2024-04-25T09:57:20.839 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:57:20.839 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049 --verbose --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/orig.config.yaml 2024-04-25T09:57:20.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:57:20.967 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673049' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJcr468dCbAyRDljN9HUUb2/PzXVQ0QK1aErESi+0DrYMfMi4fjQk7VR3P2ayslklWuqtkwYsQUGme2SgFz3kx0= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGLHf4dNqReLIpyXlLKILzvh/VptNbqJtbvpPGa6bvreCFr44OYz9LhiT49NJzSyXIJPiuJMBCnnB1EIzrKF1sk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:57:20.968 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:57:20.968 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:57:20.968 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:57:20.970 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:57:20.995 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:57:20.995 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:57:20.995 INFO:teuthology.packaging:ref: None 2024-04-25T09:57:20.995 INFO:teuthology.packaging:tag: None 2024-04-25T09:57:20.995 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:57:20.995 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:57:20.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:57:21.137 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:57:21.138 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:57:21.146 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:57:21.146 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:57:21.153 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:57:21.168 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:57:21.173 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:57:21.199 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:51:36.474344', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIuBVkbj4gQlWylxaOPBGehPIhXGrXkvbxh9Lx+Vtrs7'} 2024-04-25T09:57:21.219 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:51:36.472561', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFEZUvEZQH14EqBuBgWPFcxVXJRMVpWg5xOTlMnODapE'} 2024-04-25T09:57:21.219 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:57:21.225 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:57:21.225 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:57:21.225 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:57:21.289 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f931ca7d0d0>, signals=[15]) 2024-04-25T09:57:21.290 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:57:21.296 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:57:21.296 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T09:57:21.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:57:21.425 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T09:57:21.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:57:21.504 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:57:21.511 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-04-25T09:57:21.516 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-04-25T09:57:21.516 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-04-25T09:57:21.563 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:57:21.563 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Ubuntu" 2024-04-25T09:57:21.563 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="22.04" 2024-04-25T09:57:21.563 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_CODENAME=jammy 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:ID=ubuntu 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE=debian 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:57:21.564 INFO:teuthology.orchestra.run.smithi087.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:57:21.565 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-04-25T09:57:21.591 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-25T09:57:21.596 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-25T09:57:21.596 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="22.04" 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=jammy 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2024-04-25T09:57:21.643 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:57:21.644 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:57:21.644 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:57:21.644 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:57:21.644 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:57:21.644 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-25T09:57:21.674 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:57:21.684 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:57:21.709 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:57:21.709 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:57:21.712 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:57:21.716 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:57:21.722 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:57:21.722 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:57:21.760 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:57:21.768 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:57:21.813 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:57:21.828 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:57:21.828 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T09:57:21.828 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T09:57:21.829 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:57:21.829 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:57:21.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:57:21.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:57:21.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T09:57:21.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T09:57:22.004 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:57:22.004 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:57:22.005 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-04-25T09:57:22.011 INFO:teuthology.orchestra.run.smithi087.stdout:5.15.0-56-generic 2024-04-25T09:57:22.011 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:57:22.011 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:57:22.012 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-04-25T09:57:22.012 DEBUG:teuthology.orchestra.run.smithi087:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T09:57:22.022 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:57:22.022 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:57:22.022 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-25T09:57:22.027 INFO:teuthology.orchestra.run.smithi169.stdout:5.15.0-56-generic 2024-04-25T09:57:22.027 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:57:22.027 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:57:22.028 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-04-25T09:57:22.028 DEBUG:teuthology.orchestra.run.smithi169:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T09:57:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T09:57:22.094 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T09:57:22.202 INFO:teuthology.orchestra.run.smithi087.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T09:57:22.202 INFO:teuthology.orchestra.run.smithi087.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T09:57:22.202 INFO:teuthology.orchestra.run.smithi087.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T09:57:22.202 INFO:teuthology.orchestra.run.smithi087.stderr:hint: 2024-04-25T09:57:22.202 INFO:teuthology.orchestra.run.smithi087.stderr:hint: git config --global init.defaultBranch 2024-04-25T09:57:22.202 INFO:teuthology.orchestra.run.smithi087.stderr:hint: 2024-04-25T09:57:22.203 INFO:teuthology.orchestra.run.smithi087.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T09:57:22.203 INFO:teuthology.orchestra.run.smithi087.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T09:57:22.203 INFO:teuthology.orchestra.run.smithi087.stderr:hint: 2024-04-25T09:57:22.203 INFO:teuthology.orchestra.run.smithi087.stderr:hint: git branch -m 2024-04-25T09:57:22.203 INFO:teuthology.orchestra.run.smithi087.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T09:57:22.205 DEBUG:teuthology.orchestra.run.smithi087:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: git config --global init.defaultBranch 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: 2024-04-25T09:57:22.242 INFO:teuthology.orchestra.run.smithi169.stderr:hint: git branch -m 2024-04-25T09:57:22.243 INFO:teuthology.orchestra.run.smithi169.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T09:57:22.245 DEBUG:teuthology.orchestra.run.smithi169:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T09:57:22.270 DEBUG:teuthology.orchestra.run.smithi087:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T09:57:22.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T09:57:22.323 DEBUG:teuthology.orchestra.run.smithi087:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T09:57:22.362 DEBUG:teuthology.orchestra.run.smithi169:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T09:57:55.580 INFO:teuthology.orchestra.run.smithi087.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T09:57:55.581 INFO:teuthology.orchestra.run.smithi087.stderr: * [new branch] main -> origin/main 2024-04-25T09:57:55.614 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T09:57:55.615 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T09:57:55.615 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T09:57:55.615 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T09:57:55.615 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T09:57:55.615 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T09:57:55.615 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T09:57:55.615 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T09:57:55.616 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T09:57:55.618 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T09:57:55.619 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T09:57:55.620 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T09:57:55.623 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T09:57:55.623 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T09:57:55.623 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T09:57:55.623 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T09:57:55.624 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T09:57:55.625 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T09:57:55.625 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T09:57:55.625 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T09:57:55.625 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T09:57:55.625 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T09:57:55.625 INFO:teuthology.orchestra.run.smithi087.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T09:57:57.228 INFO:teuthology.orchestra.run.smithi169.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T09:57:57.228 INFO:teuthology.orchestra.run.smithi169.stderr: * [new branch] main -> origin/main 2024-04-25T09:57:57.262 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T09:57:57.262 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T09:57:57.262 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T09:57:57.262 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T09:57:57.262 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T09:57:57.263 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T09:57:57.263 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T09:57:57.263 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T09:57:57.263 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T09:57:57.263 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T09:57:57.263 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T09:57:57.264 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T09:57:57.264 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T09:57:57.264 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T09:57:57.266 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T09:57:57.267 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T09:57:57.267 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T09:57:57.267 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T09:57:57.267 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T09:57:57.269 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T09:57:57.270 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T09:57:57.271 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T09:57:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:Updating files: 44% (1515/3381) Updating files: 45% (1522/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2005/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 83% (2812/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T09:57:59.889 INFO:teuthology.orchestra.run.smithi087.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T09:57:59.910 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:57:59.911 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:58:00.065 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T09:58:00.065 DEBUG:teuthology.orchestra.run.smithi087:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T09:58:00.068 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi087.front.sepia.ceph.com to complete... 2024-04-25T09:58:01.556 INFO:teuthology.orchestra.run.smithi169.stderr:Updating files: 44% (1517/3381) Updating files: 45% (1522/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2005/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 83% (2808/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T09:58:01.561 INFO:teuthology.orchestra.run.smithi169.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T09:58:01.582 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:58:01.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:58:01.738 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T09:58:01.739 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T09:58:01.741 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2024-04-25T09:58:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25 09:58:01 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T09:58:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:FINISHED --2024-04-25 09:58:01-- 2024-04-25T09:58:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:Total wall clock time: 1.8s 2024-04-25T09:58:01.885 INFO:teuthology.orchestra.run.smithi087.stderr:Downloaded: 1 files, 43M in 1.6s (27.3 MB/s) 2024-04-25T09:58:01.889 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:58:01.889 DEBUG:teuthology.orchestra.run.smithi087:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T09:58:02.147 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T09:58:03.469 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25 09:58:03 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T09:58:03.469 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2024-04-25 09:58:03-- 2024-04-25T09:58:03.469 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 1.7s 2024-04-25T09:58:03.469 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 43M in 1.5s (27.6 MB/s) 2024-04-25T09:58:03.473 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:58:03.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T09:58:03.718 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T09:58:04.185 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:58:04.189 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T09:58:04.310 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T09:58:05.681 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:58:05.687 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T09:58:05.809 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T09:58:08.187 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T09:58:08.298 INFO:teuthology.orchestra.run.smithi087.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:09.745 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T09:58:09.852 INFO:teuthology.orchestra.run.smithi169.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:29.484 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:58:29.499 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:58:29.523 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-04-25T09:58:29.807 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:29.831 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:30.128 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:58:30.134 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:58:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:58:30.204 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:58:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:58:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:58:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:58:30.405 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-04-25T09:58:30.599 DEBUG:teuthology.orchestra.run.smithi087:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T09:58:30.607 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T09:58:30.607 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T09:58:30.607 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:30.624 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:30.625 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T09:58:30.646 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T09:58:30.647 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T09:58:30.647 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T09:58:30.647 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T09:58:30.647 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T09:58:30.647 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T09:58:30.649 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:58:30.649 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:58:30.649 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:58:30.649 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:58:30.649 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:58:30.649 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:58:30.649 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T09:58:30.650 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T09:58:30.651 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:58:30.651 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T09:58:30.651 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T09:58:30.651 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T09:58:30.651 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:58:30.651 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:58:30.651 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:58:30.665 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:58:30.666 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:58:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:58:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T09:58:30.669 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:58:30.670 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:58:30.671 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T09:58:30.672 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:58:30.673 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T09:58:30.674 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:58:30.736 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:58:30.737 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:58:30.738 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T09:58:30.739 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:58:30.740 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:58:30.741 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T09:58:30.742 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T09:58:30.743 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T09:58:30.744 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T09:58:30.745 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T09:58:30.746 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:58:30.747 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:58:30.748 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:58:30.749 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:58:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:58:30.751 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:58:30.752 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:58:30.753 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:58:30.754 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:58:30.755 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:58:30.756 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:58:30.757 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:58:30.758 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:58:30.759 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:58:30.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:58:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T09:58:30.762 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:58:30.763 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:58:30.764 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T09:58:30.765 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:58:30.766 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T09:58:30.767 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T09:58:30.768 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T09:58:30.769 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:58:30.770 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T09:58:30.771 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T09:58:30.772 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T09:58:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:58:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:58:30.775 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:58:30.776 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T09:58:30.777 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T09:58:30.778 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T09:58:30.779 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:58:30.780 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:58:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T09:58:30.782 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:58:30.783 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:58:30.784 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:58:30.785 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T09:58:30.786 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T09:58:30.860 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:58:30.861 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:58:30.862 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:58:30.863 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T09:58:30.864 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T09:58:30.865 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:58:30.866 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:58:30.867 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T09:58:30.868 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:58:30.869 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:58:30.870 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:58:30.871 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:58:30.872 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:58:30.873 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:58:30.874 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T09:58:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T09:58:30.876 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:58:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T09:58:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T09:58:30.880 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:58:30.881 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:58:30.882 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T09:58:30.883 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:58:30.884 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:58:30.885 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:58:30.886 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:58:30.887 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:58:30.888 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T09:58:30.889 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T09:58:30.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:58:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:58:30.893 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:58:30.894 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T09:58:30.895 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T09:58:30.896 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T09:58:30.897 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:58:30.898 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T09:58:30.899 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T09:58:30.900 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T09:58:30.901 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T09:58:30.902 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:58:30.903 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:58:30.904 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:58:30.906 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:58:30.907 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:58:30.908 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:58:30.909 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:58:30.910 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:58:30.911 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T09:58:30.912 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:58:30.913 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:58:30.914 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T09:58:30.915 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T09:58:30.916 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:58:30.917 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:58:30.918 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T09:58:30.919 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T09:58:30.920 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:58:30.976 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T09:58:30.976 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T09:58:30.977 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:58:30.978 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T09:58:30.979 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:58:30.980 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T09:58:30.981 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:58:30.982 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:58:30.983 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T09:58:30.984 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:58:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T09:58:30.986 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:58:30.987 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:58:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T09:58:30.989 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T09:58:30.990 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T09:58:30.991 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:58:30.993 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T09:58:30.994 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:58:30.995 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:58:30.996 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:58:30.997 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:58:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:58:30.999 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T09:58:31.000 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:58:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:58:31.002 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T09:58:31.003 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:58:31.004 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:58:31.005 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:58:31.006 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:58:31.007 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:58:31.008 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:58:31.009 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:58:31.010 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:58:31.011 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:58:31.012 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:58:31.013 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:58:31.014 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:58:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:58:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:58:31.017 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:58:31.018 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:58:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:58:31.020 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:58:31.021 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T09:58:31.022 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T09:58:31.027 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:31.028 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:31.028 DEBUG:teuthology.orchestra.run.smithi087:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T09:58:31.034 INFO:teuthology.orchestra.run.smithi087.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T09:58:31.034 INFO:teuthology.orchestra.run.smithi087.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T09:58:31.034 INFO:teuthology.orchestra.run.smithi087.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T09:58:31.035 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T09:58:31.035 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:31.035 DEBUG:teuthology.orchestra.run.smithi087:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T09:58:31.080 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi087.front.sepia.ceph.com to complete... 2024-04-25T09:58:31.518 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:58:31.519 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:58:31.542 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-04-25T09:58:31.754 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:31.760 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:31.989 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:58:31.996 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:58:32.002 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:58:32.003 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:58:32.029 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-04-25T09:58:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:58:32.051 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:58:32.180 INFO:teuthology.orchestra.run.smithi087.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:58:32.180 INFO:teuthology.orchestra.run.smithi087.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:58:32.181 INFO:teuthology.orchestra.run.smithi087.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:58:32.191 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-04-25T09:58:32.354 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:32.371 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:32.636 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:58:32.646 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:58:32.707 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:58:32.713 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:58:32.865 INFO:teuthology.orchestra.run.smithi169.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:58:32.865 INFO:teuthology.orchestra.run.smithi169.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:58:32.866 INFO:teuthology.orchestra.run.smithi169.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:58:32.911 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-04-25T09:58:33.077 DEBUG:teuthology.orchestra.run.smithi169:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T09:58:33.084 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T09:58:33.085 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T09:58:33.085 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:33.103 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:33.103 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:33.124 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T09:58:33.124 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T09:58:33.125 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T09:58:33.128 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:58:33.128 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:58:33.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T09:58:33.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:58:33.131 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:58:33.131 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T09:58:33.153 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:58:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:58:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:58:33.156 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T09:58:33.157 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T09:58:33.158 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:58:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:58:33.160 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:58:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:58:33.162 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:58:33.163 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T09:58:33.164 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:58:33.165 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:58:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:58:33.167 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T09:58:33.168 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:58:33.229 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T09:58:33.229 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:58:33.229 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:58:33.229 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:58:33.229 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T09:58:33.229 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:58:33.230 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T09:58:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T09:58:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T09:58:33.233 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T09:58:33.234 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:58:33.235 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:58:33.236 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:58:33.237 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:58:33.238 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:58:33.239 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T09:58:33.240 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:58:33.241 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:58:33.242 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:58:33.243 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:58:33.244 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:58:33.245 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:58:33.246 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T09:58:33.247 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:58:33.248 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:58:33.249 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:58:33.250 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T09:58:33.251 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T09:58:33.252 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:58:33.253 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T09:58:33.254 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:58:33.255 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T09:58:33.256 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T09:58:33.257 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:58:33.258 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T09:58:33.259 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:58:33.260 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:58:33.261 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T09:58:33.262 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T09:58:33.263 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:58:33.264 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T09:58:33.265 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T09:58:33.266 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T09:58:33.267 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T09:58:33.268 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T09:58:33.269 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T09:58:33.270 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:58:33.271 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:58:33.272 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:58:33.273 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:58:33.274 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:58:33.275 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T09:58:33.276 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T09:58:33.277 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T09:58:33.278 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T09:58:33.279 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T09:58:33.280 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:58:33.356 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:58:33.357 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:58:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T09:58:33.359 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:58:33.360 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:58:33.361 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:58:33.363 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:58:33.364 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:58:33.365 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:58:33.366 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T09:58:33.367 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:58:33.369 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T09:58:33.371 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:58:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T09:58:33.375 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T09:58:33.376 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:58:33.377 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T09:58:33.378 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T09:58:33.379 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:58:33.380 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T09:58:33.381 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:58:33.382 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T09:58:33.383 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:58:33.384 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T09:58:33.385 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:58:33.386 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T09:58:33.387 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:58:33.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:58:33.389 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T09:58:33.390 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:58:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:58:33.392 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T09:58:33.395 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T09:58:33.397 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T09:58:33.398 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T09:58:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T09:58:33.400 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T09:58:33.401 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:58:33.402 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:58:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:58:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:58:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:58:33.406 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:58:33.407 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:58:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T09:58:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T09:58:33.410 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:58:33.411 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T09:58:33.412 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T09:58:33.413 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:58:33.414 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:58:33.415 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T09:58:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T09:58:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:58:33.418 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T09:58:33.419 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T09:58:33.472 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T09:58:33.473 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:58:33.475 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:58:33.476 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:58:33.477 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:58:33.478 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T09:58:33.479 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T09:58:33.480 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:58:33.481 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:58:33.482 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T09:58:33.483 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:58:33.484 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T09:58:33.485 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:58:33.486 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T09:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:58:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:58:33.489 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:58:33.490 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:58:33.491 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:58:33.493 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:58:33.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:58:33.495 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:58:33.496 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:58:33.497 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:58:33.498 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T09:58:33.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:58:33.500 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:58:33.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:58:33.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:58:33.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:58:33.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T09:58:33.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:58:33.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:58:33.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:58:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:58:33.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T09:58:33.511 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:58:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:58:33.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:58:33.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T09:58:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:58:33.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:58:33.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T09:58:33.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T09:58:33.523 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:33.523 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:33.523 DEBUG:teuthology.orchestra.run.smithi169:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T09:58:33.529 INFO:teuthology.orchestra.run.smithi169.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T09:58:33.530 INFO:teuthology.orchestra.run.smithi169.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T09:58:33.530 INFO:teuthology.orchestra.run.smithi169.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T09:58:33.530 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T09:58:33.530 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:33.531 DEBUG:teuthology.orchestra.run.smithi169:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T09:58:33.575 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi169.front.sepia.ceph.com to complete... 2024-04-25T09:58:33.934 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:58:33.935 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:58:33.946 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-04-25T09:58:34.130 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:34.135 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:34.373 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:58:34.378 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:58:34.420 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:58:34.425 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:58:34.536 INFO:teuthology.orchestra.run.smithi169.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:58:34.537 INFO:teuthology.orchestra.run.smithi169.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:58:34.537 INFO:teuthology.orchestra.run.smithi169.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:58:34.554 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-04-25T09:59:03.354 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T09:59:03.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T09:59:03.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:59:05.662 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T09:59:05.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T09:59:05.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:59:21.891 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T09:59:24.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T09:59:30.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T09:59:30.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:59:33.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T09:59:33.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:59:33.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T09:59:36.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T09:59:45.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T09:59:45.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:59:48.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T09:59:48.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:00:04.387 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T10:00:06.690 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T10:00:19.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T10:00:19.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:00:19.644 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T10:00:20.795 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T10:00:20.795 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-04-25T10:00:20.803 INFO:teuthology.orchestra.run.smithi087.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T10:00:20.803 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T10:00:20.804 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T10:00:20.804 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T10:00:21.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T10:00:21.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:00:21.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T10:00:21.805 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:39.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T10:00:39.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:00:40.045 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T10:00:40.688 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T10:00:40.689 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T10:00:40.689 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-25T10:00:40.739 INFO:teuthology.orchestra.run.smithi169.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T10:00:40.739 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T10:00:40.739 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T10:00:40.739 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T10:00:41.740 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:41.741 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:00:41.754 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:00:41.754 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:00:41.757 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:00:41.766 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:00:41.773 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:00:41.781 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:00:41.782 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:00:41.813 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:00:41.844 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:00:41.852 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:00:41.852 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:00:41.880 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:00:41.944 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:00:41.965 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:00:41.980 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:00:41.999 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:00:42.001 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:00:42.010 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:00:42.010 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:00:42.017 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:00:42.038 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:00:42.048 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:00:42.048 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:00:42.080 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:00:42.088 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:00:42.142 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:00:42.194 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:00:42.194 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:00:42.264 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:00:42.273 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:00:42.324 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:00:42.324 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:00:42.401 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-04-25T10:00:42.404 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-25T10:00:42.530 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:00:42.537 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:00:42.537 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:00:42.560 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:00:42.570 DEBUG:teuthology.task.selinux:Excluding smithi087: OS 'ubuntu' does not support SELinux 2024-04-25T10:00:42.570 DEBUG:teuthology.task.selinux:Excluding smithi169: OS 'ubuntu' does not support SELinux 2024-04-25T10:00:42.570 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:00:42.570 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T10:00:42.570 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:00:42.571 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:00:42.580 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T10:00:42.586 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T10:00:42.890 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:00:42.907 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:00:42.908 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi087.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:08:25.578 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi087.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-25T10:08:25.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T10:08:25.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:08:25.666 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-04-25T10:08:25.773 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T10:08:25.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T10:08:25.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:08:25.854 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T10:08:25.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T10:08:25.982 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:08:25.995 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:08:25.996 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:08:25.996 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:08:25.999 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:08:25.999 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:08:26.034 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T10:08:26.035 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Command line: ntpd -gq 2024-04-25T10:08:26.035 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: ---------------------------------------------------- 2024-04-25T10:08:26.035 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T10:08:26.035 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T10:08:26.035 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: corporation. Support and training for ntp-4 are 2024-04-25T10:08:26.035 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: available at https://www.nwtime.org/support 2024-04-25T10:08:26.035 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: ---------------------------------------------------- 2024-04-25T10:08:26.036 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: proto: precision = 0.042 usec (-24) 2024-04-25T10:08:26.036 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: basedate set to 2022-02-04 2024-04-25T10:08:26.036 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: gps base set to 2022-02-06 (week 2196) 2024-04-25T10:08:26.036 INFO:teuthology.orchestra.run.smithi087.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:08:26.036 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:08:26.037 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:08:26.037 INFO:teuthology.orchestra.run.smithi087.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:08:26.038 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T10:08:26.038 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T10:08:26.038 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T10:08:26.038 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Listen normally on 3 enp3s0f1 172.21.15.87:123 2024-04-25T10:08:26.038 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Listen normally on 4 lo [::1]:123 2024-04-25T10:08:26.038 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5f5%5]:123 2024-04-25T10:08:26.038 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:26 ntpd[17219]: Listening on routing socket on fd #22 for interface updates 2024-04-25T10:08:26.060 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T10:08:26.061 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Command line: ntpd -gq 2024-04-25T10:08:26.061 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: ---------------------------------------------------- 2024-04-25T10:08:26.061 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T10:08:26.061 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T10:08:26.061 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: corporation. Support and training for ntp-4 are 2024-04-25T10:08:26.061 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: available at https://www.nwtime.org/support 2024-04-25T10:08:26.061 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: ---------------------------------------------------- 2024-04-25T10:08:26.062 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: proto: precision = 0.041 usec (-24) 2024-04-25T10:08:26.062 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: basedate set to 2022-02-04 2024-04-25T10:08:26.062 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: gps base set to 2022-02-06 (week 2196) 2024-04-25T10:08:26.063 INFO:teuthology.orchestra.run.smithi169.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:08:26.063 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:08:26.063 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:08:26.063 INFO:teuthology.orchestra.run.smithi169.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:08:26.064 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T10:08:26.064 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T10:08:26.064 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T10:08:26.065 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Listen normally on 3 enp3s0f1 172.21.15.169:123 2024-04-25T10:08:26.065 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Listen normally on 4 lo [::1]:123 2024-04-25T10:08:26.065 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f4d%5]:123 2024-04-25T10:08:26.065 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:26 ntpd[17196]: Listening on routing socket on fd #22 for interface updates 2024-04-25T10:08:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:27 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:27 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:27 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:27 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:27 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:27 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:27.061 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:27.061 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:27.061 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:27.061 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:27.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:27.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:27.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:27.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:27 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:28.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:28 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:28.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:28 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:28 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:28 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:28 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:28 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:29 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:29 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:29 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:29 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:29 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:29 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:29.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:29 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:30.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:30.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:30.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:30.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:30.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:30.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:30.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:30.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:30 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:31.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:31 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:31.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:31.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:31.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:31.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:31 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:32.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:32 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:32.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:32 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:32.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:32 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:32.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:32 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:32.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:32 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:32.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:32 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:33.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.035 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:33.036 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:33.036 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 10:08:33 ntpd[17219]: ntpd: time slew -0.001192 s 2024-04-25T10:08:33.036 INFO:teuthology.orchestra.run.smithi087.stdout:ntpd: time slew -0.001192s 2024-04-25T10:08:33.037 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T10:08:33.037 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 10:08:33 ntpd[17219]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stdout:25 Apr 10:08:33 ntpd[17196]: ntpd: time slew -0.002898 s 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi169.stdout:ntpd: time slew -0.002898s 2024-04-25T10:08:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T10:08:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:25 Apr 10:08:33 ntpd[17196]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T10:08:33.141 INFO:teuthology.orchestra.run.smithi087.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:08:33.141 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================== 2024-04-25T10:08:33.142 INFO:teuthology.orchestra.run.smithi087.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.142 INFO:teuthology.orchestra.run.smithi087.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.142 INFO:teuthology.orchestra.run.smithi087.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.142 INFO:teuthology.orchestra.run.smithi087.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.159 INFO:teuthology.orchestra.run.smithi169.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:08:33.159 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================== 2024-04-25T10:08:33.159 INFO:teuthology.orchestra.run.smithi169.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.159 INFO:teuthology.orchestra.run.smithi169.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.159 INFO:teuthology.orchestra.run.smithi169.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.159 INFO:teuthology.orchestra.run.smithi169.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:08:33.160 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:08:33.169 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:08:33.169 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T10:08:33.169 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T10:08:33.169 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:08:33.175 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:08:33.175 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:08:33.175 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-key list | grep Ceph 2024-04-25T10:08:33.176 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-key list | grep Ceph 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi087.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T10:08:33.306 INFO:teuthology.orchestra.run.smithi169.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T10:08:33.334 INFO:teuthology.orchestra.run.smithi087.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T10:08:33.334 INFO:teuthology.orchestra.run.smithi087.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T10:08:33.335 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T10:08:33.335 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T10:08:33.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:08:33.404 INFO:teuthology.orchestra.run.smithi169.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T10:08:33.405 INFO:teuthology.orchestra.run.smithi169.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T10:08:33.406 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T10:08:33.406 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T10:08:33.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:08:33.500 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T10:08:33.500 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:08:33.578 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T10:08:33.578 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:08:33.616 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:08:33.616 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T10:08:33.635 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get update 2024-04-25T10:08:33.703 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:08:33.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T10:08:33.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-get update 2024-04-25T10:08:33.823 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T10:08:33.833 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi087.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T10:08:33.913 INFO:teuthology.orchestra.run.smithi087.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T10:08:33.926 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T10:08:33.934 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T10:08:33.947 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T10:08:33.966 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T10:08:33.966 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:9 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T10:08:33.979 INFO:teuthology.orchestra.run.smithi169.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T10:08:34.032 INFO:teuthology.orchestra.run.smithi169.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T10:08:34.032 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T10:08:34.058 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T10:08:34.120 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T10:08:34.409 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 28.9 kB in 1s (52.3 kB/s) 2024-04-25T10:08:34.501 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 28.9 kB in 1s (52.2 kB/s) 2024-04-25T10:08:35.419 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T10:08:35.433 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:08:35.481 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2024-04-25T10:08:35.496 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:08:35.508 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T10:08:35.571 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2024-04-25T10:08:35.684 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-04-25T10:08:35.685 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-04-25T10:08:35.752 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2024-04-25T10:08:35.753 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2024-04-25T10:08:35.856 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:08:35.856 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:08:35.856 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:08:35.856 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:08:35.857 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:08:35.857 INFO:teuthology.orchestra.run.smithi087.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:08:35.857 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:08:35.857 INFO:teuthology.orchestra.run.smithi087.stdout:The following additional packages will be installed: 2024-04-25T10:08:35.857 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T10:08:35.859 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T10:08:35.860 INFO:teuthology.orchestra.run.smithi087.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout:Suggested packages: 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout:Recommended packages: 2024-04-25T10:08:35.861 INFO:teuthology.orchestra.run.smithi087.stdout: btrfs-tools 2024-04-25T10:08:35.936 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:08:35.936 INFO:teuthology.orchestra.run.smithi169.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:08:35.936 INFO:teuthology.orchestra.run.smithi169.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:08:35.936 INFO:teuthology.orchestra.run.smithi169.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:08:35.937 INFO:teuthology.orchestra.run.smithi169.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:08:35.937 INFO:teuthology.orchestra.run.smithi169.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:08:35.937 INFO:teuthology.orchestra.run.smithi169.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:08:35.939 INFO:teuthology.orchestra.run.smithi169.stdout:The following additional packages will be installed: 2024-04-25T10:08:35.939 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T10:08:35.939 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T10:08:35.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout:Suggested packages: 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T10:08:35.941 INFO:teuthology.orchestra.run.smithi169.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T10:08:35.942 INFO:teuthology.orchestra.run.smithi169.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T10:08:35.942 INFO:teuthology.orchestra.run.smithi169.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T10:08:35.942 INFO:teuthology.orchestra.run.smithi169.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T10:08:35.942 INFO:teuthology.orchestra.run.smithi169.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T10:08:35.942 INFO:teuthology.orchestra.run.smithi169.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T10:08:35.942 INFO:teuthology.orchestra.run.smithi169.stdout:Recommended packages: 2024-04-25T10:08:35.942 INFO:teuthology.orchestra.run.smithi169.stdout: btrfs-tools 2024-04-25T10:08:36.009 INFO:teuthology.orchestra.run.smithi087.stdout:The following NEW packages will be installed: 2024-04-25T10:08:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T10:08:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T10:08:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T10:08:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T10:08:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:08:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T10:08:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T10:08:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T10:08:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T10:08:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T10:08:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T10:08:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T10:08:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T10:08:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: socat unzip xmlstarlet zip 2024-04-25T10:08:36.083 INFO:teuthology.orchestra.run.smithi087.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T10:08:36.083 INFO:teuthology.orchestra.run.smithi087.stdout:Need to get 176 MB of archives. 2024-04-25T10:08:36.083 INFO:teuthology.orchestra.run.smithi087.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T10:08:36.084 INFO:teuthology.orchestra.run.smithi087.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T10:08:36.092 INFO:teuthology.orchestra.run.smithi169.stdout:The following NEW packages will be installed: 2024-04-25T10:08:36.092 INFO:teuthology.orchestra.run.smithi169.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T10:08:36.092 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T10:08:36.093 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T10:08:36.095 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T10:08:36.095 INFO:teuthology.orchestra.run.smithi169.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:08:36.095 INFO:teuthology.orchestra.run.smithi169.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T10:08:36.095 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T10:08:36.095 INFO:teuthology.orchestra.run.smithi169.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T10:08:36.095 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T10:08:36.096 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T10:08:36.097 INFO:teuthology.orchestra.run.smithi169.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T10:08:36.097 INFO:teuthology.orchestra.run.smithi169.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T10:08:36.097 INFO:teuthology.orchestra.run.smithi169.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T10:08:36.097 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T10:08:36.097 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T10:08:36.097 INFO:teuthology.orchestra.run.smithi169.stdout: socat unzip xmlstarlet zip 2024-04-25T10:08:36.103 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T10:08:36.164 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T10:08:36.169 INFO:teuthology.orchestra.run.smithi169.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T10:08:36.169 INFO:teuthology.orchestra.run.smithi169.stdout:Need to get 176 MB of archives. 2024-04-25T10:08:36.169 INFO:teuthology.orchestra.run.smithi169.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T10:08:36.169 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T10:08:36.183 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T10:08:36.189 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T10:08:36.190 INFO:teuthology.orchestra.run.smithi169.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T10:08:36.192 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T10:08:36.201 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T10:08:36.204 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T10:08:36.243 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T10:08:36.249 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T10:08:36.251 INFO:teuthology.orchestra.run.smithi087.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T10:08:36.255 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T10:08:36.258 INFO:teuthology.orchestra.run.smithi087.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T10:08:36.261 INFO:teuthology.orchestra.run.smithi087.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T10:08:36.261 INFO:teuthology.orchestra.run.smithi087.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T10:08:36.268 INFO:teuthology.orchestra.run.smithi087.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T10:08:36.269 INFO:teuthology.orchestra.run.smithi087.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T10:08:36.271 INFO:teuthology.orchestra.run.smithi087.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T10:08:36.272 INFO:teuthology.orchestra.run.smithi087.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T10:08:36.273 INFO:teuthology.orchestra.run.smithi087.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T10:08:36.280 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T10:08:36.284 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T10:08:36.294 INFO:teuthology.orchestra.run.smithi087.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T10:08:36.296 INFO:teuthology.orchestra.run.smithi087.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T10:08:36.297 INFO:teuthology.orchestra.run.smithi087.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T10:08:36.298 INFO:teuthology.orchestra.run.smithi087.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T10:08:36.298 INFO:teuthology.orchestra.run.smithi087.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T10:08:36.304 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T10:08:36.313 INFO:teuthology.orchestra.run.smithi087.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T10:08:36.314 INFO:teuthology.orchestra.run.smithi087.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T10:08:36.314 INFO:teuthology.orchestra.run.smithi087.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T10:08:36.315 INFO:teuthology.orchestra.run.smithi087.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T10:08:36.315 INFO:teuthology.orchestra.run.smithi087.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T10:08:36.334 INFO:teuthology.orchestra.run.smithi087.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T10:08:36.334 INFO:teuthology.orchestra.run.smithi087.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T10:08:36.336 INFO:teuthology.orchestra.run.smithi087.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T10:08:36.337 INFO:teuthology.orchestra.run.smithi087.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T10:08:36.338 INFO:teuthology.orchestra.run.smithi087.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T10:08:36.353 INFO:teuthology.orchestra.run.smithi087.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T10:08:36.354 INFO:teuthology.orchestra.run.smithi087.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T10:08:36.355 INFO:teuthology.orchestra.run.smithi087.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T10:08:36.355 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T10:08:36.355 INFO:teuthology.orchestra.run.smithi087.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T10:08:36.356 INFO:teuthology.orchestra.run.smithi087.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T10:08:36.358 INFO:teuthology.orchestra.run.smithi169.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T10:08:36.366 INFO:teuthology.orchestra.run.smithi169.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T10:08:36.369 INFO:teuthology.orchestra.run.smithi169.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T10:08:36.370 INFO:teuthology.orchestra.run.smithi169.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T10:08:36.376 INFO:teuthology.orchestra.run.smithi087.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T10:08:36.376 INFO:teuthology.orchestra.run.smithi169.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T10:08:36.376 INFO:teuthology.orchestra.run.smithi087.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T10:08:36.378 INFO:teuthology.orchestra.run.smithi169.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T10:08:36.380 INFO:teuthology.orchestra.run.smithi169.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T10:08:36.382 INFO:teuthology.orchestra.run.smithi087.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T10:08:36.382 INFO:teuthology.orchestra.run.smithi087.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T10:08:36.383 INFO:teuthology.orchestra.run.smithi169.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T10:08:36.383 INFO:teuthology.orchestra.run.smithi169.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T10:08:36.383 INFO:teuthology.orchestra.run.smithi087.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T10:08:36.396 INFO:teuthology.orchestra.run.smithi087.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T10:08:36.397 INFO:teuthology.orchestra.run.smithi087.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T10:08:36.399 INFO:teuthology.orchestra.run.smithi087.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T10:08:36.400 INFO:teuthology.orchestra.run.smithi087.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T10:08:36.401 INFO:teuthology.orchestra.run.smithi087.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T10:08:36.403 INFO:teuthology.orchestra.run.smithi169.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi169.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi169.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi169.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T10:08:36.408 INFO:teuthology.orchestra.run.smithi169.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T10:08:36.409 INFO:teuthology.orchestra.run.smithi169.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T10:08:36.410 INFO:teuthology.orchestra.run.smithi169.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T10:08:36.416 INFO:teuthology.orchestra.run.smithi087.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T10:08:36.417 INFO:teuthology.orchestra.run.smithi087.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T10:08:36.418 INFO:teuthology.orchestra.run.smithi087.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T10:08:36.419 INFO:teuthology.orchestra.run.smithi087.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T10:08:36.423 INFO:teuthology.orchestra.run.smithi087.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T10:08:36.436 INFO:teuthology.orchestra.run.smithi169.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T10:08:36.436 INFO:teuthology.orchestra.run.smithi169.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T10:08:36.436 INFO:teuthology.orchestra.run.smithi169.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T10:08:36.465 INFO:teuthology.orchestra.run.smithi169.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T10:08:36.465 INFO:teuthology.orchestra.run.smithi169.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T10:08:36.467 INFO:teuthology.orchestra.run.smithi169.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T10:08:36.467 INFO:teuthology.orchestra.run.smithi169.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T10:08:36.468 INFO:teuthology.orchestra.run.smithi169.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T10:08:36.468 INFO:teuthology.orchestra.run.smithi169.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T10:08:36.469 INFO:teuthology.orchestra.run.smithi169.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi087.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T10:08:36.471 INFO:teuthology.orchestra.run.smithi087.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T10:08:36.472 INFO:teuthology.orchestra.run.smithi087.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T10:08:36.473 INFO:teuthology.orchestra.run.smithi087.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi087.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T10:08:36.493 INFO:teuthology.orchestra.run.smithi087.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T10:08:36.494 INFO:teuthology.orchestra.run.smithi169.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T10:08:36.494 INFO:teuthology.orchestra.run.smithi169.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T10:08:36.495 INFO:teuthology.orchestra.run.smithi169.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T10:08:36.496 INFO:teuthology.orchestra.run.smithi087.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T10:08:36.496 INFO:teuthology.orchestra.run.smithi087.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi087.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi087.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi087.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T10:08:36.515 INFO:teuthology.orchestra.run.smithi169.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T10:08:36.515 INFO:teuthology.orchestra.run.smithi169.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T10:08:36.515 INFO:teuthology.orchestra.run.smithi087.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi087.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi087.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi169.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T10:08:36.520 INFO:teuthology.orchestra.run.smithi169.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T10:08:36.520 INFO:teuthology.orchestra.run.smithi169.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T10:08:36.521 INFO:teuthology.orchestra.run.smithi169.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T10:08:36.522 INFO:teuthology.orchestra.run.smithi169.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T10:08:36.529 INFO:teuthology.orchestra.run.smithi087.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T10:08:36.530 INFO:teuthology.orchestra.run.smithi087.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T10:08:36.534 INFO:teuthology.orchestra.run.smithi087.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T10:08:36.538 INFO:teuthology.orchestra.run.smithi169.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T10:08:36.538 INFO:teuthology.orchestra.run.smithi169.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T10:08:36.540 INFO:teuthology.orchestra.run.smithi169.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T10:08:36.551 INFO:teuthology.orchestra.run.smithi087.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T10:08:36.553 INFO:teuthology.orchestra.run.smithi087.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T10:08:36.553 INFO:teuthology.orchestra.run.smithi087.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T10:08:36.558 INFO:teuthology.orchestra.run.smithi087.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T10:08:36.560 INFO:teuthology.orchestra.run.smithi087.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T10:08:36.560 INFO:teuthology.orchestra.run.smithi087.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi087.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T10:08:36.567 INFO:teuthology.orchestra.run.smithi169.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T10:08:36.569 INFO:teuthology.orchestra.run.smithi169.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T10:08:36.570 INFO:teuthology.orchestra.run.smithi169.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T10:08:36.572 INFO:teuthology.orchestra.run.smithi169.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T10:08:36.573 INFO:teuthology.orchestra.run.smithi169.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T10:08:36.597 INFO:teuthology.orchestra.run.smithi087.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T10:08:36.605 INFO:teuthology.orchestra.run.smithi087.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T10:08:36.605 INFO:teuthology.orchestra.run.smithi087.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T10:08:36.629 INFO:teuthology.orchestra.run.smithi169.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T10:08:36.630 INFO:teuthology.orchestra.run.smithi169.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T10:08:36.631 INFO:teuthology.orchestra.run.smithi169.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T10:08:36.632 INFO:teuthology.orchestra.run.smithi169.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T10:08:36.633 INFO:teuthology.orchestra.run.smithi169.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T10:08:36.637 INFO:teuthology.orchestra.run.smithi087.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T10:08:36.637 INFO:teuthology.orchestra.run.smithi087.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T10:08:36.638 INFO:teuthology.orchestra.run.smithi087.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T10:08:36.639 INFO:teuthology.orchestra.run.smithi087.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T10:08:36.641 INFO:teuthology.orchestra.run.smithi087.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T10:08:36.642 INFO:teuthology.orchestra.run.smithi087.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T10:08:36.650 INFO:teuthology.orchestra.run.smithi087.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T10:08:36.659 INFO:teuthology.orchestra.run.smithi169.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T10:08:36.661 INFO:teuthology.orchestra.run.smithi169.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T10:08:36.662 INFO:teuthology.orchestra.run.smithi169.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T10:08:36.677 INFO:teuthology.orchestra.run.smithi087.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T10:08:36.677 INFO:teuthology.orchestra.run.smithi087.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T10:08:36.679 INFO:teuthology.orchestra.run.smithi169.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T10:08:36.679 INFO:teuthology.orchestra.run.smithi087.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T10:08:36.680 INFO:teuthology.orchestra.run.smithi169.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T10:08:36.680 INFO:teuthology.orchestra.run.smithi169.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T10:08:36.681 INFO:teuthology.orchestra.run.smithi169.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T10:08:36.681 INFO:teuthology.orchestra.run.smithi169.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T10:08:36.682 INFO:teuthology.orchestra.run.smithi169.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi087.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T10:08:36.707 INFO:teuthology.orchestra.run.smithi169.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T10:08:36.708 INFO:teuthology.orchestra.run.smithi169.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T10:08:36.715 INFO:teuthology.orchestra.run.smithi169.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T10:08:36.718 INFO:teuthology.orchestra.run.smithi169.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T10:08:36.718 INFO:teuthology.orchestra.run.smithi169.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T10:08:36.728 INFO:teuthology.orchestra.run.smithi169.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T10:08:36.729 INFO:teuthology.orchestra.run.smithi169.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T10:08:36.732 INFO:teuthology.orchestra.run.smithi169.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi169.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi169.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T10:08:36.736 INFO:teuthology.orchestra.run.smithi169.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T10:08:36.748 INFO:teuthology.orchestra.run.smithi169.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T10:08:36.753 INFO:teuthology.orchestra.run.smithi169.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T10:08:36.754 INFO:teuthology.orchestra.run.smithi169.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T10:08:36.757 INFO:teuthology.orchestra.run.smithi169.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T10:08:36.769 INFO:teuthology.orchestra.run.smithi169.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T10:08:36.769 INFO:teuthology.orchestra.run.smithi169.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T10:08:36.770 INFO:teuthology.orchestra.run.smithi169.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T10:08:36.772 INFO:teuthology.orchestra.run.smithi169.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T10:08:36.773 INFO:teuthology.orchestra.run.smithi169.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T10:08:36.780 INFO:teuthology.orchestra.run.smithi169.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T10:08:36.789 INFO:teuthology.orchestra.run.smithi169.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T10:08:36.790 INFO:teuthology.orchestra.run.smithi169.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T10:08:36.792 INFO:teuthology.orchestra.run.smithi169.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T10:08:36.792 INFO:teuthology.orchestra.run.smithi169.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T10:08:36.863 INFO:teuthology.orchestra.run.smithi087.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T10:08:36.906 INFO:teuthology.orchestra.run.smithi169.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T10:08:36.950 INFO:teuthology.orchestra.run.smithi169.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T10:08:36.967 INFO:teuthology.orchestra.run.smithi169.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T10:08:36.970 INFO:teuthology.orchestra.run.smithi169.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T10:08:36.987 INFO:teuthology.orchestra.run.smithi169.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T10:08:36.990 INFO:teuthology.orchestra.run.smithi169.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T10:08:37.006 INFO:teuthology.orchestra.run.smithi169.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T10:08:37.169 INFO:teuthology.orchestra.run.smithi087.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T10:08:37.241 INFO:teuthology.orchestra.run.smithi087.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T10:08:37.278 INFO:teuthology.orchestra.run.smithi087.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T10:08:37.281 INFO:teuthology.orchestra.run.smithi087.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T10:08:37.300 INFO:teuthology.orchestra.run.smithi087.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi087.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T10:08:37.340 INFO:teuthology.orchestra.run.smithi087.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T10:08:37.421 INFO:teuthology.orchestra.run.smithi169.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T10:08:37.423 INFO:teuthology.orchestra.run.smithi169.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T10:08:37.446 INFO:teuthology.orchestra.run.smithi169.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T10:08:37.971 INFO:teuthology.orchestra.run.smithi087.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T10:08:37.977 INFO:teuthology.orchestra.run.smithi087.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T10:08:38.025 INFO:teuthology.orchestra.run.smithi087.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T10:08:38.776 INFO:teuthology.orchestra.run.smithi169.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T10:08:39.042 INFO:teuthology.orchestra.run.smithi169.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T10:08:39.054 INFO:teuthology.orchestra.run.smithi169.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T10:08:39.062 INFO:teuthology.orchestra.run.smithi169.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T10:08:39.142 INFO:teuthology.orchestra.run.smithi169.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T10:08:39.488 INFO:teuthology.orchestra.run.smithi169.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T10:08:40.185 INFO:teuthology.orchestra.run.smithi087.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T10:08:40.747 INFO:teuthology.orchestra.run.smithi087.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T10:08:40.761 INFO:teuthology.orchestra.run.smithi087.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T10:08:40.769 INFO:teuthology.orchestra.run.smithi087.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T10:08:40.941 INFO:teuthology.orchestra.run.smithi087.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T10:08:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T10:08:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T10:08:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T10:08:41.161 INFO:teuthology.orchestra.run.smithi169.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T10:08:41.213 INFO:teuthology.orchestra.run.smithi169.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T10:08:41.222 INFO:teuthology.orchestra.run.smithi169.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T10:08:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T10:08:41.608 INFO:teuthology.orchestra.run.smithi087.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T10:08:41.978 INFO:teuthology.orchestra.run.smithi169.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T10:08:41.979 INFO:teuthology.orchestra.run.smithi169.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T10:08:44.290 INFO:teuthology.orchestra.run.smithi087.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T10:08:44.291 INFO:teuthology.orchestra.run.smithi087.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T10:08:44.357 INFO:teuthology.orchestra.run.smithi087.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T10:08:44.522 INFO:teuthology.orchestra.run.smithi087.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T10:08:44.581 INFO:teuthology.orchestra.run.smithi087.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T10:08:44.591 INFO:teuthology.orchestra.run.smithi087.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T10:08:44.737 INFO:teuthology.orchestra.run.smithi087.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T10:08:45.359 INFO:teuthology.orchestra.run.smithi087.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T10:08:45.360 INFO:teuthology.orchestra.run.smithi087.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T10:08:45.846 INFO:teuthology.orchestra.run.smithi169.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T10:08:45.847 INFO:teuthology.orchestra.run.smithi169.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T10:08:45.851 INFO:teuthology.orchestra.run.smithi169.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T10:08:45.853 INFO:teuthology.orchestra.run.smithi169.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T10:08:45.854 INFO:teuthology.orchestra.run.smithi169.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T10:08:45.858 INFO:teuthology.orchestra.run.smithi169.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T10:08:46.831 INFO:teuthology.orchestra.run.smithi169.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T10:08:47.468 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 176 MB in 11s (16.4 MB/s) 2024-04-25T10:08:47.598 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T10:08:47.664 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T10:08:47.667 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T10:08:47.713 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T10:08:48.033 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T10:08:48.046 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T10:08:48.083 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:08:48.402 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T10:08:48.416 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T10:08:48.460 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T10:08:48.526 INFO:teuthology.orchestra.run.smithi087.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T10:08:48.526 INFO:teuthology.orchestra.run.smithi087.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T10:08:48.528 INFO:teuthology.orchestra.run.smithi087.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T10:08:48.529 INFO:teuthology.orchestra.run.smithi087.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T10:08:48.529 INFO:teuthology.orchestra.run.smithi087.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T10:08:48.530 INFO:teuthology.orchestra.run.smithi087.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T10:08:49.114 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T10:08:49.128 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T10:08:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T10:08:49.214 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:08:49.584 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T10:08:49.598 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T10:08:49.642 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:08:49.814 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 176 MB in 13s (13.4 MB/s) 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T10:08:49.995 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T10:08:50.001 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T10:08:50.005 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T10:08:50.010 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T10:08:50.050 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T10:08:50.070 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:08:50.359 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T10:08:50.419 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:08:50.540 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T10:08:50.555 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:08:50.607 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:08:50.747 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T10:08:50.760 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T10:08:51.018 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T10:08:51.033 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:08:51.118 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:08:51.167 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T10:08:51.180 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T10:08:51.267 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:08:51.480 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T10:08:51.494 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:08:51.538 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:08:51.612 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T10:08:51.625 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:08:51.866 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T10:08:51.881 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T10:08:51.916 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:08:52.006 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T10:08:52.020 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T10:08:52.065 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:08:52.235 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librados2. 2024-04-25T10:08:52.250 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:52.294 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:52.518 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T10:08:52.531 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:08:52.583 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:08:52.722 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librbd1. 2024-04-25T10:08:52.736 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:52.796 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:52.988 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T10:08:53.001 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:08:53.046 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:08:53.216 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T10:08:53.230 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:53.275 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:53.374 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T10:08:53.388 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:08:53.432 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:08:53.577 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rados. 2024-04-25T10:08:53.591 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:53.643 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:53.769 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T10:08:53.782 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T10:08:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:08:54.030 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T10:08:54.044 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:08:54.090 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T10:08:54.347 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:54.382 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:54.422 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librados2. 2024-04-25T10:08:54.437 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:54.480 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T10:08:54.716 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:08:54.759 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:54.883 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librbd1. 2024-04-25T10:08:54.897 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:54.941 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:55.121 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T10:08:55.136 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T10:08:55.180 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T10:08:55.390 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:55.435 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:55.457 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T10:08:55.472 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:55.515 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:55.746 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rados. 2024-04-25T10:08:55.760 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:55.804 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:55.860 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T10:08:55.910 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:08:56.141 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T10:08:56.155 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:08:56.188 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T10:08:56.190 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T10:08:56.279 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:08:56.418 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T10:08:56.432 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:56.476 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:56.632 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T10:08:56.647 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T10:08:56.690 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:08:56.754 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T10:08:56.768 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:08:56.804 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package lua5.1. 2024-04-25T10:08:57.125 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T10:08:57.132 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T10:08:57.150 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T10:08:57.177 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:08:57.190 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T10:08:57.460 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T10:08:57.474 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:57.518 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:57.539 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package lua-any. 2024-04-25T10:08:57.554 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T10:08:57.597 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T10:08:57.871 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T10:08:57.886 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T10:08:57.900 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package zip. 2024-04-25T10:08:57.915 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T10:08:57.929 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:08:57.999 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T10:08:58.174 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T10:08:58.188 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T10:08:58.223 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:08:58.403 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package unzip. 2024-04-25T10:08:58.418 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T10:08:58.469 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:08:58.543 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T10:08:58.558 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T10:08:58.601 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:08:58.938 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua5.1. 2024-04-25T10:08:58.953 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T10:08:58.998 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:08:59.182 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package luarocks. 2024-04-25T10:08:59.197 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T10:08:59.248 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:08:59.307 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-any. 2024-04-25T10:08:59.322 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T10:08:59.357 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T10:08:59.577 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package zip. 2024-04-25T10:08:59.592 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T10:08:59.656 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librgw2. 2024-04-25T10:08:59.659 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T10:08:59.670 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:08:59.714 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:08:59.980 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package unzip. 2024-04-25T10:08:59.995 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T10:09:00.038 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:09:00.217 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T10:09:00.231 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:00.275 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:00.442 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package luarocks. 2024-04-25T10:09:00.457 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T10:09:00.500 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:09:00.662 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T10:09:00.676 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T10:09:00.720 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:09:00.908 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librgw2. 2024-04-25T10:09:00.921 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:00.983 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:00.997 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T10:09:01.013 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:01.047 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:01.452 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T10:09:01.466 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-common. 2024-04-25T10:09:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:01.501 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:01.559 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:01.788 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T10:09:01.802 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T10:09:01.846 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:09:02.166 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T10:09:02.181 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:02.224 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:02.594 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-common. 2024-04-25T10:09:02.609 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:02.652 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:02.714 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-base. 2024-04-25T10:09:02.729 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:02.782 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:03.394 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T10:09:03.408 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T10:09:03.452 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:09:03.758 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-base. 2024-04-25T10:09:03.771 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T10:09:03.773 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:03.786 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T10:09:03.825 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:09:04.183 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T10:09:04.198 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:09:04.552 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T10:09:04.567 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T10:09:04.570 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T10:09:04.584 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T10:09:04.602 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:09:04.628 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:09:04.913 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T10:09:04.930 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T10:09:04.948 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T10:09:04.962 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T10:09:04.972 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:09:05.006 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:09:05.291 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T10:09:05.306 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T10:09:05.358 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T10:09:05.383 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T10:09:05.401 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:09:05.662 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T10:09:05.677 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T10:09:05.704 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-portend. 2024-04-25T10:09:05.712 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:09:05.720 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T10:09:05.769 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T10:09:06.015 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T10:09:06.022 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T10:09:06.030 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T10:09:06.037 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T10:09:06.082 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:09:06.088 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T10:09:06.401 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T10:09:06.417 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T10:09:06.424 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T10:09:06.440 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T10:09:06.459 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T10:09:06.483 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:09:06.737 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-portend. 2024-04-25T10:09:06.752 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T10:09:06.787 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T10:09:06.845 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T10:09:06.860 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T10:09:06.895 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T10:09:07.040 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T10:09:07.055 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T10:09:07.240 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T10:09:07.255 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T10:09:07.298 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T10:09:07.410 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T10:09:07.425 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T10:09:07.468 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:09:07.626 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T10:09:07.641 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T10:09:07.676 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:09:07.788 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T10:09:07.803 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T10:09:07.846 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T10:09:07.995 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-mako. 2024-04-25T10:09:08.011 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T10:09:08.074 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:09:08.158 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T10:09:08.173 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T10:09:08.216 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T10:09:08.440 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T10:09:08.455 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T10:09:08.498 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:09:08.519 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T10:09:08.534 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T10:09:08.577 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:09:08.826 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T10:09:08.841 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T10:09:08.884 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:09:08.888 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-mako. 2024-04-25T10:09:08.903 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T10:09:08.938 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:09:09.900 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T10:09:09.915 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T10:09:09.958 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:09:10.138 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webob. 2024-04-25T10:09:10.153 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T10:09:10.196 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:09:10.286 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T10:09:10.302 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T10:09:10.344 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:09:10.594 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T10:09:10.609 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T10:09:10.647 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-webob. 2024-04-25T10:09:10.660 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:09:10.662 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T10:09:10.705 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:09:10.994 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T10:09:11.009 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T10:09:11.019 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T10:09:11.034 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T10:09:11.052 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:09:11.071 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:09:11.372 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T10:09:11.380 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-paste. 2024-04-25T10:09:11.387 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T10:09:11.395 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T10:09:11.438 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:09:11.455 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:09:11.783 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-paste. 2024-04-25T10:09:11.791 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T10:09:11.798 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T10:09:11.806 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T10:09:11.841 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:09:11.849 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:09:12.144 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T10:09:12.158 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T10:09:12.177 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T10:09:12.192 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T10:09:12.202 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:09:12.227 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:09:12.488 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T10:09:12.502 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T10:09:12.514 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T10:09:12.529 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T10:09:12.546 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:09:12.564 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T10:09:12.866 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T10:09:12.882 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T10:09:12.883 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T10:09:12.898 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T10:09:12.924 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T10:09:12.962 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:09:13.203 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T10:09:13.218 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T10:09:13.252 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:09:13.343 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T10:09:13.355 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T10:09:13.398 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T10:09:13.616 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T10:09:13.630 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T10:09:13.665 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T10:09:13.760 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-idna. 2024-04-25T10:09:13.774 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T10:09:13.818 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T10:09:13.993 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-idna. 2024-04-25T10:09:14.009 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T10:09:14.077 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T10:09:14.121 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T10:09:14.135 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T10:09:14.171 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:09:14.380 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T10:09:14.395 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T10:09:14.438 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:09:14.491 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-requests. 2024-04-25T10:09:14.505 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T10:09:14.549 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:09:14.869 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T10:09:14.883 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T10:09:14.891 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-requests. 2024-04-25T10:09:14.906 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T10:09:14.927 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:09:14.949 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:09:15.244 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T10:09:15.255 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T10:09:15.259 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T10:09:15.269 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:15.302 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:09:15.321 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:15.597 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T10:09:15.616 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:15.655 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:15.752 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T10:09:15.766 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:15.810 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:16.010 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T10:09:16.024 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:16.076 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:16.155 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T10:09:16.169 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T10:09:16.213 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:09:16.388 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T10:09:16.403 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T10:09:16.446 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:09:16.549 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T10:09:16.563 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T10:09:16.607 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T10:09:16.749 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T10:09:16.764 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T10:09:16.807 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T10:09:16.952 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T10:09:16.966 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:17.010 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:17.110 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T10:09:17.125 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:17.159 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:17.472 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T10:09:17.486 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:17.530 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:17.538 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T10:09:17.553 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:17.596 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:18.159 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T10:09:18.207 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:18.207 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T10:09:18.207 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:18.217 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:18.986 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph. 2024-04-25T10:09:19.000 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:19.029 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph. 2024-04-25T10:09:19.036 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:19.043 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:19.355 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T10:09:19.370 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:19.413 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:19.448 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T10:09:19.463 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:19.506 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:19.775 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T10:09:19.790 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:19.860 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T10:09:19.874 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:19.928 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:20.211 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephadm. 2024-04-25T10:09:20.226 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:20.269 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:20.538 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephadm. 2024-04-25T10:09:20.552 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:20.555 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T10:09:20.570 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:20.638 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T10:09:20.949 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T10:09:20.963 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T10:09:20.966 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T10:09:20.980 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:09:21.007 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T10:09:21.024 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:09:21.335 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T10:09:21.349 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:09:21.393 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:09:22.029 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-babel. 2024-04-25T10:09:22.043 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:09:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:09:22.398 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T10:09:22.412 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T10:09:22.457 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:09:22.498 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-babel. 2024-04-25T10:09:22.512 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:09:22.557 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:09:22.776 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T10:09:22.790 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:22.834 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:22.893 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T10:09:22.907 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T10:09:22.951 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:09:23.155 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T10:09:23.213 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T10:09:23.305 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T10:09:23.314 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:23.354 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:23.532 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-routes. 2024-04-25T10:09:23.548 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T10:09:23.590 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:09:23.691 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T10:09:23.704 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T10:09:23.733 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T10:09:23.935 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T10:09:23.949 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:23.993 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:24.035 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-routes. 2024-04-25T10:09:24.046 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T10:09:24.085 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:09:24.405 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T10:09:24.421 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T10:09:24.455 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T10:09:24.456 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:09:24.469 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:24.513 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:24.950 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T10:09:24.962 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T10:09:25.000 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:09:25.150 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T10:09:25.164 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T10:09:25.208 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:09:25.537 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T10:09:25.549 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T10:09:25.570 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T10:09:25.584 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T10:09:25.612 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:09:25.723 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:09:25.983 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T10:09:25.994 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T10:09:26.025 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T10:09:26.040 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T10:09:26.083 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:09:26.110 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:09:26.446 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T10:09:26.460 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T10:09:26.506 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:09:26.734 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T10:09:26.748 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:26.792 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:27.214 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T10:09:27.225 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:27.264 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T10:09:27.844 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T10:09:27.887 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T10:09:28.174 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T10:09:28.189 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T10:09:28.223 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:09:28.284 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T10:09:28.297 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T10:09:28.334 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T10:09:28.552 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T10:09:28.566 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T10:09:28.635 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:09:28.637 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T10:09:28.650 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T10:09:28.695 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:09:28.938 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T10:09:28.954 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T10:09:29.066 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T10:09:29.081 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T10:09:29.124 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:09:29.242 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T10:09:29.257 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T10:09:29.300 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T10:09:29.482 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T10:09:29.495 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T10:09:29.543 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T10:09:29.611 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T10:09:29.627 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T10:09:29.670 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:09:29.835 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T10:09:29.848 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T10:09:29.897 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T10:09:29.998 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T10:09:30.012 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T10:09:30.056 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T10:09:30.242 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T10:09:30.256 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T10:09:30.300 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:09:30.534 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T10:09:30.549 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T10:09:30.592 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:09:30.887 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T10:09:30.900 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T10:09:30.962 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T10:09:31.255 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T10:09:31.269 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:31.306 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:31.307 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T10:09:31.320 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T10:09:31.357 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:09:31.609 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T10:09:31.622 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T10:09:31.667 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:09:32.012 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T10:09:32.026 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:09:32.054 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T10:09:32.066 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:32.070 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:09:32.103 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:32.306 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package jq. 2024-04-25T10:09:32.320 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:09:32.364 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T10:09:32.457 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T10:09:32.469 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T10:09:32.506 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:09:32.608 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package socat. 2024-04-25T10:09:32.622 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T10:09:32.667 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:09:32.868 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T10:09:32.880 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:09:32.943 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:09:32.970 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T10:09:32.984 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T10:09:33.028 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:09:33.154 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package jq. 2024-04-25T10:09:33.166 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:09:33.222 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T10:09:33.349 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-test. 2024-04-25T10:09:33.363 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:33.399 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package socat. 2024-04-25T10:09:33.488 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T10:09:33.532 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:09:33.868 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T10:09:33.881 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T10:09:33.943 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:09:34.306 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-test. 2024-04-25T10:09:34.319 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:34.364 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:35.663 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T10:09:35.673 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:35.712 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:36.091 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T10:09:36.102 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:36.140 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:36.340 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T10:09:36.353 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:36.402 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:36.460 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T10:09:36.470 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T10:09:36.510 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T10:09:36.814 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:36.831 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T10:09:36.844 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:36.854 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:36.881 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:37.073 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T10:09:37.083 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:37.114 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:37.193 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T10:09:37.208 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T10:09:37.242 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T10:09:37.342 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T10:09:37.352 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:37.392 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:37.512 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T10:09:37.525 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:37.569 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:37.736 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T10:09:37.746 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T10:09:37.786 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T10:09:37.870 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:09:37.914 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:38.157 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T10:09:38.167 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T10:09:38.200 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T10:09:38.207 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:09:38.214 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:38.258 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:38.511 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T10:09:38.521 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T10:09:38.552 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:09:38.586 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T10:09:38.600 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T10:09:38.737 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:09:38.958 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package pkg-config. 2024-04-25T10:09:38.965 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T10:09:38.967 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T10:09:38.979 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T10:09:39.008 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:09:39.049 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:09:39.277 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T10:09:39.287 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T10:09:39.327 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:09:39.444 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T10:09:39.458 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T10:09:39.502 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:09:39.764 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T10:09:39.773 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T10:09:39.813 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T10:09:39.958 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package pkg-config. 2024-04-25T10:09:39.971 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T10:09:40.016 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:09:40.116 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T10:09:40.126 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T10:09:40.166 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T10:09:40.303 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T10:09:40.316 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T10:09:40.361 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:09:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T10:09:40.487 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T10:09:40.553 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:09:40.847 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T10:09:40.848 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T10:09:40.860 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T10:09:40.861 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T10:09:40.897 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T10:09:40.905 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T10:09:41.242 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-py. 2024-04-25T10:09:41.253 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T10:09:41.292 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T10:09:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T10:09:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T10:09:41.512 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T10:09:41.620 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T10:09:41.631 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T10:09:41.670 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:09:41.753 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T10:09:41.767 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T10:09:41.803 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:09:42.148 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T10:09:42.149 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T10:09:42.158 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T10:09:42.161 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T10:09:42.198 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:09:42.239 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T10:09:42.476 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-toml. 2024-04-25T10:09:42.486 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T10:09:42.518 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T10:09:42.626 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-py. 2024-04-25T10:09:42.640 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T10:09:42.684 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T10:09:42.796 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T10:09:42.805 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T10:09:42.845 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:09:43.038 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T10:09:43.051 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T10:09:43.096 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:09:43.208 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T10:09:43.217 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T10:09:43.257 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:09:43.527 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T10:09:43.536 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T10:09:43.550 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T10:09:43.563 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T10:09:43.576 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:09:43.617 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:09:43.944 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-toml. 2024-04-25T10:09:43.958 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T10:09:44.002 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T10:09:44.305 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package radosgw. 2024-04-25T10:09:44.313 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:44.330 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T10:09:44.345 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T10:09:44.355 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:44.389 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:09:44.776 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T10:09:44.790 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T10:09:44.834 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:09:44.950 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T10:09:44.958 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:44.991 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:45.137 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T10:09:45.151 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T10:09:45.195 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:09:45.280 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T10:09:45.547 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:09:45.965 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:09:46.041 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package radosgw. 2024-04-25T10:09:46.054 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:46.090 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:46.245 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T10:09:46.653 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T10:09:46.719 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T10:09:46.732 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:09:46.768 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:47.087 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T10:09:47.266 INFO:teuthology.orchestra.run.smithi087.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T10:09:47.319 INFO:teuthology.orchestra.run.smithi087.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T10:09:47.366 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:47.374 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:09:47.500 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:09:47.755 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T10:09:47.810 INFO:teuthology.orchestra.run.smithi087.stdout:Adding system user cephadm....done 2024-04-25T10:09:47.964 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:09:48.177 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T10:09:48.240 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:09:48.500 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:09:48.617 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:09:48.838 INFO:teuthology.orchestra.run.smithi169.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T10:09:48.852 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T10:09:48.883 INFO:teuthology.orchestra.run.smithi169.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T10:09:48.927 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:49.095 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:09:49.213 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T10:09:49.523 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:09:49.864 INFO:teuthology.orchestra.run.smithi169.stdout:Adding system user cephadm....done 2024-04-25T10:09:49.891 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T10:09:50.026 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:09:50.159 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T10:09:50.319 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:09:50.419 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T10:09:50.603 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:09:50.678 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:09:50.729 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:09:50.932 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:09:51.014 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T10:09:51.208 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:09:51.315 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:09:51.433 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T10:09:51.600 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:09:51.752 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:09:51.860 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:52.111 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T10:09:52.220 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:09:52.329 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T10:09:52.430 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T10:09:52.622 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:09:52.731 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T10:09:52.748 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:09:53.016 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:09:53.082 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:09:53.359 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:09:53.406 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T10:09:53.429 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T10:09:53.502 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:09:53.602 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:09:53.611 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T10:09:53.878 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T10:09:54.146 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:54.356 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:09:54.447 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:09:54.606 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:09:54.632 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:09:54.732 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T10:09:54.992 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:09:55.084 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:09:55.092 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:09:55.336 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:09:55.716 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T10:09:55.774 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T10:09:55.777 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:09:55.872 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:09:55.994 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T10:09:56.062 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:09:56.188 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:09:56.330 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:09:56.635 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:09:56.656 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T10:09:57.037 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:09:57.058 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T10:09:57.321 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:09:57.325 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:09:57.665 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T10:09:57.710 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:09:57.790 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:09:58.003 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T10:09:58.321 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:09:58.492 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:09:58.672 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:09:58.760 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:09:58.869 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:09:58.931 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:09:59.028 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:09:59.091 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:09:59.380 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T10:09:59.400 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T10:09:59.743 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:09:59.789 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T10:10:00.082 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:10:00.475 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:10:00.768 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T10:10:00.965 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:10:01.216 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:10:01.359 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T10:10:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:10:01.627 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:10:01.736 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:10:01.747 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:10:01.837 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:10:01.872 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:02.138 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:10:02.182 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T10:10:02.415 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T10:10:02.534 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:10:02.674 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:10:02.800 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:10:03.151 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:10:03.419 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T10:10:03.670 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T10:10:03.764 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:10:03.912 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:10:04.011 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:10:04.159 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T10:10:04.171 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T10:10:04.410 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:10:04.481 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:10:04.519 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:10:04.607 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T10:10:04.645 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:10:04.892 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:10:04.971 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:10:05.017 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:10:05.272 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T10:10:05.310 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:10:05.556 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:10:05.594 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:10:05.703 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:10:05.854 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:10:05.980 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:10:06.017 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:10:06.318 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T10:10:06.381 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:10:06.469 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T10:10:06.619 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T10:10:06.707 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:10:06.850 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:10:06.879 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:10:07.146 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T10:10:07.251 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:10:07.439 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:10:07.502 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T10:10:07.565 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T10:10:07.620 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T10:10:07.858 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:10:07.879 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:10:07.967 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:10:08.005 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:10:08.285 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:10:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:10:08.611 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:10:08.666 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:10:08.879 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:10:08.950 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T10:10:09.005 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:10:09.164 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:10:09.169 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T10:10:09.252 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T10:10:09.465 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T10:10:09.491 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:10:09.592 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:10:09.747 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:10:09.985 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:10:10.015 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:10:10.278 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:10:10.437 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T10:10:10.702 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:10:10.705 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:10:11.078 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:10:11.199 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:10:11.204 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:11.339 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:11.490 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:10:11.826 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:11.868 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T10:10:11.952 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:12.079 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T10:10:12.103 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:10:12.396 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:12.397 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T10:10:12.505 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:12.606 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:12.674 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:10:12.798 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:10:12.799 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:10:12.933 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:10:13.302 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:10:13.424 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:13.550 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:10:13.693 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:13.802 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:13.903 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:10:13.955 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:10:14.081 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:14.232 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:14.342 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:10:14.703 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:14.850 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:14.980 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:10:15.165 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.282 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.290 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.400 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.416 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.542 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.543 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.660 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:15.745 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:10:15.746 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:10:15.777 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:16.322 INFO:teuthology.orchestra.run.smithi087.stdout:Adding group ceph....done 2024-04-25T10:10:16.360 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:16.486 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:16.645 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:16.748 INFO:teuthology.orchestra.run.smithi087.stdout:Adding system user ceph....done 2024-04-25T10:10:16.771 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:16.900 INFO:teuthology.orchestra.run.smithi087.stdout:Setting system user ceph properties....done 2024-04-25T10:10:16.917 INFO:teuthology.orchestra.run.smithi087.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:10:16.930 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:10:17.056 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:10:17.438 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:10:18.103 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:18.251 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:18.310 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:10:18.311 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:10:18.377 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:18.503 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:18.654 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:18.780 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:18.913 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:18.939 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:19.022 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:19.484 INFO:teuthology.orchestra.run.smithi169.stdout:Adding group ceph....done 2024-04-25T10:10:19.541 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:10:19.541 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:10:19.910 INFO:teuthology.orchestra.run.smithi169.stdout:Adding system user ceph....done 2024-04-25T10:10:20.087 INFO:teuthology.orchestra.run.smithi169.stdout:Setting system user ceph properties....done 2024-04-25T10:10:20.109 INFO:teuthology.orchestra.run.smithi169.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:10:20.157 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:20.252 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:10:20.515 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:10:20.648 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:10:21.159 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:21.340 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:21.423 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:10:21.423 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:10:21.542 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:10:21.542 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:10:22.145 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:22.354 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:22.356 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:10:22.356 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:10:22.489 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:22.987 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:10:23.275 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:10:23.276 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:10:23.616 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:23.923 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:24.006 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:10:24.023 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:24.166 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:24.374 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:10:24.374 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:10:24.643 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:24.901 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:10:24.901 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:10:25.000 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:25.143 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:25.294 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:25.437 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:25.553 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:25.760 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:10:25.761 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:10:25.790 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:10:25.890 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:10:26.042 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:10:26.489 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:26.897 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:10:26.897 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:10:27.235 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:10:27.524 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:27.642 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:27.818 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:27.942 INFO:teuthology.orchestra.run.smithi087.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:10:27.948 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:10:28.021 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T10:10:28.028 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:10:28.028 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:10:28.196 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-04-25T10:10:28.196 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:10:28.371 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:10:28.372 INFO:teuthology.orchestra.run.smithi087.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:10:28.372 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:10:28.372 INFO:teuthology.orchestra.run.smithi087.stdout:The following additional packages will be installed: 2024-04-25T10:10:28.373 INFO:teuthology.orchestra.run.smithi087.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:10:28.373 INFO:teuthology.orchestra.run.smithi087.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:10:28.373 INFO:teuthology.orchestra.run.smithi087.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:10:28.373 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:10:28.373 INFO:teuthology.orchestra.run.smithi087.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:10:28.374 INFO:teuthology.orchestra.run.smithi087.stdout:Suggested packages: 2024-04-25T10:10:28.374 INFO:teuthology.orchestra.run.smithi087.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:10:28.374 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:10:28.374 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:10:28.428 INFO:teuthology.orchestra.run.smithi087.stdout:The following NEW packages will be installed: 2024-04-25T10:10:28.430 INFO:teuthology.orchestra.run.smithi087.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:10:28.430 INFO:teuthology.orchestra.run.smithi087.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:10:28.430 INFO:teuthology.orchestra.run.smithi087.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:10:28.430 INFO:teuthology.orchestra.run.smithi087.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:10:28.430 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:10:28.431 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:10:28.431 INFO:teuthology.orchestra.run.smithi087.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:10:28.431 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages will be upgraded: 2024-04-25T10:10:28.431 INFO:teuthology.orchestra.run.smithi087.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:10:28.616 INFO:teuthology.orchestra.run.smithi087.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T10:10:28.617 INFO:teuthology.orchestra.run.smithi087.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:10:28.617 INFO:teuthology.orchestra.run.smithi087.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:10:28.617 INFO:teuthology.orchestra.run.smithi087.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:10:28.627 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:28.795 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:28.921 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:29.097 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:10:29.276 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:10:29.293 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:10:29.420 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:10:29.421 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:10:29.425 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:10:29.452 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:10:29.453 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:10:29.457 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:10:29.458 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:10:29.459 INFO:teuthology.orchestra.run.smithi087.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:10:29.460 INFO:teuthology.orchestra.run.smithi087.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:10:29.551 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:10:29.626 INFO:teuthology.orchestra.run.smithi087.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:10:29.702 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:10:29.923 INFO:teuthology.orchestra.run.smithi087.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:10:29.931 INFO:teuthology.orchestra.run.smithi087.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:10:29.936 INFO:teuthology.orchestra.run.smithi087.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:10:30.021 INFO:teuthology.orchestra.run.smithi087.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:10:30.988 INFO:teuthology.orchestra.run.smithi087.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:10:31.015 INFO:teuthology.orchestra.run.smithi087.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:10:31.016 INFO:teuthology.orchestra.run.smithi087.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:10:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:10:31.019 INFO:teuthology.orchestra.run.smithi087.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:10:31.024 INFO:teuthology.orchestra.run.smithi087.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:10:31.189 INFO:teuthology.orchestra.run.smithi087.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:10:31.257 INFO:teuthology.orchestra.run.smithi169.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:10:31.262 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2024-04-25T10:10:31.509 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2024-04-25T10:10:31.510 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:10:31.683 INFO:teuthology.orchestra.run.smithi169.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:10:31.684 INFO:teuthology.orchestra.run.smithi169.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:10:31.684 INFO:teuthology.orchestra.run.smithi169.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:10:31.684 INFO:teuthology.orchestra.run.smithi169.stdout:The following additional packages will be installed: 2024-04-25T10:10:31.685 INFO:teuthology.orchestra.run.smithi169.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:10:31.685 INFO:teuthology.orchestra.run.smithi169.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:10:31.685 INFO:teuthology.orchestra.run.smithi169.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:10:31.685 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:10:31.685 INFO:teuthology.orchestra.run.smithi169.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:10:31.686 INFO:teuthology.orchestra.run.smithi169.stdout:Suggested packages: 2024-04-25T10:10:31.686 INFO:teuthology.orchestra.run.smithi169.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:10:31.686 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:10:31.686 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:10:31.740 INFO:teuthology.orchestra.run.smithi169.stdout:The following NEW packages will be installed: 2024-04-25T10:10:31.741 INFO:teuthology.orchestra.run.smithi169.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:10:31.741 INFO:teuthology.orchestra.run.smithi169.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:10:31.741 INFO:teuthology.orchestra.run.smithi169.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:10:31.742 INFO:teuthology.orchestra.run.smithi169.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:10:31.742 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:10:31.742 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:10:31.742 INFO:teuthology.orchestra.run.smithi169.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:10:31.743 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages will be upgraded: 2024-04-25T10:10:31.743 INFO:teuthology.orchestra.run.smithi169.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:10:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T10:10:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:10:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:10:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:10:31.811 INFO:teuthology.orchestra.run.smithi087.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:10:31.811 INFO:teuthology.orchestra.run.smithi087.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:10:31.972 INFO:teuthology.orchestra.run.smithi169.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:10:31.977 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi087.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi087.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:10:32.009 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:10:32.010 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:10:32.011 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:10:32.019 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:10:32.020 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:10:32.021 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:10:32.022 INFO:teuthology.orchestra.run.smithi169.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:10:32.023 INFO:teuthology.orchestra.run.smithi169.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:10:32.024 INFO:teuthology.orchestra.run.smithi169.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:10:32.032 INFO:teuthology.orchestra.run.smithi169.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:10:32.037 INFO:teuthology.orchestra.run.smithi087.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:10:32.045 INFO:teuthology.orchestra.run.smithi169.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:10:32.045 INFO:teuthology.orchestra.run.smithi169.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:10:32.046 INFO:teuthology.orchestra.run.smithi169.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:10:32.095 INFO:teuthology.orchestra.run.smithi087.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:10:32.102 INFO:teuthology.orchestra.run.smithi087.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:10:32.102 INFO:teuthology.orchestra.run.smithi087.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:10:32.103 INFO:teuthology.orchestra.run.smithi087.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:10:32.104 INFO:teuthology.orchestra.run.smithi087.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:10:32.281 INFO:teuthology.orchestra.run.smithi169.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:10:32.283 INFO:teuthology.orchestra.run.smithi169.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:10:32.284 INFO:teuthology.orchestra.run.smithi169.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:10:32.287 INFO:teuthology.orchestra.run.smithi169.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:10:32.287 INFO:teuthology.orchestra.run.smithi169.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:10:32.292 INFO:teuthology.orchestra.run.smithi169.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:10:32.294 INFO:teuthology.orchestra.run.smithi169.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:10:32.301 INFO:teuthology.orchestra.run.smithi169.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:10:32.301 INFO:teuthology.orchestra.run.smithi169.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:10:32.315 INFO:teuthology.orchestra.run.smithi087.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:10:32.323 INFO:teuthology.orchestra.run.smithi169.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:10:32.324 INFO:teuthology.orchestra.run.smithi169.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:10:32.334 INFO:teuthology.orchestra.run.smithi169.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:10:32.356 INFO:teuthology.orchestra.run.smithi169.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:10:32.360 INFO:teuthology.orchestra.run.smithi169.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:10:32.360 INFO:teuthology.orchestra.run.smithi169.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:10:32.361 INFO:teuthology.orchestra.run.smithi169.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:10:32.362 INFO:teuthology.orchestra.run.smithi169.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:10:32.374 INFO:teuthology.orchestra.run.smithi169.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:10:32.377 INFO:teuthology.orchestra.run.smithi169.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:10:32.535 INFO:teuthology.orchestra.run.smithi169.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:10:32.535 INFO:teuthology.orchestra.run.smithi169.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:10:32.535 INFO:teuthology.orchestra.run.smithi169.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:10:32.535 INFO:teuthology.orchestra.run.smithi169.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:10:32.536 INFO:teuthology.orchestra.run.smithi169.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:10:32.536 INFO:teuthology.orchestra.run.smithi169.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:10:32.549 INFO:teuthology.orchestra.run.smithi169.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:10:32.549 INFO:teuthology.orchestra.run.smithi169.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:10:32.735 INFO:teuthology.orchestra.run.smithi087.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:10:32.929 INFO:teuthology.orchestra.run.smithi169.stdout:Preconfiguring packages ... 2024-04-25T10:10:33.076 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 55.7 MB in 1s (69.9 MB/s) 2024-04-25T10:10:33.370 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:10:33.373 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:10:33.509 INFO:teuthology.orchestra.run.smithi087.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:10:33.509 INFO:teuthology.orchestra.run.smithi087.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:10:33.509 INFO:teuthology.orchestra.run.smithi087.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:10:33.509 INFO:teuthology.orchestra.run.smithi087.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:10:33.509 INFO:teuthology.orchestra.run.smithi087.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:10:33.510 INFO:teuthology.orchestra.run.smithi087.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi087.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi087.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:10:33.922 INFO:teuthology.orchestra.run.smithi087.stdout:Preconfiguring packages ... 2024-04-25T10:10:34.092 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 55.7 MB in 5s (10.9 MB/s) 2024-04-25T10:10:34.234 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:10:34.397 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:10:34.400 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:10:34.567 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:10:35.182 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:10:35.299 INFO:teuthology.orchestra.run.smithi169.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:10:35.588 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package flex. 2024-04-25T10:10:35.657 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:10:35.660 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:10:35.704 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:10:36.273 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:10:36.273 INFO:teuthology.orchestra.run.smithi087.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:10:36.434 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:10:36.520 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package flex. 2024-04-25T10:10:36.593 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:10:36.644 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:10:36.941 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:10:37.196 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:10:37.480 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:10:37.484 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:10:37.617 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:10:38.037 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:10:38.155 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:10:38.189 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:10:38.563 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:10:38.704 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:10:38.707 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:10:38.966 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:10:38.990 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package bison. 2024-04-25T10:10:39.067 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:10:39.071 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:10:39.118 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:10:39.327 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:10:39.472 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:10:39.527 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:10:39.542 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:10:39.602 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:10:39.820 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:10:39.955 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package dump. 2024-04-25T10:10:39.970 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:10:40.013 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:10:40.180 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package bison. 2024-04-25T10:10:40.257 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:10:40.261 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:10:40.307 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:10:40.325 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package indent. 2024-04-25T10:10:40.341 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:40.384 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:10:40.611 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:10:40.626 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:10:40.686 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:10:40.699 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:10:40.714 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:10:40.758 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:10:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:10:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:10:41.098 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:10:41.111 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package dump. 2024-04-25T10:10:41.126 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:10:41.169 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:10:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:10:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:10:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package indent. 2024-04-25T10:10:41.463 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:41.497 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:10:41.740 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:10:41.783 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:10:41.788 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:10:41.803 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:10:42.107 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:10:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:10:42.122 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:10:42.127 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:10:42.165 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:10:42.170 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:10:42.439 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:10:42.451 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:10:42.455 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:10:42.466 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:10:42.506 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:10:42.509 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:10:42.776 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:10:42.791 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:10:42.812 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:10:42.826 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:10:42.828 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:10:42.862 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:10:43.146 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:10:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:10:43.205 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:10:43.474 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:10:43.490 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:10:43.524 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:10:43.810 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:10:43.860 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:10:44.298 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:10:44.314 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:10:44.357 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:10:44.651 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:10:44.667 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:10:44.709 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:10:45.054 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:10:45.070 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:10:45.112 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:10:45.280 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:10:45.295 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:10:45.338 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:10:45.473 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:10:45.489 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:10:45.601 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:10:45.616 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:10:45.659 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:10:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:10:45.901 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:10:45.943 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:10:45.945 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:10:45.961 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:10:46.003 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:10:46.313 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:10:46.328 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:46.373 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:10:46.388 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:10:46.404 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:46.431 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:10:46.759 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:10:46.774 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:10:46.774 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:10:46.789 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:46.817 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:10:46.832 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:10:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:10:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:47.170 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:10:47.185 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:47.220 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:10:47.464 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:10:47.480 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:10:47.514 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:10:47.598 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:10:47.613 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:47.656 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:10:47.975 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:10:47.990 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:48.097 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:10:48.111 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:10:48.159 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:10:48.181 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:10:48.479 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:10:48.494 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:10:48.504 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:10:48.520 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:10:48.554 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:10:48.562 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:10:49.065 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:10:49.080 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:10:49.123 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:10:49.245 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:10:49.261 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:10:49.311 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:10:49.552 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:10:49.567 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:10:49.618 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:10:49.977 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ppp. 2024-04-25T10:10:49.993 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:10:50.267 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:10:50.283 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:10:50.325 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:10:50.340 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:10:50.698 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:10:50.713 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:10:50.757 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:10:50.948 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ppp. 2024-04-25T10:10:50.964 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:10:51.026 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:10:51.041 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:10:51.076 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:10:51.327 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:10:51.379 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:10:51.394 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:10:51.429 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:10:51.670 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:10:51.684 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:10:51.728 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:10:51.766 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:10:51.782 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:51.824 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:51.980 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:10:51.994 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:10:52.039 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:10:52.341 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:10:52.355 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:10:52.408 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:10:52.463 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:10:52.479 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:10:52.540 INFO:teuthology.orchestra.run.smithi169.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:10:52.603 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:10:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:10:52.667 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:52.703 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:53.103 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:10:53.119 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:53.164 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:53.307 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:10:53.322 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi087.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:10:53.839 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:10:53.853 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:10:53.891 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:54.831 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:10:54.846 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:10:54.889 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:10:55.041 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:10:55.056 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:10:55.092 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:10:55.166 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:10:55.181 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:10:55.216 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:10:55.310 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:10:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:10:55.402 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:10:55.460 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:10:55.475 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:10:55.527 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:10:55.660 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:10:55.704 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:10:55.863 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:10:55.880 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:10:55.921 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:10:56.023 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:10:56.038 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:10:56.081 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:10:56.232 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package tcl. 2024-04-25T10:10:56.248 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:10:56.295 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:10:56.358 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package tcl. 2024-04-25T10:10:56.374 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:10:56.413 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:10:56.685 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:10:56.701 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:10:56.735 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:10:56.828 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:10:56.844 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:10:56.886 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:10:57.114 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:10:57.130 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:10:57.172 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:10:57.215 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:10:57.231 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:10:57.273 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:10:57.584 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:10:57.599 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:10:57.649 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:10:57.651 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:10:57.667 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:10:57.718 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:10:57.977 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:10:58.214 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:10:58.340 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:10:58.500 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:58.626 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:10:58.785 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:58.903 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:10:59.253 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:10:59.847 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:10:59.950 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:11:00.234 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:11:00.342 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:11:00.378 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:11:00.378 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:11:00.719 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:11:00.720 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:11:01.103 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:11:01.204 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:11:01.305 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:11:01.395 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:11:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:11:01.520 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:11:01.582 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:11:01.646 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:11:01.731 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:11:01.773 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:11:01.859 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:11:01.882 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:11:01.985 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:11:02.007 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:11:02.200 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:11:02.326 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:11:02.649 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:11:02.799 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:11:02.894 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:11:02.959 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:11:02.981 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:11:03.090 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:11:03.119 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:11:03.236 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:11:03.267 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:11:03.338 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:11:03.393 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:11:03.480 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:11:03.553 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:11:03.623 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:11:03.645 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:11:03.771 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:11:03.782 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:11:03.844 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:11:03.959 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:11:04.014 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:11:04.110 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:11:04.166 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:11:04.244 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:11:04.267 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:11:04.394 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:11:04.422 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:11:04.530 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:11:04.586 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:11:04.712 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:11:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:05.224 INFO:teuthology.orchestra.run.smithi169.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:11:05.224 INFO:teuthology.orchestra.run.smithi169.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:11:05.224 INFO:teuthology.orchestra.run.smithi169.stdout:- enp3s0f1 2024-04-25T10:11:05.224 INFO:teuthology.orchestra.run.smithi169.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:11:05.224 INFO:teuthology.orchestra.run.smithi169.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:11:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:05.288 INFO:teuthology.orchestra.run.smithi087.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:11:05.288 INFO:teuthology.orchestra.run.smithi087.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:11:05.288 INFO:teuthology.orchestra.run.smithi087.stdout:- enp3s0f1 2024-04-25T10:11:05.288 INFO:teuthology.orchestra.run.smithi087.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:11:05.288 INFO:teuthology.orchestra.run.smithi087.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:11:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:11:05.419 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:11:05.757 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:11:05.799 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:11:06.159 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:11:06.196 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:11:06.889 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:11:07.009 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:11:07.040 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:11:07.134 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:11:07.141 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:11:07.284 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:11:07.294 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:11:08.233 INFO:teuthology.orchestra.run.smithi087.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:11:08.270 INFO:teuthology.orchestra.run.smithi169.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:11:08.489 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:08.489 INFO:teuthology.orchestra.run.smithi087.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:11:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:08.540 INFO:teuthology.orchestra.run.smithi169.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:11:08.740 INFO:teuthology.orchestra.run.smithi087.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:11:08.740 INFO:teuthology.orchestra.run.smithi087.stdout:Removing obsolete dictionary files: 2024-04-25T10:11:08.787 INFO:teuthology.orchestra.run.smithi169.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:11:08.787 INFO:teuthology.orchestra.run.smithi169.stdout:Removing obsolete dictionary files: 2024-04-25T10:11:09.414 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:11:09.475 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:11:09.893 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:11:09.970 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:11:10.275 INFO:teuthology.orchestra.run.smithi087.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:11:10.289 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:11:10.316 INFO:teuthology.orchestra.run.smithi169.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:11:10.334 INFO:teuthology.orchestra.run.smithi169.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout:This user must also own the server process. 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout:The default text search configuration will be set to "english". 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout:Data page checksums are disabled. 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:10.336 INFO:teuthology.orchestra.run.smithi087.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:11:10.337 INFO:teuthology.orchestra.run.smithi087.stdout:creating subdirectories ... ok 2024-04-25T10:11:10.337 INFO:teuthology.orchestra.run.smithi087.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:11:10.361 INFO:teuthology.orchestra.run.smithi087.stdout:selecting default max_connections ... 100 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout:This user must also own the server process. 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout:The default text search configuration will be set to "english". 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout:Data page checksums are disabled. 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:10.383 INFO:teuthology.orchestra.run.smithi169.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:11:10.384 INFO:teuthology.orchestra.run.smithi169.stdout:creating subdirectories ... ok 2024-04-25T10:11:10.384 INFO:teuthology.orchestra.run.smithi169.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:11:10.395 INFO:teuthology.orchestra.run.smithi087.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:11:10.408 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default max_connections ... 100 2024-04-25T10:11:10.409 INFO:teuthology.orchestra.run.smithi087.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:11:10.410 INFO:teuthology.orchestra.run.smithi087.stdout:creating configuration files ... ok 2024-04-25T10:11:10.441 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:11:10.455 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:11:10.456 INFO:teuthology.orchestra.run.smithi169.stdout:creating configuration files ... ok 2024-04-25T10:11:10.574 INFO:teuthology.orchestra.run.smithi087.stdout:running bootstrap script ... ok 2024-04-25T10:11:10.620 INFO:teuthology.orchestra.run.smithi169.stdout:running bootstrap script ... ok 2024-04-25T10:11:11.200 INFO:teuthology.orchestra.run.smithi087.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:11:11.279 INFO:teuthology.orchestra.run.smithi169.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:11:11.749 INFO:teuthology.orchestra.run.smithi087.stdout:syncing data to disk ... ok 2024-04-25T10:11:11.868 INFO:teuthology.orchestra.run.smithi169.stdout:syncing data to disk ... ok 2024-04-25T10:11:12.270 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:11:12.387 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:11:15.342 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:11:15.468 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:11:15.495 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:11:15.594 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:11:15.612 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:11:15.695 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:11:15.747 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:11:15.796 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:11:15.848 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:11:15.965 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:11:16.030 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:11:16.030 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:11:16.250 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:11:16.251 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:11:16.773 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:11:16.873 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:11:16.984 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:11:17.110 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:11:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:11:17.369 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:11:17.454 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:11:17.622 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:11:18.721 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:11:18.813 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:11:19.067 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:11:19.200 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:11:19.360 INFO:teuthology.orchestra.run.smithi087.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:11:19.368 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:19.776 INFO:teuthology.orchestra.run.smithi169.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:11:19.782 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:19.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:11:19.955 DEBUG:teuthology.orchestra.run.smithi087:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:11:19.978 INFO:teuthology.orchestra.run.smithi087.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:11:19.979 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:11:19.979 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:11:19.981 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:11:20.168 DEBUG:teuthology.orchestra.run.smithi169:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:11:20.190 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:11:20.190 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:11:20.190 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:11:20.193 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:11:20.193 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:20.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:11:20.215 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:20.215 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:11:20.253 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:11:20.253 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:20.254 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:11:20.283 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:11:20.351 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:20.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:11:20.369 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:11:20.432 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:11:20.432 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:20.432 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:11:20.453 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:11:20.518 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:20.519 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:11:20.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:11:20.600 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:11:20.600 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:20.600 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:11:20.622 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:11:20.682 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:20.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:11:20.701 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:11:20.760 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:11:20.855 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:11:20.855 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /var/log/ceph 2024-04-25T10:11:20.859 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2024-04-25T10:11:20.877 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:11:20.877 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:11:20.925 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:11:20.943 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:11:20.943 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:11:20.994 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:11:21.149 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:11:21.149 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:11:21.149 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:11:21.149 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:21.149 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:11:21.175 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:21.272 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:21.272 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:11:21.280 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:21.373 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:11:21.373 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:11:21.373 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673049', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi087.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJcr468dCbAyRDljN9HUUb2/PzXVQ0QK1aErESi+0DrYMfMi4fjQk7VR3P2ayslklWuqtkwYsQUGme2SgFz3kx0=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGLHf4dNqReLIpyXlLKILzvh/VptNbqJtbvpPGa6bvreCFr44OYz9LhiT49NJzSyXIJPiuJMBCnnB1EIzrKF1sk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:11:21.374 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:11:21.377 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:11:21.418 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:11:21.429 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:11:21.481 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:21.481 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:11:21.498 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:11:21.499 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-04-25T10:11:21.551 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:11:21.551 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:21.551 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 885 Links: 1 2024-04-25T10:11:21.551 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:21.551 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 10:04:57.474397822 +0000 2024-04-25T10:11:21.551 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 10:04:57.226400034 +0000 2024-04-25T10:11:21.551 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 10:04:57.226400034 +0000 2024-04-25T10:11:21.552 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 10:04:57.226400034 +0000 2024-04-25T10:11:21.552 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:11:21.612 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:11:21.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:11:21.613 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000396934 s, 1.3 MB/s 2024-04-25T10:11:21.615 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:11:21.669 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 904 Links: 1 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 10:04:58.086392363 +0000 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 10:04:57.810394825 +0000 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 10:04:57.810394825 +0000 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 10:04:57.810394825 +0000 2024-04-25T10:11:21.724 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:11:21.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:11:21.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:11:21.785 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000374893 s, 1.4 MB/s 2024-04-25T10:11:21.787 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:11:21.841 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-04-25T10:11:21.891 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:11:21.891 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:21.891 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 919 Links: 1 2024-04-25T10:11:21.892 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:21.892 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 10:04:58.654387296 +0000 2024-04-25T10:11:21.892 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 10:04:58.386389686 +0000 2024-04-25T10:11:21.892 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 10:04:58.386389686 +0000 2024-04-25T10:11:21.892 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 10:04:58.386389686 +0000 2024-04-25T10:11:21.892 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:11:21.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:11:21.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:11:21.953 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000377142 s, 1.4 MB/s 2024-04-25T10:11:21.955 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:11:22.009 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-04-25T10:11:22.059 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:11:22.059 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:22.059 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 937 Links: 1 2024-04-25T10:11:22.059 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:22.059 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 10:04:59.254381942 +0000 2024-04-25T10:11:22.059 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 10:04:58.978384405 +0000 2024-04-25T10:11:22.059 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 10:04:58.978384405 +0000 2024-04-25T10:11:22.060 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 10:04:58.978384405 +0000 2024-04-25T10:11:22.060 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:11:22.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:11:22.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:11:22.121 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000389244 s, 1.3 MB/s 2024-04-25T10:11:22.123 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:11:22.177 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:22.177 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:22.177 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:11:22.186 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:11:22.187 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 896 Links: 1 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:04:58.931913432 +0000 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:04:58.643916001 +0000 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:04:58.643916001 +0000 2024-04-25T10:11:22.239 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:04:58.643916001 +0000 2024-04-25T10:11:22.240 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:11:22.301 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:11:22.301 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:11:22.301 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000697184 s, 734 kB/s 2024-04-25T10:11:22.303 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:11:22.356 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-25T10:11:22.406 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:11:22.407 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:22.407 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 912 Links: 1 2024-04-25T10:11:22.407 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:22.407 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:04:59.491908436 +0000 2024-04-25T10:11:22.407 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:04:59.199911041 +0000 2024-04-25T10:11:22.407 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:04:59.199911041 +0000 2024-04-25T10:11:22.407 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:04:59.199911041 +0000 2024-04-25T10:11:22.407 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:11:22.468 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:11:22.469 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:11:22.469 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000377346 s, 1.4 MB/s 2024-04-25T10:11:22.471 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:11:22.525 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-25T10:11:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:11:22.575 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 930 Links: 1 2024-04-25T10:11:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:05:00.055903404 +0000 2024-04-25T10:11:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:04:59.767905974 +0000 2024-04-25T10:11:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:04:59.767905974 +0000 2024-04-25T10:11:22.575 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:04:59.767905974 +0000 2024-04-25T10:11:22.575 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:11:22.636 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:11:22.637 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:11:22.637 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000377821 s, 1.4 MB/s 2024-04-25T10:11:22.639 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:11:22.692 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 945 Links: 1 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 10:05:00.627898303 +0000 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 10:05:00.379900515 +0000 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 10:05:00.379900515 +0000 2024-04-25T10:11:22.747 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 10:05:00.379900515 +0000 2024-04-25T10:11:22.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:11:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T10:11:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T10:11:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000421003 s, 1.2 MB/s 2024-04-25T10:11:22.811 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:11:22.864 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:22.864 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:11:22.864 INFO:tasks.ceph:Generating config... 2024-04-25T10:11:22.866 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:11:22.866 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:11:22.866 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:11:22.866 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:11:22.866 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:11:22.866 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:11:22.866 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T10:11:22.866 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:11:22.866 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:11:22.866 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:11:22.866 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T10:11:22.866 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:11:22.867 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:11:22.867 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:11:22.867 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:11:22.868 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:11:22.870 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:11:22.870 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:11:22.915 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:11:22.940 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:11:22.987 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:11:23.049 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.87'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2024-04-25T10:11:23.050 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:11:23.051 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:23.051 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:11:23.098 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.87 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:23.189 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:23.189 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: generated fsid 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:11:23.189 INFO:teuthology.orchestra.run.smithi087.stdout:setting min_mon_release = quincy 2024-04-25T10:11:23.189 INFO:teuthology.orchestra.run.smithi087.stdout:epoch 0 2024-04-25T10:11:23.189 INFO:teuthology.orchestra.run.smithi087.stdout:fsid 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:11:23.190 INFO:teuthology.orchestra.run.smithi087.stdout:last_changed 2024-04-25T10:11:23.187117+0000 2024-04-25T10:11:23.190 INFO:teuthology.orchestra.run.smithi087.stdout:created 2024-04-25T10:11:23.187117+0000 2024-04-25T10:11:23.190 INFO:teuthology.orchestra.run.smithi087.stdout:min_mon_release 17 (quincy) 2024-04-25T10:11:23.190 INFO:teuthology.orchestra.run.smithi087.stdout:election_strategy: 1 2024-04-25T10:11:23.190 INFO:teuthology.orchestra.run.smithi087.stdout:0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-04-25T10:11:23.190 INFO:teuthology.orchestra.run.smithi087.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-25T10:11:23.191 INFO:teuthology.orchestra.run.smithi087.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2024-04-25T10:11:23.191 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:11:23.192 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:11:23.200 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b... 2024-04-25T10:11:23.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:11:23.245 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-04-25T10:11:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-04-25T10:11:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on skipped message = False 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-04-25T10:11:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-04-25T10:11:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: bdev async discard = True 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: bdev enable discard = True 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore allocator = bitmap 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-04-25T10:11:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 1/20 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 1/20 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 4/10 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-04-25T10:11:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-04-25T10:11:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-04-25T10:11:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-04-25T10:11:23.294 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: mds cache max size = 1000 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = True 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: mds log max segments = 1 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = True 2024-04-25T10:11:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:11:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:11:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:11:23.296 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-25T10:11:23.296 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-25T10:11:23.296 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:11:23.296 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-25T10:11:23.296 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:11:23.297 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-25T10:11:23.298 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-25T10:11:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-25T10:11:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-04-25T10:11:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.302 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-25T10:11:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:11:23.304 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:11:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout: mds cache max size = 1000 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2024-04-25T10:11:23.306 INFO:teuthology.orchestra.run.smithi169.stdout: mds log max segments = 1 2024-04-25T10:11:23.307 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-25T10:11:23.307 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2024-04-25T10:11:23.307 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:11:23.307 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:11:23.307 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:11:23.308 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:11:23.308 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:11:23.405 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:11:23.406 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:23.406 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:11:23.415 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:23.415 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:11:23.470 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:11:23.470 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:23.470 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:11:23.470 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:11:23.541 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:23.541 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:23.590 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:11:23.590 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:23.591 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:11:23.591 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:11:23.626 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:23.626 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:23.679 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:11:23.680 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:11:23.680 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:11:23.734 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:11:23.739 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:11:23.797 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:11:23.819 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:11:23.819 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:11:23.878 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:11:23.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:11:23.991 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:11:23.995 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:11:24.015 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:11:24.072 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:11:24.076 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:11:24.096 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:11:24.191 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:11:24.196 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:11:24.215 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:11:24.216 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:11:24.257 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:11:24.274 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:11:24.360 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:11:24.376 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:11:24.417 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:11:24.434 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:11:24.519 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:11:24.536 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:11:24.536 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:11:24.536 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:11:24.555 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:24.556 INFO:tasks.ceph:role: osd.0 2024-04-25T10:11:24.556 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:11:24.556 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:11:24.628 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:24.629 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:25.013 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T10:11:25.015 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T10:11:25.016 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:11:25.126 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:11:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:25.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:25.143 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:11:25.205 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:25.205 INFO:tasks.ceph:role: osd.1 2024-04-25T10:11:25.205 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:11:25.206 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:11:25.274 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:25.274 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:25.274 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:25.274 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:25.274 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:25.274 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:25.275 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:25.275 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:25.275 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:25.275 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:25.667 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T10:11:25.670 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T10:11:25.670 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:11:25.817 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:11:25.833 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:25.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:25.834 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:11:25.897 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:25.898 INFO:tasks.ceph:role: osd.2 2024-04-25T10:11:25.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:11:25.898 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:11:25.965 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:26.385 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T10:11:26.388 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T10:11:26.388 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:11:26.524 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:11:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:26.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:26.542 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:11:26.602 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:26.602 INFO:tasks.ceph:role: osd.3 2024-04-25T10:11:26.602 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:11:26.602 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:11:26.669 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:26.669 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:26.669 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:26.669 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:26.669 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:26.669 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:26.670 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:26.670 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:26.670 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:26.670 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:27.027 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T10:11:27.029 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T10:11:27.029 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:11:27.117 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:11:27.175 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:27.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:27.176 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:27.295+0000 7fbc0cb367c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:27.295+0000 7fbc0cb367c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:27.295+0000 7fbc0cb367c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:27.295+0000 7fbc0cb367c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:27.299+0000 7fbc0cb367c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:11:27.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:27.303+0000 7fbc0cb367c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:11:30.380 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:11:30.401 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:30.491+0000 7f2319f197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:11:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:30.491+0000 7f2319f197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:11:30.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:30.491+0000 7f2319f197c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:11:30.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:30.491+0000 7f2319f197c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:11:30.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:30.491+0000 7f2319f197c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:11:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:30.491+0000 7f2319f197c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:11:33.493 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:11:33.513 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:33.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:33.603+0000 7fc30a9d87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:11:33.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:33.603+0000 7fc30a9d87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:11:33.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:33.603+0000 7fc30a9d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:11:33.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:33.603+0000 7fc30a9d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:11:33.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:33.603+0000 7fc30a9d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:11:33.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:33.603+0000 7fc30a9d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:11:36.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:11:36.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:36.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:36.771+0000 7f8a164487c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:11:36.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:36.771+0000 7f8a164487c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:11:36.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:36.771+0000 7f8a164487c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:11:36.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:36.771+0000 7f8a164487c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:11:36.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:36.771+0000 7f8a164487c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:11:36.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:36.775+0000 7f8a164487c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:11:39.844 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:11:39.863 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:11:39.885 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:39.885 INFO:tasks.ceph:role: osd.4 2024-04-25T10:11:39.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:11:39.885 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:39.954 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:40.352 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T10:11:40.355 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T10:11:40.356 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:11:40.459 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:11:40.476 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:40.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:40.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:11:40.538 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:40.538 INFO:tasks.ceph:role: osd.5 2024-04-25T10:11:40.538 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:11:40.538 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:11:40.614 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:40.614 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:40.615 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T10:11:41.019 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T10:11:41.019 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:11:41.144 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:11:41.160 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:41.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:41.161 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:11:41.225 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:41.226 INFO:tasks.ceph:role: osd.6 2024-04-25T10:11:41.226 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:11:41.226 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:11:41.297 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:41.297 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:41.297 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:41.297 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:41.297 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:41.297 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:41.297 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:41.298 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:41.298 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:41.298 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:41.714 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T10:11:41.717 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T10:11:41.718 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:11:41.843 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:11:41.860 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:41.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:41.862 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:11:41.926 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:11:41.926 INFO:tasks.ceph:role: osd.7 2024-04-25T10:11:41.926 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:11:41.926 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:11:42.002 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:11:42.002 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:11:42.002 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:11:42.002 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:11:42.003 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:11:42.003 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:11:42.003 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:11:42.003 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:11:42.003 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:11:42.003 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:11:42.369 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T10:11:42.371 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T10:11:42.372 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:11:42.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:11:42.506 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:11:42.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:42.507 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:42.633 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:42.623+0000 7f9e67a2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:11:42.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:42.627+0000 7f9e67a2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:11:42.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:42.627+0000 7f9e67a2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:11:42.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:42.627+0000 7f9e67a2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:11:42.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:42.627+0000 7f9e67a2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:11:42.644 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:42.635+0000 7f9e67a2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:11:45.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:11:45.684 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:45.775 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:45.767+0000 7f2e04d677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:11:45.776 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:45.767+0000 7f2e04d677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:11:45.776 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:45.767+0000 7f2e04d677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:11:45.776 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:45.767+0000 7f2e04d677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:11:45.776 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:45.767+0000 7f2e04d677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:11:45.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:45.767+0000 7f2e04d677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:11:48.841 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:11:48.862 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:48.960 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:48.951+0000 7f6d40eba7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:11:48.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:48.951+0000 7f6d40eba7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:11:48.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:48.951+0000 7f6d40eba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:11:48.962 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:48.951+0000 7f6d40eba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:11:48.962 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:48.951+0000 7f6d40eba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:11:48.963 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:48.955+0000 7f6d40eba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:11:51.375 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:51.379 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:51.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:11:51.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:52.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:52.083+0000 7f6b6d0487c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:52.087+0000 7f6b6d0487c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:52.087+0000 7f6b6d0487c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:52.087+0000 7f6b6d0487c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:52.087+0000 7f6b6d0487c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:11:52.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:52.087+0000 7f6b6d0487c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:11:55.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:11:55.143 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:11:55.143 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.143 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:11:55.165 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.165 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:11:55.227 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:11:55.290 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.291 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:11:55.354 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:11:55.422 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.422 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:11:55.486 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:11:55.550 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:55.550 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:11:55.571 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:55.571 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:11:55.633 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:55.634 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:11:55.697 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:55.697 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:11:55.761 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:55.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:11:55.829 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:55.830 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:11:55.893 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:55.893 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:11:55.954 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.954 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:11:55.963 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:55.963 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:11:56.014 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:56.014 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:11:56.023 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:56.023 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:11:56.073 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:11:56.074 DEBUG:teuthology.orchestra.run.smithi087:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:11:56.077 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:11:56.121 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.y] 2024-04-25T10:11:56.121 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ== 2024-04-25T10:11:56.121 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T10:11:56.121 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw== 2024-04-25T10:11:56.121 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg== 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.0] 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg== 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.1] 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ== 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.2] 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w== 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.3] 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ== 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.x] 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg== 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.4] 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.5] 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.6] 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.7] 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[client.2] 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew== 2024-04-25T10:11:56.123 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2024-04-25T10:11:56.124 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg== 2024-04-25T10:11:56.124 INFO:teuthology.orchestra.run.smithi087.stdout:[client.3] 2024-04-25T10:11:56.124 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g== 2024-04-25T10:11:56.135 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2024-04-25T10:11:56.135 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ== 2024-04-25T10:11:56.135 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T10:11:56.135 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw== 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg== 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg== 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ== 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w== 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ== 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg== 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T10:11:56.136 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew== 2024-04-25T10:11:56.137 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-04-25T10:11:56.138 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg== 2024-04-25T10:11:56.138 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2024-04-25T10:11:56.138 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g== 2024-04-25T10:11:56.139 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:11:56.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:11:56.240 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.243 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.303 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.306 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.363 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.367 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.420 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.423 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.477 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.480 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.534 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.590 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:11:56.593 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:11:56.646 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.649 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.705 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:11:56.764 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.768 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.820 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.823 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.876 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.932 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:11:56.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:56.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:57.049 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:57.052 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:57.108 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:57.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:57.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:57.170 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:11:57.226 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:11:57.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:11:57.246 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:11:57.351 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:11:57.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:11:57.390 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:11:57.499 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:11:57.519 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:11:57.579 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:11:57.693 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:11:57.712 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:57.715 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:11:57.761 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:11:57.761 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:11:57.761 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:11:57.764 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:11:57.765 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:11:57.765 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:11:57.803 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:11:57.804 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:11:57.804 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:11:57.806 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:11:57.807 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:11:57.807 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:11:57.807 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:11:57.810 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:11:57.810 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:11:57.810 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:11:57.813 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:11:57.813 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:11:57.814 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:11:57.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6010e00640 1 Processor -- start 2024-04-25T10:11:57.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6010e00640 1 -- start start 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6010e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f600c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6010e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c0b2fd0 con 0x7f600c0aea50 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6010e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c0b32b0 con 0x7f600c06c0d0 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6010e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0af1d0 con 0x7f600c15a0e0 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6001400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f600c0aea50 0x7f600c0b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53156/0 (socket says 172.21.15.87:53156) 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6001400640 1 -- 172.21.15.87:0/2021012533 learned_addr learned my addr 172.21.15.87:0/2021012533 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6009e00640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f600c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6009e00640 1 -- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 msgr2=0x7f600c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6009e00640 1 -- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 msgr2=0x7f600c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6009e00640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f600c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:57.946+0000 7f6009e00640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f600c15b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:11:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936208045 0 0) 0x7f600c0b2fd0 con 0x7f600c0aea50 2024-04-25T10:11:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff0005740 con 0x7f600c0aea50 2024-04-25T10:11:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28659358 0 0) 0x7f5ff0005740 con 0x7f600c0aea50 2024-04-25T10:11:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 msgr2=0x7f600c15b150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:11:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f600c15b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 >> v1:172.21.15.169:6789/0 conn(0x7f600c06c0d0 legacy=0x7f600c0705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0b3590 con 0x7f600c0aea50 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5ff4001680 con 0x7f600c0aea50 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 >> v1:172.21.15.87:6789/0 conn(0x7f600c0aea50 legacy=0x7f600c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f5ff0005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ff0006280 0x7f5ff0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ff0006f60 0x7f5ff00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0af1d0 con 0x7f5ff0006f60 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff000dec0 con 0x7f5ff0006280 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00035b0 con 0x7f600c15a0e0 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6001400640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ff0006280 0x7f5ff0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6009e00640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f5ff0005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f600a800640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ff0006f60 0x7f5ff00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f600a800640 1 -- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 msgr2=0x7f5ff0005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f600a800640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f600c15a0e0 0x7f5ff0005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f600a800640 1 -- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ff0006280 msgr2=0x7f5ff0006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f600a800640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ff0006280 0x7f5ff0006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f600a800640 1 -- 172.21.15.87:0/2021012533 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0005740 con 0x7f5ff0006f60 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.102+0000 7f6001400640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ff0006280 0x7f5ff0006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:58.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f600a800640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ff0006f60 0x7f5ff00073e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6004001780 tx=0x7f600401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d1140379adf6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040013a0 con 0x7f5ff0006f60 2024-04-25T10:11:58.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60040052f0 con 0x7f5ff0006f60 2024-04-25T10:11:58.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6009400640 1 -- 172.21.15.87:0/2021012533 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004003840 con 0x7f5ff0006f60 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6010e00640 1 -- 172.21.15.87:0/2021012533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ff0006f60 msgr2=0x7f5ff00073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6010e00640 1 --2- 172.21.15.87:0/2021012533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ff0006f60 0x7f5ff00073e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6004001780 tx=0x7f600401c420 comp rx=0 tx=0).stop 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f600a800640 1 -- 172.21.15.87:0/2021012533 reap_dead start 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6010e00640 1 -- 172.21.15.87:0/2021012533 shutdown_connections 2024-04-25T10:11:58.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6010e00640 1 -- 172.21.15.87:0/2021012533 >> 172.21.15.87:0/2021012533 conn(0x7f600c0669a0 msgr2=0x7f600c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6010e00640 1 -- 172.21.15.87:0/2021012533 shutdown_connections 2024-04-25T10:11:58.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.106+0000 7f6010e00640 1 -- 172.21.15.87:0/2021012533 wait complete. 2024-04-25T10:11:58.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 Processor -- start 2024-04-25T10:11:58.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 -- start start 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ff0006f60 0x7f600c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f600c151120 0x7f600c158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600c14fba0 0x7f600c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0b44a0 con 0x7f600c14fba0 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0aee90 con 0x7f600c151120 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0b4620 con 0x7f5ff0006f60 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600c14fba0 0x7f600c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600c14fba0 0x7f600c158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57952/0 (socket says 172.21.15.87:57952) 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 -- 172.21.15.87:0/3964269158 learned_addr learned my addr 172.21.15.87:0/3964269158 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6001400640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f600c151120 0x7f600c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6009e00640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ff0006f60 0x7f600c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 -- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ff0006f60 msgr2=0x7f600c150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ff0006f60 0x7f600c150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 -- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f600c151120 msgr2=0x7f600c158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f600c151120 0x7f600c158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 -- 172.21.15.87:0/3964269158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c1f0620 con 0x7f600c14fba0 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f600a800640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600c14fba0 0x7f600c158550 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6004001780 tx=0x7f5ff0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6002a00640 1 -- 172.21.15.87:0/3964269158 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040ba020 con 0x7f600c14fba0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6002a00640 1 -- 172.21.15.87:0/3964269158 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60040054a0 con 0x7f600c14fba0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6002a00640 1 -- 172.21.15.87:0/3964269158 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4007c70 con 0x7f600c14fba0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c1516a0 con 0x7f600c14fba0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.110+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c1f08c0 con 0x7f600c14fba0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.114+0000 7f6002a00640 1 -- 172.21.15.87:0/3964269158 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f60040bd070 con 0x7f600c14fba0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.114+0000 7f6002a00640 1 -- 172.21.15.87:0/3964269158 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6004003bc0 con 0x7f600c14fba0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.114+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c04ef30 con 0x7f600c14fba0 2024-04-25T10:11:58.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.138+0000 7f6002a00640 1 -- 172.21.15.87:0/3964269158 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f60040d6dd0 con 0x7f600c14fba0 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f600c06e1f0 con 0x7f600c14fba0 2024-04-25T10:11:58.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6002a00640 1 -- 172.21.15.87:0/3964269158 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f60040c5020 con 0x7f600c14fba0 2024-04-25T10:11:58.205 INFO:teuthology.orchestra.run.smithi087.stderr:adjusted tunables profile to default 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600c14fba0 msgr2=0x7f600c158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600c14fba0 0x7f600c158550 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6004001780 tx=0x7f5ff0007a50 comp rx=0 tx=0).stop 2024-04-25T10:11:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 shutdown_connections 2024-04-25T10:11:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600c14fba0 0x7f600c158550 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f600c151120 0x7f600c158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 --2- 172.21.15.87:0/3964269158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ff0006f60 0x7f600c150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 >> 172.21.15.87:0/3964269158 conn(0x7f600c0669a0 msgr2=0x7f600c1501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 shutdown_connections 2024-04-25T10:11:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:11:58.198+0000 7f6010e00640 1 -- 172.21.15.87:0/3964269158 wait complete. 2024-04-25T10:11:58.225 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:11:58.225 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:11:58.225 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:58.225 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:11:58.244 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:58.245 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:11:58.305 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:58.305 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:11:58.332 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:11:58.327+0000 7efe9730b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:11:58.345 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:58.338+0000 7f60835ae180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:11:58.364 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:11:58.364 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:11:58.425 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:58.425 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:11:58.445 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:58.445 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:11:58.508 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:58.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:11:58.569 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:11:58.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:11:58.628 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new f9fd7de2-a5ee-4694-9239-9f30b30d37da 0 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32b6c00640 1 Processor -- start 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32b6c00640 1 -- start start 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32b6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b015ac50 0x7f32b0070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32b6c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32b0073d10 0x7f32b006df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32b6c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b00ce350 con 0x7f32b00b3c90 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32b6c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b00ce630 con 0x7f32b0073d10 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32b6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00ce910 con 0x7f32b015ac50 2024-04-25T10:11:58.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32afe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b015ac50 0x7f32b0070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32afe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b015ac50 0x7f32b0070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54550/0 (socket says 172.21.15.169:54550) 2024-04-25T10:11:58.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32afe00640 1 -- 172.21.15.169:0/245671888 learned_addr learned my addr 172.21.15.169:0/245671888 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:58.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32a7400640 1 --2- 172.21.15.169:0/245671888 >> v2:172.21.15.87:3300/0 conn(0x7f32b0073d10 0x7f32b006df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32afe00640 1 -- 172.21.15.169:0/245671888 >> v2:172.21.15.87:3300/0 conn(0x7f32b0073d10 msgr2=0x7f32b006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32afe00640 1 --2- 172.21.15.169:0/245671888 >> v2:172.21.15.87:3300/0 conn(0x7f32b0073d10 0x7f32b006df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32afe00640 1 -- 172.21.15.169:0/245671888 >> v1:172.21.15.169:6789/0 conn(0x7f32b00b3c90 legacy=0x7f32b00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.803+0000 7f32afe00640 1 -- 172.21.15.169:0/245671888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00cebf0 con 0x7f32b015ac50 2024-04-25T10:11:58.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32afe00640 1 --2- 172.21.15.169:0/245671888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b015ac50 0x7f32b0070b70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009870 tx=0x7f32a001c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd38c7b5c09fb27d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32af400640 1 -- 172.21.15.169:0/245671888 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00b0070 con 0x7f32b015ac50 2024-04-25T10:11:58.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32af400640 1 -- 172.21.15.169:0/245671888 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a00ab600 con 0x7f32b015ac50 2024-04-25T10:11:58.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32af400640 1 -- 172.21.15.169:0/245671888 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00ab9d0 con 0x7f32b015ac50 2024-04-25T10:11:58.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 -- 172.21.15.169:0/245671888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b015ac50 msgr2=0x7f32b0070b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 --2- 172.21.15.169:0/245671888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b015ac50 0x7f32b0070b70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009870 tx=0x7f32a001c520 comp rx=0 tx=0).stop 2024-04-25T10:11:58.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 -- 172.21.15.169:0/245671888 shutdown_connections 2024-04-25T10:11:58.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 --2- 172.21.15.169:0/245671888 >> v2:172.21.15.87:3300/0 conn(0x7f32b0073d10 0x7f32b006df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 --2- 172.21.15.169:0/245671888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b015ac50 0x7f32b0070b70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 -- 172.21.15.169:0/245671888 >> 172.21.15.169:0/245671888 conn(0x7f32b00b1ca0 msgr2=0x7f32b00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 -- 172.21.15.169:0/245671888 shutdown_connections 2024-04-25T10:11:58.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.807+0000 7f32b6c00640 1 -- 172.21.15.169:0/245671888 wait complete. 2024-04-25T10:11:58.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 Processor -- start 2024-04-25T10:11:58.815 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 -- start start 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32b0073d10 0x7f32b0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b00b3c90 0x7f32b015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32b015ac50 0x7f32b00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00710b0 con 0x7f32b015ac50 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b006e8d0 con 0x7f32b0073d10 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0071230 con 0x7f32b00b3c90 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32b0073d10 0x7f32b0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32a7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b00b3c90 0x7f32b015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32b0073d10 0x7f32b0152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48390/0 (socket says 172.21.15.169:48390) 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 -- 172.21.15.169:0/566147412 learned_addr learned my addr 172.21.15.169:0/566147412 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:58.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b4c00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32b015ac50 0x7f32b00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 -- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b00b3c90 msgr2=0x7f32b015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b00b3c90 0x7f32b015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 -- 172.21.15.169:0/566147412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32b015ac50 msgr2=0x7f32b00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32b015ac50 0x7f32b00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 -- 172.21.15.169:0/566147412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0151d20 con 0x7f32b0073d10 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32a7400640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b00b3c90 0x7f32b015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32afe00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32b0073d10 0x7f32b0152aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32a00ae040 tx=0x7f32a00192c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32acc00640 1 -- 172.21.15.169:0/566147412 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00b0050 con 0x7f32b0073d10 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32acc00640 1 -- 172.21.15.169:0/566147412 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a00bc070 con 0x7f32b0073d10 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32acc00640 1 -- 172.21.15.169:0/566147412 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00b5870 con 0x7f32b0073d10 2024-04-25T10:11:58.818 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b0151fc0 con 0x7f32b0073d10 2024-04-25T10:11:58.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b0152580 con 0x7f32b0073d10 2024-04-25T10:11:58.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32acc00640 1 -- 172.21.15.169:0/566147412 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f32a00b5a10 con 0x7f32b0073d10 2024-04-25T10:11:58.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.811+0000 7f32acc00640 1 -- 172.21.15.169:0/566147412 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f32a00c1a90 con 0x7f32b0073d10 2024-04-25T10:11:58.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.815+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b004ef30 con 0x7f32b0073d10 2024-04-25T10:11:58.833 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.827+0000 7f32acc00640 1 -- 172.21.15.169:0/566147412 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f32a0016110 con 0x7f32b0073d10 2024-04-25T10:11:58.847 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:58.842+0000 7f60835ae180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:11:58.849 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:11:58.843+0000 7efe9730b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:11:58.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.895+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f9fd7de2-a5ee-4694-9239-9f30b30d37da", "id": 0} v 0) v1 -- 0x7f32b0152750 con 0x7f32b0073d10 2024-04-25T10:11:58.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32acc00640 1 -- 172.21.15.169:0/566147412 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9fd7de2-a5ee-4694-9239-9f30b30d37da", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f32a00c1c90 con 0x7f32b0073d10 2024-04-25T10:11:58.907 INFO:teuthology.orchestra.run.smithi169.stdout:0 2024-04-25T10:11:58.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32b0073d10 msgr2=0x7f32b0152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32b0073d10 0x7f32b0152aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32a00ae040 tx=0x7f32a00192c0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 shutdown_connections 2024-04-25T10:11:58.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32b015ac50 0x7f32b00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32b00b3c90 0x7f32b015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 --2- 172.21.15.169:0/566147412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32b0073d10 0x7f32b0152aa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 >> 172.21.15.169:0/566147412 conn(0x7f32b00b1ca0 msgr2=0x7f32b00b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 shutdown_connections 2024-04-25T10:11:58.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:58.903+0000 7f32b6c00640 1 -- 172.21.15.169:0/566147412 wait complete. 2024-04-25T10:11:58.926 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 7baf96bf-f0b4-4b34-b5ca-78bcd8118464 1 2024-04-25T10:11:59.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f1202c00640 1 Processor -- start 2024-04-25T10:11:59.063 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f1202c00640 1 -- start start 2024-04-25T10:11:59.063 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:59.058+0000 7f60835ae180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:11:59.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f1202c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f1202c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc15fec0 con 0x7f11fc0d1830 2024-04-25T10:11:59.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f1202c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc1601a0 con 0x7f11fc15bc10 2024-04-25T10:11:59.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f1202c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0cc5b0 con 0x7f11fc0cf980 2024-04-25T10:11:59.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11fb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11fbe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f11fc15bc10 0x7f11fc15f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:58112/0 (socket says 172.21.15.169:58112) 2024-04-25T10:11:59.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11fb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54568/0 (socket says 172.21.15.169:54568) 2024-04-25T10:11:59.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11fbe00640 1 -- 172.21.15.169:0/490791121 learned_addr learned my addr 172.21.15.169:0/490791121 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T10:11:59.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11fb400640 1 -- 172.21.15.169:0/490791121 >> v1:172.21.15.169:6789/0 conn(0x7f11fc15bc10 legacy=0x7f11fc15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11fb400640 1 -- 172.21.15.169:0/490791121 >> v1:172.21.15.87:6789/0 conn(0x7f11fc0d1830 legacy=0x7f11fc0d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11fb400640 1 -- 172.21.15.169:0/490791121 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0cc890 con 0x7f11fc0cf980 2024-04-25T10:11:59.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.059+0000 7f11faa00640 1 -- 172.21.15.169:0/490791121 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989189533 0 0) 0x7f11fc1601a0 con 0x7f11fc15bc10 2024-04-25T10:11:59.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f11fb400640 1 --2- 172.21.15.169:0/490791121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0b1060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11e80097f0 tx=0x7f11e801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9235aba7ba86418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f11faa00640 1 -- 172.21.15.169:0/490791121 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80b0070 con 0x7f11fc0cf980 2024-04-25T10:11:59.068 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f11faa00640 1 -- 172.21.15.169:0/490791121 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e80a4e50 con 0x7f11fc0cf980 2024-04-25T10:11:59.068 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f11faa00640 1 -- 172.21.15.169:0/490791121 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80ab820 con 0x7f11fc0cf980 2024-04-25T10:11:59.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f1202c00640 1 -- 172.21.15.169:0/490791121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 msgr2=0x7f11fc0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f1202c00640 1 --2- 172.21.15.169:0/490791121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0b1060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11e80097f0 tx=0x7f11e801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f1202c00640 1 -- 172.21.15.169:0/490791121 shutdown_connections 2024-04-25T10:11:59.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f1202c00640 1 --2- 172.21.15.169:0/490791121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0b1060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f1202c00640 1 -- 172.21.15.169:0/490791121 >> 172.21.15.169:0/490791121 conn(0x7f11fc09a9d0 msgr2=0x7f11fc09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f1202c00640 1 -- 172.21.15.169:0/490791121 shutdown_connections 2024-04-25T10:11:59.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.063+0000 7f1202c00640 1 -- 172.21.15.169:0/490791121 wait complete. 2024-04-25T10:11:59.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 Processor -- start 2024-04-25T10:11:59.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 -- start start 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc0d1830 0x7f11fc0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11fc15bc10 0x7f11fc0c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc169970 con 0x7f11fc15bc10 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc169af0 con 0x7f11fc0d1830 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc169c70 con 0x7f11fc0cf980 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54580/0 (socket says 172.21.15.169:54580) 2024-04-25T10:11:59.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 -- 172.21.15.169:0/1293764187 learned_addr learned my addr 172.21.15.169:0/1293764187 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fb400640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc0d1830 0x7f11fc0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1200c00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11fc15bc10 0x7f11fc0c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 -- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc0d1830 msgr2=0x7f11fc0b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc0d1830 0x7f11fc0b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 -- 172.21.15.169:0/1293764187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11fc15bc10 msgr2=0x7f11fc0c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11fc15bc10 0x7f11fc0c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 -- 172.21.15.169:0/1293764187 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc15b3d0 con 0x7f11fc0cf980 2024-04-25T10:11:59.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1200c00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11fc15bc10 0x7f11fc0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:59.073 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:11:59.067+0000 7efe9730b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:11:59.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11fbe00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0c7660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00c4a0 tx=0x7f11ec00c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11f3e00640 1 -- 172.21.15.169:0/1293764187 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec018070 con 0x7f11fc0cf980 2024-04-25T10:11:59.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11f3e00640 1 -- 172.21.15.169:0/1293764187 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec0074e0 con 0x7f11fc0cf980 2024-04-25T10:11:59.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0c5ac0 con 0x7f11fc0cf980 2024-04-25T10:11:59.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11f3e00640 1 -- 172.21.15.169:0/1293764187 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec004a30 con 0x7f11fc0cf980 2024-04-25T10:11:59.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc15c480 con 0x7f11fc0cf980 2024-04-25T10:11:59.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11f3e00640 1 -- 172.21.15.169:0/1293764187 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f11ec01b070 con 0x7f11fc0cf980 2024-04-25T10:11:59.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.067+0000 7f11f3e00640 1 -- 172.21.15.169:0/1293764187 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f11ec0127b0 con 0x7f11fc0cf980 2024-04-25T10:11:59.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.071+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc04ef30 con 0x7f11fc0cf980 2024-04-25T10:11:59.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.071+0000 7f11f3e00640 1 -- 172.21.15.169:0/1293764187 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f11ec004c30 con 0x7f11fc0cf980 2024-04-25T10:11:59.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.143+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7baf96bf-f0b4-4b34-b5ca-78bcd8118464", "id": 1} v 0) v1 -- 0x7f11fc160090 con 0x7f11fc0cf980 2024-04-25T10:11:59.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.147+0000 7f11f3e00640 1 -- 172.21.15.169:0/1293764187 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7baf96bf-f0b4-4b34-b5ca-78bcd8118464", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f11ec01d440 con 0x7f11fc0cf980 2024-04-25T10:11:59.153 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-25T10:11:59.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 msgr2=0x7f11fc0c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0c7660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00c4a0 tx=0x7f11ec00c960 comp rx=0 tx=0).stop 2024-04-25T10:11:59.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 shutdown_connections 2024-04-25T10:11:59.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11fc15bc10 0x7f11fc0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc0d1830 0x7f11fc0b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 --2- 172.21.15.169:0/1293764187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11fc0cf980 0x7f11fc0c7660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 >> 172.21.15.169:0/1293764187 conn(0x7f11fc09a9d0 msgr2=0x7f11fc06fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 shutdown_connections 2024-04-25T10:11:59.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.151+0000 7f1202c00640 1 -- 172.21.15.169:0/1293764187 wait complete. 2024-04-25T10:11:59.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 7bc3a5c5-e581-4d3f-827e-038a5b979a1a 2 2024-04-25T10:11:59.307 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52e00640 1 Processor -- start 2024-04-25T10:11:59.307 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52e00640 1 -- start start 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d4c0d0920 0x7f0d4c0b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d4c15deb0 0x7f0d4c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c161840 con 0x7f0d4c15dad0 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c161b20 con 0x7f0d4c0d0920 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c161e00 con 0x7f0d4c15deb0 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d4c15deb0 0x7f0d4c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d4c15deb0 0x7f0d4c0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48412/0 (socket says 172.21.15.169:48412) 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52400640 1 -- 172.21.15.169:0/460313219 learned_addr learned my addr 172.21.15.169:0/460313219 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d51000640 1 --2- 172.21.15.169:0/460313219 >> v2:172.21.15.87:3300/0 conn(0x7f0d4c0d0920 0x7f0d4c0b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52400640 1 -- 172.21.15.169:0/460313219 >> v2:172.21.15.87:3300/0 conn(0x7f0d4c0d0920 msgr2=0x7f0d4c0b92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52400640 1 --2- 172.21.15.169:0/460313219 >> v2:172.21.15.87:3300/0 conn(0x7f0d4c0d0920 0x7f0d4c0b92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52400640 1 -- 172.21.15.169:0/460313219 >> v1:172.21.15.87:6789/0 conn(0x7f0d4c15dad0 legacy=0x7f0d4c0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.310 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.303+0000 7f0d52400640 1 -- 172.21.15.169:0/460313219 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c1620e0 con 0x7f0d4c15deb0 2024-04-25T10:11:59.312 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52400640 1 --2- 172.21.15.169:0/460313219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15deb0 0x7f0d4c0b2d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c08c3e0 tx=0x7f0d4401c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8486216371dde4be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.312 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d4be00640 1 -- 172.21.15.169:0/460313219 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440a57d0 con 0x7f0d4c15deb0 2024-04-25T10:11:59.312 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d4be00640 1 -- 172.21.15.169:0/460313219 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d440a5970 con 0x7f0d4c15deb0 2024-04-25T10:11:59.312 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d4be00640 1 -- 172.21.15.169:0/460313219 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440a5d40 con 0x7f0d4c15deb0 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 -- 172.21.15.169:0/460313219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15deb0 msgr2=0x7f0d4c0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 --2- 172.21.15.169:0/460313219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15deb0 0x7f0d4c0b2d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c08c3e0 tx=0x7f0d4401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 -- 172.21.15.169:0/460313219 shutdown_connections 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 --2- 172.21.15.169:0/460313219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15deb0 0x7f0d4c0b2d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 --2- 172.21.15.169:0/460313219 >> v2:172.21.15.87:3300/0 conn(0x7f0d4c0d0920 0x7f0d4c0b92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 -- 172.21.15.169:0/460313219 >> 172.21.15.169:0/460313219 conn(0x7f0d4c0669a0 msgr2=0x7f0d4c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 -- 172.21.15.169:0/460313219 shutdown_connections 2024-04-25T10:11:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.307+0000 7f0d52e00640 1 -- 172.21.15.169:0/460313219 wait complete. 2024-04-25T10:11:59.314 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 Processor -- start 2024-04-25T10:11:59.315 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 -- start start 2024-04-25T10:11:59.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d4c0d0920 0x7f0d4c14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15dad0 0x7f0d4c0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d4c15deb0 0x7f0d4c0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0d2680 con 0x7f0d4c0d0920 2024-04-25T10:11:59.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0d2500 con 0x7f0d4c15dad0 2024-04-25T10:11:59.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0d2800 con 0x7f0d4c15deb0 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d4c15deb0 0x7f0d4c0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d51000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15dad0 0x7f0d4c0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d4c15deb0 0x7f0d4c0cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54594/0 (socket says 172.21.15.169:54594) 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 -- 172.21.15.169:0/788334069 learned_addr learned my addr 172.21.15.169:0/788334069 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d51000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15dad0 0x7f0d4c0c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48426/0 (socket says 172.21.15.169:48426) 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d51a00640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d4c0d0920 0x7f0d4c14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 -- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15dad0 msgr2=0x7f0d4c0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15dad0 0x7f0d4c0c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 -- 172.21.15.169:0/788334069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d4c0d0920 msgr2=0x7f0d4c14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d4c0d0920 0x7f0d4c14db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 -- 172.21.15.169:0/788334069 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0c8aa0 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d51000640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15dad0 0x7f0d4c0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52400640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d4c15deb0 0x7f0d4c0cce40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d44019600 tx=0x7f0d44019790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d49600640 1 -- 172.21.15.169:0/788334069 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440bc020 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d49600640 1 -- 172.21.15.169:0/788334069 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d440b8880 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d49600640 1 -- 172.21.15.169:0/788334069 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44009510 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c0c8d80 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c0c9340 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d49600640 1 -- 172.21.15.169:0/788334069 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0d440bf020 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.311+0000 7f0d49600640 1 -- 172.21.15.169:0/788334069 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0d440c1db0 con 0x7f0d4c15deb0 2024-04-25T10:11:59.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.315+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c075120 con 0x7f0d4c15deb0 2024-04-25T10:11:59.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.315+0000 7f0d49600640 1 -- 172.21.15.169:0/788334069 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0d440c6030 con 0x7f0d4c15deb0 2024-04-25T10:11:59.386 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.383+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7bc3a5c5-e581-4d3f-827e-038a5b979a1a", "id": 2} v 0) v1 -- 0x7f0d4c0d1600 con 0x7f0d4c15deb0 2024-04-25T10:11:59.393 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.387+0000 7f0d49600640 1 -- 172.21.15.169:0/788334069 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7bc3a5c5-e581-4d3f-827e-038a5b979a1a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0d440165e0 con 0x7f0d4c15deb0 2024-04-25T10:11:59.394 INFO:teuthology.orchestra.run.smithi169.stdout:2 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d4c15deb0 msgr2=0x7f0d4c0cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d4c15deb0 0x7f0d4c0cce40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d44019600 tx=0x7f0d44019790 comp rx=0 tx=0).stop 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 shutdown_connections 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d4c15deb0 0x7f0d4c0cce40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d4c15dad0 0x7f0d4c0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 --2- 172.21.15.169:0/788334069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d4c0d0920 0x7f0d4c14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 >> 172.21.15.169:0/788334069 conn(0x7f0d4c0669a0 msgr2=0x7f0d4c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 shutdown_connections 2024-04-25T10:11:59.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.391+0000 7f0d52e00640 1 -- 172.21.15.169:0/788334069 wait complete. 2024-04-25T10:11:59.413 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new cff9fb22-0831-4513-94ef-d7839c7f5b97 3 2024-04-25T10:11:59.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7648400640 1 Processor -- start 2024-04-25T10:11:59.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7648400640 1 -- start start 2024-04-25T10:11:59.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7648400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764015dd30 0x7f764006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7648400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76400d1fb0 0x7f76400b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7648400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764006b1d0 con 0x7f76400cfb30 2024-04-25T10:11:59.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7648400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764015e110 con 0x7f76400d1fb0 2024-04-25T10:11:59.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7648400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764006b470 con 0x7f764015dd30 2024-04-25T10:11:59.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7645000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f76400cfb30 0x7f76400d18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:58122/0 (socket says 172.21.15.169:58122) 2024-04-25T10:11:59.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7645000640 1 -- 172.21.15.169:0/3372175030 learned_addr learned my addr 172.21.15.169:0/3372175030 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T10:11:59.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7645a00640 1 --2- 172.21.15.169:0/3372175030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764015dd30 0x7f764006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.543+0000 7f7646400640 1 --2- 172.21.15.169:0/3372175030 >> v2:172.21.15.87:3300/0 conn(0x7f76400d1fb0 0x7f76400b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f763fe00640 1 -- 172.21.15.169:0/3372175030 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086304860 0 0) 0x7f764006b1d0 con 0x7f76400cfb30 2024-04-25T10:11:59.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f763fe00640 1 -- 172.21.15.169:0/3372175030 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7628003530 con 0x7f76400cfb30 2024-04-25T10:11:59.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f7645a00640 1 -- 172.21.15.169:0/3372175030 >> v2:172.21.15.87:3300/0 conn(0x7f76400d1fb0 msgr2=0x7f76400b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f7645a00640 1 --2- 172.21.15.169:0/3372175030 >> v2:172.21.15.87:3300/0 conn(0x7f76400d1fb0 0x7f76400b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f7645a00640 1 -- 172.21.15.169:0/3372175030 >> v1:172.21.15.169:6789/0 conn(0x7f76400cfb30 legacy=0x7f76400d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f7645a00640 1 -- 172.21.15.169:0/3372175030 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764006b750 con 0x7f764015dd30 2024-04-25T10:11:59.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f7645a00640 1 --2- 172.21.15.169:0/3372175030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764015dd30 0x7f764006ac90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76340097c0 tx=0x7f763401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0b6125fa034299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f763fe00640 1 -- 172.21.15.169:0/3372175030 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340b2070 con 0x7f764015dd30 2024-04-25T10:11:59.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f763fe00640 1 -- 172.21.15.169:0/3372175030 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76340ad640 con 0x7f764015dd30 2024-04-25T10:11:59.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.547+0000 7f763fe00640 1 -- 172.21.15.169:0/3372175030 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340ada10 con 0x7f764015dd30 2024-04-25T10:11:59.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- 172.21.15.169:0/3372175030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764015dd30 msgr2=0x7f764006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 --2- 172.21.15.169:0/3372175030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764015dd30 0x7f764006ac90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76340097c0 tx=0x7f763401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- 172.21.15.169:0/3372175030 shutdown_connections 2024-04-25T10:11:59.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 --2- 172.21.15.169:0/3372175030 >> v2:172.21.15.87:3300/0 conn(0x7f76400d1fb0 0x7f76400b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 --2- 172.21.15.169:0/3372175030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764015dd30 0x7f764006ac90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- 172.21.15.169:0/3372175030 >> 172.21.15.169:0/3372175030 conn(0x7f764009a9d0 msgr2=0x7f764009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- 172.21.15.169:0/3372175030 shutdown_connections 2024-04-25T10:11:59.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- 172.21.15.169:0/3372175030 wait complete. 2024-04-25T10:11:59.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 Processor -- start 2024-04-25T10:11:59.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- start start 2024-04-25T10:11:59.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400cfb30 0x7f7640151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76400d1fb0 0x7f76400ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015dd30 0x7f7640159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764016ab60 con 0x7f76400d1fb0 2024-04-25T10:11:59.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76400b2890 con 0x7f764015dd30 2024-04-25T10:11:59.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7648400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764016ace0 con 0x7f76400cfb30 2024-04-25T10:11:59.558 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400cfb30 0x7f7640151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7646400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015dd30 0x7f7640159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400cfb30 0x7f7640151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54612/0 (socket says 172.21.15.169:54612) 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 -- 172.21.15.169:0/3762302415 learned_addr learned my addr 172.21.15.169:0/3762302415 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645000640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76400d1fb0 0x7f76400ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 -- 172.21.15.169:0/3762302415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015dd30 msgr2=0x7f7640159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015dd30 0x7f7640159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 -- 172.21.15.169:0/3762302415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76400d1fb0 msgr2=0x7f76400ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76400d1fb0 0x7f76400ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 -- 172.21.15.169:0/3762302415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764015ca50 con 0x7f76400cfb30 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.551+0000 7f7645a00640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400cfb30 0x7f7640151600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f763401a0d0 tx=0x7f7634019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f763d600640 1 -- 172.21.15.169:0/3762302415 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340b2040 con 0x7f76400cfb30 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f763d600640 1 -- 172.21.15.169:0/3762302415 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76340be070 con 0x7f76400cfb30 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f763d600640 1 -- 172.21.15.169:0/3762302415 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634006e20 con 0x7f76400cfb30 2024-04-25T10:11:59.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764015b9b0 con 0x7f76400cfb30 2024-04-25T10:11:59.560 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764015bf70 con 0x7f76400cfb30 2024-04-25T10:11:59.560 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f763d600640 1 -- 172.21.15.169:0/3762302415 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f76340b8680 con 0x7f76400cfb30 2024-04-25T10:11:59.560 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f763d600640 1 -- 172.21.15.169:0/3762302415 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f76340c3a80 con 0x7f76400cfb30 2024-04-25T10:11:59.560 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764004ef30 con 0x7f76400cfb30 2024-04-25T10:11:59.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.555+0000 7f763d600640 1 -- 172.21.15.169:0/3762302415 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f764015ca50 con 0x7f76400cfb30 2024-04-25T10:11:59.628 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.623+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cff9fb22-0831-4513-94ef-d7839c7f5b97", "id": 3} v 0) v1 -- 0x7f764006aea0 con 0x7f76400cfb30 2024-04-25T10:11:59.633 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.627+0000 7f763d600640 1 -- 172.21.15.169:0/3762302415 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cff9fb22-0831-4513-94ef-d7839c7f5b97", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7634016090 con 0x7f76400cfb30 2024-04-25T10:11:59.633 INFO:teuthology.orchestra.run.smithi169.stdout:3 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400cfb30 msgr2=0x7f7640151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400cfb30 0x7f7640151600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f763401a0d0 tx=0x7f7634019240 comp rx=0 tx=0).stop 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 shutdown_connections 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015dd30 0x7f7640159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76400d1fb0 0x7f76400ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 --2- 172.21.15.169:0/3762302415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400cfb30 0x7f7640151600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 >> 172.21.15.169:0/3762302415 conn(0x7f764009a9d0 msgr2=0x7f7640156940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 shutdown_connections 2024-04-25T10:11:59.635 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.631+0000 7f7648400640 1 -- 172.21.15.169:0/3762302415 wait complete. 2024-04-25T10:11:59.653 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new ccc3700f-0f90-438a-b39a-86e71ffa860b 4 2024-04-25T10:11:59.669 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:59.662+0000 7f60835ae180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:11:59.731 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:59.726+0000 7f60835ae180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:11:59.736 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:11:59.731+0000 7efe9730b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:11:59.787 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd455a00640 1 Processor -- start 2024-04-25T10:11:59.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd455a00640 1 -- start start 2024-04-25T10:11:59.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd455a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd45015dd30 0x7fd45006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd455a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45006b1d0 con 0x7fd4500d1fb0 2024-04-25T10:11:59.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd455a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45006b470 con 0x7fd4500cfb30 2024-04-25T10:11:59.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd455a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45015e110 con 0x7fd45015dd30 2024-04-25T10:11:59.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd45015dd30 0x7fd45006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd45015dd30 0x7fd45006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54624/0 (socket says 172.21.15.169:54624) 2024-04-25T10:11:59.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd44ea00640 1 -- 172.21.15.169:0/4250104374 learned_addr learned my addr 172.21.15.169:0/4250104374 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:59.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd44ea00640 1 -- 172.21.15.169:0/4250104374 >> v1:172.21.15.169:6789/0 conn(0x7fd4500cfb30 legacy=0x7fd4500d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd44ea00640 1 -- 172.21.15.169:0/4250104374 >> v1:172.21.15.87:6789/0 conn(0x7fd4500d1fb0 legacy=0x7fd4500b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd44ea00640 1 -- 172.21.15.169:0/4250104374 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45006b750 con 0x7fd45015dd30 2024-04-25T10:11:59.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.783+0000 7fd44e000640 1 -- 172.21.15.169:0/4250104374 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477111763 0 0) 0x7fd45006b470 con 0x7fd4500cfb30 2024-04-25T10:11:59.792 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:59.786+0000 7f60835ae180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:11:59.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd44ea00640 1 --2- 172.21.15.169:0/4250104374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd45015dd30 0x7fd45006ac90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd4400098a0 tx=0x7fd44001c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab34bd097a2580e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd44e000640 1 -- 172.21.15.169:0/4250104374 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400a7070 con 0x7fd45015dd30 2024-04-25T10:11:59.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd44e000640 1 -- 172.21.15.169:0/4250104374 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400b0070 con 0x7fd45015dd30 2024-04-25T10:11:59.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd44e000640 1 -- 172.21.15.169:0/4250104374 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400ab890 con 0x7fd45015dd30 2024-04-25T10:11:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd455a00640 1 -- 172.21.15.169:0/4250104374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd45015dd30 msgr2=0x7fd45006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd455a00640 1 --2- 172.21.15.169:0/4250104374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd45015dd30 0x7fd45006ac90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd4400098a0 tx=0x7fd44001c580 comp rx=0 tx=0).stop 2024-04-25T10:11:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd455a00640 1 -- 172.21.15.169:0/4250104374 shutdown_connections 2024-04-25T10:11:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd455a00640 1 --2- 172.21.15.169:0/4250104374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd45015dd30 0x7fd45006ac90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd455a00640 1 -- 172.21.15.169:0/4250104374 >> 172.21.15.169:0/4250104374 conn(0x7fd45009a9d0 msgr2=0x7fd45009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd455a00640 1 -- 172.21.15.169:0/4250104374 shutdown_connections 2024-04-25T10:11:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.787+0000 7fd455a00640 1 -- 172.21.15.169:0/4250104374 wait complete. 2024-04-25T10:11:59.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 Processor -- start 2024-04-25T10:11:59.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 -- start start 2024-04-25T10:11:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4500cfb30 0x7fd450154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4500d1fb0 0x7fd4500c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45015dd30 0x7fd45006f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45016bf00 con 0x7fd45015dd30 2024-04-25T10:11:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45016bd80 con 0x7fd4500d1fb0 2024-04-25T10:11:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45016c080 con 0x7fd4500cfb30 2024-04-25T10:11:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4500cfb30 0x7fd450154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4500cfb30 0x7fd450154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54632/0 (socket says 172.21.15.169:54632) 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 -- 172.21.15.169:0/3461687879 learned_addr learned my addr 172.21.15.169:0/3461687879 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd447e00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4500d1fb0 0x7fd4500c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44f400640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45015dd30 0x7fd45006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 -- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4500d1fb0 msgr2=0x7fd4500c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4500d1fb0 0x7fd4500c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 -- 172.21.15.169:0/3461687879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45015dd30 msgr2=0x7fd45006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45015dd30 0x7fd45006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 -- 172.21.15.169:0/3461687879 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500b7750 con 0x7fd4500cfb30 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44f400640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45015dd30 0x7fd45006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd44ea00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4500cfb30 0x7fd450154f60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4400098a0 tx=0x7fd440019320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd446a00640 1 -- 172.21.15.169:0/3461687879 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440004220 con 0x7fd4500cfb30 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd446a00640 1 -- 172.21.15.169:0/3461687879 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400043c0 con 0x7fd4500cfb30 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4501554a0 con 0x7fd4500cfb30 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd446a00640 1 -- 172.21.15.169:0/3461687879 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440003820 con 0x7fd4500cfb30 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4500c59b0 con 0x7fd4500cfb30 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd446a00640 1 -- 172.21.15.169:0/3461687879 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd4400bf420 con 0x7fd4500cfb30 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.791+0000 7fd446a00640 1 -- 172.21.15.169:0/3461687879 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fd440003df0 con 0x7fd4500cfb30 2024-04-25T10:11:59.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.795+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45004ef30 con 0x7fd4500cfb30 2024-04-25T10:11:59.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.795+0000 7fd446a00640 1 -- 172.21.15.169:0/3461687879 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd440016170 con 0x7fd4500cfb30 2024-04-25T10:11:59.802 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:11:59.799+0000 7efe9730b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:11:59.863 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:11:59.859+0000 7efe9730b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:11:59.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.863+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ccc3700f-0f90-438a-b39a-86e71ffa860b", "id": 4} v 0) v1 -- 0x7fd4500b7920 con 0x7fd4500cfb30 2024-04-25T10:11:59.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.867+0000 7fd446a00640 1 -- 172.21.15.169:0/3461687879 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ccc3700f-0f90-438a-b39a-86e71ffa860b", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd4400b4400 con 0x7fd4500cfb30 2024-04-25T10:11:59.873 INFO:teuthology.orchestra.run.smithi169.stdout:4 2024-04-25T10:11:59.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4500cfb30 msgr2=0x7fd450154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4500cfb30 0x7fd450154f60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4400098a0 tx=0x7fd440019320 comp rx=0 tx=0).stop 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 shutdown_connections 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45015dd30 0x7fd45006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4500d1fb0 0x7fd4500c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 --2- 172.21.15.169:0/3461687879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4500cfb30 0x7fd450154f60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 >> 172.21.15.169:0/3461687879 conn(0x7fd45009a9d0 msgr2=0x7fd450155700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 shutdown_connections 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:11:59.871+0000 7fd455a00640 1 -- 172.21.15.169:0/3461687879 wait complete. 2024-04-25T10:11:59.882 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:59.874+0000 7f60835ae180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:11:59.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4 5 2024-04-25T10:11:59.942 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:11:59.934+0000 7f60835ae180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:11:59.957 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:11:59.951+0000 7efe9730b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:12:00.019 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:00.015+0000 7efe9730b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:12:00.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57c6600640 1 Processor -- start 2024-04-25T10:12:00.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57c6600640 1 -- start start 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57c6600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57c015bc40 0x7f57c00cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57c6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f57c0071a80 0x7f57c0071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57c6600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00ccba0 con 0x7f57c015c020 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57c6600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c00cce80 con 0x7f57c015bc40 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57c6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c00cd160 con 0x7f57c0071a80 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f57c0071a80 0x7f57c0071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f57c0071a80 0x7f57c0071eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48464/0 (socket says 172.21.15.169:48464) 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bfe00640 1 -- 172.21.15.169:0/3849939465 learned_addr learned my addr 172.21.15.169:0/3849939465 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bf400640 1 --2- 172.21.15.169:0/3849939465 >> v2:172.21.15.87:3300/0 conn(0x7f57c015bc40 0x7f57c00cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bfe00640 1 -- 172.21.15.169:0/3849939465 >> v2:172.21.15.87:3300/0 conn(0x7f57c015bc40 msgr2=0x7f57c00cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.029 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bfe00640 1 --2- 172.21.15.169:0/3849939465 >> v2:172.21.15.87:3300/0 conn(0x7f57c015bc40 0x7f57c00cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.030 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bfe00640 1 -- 172.21.15.169:0/3849939465 >> v1:172.21.15.87:6789/0 conn(0x7f57c015c020 legacy=0x7f57c006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.030 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.023+0000 7f57bfe00640 1 -- 172.21.15.169:0/3849939465 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00cd440 con 0x7f57c0071a80 2024-04-25T10:12:00.030 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:00.022+0000 7f60835ae180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:12:00.032 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57bfe00640 1 --2- 172.21.15.169:0/3849939465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c0071a80 0x7f57c0071eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f57c00b2cc0 tx=0x7f57b801c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810e1f73882f81cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.032 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57bea00640 1 -- 172.21.15.169:0/3849939465 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80a5860 con 0x7f57c0071a80 2024-04-25T10:12:00.032 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57bea00640 1 -- 172.21.15.169:0/3849939465 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b80a5a00 con 0x7f57c0071a80 2024-04-25T10:12:00.032 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57bea00640 1 -- 172.21.15.169:0/3849939465 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80a5dd0 con 0x7f57c0071a80 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 -- 172.21.15.169:0/3849939465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c0071a80 msgr2=0x7f57c0071eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 --2- 172.21.15.169:0/3849939465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c0071a80 0x7f57c0071eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f57c00b2cc0 tx=0x7f57b801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 -- 172.21.15.169:0/3849939465 shutdown_connections 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 --2- 172.21.15.169:0/3849939465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c0071a80 0x7f57c0071eb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 --2- 172.21.15.169:0/3849939465 >> v2:172.21.15.87:3300/0 conn(0x7f57c015bc40 0x7f57c00cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 -- 172.21.15.169:0/3849939465 >> 172.21.15.169:0/3849939465 conn(0x7f57c00b1ca0 msgr2=0x7f57c00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 -- 172.21.15.169:0/3849939465 shutdown_connections 2024-04-25T10:12:00.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.027+0000 7f57c6600640 1 -- 172.21.15.169:0/3849939465 wait complete. 2024-04-25T10:12:00.034 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 Processor -- start 2024-04-25T10:12:00.034 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 -- start start 2024-04-25T10:12:00.036 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57c0071a80 0x7f57c00b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.036 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c015bc40 0x7f57c00b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.036 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c015c020 0x7f57c0142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.036 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00ce920 con 0x7f57c0071a80 2024-04-25T10:12:00.036 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00ce7a0 con 0x7f57c015bc40 2024-04-25T10:12:00.036 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00ceaa0 con 0x7f57c015c020 2024-04-25T10:12:00.036 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c015c020 0x7f57c0142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c015bc40 0x7f57c00b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bf400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57c0071a80 0x7f57c00b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c015c020 0x7f57c0142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:54644/0 (socket says 172.21.15.169:54644) 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 -- 172.21.15.169:0/749615495 learned_addr learned my addr 172.21.15.169:0/749615495 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 -- 172.21.15.169:0/749615495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c015bc40 msgr2=0x7f57c00b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 --2- 172.21.15.169:0/749615495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c015bc40 0x7f57c00b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 -- 172.21.15.169:0/749615495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57c0071a80 msgr2=0x7f57c00b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 --2- 172.21.15.169:0/749615495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57c0071a80 0x7f57c00b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 -- 172.21.15.169:0/749615495 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c014b3e0 con 0x7f57c015c020 2024-04-25T10:12:00.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57bfe00640 1 --2- 172.21.15.169:0/749615495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c015c020 0x7f57c0142bd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57b80195d0 tx=0x7f57b8019760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.038 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57b7400640 1 -- 172.21.15.169:0/749615495 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80b8670 con 0x7f57c015c020 2024-04-25T10:12:00.038 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57b7400640 1 -- 172.21.15.169:0/749615495 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b80be070 con 0x7f57c015c020 2024-04-25T10:12:00.038 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57b7400640 1 -- 172.21.15.169:0/749615495 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80b8a70 con 0x7f57c015c020 2024-04-25T10:12:00.038 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c014b6c0 con 0x7f57c015c020 2024-04-25T10:12:00.039 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c0143110 con 0x7f57c015c020 2024-04-25T10:12:00.039 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57b7400640 1 -- 172.21.15.169:0/749615495 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f57b80b8c10 con 0x7f57c015c020 2024-04-25T10:12:00.039 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.031+0000 7f57b7400640 1 -- 172.21.15.169:0/749615495 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f57b80c3d20 con 0x7f57c015c020 2024-04-25T10:12:00.039 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.035+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c004ef30 con 0x7f57c015c020 2024-04-25T10:12:00.040 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.035+0000 7f57b7400640 1 -- 172.21.15.169:0/749615495 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f57b80165e0 con 0x7f57c015c020 2024-04-25T10:12:00.091 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:00.086+0000 7f60835ae180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:12:00.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.103+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4", "id": 5} v 0) v1 -- 0x7f57c0068610 con 0x7f57c015c020 2024-04-25T10:12:00.112 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:00.107+0000 7efe9730b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:12:00.113 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.107+0000 7f57b7400640 1 -- 172.21.15.169:0/749615495 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f57b80c3460 con 0x7f57c015c020 2024-04-25T10:12:00.113 INFO:teuthology.orchestra.run.smithi169.stdout:5 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c015c020 msgr2=0x7f57c0142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 --2- 172.21.15.169:0/749615495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c015c020 0x7f57c0142bd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57b80195d0 tx=0x7f57b8019760 comp rx=0 tx=0).stop 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 shutdown_connections 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 --2- 172.21.15.169:0/749615495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c015c020 0x7f57c0142bd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 --2- 172.21.15.169:0/749615495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57c015bc40 0x7f57c00b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 --2- 172.21.15.169:0/749615495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57c0071a80 0x7f57c00b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 >> 172.21.15.169:0/749615495 conn(0x7f57c00b1ca0 msgr2=0x7f57c014e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 shutdown_connections 2024-04-25T10:12:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.111+0000 7f57c6600640 1 -- 172.21.15.169:0/749615495 wait complete. 2024-04-25T10:12:00.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 4a593c61-a6b9-47a5-a008-fc0837c29d27 6 2024-04-25T10:12:00.179 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:00.175+0000 7efe9730b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:12:00.260 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:00.254+0000 7f60835ae180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:12:00.267 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a3a00640 1 Processor -- start 2024-04-25T10:12:00.267 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a3a00640 1 -- start start 2024-04-25T10:12:00.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a3a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c0ce8a0 0x7ff49c0d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a3a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c0d2630 con 0x7ff49c06ac90 2024-04-25T10:12:00.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a3a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c0d0d60 con 0x7ff49c06ba80 2024-04-25T10:12:00.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a3a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0b1620 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c0ce8a0 0x7ff49c0d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a1a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff49c06ba80 0x7ff49c15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:58126/0 (socket says 172.21.15.169:58126) 2024-04-25T10:12:00.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a1a00640 1 -- 172.21.15.169:0/2605978988 learned_addr learned my addr 172.21.15.169:0/2605978988 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T10:12:00.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a1000640 1 -- 172.21.15.169:0/2605978988 >> v1:172.21.15.169:6789/0 conn(0x7ff49c06ba80 legacy=0x7ff49c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a1000640 1 -- 172.21.15.169:0/2605978988 >> v1:172.21.15.87:6789/0 conn(0x7ff49c06ac90 legacy=0x7ff49c06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.262+0000 7ff4a1000640 1 -- 172.21.15.169:0/2605978988 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0b0fd0 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.272 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a1000640 1 --2- 172.21.15.169:0/2605978988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c0ce8a0 0x7ff49c0d06d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0b17a0 tx=0x7ff49001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62705b87f35278e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.272 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff49b400640 1 -- 172.21.15.169:0/2605978988 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900b1070 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.272 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff49b400640 1 -- 172.21.15.169:0/2605978988 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4900ac630 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.272 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff49b400640 1 -- 172.21.15.169:0/2605978988 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900a8050 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2605978988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c0ce8a0 msgr2=0x7ff49c0d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a3a00640 1 --2- 172.21.15.169:0/2605978988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c0ce8a0 0x7ff49c0d06d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0b17a0 tx=0x7ff49001c4c0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2605978988 shutdown_connections 2024-04-25T10:12:00.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a3a00640 1 --2- 172.21.15.169:0/2605978988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c0ce8a0 0x7ff49c0d06d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2605978988 >> 172.21.15.169:0/2605978988 conn(0x7ff49c0d19c0 msgr2=0x7ff49c0d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2605978988 shutdown_connections 2024-04-25T10:12:00.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.266+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2605978988 wait complete. 2024-04-25T10:12:00.274 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 Processor -- start 2024-04-25T10:12:00.275 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 -- start start 2024-04-25T10:12:00.277 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff49c06ac90 0x7ff49c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.277 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c06ba80 0x7ff49c14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.277 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c0ce8a0 0x7ff49c154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.277 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c16c630 con 0x7ff49c06ac90 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c16c4b0 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c16c7b0 con 0x7ff49c06ba80 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c0ce8a0 0x7ff49c154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff49be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c06ba80 0x7ff49c14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c0ce8a0 0x7ff49c154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48482/0 (socket says 172.21.15.169:48482) 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 -- 172.21.15.169:0/2754689197 learned_addr learned my addr 172.21.15.169:0/2754689197 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1000640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff49c06ac90 0x7ff49c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 -- 172.21.15.169:0/2754689197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c06ba80 msgr2=0x7ff49c14ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c06ba80 0x7ff49c14ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 -- 172.21.15.169:0/2754689197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff49c06ac90 msgr2=0x7ff49c14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff49c06ac90 0x7ff49c14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 -- 172.21.15.169:0/2754689197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c1539a0 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a1a00640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c0ce8a0 0x7ff49c154a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff49400cbc0 tx=0x7ff49400a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff498c00640 1 -- 172.21.15.169:0/2754689197 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494017070 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff498c00640 1 -- 172.21.15.169:0/2754689197 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400ad40 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff498c00640 1 -- 172.21.15.169:0/2754689197 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494012600 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c153c80 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c154240 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff498c00640 1 -- 172.21.15.169:0/2754689197 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff49401a070 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.270+0000 7ff498c00640 1 -- 172.21.15.169:0/2754689197 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff49401c440 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.279 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.274+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c04ef30 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.280 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.274+0000 7ff498c00640 1 -- 172.21.15.169:0/2754689197 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff4940127a0 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.331 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:00.326+0000 7f60835ae180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:12:00.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.342+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4a593c61-a6b9-47a5-a008-fc0837c29d27", "id": 6} v 0) v1 -- 0x7ff49c0b35c0 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.349 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:00.342+0000 7efe9730b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:12:00.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff498c00640 1 -- 172.21.15.169:0/2754689197 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a593c61-a6b9-47a5-a008-fc0837c29d27", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff494012980 con 0x7ff49c0ce8a0 2024-04-25T10:12:00.353 INFO:teuthology.orchestra.run.smithi169.stdout:6 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c0ce8a0 msgr2=0x7ff49c154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c0ce8a0 0x7ff49c154a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff49400cbc0 tx=0x7ff49400a430 comp rx=0 tx=0).stop 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 shutdown_connections 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff49c0ce8a0 0x7ff49c154a40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c06ba80 0x7ff49c14ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 --2- 172.21.15.169:0/2754689197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff49c06ac90 0x7ff49c14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 >> 172.21.15.169:0/2754689197 conn(0x7ff49c0d19c0 msgr2=0x7ff49c156150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.346+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 shutdown_connections 2024-04-25T10:12:00.354 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.350+0000 7ff4a3a00640 1 -- 172.21.15.169:0/2754689197 wait complete. 2024-04-25T10:12:00.372 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new d3a5c2d1-069c-4959-bcd3-75418023a734 7 2024-04-25T10:12:00.423 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:00.418+0000 7efe9730b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:12:00.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.542+0000 7fbc28e00640 1 Processor -- start 2024-04-25T10:12:00.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.542+0000 7fbc28e00640 1 -- start start 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.542+0000 7fbc28e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbc24159540 0x7fbc240ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.542+0000 7fbc28e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc240aebd0 0x7fbc24071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.542+0000 7fbc28e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc24071360 con 0x7fbc24072020 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.542+0000 7fbc28e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc24159920 con 0x7fbc24159540 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.542+0000 7fbc28e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2415c2f0 con 0x7fbc240aebd0 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc22800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbc24072020 0x7fbc2415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:58130/0 (socket says 172.21.15.169:58130) 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc22800640 1 -- 172.21.15.169:0/2174333288 learned_addr learned my addr 172.21.15.169:0/2174333288 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T10:12:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc1be00640 1 --2- 172.21.15.169:0/2174333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc240aebd0 0x7fbc24071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc21e00640 1 --2- 172.21.15.169:0/2174333288 >> v2:172.21.15.87:3300/0 conn(0x7fbc24159540 0x7fbc240ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc21400640 1 -- 172.21.15.169:0/2174333288 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237172497 0 0) 0x7fbc24071360 con 0x7fbc24072020 2024-04-25T10:12:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc1be00640 1 -- 172.21.15.169:0/2174333288 >> v2:172.21.15.87:3300/0 conn(0x7fbc24159540 msgr2=0x7fbc240ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc1be00640 1 --2- 172.21.15.169:0/2174333288 >> v2:172.21.15.87:3300/0 conn(0x7fbc24159540 0x7fbc240ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc1be00640 1 -- 172.21.15.169:0/2174333288 >> v1:172.21.15.169:6789/0 conn(0x7fbc24072020 legacy=0x7fbc2415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.552 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.546+0000 7fbc1be00640 1 -- 172.21.15.169:0/2174333288 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2415c5d0 con 0x7fbc240aebd0 2024-04-25T10:12:00.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc1be00640 1 --2- 172.21.15.169:0/2174333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc240aebd0 0x7fbc24071ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0097c0 tx=0x7fbc0c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c18a256ba6e975 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc21400640 1 -- 172.21.15.169:0/2174333288 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0b0070 con 0x7fbc240aebd0 2024-04-25T10:12:00.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc21400640 1 -- 172.21.15.169:0/2174333288 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c0ab600 con 0x7fbc240aebd0 2024-04-25T10:12:00.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc21400640 1 -- 172.21.15.169:0/2174333288 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0ab9d0 con 0x7fbc240aebd0 2024-04-25T10:12:00.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 -- 172.21.15.169:0/2174333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc240aebd0 msgr2=0x7fbc24071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 --2- 172.21.15.169:0/2174333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc240aebd0 0x7fbc24071ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0097c0 tx=0x7fbc0c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 -- 172.21.15.169:0/2174333288 shutdown_connections 2024-04-25T10:12:00.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 --2- 172.21.15.169:0/2174333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc240aebd0 0x7fbc24071ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 --2- 172.21.15.169:0/2174333288 >> v2:172.21.15.87:3300/0 conn(0x7fbc24159540 0x7fbc240ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 -- 172.21.15.169:0/2174333288 >> 172.21.15.169:0/2174333288 conn(0x7fbc240669a0 msgr2=0x7fbc24066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 -- 172.21.15.169:0/2174333288 shutdown_connections 2024-04-25T10:12:00.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.550+0000 7fbc28e00640 1 -- 172.21.15.169:0/2174333288 wait complete. 2024-04-25T10:12:00.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 Processor -- start 2024-04-25T10:12:00.559 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 -- start start 2024-04-25T10:12:00.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc24072020 0x7fbc241573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc240aebd0 0x7fbc24148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.561 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc24159540 0x7fbc2414d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc24072430 con 0x7fbc24072020 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2406c0d0 con 0x7fbc240aebd0 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc28e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc24168930 con 0x7fbc24159540 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc240aebd0 0x7fbc24148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc22800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc24159540 0x7fbc2414d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc24072020 0x7fbc241573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc24072020 0x7fbc241573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.169:56548/0 (socket says 172.21.15.169:56548) 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc21e00640 1 -- 172.21.15.169:0/654007867 learned_addr learned my addr 172.21.15.169:0/654007867 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc1be00640 1 -- 172.21.15.169:0/654007867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc24159540 msgr2=0x7fbc2414d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc1be00640 1 --2- 172.21.15.169:0/654007867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc24159540 0x7fbc2414d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc1be00640 1 -- 172.21.15.169:0/654007867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc24072020 msgr2=0x7fbc241573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc1be00640 1 --2- 172.21.15.169:0/654007867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc24072020 0x7fbc241573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.562 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.554+0000 7fbc1be00640 1 -- 172.21.15.169:0/654007867 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24158d70 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc1be00640 1 --2- 172.21.15.169:0/654007867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc240aebd0 0x7fbc24148f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0ab000 tx=0x7fbc0c019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc1a000640 1 -- 172.21.15.169:0/654007867 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0b0040 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc1a000640 1 -- 172.21.15.169:0/654007867 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c0ba070 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc241587b0 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc1a000640 1 -- 172.21.15.169:0/654007867 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0b4850 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc24156be0 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc1a000640 1 -- 172.21.15.169:0/654007867 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbc0c0ab4c0 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc1a000640 1 -- 172.21.15.169:0/654007867 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fbc0c0bea20 con 0x7fbc240aebd0 2024-04-25T10:12:00.564 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2404ef30 con 0x7fbc240aebd0 2024-04-25T10:12:00.566 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.558+0000 7fbc1a000640 1 -- 172.21.15.169:0/654007867 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbc0c016090 con 0x7fbc240aebd0 2024-04-25T10:12:00.632 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.626+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d3a5c2d1-069c-4959-bcd3-75418023a734", "id": 7} v 0) v1 -- 0x7fbc2406d7f0 con 0x7fbc240aebd0 2024-04-25T10:12:00.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc1a000640 1 -- 172.21.15.169:0/654007867 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d3a5c2d1-069c-4959-bcd3-75418023a734", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbc0c0bec40 con 0x7fbc240aebd0 2024-04-25T10:12:00.636 INFO:teuthology.orchestra.run.smithi169.stdout:7 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc240aebd0 msgr2=0x7fbc24148f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 --2- 172.21.15.169:0/654007867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc240aebd0 0x7fbc24148f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0ab000 tx=0x7fbc0c019240 comp rx=0 tx=0).stop 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 shutdown_connections 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 --2- 172.21.15.169:0/654007867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc24159540 0x7fbc2414d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 --2- 172.21.15.169:0/654007867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc240aebd0 0x7fbc24148f80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 --2- 172.21.15.169:0/654007867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc24072020 0x7fbc241573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 >> 172.21.15.169:0/654007867 conn(0x7fbc240669a0 msgr2=0x7fbc240cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 shutdown_connections 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T10:12:00.630+0000 7fbc28e00640 1 -- 172.21.15.169:0/654007867 wait complete. 2024-04-25T10:12:00.655 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:12:00.655 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:12:00.659 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:12:00.660 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:12:00.660 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:12:00.670 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:12:00.670 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:12:00.670 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:12:00.680 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:12:00.680 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:12:00.680 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:12:00.689 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:12:00.689 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:12:00.689 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:12:00.705 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:12:00.705 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:12:00.705 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:12:00.709 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:12:00.709 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:12:00.710 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:12:00.713 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:12:00.713 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:12:00.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:12:00.716 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:12:00.717 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:00.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.898+0000 7f0be0800640 1 Processor -- start 2024-04-25T10:12:00.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.898+0000 7f0be0800640 1 -- start start 2024-04-25T10:12:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0be0800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0bd815b910 0x7f0bd815c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0be0800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd815bd60 con 0x7f0bd80d0040 2024-04-25T10:12:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0be0800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd8071050 con 0x7f0bd81550b0 2024-04-25T10:12:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0be0800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd80712f0 con 0x7f0bd815b910 2024-04-25T10:12:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bdd400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0bd80d0040 0x7f0bd815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53168/0 (socket says 172.21.15.87:53168) 2024-04-25T10:12:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bdd400640 1 -- 172.21.15.87:0/541449128 learned_addr learned my addr 172.21.15.87:0/541449128 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bde800640 1 --2- 172.21.15.87:0/541449128 >> v2:172.21.15.169:3300/0 conn(0x7f0bd815b910 0x7f0bd815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bdca00640 1 -- 172.21.15.87:0/541449128 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372542669 0 0) 0x7f0bd815bd60 con 0x7f0bd80d0040 2024-04-25T10:12:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bdca00640 1 -- 172.21.15.87:0/541449128 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc0003530 con 0x7f0bd80d0040 2024-04-25T10:12:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bdca00640 1 -- 172.21.15.87:0/541449128 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475512795 0 0) 0x7f0bd8071050 con 0x7f0bd81550b0 2024-04-25T10:12:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bdca00640 1 -- 172.21.15.87:0/541449128 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd815bd60 con 0x7f0bd81550b0 2024-04-25T10:12:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bde800640 1 -- 172.21.15.87:0/541449128 >> v1:172.21.15.169:6789/0 conn(0x7f0bd81550b0 legacy=0x7f0bd80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bde800640 1 -- 172.21.15.87:0/541449128 >> v1:172.21.15.87:6789/0 conn(0x7f0bd80d0040 legacy=0x7f0bd815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bde800640 1 -- 172.21.15.87:0/541449128 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd80715d0 con 0x7f0bd815b910 2024-04-25T10:12:00.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.902+0000 7f0bde800640 1 --2- 172.21.15.87:0/541449128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd815b910 0x7f0bd815c030 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009d90 tx=0x7f0bd401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ab4618e8626cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bdca00640 1 -- 172.21.15.87:0/541449128 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40b2070 con 0x7f0bd815b910 2024-04-25T10:12:00.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bdca00640 1 -- 172.21.15.87:0/541449128 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd40a5a00 con 0x7f0bd815b910 2024-04-25T10:12:00.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bdca00640 1 -- 172.21.15.87:0/541449128 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40a5dd0 con 0x7f0bd815b910 2024-04-25T10:12:00.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- 172.21.15.87:0/541449128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd815b910 msgr2=0x7f0bd815c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 --2- 172.21.15.87:0/541449128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd815b910 0x7f0bd815c030 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009d90 tx=0x7f0bd401c940 comp rx=0 tx=0).stop 2024-04-25T10:12:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- 172.21.15.87:0/541449128 shutdown_connections 2024-04-25T10:12:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 --2- 172.21.15.87:0/541449128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd815b910 0x7f0bd815c030 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- 172.21.15.87:0/541449128 >> 172.21.15.87:0/541449128 conn(0x7f0bd8066b70 msgr2=0x7f0bd8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- 172.21.15.87:0/541449128 shutdown_connections 2024-04-25T10:12:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- 172.21.15.87:0/541449128 wait complete. 2024-04-25T10:12:00.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 Processor -- start 2024-04-25T10:12:00.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- start start 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd80d0040 0x7f0bd8147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bd81550b0 0x7f0bd80c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd815b910 0x7f0bd806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd815cf00 con 0x7f0bd81550b0 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd815cd80 con 0x7f0bd80d0040 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0be0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd815d080 con 0x7f0bd815b910 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bdd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bd81550b0 0x7f0bd80c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bdd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bd81550b0 0x7f0bd80c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58018/0 (socket says 172.21.15.87:58018) 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bdde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd80d0040 0x7f0bd8147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bdd400640 1 -- 172.21.15.87:0/987739009 learned_addr learned my addr 172.21.15.87:0/987739009 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.906+0000 7f0bde800640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd815b910 0x7f0bd806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bdd400640 1 -- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd815b910 msgr2=0x7f0bd806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bdd400640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd815b910 0x7f0bd806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bdd400640 1 -- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd80d0040 msgr2=0x7f0bd8147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bdd400640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd80d0040 0x7f0bd8147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bdd400640 1 -- 172.21.15.87:0/987739009 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd815b6c0 con 0x7f0bd81550b0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bdde00640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd80d0040 0x7f0bd8147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bdd400640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bd81550b0 0x7f0bd80c4f60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c420 tx=0x7f0bc800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bd2000640 1 -- 172.21.15.87:0/987739009 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8017070 con 0x7f0bd81550b0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bd2000640 1 -- 172.21.15.87:0/987739009 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc8005cf0 con 0x7f0bd81550b0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd81f7e60 con 0x7f0bd81550b0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bd2000640 1 -- 172.21.15.87:0/987739009 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80072d0 con 0x7f0bd81550b0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd81f8420 con 0x7f0bd81550b0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bd2000640 1 -- 172.21.15.87:0/987739009 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0bc801a070 con 0x7f0bd81550b0 2024-04-25T10:12:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0bd2000640 1 -- 172.21.15.87:0/987739009 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0bc801cc60 con 0x7f0bd81550b0 2024-04-25T10:12:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.910+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd804ef30 con 0x7f0bd81550b0 2024-04-25T10:12:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.914+0000 7f0bd2000640 1 -- 172.21.15.87:0/987739009 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0bc8007470 con 0x7f0bd81550b0 2024-04-25T10:12:00.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd80afbf0 con 0x7f0bd81550b0 2024-04-25T10:12:00.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0bd2000640 1 -- 172.21.15.87:0/987739009 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f0bc8007650 con 0x7f0bd81550b0 2024-04-25T10:12:00.976 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:00.976 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":10,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:00.631609+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:00.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bd81550b0 msgr2=0x7f0bd80c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bd81550b0 0x7f0bd80c4f60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c420 tx=0x7f0bc800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 shutdown_connections 2024-04-25T10:12:00.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd815b910 0x7f0bd806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bd81550b0 0x7f0bd80c4f60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 --2- 172.21.15.87:0/987739009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd80d0040 0x7f0bd8147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 >> 172.21.15.87:0/987739009 conn(0x7f0bd8066b70 msgr2=0x7f0bd80b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 shutdown_connections 2024-04-25T10:12:00.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:00.970+0000 7f0be0800640 1 -- 172.21.15.87:0/987739009 wait complete. 2024-04-25T10:12:00.995 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:12:00.995 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:12:01.031 INFO:tasks.ceph.mgr.y.smithi087.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:12:01.031 INFO:tasks.ceph.mgr.y.smithi087.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:12:01.031 INFO:tasks.ceph.mgr.y.smithi087.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:12:01.082 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:01.074+0000 7f60835ae180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:12:01.154 INFO:tasks.ceph.mgr.x.smithi169.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:12:01.155 INFO:tasks.ceph.mgr.x.smithi169.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:12:01.155 INFO:tasks.ceph.mgr.x.smithi169.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:12:01.217 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:01.210+0000 7efe9730b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:12:01.230 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:01.222+0000 7f60835ae180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:12:01.362 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:01.358+0000 7efe9730b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:12:01.367 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T10:12:01.362+0000 7fe3354607c0 -1 Falling back to public interface 2024-04-25T10:12:01.379 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T10:12:01.374+0000 7f28f22807c0 -1 Falling back to public interface 2024-04-25T10:12:01.379 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T10:12:01.374+0000 7f40298d97c0 -1 Falling back to public interface 2024-04-25T10:12:01.384 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-04-25T10:12:01.378+0000 7f724db6e7c0 -1 Falling back to public interface 2024-04-25T10:12:01.384 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-04-25T10:12:01.378+0000 7f9f4ec127c0 -1 Falling back to public interface 2024-04-25T10:12:01.388 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-04-25T10:12:01.382+0000 7f70b825e7c0 -1 Falling back to public interface 2024-04-25T10:12:01.391 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T10:12:01.386+0000 7f077f7c47c0 -1 Falling back to public interface 2024-04-25T10:12:01.400 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-04-25T10:12:01.394+0000 7f77f82b27c0 -1 Falling back to public interface 2024-04-25T10:12:01.408 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:01.402+0000 7f60835ae180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:12:01.483 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:01.474+0000 7f60835ae180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:12:01.560 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:01.554+0000 7efe9730b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:12:01.631 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:01.626+0000 7f60835ae180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:12:01.647 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:01.642+0000 7efe9730b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:12:01.797 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:01.790+0000 7efe9730b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:12:01.799 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:01.924 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17b6400640 1 Processor -- start 2024-04-25T10:12:01.924 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17b6400640 1 -- start start 2024-04-25T10:12:01.926 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17b6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b00d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.926 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17b6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b00d1a40 con 0x7f17b00d0890 2024-04-25T10:12:01.926 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17b6400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b0146e10 con 0x7f17b0146a30 2024-04-25T10:12:01.926 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17b6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00d1800 con 0x7f17b00b3d60 2024-04-25T10:12:01.926 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17afe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f17b00d0890 0x7f17b006c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53180/0 (socket says 172.21.15.87:53180) 2024-04-25T10:12:01.926 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17afe00640 1 -- 172.21.15.87:0/3322911933 learned_addr learned my addr 172.21.15.87:0/3322911933 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:01.926 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17aea00640 1 --2- 172.21.15.87:0/3322911933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b00d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.927 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17ae000640 1 -- 172.21.15.87:0/3322911933 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348926560 0 0) 0x7f17b00d1a40 con 0x7f17b00d0890 2024-04-25T10:12:01.927 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17ae000640 1 -- 172.21.15.87:0/3322911933 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1794003530 con 0x7f17b00d0890 2024-04-25T10:12:01.927 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17ae000640 1 -- 172.21.15.87:0/3322911933 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924525863 0 0) 0x7f17b0146e10 con 0x7f17b0146a30 2024-04-25T10:12:01.927 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17ae000640 1 -- 172.21.15.87:0/3322911933 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b00d1a40 con 0x7f17b0146a30 2024-04-25T10:12:01.927 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17aea00640 1 -- 172.21.15.87:0/3322911933 >> v1:172.21.15.169:6789/0 conn(0x7f17b0146a30 legacy=0x7f17b00b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.927 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17aea00640 1 -- 172.21.15.87:0/3322911933 >> v1:172.21.15.87:6789/0 conn(0x7f17b00d0890 legacy=0x7f17b006c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.928 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.918+0000 7f17aea00640 1 -- 172.21.15.87:0/3322911933 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00d1ce0 con 0x7f17b00b3d60 2024-04-25T10:12:01.929 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17aea00640 1 --2- 172.21.15.87:0/3322911933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b00d0350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f179c0097f0 tx=0x7f179c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e14064f5104a4702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.930 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17ae000640 1 -- 172.21.15.87:0/3322911933 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0b0070 con 0x7f17b00b3d60 2024-04-25T10:12:01.930 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17ae000640 1 -- 172.21.15.87:0/3322911933 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c0ab600 con 0x7f17b00b3d60 2024-04-25T10:12:01.930 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17ae000640 1 -- 172.21.15.87:0/3322911933 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0ab9d0 con 0x7f17b00b3d60 2024-04-25T10:12:01.930 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17b6400640 1 -- 172.21.15.87:0/3322911933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 msgr2=0x7f17b00d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.931 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17b6400640 1 --2- 172.21.15.87:0/3322911933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b00d0350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f179c0097f0 tx=0x7f179c01c450 comp rx=0 tx=0).stop 2024-04-25T10:12:01.931 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17b6400640 1 -- 172.21.15.87:0/3322911933 shutdown_connections 2024-04-25T10:12:01.931 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17b6400640 1 --2- 172.21.15.87:0/3322911933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b00d0350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.931 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17b6400640 1 -- 172.21.15.87:0/3322911933 >> 172.21.15.87:0/3322911933 conn(0x7f17b0066920 msgr2=0x7f17b0066d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.931 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17b6400640 1 -- 172.21.15.87:0/3322911933 shutdown_connections 2024-04-25T10:12:01.931 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.922+0000 7f17b6400640 1 -- 172.21.15.87:0/3322911933 wait complete. 2024-04-25T10:12:01.932 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 Processor -- start 2024-04-25T10:12:01.932 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 -- start start 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17b00d0890 0x7f17b00cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b0146a30 0x7f17b015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b006d700 con 0x7f17b0146a30 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b006d580 con 0x7f17b00d0890 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b006d880 con 0x7f17b00b3d60 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b0146a30 0x7f17b015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17af400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17aea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17b00d0890 0x7f17b00cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.934 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17af400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b015d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:60180/0 (socket says 172.21.15.87:60180) 2024-04-25T10:12:01.935 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17af400640 1 -- 172.21.15.87:0/3110688310 learned_addr learned my addr 172.21.15.87:0/3110688310 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:01.935 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17aea00640 1 -- 172.21.15.87:0/3110688310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 msgr2=0x7f17b015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.935 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17aea00640 1 --2- 172.21.15.87:0/3110688310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.935 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17aea00640 1 -- 172.21.15.87:0/3110688310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b0146a30 msgr2=0x7f17b015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.935 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17aea00640 1 --2- 172.21.15.87:0/3110688310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b0146a30 0x7f17b015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.935 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17aea00640 1 -- 172.21.15.87:0/3110688310 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00cceb0 con 0x7f17b00d0890 2024-04-25T10:12:01.935 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17aea00640 1 --2- 172.21.15.87:0/3110688310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17b00d0890 0x7f17b00cb750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f179c0a4fd0 tx=0x7f179c0191f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17a7400640 1 -- 172.21.15.87:0/3110688310 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0b9020 con 0x7f17b00d0890 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17a7400640 1 -- 172.21.15.87:0/3110688310 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c0b5480 con 0x7f17b00d0890 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17a7400640 1 -- 172.21.15.87:0/3110688310 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0b0040 con 0x7f17b00d0890 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.926+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b0145360 con 0x7f17b00d0890 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.930+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b00cc0f0 con 0x7f17b00d0890 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.930+0000 7f17a7400640 1 -- 172.21.15.87:0/3110688310 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f179c0bc020 con 0x7f17b00d0890 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.930+0000 7f17a7400640 1 -- 172.21.15.87:0/3110688310 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f179c0bebc0 con 0x7f17b00d0890 2024-04-25T10:12:01.936 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.930+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b004ef30 con 0x7f17b00d0890 2024-04-25T10:12:01.938 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.930+0000 7f17a7400640 1 -- 172.21.15.87:0/3110688310 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f179c0c3030 con 0x7f17b00d0890 2024-04-25T10:12:01.994 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.986+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17b015adf0 con 0x7f17b00d0890 2024-04-25T10:12:01.994 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.986+0000 7f17a7400640 1 -- 172.21.15.87:0/3110688310 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f179c0160d0 con 0x7f17b00d0890 2024-04-25T10:12:01.995 INFO:teuthology.misc.health.smithi087.stdout: 2024-04-25T10:12:01.995 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":10,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:00.631609+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17b00d0890 msgr2=0x7f17b00cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 --2- 172.21.15.87:0/3110688310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17b00d0890 0x7f17b00cb750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f179c0a4fd0 tx=0x7f179c0191f0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 shutdown_connections 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 --2- 172.21.15.87:0/3110688310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b0146a30 0x7f17b015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 --2- 172.21.15.87:0/3110688310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17b00d0890 0x7f17b00cb750 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 --2- 172.21.15.87:0/3110688310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17b00b3d60 0x7f17b015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 >> 172.21.15.87:0/3110688310 conn(0x7f17b0066920 msgr2=0x7f17b00b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 shutdown_connections 2024-04-25T10:12:01.996 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:01.990+0000 7f17b6400640 1 -- 172.21.15.87:0/3110688310 wait complete. 2024-04-25T10:12:02.013 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:12:02.338 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:02.330+0000 7f60835ae180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:12:02.425 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:02.418+0000 7f60835ae180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:12:02.469 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:02.462+0000 7efe9730b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:12:02.554 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:02.550+0000 7efe9730b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:12:02.626 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:02.618+0000 7f60835ae180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:12:02.712 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:02.706+0000 7f60835ae180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:12:02.754 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:02.750+0000 7efe9730b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:12:02.838 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:02.830+0000 7efe9730b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:12:03.040 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:03.034+0000 7f60835ae180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:12:03.110 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:03.102+0000 7f60835ae180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:12:03.169 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:03.162+0000 7efe9730b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:12:03.240 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T10:12:03.234+0000 7efe9730b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:12:04.132 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:04.126+0000 7f6029600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:12:04.134 INFO:tasks.ceph.mgr.y.smithi087.stderr:2024-04-25T10:12:04.126+0000 7f6029600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:12:05.297 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-04-25T10:12:05.289+0000 7f9f4ec127c0 -1 osd.3 0 log_to_monitors true 2024-04-25T10:12:05.358 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T10:12:05.354+0000 7fe3354607c0 -1 osd.4 0 log_to_monitors true 2024-04-25T10:12:05.394 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T10:12:05.390+0000 7f077f7c47c0 -1 osd.7 0 log_to_monitors true 2024-04-25T10:12:05.465 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-04-25T10:12:05.457+0000 7f77f82b27c0 -1 osd.2 0 log_to_monitors true 2024-04-25T10:12:05.467 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-04-25T10:12:05.461+0000 7f724db6e7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T10:12:05.472 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-04-25T10:12:05.465+0000 7f70b825e7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T10:12:05.605 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T10:12:05.598+0000 7f40298d97c0 -1 osd.6 0 log_to_monitors true 2024-04-25T10:12:05.608 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T10:12:05.602+0000 7f28f22807c0 -1 osd.5 0 log_to_monitors true 2024-04-25T10:12:08.819 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:09.026 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.017+0000 7fd810a00640 1 Processor -- start 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.017+0000 7fd810a00640 1 -- start start 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd810a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd804047c20 0x7fd8040eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd810a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd804005ea0 0x7fd8040062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd810a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd804034220 con 0x7fd804033e40 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd810a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd804050210 con 0x7fd804005ea0 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd810a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd804037bc0 con 0x7fd804047c20 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd809e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd804047c20 0x7fd8040eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.029 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd809e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd804047c20 0x7fd8040eb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:54518/0 (socket says 172.21.15.87:54518) 2024-04-25T10:12:09.030 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd809e00640 1 -- 172.21.15.87:0/456355789 learned_addr learned my addr 172.21.15.87:0/456355789 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:09.031 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd809e00640 1 -- 172.21.15.87:0/456355789 >> v2:172.21.15.87:3300/0 conn(0x7fd804005ea0 msgr2=0x7fd8040062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:09.031 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd809e00640 1 --2- 172.21.15.87:0/456355789 >> v2:172.21.15.87:3300/0 conn(0x7fd804005ea0 0x7fd8040062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.031 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd809e00640 1 -- 172.21.15.87:0/456355789 >> v1:172.21.15.87:6789/0 conn(0x7fd804033e40 legacy=0x7fd8040eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.031 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.021+0000 7fd809e00640 1 -- 172.21.15.87:0/456355789 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80404fa30 con 0x7fd804047c20 2024-04-25T10:12:09.035 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd809e00640 1 --2- 172.21.15.87:0/456355789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804047c20 0x7fd8040eb890 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd80400afb0 tx=0x7fd7fc01ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4654678785619268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:09.035 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd809400640 1 -- 172.21.15.87:0/456355789 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0b5070 con 0x7fd804047c20 2024-04-25T10:12:09.035 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd809400640 1 -- 172.21.15.87:0/456355789 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc0a8a30 con 0x7fd804047c20 2024-04-25T10:12:09.036 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd809400640 1 -- 172.21.15.87:0/456355789 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0a8e00 con 0x7fd804047c20 2024-04-25T10:12:09.037 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 -- 172.21.15.87:0/456355789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804047c20 msgr2=0x7fd8040eb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.037 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 --2- 172.21.15.87:0/456355789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804047c20 0x7fd8040eb890 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd80400afb0 tx=0x7fd7fc01ed50 comp rx=0 tx=0).stop 2024-04-25T10:12:09.038 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 -- 172.21.15.87:0/456355789 shutdown_connections 2024-04-25T10:12:09.038 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 --2- 172.21.15.87:0/456355789 >> v2:172.21.15.87:3300/0 conn(0x7fd804005ea0 0x7fd8040062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.038 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 --2- 172.21.15.87:0/456355789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804047c20 0x7fd8040eb890 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.038 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 -- 172.21.15.87:0/456355789 >> 172.21.15.87:0/456355789 conn(0x7fd804009fc0 msgr2=0x7fd80400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:09.039 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 -- 172.21.15.87:0/456355789 shutdown_connections 2024-04-25T10:12:09.039 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.029+0000 7fd810a00640 1 -- 172.21.15.87:0/456355789 wait complete. 2024-04-25T10:12:09.041 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.033+0000 7fd810a00640 1 Processor -- start 2024-04-25T10:12:09.041 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.033+0000 7fd810a00640 1 -- start start 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.033+0000 7fd810a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd804005ea0 0x7fd8040d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.033+0000 7fd810a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804033e40 0x7fd8040d5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd810a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd804047c20 0x7fd80404d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd810a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80404fd80 con 0x7fd804005ea0 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd810a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80404f090 con 0x7fd804033e40 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd810a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80404f5c0 con 0x7fd804047c20 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd804047c20 0x7fd80404d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd804047c20 0x7fd80404d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:44576/0 (socket says 172.21.15.87:44576) 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 -- 172.21.15.87:0/2550514085 learned_addr learned my addr 172.21.15.87:0/2550514085 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd809e00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804033e40 0x7fd8040d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80a800640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd804005ea0 0x7fd8040d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.048 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 -- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804033e40 msgr2=0x7fd8040d5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804033e40 0x7fd8040d5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 -- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd804005ea0 msgr2=0x7fd8040d4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd804005ea0 0x7fd8040d4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 -- 172.21.15.87:0/2550514085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80404e620 con 0x7fd804047c20 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd80b200640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd804047c20 0x7fd80404d8f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd80c064760 tx=0x7fd80c04f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd802a00640 1 -- 172.21.15.87:0/2550514085 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c08f070 con 0x7fd804047c20 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd802a00640 1 -- 172.21.15.87:0/2550514085 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c073670 con 0x7fd804047c20 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd802a00640 1 -- 172.21.15.87:0/2550514085 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0933b0 con 0x7fd804047c20 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8040d0f30 con 0x7fd804047c20 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.037+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8040d14f0 con 0x7fd804047c20 2024-04-25T10:12:09.049 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.041+0000 7fd802a00640 1 -- 172.21.15.87:0/2550514085 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7fd80c092070 con 0x7fd804047c20 2024-04-25T10:12:09.050 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.041+0000 7fd802a00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7ec066ed0 0x7fd7ec069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.050 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.041+0000 7fd802a00640 1 -- 172.21.15.87:0/2550514085 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fd80c0feab0 con 0x7fd804047c20 2024-04-25T10:12:09.050 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.041+0000 7fd80a800640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7ec066ed0 0x7fd7ec069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.051 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.045+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8040053a0 con 0x7fd804047c20 2024-04-25T10:12:09.051 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.045+0000 7fd80a800640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7ec066ed0 0x7fd7ec069390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd804048d70 tx=0x7fd7f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:09.063 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.053+0000 7fd802a00640 1 -- 172.21.15.87:0/2550514085 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd80c0d0920 con 0x7fd804047c20 2024-04-25T10:12:09.341 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.333+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8040ebf00 con 0x7fd804047c20 2024-04-25T10:12:09.346 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.337+0000 7fd802a00640 1 -- 172.21.15.87:0/2550514085 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fd80c0d0220 con 0x7fd804047c20 2024-04-25T10:12:09.347 INFO:teuthology.misc.health.smithi087.stdout: 2024-04-25T10:12:09.347 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":12,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:07.169993+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:09.350 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7ec066ed0 msgr2=0x7fd7ec069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.350 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7ec066ed0 0x7fd7ec069390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd804048d70 tx=0x7fd7f8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:09.350 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd804047c20 msgr2=0x7fd80404d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.350 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd804047c20 0x7fd80404d8f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd80c064760 tx=0x7fd80c04f960 comp rx=0 tx=0).stop 2024-04-25T10:12:09.351 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 shutdown_connections 2024-04-25T10:12:09.351 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7ec066ed0 0x7fd7ec069390 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.351 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd804047c20 0x7fd80404d8f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.351 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804033e40 0x7fd8040d5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.351 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 --2- 172.21.15.87:0/2550514085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd804005ea0 0x7fd8040d4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.351 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.341+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 >> 172.21.15.87:0/2550514085 conn(0x7fd804009fc0 msgr2=0x7fd8040071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:09.352 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.345+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 shutdown_connections 2024-04-25T10:12:09.352 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:09.345+0000 7fd810a00640 1 -- 172.21.15.87:0/2550514085 wait complete. 2024-04-25T10:12:09.380 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:12:09.386 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T10:12:09.382+0000 7fe331800640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:12:09.389 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-04-25T10:12:09.377+0000 7f70b2800640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:12:09.404 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T10:12:09.398+0000 7f28ee600640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:12:09.407 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-04-25T10:12:09.401+0000 7f9f49000640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:12:09.419 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T10:12:09.414+0000 7fe326400640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:09.422 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-04-25T10:12:09.413+0000 7f70a9200640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:09.426 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T10:12:09.422+0000 7f077ba00640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:12:09.435 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T10:12:09.430+0000 7f28e3200640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:09.436 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-04-25T10:12:09.429+0000 7f9f3fa00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:09.459 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T10:12:09.454+0000 7f0770600640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:09.464 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-04-25T10:12:09.457+0000 7f77f2800640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:12:09.477 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T10:12:09.470+0000 7f4025c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:12:09.493 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-04-25T10:12:09.485+0000 7f77e9200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:09.499 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T10:12:09.494+0000 7f401a800640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:09.639 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-04-25T10:12:09.629+0000 7f7248000640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:12:09.656 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-04-25T10:12:09.649+0000 7f723ea00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:12:16.187 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:16.322 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.317+0000 7fd8f5600640 1 Processor -- start 2024-04-25T10:12:16.322 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.317+0000 7fd8f5600640 1 -- start start 2024-04-25T10:12:16.323 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.317+0000 7fd8f5600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd8f0159640 0x7fd8f006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.323 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.317+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f00cf890 0x7fd8f00cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.323 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.317+0000 7fd8f5600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f00b4bf0 con 0x7fd8f00b46e0 2024-04-25T10:12:16.323 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.317+0000 7fd8f5600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f00b4ed0 con 0x7fd8f0159640 2024-04-25T10:12:16.323 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.317+0000 7fd8f5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f006d4f0 con 0x7fd8f00cf890 2024-04-25T10:12:16.323 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8ee000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd8f00b46e0 0x7fd8f015cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58720/0 (socket says 172.21.15.87:58720) 2024-04-25T10:12:16.323 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8ee000640 1 -- 172.21.15.87:0/1208193533 learned_addr learned my addr 172.21.15.87:0/1208193533 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:16.324 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8eea00640 1 --2- 172.21.15.87:0/1208193533 >> v2:172.21.15.169:3300/0 conn(0x7fd8f0159640 0x7fd8f006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.324 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8ef400640 1 --2- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f00cf890 0x7fd8f00cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8ed600640 1 -- 172.21.15.87:0/1208193533 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452190287 0 0) 0x7fd8f00b4bf0 con 0x7fd8f00b46e0 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8ed600640 1 -- 172.21.15.87:0/1208193533 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d4003530 con 0x7fd8f00b46e0 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8eea00640 1 -- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f00cf890 msgr2=0x7fd8f00cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8eea00640 1 --2- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f00cf890 0x7fd8f00cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8eea00640 1 -- 172.21.15.87:0/1208193533 >> v1:172.21.15.87:6789/0 conn(0x7fd8f00b46e0 legacy=0x7fd8f015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8ed600640 1 -- 172.21.15.87:0/1208193533 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061033254 0 0) 0x7fd8d4003530 con 0x7fd8f00b46e0 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8eea00640 1 -- 172.21.15.87:0/1208193533 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f006cea0 con 0x7fd8f0159640 2024-04-25T10:12:16.325 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.321+0000 7fd8ef400640 1 --2- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f00cf890 0x7fd8f00cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:16.327 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8eea00640 1 --2- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f0159640 0x7fd8f006d980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009d90 tx=0x7fd8e001c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42639ce72a16da78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.327 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8ed600640 1 -- 172.21.15.87:0/1208193533 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00b2070 con 0x7fd8f0159640 2024-04-25T10:12:16.328 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8ed600640 1 -- 172.21.15.87:0/1208193533 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00a5510 con 0x7fd8f0159640 2024-04-25T10:12:16.328 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8ed600640 1 -- 172.21.15.87:0/1208193533 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8001680 con 0x7fd8f0159640 2024-04-25T10:12:16.328 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f0159640 msgr2=0x7fd8f006d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.328 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 --2- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f0159640 0x7fd8f006d980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009d90 tx=0x7fd8e001c800 comp rx=0 tx=0).stop 2024-04-25T10:12:16.329 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- 172.21.15.87:0/1208193533 shutdown_connections 2024-04-25T10:12:16.329 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 --2- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f00cf890 0x7fd8f00cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.329 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 --2- 172.21.15.87:0/1208193533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f0159640 0x7fd8f006d980 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.329 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- 172.21.15.87:0/1208193533 >> 172.21.15.87:0/1208193533 conn(0x7fd8f006abd0 msgr2=0x7fd8f006afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.329 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- 172.21.15.87:0/1208193533 shutdown_connections 2024-04-25T10:12:16.329 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- 172.21.15.87:0/1208193533 wait complete. 2024-04-25T10:12:16.330 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 Processor -- start 2024-04-25T10:12:16.330 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- start start 2024-04-25T10:12:16.332 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8f00b46e0 0x7fd8f0150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.332 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f00cf890 0x7fd8f00afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.332 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f0159640 0x7fd8f0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.332 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b35d0 con 0x7fd8f00b46e0 2024-04-25T10:12:16.332 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b3450 con 0x7fd8f00cf890 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8f5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b3750 con 0x7fd8f0159640 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.325+0000 7fd8eea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8f00b46e0 0x7fd8f0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8f00b46e0 0x7fd8f0150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40260/0 (socket says 172.21.15.87:40260) 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 -- 172.21.15.87:0/3519843992 learned_addr learned my addr 172.21.15.87:0/3519843992 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8ef400640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f0159640 0x7fd8f0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8ee000640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f00cf890 0x7fd8f00afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 -- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f0159640 msgr2=0x7fd8f0154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f0159640 0x7fd8f0154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 -- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f00cf890 msgr2=0x7fd8f00afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f00cf890 0x7fd8f00afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 -- 172.21.15.87:0/3519843992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00b27e0 con 0x7fd8f00b46e0 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8ee000640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f00cf890 0x7fd8f00afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8eea00640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8f00b46e0 0x7fd8f0150180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8e001aeb0 tx=0x7fd8e00195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.333 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8e6a00640 1 -- 172.21.15.87:0/3519843992 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00bd020 con 0x7fd8f00b46e0 2024-04-25T10:12:16.334 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8e6a00640 1 -- 172.21.15.87:0/3519843992 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00b8470 con 0x7fd8f00b46e0 2024-04-25T10:12:16.334 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8e6a00640 1 -- 172.21.15.87:0/3519843992 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00b2040 con 0x7fd8f00b46e0 2024-04-25T10:12:16.334 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f00b0d70 con 0x7fd8f00b46e0 2024-04-25T10:12:16.334 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f00b04d0 con 0x7fd8f00b46e0 2024-04-25T10:12:16.334 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f004ef30 con 0x7fd8f00b46e0 2024-04-25T10:12:16.334 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.329+0000 7fd8e6a00640 1 -- 172.21.15.87:0/3519843992 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8e00c0020 con 0x7fd8f00b46e0 2024-04-25T10:12:16.335 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.333+0000 7fd8e6a00640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8cc066f60 0x7fd8cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.335 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.333+0000 7fd8e6a00640 1 -- 172.21.15.87:0/3519843992 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd8e011f970 con 0x7fd8f00b46e0 2024-04-25T10:12:16.335 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.333+0000 7fd8ee000640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8cc066f60 0x7fd8cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.336 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.333+0000 7fd8ee000640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8cc066f60 0x7fd8cc069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8f015b330 tx=0x7fd8d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.340 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.337+0000 7fd8e6a00640 1 -- 172.21.15.87:0/3519843992 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8e00c7030 con 0x7fd8f00b46e0 2024-04-25T10:12:16.494 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.489+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8f00cfca0 con 0x7fd8f00b46e0 2024-04-25T10:12:16.495 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.489+0000 7fd8e6a00640 1 -- 172.21.15.87:0/3519843992 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7fd8e0016480 con 0x7fd8f00b46e0 2024-04-25T10:12:16.495 INFO:teuthology.misc.health.smithi087.stdout: 2024-04-25T10:12:16.495 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":16,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:13.417913+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:16.497 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8cc066f60 msgr2=0x7fd8cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8cc066f60 0x7fd8cc069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8f015b330 tx=0x7fd8d8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8f00b46e0 msgr2=0x7fd8f0150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8f00b46e0 0x7fd8f0150180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8e001aeb0 tx=0x7fd8e00195a0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 shutdown_connections 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8cc066f60 0x7fd8cc069420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8f0159640 0x7fd8f0154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f00cf890 0x7fd8f00afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 --2- 172.21.15.87:0/3519843992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8f00b46e0 0x7fd8f0150180 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 >> 172.21.15.87:0/3519843992 conn(0x7fd8f006abd0 msgr2=0x7fd8f015e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 shutdown_connections 2024-04-25T10:12:16.498 INFO:teuthology.misc.health.smithi087.stderr:2024-04-25T10:12:16.493+0000 7fd8f5600640 1 -- 172.21.15.87:0/3519843992 wait complete. 2024-04-25T10:12:16.514 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:12:16.515 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:12:16.515 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:12:16.515 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:12:16.518 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:12:16.518 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:12:16.519 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:12:16.521 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:12:16.521 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:12:16.522 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:12:16.525 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:12:16.525 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:12:16.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:12:16.528 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:12:16.529 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:12:16.529 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:16.633 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:12:16.633 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:12:16.638 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:12:16.639 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:12:16.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f199ee00640 1 Processor -- start 2024-04-25T10:12:16.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f199ee00640 1 -- start start 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f199ee00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19980d0040 0x7f199815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f199ee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f199815b740 0x7f19980b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f199ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1998071050 con 0x7f19981550b0 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f199ee00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199815d040 con 0x7f19980d0040 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f199ee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19980712f0 con 0x7f199815b740 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f1997400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19980d0040 0x7f199815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f1997400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19980d0040 0x7f199815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40300/0 (socket says 172.21.15.87:40300) 2024-04-25T10:12:16.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.669+0000 7f1997400640 1 -- 172.21.15.87:0/3131208765 learned_addr learned my addr 172.21.15.87:0/3131208765 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f199ce00640 1 --2- 172.21.15.87:0/3131208765 >> v2:172.21.15.169:3300/0 conn(0x7f199815b740 0x7f19980b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1997400640 1 -- 172.21.15.87:0/3131208765 >> v2:172.21.15.169:3300/0 conn(0x7f199815b740 msgr2=0x7f19980b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1997400640 1 --2- 172.21.15.87:0/3131208765 >> v2:172.21.15.169:3300/0 conn(0x7f199815b740 0x7f19980b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1997400640 1 -- 172.21.15.87:0/3131208765 >> v1:172.21.15.169:6789/0 conn(0x7f19981550b0 legacy=0x7f19980cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1997400640 1 -- 172.21.15.87:0/3131208765 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19980715d0 con 0x7f19980d0040 2024-04-25T10:12:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1996a00640 1 -- 172.21.15.87:0/3131208765 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973979545 0 0) 0x7f1998071050 con 0x7f19981550b0 2024-04-25T10:12:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1997400640 1 --2- 172.21.15.87:0/3131208765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19980d0040 0x7f199815b200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19980d04d0 tx=0x7f198401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65321a91641590f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1996a00640 1 -- 172.21.15.87:0/3131208765 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840aa650 con 0x7f19980d0040 2024-04-25T10:12:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1996a00640 1 -- 172.21.15.87:0/3131208765 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19840af070 con 0x7f19980d0040 2024-04-25T10:12:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.673+0000 7f1996a00640 1 -- 172.21.15.87:0/3131208765 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840aaa10 con 0x7f19980d0040 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- 172.21.15.87:0/3131208765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19980d0040 msgr2=0x7f199815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 --2- 172.21.15.87:0/3131208765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19980d0040 0x7f199815b200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19980d04d0 tx=0x7f198401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- 172.21.15.87:0/3131208765 shutdown_connections 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 --2- 172.21.15.87:0/3131208765 >> v2:172.21.15.169:3300/0 conn(0x7f199815b740 0x7f19980b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 --2- 172.21.15.87:0/3131208765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19980d0040 0x7f199815b200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- 172.21.15.87:0/3131208765 >> 172.21.15.87:0/3131208765 conn(0x7f1998066b70 msgr2=0x7f1998066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- 172.21.15.87:0/3131208765 shutdown_connections 2024-04-25T10:12:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- 172.21.15.87:0/3131208765 wait complete. 2024-04-25T10:12:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 Processor -- start 2024-04-25T10:12:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- start start 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19980d0040 0x7f19981569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19981550b0 0x7f19981504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199815b740 0x7f19980af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199815bcd0 con 0x7f19981550b0 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199815bb50 con 0x7f199815b740 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199815c030 con 0x7f19980d0040 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19981550b0 0x7f19981504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19981550b0 0x7f19981504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40310/0 (socket says 172.21.15.87:40310) 2024-04-25T10:12:16.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 -- 172.21.15.87:0/342013766 learned_addr learned my addr 172.21.15.87:0/342013766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:16.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ce00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199815b740 0x7f19980af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 -- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19980d0040 msgr2=0x7f19981569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19980d0040 0x7f19981569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 -- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199815b740 msgr2=0x7f19980af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199815b740 0x7f19980af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f1997400640 1 -- 172.21.15.87:0/342013766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998151030 con 0x7f19981550b0 2024-04-25T10:12:16.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.677+0000 7f199ce00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199815b740 0x7f19980af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f1997400640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19981550b0 0x7f19981504b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f198401aba0 tx=0x7f198401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f198fe00640 1 -- 172.21.15.87:0/342013766 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840b9070 con 0x7f19981550b0 2024-04-25T10:12:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f198fe00640 1 -- 172.21.15.87:0/342013766 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19840a6040 con 0x7f19981550b0 2024-04-25T10:12:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f198fe00640 1 -- 172.21.15.87:0/342013766 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984004420 con 0x7f19981550b0 2024-04-25T10:12:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1998151310 con 0x7f19981550b0 2024-04-25T10:12:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19981518d0 con 0x7f19981550b0 2024-04-25T10:12:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199804ef30 con 0x7f19981550b0 2024-04-25T10:12:16.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f198fe00640 1 -- 172.21.15.87:0/342013766 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19840aa8d0 con 0x7f19981550b0 2024-04-25T10:12:16.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f198fe00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1978066f60 0x7f1978069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f198fe00640 1 -- 172.21.15.87:0/342013766 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f198411e990 con 0x7f19981550b0 2024-04-25T10:12:16.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.681+0000 7f1997e00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1978066f60 0x7f1978069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.685+0000 7f1997e00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1978066f60 0x7f1978069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1988001d30 tx=0x7f1988007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.685+0000 7f198fe00640 1 -- 172.21.15.87:0/342013766 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19840c4030 con 0x7f19981550b0 2024-04-25T10:12:16.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.841+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19981509f0 con 0x7f19981550b0 2024-04-25T10:12:16.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f198fe00640 1 -- 172.21.15.87:0/342013766 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f1984016110 con 0x7f19981550b0 2024-04-25T10:12:16.848 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:16.848 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:13.417913+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:16.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1978066f60 msgr2=0x7f1978069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1978066f60 0x7f1978069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1988001d30 tx=0x7f1988007ca0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19981550b0 msgr2=0x7f19981504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19981550b0 0x7f19981504b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f198401aba0 tx=0x7f198401ad30 comp rx=0 tx=0).stop 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 shutdown_connections 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1978066f60 0x7f1978069420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199815b740 0x7f19980af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19981550b0 0x7f19981504b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 --2- 172.21.15.87:0/342013766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19980d0040 0x7f19981569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 >> 172.21.15.87:0/342013766 conn(0x7f1998066b70 msgr2=0x7f19980ade00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 shutdown_connections 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:16.845+0000 7f199ee00640 1 -- 172.21.15.87:0/342013766 wait complete. 2024-04-25T10:12:16.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:16.869 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:17.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.001+0000 7fcd71e00640 1 Processor -- start 2024-04-25T10:12:17.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.001+0000 7fcd71e00640 1 -- start start 2024-04-25T10:12:17.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd71e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd6c0aeda0 0x7fcd6c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd71e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6c159920 con 0x7fcd6c159540 2024-04-25T10:12:17.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd71e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6c15c2f0 con 0x7fcd6c072020 2024-04-25T10:12:17.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd71e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c15c5d0 con 0x7fcd6c0aeda0 2024-04-25T10:12:17.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd6aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcd6c159540 0x7fcd6c0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58742/0 (socket says 172.21.15.87:58742) 2024-04-25T10:12:17.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd6aa00640 1 -- 172.21.15.87:0/4169409602 learned_addr learned my addr 172.21.15.87:0/4169409602 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:17.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd62000640 1 --2- 172.21.15.87:0/4169409602 >> v2:172.21.15.87:3300/0 conn(0x7fcd6c0aeda0 0x7fcd6c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd62000640 1 -- 172.21.15.87:0/4169409602 >> v1:172.21.15.169:6789/0 conn(0x7fcd6c072020 legacy=0x7fcd6c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd62000640 1 -- 172.21.15.87:0/4169409602 >> v1:172.21.15.87:6789/0 conn(0x7fcd6c159540 legacy=0x7fcd6c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd62000640 1 -- 172.21.15.87:0/4169409602 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c15c8b0 con 0x7fcd6c0aeda0 2024-04-25T10:12:17.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.005+0000 7fcd6a000640 1 -- 172.21.15.87:0/4169409602 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853307194 0 0) 0x7fcd6c159920 con 0x7fcd6c159540 2024-04-25T10:12:17.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd62000640 1 --2- 172.21.15.87:0/4169409602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c0aeda0 0x7fcd6c071ae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd540097c0 tx=0x7fcd5401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5003206eba81441c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd6a000640 1 -- 172.21.15.87:0/4169409602 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd540aa5f0 con 0x7fcd6c0aeda0 2024-04-25T10:12:17.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd6a000640 1 -- 172.21.15.87:0/4169409602 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd540af070 con 0x7fcd6c0aeda0 2024-04-25T10:12:17.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd6a000640 1 -- 172.21.15.87:0/4169409602 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd540aa9b0 con 0x7fcd6c0aeda0 2024-04-25T10:12:17.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 -- 172.21.15.87:0/4169409602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c0aeda0 msgr2=0x7fcd6c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 --2- 172.21.15.87:0/4169409602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c0aeda0 0x7fcd6c071ae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd540097c0 tx=0x7fcd5401c470 comp rx=0 tx=0).stop 2024-04-25T10:12:17.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 -- 172.21.15.87:0/4169409602 shutdown_connections 2024-04-25T10:12:17.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 --2- 172.21.15.87:0/4169409602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c0aeda0 0x7fcd6c071ae0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 -- 172.21.15.87:0/4169409602 >> 172.21.15.87:0/4169409602 conn(0x7fcd6c0669a0 msgr2=0x7fcd6c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 -- 172.21.15.87:0/4169409602 shutdown_connections 2024-04-25T10:12:17.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 -- 172.21.15.87:0/4169409602 wait complete. 2024-04-25T10:12:17.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 Processor -- start 2024-04-25T10:12:17.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.009+0000 7fcd71e00640 1 -- start start 2024-04-25T10:12:17.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c072020 0x7fcd6c14b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd6c0aeda0 0x7fcd6c14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6c159540 0x7fcd6c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c169940 con 0x7fcd6c072020 2024-04-25T10:12:17.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c072430 con 0x7fcd6c159540 2024-04-25T10:12:17.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c169ac0 con 0x7fcd6c0aeda0 2024-04-25T10:12:17.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c072020 0x7fcd6c14b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c072020 0x7fcd6c14b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40332/0 (socket says 172.21.15.87:40332) 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 -- 172.21.15.87:0/2991526332 learned_addr learned my addr 172.21.15.87:0/2991526332 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd62000640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd6c0aeda0 0x7fcd6c14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6b400640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6c159540 0x7fcd6c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 -- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd6c0aeda0 msgr2=0x7fcd6c14f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd6c0aeda0 0x7fcd6c14f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 -- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6c159540 msgr2=0x7fcd6c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6c159540 0x7fcd6c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 -- 172.21.15.87:0/2991526332 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c136be0 con 0x7fcd6c072020 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd6aa00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c072020 0x7fcd6c14b1c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c410 tx=0x7fcd5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd63e00640 1 -- 172.21.15.87:0/2991526332 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0074e0 con 0x7fcd6c072020 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd63e00640 1 -- 172.21.15.87:0/2991526332 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c0047f0 con 0x7fcd6c072020 2024-04-25T10:12:17.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6c1394c0 con 0x7fcd6c072020 2024-04-25T10:12:17.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd63e00640 1 -- 172.21.15.87:0/2991526332 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c007680 con 0x7fcd6c072020 2024-04-25T10:12:17.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c139a80 con 0x7fcd6c072020 2024-04-25T10:12:17.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c04ef30 con 0x7fcd6c072020 2024-04-25T10:12:17.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.013+0000 7fcd63e00640 1 -- 172.21.15.87:0/2991526332 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd5c011070 con 0x7fcd6c072020 2024-04-25T10:12:17.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.017+0000 7fcd63e00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.017+0000 7fcd63e00640 1 -- 172.21.15.87:0/2991526332 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd5c0848f0 con 0x7fcd6c072020 2024-04-25T10:12:17.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.017+0000 7fcd62000640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.017+0000 7fcd62000640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd48066f60 0x7fcd48069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd540ae000 tx=0x7fcd540a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.021+0000 7fcd63e00640 1 -- 172.21.15.87:0/2991526332 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd5c055bb0 con 0x7fcd6c072020 2024-04-25T10:12:17.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd6c06d7f0 con 0x7fcd6c072020 2024-04-25T10:12:17.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd63e00640 1 -- 172.21.15.87:0/2991526332 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd5c0554b0 con 0x7fcd6c072020 2024-04-25T10:12:17.177 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd48066f60 msgr2=0x7fcd48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd48066f60 0x7fcd48069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd540ae000 tx=0x7fcd540a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c072020 msgr2=0x7fcd6c14b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c072020 0x7fcd6c14b1c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c410 tx=0x7fcd5c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 shutdown_connections 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6c159540 0x7fcd6c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd6c0aeda0 0x7fcd6c14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 --2- 172.21.15.87:0/2991526332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd6c072020 0x7fcd6c14b1c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 >> 172.21.15.87:0/2991526332 conn(0x7fcd6c0669a0 msgr2=0x7fcd6c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 shutdown_connections 2024-04-25T10:12:17.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.173+0000 7fcd71e00640 1 -- 172.21.15.87:0/2991526332 wait complete. 2024-04-25T10:12:17.197 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:17.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef99600640 1 Processor -- start 2024-04-25T10:12:17.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef99600640 1 -- start start 2024-04-25T10:12:17.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef99600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef9415dc20 0x7fef940b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef99600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef940b55c0 0x7fef940b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef99600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef940b6f70 con 0x7fef941617c0 2024-04-25T10:12:17.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef99600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef940b7250 con 0x7fef9415dc20 2024-04-25T10:12:17.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef99600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef940b7530 con 0x7fef940b55c0 2024-04-25T10:12:17.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef92a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef9415dc20 0x7fef940b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef92a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef9415dc20 0x7fef940b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40348/0 (socket says 172.21.15.87:40348) 2024-04-25T10:12:17.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef92a00640 1 -- 172.21.15.87:0/4240282724 learned_addr learned my addr 172.21.15.87:0/4240282724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:17.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef8a000640 1 --2- 172.21.15.87:0/4240282724 >> v2:172.21.15.169:3300/0 conn(0x7fef940b55c0 0x7fef940b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef92a00640 1 -- 172.21.15.87:0/4240282724 >> v2:172.21.15.169:3300/0 conn(0x7fef940b55c0 msgr2=0x7fef940b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef92a00640 1 --2- 172.21.15.87:0/4240282724 >> v2:172.21.15.169:3300/0 conn(0x7fef940b55c0 0x7fef940b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef92a00640 1 -- 172.21.15.87:0/4240282724 >> v1:172.21.15.169:6789/0 conn(0x7fef941617c0 legacy=0x7fef94161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef92a00640 1 -- 172.21.15.87:0/4240282724 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940b7810 con 0x7fef9415dc20 2024-04-25T10:12:17.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.329+0000 7fef8a000640 1 --2- 172.21.15.87:0/4240282724 >> v2:172.21.15.169:3300/0 conn(0x7fef940b55c0 0x7fef940b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:17.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef92a00640 1 --2- 172.21.15.87:0/4240282724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef9415dc20 0x7fef940b9380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef940b66d0 tx=0x7fef7c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7158488c70a69a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef92000640 1 -- 172.21.15.87:0/4240282724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0ab5f0 con 0x7fef9415dc20 2024-04-25T10:12:17.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef92000640 1 -- 172.21.15.87:0/4240282724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c0b0070 con 0x7fef9415dc20 2024-04-25T10:12:17.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef92000640 1 -- 172.21.15.87:0/4240282724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0ab9b0 con 0x7fef9415dc20 2024-04-25T10:12:17.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 -- 172.21.15.87:0/4240282724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef9415dc20 msgr2=0x7fef940b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 --2- 172.21.15.87:0/4240282724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef9415dc20 0x7fef940b9380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef940b66d0 tx=0x7fef7c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 -- 172.21.15.87:0/4240282724 shutdown_connections 2024-04-25T10:12:17.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 --2- 172.21.15.87:0/4240282724 >> v2:172.21.15.169:3300/0 conn(0x7fef940b55c0 0x7fef940b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 --2- 172.21.15.87:0/4240282724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef9415dc20 0x7fef940b9380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 -- 172.21.15.87:0/4240282724 >> 172.21.15.87:0/4240282724 conn(0x7fef9406aec0 msgr2=0x7fef9406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 -- 172.21.15.87:0/4240282724 shutdown_connections 2024-04-25T10:12:17.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.333+0000 7fef99600640 1 -- 172.21.15.87:0/4240282724 wait complete. 2024-04-25T10:12:17.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 Processor -- start 2024-04-25T10:12:17.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 -- start start 2024-04-25T10:12:17.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef940b55c0 0x7fef9415ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9415dc20 0x7fef9414eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941617c0 0x7fef9415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b9a40 con 0x7fef941617c0 2024-04-25T10:12:17.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b98c0 con 0x7fef9415dc20 2024-04-25T10:12:17.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b6b00 con 0x7fef940b55c0 2024-04-25T10:12:17.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941617c0 0x7fef9415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941617c0 0x7fef9415ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40352/0 (socket says 172.21.15.87:40352) 2024-04-25T10:12:17.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 -- 172.21.15.87:0/244224514 learned_addr learned my addr 172.21.15.87:0/244224514 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:17.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef92a00640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef940b55c0 0x7fef9415ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef8a000640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9415dc20 0x7fef9414eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 -- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef940b55c0 msgr2=0x7fef9415ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef940b55c0 0x7fef9415ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 -- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9415dc20 msgr2=0x7fef9414eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9415dc20 0x7fef9414eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 -- 172.21.15.87:0/244224514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9415a850 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef93400640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941617c0 0x7fef9415ec80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b970 tx=0x7fef8c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef8b400640 1 -- 172.21.15.87:0/244224514 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c019070 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef8b400640 1 -- 172.21.15.87:0/244224514 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c00cb40 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef941597b0 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef8b400640 1 -- 172.21.15.87:0/244224514 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c014440 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.337+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef94159d70 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.341+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9404ef30 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.341+0000 7fef8b400640 1 -- 172.21.15.87:0/244224514 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef8c01c070 con 0x7fef941617c0 2024-04-25T10:12:17.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.341+0000 7fef8b400640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef70066f60 0x7fef70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.341+0000 7fef8b400640 1 -- 172.21.15.87:0/244224514 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fef8c084860 con 0x7fef941617c0 2024-04-25T10:12:17.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.341+0000 7fef92a00640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef70066f60 0x7fef70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.341+0000 7fef92a00640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef70066f60 0x7fef70069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0af000 tx=0x7fef7c0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.345+0000 7fef8b400640 1 -- 172.21.15.87:0/244224514 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef8c055560 con 0x7fef941617c0 2024-04-25T10:12:17.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.529+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef94071920 con 0x7fef941617c0 2024-04-25T10:12:17.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef8b400640 1 -- 172.21.15.87:0/244224514 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fef8c059600 con 0x7fef941617c0 2024-04-25T10:12:17.536 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:17.536 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:12:17.536 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 2 2024-04-25T10:12:17.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef70066f60 msgr2=0x7fef70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef70066f60 0x7fef70069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0af000 tx=0x7fef7c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:17.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941617c0 msgr2=0x7fef9415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941617c0 0x7fef9415ec80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b970 tx=0x7fef8c00be30 comp rx=0 tx=0).stop 2024-04-25T10:12:17.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 shutdown_connections 2024-04-25T10:12:17.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef70066f60 0x7fef70069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941617c0 0x7fef9415ec80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9415dc20 0x7fef9414eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 --2- 172.21.15.87:0/244224514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef940b55c0 0x7fef9415ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 >> 172.21.15.87:0/244224514 conn(0x7fef9406aec0 msgr2=0x7fef940b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 shutdown_connections 2024-04-25T10:12:17.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:17.533+0000 7fef99600640 1 -- 172.21.15.87:0/244224514 wait complete. 2024-04-25T10:12:19.557 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:19.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.697+0000 7f25c3800640 1 Processor -- start 2024-04-25T10:12:19.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.697+0000 7f25c3800640 1 -- start start 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25c3800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f25bc071f70 0x7f25bc0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25c3800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc0b2fc0 0x7f25bc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25c3800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25bc070ed0 con 0x7f25bc15a0e0 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25c3800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc0711b0 con 0x7f25bc071f70 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25c3800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc071490 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25b3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f25bc071f70 0x7f25bc0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25b3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f25bc071f70 0x7f25bc0b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45820/0 (socket says 172.21.15.87:45820) 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25b3e00640 1 -- 172.21.15.87:0/4284776076 learned_addr learned my addr 172.21.15.87:0/4284776076 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25c1800640 1 --2- 172.21.15.87:0/4284776076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc0b2fc0 0x7f25bc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25b3e00640 1 -- 172.21.15.87:0/4284776076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc0b2fc0 msgr2=0x7f25bc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25b3e00640 1 --2- 172.21.15.87:0/4284776076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc0b2fc0 0x7f25bc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25b3e00640 1 -- 172.21.15.87:0/4284776076 >> v1:172.21.15.169:6789/0 conn(0x7f25bc15a0e0 legacy=0x7f25bc15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.701+0000 7f25b3e00640 1 -- 172.21.15.87:0/4284776076 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc071770 con 0x7f25bc071f70 2024-04-25T10:12:19.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25b3e00640 1 --2- 172.21.15.87:0/4284776076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc071f70 0x7f25bc0b2a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25a40098a0 tx=0x7f25a401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee987b1aff42079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25bbe00640 1 -- 172.21.15.87:0/4284776076 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40b0070 con 0x7f25bc071f70 2024-04-25T10:12:19.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25bbe00640 1 -- 172.21.15.87:0/4284776076 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a40ab600 con 0x7f25bc071f70 2024-04-25T10:12:19.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25bbe00640 1 -- 172.21.15.87:0/4284776076 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40ab9d0 con 0x7f25bc071f70 2024-04-25T10:12:19.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 -- 172.21.15.87:0/4284776076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc071f70 msgr2=0x7f25bc0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 --2- 172.21.15.87:0/4284776076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc071f70 0x7f25bc0b2a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25a40098a0 tx=0x7f25a401c540 comp rx=0 tx=0).stop 2024-04-25T10:12:19.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 -- 172.21.15.87:0/4284776076 shutdown_connections 2024-04-25T10:12:19.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 --2- 172.21.15.87:0/4284776076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc0b2fc0 0x7f25bc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 --2- 172.21.15.87:0/4284776076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc071f70 0x7f25bc0b2a80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 -- 172.21.15.87:0/4284776076 >> 172.21.15.87:0/4284776076 conn(0x7f25bc066b70 msgr2=0x7f25bc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 -- 172.21.15.87:0/4284776076 shutdown_connections 2024-04-25T10:12:19.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 -- 172.21.15.87:0/4284776076 wait complete. 2024-04-25T10:12:19.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 Processor -- start 2024-04-25T10:12:19.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.705+0000 7f25c3800640 1 -- start start 2024-04-25T10:12:19.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25bc071f70 0x7f25bc14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc0b2fc0 0x7f25bc157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc15a0e0 0x7f25bc14a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0723f0 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc070b50 con 0x7f25bc071f70 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc072570 con 0x7f25bc15a0e0 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc0b2fc0 0x7f25bc157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25bc071f70 0x7f25bc14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc0b2fc0 0x7f25bc157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45832/0 (socket says 172.21.15.87:45832) 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 -- 172.21.15.87:0/2946962428 learned_addr learned my addr 172.21.15.87:0/2946962428 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c1800640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc15a0e0 0x7f25bc14a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 -- 172.21.15.87:0/2946962428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc15a0e0 msgr2=0x7f25bc14a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc15a0e0 0x7f25bc14a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 -- 172.21.15.87:0/2946962428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25bc071f70 msgr2=0x7f25bc14d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25bc071f70 0x7f25bc14d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 -- 172.21.15.87:0/2946962428 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc1587c0 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b3e00640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc0b2fc0 0x7f25bc157d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ae040 tx=0x7f25a40192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b9600640 1 -- 172.21.15.87:0/2946962428 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40ba070 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b9600640 1 -- 172.21.15.87:0/2946962428 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a40b4420 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc158aa0 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25b9600640 1 -- 172.21.15.87:0/2946962428 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40b0040 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc159340 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.709+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc158c70 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.713+0000 7f25b9600640 1 -- 172.21.15.87:0/2946962428 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25a40bd070 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.713+0000 7f25b9600640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.713+0000 7f25b9600640 1 -- 172.21.15.87:0/2946962428 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25a411dcc0 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.713+0000 7f25c0e00640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.713+0000 7f25c0e00640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2598066f60 0x7f2598069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25ac001770 tx=0x7f25ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.717+0000 7f25b9600640 1 -- 172.21.15.87:0/2946962428 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25a4124080 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.901+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25bc159540 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.901+0000 7f25b9600640 1 -- 172.21.15.87:0/2946962428 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f25bc159540 con 0x7f25bc0b2fc0 2024-04-25T10:12:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:19.907 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:12:19.908 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 3 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2598066f60 msgr2=0x7f2598069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2598066f60 0x7f2598069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25ac001770 tx=0x7f25ac009040 comp rx=0 tx=0).stop 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc0b2fc0 msgr2=0x7f25bc157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc0b2fc0 0x7f25bc157d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ae040 tx=0x7f25a40192e0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 shutdown_connections 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25bc15a0e0 0x7f25bc14a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25bc0b2fc0 0x7f25bc157d90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 --2- 172.21.15.87:0/2946962428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25bc071f70 0x7f25bc14d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 >> 172.21.15.87:0/2946962428 conn(0x7f25bc066b70 msgr2=0x7f25bc0af470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 shutdown_connections 2024-04-25T10:12:19.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:19.905+0000 7f25c3800640 1 -- 172.21.15.87:0/2946962428 wait complete. 2024-04-25T10:12:19.927 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:20.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.057+0000 7ff1cca00640 1 Processor -- start 2024-04-25T10:12:20.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.057+0000 7ff1cca00640 1 -- start start 2024-04-25T10:12:20.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1cca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1c81550b0 0x7ff1c80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1cca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c8071050 con 0x7ff1c815b910 2024-04-25T10:12:20.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1cca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c815d040 con 0x7ff1c80cfe70 2024-04-25T10:12:20.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1cca00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c815bd80 con 0x7ff1c81550b0 2024-04-25T10:12:20.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1c5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1c81550b0 0x7ff1c80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1c6800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff1c815b910 0x7ff1c80b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34880/0 (socket says 172.21.15.87:34880) 2024-04-25T10:12:20.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1c6800640 1 -- 172.21.15.87:0/3160742591 learned_addr learned my addr 172.21.15.87:0/3160742591 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:20.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1c5e00640 1 -- 172.21.15.87:0/3160742591 >> v1:172.21.15.169:6789/0 conn(0x7ff1c80cfe70 legacy=0x7ff1c815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1c5e00640 1 -- 172.21.15.87:0/3160742591 >> v1:172.21.15.87:6789/0 conn(0x7ff1c815b910 legacy=0x7ff1c80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.061+0000 7ff1c5e00640 1 -- 172.21.15.87:0/3160742591 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80712f0 con 0x7ff1c81550b0 2024-04-25T10:12:20.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1c5e00640 1 --2- 172.21.15.87:0/3160742591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c81550b0 0x7ff1c80cf930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80097f0 tx=0x7ff1b801e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fd1e536bb24787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1c5400640 1 -- 172.21.15.87:0/3160742591 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80ac610 con 0x7ff1c81550b0 2024-04-25T10:12:20.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1c5400640 1 -- 172.21.15.87:0/3160742591 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b80b1070 con 0x7ff1c81550b0 2024-04-25T10:12:20.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1c5400640 1 -- 172.21.15.87:0/3160742591 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80ac9d0 con 0x7ff1c81550b0 2024-04-25T10:12:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 -- 172.21.15.87:0/3160742591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c81550b0 msgr2=0x7ff1c80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3160742591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c81550b0 0x7ff1c80cf930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80097f0 tx=0x7ff1b801e6b0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 -- 172.21.15.87:0/3160742591 shutdown_connections 2024-04-25T10:12:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3160742591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c81550b0 0x7ff1c80cf930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 -- 172.21.15.87:0/3160742591 >> 172.21.15.87:0/3160742591 conn(0x7ff1c8066b70 msgr2=0x7ff1c8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 -- 172.21.15.87:0/3160742591 shutdown_connections 2024-04-25T10:12:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 -- 172.21.15.87:0/3160742591 wait complete. 2024-04-25T10:12:20.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 Processor -- start 2024-04-25T10:12:20.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 -- start start 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c80cfe70 0x7ff1c8139f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c81550b0 0x7ff1c813f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.065+0000 7ff1cca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815b910 0x7ff1c813c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1cca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c806c090 con 0x7ff1c815b910 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1cca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80720b0 con 0x7ff1c80cfe70 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1cca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c806c210 con 0x7ff1c81550b0 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815b910 0x7ff1c813c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815b910 0x7ff1c813c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45856/0 (socket says 172.21.15.87:45856) 2024-04-25T10:12:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c6800640 1 -- 172.21.15.87:0/3395822694 learned_addr learned my addr 172.21.15.87:0/3395822694 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c80cfe70 0x7ff1c8139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1bd400640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c81550b0 0x7ff1c813f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c5e00640 1 -- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c81550b0 msgr2=0x7ff1c813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c5e00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c81550b0 0x7ff1c813f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c5e00640 1 -- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815b910 msgr2=0x7ff1c813c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c5e00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815b910 0x7ff1c813c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c5e00640 1 -- 172.21.15.87:0/3395822694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c8140350 con 0x7ff1c80cfe70 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c6800640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815b910 0x7ff1c813c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:20.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1bd400640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c81550b0 0x7ff1c813f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:20.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1c5e00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c80cfe70 0x7ff1c8139f80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80ad000 tx=0x7ff1b801cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1bea00640 1 -- 172.21.15.87:0/3395822694 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80b5620 con 0x7ff1c80cfe70 2024-04-25T10:12:20.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1bea00640 1 -- 172.21.15.87:0/3395822694 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b80be070 con 0x7ff1c80cfe70 2024-04-25T10:12:20.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1bea00640 1 -- 172.21.15.87:0/3395822694 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80b5a00 con 0x7ff1c80cfe70 2024-04-25T10:12:20.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c8156b50 con 0x7ff1c80cfe70 2024-04-25T10:12:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c8154670 con 0x7ff1c80cfe70 2024-04-25T10:12:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c804ef30 con 0x7ff1c80cfe70 2024-04-25T10:12:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.069+0000 7ff1bea00640 1 -- 172.21.15.87:0/3395822694 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1b80b5ba0 con 0x7ff1c80cfe70 2024-04-25T10:12:20.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.073+0000 7ff1bea00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1a4066f90 0x7ff1a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.073+0000 7ff1bea00640 1 -- 172.21.15.87:0/3395822694 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff1b811f720 con 0x7ff1c80cfe70 2024-04-25T10:12:20.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.073+0000 7ff1bd400640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1a4066f90 0x7ff1a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.073+0000 7ff1bd400640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1a4066f90 0x7ff1a4069450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1c81fa0d0 tx=0x7ff1b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.077+0000 7ff1bea00640 1 -- 172.21.15.87:0/3395822694 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1c8140350 con 0x7ff1c80cfe70 2024-04-25T10:12:20.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.233+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1c813cf00 con 0x7ff1c80cfe70 2024-04-25T10:12:20.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.233+0000 7ff1bea00640 1 -- 172.21.15.87:0/3395822694 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7ff1b80a9070 con 0x7ff1c80cfe70 2024-04-25T10:12:20.238 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:20.238 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:13.417913+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1a4066f90 msgr2=0x7ff1a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1a4066f90 0x7ff1a4069450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1c81fa0d0 tx=0x7ff1b0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c80cfe70 msgr2=0x7ff1c8139f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c80cfe70 0x7ff1c8139f80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80ad000 tx=0x7ff1b801cef0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 shutdown_connections 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1a4066f90 0x7ff1a4069450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c815b910 0x7ff1c813c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c81550b0 0x7ff1c813f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 --2- 172.21.15.87:0/3395822694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c80cfe70 0x7ff1c8139f80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 >> 172.21.15.87:0/3395822694 conn(0x7ff1c8066b70 msgr2=0x7ff1c815b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 shutdown_connections 2024-04-25T10:12:20.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.237+0000 7ff1cca00640 1 -- 172.21.15.87:0/3395822694 wait complete. 2024-04-25T10:12:20.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:20.258 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:20.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.389+0000 7f800f200640 1 Processor -- start 2024-04-25T10:12:20.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.389+0000 7f800f200640 1 -- start start 2024-04-25T10:12:20.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 0x7f80100b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80100cfde0 con 0x7f801015f420 2024-04-25T10:12:20.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80100d0080 con 0x7f80100ce670 2024-04-25T10:12:20.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80100d0360 con 0x7f80100ce290 2024-04-25T10:12:20.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800e800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f801015f420 0x7f80100b4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34882/0 (socket says 172.21.15.87:34882) 2024-04-25T10:12:20.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800e800640 1 -- 172.21.15.87:0/2000003780 learned_addr learned my addr 172.21.15.87:0/2000003780 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800de00640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 0x7f80100b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164453416 0 0) 0x7f80100cfde0 con 0x7f801015f420 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff4003530 con 0x7f801015f420 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 229121500 0 0) 0x7f7ff4003530 con 0x7f801015f420 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 msgr2=0x7f80100b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 0x7f80100b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 >> v1:172.21.15.169:6789/0 conn(0x7f80100ce670 legacy=0x7f801015ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100d0640 con 0x7f801015f420 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f8008001a00 con 0x7f801015f420 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 >> v1:172.21.15.87:6789/0 conn(0x7f801015f420 legacy=0x7f80100b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4003a60 0x7f7ff4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff4004810 0x7f7ff4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 0x7f7ff40051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100d0360 con 0x7f7ff4004810 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff400bae0 con 0x7f7ff4003a60 2024-04-25T10:12:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40100f0 con 0x7f80100ce290 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800d400640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff4004810 0x7f7ff4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800de00640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4003a60 0x7f7ff4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800e800640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 0x7f7ff40051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800d400640 1 -- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 msgr2=0x7f7ff40051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800d400640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80100ce290 0x7f7ff40051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800d400640 1 -- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4003a60 msgr2=0x7f7ff4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800d400640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4003a60 0x7f7ff4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.393+0000 7f800d400640 1 -- 172.21.15.87:0/2000003780 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4003530 con 0x7f7ff4004810 2024-04-25T10:12:20.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800d400640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff4004810 0x7f7ff4004c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001910 tx=0x7f7ffc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123016db5d994242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0b5070 con 0x7f7ff4004810 2024-04-25T10:12:20.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc0a8910 con 0x7f7ff4004810 2024-04-25T10:12:20.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800ca00640 1 -- 172.21.15.87:0/2000003780 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc001510 con 0x7f7ff4004810 2024-04-25T10:12:20.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800f200640 1 -- 172.21.15.87:0/2000003780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff4004810 msgr2=0x7f7ff4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800f200640 1 --2- 172.21.15.87:0/2000003780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff4004810 0x7f7ff4004c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001910 tx=0x7f7ffc01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800e800640 1 -- 172.21.15.87:0/2000003780 reap_dead start 2024-04-25T10:12:20.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800f200640 1 -- 172.21.15.87:0/2000003780 shutdown_connections 2024-04-25T10:12:20.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800f200640 1 -- 172.21.15.87:0/2000003780 >> 172.21.15.87:0/2000003780 conn(0x7f80100b32b0 msgr2=0x7f80100b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800f200640 1 -- 172.21.15.87:0/2000003780 shutdown_connections 2024-04-25T10:12:20.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.397+0000 7f800f200640 1 -- 172.21.15.87:0/2000003780 wait complete. 2024-04-25T10:12:20.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 Processor -- start 2024-04-25T10:12:20.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 -- start start 2024-04-25T10:12:20.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4004810 0x7f80101eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f801015da50 0x7f80101f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f801014fed0 0x7f80100cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100b2b40 con 0x7f801015da50 2024-04-25T10:12:20.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100b29c0 con 0x7f7ff4004810 2024-04-25T10:12:20.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80100ceaf0 con 0x7f801014fed0 2024-04-25T10:12:20.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f801015da50 0x7f80101f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f801015da50 0x7f80101f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45882/0 (socket says 172.21.15.87:45882) 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 -- 172.21.15.87:0/2719843467 learned_addr learned my addr 172.21.15.87:0/2719843467 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800de00640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4004810 0x7f80101eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800e800640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f801014fed0 0x7f80100cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 -- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f801014fed0 msgr2=0x7f80100cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f801014fed0 0x7f80100cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 -- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4004810 msgr2=0x7f80101eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4004810 0x7f80101eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 -- 172.21.15.87:0/2719843467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80101f2f40 con 0x7f801015da50 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800d400640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f801015da50 0x7f80101f2a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001910 tx=0x7f7ff4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f8006000640 1 -- 172.21.15.87:0/2719843467 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0b5040 con 0x7f801015da50 2024-04-25T10:12:20.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f8006000640 1 -- 172.21.15.87:0/2719843467 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc0c0070 con 0x7f801015da50 2024-04-25T10:12:20.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f8006000640 1 -- 172.21.15.87:0/2719843467 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc005300 con 0x7f801015da50 2024-04-25T10:12:20.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80101f31f0 con 0x7f801015da50 2024-04-25T10:12:20.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80101f37b0 con 0x7f801015da50 2024-04-25T10:12:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.401+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8010075120 con 0x7f801015da50 2024-04-25T10:12:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.405+0000 7f8006000640 1 -- 172.21.15.87:0/2719843467 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ffc003600 con 0x7f801015da50 2024-04-25T10:12:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.405+0000 7f8006000640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ff0066f60 0x7f7ff0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.405+0000 7f8006000640 1 -- 172.21.15.87:0/2719843467 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ffc122570 con 0x7f801015da50 2024-04-25T10:12:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.405+0000 7f800de00640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ff0066f60 0x7f7ff0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.405+0000 7f800de00640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ff0066f60 0x7f7ff0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f801006f9a0 tx=0x7f8000009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.409+0000 7f8006000640 1 -- 172.21.15.87:0/2719843467 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ffc0189e0 con 0x7f801015da50 2024-04-25T10:12:20.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80100ccd00 con 0x7f801015da50 2024-04-25T10:12:20.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f8006000640 1 -- 172.21.15.87:0/2719843467 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ffc0f38c0 con 0x7f801015da50 2024-04-25T10:12:20.568 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:20.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ff0066f60 msgr2=0x7f7ff0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ff0066f60 0x7f7ff0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f801006f9a0 tx=0x7f8000009190 comp rx=0 tx=0).stop 2024-04-25T10:12:20.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f801015da50 msgr2=0x7f80101f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f801015da50 0x7f80101f2a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001910 tx=0x7f7ff4005840 comp rx=0 tx=0).stop 2024-04-25T10:12:20.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 shutdown_connections 2024-04-25T10:12:20.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ff0066f60 0x7f7ff0069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f801014fed0 0x7f80100cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f801015da50 0x7f80101f2a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 --2- 172.21.15.87:0/2719843467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ff4004810 0x7f80101eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 >> 172.21.15.87:0/2719843467 conn(0x7f80100b32b0 msgr2=0x7f801006ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 shutdown_connections 2024-04-25T10:12:20.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.565+0000 7f800f200640 1 -- 172.21.15.87:0/2719843467 wait complete. 2024-04-25T10:12:20.588 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:12:20.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.717+0000 7f1df3200640 1 Processor -- start 2024-04-25T10:12:20.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df3200640 1 -- start start 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1df41527a0 0x7f1df4071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40b2260 0x7f1df40714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df3200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df415c020 con 0x7f1df40d06d0 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df415cc30 con 0x7f1df41527a0 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df415c300 con 0x7f1df40b2260 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df1400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1df40d06d0 0x7f1df40b1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34888/0 (socket says 172.21.15.87:34888) 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df1400640 1 -- 172.21.15.87:0/39092863 learned_addr learned my addr 172.21.15.87:0/39092863 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df1e00640 1 --2- 172.21.15.87:0/39092863 >> v2:172.21.15.169:3300/0 conn(0x7f1df41527a0 0x7f1df4071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df2800640 1 --2- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40b2260 0x7f1df40714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df0a00640 1 -- 172.21.15.87:0/39092863 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055146985 0 0) 0x7f1df415c020 con 0x7f1df40d06d0 2024-04-25T10:12:20.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df0a00640 1 -- 172.21.15.87:0/39092863 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd8003530 con 0x7f1df40d06d0 2024-04-25T10:12:20.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df1e00640 1 -- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40b2260 msgr2=0x7f1df40714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df1e00640 1 --2- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40b2260 0x7f1df40714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df1e00640 1 -- 172.21.15.87:0/39092863 >> v1:172.21.15.87:6789/0 conn(0x7f1df40d06d0 legacy=0x7f1df40b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.721+0000 7f1df1e00640 1 -- 172.21.15.87:0/39092863 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df415c5e0 con 0x7f1df41527a0 2024-04-25T10:12:20.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df1e00640 1 --2- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df41527a0 0x7f1df4071d50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1de40097c0 tx=0x7f1de401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a5f539a0757addf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df0a00640 1 -- 172.21.15.87:0/39092863 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40b0070 con 0x7f1df41527a0 2024-04-25T10:12:20.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df0a00640 1 -- 172.21.15.87:0/39092863 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de40ab600 con 0x7f1df41527a0 2024-04-25T10:12:20.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df0a00640 1 -- 172.21.15.87:0/39092863 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40ab9b0 con 0x7f1df41527a0 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 -- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df41527a0 msgr2=0x7f1df4071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 --2- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df41527a0 0x7f1df4071d50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1de40097c0 tx=0x7f1de401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 -- 172.21.15.87:0/39092863 shutdown_connections 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 --2- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40b2260 0x7f1df40714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 --2- 172.21.15.87:0/39092863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df41527a0 0x7f1df4071d50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 -- 172.21.15.87:0/39092863 >> 172.21.15.87:0/39092863 conn(0x7f1df406ac90 msgr2=0x7f1df406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 -- 172.21.15.87:0/39092863 shutdown_connections 2024-04-25T10:12:20.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.725+0000 7f1df3200640 1 -- 172.21.15.87:0/39092863 wait complete. 2024-04-25T10:12:20.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 Processor -- start 2024-04-25T10:12:20.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 -- start start 2024-04-25T10:12:20.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b2260 0x7f1df413a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1df40d06d0 0x7f1df40b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df41527a0 0x7f1df41536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40cfd60 con 0x7f1df40d06d0 2024-04-25T10:12:20.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40cfbe0 con 0x7f1df40b2260 2024-04-25T10:12:20.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df415f5b0 con 0x7f1df41527a0 2024-04-25T10:12:20.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1df40d06d0 0x7f1df40b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1df40d06d0 0x7f1df40b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45894/0 (socket says 172.21.15.87:45894) 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 -- 172.21.15.87:0/1071400648 learned_addr learned my addr 172.21.15.87:0/1071400648 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1e00640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b2260 0x7f1df413a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df2800640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df41527a0 0x7f1df41536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 -- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df41527a0 msgr2=0x7f1df41536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df41527a0 0x7f1df41536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 -- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b2260 msgr2=0x7f1df413a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b2260 0x7f1df413a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 -- 172.21.15.87:0/1071400648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40b0a90 con 0x7f1df40d06d0 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df1400640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1df40d06d0 0x7f1df40b00b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c700 tx=0x7f1de000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1dea000640 1 -- 172.21.15.87:0/1071400648 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0017070 con 0x7f1df40d06d0 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1dea000640 1 -- 172.21.15.87:0/1071400648 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0007270 con 0x7f1df40d06d0 2024-04-25T10:12:20.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1dea000640 1 -- 172.21.15.87:0/1071400648 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0001d50 con 0x7f1df40d06d0 2024-04-25T10:12:20.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df40b0d70 con 0x7f1df40d06d0 2024-04-25T10:12:20.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df41531a0 con 0x7f1df40d06d0 2024-04-25T10:12:20.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.729+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df4075120 con 0x7f1df40d06d0 2024-04-25T10:12:20.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.733+0000 7f1dea000640 1 -- 172.21.15.87:0/1071400648 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1de001a070 con 0x7f1df40d06d0 2024-04-25T10:12:20.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.733+0000 7f1dea000640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1dd4066f60 0x7f1dd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.733+0000 7f1dea000640 1 -- 172.21.15.87:0/1071400648 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1de0083fe0 con 0x7f1df40d06d0 2024-04-25T10:12:20.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.733+0000 7f1df1e00640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1dd4066f60 0x7f1dd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.733+0000 7f1df1e00640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1dd4066f60 0x7f1dd4069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1de40ad000 tx=0x7f1de40a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.737+0000 7f1dea000640 1 -- 172.21.15.87:0/1071400648 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1de0054ce0 con 0x7f1df40d06d0 2024-04-25T10:12:20.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.889+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1df40b2d40 con 0x7f1df40d06d0 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.889+0000 7f1dea000640 1 -- 172.21.15.87:0/1071400648 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1de0058d80 con 0x7f1df40d06d0 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:12:20.895 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1dd4066f60 msgr2=0x7f1dd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1dd4066f60 0x7f1dd4069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1de40ad000 tx=0x7f1de40a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1df40d06d0 msgr2=0x7f1df40b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1df40d06d0 0x7f1df40b00b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c700 tx=0x7f1de000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 shutdown_connections 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1dd4066f60 0x7f1dd4069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df41527a0 0x7f1df41536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1df40d06d0 0x7f1df40b00b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 --2- 172.21.15.87:0/1071400648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b2260 0x7f1df413a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 >> 172.21.15.87:0/1071400648 conn(0x7f1df406ac90 msgr2=0x7f1df415e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 shutdown_connections 2024-04-25T10:12:20.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:20.893+0000 7f1df3200640 1 -- 172.21.15.87:0/1071400648 wait complete. 2024-04-25T10:12:20.915 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:12:20.916 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:12:21.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.049+0000 7f04bcc00640 1 Processor -- start 2024-04-25T10:12:21.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.049+0000 7f04bcc00640 1 -- start start 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04bcc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b806bf80 0x7f04b80d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04bcc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f04b80d0c70 0x7f04b815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04bcc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b806aee0 con 0x7f04b81619a0 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04bcc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b806ac90 con 0x7f04b80d0c70 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04bcc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b806b1c0 con 0x7f04b806bf80 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04b6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f04b80d0c70 0x7f04b815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04b6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f04b80d0c70 0x7f04b815e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45904/0 (socket says 172.21.15.87:45904) 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04b6800640 1 -- 172.21.15.87:0/3295458466 learned_addr learned my addr 172.21.15.87:0/3295458466 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04ad400640 1 --2- 172.21.15.87:0/3295458466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b806bf80 0x7f04b80d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04b6800640 1 -- 172.21.15.87:0/3295458466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b806bf80 msgr2=0x7f04b80d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04b6800640 1 --2- 172.21.15.87:0/3295458466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b806bf80 0x7f04b80d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04b6800640 1 -- 172.21.15.87:0/3295458466 >> v1:172.21.15.169:6789/0 conn(0x7f04b81619a0 legacy=0x7f04b8161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.053+0000 7f04b6800640 1 -- 172.21.15.87:0/3295458466 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b806b4a0 con 0x7f04b80d0c70 2024-04-25T10:12:21.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04b6800640 1 --2- 172.21.15.87:0/3295458466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b80d0c70 0x7f04b815e1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009870 tx=0x7f04b001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6137f45cbc39a434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04b5400640 1 -- 172.21.15.87:0/3295458466 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00a95f0 con 0x7f04b80d0c70 2024-04-25T10:12:21.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04b5400640 1 -- 172.21.15.87:0/3295458466 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b00a9790 con 0x7f04b80d0c70 2024-04-25T10:12:21.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04b5400640 1 -- 172.21.15.87:0/3295458466 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00a9b60 con 0x7f04b80d0c70 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- 172.21.15.87:0/3295458466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b80d0c70 msgr2=0x7f04b815e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 --2- 172.21.15.87:0/3295458466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b80d0c70 0x7f04b815e1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009870 tx=0x7f04b001c530 comp rx=0 tx=0).stop 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- 172.21.15.87:0/3295458466 shutdown_connections 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 --2- 172.21.15.87:0/3295458466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b80d0c70 0x7f04b815e1c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 --2- 172.21.15.87:0/3295458466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b806bf80 0x7f04b80d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- 172.21.15.87:0/3295458466 >> 172.21.15.87:0/3295458466 conn(0x7f04b80b34a0 msgr2=0x7f04b80b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- 172.21.15.87:0/3295458466 shutdown_connections 2024-04-25T10:12:21.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- 172.21.15.87:0/3295458466 wait complete. 2024-04-25T10:12:21.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 Processor -- start 2024-04-25T10:12:21.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- start start 2024-04-25T10:12:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b806bf80 0x7f04b8148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b80d0c70 0x7f04b80cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b81619a0 0x7f04b80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b815f2a0 con 0x7f04b81619a0 2024-04-25T10:12:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b815f120 con 0x7f04b806bf80 2024-04-25T10:12:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.057+0000 7f04bcc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b816ace0 con 0x7f04b80d0c70 2024-04-25T10:12:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b81619a0 0x7f04b80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b81619a0 0x7f04b80d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45910/0 (socket says 172.21.15.87:45910) 2024-04-25T10:12:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 -- 172.21.15.87:0/2061280792 learned_addr learned my addr 172.21.15.87:0/2061280792 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:21.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04ad400640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b80d0c70 0x7f04b80cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b5e00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b806bf80 0x7f04b8148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 -- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b80d0c70 msgr2=0x7f04b80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b80d0c70 0x7f04b80cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 -- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b806bf80 msgr2=0x7f04b8148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b806bf80 0x7f04b8148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 -- 172.21.15.87:0/2061280792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b8149df0 con 0x7f04b81619a0 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04b6800640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b81619a0 0x7f04b80d0160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f04b001a160 tx=0x7f04b001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04aea00640 1 -- 172.21.15.87:0/2061280792 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00b1800 con 0x7f04b81619a0 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04aea00640 1 -- 172.21.15.87:0/2061280792 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b00b19a0 con 0x7f04b81619a0 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04aea00640 1 -- 172.21.15.87:0/2061280792 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00b1d90 con 0x7f04b81619a0 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b814a0d0 con 0x7f04b81619a0 2024-04-25T10:12:21.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b80c8ea0 con 0x7f04b81619a0 2024-04-25T10:12:21.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b804ef30 con 0x7f04b81619a0 2024-04-25T10:12:21.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.061+0000 7f04aea00640 1 -- 172.21.15.87:0/2061280792 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04b00bc450 con 0x7f04b81619a0 2024-04-25T10:12:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.065+0000 7f04aea00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0494066f60 0x7f0494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.065+0000 7f04aea00640 1 -- 172.21.15.87:0/2061280792 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04b011f7b0 con 0x7f04b81619a0 2024-04-25T10:12:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.065+0000 7f04b5e00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0494066f60 0x7f0494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.065+0000 7f04b5e00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0494066f60 0x7f0494069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a8001cb0 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.069+0000 7f04aea00640 1 -- 172.21.15.87:0/2061280792 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04b00ad020 con 0x7f04b81619a0 2024-04-25T10:12:21.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:21.236+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f04b814a2a0 con 0x7f04b81619a0 2024-04-25T10:12:21.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:21.456 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:22.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04aea00640 1 -- 172.21.15.87:0/2061280792 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f04b0016150 con 0x7f04b81619a0 2024-04-25T10:12:22.216 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0494066f60 msgr2=0x7f0494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0494066f60 0x7f0494069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a8001cb0 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b81619a0 msgr2=0x7f04b80d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b81619a0 0x7f04b80d0160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f04b001a160 tx=0x7f04b001ad70 comp rx=0 tx=0).stop 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 shutdown_connections 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0494066f60 0x7f0494069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04b81619a0 0x7f04b80d0160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04b80d0c70 0x7f04b80cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 --2- 172.21.15.87:0/2061280792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04b806bf80 0x7f04b8148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 >> 172.21.15.87:0/2061280792 conn(0x7f04b80b34a0 msgr2=0x7f04b8153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 shutdown_connections 2024-04-25T10:12:22.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.212+0000 7f04bcc00640 1 -- 172.21.15.87:0/2061280792 wait complete. 2024-04-25T10:12:22.249 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:12:22.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.388+0000 7ff476600640 1 Processor -- start 2024-04-25T10:12:22.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.388+0000 7ff476600640 1 -- start start 2024-04-25T10:12:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.388+0000 7ff476600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700af7c0 0x7ff470071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.388+0000 7ff476600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4700670b0 0x7ff4701618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.388+0000 7ff476600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47015e3c0 con 0x7ff47015b800 2024-04-25T10:12:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.388+0000 7ff476600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4700afc50 con 0x7ff4700670b0 2024-04-25T10:12:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.388+0000 7ff476600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47015ecf0 con 0x7ff4700af7c0 2024-04-25T10:12:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff47015b800 0x7ff4700669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34904/0 (socket says 172.21.15.87:34904) 2024-04-25T10:12:22.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 -- 172.21.15.87:0/1841818373 learned_addr learned my addr 172.21.15.87:0/1841818373 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:22.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff467e00640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700af7c0 0x7ff470071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46fe00640 1 --2- 172.21.15.87:0/1841818373 >> v2:172.21.15.169:3300/0 conn(0x7ff4700670b0 0x7ff4701618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175128015 0 0) 0x7ff47015e3c0 con 0x7ff47015b800 2024-04-25T10:12:22.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff454003530 con 0x7ff47015b800 2024-04-25T10:12:22.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353835521 0 0) 0x7ff454003530 con 0x7ff47015b800 2024-04-25T10:12:22.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700af7c0 msgr2=0x7ff470071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700af7c0 0x7ff470071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 >> v2:172.21.15.169:3300/0 conn(0x7ff4700670b0 msgr2=0x7ff4701618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 --2- 172.21.15.87:0/1841818373 >> v2:172.21.15.169:3300/0 conn(0x7ff4700670b0 0x7ff4701618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47015e6a0 con 0x7ff47015b800 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ff460001680 con 0x7ff47015b800 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 >> v1:172.21.15.87:6789/0 conn(0x7ff47015b800 legacy=0x7ff4700669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4540039c0 0x7ff454003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff454004810 0x7ff454004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4540051d0 0x7ff45400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700afc50 con 0x7ff454004810 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47015ecf0 con 0x7ff4540039c0 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45400bf10 con 0x7ff4540051d0 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4540039c0 0x7ff454003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff467e00640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff454004810 0x7ff454004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46fe00640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4540051d0 0x7ff45400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 -- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4540051d0 msgr2=0x7ff45400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4540051d0 0x7ff45400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 -- 172.21.15.87:0/1841818373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff454004810 msgr2=0x7ff454004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff454004810 0x7ff454004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.392+0000 7ff46f400640 1 -- 172.21.15.87:0/1841818373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff454003530 con 0x7ff4540039c0 2024-04-25T10:12:22.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff46f400640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4540039c0 0x7ff454003de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c2f0 tx=0x7ff46001f490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b1671bf70cd3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460007d60 con 0x7ff4540039c0 2024-04-25T10:12:22.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460005b00 con 0x7ff4540039c0 2024-04-25T10:12:22.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff46ea00640 1 -- 172.21.15.87:0/1841818373 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460007070 con 0x7ff4540039c0 2024-04-25T10:12:22.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 -- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4540039c0 msgr2=0x7ff454003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 --2- 172.21.15.87:0/1841818373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4540039c0 0x7ff454003de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c2f0 tx=0x7ff46001f490 comp rx=0 tx=0).stop 2024-04-25T10:12:22.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff46fe00640 1 -- 172.21.15.87:0/1841818373 reap_dead start 2024-04-25T10:12:22.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 -- 172.21.15.87:0/1841818373 shutdown_connections 2024-04-25T10:12:22.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 -- 172.21.15.87:0/1841818373 >> 172.21.15.87:0/1841818373 conn(0x7ff4700d0820 msgr2=0x7ff4700d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 -- 172.21.15.87:0/1841818373 shutdown_connections 2024-04-25T10:12:22.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 -- 172.21.15.87:0/1841818373 wait complete. 2024-04-25T10:12:22.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 Processor -- start 2024-04-25T10:12:22.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.396+0000 7ff476600640 1 -- start start 2024-04-25T10:12:22.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700cf8c0 0x7ff4700cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff470147ad0 0x7ff470147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4701561d0 0x7ff470156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700b00a0 con 0x7ff4701561d0 2024-04-25T10:12:22.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47015f180 con 0x7ff470147ad0 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47015bbe0 con 0x7ff4700cf8c0 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4701561d0 0x7ff470156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4701561d0 0x7ff470156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45914/0 (socket says 172.21.15.87:45914) 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 -- 172.21.15.87:0/4055696647 learned_addr learned my addr 172.21.15.87:0/4055696647 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46f400640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700cf8c0 0x7ff4700cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff467e00640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff470147ad0 0x7ff470147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 -- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700cf8c0 msgr2=0x7ff4700cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700cf8c0 0x7ff4700cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 -- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff470147ad0 msgr2=0x7ff470147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff470147ad0 0x7ff470147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 -- 172.21.15.87:0/4055696647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4700d01e0 con 0x7ff4701561d0 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff467e00640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff470147ad0 0x7ff470147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff46fe00640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4701561d0 0x7ff470156650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4700c9e40 tx=0x7ff46800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff467400640 1 -- 172.21.15.87:0/4055696647 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800fe10 con 0x7ff4701561d0 2024-04-25T10:12:22.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff467400640 1 -- 172.21.15.87:0/4055696647 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46801d070 con 0x7ff4701561d0 2024-04-25T10:12:22.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff470156f00 con 0x7ff4701561d0 2024-04-25T10:12:22.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff467400640 1 -- 172.21.15.87:0/4055696647 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468015670 con 0x7ff4701561d0 2024-04-25T10:12:22.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff470144930 con 0x7ff4701561d0 2024-04-25T10:12:22.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47004ef30 con 0x7ff4701561d0 2024-04-25T10:12:22.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.400+0000 7ff467400640 1 -- 172.21.15.87:0/4055696647 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff468015810 con 0x7ff4701561d0 2024-04-25T10:12:22.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.404+0000 7ff467400640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff44c066f60 0x7ff44c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.404+0000 7ff467400640 1 -- 172.21.15.87:0/4055696647 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff468085710 con 0x7ff4701561d0 2024-04-25T10:12:22.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.404+0000 7ff46f400640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff44c066f60 0x7ff44c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.404+0000 7ff46f400640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff44c066f60 0x7ff44c069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4600ae000 tx=0x7ff460007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.408+0000 7ff467400640 1 -- 172.21.15.87:0/4055696647 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff468010040 con 0x7ff4701561d0 2024-04-25T10:12:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:22.560+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff4700ca170 con 0x7ff4701561d0 2024-04-25T10:12:23.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.216+0000 7ff467400640 1 -- 172.21.15.87:0/4055696647 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff468056860 con 0x7ff4701561d0 2024-04-25T10:12:23.222 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:12:23.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff44c066f60 msgr2=0x7ff44c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff44c066f60 0x7ff44c069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4600ae000 tx=0x7ff460007040 comp rx=0 tx=0).stop 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4701561d0 msgr2=0x7ff470156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4701561d0 0x7ff470156650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4700c9e40 tx=0x7ff46800f580 comp rx=0 tx=0).stop 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 shutdown_connections 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff44c066f60 0x7ff44c069420 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4600ae000 tx=0x7ff460007040 comp rx=0 tx=0).stop 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4701561d0 0x7ff470156650 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff470147ad0 0x7ff470147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 --2- 172.21.15.87:0/4055696647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4700cf8c0 0x7ff4700cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 >> 172.21.15.87:0/4055696647 conn(0x7ff4700d0820 msgr2=0x7ff470160d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 shutdown_connections 2024-04-25T10:12:23.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.220+0000 7ff476600640 1 -- 172.21.15.87:0/4055696647 wait complete. 2024-04-25T10:12:23.270 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:12:23.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb490a00640 1 Processor -- start 2024-04-25T10:12:23.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb490a00640 1 -- start start 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb490a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c070350 0x7fb48c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb490a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c15b080 con 0x7fb48c0aef80 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb490a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c15adb0 con 0x7fb48c0b31c0 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb490a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c15eb60 con 0x7fb48c070350 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb489e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb48c0aef80 0x7fb48c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34920/0 (socket says 172.21.15.87:34920) 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb489e00640 1 -- 172.21.15.87:0/395091512 learned_addr learned my addr 172.21.15.87:0/395091512 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb483e00640 1 --2- 172.21.15.87:0/395091512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c070350 0x7fb48c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354681171 0 0) 0x7fb48c15b080 con 0x7fb48c0aef80 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb470005740 con 0x7fb48c0aef80 2024-04-25T10:12:23.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565922817 0 0) 0x7fb48c15adb0 con 0x7fb48c0b31c0 2024-04-25T10:12:23.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb48c15b080 con 0x7fb48c0b31c0 2024-04-25T10:12:23.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822531960 0 0) 0x7fb470005740 con 0x7fb48c0aef80 2024-04-25T10:12:23.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.404+0000 7fb483e00640 1 -- 172.21.15.87:0/395091512 >> v1:172.21.15.169:6789/0 conn(0x7fb48c0b31c0 legacy=0x7fb48c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb483e00640 1 -- 172.21.15.87:0/395091512 >> v1:172.21.15.87:6789/0 conn(0x7fb48c0aef80 legacy=0x7fb48c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb483e00640 1 -- 172.21.15.87:0/395091512 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c15ee40 con 0x7fb48c070350 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb483e00640 1 --2- 172.21.15.87:0/395091512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c070350 0x7fb48c15e480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097f0 tx=0x7fb47401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6164e5c8c8ec434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740b0070 con 0x7fb48c070350 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740ab400 con 0x7fb48c070350 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb489400640 1 -- 172.21.15.87:0/395091512 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c001680 con 0x7fb48c070350 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb490a00640 1 -- 172.21.15.87:0/395091512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c070350 msgr2=0x7fb48c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb490a00640 1 --2- 172.21.15.87:0/395091512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c070350 0x7fb48c15e480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097f0 tx=0x7fb47401c400 comp rx=0 tx=0).stop 2024-04-25T10:12:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb490a00640 1 -- 172.21.15.87:0/395091512 shutdown_connections 2024-04-25T10:12:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb490a00640 1 --2- 172.21.15.87:0/395091512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c070350 0x7fb48c15e480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb490a00640 1 -- 172.21.15.87:0/395091512 >> 172.21.15.87:0/395091512 conn(0x7fb48c0669a0 msgr2=0x7fb48c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.408+0000 7fb490a00640 1 -- 172.21.15.87:0/395091512 shutdown_connections 2024-04-25T10:12:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 -- 172.21.15.87:0/395091512 wait complete. 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 Processor -- start 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 -- start start 2024-04-25T10:12:23.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb48c070350 0x7fb48c13cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c0aef80 0x7fb48c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c0b31c0 0x7fb48c1519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0cfef0 con 0x7fb48c070350 2024-04-25T10:12:23.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0cfd70 con 0x7fb48c0b31c0 2024-04-25T10:12:23.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b35d0 con 0x7fb48c0aef80 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb48c070350 0x7fb48c13cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb48c070350 0x7fb48c13cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45916/0 (socket says 172.21.15.87:45916) 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 -- 172.21.15.87:0/4065221797 learned_addr learned my addr 172.21.15.87:0/4065221797 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb48a800640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c0b31c0 0x7fb48c1519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb483e00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c0aef80 0x7fb48c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 -- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c0aef80 msgr2=0x7fb48c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c0aef80 0x7fb48c159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 -- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c0b31c0 msgr2=0x7fb48c1519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c0b31c0 0x7fb48c1519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 -- 172.21.15.87:0/4065221797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c156240 con 0x7fb48c070350 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb483e00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c0aef80 0x7fb48c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb48a800640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c0b31c0 0x7fb48c1519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb489e00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb48c070350 0x7fb48c13cc60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00c780 tx=0x7fb47c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb482000640 1 -- 172.21.15.87:0/4065221797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c017070 con 0x7fb48c070350 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb482000640 1 -- 172.21.15.87:0/4065221797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c005d10 con 0x7fb48c070350 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c156520 con 0x7fb48c070350 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb482000640 1 -- 172.21.15.87:0/4065221797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0072b0 con 0x7fb48c070350 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.412+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c156ae0 con 0x7fb48c070350 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.416+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c04ef30 con 0x7fb48c070350 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.416+0000 7fb482000640 1 -- 172.21.15.87:0/4065221797 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb47c01a070 con 0x7fb48c070350 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.416+0000 7fb482000640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb468066f60 0x7fb468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.416+0000 7fb482000640 1 -- 172.21.15.87:0/4065221797 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb47c0846c0 con 0x7fb48c070350 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.416+0000 7fb483e00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb468066f60 0x7fb468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.416+0000 7fb483e00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb468066f60 0x7fb468069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4740ad000 tx=0x7fb474008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.420+0000 7fb482000640 1 -- 172.21.15.87:0/4065221797 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb47c055690 con 0x7fb48c070350 2024-04-25T10:12:23.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:23.608+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb48c15b620 con 0x7fb48c070350 2024-04-25T10:12:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.220+0000 7fb482000640 1 -- 172.21.15.87:0/4065221797 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb47c054f90 con 0x7fb48c070350 2024-04-25T10:12:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.220+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb468066f60 msgr2=0x7fb468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.220+0000 7fb490a00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb468066f60 0x7fb468069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4740ad000 tx=0x7fb474008040 comp rx=0 tx=0).stop 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.220+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb48c070350 msgr2=0x7fb48c13cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.220+0000 7fb490a00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb48c070350 0x7fb48c13cc60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00c780 tx=0x7fb47c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 shutdown_connections 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb468066f60 0x7fb468069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c0b31c0 0x7fb48c1519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb48c0aef80 0x7fb48c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 --2- 172.21.15.87:0/4065221797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb48c070350 0x7fb48c13cc60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 >> 172.21.15.87:0/4065221797 conn(0x7fb48c0669a0 msgr2=0x7fb48c0d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 shutdown_connections 2024-04-25T10:12:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.224+0000 7fb490a00640 1 -- 172.21.15.87:0/4065221797 wait complete. 2024-04-25T10:12:24.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:24.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.412+0000 7f3035400640 1 Processor -- start 2024-04-25T10:12:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.412+0000 7f3035400640 1 -- start start 2024-04-25T10:12:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f3035400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f303015b9b0 0x7f30300ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f3035400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30300b5fb0 0x7f30300ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f3035400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30300cd120 con 0x7f30300ce700 2024-04-25T10:12:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f3035400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30300d2600 con 0x7f303015b9b0 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f3035400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30300cd400 con 0x7f30300b5fb0 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f303015b9b0 0x7f30300ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f303015b9b0 0x7f30300ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45932/0 (socket says 172.21.15.87:45932) 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 -- 172.21.15.87:0/61924638 learned_addr learned my addr 172.21.15.87:0/61924638 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f3034a00640 1 --2- 172.21.15.87:0/61924638 >> v2:172.21.15.169:3300/0 conn(0x7f30300b5fb0 0x7f30300ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 -- 172.21.15.87:0/61924638 >> v2:172.21.15.169:3300/0 conn(0x7f30300b5fb0 msgr2=0x7f30300ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 --2- 172.21.15.87:0/61924638 >> v2:172.21.15.169:3300/0 conn(0x7f30300b5fb0 0x7f30300ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 -- 172.21.15.87:0/61924638 >> v1:172.21.15.169:6789/0 conn(0x7f30300ce700 legacy=0x7f30300cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 -- 172.21.15.87:0/61924638 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300b47a0 con 0x7f303015b9b0 2024-04-25T10:12:24.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.416+0000 7f302fe00640 1 --2- 172.21.15.87:0/61924638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300ce1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30300cbd90 tx=0x7f302001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d4e698418c1039 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f302f400640 1 -- 172.21.15.87:0/61924638 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30200b0070 con 0x7f303015b9b0 2024-04-25T10:12:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f302f400640 1 -- 172.21.15.87:0/61924638 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30200ab600 con 0x7f303015b9b0 2024-04-25T10:12:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f302f400640 1 -- 172.21.15.87:0/61924638 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30200ab9d0 con 0x7f303015b9b0 2024-04-25T10:12:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- 172.21.15.87:0/61924638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 msgr2=0x7f30300ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 --2- 172.21.15.87:0/61924638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300ce1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30300cbd90 tx=0x7f302001c520 comp rx=0 tx=0).stop 2024-04-25T10:12:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- 172.21.15.87:0/61924638 shutdown_connections 2024-04-25T10:12:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 --2- 172.21.15.87:0/61924638 >> v2:172.21.15.169:3300/0 conn(0x7f30300b5fb0 0x7f30300ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 --2- 172.21.15.87:0/61924638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300ce1c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- 172.21.15.87:0/61924638 >> 172.21.15.87:0/61924638 conn(0x7f30300d17b0 msgr2=0x7f30300d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- 172.21.15.87:0/61924638 shutdown_connections 2024-04-25T10:12:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- 172.21.15.87:0/61924638 wait complete. 2024-04-25T10:12:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 Processor -- start 2024-04-25T10:12:24.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- start start 2024-04-25T10:12:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30300b5fb0 0x7f3030158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30300ce700 0x7f30301525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3030168760 con 0x7f303015b9b0 2024-04-25T10:12:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30301685e0 con 0x7f30300ce700 2024-04-25T10:12:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3035400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30301688e0 con 0x7f30300b5fb0 2024-04-25T10:12:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3034a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3034a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45946/0 (socket says 172.21.15.87:45946) 2024-04-25T10:12:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3034a00640 1 -- 172.21.15.87:0/1007870536 learned_addr learned my addr 172.21.15.87:0/1007870536 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f302fe00640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30300b5fb0 0x7f3030158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.420+0000 7f3027400640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30300ce700 0x7f30301525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3034a00640 1 -- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30300b5fb0 msgr2=0x7f3030158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3034a00640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30300b5fb0 0x7f3030158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3034a00640 1 -- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30300ce700 msgr2=0x7f30301525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3034a00640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30300ce700 0x7f30301525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3034a00640 1 -- 172.21.15.87:0/1007870536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30301593b0 con 0x7f303015b9b0 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3034a00640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300c8880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3028007e70 tx=0x7f302800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f302cc00640 1 -- 172.21.15.87:0/1007870536 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028014620 con 0x7f303015b9b0 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f302cc00640 1 -- 172.21.15.87:0/1007870536 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3028019070 con 0x7f303015b9b0 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f302cc00640 1 -- 172.21.15.87:0/1007870536 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028014a20 con 0x7f303015b9b0 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3030159690 con 0x7f303015b9b0 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3030159c50 con 0x7f303015b9b0 2024-04-25T10:12:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3030075120 con 0x7f303015b9b0 2024-04-25T10:12:24.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f302cc00640 1 -- 172.21.15.87:0/1007870536 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3028014bc0 con 0x7f303015b9b0 2024-04-25T10:12:24.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f302cc00640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f300c066f60 0x7f300c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f302cc00640 1 -- 172.21.15.87:0/1007870536 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3028085860 con 0x7f303015b9b0 2024-04-25T10:12:24.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.424+0000 7f302fe00640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f300c066f60 0x7f300c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.428+0000 7f302fe00640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f300c066f60 0x7f300c069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30300c51a0 tx=0x7f3020008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.428+0000 7f302cc00640 1 -- 172.21.15.87:0/1007870536 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3028056180 con 0x7f303015b9b0 2024-04-25T10:12:24.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.584+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30300cd8c0 con 0x7f303015b9b0 2024-04-25T10:12:24.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.584+0000 7f302cc00640 1 -- 172.21.15.87:0/1007870536 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16606 (secure 0 0 0) 0x7f302805a220 con 0x7f303015b9b0 2024-04-25T10:12:24.591 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:24.591 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":19,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:23.612192+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:24.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f300c066f60 msgr2=0x7f300c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f300c066f60 0x7f300c069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30300c51a0 tx=0x7f3020008040 comp rx=0 tx=0).stop 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 msgr2=0x7f30300c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300c8880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3028007e70 tx=0x7f302800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 shutdown_connections 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f300c066f60 0x7f300c069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f303015b9b0 0x7f30300c8880 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30300ce700 0x7f30301525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 --2- 172.21.15.87:0/1007870536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30300b5fb0 0x7f3030158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 >> 172.21.15.87:0/1007870536 conn(0x7f30300d17b0 msgr2=0x7f303006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 shutdown_connections 2024-04-25T10:12:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.588+0000 7f3035400640 1 -- 172.21.15.87:0/1007870536 wait complete. 2024-04-25T10:12:24.612 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:24.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.740+0000 7fca8b200640 1 Processor -- start 2024-04-25T10:12:24.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.740+0000 7fca8b200640 1 -- start start 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca8b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca8c15db40 0x7fca8c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca8b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca8c0b5010 0x7fca8c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca8b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c15e460 con 0x7fca8c06b2c0 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca8b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0b5780 con 0x7fca8c15db40 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca8b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0b5a60 con 0x7fca8c0b5010 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca89e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca8c15db40 0x7fca8c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca89e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca8c15db40 0x7fca8c06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45950/0 (socket says 172.21.15.87:45950) 2024-04-25T10:12:24.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca89e00640 1 -- 172.21.15.87:0/312503424 learned_addr learned my addr 172.21.15.87:0/312503424 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:24.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca89e00640 1 -- 172.21.15.87:0/312503424 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0b5010 msgr2=0x7fca8c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca81400640 1 --2- 172.21.15.87:0/312503424 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0b5010 0x7fca8c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca89e00640 1 --2- 172.21.15.87:0/312503424 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0b5010 0x7fca8c15df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca89e00640 1 -- 172.21.15.87:0/312503424 >> v1:172.21.15.169:6789/0 conn(0x7fca8c06b2c0 legacy=0x7fca8c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca89e00640 1 -- 172.21.15.87:0/312503424 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0b5d40 con 0x7fca8c15db40 2024-04-25T10:12:24.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.744+0000 7fca81400640 1 --2- 172.21.15.87:0/312503424 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0b5010 0x7fca8c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:24.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca89e00640 1 --2- 172.21.15.87:0/312503424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c06ac90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca8c068400 tx=0x7fca7c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01eab655a95e1cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca89400640 1 -- 172.21.15.87:0/312503424 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b5070 con 0x7fca8c15db40 2024-04-25T10:12:24.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca89400640 1 -- 172.21.15.87:0/312503424 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c0a89d0 con 0x7fca8c15db40 2024-04-25T10:12:24.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca89400640 1 -- 172.21.15.87:0/312503424 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0a8da0 con 0x7fca8c15db40 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- 172.21.15.87:0/312503424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 msgr2=0x7fca8c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 --2- 172.21.15.87:0/312503424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c06ac90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca8c068400 tx=0x7fca7c01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- 172.21.15.87:0/312503424 shutdown_connections 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 --2- 172.21.15.87:0/312503424 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0b5010 0x7fca8c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 --2- 172.21.15.87:0/312503424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c06ac90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- 172.21.15.87:0/312503424 >> 172.21.15.87:0/312503424 conn(0x7fca8c0d17b0 msgr2=0x7fca8c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- 172.21.15.87:0/312503424 shutdown_connections 2024-04-25T10:12:24.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- 172.21.15.87:0/312503424 wait complete. 2024-04-25T10:12:24.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 Processor -- start 2024-04-25T10:12:24.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- start start 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c06b2c0 0x7fca8c1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c0b5010 0x7fca8c0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b5480 con 0x7fca8c15db40 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b6300 con 0x7fca8c0b5010 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.748+0000 7fca8b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c06b6f0 con 0x7fca8c06b2c0 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45956/0 (socket says 172.21.15.87:45956) 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 -- 172.21.15.87:0/3913827297 learned_addr learned my addr 172.21.15.87:0/3913827297 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca89e00640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c06b2c0 0x7fca8c1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca81400640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c0b5010 0x7fca8c0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 -- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c06b2c0 msgr2=0x7fca8c1f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c06b2c0 0x7fca8c1f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 -- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c0b5010 msgr2=0x7fca8c0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c0b5010 0x7fca8c0cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 -- 172.21.15.87:0/3913827297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c1f6dc0 con 0x7fca8c15db40 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8a800640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c149b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c780 tx=0x7fca8400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca82a00640 1 -- 172.21.15.87:0/3913827297 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84004b70 con 0x7fca8c15db40 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca82a00640 1 -- 172.21.15.87:0/3913827297 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca84004d10 con 0x7fca8c15db40 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca82a00640 1 -- 172.21.15.87:0/3913827297 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400aa90 con 0x7fca8c15db40 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c1f70a0 con 0x7fca8c15db40 2024-04-25T10:12:24.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c1f7660 con 0x7fca8c15db40 2024-04-25T10:12:24.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c075120 con 0x7fca8c15db40 2024-04-25T10:12:24.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca82a00640 1 -- 172.21.15.87:0/3913827297 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca8400ac30 con 0x7fca8c15db40 2024-04-25T10:12:24.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca82a00640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca68066f60 0x7fca68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.752+0000 7fca82a00640 1 -- 172.21.15.87:0/3913827297 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca84085220 con 0x7fca8c15db40 2024-04-25T10:12:24.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.756+0000 7fca89e00640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca68066f60 0x7fca68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.756+0000 7fca89e00640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca68066f60 0x7fca68069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0b2000 tx=0x7fca7c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.756+0000 7fca82a00640 1 -- 172.21.15.87:0/3913827297 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca84020070 con 0x7fca8c15db40 2024-04-25T10:12:24.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca8c0b3750 con 0x7fca8c15db40 2024-04-25T10:12:24.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca82a00640 1 -- 172.21.15.87:0/3913827297 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16606 (secure 0 0 0) 0x7fca84055b40 con 0x7fca8c15db40 2024-04-25T10:12:24.917 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:24.917 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":19,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:23.612192+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:24.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca68066f60 msgr2=0x7fca68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca68066f60 0x7fca68069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0b2000 tx=0x7fca7c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:24.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 msgr2=0x7fca8c149b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c149b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c780 tx=0x7fca8400cc40 comp rx=0 tx=0).stop 2024-04-25T10:12:24.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 shutdown_connections 2024-04-25T10:12:24.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca68066f60 0x7fca68069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c15db40 0x7fca8c149b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c0b5010 0x7fca8c0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 --2- 172.21.15.87:0/3913827297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c06b2c0 0x7fca8c1f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 >> 172.21.15.87:0/3913827297 conn(0x7fca8c0d17b0 msgr2=0x7fca8c06f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 shutdown_connections 2024-04-25T10:12:24.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:24.912+0000 7fca8b200640 1 -- 172.21.15.87:0/3913827297 wait complete. 2024-04-25T10:12:24.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:12:25.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11d5800640 1 Processor -- start 2024-04-25T10:12:25.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11d5800640 1 -- start start 2024-04-25T10:12:25.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11d5800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f11d015dcc0 0x7f11d0162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11d5800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f11d00b5480 0x7f11d00b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11d5800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d015f0b0 con 0x7f11d00669a0 2024-04-25T10:12:25.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11d5800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d0161800 con 0x7f11d015dcc0 2024-04-25T10:12:25.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11d5800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d00b3b20 con 0x7f11d00b5480 2024-04-25T10:12:25.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11cf400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f11d00669a0 0x7f11d00d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34934/0 (socket says 172.21.15.87:34934) 2024-04-25T10:12:25.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11cf400640 1 -- 172.21.15.87:0/516764766 learned_addr learned my addr 172.21.15.87:0/516764766 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:25.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11cea00640 1 --2- 172.21.15.87:0/516764766 >> v2:172.21.15.87:3300/0 conn(0x7f11d015dcc0 0x7f11d0162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11c7e00640 1 --2- 172.21.15.87:0/516764766 >> v2:172.21.15.169:3300/0 conn(0x7f11d00b5480 0x7f11d00b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11ce000640 1 -- 172.21.15.87:0/516764766 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032457708 0 0) 0x7f11d015f0b0 con 0x7f11d00669a0 2024-04-25T10:12:25.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11cea00640 1 -- 172.21.15.87:0/516764766 >> v2:172.21.15.169:3300/0 conn(0x7f11d00b5480 msgr2=0x7f11d00b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:25.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11cea00640 1 --2- 172.21.15.87:0/516764766 >> v2:172.21.15.169:3300/0 conn(0x7f11d00b5480 0x7f11d00b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11cea00640 1 -- 172.21.15.87:0/516764766 >> v1:172.21.15.87:6789/0 conn(0x7f11d00669a0 legacy=0x7f11d00d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.072+0000 7f11cea00640 1 -- 172.21.15.87:0/516764766 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00b3e00 con 0x7f11d015dcc0 2024-04-25T10:12:25.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11cea00640 1 --2- 172.21.15.87:0/516764766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d015dcc0 0x7f11d0162250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11d006bce0 tx=0x7f11c001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720748bbb054b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:25.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11ce000640 1 -- 172.21.15.87:0/516764766 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00ab650 con 0x7f11d015dcc0 2024-04-25T10:12:25.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11ce000640 1 -- 172.21.15.87:0/516764766 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c00b0070 con 0x7f11d015dcc0 2024-04-25T10:12:25.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11ce000640 1 -- 172.21.15.87:0/516764766 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00aba10 con 0x7f11d015dcc0 2024-04-25T10:12:25.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 -- 172.21.15.87:0/516764766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d015dcc0 msgr2=0x7f11d0162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 --2- 172.21.15.87:0/516764766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d015dcc0 0x7f11d0162250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11d006bce0 tx=0x7f11c001c450 comp rx=0 tx=0).stop 2024-04-25T10:12:25.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 -- 172.21.15.87:0/516764766 shutdown_connections 2024-04-25T10:12:25.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 --2- 172.21.15.87:0/516764766 >> v2:172.21.15.169:3300/0 conn(0x7f11d00b5480 0x7f11d00b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 --2- 172.21.15.87:0/516764766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d015dcc0 0x7f11d0162250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 -- 172.21.15.87:0/516764766 >> 172.21.15.87:0/516764766 conn(0x7f11d006ac90 msgr2=0x7f11d006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:25.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 -- 172.21.15.87:0/516764766 shutdown_connections 2024-04-25T10:12:25.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.076+0000 7f11d5800640 1 -- 172.21.15.87:0/516764766 wait complete. 2024-04-25T10:12:25.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 Processor -- start 2024-04-25T10:12:25.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 -- start start 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d00669a0 0x7f11d0148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d00b5480 0x7f11d0155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d015dcc0 0x7f11d0149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00b3900 con 0x7f11d00b5480 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00b3780 con 0x7f11d00669a0 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00d1050 con 0x7f11d015dcc0 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d00b5480 0x7f11d0155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d00b5480 0x7f11d0155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45966/0 (socket says 172.21.15.87:45966) 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 -- 172.21.15.87:0/2706503 learned_addr learned my addr 172.21.15.87:0/2706503 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 -- 172.21.15.87:0/2706503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d015dcc0 msgr2=0x7f11d0149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d015dcc0 0x7f11d0149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 -- 172.21.15.87:0/2706503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d00669a0 msgr2=0x7f11d0148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11cea00640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d00669a0 0x7f11d0148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d00669a0 0x7f11d0148b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 -- 172.21.15.87:0/2706503 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d014ceb0 con 0x7f11d00b5480 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c7e00640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d00b5480 0x7f11d0155770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11b800d830 tx=0x7f11b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c6a00640 1 -- 172.21.15.87:0/2706503 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800b6c0 con 0x7f11d00b5480 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c6a00640 1 -- 172.21.15.87:0/2706503 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8019070 con 0x7f11d00b5480 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11c6a00640 1 -- 172.21.15.87:0/2706503 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800ba60 con 0x7f11d00b5480 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d0150ac0 con 0x7f11d00b5480 2024-04-25T10:12:25.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.080+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d0151080 con 0x7f11d00b5480 2024-04-25T10:12:25.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.084+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d004ef30 con 0x7f11d00b5480 2024-04-25T10:12:25.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.084+0000 7f11c6a00640 1 -- 172.21.15.87:0/2706503 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11b801e460 con 0x7f11d00b5480 2024-04-25T10:12:25.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.084+0000 7f11c6a00640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11ac066f60 0x7f11ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.084+0000 7f11c6a00640 1 -- 172.21.15.87:0/2706503 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11b80856e0 con 0x7f11d00b5480 2024-04-25T10:12:25.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.084+0000 7f11cea00640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11ac066f60 0x7f11ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.084+0000 7f11cea00640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11ac066f60 0x7f11ac069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f11c00af000 tx=0x7f11c00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:25.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.092+0000 7f11c6a00640 1 -- 172.21.15.87:0/2706503 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11b8056780 con 0x7f11d00b5480 2024-04-25T10:12:25.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:25.284+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f11d0168970 con 0x7f11d00b5480 2024-04-25T10:12:26.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.248+0000 7f11c6a00640 1 -- 172.21.15.87:0/2706503 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f11b8056080 con 0x7f11d00b5480 2024-04-25T10:12:26.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11ac066f60 msgr2=0x7f11ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11ac066f60 0x7f11ac069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f11c00af000 tx=0x7f11c00a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:26.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d00b5480 msgr2=0x7f11d0155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d00b5480 0x7f11d0155770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11b800d830 tx=0x7f11b800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 shutdown_connections 2024-04-25T10:12:26.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11ac066f60 0x7f11ac069420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d015dcc0 0x7f11d0149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11d00b5480 0x7f11d0155770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 --2- 172.21.15.87:0/2706503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d00669a0 0x7f11d0148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 >> 172.21.15.87:0/2706503 conn(0x7f11d006ac90 msgr2=0x7f11d0152e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:26.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 shutdown_connections 2024-04-25T10:12:26.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.252+0000 7f11d5800640 1 -- 172.21.15.87:0/2706503 wait complete. 2024-04-25T10:12:26.281 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:12:26.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:12:26.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.416+0000 7f1708600640 1 Processor -- start 2024-04-25T10:12:26.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.416+0000 7f1708600640 1 -- start start 2024-04-25T10:12:26.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.416+0000 7f1708600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1700070350 0x7f170015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.416+0000 7f1708600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17000b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.416+0000 7f1708600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170015b080 con 0x7f17000aef80 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.416+0000 7f1708600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170015adb0 con 0x7f1700070350 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.416+0000 7f1708600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170015eb60 con 0x7f17000b31c0 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1700070350 0x7f170015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f17000aef80 0x7f170015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34950/0 (socket says 172.21.15.87:34950) 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705c00640 1 -- 172.21.15.87:0/73804594 learned_addr learned my addr 172.21.15.87:0/73804594 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1706600640 1 --2- 172.21.15.87:0/73804594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17000b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705200640 1 -- 172.21.15.87:0/73804594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 msgr2=0x7f17000b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:26.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705200640 1 --2- 172.21.15.87:0/73804594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17000b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705200640 1 -- 172.21.15.87:0/73804594 >> v1:172.21.15.87:6789/0 conn(0x7f17000aef80 legacy=0x7f170015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705200640 1 -- 172.21.15.87:0/73804594 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170015ee40 con 0x7f1700070350 2024-04-25T10:12:26.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f16ffe00640 1 -- 172.21.15.87:0/73804594 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723224982 0 0) 0x7f170015b080 con 0x7f17000aef80 2024-04-25T10:12:26.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1706600640 1 --2- 172.21.15.87:0/73804594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17000b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:26.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f1705200640 1 --2- 172.21.15.87:0/73804594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1700070350 0x7f170015e480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0097c0 tx=0x7f16ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f9e98fc4c44b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:26.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f16ffe00640 1 -- 172.21.15.87:0/73804594 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0b1070 con 0x7f1700070350 2024-04-25T10:12:26.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f16ffe00640 1 -- 172.21.15.87:0/73804594 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec0ac410 con 0x7f1700070350 2024-04-25T10:12:26.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.420+0000 7f16ffe00640 1 -- 172.21.15.87:0/73804594 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0ac7e0 con 0x7f1700070350 2024-04-25T10:12:26.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- 172.21.15.87:0/73804594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1700070350 msgr2=0x7f170015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 --2- 172.21.15.87:0/73804594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1700070350 0x7f170015e480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0097c0 tx=0x7f16ec01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- 172.21.15.87:0/73804594 shutdown_connections 2024-04-25T10:12:26.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 --2- 172.21.15.87:0/73804594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17000b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 --2- 172.21.15.87:0/73804594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1700070350 0x7f170015e480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- 172.21.15.87:0/73804594 >> 172.21.15.87:0/73804594 conn(0x7f17000669a0 msgr2=0x7f1700066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:26.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- 172.21.15.87:0/73804594 shutdown_connections 2024-04-25T10:12:26.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- 172.21.15.87:0/73804594 wait complete. 2024-04-25T10:12:26.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 Processor -- start 2024-04-25T10:12:26.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- start start 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1700070350 0x7f170014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17000aef80 0x7f1700154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17001500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17001679a0 con 0x7f17000aef80 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700167820 con 0x7f1700070350 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1708600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1700167b20 con 0x7f17000b31c0 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17000aef80 0x7f1700154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17000aef80 0x7f1700154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45992/0 (socket says 172.21.15.87:45992) 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 -- 172.21.15.87:0/2686109067 learned_addr learned my addr 172.21.15.87:0/2686109067 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:26.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705c00640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1700070350 0x7f170014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1706600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17001500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 -- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 msgr2=0x7f17001500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17001500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 -- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1700070350 msgr2=0x7f170014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1700070350 0x7f170014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.424+0000 7f1705200640 1 -- 172.21.15.87:0/2686109067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700155200 con 0x7f17000aef80 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f1705c00640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1700070350 0x7f170014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f1705200640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17000aef80 0x7f1700154740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0af040 tx=0x7f16ec019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f16fd600640 1 -- 172.21.15.87:0/2686109067 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0b1040 con 0x7f17000aef80 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f16fd600640 1 -- 172.21.15.87:0/2686109067 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec0ace20 con 0x7f17000aef80 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f16fd600640 1 -- 172.21.15.87:0/2686109067 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec006e20 con 0x7f17000aef80 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17001554e0 con 0x7f17000aef80 2024-04-25T10:12:26.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1700155aa0 con 0x7f17000aef80 2024-04-25T10:12:26.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170004ef30 con 0x7f17000aef80 2024-04-25T10:12:26.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f16fd600640 1 -- 172.21.15.87:0/2686109067 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16ec0be060 con 0x7f17000aef80 2024-04-25T10:12:26.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f16fd600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16e0066f60 0x7f16e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f16fd600640 1 -- 172.21.15.87:0/2686109067 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16ec11df80 con 0x7f17000aef80 2024-04-25T10:12:26.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.428+0000 7f1705c00640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16e0066f60 0x7f16e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.432+0000 7f1705c00640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16e0066f60 0x7f16e0069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000f70 tx=0x7f16f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.432+0000 7f16fd600640 1 -- 172.21.15.87:0/2686109067 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16ec016090 con 0x7f17000aef80 2024-04-25T10:12:26.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:26.620+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f17000b2fa0 con 0x7f17000aef80 2024-04-25T10:12:27.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f16fd600640 1 -- 172.21.15.87:0/2686109067 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f16ec0eeef0 con 0x7f17000aef80 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16e0066f60 msgr2=0x7f16e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16e0066f60 0x7f16e0069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000f70 tx=0x7f16f4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17000aef80 msgr2=0x7f1700154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17000aef80 0x7f1700154740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0af040 tx=0x7f16ec019240 comp rx=0 tx=0).stop 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 shutdown_connections 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16e0066f60 0x7f16e0069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17000b31c0 0x7f17001500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17000aef80 0x7f1700154740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 --2- 172.21.15.87:0/2686109067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1700070350 0x7f170014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.256+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 >> 172.21.15.87:0/2686109067 conn(0x7f17000669a0 msgr2=0x7f170006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.260+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 shutdown_connections 2024-04-25T10:12:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.260+0000 7f1708600640 1 -- 172.21.15.87:0/2686109067 wait complete. 2024-04-25T10:12:27.285 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:12:27.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:27.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfb6000640 1 Processor -- start 2024-04-25T10:12:27.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfb6000640 1 -- start start 2024-04-25T10:12:27.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfb6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdfb00d1820 0x7fdfb00d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfb6000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb00b1af0 con 0x7fdfb015e050 2024-04-25T10:12:27.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfb6000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb00b1dd0 con 0x7fdfb00b15b0 2024-04-25T10:12:27.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfb6000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb00b20b0 con 0x7fdfb00d1820 2024-04-25T10:12:27.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfaea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fdfb015e050 0x7fdfb00b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60498/0 (socket says 172.21.15.87:60498) 2024-04-25T10:12:27.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfaea00640 1 -- 172.21.15.87:0/266161901 learned_addr learned my addr 172.21.15.87:0/266161901 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfaf400640 1 --2- 172.21.15.87:0/266161901 >> v2:172.21.15.169:3300/0 conn(0x7fdfb00d1820 0x7fdfb00d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077059622 0 0) 0x7fdfb00b1af0 con 0x7fdfb015e050 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf94003530 con 0x7fdfb015e050 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425206815 0 0) 0x7fdfb00b1dd0 con 0x7fdfb00b15b0 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb00b1af0 con 0x7fdfb00b15b0 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2792804603 0 0) 0x7fdf94003530 con 0x7fdfb015e050 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 >> v2:172.21.15.169:3300/0 conn(0x7fdfb00d1820 msgr2=0x7fdfb00d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 --2- 172.21.15.87:0/266161901 >> v2:172.21.15.169:3300/0 conn(0x7fdfb00d1820 0x7fdfb00d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 >> v1:172.21.15.169:6789/0 conn(0x7fdfb00b15b0 legacy=0x7fdfb00cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb00b2390 con 0x7fdfb015e050 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fdf9c001600 con 0x7fdfb015e050 2024-04-25T10:12:27.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 >> v1:172.21.15.87:6789/0 conn(0x7fdfb015e050 legacy=0x7fdfb00b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94003ba0 0x7fdf94003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf94004810 0x7fdf94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf940051d0 0x7fdf9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b20b0 con 0x7fdf94004810 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9400c0a0 con 0x7fdf940051d0 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.416+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9400c2a0 con 0x7fdf94003ba0 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaea00640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf94004810 0x7fdf94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfafe00640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf940051d0 0x7fdf9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaf400640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94003ba0 0x7fdf94003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaea00640 1 -- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94003ba0 msgr2=0x7fdf94003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaea00640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94003ba0 0x7fdf94003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaea00640 1 -- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf940051d0 msgr2=0x7fdf9400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaea00640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf940051d0 0x7fdf9400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaea00640 1 -- 172.21.15.87:0/266161901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94003530 con 0x7fdf94004810 2024-04-25T10:12:27.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfafe00640 1 -- 172.21.15.87:0/266161901 reap_dead start 2024-04-25T10:12:27.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaf400640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94003ba0 0x7fdf94003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:27.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfaea00640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf94004810 0x7fdf94004c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c007f20 tx=0x7fdf9c01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5c7d0f26bd9153 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c007b70 con 0x7fdf94004810 2024-04-25T10:12:27.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c0a9020 con 0x7fdf94004810 2024-04-25T10:12:27.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.420+0000 7fdfae000640 1 -- 172.21.15.87:0/266161901 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c005ba0 con 0x7fdf94004810 2024-04-25T10:12:27.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- 172.21.15.87:0/266161901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf94004810 msgr2=0x7fdf94004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf94004810 0x7fdf94004c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c007f20 tx=0x7fdf9c01ca80 comp rx=0 tx=0).stop 2024-04-25T10:12:27.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- 172.21.15.87:0/266161901 shutdown_connections 2024-04-25T10:12:27.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 --2- 172.21.15.87:0/266161901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf94004810 0x7fdf94004c90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- 172.21.15.87:0/266161901 >> 172.21.15.87:0/266161901 conn(0x7fdfb009a9d0 msgr2=0x7fdfb009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- 172.21.15.87:0/266161901 shutdown_connections 2024-04-25T10:12:27.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- 172.21.15.87:0/266161901 wait complete. 2024-04-25T10:12:27.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 Processor -- start 2024-04-25T10:12:27.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- start start 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94004810 0x7fdfb00b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb015ea20 0x7fdfb0160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfb0161110 0x7fdfb006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b1020 con 0x7fdfb015ea20 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b0ea0 con 0x7fdfb0161110 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfb6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b11a0 con 0x7fdf94004810 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb015ea20 0x7fdfb0160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb015ea20 0x7fdfb0160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56894/0 (socket says 172.21.15.87:56894) 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 -- 172.21.15.87:0/821784066 learned_addr learned my addr 172.21.15.87:0/821784066 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfafe00640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfb0161110 0x7fdfb006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaf400640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94004810 0x7fdfb00b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 -- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94004810 msgr2=0x7fdfb00b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94004810 0x7fdfb00b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 -- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfb0161110 msgr2=0x7fdfb006eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfb0161110 0x7fdfb006eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 -- 172.21.15.87:0/821784066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb01f75c0 con 0x7fdfb015ea20 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaf400640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94004810 0x7fdfb00b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.424+0000 7fdfaea00640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb015ea20 0x7fdfb0160bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c01c8b0 tx=0x7fdf94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfa7400640 1 -- 172.21.15.87:0/821784066 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c001ab0 con 0x7fdfb015ea20 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfa7400640 1 -- 172.21.15.87:0/821784066 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c004c00 con 0x7fdfb015ea20 2024-04-25T10:12:27.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb01f78a0 con 0x7fdfb015ea20 2024-04-25T10:12:27.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfa7400640 1 -- 172.21.15.87:0/821784066 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c006e90 con 0x7fdfb015ea20 2024-04-25T10:12:27.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb01f7e60 con 0x7fdfb015ea20 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb004ef30 con 0x7fdfb015ea20 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfa7400640 1 -- 172.21.15.87:0/821784066 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf9c005040 con 0x7fdfb015ea20 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfa7400640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdf90066f60 0x7fdf90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfa7400640 1 -- 172.21.15.87:0/821784066 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf9c11e9a0 con 0x7fdfb015ea20 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfaf400640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdf90066f60 0x7fdf90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.428+0000 7fdfaf400640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdf90066f60 0x7fdf90069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfb006a000 tx=0x7fdfa0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.432+0000 7fdfa7400640 1 -- 172.21.15.87:0/821784066 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf9c016790 con 0x7fdfb015ea20 2024-04-25T10:12:27.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.620+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfb006b480 con 0x7fdfb015ea20 2024-04-25T10:12:27.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.620+0000 7fdfa7400640 1 -- 172.21.15.87:0/821784066 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fdf9c0ef9a0 con 0x7fdfb015ea20 2024-04-25T10:12:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:27.627 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:12:27.627 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdf90066f60 msgr2=0x7fdf90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdf90066f60 0x7fdf90069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfb006a000 tx=0x7fdfa0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb015ea20 msgr2=0x7fdfb0160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb015ea20 0x7fdfb0160bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c01c8b0 tx=0x7fdf94005840 comp rx=0 tx=0).stop 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 shutdown_connections 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdf90066f60 0x7fdf90069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfb0161110 0x7fdfb006eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb015ea20 0x7fdfb0160bd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 --2- 172.21.15.87:0/821784066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf94004810 0x7fdfb00b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 >> 172.21.15.87:0/821784066 conn(0x7fdfb009a9d0 msgr2=0x7fdfb00d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 shutdown_connections 2024-04-25T10:12:27.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.624+0000 7fdfb6000640 1 -- 172.21.15.87:0/821784066 wait complete. 2024-04-25T10:12:27.648 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:27.652 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-04-25T10:12:27.644+0000 7f553ec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d7a600640 1 Processor -- start 2024-04-25T10:12:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d7a600640 1 -- start start 2024-04-25T10:12:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d7a600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d7415b9e0 0x7f9d740cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d7a600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9d740d1c30 0x7f9d740cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d7a600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d740ccfb0 con 0x7f9d7415bdc0 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d7a600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d740cd290 con 0x7f9d740d1c30 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d7a600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d740cd570 con 0x7f9d7415b9e0 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d73e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9d740d1c30 0x7f9d740cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d6aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9d7415bdc0 0x7f9d740d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60506/0 (socket says 172.21.15.87:60506) 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d6aa00640 1 -- 172.21.15.87:0/895231240 learned_addr learned my addr 172.21.15.87:0/895231240 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d73400640 1 --2- 172.21.15.87:0/895231240 >> v2:172.21.15.169:3300/0 conn(0x7f9d7415b9e0 0x7f9d740cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d73e00640 1 -- 172.21.15.87:0/895231240 >> v2:172.21.15.169:3300/0 conn(0x7f9d7415b9e0 msgr2=0x7f9d740cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d73e00640 1 --2- 172.21.15.87:0/895231240 >> v2:172.21.15.169:3300/0 conn(0x7f9d7415b9e0 0x7f9d740cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d73e00640 1 -- 172.21.15.87:0/895231240 >> v1:172.21.15.87:6789/0 conn(0x7f9d7415bdc0 legacy=0x7f9d740d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d73e00640 1 -- 172.21.15.87:0/895231240 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740cd850 con 0x7f9d740d1c30 2024-04-25T10:12:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.780+0000 7f9d72a00640 1 -- 172.21.15.87:0/895231240 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799859775 0 0) 0x7f9d740ccfb0 con 0x7f9d7415bdc0 2024-04-25T10:12:27.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d73e00640 1 --2- 172.21.15.87:0/895231240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d740d1c30 0x7f9d740cc950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9d74070da0 tx=0x7f9d6c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564ee45c51aab75e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d72a00640 1 -- 172.21.15.87:0/895231240 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0b2070 con 0x7f9d740d1c30 2024-04-25T10:12:27.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d72a00640 1 -- 172.21.15.87:0/895231240 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c0a5810 con 0x7f9d740d1c30 2024-04-25T10:12:27.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d72a00640 1 -- 172.21.15.87:0/895231240 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0a5be0 con 0x7f9d740d1c30 2024-04-25T10:12:27.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 -- 172.21.15.87:0/895231240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d740d1c30 msgr2=0x7f9d740cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 --2- 172.21.15.87:0/895231240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d740d1c30 0x7f9d740cc950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9d74070da0 tx=0x7f9d6c01c8a0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 -- 172.21.15.87:0/895231240 shutdown_connections 2024-04-25T10:12:27.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 --2- 172.21.15.87:0/895231240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d740d1c30 0x7f9d740cc950 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 --2- 172.21.15.87:0/895231240 >> v2:172.21.15.169:3300/0 conn(0x7f9d7415b9e0 0x7f9d740cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 -- 172.21.15.87:0/895231240 >> 172.21.15.87:0/895231240 conn(0x7f9d7409a9d0 msgr2=0x7f9d7409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 -- 172.21.15.87:0/895231240 shutdown_connections 2024-04-25T10:12:27.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.784+0000 7f9d7a600640 1 -- 172.21.15.87:0/895231240 wait complete. 2024-04-25T10:12:27.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 Processor -- start 2024-04-25T10:12:27.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 -- start start 2024-04-25T10:12:27.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d740d1c30 0x7f9d740b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d7415b9e0 0x7f9d74156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7415bdc0 0x7f9d7415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740c6630 con 0x7f9d7415bdc0 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740c64b0 con 0x7f9d7415b9e0 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740c67b0 con 0x7f9d740d1c30 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7415bdc0 0x7f9d7415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7415bdc0 0x7f9d7415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56916/0 (socket says 172.21.15.87:56916) 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 -- 172.21.15.87:0/2608127214 learned_addr learned my addr 172.21.15.87:0/2608127214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d6aa00640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d7415b9e0 0x7f9d74156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73400640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d740d1c30 0x7f9d740b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 -- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d740d1c30 msgr2=0x7f9d740b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d740d1c30 0x7f9d740b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 -- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d7415b9e0 msgr2=0x7f9d74156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d7415b9e0 0x7f9d74156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 -- 172.21.15.87:0/2608127214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7415ecf0 con 0x7f9d7415bdc0 2024-04-25T10:12:27.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73400640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d740d1c30 0x7f9d740b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d73e00640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7415bdc0 0x7f9d7415e350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0b0040 tx=0x7f9d6c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d6be00640 1 -- 172.21.15.87:0/2608127214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0b2050 con 0x7f9d7415bdc0 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d6be00640 1 -- 172.21.15.87:0/2608127214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c0be070 con 0x7f9d7415bdc0 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d6be00640 1 -- 172.21.15.87:0/2608127214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0ada90 con 0x7f9d7415bdc0 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d740b5850 con 0x7f9d7415bdc0 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d740b5e10 con 0x7f9d7415bdc0 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.788+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7404ef30 con 0x7f9d7415bdc0 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.792+0000 7f9d6be00640 1 -- 172.21.15.87:0/2608127214 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d6c0a5690 con 0x7f9d7415bdc0 2024-04-25T10:12:27.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.792+0000 7f9d6be00640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d50066f60 0x7f9d50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.792+0000 7f9d6be00640 1 -- 172.21.15.87:0/2608127214 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d6c120270 con 0x7f9d7415bdc0 2024-04-25T10:12:27.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.792+0000 7f9d73400640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d50066f60 0x7f9d50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.792+0000 7f9d73400640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d50066f60 0x7f9d50069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009840 tx=0x7f9d64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.796+0000 7f9d6be00640 1 -- 172.21.15.87:0/2608127214 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d6c016520 con 0x7f9d7415bdc0 2024-04-25T10:12:27.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.948+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d7406ffe0 con 0x7f9d7415bdc0 2024-04-25T10:12:27.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d6be00640 1 -- 172.21.15.87:0/2608127214 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f9d6c0f11e0 con 0x7f9d7415bdc0 2024-04-25T10:12:27.957 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:27.957 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d50066f60 msgr2=0x7f9d50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d50066f60 0x7f9d50069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009840 tx=0x7f9d64008040 comp rx=0 tx=0).stop 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7415bdc0 msgr2=0x7f9d7415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7415bdc0 0x7f9d7415e350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0b0040 tx=0x7f9d6c019640 comp rx=0 tx=0).stop 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 shutdown_connections 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d50066f60 0x7f9d50069420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7415bdc0 0x7f9d7415e350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d7415b9e0 0x7f9d74156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 --2- 172.21.15.87:0/2608127214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d740d1c30 0x7f9d740b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 >> 172.21.15.87:0/2608127214 conn(0x7f9d7409a9d0 msgr2=0x7f9d7406e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 shutdown_connections 2024-04-25T10:12:27.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:27.952+0000 7f9d7a600640 1 -- 172.21.15.87:0/2608127214 wait complete. 2024-04-25T10:12:27.978 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:12:27.978 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:28.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 Processor -- start 2024-04-25T10:12:28.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 -- start start 2024-04-25T10:12:28.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4ec415da50 0x7f4ec40b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec40b16c0 0x7f4ec40669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ec4066ee0 0x7f4ec406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec415de30 con 0x7f4ec4066ee0 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec40b59f0 con 0x7f4ec415da50 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40b5cd0 con 0x7f4ec40b16c0 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ec4066ee0 0x7f4ec406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ec4066ee0 0x7f4ec406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56932/0 (socket says 172.21.15.87:56932) 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.108+0000 7f4ec2800640 1 -- 172.21.15.87:0/3750954083 learned_addr learned my addr 172.21.15.87:0/3750954083 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec2800640 1 -- 172.21.15.87:0/3750954083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec40b16c0 msgr2=0x7f4ec40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec2800640 1 --2- 172.21.15.87:0/3750954083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec40b16c0 0x7f4ec40669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec2800640 1 -- 172.21.15.87:0/3750954083 >> v2:172.21.15.169:3300/0 conn(0x7f4ec415da50 msgr2=0x7f4ec40b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec1e00640 1 --2- 172.21.15.87:0/3750954083 >> v2:172.21.15.169:3300/0 conn(0x7f4ec415da50 0x7f4ec40b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec2800640 1 --2- 172.21.15.87:0/3750954083 >> v2:172.21.15.169:3300/0 conn(0x7f4ec415da50 0x7f4ec40b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec2800640 1 -- 172.21.15.87:0/3750954083 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec40b5fb0 con 0x7f4ec4066ee0 2024-04-25T10:12:28.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec1e00640 1 --2- 172.21.15.87:0/3750954083 >> v2:172.21.15.169:3300/0 conn(0x7f4ec415da50 0x7f4ec40b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:28.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec2800640 1 --2- 172.21.15.87:0/3750954083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec4066ee0 0x7f4ec406d310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40b68f0 tx=0x7f4ebc01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c49c04d6abaa42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec1400640 1 -- 172.21.15.87:0/3750954083 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0a9430 con 0x7f4ec4066ee0 2024-04-25T10:12:28.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec1400640 1 -- 172.21.15.87:0/3750954083 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc0ae070 con 0x7f4ec4066ee0 2024-04-25T10:12:28.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec1400640 1 -- 172.21.15.87:0/3750954083 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0a97f0 con 0x7f4ec4066ee0 2024-04-25T10:12:28.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec3200640 1 -- 172.21.15.87:0/3750954083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec4066ee0 msgr2=0x7f4ec406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec3200640 1 --2- 172.21.15.87:0/3750954083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec4066ee0 0x7f4ec406d310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40b68f0 tx=0x7f4ebc01c6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec3200640 1 -- 172.21.15.87:0/3750954083 shutdown_connections 2024-04-25T10:12:28.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec3200640 1 --2- 172.21.15.87:0/3750954083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec4066ee0 0x7f4ec406d310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec3200640 1 --2- 172.21.15.87:0/3750954083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec40b16c0 0x7f4ec40669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec3200640 1 --2- 172.21.15.87:0/3750954083 >> v2:172.21.15.169:3300/0 conn(0x7f4ec415da50 0x7f4ec40b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.112+0000 7f4ec3200640 1 -- 172.21.15.87:0/3750954083 >> 172.21.15.87:0/3750954083 conn(0x7f4ec40d0690 msgr2=0x7f4ec40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 -- 172.21.15.87:0/3750954083 shutdown_connections 2024-04-25T10:12:28.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 -- 172.21.15.87:0/3750954083 wait complete. 2024-04-25T10:12:28.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 Processor -- start 2024-04-25T10:12:28.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 -- start start 2024-04-25T10:12:28.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4066ee0 0x7f4ec4148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec40b16c0 0x7f4ec40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec415da50 0x7f4ec40b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40cea40 con 0x7f4ec40b16c0 2024-04-25T10:12:28.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40ce8c0 con 0x7f4ec415da50 2024-04-25T10:12:28.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40cebc0 con 0x7f4ec4066ee0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec40b16c0 0x7f4ec40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec40b16c0 0x7f4ec40b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56936/0 (socket says 172.21.15.87:56936) 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 -- 172.21.15.87:0/1117079932 learned_addr learned my addr 172.21.15.87:0/1117079932 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec1e00640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4066ee0 0x7f4ec4148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec2800640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec415da50 0x7f4ec40b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 -- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4066ee0 msgr2=0x7f4ec4148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4066ee0 0x7f4ec4148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 -- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec415da50 msgr2=0x7f4ec40b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec415da50 0x7f4ec40b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 -- 172.21.15.87:0/1117079932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4148ff0 con 0x7f4ec40b16c0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4eb9400640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec40b16c0 0x7f4ec40b3f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00b550 tx=0x7f4eac00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ebaa00640 1 -- 172.21.15.87:0/1117079932 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac014070 con 0x7f4ec40b16c0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ebaa00640 1 -- 172.21.15.87:0/1117079932 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac010890 con 0x7f4ec40b16c0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ebaa00640 1 -- 172.21.15.87:0/1117079932 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac010c80 con 0x7f4ec40b16c0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.116+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec4158800 con 0x7f4ec40b16c0 2024-04-25T10:12:28.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.120+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec4157f60 con 0x7f4ec40b16c0 2024-04-25T10:12:28.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.120+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec4075120 con 0x7f4ec40b16c0 2024-04-25T10:12:28.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.120+0000 7f4ebaa00640 1 -- 172.21.15.87:0/1117079932 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4eac017070 con 0x7f4ec40b16c0 2024-04-25T10:12:28.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.120+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4ea0066f60 0x7f4ea0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.120+0000 7f4ebaa00640 1 -- 172.21.15.87:0/1117079932 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4eac084ca0 con 0x7f4ec40b16c0 2024-04-25T10:12:28.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.120+0000 7f4ec1e00640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4ea0066f60 0x7f4ea0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.120+0000 7f4ec1e00640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4ea0066f60 0x7f4ea0069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.124+0000 7f4ebaa00640 1 -- 172.21.15.87:0/1117079932 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ec4148ff0 con 0x7f4ec40b16c0 2024-04-25T10:12:28.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.276+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ec40b20b0 con 0x7f4ea0066f60 2024-04-25T10:12:28.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.284+0000 7f4ebaa00640 1 -- 172.21.15.87:0/1117079932 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251942 (secure 0 0 0) 0x7f4ec40b20b0 con 0x7f4ea0066f60 2024-04-25T10:12:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:28.295 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:12:28.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4ea0066f60 msgr2=0x7f4ea0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4ea0066f60 0x7f4ea0069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec40b16c0 msgr2=0x7f4ec40b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec40b16c0 0x7f4ec40b3f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00b550 tx=0x7f4eac00ba10 comp rx=0 tx=0).stop 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 shutdown_connections 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4ea0066f60 0x7f4ea0069420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec415da50 0x7f4ec40b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ec40b16c0 0x7f4ec40b3f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 --2- 172.21.15.87:0/1117079932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4066ee0 0x7f4ec4148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 >> 172.21.15.87:0/1117079932 conn(0x7f4ec40d0690 msgr2=0x7f4ec415c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 shutdown_connections 2024-04-25T10:12:28.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.288+0000 7f4ec3200640 1 -- 172.21.15.87:0/1117079932 wait complete. 2024-04-25T10:12:28.313 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:12:27.125402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218940,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755780,"statfs":{"total":773094113280,"available":772869918720,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.910725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015534+0000","last_change":"2024-04-25T10:12:24.263197+0000","last_active":"2024-04-25T10:12:26.015534+0000","last_peered":"2024-04-25T10:12:26.015534+0000","last_clean":"2024-04-25T10:12:26.015534+0000","last_became_active":"2024-04-25T10:12:24.262932+0000","last_became_peered":"2024-04-25T10:12:24.262932+0000","last_unstale":"2024-04-25T10:12:26.015534+0000","last_undegraded":"2024-04-25T10:12:26.015534+0000","last_fullsized":"2024-04-25T10:12:26.015534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247413+0000","last_change":"2024-04-25T10:12:24.255491+0000","last_active":"2024-04-25T10:12:25.247413+0000","last_peered":"2024-04-25T10:12:25.247413+0000","last_clean":"2024-04-25T10:12:25.247413+0000","last_became_active":"2024-04-25T10:12:24.255325+0000","last_became_peered":"2024-04-25T10:12:24.255325+0000","last_unstale":"2024-04-25T10:12:25.247413+0000","last_undegraded":"2024-04-25T10:12:25.247413+0000","last_fullsized":"2024-04-25T10:12:25.247413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016327+0000","last_change":"2024-04-25T10:12:24.259328+0000","last_active":"2024-04-25T10:12:26.016327+0000","last_peered":"2024-04-25T10:12:26.016327+0000","last_clean":"2024-04-25T10:12:26.016327+0000","last_became_active":"2024-04-25T10:12:24.259065+0000","last_became_peered":"2024-04-25T10:12:24.259065+0000","last_unstale":"2024-04-25T10:12:26.016327+0000","last_undegraded":"2024-04-25T10:12:26.016327+0000","last_fullsized":"2024-04-25T10:12:26.016327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247574+0000","last_change":"2024-04-25T10:12:24.263211+0000","last_active":"2024-04-25T10:12:25.247574+0000","last_peered":"2024-04-25T10:12:25.247574+0000","last_clean":"2024-04-25T10:12:25.247574+0000","last_became_active":"2024-04-25T10:12:24.262896+0000","last_became_peered":"2024-04-25T10:12:24.262896+0000","last_unstale":"2024-04-25T10:12:25.247574+0000","last_undegraded":"2024-04-25T10:12:25.247574+0000","last_fullsized":"2024-04-25T10:12:25.247574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014990+0000","last_change":"2024-04-25T10:12:24.259922+0000","last_active":"2024-04-25T10:12:26.014990+0000","last_peered":"2024-04-25T10:12:26.014990+0000","last_clean":"2024-04-25T10:12:26.014990+0000","last_became_active":"2024-04-25T10:12:24.259657+0000","last_became_peered":"2024-04-25T10:12:24.259657+0000","last_unstale":"2024-04-25T10:12:26.014990+0000","last_undegraded":"2024-04-25T10:12:26.014990+0000","last_fullsized":"2024-04-25T10:12:26.014990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245975+0000","last_change":"2024-04-25T10:12:24.254906+0000","last_active":"2024-04-25T10:12:25.245975+0000","last_peered":"2024-04-25T10:12:25.245975+0000","last_clean":"2024-04-25T10:12:25.245975+0000","last_became_active":"2024-04-25T10:12:24.254592+0000","last_became_peered":"2024-04-25T10:12:24.254592+0000","last_unstale":"2024-04-25T10:12:25.245975+0000","last_undegraded":"2024-04-25T10:12:25.245975+0000","last_fullsized":"2024-04-25T10:12:25.245975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014929+0000","last_change":"2024-04-25T10:12:24.254975+0000","last_active":"2024-04-25T10:12:26.014929+0000","last_peered":"2024-04-25T10:12:26.014929+0000","last_clean":"2024-04-25T10:12:26.014929+0000","last_became_active":"2024-04-25T10:12:24.254639+0000","last_became_peered":"2024-04-25T10:12:24.254639+0000","last_unstale":"2024-04-25T10:12:26.014929+0000","last_undegraded":"2024-04-25T10:12:26.014929+0000","last_fullsized":"2024-04-25T10:12:26.014929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015387+0000","last_change":"2024-04-25T10:12:24.264638+0000","last_active":"2024-04-25T10:12:26.015387+0000","last_peered":"2024-04-25T10:12:26.015387+0000","last_clean":"2024-04-25T10:12:26.015387+0000","last_became_active":"2024-04-25T10:12:24.264376+0000","last_became_peered":"2024-04-25T10:12:24.264376+0000","last_unstale":"2024-04-25T10:12:26.015387+0000","last_undegraded":"2024-04-25T10:12:26.015387+0000","last_fullsized":"2024-04-25T10:12:26.015387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015572+0000","last_change":"2024-04-25T10:12:24.257501+0000","last_active":"2024-04-25T10:12:26.015572+0000","last_peered":"2024-04-25T10:12:26.015572+0000","last_clean":"2024-04-25T10:12:26.015572+0000","last_became_active":"2024-04-25T10:12:24.257199+0000","last_became_peered":"2024-04-25T10:12:24.257199+0000","last_unstale":"2024-04-25T10:12:26.015572+0000","last_undegraded":"2024-04-25T10:12:26.015572+0000","last_fullsized":"2024-04-25T10:12:26.015572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012594+0000","last_change":"2024-04-25T10:12:24.250894+0000","last_active":"2024-04-25T10:12:26.012594+0000","last_peered":"2024-04-25T10:12:26.012594+0000","last_clean":"2024-04-25T10:12:26.012594+0000","last_became_active":"2024-04-25T10:12:24.250581+0000","last_became_peered":"2024-04-25T10:12:24.250581+0000","last_unstale":"2024-04-25T10:12:26.012594+0000","last_undegraded":"2024-04-25T10:12:26.012594+0000","last_fullsized":"2024-04-25T10:12:26.012594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012005+0000","last_change":"2024-04-25T10:12:24.247801+0000","last_active":"2024-04-25T10:12:26.012005+0000","last_peered":"2024-04-25T10:12:26.012005+0000","last_clean":"2024-04-25T10:12:26.012005+0000","last_became_active":"2024-04-25T10:12:24.247485+0000","last_became_peered":"2024-04-25T10:12:24.247485+0000","last_unstale":"2024-04-25T10:12:26.012005+0000","last_undegraded":"2024-04-25T10:12:26.012005+0000","last_fullsized":"2024-04-25T10:12:26.012005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.262042+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.261770+0000","last_became_peered":"2024-04-25T10:12:24.261770+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:36.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015914+0000","last_change":"2024-04-25T10:12:24.263554+0000","last_active":"2024-04-25T10:12:26.015914+0000","last_peered":"2024-04-25T10:12:26.015914+0000","last_clean":"2024-04-25T10:12:26.015914+0000","last_became_active":"2024-04-25T10:12:24.263244+0000","last_became_peered":"2024-04-25T10:12:24.263244+0000","last_unstale":"2024-04-25T10:12:26.015914+0000","last_undegraded":"2024-04-25T10:12:26.015914+0000","last_fullsized":"2024-04-25T10:12:26.015914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015884+0000","last_change":"2024-04-25T10:12:24.246682+0000","last_active":"2024-04-25T10:12:26.015884+0000","last_peered":"2024-04-25T10:12:26.015884+0000","last_clean":"2024-04-25T10:12:26.015884+0000","last_became_active":"2024-04-25T10:12:24.246353+0000","last_became_peered":"2024-04-25T10:12:24.246353+0000","last_unstale":"2024-04-25T10:12:26.015884+0000","last_undegraded":"2024-04-25T10:12:26.015884+0000","last_fullsized":"2024-04-25T10:12:26.015884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:15.421048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246997+0000","last_change":"2024-04-25T10:12:24.265431+0000","last_active":"2024-04-25T10:12:25.246997+0000","last_peered":"2024-04-25T10:12:25.246997+0000","last_clean":"2024-04-25T10:12:25.246997+0000","last_became_active":"2024-04-25T10:12:24.265047+0000","last_became_peered":"2024-04-25T10:12:24.265047+0000","last_unstale":"2024-04-25T10:12:25.246997+0000","last_undegraded":"2024-04-25T10:12:25.246997+0000","last_fullsized":"2024-04-25T10:12:25.246997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247153+0000","last_change":"2024-04-25T10:12:24.248104+0000","last_active":"2024-04-25T10:12:25.247153+0000","last_peered":"2024-04-25T10:12:25.247153+0000","last_clean":"2024-04-25T10:12:25.247153+0000","last_became_active":"2024-04-25T10:12:24.247826+0000","last_became_peered":"2024-04-25T10:12:24.247826+0000","last_unstale":"2024-04-25T10:12:25.247153+0000","last_undegraded":"2024-04-25T10:12:25.247153+0000","last_fullsized":"2024-04-25T10:12:25.247153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247053+0000","last_change":"2024-04-25T10:12:24.257638+0000","last_active":"2024-04-25T10:12:25.247053+0000","last_peered":"2024-04-25T10:12:25.247053+0000","last_clean":"2024-04-25T10:12:25.247053+0000","last_became_active":"2024-04-25T10:12:24.257332+0000","last_became_peered":"2024-04-25T10:12:24.257332+0000","last_unstale":"2024-04-25T10:12:25.247053+0000","last_undegraded":"2024-04-25T10:12:25.247053+0000","last_fullsized":"2024-04-25T10:12:25.247053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872137+0000","last_change":"2024-04-25T10:12:24.262252+0000","last_active":"2024-04-25T10:12:25.872137+0000","last_peered":"2024-04-25T10:12:25.872137+0000","last_clean":"2024-04-25T10:12:25.872137+0000","last_became_active":"2024-04-25T10:12:24.261871+0000","last_became_peered":"2024-04-25T10:12:24.261871+0000","last_unstale":"2024-04-25T10:12:25.872137+0000","last_undegraded":"2024-04-25T10:12:25.872137+0000","last_fullsized":"2024-04-25T10:12:25.872137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:13.729919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015458+0000","last_change":"2024-04-25T10:12:24.269990+0000","last_active":"2024-04-25T10:12:26.015458+0000","last_peered":"2024-04-25T10:12:26.015458+0000","last_clean":"2024-04-25T10:12:26.015458+0000","last_became_active":"2024-04-25T10:12:24.269396+0000","last_became_peered":"2024-04-25T10:12:24.269396+0000","last_unstale":"2024-04-25T10:12:26.015458+0000","last_undegraded":"2024-04-25T10:12:26.015458+0000","last_fullsized":"2024-04-25T10:12:26.015458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012013+0000","last_change":"2024-04-25T10:12:24.262832+0000","last_active":"2024-04-25T10:12:26.012013+0000","last_peered":"2024-04-25T10:12:26.012013+0000","last_clean":"2024-04-25T10:12:26.012013+0000","last_became_active":"2024-04-25T10:12:24.262090+0000","last_became_peered":"2024-04-25T10:12:24.262090+0000","last_unstale":"2024-04-25T10:12:26.012013+0000","last_undegraded":"2024-04-25T10:12:26.012013+0000","last_fullsized":"2024-04-25T10:12:26.012013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016383+0000","last_change":"2024-04-25T10:12:24.248538+0000","last_active":"2024-04-25T10:12:26.016383+0000","last_peered":"2024-04-25T10:12:26.016383+0000","last_clean":"2024-04-25T10:12:26.016383+0000","last_became_active":"2024-04-25T10:12:24.247653+0000","last_became_peered":"2024-04-25T10:12:24.247653+0000","last_unstale":"2024-04-25T10:12:26.016383+0000","last_undegraded":"2024-04-25T10:12:26.016383+0000","last_fullsized":"2024-04-25T10:12:26.016383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246115+0000","last_change":"2024-04-25T10:12:24.260603+0000","last_active":"2024-04-25T10:12:25.246115+0000","last_peered":"2024-04-25T10:12:25.246115+0000","last_clean":"2024-04-25T10:12:25.246115+0000","last_became_active":"2024-04-25T10:12:24.260288+0000","last_became_peered":"2024-04-25T10:12:24.260288+0000","last_unstale":"2024-04-25T10:12:25.246115+0000","last_undegraded":"2024-04-25T10:12:25.246115+0000","last_fullsized":"2024-04-25T10:12:25.246115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015623+0000","last_change":"2024-04-25T10:12:24.272790+0000","last_active":"2024-04-25T10:12:26.015623+0000","last_peered":"2024-04-25T10:12:26.015623+0000","last_clean":"2024-04-25T10:12:26.015623+0000","last_became_active":"2024-04-25T10:12:24.272587+0000","last_became_peered":"2024-04-25T10:12:24.272587+0000","last_unstale":"2024-04-25T10:12:26.015623+0000","last_undegraded":"2024-04-25T10:12:26.015623+0000","last_fullsized":"2024-04-25T10:12:26.015623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245637+0000","last_change":"2024-04-25T10:12:24.265627+0000","last_active":"2024-04-25T10:12:25.245637+0000","last_peered":"2024-04-25T10:12:25.245637+0000","last_clean":"2024-04-25T10:12:25.245637+0000","last_became_active":"2024-04-25T10:12:24.265274+0000","last_became_peered":"2024-04-25T10:12:24.265274+0000","last_unstale":"2024-04-25T10:12:25.245637+0000","last_undegraded":"2024-04-25T10:12:25.245637+0000","last_fullsized":"2024-04-25T10:12:25.245637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016338+0000","last_change":"2024-04-25T10:12:24.272301+0000","last_active":"2024-04-25T10:12:26.016338+0000","last_peered":"2024-04-25T10:12:26.016338+0000","last_clean":"2024-04-25T10:12:26.016338+0000","last_became_active":"2024-04-25T10:12:24.271766+0000","last_became_peered":"2024-04-25T10:12:24.271766+0000","last_unstale":"2024-04-25T10:12:26.016338+0000","last_undegraded":"2024-04-25T10:12:26.016338+0000","last_fullsized":"2024-04-25T10:12:26.016338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012456+0000","last_change":"2024-04-25T10:12:24.265368+0000","last_active":"2024-04-25T10:12:26.012456+0000","last_peered":"2024-04-25T10:12:26.012456+0000","last_clean":"2024-04-25T10:12:26.012456+0000","last_became_active":"2024-04-25T10:12:24.264950+0000","last_became_peered":"2024-04-25T10:12:24.264950+0000","last_unstale":"2024-04-25T10:12:26.012456+0000","last_undegraded":"2024-04-25T10:12:26.012456+0000","last_fullsized":"2024-04-25T10:12:26.012456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247251+0000","last_change":"2024-04-25T10:12:24.259900+0000","last_active":"2024-04-25T10:12:25.247251+0000","last_peered":"2024-04-25T10:12:25.247251+0000","last_clean":"2024-04-25T10:12:25.247251+0000","last_became_active":"2024-04-25T10:12:24.259574+0000","last_became_peered":"2024-04-25T10:12:24.259574+0000","last_unstale":"2024-04-25T10:12:25.247251+0000","last_undegraded":"2024-04-25T10:12:25.247251+0000","last_fullsized":"2024-04-25T10:12:25.247251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015996+0000","last_change":"2024-04-25T10:12:24.272315+0000","last_active":"2024-04-25T10:12:26.015996+0000","last_peered":"2024-04-25T10:12:26.015996+0000","last_clean":"2024-04-25T10:12:26.015996+0000","last_became_active":"2024-04-25T10:12:24.271843+0000","last_became_peered":"2024-04-25T10:12:24.271843+0000","last_unstale":"2024-04-25T10:12:26.015996+0000","last_undegraded":"2024-04-25T10:12:26.015996+0000","last_fullsized":"2024-04-25T10:12:26.015996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016505+0000","last_change":"2024-04-25T10:12:24.260827+0000","last_active":"2024-04-25T10:12:26.016505+0000","last_peered":"2024-04-25T10:12:26.016505+0000","last_clean":"2024-04-25T10:12:26.016505+0000","last_became_active":"2024-04-25T10:12:24.260436+0000","last_became_peered":"2024-04-25T10:12:24.260436+0000","last_unstale":"2024-04-25T10:12:26.016505+0000","last_undegraded":"2024-04-25T10:12:26.016505+0000","last_fullsized":"2024-04-25T10:12:26.016505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246276+0000","last_change":"2024-04-25T10:12:24.261679+0000","last_active":"2024-04-25T10:12:25.246276+0000","last_peered":"2024-04-25T10:12:25.246276+0000","last_clean":"2024-04-25T10:12:25.246276+0000","last_became_active":"2024-04-25T10:12:24.261420+0000","last_became_peered":"2024-04-25T10:12:24.261420+0000","last_unstale":"2024-04-25T10:12:25.246276+0000","last_undegraded":"2024-04-25T10:12:25.246276+0000","last_fullsized":"2024-04-25T10:12:25.246276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872106+0000","last_change":"2024-04-25T10:12:24.260060+0000","last_active":"2024-04-25T10:12:25.872106+0000","last_peered":"2024-04-25T10:12:25.872106+0000","last_clean":"2024-04-25T10:12:25.872106+0000","last_became_active":"2024-04-25T10:12:24.259332+0000","last_became_peered":"2024-04-25T10:12:24.259332+0000","last_unstale":"2024-04-25T10:12:25.872106+0000","last_undegraded":"2024-04-25T10:12:25.872106+0000","last_fullsized":"2024-04-25T10:12:25.872106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011814+0000","last_change":"2024-04-25T10:12:24.247021+0000","last_active":"2024-04-25T10:12:26.011814+0000","last_peered":"2024-04-25T10:12:26.011814+0000","last_clean":"2024-04-25T10:12:26.011814+0000","last_became_active":"2024-04-25T10:12:24.246699+0000","last_became_peered":"2024-04-25T10:12:24.246699+0000","last_unstale":"2024-04-25T10:12:26.011814+0000","last_undegraded":"2024-04-25T10:12:26.011814+0000","last_fullsized":"2024-04-25T10:12:26.011814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245745+0000","last_change":"2024-04-25T10:12:24.263775+0000","last_active":"2024-04-25T10:12:25.245745+0000","last_peered":"2024-04-25T10:12:25.245745+0000","last_clean":"2024-04-25T10:12:25.245745+0000","last_became_active":"2024-04-25T10:12:24.263373+0000","last_became_peered":"2024-04-25T10:12:24.263373+0000","last_unstale":"2024-04-25T10:12:25.245745+0000","last_undegraded":"2024-04-25T10:12:25.245745+0000","last_fullsized":"2024-04-25T10:12:25.245745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:23.746340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016216+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:26.016216+0000","last_peered":"2024-04-25T10:12:26.016216+0000","last_clean":"2024-04-25T10:12:26.016216+0000","last_became_active":"2024-04-25T10:12:23.249533+0000","last_became_peered":"2024-04-25T10:12:23.249533+0000","last_unstale":"2024-04-25T10:12:26.016216+0000","last_undegraded":"2024-04-25T10:12:26.016216+0000","last_fullsized":"2024-04-25T10:12:26.016216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016549+0000","last_change":"2024-04-25T10:12:24.252703+0000","last_active":"2024-04-25T10:12:26.016549+0000","last_peered":"2024-04-25T10:12:26.016549+0000","last_clean":"2024-04-25T10:12:26.016549+0000","last_became_active":"2024-04-25T10:12:24.252372+0000","last_became_peered":"2024-04-25T10:12:24.252372+0000","last_unstale":"2024-04-25T10:12:26.016549+0000","last_undegraded":"2024-04-25T10:12:26.016549+0000","last_fullsized":"2024-04-25T10:12:26.016549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012305+0000","last_change":"2024-04-25T10:12:23.248077+0000","last_active":"2024-04-25T10:12:26.012305+0000","last_peered":"2024-04-25T10:12:26.012305+0000","last_clean":"2024-04-25T10:12:26.012305+0000","last_became_active":"2024-04-25T10:12:23.247006+0000","last_became_peered":"2024-04-25T10:12:23.247006+0000","last_unstale":"2024-04-25T10:12:26.012305+0000","last_undegraded":"2024-04-25T10:12:26.012305+0000","last_fullsized":"2024-04-25T10:12:26.012305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015823+0000","last_change":"2024-04-25T10:12:23.240214+0000","last_active":"2024-04-25T10:12:26.015823+0000","last_peered":"2024-04-25T10:12:26.015823+0000","last_clean":"2024-04-25T10:12:26.015823+0000","last_became_active":"2024-04-25T10:12:23.239944+0000","last_became_peered":"2024-04-25T10:12:23.239944+0000","last_unstale":"2024-04-25T10:12:26.015823+0000","last_undegraded":"2024-04-25T10:12:26.015823+0000","last_fullsized":"2024-04-25T10:12:26.015823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015556+0000","last_change":"2024-04-25T10:12:24.258236+0000","last_active":"2024-04-25T10:12:26.015556+0000","last_peered":"2024-04-25T10:12:26.015556+0000","last_clean":"2024-04-25T10:12:26.015556+0000","last_became_active":"2024-04-25T10:12:24.256931+0000","last_became_peered":"2024-04-25T10:12:24.256931+0000","last_unstale":"2024-04-25T10:12:26.015556+0000","last_undegraded":"2024-04-25T10:12:26.015556+0000","last_fullsized":"2024-04-25T10:12:26.015556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245733+0000","last_change":"2024-04-25T10:12:23.253660+0000","last_active":"2024-04-25T10:12:25.245733+0000","last_peered":"2024-04-25T10:12:25.245733+0000","last_clean":"2024-04-25T10:12:25.245733+0000","last_became_active":"2024-04-25T10:12:23.253047+0000","last_became_peered":"2024-04-25T10:12:23.253047+0000","last_unstale":"2024-04-25T10:12:25.245733+0000","last_undegraded":"2024-04-25T10:12:25.245733+0000","last_fullsized":"2024-04-25T10:12:25.245733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246967+0000","last_change":"2024-04-25T10:12:24.260980+0000","last_active":"2024-04-25T10:12:25.246967+0000","last_peered":"2024-04-25T10:12:25.246967+0000","last_clean":"2024-04-25T10:12:25.246967+0000","last_became_active":"2024-04-25T10:12:24.260459+0000","last_became_peered":"2024-04-25T10:12:24.260459+0000","last_unstale":"2024-04-25T10:12:25.246967+0000","last_undegraded":"2024-04-25T10:12:25.246967+0000","last_fullsized":"2024-04-25T10:12:25.246967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012372+0000","last_change":"2024-04-25T10:12:23.253025+0000","last_active":"2024-04-25T10:12:26.012372+0000","last_peered":"2024-04-25T10:12:26.012372+0000","last_clean":"2024-04-25T10:12:26.012372+0000","last_became_active":"2024-04-25T10:12:23.251953+0000","last_became_peered":"2024-04-25T10:12:23.251953+0000","last_unstale":"2024-04-25T10:12:26.012372+0000","last_undegraded":"2024-04-25T10:12:26.012372+0000","last_fullsized":"2024-04-25T10:12:26.012372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246256+0000","last_change":"2024-04-25T10:12:24.273962+0000","last_active":"2024-04-25T10:12:25.246256+0000","last_peered":"2024-04-25T10:12:25.246256+0000","last_clean":"2024-04-25T10:12:25.246256+0000","last_became_active":"2024-04-25T10:12:24.273577+0000","last_became_peered":"2024-04-25T10:12:24.273577+0000","last_unstale":"2024-04-25T10:12:25.246256+0000","last_undegraded":"2024-04-25T10:12:25.246256+0000","last_fullsized":"2024-04-25T10:12:25.246256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246360+0000","last_change":"2024-04-25T10:12:23.247652+0000","last_active":"2024-04-25T10:12:25.246360+0000","last_peered":"2024-04-25T10:12:25.246360+0000","last_clean":"2024-04-25T10:12:25.246360+0000","last_became_active":"2024-04-25T10:12:23.243149+0000","last_became_peered":"2024-04-25T10:12:23.243149+0000","last_unstale":"2024-04-25T10:12:25.246360+0000","last_undegraded":"2024-04-25T10:12:25.246360+0000","last_fullsized":"2024-04-25T10:12:25.246360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016262+0000","last_change":"2024-04-25T10:12:24.261695+0000","last_active":"2024-04-25T10:12:26.016262+0000","last_peered":"2024-04-25T10:12:26.016262+0000","last_clean":"2024-04-25T10:12:26.016262+0000","last_became_active":"2024-04-25T10:12:24.261356+0000","last_became_peered":"2024-04-25T10:12:24.261356+0000","last_unstale":"2024-04-25T10:12:26.016262+0000","last_undegraded":"2024-04-25T10:12:26.016262+0000","last_fullsized":"2024-04-25T10:12:26.016262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247330+0000","last_change":"2024-04-25T10:12:23.244711+0000","last_active":"2024-04-25T10:12:25.247330+0000","last_peered":"2024-04-25T10:12:25.247330+0000","last_clean":"2024-04-25T10:12:25.247330+0000","last_became_active":"2024-04-25T10:12:23.244360+0000","last_became_peered":"2024-04-25T10:12:23.244360+0000","last_unstale":"2024-04-25T10:12:25.247330+0000","last_undegraded":"2024-04-25T10:12:25.247330+0000","last_fullsized":"2024-04-25T10:12:25.247330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246749+0000","last_change":"2024-04-25T10:12:24.259965+0000","last_active":"2024-04-25T10:12:25.246749+0000","last_peered":"2024-04-25T10:12:25.246749+0000","last_clean":"2024-04-25T10:12:25.246749+0000","last_became_active":"2024-04-25T10:12:24.249487+0000","last_became_peered":"2024-04-25T10:12:24.249487+0000","last_unstale":"2024-04-25T10:12:25.246749+0000","last_undegraded":"2024-04-25T10:12:25.246749+0000","last_fullsized":"2024-04-25T10:12:25.246749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:12.866317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012035+0000","last_change":"2024-04-25T10:12:23.240603+0000","last_active":"2024-04-25T10:12:26.012035+0000","last_peered":"2024-04-25T10:12:26.012035+0000","last_clean":"2024-04-25T10:12:26.012035+0000","last_became_active":"2024-04-25T10:12:23.240240+0000","last_became_peered":"2024-04-25T10:12:23.240240+0000","last_unstale":"2024-04-25T10:12:26.012035+0000","last_undegraded":"2024-04-25T10:12:26.012035+0000","last_fullsized":"2024-04-25T10:12:26.012035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015129+0000","last_change":"2024-04-25T10:12:24.259015+0000","last_active":"2024-04-25T10:12:26.015129+0000","last_peered":"2024-04-25T10:12:26.015129+0000","last_clean":"2024-04-25T10:12:26.015129+0000","last_became_active":"2024-04-25T10:12:24.258715+0000","last_became_peered":"2024-04-25T10:12:24.258715+0000","last_unstale":"2024-04-25T10:12:26.015129+0000","last_undegraded":"2024-04-25T10:12:26.015129+0000","last_fullsized":"2024-04-25T10:12:26.015129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016578+0000","last_change":"2024-04-25T10:12:23.253767+0000","last_active":"2024-04-25T10:12:26.016578+0000","last_peered":"2024-04-25T10:12:26.016578+0000","last_clean":"2024-04-25T10:12:26.016578+0000","last_became_active":"2024-04-25T10:12:23.253356+0000","last_became_peered":"2024-04-25T10:12:23.253356+0000","last_unstale":"2024-04-25T10:12:26.016578+0000","last_undegraded":"2024-04-25T10:12:26.016578+0000","last_fullsized":"2024-04-25T10:12:26.016578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016055+0000","last_change":"2024-04-25T10:12:24.241735+0000","last_active":"2024-04-25T10:12:26.016055+0000","last_peered":"2024-04-25T10:12:26.016055+0000","last_clean":"2024-04-25T10:12:26.016055+0000","last_became_active":"2024-04-25T10:12:24.241406+0000","last_became_peered":"2024-04-25T10:12:24.241406+0000","last_unstale":"2024-04-25T10:12:26.016055+0000","last_undegraded":"2024-04-25T10:12:26.016055+0000","last_fullsized":"2024-04-25T10:12:26.016055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:27.592446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246379+0000","last_change":"2024-04-25T10:12:23.249072+0000","last_active":"2024-04-25T10:12:25.246379+0000","last_peered":"2024-04-25T10:12:25.246379+0000","last_clean":"2024-04-25T10:12:25.246379+0000","last_became_active":"2024-04-25T10:12:23.248691+0000","last_became_peered":"2024-04-25T10:12:23.248691+0000","last_unstale":"2024-04-25T10:12:25.246379+0000","last_undegraded":"2024-04-25T10:12:25.246379+0000","last_fullsized":"2024-04-25T10:12:25.246379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015097+0000","last_change":"2024-04-25T10:12:24.269693+0000","last_active":"2024-04-25T10:12:26.015097+0000","last_peered":"2024-04-25T10:12:26.015097+0000","last_clean":"2024-04-25T10:12:26.015097+0000","last_became_active":"2024-04-25T10:12:24.269423+0000","last_became_peered":"2024-04-25T10:12:24.269423+0000","last_unstale":"2024-04-25T10:12:26.015097+0000","last_undegraded":"2024-04-25T10:12:26.015097+0000","last_fullsized":"2024-04-25T10:12:26.015097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011686+0000","last_change":"2024-04-25T10:12:23.246678+0000","last_active":"2024-04-25T10:12:26.011686+0000","last_peered":"2024-04-25T10:12:26.011686+0000","last_clean":"2024-04-25T10:12:26.011686+0000","last_became_active":"2024-04-25T10:12:23.246288+0000","last_became_peered":"2024-04-25T10:12:23.246288+0000","last_unstale":"2024-04-25T10:12:26.011686+0000","last_undegraded":"2024-04-25T10:12:26.011686+0000","last_fullsized":"2024-04-25T10:12:26.011686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011782+0000","last_change":"2024-04-25T10:12:24.245870+0000","last_active":"2024-04-25T10:12:26.011782+0000","last_peered":"2024-04-25T10:12:26.011782+0000","last_clean":"2024-04-25T10:12:26.011782+0000","last_became_active":"2024-04-25T10:12:24.244284+0000","last_became_peered":"2024-04-25T10:12:24.244284+0000","last_unstale":"2024-04-25T10:12:26.011782+0000","last_undegraded":"2024-04-25T10:12:26.011782+0000","last_fullsized":"2024-04-25T10:12:26.011782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.871923+0000","last_change":"2024-04-25T10:12:23.232086+0000","last_active":"2024-04-25T10:12:25.871923+0000","last_peered":"2024-04-25T10:12:25.871923+0000","last_clean":"2024-04-25T10:12:25.871923+0000","last_became_active":"2024-04-25T10:12:23.231694+0000","last_became_peered":"2024-04-25T10:12:23.231694+0000","last_unstale":"2024-04-25T10:12:25.871923+0000","last_undegraded":"2024-04-25T10:12:25.871923+0000","last_fullsized":"2024-04-25T10:12:25.871923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014903+0000","last_change":"2024-04-25T10:12:24.252131+0000","last_active":"2024-04-25T10:12:26.014903+0000","last_peered":"2024-04-25T10:12:26.014903+0000","last_clean":"2024-04-25T10:12:26.014903+0000","last_became_active":"2024-04-25T10:12:24.251807+0000","last_became_peered":"2024-04-25T10:12:24.251807+0000","last_unstale":"2024-04-25T10:12:26.014903+0000","last_undegraded":"2024-04-25T10:12:26.014903+0000","last_fullsized":"2024-04-25T10:12:26.014903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246755+0000","last_change":"2024-04-25T10:12:23.243507+0000","last_active":"2024-04-25T10:12:25.246755+0000","last_peered":"2024-04-25T10:12:25.246755+0000","last_clean":"2024-04-25T10:12:25.246755+0000","last_became_active":"2024-04-25T10:12:23.243082+0000","last_became_peered":"2024-04-25T10:12:23.243082+0000","last_unstale":"2024-04-25T10:12:25.246755+0000","last_undegraded":"2024-04-25T10:12:25.246755+0000","last_fullsized":"2024-04-25T10:12:25.246755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:09.593840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015480+0000","last_change":"2024-04-25T10:12:24.270975+0000","last_active":"2024-04-25T10:12:26.015480+0000","last_peered":"2024-04-25T10:12:26.015480+0000","last_clean":"2024-04-25T10:12:26.015480+0000","last_became_active":"2024-04-25T10:12:24.270678+0000","last_became_peered":"2024-04-25T10:12:24.270678+0000","last_unstale":"2024-04-25T10:12:26.015480+0000","last_undegraded":"2024-04-25T10:12:26.015480+0000","last_fullsized":"2024-04-25T10:12:26.015480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872082+0000","last_change":"2024-04-25T10:12:23.242355+0000","last_active":"2024-04-25T10:12:25.872082+0000","last_peered":"2024-04-25T10:12:25.872082+0000","last_clean":"2024-04-25T10:12:25.872082+0000","last_became_active":"2024-04-25T10:12:23.242038+0000","last_became_peered":"2024-04-25T10:12:23.242038+0000","last_unstale":"2024-04-25T10:12:25.872082+0000","last_undegraded":"2024-04-25T10:12:25.872082+0000","last_fullsized":"2024-04-25T10:12:25.872082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015634+0000","last_change":"2024-04-25T10:12:24.270959+0000","last_active":"2024-04-25T10:12:26.015634+0000","last_peered":"2024-04-25T10:12:26.015634+0000","last_clean":"2024-04-25T10:12:26.015634+0000","last_became_active":"2024-04-25T10:12:24.270658+0000","last_became_peered":"2024-04-25T10:12:24.270658+0000","last_unstale":"2024-04-25T10:12:26.015634+0000","last_undegraded":"2024-04-25T10:12:26.015634+0000","last_fullsized":"2024-04-25T10:12:26.015634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247343+0000","last_change":"2024-04-25T10:12:23.255897+0000","last_active":"2024-04-25T10:12:25.247343+0000","last_peered":"2024-04-25T10:12:25.247343+0000","last_clean":"2024-04-25T10:12:25.247343+0000","last_became_active":"2024-04-25T10:12:23.255626+0000","last_became_peered":"2024-04-25T10:12:23.255626+0000","last_unstale":"2024-04-25T10:12:25.247343+0000","last_undegraded":"2024-04-25T10:12:25.247343+0000","last_fullsized":"2024-04-25T10:12:25.247343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:51.408775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246715+0000","last_change":"2024-04-25T10:12:24.262419+0000","last_active":"2024-04-25T10:12:25.246715+0000","last_peered":"2024-04-25T10:12:25.246715+0000","last_clean":"2024-04-25T10:12:25.246715+0000","last_became_active":"2024-04-25T10:12:24.262163+0000","last_became_peered":"2024-04-25T10:12:24.262163+0000","last_unstale":"2024-04-25T10:12:25.246715+0000","last_undegraded":"2024-04-25T10:12:25.246715+0000","last_fullsized":"2024-04-25T10:12:25.246715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872045+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:25.872045+0000","last_peered":"2024-04-25T10:12:25.872045+0000","last_clean":"2024-04-25T10:12:25.872045+0000","last_became_active":"2024-04-25T10:12:23.249437+0000","last_became_peered":"2024-04-25T10:12:23.249437+0000","last_unstale":"2024-04-25T10:12:25.872045+0000","last_undegraded":"2024-04-25T10:12:25.872045+0000","last_fullsized":"2024-04-25T10:12:25.872045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015325+0000","last_change":"2024-04-25T10:12:24.266865+0000","last_active":"2024-04-25T10:12:26.015325+0000","last_peered":"2024-04-25T10:12:26.015325+0000","last_clean":"2024-04-25T10:12:26.015325+0000","last_became_active":"2024-04-25T10:12:24.266595+0000","last_became_peered":"2024-04-25T10:12:24.266595+0000","last_unstale":"2024-04-25T10:12:26.015325+0000","last_undegraded":"2024-04-25T10:12:26.015325+0000","last_fullsized":"2024-04-25T10:12:26.015325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015021+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:12:26.015021+0000","last_peered":"2024-04-25T10:12:26.015021+0000","last_clean":"2024-04-25T10:12:26.015021+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:12:26.015021+0000","last_undegraded":"2024-04-25T10:12:26.015021+0000","last_fullsized":"2024-04-25T10:12:26.015021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245447+0000","last_change":"2024-04-25T10:12:23.236000+0000","last_active":"2024-04-25T10:12:25.245447+0000","last_peered":"2024-04-25T10:12:25.245447+0000","last_clean":"2024-04-25T10:12:25.245447+0000","last_became_active":"2024-04-25T10:12:23.235473+0000","last_became_peered":"2024-04-25T10:12:23.235473+0000","last_unstale":"2024-04-25T10:12:25.245447+0000","last_undegraded":"2024-04-25T10:12:25.245447+0000","last_fullsized":"2024-04-25T10:12:25.245447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016099+0000","last_change":"2024-04-25T10:12:24.269413+0000","last_active":"2024-04-25T10:12:26.016099+0000","last_peered":"2024-04-25T10:12:26.016099+0000","last_clean":"2024-04-25T10:12:26.016099+0000","last_became_active":"2024-04-25T10:12:24.269109+0000","last_became_peered":"2024-04-25T10:12:24.269109+0000","last_unstale":"2024-04-25T10:12:26.016099+0000","last_undegraded":"2024-04-25T10:12:26.016099+0000","last_fullsized":"2024-04-25T10:12:26.016099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014753+0000","last_change":"2024-04-25T10:12:23.251616+0000","last_active":"2024-04-25T10:12:26.014753+0000","last_peered":"2024-04-25T10:12:26.014753+0000","last_clean":"2024-04-25T10:12:26.014753+0000","last_became_active":"2024-04-25T10:12:23.251356+0000","last_became_peered":"2024-04-25T10:12:23.251356+0000","last_unstale":"2024-04-25T10:12:26.014753+0000","last_undegraded":"2024-04-25T10:12:26.014753+0000","last_fullsized":"2024-04-25T10:12:26.014753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015666+0000","last_change":"2024-04-25T10:12:24.252497+0000","last_active":"2024-04-25T10:12:26.015666+0000","last_peered":"2024-04-25T10:12:26.015666+0000","last_clean":"2024-04-25T10:12:26.015666+0000","last_became_active":"2024-04-25T10:12:24.252221+0000","last_became_peered":"2024-04-25T10:12:24.252221+0000","last_unstale":"2024-04-25T10:12:26.015666+0000","last_undegraded":"2024-04-25T10:12:26.015666+0000","last_fullsized":"2024-04-25T10:12:26.015666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:52.114948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872039+0000","last_change":"2024-04-25T10:12:23.235336+0000","last_active":"2024-04-25T10:12:25.872039+0000","last_peered":"2024-04-25T10:12:25.872039+0000","last_clean":"2024-04-25T10:12:25.872039+0000","last_became_active":"2024-04-25T10:12:23.235016+0000","last_became_peered":"2024-04-25T10:12:23.235016+0000","last_unstale":"2024-04-25T10:12:25.872039+0000","last_undegraded":"2024-04-25T10:12:25.872039+0000","last_fullsized":"2024-04-25T10:12:25.872039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246990+0000","last_change":"2024-04-25T10:12:24.264913+0000","last_active":"2024-04-25T10:12:25.246990+0000","last_peered":"2024-04-25T10:12:25.246990+0000","last_clean":"2024-04-25T10:12:25.246990+0000","last_became_active":"2024-04-25T10:12:24.264498+0000","last_became_peered":"2024-04-25T10:12:24.264498+0000","last_unstale":"2024-04-25T10:12:25.246990+0000","last_undegraded":"2024-04-25T10:12:25.246990+0000","last_fullsized":"2024-04-25T10:12:25.246990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246493+0000","last_change":"2024-04-25T10:12:23.241698+0000","last_active":"2024-04-25T10:12:25.246493+0000","last_peered":"2024-04-25T10:12:25.246493+0000","last_clean":"2024-04-25T10:12:25.246493+0000","last_became_active":"2024-04-25T10:12:23.241337+0000","last_became_peered":"2024-04-25T10:12:23.241337+0000","last_unstale":"2024-04-25T10:12:25.246493+0000","last_undegraded":"2024-04-25T10:12:25.246493+0000","last_fullsized":"2024-04-25T10:12:25.246493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246479+0000","last_change":"2024-04-25T10:12:24.249493+0000","last_active":"2024-04-25T10:12:25.246479+0000","last_peered":"2024-04-25T10:12:25.246479+0000","last_clean":"2024-04-25T10:12:25.246479+0000","last_became_active":"2024-04-25T10:12:24.249067+0000","last_became_peered":"2024-04-25T10:12:24.249067+0000","last_unstale":"2024-04-25T10:12:25.246479+0000","last_undegraded":"2024-04-25T10:12:25.246479+0000","last_fullsized":"2024-04-25T10:12:25.246479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.231393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247191+0000","last_change":"2024-04-25T10:12:23.243659+0000","last_active":"2024-04-25T10:12:25.247191+0000","last_peered":"2024-04-25T10:12:25.247191+0000","last_clean":"2024-04-25T10:12:25.247191+0000","last_became_active":"2024-04-25T10:12:23.243098+0000","last_became_peered":"2024-04-25T10:12:23.243098+0000","last_unstale":"2024-04-25T10:12:25.247191+0000","last_undegraded":"2024-04-25T10:12:25.247191+0000","last_fullsized":"2024-04-25T10:12:25.247191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245987+0000","last_change":"2024-04-25T10:12:24.264189+0000","last_active":"2024-04-25T10:12:25.245987+0000","last_peered":"2024-04-25T10:12:25.245987+0000","last_clean":"2024-04-25T10:12:25.245987+0000","last_became_active":"2024-04-25T10:12:24.258438+0000","last_became_peered":"2024-04-25T10:12:24.258438+0000","last_unstale":"2024-04-25T10:12:25.245987+0000","last_undegraded":"2024-04-25T10:12:25.245987+0000","last_fullsized":"2024-04-25T10:12:25.245987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015445+0000","last_change":"2024-04-25T10:12:23.254569+0000","last_active":"2024-04-25T10:12:26.015445+0000","last_peered":"2024-04-25T10:12:26.015445+0000","last_clean":"2024-04-25T10:12:26.015445+0000","last_became_active":"2024-04-25T10:12:23.254228+0000","last_became_peered":"2024-04-25T10:12:23.254228+0000","last_unstale":"2024-04-25T10:12:26.015445+0000","last_undegraded":"2024-04-25T10:12:26.015445+0000","last_fullsized":"2024-04-25T10:12:26.015445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:12.723714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247192+0000","last_change":"2024-04-25T10:12:24.258387+0000","last_active":"2024-04-25T10:12:25.247192+0000","last_peered":"2024-04-25T10:12:25.247192+0000","last_clean":"2024-04-25T10:12:25.247192+0000","last_became_active":"2024-04-25T10:12:24.257160+0000","last_became_peered":"2024-04-25T10:12:24.257160+0000","last_unstale":"2024-04-25T10:12:25.247192+0000","last_undegraded":"2024-04-25T10:12:25.247192+0000","last_fullsized":"2024-04-25T10:12:25.247192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247441+0000","last_change":"2024-04-25T10:12:23.247450+0000","last_active":"2024-04-25T10:12:25.247441+0000","last_peered":"2024-04-25T10:12:25.247441+0000","last_clean":"2024-04-25T10:12:25.247441+0000","last_became_active":"2024-04-25T10:12:23.247166+0000","last_became_peered":"2024-04-25T10:12:23.247166+0000","last_unstale":"2024-04-25T10:12:25.247441+0000","last_undegraded":"2024-04-25T10:12:25.247441+0000","last_fullsized":"2024-04-25T10:12:25.247441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016516+0000","last_change":"2024-04-25T10:12:24.271046+0000","last_active":"2024-04-25T10:12:26.016516+0000","last_peered":"2024-04-25T10:12:26.016516+0000","last_clean":"2024-04-25T10:12:26.016516+0000","last_became_active":"2024-04-25T10:12:24.270697+0000","last_became_peered":"2024-04-25T10:12:24.270697+0000","last_unstale":"2024-04-25T10:12:26.016516+0000","last_undegraded":"2024-04-25T10:12:26.016516+0000","last_fullsized":"2024-04-25T10:12:26.016516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012514+0000","last_change":"2024-04-25T10:12:23.252034+0000","last_active":"2024-04-25T10:12:26.012514+0000","last_peered":"2024-04-25T10:12:26.012514+0000","last_clean":"2024-04-25T10:12:26.012514+0000","last_became_active":"2024-04-25T10:12:23.251537+0000","last_became_peered":"2024-04-25T10:12:23.251537+0000","last_unstale":"2024-04-25T10:12:26.012514+0000","last_undegraded":"2024-04-25T10:12:26.012514+0000","last_fullsized":"2024-04-25T10:12:26.012514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016121+0000","last_change":"2024-04-25T10:12:24.247147+0000","last_active":"2024-04-25T10:12:26.016121+0000","last_peered":"2024-04-25T10:12:26.016121+0000","last_clean":"2024-04-25T10:12:26.016121+0000","last_became_active":"2024-04-25T10:12:24.246897+0000","last_became_peered":"2024-04-25T10:12:24.246897+0000","last_unstale":"2024-04-25T10:12:26.016121+0000","last_undegraded":"2024-04-25T10:12:26.016121+0000","last_fullsized":"2024-04-25T10:12:26.016121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014793+0000","last_change":"2024-04-25T10:12:23.254396+0000","last_active":"2024-04-25T10:12:26.014793+0000","last_peered":"2024-04-25T10:12:26.014793+0000","last_clean":"2024-04-25T10:12:26.014793+0000","last_became_active":"2024-04-25T10:12:23.254125+0000","last_became_peered":"2024-04-25T10:12:23.254125+0000","last_unstale":"2024-04-25T10:12:26.014793+0000","last_undegraded":"2024-04-25T10:12:26.014793+0000","last_fullsized":"2024-04-25T10:12:26.014793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247014+0000","last_change":"2024-04-25T10:12:24.261214+0000","last_active":"2024-04-25T10:12:25.247014+0000","last_peered":"2024-04-25T10:12:25.247014+0000","last_clean":"2024-04-25T10:12:25.247014+0000","last_became_active":"2024-04-25T10:12:24.260895+0000","last_became_peered":"2024-04-25T10:12:24.260895+0000","last_unstale":"2024-04-25T10:12:25.247014+0000","last_undegraded":"2024-04-25T10:12:25.247014+0000","last_fullsized":"2024-04-25T10:12:25.247014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247067+0000","last_change":"2024-04-25T10:12:23.244316+0000","last_active":"2024-04-25T10:12:25.247067+0000","last_peered":"2024-04-25T10:12:25.247067+0000","last_clean":"2024-04-25T10:12:25.247067+0000","last_became_active":"2024-04-25T10:12:23.243075+0000","last_became_peered":"2024-04-25T10:12:23.243075+0000","last_unstale":"2024-04-25T10:12:25.247067+0000","last_undegraded":"2024-04-25T10:12:25.247067+0000","last_fullsized":"2024-04-25T10:12:25.247067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016532+0000","last_change":"2024-04-25T10:12:24.252644+0000","last_active":"2024-04-25T10:12:26.016532+0000","last_peered":"2024-04-25T10:12:26.016532+0000","last_clean":"2024-04-25T10:12:26.016532+0000","last_became_active":"2024-04-25T10:12:24.252317+0000","last_became_peered":"2024-04-25T10:12:24.252317+0000","last_unstale":"2024-04-25T10:12:26.016532+0000","last_undegraded":"2024-04-25T10:12:26.016532+0000","last_fullsized":"2024-04-25T10:12:26.016532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247152+0000","last_change":"2024-04-25T10:12:23.243515+0000","last_active":"2024-04-25T10:12:25.247152+0000","last_peered":"2024-04-25T10:12:25.247152+0000","last_clean":"2024-04-25T10:12:25.247152+0000","last_became_active":"2024-04-25T10:12:23.242920+0000","last_became_peered":"2024-04-25T10:12:23.242920+0000","last_unstale":"2024-04-25T10:12:25.247152+0000","last_undegraded":"2024-04-25T10:12:25.247152+0000","last_fullsized":"2024-04-25T10:12:25.247152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016097+0000","last_change":"2024-04-25T10:12:24.256479+0000","last_active":"2024-04-25T10:12:26.016097+0000","last_peered":"2024-04-25T10:12:26.016097+0000","last_clean":"2024-04-25T10:12:26.016097+0000","last_became_active":"2024-04-25T10:12:24.256149+0000","last_became_peered":"2024-04-25T10:12:24.256149+0000","last_unstale":"2024-04-25T10:12:26.016097+0000","last_undegraded":"2024-04-25T10:12:26.016097+0000","last_fullsized":"2024-04-25T10:12:26.016097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015287+0000","last_change":"2024-04-25T10:12:23.258511+0000","last_active":"2024-04-25T10:12:26.015287+0000","last_peered":"2024-04-25T10:12:26.015287+0000","last_clean":"2024-04-25T10:12:26.015287+0000","last_became_active":"2024-04-25T10:12:23.258235+0000","last_became_peered":"2024-04-25T10:12:23.258235+0000","last_unstale":"2024-04-25T10:12:26.015287+0000","last_undegraded":"2024-04-25T10:12:26.015287+0000","last_fullsized":"2024-04-25T10:12:26.015287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012230+0000","last_change":"2024-04-25T10:12:24.245339+0000","last_active":"2024-04-25T10:12:26.012230+0000","last_peered":"2024-04-25T10:12:26.012230+0000","last_clean":"2024-04-25T10:12:26.012230+0000","last_became_active":"2024-04-25T10:12:24.241842+0000","last_became_peered":"2024-04-25T10:12:24.241842+0000","last_unstale":"2024-04-25T10:12:26.012230+0000","last_undegraded":"2024-04-25T10:12:26.012230+0000","last_fullsized":"2024-04-25T10:12:26.012230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245817+0000","last_change":"2024-04-25T10:12:24.268837+0000","last_active":"2024-04-25T10:12:25.245817+0000","last_peered":"2024-04-25T10:12:25.245817+0000","last_clean":"2024-04-25T10:12:25.245817+0000","last_became_active":"2024-04-25T10:12:24.268028+0000","last_became_peered":"2024-04-25T10:12:24.268028+0000","last_unstale":"2024-04-25T10:12:25.245817+0000","last_undegraded":"2024-04-25T10:12:25.245817+0000","last_fullsized":"2024-04-25T10:12:25.245817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015045+0000","last_change":"2024-04-25T10:12:23.259889+0000","last_active":"2024-04-25T10:12:26.015045+0000","last_peered":"2024-04-25T10:12:26.015045+0000","last_clean":"2024-04-25T10:12:26.015045+0000","last_became_active":"2024-04-25T10:12:23.259546+0000","last_became_peered":"2024-04-25T10:12:23.259546+0000","last_unstale":"2024-04-25T10:12:26.015045+0000","last_undegraded":"2024-04-25T10:12:26.015045+0000","last_fullsized":"2024-04-25T10:12:26.015045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.267238+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.266952+0000","last_became_peered":"2024-04-25T10:12:24.266952+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015266+0000","last_change":"2024-04-25T10:12:23.258590+0000","last_active":"2024-04-25T10:12:26.015266+0000","last_peered":"2024-04-25T10:12:26.015266+0000","last_clean":"2024-04-25T10:12:26.015266+0000","last_became_active":"2024-04-25T10:12:23.258334+0000","last_became_peered":"2024-04-25T10:12:23.258334+0000","last_unstale":"2024-04-25T10:12:26.015266+0000","last_undegraded":"2024-04-25T10:12:26.015266+0000","last_fullsized":"2024-04-25T10:12:26.015266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016106+0000","last_change":"2024-04-25T10:12:24.261806+0000","last_active":"2024-04-25T10:12:26.016106+0000","last_peered":"2024-04-25T10:12:26.016106+0000","last_clean":"2024-04-25T10:12:26.016106+0000","last_became_active":"2024-04-25T10:12:24.261462+0000","last_became_peered":"2024-04-25T10:12:24.261462+0000","last_unstale":"2024-04-25T10:12:26.016106+0000","last_undegraded":"2024-04-25T10:12:26.016106+0000","last_fullsized":"2024-04-25T10:12:26.016106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016336+0000","last_change":"2024-04-25T10:12:23.246028+0000","last_active":"2024-04-25T10:12:26.016336+0000","last_peered":"2024-04-25T10:12:26.016336+0000","last_clean":"2024-04-25T10:12:26.016336+0000","last_became_active":"2024-04-25T10:12:23.245746+0000","last_became_peered":"2024-04-25T10:12:23.245746+0000","last_unstale":"2024-04-25T10:12:26.016336+0000","last_undegraded":"2024-04-25T10:12:26.016336+0000","last_fullsized":"2024-04-25T10:12:26.016336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016375+0000","last_change":"2024-04-25T10:12:24.265910+0000","last_active":"2024-04-25T10:12:26.016375+0000","last_peered":"2024-04-25T10:12:26.016375+0000","last_clean":"2024-04-25T10:12:26.016375+0000","last_became_active":"2024-04-25T10:12:24.265598+0000","last_became_peered":"2024-04-25T10:12:24.265598+0000","last_unstale":"2024-04-25T10:12:26.016375+0000","last_undegraded":"2024-04-25T10:12:26.016375+0000","last_fullsized":"2024-04-25T10:12:26.016375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016395+0000","last_change":"2024-04-25T10:12:23.252781+0000","last_active":"2024-04-25T10:12:26.016395+0000","last_peered":"2024-04-25T10:12:26.016395+0000","last_clean":"2024-04-25T10:12:26.016395+0000","last_became_active":"2024-04-25T10:12:23.252526+0000","last_became_peered":"2024-04-25T10:12:23.252526+0000","last_unstale":"2024-04-25T10:12:26.016395+0000","last_undegraded":"2024-04-25T10:12:26.016395+0000","last_fullsized":"2024-04-25T10:12:26.016395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:10.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:28.318 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.484+0000 7f35a0c00640 1 Processor -- start 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.484+0000 7f35a0c00640 1 -- start start 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f35a0c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f359c0cda10 0x7f359c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f35a0c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0b2770 con 0x7f359c0cf930 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f35a0c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c06c670 con 0x7f359c15bc40 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f35a0c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c06c950 con 0x7f359c0cda10 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f3591400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f359c0cda10 0x7f359c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f3591400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f359c0cda10 0x7f359c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56944/0 (socket says 172.21.15.87:56944) 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f3591400640 1 -- 172.21.15.87:0/2596600853 learned_addr learned my addr 172.21.15.87:0/2596600853 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f3591400640 1 -- 172.21.15.87:0/2596600853 >> v1:172.21.15.169:6789/0 conn(0x7f359c15bc40 legacy=0x7f359c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f3591400640 1 -- 172.21.15.87:0/2596600853 >> v1:172.21.15.87:6789/0 conn(0x7f359c0cf930 legacy=0x7f359c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.488+0000 7f3591400640 1 -- 172.21.15.87:0/2596600853 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c06cc30 con 0x7f359c0cda10 2024-04-25T10:12:28.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f3591400640 1 --2- 172.21.15.87:0/2596600853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c0cda10 0x7f359c06c090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3584009870 tx=0x7f358401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e9b4dbc11da56f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f3599400640 1 -- 172.21.15.87:0/2596600853 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840b0070 con 0x7f359c0cda10 2024-04-25T10:12:28.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f3599400640 1 -- 172.21.15.87:0/2596600853 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840ab600 con 0x7f359c0cda10 2024-04-25T10:12:28.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f3599400640 1 -- 172.21.15.87:0/2596600853 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840ab9d0 con 0x7f359c0cda10 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- 172.21.15.87:0/2596600853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c0cda10 msgr2=0x7f359c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2596600853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c0cda10 0x7f359c06c090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3584009870 tx=0x7f358401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- 172.21.15.87:0/2596600853 shutdown_connections 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2596600853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c0cda10 0x7f359c06c090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- 172.21.15.87:0/2596600853 >> 172.21.15.87:0/2596600853 conn(0x7f359c066b70 msgr2=0x7f359c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- 172.21.15.87:0/2596600853 shutdown_connections 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- 172.21.15.87:0/2596600853 wait complete. 2024-04-25T10:12:28.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 Processor -- start 2024-04-25T10:12:28.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- start start 2024-04-25T10:12:28.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359c0cda10 0x7f359c14f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f359c0cf930 0x7f359c1484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c15bc40 0x7f359c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c169d70 con 0x7f359c15bc40 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c169bf0 con 0x7f359c0cda10 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.492+0000 7f35a0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c169ef0 con 0x7f359c0cf930 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c15bc40 0x7f359c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c15bc40 0x7f359c15ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56956/0 (socket says 172.21.15.87:56956) 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 -- 172.21.15.87:0/2412447813 learned_addr learned my addr 172.21.15.87:0/2412447813 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f3599e00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359c0cda10 0x7f359c14f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f3591400640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f359c0cf930 0x7f359c1484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 -- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f359c0cf930 msgr2=0x7f359c1484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f359c0cf930 0x7f359c1484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 -- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359c0cda10 msgr2=0x7f359c14f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359c0cda10 0x7f359c14f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 -- 172.21.15.87:0/2412447813 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c1f68c0 con 0x7f359c15bc40 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f359a800640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c15bc40 0x7f359c15ca80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f359400c770 tx=0x7f359400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f3592a00640 1 -- 172.21.15.87:0/2412447813 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594003b80 con 0x7f359c15bc40 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f3592a00640 1 -- 172.21.15.87:0/2412447813 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3594003d20 con 0x7f359c15bc40 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f3592a00640 1 -- 172.21.15.87:0/2412447813 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359400aa90 con 0x7f359c15bc40 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c1f6ba0 con 0x7f359c15bc40 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c1f7160 con 0x7f359c15bc40 2024-04-25T10:12:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c04ef30 con 0x7f359c15bc40 2024-04-25T10:12:28.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f3592a00640 1 -- 172.21.15.87:0/2412447813 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f359400ac30 con 0x7f359c15bc40 2024-04-25T10:12:28.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.496+0000 7f3592a00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3578066f90 0x7f3578069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.500+0000 7f3592a00640 1 -- 172.21.15.87:0/2412447813 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3594084e90 con 0x7f359c15bc40 2024-04-25T10:12:28.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.500+0000 7f3599e00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3578066f90 0x7f3578069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.500+0000 7f3599e00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3578066f90 0x7f3578069450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f358c007fd0 tx=0x7f358c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.504+0000 7f3592a00640 1 -- 172.21.15.87:0/2412447813 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35940557b0 con 0x7f359c15bc40 2024-04-25T10:12:28.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.652+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f359c06d960 con 0x7f3578066f90 2024-04-25T10:12:28.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.660+0000 7f3592a00640 1 -- 172.21.15.87:0/2412447813 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251942 (secure 0 0 0) 0x7f359c06d960 con 0x7f3578066f90 2024-04-25T10:12:28.667 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3578066f90 msgr2=0x7f3578069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3578066f90 0x7f3578069450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f358c007fd0 tx=0x7f358c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c15bc40 msgr2=0x7f359c15ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c15bc40 0x7f359c15ca80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f359400c770 tx=0x7f359400cc30 comp rx=0 tx=0).stop 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 shutdown_connections 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3578066f90 0x7f3578069450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f359c15bc40 0x7f359c15ca80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f359c0cf930 0x7f359c1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 --2- 172.21.15.87:0/2412447813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359c0cda10 0x7f359c14f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 >> 172.21.15.87:0/2412447813 conn(0x7f359c066b70 msgr2=0x7f359c159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 shutdown_connections 2024-04-25T10:12:28.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.664+0000 7f35a0c00640 1 -- 172.21.15.87:0/2412447813 wait complete. 2024-04-25T10:12:28.687 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:12:27.125402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218940,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755780,"statfs":{"total":773094113280,"available":772869918720,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.910725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015534+0000","last_change":"2024-04-25T10:12:24.263197+0000","last_active":"2024-04-25T10:12:26.015534+0000","last_peered":"2024-04-25T10:12:26.015534+0000","last_clean":"2024-04-25T10:12:26.015534+0000","last_became_active":"2024-04-25T10:12:24.262932+0000","last_became_peered":"2024-04-25T10:12:24.262932+0000","last_unstale":"2024-04-25T10:12:26.015534+0000","last_undegraded":"2024-04-25T10:12:26.015534+0000","last_fullsized":"2024-04-25T10:12:26.015534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247413+0000","last_change":"2024-04-25T10:12:24.255491+0000","last_active":"2024-04-25T10:12:25.247413+0000","last_peered":"2024-04-25T10:12:25.247413+0000","last_clean":"2024-04-25T10:12:25.247413+0000","last_became_active":"2024-04-25T10:12:24.255325+0000","last_became_peered":"2024-04-25T10:12:24.255325+0000","last_unstale":"2024-04-25T10:12:25.247413+0000","last_undegraded":"2024-04-25T10:12:25.247413+0000","last_fullsized":"2024-04-25T10:12:25.247413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016327+0000","last_change":"2024-04-25T10:12:24.259328+0000","last_active":"2024-04-25T10:12:26.016327+0000","last_peered":"2024-04-25T10:12:26.016327+0000","last_clean":"2024-04-25T10:12:26.016327+0000","last_became_active":"2024-04-25T10:12:24.259065+0000","last_became_peered":"2024-04-25T10:12:24.259065+0000","last_unstale":"2024-04-25T10:12:26.016327+0000","last_undegraded":"2024-04-25T10:12:26.016327+0000","last_fullsized":"2024-04-25T10:12:26.016327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247574+0000","last_change":"2024-04-25T10:12:24.263211+0000","last_active":"2024-04-25T10:12:25.247574+0000","last_peered":"2024-04-25T10:12:25.247574+0000","last_clean":"2024-04-25T10:12:25.247574+0000","last_became_active":"2024-04-25T10:12:24.262896+0000","last_became_peered":"2024-04-25T10:12:24.262896+0000","last_unstale":"2024-04-25T10:12:25.247574+0000","last_undegraded":"2024-04-25T10:12:25.247574+0000","last_fullsized":"2024-04-25T10:12:25.247574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014990+0000","last_change":"2024-04-25T10:12:24.259922+0000","last_active":"2024-04-25T10:12:26.014990+0000","last_peered":"2024-04-25T10:12:26.014990+0000","last_clean":"2024-04-25T10:12:26.014990+0000","last_became_active":"2024-04-25T10:12:24.259657+0000","last_became_peered":"2024-04-25T10:12:24.259657+0000","last_unstale":"2024-04-25T10:12:26.014990+0000","last_undegraded":"2024-04-25T10:12:26.014990+0000","last_fullsized":"2024-04-25T10:12:26.014990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245975+0000","last_change":"2024-04-25T10:12:24.254906+0000","last_active":"2024-04-25T10:12:25.245975+0000","last_peered":"2024-04-25T10:12:25.245975+0000","last_clean":"2024-04-25T10:12:25.245975+0000","last_became_active":"2024-04-25T10:12:24.254592+0000","last_became_peered":"2024-04-25T10:12:24.254592+0000","last_unstale":"2024-04-25T10:12:25.245975+0000","last_undegraded":"2024-04-25T10:12:25.245975+0000","last_fullsized":"2024-04-25T10:12:25.245975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014929+0000","last_change":"2024-04-25T10:12:24.254975+0000","last_active":"2024-04-25T10:12:26.014929+0000","last_peered":"2024-04-25T10:12:26.014929+0000","last_clean":"2024-04-25T10:12:26.014929+0000","last_became_active":"2024-04-25T10:12:24.254639+0000","last_became_peered":"2024-04-25T10:12:24.254639+0000","last_unstale":"2024-04-25T10:12:26.014929+0000","last_undegraded":"2024-04-25T10:12:26.014929+0000","last_fullsized":"2024-04-25T10:12:26.014929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015387+0000","last_change":"2024-04-25T10:12:24.264638+0000","last_active":"2024-04-25T10:12:26.015387+0000","last_peered":"2024-04-25T10:12:26.015387+0000","last_clean":"2024-04-25T10:12:26.015387+0000","last_became_active":"2024-04-25T10:12:24.264376+0000","last_became_peered":"2024-04-25T10:12:24.264376+0000","last_unstale":"2024-04-25T10:12:26.015387+0000","last_undegraded":"2024-04-25T10:12:26.015387+0000","last_fullsized":"2024-04-25T10:12:26.015387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015572+0000","last_change":"2024-04-25T10:12:24.257501+0000","last_active":"2024-04-25T10:12:26.015572+0000","last_peered":"2024-04-25T10:12:26.015572+0000","last_clean":"2024-04-25T10:12:26.015572+0000","last_became_active":"2024-04-25T10:12:24.257199+0000","last_became_peered":"2024-04-25T10:12:24.257199+0000","last_unstale":"2024-04-25T10:12:26.015572+0000","last_undegraded":"2024-04-25T10:12:26.015572+0000","last_fullsized":"2024-04-25T10:12:26.015572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012594+0000","last_change":"2024-04-25T10:12:24.250894+0000","last_active":"2024-04-25T10:12:26.012594+0000","last_peered":"2024-04-25T10:12:26.012594+0000","last_clean":"2024-04-25T10:12:26.012594+0000","last_became_active":"2024-04-25T10:12:24.250581+0000","last_became_peered":"2024-04-25T10:12:24.250581+0000","last_unstale":"2024-04-25T10:12:26.012594+0000","last_undegraded":"2024-04-25T10:12:26.012594+0000","last_fullsized":"2024-04-25T10:12:26.012594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012005+0000","last_change":"2024-04-25T10:12:24.247801+0000","last_active":"2024-04-25T10:12:26.012005+0000","last_peered":"2024-04-25T10:12:26.012005+0000","last_clean":"2024-04-25T10:12:26.012005+0000","last_became_active":"2024-04-25T10:12:24.247485+0000","last_became_peered":"2024-04-25T10:12:24.247485+0000","last_unstale":"2024-04-25T10:12:26.012005+0000","last_undegraded":"2024-04-25T10:12:26.012005+0000","last_fullsized":"2024-04-25T10:12:26.012005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.262042+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.261770+0000","last_became_peered":"2024-04-25T10:12:24.261770+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:36.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015914+0000","last_change":"2024-04-25T10:12:24.263554+0000","last_active":"2024-04-25T10:12:26.015914+0000","last_peered":"2024-04-25T10:12:26.015914+0000","last_clean":"2024-04-25T10:12:26.015914+0000","last_became_active":"2024-04-25T10:12:24.263244+0000","last_became_peered":"2024-04-25T10:12:24.263244+0000","last_unstale":"2024-04-25T10:12:26.015914+0000","last_undegraded":"2024-04-25T10:12:26.015914+0000","last_fullsized":"2024-04-25T10:12:26.015914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015884+0000","last_change":"2024-04-25T10:12:24.246682+0000","last_active":"2024-04-25T10:12:26.015884+0000","last_peered":"2024-04-25T10:12:26.015884+0000","last_clean":"2024-04-25T10:12:26.015884+0000","last_became_active":"2024-04-25T10:12:24.246353+0000","last_became_peered":"2024-04-25T10:12:24.246353+0000","last_unstale":"2024-04-25T10:12:26.015884+0000","last_undegraded":"2024-04-25T10:12:26.015884+0000","last_fullsized":"2024-04-25T10:12:26.015884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:15.421048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246997+0000","last_change":"2024-04-25T10:12:24.265431+0000","last_active":"2024-04-25T10:12:25.246997+0000","last_peered":"2024-04-25T10:12:25.246997+0000","last_clean":"2024-04-25T10:12:25.246997+0000","last_became_active":"2024-04-25T10:12:24.265047+0000","last_became_peered":"2024-04-25T10:12:24.265047+0000","last_unstale":"2024-04-25T10:12:25.246997+0000","last_undegraded":"2024-04-25T10:12:25.246997+0000","last_fullsized":"2024-04-25T10:12:25.246997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247153+0000","last_change":"2024-04-25T10:12:24.248104+0000","last_active":"2024-04-25T10:12:25.247153+0000","last_peered":"2024-04-25T10:12:25.247153+0000","last_clean":"2024-04-25T10:12:25.247153+0000","last_became_active":"2024-04-25T10:12:24.247826+0000","last_became_peered":"2024-04-25T10:12:24.247826+0000","last_unstale":"2024-04-25T10:12:25.247153+0000","last_undegraded":"2024-04-25T10:12:25.247153+0000","last_fullsized":"2024-04-25T10:12:25.247153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247053+0000","last_change":"2024-04-25T10:12:24.257638+0000","last_active":"2024-04-25T10:12:25.247053+0000","last_peered":"2024-04-25T10:12:25.247053+0000","last_clean":"2024-04-25T10:12:25.247053+0000","last_became_active":"2024-04-25T10:12:24.257332+0000","last_became_peered":"2024-04-25T10:12:24.257332+0000","last_unstale":"2024-04-25T10:12:25.247053+0000","last_undegraded":"2024-04-25T10:12:25.247053+0000","last_fullsized":"2024-04-25T10:12:25.247053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872137+0000","last_change":"2024-04-25T10:12:24.262252+0000","last_active":"2024-04-25T10:12:25.872137+0000","last_peered":"2024-04-25T10:12:25.872137+0000","last_clean":"2024-04-25T10:12:25.872137+0000","last_became_active":"2024-04-25T10:12:24.261871+0000","last_became_peered":"2024-04-25T10:12:24.261871+0000","last_unstale":"2024-04-25T10:12:25.872137+0000","last_undegraded":"2024-04-25T10:12:25.872137+0000","last_fullsized":"2024-04-25T10:12:25.872137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:13.729919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015458+0000","last_change":"2024-04-25T10:12:24.269990+0000","last_active":"2024-04-25T10:12:26.015458+0000","last_peered":"2024-04-25T10:12:26.015458+0000","last_clean":"2024-04-25T10:12:26.015458+0000","last_became_active":"2024-04-25T10:12:24.269396+0000","last_became_peered":"2024-04-25T10:12:24.269396+0000","last_unstale":"2024-04-25T10:12:26.015458+0000","last_undegraded":"2024-04-25T10:12:26.015458+0000","last_fullsized":"2024-04-25T10:12:26.015458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012013+0000","last_change":"2024-04-25T10:12:24.262832+0000","last_active":"2024-04-25T10:12:26.012013+0000","last_peered":"2024-04-25T10:12:26.012013+0000","last_clean":"2024-04-25T10:12:26.012013+0000","last_became_active":"2024-04-25T10:12:24.262090+0000","last_became_peered":"2024-04-25T10:12:24.262090+0000","last_unstale":"2024-04-25T10:12:26.012013+0000","last_undegraded":"2024-04-25T10:12:26.012013+0000","last_fullsized":"2024-04-25T10:12:26.012013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016383+0000","last_change":"2024-04-25T10:12:24.248538+0000","last_active":"2024-04-25T10:12:26.016383+0000","last_peered":"2024-04-25T10:12:26.016383+0000","last_clean":"2024-04-25T10:12:26.016383+0000","last_became_active":"2024-04-25T10:12:24.247653+0000","last_became_peered":"2024-04-25T10:12:24.247653+0000","last_unstale":"2024-04-25T10:12:26.016383+0000","last_undegraded":"2024-04-25T10:12:26.016383+0000","last_fullsized":"2024-04-25T10:12:26.016383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246115+0000","last_change":"2024-04-25T10:12:24.260603+0000","last_active":"2024-04-25T10:12:25.246115+0000","last_peered":"2024-04-25T10:12:25.246115+0000","last_clean":"2024-04-25T10:12:25.246115+0000","last_became_active":"2024-04-25T10:12:24.260288+0000","last_became_peered":"2024-04-25T10:12:24.260288+0000","last_unstale":"2024-04-25T10:12:25.246115+0000","last_undegraded":"2024-04-25T10:12:25.246115+0000","last_fullsized":"2024-04-25T10:12:25.246115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015623+0000","last_change":"2024-04-25T10:12:24.272790+0000","last_active":"2024-04-25T10:12:26.015623+0000","last_peered":"2024-04-25T10:12:26.015623+0000","last_clean":"2024-04-25T10:12:26.015623+0000","last_became_active":"2024-04-25T10:12:24.272587+0000","last_became_peered":"2024-04-25T10:12:24.272587+0000","last_unstale":"2024-04-25T10:12:26.015623+0000","last_undegraded":"2024-04-25T10:12:26.015623+0000","last_fullsized":"2024-04-25T10:12:26.015623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245637+0000","last_change":"2024-04-25T10:12:24.265627+0000","last_active":"2024-04-25T10:12:25.245637+0000","last_peered":"2024-04-25T10:12:25.245637+0000","last_clean":"2024-04-25T10:12:25.245637+0000","last_became_active":"2024-04-25T10:12:24.265274+0000","last_became_peered":"2024-04-25T10:12:24.265274+0000","last_unstale":"2024-04-25T10:12:25.245637+0000","last_undegraded":"2024-04-25T10:12:25.245637+0000","last_fullsized":"2024-04-25T10:12:25.245637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016338+0000","last_change":"2024-04-25T10:12:24.272301+0000","last_active":"2024-04-25T10:12:26.016338+0000","last_peered":"2024-04-25T10:12:26.016338+0000","last_clean":"2024-04-25T10:12:26.016338+0000","last_became_active":"2024-04-25T10:12:24.271766+0000","last_became_peered":"2024-04-25T10:12:24.271766+0000","last_unstale":"2024-04-25T10:12:26.016338+0000","last_undegraded":"2024-04-25T10:12:26.016338+0000","last_fullsized":"2024-04-25T10:12:26.016338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012456+0000","last_change":"2024-04-25T10:12:24.265368+0000","last_active":"2024-04-25T10:12:26.012456+0000","last_peered":"2024-04-25T10:12:26.012456+0000","last_clean":"2024-04-25T10:12:26.012456+0000","last_became_active":"2024-04-25T10:12:24.264950+0000","last_became_peered":"2024-04-25T10:12:24.264950+0000","last_unstale":"2024-04-25T10:12:26.012456+0000","last_undegraded":"2024-04-25T10:12:26.012456+0000","last_fullsized":"2024-04-25T10:12:26.012456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247251+0000","last_change":"2024-04-25T10:12:24.259900+0000","last_active":"2024-04-25T10:12:25.247251+0000","last_peered":"2024-04-25T10:12:25.247251+0000","last_clean":"2024-04-25T10:12:25.247251+0000","last_became_active":"2024-04-25T10:12:24.259574+0000","last_became_peered":"2024-04-25T10:12:24.259574+0000","last_unstale":"2024-04-25T10:12:25.247251+0000","last_undegraded":"2024-04-25T10:12:25.247251+0000","last_fullsized":"2024-04-25T10:12:25.247251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015996+0000","last_change":"2024-04-25T10:12:24.272315+0000","last_active":"2024-04-25T10:12:26.015996+0000","last_peered":"2024-04-25T10:12:26.015996+0000","last_clean":"2024-04-25T10:12:26.015996+0000","last_became_active":"2024-04-25T10:12:24.271843+0000","last_became_peered":"2024-04-25T10:12:24.271843+0000","last_unstale":"2024-04-25T10:12:26.015996+0000","last_undegraded":"2024-04-25T10:12:26.015996+0000","last_fullsized":"2024-04-25T10:12:26.015996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016505+0000","last_change":"2024-04-25T10:12:24.260827+0000","last_active":"2024-04-25T10:12:26.016505+0000","last_peered":"2024-04-25T10:12:26.016505+0000","last_clean":"2024-04-25T10:12:26.016505+0000","last_became_active":"2024-04-25T10:12:24.260436+0000","last_became_peered":"2024-04-25T10:12:24.260436+0000","last_unstale":"2024-04-25T10:12:26.016505+0000","last_undegraded":"2024-04-25T10:12:26.016505+0000","last_fullsized":"2024-04-25T10:12:26.016505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246276+0000","last_change":"2024-04-25T10:12:24.261679+0000","last_active":"2024-04-25T10:12:25.246276+0000","last_peered":"2024-04-25T10:12:25.246276+0000","last_clean":"2024-04-25T10:12:25.246276+0000","last_became_active":"2024-04-25T10:12:24.261420+0000","last_became_peered":"2024-04-25T10:12:24.261420+0000","last_unstale":"2024-04-25T10:12:25.246276+0000","last_undegraded":"2024-04-25T10:12:25.246276+0000","last_fullsized":"2024-04-25T10:12:25.246276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872106+0000","last_change":"2024-04-25T10:12:24.260060+0000","last_active":"2024-04-25T10:12:25.872106+0000","last_peered":"2024-04-25T10:12:25.872106+0000","last_clean":"2024-04-25T10:12:25.872106+0000","last_became_active":"2024-04-25T10:12:24.259332+0000","last_became_peered":"2024-04-25T10:12:24.259332+0000","last_unstale":"2024-04-25T10:12:25.872106+0000","last_undegraded":"2024-04-25T10:12:25.872106+0000","last_fullsized":"2024-04-25T10:12:25.872106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011814+0000","last_change":"2024-04-25T10:12:24.247021+0000","last_active":"2024-04-25T10:12:26.011814+0000","last_peered":"2024-04-25T10:12:26.011814+0000","last_clean":"2024-04-25T10:12:26.011814+0000","last_became_active":"2024-04-25T10:12:24.246699+0000","last_became_peered":"2024-04-25T10:12:24.246699+0000","last_unstale":"2024-04-25T10:12:26.011814+0000","last_undegraded":"2024-04-25T10:12:26.011814+0000","last_fullsized":"2024-04-25T10:12:26.011814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245745+0000","last_change":"2024-04-25T10:12:24.263775+0000","last_active":"2024-04-25T10:12:25.245745+0000","last_peered":"2024-04-25T10:12:25.245745+0000","last_clean":"2024-04-25T10:12:25.245745+0000","last_became_active":"2024-04-25T10:12:24.263373+0000","last_became_peered":"2024-04-25T10:12:24.263373+0000","last_unstale":"2024-04-25T10:12:25.245745+0000","last_undegraded":"2024-04-25T10:12:25.245745+0000","last_fullsized":"2024-04-25T10:12:25.245745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:23.746340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016216+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:26.016216+0000","last_peered":"2024-04-25T10:12:26.016216+0000","last_clean":"2024-04-25T10:12:26.016216+0000","last_became_active":"2024-04-25T10:12:23.249533+0000","last_became_peered":"2024-04-25T10:12:23.249533+0000","last_unstale":"2024-04-25T10:12:26.016216+0000","last_undegraded":"2024-04-25T10:12:26.016216+0000","last_fullsized":"2024-04-25T10:12:26.016216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016549+0000","last_change":"2024-04-25T10:12:24.252703+0000","last_active":"2024-04-25T10:12:26.016549+0000","last_peered":"2024-04-25T10:12:26.016549+0000","last_clean":"2024-04-25T10:12:26.016549+0000","last_became_active":"2024-04-25T10:12:24.252372+0000","last_became_peered":"2024-04-25T10:12:24.252372+0000","last_unstale":"2024-04-25T10:12:26.016549+0000","last_undegraded":"2024-04-25T10:12:26.016549+0000","last_fullsized":"2024-04-25T10:12:26.016549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012305+0000","last_change":"2024-04-25T10:12:23.248077+0000","last_active":"2024-04-25T10:12:26.012305+0000","last_peered":"2024-04-25T10:12:26.012305+0000","last_clean":"2024-04-25T10:12:26.012305+0000","last_became_active":"2024-04-25T10:12:23.247006+0000","last_became_peered":"2024-04-25T10:12:23.247006+0000","last_unstale":"2024-04-25T10:12:26.012305+0000","last_undegraded":"2024-04-25T10:12:26.012305+0000","last_fullsized":"2024-04-25T10:12:26.012305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015823+0000","last_change":"2024-04-25T10:12:23.240214+0000","last_active":"2024-04-25T10:12:26.015823+0000","last_peered":"2024-04-25T10:12:26.015823+0000","last_clean":"2024-04-25T10:12:26.015823+0000","last_became_active":"2024-04-25T10:12:23.239944+0000","last_became_peered":"2024-04-25T10:12:23.239944+0000","last_unstale":"2024-04-25T10:12:26.015823+0000","last_undegraded":"2024-04-25T10:12:26.015823+0000","last_fullsized":"2024-04-25T10:12:26.015823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015556+0000","last_change":"2024-04-25T10:12:24.258236+0000","last_active":"2024-04-25T10:12:26.015556+0000","last_peered":"2024-04-25T10:12:26.015556+0000","last_clean":"2024-04-25T10:12:26.015556+0000","last_became_active":"2024-04-25T10:12:24.256931+0000","last_became_peered":"2024-04-25T10:12:24.256931+0000","last_unstale":"2024-04-25T10:12:26.015556+0000","last_undegraded":"2024-04-25T10:12:26.015556+0000","last_fullsized":"2024-04-25T10:12:26.015556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245733+0000","last_change":"2024-04-25T10:12:23.253660+0000","last_active":"2024-04-25T10:12:25.245733+0000","last_peered":"2024-04-25T10:12:25.245733+0000","last_clean":"2024-04-25T10:12:25.245733+0000","last_became_active":"2024-04-25T10:12:23.253047+0000","last_became_peered":"2024-04-25T10:12:23.253047+0000","last_unstale":"2024-04-25T10:12:25.245733+0000","last_undegraded":"2024-04-25T10:12:25.245733+0000","last_fullsized":"2024-04-25T10:12:25.245733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246967+0000","last_change":"2024-04-25T10:12:24.260980+0000","last_active":"2024-04-25T10:12:25.246967+0000","last_peered":"2024-04-25T10:12:25.246967+0000","last_clean":"2024-04-25T10:12:25.246967+0000","last_became_active":"2024-04-25T10:12:24.260459+0000","last_became_peered":"2024-04-25T10:12:24.260459+0000","last_unstale":"2024-04-25T10:12:25.246967+0000","last_undegraded":"2024-04-25T10:12:25.246967+0000","last_fullsized":"2024-04-25T10:12:25.246967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012372+0000","last_change":"2024-04-25T10:12:23.253025+0000","last_active":"2024-04-25T10:12:26.012372+0000","last_peered":"2024-04-25T10:12:26.012372+0000","last_clean":"2024-04-25T10:12:26.012372+0000","last_became_active":"2024-04-25T10:12:23.251953+0000","last_became_peered":"2024-04-25T10:12:23.251953+0000","last_unstale":"2024-04-25T10:12:26.012372+0000","last_undegraded":"2024-04-25T10:12:26.012372+0000","last_fullsized":"2024-04-25T10:12:26.012372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246256+0000","last_change":"2024-04-25T10:12:24.273962+0000","last_active":"2024-04-25T10:12:25.246256+0000","last_peered":"2024-04-25T10:12:25.246256+0000","last_clean":"2024-04-25T10:12:25.246256+0000","last_became_active":"2024-04-25T10:12:24.273577+0000","last_became_peered":"2024-04-25T10:12:24.273577+0000","last_unstale":"2024-04-25T10:12:25.246256+0000","last_undegraded":"2024-04-25T10:12:25.246256+0000","last_fullsized":"2024-04-25T10:12:25.246256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246360+0000","last_change":"2024-04-25T10:12:23.247652+0000","last_active":"2024-04-25T10:12:25.246360+0000","last_peered":"2024-04-25T10:12:25.246360+0000","last_clean":"2024-04-25T10:12:25.246360+0000","last_became_active":"2024-04-25T10:12:23.243149+0000","last_became_peered":"2024-04-25T10:12:23.243149+0000","last_unstale":"2024-04-25T10:12:25.246360+0000","last_undegraded":"2024-04-25T10:12:25.246360+0000","last_fullsized":"2024-04-25T10:12:25.246360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016262+0000","last_change":"2024-04-25T10:12:24.261695+0000","last_active":"2024-04-25T10:12:26.016262+0000","last_peered":"2024-04-25T10:12:26.016262+0000","last_clean":"2024-04-25T10:12:26.016262+0000","last_became_active":"2024-04-25T10:12:24.261356+0000","last_became_peered":"2024-04-25T10:12:24.261356+0000","last_unstale":"2024-04-25T10:12:26.016262+0000","last_undegraded":"2024-04-25T10:12:26.016262+0000","last_fullsized":"2024-04-25T10:12:26.016262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247330+0000","last_change":"2024-04-25T10:12:23.244711+0000","last_active":"2024-04-25T10:12:25.247330+0000","last_peered":"2024-04-25T10:12:25.247330+0000","last_clean":"2024-04-25T10:12:25.247330+0000","last_became_active":"2024-04-25T10:12:23.244360+0000","last_became_peered":"2024-04-25T10:12:23.244360+0000","last_unstale":"2024-04-25T10:12:25.247330+0000","last_undegraded":"2024-04-25T10:12:25.247330+0000","last_fullsized":"2024-04-25T10:12:25.247330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246749+0000","last_change":"2024-04-25T10:12:24.259965+0000","last_active":"2024-04-25T10:12:25.246749+0000","last_peered":"2024-04-25T10:12:25.246749+0000","last_clean":"2024-04-25T10:12:25.246749+0000","last_became_active":"2024-04-25T10:12:24.249487+0000","last_became_peered":"2024-04-25T10:12:24.249487+0000","last_unstale":"2024-04-25T10:12:25.246749+0000","last_undegraded":"2024-04-25T10:12:25.246749+0000","last_fullsized":"2024-04-25T10:12:25.246749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:12.866317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012035+0000","last_change":"2024-04-25T10:12:23.240603+0000","last_active":"2024-04-25T10:12:26.012035+0000","last_peered":"2024-04-25T10:12:26.012035+0000","last_clean":"2024-04-25T10:12:26.012035+0000","last_became_active":"2024-04-25T10:12:23.240240+0000","last_became_peered":"2024-04-25T10:12:23.240240+0000","last_unstale":"2024-04-25T10:12:26.012035+0000","last_undegraded":"2024-04-25T10:12:26.012035+0000","last_fullsized":"2024-04-25T10:12:26.012035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015129+0000","last_change":"2024-04-25T10:12:24.259015+0000","last_active":"2024-04-25T10:12:26.015129+0000","last_peered":"2024-04-25T10:12:26.015129+0000","last_clean":"2024-04-25T10:12:26.015129+0000","last_became_active":"2024-04-25T10:12:24.258715+0000","last_became_peered":"2024-04-25T10:12:24.258715+0000","last_unstale":"2024-04-25T10:12:26.015129+0000","last_undegraded":"2024-04-25T10:12:26.015129+0000","last_fullsized":"2024-04-25T10:12:26.015129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016578+0000","last_change":"2024-04-25T10:12:23.253767+0000","last_active":"2024-04-25T10:12:26.016578+0000","last_peered":"2024-04-25T10:12:26.016578+0000","last_clean":"2024-04-25T10:12:26.016578+0000","last_became_active":"2024-04-25T10:12:23.253356+0000","last_became_peered":"2024-04-25T10:12:23.253356+0000","last_unstale":"2024-04-25T10:12:26.016578+0000","last_undegraded":"2024-04-25T10:12:26.016578+0000","last_fullsized":"2024-04-25T10:12:26.016578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016055+0000","last_change":"2024-04-25T10:12:24.241735+0000","last_active":"2024-04-25T10:12:26.016055+0000","last_peered":"2024-04-25T10:12:26.016055+0000","last_clean":"2024-04-25T10:12:26.016055+0000","last_became_active":"2024-04-25T10:12:24.241406+0000","last_became_peered":"2024-04-25T10:12:24.241406+0000","last_unstale":"2024-04-25T10:12:26.016055+0000","last_undegraded":"2024-04-25T10:12:26.016055+0000","last_fullsized":"2024-04-25T10:12:26.016055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:27.592446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246379+0000","last_change":"2024-04-25T10:12:23.249072+0000","last_active":"2024-04-25T10:12:25.246379+0000","last_peered":"2024-04-25T10:12:25.246379+0000","last_clean":"2024-04-25T10:12:25.246379+0000","last_became_active":"2024-04-25T10:12:23.248691+0000","last_became_peered":"2024-04-25T10:12:23.248691+0000","last_unstale":"2024-04-25T10:12:25.246379+0000","last_undegraded":"2024-04-25T10:12:25.246379+0000","last_fullsized":"2024-04-25T10:12:25.246379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015097+0000","last_change":"2024-04-25T10:12:24.269693+0000","last_active":"2024-04-25T10:12:26.015097+0000","last_peered":"2024-04-25T10:12:26.015097+0000","last_clean":"2024-04-25T10:12:26.015097+0000","last_became_active":"2024-04-25T10:12:24.269423+0000","last_became_peered":"2024-04-25T10:12:24.269423+0000","last_unstale":"2024-04-25T10:12:26.015097+0000","last_undegraded":"2024-04-25T10:12:26.015097+0000","last_fullsized":"2024-04-25T10:12:26.015097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011686+0000","last_change":"2024-04-25T10:12:23.246678+0000","last_active":"2024-04-25T10:12:26.011686+0000","last_peered":"2024-04-25T10:12:26.011686+0000","last_clean":"2024-04-25T10:12:26.011686+0000","last_became_active":"2024-04-25T10:12:23.246288+0000","last_became_peered":"2024-04-25T10:12:23.246288+0000","last_unstale":"2024-04-25T10:12:26.011686+0000","last_undegraded":"2024-04-25T10:12:26.011686+0000","last_fullsized":"2024-04-25T10:12:26.011686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011782+0000","last_change":"2024-04-25T10:12:24.245870+0000","last_active":"2024-04-25T10:12:26.011782+0000","last_peered":"2024-04-25T10:12:26.011782+0000","last_clean":"2024-04-25T10:12:26.011782+0000","last_became_active":"2024-04-25T10:12:24.244284+0000","last_became_peered":"2024-04-25T10:12:24.244284+0000","last_unstale":"2024-04-25T10:12:26.011782+0000","last_undegraded":"2024-04-25T10:12:26.011782+0000","last_fullsized":"2024-04-25T10:12:26.011782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.871923+0000","last_change":"2024-04-25T10:12:23.232086+0000","last_active":"2024-04-25T10:12:25.871923+0000","last_peered":"2024-04-25T10:12:25.871923+0000","last_clean":"2024-04-25T10:12:25.871923+0000","last_became_active":"2024-04-25T10:12:23.231694+0000","last_became_peered":"2024-04-25T10:12:23.231694+0000","last_unstale":"2024-04-25T10:12:25.871923+0000","last_undegraded":"2024-04-25T10:12:25.871923+0000","last_fullsized":"2024-04-25T10:12:25.871923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014903+0000","last_change":"2024-04-25T10:12:24.252131+0000","last_active":"2024-04-25T10:12:26.014903+0000","last_peered":"2024-04-25T10:12:26.014903+0000","last_clean":"2024-04-25T10:12:26.014903+0000","last_became_active":"2024-04-25T10:12:24.251807+0000","last_became_peered":"2024-04-25T10:12:24.251807+0000","last_unstale":"2024-04-25T10:12:26.014903+0000","last_undegraded":"2024-04-25T10:12:26.014903+0000","last_fullsized":"2024-04-25T10:12:26.014903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246755+0000","last_change":"2024-04-25T10:12:23.243507+0000","last_active":"2024-04-25T10:12:25.246755+0000","last_peered":"2024-04-25T10:12:25.246755+0000","last_clean":"2024-04-25T10:12:25.246755+0000","last_became_active":"2024-04-25T10:12:23.243082+0000","last_became_peered":"2024-04-25T10:12:23.243082+0000","last_unstale":"2024-04-25T10:12:25.246755+0000","last_undegraded":"2024-04-25T10:12:25.246755+0000","last_fullsized":"2024-04-25T10:12:25.246755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:09.593840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015480+0000","last_change":"2024-04-25T10:12:24.270975+0000","last_active":"2024-04-25T10:12:26.015480+0000","last_peered":"2024-04-25T10:12:26.015480+0000","last_clean":"2024-04-25T10:12:26.015480+0000","last_became_active":"2024-04-25T10:12:24.270678+0000","last_became_peered":"2024-04-25T10:12:24.270678+0000","last_unstale":"2024-04-25T10:12:26.015480+0000","last_undegraded":"2024-04-25T10:12:26.015480+0000","last_fullsized":"2024-04-25T10:12:26.015480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872082+0000","last_change":"2024-04-25T10:12:23.242355+0000","last_active":"2024-04-25T10:12:25.872082+0000","last_peered":"2024-04-25T10:12:25.872082+0000","last_clean":"2024-04-25T10:12:25.872082+0000","last_became_active":"2024-04-25T10:12:23.242038+0000","last_became_peered":"2024-04-25T10:12:23.242038+0000","last_unstale":"2024-04-25T10:12:25.872082+0000","last_undegraded":"2024-04-25T10:12:25.872082+0000","last_fullsized":"2024-04-25T10:12:25.872082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015634+0000","last_change":"2024-04-25T10:12:24.270959+0000","last_active":"2024-04-25T10:12:26.015634+0000","last_peered":"2024-04-25T10:12:26.015634+0000","last_clean":"2024-04-25T10:12:26.015634+0000","last_became_active":"2024-04-25T10:12:24.270658+0000","last_became_peered":"2024-04-25T10:12:24.270658+0000","last_unstale":"2024-04-25T10:12:26.015634+0000","last_undegraded":"2024-04-25T10:12:26.015634+0000","last_fullsized":"2024-04-25T10:12:26.015634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247343+0000","last_change":"2024-04-25T10:12:23.255897+0000","last_active":"2024-04-25T10:12:25.247343+0000","last_peered":"2024-04-25T10:12:25.247343+0000","last_clean":"2024-04-25T10:12:25.247343+0000","last_became_active":"2024-04-25T10:12:23.255626+0000","last_became_peered":"2024-04-25T10:12:23.255626+0000","last_unstale":"2024-04-25T10:12:25.247343+0000","last_undegraded":"2024-04-25T10:12:25.247343+0000","last_fullsized":"2024-04-25T10:12:25.247343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:51.408775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246715+0000","last_change":"2024-04-25T10:12:24.262419+0000","last_active":"2024-04-25T10:12:25.246715+0000","last_peered":"2024-04-25T10:12:25.246715+0000","last_clean":"2024-04-25T10:12:25.246715+0000","last_became_active":"2024-04-25T10:12:24.262163+0000","last_became_peered":"2024-04-25T10:12:24.262163+0000","last_unstale":"2024-04-25T10:12:25.246715+0000","last_undegraded":"2024-04-25T10:12:25.246715+0000","last_fullsized":"2024-04-25T10:12:25.246715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872045+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:25.872045+0000","last_peered":"2024-04-25T10:12:25.872045+0000","last_clean":"2024-04-25T10:12:25.872045+0000","last_became_active":"2024-04-25T10:12:23.249437+0000","last_became_peered":"2024-04-25T10:12:23.249437+0000","last_unstale":"2024-04-25T10:12:25.872045+0000","last_undegraded":"2024-04-25T10:12:25.872045+0000","last_fullsized":"2024-04-25T10:12:25.872045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015325+0000","last_change":"2024-04-25T10:12:24.266865+0000","last_active":"2024-04-25T10:12:26.015325+0000","last_peered":"2024-04-25T10:12:26.015325+0000","last_clean":"2024-04-25T10:12:26.015325+0000","last_became_active":"2024-04-25T10:12:24.266595+0000","last_became_peered":"2024-04-25T10:12:24.266595+0000","last_unstale":"2024-04-25T10:12:26.015325+0000","last_undegraded":"2024-04-25T10:12:26.015325+0000","last_fullsized":"2024-04-25T10:12:26.015325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015021+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:12:26.015021+0000","last_peered":"2024-04-25T10:12:26.015021+0000","last_clean":"2024-04-25T10:12:26.015021+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:12:26.015021+0000","last_undegraded":"2024-04-25T10:12:26.015021+0000","last_fullsized":"2024-04-25T10:12:26.015021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245447+0000","last_change":"2024-04-25T10:12:23.236000+0000","last_active":"2024-04-25T10:12:25.245447+0000","last_peered":"2024-04-25T10:12:25.245447+0000","last_clean":"2024-04-25T10:12:25.245447+0000","last_became_active":"2024-04-25T10:12:23.235473+0000","last_became_peered":"2024-04-25T10:12:23.235473+0000","last_unstale":"2024-04-25T10:12:25.245447+0000","last_undegraded":"2024-04-25T10:12:25.245447+0000","last_fullsized":"2024-04-25T10:12:25.245447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016099+0000","last_change":"2024-04-25T10:12:24.269413+0000","last_active":"2024-04-25T10:12:26.016099+0000","last_peered":"2024-04-25T10:12:26.016099+0000","last_clean":"2024-04-25T10:12:26.016099+0000","last_became_active":"2024-04-25T10:12:24.269109+0000","last_became_peered":"2024-04-25T10:12:24.269109+0000","last_unstale":"2024-04-25T10:12:26.016099+0000","last_undegraded":"2024-04-25T10:12:26.016099+0000","last_fullsized":"2024-04-25T10:12:26.016099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014753+0000","last_change":"2024-04-25T10:12:23.251616+0000","last_active":"2024-04-25T10:12:26.014753+0000","last_peered":"2024-04-25T10:12:26.014753+0000","last_clean":"2024-04-25T10:12:26.014753+0000","last_became_active":"2024-04-25T10:12:23.251356+0000","last_became_peered":"2024-04-25T10:12:23.251356+0000","last_unstale":"2024-04-25T10:12:26.014753+0000","last_undegraded":"2024-04-25T10:12:26.014753+0000","last_fullsized":"2024-04-25T10:12:26.014753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015666+0000","last_change":"2024-04-25T10:12:24.252497+0000","last_active":"2024-04-25T10:12:26.015666+0000","last_peered":"2024-04-25T10:12:26.015666+0000","last_clean":"2024-04-25T10:12:26.015666+0000","last_became_active":"2024-04-25T10:12:24.252221+0000","last_became_peered":"2024-04-25T10:12:24.252221+0000","last_unstale":"2024-04-25T10:12:26.015666+0000","last_undegraded":"2024-04-25T10:12:26.015666+0000","last_fullsized":"2024-04-25T10:12:26.015666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:52.114948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872039+0000","last_change":"2024-04-25T10:12:23.235336+0000","last_active":"2024-04-25T10:12:25.872039+0000","last_peered":"2024-04-25T10:12:25.872039+0000","last_clean":"2024-04-25T10:12:25.872039+0000","last_became_active":"2024-04-25T10:12:23.235016+0000","last_became_peered":"2024-04-25T10:12:23.235016+0000","last_unstale":"2024-04-25T10:12:25.872039+0000","last_undegraded":"2024-04-25T10:12:25.872039+0000","last_fullsized":"2024-04-25T10:12:25.872039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246990+0000","last_change":"2024-04-25T10:12:24.264913+0000","last_active":"2024-04-25T10:12:25.246990+0000","last_peered":"2024-04-25T10:12:25.246990+0000","last_clean":"2024-04-25T10:12:25.246990+0000","last_became_active":"2024-04-25T10:12:24.264498+0000","last_became_peered":"2024-04-25T10:12:24.264498+0000","last_unstale":"2024-04-25T10:12:25.246990+0000","last_undegraded":"2024-04-25T10:12:25.246990+0000","last_fullsized":"2024-04-25T10:12:25.246990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246493+0000","last_change":"2024-04-25T10:12:23.241698+0000","last_active":"2024-04-25T10:12:25.246493+0000","last_peered":"2024-04-25T10:12:25.246493+0000","last_clean":"2024-04-25T10:12:25.246493+0000","last_became_active":"2024-04-25T10:12:23.241337+0000","last_became_peered":"2024-04-25T10:12:23.241337+0000","last_unstale":"2024-04-25T10:12:25.246493+0000","last_undegraded":"2024-04-25T10:12:25.246493+0000","last_fullsized":"2024-04-25T10:12:25.246493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246479+0000","last_change":"2024-04-25T10:12:24.249493+0000","last_active":"2024-04-25T10:12:25.246479+0000","last_peered":"2024-04-25T10:12:25.246479+0000","last_clean":"2024-04-25T10:12:25.246479+0000","last_became_active":"2024-04-25T10:12:24.249067+0000","last_became_peered":"2024-04-25T10:12:24.249067+0000","last_unstale":"2024-04-25T10:12:25.246479+0000","last_undegraded":"2024-04-25T10:12:25.246479+0000","last_fullsized":"2024-04-25T10:12:25.246479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.231393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247191+0000","last_change":"2024-04-25T10:12:23.243659+0000","last_active":"2024-04-25T10:12:25.247191+0000","last_peered":"2024-04-25T10:12:25.247191+0000","last_clean":"2024-04-25T10:12:25.247191+0000","last_became_active":"2024-04-25T10:12:23.243098+0000","last_became_peered":"2024-04-25T10:12:23.243098+0000","last_unstale":"2024-04-25T10:12:25.247191+0000","last_undegraded":"2024-04-25T10:12:25.247191+0000","last_fullsized":"2024-04-25T10:12:25.247191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245987+0000","last_change":"2024-04-25T10:12:24.264189+0000","last_active":"2024-04-25T10:12:25.245987+0000","last_peered":"2024-04-25T10:12:25.245987+0000","last_clean":"2024-04-25T10:12:25.245987+0000","last_became_active":"2024-04-25T10:12:24.258438+0000","last_became_peered":"2024-04-25T10:12:24.258438+0000","last_unstale":"2024-04-25T10:12:25.245987+0000","last_undegraded":"2024-04-25T10:12:25.245987+0000","last_fullsized":"2024-04-25T10:12:25.245987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015445+0000","last_change":"2024-04-25T10:12:23.254569+0000","last_active":"2024-04-25T10:12:26.015445+0000","last_peered":"2024-04-25T10:12:26.015445+0000","last_clean":"2024-04-25T10:12:26.015445+0000","last_became_active":"2024-04-25T10:12:23.254228+0000","last_became_peered":"2024-04-25T10:12:23.254228+0000","last_unstale":"2024-04-25T10:12:26.015445+0000","last_undegraded":"2024-04-25T10:12:26.015445+0000","last_fullsized":"2024-04-25T10:12:26.015445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:12.723714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247192+0000","last_change":"2024-04-25T10:12:24.258387+0000","last_active":"2024-04-25T10:12:25.247192+0000","last_peered":"2024-04-25T10:12:25.247192+0000","last_clean":"2024-04-25T10:12:25.247192+0000","last_became_active":"2024-04-25T10:12:24.257160+0000","last_became_peered":"2024-04-25T10:12:24.257160+0000","last_unstale":"2024-04-25T10:12:25.247192+0000","last_undegraded":"2024-04-25T10:12:25.247192+0000","last_fullsized":"2024-04-25T10:12:25.247192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247441+0000","last_change":"2024-04-25T10:12:23.247450+0000","last_active":"2024-04-25T10:12:25.247441+0000","last_peered":"2024-04-25T10:12:25.247441+0000","last_clean":"2024-04-25T10:12:25.247441+0000","last_became_active":"2024-04-25T10:12:23.247166+0000","last_became_peered":"2024-04-25T10:12:23.247166+0000","last_unstale":"2024-04-25T10:12:25.247441+0000","last_undegraded":"2024-04-25T10:12:25.247441+0000","last_fullsized":"2024-04-25T10:12:25.247441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016516+0000","last_change":"2024-04-25T10:12:24.271046+0000","last_active":"2024-04-25T10:12:26.016516+0000","last_peered":"2024-04-25T10:12:26.016516+0000","last_clean":"2024-04-25T10:12:26.016516+0000","last_became_active":"2024-04-25T10:12:24.270697+0000","last_became_peered":"2024-04-25T10:12:24.270697+0000","last_unstale":"2024-04-25T10:12:26.016516+0000","last_undegraded":"2024-04-25T10:12:26.016516+0000","last_fullsized":"2024-04-25T10:12:26.016516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012514+0000","last_change":"2024-04-25T10:12:23.252034+0000","last_active":"2024-04-25T10:12:26.012514+0000","last_peered":"2024-04-25T10:12:26.012514+0000","last_clean":"2024-04-25T10:12:26.012514+0000","last_became_active":"2024-04-25T10:12:23.251537+0000","last_became_peered":"2024-04-25T10:12:23.251537+0000","last_unstale":"2024-04-25T10:12:26.012514+0000","last_undegraded":"2024-04-25T10:12:26.012514+0000","last_fullsized":"2024-04-25T10:12:26.012514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016121+0000","last_change":"2024-04-25T10:12:24.247147+0000","last_active":"2024-04-25T10:12:26.016121+0000","last_peered":"2024-04-25T10:12:26.016121+0000","last_clean":"2024-04-25T10:12:26.016121+0000","last_became_active":"2024-04-25T10:12:24.246897+0000","last_became_peered":"2024-04-25T10:12:24.246897+0000","last_unstale":"2024-04-25T10:12:26.016121+0000","last_undegraded":"2024-04-25T10:12:26.016121+0000","last_fullsized":"2024-04-25T10:12:26.016121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014793+0000","last_change":"2024-04-25T10:12:23.254396+0000","last_active":"2024-04-25T10:12:26.014793+0000","last_peered":"2024-04-25T10:12:26.014793+0000","last_clean":"2024-04-25T10:12:26.014793+0000","last_became_active":"2024-04-25T10:12:23.254125+0000","last_became_peered":"2024-04-25T10:12:23.254125+0000","last_unstale":"2024-04-25T10:12:26.014793+0000","last_undegraded":"2024-04-25T10:12:26.014793+0000","last_fullsized":"2024-04-25T10:12:26.014793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247014+0000","last_change":"2024-04-25T10:12:24.261214+0000","last_active":"2024-04-25T10:12:25.247014+0000","last_peered":"2024-04-25T10:12:25.247014+0000","last_clean":"2024-04-25T10:12:25.247014+0000","last_became_active":"2024-04-25T10:12:24.260895+0000","last_became_peered":"2024-04-25T10:12:24.260895+0000","last_unstale":"2024-04-25T10:12:25.247014+0000","last_undegraded":"2024-04-25T10:12:25.247014+0000","last_fullsized":"2024-04-25T10:12:25.247014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247067+0000","last_change":"2024-04-25T10:12:23.244316+0000","last_active":"2024-04-25T10:12:25.247067+0000","last_peered":"2024-04-25T10:12:25.247067+0000","last_clean":"2024-04-25T10:12:25.247067+0000","last_became_active":"2024-04-25T10:12:23.243075+0000","last_became_peered":"2024-04-25T10:12:23.243075+0000","last_unstale":"2024-04-25T10:12:25.247067+0000","last_undegraded":"2024-04-25T10:12:25.247067+0000","last_fullsized":"2024-04-25T10:12:25.247067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016532+0000","last_change":"2024-04-25T10:12:24.252644+0000","last_active":"2024-04-25T10:12:26.016532+0000","last_peered":"2024-04-25T10:12:26.016532+0000","last_clean":"2024-04-25T10:12:26.016532+0000","last_became_active":"2024-04-25T10:12:24.252317+0000","last_became_peered":"2024-04-25T10:12:24.252317+0000","last_unstale":"2024-04-25T10:12:26.016532+0000","last_undegraded":"2024-04-25T10:12:26.016532+0000","last_fullsized":"2024-04-25T10:12:26.016532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247152+0000","last_change":"2024-04-25T10:12:23.243515+0000","last_active":"2024-04-25T10:12:25.247152+0000","last_peered":"2024-04-25T10:12:25.247152+0000","last_clean":"2024-04-25T10:12:25.247152+0000","last_became_active":"2024-04-25T10:12:23.242920+0000","last_became_peered":"2024-04-25T10:12:23.242920+0000","last_unstale":"2024-04-25T10:12:25.247152+0000","last_undegraded":"2024-04-25T10:12:25.247152+0000","last_fullsized":"2024-04-25T10:12:25.247152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016097+0000","last_change":"2024-04-25T10:12:24.256479+0000","last_active":"2024-04-25T10:12:26.016097+0000","last_peered":"2024-04-25T10:12:26.016097+0000","last_clean":"2024-04-25T10:12:26.016097+0000","last_became_active":"2024-04-25T10:12:24.256149+0000","last_became_peered":"2024-04-25T10:12:24.256149+0000","last_unstale":"2024-04-25T10:12:26.016097+0000","last_undegraded":"2024-04-25T10:12:26.016097+0000","last_fullsized":"2024-04-25T10:12:26.016097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015287+0000","last_change":"2024-04-25T10:12:23.258511+0000","last_active":"2024-04-25T10:12:26.015287+0000","last_peered":"2024-04-25T10:12:26.015287+0000","last_clean":"2024-04-25T10:12:26.015287+0000","last_became_active":"2024-04-25T10:12:23.258235+0000","last_became_peered":"2024-04-25T10:12:23.258235+0000","last_unstale":"2024-04-25T10:12:26.015287+0000","last_undegraded":"2024-04-25T10:12:26.015287+0000","last_fullsized":"2024-04-25T10:12:26.015287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012230+0000","last_change":"2024-04-25T10:12:24.245339+0000","last_active":"2024-04-25T10:12:26.012230+0000","last_peered":"2024-04-25T10:12:26.012230+0000","last_clean":"2024-04-25T10:12:26.012230+0000","last_became_active":"2024-04-25T10:12:24.241842+0000","last_became_peered":"2024-04-25T10:12:24.241842+0000","last_unstale":"2024-04-25T10:12:26.012230+0000","last_undegraded":"2024-04-25T10:12:26.012230+0000","last_fullsized":"2024-04-25T10:12:26.012230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245817+0000","last_change":"2024-04-25T10:12:24.268837+0000","last_active":"2024-04-25T10:12:25.245817+0000","last_peered":"2024-04-25T10:12:25.245817+0000","last_clean":"2024-04-25T10:12:25.245817+0000","last_became_active":"2024-04-25T10:12:24.268028+0000","last_became_peered":"2024-04-25T10:12:24.268028+0000","last_unstale":"2024-04-25T10:12:25.245817+0000","last_undegraded":"2024-04-25T10:12:25.245817+0000","last_fullsized":"2024-04-25T10:12:25.245817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015045+0000","last_change":"2024-04-25T10:12:23.259889+0000","last_active":"2024-04-25T10:12:26.015045+0000","last_peered":"2024-04-25T10:12:26.015045+0000","last_clean":"2024-04-25T10:12:26.015045+0000","last_became_active":"2024-04-25T10:12:23.259546+0000","last_became_peered":"2024-04-25T10:12:23.259546+0000","last_unstale":"2024-04-25T10:12:26.015045+0000","last_undegraded":"2024-04-25T10:12:26.015045+0000","last_fullsized":"2024-04-25T10:12:26.015045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.267238+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.266952+0000","last_became_peered":"2024-04-25T10:12:24.266952+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015266+0000","last_change":"2024-04-25T10:12:23.258590+0000","last_active":"2024-04-25T10:12:26.015266+0000","last_peered":"2024-04-25T10:12:26.015266+0000","last_clean":"2024-04-25T10:12:26.015266+0000","last_became_active":"2024-04-25T10:12:23.258334+0000","last_became_peered":"2024-04-25T10:12:23.258334+0000","last_unstale":"2024-04-25T10:12:26.015266+0000","last_undegraded":"2024-04-25T10:12:26.015266+0000","last_fullsized":"2024-04-25T10:12:26.015266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016106+0000","last_change":"2024-04-25T10:12:24.261806+0000","last_active":"2024-04-25T10:12:26.016106+0000","last_peered":"2024-04-25T10:12:26.016106+0000","last_clean":"2024-04-25T10:12:26.016106+0000","last_became_active":"2024-04-25T10:12:24.261462+0000","last_became_peered":"2024-04-25T10:12:24.261462+0000","last_unstale":"2024-04-25T10:12:26.016106+0000","last_undegraded":"2024-04-25T10:12:26.016106+0000","last_fullsized":"2024-04-25T10:12:26.016106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016336+0000","last_change":"2024-04-25T10:12:23.246028+0000","last_active":"2024-04-25T10:12:26.016336+0000","last_peered":"2024-04-25T10:12:26.016336+0000","last_clean":"2024-04-25T10:12:26.016336+0000","last_became_active":"2024-04-25T10:12:23.245746+0000","last_became_peered":"2024-04-25T10:12:23.245746+0000","last_unstale":"2024-04-25T10:12:26.016336+0000","last_undegraded":"2024-04-25T10:12:26.016336+0000","last_fullsized":"2024-04-25T10:12:26.016336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016375+0000","last_change":"2024-04-25T10:12:24.265910+0000","last_active":"2024-04-25T10:12:26.016375+0000","last_peered":"2024-04-25T10:12:26.016375+0000","last_clean":"2024-04-25T10:12:26.016375+0000","last_became_active":"2024-04-25T10:12:24.265598+0000","last_became_peered":"2024-04-25T10:12:24.265598+0000","last_unstale":"2024-04-25T10:12:26.016375+0000","last_undegraded":"2024-04-25T10:12:26.016375+0000","last_fullsized":"2024-04-25T10:12:26.016375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016395+0000","last_change":"2024-04-25T10:12:23.252781+0000","last_active":"2024-04-25T10:12:26.016395+0000","last_peered":"2024-04-25T10:12:26.016395+0000","last_clean":"2024-04-25T10:12:26.016395+0000","last_became_active":"2024-04-25T10:12:23.252526+0000","last_became_peered":"2024-04-25T10:12:23.252526+0000","last_unstale":"2024-04-25T10:12:26.016395+0000","last_undegraded":"2024-04-25T10:12:26.016395+0000","last_fullsized":"2024-04-25T10:12:26.016395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:10.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:28.691 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:12:28.691 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:12:28.691 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:12:28.691 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:12:28.692 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:12:28.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e49e00640 1 Processor -- start 2024-04-25T10:12:28.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e49e00640 1 -- start start 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e49e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6e440d0040 0x7f6e4415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e49e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e440b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e49e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e44071050 con 0x7f6e441550b0 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e49e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4415d040 con 0x7f6e440d0040 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e49e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440712f0 con 0x7f6e4415b740 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e42a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6e441550b0 0x7f6e440cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60528/0 (socket says 172.21.15.87:60528) 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e42a00640 1 -- 172.21.15.87:0/1214825161 learned_addr learned my addr 172.21.15.87:0/1214825161 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:28.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e3a000640 1 --2- 172.21.15.87:0/1214825161 >> v2:172.21.15.87:3300/0 conn(0x7f6e440d0040 0x7f6e4415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e43400640 1 --2- 172.21.15.87:0/1214825161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e440b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.860+0000 7f6e42000640 1 -- 172.21.15.87:0/1214825161 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073313500 0 0) 0x7f6e44071050 con 0x7f6e441550b0 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e42000640 1 -- 172.21.15.87:0/1214825161 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e28003530 con 0x7f6e441550b0 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e3a000640 1 -- 172.21.15.87:0/1214825161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 msgr2=0x7f6e440b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e3a000640 1 --2- 172.21.15.87:0/1214825161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e440b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e3a000640 1 -- 172.21.15.87:0/1214825161 >> v1:172.21.15.87:6789/0 conn(0x7f6e441550b0 legacy=0x7f6e440cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e3a000640 1 -- 172.21.15.87:0/1214825161 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440715d0 con 0x7f6e440d0040 2024-04-25T10:12:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e3a000640 1 --2- 172.21.15.87:0/1214825161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e440d0040 0x7f6e4415b200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0097c0 tx=0x7f6e2c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8679baf729e199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e42000640 1 -- 172.21.15.87:0/1214825161 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0a7070 con 0x7f6e440d0040 2024-04-25T10:12:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e42000640 1 -- 172.21.15.87:0/1214825161 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c0b0070 con 0x7f6e440d0040 2024-04-25T10:12:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e42000640 1 -- 172.21.15.87:0/1214825161 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0ab890 con 0x7f6e440d0040 2024-04-25T10:12:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 -- 172.21.15.87:0/1214825161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e440d0040 msgr2=0x7f6e4415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 --2- 172.21.15.87:0/1214825161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e440d0040 0x7f6e4415b200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0097c0 tx=0x7f6e2c01c470 comp rx=0 tx=0).stop 2024-04-25T10:12:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 -- 172.21.15.87:0/1214825161 shutdown_connections 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 --2- 172.21.15.87:0/1214825161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e440b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 --2- 172.21.15.87:0/1214825161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e440d0040 0x7f6e4415b200 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 -- 172.21.15.87:0/1214825161 >> 172.21.15.87:0/1214825161 conn(0x7f6e44066b70 msgr2=0x7f6e44066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 -- 172.21.15.87:0/1214825161 shutdown_connections 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.864+0000 7f6e49e00640 1 -- 172.21.15.87:0/1214825161 wait complete. 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 Processor -- start 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 -- start start 2024-04-25T10:12:28.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e440d0040 0x7f6e4414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e441550b0 0x7f6e440b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e4413df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415bcd0 con 0x7f6e441550b0 2024-04-25T10:12:28.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415bb50 con 0x7f6e440d0040 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415c030 con 0x7f6e4415b740 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e441550b0 0x7f6e440b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e441550b0 0x7f6e440b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56964/0 (socket says 172.21.15.87:56964) 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 -- 172.21.15.87:0/2090049498 learned_addr learned my addr 172.21.15.87:0/2090049498 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e43400640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e4413df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e42a00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e440d0040 0x7f6e4414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 -- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 msgr2=0x7f6e4413df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e4413df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 -- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e440d0040 msgr2=0x7f6e4414f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e440d0040 0x7f6e4414f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 -- 172.21.15.87:0/2090049498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4414baa0 con 0x7f6e441550b0 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3a000640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e441550b0 0x7f6e440b1c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0ad040 tx=0x7f6e2c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3be00640 1 -- 172.21.15.87:0/2090049498 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0bf610 con 0x7f6e441550b0 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3be00640 1 -- 172.21.15.87:0/2090049498 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c0bf7d0 con 0x7f6e441550b0 2024-04-25T10:12:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4414fb10 con 0x7f6e441550b0 2024-04-25T10:12:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e3be00640 1 -- 172.21.15.87:0/2090049498 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0bfbc0 con 0x7f6e441550b0 2024-04-25T10:12:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.868+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e441500d0 con 0x7f6e441550b0 2024-04-25T10:12:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.872+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4404ef30 con 0x7f6e441550b0 2024-04-25T10:12:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.872+0000 7f6e3be00640 1 -- 172.21.15.87:0/2090049498 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e2c0bd020 con 0x7f6e441550b0 2024-04-25T10:12:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.872+0000 7f6e3be00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e20066f90 0x7f6e20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.872+0000 7f6e3be00640 1 -- 172.21.15.87:0/2090049498 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e2c11f7f0 con 0x7f6e441550b0 2024-04-25T10:12:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.872+0000 7f6e42a00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e20066f90 0x7f6e20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.872+0000 7f6e42a00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e20066f90 0x7f6e20069450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e34005890 tx=0x7f6e34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:28.876+0000 7f6e3be00640 1 -- 172.21.15.87:0/2090049498 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e2c0c4060 con 0x7f6e441550b0 2024-04-25T10:12:29.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.064+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6e440ae730 con 0x7f6e441550b0 2024-04-25T10:12:29.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.068+0000 7f6e3be00640 1 -- 172.21.15.87:0/2090049498 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165733 (secure 0 0 0) 0x7f6e2c016090 con 0x7f6e441550b0 2024-04-25T10:12:29.075 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e20066f90 msgr2=0x7f6e20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e20066f90 0x7f6e20069450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e34005890 tx=0x7f6e34009040 comp rx=0 tx=0).stop 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e441550b0 msgr2=0x7f6e440b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e441550b0 0x7f6e440b1c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0ad040 tx=0x7f6e2c01acb0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 shutdown_connections 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e20066f90 0x7f6e20069450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e4415b740 0x7f6e4413df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e441550b0 0x7f6e440b1c60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 --2- 172.21.15.87:0/2090049498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e440d0040 0x7f6e4414f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 >> 172.21.15.87:0/2090049498 conn(0x7f6e44066b70 msgr2=0x7f6e44153d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 shutdown_connections 2024-04-25T10:12:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.072+0000 7f6e49e00640 1 -- 172.21.15.87:0/2090049498 wait complete. 2024-04-25T10:12:29.095 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3202965986},{"type":"v1","addr":"172.21.15.87:6833","nonce":3202965986}]},"active_addr":"172.21.15.87:6833/3202965986","active_change":"2024-04-25T10:12:03.110828+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4099,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3293640394}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3902739093}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":1829545336}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3614202722}]}]} 2024-04-25T10:12:29.098 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:12:29.098 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:12:29.098 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1d01c00640 1 Processor -- start 2024-04-25T10:12:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1d01c00640 1 -- start start 2024-04-25T10:12:29.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1d01c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1cfc15bb50 0x7f1cfc0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1d01c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc0d24d0 0x7f1cfc0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1d01c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc0ce430 con 0x7f1cfc0d0490 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1d01c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc0b1070 con 0x7f1cfc15bb50 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1d01c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc0ce6d0 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1cfb400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1cfc0d0490 0x7f1cfc0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60542/0 (socket says 172.21.15.87:60542) 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.228+0000 7f1cfb400640 1 -- 172.21.15.87:0/3671384038 learned_addr learned my addr 172.21.15.87:0/3671384038 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cfaa00640 1 --2- 172.21.15.87:0/3671384038 >> v2:172.21.15.169:3300/0 conn(0x7f1cfc15bb50 0x7f1cfc0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cfa000640 1 --2- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc0d24d0 0x7f1cfc0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cf9600640 1 -- 172.21.15.87:0/3671384038 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985605538 0 0) 0x7f1cfc0ce430 con 0x7f1cfc0d0490 2024-04-25T10:12:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cf9600640 1 -- 172.21.15.87:0/3671384038 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce0003530 con 0x7f1cfc0d0490 2024-04-25T10:12:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cfaa00640 1 -- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc0d24d0 msgr2=0x7f1cfc0cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cfaa00640 1 --2- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc0d24d0 0x7f1cfc0cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cfaa00640 1 -- 172.21.15.87:0/3671384038 >> v1:172.21.15.87:6789/0 conn(0x7f1cfc0d0490 legacy=0x7f1cfc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cfaa00640 1 -- 172.21.15.87:0/3671384038 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc0ce9b0 con 0x7f1cfc15bb50 2024-04-25T10:12:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cfaa00640 1 --2- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc15bb50 0x7f1cfc0b0770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0097c0 tx=0x7f1cec01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d917f01e00f1956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.232+0000 7f1cf9600640 1 -- 172.21.15.87:0/3671384038 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0b0070 con 0x7f1cfc15bb50 2024-04-25T10:12:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1cf9600640 1 -- 172.21.15.87:0/3671384038 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec0ab600 con 0x7f1cfc15bb50 2024-04-25T10:12:29.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1cf9600640 1 -- 172.21.15.87:0/3671384038 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0ab9b0 con 0x7f1cfc15bb50 2024-04-25T10:12:29.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc15bb50 msgr2=0x7f1cfc0b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc15bb50 0x7f1cfc0b0770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0097c0 tx=0x7f1cec01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- 172.21.15.87:0/3671384038 shutdown_connections 2024-04-25T10:12:29.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc0d24d0 0x7f1cfc0cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3671384038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc15bb50 0x7f1cfc0b0770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- 172.21.15.87:0/3671384038 >> 172.21.15.87:0/3671384038 conn(0x7f1cfc0d1970 msgr2=0x7f1cfc0d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- 172.21.15.87:0/3671384038 shutdown_connections 2024-04-25T10:12:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- 172.21.15.87:0/3671384038 wait complete. 2024-04-25T10:12:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 Processor -- start 2024-04-25T10:12:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- start start 2024-04-25T10:12:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc0d0490 0x7f1cfc1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cfc0d24d0 0x7f1cfc164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc15bb50 0x7f1cfc06c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc168830 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc15c140 con 0x7f1cfc0d0490 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1d01c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc1689b0 con 0x7f1cfc15bb50 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cfc0d24d0 0x7f1cfc164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cfc0d24d0 0x7f1cfc164d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56968/0 (socket says 172.21.15.87:56968) 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1cfa000640 1 -- 172.21.15.87:0/3597086323 learned_addr learned my addr 172.21.15.87:0/3597086323 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1cfaa00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc0d0490 0x7f1cfc1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.236+0000 7f1cfb400640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc15bb50 0x7f1cfc06c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cfa000640 1 -- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc15bb50 msgr2=0x7f1cfc06c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cfa000640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc15bb50 0x7f1cfc06c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cfa000640 1 -- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc0d0490 msgr2=0x7f1cfc1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cfa000640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc0d0490 0x7f1cfc1516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cfa000640 1 -- 172.21.15.87:0/3597086323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc06b710 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cfa000640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cfc0d24d0 0x7f1cfc164d30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e900 tx=0x7f1ce800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cf2a00640 1 -- 172.21.15.87:0/3597086323 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8017070 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cf2a00640 1 -- 172.21.15.87:0/3597086323 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce800ca00 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc06b9f0 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cf2a00640 1 -- 172.21.15.87:0/3597086323 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800cdf0 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc06bfb0 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc04ef30 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cf2a00640 1 -- 172.21.15.87:0/3597086323 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ce801a070 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.240+0000 7f1cf2a00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cdc066f60 0x7f1cdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.244+0000 7f1cf2a00640 1 -- 172.21.15.87:0/3597086323 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ce8085380 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.244+0000 7f1cfaa00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cdc066f60 0x7f1cdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.244+0000 7f1cfaa00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cdc066f60 0x7f1cdc069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0ad000 tx=0x7f1cec0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.248+0000 7f1cf2a00640 1 -- 172.21.15.87:0/3597086323 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ce8055d30 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.400+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cfc0b70c0 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.400+0000 7f1cf2a00640 1 -- 172.21.15.87:0/3597086323 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f1ce8059dd0 con 0x7f1cfc0d24d0 2024-04-25T10:12:29.407 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:29.407 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cdc066f60 msgr2=0x7f1cdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cdc066f60 0x7f1cdc069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0ad000 tx=0x7f1cec0a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cfc0d24d0 msgr2=0x7f1cfc164d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cfc0d24d0 0x7f1cfc164d30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e900 tx=0x7f1ce800edc0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 shutdown_connections 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cdc066f60 0x7f1cdc069420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cfc15bb50 0x7f1cfc06c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cfc0d24d0 0x7f1cfc164d30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 --2- 172.21.15.87:0/3597086323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cfc0d0490 0x7f1cfc1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 >> 172.21.15.87:0/3597086323 conn(0x7f1cfc0d1970 msgr2=0x7f1cfc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 shutdown_connections 2024-04-25T10:12:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.404+0000 7f1d01c00640 1 -- 172.21.15.87:0/3597086323 wait complete. 2024-04-25T10:12:29.429 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:12:29.429 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:29.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faed6000640 1 Processor -- start 2024-04-25T10:12:29.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faed6000640 1 -- start start 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faed6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faed015b9e0 0x7faed00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faed6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faed00d1c30 0x7faed00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faed6000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00ccfb0 con 0x7faed015bdc0 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faed6000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00cd290 con 0x7faed015b9e0 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faed6000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00cd570 con 0x7faed00d1c30 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faecf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faed015b9e0 0x7faed00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faecf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faed015b9e0 0x7faed00cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56972/0 (socket says 172.21.15.87:56972) 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faecf400640 1 -- 172.21.15.87:0/709371409 learned_addr learned my addr 172.21.15.87:0/709371409 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:29.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faecf400640 1 -- 172.21.15.87:0/709371409 >> v2:172.21.15.169:3300/0 conn(0x7faed00d1c30 msgr2=0x7faed00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faecf400640 1 --2- 172.21.15.87:0/709371409 >> v2:172.21.15.169:3300/0 conn(0x7faed00d1c30 0x7faed00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faecf400640 1 -- 172.21.15.87:0/709371409 >> v1:172.21.15.169:6789/0 conn(0x7faed015bdc0 legacy=0x7faed00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.560+0000 7faecf400640 1 -- 172.21.15.87:0/709371409 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00cd850 con 0x7faed015b9e0 2024-04-25T10:12:29.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faecf400640 1 --2- 172.21.15.87:0/709371409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed00cc410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faed0070e60 tx=0x7faec001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915f688bd8163d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faecea00640 1 -- 172.21.15.87:0/709371409 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00b0070 con 0x7faed015b9e0 2024-04-25T10:12:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faecea00640 1 -- 172.21.15.87:0/709371409 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec00ab600 con 0x7faed015b9e0 2024-04-25T10:12:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faecea00640 1 -- 172.21.15.87:0/709371409 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00ab9d0 con 0x7faed015b9e0 2024-04-25T10:12:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 -- 172.21.15.87:0/709371409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 msgr2=0x7faed00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 --2- 172.21.15.87:0/709371409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed00cc410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faed0070e60 tx=0x7faec001c450 comp rx=0 tx=0).stop 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 -- 172.21.15.87:0/709371409 shutdown_connections 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 --2- 172.21.15.87:0/709371409 >> v2:172.21.15.169:3300/0 conn(0x7faed00d1c30 0x7faed00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 --2- 172.21.15.87:0/709371409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed00cc410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 -- 172.21.15.87:0/709371409 >> 172.21.15.87:0/709371409 conn(0x7faed009a9d0 msgr2=0x7faed009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 -- 172.21.15.87:0/709371409 shutdown_connections 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.564+0000 7faed6000640 1 -- 172.21.15.87:0/709371409 wait complete. 2024-04-25T10:12:29.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 Processor -- start 2024-04-25T10:12:29.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 -- start start 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed00d1c30 0x7faed015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed01537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed015bdc0 0x7faed0153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c6630 con 0x7faed015b9e0 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c64b0 con 0x7faed015bdc0 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c67b0 con 0x7faed00d1c30 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed01537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed01537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56982/0 (socket says 172.21.15.87:56982) 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 -- 172.21.15.87:0/2214720595 learned_addr learned my addr 172.21.15.87:0/2214720595 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faecfe00640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed015bdc0 0x7faed0153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faecf400640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed00d1c30 0x7faed015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 -- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed00d1c30 msgr2=0x7faed015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed00d1c30 0x7faed015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 -- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed015bdc0 msgr2=0x7faed0153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed015bdc0 0x7faed0153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 -- 172.21.15.87:0/2214720595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0154250 con 0x7faed015b9e0 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faecfe00640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed015bdc0 0x7faed0153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faecf400640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed00d1c30 0x7faed015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec6a00640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed01537d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faeb800b3e0 tx=0x7faeb800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec7e00640 1 -- 172.21.15.87:0/2214720595 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8015020 con 0x7faed015b9e0 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec7e00640 1 -- 172.21.15.87:0/2214720595 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8011850 con 0x7faed015b9e0 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faec7e00640 1 -- 172.21.15.87:0/2214720595 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8011c40 con 0x7faed015b9e0 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed0157800 con 0x7faed015b9e0 2024-04-25T10:12:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed01580a0 con 0x7faed015b9e0 2024-04-25T10:12:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.568+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed004ef30 con 0x7faed015b9e0 2024-04-25T10:12:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.572+0000 7faec7e00640 1 -- 172.21.15.87:0/2214720595 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faeb8018020 con 0x7faed015b9e0 2024-04-25T10:12:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.572+0000 7faec7e00640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeac066f60 0x7faeac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.572+0000 7faec7e00640 1 -- 172.21.15.87:0/2214720595 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faeb8084f80 con 0x7faed015b9e0 2024-04-25T10:12:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.572+0000 7faecf400640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeac066f60 0x7faeac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.572+0000 7faecf400640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeac066f60 0x7faeac069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faec00ad000 tx=0x7faec0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.576+0000 7faec7e00640 1 -- 172.21.15.87:0/2214720595 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faeb801f020 con 0x7faed015b9e0 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faed00cef20 con 0x7faed015b9e0 2024-04-25T10:12:29.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faec7e00640 1 -- 172.21.15.87:0/2214720595 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7faeb80172c0 con 0x7faed015b9e0 2024-04-25T10:12:29.737 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:29.737 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:29.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeac066f60 msgr2=0x7faeac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeac066f60 0x7faeac069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faec00ad000 tx=0x7faec0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:29.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 msgr2=0x7faed01537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed01537d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faeb800b3e0 tx=0x7faeb800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 shutdown_connections 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeac066f60 0x7faeac069420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed015bdc0 0x7faed0153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed015b9e0 0x7faed01537d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 --2- 172.21.15.87:0/2214720595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed00d1c30 0x7faed015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 >> 172.21.15.87:0/2214720595 conn(0x7faed009a9d0 msgr2=0x7faed006f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 shutdown_connections 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.732+0000 7faed6000640 1 -- 172.21.15.87:0/2214720595 wait complete. 2024-04-25T10:12:29.758 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:12:29.759 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:12:29.759 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:12:29.760 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:12:29.760 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:12:29.761 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:12:29.762 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:12:29.762 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:12:29.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.988+0000 7f2315400640 1 Processor -- start 2024-04-25T10:12:29.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.988+0000 7f2315400640 1 -- start start 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f2315400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2308055620 0x7f23080380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f2315400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2308053c40 0x7f23080df180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f2315400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308033e50 0x7f230804fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f2315400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23080541b0 con 0x7f2308055620 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f2315400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2308006210 con 0x7f2308053c40 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f2315400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23080064f0 con 0x7f2308033e50 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2308053c40 0x7f23080df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2308053c40 0x7f23080df180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:42912/0 (socket says 172.21.15.87:42912) 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 -- 172.21.15.87:0/3912558350 learned_addr learned my addr 172.21.15.87:0/3912558350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230f200640 1 --2- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308033e50 0x7f230804fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 -- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308033e50 msgr2=0x7f230804fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 --2- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308033e50 0x7f230804fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 -- 172.21.15.87:0/3912558350 >> v2:172.21.15.87:3300/0 conn(0x7f2308055620 msgr2=0x7f23080380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 --2- 172.21.15.87:0/3912558350 >> v2:172.21.15.87:3300/0 conn(0x7f2308055620 0x7f23080380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.992+0000 7f230de00640 1 -- 172.21.15.87:0/3912558350 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2308054490 con 0x7f2308053c40 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.996+0000 7f13bf000640 1 Processor -- start 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f230de00640 1 --2- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 0x7f23080df180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2308006140 tx=0x7f230001c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83f182ed39d17264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:29.996+0000 7f13bf000640 1 -- start start 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13bf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b00589d0 0x7f13b00e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13bf000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13b0006440 0x7f13b0006850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13bf000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b00571b0 con 0x7f13b0053540 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13bf000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b0057490 con 0x7f13b0006440 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13bf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0057770 con 0x7f13b00589d0 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f37ff200640 1 Processor -- start 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f37ff200640 1 -- start start 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13bdc00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f13b0053540 0x7f13b0053920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39912/0 (socket says 172.21.15.87:39912) 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13bdc00640 1 -- 172.21.15.87:0/2251413906 learned_addr learned my addr 172.21.15.87:0/2251413906 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f230d400640 1 -- 172.21.15.87:0/3912558350 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000ab5f0 con 0x7f2308053c40 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f230d400640 1 -- 172.21.15.87:0/3912558350 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23000b0070 con 0x7f2308053c40 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f230d400640 1 -- 172.21.15.87:0/3912558350 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000ab9b0 con 0x7f2308053c40 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524991232 0 0) 0x7f13b00571b0 con 0x7f13b0053540 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13a0003530 con 0x7f13b0053540 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 318948924 0 0) 0x7f13a0003530 con 0x7f13b0053540 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b00589d0 msgr2=0x7f13b00e7a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b00589d0 0x7f13b00e7a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 >> v2:172.21.15.87:3300/0 conn(0x7f13b0006440 msgr2=0x7f13b0006850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 --2- 172.21.15.87:0/2251413906 >> v2:172.21.15.87:3300/0 conn(0x7f13b0006440 0x7f13b0006850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0057a50 con 0x7f13b0053540 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f13b804f450 con 0x7f13b0053540 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 >> v1:172.21.15.169:6789/0 conn(0x7f13b0053540 legacy=0x7f13b0053920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00039c0 0x7f13a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13a0004810 0x7f13a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13a00051d0 0x7f13a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0057490 con 0x7f13a00051d0 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0057770 con 0x7f13a0004810 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a000c0d0 con 0x7f13a00039c0 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f37ff200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f37f8047c20 0x7f37f80eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.000+0000 7f37ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8005ea0 0x7f37f80062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 -- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 msgr2=0x7f23080df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 --2- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 0x7f23080df180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2308006140 tx=0x7f230001c520 comp rx=0 tx=0).stop 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13be600640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13a00051d0 0x7f13a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13bd200640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13a0004810 0x7f13a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37ff200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f8034220 con 0x7f37f8033e40 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13bdc00640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00039c0 0x7f13a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fe800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8005ea0 0x7f37f80062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fe800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8005ea0 0x7f37f80062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:37936/0 (socket says 172.21.15.87:37936) 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fe800640 1 -- 172.21.15.87:0/1243309131 learned_addr learned my addr 172.21.15.87:0/1243309131 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37ff200640 1 -- 172.21.15.87:0/1243309131 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f8050210 con 0x7f37f8047c20 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37ff200640 1 -- 172.21.15.87:0/1243309131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8037bc0 con 0x7f37f8005ea0 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13be600640 1 -- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00039c0 msgr2=0x7f13a0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fd400640 1 --2- 172.21.15.87:0/1243309131 >> v2:172.21.15.87:3300/0 conn(0x7f37f8047c20 0x7f37f80eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fe800640 1 -- 172.21.15.87:0/1243309131 >> v2:172.21.15.87:3300/0 conn(0x7f37f8047c20 msgr2=0x7f37f80eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fe800640 1 --2- 172.21.15.87:0/1243309131 >> v2:172.21.15.87:3300/0 conn(0x7f37f8047c20 0x7f37f80eb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fe800640 1 -- 172.21.15.87:0/1243309131 >> v1:172.21.15.87:6789/0 conn(0x7f37f8033e40 legacy=0x7f37f80eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f37fe800640 1 -- 172.21.15.87:0/1243309131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f804fa30 con 0x7f37f8005ea0 2024-04-25T10:12:30.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13be600640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00039c0 0x7f13a0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13be600640 1 -- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13a0004810 msgr2=0x7f13a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13be600640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13a0004810 0x7f13a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13be600640 1 -- 172.21.15.87:0/2251413906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0057a50 con 0x7f13a00051d0 2024-04-25T10:12:30.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f13bdc00640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00039c0 0x7f13a0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 -- 172.21.15.87:0/3912558350 shutdown_connections 2024-04-25T10:12:30.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 --2- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308033e50 0x7f230804fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 --2- 172.21.15.87:0/3912558350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 0x7f23080df180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 --2- 172.21.15.87:0/3912558350 >> v2:172.21.15.87:3300/0 conn(0x7f2308055620 0x7f23080380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 -- 172.21.15.87:0/3912558350 >> 172.21.15.87:0/3912558350 conn(0x7f2308054b00 msgr2=0x7f2308054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f0fe2e00640 1 Processor -- start 2024-04-25T10:12:30.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 -- 172.21.15.87:0/3912558350 shutdown_connections 2024-04-25T10:12:30.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f2315400640 1 -- 172.21.15.87:0/3912558350 wait complete. 2024-04-25T10:12:30.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7f0fe2e00640 1 -- start start 2024-04-25T10:12:30.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7fbc0a200640 1 Processor -- start 2024-04-25T10:12:30.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.004+0000 7fbc0a200640 1 -- start start 2024-04-25T10:12:30.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fe2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc0cc690 0x7f0fdc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fe2e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fdc06e450 0x7f0fdc070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fe2e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fdc06eaf0 con 0x7f0fdc15bd00 2024-04-25T10:12:30.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fe2e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fdc0cd200 con 0x7f0fdc06e450 2024-04-25T10:12:30.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fe2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0cd4e0 con 0x7f0fdc0cc690 2024-04-25T10:12:30.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 Processor -- start 2024-04-25T10:12:30.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc0cc690 0x7f0fdc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc0a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc040cc7d0 0x7fbc040ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc0a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbc040cd6c0 0x7fbc040caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc0a200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc04161a90 con 0x7fbc0415dd90 2024-04-25T10:12:30.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc0a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc0406eb20 con 0x7fbc040cd6c0 2024-04-25T10:12:30.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc0a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040ce590 con 0x7fbc040cc7d0 2024-04-25T10:12:30.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 -- start start 2024-04-25T10:12:30.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0fdc15bd00 0x7f0fdc0cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60556/0 (socket says 172.21.15.87:60556) 2024-04-25T10:12:30.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc0cc690 0x7f0fdc06df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:37952/0 (socket says 172.21.15.87:37952) 2024-04-25T10:12:30.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdbe00640 1 -- 172.21.15.87:0/2200698334 learned_addr learned my addr 172.21.15.87:0/2200698334 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fe0e00640 1 --2- 172.21.15.87:0/2200698334 >> v2:172.21.15.87:3300/0 conn(0x7f0fdc06e450 0x7f0fdc070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f13be600640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13a00051d0 0x7f13a000b9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009890 tx=0x7f13a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab98208872b4334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40b0070 con 0x7f13a00051d0 2024-04-25T10:12:30.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a40ab620 con 0x7f13a00051d0 2024-04-25T10:12:30.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f13b7e00640 1 -- 172.21.15.87:0/2251413906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4009460 con 0x7f13a00051d0 2024-04-25T10:12:30.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc040cc7d0 0x7fbc040ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc040cc7d0 0x7fbc040ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:37968/0 (socket says 172.21.15.87:37968) 2024-04-25T10:12:30.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc02a00640 1 -- 172.21.15.87:0/4233712684 learned_addr learned my addr 172.21.15.87:0/4233712684 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc03e00640 1 --2- 172.21.15.87:0/4233712684 >> v2:172.21.15.169:3300/0 conn(0x7fbc040cd6c0 0x7fbc040caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2308033e50 0x7f230804e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 0x7f2308035820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f37fe800640 1 --2- 172.21.15.87:0/1243309131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8005ea0 0x7f37f80062d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009870 tx=0x7f37f001ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb24378f40e99d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdb400640 1 -- 172.21.15.87:0/2200698334 >> v2:172.21.15.87:3300/0 conn(0x7f0fdc06e450 msgr2=0x7f0fdc070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdb400640 1 --2- 172.21.15.87:0/2200698334 >> v2:172.21.15.87:3300/0 conn(0x7f0fdc06e450 0x7f0fdc070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308055620 0x7f2308036590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080075f0 con 0x7f2308033e50 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2308007470 con 0x7f2308053c40 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f2315400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2308007770 con 0x7f2308055620 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f13bf000640 1 -- 172.21.15.87:0/2251413906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13a00051d0 msgr2=0x7f13a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f13bf000640 1 --2- 172.21.15.87:0/2251413906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13a00051d0 0x7f13a000b9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009890 tx=0x7f13a401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc02a00640 1 -- 172.21.15.87:0/4233712684 >> v2:172.21.15.169:3300/0 conn(0x7fbc040cd6c0 msgr2=0x7fbc040caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc02a00640 1 --2- 172.21.15.87:0/4233712684 >> v2:172.21.15.169:3300/0 conn(0x7fbc040cd6c0 0x7fbc040caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc02a00640 1 -- 172.21.15.87:0/4233712684 >> v1:172.21.15.87:6789/0 conn(0x7fbc0415dd90 legacy=0x7fbc040cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f230de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 0x7f2308035820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f13be600640 1 -- 172.21.15.87:0/2251413906 reap_dead start 2024-04-25T10:12:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f230f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308055620 0x7f2308036590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f230f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308055620 0x7f2308036590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:37974/0 (socket says 172.21.15.87:37974) 2024-04-25T10:12:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f230f200640 1 -- 172.21.15.87:0/3417406086 learned_addr learned my addr 172.21.15.87:0/3417406086 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- 172.21.15.87:0/2251413906 shutdown_connections 2024-04-25T10:12:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- 172.21.15.87:0/2251413906 >> 172.21.15.87:0/2251413906 conn(0x7f13b0058ef0 msgr2=0x7f13b0059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7fbc02a00640 1 -- 172.21.15.87:0/4233712684 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04161d70 con 0x7fbc040cc7d0 2024-04-25T10:12:30.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- 172.21.15.87:0/2251413906 shutdown_connections 2024-04-25T10:12:30.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdb400640 1 -- 172.21.15.87:0/2200698334 >> v1:172.21.15.87:6789/0 conn(0x7f0fdc15bd00 legacy=0x7f0fdc0cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- 172.21.15.87:0/2251413906 wait complete. 2024-04-25T10:12:30.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f230f200640 1 -- 172.21.15.87:0/3417406086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 msgr2=0x7f2308035820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f230f200640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2308053c40 0x7f2308035820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f230f200640 1 -- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2308033e50 msgr2=0x7f230804e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f230f200640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2308033e50 0x7f230804e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f230f200640 1 -- 172.21.15.87:0/3417406086 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2308034780 con 0x7f2308055620 2024-04-25T10:12:30.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.008+0000 7f0fdb400640 1 -- 172.21.15.87:0/2200698334 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0cd7c0 con 0x7f0fdc0cc690 2024-04-25T10:12:30.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f230f200640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308055620 0x7f2308036590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2310073950 tx=0x7f23100687d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f37fca00640 1 -- 172.21.15.87:0/1243309131 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00b5070 con 0x7f37f8005ea0 2024-04-25T10:12:30.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f37fca00640 1 -- 172.21.15.87:0/1243309131 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f00a89d0 con 0x7f37f8005ea0 2024-04-25T10:12:30.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 Processor -- start 2024-04-25T10:12:30.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f37fca00640 1 -- 172.21.15.87:0/1243309131 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00a8da0 con 0x7f37f8005ea0 2024-04-25T10:12:30.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- start start 2024-04-25T10:12:30.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00051d0 0x7f13b00df390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b00e1530 0x7f13b00d5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b000ac50 0x7f13b004da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0056f70 con 0x7f13b00e1530 2024-04-25T10:12:30.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f37ff200640 1 -- 172.21.15.87:0/1243309131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8005ea0 msgr2=0x7f37f80062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0056df0 con 0x7f13b000ac50 2024-04-25T10:12:30.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f13bf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00e87a0 con 0x7f13a00051d0 2024-04-25T10:12:30.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7f37ff200640 1 --2- 172.21.15.87:0/1243309131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8005ea0 0x7f37f80062d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009870 tx=0x7f37f001ed10 comp rx=0 tx=0).stop 2024-04-25T10:12:30.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7fe27f000640 1 Processor -- start 2024-04-25T10:12:30.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.012+0000 7fe27f000640 1 -- start start 2024-04-25T10:12:30.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fbc03e00640 1 --2- 172.21.15.87:0/4233712684 >> v2:172.21.15.169:3300/0 conn(0x7fbc040cd6c0 0x7fbc040caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:30.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe27f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 0x7fe278072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe27f000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780cf290 con 0x7fe2780cfcc0 2024-04-25T10:12:30.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe27f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780b4ae0 con 0x7fe2780b4470 2024-04-25T10:12:30.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe27f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b2980 con 0x7fe278162080 2024-04-25T10:12:30.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13bd200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b00e1530 0x7f13b00d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13bd200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b00e1530 0x7f13b00d5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57052/0 (socket says 172.21.15.87:57052) 2024-04-25T10:12:30.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13bd200640 1 -- 172.21.15.87:0/3276737305 learned_addr learned my addr 172.21.15.87:0/3276737305 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13bdc00640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00051d0 0x7f13b00df390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310094640 con 0x7f2308055620 2024-04-25T10:12:30.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23100947e0 con 0x7f2308055620 2024-04-25T10:12:30.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310094bb0 con 0x7f2308055620 2024-04-25T10:12:30.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f37ff200640 1 -- 172.21.15.87:0/1243309131 shutdown_connections 2024-04-25T10:12:30.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f37ff200640 1 --2- 172.21.15.87:0/1243309131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8005ea0 0x7f37f80062d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f37ff200640 1 --2- 172.21.15.87:0/1243309131 >> v2:172.21.15.87:3300/0 conn(0x7f37f8047c20 0x7f37f80eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f37ff200640 1 -- 172.21.15.87:0/1243309131 >> 172.21.15.87:0/1243309131 conn(0x7f37f8009fc0 msgr2=0x7f37f800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13be600640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b000ac50 0x7f13b004da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe2780b4470 0x7fe2780cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39918/0 (socket says 172.21.15.87:39918) 2024-04-25T10:12:30.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277400640 1 -- 172.21.15.87:0/700514533 learned_addr learned my addr 172.21.15.87:0/700514533 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2308034a60 con 0x7f2308055620 2024-04-25T10:12:30.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f37ff200640 1 -- 172.21.15.87:0/1243309131 shutdown_connections 2024-04-25T10:12:30.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797848872 0 0) 0x7fe2780b4ae0 con 0x7fe2780b4470 2024-04-25T10:12:30.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13be600640 1 -- 172.21.15.87:0/3276737305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00051d0 msgr2=0x7f13b00df390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13be600640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13a00051d0 0x7f13b00df390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13be600640 1 -- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b00e1530 msgr2=0x7f13b00d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13be600640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b00e1530 0x7f13b00d5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13be600640 1 -- 172.21.15.87:0/3276737305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b00e1ca0 con 0x7f13b000ac50 2024-04-25T10:12:30.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2308035020 con 0x7f2308055620 2024-04-25T10:12:30.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe25c003530 con 0x7fe2780b4470 2024-04-25T10:12:30.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277e00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 0x7fe278072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fdb400640 1 --2- 172.21.15.87:0/2200698334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc0cc690 0x7f0fdc06df10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009d90 tx=0x7f0fcc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee24d18cade33e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f37ff200640 1 -- 172.21.15.87:0/1243309131 wait complete. 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3114930836 0 0) 0x7fe25c003530 con 0x7fe2780b4470 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 msgr2=0x7fe278072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 0x7fe278072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 >> v1:172.21.15.87:6789/0 conn(0x7fe2780cfcc0 legacy=0x7fe27806c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780b2c60 con 0x7fe2780b4470 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe264001660 con 0x7fe2780b4470 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 >> v1:172.21.15.169:6789/0 conn(0x7fe2780b4470 legacy=0x7fe2780cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe25c0039c0 0x7fe25c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c004810 0x7fe25c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 0x7fe25c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b2980 con 0x7fe25c0039c0 2024-04-25T10:12:30.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c005550 con 0x7fe25c004810 2024-04-25T10:12:30.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c00bf10 con 0x7fe278162080 2024-04-25T10:12:30.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe27d000640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 0x7fe25c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277e00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe25c0039c0 0x7fe25c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277400640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c004810 0x7fe25c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fdaa00640 1 -- 172.21.15.87:0/2200698334 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0b2070 con 0x7f0fdc0cc690 2024-04-25T10:12:30.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2308035400 con 0x7f2308055620 2024-04-25T10:12:30.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fdaa00640 1 -- 172.21.15.87:0/2200698334 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc0a58b0 con 0x7f0fdc0cc690 2024-04-25T10:12:30.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fdaa00640 1 -- 172.21.15.87:0/2200698334 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0b2070 con 0x7f0fdc0cc690 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277e00640 1 -- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 msgr2=0x7fe25c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277e00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 0x7fe25c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277e00640 1 -- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c004810 msgr2=0x7fe25c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277e00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c004810 0x7fe25c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277e00640 1 -- 172.21.15.87:0/700514533 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780b2c60 con 0x7fe25c0039c0 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe277400640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c004810 0x7fe25c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fe27d000640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe278162080 0x7fe25c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fe2e00640 1 -- 172.21.15.87:0/2200698334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc0cc690 msgr2=0x7f0fdc06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fe2e00640 1 --2- 172.21.15.87:0/2200698334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc0cc690 0x7f0fdc06df10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009d90 tx=0x7f0fcc01c970 comp rx=0 tx=0).stop 2024-04-25T10:12:30.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13be600640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b000ac50 0x7f13b004da70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f13a40ab000 tx=0x7f13a0005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f37ff200640 1 Processor -- start 2024-04-25T10:12:30.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fe2e00640 1 -- 172.21.15.87:0/2200698334 shutdown_connections 2024-04-25T10:12:30.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fe2e00640 1 --2- 172.21.15.87:0/2200698334 >> v2:172.21.15.87:3300/0 conn(0x7f0fdc06e450 0x7f0fdc070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fe2e00640 1 --2- 172.21.15.87:0/2200698334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc0cc690 0x7f0fdc06df10 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009d90 tx=0x7f0fcc01c970 comp rx=0 tx=0).stop 2024-04-25T10:12:30.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f0fe2e00640 1 -- 172.21.15.87:0/2200698334 >> 172.21.15.87:0/2200698334 conn(0x7f0fdc0b1330 msgr2=0x7f0fdc0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40be070 con 0x7f13b000ac50 2024-04-25T10:12:30.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a40ab9d0 con 0x7f13b000ac50 2024-04-25T10:12:30.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40b0040 con 0x7f13b000ac50 2024-04-25T10:12:30.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f13bf000640 1 -- 172.21.15.87:0/3276737305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b00e1f80 con 0x7f13b000ac50 2024-04-25T10:12:30.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f13bf000640 1 -- 172.21.15.87:0/3276737305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b00e2540 con 0x7f13b000ac50 2024-04-25T10:12:30.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37ff200640 1 -- start start 2024-04-25T10:12:30.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.016+0000 7fbc02a00640 1 --2- 172.21.15.87:0/4233712684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc040cc7d0 0x7fbc040ca570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009870 tx=0x7fbbf401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ceb12cc06cb518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc02000640 1 -- 172.21.15.87:0/4233712684 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40aa610 con 0x7fbc040cc7d0 2024-04-25T10:12:30.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc02000640 1 -- 172.21.15.87:0/4233712684 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf40af070 con 0x7fbc040cc7d0 2024-04-25T10:12:30.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc02000640 1 -- 172.21.15.87:0/4233712684 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40aa9d0 con 0x7fbc040cc7d0 2024-04-25T10:12:30.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f138c001040 con 0x7f13b000ac50 2024-04-25T10:12:30.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f0fe2e00640 1 -- 172.21.15.87:0/2200698334 shutdown_connections 2024-04-25T10:12:30.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23100933e0 con 0x7f2308055620 2024-04-25T10:12:30.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f2306a00640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22f0066fb0 0x7f22f0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f0fe2e00640 1 -- 172.21.15.87:0/2200698334 wait complete. 2024-04-25T10:12:30.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f230e800640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22f0066fb0 0x7f22f0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc0a200640 1 -- 172.21.15.87:0/4233712684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc040cc7d0 msgr2=0x7fbc040ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37ff200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37f8005ea0 0x7f37f8035e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37ff200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f8033e40 0x7f37f81812f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc0a200640 1 --2- 172.21.15.87:0/4233712684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc040cc7d0 0x7fbc040ca570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009870 tx=0x7fbbf401c510 comp rx=0 tx=0).stop 2024-04-25T10:12:30.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc0a200640 1 -- 172.21.15.87:0/4233712684 shutdown_connections 2024-04-25T10:12:30.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc0a200640 1 --2- 172.21.15.87:0/4233712684 >> v2:172.21.15.169:3300/0 conn(0x7fbc040cd6c0 0x7fbc040caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc0a200640 1 --2- 172.21.15.87:0/4233712684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc040cc7d0 0x7fbc040ca570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc0a200640 1 -- 172.21.15.87:0/4233712684 >> 172.21.15.87:0/4233712684 conn(0x7fbc040b1330 msgr2=0x7fbc040b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23100cd530 con 0x7f2308055620 2024-04-25T10:12:30.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f2306a00640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7f22f006f1b0 0x7f22f00715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 --> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2310070e50 con 0x7f22f006f1b0 2024-04-25T10:12:30.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23100cd160 con 0x7f2308055620 2024-04-25T10:12:30.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7fbc0a200640 1 -- 172.21.15.87:0/4233712684 shutdown_connections 2024-04-25T10:12:30.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8047c20 0x7f37f80db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37ff200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f804fd80 con 0x7f37f8005ea0 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37ff200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f804f090 con 0x7f37f8033e40 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37ff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f804f5c0 con 0x7f37f8047c20 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8047c20 0x7f37f80db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8047c20 0x7f37f80db6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38002/0 (socket says 172.21.15.87:38002) 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 -- 172.21.15.87:0/441572538 learned_addr learned my addr 172.21.15.87:0/441572538 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f230e800640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22f0066fb0 0x7f22f0069470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f230804e540 tx=0x7f22fc006c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13a40c1070 con 0x7f13b000ac50 2024-04-25T10:12:30.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7fbc0a200640 1 -- 172.21.15.87:0/4233712684 wait complete. 2024-04-25T10:12:30.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 Processor -- start 2024-04-25T10:12:30.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 -- start start 2024-04-25T10:12:30.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f79f4e00640 1 Processor -- start 2024-04-25T10:12:30.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7fe277e00640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe25c0039c0 0x7fe25c003e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe26800ba30 tx=0x7fe26801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc94846c82b49b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.020+0000 7f37fd400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f8033e40 0x7f37f81812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 -- 172.21.15.87:0/441572538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f8033e40 msgr2=0x7f37f81812f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f8033e40 0x7f37f81812f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 -- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37f8005ea0 msgr2=0x7f37f8035e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37f8005ea0 0x7f37f8035e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 -- 172.21.15.87:0/441572538 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f80dbc30 con 0x7f37f8047c20 2024-04-25T10:12:30.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f79f4e00640 1 -- start start 2024-04-25T10:12:30.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13b5600640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1398067030 0x7f13980694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f230de00640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7f22f006f1b0 0x7f22f00715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13a411e9c0 con 0x7f13b000ac50 2024-04-25T10:12:30.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2680b2070 con 0x7fe25c0039c0 2024-04-25T10:12:30.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13b5600640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7f139806f250 0x7f1398071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2680ad410 con 0x7fe25c0039c0 2024-04-25T10:12:30.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7fe276a00640 1 -- 172.21.15.87:0/700514533 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26800b600 con 0x7fe25c0039c0 2024-04-25T10:12:30.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 --> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13a40ab620 con 0x7f139806f250 2024-04-25T10:12:30.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a411edb0 con 0x7f13b000ac50 2024-04-25T10:12:30.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13bdc00640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1398067030 0x7f13980694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc06e450 0x7f0fdc0c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37fe800640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8047c20 0x7f37f80db6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f37f001d3c0 tx=0x7f37f001bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15bd00 0x7f0fdc0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0cf510 0x7f0fdc202ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc071230 con 0x7f0fdc15bd00 2024-04-25T10:12:30.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0710b0 con 0x7f0fdc0cf510 2024-04-25T10:12:30.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0713b0 con 0x7f0fdc06e450 2024-04-25T10:12:30.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13bd200640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7f139806f250 0x7f1398071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f230de00640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7f22f006f1b0 0x7f22f00715d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00be070 con 0x7f37f8047c20 2024-04-25T10:12:30.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f00b0700 con 0x7f37f8047c20 2024-04-25T10:12:30.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00b5040 con 0x7f37f8047c20 2024-04-25T10:12:30.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fdb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15bd00 0x7f0fdc0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fdb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15bd00 0x7f0fdc0c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57076/0 (socket says 172.21.15.87:57076) 2024-04-25T10:12:30.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fdb400640 1 -- 172.21.15.87:0/916772782 learned_addr learned my addr 172.21.15.87:0/916772782 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fe0e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0cf510 0x7f0fdc202ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7fbc0a200640 1 Processor -- start 2024-04-25T10:12:30.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f80364e0 con 0x7f37f8047c20 2024-04-25T10:12:30.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f8036aa0 con 0x7f37f8047c20 2024-04-25T10:12:30.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f0fdbe00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc06e450 0x7f0fdc0c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f13bdc00640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1398067030 0x7f13980694f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13b804f200 tx=0x7f13b806d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7fbc0a200640 1 -- start start 2024-04-25T10:12:30.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.024+0000 7f79f4e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f79f00cf550 0x7f79f006e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79f4e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79f006caa0 0x7f79f0164760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79f4e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00b3020 con 0x7f79f0159830 2024-04-25T10:12:30.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79f4e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f00b3c30 con 0x7f79f00cf550 2024-04-25T10:12:30.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79f4e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f00b3300 con 0x7f79f006caa0 2024-04-25T10:12:30.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37f80053a0 con 0x7f37f8047c20 2024-04-25T10:12:30.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fdbe00640 1 -- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0cf510 msgr2=0x7f0fdc202ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fdbe00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0cf510 0x7f0fdc202ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fdbe00640 1 -- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15bd00 msgr2=0x7f0fdc0c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fdbe00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15bd00 0x7f0fdc0c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fdbe00640 1 -- 172.21.15.87:0/916772782 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0c6350 con 0x7f0fdc06e450 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fe0e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0cf510 0x7f0fdc202ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fdb400640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15bd00 0x7f0fdc0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fe27f000640 1 -- 172.21.15.87:0/700514533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe25c0039c0 msgr2=0x7fe25c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fe27f000640 1 --2- 172.21.15.87:0/700514533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe25c0039c0 0x7fe25c003e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe26800ba30 tx=0x7fe26801c6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fe27d000640 1 -- 172.21.15.87:0/700514533 reap_dead start 2024-04-25T10:12:30.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79efe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79f006caa0 0x7f79f0164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79efe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79f006caa0 0x7f79f0164760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:42990/0 (socket says 172.21.15.87:42990) 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79efe00640 1 -- 172.21.15.87:0/1006569668 learned_addr learned my addr 172.21.15.87:0/1006569668 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc0a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc040cd6c0 0x7fbc0414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc0a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc0415dd90 0x7fbc04156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc0a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc040c6660 0x7fbc040c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc0a200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040cb580 con 0x7fbc040c6660 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc0a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040cb400 con 0x7fbc040cd6c0 2024-04-25T10:12:30.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc0a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040ce7e0 con 0x7fbc0415dd90 2024-04-25T10:12:30.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fe27f000640 1 -- 172.21.15.87:0/700514533 shutdown_connections 2024-04-25T10:12:30.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fe27f000640 1 -- 172.21.15.87:0/700514533 >> 172.21.15.87:0/700514533 conn(0x7fe278066b70 msgr2=0x7fe278066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fdbe00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc06e450 0x7f0fdc0c30c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c430 tx=0x7f0fc400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79efe00640 1 -- 172.21.15.87:0/1006569668 >> v2:172.21.15.87:3300/0 conn(0x7f79f00cf550 msgr2=0x7f79f006e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79ef400640 1 --2- 172.21.15.87:0/1006569668 >> v2:172.21.15.87:3300/0 conn(0x7f79f00cf550 0x7f79f006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fe27f000640 1 -- 172.21.15.87:0/700514533 shutdown_connections 2024-04-25T10:12:30.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc0415dd90 0x7fbc04156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc0415dd90 0x7fbc04156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38012/0 (socket says 172.21.15.87:38012) 2024-04-25T10:12:30.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 -- 172.21.15.87:0/1353293352 learned_addr learned my addr 172.21.15.87:0/1353293352 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4017070 con 0x7f0fdc06e450 2024-04-25T10:12:30.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc4005cf0 con 0x7f0fdc06e450 2024-04-25T10:12:30.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc15d750 con 0x7f0fdc06e450 2024-04-25T10:12:30.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== osd.0 v2:172.21.15.87:6800/2221934810 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2310070e50 con 0x7f22f006f1b0 2024-04-25T10:12:30.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40072d0 con 0x7f0fdc06e450 2024-04-25T10:12:30.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc15dd10 con 0x7f0fdc06e450 2024-04-25T10:12:30.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79efe00640 1 --2- 172.21.15.87:0/1006569668 >> v2:172.21.15.87:3300/0 conn(0x7f79f00cf550 0x7f79f006e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79efe00640 1 -- 172.21.15.87:0/1006569668 >> v1:172.21.15.87:6789/0 conn(0x7f79f0159830 legacy=0x7f79f006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79efe00640 1 -- 172.21.15.87:0/1006569668 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00b35e0 con 0x7f79f006caa0 2024-04-25T10:12:30.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fe27f000640 1 -- 172.21.15.87:0/700514533 wait complete. 2024-04-25T10:12:30.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f79ef400640 1 --2- 172.21.15.87:0/1006569668 >> v2:172.21.15.87:3300/0 conn(0x7f79f00cf550 0x7f79f006e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 -- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc040cd6c0 msgr2=0x7fbc0414f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fdc075190 con 0x7f0fdc06e450 2024-04-25T10:12:30.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f13bd200640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7f139806f250 0x7f1398071670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc03400640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc040cd6c0 0x7fbc0414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc040cd6c0 0x7fbc0414f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 -- 172.21.15.87:0/1353293352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc040c6660 msgr2=0x7fbc040c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc040c6660 0x7fbc040c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 -- 172.21.15.87:0/1353293352 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc040c6b60 con 0x7fbc0415dd90 2024-04-25T10:12:30.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37f00b0ad0 con 0x7f37f8047c20 2024-04-25T10:12:30.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc03400640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc040cd6c0 0x7fbc0414f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.028+0000 7fbc02a00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc0415dd90 0x7fbc04156290 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40ac040 tx=0x7fbbf401ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f37f6000640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f37e0066f60 0x7f37e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37f0121e00 con 0x7f37f8047c20 2024-04-25T10:12:30.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fc401a070 con 0x7f0fdc06e450 2024-04-25T10:12:30.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63be00640 1 Processor -- start 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fe27f000640 1 Processor -- start 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fe27f000640 1 -- start start 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40bd5f0 con 0x7fbc0415dd90 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf40bd790 con 0x7fbc0415dd90 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40bdb80 con 0x7fbc0415dd90 2024-04-25T10:12:30.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fbc0a200640 1 -- 172.21.15.87:0/1353293352 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc040c8f30 con 0x7fbc0415dd90 2024-04-25T10:12:30.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fbc0a200640 1 -- 172.21.15.87:0/1353293352 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc041567d0 con 0x7fbc0415dd90 2024-04-25T10:12:30.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== osd.1 v2:172.21.15.87:6816/191811739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f13a40ab620 con 0x7f139806f250 2024-04-25T10:12:30.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f37f6000640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f37e006f180 0x7f37e00715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 --> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37f00bce50 con 0x7f37e006f180 2024-04-25T10:12:30.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37f01221f0 con 0x7f37f8047c20 2024-04-25T10:12:30.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f37fde00640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f37e0066f60 0x7f37e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63be00640 1 -- start start 2024-04-25T10:12:30.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb63c162290 0x7fb63c0b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fe27f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe278155360 0x7fe2780aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb63c0d03d0 0x7fb63c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0d0d20 con 0x7fb63c165ed0 2024-04-25T10:12:30.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0d1c60 con 0x7fb63c162290 2024-04-25T10:12:30.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fe27f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe278155aa0 0x7fe2780af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f37fd400640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f37e006f180 0x7f37e00715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c166490 con 0x7fb63c0d03d0 2024-04-25T10:12:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fe27f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2781ff9e0 0x7fe2780af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbdc001040 con 0x7fbc0415dd90 2024-04-25T10:12:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7f0fd3e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066fb0 0x7f0fb8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fc4084450 con 0x7f0fdc06e450 2024-04-25T10:12:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f37fde00640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f37e0066f60 0x7f37e0069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3800073c30 tx=0x7f380006a180 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806cb10 con 0x7fe278155aa0 2024-04-25T10:12:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b4890 con 0x7fe278155360 2024-04-25T10:12:30.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f0fd3e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7f0fb806f230 0x7f0fb8071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 --> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fb8071d00 con 0x7f0fb806f230 2024-04-25T10:12:30.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc4084800 con 0x7f0fdc06e450 2024-04-25T10:12:30.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb63c162290 0x7fb63c0b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb63c162290 0x7fb63c0b6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57082/0 (socket says 172.21.15.87:57082) 2024-04-25T10:12:30.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.032+0000 7fb63aa00640 1 -- 172.21.15.87:0/2154185025 learned_addr learned my addr 172.21.15.87:0/2154185025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fb63b400640 1 --2- 172.21.15.87:0/2154185025 >> v2:172.21.15.169:3300/0 conn(0x7fb63c0d03d0 0x7fb63c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fb63aa00640 1 -- 172.21.15.87:0/2154185025 >> v2:172.21.15.169:3300/0 conn(0x7fb63c0d03d0 msgr2=0x7fb63c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fb63aa00640 1 --2- 172.21.15.87:0/2154185025 >> v2:172.21.15.169:3300/0 conn(0x7fb63c0d03d0 0x7fb63c0d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fb63aa00640 1 -- 172.21.15.87:0/2154185025 >> v1:172.21.15.169:6789/0 conn(0x7fb63c165ed0 legacy=0x7fb63c0ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806cc90 con 0x7fe2781ff9e0 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2781ff9e0 0x7fe2780af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2781ff9e0 0x7fe2780af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38028/0 (socket says 172.21.15.87:38028) 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 -- 172.21.15.87:0/4077121388 learned_addr learned my addr 172.21.15.87:0/4077121388 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe277e00640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe278155360 0x7fe2780aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fb63aa00640 1 -- 172.21.15.87:0/2154185025 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c166770 con 0x7fb63c162290 2024-04-25T10:12:30.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 -- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe278155360 msgr2=0x7fe2780aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe278155360 0x7fe2780aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 -- 172.21.15.87:0/4077121388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe278155aa0 msgr2=0x7fe2780af3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:12:30.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f0fe0e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7f0fb806f230 0x7f0fb8071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe278155aa0 0x7fe2780af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 -- 172.21.15.87:0/4077121388 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780b0870 con 0x7fe2781ff9e0 2024-04-25T10:12:30.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79efe00640 1 --2- 172.21.15.87:0/1006569668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f006caa0 0x7f79f0164760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79f00cf930 tx=0x7f79e801eb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327fb96af4ce1107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f0fdb400640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066fb0 0x7f0fb8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f0fdb400640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066fb0 0x7f0fb8069470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009d90 tx=0x7f0fcc0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbf40c8470 con 0x7fbc0415dd90 2024-04-25T10:12:30.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27d000640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2781ff9e0 0x7fe2780af920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe27806d900 tx=0x7fe27000ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264007cf0 con 0x7fe2781ff9e0 2024-04-25T10:12:30.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270018070 con 0x7fe2781ff9e0 2024-04-25T10:12:30.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270001970 con 0x7fe2781ff9e0 2024-04-25T10:12:30.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79ee000640 1 -- 172.21.15.87:0/1006569668 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800b2d0 con 0x7f79f006caa0 2024-04-25T10:12:30.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79ee000640 1 -- 172.21.15.87:0/1006569668 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e800b470 con 0x7f79f006caa0 2024-04-25T10:12:30.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79ee000640 1 -- 172.21.15.87:0/1006569668 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80a7bb0 con 0x7f79f006caa0 2024-04-25T10:12:30.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f37fd400640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f37e006f180 0x7f37e00715a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2781fef20 con 0x7fe2781ff9e0 2024-04-25T10:12:30.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2781ff4e0 con 0x7fe2781ff9e0 2024-04-25T10:12:30.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79f4e00640 1 -- 172.21.15.87:0/1006569668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f006caa0 msgr2=0x7f79f0164760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79f4e00640 1 --2- 172.21.15.87:0/1006569668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f006caa0 0x7f79f0164760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79f00cf930 tx=0x7f79e801eb30 comp rx=0 tx=0).stop 2024-04-25T10:12:30.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79f4e00640 1 -- 172.21.15.87:0/1006569668 shutdown_connections 2024-04-25T10:12:30.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79f4e00640 1 --2- 172.21.15.87:0/1006569668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f006caa0 0x7f79f0164760 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79f4e00640 1 --2- 172.21.15.87:0/1006569668 >> v2:172.21.15.87:3300/0 conn(0x7f79f00cf550 0x7f79f006e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79f4e00640 1 -- 172.21.15.87:0/1006569668 >> 172.21.15.87:0/1006569668 conn(0x7f79f0066bd0 msgr2=0x7f79f0066fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbbfb400640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbe4066fb0 0x7fbbe4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7f79f4e00640 1 -- 172.21.15.87:0/1006569668 shutdown_connections 2024-04-25T10:12:30.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7f79f4e00640 1 -- 172.21.15.87:0/1006569668 wait complete. 2024-04-25T10:12:30.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbf411fda0 con 0x7fbc0415dd90 2024-04-25T10:12:30.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.036+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe27804ef30 con 0x7fe2781ff9e0 2024-04-25T10:12:30.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7f0fe0e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7f0fb806f230 0x7f0fb8071650 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbbfb400640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fbbe406f1d0 0x7fbbe40715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 --> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbf4006800 con 0x7fbbe406f1d0 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf4120150 con 0x7fbc0415dd90 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbc03e00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fbbe406f1d0 0x7fbbe40715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbc03400640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbe4066fb0 0x7fbbe4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== osd.2 v2:172.21.15.87:6824/2913449321 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f37f00bce50 con 0x7f37e006f180 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fb63aa00640 1 --2- 172.21.15.87:0/2154185025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 0x7fb63c0b6d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0ca960 tx=0x7fb63401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f163ebbbea080d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fb639600640 1 -- 172.21.15.87:0/2154185025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6340b0070 con 0x7fb63c162290 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fb639600640 1 -- 172.21.15.87:0/2154185025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6340ab600 con 0x7fb63c162290 2024-04-25T10:12:30.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fb639600640 1 -- 172.21.15.87:0/2154185025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6340ab9d0 con 0x7fb63c162290 2024-04-25T10:12:30.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2700050d0 con 0x7fe2781ff9e0 2024-04-25T10:12:30.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== osd.3 v2:172.21.15.87:6802/327355267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0fb8071d00 con 0x7f0fb806f230 2024-04-25T10:12:30.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7f79f4e00640 1 Processor -- start 2024-04-25T10:12:30.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbc03400640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbe4066fb0 0x7fbbe4069470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000a830 tx=0x7fbbf0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fbc03e00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fbbe406f1d0 0x7fbbe40715f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fb63be00640 1 -- 172.21.15.87:0/2154185025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 msgr2=0x7fb63c0b6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fb63be00640 1 --2- 172.21.15.87:0/2154185025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 0x7fb63c0b6d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0ca960 tx=0x7fb63401c270 comp rx=0 tx=0).stop 2024-04-25T10:12:30.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7fe26fe00640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe258066fe0 0x7fe2580694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe270084d40 con 0x7fe2781ff9e0 2024-04-25T10:12:30.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fb63be00640 1 -- 172.21.15.87:0/2154185025 shutdown_connections 2024-04-25T10:12:30.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fb63be00640 1 --2- 172.21.15.87:0/2154185025 >> v2:172.21.15.169:3300/0 conn(0x7fb63c0d03d0 0x7fb63c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fb63be00640 1 --2- 172.21.15.87:0/2154185025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 0x7fb63c0b6d50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fb63be00640 1 -- 172.21.15.87:0/2154185025 >> 172.21.15.87:0/2154185025 conn(0x7fb63c0669a0 msgr2=0x7fb63c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe26fe00640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7fe25806f260 0x7fe258071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 --> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe27000ce50 con 0x7fe25806f260 2024-04-25T10:12:30.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe270087360 con 0x7fe2781ff9e0 2024-04-25T10:12:30.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fb63be00640 1 -- 172.21.15.87:0/2154185025 shutdown_connections 2024-04-25T10:12:30.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== osd.4 v2:172.21.15.169:6800/3149326747 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbbf4006800 con 0x7fbbe406f1d0 2024-04-25T10:12:30.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fb63be00640 1 -- 172.21.15.87:0/2154185025 wait complete. 2024-04-25T10:12:30.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe277400640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7fe25806f260 0x7fe258071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe277e00640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe258066fe0 0x7fe2580694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe277400640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7fe25806f260 0x7fe258071680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fe277e00640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe258066fe0 0x7fe2580694a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2680af000 tx=0x7fe2680046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7fb63be00640 1 Processor -- start 2024-04-25T10:12:30.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63be00640 1 -- start start 2024-04-25T10:12:30.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.040+0000 7f79f4e00640 1 -- start start 2024-04-25T10:12:30.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79f4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f006caa0 0x7f79f0157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79f4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00cf550 0x7f79f0158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79f4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0159830 0x7f79f01542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79f4e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b2dc0 con 0x7f79f006caa0 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79f4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b2c40 con 0x7f79f00cf550 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79f4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0070bb0 con 0x7f79f0159830 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00cf550 0x7f79f0158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00cf550 0x7f79f0158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43034/0 (socket says 172.21.15.87:43034) 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 -- 172.21.15.87:0/2124353285 learned_addr learned my addr 172.21.15.87:0/2124353285 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79ef400640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f006caa0 0x7f79f0157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 -- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0159830 msgr2=0x7f79f01542e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0159830 0x7f79f01542e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 -- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f006caa0 msgr2=0x7f79f0157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f006caa0 0x7f79f0157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 -- 172.21.15.87:0/2124353285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0157ea0 con 0x7f79f00cf550 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79eea00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00cf550 0x7f79f0158f40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c500 tx=0x7f79e000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.044+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0016020 con 0x7f79f00cf550 2024-04-25T10:12:30.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== osd.6 v2:172.21.15.169:6808/3889701998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe27000ce50 con 0x7fe25806f260 2024-04-25T10:12:30.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0005220 con 0x7f79f00cf550 2024-04-25T10:12:30.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0007030 con 0x7f79f00cf550 2024-04-25T10:12:30.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb63c0d03d0 0x7fb63c06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 0x7fb63c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb63c165ed0 0x7fb63c0c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0d2a80 con 0x7fb63c162290 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0d2900 con 0x7fb63c165ed0 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0d2c00 con 0x7fb63c0d03d0 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb63c0d03d0 0x7fb63c06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb63c165ed0 0x7fb63c0c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb63c0d03d0 0x7fb63c06d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38034/0 (socket says 172.21.15.87:38034) 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 -- 172.21.15.87:0/1682499925 learned_addr learned my addr 172.21.15.87:0/1682499925 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63a000640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 0x7fb63c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 -- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb63c165ed0 msgr2=0x7fb63c0c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb63c165ed0 0x7fb63c0c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 -- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 msgr2=0x7fb63c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb63c162290 0x7fb63c0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 -- 172.21.15.87:0/1682499925 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0d3100 con 0x7fb63c0d03d0 2024-04-25T10:12:30.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb63aa00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb63c0d03d0 0x7fb63c06d8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ae040 tx=0x7fb634019010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.048+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6340b0040 con 0x7fb63c0d03d0 2024-04-25T10:12:30.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6340b4650 con 0x7fb63c0d03d0 2024-04-25T10:12:30.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6340b4a40 con 0x7fb63c0d03d0 2024-04-25T10:12:30.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c06ddf0 con 0x7fb63c0d03d0 2024-04-25T10:12:30.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c15d930 con 0x7fb63c0d03d0 2024-04-25T10:12:30.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb63c15d400 con 0x7fb63c0d03d0 2024-04-25T10:12:30.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f0158180 con 0x7f79f00cf550 2024-04-25T10:12:30.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f0158740 con 0x7f79f00cf550 2024-04-25T10:12:30.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79f0075120 con 0x7f79f00cf550 2024-04-25T10:12:30.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.052+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79e00071d0 con 0x7f79f00cf550 2024-04-25T10:12:30.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79e7400640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0088f70 0x7f79d008b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6340b4be0 con 0x7fb63c0d03d0 2024-04-25T10:12:30.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79e0084ae0 con 0x7f79f00cf550 2024-04-25T10:12:30.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79e7400640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f79d0091110 0x7f79d0093530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 --> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79e000bf30 con 0x7f79d0091110 2024-04-25T10:12:30.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79e00892a0 con 0x7f79f00cf550 2024-04-25T10:12:30.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 --> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f23080053d0 con 0x7f22f006f1b0 2024-04-25T10:12:30.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79efe00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f79d0091110 0x7f79d0093530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb632a00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb61c066f60 0x7fb61c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb63411eff0 con 0x7fb63c0d03d0 2024-04-25T10:12:30.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb632a00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7fb61c06f100 0x7fb61c071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 --> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb61c071bd0 con 0x7fb61c06f100 2024-04-25T10:12:30.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79efe00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f79d0091110 0x7f79d0093530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f79ef400640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0088f70 0x7f79d008b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb63b400640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7fb61c06f100 0x7fb61c071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb63a000640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb61c066f60 0x7fb61c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7fb63b400640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7fb61c06f100 0x7fb61c071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6340ebc80 con 0x7fb63c0d03d0 2024-04-25T10:12:30.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== osd.7 v2:172.21.15.169:6824/2370587194 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f79e000bf30 con 0x7f79d0091110 2024-04-25T10:12:30.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== osd.5 v2:172.21.15.169:6809/1054103150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb61c071bd0 con 0x7fb61c06f100 2024-04-25T10:12:30.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.056+0000 7f2306a00640 1 -- 172.21.15.87:0/3417406086 <== osd.0 v2:172.21.15.87:6800/2221934810 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f23080053d0 con 0x7f22f006f1b0 2024-04-25T10:12:30.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7f22f006f1b0 msgr2=0x7f22f00715d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7f22f006f1b0 0x7f22f00715d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f79ef400640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0088f70 0x7f79d008b430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22f0066fb0 msgr2=0x7f22f0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22f0066fb0 0x7f22f0069470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f230804e540 tx=0x7f22fc006c00 comp rx=0 tx=0).stop 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308055620 msgr2=0x7f2308036590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 --2- 172.21.15.87:0/3417406086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2308055620 0x7f2308036590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2310073950 tx=0x7f23100687d0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f230f200640 1 -- 172.21.15.87:0/3417406086 reap_dead start 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 shutdown_connections 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.060+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 >> 172.21.15.87:0/3417406086 conn(0x7f2308054b00 msgr2=0x7f23080d5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7fb63a000640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb61c066f60 0x7fb61c069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb63c15ee60 tx=0x7fb62c0094d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 shutdown_connections 2024-04-25T10:12:30.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f2315400640 1 -- 172.21.15.87:0/3417406086 wait complete. 2024-04-25T10:12:30.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 --> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f138c002de0 con 0x7f139806f250 2024-04-25T10:12:30.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13b5600640 1 -- 172.21.15.87:0/3276737305 <== osd.1 v2:172.21.15.87:6816/191811739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f138c002de0 con 0x7f139806f250 2024-04-25T10:12:30.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7f139806f250 msgr2=0x7f1398071670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13aaa00640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7f139806f250 0x7f1398071670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1398067030 msgr2=0x7f13980694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13aaa00640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1398067030 0x7f13980694f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13b804f200 tx=0x7f13b806d040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b000ac50 msgr2=0x7f13b004da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.064+0000 7f13aaa00640 1 --2- 172.21.15.87:0/3276737305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b000ac50 0x7f13b004da70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f13a40ab000 tx=0x7f13a0005840 comp rx=0 tx=0).stop 2024-04-25T10:12:30.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.068+0000 7f13be600640 1 -- 172.21.15.87:0/3276737305 reap_dead start 2024-04-25T10:12:30.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.068+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 shutdown_connections 2024-04-25T10:12:30.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.068+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 >> 172.21.15.87:0/3276737305 conn(0x7f13b0058ef0 msgr2=0x7f13b00e9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.068+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 shutdown_connections 2024-04-25T10:12:30.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.068+0000 7f13aaa00640 1 -- 172.21.15.87:0/3276737305 wait complete. 2024-04-25T10:12:30.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 --> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbdc002de0 con 0x7fbbe406f1d0 2024-04-25T10:12:30.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 --> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f37f8036ef0 con 0x7f37e006f180 2024-04-25T10:12:30.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbfb400640 1 -- 172.21.15.87:0/1353293352 <== osd.4 v2:172.21.15.169:6800/3149326747 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbbdc002de0 con 0x7fbbe406f1d0 2024-04-25T10:12:30.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 --> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fdc067e20 con 0x7f0fb806f230 2024-04-25T10:12:30.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fbbe406f1d0 msgr2=0x7fbbe40715f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fbbe406f1d0 0x7fbbe40715f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbe4066fb0 msgr2=0x7fbbe4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7f37f6000640 1 -- 172.21.15.87:0/441572538 <== osd.2 v2:172.21.15.87:6824/2913449321 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f37f8036ef0 con 0x7f37e006f180 2024-04-25T10:12:30.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbe4066fb0 0x7fbbe4069470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000a830 tx=0x7fbbf0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc0415dd90 msgr2=0x7fbc04156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 --2- 172.21.15.87:0/1353293352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc0415dd90 0x7fbc04156290 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40ac040 tx=0x7fbbf401ad50 comp rx=0 tx=0).stop 2024-04-25T10:12:30.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbc03e00640 1 -- 172.21.15.87:0/1353293352 reap_dead start 2024-04-25T10:12:30.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 shutdown_connections 2024-04-25T10:12:30.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 >> 172.21.15.87:0/1353293352 conn(0x7fbc040b1330 msgr2=0x7fbc040b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f37e006f180 msgr2=0x7f37e00715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7f0fd3e00640 1 -- 172.21.15.87:0/916772782 <== osd.3 v2:172.21.15.87:6802/327355267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0fdc067e20 con 0x7f0fb806f230 2024-04-25T10:12:30.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 shutdown_connections 2024-04-25T10:12:30.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7fbbf8c00640 1 -- 172.21.15.87:0/1353293352 wait complete. 2024-04-25T10:12:30.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7f37ff200640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f37e006f180 0x7f37e00715a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.072+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f37e0066f60 msgr2=0x7f37e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f37ff200640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f37e0066f60 0x7f37e0069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3800073c30 tx=0x7f380006a180 comp rx=0 tx=0).stop 2024-04-25T10:12:30.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8047c20 msgr2=0x7f37f80db6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f37ff200640 1 --2- 172.21.15.87:0/441572538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f8047c20 0x7f37f80db6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f37f001d3c0 tx=0x7f37f001bab0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f37fe800640 1 -- 172.21.15.87:0/441572538 reap_dead start 2024-04-25T10:12:30.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 shutdown_connections 2024-04-25T10:12:30.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 >> 172.21.15.87:0/441572538 conn(0x7f37f8009fc0 msgr2=0x7f37f80e6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7f0fb806f230 msgr2=0x7f0fb8071650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7f0fb806f230 0x7f0fb8071650 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066fb0 msgr2=0x7f0fb8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066fb0 0x7f0fb8069470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009d90 tx=0x7f0fcc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc06e450 msgr2=0x7f0fdc0c30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 --2- 172.21.15.87:0/916772782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc06e450 0x7f0fdc0c30c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c430 tx=0x7f0fc400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe0e00640 1 -- 172.21.15.87:0/916772782 reap_dead start 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 shutdown_connections 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 >> 172.21.15.87:0/916772782 conn(0x7f0fdc0b1330 msgr2=0x7f0fdc159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 shutdown_connections 2024-04-25T10:12:30.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.076+0000 7f0fe2e00640 1 -- 172.21.15.87:0/916772782 wait complete. 2024-04-25T10:12:30.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.080+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 shutdown_connections 2024-04-25T10:12:30.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7f37ff200640 1 -- 172.21.15.87:0/441572538 wait complete. 2024-04-25T10:12:30.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 --> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe2780cf070 con 0x7fe25806f260 2024-04-25T10:12:30.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe26fe00640 1 -- 172.21.15.87:0/4077121388 <== osd.6 v2:172.21.15.169:6808/3889701998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe2780cf070 con 0x7fe25806f260 2024-04-25T10:12:30.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 --> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb63c075120 con 0x7fb61c06f100 2024-04-25T10:12:30.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb632a00640 1 -- 172.21.15.87:0/1682499925 <== osd.5 v2:172.21.15.169:6809/1054103150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb63c075120 con 0x7fb61c06f100 2024-04-25T10:12:30.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7fb61c06f100 msgr2=0x7fb61c071520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7fb61c06f100 0x7fb61c071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7fe25806f260 msgr2=0x7fe258071680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe27f000640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7fe25806f260 0x7fe258071680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb61c066f60 msgr2=0x7fb61c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb61c066f60 0x7fb61c069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb63c15ee60 tx=0x7fb62c0094d0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb63c0d03d0 msgr2=0x7fb63c06d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 --2- 172.21.15.87:0/1682499925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb63c0d03d0 0x7fb63c06d8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ae040 tx=0x7fb634019010 comp rx=0 tx=0).stop 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63b400640 1 -- 172.21.15.87:0/1682499925 reap_dead start 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 shutdown_connections 2024-04-25T10:12:30.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 >> 172.21.15.87:0/1682499925 conn(0x7fb63c0669a0 msgr2=0x7fb63c0d3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe258066fe0 msgr2=0x7fe2580694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe27f000640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe258066fe0 0x7fe2580694a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2680af000 tx=0x7fe2680046d0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2781ff9e0 msgr2=0x7fe2780af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.084+0000 7fe27f000640 1 --2- 172.21.15.87:0/4077121388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2781ff9e0 0x7fe2780af920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe27806d900 tx=0x7fe27000ff70 comp rx=0 tx=0).stop 2024-04-25T10:12:30.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 shutdown_connections 2024-04-25T10:12:30.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7fb63be00640 1 -- 172.21.15.87:0/1682499925 wait complete. 2024-04-25T10:12:30.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7fe27d000640 1 -- 172.21.15.87:0/4077121388 reap_dead start 2024-04-25T10:12:30.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 shutdown_connections 2024-04-25T10:12:30.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 >> 172.21.15.87:0/4077121388 conn(0x7fe278066b70 msgr2=0x7fe2780ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 --> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79f006e810 con 0x7f79d0091110 2024-04-25T10:12:30.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79e7400640 1 -- 172.21.15.87:0/2124353285 <== osd.7 v2:172.21.15.169:6824/2370587194 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f79f006e810 con 0x7f79d0091110 2024-04-25T10:12:30.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 shutdown_connections 2024-04-25T10:12:30.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7fe27f000640 1 -- 172.21.15.87:0/4077121388 wait complete. 2024-04-25T10:12:30.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f79d0091110 msgr2=0x7f79d0093530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f79d0091110 0x7f79d0093530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0088f70 msgr2=0x7f79d008b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0088f70 0x7f79d008b430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc008040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00cf550 msgr2=0x7f79f0158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 --2- 172.21.15.87:0/2124353285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00cf550 0x7f79f0158f40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c500 tx=0x7f79e000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79efe00640 1 -- 172.21.15.87:0/2124353285 reap_dead start 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 shutdown_connections 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 >> 172.21.15.87:0/2124353285 conn(0x7f79f0066bd0 msgr2=0x7f79f00cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 shutdown_connections 2024-04-25T10:12:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.088+0000 7f79f4e00640 1 -- 172.21.15.87:0/2124353285 wait complete. 2024-04-25T10:12:30.118 INFO:teuthology.orchestra.run.smithi087.stdout:55834574855 2024-04-25T10:12:30.118 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:12:30.119 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:30.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:12:30.121 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:30.121 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:12:30.124 INFO:teuthology.orchestra.run.smithi087.stdout:55834574855 2024-04-25T10:12:30.125 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:12:30.132 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:30.132 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:12:30.135 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:30.136 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:30.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:12:30.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:12:30.137 INFO:teuthology.orchestra.run.smithi087.stdout:55834574853 2024-04-25T10:12:30.138 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:12:30.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.328+0000 7f01f1c00640 1 Processor -- start 2024-04-25T10:12:30.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.328+0000 7f01f1c00640 1 -- start start 2024-04-25T10:12:30.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.328+0000 7f01f1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01ec06caa0 0x7f01ec0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0b3020 con 0x7f01ec159c10 2024-04-25T10:12:30.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0b3c30 con 0x7f01ec06caa0 2024-04-25T10:12:30.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b3300 con 0x7f01ec159830 2024-04-25T10:12:30.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01ec06caa0 0x7f01ec0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01ec06caa0 0x7f01ec0703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43058/0 (socket says 172.21.15.87:43058) 2024-04-25T10:12:30.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1200640 1 -- 172.21.15.87:0/1969862166 learned_addr learned my addr 172.21.15.87:0/1969862166 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01ebe00640 1 --2- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7fc7ae800640 1 Processor -- start 2024-04-25T10:12:30.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1200640 1 -- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 msgr2=0x7f01ec06e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1200640 1 --2- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec06e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1200640 1 -- 172.21.15.87:0/1969862166 >> v1:172.21.15.169:6789/0 conn(0x7f01ec159c10 legacy=0x7f01ec06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f01f1200640 1 -- 172.21.15.87:0/1969862166 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0b35e0 con 0x7f01ec06caa0 2024-04-25T10:12:30.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7fc7ae800640 1 -- start start 2024-04-25T10:12:30.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f8e13e00640 1 Processor -- start 2024-04-25T10:12:30.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.332+0000 7f8e13e00640 1 -- start start 2024-04-25T10:12:30.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a00556f0 0x7fc7a00dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ae800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a0054650 con 0x7fc7a0053c90 2024-04-25T10:12:30.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ae800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a0054930 con 0x7fc7a0050fc0 2024-04-25T10:12:30.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ae800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0039d90 con 0x7fc7a00556f0 2024-04-25T10:12:30.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a00556f0 0x7fc7a00dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e140b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8e140d03d0 0x7f8e140d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e14166490 con 0x7f8e14165ed0 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e140d0d60 con 0x7f8e140d03d0 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140d1c60 con 0x7f8e14162290 2024-04-25T10:12:30.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a00556f0 0x7fc7a00dfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38058/0 (socket says 172.21.15.87:38058) 2024-04-25T10:12:30.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ad400640 1 -- 172.21.15.87:0/700132966 learned_addr learned my addr 172.21.15.87:0/700132966 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ad400640 1 -- 172.21.15.87:0/700132966 >> v1:172.21.15.169:6789/0 conn(0x7fc7a0050fc0 legacy=0x7fc7a0006b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ad400640 1 -- 172.21.15.87:0/700132966 >> v1:172.21.15.87:6789/0 conn(0x7fc7a0053c90 legacy=0x7fc7a00508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e12000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8e14165ed0 0x7f8e140ce1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39944/0 (socket says 172.21.15.87:39944) 2024-04-25T10:12:30.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e12000640 1 -- 172.21.15.87:0/3679960226 learned_addr learned my addr 172.21.15.87:0/3679960226 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7fc7ad400640 1 -- 172.21.15.87:0/700132966 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0054c10 con 0x7fc7a00556f0 2024-04-25T10:12:30.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e12a00640 1 --2- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e140b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13400640 1 --2- 172.21.15.87:0/3679960226 >> v2:172.21.15.169:3300/0 conn(0x7f8e140d03d0 0x7f8e140d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e11600640 1 -- 172.21.15.87:0/3679960226 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700599093 0 0) 0x7f8e14166490 con 0x7f8e14165ed0 2024-04-25T10:12:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e11600640 1 -- 172.21.15.87:0/3679960226 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8df8003530 con 0x7f8e14165ed0 2024-04-25T10:12:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13400640 1 -- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 msgr2=0x7f8e140b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13400640 1 --2- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e140b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13400640 1 -- 172.21.15.87:0/3679960226 >> v1:172.21.15.169:6789/0 conn(0x7f8e14165ed0 legacy=0x7f8e140ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.336+0000 7f8e13400640 1 -- 172.21.15.87:0/3679960226 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14166770 con 0x7f8e140d03d0 2024-04-25T10:12:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1200640 1 --2- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec0703c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007830 tx=0x7f01d801d790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313a848b2cbe26f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01eaa00640 1 -- 172.21.15.87:0/1969862166 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80a6590 con 0x7f01ec06caa0 2024-04-25T10:12:30.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01eaa00640 1 -- 172.21.15.87:0/1969862166 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d80a6730 con 0x7f01ec06caa0 2024-04-25T10:12:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01eaa00640 1 -- 172.21.15.87:0/1969862166 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80a6b00 con 0x7f01ec06caa0 2024-04-25T10:12:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 -- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 msgr2=0x7f01ec0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 --2- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec0703c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007830 tx=0x7f01d801d790 comp rx=0 tx=0).stop 2024-04-25T10:12:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 -- 172.21.15.87:0/1969862166 shutdown_connections 2024-04-25T10:12:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7fc7ad400640 1 --2- 172.21.15.87:0/700132966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a00556f0 0x7fc7a00dfea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009870 tx=0x7fc79c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb8f7757418d6570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 --2- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec0703c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 --2- 172.21.15.87:0/1969862166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 -- 172.21.15.87:0/1969862166 >> 172.21.15.87:0/1969862166 conn(0x7f01ec066bd0 msgr2=0x7f01ec066fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 -- 172.21.15.87:0/1969862166 shutdown_connections 2024-04-25T10:12:30.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7fc7a7e00640 1 -- 172.21.15.87:0/700132966 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0b0070 con 0x7fc7a00556f0 2024-04-25T10:12:30.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7f01f1c00640 1 -- 172.21.15.87:0/1969862166 wait complete. 2024-04-25T10:12:30.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7fc7a7e00640 1 -- 172.21.15.87:0/700132966 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c0ab600 con 0x7fc7a00556f0 2024-04-25T10:12:30.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.340+0000 7fc7a7e00640 1 -- 172.21.15.87:0/700132966 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0ab9d0 con 0x7fc7a00556f0 2024-04-25T10:12:30.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7fc7ae800640 1 -- 172.21.15.87:0/700132966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a00556f0 msgr2=0x7fc7a00dfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7fc7ae800640 1 --2- 172.21.15.87:0/700132966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a00556f0 0x7fc7a00dfea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009870 tx=0x7fc79c01c540 comp rx=0 tx=0).stop 2024-04-25T10:12:30.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7fc7ae800640 1 -- 172.21.15.87:0/700132966 shutdown_connections 2024-04-25T10:12:30.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7fc7ae800640 1 --2- 172.21.15.87:0/700132966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a00556f0 0x7fc7a00dfea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7fc7ae800640 1 -- 172.21.15.87:0/700132966 >> 172.21.15.87:0/700132966 conn(0x7fc7a0052660 msgr2=0x7fc7a0052a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e13400640 1 --2- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140d03d0 0x7f8e140d0820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009d90 tx=0x7f8e0c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cee12a8905ffa4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7fc7ae800640 1 -- 172.21.15.87:0/700132966 shutdown_connections 2024-04-25T10:12:30.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7fc7ae800640 1 -- 172.21.15.87:0/700132966 wait complete. 2024-04-25T10:12:30.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e11600640 1 -- 172.21.15.87:0/3679960226 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0b2070 con 0x7f8e140d03d0 2024-04-25T10:12:30.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e11600640 1 -- 172.21.15.87:0/3679960226 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c0a5830 con 0x7f8e140d03d0 2024-04-25T10:12:30.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e11600640 1 -- 172.21.15.87:0/3679960226 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0a5c00 con 0x7f8e140d03d0 2024-04-25T10:12:30.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f01f1c00640 1 Processor -- start 2024-04-25T10:12:30.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f01f1c00640 1 -- start start 2024-04-25T10:12:30.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e13e00640 1 -- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140d03d0 msgr2=0x7f8e140d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e13e00640 1 --2- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140d03d0 0x7f8e140d0820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009d90 tx=0x7f8e0c01c970 comp rx=0 tx=0).stop 2024-04-25T10:12:30.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e13e00640 1 -- 172.21.15.87:0/3679960226 shutdown_connections 2024-04-25T10:12:30.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e13e00640 1 --2- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140d03d0 0x7f8e140d0820 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e13e00640 1 --2- 172.21.15.87:0/3679960226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e140b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.344+0000 7f8e13e00640 1 -- 172.21.15.87:0/3679960226 >> 172.21.15.87:0/3679960226 conn(0x7f8e140669a0 msgr2=0x7f8e14066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f5746400640 1 Processor -- start 2024-04-25T10:12:30.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 Processor -- start 2024-04-25T10:12:30.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 -- start start 2024-04-25T10:12:30.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f8e13e00640 1 -- 172.21.15.87:0/3679960226 shutdown_connections 2024-04-25T10:12:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01f1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec162da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01f1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01f1c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec159c10 0x7f01ec145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f8e13e00640 1 -- 172.21.15.87:0/3679960226 wait complete. 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01ebe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38082/0 (socket says 172.21.15.87:38082) 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01ebe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec162da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43086/0 (socket says 172.21.15.87:43086) 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 -- 172.21.15.87:0/2313732359 learned_addr learned my addr 172.21.15.87:0/2313732359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b2dc0 con 0x7f01ec159c10 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b2c40 con 0x7f01ec06caa0 2024-04-25T10:12:30.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec071110 con 0x7f01ec159830 2024-04-25T10:12:30.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a0050fc0 0x7fc7a00cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a0053c90 0x7fc7a00e0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00556f0 0x7fc7a00e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00eedc0 con 0x7fc7a0050fc0 2024-04-25T10:12:30.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a003b8d0 con 0x7fc7a00556f0 2024-04-25T10:12:30.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ae800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00eef40 con 0x7fc7a0053c90 2024-04-25T10:12:30.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f5746400640 1 -- start start 2024-04-25T10:12:30.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 -- 172.21.15.87:0/2313732359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 msgr2=0x7f01ec162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec162da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 -- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec159c10 msgr2=0x7f01ec145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec159c10 0x7f01ec145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec151e20 con 0x7f01ec159830 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01eb400640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec153390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c7d0 tx=0x7f01e400cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01e3e00640 1 -- 172.21.15.87:0/2313732359 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4016020 con 0x7f01ec159830 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01e3e00640 1 -- 172.21.15.87:0/2313732359 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e400aa50 con 0x7f01ec159830 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f01e3e00640 1 -- 172.21.15.87:0/2313732359 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e400ae40 con 0x7f01ec159830 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec152100 con 0x7f01ec159830 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec1526c0 con 0x7f01ec159830 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a0050fc0 0x7fc7a00cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a0050fc0 0x7fc7a00cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57120/0 (socket says 172.21.15.87:57120) 2024-04-25T10:12:30.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 -- 172.21.15.87:0/1413617365 learned_addr learned my addr 172.21.15.87:0/1413617365 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 -- 172.21.15.87:0/1413617365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a0053c90 msgr2=0x7fc7a00e0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a0053c90 0x7fc7a00e0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 -- 172.21.15.87:0/1413617365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00556f0 msgr2=0x7fc7a00e22b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00556f0 0x7fc7a00e22b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7fc7ad400640 1 -- 172.21.15.87:0/1413617365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a00e0e30 con 0x7fc7a0050fc0 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f5746400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5740070310 0x7f5740071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f5746400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5740072110 0x7f57400cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f5746400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574015a690 con 0x7f57400674c0 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f5746400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574015a970 con 0x7f5740072110 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f5746400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574015ac50 con 0x7f5740070310 2024-04-25T10:12:30.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f573f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f57400674c0 0x7f5740150950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60596/0 (socket says 172.21.15.87:60596) 2024-04-25T10:12:30.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f573f400640 1 -- 172.21.15.87:0/694341014 learned_addr learned my addr 172.21.15.87:0/694341014 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f8e13e00640 1 Processor -- start 2024-04-25T10:12:30.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.348+0000 7f8e13e00640 1 -- start start 2024-04-25T10:12:30.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7fc7ad400640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a0050fc0 0x7fc7a00cf190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ae040 tx=0x7fc79c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7fc7a5600640 1 -- 172.21.15.87:0/1413617365 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0ba070 con 0x7fc7a0050fc0 2024-04-25T10:12:30.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7fc7a5600640 1 -- 172.21.15.87:0/1413617365 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c0ab9d0 con 0x7fc7a0050fc0 2024-04-25T10:12:30.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7fc7a5600640 1 -- 172.21.15.87:0/1413617365 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0b0040 con 0x7fc7a0050fc0 2024-04-25T10:12:30.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7fc7ae800640 1 -- 172.21.15.87:0/1413617365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a00e1110 con 0x7fc7a0050fc0 2024-04-25T10:12:30.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7fc7ae800640 1 -- 172.21.15.87:0/1413617365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a00e0990 con 0x7fc7a0050fc0 2024-04-25T10:12:30.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f5737e00640 1 --2- 172.21.15.87:0/694341014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5740070310 0x7f5740071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f573fe00640 1 --2- 172.21.15.87:0/694341014 >> v2:172.21.15.87:3300/0 conn(0x7f5740072110 0x7f57400cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec152af0 con 0x7f01ec159830 2024-04-25T10:12:30.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f5737e00640 1 -- 172.21.15.87:0/694341014 >> v2:172.21.15.87:3300/0 conn(0x7f5740072110 msgr2=0x7f57400cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f5737e00640 1 --2- 172.21.15.87:0/694341014 >> v2:172.21.15.87:3300/0 conn(0x7f5740072110 0x7f57400cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f573ea00640 1 -- 172.21.15.87:0/694341014 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943166446 0 0) 0x7f574015a690 con 0x7f57400674c0 2024-04-25T10:12:30.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a00053a0 con 0x7fc7a0050fc0 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f5737e00640 1 -- 172.21.15.87:0/694341014 >> v1:172.21.15.87:6789/0 conn(0x7f57400674c0 legacy=0x7f5740150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e140d03d0 0x7f8e1415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e14157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e14165ed0 0x7f8e14153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f8e13e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140d2a60 con 0x7f8e140d03d0 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f8e13e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140d28e0 con 0x7f8e14165ed0 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f8e13e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140d2be0 con 0x7f8e14162290 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f8e12a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e140d03d0 0x7f8e1415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e140d03d0 0x7f8e1415aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57152/0 (socket says 172.21.15.87:57152) 2024-04-25T10:12:30.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 -- 172.21.15.87:0/2452323552 learned_addr learned my addr 172.21.15.87:0/2452323552 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12000640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e14157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e13400640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e14165ed0 0x7f8e14153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.352+0000 7f5737e00640 1 -- 172.21.15.87:0/694341014 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574015af30 con 0x7f5740070310 2024-04-25T10:12:30.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 -- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 msgr2=0x7f8e14157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e14157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 -- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e14165ed0 msgr2=0x7f8e14153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e14165ed0 0x7f8e14153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 -- 172.21.15.87:0/2452323552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14156350 con 0x7f8e140d03d0 2024-04-25T10:12:30.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12000640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e14157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f573fe00640 1 --2- 172.21.15.87:0/694341014 >> v2:172.21.15.87:3300/0 conn(0x7f5740072110 0x7f57400cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e12a00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e140d03d0 0x7f8e1415aa50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00e930 tx=0x7f8dfc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e0b400640 1 -- 172.21.15.87:0/2452323552 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc017070 con 0x7f8e140d03d0 2024-04-25T10:12:30.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e0b400640 1 -- 172.21.15.87:0/2452323552 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc00ca00 con 0x7f8e140d03d0 2024-04-25T10:12:30.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e0b400640 1 -- 172.21.15.87:0/2452323552 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00cdb0 con 0x7f8e140d03d0 2024-04-25T10:12:30.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e141565f0 con 0x7f8e140d03d0 2024-04-25T10:12:30.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e14156bb0 con 0x7f8e140d03d0 2024-04-25T10:12:30.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f01e3e00640 1 -- 172.21.15.87:0/2313732359 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01e4019020 con 0x7f01ec159830 2024-04-25T10:12:30.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7f01e3e00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01cc067030 0x7f01cc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.356+0000 7fc7a5600640 1 -- 172.21.15.87:0/1413617365 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc79c0bd070 con 0x7fc7a0050fc0 2024-04-25T10:12:30.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7fc7a5600640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc788066fb0 0x7fc788069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7fc7a5600640 1 -- 172.21.15.87:0/1413617365 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc79c11e2c0 con 0x7fc7a0050fc0 2024-04-25T10:12:30.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f5737e00640 1 --2- 172.21.15.87:0/694341014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5740070310 0x7f5740071bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57280097f0 tx=0x7f572801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17f292100ce09ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f01ebe00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01cc067030 0x7f01cc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f01e3e00640 1 -- 172.21.15.87:0/2313732359 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01e4084fe0 con 0x7f01ec159830 2024-04-25T10:12:30.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f01ebe00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01cc067030 0x7f01cc0694f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f01ec163750 tx=0x7f01dc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f573ea00640 1 -- 172.21.15.87:0/694341014 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280b5070 con 0x7f5740070310 2024-04-25T10:12:30.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f573ea00640 1 -- 172.21.15.87:0/694341014 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57280a8990 con 0x7f5740070310 2024-04-25T10:12:30.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f573ea00640 1 -- 172.21.15.87:0/694341014 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280a8d60 con 0x7f5740070310 2024-04-25T10:12:30.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7fc7aca00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc788066fb0 0x7fc788069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.360+0000 7f8e0b400640 1 -- 172.21.15.87:0/2452323552 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dfc01a070 con 0x7f8e140d03d0 2024-04-25T10:12:30.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7fc7aca00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc788066fb0 0x7fc788069470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7a01856f0 tx=0x7fc7940079e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 -- 172.21.15.87:0/694341014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5740070310 msgr2=0x7f5740071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec0052a0 con 0x7f8e140d03d0 2024-04-25T10:12:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 --2- 172.21.15.87:0/694341014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5740070310 0x7f5740071bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57280097f0 tx=0x7f572801ecb0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 -- 172.21.15.87:0/694341014 shutdown_connections 2024-04-25T10:12:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 --2- 172.21.15.87:0/694341014 >> v2:172.21.15.87:3300/0 conn(0x7f5740072110 0x7f57400cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 --2- 172.21.15.87:0/694341014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5740070310 0x7f5740071bd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 -- 172.21.15.87:0/694341014 >> 172.21.15.87:0/694341014 conn(0x7f57400669a0 msgr2=0x7f5740066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f8e0b400640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8df0066f60 0x7f8df0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f8e0b400640 1 -- 172.21.15.87:0/2452323552 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dfc084fc0 con 0x7f8e140d03d0 2024-04-25T10:12:30.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f01e3e00640 1 -- 172.21.15.87:0/2313732359 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01e4020030 con 0x7f01ec159830 2024-04-25T10:12:30.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 -- 172.21.15.87:0/694341014 shutdown_connections 2024-04-25T10:12:30.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f8e12000640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8df0066f60 0x7f8df0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.364+0000 7f5746400640 1 -- 172.21.15.87:0/694341014 wait complete. 2024-04-25T10:12:30.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f8e12000640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8df0066f60 0x7f8df0069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8e1416d1d0 tx=0x7f8e04007d40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 Processor -- start 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 -- start start 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57400674c0 0x7f5740156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5740070310 0x7f5740154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5740072110 0x7f5740153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400d0190 con 0x7f5740072110 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400d0010 con 0x7f5740070310 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f5746400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57400d0310 con 0x7f57400674c0 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.368+0000 7f573f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57400674c0 0x7f5740156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5740070310 0x7f5740154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5740070310 0x7f5740154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43124/0 (socket says 172.21.15.87:43124) 2024-04-25T10:12:30.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 -- 172.21.15.87:0/135429222 learned_addr learned my addr 172.21.15.87:0/135429222 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7fc7a5600640 1 -- 172.21.15.87:0/1413617365 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc79c016130 con 0x7fc7a0050fc0 2024-04-25T10:12:30.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f573fe00640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5740072110 0x7f5740153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 -- 172.21.15.87:0/135429222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57400674c0 msgr2=0x7f5740156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57400674c0 0x7f5740156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 -- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5740072110 msgr2=0x7f5740153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5740072110 0x7f5740153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 -- 172.21.15.87:0/135429222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740071240 con 0x7f5740070310 2024-04-25T10:12:30.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f573fe00640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5740072110 0x7f5740153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.372+0000 7f5737e00640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5740070310 0x7f5740154b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f572801b8c0 tx=0x7f572801ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.376+0000 7f5737400640 1 -- 172.21.15.87:0/135429222 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280b5040 con 0x7f5740070310 2024-04-25T10:12:30.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.376+0000 7f5737400640 1 -- 172.21.15.87:0/135429222 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57280bf070 con 0x7f5740070310 2024-04-25T10:12:30.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.376+0000 7f5737400640 1 -- 172.21.15.87:0/135429222 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280b1910 con 0x7f5740070310 2024-04-25T10:12:30.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.376+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5740133900 con 0x7f5740070310 2024-04-25T10:12:30.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.376+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5740156d50 con 0x7f5740070310 2024-04-25T10:12:30.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.380+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57180052a0 con 0x7f5740070310 2024-04-25T10:12:30.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.380+0000 7f5737400640 1 -- 172.21.15.87:0/135429222 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57280b1ab0 con 0x7f5740070310 2024-04-25T10:12:30.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.380+0000 7f8e0b400640 1 -- 172.21.15.87:0/2452323552 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dfc055a30 con 0x7f8e140d03d0 2024-04-25T10:12:30.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.380+0000 7f5737400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f571c067030 0x7f571c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.380+0000 7f5737400640 1 -- 172.21.15.87:0/135429222 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5728122010 con 0x7f5740070310 2024-04-25T10:12:30.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.384+0000 7f573f400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f571c067030 0x7f571c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.384+0000 7f573f400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f571c067030 0x7f571c0694f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f70 tx=0x7f5730007240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.384+0000 7f5737400640 1 -- 172.21.15.87:0/135429222 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5728018810 con 0x7f5740070310 2024-04-25T10:12:30.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9667c00640 1 Processor -- start 2024-04-25T10:12:30.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9667c00640 1 -- start start 2024-04-25T10:12:30.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9667c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f966006ca50 0x7f96600cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9667c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9660070310 0x7f96600707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9667c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96600b4470 con 0x7f96600cf930 2024-04-25T10:12:30.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9667c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96600cf4d0 con 0x7f9660070310 2024-04-25T10:12:30.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9667c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96600b4cd0 con 0x7f966006ca50 2024-04-25T10:12:30.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9665200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f96600cf930 0x7f96600cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39950/0 (socket says 172.21.15.87:39950) 2024-04-25T10:12:30.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9665200640 1 -- 172.21.15.87:0/2220487750 learned_addr learned my addr 172.21.15.87:0/2220487750 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f965fe00640 1 --2- 172.21.15.87:0/2220487750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f966006ca50 0x7f96600cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f9665c00640 1 --2- 172.21.15.87:0/2220487750 >> v2:172.21.15.169:3300/0 conn(0x7f9660070310 0x7f96600707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f965f400640 1 -- 172.21.15.87:0/2220487750 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400871311 0 0) 0x7f96600b4470 con 0x7f96600cf930 2024-04-25T10:12:30.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f965f400640 1 -- 172.21.15.87:0/2220487750 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9648003530 con 0x7f96600cf930 2024-04-25T10:12:30.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f965fe00640 1 -- 172.21.15.87:0/2220487750 >> v2:172.21.15.169:3300/0 conn(0x7f9660070310 msgr2=0x7f96600707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f965fe00640 1 --2- 172.21.15.87:0/2220487750 >> v2:172.21.15.169:3300/0 conn(0x7f9660070310 0x7f96600707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f965fe00640 1 -- 172.21.15.87:0/2220487750 >> v1:172.21.15.169:6789/0 conn(0x7f96600cf930 legacy=0x7f96600cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.416+0000 7f965fe00640 1 -- 172.21.15.87:0/2220487750 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96600b4710 con 0x7f966006ca50 2024-04-25T10:12:30.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.420+0000 7f965fe00640 1 --2- 172.21.15.87:0/2220487750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f966006ca50 0x7f96600cef90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96540097f0 tx=0x7f965401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15938baa3d1ef692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f965f400640 1 -- 172.21.15.87:0/2220487750 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540b0070 con 0x7f966006ca50 2024-04-25T10:12:30.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f965f400640 1 -- 172.21.15.87:0/2220487750 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96540ab600 con 0x7f966006ca50 2024-04-25T10:12:30.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f965f400640 1 -- 172.21.15.87:0/2220487750 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540ab9d0 con 0x7f966006ca50 2024-04-25T10:12:30.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 -- 172.21.15.87:0/2220487750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f966006ca50 msgr2=0x7f96600cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 --2- 172.21.15.87:0/2220487750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f966006ca50 0x7f96600cef90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96540097f0 tx=0x7f965401c410 comp rx=0 tx=0).stop 2024-04-25T10:12:30.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 -- 172.21.15.87:0/2220487750 shutdown_connections 2024-04-25T10:12:30.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 --2- 172.21.15.87:0/2220487750 >> v2:172.21.15.169:3300/0 conn(0x7f9660070310 0x7f96600707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 --2- 172.21.15.87:0/2220487750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f966006ca50 0x7f96600cef90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 -- 172.21.15.87:0/2220487750 >> 172.21.15.87:0/2220487750 conn(0x7f9660066b70 msgr2=0x7f9660066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 -- 172.21.15.87:0/2220487750 shutdown_connections 2024-04-25T10:12:30.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.424+0000 7f9667c00640 1 -- 172.21.15.87:0/2220487750 wait complete. 2024-04-25T10:12:30.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 Processor -- start 2024-04-25T10:12:30.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 -- start start 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f966006ca50 0x7f9660160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9660070310 0x7f96601429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96600cf930 0x7f9660152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9665200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f966006ca50 0x7f9660160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96600b2e30 con 0x7f9660070310 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660072050 con 0x7f966006ca50 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9665200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f966006ca50 0x7f9660160190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43150/0 (socket says 172.21.15.87:43150) 2024-04-25T10:12:30.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9665200640 1 -- 172.21.15.87:0/1171866365 learned_addr learned my addr 172.21.15.87:0/1171866365 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9667c00640 1 -- 172.21.15.87:0/1171866365 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96600b2fb0 con 0x7f96600cf930 2024-04-25T10:12:30.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9665c00640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96600cf930 0x7f9660152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f965fe00640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9660070310 0x7f96601429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7ffbb0000640 1 Processor -- start 2024-04-25T10:12:30.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7ffbb0000640 1 -- start start 2024-04-25T10:12:30.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f965fe00640 1 -- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96600cf930 msgr2=0x7f9660152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f965fe00640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96600cf930 0x7f9660152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f965fe00640 1 -- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f966006ca50 msgr2=0x7f9660160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f965fe00640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f966006ca50 0x7f9660160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f965fe00640 1 -- 172.21.15.87:0/1171866365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660160820 con 0x7f9660070310 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.428+0000 7f9665c00640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96600cf930 0x7f9660152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f965fe00640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9660070310 0x7f96601429d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96540ab000 tx=0x7f96540191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f965cc00640 1 -- 172.21.15.87:0/1171866365 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540b8020 con 0x7f9660070310 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f9667c00640 1 -- 172.21.15.87:0/1171866365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9660160b00 con 0x7f9660070310 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f965cc00640 1 -- 172.21.15.87:0/1171866365 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96540b4600 con 0x7f9660070310 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f965cc00640 1 -- 172.21.15.87:0/1171866365 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540b0040 con 0x7f9660070310 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f9667c00640 1 -- 172.21.15.87:0/1171866365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96600af6b0 con 0x7f9660070310 2024-04-25T10:12:30.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b73000640 1 Processor -- start 2024-04-25T10:12:30.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b73000640 1 -- start start 2024-04-25T10:12:30.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbb0000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffba80674c0 0x7ffba8150950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbb0000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffba80722e0 0x7ffba80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbacc00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ffba8150e90 0x7ffba8071bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39960/0 (socket says 172.21.15.87:39960) 2024-04-25T10:12:30.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7fafd7a00640 1 Processor -- start 2024-04-25T10:12:30.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbad600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffba80674c0 0x7ffba8150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbad600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffba80674c0 0x7ffba8150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43160/0 (socket says 172.21.15.87:43160) 2024-04-25T10:12:30.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbb0000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba815a690 con 0x7ffba8150e90 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbacc00640 1 -- 172.21.15.87:0/1709203906 learned_addr learned my addr 172.21.15.87:0/1709203906 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbb0000640 1 -- 172.21.15.87:0/1709203906 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba815a970 con 0x7ffba80722e0 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b73000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1b6c162f40 0x7f1b6c06c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b73000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1b6c0b3050 0x7f1b6c070110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b73000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c0b3490 con 0x7f1b6c162450 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b73000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0721b0 con 0x7f1b6c0b3050 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b73000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0b2570 con 0x7f1b6c162f40 2024-04-25T10:12:30.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7f1b71200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1b6c162f40 0x7f1b6c06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696686011 0 0) 0x7ffba815a690 con 0x7ffba8150e90 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7ffbb0000640 1 -- 172.21.15.87:0/1709203906 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba815ac50 con 0x7ffba80674c0 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb90003530 con 0x7ffba8150e90 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f1b71200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1b6c162f40 0x7f1b6c06c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43176/0 (socket says 172.21.15.87:43176) 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f1b71200640 1 -- 172.21.15.87:0/3322126631 learned_addr learned my addr 172.21.15.87:0/3322126631 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.432+0000 7fafd7a00640 1 -- start start 2024-04-25T10:12:30.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffbae000640 1 --2- 172.21.15.87:0/1709203906 >> v2:172.21.15.87:3300/0 conn(0x7ffba80722e0 0x7ffba80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3379753143 0 0) 0x7ffb90003530 con 0x7ffba8150e90 2024-04-25T10:12:30.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 >> v2:172.21.15.169:3300/0 conn(0x7ffba80674c0 msgr2=0x7ffba8150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 --2- 172.21.15.87:0/1709203906 >> v2:172.21.15.169:3300/0 conn(0x7ffba80674c0 0x7ffba8150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd7a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fafd00b6cd0 0x7fafd00b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd7a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd00d2770 con 0x7fafd015ddf0 2024-04-25T10:12:30.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd7a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd00d1490 con 0x7fafd00b6cd0 2024-04-25T10:12:30.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd7a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00d2a50 con 0x7fafd00d0b10 2024-04-25T10:12:30.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 >> v2:172.21.15.87:3300/0 conn(0x7ffba80722e0 msgr2=0x7ffba80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 --2- 172.21.15.87:0/1709203906 >> v2:172.21.15.87:3300/0 conn(0x7ffba80722e0 0x7ffba80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba815af30 con 0x7ffba8150e90 2024-04-25T10:12:30.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ffb9c001680 con 0x7ffba8150e90 2024-04-25T10:12:30.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffbae000640 1 --2- 172.21.15.87:0/1709203906 >> v2:172.21.15.87:3300/0 conn(0x7ffba80722e0 0x7ffba80cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:30.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffbad600640 1 --2- 172.21.15.87:0/1709203906 >> v2:172.21.15.169:3300/0 conn(0x7ffba80674c0 0x7ffba8150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:30.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 >> v1:172.21.15.169:6789/0 conn(0x7ffba8150e90 legacy=0x7ffba8071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd5000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fafd015ddf0 0x7fafd00d1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39976/0 (socket says 172.21.15.87:39976) 2024-04-25T10:12:30.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd5000640 1 -- 172.21.15.87:0/2451902463 learned_addr learned my addr 172.21.15.87:0/2451902463 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:30.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafcfe00640 1 --2- 172.21.15.87:0/2451902463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb900039c0 0x7ffb90003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90004810 0x7ffb90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb900051d0 0x7ffb9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba815a970 con 0x7ffb900051d0 2024-04-25T10:12:30.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba815ac50 con 0x7ffb90004810 2024-04-25T10:12:30.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9000bfc0 con 0x7ffb900039c0 2024-04-25T10:12:30.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f965cc00640 1 -- 172.21.15.87:0/1171866365 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96540bb020 con 0x7f9660070310 2024-04-25T10:12:30.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffbacc00640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90004810 0x7ffb90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffbad600640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb900039c0 0x7ffb90003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966004ef30 con 0x7f9660070310 2024-04-25T10:12:30.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f1b71200640 1 -- 172.21.15.87:0/3322126631 >> v2:172.21.15.87:3300/0 conn(0x7f1b6c0b3050 msgr2=0x7f1b6c070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f1b71200640 1 --2- 172.21.15.87:0/3322126631 >> v2:172.21.15.87:3300/0 conn(0x7f1b6c0b3050 0x7f1b6c070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f1b71200640 1 -- 172.21.15.87:0/3322126631 >> v1:172.21.15.87:6789/0 conn(0x7f1b6c162450 legacy=0x7f1b6c162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7f1b71200640 1 -- 172.21.15.87:0/3322126631 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c0b2850 con 0x7f1b6c162f40 2024-04-25T10:12:30.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafd5a00640 1 --2- 172.21.15.87:0/2451902463 >> v2:172.21.15.87:3300/0 conn(0x7fafd00b6cd0 0x7fafd00b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7ffbae000640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb900051d0 0x7ffb9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.436+0000 7fafcf400640 1 -- 172.21.15.87:0/2451902463 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772420987 0 0) 0x7fafd00d2770 con 0x7fafd015ddf0 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7f965cc00640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9640066f60 0x7f9640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7f965cc00640 1 -- 172.21.15.87:0/1171866365 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f965411ec80 con 0x7f9660070310 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7ffbacc00640 1 -- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb900039c0 msgr2=0x7ffb90003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7ffbacc00640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb900039c0 0x7ffb90003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7ffbacc00640 1 -- 172.21.15.87:0/1709203906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb900051d0 msgr2=0x7ffb9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7ffbacc00640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb900051d0 0x7ffb9000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7ffbacc00640 1 -- 172.21.15.87:0/1709203906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb90003530 con 0x7ffb90004810 2024-04-25T10:12:30.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7ffbae000640 1 -- 172.21.15.87:0/1709203906 reap_dead start 2024-04-25T10:12:30.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7fafcf400640 1 -- 172.21.15.87:0/2451902463 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb4003530 con 0x7fafd015ddf0 2024-04-25T10:12:30.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7fafcfe00640 1 -- 172.21.15.87:0/2451902463 >> v2:172.21.15.87:3300/0 conn(0x7fafd00b6cd0 msgr2=0x7fafd00b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7fafcfe00640 1 --2- 172.21.15.87:0/2451902463 >> v2:172.21.15.87:3300/0 conn(0x7fafd00b6cd0 0x7fafd00b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7fafcfe00640 1 -- 172.21.15.87:0/2451902463 >> v1:172.21.15.169:6789/0 conn(0x7fafd015ddf0 legacy=0x7fafd00d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7fafcfe00640 1 -- 172.21.15.87:0/2451902463 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd00d2d30 con 0x7fafd00d0b10 2024-04-25T10:12:30.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7f1b71200640 1 --2- 172.21.15.87:0/3322126631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c162f40 0x7f1b6c06c140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b3b40 tx=0x7f1b5801c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573a89782b45c668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7f1b6be00640 1 -- 172.21.15.87:0/3322126631 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b0070 con 0x7f1b6c162f40 2024-04-25T10:12:30.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7f1b6be00640 1 -- 172.21.15.87:0/3322126631 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b580ab600 con 0x7f1b6c162f40 2024-04-25T10:12:30.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7f1b6be00640 1 -- 172.21.15.87:0/3322126631 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580ab9d0 con 0x7f1b6c162f40 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 -- 172.21.15.87:0/3322126631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c162f40 msgr2=0x7f1b6c06c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 --2- 172.21.15.87:0/3322126631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c162f40 0x7f1b6c06c140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b3b40 tx=0x7f1b5801c320 comp rx=0 tx=0).stop 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafcfe00640 1 --2- 172.21.15.87:0/2451902463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd00d0f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009d90 tx=0x7fafc001f140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d846a3e8aa5b59d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 -- 172.21.15.87:0/3322126631 shutdown_connections 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 --2- 172.21.15.87:0/3322126631 >> v2:172.21.15.87:3300/0 conn(0x7f1b6c0b3050 0x7f1b6c070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 --2- 172.21.15.87:0/3322126631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c162f40 0x7f1b6c06c140 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 -- 172.21.15.87:0/3322126631 >> 172.21.15.87:0/3322126631 conn(0x7f1b6c0669a0 msgr2=0x7f1b6c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafcf400640 1 -- 172.21.15.87:0/2451902463 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b5070 con 0x7fafd00d0b10 2024-04-25T10:12:30.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 -- 172.21.15.87:0/3322126631 shutdown_connections 2024-04-25T10:12:30.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f1b73000640 1 -- 172.21.15.87:0/3322126631 wait complete. 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafcf400640 1 -- 172.21.15.87:0/2451902463 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc00b0440 con 0x7fafd00d0b10 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafcf400640 1 -- 172.21.15.87:0/2451902463 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b0810 con 0x7fafd00d0b10 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 -- 172.21.15.87:0/2451902463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 msgr2=0x7fafd00d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2451902463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd00d0f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009d90 tx=0x7fafc001f140 comp rx=0 tx=0).stop 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 -- 172.21.15.87:0/2451902463 shutdown_connections 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2451902463 >> v2:172.21.15.87:3300/0 conn(0x7fafd00b6cd0 0x7fafd00b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2451902463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd00d0f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 -- 172.21.15.87:0/2451902463 >> 172.21.15.87:0/2451902463 conn(0x7fafd006b090 msgr2=0x7fafd006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.440+0000 7f9665200640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9640066f60 0x7f9640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7f9665200640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9640066f60 0x7f9640069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f964c007fd0 tx=0x7f964c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7ffbacc00640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90004810 0x7ffb90004c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c005070 tx=0x7ffb9c01d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=242817f077d7a568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 -- 172.21.15.87:0/2451902463 shutdown_connections 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 Processor -- start 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 -- start start 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.444+0000 7fafd7a00640 1 -- 172.21.15.87:0/2451902463 wait complete. 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7fafd7a00640 1 Processor -- start 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c004c70 con 0x7ffb90004810 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c005ae0 con 0x7ffb90004810 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7ffba7e00640 1 -- 172.21.15.87:0/1709203906 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c008110 con 0x7ffb90004810 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7fafd7a00640 1 -- start start 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c0b3050 0x7f1b6c151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b6c162450 0x7f1b6c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7ffbb0000640 1 -- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90004810 msgr2=0x7ffb90004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90004810 0x7ffb90004c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c005070 tx=0x7ffb9c01d260 comp rx=0 tx=0).stop 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b6c162f40 0x7f1b6c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b6c162450 0x7f1b6c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b71c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c0b3050 0x7f1b6c151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b6c162450 0x7f1b6c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38156/0 (socket says 172.21.15.87:38156) 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b71c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c0b3050 0x7f1b6c151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43194/0 (socket says 172.21.15.87:43194) 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c16ec00 con 0x7f1b6c162f40 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c16ea80 con 0x7f1b6c0b3050 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b73000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c16ed80 con 0x7f1b6c162450 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b71200640 1 -- 172.21.15.87:0/2887533754 learned_addr learned my addr 172.21.15.87:0/2887533754 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f965cc00640 1 -- 172.21.15.87:0/1171866365 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9654016090 con 0x7f9660070310 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.448+0000 7f1b72600640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b6c162f40 0x7f1b6c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b71200640 1 -- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c0b3050 msgr2=0x7f1b6c151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b71200640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c0b3050 0x7f1b6c151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b71200640 1 -- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b6c162f40 msgr2=0x7f1b6c153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b71200640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b6c162f40 0x7f1b6c153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b71200640 1 -- 172.21.15.87:0/2887533754 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c15a470 con 0x7f1b6c162450 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd7a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafd00b6cd0 0x7fafd00c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b71c00640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c0b3050 0x7f1b6c151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b71200640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b6c162450 0x7f1b6c1599b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b580ae040 tx=0x7f1b580190c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7ffbb0000640 1 -- 172.21.15.87:0/1709203906 shutdown_connections 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1709203906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb90004810 0x7ffb90004c90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7ffbb0000640 1 -- 172.21.15.87:0/1709203906 >> 172.21.15.87:0/1709203906 conn(0x7ffba80669a0 msgr2=0x7ffba8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd01f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd7a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd015ddf0 0x7fafd01f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7ffbb0000640 1 -- 172.21.15.87:0/1709203906 shutdown_connections 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafd00b6cd0 0x7fafd00c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd015ddf0 0x7fafd01f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd015ddf0 0x7fafd01f4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43196/0 (socket says 172.21.15.87:43196) 2024-04-25T10:12:30.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafd00b6cd0 0x7fafd00c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57228/0 (socket says 172.21.15.87:57228) 2024-04-25T10:12:30.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd01f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b69600640 1 -- 172.21.15.87:0/2887533754 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b0050 con 0x7f1b6c162450 2024-04-25T10:12:30.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b69600640 1 -- 172.21.15.87:0/2887533754 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b580bc070 con 0x7f1b6c162450 2024-04-25T10:12:30.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b69600640 1 -- 172.21.15.87:0/2887533754 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b5a50 con 0x7f1b6c162450 2024-04-25T10:12:30.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c15a750 con 0x7f1b6c162450 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7ffbb0000640 1 -- 172.21.15.87:0/1709203906 wait complete. 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c15ad10 con 0x7f1b6c162450 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd01f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38168/0 (socket says 172.21.15.87:38168) 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00d3810 con 0x7fafd00b6cd0 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd5a00640 1 -- 172.21.15.87:0/2602386629 learned_addr learned my addr 172.21.15.87:0/2602386629 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00d32f0 con 0x7fafd015ddf0 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00b9a90 con 0x7fafd00d0b10 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 -- 172.21.15.87:0/2602386629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd015ddf0 msgr2=0x7fafd01f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd015ddf0 0x7fafd01f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 -- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafd00b6cd0 msgr2=0x7fafd00c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafd00b6cd0 0x7fafd00c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd00ccc90 con 0x7fafd00d0b10 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.452+0000 7fafcfe00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd01f3ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fafc00af000 tx=0x7fafc001bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7fafccc00640 1 -- 172.21.15.87:0/2602386629 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00c1020 con 0x7fafd00d0b10 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7fafccc00640 1 -- 172.21.15.87:0/2602386629 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc00bb690 con 0x7fafd00d0b10 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7fafccc00640 1 -- 172.21.15.87:0/2602386629 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b5040 con 0x7fafd00d0b10 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd00ccf70 con 0x7fafd00d0b10 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd00cd530 con 0x7fafd00d0b10 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7f1b69600640 1 -- 172.21.15.87:0/2887533754 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b580b5bf0 con 0x7f1b6c162450 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7f1b69600640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b4c066f60 0x7f1b4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7f1b69600640 1 -- 172.21.15.87:0/2887533754 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b5811f960 con 0x7f1b6c162450 2024-04-25T10:12:30.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7ffbb0000640 1 Processor -- start 2024-04-25T10:12:30.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7ffbb0000640 1 -- start start 2024-04-25T10:12:30.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.456+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd004ef30 con 0x7fafd00d0b10 2024-04-25T10:12:30.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c075120 con 0x7f1b6c162450 2024-04-25T10:12:30.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7fafccc00640 1 -- 172.21.15.87:0/2602386629 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafc00bba80 con 0x7fafd00d0b10 2024-04-25T10:12:30.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7f1b71c00640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b4c066f60 0x7f1b4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbb0000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb90004810 0x7ffba8146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbb0000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffba8153340 0x7ffba81537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7f1b71c00640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b4c066f60 0x7f1b4c069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b60001760 tx=0x7f1b60009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7fafccc00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafac066f60 0x7fafac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7fafccc00640 1 -- 172.21.15.87:0/2602386629 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafc0002020 con 0x7fafd00d0b10 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbb0000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba8157870 0x7ffba8157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbb0000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba8150950 con 0x7ffba8153340 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbb0000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba8070520 con 0x7ffba8157870 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbb0000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba8150ad0 con 0x7ffb90004810 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb90004810 0x7ffba8146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb90004810 0x7ffba8146c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38176/0 (socket says 172.21.15.87:38176) 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 -- 172.21.15.87:0/1146941097 learned_addr learned my addr 172.21.15.87:0/1146941097 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbae000640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba8157870 0x7ffba8157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbacc00640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffba8153340 0x7ffba81537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 -- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba8157870 msgr2=0x7ffba8157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba8157870 0x7ffba8157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 -- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffba8153340 msgr2=0x7ffba81537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffba8153340 0x7ffba81537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 -- 172.21.15.87:0/1146941097 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8155970 con 0x7ffb90004810 2024-04-25T10:12:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7fafd5000640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafac066f60 0x7fafac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.460+0000 7ffbad600640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb90004810 0x7ffba8146c90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffba000f410 tx=0x7ffba000f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.464+0000 7fafd5000640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafac066f60 0x7fafac069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafc8000c10 tx=0x7fafc8007dd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.464+0000 7f1b69600640 1 -- 172.21.15.87:0/2887533754 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b6c0b3490 con 0x7f1b6c162450 2024-04-25T10:12:30.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.472+0000 7fafccc00640 1 -- 172.21.15.87:0/2602386629 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafc0018ca0 con 0x7fafd00d0b10 2024-04-25T10:12:30.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.472+0000 7ffba5600640 1 -- 172.21.15.87:0/1146941097 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0003ba0 con 0x7ffb90004810 2024-04-25T10:12:30.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.472+0000 7ffba5600640 1 -- 172.21.15.87:0/1146941097 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba0003d40 con 0x7ffb90004810 2024-04-25T10:12:30.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.472+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba8155c50 con 0x7ffb90004810 2024-04-25T10:12:30.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.472+0000 7ffba5600640 1 -- 172.21.15.87:0/1146941097 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0004260 con 0x7ffb90004810 2024-04-25T10:12:30.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.476+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba8156210 con 0x7ffb90004810 2024-04-25T10:12:30.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.476+0000 7ffba5600640 1 -- 172.21.15.87:0/1146941097 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffba0004400 con 0x7ffb90004810 2024-04-25T10:12:30.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.476+0000 7ffba5600640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb88066fb0 0x7ffb88069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.480+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba804ef30 con 0x7ffb90004810 2024-04-25T10:12:30.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.480+0000 7ffbacc00640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb88066fb0 0x7ffb88069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.480+0000 7ffba5600640 1 -- 172.21.15.87:0/1146941097 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffba005ae00 con 0x7ffb90004810 2024-04-25T10:12:30.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.480+0000 7ffbacc00640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb88066fb0 0x7ffb88069470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c008fd0 tx=0x7ffb9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.484+0000 7ffba5600640 1 -- 172.21.15.87:0/1146941097 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffba00b5cb0 con 0x7ffb90004810 2024-04-25T10:12:30.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.700+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01ec075120 con 0x7f01ec159830 2024-04-25T10:12:30.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.700+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc7a00d4ec0 con 0x7fc7a0050fc0 2024-04-25T10:12:30.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.700+0000 7f01e3e00640 1 -- 172.21.15.87:0/2313732359 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01e4055fd0 con 0x7f01ec159830 2024-04-25T10:12:30.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.700+0000 7fc7a5600640 1 -- 172.21.15.87:0/1413617365 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc79c0ef3b0 con 0x7fc7a0050fc0 2024-04-25T10:12:30.707 INFO:teuthology.orchestra.run.smithi087.stdout:55834574853 2024-04-25T10:12:30.707 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:30.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5718005590 con 0x7f5740070310 2024-04-25T10:12:30.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7f5737400640 1 -- 172.21.15.87:0/135429222 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f57280f3090 con 0x7f5740070310 2024-04-25T10:12:30.710 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:30.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc788066fb0 msgr2=0x7fc788069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc788066fb0 0x7fc788069470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7a01856f0 tx=0x7fc7940079e0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a0050fc0 msgr2=0x7fc7a00cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a0050fc0 0x7fc7a00cf190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ae040 tx=0x7fc79c0192e0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 shutdown_connections 2024-04-25T10:12:30.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc788066fb0 0x7fc788069470 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7a01856f0 tx=0x7fc7940079e0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00556f0 0x7fc7a00e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a0053c90 0x7fc7a00e0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 --2- 172.21.15.87:0/1413617365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a0050fc0 0x7fc7a00cf190 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ae040 tx=0x7fc79c0192e0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.704+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 >> 172.21.15.87:0/1413617365 conn(0x7fc7a0052660 msgr2=0x7fc7a0055e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01cc067030 msgr2=0x7f01cc0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01cc067030 0x7f01cc0694f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f01ec163750 tx=0x7f01dc009190 comp rx=0 tx=0).stop 2024-04-25T10:12:30.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f571c067030 msgr2=0x7f571c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f571c067030 0x7f571c0694f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f70 tx=0x7f5730007240 comp rx=0 tx=0).stop 2024-04-25T10:12:30.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5740070310 msgr2=0x7f5740154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5740070310 0x7f5740154b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f572801b8c0 tx=0x7f572801ba50 comp rx=0 tx=0).stop 2024-04-25T10:12:30.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 shutdown_connections 2024-04-25T10:12:30.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 msgr2=0x7f01ec153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec153390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c7d0 tx=0x7f01e400cc90 comp rx=0 tx=0).stop 2024-04-25T10:12:30.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7fc79aa00640 1 -- 172.21.15.87:0/1413617365 wait complete. 2024-04-25T10:12:30.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 shutdown_connections 2024-04-25T10:12:30.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01cc067030 0x7f01cc0694f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec159c10 0x7f01ec145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec159830 0x7f01ec153390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 --2- 172.21.15.87:0/2313732359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06caa0 0x7f01ec162da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 >> 172.21.15.87:0/2313732359 conn(0x7f01ec066bd0 msgr2=0x7f01ec0cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 shutdown_connections 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f571c067030 0x7f571c0694f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5740072110 0x7f5740153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5740070310 0x7f5740154b30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 --2- 172.21.15.87:0/135429222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57400674c0 0x7f5740156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.708+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 >> 172.21.15.87:0/135429222 conn(0x7f57400669a0 msgr2=0x7f57400b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.712+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 shutdown_connections 2024-04-25T10:12:30.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.712+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 shutdown_connections 2024-04-25T10:12:30.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.712+0000 7f5746400640 1 -- 172.21.15.87:0/135429222 wait complete. 2024-04-25T10:12:30.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.712+0000 7f01f1c00640 1 -- 172.21.15.87:0/2313732359 wait complete. 2024-04-25T10:12:30.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.724+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8dec005590 con 0x7f8e140d03d0 2024-04-25T10:12:30.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.724+0000 7f8e0b400640 1 -- 172.21.15.87:0/2452323552 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8dfc059ad0 con 0x7f8e140d03d0 2024-04-25T10:12:30.729 INFO:teuthology.orchestra.run.smithi087.stdout:55834574853 2024-04-25T10:12:30.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8df0066f60 msgr2=0x7f8df0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8df0066f60 0x7f8df0069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8e1416d1d0 tx=0x7f8e04007d40 comp rx=0 tx=0).stop 2024-04-25T10:12:30.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e140d03d0 msgr2=0x7f8e1415aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e140d03d0 0x7f8e1415aa50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00e930 tx=0x7f8dfc00edf0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 shutdown_connections 2024-04-25T10:12:30.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8df0066f60 0x7f8df0069420 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8e1416d1d0 tx=0x7f8e04007d40 comp rx=0 tx=0).stop 2024-04-25T10:12:30.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e14165ed0 0x7f8e14153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e14162290 0x7f8e14157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 --2- 172.21.15.87:0/2452323552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e140d03d0 0x7f8e1415aa50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 >> 172.21.15.87:0/2452323552 conn(0x7f8e140669a0 msgr2=0x7f8e141647c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 shutdown_connections 2024-04-25T10:12:30.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.728+0000 7f8e13e00640 1 -- 172.21.15.87:0/2452323552 wait complete. 2024-04-25T10:12:30.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-25T10:12:30.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-25T10:12:30.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-04-25T10:12:30.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-25T10:12:30.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9660162040 con 0x7f9660070310 2024-04-25T10:12:30.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f965cc00640 1 -- 172.21.15.87:0/1171866365 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f96540ef5b0 con 0x7f9660070310 2024-04-25T10:12:30.773 INFO:teuthology.orchestra.run.smithi087.stdout:55834574853 2024-04-25T10:12:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9640066f60 msgr2=0x7f9640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9640066f60 0x7f9640069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f964c007fd0 tx=0x7f964c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9660070310 msgr2=0x7f96601429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9660070310 0x7f96601429d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96540ab000 tx=0x7f96540191b0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 shutdown_connections 2024-04-25T10:12:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9640066f60 0x7f9640069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96600cf930 0x7f9660152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9660070310 0x7f96601429d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 --2- 172.21.15.87:0/1171866365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f966006ca50 0x7f9660160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 >> 172.21.15.87:0/1171866365 conn(0x7f9660066b70 msgr2=0x7f966006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 shutdown_connections 2024-04-25T10:12:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.768+0000 7f9652000640 1 -- 172.21.15.87:0/1171866365 wait complete. 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fafd015d120 con 0x7fafd00d0b10 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafccc00640 1 -- 172.21.15.87:0/2602386629 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafc00f3bf0 con 0x7fafd00d0b10 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi087.stdout:55834574852 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafac066f60 msgr2=0x7fafac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafac066f60 0x7fafac069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafc8000c10 tx=0x7fafc8007dd0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 msgr2=0x7fafd01f3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd01f3ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fafc00af000 tx=0x7fafc001bee0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 shutdown_connections 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafac066f60 0x7fafac069420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd015ddf0 0x7fafd01f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd00d0b10 0x7fafd01f3ef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 --2- 172.21.15.87:0/2602386629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafd00b6cd0 0x7fafd00c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 >> 172.21.15.87:0/2602386629 conn(0x7fafd006b090 msgr2=0x7fafd00b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 shutdown_connections 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.776+0000 7fafd7a00640 1 -- 172.21.15.87:0/2602386629 wait complete. 2024-04-25T10:12:30.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-04-25T10:12:30.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-04-25T10:12:30.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.804+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1b6c0cf170 con 0x7f1b6c162450 2024-04-25T10:12:30.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.804+0000 7f1b69600640 1 -- 172.21.15.87:0/2887533754 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b580f0850 con 0x7f1b6c162450 2024-04-25T10:12:30.809 INFO:teuthology.orchestra.run.smithi087.stdout:55834574853 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b4c066f60 msgr2=0x7f1b4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b4c066f60 0x7f1b4c069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b60001760 tx=0x7f1b60009040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b6c162450 msgr2=0x7f1b6c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b6c162450 0x7f1b6c1599b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b580ae040 tx=0x7f1b580190c0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 shutdown_connections 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b4c066f60 0x7f1b4c069420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b6c162f40 0x7f1b6c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b6c162450 0x7f1b6c1599b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 --2- 172.21.15.87:0/2887533754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b6c0b3050 0x7f1b6c151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 >> 172.21.15.87:0/2887533754 conn(0x7f1b6c0669a0 msgr2=0x7f1b6c164400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 shutdown_connections 2024-04-25T10:12:30.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.808+0000 7f1b73000640 1 -- 172.21.15.87:0/2887533754 wait complete. 2024-04-25T10:12:30.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.812+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ffba80b3bf0 con 0x7ffb90004810 2024-04-25T10:12:30.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.812+0000 7ffba5600640 1 -- 172.21.15.87:0/1146941097 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffba0019030 con 0x7ffb90004810 2024-04-25T10:12:30.818 INFO:teuthology.orchestra.run.smithi087.stdout:55834574853 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb88066fb0 msgr2=0x7ffb88069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb88066fb0 0x7ffb88069470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c008fd0 tx=0x7ffb9c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb90004810 msgr2=0x7ffba8146c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb90004810 0x7ffba8146c90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffba000f410 tx=0x7ffba000f810 comp rx=0 tx=0).stop 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 shutdown_connections 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb88066fb0 0x7ffb88069470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba8157870 0x7ffba8157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffba8153340 0x7ffba81537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 --2- 172.21.15.87:0/1146941097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb90004810 0x7ffba8146c90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 >> 172.21.15.87:0/1146941097 conn(0x7ffba80669a0 msgr2=0x7ffba8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 shutdown_connections 2024-04-25T10:12:30.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:30.816+0000 7ffbb0000640 1 -- 172.21.15.87:0/1146941097 wait complete. 2024-04-25T10:12:30.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-04-25T10:12:30.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-25T10:12:31.754 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:12:31.755 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:12:31.756 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:12:31.762 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:12:31.799 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:12:31.808 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:12:31.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:12:31.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:12:31.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.936+0000 7f32c2c00640 1 Processor -- start 2024-04-25T10:12:31.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.936+0000 7f32c2c00640 1 -- start start 2024-04-25T10:12:31.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.936+0000 7f32c2c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32bc159e70 0x7f32bc0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.936+0000 7f32c2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc06c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32c2c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc0caa50 con 0x7f32bc070e00 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32c2c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc0cad30 con 0x7f32bc159e70 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32c2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0cb010 con 0x7f32bc0ccc20 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32bbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32bc159e70 0x7f32bc0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32bbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32bc159e70 0x7f32bc0ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57252/0 (socket says 172.21.15.87:57252) 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32bbe00640 1 -- 172.21.15.87:0/152043326 learned_addr learned my addr 172.21.15.87:0/152043326 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32bbe00640 1 -- 172.21.15.87:0/152043326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 msgr2=0x7f32bc06c180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32bbe00640 1 --2- 172.21.15.87:0/152043326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc06c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32bbe00640 1 -- 172.21.15.87:0/152043326 >> v1:172.21.15.169:6789/0 conn(0x7f32bc070e00 legacy=0x7f32bc0cc510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:12:31.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.940+0000 7f32bbe00640 1 -- 172.21.15.87:0/152043326 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0cb2f0 con 0x7f32bc159e70 2024-04-25T10:12:31.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.944+0000 7f32bbe00640 1 --2- 172.21.15.87:0/152043326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc159e70 0x7f32bc0ca150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0cd0f0 tx=0x7f32a801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6051a769a0ada0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.944+0000 7f32baa00640 1 -- 172.21.15.87:0/152043326 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80aa430 con 0x7f32bc159e70 2024-04-25T10:12:31.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.944+0000 7f32baa00640 1 -- 172.21.15.87:0/152043326 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a80af070 con 0x7f32bc159e70 2024-04-25T10:12:31.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.944+0000 7f32baa00640 1 -- 172.21.15.87:0/152043326 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80aa7f0 con 0x7f32bc159e70 2024-04-25T10:12:31.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 -- 172.21.15.87:0/152043326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc159e70 msgr2=0x7f32bc0ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 --2- 172.21.15.87:0/152043326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc159e70 0x7f32bc0ca150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0cd0f0 tx=0x7f32a801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 -- 172.21.15.87:0/152043326 shutdown_connections 2024-04-25T10:12:31.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 --2- 172.21.15.87:0/152043326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc06c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 --2- 172.21.15.87:0/152043326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc159e70 0x7f32bc0ca150 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 -- 172.21.15.87:0/152043326 >> 172.21.15.87:0/152043326 conn(0x7f32bc0b1330 msgr2=0x7f32bc0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 -- 172.21.15.87:0/152043326 shutdown_connections 2024-04-25T10:12:31.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.948+0000 7f32c2c00640 1 -- 172.21.15.87:0/152043326 wait complete. 2024-04-25T10:12:31.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.952+0000 7f32c2c00640 1 Processor -- start 2024-04-25T10:12:31.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.952+0000 7f32c2c00640 1 -- start start 2024-04-25T10:12:31.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.952+0000 7f32c2c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc070e00 0x7f32bc1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.952+0000 7f32bbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc070e00 0x7f32bc1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.952+0000 7f32bbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc070e00 0x7f32bc1513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57254/0 (socket says 172.21.15.87:57254) 2024-04-25T10:12:31.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bbe00640 1 -- 172.21.15.87:0/1709831939 learned_addr learned my addr 172.21.15.87:0/1709831939 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:31.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc159e70 0x7f32bc0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0cbc70 con 0x7f32bc070e00 2024-04-25T10:12:31.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0cbaf0 con 0x7f32bc159e70 2024-04-25T10:12:31.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0cc010 con 0x7f32bc0ccc20 2024-04-25T10:12:31.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bb400640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c0c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc159e70 0x7f32bc0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bb400640 1 -- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc159e70 msgr2=0x7f32bc0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bb400640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc159e70 0x7f32bc0c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bb400640 1 -- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc070e00 msgr2=0x7f32bc1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bb400640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc070e00 0x7f32bc1513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bb400640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0c4c20 con 0x7f32bc0ccc20 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c0c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc159e70 0x7f32bc0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bbe00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc070e00 0x7f32bc1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32bb400640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc0c8f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b440 tx=0x7f32ac00b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32b3e00640 1 -- 172.21.15.87:0/1709831939 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac015020 con 0x7f32bc0ccc20 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32b3e00640 1 -- 172.21.15.87:0/1709831939 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac011850 con 0x7f32bc0ccc20 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32b3e00640 1 -- 172.21.15.87:0/1709831939 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac011c40 con 0x7f32bc0ccc20 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc0c4f00 con 0x7f32bc0ccc20 2024-04-25T10:12:31.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc0c6b10 con 0x7f32bc0ccc20 2024-04-25T10:12:31.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.956+0000 7f5449c00640 1 Processor -- start 2024-04-25T10:12:31.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449c00640 1 -- start start 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5444165500 0x7f54440cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 0x7f54440d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54440b7f50 con 0x7f54440d0ec0 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440b8ff0 con 0x7f5444165500 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f32b3e00640 1 -- 172.21.15.87:0/1709831939 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32ac018020 con 0x7f32bc0ccc20 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54440d12f0 con 0x7f54440b2760 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f54440d0ec0 0x7f54440d1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60616/0 (socket says 172.21.15.87:60616) 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f32b3e00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f329c067030 0x7f329c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.960+0000 7f5449200640 1 -- 172.21.15.87:0/3780471446 learned_addr learned my addr 172.21.15.87:0/3780471446 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f5443e00640 1 --2- 172.21.15.87:0/3780471446 >> v2:172.21.15.169:3300/0 conn(0x7f5444165500 0x7f54440cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f32b3e00640 1 -- 172.21.15.87:0/1709831939 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32ac084e20 con 0x7f32bc0ccc20 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32940052a0 con 0x7f32bc0ccc20 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f5443400640 1 --2- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 0x7f54440d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f5443e00640 1 -- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 msgr2=0x7f54440d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f5443e00640 1 --2- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 0x7f54440d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f5443e00640 1 -- 172.21.15.87:0/3780471446 >> v1:172.21.15.87:6789/0 conn(0x7f54440d0ec0 legacy=0x7f54440d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f5443e00640 1 -- 172.21.15.87:0/3780471446 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440b8230 con 0x7f5444165500 2024-04-25T10:12:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f32bbe00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f329c067030 0x7f329c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.964+0000 7f32bbe00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f329c067030 0x7f329c0694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32a801c530 tx=0x7f32a80a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.968+0000 7f5443e00640 1 --2- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5444165500 0x7f54440cec60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54380097c0 tx=0x7f543801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92618fd83cd1bfc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.968+0000 7f5442a00640 1 -- 172.21.15.87:0/3780471446 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380b5070 con 0x7f5444165500 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.968+0000 7f5442a00640 1 -- 172.21.15.87:0/3780471446 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54380a8990 con 0x7f5444165500 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.968+0000 7f5442a00640 1 -- 172.21.15.87:0/3780471446 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380b5070 con 0x7f5444165500 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.968+0000 7f5449c00640 1 -- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5444165500 msgr2=0x7f54440cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.968+0000 7f5449c00640 1 --2- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5444165500 0x7f54440cec60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54380097c0 tx=0x7f543801ec90 comp rx=0 tx=0).stop 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.972+0000 7f5449c00640 1 -- 172.21.15.87:0/3780471446 shutdown_connections 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.972+0000 7f5449c00640 1 --2- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 0x7f54440d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.972+0000 7f5449c00640 1 --2- 172.21.15.87:0/3780471446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5444165500 0x7f54440cec60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.972+0000 7f5449c00640 1 -- 172.21.15.87:0/3780471446 >> 172.21.15.87:0/3780471446 conn(0x7f544406b090 msgr2=0x7f544406b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.968+0000 7f32b3e00640 1 -- 172.21.15.87:0/1709831939 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32ac01f020 con 0x7f32bc0ccc20 2024-04-25T10:12:31.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.972+0000 7f5449c00640 1 -- 172.21.15.87:0/3780471446 shutdown_connections 2024-04-25T10:12:31.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.972+0000 7f5449c00640 1 -- 172.21.15.87:0/3780471446 wait complete. 2024-04-25T10:12:31.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.972+0000 7f5449c00640 1 Processor -- start 2024-04-25T10:12:31.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449c00640 1 -- start start 2024-04-25T10:12:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 0x7f544415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54440d0ec0 0x7f54441557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5444165500 0x7f54440c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440b8ad0 con 0x7f5444165500 2024-04-25T10:12:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440b87f0 con 0x7f54440d0ec0 2024-04-25T10:12:31.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54441658e0 con 0x7f54440b2760 2024-04-25T10:12:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5444165500 0x7f54440c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5444165500 0x7f54440c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57262/0 (socket says 172.21.15.87:57262) 2024-04-25T10:12:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 -- 172.21.15.87:0/1486691366 learned_addr learned my addr 172.21.15.87:0/1486691366 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:31.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 -- 172.21.15.87:0/1486691366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 msgr2=0x7f544415ebb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:12:31.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 0x7f544415ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 -- 172.21.15.87:0/1486691366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54440d0ec0 msgr2=0x7f54441557c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:12:31.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54440d0ec0 0x7f54441557c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 -- 172.21.15.87:0/1486691366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544415e990 con 0x7f5444165500 2024-04-25T10:12:31.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.976+0000 7f5449200640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5444165500 0x7f54440c8bb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f543c00cb40 tx=0x7f543c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.984+0000 7f5437e00640 1 -- 172.21.15.87:0/1486691366 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c017070 con 0x7f5444165500 2024-04-25T10:12:31.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.984+0000 7f5437e00640 1 -- 172.21.15.87:0/1486691366 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c00aca0 con 0x7f5444165500 2024-04-25T10:12:31.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.984+0000 7f5437e00640 1 -- 172.21.15.87:0/1486691366 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c017070 con 0x7f5444165500 2024-04-25T10:12:31.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.984+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54440c92b0 con 0x7f5444165500 2024-04-25T10:12:31.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.984+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54440c9870 con 0x7f5444165500 2024-04-25T10:12:31.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.988+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5444075120 con 0x7f5444165500 2024-04-25T10:12:31.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.988+0000 7f5437e00640 1 -- 172.21.15.87:0/1486691366 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f543c01a070 con 0x7f5444165500 2024-04-25T10:12:32.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.996+0000 7f5437e00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5424067160 0x7f5424069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.996+0000 7f5437e00640 1 -- 172.21.15.87:0/1486691366 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f543c0849a0 con 0x7f5444165500 2024-04-25T10:12:32.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.996+0000 7f5443e00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5424067160 0x7f5424069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:31.996+0000 7f5443e00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5424067160 0x7f5424069620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54380b2000 tx=0x7f54380aa070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.000+0000 7f5437e00640 1 -- 172.21.15.87:0/1486691366 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f544415e990 con 0x7f5444165500 2024-04-25T10:12:32.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.016+0000 7f7733200640 1 Processor -- start 2024-04-25T10:12:32.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.016+0000 7f7733200640 1 -- start start 2024-04-25T10:12:32.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7733200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77340cfd20 0x7f77340d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7733200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f77340b1450 0x7f77340b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7731e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f773415bc00 0x7f77340cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60618/0 (socket says 172.21.15.87:60618) 2024-04-25T10:12:32.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7731e00640 1 -- 172.21.15.87:0/3001256101 learned_addr learned my addr 172.21.15.87:0/3001256101 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:32.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77340d06c0 con 0x7f773415bc00 2024-04-25T10:12:32.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773415bfe0 con 0x7f77340b1450 2024-04-25T10:12:32.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7731400640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77340cfd20 0x7f77340d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340cd100 con 0x7f77340cfd20 2024-04-25T10:12:32.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236823786 0 0) 0x7f77340d06c0 con 0x7f773415bc00 2024-04-25T10:12:32.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.020+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7718003530 con 0x7f773415bc00 2024-04-25T10:12:32.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 --2- 172.21.15.87:0/3001256101 >> v2:172.21.15.87:3300/0 conn(0x7f77340b1450 0x7f77340b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542010908 0 0) 0x7f7718003530 con 0x7f773415bc00 2024-04-25T10:12:32.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77340cfd20 msgr2=0x7f77340d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77340cfd20 0x7f77340d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 >> v2:172.21.15.87:3300/0 conn(0x7f77340b1450 msgr2=0x7f77340b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 --2- 172.21.15.87:0/3001256101 >> v2:172.21.15.87:3300/0 conn(0x7f77340b1450 0x7f77340b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340cd3e0 con 0x7f773415bc00 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f7724001680 con 0x7f773415bc00 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 >> v1:172.21.15.87:6789/0 conn(0x7f773415bc00 legacy=0x7f77340cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7718003a30 0x7f7718003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7718004810 0x7f7718004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f771800fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773415bfe0 con 0x7f7718004810 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340cd100 con 0x7f7718003a30 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7718007a30 con 0x7f77180051d0 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 --2- 172.21.15.87:0/3001256101 >> v2:172.21.15.87:3300/0 conn(0x7f77340b1450 0x7f77340b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7731e00640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7718003a30 0x7f7718003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f771800fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 -- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7718003a30 msgr2=0x7f7718003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7718003a30 0x7f7718003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 -- 172.21.15.87:0/3001256101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7718004810 msgr2=0x7f7718004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:12:32.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7718004810 0x7f7718004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.024+0000 7f7732800640 1 -- 172.21.15.87:0/3001256101 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340cd3e0 con 0x7f77180051d0 2024-04-25T10:12:32.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7732800640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f771800fde0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f772c0073e0 tx=0x7f772c01cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe3aba9d8aa5ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c0b2070 con 0x7f77180051d0 2024-04-25T10:12:32.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7730a00640 1 -- 172.21.15.87:0/3001256101 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c0a5e40 con 0x7f77180051d0 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 msgr2=0x7f771800fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7733200640 1 --2- 172.21.15.87:0/3001256101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f771800fde0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f772c0073e0 tx=0x7f772c01cf40 comp rx=0 tx=0).stop 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7732800640 1 -- 172.21.15.87:0/3001256101 reap_dead start 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 shutdown_connections 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 >> 172.21.15.87:0/3001256101 conn(0x7f77340d1740 msgr2=0x7f77340d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 shutdown_connections 2024-04-25T10:12:32.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.032+0000 7f7733200640 1 -- 172.21.15.87:0/3001256101 wait complete. 2024-04-25T10:12:32.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.036+0000 7f7733200640 1 Processor -- start 2024-04-25T10:12:32.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.036+0000 7f7733200640 1 -- start start 2024-04-25T10:12:32.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7733200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f77340ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7733200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773414b750 0x7f773414bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7733200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f773415c200 0x7f773415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f77340ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f77340ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38238/0 (socket says 172.21.15.87:38238) 2024-04-25T10:12:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 -- 172.21.15.87:0/1352616132 learned_addr learned my addr 172.21.15.87:0/1352616132 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7732800640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f773415c200 0x7f773415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731400640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773414b750 0x7f773414bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340b0f30 con 0x7f773415c200 2024-04-25T10:12:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340b0c50 con 0x7f773414b750 2024-04-25T10:12:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340cfa90 con 0x7f77180051d0 2024-04-25T10:12:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 -- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773414b750 msgr2=0x7f773414bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773414b750 0x7f773414bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 -- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f773415c200 msgr2=0x7f773415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f773415c200 0x7f773415c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.040+0000 7f7731e00640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773415cf30 con 0x7f77180051d0 2024-04-25T10:12:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.044+0000 7f7731e00640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f77340ca520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7724011850 tx=0x7f7724011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.044+0000 7f772a000640 1 -- 172.21.15.87:0/1352616132 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400faf0 con 0x7f77180051d0 2024-04-25T10:12:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.044+0000 7f772a000640 1 -- 172.21.15.87:0/1352616132 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772401b070 con 0x7f77180051d0 2024-04-25T10:12:32.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.044+0000 7f772a000640 1 -- 172.21.15.87:0/1352616132 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724016440 con 0x7f77180051d0 2024-04-25T10:12:32.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.044+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773415d210 con 0x7f77180051d0 2024-04-25T10:12:32.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.044+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773415d7d0 con 0x7f77180051d0 2024-04-25T10:12:32.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7f772a000640 1 -- 172.21.15.87:0/1352616132 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77240165e0 con 0x7f77180051d0 2024-04-25T10:12:32.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7734075120 con 0x7f77180051d0 2024-04-25T10:12:32.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7fce2c600640 1 Processor -- start 2024-04-25T10:12:32.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7f4100800640 1 Processor -- start 2024-04-25T10:12:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7fce2c600640 1 -- start start 2024-04-25T10:12:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7f772a000640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7714066f60 0x7f7714069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7f772a000640 1 -- 172.21.15.87:0/1352616132 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7724088350 con 0x7f77180051d0 2024-04-25T10:12:32.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7f7731400640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7714066f60 0x7f7714069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f7731400640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7714066f60 0x7f7714069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77341f2760 tx=0x7f7720008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce2c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce240b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.048+0000 7f4100800640 1 -- start start 2024-04-25T10:12:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce2c600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce240cf930 0x7fce240cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce29c00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fce24161150 0x7fce2406c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39992/0 (socket says 172.21.15.87:39992) 2024-04-25T10:12:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce29c00640 1 -- 172.21.15.87:0/2905004772 learned_addr learned my addr 172.21.15.87:0/2905004772 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce29200640 1 --2- 172.21.15.87:0/2905004772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce240b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f4100800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40f815a9e0 0x7f40f806d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f4100800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f815adc0 con 0x7f40f815e580 2024-04-25T10:12:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f4100800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f806e2d0 con 0x7f40f80cf3d0 2024-04-25T10:12:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f4100800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f806e5b0 con 0x7f40f815a9e0 2024-04-25T10:12:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce2c600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce24070530 con 0x7fce24161150 2024-04-25T10:12:32.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce2c600640 1 -- 172.21.15.87:0/2905004772 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce240b29b0 con 0x7fce240cf930 2024-04-25T10:12:32.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce2c600640 1 -- 172.21.15.87:0/2905004772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b2f50 con 0x7fce24070cd0 2024-04-25T10:12:32.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f40fde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40f815a9e0 0x7f40f806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f40fde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40f815a9e0 0x7f40f806d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43254/0 (socket says 172.21.15.87:43254) 2024-04-25T10:12:32.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f40fde00640 1 -- 172.21.15.87:0/1355163674 learned_addr learned my addr 172.21.15.87:0/1355163674 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce2a600640 1 --2- 172.21.15.87:0/2905004772 >> v2:172.21.15.87:3300/0 conn(0x7fce240cf930 0x7fce240cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce23e00640 1 -- 172.21.15.87:0/2905004772 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739719356 0 0) 0x7fce18001120 con 0x7fce24161150 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce29200640 1 -- 172.21.15.87:0/2905004772 >> v2:172.21.15.87:3300/0 conn(0x7fce240cf930 msgr2=0x7fce240cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce29200640 1 --2- 172.21.15.87:0/2905004772 >> v2:172.21.15.87:3300/0 conn(0x7fce240cf930 0x7fce240cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce29200640 1 -- 172.21.15.87:0/2905004772 >> v1:172.21.15.169:6789/0 conn(0x7fce24161150 legacy=0x7fce2406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce29200640 1 -- 172.21.15.87:0/2905004772 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240ae2f0 con 0x7fce24070cd0 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7fce2a600640 1 --2- 172.21.15.87:0/2905004772 >> v2:172.21.15.87:3300/0 conn(0x7fce240cf930 0x7fce240cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f40fde00640 1 -- 172.21.15.87:0/1355163674 >> v1:172.21.15.169:6789/0 conn(0x7f40f80cf3d0 legacy=0x7f40f80cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f40fde00640 1 -- 172.21.15.87:0/1355163674 >> v1:172.21.15.87:6789/0 conn(0x7f40f815e580 legacy=0x7f40f80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.052+0000 7f40fde00640 1 -- 172.21.15.87:0/1355163674 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f806e890 con 0x7f40f815a9e0 2024-04-25T10:12:32.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.056+0000 7f772a000640 1 -- 172.21.15.87:0/1352616132 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7724059360 con 0x7f77180051d0 2024-04-25T10:12:32.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.056+0000 7f675de00640 1 Processor -- start 2024-04-25T10:12:32.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.056+0000 7f675de00640 1 -- start start 2024-04-25T10:12:32.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f40fde00640 1 --2- 172.21.15.87:0/1355163674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f806d960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009e50 tx=0x7f40f401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3577447f7c87c57f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce29200640 1 --2- 172.21.15.87:0/2905004772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce240b43b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fce140097c0 tx=0x7fce1401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd0305866ed0c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f40fca00640 1 -- 172.21.15.87:0/1355163674 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40a58d0 con 0x7f40f815a9e0 2024-04-25T10:12:32.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f40fca00640 1 -- 172.21.15.87:0/1355163674 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f40a5a70 con 0x7f40f815a9e0 2024-04-25T10:12:32.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f40fca00640 1 -- 172.21.15.87:0/1355163674 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40a5e40 con 0x7f40f815a9e0 2024-04-25T10:12:32.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce23e00640 1 -- 172.21.15.87:0/2905004772 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce140b2070 con 0x7fce24070cd0 2024-04-25T10:12:32.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce23e00640 1 -- 172.21.15.87:0/2905004772 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce140ad640 con 0x7fce24070cd0 2024-04-25T10:12:32.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce23e00640 1 -- 172.21.15.87:0/2905004772 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce140b2070 con 0x7fce24070cd0 2024-04-25T10:12:32.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f675de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f675806ca50 0x7f67580cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f675de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6758070310 0x7f67580707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f4100800640 1 -- 172.21.15.87:0/1355163674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 msgr2=0x7f40f806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f4100800640 1 --2- 172.21.15.87:0/1355163674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f806d960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009e50 tx=0x7f40f401ca00 comp rx=0 tx=0).stop 2024-04-25T10:12:32.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f675de00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67580b4470 con 0x7f67580cf930 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f675de00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580cf4d0 con 0x7f675806ca50 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f675de00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580b4cd0 con 0x7f6758070310 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f4100800640 1 -- 172.21.15.87:0/1355163674 shutdown_connections 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f4100800640 1 --2- 172.21.15.87:0/1355163674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f806d960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f4100800640 1 -- 172.21.15.87:0/1355163674 >> 172.21.15.87:0/1355163674 conn(0x7f40f80b2380 msgr2=0x7f40f80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f6757400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6758070310 0x7f67580707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f6757400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6758070310 0x7f67580707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43268/0 (socket says 172.21.15.87:43268) 2024-04-25T10:12:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f6757400640 1 -- 172.21.15.87:0/1750904353 learned_addr learned my addr 172.21.15.87:0/1750904353 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 -- 172.21.15.87:0/2905004772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 msgr2=0x7fce240b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 --2- 172.21.15.87:0/2905004772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce240b43b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fce140097c0 tx=0x7fce1401c4c0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 -- 172.21.15.87:0/2905004772 shutdown_connections 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 --2- 172.21.15.87:0/2905004772 >> v2:172.21.15.87:3300/0 conn(0x7fce240cf930 0x7fce240cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 --2- 172.21.15.87:0/2905004772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce240b43b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 -- 172.21.15.87:0/2905004772 >> 172.21.15.87:0/2905004772 conn(0x7fce240669a0 msgr2=0x7fce24066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f674fe00640 1 --2- 172.21.15.87:0/1750904353 >> v2:172.21.15.87:3300/0 conn(0x7f675806ca50 0x7f67580cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 -- 172.21.15.87:0/2905004772 shutdown_connections 2024-04-25T10:12:32.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7fce2c600640 1 -- 172.21.15.87:0/2905004772 wait complete. 2024-04-25T10:12:32.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f6757400640 1 -- 172.21.15.87:0/1750904353 >> v2:172.21.15.87:3300/0 conn(0x7f675806ca50 msgr2=0x7f67580cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f6757400640 1 --2- 172.21.15.87:0/1750904353 >> v2:172.21.15.87:3300/0 conn(0x7f675806ca50 0x7f67580cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f6757400640 1 -- 172.21.15.87:0/1750904353 >> v1:172.21.15.87:6789/0 conn(0x7f67580cf930 legacy=0x7f67580cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.060+0000 7f6757400640 1 -- 172.21.15.87:0/1750904353 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580b4710 con 0x7f6758070310 2024-04-25T10:12:32.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.064+0000 7f4100800640 1 -- 172.21.15.87:0/1355163674 shutdown_connections 2024-04-25T10:12:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.064+0000 7fce2c600640 1 Processor -- start 2024-04-25T10:12:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.064+0000 7fce2c600640 1 -- start start 2024-04-25T10:12:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.064+0000 7f4100800640 1 -- 172.21.15.87:0/1355163674 wait complete. 2024-04-25T10:12:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.064+0000 7f4100800640 1 Processor -- start 2024-04-25T10:12:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f4100800640 1 -- start start 2024-04-25T10:12:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce2415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce240cf930 0x7fce240ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce24161150 0x7fce240ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b3d70 con 0x7fce24161150 2024-04-25T10:12:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b3bf0 con 0x7fce240cf930 2024-04-25T10:12:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b3ef0 con 0x7fce24070cd0 2024-04-25T10:12:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f6757400640 1 --2- 172.21.15.87:0/1750904353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f67580707a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6758166c50 tx=0x7f675001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57537eb4f8221d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce29200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce240cf930 0x7fce240ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce29c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce2415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce29200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce240cf930 0x7fce240ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43280/0 (socket says 172.21.15.87:43280) 2024-04-25T10:12:32.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce29200640 1 -- 172.21.15.87:0/1460989959 learned_addr learned my addr 172.21.15.87:0/1460989959 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f6756000640 1 -- 172.21.15.87:0/1750904353 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500a56d0 con 0x7f6758070310 2024-04-25T10:12:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f6756000640 1 -- 172.21.15.87:0/1750904353 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67500a5870 con 0x7f6758070310 2024-04-25T10:12:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f6756000640 1 -- 172.21.15.87:0/1750904353 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500a5c40 con 0x7f6758070310 2024-04-25T10:12:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2a600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce24161150 0x7fce240ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f4100800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40f80cf3d0 0x7f40f8151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f4100800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f4100800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40f815e580 0x7f40f80c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f40fde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40f80cf3d0 0x7f40f8151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f40fde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40f80cf3d0 0x7f40f8151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38274/0 (socket says 172.21.15.87:38274) 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f40fde00640 1 -- 172.21.15.87:0/2165875408 learned_addr learned my addr 172.21.15.87:0/2165875408 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80cc170 con 0x7f40f815e580 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f806e0b0 con 0x7f40f815a9e0 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80cc2f0 con 0x7f40f80cf3d0 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f40fd400640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2a600640 1 -- 172.21.15.87:0/1460989959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 msgr2=0x7fce2415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2a600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce2415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2a600640 1 -- 172.21.15.87:0/1460989959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce240cf930 msgr2=0x7fce240ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2a600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce240cf930 0x7fce240ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7fce2a600640 1 -- 172.21.15.87:0/1460989959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2415ebc0 con 0x7fce24161150 2024-04-25T10:12:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f40fe800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40f815e580 0x7f40f80c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f675de00640 1 -- 172.21.15.87:0/1750904353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 msgr2=0x7f67580707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.068+0000 7f675de00640 1 --2- 172.21.15.87:0/1750904353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f67580707a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6758166c50 tx=0x7f675001c920 comp rx=0 tx=0).stop 2024-04-25T10:12:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fde00640 1 -- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 msgr2=0x7f40f80c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fde00640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f80c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fde00640 1 -- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40f815e580 msgr2=0x7f40f80c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fde00640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40f815e580 0x7f40f80c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fde00640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f80c7ce0 con 0x7f40f80cf3d0 2024-04-25T10:12:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7fce2a600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce24161150 0x7fce240ac690 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00cc90 tx=0x7fce1c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fe800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40f815e580 0x7f40f80c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f675de00640 1 -- 172.21.15.87:0/1750904353 shutdown_connections 2024-04-25T10:12:32.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f675de00640 1 --2- 172.21.15.87:0/1750904353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f67580707a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f675de00640 1 --2- 172.21.15.87:0/1750904353 >> v2:172.21.15.87:3300/0 conn(0x7f675806ca50 0x7f67580cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f675de00640 1 -- 172.21.15.87:0/1750904353 >> 172.21.15.87:0/1750904353 conn(0x7f6758066b70 msgr2=0x7f6758066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fd400640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f80c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f675de00640 1 -- 172.21.15.87:0/1750904353 shutdown_connections 2024-04-25T10:12:32.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f675de00640 1 -- 172.21.15.87:0/1750904353 wait complete. 2024-04-25T10:12:32.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40fde00640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40f80cf3d0 0x7f40f8151fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40f4019610 tx=0x7f40f40197a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7fce21600640 1 -- 172.21.15.87:0/1460989959 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00f680 con 0x7fce24161150 2024-04-25T10:12:32.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40f2000640 1 -- 172.21.15.87:0/2165875408 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40bc030 con 0x7f40f80cf3d0 2024-04-25T10:12:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7fce21600640 1 -- 172.21.15.87:0/1460989959 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00f820 con 0x7fce24161150 2024-04-25T10:12:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7fce21600640 1 -- 172.21.15.87:0/1460989959 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00f680 con 0x7fce24161150 2024-04-25T10:12:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f40f2000640 1 -- 172.21.15.87:0/2165875408 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f40090d0 con 0x7f40f80cf3d0 2024-04-25T10:12:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2415eea0 con 0x7fce24161150 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f80c7fc0 con 0x7f40f80cf3d0 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.072+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f80c8580 con 0x7f40f80cf3d0 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.076+0000 7f40f2000640 1 -- 172.21.15.87:0/2165875408 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4009510 con 0x7f40f80cf3d0 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.076+0000 7f675de00640 1 Processor -- start 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.076+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce24204a40 con 0x7fce24161150 2024-04-25T10:12:32.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.076+0000 7f675de00640 1 -- start start 2024-04-25T10:12:32.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.076+0000 7f40f2000640 1 -- 172.21.15.87:0/2165875408 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40f40bf070 con 0x7f40f80cf3d0 2024-04-25T10:12:32.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.076+0000 7fce21600640 1 -- 172.21.15.87:0/1460989959 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce1c00f9d0 con 0x7fce24161150 2024-04-25T10:12:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d40052a0 con 0x7f40f80cf3d0 2024-04-25T10:12:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2404ef30 con 0x7fce24161150 2024-04-25T10:12:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f40f2000640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f40dc067030 0x7f40dc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f40f2000640 1 -- 172.21.15.87:0/2165875408 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40f41202d0 con 0x7f40f80cf3d0 2024-04-25T10:12:32.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f675de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f675806ca50 0x7f6758153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7fce21600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fce08067160 0x7fce08069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f675de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f675815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f675de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67580cf930 0x7f6758159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f674fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f675815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f674fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f675815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43304/0 (socket says 172.21.15.87:43304) 2024-04-25T10:12:32.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f674fe00640 1 -- 172.21.15.87:0/2266061402 learned_addr learned my addr 172.21.15.87:0/2266061402 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7fce29c00640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fce08067160 0x7fce08069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758071fc0 con 0x7f675806ca50 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758071e40 con 0x7f6758070310 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67580b2e30 con 0x7f67580cf930 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f6757400640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67580cf930 0x7f6758159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f6757400640 1 -- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 msgr2=0x7f675815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f6757400640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f675815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f6757400640 1 -- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f675806ca50 msgr2=0x7f6758153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f6757400640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f675806ca50 0x7f6758153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f6757400640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758156850 con 0x7f67580cf930 2024-04-25T10:12:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.080+0000 7f40fd400640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f40dc067030 0x7f40dc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f6757400640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67580cf930 0x7f6758159b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67500ab000 tx=0x7f67500196c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7fce29c00640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fce08067160 0x7fce08069620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce18000f70 tx=0x7fce18007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f40fd400640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f40dc067030 0x7f40dc0694f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40f815c0c0 tx=0x7f40ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7fce21600640 1 -- 172.21.15.87:0/1460989959 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce1c083170 con 0x7fce24161150 2024-04-25T10:12:32.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f674f400640 1 -- 172.21.15.87:0/2266061402 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500bc020 con 0x7f67580cf930 2024-04-25T10:12:32.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f674f400640 1 -- 172.21.15.87:0/2266061402 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67500b8670 con 0x7f67580cf930 2024-04-25T10:12:32.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f674f400640 1 -- 172.21.15.87:0/2266061402 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500b8a20 con 0x7f67580cf930 2024-04-25T10:12:32.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6758156b30 con 0x7f67580cf930 2024-04-25T10:12:32.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f40f2000640 1 -- 172.21.15.87:0/2165875408 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40f4016620 con 0x7f40f80cf3d0 2024-04-25T10:12:32.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.084+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6758157f70 con 0x7f67580cf930 2024-04-25T10:12:32.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.088+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675804ef30 con 0x7f67580cf930 2024-04-25T10:12:32.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.092+0000 7fce21600640 1 -- 172.21.15.87:0/1460989959 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce1c014030 con 0x7fce24161150 2024-04-25T10:12:32.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.092+0000 7f674f400640 1 -- 172.21.15.87:0/2266061402 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67500bf020 con 0x7f67580cf930 2024-04-25T10:12:32.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.092+0000 7f674f400640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6734066fe0 0x7f67340694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.092+0000 7f674f400640 1 -- 172.21.15.87:0/2266061402 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f675011fed0 con 0x7f67580cf930 2024-04-25T10:12:32.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.096+0000 7f6756a00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6734066fe0 0x7f67340694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.096+0000 7f6756a00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6734066fe0 0x7f67340694a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6758161e50 tx=0x7f6748006bb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.104+0000 7f674f400640 1 -- 172.21.15.87:0/2266061402 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67500c6030 con 0x7f67580cf930 2024-04-25T10:12:32.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.112+0000 7f0c15e00640 1 Processor -- start 2024-04-25T10:12:32.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.112+0000 7f0c15e00640 1 -- start start 2024-04-25T10:12:32.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.112+0000 7fe3fe200640 1 Processor -- start 2024-04-25T10:12:32.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.112+0000 7fe3fe200640 1 -- start start 2024-04-25T10:12:32.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.112+0000 7f0c15e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c1015de00 0x7f0c100ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.112+0000 7f0c15e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1015e1e0 0x7f0c1015e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3fe200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3f80d0040 0x7fe3f806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3fe200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f80b2770 con 0x7fe3f815bc40 2024-04-25T10:12:32.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3fe200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f806c670 con 0x7fe3f80cda10 2024-04-25T10:12:32.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3fe200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f806c950 con 0x7fe3f80d0040 2024-04-25T10:12:32.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c15e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1015ecf0 con 0x7f0c100cf9d0 2024-04-25T10:12:32.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c15e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1015efd0 con 0x7f0c1015de00 2024-04-25T10:12:32.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c15e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100ada20 con 0x7f0c1015e1e0 2024-04-25T10:12:32.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c0f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0c100cf9d0 0x7f0c100b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60648/0 (socket says 172.21.15.87:60648) 2024-04-25T10:12:32.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c0f400640 1 -- 172.21.15.87:0/3074191957 learned_addr learned my addr 172.21.15.87:0/3074191957 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:32.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f6a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe3f80cda10 0x7fe3f80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:40010/0 (socket says 172.21.15.87:40010) 2024-04-25T10:12:32.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f6a00640 1 -- 172.21.15.87:0/865278548 learned_addr learned my addr 172.21.15.87:0/865278548 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:32.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f7e00640 1 --2- 172.21.15.87:0/865278548 >> v2:172.21.15.87:3300/0 conn(0x7fe3f80d0040 0x7fe3f806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c0ea00640 1 --2- 172.21.15.87:0/3074191957 >> v2:172.21.15.87:3300/0 conn(0x7f0c1015de00 0x7f0c100ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f6000640 1 -- 172.21.15.87:0/865278548 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372109679 0 0) 0x7fe3f806c670 con 0x7fe3f80cda10 2024-04-25T10:12:32.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f6000640 1 -- 172.21.15.87:0/865278548 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3dc003530 con 0x7fe3f80cda10 2024-04-25T10:12:32.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f7e00640 1 -- 172.21.15.87:0/865278548 >> v1:172.21.15.169:6789/0 conn(0x7fe3f80cda10 legacy=0x7fe3f80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f7e00640 1 -- 172.21.15.87:0/865278548 >> v1:172.21.15.87:6789/0 conn(0x7fe3f815bc40 legacy=0x7fe3f80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7fe3f7e00640 1 -- 172.21.15.87:0/865278548 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f806cc30 con 0x7fe3f80d0040 2024-04-25T10:12:32.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c0ea00640 1 -- 172.21.15.87:0/3074191957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1015e1e0 msgr2=0x7f0c1015e620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:12:32.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c0ea00640 1 --2- 172.21.15.87:0/3074191957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1015e1e0 0x7f0c1015e620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c0ea00640 1 -- 172.21.15.87:0/3074191957 >> v1:172.21.15.87:6789/0 conn(0x7f0c100cf9d0 legacy=0x7f0c100b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.116+0000 7f0c0ea00640 1 -- 172.21.15.87:0/3074191957 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c100add00 con 0x7f0c1015de00 2024-04-25T10:12:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.120+0000 7fe3f7e00640 1 --2- 172.21.15.87:0/865278548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80d0040 0x7fe3f806c090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40097c0 tx=0x7fe3e401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd1ccb8d0699058 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3f6000640 1 -- 172.21.15.87:0/865278548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40b0070 con 0x7fe3f80d0040 2024-04-25T10:12:32.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3f6000640 1 -- 172.21.15.87:0/865278548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e40ab600 con 0x7fe3f80d0040 2024-04-25T10:12:32.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c0ea00640 1 --2- 172.21.15.87:0/3074191957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1015de00 0x7f0c100ae220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097f0 tx=0x7f0c0001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae48ec6ab211963f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c0d600640 1 -- 172.21.15.87:0/3074191957 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000b0070 con 0x7f0c1015de00 2024-04-25T10:12:32.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3fe200640 1 -- 172.21.15.87:0/865278548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80d0040 msgr2=0x7fe3f806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3f6000640 1 -- 172.21.15.87:0/865278548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40ab9b0 con 0x7fe3f80d0040 2024-04-25T10:12:32.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3fe200640 1 --2- 172.21.15.87:0/865278548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80d0040 0x7fe3f806c090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40097c0 tx=0x7fe3e401c390 comp rx=0 tx=0).stop 2024-04-25T10:12:32.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c0d600640 1 -- 172.21.15.87:0/3074191957 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c000ab600 con 0x7f0c1015de00 2024-04-25T10:12:32.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c0d600640 1 -- 172.21.15.87:0/3074191957 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000b0070 con 0x7f0c1015de00 2024-04-25T10:12:32.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3fe200640 1 -- 172.21.15.87:0/865278548 shutdown_connections 2024-04-25T10:12:32.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3fe200640 1 --2- 172.21.15.87:0/865278548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80d0040 0x7fe3f806c090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3fe200640 1 -- 172.21.15.87:0/865278548 >> 172.21.15.87:0/865278548 conn(0x7fe3f8066b70 msgr2=0x7fe3f8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3fe200640 1 -- 172.21.15.87:0/865278548 shutdown_connections 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c15e00640 1 -- 172.21.15.87:0/3074191957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1015de00 msgr2=0x7f0c100ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c15e00640 1 --2- 172.21.15.87:0/3074191957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1015de00 0x7f0c100ae220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097f0 tx=0x7f0c0001c4c0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c15e00640 1 -- 172.21.15.87:0/3074191957 shutdown_connections 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c15e00640 1 --2- 172.21.15.87:0/3074191957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1015e1e0 0x7f0c1015e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c15e00640 1 --2- 172.21.15.87:0/3074191957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1015de00 0x7f0c100ae220 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097f0 tx=0x7f0c0001c4c0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7f0c15e00640 1 -- 172.21.15.87:0/3074191957 >> 172.21.15.87:0/3074191957 conn(0x7f0c100669a0 msgr2=0x7f0c10066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.124+0000 7fe3fe200640 1 -- 172.21.15.87:0/865278548 wait complete. 2024-04-25T10:12:32.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.128+0000 7f0c15e00640 1 -- 172.21.15.87:0/3074191957 shutdown_connections 2024-04-25T10:12:32.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.128+0000 7f0c15e00640 1 -- 172.21.15.87:0/3074191957 wait complete. 2024-04-25T10:12:32.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.128+0000 7fe3fe200640 1 Processor -- start 2024-04-25T10:12:32.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.128+0000 7fe3fe200640 1 -- start start 2024-04-25T10:12:32.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.128+0000 7f0c15e00640 1 Processor -- start 2024-04-25T10:12:32.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.128+0000 7f0c15e00640 1 -- start start 2024-04-25T10:12:32.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3fe200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80cda10 0x7fe3f8156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c15e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c100cf9d0 0x7f0c100ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80cda10 0x7fe3f8156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c15e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c1015e1e0 0x7f0c1014f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c100cf9d0 0x7f0c100ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c100cf9d0 0x7f0c100ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57368/0 (socket says 172.21.15.87:57368) 2024-04-25T10:12:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80cda10 0x7fe3f8156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57364/0 (socket says 172.21.15.87:57364) 2024-04-25T10:12:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0ea00640 1 -- 172.21.15.87:0/1866437047 learned_addr learned my addr 172.21.15.87:0/1866437047 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0e000640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c1015e1e0 0x7f0c1014f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3f7400640 1 -- 172.21.15.87:0/1182540190 learned_addr learned my addr 172.21.15.87:0/1182540190 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3fe200640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f80d0040 0x7fe3f8155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c15e00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1014f900 0x7f0c101546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100ae8e0 con 0x7f0c100cf9d0 2024-04-25T10:12:32.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100ae760 con 0x7f0c1015e1e0 2024-04-25T10:12:32.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1016b070 con 0x7f0c1014f900 2024-04-25T10:12:32.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3fe200640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3f815bc40 0x7fe3f8155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3fe200640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8169980 con 0x7fe3f80cda10 2024-04-25T10:12:32.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3fe200640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8169800 con 0x7fe3f80d0040 2024-04-25T10:12:32.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3f7e00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3f815bc40 0x7fe3f8155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0e000640 1 -- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1014f900 msgr2=0x7f0c101546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0e000640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1014f900 0x7f0c101546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0e000640 1 -- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c100cf9d0 msgr2=0x7f0c100ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0e000640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c100cf9d0 0x7f0c100ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3f6a00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f80d0040 0x7fe3f8155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0e000640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10152290 con 0x7f0c1015e1e0 2024-04-25T10:12:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7fe3fe200640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8169b00 con 0x7fe3f815bc40 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.132+0000 7f0c0e000640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c1015e1e0 0x7f0c1014f3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00b410 tx=0x7f0bfc00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7f0c07400640 1 -- 172.21.15.87:0/1866437047 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc015020 con 0x7f0c1015e1e0 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c10152570 con 0x7f0c1015e1e0 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c10152b30 con 0x7f0c1015e1e0 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7f0c07400640 1 -- 172.21.15.87:0/1866437047 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc011830 con 0x7f0c1015e1e0 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7f0c07400640 1 -- 172.21.15.87:0/1866437047 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc015020 con 0x7f0c1015e1e0 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3f7e00640 1 -- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f80d0040 msgr2=0x7fe3f8155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3f7e00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f80d0040 0x7fe3f8155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3f7e00640 1 -- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80cda10 msgr2=0x7fe3f8156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3f7e00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80cda10 0x7fe3f8156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3f7e00640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8157560 con 0x7fe3f815bc40 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3f7e00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3f815bc40 0x7fe3f8155740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3f806ddb0 tx=0x7fe3e4019130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3ef400640 1 -- 172.21.15.87:0/1182540190 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40b9020 con 0x7fe3f815bc40 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3ef400640 1 -- 172.21.15.87:0/1182540190 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e40abbc0 con 0x7fe3f815bc40 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3ef400640 1 -- 172.21.15.87:0/1182540190 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40b0040 con 0x7fe3f815bc40 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3fe200640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f8157840 con 0x7fe3f815bc40 2024-04-25T10:12:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7fe3fe200640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8157e00 con 0x7fe3f815bc40 2024-04-25T10:12:32.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.140+0000 7fe3ef400640 1 -- 172.21.15.87:0/1182540190 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3e40bc020 con 0x7fe3f815bc40 2024-04-25T10:12:32.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.140+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc0052a0 con 0x7fe3f815bc40 2024-04-25T10:12:32.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.136+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1004ef30 con 0x7f0c1015e1e0 2024-04-25T10:12:32.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.140+0000 7f0c07400640 1 -- 172.21.15.87:0/1866437047 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bfc018020 con 0x7f0c1015e1e0 2024-04-25T10:12:32.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.140+0000 7f0c07400640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bf0067360 0x7f0bf0069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.140+0000 7f0c07400640 1 -- 172.21.15.87:0/1866437047 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bfc011630 con 0x7f0c1015e1e0 2024-04-25T10:12:32.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.140+0000 7f0c0ea00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bf0067360 0x7f0bf0069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.144+0000 7fe3ef400640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3d8067030 0x7fe3d80694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.144+0000 7fe3ef400640 1 -- 172.21.15.87:0/1182540190 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3e411f3c0 con 0x7fe3f815bc40 2024-04-25T10:12:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.144+0000 7f0c07400640 1 -- 172.21.15.87:0/1866437047 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bfc01f020 con 0x7f0c1015e1e0 2024-04-25T10:12:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.144+0000 7f0c0ea00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bf0067360 0x7f0bf0069820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c0001ab70 tx=0x7f0c000a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.144+0000 7fe3f7400640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3d8067030 0x7fe3d80694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.148+0000 7fe3f7400640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3d8067030 0x7fe3d80694f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80099a0 tx=0x7fe3e80090e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.148+0000 7fe3ef400640 1 -- 172.21.15.87:0/1182540190 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3e40c3030 con 0x7fe3f815bc40 2024-04-25T10:12:32.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.316+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3294005590 con 0x7f32bc0ccc20 2024-04-25T10:12:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.316+0000 7f32b3e00640 1 -- 172.21.15.87:0/1709831939 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32ac01f020 con 0x7f32bc0ccc20 2024-04-25T10:12:32.323 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:32.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f329c067030 msgr2=0x7f329c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f329c067030 0x7f329c0694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32a801c530 tx=0x7f32a80a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 msgr2=0x7f32bc0c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc0c8f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b440 tx=0x7f32ac00b900 comp rx=0 tx=0).stop 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 shutdown_connections 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f329c067030 0x7f329c0694f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc159e70 0x7f32bc0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32bc0ccc20 0x7f32bc0c8f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 --2- 172.21.15.87:0/1709831939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32bc070e00 0x7f32bc1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 >> 172.21.15.87:0/1709831939 conn(0x7f32bc0b1330 msgr2=0x7f32bc06e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 shutdown_connections 2024-04-25T10:12:32.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.320+0000 7f32c2c00640 1 -- 172.21.15.87:0/1709831939 wait complete. 2024-04-25T10:12:32.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.332+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f54440b3f90 con 0x7f5444165500 2024-04-25T10:12:32.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.336+0000 7f5437e00640 1 -- 172.21.15.87:0/1486691366 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f544415e990 con 0x7f5444165500 2024-04-25T10:12:32.350 INFO:teuthology.orchestra.run.smithi087.stdout:55834574855 2024-04-25T10:12:32.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.344+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5424067160 msgr2=0x7f5424069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.344+0000 7f5449c00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5424067160 0x7f5424069620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54380b2000 tx=0x7f54380aa070 comp rx=0 tx=0).stop 2024-04-25T10:12:32.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.344+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5444165500 msgr2=0x7f54440c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.344+0000 7f5449c00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5444165500 0x7f54440c8bb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f543c00cb40 tx=0x7f543c00a060 comp rx=0 tx=0).stop 2024-04-25T10:12:32.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.348+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 shutdown_connections 2024-04-25T10:12:32.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.348+0000 7f5449c00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5424067160 0x7f5424069620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.348+0000 7f5449c00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5444165500 0x7f54440c8bb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.348+0000 7f5449c00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54440d0ec0 0x7f54441557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.348+0000 7f5449c00640 1 --2- 172.21.15.87:0/1486691366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54440b2760 0x7f544415ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.348+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 >> 172.21.15.87:0/1486691366 conn(0x7f544406b090 msgr2=0x7f54440b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.348+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 shutdown_connections 2024-04-25T10:12:32.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-25T10:12:32.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.352+0000 7f5449c00640 1 -- 172.21.15.87:0/1486691366 wait complete. 2024-04-25T10:12:32.358 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-25T10:12:32.388 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fce24160770 con 0x7fce24161150 2024-04-25T10:12:32.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce21600640 1 -- 172.21.15.87:0/1460989959 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce1c055bd0 con 0x7fce24161150 2024-04-25T10:12:32.392 INFO:teuthology.orchestra.run.smithi087.stdout:55834574853 2024-04-25T10:12:32.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f77340cc590 con 0x7f77180051d0 2024-04-25T10:12:32.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fce08067160 msgr2=0x7fce08069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fce08067160 0x7fce08069620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce18000f70 tx=0x7fce18007c10 comp rx=0 tx=0).stop 2024-04-25T10:12:32.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce24161150 msgr2=0x7fce240ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce24161150 0x7fce240ac690 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00cc90 tx=0x7fce1c00a650 comp rx=0 tx=0).stop 2024-04-25T10:12:32.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7f772a000640 1 -- 172.21.15.87:0/1352616132 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7724059360 con 0x7f77180051d0 2024-04-25T10:12:32.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 shutdown_connections 2024-04-25T10:12:32.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fce08067160 0x7fce08069620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce24161150 0x7fce240ac690 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce240cf930 0x7fce240ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 --2- 172.21.15.87:0/1460989959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24070cd0 0x7fce2415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.396 INFO:teuthology.orchestra.run.smithi087.stdout:55834574855 2024-04-25T10:12:32.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 >> 172.21.15.87:0/1460989959 conn(0x7fce240669a0 msgr2=0x7fce240b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 shutdown_connections 2024-04-25T10:12:32.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.388+0000 7fce2c600640 1 -- 172.21.15.87:0/1460989959 wait complete. 2024-04-25T10:12:32.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7714066f60 msgr2=0x7f7714069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7714066f60 0x7f7714069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77341f2760 tx=0x7f7720008040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 msgr2=0x7f77340ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f77340ca520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7724011850 tx=0x7f7724011d10 comp rx=0 tx=0).stop 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 shutdown_connections 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7714066f60 0x7f7714069420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f773415c200 0x7f773415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773414b750 0x7f773414bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 --2- 172.21.15.87:0/1352616132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77180051d0 0x7f77340ca520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 >> 172.21.15.87:0/1352616132 conn(0x7f77340d1740 msgr2=0x7f773406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 shutdown_connections 2024-04-25T10:12:32.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.392+0000 7f7733200640 1 -- 172.21.15.87:0/1352616132 wait complete. 2024-04-25T10:12:32.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.404+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f40d4005590 con 0x7f40f80cf3d0 2024-04-25T10:12:32.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f40f2000640 1 -- 172.21.15.87:0/2165875408 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f40f40f1410 con 0x7f40f80cf3d0 2024-04-25T10:12:32.414 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f40dc067030 msgr2=0x7f40dc0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f40dc067030 0x7f40dc0694f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40f815c0c0 tx=0x7f40ec009040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40f80cf3d0 msgr2=0x7f40f8151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40f80cf3d0 0x7f40f8151fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40f4019610 tx=0x7f40f40197a0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 shutdown_connections 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f40dc067030 0x7f40dc0694f0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40f815c0c0 tx=0x7f40ec009040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40f815e580 0x7f40f80c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40f815a9e0 0x7f40f80c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 --2- 172.21.15.87:0/2165875408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40f80cf3d0 0x7f40f8151fc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 >> 172.21.15.87:0/2165875408 conn(0x7f40f80b2380 msgr2=0x7f40f815d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 shutdown_connections 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.408+0000 7f4100800640 1 -- 172.21.15.87:0/2165875408 wait complete. 2024-04-25T10:12:32.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-25T10:12:32.418 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-25T10:12:32.432 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-25T10:12:32.437 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.440+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f675806c2e0 con 0x7f67580cf930 2024-04-25T10:12:32.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.440+0000 7f674f400640 1 -- 172.21.15.87:0/2266061402 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67500165a0 con 0x7f67580cf930 2024-04-25T10:12:32.445 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:32.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6734066fe0 msgr2=0x7f67340694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6734066fe0 0x7f67340694a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6758161e50 tx=0x7f6748006bb0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67580cf930 msgr2=0x7f6758159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67580cf930 0x7f6758159b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67500ab000 tx=0x7f67500196c0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 shutdown_connections 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6734066fe0 0x7f67340694a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67580cf930 0x7f6758159b40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758070310 0x7f675815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 --2- 172.21.15.87:0/2266061402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f675806ca50 0x7f6758153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 >> 172.21.15.87:0/2266061402 conn(0x7f6758066b70 msgr2=0x7f67580afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 shutdown_connections 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.444+0000 7f675de00640 1 -- 172.21.15.87:0/2266061402 wait complete. 2024-04-25T10:12:32.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.456+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe3cc005590 con 0x7fe3f815bc40 2024-04-25T10:12:32.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.456+0000 7fe3ef400640 1 -- 172.21.15.87:0/1182540190 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3e4016010 con 0x7fe3f815bc40 2024-04-25T10:12:32.462 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:32.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3d8067030 msgr2=0x7fe3d80694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3d8067030 0x7fe3d80694f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80099a0 tx=0x7fe3e80090e0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3f815bc40 msgr2=0x7fe3f8155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3f815bc40 0x7fe3f8155740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3f806ddb0 tx=0x7fe3e4019130 comp rx=0 tx=0).stop 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 shutdown_connections 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3d8067030 0x7fe3d80694f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3f815bc40 0x7fe3f8155740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f80d0040 0x7fe3f8155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 --2- 172.21.15.87:0/1182540190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f80cda10 0x7fe3f8156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 >> 172.21.15.87:0/1182540190 conn(0x7fe3f8066b70 msgr2=0x7fe3f815a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 shutdown_connections 2024-04-25T10:12:32.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.460+0000 7fe3ecc00640 1 -- 172.21.15.87:0/1182540190 wait complete. 2024-04-25T10:12:32.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-25T10:12:32.478 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.476+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0c100b26c0 con 0x7f0c1015e1e0 2024-04-25T10:12:32.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.476+0000 7f0c07400640 1 -- 172.21.15.87:0/1866437047 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0bfc055980 con 0x7f0c1015e1e0 2024-04-25T10:12:32.482 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-04-25T10:12:32.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-25T10:12:32.484 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bf0067360 msgr2=0x7f0bf0069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bf0067360 0x7f0bf0069820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c0001ab70 tx=0x7f0c000a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c1015e1e0 msgr2=0x7f0c1014f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c1015e1e0 0x7f0c1014f3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00b410 tx=0x7f0bfc00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 shutdown_connections 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bf0067360 0x7f0bf0069820 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c1014f900 0x7f0c101546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c1015e1e0 0x7f0c1014f3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 --2- 172.21.15.87:0/1866437047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c100cf9d0 0x7f0c100ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 >> 172.21.15.87:0/1866437047 conn(0x7f0c100669a0 msgr2=0x7f0c10160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 shutdown_connections 2024-04-25T10:12:32.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.480+0000 7f0c15e00640 1 -- 172.21.15.87:0/1866437047 wait complete. 2024-04-25T10:12:32.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-04-25T10:12:32.503 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:32.503 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:12:32.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 Processor -- start 2024-04-25T10:12:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 -- start start 2024-04-25T10:12:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa43c0cf930 0x7fa43c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa43c070c20 0x7fa43c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c15bb90 con 0x7fa43c070c20 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c0d0250 con 0x7fa43c0cf930 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa443c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c0ad840 con 0x7fa43c0ae020 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa43c070c20 0x7fa43c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa43c070c20 0x7fa43c15e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57376/0 (socket says 172.21.15.87:57376) 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 -- 172.21.15.87:0/26423285 learned_addr learned my addr 172.21.15.87:0/26423285 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa441c00640 1 --2- 172.21.15.87:0/26423285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa441200640 1 --2- 172.21.15.87:0/26423285 >> v2:172.21.15.169:3300/0 conn(0x7fa43c0cf930 0x7fa43c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 -- 172.21.15.87:0/26423285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 msgr2=0x7fa43c0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 --2- 172.21.15.87:0/26423285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c0ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 -- 172.21.15.87:0/26423285 >> v2:172.21.15.169:3300/0 conn(0x7fa43c0cf930 msgr2=0x7fa43c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 --2- 172.21.15.87:0/26423285 >> v2:172.21.15.169:3300/0 conn(0x7fa43c0cf930 0x7fa43c0cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa433e00640 1 -- 172.21.15.87:0/26423285 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c15e9f0 con 0x7fa43c070c20 2024-04-25T10:12:32.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.676+0000 7fa441c00640 1 --2- 172.21.15.87:0/26423285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa433e00640 1 --2- 172.21.15.87:0/26423285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c070c20 0x7fa43c15e170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0aec50 tx=0x7fa42401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63afd797cc2886ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa43be00640 1 -- 172.21.15.87:0/26423285 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240aa610 con 0x7fa43c070c20 2024-04-25T10:12:32.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa43be00640 1 -- 172.21.15.87:0/26423285 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4240af070 con 0x7fa43c070c20 2024-04-25T10:12:32.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa43be00640 1 -- 172.21.15.87:0/26423285 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240aa9d0 con 0x7fa43c070c20 2024-04-25T10:12:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 -- 172.21.15.87:0/26423285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c070c20 msgr2=0x7fa43c15e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 --2- 172.21.15.87:0/26423285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c070c20 0x7fa43c15e170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0aec50 tx=0x7fa42401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 -- 172.21.15.87:0/26423285 shutdown_connections 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 --2- 172.21.15.87:0/26423285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c070c20 0x7fa43c15e170 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 --2- 172.21.15.87:0/26423285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 --2- 172.21.15.87:0/26423285 >> v2:172.21.15.169:3300/0 conn(0x7fa43c0cf930 0x7fa43c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 -- 172.21.15.87:0/26423285 >> 172.21.15.87:0/26423285 conn(0x7fa43c066b70 msgr2=0x7fa43c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 -- 172.21.15.87:0/26423285 shutdown_connections 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.680+0000 7fa443c00640 1 -- 172.21.15.87:0/26423285 wait complete. 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 Processor -- start 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 -- start start 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c070c20 0x7fa43c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c0cf930 0x7fa43c1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c15e6b0 con 0x7fa43c0cf930 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c06e130 con 0x7fa43c070c20 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c161790 con 0x7fa43c0ae020 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c0cf930 0x7fa43c1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c0cf930 0x7fa43c1596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57378/0 (socket says 172.21.15.87:57378) 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 -- 172.21.15.87:0/2191015003 learned_addr learned my addr 172.21.15.87:0/2191015003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441200640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c070c20 0x7fa43c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa433e00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 -- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 msgr2=0x7fa43c154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 -- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c070c20 msgr2=0x7fa43c0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c070c20 0x7fa43c0cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 -- 172.21.15.87:0/2191015003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c1fa5a0 con 0x7fa43c0cf930 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa433e00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa441c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c0cf930 0x7fa43c1596a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa434007e40 tx=0x7fa43400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa439600640 1 -- 172.21.15.87:0/2191015003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43401a070 con 0x7fa43c0cf930 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa439600640 1 -- 172.21.15.87:0/2191015003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa434003ba0 con 0x7fa43c0cf930 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa439600640 1 -- 172.21.15.87:0/2191015003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400de80 con 0x7fa43c0cf930 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c1fa880 con 0x7fa43c0cf930 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.684+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c1fae40 con 0x7fa43c0cf930 2024-04-25T10:12:32.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.688+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c04ef30 con 0x7fa43c0cf930 2024-04-25T10:12:32.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.688+0000 7fa439600640 1 -- 172.21.15.87:0/2191015003 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa43401d070 con 0x7fa43c0cf930 2024-04-25T10:12:32.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.688+0000 7fa439600640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f60 0x7fa418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.688+0000 7fa439600640 1 -- 172.21.15.87:0/2191015003 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa434085390 con 0x7fa43c0cf930 2024-04-25T10:12:32.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.688+0000 7fa441200640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f60 0x7fa418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.688+0000 7fa441200640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f60 0x7fa418069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00b930 tx=0x7fa42c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.692+0000 7fa439600640 1 -- 172.21.15.87:0/2191015003 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa434010070 con 0x7fa43c0cf930 2024-04-25T10:12:32.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.844+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa43c15f470 con 0x7fa418066f60 2024-04-25T10:12:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.848+0000 7fa439600640 1 -- 172.21.15.87:0/2191015003 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7fa43c15f470 con 0x7fa418066f60 2024-04-25T10:12:32.858 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:32.860 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f60 msgr2=0x7fa418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f60 0x7fa418069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00b930 tx=0x7fa42c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c0cf930 msgr2=0x7fa43c1596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c0cf930 0x7fa43c1596a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa434007e40 tx=0x7fa43400d4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 shutdown_connections 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f60 0x7fa418069420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c0cf930 0x7fa43c1596a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0ae020 0x7fa43c154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 --2- 172.21.15.87:0/2191015003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c070c20 0x7fa43c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 >> 172.21.15.87:0/2191015003 conn(0x7fa43c066b70 msgr2=0x7fa43c0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 shutdown_connections 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:32.852+0000 7fa443c00640 1 -- 172.21.15.87:0/2191015003 wait complete. 2024-04-25T10:12:32.879 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T10:12:31.126186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.905140"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015534+0000","last_change":"2024-04-25T10:12:24.263197+0000","last_active":"2024-04-25T10:12:26.015534+0000","last_peered":"2024-04-25T10:12:26.015534+0000","last_clean":"2024-04-25T10:12:26.015534+0000","last_became_active":"2024-04-25T10:12:24.262932+0000","last_became_peered":"2024-04-25T10:12:24.262932+0000","last_unstale":"2024-04-25T10:12:26.015534+0000","last_undegraded":"2024-04-25T10:12:26.015534+0000","last_fullsized":"2024-04-25T10:12:26.015534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247413+0000","last_change":"2024-04-25T10:12:24.255491+0000","last_active":"2024-04-25T10:12:25.247413+0000","last_peered":"2024-04-25T10:12:25.247413+0000","last_clean":"2024-04-25T10:12:25.247413+0000","last_became_active":"2024-04-25T10:12:24.255325+0000","last_became_peered":"2024-04-25T10:12:24.255325+0000","last_unstale":"2024-04-25T10:12:25.247413+0000","last_undegraded":"2024-04-25T10:12:25.247413+0000","last_fullsized":"2024-04-25T10:12:25.247413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016327+0000","last_change":"2024-04-25T10:12:24.259328+0000","last_active":"2024-04-25T10:12:26.016327+0000","last_peered":"2024-04-25T10:12:26.016327+0000","last_clean":"2024-04-25T10:12:26.016327+0000","last_became_active":"2024-04-25T10:12:24.259065+0000","last_became_peered":"2024-04-25T10:12:24.259065+0000","last_unstale":"2024-04-25T10:12:26.016327+0000","last_undegraded":"2024-04-25T10:12:26.016327+0000","last_fullsized":"2024-04-25T10:12:26.016327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247574+0000","last_change":"2024-04-25T10:12:24.263211+0000","last_active":"2024-04-25T10:12:25.247574+0000","last_peered":"2024-04-25T10:12:25.247574+0000","last_clean":"2024-04-25T10:12:25.247574+0000","last_became_active":"2024-04-25T10:12:24.262896+0000","last_became_peered":"2024-04-25T10:12:24.262896+0000","last_unstale":"2024-04-25T10:12:25.247574+0000","last_undegraded":"2024-04-25T10:12:25.247574+0000","last_fullsized":"2024-04-25T10:12:25.247574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014990+0000","last_change":"2024-04-25T10:12:24.259922+0000","last_active":"2024-04-25T10:12:26.014990+0000","last_peered":"2024-04-25T10:12:26.014990+0000","last_clean":"2024-04-25T10:12:26.014990+0000","last_became_active":"2024-04-25T10:12:24.259657+0000","last_became_peered":"2024-04-25T10:12:24.259657+0000","last_unstale":"2024-04-25T10:12:26.014990+0000","last_undegraded":"2024-04-25T10:12:26.014990+0000","last_fullsized":"2024-04-25T10:12:26.014990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245975+0000","last_change":"2024-04-25T10:12:24.254906+0000","last_active":"2024-04-25T10:12:25.245975+0000","last_peered":"2024-04-25T10:12:25.245975+0000","last_clean":"2024-04-25T10:12:25.245975+0000","last_became_active":"2024-04-25T10:12:24.254592+0000","last_became_peered":"2024-04-25T10:12:24.254592+0000","last_unstale":"2024-04-25T10:12:25.245975+0000","last_undegraded":"2024-04-25T10:12:25.245975+0000","last_fullsized":"2024-04-25T10:12:25.245975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014929+0000","last_change":"2024-04-25T10:12:24.254975+0000","last_active":"2024-04-25T10:12:26.014929+0000","last_peered":"2024-04-25T10:12:26.014929+0000","last_clean":"2024-04-25T10:12:26.014929+0000","last_became_active":"2024-04-25T10:12:24.254639+0000","last_became_peered":"2024-04-25T10:12:24.254639+0000","last_unstale":"2024-04-25T10:12:26.014929+0000","last_undegraded":"2024-04-25T10:12:26.014929+0000","last_fullsized":"2024-04-25T10:12:26.014929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015387+0000","last_change":"2024-04-25T10:12:24.264638+0000","last_active":"2024-04-25T10:12:26.015387+0000","last_peered":"2024-04-25T10:12:26.015387+0000","last_clean":"2024-04-25T10:12:26.015387+0000","last_became_active":"2024-04-25T10:12:24.264376+0000","last_became_peered":"2024-04-25T10:12:24.264376+0000","last_unstale":"2024-04-25T10:12:26.015387+0000","last_undegraded":"2024-04-25T10:12:26.015387+0000","last_fullsized":"2024-04-25T10:12:26.015387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015572+0000","last_change":"2024-04-25T10:12:24.257501+0000","last_active":"2024-04-25T10:12:26.015572+0000","last_peered":"2024-04-25T10:12:26.015572+0000","last_clean":"2024-04-25T10:12:26.015572+0000","last_became_active":"2024-04-25T10:12:24.257199+0000","last_became_peered":"2024-04-25T10:12:24.257199+0000","last_unstale":"2024-04-25T10:12:26.015572+0000","last_undegraded":"2024-04-25T10:12:26.015572+0000","last_fullsized":"2024-04-25T10:12:26.015572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012594+0000","last_change":"2024-04-25T10:12:24.250894+0000","last_active":"2024-04-25T10:12:26.012594+0000","last_peered":"2024-04-25T10:12:26.012594+0000","last_clean":"2024-04-25T10:12:26.012594+0000","last_became_active":"2024-04-25T10:12:24.250581+0000","last_became_peered":"2024-04-25T10:12:24.250581+0000","last_unstale":"2024-04-25T10:12:26.012594+0000","last_undegraded":"2024-04-25T10:12:26.012594+0000","last_fullsized":"2024-04-25T10:12:26.012594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012005+0000","last_change":"2024-04-25T10:12:24.247801+0000","last_active":"2024-04-25T10:12:26.012005+0000","last_peered":"2024-04-25T10:12:26.012005+0000","last_clean":"2024-04-25T10:12:26.012005+0000","last_became_active":"2024-04-25T10:12:24.247485+0000","last_became_peered":"2024-04-25T10:12:24.247485+0000","last_unstale":"2024-04-25T10:12:26.012005+0000","last_undegraded":"2024-04-25T10:12:26.012005+0000","last_fullsized":"2024-04-25T10:12:26.012005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.262042+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.261770+0000","last_became_peered":"2024-04-25T10:12:24.261770+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:36.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015914+0000","last_change":"2024-04-25T10:12:24.263554+0000","last_active":"2024-04-25T10:12:26.015914+0000","last_peered":"2024-04-25T10:12:26.015914+0000","last_clean":"2024-04-25T10:12:26.015914+0000","last_became_active":"2024-04-25T10:12:24.263244+0000","last_became_peered":"2024-04-25T10:12:24.263244+0000","last_unstale":"2024-04-25T10:12:26.015914+0000","last_undegraded":"2024-04-25T10:12:26.015914+0000","last_fullsized":"2024-04-25T10:12:26.015914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015884+0000","last_change":"2024-04-25T10:12:24.246682+0000","last_active":"2024-04-25T10:12:26.015884+0000","last_peered":"2024-04-25T10:12:26.015884+0000","last_clean":"2024-04-25T10:12:26.015884+0000","last_became_active":"2024-04-25T10:12:24.246353+0000","last_became_peered":"2024-04-25T10:12:24.246353+0000","last_unstale":"2024-04-25T10:12:26.015884+0000","last_undegraded":"2024-04-25T10:12:26.015884+0000","last_fullsized":"2024-04-25T10:12:26.015884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:15.421048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246997+0000","last_change":"2024-04-25T10:12:24.265431+0000","last_active":"2024-04-25T10:12:25.246997+0000","last_peered":"2024-04-25T10:12:25.246997+0000","last_clean":"2024-04-25T10:12:25.246997+0000","last_became_active":"2024-04-25T10:12:24.265047+0000","last_became_peered":"2024-04-25T10:12:24.265047+0000","last_unstale":"2024-04-25T10:12:25.246997+0000","last_undegraded":"2024-04-25T10:12:25.246997+0000","last_fullsized":"2024-04-25T10:12:25.246997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247153+0000","last_change":"2024-04-25T10:12:24.248104+0000","last_active":"2024-04-25T10:12:25.247153+0000","last_peered":"2024-04-25T10:12:25.247153+0000","last_clean":"2024-04-25T10:12:25.247153+0000","last_became_active":"2024-04-25T10:12:24.247826+0000","last_became_peered":"2024-04-25T10:12:24.247826+0000","last_unstale":"2024-04-25T10:12:25.247153+0000","last_undegraded":"2024-04-25T10:12:25.247153+0000","last_fullsized":"2024-04-25T10:12:25.247153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247053+0000","last_change":"2024-04-25T10:12:24.257638+0000","last_active":"2024-04-25T10:12:25.247053+0000","last_peered":"2024-04-25T10:12:25.247053+0000","last_clean":"2024-04-25T10:12:25.247053+0000","last_became_active":"2024-04-25T10:12:24.257332+0000","last_became_peered":"2024-04-25T10:12:24.257332+0000","last_unstale":"2024-04-25T10:12:25.247053+0000","last_undegraded":"2024-04-25T10:12:25.247053+0000","last_fullsized":"2024-04-25T10:12:25.247053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872137+0000","last_change":"2024-04-25T10:12:24.262252+0000","last_active":"2024-04-25T10:12:25.872137+0000","last_peered":"2024-04-25T10:12:25.872137+0000","last_clean":"2024-04-25T10:12:25.872137+0000","last_became_active":"2024-04-25T10:12:24.261871+0000","last_became_peered":"2024-04-25T10:12:24.261871+0000","last_unstale":"2024-04-25T10:12:25.872137+0000","last_undegraded":"2024-04-25T10:12:25.872137+0000","last_fullsized":"2024-04-25T10:12:25.872137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:13.729919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015458+0000","last_change":"2024-04-25T10:12:24.269990+0000","last_active":"2024-04-25T10:12:26.015458+0000","last_peered":"2024-04-25T10:12:26.015458+0000","last_clean":"2024-04-25T10:12:26.015458+0000","last_became_active":"2024-04-25T10:12:24.269396+0000","last_became_peered":"2024-04-25T10:12:24.269396+0000","last_unstale":"2024-04-25T10:12:26.015458+0000","last_undegraded":"2024-04-25T10:12:26.015458+0000","last_fullsized":"2024-04-25T10:12:26.015458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012013+0000","last_change":"2024-04-25T10:12:24.262832+0000","last_active":"2024-04-25T10:12:26.012013+0000","last_peered":"2024-04-25T10:12:26.012013+0000","last_clean":"2024-04-25T10:12:26.012013+0000","last_became_active":"2024-04-25T10:12:24.262090+0000","last_became_peered":"2024-04-25T10:12:24.262090+0000","last_unstale":"2024-04-25T10:12:26.012013+0000","last_undegraded":"2024-04-25T10:12:26.012013+0000","last_fullsized":"2024-04-25T10:12:26.012013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016383+0000","last_change":"2024-04-25T10:12:24.248538+0000","last_active":"2024-04-25T10:12:26.016383+0000","last_peered":"2024-04-25T10:12:26.016383+0000","last_clean":"2024-04-25T10:12:26.016383+0000","last_became_active":"2024-04-25T10:12:24.247653+0000","last_became_peered":"2024-04-25T10:12:24.247653+0000","last_unstale":"2024-04-25T10:12:26.016383+0000","last_undegraded":"2024-04-25T10:12:26.016383+0000","last_fullsized":"2024-04-25T10:12:26.016383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246115+0000","last_change":"2024-04-25T10:12:24.260603+0000","last_active":"2024-04-25T10:12:25.246115+0000","last_peered":"2024-04-25T10:12:25.246115+0000","last_clean":"2024-04-25T10:12:25.246115+0000","last_became_active":"2024-04-25T10:12:24.260288+0000","last_became_peered":"2024-04-25T10:12:24.260288+0000","last_unstale":"2024-04-25T10:12:25.246115+0000","last_undegraded":"2024-04-25T10:12:25.246115+0000","last_fullsized":"2024-04-25T10:12:25.246115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015623+0000","last_change":"2024-04-25T10:12:24.272790+0000","last_active":"2024-04-25T10:12:26.015623+0000","last_peered":"2024-04-25T10:12:26.015623+0000","last_clean":"2024-04-25T10:12:26.015623+0000","last_became_active":"2024-04-25T10:12:24.272587+0000","last_became_peered":"2024-04-25T10:12:24.272587+0000","last_unstale":"2024-04-25T10:12:26.015623+0000","last_undegraded":"2024-04-25T10:12:26.015623+0000","last_fullsized":"2024-04-25T10:12:26.015623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245637+0000","last_change":"2024-04-25T10:12:24.265627+0000","last_active":"2024-04-25T10:12:25.245637+0000","last_peered":"2024-04-25T10:12:25.245637+0000","last_clean":"2024-04-25T10:12:25.245637+0000","last_became_active":"2024-04-25T10:12:24.265274+0000","last_became_peered":"2024-04-25T10:12:24.265274+0000","last_unstale":"2024-04-25T10:12:25.245637+0000","last_undegraded":"2024-04-25T10:12:25.245637+0000","last_fullsized":"2024-04-25T10:12:25.245637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016338+0000","last_change":"2024-04-25T10:12:24.272301+0000","last_active":"2024-04-25T10:12:26.016338+0000","last_peered":"2024-04-25T10:12:26.016338+0000","last_clean":"2024-04-25T10:12:26.016338+0000","last_became_active":"2024-04-25T10:12:24.271766+0000","last_became_peered":"2024-04-25T10:12:24.271766+0000","last_unstale":"2024-04-25T10:12:26.016338+0000","last_undegraded":"2024-04-25T10:12:26.016338+0000","last_fullsized":"2024-04-25T10:12:26.016338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012456+0000","last_change":"2024-04-25T10:12:24.265368+0000","last_active":"2024-04-25T10:12:26.012456+0000","last_peered":"2024-04-25T10:12:26.012456+0000","last_clean":"2024-04-25T10:12:26.012456+0000","last_became_active":"2024-04-25T10:12:24.264950+0000","last_became_peered":"2024-04-25T10:12:24.264950+0000","last_unstale":"2024-04-25T10:12:26.012456+0000","last_undegraded":"2024-04-25T10:12:26.012456+0000","last_fullsized":"2024-04-25T10:12:26.012456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247251+0000","last_change":"2024-04-25T10:12:24.259900+0000","last_active":"2024-04-25T10:12:25.247251+0000","last_peered":"2024-04-25T10:12:25.247251+0000","last_clean":"2024-04-25T10:12:25.247251+0000","last_became_active":"2024-04-25T10:12:24.259574+0000","last_became_peered":"2024-04-25T10:12:24.259574+0000","last_unstale":"2024-04-25T10:12:25.247251+0000","last_undegraded":"2024-04-25T10:12:25.247251+0000","last_fullsized":"2024-04-25T10:12:25.247251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015996+0000","last_change":"2024-04-25T10:12:24.272315+0000","last_active":"2024-04-25T10:12:26.015996+0000","last_peered":"2024-04-25T10:12:26.015996+0000","last_clean":"2024-04-25T10:12:26.015996+0000","last_became_active":"2024-04-25T10:12:24.271843+0000","last_became_peered":"2024-04-25T10:12:24.271843+0000","last_unstale":"2024-04-25T10:12:26.015996+0000","last_undegraded":"2024-04-25T10:12:26.015996+0000","last_fullsized":"2024-04-25T10:12:26.015996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016505+0000","last_change":"2024-04-25T10:12:24.260827+0000","last_active":"2024-04-25T10:12:26.016505+0000","last_peered":"2024-04-25T10:12:26.016505+0000","last_clean":"2024-04-25T10:12:26.016505+0000","last_became_active":"2024-04-25T10:12:24.260436+0000","last_became_peered":"2024-04-25T10:12:24.260436+0000","last_unstale":"2024-04-25T10:12:26.016505+0000","last_undegraded":"2024-04-25T10:12:26.016505+0000","last_fullsized":"2024-04-25T10:12:26.016505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246276+0000","last_change":"2024-04-25T10:12:24.261679+0000","last_active":"2024-04-25T10:12:25.246276+0000","last_peered":"2024-04-25T10:12:25.246276+0000","last_clean":"2024-04-25T10:12:25.246276+0000","last_became_active":"2024-04-25T10:12:24.261420+0000","last_became_peered":"2024-04-25T10:12:24.261420+0000","last_unstale":"2024-04-25T10:12:25.246276+0000","last_undegraded":"2024-04-25T10:12:25.246276+0000","last_fullsized":"2024-04-25T10:12:25.246276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872106+0000","last_change":"2024-04-25T10:12:24.260060+0000","last_active":"2024-04-25T10:12:25.872106+0000","last_peered":"2024-04-25T10:12:25.872106+0000","last_clean":"2024-04-25T10:12:25.872106+0000","last_became_active":"2024-04-25T10:12:24.259332+0000","last_became_peered":"2024-04-25T10:12:24.259332+0000","last_unstale":"2024-04-25T10:12:25.872106+0000","last_undegraded":"2024-04-25T10:12:25.872106+0000","last_fullsized":"2024-04-25T10:12:25.872106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011814+0000","last_change":"2024-04-25T10:12:24.247021+0000","last_active":"2024-04-25T10:12:26.011814+0000","last_peered":"2024-04-25T10:12:26.011814+0000","last_clean":"2024-04-25T10:12:26.011814+0000","last_became_active":"2024-04-25T10:12:24.246699+0000","last_became_peered":"2024-04-25T10:12:24.246699+0000","last_unstale":"2024-04-25T10:12:26.011814+0000","last_undegraded":"2024-04-25T10:12:26.011814+0000","last_fullsized":"2024-04-25T10:12:26.011814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245745+0000","last_change":"2024-04-25T10:12:24.263775+0000","last_active":"2024-04-25T10:12:25.245745+0000","last_peered":"2024-04-25T10:12:25.245745+0000","last_clean":"2024-04-25T10:12:25.245745+0000","last_became_active":"2024-04-25T10:12:24.263373+0000","last_became_peered":"2024-04-25T10:12:24.263373+0000","last_unstale":"2024-04-25T10:12:25.245745+0000","last_undegraded":"2024-04-25T10:12:25.245745+0000","last_fullsized":"2024-04-25T10:12:25.245745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:23.746340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016216+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:26.016216+0000","last_peered":"2024-04-25T10:12:26.016216+0000","last_clean":"2024-04-25T10:12:26.016216+0000","last_became_active":"2024-04-25T10:12:23.249533+0000","last_became_peered":"2024-04-25T10:12:23.249533+0000","last_unstale":"2024-04-25T10:12:26.016216+0000","last_undegraded":"2024-04-25T10:12:26.016216+0000","last_fullsized":"2024-04-25T10:12:26.016216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016549+0000","last_change":"2024-04-25T10:12:24.252703+0000","last_active":"2024-04-25T10:12:26.016549+0000","last_peered":"2024-04-25T10:12:26.016549+0000","last_clean":"2024-04-25T10:12:26.016549+0000","last_became_active":"2024-04-25T10:12:24.252372+0000","last_became_peered":"2024-04-25T10:12:24.252372+0000","last_unstale":"2024-04-25T10:12:26.016549+0000","last_undegraded":"2024-04-25T10:12:26.016549+0000","last_fullsized":"2024-04-25T10:12:26.016549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012305+0000","last_change":"2024-04-25T10:12:23.248077+0000","last_active":"2024-04-25T10:12:26.012305+0000","last_peered":"2024-04-25T10:12:26.012305+0000","last_clean":"2024-04-25T10:12:26.012305+0000","last_became_active":"2024-04-25T10:12:23.247006+0000","last_became_peered":"2024-04-25T10:12:23.247006+0000","last_unstale":"2024-04-25T10:12:26.012305+0000","last_undegraded":"2024-04-25T10:12:26.012305+0000","last_fullsized":"2024-04-25T10:12:26.012305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015823+0000","last_change":"2024-04-25T10:12:23.240214+0000","last_active":"2024-04-25T10:12:26.015823+0000","last_peered":"2024-04-25T10:12:26.015823+0000","last_clean":"2024-04-25T10:12:26.015823+0000","last_became_active":"2024-04-25T10:12:23.239944+0000","last_became_peered":"2024-04-25T10:12:23.239944+0000","last_unstale":"2024-04-25T10:12:26.015823+0000","last_undegraded":"2024-04-25T10:12:26.015823+0000","last_fullsized":"2024-04-25T10:12:26.015823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015556+0000","last_change":"2024-04-25T10:12:24.258236+0000","last_active":"2024-04-25T10:12:26.015556+0000","last_peered":"2024-04-25T10:12:26.015556+0000","last_clean":"2024-04-25T10:12:26.015556+0000","last_became_active":"2024-04-25T10:12:24.256931+0000","last_became_peered":"2024-04-25T10:12:24.256931+0000","last_unstale":"2024-04-25T10:12:26.015556+0000","last_undegraded":"2024-04-25T10:12:26.015556+0000","last_fullsized":"2024-04-25T10:12:26.015556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245733+0000","last_change":"2024-04-25T10:12:23.253660+0000","last_active":"2024-04-25T10:12:25.245733+0000","last_peered":"2024-04-25T10:12:25.245733+0000","last_clean":"2024-04-25T10:12:25.245733+0000","last_became_active":"2024-04-25T10:12:23.253047+0000","last_became_peered":"2024-04-25T10:12:23.253047+0000","last_unstale":"2024-04-25T10:12:25.245733+0000","last_undegraded":"2024-04-25T10:12:25.245733+0000","last_fullsized":"2024-04-25T10:12:25.245733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246967+0000","last_change":"2024-04-25T10:12:24.260980+0000","last_active":"2024-04-25T10:12:25.246967+0000","last_peered":"2024-04-25T10:12:25.246967+0000","last_clean":"2024-04-25T10:12:25.246967+0000","last_became_active":"2024-04-25T10:12:24.260459+0000","last_became_peered":"2024-04-25T10:12:24.260459+0000","last_unstale":"2024-04-25T10:12:25.246967+0000","last_undegraded":"2024-04-25T10:12:25.246967+0000","last_fullsized":"2024-04-25T10:12:25.246967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012372+0000","last_change":"2024-04-25T10:12:23.253025+0000","last_active":"2024-04-25T10:12:26.012372+0000","last_peered":"2024-04-25T10:12:26.012372+0000","last_clean":"2024-04-25T10:12:26.012372+0000","last_became_active":"2024-04-25T10:12:23.251953+0000","last_became_peered":"2024-04-25T10:12:23.251953+0000","last_unstale":"2024-04-25T10:12:26.012372+0000","last_undegraded":"2024-04-25T10:12:26.012372+0000","last_fullsized":"2024-04-25T10:12:26.012372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246256+0000","last_change":"2024-04-25T10:12:24.273962+0000","last_active":"2024-04-25T10:12:25.246256+0000","last_peered":"2024-04-25T10:12:25.246256+0000","last_clean":"2024-04-25T10:12:25.246256+0000","last_became_active":"2024-04-25T10:12:24.273577+0000","last_became_peered":"2024-04-25T10:12:24.273577+0000","last_unstale":"2024-04-25T10:12:25.246256+0000","last_undegraded":"2024-04-25T10:12:25.246256+0000","last_fullsized":"2024-04-25T10:12:25.246256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246360+0000","last_change":"2024-04-25T10:12:23.247652+0000","last_active":"2024-04-25T10:12:25.246360+0000","last_peered":"2024-04-25T10:12:25.246360+0000","last_clean":"2024-04-25T10:12:25.246360+0000","last_became_active":"2024-04-25T10:12:23.243149+0000","last_became_peered":"2024-04-25T10:12:23.243149+0000","last_unstale":"2024-04-25T10:12:25.246360+0000","last_undegraded":"2024-04-25T10:12:25.246360+0000","last_fullsized":"2024-04-25T10:12:25.246360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016262+0000","last_change":"2024-04-25T10:12:24.261695+0000","last_active":"2024-04-25T10:12:26.016262+0000","last_peered":"2024-04-25T10:12:26.016262+0000","last_clean":"2024-04-25T10:12:26.016262+0000","last_became_active":"2024-04-25T10:12:24.261356+0000","last_became_peered":"2024-04-25T10:12:24.261356+0000","last_unstale":"2024-04-25T10:12:26.016262+0000","last_undegraded":"2024-04-25T10:12:26.016262+0000","last_fullsized":"2024-04-25T10:12:26.016262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247330+0000","last_change":"2024-04-25T10:12:23.244711+0000","last_active":"2024-04-25T10:12:25.247330+0000","last_peered":"2024-04-25T10:12:25.247330+0000","last_clean":"2024-04-25T10:12:25.247330+0000","last_became_active":"2024-04-25T10:12:23.244360+0000","last_became_peered":"2024-04-25T10:12:23.244360+0000","last_unstale":"2024-04-25T10:12:25.247330+0000","last_undegraded":"2024-04-25T10:12:25.247330+0000","last_fullsized":"2024-04-25T10:12:25.247330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246749+0000","last_change":"2024-04-25T10:12:24.259965+0000","last_active":"2024-04-25T10:12:25.246749+0000","last_peered":"2024-04-25T10:12:25.246749+0000","last_clean":"2024-04-25T10:12:25.246749+0000","last_became_active":"2024-04-25T10:12:24.249487+0000","last_became_peered":"2024-04-25T10:12:24.249487+0000","last_unstale":"2024-04-25T10:12:25.246749+0000","last_undegraded":"2024-04-25T10:12:25.246749+0000","last_fullsized":"2024-04-25T10:12:25.246749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:12.866317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012035+0000","last_change":"2024-04-25T10:12:23.240603+0000","last_active":"2024-04-25T10:12:26.012035+0000","last_peered":"2024-04-25T10:12:26.012035+0000","last_clean":"2024-04-25T10:12:26.012035+0000","last_became_active":"2024-04-25T10:12:23.240240+0000","last_became_peered":"2024-04-25T10:12:23.240240+0000","last_unstale":"2024-04-25T10:12:26.012035+0000","last_undegraded":"2024-04-25T10:12:26.012035+0000","last_fullsized":"2024-04-25T10:12:26.012035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015129+0000","last_change":"2024-04-25T10:12:24.259015+0000","last_active":"2024-04-25T10:12:26.015129+0000","last_peered":"2024-04-25T10:12:26.015129+0000","last_clean":"2024-04-25T10:12:26.015129+0000","last_became_active":"2024-04-25T10:12:24.258715+0000","last_became_peered":"2024-04-25T10:12:24.258715+0000","last_unstale":"2024-04-25T10:12:26.015129+0000","last_undegraded":"2024-04-25T10:12:26.015129+0000","last_fullsized":"2024-04-25T10:12:26.015129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016578+0000","last_change":"2024-04-25T10:12:23.253767+0000","last_active":"2024-04-25T10:12:26.016578+0000","last_peered":"2024-04-25T10:12:26.016578+0000","last_clean":"2024-04-25T10:12:26.016578+0000","last_became_active":"2024-04-25T10:12:23.253356+0000","last_became_peered":"2024-04-25T10:12:23.253356+0000","last_unstale":"2024-04-25T10:12:26.016578+0000","last_undegraded":"2024-04-25T10:12:26.016578+0000","last_fullsized":"2024-04-25T10:12:26.016578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016055+0000","last_change":"2024-04-25T10:12:24.241735+0000","last_active":"2024-04-25T10:12:26.016055+0000","last_peered":"2024-04-25T10:12:26.016055+0000","last_clean":"2024-04-25T10:12:26.016055+0000","last_became_active":"2024-04-25T10:12:24.241406+0000","last_became_peered":"2024-04-25T10:12:24.241406+0000","last_unstale":"2024-04-25T10:12:26.016055+0000","last_undegraded":"2024-04-25T10:12:26.016055+0000","last_fullsized":"2024-04-25T10:12:26.016055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:27.592446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246379+0000","last_change":"2024-04-25T10:12:23.249072+0000","last_active":"2024-04-25T10:12:25.246379+0000","last_peered":"2024-04-25T10:12:25.246379+0000","last_clean":"2024-04-25T10:12:25.246379+0000","last_became_active":"2024-04-25T10:12:23.248691+0000","last_became_peered":"2024-04-25T10:12:23.248691+0000","last_unstale":"2024-04-25T10:12:25.246379+0000","last_undegraded":"2024-04-25T10:12:25.246379+0000","last_fullsized":"2024-04-25T10:12:25.246379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015097+0000","last_change":"2024-04-25T10:12:24.269693+0000","last_active":"2024-04-25T10:12:26.015097+0000","last_peered":"2024-04-25T10:12:26.015097+0000","last_clean":"2024-04-25T10:12:26.015097+0000","last_became_active":"2024-04-25T10:12:24.269423+0000","last_became_peered":"2024-04-25T10:12:24.269423+0000","last_unstale":"2024-04-25T10:12:26.015097+0000","last_undegraded":"2024-04-25T10:12:26.015097+0000","last_fullsized":"2024-04-25T10:12:26.015097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011686+0000","last_change":"2024-04-25T10:12:23.246678+0000","last_active":"2024-04-25T10:12:26.011686+0000","last_peered":"2024-04-25T10:12:26.011686+0000","last_clean":"2024-04-25T10:12:26.011686+0000","last_became_active":"2024-04-25T10:12:23.246288+0000","last_became_peered":"2024-04-25T10:12:23.246288+0000","last_unstale":"2024-04-25T10:12:26.011686+0000","last_undegraded":"2024-04-25T10:12:26.011686+0000","last_fullsized":"2024-04-25T10:12:26.011686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011782+0000","last_change":"2024-04-25T10:12:24.245870+0000","last_active":"2024-04-25T10:12:26.011782+0000","last_peered":"2024-04-25T10:12:26.011782+0000","last_clean":"2024-04-25T10:12:26.011782+0000","last_became_active":"2024-04-25T10:12:24.244284+0000","last_became_peered":"2024-04-25T10:12:24.244284+0000","last_unstale":"2024-04-25T10:12:26.011782+0000","last_undegraded":"2024-04-25T10:12:26.011782+0000","last_fullsized":"2024-04-25T10:12:26.011782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.871923+0000","last_change":"2024-04-25T10:12:23.232086+0000","last_active":"2024-04-25T10:12:25.871923+0000","last_peered":"2024-04-25T10:12:25.871923+0000","last_clean":"2024-04-25T10:12:25.871923+0000","last_became_active":"2024-04-25T10:12:23.231694+0000","last_became_peered":"2024-04-25T10:12:23.231694+0000","last_unstale":"2024-04-25T10:12:25.871923+0000","last_undegraded":"2024-04-25T10:12:25.871923+0000","last_fullsized":"2024-04-25T10:12:25.871923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014903+0000","last_change":"2024-04-25T10:12:24.252131+0000","last_active":"2024-04-25T10:12:26.014903+0000","last_peered":"2024-04-25T10:12:26.014903+0000","last_clean":"2024-04-25T10:12:26.014903+0000","last_became_active":"2024-04-25T10:12:24.251807+0000","last_became_peered":"2024-04-25T10:12:24.251807+0000","last_unstale":"2024-04-25T10:12:26.014903+0000","last_undegraded":"2024-04-25T10:12:26.014903+0000","last_fullsized":"2024-04-25T10:12:26.014903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246755+0000","last_change":"2024-04-25T10:12:23.243507+0000","last_active":"2024-04-25T10:12:25.246755+0000","last_peered":"2024-04-25T10:12:25.246755+0000","last_clean":"2024-04-25T10:12:25.246755+0000","last_became_active":"2024-04-25T10:12:23.243082+0000","last_became_peered":"2024-04-25T10:12:23.243082+0000","last_unstale":"2024-04-25T10:12:25.246755+0000","last_undegraded":"2024-04-25T10:12:25.246755+0000","last_fullsized":"2024-04-25T10:12:25.246755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:09.593840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015480+0000","last_change":"2024-04-25T10:12:24.270975+0000","last_active":"2024-04-25T10:12:26.015480+0000","last_peered":"2024-04-25T10:12:26.015480+0000","last_clean":"2024-04-25T10:12:26.015480+0000","last_became_active":"2024-04-25T10:12:24.270678+0000","last_became_peered":"2024-04-25T10:12:24.270678+0000","last_unstale":"2024-04-25T10:12:26.015480+0000","last_undegraded":"2024-04-25T10:12:26.015480+0000","last_fullsized":"2024-04-25T10:12:26.015480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872082+0000","last_change":"2024-04-25T10:12:23.242355+0000","last_active":"2024-04-25T10:12:25.872082+0000","last_peered":"2024-04-25T10:12:25.872082+0000","last_clean":"2024-04-25T10:12:25.872082+0000","last_became_active":"2024-04-25T10:12:23.242038+0000","last_became_peered":"2024-04-25T10:12:23.242038+0000","last_unstale":"2024-04-25T10:12:25.872082+0000","last_undegraded":"2024-04-25T10:12:25.872082+0000","last_fullsized":"2024-04-25T10:12:25.872082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015634+0000","last_change":"2024-04-25T10:12:24.270959+0000","last_active":"2024-04-25T10:12:26.015634+0000","last_peered":"2024-04-25T10:12:26.015634+0000","last_clean":"2024-04-25T10:12:26.015634+0000","last_became_active":"2024-04-25T10:12:24.270658+0000","last_became_peered":"2024-04-25T10:12:24.270658+0000","last_unstale":"2024-04-25T10:12:26.015634+0000","last_undegraded":"2024-04-25T10:12:26.015634+0000","last_fullsized":"2024-04-25T10:12:26.015634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247343+0000","last_change":"2024-04-25T10:12:23.255897+0000","last_active":"2024-04-25T10:12:25.247343+0000","last_peered":"2024-04-25T10:12:25.247343+0000","last_clean":"2024-04-25T10:12:25.247343+0000","last_became_active":"2024-04-25T10:12:23.255626+0000","last_became_peered":"2024-04-25T10:12:23.255626+0000","last_unstale":"2024-04-25T10:12:25.247343+0000","last_undegraded":"2024-04-25T10:12:25.247343+0000","last_fullsized":"2024-04-25T10:12:25.247343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:51.408775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246715+0000","last_change":"2024-04-25T10:12:24.262419+0000","last_active":"2024-04-25T10:12:25.246715+0000","last_peered":"2024-04-25T10:12:25.246715+0000","last_clean":"2024-04-25T10:12:25.246715+0000","last_became_active":"2024-04-25T10:12:24.262163+0000","last_became_peered":"2024-04-25T10:12:24.262163+0000","last_unstale":"2024-04-25T10:12:25.246715+0000","last_undegraded":"2024-04-25T10:12:25.246715+0000","last_fullsized":"2024-04-25T10:12:25.246715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872045+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:25.872045+0000","last_peered":"2024-04-25T10:12:25.872045+0000","last_clean":"2024-04-25T10:12:25.872045+0000","last_became_active":"2024-04-25T10:12:23.249437+0000","last_became_peered":"2024-04-25T10:12:23.249437+0000","last_unstale":"2024-04-25T10:12:25.872045+0000","last_undegraded":"2024-04-25T10:12:25.872045+0000","last_fullsized":"2024-04-25T10:12:25.872045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015325+0000","last_change":"2024-04-25T10:12:24.266865+0000","last_active":"2024-04-25T10:12:26.015325+0000","last_peered":"2024-04-25T10:12:26.015325+0000","last_clean":"2024-04-25T10:12:26.015325+0000","last_became_active":"2024-04-25T10:12:24.266595+0000","last_became_peered":"2024-04-25T10:12:24.266595+0000","last_unstale":"2024-04-25T10:12:26.015325+0000","last_undegraded":"2024-04-25T10:12:26.015325+0000","last_fullsized":"2024-04-25T10:12:26.015325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015021+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:12:26.015021+0000","last_peered":"2024-04-25T10:12:26.015021+0000","last_clean":"2024-04-25T10:12:26.015021+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:12:26.015021+0000","last_undegraded":"2024-04-25T10:12:26.015021+0000","last_fullsized":"2024-04-25T10:12:26.015021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245447+0000","last_change":"2024-04-25T10:12:23.236000+0000","last_active":"2024-04-25T10:12:25.245447+0000","last_peered":"2024-04-25T10:12:25.245447+0000","last_clean":"2024-04-25T10:12:25.245447+0000","last_became_active":"2024-04-25T10:12:23.235473+0000","last_became_peered":"2024-04-25T10:12:23.235473+0000","last_unstale":"2024-04-25T10:12:25.245447+0000","last_undegraded":"2024-04-25T10:12:25.245447+0000","last_fullsized":"2024-04-25T10:12:25.245447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016099+0000","last_change":"2024-04-25T10:12:24.269413+0000","last_active":"2024-04-25T10:12:26.016099+0000","last_peered":"2024-04-25T10:12:26.016099+0000","last_clean":"2024-04-25T10:12:26.016099+0000","last_became_active":"2024-04-25T10:12:24.269109+0000","last_became_peered":"2024-04-25T10:12:24.269109+0000","last_unstale":"2024-04-25T10:12:26.016099+0000","last_undegraded":"2024-04-25T10:12:26.016099+0000","last_fullsized":"2024-04-25T10:12:26.016099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014753+0000","last_change":"2024-04-25T10:12:23.251616+0000","last_active":"2024-04-25T10:12:26.014753+0000","last_peered":"2024-04-25T10:12:26.014753+0000","last_clean":"2024-04-25T10:12:26.014753+0000","last_became_active":"2024-04-25T10:12:23.251356+0000","last_became_peered":"2024-04-25T10:12:23.251356+0000","last_unstale":"2024-04-25T10:12:26.014753+0000","last_undegraded":"2024-04-25T10:12:26.014753+0000","last_fullsized":"2024-04-25T10:12:26.014753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015666+0000","last_change":"2024-04-25T10:12:24.252497+0000","last_active":"2024-04-25T10:12:26.015666+0000","last_peered":"2024-04-25T10:12:26.015666+0000","last_clean":"2024-04-25T10:12:26.015666+0000","last_became_active":"2024-04-25T10:12:24.252221+0000","last_became_peered":"2024-04-25T10:12:24.252221+0000","last_unstale":"2024-04-25T10:12:26.015666+0000","last_undegraded":"2024-04-25T10:12:26.015666+0000","last_fullsized":"2024-04-25T10:12:26.015666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:52.114948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872039+0000","last_change":"2024-04-25T10:12:23.235336+0000","last_active":"2024-04-25T10:12:25.872039+0000","last_peered":"2024-04-25T10:12:25.872039+0000","last_clean":"2024-04-25T10:12:25.872039+0000","last_became_active":"2024-04-25T10:12:23.235016+0000","last_became_peered":"2024-04-25T10:12:23.235016+0000","last_unstale":"2024-04-25T10:12:25.872039+0000","last_undegraded":"2024-04-25T10:12:25.872039+0000","last_fullsized":"2024-04-25T10:12:25.872039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246990+0000","last_change":"2024-04-25T10:12:24.264913+0000","last_active":"2024-04-25T10:12:25.246990+0000","last_peered":"2024-04-25T10:12:25.246990+0000","last_clean":"2024-04-25T10:12:25.246990+0000","last_became_active":"2024-04-25T10:12:24.264498+0000","last_became_peered":"2024-04-25T10:12:24.264498+0000","last_unstale":"2024-04-25T10:12:25.246990+0000","last_undegraded":"2024-04-25T10:12:25.246990+0000","last_fullsized":"2024-04-25T10:12:25.246990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246493+0000","last_change":"2024-04-25T10:12:23.241698+0000","last_active":"2024-04-25T10:12:25.246493+0000","last_peered":"2024-04-25T10:12:25.246493+0000","last_clean":"2024-04-25T10:12:25.246493+0000","last_became_active":"2024-04-25T10:12:23.241337+0000","last_became_peered":"2024-04-25T10:12:23.241337+0000","last_unstale":"2024-04-25T10:12:25.246493+0000","last_undegraded":"2024-04-25T10:12:25.246493+0000","last_fullsized":"2024-04-25T10:12:25.246493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246479+0000","last_change":"2024-04-25T10:12:24.249493+0000","last_active":"2024-04-25T10:12:25.246479+0000","last_peered":"2024-04-25T10:12:25.246479+0000","last_clean":"2024-04-25T10:12:25.246479+0000","last_became_active":"2024-04-25T10:12:24.249067+0000","last_became_peered":"2024-04-25T10:12:24.249067+0000","last_unstale":"2024-04-25T10:12:25.246479+0000","last_undegraded":"2024-04-25T10:12:25.246479+0000","last_fullsized":"2024-04-25T10:12:25.246479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.231393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247191+0000","last_change":"2024-04-25T10:12:23.243659+0000","last_active":"2024-04-25T10:12:25.247191+0000","last_peered":"2024-04-25T10:12:25.247191+0000","last_clean":"2024-04-25T10:12:25.247191+0000","last_became_active":"2024-04-25T10:12:23.243098+0000","last_became_peered":"2024-04-25T10:12:23.243098+0000","last_unstale":"2024-04-25T10:12:25.247191+0000","last_undegraded":"2024-04-25T10:12:25.247191+0000","last_fullsized":"2024-04-25T10:12:25.247191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245987+0000","last_change":"2024-04-25T10:12:24.264189+0000","last_active":"2024-04-25T10:12:25.245987+0000","last_peered":"2024-04-25T10:12:25.245987+0000","last_clean":"2024-04-25T10:12:25.245987+0000","last_became_active":"2024-04-25T10:12:24.258438+0000","last_became_peered":"2024-04-25T10:12:24.258438+0000","last_unstale":"2024-04-25T10:12:25.245987+0000","last_undegraded":"2024-04-25T10:12:25.245987+0000","last_fullsized":"2024-04-25T10:12:25.245987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015445+0000","last_change":"2024-04-25T10:12:23.254569+0000","last_active":"2024-04-25T10:12:26.015445+0000","last_peered":"2024-04-25T10:12:26.015445+0000","last_clean":"2024-04-25T10:12:26.015445+0000","last_became_active":"2024-04-25T10:12:23.254228+0000","last_became_peered":"2024-04-25T10:12:23.254228+0000","last_unstale":"2024-04-25T10:12:26.015445+0000","last_undegraded":"2024-04-25T10:12:26.015445+0000","last_fullsized":"2024-04-25T10:12:26.015445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:12.723714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247192+0000","last_change":"2024-04-25T10:12:24.258387+0000","last_active":"2024-04-25T10:12:25.247192+0000","last_peered":"2024-04-25T10:12:25.247192+0000","last_clean":"2024-04-25T10:12:25.247192+0000","last_became_active":"2024-04-25T10:12:24.257160+0000","last_became_peered":"2024-04-25T10:12:24.257160+0000","last_unstale":"2024-04-25T10:12:25.247192+0000","last_undegraded":"2024-04-25T10:12:25.247192+0000","last_fullsized":"2024-04-25T10:12:25.247192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.188443+0000","last_change":"2024-04-25T10:12:23.247450+0000","last_active":"2024-04-25T10:12:26.188443+0000","last_peered":"2024-04-25T10:12:26.188443+0000","last_clean":"2024-04-25T10:12:26.188443+0000","last_became_active":"2024-04-25T10:12:23.247166+0000","last_became_peered":"2024-04-25T10:12:23.247166+0000","last_unstale":"2024-04-25T10:12:26.188443+0000","last_undegraded":"2024-04-25T10:12:26.188443+0000","last_fullsized":"2024-04-25T10:12:26.188443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016516+0000","last_change":"2024-04-25T10:12:24.271046+0000","last_active":"2024-04-25T10:12:26.016516+0000","last_peered":"2024-04-25T10:12:26.016516+0000","last_clean":"2024-04-25T10:12:26.016516+0000","last_became_active":"2024-04-25T10:12:24.270697+0000","last_became_peered":"2024-04-25T10:12:24.270697+0000","last_unstale":"2024-04-25T10:12:26.016516+0000","last_undegraded":"2024-04-25T10:12:26.016516+0000","last_fullsized":"2024-04-25T10:12:26.016516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012514+0000","last_change":"2024-04-25T10:12:23.252034+0000","last_active":"2024-04-25T10:12:26.012514+0000","last_peered":"2024-04-25T10:12:26.012514+0000","last_clean":"2024-04-25T10:12:26.012514+0000","last_became_active":"2024-04-25T10:12:23.251537+0000","last_became_peered":"2024-04-25T10:12:23.251537+0000","last_unstale":"2024-04-25T10:12:26.012514+0000","last_undegraded":"2024-04-25T10:12:26.012514+0000","last_fullsized":"2024-04-25T10:12:26.012514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016121+0000","last_change":"2024-04-25T10:12:24.247147+0000","last_active":"2024-04-25T10:12:26.016121+0000","last_peered":"2024-04-25T10:12:26.016121+0000","last_clean":"2024-04-25T10:12:26.016121+0000","last_became_active":"2024-04-25T10:12:24.246897+0000","last_became_peered":"2024-04-25T10:12:24.246897+0000","last_unstale":"2024-04-25T10:12:26.016121+0000","last_undegraded":"2024-04-25T10:12:26.016121+0000","last_fullsized":"2024-04-25T10:12:26.016121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014793+0000","last_change":"2024-04-25T10:12:23.254396+0000","last_active":"2024-04-25T10:12:26.014793+0000","last_peered":"2024-04-25T10:12:26.014793+0000","last_clean":"2024-04-25T10:12:26.014793+0000","last_became_active":"2024-04-25T10:12:23.254125+0000","last_became_peered":"2024-04-25T10:12:23.254125+0000","last_unstale":"2024-04-25T10:12:26.014793+0000","last_undegraded":"2024-04-25T10:12:26.014793+0000","last_fullsized":"2024-04-25T10:12:26.014793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247014+0000","last_change":"2024-04-25T10:12:24.261214+0000","last_active":"2024-04-25T10:12:25.247014+0000","last_peered":"2024-04-25T10:12:25.247014+0000","last_clean":"2024-04-25T10:12:25.247014+0000","last_became_active":"2024-04-25T10:12:24.260895+0000","last_became_peered":"2024-04-25T10:12:24.260895+0000","last_unstale":"2024-04-25T10:12:25.247014+0000","last_undegraded":"2024-04-25T10:12:25.247014+0000","last_fullsized":"2024-04-25T10:12:25.247014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247067+0000","last_change":"2024-04-25T10:12:23.244316+0000","last_active":"2024-04-25T10:12:25.247067+0000","last_peered":"2024-04-25T10:12:25.247067+0000","last_clean":"2024-04-25T10:12:25.247067+0000","last_became_active":"2024-04-25T10:12:23.243075+0000","last_became_peered":"2024-04-25T10:12:23.243075+0000","last_unstale":"2024-04-25T10:12:25.247067+0000","last_undegraded":"2024-04-25T10:12:25.247067+0000","last_fullsized":"2024-04-25T10:12:25.247067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016532+0000","last_change":"2024-04-25T10:12:24.252644+0000","last_active":"2024-04-25T10:12:26.016532+0000","last_peered":"2024-04-25T10:12:26.016532+0000","last_clean":"2024-04-25T10:12:26.016532+0000","last_became_active":"2024-04-25T10:12:24.252317+0000","last_became_peered":"2024-04-25T10:12:24.252317+0000","last_unstale":"2024-04-25T10:12:26.016532+0000","last_undegraded":"2024-04-25T10:12:26.016532+0000","last_fullsized":"2024-04-25T10:12:26.016532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247152+0000","last_change":"2024-04-25T10:12:23.243515+0000","last_active":"2024-04-25T10:12:25.247152+0000","last_peered":"2024-04-25T10:12:25.247152+0000","last_clean":"2024-04-25T10:12:25.247152+0000","last_became_active":"2024-04-25T10:12:23.242920+0000","last_became_peered":"2024-04-25T10:12:23.242920+0000","last_unstale":"2024-04-25T10:12:25.247152+0000","last_undegraded":"2024-04-25T10:12:25.247152+0000","last_fullsized":"2024-04-25T10:12:25.247152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016097+0000","last_change":"2024-04-25T10:12:24.256479+0000","last_active":"2024-04-25T10:12:26.016097+0000","last_peered":"2024-04-25T10:12:26.016097+0000","last_clean":"2024-04-25T10:12:26.016097+0000","last_became_active":"2024-04-25T10:12:24.256149+0000","last_became_peered":"2024-04-25T10:12:24.256149+0000","last_unstale":"2024-04-25T10:12:26.016097+0000","last_undegraded":"2024-04-25T10:12:26.016097+0000","last_fullsized":"2024-04-25T10:12:26.016097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015287+0000","last_change":"2024-04-25T10:12:23.258511+0000","last_active":"2024-04-25T10:12:26.015287+0000","last_peered":"2024-04-25T10:12:26.015287+0000","last_clean":"2024-04-25T10:12:26.015287+0000","last_became_active":"2024-04-25T10:12:23.258235+0000","last_became_peered":"2024-04-25T10:12:23.258235+0000","last_unstale":"2024-04-25T10:12:26.015287+0000","last_undegraded":"2024-04-25T10:12:26.015287+0000","last_fullsized":"2024-04-25T10:12:26.015287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012230+0000","last_change":"2024-04-25T10:12:24.245339+0000","last_active":"2024-04-25T10:12:26.012230+0000","last_peered":"2024-04-25T10:12:26.012230+0000","last_clean":"2024-04-25T10:12:26.012230+0000","last_became_active":"2024-04-25T10:12:24.241842+0000","last_became_peered":"2024-04-25T10:12:24.241842+0000","last_unstale":"2024-04-25T10:12:26.012230+0000","last_undegraded":"2024-04-25T10:12:26.012230+0000","last_fullsized":"2024-04-25T10:12:26.012230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245817+0000","last_change":"2024-04-25T10:12:24.268837+0000","last_active":"2024-04-25T10:12:25.245817+0000","last_peered":"2024-04-25T10:12:25.245817+0000","last_clean":"2024-04-25T10:12:25.245817+0000","last_became_active":"2024-04-25T10:12:24.268028+0000","last_became_peered":"2024-04-25T10:12:24.268028+0000","last_unstale":"2024-04-25T10:12:25.245817+0000","last_undegraded":"2024-04-25T10:12:25.245817+0000","last_fullsized":"2024-04-25T10:12:25.245817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015045+0000","last_change":"2024-04-25T10:12:23.259889+0000","last_active":"2024-04-25T10:12:26.015045+0000","last_peered":"2024-04-25T10:12:26.015045+0000","last_clean":"2024-04-25T10:12:26.015045+0000","last_became_active":"2024-04-25T10:12:23.259546+0000","last_became_peered":"2024-04-25T10:12:23.259546+0000","last_unstale":"2024-04-25T10:12:26.015045+0000","last_undegraded":"2024-04-25T10:12:26.015045+0000","last_fullsized":"2024-04-25T10:12:26.015045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.267238+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.266952+0000","last_became_peered":"2024-04-25T10:12:24.266952+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015266+0000","last_change":"2024-04-25T10:12:23.258590+0000","last_active":"2024-04-25T10:12:26.015266+0000","last_peered":"2024-04-25T10:12:26.015266+0000","last_clean":"2024-04-25T10:12:26.015266+0000","last_became_active":"2024-04-25T10:12:23.258334+0000","last_became_peered":"2024-04-25T10:12:23.258334+0000","last_unstale":"2024-04-25T10:12:26.015266+0000","last_undegraded":"2024-04-25T10:12:26.015266+0000","last_fullsized":"2024-04-25T10:12:26.015266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016106+0000","last_change":"2024-04-25T10:12:24.261806+0000","last_active":"2024-04-25T10:12:26.016106+0000","last_peered":"2024-04-25T10:12:26.016106+0000","last_clean":"2024-04-25T10:12:26.016106+0000","last_became_active":"2024-04-25T10:12:24.261462+0000","last_became_peered":"2024-04-25T10:12:24.261462+0000","last_unstale":"2024-04-25T10:12:26.016106+0000","last_undegraded":"2024-04-25T10:12:26.016106+0000","last_fullsized":"2024-04-25T10:12:26.016106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016336+0000","last_change":"2024-04-25T10:12:23.246028+0000","last_active":"2024-04-25T10:12:26.016336+0000","last_peered":"2024-04-25T10:12:26.016336+0000","last_clean":"2024-04-25T10:12:26.016336+0000","last_became_active":"2024-04-25T10:12:23.245746+0000","last_became_peered":"2024-04-25T10:12:23.245746+0000","last_unstale":"2024-04-25T10:12:26.016336+0000","last_undegraded":"2024-04-25T10:12:26.016336+0000","last_fullsized":"2024-04-25T10:12:26.016336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016375+0000","last_change":"2024-04-25T10:12:24.265910+0000","last_active":"2024-04-25T10:12:26.016375+0000","last_peered":"2024-04-25T10:12:26.016375+0000","last_clean":"2024-04-25T10:12:26.016375+0000","last_became_active":"2024-04-25T10:12:24.265598+0000","last_became_peered":"2024-04-25T10:12:24.265598+0000","last_unstale":"2024-04-25T10:12:26.016375+0000","last_undegraded":"2024-04-25T10:12:26.016375+0000","last_fullsized":"2024-04-25T10:12:26.016375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016395+0000","last_change":"2024-04-25T10:12:23.252781+0000","last_active":"2024-04-25T10:12:26.016395+0000","last_peered":"2024-04-25T10:12:26.016395+0000","last_clean":"2024-04-25T10:12:26.016395+0000","last_became_active":"2024-04-25T10:12:23.252526+0000","last_became_peered":"2024-04-25T10:12:23.252526+0000","last_unstale":"2024-04-25T10:12:26.016395+0000","last_undegraded":"2024-04-25T10:12:26.016395+0000","last_fullsized":"2024-04-25T10:12:26.016395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:10.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:32.883 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:33.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.052+0000 7f001de00640 1 Processor -- start 2024-04-25T10:12:33.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.052+0000 7f001de00640 1 -- start start 2024-04-25T10:12:33.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f001de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00181550b0 0x7f00180cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f001de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00180cfe70 0x7f001815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f001de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001815b740 0x7f001806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f001de00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0018071050 con 0x7f00181550b0 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f001de00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00180712f0 con 0x7f00180cfe70 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f001de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180715d0 con 0x7f001815b740 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00181550b0 0x7f00180cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00181550b0 0x7f00180cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57386/0 (socket says 172.21.15.87:57386) 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 -- 172.21.15.87:0/1440888320 learned_addr learned my addr 172.21.15.87:0/1440888320 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0017400640 1 --2- 172.21.15.87:0/1440888320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001815b740 0x7f001806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016000640 1 --2- 172.21.15.87:0/1440888320 >> v2:172.21.15.169:3300/0 conn(0x7f00180cfe70 0x7f001815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 -- 172.21.15.87:0/1440888320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001815b740 msgr2=0x7f001806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 --2- 172.21.15.87:0/1440888320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001815b740 0x7f001806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 -- 172.21.15.87:0/1440888320 >> v2:172.21.15.169:3300/0 conn(0x7f00180cfe70 msgr2=0x7f001815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 --2- 172.21.15.87:0/1440888320 >> v2:172.21.15.169:3300/0 conn(0x7f00180cfe70 0x7f001815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 -- 172.21.15.87:0/1440888320 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180718b0 con 0x7f00181550b0 2024-04-25T10:12:33.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0017400640 1 --2- 172.21.15.87:0/1440888320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001815b740 0x7f001806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:33.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016000640 1 --2- 172.21.15.87:0/1440888320 >> v2:172.21.15.169:3300/0 conn(0x7f00180cfe70 0x7f001815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:33.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.056+0000 7f0016a00640 1 --2- 172.21.15.87:0/1440888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00181550b0 0x7f00180cf930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0008009870 tx=0x7f000801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9333d5c01598fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f0015600640 1 -- 172.21.15.87:0/1440888320 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080b0070 con 0x7f00181550b0 2024-04-25T10:12:33.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f0015600640 1 -- 172.21.15.87:0/1440888320 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00080ab600 con 0x7f00181550b0 2024-04-25T10:12:33.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f0015600640 1 -- 172.21.15.87:0/1440888320 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080ab9d0 con 0x7f00181550b0 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- 172.21.15.87:0/1440888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00181550b0 msgr2=0x7f00180cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 --2- 172.21.15.87:0/1440888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00181550b0 0x7f00180cf930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0008009870 tx=0x7f000801c260 comp rx=0 tx=0).stop 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- 172.21.15.87:0/1440888320 shutdown_connections 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 --2- 172.21.15.87:0/1440888320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001815b740 0x7f001806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 --2- 172.21.15.87:0/1440888320 >> v2:172.21.15.169:3300/0 conn(0x7f00180cfe70 0x7f001815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 --2- 172.21.15.87:0/1440888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00181550b0 0x7f00180cf930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- 172.21.15.87:0/1440888320 >> 172.21.15.87:0/1440888320 conn(0x7f0018066b70 msgr2=0x7f0018066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- 172.21.15.87:0/1440888320 shutdown_connections 2024-04-25T10:12:33.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- 172.21.15.87:0/1440888320 wait complete. 2024-04-25T10:12:33.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 Processor -- start 2024-04-25T10:12:33.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- start start 2024-04-25T10:12:33.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00180cfe70 0x7f00180cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00181550b0 0x7f001813c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001815b740 0x7f00180cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b3260 con 0x7f00180cfe70 2024-04-25T10:12:33.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001815bd60 con 0x7f001815b740 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f001de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b33e0 con 0x7f00181550b0 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f0016a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00180cfe70 0x7f00180cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f0016a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00180cfe70 0x7f00180cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57400/0 (socket says 172.21.15.87:57400) 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.060+0000 7f0016a00640 1 -- 172.21.15.87:0/3703565155 learned_addr learned my addr 172.21.15.87:0/3703565155 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f0016a00640 1 -- 172.21.15.87:0/3703565155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00181550b0 msgr2=0x7f001813c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f0016a00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00181550b0 0x7f001813c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f0016a00640 1 -- 172.21.15.87:0/3703565155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001815b740 msgr2=0x7f00180cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f0016a00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001815b740 0x7f00180cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f0016a00640 1 -- 172.21.15.87:0/3703565155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180cc3d0 con 0x7f00180cfe70 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f0016a00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00180cfe70 0x7f00180cbdf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00080ae040 tx=0x7f000801aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f000f400640 1 -- 172.21.15.87:0/3703565155 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080b0040 con 0x7f00180cfe70 2024-04-25T10:12:33.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f000f400640 1 -- 172.21.15.87:0/3703565155 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00080bc070 con 0x7f00180cfe70 2024-04-25T10:12:33.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f000f400640 1 -- 172.21.15.87:0/3703565155 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080b6860 con 0x7f00180cfe70 2024-04-25T10:12:33.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0018153160 con 0x7f00180cfe70 2024-04-25T10:12:33.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0018153720 con 0x7f00180cfe70 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001804ef30 con 0x7f00180cfe70 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f000f400640 1 -- 172.21.15.87:0/3703565155 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00080ab480 con 0x7f00180cfe70 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f000f400640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efff8066f60 0x7efff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f000f400640 1 -- 172.21.15.87:0/3703565155 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f000811f4e0 con 0x7f00180cfe70 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.064+0000 7f0016000640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efff8066f60 0x7efff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.068+0000 7f0016000640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efff8066f60 0x7efff8069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0004006f10 tx=0x7f0004008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.072+0000 7f000f400640 1 -- 172.21.15.87:0/3703565155 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0008016110 con 0x7f00180cfe70 2024-04-25T10:12:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.224+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f001814e7d0 con 0x7efff8066f60 2024-04-25T10:12:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.228+0000 7f000f400640 1 -- 172.21.15.87:0/3703565155 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f001814e7d0 con 0x7efff8066f60 2024-04-25T10:12:33.238 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:33.239 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:12:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efff8066f60 msgr2=0x7efff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efff8066f60 0x7efff8069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0004006f10 tx=0x7f0004008040 comp rx=0 tx=0).stop 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00180cfe70 msgr2=0x7f00180cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00180cfe70 0x7f00180cbdf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00080ae040 tx=0x7f000801aaa0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 shutdown_connections 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efff8066f60 0x7efff8069420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001815b740 0x7f00180cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00181550b0 0x7f001813c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 --2- 172.21.15.87:0/3703565155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00180cfe70 0x7f00180cbdf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 >> 172.21.15.87:0/3703565155 conn(0x7f0018066b70 msgr2=0x7f00180b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 shutdown_connections 2024-04-25T10:12:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.232+0000 7f001de00640 1 -- 172.21.15.87:0/3703565155 wait complete. 2024-04-25T10:12:33.259 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T10:12:33.126614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.900670"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015534+0000","last_change":"2024-04-25T10:12:24.263197+0000","last_active":"2024-04-25T10:12:26.015534+0000","last_peered":"2024-04-25T10:12:26.015534+0000","last_clean":"2024-04-25T10:12:26.015534+0000","last_became_active":"2024-04-25T10:12:24.262932+0000","last_became_peered":"2024-04-25T10:12:24.262932+0000","last_unstale":"2024-04-25T10:12:26.015534+0000","last_undegraded":"2024-04-25T10:12:26.015534+0000","last_fullsized":"2024-04-25T10:12:26.015534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247413+0000","last_change":"2024-04-25T10:12:24.255491+0000","last_active":"2024-04-25T10:12:25.247413+0000","last_peered":"2024-04-25T10:12:25.247413+0000","last_clean":"2024-04-25T10:12:25.247413+0000","last_became_active":"2024-04-25T10:12:24.255325+0000","last_became_peered":"2024-04-25T10:12:24.255325+0000","last_unstale":"2024-04-25T10:12:25.247413+0000","last_undegraded":"2024-04-25T10:12:25.247413+0000","last_fullsized":"2024-04-25T10:12:25.247413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016327+0000","last_change":"2024-04-25T10:12:24.259328+0000","last_active":"2024-04-25T10:12:26.016327+0000","last_peered":"2024-04-25T10:12:26.016327+0000","last_clean":"2024-04-25T10:12:26.016327+0000","last_became_active":"2024-04-25T10:12:24.259065+0000","last_became_peered":"2024-04-25T10:12:24.259065+0000","last_unstale":"2024-04-25T10:12:26.016327+0000","last_undegraded":"2024-04-25T10:12:26.016327+0000","last_fullsized":"2024-04-25T10:12:26.016327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247574+0000","last_change":"2024-04-25T10:12:24.263211+0000","last_active":"2024-04-25T10:12:25.247574+0000","last_peered":"2024-04-25T10:12:25.247574+0000","last_clean":"2024-04-25T10:12:25.247574+0000","last_became_active":"2024-04-25T10:12:24.262896+0000","last_became_peered":"2024-04-25T10:12:24.262896+0000","last_unstale":"2024-04-25T10:12:25.247574+0000","last_undegraded":"2024-04-25T10:12:25.247574+0000","last_fullsized":"2024-04-25T10:12:25.247574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014990+0000","last_change":"2024-04-25T10:12:24.259922+0000","last_active":"2024-04-25T10:12:26.014990+0000","last_peered":"2024-04-25T10:12:26.014990+0000","last_clean":"2024-04-25T10:12:26.014990+0000","last_became_active":"2024-04-25T10:12:24.259657+0000","last_became_peered":"2024-04-25T10:12:24.259657+0000","last_unstale":"2024-04-25T10:12:26.014990+0000","last_undegraded":"2024-04-25T10:12:26.014990+0000","last_fullsized":"2024-04-25T10:12:26.014990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245975+0000","last_change":"2024-04-25T10:12:24.254906+0000","last_active":"2024-04-25T10:12:25.245975+0000","last_peered":"2024-04-25T10:12:25.245975+0000","last_clean":"2024-04-25T10:12:25.245975+0000","last_became_active":"2024-04-25T10:12:24.254592+0000","last_became_peered":"2024-04-25T10:12:24.254592+0000","last_unstale":"2024-04-25T10:12:25.245975+0000","last_undegraded":"2024-04-25T10:12:25.245975+0000","last_fullsized":"2024-04-25T10:12:25.245975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014929+0000","last_change":"2024-04-25T10:12:24.254975+0000","last_active":"2024-04-25T10:12:26.014929+0000","last_peered":"2024-04-25T10:12:26.014929+0000","last_clean":"2024-04-25T10:12:26.014929+0000","last_became_active":"2024-04-25T10:12:24.254639+0000","last_became_peered":"2024-04-25T10:12:24.254639+0000","last_unstale":"2024-04-25T10:12:26.014929+0000","last_undegraded":"2024-04-25T10:12:26.014929+0000","last_fullsized":"2024-04-25T10:12:26.014929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015387+0000","last_change":"2024-04-25T10:12:24.264638+0000","last_active":"2024-04-25T10:12:26.015387+0000","last_peered":"2024-04-25T10:12:26.015387+0000","last_clean":"2024-04-25T10:12:26.015387+0000","last_became_active":"2024-04-25T10:12:24.264376+0000","last_became_peered":"2024-04-25T10:12:24.264376+0000","last_unstale":"2024-04-25T10:12:26.015387+0000","last_undegraded":"2024-04-25T10:12:26.015387+0000","last_fullsized":"2024-04-25T10:12:26.015387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015572+0000","last_change":"2024-04-25T10:12:24.257501+0000","last_active":"2024-04-25T10:12:26.015572+0000","last_peered":"2024-04-25T10:12:26.015572+0000","last_clean":"2024-04-25T10:12:26.015572+0000","last_became_active":"2024-04-25T10:12:24.257199+0000","last_became_peered":"2024-04-25T10:12:24.257199+0000","last_unstale":"2024-04-25T10:12:26.015572+0000","last_undegraded":"2024-04-25T10:12:26.015572+0000","last_fullsized":"2024-04-25T10:12:26.015572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012594+0000","last_change":"2024-04-25T10:12:24.250894+0000","last_active":"2024-04-25T10:12:26.012594+0000","last_peered":"2024-04-25T10:12:26.012594+0000","last_clean":"2024-04-25T10:12:26.012594+0000","last_became_active":"2024-04-25T10:12:24.250581+0000","last_became_peered":"2024-04-25T10:12:24.250581+0000","last_unstale":"2024-04-25T10:12:26.012594+0000","last_undegraded":"2024-04-25T10:12:26.012594+0000","last_fullsized":"2024-04-25T10:12:26.012594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012005+0000","last_change":"2024-04-25T10:12:24.247801+0000","last_active":"2024-04-25T10:12:26.012005+0000","last_peered":"2024-04-25T10:12:26.012005+0000","last_clean":"2024-04-25T10:12:26.012005+0000","last_became_active":"2024-04-25T10:12:24.247485+0000","last_became_peered":"2024-04-25T10:12:24.247485+0000","last_unstale":"2024-04-25T10:12:26.012005+0000","last_undegraded":"2024-04-25T10:12:26.012005+0000","last_fullsized":"2024-04-25T10:12:26.012005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.262042+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.261770+0000","last_became_peered":"2024-04-25T10:12:24.261770+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:36.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015914+0000","last_change":"2024-04-25T10:12:24.263554+0000","last_active":"2024-04-25T10:12:26.015914+0000","last_peered":"2024-04-25T10:12:26.015914+0000","last_clean":"2024-04-25T10:12:26.015914+0000","last_became_active":"2024-04-25T10:12:24.263244+0000","last_became_peered":"2024-04-25T10:12:24.263244+0000","last_unstale":"2024-04-25T10:12:26.015914+0000","last_undegraded":"2024-04-25T10:12:26.015914+0000","last_fullsized":"2024-04-25T10:12:26.015914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015884+0000","last_change":"2024-04-25T10:12:24.246682+0000","last_active":"2024-04-25T10:12:26.015884+0000","last_peered":"2024-04-25T10:12:26.015884+0000","last_clean":"2024-04-25T10:12:26.015884+0000","last_became_active":"2024-04-25T10:12:24.246353+0000","last_became_peered":"2024-04-25T10:12:24.246353+0000","last_unstale":"2024-04-25T10:12:26.015884+0000","last_undegraded":"2024-04-25T10:12:26.015884+0000","last_fullsized":"2024-04-25T10:12:26.015884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:15.421048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246997+0000","last_change":"2024-04-25T10:12:24.265431+0000","last_active":"2024-04-25T10:12:25.246997+0000","last_peered":"2024-04-25T10:12:25.246997+0000","last_clean":"2024-04-25T10:12:25.246997+0000","last_became_active":"2024-04-25T10:12:24.265047+0000","last_became_peered":"2024-04-25T10:12:24.265047+0000","last_unstale":"2024-04-25T10:12:25.246997+0000","last_undegraded":"2024-04-25T10:12:25.246997+0000","last_fullsized":"2024-04-25T10:12:25.246997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247153+0000","last_change":"2024-04-25T10:12:24.248104+0000","last_active":"2024-04-25T10:12:25.247153+0000","last_peered":"2024-04-25T10:12:25.247153+0000","last_clean":"2024-04-25T10:12:25.247153+0000","last_became_active":"2024-04-25T10:12:24.247826+0000","last_became_peered":"2024-04-25T10:12:24.247826+0000","last_unstale":"2024-04-25T10:12:25.247153+0000","last_undegraded":"2024-04-25T10:12:25.247153+0000","last_fullsized":"2024-04-25T10:12:25.247153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.228413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247053+0000","last_change":"2024-04-25T10:12:24.257638+0000","last_active":"2024-04-25T10:12:25.247053+0000","last_peered":"2024-04-25T10:12:25.247053+0000","last_clean":"2024-04-25T10:12:25.247053+0000","last_became_active":"2024-04-25T10:12:24.257332+0000","last_became_peered":"2024-04-25T10:12:24.257332+0000","last_unstale":"2024-04-25T10:12:25.247053+0000","last_undegraded":"2024-04-25T10:12:25.247053+0000","last_fullsized":"2024-04-25T10:12:25.247053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:53.086646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872137+0000","last_change":"2024-04-25T10:12:24.262252+0000","last_active":"2024-04-25T10:12:25.872137+0000","last_peered":"2024-04-25T10:12:25.872137+0000","last_clean":"2024-04-25T10:12:25.872137+0000","last_became_active":"2024-04-25T10:12:24.261871+0000","last_became_peered":"2024-04-25T10:12:24.261871+0000","last_unstale":"2024-04-25T10:12:25.872137+0000","last_undegraded":"2024-04-25T10:12:25.872137+0000","last_fullsized":"2024-04-25T10:12:25.872137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:13.729919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015458+0000","last_change":"2024-04-25T10:12:24.269990+0000","last_active":"2024-04-25T10:12:26.015458+0000","last_peered":"2024-04-25T10:12:26.015458+0000","last_clean":"2024-04-25T10:12:26.015458+0000","last_became_active":"2024-04-25T10:12:24.269396+0000","last_became_peered":"2024-04-25T10:12:24.269396+0000","last_unstale":"2024-04-25T10:12:26.015458+0000","last_undegraded":"2024-04-25T10:12:26.015458+0000","last_fullsized":"2024-04-25T10:12:26.015458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012013+0000","last_change":"2024-04-25T10:12:24.262832+0000","last_active":"2024-04-25T10:12:26.012013+0000","last_peered":"2024-04-25T10:12:26.012013+0000","last_clean":"2024-04-25T10:12:26.012013+0000","last_became_active":"2024-04-25T10:12:24.262090+0000","last_became_peered":"2024-04-25T10:12:24.262090+0000","last_unstale":"2024-04-25T10:12:26.012013+0000","last_undegraded":"2024-04-25T10:12:26.012013+0000","last_fullsized":"2024-04-25T10:12:26.012013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016383+0000","last_change":"2024-04-25T10:12:24.248538+0000","last_active":"2024-04-25T10:12:26.016383+0000","last_peered":"2024-04-25T10:12:26.016383+0000","last_clean":"2024-04-25T10:12:26.016383+0000","last_became_active":"2024-04-25T10:12:24.247653+0000","last_became_peered":"2024-04-25T10:12:24.247653+0000","last_unstale":"2024-04-25T10:12:26.016383+0000","last_undegraded":"2024-04-25T10:12:26.016383+0000","last_fullsized":"2024-04-25T10:12:26.016383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:13.111296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246115+0000","last_change":"2024-04-25T10:12:24.260603+0000","last_active":"2024-04-25T10:12:25.246115+0000","last_peered":"2024-04-25T10:12:25.246115+0000","last_clean":"2024-04-25T10:12:25.246115+0000","last_became_active":"2024-04-25T10:12:24.260288+0000","last_became_peered":"2024-04-25T10:12:24.260288+0000","last_unstale":"2024-04-25T10:12:25.246115+0000","last_undegraded":"2024-04-25T10:12:25.246115+0000","last_fullsized":"2024-04-25T10:12:25.246115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015623+0000","last_change":"2024-04-25T10:12:24.272790+0000","last_active":"2024-04-25T10:12:26.015623+0000","last_peered":"2024-04-25T10:12:26.015623+0000","last_clean":"2024-04-25T10:12:26.015623+0000","last_became_active":"2024-04-25T10:12:24.272587+0000","last_became_peered":"2024-04-25T10:12:24.272587+0000","last_unstale":"2024-04-25T10:12:26.015623+0000","last_undegraded":"2024-04-25T10:12:26.015623+0000","last_fullsized":"2024-04-25T10:12:26.015623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245637+0000","last_change":"2024-04-25T10:12:24.265627+0000","last_active":"2024-04-25T10:12:25.245637+0000","last_peered":"2024-04-25T10:12:25.245637+0000","last_clean":"2024-04-25T10:12:25.245637+0000","last_became_active":"2024-04-25T10:12:24.265274+0000","last_became_peered":"2024-04-25T10:12:24.265274+0000","last_unstale":"2024-04-25T10:12:25.245637+0000","last_undegraded":"2024-04-25T10:12:25.245637+0000","last_fullsized":"2024-04-25T10:12:25.245637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016338+0000","last_change":"2024-04-25T10:12:24.272301+0000","last_active":"2024-04-25T10:12:26.016338+0000","last_peered":"2024-04-25T10:12:26.016338+0000","last_clean":"2024-04-25T10:12:26.016338+0000","last_became_active":"2024-04-25T10:12:24.271766+0000","last_became_peered":"2024-04-25T10:12:24.271766+0000","last_unstale":"2024-04-25T10:12:26.016338+0000","last_undegraded":"2024-04-25T10:12:26.016338+0000","last_fullsized":"2024-04-25T10:12:26.016338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012456+0000","last_change":"2024-04-25T10:12:24.265368+0000","last_active":"2024-04-25T10:12:26.012456+0000","last_peered":"2024-04-25T10:12:26.012456+0000","last_clean":"2024-04-25T10:12:26.012456+0000","last_became_active":"2024-04-25T10:12:24.264950+0000","last_became_peered":"2024-04-25T10:12:24.264950+0000","last_unstale":"2024-04-25T10:12:26.012456+0000","last_undegraded":"2024-04-25T10:12:26.012456+0000","last_fullsized":"2024-04-25T10:12:26.012456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247251+0000","last_change":"2024-04-25T10:12:24.259900+0000","last_active":"2024-04-25T10:12:25.247251+0000","last_peered":"2024-04-25T10:12:25.247251+0000","last_clean":"2024-04-25T10:12:25.247251+0000","last_became_active":"2024-04-25T10:12:24.259574+0000","last_became_peered":"2024-04-25T10:12:24.259574+0000","last_unstale":"2024-04-25T10:12:25.247251+0000","last_undegraded":"2024-04-25T10:12:25.247251+0000","last_fullsized":"2024-04-25T10:12:25.247251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015996+0000","last_change":"2024-04-25T10:12:24.272315+0000","last_active":"2024-04-25T10:12:26.015996+0000","last_peered":"2024-04-25T10:12:26.015996+0000","last_clean":"2024-04-25T10:12:26.015996+0000","last_became_active":"2024-04-25T10:12:24.271843+0000","last_became_peered":"2024-04-25T10:12:24.271843+0000","last_unstale":"2024-04-25T10:12:26.015996+0000","last_undegraded":"2024-04-25T10:12:26.015996+0000","last_fullsized":"2024-04-25T10:12:26.015996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016505+0000","last_change":"2024-04-25T10:12:24.260827+0000","last_active":"2024-04-25T10:12:26.016505+0000","last_peered":"2024-04-25T10:12:26.016505+0000","last_clean":"2024-04-25T10:12:26.016505+0000","last_became_active":"2024-04-25T10:12:24.260436+0000","last_became_peered":"2024-04-25T10:12:24.260436+0000","last_unstale":"2024-04-25T10:12:26.016505+0000","last_undegraded":"2024-04-25T10:12:26.016505+0000","last_fullsized":"2024-04-25T10:12:26.016505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246276+0000","last_change":"2024-04-25T10:12:24.261679+0000","last_active":"2024-04-25T10:12:25.246276+0000","last_peered":"2024-04-25T10:12:25.246276+0000","last_clean":"2024-04-25T10:12:25.246276+0000","last_became_active":"2024-04-25T10:12:24.261420+0000","last_became_peered":"2024-04-25T10:12:24.261420+0000","last_unstale":"2024-04-25T10:12:25.246276+0000","last_undegraded":"2024-04-25T10:12:25.246276+0000","last_fullsized":"2024-04-25T10:12:25.246276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872106+0000","last_change":"2024-04-25T10:12:24.260060+0000","last_active":"2024-04-25T10:12:25.872106+0000","last_peered":"2024-04-25T10:12:25.872106+0000","last_clean":"2024-04-25T10:12:25.872106+0000","last_became_active":"2024-04-25T10:12:24.259332+0000","last_became_peered":"2024-04-25T10:12:24.259332+0000","last_unstale":"2024-04-25T10:12:25.872106+0000","last_undegraded":"2024-04-25T10:12:25.872106+0000","last_fullsized":"2024-04-25T10:12:25.872106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011814+0000","last_change":"2024-04-25T10:12:24.247021+0000","last_active":"2024-04-25T10:12:26.011814+0000","last_peered":"2024-04-25T10:12:26.011814+0000","last_clean":"2024-04-25T10:12:26.011814+0000","last_became_active":"2024-04-25T10:12:24.246699+0000","last_became_peered":"2024-04-25T10:12:24.246699+0000","last_unstale":"2024-04-25T10:12:26.011814+0000","last_undegraded":"2024-04-25T10:12:26.011814+0000","last_fullsized":"2024-04-25T10:12:26.011814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245745+0000","last_change":"2024-04-25T10:12:24.263775+0000","last_active":"2024-04-25T10:12:25.245745+0000","last_peered":"2024-04-25T10:12:25.245745+0000","last_clean":"2024-04-25T10:12:25.245745+0000","last_became_active":"2024-04-25T10:12:24.263373+0000","last_became_peered":"2024-04-25T10:12:24.263373+0000","last_unstale":"2024-04-25T10:12:25.245745+0000","last_undegraded":"2024-04-25T10:12:25.245745+0000","last_fullsized":"2024-04-25T10:12:25.245745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:23.746340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016216+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:26.016216+0000","last_peered":"2024-04-25T10:12:26.016216+0000","last_clean":"2024-04-25T10:12:26.016216+0000","last_became_active":"2024-04-25T10:12:23.249533+0000","last_became_peered":"2024-04-25T10:12:23.249533+0000","last_unstale":"2024-04-25T10:12:26.016216+0000","last_undegraded":"2024-04-25T10:12:26.016216+0000","last_fullsized":"2024-04-25T10:12:26.016216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016549+0000","last_change":"2024-04-25T10:12:24.252703+0000","last_active":"2024-04-25T10:12:26.016549+0000","last_peered":"2024-04-25T10:12:26.016549+0000","last_clean":"2024-04-25T10:12:26.016549+0000","last_became_active":"2024-04-25T10:12:24.252372+0000","last_became_peered":"2024-04-25T10:12:24.252372+0000","last_unstale":"2024-04-25T10:12:26.016549+0000","last_undegraded":"2024-04-25T10:12:26.016549+0000","last_fullsized":"2024-04-25T10:12:26.016549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012305+0000","last_change":"2024-04-25T10:12:23.248077+0000","last_active":"2024-04-25T10:12:26.012305+0000","last_peered":"2024-04-25T10:12:26.012305+0000","last_clean":"2024-04-25T10:12:26.012305+0000","last_became_active":"2024-04-25T10:12:23.247006+0000","last_became_peered":"2024-04-25T10:12:23.247006+0000","last_unstale":"2024-04-25T10:12:26.012305+0000","last_undegraded":"2024-04-25T10:12:26.012305+0000","last_fullsized":"2024-04-25T10:12:26.012305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015823+0000","last_change":"2024-04-25T10:12:23.240214+0000","last_active":"2024-04-25T10:12:26.015823+0000","last_peered":"2024-04-25T10:12:26.015823+0000","last_clean":"2024-04-25T10:12:26.015823+0000","last_became_active":"2024-04-25T10:12:23.239944+0000","last_became_peered":"2024-04-25T10:12:23.239944+0000","last_unstale":"2024-04-25T10:12:26.015823+0000","last_undegraded":"2024-04-25T10:12:26.015823+0000","last_fullsized":"2024-04-25T10:12:26.015823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015556+0000","last_change":"2024-04-25T10:12:24.258236+0000","last_active":"2024-04-25T10:12:26.015556+0000","last_peered":"2024-04-25T10:12:26.015556+0000","last_clean":"2024-04-25T10:12:26.015556+0000","last_became_active":"2024-04-25T10:12:24.256931+0000","last_became_peered":"2024-04-25T10:12:24.256931+0000","last_unstale":"2024-04-25T10:12:26.015556+0000","last_undegraded":"2024-04-25T10:12:26.015556+0000","last_fullsized":"2024-04-25T10:12:26.015556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245733+0000","last_change":"2024-04-25T10:12:23.253660+0000","last_active":"2024-04-25T10:12:25.245733+0000","last_peered":"2024-04-25T10:12:25.245733+0000","last_clean":"2024-04-25T10:12:25.245733+0000","last_became_active":"2024-04-25T10:12:23.253047+0000","last_became_peered":"2024-04-25T10:12:23.253047+0000","last_unstale":"2024-04-25T10:12:25.245733+0000","last_undegraded":"2024-04-25T10:12:25.245733+0000","last_fullsized":"2024-04-25T10:12:25.245733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246967+0000","last_change":"2024-04-25T10:12:24.260980+0000","last_active":"2024-04-25T10:12:25.246967+0000","last_peered":"2024-04-25T10:12:25.246967+0000","last_clean":"2024-04-25T10:12:25.246967+0000","last_became_active":"2024-04-25T10:12:24.260459+0000","last_became_peered":"2024-04-25T10:12:24.260459+0000","last_unstale":"2024-04-25T10:12:25.246967+0000","last_undegraded":"2024-04-25T10:12:25.246967+0000","last_fullsized":"2024-04-25T10:12:25.246967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012372+0000","last_change":"2024-04-25T10:12:23.253025+0000","last_active":"2024-04-25T10:12:26.012372+0000","last_peered":"2024-04-25T10:12:26.012372+0000","last_clean":"2024-04-25T10:12:26.012372+0000","last_became_active":"2024-04-25T10:12:23.251953+0000","last_became_peered":"2024-04-25T10:12:23.251953+0000","last_unstale":"2024-04-25T10:12:26.012372+0000","last_undegraded":"2024-04-25T10:12:26.012372+0000","last_fullsized":"2024-04-25T10:12:26.012372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246256+0000","last_change":"2024-04-25T10:12:24.273962+0000","last_active":"2024-04-25T10:12:25.246256+0000","last_peered":"2024-04-25T10:12:25.246256+0000","last_clean":"2024-04-25T10:12:25.246256+0000","last_became_active":"2024-04-25T10:12:24.273577+0000","last_became_peered":"2024-04-25T10:12:24.273577+0000","last_unstale":"2024-04-25T10:12:25.246256+0000","last_undegraded":"2024-04-25T10:12:25.246256+0000","last_fullsized":"2024-04-25T10:12:25.246256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246360+0000","last_change":"2024-04-25T10:12:23.247652+0000","last_active":"2024-04-25T10:12:25.246360+0000","last_peered":"2024-04-25T10:12:25.246360+0000","last_clean":"2024-04-25T10:12:25.246360+0000","last_became_active":"2024-04-25T10:12:23.243149+0000","last_became_peered":"2024-04-25T10:12:23.243149+0000","last_unstale":"2024-04-25T10:12:25.246360+0000","last_undegraded":"2024-04-25T10:12:25.246360+0000","last_fullsized":"2024-04-25T10:12:25.246360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016262+0000","last_change":"2024-04-25T10:12:24.261695+0000","last_active":"2024-04-25T10:12:26.016262+0000","last_peered":"2024-04-25T10:12:26.016262+0000","last_clean":"2024-04-25T10:12:26.016262+0000","last_became_active":"2024-04-25T10:12:24.261356+0000","last_became_peered":"2024-04-25T10:12:24.261356+0000","last_unstale":"2024-04-25T10:12:26.016262+0000","last_undegraded":"2024-04-25T10:12:26.016262+0000","last_fullsized":"2024-04-25T10:12:26.016262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:13.027068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247330+0000","last_change":"2024-04-25T10:12:23.244711+0000","last_active":"2024-04-25T10:12:25.247330+0000","last_peered":"2024-04-25T10:12:25.247330+0000","last_clean":"2024-04-25T10:12:25.247330+0000","last_became_active":"2024-04-25T10:12:23.244360+0000","last_became_peered":"2024-04-25T10:12:23.244360+0000","last_unstale":"2024-04-25T10:12:25.247330+0000","last_undegraded":"2024-04-25T10:12:25.247330+0000","last_fullsized":"2024-04-25T10:12:25.247330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246749+0000","last_change":"2024-04-25T10:12:24.259965+0000","last_active":"2024-04-25T10:12:25.246749+0000","last_peered":"2024-04-25T10:12:25.246749+0000","last_clean":"2024-04-25T10:12:25.246749+0000","last_became_active":"2024-04-25T10:12:24.249487+0000","last_became_peered":"2024-04-25T10:12:24.249487+0000","last_unstale":"2024-04-25T10:12:25.246749+0000","last_undegraded":"2024-04-25T10:12:25.246749+0000","last_fullsized":"2024-04-25T10:12:25.246749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:12.866317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012035+0000","last_change":"2024-04-25T10:12:23.240603+0000","last_active":"2024-04-25T10:12:26.012035+0000","last_peered":"2024-04-25T10:12:26.012035+0000","last_clean":"2024-04-25T10:12:26.012035+0000","last_became_active":"2024-04-25T10:12:23.240240+0000","last_became_peered":"2024-04-25T10:12:23.240240+0000","last_unstale":"2024-04-25T10:12:26.012035+0000","last_undegraded":"2024-04-25T10:12:26.012035+0000","last_fullsized":"2024-04-25T10:12:26.012035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015129+0000","last_change":"2024-04-25T10:12:24.259015+0000","last_active":"2024-04-25T10:12:26.015129+0000","last_peered":"2024-04-25T10:12:26.015129+0000","last_clean":"2024-04-25T10:12:26.015129+0000","last_became_active":"2024-04-25T10:12:24.258715+0000","last_became_peered":"2024-04-25T10:12:24.258715+0000","last_unstale":"2024-04-25T10:12:26.015129+0000","last_undegraded":"2024-04-25T10:12:26.015129+0000","last_fullsized":"2024-04-25T10:12:26.015129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016578+0000","last_change":"2024-04-25T10:12:23.253767+0000","last_active":"2024-04-25T10:12:26.016578+0000","last_peered":"2024-04-25T10:12:26.016578+0000","last_clean":"2024-04-25T10:12:26.016578+0000","last_became_active":"2024-04-25T10:12:23.253356+0000","last_became_peered":"2024-04-25T10:12:23.253356+0000","last_unstale":"2024-04-25T10:12:26.016578+0000","last_undegraded":"2024-04-25T10:12:26.016578+0000","last_fullsized":"2024-04-25T10:12:26.016578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016055+0000","last_change":"2024-04-25T10:12:24.241735+0000","last_active":"2024-04-25T10:12:26.016055+0000","last_peered":"2024-04-25T10:12:26.016055+0000","last_clean":"2024-04-25T10:12:26.016055+0000","last_became_active":"2024-04-25T10:12:24.241406+0000","last_became_peered":"2024-04-25T10:12:24.241406+0000","last_unstale":"2024-04-25T10:12:26.016055+0000","last_undegraded":"2024-04-25T10:12:26.016055+0000","last_fullsized":"2024-04-25T10:12:26.016055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:27.592446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246379+0000","last_change":"2024-04-25T10:12:23.249072+0000","last_active":"2024-04-25T10:12:25.246379+0000","last_peered":"2024-04-25T10:12:25.246379+0000","last_clean":"2024-04-25T10:12:25.246379+0000","last_became_active":"2024-04-25T10:12:23.248691+0000","last_became_peered":"2024-04-25T10:12:23.248691+0000","last_unstale":"2024-04-25T10:12:25.246379+0000","last_undegraded":"2024-04-25T10:12:25.246379+0000","last_fullsized":"2024-04-25T10:12:25.246379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015097+0000","last_change":"2024-04-25T10:12:24.269693+0000","last_active":"2024-04-25T10:12:26.015097+0000","last_peered":"2024-04-25T10:12:26.015097+0000","last_clean":"2024-04-25T10:12:26.015097+0000","last_became_active":"2024-04-25T10:12:24.269423+0000","last_became_peered":"2024-04-25T10:12:24.269423+0000","last_unstale":"2024-04-25T10:12:26.015097+0000","last_undegraded":"2024-04-25T10:12:26.015097+0000","last_fullsized":"2024-04-25T10:12:26.015097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011686+0000","last_change":"2024-04-25T10:12:23.246678+0000","last_active":"2024-04-25T10:12:26.011686+0000","last_peered":"2024-04-25T10:12:26.011686+0000","last_clean":"2024-04-25T10:12:26.011686+0000","last_became_active":"2024-04-25T10:12:23.246288+0000","last_became_peered":"2024-04-25T10:12:23.246288+0000","last_unstale":"2024-04-25T10:12:26.011686+0000","last_undegraded":"2024-04-25T10:12:26.011686+0000","last_fullsized":"2024-04-25T10:12:26.011686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.011782+0000","last_change":"2024-04-25T10:12:24.245870+0000","last_active":"2024-04-25T10:12:26.011782+0000","last_peered":"2024-04-25T10:12:26.011782+0000","last_clean":"2024-04-25T10:12:26.011782+0000","last_became_active":"2024-04-25T10:12:24.244284+0000","last_became_peered":"2024-04-25T10:12:24.244284+0000","last_unstale":"2024-04-25T10:12:26.011782+0000","last_undegraded":"2024-04-25T10:12:26.011782+0000","last_fullsized":"2024-04-25T10:12:26.011782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.871923+0000","last_change":"2024-04-25T10:12:23.232086+0000","last_active":"2024-04-25T10:12:25.871923+0000","last_peered":"2024-04-25T10:12:25.871923+0000","last_clean":"2024-04-25T10:12:25.871923+0000","last_became_active":"2024-04-25T10:12:23.231694+0000","last_became_peered":"2024-04-25T10:12:23.231694+0000","last_unstale":"2024-04-25T10:12:25.871923+0000","last_undegraded":"2024-04-25T10:12:25.871923+0000","last_fullsized":"2024-04-25T10:12:25.871923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014903+0000","last_change":"2024-04-25T10:12:24.252131+0000","last_active":"2024-04-25T10:12:26.014903+0000","last_peered":"2024-04-25T10:12:26.014903+0000","last_clean":"2024-04-25T10:12:26.014903+0000","last_became_active":"2024-04-25T10:12:24.251807+0000","last_became_peered":"2024-04-25T10:12:24.251807+0000","last_unstale":"2024-04-25T10:12:26.014903+0000","last_undegraded":"2024-04-25T10:12:26.014903+0000","last_fullsized":"2024-04-25T10:12:26.014903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246755+0000","last_change":"2024-04-25T10:12:23.243507+0000","last_active":"2024-04-25T10:12:25.246755+0000","last_peered":"2024-04-25T10:12:25.246755+0000","last_clean":"2024-04-25T10:12:25.246755+0000","last_became_active":"2024-04-25T10:12:23.243082+0000","last_became_peered":"2024-04-25T10:12:23.243082+0000","last_unstale":"2024-04-25T10:12:25.246755+0000","last_undegraded":"2024-04-25T10:12:25.246755+0000","last_fullsized":"2024-04-25T10:12:25.246755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:09.593840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015480+0000","last_change":"2024-04-25T10:12:24.270975+0000","last_active":"2024-04-25T10:12:26.015480+0000","last_peered":"2024-04-25T10:12:26.015480+0000","last_clean":"2024-04-25T10:12:26.015480+0000","last_became_active":"2024-04-25T10:12:24.270678+0000","last_became_peered":"2024-04-25T10:12:24.270678+0000","last_unstale":"2024-04-25T10:12:26.015480+0000","last_undegraded":"2024-04-25T10:12:26.015480+0000","last_fullsized":"2024-04-25T10:12:26.015480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872082+0000","last_change":"2024-04-25T10:12:23.242355+0000","last_active":"2024-04-25T10:12:25.872082+0000","last_peered":"2024-04-25T10:12:25.872082+0000","last_clean":"2024-04-25T10:12:25.872082+0000","last_became_active":"2024-04-25T10:12:23.242038+0000","last_became_peered":"2024-04-25T10:12:23.242038+0000","last_unstale":"2024-04-25T10:12:25.872082+0000","last_undegraded":"2024-04-25T10:12:25.872082+0000","last_fullsized":"2024-04-25T10:12:25.872082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015634+0000","last_change":"2024-04-25T10:12:24.270959+0000","last_active":"2024-04-25T10:12:26.015634+0000","last_peered":"2024-04-25T10:12:26.015634+0000","last_clean":"2024-04-25T10:12:26.015634+0000","last_became_active":"2024-04-25T10:12:24.270658+0000","last_became_peered":"2024-04-25T10:12:24.270658+0000","last_unstale":"2024-04-25T10:12:26.015634+0000","last_undegraded":"2024-04-25T10:12:26.015634+0000","last_fullsized":"2024-04-25T10:12:26.015634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.488152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247343+0000","last_change":"2024-04-25T10:12:23.255897+0000","last_active":"2024-04-25T10:12:25.247343+0000","last_peered":"2024-04-25T10:12:25.247343+0000","last_clean":"2024-04-25T10:12:25.247343+0000","last_became_active":"2024-04-25T10:12:23.255626+0000","last_became_peered":"2024-04-25T10:12:23.255626+0000","last_unstale":"2024-04-25T10:12:25.247343+0000","last_undegraded":"2024-04-25T10:12:25.247343+0000","last_fullsized":"2024-04-25T10:12:25.247343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:51.408775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246715+0000","last_change":"2024-04-25T10:12:24.262419+0000","last_active":"2024-04-25T10:12:25.246715+0000","last_peered":"2024-04-25T10:12:25.246715+0000","last_clean":"2024-04-25T10:12:25.246715+0000","last_became_active":"2024-04-25T10:12:24.262163+0000","last_became_peered":"2024-04-25T10:12:24.262163+0000","last_unstale":"2024-04-25T10:12:25.246715+0000","last_undegraded":"2024-04-25T10:12:25.246715+0000","last_fullsized":"2024-04-25T10:12:25.246715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.632725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872045+0000","last_change":"2024-04-25T10:12:23.249865+0000","last_active":"2024-04-25T10:12:25.872045+0000","last_peered":"2024-04-25T10:12:25.872045+0000","last_clean":"2024-04-25T10:12:25.872045+0000","last_became_active":"2024-04-25T10:12:23.249437+0000","last_became_peered":"2024-04-25T10:12:23.249437+0000","last_unstale":"2024-04-25T10:12:25.872045+0000","last_undegraded":"2024-04-25T10:12:25.872045+0000","last_fullsized":"2024-04-25T10:12:25.872045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015325+0000","last_change":"2024-04-25T10:12:24.266865+0000","last_active":"2024-04-25T10:12:26.015325+0000","last_peered":"2024-04-25T10:12:26.015325+0000","last_clean":"2024-04-25T10:12:26.015325+0000","last_became_active":"2024-04-25T10:12:24.266595+0000","last_became_peered":"2024-04-25T10:12:24.266595+0000","last_unstale":"2024-04-25T10:12:26.015325+0000","last_undegraded":"2024-04-25T10:12:26.015325+0000","last_fullsized":"2024-04-25T10:12:26.015325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:19.100622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015021+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:12:26.015021+0000","last_peered":"2024-04-25T10:12:26.015021+0000","last_clean":"2024-04-25T10:12:26.015021+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:12:26.015021+0000","last_undegraded":"2024-04-25T10:12:26.015021+0000","last_fullsized":"2024-04-25T10:12:26.015021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245447+0000","last_change":"2024-04-25T10:12:23.236000+0000","last_active":"2024-04-25T10:12:25.245447+0000","last_peered":"2024-04-25T10:12:25.245447+0000","last_clean":"2024-04-25T10:12:25.245447+0000","last_became_active":"2024-04-25T10:12:23.235473+0000","last_became_peered":"2024-04-25T10:12:23.235473+0000","last_unstale":"2024-04-25T10:12:25.245447+0000","last_undegraded":"2024-04-25T10:12:25.245447+0000","last_fullsized":"2024-04-25T10:12:25.245447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:08.925143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016099+0000","last_change":"2024-04-25T10:12:24.269413+0000","last_active":"2024-04-25T10:12:26.016099+0000","last_peered":"2024-04-25T10:12:26.016099+0000","last_clean":"2024-04-25T10:12:26.016099+0000","last_became_active":"2024-04-25T10:12:24.269109+0000","last_became_peered":"2024-04-25T10:12:24.269109+0000","last_unstale":"2024-04-25T10:12:26.016099+0000","last_undegraded":"2024-04-25T10:12:26.016099+0000","last_fullsized":"2024-04-25T10:12:26.016099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014753+0000","last_change":"2024-04-25T10:12:23.251616+0000","last_active":"2024-04-25T10:12:26.014753+0000","last_peered":"2024-04-25T10:12:26.014753+0000","last_clean":"2024-04-25T10:12:26.014753+0000","last_became_active":"2024-04-25T10:12:23.251356+0000","last_became_peered":"2024-04-25T10:12:23.251356+0000","last_unstale":"2024-04-25T10:12:26.014753+0000","last_undegraded":"2024-04-25T10:12:26.014753+0000","last_fullsized":"2024-04-25T10:12:26.014753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015666+0000","last_change":"2024-04-25T10:12:24.252497+0000","last_active":"2024-04-25T10:12:26.015666+0000","last_peered":"2024-04-25T10:12:26.015666+0000","last_clean":"2024-04-25T10:12:26.015666+0000","last_became_active":"2024-04-25T10:12:24.252221+0000","last_became_peered":"2024-04-25T10:12:24.252221+0000","last_unstale":"2024-04-25T10:12:26.015666+0000","last_undegraded":"2024-04-25T10:12:26.015666+0000","last_fullsized":"2024-04-25T10:12:26.015666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:52.114948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.872039+0000","last_change":"2024-04-25T10:12:23.235336+0000","last_active":"2024-04-25T10:12:25.872039+0000","last_peered":"2024-04-25T10:12:25.872039+0000","last_clean":"2024-04-25T10:12:25.872039+0000","last_became_active":"2024-04-25T10:12:23.235016+0000","last_became_peered":"2024-04-25T10:12:23.235016+0000","last_unstale":"2024-04-25T10:12:25.872039+0000","last_undegraded":"2024-04-25T10:12:25.872039+0000","last_fullsized":"2024-04-25T10:12:25.872039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246990+0000","last_change":"2024-04-25T10:12:24.264913+0000","last_active":"2024-04-25T10:12:25.246990+0000","last_peered":"2024-04-25T10:12:25.246990+0000","last_clean":"2024-04-25T10:12:25.246990+0000","last_became_active":"2024-04-25T10:12:24.264498+0000","last_became_peered":"2024-04-25T10:12:24.264498+0000","last_unstale":"2024-04-25T10:12:25.246990+0000","last_undegraded":"2024-04-25T10:12:25.246990+0000","last_fullsized":"2024-04-25T10:12:25.246990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246493+0000","last_change":"2024-04-25T10:12:23.241698+0000","last_active":"2024-04-25T10:12:25.246493+0000","last_peered":"2024-04-25T10:12:25.246493+0000","last_clean":"2024-04-25T10:12:25.246493+0000","last_became_active":"2024-04-25T10:12:23.241337+0000","last_became_peered":"2024-04-25T10:12:23.241337+0000","last_unstale":"2024-04-25T10:12:25.246493+0000","last_undegraded":"2024-04-25T10:12:25.246493+0000","last_fullsized":"2024-04-25T10:12:25.246493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246479+0000","last_change":"2024-04-25T10:12:24.249493+0000","last_active":"2024-04-25T10:12:25.246479+0000","last_peered":"2024-04-25T10:12:25.246479+0000","last_clean":"2024-04-25T10:12:25.246479+0000","last_became_active":"2024-04-25T10:12:24.249067+0000","last_became_peered":"2024-04-25T10:12:24.249067+0000","last_unstale":"2024-04-25T10:12:25.246479+0000","last_undegraded":"2024-04-25T10:12:25.246479+0000","last_fullsized":"2024-04-25T10:12:25.246479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.231393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247191+0000","last_change":"2024-04-25T10:12:23.243659+0000","last_active":"2024-04-25T10:12:25.247191+0000","last_peered":"2024-04-25T10:12:25.247191+0000","last_clean":"2024-04-25T10:12:25.247191+0000","last_became_active":"2024-04-25T10:12:23.243098+0000","last_became_peered":"2024-04-25T10:12:23.243098+0000","last_unstale":"2024-04-25T10:12:25.247191+0000","last_undegraded":"2024-04-25T10:12:25.247191+0000","last_fullsized":"2024-04-25T10:12:25.247191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245987+0000","last_change":"2024-04-25T10:12:24.264189+0000","last_active":"2024-04-25T10:12:25.245987+0000","last_peered":"2024-04-25T10:12:25.245987+0000","last_clean":"2024-04-25T10:12:25.245987+0000","last_became_active":"2024-04-25T10:12:24.258438+0000","last_became_peered":"2024-04-25T10:12:24.258438+0000","last_unstale":"2024-04-25T10:12:25.245987+0000","last_undegraded":"2024-04-25T10:12:25.245987+0000","last_fullsized":"2024-04-25T10:12:25.245987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.960679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015445+0000","last_change":"2024-04-25T10:12:23.254569+0000","last_active":"2024-04-25T10:12:26.015445+0000","last_peered":"2024-04-25T10:12:26.015445+0000","last_clean":"2024-04-25T10:12:26.015445+0000","last_became_active":"2024-04-25T10:12:23.254228+0000","last_became_peered":"2024-04-25T10:12:23.254228+0000","last_unstale":"2024-04-25T10:12:26.015445+0000","last_undegraded":"2024-04-25T10:12:26.015445+0000","last_fullsized":"2024-04-25T10:12:26.015445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:12.723714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247192+0000","last_change":"2024-04-25T10:12:24.258387+0000","last_active":"2024-04-25T10:12:25.247192+0000","last_peered":"2024-04-25T10:12:25.247192+0000","last_clean":"2024-04-25T10:12:25.247192+0000","last_became_active":"2024-04-25T10:12:24.257160+0000","last_became_peered":"2024-04-25T10:12:24.257160+0000","last_unstale":"2024-04-25T10:12:25.247192+0000","last_undegraded":"2024-04-25T10:12:25.247192+0000","last_fullsized":"2024-04-25T10:12:25.247192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.188443+0000","last_change":"2024-04-25T10:12:23.247450+0000","last_active":"2024-04-25T10:12:26.188443+0000","last_peered":"2024-04-25T10:12:26.188443+0000","last_clean":"2024-04-25T10:12:26.188443+0000","last_became_active":"2024-04-25T10:12:23.247166+0000","last_became_peered":"2024-04-25T10:12:23.247166+0000","last_unstale":"2024-04-25T10:12:26.188443+0000","last_undegraded":"2024-04-25T10:12:26.188443+0000","last_fullsized":"2024-04-25T10:12:26.188443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016516+0000","last_change":"2024-04-25T10:12:24.271046+0000","last_active":"2024-04-25T10:12:26.016516+0000","last_peered":"2024-04-25T10:12:26.016516+0000","last_clean":"2024-04-25T10:12:26.016516+0000","last_became_active":"2024-04-25T10:12:24.270697+0000","last_became_peered":"2024-04-25T10:12:24.270697+0000","last_unstale":"2024-04-25T10:12:26.016516+0000","last_undegraded":"2024-04-25T10:12:26.016516+0000","last_fullsized":"2024-04-25T10:12:26.016516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:26.500984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012514+0000","last_change":"2024-04-25T10:12:23.252034+0000","last_active":"2024-04-25T10:12:26.012514+0000","last_peered":"2024-04-25T10:12:26.012514+0000","last_clean":"2024-04-25T10:12:26.012514+0000","last_became_active":"2024-04-25T10:12:23.251537+0000","last_became_peered":"2024-04-25T10:12:23.251537+0000","last_unstale":"2024-04-25T10:12:26.012514+0000","last_undegraded":"2024-04-25T10:12:26.012514+0000","last_fullsized":"2024-04-25T10:12:26.012514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016121+0000","last_change":"2024-04-25T10:12:24.247147+0000","last_active":"2024-04-25T10:12:26.016121+0000","last_peered":"2024-04-25T10:12:26.016121+0000","last_clean":"2024-04-25T10:12:26.016121+0000","last_became_active":"2024-04-25T10:12:24.246897+0000","last_became_peered":"2024-04-25T10:12:24.246897+0000","last_unstale":"2024-04-25T10:12:26.016121+0000","last_undegraded":"2024-04-25T10:12:26.016121+0000","last_fullsized":"2024-04-25T10:12:26.016121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:32.750899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.014793+0000","last_change":"2024-04-25T10:12:23.254396+0000","last_active":"2024-04-25T10:12:26.014793+0000","last_peered":"2024-04-25T10:12:26.014793+0000","last_clean":"2024-04-25T10:12:26.014793+0000","last_became_active":"2024-04-25T10:12:23.254125+0000","last_became_peered":"2024-04-25T10:12:23.254125+0000","last_unstale":"2024-04-25T10:12:26.014793+0000","last_undegraded":"2024-04-25T10:12:26.014793+0000","last_fullsized":"2024-04-25T10:12:26.014793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:23.007549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247014+0000","last_change":"2024-04-25T10:12:24.261214+0000","last_active":"2024-04-25T10:12:25.247014+0000","last_peered":"2024-04-25T10:12:25.247014+0000","last_clean":"2024-04-25T10:12:25.247014+0000","last_became_active":"2024-04-25T10:12:24.260895+0000","last_became_peered":"2024-04-25T10:12:24.260895+0000","last_unstale":"2024-04-25T10:12:25.247014+0000","last_undegraded":"2024-04-25T10:12:25.247014+0000","last_fullsized":"2024-04-25T10:12:25.247014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:52.414979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247067+0000","last_change":"2024-04-25T10:12:23.244316+0000","last_active":"2024-04-25T10:12:25.247067+0000","last_peered":"2024-04-25T10:12:25.247067+0000","last_clean":"2024-04-25T10:12:25.247067+0000","last_became_active":"2024-04-25T10:12:23.243075+0000","last_became_peered":"2024-04-25T10:12:23.243075+0000","last_unstale":"2024-04-25T10:12:25.247067+0000","last_undegraded":"2024-04-25T10:12:25.247067+0000","last_fullsized":"2024-04-25T10:12:25.247067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:22.740135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016532+0000","last_change":"2024-04-25T10:12:24.252644+0000","last_active":"2024-04-25T10:12:26.016532+0000","last_peered":"2024-04-25T10:12:26.016532+0000","last_clean":"2024-04-25T10:12:26.016532+0000","last_became_active":"2024-04-25T10:12:24.252317+0000","last_became_peered":"2024-04-25T10:12:24.252317+0000","last_unstale":"2024-04-25T10:12:26.016532+0000","last_undegraded":"2024-04-25T10:12:26.016532+0000","last_fullsized":"2024-04-25T10:12:26.016532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:09.926294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.247152+0000","last_change":"2024-04-25T10:12:23.243515+0000","last_active":"2024-04-25T10:12:25.247152+0000","last_peered":"2024-04-25T10:12:25.247152+0000","last_clean":"2024-04-25T10:12:25.247152+0000","last_became_active":"2024-04-25T10:12:23.242920+0000","last_became_peered":"2024-04-25T10:12:23.242920+0000","last_unstale":"2024-04-25T10:12:25.247152+0000","last_undegraded":"2024-04-25T10:12:25.247152+0000","last_fullsized":"2024-04-25T10:12:25.247152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016097+0000","last_change":"2024-04-25T10:12:24.256479+0000","last_active":"2024-04-25T10:12:26.016097+0000","last_peered":"2024-04-25T10:12:26.016097+0000","last_clean":"2024-04-25T10:12:26.016097+0000","last_became_active":"2024-04-25T10:12:24.256149+0000","last_became_peered":"2024-04-25T10:12:24.256149+0000","last_unstale":"2024-04-25T10:12:26.016097+0000","last_undegraded":"2024-04-25T10:12:26.016097+0000","last_fullsized":"2024-04-25T10:12:26.016097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015287+0000","last_change":"2024-04-25T10:12:23.258511+0000","last_active":"2024-04-25T10:12:26.015287+0000","last_peered":"2024-04-25T10:12:26.015287+0000","last_clean":"2024-04-25T10:12:26.015287+0000","last_became_active":"2024-04-25T10:12:23.258235+0000","last_became_peered":"2024-04-25T10:12:23.258235+0000","last_unstale":"2024-04-25T10:12:26.015287+0000","last_undegraded":"2024-04-25T10:12:26.015287+0000","last_fullsized":"2024-04-25T10:12:26.015287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.012230+0000","last_change":"2024-04-25T10:12:24.245339+0000","last_active":"2024-04-25T10:12:26.012230+0000","last_peered":"2024-04-25T10:12:26.012230+0000","last_clean":"2024-04-25T10:12:26.012230+0000","last_became_active":"2024-04-25T10:12:24.241842+0000","last_became_peered":"2024-04-25T10:12:24.241842+0000","last_unstale":"2024-04-25T10:12:26.012230+0000","last_undegraded":"2024-04-25T10:12:26.012230+0000","last_fullsized":"2024-04-25T10:12:26.012230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:23.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.245817+0000","last_change":"2024-04-25T10:12:24.268837+0000","last_active":"2024-04-25T10:12:25.245817+0000","last_peered":"2024-04-25T10:12:25.245817+0000","last_clean":"2024-04-25T10:12:25.245817+0000","last_became_active":"2024-04-25T10:12:24.268028+0000","last_became_peered":"2024-04-25T10:12:24.268028+0000","last_unstale":"2024-04-25T10:12:25.245817+0000","last_undegraded":"2024-04-25T10:12:25.245817+0000","last_fullsized":"2024-04-25T10:12:25.245817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:28.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015045+0000","last_change":"2024-04-25T10:12:23.259889+0000","last_active":"2024-04-25T10:12:26.015045+0000","last_peered":"2024-04-25T10:12:26.015045+0000","last_clean":"2024-04-25T10:12:26.015045+0000","last_became_active":"2024-04-25T10:12:23.259546+0000","last_became_peered":"2024-04-25T10:12:23.259546+0000","last_unstale":"2024-04-25T10:12:26.015045+0000","last_undegraded":"2024-04-25T10:12:26.015045+0000","last_fullsized":"2024-04-25T10:12:26.015045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:35.580429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:25.246566+0000","last_change":"2024-04-25T10:12:24.267238+0000","last_active":"2024-04-25T10:12:25.246566+0000","last_peered":"2024-04-25T10:12:25.246566+0000","last_clean":"2024-04-25T10:12:25.246566+0000","last_became_active":"2024-04-25T10:12:24.266952+0000","last_became_peered":"2024-04-25T10:12:24.266952+0000","last_unstale":"2024-04-25T10:12:25.246566+0000","last_undegraded":"2024-04-25T10:12:25.246566+0000","last_fullsized":"2024-04-25T10:12:25.246566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:33.842758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.015266+0000","last_change":"2024-04-25T10:12:23.258590+0000","last_active":"2024-04-25T10:12:26.015266+0000","last_peered":"2024-04-25T10:12:26.015266+0000","last_clean":"2024-04-25T10:12:26.015266+0000","last_became_active":"2024-04-25T10:12:23.258334+0000","last_became_peered":"2024-04-25T10:12:23.258334+0000","last_unstale":"2024-04-25T10:12:26.015266+0000","last_undegraded":"2024-04-25T10:12:26.015266+0000","last_fullsized":"2024-04-25T10:12:26.015266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.860112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016106+0000","last_change":"2024-04-25T10:12:24.261806+0000","last_active":"2024-04-25T10:12:26.016106+0000","last_peered":"2024-04-25T10:12:26.016106+0000","last_clean":"2024-04-25T10:12:26.016106+0000","last_became_active":"2024-04-25T10:12:24.261462+0000","last_became_peered":"2024-04-25T10:12:24.261462+0000","last_unstale":"2024-04-25T10:12:26.016106+0000","last_undegraded":"2024-04-25T10:12:26.016106+0000","last_fullsized":"2024-04-25T10:12:26.016106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:37.692318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016336+0000","last_change":"2024-04-25T10:12:23.246028+0000","last_active":"2024-04-25T10:12:26.016336+0000","last_peered":"2024-04-25T10:12:26.016336+0000","last_clean":"2024-04-25T10:12:26.016336+0000","last_became_active":"2024-04-25T10:12:23.245746+0000","last_became_peered":"2024-04-25T10:12:23.245746+0000","last_unstale":"2024-04-25T10:12:26.016336+0000","last_undegraded":"2024-04-25T10:12:26.016336+0000","last_fullsized":"2024-04-25T10:12:26.016336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.477485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016375+0000","last_change":"2024-04-25T10:12:24.265910+0000","last_active":"2024-04-25T10:12:26.016375+0000","last_peered":"2024-04-25T10:12:26.016375+0000","last_clean":"2024-04-25T10:12:26.016375+0000","last_became_active":"2024-04-25T10:12:24.265598+0000","last_became_peered":"2024-04-25T10:12:24.265598+0000","last_unstale":"2024-04-25T10:12:26.016375+0000","last_undegraded":"2024-04-25T10:12:26.016375+0000","last_fullsized":"2024-04-25T10:12:26.016375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:23.214389+0000","last_clean_scrub_stamp":"2024-04-25T10:12:23.214389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:00.147765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:12:26.016395+0000","last_change":"2024-04-25T10:12:23.252781+0000","last_active":"2024-04-25T10:12:26.016395+0000","last_peered":"2024-04-25T10:12:26.016395+0000","last_clean":"2024-04-25T10:12:26.016395+0000","last_became_active":"2024-04-25T10:12:23.252526+0000","last_became_peered":"2024-04-25T10:12:23.252526+0000","last_unstale":"2024-04-25T10:12:26.016395+0000","last_undegraded":"2024-04-25T10:12:26.016395+0000","last_fullsized":"2024-04-25T10:12:26.016395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:22.208185+0000","last_clean_scrub_stamp":"2024-04-25T10:12:22.208185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:10.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:33.263 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:12:33.263 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:12:33.263 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:12:33.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:12:33.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5f3000640 1 Processor -- start 2024-04-25T10:12:33.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5f3000640 1 -- start start 2024-04-25T10:12:33.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5f3000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5ec0d0040 0x7fb5ec15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5f3000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec071050 con 0x7fb5ec1550b0 2024-04-25T10:12:33.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5f3000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec15d040 con 0x7fb5ec15b740 2024-04-25T10:12:33.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5f3000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec0712f0 con 0x7fb5ec0d0040 2024-04-25T10:12:33.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5ebe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb5ec1550b0 0x7fb5ec0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60654/0 (socket says 172.21.15.87:60654) 2024-04-25T10:12:33.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.432+0000 7fb5ebe00640 1 -- 172.21.15.87:0/3419982458 learned_addr learned my addr 172.21.15.87:0/3419982458 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:33.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467785313 0 0) 0x7fb5ec071050 con 0x7fb5ec1550b0 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5e3400640 1 --2- 172.21.15.87:0/3419982458 >> v2:172.21.15.169:3300/0 conn(0x7fb5ec0d0040 0x7fb5ec15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d0005740 con 0x7fb5ec1550b0 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4118205404 0 0) 0x7fb5d0005740 con 0x7fb5ec1550b0 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 >> v2:172.21.15.169:3300/0 conn(0x7fb5ec0d0040 msgr2=0x7fb5ec15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 --2- 172.21.15.87:0/3419982458 >> v2:172.21.15.169:3300/0 conn(0x7fb5ec0d0040 0x7fb5ec15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 >> v1:172.21.15.169:6789/0 conn(0x7fb5ec15b740 legacy=0x7fb5ec0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0715d0 con 0x7fb5ec1550b0 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb5dc001680 con 0x7fb5ec1550b0 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 >> v1:172.21.15.87:6789/0 conn(0x7fb5ec1550b0 legacy=0x7fb5ec0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d0005bd0 0x7fb5d0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5d0006a20 0x7fb5d0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d00073e0 0x7fb5d000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0712f0 con 0x7fb5d00073e0 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d000e280 con 0x7fb5d0006a20 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d000e480 con 0x7fb5d0005bd0 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5f1000640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d00073e0 0x7fb5d000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5ebe00640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d0005bd0 0x7fb5d0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5e3400640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5d0006a20 0x7fb5d0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5f1000640 1 -- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d0005bd0 msgr2=0x7fb5d0005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5f1000640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d0005bd0 0x7fb5d0005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5f1000640 1 -- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5d0006a20 msgr2=0x7fb5d0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5f1000640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5d0006a20 0x7fb5d0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.436+0000 7fb5f1000640 1 -- 172.21.15.87:0/3419982458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0005740 con 0x7fb5d00073e0 2024-04-25T10:12:33.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f1000640 1 -- 172.21.15.87:0/3419982458 reap_dead start 2024-04-25T10:12:33.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f1000640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d00073e0 0x7fb5d000dbe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4001e70 tx=0x7fb5e401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d2c18709f4bd74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4001a90 con 0x7fb5d00073e0 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4003ba0 con 0x7fb5d00073e0 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5eb400640 1 -- 172.21.15.87:0/3419982458 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4005020 con 0x7fb5d00073e0 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- 172.21.15.87:0/3419982458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d00073e0 msgr2=0x7fb5d000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d00073e0 0x7fb5d000dbe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4001e70 tx=0x7fb5e401c780 comp rx=0 tx=0).stop 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- 172.21.15.87:0/3419982458 shutdown_connections 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3419982458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d00073e0 0x7fb5d000dbe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- 172.21.15.87:0/3419982458 >> 172.21.15.87:0/3419982458 conn(0x7fb5ec066b70 msgr2=0x7fb5ec066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- 172.21.15.87:0/3419982458 shutdown_connections 2024-04-25T10:12:33.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- 172.21.15.87:0/3419982458 wait complete. 2024-04-25T10:12:33.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 Processor -- start 2024-04-25T10:12:33.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- start start 2024-04-25T10:12:33.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d00073e0 0x7fb5ec0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5ec146d70 0x7fb5ec1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5ec157970 0x7fb5ec157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0720b0 con 0x7fb5ec157970 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec071b90 con 0x7fb5ec146d70 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.440+0000 7fb5f3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec15bb90 con 0x7fb5d00073e0 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5ec157970 0x7fb5ec157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5ec157970 0x7fb5ec157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57420/0 (socket says 172.21.15.87:57420) 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 -- 172.21.15.87:0/3317475185 learned_addr learned my addr 172.21.15.87:0/3317475185 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 -- 172.21.15.87:0/3317475185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d00073e0 msgr2=0x7fb5ec0b1710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d00073e0 0x7fb5ec0b1710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 -- 172.21.15.87:0/3317475185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5ec146d70 msgr2=0x7fb5ec1471f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5ec146d70 0x7fb5ec1471f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 -- 172.21.15.87:0/3317475185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec1ef2f0 con 0x7fb5ec157970 2024-04-25T10:12:33.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f1000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5ec157970 0x7fb5ec157df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb5e401c5b0 tx=0x7fb5d0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5e8c00640 1 -- 172.21.15.87:0/3317475185 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4001a90 con 0x7fb5ec157970 2024-04-25T10:12:33.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5e8c00640 1 -- 172.21.15.87:0/3317475185 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4005830 con 0x7fb5ec157970 2024-04-25T10:12:33.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec1ef5d0 con 0x7fb5ec157970 2024-04-25T10:12:33.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5e8c00640 1 -- 172.21.15.87:0/3317475185 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4006da0 con 0x7fb5ec157970 2024-04-25T10:12:33.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec1efb90 con 0x7fb5ec157970 2024-04-25T10:12:33.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec04ef30 con 0x7fb5ec157970 2024-04-25T10:12:33.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.444+0000 7fb5e8c00640 1 -- 172.21.15.87:0/3317475185 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5e40a6690 con 0x7fb5ec157970 2024-04-25T10:12:33.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.448+0000 7fb5e8c00640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5c8066f90 0x7fb5c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.448+0000 7fb5e8c00640 1 -- 172.21.15.87:0/3317475185 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5e411e9c0 con 0x7fb5ec157970 2024-04-25T10:12:33.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.448+0000 7fb5ebe00640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5c8066f90 0x7fb5c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.448+0000 7fb5ebe00640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5c8066f90 0x7fb5c8069450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0011f0 tx=0x7fb5dc001160 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.452+0000 7fb5e8c00640 1 -- 172.21.15.87:0/3317475185 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5e40b7060 con 0x7fb5ec157970 2024-04-25T10:12:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.648+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb5ec0b3670 con 0x7fb5ec157970 2024-04-25T10:12:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.648+0000 7fb5e8c00640 1 -- 172.21.15.87:0/3317475185 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb5e4016490 con 0x7fb5ec157970 2024-04-25T10:12:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:33.654 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:12:33.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5c8066f90 msgr2=0x7fb5c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5c8066f90 0x7fb5c8069450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0011f0 tx=0x7fb5dc001160 comp rx=0 tx=0).stop 2024-04-25T10:12:33.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5ec157970 msgr2=0x7fb5ec157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5ec157970 0x7fb5ec157df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb5e401c5b0 tx=0x7fb5d0007a50 comp rx=0 tx=0).stop 2024-04-25T10:12:33.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 shutdown_connections 2024-04-25T10:12:33.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5c8066f90 0x7fb5c8069450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5ec157970 0x7fb5ec157df0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5ec146d70 0x7fb5ec1471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 --2- 172.21.15.87:0/3317475185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5d00073e0 0x7fb5ec0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 >> 172.21.15.87:0/3317475185 conn(0x7fb5ec066b70 msgr2=0x7fb5ec0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 shutdown_connections 2024-04-25T10:12:33.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.652+0000 7fb5f3000640 1 -- 172.21.15.87:0/3317475185 wait complete. 2024-04-25T10:12:33.674 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:12:33.675 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:33.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f5034e00640 1 Processor -- start 2024-04-25T10:12:33.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f5034e00640 1 -- start start 2024-04-25T10:12:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f5034e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50300cda10 0x7f503006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f5034e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50300b2770 con 0x7f50300cf930 2024-04-25T10:12:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f5034e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503006c670 con 0x7f503015bc40 2024-04-25T10:12:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f5034e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503006c950 con 0x7f50300cda10 2024-04-25T10:12:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502e800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f50300cf930 0x7f50300b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60668/0 (socket says 172.21.15.87:60668) 2024-04-25T10:12:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50300cda10 0x7f503006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502e800640 1 -- 172.21.15.87:0/993583130 learned_addr learned my addr 172.21.15.87:0/993583130 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502ca00640 1 -- 172.21.15.87:0/993583130 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261941777 0 0) 0x7f50300b2770 con 0x7f50300cf930 2024-04-25T10:12:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502ca00640 1 -- 172.21.15.87:0/993583130 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5014003530 con 0x7f50300cf930 2024-04-25T10:12:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502d400640 1 -- 172.21.15.87:0/993583130 >> v1:172.21.15.169:6789/0 conn(0x7f503015bc40 legacy=0x7f50300b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502ca00640 1 -- 172.21.15.87:0/993583130 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269726797 0 0) 0x7f503006c670 con 0x7f503015bc40 2024-04-25T10:12:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502d400640 1 -- 172.21.15.87:0/993583130 >> v1:172.21.15.87:6789/0 conn(0x7f50300cf930 legacy=0x7f50300b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.804+0000 7f502d400640 1 -- 172.21.15.87:0/993583130 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503006cc30 con 0x7f50300cda10 2024-04-25T10:12:33.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f502d400640 1 --2- 172.21.15.87:0/993583130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50300cda10 0x7f503006c090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f501c0097c0 tx=0x7f501c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9439bb4fe38c78b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f502ca00640 1 -- 172.21.15.87:0/993583130 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c0b0070 con 0x7f50300cda10 2024-04-25T10:12:33.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f502ca00640 1 -- 172.21.15.87:0/993583130 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c0ab600 con 0x7f50300cda10 2024-04-25T10:12:33.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f502ca00640 1 -- 172.21.15.87:0/993583130 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c0ab9b0 con 0x7f50300cda10 2024-04-25T10:12:33.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f5034e00640 1 -- 172.21.15.87:0/993583130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50300cda10 msgr2=0x7f503006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f5034e00640 1 --2- 172.21.15.87:0/993583130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50300cda10 0x7f503006c090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f501c0097c0 tx=0x7f501c01c410 comp rx=0 tx=0).stop 2024-04-25T10:12:33.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f5034e00640 1 -- 172.21.15.87:0/993583130 shutdown_connections 2024-04-25T10:12:33.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f5034e00640 1 --2- 172.21.15.87:0/993583130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50300cda10 0x7f503006c090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f5034e00640 1 -- 172.21.15.87:0/993583130 >> 172.21.15.87:0/993583130 conn(0x7f5030066b70 msgr2=0x7f5030066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f5034e00640 1 -- 172.21.15.87:0/993583130 shutdown_connections 2024-04-25T10:12:33.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.808+0000 7f5034e00640 1 -- 172.21.15.87:0/993583130 wait complete. 2024-04-25T10:12:33.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 Processor -- start 2024-04-25T10:12:33.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 -- start start 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50300cda10 0x7f5030071b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50300cf930 0x7f503014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f503015bc40 0x7f503015d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5030169d70 con 0x7f503015bc40 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5030169bf0 con 0x7f50300cda10 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5030169ef0 con 0x7f50300cf930 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f503015bc40 0x7f503015d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f503015bc40 0x7f503015d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57442/0 (socket says 172.21.15.87:57442) 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 -- 172.21.15.87:0/4254150634 learned_addr learned my addr 172.21.15.87:0/4254150634 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502d400640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50300cf930 0x7f503014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502de00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50300cda10 0x7f5030071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 -- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50300cf930 msgr2=0x7f503014b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50300cf930 0x7f503014b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 -- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50300cda10 msgr2=0x7f5030071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50300cda10 0x7f5030071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 -- 172.21.15.87:0/4254150634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5030070ac0 con 0x7f503015bc40 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502de00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50300cda10 0x7f5030071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f502e800640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f503015bc40 0x7f503015d290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f502800cb70 tx=0x7f502800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5026000640 1 -- 172.21.15.87:0/4254150634 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028017070 con 0x7f503015bc40 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5026000640 1 -- 172.21.15.87:0/4254150634 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502800ad40 con 0x7f503015bc40 2024-04-25T10:12:33.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5026000640 1 -- 172.21.15.87:0/4254150634 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028012620 con 0x7f503015bc40 2024-04-25T10:12:33.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5030070da0 con 0x7f503015bc40 2024-04-25T10:12:33.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.812+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5030071360 con 0x7f503015bc40 2024-04-25T10:12:33.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.816+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503004ef30 con 0x7f503015bc40 2024-04-25T10:12:33.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.816+0000 7f5026000640 1 -- 172.21.15.87:0/4254150634 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f502801a070 con 0x7f503015bc40 2024-04-25T10:12:33.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.816+0000 7f5026000640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5010066f60 0x7f5010069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.816+0000 7f5026000640 1 -- 172.21.15.87:0/4254150634 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5028084e50 con 0x7f503015bc40 2024-04-25T10:12:33.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.816+0000 7f502de00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5010066f60 0x7f5010069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.816+0000 7f502de00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5010066f60 0x7f5010069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50300cbf30 tx=0x7f5020009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.820+0000 7f5026000640 1 -- 172.21.15.87:0/4254150634 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5028055e60 con 0x7f503015bc40 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.976+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f503006d960 con 0x7f503015bc40 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.976+0000 7f5026000640 1 -- 172.21.15.87:0/4254150634 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f5028055760 con 0x7f503015bc40 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5010066f60 msgr2=0x7f5010069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5010066f60 0x7f5010069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50300cbf30 tx=0x7f5020009040 comp rx=0 tx=0).stop 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f503015bc40 msgr2=0x7f503015d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f503015bc40 0x7f503015d290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f502800cb70 tx=0x7f502800a430 comp rx=0 tx=0).stop 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 shutdown_connections 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5010066f60 0x7f5010069420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f503015bc40 0x7f503015d290 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50300cf930 0x7f503014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 --2- 172.21.15.87:0/4254150634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50300cda10 0x7f5030071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 >> 172.21.15.87:0/4254150634 conn(0x7f5030066b70 msgr2=0x7f5030066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 shutdown_connections 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:33.980+0000 7f5034e00640 1 -- 172.21.15.87:0/4254150634 wait complete. 2024-04-25T10:12:34.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:34.004 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:34.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.136+0000 7f54a3800640 1 Processor -- start 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.136+0000 7f54a3800640 1 -- start start 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.136+0000 7f54a3800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f549c0d0d00 0x7f549c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.136+0000 7f54a3800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f549c0cc200 0x7f549c162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f54a3800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0cdab0 con 0x7f549c1626c0 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f54a3800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c0cd210 con 0x7f549c0cc200 2024-04-25T10:12:34.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f54a3800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c0cd4f0 con 0x7f549c0d0d00 2024-04-25T10:12:34.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f549c0cc200 0x7f549c162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f54a1800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f549c1626c0 0x7f549c06cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60678/0 (socket says 172.21.15.87:60678) 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f54a1800640 1 -- 172.21.15.87:0/2496371020 learned_addr learned my addr 172.21.15.87:0/2496371020 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f54a0e00640 1 --2- 172.21.15.87:0/2496371020 >> v2:172.21.15.169:3300/0 conn(0x7f549c0d0d00 0x7f549c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549be00640 1 -- 172.21.15.87:0/2496371020 >> v2:172.21.15.169:3300/0 conn(0x7f549c0d0d00 msgr2=0x7f549c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549be00640 1 --2- 172.21.15.87:0/2496371020 >> v2:172.21.15.169:3300/0 conn(0x7f549c0d0d00 0x7f549c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549be00640 1 -- 172.21.15.87:0/2496371020 >> v1:172.21.15.87:6789/0 conn(0x7f549c1626c0 legacy=0x7f549c06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549be00640 1 -- 172.21.15.87:0/2496371020 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0cd7d0 con 0x7f549c0cc200 2024-04-25T10:12:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549b400640 1 -- 172.21.15.87:0/2496371020 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909567742 0 0) 0x7f549c0cdab0 con 0x7f549c1626c0 2024-04-25T10:12:34.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549be00640 1 --2- 172.21.15.87:0/2496371020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0cc200 0x7f549c162180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f549c0cdd40 tx=0x7f548801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9fb368b2fc1c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549b400640 1 -- 172.21.15.87:0/2496371020 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880b0070 con 0x7f549c0cc200 2024-04-25T10:12:34.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.140+0000 7f549b400640 1 -- 172.21.15.87:0/2496371020 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54880ab600 con 0x7f549c0cc200 2024-04-25T10:12:34.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549b400640 1 -- 172.21.15.87:0/2496371020 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880ab9d0 con 0x7f549c0cc200 2024-04-25T10:12:34.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- 172.21.15.87:0/2496371020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0cc200 msgr2=0x7f549c162180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 --2- 172.21.15.87:0/2496371020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0cc200 0x7f549c162180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f549c0cdd40 tx=0x7f548801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- 172.21.15.87:0/2496371020 shutdown_connections 2024-04-25T10:12:34.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 --2- 172.21.15.87:0/2496371020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0cc200 0x7f549c162180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 --2- 172.21.15.87:0/2496371020 >> v2:172.21.15.169:3300/0 conn(0x7f549c0d0d00 0x7f549c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- 172.21.15.87:0/2496371020 >> 172.21.15.87:0/2496371020 conn(0x7f549c072290 msgr2=0x7f549c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- 172.21.15.87:0/2496371020 shutdown_connections 2024-04-25T10:12:34.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- 172.21.15.87:0/2496371020 wait complete. 2024-04-25T10:12:34.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 Processor -- start 2024-04-25T10:12:34.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- start start 2024-04-25T10:12:34.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549c0cc200 0x7f549c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0d0d00 0x7f549c0987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f549c1626c0 0x7f549c098d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c15e040 con 0x7f549c0d0d00 2024-04-25T10:12:34.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c06d8a0 con 0x7f549c1626c0 2024-04-25T10:12:34.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a3800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c15e1c0 con 0x7f549c0cc200 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0d0d00 0x7f549c0987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0d0d00 0x7f549c0987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57474/0 (socket says 172.21.15.87:57474) 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 -- 172.21.15.87:0/3331528629 learned_addr learned my addr 172.21.15.87:0/3331528629 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a0e00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549c0cc200 0x7f549c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a1800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f549c1626c0 0x7f549c098d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 -- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549c0cc200 msgr2=0x7f549c156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549c0cc200 0x7f549c156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 -- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f549c1626c0 msgr2=0x7f549c098d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f549c1626c0 0x7f549c098d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f549be00640 1 -- 172.21.15.87:0/3331528629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c099a40 con 0x7f549c0d0d00 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a0e00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549c0cc200 0x7f549c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.144+0000 7f54a1800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f549c1626c0 0x7f549c098d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f549be00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0d0d00 0x7f549c0987d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54880ae040 tx=0x7f5488019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f5498c00640 1 -- 172.21.15.87:0/3331528629 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880b8020 con 0x7f549c0d0d00 2024-04-25T10:12:34.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f5498c00640 1 -- 172.21.15.87:0/3331528629 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54880b4410 con 0x7f549c0d0d00 2024-04-25T10:12:34.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c1597b0 con 0x7f549c0d0d00 2024-04-25T10:12:34.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f5498c00640 1 -- 172.21.15.87:0/3331528629 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880b0040 con 0x7f549c0d0d00 2024-04-25T10:12:34.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c157010 con 0x7f549c0d0d00 2024-04-25T10:12:34.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c157640 con 0x7f549c0d0d00 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f5498c00640 1 -- 172.21.15.87:0/3331528629 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54880bb020 con 0x7f549c0d0d00 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f5498c00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f547c066f60 0x7f547c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f5498c00640 1 -- 172.21.15.87:0/3331528629 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f548811edd0 con 0x7f549c0d0d00 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f54a0e00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f547c066f60 0x7f547c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.148+0000 7f54a0e00640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f547c066f60 0x7f547c069420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f549c15cca0 tx=0x7f548c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.152+0000 7f5498c00640 1 -- 172.21.15.87:0/3331528629 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54880c2030 con 0x7f549c0d0d00 2024-04-25T10:12:34.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f549c04ef30 con 0x7f549c0d0d00 2024-04-25T10:12:34.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f5498c00640 1 -- 172.21.15.87:0/3331528629 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5488016070 con 0x7f549c0d0d00 2024-04-25T10:12:34.313 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:34.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f547c066f60 msgr2=0x7f547c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f547c066f60 0x7f547c069420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f549c15cca0 tx=0x7f548c00b040 comp rx=0 tx=0).stop 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0d0d00 msgr2=0x7f549c0987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0d0d00 0x7f549c0987d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54880ae040 tx=0x7f5488019190 comp rx=0 tx=0).stop 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 shutdown_connections 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f547c066f60 0x7f547c069420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f549c1626c0 0x7f549c098d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f549c0d0d00 0x7f549c0987d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 --2- 172.21.15.87:0/3331528629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549c0cc200 0x7f549c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 >> 172.21.15.87:0/3331528629 conn(0x7f549c072290 msgr2=0x7f549c0ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 shutdown_connections 2024-04-25T10:12:34.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.308+0000 7f54a3800640 1 -- 172.21.15.87:0/3331528629 wait complete. 2024-04-25T10:12:34.335 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:34.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.468+0000 7fad4e200640 1 Processor -- start 2024-04-25T10:12:34.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.468+0000 7fad4e200640 1 -- start start 2024-04-25T10:12:34.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.468+0000 7fad4e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad48159540 0x7fad480ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.468+0000 7fad4e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad480aebd0 0x7fad48071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.468+0000 7fad4e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad48071360 con 0x7fad48072020 2024-04-25T10:12:34.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.468+0000 7fad4e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad480af030 con 0x7fad480aebd0 2024-04-25T10:12:34.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.468+0000 7fad4e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad48159920 con 0x7fad48159540 2024-04-25T10:12:34.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad48159540 0x7fad480ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad48159540 0x7fad480ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:38392/0 (socket says 172.21.15.87:38392) 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 -- 172.21.15.87:0/1147623372 learned_addr learned my addr 172.21.15.87:0/1147623372 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47e00640 1 --1- 172.21.15.87:0/1147623372 >> v1:172.21.15.169:6789/0 conn(0x7fad48072020 0x7fad4815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:40046/0 (socket says 172.21.15.87:40046) 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad3ea00640 1 --2- 172.21.15.87:0/1147623372 >> v2:172.21.15.169:3300/0 conn(0x7fad480aebd0 0x7fad48071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 -- 172.21.15.87:0/1147623372 >> v2:172.21.15.169:3300/0 conn(0x7fad480aebd0 msgr2=0x7fad48071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 --2- 172.21.15.87:0/1147623372 >> v2:172.21.15.169:3300/0 conn(0x7fad480aebd0 0x7fad48071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 -- 172.21.15.87:0/1147623372 >> v1:172.21.15.169:6789/0 conn(0x7fad48072020 legacy=0x7fad4815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 -- 172.21.15.87:0/1147623372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4815c2f0 con 0x7fad48159540 2024-04-25T10:12:34.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.472+0000 7fad47400640 1 --2- 172.21.15.87:0/1147623372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad48159540 0x7fad480ae690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad38009e50 tx=0x7fad3801ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e4b7dbea3633d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad46a00640 1 -- 172.21.15.87:0/1147623372 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad380a57e0 con 0x7fad48159540 2024-04-25T10:12:34.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad46a00640 1 -- 172.21.15.87:0/1147623372 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad380a5980 con 0x7fad48159540 2024-04-25T10:12:34.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad46a00640 1 -- 172.21.15.87:0/1147623372 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad380a5d50 con 0x7fad48159540 2024-04-25T10:12:34.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- 172.21.15.87:0/1147623372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad48159540 msgr2=0x7fad480ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 --2- 172.21.15.87:0/1147623372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad48159540 0x7fad480ae690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad38009e50 tx=0x7fad3801ca00 comp rx=0 tx=0).stop 2024-04-25T10:12:34.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- 172.21.15.87:0/1147623372 shutdown_connections 2024-04-25T10:12:34.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 --2- 172.21.15.87:0/1147623372 >> v2:172.21.15.169:3300/0 conn(0x7fad480aebd0 0x7fad48071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 --2- 172.21.15.87:0/1147623372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad48159540 0x7fad480ae690 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- 172.21.15.87:0/1147623372 >> 172.21.15.87:0/1147623372 conn(0x7fad480669a0 msgr2=0x7fad48066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- 172.21.15.87:0/1147623372 shutdown_connections 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- 172.21.15.87:0/1147623372 wait complete. 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 Processor -- start 2024-04-25T10:12:34.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- start start 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad48072020 0x7fad480adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad480aebd0 0x7fad480b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad48159540 0x7fad4813fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4806c0d0 con 0x7fad48159540 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815bfa0 con 0x7fad48072020 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad4e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad48072430 con 0x7fad480aebd0 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.476+0000 7fad47e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad48159540 0x7fad4813fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad48072020 0x7fad480adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad48159540 0x7fad4813fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57480/0 (socket says 172.21.15.87:57480) 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 -- 172.21.15.87:0/4253271461 learned_addr learned my addr 172.21.15.87:0/4253271461 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad3ea00640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad480aebd0 0x7fad480b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 -- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad480aebd0 msgr2=0x7fad480b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad480aebd0 0x7fad480b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 -- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad48072020 msgr2=0x7fad480adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad48072020 0x7fad480adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 -- 172.21.15.87:0/4253271461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad480b32c0 con 0x7fad48159540 2024-04-25T10:12:34.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47400640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad48072020 0x7fad480adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47e00640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad48159540 0x7fad4813fb90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c410 tx=0x7fad4000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad3fe00640 1 -- 172.21.15.87:0/4253271461 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad400074e0 con 0x7fad48159540 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad3fe00640 1 -- 172.21.15.87:0/4253271461 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad40007680 con 0x7fad48159540 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad480b35a0 con 0x7fad48159540 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad3fe00640 1 -- 172.21.15.87:0/4253271461 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40004a10 con 0x7fad48159540 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad480b3b60 con 0x7fad48159540 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4804ef30 con 0x7fad48159540 2024-04-25T10:12:34.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad3fe00640 1 -- 172.21.15.87:0/4253271461 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad4001a020 con 0x7fad48159540 2024-04-25T10:12:34.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad3fe00640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad24066f60 0x7fad24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad3fe00640 1 -- 172.21.15.87:0/4253271461 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad40084780 con 0x7fad48159540 2024-04-25T10:12:34.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.480+0000 7fad47400640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad24066f60 0x7fad24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.484+0000 7fad47400640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad24066f60 0x7fad24069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fad380ae000 tx=0x7fad380a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.484+0000 7fad3fe00640 1 -- 172.21.15.87:0/4253271461 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad400550a0 con 0x7fad48159540 2024-04-25T10:12:34.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.640+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad4806d7f0 con 0x7fad48159540 2024-04-25T10:12:34.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.640+0000 7fad3fe00640 1 -- 172.21.15.87:0/4253271461 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad40059140 con 0x7fad48159540 2024-04-25T10:12:34.646 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:34.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad24066f60 msgr2=0x7fad24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad24066f60 0x7fad24069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fad380ae000 tx=0x7fad380a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:34.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad48159540 msgr2=0x7fad4813fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad48159540 0x7fad4813fb90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c410 tx=0x7fad4000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 shutdown_connections 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad24066f60 0x7fad24069420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad48159540 0x7fad4813fb90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad480aebd0 0x7fad480b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 --2- 172.21.15.87:0/4253271461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad48072020 0x7fad480adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 >> 172.21.15.87:0/4253271461 conn(0x7fad480669a0 msgr2=0x7fad4815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 shutdown_connections 2024-04-25T10:12:34.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.644+0000 7fad4e200640 1 -- 172.21.15.87:0/4253271461 wait complete. 2024-04-25T10:12:34.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:34.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe02e800640 1 Processor -- start 2024-04-25T10:12:34.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe02e800640 1 -- start start 2024-04-25T10:12:34.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe02e800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe028163140 0x7fe0280b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe02e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 0x7fe0280cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe02e800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe028071d90 con 0x7fe0280aeef0 2024-04-25T10:12:34.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe02e800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0280b2e80 con 0x7fe028163140 2024-04-25T10:12:34.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe02e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280d0010 con 0x7fe0280cf640 2024-04-25T10:12:34.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe027400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe0280aeef0 0x7fe0280b2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60692/0 (socket says 172.21.15.87:60692) 2024-04-25T10:12:34.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe027400640 1 -- 172.21.15.87:0/718779842 learned_addr learned my addr 172.21.15.87:0/718779842 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe027e00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 0x7fe0280cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe01ea00640 1 --2- 172.21.15.87:0/718779842 >> v2:172.21.15.169:3300/0 conn(0x7fe028163140 0x7fe0280b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604885929 0 0) 0x7fe0180014a0 con 0x7fe0280aeef0 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe00c003530 con 0x7fe0280aeef0 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3800589335 0 0) 0x7fe00c003530 con 0x7fe0280aeef0 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 msgr2=0x7fe0280cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 0x7fe0280cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 >> v2:172.21.15.169:3300/0 conn(0x7fe028163140 msgr2=0x7fe0280b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 --2- 172.21.15.87:0/718779842 >> v2:172.21.15.169:3300/0 conn(0x7fe028163140 0x7fe0280b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02806c170 con 0x7fe0280aeef0 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe018001bd0 con 0x7fe0280aeef0 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 >> v1:172.21.15.87:6789/0 conn(0x7fe0280aeef0 legacy=0x7fe0280b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 0x7fe00c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe00c003f90 0x7fe00c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe00c004d50 0x7fe00c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280b2e80 con 0x7fe00c003f90 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280d0010 con 0x7fe00c004d50 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c00b9d0 con 0x7fe0280cf640 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe01ea00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe00c003f90 0x7fe00c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe027400640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 0x7fe00c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.800+0000 7fe027e00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe00c004d50 0x7fe00c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe01ea00640 1 -- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 msgr2=0x7fe00c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe01ea00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0280cf640 0x7fe00c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe01ea00640 1 -- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe00c004d50 msgr2=0x7fe00c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe01ea00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe00c004d50 0x7fe00c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe01ea00640 1 -- 172.21.15.87:0/718779842 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c003530 con 0x7fe00c003f90 2024-04-25T10:12:34.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe01ea00640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe00c003f90 0x7fe00c004810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe010009b10 tx=0x7fe01001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4106f27d5bced5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100b2070 con 0x7fe00c003f90 2024-04-25T10:12:34.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0100a5590 con 0x7fe00c003f90 2024-04-25T10:12:34.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe026a00640 1 -- 172.21.15.87:0/718779842 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100096b0 con 0x7fe00c003f90 2024-04-25T10:12:34.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe02e800640 1 -- 172.21.15.87:0/718779842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe00c003f90 msgr2=0x7fe00c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe02e800640 1 --2- 172.21.15.87:0/718779842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe00c003f90 0x7fe00c004810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe010009b10 tx=0x7fe01001cab0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe027e00640 1 -- 172.21.15.87:0/718779842 reap_dead start 2024-04-25T10:12:34.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe02e800640 1 -- 172.21.15.87:0/718779842 shutdown_connections 2024-04-25T10:12:34.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe02e800640 1 -- 172.21.15.87:0/718779842 >> 172.21.15.87:0/718779842 conn(0x7fe0280669a0 msgr2=0x7fe028066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe02e800640 1 -- 172.21.15.87:0/718779842 shutdown_connections 2024-04-25T10:12:34.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.804+0000 7fe02e800640 1 -- 172.21.15.87:0/718779842 wait complete. 2024-04-25T10:12:34.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 Processor -- start 2024-04-25T10:12:34.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 -- start start 2024-04-25T10:12:34.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe00c003f90 0x7fe0281eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe028156eb0 0x7fe02815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe02815f500 0x7fe02815f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281635c0 con 0x7fe02815f500 2024-04-25T10:12:34.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280722f0 con 0x7fe028156eb0 2024-04-25T10:12:34.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281ec2d0 con 0x7fe00c003f90 2024-04-25T10:12:34.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe02815f500 0x7fe02815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe02815f500 0x7fe02815f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57486/0 (socket says 172.21.15.87:57486) 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 -- 172.21.15.87:0/598795522 learned_addr learned my addr 172.21.15.87:0/598795522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027400640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe00c003f90 0x7fe0281eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe01ea00640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe028156eb0 0x7fe02815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 -- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe00c003f90 msgr2=0x7fe0281eb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe00c003f90 0x7fe0281eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 -- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe028156eb0 msgr2=0x7fe02815a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe028156eb0 0x7fe02815a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 -- 172.21.15.87:0/598795522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0281ec5b0 con 0x7fe02815f500 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe01ea00640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe028156eb0 0x7fe02815a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe027e00640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe02815f500 0x7fe02815f980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe028071b00 tx=0x7fe020011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe01fe00640 1 -- 172.21.15.87:0/598795522 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020004030 con 0x7fe02815f500 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe01fe00640 1 -- 172.21.15.87:0/598795522 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02001b070 con 0x7fe02815f500 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe01fe00640 1 -- 172.21.15.87:0/598795522 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020004450 con 0x7fe02815f500 2024-04-25T10:12:34.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0281ec890 con 0x7fe02815f500 2024-04-25T10:12:34.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.808+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0281586d0 con 0x7fe02815f500 2024-04-25T10:12:34.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.812+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02804ef30 con 0x7fe02815f500 2024-04-25T10:12:34.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.812+0000 7fe01fe00640 1 -- 172.21.15.87:0/598795522 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe020012040 con 0x7fe02815f500 2024-04-25T10:12:34.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.812+0000 7fe01fe00640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe004066f60 0x7fe004069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.812+0000 7fe01fe00640 1 -- 172.21.15.87:0/598795522 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe020085070 con 0x7fe02815f500 2024-04-25T10:12:34.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.812+0000 7fe027400640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe004066f60 0x7fe004069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.812+0000 7fe027400640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe004066f60 0x7fe004069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe018001680 tx=0x7fe01800a160 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.816+0000 7fe01fe00640 1 -- 172.21.15.87:0/598795522 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe020055fe0 con 0x7fe02815f500 2024-04-25T10:12:34.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe028158900 con 0x7fe02815f500 2024-04-25T10:12:34.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe01fe00640 1 -- 172.21.15.87:0/598795522 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0200558e0 con 0x7fe02815f500 2024-04-25T10:12:34.973 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe004066f60 msgr2=0x7fe004069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe004066f60 0x7fe004069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe018001680 tx=0x7fe01800a160 comp rx=0 tx=0).stop 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe02815f500 msgr2=0x7fe02815f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe02815f500 0x7fe02815f980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe028071b00 tx=0x7fe020011cf0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 shutdown_connections 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe004066f60 0x7fe004069420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe02815f500 0x7fe02815f980 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe028156eb0 0x7fe02815a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 --2- 172.21.15.87:0/598795522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe00c003f90 0x7fe0281eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 >> 172.21.15.87:0/598795522 conn(0x7fe0280669a0 msgr2=0x7fe028160f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 shutdown_connections 2024-04-25T10:12:34.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:34.968+0000 7fe02e800640 1 -- 172.21.15.87:0/598795522 wait complete. 2024-04-25T10:12:34.994 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:35.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae6d800640 1 Processor -- start 2024-04-25T10:12:35.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae6d800640 1 -- start start 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae6d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae6806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae6d800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fae680cf930 0x7fae680cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae6d800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae680b2770 con 0x7fae680cda10 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae6d800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6806c670 con 0x7fae680cf930 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae6d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6806c950 con 0x7fae6815bc40 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae5e000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fae680cda10 0x7fae680b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60704/0 (socket says 172.21.15.87:60704) 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae67400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fae680cf930 0x7fae680cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.124+0000 7fae5e000640 1 -- 172.21.15.87:0/3806753182 learned_addr learned my addr 172.21.15.87:0/3806753182 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae66a00640 1 --2- 172.21.15.87:0/3806753182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae6806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae66000640 1 -- 172.21.15.87:0/3806753182 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903835781 0 0) 0x7fae680b2770 con 0x7fae680cda10 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae66000640 1 -- 172.21.15.87:0/3806753182 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae4c003530 con 0x7fae680cda10 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae67400640 1 -- 172.21.15.87:0/3806753182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 msgr2=0x7fae6806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae67400640 1 --2- 172.21.15.87:0/3806753182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae6806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae67400640 1 -- 172.21.15.87:0/3806753182 >> v1:172.21.15.87:6789/0 conn(0x7fae680cda10 legacy=0x7fae680b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae67400640 1 -- 172.21.15.87:0/3806753182 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6806cc30 con 0x7fae680cf930 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae67400640 1 --2- 172.21.15.87:0/3806753182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cf930 0x7fae680cfde0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fae60009d90 tx=0x7fae6001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d37f5cc74f6086b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae66000640 1 -- 172.21.15.87:0/3806753182 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae600b2070 con 0x7fae680cf930 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae66000640 1 -- 172.21.15.87:0/3806753182 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae600a57a0 con 0x7fae680cf930 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae66000640 1 -- 172.21.15.87:0/3806753182 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae600a5b70 con 0x7fae680cf930 2024-04-25T10:12:35.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae6d800640 1 -- 172.21.15.87:0/3806753182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cf930 msgr2=0x7fae680cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.128+0000 7fae6d800640 1 --2- 172.21.15.87:0/3806753182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cf930 0x7fae680cfde0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fae60009d90 tx=0x7fae6001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- 172.21.15.87:0/3806753182 shutdown_connections 2024-04-25T10:12:35.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 --2- 172.21.15.87:0/3806753182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cf930 0x7fae680cfde0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 --2- 172.21.15.87:0/3806753182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae6806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- 172.21.15.87:0/3806753182 >> 172.21.15.87:0/3806753182 conn(0x7fae68066b70 msgr2=0x7fae68066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- 172.21.15.87:0/3806753182 shutdown_connections 2024-04-25T10:12:35.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- 172.21.15.87:0/3806753182 wait complete. 2024-04-25T10:12:35.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 Processor -- start 2024-04-25T10:12:35.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- start start 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cda10 0x7fae6814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae680cf930 0x7fae68156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae68157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae680b3850 con 0x7fae680cda10 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae680b36d0 con 0x7fae680cf930 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae680b39d0 con 0x7fae6815bc40 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cda10 0x7fae6814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cda10 0x7fae6814e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57504/0 (socket says 172.21.15.87:57504) 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 -- 172.21.15.87:0/3295988184 learned_addr learned my addr 172.21.15.87:0/3295988184 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae67400640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae68157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae5e000640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae680cf930 0x7fae68156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 -- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 msgr2=0x7fae68157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae68157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 -- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae680cf930 msgr2=0x7fae68156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae680cf930 0x7fae68156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 -- 172.21.15.87:0/3295988184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae680b4250 con 0x7fae680cda10 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae5e000640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae680cf930 0x7fae68156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae66a00640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cda10 0x7fae6814e490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae5800e8a0 tx=0x7fae5800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae5f400640 1 -- 172.21.15.87:0/3295988184 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58017070 con 0x7fae680cda10 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae5f400640 1 -- 172.21.15.87:0/3295988184 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5800c940 con 0x7fae680cda10 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae5f400640 1 -- 172.21.15.87:0/3295988184 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800ccf0 con 0x7fae680cda10 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.132+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6814e9d0 con 0x7fae680cda10 2024-04-25T10:12:35.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.136+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6815cc00 con 0x7fae680cda10 2024-04-25T10:12:35.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.136+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6804ef30 con 0x7fae680cda10 2024-04-25T10:12:35.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.136+0000 7fae5f400640 1 -- 172.21.15.87:0/3295988184 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae5801a070 con 0x7fae680cda10 2024-04-25T10:12:35.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.136+0000 7fae5f400640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae44066f90 0x7fae44069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.136+0000 7fae5f400640 1 -- 172.21.15.87:0/3295988184 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae58084f20 con 0x7fae680cda10 2024-04-25T10:12:35.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.136+0000 7fae5e000640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae44066f90 0x7fae44069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.136+0000 7fae5e000640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae44066f90 0x7fae44069450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae68166990 tx=0x7fae50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.140+0000 7fae5f400640 1 -- 172.21.15.87:0/3295988184 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae58055f20 con 0x7fae680cda10 2024-04-25T10:12:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae6806d960 con 0x7fae680cda10 2024-04-25T10:12:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae5f400640 1 -- 172.21.15.87:0/3295988184 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fae58055820 con 0x7fae680cda10 2024-04-25T10:12:35.333 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:35.333 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:12:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae44066f90 msgr2=0x7fae44069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae44066f90 0x7fae44069450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae68166990 tx=0x7fae50009040 comp rx=0 tx=0).stop 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cda10 msgr2=0x7fae6814e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cda10 0x7fae6814e490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae5800e8a0 tx=0x7fae5800eca0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 shutdown_connections 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae44066f90 0x7fae44069450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae6815bc40 0x7fae68157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae680cf930 0x7fae68156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 --2- 172.21.15.87:0/3295988184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae680cda10 0x7fae6814e490 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 >> 172.21.15.87:0/3295988184 conn(0x7fae68066b70 msgr2=0x7fae68155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 shutdown_connections 2024-04-25T10:12:35.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.328+0000 7fae6d800640 1 -- 172.21.15.87:0/3295988184 wait complete. 2024-04-25T10:12:35.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:35.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.484+0000 7f8130400640 1 Processor -- start 2024-04-25T10:12:35.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.484+0000 7f8130400640 1 -- start start 2024-04-25T10:12:35.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f8130400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81280d0040 0x7f812806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f8130400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81280b2770 con 0x7f812815bc40 2024-04-25T10:12:35.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f8130400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812806c670 con 0x7f81280cda10 2024-04-25T10:12:35.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f8130400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812806c950 con 0x7f81280d0040 2024-04-25T10:12:35.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f812da00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f812815bc40 0x7f81280b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60706/0 (socket says 172.21.15.87:60706) 2024-04-25T10:12:35.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f812da00640 1 -- 172.21.15.87:0/3448394688 learned_addr learned my addr 172.21.15.87:0/3448394688 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:35.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f812e400640 1 --2- 172.21.15.87:0/3448394688 >> v2:172.21.15.87:3300/0 conn(0x7f81280d0040 0x7f812806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f812e400640 1 -- 172.21.15.87:0/3448394688 >> v1:172.21.15.169:6789/0 conn(0x7f81280cda10 legacy=0x7f81280cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f812e400640 1 -- 172.21.15.87:0/3448394688 >> v1:172.21.15.87:6789/0 conn(0x7f812815bc40 legacy=0x7f81280b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f812e400640 1 -- 172.21.15.87:0/3448394688 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812806cc30 con 0x7f81280d0040 2024-04-25T10:12:35.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.488+0000 7f8127e00640 1 -- 172.21.15.87:0/3448394688 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543792707 0 0) 0x7f81280b2770 con 0x7f812815bc40 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f812e400640 1 --2- 172.21.15.87:0/3448394688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280d0040 0x7f812806c090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81200097c0 tx=0x7f812001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c68262f41cc0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8127e00640 1 -- 172.21.15.87:0/3448394688 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81200b0070 con 0x7f81280d0040 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8127e00640 1 -- 172.21.15.87:0/3448394688 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81200ab600 con 0x7f81280d0040 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8127e00640 1 -- 172.21.15.87:0/3448394688 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81200ab9d0 con 0x7f81280d0040 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- 172.21.15.87:0/3448394688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280d0040 msgr2=0x7f812806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 --2- 172.21.15.87:0/3448394688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280d0040 0x7f812806c090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81200097c0 tx=0x7f812001c410 comp rx=0 tx=0).stop 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- 172.21.15.87:0/3448394688 shutdown_connections 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 --2- 172.21.15.87:0/3448394688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280d0040 0x7f812806c090 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- 172.21.15.87:0/3448394688 >> 172.21.15.87:0/3448394688 conn(0x7f8128066b70 msgr2=0x7f8128066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- 172.21.15.87:0/3448394688 shutdown_connections 2024-04-25T10:12:35.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- 172.21.15.87:0/3448394688 wait complete. 2024-04-25T10:12:35.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 Processor -- start 2024-04-25T10:12:35.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- start start 2024-04-25T10:12:35.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cda10 0x7f8128150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81280d0040 0x7f8128070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f812815bc40 0x7f81280b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128169980 con 0x7f81280cda10 2024-04-25T10:12:35.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128169800 con 0x7f812815bc40 2024-04-25T10:12:35.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f8130400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8128169b00 con 0x7f81280d0040 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f812da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cda10 0x7f8128150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f812da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cda10 0x7f8128150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57524/0 (socket says 172.21.15.87:57524) 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f812da00640 1 -- 172.21.15.87:0/2372726407 learned_addr learned my addr 172.21.15.87:0/2372726407 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f812d000640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81280d0040 0x7f8128070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.492+0000 7f812e400640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f812815bc40 0x7f81280b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f812da00640 1 -- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81280d0040 msgr2=0x7f8128070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f812da00640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81280d0040 0x7f8128070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f812da00640 1 -- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f812815bc40 msgr2=0x7f81280b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f812da00640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f812815bc40 0x7f81280b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f812da00640 1 -- 172.21.15.87:0/2372726407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8128070bd0 con 0x7f81280cda10 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f812da00640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cda10 0x7f8128150de0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c8c0 tx=0x7f811c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8125600640 1 -- 172.21.15.87:0/2372726407 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c018070 con 0x7f81280cda10 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8125600640 1 -- 172.21.15.87:0/2372726407 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c00a640 con 0x7f81280cda10 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8128070eb0 con 0x7f81280cda10 2024-04-25T10:12:35.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8125600640 1 -- 172.21.15.87:0/2372726407 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00aa70 con 0x7f81280cda10 2024-04-25T10:12:35.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8128071470 con 0x7f81280cda10 2024-04-25T10:12:35.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812804ef30 con 0x7f81280cda10 2024-04-25T10:12:35.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8125600640 1 -- 172.21.15.87:0/2372726407 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f811c01b070 con 0x7f81280cda10 2024-04-25T10:12:35.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8125600640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f8125600640 1 -- 172.21.15.87:0/2372726407 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f811c084690 con 0x7f81280cda10 2024-04-25T10:12:35.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.496+0000 7f812d000640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.500+0000 7f812d000640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f81280cc0c0 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.500+0000 7f8125600640 1 -- 172.21.15.87:0/2372726407 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f811c054fb0 con 0x7f81280cda10 2024-04-25T10:12:35.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.660+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f812806d960 con 0x7f81280cda10 2024-04-25T10:12:35.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8125600640 1 -- 172.21.15.87:0/2372726407 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f811c059050 con 0x7f81280cda10 2024-04-25T10:12:35.668 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:35.668 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:35.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 msgr2=0x7f8108069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f81280cc0c0 tx=0x7f8114009040 comp rx=0 tx=0).stop 2024-04-25T10:12:35.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cda10 msgr2=0x7f8128150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cda10 0x7f8128150de0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c8c0 tx=0x7f811c00cd80 comp rx=0 tx=0).stop 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 shutdown_connections 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f812815bc40 0x7f81280b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81280d0040 0x7f8128070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 --2- 172.21.15.87:0/2372726407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cda10 0x7f8128150de0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 >> 172.21.15.87:0/2372726407 conn(0x7f8128066b70 msgr2=0x7f81280af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 shutdown_connections 2024-04-25T10:12:35.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.664+0000 7f8130400640 1 -- 172.21.15.87:0/2372726407 wait complete. 2024-04-25T10:12:35.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:35.690 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a4200640 1 Processor -- start 2024-04-25T10:12:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a4200640 1 -- start start 2024-04-25T10:12:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c15bc10 0x7f609c15f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a4200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f609c0cf980 0x7f609c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a4200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609c15fec0 con 0x7f609c0d1830 2024-04-25T10:12:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a4200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c1601a0 con 0x7f609c0cf980 2024-04-25T10:12:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0cc5b0 con 0x7f609c15bc10 2024-04-25T10:12:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a0e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f609c0cf980 0x7f609c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a2200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f609c0d1830 0x7f609c0d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60714/0 (socket says 172.21.15.87:60714) 2024-04-25T10:12:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a2200640 1 -- 172.21.15.87:0/2610855087 learned_addr learned my addr 172.21.15.87:0/2610855087 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a1800640 1 --2- 172.21.15.87:0/2610855087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c15bc10 0x7f609c15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a0e00640 1 -- 172.21.15.87:0/2610855087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c15bc10 msgr2=0x7f609c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a0e00640 1 --2- 172.21.15.87:0/2610855087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c15bc10 0x7f609c15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a0e00640 1 -- 172.21.15.87:0/2610855087 >> v1:172.21.15.87:6789/0 conn(0x7f609c0d1830 legacy=0x7f609c0d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f60a0e00640 1 -- 172.21.15.87:0/2610855087 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0cc890 con 0x7f609c0cf980 2024-04-25T10:12:35.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.824+0000 7f609be00640 1 -- 172.21.15.87:0/2610855087 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177242357 0 0) 0x7f609c15fec0 con 0x7f609c0d1830 2024-04-25T10:12:35.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a0e00640 1 --2- 172.21.15.87:0/2610855087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c0b1060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f60880097c0 tx=0x7f608801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4318796b6b362559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f609be00640 1 -- 172.21.15.87:0/2610855087 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60880b0070 con 0x7f609c0cf980 2024-04-25T10:12:35.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f609be00640 1 -- 172.21.15.87:0/2610855087 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60880ab600 con 0x7f609c0cf980 2024-04-25T10:12:35.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f609be00640 1 -- 172.21.15.87:0/2610855087 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60880ab9d0 con 0x7f609c0cf980 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 -- 172.21.15.87:0/2610855087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 msgr2=0x7f609c0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 --2- 172.21.15.87:0/2610855087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c0b1060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f60880097c0 tx=0x7f608801c410 comp rx=0 tx=0).stop 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 -- 172.21.15.87:0/2610855087 shutdown_connections 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 --2- 172.21.15.87:0/2610855087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c0b1060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 --2- 172.21.15.87:0/2610855087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c15bc10 0x7f609c15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 -- 172.21.15.87:0/2610855087 >> 172.21.15.87:0/2610855087 conn(0x7f609c09a9d0 msgr2=0x7f609c09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 -- 172.21.15.87:0/2610855087 shutdown_connections 2024-04-25T10:12:35.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 -- 172.21.15.87:0/2610855087 wait complete. 2024-04-25T10:12:35.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 Processor -- start 2024-04-25T10:12:35.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.828+0000 7f60a4200640 1 -- start start 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c0d1830 0x7f609c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f609c15bc10 0x7f609c159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0cd740 con 0x7f609c0cf980 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0cd5c0 con 0x7f609c15bc10 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c168960 con 0x7f609c0d1830 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57544/0 (socket says 172.21.15.87:57544) 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 -- 172.21.15.87:0/2421269256 learned_addr learned my addr 172.21.15.87:0/2421269256 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a0e00640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c0d1830 0x7f609c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a2200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f609c15bc10 0x7f609c159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 -- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c0d1830 msgr2=0x7f609c0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c0d1830 0x7f609c0b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 -- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f609c15bc10 msgr2=0x7f609c159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f609c15bc10 0x7f609c159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 -- 172.21.15.87:0/2421269256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0b28c0 con 0x7f609c0cf980 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a0e00640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c0d1830 0x7f609c0b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a2200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f609c15bc10 0x7f609c159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a1800640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c155b90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f609000d800 tx=0x7f609000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f6099600640 1 -- 172.21.15.87:0/2421269256 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000b6c0 con 0x7f609c0cf980 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f6099600640 1 -- 172.21.15.87:0/2421269256 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090019070 con 0x7f609c0cf980 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f6099600640 1 -- 172.21.15.87:0/2421269256 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000ba60 con 0x7f609c0cf980 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c0b2ba0 con 0x7f609c0cf980 2024-04-25T10:12:35.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c159d90 con 0x7f609c0cf980 2024-04-25T10:12:35.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.832+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c04ef30 con 0x7f609c0cf980 2024-04-25T10:12:35.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.836+0000 7f6099600640 1 -- 172.21.15.87:0/2421269256 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f609000bc00 con 0x7f609c0cf980 2024-04-25T10:12:35.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.836+0000 7f6099600640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f607c066f60 0x7f607c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.836+0000 7f6099600640 1 -- 172.21.15.87:0/2421269256 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6090085570 con 0x7f609c0cf980 2024-04-25T10:12:35.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.836+0000 7f60a0e00640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f607c066f60 0x7f607c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.836+0000 7f60a0e00640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f607c066f60 0x7f607c069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60880ad000 tx=0x7f6088008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.840+0000 7f6099600640 1 -- 172.21.15.87:0/2421269256 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60900564e0 con 0x7f609c0cf980 2024-04-25T10:12:35.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.992+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f609c0b7920 con 0x7f609c0cf980 2024-04-25T10:12:35.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.992+0000 7f6099600640 1 -- 172.21.15.87:0/2421269256 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6090083e20 con 0x7f609c0cf980 2024-04-25T10:12:35.998 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.992+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f607c066f60 msgr2=0x7f607c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f607c066f60 0x7f607c069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60880ad000 tx=0x7f6088008040 comp rx=0 tx=0).stop 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 msgr2=0x7f609c155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c155b90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f609000d800 tx=0x7f609000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 shutdown_connections 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f607c066f60 0x7f607c069420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f609c15bc10 0x7f609c159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f609c0d1830 0x7f609c0b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 --2- 172.21.15.87:0/2421269256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f609c0cf980 0x7f609c155b90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 >> 172.21.15.87:0/2421269256 conn(0x7f609c09a9d0 msgr2=0x7f609c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 shutdown_connections 2024-04-25T10:12:36.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:35.996+0000 7f60a4200640 1 -- 172.21.15.87:0/2421269256 wait complete. 2024-04-25T10:12:36.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac04800640 1 Processor -- start 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac04800640 1 -- start start 2024-04-25T10:12:36.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac04800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fabfc15dd30 0x7fabfc06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac04800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fabfc0d1fb0 0x7fabfc0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac04800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc06b1d0 con 0x7fabfc0cfb30 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac04800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc15e110 con 0x7fabfc0d1fb0 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac04800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc06b470 con 0x7fabfc15dd30 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac02800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fabfc0d1fb0 0x7fabfc0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac01400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fabfc0cfb30 0x7fabfc0d18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60716/0 (socket says 172.21.15.87:60716) 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac01400640 1 -- 172.21.15.87:0/2384435829 learned_addr learned my addr 172.21.15.87:0/2384435829 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac00a00640 1 -- 172.21.15.87:0/2384435829 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802518925 0 0) 0x7fabfc06b1d0 con 0x7fabfc0cfb30 2024-04-25T10:12:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac01e00640 1 --2- 172.21.15.87:0/2384435829 >> v2:172.21.15.169:3300/0 conn(0x7fabfc15dd30 0x7fabfc06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac00a00640 1 -- 172.21.15.87:0/2384435829 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe4003530 con 0x7fabfc0cfb30 2024-04-25T10:12:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac02800640 1 -- 172.21.15.87:0/2384435829 >> v2:172.21.15.169:3300/0 conn(0x7fabfc15dd30 msgr2=0x7fabfc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac02800640 1 --2- 172.21.15.87:0/2384435829 >> v2:172.21.15.169:3300/0 conn(0x7fabfc15dd30 0x7fabfc06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac02800640 1 -- 172.21.15.87:0/2384435829 >> v1:172.21.15.87:6789/0 conn(0x7fabfc0cfb30 legacy=0x7fabfc0d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.152+0000 7fac02800640 1 -- 172.21.15.87:0/2384435829 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc06b750 con 0x7fabfc0d1fb0 2024-04-25T10:12:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac02800640 1 --2- 172.21.15.87:0/2384435829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0d1fb0 0x7fabfc0b1930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0b1520 tx=0x7fabf801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d5c832833f7215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac00a00640 1 -- 172.21.15.87:0/2384435829 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf80b2070 con 0x7fabfc0d1fb0 2024-04-25T10:12:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac00a00640 1 -- 172.21.15.87:0/2384435829 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf80a58e0 con 0x7fabfc0d1fb0 2024-04-25T10:12:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac00a00640 1 -- 172.21.15.87:0/2384435829 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf80a5cb0 con 0x7fabfc0d1fb0 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 -- 172.21.15.87:0/2384435829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0d1fb0 msgr2=0x7fabfc0b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 --2- 172.21.15.87:0/2384435829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0d1fb0 0x7fabfc0b1930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0b1520 tx=0x7fabf801c930 comp rx=0 tx=0).stop 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 -- 172.21.15.87:0/2384435829 shutdown_connections 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 --2- 172.21.15.87:0/2384435829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0d1fb0 0x7fabfc0b1930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 --2- 172.21.15.87:0/2384435829 >> v2:172.21.15.169:3300/0 conn(0x7fabfc15dd30 0x7fabfc06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 -- 172.21.15.87:0/2384435829 >> 172.21.15.87:0/2384435829 conn(0x7fabfc09a9d0 msgr2=0x7fabfc09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 -- 172.21.15.87:0/2384435829 shutdown_connections 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 -- 172.21.15.87:0/2384435829 wait complete. 2024-04-25T10:12:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 Processor -- start 2024-04-25T10:12:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.156+0000 7fac04800640 1 -- start start 2024-04-25T10:12:36.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0cfb30 0x7fabfc15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabfc0d1fb0 0x7fabfc1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabfc15dd30 0x7fabfc0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc16ab60 con 0x7fabfc0cfb30 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0b2890 con 0x7fabfc15dd30 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc16ace0 con 0x7fabfc0d1fb0 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0cfb30 0x7fabfc15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0cfb30 0x7fabfc15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57564/0 (socket says 172.21.15.87:57564) 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 -- 172.21.15.87:0/3346016149 learned_addr learned my addr 172.21.15.87:0/3346016149 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01400640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabfc0d1fb0 0x7fabfc1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac02800640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabfc15dd30 0x7fabfc0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 -- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabfc0d1fb0 msgr2=0x7fabfc1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabfc0d1fb0 0x7fabfc1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 -- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabfc15dd30 msgr2=0x7fabfc0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabfc15dd30 0x7fabfc0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 -- 172.21.15.87:0/3346016149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc15aef0 con 0x7fabfc0cfb30 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac01e00640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0cfb30 0x7fabfc15b990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d5f0 tx=0x7fabf000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fabf6000640 1 -- 172.21.15.87:0/3346016149 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00125f0 con 0x7fabfc0cfb30 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fabf6000640 1 -- 172.21.15.87:0/3346016149 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf0019070 con 0x7fabfc0cfb30 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc0b0890 con 0x7fabfc0cfb30 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fabf6000640 1 -- 172.21.15.87:0/3346016149 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00129b0 con 0x7fabfc0cfb30 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc154c00 con 0x7fabfc0cfb30 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc04ef30 con 0x7fabfc0cfb30 2024-04-25T10:12:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.160+0000 7fabf6000640 1 -- 172.21.15.87:0/3346016149 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabf0012b50 con 0x7fabfc0cfb30 2024-04-25T10:12:36.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.164+0000 7fabf6000640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fabe0066f60 0x7fabe0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.164+0000 7fabf6000640 1 -- 172.21.15.87:0/3346016149 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabf00853c0 con 0x7fabfc0cfb30 2024-04-25T10:12:36.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.164+0000 7fac01400640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fabe0066f60 0x7fabe0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.164+0000 7fac01400640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fabe0066f60 0x7fabe0069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fabfc1fa220 tx=0x7fabec000bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.168+0000 7fabf6000640 1 -- 172.21.15.87:0/3346016149 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabf0055ce0 con 0x7fabfc0cfb30 2024-04-25T10:12:36.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.320+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabfc0b7920 con 0x7fabfc0cfb30 2024-04-25T10:12:36.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.320+0000 7fabf6000640 1 -- 172.21.15.87:0/3346016149 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabf0059d80 con 0x7fabfc0cfb30 2024-04-25T10:12:36.327 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fabe0066f60 msgr2=0x7fabe0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fabe0066f60 0x7fabe0069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fabfc1fa220 tx=0x7fabec000bd0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0cfb30 msgr2=0x7fabfc15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0cfb30 0x7fabfc15b990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d5f0 tx=0x7fabf000dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 shutdown_connections 2024-04-25T10:12:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fabe0066f60 0x7fabe0069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabfc15dd30 0x7fabfc0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabfc0d1fb0 0x7fabfc1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 --2- 172.21.15.87:0/3346016149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabfc0cfb30 0x7fabfc15b990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 >> 172.21.15.87:0/3346016149 conn(0x7fabfc09a9d0 msgr2=0x7fabfc1567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 shutdown_connections 2024-04-25T10:12:36.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.324+0000 7fac04800640 1 -- 172.21.15.87:0/3346016149 wait complete. 2024-04-25T10:12:36.347 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:36.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.480+0000 7f2220400640 1 Processor -- start 2024-04-25T10:12:36.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.480+0000 7f2220400640 1 -- start start 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f2220400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22180d0d00 0x7f22180ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f2220400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22180cc200 0x7f2218162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f2220400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22180cdab0 con 0x7f22181626c0 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f2220400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180cd210 con 0x7f22180d0d00 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f2220400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180cd4f0 con 0x7f22180cc200 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221e400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f22181626c0 0x7f221806cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60720/0 (socket says 172.21.15.87:60720) 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221e400640 1 -- 172.21.15.87:0/3260511101 learned_addr learned my addr 172.21.15.87:0/3260511101 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:36.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221da00640 1 --2- 172.21.15.87:0/3260511101 >> v2:172.21.15.87:3300/0 conn(0x7f22180d0d00 0x7f22180ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221d000640 1 --2- 172.21.15.87:0/3260511101 >> v2:172.21.15.169:3300/0 conn(0x7f22180cc200 0x7f2218162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221da00640 1 -- 172.21.15.87:0/3260511101 >> v2:172.21.15.169:3300/0 conn(0x7f22180cc200 msgr2=0x7f2218162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221da00640 1 --2- 172.21.15.87:0/3260511101 >> v2:172.21.15.169:3300/0 conn(0x7f22180cc200 0x7f2218162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221da00640 1 -- 172.21.15.87:0/3260511101 >> v1:172.21.15.87:6789/0 conn(0x7f22181626c0 legacy=0x7f221806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f221da00640 1 -- 172.21.15.87:0/3260511101 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22180cd7d0 con 0x7f22180d0d00 2024-04-25T10:12:36.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.484+0000 7f2217e00640 1 -- 172.21.15.87:0/3260511101 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208595520 0 0) 0x7f22180cdab0 con 0x7f22181626c0 2024-04-25T10:12:36.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f221da00640 1 --2- 172.21.15.87:0/3260511101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22180d0d00 0x7f22180ccc30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2218071ca0 tx=0x7f220c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde9d07b0b494cca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2217e00640 1 -- 172.21.15.87:0/3260511101 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c0b0070 con 0x7f22180d0d00 2024-04-25T10:12:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2217e00640 1 -- 172.21.15.87:0/3260511101 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c0ab600 con 0x7f22180d0d00 2024-04-25T10:12:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2217e00640 1 -- 172.21.15.87:0/3260511101 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c0ab9d0 con 0x7f22180d0d00 2024-04-25T10:12:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- 172.21.15.87:0/3260511101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22180d0d00 msgr2=0x7f22180ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 --2- 172.21.15.87:0/3260511101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22180d0d00 0x7f22180ccc30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2218071ca0 tx=0x7f220c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- 172.21.15.87:0/3260511101 shutdown_connections 2024-04-25T10:12:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 --2- 172.21.15.87:0/3260511101 >> v2:172.21.15.169:3300/0 conn(0x7f22180cc200 0x7f2218162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 --2- 172.21.15.87:0/3260511101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22180d0d00 0x7f22180ccc30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- 172.21.15.87:0/3260511101 >> 172.21.15.87:0/3260511101 conn(0x7f2218072290 msgr2=0x7f22180726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- 172.21.15.87:0/3260511101 shutdown_connections 2024-04-25T10:12:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- 172.21.15.87:0/3260511101 wait complete. 2024-04-25T10:12:36.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 Processor -- start 2024-04-25T10:12:36.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- start start 2024-04-25T10:12:36.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22180cc200 0x7f2218154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22180d0d00 0x7f221815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22181626c0 0x7f221815cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221815e040 con 0x7f22181626c0 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221806d860 con 0x7f22180d0d00 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f2220400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221815e1c0 con 0x7f22180cc200 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f221e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22181626c0 0x7f221815cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f221e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22181626c0 0x7f221815cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57592/0 (socket says 172.21.15.87:57592) 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.488+0000 7f221e400640 1 -- 172.21.15.87:0/2033119302 learned_addr learned my addr 172.21.15.87:0/2033119302 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221da00640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22180cc200 0x7f2218154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221d000640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22180d0d00 0x7f221815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221e400640 1 -- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22180cc200 msgr2=0x7f2218154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221e400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22180cc200 0x7f2218154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221e400640 1 -- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22180d0d00 msgr2=0x7f221815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221e400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22180d0d00 0x7f221815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221e400640 1 -- 172.21.15.87:0/2033119302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221815a190 con 0x7f22181626c0 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f221e400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22181626c0 0x7f221815cb70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f221000c700 tx=0x7f221000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f2215600640 1 -- 172.21.15.87:0/2033119302 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210018070 con 0x7f22181626c0 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f2215600640 1 -- 172.21.15.87:0/2033119302 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221000ce10 con 0x7f22181626c0 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221815e650 con 0x7f22181626c0 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f2215600640 1 -- 172.21.15.87:0/2033119302 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221000a860 con 0x7f22181626c0 2024-04-25T10:12:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2218159e20 con 0x7f22181626c0 2024-04-25T10:12:36.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221804ef30 con 0x7f22181626c0 2024-04-25T10:12:36.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.492+0000 7f2215600640 1 -- 172.21.15.87:0/2033119302 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f221001b070 con 0x7f22181626c0 2024-04-25T10:12:36.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.496+0000 7f2215600640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21f8066f60 0x7f21f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.496+0000 7f2215600640 1 -- 172.21.15.87:0/2033119302 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2210084530 con 0x7f22181626c0 2024-04-25T10:12:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.496+0000 7f221da00640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21f8066f60 0x7f21f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.496+0000 7f221da00640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21f8066f60 0x7f21f8069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f220c0ad000 tx=0x7f220c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.500+0000 7f2215600640 1 -- 172.21.15.87:0/2033119302 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2210054ee0 con 0x7f22181626c0 2024-04-25T10:12:36.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.652+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2218074160 con 0x7f22181626c0 2024-04-25T10:12:36.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.652+0000 7f2215600640 1 -- 172.21.15.87:0/2033119302 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2210058f80 con 0x7f22181626c0 2024-04-25T10:12:36.658 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:36.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21f8066f60 msgr2=0x7f21f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21f8066f60 0x7f21f8069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f220c0ad000 tx=0x7f220c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22181626c0 msgr2=0x7f221815cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22181626c0 0x7f221815cb70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f221000c700 tx=0x7f221000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 shutdown_connections 2024-04-25T10:12:36.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21f8066f60 0x7f21f8069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22181626c0 0x7f221815cb70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22180d0d00 0x7f221815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 --2- 172.21.15.87:0/2033119302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22180cc200 0x7f2218154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 >> 172.21.15.87:0/2033119302 conn(0x7f2218072290 msgr2=0x7f22180ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 shutdown_connections 2024-04-25T10:12:36.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.656+0000 7f2220400640 1 -- 172.21.15.87:0/2033119302 wait complete. 2024-04-25T10:12:36.679 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:36.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe596000640 1 Processor -- start 2024-04-25T10:12:36.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe596000640 1 -- start start 2024-04-25T10:12:36.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe596000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 0x7fe5900aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe596000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe59006e690 0x7fe5900b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe596000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5900b39f0 con 0x7fe5900b2fa0 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe596000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59015e1d0 con 0x7fe59006e690 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe596000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900ae250 con 0x7fe59015ddf0 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe586a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe5900b2fa0 0x7fe59006df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60730/0 (socket says 172.21.15.87:60730) 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe586a00640 1 -- 172.21.15.87:0/2698037983 learned_addr learned my addr 172.21.15.87:0/2698037983 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58f400640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 0x7fe5900aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58fe00640 1 --2- 172.21.15.87:0/2698037983 >> v2:172.21.15.169:3300/0 conn(0x7fe59006e690 0x7fe5900b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195892123 0 0) 0x7fe5900b39f0 con 0x7fe5900b2fa0 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe574003530 con 0x7fe5900b2fa0 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2164901010 0 0) 0x7fe574003530 con 0x7fe5900b2fa0 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 msgr2=0x7fe5900aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 0x7fe5900aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 >> v2:172.21.15.169:3300/0 conn(0x7fe59006e690 msgr2=0x7fe5900b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 --2- 172.21.15.87:0/2698037983 >> v2:172.21.15.169:3300/0 conn(0x7fe59006e690 0x7fe5900b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900ae4a0 con 0x7fe5900b2fa0 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe578001680 con 0x7fe5900b2fa0 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 >> v1:172.21.15.87:6789/0 conn(0x7fe5900b2fa0 legacy=0x7fe59006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe574003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 0x7fe574004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574004d50 0x7fe5740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59015e1d0 con 0x7fe5740039c0 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900ae250 con 0x7fe574004d50 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57400b9d0 con 0x7fe59015ddf0 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe58f400640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe574003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.808+0000 7fe586a00640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 0x7fe574004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58fe00640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574004d50 0x7fe5740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58f400640 1 -- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 msgr2=0x7fe574004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58f400640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59015ddf0 0x7fe574004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58f400640 1 -- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574004d50 msgr2=0x7fe5740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58f400640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574004d50 0x7fe5740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58f400640 1 -- 172.21.15.87:0/2698037983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574003530 con 0x7fe5740039c0 2024-04-25T10:12:36.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58f400640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe574003de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe580009fc0 tx=0x7fe58001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a3e8e679905ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800b2070 con 0x7fe5740039c0 2024-04-25T10:12:36.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5800a5c40 con 0x7fe5740039c0 2024-04-25T10:12:36.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58ea00640 1 -- 172.21.15.87:0/2698037983 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580009c10 con 0x7fe5740039c0 2024-04-25T10:12:36.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe596000640 1 -- 172.21.15.87:0/2698037983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 msgr2=0x7fe574003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe596000640 1 --2- 172.21.15.87:0/2698037983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe574003de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe580009fc0 tx=0x7fe58001cc50 comp rx=0 tx=0).stop 2024-04-25T10:12:36.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe58fe00640 1 -- 172.21.15.87:0/2698037983 reap_dead start 2024-04-25T10:12:36.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe596000640 1 -- 172.21.15.87:0/2698037983 shutdown_connections 2024-04-25T10:12:36.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe596000640 1 -- 172.21.15.87:0/2698037983 >> 172.21.15.87:0/2698037983 conn(0x7fe5900b1ca0 msgr2=0x7fe5900b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe596000640 1 -- 172.21.15.87:0/2698037983 shutdown_connections 2024-04-25T10:12:36.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.812+0000 7fe596000640 1 -- 172.21.15.87:0/2698037983 wait complete. 2024-04-25T10:12:36.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 Processor -- start 2024-04-25T10:12:36.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 -- start start 2024-04-25T10:12:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe5901448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe590158940 0x7fe5901590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5901595e0 0x7fe5901e9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe590070cd0 con 0x7fe5740039c0 2024-04-25T10:12:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe590070b50 con 0x7fe5901595e0 2024-04-25T10:12:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe590070e50 con 0x7fe590158940 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe5901448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe5901448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57604/0 (socket says 172.21.15.87:57604) 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 -- 172.21.15.87:0/3995775740 learned_addr learned my addr 172.21.15.87:0/3995775740 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe586a00640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe590158940 0x7fe5901590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58fe00640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5901595e0 0x7fe5901e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 -- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe590158940 msgr2=0x7fe5901590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe590158940 0x7fe5901590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 -- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5901595e0 msgr2=0x7fe5901e9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5901595e0 0x7fe5901e9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 -- 172.21.15.87:0/3995775740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590159b60 con 0x7fe5740039c0 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe586a00640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe590158940 0x7fe5901590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe58f400640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe5901448a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe5800b0040 tx=0x7fe580009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe587e00640 1 -- 172.21.15.87:0/3995775740 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800b2040 con 0x7fe5740039c0 2024-04-25T10:12:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe587e00640 1 -- 172.21.15.87:0/3995775740 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5800a5a40 con 0x7fe5740039c0 2024-04-25T10:12:36.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe587e00640 1 -- 172.21.15.87:0/3995775740 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800adbe0 con 0x7fe5740039c0 2024-04-25T10:12:36.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590157e60 con 0x7fe5740039c0 2024-04-25T10:12:36.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.816+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe590158420 con 0x7fe5740039c0 2024-04-25T10:12:36.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.820+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59004ef30 con 0x7fe5740039c0 2024-04-25T10:12:36.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.820+0000 7fe587e00640 1 -- 172.21.15.87:0/3995775740 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5800bf030 con 0x7fe5740039c0 2024-04-25T10:12:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.820+0000 7fe587e00640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe56c066f90 0x7fe56c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.820+0000 7fe587e00640 1 -- 172.21.15.87:0/3995775740 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe58011f320 con 0x7fe5740039c0 2024-04-25T10:12:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.820+0000 7fe586a00640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe56c066f90 0x7fe56c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.820+0000 7fe586a00640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe56c066f90 0x7fe56c069450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe5901538c0 tx=0x7fe578009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:36.824+0000 7fe587e00640 1 -- 172.21.15.87:0/3995775740 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5800c6060 con 0x7fe5740039c0 2024-04-25T10:12:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.008+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe59015d4a0 con 0x7fe5740039c0 2024-04-25T10:12:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.008+0000 7fe587e00640 1 -- 172.21.15.87:0/3995775740 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fe580016990 con 0x7fe5740039c0 2024-04-25T10:12:37.014 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:37.014 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:12:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:12:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe56c066f90 msgr2=0x7fe56c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe56c066f90 0x7fe56c069450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe5901538c0 tx=0x7fe578009040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 msgr2=0x7fe5901448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe5901448a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe5800b0040 tx=0x7fe580009750 comp rx=0 tx=0).stop 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 shutdown_connections 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe56c066f90 0x7fe56c069450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5901595e0 0x7fe5901e9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe590158940 0x7fe5901590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 --2- 172.21.15.87:0/3995775740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5740039c0 0x7fe5901448a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 >> 172.21.15.87:0/3995775740 conn(0x7fe5900b1ca0 msgr2=0x7fe590153ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 shutdown_connections 2024-04-25T10:12:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.012+0000 7fe596000640 1 -- 172.21.15.87:0/3995775740 wait complete. 2024-04-25T10:12:37.036 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:37.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.164+0000 7f94da400640 1 Processor -- start 2024-04-25T10:12:37.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.164+0000 7f94da400640 1 -- start start 2024-04-25T10:12:37.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94da400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f94d415dc10 0x7f94d40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94da400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d406bdb0 0x7f94d40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94da400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d40cbff0 con 0x7f94d415dff0 2024-04-25T10:12:37.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94da400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d40b45a0 con 0x7f94d415dc10 2024-04-25T10:12:37.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94da400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40cbda0 con 0x7f94d406bdb0 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f94d415dff0 0x7f94d406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60746/0 (socket says 172.21.15.87:60746) 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d3e00640 1 -- 172.21.15.87:0/134368852 learned_addr learned my addr 172.21.15.87:0/134368852 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d9a00640 1 --2- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d406bdb0 0x7f94d40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d9000640 1 --2- 172.21.15.87:0/134368852 >> v2:172.21.15.169:3300/0 conn(0x7f94d415dc10 0x7f94d40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d3400640 1 -- 172.21.15.87:0/134368852 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122329223 0 0) 0x7f94d40cbff0 con 0x7f94d415dff0 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d3400640 1 -- 172.21.15.87:0/134368852 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94bc003530 con 0x7f94d415dff0 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d9000640 1 -- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d406bdb0 msgr2=0x7f94d40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d9000640 1 --2- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d406bdb0 0x7f94d40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d9000640 1 -- 172.21.15.87:0/134368852 >> v1:172.21.15.87:6789/0 conn(0x7f94d415dff0 legacy=0x7f94d406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.168+0000 7f94d9000640 1 -- 172.21.15.87:0/134368852 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d40cc2d0 con 0x7f94d415dc10 2024-04-25T10:12:37.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94d9000640 1 --2- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dc10 0x7f94d40d04a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94c80097c0 tx=0x7f94c801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=319fe4462a08846f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94d3400640 1 -- 172.21.15.87:0/134368852 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c80a7070 con 0x7f94d415dc10 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94d3400640 1 -- 172.21.15.87:0/134368852 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c80b0070 con 0x7f94d415dc10 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94d3400640 1 -- 172.21.15.87:0/134368852 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c80ab890 con 0x7f94d415dc10 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 -- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dc10 msgr2=0x7f94d40d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 --2- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dc10 0x7f94d40d04a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94c80097c0 tx=0x7f94c801c470 comp rx=0 tx=0).stop 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 -- 172.21.15.87:0/134368852 shutdown_connections 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 --2- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d406bdb0 0x7f94d40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 --2- 172.21.15.87:0/134368852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dc10 0x7f94d40d04a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 -- 172.21.15.87:0/134368852 >> 172.21.15.87:0/134368852 conn(0x7f94d40d1750 msgr2=0x7f94d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 -- 172.21.15.87:0/134368852 shutdown_connections 2024-04-25T10:12:37.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 -- 172.21.15.87:0/134368852 wait complete. 2024-04-25T10:12:37.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 Processor -- start 2024-04-25T10:12:37.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.172+0000 7f94da400640 1 -- start start 2024-04-25T10:12:37.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94d406bdb0 0x7f94d40b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d415dc10 0x7f94d415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dff0 0x7f94d40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40cdc80 con 0x7f94d406bdb0 2024-04-25T10:12:37.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40cdb00 con 0x7f94d415dff0 2024-04-25T10:12:37.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40cde00 con 0x7f94d415dc10 2024-04-25T10:12:37.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94d406bdb0 0x7f94d40b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94d406bdb0 0x7f94d40b6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57618/0 (socket says 172.21.15.87:57618) 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 -- 172.21.15.87:0/279091761 learned_addr learned my addr 172.21.15.87:0/279091761 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d3e00640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d415dc10 0x7f94d415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9a00640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dff0 0x7f94d40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 -- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d415dc10 msgr2=0x7f94d415ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d415dc10 0x7f94d415ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 -- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dff0 msgr2=0x7f94d40c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dff0 0x7f94d40c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 -- 172.21.15.87:0/279091761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d415b860 con 0x7f94d406bdb0 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d9000640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94d406bdb0 0x7f94d40b6f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94c80ad040 tx=0x7f94c801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d0c00640 1 -- 172.21.15.87:0/279091761 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c80bf610 con 0x7f94d406bdb0 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d0c00640 1 -- 172.21.15.87:0/279091761 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c80abe50 con 0x7f94d406bdb0 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94d0c00640 1 -- 172.21.15.87:0/279091761 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c80bfa10 con 0x7f94d406bdb0 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d415bb40 con 0x7f94d406bdb0 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d4152fb0 con 0x7f94d406bdb0 2024-04-25T10:12:37.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.176+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4075120 con 0x7f94d406bdb0 2024-04-25T10:12:37.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.180+0000 7f94d0c00640 1 -- 172.21.15.87:0/279091761 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94c80bd020 con 0x7f94d406bdb0 2024-04-25T10:12:37.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.180+0000 7f94d0c00640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94b4066f60 0x7f94b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.180+0000 7f94d0c00640 1 -- 172.21.15.87:0/279091761 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94c811f3d0 con 0x7f94d406bdb0 2024-04-25T10:12:37.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.180+0000 7f94d3e00640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94b4066f60 0x7f94b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.180+0000 7f94d3e00640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94b4066f60 0x7f94b4069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f94d413e840 tx=0x7f94c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.184+0000 7f94d0c00640 1 -- 172.21.15.87:0/279091761 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94c80c6030 con 0x7f94d406bdb0 2024-04-25T10:12:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94d4072f20 con 0x7f94d406bdb0 2024-04-25T10:12:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94d0c00640 1 -- 172.21.15.87:0/279091761 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f94c8016090 con 0x7f94d406bdb0 2024-04-25T10:12:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:37.346 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94b4066f60 msgr2=0x7f94b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94b4066f60 0x7f94b4069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f94d413e840 tx=0x7f94c0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94d406bdb0 msgr2=0x7f94d40b6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94d406bdb0 0x7f94d40b6f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94c80ad040 tx=0x7f94c801acb0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 shutdown_connections 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94b4066f60 0x7f94b4069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94d415dff0 0x7f94d40c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d415dc10 0x7f94d415ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 --2- 172.21.15.87:0/279091761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94d406bdb0 0x7f94d40b6f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.340+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 >> 172.21.15.87:0/279091761 conn(0x7f94d40d1750 msgr2=0x7f94d406f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.344+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 shutdown_connections 2024-04-25T10:12:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.344+0000 7f94da400640 1 -- 172.21.15.87:0/279091761 wait complete. 2024-04-25T10:12:37.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:37.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.496+0000 7f6282200640 1 Processor -- start 2024-04-25T10:12:37.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.496+0000 7f6282200640 1 -- start start 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6282200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f627c0cda10 0x7f627c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6282200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f627c0cf930 0x7f627c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6282200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c0b2770 con 0x7f627c15bc40 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6282200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c06c670 con 0x7f627c0cda10 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6282200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c06c950 con 0x7f627c0cf930 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6272a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f627c0cda10 0x7f627c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f627b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f627c15bc40 0x7f627c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37014/0 (socket says 172.21.15.87:37014) 2024-04-25T10:12:37.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f627b400640 1 -- 172.21.15.87:0/2330594256 learned_addr learned my addr 172.21.15.87:0/2330594256 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6272a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f627c0cda10 0x7f627c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54046/0 (socket says 172.21.15.87:54046) 2024-04-25T10:12:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f627be00640 1 --2- 172.21.15.87:0/2330594256 >> v2:172.21.15.169:3300/0 conn(0x7f627c0cf930 0x7f627c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6272a00640 1 -- 172.21.15.87:0/2330594256 >> v2:172.21.15.169:3300/0 conn(0x7f627c0cf930 msgr2=0x7f627c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6272a00640 1 --2- 172.21.15.87:0/2330594256 >> v2:172.21.15.169:3300/0 conn(0x7f627c0cf930 0x7f627c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6272a00640 1 -- 172.21.15.87:0/2330594256 >> v1:172.21.15.87:6789/0 conn(0x7f627c15bc40 legacy=0x7f627c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6272a00640 1 -- 172.21.15.87:0/2330594256 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c06cc30 con 0x7f627c0cda10 2024-04-25T10:12:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f627be00640 1 --2- 172.21.15.87:0/2330594256 >> v2:172.21.15.169:3300/0 conn(0x7f627c0cf930 0x7f627c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:37.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.500+0000 7f6272a00640 1 --2- 172.21.15.87:0/2330594256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c0cda10 0x7f627c06c090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f627c0ae4c0 tx=0x7f626401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc413177072eb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f627aa00640 1 -- 172.21.15.87:0/2330594256 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640b0070 con 0x7f627c0cda10 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f627aa00640 1 -- 172.21.15.87:0/2330594256 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62640ab600 con 0x7f627c0cda10 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f627aa00640 1 -- 172.21.15.87:0/2330594256 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640ab9d0 con 0x7f627c0cda10 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- 172.21.15.87:0/2330594256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c0cda10 msgr2=0x7f627c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 --2- 172.21.15.87:0/2330594256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c0cda10 0x7f627c06c090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f627c0ae4c0 tx=0x7f626401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- 172.21.15.87:0/2330594256 shutdown_connections 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 --2- 172.21.15.87:0/2330594256 >> v2:172.21.15.169:3300/0 conn(0x7f627c0cf930 0x7f627c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 --2- 172.21.15.87:0/2330594256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c0cda10 0x7f627c06c090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- 172.21.15.87:0/2330594256 >> 172.21.15.87:0/2330594256 conn(0x7f627c066b70 msgr2=0x7f627c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- 172.21.15.87:0/2330594256 shutdown_connections 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- 172.21.15.87:0/2330594256 wait complete. 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 Processor -- start 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- start start 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f627c0cda10 0x7f627c15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f627c0cf930 0x7f627c143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c15bc40 0x7f627c1440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0b3850 con 0x7f627c15bc40 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0b36d0 con 0x7f627c0cda10 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6282200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0b39d0 con 0x7f627c0cf930 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f627be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c15bc40 0x7f627c1440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f627be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c15bc40 0x7f627c1440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54056/0 (socket says 172.21.15.87:54056) 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f627be00640 1 -- 172.21.15.87:0/2527892815 learned_addr learned my addr 172.21.15.87:0/2527892815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f627b400640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f627c0cda10 0x7f627c15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.504+0000 7f6272a00640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f627c0cf930 0x7f627c143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f627be00640 1 -- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f627c0cf930 msgr2=0x7f627c143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f627be00640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f627c0cf930 0x7f627c143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f627be00640 1 -- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f627c0cda10 msgr2=0x7f627c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f627be00640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f627c0cda10 0x7f627c15e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f627be00640 1 -- 172.21.15.87:0/2527892815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c158e70 con 0x7f627c15bc40 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f627be00640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c15bc40 0x7f627c1440f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f627400cd80 tx=0x7f627400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6273e00640 1 -- 172.21.15.87:0/2527892815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400f680 con 0x7f627c15bc40 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6273e00640 1 -- 172.21.15.87:0/2527892815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627400f840 con 0x7f627c15bc40 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6273e00640 1 -- 172.21.15.87:0/2527892815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400fc30 con 0x7f627c15bc40 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c158b90 con 0x7f627c15bc40 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c151b20 con 0x7f627c15bc40 2024-04-25T10:12:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c04ef30 con 0x7f627c15bc40 2024-04-25T10:12:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6273e00640 1 -- 172.21.15.87:0/2527892815 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f627400e3e0 con 0x7f627c15bc40 2024-04-25T10:12:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6273e00640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f6273e00640 1 -- 172.21.15.87:0/2527892815 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6274085d40 con 0x7f627c15bc40 2024-04-25T10:12:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.508+0000 7f627b400640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.512+0000 7f627b400640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f626c000f70 tx=0x7f626c007e90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.512+0000 7f6273e00640 1 -- 172.21.15.87:0/2527892815 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6274014030 con 0x7f627c15bc40 2024-04-25T10:12:37.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.700+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f627c15b400 con 0x7f627c15bc40 2024-04-25T10:12:37.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.700+0000 7f6273e00640 1 -- 172.21.15.87:0/2527892815 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f6274056660 con 0x7f627c15bc40 2024-04-25T10:12:37.706 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:37.706 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:12:37.707 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:12:37.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6258066f60 msgr2=0x7f6258069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f626c000f70 tx=0x7f626c007e90 comp rx=0 tx=0).stop 2024-04-25T10:12:37.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c15bc40 msgr2=0x7f627c1440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c15bc40 0x7f627c1440f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f627400cd80 tx=0x7f627400a650 comp rx=0 tx=0).stop 2024-04-25T10:12:37.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 shutdown_connections 2024-04-25T10:12:37.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f627c15bc40 0x7f627c1440f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f627c0cf930 0x7f627c143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 --2- 172.21.15.87:0/2527892815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f627c0cda10 0x7f627c15e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 >> 172.21.15.87:0/2527892815 conn(0x7f627c066b70 msgr2=0x7f627c0af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 shutdown_connections 2024-04-25T10:12:37.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:37.704+0000 7f6282200640 1 -- 172.21.15.87:0/2527892815 wait complete. 2024-04-25T10:12:38.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:23.612551+0000', 'modified': '2024-04-25T10:12:26.623883+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/531083715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 531083715}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 531083715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:38.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:38.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:12:38.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f2c00640 1 Processor -- start 2024-04-25T10:12:38.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f2c00640 1 -- start start 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 0x7f99ec15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f2c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ec071050 con 0x7f99ec1550b0 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f2c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ec15d040 con 0x7f99ec15b740 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0712f0 con 0x7f99ec0d0040 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99ebe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f99ec1550b0 0x7f99ec0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37020/0 (socket says 172.21.15.87:37020) 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99ebe00640 1 -- 172.21.15.87:0/344604593 learned_addr learned my addr 172.21.15.87:0/344604593 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941293368 0 0) 0x7f99ec071050 con 0x7f99ec1550b0 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99d0005740 con 0x7f99ec1550b0 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99e3400640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 0x7f99ec15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855966789 0 0) 0x7f99d0005740 con 0x7f99ec1550b0 2024-04-25T10:12:38.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 msgr2=0x7f99ec15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 0x7f99ec15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 >> v1:172.21.15.169:6789/0 conn(0x7f99ec15b740 legacy=0x7f99ec0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0715d0 con 0x7f99ec1550b0 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f99dc001680 con 0x7f99ec1550b0 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 >> v1:172.21.15.87:6789/0 conn(0x7f99ec1550b0 legacy=0x7f99ec0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d0005c00 0x7f99d0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 0x7f99d0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99d0006f60 0x7f99d00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0712f0 con 0x7f99d0006f60 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d000dd20 con 0x7f99d0005c00 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d000df20 con 0x7f99ec0d0040 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f0c00640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99d0006f60 0x7f99d00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99e3400640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 0x7f99d0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99ebe00640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d0005c00 0x7f99d0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f0c00640 1 -- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 msgr2=0x7f99d0006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f0c00640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ec0d0040 0x7f99d0006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f0c00640 1 -- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d0005c00 msgr2=0x7f99d0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f0c00640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d0005c00 0x7f99d0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.264+0000 7f99f0c00640 1 -- 172.21.15.87:0/344604593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0005740 con 0x7f99d0006f60 2024-04-25T10:12:38.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f0c00640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99d0006f60 0x7f99d00073e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007fd0 tx=0x7f99e401c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7c14e22e8f066e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40b2070 con 0x7f99d0006f60 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e40a5730 con 0x7f99d0006f60 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99eb400640 1 -- 172.21.15.87:0/344604593 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4001b60 con 0x7f99d0006f60 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f2c00640 1 -- 172.21.15.87:0/344604593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99d0006f60 msgr2=0x7f99d00073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f2c00640 1 --2- 172.21.15.87:0/344604593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99d0006f60 0x7f99d00073e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007fd0 tx=0x7f99e401c840 comp rx=0 tx=0).stop 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f0c00640 1 -- 172.21.15.87:0/344604593 reap_dead start 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f2c00640 1 -- 172.21.15.87:0/344604593 shutdown_connections 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f2c00640 1 -- 172.21.15.87:0/344604593 >> 172.21.15.87:0/344604593 conn(0x7f99ec066b70 msgr2=0x7f99ec066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f2c00640 1 -- 172.21.15.87:0/344604593 shutdown_connections 2024-04-25T10:12:38.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.268+0000 7f99f2c00640 1 -- 172.21.15.87:0/344604593 wait complete. 2024-04-25T10:12:38.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 Processor -- start 2024-04-25T10:12:38.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 -- start start 2024-04-25T10:12:38.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99d0006f60 0x7f99ec157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ec0b3680 0x7f99ec148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99ec06c950 0x7f99ec06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0720b0 con 0x7f99ec06c950 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec071b90 con 0x7f99ec0b3680 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec15bbf0 con 0x7f99d0006f60 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99ec06c950 0x7f99ec06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99ec06c950 0x7f99ec06cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54076/0 (socket says 172.21.15.87:54076) 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 -- 172.21.15.87:0/1188446720 learned_addr learned my addr 172.21.15.87:0/1188446720 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99e3400640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ec0b3680 0x7f99ec148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99ebe00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99d0006f60 0x7f99ec157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 -- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99d0006f60 msgr2=0x7f99ec157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99d0006f60 0x7f99ec157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 -- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ec0b3680 msgr2=0x7f99ec148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ec0b3680 0x7f99ec148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 -- 172.21.15.87:0/1188446720 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec151760 con 0x7f99ec06c950 2024-04-25T10:12:38.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f0c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99ec06c950 0x7f99ec06cdd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f99e4098250 tx=0x7f99d0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99e8c00640 1 -- 172.21.15.87:0/1188446720 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40bb070 con 0x7f99ec06c950 2024-04-25T10:12:38.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99e8c00640 1 -- 172.21.15.87:0/1188446720 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e40a5b00 con 0x7f99ec06c950 2024-04-25T10:12:38.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99e8c00640 1 -- 172.21.15.87:0/1188446720 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40b2040 con 0x7f99ec06c950 2024-04-25T10:12:38.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ec151a40 con 0x7f99ec06c950 2024-04-25T10:12:38.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.272+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ec152000 con 0x7f99ec06c950 2024-04-25T10:12:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99e8c00640 1 -- 172.21.15.87:0/1188446720 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99e40be070 con 0x7f99ec06c950 2024-04-25T10:12:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99e8c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f99c8066f60 0x7f99c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99e8c00640 1 -- 172.21.15.87:0/1188446720 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99e411e410 con 0x7f99ec06c950 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99ebe00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f99c8066f60 0x7f99c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 Processor -- start 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 -- start start 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99ebe00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f99c8066f60 0x7f99c8069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0054a0 tx=0x7f99dc001440 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab732cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732d240 0x558ab72e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e2f30 0x558ab72e3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab71ca5f0 con 0x558ab72e2f30 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab7131da0 con 0x558ab732c920 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.276+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ab70ba150 con 0x558ab732d240 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e2f30 0x558ab72e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e2f30 0x558ab72e3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54086/0 (socket says 172.21.15.87:54086) 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 -- 172.21.15.87:0/1172819245 learned_addr learned my addr 172.21.15.87:0/1172819245 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99e3400640 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732d240 0x558ab72e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99ebe00640 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab732cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 -- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732d240 msgr2=0x558ab72e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732d240 0x558ab72e29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 -- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 msgr2=0x558ab732cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab732cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 -- 172.21.15.87:0/1172819245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ab70d3ca0 con 0x558ab72e2f30 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e2f30 0x558ab72e3310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99e40f6ef0 tx=0x7f99e40ee7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c5223d17848658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99e0c00640 1 -- 172.21.15.87:0/1172819245 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40f2e60 con 0x558ab72e2f30 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99e0c00640 1 -- 172.21.15.87:0/1172819245 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e40f7420 con 0x558ab72e2f30 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99e0c00640 1 -- 172.21.15.87:0/1172819245 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40ed0c0 con 0x558ab72e2f30 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- 172.21.15.87:0/1172819245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e2f30 msgr2=0x558ab72e3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e2f30 0x558ab72e3310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99e40f6ef0 tx=0x7f99e40ee7e0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- 172.21.15.87:0/1172819245 shutdown_connections 2024-04-25T10:12:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e2f30 0x558ab72e3310 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732d240 0x558ab72e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 --2- 172.21.15.87:0/1172819245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab732cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- 172.21.15.87:0/1172819245 >> 172.21.15.87:0/1172819245 conn(0x558ab72e92a0 msgr2=0x558ab72e12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- 172.21.15.87:0/1172819245 shutdown_connections 2024-04-25T10:12:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- 172.21.15.87:0/1172819245 wait complete. 2024-04-25T10:12:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- start start 2024-04-25T10:12:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 Processor -- start 2024-04-25T10:12:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e2f30 0x558ab7333c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732c920 0x558ab73341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732d240 0x558ab72e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab70ec990 con 0x558ab732d240 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab712de40 con 0x558ab72e2f30 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ab70bd900 con 0x558ab732c920 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f6105000 10 client.0 _ll_register_callbacks cb 0x558ab71e51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732d240 0x558ab72e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732d240 0x558ab72e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54094/0 (socket says 172.21.15.87:54094) 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.280+0000 7f99f0c00640 1 -- 172.21.15.87:0/489507410 learned_addr learned my addr 172.21.15.87:0/489507410 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f0c00640 1 -- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732c920 msgr2=0x558ab73341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99e3400640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732c920 0x558ab73341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99ebe00640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e2f30 0x558ab7333c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f0c00640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732c920 0x558ab73341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f0c00640 1 -- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e2f30 msgr2=0x558ab7333c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f0c00640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e2f30 0x558ab7333c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f0c00640 1 -- 172.21.15.87:0/489507410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ab70ea630 con 0x558ab732d240 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99e3400640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732c920 0x558ab73341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99ebe00640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e2f30 0x558ab7333c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f0c00640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732d240 0x558ab72e6aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99e4001780 tx=0x7f99e4007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c99b1b78a5ef697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 1 -- 172.21.15.87:0/489507410 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4106040 con 0x558ab732d240 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 1 -- 172.21.15.87:0/489507410 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99e40a5cb0 con 0x558ab732d240 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99e40f7420 con 0x558ab732d240 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 1 -- 172.21.15.87:0/489507410 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e40f2e60 con 0x558ab732d240 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 10 client.4656 fetch_fsmap learned FSMap version 8 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x558ab70ba3a0 con 0x558ab732d240 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 1 -- 172.21.15.87:0/489507410 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e40f7420 con 0x558ab732d240 2024-04-25T10:12:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 1 -- 172.21.15.87:0/489507410 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99e40b5070 con 0x558ab732d240 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 10 client.4656 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 20 client.4656 populate_metadata read hostname 'smithi087' 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732c500 0x558ab72e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 4 client.4656 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 --> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ab70cda50 con 0x558ab732c500 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99ebe00640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732c500 0x558ab72e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 10 client.4656 ms_handle_connect on v2:172.21.15.87:6835/531083715 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99ebe00640 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732c500 0x558ab72e74b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480d125f042f3cf7 server_cookie=c3315ff8b065400a in_seq=0 out_seq=0 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 1 -- 172.21.15.87:0/489507410 <== mds.0 v2:172.21.15.87:6835/531083715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ab70cda50 con 0x558ab732c500 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 10 client.4656 handle_command_reply: tid=0 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 client.4656 shutdown 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732d240 msgr2=0x558ab72e6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732d240 0x558ab72e6aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99e4001780 tx=0x7f99e4007e60 comp rx=0 tx=0).stop 2024-04-25T10:12:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 shutdown_connections 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732c500 0x558ab72e74b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732d240 0x558ab72e6aa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab732c920 0x558ab73341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99d9600640 0 client.4656 ms_handle_reset on v2:172.21.15.87:6835/531083715 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 --2- 172.21.15.87:0/489507410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e2f30 0x558ab7333c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 >> 172.21.15.87:0/489507410 conn(0x558ab72e92a0 msgr2=0x558ab72e19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 shutdown_connections 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 1 -- 172.21.15.87:0/489507410 wait complete. 2024-04-25T10:12:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.284+0000 7f99f6105000 20 client.4656 trim_cache size 0 max 16384 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 Processor -- start 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 -- start start 2024-04-25T10:12:38.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e92a0 0x558ab7318ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7319010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732c920 0x558ab72e5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec1ff650 con 0x558ab732c920 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec157070 con 0x558ab72e92a0 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec157670 con 0x558ab72e2f30 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732c920 0x558ab72e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732c920 0x558ab72e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54102/0 (socket says 172.21.15.87:54102) 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99f0c00640 1 -- 172.21.15.87:0/2014871572 learned_addr learned my addr 172.21.15.87:0/2014871572 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99ebe00640 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e92a0 0x558ab7318ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.300+0000 7f99e3400640 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7319010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f0c00640 1 -- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 msgr2=0x558ab7319010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f0c00640 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7319010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f0c00640 1 -- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e92a0 msgr2=0x558ab7318ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f0c00640 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e92a0 0x558ab7318ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f0c00640 1 -- 172.21.15.87:0/2014871572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ab732d070 con 0x558ab732c920 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f0c00640 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732c920 0x558ab72e5320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f99e40eeac0 tx=0x7f99e40f1070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b80f76f71e7ffc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99dbe00640 1 -- 172.21.15.87:0/2014871572 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4106040 con 0x558ab732c920 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99dbe00640 1 -- 172.21.15.87:0/2014871572 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e411e0c0 con 0x558ab732c920 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99dbe00640 1 -- 172.21.15.87:0/2014871572 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40f7060 con 0x558ab732c920 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- 172.21.15.87:0/2014871572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732c920 msgr2=0x558ab72e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732c920 0x558ab72e5320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f99e40eeac0 tx=0x7f99e40f1070 comp rx=0 tx=0).stop 2024-04-25T10:12:38.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- 172.21.15.87:0/2014871572 shutdown_connections 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab732c920 0x558ab72e5320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7319010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 --2- 172.21.15.87:0/2014871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab72e92a0 0x558ab7318ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- 172.21.15.87:0/2014871572 >> 172.21.15.87:0/2014871572 conn(0x558ab73343c0 msgr2=0x558ab72e86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- 172.21.15.87:0/2014871572 shutdown_connections 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- 172.21.15.87:0/2014871572 wait complete. 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- start start 2024-04-25T10:12:38.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 Processor -- start 2024-04-25T10:12:38.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7337c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e92a0 0x558ab73891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab7389720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab71363d0 con 0x558ab72e92a0 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab71edd50 con 0x558ab732c920 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ab70cc980 con 0x558ab72e2f30 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f6105000 10 client.0 _ll_register_callbacks cb 0x7f99d400b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e92a0 0x558ab73891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e92a0 0x558ab73891e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54114/0 (socket says 172.21.15.87:54114) 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 -- 172.21.15.87:0/3836736952 learned_addr learned my addr 172.21.15.87:0/3836736952 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99ebe00640 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7337c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99f0c00640 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab7389720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 -- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 msgr2=0x558ab7337c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7337c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 -- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 msgr2=0x558ab7389720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab7389720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.304+0000 7f99e3400640 1 -- 172.21.15.87:0/3836736952 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ab71b3600 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99e3400640 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e92a0 0x558ab73891e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99d4018700 tx=0x7f99d4018b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4490a775da46dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 1 -- 172.21.15.87:0/3836736952 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40166a0 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 1 -- 172.21.15.87:0/3836736952 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99d4016840 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99e411e0c0 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 1 -- 172.21.15.87:0/3836736952 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99d402c050 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 10 client.4662 fetch_fsmap learned FSMap version 8 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x558ab70c69b0 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 1 -- 172.21.15.87:0/3836736952 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e411e0c0 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 1 -- 172.21.15.87:0/3836736952 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99d402f020 con 0x558ab72e92a0 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:12:38.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 10 client.4662 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 20 client.4662 populate_metadata read hostname 'smithi087' 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732d240 0x558ab7389c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 4 client.4662 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 --> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558ab713f2e0 con 0x558ab732d240 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99ebe00640 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732d240 0x558ab7389c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99ebe00640 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732d240 0x558ab7389c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e9ad8cc95d6504 server_cookie=a30c50086e85eebf in_seq=0 out_seq=0 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 10 client.4662 ms_handle_connect on v2:172.21.15.87:6835/531083715 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 1 -- 172.21.15.87:0/3836736952 <== mds.0 v2:172.21.15.87:6835/531083715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x558ab713f2e0 con 0x558ab732d240 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 10 client.4662 handle_command_reply: tid=0 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 client.4662 shutdown 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e92a0 msgr2=0x558ab73891e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e92a0 0x558ab73891e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99d4018700 tx=0x7f99d4018b00 comp rx=0 tx=0).stop 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 shutdown_connections 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:6835/531083715,v1:172.21.15.87:6837/531083715] conn(0x558ab732d240 0x558ab7389c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ab732c920 0x558ab7389720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ab72e92a0 0x558ab73891e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 --2- 172.21.15.87:0/3836736952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ab72e2f30 0x558ab7337c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 >> 172.21.15.87:0/3836736952 conn(0x558ab73343c0 msgr2=0x558ab72e5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99d9600640 0 client.4662 ms_handle_reset on v2:172.21.15.87:6835/531083715 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 shutdown_connections 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 1 -- 172.21.15.87:0/3836736952 wait complete. 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f6105000 20 client.4662 trim_cache size 0 max 16384 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:12:38.320 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4287, 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 14.087616514, 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 8, 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 20, 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 21.669250819999998 2024-04-25T10:12:38.321 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f99c8066f60 msgr2=0x7f99c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f99c8066f60 0x7f99c8069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0054a0 tx=0x7f99dc001440 comp rx=0 tx=0).stop 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99ec06c950 msgr2=0x7f99ec06cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99ec06c950 0x7f99ec06cdd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f99e4098250 tx=0x7f99d0007a50 comp rx=0 tx=0).stop 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 shutdown_connections 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f99c8066f60 0x7f99c8069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99ec06c950 0x7f99ec06cdd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ec0b3680 0x7f99ec148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 --2- 172.21.15.87:0/1188446720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99d0006f60 0x7f99ec157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 >> 172.21.15.87:0/1188446720 conn(0x7f99ec066b70 msgr2=0x7f99ec0b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 shutdown_connections 2024-04-25T10:12:38.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.308+0000 7f99f2c00640 1 -- 172.21.15.87:0/1188446720 wait complete. 2024-04-25T10:12:38.333 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:12:38.359 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:12:38.359 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:12:38.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.360 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:12:38.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T10:12:38.437 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-04-25T10:12:38.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:38.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.439 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:12:38.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.492 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T10:12:38.558 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-04-25T10:12:38.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:38.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.561 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T10:12:38.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.571 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-25T10:12:38.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:38.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.640 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T10:12:38.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.695 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T10:12:38.762 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-25T10:12:38.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:38.764 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T10:12:38.764 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:12:38.764 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:38.765 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:38.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.892+0000 7ff2ccc00640 1 Processor -- start 2024-04-25T10:12:38.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.892+0000 7ff2ccc00640 1 -- start start 2024-04-25T10:12:38.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2ccc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff2c80b0f70 0x7ff2c80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 0x7ff2c80ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2ccc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff2c80cd110 0x7ff2c80cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2ccc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c815ef20 con 0x7ff2c80cd110 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2ccc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c815f3b0 con 0x7ff2c80b0f70 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2ccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80ca810 con 0x7ff2c80cfd90 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff2c80cd110 0x7ff2c80cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff2c80cd110 0x7ff2c80cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54128/0 (socket says 172.21.15.87:54128) 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 -- 172.21.15.87:0/1184305537 learned_addr learned my addr 172.21.15.87:0/1184305537 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c5400640 1 --2- 172.21.15.87:0/1184305537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 0x7ff2c80ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c5e00640 1 --2- 172.21.15.87:0/1184305537 >> v2:172.21.15.169:3300/0 conn(0x7ff2c80b0f70 0x7ff2c80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 -- 172.21.15.87:0/1184305537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 msgr2=0x7ff2c80ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 --2- 172.21.15.87:0/1184305537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 0x7ff2c80ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 -- 172.21.15.87:0/1184305537 >> v2:172.21.15.169:3300/0 conn(0x7ff2c80b0f70 msgr2=0x7ff2c80b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 --2- 172.21.15.87:0/1184305537 >> v2:172.21.15.169:3300/0 conn(0x7ff2c80b0f70 0x7ff2c80b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 -- 172.21.15.87:0/1184305537 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80b1890 con 0x7ff2c80cd110 2024-04-25T10:12:38.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.896+0000 7ff2c6800640 1 --2- 172.21.15.87:0/1184305537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c80cba20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8071230 tx=0x7ff2c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd8a5a1e962250b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c4a00640 1 -- 172.21.15.87:0/1184305537 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00b0070 con 0x7ff2c80cd110 2024-04-25T10:12:38.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c4a00640 1 -- 172.21.15.87:0/1184305537 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c00ab600 con 0x7ff2c80cd110 2024-04-25T10:12:38.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c4a00640 1 -- 172.21.15.87:0/1184305537 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00ab9d0 con 0x7ff2c80cd110 2024-04-25T10:12:38.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- 172.21.15.87:0/1184305537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 msgr2=0x7ff2c80cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/1184305537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c80cba20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8071230 tx=0x7ff2c001c520 comp rx=0 tx=0).stop 2024-04-25T10:12:38.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- 172.21.15.87:0/1184305537 shutdown_connections 2024-04-25T10:12:38.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/1184305537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c80cba20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/1184305537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 0x7ff2c80ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/1184305537 >> v2:172.21.15.169:3300/0 conn(0x7ff2c80b0f70 0x7ff2c80b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- 172.21.15.87:0/1184305537 >> 172.21.15.87:0/1184305537 conn(0x7ff2c80b1d50 msgr2=0x7ff2c80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- 172.21.15.87:0/1184305537 shutdown_connections 2024-04-25T10:12:38.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- 172.21.15.87:0/1184305537 wait complete. 2024-04-25T10:12:38.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 Processor -- start 2024-04-25T10:12:38.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- start start 2024-04-25T10:12:38.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2c80b0f70 0x7ff2c80c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c81606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 0x7ff2c8160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8072200 con 0x7ff2c80cd110 2024-04-25T10:12:38.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80cd580 con 0x7ff2c80b0f70 2024-04-25T10:12:38.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2ccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8072380 con 0x7ff2c80cfd90 2024-04-25T10:12:38.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c81606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c81606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54144/0 (socket says 172.21.15.87:54144) 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5400640 1 -- 172.21.15.87:0/2153544150 learned_addr learned my addr 172.21.15.87:0/2153544150 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5e00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2c80b0f70 0x7ff2c80c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5400640 1 -- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 msgr2=0x7ff2c8160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5400640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 0x7ff2c8160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5400640 1 -- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2c80b0f70 msgr2=0x7ff2c80c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.900+0000 7ff2c5400640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2c80b0f70 0x7ff2c80c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2c5400640 1 -- 172.21.15.87:0/2153544150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80c6a80 con 0x7ff2c80cd110 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2c5e00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2c80b0f70 0x7ff2c80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2c5400640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c81606a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400d5d0 tx=0x7ff2b400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2be000640 1 -- 172.21.15.87:0/2153544150 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4004240 con 0x7ff2c80cd110 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2be000640 1 -- 172.21.15.87:0/2153544150 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b40043e0 con 0x7ff2c80cd110 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2be000640 1 -- 172.21.15.87:0/2153544150 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4003840 con 0x7ff2c80cd110 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c80c67a0 con 0x7ff2c80cd110 2024-04-25T10:12:38.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c80ad420 con 0x7ff2c80cd110 2024-04-25T10:12:38.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c804ef30 con 0x7ff2c80cd110 2024-04-25T10:12:38.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2be000640 1 -- 172.21.15.87:0/2153544150 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2b40039e0 con 0x7ff2c80cd110 2024-04-25T10:12:38.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2be000640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff2a8066f60 0x7ff2a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2be000640 1 -- 172.21.15.87:0/2153544150 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2b40852a0 con 0x7ff2c80cd110 2024-04-25T10:12:38.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.904+0000 7ff2c5e00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff2a8066f60 0x7ff2a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.908+0000 7ff2c5e00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff2a8066f60 0x7ff2a8069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009f70 tx=0x7ff2b800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:38.908+0000 7ff2be000640 1 -- 172.21.15.87:0/2153544150 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2b401c360 con 0x7ff2c80cd110 2024-04-25T10:12:39.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2c80cda20 con 0x7ff2c80cd110 2024-04-25T10:12:39.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2be000640 1 -- 172.21.15.87:0/2153544150 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff2b40562a0 con 0x7ff2c80cd110 2024-04-25T10:12:39.101 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:39.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff2a8066f60 msgr2=0x7ff2a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff2a8066f60 0x7ff2a8069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009f70 tx=0x7ff2b800b040 comp rx=0 tx=0).stop 2024-04-25T10:12:39.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 msgr2=0x7ff2c81606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c81606a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400d5d0 tx=0x7ff2b400da90 comp rx=0 tx=0).stop 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 shutdown_connections 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff2a8066f60 0x7ff2a8069420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2c80cfd90 0x7ff2c8160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2c80cd110 0x7ff2c81606a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 --2- 172.21.15.87:0/2153544150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2c80b0f70 0x7ff2c80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 >> 172.21.15.87:0/2153544150 conn(0x7ff2c80b1d50 msgr2=0x7ff2c806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 shutdown_connections 2024-04-25T10:12:39.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:39.096+0000 7ff2ccc00640 1 -- 172.21.15.87:0/2153544150 wait complete. 2024-04-25T10:12:39.121 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:12:39.121 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:39.121 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:12:39.121 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:39.121 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:39.121 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:12:39.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:12:39.122 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:39.122 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:39.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.122 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:12:39.131 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:39.131 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:39.131 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:39.131 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:12:39.133 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42455sec preferred_lft 42455sec 2024-04-25T10:12:39.133 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:12:39.133 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:39.133 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:12:39.133 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:39.133 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:12:39.133 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:39.133 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:12:39.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.134 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:39.134 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:39.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:12:39.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-04-25T10:12:39.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-04-25T10:12:39.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:12:39.134 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:39.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.248 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:39.267 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:12:39.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.269 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:12:39.344 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:39.345 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:39.345 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:39.345 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:39.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.345 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:39.345 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:39.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:12:39.346 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:12:39.346 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:12:39.346 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:39.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.437 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:12:39.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.447 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-04-25T10:12:39.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.495 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:39.496 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:39.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:39.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:12:39.496 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:39.576 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:12:39.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:12:39.577 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:39.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.747 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:39.747 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:39.747 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:12:39.747 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:12:39.747 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:39.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:39.786 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:39.839 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:39.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:39.839 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:39.890 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:39.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.891 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:39.979 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:39.979 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:39.979 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:12:40.647 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:40.647 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:40.647 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:12:40.647 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:40.647 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:40.661 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:40.662 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:40.662 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:12:40.662 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:40.662 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:40.662 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:12:40.662 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:12:40.662 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:12:40.665 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:40.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:40.666 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:40.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:40.731 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:40.798 INFO:teuthology.orchestra.run.smithi087.stdout:client4582 2024-04-25T10:12:40.799 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:40.800 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:40.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:40.847 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:40.907 INFO:teuthology.orchestra.run.smithi087.stdout:client4582 2024-04-25T10:12:40.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:40.907 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:40.963 INFO:teuthology.orchestra.run.smithi087.stdout:client4582 2024-04-25T10:12:40.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:40.964 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/84c5cae3-3d36-4b8f-9ce8-349d9b4e086b.client4582/status') 2024-04-25T10:12:41.031 INFO:teuthology.orchestra.run.smithi087.stdout:instance: client.4582 (3)172.21.15.87:0/3775942443 2024-04-25T10:12:41.031 INFO:teuthology.orchestra.run.smithi087.stdout:blocklisted: false 2024-04-25T10:12:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records in 2024-04-25T10:12:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records out 2024-04-25T10:12:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:70 bytes copied, 5.2486e-05 s, 1.3 MB/s 2024-04-25T10:12:41.033 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T10:12:41.033 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:12:41.033 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:12:41.034 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:41.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4ccc00640 1 Processor -- start 2024-04-25T10:12:41.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4ccc00640 1 -- start start 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4ccc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4c8144270 0x7ff4c8071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4ccc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4c80b5f60 0x7ff4c806d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4ccc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c80cd930 con 0x7ff4c80cdba0 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4ccc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c8144650 con 0x7ff4c80b5f60 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4ccc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c8071240 con 0x7ff4c8144270 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4c80b5f60 0x7ff4c806d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4c8144270 0x7ff4c8071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4c80b5f60 0x7ff4c806d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54150/0 (socket says 172.21.15.87:54150) 2024-04-25T10:12:41.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7e00640 1 -- 172.21.15.87:0/4105927033 learned_addr learned my addr 172.21.15.87:0/4105927033 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:41.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7e00640 1 -- 172.21.15.87:0/4105927033 >> v2:172.21.15.169:3300/0 conn(0x7ff4c8144270 msgr2=0x7ff4c8071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7e00640 1 --2- 172.21.15.87:0/4105927033 >> v2:172.21.15.169:3300/0 conn(0x7ff4c8144270 0x7ff4c8071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7e00640 1 -- 172.21.15.87:0/4105927033 >> v1:172.21.15.169:6789/0 conn(0x7ff4c80cdba0 legacy=0x7ff4c80b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7e00640 1 -- 172.21.15.87:0/4105927033 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80cd370 con 0x7ff4c80b5f60 2024-04-25T10:12:41.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.212+0000 7ff4c7400640 1 --2- 172.21.15.87:0/4105927033 >> v2:172.21.15.169:3300/0 conn(0x7ff4c8144270 0x7ff4c8071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:41.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4c7e00640 1 --2- 172.21.15.87:0/4105927033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c806d3a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8070b20 tx=0x7ff4c001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2002f042c2abfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4c6a00640 1 -- 172.21.15.87:0/4105927033 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00b0070 con 0x7ff4c80b5f60 2024-04-25T10:12:41.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4c6a00640 1 -- 172.21.15.87:0/4105927033 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c00ab600 con 0x7ff4c80b5f60 2024-04-25T10:12:41.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4c6a00640 1 -- 172.21.15.87:0/4105927033 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00ab9d0 con 0x7ff4c80b5f60 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 -- 172.21.15.87:0/4105927033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 msgr2=0x7ff4c806d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/4105927033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c806d3a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8070b20 tx=0x7ff4c001c460 comp rx=0 tx=0).stop 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 -- 172.21.15.87:0/4105927033 shutdown_connections 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/4105927033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c806d3a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/4105927033 >> v2:172.21.15.169:3300/0 conn(0x7ff4c8144270 0x7ff4c8071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 -- 172.21.15.87:0/4105927033 >> 172.21.15.87:0/4105927033 conn(0x7ff4c80d0690 msgr2=0x7ff4c80d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 -- 172.21.15.87:0/4105927033 shutdown_connections 2024-04-25T10:12:41.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.216+0000 7ff4ccc00640 1 -- 172.21.15.87:0/4105927033 wait complete. 2024-04-25T10:12:41.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 Processor -- start 2024-04-25T10:12:41.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 -- start start 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c81fa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c80cdba0 0x7ff4c81565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4c8144270 0x7ff4c81638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c806ee30 con 0x7ff4c80b5f60 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c806ecb0 con 0x7ff4c80cdba0 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8071f40 con 0x7ff4c8144270 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c81fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c81fa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54158/0 (socket says 172.21.15.87:54158) 2024-04-25T10:12:41.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 -- 172.21.15.87:0/3509888616 learned_addr learned my addr 172.21.15.87:0/3509888616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7e00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4c8144270 0x7ff4c81638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4bea00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c80cdba0 0x7ff4c81565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 -- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4c8144270 msgr2=0x7ff4c81638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4c8144270 0x7ff4c81638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 -- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c80cdba0 msgr2=0x7ff4c81565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c80cdba0 0x7ff4c81565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 -- 172.21.15.87:0/3509888616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c8157090 con 0x7ff4c80b5f60 2024-04-25T10:12:41.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4c7400640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c81fa030 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800efc0 tx=0x7ff4b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4bfe00640 1 -- 172.21.15.87:0/3509888616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8018020 con 0x7ff4c80b5f60 2024-04-25T10:12:41.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4bfe00640 1 -- 172.21.15.87:0/3509888616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b8014680 con 0x7ff4c80b5f60 2024-04-25T10:12:41.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4bfe00640 1 -- 172.21.15.87:0/3509888616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8014a50 con 0x7ff4c80b5f60 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c8157370 con 0x7ff4c80b5f60 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.220+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c8157930 con 0x7ff4c80b5f60 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.224+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c8075120 con 0x7ff4c80b5f60 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.224+0000 7ff4bfe00640 1 -- 172.21.15.87:0/3509888616 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4b801b020 con 0x7ff4c80b5f60 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.224+0000 7ff4bfe00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4a4066f90 0x7ff4a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.224+0000 7ff4bfe00640 1 -- 172.21.15.87:0/3509888616 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4b8085d00 con 0x7ff4c80b5f60 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.224+0000 7ff4bea00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4a4066f90 0x7ff4a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.224+0000 7ff4bea00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4a4066f90 0x7ff4a4069450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b5750 tx=0x7ff4b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.228+0000 7ff4bfe00640 1 -- 172.21.15.87:0/3509888616 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4b8022030 con 0x7ff4c80b5f60 2024-04-25T10:12:41.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.412+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4c80b65c0 con 0x7ff4c80b5f60 2024-04-25T10:12:41.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.412+0000 7ff4bfe00640 1 -- 172.21.15.87:0/3509888616 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4b8010040 con 0x7ff4c80b5f60 2024-04-25T10:12:41.420 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:41.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4a4066f90 msgr2=0x7ff4a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4a4066f90 0x7ff4a4069450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b5750 tx=0x7ff4b0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 msgr2=0x7ff4c81fa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c81fa030 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800efc0 tx=0x7ff4b800c490 comp rx=0 tx=0).stop 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 shutdown_connections 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4a4066f90 0x7ff4a4069450 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4c8144270 0x7ff4c81638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c80cdba0 0x7ff4c81565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 --2- 172.21.15.87:0/3509888616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4c80b5f60 0x7ff4c81fa030 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 >> 172.21.15.87:0/3509888616 conn(0x7ff4c80d0690 msgr2=0x7ff4c815be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 shutdown_connections 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:41.416+0000 7ff4ccc00640 1 -- 172.21.15.87:0/3509888616 wait complete. 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:41.440 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:12:41.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.441 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:12:41.449 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:12:41.450 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42452sec preferred_lft 42452sec 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.451 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-04-25T10:12:41.452 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.452 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:41.452 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:41.452 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:12:41.452 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.453 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:41.512 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:12:41.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.514 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.573 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:41.573 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:41.573 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:12:41.573 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:12:41.573 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:12:41.573 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:41.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.664 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:12:41.719 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:41.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.720 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-04-25T10:12:41.770 INFO:teuthology.orchestra.run.smithi087.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:12:41.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.772 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:41.772 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:41.772 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:41.772 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:12:41.772 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:41.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:41.923 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether de:79:0f:06:21:92 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:12:41.924 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:12:41.924 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.924 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::dc79:fff:fe06:2192/64 scope link 2024-04-25T10:12:41.924 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.925 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether de:79:0f:06:21:92 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:12:41.987 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:12:41.988 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.988 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::dc79:fff:fe06:2192/64 scope link 2024-04-25T10:12:41.988 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:41.989 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:12:41.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.989 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:41.989 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:41.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:12:41.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:12:41.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:12:41.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:12:41.990 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:12:41.990 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:42.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.127 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:12:42.127 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:12:42.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.1 up 2024-04-25T10:12:42.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:12:42.127 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:12:42.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:42.165 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:12:42.215 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:12:42.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:42.215 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:12:42.267 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:42.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.267 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:12:42.359 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:12:42.359 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:12:42.359 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:12:42.374 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:12:42.374 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:12:42.374 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:12:42.374 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:42.374 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:12:42.384 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:12:42.388 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:42.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.388 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:12:42.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.407 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:42.459 INFO:teuthology.orchestra.run.smithi087.stdout:client4677 2024-04-25T10:12:42.459 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:42.459 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:42.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.508 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:42.562 INFO:teuthology.orchestra.run.smithi087.stdout:client4677 2024-04-25T10:12:42.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.562 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:42.613 INFO:teuthology.orchestra.run.smithi087.stdout:client4677 2024-04-25T10:12:42.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.614 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/84c5cae3-3d36-4b8f-9ce8-349d9b4e086b.client4677/status') 2024-04-25T10:12:42.674 INFO:teuthology.orchestra.run.smithi087.stdout:instance: client.4677 (3)192.168.0.2:0/4046370876 2024-04-25T10:12:42.675 INFO:teuthology.orchestra.run.smithi087.stdout:blocklisted: false 2024-04-25T10:12:42.675 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records in 2024-04-25T10:12:42.675 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records out 2024-04-25T10:12:42.675 INFO:teuthology.orchestra.run.smithi087.stderr:69 bytes copied, 4.9848e-05 s, 1.4 MB/s 2024-04-25T10:12:42.677 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T10:12:42.677 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:12:42.677 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:12:42.677 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:42.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc1be00640 1 Processor -- start 2024-04-25T10:12:42.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc1be00640 1 -- start start 2024-04-25T10:12:42.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc1be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbc140d0920 0x7fbc140d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc1be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140d2490 con 0x7fbc140d1ce0 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc1be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140d2770 con 0x7fbc1415df00 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc1be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc140d2a50 con 0x7fbc140d0920 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc18a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbc140d0920 0x7fbc140d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc18a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbc140d0920 0x7fbc140d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54166/0 (socket says 172.21.15.87:54166) 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc19e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbc140d1ce0 0x7fbc140b71e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37024/0 (socket says 172.21.15.87:37024) 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc18a00640 1 -- 172.21.15.87:0/97519092 learned_addr learned my addr 172.21.15.87:0/97519092 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc18a00640 1 -- 172.21.15.87:0/97519092 >> v1:172.21.15.169:6789/0 conn(0x7fbc1415df00 legacy=0x7fbc140d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc18a00640 1 -- 172.21.15.87:0/97519092 >> v1:172.21.15.87:6789/0 conn(0x7fbc140d1ce0 legacy=0x7fbc140b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.852+0000 7fbc18a00640 1 -- 172.21.15.87:0/97519092 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc140d2d30 con 0x7fbc140d0920 2024-04-25T10:12:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc18a00640 1 --2- 172.21.15.87:0/97519092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d0920 0x7fbc140d17a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009870 tx=0x7fbc0001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc612faa2f8ae2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc13e00640 1 -- 172.21.15.87:0/97519092 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000ab660 con 0x7fbc140d0920 2024-04-25T10:12:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc13e00640 1 -- 172.21.15.87:0/97519092 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc000b0070 con 0x7fbc140d0920 2024-04-25T10:12:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc13e00640 1 -- 172.21.15.87:0/97519092 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000aba20 con 0x7fbc140d0920 2024-04-25T10:12:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc1be00640 1 -- 172.21.15.87:0/97519092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d0920 msgr2=0x7fbc140d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc1be00640 1 --2- 172.21.15.87:0/97519092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d0920 0x7fbc140d17a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009870 tx=0x7fbc0001c530 comp rx=0 tx=0).stop 2024-04-25T10:12:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc1be00640 1 -- 172.21.15.87:0/97519092 shutdown_connections 2024-04-25T10:12:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc1be00640 1 --2- 172.21.15.87:0/97519092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d0920 0x7fbc140d17a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc1be00640 1 -- 172.21.15.87:0/97519092 >> 172.21.15.87:0/97519092 conn(0x7fbc1406b290 msgr2=0x7fbc140669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc1be00640 1 -- 172.21.15.87:0/97519092 shutdown_connections 2024-04-25T10:12:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.856+0000 7fbc1be00640 1 -- 172.21.15.87:0/97519092 wait complete. 2024-04-25T10:12:42.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 Processor -- start 2024-04-25T10:12:42.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 -- start start 2024-04-25T10:12:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc140d0920 0x7fbc140ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d1ce0 0x7fbc140b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc1415df00 0x7fbc1415f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140b8280 con 0x7fbc140d1ce0 2024-04-25T10:12:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140b8100 con 0x7fbc1415df00 2024-04-25T10:12:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140b8400 con 0x7fbc140d0920 2024-04-25T10:12:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d1ce0 0x7fbc140b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d1ce0 0x7fbc140b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54180/0 (socket says 172.21.15.87:54180) 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 -- 172.21.15.87:0/376285880 learned_addr learned my addr 172.21.15.87:0/376285880 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc19400640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc140d0920 0x7fbc140ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc19e00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc1415df00 0x7fbc1415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 -- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc140d0920 msgr2=0x7fbc140ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc140d0920 0x7fbc140ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 -- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc1415df00 msgr2=0x7fbc1415f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc1415df00 0x7fbc1415f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 -- 172.21.15.87:0/376285880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc140b6450 con 0x7fbc140d1ce0 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc19e00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc1415df00 0x7fbc1415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc18a00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d1ce0 0x7fbc140b5c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc000ad040 tx=0x7fbc0001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc11600640 1 -- 172.21.15.87:0/376285880 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000b4800 con 0x7fbc140d1ce0 2024-04-25T10:12:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc11600640 1 -- 172.21.15.87:0/376285880 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc000abe50 con 0x7fbc140d1ce0 2024-04-25T10:12:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1415e710 con 0x7fbc140d1ce0 2024-04-25T10:12:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc11600640 1 -- 172.21.15.87:0/376285880 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000b4c00 con 0x7fbc140d1ce0 2024-04-25T10:12:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.860+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1415e9b0 con 0x7fbc140d1ce0 2024-04-25T10:12:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.864+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1404ef30 con 0x7fbc140d1ce0 2024-04-25T10:12:42.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.864+0000 7fbc11600640 1 -- 172.21.15.87:0/376285880 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc000c7390 con 0x7fbc140d1ce0 2024-04-25T10:12:42.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.864+0000 7fbc11600640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbf4066f60 0x7fbbf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.864+0000 7fbc11600640 1 -- 172.21.15.87:0/376285880 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc0011f8e0 con 0x7fbc140d1ce0 2024-04-25T10:12:42.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.864+0000 7fbc19400640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbf4066f60 0x7fbbf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.864+0000 7fbc19400640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbf4066f60 0x7fbbf4069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc08007fd0 tx=0x7fbc08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:42.868+0000 7fbc11600640 1 -- 172.21.15.87:0/376285880 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc000c4060 con 0x7fbc140d1ce0 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.052+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc141593f0 con 0x7fbc140d1ce0 2024-04-25T10:12:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.052+0000 7fbc11600640 1 -- 172.21.15.87:0/376285880 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc00016150 con 0x7fbc140d1ce0 2024-04-25T10:12:43.060 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbf4066f60 msgr2=0x7fbbf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbf4066f60 0x7fbbf4069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc08007fd0 tx=0x7fbc08009040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d1ce0 msgr2=0x7fbc140b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d1ce0 0x7fbc140b5c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc000ad040 tx=0x7fbc0001ad70 comp rx=0 tx=0).stop 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 shutdown_connections 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbbf4066f60 0x7fbbf4069420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc1415df00 0x7fbc1415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc140d1ce0 0x7fbc140b5c00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 --2- 172.21.15.87:0/376285880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc140d0920 0x7fbc140ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 >> 172.21.15.87:0/376285880 conn(0x7fbc1406b290 msgr2=0x7fbc140b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 shutdown_connections 2024-04-25T10:12:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:43.056+0000 7fbc1be00640 1 -- 172.21.15.87:0/376285880 wait complete. 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:43.080 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:12:43.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.081 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-04-25T10:12:43.091 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:43.091 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:43.091 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:43.091 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:43.091 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:43.092 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42458sec preferred_lft 42458sec 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 02:42:9b:0f:4a:34 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:12:43.093 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:43.094 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:12:43.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.094 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:43.094 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:43.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:12:43.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2024-04-25T10:12:43.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2024-04-25T10:12:43.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:12:43.095 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:43.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.214 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:43.232 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.235 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-04-25T10:12:43.312 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:43.313 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:43.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.314 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:43.314 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:43.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:12:43.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:12:43.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:12:43.314 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:43.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.404 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T10:12:43.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.413 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-04-25T10:12:43.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.462 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:43.462 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:43.462 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:12:43.462 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:12:43.462 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:43.544 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:12:43.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:12:43.545 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:43.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.717 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:43.717 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:43.717 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2024-04-25T10:12:43.717 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:12:43.717 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:43.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:43.756 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:12:43.805 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:12:43.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:43.806 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:12:43.852 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:43.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.853 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:12:43.941 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:12:43.941 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:12:43.941 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-25T10:12:44.517 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:12:44.517 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:12:44.517 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-25T10:12:44.517 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:44.517 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:12:44.530 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:12:44.533 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:44.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:44.534 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:12:44.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:44.554 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:44.625 INFO:teuthology.orchestra.run.smithi169.stdout:client4597 2024-04-25T10:12:44.625 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:12:44.625 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:44.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:44.673 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:44.724 INFO:teuthology.orchestra.run.smithi169.stdout:client4597 2024-04-25T10:12:44.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:44.725 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:44.780 INFO:teuthology.orchestra.run.smithi169.stdout:client4597 2024-04-25T10:12:44.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:44.781 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/84c5cae3-3d36-4b8f-9ce8-349d9b4e086b.client4597/status') 2024-04-25T10:12:44.840 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4597 (3)192.168.0.1:0/3231804849 2024-04-25T10:12:44.841 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2024-04-25T10:12:44.841 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2024-04-25T10:12:44.841 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2024-04-25T10:12:44.841 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 5.0969e-05 s, 1.4 MB/s 2024-04-25T10:12:44.843 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T10:12:44.843 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:12:44.843 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:12:44.843 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:44.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 Processor -- start 2024-04-25T10:12:44.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 -- start start 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f05340d1820 0x7f05340d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f053415e050 0x7f05340b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05340b13e0 0x7f05340ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340b1870 con 0x7f05340b13e0 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340b1af0 con 0x7f05340d1820 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f053b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340b1dd0 con 0x7f053415e050 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05340b13e0 0x7f05340ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05340b13e0 0x7f05340ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54190/0 (socket says 172.21.15.87:54190) 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 -- 172.21.15.87:0/2620763705 learned_addr learned my addr 172.21.15.87:0/2620763705 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0533400640 1 --2- 172.21.15.87:0/2620763705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f053415e050 0x7f05340b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0533e00640 1 --2- 172.21.15.87:0/2620763705 >> v2:172.21.15.169:3300/0 conn(0x7f05340d1820 0x7f05340d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 -- 172.21.15.87:0/2620763705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f053415e050 msgr2=0x7f05340b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 --2- 172.21.15.87:0/2620763705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f053415e050 0x7f05340b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 -- 172.21.15.87:0/2620763705 >> v2:172.21.15.169:3300/0 conn(0x7f05340d1820 msgr2=0x7f05340d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 --2- 172.21.15.87:0/2620763705 >> v2:172.21.15.169:3300/0 conn(0x7f05340d1820 0x7f05340d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0539200640 1 -- 172.21.15.87:0/2620763705 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340b20b0 con 0x7f05340b13e0 2024-04-25T10:12:44.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0533400640 1 --2- 172.21.15.87:0/2620763705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f053415e050 0x7f05340b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:44.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.984+0000 7f0533e00640 1 --2- 172.21.15.87:0/2620763705 >> v2:172.21.15.169:3300/0 conn(0x7f05340d1820 0x7f05340d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f0539200640 1 --2- 172.21.15.87:0/2620763705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05340b13e0 0x7f05340ce180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0534070e90 tx=0x7f052c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19ef75070851647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f0532a00640 1 -- 172.21.15.87:0/2620763705 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0b0070 con 0x7f05340b13e0 2024-04-25T10:12:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f0532a00640 1 -- 172.21.15.87:0/2620763705 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c0ab600 con 0x7f05340b13e0 2024-04-25T10:12:44.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f0532a00640 1 -- 172.21.15.87:0/2620763705 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0ab9d0 con 0x7f05340b13e0 2024-04-25T10:12:44.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 -- 172.21.15.87:0/2620763705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05340b13e0 msgr2=0x7f05340ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 --2- 172.21.15.87:0/2620763705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05340b13e0 0x7f05340ce180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0534070e90 tx=0x7f052c01c260 comp rx=0 tx=0).stop 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 -- 172.21.15.87:0/2620763705 shutdown_connections 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 --2- 172.21.15.87:0/2620763705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05340b13e0 0x7f05340ce180 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 --2- 172.21.15.87:0/2620763705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f053415e050 0x7f05340b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 --2- 172.21.15.87:0/2620763705 >> v2:172.21.15.169:3300/0 conn(0x7f05340d1820 0x7f05340d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 -- 172.21.15.87:0/2620763705 >> 172.21.15.87:0/2620763705 conn(0x7f053409a9d0 msgr2=0x7f053409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 -- 172.21.15.87:0/2620763705 shutdown_connections 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.988+0000 7f053b200640 1 -- 172.21.15.87:0/2620763705 wait complete. 2024-04-25T10:12:44.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 Processor -- start 2024-04-25T10:12:44.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 -- start start 2024-04-25T10:12:44.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05340b13e0 0x7f0534151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05340d1820 0x7f05340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f053415e050 0x7f05340b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340cfdf0 con 0x7f053415e050 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340cfc70 con 0x7f05340d1820 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340cff70 con 0x7f05340b13e0 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f053415e050 0x7f05340b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f053415e050 0x7f05340b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54202/0 (socket says 172.21.15.87:54202) 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 -- 172.21.15.87:0/2437607634 learned_addr learned my addr 172.21.15.87:0/2437607634 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 -- 172.21.15.87:0/2437607634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05340b13e0 msgr2=0x7f0534151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05340b13e0 0x7f0534151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 -- 172.21.15.87:0/2437607634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05340d1820 msgr2=0x7f05340b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05340d1820 0x7f05340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 -- 172.21.15.87:0/2437607634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340b5bd0 con 0x7f053415e050 2024-04-25T10:12:45.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f0539200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f053415e050 0x7f05340b5590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ae040 tx=0x7f052c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f052be00640 1 -- 172.21.15.87:0/2437607634 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0b0040 con 0x7f053415e050 2024-04-25T10:12:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f052be00640 1 -- 172.21.15.87:0/2437607634 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c0b9070 con 0x7f053415e050 2024-04-25T10:12:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05340b5eb0 con 0x7f053415e050 2024-04-25T10:12:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f052be00640 1 -- 172.21.15.87:0/2437607634 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0b4690 con 0x7f053415e050 2024-04-25T10:12:45.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.992+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05340b6470 con 0x7f053415e050 2024-04-25T10:12:45.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.996+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053404ef30 con 0x7f053415e050 2024-04-25T10:12:45.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.996+0000 7f052be00640 1 -- 172.21.15.87:0/2437607634 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f052c0b4830 con 0x7f053415e050 2024-04-25T10:12:45.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.996+0000 7f052be00640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0514066f60 0x7f0514069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.996+0000 7f052be00640 1 -- 172.21.15.87:0/2437607634 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052c11e6b0 con 0x7f053415e050 2024-04-25T10:12:45.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.996+0000 7f0533e00640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0514066f60 0x7f0514069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:44.996+0000 7f0533e00640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0514066f60 0x7f0514069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0524009ed0 tx=0x7f05240094a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.000+0000 7f052be00640 1 -- 172.21.15.87:0/2437607634 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f052c0c2020 con 0x7f053415e050 2024-04-25T10:12:45.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f053415d4e0 con 0x7f053415e050 2024-04-25T10:12:45.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f052be00640 1 -- 172.21.15.87:0/2437607634 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f052c016110 con 0x7f053415e050 2024-04-25T10:12:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:45.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0514066f60 msgr2=0x7f0514069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0514066f60 0x7f0514069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0524009ed0 tx=0x7f05240094a0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f053415e050 msgr2=0x7f05340b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f053415e050 0x7f05340b5590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ae040 tx=0x7f052c019000 comp rx=0 tx=0).stop 2024-04-25T10:12:45.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 shutdown_connections 2024-04-25T10:12:45.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0514066f60 0x7f0514069420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f053415e050 0x7f05340b5590 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05340d1820 0x7f05340b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 --2- 172.21.15.87:0/2437607634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05340b13e0 0x7f0534151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 >> 172.21.15.87:0/2437607634 conn(0x7f053409a9d0 msgr2=0x7f05340ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:45.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 shutdown_connections 2024-04-25T10:12:45.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:45.192+0000 7f053b200640 1 -- 172.21.15.87:0/2437607634 wait complete. 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:45.220 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:12:45.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.221 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-04-25T10:12:45.230 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:45.230 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:45.231 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42456sec preferred_lft 42456sec 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 02:42:9b:0f:4a:34 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:12:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.233 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:12:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:12:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.233 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:45.291 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-25T10:12:45.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.293 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-04-25T10:12:45.349 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-04-25T10:12:45.349 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:45.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.351 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:45.351 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:45.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:12:45.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:12:45.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:12:45.351 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:45.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.437 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T10:12:45.490 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:12:45.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.491 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-04-25T10:12:45.545 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:12:45.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.546 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:45.546 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:45.546 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:12:45.546 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:12:45.546 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:45.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.627 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:12:45.693 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:45.693 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:45.693 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:45.693 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.693 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:45.693 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.693 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:45.694 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 9e:36:91:1f:a0:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:12:45.694 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:12:45.694 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.694 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::9c36:91ff:fe1f:a015/64 scope link 2024-04-25T10:12:45.694 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.695 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:12:45.761 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 9e:36:91:1f:a0:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::9c36:91ff:fe1f:a015/64 scope link 2024-04-25T10:12:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:45.764 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:12:45.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:12:45.764 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:45.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:45.898 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-04-25T10:12:45.898 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T10:12:45.898 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2024-04-25T10:12:45.898 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:12:45.898 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T10:12:45.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:45.937 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:12:45.989 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:12:45.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:45.990 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:12:46.041 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:46.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:12:46.128 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:12:46.128 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:12:46.129 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-25T10:12:46.143 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:12:46.143 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:12:46.144 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-25T10:12:46.144 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:46.144 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:12:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:12:46.157 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:46.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.157 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:12:46.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.217 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:46.273 INFO:teuthology.orchestra.run.smithi169.stdout:client4505 2024-04-25T10:12:46.273 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:12:46.273 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:46.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.326 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:46.384 INFO:teuthology.orchestra.run.smithi169.stdout:client4505 2024-04-25T10:12:46.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.385 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:46.436 INFO:teuthology.orchestra.run.smithi169.stdout:client4505 2024-04-25T10:12:46.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.437 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/84c5cae3-3d36-4b8f-9ce8-349d9b4e086b.client4505/status') 2024-04-25T10:12:46.501 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4505 (3)192.168.0.2:0/4254037518 2024-04-25T10:12:46.501 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2024-04-25T10:12:46.501 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2024-04-25T10:12:46.501 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2024-04-25T10:12:46.501 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 4.9532e-05 s, 1.4 MB/s 2024-04-25T10:12:46.506 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:12:46.522 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:46.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7e00640 1 Processor -- start 2024-04-25T10:12:46.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7e00640 1 -- start start 2024-04-25T10:12:46.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f58e80670b0 0x7f58e80b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e815de30 con 0x7f58e815da50 2024-04-25T10:12:46.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e80b59f0 con 0x7f58e80b16c0 2024-04-25T10:12:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80b5cd0 con 0x7f58e80670b0 2024-04-25T10:12:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e6a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f58e815da50 0x7f58e80669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37030/0 (socket says 172.21.15.87:37030) 2024-04-25T10:12:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e6a00640 1 -- 172.21.15.87:0/3144504875 learned_addr learned my addr 172.21.15.87:0/3144504875 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7400640 1 --2- 172.21.15.87:0/3144504875 >> v2:172.21.15.87:3300/0 conn(0x7f58e80670b0 0x7f58e80b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7400640 1 -- 172.21.15.87:0/3144504875 >> v1:172.21.15.169:6789/0 conn(0x7f58e80b16c0 legacy=0x7f58e80ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7400640 1 -- 172.21.15.87:0/3144504875 >> v1:172.21.15.87:6789/0 conn(0x7f58e815da50 legacy=0x7f58e80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.664+0000 7f58e7400640 1 -- 172.21.15.87:0/3144504875 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80b5fb0 con 0x7f58e80670b0 2024-04-25T10:12:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7400640 1 --2- 172.21.15.87:0/3144504875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80b1d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009d90 tx=0x7f58e001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638231a3e30ab213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e5600640 1 -- 172.21.15.87:0/3144504875 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00b2070 con 0x7f58e80670b0 2024-04-25T10:12:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e5600640 1 -- 172.21.15.87:0/3144504875 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e00a5940 con 0x7f58e80670b0 2024-04-25T10:12:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e5600640 1 -- 172.21.15.87:0/3144504875 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00a5d10 con 0x7f58e80670b0 2024-04-25T10:12:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 -- 172.21.15.87:0/3144504875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 msgr2=0x7f58e80b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 --2- 172.21.15.87:0/3144504875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80b1d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009d90 tx=0x7f58e001c970 comp rx=0 tx=0).stop 2024-04-25T10:12:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 -- 172.21.15.87:0/3144504875 shutdown_connections 2024-04-25T10:12:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 --2- 172.21.15.87:0/3144504875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80b1d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 -- 172.21.15.87:0/3144504875 >> 172.21.15.87:0/3144504875 conn(0x7f58e80d0690 msgr2=0x7f58e80d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 -- 172.21.15.87:0/3144504875 shutdown_connections 2024-04-25T10:12:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 -- 172.21.15.87:0/3144504875 wait complete. 2024-04-25T10:12:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.668+0000 7f58e7e00640 1 Processor -- start 2024-04-25T10:12:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 -- start start 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58e80b16c0 0x7f58e8140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58e815da50 0x7f58e8140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80cf9a0 con 0x7f58e80670b0 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80cf820 con 0x7f58e80b16c0 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e816b610 con 0x7f58e815da50 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54208/0 (socket says 172.21.15.87:54208) 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 -- 172.21.15.87:0/2043141107 learned_addr learned my addr 172.21.15.87:0/2043141107 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6000640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58e80b16c0 0x7f58e8140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7400640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58e815da50 0x7f58e8140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 -- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58e815da50 msgr2=0x7f58e8140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58e815da50 0x7f58e8140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 -- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58e80b16c0 msgr2=0x7f58e8140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58e80b16c0 0x7f58e8140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 -- 172.21.15.87:0/2043141107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80b48d0 con 0x7f58e80670b0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e6a00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80ca3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f58d8005890 tx=0x7f58d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58df400640 1 -- 172.21.15.87:0/2043141107 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8017070 con 0x7f58e80670b0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58df400640 1 -- 172.21.15.87:0/2043141107 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d8005d10 con 0x7f58e80670b0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58df400640 1 -- 172.21.15.87:0/2043141107 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8007310 con 0x7f58e80670b0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e80b4bb0 con 0x7f58e80670b0 2024-04-25T10:12:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e80b5170 con 0x7f58e80670b0 2024-04-25T10:12:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.672+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e8075120 con 0x7f58e80670b0 2024-04-25T10:12:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.676+0000 7f58df400640 1 -- 172.21.15.87:0/2043141107 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58d801a070 con 0x7f58e80670b0 2024-04-25T10:12:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.676+0000 7f58df400640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f58c8066f60 0x7f58c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.676+0000 7f58df400640 1 -- 172.21.15.87:0/2043141107 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58d80842e0 con 0x7f58e80670b0 2024-04-25T10:12:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.676+0000 7f58e6000640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f58c8066f60 0x7f58c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.676+0000 7f58e6000640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f58c8066f60 0x7f58c8069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ccac0 tx=0x7f58d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.680+0000 7f58df400640 1 -- 172.21.15.87:0/2043141107 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58d8054c90 con 0x7f58e80670b0 2024-04-25T10:12:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.836+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58e8151bd0 con 0x7f58e80670b0 2024-04-25T10:12:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.836+0000 7f58df400640 1 -- 172.21.15.87:0/2043141107 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f58d8058d30 con 0x7f58e80670b0 2024-04-25T10:12:46.843 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:46.843 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:46.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f58c8066f60 msgr2=0x7f58c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f58c8066f60 0x7f58c8069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ccac0 tx=0x7f58d4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:46.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 msgr2=0x7f58e80ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80ca3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f58d8005890 tx=0x7f58d800c890 comp rx=0 tx=0).stop 2024-04-25T10:12:46.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 shutdown_connections 2024-04-25T10:12:46.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f58c8066f60 0x7f58c8069420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58e815da50 0x7f58e8140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58e80b16c0 0x7f58e8140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 --2- 172.21.15.87:0/2043141107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58e80670b0 0x7f58e80ca3f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 >> 172.21.15.87:0/2043141107 conn(0x7f58e80d0690 msgr2=0x7f58e806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 shutdown_connections 2024-04-25T10:12:46.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.840+0000 7f58e7e00640 1 -- 172.21.15.87:0/2043141107 wait complete. 2024-04-25T10:12:46.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:46.865 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:47.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.996+0000 7efc9aa00640 1 Processor -- start 2024-04-25T10:12:47.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:46.996+0000 7efc9aa00640 1 -- start start 2024-04-25T10:12:47.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9415b150 0x7efc940aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc9aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efc9406c0d0 0x7efc9406c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc9aa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc940b2fd0 con 0x7efc9415a0e0 2024-04-25T10:12:47.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc9aa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc940aef90 con 0x7efc9406c0d0 2024-04-25T10:12:47.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc9aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940af1d0 con 0x7efc9415b150 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc98a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efc9406c0d0 0x7efc9406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc93e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efc9415a0e0 0x7efc940b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37036/0 (socket says 172.21.15.87:37036) 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc98a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efc9406c0d0 0x7efc9406c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54222/0 (socket says 172.21.15.87:54222) 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc93e00640 1 -- 172.21.15.87:0/93482334 learned_addr learned my addr 172.21.15.87:0/93482334 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc93400640 1 --2- 172.21.15.87:0/93482334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9415b150 0x7efc940aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc98a00640 1 -- 172.21.15.87:0/93482334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9415b150 msgr2=0x7efc940aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc98a00640 1 --2- 172.21.15.87:0/93482334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9415b150 0x7efc940aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc98a00640 1 -- 172.21.15.87:0/93482334 >> v1:172.21.15.87:6789/0 conn(0x7efc9415a0e0 legacy=0x7efc940b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc98a00640 1 -- 172.21.15.87:0/93482334 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940b32b0 con 0x7efc9406c0d0 2024-04-25T10:12:47.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.000+0000 7efc92a00640 1 -- 172.21.15.87:0/93482334 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536571581 0 0) 0x7efc940b2fd0 con 0x7efc9415a0e0 2024-04-25T10:12:47.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc98a00640 1 --2- 172.21.15.87:0/93482334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9406c0d0 0x7efc9406c520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009d90 tx=0x7efc8c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0a6e111d5ad9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc92a00640 1 -- 172.21.15.87:0/93482334 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0ac070 con 0x7efc9406c0d0 2024-04-25T10:12:47.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc92a00640 1 -- 172.21.15.87:0/93482334 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c0a6620 con 0x7efc9406c0d0 2024-04-25T10:12:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc92a00640 1 -- 172.21.15.87:0/93482334 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0a69f0 con 0x7efc9406c0d0 2024-04-25T10:12:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 -- 172.21.15.87:0/93482334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9406c0d0 msgr2=0x7efc9406c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 --2- 172.21.15.87:0/93482334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9406c0d0 0x7efc9406c520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009d90 tx=0x7efc8c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 -- 172.21.15.87:0/93482334 shutdown_connections 2024-04-25T10:12:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 --2- 172.21.15.87:0/93482334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9406c0d0 0x7efc9406c520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 --2- 172.21.15.87:0/93482334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9415b150 0x7efc940aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 -- 172.21.15.87:0/93482334 >> 172.21.15.87:0/93482334 conn(0x7efc940669a0 msgr2=0x7efc94066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 -- 172.21.15.87:0/93482334 shutdown_connections 2024-04-25T10:12:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 -- 172.21.15.87:0/93482334 wait complete. 2024-04-25T10:12:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 Processor -- start 2024-04-25T10:12:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.004+0000 7efc9aa00640 1 -- start start 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9406c0d0 0x7efc941f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9415a0e0 0x7efc94167a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc9415b150 0x7efc940ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940b4620 con 0x7efc9415a0e0 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940b44a0 con 0x7efc9415b150 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940705f0 con 0x7efc9406c0d0 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9415a0e0 0x7efc94167a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9415a0e0 0x7efc94167a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54230/0 (socket says 172.21.15.87:54230) 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 -- 172.21.15.87:0/743047355 learned_addr learned my addr 172.21.15.87:0/743047355 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93e00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9406c0d0 0x7efc941f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc98a00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc9415b150 0x7efc940ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 -- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9406c0d0 msgr2=0x7efc941f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9406c0d0 0x7efc941f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 -- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc9415b150 msgr2=0x7efc940ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc9415b150 0x7efc940ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 -- 172.21.15.87:0/743047355 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94168360 con 0x7efc9415a0e0 2024-04-25T10:12:47.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc93400640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9415a0e0 0x7efc94167a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efc8000d5f0 tx=0x7efc8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc8be00640 1 -- 172.21.15.87:0/743047355 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800125f0 con 0x7efc9415a0e0 2024-04-25T10:12:47.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc8be00640 1 -- 172.21.15.87:0/743047355 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80019070 con 0x7efc9415a0e0 2024-04-25T10:12:47.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc8be00640 1 -- 172.21.15.87:0/743047355 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800129b0 con 0x7efc9415a0e0 2024-04-25T10:12:47.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc94168640 con 0x7efc9415a0e0 2024-04-25T10:12:47.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc94148c00 con 0x7efc9415a0e0 2024-04-25T10:12:47.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.008+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9404ef30 con 0x7efc9415a0e0 2024-04-25T10:12:47.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.012+0000 7efc8be00640 1 -- 172.21.15.87:0/743047355 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc80012b50 con 0x7efc9415a0e0 2024-04-25T10:12:47.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.012+0000 7efc8be00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efc74066f60 0x7efc74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.012+0000 7efc8be00640 1 -- 172.21.15.87:0/743047355 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc80085330 con 0x7efc9415a0e0 2024-04-25T10:12:47.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.012+0000 7efc93e00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efc74066f60 0x7efc74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.012+0000 7efc93e00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efc74066f60 0x7efc74069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc84007fd0 tx=0x7efc84007d60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.016+0000 7efc8be00640 1 -- 172.21.15.87:0/743047355 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc80055c50 con 0x7efc9415a0e0 2024-04-25T10:12:47.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.168+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc9406e1f0 con 0x7efc9415a0e0 2024-04-25T10:12:47.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.168+0000 7efc8be00640 1 -- 172.21.15.87:0/743047355 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc80059cf0 con 0x7efc9415a0e0 2024-04-25T10:12:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efc74066f60 msgr2=0x7efc74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efc74066f60 0x7efc74069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc84007fd0 tx=0x7efc84007d60 comp rx=0 tx=0).stop 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9415a0e0 msgr2=0x7efc94167a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9415a0e0 0x7efc94167a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efc8000d5f0 tx=0x7efc8000dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 shutdown_connections 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efc74066f60 0x7efc74069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc9415b150 0x7efc940ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc9415a0e0 0x7efc94167a60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 --2- 172.21.15.87:0/743047355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc9406c0d0 0x7efc941f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 >> 172.21.15.87:0/743047355 conn(0x7efc940669a0 msgr2=0x7efc94071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 shutdown_connections 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.172+0000 7efc9aa00640 1 -- 172.21.15.87:0/743047355 wait complete. 2024-04-25T10:12:47.196 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:47.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 Processor -- start 2024-04-25T10:12:47.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 -- start start 2024-04-25T10:12:47.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f195c1560b0 0x7f195c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c06ea30 0x7f195c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f195c0669a0 0x7f195c066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c156490 con 0x7f195c0669a0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c15f760 con 0x7f195c1560b0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1963a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c15fa40 con 0x7f195c06ea30 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f195c0669a0 0x7f195c066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f195c0669a0 0x7f195c066e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54240/0 (socket says 172.21.15.87:54240) 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 -- 172.21.15.87:0/1668556193 learned_addr learned my addr 172.21.15.87:0/1668556193 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961000640 1 --2- 172.21.15.87:0/1668556193 >> v2:172.21.15.169:3300/0 conn(0x7f195c1560b0 0x7f195c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1953e00640 1 --2- 172.21.15.87:0/1668556193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c06ea30 0x7f195c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 -- 172.21.15.87:0/1668556193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c06ea30 msgr2=0x7f195c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 --2- 172.21.15.87:0/1668556193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c06ea30 0x7f195c06eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 -- 172.21.15.87:0/1668556193 >> v2:172.21.15.169:3300/0 conn(0x7f195c1560b0 msgr2=0x7f195c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 --2- 172.21.15.87:0/1668556193 >> v2:172.21.15.169:3300/0 conn(0x7f195c1560b0 0x7f195c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.332+0000 7f1961a00640 1 -- 172.21.15.87:0/1668556193 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c15fd20 con 0x7f195c0669a0 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1961a00640 1 --2- 172.21.15.87:0/1668556193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c066e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f195c071600 tx=0x7f195401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7215dc842a6229e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f195be00640 1 -- 172.21.15.87:0/1668556193 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19540a55e0 con 0x7f195c0669a0 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f195be00640 1 -- 172.21.15.87:0/1668556193 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19540a5780 con 0x7f195c0669a0 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f195be00640 1 -- 172.21.15.87:0/1668556193 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19540a5b50 con 0x7f195c0669a0 2024-04-25T10:12:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 -- 172.21.15.87:0/1668556193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 msgr2=0x7f195c066e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 --2- 172.21.15.87:0/1668556193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c066e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f195c071600 tx=0x7f195401c960 comp rx=0 tx=0).stop 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 -- 172.21.15.87:0/1668556193 shutdown_connections 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 --2- 172.21.15.87:0/1668556193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c066e30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 --2- 172.21.15.87:0/1668556193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c06ea30 0x7f195c06eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 --2- 172.21.15.87:0/1668556193 >> v2:172.21.15.169:3300/0 conn(0x7f195c1560b0 0x7f195c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 -- 172.21.15.87:0/1668556193 >> 172.21.15.87:0/1668556193 conn(0x7f195c0b2560 msgr2=0x7f195c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 -- 172.21.15.87:0/1668556193 shutdown_connections 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.336+0000 7f1963a00640 1 -- 172.21.15.87:0/1668556193 wait complete. 2024-04-25T10:12:47.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 Processor -- start 2024-04-25T10:12:47.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 -- start start 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c14bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f195c06ea30 0x7f195c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c1560b0 0x7f195c15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c06e060 con 0x7f195c0669a0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c06dee0 con 0x7f195c06ea30 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c06e1e0 con 0x7f195c1560b0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c14bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c14bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40572/0 (socket says 172.21.15.87:40572) 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 -- 172.21.15.87:0/2046197421 learned_addr learned my addr 172.21.15.87:0/2046197421 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961a00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c1560b0 0x7f195c15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1953e00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f195c06ea30 0x7f195c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 -- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c1560b0 msgr2=0x7f195c15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c1560b0 0x7f195c15ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 -- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f195c06ea30 msgr2=0x7f195c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f195c06ea30 0x7f195c15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 -- 172.21.15.87:0/2046197421 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0b3d40 con 0x7f195c0669a0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1961000640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c14bb40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f194c00d800 tx=0x7f194c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1959600640 1 -- 172.21.15.87:0/2046197421 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00b6c0 con 0x7f195c0669a0 2024-04-25T10:12:47.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1959600640 1 -- 172.21.15.87:0/2046197421 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c019070 con 0x7f195c0669a0 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1959600640 1 -- 172.21.15.87:0/2046197421 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00ba60 con 0x7f195c0669a0 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c0b4020 con 0x7f195c0669a0 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c0b45e0 con 0x7f195c0669a0 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.340+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c04ef30 con 0x7f195c0669a0 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.344+0000 7f1959600640 1 -- 172.21.15.87:0/2046197421 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f194c00bc00 con 0x7f195c0669a0 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.344+0000 7f1959600640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1938066f90 0x7f1938069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.344+0000 7f1959600640 1 -- 172.21.15.87:0/2046197421 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f194c085450 con 0x7f195c0669a0 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.344+0000 7f1953e00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1938066f90 0x7f1938069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.344+0000 7f1953e00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1938066f90 0x7f1938069450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f195c149f90 tx=0x7f1944008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.348+0000 7f1959600640 1 -- 172.21.15.87:0/2046197421 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f194c056450 con 0x7f195c0669a0 2024-04-25T10:12:47.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.500+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f195c072090 con 0x7f195c0669a0 2024-04-25T10:12:47.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.500+0000 7f1959600640 1 -- 172.21.15.87:0/2046197421 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f194c055d50 con 0x7f195c0669a0 2024-04-25T10:12:47.507 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1938066f90 msgr2=0x7f1938069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1938066f90 0x7f1938069450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f195c149f90 tx=0x7f1944008040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 msgr2=0x7f195c14bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c14bb40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f194c00d800 tx=0x7f194c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 shutdown_connections 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1938066f90 0x7f1938069450 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f195c1560b0 0x7f195c15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f195c06ea30 0x7f195c15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 --2- 172.21.15.87:0/2046197421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f195c0669a0 0x7f195c14bb40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 >> 172.21.15.87:0/2046197421 conn(0x7f195c0b2560 msgr2=0x7f195c153eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 shutdown_connections 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.504+0000 7f1963a00640 1 -- 172.21.15.87:0/2046197421 wait complete. 2024-04-25T10:12:47.529 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:47.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4ae400640 1 Processor -- start 2024-04-25T10:12:47.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4ae400640 1 -- start start 2024-04-25T10:12:47.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4ae400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4a8066fe0 0x7ff4a8161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4ae400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a80b40c0 con 0x7ff4a8066c00 2024-04-25T10:12:47.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4ae400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a80b5b00 con 0x7ff4a80b5560 2024-04-25T10:12:47.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4ae400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a8161d60 con 0x7ff4a8066fe0 2024-04-25T10:12:47.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff4a8066c00 0x7ff4a8072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40214/0 (socket says 172.21.15.87:40214) 2024-04-25T10:12:47.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a7400640 1 -- 172.21.15.87:0/2835128594 learned_addr learned my addr 172.21.15.87:0/2835128594 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:47.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a6a00640 1 --2- 172.21.15.87:0/2835128594 >> v2:172.21.15.87:3300/0 conn(0x7ff4a8066fe0 0x7ff4a8161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a6000640 1 -- 172.21.15.87:0/2835128594 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776956566 0 0) 0x7ff4a80b40c0 con 0x7ff4a8066c00 2024-04-25T10:12:47.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a6000640 1 -- 172.21.15.87:0/2835128594 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c003530 con 0x7ff4a8066c00 2024-04-25T10:12:47.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a6a00640 1 -- 172.21.15.87:0/2835128594 >> v1:172.21.15.169:6789/0 conn(0x7ff4a80b5560 legacy=0x7ff4a80ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a6a00640 1 -- 172.21.15.87:0/2835128594 >> v1:172.21.15.87:6789/0 conn(0x7ff4a8066c00 legacy=0x7ff4a8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.664+0000 7ff4a6a00640 1 -- 172.21.15.87:0/2835128594 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80b3a00 con 0x7ff4a8066fe0 2024-04-25T10:12:47.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4a6a00640 1 --2- 172.21.15.87:0/2835128594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a8066fe0 0x7ff4a8161820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4940097c0 tx=0x7ff49401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6edbf152984579c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4a6000640 1 -- 172.21.15.87:0/2835128594 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940b0070 con 0x7ff4a8066fe0 2024-04-25T10:12:47.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4a6000640 1 -- 172.21.15.87:0/2835128594 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4940ab600 con 0x7ff4a8066fe0 2024-04-25T10:12:47.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4a6000640 1 -- 172.21.15.87:0/2835128594 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940ab9d0 con 0x7ff4a8066fe0 2024-04-25T10:12:47.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4ae400640 1 -- 172.21.15.87:0/2835128594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a8066fe0 msgr2=0x7ff4a8161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4ae400640 1 --2- 172.21.15.87:0/2835128594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a8066fe0 0x7ff4a8161820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4940097c0 tx=0x7ff49401c340 comp rx=0 tx=0).stop 2024-04-25T10:12:47.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4ae400640 1 -- 172.21.15.87:0/2835128594 shutdown_connections 2024-04-25T10:12:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4ae400640 1 --2- 172.21.15.87:0/2835128594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a8066fe0 0x7ff4a8161820 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4ae400640 1 -- 172.21.15.87:0/2835128594 >> 172.21.15.87:0/2835128594 conn(0x7ff4a80b2560 msgr2=0x7ff4a80b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4ae400640 1 -- 172.21.15.87:0/2835128594 shutdown_connections 2024-04-25T10:12:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.668+0000 7ff4ae400640 1 -- 172.21.15.87:0/2835128594 wait complete. 2024-04-25T10:12:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 Processor -- start 2024-04-25T10:12:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 -- start start 2024-04-25T10:12:47.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a8066c00 0x7ff4a8147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a8066fe0 0x7ff4a806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a80b5560 0x7ff4a814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a815e540 con 0x7ff4a80b5560 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8158620 con 0x7ff4a8066c00 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a815e6c0 con 0x7ff4a8066fe0 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a80b5560 0x7ff4a814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a80b5560 0x7ff4a814e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40586/0 (socket says 172.21.15.87:40586) 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 -- 172.21.15.87:0/1527505679 learned_addr learned my addr 172.21.15.87:0/1527505679 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a6a00640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a8066fe0 0x7ff4a806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a8066c00 0x7ff4a8147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 -- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a8066fe0 msgr2=0x7ff4a806d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a8066fe0 0x7ff4a806d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 -- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a8066c00 msgr2=0x7ff4a8147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a8066c00 0x7ff4a8147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 -- 172.21.15.87:0/1527505679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a806e110 con 0x7ff4a80b5560 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4a7e00640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a80b5560 0x7ff4a814e280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000b970 tx=0x7ff4a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff49f400640 1 -- 172.21.15.87:0/1527505679 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0019070 con 0x7ff4a80b5560 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff49f400640 1 -- 172.21.15.87:0/1527505679 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a000cb40 con 0x7ff4a80b5560 2024-04-25T10:12:47.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff49f400640 1 -- 172.21.15.87:0/1527505679 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0014440 con 0x7ff4a80b5560 2024-04-25T10:12:47.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a8148b80 con 0x7ff4a80b5560 2024-04-25T10:12:47.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a8149140 con 0x7ff4a80b5560 2024-04-25T10:12:47.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.672+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a804ef30 con 0x7ff4a80b5560 2024-04-25T10:12:47.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.676+0000 7ff49f400640 1 -- 172.21.15.87:0/1527505679 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4a001c070 con 0x7ff4a80b5560 2024-04-25T10:12:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.676+0000 7ff49f400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff488066f60 0x7ff488069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.676+0000 7ff49f400640 1 -- 172.21.15.87:0/1527505679 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4a0084bb0 con 0x7ff4a80b5560 2024-04-25T10:12:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.676+0000 7ff4a7400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff488066f60 0x7ff488069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.676+0000 7ff4a7400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff488066f60 0x7ff488069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff498007fd0 tx=0x7ff498009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.680+0000 7ff49f400640 1 -- 172.21.15.87:0/1527505679 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4a0055560 con 0x7ff4a80b5560 2024-04-25T10:12:47.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.832+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4a8153150 con 0x7ff4a80b5560 2024-04-25T10:12:47.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.832+0000 7ff49f400640 1 -- 172.21.15.87:0/1527505679 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4a0059600 con 0x7ff4a80b5560 2024-04-25T10:12:47.839 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff488066f60 msgr2=0x7ff488069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff488066f60 0x7ff488069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff498007fd0 tx=0x7ff498009040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a80b5560 msgr2=0x7ff4a814e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a80b5560 0x7ff4a814e280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000b970 tx=0x7ff4a000be30 comp rx=0 tx=0).stop 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 shutdown_connections 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff488066f60 0x7ff488069420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a80b5560 0x7ff4a814e280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a8066fe0 0x7ff4a806d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 --2- 172.21.15.87:0/1527505679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a8066c00 0x7ff4a8147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 >> 172.21.15.87:0/1527505679 conn(0x7ff4a80b2560 msgr2=0x7ff4a806e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 shutdown_connections 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.836+0000 7ff4ae400640 1 -- 172.21.15.87:0/1527505679 wait complete. 2024-04-25T10:12:47.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:47.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a9c00640 1 Processor -- start 2024-04-25T10:12:47.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a9c00640 1 -- start start 2024-04-25T10:12:47.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14a40cda10 0x7f14a406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a9c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a40b2770 con 0x7f14a40cf930 2024-04-25T10:12:47.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a9c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a406c670 con 0x7f14a415bc40 2024-04-25T10:12:47.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a9c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a406c950 con 0x7f14a40cda10 2024-04-25T10:12:47.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f14a40cf930 0x7f14a40b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40228/0 (socket says 172.21.15.87:40228) 2024-04-25T10:12:47.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a3400640 1 -- 172.21.15.87:0/1958435722 learned_addr learned my addr 172.21.15.87:0/1958435722 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:47.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f149a000640 1 --2- 172.21.15.87:0/1958435722 >> v2:172.21.15.169:3300/0 conn(0x7f14a40cda10 0x7f14a406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a2000640 1 -- 172.21.15.87:0/1958435722 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198325561 0 0) 0x7f14a40b2770 con 0x7f14a40cf930 2024-04-25T10:12:48.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f14a2000640 1 -- 172.21.15.87:0/1958435722 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1488003530 con 0x7f14a40cf930 2024-04-25T10:12:48.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f149a000640 1 -- 172.21.15.87:0/1958435722 >> v1:172.21.15.169:6789/0 conn(0x7f14a415bc40 legacy=0x7f14a40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f149a000640 1 -- 172.21.15.87:0/1958435722 >> v1:172.21.15.87:6789/0 conn(0x7f14a40cf930 legacy=0x7f14a40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.992+0000 7f149a000640 1 -- 172.21.15.87:0/1958435722 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a406cc30 con 0x7f14a40cda10 2024-04-25T10:12:48.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f149a000640 1 --2- 172.21.15.87:0/1958435722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cda10 0x7f14a406c090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f148c0097c0 tx=0x7f148c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e9049664b455e31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a2000640 1 -- 172.21.15.87:0/1958435722 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0b0070 con 0x7f14a40cda10 2024-04-25T10:12:48.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a2000640 1 -- 172.21.15.87:0/1958435722 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c0ab600 con 0x7f14a40cda10 2024-04-25T10:12:48.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a2000640 1 -- 172.21.15.87:0/1958435722 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0ab9b0 con 0x7f14a40cda10 2024-04-25T10:12:48.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a9c00640 1 -- 172.21.15.87:0/1958435722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cda10 msgr2=0x7f14a406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a9c00640 1 --2- 172.21.15.87:0/1958435722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cda10 0x7f14a406c090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f148c0097c0 tx=0x7f148c01c410 comp rx=0 tx=0).stop 2024-04-25T10:12:48.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a9c00640 1 -- 172.21.15.87:0/1958435722 shutdown_connections 2024-04-25T10:12:48.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a9c00640 1 --2- 172.21.15.87:0/1958435722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cda10 0x7f14a406c090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a9c00640 1 -- 172.21.15.87:0/1958435722 >> 172.21.15.87:0/1958435722 conn(0x7f14a4066b70 msgr2=0x7f14a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a9c00640 1 -- 172.21.15.87:0/1958435722 shutdown_connections 2024-04-25T10:12:48.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:47.996+0000 7f14a9c00640 1 -- 172.21.15.87:0/1958435722 wait complete. 2024-04-25T10:12:48.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 Processor -- start 2024-04-25T10:12:48.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 -- start start 2024-04-25T10:12:48.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14a40cda10 0x7f14a415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cf930 0x7f14a41f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14a415bc40 0x7f14a4143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4169d70 con 0x7f14a415bc40 2024-04-25T10:12:48.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4169bf0 con 0x7f14a40cf930 2024-04-25T10:12:48.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4169ef0 con 0x7f14a40cda10 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14a415bc40 0x7f14a4143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14a415bc40 0x7f14a4143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40598/0 (socket says 172.21.15.87:40598) 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 -- 172.21.15.87:0/2000541369 learned_addr learned my addr 172.21.15.87:0/2000541369 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f149a000640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cf930 0x7f14a41f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a2a00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14a40cda10 0x7f14a415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 -- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14a40cda10 msgr2=0x7f14a415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14a40cda10 0x7f14a415d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 -- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cf930 msgr2=0x7f14a41f87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cf930 0x7f14a41f87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 -- 172.21.15.87:0/2000541369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a41f8f10 con 0x7f14a415bc40 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f149a000640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cf930 0x7f14a41f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a3400640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14a415bc40 0x7f14a4143230 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cb40 tx=0x7f149c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f149b400640 1 -- 172.21.15.87:0/2000541369 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c017070 con 0x7f14a415bc40 2024-04-25T10:12:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f149b400640 1 -- 172.21.15.87:0/2000541369 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00aca0 con 0x7f14a415bc40 2024-04-25T10:12:48.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f149b400640 1 -- 172.21.15.87:0/2000541369 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c012620 con 0x7f14a415bc40 2024-04-25T10:12:48.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a41f91f0 con 0x7f14a415bc40 2024-04-25T10:12:48.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.000+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a414f680 con 0x7f14a415bc40 2024-04-25T10:12:48.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.004+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a404ef30 con 0x7f14a415bc40 2024-04-25T10:12:48.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.004+0000 7f149b400640 1 -- 172.21.15.87:0/2000541369 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f149c01a070 con 0x7f14a415bc40 2024-04-25T10:12:48.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.004+0000 7f149b400640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1480066f60 0x7f1480069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.004+0000 7f149b400640 1 -- 172.21.15.87:0/2000541369 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f149c084e50 con 0x7f14a415bc40 2024-04-25T10:12:48.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.004+0000 7f14a2a00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1480066f60 0x7f1480069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.004+0000 7f14a2a00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1480066f60 0x7f1480069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1494001650 tx=0x7f1494009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.008+0000 7f149b400640 1 -- 172.21.15.87:0/2000541369 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f149c055e50 con 0x7f14a415bc40 2024-04-25T10:12:48.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.164+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14a406d960 con 0x7f14a415bc40 2024-04-25T10:12:48.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.164+0000 7f149b400640 1 -- 172.21.15.87:0/2000541369 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f149c055750 con 0x7f14a415bc40 2024-04-25T10:12:48.171 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:48.171 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1480066f60 msgr2=0x7f1480069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1480066f60 0x7f1480069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1494001650 tx=0x7f1494009040 comp rx=0 tx=0).stop 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14a415bc40 msgr2=0x7f14a4143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14a415bc40 0x7f14a4143230 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cb40 tx=0x7f149c00a060 comp rx=0 tx=0).stop 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 shutdown_connections 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1480066f60 0x7f1480069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14a415bc40 0x7f14a4143230 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a40cf930 0x7f14a41f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 --2- 172.21.15.87:0/2000541369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14a40cda10 0x7f14a415d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 >> 172.21.15.87:0/2000541369 conn(0x7f14a4066b70 msgr2=0x7f14a40ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 shutdown_connections 2024-04-25T10:12:48.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.168+0000 7f14a9c00640 1 -- 172.21.15.87:0/2000541369 wait complete. 2024-04-25T10:12:48.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:48.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:48.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.328+0000 7f308a200640 1 Processor -- start 2024-04-25T10:12:48.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.328+0000 7f308a200640 1 -- start start 2024-04-25T10:12:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f308a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084159540 0x7f30840ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f308a200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30840aebd0 0x7f3084071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f308a200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3084071360 con 0x7f3084072020 2024-04-25T10:12:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f308a200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30840af030 con 0x7f30840aebd0 2024-04-25T10:12:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f308a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3084159920 con 0x7f3084159540 2024-04-25T10:12:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f307aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30840aebd0 0x7f3084071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f3083e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3084072020 0x7f308415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40230/0 (socket says 172.21.15.87:40230) 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f3083e00640 1 -- 172.21.15.87:0/2145878492 learned_addr learned my addr 172.21.15.87:0/2145878492 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f3083400640 1 --2- 172.21.15.87:0/2145878492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084159540 0x7f30840ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f307aa00640 1 -- 172.21.15.87:0/2145878492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084159540 msgr2=0x7f30840ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f307aa00640 1 --2- 172.21.15.87:0/2145878492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084159540 0x7f30840ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f3082a00640 1 -- 172.21.15.87:0/2145878492 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287432983 0 0) 0x7f3084071360 con 0x7f3084072020 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f307aa00640 1 -- 172.21.15.87:0/2145878492 >> v1:172.21.15.87:6789/0 conn(0x7f3084072020 legacy=0x7f308415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.332+0000 7f307aa00640 1 -- 172.21.15.87:0/2145878492 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308415c2f0 con 0x7f30840aebd0 2024-04-25T10:12:48.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f307aa00640 1 --2- 172.21.15.87:0/2145878492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30840aebd0 0x7f3084071ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f306c0097c0 tx=0x7f306c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef5f847112a32d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f3082a00640 1 -- 172.21.15.87:0/2145878492 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0b2070 con 0x7f30840aebd0 2024-04-25T10:12:48.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f3082a00640 1 -- 172.21.15.87:0/2145878492 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c0ad640 con 0x7f30840aebd0 2024-04-25T10:12:48.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f3082a00640 1 -- 172.21.15.87:0/2145878492 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0ada10 con 0x7f30840aebd0 2024-04-25T10:12:48.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 -- 172.21.15.87:0/2145878492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30840aebd0 msgr2=0x7f3084071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 --2- 172.21.15.87:0/2145878492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30840aebd0 0x7f3084071ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f306c0097c0 tx=0x7f306c01c470 comp rx=0 tx=0).stop 2024-04-25T10:12:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 -- 172.21.15.87:0/2145878492 shutdown_connections 2024-04-25T10:12:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 --2- 172.21.15.87:0/2145878492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30840aebd0 0x7f3084071ae0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 --2- 172.21.15.87:0/2145878492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084159540 0x7f30840ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 -- 172.21.15.87:0/2145878492 >> 172.21.15.87:0/2145878492 conn(0x7f30840669a0 msgr2=0x7f3084066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 -- 172.21.15.87:0/2145878492 shutdown_connections 2024-04-25T10:12:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 -- 172.21.15.87:0/2145878492 wait complete. 2024-04-25T10:12:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 Processor -- start 2024-04-25T10:12:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.336+0000 7f308a200640 1 -- start start 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084072020 0x7f308414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30840aebd0 0x7f308413efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3084159540 0x7f3084150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308406c0d0 con 0x7f3084159540 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308415bfa0 con 0x7f30840aebd0 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3084072430 con 0x7f3084072020 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3084159540 0x7f3084150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3084159540 0x7f3084150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40618/0 (socket says 172.21.15.87:40618) 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 -- 172.21.15.87:0/106976419 learned_addr learned my addr 172.21.15.87:0/106976419 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083400640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084072020 0x7f308414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f307aa00640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30840aebd0 0x7f308413efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 -- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084072020 msgr2=0x7f308414d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084072020 0x7f308414d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 -- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30840aebd0 msgr2=0x7f308413efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30840aebd0 0x7f308413efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 -- 172.21.15.87:0/106976419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308414d8e0 con 0x7f3084159540 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083400640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084072020 0x7f308414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f3083e00640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3084159540 0x7f3084150ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c480 tx=0x7f307c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f307be00640 1 -- 172.21.15.87:0/106976419 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c018070 con 0x7f3084159540 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f307be00640 1 -- 172.21.15.87:0/106976419 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c0074e0 con 0x7f3084159540 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308414dbc0 con 0x7f3084159540 2024-04-25T10:12:48.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f307be00640 1 -- 172.21.15.87:0/106976419 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c004a30 con 0x7f3084159540 2024-04-25T10:12:48.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308414e180 con 0x7f3084159540 2024-04-25T10:12:48.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.340+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308414e3c0 con 0x7f3084159540 2024-04-25T10:12:48.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.344+0000 7f307be00640 1 -- 172.21.15.87:0/106976419 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f307c01b070 con 0x7f3084159540 2024-04-25T10:12:48.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.344+0000 7f307be00640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3060066f60 0x7f3060069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.344+0000 7f307be00640 1 -- 172.21.15.87:0/106976419 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f307c0842b0 con 0x7f3084159540 2024-04-25T10:12:48.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.344+0000 7f3083400640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3060066f60 0x7f3060069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.344+0000 7f3083400640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3060066f60 0x7f3060069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f30840ae650 tx=0x7f307400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.348+0000 7f307be00640 1 -- 172.21.15.87:0/106976419 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f307c055350 con 0x7f3084159540 2024-04-25T10:12:48.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.500+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3084166700 con 0x7f3084159540 2024-04-25T10:12:48.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.500+0000 7f307be00640 1 -- 172.21.15.87:0/106976419 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f307c054c50 con 0x7f3084159540 2024-04-25T10:12:48.508 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3060066f60 msgr2=0x7f3060069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3060066f60 0x7f3060069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f30840ae650 tx=0x7f307400b040 comp rx=0 tx=0).stop 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3084159540 msgr2=0x7f3084150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3084159540 0x7f3084150ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c480 tx=0x7f307c00c940 comp rx=0 tx=0).stop 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 shutdown_connections 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3060066f60 0x7f3060069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3084159540 0x7f3084150ad0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30840aebd0 0x7f308413efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 --2- 172.21.15.87:0/106976419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3084072020 0x7f308414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 >> 172.21.15.87:0/106976419 conn(0x7f30840669a0 msgr2=0x7f30840cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 shutdown_connections 2024-04-25T10:12:48.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.504+0000 7f308a200640 1 -- 172.21.15.87:0/106976419 wait complete. 2024-04-25T10:12:48.528 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:48.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.660+0000 7fe9e2600640 1 Processor -- start 2024-04-25T10:12:48.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.660+0000 7fe9e2600640 1 -- start start 2024-04-25T10:12:48.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9e2600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9dc066c00 0x7fe9dc161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9e2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc0b5560 0x7fe9dc0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9e2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc0b40c0 con 0x7fe9dc066fe0 2024-04-25T10:12:48.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9e2600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc161d60 con 0x7fe9dc066c00 2024-04-25T10:12:48.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9e2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0b5b00 con 0x7fe9dc0b5560 2024-04-25T10:12:48.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9dbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc0b5560 0x7fe9dc0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9db400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9dc066c00 0x7fe9dc161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9dbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc0b5560 0x7fe9dc0ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:42996/0 (socket says 172.21.15.87:42996) 2024-04-25T10:12:48.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9dbe00640 1 -- 172.21.15.87:0/3577131487 learned_addr learned my addr 172.21.15.87:0/3577131487 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:48.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9dbe00640 1 -- 172.21.15.87:0/3577131487 >> v2:172.21.15.169:3300/0 conn(0x7fe9dc066c00 msgr2=0x7fe9dc161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9dbe00640 1 --2- 172.21.15.87:0/3577131487 >> v2:172.21.15.169:3300/0 conn(0x7fe9dc066c00 0x7fe9dc161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9dbe00640 1 -- 172.21.15.87:0/3577131487 >> v1:172.21.15.169:6789/0 conn(0x7fe9dc066fe0 legacy=0x7fe9dc072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.664+0000 7fe9dbe00640 1 -- 172.21.15.87:0/3577131487 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0b3a00 con 0x7fe9dc0b5560 2024-04-25T10:12:48.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9dbe00640 1 --2- 172.21.15.87:0/3577131487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc0b5560 0x7fe9dc0ae850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009870 tx=0x7fe9d401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=586cf1ea4f4a4e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9da000640 1 -- 172.21.15.87:0/3577131487 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40b0070 con 0x7fe9dc0b5560 2024-04-25T10:12:48.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9da000640 1 -- 172.21.15.87:0/3577131487 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4004230 con 0x7fe9dc0b5560 2024-04-25T10:12:48.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9da000640 1 -- 172.21.15.87:0/3577131487 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4003620 con 0x7fe9dc0b5560 2024-04-25T10:12:48.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 -- 172.21.15.87:0/3577131487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc0b5560 msgr2=0x7fe9dc0ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3577131487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc0b5560 0x7fe9dc0ae850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009870 tx=0x7fe9d401c440 comp rx=0 tx=0).stop 2024-04-25T10:12:48.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 -- 172.21.15.87:0/3577131487 shutdown_connections 2024-04-25T10:12:48.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3577131487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc0b5560 0x7fe9dc0ae850 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3577131487 >> v2:172.21.15.169:3300/0 conn(0x7fe9dc066c00 0x7fe9dc161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 -- 172.21.15.87:0/3577131487 >> 172.21.15.87:0/3577131487 conn(0x7fe9dc0b2560 msgr2=0x7fe9dc0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 -- 172.21.15.87:0/3577131487 shutdown_connections 2024-04-25T10:12:48.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 -- 172.21.15.87:0/3577131487 wait complete. 2024-04-25T10:12:48.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 Processor -- start 2024-04-25T10:12:48.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.668+0000 7fe9e2600640 1 -- start start 2024-04-25T10:12:48.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9dc066c00 0x7fe9dc14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc066fe0 0x7fe9dc160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc0b5560 0x7fe9dc150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15e540 con 0x7fe9dc066c00 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc1585e0 con 0x7fe9dc0b5560 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15e6c0 con 0x7fe9dc066fe0 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9dc066c00 0x7fe9dc14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9dc066c00 0x7fe9dc14a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40620/0 (socket says 172.21.15.87:40620) 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 -- 172.21.15.87:0/3176978850 learned_addr learned my addr 172.21.15.87:0/3176978850 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 -- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc066fe0 msgr2=0x7fe9dc160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9dbe00640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc0b5560 0x7fe9dc150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9daa00640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc066fe0 0x7fe9dc160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc066fe0 0x7fe9dc160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 -- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc0b5560 msgr2=0x7fe9dc150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc0b5560 0x7fe9dc150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 -- 172.21.15.87:0/3176978850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc15f7f0 con 0x7fe9dc066c00 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9dbe00640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc0b5560 0x7fe9dc150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9daa00640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc066fe0 0x7fe9dc160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9db400640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9dc066c00 0x7fe9dc14a760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00efc0 tx=0x7fe9cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9d3400640 1 -- 172.21.15.87:0/3176978850 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc018020 con 0x7fe9dc066c00 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9d3400640 1 -- 172.21.15.87:0/3176978850 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc014480 con 0x7fe9dc066c00 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9d3400640 1 -- 172.21.15.87:0/3176978850 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc014850 con 0x7fe9dc066c00 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc15fa90 con 0x7fe9dc066c00 2024-04-25T10:12:48.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc160050 con 0x7fe9dc066c00 2024-04-25T10:12:48.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc04ef30 con 0x7fe9dc066c00 2024-04-25T10:12:48.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.672+0000 7fe9d3400640 1 -- 172.21.15.87:0/3176978850 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9cc01b020 con 0x7fe9dc066c00 2024-04-25T10:12:48.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.676+0000 7fe9d3400640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9bc066f60 0x7fe9bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.676+0000 7fe9d3400640 1 -- 172.21.15.87:0/3176978850 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9cc085bd0 con 0x7fe9dc066c00 2024-04-25T10:12:48.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.676+0000 7fe9daa00640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9bc066f60 0x7fe9bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.676+0000 7fe9daa00640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9bc066f60 0x7fe9bc069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc162880 tx=0x7fe9c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.680+0000 7fe9d3400640 1 -- 172.21.15.87:0/3176978850 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9cc022070 con 0x7fe9dc066c00 2024-04-25T10:12:48.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.832+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9dc1699a0 con 0x7fe9dc066c00 2024-04-25T10:12:48.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.832+0000 7fe9d3400640 1 -- 172.21.15.87:0/3176978850 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9cc010040 con 0x7fe9dc066c00 2024-04-25T10:12:48.839 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9bc066f60 msgr2=0x7fe9bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9bc066f60 0x7fe9bc069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc162880 tx=0x7fe9c8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9dc066c00 msgr2=0x7fe9dc14a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9dc066c00 0x7fe9dc14a760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00efc0 tx=0x7fe9cc00c490 comp rx=0 tx=0).stop 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 shutdown_connections 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9bc066f60 0x7fe9bc069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc0b5560 0x7fe9dc150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9dc066fe0 0x7fe9dc160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 --2- 172.21.15.87:0/3176978850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9dc066c00 0x7fe9dc14a760 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 >> 172.21.15.87:0/3176978850 conn(0x7fe9dc0b2560 msgr2=0x7fe9dc0ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 shutdown_connections 2024-04-25T10:12:48.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.836+0000 7fe9e2600640 1 -- 172.21.15.87:0/3176978850 wait complete. 2024-04-25T10:12:48.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:48.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.992+0000 7f26c1200640 1 Processor -- start 2024-04-25T10:12:48.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.992+0000 7f26c1200640 1 -- start start 2024-04-25T10:12:48.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.992+0000 7f26c1200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f26bc0cd9a0 0x7f26bc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.992+0000 7f26c1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc15d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.992+0000 7f26c1200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26bc06ac90 con 0x7f26bc06bd10 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.992+0000 7f26c1200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc06b880 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.992+0000 7f26c1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc06af50 con 0x7f26bc162020 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f26bc0cd9a0 0x7f26bc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f26bc0cd9a0 0x7f26bc0d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40630/0 (socket says 172.21.15.87:40630) 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 -- 172.21.15.87:0/3091007269 learned_addr learned my addr 172.21.15.87:0/3091007269 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:49.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26ba800640 1 --2- 172.21.15.87:0/3091007269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 -- 172.21.15.87:0/3091007269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 msgr2=0x7f26bc15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 --2- 172.21.15.87:0/3091007269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc15d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 -- 172.21.15.87:0/3091007269 >> v1:172.21.15.169:6789/0 conn(0x7f26bc06bd10 legacy=0x7f26bc161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 -- 172.21.15.87:0/3091007269 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc06b230 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9e00640 1 --2- 172.21.15.87:0/3091007269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc0d0660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009870 tx=0x7f26ac01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fb4c85fed06cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9400640 1 -- 172.21.15.87:0/3091007269 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac0b5070 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9400640 1 -- 172.21.15.87:0/3091007269 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac0a8a50 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:48.996+0000 7f26b9400640 1 -- 172.21.15.87:0/3091007269 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac0a8e20 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- 172.21.15.87:0/3091007269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 msgr2=0x7f26bc0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 --2- 172.21.15.87:0/3091007269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc0d0660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009870 tx=0x7f26ac01ed50 comp rx=0 tx=0).stop 2024-04-25T10:12:49.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- 172.21.15.87:0/3091007269 shutdown_connections 2024-04-25T10:12:49.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 --2- 172.21.15.87:0/3091007269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc15d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 --2- 172.21.15.87:0/3091007269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc0d0660 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- 172.21.15.87:0/3091007269 >> 172.21.15.87:0/3091007269 conn(0x7f26bc0d19a0 msgr2=0x7f26bc0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- 172.21.15.87:0/3091007269 shutdown_connections 2024-04-25T10:12:49.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- 172.21.15.87:0/3091007269 wait complete. 2024-04-25T10:12:49.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 Processor -- start 2024-04-25T10:12:49.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- start start 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc06bd10 0x7f26bc156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc09b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc09d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc162660 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0cdd80 con 0x7f26bc06bd10 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26c1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0d0ba0 con 0x7f26bc162020 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc09b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc09b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40638/0 (socket says 172.21.15.87:40638) 2024-04-25T10:12:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 -- 172.21.15.87:0/2288905995 learned_addr learned my addr 172.21.15.87:0/2288905995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b9e00640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc06bd10 0x7f26bc156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26ba800640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc09d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 -- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 msgr2=0x7f26bc09d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc09d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 -- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc06bd10 msgr2=0x7f26bc156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc06bd10 0x7f26bc156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26b1400640 1 -- 172.21.15.87:0/2288905995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc09c100 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.000+0000 7f26ba800640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc09d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b1400640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc09b820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f26a400c420 tx=0x7f26a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b2a00640 1 -- 172.21.15.87:0/2288905995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40052d0 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b2a00640 1 -- 172.21.15.87:0/2288905995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a4005470 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b2a00640 1 -- 172.21.15.87:0/2288905995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4007260 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc09c3e0 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc09c9a0 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc04ef30 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b2a00640 1 -- 172.21.15.87:0/2288905995 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26a4007400 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b2a00640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2698066f60 0x7f2698069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b2a00640 1 -- 172.21.15.87:0/2288905995 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26a40844c0 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.004+0000 7f26b9e00640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2698066f60 0x7f2698069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.008+0000 7f26b9e00640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2698066f60 0x7f2698069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0b2000 tx=0x7f26ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.008+0000 7f26b2a00640 1 -- 172.21.15.87:0/2288905995 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26a4054e70 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.164+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26bc0d1f90 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.164+0000 7f26b2a00640 1 -- 172.21.15.87:0/2288905995 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26a4058f10 con 0x7f26bc0cd9a0 2024-04-25T10:12:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.164+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2698066f60 msgr2=0x7f2698069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.164+0000 7f26c1200640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2698066f60 0x7f2698069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0b2000 tx=0x7f26ac009040 comp rx=0 tx=0).stop 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.164+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 msgr2=0x7f26bc09b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.164+0000 7f26c1200640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc09b820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f26a400c420 tx=0x7f26a400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 shutdown_connections 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2698066f60 0x7f2698069420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc162020 0x7f26bc09d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26bc0cd9a0 0x7f26bc09b820 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 --2- 172.21.15.87:0/2288905995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc06bd10 0x7f26bc156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 >> 172.21.15.87:0/2288905995 conn(0x7f26bc0d19a0 msgr2=0x7f26bc06ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 shutdown_connections 2024-04-25T10:12:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.168+0000 7f26c1200640 1 -- 172.21.15.87:0/2288905995 wait complete. 2024-04-25T10:12:49.190 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd316a00640 1 Processor -- start 2024-04-25T10:12:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd316a00640 1 -- start start 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd316a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd31015b9e0 0x7fd3100cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd316a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3100ccfb0 con 0x7fd31015bdc0 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd316a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3100cd290 con 0x7fd3100d1c30 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd316a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3100cd570 con 0x7fd31015b9e0 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd307400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd31015bdc0 0x7fd3100d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40232/0 (socket says 172.21.15.87:40232) 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd307400640 1 -- 172.21.15.87:0/1878267059 learned_addr learned my addr 172.21.15.87:0/1878267059 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30fe00640 1 --2- 172.21.15.87:0/1878267059 >> v2:172.21.15.169:3300/0 conn(0x7fd31015b9e0 0x7fd3100cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730934758 0 0) 0x7fd3100ccfb0 con 0x7fd31015bdc0 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f4003530 con 0x7fd31015bdc0 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3801713261 0 0) 0x7fd2f4003530 con 0x7fd31015bdc0 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 >> v2:172.21.15.169:3300/0 conn(0x7fd31015b9e0 msgr2=0x7fd3100cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 --2- 172.21.15.87:0/1878267059 >> v2:172.21.15.169:3300/0 conn(0x7fd31015b9e0 0x7fd3100cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 >> v1:172.21.15.169:6789/0 conn(0x7fd3100d1c30 legacy=0x7fd3100c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100cd850 con 0x7fd31015bdc0 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd2f8001680 con 0x7fd31015bdc0 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 >> v1:172.21.15.87:6789/0 conn(0x7fd31015bdc0 legacy=0x7fd3100d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f40039c0 0x7fd2f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f4004810 0x7fd2f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40051d0 0x7fd2f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100cd570 con 0x7fd2f40051d0 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f400c170 con 0x7fd2f4004810 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f400c370 con 0x7fd2f40039c0 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd314a00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40051d0 0x7fd2f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd30fe00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f40039c0 0x7fd2f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.320+0000 7fd307400640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f4004810 0x7fd2f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30fe00640 1 -- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f4004810 msgr2=0x7fd2f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30fe00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f4004810 0x7fd2f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30fe00640 1 -- 172.21.15.87:0/1878267059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40051d0 msgr2=0x7fd2f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30fe00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40051d0 0x7fd2f400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30fe00640 1 -- 172.21.15.87:0/1878267059 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4003530 con 0x7fd2f40039c0 2024-04-25T10:12:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd314a00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40051d0 0x7fd2f400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd314a00640 1 -- 172.21.15.87:0/1878267059 reap_dead start 2024-04-25T10:12:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30fe00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f40039c0 0x7fd2f4003de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd300009ae0 tx=0x7fd30001ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5e912cd3a010da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000096b0 con 0x7fd2f40039c0 2024-04-25T10:12:49.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3000a6480 con 0x7fd2f40039c0 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd30f400640 1 -- 172.21.15.87:0/1878267059 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000a6870 con 0x7fd2f40039c0 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd316a00640 1 -- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f40039c0 msgr2=0x7fd2f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.324+0000 7fd316a00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f40039c0 0x7fd2f4003de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd300009ae0 tx=0x7fd30001ca70 comp rx=0 tx=0).stop 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- 172.21.15.87:0/1878267059 shutdown_connections 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 --2- 172.21.15.87:0/1878267059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f40039c0 0x7fd2f4003de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- 172.21.15.87:0/1878267059 >> 172.21.15.87:0/1878267059 conn(0x7fd31009a9d0 msgr2=0x7fd31009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- 172.21.15.87:0/1878267059 shutdown_connections 2024-04-25T10:12:49.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- 172.21.15.87:0/1878267059 wait complete. 2024-04-25T10:12:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 Processor -- start 2024-04-25T10:12:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- start start 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40039c0 0x7fd31014b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31009b720 0x7fd31009bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3100cb340 0x7fd3100cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100d1520 con 0x7fd2f40039c0 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100cf740 con 0x7fd31009b720 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd316a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100d16a0 con 0x7fd3100cb340 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40039c0 0x7fd31014b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40039c0 0x7fd31014b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40666/0 (socket says 172.21.15.87:40666) 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 -- 172.21.15.87:0/4187448485 learned_addr learned my addr 172.21.15.87:0/4187448485 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 -- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3100cb340 msgr2=0x7fd3100cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3100cb340 0x7fd3100cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 -- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31009b720 msgr2=0x7fd31009bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd307400640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31009b720 0x7fd31009bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31009b720 0x7fd31009bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 -- 172.21.15.87:0/4187448485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100ca5a0 con 0x7fd2f40039c0 2024-04-25T10:12:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd307400640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31009b720 0x7fd31009bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30fe00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40039c0 0x7fd31014b450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd30001c8a0 tx=0x7fd300019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.328+0000 7fd30cc00640 1 -- 172.21.15.87:0/4187448485 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000a6e30 con 0x7fd2f40039c0 2024-04-25T10:12:49.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd30cc00640 1 -- 172.21.15.87:0/4187448485 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3000a7460 con 0x7fd2f40039c0 2024-04-25T10:12:49.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3100ca860 con 0x7fd2f40039c0 2024-04-25T10:12:49.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd30cc00640 1 -- 172.21.15.87:0/4187448485 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000a7830 con 0x7fd2f40039c0 2024-04-25T10:12:49.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3100cae20 con 0x7fd2f40039c0 2024-04-25T10:12:49.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31004ef30 con 0x7fd2f40039c0 2024-04-25T10:12:49.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd30cc00640 1 -- 172.21.15.87:0/4187448485 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3000a79d0 con 0x7fd2f40039c0 2024-04-25T10:12:49.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd30cc00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2ec066f60 0x7fd2ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd30cc00640 1 -- 172.21.15.87:0/4187448485 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd30011f120 con 0x7fd2f40039c0 2024-04-25T10:12:49.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd307400640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2ec066f60 0x7fd2ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.332+0000 7fd307400640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2ec066f60 0x7fd2ec069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8007cd0 tx=0x7fd2f8007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.336+0000 7fd30cc00640 1 -- 172.21.15.87:0/4187448485 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3000ac020 con 0x7fd2f40039c0 2024-04-25T10:12:49.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.492+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3100b7760 con 0x7fd2f40039c0 2024-04-25T10:12:49.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.492+0000 7fd30cc00640 1 -- 172.21.15.87:0/4187448485 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fd300016780 con 0x7fd2f40039c0 2024-04-25T10:12:49.499 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:49.499 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2ec066f60 msgr2=0x7fd2ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2ec066f60 0x7fd2ec069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8007cd0 tx=0x7fd2f8007900 comp rx=0 tx=0).stop 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40039c0 msgr2=0x7fd31014b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40039c0 0x7fd31014b450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd30001c8a0 tx=0x7fd300019810 comp rx=0 tx=0).stop 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 shutdown_connections 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2ec066f60 0x7fd2ec069420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3100cb340 0x7fd3100cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31009b720 0x7fd31009bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 --2- 172.21.15.87:0/4187448485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f40039c0 0x7fd31014b450 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 >> 172.21.15.87:0/4187448485 conn(0x7fd31009a9d0 msgr2=0x7fd3100c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 shutdown_connections 2024-04-25T10:12:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.496+0000 7fd316a00640 1 -- 172.21.15.87:0/4187448485 wait complete. 2024-04-25T10:12:49.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:49.520 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:49.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58b3800640 1 Processor -- start 2024-04-25T10:12:49.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58b3800640 1 -- start start 2024-04-25T10:12:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58b3800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f58ac0b38d0 0x7f58ac15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58b3800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac15b080 con 0x7f58ac070350 2024-04-25T10:12:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58b3800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac15adb0 con 0x7f58ac0aef80 2024-04-25T10:12:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58b3800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac15eb60 con 0x7f58ac0b38d0 2024-04-25T10:12:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58a3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f58ac070350 0x7f58ac0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40236/0 (socket says 172.21.15.87:40236) 2024-04-25T10:12:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58a3e00640 1 -- 172.21.15.87:0/2920766797 learned_addr learned my addr 172.21.15.87:0/2920766797 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.652+0000 7f58b1800640 1 --2- 172.21.15.87:0/2920766797 >> v2:172.21.15.87:3300/0 conn(0x7f58ac0b38d0 0x7f58ac15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b1800640 1 -- 172.21.15.87:0/2920766797 >> v1:172.21.15.169:6789/0 conn(0x7f58ac0aef80 legacy=0x7f58ac15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b1800640 1 -- 172.21.15.87:0/2920766797 >> v1:172.21.15.87:6789/0 conn(0x7f58ac070350 legacy=0x7f58ac0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b1800640 1 -- 172.21.15.87:0/2920766797 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac15ee40 con 0x7f58ac0b38d0 2024-04-25T10:12:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58abe00640 1 -- 172.21.15.87:0/2920766797 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174361070 0 0) 0x7f58ac15b080 con 0x7f58ac070350 2024-04-25T10:12:49.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b1800640 1 --2- 172.21.15.87:0/2920766797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0b38d0 0x7f58ac15e480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009d90 tx=0x7f58a401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffe36c0358272be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58abe00640 1 -- 172.21.15.87:0/2920766797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40b2070 con 0x7f58ac0b38d0 2024-04-25T10:12:49.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58abe00640 1 -- 172.21.15.87:0/2920766797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a40a5a10 con 0x7f58ac0b38d0 2024-04-25T10:12:49.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58abe00640 1 -- 172.21.15.87:0/2920766797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40a5de0 con 0x7f58ac0b38d0 2024-04-25T10:12:49.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b3800640 1 -- 172.21.15.87:0/2920766797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0b38d0 msgr2=0x7f58ac15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b3800640 1 --2- 172.21.15.87:0/2920766797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0b38d0 0x7f58ac15e480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009d90 tx=0x7f58a401c970 comp rx=0 tx=0).stop 2024-04-25T10:12:49.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b3800640 1 -- 172.21.15.87:0/2920766797 shutdown_connections 2024-04-25T10:12:49.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b3800640 1 --2- 172.21.15.87:0/2920766797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0b38d0 0x7f58ac15e480 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b3800640 1 -- 172.21.15.87:0/2920766797 >> 172.21.15.87:0/2920766797 conn(0x7f58ac0669a0 msgr2=0x7f58ac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b3800640 1 -- 172.21.15.87:0/2920766797 shutdown_connections 2024-04-25T10:12:49.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.656+0000 7f58b3800640 1 -- 172.21.15.87:0/2920766797 wait complete. 2024-04-25T10:12:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 Processor -- start 2024-04-25T10:12:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 -- start start 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac070350 0x7f58ac13d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0aef80 0x7f58ac151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac0aa380 0x7f58ac0aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac1689b0 con 0x7f58ac0aef80 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac168830 con 0x7f58ac0aa380 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac168b30 con 0x7f58ac070350 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0aef80 0x7f58ac151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0aef80 0x7f58ac151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40692/0 (socket says 172.21.15.87:40692) 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 -- 172.21.15.87:0/2416489694 learned_addr learned my addr 172.21.15.87:0/2416489694 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b0e00640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac070350 0x7f58ac13d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b1800640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac0aa380 0x7f58ac0aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 -- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac070350 msgr2=0x7f58ac13d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac070350 0x7f58ac13d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 -- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac0aa380 msgr2=0x7f58ac0aa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac0aa380 0x7f58ac0aa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 -- 172.21.15.87:0/2416489694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac152780 con 0x7f58ac0aef80 2024-04-25T10:12:49.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a3e00640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0aef80 0x7f58ac151e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f589400c4a0 tx=0x7f589400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a9600640 1 -- 172.21.15.87:0/2416489694 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894018070 con 0x7f58ac0aef80 2024-04-25T10:12:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a9600640 1 -- 172.21.15.87:0/2416489694 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58940074e0 con 0x7f58ac0aef80 2024-04-25T10:12:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58a9600640 1 -- 172.21.15.87:0/2416489694 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894004a30 con 0x7f58ac0aef80 2024-04-25T10:12:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.660+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac152a60 con 0x7f58ac0aef80 2024-04-25T10:12:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.664+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac153020 con 0x7f58ac0aef80 2024-04-25T10:12:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.664+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac04ef30 con 0x7f58ac0aef80 2024-04-25T10:12:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.664+0000 7f58a9600640 1 -- 172.21.15.87:0/2416489694 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5894004bd0 con 0x7f58ac0aef80 2024-04-25T10:12:49.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.664+0000 7f58a9600640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5888066f60 0x7f5888069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.664+0000 7f58a9600640 1 -- 172.21.15.87:0/2416489694 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5894084290 con 0x7f58ac0aef80 2024-04-25T10:12:49.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.664+0000 7f58b0e00640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5888066f60 0x7f5888069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.664+0000 7f58b0e00640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5888066f60 0x7f5888069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58ac070df0 tx=0x7f589c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.668+0000 7f58a9600640 1 -- 172.21.15.87:0/2416489694 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5894054cc0 con 0x7f58ac0aef80 2024-04-25T10:12:49.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.820+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58ac0b2fa0 con 0x7f58ac0aef80 2024-04-25T10:12:49.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.820+0000 7f58a9600640 1 -- 172.21.15.87:0/2416489694 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5894058d60 con 0x7f58ac0aef80 2024-04-25T10:12:49.827 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5888066f60 msgr2=0x7f5888069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5888066f60 0x7f5888069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58ac070df0 tx=0x7f589c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0aef80 msgr2=0x7f58ac151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0aef80 0x7f58ac151e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f589400c4a0 tx=0x7f589400c960 comp rx=0 tx=0).stop 2024-04-25T10:12:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 shutdown_connections 2024-04-25T10:12:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5888066f60 0x7f5888069420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac0aa380 0x7f58ac0aa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58ac0aef80 0x7f58ac151e10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 --2- 172.21.15.87:0/2416489694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac070350 0x7f58ac13d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 >> 172.21.15.87:0/2416489694 conn(0x7f58ac0669a0 msgr2=0x7f58ac15a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 shutdown_connections 2024-04-25T10:12:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.824+0000 7f58b3800640 1 -- 172.21.15.87:0/2416489694 wait complete. 2024-04-25T10:12:49.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.980+0000 7f8fd7a00640 1 Processor -- start 2024-04-25T10:12:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.980+0000 7f8fd7a00640 1 -- start start 2024-04-25T10:12:49.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.980+0000 7f8fd7a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8fd0159540 0x7f8fd00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.980+0000 7f8fd7a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8fd00aebd0 0x7f8fd0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.980+0000 7f8fd7a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd0071360 con 0x7f8fd0072020 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.980+0000 7f8fd7a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd0159920 con 0x7f8fd0159540 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.980+0000 7f8fd7a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd015c2f0 con 0x7f8fd00aebd0 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8fd0159540 0x7f8fd00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8fd0159540 0x7f8fd00ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40696/0 (socket says 172.21.15.87:40696) 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 -- 172.21.15.87:0/3908980733 learned_addr learned my addr 172.21.15.87:0/3908980733 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:49.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fc7e00640 1 --2- 172.21.15.87:0/3908980733 >> v2:172.21.15.169:3300/0 conn(0x7f8fd00aebd0 0x7f8fd0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 -- 172.21.15.87:0/3908980733 >> v2:172.21.15.169:3300/0 conn(0x7f8fd00aebd0 msgr2=0x7f8fd0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 --2- 172.21.15.87:0/3908980733 >> v2:172.21.15.169:3300/0 conn(0x7f8fd00aebd0 0x7f8fd0071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 -- 172.21.15.87:0/3908980733 >> v1:172.21.15.169:6789/0 conn(0x7f8fd0072020 legacy=0x7f8fd015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 -- 172.21.15.87:0/3908980733 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd015c5d0 con 0x7f8fd0159540 2024-04-25T10:12:49.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fd5000640 1 --2- 172.21.15.87:0/3908980733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00ae690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00cd4e0 tx=0x7f8fc001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd701cff2142f675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fcfe00640 1 -- 172.21.15.87:0/3908980733 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00b0070 con 0x7f8fd0159540 2024-04-25T10:12:49.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fcfe00640 1 -- 172.21.15.87:0/3908980733 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc00ab600 con 0x7f8fd0159540 2024-04-25T10:12:49.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.984+0000 7f8fcfe00640 1 -- 172.21.15.87:0/3908980733 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00ab9d0 con 0x7f8fd0159540 2024-04-25T10:12:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- 172.21.15.87:0/3908980733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 msgr2=0x7f8fd00ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/3908980733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00ae690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00cd4e0 tx=0x7f8fc001c520 comp rx=0 tx=0).stop 2024-04-25T10:12:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- 172.21.15.87:0/3908980733 shutdown_connections 2024-04-25T10:12:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/3908980733 >> v2:172.21.15.169:3300/0 conn(0x7f8fd00aebd0 0x7f8fd0071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/3908980733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00ae690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- 172.21.15.87:0/3908980733 >> 172.21.15.87:0/3908980733 conn(0x7f8fd00669a0 msgr2=0x7f8fd0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- 172.21.15.87:0/3908980733 shutdown_connections 2024-04-25T10:12:49.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- 172.21.15.87:0/3908980733 wait complete. 2024-04-25T10:12:49.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 Processor -- start 2024-04-25T10:12:49.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- start start 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fd0072020 0x7f8fd0149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd00aebd0 0x7f8fd00b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0072430 con 0x7f8fd0159540 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd006c0d0 con 0x7f8fd00aebd0 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd7a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0168930 con 0x7f8fd0072020 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40712/0 (socket says 172.21.15.87:40712) 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 -- 172.21.15.87:0/4178629919 learned_addr learned my addr 172.21.15.87:0/4178629919 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5000640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fd0072020 0x7f8fd0149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fc7e00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd00aebd0 0x7f8fd00b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 -- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fd0072020 msgr2=0x7f8fd0149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fd0072020 0x7f8fd0149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 -- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd00aebd0 msgr2=0x7f8fd00b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd00aebd0 0x7f8fd00b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.988+0000 7f8fd5a00640 1 -- 172.21.15.87:0/4178629919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00b3cb0 con 0x7f8fd0159540 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fd5a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00b14b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800c770 tx=0x7f8fc800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fcd600640 1 -- 172.21.15.87:0/4178629919 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8003b80 con 0x7f8fd0159540 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fcd600640 1 -- 172.21.15.87:0/4178629919 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc8003d20 con 0x7f8fd0159540 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd00b3f90 con 0x7f8fd0159540 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fcd600640 1 -- 172.21.15.87:0/4178629919 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800aa30 con 0x7f8fd0159540 2024-04-25T10:12:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd00b4550 con 0x7f8fd0159540 2024-04-25T10:12:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd004ef30 con 0x7f8fd0159540 2024-04-25T10:12:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fcd600640 1 -- 172.21.15.87:0/4178629919 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fc800abd0 con 0x7f8fd0159540 2024-04-25T10:12:49.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fcd600640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fac066f60 0x7f8fac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fcd600640 1 -- 172.21.15.87:0/4178629919 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fc8084de0 con 0x7f8fd0159540 2024-04-25T10:12:49.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.992+0000 7f8fd5000640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fac066f60 0x7f8fac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.996+0000 7f8fd5000640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fac066f60 0x7f8fac069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00ad000 tx=0x7f8fc0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:49.996+0000 7f8fcd600640 1 -- 172.21.15.87:0/4178629919 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fc8055790 con 0x7f8fd0159540 2024-04-25T10:12:50.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fd006d7f0 con 0x7f8fd0159540 2024-04-25T10:12:50.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fcd600640 1 -- 172.21.15.87:0/4178629919 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8fc8059830 con 0x7f8fd0159540 2024-04-25T10:12:50.157 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fac066f60 msgr2=0x7f8fac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fac066f60 0x7f8fac069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00ad000 tx=0x7f8fc0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 msgr2=0x7f8fd00b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00b14b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800c770 tx=0x7f8fc800cc30 comp rx=0 tx=0).stop 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 shutdown_connections 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fac066f60 0x7f8fac069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fd0159540 0x7f8fd00b14b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd00aebd0 0x7f8fd00b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 --2- 172.21.15.87:0/4178629919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fd0072020 0x7f8fd0149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 >> 172.21.15.87:0/4178629919 conn(0x7f8fd00669a0 msgr2=0x7f8fd006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 shutdown_connections 2024-04-25T10:12:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.152+0000 7f8fd7a00640 1 -- 172.21.15.87:0/4178629919 wait complete. 2024-04-25T10:12:50.179 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cfa800640 1 Processor -- start 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cfa800640 1 -- start start 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cfa800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6cf40721f0 0x7f6cf4166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cfa800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf4071360 con 0x7f6cf4159540 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cfa800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf4159920 con 0x7f6cf40aeda0 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cfa800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf415c2f0 con 0x7f6cf40721f0 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6cf4159540 0x7f6cf40ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40250/0 (socket says 172.21.15.87:40250) 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf3400640 1 -- 172.21.15.87:0/2392634594 learned_addr learned my addr 172.21.15.87:0/2392634594 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf3e00640 1 --2- 172.21.15.87:0/2392634594 >> v2:172.21.15.169:3300/0 conn(0x7f6cf40721f0 0x7f6cf4166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf2000640 1 -- 172.21.15.87:0/2392634594 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062434176 0 0) 0x7f6cf4071360 con 0x7f6cf4159540 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf2000640 1 -- 172.21.15.87:0/2392634594 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd8003530 con 0x7f6cf4159540 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf3e00640 1 -- 172.21.15.87:0/2392634594 >> v1:172.21.15.169:6789/0 conn(0x7f6cf40aeda0 legacy=0x7f6cf4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf2000640 1 -- 172.21.15.87:0/2392634594 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137738237 0 0) 0x7f6cf4159920 con 0x7f6cf40aeda0 2024-04-25T10:12:50.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf3e00640 1 -- 172.21.15.87:0/2392634594 >> v1:172.21.15.87:6789/0 conn(0x7f6cf4159540 legacy=0x7f6cf40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.312+0000 7f6cf3e00640 1 -- 172.21.15.87:0/2392634594 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf415c5d0 con 0x7f6cf40721f0 2024-04-25T10:12:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cf3e00640 1 --2- 172.21.15.87:0/2392634594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40721f0 0x7f6cf4166920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0097c0 tx=0x7f6cec01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eee3fa9db761c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cf2000640 1 -- 172.21.15.87:0/2392634594 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0a7070 con 0x7f6cf40721f0 2024-04-25T10:12:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cf2000640 1 -- 172.21.15.87:0/2392634594 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec0b0070 con 0x7f6cf40721f0 2024-04-25T10:12:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cf2000640 1 -- 172.21.15.87:0/2392634594 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0ab890 con 0x7f6cf40721f0 2024-04-25T10:12:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cfa800640 1 -- 172.21.15.87:0/2392634594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40721f0 msgr2=0x7f6cf4166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cfa800640 1 --2- 172.21.15.87:0/2392634594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40721f0 0x7f6cf4166920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0097c0 tx=0x7f6cec01c470 comp rx=0 tx=0).stop 2024-04-25T10:12:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cfa800640 1 -- 172.21.15.87:0/2392634594 shutdown_connections 2024-04-25T10:12:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cfa800640 1 --2- 172.21.15.87:0/2392634594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40721f0 0x7f6cf4166920 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cfa800640 1 -- 172.21.15.87:0/2392634594 >> 172.21.15.87:0/2392634594 conn(0x7f6cf40669a0 msgr2=0x7f6cf4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cfa800640 1 -- 172.21.15.87:0/2392634594 shutdown_connections 2024-04-25T10:12:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.316+0000 7f6cfa800640 1 -- 172.21.15.87:0/2392634594 wait complete. 2024-04-25T10:12:50.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 Processor -- start 2024-04-25T10:12:50.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 -- start start 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf40721f0 0x7f6cf414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40aeda0 0x7f6cf415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cf4159540 0x7f6cf4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4169680 con 0x7f6cf40721f0 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf406c0d0 con 0x7f6cf40aeda0 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4169800 con 0x7f6cf4159540 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf40721f0 0x7f6cf414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf40721f0 0x7f6cf414d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40726/0 (socket says 172.21.15.87:40726) 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 -- 172.21.15.87:0/3829268017 learned_addr learned my addr 172.21.15.87:0/3829268017 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf2a00640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40aeda0 0x7f6cf415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3e00640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cf4159540 0x7f6cf4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 -- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cf4159540 msgr2=0x7f6cf4157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cf4159540 0x7f6cf4157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 -- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40aeda0 msgr2=0x7f6cf415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40aeda0 0x7f6cf415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 -- 172.21.15.87:0/3829268017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf415b320 con 0x7f6cf40721f0 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3e00640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cf4159540 0x7f6cf4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cf3400640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf40721f0 0x7f6cf414d740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400cad0 tx=0x7f6ce400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6ceb400640 1 -- 172.21.15.87:0/3829268017 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4017070 con 0x7f6cf40721f0 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6ceb400640 1 -- 172.21.15.87:0/3829268017 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce400ac30 con 0x7f6cf40721f0 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6ceb400640 1 -- 172.21.15.87:0/3829268017 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4012400 con 0x7f6cf40721f0 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf415b600 con 0x7f6cf40721f0 2024-04-25T10:12:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.320+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf414f5e0 con 0x7f6cf40721f0 2024-04-25T10:12:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.324+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf404ef30 con 0x7f6cf40721f0 2024-04-25T10:12:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.324+0000 7f6ceb400640 1 -- 172.21.15.87:0/3829268017 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ce401a070 con 0x7f6cf40721f0 2024-04-25T10:12:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.324+0000 7f6ceb400640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6cd4066f60 0x7f6cd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.324+0000 7f6ceb400640 1 -- 172.21.15.87:0/3829268017 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ce4084e20 con 0x7f6cf40721f0 2024-04-25T10:12:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.324+0000 7f6cf2a00640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6cd4066f60 0x7f6cd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.324+0000 7f6cf2a00640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6cd4066f60 0x7f6cd4069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40ae890 tx=0x7f6ce0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.328+0000 7f6ceb400640 1 -- 172.21.15.87:0/3829268017 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ce40556c0 con 0x7f6cf40721f0 2024-04-25T10:12:50.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6cf406d7f0 con 0x7f6cf40721f0 2024-04-25T10:12:50.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6ceb400640 1 -- 172.21.15.87:0/3829268017 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ce4059760 con 0x7f6cf40721f0 2024-04-25T10:12:50.490 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6cd4066f60 msgr2=0x7f6cd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6cd4066f60 0x7f6cd4069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40ae890 tx=0x7f6ce0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf40721f0 msgr2=0x7f6cf414d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf40721f0 0x7f6cf414d740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400cad0 tx=0x7f6ce400cf90 comp rx=0 tx=0).stop 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 shutdown_connections 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6cd4066f60 0x7f6cd4069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cf4159540 0x7f6cf4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cf40aeda0 0x7f6cf415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 --2- 172.21.15.87:0/3829268017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf40721f0 0x7f6cf414d740 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 >> 172.21.15.87:0/3829268017 conn(0x7f6cf40669a0 msgr2=0x7f6cf4071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.483+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 shutdown_connections 2024-04-25T10:12:50.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.487+0000 7f6cfa800640 1 -- 172.21.15.87:0/3829268017 wait complete. 2024-04-25T10:12:50.510 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:50.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe19e000640 1 Processor -- start 2024-04-25T10:12:50.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe19e000640 1 -- start start 2024-04-25T10:12:50.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe19e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe19815bb50 0x7fe1980b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe19e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1980d24d0 0x7fe1980cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe19e000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980ce430 con 0x7fe1980d0490 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe19e000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1980d08f0 con 0x7fe1980d24d0 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe19e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1980b1070 con 0x7fe19815bb50 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe196a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1980d24d0 0x7fe1980cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe197e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe1980d0490 0x7fe1980cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40254/0 (socket says 172.21.15.87:40254) 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe197e00640 1 -- 172.21.15.87:0/1029015261 learned_addr learned my addr 172.21.15.87:0/1029015261 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe197400640 1 --2- 172.21.15.87:0/1029015261 >> v2:172.21.15.169:3300/0 conn(0x7fe19815bb50 0x7fe1980b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe196a00640 1 -- 172.21.15.87:0/1029015261 >> v2:172.21.15.169:3300/0 conn(0x7fe19815bb50 msgr2=0x7fe1980b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe196a00640 1 --2- 172.21.15.87:0/1029015261 >> v2:172.21.15.169:3300/0 conn(0x7fe19815bb50 0x7fe1980b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe196a00640 1 -- 172.21.15.87:0/1029015261 >> v1:172.21.15.87:6789/0 conn(0x7fe1980d0490 legacy=0x7fe1980cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.639+0000 7fe196a00640 1 -- 172.21.15.87:0/1029015261 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980ce6d0 con 0x7fe1980d24d0 2024-04-25T10:12:50.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe196a00640 1 --2- 172.21.15.87:0/1029015261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d24d0 0x7fe1980cff50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe1880097c0 tx=0x7fe18801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7328daa631b3e45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe196000640 1 -- 172.21.15.87:0/1029015261 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880ab430 con 0x7fe1980d24d0 2024-04-25T10:12:50.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe196000640 1 -- 172.21.15.87:0/1029015261 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1880b0070 con 0x7fe1980d24d0 2024-04-25T10:12:50.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe196000640 1 -- 172.21.15.87:0/1029015261 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880ab7f0 con 0x7fe1980d24d0 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 -- 172.21.15.87:0/1029015261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d24d0 msgr2=0x7fe1980cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 --2- 172.21.15.87:0/1029015261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d24d0 0x7fe1980cff50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe1880097c0 tx=0x7fe18801c430 comp rx=0 tx=0).stop 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 -- 172.21.15.87:0/1029015261 shutdown_connections 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 --2- 172.21.15.87:0/1029015261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d24d0 0x7fe1980cff50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 --2- 172.21.15.87:0/1029015261 >> v2:172.21.15.169:3300/0 conn(0x7fe19815bb50 0x7fe1980b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 -- 172.21.15.87:0/1029015261 >> 172.21.15.87:0/1029015261 conn(0x7fe1980d1970 msgr2=0x7fe1980d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 -- 172.21.15.87:0/1029015261 shutdown_connections 2024-04-25T10:12:50.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.643+0000 7fe19e000640 1 -- 172.21.15.87:0/1029015261 wait complete. 2024-04-25T10:12:50.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 Processor -- start 2024-04-25T10:12:50.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 -- start start 2024-04-25T10:12:50.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d0490 0x7fe198150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1980d24d0 0x7fe19815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19815bb50 0x7fe1981505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19815c0b0 con 0x7fe1980d0490 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19815bf30 con 0x7fe1980d24d0 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe198168830 con 0x7fe19815bb50 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d0490 0x7fe198150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d0490 0x7fe198150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40744/0 (socket says 172.21.15.87:40744) 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 -- 172.21.15.87:0/639690884 learned_addr learned my addr 172.21.15.87:0/639690884 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 -- 172.21.15.87:0/639690884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19815bb50 msgr2=0x7fe1981505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19815bb50 0x7fe1981505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 -- 172.21.15.87:0/639690884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1980d24d0 msgr2=0x7fe19815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1980d24d0 0x7fe19815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 -- 172.21.15.87:0/639690884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198151430 con 0x7fe1980d0490 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe197400640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d0490 0x7fe198150da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe18000d5f0 tx=0x7fe18000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe18f400640 1 -- 172.21.15.87:0/639690884 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1800125f0 con 0x7fe1980d0490 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe18f400640 1 -- 172.21.15.87:0/639690884 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe180019070 con 0x7fe1980d0490 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe18f400640 1 -- 172.21.15.87:0/639690884 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1800129b0 con 0x7fe1980d0490 2024-04-25T10:12:50.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe198151710 con 0x7fe1980d0490 2024-04-25T10:12:50.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe198151cd0 con 0x7fe1980d0490 2024-04-25T10:12:50.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.647+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19804ef30 con 0x7fe1980d0490 2024-04-25T10:12:50.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.651+0000 7fe18f400640 1 -- 172.21.15.87:0/639690884 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe180012b50 con 0x7fe1980d0490 2024-04-25T10:12:50.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.651+0000 7fe18f400640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe174066f90 0x7fe174069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.651+0000 7fe18f400640 1 -- 172.21.15.87:0/639690884 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe180085370 con 0x7fe1980d0490 2024-04-25T10:12:50.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.651+0000 7fe196a00640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe174066f90 0x7fe174069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.651+0000 7fe196a00640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe174066f90 0x7fe174069450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1880af000 tx=0x7fe188004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.655+0000 7fe18f400640 1 -- 172.21.15.87:0/639690884 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe180056410 con 0x7fe1980d0490 2024-04-25T10:12:50.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.807+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1980b1390 con 0x7fe1980d0490 2024-04-25T10:12:50.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe18f400640 1 -- 172.21.15.87:0/639690884 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fe180055d10 con 0x7fe1980d0490 2024-04-25T10:12:50.817 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:50.817 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe174066f90 msgr2=0x7fe174069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe174066f90 0x7fe174069450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1880af000 tx=0x7fe188004740 comp rx=0 tx=0).stop 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d0490 msgr2=0x7fe198150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d0490 0x7fe198150da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe18000d5f0 tx=0x7fe18000dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 shutdown_connections 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe174066f90 0x7fe174069450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19815bb50 0x7fe1981505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1980d24d0 0x7fe19815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 --2- 172.21.15.87:0/639690884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1980d0490 0x7fe198150da0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 >> 172.21.15.87:0/639690884 conn(0x7fe1980d1970 msgr2=0x7fe1980b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 shutdown_connections 2024-04-25T10:12:50.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.811+0000 7fe19e000640 1 -- 172.21.15.87:0/639690884 wait complete. 2024-04-25T10:12:50.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:50.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:50.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f76a1800640 1 Processor -- start 2024-04-25T10:12:50.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f76a1800640 1 -- start start 2024-04-25T10:12:50.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f76a1800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f769c0aef80 0x7f769c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f76a1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c070350 0x7f769c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f76a1800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c15b080 con 0x7f769c15bac0 2024-04-25T10:12:50.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f76a1800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c15adb0 con 0x7f769c0aef80 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f76a1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15eb60 con 0x7f769c070350 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f769c0aef80 0x7f769c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f769c0aef80 0x7f769c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40748/0 (socket says 172.21.15.87:40748) 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769aa00640 1 -- 172.21.15.87:0/3188593049 learned_addr learned my addr 172.21.15.87:0/3188593049 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769a000640 1 --2- 172.21.15.87:0/3188593049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c070350 0x7f769c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769aa00640 1 -- 172.21.15.87:0/3188593049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c070350 msgr2=0x7f769c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769aa00640 1 --2- 172.21.15.87:0/3188593049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c070350 0x7f769c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769aa00640 1 -- 172.21.15.87:0/3188593049 >> v1:172.21.15.169:6789/0 conn(0x7f769c15bac0 legacy=0x7f769c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.971+0000 7f769aa00640 1 -- 172.21.15.87:0/3188593049 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c15ee40 con 0x7f769c0aef80 2024-04-25T10:12:50.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f769aa00640 1 --2- 172.21.15.87:0/3188593049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c0aef80 0x7f769c15e480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f768c009e50 tx=0x7f768c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cafceae019c766 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f7699600640 1 -- 172.21.15.87:0/3188593049 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0a58e0 con 0x7f769c0aef80 2024-04-25T10:12:50.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f7699600640 1 -- 172.21.15.87:0/3188593049 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c0a5a80 con 0x7f769c0aef80 2024-04-25T10:12:50.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f7699600640 1 -- 172.21.15.87:0/3188593049 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0a5e50 con 0x7f769c0aef80 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 -- 172.21.15.87:0/3188593049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c0aef80 msgr2=0x7f769c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 --2- 172.21.15.87:0/3188593049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c0aef80 0x7f769c15e480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f768c009e50 tx=0x7f768c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 -- 172.21.15.87:0/3188593049 shutdown_connections 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 --2- 172.21.15.87:0/3188593049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c070350 0x7f769c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 --2- 172.21.15.87:0/3188593049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c0aef80 0x7f769c15e480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 -- 172.21.15.87:0/3188593049 >> 172.21.15.87:0/3188593049 conn(0x7f769c0669a0 msgr2=0x7f769c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 -- 172.21.15.87:0/3188593049 shutdown_connections 2024-04-25T10:12:50.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 -- 172.21.15.87:0/3188593049 wait complete. 2024-04-25T10:12:50.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 Processor -- start 2024-04-25T10:12:50.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.975+0000 7f76a1800640 1 -- start start 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c070350 0x7f769c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c0aef80 0x7f769c14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c15bac0 0x7f769c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c1679a0 con 0x7f769c070350 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c167820 con 0x7f769c15bac0 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c167b20 con 0x7f769c0aef80 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c070350 0x7f769c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c070350 0x7f769c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40762/0 (socket says 172.21.15.87:40762) 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 -- 172.21.15.87:0/4019385894 learned_addr learned my addr 172.21.15.87:0/4019385894 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769b400640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c15bac0 0x7f769c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769a000640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c0aef80 0x7f769c14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 -- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c0aef80 msgr2=0x7f769c14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c0aef80 0x7f769c14c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 -- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c15bac0 msgr2=0x7f769c151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c15bac0 0x7f769c151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 -- 172.21.15.87:0/4019385894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c14cd70 con 0x7f769c070350 2024-04-25T10:12:50.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f769aa00640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c070350 0x7f769c152e70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f768c01a660 tx=0x7f768c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f7692a00640 1 -- 172.21.15.87:0/4019385894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0a58e0 con 0x7f769c070350 2024-04-25T10:12:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f7692a00640 1 -- 172.21.15.87:0/4019385894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c007c50 con 0x7f769c070350 2024-04-25T10:12:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f7692a00640 1 -- 172.21.15.87:0/4019385894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c009510 con 0x7f769c070350 2024-04-25T10:12:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c151dd0 con 0x7f769c070350 2024-04-25T10:12:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c152390 con 0x7f769c070350 2024-04-25T10:12:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c04ef30 con 0x7f769c070350 2024-04-25T10:12:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.979+0000 7f7692a00640 1 -- 172.21.15.87:0/4019385894 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f768c0c0020 con 0x7f769c070350 2024-04-25T10:12:50.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.983+0000 7f7692a00640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f767c066f60 0x7f767c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.983+0000 7f7692a00640 1 -- 172.21.15.87:0/4019385894 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f768c11fc60 con 0x7f769c070350 2024-04-25T10:12:50.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.983+0000 7f769a000640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f767c066f60 0x7f767c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.983+0000 7f769a000640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f767c066f60 0x7f767c069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f769c150c10 tx=0x7f7688009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:50.987+0000 7f7692a00640 1 -- 172.21.15.87:0/4019385894 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f768c0c7020 con 0x7f769c070350 2024-04-25T10:12:51.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.139+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f769c0b3620 con 0x7f769c070350 2024-04-25T10:12:51.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.139+0000 7f7692a00640 1 -- 172.21.15.87:0/4019385894 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f769c14cd70 con 0x7f769c070350 2024-04-25T10:12:51.147 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f767c066f60 msgr2=0x7f767c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f767c066f60 0x7f767c069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f769c150c10 tx=0x7f7688009190 comp rx=0 tx=0).stop 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c070350 msgr2=0x7f769c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c070350 0x7f769c152e70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f768c01a660 tx=0x7f768c0197d0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 shutdown_connections 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f767c066f60 0x7f767c069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c15bac0 0x7f769c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c0aef80 0x7f769c14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 --2- 172.21.15.87:0/4019385894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f769c070350 0x7f769c152e70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 >> 172.21.15.87:0/4019385894 conn(0x7f769c0669a0 msgr2=0x7f769c157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 shutdown_connections 2024-04-25T10:12:51.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.143+0000 7f76a1800640 1 -- 172.21.15.87:0/4019385894 wait complete. 2024-04-25T10:12:51.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:51.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.299+0000 7f38eb400640 1 Processor -- start 2024-04-25T10:12:51.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.299+0000 7f38eb400640 1 -- start start 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38eb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f38e4066c00 0x7f38e4161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38eb400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f38e4066fe0 0x7f38e4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38eb400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40b40c0 con 0x7f38e40b5560 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38eb400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e4161d60 con 0x7f38e4066c00 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38eb400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e40b5b00 con 0x7f38e4066fe0 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f38e4066c00 0x7f38e4161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e9400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f38e40b5560 0x7f38e40ae850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40268/0 (socket says 172.21.15.87:40268) 2024-04-25T10:12:51.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f38e4066c00 0x7f38e4161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40774/0 (socket says 172.21.15.87:40774) 2024-04-25T10:12:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e9400640 1 -- 172.21.15.87:0/2329604460 learned_addr learned my addr 172.21.15.87:0/2329604460 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e3e00640 1 --2- 172.21.15.87:0/2329604460 >> v2:172.21.15.169:3300/0 conn(0x7f38e4066fe0 0x7f38e4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e8a00640 1 -- 172.21.15.87:0/2329604460 >> v2:172.21.15.169:3300/0 conn(0x7f38e4066fe0 msgr2=0x7f38e4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e8a00640 1 --2- 172.21.15.87:0/2329604460 >> v2:172.21.15.169:3300/0 conn(0x7f38e4066fe0 0x7f38e4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e8a00640 1 -- 172.21.15.87:0/2329604460 >> v1:172.21.15.87:6789/0 conn(0x7f38e40b5560 legacy=0x7f38e40ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.303+0000 7f38e8a00640 1 -- 172.21.15.87:0/2329604460 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40b3a00 con 0x7f38e4066c00 2024-04-25T10:12:51.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38e8a00640 1 --2- 172.21.15.87:0/2329604460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e4066c00 0x7f38e4161820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38e4071220 tx=0x7f38d401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a312f1d6553d1e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38e3400640 1 -- 172.21.15.87:0/2329604460 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40a66e0 con 0x7f38e4066c00 2024-04-25T10:12:51.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38e3400640 1 -- 172.21.15.87:0/2329604460 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d40a6880 con 0x7f38e4066c00 2024-04-25T10:12:51.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38e3400640 1 -- 172.21.15.87:0/2329604460 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40a6c50 con 0x7f38e4066c00 2024-04-25T10:12:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 -- 172.21.15.87:0/2329604460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e4066c00 msgr2=0x7f38e4161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 --2- 172.21.15.87:0/2329604460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e4066c00 0x7f38e4161820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38e4071220 tx=0x7f38d401c810 comp rx=0 tx=0).stop 2024-04-25T10:12:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 -- 172.21.15.87:0/2329604460 shutdown_connections 2024-04-25T10:12:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 --2- 172.21.15.87:0/2329604460 >> v2:172.21.15.169:3300/0 conn(0x7f38e4066fe0 0x7f38e4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 --2- 172.21.15.87:0/2329604460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e4066c00 0x7f38e4161820 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 -- 172.21.15.87:0/2329604460 >> 172.21.15.87:0/2329604460 conn(0x7f38e40b2560 msgr2=0x7f38e40b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 -- 172.21.15.87:0/2329604460 shutdown_connections 2024-04-25T10:12:51.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 -- 172.21.15.87:0/2329604460 wait complete. 2024-04-25T10:12:51.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 Processor -- start 2024-04-25T10:12:51.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.307+0000 7f38eb400640 1 -- start start 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38e4066c00 0x7f38e415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38e4066fe0 0x7f38e415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e40b5560 0x7f38e413d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e415e540 con 0x7f38e40b5560 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41585e0 con 0x7f38e4066fe0 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e415e6c0 con 0x7f38e4066c00 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e40b5560 0x7f38e413d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e40b5560 0x7f38e413d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40788/0 (socket says 172.21.15.87:40788) 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 -- 172.21.15.87:0/977379221 learned_addr learned my addr 172.21.15.87:0/977379221 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e8a00640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38e4066c00 0x7f38e415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e3e00640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38e4066fe0 0x7f38e415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 -- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38e4066c00 msgr2=0x7f38e415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38e4066c00 0x7f38e415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 -- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38e4066fe0 msgr2=0x7f38e415c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38e4066fe0 0x7f38e415c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 -- 172.21.15.87:0/977379221 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e413e820 con 0x7f38e40b5560 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e8a00640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38e4066c00 0x7f38e415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e3e00640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38e4066fe0 0x7f38e415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e9400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e40b5560 0x7f38e413d9a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c3c0 tx=0x7f38dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e0c00640 1 -- 172.21.15.87:0/977379221 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc017070 con 0x7f38e40b5560 2024-04-25T10:12:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e0c00640 1 -- 172.21.15.87:0/977379221 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc005cf0 con 0x7f38e40b5560 2024-04-25T10:12:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e0c00640 1 -- 172.21.15.87:0/977379221 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0072f0 con 0x7f38e40b5560 2024-04-25T10:12:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e413eb00 con 0x7f38e40b5560 2024-04-25T10:12:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e4152770 con 0x7f38e40b5560 2024-04-25T10:12:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e404ef30 con 0x7f38e40b5560 2024-04-25T10:12:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.311+0000 7f38e0c00640 1 -- 172.21.15.87:0/977379221 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38dc01a070 con 0x7f38e40b5560 2024-04-25T10:12:51.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.315+0000 7f38e0c00640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38c4066f90 0x7f38c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.315+0000 7f38e0c00640 1 -- 172.21.15.87:0/977379221 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38dc084350 con 0x7f38e40b5560 2024-04-25T10:12:51.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.315+0000 7f38e8a00640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38c4066f90 0x7f38c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.315+0000 7f38e8a00640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38c4066f90 0x7f38c4069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38d40af000 tx=0x7f38d40a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.319+0000 7f38e0c00640 1 -- 172.21.15.87:0/977379221 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38dc0552c0 con 0x7f38e40b5560 2024-04-25T10:12:51.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.471+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38e41699a0 con 0x7f38e40b5560 2024-04-25T10:12:51.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.471+0000 7f38e0c00640 1 -- 172.21.15.87:0/977379221 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38dc054bc0 con 0x7f38e40b5560 2024-04-25T10:12:51.479 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38c4066f90 msgr2=0x7f38c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38c4066f90 0x7f38c4069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38d40af000 tx=0x7f38d40a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e40b5560 msgr2=0x7f38e413d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e40b5560 0x7f38e413d9a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c3c0 tx=0x7f38dc00c880 comp rx=0 tx=0).stop 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 shutdown_connections 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38c4066f90 0x7f38c4069450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38e40b5560 0x7f38e413d9a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38e4066fe0 0x7f38e415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 --2- 172.21.15.87:0/977379221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38e4066c00 0x7f38e415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 >> 172.21.15.87:0/977379221 conn(0x7f38e40b2560 msgr2=0x7f38e406df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 shutdown_connections 2024-04-25T10:12:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.475+0000 7f38eb400640 1 -- 172.21.15.87:0/977379221 wait complete. 2024-04-25T10:12:51.499 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:51.505 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:51.546 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.631+0000 7fb96a600640 1 Processor -- start 2024-04-25T10:12:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.631+0000 7fb96a600640 1 -- start start 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.631+0000 7fb96a600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb964070350 0x7fb96415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb96a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb9640b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb96a600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96415b080 con 0x7fb9640aef80 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb96a600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96415adb0 con 0x7fb964070350 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb96a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96415eb60 con 0x7fb9640b31c0 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb95aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb964070350 0x7fb96415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb963400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb9640aef80 0x7fb96415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40280/0 (socket says 172.21.15.87:40280) 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb95aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb964070350 0x7fb96415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40798/0 (socket says 172.21.15.87:40798) 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb963400640 1 -- 172.21.15.87:0/1822451879 learned_addr learned my addr 172.21.15.87:0/1822451879 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb963e00640 1 --2- 172.21.15.87:0/1822451879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb9640b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb95aa00640 1 -- 172.21.15.87:0/1822451879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 msgr2=0x7fb9640b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb95aa00640 1 --2- 172.21.15.87:0/1822451879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb9640b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb95aa00640 1 -- 172.21.15.87:0/1822451879 >> v1:172.21.15.87:6789/0 conn(0x7fb9640aef80 legacy=0x7fb96415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb962a00640 1 -- 172.21.15.87:0/1822451879 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749376213 0 0) 0x7fb96415b080 con 0x7fb9640aef80 2024-04-25T10:12:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb95aa00640 1 -- 172.21.15.87:0/1822451879 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96415ee40 con 0x7fb964070350 2024-04-25T10:12:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.635+0000 7fb95aa00640 1 --2- 172.21.15.87:0/1822451879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964070350 0x7fb96415e480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097c0 tx=0x7fb94c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f72c17b702ed42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb962a00640 1 -- 172.21.15.87:0/1822451879 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b1070 con 0x7fb964070350 2024-04-25T10:12:51.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb962a00640 1 -- 172.21.15.87:0/1822451879 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0ac410 con 0x7fb964070350 2024-04-25T10:12:51.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb962a00640 1 -- 172.21.15.87:0/1822451879 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0ac7e0 con 0x7fb964070350 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- 172.21.15.87:0/1822451879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964070350 msgr2=0x7fb96415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 --2- 172.21.15.87:0/1822451879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964070350 0x7fb96415e480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097c0 tx=0x7fb94c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- 172.21.15.87:0/1822451879 shutdown_connections 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 --2- 172.21.15.87:0/1822451879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb9640b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 --2- 172.21.15.87:0/1822451879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964070350 0x7fb96415e480 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- 172.21.15.87:0/1822451879 >> 172.21.15.87:0/1822451879 conn(0x7fb9640669a0 msgr2=0x7fb964066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- 172.21.15.87:0/1822451879 shutdown_connections 2024-04-25T10:12:51.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- 172.21.15.87:0/1822451879 wait complete. 2024-04-25T10:12:51.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 Processor -- start 2024-04-25T10:12:51.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- start start 2024-04-25T10:12:51.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb964070350 0x7fb964151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9640aef80 0x7fb964151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb964152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641679a0 con 0x7fb9640aef80 2024-04-25T10:12:51.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964167820 con 0x7fb964070350 2024-04-25T10:12:51.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb96a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb964167b20 con 0x7fb9640b31c0 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb95aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9640aef80 0x7fb964151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb95aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9640aef80 0x7fb964151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40804/0 (socket says 172.21.15.87:40804) 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb95aa00640 1 -- 172.21.15.87:0/3662593631 learned_addr learned my addr 172.21.15.87:0/3662593631 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.639+0000 7fb963e00640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb964152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb963400640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb964070350 0x7fb964151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95aa00640 1 -- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 msgr2=0x7fb964152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95aa00640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb964152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95aa00640 1 -- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb964070350 msgr2=0x7fb964151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95aa00640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb964070350 0x7fb964151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95aa00640 1 -- 172.21.15.87:0/3662593631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9640ae4c0 con 0x7fb9640aef80 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb963400640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb964070350 0x7fb964151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95aa00640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9640aef80 0x7fb964151fd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097c0 tx=0x7fb94c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95be00640 1 -- 172.21.15.87:0/3662593631 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0bc070 con 0x7fb9640aef80 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95be00640 1 -- 172.21.15.87:0/3662593631 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0acd70 con 0x7fb9640aef80 2024-04-25T10:12:51.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb964157ec0 con 0x7fb9640aef80 2024-04-25T10:12:51.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95be00640 1 -- 172.21.15.87:0/3662593631 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b1040 con 0x7fb9640aef80 2024-04-25T10:12:51.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96414fab0 con 0x7fb9640aef80 2024-04-25T10:12:51.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96404ef30 con 0x7fb9640aef80 2024-04-25T10:12:51.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95be00640 1 -- 172.21.15.87:0/3662593631 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb94c0bf070 con 0x7fb9640aef80 2024-04-25T10:12:51.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95be00640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb940066f90 0x7fb940069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb95be00640 1 -- 172.21.15.87:0/3662593631 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb94c11e090 con 0x7fb9640aef80 2024-04-25T10:12:51.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.643+0000 7fb963400640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb940066f90 0x7fb940069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.647+0000 7fb963400640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb940066f90 0x7fb940069450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb954000f70 tx=0x7fb954009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.647+0000 7fb95be00640 1 -- 172.21.15.87:0/3662593631 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb94c016090 con 0x7fb9640aef80 2024-04-25T10:12:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9640b2fa0 con 0x7fb9640aef80 2024-04-25T10:12:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb95be00640 1 -- 172.21.15.87:0/3662593631 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb94c0ef000 con 0x7fb9640aef80 2024-04-25T10:12:51.809 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb940066f90 msgr2=0x7fb940069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb940066f90 0x7fb940069450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb954000f70 tx=0x7fb954009040 comp rx=0 tx=0).stop 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9640aef80 msgr2=0x7fb964151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9640aef80 0x7fb964151fd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097c0 tx=0x7fb94c019240 comp rx=0 tx=0).stop 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 shutdown_connections 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb940066f90 0x7fb940069450 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9640b31c0 0x7fb964152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9640aef80 0x7fb964151fd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 --2- 172.21.15.87:0/3662593631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb964070350 0x7fb964151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.803+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 >> 172.21.15.87:0/3662593631 conn(0x7fb9640669a0 msgr2=0x7fb9640ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.807+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 shutdown_connections 2024-04-25T10:12:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.807+0000 7fb96a600640 1 -- 172.21.15.87:0/3662593631 wait complete. 2024-04-25T10:12:51.841 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T10:12:51.841 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:12:51.841 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:51.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00dc800640 1 Processor -- start 2024-04-25T10:12:51.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00dc800640 1 -- start start 2024-04-25T10:12:51.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00dc800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00d40cda10 0x7f00d406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00dc800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d40b2770 con 0x7f00d415bc40 2024-04-25T10:12:51.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00dc800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d406c670 con 0x7f00d40cf930 2024-04-25T10:12:51.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00dc800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d406c950 con 0x7f00d40cda10 2024-04-25T10:12:51.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00d9e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f00d415bc40 0x7f00d40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40284/0 (socket says 172.21.15.87:40284) 2024-04-25T10:12:51.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00d9e00640 1 -- 172.21.15.87:0/663729871 learned_addr learned my addr 172.21.15.87:0/663729871 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00d9400640 1 --2- 172.21.15.87:0/663729871 >> v2:172.21.15.169:3300/0 conn(0x7f00d40cda10 0x7f00d406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00d8a00640 1 -- 172.21.15.87:0/663729871 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783052263 0 0) 0x7f00d40b2770 con 0x7f00d415bc40 2024-04-25T10:12:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.971+0000 7f00d8a00640 1 -- 172.21.15.87:0/663729871 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc003530 con 0x7f00d415bc40 2024-04-25T10:12:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d8a00640 1 -- 172.21.15.87:0/663729871 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739912022 0 0) 0x7f00d406c670 con 0x7f00d40cf930 2024-04-25T10:12:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d8a00640 1 -- 172.21.15.87:0/663729871 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d40b2770 con 0x7f00d40cf930 2024-04-25T10:12:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d9400640 1 -- 172.21.15.87:0/663729871 >> v1:172.21.15.169:6789/0 conn(0x7f00d40cf930 legacy=0x7f00d40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d9400640 1 -- 172.21.15.87:0/663729871 >> v1:172.21.15.87:6789/0 conn(0x7f00d415bc40 legacy=0x7f00d40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d9400640 1 -- 172.21.15.87:0/663729871 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d406cc30 con 0x7f00d40cda10 2024-04-25T10:12:51.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d9400640 1 --2- 172.21.15.87:0/663729871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d40cda10 0x7f00d406c090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00c40097c0 tx=0x7f00c401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b3a3a2baa9bbf7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d8a00640 1 -- 172.21.15.87:0/663729871 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40b0070 con 0x7f00d40cda10 2024-04-25T10:12:51.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d8a00640 1 -- 172.21.15.87:0/663729871 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c40ab600 con 0x7f00d40cda10 2024-04-25T10:12:51.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00d8a00640 1 -- 172.21.15.87:0/663729871 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40ab9d0 con 0x7f00d40cda10 2024-04-25T10:12:51.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00dc800640 1 -- 172.21.15.87:0/663729871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d40cda10 msgr2=0x7f00d406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00dc800640 1 --2- 172.21.15.87:0/663729871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d40cda10 0x7f00d406c090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00c40097c0 tx=0x7f00c401c410 comp rx=0 tx=0).stop 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00dc800640 1 -- 172.21.15.87:0/663729871 shutdown_connections 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00dc800640 1 --2- 172.21.15.87:0/663729871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d40cda10 0x7f00d406c090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00dc800640 1 -- 172.21.15.87:0/663729871 >> 172.21.15.87:0/663729871 conn(0x7f00d4066b70 msgr2=0x7f00d4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00dc800640 1 -- 172.21.15.87:0/663729871 shutdown_connections 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.975+0000 7f00dc800640 1 -- 172.21.15.87:0/663729871 wait complete. 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 Processor -- start 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 -- start start 2024-04-25T10:12:51.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00d40cda10 0x7f00d40af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00d40cf930 0x7f00d40afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d415bc40 0x7f00d40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4169d70 con 0x7f00d40cf930 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4169bf0 con 0x7f00d415bc40 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4169ef0 con 0x7f00d40cda10 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00d40cf930 0x7f00d40afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00d40cf930 0x7f00d40afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40810/0 (socket says 172.21.15.87:40810) 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 -- 172.21.15.87:0/2586759276 learned_addr learned my addr 172.21.15.87:0/2586759276 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00da800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d415bc40 0x7f00d40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9e00640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00d40cda10 0x7f00d40af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 -- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00d40cda10 msgr2=0x7f00d40af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00d40cda10 0x7f00d40af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 -- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d415bc40 msgr2=0x7f00d40adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d415bc40 0x7f00d40adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 -- 172.21.15.87:0/2586759276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d414f340 con 0x7f00d40cf930 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00da800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d415bc40 0x7f00d40adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9e00640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00d40cda10 0x7f00d40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00d9400640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00d40cf930 0x7f00d40afdd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00c4019020 tx=0x7f00c40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00ce000640 1 -- 172.21.15.87:0/2586759276 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40b0040 con 0x7f00d40cf930 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00ce000640 1 -- 172.21.15.87:0/2586759276 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c40b9070 con 0x7f00d40cf930 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00ce000640 1 -- 172.21.15.87:0/2586759276 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40b4650 con 0x7f00d40cf930 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.979+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d414f620 con 0x7f00d40cf930 2024-04-25T10:12:51.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.983+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d414fbe0 con 0x7f00d40cf930 2024-04-25T10:12:51.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.983+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d404ef30 con 0x7f00d40cf930 2024-04-25T10:12:51.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.983+0000 7f00ce000640 1 -- 172.21.15.87:0/2586759276 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00c40b47f0 con 0x7f00d40cf930 2024-04-25T10:12:51.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.983+0000 7f00ce000640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00b8066f60 0x7f00b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.983+0000 7f00ce000640 1 -- 172.21.15.87:0/2586759276 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00c411ed10 con 0x7f00d40cf930 2024-04-25T10:12:51.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.983+0000 7f00d9e00640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00b8066f60 0x7f00b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.983+0000 7f00d9e00640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00b8066f60 0x7f00b8069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00c8000f70 tx=0x7f00c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:51.987+0000 7f00ce000640 1 -- 172.21.15.87:0/2586759276 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00c4124020 con 0x7f00d40cf930 2024-04-25T10:12:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.143+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00d406d560 con 0x7f00d40cf930 2024-04-25T10:12:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.143+0000 7f00ce000640 1 -- 172.21.15.87:0/2586759276 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f00c4016090 con 0x7f00d40cf930 2024-04-25T10:12:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:52.150 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:52.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00b8066f60 msgr2=0x7f00b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00b8066f60 0x7f00b8069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00c8000f70 tx=0x7f00c8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:52.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00d40cf930 msgr2=0x7f00d40afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00d40cf930 0x7f00d40afdd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00c4019020 tx=0x7f00c40191b0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 shutdown_connections 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00b8066f60 0x7f00b8069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d415bc40 0x7f00d40adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00d40cf930 0x7f00d40afdd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 --2- 172.21.15.87:0/2586759276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00d40cda10 0x7f00d40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 >> 172.21.15.87:0/2586759276 conn(0x7f00d4066b70 msgr2=0x7f00d415af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 shutdown_connections 2024-04-25T10:12:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.147+0000 7f00dc800640 1 -- 172.21.15.87:0/2586759276 wait complete. 2024-04-25T10:12:52.172 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:52.172 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.303+0000 7f8a32800640 1 Processor -- start 2024-04-25T10:12:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.303+0000 7f8a32800640 1 -- start start 2024-04-25T10:12:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a32800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8a2c0aef80 0x7f8a2c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a32800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c15b080 con 0x7f8a2c070350 2024-04-25T10:12:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a32800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c15adb0 con 0x7f8a2c0b31c0 2024-04-25T10:12:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a32800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2c15eb60 con 0x7f8a2c0aef80 2024-04-25T10:12:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a2b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8a2c0aef80 0x7f8a2c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a2b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8a2c0aef80 0x7f8a2c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40824/0 (socket says 172.21.15.87:40824) 2024-04-25T10:12:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a2b400640 1 -- 172.21.15.87:0/3377227714 learned_addr learned my addr 172.21.15.87:0/3377227714 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a2b400640 1 -- 172.21.15.87:0/3377227714 >> v1:172.21.15.169:6789/0 conn(0x7f8a2c0b31c0 legacy=0x7f8a2c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a2b400640 1 -- 172.21.15.87:0/3377227714 >> v1:172.21.15.87:6789/0 conn(0x7f8a2c070350 legacy=0x7f8a2c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a2a000640 1 -- 172.21.15.87:0/3377227714 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420330691 0 0) 0x7f8a2c15b080 con 0x7f8a2c070350 2024-04-25T10:12:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.307+0000 7f8a2b400640 1 -- 172.21.15.87:0/3377227714 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c15ee40 con 0x7f8a2c0aef80 2024-04-25T10:12:52.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a2b400640 1 --2- 172.21.15.87:0/3377227714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c15e480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009e50 tx=0x7f8a1c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72b052d761890ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a2a000640 1 -- 172.21.15.87:0/3377227714 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a58a0 con 0x7f8a2c0aef80 2024-04-25T10:12:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a2a000640 1 -- 172.21.15.87:0/3377227714 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0a5a40 con 0x7f8a2c0aef80 2024-04-25T10:12:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a2a000640 1 -- 172.21.15.87:0/3377227714 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a5e10 con 0x7f8a2c0aef80 2024-04-25T10:12:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- 172.21.15.87:0/3377227714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 msgr2=0x7f8a2c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 --2- 172.21.15.87:0/3377227714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c15e480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009e50 tx=0x7f8a1c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- 172.21.15.87:0/3377227714 shutdown_connections 2024-04-25T10:12:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 --2- 172.21.15.87:0/3377227714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c15e480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- 172.21.15.87:0/3377227714 >> 172.21.15.87:0/3377227714 conn(0x7f8a2c0669a0 msgr2=0x7f8a2c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- 172.21.15.87:0/3377227714 shutdown_connections 2024-04-25T10:12:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- 172.21.15.87:0/3377227714 wait complete. 2024-04-25T10:12:52.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 Processor -- start 2024-04-25T10:12:52.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- start start 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2c070350 0x7f8a2c165f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a2c0b31c0 0x7f8a2c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c0cfef0 con 0x7f8a2c0aef80 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c0cfd70 con 0x7f8a2c0b31c0 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.311+0000 7f8a32800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c0b3610 con 0x7f8a2c070350 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2c070350 0x7f8a2c165f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c06c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40834/0 (socket says 172.21.15.87:40834) 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 -- 172.21.15.87:0/639304559 learned_addr learned my addr 172.21.15.87:0/639304559 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2be00640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a2c0b31c0 0x7f8a2c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 -- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2c070350 msgr2=0x7f8a2c165f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2c070350 0x7f8a2c165f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 -- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a2c0b31c0 msgr2=0x7f8a2c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a2c0b31c0 0x7f8a2c06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 -- 172.21.15.87:0/639304559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c0afbe0 con 0x7f8a2c0aef80 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2be00640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a2c0b31c0 0x7f8a2c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a2aa00640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c06c0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800c4d0 tx=0x7f8a1800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a23400640 1 -- 172.21.15.87:0/639304559 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18017070 con 0x7f8a2c0aef80 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a23400640 1 -- 172.21.15.87:0/639304559 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a18005480 con 0x7f8a2c0aef80 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c0b2450 con 0x7f8a2c0aef80 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a23400640 1 -- 172.21.15.87:0/639304559 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18003840 con 0x7f8a2c0aef80 2024-04-25T10:12:52.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c0b1690 con 0x7f8a2c0aef80 2024-04-25T10:12:52.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c04ef30 con 0x7f8a2c0aef80 2024-04-25T10:12:52.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a23400640 1 -- 172.21.15.87:0/639304559 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a1801a070 con 0x7f8a2c0aef80 2024-04-25T10:12:52.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.315+0000 7f8a23400640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a0c066f60 0x7f8a0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.319+0000 7f8a23400640 1 -- 172.21.15.87:0/639304559 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a180843d0 con 0x7f8a2c0aef80 2024-04-25T10:12:52.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.319+0000 7f8a2b400640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a0c066f60 0x7f8a0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.319+0000 7f8a2b400640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a0c066f60 0x7f8a0c069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ae000 tx=0x7f8a1c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.319+0000 7f8a23400640 1 -- 172.21.15.87:0/639304559 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a2c0afbe0 con 0x7f8a2c0aef80 2024-04-25T10:12:52.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.475+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a2c0b2fa0 con 0x7f8a2c0aef80 2024-04-25T10:12:52.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.475+0000 7f8a23400640 1 -- 172.21.15.87:0/639304559 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a18054db0 con 0x7f8a2c0aef80 2024-04-25T10:12:52.483 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a0c066f60 msgr2=0x7f8a0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a0c066f60 0x7f8a0c069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ae000 tx=0x7f8a1c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 msgr2=0x7f8a2c06c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c06c0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800c4d0 tx=0x7f8a1800c990 comp rx=0 tx=0).stop 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 shutdown_connections 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a0c066f60 0x7f8a0c069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a2c0b31c0 0x7f8a2c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c0aef80 0x7f8a2c06c0d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 --2- 172.21.15.87:0/639304559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2c070350 0x7f8a2c165f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 >> 172.21.15.87:0/639304559 conn(0x7f8a2c0669a0 msgr2=0x7f8a2c0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 shutdown_connections 2024-04-25T10:12:52.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.479+0000 7f8a32800640 1 -- 172.21.15.87:0/639304559 wait complete. 2024-04-25T10:12:52.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:52.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a45000640 1 Processor -- start 2024-04-25T10:12:52.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a45000640 1 -- start start 2024-04-25T10:12:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a45000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 0x7f1a400d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a45000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a400b1770 con 0x7f1a4015bc10 2024-04-25T10:12:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a45000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4015fec0 con 0x7f1a400cf980 2024-04-25T10:12:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a45000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401601a0 con 0x7f1a400d1830 2024-04-25T10:12:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3de00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1a4015bc10 0x7f1a4015f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40296/0 (socket says 172.21.15.87:40296) 2024-04-25T10:12:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3de00640 1 -- 172.21.15.87:0/2221503060 learned_addr learned my addr 172.21.15.87:0/2221503060 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:52.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185852369 0 0) 0x7f1a400b1770 con 0x7f1a4015bc10 2024-04-25T10:12:52.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3e800640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 0x7f1a400d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a24003530 con 0x7f1a4015bc10 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015366217 0 0) 0x7f1a24003530 con 0x7f1a4015bc10 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 msgr2=0x7f1a400d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 0x7f1a400d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 >> v1:172.21.15.169:6789/0 conn(0x7f1a400cf980 legacy=0x7f1a400b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400cc5b0 con 0x7f1a4015bc10 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f1a30001a00 con 0x7f1a4015bc10 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 >> v1:172.21.15.87:6789/0 conn(0x7f1a4015bc10 legacy=0x7f1a4015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a240039f0 0x7f1a24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 0x7f1a24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d50 0x7f1a240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401601a0 con 0x7f1a240039f0 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2400bab0 con 0x7f1a24004d50 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.635+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2400bcb0 con 0x7f1a400d1830 2024-04-25T10:12:52.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3de00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a240039f0 0x7f1a24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3e800640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d50 0x7f1a240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3d400640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 0x7f1a24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3de00640 1 -- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 msgr2=0x7f1a24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3de00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 0x7f1a24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3de00640 1 -- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d50 msgr2=0x7f1a240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3de00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d50 0x7f1a240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3de00640 1 -- 172.21.15.87:0/2221503060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24003530 con 0x7f1a240039f0 2024-04-25T10:12:52.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3d400640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400d1830 0x7f1a24004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:52.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3de00640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a240039f0 0x7f1a24003e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000a760 tx=0x7f1a3001d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f45f94e58d383e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000a3b0 con 0x7f1a240039f0 2024-04-25T10:12:52.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a300a9070 con 0x7f1a240039f0 2024-04-25T10:12:52.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.639+0000 7f1a3ca00640 1 -- 172.21.15.87:0/2221503060 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a300ae470 con 0x7f1a240039f0 2024-04-25T10:12:52.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- 172.21.15.87:0/2221503060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a240039f0 msgr2=0x7f1a24003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 --2- 172.21.15.87:0/2221503060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a240039f0 0x7f1a24003e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000a760 tx=0x7f1a3001d280 comp rx=0 tx=0).stop 2024-04-25T10:12:52.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3e800640 1 -- 172.21.15.87:0/2221503060 reap_dead start 2024-04-25T10:12:52.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- 172.21.15.87:0/2221503060 shutdown_connections 2024-04-25T10:12:52.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- 172.21.15.87:0/2221503060 >> 172.21.15.87:0/2221503060 conn(0x7f1a4009a9d0 msgr2=0x7f1a4009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- 172.21.15.87:0/2221503060 shutdown_connections 2024-04-25T10:12:52.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- 172.21.15.87:0/2221503060 wait complete. 2024-04-25T10:12:52.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 Processor -- start 2024-04-25T10:12:52.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- start start 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039f0 0x7f1a400b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a400b7150 0x7f1a400c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a400b2170 0x7f1a400b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400cd740 con 0x7f1a400b7150 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400cd5c0 con 0x7f1a400b2170 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a45000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40165950 con 0x7f1a240039f0 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a400b7150 0x7f1a400c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a400b7150 0x7f1a400c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40852/0 (socket says 172.21.15.87:40852) 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 -- 172.21.15.87:0/2403710613 learned_addr learned my addr 172.21.15.87:0/2403710613 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3de00640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039f0 0x7f1a400b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3e800640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a400b2170 0x7f1a400b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 -- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039f0 msgr2=0x7f1a400b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039f0 0x7f1a400b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 -- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a400b2170 msgr2=0x7f1a400b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a400b2170 0x7f1a400b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 -- 172.21.15.87:0/2403710613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400b4dd0 con 0x7f1a400b7150 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.643+0000 7f1a3d400640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a400b7150 0x7f1a400c7db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e7e0 tx=0x7f1a2c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a36000640 1 -- 172.21.15.87:0/2403710613 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0050d0 con 0x7f1a400b7150 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a36000640 1 -- 172.21.15.87:0/2403710613 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c018070 con 0x7f1a400b7150 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a36000640 1 -- 172.21.15.87:0/2403710613 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00c6c0 con 0x7f1a400b7150 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a400c8cc0 con 0x7f1a400b7150 2024-04-25T10:12:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a400c9280 con 0x7f1a400b7150 2024-04-25T10:12:52.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4004ef30 con 0x7f1a400b7150 2024-04-25T10:12:52.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a36000640 1 -- 172.21.15.87:0/2403710613 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a2c00c860 con 0x7f1a400b7150 2024-04-25T10:12:52.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a36000640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a20066f90 0x7f1a20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a36000640 1 -- 172.21.15.87:0/2403710613 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a2c084d50 con 0x7f1a400b7150 2024-04-25T10:12:52.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a3de00640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a20066f90 0x7f1a20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.647+0000 7f1a3de00640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a20066f90 0x7f1a20069450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a300ad000 tx=0x7f1a300a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.651+0000 7f1a36000640 1 -- 172.21.15.87:0/2403710613 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a2c055e10 con 0x7f1a400b7150 2024-04-25T10:12:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.803+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a400b13f0 con 0x7f1a400b7150 2024-04-25T10:12:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a36000640 1 -- 172.21.15.87:0/2403710613 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a2c055710 con 0x7f1a400b7150 2024-04-25T10:12:52.812 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a20066f90 msgr2=0x7f1a20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a20066f90 0x7f1a20069450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a300ad000 tx=0x7f1a300a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a400b7150 msgr2=0x7f1a400c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a400b7150 0x7f1a400c7db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e7e0 tx=0x7f1a2c00eca0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 shutdown_connections 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a20066f90 0x7f1a20069450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a400b2170 0x7f1a400b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a400b7150 0x7f1a400c7db0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 --2- 172.21.15.87:0/2403710613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039f0 0x7f1a400b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 >> 172.21.15.87:0/2403710613 conn(0x7f1a4009a9d0 msgr2=0x7f1a400b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 shutdown_connections 2024-04-25T10:12:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.807+0000 7f1a45000640 1 -- 172.21.15.87:0/2403710613 wait complete. 2024-04-25T10:12:52.832 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:52.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.963+0000 7fa575400640 1 Processor -- start 2024-04-25T10:12:52.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.963+0000 7fa575400640 1 -- start start 2024-04-25T10:12:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa575400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa5700cc200 0x7fa5700ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa575400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa57006cf40 0x7fa57006d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa575400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5700cdab0 con 0x7fa5700d0d00 2024-04-25T10:12:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa575400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57006d8b0 con 0x7fa5700cc200 2024-04-25T10:12:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa575400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5700cd210 con 0x7fa57006cf40 2024-04-25T10:12:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa565e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa5700cc200 0x7fa5700ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa565e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa5700cc200 0x7fa5700ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40854/0 (socket says 172.21.15.87:40854) 2024-04-25T10:12:52.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa565e00640 1 -- 172.21.15.87:0/2773295497 learned_addr learned my addr 172.21.15.87:0/2773295497 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:52.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa56f200640 1 --2- 172.21.15.87:0/2773295497 >> v2:172.21.15.169:3300/0 conn(0x7fa57006cf40 0x7fa57006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa565e00640 1 -- 172.21.15.87:0/2773295497 >> v2:172.21.15.169:3300/0 conn(0x7fa57006cf40 msgr2=0x7fa57006d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa565e00640 1 --2- 172.21.15.87:0/2773295497 >> v2:172.21.15.169:3300/0 conn(0x7fa57006cf40 0x7fa57006d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa565e00640 1 -- 172.21.15.87:0/2773295497 >> v1:172.21.15.169:6789/0 conn(0x7fa5700d0d00 legacy=0x7fa570162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.967+0000 7fa565e00640 1 -- 172.21.15.87:0/2773295497 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5700cd4f0 con 0x7fa5700cc200 2024-04-25T10:12:52.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa565e00640 1 --2- 172.21.15.87:0/2773295497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700cc200 0x7fa5700ccc30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa57006f8f0 tx=0x7fa55801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957f1916409628e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa56de00640 1 -- 172.21.15.87:0/2773295497 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580b2070 con 0x7fa5700cc200 2024-04-25T10:12:52.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa56de00640 1 -- 172.21.15.87:0/2773295497 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5580ad680 con 0x7fa5700cc200 2024-04-25T10:12:52.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa56de00640 1 -- 172.21.15.87:0/2773295497 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580ada50 con 0x7fa5700cc200 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 -- 172.21.15.87:0/2773295497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700cc200 msgr2=0x7fa5700ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 --2- 172.21.15.87:0/2773295497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700cc200 0x7fa5700ccc30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa57006f8f0 tx=0x7fa55801c490 comp rx=0 tx=0).stop 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 -- 172.21.15.87:0/2773295497 shutdown_connections 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 --2- 172.21.15.87:0/2773295497 >> v2:172.21.15.169:3300/0 conn(0x7fa57006cf40 0x7fa57006d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 --2- 172.21.15.87:0/2773295497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700cc200 0x7fa5700ccc30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 -- 172.21.15.87:0/2773295497 >> 172.21.15.87:0/2773295497 conn(0x7fa570072290 msgr2=0x7fa5700726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 -- 172.21.15.87:0/2773295497 shutdown_connections 2024-04-25T10:12:52.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 -- 172.21.15.87:0/2773295497 wait complete. 2024-04-25T10:12:52.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 Processor -- start 2024-04-25T10:12:52.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.971+0000 7fa575400640 1 -- start start 2024-04-25T10:12:52.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57006cf40 0x7fa5700997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5700cc200 0x7fa5700c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700d0d00 0x7fa5700c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa570162a10 con 0x7fa5700d0d00 2024-04-25T10:12:52.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa570162890 con 0x7fa57006cf40 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57015e040 con 0x7fa5700cc200 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700d0d00 0x7fa5700c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700d0d00 0x7fa5700c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40860/0 (socket says 172.21.15.87:40860) 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 -- 172.21.15.87:0/2092175848 learned_addr learned my addr 172.21.15.87:0/2092175848 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa565e00640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5700cc200 0x7fa5700c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56e800640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57006cf40 0x7fa5700997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 -- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5700cc200 msgr2=0x7fa5700c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5700cc200 0x7fa5700c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 -- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57006cf40 msgr2=0x7fa5700997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57006cf40 0x7fa5700997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 -- 172.21.15.87:0/2092175848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5700ccfa0 con 0x7fa5700d0d00 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56e800640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57006cf40 0x7fa5700997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:52.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa56f200640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700d0d00 0x7fa5700c72f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa568007e70 tx=0x7fa56800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa567400640 1 -- 172.21.15.87:0/2092175848 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568014620 con 0x7fa5700d0d00 2024-04-25T10:12:52.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa567400640 1 -- 172.21.15.87:0/2092175848 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa568019070 con 0x7fa5700d0d00 2024-04-25T10:12:52.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa567400640 1 -- 172.21.15.87:0/2092175848 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568014a20 con 0x7fa5700d0d00 2024-04-25T10:12:52.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57015aa10 con 0x7fa5700d0d00 2024-04-25T10:12:52.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa570099d20 con 0x7fa5700d0d00 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57004ef30 con 0x7fa5700d0d00 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa567400640 1 -- 172.21.15.87:0/2092175848 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa568014bc0 con 0x7fa5700d0d00 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.975+0000 7fa567400640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa54c066f60 0x7fa54c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.979+0000 7fa567400640 1 -- 172.21.15.87:0/2092175848 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5680859e0 con 0x7fa5700d0d00 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.979+0000 7fa56e800640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa54c066f60 0x7fa54c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.979+0000 7fa56e800640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa54c066f60 0x7fa54c069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa560001650 tx=0x7fa560009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:52.983+0000 7fa567400640 1 -- 172.21.15.87:0/2092175848 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa568011260 con 0x7fa5700d0d00 2024-04-25T10:12:53.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.135+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa570072ff0 con 0x7fa5700d0d00 2024-04-25T10:12:53.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.135+0000 7fa567400640 1 -- 172.21.15.87:0/2092175848 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa568056950 con 0x7fa5700d0d00 2024-04-25T10:12:53.143 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:53.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa54c066f60 msgr2=0x7fa54c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa54c066f60 0x7fa54c069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa560001650 tx=0x7fa560009040 comp rx=0 tx=0).stop 2024-04-25T10:12:53.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700d0d00 msgr2=0x7fa5700c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700d0d00 0x7fa5700c72f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa568007e70 tx=0x7fa56800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 shutdown_connections 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa54c066f60 0x7fa54c069420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5700d0d00 0x7fa5700c72f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5700cc200 0x7fa5700c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 --2- 172.21.15.87:0/2092175848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57006cf40 0x7fa5700997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 >> 172.21.15.87:0/2092175848 conn(0x7fa570072290 msgr2=0x7fa5700ae0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 shutdown_connections 2024-04-25T10:12:53.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.139+0000 7fa575400640 1 -- 172.21.15.87:0/2092175848 wait complete. 2024-04-25T10:12:53.165 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9419600640 1 Processor -- start 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9419600640 1 -- start start 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9419600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f941415ac70 0x7f94140ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9419600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f941406dd30 0x7f94140b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9419600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94140af4c0 con 0x7f94140b4da0 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9419600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941415b050 con 0x7f941406dd30 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9419600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94140b2d20 con 0x7f941415ac70 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f941406dd30 0x7f94140b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f941406dd30 0x7f94140b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40872/0 (socket says 172.21.15.87:40872) 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413400640 1 -- 172.21.15.87:0/4083321222 learned_addr learned my addr 172.21.15.87:0/4083321222 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:53.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413e00640 1 --2- 172.21.15.87:0/4083321222 >> v2:172.21.15.169:3300/0 conn(0x7f941415ac70 0x7f94140ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413400640 1 -- 172.21.15.87:0/4083321222 >> v2:172.21.15.169:3300/0 conn(0x7f941415ac70 msgr2=0x7f94140ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413400640 1 --2- 172.21.15.87:0/4083321222 >> v2:172.21.15.169:3300/0 conn(0x7f941415ac70 0x7f94140ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413400640 1 -- 172.21.15.87:0/4083321222 >> v1:172.21.15.169:6789/0 conn(0x7f94140b4da0 legacy=0x7f941406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413400640 1 -- 172.21.15.87:0/4083321222 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94140b2f70 con 0x7f941406dd30 2024-04-25T10:12:53.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.295+0000 7f9413e00640 1 --2- 172.21.15.87:0/4083321222 >> v2:172.21.15.169:3300/0 conn(0x7f941415ac70 0x7f94140ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9413400640 1 --2- 172.21.15.87:0/4083321222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941406dd30 0x7f94140b4860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f941406c2e0 tx=0x7f940001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36ddd9f962d73d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9412a00640 1 -- 172.21.15.87:0/4083321222 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94000b2070 con 0x7f941406dd30 2024-04-25T10:12:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9412a00640 1 -- 172.21.15.87:0/4083321222 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94000ad680 con 0x7f941406dd30 2024-04-25T10:12:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9412a00640 1 -- 172.21.15.87:0/4083321222 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94000ada50 con 0x7f941406dd30 2024-04-25T10:12:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 -- 172.21.15.87:0/4083321222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941406dd30 msgr2=0x7f94140b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 --2- 172.21.15.87:0/4083321222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941406dd30 0x7f94140b4860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f941406c2e0 tx=0x7f940001c460 comp rx=0 tx=0).stop 2024-04-25T10:12:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 -- 172.21.15.87:0/4083321222 shutdown_connections 2024-04-25T10:12:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 --2- 172.21.15.87:0/4083321222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941406dd30 0x7f94140b4860 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 --2- 172.21.15.87:0/4083321222 >> v2:172.21.15.169:3300/0 conn(0x7f941415ac70 0x7f94140ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 -- 172.21.15.87:0/4083321222 >> 172.21.15.87:0/4083321222 conn(0x7f94140b1a20 msgr2=0x7f94140b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 -- 172.21.15.87:0/4083321222 shutdown_connections 2024-04-25T10:12:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 -- 172.21.15.87:0/4083321222 wait complete. 2024-04-25T10:12:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 Processor -- start 2024-04-25T10:12:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.299+0000 7f9419600640 1 -- start start 2024-04-25T10:12:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f941406dd30 0x7f94141536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140b4da0 0x7f94141f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941415ac70 0x7f94141592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941406e8b0 con 0x7f941415ac70 2024-04-25T10:12:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941406e730 con 0x7f94140b4da0 2024-04-25T10:12:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94140cec80 con 0x7f941406dd30 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941415ac70 0x7f94141592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9413e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f941406dd30 0x7f94141536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9413400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140b4da0 0x7f94141f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941415ac70 0x7f94141592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40874/0 (socket says 172.21.15.87:40874) 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 -- 172.21.15.87:0/2761426770 learned_addr learned my addr 172.21.15.87:0/2761426770 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 -- 172.21.15.87:0/2761426770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f941406dd30 msgr2=0x7f94141536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f941406dd30 0x7f94141536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 -- 172.21.15.87:0/2761426770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140b4da0 msgr2=0x7f94141f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140b4da0 0x7f94141f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 -- 172.21.15.87:0/2761426770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94141f6320 con 0x7f941415ac70 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9418c00640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941415ac70 0x7f94141592c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c7a0 tx=0x7f940c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f940be00640 1 -- 172.21.15.87:0/2761426770 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c003b80 con 0x7f941415ac70 2024-04-25T10:12:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f940be00640 1 -- 172.21.15.87:0/2761426770 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c003d20 con 0x7f941415ac70 2024-04-25T10:12:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94141f5870 con 0x7f941415ac70 2024-04-25T10:12:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f940be00640 1 -- 172.21.15.87:0/2761426770 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c00aa30 con 0x7f941415ac70 2024-04-25T10:12:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941415c4c0 con 0x7f941415ac70 2024-04-25T10:12:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.303+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9414075120 con 0x7f941415ac70 2024-04-25T10:12:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.307+0000 7f940be00640 1 -- 172.21.15.87:0/2761426770 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f940c00abd0 con 0x7f941415ac70 2024-04-25T10:12:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.307+0000 7f940be00640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f93f4066f90 0x7f93f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.307+0000 7f940be00640 1 -- 172.21.15.87:0/2761426770 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f940c084e70 con 0x7f941415ac70 2024-04-25T10:12:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.307+0000 7f9413e00640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f93f4066f90 0x7f93f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.307+0000 7f9413e00640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f93f4066f90 0x7f93f4069450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94040045f0 tx=0x7f9404008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.311+0000 7f940be00640 1 -- 172.21.15.87:0/2761426770 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f940c055790 con 0x7f941415ac70 2024-04-25T10:12:53.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.515+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f94140b27c0 con 0x7f941415ac70 2024-04-25T10:12:53.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.911+0000 7f940be00640 1 -- 172.21.15.87:0/2761426770 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v53) v1 ==== 151+0+0 (secure 0 0 0) 0x7f940c059830 con 0x7f941415ac70 2024-04-25T10:12:53.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f93f4066f90 msgr2=0x7f93f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f93f4066f90 0x7f93f4069450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94040045f0 tx=0x7f9404008040 comp rx=0 tx=0).stop 2024-04-25T10:12:53.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941415ac70 msgr2=0x7f94141592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941415ac70 0x7f94141592c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c7a0 tx=0x7f940c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 shutdown_connections 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f93f4066f90 0x7f93f4069450 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f941415ac70 0x7f94141592c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140b4da0 0x7f94141f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 --2- 172.21.15.87:0/2761426770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f941406dd30 0x7f94141536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 >> 172.21.15.87:0/2761426770 conn(0x7f94140b1a20 msgr2=0x7f94141544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 shutdown_connections 2024-04-25T10:12:53.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:53.915+0000 7f9419600640 1 -- 172.21.15.87:0/2761426770 wait complete. 2024-04-25T10:12:53.944 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:12:54.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.075+0000 7f5315e00640 1 Processor -- start 2024-04-25T10:12:54.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.075+0000 7f5315e00640 1 -- start start 2024-04-25T10:12:54.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f5315e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 0x7f53100ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f5315e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5310159920 con 0x7f53100aebd0 2024-04-25T10:12:54.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f5315e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531015c2f0 con 0x7f53100721f0 2024-04-25T10:12:54.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f5315e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53100af030 con 0x7f5310159540 2024-04-25T10:12:54.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f53100aebd0 0x7f5310071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40298/0 (socket says 172.21.15.87:40298) 2024-04-25T10:12:54.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 -- 172.21.15.87:0/2238180163 learned_addr learned my addr 172.21.15.87:0/2238180163 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:54.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530ea00640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 0x7f53100ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448359848 0 0) 0x7f5310159920 con 0x7f53100aebd0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f4003530 con 0x7f53100aebd0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272080802 0 0) 0x7f531015c2f0 con 0x7f53100721f0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5310159920 con 0x7f53100721f0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 408388192 0 0) 0x7f52f4003530 con 0x7f53100aebd0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 msgr2=0x7f53100ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 0x7f53100ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 >> v1:172.21.15.169:6789/0 conn(0x7f53100721f0 legacy=0x7f5310070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531015c5d0 con 0x7f53100aebd0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f52fc001680 con 0x7f53100aebd0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 >> v1:172.21.15.87:6789/0 conn(0x7f53100aebd0 legacy=0x7f5310071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 0x7f52f4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f52f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52f4004d50 0x7f52f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400ba80 con 0x7f52f40040e0 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400bc30 con 0x7f52f4004d50 2024-04-25T10:12:54.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400be30 con 0x7f5310159540 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f52f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530ea00640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 0x7f52f4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530f400640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52f4004d50 0x7f52f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 -- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 msgr2=0x7f52f4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5310159540 0x7f52f4003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 -- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52f4004d50 msgr2=0x7f52f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52f4004d50 0x7f52f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.079+0000 7f530e000640 1 -- 172.21.15.87:0/2238180163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4003530 con 0x7f52f40040e0 2024-04-25T10:12:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f530e000640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f52f4004810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00c190 tx=0x7f52fc01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f967642280246b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc007cd0 con 0x7f52f40040e0 2024-04-25T10:12:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0a9020 con 0x7f52f40040e0 2024-04-25T10:12:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f530d600640 1 -- 172.21.15.87:0/2238180163 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0a5b00 con 0x7f52f40040e0 2024-04-25T10:12:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f5315e00640 1 -- 172.21.15.87:0/2238180163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 msgr2=0x7f52f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f5315e00640 1 --2- 172.21.15.87:0/2238180163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f52f4004810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00c190 tx=0x7f52fc01cc00 comp rx=0 tx=0).stop 2024-04-25T10:12:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f530f400640 1 -- 172.21.15.87:0/2238180163 reap_dead start 2024-04-25T10:12:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f5315e00640 1 -- 172.21.15.87:0/2238180163 shutdown_connections 2024-04-25T10:12:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f5315e00640 1 -- 172.21.15.87:0/2238180163 >> 172.21.15.87:0/2238180163 conn(0x7f53100669a0 msgr2=0x7f5310066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f5315e00640 1 -- 172.21.15.87:0/2238180163 shutdown_connections 2024-04-25T10:12:54.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.083+0000 7f5315e00640 1 -- 172.21.15.87:0/2238180163 wait complete. 2024-04-25T10:12:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 Processor -- start 2024-04-25T10:12:54.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 -- start start 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f5310145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53101e53b0 0x7f53100b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531015acb0 0x7f531015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531015bfa0 con 0x7f52f40040e0 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531015cb90 con 0x7f531015acb0 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531006c0d0 con 0x7f53101e53b0 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f5310145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f5310145170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40880/0 (socket says 172.21.15.87:40880) 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 -- 172.21.15.87:0/3994463806 learned_addr learned my addr 172.21.15.87:0/3994463806 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530e000640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53101e53b0 0x7f53100b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530f400640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531015acb0 0x7f531015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 -- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53101e53b0 msgr2=0x7f53100b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53101e53b0 0x7f53100b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 -- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531015acb0 msgr2=0x7f531015b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531015acb0 0x7f531015b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 -- 172.21.15.87:0/3994463806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531015b640 con 0x7f52f40040e0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530f400640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531015acb0 0x7f531015b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530e000640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53101e53b0 0x7f53100b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f530ea00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f5310145170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5300011e90 tx=0x7f530000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5307400640 1 -- 172.21.15.87:0/3994463806 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300015630 con 0x7f52f40040e0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5307400640 1 -- 172.21.15.87:0/3994463806 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53000157d0 con 0x7f52f40040e0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5307400640 1 -- 172.21.15.87:0/3994463806 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300015ba0 con 0x7f52f40040e0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53101e5840 con 0x7f52f40040e0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53101e5e00 con 0x7f52f40040e0 2024-04-25T10:12:54.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.087+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531004ef30 con 0x7f52f40040e0 2024-04-25T10:12:54.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.091+0000 7f5307400640 1 -- 172.21.15.87:0/3994463806 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f530001d400 con 0x7f52f40040e0 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.091+0000 7f5307400640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52f0066f90 0x7f52f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.091+0000 7f5307400640 1 -- 172.21.15.87:0/3994463806 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53000860f0 con 0x7f52f40040e0 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.091+0000 7f530e000640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52f0066f90 0x7f52f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.091+0000 7f530e000640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52f0066f90 0x7f52f0069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0aa000 tx=0x7f52fc0019d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.095+0000 7f5307400640 1 -- 172.21.15.87:0/3994463806 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5300019030 con 0x7f52f40040e0 2024-04-25T10:12:54.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.247+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f53100cbe80 con 0x7f52f40040e0 2024-04-25T10:12:54.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5307400640 1 -- 172.21.15.87:0/3994463806 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5300010040 con 0x7f52f40040e0 2024-04-25T10:12:54.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52f0066f90 msgr2=0x7f52f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52f0066f90 0x7f52f0069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0aa000 tx=0x7f52fc0019d0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 msgr2=0x7f5310145170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f5310145170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5300011e90 tx=0x7f530000f580 comp rx=0 tx=0).stop 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 shutdown_connections 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52f0066f90 0x7f52f0069450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531015acb0 0x7f531015b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53101e53b0 0x7f53100b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 --2- 172.21.15.87:0/3994463806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52f40040e0 0x7f5310145170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 >> 172.21.15.87:0/3994463806 conn(0x7f53100669a0 msgr2=0x7f531015d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 shutdown_connections 2024-04-25T10:12:54.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.255+0000 7f5315e00640 1 -- 172.21.15.87:0/3994463806 wait complete. 2024-04-25T10:12:54.281 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:54.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:54.366 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:54.421 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:54.421 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:54.421 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:54.430 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:54.430 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:54.481 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:54.481 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:54.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:54.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bf200640 1 Processor -- start 2024-04-25T10:12:54.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bf200640 1 -- start start 2024-04-25T10:12:54.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 0x7fb1c015d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bf200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c0161b30 con 0x7fb1c00d11c0 2024-04-25T10:12:54.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bf200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c0161e10 con 0x7fb1c00d0de0 2024-04-25T10:12:54.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01620f0 con 0x7fb1c00cee50 2024-04-25T10:12:54.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1be800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb1c00d11c0 0x7fb1c00cbef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40302/0 (socket says 172.21.15.87:40302) 2024-04-25T10:12:54.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1be800640 1 -- 172.21.15.87:0/2362095782 learned_addr learned my addr 172.21.15.87:0/2362095782 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 0x7fb1c015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778387298 0 0) 0x7fb1c0161b30 con 0x7fb1c00d11c0 2024-04-25T10:12:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a4003530 con 0x7fb1c00d11c0 2024-04-25T10:12:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821357572 0 0) 0x7fb1c0161e10 con 0x7fb1c00d0de0 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c0161b30 con 0x7fb1c00d0de0 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1702593182 0 0) 0x7fb1a4003530 con 0x7fb1c00d11c0 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 msgr2=0x7fb1c015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 0x7fb1c015d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 >> v1:172.21.15.169:6789/0 conn(0x7fb1c00d0de0 legacy=0x7fb1c00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c01623d0 con 0x7fb1c00d11c0 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb1b8001680 con 0x7fb1c00d11c0 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 0x7fb1c015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 >> v1:172.21.15.87:6789/0 conn(0x7fb1c00d11c0 legacy=0x7fb1c00cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4003b80 0x7fb1a4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a4004810 0x7fb1a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 0x7fb1a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01620f0 con 0x7fb1a4004810 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400bae0 con 0x7fb1a4003b80 2024-04-25T10:12:54.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400bce0 con 0x7fb1c00cee50 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.703+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a4004810 0x7fb1a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bde00640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4003b80 0x7fb1a4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1be800640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 0x7fb1a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bd400640 1 -- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 msgr2=0x7fb1a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00cee50 0x7fb1a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bd400640 1 -- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4003b80 msgr2=0x7fb1a4003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4003b80 0x7fb1a4003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bd400640 1 -- 172.21.15.87:0/2362095782 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4003530 con 0x7fb1a4004810 2024-04-25T10:12:54.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a4004810 0x7fb1a4004c90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009c80 tx=0x7fb1ac01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c910c28bb268fd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0b2070 con 0x7fb1a4004810 2024-04-25T10:12:54.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac0a5840 con 0x7fb1a4004810 2024-04-25T10:12:54.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bca00640 1 -- 172.21.15.87:0/2362095782 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac009880 con 0x7fb1a4004810 2024-04-25T10:12:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bf200640 1 -- 172.21.15.87:0/2362095782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a4004810 msgr2=0x7fb1a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1bf200640 1 --2- 172.21.15.87:0/2362095782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a4004810 0x7fb1a4004c90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009c80 tx=0x7fb1ac01cb80 comp rx=0 tx=0).stop 2024-04-25T10:12:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.707+0000 7fb1be800640 1 -- 172.21.15.87:0/2362095782 reap_dead start 2024-04-25T10:12:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- 172.21.15.87:0/2362095782 shutdown_connections 2024-04-25T10:12:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- 172.21.15.87:0/2362095782 >> 172.21.15.87:0/2362095782 conn(0x7fb1c0072800 msgr2=0x7fb1c0072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- 172.21.15.87:0/2362095782 shutdown_connections 2024-04-25T10:12:54.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- 172.21.15.87:0/2362095782 wait complete. 2024-04-25T10:12:54.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 Processor -- start 2024-04-25T10:12:54.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- start start 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4004810 0x7fb1c015e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c0160f90 0x7fb1c0158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00ae210 0x7fb1c0163e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00cd1e0 con 0x7fb1c0160f90 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00cd060 con 0x7fb1a4004810 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0169d40 con 0x7fb1c00ae210 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c0160f90 0x7fb1c0158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c0160f90 0x7fb1c0158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40896/0 (socket says 172.21.15.87:40896) 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 -- 172.21.15.87:0/2974077255 learned_addr learned my addr 172.21.15.87:0/2974077255 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1be800640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00ae210 0x7fb1c0163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bde00640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4004810 0x7fb1c015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 -- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00ae210 msgr2=0x7fb1c0163e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00ae210 0x7fb1c0163e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 -- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4004810 msgr2=0x7fb1c015e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4004810 0x7fb1c015e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 -- 172.21.15.87:0/2974077255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00b10f0 con 0x7fb1c0160f90 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.711+0000 7fb1bd400640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c0160f90 0x7fb1c0158c30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac01c9b0 tx=0x7fb1a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1b6000640 1 -- 172.21.15.87:0/2974077255 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0bd020 con 0x7fb1c0160f90 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1b6000640 1 -- 172.21.15.87:0/2974077255 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac009400 con 0x7fb1c0160f90 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1b6000640 1 -- 172.21.15.87:0/2974077255 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0b2040 con 0x7fb1c0160f90 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c00aee90 con 0x7fb1c0160f90 2024-04-25T10:12:54.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c00b1fd0 con 0x7fb1c0160f90 2024-04-25T10:12:54.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c0075120 con 0x7fb1c0160f90 2024-04-25T10:12:54.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1b6000640 1 -- 172.21.15.87:0/2974077255 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1ac004260 con 0x7fb1c0160f90 2024-04-25T10:12:54.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1b6000640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb1a0066f90 0x7fb1a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1b6000640 1 -- 172.21.15.87:0/2974077255 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1ac11eb50 con 0x7fb1c0160f90 2024-04-25T10:12:54.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1bde00640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb1a0066f90 0x7fb1a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.715+0000 7fb1bde00640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb1a0066f90 0x7fb1a0069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0007fd0 tx=0x7fb1b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.719+0000 7fb1b6000640 1 -- 172.21.15.87:0/2974077255 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1ac0168c0 con 0x7fb1c0160f90 2024-04-25T10:12:54.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1c00ce440 con 0x7fb1c0160f90 2024-04-25T10:12:54.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1b6000640 1 -- 172.21.15.87:0/2974077255 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fb1ac0efb60 con 0x7fb1c0160f90 2024-04-25T10:12:54.881 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:54.881 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:54.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb1a0066f90 msgr2=0x7fb1a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb1a0066f90 0x7fb1a0069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0007fd0 tx=0x7fb1b0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:54.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c0160f90 msgr2=0x7fb1c0158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c0160f90 0x7fb1c0158c30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac01c9b0 tx=0x7fb1a4005840 comp rx=0 tx=0).stop 2024-04-25T10:12:54.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 shutdown_connections 2024-04-25T10:12:54.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb1a0066f90 0x7fb1a0069450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00ae210 0x7fb1c0163e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c0160f90 0x7fb1c0158c30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 --2- 172.21.15.87:0/2974077255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a4004810 0x7fb1c015e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 >> 172.21.15.87:0/2974077255 conn(0x7fb1c0072800 msgr2=0x7fb1c015cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 shutdown_connections 2024-04-25T10:12:54.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:54.875+0000 7fb1bf200640 1 -- 172.21.15.87:0/2974077255 wait complete. 2024-04-25T10:12:54.902 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:54.902 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:55.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9bae200640 1 Processor -- start 2024-04-25T10:12:55.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9bae200640 1 -- start start 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9bae200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ba8159540 0x7f9ba80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9bae200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba80aebd0 0x7f9ba8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9bae200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba8071360 con 0x7f9ba8072020 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9bae200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba8159920 con 0x7f9ba8159540 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9bae200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba815c2f0 con 0x7f9ba80aebd0 2024-04-25T10:12:55.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba7400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ba8159540 0x7f9ba80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9ba8072020 0x7f9ba815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40318/0 (socket says 172.21.15.87:40318) 2024-04-25T10:12:55.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba7e00640 1 -- 172.21.15.87:0/3862625353 learned_addr learned my addr 172.21.15.87:0/3862625353 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:55.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba6a00640 1 --2- 172.21.15.87:0/3862625353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba80aebd0 0x7f9ba8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba7400640 1 -- 172.21.15.87:0/3862625353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba80aebd0 msgr2=0x7f9ba8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba7400640 1 --2- 172.21.15.87:0/3862625353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba80aebd0 0x7f9ba8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba7400640 1 -- 172.21.15.87:0/3862625353 >> v1:172.21.15.87:6789/0 conn(0x7f9ba8072020 legacy=0x7f9ba815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba6000640 1 -- 172.21.15.87:0/3862625353 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287940318 0 0) 0x7f9ba8071360 con 0x7f9ba8072020 2024-04-25T10:12:55.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.035+0000 7f9ba7400640 1 -- 172.21.15.87:0/3862625353 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba815c5d0 con 0x7f9ba8159540 2024-04-25T10:12:55.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9ba7400640 1 --2- 172.21.15.87:0/3862625353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba8159540 0x7f9ba80ae690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9b980097c0 tx=0x7f9b9801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a55c5fec13408f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9ba6000640 1 -- 172.21.15.87:0/3862625353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b980b0070 con 0x7f9ba8159540 2024-04-25T10:12:55.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9ba6000640 1 -- 172.21.15.87:0/3862625353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b980ab600 con 0x7f9ba8159540 2024-04-25T10:12:55.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9ba6000640 1 -- 172.21.15.87:0/3862625353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b980ab9d0 con 0x7f9ba8159540 2024-04-25T10:12:55.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 -- 172.21.15.87:0/3862625353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba8159540 msgr2=0x7f9ba80ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 --2- 172.21.15.87:0/3862625353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba8159540 0x7f9ba80ae690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9b980097c0 tx=0x7f9b9801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 -- 172.21.15.87:0/3862625353 shutdown_connections 2024-04-25T10:12:55.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 --2- 172.21.15.87:0/3862625353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba80aebd0 0x7f9ba8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 --2- 172.21.15.87:0/3862625353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba8159540 0x7f9ba80ae690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 -- 172.21.15.87:0/3862625353 >> 172.21.15.87:0/3862625353 conn(0x7f9ba80669a0 msgr2=0x7f9ba8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 -- 172.21.15.87:0/3862625353 shutdown_connections 2024-04-25T10:12:55.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 -- 172.21.15.87:0/3862625353 wait complete. 2024-04-25T10:12:55.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.039+0000 7f9bae200640 1 Processor -- start 2024-04-25T10:12:55.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 -- start start 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8072020 0x7f9ba8154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba80aebd0 0x7f9ba80af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba8159540 0x7f9ba8142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8072430 con 0x7f9ba80aebd0 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba806c0d0 con 0x7f9ba8072020 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8168930 con 0x7f9ba8159540 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba80aebd0 0x7f9ba80af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba80aebd0 0x7f9ba80af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40904/0 (socket says 172.21.15.87:40904) 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 -- 172.21.15.87:0/3852144241 learned_addr learned my addr 172.21.15.87:0/3852144241 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba7e00640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba8159540 0x7f9ba8142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba7400640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8072020 0x7f9ba8154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 -- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba8159540 msgr2=0x7f9ba8142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba8159540 0x7f9ba8142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 -- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8072020 msgr2=0x7f9ba8154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8072020 0x7f9ba8154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 -- 172.21.15.87:0/3852144241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8142680 con 0x7f9ba80aebd0 2024-04-25T10:12:55.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9ba6a00640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba80aebd0 0x7f9ba80af270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d5f0 tx=0x7f9b9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9b9f400640 1 -- 172.21.15.87:0/3852144241 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940125f0 con 0x7f9ba80aebd0 2024-04-25T10:12:55.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9b9f400640 1 -- 172.21.15.87:0/3852144241 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b94019070 con 0x7f9ba80aebd0 2024-04-25T10:12:55.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9b9f400640 1 -- 172.21.15.87:0/3852144241 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940129b0 con 0x7f9ba80aebd0 2024-04-25T10:12:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba8153210 con 0x7f9ba80aebd0 2024-04-25T10:12:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba81537d0 con 0x7f9ba80aebd0 2024-04-25T10:12:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.043+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba804ef30 con 0x7f9ba80aebd0 2024-04-25T10:12:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.047+0000 7f9b9f400640 1 -- 172.21.15.87:0/3852144241 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b94012b50 con 0x7f9ba80aebd0 2024-04-25T10:12:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.047+0000 7f9b9f400640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b88066f60 0x7f9b88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.047+0000 7f9b9f400640 1 -- 172.21.15.87:0/3852144241 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b94085300 con 0x7f9ba80aebd0 2024-04-25T10:12:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.047+0000 7f9ba7400640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b88066f60 0x7f9b88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.047+0000 7f9ba7400640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b88066f60 0x7f9b88069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9b980ad000 tx=0x7f9b980a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.051+0000 7f9b9f400640 1 -- 172.21.15.87:0/3852144241 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b94055cb0 con 0x7f9ba80aebd0 2024-04-25T10:12:55.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.203+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ba806d7f0 con 0x7f9ba80aebd0 2024-04-25T10:12:55.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.203+0000 7f9b9f400640 1 -- 172.21.15.87:0/3852144241 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b94059d50 con 0x7f9ba80aebd0 2024-04-25T10:12:55.211 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b88066f60 msgr2=0x7f9b88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b88066f60 0x7f9b88069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9b980ad000 tx=0x7f9b980a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba80aebd0 msgr2=0x7f9ba80af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba80aebd0 0x7f9ba80af270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d5f0 tx=0x7f9b9400dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 shutdown_connections 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b88066f60 0x7f9b88069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba8159540 0x7f9ba8142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ba80aebd0 0x7f9ba80af270 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 --2- 172.21.15.87:0/3852144241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba8072020 0x7f9ba8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 >> 172.21.15.87:0/3852144241 conn(0x7f9ba80669a0 msgr2=0x7f9ba806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 shutdown_connections 2024-04-25T10:12:55.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.207+0000 7f9bae200640 1 -- 172.21.15.87:0/3852144241 wait complete. 2024-04-25T10:12:55.231 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:55.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.363+0000 7fc6e4400640 1 Processor -- start 2024-04-25T10:12:55.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e4400640 1 -- start start 2024-04-25T10:12:55.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e4400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 0x7fc6dc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e4400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc159920 con 0x7fc6dc0aebd0 2024-04-25T10:12:55.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e4400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc15c2f0 con 0x7fc6dc0721f0 2024-04-25T10:12:55.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e4400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0af030 con 0x7fc6dc159540 2024-04-25T10:12:55.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc6dc0aebd0 0x7fc6dc071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40334/0 (socket says 172.21.15.87:40334) 2024-04-25T10:12:55.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 -- 172.21.15.87:0/3612588494 learned_addr learned my addr 172.21.15.87:0/3612588494 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1a00640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 0x7fc6dc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729210530 0 0) 0x7fc6dc159920 con 0x7fc6dc0aebd0 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c4003530 con 0x7fc6dc0aebd0 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207127310 0 0) 0x7fc6c4003530 con 0x7fc6dc0aebd0 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 msgr2=0x7fc6dc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 0x7fc6dc0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 >> v1:172.21.15.169:6789/0 conn(0x7fc6dc0721f0 legacy=0x7fc6dc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc15c5d0 con 0x7fc6dc0aebd0 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc6c8001680 con 0x7fc6dc0aebd0 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 >> v1:172.21.15.87:6789/0 conn(0x7fc6dc0aebd0 legacy=0x7fc6dc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40039f0 0x7fc6c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c4004810 0x7fc6c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 0x7fc6c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c400ba80 con 0x7fc6c4004810 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c400bc00 con 0x7fc6c40039f0 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c400be00 con 0x7fc6dc159540 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c4004810 0x7fc6c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e2400640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 0x7fc6c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1a00640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40039f0 0x7fc6c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 -- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 msgr2=0x7fc6c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc159540 0x7fc6c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 -- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40039f0 msgr2=0x7fc6c4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40039f0 0x7fc6c4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.367+0000 7fc6e1000640 1 -- 172.21.15.87:0/3612588494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4003530 con 0x7fc6c4004810 2024-04-25T10:12:55.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e1000640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c4004810 0x7fc6c4004c90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800c210 tx=0x7fc6c801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5a409c6d297eab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8007d20 con 0x7fc6c4004810 2024-04-25T10:12:55.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c8005ae0 con 0x7fc6c4004810 2024-04-25T10:12:55.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6dbe00640 1 -- 172.21.15.87:0/3612588494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8007070 con 0x7fc6c4004810 2024-04-25T10:12:55.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e4400640 1 -- 172.21.15.87:0/3612588494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c4004810 msgr2=0x7fc6c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e4400640 1 --2- 172.21.15.87:0/3612588494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c4004810 0x7fc6c4004c90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800c210 tx=0x7fc6c801f390 comp rx=0 tx=0).stop 2024-04-25T10:12:55.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e2400640 1 -- 172.21.15.87:0/3612588494 reap_dead start 2024-04-25T10:12:55.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e4400640 1 -- 172.21.15.87:0/3612588494 shutdown_connections 2024-04-25T10:12:55.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e4400640 1 -- 172.21.15.87:0/3612588494 >> 172.21.15.87:0/3612588494 conn(0x7fc6dc0669a0 msgr2=0x7fc6dc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e4400640 1 -- 172.21.15.87:0/3612588494 shutdown_connections 2024-04-25T10:12:55.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.371+0000 7fc6e4400640 1 -- 172.21.15.87:0/3612588494 wait complete. 2024-04-25T10:12:55.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 Processor -- start 2024-04-25T10:12:55.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 -- start start 2024-04-25T10:12:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6dc14d920 0x7fc6dc0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc14dd00 0x7fc6dc0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc1f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc15bfa0 con 0x7fc6dc14d920 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc15cb90 con 0x7fc6dc0b1a80 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc06c0d0 con 0x7fc6dc14dd00 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6dc14d920 0x7fc6dc0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6dc14d920 0x7fc6dc0afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40924/0 (socket says 172.21.15.87:40924) 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 -- 172.21.15.87:0/1359056574 learned_addr learned my addr 172.21.15.87:0/1359056574 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1000640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc14dd00 0x7fc6dc0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e2400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc1f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 -- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc14dd00 msgr2=0x7fc6dc0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc14dd00 0x7fc6dc0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 -- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6dc0b1a80 msgr2=0x7fc6dc1f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc1f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 -- 172.21.15.87:0/1359056574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc1f0a80 con 0x7fc6dc14d920 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e2400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc1f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e1a00640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6dc14d920 0x7fc6dc0afa00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00119d0 tx=0x7fc6d0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6d9600640 1 -- 172.21.15.87:0/1359056574 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d000faf0 con 0x7fc6dc14d920 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6d9600640 1 -- 172.21.15.87:0/1359056574 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d000fc90 con 0x7fc6dc14d920 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc1f0d60 con 0x7fc6dc14d920 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6d9600640 1 -- 172.21.15.87:0/1359056574 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0014440 con 0x7fc6dc14d920 2024-04-25T10:12:55.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc0b24f0 con 0x7fc6dc14d920 2024-04-25T10:12:55.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.375+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc04ef30 con 0x7fc6dc14d920 2024-04-25T10:12:55.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.379+0000 7fc6d9600640 1 -- 172.21.15.87:0/1359056574 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6d0012040 con 0x7fc6dc14d920 2024-04-25T10:12:55.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.379+0000 7fc6d9600640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc6bc066f90 0x7fc6bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.379+0000 7fc6d9600640 1 -- 172.21.15.87:0/1359056574 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6d00855c0 con 0x7fc6dc14d920 2024-04-25T10:12:55.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.379+0000 7fc6e1000640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc6bc066f90 0x7fc6bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.379+0000 7fc6e1000640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc6bc066f90 0x7fc6bc069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80af000 tx=0x7fc6c80aa000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.383+0000 7fc6d9600640 1 -- 172.21.15.87:0/1359056574 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6d0019030 con 0x7fc6dc14d920 2024-04-25T10:12:55.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.535+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6dc0b2260 con 0x7fc6dc14d920 2024-04-25T10:12:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.535+0000 7fc6d9600640 1 -- 172.21.15.87:0/1359056574 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6d0056680 con 0x7fc6dc14d920 2024-04-25T10:12:55.542 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:55.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc6bc066f90 msgr2=0x7fc6bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc6bc066f90 0x7fc6bc069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80af000 tx=0x7fc6c80aa000 comp rx=0 tx=0).stop 2024-04-25T10:12:55.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6dc14d920 msgr2=0x7fc6dc0afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6dc14d920 0x7fc6dc0afa00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00119d0 tx=0x7fc6d0011e90 comp rx=0 tx=0).stop 2024-04-25T10:12:55.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 shutdown_connections 2024-04-25T10:12:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc6bc066f90 0x7fc6bc069450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc1f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6dc14dd00 0x7fc6dc0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 --2- 172.21.15.87:0/1359056574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6dc14d920 0x7fc6dc0afa00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 >> 172.21.15.87:0/1359056574 conn(0x7fc6dc0669a0 msgr2=0x7fc6dc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 shutdown_connections 2024-04-25T10:12:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.539+0000 7fc6e4400640 1 -- 172.21.15.87:0/1359056574 wait complete. 2024-04-25T10:12:55.563 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:55.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00eba00640 1 Processor -- start 2024-04-25T10:12:55.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00eba00640 1 -- start start 2024-04-25T10:12:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00eba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00e4159540 0x7f00e40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00eba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00e40aebd0 0x7f00e4071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00eba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e4071360 con 0x7f00e4072020 2024-04-25T10:12:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00eba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e4159920 con 0x7f00e40aebd0 2024-04-25T10:12:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00eba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e415c2f0 con 0x7f00e4159540 2024-04-25T10:12:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00dbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00e40aebd0 0x7f00e4071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00dbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00e40aebd0 0x7f00e4071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40930/0 (socket says 172.21.15.87:40930) 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00dbe00640 1 -- 172.21.15.87:0/2864000055 learned_addr learned my addr 172.21.15.87:0/2864000055 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00e9a00640 1 --1- 172.21.15.87:0/2864000055 >> v1:172.21.15.87:6789/0 conn(0x7f00e4072020 0x7f00e415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40340/0 (socket says 172.21.15.87:40340) 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.695+0000 7f00e9000640 1 --2- 172.21.15.87:0/2864000055 >> v2:172.21.15.169:3300/0 conn(0x7f00e4159540 0x7f00e40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00dbe00640 1 -- 172.21.15.87:0/2864000055 >> v2:172.21.15.169:3300/0 conn(0x7f00e4159540 msgr2=0x7f00e40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00dbe00640 1 --2- 172.21.15.87:0/2864000055 >> v2:172.21.15.169:3300/0 conn(0x7f00e4159540 0x7f00e40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00dbe00640 1 -- 172.21.15.87:0/2864000055 >> v1:172.21.15.87:6789/0 conn(0x7f00e4072020 legacy=0x7f00e415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00dbe00640 1 -- 172.21.15.87:0/2864000055 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e415c5d0 con 0x7f00e40aebd0 2024-04-25T10:12:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00e3e00640 1 -- 172.21.15.87:0/2864000055 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112091365 0 0) 0x7f00e4071360 con 0x7f00e4072020 2024-04-25T10:12:55.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00dbe00640 1 --2- 172.21.15.87:0/2864000055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e40aebd0 0x7f00e4071ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00e40b4c20 tx=0x7f00cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1f0dd85114723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00e3e00640 1 -- 172.21.15.87:0/2864000055 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0b0070 con 0x7f00e40aebd0 2024-04-25T10:12:55.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00e3e00640 1 -- 172.21.15.87:0/2864000055 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc0ab600 con 0x7f00e40aebd0 2024-04-25T10:12:55.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00e3e00640 1 -- 172.21.15.87:0/2864000055 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0ab9d0 con 0x7f00e40aebd0 2024-04-25T10:12:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00eba00640 1 -- 172.21.15.87:0/2864000055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e40aebd0 msgr2=0x7f00e4071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.699+0000 7f00eba00640 1 --2- 172.21.15.87:0/2864000055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e40aebd0 0x7f00e4071ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00e40b4c20 tx=0x7f00cc01c520 comp rx=0 tx=0).stop 2024-04-25T10:12:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- 172.21.15.87:0/2864000055 shutdown_connections 2024-04-25T10:12:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 --2- 172.21.15.87:0/2864000055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e40aebd0 0x7f00e4071ae0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 --2- 172.21.15.87:0/2864000055 >> v2:172.21.15.169:3300/0 conn(0x7f00e4159540 0x7f00e40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- 172.21.15.87:0/2864000055 >> 172.21.15.87:0/2864000055 conn(0x7f00e40669a0 msgr2=0x7f00e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- 172.21.15.87:0/2864000055 shutdown_connections 2024-04-25T10:12:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- 172.21.15.87:0/2864000055 wait complete. 2024-04-25T10:12:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 Processor -- start 2024-04-25T10:12:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- start start 2024-04-25T10:12:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4072020 0x7f00e40b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e40aebd0 0x7f00e4158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e4159540 0x7f00e4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4072430 con 0x7f00e4159540 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e406c0d0 con 0x7f00e40aebd0 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4168930 con 0x7f00e4072020 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e4159540 0x7f00e4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e4159540 0x7f00e4159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40944/0 (socket says 172.21.15.87:40944) 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 -- 172.21.15.87:0/124665552 learned_addr learned my addr 172.21.15.87:0/124665552 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9000640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4072020 0x7f00e40b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00dbe00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e40aebd0 0x7f00e4158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 -- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4072020 msgr2=0x7f00e40b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4072020 0x7f00e40b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 -- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e40aebd0 msgr2=0x7f00e4158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e40aebd0 0x7f00e4158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 -- 172.21.15.87:0/124665552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4156fd0 con 0x7f00e4159540 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9000640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4072020 0x7f00e40b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e9a00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e4159540 0x7f00e4159af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c7d0 tx=0x7f00dc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e1600640 1 -- 172.21.15.87:0/124665552 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc018070 con 0x7f00e4159540 2024-04-25T10:12:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e1600640 1 -- 172.21.15.87:0/124665552 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc00a640 con 0x7f00e4159540 2024-04-25T10:12:55.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00e1600640 1 -- 172.21.15.87:0/124665552 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00aa70 con 0x7f00e4159540 2024-04-25T10:12:55.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.703+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e41572b0 con 0x7f00e4159540 2024-04-25T10:12:55.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.707+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e4157870 con 0x7f00e4159540 2024-04-25T10:12:55.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.707+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e404ef30 con 0x7f00e4159540 2024-04-25T10:12:55.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.707+0000 7f00e1600640 1 -- 172.21.15.87:0/124665552 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00dc00ac10 con 0x7f00e4159540 2024-04-25T10:12:55.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.707+0000 7f00e1600640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00c0066f60 0x7f00c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.707+0000 7f00e1600640 1 -- 172.21.15.87:0/124665552 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00dc084660 con 0x7f00e4159540 2024-04-25T10:12:55.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.707+0000 7f00e9000640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00c0066f60 0x7f00c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.707+0000 7f00e9000640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00c0066f60 0x7f00c0069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009b10 tx=0x7f00d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.711+0000 7f00e1600640 1 -- 172.21.15.87:0/124665552 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00dc055670 con 0x7f00e4159540 2024-04-25T10:12:55.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.863+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00e4166700 con 0x7f00e4159540 2024-04-25T10:12:55.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.863+0000 7f00e1600640 1 -- 172.21.15.87:0/124665552 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00dc054f70 con 0x7f00e4159540 2024-04-25T10:12:55.871 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00c0066f60 msgr2=0x7f00c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00c0066f60 0x7f00c0069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009b10 tx=0x7f00d4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e4159540 msgr2=0x7f00e4159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e4159540 0x7f00e4159af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c7d0 tx=0x7f00dc00cc90 comp rx=0 tx=0).stop 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 shutdown_connections 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00c0066f60 0x7f00c0069420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e4159540 0x7f00e4159af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e40aebd0 0x7f00e4158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 --2- 172.21.15.87:0/124665552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4072020 0x7f00e40b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 >> 172.21.15.87:0/124665552 conn(0x7f00e40669a0 msgr2=0x7f00e40708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 shutdown_connections 2024-04-25T10:12:55.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:55.867+0000 7f00eba00640 1 -- 172.21.15.87:0/124665552 wait complete. 2024-04-25T10:12:55.892 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:56.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59a00640 1 Processor -- start 2024-04-25T10:12:56.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59a00640 1 -- start start 2024-04-25T10:12:56.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d5415db40 0x7f1d5406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d540b5010 0x7f1d5415df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5415e460 con 0x7f1d5406b2c0 2024-04-25T10:12:56.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d540b5780 con 0x7f1d540b5010 2024-04-25T10:12:56.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d540b5a60 con 0x7f1d5415db40 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d4b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d540b5010 0x7f1d5415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d4b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d540b5010 0x7f1d5415df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40960/0 (socket says 172.21.15.87:40960) 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d59000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1d5406b2c0 0x7f1d540b6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40350/0 (socket says 172.21.15.87:40350) 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d4b400640 1 -- 172.21.15.87:0/3736712837 learned_addr learned my addr 172.21.15.87:0/3736712837 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d53e00640 1 --2- 172.21.15.87:0/3736712837 >> v2:172.21.15.169:3300/0 conn(0x7f1d5415db40 0x7f1d5406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d4b400640 1 -- 172.21.15.87:0/3736712837 >> v2:172.21.15.169:3300/0 conn(0x7f1d5415db40 msgr2=0x7f1d5406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d4b400640 1 --2- 172.21.15.87:0/3736712837 >> v2:172.21.15.169:3300/0 conn(0x7f1d5415db40 0x7f1d5406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d4b400640 1 -- 172.21.15.87:0/3736712837 >> v1:172.21.15.87:6789/0 conn(0x7f1d5406b2c0 legacy=0x7f1d540b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d4b400640 1 -- 172.21.15.87:0/3736712837 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540b5d40 con 0x7f1d540b5010 2024-04-25T10:12:56.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.023+0000 7f1d53400640 1 -- 172.21.15.87:0/3736712837 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173764303 0 0) 0x7f1d5415e460 con 0x7f1d5406b2c0 2024-04-25T10:12:56.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d4b400640 1 --2- 172.21.15.87:0/3736712837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b5010 0x7f1d5415df20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d54068870 tx=0x7f1d3c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d541afa8441bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d53400640 1 -- 172.21.15.87:0/3736712837 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0b5070 con 0x7f1d540b5010 2024-04-25T10:12:56.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d53400640 1 -- 172.21.15.87:0/3736712837 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c0a8a10 con 0x7f1d540b5010 2024-04-25T10:12:56.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d53400640 1 -- 172.21.15.87:0/3736712837 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0a8de0 con 0x7f1d540b5010 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 -- 172.21.15.87:0/3736712837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b5010 msgr2=0x7f1d5415df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 --2- 172.21.15.87:0/3736712837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b5010 0x7f1d5415df20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d54068870 tx=0x7f1d3c01ed10 comp rx=0 tx=0).stop 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 -- 172.21.15.87:0/3736712837 shutdown_connections 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 --2- 172.21.15.87:0/3736712837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b5010 0x7f1d5415df20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 --2- 172.21.15.87:0/3736712837 >> v2:172.21.15.169:3300/0 conn(0x7f1d5415db40 0x7f1d5406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 -- 172.21.15.87:0/3736712837 >> 172.21.15.87:0/3736712837 conn(0x7f1d540d17b0 msgr2=0x7f1d540d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 -- 172.21.15.87:0/3736712837 shutdown_connections 2024-04-25T10:12:56.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.027+0000 7f1d59a00640 1 -- 172.21.15.87:0/3736712837 wait complete. 2024-04-25T10:12:56.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 Processor -- start 2024-04-25T10:12:56.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 -- start start 2024-04-25T10:12:56.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d5406b2c0 0x7f1d54152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d540b5010 0x7f1d540c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d5415db40 0x7f1d540ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540b5480 con 0x7f1d5406b2c0 2024-04-25T10:12:56.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540b6300 con 0x7f1d540b5010 2024-04-25T10:12:56.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5406b6f0 con 0x7f1d5415db40 2024-04-25T10:12:56.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d5406b2c0 0x7f1d54152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d5406b2c0 0x7f1d54152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40974/0 (socket says 172.21.15.87:40974) 2024-04-25T10:12:56.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 -- 172.21.15.87:0/1327294682 learned_addr learned my addr 172.21.15.87:0/1327294682 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:56.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d4b400640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d540b5010 0x7f1d540c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59000640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d5415db40 0x7f1d540ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 -- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d5415db40 msgr2=0x7f1d540ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d5415db40 0x7f1d540ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 -- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d540b5010 msgr2=0x7f1d540c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d540b5010 0x7f1d540c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 -- 172.21.15.87:0/1327294682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5415b560 con 0x7f1d5406b2c0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59000640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d5415db40 0x7f1d540ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d4b400640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d540b5010 0x7f1d540c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d53e00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d5406b2c0 0x7f1d54152a20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400e870 tx=0x7f1d4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d50c00640 1 -- 172.21.15.87:0/1327294682 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44017070 con 0x7f1d5406b2c0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d50c00640 1 -- 172.21.15.87:0/1327294682 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4400c940 con 0x7f1d5406b2c0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5409edd0 con 0x7f1d5406b2c0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d50c00640 1 -- 172.21.15.87:0/1327294682 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400cd10 con 0x7f1d5406b2c0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.031+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540cce10 con 0x7f1d5406b2c0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.035+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54075120 con 0x7f1d5406b2c0 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.035+0000 7f1d50c00640 1 -- 172.21.15.87:0/1327294682 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d4401a070 con 0x7f1d5406b2c0 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.035+0000 7f1d50c00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d30066f90 0x7f1d30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.035+0000 7f1d50c00640 1 -- 172.21.15.87:0/1327294682 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d44084f20 con 0x7f1d5406b2c0 2024-04-25T10:12:56.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.035+0000 7f1d4b400640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d30066f90 0x7f1d30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.035+0000 7f1d4b400640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d30066f90 0x7f1d30069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0b2000 tx=0x7f1d3c0aa070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.039+0000 7f1d50c00640 1 -- 172.21.15.87:0/1327294682 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d44055f30 con 0x7f1d5406b2c0 2024-04-25T10:12:56.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.203+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1d5406e460 con 0x7f1d5406b2c0 2024-04-25T10:12:56.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.203+0000 7f1d50c00640 1 -- 172.21.15.87:0/1327294682 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1d44055830 con 0x7f1d5406b2c0 2024-04-25T10:12:56.211 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d30066f90 msgr2=0x7f1d30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d30066f90 0x7f1d30069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0b2000 tx=0x7f1d3c0aa070 comp rx=0 tx=0).stop 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d5406b2c0 msgr2=0x7f1d54152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d5406b2c0 0x7f1d54152a20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400e870 tx=0x7f1d4400ed30 comp rx=0 tx=0).stop 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 shutdown_connections 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d30066f90 0x7f1d30069450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d5415db40 0x7f1d540ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d540b5010 0x7f1d540c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1327294682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d5406b2c0 0x7f1d54152a20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 >> 172.21.15.87:0/1327294682 conn(0x7f1d540d17b0 msgr2=0x7f1d540d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 shutdown_connections 2024-04-25T10:12:56.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.207+0000 7f1d59a00640 1 -- 172.21.15.87:0/1327294682 wait complete. 2024-04-25T10:12:56.232 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:12:56.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.232 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T10:12:56.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:12:56.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.271 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:12:56.332 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:12:56.332 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.342 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:12:56.342 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.394 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:12:56.394 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.446 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:56.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.575+0000 7f402b400640 1 Processor -- start 2024-04-25T10:12:56.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.575+0000 7f402b400640 1 -- start start 2024-04-25T10:12:56.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.575+0000 7f402b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40240d0040 0x7f402415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.575+0000 7f402b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f402415b740 0x7f40240b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.575+0000 7f402b400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4024071050 con 0x7f40241550b0 2024-04-25T10:12:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.575+0000 7f402b400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402415d040 con 0x7f40240d0040 2024-04-25T10:12:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.575+0000 7f402b400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40240712f0 con 0x7f402415b740 2024-04-25T10:12:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4028a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f40241550b0 0x7f40240cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40356/0 (socket says 172.21.15.87:40356) 2024-04-25T10:12:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4028a00640 1 -- 172.21.15.87:0/4207302088 learned_addr learned my addr 172.21.15.87:0/4207302088 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f401be00640 1 --2- 172.21.15.87:0/4207302088 >> v2:172.21.15.87:3300/0 conn(0x7f40240d0040 0x7f402415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4029400640 1 --2- 172.21.15.87:0/4207302088 >> v2:172.21.15.169:3300/0 conn(0x7f402415b740 0x7f40240b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4023e00640 1 -- 172.21.15.87:0/4207302088 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829442938 0 0) 0x7f4024071050 con 0x7f40241550b0 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4023e00640 1 -- 172.21.15.87:0/4207302088 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4008003530 con 0x7f40241550b0 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f401be00640 1 -- 172.21.15.87:0/4207302088 >> v2:172.21.15.169:3300/0 conn(0x7f402415b740 msgr2=0x7f40240b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f401be00640 1 --2- 172.21.15.87:0/4207302088 >> v2:172.21.15.169:3300/0 conn(0x7f402415b740 0x7f40240b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f401be00640 1 -- 172.21.15.87:0/4207302088 >> v1:172.21.15.87:6789/0 conn(0x7f40241550b0 legacy=0x7f40240cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f401be00640 1 -- 172.21.15.87:0/4207302088 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240715d0 con 0x7f40240d0040 2024-04-25T10:12:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f401be00640 1 --2- 172.21.15.87:0/4207302088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40240d0040 0x7f402415b200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40240d04d0 tx=0x7f400c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68bf6d1e3c69dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4023e00640 1 -- 172.21.15.87:0/4207302088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0ab650 con 0x7f40240d0040 2024-04-25T10:12:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4023e00640 1 -- 172.21.15.87:0/4207302088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c0b0070 con 0x7f40240d0040 2024-04-25T10:12:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.579+0000 7f4023e00640 1 -- 172.21.15.87:0/4207302088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0aba10 con 0x7f40240d0040 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- 172.21.15.87:0/4207302088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40240d0040 msgr2=0x7f402415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 --2- 172.21.15.87:0/4207302088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40240d0040 0x7f402415b200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40240d04d0 tx=0x7f400c01c450 comp rx=0 tx=0).stop 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- 172.21.15.87:0/4207302088 shutdown_connections 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 --2- 172.21.15.87:0/4207302088 >> v2:172.21.15.169:3300/0 conn(0x7f402415b740 0x7f40240b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 --2- 172.21.15.87:0/4207302088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40240d0040 0x7f402415b200 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- 172.21.15.87:0/4207302088 >> 172.21.15.87:0/4207302088 conn(0x7f4024066b70 msgr2=0x7f4024066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- 172.21.15.87:0/4207302088 shutdown_connections 2024-04-25T10:12:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- 172.21.15.87:0/4207302088 wait complete. 2024-04-25T10:12:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 Processor -- start 2024-04-25T10:12:56.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- start start 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40240d0040 0x7f40241506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40241550b0 0x7f402413e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f402415b740 0x7f40241541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402415bcd0 con 0x7f402415b740 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402415bb50 con 0x7f40240d0040 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f402b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402415c030 con 0x7f40241550b0 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f402415b740 0x7f40241541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f402415b740 0x7f40241541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40992/0 (socket says 172.21.15.87:40992) 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 -- 172.21.15.87:0/4271354494 learned_addr learned my addr 172.21.15.87:0/4271354494 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4028a00640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40240d0040 0x7f40241506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 -- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40241550b0 msgr2=0x7f402413e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40241550b0 0x7f402413e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 -- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40240d0040 msgr2=0x7f40241506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40240d0040 0x7f40241506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4029400640 1 -- 172.21.15.87:0/4271354494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402413e5d0 con 0x7f402415b740 2024-04-25T10:12:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.583+0000 7f4028a00640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40240d0040 0x7f40241506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4029400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f402415b740 0x7f40241541e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f401c007e60 tx=0x7f401c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4021600640 1 -- 172.21.15.87:0/4271354494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c014420 con 0x7f402415b740 2024-04-25T10:12:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4021600640 1 -- 172.21.15.87:0/4271354494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c019070 con 0x7f402415b740 2024-04-25T10:12:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402414f640 con 0x7f402415b740 2024-04-25T10:12:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4021600640 1 -- 172.21.15.87:0/4271354494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c014820 con 0x7f402415b740 2024-04-25T10:12:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402414fc00 con 0x7f402415b740 2024-04-25T10:12:56.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4024150c20 con 0x7f402415b740 2024-04-25T10:12:56.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4021600640 1 -- 172.21.15.87:0/4271354494 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f401c0149c0 con 0x7f402415b740 2024-04-25T10:12:56.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4021600640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4000066f60 0x7f4000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4021600640 1 -- 172.21.15.87:0/4271354494 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f401c085880 con 0x7f402415b740 2024-04-25T10:12:56.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4028a00640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4000066f60 0x7f4000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.587+0000 7f4028a00640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4000066f60 0x7f4000069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4014005890 tx=0x7f4014009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.591+0000 7f4021600640 1 -- 172.21.15.87:0/4271354494 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f401c056890 con 0x7f402415b740 2024-04-25T10:12:56.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.747+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40240cf400 con 0x7f402415b740 2024-04-25T10:12:56.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.747+0000 7f4021600640 1 -- 172.21.15.87:0/4271354494 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f401c056190 con 0x7f402415b740 2024-04-25T10:12:56.755 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:56.755 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4000066f60 msgr2=0x7f4000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4000066f60 0x7f4000069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4014005890 tx=0x7f4014009040 comp rx=0 tx=0).stop 2024-04-25T10:12:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f402415b740 msgr2=0x7f40241541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f402415b740 0x7f40241541e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f401c007e60 tx=0x7f401c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 shutdown_connections 2024-04-25T10:12:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4000066f60 0x7f4000069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f402415b740 0x7f40241541e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40241550b0 0x7f402413e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 --2- 172.21.15.87:0/4271354494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40240d0040 0x7f40241506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 >> 172.21.15.87:0/4271354494 conn(0x7f4024066b70 msgr2=0x7f40240b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 shutdown_connections 2024-04-25T10:12:56.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.751+0000 7f402b400640 1 -- 172.21.15.87:0/4271354494 wait complete. 2024-04-25T10:12:56.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:56.776 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:56.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.907+0000 7ff4fd800640 1 Processor -- start 2024-04-25T10:12:56.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.907+0000 7ff4fd800640 1 -- start start 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f8071260 0x7ff4f80716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fd800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4f8071be0 0x7ff4f80b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fd800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f806de60 con 0x7ff4f815a780 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fd800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f806cdc0 con 0x7ff4f8071be0 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f806d0a0 con 0x7ff4f8071260 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4f7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff4f815a780 0x7ff4f815edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40364/0 (socket says 172.21.15.87:40364) 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4f7e00640 1 -- 172.21.15.87:0/1070095905 learned_addr learned my addr 172.21.15.87:0/1070095905 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:56.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fce00640 1 --2- 172.21.15.87:0/1070095905 >> v2:172.21.15.169:3300/0 conn(0x7ff4f8071be0 0x7ff4f80b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4ef400640 1 --2- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f8071260 0x7ff4f80716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4f7400640 1 -- 172.21.15.87:0/1070095905 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687127847 0 0) 0x7ff4f806de60 con 0x7ff4f815a780 2024-04-25T10:12:56.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4f7400640 1 -- 172.21.15.87:0/1070095905 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4dc003530 con 0x7ff4f815a780 2024-04-25T10:12:56.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fce00640 1 -- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f8071260 msgr2=0x7ff4f80716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fce00640 1 --2- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f8071260 0x7ff4f80716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fce00640 1 -- 172.21.15.87:0/1070095905 >> v1:172.21.15.87:6789/0 conn(0x7ff4f815a780 legacy=0x7ff4f815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4f7400640 1 -- 172.21.15.87:0/1070095905 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2913674714 0 0) 0x7ff4dc003530 con 0x7ff4f815a780 2024-04-25T10:12:56.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.911+0000 7ff4fce00640 1 -- 172.21.15.87:0/1070095905 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f806d380 con 0x7ff4f8071be0 2024-04-25T10:12:56.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fce00640 1 --2- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 0x7ff4f80b2460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009d90 tx=0x7ff4f001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b6dfe0c76e3682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4f7400640 1 -- 172.21.15.87:0/1070095905 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00b2070 con 0x7ff4f8071be0 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4f7400640 1 -- 172.21.15.87:0/1070095905 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f00a5830 con 0x7ff4f8071be0 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4f7400640 1 -- 172.21.15.87:0/1070095905 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00a5be0 con 0x7ff4f8071be0 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 -- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 msgr2=0x7ff4f80b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 --2- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 0x7ff4f80b2460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009d90 tx=0x7ff4f001c970 comp rx=0 tx=0).stop 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 -- 172.21.15.87:0/1070095905 shutdown_connections 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 --2- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 0x7ff4f80b2460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 --2- 172.21.15.87:0/1070095905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f8071260 0x7ff4f80716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 -- 172.21.15.87:0/1070095905 >> 172.21.15.87:0/1070095905 conn(0x7ff4f80743f0 msgr2=0x7ff4f80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 -- 172.21.15.87:0/1070095905 shutdown_connections 2024-04-25T10:12:56.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 -- 172.21.15.87:0/1070095905 wait complete. 2024-04-25T10:12:56.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 Processor -- start 2024-04-25T10:12:56.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.915+0000 7ff4fd800640 1 -- start start 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f8071260 0x7ff4f80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 0x7ff4f8168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f815a780 0x7ff4f80cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80b3330 con 0x7ff4f8071260 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80b31b0 con 0x7ff4f8071be0 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80b34b0 con 0x7ff4f815a780 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f8071260 0x7ff4f80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f8071260 0x7ff4f80cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41006/0 (socket says 172.21.15.87:41006) 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 -- 172.21.15.87:0/642278134 learned_addr learned my addr 172.21.15.87:0/642278134 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4ef400640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 0x7ff4f8168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fce00640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f815a780 0x7ff4f80cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 -- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f815a780 msgr2=0x7ff4f80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f815a780 0x7ff4f80cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 -- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 msgr2=0x7ff4f8168ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 0x7ff4f8168ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 -- 172.21.15.87:0/642278134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f8167e00 con 0x7ff4f8071260 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f7e00640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f8071260 0x7ff4f80cc450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800c440 tx=0x7ff4e800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f4c00640 1 -- 172.21.15.87:0/642278134 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8005cf0 con 0x7ff4f8071260 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f4c00640 1 -- 172.21.15.87:0/642278134 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e8007070 con 0x7ff4f8071260 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f4c00640 1 -- 172.21.15.87:0/642278134 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8007460 con 0x7ff4f8071260 2024-04-25T10:12:56.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f81680e0 con 0x7ff4f8071260 2024-04-25T10:12:56.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f81686a0 con 0x7ff4f8071260 2024-04-25T10:12:56.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f8075120 con 0x7ff4f8071260 2024-04-25T10:12:56.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.919+0000 7ff4f4c00640 1 -- 172.21.15.87:0/642278134 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4e8007600 con 0x7ff4f8071260 2024-04-25T10:12:56.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.923+0000 7ff4f4c00640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4d4066f60 0x7ff4d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.923+0000 7ff4f4c00640 1 -- 172.21.15.87:0/642278134 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4e8084d80 con 0x7ff4f8071260 2024-04-25T10:12:56.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.923+0000 7ff4ef400640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4d4066f60 0x7ff4d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.923+0000 7ff4ef400640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4d4066f60 0x7ff4d4069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8066ce0 tx=0x7ff4e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:56.927+0000 7ff4f4c00640 1 -- 172.21.15.87:0/642278134 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4e8055cf0 con 0x7ff4f8071260 2024-04-25T10:12:57.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.079+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4f806c8d0 con 0x7ff4f8071260 2024-04-25T10:12:57.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4f4c00640 1 -- 172.21.15.87:0/642278134 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4e80555f0 con 0x7ff4f8071260 2024-04-25T10:12:57.088 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:57.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4d4066f60 msgr2=0x7ff4d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4d4066f60 0x7ff4d4069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8066ce0 tx=0x7ff4e0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:57.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f8071260 msgr2=0x7ff4f80cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f8071260 0x7ff4f80cc450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800c440 tx=0x7ff4e800c900 comp rx=0 tx=0).stop 2024-04-25T10:12:57.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 shutdown_connections 2024-04-25T10:12:57.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4d4066f60 0x7ff4d4069420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4f815a780 0x7ff4f80cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4f8071be0 0x7ff4f8168ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 --2- 172.21.15.87:0/642278134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f8071260 0x7ff4f80cc450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 >> 172.21.15.87:0/642278134 conn(0x7ff4f80743f0 msgr2=0x7ff4f8153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 shutdown_connections 2024-04-25T10:12:57.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.083+0000 7ff4fd800640 1 -- 172.21.15.87:0/642278134 wait complete. 2024-04-25T10:12:57.108 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:57.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2ce0e00640 1 Processor -- start 2024-04-25T10:12:57.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2ce0e00640 1 -- start start 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2ce0e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2cdc15a0e0 0x7f2cdc15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2ce0e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2cdc0aea50 0x7f2cdc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2ce0e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0b2fd0 con 0x7f2cdc0b42d0 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2ce0e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0af1d0 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2ce0e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0b32b0 con 0x7f2cdc0aea50 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cd9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2cdc15a0e0 0x7f2cdc15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cda800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2cdc0b42d0 0x7f2cdc06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40366/0 (socket says 172.21.15.87:40366) 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cda800640 1 -- 172.21.15.87:0/2429673677 learned_addr learned my addr 172.21.15.87:0/2429673677 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:57.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cd9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2cdc15a0e0 0x7f2cdc15b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41018/0 (socket says 172.21.15.87:41018) 2024-04-25T10:12:57.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cd1400640 1 --2- 172.21.15.87:0/2429673677 >> v2:172.21.15.169:3300/0 conn(0x7f2cdc0aea50 0x7f2cdc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cd9e00640 1 -- 172.21.15.87:0/2429673677 >> v2:172.21.15.169:3300/0 conn(0x7f2cdc0aea50 msgr2=0x7f2cdc0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cd9e00640 1 --2- 172.21.15.87:0/2429673677 >> v2:172.21.15.169:3300/0 conn(0x7f2cdc0aea50 0x7f2cdc0b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.239+0000 7f2cd9e00640 1 -- 172.21.15.87:0/2429673677 >> v1:172.21.15.87:6789/0 conn(0x7f2cdc0b42d0 legacy=0x7f2cdc06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2cd9e00640 1 -- 172.21.15.87:0/2429673677 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0b3590 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2cd9400640 1 -- 172.21.15.87:0/2429673677 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076981752 0 0) 0x7f2cdc0b2fd0 con 0x7f2cdc0b42d0 2024-04-25T10:12:57.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2cd1400640 1 --2- 172.21.15.87:0/2429673677 >> v2:172.21.15.169:3300/0 conn(0x7f2cdc0aea50 0x7f2cdc0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:57.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2cd9e00640 1 --2- 172.21.15.87:0/2429673677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc15b150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc15dae0 tx=0x7f2ccc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc04a31538f98b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2cd9400640 1 -- 172.21.15.87:0/2429673677 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0b0070 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2cd9400640 1 -- 172.21.15.87:0/2429673677 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0ab600 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2cd9400640 1 -- 172.21.15.87:0/2429673677 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0ab9d0 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 -- 172.21.15.87:0/2429673677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 msgr2=0x7f2cdc15b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/2429673677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc15b150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc15dae0 tx=0x7f2ccc01c360 comp rx=0 tx=0).stop 2024-04-25T10:12:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 -- 172.21.15.87:0/2429673677 shutdown_connections 2024-04-25T10:12:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/2429673677 >> v2:172.21.15.169:3300/0 conn(0x7f2cdc0aea50 0x7f2cdc0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/2429673677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc15b150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 -- 172.21.15.87:0/2429673677 >> 172.21.15.87:0/2429673677 conn(0x7f2cdc0669a0 msgr2=0x7f2cdc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 -- 172.21.15.87:0/2429673677 shutdown_connections 2024-04-25T10:12:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.243+0000 7f2ce0e00640 1 -- 172.21.15.87:0/2429673677 wait complete. 2024-04-25T10:12:57.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 Processor -- start 2024-04-25T10:12:57.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 -- start start 2024-04-25T10:12:57.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cdc0aea50 0x7f2cdc154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cdc0b42d0 0x7f2cdc149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1670c0 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc166f40 con 0x7f2cdc0b42d0 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc167240 con 0x7f2cdc0aea50 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc0b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41030/0 (socket says 172.21.15.87:41030) 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 -- 172.21.15.87:0/409005794 learned_addr learned my addr 172.21.15.87:0/409005794 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cd9e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cdc0aea50 0x7f2cdc154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cd1400640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cdc0b42d0 0x7f2cdc149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 -- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cdc0aea50 msgr2=0x7f2cdc154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cdc0aea50 0x7f2cdc154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 -- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cdc0b42d0 msgr2=0x7f2cdc149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cdc0b42d0 0x7f2cdc149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 -- 172.21.15.87:0/409005794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc1555d0 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cda800640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc0b1880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c7d0 tx=0x7f2cd400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cd2a00640 1 -- 172.21.15.87:0/409005794 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4018070 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cd2a00640 1 -- 172.21.15.87:0/409005794 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd400a640 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2cd2a00640 1 -- 172.21.15.87:0/409005794 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400aa70 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1558b0 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.247+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc14bf80 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.251+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc04ef30 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.251+0000 7f2cd2a00640 1 -- 172.21.15.87:0/409005794 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2cd401b070 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.251+0000 7f2cd2a00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2cb8066f90 0x7f2cb8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.251+0000 7f2cd2a00640 1 -- 172.21.15.87:0/409005794 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cd4084580 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.251+0000 7f2cd9e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2cb8066f90 0x7f2cb8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.251+0000 7f2cd9e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2cb8066f90 0x7f2cb8069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0ad000 tx=0x7f2ccc0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.255+0000 7f2cd2a00640 1 -- 172.21.15.87:0/409005794 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2cd4054f40 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cdc066f20 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2cd2a00640 1 -- 172.21.15.87:0/409005794 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cd4058fe0 con 0x7f2cdc15a0e0 2024-04-25T10:12:57.417 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2cb8066f90 msgr2=0x7f2cb8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2cb8066f90 0x7f2cb8069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0ad000 tx=0x7f2ccc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 msgr2=0x7f2cdc0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc0b1880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c7d0 tx=0x7f2cd400cc90 comp rx=0 tx=0).stop 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 shutdown_connections 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2cb8066f90 0x7f2cb8069450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cdc15a0e0 0x7f2cdc0b1880 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cdc0b42d0 0x7f2cdc149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 --2- 172.21.15.87:0/409005794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cdc0aea50 0x7f2cdc154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 >> 172.21.15.87:0/409005794 conn(0x7f2cdc0669a0 msgr2=0x7f2cdc15cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 shutdown_connections 2024-04-25T10:12:57.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.411+0000 7f2ce0e00640 1 -- 172.21.15.87:0/409005794 wait complete. 2024-04-25T10:12:57.438 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:57.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d84a00640 1 Processor -- start 2024-04-25T10:12:57.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d84a00640 1 -- start start 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d84a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d80159540 0x7f9d800ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d84a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9d800aebd0 0x7f9d80071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d84a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d80071360 con 0x7f9d80072020 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d84a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d80159920 con 0x7f9d800aebd0 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d84a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8015c2f0 con 0x7f9d80159540 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d75400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9d800aebd0 0x7f9d80071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d7e800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9d80072020 0x7f9d8015f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34766/0 (socket says 172.21.15.87:34766) 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d7e800640 1 -- 172.21.15.87:0/3430984498 learned_addr learned my addr 172.21.15.87:0/3430984498 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:57.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d7de00640 1 --2- 172.21.15.87:0/3430984498 >> v2:172.21.15.169:3300/0 conn(0x7f9d80159540 0x7f9d800ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d75400640 1 -- 172.21.15.87:0/3430984498 >> v2:172.21.15.169:3300/0 conn(0x7f9d80159540 msgr2=0x7f9d800ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d75400640 1 --2- 172.21.15.87:0/3430984498 >> v2:172.21.15.169:3300/0 conn(0x7f9d80159540 0x7f9d800ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d75400640 1 -- 172.21.15.87:0/3430984498 >> v1:172.21.15.87:6789/0 conn(0x7f9d80072020 legacy=0x7f9d8015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d75400640 1 -- 172.21.15.87:0/3430984498 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8015c5d0 con 0x7f9d800aebd0 2024-04-25T10:12:57.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.571+0000 7f9d7d400640 1 -- 172.21.15.87:0/3430984498 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024855762 0 0) 0x7f9d80071360 con 0x7f9d80072020 2024-04-25T10:12:57.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d75400640 1 --2- 172.21.15.87:0/3430984498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800aebd0 0x7f9d80071ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d800b4c20 tx=0x7f9d6801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8d2ebeebbca3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d7d400640 1 -- 172.21.15.87:0/3430984498 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680b0070 con 0x7f9d800aebd0 2024-04-25T10:12:57.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d7d400640 1 -- 172.21.15.87:0/3430984498 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d680ab600 con 0x7f9d800aebd0 2024-04-25T10:12:57.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d7d400640 1 -- 172.21.15.87:0/3430984498 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680ab9d0 con 0x7f9d800aebd0 2024-04-25T10:12:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 -- 172.21.15.87:0/3430984498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800aebd0 msgr2=0x7f9d80071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3430984498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800aebd0 0x7f9d80071ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d800b4c20 tx=0x7f9d6801c480 comp rx=0 tx=0).stop 2024-04-25T10:12:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 -- 172.21.15.87:0/3430984498 shutdown_connections 2024-04-25T10:12:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3430984498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800aebd0 0x7f9d80071ae0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3430984498 >> v2:172.21.15.169:3300/0 conn(0x7f9d80159540 0x7f9d800ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 -- 172.21.15.87:0/3430984498 >> 172.21.15.87:0/3430984498 conn(0x7f9d800669a0 msgr2=0x7f9d80066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 -- 172.21.15.87:0/3430984498 shutdown_connections 2024-04-25T10:12:57.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.575+0000 7f9d84a00640 1 -- 172.21.15.87:0/3430984498 wait complete. 2024-04-25T10:12:57.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 Processor -- start 2024-04-25T10:12:57.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 -- start start 2024-04-25T10:12:57.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d80072020 0x7f9d80152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d800aebd0 0x7f9d80157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d80159540 0x7f9d801580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80072430 con 0x7f9d80159540 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8006c0d0 con 0x7f9d800aebd0 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80168930 con 0x7f9d80072020 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d80159540 0x7f9d801580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d80159540 0x7f9d801580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54794/0 (socket says 172.21.15.87:54794) 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 -- 172.21.15.87:0/3700918564 learned_addr learned my addr 172.21.15.87:0/3700918564 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7de00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d80072020 0x7f9d80152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d75400640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d800aebd0 0x7f9d80157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 -- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d80072020 msgr2=0x7f9d80152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d80072020 0x7f9d80152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 -- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d800aebd0 msgr2=0x7f9d80157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d800aebd0 0x7f9d80157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 -- 172.21.15.87:0/3700918564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d801fa930 con 0x7f9d80159540 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d7e800640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d80159540 0x7f9d801580d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c7a0 tx=0x7f9d7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d76a00640 1 -- 172.21.15.87:0/3700918564 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78018070 con 0x7f9d80159540 2024-04-25T10:12:57.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d76a00640 1 -- 172.21.15.87:0/3700918564 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7800a640 con 0x7f9d80159540 2024-04-25T10:12:57.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d76a00640 1 -- 172.21.15.87:0/3700918564 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800aab0 con 0x7f9d80159540 2024-04-25T10:12:57.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d80155320 con 0x7f9d80159540 2024-04-25T10:12:57.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d801558e0 con 0x7f9d80159540 2024-04-25T10:12:57.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.579+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8004ef30 con 0x7f9d80159540 2024-04-25T10:12:57.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.583+0000 7f9d76a00640 1 -- 172.21.15.87:0/3700918564 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d7800ac50 con 0x7f9d80159540 2024-04-25T10:12:57.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.583+0000 7f9d76a00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f60 0x7f9d5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.583+0000 7f9d76a00640 1 -- 172.21.15.87:0/3700918564 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d78084670 con 0x7f9d80159540 2024-04-25T10:12:57.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.583+0000 7f9d7de00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f60 0x7f9d5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.583+0000 7f9d7de00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f60 0x7f9d5c069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9d700079c0 tx=0x7f9d70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.587+0000 7f9d76a00640 1 -- 172.21.15.87:0/3700918564 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d78054f90 con 0x7f9d80159540 2024-04-25T10:12:57.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.739+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d800cf470 con 0x7f9d80159540 2024-04-25T10:12:57.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.739+0000 7f9d76a00640 1 -- 172.21.15.87:0/3700918564 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d78059030 con 0x7f9d80159540 2024-04-25T10:12:57.746 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f60 msgr2=0x7f9d5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f60 0x7f9d5c069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9d700079c0 tx=0x7f9d70008040 comp rx=0 tx=0).stop 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d80159540 msgr2=0x7f9d801580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d80159540 0x7f9d801580d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c7a0 tx=0x7f9d7800cc60 comp rx=0 tx=0).stop 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 shutdown_connections 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f60 0x7f9d5c069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d80159540 0x7f9d801580d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d800aebd0 0x7f9d80157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3700918564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d80072020 0x7f9d80152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 >> 172.21.15.87:0/3700918564 conn(0x7f9d800669a0 msgr2=0x7f9d800cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 shutdown_connections 2024-04-25T10:12:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.743+0000 7f9d84a00640 1 -- 172.21.15.87:0/3700918564 wait complete. 2024-04-25T10:12:57.767 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa617800640 1 Processor -- start 2024-04-25T10:12:57.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa617800640 1 -- start start 2024-04-25T10:12:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa617800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa610156150 0x7fa610071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa617800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa610156530 con 0x7fa6100670b0 2024-04-25T10:12:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa617800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61015d220 con 0x7fa61015c080 2024-04-25T10:12:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa617800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61015c640 con 0x7fa610156150 2024-04-25T10:12:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa614e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa610156150 0x7fa610071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa615800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa6100670b0 0x7fa6100af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34768/0 (socket says 172.21.15.87:34768) 2024-04-25T10:12:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa615800640 1 -- 172.21.15.87:0/2012888489 learned_addr learned my addr 172.21.15.87:0/2012888489 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa614e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa610156150 0x7fa610071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54806/0 (socket says 172.21.15.87:54806) 2024-04-25T10:12:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa614e00640 1 -- 172.21.15.87:0/2012888489 >> v1:172.21.15.169:6789/0 conn(0x7fa61015c080 legacy=0x7fa6100669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa614e00640 1 -- 172.21.15.87:0/2012888489 >> v1:172.21.15.87:6789/0 conn(0x7fa6100670b0 legacy=0x7fa6100af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.899+0000 7fa614e00640 1 -- 172.21.15.87:0/2012888489 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61015c920 con 0x7fa610156150 2024-04-25T10:12:57.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa614e00640 1 --2- 172.21.15.87:0/2012888489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa610071f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa600009d90 tx=0x7fa60001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef41fb2deca2425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa60f400640 1 -- 172.21.15.87:0/2012888489 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000b2070 con 0x7fa610156150 2024-04-25T10:12:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa60f400640 1 -- 172.21.15.87:0/2012888489 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6000a5a00 con 0x7fa610156150 2024-04-25T10:12:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa60f400640 1 -- 172.21.15.87:0/2012888489 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000a5dd0 con 0x7fa610156150 2024-04-25T10:12:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa617800640 1 -- 172.21.15.87:0/2012888489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 msgr2=0x7fa610071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa617800640 1 --2- 172.21.15.87:0/2012888489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa610071f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa600009d90 tx=0x7fa60001c940 comp rx=0 tx=0).stop 2024-04-25T10:12:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa617800640 1 -- 172.21.15.87:0/2012888489 shutdown_connections 2024-04-25T10:12:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa617800640 1 --2- 172.21.15.87:0/2012888489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa610071f30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa617800640 1 -- 172.21.15.87:0/2012888489 >> 172.21.15.87:0/2012888489 conn(0x7fa61006ac90 msgr2=0x7fa61006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa617800640 1 -- 172.21.15.87:0/2012888489 shutdown_connections 2024-04-25T10:12:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.903+0000 7fa617800640 1 -- 172.21.15.87:0/2012888489 wait complete. 2024-04-25T10:12:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 Processor -- start 2024-04-25T10:12:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 -- start start 2024-04-25T10:12:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100670b0 0x7fa610140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa6101531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61015c080 0x7fa6101536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa610169830 con 0x7fa610156150 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6101696b0 con 0x7fa61015c080 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6101699b0 con 0x7fa6100670b0 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa6101531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa6101531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54808/0 (socket says 172.21.15.87:54808) 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 -- 172.21.15.87:0/2524610134 learned_addr learned my addr 172.21.15.87:0/2524610134 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa615800640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61015c080 0x7fa6101536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa614e00640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100670b0 0x7fa610140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 -- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100670b0 msgr2=0x7fa610140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100670b0 0x7fa610140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 -- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61015c080 msgr2=0x7fa6101536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61015c080 0x7fa6101536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 -- 172.21.15.87:0/2524610134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61014fb40 con 0x7fa610156150 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60fe00640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa6101531b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c370 tx=0x7fa5fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60cc00640 1 -- 172.21.15.87:0/2524610134 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0052d0 con 0x7fa610156150 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60cc00640 1 -- 172.21.15.87:0/2524610134 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc005470 con 0x7fa610156150 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa60cc00640 1 -- 172.21.15.87:0/2524610134 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc007260 con 0x7fa610156150 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61014fe20 con 0x7fa610156150 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6101503e0 con 0x7fa610156150 2024-04-25T10:12:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.907+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61004ef30 con 0x7fa610156150 2024-04-25T10:12:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.911+0000 7fa60cc00640 1 -- 172.21.15.87:0/2524610134 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5fc007400 con 0x7fa610156150 2024-04-25T10:12:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.911+0000 7fa60cc00640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.911+0000 7fa60cc00640 1 -- 172.21.15.87:0/2524610134 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5fc0844c0 con 0x7fa610156150 2024-04-25T10:12:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.911+0000 7fa614e00640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.911+0000 7fa614e00640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6000af000 tx=0x7fa6000a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:57.915+0000 7fa60cc00640 1 -- 172.21.15.87:0/2524610134 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5fc054de0 con 0x7fa610156150 2024-04-25T10:12:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.079+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa6100b1e70 con 0x7fa610156150 2024-04-25T10:12:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.079+0000 7fa60cc00640 1 -- 172.21.15.87:0/2524610134 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa5fc058e80 con 0x7fa610156150 2024-04-25T10:12:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 msgr2=0x7fa5f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6000af000 tx=0x7fa6000a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 msgr2=0x7fa6101531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa6101531b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c370 tx=0x7fa5fc00c830 comp rx=0 tx=0).stop 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 shutdown_connections 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61015c080 0x7fa6101536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa610156150 0x7fa6101531b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 --2- 172.21.15.87:0/2524610134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100670b0 0x7fa610140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 >> 172.21.15.87:0/2524610134 conn(0x7fa61006ac90 msgr2=0x7fa61015d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 shutdown_connections 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.083+0000 7fa617800640 1 -- 172.21.15.87:0/2524610134 wait complete. 2024-04-25T10:12:58.108 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T10:12:58.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:58.108 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T10:12:58.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:58.145 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:12:58.202 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:58.202 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:58.211 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:58.211 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:58.259 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:12:58.259 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:58.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:58.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 Processor -- start 2024-04-25T10:12:58.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 -- start start 2024-04-25T10:12:58.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e800cdb60 0x7f7e8006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e800cbdf0 0x7f7e8006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cec70 0x7f7e8006bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e8006b3e0 con 0x7f7e800cdb60 2024-04-25T10:12:58.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e800cdf40 con 0x7f7e800cbdf0 2024-04-25T10:12:58.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e85e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b1230 con 0x7f7e800cec70 2024-04-25T10:12:58.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e84a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e800cdb60 0x7f7e8006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e84a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e800cdb60 0x7f7e8006aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54814/0 (socket says 172.21.15.87:54814) 2024-04-25T10:12:58.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.479+0000 7f7e84a00640 1 -- 172.21.15.87:0/4198769887 learned_addr learned my addr 172.21.15.87:0/4198769887 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:58.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e77e00640 1 --2- 172.21.15.87:0/4198769887 >> v2:172.21.15.169:3300/0 conn(0x7f7e800cbdf0 0x7f7e8006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e85400640 1 --2- 172.21.15.87:0/4198769887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cec70 0x7f7e8006bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e84a00640 1 -- 172.21.15.87:0/4198769887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cec70 msgr2=0x7f7e8006bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e84a00640 1 --2- 172.21.15.87:0/4198769887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cec70 0x7f7e8006bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e84a00640 1 -- 172.21.15.87:0/4198769887 >> v2:172.21.15.169:3300/0 conn(0x7f7e800cbdf0 msgr2=0x7f7e8006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e84a00640 1 --2- 172.21.15.87:0/4198769887 >> v2:172.21.15.169:3300/0 conn(0x7f7e800cbdf0 0x7f7e8006b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e84a00640 1 -- 172.21.15.87:0/4198769887 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e800b1510 con 0x7f7e800cdb60 2024-04-25T10:12:58.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e84a00640 1 --2- 172.21.15.87:0/4198769887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e8006aea0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7e80071c30 tx=0x7f7e7001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2e164758bbd876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e7fe00640 1 -- 172.21.15.87:0/4198769887 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700aa610 con 0x7f7e800cdb60 2024-04-25T10:12:58.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e7fe00640 1 -- 172.21.15.87:0/4198769887 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e700af070 con 0x7f7e800cdb60 2024-04-25T10:12:58.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.483+0000 7f7e7fe00640 1 -- 172.21.15.87:0/4198769887 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700aa9d0 con 0x7f7e800cdb60 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- 172.21.15.87:0/4198769887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 msgr2=0x7f7e8006aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 --2- 172.21.15.87:0/4198769887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e8006aea0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7e80071c30 tx=0x7f7e7001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- 172.21.15.87:0/4198769887 shutdown_connections 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 --2- 172.21.15.87:0/4198769887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cec70 0x7f7e8006bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 --2- 172.21.15.87:0/4198769887 >> v2:172.21.15.169:3300/0 conn(0x7f7e800cbdf0 0x7f7e8006b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 --2- 172.21.15.87:0/4198769887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e8006aea0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- 172.21.15.87:0/4198769887 >> 172.21.15.87:0/4198769887 conn(0x7f7e800d17a0 msgr2=0x7f7e800d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- 172.21.15.87:0/4198769887 shutdown_connections 2024-04-25T10:12:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- 172.21.15.87:0/4198769887 wait complete. 2024-04-25T10:12:58.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 Processor -- start 2024-04-25T10:12:58.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- start start 2024-04-25T10:12:58.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cbdf0 0x7f7e8006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e800c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800cec70 0x7f7e80159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b2c60 con 0x7f7e800cdb60 2024-04-25T10:12:58.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b2ae0 con 0x7f7e800cec70 2024-04-25T10:12:58.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b2de0 con 0x7f7e800cbdf0 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e800c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e800c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54828/0 (socket says 172.21.15.87:54828) 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 -- 172.21.15.87:0/504416795 learned_addr learned my addr 172.21.15.87:0/504416795 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e84a00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cbdf0 0x7f7e8006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e85400640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800cec70 0x7f7e80159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 -- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cbdf0 msgr2=0x7f7e8006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cbdf0 0x7f7e8006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 -- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800cec70 msgr2=0x7f7e80159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800cec70 0x7f7e80159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 -- 172.21.15.87:0/504416795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e80067da0 con 0x7f7e800cdb60 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.487+0000 7f7e77e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e800c79a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d800 tx=0x7f7e6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e7d600640 1 -- 172.21.15.87:0/504416795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68018020 con 0x7f7e800cdb60 2024-04-25T10:12:58.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e7d600640 1 -- 172.21.15.87:0/504416795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e680096a0 con 0x7f7e800cdb60 2024-04-25T10:12:58.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e7d600640 1 -- 172.21.15.87:0/504416795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68009a50 con 0x7f7e800cdb60 2024-04-25T10:12:58.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e800c6600 con 0x7f7e800cdb60 2024-04-25T10:12:58.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e800c6bc0 con 0x7f7e800cdb60 2024-04-25T10:12:58.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e80075120 con 0x7f7e800cdb60 2024-04-25T10:12:58.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e7d600640 1 -- 172.21.15.87:0/504416795 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e6801b020 con 0x7f7e800cdb60 2024-04-25T10:12:58.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e7d600640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e5c066f60 0x7f7e5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e7d600640 1 -- 172.21.15.87:0/504416795 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e68085540 con 0x7f7e800cdb60 2024-04-25T10:12:58.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.491+0000 7f7e84a00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e5c066f60 0x7f7e5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.495+0000 7f7e84a00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e5c066f60 0x7f7e5c069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7e700ae000 tx=0x7f7e700a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.495+0000 7f7e7d600640 1 -- 172.21.15.87:0/504416795 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e68022020 con 0x7f7e800cdb60 2024-04-25T10:12:58.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.655+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e800c6fc0 con 0x7f7e800cdb60 2024-04-25T10:12:58.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.655+0000 7f7e7d600640 1 -- 172.21.15.87:0/504416795 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f7e6805ae20 con 0x7f7e800cdb60 2024-04-25T10:12:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:12:58.663 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:58.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e5c066f60 msgr2=0x7f7e5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e5c066f60 0x7f7e5c069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7e700ae000 tx=0x7f7e700a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 msgr2=0x7f7e800c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e800c79a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d800 tx=0x7f7e6800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 shutdown_connections 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e5c066f60 0x7f7e5c069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800cec70 0x7f7e80159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e800cdb60 0x7f7e800c79a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 --2- 172.21.15.87:0/504416795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e800cbdf0 0x7f7e8006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 >> 172.21.15.87:0/504416795 conn(0x7f7e800d17a0 msgr2=0x7f7e800b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 shutdown_connections 2024-04-25T10:12:58.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.659+0000 7f7e85e00640 1 -- 172.21.15.87:0/504416795 wait complete. 2024-04-25T10:12:58.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:58.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e2a00640 1 Processor -- start 2024-04-25T10:12:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e2a00640 1 -- start start 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31dc0cf980 0x7f31dc0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc0d1830 0x7f31dc0d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e2a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc15fec0 con 0x7f31dc15bc10 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e2a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31dc1601a0 con 0x7f31dc0cf980 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0cc5b0 con 0x7f31dc0d1830 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31db400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31dc0cf980 0x7f31dc0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31dbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f31dc15bc10 0x7f31dc15f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34776/0 (socket says 172.21.15.87:34776) 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31db400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31dc0cf980 0x7f31dc0b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54836/0 (socket says 172.21.15.87:54836) 2024-04-25T10:12:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31dbe00640 1 -- 172.21.15.87:0/3136435603 learned_addr learned my addr 172.21.15.87:0/3136435603 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31e0a00640 1 --2- 172.21.15.87:0/3136435603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc0d1830 0x7f31dc0d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31db400640 1 -- 172.21.15.87:0/3136435603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc0d1830 msgr2=0x7f31dc0d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31db400640 1 --2- 172.21.15.87:0/3136435603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc0d1830 0x7f31dc0d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31db400640 1 -- 172.21.15.87:0/3136435603 >> v1:172.21.15.87:6789/0 conn(0x7f31dc15bc10 legacy=0x7f31dc15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31db400640 1 -- 172.21.15.87:0/3136435603 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc0cc890 con 0x7f31dc0cf980 2024-04-25T10:12:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.815+0000 7f31daa00640 1 -- 172.21.15.87:0/3136435603 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839949216 0 0) 0x7f31dc15fec0 con 0x7f31dc15bc10 2024-04-25T10:12:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31db400640 1 --2- 172.21.15.87:0/3136435603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0cf980 0x7f31dc0b1060 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfc7a58a9990237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31daa00640 1 -- 172.21.15.87:0/3136435603 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80b0070 con 0x7f31dc0cf980 2024-04-25T10:12:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31daa00640 1 -- 172.21.15.87:0/3136435603 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c80ab600 con 0x7f31dc0cf980 2024-04-25T10:12:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31daa00640 1 -- 172.21.15.87:0/3136435603 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80ab9d0 con 0x7f31dc0cf980 2024-04-25T10:12:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 -- 172.21.15.87:0/3136435603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0cf980 msgr2=0x7f31dc0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 --2- 172.21.15.87:0/3136435603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0cf980 0x7f31dc0b1060 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 -- 172.21.15.87:0/3136435603 shutdown_connections 2024-04-25T10:12:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 --2- 172.21.15.87:0/3136435603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc0d1830 0x7f31dc0d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 --2- 172.21.15.87:0/3136435603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0cf980 0x7f31dc0b1060 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 -- 172.21.15.87:0/3136435603 >> 172.21.15.87:0/3136435603 conn(0x7f31dc09a9d0 msgr2=0x7f31dc09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 -- 172.21.15.87:0/3136435603 shutdown_connections 2024-04-25T10:12:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 -- 172.21.15.87:0/3136435603 wait complete. 2024-04-25T10:12:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.819+0000 7f31e2a00640 1 Processor -- start 2024-04-25T10:12:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 -- start start 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31dc0cf980 0x7f31dc152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0d1830 0x7f31dc0c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc15bc10 0x7f31dc15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0cd5c0 con 0x7f31dc0d1830 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0b17b0 con 0x7f31dc0cf980 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0cd740 con 0x7f31dc15bc10 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0d1830 0x7f31dc0c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0d1830 0x7f31dc0c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54840/0 (socket says 172.21.15.87:54840) 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 -- 172.21.15.87:0/31586256 learned_addr learned my addr 172.21.15.87:0/31586256 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31dbe00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31dc0cf980 0x7f31dc152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e0a00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc15bc10 0x7f31dc15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 -- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc15bc10 msgr2=0x7f31dc15e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc15bc10 0x7f31dc15e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 -- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31dc0cf980 msgr2=0x7f31dc152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31dc0cf980 0x7f31dc152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 -- 172.21.15.87:0/31586256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc15ce40 con 0x7f31dc0d1830 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31db400640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0d1830 0x7f31dc0c54d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31d3e00640 1 -- 172.21.15.87:0/31586256 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80b0050 con 0x7f31dc0d1830 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31d3e00640 1 -- 172.21.15.87:0/31586256 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c80b1040 con 0x7f31dc0d1830 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31d3e00640 1 -- 172.21.15.87:0/31586256 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8006e20 con 0x7f31dc0d1830 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc15eb70 con 0x7f31dc0d1830 2024-04-25T10:12:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc0b5980 con 0x7f31dc0d1830 2024-04-25T10:12:58.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.823+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31dc04ef30 con 0x7f31dc0d1830 2024-04-25T10:12:58.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.827+0000 7f31d3e00640 1 -- 172.21.15.87:0/31586256 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31c80be060 con 0x7f31dc0d1830 2024-04-25T10:12:58.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.827+0000 7f31d3e00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f31bc066f60 0x7f31bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.827+0000 7f31d3e00640 1 -- 172.21.15.87:0/31586256 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31c811e960 con 0x7f31dc0d1830 2024-04-25T10:12:58.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.827+0000 7f31dbe00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f31bc066f60 0x7f31bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.827+0000 7f31dbe00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f31bc066f60 0x7f31bc069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007fd0 tx=0x7f31cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.831+0000 7f31d3e00640 1 -- 172.21.15.87:0/31586256 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31c8016090 con 0x7f31dc0d1830 2024-04-25T10:12:58.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.983+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31dc0b7920 con 0x7f31dc0d1830 2024-04-25T10:12:58.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.983+0000 7f31d3e00640 1 -- 172.21.15.87:0/31586256 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31c80ef8d0 con 0x7f31dc0d1830 2024-04-25T10:12:58.991 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:12:58.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f31bc066f60 msgr2=0x7f31bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f31bc066f60 0x7f31bc069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007fd0 tx=0x7f31cc009040 comp rx=0 tx=0).stop 2024-04-25T10:12:58.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0d1830 msgr2=0x7f31dc0c54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0d1830 0x7f31dc0c54d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c801ace0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 shutdown_connections 2024-04-25T10:12:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f31bc066f60 0x7f31bc069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31dc15bc10 0x7f31dc15e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31dc0d1830 0x7f31dc0c54d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 --2- 172.21.15.87:0/31586256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31dc0cf980 0x7f31dc152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 >> 172.21.15.87:0/31586256 conn(0x7f31dc09a9d0 msgr2=0x7f31dc154920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 shutdown_connections 2024-04-25T10:12:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:58.987+0000 7f31e2a00640 1 -- 172.21.15.87:0/31586256 wait complete. 2024-04-25T10:12:59.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:59.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.143+0000 7f3283e00640 1 Processor -- start 2024-04-25T10:12:59.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.143+0000 7f3283e00640 1 -- start start 2024-04-25T10:12:59.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f32840b1f50 0x7f32840b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 0x7f32840cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328406be90 con 0x7f32840b3dd0 2024-04-25T10:12:59.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32840d0b50 con 0x7f32840b1f50 2024-04-25T10:12:59.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328406b680 con 0x7f32840cd780 2024-04-25T10:12:59.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3282000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f32840b3dd0 0x7f328406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34784/0 (socket says 172.21.15.87:34784) 2024-04-25T10:12:59.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3282000640 1 -- 172.21.15.87:0/2065955574 learned_addr learned my addr 172.21.15.87:0/2065955574 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283400640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 0x7f32840cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3282a00640 1 --2- 172.21.15.87:0/2065955574 >> v2:172.21.15.169:3300/0 conn(0x7f32840b1f50 0x7f32840b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048302198 0 0) 0x7f3270001120 con 0x7f32840b3dd0 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3268003530 con 0x7f32840b3dd0 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3053528788 0 0) 0x7f3268003530 con 0x7f32840b3dd0 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 msgr2=0x7f32840cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 0x7f32840cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 >> v2:172.21.15.169:3300/0 conn(0x7f32840b1f50 msgr2=0x7f32840b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 --2- 172.21.15.87:0/2065955574 >> v2:172.21.15.169:3300/0 conn(0x7f32840b1f50 0x7f32840b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328406b8d0 con 0x7f32840b3dd0 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3282a00640 1 --2- 172.21.15.87:0/2065955574 >> v2:172.21.15.169:3300/0 conn(0x7f32840b1f50 0x7f32840b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:59.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f3270001850 con 0x7f32840b3dd0 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 >> v1:172.21.15.87:6789/0 conn(0x7f32840b3dd0 legacy=0x7f328406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 0x7f3268003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268003fb0 0x7f3268004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3268004d50 0x7f32680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328406b680 con 0x7f3268004d50 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326800bab0 con 0x7f3268003fb0 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326800bcb0 con 0x7f32840cd780 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283400640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3268004d50 0x7f32680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3282000640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268003fb0 0x7f3268004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3282a00640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 0x7f3268003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283400640 1 -- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 msgr2=0x7f3268003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283400640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 0x7f3268003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283400640 1 -- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268003fb0 msgr2=0x7f3268004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283400640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268003fb0 0x7f3268004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3283400640 1 -- 172.21.15.87:0/2065955574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268003530 con 0x7f3268004d50 2024-04-25T10:12:59.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.147+0000 7f3282a00640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840cd780 0x7f3268003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:59.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283400640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3268004d50 0x7f32680051d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f327c009fc0 tx=0x7f327c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224bcadd32b8615c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0b2070 con 0x7f3268004d50 2024-04-25T10:12:59.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c0a5c70 con 0x7f3268004d50 2024-04-25T10:12:59.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3281600640 1 -- 172.21.15.87:0/2065955574 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c009c10 con 0x7f3268004d50 2024-04-25T10:12:59.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 -- 172.21.15.87:0/2065955574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3268004d50 msgr2=0x7f32680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 --2- 172.21.15.87:0/2065955574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3268004d50 0x7f32680051d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f327c009fc0 tx=0x7f327c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:12:59.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283400640 1 -- 172.21.15.87:0/2065955574 reap_dead start 2024-04-25T10:12:59.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 -- 172.21.15.87:0/2065955574 shutdown_connections 2024-04-25T10:12:59.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 -- 172.21.15.87:0/2065955574 >> 172.21.15.87:0/2065955574 conn(0x7f32840b32b0 msgr2=0x7f32840b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 -- 172.21.15.87:0/2065955574 shutdown_connections 2024-04-25T10:12:59.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 -- 172.21.15.87:0/2065955574 wait complete. 2024-04-25T10:12:59.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 Processor -- start 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.151+0000 7f3283e00640 1 -- start start 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268004d50 0x7f32840c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f328415cac0 0x7f328415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32840cca80 0x7f32840ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840ce0f0 con 0x7f32840cca80 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840cdf70 con 0x7f3268004d50 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840d0240 con 0x7f328415cac0 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32840cca80 0x7f32840ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32840cca80 0x7f32840ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54852/0 (socket says 172.21.15.87:54852) 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 -- 172.21.15.87:0/1763591468 learned_addr learned my addr 172.21.15.87:0/1763591468 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3282a00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268004d50 0x7f32840c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3282000640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f328415cac0 0x7f328415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 -- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f328415cac0 msgr2=0x7f328415cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f328415cac0 0x7f328415cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 -- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268004d50 msgr2=0x7f32840c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268004d50 0x7f32840c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 -- 172.21.15.87:0/1763591468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840c6c50 con 0x7f32840cca80 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283400640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32840cca80 0x7f32840ccf10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f327c002240 tx=0x7f327c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f327aa00640 1 -- 172.21.15.87:0/1763591468 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0bd070 con 0x7f32840cca80 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f327aa00640 1 -- 172.21.15.87:0/1763591468 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c0a5a70 con 0x7f32840cca80 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32840c6f30 con 0x7f32840cca80 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f327aa00640 1 -- 172.21.15.87:0/1763591468 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0b2040 con 0x7f32840cca80 2024-04-25T10:12:59.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32840c74f0 con 0x7f32840cca80 2024-04-25T10:12:59.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284075120 con 0x7f32840cca80 2024-04-25T10:12:59.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.155+0000 7f327aa00640 1 -- 172.21.15.87:0/1763591468 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f327c0c0070 con 0x7f32840cca80 2024-04-25T10:12:59.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.159+0000 7f327aa00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3264066f90 0x7f3264069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.159+0000 7f327aa00640 1 -- 172.21.15.87:0/1763591468 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f327c11ec50 con 0x7f32840cca80 2024-04-25T10:12:59.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.159+0000 7f3282a00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3264066f90 0x7f3264069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.159+0000 7f3282a00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3264066f90 0x7f3264069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f32740093f0 tx=0x7f3274008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.163+0000 7f327aa00640 1 -- 172.21.15.87:0/1763591468 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f327c0a5e20 con 0x7f32840cca80 2024-04-25T10:12:59.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.315+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32840b72c0 con 0x7f32840cca80 2024-04-25T10:12:59.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.315+0000 7f327aa00640 1 -- 172.21.15.87:0/1763591468 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f327c016990 con 0x7f32840cca80 2024-04-25T10:12:59.323 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:12:59.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3264066f90 msgr2=0x7f3264069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3264066f90 0x7f3264069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f32740093f0 tx=0x7f3274008040 comp rx=0 tx=0).stop 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32840cca80 msgr2=0x7f32840ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32840cca80 0x7f32840ccf10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f327c002240 tx=0x7f327c009750 comp rx=0 tx=0).stop 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 shutdown_connections 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3264066f90 0x7f3264069450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32840cca80 0x7f32840ccf10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f328415cac0 0x7f328415cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 --2- 172.21.15.87:0/1763591468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3268004d50 0x7f32840c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 >> 172.21.15.87:0/1763591468 conn(0x7f32840b32b0 msgr2=0x7f32840ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 shutdown_connections 2024-04-25T10:12:59.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.319+0000 7f3283e00640 1 -- 172.21.15.87:0/1763591468 wait complete. 2024-04-25T10:12:59.344 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:59.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faedb600640 1 Processor -- start 2024-04-25T10:12:59.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faedb600640 1 -- start start 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faedb600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faed4159540 0x7faed40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faedb600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faed40721f0 0x7faed4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faedb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed4159920 con 0x7faed40aebd0 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faedb600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed415c2f0 con 0x7faed4159540 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faedb600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed415c5d0 con 0x7faed40721f0 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faed8c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faed4159540 0x7faed40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faed8c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faed4159540 0x7faed40ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54860/0 (socket says 172.21.15.87:54860) 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faed8c00640 1 -- 172.21.15.87:0/1971262619 learned_addr learned my addr 172.21.15.87:0/1971262619 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.475+0000 7faed9600640 1 --2- 172.21.15.87:0/1971262619 >> v2:172.21.15.169:3300/0 conn(0x7faed40721f0 0x7faed4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed8c00640 1 -- 172.21.15.87:0/1971262619 >> v2:172.21.15.169:3300/0 conn(0x7faed40721f0 msgr2=0x7faed4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed8c00640 1 --2- 172.21.15.87:0/1971262619 >> v2:172.21.15.169:3300/0 conn(0x7faed40721f0 0x7faed4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed8c00640 1 -- 172.21.15.87:0/1971262619 >> v1:172.21.15.169:6789/0 conn(0x7faed40aebd0 legacy=0x7faed4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed8c00640 1 -- 172.21.15.87:0/1971262619 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed415c8b0 con 0x7faed4159540 2024-04-25T10:12:59.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed8c00640 1 --2- 172.21.15.87:0/1971262619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed40ae690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faed40cd200 tx=0x7faec401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb6c40ac729b0b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed3e00640 1 -- 172.21.15.87:0/1971262619 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec40ab430 con 0x7faed4159540 2024-04-25T10:12:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed3e00640 1 -- 172.21.15.87:0/1971262619 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec40b0070 con 0x7faed4159540 2024-04-25T10:12:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faed3e00640 1 -- 172.21.15.87:0/1971262619 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec40ab7f0 con 0x7faed4159540 2024-04-25T10:12:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faedb600640 1 -- 172.21.15.87:0/1971262619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 msgr2=0x7faed40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faedb600640 1 --2- 172.21.15.87:0/1971262619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed40ae690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faed40cd200 tx=0x7faec401c520 comp rx=0 tx=0).stop 2024-04-25T10:12:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faedb600640 1 -- 172.21.15.87:0/1971262619 shutdown_connections 2024-04-25T10:12:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faedb600640 1 --2- 172.21.15.87:0/1971262619 >> v2:172.21.15.169:3300/0 conn(0x7faed40721f0 0x7faed4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faedb600640 1 --2- 172.21.15.87:0/1971262619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed40ae690 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faedb600640 1 -- 172.21.15.87:0/1971262619 >> 172.21.15.87:0/1971262619 conn(0x7faed40669a0 msgr2=0x7faed4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.479+0000 7faedb600640 1 -- 172.21.15.87:0/1971262619 shutdown_connections 2024-04-25T10:12:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 -- 172.21.15.87:0/1971262619 wait complete. 2024-04-25T10:12:59.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 Processor -- start 2024-04-25T10:12:59.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 -- start start 2024-04-25T10:12:59.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed40721f0 0x7faed41faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed40aebd0 0x7faed414d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed4149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed4071220 con 0x7faed4159540 2024-04-25T10:12:59.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40710a0 con 0x7faed40aebd0 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40713a0 con 0x7faed40721f0 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed4149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed4149d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54864/0 (socket says 172.21.15.87:54864) 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed40721f0 0x7faed41faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 -- 172.21.15.87:0/3971361663 learned_addr learned my addr 172.21.15.87:0/3971361663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faecbe00640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed40aebd0 0x7faed414d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 -- 172.21.15.87:0/3971361663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed40721f0 msgr2=0x7faed41faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed40721f0 0x7faed41faf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 -- 172.21.15.87:0/3971361663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed40aebd0 msgr2=0x7faed414d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed40aebd0 0x7faed414d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 -- 172.21.15.87:0/3971361663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed41fb4b0 con 0x7faed4159540 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed9600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed4149d70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faecc007e40 tx=0x7faecc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed1600640 1 -- 172.21.15.87:0/3971361663 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc014620 con 0x7faed4159540 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed1600640 1 -- 172.21.15.87:0/3971361663 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc019070 con 0x7faed4159540 2024-04-25T10:12:59.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faed1600640 1 -- 172.21.15.87:0/3971361663 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc014a20 con 0x7faed4159540 2024-04-25T10:12:59.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.483+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed414edb0 con 0x7faed4159540 2024-04-25T10:12:59.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.487+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed41fa770 con 0x7faed4159540 2024-04-25T10:12:59.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.487+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed404ef30 con 0x7faed4159540 2024-04-25T10:12:59.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.487+0000 7faed1600640 1 -- 172.21.15.87:0/3971361663 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faecc014bc0 con 0x7faed4159540 2024-04-25T10:12:59.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.487+0000 7faed1600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeb0066f60 0x7faeb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.487+0000 7faed1600640 1 -- 172.21.15.87:0/3971361663 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faecc0857b0 con 0x7faed4159540 2024-04-25T10:12:59.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.487+0000 7faed8c00640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeb0066f60 0x7faeb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.487+0000 7faed8c00640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeb0066f60 0x7faeb0069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faec40af000 tx=0x7faec40a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.491+0000 7faed1600640 1 -- 172.21.15.87:0/3971361663 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faed41fb4b0 con 0x7faed4159540 2024-04-25T10:12:59.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faed4166700 con 0x7faed4159540 2024-04-25T10:12:59.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faed1600640 1 -- 172.21.15.87:0/3971361663 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7faecc056160 con 0x7faed4159540 2024-04-25T10:12:59.652 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:12:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeb0066f60 msgr2=0x7faeb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeb0066f60 0x7faeb0069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faec40af000 tx=0x7faec40a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 msgr2=0x7faed4149d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed4149d70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faecc007e40 tx=0x7faecc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 shutdown_connections 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faeb0066f60 0x7faeb0069420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faed4159540 0x7faed4149d70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faed40aebd0 0x7faed414d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 --2- 172.21.15.87:0/3971361663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faed40721f0 0x7faed41faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 >> 172.21.15.87:0/3971361663 conn(0x7faed40669a0 msgr2=0x7faed415f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 shutdown_connections 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.647+0000 7faedb600640 1 -- 172.21.15.87:0/3971361663 wait complete. 2024-04-25T10:12:59.673 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.803+0000 7fe9ab800640 1 Processor -- start 2024-04-25T10:12:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.803+0000 7fe9ab800640 1 -- start start 2024-04-25T10:12:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cda10 0x7fe9a406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9ab800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9a40cf930 0x7fe9a40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9ab800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a40b2770 con 0x7fe9a415bc40 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9ab800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a406c670 con 0x7fe9a40cf930 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9ab800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a406c950 con 0x7fe9a40cda10 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a9800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9a40cf930 0x7fe9a40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a9800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9a40cf930 0x7fe9a40cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54876/0 (socket says 172.21.15.87:54876) 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a9800640 1 -- 172.21.15.87:0/3862354033 learned_addr learned my addr 172.21.15.87:0/3862354033 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a3e00640 1 --2- 172.21.15.87:0/3862354033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cda10 0x7fe9a406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a9800640 1 -- 172.21.15.87:0/3862354033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cda10 msgr2=0x7fe9a406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a9800640 1 --2- 172.21.15.87:0/3862354033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cda10 0x7fe9a406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a9800640 1 -- 172.21.15.87:0/3862354033 >> v1:172.21.15.169:6789/0 conn(0x7fe9a415bc40 legacy=0x7fe9a40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.807+0000 7fe9a9800640 1 -- 172.21.15.87:0/3862354033 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a406cc30 con 0x7fe9a40cf930 2024-04-25T10:12:59.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9a9800640 1 --2- 172.21.15.87:0/3862354033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cf930 0x7fe9a40cfde0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009870 tx=0x7fe99c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdd3a6b0cdb589e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9a3400640 1 -- 172.21.15.87:0/3862354033 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0b0070 con 0x7fe9a40cf930 2024-04-25T10:12:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9a3400640 1 -- 172.21.15.87:0/3862354033 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c0ab600 con 0x7fe9a40cf930 2024-04-25T10:12:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9a3400640 1 -- 172.21.15.87:0/3862354033 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0ab9d0 con 0x7fe9a40cf930 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- 172.21.15.87:0/3862354033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cf930 msgr2=0x7fe9a40cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 --2- 172.21.15.87:0/3862354033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cf930 0x7fe9a40cfde0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009870 tx=0x7fe99c01c490 comp rx=0 tx=0).stop 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- 172.21.15.87:0/3862354033 shutdown_connections 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 --2- 172.21.15.87:0/3862354033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cf930 0x7fe9a40cfde0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 --2- 172.21.15.87:0/3862354033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cda10 0x7fe9a406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- 172.21.15.87:0/3862354033 >> 172.21.15.87:0/3862354033 conn(0x7fe9a4066b70 msgr2=0x7fe9a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- 172.21.15.87:0/3862354033 shutdown_connections 2024-04-25T10:12:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- 172.21.15.87:0/3862354033 wait complete. 2024-04-25T10:12:59.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 Processor -- start 2024-04-25T10:12:59.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- start start 2024-04-25T10:12:59.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cda10 0x7fe9a4146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cf930 0x7fe9a415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a415bc40 0x7fe9a415fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a40b3850 con 0x7fe9a40cda10 2024-04-25T10:12:59.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a40b36d0 con 0x7fe9a415bc40 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9ab800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a40b39d0 con 0x7fe9a40cf930 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9a8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cda10 0x7fe9a4146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9a8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cda10 0x7fe9a4146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54892/0 (socket says 172.21.15.87:54892) 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.811+0000 7fe9a8e00640 1 -- 172.21.15.87:0/530457985 learned_addr learned my addr 172.21.15.87:0/530457985 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a3e00640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cf930 0x7fe9a415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a9800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a415bc40 0x7fe9a415fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a8e00640 1 -- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cf930 msgr2=0x7fe9a415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a8e00640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cf930 0x7fe9a415f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a8e00640 1 -- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a415bc40 msgr2=0x7fe9a415fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a8e00640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a415bc40 0x7fe9a415fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a8e00640 1 -- 172.21.15.87:0/530457985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a41484a0 con 0x7fe9a40cda10 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a9800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a415bc40 0x7fe9a415fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a8e00640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cda10 0x7fe9a4146ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe99400ba50 tx=0x7fe99400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a0c00640 1 -- 172.21.15.87:0/530457985 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994019070 con 0x7fe9a40cda10 2024-04-25T10:12:59.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a0c00640 1 -- 172.21.15.87:0/530457985 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99400cc20 con 0x7fe9a40cda10 2024-04-25T10:12:59.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a0c00640 1 -- 172.21.15.87:0/530457985 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994014680 con 0x7fe9a40cda10 2024-04-25T10:12:59.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a4148780 con 0x7fe9a40cda10 2024-04-25T10:12:59.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a4147410 con 0x7fe9a40cda10 2024-04-25T10:12:59.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a404ef30 con 0x7fe9a40cda10 2024-04-25T10:12:59.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a0c00640 1 -- 172.21.15.87:0/530457985 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe99401c070 con 0x7fe9a40cda10 2024-04-25T10:12:59.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a0c00640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe984066f90 0x7fe984069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a0c00640 1 -- 172.21.15.87:0/530457985 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe994084d70 con 0x7fe9a40cda10 2024-04-25T10:12:59.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.815+0000 7fe9a3e00640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe984066f90 0x7fe984069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.819+0000 7fe9a3e00640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe984066f90 0x7fe984069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9a414aab0 tx=0x7fe990008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.819+0000 7fe9a0c00640 1 -- 172.21.15.87:0/530457985 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9940556a0 con 0x7fe9a40cda10 2024-04-25T10:12:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.987+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe9a406d960 con 0x7fe9a40cda10 2024-04-25T10:12:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.987+0000 7fe9a0c00640 1 -- 172.21.15.87:0/530457985 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe994059740 con 0x7fe9a40cda10 2024-04-25T10:12:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:12:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe984066f90 msgr2=0x7fe984069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe984066f90 0x7fe984069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9a414aab0 tx=0x7fe990008040 comp rx=0 tx=0).stop 2024-04-25T10:12:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cda10 msgr2=0x7fe9a4146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cda10 0x7fe9a4146ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe99400ba50 tx=0x7fe99400bf10 comp rx=0 tx=0).stop 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 shutdown_connections 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe984066f90 0x7fe984069450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a415bc40 0x7fe9a415fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a40cf930 0x7fe9a415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 --2- 172.21.15.87:0/530457985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a40cda10 0x7fe9a4146ed0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 >> 172.21.15.87:0/530457985 conn(0x7fe9a4066b70 msgr2=0x7fe9a4066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 shutdown_connections 2024-04-25T10:12:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:12:59.991+0000 7fe9ab800640 1 -- 172.21.15.87:0/530457985 wait complete. 2024-04-25T10:13:00.022 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T10:13:00.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.023 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T10:13:00.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:13:00.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.046 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:13:00.100 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:13:00.100 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:00.110 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:13:00.110 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:00.162 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:13:00.162 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:00.214 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:00.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.343+0000 7fcb1be00640 1 Processor -- start 2024-04-25T10:13:00.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.343+0000 7fcb1be00640 1 -- start start 2024-04-25T10:13:00.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb1c15bc10 0x7fcb1c0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1c15bff0 con 0x7fcb1c0ccd40 2024-04-25T10:13:00.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1c0cd550 con 0x7fcb1c0d0440 2024-04-25T10:13:00.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1c0b0da0 con 0x7fcb1c15bc10 2024-04-25T10:13:00.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1a000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcb1c0ccd40 0x7fcb1c0cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34790/0 (socket says 172.21.15.87:34790) 2024-04-25T10:13:00.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1a000640 1 -- 172.21.15.87:0/4246485229 learned_addr learned my addr 172.21.15.87:0/4246485229 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/4246485229 >> v2:172.21.15.169:3300/0 conn(0x7fcb1c15bc10 0x7fcb1c0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507072087 0 0) 0x7fcb1c15bff0 con 0x7fcb1c0ccd40 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb00003530 con 0x7fcb1c0ccd40 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3835982868 0 0) 0x7fcb00003530 con 0x7fcb1c0ccd40 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 >> v2:172.21.15.169:3300/0 conn(0x7fcb1c15bc10 msgr2=0x7fcb1c0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 --2- 172.21.15.87:0/4246485229 >> v2:172.21.15.169:3300/0 conn(0x7fcb1c15bc10 0x7fcb1c0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 >> v1:172.21.15.169:6789/0 conn(0x7fcb1c0d0440 legacy=0x7fcb1c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c0b1080 con 0x7fcb1c0ccd40 2024-04-25T10:13:00.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fcb08001680 con 0x7fcb1c0ccd40 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 >> v1:172.21.15.87:6789/0 conn(0x7fcb1c0ccd40 legacy=0x7fcb1c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb00003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb000043c0 0x7fcb00004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb00005090 0x7fcb0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0b0da0 con 0x7fcb000039c0 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000bff0 con 0x7fcb000043c0 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000c1f0 con 0x7fcb00005090 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb00003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1a000640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb000043c0 0x7fcb00004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1b400640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb00005090 0x7fcb0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1aa00640 1 -- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb00005090 msgr2=0x7fcb0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb00005090 0x7fcb0000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1aa00640 1 -- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb000043c0 msgr2=0x7fcb00004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb000043c0 0x7fcb00004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1aa00640 1 -- 172.21.15.87:0/4246485229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00003530 con 0x7fcb000039c0 2024-04-25T10:13:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1b400640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb00005090 0x7fcb0000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.347+0000 7fcb1b400640 1 -- 172.21.15.87:0/4246485229 reap_dead start 2024-04-25T10:13:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb00003da0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009ae0 tx=0x7fcb0c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b70d8c92d8f2b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0b2070 con 0x7fcb000039c0 2024-04-25T10:13:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c0a5660 con 0x7fcb000039c0 2024-04-25T10:13:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb19600640 1 -- 172.21.15.87:0/4246485229 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0096b0 con 0x7fcb000039c0 2024-04-25T10:13:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1be00640 1 -- 172.21.15.87:0/4246485229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 msgr2=0x7fcb00003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1be00640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb00003da0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009ae0 tx=0x7fcb0c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:13:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1be00640 1 -- 172.21.15.87:0/4246485229 shutdown_connections 2024-04-25T10:13:00.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1be00640 1 --2- 172.21.15.87:0/4246485229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb00003da0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1be00640 1 -- 172.21.15.87:0/4246485229 >> 172.21.15.87:0/4246485229 conn(0x7fcb1c0d1750 msgr2=0x7fcb1c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1be00640 1 -- 172.21.15.87:0/4246485229 shutdown_connections 2024-04-25T10:13:00.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.351+0000 7fcb1be00640 1 -- 172.21.15.87:0/4246485229 wait complete. 2024-04-25T10:13:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 Processor -- start 2024-04-25T10:13:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 -- start start 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb1c1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0aef10 0x7fcb1c1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c141ce0 0x7fcb1c13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0cd330 con 0x7fcb000039c0 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0cd1b0 con 0x7fcb1c0aef10 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0cf760 con 0x7fcb1c141ce0 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb1c1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb1c1506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54910/0 (socket says 172.21.15.87:54910) 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 -- 172.21.15.87:0/3505924187 learned_addr learned my addr 172.21.15.87:0/3505924187 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1a000640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0aef10 0x7fcb1c1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1b400640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c141ce0 0x7fcb1c13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 -- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c141ce0 msgr2=0x7fcb1c13fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c141ce0 0x7fcb1c13fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 -- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0aef10 msgr2=0x7fcb1c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0aef10 0x7fcb1c1417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.355+0000 7fcb1aa00640 1 -- 172.21.15.87:0/3505924187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c0af410 con 0x7fcb000039c0 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb1aa00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb1c1506d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c01c8a0 tx=0x7fcb0c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb12a00640 1 -- 172.21.15.87:0/3505924187 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0b2040 con 0x7fcb000039c0 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb12a00640 1 -- 172.21.15.87:0/3505924187 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c00f000 con 0x7fcb000039c0 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb12a00640 1 -- 172.21.15.87:0/3505924187 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c004240 con 0x7fcb000039c0 2024-04-25T10:13:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c0af6f0 con 0x7fcb000039c0 2024-04-25T10:13:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c0afcb0 con 0x7fcb000039c0 2024-04-25T10:13:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c075120 con 0x7fcb000039c0 2024-04-25T10:13:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb12a00640 1 -- 172.21.15.87:0/3505924187 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb0c0bf030 con 0x7fcb000039c0 2024-04-25T10:13:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb12a00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcafc066f60 0x7fcafc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb12a00640 1 -- 172.21.15.87:0/3505924187 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb0c11edc0 con 0x7fcb000039c0 2024-04-25T10:13:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.359+0000 7fcb1a000640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcafc066f60 0x7fcafc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.363+0000 7fcb1a000640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcafc066f60 0x7fcafc069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0b7470 tx=0x7fcb080015c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.363+0000 7fcb12a00640 1 -- 172.21.15.87:0/3505924187 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb0c0c6060 con 0x7fcb000039c0 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.519+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb1c0cf540 con 0x7fcb000039c0 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb12a00640 1 -- 172.21.15.87:0/3505924187 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fcb0c016780 con 0x7fcb000039c0 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcafc066f60 msgr2=0x7fcafc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcafc066f60 0x7fcafc069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0b7470 tx=0x7fcb080015c0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 msgr2=0x7fcb1c1506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb1c1506d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c01c8a0 tx=0x7fcb0c019810 comp rx=0 tx=0).stop 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 shutdown_connections 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcafc066f60 0x7fcafc069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c141ce0 0x7fcb1c13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0aef10 0x7fcb1c1417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 --2- 172.21.15.87:0/3505924187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb000039c0 0x7fcb1c1506d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 >> 172.21.15.87:0/3505924187 conn(0x7fcb1c0d1750 msgr2=0x7fcb1c15aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 shutdown_connections 2024-04-25T10:13:00.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.523+0000 7fcb1be00640 1 -- 172.21.15.87:0/3505924187 wait complete. 2024-04-25T10:13:00.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:00.550 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:00.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.683+0000 7f6103e00640 1 Processor -- start 2024-04-25T10:13:00.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.683+0000 7f6103e00640 1 -- start start 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6103e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6104152b80 0x7f610415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6103e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f610415fc60 con 0x7f61040ccde0 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6103e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f610415ff40 con 0x7f61040cca00 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6103e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6104160220 con 0x7f6104152b80 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f60fa000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6104152b80 0x7f610415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6103400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f61040ccde0 0x7f6104071a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34794/0 (socket says 172.21.15.87:34794) 2024-04-25T10:13:00.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6103400640 1 -- 172.21.15.87:0/615494817 learned_addr learned my addr 172.21.15.87:0/615494817 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:00.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f60fa000640 1 -- 172.21.15.87:0/615494817 >> v1:172.21.15.169:6789/0 conn(0x7f61040cca00 legacy=0x7f6104152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f60fa000640 1 -- 172.21.15.87:0/615494817 >> v1:172.21.15.87:6789/0 conn(0x7f61040ccde0 legacy=0x7f6104071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f60fa000640 1 -- 172.21.15.87:0/615494817 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610406dae0 con 0x7f6104152b80 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f60fa000640 1 --2- 172.21.15.87:0/615494817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6104152b80 0x7f610415c9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0097c0 tx=0x7f60ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d779f199b006686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6102000640 1 -- 172.21.15.87:0/615494817 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0b1070 con 0x7f6104152b80 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.687+0000 7f6102000640 1 -- 172.21.15.87:0/615494817 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec0ac410 con 0x7f6104152b80 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102000640 1 -- 172.21.15.87:0/615494817 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0ac7e0 con 0x7f6104152b80 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- 172.21.15.87:0/615494817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6104152b80 msgr2=0x7f610415c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 --2- 172.21.15.87:0/615494817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6104152b80 0x7f610415c9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0097c0 tx=0x7f60ec01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- 172.21.15.87:0/615494817 shutdown_connections 2024-04-25T10:13:00.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 --2- 172.21.15.87:0/615494817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6104152b80 0x7f610415c9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- 172.21.15.87:0/615494817 >> 172.21.15.87:0/615494817 conn(0x7f610406b8a0 msgr2=0x7f61040b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- 172.21.15.87:0/615494817 shutdown_connections 2024-04-25T10:13:00.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- 172.21.15.87:0/615494817 wait complete. 2024-04-25T10:13:00.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 Processor -- start 2024-04-25T10:13:00.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- start start 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61040cca00 0x7f6104149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61040ccde0 0x7f61041544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6104152b80 0x7f6104154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6104169c50 con 0x7f61040cca00 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6104169ad0 con 0x7f6104152b80 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6104169dd0 con 0x7f61040ccde0 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61040cca00 0x7f6104149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61040cca00 0x7f6104149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54928/0 (socket says 172.21.15.87:54928) 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 -- 172.21.15.87:0/2142001922 learned_addr learned my addr 172.21.15.87:0/2142001922 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6103400640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6104152b80 0x7f6104154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f60fa000640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61040ccde0 0x7f61041544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 -- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61040ccde0 msgr2=0x7f61041544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61040ccde0 0x7f61041544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 -- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6104152b80 msgr2=0x7f6104154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6104152b80 0x7f6104154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.691+0000 7f6102a00640 1 -- 172.21.15.87:0/2142001922 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61040af750 con 0x7f61040cca00 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f6102a00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61040cca00 0x7f6104149210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60f400c420 tx=0x7f60f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fb400640 1 -- 172.21.15.87:0/2142001922 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f40052d0 con 0x7f61040cca00 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fb400640 1 -- 172.21.15.87:0/2142001922 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f4005470 con 0x7f61040cca00 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61040afa30 con 0x7f61040cca00 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fb400640 1 -- 172.21.15.87:0/2142001922 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f4007260 con 0x7f61040cca00 2024-04-25T10:13:00.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61041550a0 con 0x7f61040cca00 2024-04-25T10:13:00.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6104075120 con 0x7f61040cca00 2024-04-25T10:13:00.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fb400640 1 -- 172.21.15.87:0/2142001922 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60f4007400 con 0x7f61040cca00 2024-04-25T10:13:00.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fb400640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f60e0066f60 0x7f60e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fb400640 1 -- 172.21.15.87:0/2142001922 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60f4084550 con 0x7f61040cca00 2024-04-25T10:13:00.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fa000640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f60e0066f60 0x7f60e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.695+0000 7f60fa000640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f60e0066f60 0x7f60e0069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ae000 tx=0x7f60ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.699+0000 7f60fb400640 1 -- 172.21.15.87:0/2142001922 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60f4055550 con 0x7f61040cca00 2024-04-25T10:13:00.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.855+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6104074020 con 0x7f61040cca00 2024-04-25T10:13:00.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.855+0000 7f60fb400640 1 -- 172.21.15.87:0/2142001922 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60f4054e50 con 0x7f61040cca00 2024-04-25T10:13:00.863 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f60e0066f60 msgr2=0x7f60e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f60e0066f60 0x7f60e0069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ae000 tx=0x7f60ec008040 comp rx=0 tx=0).stop 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61040cca00 msgr2=0x7f6104149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61040cca00 0x7f6104149210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60f400c420 tx=0x7f60f400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 shutdown_connections 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f60e0066f60 0x7f60e0069420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6104152b80 0x7f6104154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61040ccde0 0x7f61041544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 --2- 172.21.15.87:0/2142001922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61040cca00 0x7f6104149210 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 >> 172.21.15.87:0/2142001922 conn(0x7f610406b8a0 msgr2=0x7f61040cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 shutdown_connections 2024-04-25T10:13:00.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:00.859+0000 7f6103e00640 1 -- 172.21.15.87:0/2142001922 wait complete. 2024-04-25T10:13:00.884 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.015+0000 7f6721200640 1 Processor -- start 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.015+0000 7f6721200640 1 -- start start 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6721200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f671c071260 0x7f671c0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6721200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071be0 0x7f671c0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6721200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671c06de60 con 0x7f671c15a780 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6721200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671c06cdc0 con 0x7f671c071260 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6721200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c06d0a0 con 0x7f671c071be0 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f671b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f671c15a780 0x7f671c15edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34810/0 (socket says 172.21.15.87:34810) 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f671b400640 1 -- 172.21.15.87:0/2443984777 learned_addr learned my addr 172.21.15.87:0/2443984777 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6712a00640 1 --2- 172.21.15.87:0/2443984777 >> v2:172.21.15.87:3300/0 conn(0x7f671c071260 0x7f671c0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f671be00640 1 --2- 172.21.15.87:0/2443984777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071be0 0x7f671c0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f671aa00640 1 -- 172.21.15.87:0/2443984777 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331951371 0 0) 0x7f671c06de60 con 0x7f671c15a780 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f671aa00640 1 -- 172.21.15.87:0/2443984777 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6700005740 con 0x7f671c15a780 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6712a00640 1 -- 172.21.15.87:0/2443984777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071be0 msgr2=0x7f671c0b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6712a00640 1 --2- 172.21.15.87:0/2443984777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071be0 0x7f671c0b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6712a00640 1 -- 172.21.15.87:0/2443984777 >> v1:172.21.15.87:6789/0 conn(0x7f671c15a780 legacy=0x7f671c15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6712a00640 1 -- 172.21.15.87:0/2443984777 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c06d380 con 0x7f671c071260 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f671be00640 1 --2- 172.21.15.87:0/2443984777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071be0 0x7f671c0b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.019+0000 7f6712a00640 1 --2- 172.21.15.87:0/2443984777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c071260 0x7f671c0716a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67040097c0 tx=0x7f670401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa266f48da4b366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f671aa00640 1 -- 172.21.15.87:0/2443984777 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040b0070 con 0x7f671c071260 2024-04-25T10:13:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f671aa00640 1 -- 172.21.15.87:0/2443984777 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040ab600 con 0x7f671c071260 2024-04-25T10:13:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f671aa00640 1 -- 172.21.15.87:0/2443984777 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040ab9d0 con 0x7f671c071260 2024-04-25T10:13:01.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- 172.21.15.87:0/2443984777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c071260 msgr2=0x7f671c0716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 --2- 172.21.15.87:0/2443984777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c071260 0x7f671c0716a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67040097c0 tx=0x7f670401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- 172.21.15.87:0/2443984777 shutdown_connections 2024-04-25T10:13:01.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 --2- 172.21.15.87:0/2443984777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071be0 0x7f671c0b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 --2- 172.21.15.87:0/2443984777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c071260 0x7f671c0716a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- 172.21.15.87:0/2443984777 >> 172.21.15.87:0/2443984777 conn(0x7f671c0743f0 msgr2=0x7f671c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- 172.21.15.87:0/2443984777 shutdown_connections 2024-04-25T10:13:01.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- 172.21.15.87:0/2443984777 wait complete. 2024-04-25T10:13:01.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 Processor -- start 2024-04-25T10:13:01.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- start start 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071260 0x7f671c0aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c071be0 0x7f671c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c15a780 0x7f671c144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0b3330 con 0x7f671c15a780 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0b31b0 con 0x7f671c071be0 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6721200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0b34b0 con 0x7f671c071260 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f671be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c15a780 0x7f671c144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f671be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c15a780 0x7f671c144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54948/0 (socket says 172.21.15.87:54948) 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f671be00640 1 -- 172.21.15.87:0/711266992 learned_addr learned my addr 172.21.15.87:0/711266992 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f6712a00640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c071be0 0x7f671c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.023+0000 7f671b400640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071260 0x7f671c0aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f671be00640 1 -- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071260 msgr2=0x7f671c0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f671be00640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071260 0x7f671c0aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f671be00640 1 -- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c071be0 msgr2=0x7f671c154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f671be00640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c071be0 0x7f671c154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f671be00640 1 -- 172.21.15.87:0/711266992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c0af5a0 con 0x7f671c15a780 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f671be00640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c15a780 0x7f671c144910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6714007e20 tx=0x7f671400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6713e00640 1 -- 172.21.15.87:0/711266992 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671401a070 con 0x7f671c15a780 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6713e00640 1 -- 172.21.15.87:0/711266992 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714003ba0 con 0x7f671c15a780 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c0af880 con 0x7f671c15a780 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6713e00640 1 -- 172.21.15.87:0/711266992 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400de80 con 0x7f671c15a780 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c0afe40 con 0x7f671c15a780 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c075120 con 0x7f671c15a780 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6713e00640 1 -- 172.21.15.87:0/711266992 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f671401d070 con 0x7f671c15a780 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6713e00640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f8066f60 0x7f66f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f6713e00640 1 -- 172.21.15.87:0/711266992 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6714085290 con 0x7f671c15a780 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.027+0000 7f671b400640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f8066f60 0x7f66f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.031+0000 7f671b400640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f8066f60 0x7f66f8069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f670c005890 tx=0x7f670c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.031+0000 7f6713e00640 1 -- 172.21.15.87:0/711266992 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6714010070 con 0x7f671c15a780 2024-04-25T10:13:01.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.187+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f671c152a30 con 0x7f671c15a780 2024-04-25T10:13:01.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.187+0000 7f6713e00640 1 -- 172.21.15.87:0/711266992 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6714055c40 con 0x7f671c15a780 2024-04-25T10:13:01.195 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:13:01.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f8066f60 msgr2=0x7f66f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f8066f60 0x7f66f8069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f670c005890 tx=0x7f670c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c15a780 msgr2=0x7f671c144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c15a780 0x7f671c144910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6714007e20 tx=0x7f671400d4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 shutdown_connections 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f8066f60 0x7f66f8069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c15a780 0x7f671c144910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c071be0 0x7f671c154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 --2- 172.21.15.87:0/711266992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071260 0x7f671c0aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 >> 172.21.15.87:0/711266992 conn(0x7f671c0743f0 msgr2=0x7f671c0ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 shutdown_connections 2024-04-25T10:13:01.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.191+0000 7f6721200640 1 -- 172.21.15.87:0/711266992 wait complete. 2024-04-25T10:13:01.215 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:01.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0874000640 1 Processor -- start 2024-04-25T10:13:01.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0874000640 1 -- start start 2024-04-25T10:13:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0874000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f086c15b150 0x7f086c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0874000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086c0b2fd0 con 0x7f086c15a0e0 2024-04-25T10:13:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0874000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086c0b32b0 con 0x7f086c06c0d0 2024-04-25T10:13:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0874000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086c15b5c0 con 0x7f086c15b150 2024-04-25T10:13:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0871600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f086c15a0e0 0x7f086c0b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34826/0 (socket says 172.21.15.87:34826) 2024-04-25T10:13:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0871600640 1 -- 172.21.15.87:0/2501153041 learned_addr learned my addr 172.21.15.87:0/2501153041 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.347+0000 7f0870c00640 1 --2- 172.21.15.87:0/2501153041 >> v2:172.21.15.87:3300/0 conn(0x7f086c15b150 0x7f086c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f0870c00640 1 -- 172.21.15.87:0/2501153041 >> v1:172.21.15.169:6789/0 conn(0x7f086c06c0d0 legacy=0x7f086c0705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f0870c00640 1 -- 172.21.15.87:0/2501153041 >> v1:172.21.15.87:6789/0 conn(0x7f086c15a0e0 legacy=0x7f086c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f0870c00640 1 -- 172.21.15.87:0/2501153041 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c0b3590 con 0x7f086c15b150 2024-04-25T10:13:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f0870c00640 1 --2- 172.21.15.87:0/2501153041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c0aea50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f08580097c0 tx=0x7f085801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa142ff57e6cb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f086be00640 1 -- 172.21.15.87:0/2501153041 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580b0070 con 0x7f086c15b150 2024-04-25T10:13:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f086be00640 1 -- 172.21.15.87:0/2501153041 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08580ab600 con 0x7f086c15b150 2024-04-25T10:13:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f086be00640 1 -- 172.21.15.87:0/2501153041 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580ab9d0 con 0x7f086c15b150 2024-04-25T10:13:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f0874000640 1 -- 172.21.15.87:0/2501153041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 msgr2=0x7f086c0aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.351+0000 7f0874000640 1 --2- 172.21.15.87:0/2501153041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c0aea50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f08580097c0 tx=0x7f085801c380 comp rx=0 tx=0).stop 2024-04-25T10:13:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- 172.21.15.87:0/2501153041 shutdown_connections 2024-04-25T10:13:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 --2- 172.21.15.87:0/2501153041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c0aea50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- 172.21.15.87:0/2501153041 >> 172.21.15.87:0/2501153041 conn(0x7f086c0669a0 msgr2=0x7f086c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- 172.21.15.87:0/2501153041 shutdown_connections 2024-04-25T10:13:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- 172.21.15.87:0/2501153041 wait complete. 2024-04-25T10:13:01.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 Processor -- start 2024-04-25T10:13:01.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- start start 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c06c0d0 0x7f086c150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f086c15a0e0 0x7f086c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c1f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0aef90 con 0x7f086c15b150 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c070f50 con 0x7f086c06c0d0 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0874000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0af1d0 con 0x7f086c15a0e0 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c1f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c1f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54964/0 (socket says 172.21.15.87:54964) 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 -- 172.21.15.87:0/246219527 learned_addr learned my addr 172.21.15.87:0/246219527 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0871600640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c06c0d0 0x7f086c150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0870c00640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f086c15a0e0 0x7f086c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 -- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f086c15a0e0 msgr2=0x7f086c158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f086c15a0e0 0x7f086c158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 -- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c06c0d0 msgr2=0x7f086c150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c06c0d0 0x7f086c150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 -- 172.21.15.87:0/246219527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c151260 con 0x7f086c15b150 2024-04-25T10:13:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0872000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c1f6320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f086400c460 tx=0x7f086400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.355+0000 7f0869600640 1 -- 172.21.15.87:0/246219527 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864016020 con 0x7f086c15b150 2024-04-25T10:13:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0869600640 1 -- 172.21.15.87:0/246219527 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864005220 con 0x7f086c15b150 2024-04-25T10:13:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c0cc2a0 con 0x7f086c15b150 2024-04-25T10:13:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0869600640 1 -- 172.21.15.87:0/246219527 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864007030 con 0x7f086c15b150 2024-04-25T10:13:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c0cc860 con 0x7f086c15b150 2024-04-25T10:13:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c04ef30 con 0x7f086c15b150 2024-04-25T10:13:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0869600640 1 -- 172.21.15.87:0/246219527 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08640071d0 con 0x7f086c15b150 2024-04-25T10:13:01.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0869600640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f084c066f60 0x7f084c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0869600640 1 -- 172.21.15.87:0/246219527 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0864084750 con 0x7f086c15b150 2024-04-25T10:13:01.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0871600640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f084c066f60 0x7f084c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.359+0000 7f0871600640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f084c066f60 0x7f084c069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0860005c10 tx=0x7f0860009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.363+0000 7f0869600640 1 -- 172.21.15.87:0/246219527 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0864020020 con 0x7f086c15b150 2024-04-25T10:13:01.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f086c070790 con 0x7f086c15b150 2024-04-25T10:13:01.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0869600640 1 -- 172.21.15.87:0/246219527 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f086c151260 con 0x7f086c15b150 2024-04-25T10:13:01.525 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f084c066f60 msgr2=0x7f084c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f084c066f60 0x7f084c069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0860005c10 tx=0x7f0860009040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 msgr2=0x7f086c1f6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c1f6320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f086400c460 tx=0x7f086400c920 comp rx=0 tx=0).stop 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 shutdown_connections 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f084c066f60 0x7f084c069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f086c15b150 0x7f086c1f6320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f086c15a0e0 0x7f086c158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 --2- 172.21.15.87:0/246219527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f086c06c0d0 0x7f086c150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 >> 172.21.15.87:0/246219527 conn(0x7f086c0669a0 msgr2=0x7f086c167f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 shutdown_connections 2024-04-25T10:13:01.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.519+0000 7f0874000640 1 -- 172.21.15.87:0/246219527 wait complete. 2024-04-25T10:13:01.546 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:01.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.675+0000 7f7cc9000640 1 Processor -- start 2024-04-25T10:13:01.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.675+0000 7f7cc9000640 1 -- start start 2024-04-25T10:13:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 0x7f7cc406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7cc40cf930 0x7f7cc40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc9000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc40b2770 con 0x7f7cc415bc40 2024-04-25T10:13:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc9000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc406c670 con 0x7f7cc40cf930 2024-04-25T10:13:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc406c950 con 0x7f7cc40cda10 2024-04-25T10:13:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7cc40cf930 0x7f7cc40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc1e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7cc415bc40 0x7f7cc40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34836/0 (socket says 172.21.15.87:34836) 2024-04-25T10:13:01.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc1e00640 1 -- 172.21.15.87:0/1805687870 learned_addr learned my addr 172.21.15.87:0/1805687870 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:01.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cb9400640 1 --2- 172.21.15.87:0/1805687870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 0x7f7cc406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc2800640 1 -- 172.21.15.87:0/1805687870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 msgr2=0x7f7cc406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc2800640 1 --2- 172.21.15.87:0/1805687870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 0x7f7cc406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc2800640 1 -- 172.21.15.87:0/1805687870 >> v1:172.21.15.87:6789/0 conn(0x7f7cc415bc40 legacy=0x7f7cc40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc2800640 1 -- 172.21.15.87:0/1805687870 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc406cc30 con 0x7f7cc40cf930 2024-04-25T10:13:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.679+0000 7f7cc1400640 1 -- 172.21.15.87:0/1805687870 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564944908 0 0) 0x7f7cc40b2770 con 0x7f7cc415bc40 2024-04-25T10:13:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc2800640 1 --2- 172.21.15.87:0/1805687870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc40cfde0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009d90 tx=0x7f7cbc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b3169e74596751 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc1400640 1 -- 172.21.15.87:0/1805687870 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0b2070 con 0x7f7cc40cf930 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc1400640 1 -- 172.21.15.87:0/1805687870 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc0a57a0 con 0x7f7cc40cf930 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc1400640 1 -- 172.21.15.87:0/1805687870 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0a5b70 con 0x7f7cc40cf930 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- 172.21.15.87:0/1805687870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 msgr2=0x7f7cc40cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 --2- 172.21.15.87:0/1805687870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc40cfde0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009d90 tx=0x7f7cbc01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- 172.21.15.87:0/1805687870 shutdown_connections 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 --2- 172.21.15.87:0/1805687870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc40cfde0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 --2- 172.21.15.87:0/1805687870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 0x7f7cc406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- 172.21.15.87:0/1805687870 >> 172.21.15.87:0/1805687870 conn(0x7f7cc4066b70 msgr2=0x7f7cc4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- 172.21.15.87:0/1805687870 shutdown_connections 2024-04-25T10:13:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- 172.21.15.87:0/1805687870 wait complete. 2024-04-25T10:13:01.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 Processor -- start 2024-04-25T10:13:01.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- start start 2024-04-25T10:13:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 0x7f7cc4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc413d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc415bc40 0x7f7cc40af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b3850 con 0x7f7cc40cf930 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b36d0 con 0x7f7cc415bc40 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cc9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b39d0 con 0x7f7cc40cda10 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.683+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc413d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc413d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54986/0 (socket says 172.21.15.87:54986) 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 -- 172.21.15.87:0/3137330687 learned_addr learned my addr 172.21.15.87:0/3137330687 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 -- 172.21.15.87:0/3137330687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 msgr2=0x7f7cc4157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 0x7f7cc4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 -- 172.21.15.87:0/3137330687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc415bc40 msgr2=0x7f7cc40af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc415bc40 0x7f7cc40af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 -- 172.21.15.87:0/3137330687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40afae0 con 0x7f7cc40cf930 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cb9400640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc413d160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00d620 tx=0x7f7cac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cbaa00640 1 -- 172.21.15.87:0/3137330687 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00b620 con 0x7f7cc40cf930 2024-04-25T10:13:01.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cbaa00640 1 -- 172.21.15.87:0/3137330687 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac019070 con 0x7f7cc40cf930 2024-04-25T10:13:01.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cbaa00640 1 -- 172.21.15.87:0/3137330687 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00b9e0 con 0x7f7cc40cf930 2024-04-25T10:13:01.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc40b1b80 con 0x7f7cc40cf930 2024-04-25T10:13:01.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc40b2140 con 0x7f7cc40cf930 2024-04-25T10:13:01.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc404ef30 con 0x7f7cc40cf930 2024-04-25T10:13:01.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cbaa00640 1 -- 172.21.15.87:0/3137330687 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cac00bb80 con 0x7f7cc40cf930 2024-04-25T10:13:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cbaa00640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ca0066f90 0x7f7ca0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.687+0000 7f7cbaa00640 1 -- 172.21.15.87:0/3137330687 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cac0853b0 con 0x7f7cc40cf930 2024-04-25T10:13:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.691+0000 7f7cc1e00640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ca0066f90 0x7f7ca0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.691+0000 7f7cc1e00640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ca0066f90 0x7f7ca0069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40015a0 tx=0x7f7cb4007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.691+0000 7f7cbaa00640 1 -- 172.21.15.87:0/3137330687 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cc40afae0 con 0x7f7cc40cf930 2024-04-25T10:13:01.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.859+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7cc40af1c0 con 0x7f7cc40cf930 2024-04-25T10:13:01.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.859+0000 7f7cbaa00640 1 -- 172.21.15.87:0/3137330687 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7cac0563b0 con 0x7f7cc40cf930 2024-04-25T10:13:01.866 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:13:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ca0066f90 msgr2=0x7f7ca0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ca0066f90 0x7f7ca0069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40015a0 tx=0x7f7cb4007970 comp rx=0 tx=0).stop 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 msgr2=0x7f7cc413d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc413d160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00d620 tx=0x7f7cac00dae0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 shutdown_connections 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7ca0066f90 0x7f7ca0069450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc415bc40 0x7f7cc40af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc40cf930 0x7f7cc413d160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 --2- 172.21.15.87:0/3137330687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cc40cda10 0x7f7cc4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 >> 172.21.15.87:0/3137330687 conn(0x7f7cc4066b70 msgr2=0x7f7cc415feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 shutdown_connections 2024-04-25T10:13:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:01.863+0000 7f7cc9000640 1 -- 172.21.15.87:0/3137330687 wait complete. 2024-04-25T10:13:01.888 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T10:13:01.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.888 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T10:13:01.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:13:01.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.914 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:13:01.970 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:13:01.971 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:02.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfdbe00640 1 Processor -- start 2024-04-25T10:13:02.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfdbe00640 1 -- start start 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfdbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfd41550b0 0x7fcfd40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd40cfe70 0x7fcfd415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfdbe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd4071050 con 0x7fcfd415b740 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfdbe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd415d040 con 0x7fcfd41550b0 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfdbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd40712f0 con 0x7fcfd40cfe70 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfd41550b0 0x7fcfd40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfd41550b0 0x7fcfd40cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54990/0 (socket says 172.21.15.87:54990) 2024-04-25T10:13:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd9400640 1 -- 172.21.15.87:0/1205801529 learned_addr learned my addr 172.21.15.87:0/1205801529 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd8a00640 1 --2- 172.21.15.87:0/1205801529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd40cfe70 0x7fcfd415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd9400640 1 -- 172.21.15.87:0/1205801529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd40cfe70 msgr2=0x7fcfd415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd9400640 1 --2- 172.21.15.87:0/1205801529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd40cfe70 0x7fcfd415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd9400640 1 -- 172.21.15.87:0/1205801529 >> v1:172.21.15.169:6789/0 conn(0x7fcfd415b740 legacy=0x7fcfd406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.099+0000 7fcfd9400640 1 -- 172.21.15.87:0/1205801529 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd40715d0 con 0x7fcfd41550b0 2024-04-25T10:13:02.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfd9400640 1 --2- 172.21.15.87:0/1205801529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd41550b0 0x7fcfd40cf930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009870 tx=0x7fcfc801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837dd573669bdea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfd3e00640 1 -- 172.21.15.87:0/1205801529 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80b0070 con 0x7fcfd41550b0 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfd3e00640 1 -- 172.21.15.87:0/1205801529 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc80ab600 con 0x7fcfd41550b0 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfd3e00640 1 -- 172.21.15.87:0/1205801529 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80ab9d0 con 0x7fcfd41550b0 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 -- 172.21.15.87:0/1205801529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd41550b0 msgr2=0x7fcfd40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/1205801529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd41550b0 0x7fcfd40cf930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009870 tx=0x7fcfc801c520 comp rx=0 tx=0).stop 2024-04-25T10:13:02.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 -- 172.21.15.87:0/1205801529 shutdown_connections 2024-04-25T10:13:02.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/1205801529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd40cfe70 0x7fcfd415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/1205801529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd41550b0 0x7fcfd40cf930 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 -- 172.21.15.87:0/1205801529 >> 172.21.15.87:0/1205801529 conn(0x7fcfd4066b70 msgr2=0x7fcfd4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 -- 172.21.15.87:0/1205801529 shutdown_connections 2024-04-25T10:13:02.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.103+0000 7fcfdbe00640 1 -- 172.21.15.87:0/1205801529 wait complete. 2024-04-25T10:13:02.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 Processor -- start 2024-04-25T10:13:02.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 -- start start 2024-04-25T10:13:02.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd40cfe70 0x7fcfd4140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd41550b0 0x7fcfd4143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd415b740 0x7fcfd413fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd415bd50 con 0x7fcfd415b740 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd415bbd0 con 0x7fcfd40cfe70 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd415c030 con 0x7fcfd41550b0 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd415b740 0x7fcfd413fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd415b740 0x7fcfd413fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55002/0 (socket says 172.21.15.87:55002) 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 -- 172.21.15.87:0/670225485 learned_addr learned my addr 172.21.15.87:0/670225485 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9400640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd40cfe70 0x7fcfd4140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd8a00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd41550b0 0x7fcfd4143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 -- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd41550b0 msgr2=0x7fcfd4143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd41550b0 0x7fcfd4143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 -- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd40cfe70 msgr2=0x7fcfd4140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd40cfe70 0x7fcfd4140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 -- 172.21.15.87:0/670225485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4140bf0 con 0x7fcfd415b740 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd8a00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd41550b0 0x7fcfd4143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd9e00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd415b740 0x7fcfd413fc40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b970 tx=0x7fcfcc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd1600640 1 -- 172.21.15.87:0/670225485 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc019070 con 0x7fcfd415b740 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd1600640 1 -- 172.21.15.87:0/670225485 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00cb40 con 0x7fcfd415b740 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfd1600640 1 -- 172.21.15.87:0/670225485 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc014440 con 0x7fcfd415b740 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd4140ed0 con 0x7fcfd415b740 2024-04-25T10:13:02.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.107+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd4141490 con 0x7fcfd415b740 2024-04-25T10:13:02.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.111+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd404ef30 con 0x7fcfd415b740 2024-04-25T10:13:02.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.111+0000 7fcfd1600640 1 -- 172.21.15.87:0/670225485 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfcc01c070 con 0x7fcfd415b740 2024-04-25T10:13:02.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.111+0000 7fcfd1600640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfb4066f60 0x7fcfb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.111+0000 7fcfd1600640 1 -- 172.21.15.87:0/670225485 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfcc084bd0 con 0x7fcfd415b740 2024-04-25T10:13:02.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.111+0000 7fcfd9400640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfb4066f60 0x7fcfb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.111+0000 7fcfd9400640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfb4066f60 0x7fcfb4069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80ad000 tx=0x7fcfc80a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.115+0000 7fcfd1600640 1 -- 172.21.15.87:0/670225485 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfcc055580 con 0x7fcfd415b740 2024-04-25T10:13:02.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfd40b1070 con 0x7fcfd415b740 2024-04-25T10:13:02.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfd1600640 1 -- 172.21.15.87:0/670225485 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fcfcc059620 con 0x7fcfd415b740 2024-04-25T10:13:02.309 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:02.309 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:13:02.310 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfb4066f60 msgr2=0x7fcfb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfb4066f60 0x7fcfb4069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80ad000 tx=0x7fcfc80a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd415b740 msgr2=0x7fcfd413fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd415b740 0x7fcfd413fc40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b970 tx=0x7fcfcc00be30 comp rx=0 tx=0).stop 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 shutdown_connections 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfb4066f60 0x7fcfb4069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfd415b740 0x7fcfd413fc40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfd41550b0 0x7fcfd4143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 --2- 172.21.15.87:0/670225485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd40cfe70 0x7fcfd4140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 >> 172.21.15.87:0/670225485 conn(0x7fcfd4066b70 msgr2=0x7fcfd40b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 shutdown_connections 2024-04-25T10:13:02.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.303+0000 7fcfdbe00640 1 -- 172.21.15.87:0/670225485 wait complete. 2024-04-25T10:13:02.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:02.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64d2c00640 1 Processor -- start 2024-04-25T10:13:02.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64d2c00640 1 -- start start 2024-04-25T10:13:02.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64d2c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64cc0af9c0 0x7f64cc06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64d2c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64cc0afda0 0x7f64cc0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64d2c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc06f200 con 0x7f64cc071f50 2024-04-25T10:13:02.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64d2c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc06d380 con 0x7f64cc0afda0 2024-04-25T10:13:02.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64d2c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc0ccc10 con 0x7f64cc0af9c0 2024-04-25T10:13:02.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64cc0afda0 0x7f64cc0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64cc0afda0 0x7f64cc0cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55006/0 (socket says 172.21.15.87:55006) 2024-04-25T10:13:02.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cb400640 1 -- 172.21.15.87:0/1561361680 learned_addr learned my addr 172.21.15.87:0/1561361680 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:02.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cbe00640 1 --2- 172.21.15.87:0/1561361680 >> v2:172.21.15.169:3300/0 conn(0x7f64cc0af9c0 0x7f64cc06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cb400640 1 -- 172.21.15.87:0/1561361680 >> v2:172.21.15.169:3300/0 conn(0x7f64cc0af9c0 msgr2=0x7f64cc06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cb400640 1 --2- 172.21.15.87:0/1561361680 >> v2:172.21.15.169:3300/0 conn(0x7f64cc0af9c0 0x7f64cc06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cb400640 1 -- 172.21.15.87:0/1561361680 >> v1:172.21.15.169:6789/0 conn(0x7f64cc071f50 legacy=0x7f64cc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cb400640 1 -- 172.21.15.87:0/1561361680 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0ccef0 con 0x7f64cc0afda0 2024-04-25T10:13:02.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.459+0000 7f64cbe00640 1 --2- 172.21.15.87:0/1561361680 >> v2:172.21.15.169:3300/0 conn(0x7f64cc0af9c0 0x7f64cc06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64cb400640 1 --2- 172.21.15.87:0/1561361680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc0afda0 0x7f64cc0cc3a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ce370 tx=0x7f64b801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b390b252f31d754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64caa00640 1 -- 172.21.15.87:0/1561361680 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80ab660 con 0x7f64cc0afda0 2024-04-25T10:13:02.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64caa00640 1 -- 172.21.15.87:0/1561361680 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b80b0070 con 0x7f64cc0afda0 2024-04-25T10:13:02.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64caa00640 1 -- 172.21.15.87:0/1561361680 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80aba20 con 0x7f64cc0afda0 2024-04-25T10:13:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 -- 172.21.15.87:0/1561361680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc0afda0 msgr2=0x7f64cc0cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 --2- 172.21.15.87:0/1561361680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc0afda0 0x7f64cc0cc3a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ce370 tx=0x7f64b801c460 comp rx=0 tx=0).stop 2024-04-25T10:13:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 -- 172.21.15.87:0/1561361680 shutdown_connections 2024-04-25T10:13:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 --2- 172.21.15.87:0/1561361680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc0afda0 0x7f64cc0cc3a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 --2- 172.21.15.87:0/1561361680 >> v2:172.21.15.169:3300/0 conn(0x7f64cc0af9c0 0x7f64cc06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 -- 172.21.15.87:0/1561361680 >> 172.21.15.87:0/1561361680 conn(0x7f64cc0b2410 msgr2=0x7f64cc0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 -- 172.21.15.87:0/1561361680 shutdown_connections 2024-04-25T10:13:02.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.463+0000 7f64d2c00640 1 -- 172.21.15.87:0/1561361680 wait complete. 2024-04-25T10:13:02.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 Processor -- start 2024-04-25T10:13:02.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 -- start start 2024-04-25T10:13:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc071f50 0x7f64cc0aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64cc0af9c0 0x7f64cc15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64cc0afda0 0x7f64cc156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc067230 con 0x7f64cc071f50 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0670b0 con 0x7f64cc0af9c0 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1600d0 con 0x7f64cc0afda0 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc071f50 0x7f64cc0aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc071f50 0x7f64cc0aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55012/0 (socket says 172.21.15.87:55012) 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 -- 172.21.15.87:0/550876115 learned_addr learned my addr 172.21.15.87:0/550876115 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d0c00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64cc0afda0 0x7f64cc156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cb400640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64cc0af9c0 0x7f64cc15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 -- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64cc0afda0 msgr2=0x7f64cc156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64cc0afda0 0x7f64cc156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 -- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64cc0af9c0 msgr2=0x7f64cc15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64cc0af9c0 0x7f64cc15aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 -- 172.21.15.87:0/550876115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0c7950 con 0x7f64cc071f50 2024-04-25T10:13:02.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64cbe00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc071f50 0x7f64cc0aed50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009fc0 tx=0x7f64bc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64c3e00640 1 -- 172.21.15.87:0/550876115 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc01a070 con 0x7f64cc071f50 2024-04-25T10:13:02.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64c3e00640 1 -- 172.21.15.87:0/550876115 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc003ba0 con 0x7f64cc071f50 2024-04-25T10:13:02.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc0ce5f0 con 0x7f64cc071f50 2024-04-25T10:13:02.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64c3e00640 1 -- 172.21.15.87:0/550876115 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc004030 con 0x7f64cc071f50 2024-04-25T10:13:02.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc0cebb0 con 0x7f64cc071f50 2024-04-25T10:13:02.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.467+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc04ef30 con 0x7f64cc071f50 2024-04-25T10:13:02.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.471+0000 7f64c3e00640 1 -- 172.21.15.87:0/550876115 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64bc01d070 con 0x7f64cc071f50 2024-04-25T10:13:02.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.471+0000 7f64c3e00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64ac066f60 0x7f64ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.471+0000 7f64c3e00640 1 -- 172.21.15.87:0/550876115 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64bc085250 con 0x7f64cc071f50 2024-04-25T10:13:02.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.471+0000 7f64cb400640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64ac066f60 0x7f64ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.471+0000 7f64cb400640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64ac066f60 0x7f64ac069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64b80af000 tx=0x7f64b80a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.475+0000 7f64c3e00640 1 -- 172.21.15.87:0/550876115 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64bc010070 con 0x7f64cc071f50 2024-04-25T10:13:02.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.627+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64cc071420 con 0x7f64cc071f50 2024-04-25T10:13:02.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64c3e00640 1 -- 172.21.15.87:0/550876115 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f64bc055c00 con 0x7f64cc071f50 2024-04-25T10:13:02.636 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:02.636 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:12:25.234151+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:02.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64ac066f60 msgr2=0x7f64ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64ac066f60 0x7f64ac069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64b80af000 tx=0x7f64b80a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:02.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc071f50 msgr2=0x7f64cc0aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc071f50 0x7f64cc0aed50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009fc0 tx=0x7f64bc00d560 comp rx=0 tx=0).stop 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 shutdown_connections 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64ac066f60 0x7f64ac069420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64cc0afda0 0x7f64cc156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64cc0af9c0 0x7f64cc15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 --2- 172.21.15.87:0/550876115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64cc071f50 0x7f64cc0aed50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 >> 172.21.15.87:0/550876115 conn(0x7f64cc0b2410 msgr2=0x7f64cc0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 shutdown_connections 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.631+0000 7f64d2c00640 1 -- 172.21.15.87:0/550876115 wait complete. 2024-04-25T10:13:02.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:21.283917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:22.608130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:02.658 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:02.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2cd800640 1 Processor -- start 2024-04-25T10:13:02.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2cd800640 1 -- start start 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2cd800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc2c8159540 0x7fc2c80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2cd800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc2c80721f0 0x7fc2c8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2cd800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c8159920 con 0x7fc2c80aebd0 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2cd800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c815c2f0 con 0x7fc2c8159540 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2cd800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c815c5d0 con 0x7fc2c80721f0 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2c6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc2c8159540 0x7fc2c80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2be000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc2c80aebd0 0x7fc2c8071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34842/0 (socket says 172.21.15.87:34842) 2024-04-25T10:13:02.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2be000640 1 -- 172.21.15.87:0/585877869 learned_addr learned my addr 172.21.15.87:0/585877869 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:02.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2c7400640 1 --2- 172.21.15.87:0/585877869 >> v2:172.21.15.169:3300/0 conn(0x7fc2c80721f0 0x7fc2c8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2c6a00640 1 -- 172.21.15.87:0/585877869 >> v2:172.21.15.169:3300/0 conn(0x7fc2c80721f0 msgr2=0x7fc2c8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2c6a00640 1 --2- 172.21.15.87:0/585877869 >> v2:172.21.15.169:3300/0 conn(0x7fc2c80721f0 0x7fc2c8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2c6a00640 1 -- 172.21.15.87:0/585877869 >> v1:172.21.15.87:6789/0 conn(0x7fc2c80aebd0 legacy=0x7fc2c8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2c6a00640 1 -- 172.21.15.87:0/585877869 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c815c8b0 con 0x7fc2c8159540 2024-04-25T10:13:02.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.791+0000 7fc2c6000640 1 -- 172.21.15.87:0/585877869 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758290979 0 0) 0x7fc2c8159920 con 0x7fc2c80aebd0 2024-04-25T10:13:02.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2c6a00640 1 --2- 172.21.15.87:0/585877869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c8159540 0x7fc2c80ae690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80cd200 tx=0x7fc2b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d561f274cd6fa22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2c6000640 1 -- 172.21.15.87:0/585877869 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80b1070 con 0x7fc2c8159540 2024-04-25T10:13:02.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2c6000640 1 -- 172.21.15.87:0/585877869 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b80ac5f0 con 0x7fc2c8159540 2024-04-25T10:13:02.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2c6000640 1 -- 172.21.15.87:0/585877869 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80ac9c0 con 0x7fc2c8159540 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 -- 172.21.15.87:0/585877869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c8159540 msgr2=0x7fc2c80ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 --2- 172.21.15.87:0/585877869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c8159540 0x7fc2c80ae690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80cd200 tx=0x7fc2b801c480 comp rx=0 tx=0).stop 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 -- 172.21.15.87:0/585877869 shutdown_connections 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 --2- 172.21.15.87:0/585877869 >> v2:172.21.15.169:3300/0 conn(0x7fc2c80721f0 0x7fc2c8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 --2- 172.21.15.87:0/585877869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c8159540 0x7fc2c80ae690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 -- 172.21.15.87:0/585877869 >> 172.21.15.87:0/585877869 conn(0x7fc2c80669a0 msgr2=0x7fc2c8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 -- 172.21.15.87:0/585877869 shutdown_connections 2024-04-25T10:13:02.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.795+0000 7fc2cd800640 1 -- 172.21.15.87:0/585877869 wait complete. 2024-04-25T10:13:02.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 Processor -- start 2024-04-25T10:13:02.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 -- start start 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80721f0 0x7fc2c80af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c80aebd0 0x7fc2c81507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c8159540 0x7fc2c80cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8071220 con 0x7fc2c80aebd0 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80710a0 con 0x7fc2c8159540 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80713a0 con 0x7fc2c80721f0 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c80aebd0 0x7fc2c81507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c80aebd0 0x7fc2c81507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55018/0 (socket says 172.21.15.87:55018) 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 -- 172.21.15.87:0/3090341796 learned_addr learned my addr 172.21.15.87:0/3090341796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2c6a00640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80721f0 0x7fc2c80af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2c7400640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c8159540 0x7fc2c80cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 -- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80721f0 msgr2=0x7fc2c80af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80721f0 0x7fc2c80af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 -- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c8159540 msgr2=0x7fc2c80cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c8159540 0x7fc2c80cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 -- 172.21.15.87:0/3090341796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c81517f0 con 0x7fc2c80aebd0 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2c7400640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c8159540 0x7fc2c80cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2be000640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c80aebd0 0x7fc2c81507c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00011f0 tx=0x7fc2b000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2bf400640 1 -- 172.21.15.87:0/3090341796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0018070 con 0x7fc2c80aebd0 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2bf400640 1 -- 172.21.15.87:0/3090341796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b00074e0 con 0x7fc2c80aebd0 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c8151ad0 con 0x7fc2c80aebd0 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2bf400640 1 -- 172.21.15.87:0/3090341796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0004a10 con 0x7fc2c80aebd0 2024-04-25T10:13:02.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c8150d00 con 0x7fc2c80aebd0 2024-04-25T10:13:02.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.799+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c8151ec0 con 0x7fc2c80aebd0 2024-04-25T10:13:02.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.803+0000 7fc2bf400640 1 -- 172.21.15.87:0/3090341796 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2b001b070 con 0x7fc2c80aebd0 2024-04-25T10:13:02.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.803+0000 7fc2bf400640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2a4066f60 0x7fc2a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.803+0000 7fc2bf400640 1 -- 172.21.15.87:0/3090341796 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2b00842b0 con 0x7fc2c80aebd0 2024-04-25T10:13:02.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.803+0000 7fc2c6a00640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2a4066f60 0x7fc2a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.803+0000 7fc2c6a00640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2a4066f60 0x7fc2a4069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80ae000 tx=0x7fc2b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.807+0000 7fc2bf400640 1 -- 172.21.15.87:0/3090341796 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2b0054d00 con 0x7fc2c80aebd0 2024-04-25T10:13:02.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2c8151ca0 con 0x7fc2c80aebd0 2024-04-25T10:13:02.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2bf400640 1 -- 172.21.15.87:0/3090341796 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2b0058da0 con 0x7fc2c80aebd0 2024-04-25T10:13:02.969 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2a4066f60 msgr2=0x7fc2a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2a4066f60 0x7fc2a4069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80ae000 tx=0x7fc2b8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c80aebd0 msgr2=0x7fc2c81507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c80aebd0 0x7fc2c81507c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00011f0 tx=0x7fc2b000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 shutdown_connections 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2a4066f60 0x7fc2a4069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c8159540 0x7fc2c80cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2c80aebd0 0x7fc2c81507c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 --2- 172.21.15.87:0/3090341796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80721f0 0x7fc2c80af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 >> 172.21.15.87:0/3090341796 conn(0x7fc2c80669a0 msgr2=0x7fc2c8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 shutdown_connections 2024-04-25T10:13:02.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:02.963+0000 7fc2cd800640 1 -- 172.21.15.87:0/3090341796 wait complete. 2024-04-25T10:13:02.990 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:03.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba3000640 1 Processor -- start 2024-04-25T10:13:03.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba3000640 1 -- start start 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c147640 0x7f0b9c0ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba3000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0b9c06dab0 0x7f0b9c0717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba3000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c147a20 con 0x7f0b9c071ce0 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba3000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c0ae810 con 0x7f0b9c06dab0 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c072110 con 0x7f0b9c147640 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba1000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0b9c071ce0 0x7f0b9c161770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34858/0 (socket says 172.21.15.87:34858) 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0b93400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0b9c06dab0 0x7f0b9c0717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0ba1000640 1 -- 172.21.15.87:0/1955410251 learned_addr learned my addr 172.21.15.87:0/1955410251 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0b9be00640 1 --2- 172.21.15.87:0/1955410251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c147640 0x7f0b9c0ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0b9b400640 1 -- 172.21.15.87:0/1955410251 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269533762 0 0) 0x7f0b9c147a20 con 0x7f0b9c071ce0 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0b93400640 1 -- 172.21.15.87:0/1955410251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c147640 msgr2=0x7f0b9c0ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0b93400640 1 --2- 172.21.15.87:0/1955410251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c147640 0x7f0b9c0ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0b93400640 1 -- 172.21.15.87:0/1955410251 >> v1:172.21.15.87:6789/0 conn(0x7f0b9c071ce0 legacy=0x7f0b9c161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.123+0000 7f0b93400640 1 -- 172.21.15.87:0/1955410251 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c06e290 con 0x7f0b9c06dab0 2024-04-25T10:13:03.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0b93400640 1 --2- 172.21.15.87:0/1955410251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c0717a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0ae990 tx=0x7f0b8401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92af710d2d92b6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0b9b400640 1 -- 172.21.15.87:0/1955410251 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840a7070 con 0x7f0b9c06dab0 2024-04-25T10:13:03.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0b9b400640 1 -- 172.21.15.87:0/1955410251 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b840b0070 con 0x7f0b9c06dab0 2024-04-25T10:13:03.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0b9b400640 1 -- 172.21.15.87:0/1955410251 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840ab890 con 0x7f0b9c06dab0 2024-04-25T10:13:03.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 -- 172.21.15.87:0/1955410251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 msgr2=0x7f0b9c0717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 --2- 172.21.15.87:0/1955410251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c0717a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0ae990 tx=0x7f0b8401c470 comp rx=0 tx=0).stop 2024-04-25T10:13:03.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 -- 172.21.15.87:0/1955410251 shutdown_connections 2024-04-25T10:13:03.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 --2- 172.21.15.87:0/1955410251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c0717a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 --2- 172.21.15.87:0/1955410251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c147640 0x7f0b9c0ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 -- 172.21.15.87:0/1955410251 >> 172.21.15.87:0/1955410251 conn(0x7f0b9c0b21c0 msgr2=0x7f0b9c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 -- 172.21.15.87:0/1955410251 shutdown_connections 2024-04-25T10:13:03.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.127+0000 7f0ba3000640 1 -- 172.21.15.87:0/1955410251 wait complete. 2024-04-25T10:13:03.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 Processor -- start 2024-04-25T10:13:03.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 -- start start 2024-04-25T10:13:03.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c071ce0 0x7f0b9c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b9c147640 0x7f0b9c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c06df10 con 0x7f0b9c06dab0 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c06ee10 con 0x7f0b9c147640 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c16a8e0 con 0x7f0b9c071ce0 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c15fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55030/0 (socket says 172.21.15.87:55030) 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 -- 172.21.15.87:0/374247499 learned_addr learned my addr 172.21.15.87:0/374247499 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b93400640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c071ce0 0x7f0b9c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba1000640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b9c147640 0x7f0b9c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 -- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c071ce0 msgr2=0x7f0b9c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c071ce0 0x7f0b9c160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 -- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b9c147640 msgr2=0x7f0b9c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b9c147640 0x7f0b9c15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 -- 172.21.15.87:0/374247499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c160540 con 0x7f0b9c06dab0 2024-04-25T10:13:03.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b9be00640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c15fa00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00d620 tx=0x7f0b8c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b98c00640 1 -- 172.21.15.87:0/374247499 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c00b620 con 0x7f0b9c06dab0 2024-04-25T10:13:03.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b98c00640 1 -- 172.21.15.87:0/374247499 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c019070 con 0x7f0b9c06dab0 2024-04-25T10:13:03.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0b98c00640 1 -- 172.21.15.87:0/374247499 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c00b9e0 con 0x7f0b9c06dab0 2024-04-25T10:13:03.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c13c020 con 0x7f0b9c06dab0 2024-04-25T10:13:03.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c13c5e0 con 0x7f0b9c06dab0 2024-04-25T10:13:03.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.131+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c04ef30 con 0x7f0b9c06dab0 2024-04-25T10:13:03.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.135+0000 7f0b98c00640 1 -- 172.21.15.87:0/374247499 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b8c00bb80 con 0x7f0b9c06dab0 2024-04-25T10:13:03.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.135+0000 7f0b98c00640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b78066f90 0x7f0b78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.135+0000 7f0b98c00640 1 -- 172.21.15.87:0/374247499 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b8c0853b0 con 0x7f0b9c06dab0 2024-04-25T10:13:03.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.135+0000 7f0b93400640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b78066f90 0x7f0b78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.135+0000 7f0b93400640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b78066f90 0x7f0b78069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b840af000 tx=0x7f0b840a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.139+0000 7f0b98c00640 1 -- 172.21.15.87:0/374247499 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b8c056450 con 0x7f0b9c06dab0 2024-04-25T10:13:03.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.295+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b9c06b9d0 con 0x7f0b9c06dab0 2024-04-25T10:13:03.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.295+0000 7f0b98c00640 1 -- 172.21.15.87:0/374247499 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b8c055d50 con 0x7f0b9c06dab0 2024-04-25T10:13:03.304 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b78066f90 msgr2=0x7f0b78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b78066f90 0x7f0b78069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b840af000 tx=0x7f0b840a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 msgr2=0x7f0b9c15fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c15fa00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00d620 tx=0x7f0b8c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 shutdown_connections 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b78066f90 0x7f0b78069450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b9c147640 0x7f0b9c15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b9c071ce0 0x7f0b9c160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 --2- 172.21.15.87:0/374247499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b9c06dab0 0x7f0b9c15fa00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 >> 172.21.15.87:0/374247499 conn(0x7f0b9c0b21c0 msgr2=0x7f0b9c160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 shutdown_connections 2024-04-25T10:13:03.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.299+0000 7f0ba3000640 1 -- 172.21.15.87:0/374247499 wait complete. 2024-04-25T10:13:03.325 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f0781000640 1 Processor -- start 2024-04-25T10:13:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f0781000640 1 -- start start 2024-04-25T10:13:03.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f0781000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f077c0b2e70 0x7f077c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f0781000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0cc080 con 0x7f077c0cd7f0 2024-04-25T10:13:03.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f0781000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0cc320 con 0x7f077c06aea0 2024-04-25T10:13:03.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f0781000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c0cc600 con 0x7f077c0b2e70 2024-04-25T10:13:03.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f077a800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f077c0cd7f0 0x7f077c15d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34866/0 (socket says 172.21.15.87:34866) 2024-04-25T10:13:03.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.455+0000 7f077a800640 1 -- 172.21.15.87:0/402845346 learned_addr learned my addr 172.21.15.87:0/402845346 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779e00640 1 --2- 172.21.15.87:0/402845346 >> v2:172.21.15.169:3300/0 conn(0x7f077c0b2e70 0x7f077c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117254149 0 0) 0x7f077c0cc080 con 0x7f077c0cd7f0 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0760003530 con 0x7f077c0cd7f0 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198214136 0 0) 0x7f0760003530 con 0x7f077c0cd7f0 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 >> v2:172.21.15.169:3300/0 conn(0x7f077c0b2e70 msgr2=0x7f077c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 --2- 172.21.15.87:0/402845346 >> v2:172.21.15.169:3300/0 conn(0x7f077c0b2e70 0x7f077c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 >> v1:172.21.15.169:6789/0 conn(0x7f077c06aea0 legacy=0x7f077c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0cc8e0 con 0x7f077c0cd7f0 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0774001a20 con 0x7f077c0cd7f0 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 >> v1:172.21.15.87:6789/0 conn(0x7f077c0cd7f0 legacy=0x7f077c15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07600039c0 0x7f0760003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0760004810 0x7f0760004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07600051d0 0x7f076000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0cc600 con 0x7f07600039c0 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076000c080 con 0x7f07600051d0 2024-04-25T10:13:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076000c280 con 0x7f0760004810 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779e00640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07600039c0 0x7f0760003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f077a800640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07600051d0 0x7f076000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779400640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0760004810 0x7f0760004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779e00640 1 -- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0760004810 msgr2=0x7f0760004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779e00640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0760004810 0x7f0760004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779e00640 1 -- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07600051d0 msgr2=0x7f076000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779e00640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07600051d0 0x7f076000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f0779e00640 1 -- 172.21.15.87:0/402845346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0760003530 con 0x7f07600039c0 2024-04-25T10:13:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.459+0000 7f077a800640 1 -- 172.21.15.87:0/402845346 reap_dead start 2024-04-25T10:13:03.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0779e00640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07600039c0 0x7f0760003de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f076c009ae0 tx=0x7f076c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bccaa1724354a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c0b2070 con 0x7f07600039c0 2024-04-25T10:13:03.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c0a5550 con 0x7f07600039c0 2024-04-25T10:13:03.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0778a00640 1 -- 172.21.15.87:0/402845346 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c0096b0 con 0x7f07600039c0 2024-04-25T10:13:03.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- 172.21.15.87:0/402845346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07600039c0 msgr2=0x7f0760003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07600039c0 0x7f0760003de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f076c009ae0 tx=0x7f076c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:13:03.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- 172.21.15.87:0/402845346 shutdown_connections 2024-04-25T10:13:03.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 --2- 172.21.15.87:0/402845346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07600039c0 0x7f0760003de0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- 172.21.15.87:0/402845346 >> 172.21.15.87:0/402845346 conn(0x7f077c0b34a0 msgr2=0x7f077c0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- 172.21.15.87:0/402845346 shutdown_connections 2024-04-25T10:13:03.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- 172.21.15.87:0/402845346 wait complete. 2024-04-25T10:13:03.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 Processor -- start 2024-04-25T10:13:03.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- start start 2024-04-25T10:13:03.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07600039c0 0x7f077c14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f077c15b300 0x7f077c0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f077c0c5a00 0x7f077c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0cdc10 con 0x7f077c0c5a00 2024-04-25T10:13:03.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0b3250 con 0x7f077c15b300 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0781000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0cdd90 con 0x7f07600039c0 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f077a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f077c0c5a00 0x7f077c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f077a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f077c0c5a00 0x7f077c1581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55038/0 (socket says 172.21.15.87:55038) 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f077a800640 1 -- 172.21.15.87:0/4086761073 learned_addr learned my addr 172.21.15.87:0/4086761073 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0779400640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f077c15b300 0x7f077c0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.463+0000 7f0779e00640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07600039c0 0x7f077c14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f077a800640 1 -- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07600039c0 msgr2=0x7f077c14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f077a800640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07600039c0 0x7f077c14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f077a800640 1 -- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f077c15b300 msgr2=0x7f077c0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f077a800640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f077c15b300 0x7f077c0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f077a800640 1 -- 172.21.15.87:0/4086761073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0c5e80 con 0x7f077c0c5a00 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f077a800640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f077c0c5a00 0x7f077c1581d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cc250 tx=0x7f0774011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0772000640 1 -- 172.21.15.87:0/4086761073 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774010040 con 0x7f077c0c5a00 2024-04-25T10:13:03.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0772000640 1 -- 172.21.15.87:0/4086761073 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077401a070 con 0x7f077c0c5a00 2024-04-25T10:13:03.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c0c6160 con 0x7f077c0c5a00 2024-04-25T10:13:03.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0772000640 1 -- 172.21.15.87:0/4086761073 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740139e0 con 0x7f077c0c5a00 2024-04-25T10:13:03.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c0c6a00 con 0x7f077c0c5a00 2024-04-25T10:13:03.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c04ef30 con 0x7f077c0c5a00 2024-04-25T10:13:03.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0772000640 1 -- 172.21.15.87:0/4086761073 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0774013b80 con 0x7f077c0c5a00 2024-04-25T10:13:03.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0772000640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f075c066f60 0x7f075c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0772000640 1 -- 172.21.15.87:0/4086761073 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0774087df0 con 0x7f077c0c5a00 2024-04-25T10:13:03.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.467+0000 7f0779e00640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f075c066f60 0x7f075c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.471+0000 7f0779e00640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f075c066f60 0x7f075c069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f076c0af000 tx=0x7f076c0046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.471+0000 7f0772000640 1 -- 172.21.15.87:0/4086761073 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07740587c0 con 0x7f077c0c5a00 2024-04-25T10:13:03.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f077c06b480 con 0x7f077c0c5a00 2024-04-25T10:13:03.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0772000640 1 -- 172.21.15.87:0/4086761073 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f077405c860 con 0x7f077c0c5a00 2024-04-25T10:13:03.633 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f075c066f60 msgr2=0x7f075c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f075c066f60 0x7f075c069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f076c0af000 tx=0x7f076c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f077c0c5a00 msgr2=0x7f077c1581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f077c0c5a00 0x7f077c1581d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cc250 tx=0x7f0774011a70 comp rx=0 tx=0).stop 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 shutdown_connections 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f075c066f60 0x7f075c069420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f077c0c5a00 0x7f077c1581d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f077c15b300 0x7f077c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 --2- 172.21.15.87:0/4086761073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07600039c0 0x7f077c14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 >> 172.21.15.87:0/4086761073 conn(0x7f077c0b34a0 msgr2=0x7f077c06f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 shutdown_connections 2024-04-25T10:13:03.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.627+0000 7f0781000640 1 -- 172.21.15.87:0/4086761073 wait complete. 2024-04-25T10:13:03.654 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:03.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91e5000640 1 Processor -- start 2024-04-25T10:13:03.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91e5000640 1 -- start start 2024-04-25T10:13:03.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91e5000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91e015df00 0x7f91e00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91e5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d1eb0 0x7f91e00b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91e5000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00d0c00 con 0x7f91e00d0750 2024-04-25T10:13:03.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91e5000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00d2490 con 0x7f91e015df00 2024-04-25T10:13:03.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91e5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00d2770 con 0x7f91e00d1eb0 2024-04-25T10:13:03.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91dde00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91e015df00 0x7f91e00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91dde00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91e015df00 0x7f91e00d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55044/0 (socket says 172.21.15.87:55044) 2024-04-25T10:13:03.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91dde00640 1 -- 172.21.15.87:0/2023433433 learned_addr learned my addr 172.21.15.87:0/2023433433 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91dde00640 1 -- 172.21.15.87:0/2023433433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d1eb0 msgr2=0x7f91e00b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91de800640 1 --2- 172.21.15.87:0/2023433433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d1eb0 0x7f91e00b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91dde00640 1 --2- 172.21.15.87:0/2023433433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d1eb0 0x7f91e00b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91dde00640 1 -- 172.21.15.87:0/2023433433 >> v1:172.21.15.169:6789/0 conn(0x7f91e00d0750 legacy=0x7f91e00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91dde00640 1 -- 172.21.15.87:0/2023433433 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00d2a50 con 0x7f91e015df00 2024-04-25T10:13:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.783+0000 7f91de800640 1 --2- 172.21.15.87:0/2023433433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d1eb0 0x7f91e00b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91dde00640 1 --2- 172.21.15.87:0/2023433433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e015df00 0x7f91e00d0210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009870 tx=0x7f91d001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9644154c4696b34b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91dca00640 1 -- 172.21.15.87:0/2023433433 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00ab640 con 0x7f91e015df00 2024-04-25T10:13:03.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91dca00640 1 -- 172.21.15.87:0/2023433433 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00b0070 con 0x7f91e015df00 2024-04-25T10:13:03.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91dca00640 1 -- 172.21.15.87:0/2023433433 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00aba00 con 0x7f91e015df00 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 -- 172.21.15.87:0/2023433433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e015df00 msgr2=0x7f91e00d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 --2- 172.21.15.87:0/2023433433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e015df00 0x7f91e00d0210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009870 tx=0x7f91d001c530 comp rx=0 tx=0).stop 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 -- 172.21.15.87:0/2023433433 shutdown_connections 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 --2- 172.21.15.87:0/2023433433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d1eb0 0x7f91e00b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 --2- 172.21.15.87:0/2023433433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e015df00 0x7f91e00d0210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 -- 172.21.15.87:0/2023433433 >> 172.21.15.87:0/2023433433 conn(0x7f91e006b290 msgr2=0x7f91e00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 -- 172.21.15.87:0/2023433433 shutdown_connections 2024-04-25T10:13:03.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.787+0000 7f91e5000640 1 -- 172.21.15.87:0/2023433433 wait complete. 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 Processor -- start 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 -- start start 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d0750 0x7f91e0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00d1eb0 0x7f91e00d4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e015df00 0x7f91e0144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00b80b0 con 0x7f91e00d1eb0 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00b7f30 con 0x7f91e015df00 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00b8230 con 0x7f91e00d0750 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00d1eb0 0x7f91e00d4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00d1eb0 0x7f91e00d4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55052/0 (socket says 172.21.15.87:55052) 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 -- 172.21.15.87:0/1081671774 learned_addr learned my addr 172.21.15.87:0/1081671774 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91de800640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e015df00 0x7f91e0144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dde00640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d0750 0x7f91e0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 -- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d0750 msgr2=0x7f91e0151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d0750 0x7f91e0151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 -- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e015df00 msgr2=0x7f91e0144070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e015df00 0x7f91e0144070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 -- 172.21.15.87:0/1081671774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00d46d0 con 0x7f91e00d1eb0 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91dd400640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00d1eb0 0x7f91e00d4190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c420 tx=0x7f91cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91d6000640 1 -- 172.21.15.87:0/1081671774 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0052d0 con 0x7f91e00d1eb0 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91d6000640 1 -- 172.21.15.87:0/1081671774 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc005470 con 0x7f91e00d1eb0 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e014fed0 con 0x7f91e00d1eb0 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91d6000640 1 -- 172.21.15.87:0/1081671774 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc007260 con 0x7f91e00d1eb0 2024-04-25T10:13:03.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.791+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0150490 con 0x7f91e00d1eb0 2024-04-25T10:13:03.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.795+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e004ef30 con 0x7f91e00d1eb0 2024-04-25T10:13:03.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.795+0000 7f91d6000640 1 -- 172.21.15.87:0/1081671774 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91cc007400 con 0x7f91e00d1eb0 2024-04-25T10:13:03.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.795+0000 7f91d6000640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f90 0x7f91c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.795+0000 7f91d6000640 1 -- 172.21.15.87:0/1081671774 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91cc084510 con 0x7f91e00d1eb0 2024-04-25T10:13:03.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.795+0000 7f91dde00640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f90 0x7f91c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.795+0000 7f91dde00640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f90 0x7f91c0069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af000 tx=0x7f91d00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.799+0000 7f91d6000640 1 -- 172.21.15.87:0/1081671774 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91cc055490 con 0x7f91e00d1eb0 2024-04-25T10:13:03.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.987+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91e00c73f0 con 0x7f91e00d1eb0 2024-04-25T10:13:03.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91d6000640 1 -- 172.21.15.87:0/1081671774 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f91cc054d90 con 0x7f91e00d1eb0 2024-04-25T10:13:03.996 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:03.996 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:13:03.996 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:13:03.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f90 msgr2=0x7f91c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f90 0x7f91c0069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af000 tx=0x7f91d00a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00d1eb0 msgr2=0x7f91e00d4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00d1eb0 0x7f91e00d4190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c420 tx=0x7f91cc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 shutdown_connections 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f90 0x7f91c0069450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e015df00 0x7f91e0144070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00d1eb0 0x7f91e00d4190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 --2- 172.21.15.87:0/1081671774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e00d0750 0x7f91e0151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 >> 172.21.15.87:0/1081671774 conn(0x7f91e006b290 msgr2=0x7f91e00c6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 shutdown_connections 2024-04-25T10:13:04.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:03.991+0000 7f91e5000640 1 -- 172.21.15.87:0/1081671774 wait complete. 2024-04-25T10:13:04.016 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:04.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3ace00640 1 Processor -- start 2024-04-25T10:13:04.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3ace00640 1 -- start start 2024-04-25T10:13:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3ace00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3a8066c00 0x7fe3a815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3ace00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80b5970 con 0x7fe3a8066fe0 2024-04-25T10:13:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3ace00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80b5c20 con 0x7fe3a80b5560 2024-04-25T10:13:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3ace00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a80af6e0 con 0x7fe3a8066c00 2024-04-25T10:13:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3a5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3a8066c00 0x7fe3a815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe39d400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe3a8066fe0 0x7fe3a806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34872/0 (socket says 172.21.15.87:34872) 2024-04-25T10:13:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe39d400640 1 -- 172.21.15.87:0/1418001628 learned_addr learned my addr 172.21.15.87:0/1418001628 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3a5e00640 1 -- 172.21.15.87:0/1418001628 >> v1:172.21.15.169:6789/0 conn(0x7fe3a80b5560 legacy=0x7fe3a8157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3a5e00640 1 -- 172.21.15.87:0/1418001628 >> v1:172.21.15.87:6789/0 conn(0x7fe3a8066fe0 legacy=0x7fe3a806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3a5400640 1 -- 172.21.15.87:0/1418001628 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754958496 0 0) 0x7fe3a80b5970 con 0x7fe3a8066fe0 2024-04-25T10:13:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.147+0000 7fe3a5e00640 1 -- 172.21.15.87:0/1418001628 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80af9c0 con 0x7fe3a8066c00 2024-04-25T10:13:04.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3a5e00640 1 --2- 172.21.15.87:0/1418001628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a8066c00 0x7fe3a815e110 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe39801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4beb49275a156e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3a5400640 1 -- 172.21.15.87:0/1418001628 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980b2070 con 0x7fe3a8066c00 2024-04-25T10:13:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3a5400640 1 -- 172.21.15.87:0/1418001628 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980a59e0 con 0x7fe3a8066c00 2024-04-25T10:13:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3a5400640 1 -- 172.21.15.87:0/1418001628 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980a5db0 con 0x7fe3a8066c00 2024-04-25T10:13:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3ace00640 1 -- 172.21.15.87:0/1418001628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a8066c00 msgr2=0x7fe3a815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3ace00640 1 --2- 172.21.15.87:0/1418001628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a8066c00 0x7fe3a815e110 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe39801c900 comp rx=0 tx=0).stop 2024-04-25T10:13:04.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3ace00640 1 -- 172.21.15.87:0/1418001628 shutdown_connections 2024-04-25T10:13:04.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3ace00640 1 --2- 172.21.15.87:0/1418001628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a8066c00 0x7fe3a815e110 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3ace00640 1 -- 172.21.15.87:0/1418001628 >> 172.21.15.87:0/1418001628 conn(0x7fe3a80b24f0 msgr2=0x7fe3a80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3ace00640 1 -- 172.21.15.87:0/1418001628 shutdown_connections 2024-04-25T10:13:04.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.151+0000 7fe3ace00640 1 -- 172.21.15.87:0/1418001628 wait complete. 2024-04-25T10:13:04.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 Processor -- start 2024-04-25T10:13:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 -- start start 2024-04-25T10:13:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3a8066c00 0x7fe3a8151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3a8066fe0 0x7fe3a81512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a80b5560 0x7fe3a8160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a816bb00 con 0x7fe3a80b5560 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80af270 con 0x7fe3a8066c00 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a816bc80 con 0x7fe3a8066fe0 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a80b5560 0x7fe3a8160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a80b5560 0x7fe3a8160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55074/0 (socket says 172.21.15.87:55074) 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 -- 172.21.15.87:0/211546108 learned_addr learned my addr 172.21.15.87:0/211546108 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe39d400640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3a8066fe0 0x7fe3a81512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a5e00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3a8066c00 0x7fe3a8151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 -- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3a8066fe0 msgr2=0x7fe3a81512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3a8066fe0 0x7fe3a81512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 -- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3a8066c00 msgr2=0x7fe3a8151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3a8066c00 0x7fe3a8151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 -- 172.21.15.87:0/211546108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a81fa5b0 con 0x7fe3a80b5560 2024-04-25T10:13:04.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3a6800640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a80b5560 0x7fe3a8160030 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000b440 tx=0x7fe3a000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe39ea00640 1 -- 172.21.15.87:0/211546108 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0015020 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe39ea00640 1 -- 172.21.15.87:0/211546108 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0011850 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe39ea00640 1 -- 172.21.15.87:0/211546108 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0011c40 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a81fa850 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a81fae10 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.155+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a804ef30 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.159+0000 7fe39ea00640 1 -- 172.21.15.87:0/211546108 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3a0018020 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.159+0000 7fe39ea00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.159+0000 7fe39ea00640 1 -- 172.21.15.87:0/211546108 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe3a0084a80 con 0x7fe3a80b5560 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.159+0000 7fe3a5e00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.159+0000 7fe3a5e00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe384066f60 0x7fe384069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3980af000 tx=0x7fe3980a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.163+0000 7fe39ea00640 1 -- 172.21.15.87:0/211546108 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3a0086e20 con 0x7fe3a80b5560 2024-04-25T10:13:04.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.327+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3a8152580 con 0x7fe3a80b5560 2024-04-25T10:13:04.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe39ea00640 1 -- 172.21.15.87:0/211546108 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16843 (secure 0 0 0) 0x7fe3a001f030 con 0x7fe3a80b5560 2024-04-25T10:13:04.337 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:04.337 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":21,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:13:03.947022+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe384066f60 msgr2=0x7fe384069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe384066f60 0x7fe384069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3980af000 tx=0x7fe3980a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a80b5560 msgr2=0x7fe3a8160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a80b5560 0x7fe3a8160030 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000b440 tx=0x7fe3a000b900 comp rx=0 tx=0).stop 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 shutdown_connections 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3a80b5560 0x7fe3a8160030 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3a8066fe0 0x7fe3a81512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 --2- 172.21.15.87:0/211546108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3a8066c00 0x7fe3a8151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 >> 172.21.15.87:0/211546108 conn(0x7fe3a80b24f0 msgr2=0x7fe3a81540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 shutdown_connections 2024-04-25T10:13:04.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.331+0000 7fe3ace00640 1 -- 172.21.15.87:0/211546108 wait complete. 2024-04-25T10:13:04.358 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:13:04.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:04.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ad6800640 1 Processor -- start 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ad6800640 1 -- start start 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ad6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8ad00ccb20 0x7f8ad00d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ad6800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad00ccfb0 con 0x7f8ad015bdc0 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ad6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad00cd290 con 0x7f8ad015b9e0 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ad6800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad00cd570 con 0x7f8ad00ccb20 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ac6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8ad00ccb20 0x7f8ad00d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8acfe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8ad015bdc0 0x7f8ad00c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34886/0 (socket says 172.21.15.87:34886) 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8acfe00640 1 -- 172.21.15.87:0/1269014791 learned_addr learned my addr 172.21.15.87:0/1269014791 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:04.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ac6a00640 1 -- 172.21.15.87:0/1269014791 >> v1:172.21.15.169:6789/0 conn(0x7f8ad015b9e0 legacy=0x7f8ad00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ac6a00640 1 -- 172.21.15.87:0/1269014791 >> v1:172.21.15.87:6789/0 conn(0x7f8ad015bdc0 legacy=0x7f8ad00c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8ac6a00640 1 -- 172.21.15.87:0/1269014791 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00cd850 con 0x7f8ad00ccb20 2024-04-25T10:13:04.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.487+0000 7f8acea00640 1 -- 172.21.15.87:0/1269014791 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735512367 0 0) 0x7f8ad00ccfb0 con 0x7f8ad015bdc0 2024-04-25T10:13:04.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ac6a00640 1 --2- 172.21.15.87:0/1269014791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad00ccb20 0x7f8ad00d1520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80097c0 tx=0x7f8ab801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ea2f2f8dd9b6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8acea00640 1 -- 172.21.15.87:0/1269014791 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80b1070 con 0x7f8ad00ccb20 2024-04-25T10:13:04.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8acea00640 1 -- 172.21.15.87:0/1269014791 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab80ac640 con 0x7f8ad00ccb20 2024-04-25T10:13:04.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8acea00640 1 -- 172.21.15.87:0/1269014791 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80aca10 con 0x7f8ad00ccb20 2024-04-25T10:13:04.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ad6800640 1 -- 172.21.15.87:0/1269014791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad00ccb20 msgr2=0x7f8ad00d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ad6800640 1 --2- 172.21.15.87:0/1269014791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad00ccb20 0x7f8ad00d1520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80097c0 tx=0x7f8ab801c460 comp rx=0 tx=0).stop 2024-04-25T10:13:04.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ad6800640 1 -- 172.21.15.87:0/1269014791 shutdown_connections 2024-04-25T10:13:04.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ad6800640 1 --2- 172.21.15.87:0/1269014791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad00ccb20 0x7f8ad00d1520 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ad6800640 1 -- 172.21.15.87:0/1269014791 >> 172.21.15.87:0/1269014791 conn(0x7f8ad009a9d0 msgr2=0x7f8ad009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ad6800640 1 -- 172.21.15.87:0/1269014791 shutdown_connections 2024-04-25T10:13:04.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.491+0000 7f8ad6800640 1 -- 172.21.15.87:0/1269014791 wait complete. 2024-04-25T10:13:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 Processor -- start 2024-04-25T10:13:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 -- start start 2024-04-25T10:13:04.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad015b9e0 0x7f8ad00c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad015bdc0 0x7f8ad00c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad01f7190 0x7f8ad01f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00c6ab0 con 0x7f8ad015b9e0 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00c6930 con 0x7f8ad01f7190 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00c6c30 con 0x7f8ad015bdc0 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad015b9e0 0x7f8ad00c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad015b9e0 0x7f8ad00c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55100/0 (socket says 172.21.15.87:55100) 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 -- 172.21.15.87:0/3248299034 learned_addr learned my addr 172.21.15.87:0/3248299034 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ac6a00640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad015bdc0 0x7f8ad00c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acfe00640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad01f7190 0x7f8ad01f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 -- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad015bdc0 msgr2=0x7f8ad00c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad015bdc0 0x7f8ad00c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 -- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad01f7190 msgr2=0x7f8ad01f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad01f7190 0x7f8ad01f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 -- 172.21.15.87:0/3248299034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00701f0 con 0x7f8ad015b9e0 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acfe00640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad01f7190 0x7f8ad01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8acf400640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad015b9e0 0x7f8ad00c89d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c480 tx=0x7f8ac000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ac7e00640 1 -- 172.21.15.87:0/3248299034 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0012830 con 0x7f8ad015b9e0 2024-04-25T10:13:04.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ac7e00640 1 -- 172.21.15.87:0/3248299034 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac0017070 con 0x7f8ad015b9e0 2024-04-25T10:13:04.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ac7e00640 1 -- 172.21.15.87:0/3248299034 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0012c30 con 0x7f8ad015b9e0 2024-04-25T10:13:04.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad00cbb60 con 0x7f8ad015b9e0 2024-04-25T10:13:04.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad00c9500 con 0x7f8ad015b9e0 2024-04-25T10:13:04.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.495+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad004ef30 con 0x7f8ad015b9e0 2024-04-25T10:13:04.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.499+0000 7f8ac7e00640 1 -- 172.21.15.87:0/3248299034 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ac001c460 con 0x7f8ad015b9e0 2024-04-25T10:13:04.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.499+0000 7f8ac7e00640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8aac066f90 0x7f8aac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.499+0000 7f8ac7e00640 1 -- 172.21.15.87:0/3248299034 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8ac0084e30 con 0x7f8ad015b9e0 2024-04-25T10:13:04.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.499+0000 7f8ac6a00640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8aac066f90 0x7f8aac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.499+0000 7f8ac6a00640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8aac066f90 0x7f8aac069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ae000 tx=0x7f8ab80a6070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.503+0000 7f8ac7e00640 1 -- 172.21.15.87:0/3248299034 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ac0055db0 con 0x7f8ad015b9e0 2024-04-25T10:13:04.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.687+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ad00cf1a0 con 0x7f8ad015b9e0 2024-04-25T10:13:04.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.687+0000 7f8ac7e00640 1 -- 172.21.15.87:0/3248299034 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f8ac00556b0 con 0x7f8ad015b9e0 2024-04-25T10:13:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:04.695 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:13:04.695 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8aac066f90 msgr2=0x7f8aac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8aac066f90 0x7f8aac069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ae000 tx=0x7f8ab80a6070 comp rx=0 tx=0).stop 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad015b9e0 msgr2=0x7f8ad00c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad015b9e0 0x7f8ad00c89d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c480 tx=0x7f8ac000c940 comp rx=0 tx=0).stop 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 shutdown_connections 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8aac066f90 0x7f8aac069450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad01f7190 0x7f8ad01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad015bdc0 0x7f8ad00c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 --2- 172.21.15.87:0/3248299034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad015b9e0 0x7f8ad00c89d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 >> 172.21.15.87:0/3248299034 conn(0x7f8ad009a9d0 msgr2=0x7f8ad006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 shutdown_connections 2024-04-25T10:13:04.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.691+0000 7f8ad6800640 1 -- 172.21.15.87:0/3248299034 wait complete. 2024-04-25T10:13:04.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f07a5a00640 1 Processor -- start 2024-04-25T10:13:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f07a5a00640 1 -- start start 2024-04-25T10:13:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f07a5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f07a5a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07a00d1eb0 0x7f07a00b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f07a5a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00d2490 con 0x7f07a00d0750 2024-04-25T10:13:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f07a5a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00d2770 con 0x7f07a00d1eb0 2024-04-25T10:13:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f07a5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00d2a50 con 0x7f07a015df00 2024-04-25T10:13:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07a00d1eb0 0x7f07a00b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07a00d1eb0 0x7f07a00b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55112/0 (socket says 172.21.15.87:55112) 2024-04-25T10:13:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079f400640 1 -- 172.21.15.87:0/1034860272 learned_addr learned my addr 172.21.15.87:0/1034860272 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079f400640 1 -- 172.21.15.87:0/1034860272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 msgr2=0x7f07a00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079ea00640 1 --2- 172.21.15.87:0/1034860272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079f400640 1 --2- 172.21.15.87:0/1034860272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079f400640 1 -- 172.21.15.87:0/1034860272 >> v1:172.21.15.169:6789/0 conn(0x7f07a00d0750 legacy=0x7f07a00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079f400640 1 -- 172.21.15.87:0/1034860272 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00d2d30 con 0x7f07a00d1eb0 2024-04-25T10:13:04.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.851+0000 7f079ea00640 1 --2- 172.21.15.87:0/1034860272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f079f400640 1 --2- 172.21.15.87:0/1034860272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a00b71e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0798009e50 tx=0x7f079801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a6c44d24aada50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f079d600640 1 -- 172.21.15.87:0/1034860272 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980a5820 con 0x7f07a00d1eb0 2024-04-25T10:13:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f079d600640 1 -- 172.21.15.87:0/1034860272 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07980a59c0 con 0x7f07a00d1eb0 2024-04-25T10:13:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f079d600640 1 -- 172.21.15.87:0/1034860272 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980a5d90 con 0x7f07a00d1eb0 2024-04-25T10:13:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 -- 172.21.15.87:0/1034860272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 msgr2=0x7f07a00b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1034860272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a00b71e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0798009e50 tx=0x7f079801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 -- 172.21.15.87:0/1034860272 shutdown_connections 2024-04-25T10:13:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1034860272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a00b71e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1034860272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 -- 172.21.15.87:0/1034860272 >> 172.21.15.87:0/1034860272 conn(0x7f07a006b290 msgr2=0x7f07a00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 -- 172.21.15.87:0/1034860272 shutdown_connections 2024-04-25T10:13:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.855+0000 7f07a5a00640 1 -- 172.21.15.87:0/1034860272 wait complete. 2024-04-25T10:13:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 Processor -- start 2024-04-25T10:13:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 -- start start 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a00d0750 0x7f07a0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a0153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b80b0 con 0x7f07a00d1eb0 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b7f30 con 0x7f07a00d0750 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b8230 con 0x7f07a015df00 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a0153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a00d0750 0x7f07a0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a0153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55128/0 (socket says 172.21.15.87:55128) 2024-04-25T10:13:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 -- 172.21.15.87:0/1969929464 learned_addr learned my addr 172.21.15.87:0/1969929464 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079f400640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 -- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 msgr2=0x7f07a00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 -- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a00d0750 msgr2=0x7f07a0150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a00d0750 0x7f07a0150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 -- 172.21.15.87:0/1969929464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0154380 con 0x7f07a00d1eb0 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079f400640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f079e000640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a0153be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f079000c480 tx=0x7f079000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f0796a00640 1 -- 172.21.15.87:0/1969929464 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790012830 con 0x7f07a00d1eb0 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f0796a00640 1 -- 172.21.15.87:0/1969929464 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790017070 con 0x7f07a00d1eb0 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f0796a00640 1 -- 172.21.15.87:0/1969929464 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790012c30 con 0x7f07a00d1eb0 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a0154660 con 0x7f07a00d1eb0 2024-04-25T10:13:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a00cad90 con 0x7f07a00d1eb0 2024-04-25T10:13:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.859+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a00caf90 con 0x7f07a00d1eb0 2024-04-25T10:13:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.863+0000 7f0796a00640 1 -- 172.21.15.87:0/1969929464 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f079001c460 con 0x7f07a00d1eb0 2024-04-25T10:13:04.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.863+0000 7f0796a00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f077c066f60 0x7f077c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.863+0000 7f0796a00640 1 -- 172.21.15.87:0/1969929464 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0790084df0 con 0x7f07a00d1eb0 2024-04-25T10:13:04.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.863+0000 7f079ea00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f077c066f60 0x7f077c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.863+0000 7f079ea00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f077c066f60 0x7f077c069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0788006fd0 tx=0x7f0788008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:04.867+0000 7f0796a00640 1 -- 172.21.15.87:0/1969929464 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07a00caf90 con 0x7f07a00d1eb0 2024-04-25T10:13:05.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.059+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07a004ef30 con 0x7f07a00d1eb0 2024-04-25T10:13:05.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.059+0000 7f0796a00640 1 -- 172.21.15.87:0/1969929464 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f0790055e10 con 0x7f07a00d1eb0 2024-04-25T10:13:05.067 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:05.067 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3680389095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3680389095},{"type":"v1","addr":"172.21.15.169:6834","nonce":3680389095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/221434036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":221434036},{"type":"v1","addr":"172.21.15.169:6835","nonce":221434036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2707660301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2707660301},{"type":"v1","addr":"172.21.15.87:6836","nonce":2707660301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:23.612551+0000","modified":"2024-04-25T10:12:26.623883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/531083715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":531083715},{"type":"v1","addr":"172.21.15.87:6837","nonce":531083715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:13:05.068 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f077c066f60 msgr2=0x7f077c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f077c066f60 0x7f077c069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0788006fd0 tx=0x7f0788008040 comp rx=0 tx=0).stop 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 msgr2=0x7f07a0153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a0153be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f079000c480 tx=0x7f079000c940 comp rx=0 tx=0).stop 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 shutdown_connections 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f077c066f60 0x7f077c069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07a015df00 0x7f07a00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a00d1eb0 0x7f07a0153be0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 --2- 172.21.15.87:0/1969929464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a00d0750 0x7f07a0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 >> 172.21.15.87:0/1969929464 conn(0x7f07a006b290 msgr2=0x7f07a0152540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 shutdown_connections 2024-04-25T10:13:05.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.063+0000 7f07a5a00640 1 -- 172.21.15.87:0/1969929464 wait complete. 2024-04-25T10:13:05.087 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:05.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.215+0000 7fad91400640 1 Processor -- start 2024-04-25T10:13:05.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.215+0000 7fad91400640 1 -- start start 2024-04-25T10:13:05.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad91400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066c00 0x7fad8c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad91400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad8c066fe0 0x7fad8c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad91400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c0b5c20 con 0x7fad8c06ed30 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad91400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c0af6e0 con 0x7fad8c066fe0 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad91400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0af9c0 con 0x7fad8c066c00 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad8b200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fad8c06ed30 0x7fad8c157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34894/0 (socket says 172.21.15.87:34894) 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad8b200640 1 -- 172.21.15.87:0/3027972567 learned_addr learned my addr 172.21.15.87:0/3027972567 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad81e00640 1 --2- 172.21.15.87:0/3027972567 >> v2:172.21.15.169:3300/0 conn(0x7fad8c066fe0 0x7fad8c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad8a800640 1 --2- 172.21.15.87:0/3027972567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066c00 0x7fad8c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad89e00640 1 -- 172.21.15.87:0/3027972567 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721760587 0 0) 0x7fad8c0b5c20 con 0x7fad8c06ed30 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad89e00640 1 -- 172.21.15.87:0/3027972567 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad70003530 con 0x7fad8c06ed30 2024-04-25T10:13:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad8a800640 1 -- 172.21.15.87:0/3027972567 >> v2:172.21.15.169:3300/0 conn(0x7fad8c066fe0 msgr2=0x7fad8c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:05.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad8a800640 1 --2- 172.21.15.87:0/3027972567 >> v2:172.21.15.169:3300/0 conn(0x7fad8c066fe0 0x7fad8c0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad8a800640 1 -- 172.21.15.87:0/3027972567 >> v1:172.21.15.87:6789/0 conn(0x7fad8c06ed30 legacy=0x7fad8c157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.219+0000 7fad8a800640 1 -- 172.21.15.87:0/3027972567 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0afca0 con 0x7fad8c066c00 2024-04-25T10:13:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad8a800640 1 --2- 172.21.15.87:0/3027972567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066c00 0x7fad8c15e110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009d90 tx=0x7fad7c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3415fc584eb28c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad89e00640 1 -- 172.21.15.87:0/3027972567 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0b2070 con 0x7fad8c066c00 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad89e00640 1 -- 172.21.15.87:0/3027972567 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c0a5a20 con 0x7fad8c066c00 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad89e00640 1 -- 172.21.15.87:0/3027972567 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0a5dd0 con 0x7fad8c066c00 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 -- 172.21.15.87:0/3027972567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066c00 msgr2=0x7fad8c15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 --2- 172.21.15.87:0/3027972567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066c00 0x7fad8c15e110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009d90 tx=0x7fad7c01c940 comp rx=0 tx=0).stop 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 -- 172.21.15.87:0/3027972567 shutdown_connections 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 --2- 172.21.15.87:0/3027972567 >> v2:172.21.15.169:3300/0 conn(0x7fad8c066fe0 0x7fad8c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 --2- 172.21.15.87:0/3027972567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066c00 0x7fad8c15e110 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 -- 172.21.15.87:0/3027972567 >> 172.21.15.87:0/3027972567 conn(0x7fad8c0b24f0 msgr2=0x7fad8c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 -- 172.21.15.87:0/3027972567 shutdown_connections 2024-04-25T10:13:05.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 -- 172.21.15.87:0/3027972567 wait complete. 2024-04-25T10:13:05.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 Processor -- start 2024-04-25T10:13:05.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.223+0000 7fad91400640 1 -- start start 2024-04-25T10:13:05.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c066c00 0x7fad8c14a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066fe0 0x7fad8c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c06ed30 0x7fad8c14f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c06ac90 con 0x7fad8c066c00 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0af270 con 0x7fad8c06ed30 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c06ae10 con 0x7fad8c066fe0 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c066c00 0x7fad8c14a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c066c00 0x7fad8c14a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55140/0 (socket says 172.21.15.87:55140) 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 -- 172.21.15.87:0/1758662127 learned_addr learned my addr 172.21.15.87:0/1758662127 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad81e00640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066fe0 0x7fad8c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8b200640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c06ed30 0x7fad8c14f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 -- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066fe0 msgr2=0x7fad8c14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066fe0 0x7fad8c14f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 -- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c06ed30 msgr2=0x7fad8c14f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c06ed30 0x7fad8c14f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 -- 172.21.15.87:0/1758662127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c1607e0 con 0x7fad8c066c00 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad81e00640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066fe0 0x7fad8c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad8a800640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c066c00 0x7fad8c14a3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000f90 tx=0x7fad7c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad83400640 1 -- 172.21.15.87:0/1758662127 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0be020 con 0x7fad8c066c00 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad83400640 1 -- 172.21.15.87:0/1758662127 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c003ba0 con 0x7fad8c066c00 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad83400640 1 -- 172.21.15.87:0/1758662127 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0b2040 con 0x7fad8c066c00 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c160ac0 con 0x7fad8c066c00 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c14a900 con 0x7fad8c066c00 2024-04-25T10:13:05.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.227+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c04ef30 con 0x7fad8c066c00 2024-04-25T10:13:05.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.231+0000 7fad83400640 1 -- 172.21.15.87:0/1758662127 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad7c0c1070 con 0x7fad8c066c00 2024-04-25T10:13:05.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.231+0000 7fad83400640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad68066f60 0x7fad68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.231+0000 7fad83400640 1 -- 172.21.15.87:0/1758662127 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fad7c11f5e0 con 0x7fad8c066c00 2024-04-25T10:13:05.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.231+0000 7fad81e00640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad68066f60 0x7fad68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.231+0000 7fad81e00640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad68066f60 0x7fad68069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad8c15de50 tx=0x7fad74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.235+0000 7fad83400640 1 -- 172.21.15.87:0/1758662127 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad7c016560 con 0x7fad8c066c00 2024-04-25T10:13:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.387+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad8c071c70 con 0x7fad8c066c00 2024-04-25T10:13:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad83400640 1 -- 172.21.15.87:0/1758662127 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16843 (secure 0 0 0) 0x7fad7c0f0570 con 0x7fad8c066c00 2024-04-25T10:13:05.397 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:05.397 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":22,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:13:04.952617+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:21.283917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:22.608130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad68066f60 msgr2=0x7fad68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad68066f60 0x7fad68069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad8c15de50 tx=0x7fad74008040 comp rx=0 tx=0).stop 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c066c00 msgr2=0x7fad8c14a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c066c00 0x7fad8c14a3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000f90 tx=0x7fad7c0196e0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 shutdown_connections 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad68066f60 0x7fad68069420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c06ed30 0x7fad8c14f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c066fe0 0x7fad8c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 --2- 172.21.15.87:0/1758662127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c066c00 0x7fad8c14a3c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 >> 172.21.15.87:0/1758662127 conn(0x7fad8c0b24f0 msgr2=0x7fad8c0b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 shutdown_connections 2024-04-25T10:13:05.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.391+0000 7fad91400640 1 -- 172.21.15.87:0/1758662127 wait complete. 2024-04-25T10:13:05.419 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:05.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.547+0000 7f1703e00640 1 Processor -- start 2024-04-25T10:13:05.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.547+0000 7f1703e00640 1 -- start start 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1703e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16fc1550b0 0x7f16fc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1703e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16fc15b910 0x7f16fc0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1703e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16fc071050 con 0x7f16fc0cfe70 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1703e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc15bd80 con 0x7f16fc1550b0 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1703e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc15d040 con 0x7f16fc15b910 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16fc1550b0 0x7f16fc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16fc1550b0 0x7f16fc0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55142/0 (socket says 172.21.15.87:55142) 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701400640 1 -- 172.21.15.87:0/3921301962 learned_addr learned my addr 172.21.15.87:0/3921301962 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701e00640 1 --2- 172.21.15.87:0/3921301962 >> v2:172.21.15.169:3300/0 conn(0x7f16fc15b910 0x7f16fc0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701400640 1 -- 172.21.15.87:0/3921301962 >> v2:172.21.15.169:3300/0 conn(0x7f16fc15b910 msgr2=0x7f16fc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701400640 1 --2- 172.21.15.87:0/3921301962 >> v2:172.21.15.169:3300/0 conn(0x7f16fc15b910 0x7f16fc0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701400640 1 -- 172.21.15.87:0/3921301962 >> v1:172.21.15.169:6789/0 conn(0x7f16fc0cfe70 legacy=0x7f16fc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.551+0000 7f1701400640 1 -- 172.21.15.87:0/3921301962 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0712f0 con 0x7f16fc1550b0 2024-04-25T10:13:05.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1701400640 1 --2- 172.21.15.87:0/3921301962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc1550b0 0x7f16fc0cf930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f16f000bb10 tx=0x7f16f001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2768896de2bd6f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f16fbe00640 1 -- 172.21.15.87:0/3921301962 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00b4070 con 0x7f16fc1550b0 2024-04-25T10:13:05.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f16fbe00640 1 -- 172.21.15.87:0/3921301962 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f00a7bb0 con 0x7f16fc1550b0 2024-04-25T10:13:05.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f16fbe00640 1 -- 172.21.15.87:0/3921301962 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00af440 con 0x7f16fc1550b0 2024-04-25T10:13:05.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 -- 172.21.15.87:0/3921301962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc1550b0 msgr2=0x7f16fc0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 --2- 172.21.15.87:0/3921301962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc1550b0 0x7f16fc0cf930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f16f000bb10 tx=0x7f16f001ebf0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 -- 172.21.15.87:0/3921301962 shutdown_connections 2024-04-25T10:13:05.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 --2- 172.21.15.87:0/3921301962 >> v2:172.21.15.169:3300/0 conn(0x7f16fc15b910 0x7f16fc0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 --2- 172.21.15.87:0/3921301962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc1550b0 0x7f16fc0cf930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 -- 172.21.15.87:0/3921301962 >> 172.21.15.87:0/3921301962 conn(0x7f16fc066b70 msgr2=0x7f16fc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 -- 172.21.15.87:0/3921301962 shutdown_connections 2024-04-25T10:13:05.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 -- 172.21.15.87:0/3921301962 wait complete. 2024-04-25T10:13:05.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 Processor -- start 2024-04-25T10:13:05.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.555+0000 7f1703e00640 1 -- start start 2024-04-25T10:13:05.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0cfe70 0x7f16fc0b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc1550b0 0x7f16fc140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc15b910 0x7f16fc156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc15c030 con 0x7f16fc0cfe70 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0720b0 con 0x7f16fc1550b0 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc15c1b0 con 0x7f16fc15b910 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0cfe70 0x7f16fc0b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0cfe70 0x7f16fc0b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55154/0 (socket says 172.21.15.87:55154) 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 -- 172.21.15.87:0/4101297124 learned_addr learned my addr 172.21.15.87:0/4101297124 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1700a00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc1550b0 0x7f16fc140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701e00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc15b910 0x7f16fc156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 -- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc15b910 msgr2=0x7f16fc156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc15b910 0x7f16fc156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 -- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc1550b0 msgr2=0x7f16fc140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc1550b0 0x7f16fc140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 -- 172.21.15.87:0/4101297124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc157410 con 0x7f16fc0cfe70 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1700a00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc1550b0 0x7f16fc140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1701400640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0cfe70 0x7f16fc0b0e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f16f001b800 tx=0x7f16f001d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f16f9600640 1 -- 172.21.15.87:0/4101297124 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00bf070 con 0x7f16fc0cfe70 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f16f9600640 1 -- 172.21.15.87:0/4101297124 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f00ba440 con 0x7f16fc0cfe70 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16fc1576f0 con 0x7f16fc0cfe70 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f16f9600640 1 -- 172.21.15.87:0/4101297124 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00b4070 con 0x7f16fc0cfe70 2024-04-25T10:13:05.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc157cb0 con 0x7f16fc0cfe70 2024-04-25T10:13:05.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc04ef30 con 0x7f16fc0cfe70 2024-04-25T10:13:05.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.559+0000 7f16f9600640 1 -- 172.21.15.87:0/4101297124 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16f00c2070 con 0x7f16fc0cfe70 2024-04-25T10:13:05.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.563+0000 7f16f9600640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.563+0000 7f16f9600640 1 -- 172.21.15.87:0/4101297124 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f16f011f710 con 0x7f16fc0cfe70 2024-04-25T10:13:05.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.563+0000 7f1700a00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.563+0000 7f1700a00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0ae500 tx=0x7f16e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.567+0000 7f16f9600640 1 -- 172.21.15.87:0/4101297124 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16f00187e0 con 0x7f16fc0cfe70 2024-04-25T10:13:05.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.751+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f16fc0cf250 con 0x7f16fc0cfe70 2024-04-25T10:13:05.758 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:13:05.751+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:05.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f16f9600640 1 -- 172.21.15.87:0/4101297124 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f16f00f0600 con 0x7f16fc0cfe70 2024-04-25T10:13:05.760 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:05.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 msgr2=0x7f16dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0ae500 tx=0x7f16e8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:05.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0cfe70 msgr2=0x7f16fc0b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0cfe70 0x7f16fc0b0e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f16f001b800 tx=0x7f16f001d430 comp rx=0 tx=0).stop 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 shutdown_connections 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc15b910 0x7f16fc156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc1550b0 0x7f16fc140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 --2- 172.21.15.87:0/4101297124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0cfe70 0x7f16fc0b0e10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.755+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 >> 172.21.15.87:0/4101297124 conn(0x7f16fc066b70 msgr2=0x7f16fc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.759+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 shutdown_connections 2024-04-25T10:13:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.759+0000 7f1703e00640 1 -- 172.21.15.87:0/4101297124 wait complete. 2024-04-25T10:13:05.774 INFO:tasks.ceph.mds.c.smithi087.stderr: -6915> 2024-04-25T10:12:27.644+0000 7f553ec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:05.785 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:13:05.824 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:13:05.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82ccc00640 1 Processor -- start 2024-04-25T10:13:05.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82ccc00640 1 -- start start 2024-04-25T10:13:05.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82ccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c80cfba0 0x7f82c806cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82ccc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f82c815e2e0 0x7f82c80b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82ccc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c806de90 con 0x7f82c815a7c0 2024-04-25T10:13:05.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82ccc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c815aba0 con 0x7f82c815e2e0 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82ccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c806d5f0 con 0x7f82c80cfba0 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82c7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f82c815a7c0 0x7f82c80cf490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34902/0 (socket says 172.21.15.87:34902) 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82c7400640 1 -- 172.21.15.87:0/3156171076 learned_addr learned my addr 172.21.15.87:0/3156171076 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82bea00640 1 --2- 172.21.15.87:0/3156171076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c80cfba0 0x7f82c806cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.919+0000 7f82c7e00640 1 --2- 172.21.15.87:0/3156171076 >> v2:172.21.15.169:3300/0 conn(0x7f82c815e2e0 0x7f82c80b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82c6a00640 1 -- 172.21.15.87:0/3156171076 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804125462 0 0) 0x7f82c806de90 con 0x7f82c815a7c0 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82c6a00640 1 -- 172.21.15.87:0/3156171076 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ac003530 con 0x7f82c815a7c0 2024-04-25T10:13:05.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82bea00640 1 -- 172.21.15.87:0/3156171076 >> v2:172.21.15.169:3300/0 conn(0x7f82c815e2e0 msgr2=0x7f82c80b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:05.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82bea00640 1 --2- 172.21.15.87:0/3156171076 >> v2:172.21.15.169:3300/0 conn(0x7f82c815e2e0 0x7f82c80b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82bea00640 1 -- 172.21.15.87:0/3156171076 >> v1:172.21.15.87:6789/0 conn(0x7f82c815a7c0 legacy=0x7f82c80cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82bea00640 1 -- 172.21.15.87:0/3156171076 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c806d8d0 con 0x7f82c80cfba0 2024-04-25T10:13:05.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82bea00640 1 --2- 172.21.15.87:0/3156171076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c80cfba0 0x7f82c806cee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f82b00097c0 tx=0x7f82b001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6452e6296be9820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82c6a00640 1 -- 172.21.15.87:0/3156171076 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b00b0070 con 0x7f82c80cfba0 2024-04-25T10:13:05.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82c6a00640 1 -- 172.21.15.87:0/3156171076 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b00ab600 con 0x7f82c80cfba0 2024-04-25T10:13:05.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.923+0000 7f82c6a00640 1 -- 172.21.15.87:0/3156171076 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b00ab9d0 con 0x7f82c80cfba0 2024-04-25T10:13:05.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- 172.21.15.87:0/3156171076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c80cfba0 msgr2=0x7f82c806cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 --2- 172.21.15.87:0/3156171076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c80cfba0 0x7f82c806cee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f82b00097c0 tx=0x7f82b001c410 comp rx=0 tx=0).stop 2024-04-25T10:13:05.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- 172.21.15.87:0/3156171076 shutdown_connections 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 --2- 172.21.15.87:0/3156171076 >> v2:172.21.15.169:3300/0 conn(0x7f82c815e2e0 0x7f82c80b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 --2- 172.21.15.87:0/3156171076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c80cfba0 0x7f82c806cee0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- 172.21.15.87:0/3156171076 >> 172.21.15.87:0/3156171076 conn(0x7f82c80669a0 msgr2=0x7f82c8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- 172.21.15.87:0/3156171076 shutdown_connections 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- 172.21.15.87:0/3156171076 wait complete. 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 Processor -- start 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- start start 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c80cfba0 0x7f82c8157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c815a7c0 0x7f82c814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c815e2e0 0x7f82c80b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80cc5e0 con 0x7f82c80cfba0 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80cc460 con 0x7f82c815e2e0 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82ccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80b3870 con 0x7f82c815a7c0 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c80cfba0 0x7f82c8157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c80cfba0 0x7f82c8157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55162/0 (socket says 172.21.15.87:55162) 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 -- 172.21.15.87:0/965390266 learned_addr learned my addr 172.21.15.87:0/965390266 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82bea00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c815a7c0 0x7f82c814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7e00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c815e2e0 0x7f82c80b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 -- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c815a7c0 msgr2=0x7f82c814e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c815a7c0 0x7f82c814e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 -- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c815e2e0 msgr2=0x7f82c80b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c815e2e0 0x7f82c80b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.927+0000 7f82c7400640 1 -- 172.21.15.87:0/965390266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c81575d0 con 0x7f82c80cfba0 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82c7400640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c80cfba0 0x7f82c8157090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82b800c450 tx=0x7f82b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82bfe00640 1 -- 172.21.15.87:0/965390266 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8017070 con 0x7f82c80cfba0 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82bfe00640 1 -- 172.21.15.87:0/965390266 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b8005cf0 con 0x7f82c80cfba0 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c8155ff0 con 0x7f82c80cfba0 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82bfe00640 1 -- 172.21.15.87:0/965390266 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8007290 con 0x7f82c80cfba0 2024-04-25T10:13:05.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c81565b0 con 0x7f82c80cfba0 2024-04-25T10:13:05.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c8075120 con 0x7f82c80cfba0 2024-04-25T10:13:05.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82bfe00640 1 -- 172.21.15.87:0/965390266 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82b801a070 con 0x7f82c80cfba0 2024-04-25T10:13:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82bfe00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82a4066f90 0x7f82a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82bfe00640 1 -- 172.21.15.87:0/965390266 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f82b80843a0 con 0x7f82c80cfba0 2024-04-25T10:13:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.931+0000 7f82bea00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82a4066f90 0x7f82a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.935+0000 7f82bea00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82a4066f90 0x7f82a4069450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f82b00097c0 tx=0x7f82b0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:05.935+0000 7f82bfe00640 1 -- 172.21.15.87:0/965390266 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82b8054d00 con 0x7f82c80cfba0 2024-04-25T10:13:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.135+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f82c806c8e0 con 0x7f82c80cfba0 2024-04-25T10:13:06.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.755+0000 7f82bfe00640 1 -- 172.21.15.87:0/965390266 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f82b8058da0 con 0x7f82c80cfba0 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82a4066f90 msgr2=0x7f82a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82a4066f90 0x7f82a4069450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f82b00097c0 tx=0x7f82b0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c80cfba0 msgr2=0x7f82c8157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c80cfba0 0x7f82c8157090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82b800c450 tx=0x7f82b800c910 comp rx=0 tx=0).stop 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 shutdown_connections 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82a4066f90 0x7f82a4069450 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c815e2e0 0x7f82c80b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82c815a7c0 0x7f82c814e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 --2- 172.21.15.87:0/965390266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c80cfba0 0x7f82c8157090 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 >> 172.21.15.87:0/965390266 conn(0x7f82c80669a0 msgr2=0x7f82c8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 shutdown_connections 2024-04-25T10:13:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.759+0000 7f82ccc00640 1 -- 172.21.15.87:0/965390266 wait complete. 2024-04-25T10:13:06.788 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:13:06.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff51e800640 1 Processor -- start 2024-04-25T10:13:06.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff51e800640 1 -- start start 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff51e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff51806b6a0 0x7ff51806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff51e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff5180cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff51e800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5180b2f50 con 0x7ff5180b3fc0 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff51e800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51806bfc0 con 0x7ff51806b6a0 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff51e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180d0270 con 0x7ff5180b26b0 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff517400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff51806b6a0 0x7ff51806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff517400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff51806b6a0 0x7ff51806ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55164/0 (socket says 172.21.15.87:55164) 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff517400640 1 -- 172.21.15.87:0/28171666 learned_addr learned my addr 172.21.15.87:0/28171666 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.967+0000 7ff50ea00640 1 --2- 172.21.15.87:0/28171666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff5180cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff517400640 1 -- 172.21.15.87:0/28171666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 msgr2=0x7ff5180cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff517400640 1 --2- 172.21.15.87:0/28171666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff5180cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff517400640 1 -- 172.21.15.87:0/28171666 >> v1:172.21.15.169:6789/0 conn(0x7ff5180b3fc0 legacy=0x7ff5180b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff517400640 1 -- 172.21.15.87:0/28171666 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51806b040 con 0x7ff51806b6a0 2024-04-25T10:13:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff50ea00640 1 --2- 172.21.15.87:0/28171666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff5180cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff517400640 1 --2- 172.21.15.87:0/28171666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff51806b6a0 0x7ff51806ba80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff508009e50 tx=0x7ff50801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9c6111eae30373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff516a00640 1 -- 172.21.15.87:0/28171666 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080a5700 con 0x7ff51806b6a0 2024-04-25T10:13:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff516a00640 1 -- 172.21.15.87:0/28171666 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5080a58a0 con 0x7ff51806b6a0 2024-04-25T10:13:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff516a00640 1 -- 172.21.15.87:0/28171666 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080a5c70 con 0x7ff51806b6a0 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff51e800640 1 -- 172.21.15.87:0/28171666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff51806b6a0 msgr2=0x7ff51806ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.971+0000 7ff51e800640 1 --2- 172.21.15.87:0/28171666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff51806b6a0 0x7ff51806ba80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff508009e50 tx=0x7ff50801ca30 comp rx=0 tx=0).stop 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- 172.21.15.87:0/28171666 shutdown_connections 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/28171666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff5180cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/28171666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff51806b6a0 0x7ff51806ba80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- 172.21.15.87:0/28171666 >> 172.21.15.87:0/28171666 conn(0x7ff5180b34a0 msgr2=0x7ff5180b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- 172.21.15.87:0/28171666 shutdown_connections 2024-04-25T10:13:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- 172.21.15.87:0/28171666 wait complete. 2024-04-25T10:13:06.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 Processor -- start 2024-04-25T10:13:06.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- start start 2024-04-25T10:13:06.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51806b6a0 0x7ff51814abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff51809c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5180b3fc0 0x7ff5180af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51815f0c0 con 0x7ff5180b3fc0 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51815ef40 con 0x7ff51806b6a0 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51816b800 con 0x7ff5180b26b0 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5180b3fc0 0x7ff5180af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5180b3fc0 0x7ff5180af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55178/0 (socket says 172.21.15.87:55178) 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 -- 172.21.15.87:0/1593742654 learned_addr learned my addr 172.21.15.87:0/1593742654 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff50ea00640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff51809c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517400640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51806b6a0 0x7ff51814abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 -- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 msgr2=0x7ff51809c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff51809c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 -- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51806b6a0 msgr2=0x7ff51814abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51806b6a0 0x7ff51814abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 -- 172.21.15.87:0/1593742654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51809eca0 con 0x7ff5180b3fc0 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff517e00640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5180b3fc0 0x7ff5180af7d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c460 tx=0x7ff51000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff50fe00640 1 -- 172.21.15.87:0/1593742654 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510016020 con 0x7ff5180b3fc0 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff50fe00640 1 -- 172.21.15.87:0/1593742654 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510005220 con 0x7ff5180b3fc0 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.975+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51809ef80 con 0x7ff5180b3fc0 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff50fe00640 1 -- 172.21.15.87:0/1593742654 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510007030 con 0x7ff5180b3fc0 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51809f540 con 0x7ff5180b3fc0 2024-04-25T10:13:06.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51804ef30 con 0x7ff5180b3fc0 2024-04-25T10:13:06.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff50fe00640 1 -- 172.21.15.87:0/1593742654 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5100071d0 con 0x7ff5180b3fc0 2024-04-25T10:13:06.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff50fe00640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f60 0x7ff4f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff50fe00640 1 -- 172.21.15.87:0/1593742654 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7ff510084950 con 0x7ff5180b3fc0 2024-04-25T10:13:06.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff517400640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f60 0x7ff4f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.979+0000 7ff517400640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f60 0x7ff4f4069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5080ae000 tx=0x7ff5080a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:06.983+0000 7ff50fe00640 1 -- 172.21.15.87:0/1593742654 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff510020020 con 0x7ff5180b3fc0 2024-04-25T10:13:07.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.139+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff51806b480 con 0x7ff5180b3fc0 2024-04-25T10:13:07.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.971+0000 7ff50fe00640 1 -- 172.21.15.87:0/1593742654 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff5180b2f50 con 0x7ff5180b3fc0 2024-04-25T10:13:07.980 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:13:07.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f60 msgr2=0x7ff4f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f60 0x7ff4f4069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5080ae000 tx=0x7ff5080a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:07.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5180b3fc0 msgr2=0x7ff5180af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5180b3fc0 0x7ff5180af7d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c460 tx=0x7ff51000c920 comp rx=0 tx=0).stop 2024-04-25T10:13:07.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 shutdown_connections 2024-04-25T10:13:07.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f60 0x7ff4f4069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5180b3fc0 0x7ff5180af7d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5180b26b0 0x7ff51809c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 --2- 172.21.15.87:0/1593742654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51806b6a0 0x7ff51814abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 >> 172.21.15.87:0/1593742654 conn(0x7ff5180b34a0 msgr2=0x7ff51806e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 shutdown_connections 2024-04-25T10:13:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:07.975+0000 7ff51e800640 1 -- 172.21.15.87:0/1593742654 wait complete. 2024-04-25T10:13:08.006 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:13:08.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd707000640 1 Processor -- start 2024-04-25T10:13:08.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd707000640 1 -- start start 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd707000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd70015bd10 0x7fd7000b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd707000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd7000cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd707000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70006c0e0 con 0x7fd70015c0f0 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd707000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7000b2ef0 con 0x7fd70015bd10 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd707000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000d0390 con 0x7fd7000cf9e0 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd6f7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd70015c0f0 0x7fd7000b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52024/0 (socket says 172.21.15.87:52024) 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd6f7400640 1 -- 172.21.15.87:0/2422316999 learned_addr learned my addr 172.21.15.87:0/2422316999 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd6ffe00640 1 --2- 172.21.15.87:0/2422316999 >> v2:172.21.15.169:3300/0 conn(0x7fd70015bd10 0x7fd7000b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.139+0000 7fd705000640 1 --2- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd7000cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ff400640 1 -- 172.21.15.87:0/2422316999 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562878884 0 0) 0x7fd70006c0e0 con 0x7fd70015c0f0 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ff400640 1 -- 172.21.15.87:0/2422316999 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e4003530 con 0x7fd70015c0f0 2024-04-25T10:13:08.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ffe00640 1 -- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 msgr2=0x7fd7000cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ffe00640 1 --2- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd7000cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ffe00640 1 -- 172.21.15.87:0/2422316999 >> v1:172.21.15.87:6789/0 conn(0x7fd70015c0f0 legacy=0x7fd7000b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ffe00640 1 -- 172.21.15.87:0/2422316999 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70015c6a0 con 0x7fd70015bd10 2024-04-25T10:13:08.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd705000640 1 --2- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd7000cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:08.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ffe00640 1 --2- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015bd10 0x7fd7000b4610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009d90 tx=0x7fd6f001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34564766477ad01d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ff400640 1 -- 172.21.15.87:0/2422316999 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00b2070 con 0x7fd70015bd10 2024-04-25T10:13:08.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ff400640 1 -- 172.21.15.87:0/2422316999 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00a5a00 con 0x7fd70015bd10 2024-04-25T10:13:08.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.143+0000 7fd6ff400640 1 -- 172.21.15.87:0/2422316999 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00a5db0 con 0x7fd70015bd10 2024-04-25T10:13:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015bd10 msgr2=0x7fd7000b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 --2- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015bd10 0x7fd7000b4610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009d90 tx=0x7fd6f001c940 comp rx=0 tx=0).stop 2024-04-25T10:13:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- 172.21.15.87:0/2422316999 shutdown_connections 2024-04-25T10:13:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 --2- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd7000cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 --2- 172.21.15.87:0/2422316999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015bd10 0x7fd7000b4610 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- 172.21.15.87:0/2422316999 >> 172.21.15.87:0/2422316999 conn(0x7fd700066b70 msgr2=0x7fd700066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- 172.21.15.87:0/2422316999 shutdown_connections 2024-04-25T10:13:08.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- 172.21.15.87:0/2422316999 wait complete. 2024-04-25T10:13:08.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 Processor -- start 2024-04-25T10:13:08.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- start start 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd700148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd70015bd10 0x7fd700146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015c0f0 0x7fd7000b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000b4cd0 con 0x7fd70015bd10 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000b4b50 con 0x7fd70015c0f0 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd707000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70006cdf0 con 0x7fd7000cf9e0 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd70015bd10 0x7fd700146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd70015bd10 0x7fd700146880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53932/0 (socket says 172.21.15.87:53932) 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 -- 172.21.15.87:0/330328399 learned_addr learned my addr 172.21.15.87:0/330328399 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6ffe00640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd700148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd705000640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015c0f0 0x7fd7000b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 -- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 msgr2=0x7fd700148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd700148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 -- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015c0f0 msgr2=0x7fd7000b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015c0f0 0x7fd7000b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 -- 172.21.15.87:0/330328399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70014b470 con 0x7fd70015bd10 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.147+0000 7fd6f7400640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd70015bd10 0x7fd700146880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800c3f0 tx=0x7fd6e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6fcc00640 1 -- 172.21.15.87:0/330328399 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8017070 con 0x7fd70015bd10 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6fcc00640 1 -- 172.21.15.87:0/330328399 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e8005cf0 con 0x7fd70015bd10 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6fcc00640 1 -- 172.21.15.87:0/330328399 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80072f0 con 0x7fd70015bd10 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70014b750 con 0x7fd70015bd10 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70014bd10 con 0x7fd70015bd10 2024-04-25T10:13:08.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70004ef30 con 0x7fd70015bd10 2024-04-25T10:13:08.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6fcc00640 1 -- 172.21.15.87:0/330328399 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6e801a070 con 0x7fd70015bd10 2024-04-25T10:13:08.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6fcc00640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd6dc066f90 0x7fd6dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6fcc00640 1 -- 172.21.15.87:0/330328399 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fd6e8084290 con 0x7fd70015bd10 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6ffe00640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd6dc066f90 0x7fd6dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.151+0000 7fd6ffe00640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd6dc066f90 0x7fd6dc069450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00af000 tx=0x7fd6f00a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.155+0000 7fd6fcc00640 1 -- 172.21.15.87:0/330328399 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6e8054c90 con 0x7fd70015bd10 2024-04-25T10:13:08.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.323+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7000d0170 con 0x7fd70015bd10 2024-04-25T10:13:08.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.975+0000 7fd6fcc00640 1 -- 172.21.15.87:0/330328399 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd6e8058d30 con 0x7fd70015bd10 2024-04-25T10:13:08.983 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:13:08.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd6dc066f90 msgr2=0x7fd6dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd6dc066f90 0x7fd6dc069450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00af000 tx=0x7fd6f00a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:08.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd70015bd10 msgr2=0x7fd700146880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd70015bd10 0x7fd700146880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800c3f0 tx=0x7fd6e800c7f0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 shutdown_connections 2024-04-25T10:13:08.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd6dc066f90 0x7fd6dc069450 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70015c0f0 0x7fd7000b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd70015bd10 0x7fd700146880 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 --2- 172.21.15.87:0/330328399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7000cf9e0 0x7fd700148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 >> 172.21.15.87:0/330328399 conn(0x7fd700066b70 msgr2=0x7fd7000cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 shutdown_connections 2024-04-25T10:13:08.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:08.979+0000 7fd707000640 1 -- 172.21.15.87:0/330328399 wait complete. 2024-04-25T10:13:09.012 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:13:09.013 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:13:09.013 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:09.013 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:13:09.013 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:13:09.013 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:09.014 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:13:09.014 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:13:09.014 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:09.014 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:13:09.014 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:13:09.015 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:09.027 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:13:09.027 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:13:09.027 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:13:09.028 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:13:09.029 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:13:09.029 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:13:09.030 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:13:09.030 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:13:09.070 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:13:09.070 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:09.072 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:13:09.072 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:09.076 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:13:09.077 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:09.078 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:13:09.078 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:09.079 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:13:09.165 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:13:09.165 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:13:09.166 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:13:09.167 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:13:09.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb5800640 1 Processor -- start 2024-04-25T10:13:09.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb5800640 1 -- start start 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb5800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0bb015a780 0x7f0bb015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb5800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00b3470 con 0x7f0bb00cfd30 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb5800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb006de60 con 0x7f0bb015a780 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb015f2f0 con 0x7f0bb00cf3b0 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bafe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0bb015a780 0x7f0bb015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bafe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0bb015a780 0x7f0bb015edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53958/0 (socket says 172.21.15.87:53958) 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bafe00640 1 -- 172.21.15.87:0/2359428671 learned_addr learned my addr 172.21.15.87:0/2359428671 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0bb4e00640 1 --1- 172.21.15.87:0/2359428671 >> v1:172.21.15.87:6789/0 conn(0x7f0bb00cfd30 0x7f0bb0071260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52052/0 (socket says 172.21.15.87:52052) 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.223+0000 7f0ba7400640 1 --2- 172.21.15.87:0/2359428671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0baf400640 1 -- 172.21.15.87:0/2359428671 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885509248 0 0) 0x7f0ba80014a0 con 0x7f0bb00cfd30 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bafe00640 1 -- 172.21.15.87:0/2359428671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 msgr2=0x7f0bb00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bafe00640 1 --2- 172.21.15.87:0/2359428671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb00cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bafe00640 1 -- 172.21.15.87:0/2359428671 >> v1:172.21.15.87:6789/0 conn(0x7f0bb00cfd30 legacy=0x7f0bb0071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bafe00640 1 -- 172.21.15.87:0/2359428671 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00d01c0 con 0x7f0bb015a780 2024-04-25T10:13:09.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bafe00640 1 --2- 172.21.15.87:0/2359428671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb015edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009870 tx=0x7f0ba001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d561670685e5d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0baf400640 1 -- 172.21.15.87:0/2359428671 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00b2070 con 0x7f0bb015a780 2024-04-25T10:13:09.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0baf400640 1 -- 172.21.15.87:0/2359428671 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba00a9070 con 0x7f0bb015a780 2024-04-25T10:13:09.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0baf400640 1 -- 172.21.15.87:0/2359428671 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00ad880 con 0x7f0bb015a780 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bb5800640 1 -- 172.21.15.87:0/2359428671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 msgr2=0x7f0bb015edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bb5800640 1 --2- 172.21.15.87:0/2359428671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb015edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009870 tx=0x7f0ba001c560 comp rx=0 tx=0).stop 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bb5800640 1 -- 172.21.15.87:0/2359428671 shutdown_connections 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bb5800640 1 --2- 172.21.15.87:0/2359428671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb00cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bb5800640 1 --2- 172.21.15.87:0/2359428671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb015edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.227+0000 7f0bb5800640 1 -- 172.21.15.87:0/2359428671 >> 172.21.15.87:0/2359428671 conn(0x7f0bb00743f0 msgr2=0x7f0bb00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 -- 172.21.15.87:0/2359428671 shutdown_connections 2024-04-25T10:13:09.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 -- 172.21.15.87:0/2359428671 wait complete. 2024-04-25T10:13:09.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 Processor -- start 2024-04-25T10:13:09.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 -- start start 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb0144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cfd30 0x7f0bb01f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb0158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb015ab60 con 0x7f0bb015a780 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb006d660 con 0x7f0bb00cfd30 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0167740 con 0x7f0bb00cf3b0 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb0158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb0158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53964/0 (socket says 172.21.15.87:53964) 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 -- 172.21.15.87:0/3893547236 learned_addr learned my addr 172.21.15.87:0/3893547236 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bafe00640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb0144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0ba7400640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cfd30 0x7f0bb01f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 -- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 msgr2=0x7f0bb0144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb0144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 -- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cfd30 msgr2=0x7f0bb01f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cfd30 0x7f0bb01f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 -- 172.21.15.87:0/3893547236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb01f7470 con 0x7f0bb015a780 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bb4e00640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb0158900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800c730 tx=0x7f0ba800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.231+0000 7f0bacc00640 1 -- 172.21.15.87:0/3893547236 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8018070 con 0x7f0bb015a780 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bacc00640 1 -- 172.21.15.87:0/3893547236 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba800ce40 con 0x7f0bb015a780 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb01f7750 con 0x7f0bb015a780 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bacc00640 1 -- 172.21.15.87:0/3893547236 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800a860 con 0x7f0bb015a780 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb01f80e0 con 0x7f0bb015a780 2024-04-25T10:13:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb0075120 con 0x7f0bb015a780 2024-04-25T10:13:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bacc00640 1 -- 172.21.15.87:0/3893547236 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ba801b070 con 0x7f0bb015a780 2024-04-25T10:13:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bacc00640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b8c066f60 0x7f0b8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bacc00640 1 -- 172.21.15.87:0/3893547236 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0ba80842c0 con 0x7f0bb015a780 2024-04-25T10:13:09.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bafe00640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b8c066f60 0x7f0b8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.235+0000 7f0bafe00640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b8c066f60 0x7f0b8c069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00af000 tx=0x7f0ba00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.239+0000 7f0bacc00640 1 -- 172.21.15.87:0/3893547236 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bb00b3470 con 0x7f0bb015a780 2024-04-25T10:13:09.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.407+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0bb006c8d0 con 0x7f0bb015a780 2024-04-25T10:13:10.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.995+0000 7f0bacc00640 1 -- 172.21.15.87:0/3893547236 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0ba8054ee0 con 0x7f0bb015a780 2024-04-25T10:13:10.003 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b8c066f60 msgr2=0x7f0b8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b8c066f60 0x7f0b8c069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00af000 tx=0x7f0ba00a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 msgr2=0x7f0bb0158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb0158900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800c730 tx=0x7f0ba800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 shutdown_connections 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b8c066f60 0x7f0b8c069420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb015a780 0x7f0bb0158900 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cfd30 0x7f0bb01f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 --2- 172.21.15.87:0/3893547236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cf3b0 0x7f0bb0144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 >> 172.21.15.87:0/3893547236 conn(0x7f0bb00743f0 msgr2=0x7f0bb00b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 shutdown_connections 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:09.999+0000 7f0bb5800640 1 -- 172.21.15.87:0/3893547236 wait complete. 2024-04-25T10:13:10.032 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:13:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d8800640 1 Processor -- start 2024-04-25T10:13:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d8800640 1 -- start start 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d8800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff3d015e050 0x7ff3d00b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d8800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d00b1af0 con 0x7ff3d00b13e0 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d8800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d00b1dd0 con 0x7ff3d00d1820 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d8800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d00b20b0 con 0x7ff3d015e050 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d5400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff3d015e050 0x7ff3d00b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d6800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff3d00b13e0 0x7ff3d00cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52054/0 (socket says 172.21.15.87:52054) 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d6800640 1 -- 172.21.15.87:0/1266901894 learned_addr learned my addr 172.21.15.87:0/1266901894 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d5400640 1 -- 172.21.15.87:0/1266901894 >> v1:172.21.15.169:6789/0 conn(0x7ff3d00d1820 legacy=0x7ff3d00d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d5400640 1 -- 172.21.15.87:0/1266901894 >> v1:172.21.15.87:6789/0 conn(0x7ff3d00b13e0 legacy=0x7ff3d00cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d5400640 1 -- 172.21.15.87:0/1266901894 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d00b2390 con 0x7ff3d015e050 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.163+0000 7ff3d4a00640 1 -- 172.21.15.87:0/1266901894 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231170014 0 0) 0x7ff3d00b1af0 con 0x7ff3d00b13e0 2024-04-25T10:13:10.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d5400640 1 --2- 172.21.15.87:0/1266901894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d015e050 0x7ff3d00b0ea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097c0 tx=0x7ff3c001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35043f4c7fd7d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d4a00640 1 -- 172.21.15.87:0/1266901894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00b1070 con 0x7ff3d015e050 2024-04-25T10:13:10.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d4a00640 1 -- 172.21.15.87:0/1266901894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c00ac410 con 0x7ff3d015e050 2024-04-25T10:13:10.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d4a00640 1 -- 172.21.15.87:0/1266901894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00ac7e0 con 0x7ff3d015e050 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 -- 172.21.15.87:0/1266901894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d015e050 msgr2=0x7ff3d00b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 --2- 172.21.15.87:0/1266901894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d015e050 0x7ff3d00b0ea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097c0 tx=0x7ff3c001c410 comp rx=0 tx=0).stop 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 -- 172.21.15.87:0/1266901894 shutdown_connections 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 --2- 172.21.15.87:0/1266901894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d015e050 0x7ff3d00b0ea0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 -- 172.21.15.87:0/1266901894 >> 172.21.15.87:0/1266901894 conn(0x7ff3d009a9d0 msgr2=0x7ff3d009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 -- 172.21.15.87:0/1266901894 shutdown_connections 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 -- 172.21.15.87:0/1266901894 wait complete. 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 Processor -- start 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.167+0000 7ff3d8800640 1 -- start start 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00b13e0 0x7ff3d0150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d00d1820 0x7ff3d00b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d015e050 0x7ff3d00b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00d0fa0 con 0x7ff3d00d1820 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00d0e20 con 0x7ff3d00b13e0 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d016b9b0 con 0x7ff3d015e050 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d00d1820 0x7ff3d00b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d00d1820 0x7ff3d00b6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53970/0 (socket says 172.21.15.87:53970) 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 -- 172.21.15.87:0/2634733353 learned_addr learned my addr 172.21.15.87:0/2634733353 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5e00640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00b13e0 0x7ff3d0150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d6800640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d015e050 0x7ff3d00b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 -- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d015e050 msgr2=0x7ff3d00b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d015e050 0x7ff3d00b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 -- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00b13e0 msgr2=0x7ff3d0150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00b13e0 0x7ff3d0150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 -- 172.21.15.87:0/2634733353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0160c90 con 0x7ff3d00d1820 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5e00640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00b13e0 0x7ff3d0150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d5400640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d00d1820 0x7ff3d00b6e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097c0 tx=0x7ff3c00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3ca000640 1 -- 172.21.15.87:0/2634733353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00b1040 con 0x7ff3d00d1820 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3ca000640 1 -- 172.21.15.87:0/2634733353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c00acda0 con 0x7ff3d00d1820 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3ca000640 1 -- 172.21.15.87:0/2634733353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0004270 con 0x7ff3d00d1820 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d013d5f0 con 0x7ff3d00d1820 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d015fe80 con 0x7ff3d00d1820 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.171+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d004ef30 con 0x7ff3d00d1820 2024-04-25T10:13:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.175+0000 7ff3ca000640 1 -- 172.21.15.87:0/2634733353 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3c0003620 con 0x7ff3d00d1820 2024-04-25T10:13:10.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.175+0000 7ff3ca000640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.175+0000 7ff3ca000640 1 -- 172.21.15.87:0/2634733353 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff3c011e8c0 con 0x7ff3d00d1820 2024-04-25T10:13:10.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.175+0000 7ff3d5e00640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.175+0000 7ff3d5e00640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3b4066f60 0x7ff3b4069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000f70 tx=0x7ff3c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.179+0000 7ff3ca000640 1 -- 172.21.15.87:0/2634733353 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3c00b9020 con 0x7ff3d00d1820 2024-04-25T10:13:10.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.383+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff3d006b480 con 0x7ff3d00d1820 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.383+0000 7ff3ca000640 1 -- 172.21.15.87:0/2634733353 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff3c0016090 con 0x7ff3d00d1820 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.395 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.398 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBuLCpmMXUtBRAAW8dVU/VwdSLh6jbuG69Knw==", 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.403 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBuLCpmIX8tBRAAp3ix3uwOvzCfF4czss4UNA==", 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBuLCpmvoctBRAAko4rrzzKr/3ZRWeNbvC4tA==", 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:13:10.404 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBuLCpm9o8tBRAA0s22w9PHvjTtEhoP/SzyhQ==", 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBuLCpma5gtBRAAFum9rFYy7TYEYJRZhwdJVQ==", 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBuLCpmG6EtBRAAR/E8uvTklKZ2gt/Wvdkc+A==", 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.406 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:13:10.407 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:13:10.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3b4066f60 msgr2=0x7ff3b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3b4066f60 0x7ff3b4069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000f70 tx=0x7ff3c4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:10.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d00d1820 msgr2=0x7ff3d00b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d00d1820 0x7ff3d00b6e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097c0 tx=0x7ff3c00191b0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 shutdown_connections 2024-04-25T10:13:10.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3d015e050 0x7ff3d00b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3d00d1820 0x7ff3d00b6e00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 --2- 172.21.15.87:0/2634733353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3d00b13e0 0x7ff3d0150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 >> 172.21.15.87:0/2634733353 conn(0x7ff3d009a9d0 msgr2=0x7ff3d015cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 shutdown_connections 2024-04-25T10:13:10.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.387+0000 7ff3d8800640 1 -- 172.21.15.87:0/2634733353 wait complete. 2024-04-25T10:13:10.412 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:13:10.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f3be00640 1 Processor -- start 2024-04-25T10:13:10.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f3be00640 1 -- start start 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f3be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f3415de60 0x7f7f340ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f3be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7f3415e240 0x7f7f3406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f3be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f340d0950 con 0x7f7f340b2af0 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f3be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f340d0c30 con 0x7f7f3415de60 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f3be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f340d0f10 con 0x7f7f3415e240 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f3415de60 0x7f7f340ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f3415de60 0x7f7f340ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53978/0 (socket says 172.21.15.87:53978) 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f38a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7f340b2af0 0x7f7f340cc340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52060/0 (socket says 172.21.15.87:52060) 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39400640 1 -- 172.21.15.87:0/3720405032 learned_addr learned my addr 172.21.15.87:0/3720405032 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:10.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39e00640 1 --2- 172.21.15.87:0/3720405032 >> v2:172.21.15.169:3300/0 conn(0x7f7f3415e240 0x7f7f3406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39400640 1 -- 172.21.15.87:0/3720405032 >> v2:172.21.15.169:3300/0 conn(0x7f7f3415e240 msgr2=0x7f7f3406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39400640 1 --2- 172.21.15.87:0/3720405032 >> v2:172.21.15.169:3300/0 conn(0x7f7f3415e240 0x7f7f3406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39400640 1 -- 172.21.15.87:0/3720405032 >> v1:172.21.15.87:6789/0 conn(0x7f7f340b2af0 legacy=0x7f7f340cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.543+0000 7f7f39400640 1 -- 172.21.15.87:0/3720405032 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340d11f0 con 0x7f7f3415de60 2024-04-25T10:13:10.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f39400640 1 --2- 172.21.15.87:0/3720405032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3415de60 0x7f7f340ce190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009e50 tx=0x7f7f2801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e482e4170c5251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f33e00640 1 -- 172.21.15.87:0/3720405032 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280a5620 con 0x7f7f3415de60 2024-04-25T10:13:10.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f33e00640 1 -- 172.21.15.87:0/3720405032 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f280a57c0 con 0x7f7f3415de60 2024-04-25T10:13:10.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f33e00640 1 -- 172.21.15.87:0/3720405032 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280a5b90 con 0x7f7f3415de60 2024-04-25T10:13:10.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 -- 172.21.15.87:0/3720405032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3415de60 msgr2=0x7f7f340ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3720405032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3415de60 0x7f7f340ce190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009e50 tx=0x7f7f2801c960 comp rx=0 tx=0).stop 2024-04-25T10:13:10.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 -- 172.21.15.87:0/3720405032 shutdown_connections 2024-04-25T10:13:10.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3720405032 >> v2:172.21.15.169:3300/0 conn(0x7f7f3415e240 0x7f7f3406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3720405032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3415de60 0x7f7f340ce190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 -- 172.21.15.87:0/3720405032 >> 172.21.15.87:0/3720405032 conn(0x7f7f340d1910 msgr2=0x7f7f340d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 -- 172.21.15.87:0/3720405032 shutdown_connections 2024-04-25T10:13:10.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 -- 172.21.15.87:0/3720405032 wait complete. 2024-04-25T10:13:10.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.547+0000 7f7f3be00640 1 Processor -- start 2024-04-25T10:13:10.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 -- start start 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f340b2af0 0x7f7f34140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3415de60 0x7f7f34141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f3415e240 0x7f7f3415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340cd630 con 0x7f7f340b2af0 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340cd4b0 con 0x7f7f3415e240 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3416ac50 con 0x7f7f3415de60 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f340b2af0 0x7f7f34140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f340b2af0 0x7f7f34140bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53992/0 (socket says 172.21.15.87:53992) 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 -- 172.21.15.87:0/3888443815 learned_addr learned my addr 172.21.15.87:0/3888443815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f38a00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3415de60 0x7f7f34141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39e00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f3415e240 0x7f7f3415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 -- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3415de60 msgr2=0x7f7f34141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3415de60 0x7f7f34141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 -- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f3415e240 msgr2=0x7f7f3415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f3415e240 0x7f7f3415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 -- 172.21.15.87:0/3888443815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340ca580 con 0x7f7f340b2af0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f39400640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f340b2af0 0x7f7f34140bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f28019570 tx=0x7f7f28019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f31600640 1 -- 172.21.15.87:0/3888443815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280bc020 con 0x7f7f340b2af0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f31600640 1 -- 172.21.15.87:0/3888443815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f280b8880 con 0x7f7f340b2af0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f340ca860 con 0x7f7f340b2af0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f31600640 1 -- 172.21.15.87:0/3888443815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28009510 con 0x7f7f340b2af0 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f340cae20 con 0x7f7f340b2af0 2024-04-25T10:13:10.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.551+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3404ef30 con 0x7f7f340b2af0 2024-04-25T10:13:10.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.555+0000 7f7f31600640 1 -- 172.21.15.87:0/3888443815 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f280bf020 con 0x7f7f340b2af0 2024-04-25T10:13:10.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.555+0000 7f7f31600640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f14066f90 0x7f7f14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.555+0000 7f7f31600640 1 -- 172.21.15.87:0/3888443815 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7f2811f970 con 0x7f7f340b2af0 2024-04-25T10:13:10.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.555+0000 7f7f38a00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f14066f90 0x7f7f14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.555+0000 7f7f38a00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f14066f90 0x7f7f14069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f3406be50 tx=0x7f7f20007cf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.559+0000 7f7f31600640 1 -- 172.21.15.87:0/3888443815 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f280165e0 con 0x7f7f340b2af0 2024-04-25T10:13:10.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.767+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7f340ce340 con 0x7f7f340b2af0 2024-04-25T10:13:10.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.767+0000 7f7f31600640 1 -- 172.21.15.87:0/3888443815 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7f280f0c30 con 0x7f7f340b2af0 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f14066f90 msgr2=0x7f7f14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f14066f90 0x7f7f14069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f3406be50 tx=0x7f7f20007cf0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f340b2af0 msgr2=0x7f7f34140bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f340b2af0 0x7f7f34140bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f28019570 tx=0x7f7f28019700 comp rx=0 tx=0).stop 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 shutdown_connections 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f14066f90 0x7f7f14069450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f3415e240 0x7f7f3415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3415de60 0x7f7f34141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 --2- 172.21.15.87:0/3888443815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f340b2af0 0x7f7f34140bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 >> 172.21.15.87:0/3888443815 conn(0x7f7f340d1910 msgr2=0x7f7f340d1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 shutdown_connections 2024-04-25T10:13:10.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.771+0000 7f7f3be00640 1 -- 172.21.15.87:0/3888443815 wait complete. 2024-04-25T10:13:10.798 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:13:10.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.927+0000 7f132d000640 1 Processor -- start 2024-04-25T10:13:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f132d000640 1 -- start start 2024-04-25T10:13:10.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f132d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13281550b0 0x7f13280cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f132d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280cfe70 0x7f132815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f132d000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1328071050 con 0x7f132815b740 2024-04-25T10:13:10.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f132d000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132815d040 con 0x7f13281550b0 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f132d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280712f0 con 0x7f13280cfe70 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13281550b0 0x7f13280cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13281550b0 0x7f13280cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53994/0 (socket says 172.21.15.87:53994) 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325e00640 1 -- 172.21.15.87:0/1497863221 learned_addr learned my addr 172.21.15.87:0/1497863221 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325400640 1 --2- 172.21.15.87:0/1497863221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280cfe70 0x7f132815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325e00640 1 -- 172.21.15.87:0/1497863221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280cfe70 msgr2=0x7f132815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325e00640 1 --2- 172.21.15.87:0/1497863221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280cfe70 0x7f132815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325e00640 1 -- 172.21.15.87:0/1497863221 >> v1:172.21.15.169:6789/0 conn(0x7f132815b740 legacy=0x7f132806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.931+0000 7f1325e00640 1 -- 172.21.15.87:0/1497863221 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280715d0 con 0x7f13281550b0 2024-04-25T10:13:10.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f1325e00640 1 --2- 172.21.15.87:0/1497863221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f13280cf930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1318009870 tx=0x7f131801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e53fdcf9f43c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f1324a00640 1 -- 172.21.15.87:0/1497863221 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180b0070 con 0x7f13281550b0 2024-04-25T10:13:10.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f1324a00640 1 -- 172.21.15.87:0/1497863221 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13180ab600 con 0x7f13281550b0 2024-04-25T10:13:10.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f1324a00640 1 -- 172.21.15.87:0/1497863221 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180ab9d0 con 0x7f13281550b0 2024-04-25T10:13:10.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 -- 172.21.15.87:0/1497863221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 msgr2=0x7f13280cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 --2- 172.21.15.87:0/1497863221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f13280cf930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1318009870 tx=0x7f131801c520 comp rx=0 tx=0).stop 2024-04-25T10:13:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 -- 172.21.15.87:0/1497863221 shutdown_connections 2024-04-25T10:13:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 --2- 172.21.15.87:0/1497863221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280cfe70 0x7f132815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 --2- 172.21.15.87:0/1497863221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f13280cf930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 -- 172.21.15.87:0/1497863221 >> 172.21.15.87:0/1497863221 conn(0x7f1328066b70 msgr2=0x7f1328066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 -- 172.21.15.87:0/1497863221 shutdown_connections 2024-04-25T10:13:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 -- 172.21.15.87:0/1497863221 wait complete. 2024-04-25T10:13:10.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 Processor -- start 2024-04-25T10:13:10.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.935+0000 7f132d000640 1 -- start start 2024-04-25T10:13:10.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13280cfe70 0x7f1328140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f1328143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f132815b740 0x7f132813fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132815bd50 con 0x7f13281550b0 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132815bbd0 con 0x7f13280cfe70 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132815c030 con 0x7f132815b740 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f1328143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f1328143040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54002/0 (socket says 172.21.15.87:54002) 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 -- 172.21.15.87:0/325863550 learned_addr learned my addr 172.21.15.87:0/325863550 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325e00640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13280cfe70 0x7f1328140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1326800640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f132815b740 0x7f132813fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 -- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f132815b740 msgr2=0x7f132813fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f132815b740 0x7f132813fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 -- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13280cfe70 msgr2=0x7f1328140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13280cfe70 0x7f1328140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 -- 172.21.15.87:0/325863550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1328140bf0 con 0x7f13281550b0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f1325400640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f1328143040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f131400c7f0 tx=0x7f131400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f131e000640 1 -- 172.21.15.87:0/325863550 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314015070 con 0x7f13281550b0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f131e000640 1 -- 172.21.15.87:0/325863550 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1314004240 con 0x7f13281550b0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f131e000640 1 -- 172.21.15.87:0/325863550 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314003620 con 0x7f13281550b0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1328140ed0 con 0x7f13281550b0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1328141490 con 0x7f13281550b0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132804ef30 con 0x7f13281550b0 2024-04-25T10:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.939+0000 7f131e000640 1 -- 172.21.15.87:0/325863550 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13140039d0 con 0x7f13281550b0 2024-04-25T10:13:10.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.943+0000 7f131e000640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1308066f60 0x7f1308069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.943+0000 7f131e000640 1 -- 172.21.15.87:0/325863550 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1314084d00 con 0x7f13281550b0 2024-04-25T10:13:10.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.943+0000 7f1325e00640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1308066f60 0x7f1308069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.943+0000 7f1325e00640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1308066f60 0x7f1308069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f13180ad000 tx=0x7f13180a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:10.947+0000 7f131e000640 1 -- 172.21.15.87:0/325863550 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1314055a00 con 0x7f13281550b0 2024-04-25T10:13:11.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.159+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f13280b1070 con 0x7f13281550b0 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f131e000640 1 -- 172.21.15.87:0/325863550 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1314059aa0 con 0x7f13281550b0 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1308066f60 msgr2=0x7f1308069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1308066f60 0x7f1308069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f13180ad000 tx=0x7f13180a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 msgr2=0x7f1328143040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f1328143040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f131400c7f0 tx=0x7f131400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 shutdown_connections 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1308066f60 0x7f1308069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f132815b740 0x7f132813fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13281550b0 0x7f1328143040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 --2- 172.21.15.87:0/325863550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13280cfe70 0x7f1328140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 >> 172.21.15.87:0/325863550 conn(0x7f1328066b70 msgr2=0x7f13280b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 shutdown_connections 2024-04-25T10:13:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.163+0000 7f132d000640 1 -- 172.21.15.87:0/325863550 wait complete. 2024-04-25T10:13:11.189 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:13:11.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad6ee00640 1 Processor -- start 2024-04-25T10:13:11.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad6ee00640 1 -- start start 2024-04-25T10:13:11.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad6ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 0x7fad6806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad6ee00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680b2770 con 0x7fad6815bc40 2024-04-25T10:13:11.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad6ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6806c670 con 0x7fad680cda10 2024-04-25T10:13:11.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad6ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6806c950 con 0x7fad680d0040 2024-04-25T10:13:11.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fad6815bc40 0x7fad680b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52070/0 (socket says 172.21.15.87:52070) 2024-04-25T10:13:11.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67e00640 1 -- 172.21.15.87:0/2807807858 learned_addr learned my addr 172.21.15.87:0/2807807858 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:11.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad6ce00640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 0x7fad6806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991258823 0 0) 0x7fad680b2770 con 0x7fad6815bc40 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad4c003530 con 0x7fad6815bc40 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 681155313 0 0) 0x7fad4c003530 con 0x7fad6815bc40 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 msgr2=0x7fad6806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 0x7fad6806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 >> v1:172.21.15.169:6789/0 conn(0x7fad680cda10 legacy=0x7fad680cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6806cc30 con 0x7fad6815bc40 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fad58001a00 con 0x7fad6815bc40 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 >> v1:172.21.15.87:6789/0 conn(0x7fad6815bc40 legacy=0x7fad680b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0039f0 0x7fad4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 0x7fad4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c004d50 0x7fad4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6806c950 con 0x7fad4c004d50 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c00bb30 con 0x7fad4c0039f0 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.319+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c00bd30 con 0x7fad680d0040 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad6ce00640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c004d50 0x7fad4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad67e00640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0039f0 0x7fad4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad5f400640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 0x7fad4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad6ce00640 1 -- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 msgr2=0x7fad4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad6ce00640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad680d0040 0x7fad4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad6ce00640 1 -- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0039f0 msgr2=0x7fad4c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad6ce00640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0039f0 0x7fad4c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad6ce00640 1 -- 172.21.15.87:0/2807807858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c003530 con 0x7fad4c004d50 2024-04-25T10:13:11.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad6ce00640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c004d50 0x7fad4c0051d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad680b4510 tx=0x7fad6001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fd8b46c59363ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600b2070 con 0x7fad4c004d50 2024-04-25T10:13:11.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad600a5600 con 0x7fad4c004d50 2024-04-25T10:13:11.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.323+0000 7fad67400640 1 -- 172.21.15.87:0/2807807858 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600096b0 con 0x7fad4c004d50 2024-04-25T10:13:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- 172.21.15.87:0/2807807858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c004d50 msgr2=0x7fad4c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 --2- 172.21.15.87:0/2807807858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c004d50 0x7fad4c0051d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad680b4510 tx=0x7fad6001ca10 comp rx=0 tx=0).stop 2024-04-25T10:13:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 -- 172.21.15.87:0/2807807858 reap_dead start 2024-04-25T10:13:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- 172.21.15.87:0/2807807858 shutdown_connections 2024-04-25T10:13:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- 172.21.15.87:0/2807807858 >> 172.21.15.87:0/2807807858 conn(0x7fad68066b70 msgr2=0x7fad68066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- 172.21.15.87:0/2807807858 shutdown_connections 2024-04-25T10:13:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- 172.21.15.87:0/2807807858 wait complete. 2024-04-25T10:13:11.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 Processor -- start 2024-04-25T10:13:11.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- start start 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c004d50 0x7fad681495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad6815d190 0x7fad680ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad680add80 0x7fad68155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6806c210 con 0x7fad680add80 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6806c090 con 0x7fad4c004d50 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6806c390 con 0x7fad6815d190 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad680add80 0x7fad68155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad680add80 0x7fad68155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54018/0 (socket says 172.21.15.87:54018) 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 -- 172.21.15.87:0/4019565994 learned_addr learned my addr 172.21.15.87:0/4019565994 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad5f400640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad6815d190 0x7fad680ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad67e00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c004d50 0x7fad681495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 -- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad6815d190 msgr2=0x7fad680ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad6815d190 0x7fad680ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 -- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c004d50 msgr2=0x7fad681495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c004d50 0x7fad681495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 -- 172.21.15.87:0/4019565994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6815f130 con 0x7fad680add80 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad5f400640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad6815d190 0x7fad680ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.327+0000 7fad6ce00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad680add80 0x7fad68155890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad6001cef0 tx=0x7fad4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad64c00640 1 -- 172.21.15.87:0/4019565994 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600b2040 con 0x7fad680add80 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad64c00640 1 -- 172.21.15.87:0/4019565994 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60000e50 con 0x7fad680add80 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad68071030 con 0x7fad680add80 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad64c00640 1 -- 172.21.15.87:0/4019565994 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600ad440 con 0x7fad680add80 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad680715f0 con 0x7fad680add80 2024-04-25T10:13:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad680720b0 con 0x7fad680add80 2024-04-25T10:13:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad64c00640 1 -- 172.21.15.87:0/4019565994 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad600bd030 con 0x7fad680add80 2024-04-25T10:13:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad64c00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad44066f90 0x7fad44069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad64c00640 1 -- 172.21.15.87:0/4019565994 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad6011e310 con 0x7fad680add80 2024-04-25T10:13:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad67e00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad44066f90 0x7fad44069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.331+0000 7fad67e00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad44066f90 0x7fad44069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad58001590 tx=0x7fad580014a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.335+0000 7fad64c00640 1 -- 172.21.15.87:0/4019565994 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad680720b0 con 0x7fad680add80 2024-04-25T10:13:11.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.543+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fad6804ef30 con 0x7fad680add80 2024-04-25T10:13:11.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad64c00640 1 -- 172.21.15.87:0/4019565994 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fad600c4060 con 0x7fad680add80 2024-04-25T10:13:11.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad44066f90 msgr2=0x7fad44069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad44066f90 0x7fad44069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad58001590 tx=0x7fad580014a0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad680add80 msgr2=0x7fad68155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad680add80 0x7fad68155890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad6001cef0 tx=0x7fad4c005840 comp rx=0 tx=0).stop 2024-04-25T10:13:11.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 shutdown_connections 2024-04-25T10:13:11.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad44066f90 0x7fad44069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad680add80 0x7fad68155890 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad6815d190 0x7fad680ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 --2- 172.21.15.87:0/4019565994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c004d50 0x7fad681495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 >> 172.21.15.87:0/4019565994 conn(0x7fad68066b70 msgr2=0x7fad680b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 shutdown_connections 2024-04-25T10:13:11.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.547+0000 7fad6ee00640 1 -- 172.21.15.87:0/4019565994 wait complete. 2024-04-25T10:13:11.575 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:13:11.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0c2800640 1 Processor -- start 2024-04-25T10:13:11.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0c2800640 1 -- start start 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0c2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0bc15bc40 0x7fb0bc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0c2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0bc0cda10 0x7fb0bc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0c2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0b2770 con 0x7fb0bc0cf930 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0c2800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc06c670 con 0x7fb0bc15bc40 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0c2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc06c950 con 0x7fb0bc0cda10 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0bb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0bc15bc40 0x7fb0bc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0bb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0bc15bc40 0x7fb0bc06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54028/0 (socket says 172.21.15.87:54028) 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0bb400640 1 -- 172.21.15.87:0/2584860987 learned_addr learned my addr 172.21.15.87:0/2584860987 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0baa00640 1 --2- 172.21.15.87:0/2584860987 >> v2:172.21.15.169:3300/0 conn(0x7fb0bc0cda10 0x7fb0bc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0bb400640 1 -- 172.21.15.87:0/2584860987 >> v2:172.21.15.169:3300/0 conn(0x7fb0bc0cda10 msgr2=0x7fb0bc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0bb400640 1 --2- 172.21.15.87:0/2584860987 >> v2:172.21.15.169:3300/0 conn(0x7fb0bc0cda10 0x7fb0bc0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0bb400640 1 -- 172.21.15.87:0/2584860987 >> v1:172.21.15.169:6789/0 conn(0x7fb0bc0cf930 legacy=0x7fb0bc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.707+0000 7fb0bb400640 1 -- 172.21.15.87:0/2584860987 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc06cc30 con 0x7fb0bc15bc40 2024-04-25T10:13:11.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0bb400640 1 --2- 172.21.15.87:0/2584860987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc15bc40 0x7fb0bc06c090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0726f0 tx=0x7fb0ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca24ed4b543e2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0ba000640 1 -- 172.21.15.87:0/2584860987 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0ab430 con 0x7fb0bc15bc40 2024-04-25T10:13:11.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0ba000640 1 -- 172.21.15.87:0/2584860987 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac0b0070 con 0x7fb0bc15bc40 2024-04-25T10:13:11.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0ba000640 1 -- 172.21.15.87:0/2584860987 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0ab7f0 con 0x7fb0bc15bc40 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 -- 172.21.15.87:0/2584860987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc15bc40 msgr2=0x7fb0bc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 --2- 172.21.15.87:0/2584860987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc15bc40 0x7fb0bc06c090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0726f0 tx=0x7fb0ac01c520 comp rx=0 tx=0).stop 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 -- 172.21.15.87:0/2584860987 shutdown_connections 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 --2- 172.21.15.87:0/2584860987 >> v2:172.21.15.169:3300/0 conn(0x7fb0bc0cda10 0x7fb0bc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 --2- 172.21.15.87:0/2584860987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc15bc40 0x7fb0bc06c090 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 -- 172.21.15.87:0/2584860987 >> 172.21.15.87:0/2584860987 conn(0x7fb0bc066b70 msgr2=0x7fb0bc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 -- 172.21.15.87:0/2584860987 shutdown_connections 2024-04-25T10:13:11.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 -- 172.21.15.87:0/2584860987 wait complete. 2024-04-25T10:13:11.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 Processor -- start 2024-04-25T10:13:11.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.711+0000 7fb0c2800640 1 -- start start 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc0cda10 0x7fb0bc14b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc0cf930 0x7fb0bc15e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0bc15bc40 0x7fb0bc15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc168b50 con 0x7fb0bc0cda10 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc1689d0 con 0x7fb0bc0cf930 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc168cd0 con 0x7fb0bc15bc40 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc0cda10 0x7fb0bc14b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc0cda10 0x7fb0bc14b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54036/0 (socket says 172.21.15.87:54036) 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 -- 172.21.15.87:0/3435647531 learned_addr learned my addr 172.21.15.87:0/3435647531 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 -- 172.21.15.87:0/3435647531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0bc15bc40 msgr2=0x7fb0bc15ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0bc15bc40 0x7fb0bc15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 -- 172.21.15.87:0/3435647531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc0cf930 msgr2=0x7fb0bc15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc0cf930 0x7fb0bc15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 -- 172.21.15.87:0/3435647531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc15c3a0 con 0x7fb0bc0cda10 2024-04-25T10:13:11.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0bb400640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc0cda10 0x7fb0bc14b460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0ad040 tx=0x7fb0ac01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0b3400640 1 -- 172.21.15.87:0/3435647531 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0be5f0 con 0x7fb0bc0cda10 2024-04-25T10:13:11.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0b3400640 1 -- 172.21.15.87:0/3435647531 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac0be790 con 0x7fb0bc0cda10 2024-04-25T10:13:11.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0b3400640 1 -- 172.21.15.87:0/3435647531 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0beb60 con 0x7fb0bc0cda10 2024-04-25T10:13:11.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc15c680 con 0x7fb0bc0cda10 2024-04-25T10:13:11.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc15cf20 con 0x7fb0bc0cda10 2024-04-25T10:13:11.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc04ef30 con 0x7fb0bc0cda10 2024-04-25T10:13:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.715+0000 7fb0b3400640 1 -- 172.21.15.87:0/3435647531 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0ac0b43f0 con 0x7fb0bc0cda10 2024-04-25T10:13:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.719+0000 7fb0b3400640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb09c066f60 0x7fb09c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.719+0000 7fb0b3400640 1 -- 172.21.15.87:0/3435647531 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb0ac0a8070 con 0x7fb0bc0cda10 2024-04-25T10:13:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.719+0000 7fb0baa00640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb09c066f60 0x7fb09c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.719+0000 7fb0baa00640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb09c066f60 0x7fb09c069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc1fa700 tx=0x7fb0a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.723+0000 7fb0b3400640 1 -- 172.21.15.87:0/3435647531 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0ac016110 con 0x7fb0bc0cda10 2024-04-25T10:13:11.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.931+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fb0bc06d960 con 0x7fb0bc0cda10 2024-04-25T10:13:11.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0b3400640 1 -- 172.21.15.87:0/3435647531 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb0ac0f08f0 con 0x7fb0bc0cda10 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb09c066f60 msgr2=0x7fb09c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb09c066f60 0x7fb09c069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc1fa700 tx=0x7fb0a8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc0cda10 msgr2=0x7fb0bc14b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc0cda10 0x7fb0bc14b460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0ad040 tx=0x7fb0ac01ad60 comp rx=0 tx=0).stop 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 shutdown_connections 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb09c066f60 0x7fb09c069420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0bc15bc40 0x7fb0bc15ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc0cf930 0x7fb0bc15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 --2- 172.21.15.87:0/3435647531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0bc0cda10 0x7fb0bc14b460 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 >> 172.21.15.87:0/3435647531 conn(0x7fb0bc066b70 msgr2=0x7fb0bc0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 shutdown_connections 2024-04-25T10:13:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:11.935+0000 7fb0c2800640 1 -- 172.21.15.87:0/3435647531 wait complete. 2024-04-25T10:13:11.962 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:13:12.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d1e600640 1 Processor -- start 2024-04-25T10:13:12.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d1e600640 1 -- start start 2024-04-25T10:13:12.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d1e600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d1815b910 0x7f5d1815c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d1e600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1815bd60 con 0x7f5d181550b0 2024-04-25T10:13:12.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d1e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d18071050 con 0x7f5d180d0040 2024-04-25T10:13:12.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d1e600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d180712f0 con 0x7f5d1815b910 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d17400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5d181550b0 0x7f5d180cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52074/0 (socket says 172.21.15.87:52074) 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d17400640 1 -- 172.21.15.87:0/2846471554 learned_addr learned my addr 172.21.15.87:0/2846471554 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d17e00640 1 --2- 172.21.15.87:0/2846471554 >> v2:172.21.15.87:3300/0 conn(0x7f5d1815b910 0x7f5d1815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d16a00640 1 -- 172.21.15.87:0/2846471554 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020442428 0 0) 0x7f5d1815bd60 con 0x7f5d181550b0 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d16a00640 1 -- 172.21.15.87:0/2846471554 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cfc003530 con 0x7f5d181550b0 2024-04-25T10:13:12.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d17e00640 1 -- 172.21.15.87:0/2846471554 >> v1:172.21.15.169:6789/0 conn(0x7f5d180d0040 legacy=0x7f5d1815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d17e00640 1 -- 172.21.15.87:0/2846471554 >> v1:172.21.15.87:6789/0 conn(0x7f5d181550b0 legacy=0x7f5d180cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.095+0000 7f5d17e00640 1 -- 172.21.15.87:0/2846471554 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d180715d0 con 0x7f5d1815b910 2024-04-25T10:13:12.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d17e00640 1 --2- 172.21.15.87:0/2846471554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d1815b910 0x7f5d1815c030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5d100097c0 tx=0x7f5d1001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33454263c2ff14cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d16a00640 1 -- 172.21.15.87:0/2846471554 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d100a7070 con 0x7f5d1815b910 2024-04-25T10:13:12.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d16a00640 1 -- 172.21.15.87:0/2846471554 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d100b0070 con 0x7f5d1815b910 2024-04-25T10:13:12.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d16a00640 1 -- 172.21.15.87:0/2846471554 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d100ab870 con 0x7f5d1815b910 2024-04-25T10:13:12.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 -- 172.21.15.87:0/2846471554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d1815b910 msgr2=0x7f5d1815c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 --2- 172.21.15.87:0/2846471554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d1815b910 0x7f5d1815c030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5d100097c0 tx=0x7f5d1001c470 comp rx=0 tx=0).stop 2024-04-25T10:13:12.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 -- 172.21.15.87:0/2846471554 shutdown_connections 2024-04-25T10:13:12.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 --2- 172.21.15.87:0/2846471554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d1815b910 0x7f5d1815c030 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 -- 172.21.15.87:0/2846471554 >> 172.21.15.87:0/2846471554 conn(0x7f5d18066b70 msgr2=0x7f5d18066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 -- 172.21.15.87:0/2846471554 shutdown_connections 2024-04-25T10:13:12.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 -- 172.21.15.87:0/2846471554 wait complete. 2024-04-25T10:13:12.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 Processor -- start 2024-04-25T10:13:12.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.099+0000 7f5d1e600640 1 -- start start 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d180d0040 0x7f5d18153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d181550b0 0x7f5d1814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1815b910 0x7f5d1815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1815cf00 con 0x7f5d180d0040 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1815cd80 con 0x7f5d1815b910 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1815d080 con 0x7f5d181550b0 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d180d0040 0x7f5d18153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d180d0040 0x7f5d18153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54050/0 (socket says 172.21.15.87:54050) 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 -- 172.21.15.87:0/837367587 learned_addr learned my addr 172.21.15.87:0/837367587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:12.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d0ea00640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d181550b0 0x7f5d1814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17e00640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1815b910 0x7f5d1815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 -- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d181550b0 msgr2=0x7f5d1814fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d181550b0 0x7f5d1814fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 -- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1815b910 msgr2=0x7f5d1815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1815b910 0x7f5d1815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 -- 172.21.15.87:0/837367587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d18154420 con 0x7f5d180d0040 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17e00640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1815b910 0x7f5d1815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d17400640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d180d0040 0x7f5d18153ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800cbc0 tx=0x7f5d0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d0fe00640 1 -- 172.21.15.87:0/837367587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08017070 con 0x7f5d180d0040 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d0fe00640 1 -- 172.21.15.87:0/837367587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0800ad40 con 0x7f5d180d0040 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d0fe00640 1 -- 172.21.15.87:0/837367587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08012600 con 0x7f5d180d0040 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d181546c0 con 0x7f5d180d0040 2024-04-25T10:13:12.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1814f570 con 0x7f5d180d0040 2024-04-25T10:13:12.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.103+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1804ef30 con 0x7f5d180d0040 2024-04-25T10:13:12.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.107+0000 7f5d0fe00640 1 -- 172.21.15.87:0/837367587 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d0801a070 con 0x7f5d180d0040 2024-04-25T10:13:12.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.107+0000 7f5d0fe00640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cf4066f60 0x7f5cf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.107+0000 7f5d0fe00640 1 -- 172.21.15.87:0/837367587 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d08084ad0 con 0x7f5d180d0040 2024-04-25T10:13:12.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.107+0000 7f5d0ea00640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cf4066f60 0x7f5cf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.107+0000 7f5d0ea00640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cf4066f60 0x7f5cf4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5d181f9830 tx=0x7f5d00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.111+0000 7f5d0fe00640 1 -- 172.21.15.87:0/837367587 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d08055740 con 0x7f5d180d0040 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.319+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f5d1814f010 con 0x7f5d180d0040 2024-04-25T10:13:12.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d0fe00640 1 -- 172.21.15.87:0/837367587 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5d080597e0 con 0x7f5d180d0040 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cf4066f60 msgr2=0x7f5cf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cf4066f60 0x7f5cf4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5d181f9830 tx=0x7f5d00009040 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d180d0040 msgr2=0x7f5d18153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d180d0040 0x7f5d18153ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800cbc0 tx=0x7f5d0800a430 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 shutdown_connections 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cf4066f60 0x7f5cf4069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1815b910 0x7f5d1815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d181550b0 0x7f5d1814fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 --2- 172.21.15.87:0/837367587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d180d0040 0x7f5d18153ee0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 >> 172.21.15.87:0/837367587 conn(0x7f5d18066b70 msgr2=0x7f5d180ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 shutdown_connections 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.323+0000 7f5d1e600640 1 -- 172.21.15.87:0/837367587 wait complete. 2024-04-25T10:13:12.350 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:13:12.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.479+0000 7fe5af600640 1 Processor -- start 2024-04-25T10:13:12.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.479+0000 7fe5af600640 1 -- start start 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.479+0000 7fe5af600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe5a81550b0 0x7fe5a80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5af600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a80cfe70 0x7fe5a815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5af600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5a815b740 0x7fe5a806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5af600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a8071050 con 0x7fe5a81550b0 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5af600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a80712f0 con 0x7fe5a815b740 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5af600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a80715d0 con 0x7fe5a80cfe70 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe5a81550b0 0x7fe5a80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe5a81550b0 0x7fe5a80cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54058/0 (socket says 172.21.15.87:54058) 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 -- 172.21.15.87:0/3842266716 learned_addr learned my addr 172.21.15.87:0/3842266716 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 -- 172.21.15.87:0/3842266716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a80cfe70 msgr2=0x7fe5a815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 --2- 172.21.15.87:0/3842266716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a80cfe70 0x7fe5a815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5ad600640 1 --2- 172.21.15.87:0/3842266716 >> v2:172.21.15.169:3300/0 conn(0x7fe5a815b740 0x7fe5a806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 -- 172.21.15.87:0/3842266716 >> v2:172.21.15.169:3300/0 conn(0x7fe5a815b740 msgr2=0x7fe5a806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 --2- 172.21.15.87:0/3842266716 >> v2:172.21.15.169:3300/0 conn(0x7fe5a815b740 0x7fe5a806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 -- 172.21.15.87:0/3842266716 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a80718b0 con 0x7fe5a81550b0 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5ad600640 1 --2- 172.21.15.87:0/3842266716 >> v2:172.21.15.169:3300/0 conn(0x7fe5a815b740 0x7fe5a806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.483+0000 7fe5acc00640 1 --2- 172.21.15.87:0/3842266716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a81550b0 0x7fe5a80cf930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80d04d0 tx=0x7fe59c01e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a96dd4602ff36c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5a7400640 1 -- 172.21.15.87:0/3842266716 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0b4070 con 0x7fe5a81550b0 2024-04-25T10:13:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5a7400640 1 -- 172.21.15.87:0/3842266716 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c0a7840 con 0x7fe5a81550b0 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5a7400640 1 -- 172.21.15.87:0/3842266716 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0a7c10 con 0x7fe5a81550b0 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- 172.21.15.87:0/3842266716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a81550b0 msgr2=0x7fe5a80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 --2- 172.21.15.87:0/3842266716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a81550b0 0x7fe5a80cf930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80d04d0 tx=0x7fe59c01e930 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- 172.21.15.87:0/3842266716 shutdown_connections 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 --2- 172.21.15.87:0/3842266716 >> v2:172.21.15.169:3300/0 conn(0x7fe5a815b740 0x7fe5a806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 --2- 172.21.15.87:0/3842266716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a80cfe70 0x7fe5a815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 --2- 172.21.15.87:0/3842266716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a81550b0 0x7fe5a80cf930 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- 172.21.15.87:0/3842266716 >> 172.21.15.87:0/3842266716 conn(0x7fe5a8066b70 msgr2=0x7fe5a8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- 172.21.15.87:0/3842266716 shutdown_connections 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- 172.21.15.87:0/3842266716 wait complete. 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 Processor -- start 2024-04-25T10:13:12.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- start start 2024-04-25T10:13:12.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a80cfe70 0x7fe5a80cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a81550b0 0x7fe5a80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a815b740 0x7fe5a80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a80b3260 con 0x7fe5a815b740 2024-04-25T10:13:12.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a815bd60 con 0x7fe5a80cfe70 2024-04-25T10:13:12.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5af600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a80b33e0 con 0x7fe5a81550b0 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5ad600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a815b740 0x7fe5a80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5ad600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a815b740 0x7fe5a80cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54066/0 (socket says 172.21.15.87:54066) 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.487+0000 7fe5ad600640 1 -- 172.21.15.87:0/3690668937 learned_addr learned my addr 172.21.15.87:0/3690668937 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5acc00640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a80cfe70 0x7fe5a80cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5a7e00640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a81550b0 0x7fe5a80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5ad600640 1 -- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a81550b0 msgr2=0x7fe5a80ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5ad600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a81550b0 0x7fe5a80ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5ad600640 1 -- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a80cfe70 msgr2=0x7fe5a80cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5ad600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a80cfe70 0x7fe5a80cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5ad600640 1 -- 172.21.15.87:0/3690668937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a80ccdd0 con 0x7fe5a815b740 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5ad600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a815b740 0x7fe5a80cede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000cd50 tx=0x7fe5a000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5a4c00640 1 -- 172.21.15.87:0/3690668937 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000f680 con 0x7fe5a815b740 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5a4c00640 1 -- 172.21.15.87:0/3690668937 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a000f840 con 0x7fe5a815b740 2024-04-25T10:13:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5a4c00640 1 -- 172.21.15.87:0/3690668937 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000fc30 con 0x7fe5a815b740 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a80cd0b0 con 0x7fe5a815b740 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a80cd670 con 0x7fe5a815b740 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a804ef30 con 0x7fe5a815b740 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5a4c00640 1 -- 172.21.15.87:0/3690668937 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5a000e3e0 con 0x7fe5a815b740 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.491+0000 7fe5a4c00640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe588066f90 0x7fe588069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.495+0000 7fe5a4c00640 1 -- 172.21.15.87:0/3690668937 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe5a0085a70 con 0x7fe5a815b740 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.495+0000 7fe5acc00640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe588066f90 0x7fe588069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.495+0000 7fe5acc00640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe588066f90 0x7fe588069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0b1000 tx=0x7fe59c0a9070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.499+0000 7fe5a4c00640 1 -- 172.21.15.87:0/3690668937 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5a0014030 con 0x7fe5a815b740 2024-04-25T10:13:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.707+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe5a814bee0 con 0x7fe5a815b740 2024-04-25T10:13:12.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5a4c00640 1 -- 172.21.15.87:0/3690668937 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe5a0056dd0 con 0x7fe5a815b740 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe588066f90 msgr2=0x7fe588069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe588066f90 0x7fe588069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0b1000 tx=0x7fe59c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a815b740 msgr2=0x7fe5a80cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a815b740 0x7fe5a80cede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000cd50 tx=0x7fe5a000a650 comp rx=0 tx=0).stop 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 shutdown_connections 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe588066f90 0x7fe588069450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5a815b740 0x7fe5a80cede0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5a81550b0 0x7fe5a80ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 --2- 172.21.15.87:0/3690668937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a80cfe70 0x7fe5a80cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 >> 172.21.15.87:0/3690668937 conn(0x7fe5a8066b70 msgr2=0x7fe5a806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 shutdown_connections 2024-04-25T10:13:12.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.711+0000 7fe5af600640 1 -- 172.21.15.87:0/3690668937 wait complete. 2024-04-25T10:13:12.738 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:12.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7da00640 1 Processor -- start 2024-04-25T10:13:12.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7da00640 1 -- start start 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c780cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7da00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2c780b4690 0x7f2c780cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7da00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c780b2460 con 0x7f2c780cd590 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7da00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c780b2d70 con 0x7f2c780b4690 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78070fb0 con 0x7f2c7815b750 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2c780b4690 0x7f2c780cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c77400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2c780cd590 0x7f2c780b3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52082/0 (socket says 172.21.15.87:52082) 2024-04-25T10:13:12.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c77400640 1 -- 172.21.15.87:0/329456840 learned_addr learned my addr 172.21.15.87:0/329456840 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:12.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c77e00640 1 --2- 172.21.15.87:0/329456840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c780cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c76a00640 1 -- 172.21.15.87:0/329456840 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747097577 0 0) 0x7f2c780b2460 con 0x7f2c780cd590 2024-04-25T10:13:12.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c76a00640 1 -- 172.21.15.87:0/329456840 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c5c003530 con 0x7f2c780cd590 2024-04-25T10:13:12.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7d000640 1 -- 172.21.15.87:0/329456840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 msgr2=0x7f2c780cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7d000640 1 --2- 172.21.15.87:0/329456840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c780cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7d000640 1 -- 172.21.15.87:0/329456840 >> v1:172.21.15.87:6789/0 conn(0x7f2c780cd590 legacy=0x7f2c780b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.867+0000 7f2c7d000640 1 -- 172.21.15.87:0/329456840 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c780b2720 con 0x7f2c780b4690 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7d000640 1 --2- 172.21.15.87:0/329456840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780b4690 0x7f2c780cd9b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009d90 tx=0x7f2c7001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60f39c9eafbda54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c76a00640 1 -- 172.21.15.87:0/329456840 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c700b2070 con 0x7f2c780b4690 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c76a00640 1 -- 172.21.15.87:0/329456840 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c700a59c0 con 0x7f2c780b4690 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c76a00640 1 -- 172.21.15.87:0/329456840 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c700a5d90 con 0x7f2c780b4690 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 -- 172.21.15.87:0/329456840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780b4690 msgr2=0x7f2c780cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 --2- 172.21.15.87:0/329456840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780b4690 0x7f2c780cd9b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009d90 tx=0x7f2c7001c940 comp rx=0 tx=0).stop 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 -- 172.21.15.87:0/329456840 shutdown_connections 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 --2- 172.21.15.87:0/329456840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780b4690 0x7f2c780cd9b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 --2- 172.21.15.87:0/329456840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c780cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 -- 172.21.15.87:0/329456840 >> 172.21.15.87:0/329456840 conn(0x7f2c780743f0 msgr2=0x7f2c780669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 -- 172.21.15.87:0/329456840 shutdown_connections 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 -- 172.21.15.87:0/329456840 wait complete. 2024-04-25T10:13:12.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 Processor -- start 2024-04-25T10:13:12.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.871+0000 7f2c7da00640 1 -- start start 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780b4690 0x7f2c780adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780cd590 0x7f2c7814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c7814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78070ce0 con 0x7f2c780cd590 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78070b60 con 0x7f2c780b4690 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780b39f0 con 0x7f2c7815b750 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780cd590 0x7f2c7814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780cd590 0x7f2c7814d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54084/0 (socket says 172.21.15.87:54084) 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 -- 172.21.15.87:0/1964278217 learned_addr learned my addr 172.21.15.87:0/1964278217 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:12.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77e00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780b4690 0x7f2c780adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7d000640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c7814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 -- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 msgr2=0x7f2c7814deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c7814deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 -- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780b4690 msgr2=0x7f2c780adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780b4690 0x7f2c780adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 -- 172.21.15.87:0/1964278217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7814ecb0 con 0x7f2c780cd590 2024-04-25T10:13:12.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c77400640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780cd590 0x7f2c7814d970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c480 tx=0x7f2c6400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c6fe00640 1 -- 172.21.15.87:0/1964278217 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64017070 con 0x7f2c780cd590 2024-04-25T10:13:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c6fe00640 1 -- 172.21.15.87:0/1964278217 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64005cf0 con 0x7f2c780cd590 2024-04-25T10:13:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c780b0550 con 0x7f2c780cd590 2024-04-25T10:13:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c6fe00640 1 -- 172.21.15.87:0/1964278217 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64007290 con 0x7f2c780cd590 2024-04-25T10:13:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c780b0b10 con 0x7f2c780cd590 2024-04-25T10:13:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c78075120 con 0x7f2c780cd590 2024-04-25T10:13:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.875+0000 7f2c6fe00640 1 -- 172.21.15.87:0/1964278217 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c6401a070 con 0x7f2c780cd590 2024-04-25T10:13:12.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.879+0000 7f2c6fe00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c58066f60 0x7f2c58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.879+0000 7f2c6fe00640 1 -- 172.21.15.87:0/1964278217 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2c640840a0 con 0x7f2c780cd590 2024-04-25T10:13:12.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.879+0000 7f2c77e00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c58066f60 0x7f2c58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.879+0000 7f2c77e00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c58066f60 0x7f2c58069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009ac0 tx=0x7f2c68009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:12.883+0000 7f2c6fe00640 1 -- 172.21.15.87:0/1964278217 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c7814ecb0 con 0x7f2c780cd590 2024-04-25T10:13:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c7815f130 con 0x7f2c780cd590 2024-04-25T10:13:13.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c6fe00640 1 -- 172.21.15.87:0/1964278217 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f2c64054da0 con 0x7f2c780cd590 2024-04-25T10:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":29,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:13:10.015168+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:13.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c58066f60 msgr2=0x7f2c58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c58066f60 0x7f2c58069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009ac0 tx=0x7f2c68009190 comp rx=0 tx=0).stop 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780cd590 msgr2=0x7f2c7814d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780cd590 0x7f2c7814d970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c480 tx=0x7f2c6400c940 comp rx=0 tx=0).stop 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 shutdown_connections 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c58066f60 0x7f2c58069420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c7815b750 0x7f2c7814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c780cd590 0x7f2c7814d970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 --2- 172.21.15.87:0/1964278217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780b4690 0x7f2c780adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 >> 172.21.15.87:0/1964278217 conn(0x7f2c780743f0 msgr2=0x7f2c78151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 shutdown_connections 2024-04-25T10:13:13.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.039+0000 7f2c7da00640 1 -- 172.21.15.87:0/1964278217 wait complete. 2024-04-25T10:13:13.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:13.067 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:13.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.199+0000 7f3393200640 1 Processor -- start 2024-04-25T10:13:13.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.199+0000 7f3393200640 1 -- start start 2024-04-25T10:13:13.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3393200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f339415b750 0x7f33940cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3393200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33940cd590 0x7f33940cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3393200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33940b2460 con 0x7f33940b3f80 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3393200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33940b2d70 con 0x7f33940cd590 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3393200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3394070fb0 con 0x7f339415b750 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f338be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33940cd590 0x7f33940cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3392800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f33940b3f80 0x7f33940b3200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:55814/0 (socket says 172.21.15.87:55814) 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3392800640 1 -- 172.21.15.87:0/3404480921 learned_addr learned my addr 172.21.15.87:0/3404480921 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3391e00640 1 --2- 172.21.15.87:0/3404480921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f339415b750 0x7f33940cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3391e00640 1 -- 172.21.15.87:0/3404480921 >> v2:172.21.15.169:3300/0 conn(0x7f33940cd590 msgr2=0x7f33940cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3391e00640 1 --2- 172.21.15.87:0/3404480921 >> v2:172.21.15.169:3300/0 conn(0x7f33940cd590 0x7f33940cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3391e00640 1 -- 172.21.15.87:0/3404480921 >> v1:172.21.15.169:6789/0 conn(0x7f33940b3f80 legacy=0x7f33940b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f3391e00640 1 -- 172.21.15.87:0/3404480921 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940b2720 con 0x7f339415b750 2024-04-25T10:13:13.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.203+0000 7f338be00640 1 --2- 172.21.15.87:0/3404480921 >> v2:172.21.15.169:3300/0 conn(0x7f33940cd590 0x7f33940cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:13.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3391e00640 1 --2- 172.21.15.87:0/3404480921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f339415b750 0x7f33940cbc30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33940b2ef0 tx=0x7f338401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d559ee92b1179ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3391400640 1 -- 172.21.15.87:0/3404480921 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33840ab430 con 0x7f339415b750 2024-04-25T10:13:13.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3391400640 1 -- 172.21.15.87:0/3404480921 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33840b0070 con 0x7f339415b750 2024-04-25T10:13:13.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3391400640 1 -- 172.21.15.87:0/3404480921 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33840ab7f0 con 0x7f339415b750 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 -- 172.21.15.87:0/3404480921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f339415b750 msgr2=0x7f33940cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 --2- 172.21.15.87:0/3404480921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f339415b750 0x7f33940cbc30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33940b2ef0 tx=0x7f338401c470 comp rx=0 tx=0).stop 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 -- 172.21.15.87:0/3404480921 shutdown_connections 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 --2- 172.21.15.87:0/3404480921 >> v2:172.21.15.169:3300/0 conn(0x7f33940cd590 0x7f33940cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 --2- 172.21.15.87:0/3404480921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f339415b750 0x7f33940cbc30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 -- 172.21.15.87:0/3404480921 >> 172.21.15.87:0/3404480921 conn(0x7f33940743f0 msgr2=0x7f33940669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 -- 172.21.15.87:0/3404480921 shutdown_connections 2024-04-25T10:13:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 -- 172.21.15.87:0/3404480921 wait complete. 2024-04-25T10:13:13.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.207+0000 7f3393200640 1 Processor -- start 2024-04-25T10:13:13.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 -- start start 2024-04-25T10:13:13.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33940b3f80 0x7f339415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33940cd590 0x7f339415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f339415b750 0x7f339414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33940b47a0 con 0x7f339415b750 2024-04-25T10:13:13.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394070b60 con 0x7f33940cd590 2024-04-25T10:13:13.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33940abd70 con 0x7f33940b3f80 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3392800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f339415b750 0x7f339414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33940cd590 0x7f339415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3391e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33940b3f80 0x7f339415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33940cd590 0x7f339415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:35636/0 (socket says 172.21.15.87:35636) 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 -- 172.21.15.87:0/1996754373 learned_addr learned my addr 172.21.15.87:0/1996754373 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 -- 172.21.15.87:0/1996754373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33940b3f80 msgr2=0x7f339415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33940b3f80 0x7f339415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 -- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f339415b750 msgr2=0x7f339414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f339415b750 0x7f339414ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 -- 172.21.15.87:0/1996754373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339415d820 con 0x7f33940cd590 2024-04-25T10:13:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3392800640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f339415b750 0x7f339414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338be00640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33940cd590 0x7f339415e8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f337c00ebe0 tx=0x7f337c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338a000640 1 -- 172.21.15.87:0/1996754373 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c017070 con 0x7f33940cd590 2024-04-25T10:13:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338a000640 1 -- 172.21.15.87:0/1996754373 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c00cc30 con 0x7f33940cd590 2024-04-25T10:13:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f338a000640 1 -- 172.21.15.87:0/1996754373 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c013470 con 0x7f33940cd590 2024-04-25T10:13:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339415db00 con 0x7f33940cd590 2024-04-25T10:13:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339415e3a0 con 0x7f33940cd590 2024-04-25T10:13:13.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.211+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339415dcd0 con 0x7f33940cd590 2024-04-25T10:13:13.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.215+0000 7f338a000640 1 -- 172.21.15.87:0/1996754373 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f337c01a070 con 0x7f33940cd590 2024-04-25T10:13:13.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.215+0000 7f338a000640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3370066f60 0x7f3370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.215+0000 7f338a000640 1 -- 172.21.15.87:0/1996754373 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f337c084fd0 con 0x7f33940cd590 2024-04-25T10:13:13.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.215+0000 7f3391e00640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3370066f60 0x7f3370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.215+0000 7f3391e00640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3370066f60 0x7f3370069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f33840af000 tx=0x7f33840b6040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.219+0000 7f338a000640 1 -- 172.21.15.87:0/1996754373 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f337c055c40 con 0x7f33940cd590 2024-04-25T10:13:13.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.375+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3394075120 con 0x7f33940cd590 2024-04-25T10:13:13.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f338a000640 1 -- 172.21.15.87:0/1996754373 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f337c059ce0 con 0x7f33940cd590 2024-04-25T10:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3370066f60 msgr2=0x7f3370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3370066f60 0x7f3370069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f33840af000 tx=0x7f33840b6040 comp rx=0 tx=0).stop 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33940cd590 msgr2=0x7f339415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33940cd590 0x7f339415e8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f337c00ebe0 tx=0x7f337c00c520 comp rx=0 tx=0).stop 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 shutdown_connections 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3370066f60 0x7f3370069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f339415b750 0x7f339414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33940cd590 0x7f339415e8c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 --2- 172.21.15.87:0/1996754373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33940b3f80 0x7f339415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 >> 172.21.15.87:0/1996754373 conn(0x7f33940743f0 msgr2=0x7f33940b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 shutdown_connections 2024-04-25T10:13:13.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.379+0000 7f3393200640 1 -- 172.21.15.87:0/1996754373 wait complete. 2024-04-25T10:13:13.406 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:13:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f51e00640 1 Processor -- start 2024-04-25T10:13:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f51e00640 1 -- start start 2024-04-25T10:13:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f51e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f4c15b9e0 0x7f5f4c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f51e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0ccfb0 con 0x7f5f4c15bdc0 2024-04-25T10:13:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f51e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0cd290 con 0x7f5f4c0d1c30 2024-04-25T10:13:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f51e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cd570 con 0x7f5f4c15b9e0 2024-04-25T10:13:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f4a000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5f4c15bdc0 0x7f5f4c0d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52096/0 (socket says 172.21.15.87:52096) 2024-04-25T10:13:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f4a000640 1 -- 172.21.15.87:0/2492462021 learned_addr learned my addr 172.21.15.87:0/2492462021 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f4aa00640 1 --2- 172.21.15.87:0/2492462021 >> v2:172.21.15.169:3300/0 conn(0x7f5f4c15b9e0 0x7f5f4c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807281966 0 0) 0x7f5f4c0ccfb0 con 0x7f5f4c15bdc0 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f30003530 con 0x7f5f4c15bdc0 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3958474095 0 0) 0x7f5f30003530 con 0x7f5f4c15bdc0 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 >> v2:172.21.15.169:3300/0 conn(0x7f5f4c15b9e0 msgr2=0x7f5f4c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 --2- 172.21.15.87:0/2492462021 >> v2:172.21.15.169:3300/0 conn(0x7f5f4c15b9e0 0x7f5f4c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 >> v1:172.21.15.169:6789/0 conn(0x7f5f4c0d1c30 legacy=0x7f5f4c0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0cd850 con 0x7f5f4c15bdc0 2024-04-25T10:13:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5f38001680 con 0x7f5f4c15bdc0 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 >> v1:172.21.15.87:6789/0 conn(0x7f5f4c15bdc0 legacy=0x7f5f4c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300039c0 0x7f5f30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f30004810 0x7f5f30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f300051d0 0x7f5f3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cd570 con 0x7f5f300051d0 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3000c170 con 0x7f5f30004810 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3000c370 con 0x7f5f300039c0 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f4b400640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f300051d0 0x7f5f3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f4a000640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f30004810 0x7f5f30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.535+0000 7f5f4aa00640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300039c0 0x7f5f30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f4b400640 1 -- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300039c0 msgr2=0x7f5f30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f4b400640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300039c0 0x7f5f30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f4b400640 1 -- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f30004810 msgr2=0x7f5f30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f4b400640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f30004810 0x7f5f30004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f4b400640 1 -- 172.21.15.87:0/2492462021 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30003530 con 0x7f5f300051d0 2024-04-25T10:13:13.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f4b400640 1 -- 172.21.15.87:0/2492462021 reap_dead start 2024-04-25T10:13:13.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f4b400640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f300051d0 0x7f5f3000b9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f44001e70 tx=0x7f5f4401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e40e089d1aaea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44001a90 con 0x7f5f300051d0 2024-04-25T10:13:13.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f44003bc0 con 0x7f5f300051d0 2024-04-25T10:13:13.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f49600640 1 -- 172.21.15.87:0/2492462021 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440050d0 con 0x7f5f300051d0 2024-04-25T10:13:13.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f51e00640 1 -- 172.21.15.87:0/2492462021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f300051d0 msgr2=0x7f5f3000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f300051d0 0x7f5f3000b9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f44001e70 tx=0x7f5f4401c780 comp rx=0 tx=0).stop 2024-04-25T10:13:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f51e00640 1 -- 172.21.15.87:0/2492462021 shutdown_connections 2024-04-25T10:13:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2492462021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f300051d0 0x7f5f3000b9d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f51e00640 1 -- 172.21.15.87:0/2492462021 >> 172.21.15.87:0/2492462021 conn(0x7f5f4c09a9d0 msgr2=0x7f5f4c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f51e00640 1 -- 172.21.15.87:0/2492462021 shutdown_connections 2024-04-25T10:13:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.539+0000 7f5f51e00640 1 -- 172.21.15.87:0/2492462021 wait complete. 2024-04-25T10:13:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 Processor -- start 2024-04-25T10:13:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 -- start start 2024-04-25T10:13:13.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300051d0 0x7f5f4c14c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c14c8a0 0x7f5f4c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c06c230 0x7f5f4c151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0d1520 con 0x7f5f4c06c230 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cf740 con 0x7f5f4c14c8a0 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0d16a0 con 0x7f5f300051d0 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c06c230 0x7f5f4c151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c06c230 0x7f5f4c151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54116/0 (socket says 172.21.15.87:54116) 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 -- 172.21.15.87:0/2367619713 learned_addr learned my addr 172.21.15.87:0/2367619713 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4a000640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c14c8a0 0x7f5f4c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4aa00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300051d0 0x7f5f4c14c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 -- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300051d0 msgr2=0x7f5f4c14c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300051d0 0x7f5f4c14c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 -- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c14c8a0 msgr2=0x7f5f4c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c14c8a0 0x7f5f4c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 -- 172.21.15.87:0/2367619713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0cbae0 con 0x7f5f4c06c230 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4a000640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c14c8a0 0x7f5f4c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:13.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4aa00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300051d0 0x7f5f4c14c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f4b400640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c06c230 0x7f5f4c151870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401ccf0 tx=0x7f5f30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f43400640 1 -- 172.21.15.87:0/2367619713 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440ae070 con 0x7f5f4c06c230 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f43400640 1 -- 172.21.15.87:0/2367619713 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f440012e0 con 0x7f5f4c06c230 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f43400640 1 -- 172.21.15.87:0/2367619713 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44006d60 con 0x7f5f4c06c230 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c0c8a70 con 0x7f5f4c06c230 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.543+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c152620 con 0x7f5f4c06c230 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.547+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c04ef30 con 0x7f5f4c06c230 2024-04-25T10:13:13.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.547+0000 7f5f43400640 1 -- 172.21.15.87:0/2367619713 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f440052e0 con 0x7f5f4c06c230 2024-04-25T10:13:13.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.547+0000 7f5f43400640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f2c066f60 0x7f5f2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.547+0000 7f5f43400640 1 -- 172.21.15.87:0/2367619713 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5f4411d730 con 0x7f5f4c06c230 2024-04-25T10:13:13.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.547+0000 7f5f4aa00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f2c066f60 0x7f5f2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.547+0000 7f5f4aa00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f2c066f60 0x7f5f2c069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009a60 tx=0x7f5f3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.551+0000 7f5f43400640 1 -- 172.21.15.87:0/2367619713 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f440b9030 con 0x7f5f4c06c230 2024-04-25T10:13:13.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.707+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f4c0d05b0 con 0x7f5f4c06c230 2024-04-25T10:13:13.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.707+0000 7f5f43400640 1 -- 172.21.15.87:0/2367619713 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5f44016490 con 0x7f5f4c06c230 2024-04-25T10:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:13:13.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f2c066f60 msgr2=0x7f5f2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f2c066f60 0x7f5f2c069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009a60 tx=0x7f5f3c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:13.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c06c230 msgr2=0x7f5f4c151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c06c230 0x7f5f4c151870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401ccf0 tx=0x7f5f30005840 comp rx=0 tx=0).stop 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 shutdown_connections 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f2c066f60 0x7f5f2c069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c06c230 0x7f5f4c151870 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c14c8a0 0x7f5f4c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 --2- 172.21.15.87:0/2367619713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f300051d0 0x7f5f4c14c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 >> 172.21.15.87:0/2367619713 conn(0x7f5f4c09a9d0 msgr2=0x7f5f4c0c58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 shutdown_connections 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:13.711+0000 7f5f51e00640 1 -- 172.21.15.87:0/2367619713 wait complete. 2024-04-25T10:13:13.884 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:13:13.884 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:13:14.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a7200640 1 Processor -- start 2024-04-25T10:13:14.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a7200640 1 -- start start 2024-04-25T10:13:14.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a7200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5a815bb50 0x7fb5a80b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a80677c0 con 0x7fb5a806d760 2024-04-25T10:13:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a80cc8f0 con 0x7fb5a80cc300 2024-04-25T10:13:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a7200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a80cceb0 con 0x7fb5a815bb50 2024-04-25T10:13:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5a815bb50 0x7fb5a80b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a5400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb5a806d760 0x7fb5a80669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52104/0 (socket says 172.21.15.87:52104) 2024-04-25T10:13:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a5400640 1 -- 172.21.15.87:0/3905724645 learned_addr learned my addr 172.21.15.87:0/3905724645 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:14.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a5e00640 1 -- 172.21.15.87:0/3905724645 >> v1:172.21.15.169:6789/0 conn(0x7fb5a80cc300 legacy=0x7fb5a80670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a5e00640 1 -- 172.21.15.87:0/3905724645 >> v1:172.21.15.87:6789/0 conn(0x7fb5a806d760 legacy=0x7fb5a80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a5e00640 1 -- 172.21.15.87:0/3905724645 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80ccbd0 con 0x7fb5a815bb50 2024-04-25T10:13:14.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.019+0000 7fb5a4a00640 1 -- 172.21.15.87:0/3905724645 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359144819 0 0) 0x7fb5a80677c0 con 0x7fb5a806d760 2024-04-25T10:13:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a5e00640 1 --2- 172.21.15.87:0/3905724645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a815bb50 0x7fb5a80b2350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb5980097c0 tx=0x7fb59801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d775a1fd7b436d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a4a00640 1 -- 172.21.15.87:0/3905724645 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980b0070 con 0x7fb5a815bb50 2024-04-25T10:13:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a4a00640 1 -- 172.21.15.87:0/3905724645 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5980ab600 con 0x7fb5a815bb50 2024-04-25T10:13:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a4a00640 1 -- 172.21.15.87:0/3905724645 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980ab9d0 con 0x7fb5a815bb50 2024-04-25T10:13:14.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 -- 172.21.15.87:0/3905724645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a815bb50 msgr2=0x7fb5a80b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3905724645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a815bb50 0x7fb5a80b2350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb5980097c0 tx=0x7fb59801c3e0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 -- 172.21.15.87:0/3905724645 shutdown_connections 2024-04-25T10:13:14.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3905724645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a815bb50 0x7fb5a80b2350 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 -- 172.21.15.87:0/3905724645 >> 172.21.15.87:0/3905724645 conn(0x7fb5a80cd4b0 msgr2=0x7fb5a80cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 -- 172.21.15.87:0/3905724645 shutdown_connections 2024-04-25T10:13:14.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 -- 172.21.15.87:0/3905724645 wait complete. 2024-04-25T10:13:14.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.023+0000 7fb5a7200640 1 Processor -- start 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 -- start start 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5a806d760 0x7fb5a80b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a80cc300 0x7fb5a8153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5a815bb50 0x7fb5a81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a815bf30 con 0x7fb5a80cc300 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80b2ca0 con 0x7fb5a806d760 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a816a350 con 0x7fb5a815bb50 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a80cc300 0x7fb5a8153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a80cc300 0x7fb5a8153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54140/0 (socket says 172.21.15.87:54140) 2024-04-25T10:13:14.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 -- 172.21.15.87:0/3796505181 learned_addr learned my addr 172.21.15.87:0/3796505181 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5e00640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5a806d760 0x7fb5a80b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a6800640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5a815bb50 0x7fb5a81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 -- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5a815bb50 msgr2=0x7fb5a81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5a815bb50 0x7fb5a81535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 -- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5a806d760 msgr2=0x7fb5a80b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5a806d760 0x7fb5a80b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 -- 172.21.15.87:0/3796505181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80c5ca0 con 0x7fb5a80cc300 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a5400640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a80cc300 0x7fb5a8153060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c440 tx=0x7fb59400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb59e000640 1 -- 172.21.15.87:0/3796505181 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594018070 con 0x7fb5a80cc300 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb59e000640 1 -- 172.21.15.87:0/3796505181 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5940074e0 con 0x7fb5a80cc300 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a80c5f80 con 0x7fb5a80cc300 2024-04-25T10:13:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb59e000640 1 -- 172.21.15.87:0/3796505181 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594004a10 con 0x7fb5a80cc300 2024-04-25T10:13:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a80c6540 con 0x7fb5a80cc300 2024-04-25T10:13:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a8075120 con 0x7fb5a80cc300 2024-04-25T10:13:14.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.031+0000 7fb59e000640 1 -- 172.21.15.87:0/3796505181 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb59401b070 con 0x7fb5a80cc300 2024-04-25T10:13:14.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.031+0000 7fb59e000640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb588066f60 0x7fb588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.031+0000 7fb59e000640 1 -- 172.21.15.87:0/3796505181 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb594083f80 con 0x7fb5a80cc300 2024-04-25T10:13:14.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.031+0000 7fb5a5e00640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb588066f60 0x7fb588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.031+0000 7fb5a5e00640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb588066f60 0x7fb588069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ad000 tx=0x7fb5980a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.035+0000 7fb59e000640 1 -- 172.21.15.87:0/3796505181 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb594054bf0 con 0x7fb5a80cc300 2024-04-25T10:13:14.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:14.191+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb5a80cdaa0 con 0x7fb5a80cc300 2024-04-25T10:13:15.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.023+0000 7fb59e000640 1 -- 172.21.15.87:0/3796505181 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb594058c90 con 0x7fb5a80cc300 2024-04-25T10:13:15.032 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb588066f60 msgr2=0x7fb588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb588066f60 0x7fb588069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ad000 tx=0x7fb5980a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a80cc300 msgr2=0x7fb5a8153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a80cc300 0x7fb5a8153060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c440 tx=0x7fb59400c900 comp rx=0 tx=0).stop 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 shutdown_connections 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb588066f60 0x7fb588069420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5a815bb50 0x7fb5a81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5a80cc300 0x7fb5a8153060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 --2- 172.21.15.87:0/3796505181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5a806d760 0x7fb5a80b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 >> 172.21.15.87:0/3796505181 conn(0x7fb5a80cd4b0 msgr2=0x7fb5a80ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 shutdown_connections 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.027+0000 7fb5a7200640 1 -- 172.21.15.87:0/3796505181 wait complete. 2024-04-25T10:13:15.061 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:13:15.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.199+0000 7f205ae00640 1 Processor -- start 2024-04-25T10:13:15.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.199+0000 7f205ae00640 1 -- start start 2024-04-25T10:13:15.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f205ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f20540cf960 0x7f20540cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f205ae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205406ca90 con 0x7f205415bbb0 2024-04-25T10:13:15.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f205ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205406cd70 con 0x7f20540b4660 2024-04-25T10:13:15.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f205ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205406d050 con 0x7f20540cf960 2024-04-25T10:13:15.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2053e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f205415bbb0 0x7f20540cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52112/0 (socket says 172.21.15.87:52112) 2024-04-25T10:13:15.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2053e00640 1 -- 172.21.15.87:0/2444857771 learned_addr learned my addr 172.21.15.87:0/2444857771 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:15.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2058e00640 1 --2- 172.21.15.87:0/2444857771 >> v2:172.21.15.169:3300/0 conn(0x7f20540cf960 0x7f20540cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438500671 0 0) 0x7f205406ca90 con 0x7f205415bbb0 2024-04-25T10:13:15.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2038003530 con 0x7f205415bbb0 2024-04-25T10:13:15.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109875325 0 0) 0x7f205406cd70 con 0x7f20540b4660 2024-04-25T10:13:15.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f205406ca90 con 0x7f20540b4660 2024-04-25T10:13:15.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2058e00640 1 -- 172.21.15.87:0/2444857771 >> v1:172.21.15.169:6789/0 conn(0x7f20540b4660 legacy=0x7f205406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2905960521 0 0) 0x7f2038003530 con 0x7f205415bbb0 2024-04-25T10:13:15.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2058e00640 1 -- 172.21.15.87:0/2444857771 >> v1:172.21.15.87:6789/0 conn(0x7f205415bbb0 legacy=0x7f20540cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.203+0000 7f2058e00640 1 -- 172.21.15.87:0/2444857771 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540b2880 con 0x7f20540cf960 2024-04-25T10:13:15.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f2058e00640 1 --2- 172.21.15.87:0/2444857771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f20540cfe10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f204c009d90 tx=0x7f204c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e655d6a081c9c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044001680 con 0x7f20540cf960 2024-04-25T10:13:15.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c0a64e0 con 0x7f20540cf960 2024-04-25T10:13:15.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f2053400640 1 -- 172.21.15.87:0/2444857771 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0a68b0 con 0x7f20540cf960 2024-04-25T10:13:15.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 -- 172.21.15.87:0/2444857771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 msgr2=0x7f20540cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 --2- 172.21.15.87:0/2444857771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f20540cfe10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f204c009d90 tx=0x7f204c01c890 comp rx=0 tx=0).stop 2024-04-25T10:13:15.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 -- 172.21.15.87:0/2444857771 shutdown_connections 2024-04-25T10:13:15.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 --2- 172.21.15.87:0/2444857771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f20540cfe10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 -- 172.21.15.87:0/2444857771 >> 172.21.15.87:0/2444857771 conn(0x7f2054066b70 msgr2=0x7f2054066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 -- 172.21.15.87:0/2444857771 shutdown_connections 2024-04-25T10:13:15.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 -- 172.21.15.87:0/2444857771 wait complete. 2024-04-25T10:13:15.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.207+0000 7f205ae00640 1 Processor -- start 2024-04-25T10:13:15.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 -- start start 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20540b4660 0x7f205414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f2054144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f205415bbb0 0x7f20541568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20540b3c20 con 0x7f205415bbb0 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20540b3aa0 con 0x7f20540cf960 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205406c830 con 0x7f20540b4660 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f205415bbb0 0x7f20541568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f205415bbb0 0x7f20541568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54154/0 (socket says 172.21.15.87:54154) 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 -- 172.21.15.87:0/2623006411 learned_addr learned my addr 172.21.15.87:0/2623006411 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f204b400640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f2054144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2053e00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20540b4660 0x7f205414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 -- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20540b4660 msgr2=0x7f205414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20540b4660 0x7f205414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 -- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 msgr2=0x7f2054144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f2054144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 -- 172.21.15.87:0/2623006411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20541461d0 con 0x7f205415bbb0 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f204b400640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f2054144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2058e00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f205415bbb0 0x7f20541568f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f204c0194a0 tx=0x7f204c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2050c00640 1 -- 172.21.15.87:0/2623006411 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0b8840 con 0x7f205415bbb0 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2050c00640 1 -- 172.21.15.87:0/2623006411 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c0c1070 con 0x7f205415bbb0 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f2050c00640 1 -- 172.21.15.87:0/2623006411 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0b8c20 con 0x7f205415bbb0 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20541464b0 con 0x7f205415bbb0 2024-04-25T10:13:15.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2054146a70 con 0x7f205415bbb0 2024-04-25T10:13:15.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.211+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205404ef30 con 0x7f205415bbb0 2024-04-25T10:13:15.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.215+0000 7f2050c00640 1 -- 172.21.15.87:0/2623006411 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f204c0c6460 con 0x7f205415bbb0 2024-04-25T10:13:15.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.215+0000 7f2050c00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2030066f60 0x7f2030069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.215+0000 7f2050c00640 1 -- 172.21.15.87:0/2623006411 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f204c120ad0 con 0x7f205415bbb0 2024-04-25T10:13:15.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.215+0000 7f2053e00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2030066f60 0x7f2030069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.215+0000 7f2053e00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2030066f60 0x7f2030069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2044001ec0 tx=0x7f2044001cb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.219+0000 7f2050c00640 1 -- 172.21.15.87:0/2623006411 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f204c016480 con 0x7f205415bbb0 2024-04-25T10:13:15.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:15.375+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f205406d980 con 0x7f205415bbb0 2024-04-25T10:13:16.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.031+0000 7f2050c00640 1 -- 172.21.15.87:0/2623006411 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f204c0f1650 con 0x7f205415bbb0 2024-04-25T10:13:16.039 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2030066f60 msgr2=0x7f2030069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2030066f60 0x7f2030069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2044001ec0 tx=0x7f2044001cb0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f205415bbb0 msgr2=0x7f20541568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f205415bbb0 0x7f20541568f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f204c0194a0 tx=0x7f204c019630 comp rx=0 tx=0).stop 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 shutdown_connections 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2030066f60 0x7f2030069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f205415bbb0 0x7f20541568f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20540cf960 0x7f2054144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 --2- 172.21.15.87:0/2623006411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20540b4660 0x7f205414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 >> 172.21.15.87:0/2623006411 conn(0x7f2054066b70 msgr2=0x7f2054066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 shutdown_connections 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.035+0000 7f205ae00640 1 -- 172.21.15.87:0/2623006411 wait complete. 2024-04-25T10:13:16.080 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:13:16.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ead600640 1 Processor -- start 2024-04-25T10:13:16.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ead600640 1 -- start start 2024-04-25T10:13:16.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ead600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ea8159540 0x7f4ea80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ead600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80721f0 0x7f4ea8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ead600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea8159920 con 0x7f4ea80aebd0 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ead600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea815c2f0 con 0x7f4ea8159540 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ead600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea815c5d0 con 0x7f4ea80721f0 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ea8159540 0x7f4ea80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ea8159540 0x7f4ea80ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54168/0 (socket says 172.21.15.87:54168) 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea6a00640 1 -- 172.21.15.87:0/4292907628 learned_addr learned my addr 172.21.15.87:0/4292907628 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:16.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea7400640 1 --2- 172.21.15.87:0/4292907628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80721f0 0x7f4ea8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea6a00640 1 -- 172.21.15.87:0/4292907628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80721f0 msgr2=0x7f4ea8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea6a00640 1 --2- 172.21.15.87:0/4292907628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80721f0 0x7f4ea8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea6a00640 1 -- 172.21.15.87:0/4292907628 >> v1:172.21.15.169:6789/0 conn(0x7f4ea80aebd0 legacy=0x7f4ea8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea6a00640 1 -- 172.21.15.87:0/4292907628 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea815c8b0 con 0x7f4ea8159540 2024-04-25T10:13:16.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.215+0000 7f4ea7400640 1 --2- 172.21.15.87:0/4292907628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80721f0 0x7f4ea8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:16.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ea6a00640 1 --2- 172.21.15.87:0/4292907628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea8159540 0x7f4ea80ae690 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e9801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c329586c92366d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ea6000640 1 -- 172.21.15.87:0/4292907628 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980ab5f0 con 0x7f4ea8159540 2024-04-25T10:13:16.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ea6000640 1 -- 172.21.15.87:0/4292907628 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e980b0070 con 0x7f4ea8159540 2024-04-25T10:13:16.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ea6000640 1 -- 172.21.15.87:0/4292907628 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980ab9b0 con 0x7f4ea8159540 2024-04-25T10:13:16.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 -- 172.21.15.87:0/4292907628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea8159540 msgr2=0x7f4ea80ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 --2- 172.21.15.87:0/4292907628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea8159540 0x7f4ea80ae690 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e9801c520 comp rx=0 tx=0).stop 2024-04-25T10:13:16.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 -- 172.21.15.87:0/4292907628 shutdown_connections 2024-04-25T10:13:16.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 --2- 172.21.15.87:0/4292907628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80721f0 0x7f4ea8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 --2- 172.21.15.87:0/4292907628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea8159540 0x7f4ea80ae690 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 -- 172.21.15.87:0/4292907628 >> 172.21.15.87:0/4292907628 conn(0x7f4ea80669a0 msgr2=0x7f4ea8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 -- 172.21.15.87:0/4292907628 shutdown_connections 2024-04-25T10:13:16.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.219+0000 7f4ead600640 1 -- 172.21.15.87:0/4292907628 wait complete. 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 Processor -- start 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 -- start start 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea80721f0 0x7f4ea814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80aebd0 0x7f4ea8151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8159540 0x7f4ea8153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea8071220 con 0x7f4ea80721f0 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80710a0 con 0x7f4ea8159540 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80713a0 con 0x7f4ea80aebd0 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea80721f0 0x7f4ea814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea80721f0 0x7f4ea814d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54170/0 (socket says 172.21.15.87:54170) 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 -- 172.21.15.87:0/2832038492 learned_addr learned my addr 172.21.15.87:0/2832038492 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4e9e000640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80aebd0 0x7f4ea8151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea7400640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8159540 0x7f4ea8153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 -- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80aebd0 msgr2=0x7f4ea8151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80aebd0 0x7f4ea8151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 -- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8159540 msgr2=0x7f4ea8153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8159540 0x7f4ea8153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 -- 172.21.15.87:0/2832038492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea80cf220 con 0x7f4ea80721f0 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ea6a00640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea80721f0 0x7f4ea814d4a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4e980ad040 tx=0x7f4e9801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4e9f400640 1 -- 172.21.15.87:0/2832038492 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980be5f0 con 0x7f4ea80721f0 2024-04-25T10:13:16.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4e9f400640 1 -- 172.21.15.87:0/2832038492 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e980be7b0 con 0x7f4ea80721f0 2024-04-25T10:13:16.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4e9f400640 1 -- 172.21.15.87:0/2832038492 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980beba0 con 0x7f4ea80721f0 2024-04-25T10:13:16.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea8151de0 con 0x7f4ea80721f0 2024-04-25T10:13:16.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.223+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea81523a0 con 0x7f4ea80721f0 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.227+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea804ef30 con 0x7f4ea80721f0 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.227+0000 7f4e9f400640 1 -- 172.21.15.87:0/2832038492 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e980c9430 con 0x7f4ea80721f0 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.227+0000 7f4e9f400640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e84066f60 0x7f4e84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.227+0000 7f4e9f400640 1 -- 172.21.15.87:0/2832038492 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4e9811f8b0 con 0x7f4ea80721f0 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.227+0000 7f4e9e000640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e84066f60 0x7f4e84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.227+0000 7f4e9e000640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e84066f60 0x7f4e84069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4ea81f8360 tx=0x7f4e90009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.231+0000 7f4e9f400640 1 -- 172.21.15.87:0/2832038492 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e98016110 con 0x7f4ea80721f0 2024-04-25T10:13:16.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:16.415+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4ea806c5a0 con 0x7f4ea80721f0 2024-04-25T10:13:17.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.035+0000 7f4e9f400640 1 -- 172.21.15.87:0/2832038492 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4e980f0440 con 0x7f4ea80721f0 2024-04-25T10:13:17.041 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T10:13:17.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e84066f60 msgr2=0x7f4e84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e84066f60 0x7f4e84069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4ea81f8360 tx=0x7f4e90009040 comp rx=0 tx=0).stop 2024-04-25T10:13:17.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea80721f0 msgr2=0x7f4ea814d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea80721f0 0x7f4ea814d4a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4e980ad040 tx=0x7f4e9801ad60 comp rx=0 tx=0).stop 2024-04-25T10:13:17.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 shutdown_connections 2024-04-25T10:13:17.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e84066f60 0x7f4e84069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8159540 0x7f4ea8153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80aebd0 0x7f4ea8151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 --2- 172.21.15.87:0/2832038492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea80721f0 0x7f4ea814d4a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.039+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 >> 172.21.15.87:0/2832038492 conn(0x7f4ea80669a0 msgr2=0x7f4ea815a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.043+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 shutdown_connections 2024-04-25T10:13:17.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.043+0000 7f4ead600640 1 -- 172.21.15.87:0/2832038492 wait complete. 2024-04-25T10:13:17.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:17.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24faa00640 1 Processor -- start 2024-04-25T10:13:17.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24faa00640 1 -- start start 2024-04-25T10:13:17.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24faa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f24f41550b0 0x7f24f40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24faa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f4071050 con 0x7f24f40cfe70 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24faa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f415d040 con 0x7f24f415b910 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24faa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f415bd80 con 0x7f24f41550b0 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24f3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f24f40cfe70 0x7f24f415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52122/0 (socket says 172.21.15.87:52122) 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24f3400640 1 -- 172.21.15.87:0/3029581718 learned_addr learned my addr 172.21.15.87:0/3029581718 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.223+0000 7f24f3e00640 1 --2- 172.21.15.87:0/3029581718 >> v2:172.21.15.169:3300/0 conn(0x7f24f41550b0 0x7f24f40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739282650 0 0) 0x7f24f4071050 con 0x7f24f40cfe70 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d8005740 con 0x7f24f40cfe70 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934791809 0 0) 0x7f24f415d040 con 0x7f24f415b910 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f4071050 con 0x7f24f415b910 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283766289 0 0) 0x7f24d8005740 con 0x7f24f40cfe70 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 >> v2:172.21.15.169:3300/0 conn(0x7f24f41550b0 msgr2=0x7f24f40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 --2- 172.21.15.87:0/3029581718 >> v2:172.21.15.169:3300/0 conn(0x7f24f41550b0 0x7f24f40cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 >> v1:172.21.15.169:6789/0 conn(0x7f24f415b910 legacy=0x7f24f40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40712f0 con 0x7f24f40cfe70 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f24e4001680 con 0x7f24f40cfe70 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f3e00640 1 --2- 172.21.15.87:0/3029581718 >> v2:172.21.15.169:3300/0 conn(0x7f24f41550b0 0x7f24f40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 >> v1:172.21.15.87:6789/0 conn(0x7f24f40cfe70 legacy=0x7f24f415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d8005d60 0x7f24d8006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d8006a20 0x7f24d8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d80073e0 0x7f24d800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f415bd80 con 0x7f24d80073e0 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d800e260 con 0x7f24d8006a20 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d800e460 con 0x7f24d8005d60 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f8a00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d80073e0 0x7f24d800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f3400640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d8006a20 0x7f24d8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f3e00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d8005d60 0x7f24d8006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f8a00640 1 -- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d8005d60 msgr2=0x7f24d8006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f8a00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d8005d60 0x7f24d8006160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f8a00640 1 -- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d8006a20 msgr2=0x7f24d8006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f8a00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d8006a20 0x7f24d8006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.227+0000 7f24f8a00640 1 -- 172.21.15.87:0/3029581718 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8005740 con 0x7f24d80073e0 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24f8a00640 1 -- 172.21.15.87:0/3029581718 reap_dead start 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24f8a00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d80073e0 0x7f24d800dbe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f24ec001fd0 tx=0x7f24ec01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9222770ce0e47233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec001c20 con 0x7f24d80073e0 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec003b80 con 0x7f24d80073e0 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24f2a00640 1 -- 172.21.15.87:0/3029581718 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0050d0 con 0x7f24d80073e0 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- 172.21.15.87:0/3029581718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d80073e0 msgr2=0x7f24d800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d80073e0 0x7f24d800dbe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f24ec001fd0 tx=0x7f24ec01c900 comp rx=0 tx=0).stop 2024-04-25T10:13:17.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- 172.21.15.87:0/3029581718 shutdown_connections 2024-04-25T10:13:17.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 --2- 172.21.15.87:0/3029581718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d80073e0 0x7f24d800dbe0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- 172.21.15.87:0/3029581718 >> 172.21.15.87:0/3029581718 conn(0x7f24f4066b70 msgr2=0x7f24f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- 172.21.15.87:0/3029581718 shutdown_connections 2024-04-25T10:13:17.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- 172.21.15.87:0/3029581718 wait complete. 2024-04-25T10:13:17.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 Processor -- start 2024-04-25T10:13:17.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- start start 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80073e0 0x7f24f40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f40ce5f0 0x7f24f41f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24f4157840 0x7f24f4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f4071b90 con 0x7f24f40ce5f0 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40718b0 con 0x7f24d80073e0 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24faa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40720b0 con 0x7f24f4157840 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.231+0000 7f24f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f40ce5f0 0x7f24f41f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f40ce5f0 0x7f24f41f2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54178/0 (socket says 172.21.15.87:54178) 2024-04-25T10:13:17.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 -- 172.21.15.87:0/746763879 learned_addr learned my addr 172.21.15.87:0/746763879 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3e00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80073e0 0x7f24f40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f8a00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24f4157840 0x7f24f4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 -- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24f4157840 msgr2=0x7f24f4157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24f4157840 0x7f24f4157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 -- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80073e0 msgr2=0x7f24f40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80073e0 0x7f24f40cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 -- 172.21.15.87:0/746763879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40cea70 con 0x7f24f40ce5f0 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3e00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80073e0 0x7f24f40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3400640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f40ce5f0 0x7f24f41f2c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24e40017f0 tx=0x7f24e40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24ebe00640 1 -- 172.21.15.87:0/746763879 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e401c070 con 0x7f24f40ce5f0 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24ebe00640 1 -- 172.21.15.87:0/746763879 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e4014750 con 0x7f24f40ce5f0 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f40cd920 con 0x7f24f40ce5f0 2024-04-25T10:13:17.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24ebe00640 1 -- 172.21.15.87:0/746763879 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4014b40 con 0x7f24f40ce5f0 2024-04-25T10:13:17.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f40cdee0 con 0x7f24f40ce5f0 2024-04-25T10:13:17.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f404ef30 con 0x7f24f40ce5f0 2024-04-25T10:13:17.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24ebe00640 1 -- 172.21.15.87:0/746763879 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24e401f070 con 0x7f24f40ce5f0 2024-04-25T10:13:17.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24ebe00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24ebe00640 1 -- 172.21.15.87:0/746763879 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24e4087a20 con 0x7f24f40ce5f0 2024-04-25T10:13:17.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.235+0000 7f24f3e00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.239+0000 7f24f3e00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24d0066f90 0x7f24d0069450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009cf0 tx=0x7f24dc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.243+0000 7f24ebe00640 1 -- 172.21.15.87:0/746763879 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24e4058990 con 0x7f24f40ce5f0 2024-04-25T10:13:17.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.395+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24f40b26a0 con 0x7f24f40ce5f0 2024-04-25T10:13:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24ebe00640 1 -- 172.21.15.87:0/746763879 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16670 (secure 0 0 0) 0x7f24e4058290 con 0x7f24f40ce5f0 2024-04-25T10:13:17.404 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:17.404 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:13:16.421443+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:14.236622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:15.422837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24d0066f90 msgr2=0x7f24d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24d0066f90 0x7f24d0069450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009cf0 tx=0x7f24dc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f40ce5f0 msgr2=0x7f24f41f2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f40ce5f0 0x7f24f41f2c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24e40017f0 tx=0x7f24e40128e0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 shutdown_connections 2024-04-25T10:13:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24f4157840 0x7f24f4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f40ce5f0 0x7f24f41f2c40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 --2- 172.21.15.87:0/746763879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80073e0 0x7f24f40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 >> 172.21.15.87:0/746763879 conn(0x7f24f4066b70 msgr2=0x7f24f415a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 shutdown_connections 2024-04-25T10:13:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.399+0000 7f24faa00640 1 -- 172.21.15.87:0/746763879 wait complete. 2024-04-25T10:13:17.425 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:17.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 Processor -- start 2024-04-25T10:13:17.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 -- start start 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2080157f40 0x7f20800aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20800af380 0x7f20800669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2080066ee0 0x7f208006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2080067350 con 0x7f2080157f40 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20800ad840 con 0x7f2080066ee0 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2085800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208006e7a0 con 0x7f20800af380 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2080157f40 0x7f20800aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2080157f40 0x7f20800aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44810/0 (socket says 172.21.15.87:44810) 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 -- 172.21.15.87:0/1336207264 learned_addr learned my addr 172.21.15.87:0/1336207264 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f2084e00640 1 --2- 172.21.15.87:0/1336207264 >> v2:172.21.15.169:3300/0 conn(0x7f2080066ee0 0x7f208006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207f400640 1 --2- 172.21.15.87:0/1336207264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20800af380 0x7f20800669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 -- 172.21.15.87:0/1336207264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20800af380 msgr2=0x7f20800669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 --2- 172.21.15.87:0/1336207264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20800af380 0x7f20800669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 -- 172.21.15.87:0/1336207264 >> v2:172.21.15.169:3300/0 conn(0x7f2080066ee0 msgr2=0x7f208006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 --2- 172.21.15.87:0/1336207264 >> v2:172.21.15.169:3300/0 conn(0x7f2080066ee0 0x7f208006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.555+0000 7f207fe00640 1 -- 172.21.15.87:0/1336207264 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20800af9b0 con 0x7f2080157f40 2024-04-25T10:13:17.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f207fe00640 1 --2- 172.21.15.87:0/1336207264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2080157f40 0x7f20800aee40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f20800aed00 tx=0x7f207001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3f106e79a72158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f207ea00640 1 -- 172.21.15.87:0/1336207264 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700b0070 con 0x7f2080157f40 2024-04-25T10:13:17.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f207ea00640 1 -- 172.21.15.87:0/1336207264 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20700ab600 con 0x7f2080157f40 2024-04-25T10:13:17.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f207ea00640 1 -- 172.21.15.87:0/1336207264 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700ab9d0 con 0x7f2080157f40 2024-04-25T10:13:17.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 -- 172.21.15.87:0/1336207264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2080157f40 msgr2=0x7f20800aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 --2- 172.21.15.87:0/1336207264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2080157f40 0x7f20800aee40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f20800aed00 tx=0x7f207001c260 comp rx=0 tx=0).stop 2024-04-25T10:13:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 -- 172.21.15.87:0/1336207264 shutdown_connections 2024-04-25T10:13:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 --2- 172.21.15.87:0/1336207264 >> v2:172.21.15.169:3300/0 conn(0x7f2080066ee0 0x7f208006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 --2- 172.21.15.87:0/1336207264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20800af380 0x7f20800669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 --2- 172.21.15.87:0/1336207264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2080157f40 0x7f20800aee40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 -- 172.21.15.87:0/1336207264 >> 172.21.15.87:0/1336207264 conn(0x7f2080074310 msgr2=0x7f208006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 -- 172.21.15.87:0/1336207264 shutdown_connections 2024-04-25T10:13:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 -- 172.21.15.87:0/1336207264 wait complete. 2024-04-25T10:13:17.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 Processor -- start 2024-04-25T10:13:17.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.559+0000 7f2085800640 1 -- start start 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2080066ee0 0x7f2080154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20800af380 0x7f20801549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2080157f40 0x7f2080163f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208015eee0 con 0x7f20800af380 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208015e9c0 con 0x7f2080066ee0 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2080158320 con 0x7f2080157f40 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20800af380 0x7f20801549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20800af380 0x7f20801549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44818/0 (socket says 172.21.15.87:44818) 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 -- 172.21.15.87:0/3986691603 learned_addr learned my addr 172.21.15.87:0/3986691603 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:17.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207fe00640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2080066ee0 0x7f2080154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2084e00640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2080157f40 0x7f2080163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 -- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2080157f40 msgr2=0x7f2080163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2080157f40 0x7f2080163f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 -- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2080066ee0 msgr2=0x7f2080154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2080066ee0 0x7f2080154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 -- 172.21.15.87:0/3986691603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2080165f40 con 0x7f20800af380 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207fe00640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2080066ee0 0x7f2080154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f207f400640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20800af380 0x7f20801549b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f206c00d800 tx=0x7f206c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2077e00640 1 -- 172.21.15.87:0/3986691603 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00b6c0 con 0x7f20800af380 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2077e00640 1 -- 172.21.15.87:0/3986691603 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c019070 con 0x7f20800af380 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2080154fe0 con 0x7f20800af380 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2077e00640 1 -- 172.21.15.87:0/3986691603 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00ba60 con 0x7f20800af380 2024-04-25T10:13:17.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20801555a0 con 0x7f20800af380 2024-04-25T10:13:17.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2080075120 con 0x7f20800af380 2024-04-25T10:13:17.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.563+0000 7f2077e00640 1 -- 172.21.15.87:0/3986691603 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f206c00bc00 con 0x7f20800af380 2024-04-25T10:13:17.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.567+0000 7f2077e00640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2060066f60 0x7f2060069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.567+0000 7f2077e00640 1 -- 172.21.15.87:0/3986691603 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f206c085580 con 0x7f20800af380 2024-04-25T10:13:17.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.567+0000 7f207fe00640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2060066f60 0x7f2060069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.567+0000 7f207fe00640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2060066f60 0x7f2060069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20700ad000 tx=0x7f20700a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.571+0000 7f2077e00640 1 -- 172.21.15.87:0/3986691603 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f206c056460 con 0x7f20800af380 2024-04-25T10:13:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.723+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20800b54e0 con 0x7f20800af380 2024-04-25T10:13:17.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2077e00640 1 -- 172.21.15.87:0/3986691603 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16670 (secure 0 0 0) 0x7f206c055d60 con 0x7f20800af380 2024-04-25T10:13:17.733 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:17.733 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:13:16.421443+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:14.236622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:15.422837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:17.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2060066f60 msgr2=0x7f2060069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2060066f60 0x7f2060069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20700ad000 tx=0x7f20700a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:17.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20800af380 msgr2=0x7f20801549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20800af380 0x7f20801549b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f206c00d800 tx=0x7f206c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 shutdown_connections 2024-04-25T10:13:17.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2060066f60 0x7f2060069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2080157f40 0x7f2080163f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20800af380 0x7f20801549b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 --2- 172.21.15.87:0/3986691603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2080066ee0 0x7f2080154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 >> 172.21.15.87:0/3986691603 conn(0x7f2080074310 msgr2=0x7f2080156d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 shutdown_connections 2024-04-25T10:13:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.727+0000 7f2085800640 1 -- 172.21.15.87:0/3986691603 wait complete. 2024-04-25T10:13:17.754 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:13:17.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc97bc00640 1 Processor -- start 2024-04-25T10:13:17.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc97bc00640 1 -- start start 2024-04-25T10:13:17.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc97bc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc974159540 0x7fc9740ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc97bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740721f0 0x7fc974070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc97bc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc974159920 con 0x7fc9740aebd0 2024-04-25T10:13:17.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc97bc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97415c2f0 con 0x7fc974159540 2024-04-25T10:13:17.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc97bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97415c5d0 con 0x7fc9740721f0 2024-04-25T10:13:17.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc979c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740721f0 0x7fc974070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc96be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc9740aebd0 0x7fc974071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:51100/0 (socket says 172.21.15.87:51100) 2024-04-25T10:13:17.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc96be00640 1 -- 172.21.15.87:0/2584236152 learned_addr learned my addr 172.21.15.87:0/2584236152 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:17.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc979200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc974159540 0x7fc9740ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc973e00640 1 -- 172.21.15.87:0/2584236152 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507624152 0 0) 0x7fc974159920 con 0x7fc9740aebd0 2024-04-25T10:13:17.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc973e00640 1 -- 172.21.15.87:0/2584236152 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc958003530 con 0x7fc9740aebd0 2024-04-25T10:13:17.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc979200640 1 -- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740721f0 msgr2=0x7fc974070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc979200640 1 --2- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740721f0 0x7fc974070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.887+0000 7fc979200640 1 -- 172.21.15.87:0/2584236152 >> v1:172.21.15.169:6789/0 conn(0x7fc9740aebd0 legacy=0x7fc974071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.891+0000 7fc979200640 1 -- 172.21.15.87:0/2584236152 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97415c8b0 con 0x7fc974159540 2024-04-25T10:13:17.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.891+0000 7fc979c00640 1 --2- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740721f0 0x7fc974070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:17.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.891+0000 7fc979200640 1 --2- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9740ae690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097c0 tx=0x7fc96401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0498c49980ff25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.891+0000 7fc973e00640 1 -- 172.21.15.87:0/2584236152 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640b1070 con 0x7fc974159540 2024-04-25T10:13:17.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.891+0000 7fc973e00640 1 -- 172.21.15.87:0/2584236152 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9640ac410 con 0x7fc974159540 2024-04-25T10:13:17.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.891+0000 7fc973e00640 1 -- 172.21.15.87:0/2584236152 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640ac7c0 con 0x7fc974159540 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 msgr2=0x7fc9740ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9740ae690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097c0 tx=0x7fc96401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- 172.21.15.87:0/2584236152 shutdown_connections 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740721f0 0x7fc974070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2584236152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9740ae690 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- 172.21.15.87:0/2584236152 >> 172.21.15.87:0/2584236152 conn(0x7fc9740669a0 msgr2=0x7fc974066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- 172.21.15.87:0/2584236152 shutdown_connections 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- 172.21.15.87:0/2584236152 wait complete. 2024-04-25T10:13:17.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 Processor -- start 2024-04-25T10:13:17.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- start start 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9740721f0 0x7fc9740b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740aebd0 0x7fc9740b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9741543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc974071220 con 0x7fc9740721f0 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740710a0 con 0x7fc974159540 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc97bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740713a0 con 0x7fc9740aebd0 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9740721f0 0x7fc9740b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9740721f0 0x7fc9740b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44822/0 (socket says 172.21.15.87:44822) 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 -- 172.21.15.87:0/2036575400 learned_addr learned my addr 172.21.15.87:0/2036575400 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979c00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9741543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc96be00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740aebd0 0x7fc9740b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 -- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740aebd0 msgr2=0x7fc9740b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740aebd0 0x7fc9740b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 -- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 msgr2=0x7fc9741543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9741543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 -- 172.21.15.87:0/2036575400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740b0ea0 con 0x7fc9740721f0 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979c00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9741543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc96be00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740aebd0 0x7fc9740b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc979200640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9740721f0 0x7fc9740b1f40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c10 tx=0x7fc964019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc971600640 1 -- 172.21.15.87:0/2036575400 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640bb070 con 0x7fc9740721f0 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.895+0000 7fc971600640 1 -- 172.21.15.87:0/2036575400 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9640036a0 con 0x7fc9740721f0 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9740b1180 con 0x7fc9740721f0 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc971600640 1 -- 172.21.15.87:0/2036575400 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640b1040 con 0x7fc9740721f0 2024-04-25T10:13:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9740b1740 con 0x7fc9740721f0 2024-04-25T10:13:17.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9740b1c30 con 0x7fc9740721f0 2024-04-25T10:13:17.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc971600640 1 -- 172.21.15.87:0/2036575400 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9640be070 con 0x7fc9740721f0 2024-04-25T10:13:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc971600640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc950066f60 0x7fc950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc971600640 1 -- 172.21.15.87:0/2036575400 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc96411dfb0 con 0x7fc9740721f0 2024-04-25T10:13:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc96be00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc950066f60 0x7fc950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.899+0000 7fc96be00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc950066f60 0x7fc950069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc974067170 tx=0x7fc95c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:17.903+0000 7fc971600640 1 -- 172.21.15.87:0/2036575400 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc964016090 con 0x7fc9740721f0 2024-04-25T10:13:18.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:18.095+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc97404ef30 con 0x7fc9740721f0 2024-04-25T10:13:19.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.059+0000 7fc971600640 1 -- 172.21.15.87:0/2036575400 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc9640ef050 con 0x7fc9740721f0 2024-04-25T10:13:19.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc950066f60 msgr2=0x7fc950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc950066f60 0x7fc950069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc974067170 tx=0x7fc95c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9740721f0 msgr2=0x7fc9740b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9740721f0 0x7fc9740b1f40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c10 tx=0x7fc964019240 comp rx=0 tx=0).stop 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 shutdown_connections 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc950066f60 0x7fc950069420 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc974159540 0x7fc9741543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9740aebd0 0x7fc9740b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 --2- 172.21.15.87:0/2036575400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9740721f0 0x7fc9740b1f40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 >> 172.21.15.87:0/2036575400 conn(0x7fc9740669a0 msgr2=0x7fc9740ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 shutdown_connections 2024-04-25T10:13:19.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.063+0000 7fc97bc00640 1 -- 172.21.15.87:0/2036575400 wait complete. 2024-04-25T10:13:19.089 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:13:19.089 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01c400640 1 Processor -- start 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01c400640 1 -- start start 2024-04-25T10:13:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0140aeda0 0x7fb014071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb014072020 0x7fb014070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb014159920 con 0x7fb014159540 2024-04-25T10:13:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01415c2f0 con 0x7fb0140aeda0 2024-04-25T10:13:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01415c5d0 con 0x7fb014072020 2024-04-25T10:13:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb014072020 0x7fb014070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb019a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb014159540 0x7fb0140ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:51110/0 (socket says 172.21.15.87:51110) 2024-04-25T10:13:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb019000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0140aeda0 0x7fb014071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb019a00640 1 -- 172.21.15.87:0/45831540 learned_addr learned my addr 172.21.15.87:0/45831540 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb013e00640 1 -- 172.21.15.87:0/45831540 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359940023 0 0) 0x7fb014159920 con 0x7fb014159540 2024-04-25T10:13:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb013e00640 1 -- 172.21.15.87:0/45831540 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faffc003530 con 0x7fb014159540 2024-04-25T10:13:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01a400640 1 -- 172.21.15.87:0/45831540 >> v2:172.21.15.169:3300/0 conn(0x7fb0140aeda0 msgr2=0x7fb014071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01a400640 1 --2- 172.21.15.87:0/45831540 >> v2:172.21.15.169:3300/0 conn(0x7fb0140aeda0 0x7fb014071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01a400640 1 -- 172.21.15.87:0/45831540 >> v1:172.21.15.169:6789/0 conn(0x7fb014159540 legacy=0x7fb0140ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb01a400640 1 -- 172.21.15.87:0/45831540 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01415c8b0 con 0x7fb014072020 2024-04-25T10:13:19.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.223+0000 7fb019000640 1 --2- 172.21.15.87:0/45831540 >> v2:172.21.15.169:3300/0 conn(0x7fb0140aeda0 0x7fb014071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:19.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01a400640 1 --2- 172.21.15.87:0/45831540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb014072020 0x7fb014070350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009d90 tx=0x7fb00c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4557cedf7a3f78d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb013e00640 1 -- 172.21.15.87:0/45831540 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0b2070 con 0x7fb014072020 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb013e00640 1 -- 172.21.15.87:0/45831540 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c0a59e0 con 0x7fb014072020 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb013e00640 1 -- 172.21.15.87:0/45831540 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0a5db0 con 0x7fb014072020 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 -- 172.21.15.87:0/45831540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb014072020 msgr2=0x7fb014070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 --2- 172.21.15.87:0/45831540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb014072020 0x7fb014070350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009d90 tx=0x7fb00c01c940 comp rx=0 tx=0).stop 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 -- 172.21.15.87:0/45831540 shutdown_connections 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 --2- 172.21.15.87:0/45831540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb014072020 0x7fb014070350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 --2- 172.21.15.87:0/45831540 >> v2:172.21.15.169:3300/0 conn(0x7fb0140aeda0 0x7fb014071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 -- 172.21.15.87:0/45831540 >> 172.21.15.87:0/45831540 conn(0x7fb0140669a0 msgr2=0x7fb014066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 -- 172.21.15.87:0/45831540 shutdown_connections 2024-04-25T10:13:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.227+0000 7fb01c400640 1 -- 172.21.15.87:0/45831540 wait complete. 2024-04-25T10:13:19.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 Processor -- start 2024-04-25T10:13:19.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 -- start start 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014072020 0x7fb0140b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0140aeda0 0x7fb01415a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb014159540 0x7fb01414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140710a0 con 0x7fb014072020 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01406c0d0 con 0x7fb014159540 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb014071220 con 0x7fb0140aeda0 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014072020 0x7fb0140b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014072020 0x7fb0140b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44838/0 (socket says 172.21.15.87:44838) 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 -- 172.21.15.87:0/31702487 learned_addr learned my addr 172.21.15.87:0/31702487 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019000640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0140aeda0 0x7fb01415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01a400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb014159540 0x7fb01414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 -- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0140aeda0 msgr2=0x7fb01415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0140aeda0 0x7fb01415a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 -- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb014159540 msgr2=0x7fb01414d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb014159540 0x7fb01414d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 -- 172.21.15.87:0/31702487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140b0c90 con 0x7fb014072020 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01a400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb014159540 0x7fb01414d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb019a00640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014072020 0x7fb0140b0750 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c480 tx=0x7fb00800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb011600640 1 -- 172.21.15.87:0/31702487 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008017070 con 0x7fb014072020 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb011600640 1 -- 172.21.15.87:0/31702487 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb008005cf0 con 0x7fb014072020 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb011600640 1 -- 172.21.15.87:0/31702487 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008007290 con 0x7fb014072020 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0140b0f30 con 0x7fb014072020 2024-04-25T10:13:19.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.231+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0140b14f0 con 0x7fb014072020 2024-04-25T10:13:19.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.235+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01404ef30 con 0x7fb014072020 2024-04-25T10:13:19.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.235+0000 7fb011600640 1 -- 172.21.15.87:0/31702487 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb00801a070 con 0x7fb014072020 2024-04-25T10:13:19.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.235+0000 7fb011600640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faff4066f60 0x7faff4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.235+0000 7fb011600640 1 -- 172.21.15.87:0/31702487 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb008084360 con 0x7fb014072020 2024-04-25T10:13:19.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.235+0000 7fb019000640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faff4066f60 0x7faff4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.235+0000 7fb019000640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faff4066f60 0x7faff4069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb000009580 tx=0x7fb000009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.239+0000 7fb011600640 1 -- 172.21.15.87:0/31702487 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb008054d10 con 0x7fb014072020 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:19.427+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb0140cf470 con 0x7fb014072020 2024-04-25T10:13:20.007 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:13:19.999+0000 7f5dafa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb011600640 1 -- 172.21.15.87:0/31702487 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb008058db0 con 0x7fb014072020 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faff4066f60 msgr2=0x7faff4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faff4066f60 0x7faff4069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb000009580 tx=0x7fb000009190 comp rx=0 tx=0).stop 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014072020 msgr2=0x7fb0140b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014072020 0x7fb0140b0750 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c480 tx=0x7fb00800c940 comp rx=0 tx=0).stop 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 shutdown_connections 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faff4066f60 0x7faff4069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb014159540 0x7fb01414d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0140aeda0 0x7fb01415a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 --2- 172.21.15.87:0/31702487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014072020 0x7fb0140b0750 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 >> 172.21.15.87:0/31702487 conn(0x7fb0140669a0 msgr2=0x7fb014159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 shutdown_connections 2024-04-25T10:13:20.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.067+0000 7fb01c400640 1 -- 172.21.15.87:0/31702487 wait complete. 2024-04-25T10:13:20.097 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:13:20.097 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:20.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7c600640 1 Processor -- start 2024-04-25T10:13:20.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7c600640 1 -- start start 2024-04-25T10:13:20.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 0x7fdd7406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdd740cf930 0x7fdd740cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7c600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd740b2770 con 0x7fdd740cda10 2024-04-25T10:13:20.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7406c670 con 0x7fdd740cf930 2024-04-25T10:13:20.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7406c950 con 0x7fdd7415bc40 2024-04-25T10:13:20.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd79200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fdd740cda10 0x7fdd740b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56688/0 (socket says 172.21.15.87:56688) 2024-04-25T10:13:20.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd79200640 1 -- 172.21.15.87:0/3990257372 learned_addr learned my addr 172.21.15.87:0/3990257372 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:20.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd79c00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 0x7fdd7406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7a600640 1 --2- 172.21.15.87:0/3990257372 >> v2:172.21.15.169:3300/0 conn(0x7fdd740cf930 0x7fdd740cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113453426 0 0) 0x7fdd740b2770 con 0x7fdd740cda10 2024-04-25T10:13:20.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd5c003530 con 0x7fdd740cda10 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2164094796 0 0) 0x7fdd5c003530 con 0x7fdd740cda10 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 msgr2=0x7fdd7406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 0x7fdd7406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 >> v2:172.21.15.169:3300/0 conn(0x7fdd740cf930 msgr2=0x7fdd740cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 --2- 172.21.15.87:0/3990257372 >> v2:172.21.15.169:3300/0 conn(0x7fdd740cf930 0x7fdd740cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7406cc30 con 0x7fdd740cda10 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fdd60001680 con 0x7fdd740cda10 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 >> v1:172.21.15.87:6789/0 conn(0x7fdd740cda10 legacy=0x7fdd740b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0039c0 0x7fdd5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 0x7fdd5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c004d50 0x7fdd5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7406c670 con 0x7fdd5c0039c0 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7406c950 con 0x7fdd5c004d50 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c00b9d0 con 0x7fdd7415bc40 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd79c00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0039c0 0x7fdd5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd79200640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 0x7fdd5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.227+0000 7fdd7a600640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c004d50 0x7fdd5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd79c00640 1 -- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 msgr2=0x7fdd5c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd79c00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7415bc40 0x7fdd5c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd79c00640 1 -- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c004d50 msgr2=0x7fdd5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd79c00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c004d50 0x7fdd5c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd79c00640 1 -- 172.21.15.87:0/3990257372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c003530 con 0x7fdd5c0039c0 2024-04-25T10:13:20.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd79c00640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0039c0 0x7fdd5c003de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009ae0 tx=0x7fdd6801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93390918be26cec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680096b0 con 0x7fdd5c0039c0 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd680a6450 con 0x7fdd5c0039c0 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd73e00640 1 -- 172.21.15.87:0/3990257372 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680a6840 con 0x7fdd5c0039c0 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd7c600640 1 -- 172.21.15.87:0/3990257372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0039c0 msgr2=0x7fdd5c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd7c600640 1 --2- 172.21.15.87:0/3990257372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0039c0 0x7fdd5c003de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009ae0 tx=0x7fdd6801ca70 comp rx=0 tx=0).stop 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd7a600640 1 -- 172.21.15.87:0/3990257372 reap_dead start 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd7c600640 1 -- 172.21.15.87:0/3990257372 shutdown_connections 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd7c600640 1 -- 172.21.15.87:0/3990257372 >> 172.21.15.87:0/3990257372 conn(0x7fdd74066b70 msgr2=0x7fdd74066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd7c600640 1 -- 172.21.15.87:0/3990257372 shutdown_connections 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.231+0000 7fdd7c600640 1 -- 172.21.15.87:0/3990257372 wait complete. 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 Processor -- start 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 -- start start 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c0039c0 0x7fdd74144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd741fa560 0x7fdd741fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd74157720 0x7fdd74157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740b3810 con 0x7fdd74157720 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740b3690 con 0x7fdd741fa560 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740b3990 con 0x7fdd5c0039c0 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd74157720 0x7fdd74157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd74157720 0x7fdd74157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44846/0 (socket says 172.21.15.87:44846) 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 -- 172.21.15.87:0/1682645410 learned_addr learned my addr 172.21.15.87:0/1682645410 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd79c00640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c0039c0 0x7fdd74144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd79200640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd741fa560 0x7fdd741fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 -- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c0039c0 msgr2=0x7fdd74144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c0039c0 0x7fdd74144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 -- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd741fa560 msgr2=0x7fdd741fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd741fa560 0x7fdd741fa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 -- 172.21.15.87:0/1682645410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd741e79e0 con 0x7fdd74157720 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7a600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd74157720 0x7fdd74157ba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdd740b44f0 tx=0x7fdd6c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd71600640 1 -- 172.21.15.87:0/1682645410 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c011e10 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd71600640 1 -- 172.21.15.87:0/1682645410 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c01b070 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd74158300 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd71600640 1 -- 172.21.15.87:0/1682645410 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0039a0 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.235+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd741451d0 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.239+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7404ef30 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.239+0000 7fdd71600640 1 -- 172.21.15.87:0/1682645410 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd6c003b60 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.239+0000 7fdd71600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd54066f60 0x7fdd54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.239+0000 7fdd71600640 1 -- 172.21.15.87:0/1682645410 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd6c085580 con 0x7fdd74157720 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.239+0000 7fdd79c00640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd54066f60 0x7fdd54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.239+0000 7fdd79c00640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd54066f60 0x7fdd54069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdd680ad000 tx=0x7fdd680b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.243+0000 7fdd71600640 1 -- 172.21.15.87:0/1682645410 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd741e79e0 con 0x7fdd74157720 2024-04-25T10:13:20.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.427+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd740af1c0 con 0x7fdd74157720 2024-04-25T10:13:20.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.427+0000 7fdd71600640 1 -- 172.21.15.87:0/1682645410 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fdd6c055f40 con 0x7fdd74157720 2024-04-25T10:13:20.435 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:20.435 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3059476782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3059476782},{"type":"v1","addr":"172.21.15.169:6834","nonce":3059476782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/685777206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":685777206},{"type":"v1","addr":"172.21.15.87:6837","nonce":685777206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2281421407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2281421407},{"type":"v1","addr":"172.21.15.169:6835","nonce":2281421407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:16.421834+0000","modified":"2024-04-25T10:13:19.431751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/4156145333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4156145333},{"type":"v1","addr":"172.21.15.87:6835","nonce":4156145333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:13:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 16 2024-04-25T10:13:20.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd54066f60 msgr2=0x7fdd54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd54066f60 0x7fdd54069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdd680ad000 tx=0x7fdd680b3040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd74157720 msgr2=0x7fdd74157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd74157720 0x7fdd74157ba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdd740b44f0 tx=0x7fdd6c0115a0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 shutdown_connections 2024-04-25T10:13:20.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd54066f60 0x7fdd54069420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd74157720 0x7fdd74157ba0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd741fa560 0x7fdd741fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 --2- 172.21.15.87:0/1682645410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c0039c0 0x7fdd74144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 >> 172.21.15.87:0/1682645410 conn(0x7fdd74066b70 msgr2=0x7fdd740ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 shutdown_connections 2024-04-25T10:13:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.431+0000 7fdd7c600640 1 -- 172.21.15.87:0/1682645410 wait complete. 2024-04-25T10:13:20.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:20.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa205000640 1 Processor -- start 2024-04-25T10:13:20.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa205000640 1 -- start start 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa205000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa20015bc40 0x7fa20006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa205000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000cf930 0x7fa2000cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa205000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000b2770 con 0x7fa2000cda10 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa205000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20006c670 con 0x7fa20015bc40 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa205000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20006c950 con 0x7fa2000cf930 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa1fde00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa20015bc40 0x7fa20006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa1fde00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa20015bc40 0x7fa20006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44854/0 (socket says 172.21.15.87:44854) 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa1fde00640 1 -- 172.21.15.87:0/3716500534 learned_addr learned my addr 172.21.15.87:0/3716500534 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa1f5400640 1 --1- 172.21.15.87:0/3716500534 >> v1:172.21.15.87:6789/0 conn(0x7fa2000cda10 0x7fa2000b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56694/0 (socket says 172.21.15.87:56694) 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.586+0000 7fa1fe800640 1 --2- 172.21.15.87:0/3716500534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000cf930 0x7fa2000cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fde00640 1 -- 172.21.15.87:0/3716500534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000cf930 msgr2=0x7fa2000cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fde00640 1 --2- 172.21.15.87:0/3716500534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000cf930 0x7fa2000cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fde00640 1 -- 172.21.15.87:0/3716500534 >> v1:172.21.15.87:6789/0 conn(0x7fa2000cda10 legacy=0x7fa2000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fde00640 1 -- 172.21.15.87:0/3716500534 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20006cc30 con 0x7fa20015bc40 2024-04-25T10:13:20.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fd400640 1 -- 172.21.15.87:0/3716500534 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961041768 0 0) 0x7fa2000b2770 con 0x7fa2000cda10 2024-04-25T10:13:20.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fde00640 1 --2- 172.21.15.87:0/3716500534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa20015bc40 0x7fa20006c090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009e50 tx=0x7fa1f001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468809f5806cd069 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fd400640 1 -- 172.21.15.87:0/3716500534 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00a5660 con 0x7fa20015bc40 2024-04-25T10:13:20.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fd400640 1 -- 172.21.15.87:0/3716500534 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f00a5800 con 0x7fa20015bc40 2024-04-25T10:13:20.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa1fd400640 1 -- 172.21.15.87:0/3716500534 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00a5bd0 con 0x7fa20015bc40 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa205000640 1 -- 172.21.15.87:0/3716500534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa20015bc40 msgr2=0x7fa20006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa205000640 1 --2- 172.21.15.87:0/3716500534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa20015bc40 0x7fa20006c090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009e50 tx=0x7fa1f001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa205000640 1 -- 172.21.15.87:0/3716500534 shutdown_connections 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa205000640 1 --2- 172.21.15.87:0/3716500534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000cf930 0x7fa2000cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa205000640 1 --2- 172.21.15.87:0/3716500534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa20015bc40 0x7fa20006c090 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.590+0000 7fa205000640 1 -- 172.21.15.87:0/3716500534 >> 172.21.15.87:0/3716500534 conn(0x7fa200066b70 msgr2=0x7fa200066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 -- 172.21.15.87:0/3716500534 shutdown_connections 2024-04-25T10:13:20.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 -- 172.21.15.87:0/3716500534 wait complete. 2024-04-25T10:13:20.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 Processor -- start 2024-04-25T10:13:20.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 -- start start 2024-04-25T10:13:20.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2000cda10 0x7fa2001faea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2000cf930 0x7fa20014a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa20015bc40 0x7fa200155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000b3890 con 0x7fa2000cda10 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000b3710 con 0x7fa2000cf930 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000b3a10 con 0x7fa20015bc40 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2000cda10 0x7fa2001faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2000cda10 0x7fa2001faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44860/0 (socket says 172.21.15.87:44860) 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 -- 172.21.15.87:0/4195432371 learned_addr learned my addr 172.21.15.87:0/4195432371 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fe800640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa20015bc40 0x7fa200155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1f5400640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2000cf930 0x7fa20014a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 -- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa20015bc40 msgr2=0x7fa200155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa20015bc40 0x7fa200155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 -- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2000cf930 msgr2=0x7fa20014a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2000cf930 0x7fa20014a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 -- 172.21.15.87:0/4195432371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000cfdb0 con 0x7fa2000cda10 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fe800640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa20015bc40 0x7fa200155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1f5400640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2000cf930 0x7fa20014a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1fde00640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2000cda10 0x7fa2001faea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00af040 tx=0x7fa1f0019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1f6a00640 1 -- 172.21.15.87:0/4195432371 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0009510 con 0x7fa2000cda10 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1f6a00640 1 -- 172.21.15.87:0/4195432371 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0003ba0 con 0x7fa2000cda10 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa1f6a00640 1 -- 172.21.15.87:0/4195432371 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00b8680 con 0x7fa2000cda10 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.594+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20015e4c0 con 0x7fa2000cda10 2024-04-25T10:13:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.598+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20015ea80 con 0x7fa2000cda10 2024-04-25T10:13:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.598+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20015d690 con 0x7fa2000cda10 2024-04-25T10:13:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.598+0000 7fa1f6a00640 1 -- 172.21.15.87:0/4195432371 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1f00b8820 con 0x7fa2000cda10 2024-04-25T10:13:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.598+0000 7fa1f6a00640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa1dc066f60 0x7fa1dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.598+0000 7fa1f6a00640 1 -- 172.21.15.87:0/4195432371 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1f0120640 con 0x7fa2000cda10 2024-04-25T10:13:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.598+0000 7fa1f5400640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa1dc066f60 0x7fa1dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.598+0000 7fa1f5400640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa1dc066f60 0x7fa1dc069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8007fd0 tx=0x7fa1e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.602+0000 7fa1f6a00640 1 -- 172.21.15.87:0/4195432371 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa20015d690 con 0x7fa2000cda10 2024-04-25T10:13:20.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.758+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa20004ef30 con 0x7fa2000cda10 2024-04-25T10:13:20.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.758+0000 7fa1f6a00640 1 -- 172.21.15.87:0/4195432371 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16657 (secure 0 0 0) 0x7fa1f0016620 con 0x7fa2000cda10 2024-04-25T10:13:20.767 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:20.768 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":33,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:13:18.051652+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:14.236622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:15.422837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:20.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa1dc066f60 msgr2=0x7fa1dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa1dc066f60 0x7fa1dc069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8007fd0 tx=0x7fa1e8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2000cda10 msgr2=0x7fa2001faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2000cda10 0x7fa2001faea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00af040 tx=0x7fa1f0019740 comp rx=0 tx=0).stop 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 shutdown_connections 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa1dc066f60 0x7fa1dc069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa20015bc40 0x7fa200155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2000cf930 0x7fa20014a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 --2- 172.21.15.87:0/4195432371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2000cda10 0x7fa2001faea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 >> 172.21.15.87:0/4195432371 conn(0x7fa200066b70 msgr2=0x7fa200159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 shutdown_connections 2024-04-25T10:13:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.762+0000 7fa205000640 1 -- 172.21.15.87:0/4195432371 wait complete. 2024-04-25T10:13:20.789 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:13:20.789 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.914+0000 7f53e6800640 1 Processor -- start 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.914+0000 7f53e6800640 1 -- start start 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53e6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e01550b0 0x7f53e00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53e6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f53e00cfe70 0x7f53e015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53e6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53e015b740 0x7f53e006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53e6800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e0071050 con 0x7f53e015b740 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53e6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00712f0 con 0x7f53e00cfe70 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53e6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00715d0 con 0x7f53e01550b0 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53e015b740 0x7f53e006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53e015b740 0x7f53e006d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44876/0 (socket says 172.21.15.87:44876) 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 -- 172.21.15.87:0/1092700992 learned_addr learned my addr 172.21.15.87:0/1092700992 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53df400640 1 --2- 172.21.15.87:0/1092700992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e01550b0 0x7f53e00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53d6a00640 1 --2- 172.21.15.87:0/1092700992 >> v2:172.21.15.169:3300/0 conn(0x7f53e00cfe70 0x7f53e015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 -- 172.21.15.87:0/1092700992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e01550b0 msgr2=0x7f53e00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 --2- 172.21.15.87:0/1092700992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e01550b0 0x7f53e00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 -- 172.21.15.87:0/1092700992 >> v2:172.21.15.169:3300/0 conn(0x7f53e00cfe70 msgr2=0x7f53e015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 --2- 172.21.15.87:0/1092700992 >> v2:172.21.15.169:3300/0 conn(0x7f53e00cfe70 0x7f53e015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53dfe00640 1 -- 172.21.15.87:0/1092700992 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00718b0 con 0x7f53e015b740 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.918+0000 7f53df400640 1 --2- 172.21.15.87:0/1092700992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e01550b0 0x7f53e00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53dfe00640 1 --2- 172.21.15.87:0/1092700992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e015b740 0x7f53e006d1a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f53e006e290 tx=0x7f53d801e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b9d0abc980b8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53dea00640 1 -- 172.21.15.87:0/1092700992 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d800b2d0 con 0x7f53e015b740 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53dea00640 1 -- 172.21.15.87:0/1092700992 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d800b470 con 0x7f53e015b740 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53dea00640 1 -- 172.21.15.87:0/1092700992 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d80a7860 con 0x7f53e015b740 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- 172.21.15.87:0/1092700992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e015b740 msgr2=0x7f53e006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 --2- 172.21.15.87:0/1092700992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e015b740 0x7f53e006d1a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f53e006e290 tx=0x7f53d801e930 comp rx=0 tx=0).stop 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- 172.21.15.87:0/1092700992 shutdown_connections 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 --2- 172.21.15.87:0/1092700992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e015b740 0x7f53e006d1a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 --2- 172.21.15.87:0/1092700992 >> v2:172.21.15.169:3300/0 conn(0x7f53e00cfe70 0x7f53e015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 --2- 172.21.15.87:0/1092700992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e01550b0 0x7f53e00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- 172.21.15.87:0/1092700992 >> 172.21.15.87:0/1092700992 conn(0x7f53e0066b70 msgr2=0x7f53e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- 172.21.15.87:0/1092700992 shutdown_connections 2024-04-25T10:13:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- 172.21.15.87:0/1092700992 wait complete. 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 Processor -- start 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- start start 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e00cfe70 0x7f53e0147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e01550b0 0x7f53e00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e015b740 0x7f53e00cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b33e0 con 0x7f53e01550b0 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b3260 con 0x7f53e015b740 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.922+0000 7f53e6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b3560 con 0x7f53e00cfe70 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e01550b0 0x7f53e00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e01550b0 0x7f53e00cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44890/0 (socket says 172.21.15.87:44890) 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 -- 172.21.15.87:0/3889809891 learned_addr learned my addr 172.21.15.87:0/3889809891 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53df400640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e00cfe70 0x7f53e0147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53dfe00640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e015b740 0x7f53e00cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 -- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e00cfe70 msgr2=0x7f53e0147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e00cfe70 0x7f53e0147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 -- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e015b740 msgr2=0x7f53e00cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e015b740 0x7f53e00cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 -- 172.21.15.87:0/3889809891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e0150180 con 0x7f53e01550b0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d6a00640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e01550b0 0x7f53e00cd840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f53c800d800 tx=0x7f53c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d7e00640 1 -- 172.21.15.87:0/3889809891 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800b6c0 con 0x7f53e01550b0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d7e00640 1 -- 172.21.15.87:0/3889809891 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8019070 con 0x7f53e01550b0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d7e00640 1 -- 172.21.15.87:0/3889809891 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800ba60 con 0x7f53e01550b0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e014c510 con 0x7f53e01550b0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e014cad0 con 0x7f53e01550b0 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e004ef30 con 0x7f53e01550b0 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d7e00640 1 -- 172.21.15.87:0/3889809891 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53c800bc00 con 0x7f53e01550b0 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d7e00640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f53bc066f60 0x7f53bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.926+0000 7f53d7e00640 1 -- 172.21.15.87:0/3889809891 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53c8085570 con 0x7f53e01550b0 2024-04-25T10:13:20.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.930+0000 7f53df400640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f53bc066f60 0x7f53bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.930+0000 7f53df400640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f53bc066f60 0x7f53bc069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009830 tx=0x7f53d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:20.934+0000 7f53d7e00640 1 -- 172.21.15.87:0/3889809891 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53c80564e0 con 0x7f53e01550b0 2024-04-25T10:13:21.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.086+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53e00cc430 con 0x7f53bc066f60 2024-04-25T10:13:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.090+0000 7f53d7e00640 1 -- 172.21.15.87:0/3889809891 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246632 (secure 0 0 0) 0x7f53e00cc430 con 0x7f53bc066f60 2024-04-25T10:13:21.102 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:21.104 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:13:21.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f53bc066f60 msgr2=0x7f53bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f53bc066f60 0x7f53bc069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009830 tx=0x7f53d0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:21.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e01550b0 msgr2=0x7f53e00cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e01550b0 0x7f53e00cd840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f53c800d800 tx=0x7f53c800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 shutdown_connections 2024-04-25T10:13:21.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f53bc066f60 0x7f53bc069420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e015b740 0x7f53e00cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53e01550b0 0x7f53e00cd840 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 --2- 172.21.15.87:0/3889809891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53e00cfe70 0x7f53e0147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 >> 172.21.15.87:0/3889809891 conn(0x7f53e0066b70 msgr2=0x7f53e006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 shutdown_connections 2024-04-25T10:13:21.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.094+0000 7f53e6800640 1 -- 172.21.15.87:0/3889809891 wait complete. 2024-04-25T10:13:21.122 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-04-25T10:13:19.135649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219960,"kb_used_data":3796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754760,"statfs":{"total":773094113280,"available":772868874240,"internally_reserved":0,"allocated":3887104,"data_stored":2205195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.106044"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051524+0000","last_change":"2024-04-25T10:13:16.051524+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051524+0000","last_undegraded":"2024-04-25T10:13:16.051524+0000","last_fullsized":"2024-04-25T10:13:16.051524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049178+0000","last_change":"2024-04-25T10:13:16.049178+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049178+0000","last_undegraded":"2024-04-25T10:13:16.049178+0000","last_fullsized":"2024-04-25T10:13:16.049178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042138+0000","last_change":"2024-04-25T10:13:16.042138+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042138+0000","last_undegraded":"2024-04-25T10:13:16.042138+0000","last_fullsized":"2024-04-25T10:13:16.042138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043287+0000","last_change":"2024-04-25T10:13:16.043287+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043287+0000","last_undegraded":"2024-04-25T10:13:16.043287+0000","last_fullsized":"2024-04-25T10:13:16.043287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051209+0000","last_change":"2024-04-25T10:13:16.051209+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051209+0000","last_undegraded":"2024-04-25T10:13:16.051209+0000","last_fullsized":"2024-04-25T10:13:16.051209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048910+0000","last_change":"2024-04-25T10:13:16.048910+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048910+0000","last_undegraded":"2024-04-25T10:13:16.048910+0000","last_fullsized":"2024-04-25T10:13:16.048910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050714+0000","last_change":"2024-04-25T10:13:16.050714+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050714+0000","last_undegraded":"2024-04-25T10:13:16.050714+0000","last_fullsized":"2024-04-25T10:13:16.050714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053901+0000","last_change":"2024-04-25T10:13:16.053901+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053901+0000","last_undegraded":"2024-04-25T10:13:16.053901+0000","last_fullsized":"2024-04-25T10:13:16.053901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044137+0000","last_change":"2024-04-25T10:13:16.044137+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044137+0000","last_undegraded":"2024-04-25T10:13:16.044137+0000","last_fullsized":"2024-04-25T10:13:16.044137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043363+0000","last_change":"2024-04-25T10:13:16.043363+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043363+0000","last_undegraded":"2024-04-25T10:13:16.043363+0000","last_fullsized":"2024-04-25T10:13:16.043363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.054965+0000","last_change":"2024-04-25T10:13:16.054965+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.054965+0000","last_undegraded":"2024-04-25T10:13:16.054965+0000","last_fullsized":"2024-04-25T10:13:16.054965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043819+0000","last_change":"2024-04-25T10:13:16.043819+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043819+0000","last_undegraded":"2024-04-25T10:13:16.043819+0000","last_fullsized":"2024-04-25T10:13:16.043819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046803+0000","last_change":"2024-04-25T10:13:16.046803+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046803+0000","last_undegraded":"2024-04-25T10:13:16.046803+0000","last_fullsized":"2024-04-25T10:13:16.046803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048465+0000","last_change":"2024-04-25T10:13:16.048465+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048465+0000","last_undegraded":"2024-04-25T10:13:16.048465+0000","last_fullsized":"2024-04-25T10:13:16.048465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041761+0000","last_change":"2024-04-25T10:13:16.041761+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041761+0000","last_undegraded":"2024-04-25T10:13:16.041761+0000","last_fullsized":"2024-04-25T10:13:16.041761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051289+0000","last_change":"2024-04-25T10:13:16.051289+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051289+0000","last_undegraded":"2024-04-25T10:13:16.051289+0000","last_fullsized":"2024-04-25T10:13:16.051289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.057085+0000","last_change":"2024-04-25T10:13:16.057085+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.057085+0000","last_undegraded":"2024-04-25T10:13:16.057085+0000","last_fullsized":"2024-04-25T10:13:16.057085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048752+0000","last_change":"2024-04-25T10:13:16.048752+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048752+0000","last_undegraded":"2024-04-25T10:13:16.048752+0000","last_fullsized":"2024-04-25T10:13:16.048752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050699+0000","last_change":"2024-04-25T10:13:16.050699+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050699+0000","last_undegraded":"2024-04-25T10:13:16.050699+0000","last_fullsized":"2024-04-25T10:13:16.050699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050639+0000","last_change":"2024-04-25T10:13:16.050639+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050639+0000","last_undegraded":"2024-04-25T10:13:16.050639+0000","last_fullsized":"2024-04-25T10:13:16.050639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043315+0000","last_change":"2024-04-25T10:13:16.043315+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043315+0000","last_undegraded":"2024-04-25T10:13:16.043315+0000","last_fullsized":"2024-04-25T10:13:16.043315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042542+0000","last_change":"2024-04-25T10:13:16.042542+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042542+0000","last_undegraded":"2024-04-25T10:13:16.042542+0000","last_fullsized":"2024-04-25T10:13:16.042542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047344+0000","last_change":"2024-04-25T10:13:16.047344+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047344+0000","last_undegraded":"2024-04-25T10:13:16.047344+0000","last_fullsized":"2024-04-25T10:13:16.047344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041284+0000","last_change":"2024-04-25T10:13:16.041284+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041284+0000","last_undegraded":"2024-04-25T10:13:16.041284+0000","last_fullsized":"2024-04-25T10:13:16.041284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043168+0000","last_change":"2024-04-25T10:13:16.043168+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043168+0000","last_undegraded":"2024-04-25T10:13:16.043168+0000","last_fullsized":"2024-04-25T10:13:16.043168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048337+0000","last_change":"2024-04-25T10:13:16.048337+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048337+0000","last_undegraded":"2024-04-25T10:13:16.048337+0000","last_fullsized":"2024-04-25T10:13:16.048337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044692+0000","last_change":"2024-04-25T10:13:16.044692+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044692+0000","last_undegraded":"2024-04-25T10:13:16.044692+0000","last_fullsized":"2024-04-25T10:13:16.044692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043349+0000","last_change":"2024-04-25T10:13:16.043349+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043349+0000","last_undegraded":"2024-04-25T10:13:16.043349+0000","last_fullsized":"2024-04-25T10:13:16.043349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042136+0000","last_change":"2024-04-25T10:13:16.042136+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042136+0000","last_undegraded":"2024-04-25T10:13:16.042136+0000","last_fullsized":"2024-04-25T10:13:16.042136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050728+0000","last_change":"2024-04-25T10:13:16.050728+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050728+0000","last_undegraded":"2024-04-25T10:13:16.050728+0000","last_fullsized":"2024-04-25T10:13:16.050728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050320+0000","last_change":"2024-04-25T10:13:16.050320+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050320+0000","last_undegraded":"2024-04-25T10:13:16.050320+0000","last_fullsized":"2024-04-25T10:13:16.050320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053491+0000","last_change":"2024-04-25T10:13:16.053491+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053491+0000","last_undegraded":"2024-04-25T10:13:16.053491+0000","last_fullsized":"2024-04-25T10:13:16.053491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047586+0000","last_change":"2024-04-25T10:13:16.047586+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047586+0000","last_undegraded":"2024-04-25T10:13:16.047586+0000","last_fullsized":"2024-04-25T10:13:16.047586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063422+0000","last_change":"2024-04-25T10:13:16.059863+0000","last_active":"2024-04-25T10:13:16.063422+0000","last_peered":"2024-04-25T10:13:16.063422+0000","last_clean":"2024-04-25T10:13:16.063422+0000","last_became_active":"2024-04-25T10:13:16.059568+0000","last_became_peered":"2024-04-25T10:13:16.059568+0000","last_unstale":"2024-04-25T10:13:16.063422+0000","last_undegraded":"2024-04-25T10:13:16.063422+0000","last_fullsized":"2024-04-25T10:13:16.063422+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049779+0000","last_change":"2024-04-25T10:13:16.049779+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049779+0000","last_undegraded":"2024-04-25T10:13:16.049779+0000","last_fullsized":"2024-04-25T10:13:16.049779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073269+0000","last_change":"2024-04-25T10:13:16.067950+0000","last_active":"2024-04-25T10:13:16.073269+0000","last_peered":"2024-04-25T10:13:16.073269+0000","last_clean":"2024-04-25T10:13:16.073269+0000","last_became_active":"2024-04-25T10:13:16.067755+0000","last_became_peered":"2024-04-25T10:13:16.067755+0000","last_unstale":"2024-04-25T10:13:16.073269+0000","last_undegraded":"2024-04-25T10:13:16.073269+0000","last_fullsized":"2024-04-25T10:13:16.073269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044196+0000","last_change":"2024-04-25T10:13:16.044196+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044196+0000","last_undegraded":"2024-04-25T10:13:16.044196+0000","last_fullsized":"2024-04-25T10:13:16.044196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064555+0000","last_change":"2024-04-25T10:13:16.058185+0000","last_active":"2024-04-25T10:13:16.064555+0000","last_peered":"2024-04-25T10:13:16.064555+0000","last_clean":"2024-04-25T10:13:16.064555+0000","last_became_active":"2024-04-25T10:13:16.057713+0000","last_became_peered":"2024-04-25T10:13:16.057713+0000","last_unstale":"2024-04-25T10:13:16.064555+0000","last_undegraded":"2024-04-25T10:13:16.064555+0000","last_fullsized":"2024-04-25T10:13:16.064555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:08.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043030+0000","last_change":"2024-04-25T10:13:16.043030+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043030+0000","last_undegraded":"2024-04-25T10:13:16.043030+0000","last_fullsized":"2024-04-25T10:13:16.043030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060374+0000","last_change":"2024-04-25T10:13:16.058303+0000","last_active":"2024-04-25T10:13:16.060374+0000","last_peered":"2024-04-25T10:13:16.060374+0000","last_clean":"2024-04-25T10:13:16.060374+0000","last_became_active":"2024-04-25T10:13:16.057794+0000","last_became_peered":"2024-04-25T10:13:16.057794+0000","last_unstale":"2024-04-25T10:13:16.060374+0000","last_undegraded":"2024-04-25T10:13:16.060374+0000","last_fullsized":"2024-04-25T10:13:16.060374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050240+0000","last_change":"2024-04-25T10:13:16.050240+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050240+0000","last_undegraded":"2024-04-25T10:13:16.050240+0000","last_fullsized":"2024-04-25T10:13:16.050240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064044+0000","last_change":"2024-04-25T10:13:16.058396+0000","last_active":"2024-04-25T10:13:16.064044+0000","last_peered":"2024-04-25T10:13:16.064044+0000","last_clean":"2024-04-25T10:13:16.064044+0000","last_became_active":"2024-04-25T10:13:16.057548+0000","last_became_peered":"2024-04-25T10:13:16.057548+0000","last_unstale":"2024-04-25T10:13:16.064044+0000","last_undegraded":"2024-04-25T10:13:16.064044+0000","last_fullsized":"2024-04-25T10:13:16.064044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:44.890367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041080+0000","last_change":"2024-04-25T10:13:16.041080+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041080+0000","last_undegraded":"2024-04-25T10:13:16.041080+0000","last_fullsized":"2024-04-25T10:13:16.041080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063961+0000","last_change":"2024-04-25T10:13:16.056396+0000","last_active":"2024-04-25T10:13:16.063961+0000","last_peered":"2024-04-25T10:13:16.063961+0000","last_clean":"2024-04-25T10:13:16.063961+0000","last_became_active":"2024-04-25T10:13:16.056119+0000","last_became_peered":"2024-04-25T10:13:16.056119+0000","last_unstale":"2024-04-25T10:13:16.063961+0000","last_undegraded":"2024-04-25T10:13:16.063961+0000","last_fullsized":"2024-04-25T10:13:16.063961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046223+0000","last_change":"2024-04-25T10:13:16.046223+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046223+0000","last_undegraded":"2024-04-25T10:13:16.046223+0000","last_fullsized":"2024-04-25T10:13:16.046223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066147+0000","last_change":"2024-04-25T10:13:16.063791+0000","last_active":"2024-04-25T10:13:16.066147+0000","last_peered":"2024-04-25T10:13:16.066147+0000","last_clean":"2024-04-25T10:13:16.066147+0000","last_became_active":"2024-04-25T10:13:16.063534+0000","last_became_peered":"2024-04-25T10:13:16.063534+0000","last_unstale":"2024-04-25T10:13:16.066147+0000","last_undegraded":"2024-04-25T10:13:16.066147+0000","last_fullsized":"2024-04-25T10:13:16.066147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:25.604894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043697+0000","last_change":"2024-04-25T10:13:16.043697+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043697+0000","last_undegraded":"2024-04-25T10:13:16.043697+0000","last_fullsized":"2024-04-25T10:13:16.043697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066302+0000","last_change":"2024-04-25T10:13:16.061738+0000","last_active":"2024-04-25T10:13:16.066302+0000","last_peered":"2024-04-25T10:13:16.066302+0000","last_clean":"2024-04-25T10:13:16.066302+0000","last_became_active":"2024-04-25T10:13:16.061207+0000","last_became_peered":"2024-04-25T10:13:16.061207+0000","last_unstale":"2024-04-25T10:13:16.066302+0000","last_undegraded":"2024-04-25T10:13:16.066302+0000","last_fullsized":"2024-04-25T10:13:16.066302+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:27.291874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044636+0000","last_change":"2024-04-25T10:13:16.044636+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044636+0000","last_undegraded":"2024-04-25T10:13:16.044636+0000","last_fullsized":"2024-04-25T10:13:16.044636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073358+0000","last_change":"2024-04-25T10:13:16.067967+0000","last_active":"2024-04-25T10:13:16.073358+0000","last_peered":"2024-04-25T10:13:16.073358+0000","last_clean":"2024-04-25T10:13:16.073358+0000","last_became_active":"2024-04-25T10:13:16.067640+0000","last_became_peered":"2024-04-25T10:13:16.067640+0000","last_unstale":"2024-04-25T10:13:16.073358+0000","last_undegraded":"2024-04-25T10:13:16.073358+0000","last_fullsized":"2024-04-25T10:13:16.073358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047672+0000","last_change":"2024-04-25T10:13:16.047672+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047672+0000","last_undegraded":"2024-04-25T10:13:16.047672+0000","last_fullsized":"2024-04-25T10:13:16.047672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060814+0000","last_change":"2024-04-25T10:13:16.058919+0000","last_active":"2024-04-25T10:13:16.060814+0000","last_peered":"2024-04-25T10:13:16.060814+0000","last_clean":"2024-04-25T10:13:16.060814+0000","last_became_active":"2024-04-25T10:13:16.058553+0000","last_became_peered":"2024-04-25T10:13:16.058553+0000","last_unstale":"2024-04-25T10:13:16.060814+0000","last_undegraded":"2024-04-25T10:13:16.060814+0000","last_fullsized":"2024-04-25T10:13:16.060814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:26.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047789+0000","last_change":"2024-04-25T10:13:16.047789+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047789+0000","last_undegraded":"2024-04-25T10:13:16.047789+0000","last_fullsized":"2024-04-25T10:13:16.047789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.068891+0000","last_change":"2024-04-25T10:13:16.064906+0000","last_active":"2024-04-25T10:13:16.068891+0000","last_peered":"2024-04-25T10:13:16.068891+0000","last_clean":"2024-04-25T10:13:16.068891+0000","last_became_active":"2024-04-25T10:13:16.064562+0000","last_became_peered":"2024-04-25T10:13:16.064562+0000","last_unstale":"2024-04-25T10:13:16.068891+0000","last_undegraded":"2024-04-25T10:13:16.068891+0000","last_fullsized":"2024-04-25T10:13:16.068891+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042594+0000","last_change":"2024-04-25T10:13:16.042594+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042594+0000","last_undegraded":"2024-04-25T10:13:16.042594+0000","last_fullsized":"2024-04-25T10:13:16.042594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064513+0000","last_change":"2024-04-25T10:13:16.056267+0000","last_active":"2024-04-25T10:13:16.064513+0000","last_peered":"2024-04-25T10:13:16.064513+0000","last_clean":"2024-04-25T10:13:16.064513+0000","last_became_active":"2024-04-25T10:13:16.055804+0000","last_became_peered":"2024-04-25T10:13:16.055804+0000","last_unstale":"2024-04-25T10:13:16.064513+0000","last_undegraded":"2024-04-25T10:13:16.064513+0000","last_fullsized":"2024-04-25T10:13:16.064513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043721+0000","last_change":"2024-04-25T10:13:16.043721+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043721+0000","last_undegraded":"2024-04-25T10:13:16.043721+0000","last_fullsized":"2024-04-25T10:13:16.043721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066328+0000","last_change":"2024-04-25T10:13:16.061578+0000","last_active":"2024-04-25T10:13:16.066328+0000","last_peered":"2024-04-25T10:13:16.066328+0000","last_clean":"2024-04-25T10:13:16.066328+0000","last_became_active":"2024-04-25T10:13:16.061283+0000","last_became_peered":"2024-04-25T10:13:16.061283+0000","last_unstale":"2024-04-25T10:13:16.066328+0000","last_undegraded":"2024-04-25T10:13:16.066328+0000","last_fullsized":"2024-04-25T10:13:16.066328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:13.032134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046150+0000","last_change":"2024-04-25T10:13:16.046150+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046150+0000","last_undegraded":"2024-04-25T10:13:16.046150+0000","last_fullsized":"2024-04-25T10:13:16.046150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065293+0000","last_change":"2024-04-25T10:13:16.061513+0000","last_active":"2024-04-25T10:13:16.065293+0000","last_peered":"2024-04-25T10:13:16.065293+0000","last_clean":"2024-04-25T10:13:16.065293+0000","last_became_active":"2024-04-25T10:13:16.061216+0000","last_became_peered":"2024-04-25T10:13:16.061216+0000","last_unstale":"2024-04-25T10:13:16.065293+0000","last_undegraded":"2024-04-25T10:13:16.065293+0000","last_fullsized":"2024-04-25T10:13:16.065293+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053049+0000","last_change":"2024-04-25T10:13:16.053049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053049+0000","last_undegraded":"2024-04-25T10:13:16.053049+0000","last_fullsized":"2024-04-25T10:13:16.053049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066124+0000","last_change":"2024-04-25T10:13:16.063235+0000","last_active":"2024-04-25T10:13:16.066124+0000","last_peered":"2024-04-25T10:13:16.066124+0000","last_clean":"2024-04-25T10:13:16.066124+0000","last_became_active":"2024-04-25T10:13:16.062820+0000","last_became_peered":"2024-04-25T10:13:16.062820+0000","last_unstale":"2024-04-25T10:13:16.066124+0000","last_undegraded":"2024-04-25T10:13:16.066124+0000","last_fullsized":"2024-04-25T10:13:16.066124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:09.118013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049640+0000","last_change":"2024-04-25T10:13:16.049640+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049640+0000","last_undegraded":"2024-04-25T10:13:16.049640+0000","last_fullsized":"2024-04-25T10:13:16.049640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064465+0000","last_change":"2024-04-25T10:13:16.057566+0000","last_active":"2024-04-25T10:13:16.064465+0000","last_peered":"2024-04-25T10:13:16.064465+0000","last_clean":"2024-04-25T10:13:16.064465+0000","last_became_active":"2024-04-25T10:13:16.055961+0000","last_became_peered":"2024-04-25T10:13:16.055961+0000","last_unstale":"2024-04-25T10:13:16.064465+0000","last_undegraded":"2024-04-25T10:13:16.064465+0000","last_fullsized":"2024-04-25T10:13:16.064465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043272+0000","last_change":"2024-04-25T10:13:16.043272+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043272+0000","last_undegraded":"2024-04-25T10:13:16.043272+0000","last_fullsized":"2024-04-25T10:13:16.043272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065693+0000","last_change":"2024-04-25T10:13:16.058726+0000","last_active":"2024-04-25T10:13:16.065693+0000","last_peered":"2024-04-25T10:13:16.065693+0000","last_clean":"2024-04-25T10:13:16.065693+0000","last_became_active":"2024-04-25T10:13:16.058253+0000","last_became_peered":"2024-04-25T10:13:16.058253+0000","last_unstale":"2024-04-25T10:13:16.065693+0000","last_undegraded":"2024-04-25T10:13:16.065693+0000","last_fullsized":"2024-04-25T10:13:16.065693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044193+0000","last_change":"2024-04-25T10:13:16.044193+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044193+0000","last_undegraded":"2024-04-25T10:13:16.044193+0000","last_fullsized":"2024-04-25T10:13:16.044193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066628+0000","last_change":"2024-04-25T10:13:16.062295+0000","last_active":"2024-04-25T10:13:16.066628+0000","last_peered":"2024-04-25T10:13:16.066628+0000","last_clean":"2024-04-25T10:13:16.066628+0000","last_became_active":"2024-04-25T10:13:16.061270+0000","last_became_peered":"2024-04-25T10:13:16.061270+0000","last_unstale":"2024-04-25T10:13:16.066628+0000","last_undegraded":"2024-04-25T10:13:16.066628+0000","last_fullsized":"2024-04-25T10:13:16.066628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:11.953599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042049+0000","last_change":"2024-04-25T10:13:16.042049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042049+0000","last_undegraded":"2024-04-25T10:13:16.042049+0000","last_fullsized":"2024-04-25T10:13:16.042049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063994+0000","last_change":"2024-04-25T10:13:16.055470+0000","last_active":"2024-04-25T10:13:16.063994+0000","last_peered":"2024-04-25T10:13:16.063994+0000","last_clean":"2024-04-25T10:13:16.063994+0000","last_became_active":"2024-04-25T10:13:16.055137+0000","last_became_peered":"2024-04-25T10:13:16.055137+0000","last_unstale":"2024-04-25T10:13:16.063994+0000","last_undegraded":"2024-04-25T10:13:16.063994+0000","last_fullsized":"2024-04-25T10:13:16.063994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:10.904343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046361+0000","last_change":"2024-04-25T10:13:16.046361+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046361+0000","last_undegraded":"2024-04-25T10:13:16.046361+0000","last_fullsized":"2024-04-25T10:13:16.046361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064028+0000","last_change":"2024-04-25T10:13:16.057838+0000","last_active":"2024-04-25T10:13:16.064028+0000","last_peered":"2024-04-25T10:13:16.064028+0000","last_clean":"2024-04-25T10:13:16.064028+0000","last_became_active":"2024-04-25T10:13:16.057476+0000","last_became_peered":"2024-04-25T10:13:16.057476+0000","last_unstale":"2024-04-25T10:13:16.064028+0000","last_undegraded":"2024-04-25T10:13:16.064028+0000","last_fullsized":"2024-04-25T10:13:16.064028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:39.436447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046108+0000","last_change":"2024-04-25T10:13:16.046108+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046108+0000","last_undegraded":"2024-04-25T10:13:16.046108+0000","last_fullsized":"2024-04-25T10:13:16.046108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070376+0000","last_change":"2024-04-25T10:13:16.065980+0000","last_active":"2024-04-25T10:13:16.070376+0000","last_peered":"2024-04-25T10:13:16.070376+0000","last_clean":"2024-04-25T10:13:16.070376+0000","last_became_active":"2024-04-25T10:13:16.065704+0000","last_became_peered":"2024-04-25T10:13:16.065704+0000","last_unstale":"2024-04-25T10:13:16.070376+0000","last_undegraded":"2024-04-25T10:13:16.070376+0000","last_fullsized":"2024-04-25T10:13:16.070376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046010+0000","last_change":"2024-04-25T10:13:16.046010+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046010+0000","last_undegraded":"2024-04-25T10:13:16.046010+0000","last_fullsized":"2024-04-25T10:13:16.046010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.069159+0000","last_change":"2024-04-25T10:13:16.065367+0000","last_active":"2024-04-25T10:13:16.069159+0000","last_peered":"2024-04-25T10:13:16.069159+0000","last_clean":"2024-04-25T10:13:16.069159+0000","last_became_active":"2024-04-25T10:13:16.065198+0000","last_became_peered":"2024-04-25T10:13:16.065198+0000","last_unstale":"2024-04-25T10:13:16.069159+0000","last_undegraded":"2024-04-25T10:13:16.069159+0000","last_fullsized":"2024-04-25T10:13:16.069159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051599+0000","last_change":"2024-04-25T10:13:16.051599+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051599+0000","last_undegraded":"2024-04-25T10:13:16.051599+0000","last_fullsized":"2024-04-25T10:13:16.051599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066646+0000","last_change":"2024-04-25T10:13:16.062433+0000","last_active":"2024-04-25T10:13:16.066646+0000","last_peered":"2024-04-25T10:13:16.066646+0000","last_clean":"2024-04-25T10:13:16.066646+0000","last_became_active":"2024-04-25T10:13:16.062227+0000","last_became_peered":"2024-04-25T10:13:16.062227+0000","last_unstale":"2024-04-25T10:13:16.066646+0000","last_undegraded":"2024-04-25T10:13:16.066646+0000","last_fullsized":"2024-04-25T10:13:16.066646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:33.301434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051355+0000","last_change":"2024-04-25T10:13:16.051355+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051355+0000","last_undegraded":"2024-04-25T10:13:16.051355+0000","last_fullsized":"2024-04-25T10:13:16.051355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063289+0000","last_change":"2024-04-25T10:13:16.057718+0000","last_active":"2024-04-25T10:13:16.063289+0000","last_peered":"2024-04-25T10:13:16.063289+0000","last_clean":"2024-04-25T10:13:16.063289+0000","last_became_active":"2024-04-25T10:13:16.057007+0000","last_became_peered":"2024-04-25T10:13:16.057007+0000","last_unstale":"2024-04-25T10:13:16.063289+0000","last_undegraded":"2024-04-25T10:13:16.063289+0000","last_fullsized":"2024-04-25T10:13:16.063289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046460+0000","last_change":"2024-04-25T10:13:16.046460+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046460+0000","last_undegraded":"2024-04-25T10:13:16.046460+0000","last_fullsized":"2024-04-25T10:13:16.046460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.056100+0000","last_change":"2024-04-25T10:13:16.051248+0000","last_active":"2024-04-25T10:13:16.056100+0000","last_peered":"2024-04-25T10:13:16.056100+0000","last_clean":"2024-04-25T10:13:16.056100+0000","last_became_active":"2024-04-25T10:13:16.050951+0000","last_became_peered":"2024-04-25T10:13:16.050951+0000","last_unstale":"2024-04-25T10:13:16.056100+0000","last_undegraded":"2024-04-25T10:13:16.056100+0000","last_fullsized":"2024-04-25T10:13:16.056100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043744+0000","last_change":"2024-04-25T10:13:16.043744+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043744+0000","last_undegraded":"2024-04-25T10:13:16.043744+0000","last_fullsized":"2024-04-25T10:13:16.043744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070444+0000","last_change":"2024-04-25T10:13:16.066644+0000","last_active":"2024-04-25T10:13:16.070444+0000","last_peered":"2024-04-25T10:13:16.070444+0000","last_clean":"2024-04-25T10:13:16.070444+0000","last_became_active":"2024-04-25T10:13:16.064572+0000","last_became_peered":"2024-04-25T10:13:16.064572+0000","last_unstale":"2024-04-25T10:13:16.070444+0000","last_undegraded":"2024-04-25T10:13:16.070444+0000","last_fullsized":"2024-04-25T10:13:16.070444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048564+0000","last_change":"2024-04-25T10:13:16.048564+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048564+0000","last_undegraded":"2024-04-25T10:13:16.048564+0000","last_fullsized":"2024-04-25T10:13:16.048564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.061896+0000","last_change":"2024-04-25T10:13:16.057038+0000","last_active":"2024-04-25T10:13:16.061896+0000","last_peered":"2024-04-25T10:13:16.061896+0000","last_clean":"2024-04-25T10:13:16.061896+0000","last_became_active":"2024-04-25T10:13:16.056758+0000","last_became_peered":"2024-04-25T10:13:16.056758+0000","last_unstale":"2024-04-25T10:13:16.061896+0000","last_undegraded":"2024-04-25T10:13:16.061896+0000","last_fullsized":"2024-04-25T10:13:16.061896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050996+0000","last_change":"2024-04-25T10:13:16.050996+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050996+0000","last_undegraded":"2024-04-25T10:13:16.050996+0000","last_fullsized":"2024-04-25T10:13:16.050996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060591+0000","last_change":"2024-04-25T10:13:16.055447+0000","last_active":"2024-04-25T10:13:16.060591+0000","last_peered":"2024-04-25T10:13:16.060591+0000","last_clean":"2024-04-25T10:13:16.060591+0000","last_became_active":"2024-04-25T10:13:16.055157+0000","last_became_peered":"2024-04-25T10:13:16.055157+0000","last_unstale":"2024-04-25T10:13:16.060591+0000","last_undegraded":"2024-04-25T10:13:16.060591+0000","last_fullsized":"2024-04-25T10:13:16.060591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:13.025816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047880+0000","last_change":"2024-04-25T10:13:16.047880+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047880+0000","last_undegraded":"2024-04-25T10:13:16.047880+0000","last_fullsized":"2024-04-25T10:13:16.047880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.054639+0000","last_change":"2024-04-25T10:13:16.049162+0000","last_active":"2024-04-25T10:13:16.054639+0000","last_peered":"2024-04-25T10:13:16.054639+0000","last_clean":"2024-04-25T10:13:16.054639+0000","last_became_active":"2024-04-25T10:13:16.048699+0000","last_became_peered":"2024-04-25T10:13:16.048699+0000","last_unstale":"2024-04-25T10:13:16.054639+0000","last_undegraded":"2024-04-25T10:13:16.054639+0000","last_fullsized":"2024-04-25T10:13:16.054639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043077+0000","last_change":"2024-04-25T10:13:16.043077+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043077+0000","last_undegraded":"2024-04-25T10:13:16.043077+0000","last_fullsized":"2024-04-25T10:13:16.043077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.055963+0000","last_change":"2024-04-25T10:13:16.052633+0000","last_active":"2024-04-25T10:13:16.055963+0000","last_peered":"2024-04-25T10:13:16.055963+0000","last_clean":"2024-04-25T10:13:16.055963+0000","last_became_active":"2024-04-25T10:13:16.052308+0000","last_became_peered":"2024-04-25T10:13:16.052308+0000","last_unstale":"2024-04-25T10:13:16.055963+0000","last_undegraded":"2024-04-25T10:13:16.055963+0000","last_fullsized":"2024-04-25T10:13:16.055963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:01.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043595+0000","last_change":"2024-04-25T10:13:16.043595+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043595+0000","last_undegraded":"2024-04-25T10:13:16.043595+0000","last_fullsized":"2024-04-25T10:13:16.043595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070420+0000","last_change":"2024-04-25T10:13:16.065524+0000","last_active":"2024-04-25T10:13:16.070420+0000","last_peered":"2024-04-25T10:13:16.070420+0000","last_clean":"2024-04-25T10:13:16.070420+0000","last_became_active":"2024-04-25T10:13:16.065056+0000","last_became_peered":"2024-04-25T10:13:16.065056+0000","last_unstale":"2024-04-25T10:13:16.070420+0000","last_undegraded":"2024-04-25T10:13:16.070420+0000","last_fullsized":"2024-04-25T10:13:16.070420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:18.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.052506+0000","last_change":"2024-04-25T10:13:16.052506+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.052506+0000","last_undegraded":"2024-04-25T10:13:16.052506+0000","last_fullsized":"2024-04-25T10:13:16.052506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.072389+0000","last_change":"2024-04-25T10:13:16.066995+0000","last_active":"2024-04-25T10:13:16.072389+0000","last_peered":"2024-04-25T10:13:16.072389+0000","last_clean":"2024-04-25T10:13:16.072389+0000","last_became_active":"2024-04-25T10:13:16.066686+0000","last_became_peered":"2024-04-25T10:13:16.066686+0000","last_unstale":"2024-04-25T10:13:16.072389+0000","last_undegraded":"2024-04-25T10:13:16.072389+0000","last_fullsized":"2024-04-25T10:13:16.072389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.059855+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:13:16.059855+0000","last_peered":"2024-04-25T10:13:16.059855+0000","last_clean":"2024-04-25T10:13:16.059855+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:13:16.059855+0000","last_undegraded":"2024-04-25T10:13:16.059855+0000","last_fullsized":"2024-04-25T10:13:16.059855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:21.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:21.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.290+0000 7fd660e00640 1 Processor -- start 2024-04-25T10:13:21.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.290+0000 7fd660e00640 1 -- start start 2024-04-25T10:13:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd660e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd65c15bc40 0x7fd65c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd660e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd660e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0b2770 con 0x7fd65c0cf930 2024-04-25T10:13:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd660e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c06c670 con 0x7fd65c15bc40 2024-04-25T10:13:21.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd660e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c06c950 con 0x7fd65c0cda10 2024-04-25T10:13:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd65c15bc40 0x7fd65c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd65c15bc40 0x7fd65c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44894/0 (socket says 172.21.15.87:44894) 2024-04-25T10:13:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659e00640 1 -- 172.21.15.87:0/2005096701 learned_addr learned my addr 172.21.15.87:0/2005096701 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:21.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd65a800640 1 --1- 172.21.15.87:0/2005096701 >> v1:172.21.15.87:6789/0 conn(0x7fd65c0cf930 0x7fd65c15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56706/0 (socket says 172.21.15.87:56706) 2024-04-25T10:13:21.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd651400640 1 --2- 172.21.15.87:0/2005096701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659e00640 1 -- 172.21.15.87:0/2005096701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 msgr2=0x7fd65c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659e00640 1 --2- 172.21.15.87:0/2005096701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659e00640 1 -- 172.21.15.87:0/2005096701 >> v1:172.21.15.87:6789/0 conn(0x7fd65c0cf930 legacy=0x7fd65c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659400640 1 -- 172.21.15.87:0/2005096701 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519123145 0 0) 0x7fd65c0b2770 con 0x7fd65c0cf930 2024-04-25T10:13:21.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd659e00640 1 -- 172.21.15.87:0/2005096701 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c06cc30 con 0x7fd65c15bc40 2024-04-25T10:13:21.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.294+0000 7fd651400640 1 --2- 172.21.15.87:0/2005096701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd659e00640 1 --2- 172.21.15.87:0/2005096701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c06c090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009e50 tx=0x7fd64c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb78ed81b2c3b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd659400640 1 -- 172.21.15.87:0/2005096701 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0a5810 con 0x7fd65c15bc40 2024-04-25T10:13:21.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd659400640 1 -- 172.21.15.87:0/2005096701 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c0a59b0 con 0x7fd65c15bc40 2024-04-25T10:13:21.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd659400640 1 -- 172.21.15.87:0/2005096701 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0a5d80 con 0x7fd65c15bc40 2024-04-25T10:13:21.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- 172.21.15.87:0/2005096701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 msgr2=0x7fd65c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 --2- 172.21.15.87:0/2005096701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c06c090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009e50 tx=0x7fd64c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:13:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- 172.21.15.87:0/2005096701 shutdown_connections 2024-04-25T10:13:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 --2- 172.21.15.87:0/2005096701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 --2- 172.21.15.87:0/2005096701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c06c090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- 172.21.15.87:0/2005096701 >> 172.21.15.87:0/2005096701 conn(0x7fd65c066b70 msgr2=0x7fd65c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- 172.21.15.87:0/2005096701 shutdown_connections 2024-04-25T10:13:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- 172.21.15.87:0/2005096701 wait complete. 2024-04-25T10:13:21.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 Processor -- start 2024-04-25T10:13:21.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- start start 2024-04-25T10:13:21.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd65c0cf930 0x7fd65c154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c168b50 con 0x7fd65c15bc40 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1689d0 con 0x7fd65c0cf930 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd660e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c168cd0 con 0x7fd65c0cda10 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd65a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd65a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c1542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44906/0 (socket says 172.21.15.87:44906) 2024-04-25T10:13:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.298+0000 7fd65a800640 1 -- 172.21.15.87:0/2748977847 learned_addr learned my addr 172.21.15.87:0/2748977847 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd651400640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd65c0cf930 0x7fd65c154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd659e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd65a800640 1 -- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 msgr2=0x7fd65c15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd65a800640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c15d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd65a800640 1 -- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd65c0cf930 msgr2=0x7fd65c154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd65a800640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd65c0cf930 0x7fd65c154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd65a800640 1 -- 172.21.15.87:0/2748977847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c15d9d0 con 0x7fd65c15bc40 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd651400640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd65c0cf930 0x7fd65c154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd659e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd65a800640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c1542b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c3e0 tx=0x7fd65400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd652a00640 1 -- 172.21.15.87:0/2748977847 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540074e0 con 0x7fd65c15bc40 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd652a00640 1 -- 172.21.15.87:0/2748977847 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6540047f0 con 0x7fd65c15bc40 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd652a00640 1 -- 172.21.15.87:0/2748977847 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654004be0 con 0x7fd65c15bc40 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c15dcb0 con 0x7fd65c15bc40 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c153ab0 con 0x7fd65c15bc40 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c04ef30 con 0x7fd65c15bc40 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd652a00640 1 -- 172.21.15.87:0/2748977847 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd654011070 con 0x7fd65c15bc40 2024-04-25T10:13:21.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd652a00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd638066f90 0x7fd638069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd652a00640 1 -- 172.21.15.87:0/2748977847 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd654084c40 con 0x7fd65c15bc40 2024-04-25T10:13:21.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.302+0000 7fd659e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd638066f90 0x7fd638069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.306+0000 7fd659e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd638066f90 0x7fd638069450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0ae000 tx=0x7fd64c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.306+0000 7fd652a00640 1 -- 172.21.15.87:0/2748977847 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd654055bb0 con 0x7fd65c15bc40 2024-04-25T10:13:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.458+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd65c067050 con 0x7fd638066f90 2024-04-25T10:13:21.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd652a00640 1 -- 172.21.15.87:0/2748977847 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246632 (secure 0 0 0) 0x7fd65c067050 con 0x7fd638066f90 2024-04-25T10:13:21.475 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:21.476 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:13:21.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd638066f90 msgr2=0x7fd638069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd638066f90 0x7fd638069450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0ae000 tx=0x7fd64c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 msgr2=0x7fd65c1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c1542b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c3e0 tx=0x7fd65400c8a0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 shutdown_connections 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd638066f90 0x7fd638069450 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c15bc40 0x7fd65c1542b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd65c0cf930 0x7fd65c154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 --2- 172.21.15.87:0/2748977847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd65c0cda10 0x7fd65c15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.466+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 >> 172.21.15.87:0/2748977847 conn(0x7fd65c066b70 msgr2=0x7fd65c15a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.470+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 shutdown_connections 2024-04-25T10:13:21.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.470+0000 7fd660e00640 1 -- 172.21.15.87:0/2748977847 wait complete. 2024-04-25T10:13:21.495 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-25T10:13:21.135987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219960,"kb_used_data":3796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754760,"statfs":{"total":773094113280,"available":772868874240,"internally_reserved":0,"allocated":3887104,"data_stored":2205195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000989"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051524+0000","last_change":"2024-04-25T10:13:16.051524+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051524+0000","last_undegraded":"2024-04-25T10:13:16.051524+0000","last_fullsized":"2024-04-25T10:13:16.051524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049178+0000","last_change":"2024-04-25T10:13:16.049178+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049178+0000","last_undegraded":"2024-04-25T10:13:16.049178+0000","last_fullsized":"2024-04-25T10:13:16.049178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042138+0000","last_change":"2024-04-25T10:13:16.042138+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042138+0000","last_undegraded":"2024-04-25T10:13:16.042138+0000","last_fullsized":"2024-04-25T10:13:16.042138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043287+0000","last_change":"2024-04-25T10:13:16.043287+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043287+0000","last_undegraded":"2024-04-25T10:13:16.043287+0000","last_fullsized":"2024-04-25T10:13:16.043287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051209+0000","last_change":"2024-04-25T10:13:16.051209+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051209+0000","last_undegraded":"2024-04-25T10:13:16.051209+0000","last_fullsized":"2024-04-25T10:13:16.051209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048910+0000","last_change":"2024-04-25T10:13:16.048910+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048910+0000","last_undegraded":"2024-04-25T10:13:16.048910+0000","last_fullsized":"2024-04-25T10:13:16.048910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050714+0000","last_change":"2024-04-25T10:13:16.050714+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050714+0000","last_undegraded":"2024-04-25T10:13:16.050714+0000","last_fullsized":"2024-04-25T10:13:16.050714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053901+0000","last_change":"2024-04-25T10:13:16.053901+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053901+0000","last_undegraded":"2024-04-25T10:13:16.053901+0000","last_fullsized":"2024-04-25T10:13:16.053901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044137+0000","last_change":"2024-04-25T10:13:16.044137+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044137+0000","last_undegraded":"2024-04-25T10:13:16.044137+0000","last_fullsized":"2024-04-25T10:13:16.044137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043363+0000","last_change":"2024-04-25T10:13:16.043363+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043363+0000","last_undegraded":"2024-04-25T10:13:16.043363+0000","last_fullsized":"2024-04-25T10:13:16.043363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.054965+0000","last_change":"2024-04-25T10:13:16.054965+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.054965+0000","last_undegraded":"2024-04-25T10:13:16.054965+0000","last_fullsized":"2024-04-25T10:13:16.054965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043819+0000","last_change":"2024-04-25T10:13:16.043819+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043819+0000","last_undegraded":"2024-04-25T10:13:16.043819+0000","last_fullsized":"2024-04-25T10:13:16.043819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046803+0000","last_change":"2024-04-25T10:13:16.046803+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046803+0000","last_undegraded":"2024-04-25T10:13:16.046803+0000","last_fullsized":"2024-04-25T10:13:16.046803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048465+0000","last_change":"2024-04-25T10:13:16.048465+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048465+0000","last_undegraded":"2024-04-25T10:13:16.048465+0000","last_fullsized":"2024-04-25T10:13:16.048465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041761+0000","last_change":"2024-04-25T10:13:16.041761+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041761+0000","last_undegraded":"2024-04-25T10:13:16.041761+0000","last_fullsized":"2024-04-25T10:13:16.041761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051289+0000","last_change":"2024-04-25T10:13:16.051289+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051289+0000","last_undegraded":"2024-04-25T10:13:16.051289+0000","last_fullsized":"2024-04-25T10:13:16.051289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.057085+0000","last_change":"2024-04-25T10:13:16.057085+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.057085+0000","last_undegraded":"2024-04-25T10:13:16.057085+0000","last_fullsized":"2024-04-25T10:13:16.057085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048752+0000","last_change":"2024-04-25T10:13:16.048752+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048752+0000","last_undegraded":"2024-04-25T10:13:16.048752+0000","last_fullsized":"2024-04-25T10:13:16.048752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050699+0000","last_change":"2024-04-25T10:13:16.050699+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050699+0000","last_undegraded":"2024-04-25T10:13:16.050699+0000","last_fullsized":"2024-04-25T10:13:16.050699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050639+0000","last_change":"2024-04-25T10:13:16.050639+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050639+0000","last_undegraded":"2024-04-25T10:13:16.050639+0000","last_fullsized":"2024-04-25T10:13:16.050639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043315+0000","last_change":"2024-04-25T10:13:16.043315+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043315+0000","last_undegraded":"2024-04-25T10:13:16.043315+0000","last_fullsized":"2024-04-25T10:13:16.043315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042542+0000","last_change":"2024-04-25T10:13:16.042542+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042542+0000","last_undegraded":"2024-04-25T10:13:16.042542+0000","last_fullsized":"2024-04-25T10:13:16.042542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047344+0000","last_change":"2024-04-25T10:13:16.047344+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047344+0000","last_undegraded":"2024-04-25T10:13:16.047344+0000","last_fullsized":"2024-04-25T10:13:16.047344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041284+0000","last_change":"2024-04-25T10:13:16.041284+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041284+0000","last_undegraded":"2024-04-25T10:13:16.041284+0000","last_fullsized":"2024-04-25T10:13:16.041284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043168+0000","last_change":"2024-04-25T10:13:16.043168+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043168+0000","last_undegraded":"2024-04-25T10:13:16.043168+0000","last_fullsized":"2024-04-25T10:13:16.043168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048337+0000","last_change":"2024-04-25T10:13:16.048337+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048337+0000","last_undegraded":"2024-04-25T10:13:16.048337+0000","last_fullsized":"2024-04-25T10:13:16.048337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044692+0000","last_change":"2024-04-25T10:13:16.044692+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044692+0000","last_undegraded":"2024-04-25T10:13:16.044692+0000","last_fullsized":"2024-04-25T10:13:16.044692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043349+0000","last_change":"2024-04-25T10:13:16.043349+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043349+0000","last_undegraded":"2024-04-25T10:13:16.043349+0000","last_fullsized":"2024-04-25T10:13:16.043349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042136+0000","last_change":"2024-04-25T10:13:16.042136+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042136+0000","last_undegraded":"2024-04-25T10:13:16.042136+0000","last_fullsized":"2024-04-25T10:13:16.042136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050728+0000","last_change":"2024-04-25T10:13:16.050728+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050728+0000","last_undegraded":"2024-04-25T10:13:16.050728+0000","last_fullsized":"2024-04-25T10:13:16.050728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050320+0000","last_change":"2024-04-25T10:13:16.050320+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050320+0000","last_undegraded":"2024-04-25T10:13:16.050320+0000","last_fullsized":"2024-04-25T10:13:16.050320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053491+0000","last_change":"2024-04-25T10:13:16.053491+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053491+0000","last_undegraded":"2024-04-25T10:13:16.053491+0000","last_fullsized":"2024-04-25T10:13:16.053491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047586+0000","last_change":"2024-04-25T10:13:16.047586+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047586+0000","last_undegraded":"2024-04-25T10:13:16.047586+0000","last_fullsized":"2024-04-25T10:13:16.047586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063422+0000","last_change":"2024-04-25T10:13:16.059863+0000","last_active":"2024-04-25T10:13:16.063422+0000","last_peered":"2024-04-25T10:13:16.063422+0000","last_clean":"2024-04-25T10:13:16.063422+0000","last_became_active":"2024-04-25T10:13:16.059568+0000","last_became_peered":"2024-04-25T10:13:16.059568+0000","last_unstale":"2024-04-25T10:13:16.063422+0000","last_undegraded":"2024-04-25T10:13:16.063422+0000","last_fullsized":"2024-04-25T10:13:16.063422+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049779+0000","last_change":"2024-04-25T10:13:16.049779+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049779+0000","last_undegraded":"2024-04-25T10:13:16.049779+0000","last_fullsized":"2024-04-25T10:13:16.049779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073269+0000","last_change":"2024-04-25T10:13:16.067950+0000","last_active":"2024-04-25T10:13:16.073269+0000","last_peered":"2024-04-25T10:13:16.073269+0000","last_clean":"2024-04-25T10:13:16.073269+0000","last_became_active":"2024-04-25T10:13:16.067755+0000","last_became_peered":"2024-04-25T10:13:16.067755+0000","last_unstale":"2024-04-25T10:13:16.073269+0000","last_undegraded":"2024-04-25T10:13:16.073269+0000","last_fullsized":"2024-04-25T10:13:16.073269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044196+0000","last_change":"2024-04-25T10:13:16.044196+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044196+0000","last_undegraded":"2024-04-25T10:13:16.044196+0000","last_fullsized":"2024-04-25T10:13:16.044196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064555+0000","last_change":"2024-04-25T10:13:16.058185+0000","last_active":"2024-04-25T10:13:16.064555+0000","last_peered":"2024-04-25T10:13:16.064555+0000","last_clean":"2024-04-25T10:13:16.064555+0000","last_became_active":"2024-04-25T10:13:16.057713+0000","last_became_peered":"2024-04-25T10:13:16.057713+0000","last_unstale":"2024-04-25T10:13:16.064555+0000","last_undegraded":"2024-04-25T10:13:16.064555+0000","last_fullsized":"2024-04-25T10:13:16.064555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:08.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043030+0000","last_change":"2024-04-25T10:13:16.043030+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043030+0000","last_undegraded":"2024-04-25T10:13:16.043030+0000","last_fullsized":"2024-04-25T10:13:16.043030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060374+0000","last_change":"2024-04-25T10:13:16.058303+0000","last_active":"2024-04-25T10:13:16.060374+0000","last_peered":"2024-04-25T10:13:16.060374+0000","last_clean":"2024-04-25T10:13:16.060374+0000","last_became_active":"2024-04-25T10:13:16.057794+0000","last_became_peered":"2024-04-25T10:13:16.057794+0000","last_unstale":"2024-04-25T10:13:16.060374+0000","last_undegraded":"2024-04-25T10:13:16.060374+0000","last_fullsized":"2024-04-25T10:13:16.060374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050240+0000","last_change":"2024-04-25T10:13:16.050240+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050240+0000","last_undegraded":"2024-04-25T10:13:16.050240+0000","last_fullsized":"2024-04-25T10:13:16.050240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064044+0000","last_change":"2024-04-25T10:13:16.058396+0000","last_active":"2024-04-25T10:13:16.064044+0000","last_peered":"2024-04-25T10:13:16.064044+0000","last_clean":"2024-04-25T10:13:16.064044+0000","last_became_active":"2024-04-25T10:13:16.057548+0000","last_became_peered":"2024-04-25T10:13:16.057548+0000","last_unstale":"2024-04-25T10:13:16.064044+0000","last_undegraded":"2024-04-25T10:13:16.064044+0000","last_fullsized":"2024-04-25T10:13:16.064044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:44.890367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041080+0000","last_change":"2024-04-25T10:13:16.041080+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041080+0000","last_undegraded":"2024-04-25T10:13:16.041080+0000","last_fullsized":"2024-04-25T10:13:16.041080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063961+0000","last_change":"2024-04-25T10:13:16.056396+0000","last_active":"2024-04-25T10:13:16.063961+0000","last_peered":"2024-04-25T10:13:16.063961+0000","last_clean":"2024-04-25T10:13:16.063961+0000","last_became_active":"2024-04-25T10:13:16.056119+0000","last_became_peered":"2024-04-25T10:13:16.056119+0000","last_unstale":"2024-04-25T10:13:16.063961+0000","last_undegraded":"2024-04-25T10:13:16.063961+0000","last_fullsized":"2024-04-25T10:13:16.063961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046223+0000","last_change":"2024-04-25T10:13:16.046223+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046223+0000","last_undegraded":"2024-04-25T10:13:16.046223+0000","last_fullsized":"2024-04-25T10:13:16.046223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066147+0000","last_change":"2024-04-25T10:13:16.063791+0000","last_active":"2024-04-25T10:13:16.066147+0000","last_peered":"2024-04-25T10:13:16.066147+0000","last_clean":"2024-04-25T10:13:16.066147+0000","last_became_active":"2024-04-25T10:13:16.063534+0000","last_became_peered":"2024-04-25T10:13:16.063534+0000","last_unstale":"2024-04-25T10:13:16.066147+0000","last_undegraded":"2024-04-25T10:13:16.066147+0000","last_fullsized":"2024-04-25T10:13:16.066147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:25.604894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043697+0000","last_change":"2024-04-25T10:13:16.043697+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043697+0000","last_undegraded":"2024-04-25T10:13:16.043697+0000","last_fullsized":"2024-04-25T10:13:16.043697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066302+0000","last_change":"2024-04-25T10:13:16.061738+0000","last_active":"2024-04-25T10:13:16.066302+0000","last_peered":"2024-04-25T10:13:16.066302+0000","last_clean":"2024-04-25T10:13:16.066302+0000","last_became_active":"2024-04-25T10:13:16.061207+0000","last_became_peered":"2024-04-25T10:13:16.061207+0000","last_unstale":"2024-04-25T10:13:16.066302+0000","last_undegraded":"2024-04-25T10:13:16.066302+0000","last_fullsized":"2024-04-25T10:13:16.066302+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:27.291874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044636+0000","last_change":"2024-04-25T10:13:16.044636+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044636+0000","last_undegraded":"2024-04-25T10:13:16.044636+0000","last_fullsized":"2024-04-25T10:13:16.044636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073358+0000","last_change":"2024-04-25T10:13:16.067967+0000","last_active":"2024-04-25T10:13:16.073358+0000","last_peered":"2024-04-25T10:13:16.073358+0000","last_clean":"2024-04-25T10:13:16.073358+0000","last_became_active":"2024-04-25T10:13:16.067640+0000","last_became_peered":"2024-04-25T10:13:16.067640+0000","last_unstale":"2024-04-25T10:13:16.073358+0000","last_undegraded":"2024-04-25T10:13:16.073358+0000","last_fullsized":"2024-04-25T10:13:16.073358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047672+0000","last_change":"2024-04-25T10:13:16.047672+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047672+0000","last_undegraded":"2024-04-25T10:13:16.047672+0000","last_fullsized":"2024-04-25T10:13:16.047672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060814+0000","last_change":"2024-04-25T10:13:16.058919+0000","last_active":"2024-04-25T10:13:16.060814+0000","last_peered":"2024-04-25T10:13:16.060814+0000","last_clean":"2024-04-25T10:13:16.060814+0000","last_became_active":"2024-04-25T10:13:16.058553+0000","last_became_peered":"2024-04-25T10:13:16.058553+0000","last_unstale":"2024-04-25T10:13:16.060814+0000","last_undegraded":"2024-04-25T10:13:16.060814+0000","last_fullsized":"2024-04-25T10:13:16.060814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:26.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047789+0000","last_change":"2024-04-25T10:13:16.047789+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047789+0000","last_undegraded":"2024-04-25T10:13:16.047789+0000","last_fullsized":"2024-04-25T10:13:16.047789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.068891+0000","last_change":"2024-04-25T10:13:16.064906+0000","last_active":"2024-04-25T10:13:16.068891+0000","last_peered":"2024-04-25T10:13:16.068891+0000","last_clean":"2024-04-25T10:13:16.068891+0000","last_became_active":"2024-04-25T10:13:16.064562+0000","last_became_peered":"2024-04-25T10:13:16.064562+0000","last_unstale":"2024-04-25T10:13:16.068891+0000","last_undegraded":"2024-04-25T10:13:16.068891+0000","last_fullsized":"2024-04-25T10:13:16.068891+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042594+0000","last_change":"2024-04-25T10:13:16.042594+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042594+0000","last_undegraded":"2024-04-25T10:13:16.042594+0000","last_fullsized":"2024-04-25T10:13:16.042594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064513+0000","last_change":"2024-04-25T10:13:16.056267+0000","last_active":"2024-04-25T10:13:16.064513+0000","last_peered":"2024-04-25T10:13:16.064513+0000","last_clean":"2024-04-25T10:13:16.064513+0000","last_became_active":"2024-04-25T10:13:16.055804+0000","last_became_peered":"2024-04-25T10:13:16.055804+0000","last_unstale":"2024-04-25T10:13:16.064513+0000","last_undegraded":"2024-04-25T10:13:16.064513+0000","last_fullsized":"2024-04-25T10:13:16.064513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043721+0000","last_change":"2024-04-25T10:13:16.043721+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043721+0000","last_undegraded":"2024-04-25T10:13:16.043721+0000","last_fullsized":"2024-04-25T10:13:16.043721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066328+0000","last_change":"2024-04-25T10:13:16.061578+0000","last_active":"2024-04-25T10:13:16.066328+0000","last_peered":"2024-04-25T10:13:16.066328+0000","last_clean":"2024-04-25T10:13:16.066328+0000","last_became_active":"2024-04-25T10:13:16.061283+0000","last_became_peered":"2024-04-25T10:13:16.061283+0000","last_unstale":"2024-04-25T10:13:16.066328+0000","last_undegraded":"2024-04-25T10:13:16.066328+0000","last_fullsized":"2024-04-25T10:13:16.066328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:13.032134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046150+0000","last_change":"2024-04-25T10:13:16.046150+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046150+0000","last_undegraded":"2024-04-25T10:13:16.046150+0000","last_fullsized":"2024-04-25T10:13:16.046150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065293+0000","last_change":"2024-04-25T10:13:16.061513+0000","last_active":"2024-04-25T10:13:16.065293+0000","last_peered":"2024-04-25T10:13:16.065293+0000","last_clean":"2024-04-25T10:13:16.065293+0000","last_became_active":"2024-04-25T10:13:16.061216+0000","last_became_peered":"2024-04-25T10:13:16.061216+0000","last_unstale":"2024-04-25T10:13:16.065293+0000","last_undegraded":"2024-04-25T10:13:16.065293+0000","last_fullsized":"2024-04-25T10:13:16.065293+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053049+0000","last_change":"2024-04-25T10:13:16.053049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053049+0000","last_undegraded":"2024-04-25T10:13:16.053049+0000","last_fullsized":"2024-04-25T10:13:16.053049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066124+0000","last_change":"2024-04-25T10:13:16.063235+0000","last_active":"2024-04-25T10:13:16.066124+0000","last_peered":"2024-04-25T10:13:16.066124+0000","last_clean":"2024-04-25T10:13:16.066124+0000","last_became_active":"2024-04-25T10:13:16.062820+0000","last_became_peered":"2024-04-25T10:13:16.062820+0000","last_unstale":"2024-04-25T10:13:16.066124+0000","last_undegraded":"2024-04-25T10:13:16.066124+0000","last_fullsized":"2024-04-25T10:13:16.066124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:09.118013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049640+0000","last_change":"2024-04-25T10:13:16.049640+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049640+0000","last_undegraded":"2024-04-25T10:13:16.049640+0000","last_fullsized":"2024-04-25T10:13:16.049640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064465+0000","last_change":"2024-04-25T10:13:16.057566+0000","last_active":"2024-04-25T10:13:16.064465+0000","last_peered":"2024-04-25T10:13:16.064465+0000","last_clean":"2024-04-25T10:13:16.064465+0000","last_became_active":"2024-04-25T10:13:16.055961+0000","last_became_peered":"2024-04-25T10:13:16.055961+0000","last_unstale":"2024-04-25T10:13:16.064465+0000","last_undegraded":"2024-04-25T10:13:16.064465+0000","last_fullsized":"2024-04-25T10:13:16.064465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043272+0000","last_change":"2024-04-25T10:13:16.043272+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043272+0000","last_undegraded":"2024-04-25T10:13:16.043272+0000","last_fullsized":"2024-04-25T10:13:16.043272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065693+0000","last_change":"2024-04-25T10:13:16.058726+0000","last_active":"2024-04-25T10:13:16.065693+0000","last_peered":"2024-04-25T10:13:16.065693+0000","last_clean":"2024-04-25T10:13:16.065693+0000","last_became_active":"2024-04-25T10:13:16.058253+0000","last_became_peered":"2024-04-25T10:13:16.058253+0000","last_unstale":"2024-04-25T10:13:16.065693+0000","last_undegraded":"2024-04-25T10:13:16.065693+0000","last_fullsized":"2024-04-25T10:13:16.065693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044193+0000","last_change":"2024-04-25T10:13:16.044193+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044193+0000","last_undegraded":"2024-04-25T10:13:16.044193+0000","last_fullsized":"2024-04-25T10:13:16.044193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066628+0000","last_change":"2024-04-25T10:13:16.062295+0000","last_active":"2024-04-25T10:13:16.066628+0000","last_peered":"2024-04-25T10:13:16.066628+0000","last_clean":"2024-04-25T10:13:16.066628+0000","last_became_active":"2024-04-25T10:13:16.061270+0000","last_became_peered":"2024-04-25T10:13:16.061270+0000","last_unstale":"2024-04-25T10:13:16.066628+0000","last_undegraded":"2024-04-25T10:13:16.066628+0000","last_fullsized":"2024-04-25T10:13:16.066628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:11.953599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042049+0000","last_change":"2024-04-25T10:13:16.042049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042049+0000","last_undegraded":"2024-04-25T10:13:16.042049+0000","last_fullsized":"2024-04-25T10:13:16.042049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063994+0000","last_change":"2024-04-25T10:13:16.055470+0000","last_active":"2024-04-25T10:13:16.063994+0000","last_peered":"2024-04-25T10:13:16.063994+0000","last_clean":"2024-04-25T10:13:16.063994+0000","last_became_active":"2024-04-25T10:13:16.055137+0000","last_became_peered":"2024-04-25T10:13:16.055137+0000","last_unstale":"2024-04-25T10:13:16.063994+0000","last_undegraded":"2024-04-25T10:13:16.063994+0000","last_fullsized":"2024-04-25T10:13:16.063994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:10.904343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046361+0000","last_change":"2024-04-25T10:13:16.046361+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046361+0000","last_undegraded":"2024-04-25T10:13:16.046361+0000","last_fullsized":"2024-04-25T10:13:16.046361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064028+0000","last_change":"2024-04-25T10:13:16.057838+0000","last_active":"2024-04-25T10:13:16.064028+0000","last_peered":"2024-04-25T10:13:16.064028+0000","last_clean":"2024-04-25T10:13:16.064028+0000","last_became_active":"2024-04-25T10:13:16.057476+0000","last_became_peered":"2024-04-25T10:13:16.057476+0000","last_unstale":"2024-04-25T10:13:16.064028+0000","last_undegraded":"2024-04-25T10:13:16.064028+0000","last_fullsized":"2024-04-25T10:13:16.064028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:39.436447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046108+0000","last_change":"2024-04-25T10:13:16.046108+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046108+0000","last_undegraded":"2024-04-25T10:13:16.046108+0000","last_fullsized":"2024-04-25T10:13:16.046108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070376+0000","last_change":"2024-04-25T10:13:16.065980+0000","last_active":"2024-04-25T10:13:16.070376+0000","last_peered":"2024-04-25T10:13:16.070376+0000","last_clean":"2024-04-25T10:13:16.070376+0000","last_became_active":"2024-04-25T10:13:16.065704+0000","last_became_peered":"2024-04-25T10:13:16.065704+0000","last_unstale":"2024-04-25T10:13:16.070376+0000","last_undegraded":"2024-04-25T10:13:16.070376+0000","last_fullsized":"2024-04-25T10:13:16.070376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046010+0000","last_change":"2024-04-25T10:13:16.046010+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046010+0000","last_undegraded":"2024-04-25T10:13:16.046010+0000","last_fullsized":"2024-04-25T10:13:16.046010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.069159+0000","last_change":"2024-04-25T10:13:16.065367+0000","last_active":"2024-04-25T10:13:16.069159+0000","last_peered":"2024-04-25T10:13:16.069159+0000","last_clean":"2024-04-25T10:13:16.069159+0000","last_became_active":"2024-04-25T10:13:16.065198+0000","last_became_peered":"2024-04-25T10:13:16.065198+0000","last_unstale":"2024-04-25T10:13:16.069159+0000","last_undegraded":"2024-04-25T10:13:16.069159+0000","last_fullsized":"2024-04-25T10:13:16.069159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051599+0000","last_change":"2024-04-25T10:13:16.051599+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051599+0000","last_undegraded":"2024-04-25T10:13:16.051599+0000","last_fullsized":"2024-04-25T10:13:16.051599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066646+0000","last_change":"2024-04-25T10:13:16.062433+0000","last_active":"2024-04-25T10:13:16.066646+0000","last_peered":"2024-04-25T10:13:16.066646+0000","last_clean":"2024-04-25T10:13:16.066646+0000","last_became_active":"2024-04-25T10:13:16.062227+0000","last_became_peered":"2024-04-25T10:13:16.062227+0000","last_unstale":"2024-04-25T10:13:16.066646+0000","last_undegraded":"2024-04-25T10:13:16.066646+0000","last_fullsized":"2024-04-25T10:13:16.066646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:33.301434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051355+0000","last_change":"2024-04-25T10:13:16.051355+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051355+0000","last_undegraded":"2024-04-25T10:13:16.051355+0000","last_fullsized":"2024-04-25T10:13:16.051355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063289+0000","last_change":"2024-04-25T10:13:16.057718+0000","last_active":"2024-04-25T10:13:16.063289+0000","last_peered":"2024-04-25T10:13:16.063289+0000","last_clean":"2024-04-25T10:13:16.063289+0000","last_became_active":"2024-04-25T10:13:16.057007+0000","last_became_peered":"2024-04-25T10:13:16.057007+0000","last_unstale":"2024-04-25T10:13:16.063289+0000","last_undegraded":"2024-04-25T10:13:16.063289+0000","last_fullsized":"2024-04-25T10:13:16.063289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046460+0000","last_change":"2024-04-25T10:13:16.046460+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046460+0000","last_undegraded":"2024-04-25T10:13:16.046460+0000","last_fullsized":"2024-04-25T10:13:16.046460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.056100+0000","last_change":"2024-04-25T10:13:16.051248+0000","last_active":"2024-04-25T10:13:16.056100+0000","last_peered":"2024-04-25T10:13:16.056100+0000","last_clean":"2024-04-25T10:13:16.056100+0000","last_became_active":"2024-04-25T10:13:16.050951+0000","last_became_peered":"2024-04-25T10:13:16.050951+0000","last_unstale":"2024-04-25T10:13:16.056100+0000","last_undegraded":"2024-04-25T10:13:16.056100+0000","last_fullsized":"2024-04-25T10:13:16.056100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043744+0000","last_change":"2024-04-25T10:13:16.043744+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043744+0000","last_undegraded":"2024-04-25T10:13:16.043744+0000","last_fullsized":"2024-04-25T10:13:16.043744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070444+0000","last_change":"2024-04-25T10:13:16.066644+0000","last_active":"2024-04-25T10:13:16.070444+0000","last_peered":"2024-04-25T10:13:16.070444+0000","last_clean":"2024-04-25T10:13:16.070444+0000","last_became_active":"2024-04-25T10:13:16.064572+0000","last_became_peered":"2024-04-25T10:13:16.064572+0000","last_unstale":"2024-04-25T10:13:16.070444+0000","last_undegraded":"2024-04-25T10:13:16.070444+0000","last_fullsized":"2024-04-25T10:13:16.070444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048564+0000","last_change":"2024-04-25T10:13:16.048564+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048564+0000","last_undegraded":"2024-04-25T10:13:16.048564+0000","last_fullsized":"2024-04-25T10:13:16.048564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.061896+0000","last_change":"2024-04-25T10:13:16.057038+0000","last_active":"2024-04-25T10:13:16.061896+0000","last_peered":"2024-04-25T10:13:16.061896+0000","last_clean":"2024-04-25T10:13:16.061896+0000","last_became_active":"2024-04-25T10:13:16.056758+0000","last_became_peered":"2024-04-25T10:13:16.056758+0000","last_unstale":"2024-04-25T10:13:16.061896+0000","last_undegraded":"2024-04-25T10:13:16.061896+0000","last_fullsized":"2024-04-25T10:13:16.061896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050996+0000","last_change":"2024-04-25T10:13:16.050996+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050996+0000","last_undegraded":"2024-04-25T10:13:16.050996+0000","last_fullsized":"2024-04-25T10:13:16.050996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060591+0000","last_change":"2024-04-25T10:13:16.055447+0000","last_active":"2024-04-25T10:13:16.060591+0000","last_peered":"2024-04-25T10:13:16.060591+0000","last_clean":"2024-04-25T10:13:16.060591+0000","last_became_active":"2024-04-25T10:13:16.055157+0000","last_became_peered":"2024-04-25T10:13:16.055157+0000","last_unstale":"2024-04-25T10:13:16.060591+0000","last_undegraded":"2024-04-25T10:13:16.060591+0000","last_fullsized":"2024-04-25T10:13:16.060591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:13.025816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047880+0000","last_change":"2024-04-25T10:13:16.047880+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047880+0000","last_undegraded":"2024-04-25T10:13:16.047880+0000","last_fullsized":"2024-04-25T10:13:16.047880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.054639+0000","last_change":"2024-04-25T10:13:16.049162+0000","last_active":"2024-04-25T10:13:16.054639+0000","last_peered":"2024-04-25T10:13:16.054639+0000","last_clean":"2024-04-25T10:13:16.054639+0000","last_became_active":"2024-04-25T10:13:16.048699+0000","last_became_peered":"2024-04-25T10:13:16.048699+0000","last_unstale":"2024-04-25T10:13:16.054639+0000","last_undegraded":"2024-04-25T10:13:16.054639+0000","last_fullsized":"2024-04-25T10:13:16.054639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043077+0000","last_change":"2024-04-25T10:13:16.043077+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043077+0000","last_undegraded":"2024-04-25T10:13:16.043077+0000","last_fullsized":"2024-04-25T10:13:16.043077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.055963+0000","last_change":"2024-04-25T10:13:16.052633+0000","last_active":"2024-04-25T10:13:16.055963+0000","last_peered":"2024-04-25T10:13:16.055963+0000","last_clean":"2024-04-25T10:13:16.055963+0000","last_became_active":"2024-04-25T10:13:16.052308+0000","last_became_peered":"2024-04-25T10:13:16.052308+0000","last_unstale":"2024-04-25T10:13:16.055963+0000","last_undegraded":"2024-04-25T10:13:16.055963+0000","last_fullsized":"2024-04-25T10:13:16.055963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:01.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043595+0000","last_change":"2024-04-25T10:13:16.043595+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043595+0000","last_undegraded":"2024-04-25T10:13:16.043595+0000","last_fullsized":"2024-04-25T10:13:16.043595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070420+0000","last_change":"2024-04-25T10:13:16.065524+0000","last_active":"2024-04-25T10:13:16.070420+0000","last_peered":"2024-04-25T10:13:16.070420+0000","last_clean":"2024-04-25T10:13:16.070420+0000","last_became_active":"2024-04-25T10:13:16.065056+0000","last_became_peered":"2024-04-25T10:13:16.065056+0000","last_unstale":"2024-04-25T10:13:16.070420+0000","last_undegraded":"2024-04-25T10:13:16.070420+0000","last_fullsized":"2024-04-25T10:13:16.070420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:18.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.052506+0000","last_change":"2024-04-25T10:13:16.052506+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.052506+0000","last_undegraded":"2024-04-25T10:13:16.052506+0000","last_fullsized":"2024-04-25T10:13:16.052506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.072389+0000","last_change":"2024-04-25T10:13:16.066995+0000","last_active":"2024-04-25T10:13:16.072389+0000","last_peered":"2024-04-25T10:13:16.072389+0000","last_clean":"2024-04-25T10:13:16.072389+0000","last_became_active":"2024-04-25T10:13:16.066686+0000","last_became_peered":"2024-04-25T10:13:16.066686+0000","last_unstale":"2024-04-25T10:13:16.072389+0000","last_undegraded":"2024-04-25T10:13:16.072389+0000","last_fullsized":"2024-04-25T10:13:16.072389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.059855+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:13:16.059855+0000","last_peered":"2024-04-25T10:13:16.059855+0000","last_clean":"2024-04-25T10:13:16.059855+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:13:16.059855+0000","last_undegraded":"2024-04-25T10:13:16.059855+0000","last_fullsized":"2024-04-25T10:13:16.059855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:21.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:21.598 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:21.601 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.670+0000 7fdc24800640 1 Processor -- start 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.670+0000 7fdc24800640 1 -- start start 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc24800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdc1c0d0040 0x7fdc1c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc24800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c071050 con 0x7fdc1c1550b0 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc24800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c15d040 con 0x7fdc1c15b740 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc24800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0712f0 con 0x7fdc1c0d0040 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc21400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdc1c0d0040 0x7fdc1c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc21400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdc1c0d0040 0x7fdc1c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44920/0 (socket says 172.21.15.87:44920) 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc21400640 1 -- 172.21.15.87:0/1408094768 learned_addr learned my addr 172.21.15.87:0/1408094768 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc21400640 1 -- 172.21.15.87:0/1408094768 >> v1:172.21.15.169:6789/0 conn(0x7fdc1c15b740 legacy=0x7fdc1c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc21400640 1 -- 172.21.15.87:0/1408094768 >> v1:172.21.15.87:6789/0 conn(0x7fdc1c1550b0 legacy=0x7fdc1c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc21400640 1 -- 172.21.15.87:0/1408094768 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0715d0 con 0x7fdc1c0d0040 2024-04-25T10:13:21.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.674+0000 7fdc20a00640 1 -- 172.21.15.87:0/1408094768 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684877444 0 0) 0x7fdc1c071050 con 0x7fdc1c1550b0 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc21400640 1 --2- 172.21.15.87:0/1408094768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c15b200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00bb10 tx=0x7fdc0c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa84732c113b8ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc20a00640 1 -- 172.21.15.87:0/1408094768 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0b4070 con 0x7fdc1c0d0040 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc20a00640 1 -- 172.21.15.87:0/1408094768 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c0a7b90 con 0x7fdc1c0d0040 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc20a00640 1 -- 172.21.15.87:0/1408094768 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0af440 con 0x7fdc1c0d0040 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- 172.21.15.87:0/1408094768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 msgr2=0x7fdc1c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 --2- 172.21.15.87:0/1408094768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c15b200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00bb10 tx=0x7fdc0c01ebc0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- 172.21.15.87:0/1408094768 shutdown_connections 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 --2- 172.21.15.87:0/1408094768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c15b200 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- 172.21.15.87:0/1408094768 >> 172.21.15.87:0/1408094768 conn(0x7fdc1c066b70 msgr2=0x7fdc1c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- 172.21.15.87:0/1408094768 shutdown_connections 2024-04-25T10:13:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- 172.21.15.87:0/1408094768 wait complete. 2024-04-25T10:13:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 Processor -- start 2024-04-25T10:13:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- start start 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc1c1550b0 0x7fdc1c14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc1c15b740 0x7fdc1c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c15bd10 con 0x7fdc1c0d0040 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c15bb90 con 0x7fdc1c15b740 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.678+0000 7fdc24800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c06c090 con 0x7fdc1c1550b0 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44922/0 (socket says 172.21.15.87:44922) 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 -- 172.21.15.87:0/691808916 learned_addr learned my addr 172.21.15.87:0/691808916 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 -- 172.21.15.87:0/691808916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc1c1550b0 msgr2=0x7fdc1c14c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc1c1550b0 0x7fdc1c14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 -- 172.21.15.87:0/691808916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc1c15b740 msgr2=0x7fdc1c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc1c15b740 0x7fdc1c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 -- 172.21.15.87:0/691808916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c14b710 con 0x7fdc1c0d0040 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc21e00640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c142e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c500 tx=0x7fdc1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc16000640 1 -- 172.21.15.87:0/691808916 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10017070 con 0x7fdc1c0d0040 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc16000640 1 -- 172.21.15.87:0/691808916 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10003600 con 0x7fdc1c0d0040 2024-04-25T10:13:21.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc16000640 1 -- 172.21.15.87:0/691808916 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc100039f0 con 0x7fdc1c0d0040 2024-04-25T10:13:21.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c14b9f0 con 0x7fdc1c0d0040 2024-04-25T10:13:21.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c14bfb0 con 0x7fdc1c0d0040 2024-04-25T10:13:21.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c04ef30 con 0x7fdc1c0d0040 2024-04-25T10:13:21.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc16000640 1 -- 172.21.15.87:0/691808916 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc1001a070 con 0x7fdc1c0d0040 2024-04-25T10:13:21.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc16000640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc00066f60 0x7fdc00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.682+0000 7fdc16000640 1 -- 172.21.15.87:0/691808916 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc10084450 con 0x7fdc1c0d0040 2024-04-25T10:13:21.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.686+0000 7fdc21400640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc00066f60 0x7fdc00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.686+0000 7fdc21400640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc00066f60 0x7fdc00069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0b1000 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.690+0000 7fdc16000640 1 -- 172.21.15.87:0/691808916 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc100555b0 con 0x7fdc1c0d0040 2024-04-25T10:13:21.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.838+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc1c0ceb40 con 0x7fdc00066f60 2024-04-25T10:13:21.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.842+0000 7fdc16000640 1 -- 172.21.15.87:0/691808916 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246632 (secure 0 0 0) 0x7fdc1c0ceb40 con 0x7fdc00066f60 2024-04-25T10:13:21.853 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:21.855 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc00066f60 msgr2=0x7fdc00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc00066f60 0x7fdc00069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0b1000 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 msgr2=0x7fdc1c142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c142e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c500 tx=0x7fdc1000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 shutdown_connections 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc00066f60 0x7fdc00069420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc1c15b740 0x7fdc1c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc1c1550b0 0x7fdc1c14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 --2- 172.21.15.87:0/691808916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc1c0d0040 0x7fdc1c142e00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 >> 172.21.15.87:0/691808916 conn(0x7fdc1c066b70 msgr2=0x7fdc1c0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 shutdown_connections 2024-04-25T10:13:21.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:21.846+0000 7fdc24800640 1 -- 172.21.15.87:0/691808916 wait complete. 2024-04-25T10:13:21.874 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-25T10:13:21.135987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219960,"kb_used_data":3796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754760,"statfs":{"total":773094113280,"available":772868874240,"internally_reserved":0,"allocated":3887104,"data_stored":2205195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000989"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051524+0000","last_change":"2024-04-25T10:13:16.051524+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051524+0000","last_undegraded":"2024-04-25T10:13:16.051524+0000","last_fullsized":"2024-04-25T10:13:16.051524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049178+0000","last_change":"2024-04-25T10:13:16.049178+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049178+0000","last_undegraded":"2024-04-25T10:13:16.049178+0000","last_fullsized":"2024-04-25T10:13:16.049178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042138+0000","last_change":"2024-04-25T10:13:16.042138+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042138+0000","last_undegraded":"2024-04-25T10:13:16.042138+0000","last_fullsized":"2024-04-25T10:13:16.042138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043287+0000","last_change":"2024-04-25T10:13:16.043287+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043287+0000","last_undegraded":"2024-04-25T10:13:16.043287+0000","last_fullsized":"2024-04-25T10:13:16.043287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051209+0000","last_change":"2024-04-25T10:13:16.051209+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051209+0000","last_undegraded":"2024-04-25T10:13:16.051209+0000","last_fullsized":"2024-04-25T10:13:16.051209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048910+0000","last_change":"2024-04-25T10:13:16.048910+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048910+0000","last_undegraded":"2024-04-25T10:13:16.048910+0000","last_fullsized":"2024-04-25T10:13:16.048910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050714+0000","last_change":"2024-04-25T10:13:16.050714+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050714+0000","last_undegraded":"2024-04-25T10:13:16.050714+0000","last_fullsized":"2024-04-25T10:13:16.050714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053901+0000","last_change":"2024-04-25T10:13:16.053901+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053901+0000","last_undegraded":"2024-04-25T10:13:16.053901+0000","last_fullsized":"2024-04-25T10:13:16.053901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044137+0000","last_change":"2024-04-25T10:13:16.044137+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044137+0000","last_undegraded":"2024-04-25T10:13:16.044137+0000","last_fullsized":"2024-04-25T10:13:16.044137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043363+0000","last_change":"2024-04-25T10:13:16.043363+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043363+0000","last_undegraded":"2024-04-25T10:13:16.043363+0000","last_fullsized":"2024-04-25T10:13:16.043363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.054965+0000","last_change":"2024-04-25T10:13:16.054965+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.054965+0000","last_undegraded":"2024-04-25T10:13:16.054965+0000","last_fullsized":"2024-04-25T10:13:16.054965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043819+0000","last_change":"2024-04-25T10:13:16.043819+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043819+0000","last_undegraded":"2024-04-25T10:13:16.043819+0000","last_fullsized":"2024-04-25T10:13:16.043819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046803+0000","last_change":"2024-04-25T10:13:16.046803+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046803+0000","last_undegraded":"2024-04-25T10:13:16.046803+0000","last_fullsized":"2024-04-25T10:13:16.046803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048465+0000","last_change":"2024-04-25T10:13:16.048465+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048465+0000","last_undegraded":"2024-04-25T10:13:16.048465+0000","last_fullsized":"2024-04-25T10:13:16.048465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041761+0000","last_change":"2024-04-25T10:13:16.041761+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041761+0000","last_undegraded":"2024-04-25T10:13:16.041761+0000","last_fullsized":"2024-04-25T10:13:16.041761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051289+0000","last_change":"2024-04-25T10:13:16.051289+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051289+0000","last_undegraded":"2024-04-25T10:13:16.051289+0000","last_fullsized":"2024-04-25T10:13:16.051289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.057085+0000","last_change":"2024-04-25T10:13:16.057085+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.057085+0000","last_undegraded":"2024-04-25T10:13:16.057085+0000","last_fullsized":"2024-04-25T10:13:16.057085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048752+0000","last_change":"2024-04-25T10:13:16.048752+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048752+0000","last_undegraded":"2024-04-25T10:13:16.048752+0000","last_fullsized":"2024-04-25T10:13:16.048752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050699+0000","last_change":"2024-04-25T10:13:16.050699+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050699+0000","last_undegraded":"2024-04-25T10:13:16.050699+0000","last_fullsized":"2024-04-25T10:13:16.050699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050639+0000","last_change":"2024-04-25T10:13:16.050639+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050639+0000","last_undegraded":"2024-04-25T10:13:16.050639+0000","last_fullsized":"2024-04-25T10:13:16.050639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043315+0000","last_change":"2024-04-25T10:13:16.043315+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043315+0000","last_undegraded":"2024-04-25T10:13:16.043315+0000","last_fullsized":"2024-04-25T10:13:16.043315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042542+0000","last_change":"2024-04-25T10:13:16.042542+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042542+0000","last_undegraded":"2024-04-25T10:13:16.042542+0000","last_fullsized":"2024-04-25T10:13:16.042542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047344+0000","last_change":"2024-04-25T10:13:16.047344+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047344+0000","last_undegraded":"2024-04-25T10:13:16.047344+0000","last_fullsized":"2024-04-25T10:13:16.047344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041284+0000","last_change":"2024-04-25T10:13:16.041284+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041284+0000","last_undegraded":"2024-04-25T10:13:16.041284+0000","last_fullsized":"2024-04-25T10:13:16.041284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043168+0000","last_change":"2024-04-25T10:13:16.043168+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043168+0000","last_undegraded":"2024-04-25T10:13:16.043168+0000","last_fullsized":"2024-04-25T10:13:16.043168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048337+0000","last_change":"2024-04-25T10:13:16.048337+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048337+0000","last_undegraded":"2024-04-25T10:13:16.048337+0000","last_fullsized":"2024-04-25T10:13:16.048337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044692+0000","last_change":"2024-04-25T10:13:16.044692+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044692+0000","last_undegraded":"2024-04-25T10:13:16.044692+0000","last_fullsized":"2024-04-25T10:13:16.044692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043349+0000","last_change":"2024-04-25T10:13:16.043349+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043349+0000","last_undegraded":"2024-04-25T10:13:16.043349+0000","last_fullsized":"2024-04-25T10:13:16.043349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042136+0000","last_change":"2024-04-25T10:13:16.042136+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042136+0000","last_undegraded":"2024-04-25T10:13:16.042136+0000","last_fullsized":"2024-04-25T10:13:16.042136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050728+0000","last_change":"2024-04-25T10:13:16.050728+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050728+0000","last_undegraded":"2024-04-25T10:13:16.050728+0000","last_fullsized":"2024-04-25T10:13:16.050728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050320+0000","last_change":"2024-04-25T10:13:16.050320+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050320+0000","last_undegraded":"2024-04-25T10:13:16.050320+0000","last_fullsized":"2024-04-25T10:13:16.050320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053491+0000","last_change":"2024-04-25T10:13:16.053491+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053491+0000","last_undegraded":"2024-04-25T10:13:16.053491+0000","last_fullsized":"2024-04-25T10:13:16.053491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047586+0000","last_change":"2024-04-25T10:13:16.047586+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047586+0000","last_undegraded":"2024-04-25T10:13:16.047586+0000","last_fullsized":"2024-04-25T10:13:16.047586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063422+0000","last_change":"2024-04-25T10:13:16.059863+0000","last_active":"2024-04-25T10:13:16.063422+0000","last_peered":"2024-04-25T10:13:16.063422+0000","last_clean":"2024-04-25T10:13:16.063422+0000","last_became_active":"2024-04-25T10:13:16.059568+0000","last_became_peered":"2024-04-25T10:13:16.059568+0000","last_unstale":"2024-04-25T10:13:16.063422+0000","last_undegraded":"2024-04-25T10:13:16.063422+0000","last_fullsized":"2024-04-25T10:13:16.063422+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049779+0000","last_change":"2024-04-25T10:13:16.049779+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049779+0000","last_undegraded":"2024-04-25T10:13:16.049779+0000","last_fullsized":"2024-04-25T10:13:16.049779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073269+0000","last_change":"2024-04-25T10:13:16.067950+0000","last_active":"2024-04-25T10:13:16.073269+0000","last_peered":"2024-04-25T10:13:16.073269+0000","last_clean":"2024-04-25T10:13:16.073269+0000","last_became_active":"2024-04-25T10:13:16.067755+0000","last_became_peered":"2024-04-25T10:13:16.067755+0000","last_unstale":"2024-04-25T10:13:16.073269+0000","last_undegraded":"2024-04-25T10:13:16.073269+0000","last_fullsized":"2024-04-25T10:13:16.073269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044196+0000","last_change":"2024-04-25T10:13:16.044196+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044196+0000","last_undegraded":"2024-04-25T10:13:16.044196+0000","last_fullsized":"2024-04-25T10:13:16.044196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064555+0000","last_change":"2024-04-25T10:13:16.058185+0000","last_active":"2024-04-25T10:13:16.064555+0000","last_peered":"2024-04-25T10:13:16.064555+0000","last_clean":"2024-04-25T10:13:16.064555+0000","last_became_active":"2024-04-25T10:13:16.057713+0000","last_became_peered":"2024-04-25T10:13:16.057713+0000","last_unstale":"2024-04-25T10:13:16.064555+0000","last_undegraded":"2024-04-25T10:13:16.064555+0000","last_fullsized":"2024-04-25T10:13:16.064555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:08.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043030+0000","last_change":"2024-04-25T10:13:16.043030+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043030+0000","last_undegraded":"2024-04-25T10:13:16.043030+0000","last_fullsized":"2024-04-25T10:13:16.043030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060374+0000","last_change":"2024-04-25T10:13:16.058303+0000","last_active":"2024-04-25T10:13:16.060374+0000","last_peered":"2024-04-25T10:13:16.060374+0000","last_clean":"2024-04-25T10:13:16.060374+0000","last_became_active":"2024-04-25T10:13:16.057794+0000","last_became_peered":"2024-04-25T10:13:16.057794+0000","last_unstale":"2024-04-25T10:13:16.060374+0000","last_undegraded":"2024-04-25T10:13:16.060374+0000","last_fullsized":"2024-04-25T10:13:16.060374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050240+0000","last_change":"2024-04-25T10:13:16.050240+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050240+0000","last_undegraded":"2024-04-25T10:13:16.050240+0000","last_fullsized":"2024-04-25T10:13:16.050240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064044+0000","last_change":"2024-04-25T10:13:16.058396+0000","last_active":"2024-04-25T10:13:16.064044+0000","last_peered":"2024-04-25T10:13:16.064044+0000","last_clean":"2024-04-25T10:13:16.064044+0000","last_became_active":"2024-04-25T10:13:16.057548+0000","last_became_peered":"2024-04-25T10:13:16.057548+0000","last_unstale":"2024-04-25T10:13:16.064044+0000","last_undegraded":"2024-04-25T10:13:16.064044+0000","last_fullsized":"2024-04-25T10:13:16.064044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:44.890367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041080+0000","last_change":"2024-04-25T10:13:16.041080+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041080+0000","last_undegraded":"2024-04-25T10:13:16.041080+0000","last_fullsized":"2024-04-25T10:13:16.041080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063961+0000","last_change":"2024-04-25T10:13:16.056396+0000","last_active":"2024-04-25T10:13:16.063961+0000","last_peered":"2024-04-25T10:13:16.063961+0000","last_clean":"2024-04-25T10:13:16.063961+0000","last_became_active":"2024-04-25T10:13:16.056119+0000","last_became_peered":"2024-04-25T10:13:16.056119+0000","last_unstale":"2024-04-25T10:13:16.063961+0000","last_undegraded":"2024-04-25T10:13:16.063961+0000","last_fullsized":"2024-04-25T10:13:16.063961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046223+0000","last_change":"2024-04-25T10:13:16.046223+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046223+0000","last_undegraded":"2024-04-25T10:13:16.046223+0000","last_fullsized":"2024-04-25T10:13:16.046223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066147+0000","last_change":"2024-04-25T10:13:16.063791+0000","last_active":"2024-04-25T10:13:16.066147+0000","last_peered":"2024-04-25T10:13:16.066147+0000","last_clean":"2024-04-25T10:13:16.066147+0000","last_became_active":"2024-04-25T10:13:16.063534+0000","last_became_peered":"2024-04-25T10:13:16.063534+0000","last_unstale":"2024-04-25T10:13:16.066147+0000","last_undegraded":"2024-04-25T10:13:16.066147+0000","last_fullsized":"2024-04-25T10:13:16.066147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:25.604894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043697+0000","last_change":"2024-04-25T10:13:16.043697+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043697+0000","last_undegraded":"2024-04-25T10:13:16.043697+0000","last_fullsized":"2024-04-25T10:13:16.043697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066302+0000","last_change":"2024-04-25T10:13:16.061738+0000","last_active":"2024-04-25T10:13:16.066302+0000","last_peered":"2024-04-25T10:13:16.066302+0000","last_clean":"2024-04-25T10:13:16.066302+0000","last_became_active":"2024-04-25T10:13:16.061207+0000","last_became_peered":"2024-04-25T10:13:16.061207+0000","last_unstale":"2024-04-25T10:13:16.066302+0000","last_undegraded":"2024-04-25T10:13:16.066302+0000","last_fullsized":"2024-04-25T10:13:16.066302+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:27.291874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044636+0000","last_change":"2024-04-25T10:13:16.044636+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044636+0000","last_undegraded":"2024-04-25T10:13:16.044636+0000","last_fullsized":"2024-04-25T10:13:16.044636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073358+0000","last_change":"2024-04-25T10:13:16.067967+0000","last_active":"2024-04-25T10:13:16.073358+0000","last_peered":"2024-04-25T10:13:16.073358+0000","last_clean":"2024-04-25T10:13:16.073358+0000","last_became_active":"2024-04-25T10:13:16.067640+0000","last_became_peered":"2024-04-25T10:13:16.067640+0000","last_unstale":"2024-04-25T10:13:16.073358+0000","last_undegraded":"2024-04-25T10:13:16.073358+0000","last_fullsized":"2024-04-25T10:13:16.073358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047672+0000","last_change":"2024-04-25T10:13:16.047672+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047672+0000","last_undegraded":"2024-04-25T10:13:16.047672+0000","last_fullsized":"2024-04-25T10:13:16.047672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060814+0000","last_change":"2024-04-25T10:13:16.058919+0000","last_active":"2024-04-25T10:13:16.060814+0000","last_peered":"2024-04-25T10:13:16.060814+0000","last_clean":"2024-04-25T10:13:16.060814+0000","last_became_active":"2024-04-25T10:13:16.058553+0000","last_became_peered":"2024-04-25T10:13:16.058553+0000","last_unstale":"2024-04-25T10:13:16.060814+0000","last_undegraded":"2024-04-25T10:13:16.060814+0000","last_fullsized":"2024-04-25T10:13:16.060814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:26.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047789+0000","last_change":"2024-04-25T10:13:16.047789+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047789+0000","last_undegraded":"2024-04-25T10:13:16.047789+0000","last_fullsized":"2024-04-25T10:13:16.047789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.068891+0000","last_change":"2024-04-25T10:13:16.064906+0000","last_active":"2024-04-25T10:13:16.068891+0000","last_peered":"2024-04-25T10:13:16.068891+0000","last_clean":"2024-04-25T10:13:16.068891+0000","last_became_active":"2024-04-25T10:13:16.064562+0000","last_became_peered":"2024-04-25T10:13:16.064562+0000","last_unstale":"2024-04-25T10:13:16.068891+0000","last_undegraded":"2024-04-25T10:13:16.068891+0000","last_fullsized":"2024-04-25T10:13:16.068891+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042594+0000","last_change":"2024-04-25T10:13:16.042594+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042594+0000","last_undegraded":"2024-04-25T10:13:16.042594+0000","last_fullsized":"2024-04-25T10:13:16.042594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064513+0000","last_change":"2024-04-25T10:13:16.056267+0000","last_active":"2024-04-25T10:13:16.064513+0000","last_peered":"2024-04-25T10:13:16.064513+0000","last_clean":"2024-04-25T10:13:16.064513+0000","last_became_active":"2024-04-25T10:13:16.055804+0000","last_became_peered":"2024-04-25T10:13:16.055804+0000","last_unstale":"2024-04-25T10:13:16.064513+0000","last_undegraded":"2024-04-25T10:13:16.064513+0000","last_fullsized":"2024-04-25T10:13:16.064513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043721+0000","last_change":"2024-04-25T10:13:16.043721+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043721+0000","last_undegraded":"2024-04-25T10:13:16.043721+0000","last_fullsized":"2024-04-25T10:13:16.043721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066328+0000","last_change":"2024-04-25T10:13:16.061578+0000","last_active":"2024-04-25T10:13:16.066328+0000","last_peered":"2024-04-25T10:13:16.066328+0000","last_clean":"2024-04-25T10:13:16.066328+0000","last_became_active":"2024-04-25T10:13:16.061283+0000","last_became_peered":"2024-04-25T10:13:16.061283+0000","last_unstale":"2024-04-25T10:13:16.066328+0000","last_undegraded":"2024-04-25T10:13:16.066328+0000","last_fullsized":"2024-04-25T10:13:16.066328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:13.032134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046150+0000","last_change":"2024-04-25T10:13:16.046150+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046150+0000","last_undegraded":"2024-04-25T10:13:16.046150+0000","last_fullsized":"2024-04-25T10:13:16.046150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065293+0000","last_change":"2024-04-25T10:13:16.061513+0000","last_active":"2024-04-25T10:13:16.065293+0000","last_peered":"2024-04-25T10:13:16.065293+0000","last_clean":"2024-04-25T10:13:16.065293+0000","last_became_active":"2024-04-25T10:13:16.061216+0000","last_became_peered":"2024-04-25T10:13:16.061216+0000","last_unstale":"2024-04-25T10:13:16.065293+0000","last_undegraded":"2024-04-25T10:13:16.065293+0000","last_fullsized":"2024-04-25T10:13:16.065293+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053049+0000","last_change":"2024-04-25T10:13:16.053049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053049+0000","last_undegraded":"2024-04-25T10:13:16.053049+0000","last_fullsized":"2024-04-25T10:13:16.053049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066124+0000","last_change":"2024-04-25T10:13:16.063235+0000","last_active":"2024-04-25T10:13:16.066124+0000","last_peered":"2024-04-25T10:13:16.066124+0000","last_clean":"2024-04-25T10:13:16.066124+0000","last_became_active":"2024-04-25T10:13:16.062820+0000","last_became_peered":"2024-04-25T10:13:16.062820+0000","last_unstale":"2024-04-25T10:13:16.066124+0000","last_undegraded":"2024-04-25T10:13:16.066124+0000","last_fullsized":"2024-04-25T10:13:16.066124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:09.118013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049640+0000","last_change":"2024-04-25T10:13:16.049640+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049640+0000","last_undegraded":"2024-04-25T10:13:16.049640+0000","last_fullsized":"2024-04-25T10:13:16.049640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064465+0000","last_change":"2024-04-25T10:13:16.057566+0000","last_active":"2024-04-25T10:13:16.064465+0000","last_peered":"2024-04-25T10:13:16.064465+0000","last_clean":"2024-04-25T10:13:16.064465+0000","last_became_active":"2024-04-25T10:13:16.055961+0000","last_became_peered":"2024-04-25T10:13:16.055961+0000","last_unstale":"2024-04-25T10:13:16.064465+0000","last_undegraded":"2024-04-25T10:13:16.064465+0000","last_fullsized":"2024-04-25T10:13:16.064465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043272+0000","last_change":"2024-04-25T10:13:16.043272+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043272+0000","last_undegraded":"2024-04-25T10:13:16.043272+0000","last_fullsized":"2024-04-25T10:13:16.043272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065693+0000","last_change":"2024-04-25T10:13:16.058726+0000","last_active":"2024-04-25T10:13:16.065693+0000","last_peered":"2024-04-25T10:13:16.065693+0000","last_clean":"2024-04-25T10:13:16.065693+0000","last_became_active":"2024-04-25T10:13:16.058253+0000","last_became_peered":"2024-04-25T10:13:16.058253+0000","last_unstale":"2024-04-25T10:13:16.065693+0000","last_undegraded":"2024-04-25T10:13:16.065693+0000","last_fullsized":"2024-04-25T10:13:16.065693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044193+0000","last_change":"2024-04-25T10:13:16.044193+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044193+0000","last_undegraded":"2024-04-25T10:13:16.044193+0000","last_fullsized":"2024-04-25T10:13:16.044193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066628+0000","last_change":"2024-04-25T10:13:16.062295+0000","last_active":"2024-04-25T10:13:16.066628+0000","last_peered":"2024-04-25T10:13:16.066628+0000","last_clean":"2024-04-25T10:13:16.066628+0000","last_became_active":"2024-04-25T10:13:16.061270+0000","last_became_peered":"2024-04-25T10:13:16.061270+0000","last_unstale":"2024-04-25T10:13:16.066628+0000","last_undegraded":"2024-04-25T10:13:16.066628+0000","last_fullsized":"2024-04-25T10:13:16.066628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:11.953599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042049+0000","last_change":"2024-04-25T10:13:16.042049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042049+0000","last_undegraded":"2024-04-25T10:13:16.042049+0000","last_fullsized":"2024-04-25T10:13:16.042049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063994+0000","last_change":"2024-04-25T10:13:16.055470+0000","last_active":"2024-04-25T10:13:16.063994+0000","last_peered":"2024-04-25T10:13:16.063994+0000","last_clean":"2024-04-25T10:13:16.063994+0000","last_became_active":"2024-04-25T10:13:16.055137+0000","last_became_peered":"2024-04-25T10:13:16.055137+0000","last_unstale":"2024-04-25T10:13:16.063994+0000","last_undegraded":"2024-04-25T10:13:16.063994+0000","last_fullsized":"2024-04-25T10:13:16.063994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:10.904343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046361+0000","last_change":"2024-04-25T10:13:16.046361+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046361+0000","last_undegraded":"2024-04-25T10:13:16.046361+0000","last_fullsized":"2024-04-25T10:13:16.046361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064028+0000","last_change":"2024-04-25T10:13:16.057838+0000","last_active":"2024-04-25T10:13:16.064028+0000","last_peered":"2024-04-25T10:13:16.064028+0000","last_clean":"2024-04-25T10:13:16.064028+0000","last_became_active":"2024-04-25T10:13:16.057476+0000","last_became_peered":"2024-04-25T10:13:16.057476+0000","last_unstale":"2024-04-25T10:13:16.064028+0000","last_undegraded":"2024-04-25T10:13:16.064028+0000","last_fullsized":"2024-04-25T10:13:16.064028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:39.436447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046108+0000","last_change":"2024-04-25T10:13:16.046108+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046108+0000","last_undegraded":"2024-04-25T10:13:16.046108+0000","last_fullsized":"2024-04-25T10:13:16.046108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070376+0000","last_change":"2024-04-25T10:13:16.065980+0000","last_active":"2024-04-25T10:13:16.070376+0000","last_peered":"2024-04-25T10:13:16.070376+0000","last_clean":"2024-04-25T10:13:16.070376+0000","last_became_active":"2024-04-25T10:13:16.065704+0000","last_became_peered":"2024-04-25T10:13:16.065704+0000","last_unstale":"2024-04-25T10:13:16.070376+0000","last_undegraded":"2024-04-25T10:13:16.070376+0000","last_fullsized":"2024-04-25T10:13:16.070376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046010+0000","last_change":"2024-04-25T10:13:16.046010+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046010+0000","last_undegraded":"2024-04-25T10:13:16.046010+0000","last_fullsized":"2024-04-25T10:13:16.046010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.069159+0000","last_change":"2024-04-25T10:13:16.065367+0000","last_active":"2024-04-25T10:13:16.069159+0000","last_peered":"2024-04-25T10:13:16.069159+0000","last_clean":"2024-04-25T10:13:16.069159+0000","last_became_active":"2024-04-25T10:13:16.065198+0000","last_became_peered":"2024-04-25T10:13:16.065198+0000","last_unstale":"2024-04-25T10:13:16.069159+0000","last_undegraded":"2024-04-25T10:13:16.069159+0000","last_fullsized":"2024-04-25T10:13:16.069159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051599+0000","last_change":"2024-04-25T10:13:16.051599+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051599+0000","last_undegraded":"2024-04-25T10:13:16.051599+0000","last_fullsized":"2024-04-25T10:13:16.051599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066646+0000","last_change":"2024-04-25T10:13:16.062433+0000","last_active":"2024-04-25T10:13:16.066646+0000","last_peered":"2024-04-25T10:13:16.066646+0000","last_clean":"2024-04-25T10:13:16.066646+0000","last_became_active":"2024-04-25T10:13:16.062227+0000","last_became_peered":"2024-04-25T10:13:16.062227+0000","last_unstale":"2024-04-25T10:13:16.066646+0000","last_undegraded":"2024-04-25T10:13:16.066646+0000","last_fullsized":"2024-04-25T10:13:16.066646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:33.301434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051355+0000","last_change":"2024-04-25T10:13:16.051355+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051355+0000","last_undegraded":"2024-04-25T10:13:16.051355+0000","last_fullsized":"2024-04-25T10:13:16.051355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063289+0000","last_change":"2024-04-25T10:13:16.057718+0000","last_active":"2024-04-25T10:13:16.063289+0000","last_peered":"2024-04-25T10:13:16.063289+0000","last_clean":"2024-04-25T10:13:16.063289+0000","last_became_active":"2024-04-25T10:13:16.057007+0000","last_became_peered":"2024-04-25T10:13:16.057007+0000","last_unstale":"2024-04-25T10:13:16.063289+0000","last_undegraded":"2024-04-25T10:13:16.063289+0000","last_fullsized":"2024-04-25T10:13:16.063289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046460+0000","last_change":"2024-04-25T10:13:16.046460+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046460+0000","last_undegraded":"2024-04-25T10:13:16.046460+0000","last_fullsized":"2024-04-25T10:13:16.046460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.056100+0000","last_change":"2024-04-25T10:13:16.051248+0000","last_active":"2024-04-25T10:13:16.056100+0000","last_peered":"2024-04-25T10:13:16.056100+0000","last_clean":"2024-04-25T10:13:16.056100+0000","last_became_active":"2024-04-25T10:13:16.050951+0000","last_became_peered":"2024-04-25T10:13:16.050951+0000","last_unstale":"2024-04-25T10:13:16.056100+0000","last_undegraded":"2024-04-25T10:13:16.056100+0000","last_fullsized":"2024-04-25T10:13:16.056100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043744+0000","last_change":"2024-04-25T10:13:16.043744+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043744+0000","last_undegraded":"2024-04-25T10:13:16.043744+0000","last_fullsized":"2024-04-25T10:13:16.043744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070444+0000","last_change":"2024-04-25T10:13:16.066644+0000","last_active":"2024-04-25T10:13:16.070444+0000","last_peered":"2024-04-25T10:13:16.070444+0000","last_clean":"2024-04-25T10:13:16.070444+0000","last_became_active":"2024-04-25T10:13:16.064572+0000","last_became_peered":"2024-04-25T10:13:16.064572+0000","last_unstale":"2024-04-25T10:13:16.070444+0000","last_undegraded":"2024-04-25T10:13:16.070444+0000","last_fullsized":"2024-04-25T10:13:16.070444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048564+0000","last_change":"2024-04-25T10:13:16.048564+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048564+0000","last_undegraded":"2024-04-25T10:13:16.048564+0000","last_fullsized":"2024-04-25T10:13:16.048564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.061896+0000","last_change":"2024-04-25T10:13:16.057038+0000","last_active":"2024-04-25T10:13:16.061896+0000","last_peered":"2024-04-25T10:13:16.061896+0000","last_clean":"2024-04-25T10:13:16.061896+0000","last_became_active":"2024-04-25T10:13:16.056758+0000","last_became_peered":"2024-04-25T10:13:16.056758+0000","last_unstale":"2024-04-25T10:13:16.061896+0000","last_undegraded":"2024-04-25T10:13:16.061896+0000","last_fullsized":"2024-04-25T10:13:16.061896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050996+0000","last_change":"2024-04-25T10:13:16.050996+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050996+0000","last_undegraded":"2024-04-25T10:13:16.050996+0000","last_fullsized":"2024-04-25T10:13:16.050996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060591+0000","last_change":"2024-04-25T10:13:16.055447+0000","last_active":"2024-04-25T10:13:16.060591+0000","last_peered":"2024-04-25T10:13:16.060591+0000","last_clean":"2024-04-25T10:13:16.060591+0000","last_became_active":"2024-04-25T10:13:16.055157+0000","last_became_peered":"2024-04-25T10:13:16.055157+0000","last_unstale":"2024-04-25T10:13:16.060591+0000","last_undegraded":"2024-04-25T10:13:16.060591+0000","last_fullsized":"2024-04-25T10:13:16.060591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:13.025816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047880+0000","last_change":"2024-04-25T10:13:16.047880+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047880+0000","last_undegraded":"2024-04-25T10:13:16.047880+0000","last_fullsized":"2024-04-25T10:13:16.047880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.054639+0000","last_change":"2024-04-25T10:13:16.049162+0000","last_active":"2024-04-25T10:13:16.054639+0000","last_peered":"2024-04-25T10:13:16.054639+0000","last_clean":"2024-04-25T10:13:16.054639+0000","last_became_active":"2024-04-25T10:13:16.048699+0000","last_became_peered":"2024-04-25T10:13:16.048699+0000","last_unstale":"2024-04-25T10:13:16.054639+0000","last_undegraded":"2024-04-25T10:13:16.054639+0000","last_fullsized":"2024-04-25T10:13:16.054639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043077+0000","last_change":"2024-04-25T10:13:16.043077+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043077+0000","last_undegraded":"2024-04-25T10:13:16.043077+0000","last_fullsized":"2024-04-25T10:13:16.043077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.055963+0000","last_change":"2024-04-25T10:13:16.052633+0000","last_active":"2024-04-25T10:13:16.055963+0000","last_peered":"2024-04-25T10:13:16.055963+0000","last_clean":"2024-04-25T10:13:16.055963+0000","last_became_active":"2024-04-25T10:13:16.052308+0000","last_became_peered":"2024-04-25T10:13:16.052308+0000","last_unstale":"2024-04-25T10:13:16.055963+0000","last_undegraded":"2024-04-25T10:13:16.055963+0000","last_fullsized":"2024-04-25T10:13:16.055963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:01.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043595+0000","last_change":"2024-04-25T10:13:16.043595+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043595+0000","last_undegraded":"2024-04-25T10:13:16.043595+0000","last_fullsized":"2024-04-25T10:13:16.043595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070420+0000","last_change":"2024-04-25T10:13:16.065524+0000","last_active":"2024-04-25T10:13:16.070420+0000","last_peered":"2024-04-25T10:13:16.070420+0000","last_clean":"2024-04-25T10:13:16.070420+0000","last_became_active":"2024-04-25T10:13:16.065056+0000","last_became_peered":"2024-04-25T10:13:16.065056+0000","last_unstale":"2024-04-25T10:13:16.070420+0000","last_undegraded":"2024-04-25T10:13:16.070420+0000","last_fullsized":"2024-04-25T10:13:16.070420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:18.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.052506+0000","last_change":"2024-04-25T10:13:16.052506+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.052506+0000","last_undegraded":"2024-04-25T10:13:16.052506+0000","last_fullsized":"2024-04-25T10:13:16.052506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.072389+0000","last_change":"2024-04-25T10:13:16.066995+0000","last_active":"2024-04-25T10:13:16.072389+0000","last_peered":"2024-04-25T10:13:16.072389+0000","last_clean":"2024-04-25T10:13:16.072389+0000","last_became_active":"2024-04-25T10:13:16.066686+0000","last_became_peered":"2024-04-25T10:13:16.066686+0000","last_unstale":"2024-04-25T10:13:16.072389+0000","last_undegraded":"2024-04-25T10:13:16.072389+0000","last_fullsized":"2024-04-25T10:13:16.072389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.059855+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:13:16.059855+0000","last_peered":"2024-04-25T10:13:16.059855+0000","last_clean":"2024-04-25T10:13:16.059855+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:13:16.059855+0000","last_undegraded":"2024-04-25T10:13:16.059855+0000","last_fullsized":"2024-04-25T10:13:16.059855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:21.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-25T10:13:21.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.051524+0000', 'last_change': '2024-04-25T10:13:16.051524+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.051524+0000', 'last_undegraded': '2024-04-25T10:13:16.051524+0000', 'last_fullsized': '2024-04-25T10:13:16.051524+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.049178+0000', 'last_change': '2024-04-25T10:13:16.049178+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.049178+0000', 'last_undegraded': '2024-04-25T10:13:16.049178+0000', 'last_fullsized': '2024-04-25T10:13:16.049178+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.042138+0000', 'last_change': '2024-04-25T10:13:16.042138+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.042138+0000', 'last_undegraded': '2024-04-25T10:13:16.042138+0000', 'last_fullsized': '2024-04-25T10:13:16.042138+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043287+0000', 'last_change': '2024-04-25T10:13:16.043287+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043287+0000', 'last_undegraded': '2024-04-25T10:13:16.043287+0000', 'last_fullsized': '2024-04-25T10:13:16.043287+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-25T10:13:21.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.051209+0000', 'last_change': '2024-04-25T10:13:16.051209+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.051209+0000', 'last_undegraded': '2024-04-25T10:13:16.051209+0000', 'last_fullsized': '2024-04-25T10:13:16.051209+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.048910+0000', 'last_change': '2024-04-25T10:13:16.048910+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.048910+0000', 'last_undegraded': '2024-04-25T10:13:16.048910+0000', 'last_fullsized': '2024-04-25T10:13:16.048910+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.050714+0000', 'last_change': '2024-04-25T10:13:16.050714+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.050714+0000', 'last_undegraded': '2024-04-25T10:13:16.050714+0000', 'last_fullsized': '2024-04-25T10:13:16.050714+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.053901+0000', 'last_change': '2024-04-25T10:13:16.053901+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.053901+0000', 'last_undegraded': '2024-04-25T10:13:16.053901+0000', 'last_fullsized': '2024-04-25T10:13:16.053901+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.044137+0000', 'last_change': '2024-04-25T10:13:16.044137+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.044137+0000', 'last_undegraded': '2024-04-25T10:13:16.044137+0000', 'last_fullsized': '2024-04-25T10:13:16.044137+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-25T10:13:21.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043363+0000', 'last_change': '2024-04-25T10:13:16.043363+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043363+0000', 'last_undegraded': '2024-04-25T10:13:16.043363+0000', 'last_fullsized': '2024-04-25T10:13:16.043363+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.054965+0000', 'last_change': '2024-04-25T10:13:16.054965+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.054965+0000', 'last_undegraded': '2024-04-25T10:13:16.054965+0000', 'last_fullsized': '2024-04-25T10:13:16.054965+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043819+0000', 'last_change': '2024-04-25T10:13:16.043819+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043819+0000', 'last_undegraded': '2024-04-25T10:13:16.043819+0000', 'last_fullsized': '2024-04-25T10:13:16.043819+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.046803+0000', 'last_change': '2024-04-25T10:13:16.046803+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.046803+0000', 'last_undegraded': '2024-04-25T10:13:16.046803+0000', 'last_fullsized': '2024-04-25T10:13:16.046803+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.048465+0000', 'last_change': '2024-04-25T10:13:16.048465+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.048465+0000', 'last_undegraded': '2024-04-25T10:13:16.048465+0000', 'last_fullsized': '2024-04-25T10:13:16.048465+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.041761+0000', 'last_change': '2024-04-25T10:13:16.041761+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.041761+0000', 'last_undegraded': '2024-04-25T10:13:16.041761+0000', 'last_fullsized': '2024-04-25T10:13:16.041761+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.051289+0000', 'last_change': '2024-04-25T10:13:16.051289+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.051289+0000', 'last_undegraded': '2024-04-25T10:13:16.051289+0000', 'last_fullsized': '2024-04-25T10:13:16.051289+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.057085+0000', 'last_change': '2024-04-25T10:13:16.057085+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.057085+0000', 'last_undegraded': '2024-04-25T10:13:16.057085+0000', 'last_fullsized': '2024-04-25T10:13:16.057085+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.048752+0000', 'last_change': '2024-04-25T10:13:16.048752+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.048752+0000', 'last_undegraded': '2024-04-25T10:13:16.048752+0000', 'last_fullsized': '2024-04-25T10:13:16.048752+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.050699+0000', 'last_change': '2024-04-25T10:13:16.050699+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.050699+0000', 'last_undegraded': '2024-04-25T10:13:16.050699+0000', 'last_fullsized': '2024-04-25T10:13:16.050699+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.050639+0000', 'last_change': '2024-04-25T10:13:16.050639+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.050639+0000', 'last_undegraded': '2024-04-25T10:13:16.050639+0000', 'last_fullsized': '2024-04-25T10:13:16.050639+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-25T10:13:21.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043315+0000', 'last_change': '2024-04-25T10:13:16.043315+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043315+0000', 'last_undegraded': '2024-04-25T10:13:16.043315+0000', 'last_fullsized': '2024-04-25T10:13:16.043315+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-25T10:13:21.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.042542+0000', 'last_change': '2024-04-25T10:13:16.042542+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.042542+0000', 'last_undegraded': '2024-04-25T10:13:16.042542+0000', 'last_fullsized': '2024-04-25T10:13:16.042542+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-25T10:13:21.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.047344+0000', 'last_change': '2024-04-25T10:13:16.047344+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.047344+0000', 'last_undegraded': '2024-04-25T10:13:16.047344+0000', 'last_fullsized': '2024-04-25T10:13:16.047344+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:13:21.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-25T10:13:21.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.041284+0000', 'last_change': '2024-04-25T10:13:16.041284+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.041284+0000', 'last_undegraded': '2024-04-25T10:13:16.041284+0000', 'last_fullsized': '2024-04-25T10:13:16.041284+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043168+0000', 'last_change': '2024-04-25T10:13:16.043168+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043168+0000', 'last_undegraded': '2024-04-25T10:13:16.043168+0000', 'last_fullsized': '2024-04-25T10:13:16.043168+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.048337+0000', 'last_change': '2024-04-25T10:13:16.048337+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.048337+0000', 'last_undegraded': '2024-04-25T10:13:16.048337+0000', 'last_fullsized': '2024-04-25T10:13:16.048337+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.044692+0000', 'last_change': '2024-04-25T10:13:16.044692+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.044692+0000', 'last_undegraded': '2024-04-25T10:13:16.044692+0000', 'last_fullsized': '2024-04-25T10:13:16.044692+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043349+0000', 'last_change': '2024-04-25T10:13:16.043349+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043349+0000', 'last_undegraded': '2024-04-25T10:13:16.043349+0000', 'last_fullsized': '2024-04-25T10:13:16.043349+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.042136+0000', 'last_change': '2024-04-25T10:13:16.042136+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.042136+0000', 'last_undegraded': '2024-04-25T10:13:16.042136+0000', 'last_fullsized': '2024-04-25T10:13:16.042136+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.050728+0000', 'last_change': '2024-04-25T10:13:16.050728+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.050728+0000', 'last_undegraded': '2024-04-25T10:13:16.050728+0000', 'last_fullsized': '2024-04-25T10:13:16.050728+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.050320+0000', 'last_change': '2024-04-25T10:13:16.050320+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.050320+0000', 'last_undegraded': '2024-04-25T10:13:16.050320+0000', 'last_fullsized': '2024-04-25T10:13:16.050320+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.053491+0000', 'last_change': '2024-04-25T10:13:16.053491+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.053491+0000', 'last_undegraded': '2024-04-25T10:13:16.053491+0000', 'last_fullsized': '2024-04-25T10:13:16.053491+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.047586+0000', 'last_change': '2024-04-25T10:13:16.047586+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.047586+0000', 'last_undegraded': '2024-04-25T10:13:16.047586+0000', 'last_fullsized': '2024-04-25T10:13:16.047586+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.049779+0000', 'last_change': '2024-04-25T10:13:16.049779+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.049779+0000', 'last_undegraded': '2024-04-25T10:13:16.049779+0000', 'last_fullsized': '2024-04-25T10:13:16.049779+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-25T10:13:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.044196+0000', 'last_change': '2024-04-25T10:13:16.044196+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.044196+0000', 'last_undegraded': '2024-04-25T10:13:16.044196+0000', 'last_fullsized': '2024-04-25T10:13:16.044196+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043030+0000', 'last_change': '2024-04-25T10:13:16.043030+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043030+0000', 'last_undegraded': '2024-04-25T10:13:16.043030+0000', 'last_fullsized': '2024-04-25T10:13:16.043030+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.050240+0000', 'last_change': '2024-04-25T10:13:16.050240+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.050240+0000', 'last_undegraded': '2024-04-25T10:13:16.050240+0000', 'last_fullsized': '2024-04-25T10:13:16.050240+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.041080+0000', 'last_change': '2024-04-25T10:13:16.041080+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.041080+0000', 'last_undegraded': '2024-04-25T10:13:16.041080+0000', 'last_fullsized': '2024-04-25T10:13:16.041080+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.046223+0000', 'last_change': '2024-04-25T10:13:16.046223+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.046223+0000', 'last_undegraded': '2024-04-25T10:13:16.046223+0000', 'last_fullsized': '2024-04-25T10:13:16.046223+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-25T10:13:21.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043697+0000', 'last_change': '2024-04-25T10:13:16.043697+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043697+0000', 'last_undegraded': '2024-04-25T10:13:16.043697+0000', 'last_fullsized': '2024-04-25T10:13:16.043697+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.044636+0000', 'last_change': '2024-04-25T10:13:16.044636+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.044636+0000', 'last_undegraded': '2024-04-25T10:13:16.044636+0000', 'last_fullsized': '2024-04-25T10:13:16.044636+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.047672+0000', 'last_change': '2024-04-25T10:13:16.047672+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.047672+0000', 'last_undegraded': '2024-04-25T10:13:16.047672+0000', 'last_fullsized': '2024-04-25T10:13:16.047672+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.047789+0000', 'last_change': '2024-04-25T10:13:16.047789+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.047789+0000', 'last_undegraded': '2024-04-25T10:13:16.047789+0000', 'last_fullsized': '2024-04-25T10:13:16.047789+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.042594+0000', 'last_change': '2024-04-25T10:13:16.042594+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.042594+0000', 'last_undegraded': '2024-04-25T10:13:16.042594+0000', 'last_fullsized': '2024-04-25T10:13:16.042594+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043721+0000', 'last_change': '2024-04-25T10:13:16.043721+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043721+0000', 'last_undegraded': '2024-04-25T10:13:16.043721+0000', 'last_fullsized': '2024-04-25T10:13:16.043721+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.046150+0000', 'last_change': '2024-04-25T10:13:16.046150+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.046150+0000', 'last_undegraded': '2024-04-25T10:13:16.046150+0000', 'last_fullsized': '2024-04-25T10:13:16.046150+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.053049+0000', 'last_change': '2024-04-25T10:13:16.053049+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.053049+0000', 'last_undegraded': '2024-04-25T10:13:16.053049+0000', 'last_fullsized': '2024-04-25T10:13:16.053049+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.049640+0000', 'last_change': '2024-04-25T10:13:16.049640+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.049640+0000', 'last_undegraded': '2024-04-25T10:13:16.049640+0000', 'last_fullsized': '2024-04-25T10:13:16.049640+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043272+0000', 'last_change': '2024-04-25T10:13:16.043272+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043272+0000', 'last_undegraded': '2024-04-25T10:13:16.043272+0000', 'last_fullsized': '2024-04-25T10:13:16.043272+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.044193+0000', 'last_change': '2024-04-25T10:13:16.044193+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.044193+0000', 'last_undegraded': '2024-04-25T10:13:16.044193+0000', 'last_fullsized': '2024-04-25T10:13:16.044193+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.042049+0000', 'last_change': '2024-04-25T10:13:16.042049+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.042049+0000', 'last_undegraded': '2024-04-25T10:13:16.042049+0000', 'last_fullsized': '2024-04-25T10:13:16.042049+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:13:21.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-25T10:13:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.046361+0000', 'last_change': '2024-04-25T10:13:16.046361+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.046361+0000', 'last_undegraded': '2024-04-25T10:13:16.046361+0000', 'last_fullsized': '2024-04-25T10:13:16.046361+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-25T10:13:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.046108+0000', 'last_change': '2024-04-25T10:13:16.046108+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.046108+0000', 'last_undegraded': '2024-04-25T10:13:16.046108+0000', 'last_fullsized': '2024-04-25T10:13:16.046108+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T10:13:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.046010+0000', 'last_change': '2024-04-25T10:13:16.046010+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.046010+0000', 'last_undegraded': '2024-04-25T10:13:16.046010+0000', 'last_fullsized': '2024-04-25T10:13:16.046010+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-25T10:13:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.051599+0000', 'last_change': '2024-04-25T10:13:16.051599+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.051599+0000', 'last_undegraded': '2024-04-25T10:13:16.051599+0000', 'last_fullsized': '2024-04-25T10:13:16.051599+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.051355+0000', 'last_change': '2024-04-25T10:13:16.051355+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.051355+0000', 'last_undegraded': '2024-04-25T10:13:16.051355+0000', 'last_fullsized': '2024-04-25T10:13:16.051355+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.046460+0000', 'last_change': '2024-04-25T10:13:16.046460+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.046460+0000', 'last_undegraded': '2024-04-25T10:13:16.046460+0000', 'last_fullsized': '2024-04-25T10:13:16.046460+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043744+0000', 'last_change': '2024-04-25T10:13:16.043744+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043744+0000', 'last_undegraded': '2024-04-25T10:13:16.043744+0000', 'last_fullsized': '2024-04-25T10:13:16.043744+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.048564+0000', 'last_change': '2024-04-25T10:13:16.048564+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.048564+0000', 'last_undegraded': '2024-04-25T10:13:16.048564+0000', 'last_fullsized': '2024-04-25T10:13:16.048564+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.050996+0000', 'last_change': '2024-04-25T10:13:16.050996+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.050996+0000', 'last_undegraded': '2024-04-25T10:13:16.050996+0000', 'last_fullsized': '2024-04-25T10:13:16.050996+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:13:21.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-25T10:13:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.047880+0000', 'last_change': '2024-04-25T10:13:16.047880+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.047880+0000', 'last_undegraded': '2024-04-25T10:13:16.047880+0000', 'last_fullsized': '2024-04-25T10:13:16.047880+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:13:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-25T10:13:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043077+0000', 'last_change': '2024-04-25T10:13:16.043077+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043077+0000', 'last_undegraded': '2024-04-25T10:13:16.043077+0000', 'last_fullsized': '2024-04-25T10:13:16.043077+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:13:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-25T10:13:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.043595+0000', 'last_change': '2024-04-25T10:13:16.043595+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.043595+0000', 'last_undegraded': '2024-04-25T10:13:16.043595+0000', 'last_fullsized': '2024-04-25T10:13:16.043595+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:13:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-25T10:13:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:13:16.052506+0000', 'last_change': '2024-04-25T10:13:16.052506+0000', 'last_active': '2024-04-25T10:13:16.029897+0000', 'last_peered': '2024-04-25T10:13:16.029897+0000', 'last_clean': '2024-04-25T10:13:16.029897+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:13:16.052506+0000', 'last_undegraded': '2024-04-25T10:13:16.052506+0000', 'last_fullsized': '2024-04-25T10:13:16.052506+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'last_clean_scrub_stamp': '2024-04-25T10:13:16.029897+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:13:21.893 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:22.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdf5000640 1 Processor -- start 2024-04-25T10:13:22.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdf5000640 1 -- start start 2024-04-25T10:13:22.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdf5000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcdf015bc40 0x7fcdf006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdf5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdf5000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00b2770 con 0x7fcdf00cda10 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdf5000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf006c670 con 0x7fcdf015bc40 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdf5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf006c950 con 0x7fcdf00cf930 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdede00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcdf015bc40 0x7fcdf006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdede00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcdf015bc40 0x7fcdf006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44930/0 (socket says 172.21.15.87:44930) 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdede00640 1 -- 172.21.15.87:0/1746383971 learned_addr learned my addr 172.21.15.87:0/1746383971 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdee800640 1 --2- 172.21.15.87:0/1746383971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdede00640 1 -- 172.21.15.87:0/1746383971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 msgr2=0x7fcdf00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdede00640 1 --2- 172.21.15.87:0/1746383971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdede00640 1 -- 172.21.15.87:0/1746383971 >> v1:172.21.15.169:6789/0 conn(0x7fcdf00cda10 legacy=0x7fcdf00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.046+0000 7fcdede00640 1 -- 172.21.15.87:0/1746383971 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf006cc30 con 0x7fcdf015bc40 2024-04-25T10:13:22.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdede00640 1 --2- 172.21.15.87:0/1746383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf006c090 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009870 tx=0x7fcde001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a36d47d8296f780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdeca00640 1 -- 172.21.15.87:0/1746383971 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde00b0070 con 0x7fcdf015bc40 2024-04-25T10:13:22.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdeca00640 1 -- 172.21.15.87:0/1746383971 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde00ab600 con 0x7fcdf015bc40 2024-04-25T10:13:22.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdeca00640 1 -- 172.21.15.87:0/1746383971 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde00ab9d0 con 0x7fcdf015bc40 2024-04-25T10:13:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 -- 172.21.15.87:0/1746383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 msgr2=0x7fcdf006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 --2- 172.21.15.87:0/1746383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf006c090 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009870 tx=0x7fcde001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 -- 172.21.15.87:0/1746383971 shutdown_connections 2024-04-25T10:13:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 --2- 172.21.15.87:0/1746383971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 --2- 172.21.15.87:0/1746383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf006c090 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 -- 172.21.15.87:0/1746383971 >> 172.21.15.87:0/1746383971 conn(0x7fcdf0066b70 msgr2=0x7fcdf0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 -- 172.21.15.87:0/1746383971 shutdown_connections 2024-04-25T10:13:22.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.050+0000 7fcdf5000640 1 -- 172.21.15.87:0/1746383971 wait complete. 2024-04-25T10:13:22.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 Processor -- start 2024-04-25T10:13:22.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 -- start start 2024-04-25T10:13:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cda10 0x7fcdf0071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b3890 con 0x7fcdf015bc40 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b3710 con 0x7fcdf00cda10 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b3a10 con 0x7fcdf00cf930 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf00706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44936/0 (socket says 172.21.15.87:44936) 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 -- 172.21.15.87:0/3805422672 learned_addr learned my addr 172.21.15.87:0/3805422672 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdede00640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cda10 0x7fcdf0071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcded400640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 -- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 msgr2=0x7fcdf014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf014e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 -- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cda10 msgr2=0x7fcdf0071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cda10 0x7fcdf0071f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 -- 172.21.15.87:0/3805422672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0070e80 con 0x7fcdf015bc40 2024-04-25T10:13:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdee800640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf00706e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcde8007e90 tx=0x7fcde800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcde6000640 1 -- 172.21.15.87:0/3805422672 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8014420 con 0x7fcdf015bc40 2024-04-25T10:13:22.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcde6000640 1 -- 172.21.15.87:0/3805422672 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8019070 con 0x7fcdf015bc40 2024-04-25T10:13:22.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf0071160 con 0x7fcdf015bc40 2024-04-25T10:13:22.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcde6000640 1 -- 172.21.15.87:0/3805422672 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8014800 con 0x7fcdf015bc40 2024-04-25T10:13:22.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf0071720 con 0x7fcdf015bc40 2024-04-25T10:13:22.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.054+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf004ef30 con 0x7fcdf015bc40 2024-04-25T10:13:22.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.058+0000 7fcde6000640 1 -- 172.21.15.87:0/3805422672 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcde80149a0 con 0x7fcdf015bc40 2024-04-25T10:13:22.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.058+0000 7fcde6000640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdd0066f60 0x7fcdd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.058+0000 7fcde6000640 1 -- 172.21.15.87:0/3805422672 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde8085710 con 0x7fcdf015bc40 2024-04-25T10:13:22.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.058+0000 7fcdede00640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdd0066f60 0x7fcdd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.058+0000 7fcdede00640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdd0066f60 0x7fcdd0069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ad000 tx=0x7fcde00a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.062+0000 7fcde6000640 1 -- 172.21.15.87:0/3805422672 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcde80560c0 con 0x7fcdf015bc40 2024-04-25T10:13:22.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcdf006d960 con 0x7fcdf015bc40 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcde6000640 1 -- 172.21.15.87:0/3805422672 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1581 (secure 0 0 0) 0x7fcde805a160 con 0x7fcdf015bc40 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":84,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225239040,"bytes_avail":772868874240,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5127}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-25T10:13:13.134778+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdd0066f60 msgr2=0x7fcdd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdd0066f60 0x7fcdd0069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ad000 tx=0x7fcde00a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 msgr2=0x7fcdf00706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf00706e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcde8007e90 tx=0x7fcde800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 shutdown_connections 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdd0066f60 0x7fcdd0069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf015bc40 0x7fcdf00706e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00cf930 0x7fcdf014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 --2- 172.21.15.87:0/3805422672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cda10 0x7fcdf0071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 >> 172.21.15.87:0/3805422672 conn(0x7fcdf0066b70 msgr2=0x7fcdf015ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 shutdown_connections 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.274+0000 7fcdf5000640 1 -- 172.21.15.87:0/3805422672 wait complete. 2024-04-25T10:13:22.302 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:13:22.303 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:22.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd97f800640 1 Processor -- start 2024-04-25T10:13:22.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd97f800640 1 -- start start 2024-04-25T10:13:22.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd97f800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9780d07b0 0x7fd97806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd97f800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9780ccaa0 con 0x7fd9780d0350 2024-04-25T10:13:22.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd97f800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9780ccd80 con 0x7fd97815df60 2024-04-25T10:13:22.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd97f800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97815e340 con 0x7fd9780d07b0 2024-04-25T10:13:22.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd9780d0350 0x7fd97806aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56732/0 (socket says 172.21.15.87:56732) 2024-04-25T10:13:22.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977e00640 1 -- 172.21.15.87:0/3341102416 learned_addr learned my addr 172.21.15.87:0/3341102416 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757745522 0 0) 0x7fd9780ccaa0 con 0x7fd9780d0350 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd97d800640 1 --2- 172.21.15.87:0/3341102416 >> v2:172.21.15.169:3300/0 conn(0x7fd9780d07b0 0x7fd97806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd95c003530 con 0x7fd9780d0350 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951595499 0 0) 0x7fd95c003530 con 0x7fd9780d0350 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 >> v2:172.21.15.169:3300/0 conn(0x7fd9780d07b0 msgr2=0x7fd97806b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 --2- 172.21.15.87:0/3341102416 >> v2:172.21.15.169:3300/0 conn(0x7fd9780d07b0 0x7fd97806b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 >> v1:172.21.15.169:6789/0 conn(0x7fd97815df60 legacy=0x7fd9780cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780cd060 con 0x7fd9780d0350 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd964001680 con 0x7fd9780d0350 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.430+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 >> v1:172.21.15.87:6789/0 conn(0x7fd9780d0350 legacy=0x7fd97806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd95c0039c0 0x7fd95c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c004810 0x7fd95c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd95c0051d0 0x7fd95c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c00c140 con 0x7fd95c0051d0 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c00c2f0 con 0x7fd95c004810 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c00c4f0 con 0x7fd95c0039c0 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd95c0051d0 0x7fd95c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977e00640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c004810 0x7fd95c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97ce00640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd95c0039c0 0x7fd95c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 -- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd95c0039c0 msgr2=0x7fd95c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd95c0039c0 0x7fd95c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 -- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c004810 msgr2=0x7fd95c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c004810 0x7fd95c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 -- 172.21.15.87:0/3341102416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c003530 con 0x7fd95c0051d0 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 -- 172.21.15.87:0/3341102416 reap_dead start 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd97d800640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd95c0051d0 0x7fd95c00b9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd970009ae0 tx=0x7fd97001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2b765d9623c8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700096b0 con 0x7fd95c0051d0 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd970004020 con 0x7fd95c0051d0 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.434+0000 7fd977400640 1 -- 172.21.15.87:0/3341102416 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970004410 con 0x7fd95c0051d0 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- 172.21.15.87:0/3341102416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd95c0051d0 msgr2=0x7fd95c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd95c0051d0 0x7fd95c00b9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd970009ae0 tx=0x7fd97001cab0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- 172.21.15.87:0/3341102416 shutdown_connections 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 --2- 172.21.15.87:0/3341102416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd95c0051d0 0x7fd95c00b9d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- 172.21.15.87:0/3341102416 >> 172.21.15.87:0/3341102416 conn(0x7fd9780d1910 msgr2=0x7fd9780d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- 172.21.15.87:0/3341102416 shutdown_connections 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- 172.21.15.87:0/3341102416 wait complete. 2024-04-25T10:13:22.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 Processor -- start 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- start start 2024-04-25T10:13:22.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c0051d0 0x7fd9781f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780c89e0 0x7fd9781f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9780c7480 0x7fd9780c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780cd620 con 0x7fd9780c7480 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780d0e70 con 0x7fd95c0051d0 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780d0ff0 con 0x7fd9780c89e0 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9780c7480 0x7fd9780c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9780c7480 0x7fd9780c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44946/0 (socket says 172.21.15.87:44946) 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 -- 172.21.15.87:0/3137516728 learned_addr learned my addr 172.21.15.87:0/3137516728 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd977e00640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780c89e0 0x7fd9781f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97ce00640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c0051d0 0x7fd9781f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 -- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780c89e0 msgr2=0x7fd9781f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780c89e0 0x7fd9781f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 -- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c0051d0 msgr2=0x7fd9781f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c0051d0 0x7fd9781f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd97d800640 1 -- 172.21.15.87:0/3137516728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9781f4660 con 0x7fd9780c7480 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.438+0000 7fd977e00640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780c89e0 0x7fd9781f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd97d800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9780c7480 0x7fd9780c7910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd97001c8e0 tx=0x7fd95c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd974c00640 1 -- 172.21.15.87:0/3137516728 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700b2070 con 0x7fd9780c7480 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd974c00640 1 -- 172.21.15.87:0/3137516728 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9700bb070 con 0x7fd9780c7480 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd974c00640 1 -- 172.21.15.87:0/3137516728 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700096b0 con 0x7fd9780c7480 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9780c8ea0 con 0x7fd9780c7480 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9780c9460 con 0x7fd9780c7480 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97804ef30 con 0x7fd9780c7480 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd974c00640 1 -- 172.21.15.87:0/3137516728 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd970003a60 con 0x7fd9780c7480 2024-04-25T10:13:22.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd974c00640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd958066f60 0x7fd958069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd974c00640 1 -- 172.21.15.87:0/3137516728 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd97011ecf0 con 0x7fd9780c7480 2024-04-25T10:13:22.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.442+0000 7fd97ce00640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd958066f60 0x7fd958069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.446+0000 7fd97ce00640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd958066f60 0x7fd958069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd968000f90 tx=0x7fd968009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.446+0000 7fd974c00640 1 -- 172.21.15.87:0/3137516728 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9781f4660 con 0x7fd9780c7480 2024-04-25T10:13:22.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.598+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd978161360 con 0x7fd958066f60 2024-04-25T10:13:22.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.602+0000 7fd974c00640 1 -- 172.21.15.87:0/3137516728 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246632 (secure 0 0 0) 0x7fd978161360 con 0x7fd958066f60 2024-04-25T10:13:22.613 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd958066f60 msgr2=0x7fd958069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd958066f60 0x7fd958069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd968000f90 tx=0x7fd968009040 comp rx=0 tx=0).stop 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9780c7480 msgr2=0x7fd9780c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9780c7480 0x7fd9780c7910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd97001c8e0 tx=0x7fd95c005840 comp rx=0 tx=0).stop 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 shutdown_connections 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd958066f60 0x7fd958069420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9780c7480 0x7fd9780c7910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780c89e0 0x7fd9781f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 --2- 172.21.15.87:0/3137516728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd95c0051d0 0x7fd9781f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 >> 172.21.15.87:0/3137516728 conn(0x7fd9780d1910 msgr2=0x7fd97815d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 shutdown_connections 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:22.606+0000 7fd97f800640 1 -- 172.21.15.87:0/3137516728 wait complete. 2024-04-25T10:13:22.635 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-25T10:13:21.135987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219960,"kb_used_data":3796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754760,"statfs":{"total":773094113280,"available":772868874240,"internally_reserved":0,"allocated":3887104,"data_stored":2205195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000989"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051524+0000","last_change":"2024-04-25T10:13:16.051524+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051524+0000","last_undegraded":"2024-04-25T10:13:16.051524+0000","last_fullsized":"2024-04-25T10:13:16.051524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049178+0000","last_change":"2024-04-25T10:13:16.049178+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049178+0000","last_undegraded":"2024-04-25T10:13:16.049178+0000","last_fullsized":"2024-04-25T10:13:16.049178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042138+0000","last_change":"2024-04-25T10:13:16.042138+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042138+0000","last_undegraded":"2024-04-25T10:13:16.042138+0000","last_fullsized":"2024-04-25T10:13:16.042138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043287+0000","last_change":"2024-04-25T10:13:16.043287+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043287+0000","last_undegraded":"2024-04-25T10:13:16.043287+0000","last_fullsized":"2024-04-25T10:13:16.043287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051209+0000","last_change":"2024-04-25T10:13:16.051209+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051209+0000","last_undegraded":"2024-04-25T10:13:16.051209+0000","last_fullsized":"2024-04-25T10:13:16.051209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048910+0000","last_change":"2024-04-25T10:13:16.048910+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048910+0000","last_undegraded":"2024-04-25T10:13:16.048910+0000","last_fullsized":"2024-04-25T10:13:16.048910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050714+0000","last_change":"2024-04-25T10:13:16.050714+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050714+0000","last_undegraded":"2024-04-25T10:13:16.050714+0000","last_fullsized":"2024-04-25T10:13:16.050714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053901+0000","last_change":"2024-04-25T10:13:16.053901+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053901+0000","last_undegraded":"2024-04-25T10:13:16.053901+0000","last_fullsized":"2024-04-25T10:13:16.053901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044137+0000","last_change":"2024-04-25T10:13:16.044137+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044137+0000","last_undegraded":"2024-04-25T10:13:16.044137+0000","last_fullsized":"2024-04-25T10:13:16.044137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043363+0000","last_change":"2024-04-25T10:13:16.043363+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043363+0000","last_undegraded":"2024-04-25T10:13:16.043363+0000","last_fullsized":"2024-04-25T10:13:16.043363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.054965+0000","last_change":"2024-04-25T10:13:16.054965+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.054965+0000","last_undegraded":"2024-04-25T10:13:16.054965+0000","last_fullsized":"2024-04-25T10:13:16.054965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043819+0000","last_change":"2024-04-25T10:13:16.043819+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043819+0000","last_undegraded":"2024-04-25T10:13:16.043819+0000","last_fullsized":"2024-04-25T10:13:16.043819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046803+0000","last_change":"2024-04-25T10:13:16.046803+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046803+0000","last_undegraded":"2024-04-25T10:13:16.046803+0000","last_fullsized":"2024-04-25T10:13:16.046803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048465+0000","last_change":"2024-04-25T10:13:16.048465+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048465+0000","last_undegraded":"2024-04-25T10:13:16.048465+0000","last_fullsized":"2024-04-25T10:13:16.048465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041761+0000","last_change":"2024-04-25T10:13:16.041761+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041761+0000","last_undegraded":"2024-04-25T10:13:16.041761+0000","last_fullsized":"2024-04-25T10:13:16.041761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051289+0000","last_change":"2024-04-25T10:13:16.051289+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051289+0000","last_undegraded":"2024-04-25T10:13:16.051289+0000","last_fullsized":"2024-04-25T10:13:16.051289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.057085+0000","last_change":"2024-04-25T10:13:16.057085+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.057085+0000","last_undegraded":"2024-04-25T10:13:16.057085+0000","last_fullsized":"2024-04-25T10:13:16.057085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048752+0000","last_change":"2024-04-25T10:13:16.048752+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048752+0000","last_undegraded":"2024-04-25T10:13:16.048752+0000","last_fullsized":"2024-04-25T10:13:16.048752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050699+0000","last_change":"2024-04-25T10:13:16.050699+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050699+0000","last_undegraded":"2024-04-25T10:13:16.050699+0000","last_fullsized":"2024-04-25T10:13:16.050699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050639+0000","last_change":"2024-04-25T10:13:16.050639+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050639+0000","last_undegraded":"2024-04-25T10:13:16.050639+0000","last_fullsized":"2024-04-25T10:13:16.050639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043315+0000","last_change":"2024-04-25T10:13:16.043315+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043315+0000","last_undegraded":"2024-04-25T10:13:16.043315+0000","last_fullsized":"2024-04-25T10:13:16.043315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042542+0000","last_change":"2024-04-25T10:13:16.042542+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042542+0000","last_undegraded":"2024-04-25T10:13:16.042542+0000","last_fullsized":"2024-04-25T10:13:16.042542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047344+0000","last_change":"2024-04-25T10:13:16.047344+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047344+0000","last_undegraded":"2024-04-25T10:13:16.047344+0000","last_fullsized":"2024-04-25T10:13:16.047344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041284+0000","last_change":"2024-04-25T10:13:16.041284+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041284+0000","last_undegraded":"2024-04-25T10:13:16.041284+0000","last_fullsized":"2024-04-25T10:13:16.041284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043168+0000","last_change":"2024-04-25T10:13:16.043168+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043168+0000","last_undegraded":"2024-04-25T10:13:16.043168+0000","last_fullsized":"2024-04-25T10:13:16.043168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048337+0000","last_change":"2024-04-25T10:13:16.048337+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048337+0000","last_undegraded":"2024-04-25T10:13:16.048337+0000","last_fullsized":"2024-04-25T10:13:16.048337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044692+0000","last_change":"2024-04-25T10:13:16.044692+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044692+0000","last_undegraded":"2024-04-25T10:13:16.044692+0000","last_fullsized":"2024-04-25T10:13:16.044692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043349+0000","last_change":"2024-04-25T10:13:16.043349+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043349+0000","last_undegraded":"2024-04-25T10:13:16.043349+0000","last_fullsized":"2024-04-25T10:13:16.043349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042136+0000","last_change":"2024-04-25T10:13:16.042136+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042136+0000","last_undegraded":"2024-04-25T10:13:16.042136+0000","last_fullsized":"2024-04-25T10:13:16.042136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050728+0000","last_change":"2024-04-25T10:13:16.050728+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050728+0000","last_undegraded":"2024-04-25T10:13:16.050728+0000","last_fullsized":"2024-04-25T10:13:16.050728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050320+0000","last_change":"2024-04-25T10:13:16.050320+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050320+0000","last_undegraded":"2024-04-25T10:13:16.050320+0000","last_fullsized":"2024-04-25T10:13:16.050320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053491+0000","last_change":"2024-04-25T10:13:16.053491+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053491+0000","last_undegraded":"2024-04-25T10:13:16.053491+0000","last_fullsized":"2024-04-25T10:13:16.053491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047586+0000","last_change":"2024-04-25T10:13:16.047586+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047586+0000","last_undegraded":"2024-04-25T10:13:16.047586+0000","last_fullsized":"2024-04-25T10:13:16.047586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063422+0000","last_change":"2024-04-25T10:13:16.059863+0000","last_active":"2024-04-25T10:13:16.063422+0000","last_peered":"2024-04-25T10:13:16.063422+0000","last_clean":"2024-04-25T10:13:16.063422+0000","last_became_active":"2024-04-25T10:13:16.059568+0000","last_became_peered":"2024-04-25T10:13:16.059568+0000","last_unstale":"2024-04-25T10:13:16.063422+0000","last_undegraded":"2024-04-25T10:13:16.063422+0000","last_fullsized":"2024-04-25T10:13:16.063422+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049779+0000","last_change":"2024-04-25T10:13:16.049779+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049779+0000","last_undegraded":"2024-04-25T10:13:16.049779+0000","last_fullsized":"2024-04-25T10:13:16.049779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073269+0000","last_change":"2024-04-25T10:13:16.067950+0000","last_active":"2024-04-25T10:13:16.073269+0000","last_peered":"2024-04-25T10:13:16.073269+0000","last_clean":"2024-04-25T10:13:16.073269+0000","last_became_active":"2024-04-25T10:13:16.067755+0000","last_became_peered":"2024-04-25T10:13:16.067755+0000","last_unstale":"2024-04-25T10:13:16.073269+0000","last_undegraded":"2024-04-25T10:13:16.073269+0000","last_fullsized":"2024-04-25T10:13:16.073269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044196+0000","last_change":"2024-04-25T10:13:16.044196+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044196+0000","last_undegraded":"2024-04-25T10:13:16.044196+0000","last_fullsized":"2024-04-25T10:13:16.044196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064555+0000","last_change":"2024-04-25T10:13:16.058185+0000","last_active":"2024-04-25T10:13:16.064555+0000","last_peered":"2024-04-25T10:13:16.064555+0000","last_clean":"2024-04-25T10:13:16.064555+0000","last_became_active":"2024-04-25T10:13:16.057713+0000","last_became_peered":"2024-04-25T10:13:16.057713+0000","last_unstale":"2024-04-25T10:13:16.064555+0000","last_undegraded":"2024-04-25T10:13:16.064555+0000","last_fullsized":"2024-04-25T10:13:16.064555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:08.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043030+0000","last_change":"2024-04-25T10:13:16.043030+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043030+0000","last_undegraded":"2024-04-25T10:13:16.043030+0000","last_fullsized":"2024-04-25T10:13:16.043030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060374+0000","last_change":"2024-04-25T10:13:16.058303+0000","last_active":"2024-04-25T10:13:16.060374+0000","last_peered":"2024-04-25T10:13:16.060374+0000","last_clean":"2024-04-25T10:13:16.060374+0000","last_became_active":"2024-04-25T10:13:16.057794+0000","last_became_peered":"2024-04-25T10:13:16.057794+0000","last_unstale":"2024-04-25T10:13:16.060374+0000","last_undegraded":"2024-04-25T10:13:16.060374+0000","last_fullsized":"2024-04-25T10:13:16.060374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050240+0000","last_change":"2024-04-25T10:13:16.050240+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050240+0000","last_undegraded":"2024-04-25T10:13:16.050240+0000","last_fullsized":"2024-04-25T10:13:16.050240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064044+0000","last_change":"2024-04-25T10:13:16.058396+0000","last_active":"2024-04-25T10:13:16.064044+0000","last_peered":"2024-04-25T10:13:16.064044+0000","last_clean":"2024-04-25T10:13:16.064044+0000","last_became_active":"2024-04-25T10:13:16.057548+0000","last_became_peered":"2024-04-25T10:13:16.057548+0000","last_unstale":"2024-04-25T10:13:16.064044+0000","last_undegraded":"2024-04-25T10:13:16.064044+0000","last_fullsized":"2024-04-25T10:13:16.064044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:44.890367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.041080+0000","last_change":"2024-04-25T10:13:16.041080+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.041080+0000","last_undegraded":"2024-04-25T10:13:16.041080+0000","last_fullsized":"2024-04-25T10:13:16.041080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063961+0000","last_change":"2024-04-25T10:13:16.056396+0000","last_active":"2024-04-25T10:13:16.063961+0000","last_peered":"2024-04-25T10:13:16.063961+0000","last_clean":"2024-04-25T10:13:16.063961+0000","last_became_active":"2024-04-25T10:13:16.056119+0000","last_became_peered":"2024-04-25T10:13:16.056119+0000","last_unstale":"2024-04-25T10:13:16.063961+0000","last_undegraded":"2024-04-25T10:13:16.063961+0000","last_fullsized":"2024-04-25T10:13:16.063961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:25.646479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046223+0000","last_change":"2024-04-25T10:13:16.046223+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046223+0000","last_undegraded":"2024-04-25T10:13:16.046223+0000","last_fullsized":"2024-04-25T10:13:16.046223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066147+0000","last_change":"2024-04-25T10:13:16.063791+0000","last_active":"2024-04-25T10:13:16.066147+0000","last_peered":"2024-04-25T10:13:16.066147+0000","last_clean":"2024-04-25T10:13:16.066147+0000","last_became_active":"2024-04-25T10:13:16.063534+0000","last_became_peered":"2024-04-25T10:13:16.063534+0000","last_unstale":"2024-04-25T10:13:16.066147+0000","last_undegraded":"2024-04-25T10:13:16.066147+0000","last_fullsized":"2024-04-25T10:13:16.066147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:25.604894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043697+0000","last_change":"2024-04-25T10:13:16.043697+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043697+0000","last_undegraded":"2024-04-25T10:13:16.043697+0000","last_fullsized":"2024-04-25T10:13:16.043697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066302+0000","last_change":"2024-04-25T10:13:16.061738+0000","last_active":"2024-04-25T10:13:16.066302+0000","last_peered":"2024-04-25T10:13:16.066302+0000","last_clean":"2024-04-25T10:13:16.066302+0000","last_became_active":"2024-04-25T10:13:16.061207+0000","last_became_peered":"2024-04-25T10:13:16.061207+0000","last_unstale":"2024-04-25T10:13:16.066302+0000","last_undegraded":"2024-04-25T10:13:16.066302+0000","last_fullsized":"2024-04-25T10:13:16.066302+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:27.291874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044636+0000","last_change":"2024-04-25T10:13:16.044636+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044636+0000","last_undegraded":"2024-04-25T10:13:16.044636+0000","last_fullsized":"2024-04-25T10:13:16.044636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.073358+0000","last_change":"2024-04-25T10:13:16.067967+0000","last_active":"2024-04-25T10:13:16.073358+0000","last_peered":"2024-04-25T10:13:16.073358+0000","last_clean":"2024-04-25T10:13:16.073358+0000","last_became_active":"2024-04-25T10:13:16.067640+0000","last_became_peered":"2024-04-25T10:13:16.067640+0000","last_unstale":"2024-04-25T10:13:16.073358+0000","last_undegraded":"2024-04-25T10:13:16.073358+0000","last_fullsized":"2024-04-25T10:13:16.073358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047672+0000","last_change":"2024-04-25T10:13:16.047672+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047672+0000","last_undegraded":"2024-04-25T10:13:16.047672+0000","last_fullsized":"2024-04-25T10:13:16.047672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060814+0000","last_change":"2024-04-25T10:13:16.058919+0000","last_active":"2024-04-25T10:13:16.060814+0000","last_peered":"2024-04-25T10:13:16.060814+0000","last_clean":"2024-04-25T10:13:16.060814+0000","last_became_active":"2024-04-25T10:13:16.058553+0000","last_became_peered":"2024-04-25T10:13:16.058553+0000","last_unstale":"2024-04-25T10:13:16.060814+0000","last_undegraded":"2024-04-25T10:13:16.060814+0000","last_fullsized":"2024-04-25T10:13:16.060814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:26.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047789+0000","last_change":"2024-04-25T10:13:16.047789+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047789+0000","last_undegraded":"2024-04-25T10:13:16.047789+0000","last_fullsized":"2024-04-25T10:13:16.047789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.068891+0000","last_change":"2024-04-25T10:13:16.064906+0000","last_active":"2024-04-25T10:13:16.068891+0000","last_peered":"2024-04-25T10:13:16.068891+0000","last_clean":"2024-04-25T10:13:16.068891+0000","last_became_active":"2024-04-25T10:13:16.064562+0000","last_became_peered":"2024-04-25T10:13:16.064562+0000","last_unstale":"2024-04-25T10:13:16.068891+0000","last_undegraded":"2024-04-25T10:13:16.068891+0000","last_fullsized":"2024-04-25T10:13:16.068891+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042594+0000","last_change":"2024-04-25T10:13:16.042594+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042594+0000","last_undegraded":"2024-04-25T10:13:16.042594+0000","last_fullsized":"2024-04-25T10:13:16.042594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064513+0000","last_change":"2024-04-25T10:13:16.056267+0000","last_active":"2024-04-25T10:13:16.064513+0000","last_peered":"2024-04-25T10:13:16.064513+0000","last_clean":"2024-04-25T10:13:16.064513+0000","last_became_active":"2024-04-25T10:13:16.055804+0000","last_became_peered":"2024-04-25T10:13:16.055804+0000","last_unstale":"2024-04-25T10:13:16.064513+0000","last_undegraded":"2024-04-25T10:13:16.064513+0000","last_fullsized":"2024-04-25T10:13:16.064513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043721+0000","last_change":"2024-04-25T10:13:16.043721+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043721+0000","last_undegraded":"2024-04-25T10:13:16.043721+0000","last_fullsized":"2024-04-25T10:13:16.043721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066328+0000","last_change":"2024-04-25T10:13:16.061578+0000","last_active":"2024-04-25T10:13:16.066328+0000","last_peered":"2024-04-25T10:13:16.066328+0000","last_clean":"2024-04-25T10:13:16.066328+0000","last_became_active":"2024-04-25T10:13:16.061283+0000","last_became_peered":"2024-04-25T10:13:16.061283+0000","last_unstale":"2024-04-25T10:13:16.066328+0000","last_undegraded":"2024-04-25T10:13:16.066328+0000","last_fullsized":"2024-04-25T10:13:16.066328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:13.032134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046150+0000","last_change":"2024-04-25T10:13:16.046150+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046150+0000","last_undegraded":"2024-04-25T10:13:16.046150+0000","last_fullsized":"2024-04-25T10:13:16.046150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065293+0000","last_change":"2024-04-25T10:13:16.061513+0000","last_active":"2024-04-25T10:13:16.065293+0000","last_peered":"2024-04-25T10:13:16.065293+0000","last_clean":"2024-04-25T10:13:16.065293+0000","last_became_active":"2024-04-25T10:13:16.061216+0000","last_became_peered":"2024-04-25T10:13:16.061216+0000","last_unstale":"2024-04-25T10:13:16.065293+0000","last_undegraded":"2024-04-25T10:13:16.065293+0000","last_fullsized":"2024-04-25T10:13:16.065293+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.053049+0000","last_change":"2024-04-25T10:13:16.053049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.053049+0000","last_undegraded":"2024-04-25T10:13:16.053049+0000","last_fullsized":"2024-04-25T10:13:16.053049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066124+0000","last_change":"2024-04-25T10:13:16.063235+0000","last_active":"2024-04-25T10:13:16.066124+0000","last_peered":"2024-04-25T10:13:16.066124+0000","last_clean":"2024-04-25T10:13:16.066124+0000","last_became_active":"2024-04-25T10:13:16.062820+0000","last_became_peered":"2024-04-25T10:13:16.062820+0000","last_unstale":"2024-04-25T10:13:16.066124+0000","last_undegraded":"2024-04-25T10:13:16.066124+0000","last_fullsized":"2024-04-25T10:13:16.066124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:09.118013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.049640+0000","last_change":"2024-04-25T10:13:16.049640+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.049640+0000","last_undegraded":"2024-04-25T10:13:16.049640+0000","last_fullsized":"2024-04-25T10:13:16.049640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064465+0000","last_change":"2024-04-25T10:13:16.057566+0000","last_active":"2024-04-25T10:13:16.064465+0000","last_peered":"2024-04-25T10:13:16.064465+0000","last_clean":"2024-04-25T10:13:16.064465+0000","last_became_active":"2024-04-25T10:13:16.055961+0000","last_became_peered":"2024-04-25T10:13:16.055961+0000","last_unstale":"2024-04-25T10:13:16.064465+0000","last_undegraded":"2024-04-25T10:13:16.064465+0000","last_fullsized":"2024-04-25T10:13:16.064465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043272+0000","last_change":"2024-04-25T10:13:16.043272+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043272+0000","last_undegraded":"2024-04-25T10:13:16.043272+0000","last_fullsized":"2024-04-25T10:13:16.043272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.065693+0000","last_change":"2024-04-25T10:13:16.058726+0000","last_active":"2024-04-25T10:13:16.065693+0000","last_peered":"2024-04-25T10:13:16.065693+0000","last_clean":"2024-04-25T10:13:16.065693+0000","last_became_active":"2024-04-25T10:13:16.058253+0000","last_became_peered":"2024-04-25T10:13:16.058253+0000","last_unstale":"2024-04-25T10:13:16.065693+0000","last_undegraded":"2024-04-25T10:13:16.065693+0000","last_fullsized":"2024-04-25T10:13:16.065693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.044193+0000","last_change":"2024-04-25T10:13:16.044193+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.044193+0000","last_undegraded":"2024-04-25T10:13:16.044193+0000","last_fullsized":"2024-04-25T10:13:16.044193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066628+0000","last_change":"2024-04-25T10:13:16.062295+0000","last_active":"2024-04-25T10:13:16.066628+0000","last_peered":"2024-04-25T10:13:16.066628+0000","last_clean":"2024-04-25T10:13:16.066628+0000","last_became_active":"2024-04-25T10:13:16.061270+0000","last_became_peered":"2024-04-25T10:13:16.061270+0000","last_unstale":"2024-04-25T10:13:16.066628+0000","last_undegraded":"2024-04-25T10:13:16.066628+0000","last_fullsized":"2024-04-25T10:13:16.066628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:11.953599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.042049+0000","last_change":"2024-04-25T10:13:16.042049+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.042049+0000","last_undegraded":"2024-04-25T10:13:16.042049+0000","last_fullsized":"2024-04-25T10:13:16.042049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063994+0000","last_change":"2024-04-25T10:13:16.055470+0000","last_active":"2024-04-25T10:13:16.063994+0000","last_peered":"2024-04-25T10:13:16.063994+0000","last_clean":"2024-04-25T10:13:16.063994+0000","last_became_active":"2024-04-25T10:13:16.055137+0000","last_became_peered":"2024-04-25T10:13:16.055137+0000","last_unstale":"2024-04-25T10:13:16.063994+0000","last_undegraded":"2024-04-25T10:13:16.063994+0000","last_fullsized":"2024-04-25T10:13:16.063994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:10.904343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046361+0000","last_change":"2024-04-25T10:13:16.046361+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046361+0000","last_undegraded":"2024-04-25T10:13:16.046361+0000","last_fullsized":"2024-04-25T10:13:16.046361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.064028+0000","last_change":"2024-04-25T10:13:16.057838+0000","last_active":"2024-04-25T10:13:16.064028+0000","last_peered":"2024-04-25T10:13:16.064028+0000","last_clean":"2024-04-25T10:13:16.064028+0000","last_became_active":"2024-04-25T10:13:16.057476+0000","last_became_peered":"2024-04-25T10:13:16.057476+0000","last_unstale":"2024-04-25T10:13:16.064028+0000","last_undegraded":"2024-04-25T10:13:16.064028+0000","last_fullsized":"2024-04-25T10:13:16.064028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:39.436447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046108+0000","last_change":"2024-04-25T10:13:16.046108+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046108+0000","last_undegraded":"2024-04-25T10:13:16.046108+0000","last_fullsized":"2024-04-25T10:13:16.046108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070376+0000","last_change":"2024-04-25T10:13:16.065980+0000","last_active":"2024-04-25T10:13:16.070376+0000","last_peered":"2024-04-25T10:13:16.070376+0000","last_clean":"2024-04-25T10:13:16.070376+0000","last_became_active":"2024-04-25T10:13:16.065704+0000","last_became_peered":"2024-04-25T10:13:16.065704+0000","last_unstale":"2024-04-25T10:13:16.070376+0000","last_undegraded":"2024-04-25T10:13:16.070376+0000","last_fullsized":"2024-04-25T10:13:16.070376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046010+0000","last_change":"2024-04-25T10:13:16.046010+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046010+0000","last_undegraded":"2024-04-25T10:13:16.046010+0000","last_fullsized":"2024-04-25T10:13:16.046010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.069159+0000","last_change":"2024-04-25T10:13:16.065367+0000","last_active":"2024-04-25T10:13:16.069159+0000","last_peered":"2024-04-25T10:13:16.069159+0000","last_clean":"2024-04-25T10:13:16.069159+0000","last_became_active":"2024-04-25T10:13:16.065198+0000","last_became_peered":"2024-04-25T10:13:16.065198+0000","last_unstale":"2024-04-25T10:13:16.069159+0000","last_undegraded":"2024-04-25T10:13:16.069159+0000","last_fullsized":"2024-04-25T10:13:16.069159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051599+0000","last_change":"2024-04-25T10:13:16.051599+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051599+0000","last_undegraded":"2024-04-25T10:13:16.051599+0000","last_fullsized":"2024-04-25T10:13:16.051599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.066646+0000","last_change":"2024-04-25T10:13:16.062433+0000","last_active":"2024-04-25T10:13:16.066646+0000","last_peered":"2024-04-25T10:13:16.066646+0000","last_clean":"2024-04-25T10:13:16.066646+0000","last_became_active":"2024-04-25T10:13:16.062227+0000","last_became_peered":"2024-04-25T10:13:16.062227+0000","last_unstale":"2024-04-25T10:13:16.066646+0000","last_undegraded":"2024-04-25T10:13:16.066646+0000","last_fullsized":"2024-04-25T10:13:16.066646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:33.301434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.051355+0000","last_change":"2024-04-25T10:13:16.051355+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.051355+0000","last_undegraded":"2024-04-25T10:13:16.051355+0000","last_fullsized":"2024-04-25T10:13:16.051355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.063289+0000","last_change":"2024-04-25T10:13:16.057718+0000","last_active":"2024-04-25T10:13:16.063289+0000","last_peered":"2024-04-25T10:13:16.063289+0000","last_clean":"2024-04-25T10:13:16.063289+0000","last_became_active":"2024-04-25T10:13:16.057007+0000","last_became_peered":"2024-04-25T10:13:16.057007+0000","last_unstale":"2024-04-25T10:13:16.063289+0000","last_undegraded":"2024-04-25T10:13:16.063289+0000","last_fullsized":"2024-04-25T10:13:16.063289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.046460+0000","last_change":"2024-04-25T10:13:16.046460+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.046460+0000","last_undegraded":"2024-04-25T10:13:16.046460+0000","last_fullsized":"2024-04-25T10:13:16.046460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.056100+0000","last_change":"2024-04-25T10:13:16.051248+0000","last_active":"2024-04-25T10:13:16.056100+0000","last_peered":"2024-04-25T10:13:16.056100+0000","last_clean":"2024-04-25T10:13:16.056100+0000","last_became_active":"2024-04-25T10:13:16.050951+0000","last_became_peered":"2024-04-25T10:13:16.050951+0000","last_unstale":"2024-04-25T10:13:16.056100+0000","last_undegraded":"2024-04-25T10:13:16.056100+0000","last_fullsized":"2024-04-25T10:13:16.056100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043744+0000","last_change":"2024-04-25T10:13:16.043744+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043744+0000","last_undegraded":"2024-04-25T10:13:16.043744+0000","last_fullsized":"2024-04-25T10:13:16.043744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070444+0000","last_change":"2024-04-25T10:13:16.066644+0000","last_active":"2024-04-25T10:13:16.070444+0000","last_peered":"2024-04-25T10:13:16.070444+0000","last_clean":"2024-04-25T10:13:16.070444+0000","last_became_active":"2024-04-25T10:13:16.064572+0000","last_became_peered":"2024-04-25T10:13:16.064572+0000","last_unstale":"2024-04-25T10:13:16.070444+0000","last_undegraded":"2024-04-25T10:13:16.070444+0000","last_fullsized":"2024-04-25T10:13:16.070444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.048564+0000","last_change":"2024-04-25T10:13:16.048564+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.048564+0000","last_undegraded":"2024-04-25T10:13:16.048564+0000","last_fullsized":"2024-04-25T10:13:16.048564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.061896+0000","last_change":"2024-04-25T10:13:16.057038+0000","last_active":"2024-04-25T10:13:16.061896+0000","last_peered":"2024-04-25T10:13:16.061896+0000","last_clean":"2024-04-25T10:13:16.061896+0000","last_became_active":"2024-04-25T10:13:16.056758+0000","last_became_peered":"2024-04-25T10:13:16.056758+0000","last_unstale":"2024-04-25T10:13:16.061896+0000","last_undegraded":"2024-04-25T10:13:16.061896+0000","last_fullsized":"2024-04-25T10:13:16.061896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.050996+0000","last_change":"2024-04-25T10:13:16.050996+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.050996+0000","last_undegraded":"2024-04-25T10:13:16.050996+0000","last_fullsized":"2024-04-25T10:13:16.050996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.060591+0000","last_change":"2024-04-25T10:13:16.055447+0000","last_active":"2024-04-25T10:13:16.060591+0000","last_peered":"2024-04-25T10:13:16.060591+0000","last_clean":"2024-04-25T10:13:16.060591+0000","last_became_active":"2024-04-25T10:13:16.055157+0000","last_became_peered":"2024-04-25T10:13:16.055157+0000","last_unstale":"2024-04-25T10:13:16.060591+0000","last_undegraded":"2024-04-25T10:13:16.060591+0000","last_fullsized":"2024-04-25T10:13:16.060591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:13.025816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.047880+0000","last_change":"2024-04-25T10:13:16.047880+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.047880+0000","last_undegraded":"2024-04-25T10:13:16.047880+0000","last_fullsized":"2024-04-25T10:13:16.047880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.054639+0000","last_change":"2024-04-25T10:13:16.049162+0000","last_active":"2024-04-25T10:13:16.054639+0000","last_peered":"2024-04-25T10:13:16.054639+0000","last_clean":"2024-04-25T10:13:16.054639+0000","last_became_active":"2024-04-25T10:13:16.048699+0000","last_became_peered":"2024-04-25T10:13:16.048699+0000","last_unstale":"2024-04-25T10:13:16.054639+0000","last_undegraded":"2024-04-25T10:13:16.054639+0000","last_fullsized":"2024-04-25T10:13:16.054639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043077+0000","last_change":"2024-04-25T10:13:16.043077+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043077+0000","last_undegraded":"2024-04-25T10:13:16.043077+0000","last_fullsized":"2024-04-25T10:13:16.043077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.055963+0000","last_change":"2024-04-25T10:13:16.052633+0000","last_active":"2024-04-25T10:13:16.055963+0000","last_peered":"2024-04-25T10:13:16.055963+0000","last_clean":"2024-04-25T10:13:16.055963+0000","last_became_active":"2024-04-25T10:13:16.052308+0000","last_became_peered":"2024-04-25T10:13:16.052308+0000","last_unstale":"2024-04-25T10:13:16.055963+0000","last_undegraded":"2024-04-25T10:13:16.055963+0000","last_fullsized":"2024-04-25T10:13:16.055963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:01.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.043595+0000","last_change":"2024-04-25T10:13:16.043595+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.043595+0000","last_undegraded":"2024-04-25T10:13:16.043595+0000","last_fullsized":"2024-04-25T10:13:16.043595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.070420+0000","last_change":"2024-04-25T10:13:16.065524+0000","last_active":"2024-04-25T10:13:16.070420+0000","last_peered":"2024-04-25T10:13:16.070420+0000","last_clean":"2024-04-25T10:13:16.070420+0000","last_became_active":"2024-04-25T10:13:16.065056+0000","last_became_peered":"2024-04-25T10:13:16.065056+0000","last_unstale":"2024-04-25T10:13:16.070420+0000","last_undegraded":"2024-04-25T10:13:16.070420+0000","last_fullsized":"2024-04-25T10:13:16.070420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:18.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T10:13:16.052506+0000","last_change":"2024-04-25T10:13:16.052506+0000","last_active":"2024-04-25T10:13:16.029897+0000","last_peered":"2024-04-25T10:13:16.029897+0000","last_clean":"2024-04-25T10:13:16.029897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:16.052506+0000","last_undegraded":"2024-04-25T10:13:16.052506+0000","last_fullsized":"2024-04-25T10:13:16.052506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.072389+0000","last_change":"2024-04-25T10:13:16.066995+0000","last_active":"2024-04-25T10:13:16.072389+0000","last_peered":"2024-04-25T10:13:16.072389+0000","last_clean":"2024-04-25T10:13:16.072389+0000","last_became_active":"2024-04-25T10:13:16.066686+0000","last_became_peered":"2024-04-25T10:13:16.066686+0000","last_unstale":"2024-04-25T10:13:16.072389+0000","last_undegraded":"2024-04-25T10:13:16.072389+0000","last_fullsized":"2024-04-25T10:13:16.072389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:13:16.059855+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:13:16.059855+0000","last_peered":"2024-04-25T10:13:16.059855+0000","last_clean":"2024-04-25T10:13:16.059855+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:13:16.059855+0000","last_undegraded":"2024-04-25T10:13:16.059855+0000","last_fullsized":"2024-04-25T10:13:16.059855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:25.640 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:25.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.778+0000 7efd4e000640 1 Processor -- start 2024-04-25T10:13:25.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.778+0000 7efd4e000640 1 -- start start 2024-04-25T10:13:25.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd4e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd480d0040 0x7efd4815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd4e000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd48071050 con 0x7efd4815b740 2024-04-25T10:13:25.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd4e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4815d040 con 0x7efd481550b0 2024-04-25T10:13:25.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd4e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd480712f0 con 0x7efd480d0040 2024-04-25T10:13:25.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd47e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efd4815b740 0x7efd480b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56734/0 (socket says 172.21.15.87:56734) 2024-04-25T10:13:25.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd47e00640 1 -- 172.21.15.87:0/3520226320 learned_addr learned my addr 172.21.15.87:0/3520226320 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:25.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46a00640 1 --2- 172.21.15.87:0/3520226320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd480d0040 0x7efd4815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46000640 1 -- 172.21.15.87:0/3520226320 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894387832 0 0) 0x7efd48071050 con 0x7efd4815b740 2024-04-25T10:13:25.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46000640 1 -- 172.21.15.87:0/3520226320 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd2c003530 con 0x7efd4815b740 2024-04-25T10:13:25.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46000640 1 -- 172.21.15.87:0/3520226320 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92359523 0 0) 0x7efd4815d040 con 0x7efd481550b0 2024-04-25T10:13:25.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46000640 1 -- 172.21.15.87:0/3520226320 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd48071050 con 0x7efd481550b0 2024-04-25T10:13:25.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46a00640 1 -- 172.21.15.87:0/3520226320 >> v1:172.21.15.169:6789/0 conn(0x7efd481550b0 legacy=0x7efd480cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46a00640 1 -- 172.21.15.87:0/3520226320 >> v1:172.21.15.87:6789/0 conn(0x7efd4815b740 legacy=0x7efd480b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.782+0000 7efd46a00640 1 -- 172.21.15.87:0/3520226320 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480715d0 con 0x7efd480d0040 2024-04-25T10:13:25.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd46a00640 1 --2- 172.21.15.87:0/3520226320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd480d0040 0x7efd4815b200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd340097f0 tx=0x7efd3401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59df70905131de0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd46000640 1 -- 172.21.15.87:0/3520226320 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340b1070 con 0x7efd480d0040 2024-04-25T10:13:25.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd46000640 1 -- 172.21.15.87:0/3520226320 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340ac410 con 0x7efd480d0040 2024-04-25T10:13:25.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd46000640 1 -- 172.21.15.87:0/3520226320 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340ac7e0 con 0x7efd480d0040 2024-04-25T10:13:25.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 -- 172.21.15.87:0/3520226320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd480d0040 msgr2=0x7efd4815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 --2- 172.21.15.87:0/3520226320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd480d0040 0x7efd4815b200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd340097f0 tx=0x7efd3401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 -- 172.21.15.87:0/3520226320 shutdown_connections 2024-04-25T10:13:25.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 --2- 172.21.15.87:0/3520226320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd480d0040 0x7efd4815b200 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 -- 172.21.15.87:0/3520226320 >> 172.21.15.87:0/3520226320 conn(0x7efd48066b70 msgr2=0x7efd48066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 -- 172.21.15.87:0/3520226320 shutdown_connections 2024-04-25T10:13:25.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 -- 172.21.15.87:0/3520226320 wait complete. 2024-04-25T10:13:25.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 Processor -- start 2024-04-25T10:13:25.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.786+0000 7efd4e000640 1 -- start start 2024-04-25T10:13:25.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd480d0040 0x7efd4814d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd481550b0 0x7efd4814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd4815b740 0x7efd48159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4815bcf0 con 0x7efd4815b740 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4815bb70 con 0x7efd480d0040 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd481684c0 con 0x7efd481550b0 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd4815b740 0x7efd48159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd4815b740 0x7efd48159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44960/0 (socket says 172.21.15.87:44960) 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 -- 172.21.15.87:0/2300811785 learned_addr learned my addr 172.21.15.87:0/2300811785 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 -- 172.21.15.87:0/2300811785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd481550b0 msgr2=0x7efd4814f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd481550b0 0x7efd4814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 -- 172.21.15.87:0/2300811785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd480d0040 msgr2=0x7efd4814d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd480d0040 0x7efd4814d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 -- 172.21.15.87:0/2300811785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4814d830 con 0x7efd4815b740 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd47e00640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd4815b740 0x7efd48159fd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c460 tx=0x7efd4000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd3f400640 1 -- 172.21.15.87:0/2300811785 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40017070 con 0x7efd4815b740 2024-04-25T10:13:25.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd3f400640 1 -- 172.21.15.87:0/2300811785 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd40005cf0 con 0x7efd4815b740 2024-04-25T10:13:25.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd48158f30 con 0x7efd4815b740 2024-04-25T10:13:25.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd3f400640 1 -- 172.21.15.87:0/2300811785 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400072f0 con 0x7efd4815b740 2024-04-25T10:13:25.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd481594f0 con 0x7efd4815b740 2024-04-25T10:13:25.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4804ef30 con 0x7efd4815b740 2024-04-25T10:13:25.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd3f400640 1 -- 172.21.15.87:0/2300811785 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd4001a070 con 0x7efd4815b740 2024-04-25T10:13:25.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.790+0000 7efd3f400640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd28066f90 0x7efd28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.794+0000 7efd3f400640 1 -- 172.21.15.87:0/2300811785 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd400844f0 con 0x7efd4815b740 2024-04-25T10:13:25.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.794+0000 7efd47400640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd28066f90 0x7efd28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.794+0000 7efd47400640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd28066f90 0x7efd28069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efd38001570 tx=0x7efd38007c80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.798+0000 7efd3f400640 1 -- 172.21.15.87:0/2300811785 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd40055460 con 0x7efd4815b740 2024-04-25T10:13:25.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.946+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd4814fb70 con 0x7efd28066f90 2024-04-25T10:13:25.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.950+0000 7efd3f400640 1 -- 172.21.15.87:0/2300811785 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251950 (secure 0 0 0) 0x7efd4814fb70 con 0x7efd28066f90 2024-04-25T10:13:25.961 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:25.963 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd28066f90 msgr2=0x7efd28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd28066f90 0x7efd28069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efd38001570 tx=0x7efd38007c80 comp rx=0 tx=0).stop 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd4815b740 msgr2=0x7efd48159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd4815b740 0x7efd48159fd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c460 tx=0x7efd4000c920 comp rx=0 tx=0).stop 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 shutdown_connections 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd28066f90 0x7efd28069450 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd4815b740 0x7efd48159fd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd481550b0 0x7efd4814f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 --2- 172.21.15.87:0/2300811785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd480d0040 0x7efd4814d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 >> 172.21.15.87:0/2300811785 conn(0x7efd48066b70 msgr2=0x7efd4815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 shutdown_connections 2024-04-25T10:13:25.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:25.954+0000 7efd4e000640 1 -- 172.21.15.87:0/2300811785 wait complete. 2024-04-25T10:13:25.981 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-25T10:13:25.137160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220300,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754420,"statfs":{"total":773094113280,"available":772868526080,"internally_reserved":0,"allocated":4235264,"data_stored":2358484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.097655"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787131+0000","last_change":"2024-04-25T10:13:17.066942+0000","last_active":"2024-04-25T10:13:18.787131+0000","last_peered":"2024-04-25T10:13:18.787131+0000","last_clean":"2024-04-25T10:13:18.787131+0000","last_became_active":"2024-04-25T10:13:17.066652+0000","last_became_peered":"2024-04-25T10:13:17.066652+0000","last_unstale":"2024-04-25T10:13:18.787131+0000","last_undegraded":"2024-04-25T10:13:18.787131+0000","last_fullsized":"2024-04-25T10:13:18.787131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:51.952858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787565+0000","last_change":"2024-04-25T10:13:17.067864+0000","last_active":"2024-04-25T10:13:18.787565+0000","last_peered":"2024-04-25T10:13:18.787565+0000","last_clean":"2024-04-25T10:13:18.787565+0000","last_became_active":"2024-04-25T10:13:17.067098+0000","last_became_peered":"2024-04-25T10:13:17.067098+0000","last_unstale":"2024-04-25T10:13:18.787565+0000","last_undegraded":"2024-04-25T10:13:18.787565+0000","last_fullsized":"2024-04-25T10:13:18.787565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:57.954125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061463+0000","last_change":"2024-04-25T10:13:17.076114+0000","last_active":"2024-04-25T10:13:18.061463+0000","last_peered":"2024-04-25T10:13:18.061463+0000","last_clean":"2024-04-25T10:13:18.061463+0000","last_became_active":"2024-04-25T10:13:17.075425+0000","last_became_peered":"2024-04-25T10:13:17.075425+0000","last_unstale":"2024-04-25T10:13:18.061463+0000","last_undegraded":"2024-04-25T10:13:18.061463+0000","last_fullsized":"2024-04-25T10:13:18.061463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:30.745849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059508+0000","last_change":"2024-04-25T10:13:17.082662+0000","last_active":"2024-04-25T10:13:18.059508+0000","last_peered":"2024-04-25T10:13:18.059508+0000","last_clean":"2024-04-25T10:13:18.059508+0000","last_became_active":"2024-04-25T10:13:17.082395+0000","last_became_peered":"2024-04-25T10:13:17.082395+0000","last_unstale":"2024-04-25T10:13:18.059508+0000","last_undegraded":"2024-04-25T10:13:18.059508+0000","last_fullsized":"2024-04-25T10:13:18.059508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:17.711307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787089+0000","last_change":"2024-04-25T10:13:17.074866+0000","last_active":"2024-04-25T10:13:18.787089+0000","last_peered":"2024-04-25T10:13:18.787089+0000","last_clean":"2024-04-25T10:13:18.787089+0000","last_became_active":"2024-04-25T10:13:17.074514+0000","last_became_peered":"2024-04-25T10:13:17.074514+0000","last_unstale":"2024-04-25T10:13:18.787089+0000","last_undegraded":"2024-04-25T10:13:18.787089+0000","last_fullsized":"2024-04-25T10:13:18.787089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:27.136994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787000+0000","last_change":"2024-04-25T10:13:17.057285+0000","last_active":"2024-04-25T10:13:18.787000+0000","last_peered":"2024-04-25T10:13:18.787000+0000","last_clean":"2024-04-25T10:13:18.787000+0000","last_became_active":"2024-04-25T10:13:17.056999+0000","last_became_peered":"2024-04-25T10:13:17.056999+0000","last_unstale":"2024-04-25T10:13:18.787000+0000","last_undegraded":"2024-04-25T10:13:18.787000+0000","last_fullsized":"2024-04-25T10:13:18.787000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:58.098204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787177+0000","last_change":"2024-04-25T10:13:17.071207+0000","last_active":"2024-04-25T10:13:18.787177+0000","last_peered":"2024-04-25T10:13:18.787177+0000","last_clean":"2024-04-25T10:13:18.787177+0000","last_became_active":"2024-04-25T10:13:17.070861+0000","last_became_peered":"2024-04-25T10:13:17.070861+0000","last_unstale":"2024-04-25T10:13:18.787177+0000","last_undegraded":"2024-04-25T10:13:18.787177+0000","last_fullsized":"2024-04-25T10:13:18.787177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:12.381668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786759+0000","last_change":"2024-04-25T10:13:17.061861+0000","last_active":"2024-04-25T10:13:18.786759+0000","last_peered":"2024-04-25T10:13:18.786759+0000","last_clean":"2024-04-25T10:13:18.786759+0000","last_became_active":"2024-04-25T10:13:17.061372+0000","last_became_peered":"2024-04-25T10:13:17.061372+0000","last_unstale":"2024-04-25T10:13:18.786759+0000","last_undegraded":"2024-04-25T10:13:18.786759+0000","last_fullsized":"2024-04-25T10:13:18.786759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:17.711307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061716+0000","last_change":"2024-04-25T10:13:17.081365+0000","last_active":"2024-04-25T10:13:18.061716+0000","last_peered":"2024-04-25T10:13:18.061716+0000","last_clean":"2024-04-25T10:13:18.061716+0000","last_became_active":"2024-04-25T10:13:17.080945+0000","last_became_peered":"2024-04-25T10:13:17.080945+0000","last_unstale":"2024-04-25T10:13:18.061716+0000","last_undegraded":"2024-04-25T10:13:18.061716+0000","last_fullsized":"2024-04-25T10:13:18.061716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:21.473698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059763+0000","last_change":"2024-04-25T10:13:17.084021+0000","last_active":"2024-04-25T10:13:18.059763+0000","last_peered":"2024-04-25T10:13:18.059763+0000","last_clean":"2024-04-25T10:13:18.059763+0000","last_became_active":"2024-04-25T10:13:17.083655+0000","last_became_peered":"2024-04-25T10:13:17.083655+0000","last_unstale":"2024-04-25T10:13:18.059763+0000","last_undegraded":"2024-04-25T10:13:18.059763+0000","last_fullsized":"2024-04-25T10:13:18.059763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:33.747462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787500+0000","last_change":"2024-04-25T10:13:17.073903+0000","last_active":"2024-04-25T10:13:18.787500+0000","last_peered":"2024-04-25T10:13:18.787500+0000","last_clean":"2024-04-25T10:13:18.787500+0000","last_became_active":"2024-04-25T10:13:17.073617+0000","last_became_peered":"2024-04-25T10:13:17.073617+0000","last_unstale":"2024-04-25T10:13:18.787500+0000","last_undegraded":"2024-04-25T10:13:18.787500+0000","last_fullsized":"2024-04-25T10:13:18.787500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:23.488785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060925+0000","last_change":"2024-04-25T10:13:17.057931+0000","last_active":"2024-04-25T10:13:18.060925+0000","last_peered":"2024-04-25T10:13:18.060925+0000","last_clean":"2024-04-25T10:13:18.060925+0000","last_became_active":"2024-04-25T10:13:17.057642+0000","last_became_peered":"2024-04-25T10:13:17.057642+0000","last_unstale":"2024-04-25T10:13:18.060925+0000","last_undegraded":"2024-04-25T10:13:18.060925+0000","last_fullsized":"2024-04-25T10:13:18.060925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:32.738164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787244+0000","last_change":"2024-04-25T10:13:17.081870+0000","last_active":"2024-04-25T10:13:18.787244+0000","last_peered":"2024-04-25T10:13:18.787244+0000","last_clean":"2024-04-25T10:13:18.787244+0000","last_became_active":"2024-04-25T10:13:17.081567+0000","last_became_peered":"2024-04-25T10:13:17.081567+0000","last_unstale":"2024-04-25T10:13:18.787244+0000","last_undegraded":"2024-04-25T10:13:18.787244+0000","last_fullsized":"2024-04-25T10:13:18.787244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:24.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787104+0000","last_change":"2024-04-25T10:13:17.065877+0000","last_active":"2024-04-25T10:13:18.787104+0000","last_peered":"2024-04-25T10:13:18.787104+0000","last_clean":"2024-04-25T10:13:18.787104+0000","last_became_active":"2024-04-25T10:13:17.062728+0000","last_became_peered":"2024-04-25T10:13:17.062728+0000","last_unstale":"2024-04-25T10:13:18.787104+0000","last_undegraded":"2024-04-25T10:13:18.787104+0000","last_fullsized":"2024-04-25T10:13:18.787104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:30.745849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060612+0000","last_change":"2024-04-25T10:13:17.061862+0000","last_active":"2024-04-25T10:13:18.060612+0000","last_peered":"2024-04-25T10:13:18.060612+0000","last_clean":"2024-04-25T10:13:18.060612+0000","last_became_active":"2024-04-25T10:13:17.061614+0000","last_became_peered":"2024-04-25T10:13:17.061614+0000","last_unstale":"2024-04-25T10:13:18.060612+0000","last_undegraded":"2024-04-25T10:13:18.060612+0000","last_fullsized":"2024-04-25T10:13:18.060612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:33.747462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787543+0000","last_change":"2024-04-25T10:13:17.072602+0000","last_active":"2024-04-25T10:13:18.787543+0000","last_peered":"2024-04-25T10:13:18.787543+0000","last_clean":"2024-04-25T10:13:18.787543+0000","last_became_active":"2024-04-25T10:13:17.072325+0000","last_became_peered":"2024-04-25T10:13:17.072325+0000","last_unstale":"2024-04-25T10:13:18.787543+0000","last_undegraded":"2024-04-25T10:13:18.787543+0000","last_fullsized":"2024-04-25T10:13:18.787543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:06.565345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787444+0000","last_change":"2024-04-25T10:13:17.070634+0000","last_active":"2024-04-25T10:13:18.787444+0000","last_peered":"2024-04-25T10:13:18.787444+0000","last_clean":"2024-04-25T10:13:18.787444+0000","last_became_active":"2024-04-25T10:13:17.070364+0000","last_became_peered":"2024-04-25T10:13:17.070364+0000","last_unstale":"2024-04-25T10:13:18.787444+0000","last_undegraded":"2024-04-25T10:13:18.787444+0000","last_fullsized":"2024-04-25T10:13:18.787444+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:30.745849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787642+0000","last_change":"2024-04-25T10:13:17.081417+0000","last_active":"2024-04-25T10:13:18.787642+0000","last_peered":"2024-04-25T10:13:18.787642+0000","last_clean":"2024-04-25T10:13:18.787642+0000","last_became_active":"2024-04-25T10:13:17.081147+0000","last_became_peered":"2024-04-25T10:13:17.081147+0000","last_unstale":"2024-04-25T10:13:18.787642+0000","last_undegraded":"2024-04-25T10:13:18.787642+0000","last_fullsized":"2024-04-25T10:13:18.787642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:25.111431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787709+0000","last_change":"2024-04-25T10:13:17.076084+0000","last_active":"2024-04-25T10:13:18.787709+0000","last_peered":"2024-04-25T10:13:18.787709+0000","last_clean":"2024-04-25T10:13:18.787709+0000","last_became_active":"2024-04-25T10:13:17.075069+0000","last_became_peered":"2024-04-25T10:13:17.075069+0000","last_unstale":"2024-04-25T10:13:18.787709+0000","last_undegraded":"2024-04-25T10:13:18.787709+0000","last_fullsized":"2024-04-25T10:13:18.787709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:56.287212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787583+0000","last_change":"2024-04-25T10:13:17.059465+0000","last_active":"2024-04-25T10:13:18.787583+0000","last_peered":"2024-04-25T10:13:18.787583+0000","last_clean":"2024-04-25T10:13:18.787583+0000","last_became_active":"2024-04-25T10:13:17.059159+0000","last_became_peered":"2024-04-25T10:13:17.059159+0000","last_unstale":"2024-04-25T10:13:18.787583+0000","last_undegraded":"2024-04-25T10:13:18.787583+0000","last_fullsized":"2024-04-25T10:13:18.787583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:05.751743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060900+0000","last_change":"2024-04-25T10:13:17.059728+0000","last_active":"2024-04-25T10:13:18.060900+0000","last_peered":"2024-04-25T10:13:18.060900+0000","last_clean":"2024-04-25T10:13:18.060900+0000","last_became_active":"2024-04-25T10:13:17.059468+0000","last_became_peered":"2024-04-25T10:13:17.059468+0000","last_unstale":"2024-04-25T10:13:18.060900+0000","last_undegraded":"2024-04-25T10:13:18.060900+0000","last_fullsized":"2024-04-25T10:13:18.060900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:51.952858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061003+0000","last_change":"2024-04-25T10:13:17.061341+0000","last_active":"2024-04-25T10:13:18.061003+0000","last_peered":"2024-04-25T10:13:18.061003+0000","last_clean":"2024-04-25T10:13:18.061003+0000","last_became_active":"2024-04-25T10:13:17.061041+0000","last_became_peered":"2024-04-25T10:13:17.061041+0000","last_unstale":"2024-04-25T10:13:18.061003+0000","last_undegraded":"2024-04-25T10:13:18.061003+0000","last_fullsized":"2024-04-25T10:13:18.061003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:12.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788354+0000","last_change":"2024-04-25T10:13:17.073959+0000","last_active":"2024-04-25T10:13:18.788354+0000","last_peered":"2024-04-25T10:13:18.788354+0000","last_clean":"2024-04-25T10:13:18.788354+0000","last_became_active":"2024-04-25T10:13:17.068652+0000","last_became_peered":"2024-04-25T10:13:17.068652+0000","last_unstale":"2024-04-25T10:13:18.788354+0000","last_undegraded":"2024-04-25T10:13:18.788354+0000","last_fullsized":"2024-04-25T10:13:18.788354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:09.070657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060704+0000","last_change":"2024-04-25T10:13:17.062742+0000","last_active":"2024-04-25T10:13:18.060704+0000","last_peered":"2024-04-25T10:13:18.060704+0000","last_clean":"2024-04-25T10:13:18.060704+0000","last_became_active":"2024-04-25T10:13:17.062462+0000","last_became_peered":"2024-04-25T10:13:17.062462+0000","last_unstale":"2024-04-25T10:13:18.060704+0000","last_undegraded":"2024-04-25T10:13:18.060704+0000","last_fullsized":"2024-04-25T10:13:18.060704+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:39.282949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059997+0000","last_change":"2024-04-25T10:13:17.082755+0000","last_active":"2024-04-25T10:13:18.059997+0000","last_peered":"2024-04-25T10:13:18.059997+0000","last_clean":"2024-04-25T10:13:18.059997+0000","last_became_active":"2024-04-25T10:13:17.081262+0000","last_became_peered":"2024-04-25T10:13:17.081262+0000","last_unstale":"2024-04-25T10:13:18.059997+0000","last_undegraded":"2024-04-25T10:13:18.059997+0000","last_fullsized":"2024-04-25T10:13:18.059997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:32.738164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787726+0000","last_change":"2024-04-25T10:13:17.068048+0000","last_active":"2024-04-25T10:13:18.787726+0000","last_peered":"2024-04-25T10:13:18.787726+0000","last_clean":"2024-04-25T10:13:18.787726+0000","last_became_active":"2024-04-25T10:13:17.067620+0000","last_became_peered":"2024-04-25T10:13:17.067620+0000","last_unstale":"2024-04-25T10:13:18.787726+0000","last_undegraded":"2024-04-25T10:13:18.787726+0000","last_fullsized":"2024-04-25T10:13:18.787726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:54.062980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059860+0000","last_change":"2024-04-25T10:13:17.072365+0000","last_active":"2024-04-25T10:13:18.059860+0000","last_peered":"2024-04-25T10:13:18.059860+0000","last_clean":"2024-04-25T10:13:18.059860+0000","last_became_active":"2024-04-25T10:13:17.072156+0000","last_became_peered":"2024-04-25T10:13:17.072156+0000","last_unstale":"2024-04-25T10:13:18.059860+0000","last_undegraded":"2024-04-25T10:13:18.059860+0000","last_fullsized":"2024-04-25T10:13:18.059860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:39.282949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061079+0000","last_change":"2024-04-25T10:13:17.064343+0000","last_active":"2024-04-25T10:13:18.061079+0000","last_peered":"2024-04-25T10:13:18.061079+0000","last_clean":"2024-04-25T10:13:18.061079+0000","last_became_active":"2024-04-25T10:13:17.063964+0000","last_became_peered":"2024-04-25T10:13:17.063964+0000","last_unstale":"2024-04-25T10:13:18.061079+0000","last_undegraded":"2024-04-25T10:13:18.061079+0000","last_fullsized":"2024-04-25T10:13:18.061079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:12.381668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061095+0000","last_change":"2024-04-25T10:13:17.059857+0000","last_active":"2024-04-25T10:13:18.061095+0000","last_peered":"2024-04-25T10:13:18.061095+0000","last_clean":"2024-04-25T10:13:18.061095+0000","last_became_active":"2024-04-25T10:13:17.059581+0000","last_became_peered":"2024-04-25T10:13:17.059581+0000","last_unstale":"2024-04-25T10:13:18.061095+0000","last_undegraded":"2024-04-25T10:13:18.061095+0000","last_fullsized":"2024-04-25T10:13:18.061095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:57.055382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786899+0000","last_change":"2024-04-25T10:13:17.070154+0000","last_active":"2024-04-25T10:13:18.786899+0000","last_peered":"2024-04-25T10:13:18.786899+0000","last_clean":"2024-04-25T10:13:18.786899+0000","last_became_active":"2024-04-25T10:13:17.069891+0000","last_became_peered":"2024-04-25T10:13:17.069891+0000","last_unstale":"2024-04-25T10:13:18.786899+0000","last_undegraded":"2024-04-25T10:13:18.786899+0000","last_fullsized":"2024-04-25T10:13:18.786899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:20.999645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786687+0000","last_change":"2024-04-25T10:13:17.060496+0000","last_active":"2024-04-25T10:13:18.786687+0000","last_peered":"2024-04-25T10:13:18.786687+0000","last_clean":"2024-04-25T10:13:18.786687+0000","last_became_active":"2024-04-25T10:13:17.060197+0000","last_became_peered":"2024-04-25T10:13:17.060197+0000","last_unstale":"2024-04-25T10:13:18.786687+0000","last_undegraded":"2024-04-25T10:13:18.786687+0000","last_fullsized":"2024-04-25T10:13:18.786687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.206588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786943+0000","last_change":"2024-04-25T10:13:17.065411+0000","last_active":"2024-04-25T10:13:18.786943+0000","last_peered":"2024-04-25T10:13:18.786943+0000","last_clean":"2024-04-25T10:13:18.786943+0000","last_became_active":"2024-04-25T10:13:17.064856+0000","last_became_peered":"2024-04-25T10:13:17.064856+0000","last_unstale":"2024-04-25T10:13:18.786943+0000","last_undegraded":"2024-04-25T10:13:18.786943+0000","last_fullsized":"2024-04-25T10:13:18.786943+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:33.747462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788603+0000","last_change":"2024-04-25T10:13:17.067919+0000","last_active":"2024-04-25T10:13:18.788603+0000","last_peered":"2024-04-25T10:13:18.788603+0000","last_clean":"2024-04-25T10:13:18.788603+0000","last_became_active":"2024-04-25T10:13:17.067661+0000","last_became_peered":"2024-04-25T10:13:17.067661+0000","last_unstale":"2024-04-25T10:13:18.788603+0000","last_undegraded":"2024-04-25T10:13:18.788603+0000","last_fullsized":"2024-04-25T10:13:18.788603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:10.343457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787420+0000","last_change":"2024-04-25T10:13:16.059863+0000","last_active":"2024-04-25T10:13:18.787420+0000","last_peered":"2024-04-25T10:13:18.787420+0000","last_clean":"2024-04-25T10:13:18.787420+0000","last_became_active":"2024-04-25T10:13:16.059568+0000","last_became_peered":"2024-04-25T10:13:16.059568+0000","last_unstale":"2024-04-25T10:13:18.787420+0000","last_undegraded":"2024-04-25T10:13:18.787420+0000","last_fullsized":"2024-04-25T10:13:18.787420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:08.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787309+0000","last_change":"2024-04-25T10:13:17.066983+0000","last_active":"2024-04-25T10:13:18.787309+0000","last_peered":"2024-04-25T10:13:18.787309+0000","last_clean":"2024-04-25T10:13:18.787309+0000","last_became_active":"2024-04-25T10:13:17.066711+0000","last_became_peered":"2024-04-25T10:13:17.066711+0000","last_unstale":"2024-04-25T10:13:18.787309+0000","last_undegraded":"2024-04-25T10:13:18.787309+0000","last_fullsized":"2024-04-25T10:13:18.787309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:12.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788617+0000","last_change":"2024-04-25T10:13:16.067950+0000","last_active":"2024-04-25T10:13:18.788617+0000","last_peered":"2024-04-25T10:13:18.788617+0000","last_clean":"2024-04-25T10:13:18.788617+0000","last_became_active":"2024-04-25T10:13:16.067755+0000","last_became_peered":"2024-04-25T10:13:16.067755+0000","last_unstale":"2024-04-25T10:13:18.788617+0000","last_undegraded":"2024-04-25T10:13:18.788617+0000","last_fullsized":"2024-04-25T10:13:18.788617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061336+0000","last_change":"2024-04-25T10:13:17.060763+0000","last_active":"2024-04-25T10:13:18.061336+0000","last_peered":"2024-04-25T10:13:18.061336+0000","last_clean":"2024-04-25T10:13:18.061336+0000","last_became_active":"2024-04-25T10:13:17.060472+0000","last_became_peered":"2024-04-25T10:13:17.060472+0000","last_unstale":"2024-04-25T10:13:18.061336+0000","last_undegraded":"2024-04-25T10:13:18.061336+0000","last_fullsized":"2024-04-25T10:13:18.061336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.839071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787775+0000","last_change":"2024-04-25T10:13:16.058185+0000","last_active":"2024-04-25T10:13:18.787775+0000","last_peered":"2024-04-25T10:13:18.787775+0000","last_clean":"2024-04-25T10:13:18.787775+0000","last_became_active":"2024-04-25T10:13:16.057713+0000","last_became_peered":"2024-04-25T10:13:16.057713+0000","last_unstale":"2024-04-25T10:13:18.787775+0000","last_undegraded":"2024-04-25T10:13:18.787775+0000","last_fullsized":"2024-04-25T10:13:18.787775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:50.941071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059844+0000","last_change":"2024-04-25T10:13:17.084663+0000","last_active":"2024-04-25T10:13:18.059844+0000","last_peered":"2024-04-25T10:13:18.059844+0000","last_clean":"2024-04-25T10:13:18.059844+0000","last_became_active":"2024-04-25T10:13:17.083095+0000","last_became_peered":"2024-04-25T10:13:17.083095+0000","last_unstale":"2024-04-25T10:13:18.059844+0000","last_undegraded":"2024-04-25T10:13:18.059844+0000","last_fullsized":"2024-04-25T10:13:18.059844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:14.646796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787771+0000","last_change":"2024-04-25T10:13:16.058303+0000","last_active":"2024-04-25T10:13:18.787771+0000","last_peered":"2024-04-25T10:13:18.787771+0000","last_clean":"2024-04-25T10:13:18.787771+0000","last_became_active":"2024-04-25T10:13:16.057794+0000","last_became_peered":"2024-04-25T10:13:16.057794+0000","last_unstale":"2024-04-25T10:13:18.787771+0000","last_undegraded":"2024-04-25T10:13:18.787771+0000","last_fullsized":"2024-04-25T10:13:18.787771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:39.436447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787059+0000","last_change":"2024-04-25T10:13:17.064729+0000","last_active":"2024-04-25T10:13:18.787059+0000","last_peered":"2024-04-25T10:13:18.787059+0000","last_clean":"2024-04-25T10:13:18.787059+0000","last_became_active":"2024-04-25T10:13:17.064386+0000","last_became_peered":"2024-04-25T10:13:17.064386+0000","last_unstale":"2024-04-25T10:13:18.787059+0000","last_undegraded":"2024-04-25T10:13:18.787059+0000","last_fullsized":"2024-04-25T10:13:18.787059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:39.282949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061150+0000","last_change":"2024-04-25T10:13:16.058396+0000","last_active":"2024-04-25T10:13:18.061150+0000","last_peered":"2024-04-25T10:13:18.061150+0000","last_clean":"2024-04-25T10:13:18.061150+0000","last_became_active":"2024-04-25T10:13:16.057548+0000","last_became_peered":"2024-04-25T10:13:16.057548+0000","last_unstale":"2024-04-25T10:13:18.061150+0000","last_undegraded":"2024-04-25T10:13:18.061150+0000","last_fullsized":"2024-04-25T10:13:18.061150+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.485272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060696+0000","last_change":"2024-04-25T10:13:17.062419+0000","last_active":"2024-04-25T10:13:18.060696+0000","last_peered":"2024-04-25T10:13:18.060696+0000","last_clean":"2024-04-25T10:13:18.060696+0000","last_became_active":"2024-04-25T10:13:17.062156+0000","last_became_peered":"2024-04-25T10:13:17.062156+0000","last_unstale":"2024-04-25T10:13:18.060696+0000","last_undegraded":"2024-04-25T10:13:18.060696+0000","last_fullsized":"2024-04-25T10:13:18.060696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:14.646796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061071+0000","last_change":"2024-04-25T10:13:16.056396+0000","last_active":"2024-04-25T10:13:18.061071+0000","last_peered":"2024-04-25T10:13:18.061071+0000","last_clean":"2024-04-25T10:13:18.061071+0000","last_became_active":"2024-04-25T10:13:16.056119+0000","last_became_peered":"2024-04-25T10:13:16.056119+0000","last_unstale":"2024-04-25T10:13:18.061071+0000","last_undegraded":"2024-04-25T10:13:18.061071+0000","last_fullsized":"2024-04-25T10:13:18.061071+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:12.752982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060300+0000","last_change":"2024-04-25T10:13:17.063694+0000","last_active":"2024-04-25T10:13:18.060300+0000","last_peered":"2024-04-25T10:13:18.060300+0000","last_clean":"2024-04-25T10:13:18.060300+0000","last_became_active":"2024-04-25T10:13:17.063460+0000","last_became_peered":"2024-04-25T10:13:17.063460+0000","last_unstale":"2024-04-25T10:13:18.060300+0000","last_undegraded":"2024-04-25T10:13:18.060300+0000","last_fullsized":"2024-04-25T10:13:18.060300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:10.343457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786892+0000","last_change":"2024-04-25T10:13:16.063791+0000","last_active":"2024-04-25T10:13:18.786892+0000","last_peered":"2024-04-25T10:13:18.786892+0000","last_clean":"2024-04-25T10:13:18.786892+0000","last_became_active":"2024-04-25T10:13:16.063534+0000","last_became_peered":"2024-04-25T10:13:16.063534+0000","last_unstale":"2024-04-25T10:13:18.786892+0000","last_undegraded":"2024-04-25T10:13:18.786892+0000","last_fullsized":"2024-04-25T10:13:18.786892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:59.194801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059317+0000","last_change":"2024-04-25T10:13:17.073695+0000","last_active":"2024-04-25T10:13:18.059317+0000","last_peered":"2024-04-25T10:13:18.059317+0000","last_clean":"2024-04-25T10:13:18.059317+0000","last_became_active":"2024-04-25T10:13:17.073392+0000","last_became_peered":"2024-04-25T10:13:17.073392+0000","last_unstale":"2024-04-25T10:13:18.059317+0000","last_undegraded":"2024-04-25T10:13:18.059317+0000","last_fullsized":"2024-04-25T10:13:18.059317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:22.985621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060296+0000","last_change":"2024-04-25T10:13:16.061738+0000","last_active":"2024-04-25T10:13:18.060296+0000","last_peered":"2024-04-25T10:13:18.060296+0000","last_clean":"2024-04-25T10:13:18.060296+0000","last_became_active":"2024-04-25T10:13:16.061207+0000","last_became_peered":"2024-04-25T10:13:16.061207+0000","last_unstale":"2024-04-25T10:13:18.060296+0000","last_undegraded":"2024-04-25T10:13:18.060296+0000","last_fullsized":"2024-04-25T10:13:18.060296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:13.025816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061782+0000","last_change":"2024-04-25T10:13:17.058646+0000","last_active":"2024-04-25T10:13:18.061782+0000","last_peered":"2024-04-25T10:13:18.061782+0000","last_clean":"2024-04-25T10:13:18.061782+0000","last_became_active":"2024-04-25T10:13:17.058411+0000","last_became_peered":"2024-04-25T10:13:17.058411+0000","last_unstale":"2024-04-25T10:13:18.061782+0000","last_undegraded":"2024-04-25T10:13:18.061782+0000","last_fullsized":"2024-04-25T10:13:18.061782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:23.488785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787372+0000","last_change":"2024-04-25T10:13:16.067967+0000","last_active":"2024-04-25T10:13:18.787372+0000","last_peered":"2024-04-25T10:13:18.787372+0000","last_clean":"2024-04-25T10:13:18.787372+0000","last_became_active":"2024-04-25T10:13:16.067640+0000","last_became_peered":"2024-04-25T10:13:16.067640+0000","last_unstale":"2024-04-25T10:13:18.787372+0000","last_undegraded":"2024-04-25T10:13:18.787372+0000","last_fullsized":"2024-04-25T10:13:18.787372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:11.345445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787222+0000","last_change":"2024-04-25T10:13:17.081373+0000","last_active":"2024-04-25T10:13:18.787222+0000","last_peered":"2024-04-25T10:13:18.787222+0000","last_clean":"2024-04-25T10:13:18.787222+0000","last_became_active":"2024-04-25T10:13:17.081113+0000","last_became_peered":"2024-04-25T10:13:17.081113+0000","last_unstale":"2024-04-25T10:13:18.787222+0000","last_undegraded":"2024-04-25T10:13:18.787222+0000","last_fullsized":"2024-04-25T10:13:18.787222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:06.565345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788705+0000","last_change":"2024-04-25T10:13:16.058919+0000","last_active":"2024-04-25T10:13:18.788705+0000","last_peered":"2024-04-25T10:13:18.788705+0000","last_clean":"2024-04-25T10:13:18.788705+0000","last_became_active":"2024-04-25T10:13:16.058553+0000","last_became_peered":"2024-04-25T10:13:16.058553+0000","last_unstale":"2024-04-25T10:13:18.788705+0000","last_undegraded":"2024-04-25T10:13:18.788705+0000","last_fullsized":"2024-04-25T10:13:18.788705+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:09.118013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787920+0000","last_change":"2024-04-25T10:13:17.066834+0000","last_active":"2024-04-25T10:13:18.787920+0000","last_peered":"2024-04-25T10:13:18.787920+0000","last_clean":"2024-04-25T10:13:18.787920+0000","last_became_active":"2024-04-25T10:13:17.066524+0000","last_became_peered":"2024-04-25T10:13:17.066524+0000","last_unstale":"2024-04-25T10:13:18.787920+0000","last_undegraded":"2024-04-25T10:13:18.787920+0000","last_fullsized":"2024-04-25T10:13:18.787920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:18.940717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061549+0000","last_change":"2024-04-25T10:13:16.064906+0000","last_active":"2024-04-25T10:13:18.061549+0000","last_peered":"2024-04-25T10:13:18.061549+0000","last_clean":"2024-04-25T10:13:18.061549+0000","last_became_active":"2024-04-25T10:13:16.064562+0000","last_became_peered":"2024-04-25T10:13:16.064562+0000","last_unstale":"2024-04-25T10:13:18.061549+0000","last_undegraded":"2024-04-25T10:13:18.061549+0000","last_fullsized":"2024-04-25T10:13:18.061549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:59.194801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061329+0000","last_change":"2024-04-25T10:13:17.076642+0000","last_active":"2024-04-25T10:13:18.061329+0000","last_peered":"2024-04-25T10:13:18.061329+0000","last_clean":"2024-04-25T10:13:18.061329+0000","last_became_active":"2024-04-25T10:13:17.075845+0000","last_became_peered":"2024-04-25T10:13:17.075845+0000","last_unstale":"2024-04-25T10:13:18.061329+0000","last_undegraded":"2024-04-25T10:13:18.061329+0000","last_fullsized":"2024-04-25T10:13:18.061329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:58.098204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788004+0000","last_change":"2024-04-25T10:13:16.056267+0000","last_active":"2024-04-25T10:13:18.788004+0000","last_peered":"2024-04-25T10:13:18.788004+0000","last_clean":"2024-04-25T10:13:18.788004+0000","last_became_active":"2024-04-25T10:13:16.055804+0000","last_became_peered":"2024-04-25T10:13:16.055804+0000","last_unstale":"2024-04-25T10:13:18.788004+0000","last_undegraded":"2024-04-25T10:13:18.788004+0000","last_fullsized":"2024-04-25T10:13:18.788004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:09.331670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059400+0000","last_change":"2024-04-25T10:13:17.073737+0000","last_active":"2024-04-25T10:13:18.059400+0000","last_peered":"2024-04-25T10:13:18.059400+0000","last_clean":"2024-04-25T10:13:18.059400+0000","last_became_active":"2024-04-25T10:13:17.073428+0000","last_became_peered":"2024-04-25T10:13:17.073428+0000","last_unstale":"2024-04-25T10:13:18.059400+0000","last_undegraded":"2024-04-25T10:13:18.059400+0000","last_fullsized":"2024-04-25T10:13:18.059400+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:21.473698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061197+0000","last_change":"2024-04-25T10:13:16.061578+0000","last_active":"2024-04-25T10:13:18.061197+0000","last_peered":"2024-04-25T10:13:18.061197+0000","last_clean":"2024-04-25T10:13:18.061197+0000","last_became_active":"2024-04-25T10:13:16.061283+0000","last_became_peered":"2024-04-25T10:13:16.061283+0000","last_unstale":"2024-04-25T10:13:18.061197+0000","last_undegraded":"2024-04-25T10:13:18.061197+0000","last_fullsized":"2024-04-25T10:13:18.061197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:42.226386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787199+0000","last_change":"2024-04-25T10:13:17.057859+0000","last_active":"2024-04-25T10:13:18.787199+0000","last_peered":"2024-04-25T10:13:18.787199+0000","last_clean":"2024-04-25T10:13:18.787199+0000","last_became_active":"2024-04-25T10:13:17.057578+0000","last_became_peered":"2024-04-25T10:13:17.057578+0000","last_unstale":"2024-04-25T10:13:18.787199+0000","last_undegraded":"2024-04-25T10:13:18.787199+0000","last_fullsized":"2024-04-25T10:13:18.787199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:23.488785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059385+0000","last_change":"2024-04-25T10:13:16.061513+0000","last_active":"2024-04-25T10:13:18.059385+0000","last_peered":"2024-04-25T10:13:18.059385+0000","last_clean":"2024-04-25T10:13:18.059385+0000","last_became_active":"2024-04-25T10:13:16.061216+0000","last_became_peered":"2024-04-25T10:13:16.061216+0000","last_unstale":"2024-04-25T10:13:18.059385+0000","last_undegraded":"2024-04-25T10:13:18.059385+0000","last_fullsized":"2024-04-25T10:13:18.059385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:08.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787130+0000","last_change":"2024-04-25T10:13:17.065157+0000","last_active":"2024-04-25T10:13:18.787130+0000","last_peered":"2024-04-25T10:13:18.787130+0000","last_clean":"2024-04-25T10:13:18.787130+0000","last_became_active":"2024-04-25T10:13:17.064501+0000","last_became_peered":"2024-04-25T10:13:17.064501+0000","last_unstale":"2024-04-25T10:13:18.787130+0000","last_undegraded":"2024-04-25T10:13:18.787130+0000","last_fullsized":"2024-04-25T10:13:18.787130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:32.738164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786959+0000","last_change":"2024-04-25T10:13:16.063235+0000","last_active":"2024-04-25T10:13:18.786959+0000","last_peered":"2024-04-25T10:13:18.786959+0000","last_clean":"2024-04-25T10:13:18.786959+0000","last_became_active":"2024-04-25T10:13:16.062820+0000","last_became_peered":"2024-04-25T10:13:16.062820+0000","last_unstale":"2024-04-25T10:13:18.786959+0000","last_undegraded":"2024-04-25T10:13:18.786959+0000","last_fullsized":"2024-04-25T10:13:18.786959+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:08.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786787+0000","last_change":"2024-04-25T10:13:17.062972+0000","last_active":"2024-04-25T10:13:18.786787+0000","last_peered":"2024-04-25T10:13:18.786787+0000","last_clean":"2024-04-25T10:13:18.786787+0000","last_became_active":"2024-04-25T10:13:17.062704+0000","last_became_peered":"2024-04-25T10:13:17.062704+0000","last_unstale":"2024-04-25T10:13:18.786787+0000","last_undegraded":"2024-04-25T10:13:18.786787+0000","last_fullsized":"2024-04-25T10:13:18.786787+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:09.070657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786828+0000","last_change":"2024-04-25T10:13:16.057566+0000","last_active":"2024-04-25T10:13:18.786828+0000","last_peered":"2024-04-25T10:13:18.786828+0000","last_clean":"2024-04-25T10:13:18.786828+0000","last_became_active":"2024-04-25T10:13:16.055961+0000","last_became_peered":"2024-04-25T10:13:16.055961+0000","last_unstale":"2024-04-25T10:13:18.786828+0000","last_undegraded":"2024-04-25T10:13:18.786828+0000","last_fullsized":"2024-04-25T10:13:18.786828+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:08.058870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061429+0000","last_change":"2024-04-25T10:13:17.053406+0000","last_active":"2024-04-25T10:13:18.061429+0000","last_peered":"2024-04-25T10:13:18.061429+0000","last_clean":"2024-04-25T10:13:18.061429+0000","last_became_active":"2024-04-25T10:13:17.053124+0000","last_became_peered":"2024-04-25T10:13:17.053124+0000","last_unstale":"2024-04-25T10:13:18.061429+0000","last_undegraded":"2024-04-25T10:13:18.061429+0000","last_fullsized":"2024-04-25T10:13:18.061429+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:22.985621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060159+0000","last_change":"2024-04-25T10:13:16.058726+0000","last_active":"2024-04-25T10:13:18.060159+0000","last_peered":"2024-04-25T10:13:18.060159+0000","last_clean":"2024-04-25T10:13:18.060159+0000","last_became_active":"2024-04-25T10:13:16.058253+0000","last_became_peered":"2024-04-25T10:13:16.058253+0000","last_unstale":"2024-04-25T10:13:18.060159+0000","last_undegraded":"2024-04-25T10:13:18.060159+0000","last_fullsized":"2024-04-25T10:13:18.060159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:18.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061886+0000","last_change":"2024-04-25T10:13:17.052649+0000","last_active":"2024-04-25T10:13:18.061886+0000","last_peered":"2024-04-25T10:13:18.061886+0000","last_clean":"2024-04-25T10:13:18.061886+0000","last_became_active":"2024-04-25T10:13:17.052337+0000","last_became_peered":"2024-04-25T10:13:17.052337+0000","last_unstale":"2024-04-25T10:13:18.061886+0000","last_undegraded":"2024-04-25T10:13:18.061886+0000","last_fullsized":"2024-04-25T10:13:18.061886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:06.565345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061400+0000","last_change":"2024-04-25T10:13:16.062295+0000","last_active":"2024-04-25T10:13:18.061400+0000","last_peered":"2024-04-25T10:13:18.061400+0000","last_clean":"2024-04-25T10:13:18.061400+0000","last_became_active":"2024-04-25T10:13:16.061270+0000","last_became_peered":"2024-04-25T10:13:16.061270+0000","last_unstale":"2024-04-25T10:13:18.061400+0000","last_undegraded":"2024-04-25T10:13:18.061400+0000","last_fullsized":"2024-04-25T10:13:18.061400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059502+0000","last_change":"2024-04-25T10:13:17.077411+0000","last_active":"2024-04-25T10:13:18.059502+0000","last_peered":"2024-04-25T10:13:18.059502+0000","last_clean":"2024-04-25T10:13:18.059502+0000","last_became_active":"2024-04-25T10:13:17.065203+0000","last_became_peered":"2024-04-25T10:13:17.065203+0000","last_unstale":"2024-04-25T10:13:18.059502+0000","last_undegraded":"2024-04-25T10:13:18.059502+0000","last_fullsized":"2024-04-25T10:13:18.059502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:20.999645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061567+0000","last_change":"2024-04-25T10:13:16.055470+0000","last_active":"2024-04-25T10:13:18.061567+0000","last_peered":"2024-04-25T10:13:18.061567+0000","last_clean":"2024-04-25T10:13:18.061567+0000","last_became_active":"2024-04-25T10:13:16.055137+0000","last_became_peered":"2024-04-25T10:13:16.055137+0000","last_unstale":"2024-04-25T10:13:18.061567+0000","last_undegraded":"2024-04-25T10:13:18.061567+0000","last_fullsized":"2024-04-25T10:13:18.061567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:16.640757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787672+0000","last_change":"2024-04-25T10:13:17.071509+0000","last_active":"2024-04-25T10:13:18.787672+0000","last_peered":"2024-04-25T10:13:18.787672+0000","last_clean":"2024-04-25T10:13:18.787672+0000","last_became_active":"2024-04-25T10:13:17.070619+0000","last_became_peered":"2024-04-25T10:13:17.070619+0000","last_unstale":"2024-04-25T10:13:18.787672+0000","last_undegraded":"2024-04-25T10:13:18.787672+0000","last_fullsized":"2024-04-25T10:13:18.787672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:12.138732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061295+0000","last_change":"2024-04-25T10:13:16.057838+0000","last_active":"2024-04-25T10:13:18.061295+0000","last_peered":"2024-04-25T10:13:18.061295+0000","last_clean":"2024-04-25T10:13:18.061295+0000","last_became_active":"2024-04-25T10:13:16.057476+0000","last_became_peered":"2024-04-25T10:13:16.057476+0000","last_unstale":"2024-04-25T10:13:18.061295+0000","last_undegraded":"2024-04-25T10:13:18.061295+0000","last_fullsized":"2024-04-25T10:13:18.061295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:26.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787997+0000","last_change":"2024-04-25T10:13:17.070014+0000","last_active":"2024-04-25T10:13:18.787997+0000","last_peered":"2024-04-25T10:13:18.787997+0000","last_clean":"2024-04-25T10:13:18.787997+0000","last_became_active":"2024-04-25T10:13:17.069586+0000","last_became_peered":"2024-04-25T10:13:17.069586+0000","last_unstale":"2024-04-25T10:13:18.787997+0000","last_undegraded":"2024-04-25T10:13:18.787997+0000","last_fullsized":"2024-04-25T10:13:18.787997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.859203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788606+0000","last_change":"2024-04-25T10:13:16.065980+0000","last_active":"2024-04-25T10:13:18.788606+0000","last_peered":"2024-04-25T10:13:18.788606+0000","last_clean":"2024-04-25T10:13:18.788606+0000","last_became_active":"2024-04-25T10:13:16.065704+0000","last_became_peered":"2024-04-25T10:13:16.065704+0000","last_unstale":"2024-04-25T10:13:18.788606+0000","last_undegraded":"2024-04-25T10:13:18.788606+0000","last_fullsized":"2024-04-25T10:13:18.788606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:18.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787480+0000","last_change":"2024-04-25T10:13:17.068318+0000","last_active":"2024-04-25T10:13:18.787480+0000","last_peered":"2024-04-25T10:13:18.787480+0000","last_clean":"2024-04-25T10:13:18.787480+0000","last_became_active":"2024-04-25T10:13:17.068044+0000","last_became_peered":"2024-04-25T10:13:17.068044+0000","last_unstale":"2024-04-25T10:13:18.787480+0000","last_undegraded":"2024-04-25T10:13:18.787480+0000","last_fullsized":"2024-04-25T10:13:18.787480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:43.238172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787102+0000","last_change":"2024-04-25T10:13:16.065367+0000","last_active":"2024-04-25T10:13:18.787102+0000","last_peered":"2024-04-25T10:13:18.787102+0000","last_clean":"2024-04-25T10:13:18.787102+0000","last_became_active":"2024-04-25T10:13:16.065198+0000","last_became_peered":"2024-04-25T10:13:16.065198+0000","last_unstale":"2024-04-25T10:13:18.787102+0000","last_undegraded":"2024-04-25T10:13:18.787102+0000","last_fullsized":"2024-04-25T10:13:18.787102+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:11.126945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788090+0000","last_change":"2024-04-25T10:13:17.056344+0000","last_active":"2024-04-25T10:13:18.788090+0000","last_peered":"2024-04-25T10:13:18.788090+0000","last_clean":"2024-04-25T10:13:18.788090+0000","last_became_active":"2024-04-25T10:13:17.055945+0000","last_became_peered":"2024-04-25T10:13:17.055945+0000","last_unstale":"2024-04-25T10:13:18.788090+0000","last_undegraded":"2024-04-25T10:13:18.788090+0000","last_fullsized":"2024-04-25T10:13:18.788090+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:10.129800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060135+0000","last_change":"2024-04-25T10:13:16.062433+0000","last_active":"2024-04-25T10:13:18.060135+0000","last_peered":"2024-04-25T10:13:18.060135+0000","last_clean":"2024-04-25T10:13:18.060135+0000","last_became_active":"2024-04-25T10:13:16.062227+0000","last_became_peered":"2024-04-25T10:13:16.062227+0000","last_unstale":"2024-04-25T10:13:18.060135+0000","last_undegraded":"2024-04-25T10:13:18.060135+0000","last_fullsized":"2024-04-25T10:13:18.060135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:14.008832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787036+0000","last_change":"2024-04-25T10:13:17.066221+0000","last_active":"2024-04-25T10:13:18.787036+0000","last_peered":"2024-04-25T10:13:18.787036+0000","last_clean":"2024-04-25T10:13:18.787036+0000","last_became_active":"2024-04-25T10:13:17.065912+0000","last_became_peered":"2024-04-25T10:13:17.065912+0000","last_unstale":"2024-04-25T10:13:18.787036+0000","last_undegraded":"2024-04-25T10:13:18.787036+0000","last_fullsized":"2024-04-25T10:13:18.787036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:14.646796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060807+0000","last_change":"2024-04-25T10:13:16.057718+0000","last_active":"2024-04-25T10:13:18.060807+0000","last_peered":"2024-04-25T10:13:18.060807+0000","last_clean":"2024-04-25T10:13:18.060807+0000","last_became_active":"2024-04-25T10:13:16.057007+0000","last_became_peered":"2024-04-25T10:13:16.057007+0000","last_unstale":"2024-04-25T10:13:18.060807+0000","last_undegraded":"2024-04-25T10:13:18.060807+0000","last_fullsized":"2024-04-25T10:13:18.060807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:08.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787548+0000","last_change":"2024-04-25T10:13:17.066896+0000","last_active":"2024-04-25T10:13:18.787548+0000","last_peered":"2024-04-25T10:13:18.787548+0000","last_clean":"2024-04-25T10:13:18.787548+0000","last_became_active":"2024-04-25T10:13:17.066594+0000","last_became_peered":"2024-04-25T10:13:17.066594+0000","last_unstale":"2024-04-25T10:13:18.787548+0000","last_undegraded":"2024-04-25T10:13:18.787548+0000","last_fullsized":"2024-04-25T10:13:18.787548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:17.652543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060173+0000","last_change":"2024-04-25T10:13:16.051248+0000","last_active":"2024-04-25T10:13:18.060173+0000","last_peered":"2024-04-25T10:13:18.060173+0000","last_clean":"2024-04-25T10:13:18.060173+0000","last_became_active":"2024-04-25T10:13:16.050951+0000","last_became_peered":"2024-04-25T10:13:16.050951+0000","last_unstale":"2024-04-25T10:13:18.060173+0000","last_undegraded":"2024-04-25T10:13:18.060173+0000","last_fullsized":"2024-04-25T10:13:18.060173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.930692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.061938+0000","last_change":"2024-04-25T10:13:17.081518+0000","last_active":"2024-04-25T10:13:18.061938+0000","last_peered":"2024-04-25T10:13:18.061938+0000","last_clean":"2024-04-25T10:13:18.061938+0000","last_became_active":"2024-04-25T10:13:17.081001+0000","last_became_peered":"2024-04-25T10:13:17.081001+0000","last_unstale":"2024-04-25T10:13:18.061938+0000","last_undegraded":"2024-04-25T10:13:18.061938+0000","last_fullsized":"2024-04-25T10:13:18.061938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:57.954125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788698+0000","last_change":"2024-04-25T10:13:16.066644+0000","last_active":"2024-04-25T10:13:18.788698+0000","last_peered":"2024-04-25T10:13:18.788698+0000","last_clean":"2024-04-25T10:13:18.788698+0000","last_became_active":"2024-04-25T10:13:16.064572+0000","last_became_peered":"2024-04-25T10:13:16.064572+0000","last_unstale":"2024-04-25T10:13:18.788698+0000","last_undegraded":"2024-04-25T10:13:18.788698+0000","last_fullsized":"2024-04-25T10:13:18.788698+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:08.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788596+0000","last_change":"2024-04-25T10:13:17.070359+0000","last_active":"2024-04-25T10:13:18.788596+0000","last_peered":"2024-04-25T10:13:18.788596+0000","last_clean":"2024-04-25T10:13:18.788596+0000","last_became_active":"2024-04-25T10:13:17.070045+0000","last_became_peered":"2024-04-25T10:13:17.070045+0000","last_unstale":"2024-04-25T10:13:18.788596+0000","last_undegraded":"2024-04-25T10:13:18.788596+0000","last_fullsized":"2024-04-25T10:13:18.788596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:51.952858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059603+0000","last_change":"2024-04-25T10:13:16.057038+0000","last_active":"2024-04-25T10:13:18.059603+0000","last_peered":"2024-04-25T10:13:18.059603+0000","last_clean":"2024-04-25T10:13:18.059603+0000","last_became_active":"2024-04-25T10:13:16.056758+0000","last_became_peered":"2024-04-25T10:13:16.056758+0000","last_unstale":"2024-04-25T10:13:18.059603+0000","last_undegraded":"2024-04-25T10:13:18.059603+0000","last_fullsized":"2024-04-25T10:13:18.059603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:25.604894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786956+0000","last_change":"2024-04-25T10:13:17.060592+0000","last_active":"2024-04-25T10:13:18.786956+0000","last_peered":"2024-04-25T10:13:18.786956+0000","last_clean":"2024-04-25T10:13:18.786956+0000","last_became_active":"2024-04-25T10:13:17.059645+0000","last_became_peered":"2024-04-25T10:13:17.059645+0000","last_unstale":"2024-04-25T10:13:18.786956+0000","last_undegraded":"2024-04-25T10:13:18.786956+0000","last_fullsized":"2024-04-25T10:13:18.786956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:10.343457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788757+0000","last_change":"2024-04-25T10:13:16.055447+0000","last_active":"2024-04-25T10:13:18.788757+0000","last_peered":"2024-04-25T10:13:18.788757+0000","last_clean":"2024-04-25T10:13:18.788757+0000","last_became_active":"2024-04-25T10:13:16.055157+0000","last_became_peered":"2024-04-25T10:13:16.055157+0000","last_unstale":"2024-04-25T10:13:18.788757+0000","last_undegraded":"2024-04-25T10:13:18.788757+0000","last_fullsized":"2024-04-25T10:13:18.788757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:25.604894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.788610+0000","last_change":"2024-04-25T10:13:17.060933+0000","last_active":"2024-04-25T10:13:18.788610+0000","last_peered":"2024-04-25T10:13:18.788610+0000","last_clean":"2024-04-25T10:13:18.788610+0000","last_became_active":"2024-04-25T10:13:17.060675+0000","last_became_peered":"2024-04-25T10:13:17.060675+0000","last_unstale":"2024-04-25T10:13:18.788610+0000","last_undegraded":"2024-04-25T10:13:18.788610+0000","last_fullsized":"2024-04-25T10:13:18.788610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:12.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.059466+0000","last_change":"2024-04-25T10:13:16.049162+0000","last_active":"2024-04-25T10:13:18.059466+0000","last_peered":"2024-04-25T10:13:18.059466+0000","last_clean":"2024-04-25T10:13:18.059466+0000","last_became_active":"2024-04-25T10:13:16.048699+0000","last_became_peered":"2024-04-25T10:13:16.048699+0000","last_unstale":"2024-04-25T10:13:18.059466+0000","last_undegraded":"2024-04-25T10:13:18.059466+0000","last_fullsized":"2024-04-25T10:13:18.059466+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:09.118013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786730+0000","last_change":"2024-04-25T10:13:17.077189+0000","last_active":"2024-04-25T10:13:18.786730+0000","last_peered":"2024-04-25T10:13:18.786730+0000","last_clean":"2024-04-25T10:13:18.786730+0000","last_became_active":"2024-04-25T10:13:17.077019+0000","last_became_peered":"2024-04-25T10:13:17.077019+0000","last_unstale":"2024-04-25T10:13:18.786730+0000","last_undegraded":"2024-04-25T10:13:18.786730+0000","last_fullsized":"2024-04-25T10:13:18.786730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:12.381668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060234+0000","last_change":"2024-04-25T10:13:16.052633+0000","last_active":"2024-04-25T10:13:18.060234+0000","last_peered":"2024-04-25T10:13:18.060234+0000","last_clean":"2024-04-25T10:13:18.060234+0000","last_became_active":"2024-04-25T10:13:16.052308+0000","last_became_peered":"2024-04-25T10:13:16.052308+0000","last_unstale":"2024-04-25T10:13:18.060234+0000","last_undegraded":"2024-04-25T10:13:18.060234+0000","last_fullsized":"2024-04-25T10:13:18.060234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:33.833365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.060143+0000","last_change":"2024-04-25T10:13:17.059659+0000","last_active":"2024-04-25T10:13:18.060143+0000","last_peered":"2024-04-25T10:13:18.060143+0000","last_clean":"2024-04-25T10:13:18.060143+0000","last_became_active":"2024-04-25T10:13:17.059332+0000","last_became_peered":"2024-04-25T10:13:17.059332+0000","last_unstale":"2024-04-25T10:13:18.060143+0000","last_undegraded":"2024-04-25T10:13:18.060143+0000","last_fullsized":"2024-04-25T10:13:18.060143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:09.070657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786749+0000","last_change":"2024-04-25T10:13:16.065524+0000","last_active":"2024-04-25T10:13:18.786749+0000","last_peered":"2024-04-25T10:13:18.786749+0000","last_clean":"2024-04-25T10:13:18.786749+0000","last_became_active":"2024-04-25T10:13:16.065056+0000","last_became_peered":"2024-04-25T10:13:16.065056+0000","last_unstale":"2024-04-25T10:13:18.786749+0000","last_undegraded":"2024-04-25T10:13:18.786749+0000","last_fullsized":"2024-04-25T10:13:18.786749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787305+0000","last_change":"2024-04-25T10:13:17.063185+0000","last_active":"2024-04-25T10:13:18.787305+0000","last_peered":"2024-04-25T10:13:18.787305+0000","last_clean":"2024-04-25T10:13:18.787305+0000","last_became_active":"2024-04-25T10:13:17.062927+0000","last_became_peered":"2024-04-25T10:13:17.062927+0000","last_unstale":"2024-04-25T10:13:18.787305+0000","last_undegraded":"2024-04-25T10:13:18.787305+0000","last_fullsized":"2024-04-25T10:13:18.787305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:16.029897+0000","last_clean_scrub_stamp":"2024-04-25T10:13:16.029897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:58.098204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.786634+0000","last_change":"2024-04-25T10:13:16.066995+0000","last_active":"2024-04-25T10:13:18.786634+0000","last_peered":"2024-04-25T10:13:18.786634+0000","last_clean":"2024-04-25T10:13:18.786634+0000","last_became_active":"2024-04-25T10:13:16.066686+0000","last_became_peered":"2024-04-25T10:13:16.066686+0000","last_unstale":"2024-04-25T10:13:18.786634+0000","last_undegraded":"2024-04-25T10:13:18.786634+0000","last_fullsized":"2024-04-25T10:13:18.786634+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:15.018110+0000","last_clean_scrub_stamp":"2024-04-25T10:13:15.018110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:04.739957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T10:13:18.787562+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:13:18.787562+0000","last_peered":"2024-04-25T10:13:18.787562+0000","last_clean":"2024-04-25T10:13:18.787562+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:13:18.787562+0000","last_undegraded":"2024-04-25T10:13:18.787562+0000","last_fullsized":"2024-04-25T10:13:18.787562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:25.985 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:13:25.985 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:13:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edd600640 1 Processor -- start 2024-04-25T10:13:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edd600640 1 -- start start 2024-04-25T10:13:26.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edd600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ed8074210 0x7f2ed80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edd600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ed80cc370 0x7f2ed80cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edd600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed80cd870 con 0x7f2ed815bdc0 2024-04-25T10:13:26.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edd600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed80ca910 con 0x7f2ed80cc370 2024-04-25T10:13:26.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edd600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed80cabf0 con 0x7f2ed8074210 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edcc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ed80cc370 0x7f2ed80cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edcc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ed80cc370 0x7f2ed80cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44966/0 (socket says 172.21.15.87:44966) 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edcc00640 1 -- 172.21.15.87:0/1982495547 learned_addr learned my addr 172.21.15.87:0/1982495547 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2ed7400640 1 --2- 172.21.15.87:0/1982495547 >> v2:172.21.15.169:3300/0 conn(0x7f2ed8074210 0x7f2ed80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edcc00640 1 -- 172.21.15.87:0/1982495547 >> v2:172.21.15.169:3300/0 conn(0x7f2ed8074210 msgr2=0x7f2ed80728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edcc00640 1 --2- 172.21.15.87:0/1982495547 >> v2:172.21.15.169:3300/0 conn(0x7f2ed8074210 0x7f2ed80728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edcc00640 1 -- 172.21.15.87:0/1982495547 >> v1:172.21.15.169:6789/0 conn(0x7f2ed815bdc0 legacy=0x7f2ed80cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.162+0000 7f2edcc00640 1 -- 172.21.15.87:0/1982495547 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed80caed0 con 0x7f2ed80cc370 2024-04-25T10:13:26.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edcc00640 1 --2- 172.21.15.87:0/1982495547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed80cc370 0x7f2ed80cddf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80b2ae0 tx=0x7f2ed001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620fff318837c5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2ed6a00640 1 -- 172.21.15.87:0/1982495547 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00a58a0 con 0x7f2ed80cc370 2024-04-25T10:13:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2ed6a00640 1 -- 172.21.15.87:0/1982495547 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed00a5a40 con 0x7f2ed80cc370 2024-04-25T10:13:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2ed6a00640 1 -- 172.21.15.87:0/1982495547 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00a5e10 con 0x7f2ed80cc370 2024-04-25T10:13:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 -- 172.21.15.87:0/1982495547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed80cc370 msgr2=0x7f2ed80cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 --2- 172.21.15.87:0/1982495547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed80cc370 0x7f2ed80cddf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80b2ae0 tx=0x7f2ed001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 -- 172.21.15.87:0/1982495547 shutdown_connections 2024-04-25T10:13:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 --2- 172.21.15.87:0/1982495547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed80cc370 0x7f2ed80cddf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 --2- 172.21.15.87:0/1982495547 >> v2:172.21.15.169:3300/0 conn(0x7f2ed8074210 0x7f2ed80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 -- 172.21.15.87:0/1982495547 >> 172.21.15.87:0/1982495547 conn(0x7f2ed80cc970 msgr2=0x7f2ed80ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 -- 172.21.15.87:0/1982495547 shutdown_connections 2024-04-25T10:13:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.166+0000 7f2edd600640 1 -- 172.21.15.87:0/1982495547 wait complete. 2024-04-25T10:13:26.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 Processor -- start 2024-04-25T10:13:26.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 -- start start 2024-04-25T10:13:26.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed8074210 0x7f2ed8153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ed80cc370 0x7f2ed814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ed815bdc0 0x7f2ed806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed8168d30 con 0x7f2ed8074210 2024-04-25T10:13:26.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed8168bb0 con 0x7f2ed80cc370 2024-04-25T10:13:26.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed8168eb0 con 0x7f2ed815bdc0 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed8074210 0x7f2ed8153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed8074210 0x7f2ed8153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44968/0 (socket says 172.21.15.87:44968) 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 -- 172.21.15.87:0/589135675 learned_addr learned my addr 172.21.15.87:0/589135675 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7400640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ed80cc370 0x7f2ed814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edcc00640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ed815bdc0 0x7f2ed806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 -- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ed815bdc0 msgr2=0x7f2ed806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ed815bdc0 0x7f2ed806db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 -- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ed80cc370 msgr2=0x7f2ed814f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ed80cc370 0x7f2ed814f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 -- 172.21.15.87:0/589135675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8153880 con 0x7f2ed8074210 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ed7e00640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed8074210 0x7f2ed8153340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c420 tx=0x7f2ec800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ecfe00640 1 -- 172.21.15.87:0/589135675 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80052d0 con 0x7f2ed8074210 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ecfe00640 1 -- 172.21.15.87:0/589135675 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec8005470 con 0x7f2ed8074210 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2ecfe00640 1 -- 172.21.15.87:0/589135675 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8007260 con 0x7f2ed8074210 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed806e570 con 0x7f2ed8074210 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.170+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed80afdc0 con 0x7f2ed8074210 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.174+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed8067940 con 0x7f2ed8074210 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.174+0000 7f2ecfe00640 1 -- 172.21.15.87:0/589135675 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ec8007400 con 0x7f2ed8074210 2024-04-25T10:13:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.174+0000 7f2ecfe00640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2eb8066f90 0x7f2eb8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.174+0000 7f2ecfe00640 1 -- 172.21.15.87:0/589135675 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ec8084440 con 0x7f2ed8074210 2024-04-25T10:13:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.174+0000 7f2ed7400640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2eb8066f90 0x7f2eb8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.174+0000 7f2ed7400640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2eb8066f90 0x7f2eb8069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ed806cc80 tx=0x7f2ec4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.178+0000 7f2ecfe00640 1 -- 172.21.15.87:0/589135675 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ec8054df0 con 0x7f2ed8074210 2024-04-25T10:13:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.386+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2ed8075120 con 0x7f2ed8074210 2024-04-25T10:13:26.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2ecfe00640 1 -- 172.21.15.87:0/589135675 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2ec8058e90 con 0x7f2ed8074210 2024-04-25T10:13:26.397 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:13:26.397 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:13:26.397 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:13:26.397 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:13:26.397 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:26.398 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:13:26.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2eb8066f90 msgr2=0x7f2eb8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2eb8066f90 0x7f2eb8069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ed806cc80 tx=0x7f2ec4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:26.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed8074210 msgr2=0x7f2ed8153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed8074210 0x7f2ed8153340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c420 tx=0x7f2ec800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 shutdown_connections 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2eb8066f90 0x7f2eb8069450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ed815bdc0 0x7f2ed806db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ed80cc370 0x7f2ed814f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 --2- 172.21.15.87:0/589135675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed8074210 0x7f2ed8153340 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 >> 172.21.15.87:0/589135675 conn(0x7f2ed80cc970 msgr2=0x7f2ed80725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 shutdown_connections 2024-04-25T10:13:26.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.390+0000 7f2edd600640 1 -- 172.21.15.87:0/589135675 wait complete. 2024-04-25T10:13:26.419 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:26.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.546+0000 7fe606e00640 1 Processor -- start 2024-04-25T10:13:26.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.546+0000 7fe606e00640 1 -- start start 2024-04-25T10:13:26.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe606e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6000b57e0 0x7fe600162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe606e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe6000669a0 0x7fe600066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe606e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6000b0210 con 0x7fe60015dab0 2024-04-25T10:13:26.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe606e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6000aff30 con 0x7fe6000669a0 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe606e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe600161840 con 0x7fe6000b57e0 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe604e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe6000669a0 0x7fe600066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe5ffe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe60015dab0 0x7fe6000af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56748/0 (socket says 172.21.15.87:56748) 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe5ffe00640 1 -- 172.21.15.87:0/2621264653 learned_addr learned my addr 172.21.15.87:0/2621264653 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe5f7400640 1 --2- 172.21.15.87:0/2621264653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6000b57e0 0x7fe600162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe604e00640 1 -- 172.21.15.87:0/2621264653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6000b57e0 msgr2=0x7fe600162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe604e00640 1 --2- 172.21.15.87:0/2621264653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6000b57e0 0x7fe600162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe604e00640 1 -- 172.21.15.87:0/2621264653 >> v1:172.21.15.87:6789/0 conn(0x7fe60015dab0 legacy=0x7fe6000af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.550+0000 7fe604e00640 1 -- 172.21.15.87:0/2621264653 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600161b20 con 0x7fe6000669a0 2024-04-25T10:13:26.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe604e00640 1 --2- 172.21.15.87:0/2621264653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000669a0 0x7fe600066e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009d90 tx=0x7fe5f801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2867dcc754005344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe5ff400640 1 -- 172.21.15.87:0/2621264653 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80b2070 con 0x7fe6000669a0 2024-04-25T10:13:26.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe5ff400640 1 -- 172.21.15.87:0/2621264653 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f80a5830 con 0x7fe6000669a0 2024-04-25T10:13:26.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe5ff400640 1 -- 172.21.15.87:0/2621264653 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80a5c00 con 0x7fe6000669a0 2024-04-25T10:13:26.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 -- 172.21.15.87:0/2621264653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000669a0 msgr2=0x7fe600066e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 --2- 172.21.15.87:0/2621264653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000669a0 0x7fe600066e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009d90 tx=0x7fe5f801c970 comp rx=0 tx=0).stop 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 -- 172.21.15.87:0/2621264653 shutdown_connections 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 --2- 172.21.15.87:0/2621264653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000669a0 0x7fe600066e30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 --2- 172.21.15.87:0/2621264653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6000b57e0 0x7fe600162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 -- 172.21.15.87:0/2621264653 >> 172.21.15.87:0/2621264653 conn(0x7fe60006ac90 msgr2=0x7fe60006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 -- 172.21.15.87:0/2621264653 shutdown_connections 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 -- 172.21.15.87:0/2621264653 wait complete. 2024-04-25T10:13:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 Processor -- start 2024-04-25T10:13:26.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.554+0000 7fe606e00640 1 -- start start 2024-04-25T10:13:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6000669a0 0x7fe60014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000b57e0 0x7fe6001473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe60015dab0 0x7fe600160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000d10e0 con 0x7fe6000b57e0 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000d0f60 con 0x7fe6000669a0 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000d1260 con 0x7fe60015dab0 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000b57e0 0x7fe6001473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000b57e0 0x7fe6001473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44980/0 (socket says 172.21.15.87:44980) 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 -- 172.21.15.87:0/857446669 learned_addr learned my addr 172.21.15.87:0/857446669 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5ffe00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6000669a0 0x7fe60014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe604e00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe60015dab0 0x7fe600160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 -- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe60015dab0 msgr2=0x7fe600160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe60015dab0 0x7fe600160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 -- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6000669a0 msgr2=0x7fe60014cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6000669a0 0x7fe60014cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 -- 172.21.15.87:0/857446669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60014bed0 con 0x7fe6000b57e0 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5f7400640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000b57e0 0x7fe6001473e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800d800 tx=0x7fe5e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5fcc00640 1 -- 172.21.15.87:0/857446669 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800b6c0 con 0x7fe6000b57e0 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5fcc00640 1 -- 172.21.15.87:0/857446669 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e8019070 con 0x7fe6000b57e0 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5fcc00640 1 -- 172.21.15.87:0/857446669 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800ba60 con 0x7fe6000b57e0 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60014c1b0 con 0x7fe6000b57e0 2024-04-25T10:13:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60014c770 con 0x7fe6000b57e0 2024-04-25T10:13:26.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60004ef30 con 0x7fe6000b57e0 2024-04-25T10:13:26.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.558+0000 7fe5fcc00640 1 -- 172.21.15.87:0/857446669 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5e800bc00 con 0x7fe6000b57e0 2024-04-25T10:13:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.562+0000 7fe5fcc00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe5dc066f60 0x7fe5dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.562+0000 7fe5fcc00640 1 -- 172.21.15.87:0/857446669 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5e80854d0 con 0x7fe6000b57e0 2024-04-25T10:13:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.562+0000 7fe5ffe00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe5dc066f60 0x7fe5dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.562+0000 7fe5ffe00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe5dc066f60 0x7fe5dc069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0000f70 tx=0x7fe5f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.566+0000 7fe5fcc00640 1 -- 172.21.15.87:0/857446669 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5e80564e0 con 0x7fe6000b57e0 2024-04-25T10:13:26.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.750+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6000719b0 con 0x7fe6000b57e0 2024-04-25T10:13:26.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.750+0000 7fe5fcc00640 1 -- 172.21.15.87:0/857446669 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fe5e8083e20 con 0x7fe6000b57e0 2024-04-25T10:13:26.758 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:13:26.758 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3059476782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3059476782},{"type":"v1","addr":"172.21.15.169:6834","nonce":3059476782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/685777206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":685777206},{"type":"v1","addr":"172.21.15.87:6837","nonce":685777206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2281421407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2281421407},{"type":"v1","addr":"172.21.15.169:6835","nonce":2281421407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:16.421834+0000","modified":"2024-04-25T10:13:19.431751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/4156145333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4156145333},{"type":"v1","addr":"172.21.15.87:6835","nonce":4156145333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:13:26.759 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 16 2024-04-25T10:13:26.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe5dc066f60 msgr2=0x7fe5dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe5dc066f60 0x7fe5dc069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0000f70 tx=0x7fe5f0009040 comp rx=0 tx=0).stop 2024-04-25T10:13:26.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000b57e0 msgr2=0x7fe6001473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000b57e0 0x7fe6001473e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800d800 tx=0x7fe5e800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 shutdown_connections 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe5dc066f60 0x7fe5dc069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe60015dab0 0x7fe600160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6000b57e0 0x7fe6001473e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 --2- 172.21.15.87:0/857446669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6000669a0 0x7fe60014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 >> 172.21.15.87:0/857446669 conn(0x7fe60006ac90 msgr2=0x7fe60006e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 shutdown_connections 2024-04-25T10:13:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:26.754+0000 7fe606e00640 1 -- 172.21.15.87:0/857446669 wait complete. 2024-04-25T10:13:27.181 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:16.421834+0000', 'modified': '2024-04-25T10:13:19.431751+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5127': {'gid': 5127, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6835/4156145333', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 4156145333}, {'type': 'v1', 'addr': '172.21.15.87:6835', 'nonce': 4156145333}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:27.182 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:27.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:13:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ffa800640 1 Processor -- start 2024-04-25T10:13:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ffa800640 1 -- start start 2024-04-25T10:13:27.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ffa800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3ff4071170 0x7f3ff406cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ffa800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff40ad9e0 con 0x7f3ff4073ea0 2024-04-25T10:13:27.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ffa800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff40b40a0 con 0x7f3ff415db00 2024-04-25T10:13:27.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ffa800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff40b4380 con 0x7f3ff4071170 2024-04-25T10:13:27.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ff8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3ff4071170 0x7f3ff406cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ff9e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3ff4073ea0 0x7f3ff40ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56760/0 (socket says 172.21.15.87:56760) 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ff9e00640 1 -- 172.21.15.87:0/3200036311 learned_addr learned my addr 172.21.15.87:0/3200036311 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ff8a00640 1 -- 172.21.15.87:0/3200036311 >> v1:172.21.15.169:6789/0 conn(0x7f3ff415db00 legacy=0x7f3ff4070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ff8a00640 1 -- 172.21.15.87:0/3200036311 >> v1:172.21.15.87:6789/0 conn(0x7f3ff4073ea0 legacy=0x7f3ff40ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ff8a00640 1 -- 172.21.15.87:0/3200036311 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40adc80 con 0x7f3ff4071170 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.314+0000 7f3ff3e00640 1 -- 172.21.15.87:0/3200036311 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496178310 0 0) 0x7f3ff40ad9e0 con 0x7f3ff4073ea0 2024-04-25T10:13:27.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/3200036311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff4071170 0x7f3ff406cc00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00097c0 tx=0x7f3fe001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f43421a74590d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ff3e00640 1 -- 172.21.15.87:0/3200036311 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00b1070 con 0x7f3ff4071170 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ff3e00640 1 -- 172.21.15.87:0/3200036311 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00ac5f0 con 0x7f3ff4071170 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ff3e00640 1 -- 172.21.15.87:0/3200036311 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00ac9c0 con 0x7f3ff4071170 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ffa800640 1 -- 172.21.15.87:0/3200036311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff4071170 msgr2=0x7f3ff406cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3200036311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff4071170 0x7f3ff406cc00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00097c0 tx=0x7f3fe001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ffa800640 1 -- 172.21.15.87:0/3200036311 shutdown_connections 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3200036311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff4071170 0x7f3ff406cc00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ffa800640 1 -- 172.21.15.87:0/3200036311 >> 172.21.15.87:0/3200036311 conn(0x7f3ff40b1b50 msgr2=0x7f3ff40b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ffa800640 1 -- 172.21.15.87:0/3200036311 shutdown_connections 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.318+0000 7f3ffa800640 1 -- 172.21.15.87:0/3200036311 wait complete. 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 Processor -- start 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 -- start start 2024-04-25T10:13:27.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff4071170 0x7f3ff414f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff4073ea0 0x7f3ff4144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff415db00 0x7f3ff414c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff416b9d0 con 0x7f3ff415db00 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff416b850 con 0x7f3ff4071170 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff416bb50 con 0x7f3ff4073ea0 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff415db00 0x7f3ff414c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff415db00 0x7f3ff414c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44996/0 (socket says 172.21.15.87:44996) 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 -- 172.21.15.87:0/3263580931 learned_addr learned my addr 172.21.15.87:0/3263580931 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff4073ea0 0x7f3ff4144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9400640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff4071170 0x7f3ff414f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 -- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff4073ea0 msgr2=0x7f3ff4144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff4073ea0 0x7f3ff4144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 -- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff4071170 msgr2=0x7f3ff414f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff4071170 0x7f3ff414f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 -- 172.21.15.87:0/3263580931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff414f980 con 0x7f3ff415db00 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff415db00 0x7f3ff414c0b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c7d0 tx=0x7f3fec00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff1600640 1 -- 172.21.15.87:0/3263580931 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec018070 con 0x7f3ff415db00 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff1600640 1 -- 172.21.15.87:0/3263580931 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec00a640 con 0x7f3ff415db00 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ff1600640 1 -- 172.21.15.87:0/3263580931 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec00aa70 con 0x7f3ff415db00 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff415bb10 con 0x7f3ff415db00 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.322+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff415c0d0 con 0x7f3ff415db00 2024-04-25T10:13:27.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff1600640 1 -- 172.21.15.87:0/3263580931 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3fec00ac10 con 0x7f3ff415db00 2024-04-25T10:13:27.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff1600640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3fd4066f90 0x7f3fd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff1600640 1 -- 172.21.15.87:0/3263580931 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fec084580 con 0x7f3ff415db00 2024-04-25T10:13:27.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff9400640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3fd4066f90 0x7f3fd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 Processor -- start 2024-04-25T10:13:27.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 -- start start 2024-04-25T10:13:27.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff9400640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3fd4066f90 0x7f3fd4069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8007fd0 tx=0x7f3fe8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1c920 0x55be73f1cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73ed29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73ed3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be73dba5f0 con 0x55be73f1d240 2024-04-25T10:13:27.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be73d21da0 con 0x55be73f1c920 2024-04-25T10:13:27.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be73caa150 con 0x55be73ed2f30 2024-04-25T10:13:27.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73ed29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73ed29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45000/0 (socket says 172.21.15.87:45000) 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 -- 172.21.15.87:0/449036261 learned_addr learned my addr 172.21.15.87:0/449036261 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff9400640 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1c920 0x55be73f1cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73ed3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 -- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 msgr2=0x55be73ed3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73ed3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 -- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1c920 msgr2=0x55be73f1cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1c920 0x55be73f1cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.326+0000 7f3ff8a00640 1 -- 172.21.15.87:0/449036261 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be73cc3ca0 con 0x55be73f1d240 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73ed29f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3fe001a010 tx=0x7f3fe00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eada70a92acecee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fe6000640 1 -- 172.21.15.87:0/449036261 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00bb070 con 0x55be73f1d240 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fe6000640 1 -- 172.21.15.87:0/449036261 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00ac960 con 0x55be73f1d240 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fe6000640 1 -- 172.21.15.87:0/449036261 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00b1040 con 0x55be73f1d240 2024-04-25T10:13:27.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- 172.21.15.87:0/449036261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 msgr2=0x55be73ed29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73ed29f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3fe001a010 tx=0x7f3fe00a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- 172.21.15.87:0/449036261 shutdown_connections 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73ed3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73ed29f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 --2- 172.21.15.87:0/449036261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1c920 0x55be73f1cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- 172.21.15.87:0/449036261 >> 172.21.15.87:0/449036261 conn(0x55be73ed92a0 msgr2=0x55be73ed12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- 172.21.15.87:0/449036261 shutdown_connections 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- 172.21.15.87:0/449036261 wait complete. 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- start start 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 Processor -- start 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f23d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73ed6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be73cdc990 con 0x55be73f1c920 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be73d1de40 con 0x55be73f1d240 2024-04-25T10:13:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be73cad900 con 0x55be73ed2f30 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3fff211000 10 client.0 _ll_register_callbacks cb 0x55be73dd51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f242a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45006/0 (socket says 172.21.15.87:45006) 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 -- 172.21.15.87:0/196888878 learned_addr learned my addr 172.21.15.87:0/196888878 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff9400640 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f23d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73ed6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 -- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 msgr2=0x55be73f23d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f23d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 -- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 msgr2=0x55be73ed6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73ed6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.330+0000 7f3ff8a00640 1 -- 172.21.15.87:0/196888878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be73cda630 con 0x55be73f1c920 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f242a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3fe001a010 tx=0x7f3fe001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1ffd9fa009a5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 1 -- 172.21.15.87:0/196888878 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00bb040 con 0x55be73f1c920 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 1 -- 172.21.15.87:0/196888878 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fe00b7440 con 0x55be73f1c920 2024-04-25T10:13:27.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fe00ac960 con 0x55be73f1c920 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 1 -- 172.21.15.87:0/196888878 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fe00b1040 con 0x55be73f1c920 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 10 client.5334 fetch_fsmap learned FSMap version 16 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55be73caa3a0 con 0x55be73f1c920 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 1 -- 172.21.15.87:0/196888878 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00ac960 con 0x55be73f1c920 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 1 -- 172.21.15.87:0/196888878 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3fe00ab070 con 0x55be73f1c920 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 10 client.5334 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 10 client.5334 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 20 client.5334 populate_metadata read hostname 'smithi087' 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73ed8160 0x55be73f2ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 4 client.5334 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be73cbda50 con 0x55be73ed8160 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3ff9400640 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73ed8160 0x55be73f2ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 10 client.5334 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3ff9400640 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73ed8160 0x55be73f2ddc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3292995d22dba30e server_cookie=db0b230e042bbd38 in_seq=0 out_seq=0 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 1 -- 172.21.15.87:0/196888878 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be73cbda50 con 0x55be73ed8160 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 10 client.5334 handle_command_reply: tid=0 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 client.5334 shutdown 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 msgr2=0x55be73f242a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f242a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3fe001a010 tx=0x7f3fe001ac20 comp rx=0 tx=0).stop 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 shutdown_connections 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73ed8160 0x55be73f2ddc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73ed6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f242a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 --2- 172.21.15.87:0/196888878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f23d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fdaa00640 0 client.5334 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 >> 172.21.15.87:0/196888878 conn(0x55be73ed92a0 msgr2=0x55be73ed19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 shutdown_connections 2024-04-25T10:13:27.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 1 -- 172.21.15.87:0/196888878 wait complete. 2024-04-25T10:13:27.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.334+0000 7f3fff211000 20 client.5334 trim_cache size 0 max 16384 2024-04-25T10:13:27.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 Processor -- start 2024-04-25T10:13:27.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 -- start start 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73f78cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff4202190 con 0x55be73f1c920 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff41462d0 con 0x55be73f1d240 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff41505d0 con 0x55be73ed2f30 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f787b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53882/0 (socket says 172.21.15.87:53882) 2024-04-25T10:13:27.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 -- 172.21.15.87:0/590427872 learned_addr learned my addr 172.21.15.87:0/590427872 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff9400640 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73f78cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 -- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 msgr2=0x55be73f78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f78270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 -- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 msgr2=0x55be73f78cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73f78cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.350+0000 7f3ff8a00640 1 -- 172.21.15.87:0/590427872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be73f1d070 con 0x55be73f1c920 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f787b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3fe001ac60 tx=0x7f3fe00bf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d30fb2380b9fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fe5600640 1 -- 172.21.15.87:0/590427872 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00bb070 con 0x55be73f1c920 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fe5600640 1 -- 172.21.15.87:0/590427872 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00c6070 con 0x55be73f1c920 2024-04-25T10:13:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fe5600640 1 -- 172.21.15.87:0/590427872 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0004020 con 0x55be73f1c920 2024-04-25T10:13:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- 172.21.15.87:0/590427872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 msgr2=0x55be73f787b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f787b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3fe001ac60 tx=0x7f3fe00bf000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- 172.21.15.87:0/590427872 shutdown_connections 2024-04-25T10:13:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73f1d240 0x55be73f78cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1c920 0x55be73f787b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 --2- 172.21.15.87:0/590427872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73ed2f30 0x55be73f78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- 172.21.15.87:0/590427872 >> 172.21.15.87:0/590427872 conn(0x55be73ed92a0 msgr2=0x55be73ed6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- 172.21.15.87:0/590427872 shutdown_connections 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- 172.21.15.87:0/590427872 wait complete. 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- start start 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 Processor -- start 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73ed2f30 0x55be73f7d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73f1c920 0x55be73f7dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73f37840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be73d263d0 con 0x55be73f1d240 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be73dddd50 con 0x55be73ed2f30 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be73cbc980 con 0x55be73f1c920 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fff211000 10 client.0 _ll_register_callbacks cb 0x7f3fe0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73f37840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73ed2f30 0x55be73f7d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73f37840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53890/0 (socket says 172.21.15.87:53890) 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 -- 172.21.15.87:0/1454836916 learned_addr learned my addr 172.21.15.87:0/1454836916 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff8a00640 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73f1c920 0x55be73f7dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 -- 172.21.15.87:0/1454836916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73f1c920 msgr2=0x55be73f7dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73f1c920 0x55be73f7dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 -- 172.21.15.87:0/1454836916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73ed2f30 msgr2=0x55be73f7d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73ed2f30 0x55be73f7d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 -- 172.21.15.87:0/1454836916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be73da3600 con 0x55be73f1d240 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3ff9e00640 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73f37840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3fec082a10 tx=0x7f3fec054ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13ab2c02473a4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.354+0000 7f3fdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 1 -- 172.21.15.87:0/1454836916 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec082060 con 0x55be73f1d240 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 1 -- 172.21.15.87:0/1454836916 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fec0840b0 con 0x55be73f1d240 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fe00c6070 con 0x55be73f1d240 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 1 -- 172.21.15.87:0/1454836916 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fec059e60 con 0x55be73f1d240 2024-04-25T10:13:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 10 client.5340 fetch_fsmap learned FSMap version 16 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55be73cb69b0 con 0x55be73f1d240 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 1 -- 172.21.15.87:0/1454836916 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00c6070 con 0x55be73f1d240 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 1 -- 172.21.15.87:0/1454836916 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3fec06cb80 con 0x55be73f1d240 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 10 client.5340 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 10 client.5340 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 20 client.5340 populate_metadata read hostname 'smithi087' 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73f37d80 0x55be73f38160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 4 client.5340 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be73d2f2e0 con 0x55be73f37d80 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ff9400640 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73f37d80 0x55be73f38160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ff9400640 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73f37d80 0x55be73f38160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c468a4465252a830 server_cookie=80192b1ce8585aad in_seq=0 out_seq=0 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 10 client.5340 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 1 -- 172.21.15.87:0/1454836916 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55be73d2f2e0 con 0x55be73f37d80 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 10 client.5340 handle_command_reply: tid=0 2024-04-25T10:13:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 client.5340 shutdown 2024-04-25T10:13:27.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 msgr2=0x55be73f37840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73f37840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3fec082a10 tx=0x7f3fec054ef0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 shutdown_connections 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be73f1d240 0x55be73f37840 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be73f1c920 0x55be73f7dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55be73f37d80 0x55be73f38160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 --2- 172.21.15.87:0/1454836916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be73ed2f30 0x55be73f7d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 >> 172.21.15.87:0/1454836916 conn(0x55be73ed92a0 msgr2=0x55be73f24920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fdaa00640 0 client.5340 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 shutdown_connections 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 1 -- 172.21.15.87:0/1454836916 wait complete. 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3fff211000 20 client.5340 trim_cache size 0 max 16384 2024-04-25T10:13:27.370 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5127, 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 10.323613782000001, 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 16, 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 33, 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 33, 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 18.196732489999999 2024-04-25T10:13:27.371 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:13:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3fd4066f90 msgr2=0x7f3fd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3fd4066f90 0x7f3fd4069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8007fd0 tx=0x7f3fe8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff415db00 msgr2=0x7f3ff414c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff415db00 0x7f3ff414c0b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c7d0 tx=0x7f3fec00cc90 comp rx=0 tx=0).stop 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 shutdown_connections 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3fd4066f90 0x7f3fd4069450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff415db00 0x7f3ff414c0b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff4073ea0 0x7f3ff4144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 --2- 172.21.15.87:0/3263580931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff4071170 0x7f3ff414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 >> 172.21.15.87:0/3263580931 conn(0x7f3ff40b1b50 msgr2=0x7f3ff40b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 shutdown_connections 2024-04-25T10:13:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.358+0000 7f3ffa800640 1 -- 172.21.15.87:0/3263580931 wait complete. 2024-04-25T10:13:27.384 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:27.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.510+0000 7effd9a00640 1 Processor -- start 2024-04-25T10:13:27.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.510+0000 7effd9a00640 1 -- start start 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd415dab0 0x7effd4162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd9a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7effd40b57e0 0x7effd40af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd9a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd40b0210 con 0x7effd40669a0 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd9a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40aff30 con 0x7effd40b57e0 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd4161840 con 0x7effd415dab0 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7effd40669a0 0x7effd40d0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33916/0 (socket says 172.21.15.87:33916) 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd3400640 1 -- 172.21.15.87:0/3714798739 learned_addr learned my addr 172.21.15.87:0/3714798739 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7effd40b57e0 0x7effd40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd2a00640 1 --2- 172.21.15.87:0/3714798739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd415dab0 0x7effd4162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd1600640 1 -- 172.21.15.87:0/3714798739 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131234105 0 0) 0x7effd40b0210 con 0x7effd40669a0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd1600640 1 -- 172.21.15.87:0/3714798739 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effb8003530 con 0x7effd40669a0 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd2000640 1 -- 172.21.15.87:0/3714798739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd415dab0 msgr2=0x7effd4162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd2000640 1 --2- 172.21.15.87:0/3714798739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd415dab0 0x7effd4162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd2000640 1 -- 172.21.15.87:0/3714798739 >> v1:172.21.15.87:6789/0 conn(0x7effd40669a0 legacy=0x7effd40d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd2000640 1 -- 172.21.15.87:0/3714798739 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4161b20 con 0x7effd40b57e0 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.514+0000 7effd2a00640 1 --2- 172.21.15.87:0/3714798739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd415dab0 0x7effd4162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:27.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd2000640 1 --2- 172.21.15.87:0/3714798739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40b57e0 0x7effd40af690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7effc00097c0 tx=0x7effd40d1460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89207edb4451d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd1600640 1 -- 172.21.15.87:0/3714798739 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc00aa5f0 con 0x7effd40b57e0 2024-04-25T10:13:27.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd1600640 1 -- 172.21.15.87:0/3714798739 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc00af070 con 0x7effd40b57e0 2024-04-25T10:13:27.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd1600640 1 -- 172.21.15.87:0/3714798739 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc00aa9b0 con 0x7effd40b57e0 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 -- 172.21.15.87:0/3714798739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40b57e0 msgr2=0x7effd40af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 --2- 172.21.15.87:0/3714798739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40b57e0 0x7effd40af690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7effc00097c0 tx=0x7effd40d1460 comp rx=0 tx=0).stop 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 -- 172.21.15.87:0/3714798739 shutdown_connections 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 --2- 172.21.15.87:0/3714798739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40b57e0 0x7effd40af690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 --2- 172.21.15.87:0/3714798739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd415dab0 0x7effd4162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 -- 172.21.15.87:0/3714798739 >> 172.21.15.87:0/3714798739 conn(0x7effd406ac90 msgr2=0x7effd406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 -- 172.21.15.87:0/3714798739 shutdown_connections 2024-04-25T10:13:27.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 -- 172.21.15.87:0/3714798739 wait complete. 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 Processor -- start 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.518+0000 7effd9a00640 1 -- start start 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40669a0 0x7effd415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd40b57e0 0x7effd40ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd415dab0 0x7effd40ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd415f020 con 0x7effd40669a0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd415eea0 con 0x7effd415dab0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd416ab40 con 0x7effd40b57e0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40669a0 0x7effd415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40669a0 0x7effd415cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53922/0 (socket says 172.21.15.87:53922) 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 -- 172.21.15.87:0/3667997251 learned_addr learned my addr 172.21.15.87:0/3667997251 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2000640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd40b57e0 0x7effd40ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd3400640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd415dab0 0x7effd40ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 -- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd40b57e0 msgr2=0x7effd40ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd40b57e0 0x7effd40ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 -- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd415dab0 msgr2=0x7effd40ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd415dab0 0x7effd40ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 -- 172.21.15.87:0/3667997251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4151750 con 0x7effd40669a0 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd2a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40669a0 0x7effd415cb10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effc400d800 tx=0x7effc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effcaa00640 1 -- 172.21.15.87:0/3667997251 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400b6c0 con 0x7effd40669a0 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effcaa00640 1 -- 172.21.15.87:0/3667997251 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4019070 con 0x7effd40669a0 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effcaa00640 1 -- 172.21.15.87:0/3667997251 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400baa0 con 0x7effd40669a0 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd4151a30 con 0x7effd40669a0 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd4151ff0 con 0x7effd40669a0 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd404ef30 con 0x7effd40669a0 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.522+0000 7effcaa00640 1 -- 172.21.15.87:0/3667997251 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effc400bc40 con 0x7effd40669a0 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.526+0000 7effcaa00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effb4066f60 0x7effb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.526+0000 7effcaa00640 1 -- 172.21.15.87:0/3667997251 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effc4085440 con 0x7effd40669a0 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.526+0000 7effd2000640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effb4066f60 0x7effb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.526+0000 7effd2000640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effb4066f60 0x7effb4069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7effc00ae000 tx=0x7effc0004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.530+0000 7effcaa00640 1 -- 172.21.15.87:0/3667997251 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effc4056450 con 0x7effd40669a0 2024-04-25T10:13:27.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7effd40719b0 con 0x7effd40669a0 2024-04-25T10:13:27.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effcaa00640 1 -- 172.21.15.87:0/3667997251 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7effc4055d50 con 0x7effd40669a0 2024-04-25T10:13:27.725 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effb4066f60 msgr2=0x7effb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effb4066f60 0x7effb4069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7effc00ae000 tx=0x7effc0004740 comp rx=0 tx=0).stop 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40669a0 msgr2=0x7effd415cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40669a0 0x7effd415cb10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effc400d800 tx=0x7effc400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 shutdown_connections 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effb4066f60 0x7effb4069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd415dab0 0x7effd40ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd40b57e0 0x7effd40ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 --2- 172.21.15.87:0/3667997251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd40669a0 0x7effd415cb10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 >> 172.21.15.87:0/3667997251 conn(0x7effd406ac90 msgr2=0x7effd415fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 shutdown_connections 2024-04-25T10:13:27.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:27.718+0000 7effd9a00640 1 -- 172.21.15.87:0/3667997251 wait complete. 2024-04-25T10:13:27.745 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:27.746 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:27.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.747 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42406sec preferred_lft 42406sec 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:27.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.759 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:27.821 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:13:27.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.823 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:13:27.884 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:13:27.884 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:27.884 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:27.885 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:27.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.886 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:13:27.886 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:13:27.886 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:27.886 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:27.886 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:27.886 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:13:27.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.976 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:13:27.985 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:27.985 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:27.985 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:27.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.986 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:13:28.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:28.051 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:28.103 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:28.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:28.104 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:28.155 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.155 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:28.247 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:28.247 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:28.247 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:13:28.262 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:28.262 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:28.262 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:13:28.262 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:13:28.262 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:28.271 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:28.271 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:13:28.275 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:28.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.275 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:28.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.343 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:28.399 INFO:teuthology.orchestra.run.smithi087.stdout:client4868 2024-04-25T10:13:28.400 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:13:28.400 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:28.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe062400640 1 Processor -- start 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe062400640 1 -- start start 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe062400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe062400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe05c161640 0x7fe05c071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe062400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05c0672c0 con 0x7fe05c06ac90 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe062400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05c161ad0 con 0x7fe05c161640 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe062400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c161db0 con 0x7fe05c0669a0 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe053e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe05c161640 0x7fe05c071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe061a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe05c06ac90 0x7fe05c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33928/0 (socket says 172.21.15.87:33928) 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe053e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe05c161640 0x7fe05c071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53934/0 (socket says 172.21.15.87:53934) 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe061a00640 1 -- 172.21.15.87:0/1730996167 learned_addr learned my addr 172.21.15.87:0/1730996167 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe061000640 1 --2- 172.21.15.87:0/1730996167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe053e00640 1 -- 172.21.15.87:0/1730996167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 msgr2=0x7fe05c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe053e00640 1 --2- 172.21.15.87:0/1730996167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe053e00640 1 -- 172.21.15.87:0/1730996167 >> v1:172.21.15.87:6789/0 conn(0x7fe05c06ac90 legacy=0x7fe05c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe053e00640 1 -- 172.21.15.87:0/1730996167 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c162090 con 0x7fe05c161640 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.630+0000 7fe05be00640 1 -- 172.21.15.87:0/1730996167 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373983236 0 0) 0x7fe05c0672c0 con 0x7fe05c06ac90 2024-04-25T10:13:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe053e00640 1 --2- 172.21.15.87:0/1730996167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c161640 0x7fe05c071e50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe0440097c0 tx=0x7fe04401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea50489ad05fcea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe05be00640 1 -- 172.21.15.87:0/1730996167 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0440b0070 con 0x7fe05c161640 2024-04-25T10:13:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe05be00640 1 -- 172.21.15.87:0/1730996167 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0440ab600 con 0x7fe05c161640 2024-04-25T10:13:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe05be00640 1 -- 172.21.15.87:0/1730996167 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0440ab9d0 con 0x7fe05c161640 2024-04-25T10:13:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 -- 172.21.15.87:0/1730996167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c161640 msgr2=0x7fe05c071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 --2- 172.21.15.87:0/1730996167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c161640 0x7fe05c071e50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe0440097c0 tx=0x7fe04401c410 comp rx=0 tx=0).stop 2024-04-25T10:13:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 -- 172.21.15.87:0/1730996167 shutdown_connections 2024-04-25T10:13:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 --2- 172.21.15.87:0/1730996167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c161640 0x7fe05c071e50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 --2- 172.21.15.87:0/1730996167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 -- 172.21.15.87:0/1730996167 >> 172.21.15.87:0/1730996167 conn(0x7fe05c0d06f0 msgr2=0x7fe05c0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 -- 172.21.15.87:0/1730996167 shutdown_connections 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.634+0000 7fe062400640 1 -- 172.21.15.87:0/1730996167 wait complete. 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 Processor -- start 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 -- start start 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe05c06ac90 0x7fe05c159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c154a50 0x7fe05c159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c16b1d0 con 0x7fe05c154a50 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c15df60 con 0x7fe05c06ac90 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c16b350 con 0x7fe05c0669a0 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c154a50 0x7fe05c159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c154a50 0x7fe05c159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53938/0 (socket says 172.21.15.87:53938) 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 -- 172.21.15.87:0/2566597918 learned_addr learned my addr 172.21.15.87:0/2566597918 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:13:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061000640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe053e00640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe05c06ac90 0x7fe05c159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 -- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 msgr2=0x7fe05c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 -- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe05c06ac90 msgr2=0x7fe05c159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe05c06ac90 0x7fe05c159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 -- 172.21.15.87:0/2566597918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c154860 con 0x7fe05c154a50 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe053e00640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe05c06ac90 0x7fe05c159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe061a00640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c154a50 0x7fe05c159f40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe05400c760 tx=0x7fe05400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe059600640 1 -- 172.21.15.87:0/2566597918 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054018070 con 0x7fe05c154a50 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe059600640 1 -- 172.21.15.87:0/2566597918 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05400a640 con 0x7fe05c154a50 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe059600640 1 -- 172.21.15.87:0/2566597918 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05400aa30 con 0x7fe05c154a50 2024-04-25T10:13:28.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05c154fc0 con 0x7fe05c154a50 2024-04-25T10:13:28.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.638+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c155580 con 0x7fe05c154a50 2024-04-25T10:13:28.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.642+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c075120 con 0x7fe05c154a50 2024-04-25T10:13:28.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.642+0000 7fe059600640 1 -- 172.21.15.87:0/2566597918 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe05401b070 con 0x7fe05c154a50 2024-04-25T10:13:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.642+0000 7fe059600640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe038066fe0 0x7fe0380694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.642+0000 7fe059600640 1 -- 172.21.15.87:0/2566597918 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe054084600 con 0x7fe05c154a50 2024-04-25T10:13:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.642+0000 7fe061000640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe038066fe0 0x7fe0380694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.642+0000 7fe061000640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe038066fe0 0x7fe0380694a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe05c160b20 tx=0x7fe04c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.646+0000 7fe059600640 1 -- 172.21.15.87:0/2566597918 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0540556f0 con 0x7fe05c154a50 2024-04-25T10:13:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:28.834+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe05c0af0e0 con 0x7fe05c154a50 2024-04-25T10:13:29.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.398+0000 7fe059600640 1 -- 172.21.15.87:0/2566597918 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe054054ff0 con 0x7fe05c154a50 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi087.stderr:enabled new snapshots 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe038066fe0 msgr2=0x7fe0380694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe038066fe0 0x7fe0380694a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe05c160b20 tx=0x7fe04c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c154a50 msgr2=0x7fe05c159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c154a50 0x7fe05c159f40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe05400c760 tx=0x7fe05400cc20 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 shutdown_connections 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe038066fe0 0x7fe0380694a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe05c154a50 0x7fe05c159f40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe05c06ac90 0x7fe05c159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 --2- 172.21.15.87:0/2566597918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe05c0669a0 0x7fe05c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 >> 172.21.15.87:0/2566597918 conn(0x7fe05c0d06f0 msgr2=0x7fe05c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 shutdown_connections 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:29.402+0000 7fe062400640 1 -- 172.21.15.87:0/2566597918 wait complete. 2024-04-25T10:13:29.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.429 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:13:29.442 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:13:29.443 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:29.492 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:13:29.492 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> import uuid 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> n = 2000 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> create_hard_links = False 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> if 3 > 0: 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> create_hard_links = True 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> dpath = os.path.dirname(path) 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> fnameprefix = os.path.basename(path) 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:13:29.493 DEBUG:teuthology.orchestra.run.smithi087:> for i in range(n): 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> f.write(f"{i}") 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> f.flush() 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(f.fileno()) 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> os.unlink(fpath) 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(dirfd) 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> if create_hard_links: 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> for j in range(3): 2024-04-25T10:13:29.494 DEBUG:teuthology.orchestra.run.smithi087:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:13:29.495 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:13:29.495 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(dirfd) 2024-04-25T10:13:29.495 DEBUG:teuthology.orchestra.run.smithi087:> finally: 2024-04-25T10:13:29.495 DEBUG:teuthology.orchestra.run.smithi087:> os.close(dirfd) 2024-04-25T10:13:29.495 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:13:50.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:13:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:50.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:50.467 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:13:50.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:50.529 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:13:51.655 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:51.659 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:17.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:17.706 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:14:17.764 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:14:17.765 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:14:17.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cdc00640 1 Processor -- start 2024-04-25T10:14:17.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cdc00640 1 -- start start 2024-04-25T10:14:17.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cdc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2c815db50 0x7fe2c80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c80ae490 0x7fe2c80ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cdc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c806bf40 con 0x7fe2c80aee30 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cdc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c80b1cb0 con 0x7fe2c815db50 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80b1f90 con 0x7fe2c80ae490 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cd200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe2c80aee30 0x7fe2c806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55084/0 (socket says 172.21.15.87:55084) 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2cd200640 1 -- 172.21.15.87:0/1055107820 learned_addr learned my addr 172.21.15.87:0/1055107820 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/1055107820 >> v2:172.21.15.169:3300/0 conn(0x7fe2c815db50 0x7fe2c80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.893+0000 7fe2c7400640 1 --2- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c80ae490 0x7fe2c80ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c6a00640 1 -- 172.21.15.87:0/1055107820 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144713721 0 0) 0x7fe2c806bf40 con 0x7fe2c80aee30 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c6a00640 1 -- 172.21.15.87:0/1055107820 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ac003530 con 0x7fe2c80aee30 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c7e00640 1 -- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c80ae490 msgr2=0x7fe2c80ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c80ae490 0x7fe2c80ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c7e00640 1 -- 172.21.15.87:0/1055107820 >> v1:172.21.15.87:6789/0 conn(0x7fe2c80aee30 legacy=0x7fe2c806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c7e00640 1 -- 172.21.15.87:0/1055107820 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80b2270 con 0x7fe2c815db50 2024-04-25T10:14:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c815db50 0x7fe2c80b2d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00097c0 tx=0x7fe2b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2563c7fade68611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c6a00640 1 -- 172.21.15.87:0/1055107820 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00b0070 con 0x7fe2c815db50 2024-04-25T10:14:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c6a00640 1 -- 172.21.15.87:0/1055107820 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b00ab600 con 0x7fe2c815db50 2024-04-25T10:14:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.897+0000 7fe2c6a00640 1 -- 172.21.15.87:0/1055107820 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00ab9d0 con 0x7fe2c815db50 2024-04-25T10:14:17.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c815db50 msgr2=0x7fe2c80b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c815db50 0x7fe2c80b2d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00097c0 tx=0x7fe2b001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- 172.21.15.87:0/1055107820 shutdown_connections 2024-04-25T10:14:17.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c80ae490 0x7fe2c80ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/1055107820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c815db50 0x7fe2c80b2d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- 172.21.15.87:0/1055107820 >> 172.21.15.87:0/1055107820 conn(0x7fe2c80d17a0 msgr2=0x7fe2c80d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- 172.21.15.87:0/1055107820 shutdown_connections 2024-04-25T10:14:17.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- 172.21.15.87:0/1055107820 wait complete. 2024-04-25T10:14:17.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 Processor -- start 2024-04-25T10:14:17.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- start start 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c80ae490 0x7fe2c81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c80aee30 0x7fe2c8153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c815db50 0x7fe2c806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c816a8e0 con 0x7fe2c80ae490 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c816a760 con 0x7fe2c80aee30 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c816aa60 con 0x7fe2c815db50 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c80ae490 0x7fe2c81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c80ae490 0x7fe2c81527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33820/0 (socket says 172.21.15.87:33820) 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2131080894 learned_addr learned my addr 172.21.15.87:0/2131080894 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7400640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c80aee30 0x7fe2c8153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cd200640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c815db50 0x7fe2c806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c815db50 msgr2=0x7fe2c806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c815db50 0x7fe2c806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c80aee30 msgr2=0x7fe2c8153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c80aee30 0x7fe2c8153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2131080894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c8074840 con 0x7fe2c80ae490 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2cd200640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c815db50 0x7fe2c806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.901+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c80ae490 0x7fe2c81527f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe2b001a0d0 tx=0x7fe2b001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2bfe00640 1 -- 172.21.15.87:0/2131080894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00b0050 con 0x7fe2c80ae490 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2bfe00640 1 -- 172.21.15.87:0/2131080894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b00bc070 con 0x7fe2c80ae490 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c80d26c0 con 0x7fe2c80ae490 2024-04-25T10:14:17.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2bfe00640 1 -- 172.21.15.87:0/2131080894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0006e20 con 0x7fe2c80ae490 2024-04-25T10:14:17.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c81530e0 con 0x7fe2c80ae490 2024-04-25T10:14:17.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2bfe00640 1 -- 172.21.15.87:0/2131080894 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2b00b5830 con 0x7fe2c80ae490 2024-04-25T10:14:17.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2bfe00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe2a4066f60 0x7fe2a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2bfe00640 1 -- 172.21.15.87:0/2131080894 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b011f620 con 0x7fe2c80ae490 2024-04-25T10:14:17.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.905+0000 7fe2c7400640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe2a4066f60 0x7fe2a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 Processor -- start 2024-04-25T10:14:17.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7400640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe2a4066f60 0x7fe2a4069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806c670 tx=0x7fe2b8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- start start 2024-04-25T10:14:17.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d73aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d7369850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d736a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613d724c1f0 con 0x5613d7369d90 2024-04-25T10:14:17.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613d71b2e90 con 0x5613d73aea40 2024-04-25T10:14:17.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613d713af50 con 0x5613d73af360 2024-04-25T10:14:17.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2cd200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d736a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d73aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d7369850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d73aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:46284/0 (socket says 172.21.15.87:46284) 2024-04-25T10:14:17.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d7369850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:59184/0 (socket says 172.21.15.87:59184) 2024-04-25T10:14:17.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 -- 172.21.15.87:0/3025417812 learned_addr learned my addr 172.21.15.87:0/3025417812 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 -- 172.21.15.87:0/3025417812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 msgr2=0x5613d7369850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/3025417812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d7369850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 -- 172.21.15.87:0/3025417812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 msgr2=0x5613d736a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/3025417812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d736a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 -- 172.21.15.87:0/3025417812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613d718a8b0 con 0x5613d73aea40 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/3025417812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d73aee20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00f8100 tx=0x7fe2b00f4f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f1b129f785bb439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2bcc00640 1 -- 172.21.15.87:0/3025417812 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00eff20 con 0x5613d73aea40 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2bcc00640 1 -- 172.21.15.87:0/3025417812 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b00b1040 con 0x5613d73aea40 2024-04-25T10:14:17.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2bcc00640 1 -- 172.21.15.87:0/3025417812 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00f4070 con 0x5613d73aea40 2024-04-25T10:14:17.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3025417812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 msgr2=0x5613d73aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3025417812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d73aee20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00f8100 tx=0x7fe2b00f4f90 comp rx=0 tx=0).stop 2024-04-25T10:14:17.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3025417812 shutdown_connections 2024-04-25T10:14:17.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3025417812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d736a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3025417812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d7369850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3025417812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d73aee20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3025417812 >> 172.21.15.87:0/3025417812 conn(0x5613d7365d40 msgr2=0x5613d7368130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3025417812 shutdown_connections 2024-04-25T10:14:17.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.909+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3025417812 wait complete. 2024-04-25T10:14:17.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 -- start start 2024-04-25T10:14:17.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 Processor -- start 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d7369d90 0x5613d73ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d73aea40 0x5613d736fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73af360 0x5613d7370430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613d716d540 con 0x5613d73aea40 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613d71aef30 con 0x5613d73af360 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613d71b2e90 con 0x5613d7369d90 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 10 client.0 _ll_register_callbacks cb 0x5613d7267180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:17.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d73aea40 0x5613d736fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d73aea40 0x5613d736fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33850/0 (socket says 172.21.15.87:33850) 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 -- 172.21.15.87:0/3266681929 learned_addr learned my addr 172.21.15.87:0/3266681929 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2cd200640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73af360 0x5613d7370430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d7369d90 0x5613d73ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 -- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d7369d90 msgr2=0x5613d73ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d7369d90 0x5613d73ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 -- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73af360 msgr2=0x5613d7370430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73af360 0x5613d7370430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 -- 172.21.15.87:0/3266681929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613d713b1a0 con 0x5613d73aea40 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2cd200640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73af360 0x5613d7370430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7400640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d73aea40 0x5613d736fef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8003650 tx=0x7fe2b80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94baf54ea9978c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2b5600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2b5600640 1 -- 172.21.15.87:0/3266681929 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8027070 con 0x5613d73aea40 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2b5600640 1 -- 172.21.15.87:0/3266681929 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b8014440 con 0x5613d73aea40 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2b00b1040 con 0x5613d73aea40 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2b5600640 1 -- 172.21.15.87:0/3266681929 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b801b070 con 0x5613d73aea40 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 10 client.5370 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5613d7235840 con 0x5613d73aea40 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2b5600640 1 -- 172.21.15.87:0/3266681929 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b00b1040 con 0x5613d73aea40 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2b5600640 1 -- 172.21.15.87:0/3266681929 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe2b8035080 con 0x5613d73aea40 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 10 client.5370 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 10 client.5370 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 20 client.5370 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d7371170 0x5613d73c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 4 client.5370 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613d714ebe0 con 0x5613d7371170 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d7371170 0x5613d73c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2b5600640 10 client.5370 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.913+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d7371170 0x5613d73c0d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ca0651afa3d9c2 server_cookie=b10cbbed31db5c4e in_seq=0 out_seq=0 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2b5600640 1 -- 172.21.15.87:0/3266681929 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5613d714ebe0 con 0x5613d7371170 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2b5600640 10 client.5370 handle_command_reply: tid=0 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 client.5370 shutdown 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d73aea40 msgr2=0x5613d736fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d73aea40 0x5613d736fef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8003650 tx=0x7fe2b80046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 shutdown_connections 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d7371170 0x5613d73c0d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73af360 0x5613d7370430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d73aea40 0x5613d736fef0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2b5600640 0 client.5370 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/3266681929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d7369d90 0x5613d73ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 >> 172.21.15.87:0/3266681929 conn(0x5613d7365d40 msgr2=0x5613d73689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 shutdown_connections 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 1 -- 172.21.15.87:0/3266681929 wait complete. 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.917+0000 7fe2d25cd000 20 client.5370 trim_cache size 0 max 16384 2024-04-25T10:14:17.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 Processor -- start 2024-04-25T10:14:17.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 -- start start 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d740b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c82030a0 con 0x5613d7369d90 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c815ec90 con 0x5613d73aea40 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80b5be0 con 0x5613d73af360 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33852/0 (socket says 172.21.15.87:33852) 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.929+0000 7fe2c7e00640 1 -- 172.21.15.87:0/4111753529 learned_addr learned my addr 172.21.15.87:0/4111753529 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7400640 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2cd200640 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d740b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 -- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 msgr2=0x5613d740b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d740b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 -- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 msgr2=0x5613d740b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 -- 172.21.15.87:0/4111753529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613d720ca10 con 0x5613d7369d90 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7400640 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ab80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe2b001c980 tx=0x7fe2b00abf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85da86cb04272c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2b7e00640 1 -- 172.21.15.87:0/4111753529 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0107040 con 0x5613d7369d90 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2b7e00640 1 -- 172.21.15.87:0/4111753529 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b010c070 con 0x5613d7369d90 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2b7e00640 1 -- 172.21.15.87:0/4111753529 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00f4070 con 0x5613d7369d90 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- 172.21.15.87:0/4111753529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 msgr2=0x5613d740ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ab80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe2b001c980 tx=0x7fe2b00abf90 comp rx=0 tx=0).stop 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- 172.21.15.87:0/4111753529 shutdown_connections 2024-04-25T10:14:17.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d740b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/4111753529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ab80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- 172.21.15.87:0/4111753529 >> 172.21.15.87:0/4111753529 conn(0x5613d7365d40 msgr2=0x5613d736ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- 172.21.15.87:0/4111753529 shutdown_connections 2024-04-25T10:14:17.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- 172.21.15.87:0/4111753529 wait complete. 2024-04-25T10:14:17.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- start start 2024-04-25T10:14:17.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 Processor -- start 2024-04-25T10:14:17.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d73ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613d71aef30 con 0x5613d7369d90 2024-04-25T10:14:17.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c815e690 con 0x5613d73aea40 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613d71b74c0 con 0x5613d73af360 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2d25cd000 10 client.0 _ll_register_callbacks cb 0x5613d7175a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33858/0 (socket says 172.21.15.87:33858) 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2097929913 learned_addr learned my addr 172.21.15.87:0/2097929913 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7400640 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2cd200640 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d73ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 msgr2=0x5613d73ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d73ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 msgr2=0x5613d740f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 -- 172.21.15.87:0/2097929913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613d7147b00 con 0x5613d7369d90 2024-04-25T10:14:17.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2c7e00640 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ece0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe2b001ca10 tx=0x7fe2b00b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ab22d4b64d2394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.933+0000 7fe2b5600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 1 -- 172.21.15.87:0/2097929913 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0107040 con 0x5613d7369d90 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 1 -- 172.21.15.87:0/2097929913 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b00ec590 con 0x5613d7369d90 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2b010c070 con 0x5613d7369d90 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 1 -- 172.21.15.87:0/2097929913 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b00f4070 con 0x5613d7369d90 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 10 client.5376 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5613d7159f80 con 0x5613d7369d90 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 1 -- 172.21.15.87:0/2097929913 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b010c070 con 0x5613d7369d90 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 1 -- 172.21.15.87:0/2097929913 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe2b00f3030 con 0x5613d7369d90 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 10 client.5376 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 10 client.5376 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 20 client.5376 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d73cb020 0x5613d73cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 4 client.5376 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:14:17.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5613d71c03d0 con 0x5613d73cb020 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2c7400640 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d73cb020 0x5613d73cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2c7400640 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d73cb020 0x5613d73cb430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6819abb8ba4cbfdb server_cookie=611fd785de7f48f5 in_seq=0 out_seq=0 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 10 client.5376 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 1 -- 172.21.15.87:0/2097929913 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17921 (crc 0 0 0) 0x5613d71c03d0 con 0x5613d73cb020 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 10 client.5376 handle_command_reply: tid=0 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 client.5376 shutdown 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 msgr2=0x5613d740ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ece0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe2b001ca10 tx=0x7fe2b00b5000 comp rx=0 tx=0).stop 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 shutdown_connections 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613d73af360 0x5613d73ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x5613d73cb020 0x5613d73cb430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613d73aea40 0x5613d740f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 --2- 172.21.15.87:0/2097929913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5613d7369d90 0x5613d740ece0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2b5600640 0 client.5376 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 >> 172.21.15.87:0/2097929913 conn(0x5613d7365d40 msgr2=0x5613d736f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 shutdown_connections 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 1 -- 172.21.15.87:0/2097929913 wait complete. 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2d25cd000 20 client.5376 trim_cache size 0 max 16384 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 1414, 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 1406, 2024-04-25T10:14:17.951 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 451740, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 12081609, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 8, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.166262058, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.077824067, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.042117021, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.036322026, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1406, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.262733099, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000186865 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:14:17.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000449, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000224 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 114272, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 18608 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 29756, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 50749, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 10220019, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 44942568, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 8, 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 3.587393530, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 2.664895426, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 1.210742079, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.032415642, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 50748, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.461433451, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000028797 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 47556, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.009304234, 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000195 2024-04-25T10:14:17.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 108544 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 3756, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 3758, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1136134, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 63099919, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.447935228, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.228347139, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.088781445, 2024-04-25T10:14:17.955 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.104130154, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3757, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.688929201, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000183372 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.956 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 1, 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-MDSRank": { 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4305, 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 13.226672983, 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003072397 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-PurgeQueue": { 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:14:17.957 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2155, 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.691246814, 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000320764 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": { 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "request": 26231, 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "reply": 26231, 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "reply_latency": { 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 26231, 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 247.672319030, 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.009441970 2024-04-25T10:14:17.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_reply": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "forward": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_complete": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_keys": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_commit": 12, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_split": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_merge": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes": 19742, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_top": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_bottom": 19742, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pin_tail": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pinned": 19742, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_expired": 0, 2024-04-25T10:14:17.959 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_with_caps": 135, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": 135, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "subtrees": 2, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse": 49017, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_hit": 48533, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_forward": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_discover": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_remote_ino": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_lock": 142, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "load_cent": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "q": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "exported": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "exported_inodes": 0, 2024-04-25T10:14:17.960 INFO:teuthology.orchestra.run.smithi087.stdout: "imported": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "imported_inodes": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_dir_fetch": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_peer_discover": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rfiles": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rbytes": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rsnaps": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_set_tag": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:14:17.961 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_file_inodes": 0, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_revoke": 1575, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_grant": 4040, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps": 4576, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_cap_release": 16, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "process_request_cap_release": 14081 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_cache": { 2024-04-25T10:14:17.962 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update_receipt": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_try_discover": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_send_discover": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_handle_discover": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays": 2000, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_delayed": 1, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_created": 4119, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_enqueued": 1867, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_reintegrated": 2214, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_migrated": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_processing": 0, 2024-04-25T10:14:17.963 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_started": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_completed": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_exportdir": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_flush": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragstats": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_inodestats": 0 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_log": { 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "evlrg": 0, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "evadd": 30124, 2024-04-25T10:14:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: "evex": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "evtrm": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "ev": 30124, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "evexg": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "evexd": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "segadd": 46, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "segex": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "segtrm": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "seg": 46, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "segmjr": 4, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "segexg": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "segexd": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "expos": 4194304, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "wrpos": 98293615, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "rdpos": 0, 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi087.stdout: "jlat": { 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4084, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 67.595533450, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.016551305 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "replayed": 0 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_mem": { 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 2014, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "ino+": 2014, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "ino-": 0, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "dir": 13, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "dir+": 13, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "dir-": 0, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "dn": 19742, 2024-04-25T10:14:17.966 INFO:teuthology.orchestra.run.smithi087.stdout: "dn+": 19743, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "dn-": 1, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "cap": 135, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "cap+": 2094, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "cap-": 1959, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "rss": 111576, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "heap": 215340 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_server": { 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_client_request": 29791, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_server_request": 0, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_request": 26231, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_session": 3, 2024-04-25T10:14:17.967 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_peer_request": 0, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "req_create_latency": { 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2000, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.264677861, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000632338 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getattr_latency": { 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 23, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.003196112, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000138961 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getfilelock_latency": { 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "req_link_latency": { 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6000, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6.675992649, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001112665 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookup_latency": { 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 7981, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6.703181472, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000839892 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookuphash_latency": { 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupino_latency": { 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupname_latency": { 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupparent_latency": { 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.970 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupsnap_latency": { 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000062806, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000062806 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lssnap_latency": { 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mkdir_latency": { 2024-04-25T10:14:17.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.002548807, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002548807 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mknod_latency": { 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mksnap_latency": { 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.007231519, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.007231519 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "req_open_latency": { 2024-04-25T10:14:17.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "req_readdir_latency": { 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.162287251, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.020285906 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rename_latency": { 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2214, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 216.190019708, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.097646802 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.973 INFO:teuthology.orchestra.run.smithi087.stdout: "req_renamesnap_latency": { 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "req_snapdiff_latency": { 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmdir_latency": { 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.974 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmsnap_latency": { 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.046056440, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.046056440 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmxattr_latency": { 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setattr_latency": { 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000277061, 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000277061 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setdirlayout_latency": { 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setfilelock_latency": { 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setlayout_latency": { 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setxattr_latency": { 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "req_symlink_latency": { 2024-04-25T10:14:17.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "req_unlink_latency": { 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8000, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 16.616787344, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002077098 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getvxattr_latency": { 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_sessions": { 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "session_count": 1, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "session_add": 1, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "session_remove": 0, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_open": 1, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_stale": 0, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "total_load": 18907, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "average_load": 18907, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "avg_session_uptime": 45, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "mdthresh_evicted": 0 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-04-25T10:14:17.980 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:14:17.981 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:14:17.982 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 17226683, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 4375, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:14:17.983 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 5312, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 75, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 20239526, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 794602, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-04-25T10:14:17.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "objecter": { 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_active": 0, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_laggy": 0, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send": 6145, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send_bytes": 94523185, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_resend": 31, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_reply": 6145, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_latency": { 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6145, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 30.587001093, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.004977542 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "op_inflight": 0, 2024-04-25T10:14:17.985 INFO:teuthology.orchestra.run.smithi087.stdout: "oplen_avg": { 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6145, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6237 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "op": 6145, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "op_r": 32, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "op_w": 6113, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "op_rmw": 0, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "op_pg": 0, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_stat": 0, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_create": 11, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_read": 32, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_write": 4116, 2024-04-25T10:14:17.986 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writefull": 23, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writesame": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_append": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_zero": 1867, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_truncate": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_delete": 34, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_mapext": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_sparse_read": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_clonerange": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_getxattr": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_setxattr": 22, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_rmxattr": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_call": 0, 2024-04-25T10:14:17.987 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_watch": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_notify": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_other": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_active": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_send": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_resend": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_ping": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_active": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_send": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_resend": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_active": 0, 2024-04-25T10:14:17.988 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_send": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_resend": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_active": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_send": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_resend": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "command_active": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "command_send": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "command_resend": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "map_epoch": 33, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "map_full": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "map_inc": 1, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_sessions": 8, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_open": 8, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_close": 0, 2024-04-25T10:14:17.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_laggy": 0, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_wr": 102, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_rd": 0, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_del": 30 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "oft": { 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_objs": 1, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_updates": 0, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_removes": 0 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "osd-slow-ops": { 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_ops_count": 0 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.990 INFO:teuthology.orchestra.run.smithi087.stdout: "purge_queue": { 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops": 0, 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing": 0, 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_high_water": 32, 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed_ops": 1867, 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed": 1867, 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_item_in_journal": 0 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:14:17.991 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 34918, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 9118947, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 34918, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 34918, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 9118947, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.992 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_bytes": { 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 6145, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 94726688, 2024-04-25T10:14:17.993 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 6145, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 94726688, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_ops": { 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 1024, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 6145, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 6145, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 6145, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 6145, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:14:17.995 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 1867, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 203503, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 1867, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 32, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 203503, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.996 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle-0x56125a8cca00": { 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 84, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 30124, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 94099311, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 30124, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 4084, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 94099227, 2024-04-25T10:14:17.997 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:17.998 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:17.998 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:17.998 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:17.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:17.998 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:14:17.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe2a4066f60 msgr2=0x7fe2a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe2a4066f60 0x7fe2a4069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806c670 tx=0x7fe2b8009190 comp rx=0 tx=0).stop 2024-04-25T10:14:17.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c80ae490 msgr2=0x7fe2c81527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c80ae490 0x7fe2c81527f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe2b001a0d0 tx=0x7fe2b001ace0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 shutdown_connections 2024-04-25T10:14:18.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe2a4066f60 0x7fe2a4069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2c815db50 0x7fe2c806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2c80aee30 0x7fe2c8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 --2- 172.21.15.87:0/2131080894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c80ae490 0x7fe2c81527f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 >> 172.21.15.87:0/2131080894 conn(0x7fe2c80d17a0 msgr2=0x7fe2c806fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 shutdown_connections 2024-04-25T10:14:18.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:17.937+0000 7fe2cdc00640 1 -- 172.21.15.87:0/2131080894 wait complete. 2024-04-25T10:14:18.001 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T10:14:18.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:18.001 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T10:14:18.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d9ba00640 1 Processor -- start 2024-04-25T10:14:18.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d9ba00640 1 -- start start 2024-04-25T10:14:18.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d9ba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3d940b3b20 0x7f3d9415e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d9ba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d940cf9e0 0x7f3d940cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d9ba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940b3f00 con 0x7f3d94070c70 2024-04-25T10:14:18.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d9ba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d940d0370 con 0x7f3d940cf9e0 2024-04-25T10:14:18.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d9ba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d940ad900 con 0x7f3d940b3b20 2024-04-25T10:14:18.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d940cf9e0 0x7f3d940cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d940cf9e0 0x7f3d940cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33862/0 (socket says 172.21.15.87:33862) 2024-04-25T10:14:18.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99a00640 1 -- 172.21.15.87:0/1211985670 learned_addr learned my addr 172.21.15.87:0/1211985670 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99000640 1 --2- 172.21.15.87:0/1211985670 >> v2:172.21.15.169:3300/0 conn(0x7f3d940b3b20 0x7f3d9415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99a00640 1 -- 172.21.15.87:0/1211985670 >> v2:172.21.15.169:3300/0 conn(0x7f3d940b3b20 msgr2=0x7f3d9415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99a00640 1 --2- 172.21.15.87:0/1211985670 >> v2:172.21.15.169:3300/0 conn(0x7f3d940b3b20 0x7f3d9415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99a00640 1 -- 172.21.15.87:0/1211985670 >> v1:172.21.15.87:6789/0 conn(0x7f3d94070c70 legacy=0x7f3d940b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.133+0000 7f3d99a00640 1 -- 172.21.15.87:0/1211985670 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9415e8a0 con 0x7f3d940cf9e0 2024-04-25T10:14:18.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d99a00640 1 --2- 172.21.15.87:0/1211985670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d940cfe30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3d940d0670 tx=0x7f3d8c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496a5b082554ba37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d93400640 1 -- 172.21.15.87:0/1211985670 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0a5880 con 0x7f3d940cf9e0 2024-04-25T10:14:18.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d93400640 1 -- 172.21.15.87:0/1211985670 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c0a5a20 con 0x7f3d940cf9e0 2024-04-25T10:14:18.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d93400640 1 -- 172.21.15.87:0/1211985670 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0a5df0 con 0x7f3d940cf9e0 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 -- 172.21.15.87:0/1211985670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 msgr2=0x7f3d940cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/1211985670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d940cfe30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3d940d0670 tx=0x7f3d8c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 -- 172.21.15.87:0/1211985670 shutdown_connections 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/1211985670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d940cfe30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/1211985670 >> v2:172.21.15.169:3300/0 conn(0x7f3d940b3b20 0x7f3d9415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 -- 172.21.15.87:0/1211985670 >> 172.21.15.87:0/1211985670 conn(0x7f3d94066b70 msgr2=0x7f3d94066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 -- 172.21.15.87:0/1211985670 shutdown_connections 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.137+0000 7f3d9ba00640 1 -- 172.21.15.87:0/1211985670 wait complete. 2024-04-25T10:14:18.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 Processor -- start 2024-04-25T10:14:18.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 -- start start 2024-04-25T10:14:18.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d94070c70 0x7f3d94155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d940b3b20 0x7f3d940cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d9415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9415f140 con 0x7f3d940cf9e0 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94070440 con 0x7f3d940b3b20 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940ae580 con 0x7f3d94070c70 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d9415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d9415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33874/0 (socket says 172.21.15.87:33874) 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 -- 172.21.15.87:0/4076109950 learned_addr learned my addr 172.21.15.87:0/4076109950 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d93e00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d940b3b20 0x7f3d940cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99000640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d94070c70 0x7f3d94155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 -- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d94070c70 msgr2=0x7f3d94155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d94070c70 0x7f3d94155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 -- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d940b3b20 msgr2=0x7f3d940cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d940b3b20 0x7f3d940cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 -- 172.21.15.87:0/4076109950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d941629b0 con 0x7f3d940cf9e0 2024-04-25T10:14:18.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d99a00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d9415a460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0195d0 tx=0x7f3d8c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d90c00640 1 -- 172.21.15.87:0/4076109950 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0bc020 con 0x7f3d940cf9e0 2024-04-25T10:14:18.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d90c00640 1 -- 172.21.15.87:0/4076109950 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c0b8670 con 0x7f3d940cf9e0 2024-04-25T10:14:18.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d94162c90 con 0x7f3d940cf9e0 2024-04-25T10:14:18.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d90c00640 1 -- 172.21.15.87:0/4076109950 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0b8a60 con 0x7f3d940cf9e0 2024-04-25T10:14:18.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.141+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d94156770 con 0x7f3d940cf9e0 2024-04-25T10:14:18.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d90c00640 1 -- 172.21.15.87:0/4076109950 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d8c0bf020 con 0x7f3d940cf9e0 2024-04-25T10:14:18.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d90c00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d74066f60 0x7f3d74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d90c00640 1 -- 172.21.15.87:0/4076109950 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d8c11fe80 con 0x7f3d940cf9e0 2024-04-25T10:14:18.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d99000640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d74066f60 0x7f3d74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 Processor -- start 2024-04-25T10:14:18.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 -- start start 2024-04-25T10:14:18.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d99000640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d74066f60 0x7f3d74069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d940cd6f0 tx=0x7f3d84009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006a520 0x55991006a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006ae40 0x559910021430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x559910021d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990fe6ed00 con 0x559910021970 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990fdf6df0 con 0x55991006ae40 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55990fe6ada0 con 0x55991006a520 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x559910021d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x559910021d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33890/0 (socket says 172.21.15.87:33890) 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.145+0000 7f3d99a00640 1 -- 172.21.15.87:0/3372491460 learned_addr learned my addr 172.21.15.87:0/3372491460 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99a00640 1 -- 172.21.15.87:0/3372491460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006a520 msgr2=0x55991006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99a00640 1 --2- 172.21.15.87:0/3372491460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006a520 0x55991006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99a00640 1 -- 172.21.15.87:0/3372491460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006ae40 msgr2=0x559910021430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99a00640 1 --2- 172.21.15.87:0/3372491460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006ae40 0x559910021430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99a00640 1 -- 172.21.15.87:0/3372491460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990fe29580 con 0x559910021970 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99a00640 1 --2- 172.21.15.87:0/3372491460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x559910021d50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0f8960 tx=0x7f3d8c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e14b6e14b94af61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d89600640 1 -- 172.21.15.87:0/3372491460 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0f48d0 con 0x559910021970 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d89600640 1 -- 172.21.15.87:0/3372491460 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c0f8e90 con 0x559910021970 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d89600640 1 -- 172.21.15.87:0/3372491460 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c1081c0 con 0x559910021970 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3372491460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 msgr2=0x559910021d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3372491460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x559910021d50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0f8960 tx=0x7f3d8c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3372491460 shutdown_connections 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3372491460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x559910021d50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3372491460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006ae40 0x559910021430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3372491460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006a520 0x55991006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3372491460 >> 172.21.15.87:0/3372491460 conn(0x55990ff08c10 msgr2=0x55991001fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3372491460 shutdown_connections 2024-04-25T10:14:18.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3372491460 wait complete. 2024-04-25T10:14:18.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- start start 2024-04-25T10:14:18.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 Processor -- start 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x5599100708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x559910070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990fe076d0 con 0x559910021970 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990fe29950 con 0x55991006a520 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55990fe6ed00 con 0x55991006ae40 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d9ee8c000 10 client.0 _ll_register_callbacks cb 0x55991000d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x5599100708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x5599100708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33896/0 (socket says 172.21.15.87:33896) 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 -- 172.21.15.87:0/2315406775 learned_addr learned my addr 172.21.15.87:0/2315406775 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d93e00640 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x559910070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99a00640 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 -- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 msgr2=0x5599100259f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100259f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 -- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 msgr2=0x559910070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x559910070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.149+0000 7f3d99000640 1 -- 172.21.15.87:0/2315406775 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55990fe0bd40 con 0x559910021970 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d99000640 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x5599100708e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3d840149c0 tx=0x7f3d840149f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2d7e2d22b9a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:18.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 1 -- 172.21.15.87:0/2315406775 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84025020 con 0x559910021970 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 1 -- 172.21.15.87:0/2315406775 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d840214b0 con 0x559910021970 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d8c0f8e90 con 0x559910021970 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 1 -- 172.21.15.87:0/2315406775 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d8401d040 con 0x559910021970 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 10 client.5391 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55990fe69cf0 con 0x559910021970 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 1 -- 172.21.15.87:0/2315406775 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c0f8e90 con 0x559910021970 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 1 -- 172.21.15.87:0/2315406775 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d84032080 con 0x559910021970 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 10 client.5391 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 10 client.5391 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 20 client.5391 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910026760 0x559910026b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 4 client.5391 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55990fe0a9d0 con 0x559910026760 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d93e00640 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910026760 0x559910026b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d93e00640 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910026760 0x559910026b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1b320e947cf481 server_cookie=94685297486dca11 in_seq=0 out_seq=0 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 10 client.5391 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 1 -- 172.21.15.87:0/2315406775 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55990fe0a9d0 con 0x559910026760 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 10 client.5391 handle_command_reply: tid=0 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 client.5391 shutdown 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 msgr2=0x5599100708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x5599100708e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3d840149c0 tx=0x7f3d840149f0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 shutdown_connections 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910026760 0x559910026b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x559910070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2315406775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559910021970 0x5599100708e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d7e000640 0 client.5391 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 >> 172.21.15.87:0/2315406775 conn(0x55990ff08c10 msgr2=0x559910020420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 shutdown_connections 2024-04-25T10:14:18.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2315406775 wait complete. 2024-04-25T10:14:18.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.153+0000 7f3d9ee8c000 20 client.5391 trim_cache size 0 max 16384 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 Processor -- start 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 -- start start 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559910021970 0x5599100c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006a520 0x5599100c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94202ba0 con 0x55991006a520 2024-04-25T10:14:18.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94157a80 con 0x559910021970 2024-04-25T10:14:18.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94156e80 con 0x55991006ae40 2024-04-25T10:14:18.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006a520 0x5599100c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006a520 0x5599100c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33912/0 (socket says 172.21.15.87:33912) 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 -- 172.21.15.87:0/3656747777 learned_addr learned my addr 172.21.15.87:0/3656747777 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d99000640 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559910021970 0x5599100c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d99a00640 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 -- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 msgr2=0x5599100c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 -- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559910021970 msgr2=0x5599100c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559910021970 0x5599100c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 -- 172.21.15.87:0/3656747777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990fef1d30 con 0x55991006a520 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d99a00640 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d93e00640 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006a520 0x5599100c5f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001480 tx=0x7f3d80007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831e00e641244685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d88c00640 1 -- 172.21.15.87:0/3656747777 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80019e60 con 0x55991006a520 2024-04-25T10:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d88c00640 1 -- 172.21.15.87:0/3656747777 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d800067e0 con 0x55991006a520 2024-04-25T10:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.169+0000 7f3d88c00640 1 -- 172.21.15.87:0/3656747777 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80006bb0 con 0x55991006a520 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3656747777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006a520 msgr2=0x5599100c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006a520 0x5599100c5f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001480 tx=0x7f3d80007e60 comp rx=0 tx=0).stop 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3656747777 shutdown_connections 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55991006ae40 0x5599100c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006a520 0x5599100c5f70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/3656747777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559910021970 0x5599100c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3656747777 >> 172.21.15.87:0/3656747777 conn(0x55990ff08c10 msgr2=0x559910069820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3656747777 shutdown_connections 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/3656747777 wait complete. 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- start start 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 Processor -- start 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559910021970 0x5599100cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x5599100cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006ae40 0x559910085000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990fe6e010 con 0x55991006ae40 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94157480 con 0x55991006a520 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55990fe076d0 con 0x559910021970 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 10 client.0 _ll_register_callbacks cb 0x7f3d8c0092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006ae40 0x559910085000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006ae40 0x559910085000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33928/0 (socket says 172.21.15.87:33928) 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 -- 172.21.15.87:0/2488488264 learned_addr learned my addr 172.21.15.87:0/2488488264 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99000640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559910021970 0x5599100cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d93e00640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x5599100cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 -- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559910021970 msgr2=0x5599100cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559910021970 0x5599100cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 -- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 msgr2=0x5599100cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x5599100cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 -- 172.21.15.87:0/2488488264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55990fe3b5a0 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d93e00640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x5599100cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99000640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559910021970 0x5599100cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d99a00640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006ae40 0x559910085000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0985a0 tx=0x7f3d8c0c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723265fc4cc36ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d7e000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d7e000640 1 -- 172.21.15.87:0/2488488264 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c10e060 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d7e000640 1 -- 172.21.15.87:0/2488488264 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d8c1081c0 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d800067e0 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d7e000640 1 -- 172.21.15.87:0/2488488264 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d8c0f48d0 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 10 client.5397 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55990fe15f90 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d7e000640 1 -- 172.21.15.87:0/2488488264 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d800067e0 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d7e000640 1 -- 172.21.15.87:0/2488488264 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d8c0f0a90 con 0x55991006ae40 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 10 client.5397 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 10 client.5397 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 20 client.5397 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:18.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910085540 0x559910085950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 4 client.5397 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T10:14:18.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.173+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x55990fe4b610 con 0x559910085540 2024-04-25T10:14:18.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.177+0000 7f3d99000640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910085540 0x559910085950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.177+0000 7f3d7e000640 10 client.5397 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.177+0000 7f3d99000640 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910085540 0x559910085950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5019b81450d50db server_cookie=c0de5cdf89d08d6e in_seq=0 out_seq=0 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d7e000640 1 -- 172.21.15.87:0/2488488264 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55990fe4b610 con 0x559910085540 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d7e000640 10 client.5397 handle_command_reply: tid=0 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 client.5397 shutdown 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006ae40 msgr2=0x559910085000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006ae40 0x559910085000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0985a0 tx=0x7f3d8c0c3000 comp rx=0 tx=0).stop 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 shutdown_connections 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55991006ae40 0x559910085000 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x559910085540 0x559910085950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55991006a520 0x5599100cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 --2- 172.21.15.87:0/2488488264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559910021970 0x5599100cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d7e000640 0 client.5397 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 >> 172.21.15.87:0/2488488264 conn(0x55990ff08c10 msgr2=0x55991006b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 shutdown_connections 2024-04-25T10:14:18.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 1 -- 172.21.15.87:0/2488488264 wait complete. 2024-04-25T10:14:18.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ee8c000 20 client.5397 trim_cache size 0 max 16384 2024-04-25T10:14:18.196 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:14:18.196 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:14:18.196 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "8c5596fc-c04d-4dda-96c0-6f0ba4c9eeb6", 2024-04-25T10:14:18.196 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:14:18.196 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d74066f60 msgr2=0x7f3d74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d74066f60 0x7f3d74069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d940cd6f0 tx=0x7f3d84009190 comp rx=0 tx=0).stop 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 msgr2=0x7f3d9415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d9415a460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0195d0 tx=0x7f3d8c019760 comp rx=0 tx=0).stop 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 shutdown_connections 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d74066f60 0x7f3d74069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d940cf9e0 0x7f3d9415a460 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d940b3b20 0x7f3d940cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 --2- 172.21.15.87:0/4076109950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d94070c70 0x7f3d94155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 >> 172.21.15.87:0/4076109950 conn(0x7f3d94066b70 msgr2=0x7f3d940af270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 shutdown_connections 2024-04-25T10:14:18.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.185+0000 7f3d9ba00640 1 -- 172.21.15.87:0/4076109950 wait complete. 2024-04-25T10:14:18.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:18.215 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:14:18.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.389+0000 7f1103800640 1 Processor -- start 2024-04-25T10:14:18.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.389+0000 7f1103800640 1 -- start start 2024-04-25T10:14:18.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1103800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc15dcc0 0x7f10fc162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1103800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10fc0670b0 0x7f10fc0b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1103800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc15f0b0 con 0x7f10fc0b5480 2024-04-25T10:14:18.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1103800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10fc161800 con 0x7f10fc0670b0 2024-04-25T10:14:18.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1103800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0b3b20 con 0x7f10fc15dcc0 2024-04-25T10:14:18.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f10fbe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f10fc0b5480 0x7f10fc0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:42266/0 (socket says 172.21.15.87:42266) 2024-04-25T10:14:18.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f10fbe00640 1 -- 172.21.15.87:0/2945190143 learned_addr learned my addr 172.21.15.87:0/2945190143 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:18.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1100e00640 1 --2- 172.21.15.87:0/2945190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc15dcc0 0x7f10fc162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1101800640 1 --2- 172.21.15.87:0/2945190143 >> v2:172.21.15.169:3300/0 conn(0x7f10fc0670b0 0x7f10fc0b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1100e00640 1 -- 172.21.15.87:0/2945190143 >> v2:172.21.15.169:3300/0 conn(0x7f10fc0670b0 msgr2=0x7f10fc0b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1100e00640 1 --2- 172.21.15.87:0/2945190143 >> v2:172.21.15.169:3300/0 conn(0x7f10fc0670b0 0x7f10fc0b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f10fb400640 1 -- 172.21.15.87:0/2945190143 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131809785 0 0) 0x7f10fc15f0b0 con 0x7f10fc0b5480 2024-04-25T10:14:18.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1100e00640 1 -- 172.21.15.87:0/2945190143 >> v1:172.21.15.169:6789/0 conn(0x7f10fc0b5480 legacy=0x7f10fc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.393+0000 7f1100e00640 1 -- 172.21.15.87:0/2945190143 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc0b3e00 con 0x7f10fc15dcc0 2024-04-25T10:14:18.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1100e00640 1 --2- 172.21.15.87:0/2945190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc15dcc0 0x7f10fc162250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097c0 tx=0x7f10f001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241d535e72241d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f10fb400640 1 -- 172.21.15.87:0/2945190143 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00b1070 con 0x7f10fc15dcc0 2024-04-25T10:14:18.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f10fb400640 1 -- 172.21.15.87:0/2945190143 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00a8050 con 0x7f10fc15dcc0 2024-04-25T10:14:18.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f10fb400640 1 -- 172.21.15.87:0/2945190143 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00ac860 con 0x7f10fc15dcc0 2024-04-25T10:14:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 -- 172.21.15.87:0/2945190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc15dcc0 msgr2=0x7f10fc162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 --2- 172.21.15.87:0/2945190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc15dcc0 0x7f10fc162250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097c0 tx=0x7f10f001c400 comp rx=0 tx=0).stop 2024-04-25T10:14:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 -- 172.21.15.87:0/2945190143 shutdown_connections 2024-04-25T10:14:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 --2- 172.21.15.87:0/2945190143 >> v2:172.21.15.169:3300/0 conn(0x7f10fc0670b0 0x7f10fc0b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 --2- 172.21.15.87:0/2945190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc15dcc0 0x7f10fc162250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 -- 172.21.15.87:0/2945190143 >> 172.21.15.87:0/2945190143 conn(0x7f10fc06ac90 msgr2=0x7f10fc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 -- 172.21.15.87:0/2945190143 shutdown_connections 2024-04-25T10:14:18.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.397+0000 7f1103800640 1 -- 172.21.15.87:0/2945190143 wait complete. 2024-04-25T10:14:18.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 Processor -- start 2024-04-25T10:14:18.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 -- start start 2024-04-25T10:14:18.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10fc0670b0 0x7f10fc0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc0b5480 0x7f10fc1fa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10fc15dcc0 0x7f10fc1fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0d0ec0 con 0x7f10fc15dcc0 2024-04-25T10:14:18.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0d0d40 con 0x7f10fc0670b0 2024-04-25T10:14:18.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.401+0000 7f1103800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0d1040 con 0x7f10fc0b5480 2024-04-25T10:14:18.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1100e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10fc0670b0 0x7f10fc0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10fc15dcc0 0x7f10fc1fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10fc15dcc0 0x7f10fc1fa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33936/0 (socket says 172.21.15.87:33936) 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 -- 172.21.15.87:0/1462265208 learned_addr learned my addr 172.21.15.87:0/1462265208 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f10fbe00640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc0b5480 0x7f10fc1fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 -- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc0b5480 msgr2=0x7f10fc1fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc0b5480 0x7f10fc1fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 -- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10fc0670b0 msgr2=0x7f10fc0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10fc0670b0 0x7f10fc0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 -- 172.21.15.87:0/1462265208 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc152b80 con 0x7f10fc15dcc0 2024-04-25T10:14:18.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1100e00640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10fc0670b0 0x7f10fc0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1101800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10fc15dcc0 0x7f10fc1fa7c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009e30 tx=0x7f10f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f10f8c00640 1 -- 172.21.15.87:0/1462265208 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4018020 con 0x7f10fc15dcc0 2024-04-25T10:14:18.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f10f8c00640 1 -- 172.21.15.87:0/1462265208 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4014480 con 0x7f10fc15dcc0 2024-04-25T10:14:18.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f10f8c00640 1 -- 172.21.15.87:0/1462265208 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4014870 con 0x7f10fc15dcc0 2024-04-25T10:14:18.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10fc152640 con 0x7f10fc15dcc0 2024-04-25T10:14:18.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.405+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc0cd770 con 0x7f10fc15dcc0 2024-04-25T10:14:18.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.409+0000 7f10f8c00640 1 -- 172.21.15.87:0/1462265208 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10f401b020 con 0x7f10fc15dcc0 2024-04-25T10:14:18.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.409+0000 7f10f8c00640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.409+0000 7f10f8c00640 1 -- 172.21.15.87:0/1462265208 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10f4085e70 con 0x7f10fc15dcc0 2024-04-25T10:14:18.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.409+0000 7f1106cda000 1 Processor -- start 2024-04-25T10:14:18.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 -- start start 2024-04-25T10:14:18.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1100e00640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1100e00640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f10dc066f60 0x7f10dc069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10f00ae000 tx=0x7f10f00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a873364b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873365480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8731b1ff0 con 0x55a8733af2f0 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87313a150 con 0x55a8733ae9d0 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8731ae090 con 0x55a8733650a0 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f10fbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a873364b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1100e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1101800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873365480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f10fbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a873364b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33948/0 (socket says 172.21.15.87:33948) 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f10fbe00640 1 -- 172.21.15.87:0/1575467167 learned_addr learned my addr 172.21.15.87:0/1575467167 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1101800640 1 -- 172.21.15.87:0/1575467167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 msgr2=0x55a8733aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1101800640 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1101800640 1 -- 172.21.15.87:0/1575467167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 msgr2=0x55a873364b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1101800640 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a873364b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1101800640 1 -- 172.21.15.87:0/1575467167 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8733480d0 con 0x55a8733650a0 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f10fbe00640 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a873364b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1101800640 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873365480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10f4056470 tx=0x7f10f4056770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4814edf98e95bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f10ed600640 1 -- 172.21.15.87:0/1575467167 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4082070 con 0x55a8733650a0 2024-04-25T10:14:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f10ed600640 1 -- 172.21.15.87:0/1575467167 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4084410 con 0x55a8733650a0 2024-04-25T10:14:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f10ed600640 1 -- 172.21.15.87:0/1575467167 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f406e070 con 0x55a8733650a0 2024-04-25T10:14:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 -- 172.21.15.87:0/1575467167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 msgr2=0x55a873365480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.413+0000 7f1106cda000 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873365480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10f4056470 tx=0x7f10f4056770 comp rx=0 tx=0).stop 2024-04-25T10:14:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- 172.21.15.87:0/1575467167 shutdown_connections 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873365480 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a873364b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 --2- 172.21.15.87:0/1575467167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- 172.21.15.87:0/1575467167 >> 172.21.15.87:0/1575467167 conn(0x55a873361020 msgr2=0x55a873363440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- 172.21.15.87:0/1575467167 shutdown_connections 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- 172.21.15.87:0/1575467167 wait complete. 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- start start 2024-04-25T10:14:18.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 Processor -- start 2024-04-25T10:14:18.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a8733b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a8733b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a873176250 con 0x55a8733af2f0 2024-04-25T10:14:18.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87316c690 con 0x55a8733650a0 2024-04-25T10:14:18.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f10fbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f10fbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:59264/0 (socket says 172.21.15.87:59264) 2024-04-25T10:14:18.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f10fbe00640 1 -- 172.21.15.87:0/3854946367 learned_addr learned my addr 172.21.15.87:0/3854946367 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a87313aa40 con 0x55a8733ae9d0 2024-04-25T10:14:18.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1100e00640 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a8733b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.417+0000 7f1106cda000 10 client.0 _ll_register_callbacks cb 0x55a87334fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:18.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1101800640 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a8733b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10fbe00640 1 -- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 msgr2=0x55a8733b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10fbe00640 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a8733b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10fbe00640 1 -- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 msgr2=0x55a8733b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10fbe00640 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a8733b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10fbe00640 1 -- 172.21.15.87:0/3854946367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a87325f1c0 con 0x55a8733ae9d0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10fbe00640 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b0e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10e8004de0 tx=0x7f10e80016d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d6928baf03f2f87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10e2000640 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10e2000640 1 -- 172.21.15.87:0/3854946367 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80145e0 con 0x55a8733ae9d0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10e2000640 1 -- 172.21.15.87:0/3854946367 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e800c780 con 0x55a8733ae9d0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10f4084410 con 0x55a8733ae9d0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10e2000640 1 -- 172.21.15.87:0/3854946367 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e800c9c0 con 0x55a8733ae9d0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 10 client.4916 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a87325b170 con 0x55a8733ae9d0 2024-04-25T10:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10e2000640 1 -- 172.21.15.87:0/3854946367 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4084410 con 0x55a8733ae9d0 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10e2000640 1 -- 172.21.15.87:0/3854946367 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10e800cb60 con 0x55a8733ae9d0 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 10 client.4916 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 10 client.4916 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 20 client.4916 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733b1dc0 0x55a8733c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 4 client.4916 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a87314dc60 con 0x55a8733b1dc0 2024-04-25T10:14:18.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1100e00640 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733b1dc0 0x55a8733c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f1100e00640 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733b1dc0 0x55a8733c9780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a78ceb5f8b33f6a server_cookie=85d00e80259205cd in_seq=0 out_seq=0 2024-04-25T10:14:18.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.421+0000 7f10e2000640 10 client.4916 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f10e2000640 1 -- 172.21.15.87:0/3854946367 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a87314dc60 con 0x55a8733b1dc0 2024-04-25T10:14:18.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f10e2000640 10 client.4916 handle_command_reply: tid=0 2024-04-25T10:14:18.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 client.4916 shutdown 2024-04-25T10:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 msgr2=0x55a8733b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b0e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10e8004de0 tx=0x7f10e80016d0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 shutdown_connections 2024-04-25T10:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a8733af2f0 0x55a8733b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b0e70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733b1dc0 0x55a8733c9780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 --2- 172.21.15.87:0/3854946367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a8733b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f10e2000640 0 client.4916 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 >> 172.21.15.87:0/3854946367 conn(0x55a873361020 msgr2=0x55a873363ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 shutdown_connections 2024-04-25T10:14:18.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 1 -- 172.21.15.87:0/3854946367 wait complete. 2024-04-25T10:14:18.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.425+0000 7f1106cda000 20 client.4916 trim_cache size 0 max 16384 2024-04-25T10:14:18.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 Processor -- start 2024-04-25T10:14:18.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 -- start start 2024-04-25T10:14:18.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873364420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a873364960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0aa9f0 con 0x55a873361020 2024-04-25T10:14:18.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0a9bd0 con 0x55a8733650a0 2024-04-25T10:14:18.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1106cda000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc151f60 con 0x55a8733ae9d0 2024-04-25T10:14:18.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1100e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873364420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1100e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873364420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33958/0 (socket says 172.21.15.87:33958) 2024-04-25T10:14:18.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1100e00640 1 -- 172.21.15.87:0/1415592277 learned_addr learned my addr 172.21.15.87:0/1415592277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f10fbe00640 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a873364960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.445+0000 7f1101800640 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 -- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 msgr2=0x55a8733b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 -- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 msgr2=0x55a873364960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a873364960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 -- 172.21.15.87:0/1415592277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8731f8de0 con 0x55a873361020 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1101800640 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873364420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10f001a030 tx=0x7f10f00a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27bbfafcadeeb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f10ecc00640 1 -- 172.21.15.87:0/1415592277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00cd070 con 0x55a873361020 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f10ecc00640 1 -- 172.21.15.87:0/1415592277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00a8040 con 0x55a873361020 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f10ecc00640 1 -- 172.21.15.87:0/1415592277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00b1040 con 0x55a873361020 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- 172.21.15.87:0/1415592277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 msgr2=0x55a873364420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873364420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10f001a030 tx=0x7f10f00a9000 comp rx=0 tx=0).stop 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- 172.21.15.87:0/1415592277 shutdown_connections 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733ae9d0 0x55a8733b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733650a0 0x55a873364960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 --2- 172.21.15.87:0/1415592277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873364420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- 172.21.15.87:0/1415592277 >> 172.21.15.87:0/1415592277 conn(0x55a8733917e0 msgr2=0x55a8733b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- 172.21.15.87:0/1415592277 shutdown_connections 2024-04-25T10:14:18.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- 172.21.15.87:0/1415592277 wait complete. 2024-04-25T10:14:18.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- start start 2024-04-25T10:14:18.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 Processor -- start 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873417c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873418170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87314ce00 con 0x55a873361020 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0a95d0 con 0x55a8733ae9d0 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8731b1300 con 0x55a8733650a0 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1106cda000 10 client.0 _ll_register_callbacks cb 0x55a873330040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873417c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873417c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33968/0 (socket says 172.21.15.87:33968) 2024-04-25T10:14:18.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1100e00640 1 -- 172.21.15.87:0/4253974448 learned_addr learned my addr 172.21.15.87:0/4253974448 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f10fbe00640 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873418170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.449+0000 7f1101800640 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1100e00640 1 -- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 msgr2=0x55a873418170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1100e00640 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873418170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1100e00640 1 -- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 msgr2=0x55a8733c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1100e00640 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1100e00640 1 -- 172.21.15.87:0/4253974448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8731538e0 con 0x55a873361020 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1100e00640 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873417c30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10f001a030 tx=0x7f10f001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b564aa98f5c47c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10e2000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10e2000640 1 -- 172.21.15.87:0/4253974448 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00cd030 con 0x55a873361020 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10e2000640 1 -- 172.21.15.87:0/4253974448 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10f00c8670 con 0x55a873361020 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10f00a8040 con 0x55a873361020 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10e2000640 1 -- 172.21.15.87:0/4253974448 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10f00db050 con 0x55a873361020 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 10 client.5412 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:18.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a87320b8f0 con 0x55a873361020 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10e2000640 1 -- 172.21.15.87:0/4253974448 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00a8040 con 0x55a873361020 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10e2000640 1 -- 172.21.15.87:0/4253974448 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10f00de020 con 0x55a873361020 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 10 client.5412 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 10 client.5412 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 20 client.5412 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733af2f0 0x55a8733ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 4 client.5412 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a8731bf530 con 0x55a8733af2f0 2024-04-25T10:14:18.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10fbe00640 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733af2f0 0x55a8733ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10fbe00640 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733af2f0 0x55a8733ca8e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9033797d060e7497 server_cookie=c3936814f6e5420 in_seq=0 out_seq=0 2024-04-25T10:14:18.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.453+0000 7f10e2000640 10 client.5412 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f10e2000640 1 -- 172.21.15.87:0/4253974448 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x55a8731bf530 con 0x55a8733af2f0 2024-04-25T10:14:18.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f10e2000640 10 client.5412 handle_command_reply: tid=0 2024-04-25T10:14:18.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 client.5412 shutdown 2024-04-25T10:14:18.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 msgr2=0x55a873417c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873417c30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10f001a030 tx=0x7f10f001ac40 comp rx=0 tx=0).stop 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 shutdown_connections 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55a8733af2f0 0x55a8733ca8e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8733ae9d0 0x55a8733c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a8733650a0 0x55a873418170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f10e2000640 0 client.5412 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 --2- 172.21.15.87:0/4253974448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a873361020 0x55a873417c30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 >> 172.21.15.87:0/4253974448 conn(0x55a8733917e0 msgr2=0x55a8733b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.489+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 shutdown_connections 2024-04-25T10:14:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1106cda000 1 -- 172.21.15.87:0/4253974448 wait complete. 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1106cda000 20 client.5412 trim_cache size 0 max 16384 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "scrub active (441 inodes in the stack)", 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout: "8c5596fc-c04d-4dda-96c0-6f0ba4c9eeb6": { 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/", 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "8c5596fc-c04d-4dda-96c0-6f0ba4c9eeb6", 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T10:14:18.501 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:18.502 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:18.502 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:14:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f10dc066f60 msgr2=0x7f10dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f10dc066f60 0x7f10dc069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10f00ae000 tx=0x7f10f00a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10fc15dcc0 msgr2=0x7f10fc1fa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10fc15dcc0 0x7f10fc1fa7c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009e30 tx=0x7f10f400c490 comp rx=0 tx=0).stop 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 shutdown_connections 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10fc15dcc0 0x7f10fc1fa7c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10fc0b5480 0x7f10fc1fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 --2- 172.21.15.87:0/1462265208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10fc0670b0 0x7f10fc0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 >> 172.21.15.87:0/1462265208 conn(0x7f10fc06ac90 msgr2=0x7f10fc0afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 shutdown_connections 2024-04-25T10:14:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:18.493+0000 7f1103800640 1 -- 172.21.15.87:0/1462265208 wait complete. 2024-04-25T10:14:18.528 INFO:tasks.cephfs.filesystem:scrub status for tag:8c5596fc-c04d-4dda-96c0-6f0ba4c9eeb6 - {'path': '/', 'tag': '8c5596fc-c04d-4dda-96c0-6f0ba4c9eeb6', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T10:14:21.709 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:21.712 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:48.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:48.530 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:14:48.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.672+0000 7f8310600640 1 Processor -- start 2024-04-25T10:14:48.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f8310600640 1 -- start start 2024-04-25T10:14:48.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f8310600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83080aef80 0x7f830815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f8310600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8308070350 0x7f830815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f8310600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830815b080 con 0x7f830815bac0 2024-04-25T10:14:48.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f8310600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830815adb0 con 0x7f83080aef80 2024-04-25T10:14:48.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f8310600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830815eb60 con 0x7f8308070350 2024-04-25T10:14:48.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8308070350 0x7f830815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83080aef80 0x7f830815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8308070350 0x7f830815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:43252/0 (socket says 172.21.15.87:43252) 2024-04-25T10:14:48.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830d200640 1 -- 172.21.15.87:0/3306476832 learned_addr learned my addr 172.21.15.87:0/3306476832 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830e600640 1 --1- 172.21.15.87:0/3306476832 >> v1:172.21.15.169:6789/0 conn(0x7f830815bac0 0x7f83080b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:60816/0 (socket says 172.21.15.87:60816) 2024-04-25T10:14:48.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830d200640 1 -- 172.21.15.87:0/3306476832 >> v2:172.21.15.169:3300/0 conn(0x7f83080aef80 msgr2=0x7f830815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830d200640 1 --2- 172.21.15.87:0/3306476832 >> v2:172.21.15.169:3300/0 conn(0x7f83080aef80 0x7f830815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830d200640 1 -- 172.21.15.87:0/3306476832 >> v1:172.21.15.169:6789/0 conn(0x7f830815bac0 legacy=0x7f83080b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830d200640 1 -- 172.21.15.87:0/3306476832 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830815ee40 con 0x7f8308070350 2024-04-25T10:14:48.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.676+0000 7f830dc00640 1 --2- 172.21.15.87:0/3306476832 >> v2:172.21.15.169:3300/0 conn(0x7f83080aef80 0x7f830815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:48.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f830d200640 1 --2- 172.21.15.87:0/3306476832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8308070350 0x7f830815b580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009870 tx=0x7f82f401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33da7931c9612a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8307e00640 1 -- 172.21.15.87:0/3306476832 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40a9400 con 0x7f8308070350 2024-04-25T10:14:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8307e00640 1 -- 172.21.15.87:0/3306476832 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40a95a0 con 0x7f8308070350 2024-04-25T10:14:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8307e00640 1 -- 172.21.15.87:0/3306476832 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40a9970 con 0x7f8308070350 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 -- 172.21.15.87:0/3306476832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8308070350 msgr2=0x7f830815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 --2- 172.21.15.87:0/3306476832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8308070350 0x7f830815b580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009870 tx=0x7f82f401c560 comp rx=0 tx=0).stop 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 -- 172.21.15.87:0/3306476832 shutdown_connections 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 --2- 172.21.15.87:0/3306476832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8308070350 0x7f830815b580 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 --2- 172.21.15.87:0/3306476832 >> v2:172.21.15.169:3300/0 conn(0x7f83080aef80 0x7f830815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 -- 172.21.15.87:0/3306476832 >> 172.21.15.87:0/3306476832 conn(0x7f83080669a0 msgr2=0x7f8308066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 -- 172.21.15.87:0/3306476832 shutdown_connections 2024-04-25T10:14:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.680+0000 7f8310600640 1 -- 172.21.15.87:0/3306476832 wait complete. 2024-04-25T10:14:48.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 Processor -- start 2024-04-25T10:14:48.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 -- start start 2024-04-25T10:14:48.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8308070350 0x7f830806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83080aef80 0x7f83081f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f830815bac0 0x7f8308159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308165810 con 0x7f8308070350 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83080b3ae0 con 0x7f83080aef80 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8308165990 con 0x7f830815bac0 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8308070350 0x7f830806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8308070350 0x7f830806dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51800/0 (socket says 172.21.15.87:51800) 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 -- 172.21.15.87:0/2199686388 learned_addr learned my addr 172.21.15.87:0/2199686388 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830e600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f830815bac0 0x7f8308159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830d200640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83080aef80 0x7f83081f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 -- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f830815bac0 msgr2=0x7f8308159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f830815bac0 0x7f8308159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 -- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83080aef80 msgr2=0x7f83081f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83080aef80 0x7f83081f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 -- 172.21.15.87:0/2199686388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830815a160 con 0x7f8308070350 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f830dc00640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8308070350 0x7f830806dc40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00efc0 tx=0x7f82fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8305600640 1 -- 172.21.15.87:0/2199686388 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc018020 con 0x7f8308070350 2024-04-25T10:14:48.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8305600640 1 -- 172.21.15.87:0/2199686388 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc014680 con 0x7f8308070350 2024-04-25T10:14:48.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081f6fa0 con 0x7f8308070350 2024-04-25T10:14:48.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8305600640 1 -- 172.21.15.87:0/2199686388 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc014a50 con 0x7f8308070350 2024-04-25T10:14:48.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.684+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081f7560 con 0x7f8308070350 2024-04-25T10:14:48.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8305600640 1 -- 172.21.15.87:0/2199686388 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82fc01b020 con 0x7f8308070350 2024-04-25T10:14:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8305600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82e8066f60 0x7f82e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8305600640 1 -- 172.21.15.87:0/2199686388 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f82fc085ef0 con 0x7f8308070350 2024-04-25T10:14:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f830d200640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82e8066f60 0x7f82e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 Processor -- start 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 -- start start 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f830d200640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82e8066f60 0x7f82e8069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ae000 tx=0x7f82f40b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255ec2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255e79480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561255cc5ff0 con 0x561255ec32f0 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561255c4e150 con 0x561255ec29d0 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561255cc2090 con 0x561255e790a0 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.688+0000 7f830d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51804/0 (socket says 172.21.15.87:51804) 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 -- 172.21.15.87:0/586486015 learned_addr learned my addr 172.21.15.87:0/586486015 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830dc00640 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255ec2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830e600640 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255e79480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 -- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 msgr2=0x561255e79480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255e79480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 -- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 msgr2=0x561255ec2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255ec2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 -- 172.21.15.87:0/586486015 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561255e5c0d0 con 0x561255ec32f0 2024-04-25T10:14:48.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78b60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f82f401a190 tx=0x7f82f4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c700a33ed9ce409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f82fa000640 1 -- 172.21.15.87:0/586486015 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40cb070 con 0x561255ec32f0 2024-04-25T10:14:48.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f82fa000640 1 -- 172.21.15.87:0/586486015 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40bb540 con 0x561255ec32f0 2024-04-25T10:14:48.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f82fa000640 1 -- 172.21.15.87:0/586486015 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40bb8f0 con 0x561255ec32f0 2024-04-25T10:14:48.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- 172.21.15.87:0/586486015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 msgr2=0x561255e78b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78b60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f82f401a190 tx=0x7f82f4004740 comp rx=0 tx=0).stop 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- 172.21.15.87:0/586486015 shutdown_connections 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255e79480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78b60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 --2- 172.21.15.87:0/586486015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255ec2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- 172.21.15.87:0/586486015 >> 172.21.15.87:0/586486015 conn(0x561255e75020 msgr2=0x561255e77440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- 172.21.15.87:0/586486015 shutdown_connections 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- 172.21.15.87:0/586486015 wait complete. 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- start start 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 Processor -- start 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ea77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec29d0 0x561255ea7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ea8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561255c8a250 con 0x561255ec29d0 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561255c80690 con 0x561255ec32f0 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561255c4ea40 con 0x561255e790a0 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f8313a1d000 10 client.0 _ll_register_callbacks cb 0x561255e63e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec29d0 0x561255ea7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec29d0 0x561255ea7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51808/0 (socket says 172.21.15.87:51808) 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830d200640 1 -- 172.21.15.87:0/3802984954 learned_addr learned my addr 172.21.15.87:0/3802984954 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830dc00640 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ea77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.692+0000 7f830e600640 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ea8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830d200640 1 -- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 msgr2=0x561255ea77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830d200640 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ea77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830d200640 1 -- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 msgr2=0x561255ea8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830d200640 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ea8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830d200640 1 -- 172.21.15.87:0/3802984954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561255d731c0 con 0x561255ec29d0 2024-04-25T10:14:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830d200640 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec29d0 0x561255ea7d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f82f401a190 tx=0x7f82f401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963a904de4c80630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 1 -- 172.21.15.87:0/3802984954 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40cb040 con 0x561255ec29d0 2024-04-25T10:14:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 1 -- 172.21.15.87:0/3802984954 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f82f40bbbf0 con 0x561255ec29d0 2024-04-25T10:14:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f40bb540 con 0x561255ec29d0 2024-04-25T10:14:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 1 -- 172.21.15.87:0/3802984954 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f40dc050 con 0x561255ec29d0 2024-04-25T10:14:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 10 client.5421 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x561255d6f170 con 0x561255ec29d0 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 1 -- 172.21.15.87:0/3802984954 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40bb540 con 0x561255ec29d0 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 1 -- 172.21.15.87:0/3802984954 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82f40df020 con 0x561255ec29d0 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 10 client.5421 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 10 client.5421 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 20 client.5421 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed2c80 0x561255ed5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 4 client.5421 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561255c61c60 con 0x561255ed2c80 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830dc00640 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed2c80 0x561255ed5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f830dc00640 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed2c80 0x561255ed5070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf06d1b8d54daf53 server_cookie=cd48ff5ab1c882b8 in_seq=0 out_seq=0 2024-04-25T10:14:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 10 client.5421 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 1 -- 172.21.15.87:0/3802984954 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561255c61c60 con 0x561255ed2c80 2024-04-25T10:14:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 10 client.5421 handle_command_reply: tid=0 2024-04-25T10:14:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 client.5421 shutdown 2024-04-25T10:14:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec29d0 msgr2=0x561255ea7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec29d0 0x561255ea7d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f82f401a190 tx=0x7f82f401ada0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 shutdown_connections 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed2c80 0x561255ed5070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ea8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec29d0 0x561255ea7d30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f82eea00640 0 client.5421 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 --2- 172.21.15.87:0/3802984954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ea77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 >> 172.21.15.87:0/3802984954 conn(0x561255e75020 msgr2=0x561255e77ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 shutdown_connections 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 1 -- 172.21.15.87:0/3802984954 wait complete. 2024-04-25T10:14:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.696+0000 7f8313a1d000 20 client.5421 trim_cache size 0 max 16384 2024-04-25T10:14:48.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 Processor -- start 2024-04-25T10:14:48.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 -- start start 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ec1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255e78420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308203030 con 0x561255ec32f0 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308202240 con 0x561255ec29d0 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830815dd80 con 0x561255e790a0 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51814/0 (socket says 172.21.15.87:51814) 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 -- 172.21.15.87:0/1571005619 learned_addr learned my addr 172.21.15.87:0/1571005619 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830d200640 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255e78420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830dc00640 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ec1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 -- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 msgr2=0x561255ec1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ec1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 -- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 msgr2=0x561255e78420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255e78420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 -- 172.21.15.87:0/1571005619 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561255d0cde0 con 0x561255ec32f0 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830d200640 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255e78420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f830e600640 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8300005cf0 tx=0x7f8300007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59119fa1c3c96cca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f82f9600640 1 -- 172.21.15.87:0/1571005619 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830001a070 con 0x561255ec32f0 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f82f9600640 1 -- 172.21.15.87:0/1571005619 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8300007550 con 0x561255ec32f0 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.712+0000 7f82f9600640 1 -- 172.21.15.87:0/1571005619 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300013ab0 con 0x561255ec32f0 2024-04-25T10:14:48.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- 172.21.15.87:0/1571005619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 msgr2=0x561255e78960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8300005cf0 tx=0x7f8300007420 comp rx=0 tx=0).stop 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- 172.21.15.87:0/1571005619 shutdown_connections 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255ec32f0 0x561255e78960 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec29d0 0x561255e78420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 --2- 172.21.15.87:0/1571005619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255e790a0 0x561255ec1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- 172.21.15.87:0/1571005619 >> 172.21.15.87:0/1571005619 conn(0x561255e75020 msgr2=0x561255ec10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- 172.21.15.87:0/1571005619 shutdown_connections 2024-04-25T10:14:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- 172.21.15.87:0/1571005619 wait complete. 2024-04-25T10:14:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- start start 2024-04-25T10:14:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 Processor -- start 2024-04-25T10:14:48.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255e790a0 0x561255f2bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255ec29d0 0x561255f2c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ed5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561255c60e00 con 0x561255e790a0 2024-04-25T10:14:48.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081532c0 con 0x561255ec32f0 2024-04-25T10:14:48.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561255cc5300 con 0x561255ec29d0 2024-04-25T10:14:48.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 10 client.0 _ll_register_callbacks cb 0x561255e44040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:48.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255e790a0 0x561255f2bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255e790a0 0x561255f2bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51818/0 (socket says 172.21.15.87:51818) 2024-04-25T10:14:48.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 -- 172.21.15.87:0/2869528003 learned_addr learned my addr 172.21.15.87:0/2869528003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830d200640 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255ec29d0 0x561255f2c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830e600640 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ed5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 -- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255ec29d0 msgr2=0x561255f2c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255ec29d0 0x561255f2c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 -- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 msgr2=0x561255ed5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ed5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 -- 172.21.15.87:0/2869528003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561255c678e0 con 0x561255e790a0 2024-04-25T10:14:48.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f830dc00640 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255e790a0 0x561255f2bb20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0566d0 tx=0x7f82fc016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb83c7b6a3664f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f82eea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:48.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f82eea00640 1 -- 172.21.15.87:0/2869528003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc073060 con 0x561255e790a0 2024-04-25T10:14:48.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f82eea00640 1 -- 172.21.15.87:0/2869528003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f82fc024d90 con 0x561255e790a0 2024-04-25T10:14:48.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8300007550 con 0x561255e790a0 2024-04-25T10:14:48.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f82eea00640 1 -- 172.21.15.87:0/2869528003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82fc079050 con 0x561255e790a0 2024-04-25T10:14:48.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 10 client.5427 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:48.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x561255d1f8f0 con 0x561255e790a0 2024-04-25T10:14:48.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f82eea00640 1 -- 172.21.15.87:0/2869528003 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8300007550 con 0x561255e790a0 2024-04-25T10:14:48.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.716+0000 7f82eea00640 1 -- 172.21.15.87:0/2869528003 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82fc07c020 con 0x561255e790a0 2024-04-25T10:14:48.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 10 client.5427 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:48.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 10 client.5427 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:48.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 20 client.5427 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:48.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed6020 0x561255ed6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 4 client.5427 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:48.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561255cd3530 con 0x561255ed6020 2024-04-25T10:14:48.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f830d200640 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed6020 0x561255ed6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f82eea00640 10 client.5427 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f830d200640 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed6020 0x561255ed6430 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cffa0c1f129868a server_cookie=46d6750d37e4cc04 in_seq=0 out_seq=0 2024-04-25T10:14:48.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f82eea00640 1 -- 172.21.15.87:0/2869528003 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561255cd3530 con 0x561255ed6020 2024-04-25T10:14:48.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f82eea00640 10 client.5427 handle_command_reply: tid=0 2024-04-25T10:14:48.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 client.5427 shutdown 2024-04-25T10:14:48.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255e790a0 msgr2=0x561255f2bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255e790a0 0x561255f2bb20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0566d0 tx=0x7f82fc016040 comp rx=0 tx=0).stop 2024-04-25T10:14:48.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 shutdown_connections 2024-04-25T10:14:48.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x561255ed6020 0x561255ed6430 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561255ec32f0 0x561255ed5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561255ec29d0 0x561255f2c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f82eea00640 0 client.5427 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 --2- 172.21.15.87:0/2869528003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561255e790a0 0x561255f2bb20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 >> 172.21.15.87:0/2869528003 conn(0x561255e75020 msgr2=0x561255ec1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 shutdown_connections 2024-04-25T10:14:48.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 1 -- 172.21.15.87:0/2869528003 wait complete. 2024-04-25T10:14:48.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8313a1d000 20 client.5427 trim_cache size 0 max 16384 2024-04-25T10:14:48.738 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:14:48.738 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:14:48.739 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:14:48.739 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:14:48.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82e8066f60 msgr2=0x7f82e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82e8066f60 0x7f82e8069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ae000 tx=0x7f82f40b3040 comp rx=0 tx=0).stop 2024-04-25T10:14:48.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8308070350 msgr2=0x7f830806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8308070350 0x7f830806dc40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00efc0 tx=0x7f82fc00c490 comp rx=0 tx=0).stop 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 shutdown_connections 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f82e8066f60 0x7f82e8069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f830815bac0 0x7f8308159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83080aef80 0x7f83081f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 --2- 172.21.15.87:0/2199686388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8308070350 0x7f830806dc40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 >> 172.21.15.87:0/2199686388 conn(0x7f83080669a0 msgr2=0x7f83080ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 shutdown_connections 2024-04-25T10:14:48.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.720+0000 7f8310600640 1 -- 172.21.15.87:0/2199686388 wait complete. 2024-04-25T10:14:48.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:14:48.745 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:14:48.745 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:14:48.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.876+0000 7f8666600640 1 Processor -- start 2024-04-25T10:14:48.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.876+0000 7f8666600640 1 -- start start 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f8666600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86600b3480 0x7f866015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f8666600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8660071bc0 0x7f8660072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f8666600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86600ad840 con 0x7f86600cfb50 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f8666600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8660072570 con 0x7f86600b3480 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f8666600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866015bb90 con 0x7f8660071bc0 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86600b3480 0x7f866015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86600b3480 0x7f866015e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51832/0 (socket says 172.21.15.87:51832) 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865f400640 1 -- 172.21.15.87:0/1010358353 learned_addr learned my addr 172.21.15.87:0/1010358353 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865fe00640 1 --2- 172.21.15.87:0/1010358353 >> v2:172.21.15.169:3300/0 conn(0x7f8660071bc0 0x7f8660072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865f400640 1 -- 172.21.15.87:0/1010358353 >> v2:172.21.15.169:3300/0 conn(0x7f8660071bc0 msgr2=0x7f8660072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865f400640 1 --2- 172.21.15.87:0/1010358353 >> v2:172.21.15.169:3300/0 conn(0x7f8660071bc0 0x7f8660072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865f400640 1 -- 172.21.15.87:0/1010358353 >> v1:172.21.15.169:6789/0 conn(0x7f86600cfb50 legacy=0x7f8660070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.880+0000 7f865f400640 1 -- 172.21.15.87:0/1010358353 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866015e710 con 0x7f86600b3480 2024-04-25T10:14:48.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f865f400640 1 --2- 172.21.15.87:0/1010358353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86600b3480 0x7f866015e130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86601627c0 tx=0x7f864801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39858f3df925e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f865e000640 1 -- 172.21.15.87:0/1010358353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480b0070 con 0x7f86600b3480 2024-04-25T10:14:48.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f865e000640 1 -- 172.21.15.87:0/1010358353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86480ab600 con 0x7f86600b3480 2024-04-25T10:14:48.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f865e000640 1 -- 172.21.15.87:0/1010358353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480ab9d0 con 0x7f86600b3480 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- 172.21.15.87:0/1010358353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86600b3480 msgr2=0x7f866015e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 --2- 172.21.15.87:0/1010358353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86600b3480 0x7f866015e130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86601627c0 tx=0x7f864801c520 comp rx=0 tx=0).stop 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- 172.21.15.87:0/1010358353 shutdown_connections 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 --2- 172.21.15.87:0/1010358353 >> v2:172.21.15.169:3300/0 conn(0x7f8660071bc0 0x7f8660072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 --2- 172.21.15.87:0/1010358353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86600b3480 0x7f866015e130 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- 172.21.15.87:0/1010358353 >> 172.21.15.87:0/1010358353 conn(0x7f86600669a0 msgr2=0x7f8660066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- 172.21.15.87:0/1010358353 shutdown_connections 2024-04-25T10:14:48.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- 172.21.15.87:0/1010358353 wait complete. 2024-04-25T10:14:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 Processor -- start 2024-04-25T10:14:48.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- start start 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8660071bc0 0x7f86601491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86600b3480 0x7f866014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86600cfb50 0x7f866014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600ae530 con 0x7f8660071bc0 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600cff60 con 0x7f86600b3480 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.884+0000 7f8666600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86600ae6b0 con 0x7f86600cfb50 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8660071bc0 0x7f86601491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8660071bc0 0x7f86601491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51844/0 (socket says 172.21.15.87:51844) 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 -- 172.21.15.87:0/681560177 learned_addr learned my addr 172.21.15.87:0/681560177 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865fe00640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86600cfb50 0x7f866014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865ea00640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86600b3480 0x7f866014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 -- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86600cfb50 msgr2=0x7f866014df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86600cfb50 0x7f866014df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 -- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86600b3480 msgr2=0x7f866014da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86600b3480 0x7f866014da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 -- 172.21.15.87:0/681560177 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866014e480 con 0x7f8660071bc0 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f865f400640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8660071bc0 0x7f86601491b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f864801abd0 tx=0x7f864801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f8657400640 1 -- 172.21.15.87:0/681560177 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480b0040 con 0x7f8660071bc0 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f8657400640 1 -- 172.21.15.87:0/681560177 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86480ab400 con 0x7f8660071bc0 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f8657400640 1 -- 172.21.15.87:0/681560177 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480b6660 con 0x7f8660071bc0 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866014e730 con 0x7f8660071bc0 2024-04-25T10:14:48.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866014ecf0 con 0x7f8660071bc0 2024-04-25T10:14:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.888+0000 7f8657400640 1 -- 172.21.15.87:0/681560177 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86480be030 con 0x7f8660071bc0 2024-04-25T10:14:48.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8657400640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f863c066f60 0x7f863c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8657400640 1 -- 172.21.15.87:0/681560177 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f864811e9e0 con 0x7f8660071bc0 2024-04-25T10:14:48.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f863c066f60 0x7f863c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 Processor -- start 2024-04-25T10:14:48.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 -- start start 2024-04-25T10:14:48.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f863c066f60 0x7f863c069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8660067090 tx=0x7f8650009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f7984e4360 0x55f79849e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f79849ed90 0x55f79849f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7983811f0 con 0x55f7984e4360 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7982e7e90 con 0x55f7984e3a40 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f79826ff50 con 0x55f79849ed90 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f7984e4360 0x55f79849e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f7984e4360 0x55f79849e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51850/0 (socket says 172.21.15.87:51850) 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 -- 172.21.15.87:0/641786601 learned_addr learned my addr 172.21.15.87:0/641786601 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865f400640 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865fe00640 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f79849ed90 0x55f79849f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 -- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f79849ed90 msgr2=0x55f79849f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f79849ed90 0x55f79849f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 -- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 msgr2=0x55f7984e3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 -- 172.21.15.87:0/641786601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7982bf8b0 con 0x55f7984e4360 2024-04-25T10:14:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f865ea00640 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f7984e4360 0x55f79849e850 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f86600ada10 tx=0x7f8650011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbb83310057578c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f864fe00640 1 -- 172.21.15.87:0/641786601 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650021450 con 0x55f7984e4360 2024-04-25T10:14:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f864fe00640 1 -- 172.21.15.87:0/641786601 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86500215f0 con 0x55f7984e4360 2024-04-25T10:14:48.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f864fe00640 1 -- 172.21.15.87:0/641786601 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86500219a0 con 0x55f7984e4360 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 -- 172.21.15.87:0/641786601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f7984e4360 msgr2=0x55f79849e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f7984e4360 0x55f79849e850 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f86600ada10 tx=0x7f8650011000 comp rx=0 tx=0).stop 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 -- 172.21.15.87:0/641786601 shutdown_connections 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f79849ed90 0x55f79849f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f7984e4360 0x55f79849e850 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 --2- 172.21.15.87:0/641786601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.892+0000 7f8669ade000 1 -- 172.21.15.87:0/641786601 >> 172.21.15.87:0/641786601 conn(0x55f79849ad40 msgr2=0x55f79849d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- 172.21.15.87:0/641786601 shutdown_connections 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- 172.21.15.87:0/641786601 wait complete. 2024-04-25T10:14:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- start start 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 Processor -- start 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7982a2540 con 0x55f79849ed90 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7982e3f30 con 0x55f7984e3a40 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7984855b0 con 0x55f7984e4360 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 10 client.0 _ll_register_callbacks cb 0x55f79839c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984eb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51858/0 (socket says 172.21.15.87:51858) 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 -- 172.21.15.87:0/252118211 learned_addr learned my addr 172.21.15.87:0/252118211 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865ea00640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865fe00640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 -- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 msgr2=0x55f7984a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 -- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 msgr2=0x55f7984eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 -- 172.21.15.87:0/252118211 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7982701a0 con 0x55f79849ed90 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865fe00640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865f400640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984eb1c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8648097df0 tx=0x7f86480ef9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d1011c2ffdf5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f864cc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f864cc00640 1 -- 172.21.15.87:0/252118211 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864811b060 con 0x55f79849ed90 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f864cc00640 1 -- 172.21.15.87:0/252118211 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f86480ed5c0 con 0x55f79849ed90 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86500215f0 con 0x55f79849ed90 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f864cc00640 1 -- 172.21.15.87:0/252118211 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86480f79d0 con 0x55f79849ed90 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 10 client.5439 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55f79836a840 con 0x55f79849ed90 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f864cc00640 1 -- 172.21.15.87:0/252118211 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86500215f0 con 0x55f79849ed90 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f864cc00640 1 -- 172.21.15.87:0/252118211 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86480b3030 con 0x55f79849ed90 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 10 client.5439 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 10 client.5439 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 20 client.5439 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984a5bd0 0x55f7984a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 4 client.5439 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f798283be0 con 0x55f7984a5bd0 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.896+0000 7f865ea00640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984a5bd0 0x55f7984a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f865ea00640 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984a5bd0 0x55f7984a5fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0df3797d9fb9c8e server_cookie=a5aecbc156cd5fed in_seq=0 out_seq=0 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f864cc00640 10 client.5439 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f864cc00640 1 -- 172.21.15.87:0/252118211 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f798283be0 con 0x55f7984a5bd0 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f864cc00640 10 client.5439 handle_command_reply: tid=0 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 client.5439 shutdown 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 msgr2=0x55f7984eb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984eb1c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8648097df0 tx=0x7f86480ef9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 shutdown_connections 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984a5bd0 0x55f7984a5fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 --2- 172.21.15.87:0/252118211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984eb1c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 >> 172.21.15.87:0/252118211 conn(0x55f79849ad40 msgr2=0x55f79849d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f864cc00640 0 client.5439 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 shutdown_connections 2024-04-25T10:14:48.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 1 -- 172.21.15.87:0/252118211 wait complete. 2024-04-25T10:14:48.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.900+0000 7f8669ade000 20 client.5439 trim_cache size 0 max 16384 2024-04-25T10:14:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.912+0000 7f8669ade000 1 Processor -- start 2024-04-25T10:14:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.912+0000 7f8669ade000 1 -- start start 2024-04-25T10:14:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f79849d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e3a40 0x55f79849d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e4360 0x55f79849de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8660202c50 con 0x55f79849ed90 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8660201e50 con 0x55f7984e4360 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8660201250 con 0x55f7984e3a40 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f79849d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f79849d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51870/0 (socket says 172.21.15.87:51870) 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 -- 172.21.15.87:0/3312879488 learned_addr learned my addr 172.21.15.87:0/3312879488 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865fe00640 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e4360 0x55f79849de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865ea00640 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e3a40 0x55f79849d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 -- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e3a40 msgr2=0x55f79849d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e3a40 0x55f79849d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 -- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e4360 msgr2=0x55f79849de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e4360 0x55f79849de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 -- 172.21.15.87:0/3312879488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f798341a10 con 0x55f79849ed90 2024-04-25T10:14:48.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f79849d130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f864801ca00 tx=0x7f86480f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32f51cd028d57d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f864f400640 1 -- 172.21.15.87:0/3312879488 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480eee00 con 0x55f79849ed90 2024-04-25T10:14:48.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f864f400640 1 -- 172.21.15.87:0/3312879488 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864810d360 con 0x55f79849ed90 2024-04-25T10:14:48.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f864f400640 1 -- 172.21.15.87:0/3312879488 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480f7540 con 0x55f79849ed90 2024-04-25T10:14:48.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- 172.21.15.87:0/3312879488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 msgr2=0x55f79849d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f79849d130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f864801ca00 tx=0x7f86480f3000 comp rx=0 tx=0).stop 2024-04-25T10:14:48.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- 172.21.15.87:0/3312879488 shutdown_connections 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e4360 0x55f79849de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e3a40 0x55f79849d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- 172.21.15.87:0/3312879488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f79849d130 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- 172.21.15.87:0/3312879488 >> 172.21.15.87:0/3312879488 conn(0x55f79849ad40 msgr2=0x55f7984f18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- 172.21.15.87:0/3312879488 shutdown_connections 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- 172.21.15.87:0/3312879488 wait complete. 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- start start 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 Processor -- start 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984e9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7982ec4c0 con 0x55f79849ed90 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8660201850 con 0x55f7984e3a40 2024-04-25T10:14:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7982e3f30 con 0x55f7984e4360 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f8669ade000 10 client.0 _ll_register_callbacks cb 0x55f7982aaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984e8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51886/0 (socket says 172.21.15.87:51886) 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.916+0000 7f865f400640 1 -- 172.21.15.87:0/4082657248 learned_addr learned my addr 172.21.15.87:0/4082657248 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865ea00640 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865fe00640 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865f400640 1 -- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 msgr2=0x55f7984e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865f400640 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865f400640 1 -- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 msgr2=0x55f7984e8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865f400640 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865f400640 1 -- 172.21.15.87:0/4082657248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f79827cb00 con 0x55f79849ed90 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865f400640 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984e8890 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f86480abdb0 tx=0x7f8648000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed38d354a6bf27c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:14:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 1 -- 172.21.15.87:0/4082657248 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864811b020 con 0x55f79849ed90 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 1 -- 172.21.15.87:0/4082657248 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f86480ed5c0 con 0x55f79849ed90 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f864810d360 con 0x55f79849ed90 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 1 -- 172.21.15.87:0/4082657248 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86480eee00 con 0x55f79849ed90 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 10 client.5445 fetch_fsmap learned FSMap version 17 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55f79828ef80 con 0x55f79849ed90 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 1 -- 172.21.15.87:0/4082657248 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864810d360 con 0x55f79849ed90 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 1 -- 172.21.15.87:0/4082657248 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8648116390 con 0x55f79849ed90 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 10 client.5445 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 10 client.5445 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 20 client.5445 populate_metadata read hostname 'smithi087' 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984e9cd0 0x55f79854bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 4 client.5445 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 --> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55f7982f53d0 con 0x55f7984e9cd0 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865ea00640 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984e9cd0 0x55f79854bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f865ea00640 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984e9cd0 0x55f79854bce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482096cf0698d56b server_cookie=6916ee719ca6b559 in_seq=0 out_seq=0 2024-04-25T10:14:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 10 client.5445 ms_handle_connect on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 1 -- 172.21.15.87:0/4082657248 <== mds.0 v2:172.21.15.87:6834/4156145333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17944 (crc 0 0 0) 0x55f7982f53d0 con 0x55f7984e9cd0 2024-04-25T10:14:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 10 client.5445 handle_command_reply: tid=0 2024-04-25T10:14:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 client.5445 shutdown 2024-04-25T10:14:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 msgr2=0x55f7984e8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984e8890 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f86480abdb0 tx=0x7f8648000bd0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 shutdown_connections 2024-04-25T10:14:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f7984e4360 0x55f7984e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:6834/4156145333,v1:172.21.15.87:6835/4156145333] conn(0x55f7984e9cd0 0x55f79854bce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f7984e3a40 0x55f7984e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 --2- 172.21.15.87:0/4082657248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f79849ed90 0x55f7984e8890 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f864cc00640 0 client.5445 ms_handle_reset on v2:172.21.15.87:6834/4156145333 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 >> 172.21.15.87:0/4082657248 conn(0x55f79849ad40 msgr2=0x55f7984e2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 shutdown_connections 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 1 -- 172.21.15.87:0/4082657248 wait complete. 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8669ade000 20 client.5445 trim_cache size 0 max 16384 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:14:48.931 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 2268, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2260, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 668528, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 12457730, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 16, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 6, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.238079070, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.104420060, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.061974421, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.054141895, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2260, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.355445164, 2024-04-25T10:14:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000157276 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 10, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000001276, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000127 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 118144, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 24832 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 30796, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 51763, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 10491893, 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 45284538, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 8, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 3.688608940, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 2.710741448, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 1.231532547, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.059251710, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 51762, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.591330060, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000030743 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 47560, 2024-04-25T10:14:48.934 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.009306503, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000195 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 119344 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 5955, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 5957, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1898074, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 64638620, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.626276511, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.298322521, 2024-04-25T10:14:48.935 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.129511329, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.158343087, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 5956, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.017316938, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000170805 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:14:48.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 1, 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-MDSRank": { 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 5733, 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 13.670059690, 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002384451 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-PurgeQueue": { 2024-04-25T10:14:48.937 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4118, 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.970071106, 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000235568 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": { 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "request": 26231, 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "reply": 26231, 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "reply_latency": { 2024-04-25T10:14:48.938 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 26231, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 247.672319030, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.009441970 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_reply": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "forward": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_complete": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_keys": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_commit": 12, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_split": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_merge": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes": 17742, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_top": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_bottom": 17742, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pin_tail": 0, 2024-04-25T10:14:48.939 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pinned": 17742, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_expired": 0, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_with_caps": 2, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": 2, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "subtrees": 2, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse": 49018, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_hit": 48534, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_forward": 0, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_discover": 0, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_remote_ino": 0, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_lock": 142, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "load_cent": 0, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "q": 0, 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi087.stdout: "exported": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "exported_inodes": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "imported": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "imported_inodes": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_dir_fetch": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_peer_discover": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rfiles": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rbytes": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rsnaps": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_set_tag": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:14:48.941 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_inodes": 13, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_file_inodes": 2000, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_revoke": 1575, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_grant": 4040, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps": 4576, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_cap_release": 17, 2024-04-25T10:14:48.942 INFO:teuthology.orchestra.run.smithi087.stdout: "process_request_cap_release": 14081 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_cache": { 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update_receipt": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_try_discover": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_send_discover": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_handle_discover": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_delayed": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_created": 4119, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_enqueued": 3867, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_reintegrated": 2214, 2024-04-25T10:14:48.943 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_migrated": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_processing": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_started": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_completed": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_exportdir": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_flush": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragstats": 0, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_inodestats": 0 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_log": { 2024-04-25T10:14:48.944 INFO:teuthology.orchestra.run.smithi087.stdout: "evlrg": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "evadd": 32131, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "evex": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "evtrm": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "ev": 32131, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "evexg": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "evexd": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "segadd": 48, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "segex": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "segtrm": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "seg": 48, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "segmjr": 4, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "segexg": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "segexd": 0, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "expos": 4194304, 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi087.stdout: "wrpos": 99496171, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "rdpos": 0, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "jlat": { 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4088, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 67.638557187, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.016545635 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "replayed": 0 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_mem": { 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 14, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "ino+": 2016, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "ino-": 2002, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "dir": 13, 2024-04-25T10:14:48.946 INFO:teuthology.orchestra.run.smithi087.stdout: "dir+": 13, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "dir-": 0, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "dn": 17742, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "dn+": 19743, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "dn-": 2001, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "cap": 2, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "cap+": 2094, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "cap-": 2092, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "rss": 96632, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "heap": 215340 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_server": { 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_client_request": 29791, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_server_request": 0, 2024-04-25T10:14:48.947 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_request": 26231, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_session": 5, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_peer_request": 0, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "req_create_latency": { 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2000, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.264677861, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000632338 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getattr_latency": { 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 23, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.003196112, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000138961 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getfilelock_latency": { 2024-04-25T10:14:48.948 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "req_link_latency": { 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6000, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6.675992649, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001112665 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookup_latency": { 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 7981, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6.703181472, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000839892 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookuphash_latency": { 2024-04-25T10:14:48.949 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupino_latency": { 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupname_latency": { 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.950 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupparent_latency": { 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupsnap_latency": { 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000062806, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000062806 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lssnap_latency": { 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mkdir_latency": { 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.002548807, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002548807 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mknod_latency": { 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mksnap_latency": { 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.007231519, 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.007231519 2024-04-25T10:14:48.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "req_open_latency": { 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "req_readdir_latency": { 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.162287251, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.020285906 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rename_latency": { 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2214, 2024-04-25T10:14:48.953 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 216.190019708, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.097646802 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "req_renamesnap_latency": { 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "req_snapdiff_latency": { 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmdir_latency": { 2024-04-25T10:14:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmsnap_latency": { 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.046056440, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.046056440 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmxattr_latency": { 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setattr_latency": { 2024-04-25T10:14:48.955 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000277061, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000277061 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setdirlayout_latency": { 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setfilelock_latency": { 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setlayout_latency": { 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setxattr_latency": { 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "req_symlink_latency": { 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi087.stdout: "req_unlink_latency": { 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8000, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 16.616787344, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002077098 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getvxattr_latency": { 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_sessions": { 2024-04-25T10:14:48.958 INFO:teuthology.orchestra.run.smithi087.stdout: "session_count": 1, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "session_add": 1, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "session_remove": 0, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_open": 1, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_stale": 0, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "total_load": 18907, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "average_load": 18907, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "avg_session_uptime": 75, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "mdthresh_evicted": 0 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:14:48.959 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-04-25T10:14:48.960 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:14:48.961 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 250610, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 86, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 5416, 2024-04-25T10:14:48.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 77, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 13362540, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 660214, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "objecter": { 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "op_active": 0, 2024-04-25T10:14:48.963 INFO:teuthology.orchestra.run.smithi087.stdout: "op_laggy": 0, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send": 10183, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send_bytes": 95940364, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "op_resend": 31, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "op_reply": 10183, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "op_latency": { 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 10183, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 45.798515517, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.004497546 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "op_inflight": 0, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "oplen_avg": { 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 10183, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 10277 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.964 INFO:teuthology.orchestra.run.smithi087.stdout: "op": 10183, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "op_r": 2055, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "op_w": 8128, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "op_rmw": 0, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "op_pg": 0, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_stat": 0, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_create": 11, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_read": 40, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_write": 4126, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writefull": 26, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writesame": 0, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_append": 0, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_zero": 1867, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_truncate": 0, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_delete": 2034, 2024-04-25T10:14:48.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_mapext": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_sparse_read": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_clonerange": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_getxattr": 2015, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_setxattr": 22, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_rmxattr": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_call": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_watch": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_notify": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:14:48.966 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_other": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_active": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_send": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_resend": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_ping": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_active": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_send": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_resend": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_active": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_send": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_resend": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_active": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_send": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_resend": 0, 2024-04-25T10:14:48.967 INFO:teuthology.orchestra.run.smithi087.stdout: "command_active": 0, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "command_send": 0, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "command_resend": 0, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "map_epoch": 35, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "map_full": 0, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "map_inc": 3, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_sessions": 8, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_open": 8, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_close": 0, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_laggy": 0, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_wr": 105, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_rd": 0, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_del": 31 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.968 INFO:teuthology.orchestra.run.smithi087.stdout: "oft": { 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_objs": 1, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_updates": 0, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_removes": 0 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "osd-slow-ops": { 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_ops_count": 0 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "purge_queue": { 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops": 0, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing": 0, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_high_water": 32, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed_ops": 3867, 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed": 3867, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_item_in_journal": 0 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 39011, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 10001102, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 39011, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 39011, 2024-04-25T10:14:48.970 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 10001102, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_bytes": { 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:48.971 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 10183, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 96345867, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 10183, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 96345867, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_ops": { 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 1024, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:48.972 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 10183, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 10183, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 10183, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 10183, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.973 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 3867, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 405503, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 3867, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 38, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 405503, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.974 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle-0x56125a8cca00": { 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 32131, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 95301867, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 32131, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:14:48.975 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 4088, 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 95301867, 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:14:48.976 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:14:48.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f863c066f60 msgr2=0x7f863c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f863c066f60 0x7f863c069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8660067090 tx=0x7f8650009040 comp rx=0 tx=0).stop 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8660071bc0 msgr2=0x7f86601491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8660071bc0 0x7f86601491b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f864801abd0 tx=0x7f864801ad60 comp rx=0 tx=0).stop 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 shutdown_connections 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f863c066f60 0x7f863c069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86600cfb50 0x7f866014df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86600b3480 0x7f866014da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 --2- 172.21.15.87:0/681560177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8660071bc0 0x7f86601491b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 >> 172.21.15.87:0/681560177 conn(0x7f86600669a0 msgr2=0x7f866015bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 shutdown_connections 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:48.920+0000 7f8666600640 1 -- 172.21.15.87:0/681560177 wait complete. 2024-04-25T10:14:48.979 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:48.979 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:49.086 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:49.136 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:49.136 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:14:49.136 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:49.146 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:14:49.146 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:49.195 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:49.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.368+0000 7fda53a00640 1 Processor -- start 2024-04-25T10:14:49.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.368+0000 7fda53a00640 1 -- start start 2024-04-25T10:14:49.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda53a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c15bc40 0x7fda4c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda53a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda4c0cf930 0x7fda4c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda53a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0b2770 con 0x7fda4c0cda10 2024-04-25T10:14:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda53a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c06c670 con 0x7fda4c0cf930 2024-04-25T10:14:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda53a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c06c950 con 0x7fda4c15bc40 2024-04-25T10:14:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda51000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c15bc40 0x7fda4c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda4be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fda4c0cda10 0x7fda4c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:60826/0 (socket says 172.21.15.87:60826) 2024-04-25T10:14:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda4be00640 1 -- 172.21.15.87:0/1481490866 learned_addr learned my addr 172.21.15.87:0/1481490866 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda51a00640 1 --2- 172.21.15.87:0/1481490866 >> v2:172.21.15.169:3300/0 conn(0x7fda4c0cf930 0x7fda4c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda4b400640 1 -- 172.21.15.87:0/1481490866 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282488481 0 0) 0x7fda4c0b2770 con 0x7fda4c0cda10 2024-04-25T10:14:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda4b400640 1 -- 172.21.15.87:0/1481490866 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda30003530 con 0x7fda4c0cda10 2024-04-25T10:14:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda51a00640 1 -- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c15bc40 msgr2=0x7fda4c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda51a00640 1 --2- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c15bc40 0x7fda4c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda51a00640 1 -- 172.21.15.87:0/1481490866 >> v1:172.21.15.169:6789/0 conn(0x7fda4c0cda10 legacy=0x7fda4c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda51a00640 1 -- 172.21.15.87:0/1481490866 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c06cc30 con 0x7fda4c0cf930 2024-04-25T10:14:49.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.372+0000 7fda51000640 1 --2- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c15bc40 0x7fda4c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda51a00640 1 --2- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c0cf930 0x7fda4c0cfde0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda440097c0 tx=0x7fda4401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=320c6716d0574849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda4b400640 1 -- 172.21.15.87:0/1481490866 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440b0070 con 0x7fda4c0cf930 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda4b400640 1 -- 172.21.15.87:0/1481490866 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda440ab600 con 0x7fda4c0cf930 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda4b400640 1 -- 172.21.15.87:0/1481490866 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440ab9d0 con 0x7fda4c0cf930 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 -- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c0cf930 msgr2=0x7fda4c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 --2- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c0cf930 0x7fda4c0cfde0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda440097c0 tx=0x7fda4401c410 comp rx=0 tx=0).stop 2024-04-25T10:14:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 -- 172.21.15.87:0/1481490866 shutdown_connections 2024-04-25T10:14:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 --2- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c0cf930 0x7fda4c0cfde0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 --2- 172.21.15.87:0/1481490866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c15bc40 0x7fda4c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 -- 172.21.15.87:0/1481490866 >> 172.21.15.87:0/1481490866 conn(0x7fda4c066b70 msgr2=0x7fda4c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 -- 172.21.15.87:0/1481490866 shutdown_connections 2024-04-25T10:14:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.376+0000 7fda53a00640 1 -- 172.21.15.87:0/1481490866 wait complete. 2024-04-25T10:14:49.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 Processor -- start 2024-04-25T10:14:49.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 -- start start 2024-04-25T10:14:49.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c0cda10 0x7fda4c149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda4c0cf930 0x7fda4c071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c15bc40 0x7fda4c14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b3850 con 0x7fda4c0cf930 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b36d0 con 0x7fda4c15bc40 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b39d0 con 0x7fda4c0cda10 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda4c0cf930 0x7fda4c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda4c0cf930 0x7fda4c071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51898/0 (socket says 172.21.15.87:51898) 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 -- 172.21.15.87:0/2015786243 learned_addr learned my addr 172.21.15.87:0/2015786243 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda51000640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c0cda10 0x7fda4c149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda51a00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c15bc40 0x7fda4c14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 -- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c0cda10 msgr2=0x7fda4c149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c0cda10 0x7fda4c149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 -- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c15bc40 msgr2=0x7fda4c14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c15bc40 0x7fda4c14ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 -- 172.21.15.87:0/2015786243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c14f5b0 con 0x7fda4c0cf930 2024-04-25T10:14:49.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda4be00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda4c0cf930 0x7fda4c071470 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c3f0 tx=0x7fda3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda48c00640 1 -- 172.21.15.87:0/2015786243 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38017070 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda48c00640 1 -- 172.21.15.87:0/2015786243 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda38005cf0 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda48c00640 1 -- 172.21.15.87:0/2015786243 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda380072f0 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c0706b0 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c070c70 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.380+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c04ef30 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.384+0000 7fda48c00640 1 -- 172.21.15.87:0/2015786243 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda3801a070 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.384+0000 7fda48c00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda2c066f90 0x7fda2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.384+0000 7fda48c00640 1 -- 172.21.15.87:0/2015786243 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fda38084500 con 0x7fda4c0cf930 2024-04-25T10:14:49.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.384+0000 7fda51000640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda2c066f90 0x7fda2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.384+0000 7fda51000640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda2c066f90 0x7fda2c069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0680b0 tx=0x7fda3c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.388+0000 7fda48c00640 1 -- 172.21.15.87:0/2015786243 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda38055450 con 0x7fda4c0cf930 2024-04-25T10:14:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.540+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda4c0af1c0 con 0x7fda4c0cf930 2024-04-25T10:14:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda48c00640 1 -- 172.21.15.87:0/2015786243 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16722 (secure 0 0 0) 0x7fda38054d50 con 0x7fda4c0cf930 2024-04-25T10:14:49.548 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:49.548 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":35,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:14:27.878686+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:14.236622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:15.422837+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda2c066f90 msgr2=0x7fda2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda2c066f90 0x7fda2c069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0680b0 tx=0x7fda3c00b040 comp rx=0 tx=0).stop 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda4c0cf930 msgr2=0x7fda4c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda4c0cf930 0x7fda4c071470 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c3f0 tx=0x7fda3800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 shutdown_connections 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda2c066f90 0x7fda2c069450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4c15bc40 0x7fda4c14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda4c0cf930 0x7fda4c071470 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 --2- 172.21.15.87:0/2015786243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4c0cda10 0x7fda4c149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 >> 172.21.15.87:0/2015786243 conn(0x7fda4c066b70 msgr2=0x7fda4c158630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 shutdown_connections 2024-04-25T10:14:49.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.544+0000 7fda53a00640 1 -- 172.21.15.87:0/2015786243 wait complete. 2024-04-25T10:14:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:14.236622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:15.422837+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:49.570 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.700+0000 7f9df4c00640 1 Processor -- start 2024-04-25T10:14:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.700+0000 7f9df4c00640 1 -- start start 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9df4c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9df00aef80 0x7f9df015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9df4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 0x7f9df015b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9df4c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df015b080 con 0x7f9df015bac0 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9df4c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df015adb0 con 0x7f9df00aef80 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9df4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df015eb60 con 0x7f9df0070350 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9df015bac0 0x7f9df00b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38252/0 (socket says 172.21.15.87:38252) 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 -- 172.21.15.87:0/3240517040 learned_addr learned my addr 172.21.15.87:0/3240517040 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dede00640 1 --2- 172.21.15.87:0/3240517040 >> v2:172.21.15.169:3300/0 conn(0x7f9df00aef80 0x7f9df015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9de5400640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 0x7f9df015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094349026 0 0) 0x7f9df015b080 con 0x7f9df015bac0 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd4005740 con 0x7f9df015bac0 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2787691987 0 0) 0x7f9dd4005740 con 0x7f9df015bac0 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 msgr2=0x7f9df015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 0x7f9df015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 >> v2:172.21.15.169:3300/0 conn(0x7f9df00aef80 msgr2=0x7f9df015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 --2- 172.21.15.87:0/3240517040 >> v2:172.21.15.169:3300/0 conn(0x7f9df00aef80 0x7f9df015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df015ee40 con 0x7f9df015bac0 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f9de8001680 con 0x7f9df015bac0 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 >> v1:172.21.15.87:6789/0 conn(0x7f9df015bac0 legacy=0x7f9df00b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 0x7f9dd4005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dd4006150 0x7f9dd4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9dd40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df015adb0 con 0x7f9dd4006f60 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df015eb60 con 0x7f9dd4006150 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400dbe0 con 0x7f9df0070350 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9dd40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9de5400640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dd4006150 0x7f9dd4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dede00640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 0x7f9dd4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 -- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 msgr2=0x7f9dd4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 0x7f9dd4005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 -- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dd4006150 msgr2=0x7f9dd4006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dd4006150 0x7f9dd4006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dee800640 1 -- 172.21.15.87:0/3240517040 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4005740 con 0x7f9dd4006f60 2024-04-25T10:14:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9dede00640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df0070350 0x7f9dd4005bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.704+0000 7f9de5400640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dd4006150 0x7f9dd4006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:49.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9dee800640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9dd40073e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c2c0 tx=0x7f9de801cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7717f2e5ae6fc6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8007d60 con 0x7f9dd4006f60 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de80a9020 con 0x7f9dd4006f60 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9ded400640 1 -- 172.21.15.87:0/3240517040 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de80a5bb0 con 0x7f9dd4006f60 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9df4c00640 1 -- 172.21.15.87:0/3240517040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 msgr2=0x7f9dd40073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9df4c00640 1 --2- 172.21.15.87:0/3240517040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9dd40073e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c2c0 tx=0x7f9de801cce0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9dee800640 1 -- 172.21.15.87:0/3240517040 reap_dead start 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9df4c00640 1 -- 172.21.15.87:0/3240517040 shutdown_connections 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9df4c00640 1 -- 172.21.15.87:0/3240517040 >> 172.21.15.87:0/3240517040 conn(0x7f9df00669a0 msgr2=0x7f9df0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9df4c00640 1 -- 172.21.15.87:0/3240517040 shutdown_connections 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.708+0000 7f9df4c00640 1 -- 172.21.15.87:0/3240517040 wait complete. 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 Processor -- start 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 -- start start 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9df014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df00ad840 0x7f9df00adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df0150d40 0x7f9df01511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b38d0 con 0x7f9dd4006f60 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0071a00 con 0x7f9df0150d40 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b3a50 con 0x7f9df00ad840 2024-04-25T10:14:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9df014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9df014dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51914/0 (socket says 172.21.15.87:51914) 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 -- 172.21.15.87:0/1762825123 learned_addr learned my addr 172.21.15.87:0/1762825123 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 -- 172.21.15.87:0/1762825123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df00ad840 msgr2=0x7f9df00adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df00ad840 0x7f9df00adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 -- 172.21.15.87:0/1762825123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df0150d40 msgr2=0x7f9df01511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df0150d40 0x7f9df01511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 -- 172.21.15.87:0/1762825123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df014e500 con 0x7f9dd4006f60 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9dede00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9df014dfc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009fc0 tx=0x7f9de0010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9de6a00640 1 -- 172.21.15.87:0/1762825123 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de001a030 con 0x7f9dd4006f60 2024-04-25T10:14:49.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9de6a00640 1 -- 172.21.15.87:0/1762825123 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00039a0 con 0x7f9dd4006f60 2024-04-25T10:14:49.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9de6a00640 1 -- 172.21.15.87:0/1762825123 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0003d70 con 0x7f9dd4006f60 2024-04-25T10:14:49.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df00af3c0 con 0x7f9dd4006f60 2024-04-25T10:14:49.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df00af980 con 0x7f9dd4006f60 2024-04-25T10:14:49.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.712+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df004ef30 con 0x7f9dd4006f60 2024-04-25T10:14:49.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.716+0000 7f9de6a00640 1 -- 172.21.15.87:0/1762825123 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9de001d070 con 0x7f9dd4006f60 2024-04-25T10:14:49.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.716+0000 7f9de6a00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dcc066f90 0x7f9dcc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.716+0000 7f9de6a00640 1 -- 172.21.15.87:0/1762825123 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9de0085ef0 con 0x7f9dd4006f60 2024-04-25T10:14:49.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.716+0000 7f9de5400640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dcc066f90 0x7f9dcc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.716+0000 7f9de5400640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dcc066f90 0x7f9dcc069450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009850 tx=0x7f9dd8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.720+0000 7f9de6a00640 1 -- 172.21.15.87:0/1762825123 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9de0056ee0 con 0x7f9dd4006f60 2024-04-25T10:14:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9df00b2fa0 con 0x7f9dd4006f60 2024-04-25T10:14:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9de6a00640 1 -- 172.21.15.87:0/1762825123 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9de00567e0 con 0x7f9dd4006f60 2024-04-25T10:14:49.881 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:14:49.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dcc066f90 msgr2=0x7f9dcc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dcc066f90 0x7f9dcc069450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009850 tx=0x7f9dd8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:49.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 msgr2=0x7f9df014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9df014dfc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009fc0 tx=0x7f9de0010650 comp rx=0 tx=0).stop 2024-04-25T10:14:49.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 shutdown_connections 2024-04-25T10:14:49.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dcc066f90 0x7f9dcc069450 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df0150d40 0x7f9df01511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df00ad840 0x7f9df00adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 --2- 172.21.15.87:0/1762825123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd4006f60 0x7f9df014dfc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 >> 172.21.15.87:0/1762825123 conn(0x7f9df00669a0 msgr2=0x7f9df00cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 shutdown_connections 2024-04-25T10:14:49.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:49.876+0000 7f9df4c00640 1 -- 172.21.15.87:0/1762825123 wait complete. 2024-04-25T10:14:49.902 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:50.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbc0200640 1 Processor -- start 2024-04-25T10:14:50.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbc0200640 1 -- start start 2024-04-25T10:14:50.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbc0200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbb80cfb30 0x7fdbb80d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbc0200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdbb80ce220 0x7fdbb80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbc0200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb806b470 con 0x7fdbb815dd30 2024-04-25T10:14:50.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbc0200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb815e110 con 0x7fdbb80cfb30 2024-04-25T10:14:50.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbc0200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb80b2940 con 0x7fdbb80ce220 2024-04-25T10:14:50.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbb80cfb30 0x7fdbb80d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbb80cfb30 0x7fdbb80d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51922/0 (socket says 172.21.15.87:51922) 2024-04-25T10:14:50.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbce00640 1 -- 172.21.15.87:0/2856762695 learned_addr learned my addr 172.21.15.87:0/2856762695 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:50.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbe200640 1 --2- 172.21.15.87:0/2856762695 >> v2:172.21.15.169:3300/0 conn(0x7fdbb80ce220 0x7fdbb80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbce00640 1 -- 172.21.15.87:0/2856762695 >> v2:172.21.15.169:3300/0 conn(0x7fdbb80ce220 msgr2=0x7fdbb80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbce00640 1 --2- 172.21.15.87:0/2856762695 >> v2:172.21.15.169:3300/0 conn(0x7fdbb80ce220 0x7fdbb80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbce00640 1 -- 172.21.15.87:0/2856762695 >> v1:172.21.15.169:6789/0 conn(0x7fdbb815dd30 legacy=0x7fdbb806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.036+0000 7fdbbce00640 1 -- 172.21.15.87:0/2856762695 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb806b750 con 0x7fdbb80cfb30 2024-04-25T10:14:50.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbbce00640 1 --2- 172.21.15.87:0/2856762695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80cfb30 0x7fdbb80d18a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80b1180 tx=0x7fdba401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94b7d5d50fc32d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbb7e00640 1 -- 172.21.15.87:0/2856762695 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40ab5f0 con 0x7fdbb80cfb30 2024-04-25T10:14:50.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbb7e00640 1 -- 172.21.15.87:0/2856762695 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba40b0070 con 0x7fdbb80cfb30 2024-04-25T10:14:50.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbb7e00640 1 -- 172.21.15.87:0/2856762695 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40ab9b0 con 0x7fdbb80cfb30 2024-04-25T10:14:50.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 -- 172.21.15.87:0/2856762695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80cfb30 msgr2=0x7fdbb80d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2856762695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80cfb30 0x7fdbb80d18a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80b1180 tx=0x7fdba401c500 comp rx=0 tx=0).stop 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 -- 172.21.15.87:0/2856762695 shutdown_connections 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2856762695 >> v2:172.21.15.169:3300/0 conn(0x7fdbb80ce220 0x7fdbb80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2856762695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80cfb30 0x7fdbb80d18a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 -- 172.21.15.87:0/2856762695 >> 172.21.15.87:0/2856762695 conn(0x7fdbb809a9d0 msgr2=0x7fdbb809adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 -- 172.21.15.87:0/2856762695 shutdown_connections 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 -- 172.21.15.87:0/2856762695 wait complete. 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 Processor -- start 2024-04-25T10:14:50.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.040+0000 7fdbc0200640 1 -- start start 2024-04-25T10:14:50.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80ce220 0x7fdbb80af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb80cfb30 0x7fdbb814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb815dd30 0x7fdbb814bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80b24f0 con 0x7fdbb80ce220 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80b2370 con 0x7fdbb815dd30 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80b2670 con 0x7fdbb80cfb30 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80ce220 0x7fdbb80af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80ce220 0x7fdbb80af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51924/0 (socket says 172.21.15.87:51924) 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 -- 172.21.15.87:0/2842002347 learned_addr learned my addr 172.21.15.87:0/2842002347 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbe200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb815dd30 0x7fdbb814bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbce00640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb80cfb30 0x7fdbb814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 -- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb80cfb30 msgr2=0x7fdbb814b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb80cfb30 0x7fdbb814b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 -- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb815dd30 msgr2=0x7fdbb814bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb815dd30 0x7fdbb814bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 -- 172.21.15.87:0/2842002347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb814b970 con 0x7fdbb80ce220 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbe200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb815dd30 0x7fdbb814bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbbd800640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80ce220 0x7fdbb80af090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c370 tx=0x7fdbac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbb5600640 1 -- 172.21.15.87:0/2842002347 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0052d0 con 0x7fdbb80ce220 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbb5600640 1 -- 172.21.15.87:0/2842002347 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac005470 con 0x7fdbb80ce220 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbb5600640 1 -- 172.21.15.87:0/2842002347 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac007260 con 0x7fdbb80ce220 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb8155690 con 0x7fdbb80ce220 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb815af40 con 0x7fdbb80ce220 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb8155bb0 con 0x7fdbb80ce220 2024-04-25T10:14:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.044+0000 7fdbb5600640 1 -- 172.21.15.87:0/2842002347 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbac007400 con 0x7fdbb80ce220 2024-04-25T10:14:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.048+0000 7fdbb5600640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdb98066f60 0x7fdb98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.048+0000 7fdbb5600640 1 -- 172.21.15.87:0/2842002347 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdbac084530 con 0x7fdbb80ce220 2024-04-25T10:14:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.048+0000 7fdbbce00640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdb98066f60 0x7fdb98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.048+0000 7fdbbce00640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdb98066f60 0x7fdb98069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdba40af000 tx=0x7fdba40a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.052+0000 7fdbb5600640 1 -- 172.21.15.87:0/2842002347 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdbb8155bb0 con 0x7fdbb80ce220 2024-04-25T10:14:50.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdbb804ef30 con 0x7fdbb80ce220 2024-04-25T10:14:50.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbb5600640 1 -- 172.21.15.87:0/2842002347 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdbac054da0 con 0x7fdbb80ce220 2024-04-25T10:14:50.210 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:14:50.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdb98066f60 msgr2=0x7fdb98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdb98066f60 0x7fdb98069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdba40af000 tx=0x7fdba40a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80ce220 msgr2=0x7fdbb80af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80ce220 0x7fdbb80af090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c370 tx=0x7fdbac00c830 comp rx=0 tx=0).stop 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 shutdown_connections 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdb98066f60 0x7fdb98069420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb815dd30 0x7fdbb814bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb80cfb30 0x7fdbb814b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 --2- 172.21.15.87:0/2842002347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbb80ce220 0x7fdbb80af090 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.204+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 >> 172.21.15.87:0/2842002347 conn(0x7fdbb809a9d0 msgr2=0x7fdbb80cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.208+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 shutdown_connections 2024-04-25T10:14:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.208+0000 7fdbc0200640 1 -- 172.21.15.87:0/2842002347 wait complete. 2024-04-25T10:14:50.230 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc69600640 1 Processor -- start 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc69600640 1 -- start start 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc69600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc64070350 0x7fcc6415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc69600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc69600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6415b080 con 0x7fcc640aef80 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc69600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6415adb0 con 0x7fcc64070350 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc69600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6415eb60 con 0x7fcc640b31c0 2024-04-25T10:14:50.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc62a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcc640aef80 0x7fcc6415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:60838/0 (socket says 172.21.15.87:60838) 2024-04-25T10:14:50.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc62a00640 1 -- 172.21.15.87:0/1467431160 learned_addr learned my addr 172.21.15.87:0/1467431160 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc62000640 1 --2- 172.21.15.87:0/1467431160 >> v2:172.21.15.169:3300/0 conn(0x7fcc64070350 0x7fcc6415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc63400640 1 --2- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc61600640 1 -- 172.21.15.87:0/1467431160 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397247314 0 0) 0x7fcc6415b080 con 0x7fcc640aef80 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc61600640 1 -- 172.21.15.87:0/1467431160 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc48003530 con 0x7fcc640aef80 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc62000640 1 -- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 msgr2=0x7fcc640b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc62000640 1 --2- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc62000640 1 -- 172.21.15.87:0/1467431160 >> v1:172.21.15.169:6789/0 conn(0x7fcc640aef80 legacy=0x7fcc6415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.364+0000 7fcc62000640 1 -- 172.21.15.87:0/1467431160 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6415ee40 con 0x7fcc64070350 2024-04-25T10:14:50.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc62000640 1 --2- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 0x7fcc6415e480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc500097c0 tx=0x7fcc5001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41828d54be1061b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc61600640 1 -- 172.21.15.87:0/1467431160 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500b1070 con 0x7fcc64070350 2024-04-25T10:14:50.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc61600640 1 -- 172.21.15.87:0/1467431160 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc500ac410 con 0x7fcc64070350 2024-04-25T10:14:50.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc61600640 1 -- 172.21.15.87:0/1467431160 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500ac7c0 con 0x7fcc64070350 2024-04-25T10:14:50.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 -- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 msgr2=0x7fcc6415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 --2- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 0x7fcc6415e480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc500097c0 tx=0x7fcc5001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 -- 172.21.15.87:0/1467431160 shutdown_connections 2024-04-25T10:14:50.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 --2- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 --2- 172.21.15.87:0/1467431160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 0x7fcc6415e480 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 -- 172.21.15.87:0/1467431160 >> 172.21.15.87:0/1467431160 conn(0x7fcc640669a0 msgr2=0x7fcc64066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 -- 172.21.15.87:0/1467431160 shutdown_connections 2024-04-25T10:14:50.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.368+0000 7fcc69600640 1 -- 172.21.15.87:0/1467431160 wait complete. 2024-04-25T10:14:50.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 Processor -- start 2024-04-25T10:14:50.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 -- start start 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 0x7fcc6414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc640aef80 0x7fcc6406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641679a0 con 0x7fcc640aef80 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64167820 con 0x7fcc64070350 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64167b20 con 0x7fcc640b31c0 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc640aef80 0x7fcc6406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 0x7fcc6414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc640aef80 0x7fcc6406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51932/0 (socket says 172.21.15.87:51932) 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 -- 172.21.15.87:0/2017820121 learned_addr learned my addr 172.21.15.87:0/2017820121 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc63400640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 -- 172.21.15.87:0/2017820121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 msgr2=0x7fcc640adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 -- 172.21.15.87:0/2017820121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 msgr2=0x7fcc6414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 0x7fcc6414f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 -- 172.21.15.87:0/2017820121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6414ff10 con 0x7fcc640aef80 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc62000640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc640aef80 0x7fcc6406d030 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcc500af040 tx=0x7fcc50019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2017820121 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500b9020 con 0x7fcc640aef80 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2017820121 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc500accc0 con 0x7fcc640aef80 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc641501f0 con 0x7fcc640aef80 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2017820121 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500b1040 con 0x7fcc640aef80 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc64150a90 con 0x7fcc640aef80 2024-04-25T10:14:50.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.372+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6404ef30 con 0x7fcc640aef80 2024-04-25T10:14:50.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.376+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2017820121 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc500bc020 con 0x7fcc640aef80 2024-04-25T10:14:50.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.376+0000 7fcc5aa00640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc44066f60 0x7fcc44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.376+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2017820121 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcc5011e830 con 0x7fcc640aef80 2024-04-25T10:14:50.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.376+0000 7fcc62a00640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc44066f60 0x7fcc44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.376+0000 7fcc62a00640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc44066f60 0x7fcc44069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc641fa130 tx=0x7fcc54000bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.380+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2017820121 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc500c3030 con 0x7fcc640aef80 2024-04-25T10:14:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.532+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc640b2fa0 con 0x7fcc640aef80 2024-04-25T10:14:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.532+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2017820121 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc50016090 con 0x7fcc640aef80 2024-04-25T10:14:50.539 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc44066f60 msgr2=0x7fcc44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc44066f60 0x7fcc44069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc641fa130 tx=0x7fcc54000bd0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc640aef80 msgr2=0x7fcc6406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc640aef80 0x7fcc6406d030 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcc500af040 tx=0x7fcc50019240 comp rx=0 tx=0).stop 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 shutdown_connections 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc44066f60 0x7fcc44069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc640b31c0 0x7fcc640adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc640aef80 0x7fcc6406d030 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 --2- 172.21.15.87:0/2017820121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc64070350 0x7fcc6414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 >> 172.21.15.87:0/2017820121 conn(0x7fcc640669a0 msgr2=0x7fcc640b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 shutdown_connections 2024-04-25T10:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.536+0000 7fcc69600640 1 -- 172.21.15.87:0/2017820121 wait complete. 2024-04-25T10:14:50.559 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.688+0000 7faeb9600640 1 Processor -- start 2024-04-25T10:14:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.688+0000 7faeb9600640 1 -- start start 2024-04-25T10:14:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb415bc40 0x7faeb406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb9600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb40b2770 con 0x7faeb40cda10 2024-04-25T10:14:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb9600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb406c670 con 0x7faeb40cf930 2024-04-25T10:14:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb406c950 con 0x7faeb415bc40 2024-04-25T10:14:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb2000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7faeb40cda10 0x7faeb40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38262/0 (socket says 172.21.15.87:38262) 2024-04-25T10:14:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb2000640 1 -- 172.21.15.87:0/2913112608 learned_addr learned my addr 172.21.15.87:0/2913112608 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:50.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb2a00640 1 --2- 172.21.15.87:0/2913112608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb415bc40 0x7faeb406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb1600640 1 -- 172.21.15.87:0/2913112608 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480775265 0 0) 0x7faeb40b2770 con 0x7faeb40cda10 2024-04-25T10:14:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb1600640 1 -- 172.21.15.87:0/2913112608 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae98003530 con 0x7faeb40cda10 2024-04-25T10:14:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb1600640 1 -- 172.21.15.87:0/2913112608 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383337446 0 0) 0x7faeb406c670 con 0x7faeb40cf930 2024-04-25T10:14:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb1600640 1 -- 172.21.15.87:0/2913112608 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb40b2770 con 0x7faeb40cf930 2024-04-25T10:14:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb2a00640 1 -- 172.21.15.87:0/2913112608 >> v1:172.21.15.169:6789/0 conn(0x7faeb40cf930 legacy=0x7faeb40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb2a00640 1 -- 172.21.15.87:0/2913112608 >> v1:172.21.15.87:6789/0 conn(0x7faeb40cda10 legacy=0x7faeb40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.692+0000 7faeb2a00640 1 -- 172.21.15.87:0/2913112608 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb406cc30 con 0x7faeb415bc40 2024-04-25T10:14:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb2a00640 1 --2- 172.21.15.87:0/2913112608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb415bc40 0x7faeb406c090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faea40097f0 tx=0x7faea401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b624140f1d03c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb1600640 1 -- 172.21.15.87:0/2913112608 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40b2070 con 0x7faeb415bc40 2024-04-25T10:14:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb1600640 1 -- 172.21.15.87:0/2913112608 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea40a9070 con 0x7faeb415bc40 2024-04-25T10:14:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb1600640 1 -- 172.21.15.87:0/2913112608 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40ad880 con 0x7faeb415bc40 2024-04-25T10:14:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 -- 172.21.15.87:0/2913112608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb415bc40 msgr2=0x7faeb406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 --2- 172.21.15.87:0/2913112608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb415bc40 0x7faeb406c090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faea40097f0 tx=0x7faea401c450 comp rx=0 tx=0).stop 2024-04-25T10:14:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 -- 172.21.15.87:0/2913112608 shutdown_connections 2024-04-25T10:14:50.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 --2- 172.21.15.87:0/2913112608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb415bc40 0x7faeb406c090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 -- 172.21.15.87:0/2913112608 >> 172.21.15.87:0/2913112608 conn(0x7faeb4066b70 msgr2=0x7faeb4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 -- 172.21.15.87:0/2913112608 shutdown_connections 2024-04-25T10:14:50.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 -- 172.21.15.87:0/2913112608 wait complete. 2024-04-25T10:14:50.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 Processor -- start 2024-04-25T10:14:50.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.696+0000 7faeb9600640 1 -- start start 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb40cda10 0x7faeb4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb40cf930 0x7faeb4153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb415bc40 0x7faeb415e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4169d70 con 0x7faeb415bc40 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4169bf0 con 0x7faeb40cf930 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4169ef0 con 0x7faeb40cda10 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb415bc40 0x7faeb415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb415bc40 0x7faeb415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51948/0 (socket says 172.21.15.87:51948) 2024-04-25T10:14:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 -- 172.21.15.87:0/1007204785 learned_addr learned my addr 172.21.15.87:0/1007204785 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb2a00640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb40cda10 0x7faeb4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb2000640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb40cf930 0x7faeb4153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 -- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb40cda10 msgr2=0x7faeb4156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb40cda10 0x7faeb4156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 -- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb40cf930 msgr2=0x7faeb4153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb40cf930 0x7faeb4153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 -- 172.21.15.87:0/1007204785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb415cc10 con 0x7faeb415bc40 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb2000640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb40cf930 0x7faeb4153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb3400640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb415bc40 0x7faeb415e480 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c790 tx=0x7faeac00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeaaa00640 1 -- 172.21.15.87:0/1007204785 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac003b80 con 0x7faeb415bc40 2024-04-25T10:14:50.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeaaa00640 1 -- 172.21.15.87:0/1007204785 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac003d20 con 0x7faeb415bc40 2024-04-25T10:14:50.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeaaa00640 1 -- 172.21.15.87:0/1007204785 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00aa50 con 0x7faeb415bc40 2024-04-25T10:14:50.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb41542d0 con 0x7faeb415bc40 2024-04-25T10:14:50.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb4154890 con 0x7faeb415bc40 2024-04-25T10:14:50.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb4154ad0 con 0x7faeb415bc40 2024-04-25T10:14:50.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.700+0000 7faeaaa00640 1 -- 172.21.15.87:0/1007204785 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faeac00abf0 con 0x7faeb415bc40 2024-04-25T10:14:50.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.704+0000 7faeaaa00640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae94066f60 0x7fae94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.704+0000 7faeaaa00640 1 -- 172.21.15.87:0/1007204785 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7faeac084fa0 con 0x7faeb415bc40 2024-04-25T10:14:50.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.704+0000 7faeb2a00640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae94066f60 0x7fae94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.704+0000 7faeb2a00640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae94066f60 0x7fae94069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faea40af000 tx=0x7faea40a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.708+0000 7faeaaa00640 1 -- 172.21.15.87:0/1007204785 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faeac056020 con 0x7faeb415bc40 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.872+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faeb404ef30 con 0x7faeb415bc40 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.872+0000 7faeaaa00640 1 -- 172.21.15.87:0/1007204785 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7faeac055920 con 0x7faeb415bc40 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae94066f60 msgr2=0x7fae94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae94066f60 0x7fae94069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faea40af000 tx=0x7faea40a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb415bc40 msgr2=0x7faeb415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb415bc40 0x7faeb415e480 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c790 tx=0x7faeac00cc50 comp rx=0 tx=0).stop 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 shutdown_connections 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae94066f60 0x7fae94069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb415bc40 0x7faeb415e480 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb40cf930 0x7faeb4153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 --2- 172.21.15.87:0/1007204785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeb40cda10 0x7faeb4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 >> 172.21.15.87:0/1007204785 conn(0x7faeb4066b70 msgr2=0x7faeb4159830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 shutdown_connections 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:50.876+0000 7faeb9600640 1 -- 172.21.15.87:0/1007204785 wait complete. 2024-04-25T10:14:50.898 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:50.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:50.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:14:50.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:50.938 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:50.992 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:14:50.992 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:51.001 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:51.002 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:14:51.002 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:51.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:51.048 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:14:51.048 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:51.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:51.056 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:51.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.224+0000 7f0fc4e00640 1 Processor -- start 2024-04-25T10:14:51.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.224+0000 7f0fc4e00640 1 -- start start 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fc4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc015dce0 0x7f0fc00669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fc4e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fc006eec0 0x7f0fc006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fc4e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fc00ad850 0x7f0fc00adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fc4e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc00674d0 con 0x7f0fc00ad850 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fc4e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc015e0c0 con 0x7f0fc006eec0 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fc4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00671f0 con 0x7f0fc015dce0 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbd400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fc006eec0 0x7f0fc006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbe800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fc00ad850 0x7f0fc00adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc015dce0 0x7f0fc00669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbd400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fc006eec0 0x7f0fc006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:53444/0 (socket says 172.21.15.87:53444) 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbe800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fc00ad850 0x7f0fc00adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51962/0 (socket says 172.21.15.87:51962) 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc015dce0 0x7f0fc00669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:43460/0 (socket says 172.21.15.87:43460) 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbd400640 1 -- 172.21.15.87:0/114552207 learned_addr learned my addr 172.21.15.87:0/114552207 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbe800640 1 -- 172.21.15.87:0/114552207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc015dce0 msgr2=0x7f0fc00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbe800640 1 --2- 172.21.15.87:0/114552207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc015dce0 0x7f0fc00669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbe800640 1 -- 172.21.15.87:0/114552207 >> v2:172.21.15.169:3300/0 conn(0x7f0fc006eec0 msgr2=0x7f0fc006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbe800640 1 --2- 172.21.15.87:0/114552207 >> v2:172.21.15.169:3300/0 conn(0x7f0fc006eec0 0x7f0fc006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.228+0000 7f0fbe800640 1 -- 172.21.15.87:0/114552207 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc006d4b0 con 0x7f0fc00ad850 2024-04-25T10:14:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fbe800640 1 --2- 172.21.15.87:0/114552207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc00ad850 0x7f0fc00adca0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00ce3a0 tx=0x7f0fb801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dedaa7304fa787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fbca00640 1 -- 172.21.15.87:0/114552207 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80b2070 con 0x7f0fc00ad850 2024-04-25T10:14:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fbca00640 1 -- 172.21.15.87:0/114552207 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb80a5720 con 0x7f0fc00ad850 2024-04-25T10:14:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fbca00640 1 -- 172.21.15.87:0/114552207 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80a5af0 con 0x7f0fc00ad850 2024-04-25T10:14:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 -- 172.21.15.87:0/114552207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc00ad850 msgr2=0x7f0fc00adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/114552207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc00ad850 0x7f0fc00adca0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00ce3a0 tx=0x7f0fb801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 -- 172.21.15.87:0/114552207 shutdown_connections 2024-04-25T10:14:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/114552207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc00ad850 0x7f0fc00adca0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/114552207 >> v2:172.21.15.169:3300/0 conn(0x7f0fc006eec0 0x7f0fc006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/114552207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc015dce0 0x7f0fc00669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 -- 172.21.15.87:0/114552207 >> 172.21.15.87:0/114552207 conn(0x7f0fc00b2410 msgr2=0x7f0fc00b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 -- 172.21.15.87:0/114552207 shutdown_connections 2024-04-25T10:14:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 -- 172.21.15.87:0/114552207 wait complete. 2024-04-25T10:14:51.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 Processor -- start 2024-04-25T10:14:51.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.232+0000 7f0fc4e00640 1 -- start start 2024-04-25T10:14:51.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc006eec0 0x7f0fc00c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc00ad850 0x7f0fc00c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fc015dce0 0x7f0fc015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc006e650 con 0x7f0fc006eec0 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc006e4d0 con 0x7f0fc015dce0 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00d0320 con 0x7f0fc00ad850 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc006eec0 0x7f0fc00c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc006eec0 0x7f0fc00c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51976/0 (socket says 172.21.15.87:51976) 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 -- 172.21.15.87:0/3698683941 learned_addr learned my addr 172.21.15.87:0/3698683941 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbe800640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fc015dce0 0x7f0fc015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 -- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc00ad850 msgr2=0x7f0fc00c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc00ad850 0x7f0fc00c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 -- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fc015dce0 msgr2=0x7f0fc015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fc015dce0 0x7f0fc015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 -- 172.21.15.87:0/3698683941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00c3b30 con 0x7f0fc006eec0 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbe800640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fc015dce0 0x7f0fc015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fbde00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc006eec0 0x7f0fc00c4920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000e870 tx=0x7f0fb000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fb6000640 1 -- 172.21.15.87:0/3698683941 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0017070 con 0x7f0fc006eec0 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fb6000640 1 -- 172.21.15.87:0/3698683941 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb000c940 con 0x7f0fc006eec0 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fb6000640 1 -- 172.21.15.87:0/3698683941 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000ccf0 con 0x7f0fc006eec0 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc00c8490 con 0x7f0fc006eec0 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc0156ca0 con 0x7f0fc006eec0 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc004ef30 con 0x7f0fc006eec0 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.236+0000 7f0fb6000640 1 -- 172.21.15.87:0/3698683941 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fb001a070 con 0x7f0fc006eec0 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.240+0000 7f0fb6000640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fa0066f60 0x7f0fa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.240+0000 7f0fb6000640 1 -- 172.21.15.87:0/3698683941 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0fb0084ec0 con 0x7f0fc006eec0 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.240+0000 7f0fbd400640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fa0066f60 0x7f0fa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.240+0000 7f0fbd400640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fa0066f60 0x7f0fa0069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0fac006aa0 tx=0x7f0fac006a30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.244+0000 7f0fb6000640 1 -- 172.21.15.87:0/3698683941 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0fb0055f20 con 0x7f0fc006eec0 2024-04-25T10:14:51.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.428+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fc0068180 con 0x7f0fc006eec0 2024-04-25T10:14:51.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fb6000640 1 -- 172.21.15.87:0/3698683941 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0fb0055820 con 0x7f0fc006eec0 2024-04-25T10:14:51.436 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:51.436 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3059476782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3059476782},{"type":"v1","addr":"172.21.15.169:6834","nonce":3059476782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/685777206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":685777206},{"type":"v1","addr":"172.21.15.87:6837","nonce":685777206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2281421407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2281421407},{"type":"v1","addr":"172.21.15.169:6835","nonce":2281421407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:13:16.421834+0000","modified":"2024-04-25T10:13:28.839800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/4156145333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4156145333},{"type":"v1","addr":"172.21.15.87:6835","nonce":4156145333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:14:51.436 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 17 2024-04-25T10:14:51.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fa0066f60 msgr2=0x7f0fa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fa0066f60 0x7f0fa0069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0fac006aa0 tx=0x7f0fac006a30 comp rx=0 tx=0).stop 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc006eec0 msgr2=0x7f0fc00c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc006eec0 0x7f0fc00c4920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000e870 tx=0x7f0fb000ec70 comp rx=0 tx=0).stop 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 shutdown_connections 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fa0066f60 0x7f0fa0069420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fc015dce0 0x7f0fc015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fc00ad850 0x7f0fc00c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 --2- 172.21.15.87:0/3698683941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc006eec0 0x7f0fc00c4920 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 >> 172.21.15.87:0/3698683941 conn(0x7f0fc00b2410 msgr2=0x7f0fc0160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 shutdown_connections 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.432+0000 7f0fc4e00640 1 -- 172.21.15.87:0/3698683941 wait complete. 2024-04-25T10:14:51.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:51.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.584+0000 7f2b43600640 1 Processor -- start 2024-04-25T10:14:51.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.584+0000 7f2b43600640 1 -- start start 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b43600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b3c15bc40 0x7f2b3c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b43600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cf930 0x7f2b3c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b43600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c0b2770 con 0x7f2b3c0cda10 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b43600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3c06c670 con 0x7f2b3c15bc40 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b43600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c06c950 con 0x7f2b3c0cf930 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b40c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b3c15bc40 0x7f2b3c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b40c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b3c15bc40 0x7f2b3c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51984/0 (socket says 172.21.15.87:51984) 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b40c00640 1 -- 172.21.15.87:0/137211601 learned_addr learned my addr 172.21.15.87:0/137211601 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b41600640 1 --2- 172.21.15.87:0/137211601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cf930 0x7f2b3c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b40c00640 1 -- 172.21.15.87:0/137211601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cf930 msgr2=0x7f2b3c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b40c00640 1 --2- 172.21.15.87:0/137211601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cf930 0x7f2b3c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b40c00640 1 -- 172.21.15.87:0/137211601 >> v1:172.21.15.169:6789/0 conn(0x7f2b3c0cda10 legacy=0x7f2b3c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b3be00640 1 -- 172.21.15.87:0/137211601 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442021494 0 0) 0x7f2b3c0b2770 con 0x7f2b3c0cda10 2024-04-25T10:14:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.588+0000 7f2b40c00640 1 -- 172.21.15.87:0/137211601 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c06cc30 con 0x7f2b3c15bc40 2024-04-25T10:14:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b40c00640 1 --2- 172.21.15.87:0/137211601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c15bc40 0x7f2b3c06c090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009870 tx=0x7f2b2c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edafb6ef9f8e993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b3be00640 1 -- 172.21.15.87:0/137211601 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0b0070 con 0x7f2b3c15bc40 2024-04-25T10:14:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b3be00640 1 -- 172.21.15.87:0/137211601 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c0ab600 con 0x7f2b3c15bc40 2024-04-25T10:14:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b3be00640 1 -- 172.21.15.87:0/137211601 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0ab9d0 con 0x7f2b3c15bc40 2024-04-25T10:14:51.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 -- 172.21.15.87:0/137211601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c15bc40 msgr2=0x7f2b3c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 --2- 172.21.15.87:0/137211601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c15bc40 0x7f2b3c06c090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009870 tx=0x7f2b2c01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 -- 172.21.15.87:0/137211601 shutdown_connections 2024-04-25T10:14:51.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 --2- 172.21.15.87:0/137211601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cf930 0x7f2b3c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 --2- 172.21.15.87:0/137211601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c15bc40 0x7f2b3c06c090 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 -- 172.21.15.87:0/137211601 >> 172.21.15.87:0/137211601 conn(0x7f2b3c066b70 msgr2=0x7f2b3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 -- 172.21.15.87:0/137211601 shutdown_connections 2024-04-25T10:14:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 -- 172.21.15.87:0/137211601 wait complete. 2024-04-25T10:14:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 Processor -- start 2024-04-25T10:14:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.592+0000 7f2b43600640 1 -- start start 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cda10 0x7f2b3c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c0cf930 0x7f2b3c14ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b3c15bc40 0x7f2b3c1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0b3890 con 0x7f2b3c0cf930 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0b3710 con 0x7f2b3c15bc40 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0b3a10 con 0x7f2b3c0cda10 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c0cf930 0x7f2b3c14ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c0cf930 0x7f2b3c14ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51992/0 (socket says 172.21.15.87:51992) 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 -- 172.21.15.87:0/867641019 learned_addr learned my addr 172.21.15.87:0/867641019 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b40c00640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cda10 0x7f2b3c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b41600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b3c15bc40 0x7f2b3c1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 -- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cda10 msgr2=0x7f2b3c0b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cda10 0x7f2b3c0b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 -- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b3c15bc40 msgr2=0x7f2b3c1529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b3c15bc40 0x7f2b3c1529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 -- 172.21.15.87:0/867641019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c0b0490 con 0x7f2b3c0cf930 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b40c00640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cda10 0x7f2b3c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:51.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b33e00640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c0cf930 0x7f2b3c14ce30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c450 tx=0x7f2b2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b39600640 1 -- 172.21.15.87:0/867641019 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24017070 con 0x7f2b3c0cf930 2024-04-25T10:14:51.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b39600640 1 -- 172.21.15.87:0/867641019 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b24005480 con 0x7f2b3c0cf930 2024-04-25T10:14:51.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b39600640 1 -- 172.21.15.87:0/867641019 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24003840 con 0x7f2b3c0cf930 2024-04-25T10:14:51.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c0b0770 con 0x7f2b3c0cf930 2024-04-25T10:14:51.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c0b0d30 con 0x7f2b3c0cf930 2024-04-25T10:14:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c04ef30 con 0x7f2b3c0cf930 2024-04-25T10:14:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.596+0000 7f2b39600640 1 -- 172.21.15.87:0/867641019 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b2401a070 con 0x7f2b3c0cf930 2024-04-25T10:14:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.600+0000 7f2b39600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b18066f60 0x7f2b18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.600+0000 7f2b39600640 1 -- 172.21.15.87:0/867641019 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2b24084330 con 0x7f2b3c0cf930 2024-04-25T10:14:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.600+0000 7f2b40c00640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b18066f60 0x7f2b18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.600+0000 7f2b40c00640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b18066f60 0x7f2b18069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0ad000 tx=0x7f2b2c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.604+0000 7f2b39600640 1 -- 172.21.15.87:0/867641019 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b24054c30 con 0x7f2b3c0cf930 2024-04-25T10:14:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.760+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b3c0af1c0 con 0x7f2b3c0cf930 2024-04-25T10:14:51.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.760+0000 7f2b39600640 1 -- 172.21.15.87:0/867641019 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16722 (secure 0 0 0) 0x7f2b24058cd0 con 0x7f2b3c0cf930 2024-04-25T10:14:51.766 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:51.766 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":35,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:14:27.878686+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:14.236622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:15.422837+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b18066f60 msgr2=0x7f2b18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b18066f60 0x7f2b18069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0ad000 tx=0x7f2b2c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c0cf930 msgr2=0x7f2b3c14ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c0cf930 0x7f2b3c14ce30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c450 tx=0x7f2b2400c910 comp rx=0 tx=0).stop 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 shutdown_connections 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b18066f60 0x7f2b18069420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b3c15bc40 0x7f2b3c1529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b3c0cf930 0x7f2b3c14ce30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 --2- 172.21.15.87:0/867641019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b3c0cda10 0x7f2b3c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 >> 172.21.15.87:0/867641019 conn(0x7f2b3c066b70 msgr2=0x7f2b3c158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 shutdown_connections 2024-04-25T10:14:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:51.764+0000 7f2b43600640 1 -- 172.21.15.87:0/867641019 wait complete. 2024-04-25T10:14:51.772 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:51.775 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:51.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:14.236622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:15.422837+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:51.788 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:52.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.000+0000 7f8292000640 1 Processor -- start 2024-04-25T10:14:52.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.000+0000 7f8292000640 1 -- start start 2024-04-25T10:14:52.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8292000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f828c071db0 0x7f828c165730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8292000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0b3470 con 0x7f828c071260 2024-04-25T10:14:52.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8292000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c06de60 con 0x7f828c15a780 2024-04-25T10:14:52.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8292000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c06cdc0 con 0x7f828c071db0 2024-04-25T10:14:52.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8283e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f828c071260 0x7f828c0716a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38270/0 (socket says 172.21.15.87:38270) 2024-04-25T10:14:52.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8283e00640 1 -- 172.21.15.87:0/2522589351 learned_addr learned my addr 172.21.15.87:0/2522589351 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:52.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110353431 0 0) 0x7f8274001120 con 0x7f828c071260 2024-04-25T10:14:52.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8291600640 1 --2- 172.21.15.87:0/2522589351 >> v2:172.21.15.169:3300/0 conn(0x7f828c071db0 0x7f828c165730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8270003530 con 0x7f828c071260 2024-04-25T10:14:52.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1586999290 0 0) 0x7f8270003530 con 0x7f828c071260 2024-04-25T10:14:52.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 >> v2:172.21.15.169:3300/0 conn(0x7f828c071db0 msgr2=0x7f828c165730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 --2- 172.21.15.87:0/2522589351 >> v2:172.21.15.169:3300/0 conn(0x7f828c071db0 0x7f828c165730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 >> v1:172.21.15.169:6789/0 conn(0x7f828c15a780 legacy=0x7f828c15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c06d0a0 con 0x7f828c071260 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f8274001850 con 0x7f828c071260 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 >> v1:172.21.15.87:6789/0 conn(0x7f828c071260 legacy=0x7f828c0716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270003a80 0x7f8270003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270004810 0x7f8270004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8270005220 0x7f827000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c06cdc0 con 0x7f8270003a80 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827000c150 con 0x7f8270005220 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827000c350 con 0x7f8270004810 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8290c00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270003a80 0x7f8270003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8283e00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270004810 0x7f8270004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8291600640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8270005220 0x7f827000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8290c00640 1 -- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270004810 msgr2=0x7f8270004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8290c00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270004810 0x7f8270004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8290c00640 1 -- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8270005220 msgr2=0x7f827000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8290c00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8270005220 0x7f827000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8290c00640 1 -- 172.21.15.87:0/2522589351 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270003530 con 0x7f8270003a80 2024-04-25T10:14:52.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8291600640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8270005220 0x7f827000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:52.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.004+0000 7f8291600640 1 -- 172.21.15.87:0/2522589351 reap_dead start 2024-04-25T10:14:52.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8290c00640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270003a80 0x7f8270003ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f827c001780 tx=0x7f827c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a214f8ef9577861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c0013a0 con 0x7f8270003a80 2024-04-25T10:14:52.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c0052f0 con 0x7f8270003a80 2024-04-25T10:14:52.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f828be00640 1 -- 172.21.15.87:0/2522589351 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c003840 con 0x7f8270003a80 2024-04-25T10:14:52.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 -- 172.21.15.87:0/2522589351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270003a80 msgr2=0x7f8270003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270003a80 0x7f8270003ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f827c001780 tx=0x7f827c01c400 comp rx=0 tx=0).stop 2024-04-25T10:14:52.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 -- 172.21.15.87:0/2522589351 shutdown_connections 2024-04-25T10:14:52.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 --2- 172.21.15.87:0/2522589351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270003a80 0x7f8270003ea0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 -- 172.21.15.87:0/2522589351 >> 172.21.15.87:0/2522589351 conn(0x7f828c0743f0 msgr2=0x7f828c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 -- 172.21.15.87:0/2522589351 shutdown_connections 2024-04-25T10:14:52.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 -- 172.21.15.87:0/2522589351 wait complete. 2024-04-25T10:14:52.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 Processor -- start 2024-04-25T10:14:52.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.008+0000 7f8292000640 1 -- start start 2024-04-25T10:14:52.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270003a80 0x7f828c1fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828c14f1f0 0x7f828c158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f828c14f670 0x7f828c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c15f340 con 0x7f828c14f670 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c15f1c0 con 0x7f828c14f1f0 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c071a50 con 0x7f8270003a80 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f828c14f670 0x7f828c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f828c14f670 0x7f828c158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52004/0 (socket says 172.21.15.87:52004) 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 -- 172.21.15.87:0/1706138893 learned_addr learned my addr 172.21.15.87:0/1706138893 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8290c00640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270003a80 0x7f828c1fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8283e00640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828c14f1f0 0x7f828c158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 -- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270003a80 msgr2=0x7f828c1fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270003a80 0x7f828c1fb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 -- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828c14f1f0 msgr2=0x7f828c158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828c14f1f0 0x7f828c158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 -- 172.21.15.87:0/1706138893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c1fb8a0 con 0x7f828c14f670 2024-04-25T10:14:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8291600640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f828c14f670 0x7f828c158cb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f828c0cf210 tx=0x7f8284010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8289600640 1 -- 172.21.15.87:0/1706138893 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284010da0 con 0x7f828c14f670 2024-04-25T10:14:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8289600640 1 -- 172.21.15.87:0/1706138893 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82840039a0 con 0x7f828c14f670 2024-04-25T10:14:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8289600640 1 -- 172.21.15.87:0/1706138893 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284003d90 con 0x7f828c14f670 2024-04-25T10:14:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c0cdb50 con 0x7f828c14f670 2024-04-25T10:14:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c0ce0d0 con 0x7f828c14f670 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c0ce9c0 con 0x7f828c14f670 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.012+0000 7f8289600640 1 -- 172.21.15.87:0/1706138893 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f828401e020 con 0x7f828c14f670 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.016+0000 7f8289600640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8268066f60 0x7f8268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.016+0000 7f8289600640 1 -- 172.21.15.87:0/1706138893 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8284085850 con 0x7f828c14f670 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.016+0000 7f8290c00640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8268066f60 0x7f8268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.016+0000 7f8290c00640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8268066f60 0x7f8268069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f827c001750 tx=0x7f827c0011a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.020+0000 7f8289600640 1 -- 172.21.15.87:0/1706138893 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f828c1fb8a0 con 0x7f828c14f670 2024-04-25T10:14:52.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.176+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f828c075120 con 0x7f828c14f670 2024-04-25T10:14:52.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.176+0000 7f8289600640 1 -- 172.21.15.87:0/1706138893 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82840567a0 con 0x7f828c14f670 2024-04-25T10:14:52.182 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:14:52.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8268066f60 msgr2=0x7f8268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8268066f60 0x7f8268069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f827c001750 tx=0x7f827c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f828c14f670 msgr2=0x7f828c158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f828c14f670 0x7f828c158cb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f828c0cf210 tx=0x7f8284010590 comp rx=0 tx=0).stop 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 shutdown_connections 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8268066f60 0x7f8268069420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f828c14f670 0x7f828c158cb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828c14f1f0 0x7f828c158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 --2- 172.21.15.87:0/1706138893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8270003a80 0x7f828c1fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 >> 172.21.15.87:0/1706138893 conn(0x7f828c0743f0 msgr2=0x7f828c0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 shutdown_connections 2024-04-25T10:14:52.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.180+0000 7f8292000640 1 -- 172.21.15.87:0/1706138893 wait complete. 2024-04-25T10:14:52.203 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:52.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.336+0000 7f7818e00640 1 Processor -- start 2024-04-25T10:14:52.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.336+0000 7f7818e00640 1 -- start start 2024-04-25T10:14:52.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.336+0000 7f7818e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7814070350 0x7f781415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.336+0000 7f7818e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78140b31c0 0x7f78140b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.336+0000 7f7818e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781415b080 con 0x7f78140aef80 2024-04-25T10:14:52.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.336+0000 7f7818e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781415adb0 con 0x7f78140b31c0 2024-04-25T10:14:52.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.336+0000 7f7818e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781415eb60 con 0x7f7814070350 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7809400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7814070350 0x7f781415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7811e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f78140aef80 0x7f781415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:60860/0 (socket says 172.21.15.87:60860) 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7811e00640 1 -- 172.21.15.87:0/3440788942 learned_addr learned my addr 172.21.15.87:0/3440788942 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7812800640 1 --2- 172.21.15.87:0/3440788942 >> v2:172.21.15.169:3300/0 conn(0x7f78140b31c0 0x7f78140b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7809400640 1 -- 172.21.15.87:0/3440788942 >> v2:172.21.15.169:3300/0 conn(0x7f78140b31c0 msgr2=0x7f78140b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7809400640 1 --2- 172.21.15.87:0/3440788942 >> v2:172.21.15.169:3300/0 conn(0x7f78140b31c0 0x7f78140b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7809400640 1 -- 172.21.15.87:0/3440788942 >> v1:172.21.15.169:6789/0 conn(0x7f78140aef80 legacy=0x7f781415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7809400640 1 -- 172.21.15.87:0/3440788942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781415ee40 con 0x7f7814070350 2024-04-25T10:14:52.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7812800640 1 --2- 172.21.15.87:0/3440788942 >> v2:172.21.15.169:3300/0 conn(0x7f78140b31c0 0x7f78140b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:52.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7809400640 1 --2- 172.21.15.87:0/3440788942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7814070350 0x7f781415e480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0097c0 tx=0x7f77fc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c513877bf64f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.340+0000 7f7811400640 1 -- 172.21.15.87:0/3440788942 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0b1070 con 0x7f7814070350 2024-04-25T10:14:52.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7811400640 1 -- 172.21.15.87:0/3440788942 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc0ac410 con 0x7f7814070350 2024-04-25T10:14:52.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7811400640 1 -- 172.21.15.87:0/3440788942 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0ac7e0 con 0x7f7814070350 2024-04-25T10:14:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- 172.21.15.87:0/3440788942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7814070350 msgr2=0x7f781415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 --2- 172.21.15.87:0/3440788942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7814070350 0x7f781415e480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0097c0 tx=0x7f77fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- 172.21.15.87:0/3440788942 shutdown_connections 2024-04-25T10:14:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 --2- 172.21.15.87:0/3440788942 >> v2:172.21.15.169:3300/0 conn(0x7f78140b31c0 0x7f78140b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 --2- 172.21.15.87:0/3440788942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7814070350 0x7f781415e480 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- 172.21.15.87:0/3440788942 >> 172.21.15.87:0/3440788942 conn(0x7f78140669a0 msgr2=0x7f7814066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- 172.21.15.87:0/3440788942 shutdown_connections 2024-04-25T10:14:52.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- 172.21.15.87:0/3440788942 wait complete. 2024-04-25T10:14:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 Processor -- start 2024-04-25T10:14:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- start start 2024-04-25T10:14:52.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7814070350 0x7f7814151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78140aef80 0x7f78140af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78140b31c0 0x7f78140b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781406c250 con 0x7f78140aef80 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781406c0d0 con 0x7f7814070350 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7818e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781406c3d0 con 0x7f78140b31c0 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7809400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78140aef80 0x7f78140af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7809400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78140aef80 0x7f78140af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52016/0 (socket says 172.21.15.87:52016) 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.344+0000 7f7809400640 1 -- 172.21.15.87:0/1698525702 learned_addr learned my addr 172.21.15.87:0/1698525702 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7812800640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78140b31c0 0x7f78140b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7811e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7814070350 0x7f7814151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7809400640 1 -- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78140b31c0 msgr2=0x7f78140b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7809400640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78140b31c0 0x7f78140b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7809400640 1 -- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7814070350 msgr2=0x7f7814151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7809400640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7814070350 0x7f7814151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7809400640 1 -- 172.21.15.87:0/1698525702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78140afd30 con 0x7f78140aef80 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7812800640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78140b31c0 0x7f78140b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7809400640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78140aef80 0x7f78140af3e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0af040 tx=0x7f77fc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f780aa00640 1 -- 172.21.15.87:0/1698525702 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0b9020 con 0x7f78140aef80 2024-04-25T10:14:52.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f780aa00640 1 -- 172.21.15.87:0/1698525702 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc0acd90 con 0x7f78140aef80 2024-04-25T10:14:52.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78140b0010 con 0x7f78140aef80 2024-04-25T10:14:52.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f780aa00640 1 -- 172.21.15.87:0/1698525702 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0b1040 con 0x7f78140aef80 2024-04-25T10:14:52.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78140b05d0 con 0x7f78140aef80 2024-04-25T10:14:52.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781404ef30 con 0x7f78140aef80 2024-04-25T10:14:52.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f780aa00640 1 -- 172.21.15.87:0/1698525702 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77fc0bc020 con 0x7f78140aef80 2024-04-25T10:14:52.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f780aa00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f0066f60 0x7f77f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f780aa00640 1 -- 172.21.15.87:0/1698525702 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f77fc11e960 con 0x7f78140aef80 2024-04-25T10:14:52.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.348+0000 7f7811e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f0066f60 0x7f77f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.352+0000 7f7811e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f0066f60 0x7f77f0069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7804000f70 tx=0x7f7804009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.352+0000 7f780aa00640 1 -- 172.21.15.87:0/1698525702 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77fc0c3030 con 0x7f78140aef80 2024-04-25T10:14:52.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.504+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f781415a620 con 0x7f78140aef80 2024-04-25T10:14:52.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f780aa00640 1 -- 172.21.15.87:0/1698525702 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77fc016090 con 0x7f78140aef80 2024-04-25T10:14:52.512 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f0066f60 msgr2=0x7f77f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f0066f60 0x7f77f0069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7804000f70 tx=0x7f7804009040 comp rx=0 tx=0).stop 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78140aef80 msgr2=0x7f78140af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78140aef80 0x7f78140af3e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0af040 tx=0x7f77fc019240 comp rx=0 tx=0).stop 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 shutdown_connections 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f0066f60 0x7f77f0069420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78140b31c0 0x7f78140b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78140aef80 0x7f78140af3e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 --2- 172.21.15.87:0/1698525702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7814070350 0x7f7814151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 >> 172.21.15.87:0/1698525702 conn(0x7f78140669a0 msgr2=0x7f78141565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 shutdown_connections 2024-04-25T10:14:52.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.508+0000 7f7818e00640 1 -- 172.21.15.87:0/1698525702 wait complete. 2024-04-25T10:14:52.533 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:52.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.664+0000 7f5a1f000640 1 Processor -- start 2024-04-25T10:14:52.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.664+0000 7f5a1f000640 1 -- start start 2024-04-25T10:14:52.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a18071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1f000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5a18072020 0x7f5a18070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1f000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a18159920 con 0x7f5a18159540 2024-04-25T10:14:52.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1f000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1815c2f0 con 0x7f5a18072020 2024-04-25T10:14:52.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1815c5d0 con 0x7f5a180aeda0 2024-04-25T10:14:52.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a17e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5a18159540 0x7f5a180ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38274/0 (socket says 172.21.15.87:38274) 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a17e00640 1 -- 172.21.15.87:0/2915621814 learned_addr learned my addr 172.21.15.87:0/2915621814 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1d000640 1 --2- 172.21.15.87:0/2915621814 >> v2:172.21.15.87:3300/0 conn(0x7f5a18072020 0x7f5a18070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a0f400640 1 --2- 172.21.15.87:0/2915621814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a18071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a17400640 1 -- 172.21.15.87:0/2915621814 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902805708 0 0) 0x7f5a18159920 con 0x7f5a18159540 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a17400640 1 -- 172.21.15.87:0/2915621814 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59fc003530 con 0x7f5a18159540 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1d000640 1 -- 172.21.15.87:0/2915621814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 msgr2=0x7f5a18071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1d000640 1 --2- 172.21.15.87:0/2915621814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a18071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1d000640 1 -- 172.21.15.87:0/2915621814 >> v1:172.21.15.87:6789/0 conn(0x7f5a18159540 legacy=0x7f5a180ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1d000640 1 -- 172.21.15.87:0/2915621814 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1815c8b0 con 0x7f5a18072020 2024-04-25T10:14:52.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.668+0000 7f5a1d000640 1 --2- 172.21.15.87:0/2915621814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a18070350 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009d90 tx=0x7f5a1001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2d866390ef7a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17400640 1 -- 172.21.15.87:0/2915621814 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a100b2070 con 0x7f5a18072020 2024-04-25T10:14:52.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17400640 1 -- 172.21.15.87:0/2915621814 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a100a5a20 con 0x7f5a18072020 2024-04-25T10:14:52.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17400640 1 -- 172.21.15.87:0/2915621814 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a100a5df0 con 0x7f5a18072020 2024-04-25T10:14:52.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- 172.21.15.87:0/2915621814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 msgr2=0x7f5a18070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2915621814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a18070350 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009d90 tx=0x7f5a1001c940 comp rx=0 tx=0).stop 2024-04-25T10:14:52.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- 172.21.15.87:0/2915621814 shutdown_connections 2024-04-25T10:14:52.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2915621814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a18070350 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2915621814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a18071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- 172.21.15.87:0/2915621814 >> 172.21.15.87:0/2915621814 conn(0x7f5a180669a0 msgr2=0x7f5a18066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- 172.21.15.87:0/2915621814 shutdown_connections 2024-04-25T10:14:52.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- 172.21.15.87:0/2915621814 wait complete. 2024-04-25T10:14:52.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 Processor -- start 2024-04-25T10:14:52.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- start start 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a1814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a1815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a18159540 0x7f5a18153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180710a0 con 0x7f5a18072020 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1806c0d0 con 0x7f5a18159540 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18071220 con 0x7f5a180aeda0 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a1814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a1814ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52034/0 (socket says 172.21.15.87:52034) 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 -- 172.21.15.87:0/2312092760 learned_addr learned my addr 172.21.15.87:0/2312092760 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a0f400640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a1815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a1d000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a18159540 0x7f5a18153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 -- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 msgr2=0x7f5a1815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a1815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 -- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a18159540 msgr2=0x7f5a18153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a18159540 0x7f5a18153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.672+0000 7f5a17e00640 1 -- 172.21.15.87:0/2312092760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1815bbe0 con 0x7f5a18072020 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a1d000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a18159540 0x7f5a18153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a17e00640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a1814ec30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c450 tx=0x7f5a0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a14c00640 1 -- 172.21.15.87:0/2312092760 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08017070 con 0x7f5a18072020 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a14c00640 1 -- 172.21.15.87:0/2312092760 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08005cf0 con 0x7f5a18072020 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1815b060 con 0x7f5a18072020 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a14c00640 1 -- 172.21.15.87:0/2312092760 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08007290 con 0x7f5a18072020 2024-04-25T10:14:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1815b620 con 0x7f5a18072020 2024-04-25T10:14:52.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a18159af0 con 0x7f5a18072020 2024-04-25T10:14:52.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a14c00640 1 -- 172.21.15.87:0/2312092760 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a0801a070 con 0x7f5a18072020 2024-04-25T10:14:52.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a14c00640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f59f4066f60 0x7f59f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a14c00640 1 -- 172.21.15.87:0/2312092760 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5a08084460 con 0x7f5a18072020 2024-04-25T10:14:52.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.676+0000 7f5a0f400640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f59f4066f60 0x7f59f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.680+0000 7f5a0f400640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f59f4066f60 0x7f59f4069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5a1815e390 tx=0x7f5a00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.680+0000 7f5a14c00640 1 -- 172.21.15.87:0/2312092760 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a18159af0 con 0x7f5a18072020 2024-04-25T10:14:52.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.832+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a1804ef30 con 0x7f5a18072020 2024-04-25T10:14:52.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.832+0000 7f5a14c00640 1 -- 172.21.15.87:0/2312092760 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a08055450 con 0x7f5a18072020 2024-04-25T10:14:52.839 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f59f4066f60 msgr2=0x7f59f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f59f4066f60 0x7f59f4069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5a1815e390 tx=0x7f5a00008040 comp rx=0 tx=0).stop 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 msgr2=0x7f5a1814ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a1814ec30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c450 tx=0x7f5a0800c910 comp rx=0 tx=0).stop 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 shutdown_connections 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f59f4066f60 0x7f59f4069420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a18159540 0x7f5a18153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a180aeda0 0x7f5a1815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 --2- 172.21.15.87:0/2312092760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a18072020 0x7f5a1814ec30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 >> 172.21.15.87:0/2312092760 conn(0x7f5a180669a0 msgr2=0x7f5a18066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 shutdown_connections 2024-04-25T10:14:52.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.836+0000 7f5a1f000640 1 -- 172.21.15.87:0/2312092760 wait complete. 2024-04-25T10:14:52.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:52.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.988+0000 7fe6db800640 1 Processor -- start 2024-04-25T10:14:52.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.988+0000 7fe6db800640 1 -- start start 2024-04-25T10:14:52.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6db800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6db800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d4071050 con 0x7fe6d415b910 2024-04-25T10:14:52.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6db800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d415d040 con 0x7fe6d40cfe70 2024-04-25T10:14:52.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6db800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d415bda0 con 0x7fe6d41550b0 2024-04-25T10:14:52.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d9800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe6d415b910 0x7fe6d40b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38282/0 (socket says 172.21.15.87:38282) 2024-04-25T10:14:52.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d9800640 1 -- 172.21.15.87:0/434067406 learned_addr learned my addr 172.21.15.87:0/434067406 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:52.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d8e00640 1 --2- 172.21.15.87:0/434067406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d3400640 1 -- 172.21.15.87:0/434067406 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834520740 0 0) 0x7fe6d4071050 con 0x7fe6d415b910 2024-04-25T10:14:52.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d3400640 1 -- 172.21.15.87:0/434067406 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b8003530 con 0x7fe6d415b910 2024-04-25T10:14:52.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d8e00640 1 -- 172.21.15.87:0/434067406 >> v1:172.21.15.169:6789/0 conn(0x7fe6d40cfe70 legacy=0x7fe6d415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d8e00640 1 -- 172.21.15.87:0/434067406 >> v1:172.21.15.87:6789/0 conn(0x7fe6d415b910 legacy=0x7fe6d40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.992+0000 7fe6d8e00640 1 -- 172.21.15.87:0/434067406 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d40712f0 con 0x7fe6d41550b0 2024-04-25T10:14:53.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6d8e00640 1 --2- 172.21.15.87:0/434067406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cf930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097f0 tx=0x7fe6c401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dfa85b0fce9b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6d3400640 1 -- 172.21.15.87:0/434067406 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40b0070 con 0x7fe6d41550b0 2024-04-25T10:14:53.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6d3400640 1 -- 172.21.15.87:0/434067406 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c40ab600 con 0x7fe6d41550b0 2024-04-25T10:14:53.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6d3400640 1 -- 172.21.15.87:0/434067406 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40ab9d0 con 0x7fe6d41550b0 2024-04-25T10:14:53.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 -- 172.21.15.87:0/434067406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 msgr2=0x7fe6d40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 --2- 172.21.15.87:0/434067406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cf930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097f0 tx=0x7fe6c401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 -- 172.21.15.87:0/434067406 shutdown_connections 2024-04-25T10:14:53.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 --2- 172.21.15.87:0/434067406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cf930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 -- 172.21.15.87:0/434067406 >> 172.21.15.87:0/434067406 conn(0x7fe6d4066b70 msgr2=0x7fe6d4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 -- 172.21.15.87:0/434067406 shutdown_connections 2024-04-25T10:14:53.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 -- 172.21.15.87:0/434067406 wait complete. 2024-04-25T10:14:53.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 Processor -- start 2024-04-25T10:14:53.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:52.996+0000 7fe6db800640 1 -- start start 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6d40cfe70 0x7fe6d413ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d415b910 0x7fe6d40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d41684c0 con 0x7fe6d415b910 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d40720b0 con 0x7fe6d40cfe70 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4168640 con 0x7fe6d41550b0 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d415b910 0x7fe6d40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d415b910 0x7fe6d40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52038/0 (socket says 172.21.15.87:52038) 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 -- 172.21.15.87:0/2978486148 learned_addr learned my addr 172.21.15.87:0/2978486148 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d3e00640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d8e00640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6d40cfe70 0x7fe6d413ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 -- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 msgr2=0x7fe6d40cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 -- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6d40cfe70 msgr2=0x7fe6d413ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6d40cfe70 0x7fe6d413ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 -- 172.21.15.87:0/2978486148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4140510 con 0x7fe6d415b910 2024-04-25T10:14:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d9800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d415b910 0x7fe6d40ad840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00cb40 tx=0x7fe6cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d0c00640 1 -- 172.21.15.87:0/2978486148 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc017070 con 0x7fe6d415b910 2024-04-25T10:14:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d0c00640 1 -- 172.21.15.87:0/2978486148 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc00aca0 con 0x7fe6d415b910 2024-04-25T10:14:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d0c00640 1 -- 172.21.15.87:0/2978486148 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc012620 con 0x7fe6d415b910 2024-04-25T10:14:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d4159fd0 con 0x7fe6d415b910 2024-04-25T10:14:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d415a590 con 0x7fe6d415b910 2024-04-25T10:14:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d404ef30 con 0x7fe6d415b910 2024-04-25T10:14:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.000+0000 7fe6d0c00640 1 -- 172.21.15.87:0/2978486148 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6cc01a070 con 0x7fe6d415b910 2024-04-25T10:14:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.004+0000 7fe6d0c00640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6b4066f60 0x7fe6b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.004+0000 7fe6d0c00640 1 -- 172.21.15.87:0/2978486148 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe6cc084e80 con 0x7fe6d415b910 2024-04-25T10:14:53.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.004+0000 7fe6d8e00640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6b4066f60 0x7fe6b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.004+0000 7fe6d8e00640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6b4066f60 0x7fe6b4069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40ad000 tx=0x7fe6c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.008+0000 7fe6d0c00640 1 -- 172.21.15.87:0/2978486148 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6cc055de0 con 0x7fe6d415b910 2024-04-25T10:14:53.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.196+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6d4067230 con 0x7fe6d415b910 2024-04-25T10:14:53.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.196+0000 7fe6d0c00640 1 -- 172.21.15.87:0/2978486148 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fe6cc0556e0 con 0x7fe6d415b910 2024-04-25T10:14:53.203 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:53.203 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3059476782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3059476782},{"type":"v1","addr":"172.21.15.169:6834","nonce":3059476782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/685777206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":685777206},{"type":"v1","addr":"172.21.15.87:6837","nonce":685777206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2281421407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2281421407},{"type":"v1","addr":"172.21.15.169:6835","nonce":2281421407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:13:16.421834+0000","modified":"2024-04-25T10:13:28.839800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/4156145333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4156145333},{"type":"v1","addr":"172.21.15.87:6835","nonce":4156145333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:14:53.204 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 17 2024-04-25T10:14:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6b4066f60 msgr2=0x7fe6b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6b4066f60 0x7fe6b4069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40ad000 tx=0x7fe6c4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d415b910 msgr2=0x7fe6d40ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d415b910 0x7fe6d40ad840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00cb40 tx=0x7fe6cc00a060 comp rx=0 tx=0).stop 2024-04-25T10:14:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 shutdown_connections 2024-04-25T10:14:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6b4066f60 0x7fe6b4069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d415b910 0x7fe6d40ad840 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6d41550b0 0x7fe6d40cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 --2- 172.21.15.87:0/2978486148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6d40cfe70 0x7fe6d413ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 >> 172.21.15.87:0/2978486148 conn(0x7fe6d4066b70 msgr2=0x7fe6d406c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 shutdown_connections 2024-04-25T10:14:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.200+0000 7fe6db800640 1 -- 172.21.15.87:0/2978486148 wait complete. 2024-04-25T10:14:53.224 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:53.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 Processor -- start 2024-04-25T10:14:53.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 -- start start 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3141550b0 0x7fb3140cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3140cfe70 0x7fb31415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb31406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb314071050 con 0x7fb3140cfe70 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3140712f0 con 0x7fb3141550b0 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.356+0000 7fb31ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140715d0 con 0x7fb31415b740 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3140cfe70 0x7fb31415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3140cfe70 0x7fb31415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52052/0 (socket says 172.21.15.87:52052) 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 -- 172.21.15.87:0/1752905795 learned_addr learned my addr 172.21.15.87:0/1752905795 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb318c00640 1 --2- 172.21.15.87:0/1752905795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb31406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313e00640 1 --2- 172.21.15.87:0/1752905795 >> v2:172.21.15.169:3300/0 conn(0x7fb3141550b0 0x7fb3140cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 -- 172.21.15.87:0/1752905795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 msgr2=0x7fb31406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 --2- 172.21.15.87:0/1752905795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb31406d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 -- 172.21.15.87:0/1752905795 >> v2:172.21.15.169:3300/0 conn(0x7fb3141550b0 msgr2=0x7fb3140cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 --2- 172.21.15.87:0/1752905795 >> v2:172.21.15.169:3300/0 conn(0x7fb3141550b0 0x7fb3140cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 -- 172.21.15.87:0/1752905795 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140718b0 con 0x7fb3140cfe70 2024-04-25T10:14:53.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb313400640 1 --2- 172.21.15.87:0/1752905795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31415b200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb300009870 tx=0x7fb30001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2480922e588372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb312a00640 1 -- 172.21.15.87:0/1752905795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000b0070 con 0x7fb3140cfe70 2024-04-25T10:14:53.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb312a00640 1 -- 172.21.15.87:0/1752905795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3000ab600 con 0x7fb3140cfe70 2024-04-25T10:14:53.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.360+0000 7fb312a00640 1 -- 172.21.15.87:0/1752905795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000ab9d0 con 0x7fb3140cfe70 2024-04-25T10:14:53.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- 172.21.15.87:0/1752905795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 msgr2=0x7fb31415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 --2- 172.21.15.87:0/1752905795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31415b200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb300009870 tx=0x7fb30001c260 comp rx=0 tx=0).stop 2024-04-25T10:14:53.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- 172.21.15.87:0/1752905795 shutdown_connections 2024-04-25T10:14:53.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 --2- 172.21.15.87:0/1752905795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31415b200 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 --2- 172.21.15.87:0/1752905795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb31406d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 --2- 172.21.15.87:0/1752905795 >> v2:172.21.15.169:3300/0 conn(0x7fb3141550b0 0x7fb3140cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- 172.21.15.87:0/1752905795 >> 172.21.15.87:0/1752905795 conn(0x7fb314066b70 msgr2=0x7fb314066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- 172.21.15.87:0/1752905795 shutdown_connections 2024-04-25T10:14:53.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- 172.21.15.87:0/1752905795 wait complete. 2024-04-25T10:14:53.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 Processor -- start 2024-04-25T10:14:53.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- start start 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31414ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3141550b0 0x7fb3141593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb314150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b3260 con 0x7fb3140cfe70 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31415bd60 con 0x7fb3141550b0 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb31ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b33e0 con 0x7fb31415b740 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31414ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31414ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52068/0 (socket says 172.21.15.87:52068) 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 -- 172.21.15.87:0/3070990429 learned_addr learned my addr 172.21.15.87:0/3070990429 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313400640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3141550b0 0x7fb3141593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb318c00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb314150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 -- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 msgr2=0x7fb314150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb314150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 -- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3141550b0 msgr2=0x7fb3141593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3141550b0 0x7fb3141593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 -- 172.21.15.87:0/3070990429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314158d30 con 0x7fb3140cfe70 2024-04-25T10:14:53.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.364+0000 7fb313e00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31414ad80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb304007ef0 tx=0x7fb30400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb30be00640 1 -- 172.21.15.87:0/3070990429 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304014620 con 0x7fb3140cfe70 2024-04-25T10:14:53.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb30be00640 1 -- 172.21.15.87:0/3070990429 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb304019070 con 0x7fb3140cfe70 2024-04-25T10:14:53.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb30be00640 1 -- 172.21.15.87:0/3070990429 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304014a00 con 0x7fb3140cfe70 2024-04-25T10:14:53.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb314159010 con 0x7fb3140cfe70 2024-04-25T10:14:53.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3141500e0 con 0x7fb3140cfe70 2024-04-25T10:14:53.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3140b4720 con 0x7fb3140cfe70 2024-04-25T10:14:53.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb30be00640 1 -- 172.21.15.87:0/3070990429 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb304014ba0 con 0x7fb3140cfe70 2024-04-25T10:14:53.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb30be00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2f4066f60 0x7fb2f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb30be00640 1 -- 172.21.15.87:0/3070990429 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb3040859c0 con 0x7fb3140cfe70 2024-04-25T10:14:53.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb313400640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2f4066f60 0x7fb2f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.368+0000 7fb313400640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2f4066f60 0x7fb2f4069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3000ad000 tx=0x7fb300008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.372+0000 7fb30be00640 1 -- 172.21.15.87:0/3070990429 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3040569a0 con 0x7fb3140cfe70 2024-04-25T10:14:53.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.528+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb31404ef30 con 0x7fb3140cfe70 2024-04-25T10:14:53.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb30be00640 1 -- 172.21.15.87:0/3070990429 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16722 (secure 0 0 0) 0x7fb3040562a0 con 0x7fb3140cfe70 2024-04-25T10:14:53.536 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:53.536 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":35,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:14:27.878686+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:14.236622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:15.422837+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2f4066f60 msgr2=0x7fb2f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2f4066f60 0x7fb2f4069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3000ad000 tx=0x7fb300008040 comp rx=0 tx=0).stop 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 msgr2=0x7fb31414ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31414ad80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb304007ef0 tx=0x7fb30400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 shutdown_connections 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2f4066f60 0x7fb2f4069420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb31415b740 0x7fb314150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3141550b0 0x7fb3141593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 --2- 172.21.15.87:0/3070990429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3140cfe70 0x7fb31414ad80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 >> 172.21.15.87:0/3070990429 conn(0x7fb314066b70 msgr2=0x7fb31415c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 shutdown_connections 2024-04-25T10:14:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.532+0000 7fb31ac00640 1 -- 172.21.15.87:0/3070990429 wait complete. 2024-04-25T10:14:53.557 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:14:53.557 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:53.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc849800640 1 Processor -- start 2024-04-25T10:14:53.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc849800640 1 -- start start 2024-04-25T10:14:53.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc849800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440af9c0 0x7fc84406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc849800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc8440afda0 0x7fc8440cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc849800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84406f200 con 0x7fc844071f50 2024-04-25T10:14:53.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc849800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84406d380 con 0x7fc8440afda0 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc849800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440ccc10 con 0x7fc8440af9c0 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc83a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc8440afda0 0x7fc8440cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc843400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc844071f50 0x7fc8440669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38294/0 (socket says 172.21.15.87:38294) 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc843400640 1 -- 172.21.15.87:0/1850168634 learned_addr learned my addr 172.21.15.87:0/1850168634 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc842a00640 1 --2- 172.21.15.87:0/1850168634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440af9c0 0x7fc84406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc83a000640 1 -- 172.21.15.87:0/1850168634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440af9c0 msgr2=0x7fc84406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc83a000640 1 --2- 172.21.15.87:0/1850168634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440af9c0 0x7fc84406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc83a000640 1 -- 172.21.15.87:0/1850168634 >> v1:172.21.15.87:6789/0 conn(0x7fc844071f50 legacy=0x7fc8440669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc83a000640 1 -- 172.21.15.87:0/1850168634 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8440ccef0 con 0x7fc8440afda0 2024-04-25T10:14:53.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.688+0000 7fc842a00640 1 --2- 172.21.15.87:0/1850168634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440af9c0 0x7fc84406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc83a000640 1 --2- 172.21.15.87:0/1850168634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8440afda0 0x7fc8440cc3a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097c0 tx=0x7fc82c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1f293dd31f7414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc842000640 1 -- 172.21.15.87:0/1850168634 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0b0070 con 0x7fc8440afda0 2024-04-25T10:14:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc842000640 1 -- 172.21.15.87:0/1850168634 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c004230 con 0x7fc8440afda0 2024-04-25T10:14:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc842000640 1 -- 172.21.15.87:0/1850168634 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c003620 con 0x7fc8440afda0 2024-04-25T10:14:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 -- 172.21.15.87:0/1850168634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8440afda0 msgr2=0x7fc8440cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 --2- 172.21.15.87:0/1850168634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8440afda0 0x7fc8440cc3a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097c0 tx=0x7fc82c01c350 comp rx=0 tx=0).stop 2024-04-25T10:14:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 -- 172.21.15.87:0/1850168634 shutdown_connections 2024-04-25T10:14:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 --2- 172.21.15.87:0/1850168634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8440afda0 0x7fc8440cc3a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 --2- 172.21.15.87:0/1850168634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440af9c0 0x7fc84406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 -- 172.21.15.87:0/1850168634 >> 172.21.15.87:0/1850168634 conn(0x7fc8440b2410 msgr2=0x7fc8440b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 -- 172.21.15.87:0/1850168634 shutdown_connections 2024-04-25T10:14:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.692+0000 7fc849800640 1 -- 172.21.15.87:0/1850168634 wait complete. 2024-04-25T10:14:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 Processor -- start 2024-04-25T10:14:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 -- start start 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc844071f50 0x7fc8440c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8440af9c0 0x7fc8440c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440afda0 0x7fc8440c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc844067230 con 0x7fc844071f50 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440670b0 con 0x7fc8440af9c0 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8441600d0 con 0x7fc8440afda0 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc844071f50 0x7fc8440c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc844071f50 0x7fc8440c67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52084/0 (socket says 172.21.15.87:52084) 2024-04-25T10:14:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 -- 172.21.15.87:0/1163718270 learned_addr learned my addr 172.21.15.87:0/1163718270 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 -- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440afda0 msgr2=0x7fc8440c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc843400640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440afda0 0x7fc8440c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440afda0 0x7fc8440c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc83a000640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8440af9c0 0x7fc8440c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 -- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8440af9c0 msgr2=0x7fc8440c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8440af9c0 0x7fc8440c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 -- 172.21.15.87:0/1163718270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84415f050 con 0x7fc844071f50 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc843400640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440afda0 0x7fc8440c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc83a000640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8440af9c0 0x7fc8440c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc842a00640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc844071f50 0x7fc8440c67f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc834009fc0 tx=0x7fc83400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc83b400640 1 -- 172.21.15.87:0/1163718270 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83401a070 con 0x7fc844071f50 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc83b400640 1 -- 172.21.15.87:0/1163718270 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834003ba0 con 0x7fc844071f50 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc844158900 con 0x7fc844071f50 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc83b400640 1 -- 172.21.15.87:0/1163718270 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834004030 con 0x7fc844071f50 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8440c6d30 con 0x7fc844071f50 2024-04-25T10:14:53.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.696+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84404ef30 con 0x7fc844071f50 2024-04-25T10:14:53.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.700+0000 7fc83b400640 1 -- 172.21.15.87:0/1163718270 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc83401d070 con 0x7fc844071f50 2024-04-25T10:14:53.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.700+0000 7fc83b400640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc820066f60 0x7fc820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.700+0000 7fc83b400640 1 -- 172.21.15.87:0/1163718270 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc8340852d0 con 0x7fc844071f50 2024-04-25T10:14:53.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.700+0000 7fc83a000640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc820066f60 0x7fc820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.700+0000 7fc83a000640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc820066f60 0x7fc820069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0ad000 tx=0x7fc82c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.704+0000 7fc83b400640 1 -- 172.21.15.87:0/1163718270 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc834010070 con 0x7fc844071f50 2024-04-25T10:14:53.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.888+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8440c7470 con 0x7fc844071f50 2024-04-25T10:14:53.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.888+0000 7fc83b400640 1 -- 172.21.15.87:0/1163718270 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fc834055c60 con 0x7fc844071f50 2024-04-25T10:14:53.895 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:53.895 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3059476782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3059476782},{"type":"v1","addr":"172.21.15.169:6834","nonce":3059476782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/685777206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":685777206},{"type":"v1","addr":"172.21.15.87:6837","nonce":685777206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2281421407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2281421407},{"type":"v1","addr":"172.21.15.169:6835","nonce":2281421407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:13:16.421834+0000","modified":"2024-04-25T10:13:28.839800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/4156145333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4156145333},{"type":"v1","addr":"172.21.15.87:6835","nonce":4156145333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:14:53.896 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 17 2024-04-25T10:14:53.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc820066f60 msgr2=0x7fc820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc820066f60 0x7fc820069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0ad000 tx=0x7fc82c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:53.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc844071f50 msgr2=0x7fc8440c67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc844071f50 0x7fc8440c67f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc834009fc0 tx=0x7fc83400d560 comp rx=0 tx=0).stop 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 shutdown_connections 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc820066f60 0x7fc820069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8440afda0 0x7fc8440c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8440af9c0 0x7fc8440c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 --2- 172.21.15.87:0/1163718270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc844071f50 0x7fc8440c67f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 >> 172.21.15.87:0/1163718270 conn(0x7fc8440b2410 msgr2=0x7fc8440ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 shutdown_connections 2024-04-25T10:14:53.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:53.892+0000 7fc849800640 1 -- 172.21.15.87:0/1163718270 wait complete. 2024-04-25T10:14:53.916 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:54.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4d0e00640 1 Processor -- start 2024-04-25T10:14:54.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4d0e00640 1 -- start start 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4d0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4d0e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4cc0cda10 0x7ff4cc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4d0e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc0b2770 con 0x7ff4cc0cf930 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4d0e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4cc06c670 con 0x7ff4cc0cda10 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4d0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc06c950 con 0x7ff4cc15bc40 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4ca800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff4cc0cf930 0x7ff4cc15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38304/0 (socket says 172.21.15.87:38304) 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.044+0000 7ff4ca800640 1 -- 172.21.15.87:0/649076637 learned_addr learned my addr 172.21.15.87:0/649076637 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c9e00640 1 --2- 172.21.15.87:0/649076637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c9400640 1 --2- 172.21.15.87:0/649076637 >> v2:172.21.15.169:3300/0 conn(0x7ff4cc0cda10 0x7ff4cc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c8a00640 1 -- 172.21.15.87:0/649076637 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571340090 0 0) 0x7ff4cc0b2770 con 0x7ff4cc0cf930 2024-04-25T10:14:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c8a00640 1 -- 172.21.15.87:0/649076637 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b0003530 con 0x7ff4cc0cf930 2024-04-25T10:14:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c8a00640 1 -- 172.21.15.87:0/649076637 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3845272640 0 0) 0x7ff4b0003530 con 0x7ff4cc0cf930 2024-04-25T10:14:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c9e00640 1 -- 172.21.15.87:0/649076637 >> v2:172.21.15.169:3300/0 conn(0x7ff4cc0cda10 msgr2=0x7ff4cc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c9e00640 1 --2- 172.21.15.87:0/649076637 >> v2:172.21.15.169:3300/0 conn(0x7ff4cc0cda10 0x7ff4cc0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c9e00640 1 -- 172.21.15.87:0/649076637 >> v1:172.21.15.87:6789/0 conn(0x7ff4cc0cf930 legacy=0x7ff4cc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c9e00640 1 -- 172.21.15.87:0/649076637 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc06cc30 con 0x7ff4cc15bc40 2024-04-25T10:14:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.048+0000 7ff4c9e00640 1 --2- 172.21.15.87:0/649076637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc06c090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009d90 tx=0x7ff4bc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c011daaaa811ca4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4c8a00640 1 -- 172.21.15.87:0/649076637 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0b2070 con 0x7ff4cc15bc40 2024-04-25T10:14:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4c8a00640 1 -- 172.21.15.87:0/649076637 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc0a5940 con 0x7ff4cc15bc40 2024-04-25T10:14:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4c8a00640 1 -- 172.21.15.87:0/649076637 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0a5d10 con 0x7ff4cc15bc40 2024-04-25T10:14:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- 172.21.15.87:0/649076637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 msgr2=0x7ff4cc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/649076637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc06c090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009d90 tx=0x7ff4bc01c970 comp rx=0 tx=0).stop 2024-04-25T10:14:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- 172.21.15.87:0/649076637 shutdown_connections 2024-04-25T10:14:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/649076637 >> v2:172.21.15.169:3300/0 conn(0x7ff4cc0cda10 0x7ff4cc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/649076637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc06c090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- 172.21.15.87:0/649076637 >> 172.21.15.87:0/649076637 conn(0x7ff4cc066b70 msgr2=0x7ff4cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- 172.21.15.87:0/649076637 shutdown_connections 2024-04-25T10:14:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- 172.21.15.87:0/649076637 wait complete. 2024-04-25T10:14:54.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 Processor -- start 2024-04-25T10:14:54.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- start start 2024-04-25T10:14:54.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4cc0cda10 0x7ff4cc1459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4cc0cf930 0x7ff4cc1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc15e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc168b50 con 0x7ff4cc0cda10 2024-04-25T10:14:54.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1689d0 con 0x7ff4cc0cf930 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4d0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc168cd0 con 0x7ff4cc15bc40 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4c9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4cc0cda10 0x7ff4cc1459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4c9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4cc0cda10 0x7ff4cc1459a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52098/0 (socket says 172.21.15.87:52098) 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.052+0000 7ff4c9e00640 1 -- 172.21.15.87:0/1399358372 learned_addr learned my addr 172.21.15.87:0/1399358372 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9400640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4cc0cf930 0x7ff4cc1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9e00640 1 -- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 msgr2=0x7ff4cc15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9e00640 1 -- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4cc0cf930 msgr2=0x7ff4cc1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4cc0cf930 0x7ff4cc1570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9e00640 1 -- 172.21.15.87:0/1399358372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0b07a0 con 0x7ff4cc0cda10 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9400640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4cc0cf930 0x7ff4cc1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4cc0cda10 0x7ff4cc1459a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0b0040 tx=0x7ff4bc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c2000640 1 -- 172.21.15.87:0/1399358372 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0b2040 con 0x7ff4cc0cda10 2024-04-25T10:14:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c2000640 1 -- 172.21.15.87:0/1399358372 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc0bf070 con 0x7ff4cc0cda10 2024-04-25T10:14:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc0b0a80 con 0x7ff4cc0cda10 2024-04-25T10:14:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c2000640 1 -- 172.21.15.87:0/1399358372 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0b88b0 con 0x7ff4cc0cda10 2024-04-25T10:14:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc0b1040 con 0x7ff4cc0cda10 2024-04-25T10:14:54.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc04ef30 con 0x7ff4cc0cda10 2024-04-25T10:14:54.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c2000640 1 -- 172.21.15.87:0/1399358372 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4bc0a57c0 con 0x7ff4cc0cda10 2024-04-25T10:14:54.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c2000640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4ac066f60 0x7ff4ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c2000640 1 -- 172.21.15.87:0/1399358372 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff4bc120090 con 0x7ff4cc0cda10 2024-04-25T10:14:54.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.056+0000 7ff4c9400640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4ac066f60 0x7ff4ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.060+0000 7ff4c9400640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4ac066f60 0x7ff4ac069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc157c00 tx=0x7ff4b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.060+0000 7ff4c2000640 1 -- 172.21.15.87:0/1399358372 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4bc016560 con 0x7ff4cc0cda10 2024-04-25T10:14:54.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.248+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4cc06d960 con 0x7ff4cc0cda10 2024-04-25T10:14:54.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.248+0000 7ff4c2000640 1 -- 172.21.15.87:0/1399358372 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7ff4bc0b6e20 con 0x7ff4cc0cda10 2024-04-25T10:14:54.256 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:54.256 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3059476782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3059476782},{"type":"v1","addr":"172.21.15.169:6834","nonce":3059476782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/685777206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":685777206},{"type":"v1","addr":"172.21.15.87:6837","nonce":685777206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2281421407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2281421407},{"type":"v1","addr":"172.21.15.169:6835","nonce":2281421407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:13:16.421834+0000","modified":"2024-04-25T10:13:28.839800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/4156145333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4156145333},{"type":"v1","addr":"172.21.15.87:6835","nonce":4156145333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:14:54.256 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 17 2024-04-25T10:14:54.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4ac066f60 msgr2=0x7ff4ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4ac066f60 0x7ff4ac069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc157c00 tx=0x7ff4b8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4cc0cda10 msgr2=0x7ff4cc1459a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4cc0cda10 0x7ff4cc1459a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0b0040 tx=0x7ff4bc019710 comp rx=0 tx=0).stop 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 shutdown_connections 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4ac066f60 0x7ff4ac069420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4cc15bc40 0x7ff4cc15e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4cc0cf930 0x7ff4cc1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 --2- 172.21.15.87:0/1399358372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4cc0cda10 0x7ff4cc1459a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 >> 172.21.15.87:0/1399358372 conn(0x7ff4cc066b70 msgr2=0x7ff4cc157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 shutdown_connections 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.252+0000 7ff4d0e00640 1 -- 172.21.15.87:0/1399358372 wait complete. 2024-04-25T10:14:54.276 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:54.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.404+0000 7f52d9a00640 1 Processor -- start 2024-04-25T10:14:54.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.404+0000 7f52d9a00640 1 -- start start 2024-04-25T10:14:54.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d9a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52d415b9e0 0x7f52d40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d9a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40ccfb0 con 0x7f52d40d1c30 2024-04-25T10:14:54.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d9a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40cd290 con 0x7f52d415bdc0 2024-04-25T10:14:54.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d9a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d40cd570 con 0x7f52d415b9e0 2024-04-25T10:14:54.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f52d40d1c30 0x7f52d40c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38314/0 (socket says 172.21.15.87:38314) 2024-04-25T10:14:54.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d3400640 1 -- 172.21.15.87:0/2233375655 learned_addr learned my addr 172.21.15.87:0/2233375655 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:54.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306750180 0 0) 0x7f52d40ccfb0 con 0x7f52d40d1c30 2024-04-25T10:14:54.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2233375655 >> v2:172.21.15.169:3300/0 conn(0x7f52d415b9e0 0x7f52d40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b8003530 con 0x7f52d40d1c30 2024-04-25T10:14:54.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4275752944 0 0) 0x7f52b8003530 con 0x7f52d40d1c30 2024-04-25T10:14:54.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 >> v2:172.21.15.169:3300/0 conn(0x7f52d415b9e0 msgr2=0x7f52d40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 --2- 172.21.15.87:0/2233375655 >> v2:172.21.15.169:3300/0 conn(0x7f52d415b9e0 0x7f52d40cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 >> v1:172.21.15.169:6789/0 conn(0x7f52d415bdc0 legacy=0x7f52d40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d40cd850 con 0x7f52d40d1c30 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f52cc001680 con 0x7f52d40d1c30 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 >> v1:172.21.15.87:6789/0 conn(0x7f52d40d1c30 legacy=0x7f52d40c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b80039c0 0x7f52b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b8004810 0x7f52b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b80051d0 0x7f52b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40cd570 con 0x7f52b80039c0 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b800c0d0 con 0x7f52b8004810 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b800c2d0 con 0x7f52b80051d0 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b80039c0 0x7f52b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52ca000640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b8004810 0x7f52b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d3400640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b80051d0 0x7f52b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2a00640 1 -- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b80051d0 msgr2=0x7f52b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b80051d0 0x7f52b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2a00640 1 -- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b8004810 msgr2=0x7f52b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b8004810 0x7f52b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d2a00640 1 -- 172.21.15.87:0/2233375655 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8003530 con 0x7f52b80039c0 2024-04-25T10:14:54.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d3400640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b80051d0 0x7f52b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:54.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.408+0000 7f52d3400640 1 -- 172.21.15.87:0/2233375655 reap_dead start 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b80039c0 0x7f52b8003da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009fc0 tx=0x7f52c401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63db335f88a866c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40b2070 con 0x7f52b80039c0 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40a5b20 con 0x7f52b80039c0 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d2000640 1 -- 172.21.15.87:0/2233375655 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4009c10 con 0x7f52b80039c0 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d9a00640 1 -- 172.21.15.87:0/2233375655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b80039c0 msgr2=0x7f52b8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b80039c0 0x7f52b8003da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009fc0 tx=0x7f52c401cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d9a00640 1 -- 172.21.15.87:0/2233375655 shutdown_connections 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2233375655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b80039c0 0x7f52b8003da0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d9a00640 1 -- 172.21.15.87:0/2233375655 >> 172.21.15.87:0/2233375655 conn(0x7f52d409a9d0 msgr2=0x7f52d409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d9a00640 1 -- 172.21.15.87:0/2233375655 shutdown_connections 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.412+0000 7f52d9a00640 1 -- 172.21.15.87:0/2233375655 wait complete. 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 Processor -- start 2024-04-25T10:14:54.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 -- start start 2024-04-25T10:14:54.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b80039c0 0x7f52d4154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d4155f90 0x7f52d40cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52d40cb3a0 0x7f52d40b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40ce3c0 con 0x7f52d4155f90 2024-04-25T10:14:54.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40ce240 con 0x7f52b80039c0 2024-04-25T10:14:54.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40ce540 con 0x7f52d40cb3a0 2024-04-25T10:14:54.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d4155f90 0x7f52d40cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d4155f90 0x7f52d40cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52116/0 (socket says 172.21.15.87:52116) 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 -- 172.21.15.87:0/2918865398 learned_addr learned my addr 172.21.15.87:0/2918865398 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b80039c0 0x7f52d4154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d3400640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52d40cb3a0 0x7f52d40b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 -- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52d40cb3a0 msgr2=0x7f52d40b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52d40cb3a0 0x7f52d40b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 -- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b80039c0 msgr2=0x7f52d4154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b80039c0 0x7f52d4154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 -- 172.21.15.87:0/2918865398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d40c6890 con 0x7f52d4155f90 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52ca000640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d4155f90 0x7f52d40cae60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00f7f0 tx=0x7f52bc00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52cb400640 1 -- 172.21.15.87:0/2918865398 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0050d0 con 0x7f52d4155f90 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52cb400640 1 -- 172.21.15.87:0/2918865398 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc018070 con 0x7f52d4155f90 2024-04-25T10:14:54.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52cb400640 1 -- 172.21.15.87:0/2918865398 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00d650 con 0x7f52d4155f90 2024-04-25T10:14:54.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d40b6750 con 0x7f52d4155f90 2024-04-25T10:14:54.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.416+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d40c8c20 con 0x7f52d4155f90 2024-04-25T10:14:54.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.420+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d40cb7b0 con 0x7f52d4155f90 2024-04-25T10:14:54.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.420+0000 7f52cb400640 1 -- 172.21.15.87:0/2918865398 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52bc00d7f0 con 0x7f52d4155f90 2024-04-25T10:14:54.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.420+0000 7f52cb400640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52b0066f60 0x7f52b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.420+0000 7f52cb400640 1 -- 172.21.15.87:0/2918865398 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f52bc084e00 con 0x7f52d4155f90 2024-04-25T10:14:54.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.420+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52b0066f60 0x7f52b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.420+0000 7f52d2a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52b0066f60 0x7f52b0069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f52c40af000 tx=0x7f52c40a7040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.424+0000 7f52cb400640 1 -- 172.21.15.87:0/2918865398 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52bc055790 con 0x7f52d4155f90 2024-04-25T10:14:54.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52d4068e00 con 0x7f52d4155f90 2024-04-25T10:14:54.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52cb400640 1 -- 172.21.15.87:0/2918865398 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16722 (secure 0 0 0) 0x7f52bc059830 con 0x7f52d4155f90 2024-04-25T10:14:54.585 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:54.585 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":35,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:14:27.878686+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:14.236622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:15.422837+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52b0066f60 msgr2=0x7f52b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52b0066f60 0x7f52b0069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f52c40af000 tx=0x7f52c40a7040 comp rx=0 tx=0).stop 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d4155f90 msgr2=0x7f52d40cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d4155f90 0x7f52d40cae60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00f7f0 tx=0x7f52bc00fcb0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 shutdown_connections 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52b0066f60 0x7f52b0069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52d40cb3a0 0x7f52d40b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52d4155f90 0x7f52d40cae60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 --2- 172.21.15.87:0/2918865398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b80039c0 0x7f52d4154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 >> 172.21.15.87:0/2918865398 conn(0x7f52d409a9d0 msgr2=0x7f52d40ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 shutdown_connections 2024-04-25T10:14:54.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.580+0000 7f52d9a00640 1 -- 172.21.15.87:0/2918865398 wait complete. 2024-04-25T10:14:54.607 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:54.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.736+0000 7ff143a00640 1 Processor -- start 2024-04-25T10:14:54.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.736+0000 7ff143a00640 1 -- start start 2024-04-25T10:14:54.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff143a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 0x7ff13c0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff143a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0b1330 con 0x7ff13c072b20 2024-04-25T10:14:54.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff143a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0b1610 con 0x7ff13c0cd380 2024-04-25T10:14:54.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff143a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0cbee0 con 0x7ff13c072200 2024-04-25T10:14:54.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff13c072b20 0x7ff13c0b0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38324/0 (socket says 172.21.15.87:38324) 2024-04-25T10:14:54.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13be00640 1 -- 172.21.15.87:0/497273601 learned_addr learned my addr 172.21.15.87:0/497273601 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:54.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141000640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 0x7ff13c0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240699926 0 0) 0x7ff13c0b1330 con 0x7ff13c072b20 2024-04-25T10:14:54.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff120003530 con 0x7ff13c072b20 2024-04-25T10:14:54.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053485113 0 0) 0x7ff13c0b1610 con 0x7ff13c0cd380 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff13c0b1330 con 0x7ff13c0cd380 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319942852 0 0) 0x7ff120003530 con 0x7ff13c072b20 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 msgr2=0x7ff13c0725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 0x7ff13c0725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 >> v1:172.21.15.169:6789/0 conn(0x7ff13c0cd380 legacy=0x7ff13c165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0b18f0 con 0x7ff13c072b20 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ff128001680 con 0x7ff13c072b20 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 >> v1:172.21.15.87:6789/0 conn(0x7ff13c072b20 legacy=0x7ff13c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 0x7ff120003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1200040e0 0x7ff120004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff1200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0cbee0 con 0x7ff120004d50 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12000bab0 con 0x7ff1200040e0 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12000bcb0 con 0x7ff13c072200 2024-04-25T10:14:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141a00640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff1200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff13be00640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1200040e0 0x7ff120004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141000640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 0x7ff120003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141a00640 1 -- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 msgr2=0x7ff120003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141a00640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c072200 0x7ff120003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141a00640 1 -- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1200040e0 msgr2=0x7ff120004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141a00640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1200040e0 0x7ff120004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.740+0000 7ff141a00640 1 -- 172.21.15.87:0/497273601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120003530 con 0x7ff120004d50 2024-04-25T10:14:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff141a00640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff1200051d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0cd810 tx=0x7ff13401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce6f96a8c694d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134001c20 con 0x7ff120004d50 2024-04-25T10:14:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff134003ba0 con 0x7ff120004d50 2024-04-25T10:14:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff13b400640 1 -- 172.21.15.87:0/497273601 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1340050d0 con 0x7ff120004d50 2024-04-25T10:14:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 -- 172.21.15.87:0/497273601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 msgr2=0x7ff1200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 --2- 172.21.15.87:0/497273601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff1200051d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0cd810 tx=0x7ff13401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff141a00640 1 -- 172.21.15.87:0/497273601 reap_dead start 2024-04-25T10:14:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 -- 172.21.15.87:0/497273601 shutdown_connections 2024-04-25T10:14:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 -- 172.21.15.87:0/497273601 >> 172.21.15.87:0/497273601 conn(0x7ff13c0b1d50 msgr2=0x7ff13c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 -- 172.21.15.87:0/497273601 shutdown_connections 2024-04-25T10:14:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 -- 172.21.15.87:0/497273601 wait complete. 2024-04-25T10:14:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 Processor -- start 2024-04-25T10:14:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.744+0000 7ff143a00640 1 -- start start 2024-04-25T10:14:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff13c0c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c0aed80 0x7ff13c0af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff13c153340 0x7ff13c1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0cbc10 con 0x7ff120004d50 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0cba90 con 0x7ff13c153340 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0725e0 con 0x7ff13c0aed80 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff13c0c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff13c0c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52128/0 (socket says 172.21.15.87:52128) 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 -- 172.21.15.87:0/236231955 learned_addr learned my addr 172.21.15.87:0/236231955 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141a00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff13c153340 0x7ff13c1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff13be00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c0aed80 0x7ff13c0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 -- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c0aed80 msgr2=0x7ff13c0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c0aed80 0x7ff13c0af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 -- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff13c153340 msgr2=0x7ff13c1537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff13c153340 0x7ff13c1537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 -- 172.21.15.87:0/236231955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c1f3b50 con 0x7ff120004d50 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff141000640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff13c0c2960 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0119d0 tx=0x7ff12c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff138c00640 1 -- 172.21.15.87:0/236231955 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00fb50 con 0x7ff120004d50 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff138c00640 1 -- 172.21.15.87:0/236231955 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c00fcf0 con 0x7ff120004d50 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff138c00640 1 -- 172.21.15.87:0/236231955 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00fea0 con 0x7ff120004d50 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c1f3e30 con 0x7ff120004d50 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c1f43f0 con 0x7ff120004d50 2024-04-25T10:14:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c04ef30 con 0x7ff120004d50 2024-04-25T10:14:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.748+0000 7ff138c00640 1 -- 172.21.15.87:0/236231955 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff12c012040 con 0x7ff120004d50 2024-04-25T10:14:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.752+0000 7ff138c00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff11c066f60 0x7ff11c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.752+0000 7ff138c00640 1 -- 172.21.15.87:0/236231955 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff12c085770 con 0x7ff120004d50 2024-04-25T10:14:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.752+0000 7ff13be00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff11c066f60 0x7ff11c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.752+0000 7ff13be00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff11c066f60 0x7ff11c069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff128001060 tx=0x7ff128009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.756+0000 7ff138c00640 1 -- 172.21.15.87:0/236231955 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff12c019030 con 0x7ff120004d50 2024-04-25T10:14:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.940+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff13c0af740 con 0x7ff120004d50 2024-04-25T10:14:54.945 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:14:54.940+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:54.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.940+0000 7ff138c00640 1 -- 172.21.15.87:0/236231955 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff12c056810 con 0x7ff120004d50 2024-04-25T10:14:54.947 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff11c066f60 msgr2=0x7ff11c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff11c066f60 0x7ff11c069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff128001060 tx=0x7ff128009040 comp rx=0 tx=0).stop 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 msgr2=0x7ff13c0c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff13c0c2960 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0119d0 tx=0x7ff12c011e90 comp rx=0 tx=0).stop 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 shutdown_connections 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff11c066f60 0x7ff11c069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff13c153340 0x7ff13c1537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff13c0aed80 0x7ff13c0af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 --2- 172.21.15.87:0/236231955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff120004d50 0x7ff13c0c2960 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 >> 172.21.15.87:0/236231955 conn(0x7ff13c0b1d50 msgr2=0x7ff13c0ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 shutdown_connections 2024-04-25T10:14:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:54.944+0000 7ff143a00640 1 -- 172.21.15.87:0/236231955 wait complete. 2024-04-25T10:14:54.979 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:14:55.031 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:14:55.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.112+0000 7f156b400640 1 Processor -- start 2024-04-25T10:14:55.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.112+0000 7f156b400640 1 -- start start 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f156b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1564159540 0x7f15640ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f156b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15640aebd0 0x7f1564071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f156b400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1564071360 con 0x7f1564072020 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f156b400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1564159920 con 0x7f15640aebd0 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f156b400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156415c2f0 con 0x7f1564159540 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1563e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15640aebd0 0x7f1564071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1563e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15640aebd0 0x7f1564071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52148/0 (socket says 172.21.15.87:52148) 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1563e00640 1 -- 172.21.15.87:0/717729478 learned_addr learned my addr 172.21.15.87:0/717729478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:55.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1568a00640 1 --2- 172.21.15.87:0/717729478 >> v2:172.21.15.169:3300/0 conn(0x7f1564159540 0x7f15640ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1563e00640 1 -- 172.21.15.87:0/717729478 >> v2:172.21.15.169:3300/0 conn(0x7f1564159540 msgr2=0x7f15640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1563e00640 1 --2- 172.21.15.87:0/717729478 >> v2:172.21.15.169:3300/0 conn(0x7f1564159540 0x7f15640ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1563e00640 1 -- 172.21.15.87:0/717729478 >> v1:172.21.15.169:6789/0 conn(0x7f1564072020 legacy=0x7f156415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.116+0000 7f1563e00640 1 -- 172.21.15.87:0/717729478 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156415c5d0 con 0x7f15640aebd0 2024-04-25T10:14:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1563e00640 1 --2- 172.21.15.87:0/717729478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15640aebd0 0x7f1564071ae0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f15640b4c20 tx=0x7f155001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d74df91d239f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1563400640 1 -- 172.21.15.87:0/717729478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15500b0070 con 0x7f15640aebd0 2024-04-25T10:14:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1563400640 1 -- 172.21.15.87:0/717729478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15500ab600 con 0x7f15640aebd0 2024-04-25T10:14:55.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1563400640 1 -- 172.21.15.87:0/717729478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15500ab9d0 con 0x7f15640aebd0 2024-04-25T10:14:55.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- 172.21.15.87:0/717729478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15640aebd0 msgr2=0x7f1564071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 --2- 172.21.15.87:0/717729478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15640aebd0 0x7f1564071ae0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f15640b4c20 tx=0x7f155001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- 172.21.15.87:0/717729478 shutdown_connections 2024-04-25T10:14:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 --2- 172.21.15.87:0/717729478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15640aebd0 0x7f1564071ae0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 --2- 172.21.15.87:0/717729478 >> v2:172.21.15.169:3300/0 conn(0x7f1564159540 0x7f15640ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- 172.21.15.87:0/717729478 >> 172.21.15.87:0/717729478 conn(0x7f15640669a0 msgr2=0x7f1564066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- 172.21.15.87:0/717729478 shutdown_connections 2024-04-25T10:14:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- 172.21.15.87:0/717729478 wait complete. 2024-04-25T10:14:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 Processor -- start 2024-04-25T10:14:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- start start 2024-04-25T10:14:55.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1564072020 0x7f156414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15640aebd0 0x7f1564141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1564159540 0x7f15641f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156406c0d0 con 0x7f1564072020 2024-04-25T10:14:55.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1564072450 con 0x7f15640aebd0 2024-04-25T10:14:55.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f156b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1564168930 con 0x7f1564159540 2024-04-25T10:14:55.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1568a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1564072020 0x7f156414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1568a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1564072020 0x7f156414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52152/0 (socket says 172.21.15.87:52152) 2024-04-25T10:14:55.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1568a00640 1 -- 172.21.15.87:0/2494097109 learned_addr learned my addr 172.21.15.87:0/2494097109 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:55.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1563e00640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15640aebd0 0x7f1564141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.120+0000 7f1569400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1564159540 0x7f15641f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1568a00640 1 -- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1564159540 msgr2=0x7f15641f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1568a00640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1564159540 0x7f15641f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1568a00640 1 -- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15640aebd0 msgr2=0x7f1564141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1568a00640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15640aebd0 0x7f1564141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1568a00640 1 -- 172.21.15.87:0/2494097109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15641566d0 con 0x7f1564072020 2024-04-25T10:14:55.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1569400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1564159540 0x7f15641f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:55.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1568a00640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1564072020 0x7f156414e2b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f155400d620 tx=0x7f155400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1560c00640 1 -- 172.21.15.87:0/2494097109 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155400b620 con 0x7f1564072020 2024-04-25T10:14:55.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1560c00640 1 -- 172.21.15.87:0/2494097109 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1554019070 con 0x7f1564072020 2024-04-25T10:14:55.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1560c00640 1 -- 172.21.15.87:0/2494097109 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155400b9e0 con 0x7f1564072020 2024-04-25T10:14:55.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1564152a70 con 0x7f1564072020 2024-04-25T10:14:55.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15641562f0 con 0x7f1564072020 2024-04-25T10:14:55.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156404ef30 con 0x7f1564072020 2024-04-25T10:14:55.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1560c00640 1 -- 172.21.15.87:0/2494097109 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f155400bb80 con 0x7f1564072020 2024-04-25T10:14:55.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1560c00640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1544066f90 0x7f1544069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1560c00640 1 -- 172.21.15.87:0/2494097109 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1554085350 con 0x7f1564072020 2024-04-25T10:14:55.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.124+0000 7f1563e00640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1544066f90 0x7f1544069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.128+0000 7f1563e00640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1544066f90 0x7f1544069450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15500ad000 tx=0x7f1550008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.128+0000 7f1560c00640 1 -- 172.21.15.87:0/2494097109 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1554056390 con 0x7f1564072020 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.316+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f15640cf470 con 0x7f1564072020 2024-04-25T10:14:55.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f1560c00640 1 -- 172.21.15.87:0/2494097109 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1554055c90 con 0x7f1564072020 2024-04-25T10:14:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1544066f90 msgr2=0x7f1544069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1544066f90 0x7f1544069450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15500ad000 tx=0x7f1550008040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1564072020 msgr2=0x7f156414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1564072020 0x7f156414e2b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f155400d620 tx=0x7f155400dae0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 shutdown_connections 2024-04-25T10:14:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1544066f90 0x7f1544069450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1564159540 0x7f15641f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15640aebd0 0x7f1564141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 --2- 172.21.15.87:0/2494097109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1564072020 0x7f156414e2b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 >> 172.21.15.87:0/2494097109 conn(0x7f15640669a0 msgr2=0x7f156415d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 shutdown_connections 2024-04-25T10:14:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:55.948+0000 7f156b400640 1 -- 172.21.15.87:0/2494097109 wait complete. 2024-04-25T10:14:55.975 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:14:56.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.108+0000 7ff284400640 1 Processor -- start 2024-04-25T10:14:56.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.108+0000 7ff284400640 1 -- start start 2024-04-25T10:14:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.108+0000 7ff284400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff27c0cf040 0x7ff27c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff284400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c071ab0 0x7ff27c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff284400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27c071f80 con 0x7ff27c15bc00 2024-04-25T10:14:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff284400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c072260 con 0x7ff27c0cf040 2024-04-25T10:14:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff284400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0b3700 con 0x7ff27c071ab0 2024-04-25T10:14:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff27c0cf040 0x7ff27c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff27c0cf040 0x7ff27c0cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52158/0 (socket says 172.21.15.87:52158) 2024-04-25T10:14:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 -- 172.21.15.87:0/1646338616 learned_addr learned my addr 172.21.15.87:0/1646338616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff282400640 1 --2- 172.21.15.87:0/1646338616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c071ab0 0x7ff27c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 -- 172.21.15.87:0/1646338616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c071ab0 msgr2=0x7ff27c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 --2- 172.21.15.87:0/1646338616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c071ab0 0x7ff27c0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 -- 172.21.15.87:0/1646338616 >> v1:172.21.15.169:6789/0 conn(0x7ff27c15bc00 legacy=0x7ff27c0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 -- 172.21.15.87:0/1646338616 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c15b120 con 0x7ff27c0cf040 2024-04-25T10:14:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.112+0000 7ff281a00640 1 --2- 172.21.15.87:0/1646338616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c0cf040 0x7ff27c0cf9e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0d05d0 tx=0x7ff27001ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb78afc8d2da6ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff27be00640 1 -- 172.21.15.87:0/1646338616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700b4070 con 0x7ff27c0cf040 2024-04-25T10:14:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff27be00640 1 -- 172.21.15.87:0/1646338616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2700a7b50 con 0x7ff27c0cf040 2024-04-25T10:14:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff27be00640 1 -- 172.21.15.87:0/1646338616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700af440 con 0x7ff27c0cf040 2024-04-25T10:14:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- 172.21.15.87:0/1646338616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c0cf040 msgr2=0x7ff27c0cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 --2- 172.21.15.87:0/1646338616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c0cf040 0x7ff27c0cf9e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0d05d0 tx=0x7ff27001ebd0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- 172.21.15.87:0/1646338616 shutdown_connections 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 --2- 172.21.15.87:0/1646338616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c071ab0 0x7ff27c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 --2- 172.21.15.87:0/1646338616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c0cf040 0x7ff27c0cf9e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- 172.21.15.87:0/1646338616 >> 172.21.15.87:0/1646338616 conn(0x7ff27c066b70 msgr2=0x7ff27c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- 172.21.15.87:0/1646338616 shutdown_connections 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- 172.21.15.87:0/1646338616 wait complete. 2024-04-25T10:14:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 Processor -- start 2024-04-25T10:14:56.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- start start 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c071ab0 0x7ff27c1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c0cf040 0x7ff27c153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27c15bc00 0x7ff27c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0aec90 con 0x7ff27c071ab0 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0cf630 con 0x7ff27c15bc00 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff284400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0aee10 con 0x7ff27c0cf040 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c071ab0 0x7ff27c1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c071ab0 0x7ff27c1f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52162/0 (socket says 172.21.15.87:52162) 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 -- 172.21.15.87:0/1876029789 learned_addr learned my addr 172.21.15.87:0/1876029789 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281000640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c0cf040 0x7ff27c153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff282400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27c15bc00 0x7ff27c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 -- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c0cf040 msgr2=0x7ff27c153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c0cf040 0x7ff27c153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 -- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27c15bc00 msgr2=0x7ff27c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27c15bc00 0x7ff27c15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff281a00640 1 -- 172.21.15.87:0/1876029789 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c150be0 con 0x7ff27c071ab0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.116+0000 7ff282400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27c15bc00 0x7ff27c15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff281a00640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c071ab0 0x7ff27c1f61c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b2c0 tx=0x7ff27001b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff279600640 1 -- 172.21.15.87:0/1876029789 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700b4040 con 0x7ff27c071ab0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff279600640 1 -- 172.21.15.87:0/1876029789 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2700c1070 con 0x7ff27c071ab0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff279600640 1 -- 172.21.15.87:0/1876029789 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700b9460 con 0x7ff27c071ab0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c150ec0 con 0x7ff27c071ab0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c151510 con 0x7ff27c071ab0 2024-04-25T10:14:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27c04ef30 con 0x7ff27c071ab0 2024-04-25T10:14:56.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff279600640 1 -- 172.21.15.87:0/1876029789 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2700b7070 con 0x7ff27c071ab0 2024-04-25T10:14:56.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff279600640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff25c066f60 0x7ff25c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff279600640 1 -- 172.21.15.87:0/1876029789 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff2701221a0 con 0x7ff27c071ab0 2024-04-25T10:14:56.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff281000640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff25c066f60 0x7ff25c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.120+0000 7ff281000640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff25c066f60 0x7ff25c069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff27c15ec40 tx=0x7ff268009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.124+0000 7ff279600640 1 -- 172.21.15.87:0/1876029789 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2700187c0 con 0x7ff27c071ab0 2024-04-25T10:14:56.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:56.284+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff27c0b2a10 con 0x7ff27c071ab0 2024-04-25T10:14:57.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.252+0000 7ff279600640 1 -- 172.21.15.87:0/1876029789 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff2700f30b0 con 0x7ff27c071ab0 2024-04-25T10:14:57.258 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:14:57.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff25c066f60 msgr2=0x7ff25c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff25c066f60 0x7ff25c069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff27c15ec40 tx=0x7ff268009040 comp rx=0 tx=0).stop 2024-04-25T10:14:57.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c071ab0 msgr2=0x7ff27c1f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c071ab0 0x7ff27c1f61c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b2c0 tx=0x7ff27001b970 comp rx=0 tx=0).stop 2024-04-25T10:14:57.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 shutdown_connections 2024-04-25T10:14:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff25c066f60 0x7ff25c069420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27c15bc00 0x7ff27c15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27c0cf040 0x7ff27c153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 --2- 172.21.15.87:0/1876029789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff27c071ab0 0x7ff27c1f61c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 >> 172.21.15.87:0/1876029789 conn(0x7ff27c066b70 msgr2=0x7ff27c0713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 shutdown_connections 2024-04-25T10:14:57.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.256+0000 7ff284400640 1 -- 172.21.15.87:0/1876029789 wait complete. 2024-04-25T10:14:57.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:14:57.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.420+0000 7f0b44600640 1 Processor -- start 2024-04-25T10:14:57.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.420+0000 7f0b44600640 1 -- start start 2024-04-25T10:14:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b44600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0b3c15bd10 0x7f0b3c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b44600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0b3c0cf9e0 0x7f0b3c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b44600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c06c0e0 con 0x7f0b3c15c0f0 2024-04-25T10:14:57.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b44600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b2ef0 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b44600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0d0350 con 0x7f0b3c15bd10 2024-04-25T10:14:57.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0b3c0cf9e0 0x7f0b3c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0b3c0cf9e0 0x7f0b3c0cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53360/0 (socket says 172.21.15.87:53360) 2024-04-25T10:14:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 -- 172.21.15.87:0/3933559213 learned_addr learned my addr 172.21.15.87:0/3933559213 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b41c00640 1 --2- 172.21.15.87:0/3933559213 >> v2:172.21.15.169:3300/0 conn(0x7f0b3c15bd10 0x7f0b3c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 -- 172.21.15.87:0/3933559213 >> v2:172.21.15.169:3300/0 conn(0x7f0b3c15bd10 msgr2=0x7f0b3c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 --2- 172.21.15.87:0/3933559213 >> v2:172.21.15.169:3300/0 conn(0x7f0b3c15bd10 0x7f0b3c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 -- 172.21.15.87:0/3933559213 >> v1:172.21.15.169:6789/0 conn(0x7f0b3c15c0f0 legacy=0x7f0b3c0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 -- 172.21.15.87:0/3933559213 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c15c6a0 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b41c00640 1 --2- 172.21.15.87:0/3933559213 >> v2:172.21.15.169:3300/0 conn(0x7f0b3c15bd10 0x7f0b3c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:57.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.424+0000 7f0b42600640 1 --2- 172.21.15.87:0/3933559213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c0cfe10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c072810 tx=0x7f0b3401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df51058d0360e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b3be00640 1 -- 172.21.15.87:0/3933559213 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340aa650 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b3be00640 1 -- 172.21.15.87:0/3933559213 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b340af070 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b3be00640 1 -- 172.21.15.87:0/3933559213 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340aaa10 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- 172.21.15.87:0/3933559213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 msgr2=0x7f0b3c0cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 --2- 172.21.15.87:0/3933559213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c0cfe10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c072810 tx=0x7f0b3401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- 172.21.15.87:0/3933559213 shutdown_connections 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 --2- 172.21.15.87:0/3933559213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c0cfe10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 --2- 172.21.15.87:0/3933559213 >> v2:172.21.15.169:3300/0 conn(0x7f0b3c15bd10 0x7f0b3c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- 172.21.15.87:0/3933559213 >> 172.21.15.87:0/3933559213 conn(0x7f0b3c066b70 msgr2=0x7f0b3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- 172.21.15.87:0/3933559213 shutdown_connections 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- 172.21.15.87:0/3933559213 wait complete. 2024-04-25T10:14:57.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 Processor -- start 2024-04-25T10:14:57.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- start start 2024-04-25T10:14:57.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c15e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b3c15bd10 0x7f0b3c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b3c15c0f0 0x7f0b3c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b4cd0 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b4b50 con 0x7f0b3c15c0f0 2024-04-25T10:14:57.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b44600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c06cdd0 con 0x7f0b3c15bd10 2024-04-25T10:14:57.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b41c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b41c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c15e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53364/0 (socket says 172.21.15.87:53364) 2024-04-25T10:14:57.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b41c00640 1 -- 172.21.15.87:0/2078776392 learned_addr learned my addr 172.21.15.87:0/2078776392 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:57.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b41200640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b3c15bd10 0x7f0b3c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.428+0000 7f0b42600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b3c15c0f0 0x7f0b3c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b41c00640 1 -- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b3c15bd10 msgr2=0x7f0b3c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b41c00640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b3c15bd10 0x7f0b3c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b41c00640 1 -- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b3c15c0f0 msgr2=0x7f0b3c1526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b41c00640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b3c15c0f0 0x7f0b3c1526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b41c00640 1 -- 172.21.15.87:0/2078776392 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c158340 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b41c00640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c15e510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009fc0 tx=0x7f0b3000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b39600640 1 -- 172.21.15.87:0/2078776392 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3001a070 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b39600640 1 -- 172.21.15.87:0/2078776392 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b30003bc0 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b39600640 1 -- 172.21.15.87:0/2078776392 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30004030 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c158060 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c154aa0 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c04ef30 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b39600640 1 -- 172.21.15.87:0/2078776392 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b3001d070 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b39600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b1c066fb0 0x7f0b1c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b39600640 1 -- 172.21.15.87:0/2078776392 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f0b30085160 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.432+0000 7f0b41200640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b1c066fb0 0x7f0b1c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.436+0000 7f0b41200640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b1c066fb0 0x7f0b1c069470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c1f9ab0 tx=0x7f0b28009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.436+0000 7f0b39600640 1 -- 172.21.15.87:0/2078776392 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b30010070 con 0x7f0b3c0cf9e0 2024-04-25T10:14:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:57.592+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b3c06d9b0 con 0x7f0b3c0cf9e0 2024-04-25T10:14:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.252+0000 7f0b39600640 1 -- 172.21.15.87:0/2078776392 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0b30055c60 con 0x7f0b3c0cf9e0 2024-04-25T10:14:58.260 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b1c066fb0 msgr2=0x7f0b1c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b1c066fb0 0x7f0b1c069470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c1f9ab0 tx=0x7f0b28009040 comp rx=0 tx=0).stop 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 msgr2=0x7f0b3c15e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c15e510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009fc0 tx=0x7f0b3000d560 comp rx=0 tx=0).stop 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 shutdown_connections 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0b1c066fb0 0x7f0b1c069470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b3c15c0f0 0x7f0b3c1526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b3c15bd10 0x7f0b3c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 --2- 172.21.15.87:0/2078776392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b3c0cf9e0 0x7f0b3c15e510 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009fc0 tx=0x7f0b3000d560 comp rx=0 tx=0).stop 2024-04-25T10:14:58.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 >> 172.21.15.87:0/2078776392 conn(0x7f0b3c066b70 msgr2=0x7f0b3c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.256+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 shutdown_connections 2024-04-25T10:14:58.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.260+0000 7f0b44600640 1 -- 172.21.15.87:0/2078776392 wait complete. 2024-04-25T10:14:58.286 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:14:58.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.416+0000 7fe496c00640 1 Processor -- start 2024-04-25T10:14:58.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.416+0000 7fe496c00640 1 -- start start 2024-04-25T10:14:58.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe496c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe4900b3480 0x7fe49015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe496c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe4900cfb50 0x7fe490070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe496c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe490072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe496c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49015bb90 con 0x7fe4900b3480 2024-04-25T10:14:58.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe496c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4900ad840 con 0x7fe4900cfb50 2024-04-25T10:14:58.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe496c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49015e710 con 0x7fe490071bc0 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe4900cfb50 0x7fe490070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe4900cfb50 0x7fe490070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:60096/0 (socket says 172.21.15.87:60096) 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 -- 172.21.15.87:0/1395601774 learned_addr learned my addr 172.21.15.87:0/1395601774 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe494c00640 1 --2- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe490072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48fe00640 1 --2- 172.21.15.87:0/1395601774 >> v2:172.21.15.87:3300/0 conn(0x7fe4900b3480 0x7fe49015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 -- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 msgr2=0x7fe490072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 --2- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe490072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 -- 172.21.15.87:0/1395601774 >> v2:172.21.15.87:3300/0 conn(0x7fe4900b3480 msgr2=0x7fe49015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 --2- 172.21.15.87:0/1395601774 >> v2:172.21.15.87:3300/0 conn(0x7fe4900b3480 0x7fe49015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48f400640 1 -- 172.21.15.87:0/1395601774 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49015e9f0 con 0x7fe4900cfb50 2024-04-25T10:14:58.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe494c00640 1 --2- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe490072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:58.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.420+0000 7fe48fe00640 1 --2- 172.21.15.87:0/1395601774 >> v2:172.21.15.87:3300/0 conn(0x7fe4900b3480 0x7fe49015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.424+0000 7fe48f400640 1 --2- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900cfb50 0x7fe490070350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe4900b4c40 tx=0x7fe47c01c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b286d140981214a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe48ea00640 1 -- 172.21.15.87:0/1395601774 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0b0070 con 0x7fe4900cfb50 2024-04-25T10:14:58.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe48ea00640 1 -- 172.21.15.87:0/1395601774 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0ab600 con 0x7fe4900cfb50 2024-04-25T10:14:58.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe48ea00640 1 -- 172.21.15.87:0/1395601774 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0ab9d0 con 0x7fe4900cfb50 2024-04-25T10:14:58.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 -- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900cfb50 msgr2=0x7fe490070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 --2- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900cfb50 0x7fe490070350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe4900b4c40 tx=0x7fe47c01c520 comp rx=0 tx=0).stop 2024-04-25T10:14:58.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 -- 172.21.15.87:0/1395601774 shutdown_connections 2024-04-25T10:14:58.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 --2- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe490072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 --2- 172.21.15.87:0/1395601774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900cfb50 0x7fe490070350 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 --2- 172.21.15.87:0/1395601774 >> v2:172.21.15.87:3300/0 conn(0x7fe4900b3480 0x7fe49015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 -- 172.21.15.87:0/1395601774 >> 172.21.15.87:0/1395601774 conn(0x7fe4900669a0 msgr2=0x7fe490066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 -- 172.21.15.87:0/1395601774 shutdown_connections 2024-04-25T10:14:58.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.428+0000 7fe496c00640 1 -- 172.21.15.87:0/1395601774 wait complete. 2024-04-25T10:14:58.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 Processor -- start 2024-04-25T10:14:58.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 -- start start 2024-04-25T10:14:58.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe49014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900b3480 0x7fe49014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4900cfb50 0x7fe490153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490070a10 con 0x7fe4900cfb50 2024-04-25T10:14:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490070890 con 0x7fe4900b3480 2024-04-25T10:14:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe496c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900ae530 con 0x7fe490071bc0 2024-04-25T10:14:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe48f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900b3480 0x7fe49014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe48fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe49014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe48f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900b3480 0x7fe49014b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:60104/0 (socket says 172.21.15.87:60104) 2024-04-25T10:14:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.432+0000 7fe48f400640 1 -- 172.21.15.87:0/3838780368 learned_addr learned my addr 172.21.15.87:0/3838780368 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe48f400640 1 -- 172.21.15.87:0/3838780368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 msgr2=0x7fe49014a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe48f400640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe49014a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe48f400640 1 -- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4900cfb50 msgr2=0x7fe490153770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:14:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe48f400640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4900cfb50 0x7fe490153770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe48f400640 1 -- 172.21.15.87:0/3838780368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49014a770 con 0x7fe4900b3480 2024-04-25T10:14:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe48f400640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900b3480 0x7fe49014b260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0ae040 tx=0x7fe47c0192c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe487e00640 1 -- 172.21.15.87:0/3838780368 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0b0040 con 0x7fe4900b3480 2024-04-25T10:14:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe487e00640 1 -- 172.21.15.87:0/3838780368 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0bc070 con 0x7fe4900b3480 2024-04-25T10:14:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4900afe90 con 0x7fe4900b3480 2024-04-25T10:14:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe487e00640 1 -- 172.21.15.87:0/3838780368 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0b6620 con 0x7fe4900b3480 2024-04-25T10:14:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4900b0130 con 0x7fe4900b3480 2024-04-25T10:14:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49004ef30 con 0x7fe4900b3480 2024-04-25T10:14:58.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.436+0000 7fe487e00640 1 -- 172.21.15.87:0/3838780368 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe47c0ac030 con 0x7fe4900b3480 2024-04-25T10:14:58.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.440+0000 7fe487e00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe470066f60 0x7fe470069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.440+0000 7fe487e00640 1 -- 172.21.15.87:0/3838780368 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe47c11ee50 con 0x7fe4900b3480 2024-04-25T10:14:58.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.440+0000 7fe48fe00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe470066f60 0x7fe470069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.440+0000 7fe48fe00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe470066f60 0x7fe470069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4901543b0 tx=0x7fe480008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.444+0000 7fe487e00640 1 -- 172.21.15.87:0/3838780368 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe47c016110 con 0x7fe4900b3480 2024-04-25T10:14:58.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.600+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe49006caf0 con 0x7fe4900b3480 2024-04-25T10:14:58.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.604+0000 7fe487e00640 1 -- 172.21.15.87:0/3838780368 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe47c0f0140 con 0x7fe4900b3480 2024-04-25T10:14:58.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe470066f60 msgr2=0x7fe470069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe470066f60 0x7fe470069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4901543b0 tx=0x7fe480008040 comp rx=0 tx=0).stop 2024-04-25T10:14:58.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900b3480 msgr2=0x7fe49014b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900b3480 0x7fe49014b260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0ae040 tx=0x7fe47c0192c0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 shutdown_connections 2024-04-25T10:14:58.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe470066f60 0x7fe470069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4900cfb50 0x7fe490153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4900b3480 0x7fe49014b260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 --2- 172.21.15.87:0/3838780368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe490071bc0 0x7fe49014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 >> 172.21.15.87:0/3838780368 conn(0x7fe4900669a0 msgr2=0x7fe490070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 shutdown_connections 2024-04-25T10:14:58.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.608+0000 7fe496c00640 1 -- 172.21.15.87:0/3838780368 wait complete. 2024-04-25T10:14:58.631 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:14:58.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.760+0000 7fa823600640 1 Processor -- start 2024-04-25T10:14:58.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.760+0000 7fa823600640 1 -- start start 2024-04-25T10:14:58.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa823600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa81c1550b0 0x7fa81c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa823600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa81c15b910 0x7fa81c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa823600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c071050 con 0x7fa81c0cfe70 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa823600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c15bd80 con 0x7fa81c1550b0 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa823600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c15d040 con 0x7fa81c15b910 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa820c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa81c1550b0 0x7fa81c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa820c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa81c1550b0 0x7fa81c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53392/0 (socket says 172.21.15.87:53392) 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa820c00640 1 -- 172.21.15.87:0/1129525023 learned_addr learned my addr 172.21.15.87:0/1129525023 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa813e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa81c0cfe70 0x7fa81c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41626/0 (socket says 172.21.15.87:41626) 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa820c00640 1 -- 172.21.15.87:0/1129525023 >> v2:172.21.15.169:3300/0 conn(0x7fa81c15b910 msgr2=0x7fa81c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa820c00640 1 --2- 172.21.15.87:0/1129525023 >> v2:172.21.15.169:3300/0 conn(0x7fa81c15b910 0x7fa81c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa820c00640 1 -- 172.21.15.87:0/1129525023 >> v1:172.21.15.87:6789/0 conn(0x7fa81c0cfe70 legacy=0x7fa81c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.764+0000 7fa820c00640 1 -- 172.21.15.87:0/1129525023 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0712f0 con 0x7fa81c1550b0 2024-04-25T10:14:58.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa820c00640 1 --2- 172.21.15.87:0/1129525023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0cf930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009870 tx=0x7fa80c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6ed7d3e67c9f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa81be00640 1 -- 172.21.15.87:0/1129525023 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0ab610 con 0x7fa81c1550b0 2024-04-25T10:14:58.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa81be00640 1 -- 172.21.15.87:0/1129525023 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c0b0070 con 0x7fa81c1550b0 2024-04-25T10:14:58.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa81be00640 1 -- 172.21.15.87:0/1129525023 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0ab9d0 con 0x7fa81c1550b0 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 -- 172.21.15.87:0/1129525023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 msgr2=0x7fa81c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 --2- 172.21.15.87:0/1129525023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0cf930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009870 tx=0x7fa80c01c520 comp rx=0 tx=0).stop 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 -- 172.21.15.87:0/1129525023 shutdown_connections 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 --2- 172.21.15.87:0/1129525023 >> v2:172.21.15.169:3300/0 conn(0x7fa81c15b910 0x7fa81c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 --2- 172.21.15.87:0/1129525023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0cf930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 -- 172.21.15.87:0/1129525023 >> 172.21.15.87:0/1129525023 conn(0x7fa81c066b70 msgr2=0x7fa81c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 -- 172.21.15.87:0/1129525023 shutdown_connections 2024-04-25T10:14:58.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 -- 172.21.15.87:0/1129525023 wait complete. 2024-04-25T10:14:58.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 Processor -- start 2024-04-25T10:14:58.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 -- start start 2024-04-25T10:14:58.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.768+0000 7fa823600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c0cfe70 0x7fa81c13c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c15b910 0x7fa81c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c15c030 con 0x7fa81c1550b0 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0720b0 con 0x7fa81c0cfe70 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c15c1b0 con 0x7fa81c15b910 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53402/0 (socket says 172.21.15.87:53402) 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 -- 172.21.15.87:0/853915490 learned_addr learned my addr 172.21.15.87:0/853915490 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa820c00640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c0cfe70 0x7fa81c13c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa821600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c15b910 0x7fa81c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 -- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c15b910 msgr2=0x7fa81c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c15b910 0x7fa81c0b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 -- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c0cfe70 msgr2=0x7fa81c13c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c0cfe70 0x7fa81c13c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 -- 172.21.15.87:0/853915490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c1f7e20 con 0x7fa81c1550b0 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa820c00640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c0cfe70 0x7fa81c13c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa813e00640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0b0660 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c420 tx=0x7fa80400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa819600640 1 -- 172.21.15.87:0/853915490 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8040074e0 con 0x7fa81c1550b0 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa819600640 1 -- 172.21.15.87:0/853915490 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa804007680 con 0x7fa81c1550b0 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa819600640 1 -- 172.21.15.87:0/853915490 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804004a30 con 0x7fa81c1550b0 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c159ca0 con 0x7fa81c1550b0 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c15a260 con 0x7fa81c1550b0 2024-04-25T10:14:58.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c04ef30 con 0x7fa81c1550b0 2024-04-25T10:14:58.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa819600640 1 -- 172.21.15.87:0/853915490 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa80401a020 con 0x7fa81c1550b0 2024-04-25T10:14:58.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.772+0000 7fa819600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7f8066f60 0x7fa7f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.776+0000 7fa819600640 1 -- 172.21.15.87:0/853915490 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa8040846d0 con 0x7fa81c1550b0 2024-04-25T10:14:58.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.776+0000 7fa820c00640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7f8066f60 0x7fa7f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.776+0000 7fa820c00640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7f8066f60 0x7fa7f8069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0af000 tx=0x7fa80c0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.780+0000 7fa819600640 1 -- 172.21.15.87:0/853915490 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8040558c0 con 0x7fa81c1550b0 2024-04-25T10:14:58.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:58.984+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fa81c153ae0 con 0x7fa81c1550b0 2024-04-25T10:14:59.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.264+0000 7fa819600640 1 -- 172.21.15.87:0/853915490 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v142) v1 ==== 148+0+0 (secure 0 0 0) 0x7fa8040551c0 con 0x7fa81c1550b0 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7f8066f60 msgr2=0x7fa7f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7f8066f60 0x7fa7f8069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0af000 tx=0x7fa80c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 msgr2=0x7fa81c0b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0b0660 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c420 tx=0x7fa80400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 shutdown_connections 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7f8066f60 0x7fa7f8069420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c15b910 0x7fa81c0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa81c1550b0 0x7fa81c0b0660 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 --2- 172.21.15.87:0/853915490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c0cfe70 0x7fa81c13c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 >> 172.21.15.87:0/853915490 conn(0x7fa81c066b70 msgr2=0x7fa81c154000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 shutdown_connections 2024-04-25T10:14:59.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.268+0000 7fa823600640 1 -- 172.21.15.87:0/853915490 wait complete. 2024-04-25T10:14:59.293 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T10:14:59.293 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:14:59.294 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:59.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ee00640 1 Processor -- start 2024-04-25T10:14:59.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ee00640 1 -- start start 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f47680d0040 0x7f476815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ee00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f476815b740 0x7f47680b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ee00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4768071050 con 0x7f47681550b0 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ee00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476815d040 con 0x7f476815b740 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47680712f0 con 0x7f47680d0040 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f476ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f476815b740 0x7f47680b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f4767e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f47681550b0 0x7f47680cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41630/0 (socket says 172.21.15.87:41630) 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.424+0000 7f4767e00640 1 -- 172.21.15.87:0/2830085091 learned_addr learned my addr 172.21.15.87:0/2830085091 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f475f400640 1 --2- 172.21.15.87:0/2830085091 >> v2:172.21.15.169:3300/0 conn(0x7f47680d0040 0x7f476815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f476ce00640 1 -- 172.21.15.87:0/2830085091 >> v2:172.21.15.169:3300/0 conn(0x7f47680d0040 msgr2=0x7f476815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f476ce00640 1 --2- 172.21.15.87:0/2830085091 >> v2:172.21.15.169:3300/0 conn(0x7f47680d0040 0x7f476815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f476ce00640 1 -- 172.21.15.87:0/2830085091 >> v1:172.21.15.87:6789/0 conn(0x7f47681550b0 legacy=0x7f47680cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f476ce00640 1 -- 172.21.15.87:0/2830085091 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680715d0 con 0x7f476815b740 2024-04-25T10:14:59.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f475f400640 1 --2- 172.21.15.87:0/2830085091 >> v2:172.21.15.169:3300/0 conn(0x7f47680d0040 0x7f476815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:59.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f4767400640 1 -- 172.21.15.87:0/2830085091 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368735773 0 0) 0x7f4768071050 con 0x7f47681550b0 2024-04-25T10:14:59.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f476ce00640 1 --2- 172.21.15.87:0/2830085091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476815b740 0x7f47680b3260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4768070160 tx=0x7f476001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d023318e2c8df39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f4767400640 1 -- 172.21.15.87:0/2830085091 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600b2070 con 0x7f476815b740 2024-04-25T10:14:59.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f4767400640 1 -- 172.21.15.87:0/2830085091 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600a59c0 con 0x7f476815b740 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.428+0000 7f4767400640 1 -- 172.21.15.87:0/2830085091 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600a5d90 con 0x7f476815b740 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- 172.21.15.87:0/2830085091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476815b740 msgr2=0x7f47680b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 --2- 172.21.15.87:0/2830085091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476815b740 0x7f47680b3260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4768070160 tx=0x7f476001c900 comp rx=0 tx=0).stop 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- 172.21.15.87:0/2830085091 shutdown_connections 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 --2- 172.21.15.87:0/2830085091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476815b740 0x7f47680b3260 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 --2- 172.21.15.87:0/2830085091 >> v2:172.21.15.169:3300/0 conn(0x7f47680d0040 0x7f476815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- 172.21.15.87:0/2830085091 >> 172.21.15.87:0/2830085091 conn(0x7f4768066b70 msgr2=0x7f4768066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- 172.21.15.87:0/2830085091 shutdown_connections 2024-04-25T10:14:59.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- 172.21.15.87:0/2830085091 wait complete. 2024-04-25T10:14:59.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 Processor -- start 2024-04-25T10:14:59.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- start start 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47680d0040 0x7f4768153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47681550b0 0x7f4768154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476815b740 0x7f47681526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476815bd10 con 0x7f47680d0040 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476815bb90 con 0x7f476815b740 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f476ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476815c030 con 0x7f47681550b0 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47680d0040 0x7f4768153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47680d0040 0x7f4768153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53414/0 (socket says 172.21.15.87:53414) 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 -- 172.21.15.87:0/1530632697 learned_addr learned my addr 172.21.15.87:0/1530632697 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 -- 172.21.15.87:0/1530632697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47681550b0 msgr2=0x7f4768154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47681550b0 0x7f4768154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 -- 172.21.15.87:0/1530632697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476815b740 msgr2=0x7f47681526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476815b740 0x7f47681526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 -- 172.21.15.87:0/1530632697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47681409d0 con 0x7f47680d0040 2024-04-25T10:14:59.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.432+0000 7f4767e00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47680d0040 0x7f4768153d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f47580011f0 tx=0x7f475800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f4764c00640 1 -- 172.21.15.87:0/1530632697 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758018070 con 0x7f47680d0040 2024-04-25T10:14:59.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f4764c00640 1 -- 172.21.15.87:0/1530632697 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580074e0 con 0x7f47680d0040 2024-04-25T10:14:59.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47681574d0 con 0x7f47680d0040 2024-04-25T10:14:59.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f4764c00640 1 -- 172.21.15.87:0/1530632697 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758004a10 con 0x7f47680d0040 2024-04-25T10:14:59.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768157a90 con 0x7f47680d0040 2024-04-25T10:14:59.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476804ef30 con 0x7f47680d0040 2024-04-25T10:14:59.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f4764c00640 1 -- 172.21.15.87:0/1530632697 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f475801b070 con 0x7f47680d0040 2024-04-25T10:14:59.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f4764c00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f90 0x7f4744069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f4764c00640 1 -- 172.21.15.87:0/1530632697 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f47580840a0 con 0x7f47680d0040 2024-04-25T10:14:59.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f475f400640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f90 0x7f4744069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.436+0000 7f475f400640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f90 0x7f4744069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4750006e90 tx=0x7f4750008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.440+0000 7f4764c00640 1 -- 172.21.15.87:0/1530632697 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47580553b0 con 0x7f47680d0040 2024-04-25T10:14:59.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.604+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f476815dbd0 con 0x7f47680d0040 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.604+0000 7f4764c00640 1 -- 172.21.15.87:0/1530632697 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13008 (secure 0 0 0) 0x7f4758054cb0 con 0x7f47680d0040 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":38,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:14:58.255192+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6835/4156145333":"2024-04-26T10:14:54.943068+0000","172.21.15.87:6834/4156145333":"2024-04-26T10:14:54.943068+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:59.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f90 msgr2=0x7f4744069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f90 0x7f4744069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4750006e90 tx=0x7f4750008040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47680d0040 msgr2=0x7f4768153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47680d0040 0x7f4768153d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f47580011f0 tx=0x7f475800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 shutdown_connections 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f90 0x7f4744069450 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476815b740 0x7f47681526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47681550b0 0x7f4768154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 --2- 172.21.15.87:0/1530632697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47680d0040 0x7f4768153d20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 >> 172.21.15.87:0/1530632697 conn(0x7f4768066b70 msgr2=0x7f47680cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 shutdown_connections 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.608+0000 7f476ee00640 1 -- 172.21.15.87:0/1530632697 wait complete. 2024-04-25T10:14:59.632 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:59.632 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a61e00640 1 Processor -- start 2024-04-25T10:14:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a61e00640 1 -- start start 2024-04-25T10:14:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a61e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a5c0aef80 0x7f9a5c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a61e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c070350 0x7f9a5c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a61e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5c15b080 con 0x7f9a5c15bac0 2024-04-25T10:14:59.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a61e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c15adb0 con 0x7f9a5c0aef80 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a61e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c15eb60 con 0x7f9a5c070350 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a5aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a5c0aef80 0x7f9a5c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a52000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c070350 0x7f9a5c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a5aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a5c0aef80 0x7f9a5c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:60160/0 (socket says 172.21.15.87:60160) 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a5aa00640 1 -- 172.21.15.87:0/1180196614 learned_addr learned my addr 172.21.15.87:0/1180196614 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a5aa00640 1 -- 172.21.15.87:0/1180196614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c070350 msgr2=0x7f9a5c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a5aa00640 1 --2- 172.21.15.87:0/1180196614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c070350 0x7f9a5c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a5aa00640 1 -- 172.21.15.87:0/1180196614 >> v1:172.21.15.169:6789/0 conn(0x7f9a5c15bac0 legacy=0x7f9a5c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.776+0000 7f9a5aa00640 1 -- 172.21.15.87:0/1180196614 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c15ee40 con 0x7f9a5c0aef80 2024-04-25T10:14:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a5aa00640 1 --2- 172.21.15.87:0/1180196614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c15e480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009870 tx=0x7f9a4c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7379f8974b7920fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a5a000640 1 -- 172.21.15.87:0/1180196614 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0a9400 con 0x7f9a5c0aef80 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a5a000640 1 -- 172.21.15.87:0/1180196614 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0a95a0 con 0x7f9a5c0aef80 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a5a000640 1 -- 172.21.15.87:0/1180196614 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0a9970 con 0x7f9a5c0aef80 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 -- 172.21.15.87:0/1180196614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c0aef80 msgr2=0x7f9a5c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 --2- 172.21.15.87:0/1180196614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c15e480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009870 tx=0x7f9a4c01c560 comp rx=0 tx=0).stop 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 -- 172.21.15.87:0/1180196614 shutdown_connections 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 --2- 172.21.15.87:0/1180196614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c070350 0x7f9a5c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 --2- 172.21.15.87:0/1180196614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c15e480 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 -- 172.21.15.87:0/1180196614 >> 172.21.15.87:0/1180196614 conn(0x7f9a5c0669a0 msgr2=0x7f9a5c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 -- 172.21.15.87:0/1180196614 shutdown_connections 2024-04-25T10:14:59.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.780+0000 7f9a61e00640 1 -- 172.21.15.87:0/1180196614 wait complete. 2024-04-25T10:14:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 Processor -- start 2024-04-25T10:14:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 -- start start 2024-04-25T10:14:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c070350 0x7f9a5c168270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c15bac0 0x7f9a5c152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c165990 con 0x7f9a5c0aef80 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c165810 con 0x7f9a5c070350 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c165b10 con 0x7f9a5c15bac0 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53428/0 (socket says 172.21.15.87:53428) 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 -- 172.21.15.87:0/2665725848 learned_addr learned my addr 172.21.15.87:0/2665725848 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a5aa00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c070350 0x7f9a5c168270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a5b400640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c15bac0 0x7f9a5c152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 -- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c15bac0 msgr2=0x7f9a5c152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c15bac0 0x7f9a5c152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 -- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c070350 msgr2=0x7f9a5c168270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c070350 0x7f9a5c168270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 -- 172.21.15.87:0/2665725848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c15a620 con 0x7f9a5c0aef80 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a52000640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c151430 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e930 tx=0x7f9a4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a53e00640 1 -- 172.21.15.87:0/2665725848 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44017070 con 0x7f9a5c0aef80 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a53e00640 1 -- 172.21.15.87:0/2665725848 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4400ca00 con 0x7f9a5c0aef80 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a53e00640 1 -- 172.21.15.87:0/2665725848 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400cdb0 con 0x7f9a5c0aef80 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c158ea0 con 0x7f9a5c0aef80 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.784+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c159460 con 0x7f9a5c0aef80 2024-04-25T10:14:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.788+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c04ef30 con 0x7f9a5c0aef80 2024-04-25T10:14:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.788+0000 7f9a53e00640 1 -- 172.21.15.87:0/2665725848 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a4401a070 con 0x7f9a5c0aef80 2024-04-25T10:14:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.788+0000 7f9a53e00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9a38066f60 0x7f9a38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.788+0000 7f9a53e00640 1 -- 172.21.15.87:0/2665725848 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a44085090 con 0x7f9a5c0aef80 2024-04-25T10:14:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.788+0000 7f9a5aa00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9a38066f60 0x7f9a38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.788+0000 7f9a5aa00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9a38066f60 0x7f9a38069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0ae000 tx=0x7f9a4c0b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.792+0000 7f9a53e00640 1 -- 172.21.15.87:0/2665725848 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a44055e00 con 0x7f9a5c0aef80 2024-04-25T10:14:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.944+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a5c156470 con 0x7f9a5c0aef80 2024-04-25T10:14:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.944+0000 7f9a53e00640 1 -- 172.21.15.87:0/2665725848 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a44059ea0 con 0x7f9a5c0aef80 2024-04-25T10:14:59.951 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:14:59.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9a38066f60 msgr2=0x7f9a38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9a38066f60 0x7f9a38069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0ae000 tx=0x7f9a4c0b3040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5c0aef80 msgr2=0x7f9a5c151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c151430 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e930 tx=0x7f9a4400edf0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 shutdown_connections 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9a38066f60 0x7f9a38069420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5c15bac0 0x7f9a5c152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5c0aef80 0x7f9a5c151430 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 --2- 172.21.15.87:0/2665725848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a5c070350 0x7f9a5c168270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 >> 172.21.15.87:0/2665725848 conn(0x7f9a5c0669a0 msgr2=0x7f9a5c15d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 shutdown_connections 2024-04-25T10:14:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:14:59.948+0000 7f9a61e00640 1 -- 172.21.15.87:0/2665725848 wait complete. 2024-04-25T10:14:59.972 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:15:00.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.104+0000 7f937e800640 1 Processor -- start 2024-04-25T10:15:00.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.104+0000 7f937e800640 1 -- start start 2024-04-25T10:15:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.104+0000 7f937e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f937815b9e0 0x7f93780cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.104+0000 7f937e800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93780ccfb0 con 0x7f937815bdc0 2024-04-25T10:15:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.104+0000 7f937e800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93780cd290 con 0x7f93780d1c30 2024-04-25T10:15:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.104+0000 7f937e800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93780cd570 con 0x7f937815b9e0 2024-04-25T10:15:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.104+0000 7f9377400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f937815b9e0 0x7f93780cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9377400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f937815b9e0 0x7f93780cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53432/0 (socket says 172.21.15.87:53432) 2024-04-25T10:15:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9377400640 1 -- 172.21.15.87:0/3184945364 learned_addr learned my addr 172.21.15.87:0/3184945364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9377400640 1 -- 172.21.15.87:0/3184945364 >> v1:172.21.15.169:6789/0 conn(0x7f93780d1c30 legacy=0x7f93780c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9377400640 1 -- 172.21.15.87:0/3184945364 >> v1:172.21.15.87:6789/0 conn(0x7f937815bdc0 legacy=0x7f93780d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9377400640 1 -- 172.21.15.87:0/3184945364 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780cd850 con 0x7f937815b9e0 2024-04-25T10:15:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9377400640 1 --2- 172.21.15.87:0/3184945364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815b9e0 0x7f93780cc410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9368009870 tx=0x7f936801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e708f15c5ff38d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9376000640 1 -- 172.21.15.87:0/3184945364 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680b0070 con 0x7f937815b9e0 2024-04-25T10:15:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9376000640 1 -- 172.21.15.87:0/3184945364 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93680ab600 con 0x7f937815b9e0 2024-04-25T10:15:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f9376000640 1 -- 172.21.15.87:0/3184945364 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680ab9d0 con 0x7f937815b9e0 2024-04-25T10:15:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f937e800640 1 -- 172.21.15.87:0/3184945364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815b9e0 msgr2=0x7f93780cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.108+0000 7f937e800640 1 --2- 172.21.15.87:0/3184945364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815b9e0 0x7f93780cc410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9368009870 tx=0x7f936801c490 comp rx=0 tx=0).stop 2024-04-25T10:15:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- 172.21.15.87:0/3184945364 shutdown_connections 2024-04-25T10:15:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 --2- 172.21.15.87:0/3184945364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815b9e0 0x7f93780cc410 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- 172.21.15.87:0/3184945364 >> 172.21.15.87:0/3184945364 conn(0x7f937809a9d0 msgr2=0x7f937809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- 172.21.15.87:0/3184945364 shutdown_connections 2024-04-25T10:15:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- 172.21.15.87:0/3184945364 wait complete. 2024-04-25T10:15:00.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 Processor -- start 2024-04-25T10:15:00.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- start start 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93780d1c30 0x7f93781546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f937815b9e0 0x7f93780cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815bdc0 0x7f93780cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93780c6d40 con 0x7f937815bdc0 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93780c6bc0 con 0x7f93780d1c30 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f937e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9378169840 con 0x7f937815b9e0 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815bdc0 0x7f93780cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815bdc0 0x7f93780cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53446/0 (socket says 172.21.15.87:53446) 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 -- 172.21.15.87:0/1443069105 learned_addr learned my addr 172.21.15.87:0/1443069105 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9376a00640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f937815b9e0 0x7f93780cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377400640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93780d1c30 0x7f93781546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 -- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f937815b9e0 msgr2=0x7f93780cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f937815b9e0 0x7f93780cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 -- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93780d1c30 msgr2=0x7f93781546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93780d1c30 0x7f93781546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 -- 172.21.15.87:0/1443069105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780c9700 con 0x7f937815bdc0 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9376a00640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f937815b9e0 0x7f93780cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.112+0000 7f9377e00640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815bdc0 0x7f93780cb590 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f937000b9c0 tx=0x7f937000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f936f400640 1 -- 172.21.15.87:0/1443069105 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370019070 con 0x7f937815bdc0 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f936f400640 1 -- 172.21.15.87:0/1443069105 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937000cba0 con 0x7f937815bdc0 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f936f400640 1 -- 172.21.15.87:0/1443069105 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370014440 con 0x7f937815bdc0 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93780c99e0 con 0x7f937815bdc0 2024-04-25T10:15:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93780c9fa0 con 0x7f937815bdc0 2024-04-25T10:15:00.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93780c7de0 con 0x7f937815bdc0 2024-04-25T10:15:00.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f936f400640 1 -- 172.21.15.87:0/1443069105 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f937001c070 con 0x7f937815bdc0 2024-04-25T10:15:00.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f936f400640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f935806b370 0x7f935806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f936f400640 1 -- 172.21.15.87:0/1443069105 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9370084a80 con 0x7f937815bdc0 2024-04-25T10:15:00.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f9377400640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f935806b370 0x7f935806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.116+0000 7f9377400640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f935806b370 0x7f935806d830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f93680ad000 tx=0x7f93680a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.120+0000 7f936f400640 1 -- 172.21.15.87:0/1443069105 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9370086e20 con 0x7f937815bdc0 2024-04-25T10:15:00.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:00.352+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f937806d4d0 con 0x7f937815bdc0 2024-04-25T10:15:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.272+0000 7f936f400640 1 -- 172.21.15.87:0/1443069105 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v144) v1 ==== 146+0+0 (secure 0 0 0) 0x7f93700556a0 con 0x7f937815bdc0 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f935806b370 msgr2=0x7f935806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f935806b370 0x7f935806d830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f93680ad000 tx=0x7f93680a6000 comp rx=0 tx=0).stop 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815bdc0 msgr2=0x7f93780cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815bdc0 0x7f93780cb590 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f937000b9c0 tx=0x7f937000be80 comp rx=0 tx=0).stop 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 shutdown_connections 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f935806b370 0x7f935806d830 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f937815bdc0 0x7f93780cb590 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f937815b9e0 0x7f93780cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 --2- 172.21.15.87:0/1443069105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93780d1c30 0x7f93781546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 >> 172.21.15.87:0/1443069105 conn(0x7f937809a9d0 msgr2=0x7f93780d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 shutdown_connections 2024-04-25T10:15:01.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.276+0000 7f937e800640 1 -- 172.21.15.87:0/1443069105 wait complete. 2024-04-25T10:15:01.304 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:15:01.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.436+0000 7fd592400640 1 Processor -- start 2024-04-25T10:15:01.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd592400640 1 -- start start 2024-04-25T10:15:01.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd592400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c15b150 0x7fd58c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd592400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd58c06c0d0 0x7fd58c06c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd592400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58c0b2fd0 con 0x7fd58c15a0e0 2024-04-25T10:15:01.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd592400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c0aef90 con 0x7fd58c06c0d0 2024-04-25T10:15:01.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd592400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0af1d0 con 0x7fd58c15b150 2024-04-25T10:15:01.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd58c15a0e0 0x7fd58c0b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41656/0 (socket says 172.21.15.87:41656) 2024-04-25T10:15:01.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 -- 172.21.15.87:0/3679166667 learned_addr learned my addr 172.21.15.87:0/3679166667 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:01.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58be00640 1 --2- 172.21.15.87:0/3679166667 >> v2:172.21.15.169:3300/0 conn(0x7fd58c06c0d0 0x7fd58c06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd582a00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c15b150 0x7fd58c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776273773 0 0) 0x7fd58c0b2fd0 con 0x7fd58c15a0e0 2024-04-25T10:15:01.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd570003530 con 0x7fd58c15a0e0 2024-04-25T10:15:01.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1754066429 0 0) 0x7fd570003530 con 0x7fd58c15a0e0 2024-04-25T10:15:01.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c15b150 msgr2=0x7fd58c0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c15b150 0x7fd58c0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 >> v2:172.21.15.169:3300/0 conn(0x7fd58c06c0d0 msgr2=0x7fd58c06c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 --2- 172.21.15.87:0/3679166667 >> v2:172.21.15.169:3300/0 conn(0x7fd58c06c0d0 0x7fd58c06c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0b32b0 con 0x7fd58c15a0e0 2024-04-25T10:15:01.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd57c001680 con 0x7fd58c15a0e0 2024-04-25T10:15:01.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 >> v1:172.21.15.87:6789/0 conn(0x7fd58c15a0e0 legacy=0x7fd58c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5700039c0 0x7fd570003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd570004810 0x7fd570004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700051d0 0x7fd57000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0aef90 con 0x7fd5700039c0 2024-04-25T10:15:01.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0af1d0 con 0x7fd5700051d0 2024-04-25T10:15:01.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57000bf60 con 0x7fd570004810 2024-04-25T10:15:01.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58be00640 1 --2- 172.21.15.87:0/3679166667 >> v2:172.21.15.169:3300/0 conn(0x7fd58c06c0d0 0x7fd58c06c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:01.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd582a00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c15b150 0x7fd58c0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:01.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5700039c0 0x7fd570003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58be00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700051d0 0x7fd57000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd582a00640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd570004810 0x7fd570004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 -- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd570004810 msgr2=0x7fd570004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd570004810 0x7fd570004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 -- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700051d0 msgr2=0x7fd57000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700051d0 0x7fd57000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.440+0000 7fd58b400640 1 -- 172.21.15.87:0/3679166667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570003530 con 0x7fd5700039c0 2024-04-25T10:15:01.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd58b400640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5700039c0 0x7fd570003de0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00c270 tx=0x7fd57c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91814af238bbe0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c007d60 con 0x7fd5700039c0 2024-04-25T10:15:01.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c0a9020 con 0x7fd5700039c0 2024-04-25T10:15:01.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd58aa00640 1 -- 172.21.15.87:0/3679166667 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c005ae0 con 0x7fd5700039c0 2024-04-25T10:15:01.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd592400640 1 -- 172.21.15.87:0/3679166667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5700039c0 msgr2=0x7fd570003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd592400640 1 --2- 172.21.15.87:0/3679166667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5700039c0 0x7fd570003de0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00c270 tx=0x7fd57c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd58be00640 1 -- 172.21.15.87:0/3679166667 reap_dead start 2024-04-25T10:15:01.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd592400640 1 -- 172.21.15.87:0/3679166667 shutdown_connections 2024-04-25T10:15:01.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd592400640 1 -- 172.21.15.87:0/3679166667 >> 172.21.15.87:0/3679166667 conn(0x7fd58c0669a0 msgr2=0x7fd58c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd592400640 1 -- 172.21.15.87:0/3679166667 shutdown_connections 2024-04-25T10:15:01.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.444+0000 7fd592400640 1 -- 172.21.15.87:0/3679166667 wait complete. 2024-04-25T10:15:01.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 Processor -- start 2024-04-25T10:15:01.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 -- start start 2024-04-25T10:15:01.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700039c0 0x7fd58c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c14fdd0 0x7fd58c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd58c150250 0x7fd58c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0b4620 con 0x7fd58c150250 2024-04-25T10:15:01.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0b44a0 con 0x7fd5700039c0 2024-04-25T10:15:01.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0705f0 con 0x7fd58c14fdd0 2024-04-25T10:15:01.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd58c150250 0x7fd58c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd58c150250 0x7fd58c158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53456/0 (socket says 172.21.15.87:53456) 2024-04-25T10:15:01.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 -- 172.21.15.87:0/1060173365 learned_addr learned my addr 172.21.15.87:0/1060173365 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:01.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58b400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700039c0 0x7fd58c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd582a00640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c14fdd0 0x7fd58c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 -- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c14fdd0 msgr2=0x7fd58c1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c14fdd0 0x7fd58c1585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 -- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700039c0 msgr2=0x7fd58c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700039c0 0x7fd58c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 -- 172.21.15.87:0/1060173365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c1592d0 con 0x7fd58c150250 2024-04-25T10:15:01.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd58be00640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd58c150250 0x7fd58c158b30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0cc120 tx=0x7fd584009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd583e00640 1 -- 172.21.15.87:0/1060173365 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584011e90 con 0x7fd58c150250 2024-04-25T10:15:01.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd583e00640 1 -- 172.21.15.87:0/1060173365 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58401b070 con 0x7fd58c150250 2024-04-25T10:15:01.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd583e00640 1 -- 172.21.15.87:0/1060173365 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5840039a0 con 0x7fd58c150250 2024-04-25T10:15:01.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c0b0e80 con 0x7fd58c150250 2024-04-25T10:15:01.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.448+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c0b1440 con 0x7fd58c150250 2024-04-25T10:15:01.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.452+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c04ef30 con 0x7fd58c150250 2024-04-25T10:15:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.452+0000 7fd583e00640 1 -- 172.21.15.87:0/1060173365 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd584003b40 con 0x7fd58c150250 2024-04-25T10:15:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.452+0000 7fd583e00640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd568067130 0x7fd5680695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.452+0000 7fd583e00640 1 -- 172.21.15.87:0/1060173365 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd5840855f0 con 0x7fd58c150250 2024-04-25T10:15:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.452+0000 7fd58b400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd568067130 0x7fd5680695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.452+0000 7fd58b400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd568067130 0x7fd5680695f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0aa000 tx=0x7fd57c001ac0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.456+0000 7fd583e00640 1 -- 172.21.15.87:0/1060173365 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd584025020 con 0x7fd58c150250 2024-04-25T10:15:01.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.612+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd58c06e1f0 con 0x7fd58c150250 2024-04-25T10:15:01.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd583e00640 1 -- 172.21.15.87:0/1060173365 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd584056900 con 0x7fd58c150250 2024-04-25T10:15:01.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd568067130 msgr2=0x7fd5680695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd568067130 0x7fd5680695f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0aa000 tx=0x7fd57c001ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd58c150250 msgr2=0x7fd58c158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd58c150250 0x7fd58c158b30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0cc120 tx=0x7fd584009f90 comp rx=0 tx=0).stop 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 shutdown_connections 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd568067130 0x7fd5680695f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd58c150250 0x7fd58c158b30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd58c14fdd0 0x7fd58c1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 --2- 172.21.15.87:0/1060173365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700039c0 0x7fd58c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 >> 172.21.15.87:0/1060173365 conn(0x7fd58c0669a0 msgr2=0x7fd58c06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 shutdown_connections 2024-04-25T10:15:01.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:01.616+0000 7fd592400640 1 -- 172.21.15.87:0/1060173365 wait complete. 2024-04-25T10:15:01.641 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:01.641 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:01.734 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:01.832 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:01.832 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:01.832 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:01.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:01.843 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:15:01.843 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:01.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:01.853 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:01.853 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:01.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:01.899 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:15:01.899 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:01.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:01.908 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:15:01.908 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a3400640 1 Processor -- start 2024-04-25T10:15:02.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a3400640 1 -- start start 2024-04-25T10:15:02.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f689c0cda10 0x7f689c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f689c0cf930 0x7f689c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a3400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c0b2770 con 0x7f689c15bc40 2024-04-25T10:15:02.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a3400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689c06c670 con 0x7f689c0cda10 2024-04-25T10:15:02.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a3400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689c06c950 con 0x7f689c0cf930 2024-04-25T10:15:02.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f6893e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f689c0cda10 0x7f689c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a0a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f689c15bc40 0x7f689c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41658/0 (socket says 172.21.15.87:41658) 2024-04-25T10:15:02.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f68a0a00640 1 -- 172.21.15.87:0/3640906603 learned_addr learned my addr 172.21.15.87:0/3640906603 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:02.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f6893e00640 1 -- 172.21.15.87:0/3640906603 >> v2:172.21.15.169:3300/0 conn(0x7f689c0cf930 msgr2=0x7f689c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f6893e00640 1 --2- 172.21.15.87:0/3640906603 >> v2:172.21.15.169:3300/0 conn(0x7f689c0cf930 0x7f689c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f6893e00640 1 -- 172.21.15.87:0/3640906603 >> v1:172.21.15.87:6789/0 conn(0x7f689c15bc40 legacy=0x7f689c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.072+0000 7f6893e00640 1 -- 172.21.15.87:0/3640906603 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c06cc30 con 0x7f689c0cda10 2024-04-25T10:15:02.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f6893e00640 1 --2- 172.21.15.87:0/3640906603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c06c090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ae4c0 tx=0x7f688401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212d0f47a5d8a67f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f689be00640 1 -- 172.21.15.87:0/3640906603 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68840b0070 con 0x7f689c0cda10 2024-04-25T10:15:02.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f689be00640 1 -- 172.21.15.87:0/3640906603 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68840ab600 con 0x7f689c0cda10 2024-04-25T10:15:02.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f689be00640 1 -- 172.21.15.87:0/3640906603 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68840ab9d0 con 0x7f689c0cda10 2024-04-25T10:15:02.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 -- 172.21.15.87:0/3640906603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 msgr2=0x7f689c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 --2- 172.21.15.87:0/3640906603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c06c090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ae4c0 tx=0x7f688401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 -- 172.21.15.87:0/3640906603 shutdown_connections 2024-04-25T10:15:02.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 --2- 172.21.15.87:0/3640906603 >> v2:172.21.15.169:3300/0 conn(0x7f689c0cf930 0x7f689c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 --2- 172.21.15.87:0/3640906603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c06c090 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 -- 172.21.15.87:0/3640906603 >> 172.21.15.87:0/3640906603 conn(0x7f689c066b70 msgr2=0x7f689c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 -- 172.21.15.87:0/3640906603 shutdown_connections 2024-04-25T10:15:02.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.076+0000 7f68a3400640 1 -- 172.21.15.87:0/3640906603 wait complete. 2024-04-25T10:15:02.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 Processor -- start 2024-04-25T10:15:02.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 -- start start 2024-04-25T10:15:02.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c0cf930 0x7f689c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f689c15bc40 0x7f689c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0b3850 con 0x7f689c0cda10 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0b36d0 con 0x7f689c0cf930 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0b39d0 con 0x7f689c15bc40 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c15e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53472/0 (socket says 172.21.15.87:53472) 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 -- 172.21.15.87:0/1186703063 learned_addr learned my addr 172.21.15.87:0/1186703063 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f6893e00640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c0cf930 0x7f689c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 -- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f689c15bc40 msgr2=0x7f689c15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a1400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f689c15bc40 0x7f689c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f689c15bc40 0x7f689c15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 -- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c0cf930 msgr2=0x7f689c15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c0cf930 0x7f689c15cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 -- 172.21.15.87:0/1186703063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c15d940 con 0x7f689c0cda10 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a1400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f689c15bc40 0x7f689c15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a0a00640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c15e9b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c370 tx=0x7f688c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f6899600640 1 -- 172.21.15.87:0/1186703063 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c017070 con 0x7f689c0cda10 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f6899600640 1 -- 172.21.15.87:0/1186703063 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c005cf0 con 0x7f689c0cda10 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f6899600640 1 -- 172.21.15.87:0/1186703063 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c0072f0 con 0x7f689c0cda10 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c15dc20 con 0x7f689c0cda10 2024-04-25T10:15:02.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c15e1e0 con 0x7f689c0cda10 2024-04-25T10:15:02.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.080+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689c04ef30 con 0x7f689c0cda10 2024-04-25T10:15:02.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.084+0000 7f6899600640 1 -- 172.21.15.87:0/1186703063 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f688c01a070 con 0x7f689c0cda10 2024-04-25T10:15:02.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.084+0000 7f6899600640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.084+0000 7f6899600640 1 -- 172.21.15.87:0/1186703063 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f688c0840d0 con 0x7f689c0cda10 2024-04-25T10:15:02.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.084+0000 7f6893e00640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.084+0000 7f6893e00640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6878066f60 0x7f6878069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68840ad000 tx=0x7f6884008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.088+0000 7f6899600640 1 -- 172.21.15.87:0/1186703063 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f688c0553e0 con 0x7f689c0cda10 2024-04-25T10:15:02.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.272+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f689c0af1c0 con 0x7f689c0cda10 2024-04-25T10:15:02.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.272+0000 7f6899600640 1 -- 172.21.15.87:0/1186703063 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f688c05b0e0 con 0x7f689c0cda10 2024-04-25T10:15:02.279 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:02.279 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3059476782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3059476782},{"type":"v1","addr":"172.21.15.169:6834","nonce":3059476782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/685777206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":685777206},{"type":"v1","addr":"172.21.15.87:6837","nonce":685777206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2281421407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2281421407},{"type":"v1","addr":"172.21.15.169:6835","nonce":2281421407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2441068259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2441068259},{"type":"v1","addr":"172.21.15.87:6835","nonce":2441068259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T10:15:02.279 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 19 2024-04-25T10:15:02.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6878066f60 msgr2=0x7f6878069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6878066f60 0x7f6878069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68840ad000 tx=0x7f6884008040 comp rx=0 tx=0).stop 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 msgr2=0x7f689c15e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c15e9b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c370 tx=0x7f688c00c830 comp rx=0 tx=0).stop 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 shutdown_connections 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f689c15bc40 0x7f689c15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c0cf930 0x7f689c15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 --2- 172.21.15.87:0/1186703063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f689c0cda10 0x7f689c15e9b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 >> 172.21.15.87:0/1186703063 conn(0x7f689c066b70 msgr2=0x7f689c153030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 shutdown_connections 2024-04-25T10:15:02.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.276+0000 7f68a3400640 1 -- 172.21.15.87:0/1186703063 wait complete. 2024-04-25T10:15:02.300 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:15:02.301 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:15:02.301 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:15:02.301 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:02.301 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:15:02.301 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:15:02.302 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:15:02.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:02.302 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:15:02.302 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:15:02.302 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:15:02.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:02.303 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:15:02.303 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:15:02.303 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:15:02.303 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:02.313 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:15:02.314 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:15:02.314 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:15:02.315 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:15:02.317 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:15:02.317 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:15:02.318 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:15:02.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:15:02.358 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:15:02.358 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:02.359 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:15:02.359 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:02.367 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:15:02.367 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:02.368 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:15:02.368 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:02.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:15:02.445 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:15:02.445 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:15:02.456 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:15:02.456 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:15:02.458 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-25T10:15:02.451+0000 7fdda0200640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T10:15:02.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.504+0000 7ffa81200640 1 Processor -- start 2024-04-25T10:15:02.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.504+0000 7ffa81200640 1 -- start start 2024-04-25T10:15:02.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa81200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffa7c1550b0 0x7ffa7c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa81200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c0cfe70 0x7ffa7c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa81200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7c071050 con 0x7ffa7c15b740 2024-04-25T10:15:02.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa81200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c15d040 con 0x7ffa7c1550b0 2024-04-25T10:15:02.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa81200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0712f0 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa79e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffa7c1550b0 0x7ffa7c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa7a800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ffa7c15b740 0x7ffa7c06d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41680/0 (socket says 172.21.15.87:41680) 2024-04-25T10:15:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa7a800640 1 -- 172.21.15.87:0/133736315 learned_addr learned my addr 172.21.15.87:0/133736315 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa79400640 1 --2- 172.21.15.87:0/133736315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c0cfe70 0x7ffa7c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa79e00640 1 -- 172.21.15.87:0/133736315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c0cfe70 msgr2=0x7ffa7c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa79e00640 1 --2- 172.21.15.87:0/133736315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c0cfe70 0x7ffa7c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa79e00640 1 -- 172.21.15.87:0/133736315 >> v1:172.21.15.87:6789/0 conn(0x7ffa7c15b740 legacy=0x7ffa7c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa79e00640 1 -- 172.21.15.87:0/133736315 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c0715d0 con 0x7ffa7c1550b0 2024-04-25T10:15:02.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.508+0000 7ffa78a00640 1 -- 172.21.15.87:0/133736315 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039741547 0 0) 0x7ffa7c071050 con 0x7ffa7c15b740 2024-04-25T10:15:02.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa79e00640 1 --2- 172.21.15.87:0/133736315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c1550b0 0x7ffa7c0cf930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0097c0 tx=0x7ffa6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291d9c040965d2a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa78a00640 1 -- 172.21.15.87:0/133736315 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0b0070 con 0x7ffa7c1550b0 2024-04-25T10:15:02.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa78a00640 1 -- 172.21.15.87:0/133736315 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0ab600 con 0x7ffa7c1550b0 2024-04-25T10:15:02.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa78a00640 1 -- 172.21.15.87:0/133736315 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0ab9d0 con 0x7ffa7c1550b0 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 -- 172.21.15.87:0/133736315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c1550b0 msgr2=0x7ffa7c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 --2- 172.21.15.87:0/133736315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c1550b0 0x7ffa7c0cf930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0097c0 tx=0x7ffa6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 -- 172.21.15.87:0/133736315 shutdown_connections 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 --2- 172.21.15.87:0/133736315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c0cfe70 0x7ffa7c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 --2- 172.21.15.87:0/133736315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c1550b0 0x7ffa7c0cf930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 -- 172.21.15.87:0/133736315 >> 172.21.15.87:0/133736315 conn(0x7ffa7c066b70 msgr2=0x7ffa7c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 -- 172.21.15.87:0/133736315 shutdown_connections 2024-04-25T10:15:02.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 -- 172.21.15.87:0/133736315 wait complete. 2024-04-25T10:15:02.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 Processor -- start 2024-04-25T10:15:02.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.512+0000 7ffa81200640 1 -- start start 2024-04-25T10:15:02.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c0cfe70 0x7ffa7c14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa7c1550b0 0x7ffa7c149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c15b740 0x7ffa7c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c15bd10 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c15bb90 con 0x7ffa7c1550b0 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c15c030 con 0x7ffa7c15b740 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c0cfe70 0x7ffa7c14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c0cfe70 0x7ffa7c14aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53524/0 (socket says 172.21.15.87:53524) 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 -- 172.21.15.87:0/1302931696 learned_addr learned my addr 172.21.15.87:0/1302931696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79400640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa7c1550b0 0x7ffa7c149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa7a800640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c15b740 0x7ffa7c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 -- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c15b740 msgr2=0x7ffa7c14d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c15b740 0x7ffa7c14d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 -- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa7c1550b0 msgr2=0x7ffa7c149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa7c1550b0 0x7ffa7c149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 -- 172.21.15.87:0/1302931696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c14a600 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa79e00640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c0cfe70 0x7ffa7c14aea0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0ae040 tx=0x7ffa6c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa72000640 1 -- 172.21.15.87:0/1302931696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0b0050 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa72000640 1 -- 172.21.15.87:0/1302931696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0bc070 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa72000640 1 -- 172.21.15.87:0/1302931696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0b5870 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c14a8e0 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c14ef10 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c04ef30 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.516+0000 7ffa72000640 1 -- 172.21.15.87:0/1302931696 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa6c0b5a10 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.520+0000 7ffa72000640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa5c066f60 0x7ffa5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.520+0000 7ffa72000640 1 -- 172.21.15.87:0/1302931696 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa6c11f390 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.520+0000 7ffa79400640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa5c066f60 0x7ffa5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.520+0000 7ffa79400640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa5c066f60 0x7ffa5c069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c157920 tx=0x7ffa68009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.524+0000 7ffa72000640 1 -- 172.21.15.87:0/1302931696 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa6c016090 con 0x7ffa7c0cfe70 2024-04-25T10:15:02.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:02.684+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffa7c06b390 con 0x7ffa7c0cfe70 2024-04-25T10:15:03.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.288+0000 7ffa72000640 1 -- 172.21.15.87:0/1302931696 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffa6c0f0690 con 0x7ffa7c0cfe70 2024-04-25T10:15:03.294 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:15:03.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa5c066f60 msgr2=0x7ffa5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa5c066f60 0x7ffa5c069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c157920 tx=0x7ffa68009190 comp rx=0 tx=0).stop 2024-04-25T10:15:03.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c0cfe70 msgr2=0x7ffa7c14aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c0cfe70 0x7ffa7c14aea0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0ae040 tx=0x7ffa6c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 shutdown_connections 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa5c066f60 0x7ffa5c069420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa7c15b740 0x7ffa7c14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa7c1550b0 0x7ffa7c149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 --2- 172.21.15.87:0/1302931696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa7c0cfe70 0x7ffa7c14aea0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 >> 172.21.15.87:0/1302931696 conn(0x7ffa7c066b70 msgr2=0x7ffa7c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 shutdown_connections 2024-04-25T10:15:03.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.292+0000 7ffa81200640 1 -- 172.21.15.87:0/1302931696 wait complete. 2024-04-25T10:15:03.318 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:15:03.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97f6800640 1 Processor -- start 2024-04-25T10:15:03.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97f6800640 1 -- start start 2024-04-25T10:15:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97f6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 0x7f97f015c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97f6800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f0071050 con 0x7f97f01550b0 2024-04-25T10:15:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97f6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00712f0 con 0x7f97f00d0040 2024-04-25T10:15:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97f6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f015bda0 con 0x7f97f015b910 2024-04-25T10:15:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ef400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f97f01550b0 0x7f97f00cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41692/0 (socket says 172.21.15.87:41692) 2024-04-25T10:15:03.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ef400640 1 -- 172.21.15.87:0/632685899 learned_addr learned my addr 172.21.15.87:0/632685899 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:03.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97efe00640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 0x7f97f015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114915013 0 0) 0x7f97f0071050 con 0x7f97f01550b0 2024-04-25T10:15:03.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97d4003530 con 0x7f97f01550b0 2024-04-25T10:15:03.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1909066414 0 0) 0x7f97d4003530 con 0x7f97f01550b0 2024-04-25T10:15:03.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 msgr2=0x7f97f015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ee000640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 0x7f97f015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.448+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 >> v1:172.21.15.169:6789/0 conn(0x7f97f00d0040 legacy=0x7f97f015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00715d0 con 0x7f97f01550b0 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f97e0001a00 con 0x7f97f01550b0 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 >> v1:172.21.15.87:6789/0 conn(0x7f97f01550b0 legacy=0x7f97f00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d40039f0 0x7f97d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d40043a0 0x7f97d4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 0x7f97d40050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d400b9d0 con 0x7f97d40043a0 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d400bb50 con 0x7f97d40039f0 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d400bd50 con 0x7f97f015b910 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97eea00640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d40043a0 0x7f97d4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ef400640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d40039f0 0x7f97d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97efe00640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 0x7f97d40050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97eea00640 1 -- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 msgr2=0x7f97d40050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97eea00640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97f015b910 0x7f97d40050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97eea00640 1 -- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d40039f0 msgr2=0x7f97d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97eea00640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d40039f0 0x7f97d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97eea00640 1 -- 172.21.15.87:0/632685899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d4003530 con 0x7f97d40043a0 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97eea00640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d40043a0 0x7f97d4004870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f97dc001860 tx=0x7f97dc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140642fbfe16a709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0b5070 con 0x7f97d40043a0 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc0a8890 con 0x7f97d40043a0 2024-04-25T10:15:03.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.452+0000 7f97ee000640 1 -- 172.21.15.87:0/632685899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc001460 con 0x7f97d40043a0 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- 172.21.15.87:0/632685899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d40043a0 msgr2=0x7f97d4004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 --2- 172.21.15.87:0/632685899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d40043a0 0x7f97d4004870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f97dc001860 tx=0x7f97dc01ec70 comp rx=0 tx=0).stop 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97efe00640 1 -- 172.21.15.87:0/632685899 reap_dead start 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- 172.21.15.87:0/632685899 shutdown_connections 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- 172.21.15.87:0/632685899 >> 172.21.15.87:0/632685899 conn(0x7f97f0066b70 msgr2=0x7f97f0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- 172.21.15.87:0/632685899 shutdown_connections 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- 172.21.15.87:0/632685899 wait complete. 2024-04-25T10:15:03.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 Processor -- start 2024-04-25T10:15:03.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- start start 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97d40043a0 0x7f97f00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97f00b3620 0x7f97f00b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0146ae0 0x7f97f0146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00720b0 con 0x7f97f00b3620 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0071b90 con 0x7f97f0146ae0 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97f6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f015c030 con 0x7f97d40043a0 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97eea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97f00b3620 0x7f97f00b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97eea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97f00b3620 0x7f97f00b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53540/0 (socket says 172.21.15.87:53540) 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97ef400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97d40043a0 0x7f97f00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97eea00640 1 -- 172.21.15.87:0/3879523556 learned_addr learned my addr 172.21.15.87:0/3879523556 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.456+0000 7f97efe00640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0146ae0 0x7f97f0146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97eea00640 1 -- 172.21.15.87:0/3879523556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97d40043a0 msgr2=0x7f97f00adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97eea00640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97d40043a0 0x7f97f00adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97eea00640 1 -- 172.21.15.87:0/3879523556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0146ae0 msgr2=0x7f97f0146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97eea00640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0146ae0 0x7f97f0146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97eea00640 1 -- 172.21.15.87:0/3879523556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00b42c0 con 0x7f97f00b3620 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97eea00640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97f00b3620 0x7f97f00b3a80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0b3040 tx=0x7f97d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97e7400640 1 -- 172.21.15.87:0/3879523556 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0b5040 con 0x7f97f00b3620 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97e7400640 1 -- 172.21.15.87:0/3879523556 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc005070 con 0x7f97f00b3620 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97e7400640 1 -- 172.21.15.87:0/3879523556 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc005460 con 0x7f97f00b3620 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f0159b20 con 0x7f97f00b3620 2024-04-25T10:15:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f015a0e0 con 0x7f97f00b3620 2024-04-25T10:15:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f004ef30 con 0x7f97f00b3620 2024-04-25T10:15:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97e7400640 1 -- 172.21.15.87:0/3879523556 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97dc0c3060 con 0x7f97f00b3620 2024-04-25T10:15:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97e7400640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f97d0066f90 0x7f97d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97e7400640 1 -- 172.21.15.87:0/3879523556 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f97dc1212b0 con 0x7f97f00b3620 2024-04-25T10:15:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.460+0000 7f97ef400640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f97d0066f90 0x7f97d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.464+0000 7f97ef400640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f97d0066f90 0x7f97d0069450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f97e0002000 tx=0x7f97e0006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.464+0000 7f97e7400640 1 -- 172.21.15.87:0/3879523556 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97dc018980 con 0x7f97f00b3620 2024-04-25T10:15:03.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.668+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f97f006dc40 con 0x7f97f00b3620 2024-04-25T10:15:03.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97e7400640 1 -- 172.21.15.87:0/3879523556 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f97dc0f1f20 con 0x7f97f00b3620 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.677 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:03.678 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:15:03.679 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.680 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.681 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.684 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.685 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:03.686 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:15:03.688 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:03.689 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:03.689 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.689 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:03.689 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:15:03.689 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f97d0066f90 msgr2=0x7f97d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f97d0066f90 0x7f97d0069450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f97e0002000 tx=0x7f97e0006040 comp rx=0 tx=0).stop 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97f00b3620 msgr2=0x7f97f00b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97f00b3620 0x7f97f00b3a80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0b3040 tx=0x7f97d4005840 comp rx=0 tx=0).stop 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 shutdown_connections 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f97d0066f90 0x7f97d0069450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0146ae0 0x7f97f0146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97f00b3620 0x7f97f00b3a80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 --2- 172.21.15.87:0/3879523556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97d40043a0 0x7f97f00adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 >> 172.21.15.87:0/3879523556 conn(0x7f97f0066b70 msgr2=0x7f97f006cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 shutdown_connections 2024-04-25T10:15:03.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.672+0000 7f97f6800640 1 -- 172.21.15.87:0/3879523556 wait complete. 2024-04-25T10:15:03.698 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:03.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63e5800640 1 Processor -- start 2024-04-25T10:15:03.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63e5800640 1 -- start start 2024-04-25T10:15:03.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63e5800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63e00cfd90 0x7f63e00ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63e5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63e5800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e015f3b0 con 0x7f63e00b0f70 2024-04-25T10:15:03.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63e5800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e015ef20 con 0x7f63e00cfd90 2024-04-25T10:15:03.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63e5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00ca810 con 0x7f63e00cd110 2024-04-25T10:15:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63dea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f63e00b0f70 0x7f63e00b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41694/0 (socket says 172.21.15.87:41694) 2024-04-25T10:15:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63dea00640 1 -- 172.21.15.87:0/1985470159 learned_addr learned my addr 172.21.15.87:0/1985470159 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63df400640 1 --2- 172.21.15.87:0/1985470159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63d6000640 1 --2- 172.21.15.87:0/1985470159 >> v2:172.21.15.169:3300/0 conn(0x7f63e00cfd90 0x7f63e00ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63de000640 1 -- 172.21.15.87:0/1985470159 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587637780 0 0) 0x7f63e015f3b0 con 0x7f63e00b0f70 2024-04-25T10:15:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63de000640 1 -- 172.21.15.87:0/1985470159 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c4003530 con 0x7f63e00b0f70 2024-04-25T10:15:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63df400640 1 -- 172.21.15.87:0/1985470159 >> v2:172.21.15.169:3300/0 conn(0x7f63e00cfd90 msgr2=0x7f63e00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63df400640 1 --2- 172.21.15.87:0/1985470159 >> v2:172.21.15.169:3300/0 conn(0x7f63e00cfd90 0x7f63e00ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63df400640 1 -- 172.21.15.87:0/1985470159 >> v1:172.21.15.87:6789/0 conn(0x7f63e00b0f70 legacy=0x7f63e00b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.828+0000 7f63de000640 1 -- 172.21.15.87:0/1985470159 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207474661 0 0) 0x7f63c4003530 con 0x7f63e00b0f70 2024-04-25T10:15:03.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63df400640 1 -- 172.21.15.87:0/1985470159 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e015e5f0 con 0x7f63e00cd110 2024-04-25T10:15:03.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63df400640 1 --2- 172.21.15.87:0/1985470159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e00cba20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009d90 tx=0x7f63d801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53675345fb7c407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63de000640 1 -- 172.21.15.87:0/1985470159 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ac070 con 0x7f63e00cd110 2024-04-25T10:15:03.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63de000640 1 -- 172.21.15.87:0/1985470159 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d80a64a0 con 0x7f63e00cd110 2024-04-25T10:15:03.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63de000640 1 -- 172.21.15.87:0/1985470159 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0001680 con 0x7f63e00cd110 2024-04-25T10:15:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63e5800640 1 -- 172.21.15.87:0/1985470159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 msgr2=0x7f63e00cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63e5800640 1 --2- 172.21.15.87:0/1985470159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e00cba20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009d90 tx=0x7f63d801c890 comp rx=0 tx=0).stop 2024-04-25T10:15:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63e5800640 1 -- 172.21.15.87:0/1985470159 shutdown_connections 2024-04-25T10:15:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63e5800640 1 --2- 172.21.15.87:0/1985470159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e00cba20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63e5800640 1 --2- 172.21.15.87:0/1985470159 >> v2:172.21.15.169:3300/0 conn(0x7f63e00cfd90 0x7f63e00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63e5800640 1 -- 172.21.15.87:0/1985470159 >> 172.21.15.87:0/1985470159 conn(0x7f63e00b1d50 msgr2=0x7f63e00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.832+0000 7f63e5800640 1 -- 172.21.15.87:0/1985470159 shutdown_connections 2024-04-25T10:15:03.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 -- 172.21.15.87:0/1985470159 wait complete. 2024-04-25T10:15:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 Processor -- start 2024-04-25T10:15:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 -- start start 2024-04-25T10:15:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e00b0f70 0x7f63e00c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e0154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e00cfd90 0x7f63e00c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e0161b50 con 0x7f63e00cfd90 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e01619d0 con 0x7f63e00b0f70 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e0161cd0 con 0x7f63e00cd110 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e00cfd90 0x7f63e00c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e00cfd90 0x7f63e00c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53546/0 (socket says 172.21.15.87:53546) 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 -- 172.21.15.87:0/3529959811 learned_addr learned my addr 172.21.15.87:0/3529959811 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63dea00640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e00b0f70 0x7f63e00c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63d6000640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e0154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 -- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 msgr2=0x7f63e0154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e0154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 -- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e00b0f70 msgr2=0x7f63e00c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e00b0f70 0x7f63e00c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 -- 172.21.15.87:0/3529959811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0155460 con 0x7f63e00cfd90 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63dea00640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e00b0f70 0x7f63e00c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63df400640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e00cfd90 0x7f63e00c69c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f63d801af40 tx=0x7f63d8019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63d7400640 1 -- 172.21.15.87:0/3529959811 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80c0070 con 0x7f63e00cfd90 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63d7400640 1 -- 172.21.15.87:0/3529959811 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d80a7480 con 0x7f63e00cfd90 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63d7400640 1 -- 172.21.15.87:0/3529959811 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ac050 con 0x7f63e00cfd90 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e00c5920 con 0x7f63e00cfd90 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.836+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e00c5ee0 con 0x7f63e00cfd90 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.840+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e004ef30 con 0x7f63e00cfd90 2024-04-25T10:15:03.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.840+0000 7f63d7400640 1 -- 172.21.15.87:0/3529959811 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63d80af070 con 0x7f63e00cfd90 2024-04-25T10:15:03.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.840+0000 7f63d7400640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63bc066f60 0x7f63bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.840+0000 7f63d7400640 1 -- 172.21.15.87:0/3529959811 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63d811f250 con 0x7f63e00cfd90 2024-04-25T10:15:03.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.840+0000 7f63dea00640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63bc066f60 0x7f63bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.840+0000 7f63dea00640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63bc066f60 0x7f63bc069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63d0001b80 tx=0x7f63d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:03.844+0000 7f63d7400640 1 -- 172.21.15.87:0/3529959811 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63d8016480 con 0x7f63e00cfd90 2024-04-25T10:15:04.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63e00cc9b0 con 0x7f63e00cfd90 2024-04-25T10:15:04.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63d7400640 1 -- 172.21.15.87:0/3529959811 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f63d80f0510 con 0x7f63e00cfd90 2024-04-25T10:15:04.005 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:04.005 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":42,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:03.301022+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:04.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63bc066f60 msgr2=0x7f63bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63bc066f60 0x7f63bc069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63d0001b80 tx=0x7f63d0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:04.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e00cfd90 msgr2=0x7f63e00c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e00cfd90 0x7f63e00c69c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f63d801af40 tx=0x7f63d8019630 comp rx=0 tx=0).stop 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 shutdown_connections 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63bc066f60 0x7f63bc069420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e00cfd90 0x7f63e00c69c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e00cd110 0x7f63e0154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 --2- 172.21.15.87:0/3529959811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e00b0f70 0x7f63e00c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 >> 172.21.15.87:0/3529959811 conn(0x7f63e00b1d50 msgr2=0x7f63e006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 shutdown_connections 2024-04-25T10:15:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.000+0000 7f63e5800640 1 -- 172.21.15.87:0/3529959811 wait complete. 2024-04-25T10:15:04.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:04.027 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.160+0000 7f4ba3000640 1 Processor -- start 2024-04-25T10:15:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.160+0000 7f4ba3000640 1 -- start start 2024-04-25T10:15:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 0x7f4b9c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba3000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9c0b2460 con 0x7f4b9c0b44c0 2024-04-25T10:15:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba3000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9c070fb0 con 0x7f4b9c15b750 2024-04-25T10:15:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c0cc050 con 0x7f4b9c0cbc30 2024-04-25T10:15:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba2600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4b9c0b44c0 0x7f4b9c0b3200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41698/0 (socket says 172.21.15.87:41698) 2024-04-25T10:15:04.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba2600640 1 -- 172.21.15.87:0/1731369589 learned_addr learned my addr 172.21.15.87:0/1731369589 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762377759 0 0) 0x7f4b9c0b2460 con 0x7f4b9c0b44c0 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 0x7f4b9c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b84003530 con 0x7f4b9c0b44c0 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3819689110 0 0) 0x7f4b84003530 con 0x7f4b9c0b44c0 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 msgr2=0x7f4b9c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 0x7f4b9c0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 >> v1:172.21.15.169:6789/0 conn(0x7f4b9c15b750 legacy=0x7f4b9c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c0b2720 con 0x7f4b9c0b44c0 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f4b94001a00 con 0x7f4b9c0b44c0 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 >> v1:172.21.15.87:6789/0 conn(0x7f4b9c0b44c0 legacy=0x7f4b9c0b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 0x7f4b84003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b84004d50 0x7f4b840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c0cc050 con 0x7f4b84004030 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8400bb90 con 0x7f4b84004d50 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8400bd90 con 0x7f4b9c0cbc30 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1c00640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 0x7f4b84003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba2600640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b84004d50 0x7f4b840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1200640 1 -- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 msgr2=0x7f4b84003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c0cbc30 0x7f4b84003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1200640 1 -- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b84004d50 msgr2=0x7f4b840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b84004d50 0x7f4b840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.164+0000 7f4ba1200640 1 -- 172.21.15.87:0/1731369589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84003530 con 0x7f4b84004030 2024-04-25T10:15:04.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b84004810 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009ae0 tx=0x7f4b8801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371598e4f97c1cff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b880b2070 con 0x7f4b84004030 2024-04-25T10:15:04.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b880a5660 con 0x7f4b84004030 2024-04-25T10:15:04.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4b9be00640 1 -- 172.21.15.87:0/1731369589 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b880096b0 con 0x7f4b84004030 2024-04-25T10:15:04.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 -- 172.21.15.87:0/1731369589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 msgr2=0x7f4b84004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 --2- 172.21.15.87:0/1731369589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b84004810 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009ae0 tx=0x7f4b8801ca70 comp rx=0 tx=0).stop 2024-04-25T10:15:04.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba2600640 1 -- 172.21.15.87:0/1731369589 reap_dead start 2024-04-25T10:15:04.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 -- 172.21.15.87:0/1731369589 shutdown_connections 2024-04-25T10:15:04.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 -- 172.21.15.87:0/1731369589 >> 172.21.15.87:0/1731369589 conn(0x7f4b9c0743f0 msgr2=0x7f4b9c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 -- 172.21.15.87:0/1731369589 shutdown_connections 2024-04-25T10:15:04.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 -- 172.21.15.87:0/1731369589 wait complete. 2024-04-25T10:15:04.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 Processor -- start 2024-04-25T10:15:04.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.168+0000 7f4ba3000640 1 -- start start 2024-04-25T10:15:04.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b9c146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c154300 0x7f4b9c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c15a1c0 0x7f4b9c1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c0aead0 con 0x7f4b84004030 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c0b2d70 con 0x7f4b9c15a1c0 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c0cdca0 con 0x7f4b9c154300 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b9c146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b9c146150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53570/0 (socket says 172.21.15.87:53570) 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 -- 172.21.15.87:0/1768170856 learned_addr learned my addr 172.21.15.87:0/1768170856 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c154300 0x7f4b9c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba2600640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c15a1c0 0x7f4b9c1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 -- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c154300 msgr2=0x7f4b9c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c154300 0x7f4b9c159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 -- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c15a1c0 msgr2=0x7f4b9c1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c15a1c0 0x7f4b9c1437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 -- 172.21.15.87:0/1768170856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c1547c0 con 0x7f4b84004030 2024-04-25T10:15:04.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba1c00640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b9c146150 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000f7f0 tx=0x7f4b9000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4b99600640 1 -- 172.21.15.87:0/1768170856 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b900050d0 con 0x7f4b84004030 2024-04-25T10:15:04.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4b99600640 1 -- 172.21.15.87:0/1768170856 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90018070 con 0x7f4b84004030 2024-04-25T10:15:04.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4b99600640 1 -- 172.21.15.87:0/1768170856 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000d730 con 0x7f4b84004030 2024-04-25T10:15:04.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c1527c0 con 0x7f4b84004030 2024-04-25T10:15:04.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c152d80 con 0x7f4b84004030 2024-04-25T10:15:04.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9c151670 con 0x7f4b84004030 2024-04-25T10:15:04.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.172+0000 7f4b99600640 1 -- 172.21.15.87:0/1768170856 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b9000d8d0 con 0x7f4b84004030 2024-04-25T10:15:04.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.176+0000 7f4b99600640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b7c066f60 0x7f4b7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.176+0000 7f4b99600640 1 -- 172.21.15.87:0/1768170856 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b90084c30 con 0x7f4b84004030 2024-04-25T10:15:04.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.176+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b7c066f60 0x7f4b7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.176+0000 7f4ba1200640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b7c066f60 0x7f4b7c069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4b880af000 tx=0x7f4b880046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.180+0000 7f4b99600640 1 -- 172.21.15.87:0/1768170856 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b90055ef0 con 0x7f4b84004030 2024-04-25T10:15:04.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.344+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b9c15a5d0 con 0x7f4b84004030 2024-04-25T10:15:04.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4b99600640 1 -- 172.21.15.87:0/1768170856 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b9c15a5d0 con 0x7f4b84004030 2024-04-25T10:15:04.351 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:15:04.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b7c066f60 msgr2=0x7f4b7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b7c066f60 0x7f4b7c069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4b880af000 tx=0x7f4b880046d0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 msgr2=0x7f4b9c146150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b9c146150 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000f7f0 tx=0x7f4b9000fcb0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 shutdown_connections 2024-04-25T10:15:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b7c066f60 0x7f4b7c069420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c15a1c0 0x7f4b9c1437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b9c154300 0x7f4b9c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 --2- 172.21.15.87:0/1768170856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b84004030 0x7f4b9c146150 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 >> 172.21.15.87:0/1768170856 conn(0x7f4b9c0743f0 msgr2=0x7f4b9c15d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 shutdown_connections 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.348+0000 7f4ba3000640 1 -- 172.21.15.87:0/1768170856 wait complete. 2024-04-25T10:15:04.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:15:04.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738ec00640 1 Processor -- start 2024-04-25T10:15:04.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738ec00640 1 -- start start 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815b9e0 0x7f73880cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738ec00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73880d1c30 0x7f73880cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738ec00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73880ccfb0 con 0x7f738815bdc0 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738ec00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73880cd290 con 0x7f73880d1c30 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880cd570 con 0x7f738815b9e0 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73880d1c30 0x7f73880cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f7387400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f738815bdc0 0x7f73880d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41714/0 (socket says 172.21.15.87:41714) 2024-04-25T10:15:04.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f7387400640 1 -- 172.21.15.87:0/1919863945 learned_addr learned my addr 172.21.15.87:0/1919863945 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f7387e00640 1 --2- 172.21.15.87:0/1919863945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815b9e0 0x7f73880cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738cc00640 1 -- 172.21.15.87:0/1919863945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815b9e0 msgr2=0x7f73880cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738cc00640 1 --2- 172.21.15.87:0/1919863945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815b9e0 0x7f73880cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738cc00640 1 -- 172.21.15.87:0/1919863945 >> v1:172.21.15.87:6789/0 conn(0x7f738815bdc0 legacy=0x7f73880d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f738cc00640 1 -- 172.21.15.87:0/1919863945 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880cd850 con 0x7f73880d1c30 2024-04-25T10:15:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f7386a00640 1 -- 172.21.15.87:0/1919863945 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880745815 0 0) 0x7f73880ccfb0 con 0x7f738815bdc0 2024-04-25T10:15:04.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.504+0000 7f7387e00640 1 --2- 172.21.15.87:0/1919863945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815b9e0 0x7f73880cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738cc00640 1 --2- 172.21.15.87:0/1919863945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73880d1c30 0x7f73880cc950 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f73800097c0 tx=0x7f738001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3d4d2bdc379c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f7386a00640 1 -- 172.21.15.87:0/1919863945 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73800b0070 con 0x7f73880d1c30 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f7386a00640 1 -- 172.21.15.87:0/1919863945 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7380004230 con 0x7f73880d1c30 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f7386a00640 1 -- 172.21.15.87:0/1919863945 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380003620 con 0x7f73880d1c30 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 -- 172.21.15.87:0/1919863945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73880d1c30 msgr2=0x7f73880cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 --2- 172.21.15.87:0/1919863945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73880d1c30 0x7f73880cc950 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f73800097c0 tx=0x7f738001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 -- 172.21.15.87:0/1919863945 shutdown_connections 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 --2- 172.21.15.87:0/1919863945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73880d1c30 0x7f73880cc950 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 --2- 172.21.15.87:0/1919863945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815b9e0 0x7f73880cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 -- 172.21.15.87:0/1919863945 >> 172.21.15.87:0/1919863945 conn(0x7f738809a9d0 msgr2=0x7f738809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 -- 172.21.15.87:0/1919863945 shutdown_connections 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.508+0000 7f738ec00640 1 -- 172.21.15.87:0/1919863945 wait complete. 2024-04-25T10:15:04.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 Processor -- start 2024-04-25T10:15:04.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 -- start start 2024-04-25T10:15:04.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73880d1c30 0x7f7388155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738815b9e0 0x7f73881599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815bdc0 0x7f7388159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73880c6630 con 0x7f738815b9e0 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73880c64b0 con 0x7f73880d1c30 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880c67b0 con 0x7f738815bdc0 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738815b9e0 0x7f73881599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738815b9e0 0x7f73881599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53586/0 (socket says 172.21.15.87:53586) 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 -- 172.21.15.87:0/4269788839 learned_addr learned my addr 172.21.15.87:0/4269788839 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738cc00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815bdc0 0x7f7388159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387e00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73880d1c30 0x7f7388155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 -- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815bdc0 msgr2=0x7f7388159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815bdc0 0x7f7388159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 -- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73880d1c30 msgr2=0x7f7388155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73880d1c30 0x7f7388155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 -- 172.21.15.87:0/4269788839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880ca980 con 0x7f738815b9e0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387e00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73880d1c30 0x7f7388155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f7387400640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738815b9e0 0x7f73881599d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f737400c480 tx=0x7f737400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f737fe00640 1 -- 172.21.15.87:0/4269788839 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374018070 con 0x7f738815b9e0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f737fe00640 1 -- 172.21.15.87:0/4269788839 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7374007500 con 0x7f738815b9e0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f737fe00640 1 -- 172.21.15.87:0/4269788839 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374004a30 con 0x7f738815b9e0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738815e1d0 con 0x7f738815b9e0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.512+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73880caec0 con 0x7f738815b9e0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.516+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738804ef30 con 0x7f738815b9e0 2024-04-25T10:15:04.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.516+0000 7f737fe00640 1 -- 172.21.15.87:0/4269788839 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7374004bd0 con 0x7f738815b9e0 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.516+0000 7f737fe00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7368066f60 0x7f7368069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.516+0000 7f737fe00640 1 -- 172.21.15.87:0/4269788839 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7374083ed0 con 0x7f738815b9e0 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.516+0000 7f7387e00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7368066f60 0x7f7368069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.516+0000 7f7387e00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7368066f60 0x7f7368069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f737800e550 tx=0x7f7378007ab0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.520+0000 7f737fe00640 1 -- 172.21.15.87:0/4269788839 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73880ca980 con 0x7f738815b9e0 2024-04-25T10:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.676+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f738815a970 con 0x7f738815b9e0 2024-04-25T10:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.676+0000 7f737fe00640 1 -- 172.21.15.87:0/4269788839 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7374054bd0 con 0x7f738815b9e0 2024-04-25T10:15:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:04.683 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:15:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:15:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:15:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:04.683 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7368066f60 msgr2=0x7f7368069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7368066f60 0x7f7368069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f737800e550 tx=0x7f7378007ab0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738815b9e0 msgr2=0x7f73881599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738815b9e0 0x7f73881599d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f737400c480 tx=0x7f737400c940 comp rx=0 tx=0).stop 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 shutdown_connections 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7368066f60 0x7f7368069420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f738815bdc0 0x7f7388159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738815b9e0 0x7f73881599d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 --2- 172.21.15.87:0/4269788839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73880d1c30 0x7f7388155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 >> 172.21.15.87:0/4269788839 conn(0x7f738809a9d0 msgr2=0x7f738806e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 shutdown_connections 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.680+0000 7f738ec00640 1 -- 172.21.15.87:0/4269788839 wait complete. 2024-04-25T10:15:04.703 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:15:04.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:15:04.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.836+0000 7ff698a00640 1 Processor -- start 2024-04-25T10:15:04.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.836+0000 7ff698a00640 1 -- start start 2024-04-25T10:15:04.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.836+0000 7ff698a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff694162310 0x7ff69415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.836+0000 7ff698a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6940b4f50 0x7ff6940b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.836+0000 7ff698a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69415e9b0 con 0x7ff6941618a0 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.836+0000 7ff698a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69415f250 con 0x7ff6940b4f50 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.836+0000 7ff698a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6940afce0 con 0x7ff694162310 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6940b4f50 0x7ff6940b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6940b4f50 0x7ff6940b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53596/0 (socket says 172.21.15.87:53596) 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 -- 172.21.15.87:0/1260727191 learned_addr learned my addr 172.21.15.87:0/1260727191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff691e00640 1 --2- 172.21.15.87:0/1260727191 >> v2:172.21.15.169:3300/0 conn(0x7ff694162310 0x7ff69415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 -- 172.21.15.87:0/1260727191 >> v2:172.21.15.169:3300/0 conn(0x7ff694162310 msgr2=0x7ff69415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 --2- 172.21.15.87:0/1260727191 >> v2:172.21.15.169:3300/0 conn(0x7ff694162310 0x7ff69415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 -- 172.21.15.87:0/1260727191 >> v1:172.21.15.169:6789/0 conn(0x7ff6941618a0 legacy=0x7ff6940b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 -- 172.21.15.87:0/1260727191 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69415ec90 con 0x7ff6940b4f50 2024-04-25T10:15:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff691e00640 1 --2- 172.21.15.87:0/1260727191 >> v2:172.21.15.169:3300/0 conn(0x7ff694162310 0x7ff69415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:04.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff692800640 1 --2- 172.21.15.87:0/1260727191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6940b4f50 0x7ff6940b0170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff6940720f0 tx=0x7ff68c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c95f3adff20a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff691400640 1 -- 172.21.15.87:0/1260727191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0a5730 con 0x7ff6940b4f50 2024-04-25T10:15:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff691400640 1 -- 172.21.15.87:0/1260727191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c0a58d0 con 0x7ff6940b4f50 2024-04-25T10:15:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.840+0000 7ff691400640 1 -- 172.21.15.87:0/1260727191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0a5ca0 con 0x7ff6940b4f50 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- 172.21.15.87:0/1260727191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6940b4f50 msgr2=0x7ff6940b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 --2- 172.21.15.87:0/1260727191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6940b4f50 0x7ff6940b0170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff6940720f0 tx=0x7ff68c01c890 comp rx=0 tx=0).stop 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- 172.21.15.87:0/1260727191 shutdown_connections 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 --2- 172.21.15.87:0/1260727191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6940b4f50 0x7ff6940b0170 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 --2- 172.21.15.87:0/1260727191 >> v2:172.21.15.169:3300/0 conn(0x7ff694162310 0x7ff69415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- 172.21.15.87:0/1260727191 >> 172.21.15.87:0/1260727191 conn(0x7ff6940b5610 msgr2=0x7ff6940b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- 172.21.15.87:0/1260727191 shutdown_connections 2024-04-25T10:15:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- 172.21.15.87:0/1260727191 wait complete. 2024-04-25T10:15:04.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 Processor -- start 2024-04-25T10:15:04.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- start start 2024-04-25T10:15:04.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6940b4f50 0x7ff694152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6941618a0 0x7ff694150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff694162310 0x7ff694142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6940b1040 con 0x7ff694162310 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6940b0ec0 con 0x7ff6940b4f50 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff698a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6940b11c0 con 0x7ff6941618a0 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff694162310 0x7ff694142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff694162310 0x7ff694142cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53606/0 (socket says 172.21.15.87:53606) 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 -- 172.21.15.87:0/1618129210 learned_addr learned my addr 172.21.15.87:0/1618129210 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff691e00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6940b4f50 0x7ff694152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff689400640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6941618a0 0x7ff694150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 -- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6941618a0 msgr2=0x7ff694150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6941618a0 0x7ff694150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 -- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6940b4f50 msgr2=0x7ff694152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6940b4f50 0x7ff694152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 -- 172.21.15.87:0/1618129210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6941533e0 con 0x7ff694162310 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff691e00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6940b4f50 0x7ff694152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:04.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.844+0000 7ff692800640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff694162310 0x7ff694142cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0af040 tx=0x7ff68c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff68aa00640 1 -- 172.21.15.87:0/1618129210 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0bc020 con 0x7ff694162310 2024-04-25T10:15:04.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff68aa00640 1 -- 172.21.15.87:0/1618129210 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c0b8670 con 0x7ff694162310 2024-04-25T10:15:04.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff68aa00640 1 -- 172.21.15.87:0/1618129210 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0b8a60 con 0x7ff694162310 2024-04-25T10:15:04.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff694098a60 con 0x7ff694162310 2024-04-25T10:15:04.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff694099020 con 0x7ff694162310 2024-04-25T10:15:04.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69404ef30 con 0x7ff694162310 2024-04-25T10:15:04.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff68aa00640 1 -- 172.21.15.87:0/1618129210 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff68c0bf020 con 0x7ff694162310 2024-04-25T10:15:04.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff68aa00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff670066f60 0x7ff670069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff68aa00640 1 -- 172.21.15.87:0/1618129210 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff68c11fcf0 con 0x7ff694162310 2024-04-25T10:15:04.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff691e00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff670066f60 0x7ff670069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.848+0000 7ff691e00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff670066f60 0x7ff670069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff68400a9b0 tx=0x7ff684008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:04.852+0000 7ff68aa00640 1 -- 172.21.15.87:0/1618129210 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff68c124070 con 0x7ff694162310 2024-04-25T10:15:05.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.008+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff6940b2a10 con 0x7ff694162310 2024-04-25T10:15:05.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.296+0000 7ff68aa00640 1 -- 172.21.15.87:0/1618129210 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff68c0c6030 con 0x7ff694162310 2024-04-25T10:15:05.301 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:15:05.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff670066f60 msgr2=0x7ff670069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff670066f60 0x7ff670069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff68400a9b0 tx=0x7ff684008040 comp rx=0 tx=0).stop 2024-04-25T10:15:05.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff694162310 msgr2=0x7ff694142cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff694162310 0x7ff694142cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0af040 tx=0x7ff68c019630 comp rx=0 tx=0).stop 2024-04-25T10:15:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 shutdown_connections 2024-04-25T10:15:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff670066f60 0x7ff670069420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff694162310 0x7ff694142cc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6941618a0 0x7ff694150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 --2- 172.21.15.87:0/1618129210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6940b4f50 0x7ff694152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 >> 172.21.15.87:0/1618129210 conn(0x7ff6940b5610 msgr2=0x7ff694160620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 shutdown_connections 2024-04-25T10:15:05.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.300+0000 7ff698a00640 1 -- 172.21.15.87:0/1618129210 wait complete. 2024-04-25T10:15:05.328 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.468+0000 7f6b57000640 1 Processor -- start 2024-04-25T10:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.468+0000 7f6b57000640 1 -- start start 2024-04-25T10:15:05.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b57000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b5015bbb0 0x7f6b500cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b57000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5006ca90 con 0x7f6b500b4660 2024-04-25T10:15:05.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b57000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5006cd70 con 0x7f6b500cf960 2024-04-25T10:15:05.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b57000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5006d050 con 0x7f6b5015bbb0 2024-04-25T10:15:05.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b4fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b5015bbb0 0x7f6b500cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b4f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6b500b4660 0x7f6b5006c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41724/0 (socket says 172.21.15.87:41724) 2024-04-25T10:15:05.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b4f400640 1 -- 172.21.15.87:0/1283073077 learned_addr learned my addr 172.21.15.87:0/1283073077 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b4fe00640 1 -- 172.21.15.87:0/1283073077 >> v1:172.21.15.169:6789/0 conn(0x7f6b500cf960 legacy=0x7f6b500b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b4fe00640 1 -- 172.21.15.87:0/1283073077 >> v1:172.21.15.87:6789/0 conn(0x7f6b500b4660 legacy=0x7f6b5006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b4fe00640 1 -- 172.21.15.87:0/1283073077 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500b2880 con 0x7f6b5015bbb0 2024-04-25T10:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.472+0000 7f6b4ea00640 1 -- 172.21.15.87:0/1283073077 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855326571 0 0) 0x7f6b5006ca90 con 0x7f6b500b4660 2024-04-25T10:15:05.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4fe00640 1 --2- 172.21.15.87:0/1283073077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5015bbb0 0x7f6b500cda80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6b400097c0 tx=0x7f6b4001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbc5d0e52141011 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4ea00640 1 -- 172.21.15.87:0/1283073077 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400b0070 con 0x7f6b5015bbb0 2024-04-25T10:15:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4ea00640 1 -- 172.21.15.87:0/1283073077 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b400ab600 con 0x7f6b5015bbb0 2024-04-25T10:15:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4ea00640 1 -- 172.21.15.87:0/1283073077 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400ab9d0 con 0x7f6b5015bbb0 2024-04-25T10:15:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- 172.21.15.87:0/1283073077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5015bbb0 msgr2=0x7f6b500cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 --2- 172.21.15.87:0/1283073077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5015bbb0 0x7f6b500cda80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6b400097c0 tx=0x7f6b4001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- 172.21.15.87:0/1283073077 shutdown_connections 2024-04-25T10:15:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 --2- 172.21.15.87:0/1283073077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5015bbb0 0x7f6b500cda80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- 172.21.15.87:0/1283073077 >> 172.21.15.87:0/1283073077 conn(0x7f6b50066b70 msgr2=0x7f6b50066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- 172.21.15.87:0/1283073077 shutdown_connections 2024-04-25T10:15:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- 172.21.15.87:0/1283073077 wait complete. 2024-04-25T10:15:05.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 Processor -- start 2024-04-25T10:15:05.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- start start 2024-04-25T10:15:05.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b500b4660 0x7f6b50153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b500cf960 0x7f6b500c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b5015bbb0 0x7f6b500ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500ce140 con 0x7f6b500cf960 2024-04-25T10:15:05.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500cdfc0 con 0x7f6b500b4660 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b57000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5016a180 con 0x7f6b5015bbb0 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b500cf960 0x7f6b500c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b500cf960 0x7f6b500c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53620/0 (socket says 172.21.15.87:53620) 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4f400640 1 -- 172.21.15.87:0/2604308717 learned_addr learned my addr 172.21.15.87:0/2604308717 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.476+0000 7f6b4fe00640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b500b4660 0x7f6b50153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b55000640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b5015bbb0 0x7f6b500ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b4f400640 1 -- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b5015bbb0 msgr2=0x7f6b500ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b4f400640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b5015bbb0 0x7f6b500ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b4f400640 1 -- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b500b4660 msgr2=0x7f6b50153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b4f400640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b500b4660 0x7f6b50153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b4f400640 1 -- 172.21.15.87:0/2604308717 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b501538a0 con 0x7f6b500cf960 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b4f400640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b500cf960 0x7f6b500c4750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c480 tx=0x7f6b3c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b47e00640 1 -- 172.21.15.87:0/2604308717 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c018070 con 0x7f6b500cf960 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b47e00640 1 -- 172.21.15.87:0/2604308717 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c0074e0 con 0x7f6b500cf960 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b47e00640 1 -- 172.21.15.87:0/2604308717 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004a30 con 0x7f6b500cf960 2024-04-25T10:15:05.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b50154c10 con 0x7f6b500cf960 2024-04-25T10:15:05.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501551d0 con 0x7f6b500cf960 2024-04-25T10:15:05.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5004ef30 con 0x7f6b500cf960 2024-04-25T10:15:05.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b47e00640 1 -- 172.21.15.87:0/2604308717 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b3c004bd0 con 0x7f6b500cf960 2024-04-25T10:15:05.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.480+0000 7f6b47e00640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b30066f60 0x7f6b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.484+0000 7f6b47e00640 1 -- 172.21.15.87:0/2604308717 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6b3c0841a0 con 0x7f6b500cf960 2024-04-25T10:15:05.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.484+0000 7f6b4fe00640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b30066f60 0x7f6b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.484+0000 7f6b4fe00640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b30066f60 0x7f6b30069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6b400ad000 tx=0x7f6b40008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.488+0000 7f6b47e00640 1 -- 172.21.15.87:0/2604308717 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b3c054d30 con 0x7f6b500cf960 2024-04-25T10:15:05.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:05.644+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6b5006d980 con 0x7f6b500cf960 2024-04-25T10:15:06.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.300+0000 7f6b47e00640 1 -- 172.21.15.87:0/2604308717 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6b3c058dd0 con 0x7f6b500cf960 2024-04-25T10:15:06.305 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b30066f60 msgr2=0x7f6b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b30066f60 0x7f6b30069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6b400ad000 tx=0x7f6b40008040 comp rx=0 tx=0).stop 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b500cf960 msgr2=0x7f6b500c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b500cf960 0x7f6b500c4750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c480 tx=0x7f6b3c00c940 comp rx=0 tx=0).stop 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 shutdown_connections 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b30066f60 0x7f6b30069420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b5015bbb0 0x7f6b500ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b500cf960 0x7f6b500c4750 secure :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c480 tx=0x7f6b3c00c940 comp rx=0 tx=0).stop 2024-04-25T10:15:06.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 --2- 172.21.15.87:0/2604308717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b500b4660 0x7f6b50153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 >> 172.21.15.87:0/2604308717 conn(0x7f6b50066b70 msgr2=0x7f6b50066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.304+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 shutdown_connections 2024-04-25T10:15:06.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.308+0000 7f6b57000640 1 -- 172.21.15.87:0/2604308717 wait complete. 2024-04-25T10:15:06.350 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:15:06.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b0a400640 1 Processor -- start 2024-04-25T10:15:06.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b0a400640 1 -- start start 2024-04-25T10:15:06.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b0a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b040aef80 0x7f3b0415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b0a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b04070350 0x7f3b0415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b0a400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0415b080 con 0x7f3b0415bac0 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b0a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0415adb0 con 0x7f3b04070350 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b0a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0415eb60 con 0x7f3b040aef80 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b03e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3b0415bac0 0x7f3b040b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41732/0 (socket says 172.21.15.87:41732) 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.484+0000 7f3b03e00640 1 -- 172.21.15.87:0/3545152601 learned_addr learned my addr 172.21.15.87:0/3545152601 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b03400640 1 --2- 172.21.15.87:0/3545152601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b040aef80 0x7f3b0415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b02a00640 1 --2- 172.21.15.87:0/3545152601 >> v2:172.21.15.169:3300/0 conn(0x7f3b04070350 0x7f3b0415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b02000640 1 -- 172.21.15.87:0/3545152601 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891462422 0 0) 0x7f3b0415b080 con 0x7f3b0415bac0 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b02000640 1 -- 172.21.15.87:0/3545152601 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae8005740 con 0x7f3b0415bac0 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b03400640 1 -- 172.21.15.87:0/3545152601 >> v2:172.21.15.169:3300/0 conn(0x7f3b04070350 msgr2=0x7f3b0415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b03400640 1 --2- 172.21.15.87:0/3545152601 >> v2:172.21.15.169:3300/0 conn(0x7f3b04070350 0x7f3b0415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b03400640 1 -- 172.21.15.87:0/3545152601 >> v1:172.21.15.87:6789/0 conn(0x7f3b0415bac0 legacy=0x7f3b040b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b03400640 1 -- 172.21.15.87:0/3545152601 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0415ee40 con 0x7f3b040aef80 2024-04-25T10:15:06.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b03400640 1 --2- 172.21.15.87:0/3545152601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b040aef80 0x7f3b0415e480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009d90 tx=0x7f3af401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dd8c1170b773790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.488+0000 7f3b02000640 1 -- 172.21.15.87:0/3545152601 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af40b2070 con 0x7f3b040aef80 2024-04-25T10:15:06.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b02000640 1 -- 172.21.15.87:0/3545152601 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af40a5a10 con 0x7f3b040aef80 2024-04-25T10:15:06.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b02000640 1 -- 172.21.15.87:0/3545152601 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af40a5de0 con 0x7f3b040aef80 2024-04-25T10:15:06.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- 172.21.15.87:0/3545152601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b040aef80 msgr2=0x7f3b0415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 --2- 172.21.15.87:0/3545152601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b040aef80 0x7f3b0415e480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009d90 tx=0x7f3af401c970 comp rx=0 tx=0).stop 2024-04-25T10:15:06.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- 172.21.15.87:0/3545152601 shutdown_connections 2024-04-25T10:15:06.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 --2- 172.21.15.87:0/3545152601 >> v2:172.21.15.169:3300/0 conn(0x7f3b04070350 0x7f3b0415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 --2- 172.21.15.87:0/3545152601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b040aef80 0x7f3b0415e480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- 172.21.15.87:0/3545152601 >> 172.21.15.87:0/3545152601 conn(0x7f3b040669a0 msgr2=0x7f3b04066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- 172.21.15.87:0/3545152601 shutdown_connections 2024-04-25T10:15:06.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- 172.21.15.87:0/3545152601 wait complete. 2024-04-25T10:15:06.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 Processor -- start 2024-04-25T10:15:06.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- start start 2024-04-25T10:15:06.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b04070350 0x7f3b04154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b040aef80 0x7f3b04157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0415bac0 0x7f3b041430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0406c250 con 0x7f3b0415bac0 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0406c0d0 con 0x7f3b040aef80 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b0a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0406c3d0 con 0x7f3b04070350 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.492+0000 7f3b03e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0415bac0 0x7f3b041430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0415bac0 0x7f3b041430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53622/0 (socket says 172.21.15.87:53622) 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 -- 172.21.15.87:0/2142128964 learned_addr learned my addr 172.21.15.87:0/2142128964 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b02a00640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b040aef80 0x7f3b04157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b04070350 0x7f3b04154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 -- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b04070350 msgr2=0x7f3b04154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b04070350 0x7f3b04154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 -- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b040aef80 msgr2=0x7f3b04157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b040aef80 0x7f3b04157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 -- 172.21.15.87:0/2142128964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040b0ea0 con 0x7f3b0415bac0 2024-04-25T10:15:06.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b03e00640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0415bac0 0x7f3b041430e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c3c0 tx=0x7f3afc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3afb400640 1 -- 172.21.15.87:0/2142128964 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc017070 con 0x7f3b0415bac0 2024-04-25T10:15:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3afb400640 1 -- 172.21.15.87:0/2142128964 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc005cf0 con 0x7f3b0415bac0 2024-04-25T10:15:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b040b1180 con 0x7f3b0415bac0 2024-04-25T10:15:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3afb400640 1 -- 172.21.15.87:0/2142128964 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc007310 con 0x7f3b0415bac0 2024-04-25T10:15:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b040b1740 con 0x7f3b0415bac0 2024-04-25T10:15:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0404ef30 con 0x7f3b0415bac0 2024-04-25T10:15:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.496+0000 7f3afb400640 1 -- 172.21.15.87:0/2142128964 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3afc01a070 con 0x7f3b0415bac0 2024-04-25T10:15:06.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.500+0000 7f3afb400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ae4066f60 0x7f3ae4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.500+0000 7f3afb400640 1 -- 172.21.15.87:0/2142128964 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3afc0845e0 con 0x7f3b0415bac0 2024-04-25T10:15:06.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.500+0000 7f3b03400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ae4066f60 0x7f3ae4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.500+0000 7f3b03400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ae4066f60 0x7f3ae4069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3af40af000 tx=0x7f3af40a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.500+0000 7f3afb400640 1 -- 172.21.15.87:0/2142128964 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3afc054fe0 con 0x7f3b0415bac0 2024-04-25T10:15:06.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:06.688+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b040b2fa0 con 0x7f3b0415bac0 2024-04-25T10:15:07.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.328+0000 7f3afb400640 1 -- 172.21.15.87:0/2142128964 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3afc059080 con 0x7f3b0415bac0 2024-04-25T10:15:07.343 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T10:15:07.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ae4066f60 msgr2=0x7f3ae4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ae4066f60 0x7f3ae4069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3af40af000 tx=0x7f3af40a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:07.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0415bac0 msgr2=0x7f3b041430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0415bac0 0x7f3b041430e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c3c0 tx=0x7f3afc00c880 comp rx=0 tx=0).stop 2024-04-25T10:15:07.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 shutdown_connections 2024-04-25T10:15:07.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ae4066f60 0x7f3ae4069420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0415bac0 0x7f3b041430e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b040aef80 0x7f3b04157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 --2- 172.21.15.87:0/2142128964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b04070350 0x7f3b04154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 >> 172.21.15.87:0/2142128964 conn(0x7f3b040669a0 msgr2=0x7f3b040b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 shutdown_connections 2024-04-25T10:15:07.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.340+0000 7f3b0a400640 1 -- 172.21.15.87:0/2142128964 wait complete. 2024-04-25T10:15:07.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:07.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.500+0000 7f3407e00640 1 Processor -- start 2024-04-25T10:15:07.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.500+0000 7f3407e00640 1 -- start start 2024-04-25T10:15:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3407e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 0x7f34000cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3407e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3400071050 con 0x7f34000cfe70 2024-04-25T10:15:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3407e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340015d040 con 0x7f340015b910 2024-04-25T10:15:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3407e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340015bda0 con 0x7f34001550b0 2024-04-25T10:15:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3404a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f34000cfe70 0x7f340015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56138/0 (socket says 172.21.15.87:56138) 2024-04-25T10:15:07.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3404a00640 1 -- 172.21.15.87:0/3789218908 learned_addr learned my addr 172.21.15.87:0/3789218908 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405400640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 0x7f34000cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392399086 0 0) 0x7f3400071050 con 0x7f34000cfe70 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e8003530 con 0x7f34000cfe70 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3901167664 0 0) 0x7f33e8003530 con 0x7f34000cfe70 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 msgr2=0x7f34000cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 0x7f34000cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 >> v1:172.21.15.169:6789/0 conn(0x7f340015b910 legacy=0x7f34000b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000712f0 con 0x7f34000cfe70 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f33ec001680 con 0x7f34000cfe70 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 >> v1:172.21.15.87:6789/0 conn(0x7f34000cfe70 legacy=0x7f340015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e80039f0 0x7f33e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 0x7f33e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e8004d50 0x7f33e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e800ba80 con 0x7f33e8004d50 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e800bc00 con 0x7f33e80039f0 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e800e010 con 0x7f34001550b0 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405400640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e80039f0 0x7f33e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405e00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e8004d50 0x7f33e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3404a00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 0x7f33e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405e00640 1 -- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 msgr2=0x7f33e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405e00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34001550b0 0x7f33e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405e00640 1 -- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e80039f0 msgr2=0x7f33e8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405e00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e80039f0 0x7f33e8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.504+0000 7f3405e00640 1 -- 172.21.15.87:0/3789218908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e8003530 con 0x7f33e8004d50 2024-04-25T10:15:07.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3405e00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e8004d50 0x7f33e80051d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f33f8001e40 tx=0x7f33f801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319cfbf2ac4ea1c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8001a90 con 0x7f33e8004d50 2024-04-25T10:15:07.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f8003b80 con 0x7f33e8004d50 2024-04-25T10:15:07.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f33ffe00640 1 -- 172.21.15.87:0/3789218908 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8005080 con 0x7f33e8004d50 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3407e00640 1 -- 172.21.15.87:0/3789218908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e8004d50 msgr2=0x7f33e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3407e00640 1 --2- 172.21.15.87:0/3789218908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e8004d50 0x7f33e80051d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f33f8001e40 tx=0x7f33f801c760 comp rx=0 tx=0).stop 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3405e00640 1 -- 172.21.15.87:0/3789218908 reap_dead start 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3407e00640 1 -- 172.21.15.87:0/3789218908 shutdown_connections 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3407e00640 1 -- 172.21.15.87:0/3789218908 >> 172.21.15.87:0/3789218908 conn(0x7f3400066b70 msgr2=0x7f3400066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3407e00640 1 -- 172.21.15.87:0/3789218908 shutdown_connections 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.508+0000 7f3407e00640 1 -- 172.21.15.87:0/3789218908 wait complete. 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 Processor -- start 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 -- start start 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e8004d50 0x7f3400155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3400158190 0x7f3400159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34001e3100 0x7f340015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3400071b90 con 0x7f34001e3100 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000718b0 con 0x7f3400158190 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000720b0 con 0x7f33e8004d50 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34001e3100 0x7f340015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3404a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3400158190 0x7f3400159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e8004d50 0x7f3400155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e8004d50 0x7f3400155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:57544/0 (socket says 172.21.15.87:57544) 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 -- 172.21.15.87:0/1245617843 learned_addr learned my addr 172.21.15.87:0/1245617843 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 -- 172.21.15.87:0/1245617843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3400158190 msgr2=0x7f3400159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3400158190 0x7f3400159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 -- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34001e3100 msgr2=0x7f340015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34001e3100 0x7f340015a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 -- 172.21.15.87:0/1245617843 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34001563e0 con 0x7f33e8004d50 2024-04-25T10:15:07.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405e00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34001e3100 0x7f340015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:07.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3405400640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e8004d50 0x7f3400155ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011a30 tx=0x7f33f4011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f33fd600640 1 -- 172.21.15.87:0/1245617843 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400fa90 con 0x7f33e8004d50 2024-04-25T10:15:07.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f33fd600640 1 -- 172.21.15.87:0/1245617843 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f400fc30 con 0x7f33e8004d50 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34000cc940 con 0x7f33e8004d50 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f33fd600640 1 -- 172.21.15.87:0/1245617843 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4014440 con 0x7f33e8004d50 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34000ccf00 con 0x7f33e8004d50 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.512+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3400155ca0 con 0x7f33e8004d50 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.516+0000 7f33fd600640 1 -- 172.21.15.87:0/1245617843 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33f4012040 con 0x7f33e8004d50 2024-04-25T10:15:07.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.516+0000 7f33fd600640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33e0066f90 0x7f33e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.516+0000 7f33fd600640 1 -- 172.21.15.87:0/1245617843 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33f40855d0 con 0x7f33e8004d50 2024-04-25T10:15:07.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.516+0000 7f3404a00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33e0066f90 0x7f33e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.516+0000 7f3404a00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33e0066f90 0x7f33e0069450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34000b2800 tx=0x7f33ec0019b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.520+0000 7f33fd600640 1 -- 172.21.15.87:0/1245617843 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33f4019030 con 0x7f33e8004d50 2024-04-25T10:15:07.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.676+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f340004ef30 con 0x7f33e8004d50 2024-04-25T10:15:07.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.676+0000 7f33fd600640 1 -- 172.21.15.87:0/1245617843 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16614 (secure 0 0 0) 0x7f33f4056610 con 0x7f33e8004d50 2024-04-25T10:15:07.682 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":45,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:06.691544+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:05.052743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:05.692130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.676+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33e0066f90 msgr2=0x7f33e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33e0066f90 0x7f33e0069450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34000b2800 tx=0x7f33ec0019b0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e8004d50 msgr2=0x7f3400155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e8004d50 0x7f3400155ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011a30 tx=0x7f33f4011e30 comp rx=0 tx=0).stop 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 shutdown_connections 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33e0066f90 0x7f33e0069450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34001e3100 0x7f340015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3400158190 0x7f3400159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 --2- 172.21.15.87:0/1245617843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e8004d50 0x7f3400155ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 >> 172.21.15.87:0/1245617843 conn(0x7f3400066b70 msgr2=0x7f34001e91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 shutdown_connections 2024-04-25T10:15:07.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.680+0000 7f3407e00640 1 -- 172.21.15.87:0/1245617843 wait complete. 2024-04-25T10:15:07.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eea00640 1 Processor -- start 2024-04-25T10:15:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eea00640 1 -- start start 2024-04-25T10:15:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 0x7f20e806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e806f200 con 0x7f20e80afda0 2024-04-25T10:15:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e80ccc10 con 0x7f20e80af9c0 2024-04-25T10:15:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e806d380 con 0x7f20e80670b0 2024-04-25T10:15:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20df400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f20e80afda0 0x7f20e80669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56140/0 (socket says 172.21.15.87:56140) 2024-04-25T10:15:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20df400640 1 -- 172.21.15.87:0/2773628827 learned_addr learned my addr 172.21.15.87:0/2773628827 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eca00640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 0x7f20e806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118632098 0 0) 0x7f20d0001120 con 0x7f20e80afda0 2024-04-25T10:15:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20cc003530 con 0x7f20e80afda0 2024-04-25T10:15:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681565157 0 0) 0x7f20e80ccc10 con 0x7f20e80af9c0 2024-04-25T10:15:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d0001120 con 0x7f20e80af9c0 2024-04-25T10:15:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036855352 0 0) 0x7f20cc003530 con 0x7f20e80afda0 2024-04-25T10:15:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 msgr2=0x7f20e806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 0x7f20e806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 >> v1:172.21.15.169:6789/0 conn(0x7f20e80af9c0 legacy=0x7f20e80cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80ccef0 con 0x7f20e80afda0 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f20d0001870 con 0x7f20e80afda0 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20eca00640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 0x7f20e806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 >> v1:172.21.15.87:6789/0 conn(0x7f20e80afda0 legacy=0x7f20e80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc003b80 0x7f20cc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20cc004810 0x7f20cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 0x7f20cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e806d380 con 0x7f20cc004810 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc00bae0 con 0x7f20cc003b80 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc00bce0 con 0x7f20e80670b0 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.836+0000 7f20df400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20cc004810 0x7f20cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20e7e00640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc003b80 0x7f20cc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eca00640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 0x7f20cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20df400640 1 -- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 msgr2=0x7f20cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20df400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80670b0 0x7f20cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20df400640 1 -- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc003b80 msgr2=0x7f20cc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20df400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc003b80 0x7f20cc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20df400640 1 -- 172.21.15.87:0/2773628827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc003530 con 0x7f20cc004810 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20df400640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20cc004810 0x7f20cc004c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f20e80b01d0 tx=0x7f20d001cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade68f5f40085e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d000c0e0 con 0x7f20cc004810 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d00a9020 con 0x7f20cc004810 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20e7400640 1 -- 172.21.15.87:0/2773628827 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0005b00 con 0x7f20cc004810 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eea00640 1 -- 172.21.15.87:0/2773628827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20cc004810 msgr2=0x7f20cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eea00640 1 --2- 172.21.15.87:0/2773628827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20cc004810 0x7f20cc004c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f20e80b01d0 tx=0x7f20d001cb50 comp rx=0 tx=0).stop 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eca00640 1 -- 172.21.15.87:0/2773628827 reap_dead start 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eea00640 1 -- 172.21.15.87:0/2773628827 shutdown_connections 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eea00640 1 -- 172.21.15.87:0/2773628827 >> 172.21.15.87:0/2773628827 conn(0x7f20e80b2410 msgr2=0x7f20e80b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eea00640 1 -- 172.21.15.87:0/2773628827 shutdown_connections 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.840+0000 7f20eea00640 1 -- 172.21.15.87:0/2773628827 wait complete. 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 Processor -- start 2024-04-25T10:15:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 -- start start 2024-04-25T10:15:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc004810 0x7f20e8157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80c7b60 0x7f20e8156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20e8157290 0x7f20e815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e8160250 con 0x7f20e8157290 2024-04-25T10:15:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81600d0 con 0x7f20cc004810 2024-04-25T10:15:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81603d0 con 0x7f20e80c7b60 2024-04-25T10:15:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20e8157290 0x7f20e815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20e8157290 0x7f20e815cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54722/0 (socket says 172.21.15.87:54722) 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 -- 172.21.15.87:0/2449607210 learned_addr learned my addr 172.21.15.87:0/2449607210 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20e7e00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc004810 0x7f20e8157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20df400640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80c7b60 0x7f20e8156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 -- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80c7b60 msgr2=0x7f20e8156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80c7b60 0x7f20e8156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 -- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc004810 msgr2=0x7f20e8157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc004810 0x7f20e8157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 -- 172.21.15.87:0/2449607210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80c8100 con 0x7f20e8157290 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eca00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20e8157290 0x7f20e815cd40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f20cc004790 tx=0x7f20e0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20e4c00640 1 -- 172.21.15.87:0/2449607210 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000fdb0 con 0x7f20e8157290 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20e4c00640 1 -- 172.21.15.87:0/2449607210 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e00039a0 con 0x7f20e8157290 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e80c83e0 con 0x7f20e8157290 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20e4c00640 1 -- 172.21.15.87:0/2449607210 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0003d90 con 0x7f20e8157290 2024-04-25T10:15:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.844+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e80c89a0 con 0x7f20e8157290 2024-04-25T10:15:07.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.848+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e804ef30 con 0x7f20e8157290 2024-04-25T10:15:07.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.848+0000 7f20e4c00640 1 -- 172.21.15.87:0/2449607210 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20e001f020 con 0x7f20e8157290 2024-04-25T10:15:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.848+0000 7f20e4c00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20c4066f90 0x7f20c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.848+0000 7f20e4c00640 1 -- 172.21.15.87:0/2449607210 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e0085410 con 0x7f20e8157290 2024-04-25T10:15:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.848+0000 7f20e7e00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20c4066f90 0x7f20c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.848+0000 7f20e7e00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20c4066f90 0x7f20c4069450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c10 tx=0x7f20d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:07.852+0000 7f20e4c00640 1 -- 172.21.15.87:0/2449607210 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20e0010040 con 0x7f20e8157290 2024-04-25T10:15:08.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.016+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20e8071420 con 0x7f20e8157290 2024-04-25T10:15:08.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20e4c00640 1 -- 172.21.15.87:0/2449607210 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16614 (secure 0 0 0) 0x7f20e0056270 con 0x7f20e8157290 2024-04-25T10:15:08.024 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:08.024 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":45,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:06.691544+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:05.052743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:05.692130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20c4066f90 msgr2=0x7f20c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20c4066f90 0x7f20c4069450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c10 tx=0x7f20d8009040 comp rx=0 tx=0).stop 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20e8157290 msgr2=0x7f20e815cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20e8157290 0x7f20e815cd40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f20cc004790 tx=0x7f20e0009f90 comp rx=0 tx=0).stop 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 shutdown_connections 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20c4066f90 0x7f20c4069450 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20e8157290 0x7f20e815cd40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e80c7b60 0x7f20e8156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 --2- 172.21.15.87:0/2449607210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20cc004810 0x7f20e8157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 >> 172.21.15.87:0/2449607210 conn(0x7f20e80b2410 msgr2=0x7f20e80cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 shutdown_connections 2024-04-25T10:15:08.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.020+0000 7f20eea00640 1 -- 172.21.15.87:0/2449607210 wait complete. 2024-04-25T10:15:08.046 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:15:08.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facb6000640 1 Processor -- start 2024-04-25T10:15:08.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facb6000640 1 -- start start 2024-04-25T10:15:08.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facb6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 0x7facb0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facb6000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb0159920 con 0x7facb0072020 2024-04-25T10:15:08.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facb6000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb015c2f0 con 0x7facb0159540 2024-04-25T10:15:08.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facb6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb015c5d0 con 0x7facb00aeda0 2024-04-25T10:15:08.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facafe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7facb0072020 0x7facb0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56144/0 (socket says 172.21.15.87:56144) 2024-04-25T10:15:08.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facafe00640 1 -- 172.21.15.87:0/3242421887 learned_addr learned my addr 172.21.15.87:0/3242421887 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facaea00640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 0x7facb0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144604503 0 0) 0x7facb0159920 con 0x7facb0072020 2024-04-25T10:15:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac94003530 con 0x7facb0072020 2024-04-25T10:15:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100643657 0 0) 0x7fac94003530 con 0x7facb0072020 2024-04-25T10:15:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 msgr2=0x7facb0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:08.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 0x7facb0071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 >> v1:172.21.15.169:6789/0 conn(0x7facb0159540 legacy=0x7facb00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb015c8b0 con 0x7facb0072020 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7faca8001a00 con 0x7facb0072020 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 >> v1:172.21.15.87:6789/0 conn(0x7facb0072020 legacy=0x7facb0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac940039f0 0x7fac94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940043a0 0x7fac94004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 0x7fac940050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb015c5d0 con 0x7fac940039f0 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9400ba00 con 0x7fac940043a0 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.180+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9400bc00 con 0x7facb00aeda0 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaf400640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac940039f0 0x7fac94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaea00640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940043a0 0x7fac94004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facafe00640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 0x7fac940050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaf400640 1 -- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 msgr2=0x7fac940050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:08.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaf400640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb00aeda0 0x7fac940050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaf400640 1 -- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940043a0 msgr2=0x7fac94004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:08.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaf400640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940043a0 0x7fac94004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaf400640 1 -- 172.21.15.87:0/3242421887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac94003530 con 0x7fac940039f0 2024-04-25T10:15:08.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facaf400640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac940039f0 0x7fac94003dd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faca0001860 tx=0x7faca001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ee61dafefc12df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00b0070 con 0x7fac940039f0 2024-04-25T10:15:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00ab600 con 0x7fac940039f0 2024-04-25T10:15:08.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.184+0000 7facae000640 1 -- 172.21.15.87:0/3242421887 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0001460 con 0x7fac940039f0 2024-04-25T10:15:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- 172.21.15.87:0/3242421887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac940039f0 msgr2=0x7fac94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 --2- 172.21.15.87:0/3242421887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac940039f0 0x7fac94003dd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faca0001860 tx=0x7faca001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 -- 172.21.15.87:0/3242421887 reap_dead start 2024-04-25T10:15:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- 172.21.15.87:0/3242421887 shutdown_connections 2024-04-25T10:15:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- 172.21.15.87:0/3242421887 >> 172.21.15.87:0/3242421887 conn(0x7facb00669a0 msgr2=0x7facb0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- 172.21.15.87:0/3242421887 shutdown_connections 2024-04-25T10:15:08.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- 172.21.15.87:0/3242421887 wait complete. 2024-04-25T10:15:08.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 Processor -- start 2024-04-25T10:15:08.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- start start 2024-04-25T10:15:08.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940039f0 0x7facb0160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb014bd90 0x7facb01612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb0156b40 0x7facb0156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb00b2870 con 0x7facb0156b40 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb0072430 con 0x7fac940039f0 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facb6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb00b29f0 con 0x7facb014bd90 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb0156b40 0x7facb0156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb0156b40 0x7facb0156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54738/0 (socket says 172.21.15.87:54738) 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 -- 172.21.15.87:0/861791879 learned_addr learned my addr 172.21.15.87:0/861791879 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 -- 172.21.15.87:0/861791879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb014bd90 msgr2=0x7facb01612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb014bd90 0x7facb01612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 -- 172.21.15.87:0/861791879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940039f0 msgr2=0x7facb0160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940039f0 0x7facb0160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 -- 172.21.15.87:0/861791879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb0158c30 con 0x7facb0156b40 2024-04-25T10:15:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.188+0000 7facafe00640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb0156b40 0x7facb0156ff0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7facb00b4ab0 tx=0x7faca800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7faca7400640 1 -- 172.21.15.87:0/861791879 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca800ad90 con 0x7facb0156b40 2024-04-25T10:15:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7faca7400640 1 -- 172.21.15.87:0/861791879 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca8018070 con 0x7facb0156b40 2024-04-25T10:15:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb01578e0 con 0x7facb0156b40 2024-04-25T10:15:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7faca7400640 1 -- 172.21.15.87:0/861791879 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca8013630 con 0x7facb0156b40 2024-04-25T10:15:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb0158150 con 0x7facb0156b40 2024-04-25T10:15:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb004ef30 con 0x7facb0156b40 2024-04-25T10:15:08.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7faca7400640 1 -- 172.21.15.87:0/861791879 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faca80137d0 con 0x7facb0156b40 2024-04-25T10:15:08.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7faca7400640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fac90066f90 0x7fac90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7faca7400640 1 -- 172.21.15.87:0/861791879 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faca80857a0 con 0x7facb0156b40 2024-04-25T10:15:08.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.192+0000 7facaf400640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fac90066f90 0x7fac90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.196+0000 7facaf400640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fac90066f90 0x7fac90069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faca00ad000 tx=0x7faca0001240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.196+0000 7faca7400640 1 -- 172.21.15.87:0/861791879 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faca8056860 con 0x7facb0156b40 2024-04-25T10:15:08.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:08.392+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7facb00703f0 con 0x7facb0156b40 2024-04-25T10:15:09.302 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T10:15:09.295+0000 7f6ac6a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:09.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.340+0000 7faca7400640 1 -- 172.21.15.87:0/861791879 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7faca8056160 con 0x7facb0156b40 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fac90066f90 msgr2=0x7fac90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fac90066f90 0x7fac90069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faca00ad000 tx=0x7faca0001240 comp rx=0 tx=0).stop 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb0156b40 msgr2=0x7facb0156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb0156b40 0x7facb0156ff0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7facb00b4ab0 tx=0x7faca800a730 comp rx=0 tx=0).stop 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 shutdown_connections 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fac90066f90 0x7fac90069450 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb0156b40 0x7facb0156ff0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb014bd90 0x7facb01612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 --2- 172.21.15.87:0/861791879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac940039f0 0x7facb0160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 >> 172.21.15.87:0/861791879 conn(0x7facb00669a0 msgr2=0x7facb00b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 shutdown_connections 2024-04-25T10:15:09.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.344+0000 7facb6000640 1 -- 172.21.15.87:0/861791879 wait complete. 2024-04-25T10:15:09.370 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:15:09.370 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:15:09.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.504+0000 7f1d5c400640 1 Processor -- start 2024-04-25T10:15:09.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.504+0000 7f1d5c400640 1 -- start start 2024-04-25T10:15:09.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d540b38d0 0x7f1d5415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5c400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5415b080 con 0x7f1d540aef80 2024-04-25T10:15:09.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5415adb0 con 0x7f1d54070350 2024-04-25T10:15:09.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5415eb60 con 0x7f1d540b38d0 2024-04-25T10:15:09.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5a400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d540b38d0 0x7f1d5415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d59a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1d540aef80 0x7f1d5415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56154/0 (socket says 172.21.15.87:56154) 2024-04-25T10:15:09.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d59a00640 1 -- 172.21.15.87:0/2972092229 learned_addr learned my addr 172.21.15.87:0/2972092229 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:09.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5a400640 1 -- 172.21.15.87:0/2972092229 >> v1:172.21.15.169:6789/0 conn(0x7f1d54070350 legacy=0x7f1d540b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5a400640 1 -- 172.21.15.87:0/2972092229 >> v1:172.21.15.87:6789/0 conn(0x7f1d540aef80 legacy=0x7f1d5415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d5a400640 1 -- 172.21.15.87:0/2972092229 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5415ee40 con 0x7f1d540b38d0 2024-04-25T10:15:09.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.508+0000 7f1d53e00640 1 -- 172.21.15.87:0/2972092229 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000499405 0 0) 0x7f1d5415b080 con 0x7f1d540aef80 2024-04-25T10:15:09.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5a400640 1 --2- 172.21.15.87:0/2972092229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b38d0 0x7f1d5415e480 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0097c0 tx=0x7f1d4c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f121d30a03757e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d53e00640 1 -- 172.21.15.87:0/2972092229 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0ab640 con 0x7f1d540b38d0 2024-04-25T10:15:09.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d53e00640 1 -- 172.21.15.87:0/2972092229 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c0b0070 con 0x7f1d540b38d0 2024-04-25T10:15:09.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d53e00640 1 -- 172.21.15.87:0/2972092229 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0aba00 con 0x7f1d540b38d0 2024-04-25T10:15:09.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 -- 172.21.15.87:0/2972092229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b38d0 msgr2=0x7f1d5415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 --2- 172.21.15.87:0/2972092229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b38d0 0x7f1d5415e480 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0097c0 tx=0x7f1d4c01c460 comp rx=0 tx=0).stop 2024-04-25T10:15:09.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 -- 172.21.15.87:0/2972092229 shutdown_connections 2024-04-25T10:15:09.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 --2- 172.21.15.87:0/2972092229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540b38d0 0x7f1d5415e480 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 -- 172.21.15.87:0/2972092229 >> 172.21.15.87:0/2972092229 conn(0x7f1d540669a0 msgr2=0x7f1d54066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 -- 172.21.15.87:0/2972092229 shutdown_connections 2024-04-25T10:15:09.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 -- 172.21.15.87:0/2972092229 wait complete. 2024-04-25T10:15:09.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 Processor -- start 2024-04-25T10:15:09.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.512+0000 7f1d5c400640 1 -- start start 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d54070350 0x7f1d54156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540aef80 0x7f1d54151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d54150710 0x7f1d5414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d541689b0 con 0x7f1d540aef80 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d54168830 con 0x7f1d54070350 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d54168b30 con 0x7f1d54150710 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540aef80 0x7f1d54151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540aef80 0x7f1d54151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54756/0 (socket says 172.21.15.87:54756) 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 -- 172.21.15.87:0/1617629741 learned_addr learned my addr 172.21.15.87:0/1617629741 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d54070350 0x7f1d54156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5a400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d54150710 0x7f1d5414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 -- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d54150710 msgr2=0x7f1d5414ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d54150710 0x7f1d5414ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 -- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d54070350 msgr2=0x7f1d54156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d54070350 0x7f1d54156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 -- 172.21.15.87:0/1617629741 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54150c80 con 0x7f1d540aef80 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d54070350 0x7f1d54156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5a400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d54150710 0x7f1d5414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d59000640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540aef80 0x7f1d54151d20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c490 tx=0x7f1d40001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d51600640 1 -- 172.21.15.87:0/1617629741 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40017070 con 0x7f1d540aef80 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d51600640 1 -- 172.21.15.87:0/1617629741 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d40003600 con 0x7f1d540aef80 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d54150f60 con 0x7f1d540aef80 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d51600640 1 -- 172.21.15.87:0/1617629741 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40003a30 con 0x7f1d540aef80 2024-04-25T10:15:09.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d54151520 con 0x7f1d540aef80 2024-04-25T10:15:09.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5404ef30 con 0x7f1d540aef80 2024-04-25T10:15:09.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.516+0000 7f1d51600640 1 -- 172.21.15.87:0/1617629741 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d4001a070 con 0x7f1d540aef80 2024-04-25T10:15:09.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.520+0000 7f1d51600640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d34066f90 0x7f1d34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.520+0000 7f1d51600640 1 -- 172.21.15.87:0/1617629741 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d40084540 con 0x7f1d540aef80 2024-04-25T10:15:09.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.520+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d34066f90 0x7f1d34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.520+0000 7f1d59a00640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d34066f90 0x7f1d34069450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007fd0 tx=0x7f1d48009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.524+0000 7f1d51600640 1 -- 172.21.15.87:0/1617629741 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d400554b0 con 0x7f1d540aef80 2024-04-25T10:15:09.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:09.708+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1d540b2fa0 con 0x7f1d540aef80 2024-04-25T10:15:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d51600640 1 -- 172.21.15.87:0/1617629741 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1d40059e20 con 0x7f1d540aef80 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d34066f90 msgr2=0x7f1d34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d34066f90 0x7f1d34069450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007fd0 tx=0x7f1d48009040 comp rx=0 tx=0).stop 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540aef80 msgr2=0x7f1d54151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540aef80 0x7f1d54151d20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c490 tx=0x7f1d40001e80 comp rx=0 tx=0).stop 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 shutdown_connections 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d34066f90 0x7f1d34069450 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d54150710 0x7f1d5414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d540aef80 0x7f1d54151d20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 --2- 172.21.15.87:0/1617629741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d54070350 0x7f1d54156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 >> 172.21.15.87:0/1617629741 conn(0x7f1d540669a0 msgr2=0x7f1d540b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 shutdown_connections 2024-04-25T10:15:10.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.348+0000 7f1d5c400640 1 -- 172.21.15.87:0/1617629741 wait complete. 2024-04-25T10:15:10.375 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:15:10.376 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:10.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e8000640 1 Processor -- start 2024-04-25T10:15:10.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e8000640 1 -- start start 2024-04-25T10:15:10.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e8000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32e00d0040 0x7f32e015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e8000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e0071050 con 0x7f32e01550b0 2024-04-25T10:15:10.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e8000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e015d040 con 0x7f32e015b740 2024-04-25T10:15:10.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e8000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e00712f0 con 0x7f32e00d0040 2024-04-25T10:15:10.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e5600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f32e01550b0 0x7f32e00cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56164/0 (socket says 172.21.15.87:56164) 2024-04-25T10:15:10.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e5600640 1 -- 172.21.15.87:0/661452789 learned_addr learned my addr 172.21.15.87:0/661452789 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:10.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e4c00640 1 --2- 172.21.15.87:0/661452789 >> v2:172.21.15.87:3300/0 conn(0x7f32e00d0040 0x7f32e015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32dfe00640 1 -- 172.21.15.87:0/661452789 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514227374 0 0) 0x7f32e0071050 con 0x7f32e01550b0 2024-04-25T10:15:10.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32dfe00640 1 -- 172.21.15.87:0/661452789 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c8005740 con 0x7f32e01550b0 2024-04-25T10:15:10.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e4c00640 1 -- 172.21.15.87:0/661452789 >> v1:172.21.15.169:6789/0 conn(0x7f32e015b740 legacy=0x7f32e00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32dfe00640 1 -- 172.21.15.87:0/661452789 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091184469 0 0) 0x7f32e015d040 con 0x7f32e015b740 2024-04-25T10:15:10.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e4c00640 1 -- 172.21.15.87:0/661452789 >> v1:172.21.15.87:6789/0 conn(0x7f32e01550b0 legacy=0x7f32e00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.508+0000 7f32e4c00640 1 -- 172.21.15.87:0/661452789 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00715d0 con 0x7f32e00d0040 2024-04-25T10:15:10.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e4c00640 1 --2- 172.21.15.87:0/661452789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e015b200 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097c0 tx=0x7f32cc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5a11bf9d202b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32dfe00640 1 -- 172.21.15.87:0/661452789 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0b0070 con 0x7f32e00d0040 2024-04-25T10:15:10.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32dfe00640 1 -- 172.21.15.87:0/661452789 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc0ab600 con 0x7f32e00d0040 2024-04-25T10:15:10.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32dfe00640 1 -- 172.21.15.87:0/661452789 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0ab9d0 con 0x7f32e00d0040 2024-04-25T10:15:10.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e8000640 1 -- 172.21.15.87:0/661452789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 msgr2=0x7f32e015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e8000640 1 --2- 172.21.15.87:0/661452789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e015b200 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097c0 tx=0x7f32cc01c470 comp rx=0 tx=0).stop 2024-04-25T10:15:10.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e8000640 1 -- 172.21.15.87:0/661452789 shutdown_connections 2024-04-25T10:15:10.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e8000640 1 --2- 172.21.15.87:0/661452789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e015b200 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e8000640 1 -- 172.21.15.87:0/661452789 >> 172.21.15.87:0/661452789 conn(0x7f32e0066b70 msgr2=0x7f32e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e8000640 1 -- 172.21.15.87:0/661452789 shutdown_connections 2024-04-25T10:15:10.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.512+0000 7f32e8000640 1 -- 172.21.15.87:0/661452789 wait complete. 2024-04-25T10:15:10.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 Processor -- start 2024-04-25T10:15:10.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 -- start start 2024-04-25T10:15:10.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e01510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e01550b0 0x7f32e0152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e015b740 0x7f32e013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e015bcd0 con 0x7f32e00d0040 2024-04-25T10:15:10.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e015bb50 con 0x7f32e015b740 2024-04-25T10:15:10.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e006c090 con 0x7f32e01550b0 2024-04-25T10:15:10.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e01510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e01510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54776/0 (socket says 172.21.15.87:54776) 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 -- 172.21.15.87:0/3169262312 learned_addr learned my addr 172.21.15.87:0/3169262312 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e4c00640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e01550b0 0x7f32e0152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e6000640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e015b740 0x7f32e013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 -- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e01550b0 msgr2=0x7f32e0152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e01550b0 0x7f32e0152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 -- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e015b740 msgr2=0x7f32e013fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e015b740 0x7f32e013fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 -- 172.21.15.87:0/3169262312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0153820 con 0x7f32e00d0040 2024-04-25T10:15:10.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e5600640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e01510b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c450 tx=0x7f32d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32dd600640 1 -- 172.21.15.87:0/3169262312 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4017070 con 0x7f32e00d0040 2024-04-25T10:15:10.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32dd600640 1 -- 172.21.15.87:0/3169262312 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d4005cf0 con 0x7f32e00d0040 2024-04-25T10:15:10.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32dd600640 1 -- 172.21.15.87:0/3169262312 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4007290 con 0x7f32e00d0040 2024-04-25T10:15:10.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e0153b00 con 0x7f32e00d0040 2024-04-25T10:15:10.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e01540c0 con 0x7f32e00d0040 2024-04-25T10:15:10.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.516+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e004ef30 con 0x7f32e00d0040 2024-04-25T10:15:10.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.520+0000 7f32dd600640 1 -- 172.21.15.87:0/3169262312 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32d401a070 con 0x7f32e00d0040 2024-04-25T10:15:10.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.520+0000 7f32dd600640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f32c0066f60 0x7f32c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.520+0000 7f32dd600640 1 -- 172.21.15.87:0/3169262312 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32d40843a0 con 0x7f32e00d0040 2024-04-25T10:15:10.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.520+0000 7f32e4c00640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f32c0066f60 0x7f32c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.520+0000 7f32e4c00640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f32c0066f60 0x7f32c0069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0ad000 tx=0x7f32cc0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.524+0000 7f32dd600640 1 -- 172.21.15.87:0/3169262312 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32d4054d50 con 0x7f32e00d0040 2024-04-25T10:15:10.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.708+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32e01547e0 con 0x7f32e00d0040 2024-04-25T10:15:10.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.708+0000 7f32dd600640 1 -- 172.21.15.87:0/3169262312 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f32d4058df0 con 0x7f32e00d0040 2024-04-25T10:15:10.714 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:10.714 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f32c0066f60 msgr2=0x7f32c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f32c0066f60 0x7f32c0069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0ad000 tx=0x7f32cc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 msgr2=0x7f32e01510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e01510b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c450 tx=0x7f32d400c910 comp rx=0 tx=0).stop 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 shutdown_connections 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f32c0066f60 0x7f32c0069420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e015b740 0x7f32e013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e01550b0 0x7f32e0152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 --2- 172.21.15.87:0/3169262312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32e00d0040 0x7f32e01510b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 >> 172.21.15.87:0/3169262312 conn(0x7f32e0066b70 msgr2=0x7f32e00b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 shutdown_connections 2024-04-25T10:15:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.712+0000 7f32e8000640 1 -- 172.21.15.87:0/3169262312 wait complete. 2024-04-25T10:15:10.735 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:10.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.864+0000 7f931be00640 1 Processor -- start 2024-04-25T10:15:10.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.864+0000 7f931be00640 1 -- start start 2024-04-25T10:15:10.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f931be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f93140c6a20 0x7f9314166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f931be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93140ccfb0 con 0x7f93140ccb20 2024-04-25T10:15:10.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f931be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93140cd290 con 0x7f931415b9e0 2024-04-25T10:15:10.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f931be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93140cd570 con 0x7f93140c6a20 2024-04-25T10:15:10.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f9319e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f93140c6a20 0x7f9314166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f9318a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f93140ccb20 0x7f93140d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56176/0 (socket says 172.21.15.87:56176) 2024-04-25T10:15:10.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f9318a00640 1 -- 172.21.15.87:0/489472057 learned_addr learned my addr 172.21.15.87:0/489472057 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:10.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f9319e00640 1 -- 172.21.15.87:0/489472057 >> v1:172.21.15.169:6789/0 conn(0x7f931415b9e0 legacy=0x7f93140cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f9319e00640 1 -- 172.21.15.87:0/489472057 >> v1:172.21.15.87:6789/0 conn(0x7f93140ccb20 legacy=0x7f93140d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.868+0000 7f9319e00640 1 -- 172.21.15.87:0/489472057 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140cd850 con 0x7f93140c6a20 2024-04-25T10:15:10.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f9319e00640 1 --2- 172.21.15.87:0/489472057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93140c6a20 0x7f9314166820 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097c0 tx=0x7f930c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c944fff07ced6bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f9313e00640 1 -- 172.21.15.87:0/489472057 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0b0070 con 0x7f93140c6a20 2024-04-25T10:15:10.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f9313e00640 1 -- 172.21.15.87:0/489472057 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c004230 con 0x7f93140c6a20 2024-04-25T10:15:10.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f9313e00640 1 -- 172.21.15.87:0/489472057 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c003620 con 0x7f93140c6a20 2024-04-25T10:15:10.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- 172.21.15.87:0/489472057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93140c6a20 msgr2=0x7f9314166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 --2- 172.21.15.87:0/489472057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93140c6a20 0x7f9314166820 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097c0 tx=0x7f930c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- 172.21.15.87:0/489472057 shutdown_connections 2024-04-25T10:15:10.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 --2- 172.21.15.87:0/489472057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93140c6a20 0x7f9314166820 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- 172.21.15.87:0/489472057 >> 172.21.15.87:0/489472057 conn(0x7f931409a9d0 msgr2=0x7f931409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- 172.21.15.87:0/489472057 shutdown_connections 2024-04-25T10:15:10.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- 172.21.15.87:0/489472057 wait complete. 2024-04-25T10:15:10.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 Processor -- start 2024-04-25T10:15:10.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- start start 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93140c6a20 0x7f931415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931415b9e0 0x7f931414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f931414f040 0x7f93140c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9314169910 con 0x7f931414f040 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.872+0000 7f931be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9314169790 con 0x7f93140c6a20 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f931be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9314169a90 con 0x7f931415b9e0 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f931414f040 0x7f93140c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93140c6a20 0x7f931415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f931414f040 0x7f93140c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54798/0 (socket says 172.21.15.87:54798) 2024-04-25T10:15:10.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 -- 172.21.15.87:0/2149353755 learned_addr learned my addr 172.21.15.87:0/2149353755 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9318a00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931415b9e0 0x7f931414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 -- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931415b9e0 msgr2=0x7f931414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931415b9e0 0x7f931414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 -- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93140c6a20 msgr2=0x7f931415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93140c6a20 0x7f931415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 -- 172.21.15.87:0/2149353755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140ca940 con 0x7f931414f040 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9318a00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931415b9e0 0x7f931414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:10.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9319e00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f931414f040 0x7f93140c9b10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f930c0ae040 tx=0x7f930c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9311600640 1 -- 172.21.15.87:0/2149353755 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0b0040 con 0x7f931414f040 2024-04-25T10:15:10.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9311600640 1 -- 172.21.15.87:0/2149353755 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c0bb070 con 0x7f931414f040 2024-04-25T10:15:10.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93140cac20 con 0x7f931414f040 2024-04-25T10:15:10.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9311600640 1 -- 172.21.15.87:0/2149353755 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c003e20 con 0x7f931414f040 2024-04-25T10:15:10.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9314152e40 con 0x7f931414f040 2024-04-25T10:15:10.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931404ef30 con 0x7f931414f040 2024-04-25T10:15:10.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.876+0000 7f9311600640 1 -- 172.21.15.87:0/2149353755 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f930c0b3070 con 0x7f931414f040 2024-04-25T10:15:10.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.880+0000 7f9311600640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92f4066f60 0x7f92f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.880+0000 7f9311600640 1 -- 172.21.15.87:0/2149353755 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f930c11ee90 con 0x7f931414f040 2024-04-25T10:15:10.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.880+0000 7f9319400640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92f4066f60 0x7f92f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.880+0000 7f9319400640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92f4066f60 0x7f92f4069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9308001e00 tx=0x7f9308009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:10.884+0000 7f9311600640 1 -- 172.21.15.87:0/2149353755 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f930c016090 con 0x7f931414f040 2024-04-25T10:15:11.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.036+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93140cb7e0 con 0x7f931414f040 2024-04-25T10:15:11.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f9311600640 1 -- 172.21.15.87:0/2149353755 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f930c120090 con 0x7f931414f040 2024-04-25T10:15:11.046 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:11.046 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:08.327470+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:05.052743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:05.692130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:11.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92f4066f60 msgr2=0x7f92f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92f4066f60 0x7f92f4069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9308001e00 tx=0x7f9308009040 comp rx=0 tx=0).stop 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f931414f040 msgr2=0x7f93140c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f931414f040 0x7f93140c9b10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f930c0ae040 tx=0x7f930c019240 comp rx=0 tx=0).stop 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 shutdown_connections 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92f4066f60 0x7f92f4069420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f931414f040 0x7f93140c9b10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931415b9e0 0x7f931414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 --2- 172.21.15.87:0/2149353755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93140c6a20 0x7f931415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 >> 172.21.15.87:0/2149353755 conn(0x7f931409a9d0 msgr2=0x7f93140c5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 shutdown_connections 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.040+0000 7f931be00640 1 -- 172.21.15.87:0/2149353755 wait complete. 2024-04-25T10:15:11.066 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:15:11.067 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.196+0000 7f2f05600640 1 Processor -- start 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.196+0000 7f2f05600640 1 -- start start 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2f05600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f0015bc40 0x7f2f0006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2f05600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2f000cf930 0x7f2f000cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2f05600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f000b2770 con 0x7f2f000cda10 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2f05600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0006c670 con 0x7f2f000cf930 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2f05600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0006c950 con 0x7f2f0015bc40 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2efe000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2f000cda10 0x7f2f000b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56182/0 (socket says 172.21.15.87:56182) 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2efe000640 1 -- 172.21.15.87:0/2099470648 learned_addr learned my addr 172.21.15.87:0/2099470648 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2eff400640 1 --2- 172.21.15.87:0/2099470648 >> v2:172.21.15.87:3300/0 conn(0x7f2f000cf930 0x7f2f000cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2efea00640 1 --2- 172.21.15.87:0/2099470648 >> v2:172.21.15.169:3300/0 conn(0x7f2f0015bc40 0x7f2f0006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2efea00640 1 -- 172.21.15.87:0/2099470648 >> v2:172.21.15.87:3300/0 conn(0x7f2f000cf930 msgr2=0x7f2f000cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2efea00640 1 --2- 172.21.15.87:0/2099470648 >> v2:172.21.15.87:3300/0 conn(0x7f2f000cf930 0x7f2f000cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2efea00640 1 -- 172.21.15.87:0/2099470648 >> v1:172.21.15.87:6789/0 conn(0x7f2f000cda10 legacy=0x7f2f000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.200+0000 7f2efea00640 1 -- 172.21.15.87:0/2099470648 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0006cc30 con 0x7f2f0015bc40 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2efea00640 1 --2- 172.21.15.87:0/2099470648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 0x7f2f0006c090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f000d0800 tx=0x7f2ef001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c541892a0e257ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2efd600640 1 -- 172.21.15.87:0/2099470648 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00b0070 con 0x7f2f0015bc40 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2efd600640 1 -- 172.21.15.87:0/2099470648 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef00ab600 con 0x7f2f0015bc40 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2efd600640 1 -- 172.21.15.87:0/2099470648 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00ab9d0 con 0x7f2f0015bc40 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 -- 172.21.15.87:0/2099470648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 msgr2=0x7f2f0006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 --2- 172.21.15.87:0/2099470648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 0x7f2f0006c090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f000d0800 tx=0x7f2ef001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 -- 172.21.15.87:0/2099470648 shutdown_connections 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 --2- 172.21.15.87:0/2099470648 >> v2:172.21.15.87:3300/0 conn(0x7f2f000cf930 0x7f2f000cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 --2- 172.21.15.87:0/2099470648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 0x7f2f0006c090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 -- 172.21.15.87:0/2099470648 >> 172.21.15.87:0/2099470648 conn(0x7f2f00066b70 msgr2=0x7f2f00066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 -- 172.21.15.87:0/2099470648 shutdown_connections 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 -- 172.21.15.87:0/2099470648 wait complete. 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 Processor -- start 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.204+0000 7f2f05600640 1 -- start start 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f000cda10 0x7f2f0015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f000cf930 0x7f2f0013ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 0x7f2f0014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000b3850 con 0x7f2f000cf930 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000b36d0 con 0x7f2f0015bc40 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000b39d0 con 0x7f2f000cda10 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f000cf930 0x7f2f0013ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f000cf930 0x7f2f0013ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54820/0 (socket says 172.21.15.87:54820) 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 -- 172.21.15.87:0/368053352 learned_addr learned my addr 172.21.15.87:0/368053352 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efea00640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f000cda10 0x7f2f0015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2eff400640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 0x7f2f0014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 -- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f000cda10 msgr2=0x7f2f0015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f000cda10 0x7f2f0015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 -- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 msgr2=0x7f2f0014ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 0x7f2f0014ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 -- 172.21.15.87:0/368053352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0015d720 con 0x7f2f000cf930 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2efe000640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f000cf930 0x7f2f0013ff80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c370 tx=0x7f2eec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2ef6a00640 1 -- 172.21.15.87:0/368053352 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec017070 con 0x7f2f000cf930 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2ef6a00640 1 -- 172.21.15.87:0/368053352 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec005cf0 con 0x7f2f000cf930 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f001404c0 con 0x7f2f000cf930 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2ef6a00640 1 -- 172.21.15.87:0/368053352 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0072f0 con 0x7f2f000cf930 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0015dc80 con 0x7f2f000cf930 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0004ef30 con 0x7f2f000cf930 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.208+0000 7f2ef6a00640 1 -- 172.21.15.87:0/368053352 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eec01a070 con 0x7f2f000cf930 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.212+0000 7f2ef6a00640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2ee0066f60 0x7f2ee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.212+0000 7f2ef6a00640 1 -- 172.21.15.87:0/368053352 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2eec0842e0 con 0x7f2f000cf930 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.212+0000 7f2efea00640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2ee0066f60 0x7f2ee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.212+0000 7f2efea00640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2ee0066f60 0x7f2ee0069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ad000 tx=0x7f2ef0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.216+0000 7f2ef6a00640 1 -- 172.21.15.87:0/368053352 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f0015d720 con 0x7f2f000cf930 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.376+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f000af1c0 con 0x7f2ee0066f60 2024-04-25T10:15:11.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.384+0000 7f2ef6a00640 1 -- 172.21.15.87:0/368053352 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239860 (secure 0 0 0) 0x7f2f000af1c0 con 0x7f2ee0066f60 2024-04-25T10:15:11.392 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:11.394 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2ee0066f60 msgr2=0x7f2ee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2ee0066f60 0x7f2ee0069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ad000 tx=0x7f2ef0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f000cf930 msgr2=0x7f2f0013ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f000cf930 0x7f2f0013ff80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c370 tx=0x7f2eec00c830 comp rx=0 tx=0).stop 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 shutdown_connections 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2ee0066f60 0x7f2ee0069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f0015bc40 0x7f2f0014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f000cf930 0x7f2f0013ff80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 --2- 172.21.15.87:0/368053352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f000cda10 0x7f2f0015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 >> 172.21.15.87:0/368053352 conn(0x7f2f00066b70 msgr2=0x7f2f000b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 shutdown_connections 2024-04-25T10:15:11.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.388+0000 7f2f05600640 1 -- 172.21.15.87:0/368053352 wait complete. 2024-04-25T10:15:11.413 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-04-25T10:15:11.160158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":296172,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754678548,"statfs":{"total":773094113280,"available":772790833152,"internally_reserved":0,"allocated":5218304,"data_stored":2803256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":297913905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.860823"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306463+0000","last_change":"2024-04-25T10:15:05.306463+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306463+0000","last_undegraded":"2024-04-25T10:15:05.306463+0000","last_fullsized":"2024-04-25T10:15:05.306463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302921+0000","last_change":"2024-04-25T10:15:05.302921+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302921+0000","last_undegraded":"2024-04-25T10:15:05.302921+0000","last_fullsized":"2024-04-25T10:15:05.302921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306106+0000","last_change":"2024-04-25T10:15:05.306106+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306106+0000","last_undegraded":"2024-04-25T10:15:05.306106+0000","last_fullsized":"2024-04-25T10:15:05.306106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305793+0000","last_change":"2024-04-25T10:15:05.305793+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305793+0000","last_undegraded":"2024-04-25T10:15:05.305793+0000","last_fullsized":"2024-04-25T10:15:05.305793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306639+0000","last_change":"2024-04-25T10:15:05.306639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306639+0000","last_undegraded":"2024-04-25T10:15:05.306639+0000","last_fullsized":"2024-04-25T10:15:05.306639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303182+0000","last_change":"2024-04-25T10:15:05.303182+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303182+0000","last_undegraded":"2024-04-25T10:15:05.303182+0000","last_fullsized":"2024-04-25T10:15:05.303182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305140+0000","last_change":"2024-04-25T10:15:05.305140+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305140+0000","last_undegraded":"2024-04-25T10:15:05.305140+0000","last_fullsized":"2024-04-25T10:15:05.305140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302359+0000","last_change":"2024-04-25T10:15:05.302359+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302359+0000","last_undegraded":"2024-04-25T10:15:05.302359+0000","last_fullsized":"2024-04-25T10:15:05.302359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302542+0000","last_change":"2024-04-25T10:15:05.302542+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302542+0000","last_undegraded":"2024-04-25T10:15:05.302542+0000","last_fullsized":"2024-04-25T10:15:05.302542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301350+0000","last_change":"2024-04-25T10:15:05.301350+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301350+0000","last_undegraded":"2024-04-25T10:15:05.301350+0000","last_fullsized":"2024-04-25T10:15:05.301350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306466+0000","last_change":"2024-04-25T10:15:05.306466+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306466+0000","last_undegraded":"2024-04-25T10:15:05.306466+0000","last_fullsized":"2024-04-25T10:15:05.306466+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303789+0000","last_change":"2024-04-25T10:15:05.303789+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303789+0000","last_undegraded":"2024-04-25T10:15:05.303789+0000","last_fullsized":"2024-04-25T10:15:05.303789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305886+0000","last_change":"2024-04-25T10:15:05.305886+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305886+0000","last_undegraded":"2024-04-25T10:15:05.305886+0000","last_fullsized":"2024-04-25T10:15:05.305886+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305735+0000","last_change":"2024-04-25T10:15:05.305735+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305735+0000","last_undegraded":"2024-04-25T10:15:05.305735+0000","last_fullsized":"2024-04-25T10:15:05.305735+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304481+0000","last_change":"2024-04-25T10:15:05.304481+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304481+0000","last_undegraded":"2024-04-25T10:15:05.304481+0000","last_fullsized":"2024-04-25T10:15:05.304481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301889+0000","last_change":"2024-04-25T10:15:05.301889+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301889+0000","last_undegraded":"2024-04-25T10:15:05.301889+0000","last_fullsized":"2024-04-25T10:15:05.301889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303774+0000","last_change":"2024-04-25T10:15:05.303774+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303774+0000","last_undegraded":"2024-04-25T10:15:05.303774+0000","last_fullsized":"2024-04-25T10:15:05.303774+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302351+0000","last_change":"2024-04-25T10:15:05.302351+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302351+0000","last_undegraded":"2024-04-25T10:15:05.302351+0000","last_fullsized":"2024-04-25T10:15:05.302351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305363+0000","last_change":"2024-04-25T10:15:05.305363+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305363+0000","last_undegraded":"2024-04-25T10:15:05.305363+0000","last_fullsized":"2024-04-25T10:15:05.305363+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304418+0000","last_change":"2024-04-25T10:15:05.304418+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304418+0000","last_undegraded":"2024-04-25T10:15:05.304418+0000","last_fullsized":"2024-04-25T10:15:05.304418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304517+0000","last_change":"2024-04-25T10:15:05.304517+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304517+0000","last_undegraded":"2024-04-25T10:15:05.304517+0000","last_fullsized":"2024-04-25T10:15:05.304517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305639+0000","last_change":"2024-04-25T10:15:05.305639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305639+0000","last_undegraded":"2024-04-25T10:15:05.305639+0000","last_fullsized":"2024-04-25T10:15:05.305639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301358+0000","last_change":"2024-04-25T10:15:05.301358+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301358+0000","last_undegraded":"2024-04-25T10:15:05.301358+0000","last_fullsized":"2024-04-25T10:15:05.301358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304717+0000","last_change":"2024-04-25T10:15:05.304717+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304717+0000","last_undegraded":"2024-04-25T10:15:05.304717+0000","last_fullsized":"2024-04-25T10:15:05.304717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304425+0000","last_change":"2024-04-25T10:15:05.304425+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304425+0000","last_undegraded":"2024-04-25T10:15:05.304425+0000","last_fullsized":"2024-04-25T10:15:05.304425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306086+0000","last_change":"2024-04-25T10:15:05.306086+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306086+0000","last_undegraded":"2024-04-25T10:15:05.306086+0000","last_fullsized":"2024-04-25T10:15:05.306086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305180+0000","last_change":"2024-04-25T10:15:05.305180+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305180+0000","last_undegraded":"2024-04-25T10:15:05.305180+0000","last_fullsized":"2024-04-25T10:15:05.305180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304746+0000","last_change":"2024-04-25T10:15:05.304746+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304746+0000","last_undegraded":"2024-04-25T10:15:05.304746+0000","last_fullsized":"2024-04-25T10:15:05.304746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302296+0000","last_change":"2024-04-25T10:15:05.302296+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302296+0000","last_undegraded":"2024-04-25T10:15:05.302296+0000","last_fullsized":"2024-04-25T10:15:05.302296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306536+0000","last_change":"2024-04-25T10:15:05.306536+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306536+0000","last_undegraded":"2024-04-25T10:15:05.306536+0000","last_fullsized":"2024-04-25T10:15:05.306536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301360+0000","last_change":"2024-04-25T10:15:05.301360+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301360+0000","last_undegraded":"2024-04-25T10:15:05.301360+0000","last_fullsized":"2024-04-25T10:15:05.301360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305405+0000","last_change":"2024-04-25T10:15:05.305405+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305405+0000","last_undegraded":"2024-04-25T10:15:05.305405+0000","last_fullsized":"2024-04-25T10:15:05.305405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:15:05.307740+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:15:05.307740+0000","last_peered":"2024-04-25T10:15:05.307740+0000","last_clean":"2024-04-25T10:15:05.307740+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:15:05.307740+0000","last_undegraded":"2024-04-25T10:15:05.307740+0000","last_fullsized":"2024-04-25T10:15:05.307740+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35836,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35832,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336008,"statfs":{"total":96636764160,"available":96600072192,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:11.419 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:11.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.584+0000 7f9185600640 1 Processor -- start 2024-04-25T10:15:11.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.584+0000 7f9185600640 1 -- start start 2024-04-25T10:15:11.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.584+0000 7f9185600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f91800b3190 0x7f918006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.584+0000 7f9185600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9180070ed0 con 0x7f918015a0e0 2024-04-25T10:15:11.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.584+0000 7f9185600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91800711b0 con 0x7f9180071f70 2024-04-25T10:15:11.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.584+0000 7f9185600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9180071490 con 0x7f91800b3190 2024-04-25T10:15:11.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f918015a0e0 0x7f918015b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56198/0 (socket says 172.21.15.87:56198) 2024-04-25T10:15:11.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 -- 172.21.15.87:0/1357893197 learned_addr learned my addr 172.21.15.87:0/1357893197 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:11.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917f400640 1 --2- 172.21.15.87:0/1357893197 >> v2:172.21.15.169:3300/0 conn(0x7f91800b3190 0x7f918006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738599092 0 0) 0x7f9180070ed0 con 0x7f918015a0e0 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9164003530 con 0x7f918015a0e0 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 293705778 0 0) 0x7f9164003530 con 0x7f918015a0e0 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 >> v2:172.21.15.169:3300/0 conn(0x7f91800b3190 msgr2=0x7f918006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 --2- 172.21.15.87:0/1357893197 >> v2:172.21.15.169:3300/0 conn(0x7f91800b3190 0x7f918006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 >> v1:172.21.15.169:6789/0 conn(0x7f9180071f70 legacy=0x7f91800b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180071770 con 0x7f918015a0e0 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f9170001a00 con 0x7f918015a0e0 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 >> v1:172.21.15.87:6789/0 conn(0x7f918015a0e0 legacy=0x7f918015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91640039c0 0x7f9164003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9164004810 0x7f9164004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91640051d0 0x7f916400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9180071490 con 0x7f91640039c0 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916400c0d0 con 0x7f91640051d0 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916400c2d0 con 0x7f9164004810 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91640039c0 0x7f9164003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917e000640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9164004810 0x7f9164004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917f400640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91640051d0 0x7f916400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 -- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9164004810 msgr2=0x7f9164004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9164004810 0x7f9164004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 -- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91640051d0 msgr2=0x7f916400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91640051d0 0x7f916400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917ea00640 1 -- 172.21.15.87:0/1357893197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9164003530 con 0x7f91640039c0 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917f400640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91640051d0 0x7f916400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917e000640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9164004810 0x7f9164004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:11.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.588+0000 7f917f400640 1 -- 172.21.15.87:0/1357893197 reap_dead start 2024-04-25T10:15:11.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f917ea00640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91640039c0 0x7f9164003da0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f917000a790 tx=0x7f917001d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f39a517e62658cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917000a3b0 con 0x7f91640039c0 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91700ad650 con 0x7f91640039c0 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f917d600640 1 -- 172.21.15.87:0/1357893197 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91700ada80 con 0x7f91640039c0 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 -- 172.21.15.87:0/1357893197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91640039c0 msgr2=0x7f9164003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91640039c0 0x7f9164003da0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f917000a790 tx=0x7f917001d250 comp rx=0 tx=0).stop 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 -- 172.21.15.87:0/1357893197 shutdown_connections 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 --2- 172.21.15.87:0/1357893197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91640039c0 0x7f9164003da0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 -- 172.21.15.87:0/1357893197 >> 172.21.15.87:0/1357893197 conn(0x7f9180066b70 msgr2=0x7f9180066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 -- 172.21.15.87:0/1357893197 shutdown_connections 2024-04-25T10:15:11.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 -- 172.21.15.87:0/1357893197 wait complete. 2024-04-25T10:15:11.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 Processor -- start 2024-04-25T10:15:11.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.592+0000 7f9185600640 1 -- start start 2024-04-25T10:15:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91640039c0 0x7f91801502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91801f7a50 0x7f9180158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91801593e0 0x7f91801fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91800b2e90 con 0x7f91801593e0 2024-04-25T10:15:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9180070b50 con 0x7f91801f7a50 2024-04-25T10:15:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91800b3010 con 0x7f91640039c0 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91801593e0 0x7f91801fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91801f7a50 0x7f9180158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91640039c0 0x7f91801502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91801f7a50 0x7f9180158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:35418/0 (socket says 172.21.15.87:35418) 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91801593e0 0x7f91801fa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54828/0 (socket says 172.21.15.87:54828) 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 -- 172.21.15.87:0/245710962 learned_addr learned my addr 172.21.15.87:0/245710962 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 -- 172.21.15.87:0/245710962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91640039c0 msgr2=0x7f91801502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91640039c0 0x7f91801502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 -- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91801593e0 msgr2=0x7f91801fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91801593e0 0x7f91801fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 -- 172.21.15.87:0/245710962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180150f10 con 0x7f91801f7a50 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917f400640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91801593e0 0x7f91801fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:11.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f917e000640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91801f7a50 0x7f9180158ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f916c00e550 tx=0x7f916c00ea10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9176a00640 1 -- 172.21.15.87:0/245710962 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0052e0 con 0x7f91801f7a50 2024-04-25T10:15:11.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9176a00640 1 -- 172.21.15.87:0/245710962 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c018070 con 0x7f91801f7a50 2024-04-25T10:15:11.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9176a00640 1 -- 172.21.15.87:0/245710962 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c005480 con 0x7f91801f7a50 2024-04-25T10:15:11.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91801511f0 con 0x7f91801f7a50 2024-04-25T10:15:11.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91801517b0 con 0x7f91801f7a50 2024-04-25T10:15:11.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918004ef30 con 0x7f91801f7a50 2024-04-25T10:15:11.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.596+0000 7f9176a00640 1 -- 172.21.15.87:0/245710962 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f916c003840 con 0x7f91801f7a50 2024-04-25T10:15:11.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.600+0000 7f9176a00640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9160066f60 0x7f9160069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.600+0000 7f9176a00640 1 -- 172.21.15.87:0/245710962 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f916c084cb0 con 0x7f91801f7a50 2024-04-25T10:15:11.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.600+0000 7f917ea00640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9160066f60 0x7f9160069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.600+0000 7f917ea00640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9160066f60 0x7f9160069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f91700ad000 tx=0x7f91700af040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.604+0000 7f9176a00640 1 -- 172.21.15.87:0/245710962 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f916c0555d0 con 0x7f91801f7a50 2024-04-25T10:15:11.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.760+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91801597c0 con 0x7f9160066f60 2024-04-25T10:15:11.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.764+0000 7f9176a00640 1 -- 172.21.15.87:0/245710962 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239860 (secure 0 0 0) 0x7f91801597c0 con 0x7f9160066f60 2024-04-25T10:15:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:11.774 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:15:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9160066f60 msgr2=0x7f9160069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9160066f60 0x7f9160069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f91700ad000 tx=0x7f91700af040 comp rx=0 tx=0).stop 2024-04-25T10:15:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91801f7a50 msgr2=0x7f9180158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91801f7a50 0x7f9180158ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f916c00e550 tx=0x7f916c00ea10 comp rx=0 tx=0).stop 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 shutdown_connections 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9160066f60 0x7f9160069420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91801593e0 0x7f91801fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91801f7a50 0x7f9180158ea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 --2- 172.21.15.87:0/245710962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91640039c0 0x7f91801502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 >> 172.21.15.87:0/245710962 conn(0x7f9180066b70 msgr2=0x7f918015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 shutdown_connections 2024-04-25T10:15:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.768+0000 7f9185600640 1 -- 172.21.15.87:0/245710962 wait complete. 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-04-25T10:15:11.160158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":296172,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754678548,"statfs":{"total":773094113280,"available":772790833152,"internally_reserved":0,"allocated":5218304,"data_stored":2803256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":297913905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.860823"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306463+0000","last_change":"2024-04-25T10:15:05.306463+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306463+0000","last_undegraded":"2024-04-25T10:15:05.306463+0000","last_fullsized":"2024-04-25T10:15:05.306463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302921+0000","last_change":"2024-04-25T10:15:05.302921+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302921+0000","last_undegraded":"2024-04-25T10:15:05.302921+0000","last_fullsized":"2024-04-25T10:15:05.302921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306106+0000","last_change":"2024-04-25T10:15:05.306106+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306106+0000","last_undegraded":"2024-04-25T10:15:05.306106+0000","last_fullsized":"2024-04-25T10:15:05.306106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305793+0000","last_change":"2024-04-25T10:15:05.305793+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305793+0000","last_undegraded":"2024-04-25T10:15:05.305793+0000","last_fullsized":"2024-04-25T10:15:05.305793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306639+0000","last_change":"2024-04-25T10:15:05.306639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306639+0000","last_undegraded":"2024-04-25T10:15:05.306639+0000","last_fullsized":"2024-04-25T10:15:05.306639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303182+0000","last_change":"2024-04-25T10:15:05.303182+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303182+0000","last_undegraded":"2024-04-25T10:15:05.303182+0000","last_fullsized":"2024-04-25T10:15:05.303182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305140+0000","last_change":"2024-04-25T10:15:05.305140+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305140+0000","last_undegraded":"2024-04-25T10:15:05.305140+0000","last_fullsized":"2024-04-25T10:15:05.305140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302359+0000","last_change":"2024-04-25T10:15:05.302359+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302359+0000","last_undegraded":"2024-04-25T10:15:05.302359+0000","last_fullsized":"2024-04-25T10:15:05.302359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302542+0000","last_change":"2024-04-25T10:15:05.302542+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302542+0000","last_undegraded":"2024-04-25T10:15:05.302542+0000","last_fullsized":"2024-04-25T10:15:05.302542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301350+0000","last_change":"2024-04-25T10:15:05.301350+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301350+0000","last_undegraded":"2024-04-25T10:15:05.301350+0000","last_fullsized":"2024-04-25T10:15:05.301350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306466+0000","last_change":"2024-04-25T10:15:05.306466+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306466+0000","last_undegraded":"2024-04-25T10:15:05.306466+0000","last_fullsized":"2024-04-25T10:15:05.306466+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303789+0000","last_change":"2024-04-25T10:15:05.303789+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303789+0000","last_undegraded":"2024-04-25T10:15:05.303789+0000","last_fullsized":"2024-04-25T10:15:05.303789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305886+0000","last_change":"2024-04-25T10:15:05.305886+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305886+0000","last_undegraded":"2024-04-25T10:15:05.305886+0000","last_fullsized":"2024-04-25T10:15:05.305886+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305735+0000","last_change":"2024-04-25T10:15:05.305735+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305735+0000","last_undegraded":"2024-04-25T10:15:05.305735+0000","last_fullsized":"2024-04-25T10:15:05.305735+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304481+0000","last_change":"2024-04-25T10:15:05.304481+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304481+0000","last_undegraded":"2024-04-25T10:15:05.304481+0000","last_fullsized":"2024-04-25T10:15:05.304481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301889+0000","last_change":"2024-04-25T10:15:05.301889+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301889+0000","last_undegraded":"2024-04-25T10:15:05.301889+0000","last_fullsized":"2024-04-25T10:15:05.301889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303774+0000","last_change":"2024-04-25T10:15:05.303774+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303774+0000","last_undegraded":"2024-04-25T10:15:05.303774+0000","last_fullsized":"2024-04-25T10:15:05.303774+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302351+0000","last_change":"2024-04-25T10:15:05.302351+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302351+0000","last_undegraded":"2024-04-25T10:15:05.302351+0000","last_fullsized":"2024-04-25T10:15:05.302351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305363+0000","last_change":"2024-04-25T10:15:05.305363+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305363+0000","last_undegraded":"2024-04-25T10:15:05.305363+0000","last_fullsized":"2024-04-25T10:15:05.305363+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304418+0000","last_change":"2024-04-25T10:15:05.304418+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304418+0000","last_undegraded":"2024-04-25T10:15:05.304418+0000","last_fullsized":"2024-04-25T10:15:05.304418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304517+0000","last_change":"2024-04-25T10:15:05.304517+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304517+0000","last_undegraded":"2024-04-25T10:15:05.304517+0000","last_fullsized":"2024-04-25T10:15:05.304517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305639+0000","last_change":"2024-04-25T10:15:05.305639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305639+0000","last_undegraded":"2024-04-25T10:15:05.305639+0000","last_fullsized":"2024-04-25T10:15:05.305639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301358+0000","last_change":"2024-04-25T10:15:05.301358+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301358+0000","last_undegraded":"2024-04-25T10:15:05.301358+0000","last_fullsized":"2024-04-25T10:15:05.301358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304717+0000","last_change":"2024-04-25T10:15:05.304717+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304717+0000","last_undegraded":"2024-04-25T10:15:05.304717+0000","last_fullsized":"2024-04-25T10:15:05.304717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304425+0000","last_change":"2024-04-25T10:15:05.304425+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304425+0000","last_undegraded":"2024-04-25T10:15:05.304425+0000","last_fullsized":"2024-04-25T10:15:05.304425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306086+0000","last_change":"2024-04-25T10:15:05.306086+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306086+0000","last_undegraded":"2024-04-25T10:15:05.306086+0000","last_fullsized":"2024-04-25T10:15:05.306086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305180+0000","last_change":"2024-04-25T10:15:05.305180+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305180+0000","last_undegraded":"2024-04-25T10:15:05.305180+0000","last_fullsized":"2024-04-25T10:15:05.305180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304746+0000","last_change":"2024-04-25T10:15:05.304746+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304746+0000","last_undegraded":"2024-04-25T10:15:05.304746+0000","last_fullsized":"2024-04-25T10:15:05.304746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302296+0000","last_change":"2024-04-25T10:15:05.302296+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302296+0000","last_undegraded":"2024-04-25T10:15:05.302296+0000","last_fullsized":"2024-04-25T10:15:05.302296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306536+0000","last_change":"2024-04-25T10:15:05.306536+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306536+0000","last_undegraded":"2024-04-25T10:15:05.306536+0000","last_fullsized":"2024-04-25T10:15:05.306536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301360+0000","last_change":"2024-04-25T10:15:05.301360+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301360+0000","last_undegraded":"2024-04-25T10:15:05.301360+0000","last_fullsized":"2024-04-25T10:15:05.301360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305405+0000","last_change":"2024-04-25T10:15:05.305405+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305405+0000","last_undegraded":"2024-04-25T10:15:05.305405+0000","last_fullsized":"2024-04-25T10:15:05.305405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:15:05.307740+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:15:05.307740+0000","last_peered":"2024-04-25T10:15:05.307740+0000","last_clean":"2024-04-25T10:15:05.307740+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:15:05.307740+0000","last_undegraded":"2024-04-25T10:15:05.307740+0000","last_fullsized":"2024-04-25T10:15:05.307740+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35836,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35832,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336008,"statfs":{"total":96636764160,"available":96600072192,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:11.795 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:11.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.968+0000 7f5a7b800640 1 Processor -- start 2024-04-25T10:15:11.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.968+0000 7f5a7b800640 1 -- start start 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a7b800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5a7415b910 0x7f5a7415c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a7b800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7415bd60 con 0x7f5a740d0040 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a7b800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a74071050 con 0x7f5a741550b0 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a7b800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a740712f0 con 0x7f5a7415b910 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a6be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5a740d0040 0x7f5a7415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56212/0 (socket says 172.21.15.87:56212) 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a6be00640 1 -- 172.21.15.87:0/2928571558 learned_addr learned my addr 172.21.15.87:0/2928571558 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a79800640 1 --2- 172.21.15.87:0/2928571558 >> v2:172.21.15.87:3300/0 conn(0x7f5a7415b910 0x7f5a7415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a73e00640 1 -- 172.21.15.87:0/2928571558 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581839356 0 0) 0x7f5a7415bd60 con 0x7f5a740d0040 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a73e00640 1 -- 172.21.15.87:0/2928571558 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a58003530 con 0x7f5a740d0040 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a73e00640 1 -- 172.21.15.87:0/2928571558 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960608759 0 0) 0x7f5a74071050 con 0x7f5a741550b0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a73e00640 1 -- 172.21.15.87:0/2928571558 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a7415bd60 con 0x7f5a741550b0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a79800640 1 -- 172.21.15.87:0/2928571558 >> v1:172.21.15.169:6789/0 conn(0x7f5a741550b0 legacy=0x7f5a740cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a79800640 1 -- 172.21.15.87:0/2928571558 >> v1:172.21.15.87:6789/0 conn(0x7f5a740d0040 legacy=0x7f5a7415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a79800640 1 -- 172.21.15.87:0/2928571558 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740715d0 con 0x7f5a7415b910 2024-04-25T10:15:11.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.972+0000 7f5a79800640 1 --2- 172.21.15.87:0/2928571558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a7415c030 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009d90 tx=0x7f5a6c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e187454869919848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a73e00640 1 -- 172.21.15.87:0/2928571558 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0b2070 con 0x7f5a7415b910 2024-04-25T10:15:11.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a73e00640 1 -- 172.21.15.87:0/2928571558 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c0a5a00 con 0x7f5a7415b910 2024-04-25T10:15:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a73e00640 1 -- 172.21.15.87:0/2928571558 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0a5db0 con 0x7f5a7415b910 2024-04-25T10:15:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- 172.21.15.87:0/2928571558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 msgr2=0x7f5a7415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 --2- 172.21.15.87:0/2928571558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a7415c030 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009d90 tx=0x7f5a6c01c940 comp rx=0 tx=0).stop 2024-04-25T10:15:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- 172.21.15.87:0/2928571558 shutdown_connections 2024-04-25T10:15:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 --2- 172.21.15.87:0/2928571558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a7415c030 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- 172.21.15.87:0/2928571558 >> 172.21.15.87:0/2928571558 conn(0x7f5a74066b70 msgr2=0x7f5a74066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- 172.21.15.87:0/2928571558 shutdown_connections 2024-04-25T10:15:11.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- 172.21.15.87:0/2928571558 wait complete. 2024-04-25T10:15:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 Processor -- start 2024-04-25T10:15:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- start start 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0040 0x7f5a74157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a741550b0 0x7f5a740b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a74153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7415cf00 con 0x7f5a7415b910 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7415cd80 con 0x7f5a741550b0 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a7b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7415d080 con 0x7f5a740d0040 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a79800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a74153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a79800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a74153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54838/0 (socket says 172.21.15.87:54838) 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a79800640 1 -- 172.21.15.87:0/734744124 learned_addr learned my addr 172.21.15.87:0/734744124 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a6be00640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a741550b0 0x7f5a740b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.976+0000 7f5a78e00640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0040 0x7f5a74157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a79800640 1 -- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0040 msgr2=0x7f5a74157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a79800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0040 0x7f5a74157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a79800640 1 -- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a741550b0 msgr2=0x7f5a740b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a79800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a741550b0 0x7f5a740b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a79800640 1 -- 172.21.15.87:0/734744124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740b0d80 con 0x7f5a7415b910 2024-04-25T10:15:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a79800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a74153a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c01a570 tx=0x7f5a6c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a71600640 1 -- 172.21.15.87:0/734744124 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0b2050 con 0x7f5a7415b910 2024-04-25T10:15:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a71600640 1 -- 172.21.15.87:0/734744124 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c0be070 con 0x7f5a7415b910 2024-04-25T10:15:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a71600640 1 -- 172.21.15.87:0/734744124 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0adb50 con 0x7f5a7415b910 2024-04-25T10:15:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a74155530 con 0x7f5a7415b910 2024-04-25T10:15:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a74155af0 con 0x7f5a7415b910 2024-04-25T10:15:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7404ef30 con 0x7f5a7415b910 2024-04-25T10:15:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a71600640 1 -- 172.21.15.87:0/734744124 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a6c0adcf0 con 0x7f5a7415b910 2024-04-25T10:15:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a71600640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a50066f60 0x7f5a50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.980+0000 7f5a71600640 1 -- 172.21.15.87:0/734744124 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a6c120200 con 0x7f5a7415b910 2024-04-25T10:15:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.984+0000 7f5a78e00640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a50066f60 0x7f5a50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.984+0000 7f5a78e00640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a50066f60 0x7f5a50069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a640017c0 tx=0x7f5a64009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:11.984+0000 7f5a71600640 1 -- 172.21.15.87:0/734744124 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a6c016560 con 0x7f5a7415b910 2024-04-25T10:15:12.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.136+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a740af630 con 0x7f5a50066f60 2024-04-25T10:15:12.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.144+0000 7f5a71600640 1 -- 172.21.15.87:0/734744124 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239860 (secure 0 0 0) 0x7f5a740af630 con 0x7f5a50066f60 2024-04-25T10:15:12.151 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:12.152 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a50066f60 msgr2=0x7f5a50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a50066f60 0x7f5a50069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a640017c0 tx=0x7f5a64009040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 msgr2=0x7f5a74153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a74153a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c01a570 tx=0x7f5a6c0196e0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 shutdown_connections 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a50066f60 0x7f5a50069420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a7415b910 0x7f5a74153a60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a741550b0 0x7f5a740b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 --2- 172.21.15.87:0/734744124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0040 0x7f5a74157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 >> 172.21.15.87:0/734744124 conn(0x7f5a74066b70 msgr2=0x7f5a740ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 shutdown_connections 2024-04-25T10:15:12.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.148+0000 7f5a7b800640 1 -- 172.21.15.87:0/734744124 wait complete. 2024-04-25T10:15:12.171 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-04-25T10:15:11.160158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":296172,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754678548,"statfs":{"total":773094113280,"available":772790833152,"internally_reserved":0,"allocated":5218304,"data_stored":2803256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":297913905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.860823"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306463+0000","last_change":"2024-04-25T10:15:05.306463+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306463+0000","last_undegraded":"2024-04-25T10:15:05.306463+0000","last_fullsized":"2024-04-25T10:15:05.306463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302921+0000","last_change":"2024-04-25T10:15:05.302921+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302921+0000","last_undegraded":"2024-04-25T10:15:05.302921+0000","last_fullsized":"2024-04-25T10:15:05.302921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306106+0000","last_change":"2024-04-25T10:15:05.306106+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306106+0000","last_undegraded":"2024-04-25T10:15:05.306106+0000","last_fullsized":"2024-04-25T10:15:05.306106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305793+0000","last_change":"2024-04-25T10:15:05.305793+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305793+0000","last_undegraded":"2024-04-25T10:15:05.305793+0000","last_fullsized":"2024-04-25T10:15:05.305793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306639+0000","last_change":"2024-04-25T10:15:05.306639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306639+0000","last_undegraded":"2024-04-25T10:15:05.306639+0000","last_fullsized":"2024-04-25T10:15:05.306639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303182+0000","last_change":"2024-04-25T10:15:05.303182+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303182+0000","last_undegraded":"2024-04-25T10:15:05.303182+0000","last_fullsized":"2024-04-25T10:15:05.303182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305140+0000","last_change":"2024-04-25T10:15:05.305140+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305140+0000","last_undegraded":"2024-04-25T10:15:05.305140+0000","last_fullsized":"2024-04-25T10:15:05.305140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302359+0000","last_change":"2024-04-25T10:15:05.302359+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302359+0000","last_undegraded":"2024-04-25T10:15:05.302359+0000","last_fullsized":"2024-04-25T10:15:05.302359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302542+0000","last_change":"2024-04-25T10:15:05.302542+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302542+0000","last_undegraded":"2024-04-25T10:15:05.302542+0000","last_fullsized":"2024-04-25T10:15:05.302542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301350+0000","last_change":"2024-04-25T10:15:05.301350+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301350+0000","last_undegraded":"2024-04-25T10:15:05.301350+0000","last_fullsized":"2024-04-25T10:15:05.301350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306466+0000","last_change":"2024-04-25T10:15:05.306466+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306466+0000","last_undegraded":"2024-04-25T10:15:05.306466+0000","last_fullsized":"2024-04-25T10:15:05.306466+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303789+0000","last_change":"2024-04-25T10:15:05.303789+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303789+0000","last_undegraded":"2024-04-25T10:15:05.303789+0000","last_fullsized":"2024-04-25T10:15:05.303789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305886+0000","last_change":"2024-04-25T10:15:05.305886+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305886+0000","last_undegraded":"2024-04-25T10:15:05.305886+0000","last_fullsized":"2024-04-25T10:15:05.305886+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305735+0000","last_change":"2024-04-25T10:15:05.305735+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305735+0000","last_undegraded":"2024-04-25T10:15:05.305735+0000","last_fullsized":"2024-04-25T10:15:05.305735+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304481+0000","last_change":"2024-04-25T10:15:05.304481+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304481+0000","last_undegraded":"2024-04-25T10:15:05.304481+0000","last_fullsized":"2024-04-25T10:15:05.304481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301889+0000","last_change":"2024-04-25T10:15:05.301889+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301889+0000","last_undegraded":"2024-04-25T10:15:05.301889+0000","last_fullsized":"2024-04-25T10:15:05.301889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303774+0000","last_change":"2024-04-25T10:15:05.303774+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303774+0000","last_undegraded":"2024-04-25T10:15:05.303774+0000","last_fullsized":"2024-04-25T10:15:05.303774+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302351+0000","last_change":"2024-04-25T10:15:05.302351+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302351+0000","last_undegraded":"2024-04-25T10:15:05.302351+0000","last_fullsized":"2024-04-25T10:15:05.302351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305363+0000","last_change":"2024-04-25T10:15:05.305363+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305363+0000","last_undegraded":"2024-04-25T10:15:05.305363+0000","last_fullsized":"2024-04-25T10:15:05.305363+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304418+0000","last_change":"2024-04-25T10:15:05.304418+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304418+0000","last_undegraded":"2024-04-25T10:15:05.304418+0000","last_fullsized":"2024-04-25T10:15:05.304418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304517+0000","last_change":"2024-04-25T10:15:05.304517+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304517+0000","last_undegraded":"2024-04-25T10:15:05.304517+0000","last_fullsized":"2024-04-25T10:15:05.304517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305639+0000","last_change":"2024-04-25T10:15:05.305639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305639+0000","last_undegraded":"2024-04-25T10:15:05.305639+0000","last_fullsized":"2024-04-25T10:15:05.305639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301358+0000","last_change":"2024-04-25T10:15:05.301358+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301358+0000","last_undegraded":"2024-04-25T10:15:05.301358+0000","last_fullsized":"2024-04-25T10:15:05.301358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304717+0000","last_change":"2024-04-25T10:15:05.304717+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304717+0000","last_undegraded":"2024-04-25T10:15:05.304717+0000","last_fullsized":"2024-04-25T10:15:05.304717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304425+0000","last_change":"2024-04-25T10:15:05.304425+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304425+0000","last_undegraded":"2024-04-25T10:15:05.304425+0000","last_fullsized":"2024-04-25T10:15:05.304425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306086+0000","last_change":"2024-04-25T10:15:05.306086+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306086+0000","last_undegraded":"2024-04-25T10:15:05.306086+0000","last_fullsized":"2024-04-25T10:15:05.306086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305180+0000","last_change":"2024-04-25T10:15:05.305180+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305180+0000","last_undegraded":"2024-04-25T10:15:05.305180+0000","last_fullsized":"2024-04-25T10:15:05.305180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304746+0000","last_change":"2024-04-25T10:15:05.304746+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304746+0000","last_undegraded":"2024-04-25T10:15:05.304746+0000","last_fullsized":"2024-04-25T10:15:05.304746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302296+0000","last_change":"2024-04-25T10:15:05.302296+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302296+0000","last_undegraded":"2024-04-25T10:15:05.302296+0000","last_fullsized":"2024-04-25T10:15:05.302296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306536+0000","last_change":"2024-04-25T10:15:05.306536+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306536+0000","last_undegraded":"2024-04-25T10:15:05.306536+0000","last_fullsized":"2024-04-25T10:15:05.306536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301360+0000","last_change":"2024-04-25T10:15:05.301360+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301360+0000","last_undegraded":"2024-04-25T10:15:05.301360+0000","last_fullsized":"2024-04-25T10:15:05.301360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305405+0000","last_change":"2024-04-25T10:15:05.305405+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305405+0000","last_undegraded":"2024-04-25T10:15:05.305405+0000","last_fullsized":"2024-04-25T10:15:05.305405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:15:05.307740+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:15:05.307740+0000","last_peered":"2024-04-25T10:15:05.307740+0000","last_clean":"2024-04-25T10:15:05.307740+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:15:05.307740+0000","last_undegraded":"2024-04-25T10:15:05.307740+0000","last_fullsized":"2024-04-25T10:15:05.307740+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35836,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35832,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336008,"statfs":{"total":96636764160,"available":96600072192,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-25T10:15:12.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-25T10:15:12.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-25T10:15:12.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.306463+0000', 'last_change': '2024-04-25T10:15:05.306463+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.306463+0000', 'last_undegraded': '2024-04-25T10:15:05.306463+0000', 'last_fullsized': '2024-04-25T10:15:05.306463+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.302921+0000', 'last_change': '2024-04-25T10:15:05.302921+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.302921+0000', 'last_undegraded': '2024-04-25T10:15:05.302921+0000', 'last_fullsized': '2024-04-25T10:15:05.302921+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.306106+0000', 'last_change': '2024-04-25T10:15:05.306106+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.306106+0000', 'last_undegraded': '2024-04-25T10:15:05.306106+0000', 'last_fullsized': '2024-04-25T10:15:05.306106+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T10:15:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305793+0000', 'last_change': '2024-04-25T10:15:05.305793+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305793+0000', 'last_undegraded': '2024-04-25T10:15:05.305793+0000', 'last_fullsized': '2024-04-25T10:15:05.305793+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.306639+0000', 'last_change': '2024-04-25T10:15:05.306639+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.306639+0000', 'last_undegraded': '2024-04-25T10:15:05.306639+0000', 'last_fullsized': '2024-04-25T10:15:05.306639+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.303182+0000', 'last_change': '2024-04-25T10:15:05.303182+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.303182+0000', 'last_undegraded': '2024-04-25T10:15:05.303182+0000', 'last_fullsized': '2024-04-25T10:15:05.303182+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:15:12.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305140+0000', 'last_change': '2024-04-25T10:15:05.305140+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305140+0000', 'last_undegraded': '2024-04-25T10:15:05.305140+0000', 'last_fullsized': '2024-04-25T10:15:05.305140+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.302359+0000', 'last_change': '2024-04-25T10:15:05.302359+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.302359+0000', 'last_undegraded': '2024-04-25T10:15:05.302359+0000', 'last_fullsized': '2024-04-25T10:15:05.302359+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-25T10:15:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.302542+0000', 'last_change': '2024-04-25T10:15:05.302542+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.302542+0000', 'last_undegraded': '2024-04-25T10:15:05.302542+0000', 'last_fullsized': '2024-04-25T10:15:05.302542+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.301350+0000', 'last_change': '2024-04-25T10:15:05.301350+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.301350+0000', 'last_undegraded': '2024-04-25T10:15:05.301350+0000', 'last_fullsized': '2024-04-25T10:15:05.301350+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.306466+0000', 'last_change': '2024-04-25T10:15:05.306466+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.306466+0000', 'last_undegraded': '2024-04-25T10:15:05.306466+0000', 'last_fullsized': '2024-04-25T10:15:05.306466+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.303789+0000', 'last_change': '2024-04-25T10:15:05.303789+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.303789+0000', 'last_undegraded': '2024-04-25T10:15:05.303789+0000', 'last_fullsized': '2024-04-25T10:15:05.303789+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305886+0000', 'last_change': '2024-04-25T10:15:05.305886+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305886+0000', 'last_undegraded': '2024-04-25T10:15:05.305886+0000', 'last_fullsized': '2024-04-25T10:15:05.305886+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305735+0000', 'last_change': '2024-04-25T10:15:05.305735+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305735+0000', 'last_undegraded': '2024-04-25T10:15:05.305735+0000', 'last_fullsized': '2024-04-25T10:15:05.305735+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-25T10:15:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.304481+0000', 'last_change': '2024-04-25T10:15:05.304481+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.304481+0000', 'last_undegraded': '2024-04-25T10:15:05.304481+0000', 'last_fullsized': '2024-04-25T10:15:05.304481+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.301889+0000', 'last_change': '2024-04-25T10:15:05.301889+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.301889+0000', 'last_undegraded': '2024-04-25T10:15:05.301889+0000', 'last_fullsized': '2024-04-25T10:15:05.301889+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-25T10:15:12.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.303774+0000', 'last_change': '2024-04-25T10:15:05.303774+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.303774+0000', 'last_undegraded': '2024-04-25T10:15:05.303774+0000', 'last_fullsized': '2024-04-25T10:15:05.303774+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.302351+0000', 'last_change': '2024-04-25T10:15:05.302351+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.302351+0000', 'last_undegraded': '2024-04-25T10:15:05.302351+0000', 'last_fullsized': '2024-04-25T10:15:05.302351+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305363+0000', 'last_change': '2024-04-25T10:15:05.305363+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305363+0000', 'last_undegraded': '2024-04-25T10:15:05.305363+0000', 'last_fullsized': '2024-04-25T10:15:05.305363+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.304418+0000', 'last_change': '2024-04-25T10:15:05.304418+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.304418+0000', 'last_undegraded': '2024-04-25T10:15:05.304418+0000', 'last_fullsized': '2024-04-25T10:15:05.304418+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.304517+0000', 'last_change': '2024-04-25T10:15:05.304517+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.304517+0000', 'last_undegraded': '2024-04-25T10:15:05.304517+0000', 'last_fullsized': '2024-04-25T10:15:05.304517+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305639+0000', 'last_change': '2024-04-25T10:15:05.305639+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305639+0000', 'last_undegraded': '2024-04-25T10:15:05.305639+0000', 'last_fullsized': '2024-04-25T10:15:05.305639+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.301358+0000', 'last_change': '2024-04-25T10:15:05.301358+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.301358+0000', 'last_undegraded': '2024-04-25T10:15:05.301358+0000', 'last_fullsized': '2024-04-25T10:15:05.301358+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.304717+0000', 'last_change': '2024-04-25T10:15:05.304717+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.304717+0000', 'last_undegraded': '2024-04-25T10:15:05.304717+0000', 'last_fullsized': '2024-04-25T10:15:05.304717+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.304425+0000', 'last_change': '2024-04-25T10:15:05.304425+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.304425+0000', 'last_undegraded': '2024-04-25T10:15:05.304425+0000', 'last_fullsized': '2024-04-25T10:15:05.304425+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.306086+0000', 'last_change': '2024-04-25T10:15:05.306086+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.306086+0000', 'last_undegraded': '2024-04-25T10:15:05.306086+0000', 'last_fullsized': '2024-04-25T10:15:05.306086+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305180+0000', 'last_change': '2024-04-25T10:15:05.305180+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305180+0000', 'last_undegraded': '2024-04-25T10:15:05.305180+0000', 'last_fullsized': '2024-04-25T10:15:05.305180+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.304746+0000', 'last_change': '2024-04-25T10:15:05.304746+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.304746+0000', 'last_undegraded': '2024-04-25T10:15:05.304746+0000', 'last_fullsized': '2024-04-25T10:15:05.304746+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T10:15:12.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.302296+0000', 'last_change': '2024-04-25T10:15:05.302296+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.302296+0000', 'last_undegraded': '2024-04-25T10:15:05.302296+0000', 'last_fullsized': '2024-04-25T10:15:05.302296+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.306536+0000', 'last_change': '2024-04-25T10:15:05.306536+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.306536+0000', 'last_undegraded': '2024-04-25T10:15:05.306536+0000', 'last_fullsized': '2024-04-25T10:15:05.306536+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.301360+0000', 'last_change': '2024-04-25T10:15:05.301360+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.301360+0000', 'last_undegraded': '2024-04-25T10:15:05.301360+0000', 'last_fullsized': '2024-04-25T10:15:05.301360+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T10:15:12.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:15:06.298784+0000', 'last_change': '2024-04-25T10:15:06.298784+0000', 'last_active': '2024-04-25T10:15:06.298784+0000', 'last_peered': '2024-04-25T10:15:06.298784+0000', 'last_clean': '2024-04-25T10:15:06.298784+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:06.298784+0000', 'last_undegraded': '2024-04-25T10:15:06.298784+0000', 'last_fullsized': '2024-04-25T10:15:06.298784+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:06.298784+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:15:12.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-25T10:15:12.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:15:05.305405+0000', 'last_change': '2024-04-25T10:15:05.305405+0000', 'last_active': '2024-04-25T10:15:05.293212+0000', 'last_peered': '2024-04-25T10:15:05.293212+0000', 'last_clean': '2024-04-25T10:15:05.293212+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:15:05.305405+0000', 'last_undegraded': '2024-04-25T10:15:05.305405+0000', 'last_fullsized': '2024-04-25T10:15:05.305405+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'last_clean_scrub_stamp': '2024-04-25T10:15:05.293212+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:15:12.192 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:15:12.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbec00640 1 Processor -- start 2024-04-25T10:15:12.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbec00640 1 -- start start 2024-04-25T10:15:12.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdb815bc00 0x7efdb80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb80cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb80d0710 con 0x7efdb80d0260 2024-04-25T10:15:12.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb815bfe0 con 0x7efdb815bc00 2024-04-25T10:15:12.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80cd100 con 0x7efdb80cf8c0 2024-04-25T10:15:12.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbd800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdb815bc00 0x7efdb80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb80cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbd800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdb815bc00 0x7efdb80cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:35444/0 (socket says 172.21.15.87:35444) 2024-04-25T10:15:12.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbd800640 1 -- 172.21.15.87:0/3123259937 learned_addr learned my addr 172.21.15.87:0/3123259937 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:12.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbd800640 1 -- 172.21.15.87:0/3123259937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 msgr2=0x7efdb80cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbd800640 1 --2- 172.21.15.87:0/3123259937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb80cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbd800640 1 -- 172.21.15.87:0/3123259937 >> v1:172.21.15.169:6789/0 conn(0x7efdb80d0260 legacy=0x7efdb80ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.348+0000 7efdbd800640 1 -- 172.21.15.87:0/3123259937 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80cd3e0 con 0x7efdb815bc00 2024-04-25T10:15:12.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.352+0000 7efdbd800640 1 --2- 172.21.15.87:0/3123259937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 0x7efdb80cf380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efdac009870 tx=0x7efdac01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510826db183ea605 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdb7e00640 1 -- 172.21.15.87:0/3123259937 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac0a95f0 con 0x7efdb815bc00 2024-04-25T10:15:12.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdb7e00640 1 -- 172.21.15.87:0/3123259937 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac0a9790 con 0x7efdb815bc00 2024-04-25T10:15:12.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdb7e00640 1 -- 172.21.15.87:0/3123259937 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac0a9b60 con 0x7efdb815bc00 2024-04-25T10:15:12.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 -- 172.21.15.87:0/3123259937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 msgr2=0x7efdb80cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 --2- 172.21.15.87:0/3123259937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 0x7efdb80cf380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efdac009870 tx=0x7efdac01c530 comp rx=0 tx=0).stop 2024-04-25T10:15:12.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 -- 172.21.15.87:0/3123259937 shutdown_connections 2024-04-25T10:15:12.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 --2- 172.21.15.87:0/3123259937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb80cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 --2- 172.21.15.87:0/3123259937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 0x7efdb80cf380 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 -- 172.21.15.87:0/3123259937 >> 172.21.15.87:0/3123259937 conn(0x7efdb80d1740 msgr2=0x7efdb80d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 -- 172.21.15.87:0/3123259937 shutdown_connections 2024-04-25T10:15:12.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.356+0000 7efdbec00640 1 -- 172.21.15.87:0/3123259937 wait complete. 2024-04-25T10:15:12.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 Processor -- start 2024-04-25T10:15:12.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 -- start start 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb8151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdb80d0260 0x7efdb81f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 0x7efdb80cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80cce70 con 0x7efdb80d0260 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80cccf0 con 0x7efdb815bc00 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb81686e0 con 0x7efdb80cf8c0 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdb80d0260 0x7efdb81f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdb80d0260 0x7efdb81f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54850/0 (socket says 172.21.15.87:54850) 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 -- 172.21.15.87:0/1469019769 learned_addr learned my addr 172.21.15.87:0/1469019769 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbd800640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb8151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbe200640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 0x7efdb80cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 -- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 msgr2=0x7efdb8151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb8151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 -- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 msgr2=0x7efdb80cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 0x7efdb80cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 -- 172.21.15.87:0/1469019769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80c8920 con 0x7efdb80d0260 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbd800640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbce00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdb80d0260 0x7efdb81f45e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efda400e840 tx=0x7efda400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdb5600640 1 -- 172.21.15.87:0/1469019769 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4017070 con 0x7efdb80d0260 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdb5600640 1 -- 172.21.15.87:0/1469019769 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda400ee90 con 0x7efdb80d0260 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdb5600640 1 -- 172.21.15.87:0/1469019769 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda400cb80 con 0x7efdb80d0260 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb80c8c00 con 0x7efdb80d0260 2024-04-25T10:15:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb80c91c0 con 0x7efdb80d0260 2024-04-25T10:15:12.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.360+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb80b6cf0 con 0x7efdb80d0260 2024-04-25T10:15:12.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.364+0000 7efdb5600640 1 -- 172.21.15.87:0/1469019769 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efda401a070 con 0x7efdb80d0260 2024-04-25T10:15:12.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.364+0000 7efdb5600640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd98066f60 0x7efd98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.364+0000 7efdb5600640 1 -- 172.21.15.87:0/1469019769 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efda40851d0 con 0x7efdb80d0260 2024-04-25T10:15:12.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.364+0000 7efdbd800640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd98066f60 0x7efd98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.364+0000 7efdbd800640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd98066f60 0x7efd98069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efdac0ae000 tx=0x7efdac0b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.368+0000 7efdb5600640 1 -- 172.21.15.87:0/1469019769 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efda4055b80 con 0x7efdb80d0260 2024-04-25T10:15:12.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.572+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efdb8075120 con 0x7efdb80d0260 2024-04-25T10:15:12.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdb5600640 1 -- 172.21.15.87:0/1469019769 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1109 (secure 0 0 0) 0x7efda4059c20 con 0x7efdb80d0260 2024-04-25T10:15:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:12.580 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":194,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":303280128,"bytes_avail":772790833152,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5631}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T10:14:55.156926+0000","services":{}},"progress_events":{}} 2024-04-25T10:15:12.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd98066f60 msgr2=0x7efd98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd98066f60 0x7efd98069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efdac0ae000 tx=0x7efdac0b3040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdb80d0260 msgr2=0x7efdb81f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdb80d0260 0x7efdb81f45e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efda400e840 tx=0x7efda400ec40 comp rx=0 tx=0).stop 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 shutdown_connections 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd98066f60 0x7efd98069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb815bc00 0x7efdb80cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdb80d0260 0x7efdb81f45e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 --2- 172.21.15.87:0/1469019769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdb80cf8c0 0x7efdb8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 >> 172.21.15.87:0/1469019769 conn(0x7efdb80d1740 msgr2=0x7efdb8152370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 shutdown_connections 2024-04-25T10:15:12.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.576+0000 7efdbec00640 1 -- 172.21.15.87:0/1469019769 wait complete. 2024-04-25T10:15:12.600 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:15:12.600 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:12.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.728+0000 7fd56a400640 1 Processor -- start 2024-04-25T10:15:12.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.728+0000 7fd56a400640 1 -- start start 2024-04-25T10:15:12.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd56a400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd564146ed0 0x7fd5641472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd56a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5640726b0 0x7fd564071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd56a400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56406c600 con 0x7fd564071c80 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd56a400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd564161980 con 0x7fd564146ed0 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd56a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd564072080 con 0x7fd5640726b0 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd563e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd564071c80 0x7fd56406c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56220/0 (socket says 172.21.15.87:56220) 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd563e00640 1 -- 172.21.15.87:0/215191700 learned_addr learned my addr 172.21.15.87:0/215191700 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd563400640 1 --2- 172.21.15.87:0/215191700 >> v2:172.21.15.87:3300/0 conn(0x7fd564146ed0 0x7fd5641472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd562a00640 1 --2- 172.21.15.87:0/215191700 >> v2:172.21.15.169:3300/0 conn(0x7fd5640726b0 0x7fd564071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd562000640 1 -- 172.21.15.87:0/215191700 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607077095 0 0) 0x7fd56406c600 con 0x7fd564071c80 2024-04-25T10:15:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd562000640 1 -- 172.21.15.87:0/215191700 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd548003530 con 0x7fd564071c80 2024-04-25T10:15:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd563400640 1 -- 172.21.15.87:0/215191700 >> v2:172.21.15.169:3300/0 conn(0x7fd5640726b0 msgr2=0x7fd564071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd563400640 1 --2- 172.21.15.87:0/215191700 >> v2:172.21.15.169:3300/0 conn(0x7fd5640726b0 0x7fd564071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd563400640 1 -- 172.21.15.87:0/215191700 >> v1:172.21.15.87:6789/0 conn(0x7fd564071c80 legacy=0x7fd56406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.732+0000 7fd563400640 1 -- 172.21.15.87:0/215191700 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564161bd0 con 0x7fd564146ed0 2024-04-25T10:15:12.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd563400640 1 --2- 172.21.15.87:0/215191700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564146ed0 0x7fd5641472b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd56406da70 tx=0x7fd55401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293c2e74b72bb8c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd562000640 1 -- 172.21.15.87:0/215191700 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540b2070 con 0x7fd564146ed0 2024-04-25T10:15:12.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd562000640 1 -- 172.21.15.87:0/215191700 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540a59c0 con 0x7fd564146ed0 2024-04-25T10:15:12.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd562000640 1 -- 172.21.15.87:0/215191700 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540a5d90 con 0x7fd564146ed0 2024-04-25T10:15:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 -- 172.21.15.87:0/215191700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564146ed0 msgr2=0x7fd5641472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 --2- 172.21.15.87:0/215191700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564146ed0 0x7fd5641472b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd56406da70 tx=0x7fd55401c900 comp rx=0 tx=0).stop 2024-04-25T10:15:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 -- 172.21.15.87:0/215191700 shutdown_connections 2024-04-25T10:15:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 --2- 172.21.15.87:0/215191700 >> v2:172.21.15.169:3300/0 conn(0x7fd5640726b0 0x7fd564071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 --2- 172.21.15.87:0/215191700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564146ed0 0x7fd5641472b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 -- 172.21.15.87:0/215191700 >> 172.21.15.87:0/215191700 conn(0x7fd5640669a0 msgr2=0x7fd564066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 -- 172.21.15.87:0/215191700 shutdown_connections 2024-04-25T10:15:12.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 -- 172.21.15.87:0/215191700 wait complete. 2024-04-25T10:15:12.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 Processor -- start 2024-04-25T10:15:12.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.736+0000 7fd56a400640 1 -- start start 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564071c80 0x7fd564160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640726b0 0x7fd56415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd564146ed0 0x7fd564142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564147bf0 con 0x7fd564071c80 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564147a70 con 0x7fd564146ed0 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640ae850 con 0x7fd5640726b0 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564071c80 0x7fd564160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564071c80 0x7fd564160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54870/0 (socket says 172.21.15.87:54870) 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 -- 172.21.15.87:0/501915079 learned_addr learned my addr 172.21.15.87:0/501915079 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd562a00640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640726b0 0x7fd56415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563e00640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd564146ed0 0x7fd564142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 -- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640726b0 msgr2=0x7fd56415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640726b0 0x7fd56415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 -- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd564146ed0 msgr2=0x7fd564142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd564146ed0 0x7fd564142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 -- 172.21.15.87:0/501915079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564160270 con 0x7fd564071c80 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd562a00640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640726b0 0x7fd56415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563e00640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd564146ed0 0x7fd564142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd563400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564071c80 0x7fd564160e30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd5540b0040 tx=0x7fd5540196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd55b400640 1 -- 172.21.15.87:0/501915079 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540bd020 con 0x7fd564071c80 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd55b400640 1 -- 172.21.15.87:0/501915079 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540b8670 con 0x7fd564071c80 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd55b400640 1 -- 172.21.15.87:0/501915079 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540b2040 con 0x7fd564071c80 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd564141d90 con 0x7fd564071c80 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56415d830 con 0x7fd564071c80 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56404ef30 con 0x7fd564071c80 2024-04-25T10:15:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.740+0000 7fd55b400640 1 -- 172.21.15.87:0/501915079 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5540c0020 con 0x7fd564071c80 2024-04-25T10:15:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.744+0000 7fd55b400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd544066f90 0x7fd544069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.744+0000 7fd55b400640 1 -- 172.21.15.87:0/501915079 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd55411ffe0 con 0x7fd564071c80 2024-04-25T10:15:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.744+0000 7fd562a00640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd544066f90 0x7fd544069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.744+0000 7fd562a00640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd544066f90 0x7fd544069450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5641f9d00 tx=0x7fd550008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.748+0000 7fd55b400640 1 -- 172.21.15.87:0/501915079 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5540c7030 con 0x7fd564071c80 2024-04-25T10:15:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.896+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd564160c10 con 0x7fd544066f90 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.904+0000 7fd55b400640 1 -- 172.21.15.87:0/501915079 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239860 (secure 0 0 0) 0x7fd564160c10 con 0x7fd544066f90 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:12.912 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:15:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd544066f90 msgr2=0x7fd544069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd544066f90 0x7fd544069450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5641f9d00 tx=0x7fd550008040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564071c80 msgr2=0x7fd564160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564071c80 0x7fd564160e30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd5540b0040 tx=0x7fd5540196a0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 shutdown_connections 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd544066f90 0x7fd544069450 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd564146ed0 0x7fd564142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640726b0 0x7fd56415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 --2- 172.21.15.87:0/501915079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564071c80 0x7fd564160e30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 >> 172.21.15.87:0/501915079 conn(0x7fd5640669a0 msgr2=0x7fd564145fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 shutdown_connections 2024-04-25T10:15:12.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:12.908+0000 7fd56a400640 1 -- 172.21.15.87:0/501915079 wait complete. 2024-04-25T10:15:12.932 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-04-25T10:15:11.160158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":296172,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754678548,"statfs":{"total":773094113280,"available":772790833152,"internally_reserved":0,"allocated":5218304,"data_stored":2803256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":297913905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.860823"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306463+0000","last_change":"2024-04-25T10:15:05.306463+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306463+0000","last_undegraded":"2024-04-25T10:15:05.306463+0000","last_fullsized":"2024-04-25T10:15:05.306463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302921+0000","last_change":"2024-04-25T10:15:05.302921+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302921+0000","last_undegraded":"2024-04-25T10:15:05.302921+0000","last_fullsized":"2024-04-25T10:15:05.302921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306106+0000","last_change":"2024-04-25T10:15:05.306106+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306106+0000","last_undegraded":"2024-04-25T10:15:05.306106+0000","last_fullsized":"2024-04-25T10:15:05.306106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305793+0000","last_change":"2024-04-25T10:15:05.305793+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305793+0000","last_undegraded":"2024-04-25T10:15:05.305793+0000","last_fullsized":"2024-04-25T10:15:05.305793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306639+0000","last_change":"2024-04-25T10:15:05.306639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306639+0000","last_undegraded":"2024-04-25T10:15:05.306639+0000","last_fullsized":"2024-04-25T10:15:05.306639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303182+0000","last_change":"2024-04-25T10:15:05.303182+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303182+0000","last_undegraded":"2024-04-25T10:15:05.303182+0000","last_fullsized":"2024-04-25T10:15:05.303182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305140+0000","last_change":"2024-04-25T10:15:05.305140+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305140+0000","last_undegraded":"2024-04-25T10:15:05.305140+0000","last_fullsized":"2024-04-25T10:15:05.305140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302359+0000","last_change":"2024-04-25T10:15:05.302359+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302359+0000","last_undegraded":"2024-04-25T10:15:05.302359+0000","last_fullsized":"2024-04-25T10:15:05.302359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302542+0000","last_change":"2024-04-25T10:15:05.302542+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302542+0000","last_undegraded":"2024-04-25T10:15:05.302542+0000","last_fullsized":"2024-04-25T10:15:05.302542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301350+0000","last_change":"2024-04-25T10:15:05.301350+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301350+0000","last_undegraded":"2024-04-25T10:15:05.301350+0000","last_fullsized":"2024-04-25T10:15:05.301350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306466+0000","last_change":"2024-04-25T10:15:05.306466+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306466+0000","last_undegraded":"2024-04-25T10:15:05.306466+0000","last_fullsized":"2024-04-25T10:15:05.306466+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303789+0000","last_change":"2024-04-25T10:15:05.303789+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303789+0000","last_undegraded":"2024-04-25T10:15:05.303789+0000","last_fullsized":"2024-04-25T10:15:05.303789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305886+0000","last_change":"2024-04-25T10:15:05.305886+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305886+0000","last_undegraded":"2024-04-25T10:15:05.305886+0000","last_fullsized":"2024-04-25T10:15:05.305886+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305735+0000","last_change":"2024-04-25T10:15:05.305735+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305735+0000","last_undegraded":"2024-04-25T10:15:05.305735+0000","last_fullsized":"2024-04-25T10:15:05.305735+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304481+0000","last_change":"2024-04-25T10:15:05.304481+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304481+0000","last_undegraded":"2024-04-25T10:15:05.304481+0000","last_fullsized":"2024-04-25T10:15:05.304481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301889+0000","last_change":"2024-04-25T10:15:05.301889+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301889+0000","last_undegraded":"2024-04-25T10:15:05.301889+0000","last_fullsized":"2024-04-25T10:15:05.301889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.303774+0000","last_change":"2024-04-25T10:15:05.303774+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.303774+0000","last_undegraded":"2024-04-25T10:15:05.303774+0000","last_fullsized":"2024-04-25T10:15:05.303774+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302351+0000","last_change":"2024-04-25T10:15:05.302351+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302351+0000","last_undegraded":"2024-04-25T10:15:05.302351+0000","last_fullsized":"2024-04-25T10:15:05.302351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305363+0000","last_change":"2024-04-25T10:15:05.305363+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305363+0000","last_undegraded":"2024-04-25T10:15:05.305363+0000","last_fullsized":"2024-04-25T10:15:05.305363+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304418+0000","last_change":"2024-04-25T10:15:05.304418+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304418+0000","last_undegraded":"2024-04-25T10:15:05.304418+0000","last_fullsized":"2024-04-25T10:15:05.304418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304517+0000","last_change":"2024-04-25T10:15:05.304517+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304517+0000","last_undegraded":"2024-04-25T10:15:05.304517+0000","last_fullsized":"2024-04-25T10:15:05.304517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305639+0000","last_change":"2024-04-25T10:15:05.305639+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305639+0000","last_undegraded":"2024-04-25T10:15:05.305639+0000","last_fullsized":"2024-04-25T10:15:05.305639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301358+0000","last_change":"2024-04-25T10:15:05.301358+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301358+0000","last_undegraded":"2024-04-25T10:15:05.301358+0000","last_fullsized":"2024-04-25T10:15:05.301358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304717+0000","last_change":"2024-04-25T10:15:05.304717+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304717+0000","last_undegraded":"2024-04-25T10:15:05.304717+0000","last_fullsized":"2024-04-25T10:15:05.304717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304425+0000","last_change":"2024-04-25T10:15:05.304425+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304425+0000","last_undegraded":"2024-04-25T10:15:05.304425+0000","last_fullsized":"2024-04-25T10:15:05.304425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306086+0000","last_change":"2024-04-25T10:15:05.306086+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306086+0000","last_undegraded":"2024-04-25T10:15:05.306086+0000","last_fullsized":"2024-04-25T10:15:05.306086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305180+0000","last_change":"2024-04-25T10:15:05.305180+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305180+0000","last_undegraded":"2024-04-25T10:15:05.305180+0000","last_fullsized":"2024-04-25T10:15:05.305180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.304746+0000","last_change":"2024-04-25T10:15:05.304746+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.304746+0000","last_undegraded":"2024-04-25T10:15:05.304746+0000","last_fullsized":"2024-04-25T10:15:05.304746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.302296+0000","last_change":"2024-04-25T10:15:05.302296+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.302296+0000","last_undegraded":"2024-04-25T10:15:05.302296+0000","last_fullsized":"2024-04-25T10:15:05.302296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.306536+0000","last_change":"2024-04-25T10:15:05.306536+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.306536+0000","last_undegraded":"2024-04-25T10:15:05.306536+0000","last_fullsized":"2024-04-25T10:15:05.306536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.301360+0000","last_change":"2024-04-25T10:15:05.301360+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.301360+0000","last_undegraded":"2024-04-25T10:15:05.301360+0000","last_fullsized":"2024-04-25T10:15:05.301360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:15:06.298784+0000","last_change":"2024-04-25T10:15:06.298784+0000","last_active":"2024-04-25T10:15:06.298784+0000","last_peered":"2024-04-25T10:15:06.298784+0000","last_clean":"2024-04-25T10:15:06.298784+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:06.298784+0000","last_undegraded":"2024-04-25T10:15:06.298784+0000","last_fullsized":"2024-04-25T10:15:06.298784+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T10:15:05.305405+0000","last_change":"2024-04-25T10:15:05.305405+0000","last_active":"2024-04-25T10:15:05.293212+0000","last_peered":"2024-04-25T10:15:05.293212+0000","last_clean":"2024-04-25T10:15:05.293212+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:15:05.305405+0000","last_undegraded":"2024-04-25T10:15:05.305405+0000","last_fullsized":"2024-04-25T10:15:05.305405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:15:05.307740+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:15:05.307740+0000","last_peered":"2024-04-25T10:15:05.307740+0000","last_clean":"2024-04-25T10:15:05.307740+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:15:05.307740+0000","last_undegraded":"2024-04-25T10:15:05.307740+0000","last_fullsized":"2024-04-25T10:15:05.307740+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35836,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35832,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336008,"statfs":{"total":96636764160,"available":96600072192,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":503808,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1097728,"data_stored":793183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:15.937 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:16.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a34a00640 1 Processor -- start 2024-04-25T10:15:16.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a34a00640 1 -- start start 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a34a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4a300d1820 0x7f4a300d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a34a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4a3015e050 0x7f4a300b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a34a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a300b1af0 con 0x7f4a300b13e0 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a34a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a300b1dd0 con 0x7f4a300d1820 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a34a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a300b20b0 con 0x7f4a3015e050 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4a300d1820 0x7f4a300d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2e800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4a300b13e0 0x7f4a300ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56234/0 (socket says 172.21.15.87:56234) 2024-04-25T10:15:16.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2e800640 1 -- 172.21.15.87:0/3734474005 learned_addr learned my addr 172.21.15.87:0/3734474005 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:16.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a27e00640 1 --2- 172.21.15.87:0/3734474005 >> v2:172.21.15.169:3300/0 conn(0x7f4a3015e050 0x7f4a300b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2de00640 1 -- 172.21.15.87:0/3734474005 >> v2:172.21.15.169:3300/0 conn(0x7f4a3015e050 msgr2=0x7f4a300b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2de00640 1 --2- 172.21.15.87:0/3734474005 >> v2:172.21.15.169:3300/0 conn(0x7f4a3015e050 0x7f4a300b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2de00640 1 -- 172.21.15.87:0/3734474005 >> v1:172.21.15.87:6789/0 conn(0x7f4a300b13e0 legacy=0x7f4a300ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2de00640 1 -- 172.21.15.87:0/3734474005 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300b2390 con 0x7f4a300d1820 2024-04-25T10:15:16.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.080+0000 7f4a2d400640 1 -- 172.21.15.87:0/3734474005 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804306121 0 0) 0x7f4a300b1af0 con 0x7f4a300b13e0 2024-04-25T10:15:16.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a2de00640 1 --2- 172.21.15.87:0/3734474005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300d1820 0x7f4a300d1c00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4a300c5400 tx=0x7f4a2001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaf7d68d5ea60e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a2d400640 1 -- 172.21.15.87:0/3734474005 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a200aa600 con 0x7f4a300d1820 2024-04-25T10:15:16.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a2d400640 1 -- 172.21.15.87:0/3734474005 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a200af070 con 0x7f4a300d1820 2024-04-25T10:15:16.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a2d400640 1 -- 172.21.15.87:0/3734474005 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a200aa9c0 con 0x7f4a300d1820 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 -- 172.21.15.87:0/3734474005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300d1820 msgr2=0x7f4a300d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3734474005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300d1820 0x7f4a300d1c00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4a300c5400 tx=0x7f4a2001c480 comp rx=0 tx=0).stop 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 -- 172.21.15.87:0/3734474005 shutdown_connections 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3734474005 >> v2:172.21.15.169:3300/0 conn(0x7f4a3015e050 0x7f4a300b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3734474005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300d1820 0x7f4a300d1c00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 -- 172.21.15.87:0/3734474005 >> 172.21.15.87:0/3734474005 conn(0x7f4a3009a9d0 msgr2=0x7f4a3009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 -- 172.21.15.87:0/3734474005 shutdown_connections 2024-04-25T10:15:16.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.084+0000 7f4a34a00640 1 -- 172.21.15.87:0/3734474005 wait complete. 2024-04-25T10:15:16.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 Processor -- start 2024-04-25T10:15:16.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 -- start start 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300b13e0 0x7f4a30150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a300d1820 0x7f4a301573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a3015e050 0x7f4a301518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3016ab20 con 0x7f4a300b13e0 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3016a9a0 con 0x7f4a300d1820 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3016aca0 con 0x7f4a3015e050 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300b13e0 0x7f4a30150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300b13e0 0x7f4a30150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54892/0 (socket says 172.21.15.87:54892) 2024-04-25T10:15:16.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 -- 172.21.15.87:0/3194948591 learned_addr learned my addr 172.21.15.87:0/3194948591 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 -- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a3015e050 msgr2=0x7f4a301518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a27e00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a300d1820 0x7f4a301573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a3015e050 0x7f4a301518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 -- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a300d1820 msgr2=0x7f4a301573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a300d1820 0x7f4a301573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 -- 172.21.15.87:0/3194948591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30151e20 con 0x7f4a300b13e0 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a27e00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a300d1820 0x7f4a301573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a2de00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300b13e0 0x7f4a30150930 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4a200ac040 tx=0x7f4a2001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a26000640 1 -- 172.21.15.87:0/3194948591 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a200be610 con 0x7f4a300b13e0 2024-04-25T10:15:16.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a26000640 1 -- 172.21.15.87:0/3194948591 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a200be7b0 con 0x7f4a300b13e0 2024-04-25T10:15:16.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a26000640 1 -- 172.21.15.87:0/3194948591 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a200beba0 con 0x7f4a300b13e0 2024-04-25T10:15:16.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a301520c0 con 0x7f4a300b13e0 2024-04-25T10:15:16.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a300b52b0 con 0x7f4a300b13e0 2024-04-25T10:15:16.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.088+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a300b55e0 con 0x7f4a300b13e0 2024-04-25T10:15:16.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.092+0000 7f4a26000640 1 -- 172.21.15.87:0/3194948591 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a200bc020 con 0x7f4a300b13e0 2024-04-25T10:15:16.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.092+0000 7f4a26000640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4a0c066f60 0x7f4a0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.092+0000 7f4a26000640 1 -- 172.21.15.87:0/3194948591 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a2011f3a0 con 0x7f4a300b13e0 2024-04-25T10:15:16.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.092+0000 7f4a27e00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4a0c066f60 0x7f4a0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.092+0000 7f4a27e00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4a0c066f60 0x7f4a0c069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a300cdeb0 tx=0x7f4a18008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.096+0000 7f4a26000640 1 -- 172.21.15.87:0/3194948591 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a200c5030 con 0x7f4a300b13e0 2024-04-25T10:15:16.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.252+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a30152430 con 0x7f4a0c066f60 2024-04-25T10:15:16.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.256+0000 7f4a26000640 1 -- 172.21.15.87:0/3194948591 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251960 (secure 0 0 0) 0x7f4a30152430 con 0x7f4a0c066f60 2024-04-25T10:15:16.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:16.266 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:15:16.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4a0c066f60 msgr2=0x7f4a0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4a0c066f60 0x7f4a0c069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a300cdeb0 tx=0x7f4a18008040 comp rx=0 tx=0).stop 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300b13e0 msgr2=0x7f4a30150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300b13e0 0x7f4a30150930 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4a200ac040 tx=0x7f4a2001acc0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 shutdown_connections 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4a0c066f60 0x7f4a0c069420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a3015e050 0x7f4a301518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a300d1820 0x7f4a301573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 --2- 172.21.15.87:0/3194948591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a300b13e0 0x7f4a30150930 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 >> 172.21.15.87:0/3194948591 conn(0x7f4a3009a9d0 msgr2=0x7f4a3006fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 shutdown_connections 2024-04-25T10:15:16.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.260+0000 7f4a34a00640 1 -- 172.21.15.87:0/3194948591 wait complete. 2024-04-25T10:15:16.284 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-04-25T10:15:15.160964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296496,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754678224,"statfs":{"total":773094113280,"available":772790501376,"internally_reserved":0,"allocated":5578752,"data_stored":2962812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":297913905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001470"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036130+0000","last_change":"2024-04-25T10:15:07.337142+0000","last_active":"2024-04-25T10:15:09.036130+0000","last_peered":"2024-04-25T10:15:09.036130+0000","last_clean":"2024-04-25T10:15:09.036130+0000","last_became_active":"2024-04-25T10:15:07.336863+0000","last_became_peered":"2024-04-25T10:15:07.336863+0000","last_unstale":"2024-04-25T10:15:09.036130+0000","last_undegraded":"2024-04-25T10:15:09.036130+0000","last_fullsized":"2024-04-25T10:15:09.036130+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:41.797772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341013+0000","last_change":"2024-04-25T10:15:07.335631+0000","last_active":"2024-04-25T10:15:08.341013+0000","last_peered":"2024-04-25T10:15:08.341013+0000","last_clean":"2024-04-25T10:15:08.341013+0000","last_became_active":"2024-04-25T10:15:07.335369+0000","last_became_peered":"2024-04-25T10:15:07.335369+0000","last_unstale":"2024-04-25T10:15:08.341013+0000","last_undegraded":"2024-04-25T10:15:08.341013+0000","last_fullsized":"2024-04-25T10:15:08.341013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:08.507453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036025+0000","last_change":"2024-04-25T10:15:07.335610+0000","last_active":"2024-04-25T10:15:09.036025+0000","last_peered":"2024-04-25T10:15:09.036025+0000","last_clean":"2024-04-25T10:15:09.036025+0000","last_became_active":"2024-04-25T10:15:07.335355+0000","last_became_peered":"2024-04-25T10:15:07.335355+0000","last_unstale":"2024-04-25T10:15:09.036025+0000","last_undegraded":"2024-04-25T10:15:09.036025+0000","last_fullsized":"2024-04-25T10:15:09.036025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:45.487841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035183+0000","last_change":"2024-04-25T10:15:07.342047+0000","last_active":"2024-04-25T10:15:09.035183+0000","last_peered":"2024-04-25T10:15:09.035183+0000","last_clean":"2024-04-25T10:15:09.035183+0000","last_became_active":"2024-04-25T10:15:07.341763+0000","last_became_peered":"2024-04-25T10:15:07.341763+0000","last_unstale":"2024-04-25T10:15:09.035183+0000","last_undegraded":"2024-04-25T10:15:09.035183+0000","last_fullsized":"2024-04-25T10:15:09.035183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:15.327782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341210+0000","last_change":"2024-04-25T10:15:07.334890+0000","last_active":"2024-04-25T10:15:08.341210+0000","last_peered":"2024-04-25T10:15:08.341210+0000","last_clean":"2024-04-25T10:15:08.341210+0000","last_became_active":"2024-04-25T10:15:07.334512+0000","last_became_peered":"2024-04-25T10:15:07.334512+0000","last_unstale":"2024-04-25T10:15:08.341210+0000","last_undegraded":"2024-04-25T10:15:08.341210+0000","last_fullsized":"2024-04-25T10:15:08.341210+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:46.920836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035301+0000","last_change":"2024-04-25T10:15:07.341924+0000","last_active":"2024-04-25T10:15:09.035301+0000","last_peered":"2024-04-25T10:15:09.035301+0000","last_clean":"2024-04-25T10:15:09.035301+0000","last_became_active":"2024-04-25T10:15:07.341335+0000","last_became_peered":"2024-04-25T10:15:07.341335+0000","last_unstale":"2024-04-25T10:15:09.035301+0000","last_undegraded":"2024-04-25T10:15:09.035301+0000","last_fullsized":"2024-04-25T10:15:09.035301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:49.424793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036111+0000","last_change":"2024-04-25T10:15:07.346189+0000","last_active":"2024-04-25T10:15:09.036111+0000","last_peered":"2024-04-25T10:15:09.036111+0000","last_clean":"2024-04-25T10:15:09.036111+0000","last_became_active":"2024-04-25T10:15:07.345899+0000","last_became_peered":"2024-04-25T10:15:07.345899+0000","last_unstale":"2024-04-25T10:15:09.036111+0000","last_undegraded":"2024-04-25T10:15:09.036111+0000","last_fullsized":"2024-04-25T10:15:09.036111+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:46.920836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.034905+0000","last_change":"2024-04-25T10:15:07.332133+0000","last_active":"2024-04-25T10:15:09.034905+0000","last_peered":"2024-04-25T10:15:09.034905+0000","last_clean":"2024-04-25T10:15:09.034905+0000","last_became_active":"2024-04-25T10:15:07.331774+0000","last_became_peered":"2024-04-25T10:15:07.331774+0000","last_unstale":"2024-04-25T10:15:09.034905+0000","last_undegraded":"2024-04-25T10:15:09.034905+0000","last_fullsized":"2024-04-25T10:15:09.034905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:07.125887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036399+0000","last_change":"2024-04-25T10:15:07.345791+0000","last_active":"2024-04-25T10:15:09.036399+0000","last_peered":"2024-04-25T10:15:09.036399+0000","last_clean":"2024-04-25T10:15:09.036399+0000","last_became_active":"2024-04-25T10:15:07.345256+0000","last_became_peered":"2024-04-25T10:15:07.345256+0000","last_unstale":"2024-04-25T10:15:09.036399+0000","last_undegraded":"2024-04-25T10:15:09.036399+0000","last_fullsized":"2024-04-25T10:15:09.036399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:32.633323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036711+0000","last_change":"2024-04-25T10:15:07.329707+0000","last_active":"2024-04-25T10:15:09.036711+0000","last_peered":"2024-04-25T10:15:09.036711+0000","last_clean":"2024-04-25T10:15:09.036711+0000","last_became_active":"2024-04-25T10:15:07.329154+0000","last_became_peered":"2024-04-25T10:15:07.329154+0000","last_unstale":"2024-04-25T10:15:09.036711+0000","last_undegraded":"2024-04-25T10:15:09.036711+0000","last_fullsized":"2024-04-25T10:15:09.036711+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:29.699826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036093+0000","last_change":"2024-04-25T10:15:07.329448+0000","last_active":"2024-04-25T10:15:09.036093+0000","last_peered":"2024-04-25T10:15:09.036093+0000","last_clean":"2024-04-25T10:15:09.036093+0000","last_became_active":"2024-04-25T10:15:07.329171+0000","last_became_peered":"2024-04-25T10:15:07.329171+0000","last_unstale":"2024-04-25T10:15:09.036093+0000","last_undegraded":"2024-04-25T10:15:09.036093+0000","last_fullsized":"2024-04-25T10:15:09.036093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:25.752581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341411+0000","last_change":"2024-04-25T10:15:07.345043+0000","last_active":"2024-04-25T10:15:08.341411+0000","last_peered":"2024-04-25T10:15:08.341411+0000","last_clean":"2024-04-25T10:15:08.341411+0000","last_became_active":"2024-04-25T10:15:07.344602+0000","last_became_peered":"2024-04-25T10:15:07.344602+0000","last_unstale":"2024-04-25T10:15:08.341411+0000","last_undegraded":"2024-04-25T10:15:08.341411+0000","last_fullsized":"2024-04-25T10:15:08.341411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:41.088042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035601+0000","last_change":"2024-04-25T10:15:07.332942+0000","last_active":"2024-04-25T10:15:09.035601+0000","last_peered":"2024-04-25T10:15:09.035601+0000","last_clean":"2024-04-25T10:15:09.035601+0000","last_became_active":"2024-04-25T10:15:07.332429+0000","last_became_peered":"2024-04-25T10:15:07.332429+0000","last_unstale":"2024-04-25T10:15:09.035601+0000","last_undegraded":"2024-04-25T10:15:09.035601+0000","last_fullsized":"2024-04-25T10:15:09.035601+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:49.053655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035100+0000","last_change":"2024-04-25T10:15:07.331563+0000","last_active":"2024-04-25T10:15:09.035100+0000","last_peered":"2024-04-25T10:15:09.035100+0000","last_clean":"2024-04-25T10:15:09.035100+0000","last_became_active":"2024-04-25T10:15:07.330167+0000","last_became_peered":"2024-04-25T10:15:07.330167+0000","last_unstale":"2024-04-25T10:15:09.035100+0000","last_undegraded":"2024-04-25T10:15:09.035100+0000","last_fullsized":"2024-04-25T10:15:09.035100+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:47.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036425+0000","last_change":"2024-04-25T10:15:07.327216+0000","last_active":"2024-04-25T10:15:09.036425+0000","last_peered":"2024-04-25T10:15:09.036425+0000","last_clean":"2024-04-25T10:15:09.036425+0000","last_became_active":"2024-04-25T10:15:07.326904+0000","last_became_peered":"2024-04-25T10:15:07.326904+0000","last_unstale":"2024-04-25T10:15:09.036425+0000","last_undegraded":"2024-04-25T10:15:09.036425+0000","last_fullsized":"2024-04-25T10:15:09.036425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:07.997922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340600+0000","last_change":"2024-04-25T10:15:07.341473+0000","last_active":"2024-04-25T10:15:08.340600+0000","last_peered":"2024-04-25T10:15:08.340600+0000","last_clean":"2024-04-25T10:15:08.340600+0000","last_became_active":"2024-04-25T10:15:07.341131+0000","last_became_peered":"2024-04-25T10:15:07.341131+0000","last_unstale":"2024-04-25T10:15:08.340600+0000","last_undegraded":"2024-04-25T10:15:08.340600+0000","last_fullsized":"2024-04-25T10:15:08.340600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:12.473377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036206+0000","last_change":"2024-04-25T10:15:07.340254+0000","last_active":"2024-04-25T10:15:09.036206+0000","last_peered":"2024-04-25T10:15:09.036206+0000","last_clean":"2024-04-25T10:15:09.036206+0000","last_became_active":"2024-04-25T10:15:07.339933+0000","last_became_peered":"2024-04-25T10:15:07.339933+0000","last_unstale":"2024-04-25T10:15:09.036206+0000","last_undegraded":"2024-04-25T10:15:09.036206+0000","last_fullsized":"2024-04-25T10:15:09.036206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:25.752581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036134+0000","last_change":"2024-04-25T10:15:07.333150+0000","last_active":"2024-04-25T10:15:09.036134+0000","last_peered":"2024-04-25T10:15:09.036134+0000","last_clean":"2024-04-25T10:15:09.036134+0000","last_became_active":"2024-04-25T10:15:07.332857+0000","last_became_peered":"2024-04-25T10:15:07.332857+0000","last_unstale":"2024-04-25T10:15:09.036134+0000","last_undegraded":"2024-04-25T10:15:09.036134+0000","last_fullsized":"2024-04-25T10:15:09.036134+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:25.804657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341216+0000","last_change":"2024-04-25T10:15:07.339542+0000","last_active":"2024-04-25T10:15:08.341216+0000","last_peered":"2024-04-25T10:15:08.341216+0000","last_clean":"2024-04-25T10:15:08.341216+0000","last_became_active":"2024-04-25T10:15:07.334477+0000","last_became_peered":"2024-04-25T10:15:07.334477+0000","last_unstale":"2024-04-25T10:15:08.341216+0000","last_undegraded":"2024-04-25T10:15:08.341216+0000","last_fullsized":"2024-04-25T10:15:08.341216+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:14.919828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036047+0000","last_change":"2024-04-25T10:15:07.332707+0000","last_active":"2024-04-25T10:15:09.036047+0000","last_peered":"2024-04-25T10:15:09.036047+0000","last_clean":"2024-04-25T10:15:09.036047+0000","last_became_active":"2024-04-25T10:15:07.332434+0000","last_became_peered":"2024-04-25T10:15:07.332434+0000","last_unstale":"2024-04-25T10:15:09.036047+0000","last_undegraded":"2024-04-25T10:15:09.036047+0000","last_fullsized":"2024-04-25T10:15:09.036047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:35.762772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035628+0000","last_change":"2024-04-25T10:15:07.333931+0000","last_active":"2024-04-25T10:15:09.035628+0000","last_peered":"2024-04-25T10:15:09.035628+0000","last_clean":"2024-04-25T10:15:09.035628+0000","last_became_active":"2024-04-25T10:15:07.333627+0000","last_became_peered":"2024-04-25T10:15:07.333627+0000","last_unstale":"2024-04-25T10:15:09.035628+0000","last_undegraded":"2024-04-25T10:15:09.035628+0000","last_fullsized":"2024-04-25T10:15:09.035628+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:41.222078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035959+0000","last_change":"2024-04-25T10:15:07.341906+0000","last_active":"2024-04-25T10:15:09.035959+0000","last_peered":"2024-04-25T10:15:09.035959+0000","last_clean":"2024-04-25T10:15:09.035959+0000","last_became_active":"2024-04-25T10:15:07.341636+0000","last_became_peered":"2024-04-25T10:15:07.341636+0000","last_unstale":"2024-04-25T10:15:09.035959+0000","last_undegraded":"2024-04-25T10:15:09.035959+0000","last_fullsized":"2024-04-25T10:15:09.035959+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:25.804657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340524+0000","last_change":"2024-04-25T10:15:07.341355+0000","last_active":"2024-04-25T10:15:08.340524+0000","last_peered":"2024-04-25T10:15:08.340524+0000","last_clean":"2024-04-25T10:15:08.340524+0000","last_became_active":"2024-04-25T10:15:07.340600+0000","last_became_peered":"2024-04-25T10:15:07.340600+0000","last_unstale":"2024-04-25T10:15:08.340524+0000","last_undegraded":"2024-04-25T10:15:08.340524+0000","last_fullsized":"2024-04-25T10:15:08.340524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.339908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340312+0000","last_change":"2024-04-25T10:15:07.340723+0000","last_active":"2024-04-25T10:15:08.340312+0000","last_peered":"2024-04-25T10:15:08.340312+0000","last_clean":"2024-04-25T10:15:08.340312+0000","last_became_active":"2024-04-25T10:15:07.340437+0000","last_became_peered":"2024-04-25T10:15:07.340437+0000","last_unstale":"2024-04-25T10:15:08.340312+0000","last_undegraded":"2024-04-25T10:15:08.340312+0000","last_fullsized":"2024-04-25T10:15:08.340312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:54.203617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341687+0000","last_change":"2024-04-25T10:15:07.348759+0000","last_active":"2024-04-25T10:15:08.341687+0000","last_peered":"2024-04-25T10:15:08.341687+0000","last_clean":"2024-04-25T10:15:08.341687+0000","last_became_active":"2024-04-25T10:15:07.348468+0000","last_became_peered":"2024-04-25T10:15:07.348468+0000","last_unstale":"2024-04-25T10:15:08.341687+0000","last_undegraded":"2024-04-25T10:15:08.341687+0000","last_fullsized":"2024-04-25T10:15:08.341687+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:53.289220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036339+0000","last_change":"2024-04-25T10:15:07.335001+0000","last_active":"2024-04-25T10:15:09.036339+0000","last_peered":"2024-04-25T10:15:09.036339+0000","last_clean":"2024-04-25T10:15:09.036339+0000","last_became_active":"2024-04-25T10:15:07.334703+0000","last_became_peered":"2024-04-25T10:15:07.334703+0000","last_unstale":"2024-04-25T10:15:09.036339+0000","last_undegraded":"2024-04-25T10:15:09.036339+0000","last_fullsized":"2024-04-25T10:15:09.036339+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:20.586317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036019+0000","last_change":"2024-04-25T10:15:07.340262+0000","last_active":"2024-04-25T10:15:09.036019+0000","last_peered":"2024-04-25T10:15:09.036019+0000","last_clean":"2024-04-25T10:15:09.036019+0000","last_became_active":"2024-04-25T10:15:07.339947+0000","last_became_peered":"2024-04-25T10:15:07.339947+0000","last_unstale":"2024-04-25T10:15:09.036019+0000","last_undegraded":"2024-04-25T10:15:09.036019+0000","last_fullsized":"2024-04-25T10:15:09.036019+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:25.066037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341108+0000","last_change":"2024-04-25T10:15:07.345708+0000","last_active":"2024-04-25T10:15:08.341108+0000","last_peered":"2024-04-25T10:15:08.341108+0000","last_clean":"2024-04-25T10:15:08.341108+0000","last_became_active":"2024-04-25T10:15:07.343742+0000","last_became_peered":"2024-04-25T10:15:07.343742+0000","last_unstale":"2024-04-25T10:15:08.341108+0000","last_undegraded":"2024-04-25T10:15:08.341108+0000","last_fullsized":"2024-04-25T10:15:08.341108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:29.699826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341453+0000","last_change":"2024-04-25T10:15:07.345177+0000","last_active":"2024-04-25T10:15:08.341453+0000","last_peered":"2024-04-25T10:15:08.341453+0000","last_clean":"2024-04-25T10:15:08.341453+0000","last_became_active":"2024-04-25T10:15:07.344901+0000","last_became_peered":"2024-04-25T10:15:07.344901+0000","last_unstale":"2024-04-25T10:15:08.341453+0000","last_undegraded":"2024-04-25T10:15:08.341453+0000","last_fullsized":"2024-04-25T10:15:08.341453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:49.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035672+0000","last_change":"2024-04-25T10:15:07.334597+0000","last_active":"2024-04-25T10:15:09.035672+0000","last_peered":"2024-04-25T10:15:09.035672+0000","last_clean":"2024-04-25T10:15:09.035672+0000","last_became_active":"2024-04-25T10:15:07.334262+0000","last_became_peered":"2024-04-25T10:15:07.334262+0000","last_unstale":"2024-04-25T10:15:09.035672+0000","last_undegraded":"2024-04-25T10:15:09.035672+0000","last_fullsized":"2024-04-25T10:15:09.035672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:47.448092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036282+0000","last_change":"2024-04-25T10:15:07.333985+0000","last_active":"2024-04-25T10:15:09.036282+0000","last_peered":"2024-04-25T10:15:09.036282+0000","last_clean":"2024-04-25T10:15:09.036282+0000","last_became_active":"2024-04-25T10:15:07.333707+0000","last_became_peered":"2024-04-25T10:15:07.333707+0000","last_unstale":"2024-04-25T10:15:09.036282+0000","last_undegraded":"2024-04-25T10:15:09.036282+0000","last_fullsized":"2024-04-25T10:15:09.036282+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:14.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035584+0000","last_change":"2024-04-25T10:15:07.333855+0000","last_active":"2024-04-25T10:15:09.035584+0000","last_peered":"2024-04-25T10:15:09.035584+0000","last_clean":"2024-04-25T10:15:09.035584+0000","last_became_active":"2024-04-25T10:15:07.333591+0000","last_became_peered":"2024-04-25T10:15:07.333591+0000","last_unstale":"2024-04-25T10:15:09.035584+0000","last_undegraded":"2024-04-25T10:15:09.035584+0000","last_fullsized":"2024-04-25T10:15:09.035584+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:13.127430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.338737+0000","last_change":"2024-04-25T10:15:07.334408+0000","last_active":"2024-04-25T10:15:08.338737+0000","last_peered":"2024-04-25T10:15:08.338737+0000","last_clean":"2024-04-25T10:15:08.338737+0000","last_became_active":"2024-04-25T10:15:07.334113+0000","last_became_peered":"2024-04-25T10:15:07.334113+0000","last_unstale":"2024-04-25T10:15:08.338737+0000","last_undegraded":"2024-04-25T10:15:08.338737+0000","last_fullsized":"2024-04-25T10:15:08.338737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:32.247504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035829+0000","last_change":"2024-04-25T10:15:06.327089+0000","last_active":"2024-04-25T10:15:09.035829+0000","last_peered":"2024-04-25T10:15:09.035829+0000","last_clean":"2024-04-25T10:15:09.035829+0000","last_became_active":"2024-04-25T10:15:06.326473+0000","last_became_peered":"2024-04-25T10:15:06.326473+0000","last_unstale":"2024-04-25T10:15:09.035829+0000","last_undegraded":"2024-04-25T10:15:09.035829+0000","last_fullsized":"2024-04-25T10:15:09.035829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:31.627751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341647+0000","last_change":"2024-04-25T10:15:07.339748+0000","last_active":"2024-04-25T10:15:08.341647+0000","last_peered":"2024-04-25T10:15:08.341647+0000","last_clean":"2024-04-25T10:15:08.341647+0000","last_became_active":"2024-04-25T10:15:07.339234+0000","last_became_peered":"2024-04-25T10:15:07.339234+0000","last_unstale":"2024-04-25T10:15:08.341647+0000","last_undegraded":"2024-04-25T10:15:08.341647+0000","last_fullsized":"2024-04-25T10:15:08.341647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:32.247504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036194+0000","last_change":"2024-04-25T10:15:06.328605+0000","last_active":"2024-04-25T10:15:09.036194+0000","last_peered":"2024-04-25T10:15:09.036194+0000","last_clean":"2024-04-25T10:15:09.036194+0000","last_became_active":"2024-04-25T10:15:06.326800+0000","last_became_peered":"2024-04-25T10:15:06.326800+0000","last_unstale":"2024-04-25T10:15:09.036194+0000","last_undegraded":"2024-04-25T10:15:09.036194+0000","last_fullsized":"2024-04-25T10:15:09.036194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:11.467805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341306+0000","last_change":"2024-04-25T10:15:07.333785+0000","last_active":"2024-04-25T10:15:08.341306+0000","last_peered":"2024-04-25T10:15:08.341306+0000","last_clean":"2024-04-25T10:15:08.341306+0000","last_became_active":"2024-04-25T10:15:07.333499+0000","last_became_peered":"2024-04-25T10:15:07.333499+0000","last_unstale":"2024-04-25T10:15:08.341306+0000","last_undegraded":"2024-04-25T10:15:08.341306+0000","last_fullsized":"2024-04-25T10:15:08.341306+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:41.797772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.338817+0000","last_change":"2024-04-25T10:15:06.326898+0000","last_active":"2024-04-25T10:15:08.338817+0000","last_peered":"2024-04-25T10:15:08.338817+0000","last_clean":"2024-04-25T10:15:08.338817+0000","last_became_active":"2024-04-25T10:15:06.326624+0000","last_became_peered":"2024-04-25T10:15:06.326624+0000","last_unstale":"2024-04-25T10:15:08.338817+0000","last_undegraded":"2024-04-25T10:15:08.338817+0000","last_fullsized":"2024-04-25T10:15:08.338817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:11.467805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.338755+0000","last_change":"2024-04-25T10:15:07.344066+0000","last_active":"2024-04-25T10:15:08.338755+0000","last_peered":"2024-04-25T10:15:08.338755+0000","last_clean":"2024-04-25T10:15:08.338755+0000","last_became_active":"2024-04-25T10:15:07.342886+0000","last_became_peered":"2024-04-25T10:15:07.342886+0000","last_unstale":"2024-04-25T10:15:08.338755+0000","last_undegraded":"2024-04-25T10:15:08.338755+0000","last_fullsized":"2024-04-25T10:15:08.338755+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:53.289220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036227+0000","last_change":"2024-04-25T10:15:06.326297+0000","last_active":"2024-04-25T10:15:09.036227+0000","last_peered":"2024-04-25T10:15:09.036227+0000","last_clean":"2024-04-25T10:15:09.036227+0000","last_became_active":"2024-04-25T10:15:06.325842+0000","last_became_peered":"2024-04-25T10:15:06.325842+0000","last_unstale":"2024-04-25T10:15:09.036227+0000","last_undegraded":"2024-04-25T10:15:09.036227+0000","last_fullsized":"2024-04-25T10:15:09.036227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:46.442520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036492+0000","last_change":"2024-04-25T10:15:07.333694+0000","last_active":"2024-04-25T10:15:09.036492+0000","last_peered":"2024-04-25T10:15:09.036492+0000","last_clean":"2024-04-25T10:15:09.036492+0000","last_became_active":"2024-04-25T10:15:07.332344+0000","last_became_peered":"2024-04-25T10:15:07.332344+0000","last_unstale":"2024-04-25T10:15:09.036492+0000","last_undegraded":"2024-04-25T10:15:09.036492+0000","last_fullsized":"2024-04-25T10:15:09.036492+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:32.633323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035827+0000","last_change":"2024-04-25T10:15:06.325468+0000","last_active":"2024-04-25T10:15:09.035827+0000","last_peered":"2024-04-25T10:15:09.035827+0000","last_clean":"2024-04-25T10:15:09.035827+0000","last_became_active":"2024-04-25T10:15:06.325105+0000","last_became_peered":"2024-04-25T10:15:06.325105+0000","last_unstale":"2024-04-25T10:15:09.035827+0000","last_undegraded":"2024-04-25T10:15:09.035827+0000","last_fullsized":"2024-04-25T10:15:09.035827+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:59.230266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035286+0000","last_change":"2024-04-25T10:15:07.331446+0000","last_active":"2024-04-25T10:15:09.035286+0000","last_peered":"2024-04-25T10:15:09.035286+0000","last_clean":"2024-04-25T10:15:09.035286+0000","last_became_active":"2024-04-25T10:15:07.331084+0000","last_became_peered":"2024-04-25T10:15:07.331084+0000","last_unstale":"2024-04-25T10:15:09.035286+0000","last_undegraded":"2024-04-25T10:15:09.035286+0000","last_fullsized":"2024-04-25T10:15:09.035286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:25.066037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035927+0000","last_change":"2024-04-25T10:15:06.324442+0000","last_active":"2024-04-25T10:15:09.035927+0000","last_peered":"2024-04-25T10:15:09.035927+0000","last_clean":"2024-04-25T10:15:09.035927+0000","last_became_active":"2024-04-25T10:15:06.324121+0000","last_became_peered":"2024-04-25T10:15:06.324121+0000","last_unstale":"2024-04-25T10:15:09.035927+0000","last_undegraded":"2024-04-25T10:15:09.035927+0000","last_fullsized":"2024-04-25T10:15:09.035927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:53.198045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.034973+0000","last_change":"2024-04-25T10:15:07.331006+0000","last_active":"2024-04-25T10:15:09.034973+0000","last_peered":"2024-04-25T10:15:09.034973+0000","last_clean":"2024-04-25T10:15:09.034973+0000","last_became_active":"2024-04-25T10:15:07.330625+0000","last_became_peered":"2024-04-25T10:15:07.330625+0000","last_unstale":"2024-04-25T10:15:09.034973+0000","last_undegraded":"2024-04-25T10:15:09.034973+0000","last_fullsized":"2024-04-25T10:15:09.034973+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:03.467805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340657+0000","last_change":"2024-04-25T10:15:06.331272+0000","last_active":"2024-04-25T10:15:08.340657+0000","last_peered":"2024-04-25T10:15:08.340657+0000","last_clean":"2024-04-25T10:15:08.340657+0000","last_became_active":"2024-04-25T10:15:06.330996+0000","last_became_peered":"2024-04-25T10:15:06.330996+0000","last_unstale":"2024-04-25T10:15:08.340657+0000","last_undegraded":"2024-04-25T10:15:08.340657+0000","last_fullsized":"2024-04-25T10:15:08.340657+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:14.163148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340279+0000","last_change":"2024-04-25T10:15:07.341609+0000","last_active":"2024-04-25T10:15:08.340279+0000","last_peered":"2024-04-25T10:15:08.340279+0000","last_clean":"2024-04-25T10:15:08.340279+0000","last_became_active":"2024-04-25T10:15:07.341064+0000","last_became_peered":"2024-04-25T10:15:07.341064+0000","last_unstale":"2024-04-25T10:15:08.340279+0000","last_undegraded":"2024-04-25T10:15:08.340279+0000","last_fullsized":"2024-04-25T10:15:08.340279+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:32.578652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.034979+0000","last_change":"2024-04-25T10:15:06.326829+0000","last_active":"2024-04-25T10:15:09.034979+0000","last_peered":"2024-04-25T10:15:09.034979+0000","last_clean":"2024-04-25T10:15:09.034979+0000","last_became_active":"2024-04-25T10:15:06.322670+0000","last_became_peered":"2024-04-25T10:15:06.322670+0000","last_unstale":"2024-04-25T10:15:09.034979+0000","last_undegraded":"2024-04-25T10:15:09.034979+0000","last_fullsized":"2024-04-25T10:15:09.034979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:48.048083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036271+0000","last_change":"2024-04-25T10:15:07.332163+0000","last_active":"2024-04-25T10:15:09.036271+0000","last_peered":"2024-04-25T10:15:09.036271+0000","last_clean":"2024-04-25T10:15:09.036271+0000","last_became_active":"2024-04-25T10:15:07.331471+0000","last_became_peered":"2024-04-25T10:15:07.331471+0000","last_unstale":"2024-04-25T10:15:09.036271+0000","last_undegraded":"2024-04-25T10:15:09.036271+0000","last_fullsized":"2024-04-25T10:15:09.036271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:00.235837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036439+0000","last_change":"2024-04-25T10:15:06.321801+0000","last_active":"2024-04-25T10:15:09.036439+0000","last_peered":"2024-04-25T10:15:09.036439+0000","last_clean":"2024-04-25T10:15:09.036439+0000","last_became_active":"2024-04-25T10:15:06.321450+0000","last_became_peered":"2024-04-25T10:15:06.321450+0000","last_unstale":"2024-04-25T10:15:09.036439+0000","last_undegraded":"2024-04-25T10:15:09.036439+0000","last_fullsized":"2024-04-25T10:15:09.036439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:37.849716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036322+0000","last_change":"2024-04-25T10:15:07.336595+0000","last_active":"2024-04-25T10:15:09.036322+0000","last_peered":"2024-04-25T10:15:09.036322+0000","last_clean":"2024-04-25T10:15:09.036322+0000","last_became_active":"2024-04-25T10:15:07.336241+0000","last_became_peered":"2024-04-25T10:15:07.336241+0000","last_unstale":"2024-04-25T10:15:09.036322+0000","last_undegraded":"2024-04-25T10:15:09.036322+0000","last_fullsized":"2024-04-25T10:15:09.036322+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:10.289904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036740+0000","last_change":"2024-04-25T10:15:06.323274+0000","last_active":"2024-04-25T10:15:09.036740+0000","last_peered":"2024-04-25T10:15:09.036740+0000","last_clean":"2024-04-25T10:15:09.036740+0000","last_became_active":"2024-04-25T10:15:06.323022+0000","last_became_peered":"2024-04-25T10:15:06.323022+0000","last_unstale":"2024-04-25T10:15:09.036740+0000","last_undegraded":"2024-04-25T10:15:09.036740+0000","last_fullsized":"2024-04-25T10:15:09.036740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:16.681408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036059+0000","last_change":"2024-04-25T10:15:07.337616+0000","last_active":"2024-04-25T10:15:09.036059+0000","last_peered":"2024-04-25T10:15:09.036059+0000","last_clean":"2024-04-25T10:15:09.036059+0000","last_became_active":"2024-04-25T10:15:07.336945+0000","last_became_peered":"2024-04-25T10:15:07.336945+0000","last_unstale":"2024-04-25T10:15:09.036059+0000","last_undegraded":"2024-04-25T10:15:09.036059+0000","last_fullsized":"2024-04-25T10:15:09.036059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:07.170798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036572+0000","last_change":"2024-04-25T10:15:06.327519+0000","last_active":"2024-04-25T10:15:09.036572+0000","last_peered":"2024-04-25T10:15:09.036572+0000","last_clean":"2024-04-25T10:15:09.036572+0000","last_became_active":"2024-04-25T10:15:06.327228+0000","last_became_peered":"2024-04-25T10:15:06.327228+0000","last_unstale":"2024-04-25T10:15:09.036572+0000","last_undegraded":"2024-04-25T10:15:09.036572+0000","last_fullsized":"2024-04-25T10:15:09.036572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:24.060465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036514+0000","last_change":"2024-04-25T10:15:07.340492+0000","last_active":"2024-04-25T10:15:09.036514+0000","last_peered":"2024-04-25T10:15:09.036514+0000","last_clean":"2024-04-25T10:15:09.036514+0000","last_became_active":"2024-04-25T10:15:07.340217+0000","last_became_peered":"2024-04-25T10:15:07.340217+0000","last_unstale":"2024-04-25T10:15:09.036514+0000","last_undegraded":"2024-04-25T10:15:09.036514+0000","last_fullsized":"2024-04-25T10:15:09.036514+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:55.345387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036100+0000","last_change":"2024-04-25T10:15:06.319051+0000","last_active":"2024-04-25T10:15:09.036100+0000","last_peered":"2024-04-25T10:15:09.036100+0000","last_clean":"2024-04-25T10:15:09.036100+0000","last_became_active":"2024-04-25T10:15:06.318781+0000","last_became_peered":"2024-04-25T10:15:06.318781+0000","last_unstale":"2024-04-25T10:15:09.036100+0000","last_undegraded":"2024-04-25T10:15:09.036100+0000","last_fullsized":"2024-04-25T10:15:09.036100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:15.471778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340553+0000","last_change":"2024-04-25T10:15:07.341736+0000","last_active":"2024-04-25T10:15:08.340553+0000","last_peered":"2024-04-25T10:15:08.340553+0000","last_clean":"2024-04-25T10:15:08.340553+0000","last_became_active":"2024-04-25T10:15:07.341447+0000","last_became_peered":"2024-04-25T10:15:07.341447+0000","last_unstale":"2024-04-25T10:15:08.340553+0000","last_undegraded":"2024-04-25T10:15:08.340553+0000","last_fullsized":"2024-04-25T10:15:08.340553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:38.855287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035941+0000","last_change":"2024-04-25T10:15:06.322391+0000","last_active":"2024-04-25T10:15:09.035941+0000","last_peered":"2024-04-25T10:15:09.035941+0000","last_clean":"2024-04-25T10:15:09.035941+0000","last_became_active":"2024-04-25T10:15:06.322042+0000","last_became_peered":"2024-04-25T10:15:06.322042+0000","last_unstale":"2024-04-25T10:15:09.035941+0000","last_undegraded":"2024-04-25T10:15:09.035941+0000","last_fullsized":"2024-04-25T10:15:09.035941+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:07.501881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035428+0000","last_change":"2024-04-25T10:15:07.342787+0000","last_active":"2024-04-25T10:15:09.035428+0000","last_peered":"2024-04-25T10:15:09.035428+0000","last_clean":"2024-04-25T10:15:09.035428+0000","last_became_active":"2024-04-25T10:15:07.342519+0000","last_became_peered":"2024-04-25T10:15:07.342519+0000","last_unstale":"2024-04-25T10:15:09.035428+0000","last_undegraded":"2024-04-25T10:15:09.035428+0000","last_fullsized":"2024-04-25T10:15:09.035428+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:45.487841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036021+0000","last_change":"2024-04-25T10:15:06.331839+0000","last_active":"2024-04-25T10:15:09.036021+0000","last_peered":"2024-04-25T10:15:09.036021+0000","last_clean":"2024-04-25T10:15:09.036021+0000","last_became_active":"2024-04-25T10:15:06.331259+0000","last_became_peered":"2024-04-25T10:15:06.331259+0000","last_unstale":"2024-04-25T10:15:09.036021+0000","last_undegraded":"2024-04-25T10:15:09.036021+0000","last_fullsized":"2024-04-25T10:15:09.036021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:46.626933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035997+0000","last_change":"2024-04-25T10:15:07.337405+0000","last_active":"2024-04-25T10:15:09.035997+0000","last_peered":"2024-04-25T10:15:09.035997+0000","last_clean":"2024-04-25T10:15:09.035997+0000","last_became_active":"2024-04-25T10:15:07.337127+0000","last_became_peered":"2024-04-25T10:15:07.337127+0000","last_unstale":"2024-04-25T10:15:09.035997+0000","last_undegraded":"2024-04-25T10:15:09.035997+0000","last_fullsized":"2024-04-25T10:15:09.035997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:06.093371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036331+0000","last_change":"2024-04-25T10:15:06.325152+0000","last_active":"2024-04-25T10:15:09.036331+0000","last_peered":"2024-04-25T10:15:09.036331+0000","last_clean":"2024-04-25T10:15:09.036331+0000","last_became_active":"2024-04-25T10:15:06.323639+0000","last_became_peered":"2024-04-25T10:15:06.323639+0000","last_unstale":"2024-04-25T10:15:09.036331+0000","last_undegraded":"2024-04-25T10:15:09.036331+0000","last_fullsized":"2024-04-25T10:15:09.036331+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:24.747010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036284+0000","last_change":"2024-04-25T10:15:07.332766+0000","last_active":"2024-04-25T10:15:09.036284+0000","last_peered":"2024-04-25T10:15:09.036284+0000","last_clean":"2024-04-25T10:15:09.036284+0000","last_became_active":"2024-04-25T10:15:07.332520+0000","last_became_peered":"2024-04-25T10:15:07.332520+0000","last_unstale":"2024-04-25T10:15:09.036284+0000","last_undegraded":"2024-04-25T10:15:09.036284+0000","last_fullsized":"2024-04-25T10:15:09.036284+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:41.088042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035077+0000","last_change":"2024-04-25T10:15:06.326272+0000","last_active":"2024-04-25T10:15:09.035077+0000","last_peered":"2024-04-25T10:15:09.035077+0000","last_clean":"2024-04-25T10:15:09.035077+0000","last_became_active":"2024-04-25T10:15:06.326024+0000","last_became_peered":"2024-04-25T10:15:06.326024+0000","last_unstale":"2024-04-25T10:15:09.035077+0000","last_undegraded":"2024-04-25T10:15:09.035077+0000","last_fullsized":"2024-04-25T10:15:09.035077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:19.580746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036489+0000","last_change":"2024-04-25T10:15:07.343370+0000","last_active":"2024-04-25T10:15:09.036489+0000","last_peered":"2024-04-25T10:15:09.036489+0000","last_clean":"2024-04-25T10:15:09.036489+0000","last_became_active":"2024-04-25T10:15:07.343118+0000","last_became_peered":"2024-04-25T10:15:07.343118+0000","last_unstale":"2024-04-25T10:15:09.036489+0000","last_undegraded":"2024-04-25T10:15:09.036489+0000","last_fullsized":"2024-04-25T10:15:09.036489+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:39.589103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035385+0000","last_change":"2024-04-25T10:15:06.319671+0000","last_active":"2024-04-25T10:15:09.035385+0000","last_peered":"2024-04-25T10:15:09.035385+0000","last_clean":"2024-04-25T10:15:09.035385+0000","last_became_active":"2024-04-25T10:15:06.319411+0000","last_became_peered":"2024-04-25T10:15:06.319411+0000","last_unstale":"2024-04-25T10:15:09.035385+0000","last_undegraded":"2024-04-25T10:15:09.035385+0000","last_fullsized":"2024-04-25T10:15:09.035385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:37.756996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036525+0000","last_change":"2024-04-25T10:15:07.335532+0000","last_active":"2024-04-25T10:15:09.036525+0000","last_peered":"2024-04-25T10:15:09.036525+0000","last_clean":"2024-04-25T10:15:09.036525+0000","last_became_active":"2024-04-25T10:15:07.335262+0000","last_became_peered":"2024-04-25T10:15:07.335262+0000","last_unstale":"2024-04-25T10:15:09.036525+0000","last_undegraded":"2024-04-25T10:15:09.036525+0000","last_fullsized":"2024-04-25T10:15:09.036525+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:08.507453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036330+0000","last_change":"2024-04-25T10:15:06.324070+0000","last_active":"2024-04-25T10:15:09.036330+0000","last_peered":"2024-04-25T10:15:09.036330+0000","last_clean":"2024-04-25T10:15:09.036330+0000","last_became_active":"2024-04-25T10:15:06.323428+0000","last_became_peered":"2024-04-25T10:15:06.323428+0000","last_unstale":"2024-04-25T10:15:09.036330+0000","last_undegraded":"2024-04-25T10:15:09.036330+0000","last_fullsized":"2024-04-25T10:15:09.036330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:19.474322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340667+0000","last_change":"2024-04-25T10:15:07.338373+0000","last_active":"2024-04-25T10:15:08.340667+0000","last_peered":"2024-04-25T10:15:08.340667+0000","last_clean":"2024-04-25T10:15:08.340667+0000","last_became_active":"2024-04-25T10:15:07.338128+0000","last_became_peered":"2024-04-25T10:15:07.338128+0000","last_unstale":"2024-04-25T10:15:08.340667+0000","last_undegraded":"2024-04-25T10:15:08.340667+0000","last_fullsized":"2024-04-25T10:15:08.340667+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:20.479894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340997+0000","last_change":"2024-04-25T10:15:06.328768+0000","last_active":"2024-04-25T10:15:08.340997+0000","last_peered":"2024-04-25T10:15:08.340997+0000","last_clean":"2024-04-25T10:15:08.340997+0000","last_became_active":"2024-04-25T10:15:06.328497+0000","last_became_peered":"2024-04-25T10:15:06.328497+0000","last_unstale":"2024-04-25T10:15:08.340997+0000","last_undegraded":"2024-04-25T10:15:08.340997+0000","last_fullsized":"2024-04-25T10:15:08.340997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:02.462234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340990+0000","last_change":"2024-04-25T10:15:07.346683+0000","last_active":"2024-04-25T10:15:08.340990+0000","last_peered":"2024-04-25T10:15:08.340990+0000","last_clean":"2024-04-25T10:15:08.340990+0000","last_became_active":"2024-04-25T10:15:07.346315+0000","last_became_peered":"2024-04-25T10:15:07.346315+0000","last_unstale":"2024-04-25T10:15:08.340990+0000","last_undegraded":"2024-04-25T10:15:08.340990+0000","last_fullsized":"2024-04-25T10:15:08.340990+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:38.394606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036108+0000","last_change":"2024-04-25T10:15:06.323548+0000","last_active":"2024-04-25T10:15:09.036108+0000","last_peered":"2024-04-25T10:15:09.036108+0000","last_clean":"2024-04-25T10:15:09.036108+0000","last_became_active":"2024-04-25T10:15:06.323306+0000","last_became_peered":"2024-04-25T10:15:06.323306+0000","last_unstale":"2024-04-25T10:15:09.036108+0000","last_undegraded":"2024-04-25T10:15:09.036108+0000","last_fullsized":"2024-04-25T10:15:09.036108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:24.799085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035437+0000","last_change":"2024-04-25T10:15:07.341004+0000","last_active":"2024-04-25T10:15:09.035437+0000","last_peered":"2024-04-25T10:15:09.035437+0000","last_clean":"2024-04-25T10:15:09.035437+0000","last_became_active":"2024-04-25T10:15:07.337446+0000","last_became_peered":"2024-04-25T10:15:07.337446+0000","last_unstale":"2024-04-25T10:15:09.035437+0000","last_undegraded":"2024-04-25T10:15:09.035437+0000","last_fullsized":"2024-04-25T10:15:09.035437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.731535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036106+0000","last_change":"2024-04-25T10:15:06.325907+0000","last_active":"2024-04-25T10:15:09.036106+0000","last_peered":"2024-04-25T10:15:09.036106+0000","last_clean":"2024-04-25T10:15:09.036106+0000","last_became_active":"2024-04-25T10:15:06.325156+0000","last_became_peered":"2024-04-25T10:15:06.325156+0000","last_unstale":"2024-04-25T10:15:09.036106+0000","last_undegraded":"2024-04-25T10:15:09.036106+0000","last_fullsized":"2024-04-25T10:15:09.036106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:28.694254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.338558+0000","last_change":"2024-04-25T10:15:07.338412+0000","last_active":"2024-04-25T10:15:08.338558+0000","last_peered":"2024-04-25T10:15:08.338558+0000","last_clean":"2024-04-25T10:15:08.338558+0000","last_became_active":"2024-04-25T10:15:07.338123+0000","last_became_peered":"2024-04-25T10:15:07.338123+0000","last_unstale":"2024-04-25T10:15:08.338558+0000","last_undegraded":"2024-04-25T10:15:08.338558+0000","last_fullsized":"2024-04-25T10:15:08.338558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:49.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340779+0000","last_change":"2024-04-25T10:15:06.327458+0000","last_active":"2024-04-25T10:15:08.340779+0000","last_peered":"2024-04-25T10:15:08.340779+0000","last_clean":"2024-04-25T10:15:08.340779+0000","last_became_active":"2024-04-25T10:15:06.326963+0000","last_became_peered":"2024-04-25T10:15:06.326963+0000","last_unstale":"2024-04-25T10:15:08.340779+0000","last_undegraded":"2024-04-25T10:15:08.340779+0000","last_fullsized":"2024-04-25T10:15:08.340779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:46.626933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035571+0000","last_change":"2024-04-25T10:15:07.342694+0000","last_active":"2024-04-25T10:15:09.035571+0000","last_peered":"2024-04-25T10:15:09.035571+0000","last_clean":"2024-04-25T10:15:09.035571+0000","last_became_active":"2024-04-25T10:15:07.342438+0000","last_became_peered":"2024-04-25T10:15:07.342438+0000","last_unstale":"2024-04-25T10:15:09.035571+0000","last_undegraded":"2024-04-25T10:15:09.035571+0000","last_fullsized":"2024-04-25T10:15:09.035571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:21.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.034961+0000","last_change":"2024-04-25T10:15:06.320401+0000","last_active":"2024-04-25T10:15:09.034961+0000","last_peered":"2024-04-25T10:15:09.034961+0000","last_clean":"2024-04-25T10:15:09.034961+0000","last_became_active":"2024-04-25T10:15:06.319452+0000","last_became_peered":"2024-04-25T10:15:06.319452+0000","last_unstale":"2024-04-25T10:15:09.034961+0000","last_undegraded":"2024-04-25T10:15:09.034961+0000","last_fullsized":"2024-04-25T10:15:09.034961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:34.757200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035112+0000","last_change":"2024-04-25T10:15:07.338344+0000","last_active":"2024-04-25T10:15:09.035112+0000","last_peered":"2024-04-25T10:15:09.035112+0000","last_clean":"2024-04-25T10:15:09.035112+0000","last_became_active":"2024-04-25T10:15:07.338097+0000","last_became_peered":"2024-04-25T10:15:07.338097+0000","last_unstale":"2024-04-25T10:15:09.035112+0000","last_undegraded":"2024-04-25T10:15:09.035112+0000","last_fullsized":"2024-04-25T10:15:09.035112+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:47.448092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035708+0000","last_change":"2024-04-25T10:15:06.331567+0000","last_active":"2024-04-25T10:15:09.035708+0000","last_peered":"2024-04-25T10:15:09.035708+0000","last_clean":"2024-04-25T10:15:09.035708+0000","last_became_active":"2024-04-25T10:15:06.331122+0000","last_became_peered":"2024-04-25T10:15:06.331122+0000","last_unstale":"2024-04-25T10:15:09.035708+0000","last_undegraded":"2024-04-25T10:15:09.035708+0000","last_fullsized":"2024-04-25T10:15:09.035708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:02.462234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341741+0000","last_change":"2024-04-25T10:15:07.338294+0000","last_active":"2024-04-25T10:15:08.341741+0000","last_peered":"2024-04-25T10:15:08.341741+0000","last_clean":"2024-04-25T10:15:08.341741+0000","last_became_active":"2024-04-25T10:15:07.337996+0000","last_became_peered":"2024-04-25T10:15:07.337996+0000","last_unstale":"2024-04-25T10:15:08.341741+0000","last_undegraded":"2024-04-25T10:15:08.341741+0000","last_fullsized":"2024-04-25T10:15:08.341741+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:15.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035815+0000","last_change":"2024-04-25T10:15:06.326879+0000","last_active":"2024-04-25T10:15:09.035815+0000","last_peered":"2024-04-25T10:15:09.035815+0000","last_clean":"2024-04-25T10:15:09.035815+0000","last_became_active":"2024-04-25T10:15:06.326588+0000","last_became_peered":"2024-04-25T10:15:06.326588+0000","last_unstale":"2024-04-25T10:15:09.035815+0000","last_undegraded":"2024-04-25T10:15:09.035815+0000","last_fullsized":"2024-04-25T10:15:09.035815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:45.915265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036282+0000","last_change":"2024-04-25T10:15:07.329275+0000","last_active":"2024-04-25T10:15:09.036282+0000","last_peered":"2024-04-25T10:15:09.036282+0000","last_clean":"2024-04-25T10:15:09.036282+0000","last_became_active":"2024-04-25T10:15:07.328988+0000","last_became_peered":"2024-04-25T10:15:07.328988+0000","last_unstale":"2024-04-25T10:15:09.036282+0000","last_undegraded":"2024-04-25T10:15:09.036282+0000","last_fullsized":"2024-04-25T10:15:09.036282+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:00.235837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036588+0000","last_change":"2024-04-25T10:15:06.327460+0000","last_active":"2024-04-25T10:15:09.036588+0000","last_peered":"2024-04-25T10:15:09.036588+0000","last_clean":"2024-04-25T10:15:09.036588+0000","last_became_active":"2024-04-25T10:15:06.327184+0000","last_became_peered":"2024-04-25T10:15:06.327184+0000","last_unstale":"2024-04-25T10:15:09.036588+0000","last_undegraded":"2024-04-25T10:15:09.036588+0000","last_fullsized":"2024-04-25T10:15:09.036588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:44.482270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036512+0000","last_change":"2024-04-25T10:15:07.344572+0000","last_active":"2024-04-25T10:15:09.036512+0000","last_peered":"2024-04-25T10:15:09.036512+0000","last_clean":"2024-04-25T10:15:09.036512+0000","last_became_active":"2024-04-25T10:15:07.344271+0000","last_became_peered":"2024-04-25T10:15:07.344271+0000","last_unstale":"2024-04-25T10:15:09.036512+0000","last_undegraded":"2024-04-25T10:15:09.036512+0000","last_fullsized":"2024-04-25T10:15:09.036512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:30.509300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.338429+0000","last_change":"2024-04-25T10:15:06.326561+0000","last_active":"2024-04-25T10:15:08.338429+0000","last_peered":"2024-04-25T10:15:08.338429+0000","last_clean":"2024-04-25T10:15:08.338429+0000","last_became_active":"2024-04-25T10:15:06.326274+0000","last_became_peered":"2024-04-25T10:15:06.326274+0000","last_unstale":"2024-04-25T10:15:08.338429+0000","last_undegraded":"2024-04-25T10:15:08.338429+0000","last_fullsized":"2024-04-25T10:15:08.338429+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:37.849716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036220+0000","last_change":"2024-04-25T10:15:07.345757+0000","last_active":"2024-04-25T10:15:09.036220+0000","last_peered":"2024-04-25T10:15:09.036220+0000","last_clean":"2024-04-25T10:15:09.036220+0000","last_became_active":"2024-04-25T10:15:07.341686+0000","last_became_peered":"2024-04-25T10:15:07.341686+0000","last_unstale":"2024-04-25T10:15:09.036220+0000","last_undegraded":"2024-04-25T10:15:09.036220+0000","last_fullsized":"2024-04-25T10:15:09.036220+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:45.036293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036167+0000","last_change":"2024-04-25T10:15:06.328798+0000","last_active":"2024-04-25T10:15:09.036167+0000","last_peered":"2024-04-25T10:15:09.036167+0000","last_clean":"2024-04-25T10:15:09.036167+0000","last_became_active":"2024-04-25T10:15:06.328478+0000","last_became_peered":"2024-04-25T10:15:06.328478+0000","last_unstale":"2024-04-25T10:15:09.036167+0000","last_undegraded":"2024-04-25T10:15:09.036167+0000","last_fullsized":"2024-04-25T10:15:09.036167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:14.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035775+0000","last_change":"2024-04-25T10:15:07.340740+0000","last_active":"2024-04-25T10:15:09.035775+0000","last_peered":"2024-04-25T10:15:09.035775+0000","last_clean":"2024-04-25T10:15:09.035775+0000","last_became_active":"2024-04-25T10:15:07.340518+0000","last_became_peered":"2024-04-25T10:15:07.340518+0000","last_unstale":"2024-04-25T10:15:09.035775+0000","last_undegraded":"2024-04-25T10:15:09.035775+0000","last_fullsized":"2024-04-25T10:15:09.035775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:58.045837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036056+0000","last_change":"2024-04-25T10:15:06.322199+0000","last_active":"2024-04-25T10:15:09.036056+0000","last_peered":"2024-04-25T10:15:09.036056+0000","last_clean":"2024-04-25T10:15:09.036056+0000","last_became_active":"2024-04-25T10:15:06.321866+0000","last_became_peered":"2024-04-25T10:15:06.321866+0000","last_unstale":"2024-04-25T10:15:09.036056+0000","last_undegraded":"2024-04-25T10:15:09.036056+0000","last_fullsized":"2024-04-25T10:15:09.036056+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:43.334337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.340115+0000","last_change":"2024-04-25T10:15:07.348217+0000","last_active":"2024-04-25T10:15:08.340115+0000","last_peered":"2024-04-25T10:15:08.340115+0000","last_clean":"2024-04-25T10:15:08.340115+0000","last_became_active":"2024-04-25T10:15:07.346905+0000","last_became_peered":"2024-04-25T10:15:07.346905+0000","last_unstale":"2024-04-25T10:15:08.340115+0000","last_undegraded":"2024-04-25T10:15:08.340115+0000","last_fullsized":"2024-04-25T10:15:08.340115+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:32.633323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.338478+0000","last_change":"2024-04-25T10:15:06.327009+0000","last_active":"2024-04-25T10:15:08.338478+0000","last_peered":"2024-04-25T10:15:08.338478+0000","last_clean":"2024-04-25T10:15:08.338478+0000","last_became_active":"2024-04-25T10:15:06.326394+0000","last_became_peered":"2024-04-25T10:15:06.326394+0000","last_unstale":"2024-04-25T10:15:08.338478+0000","last_undegraded":"2024-04-25T10:15:08.338478+0000","last_fullsized":"2024-04-25T10:15:08.338478+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:43.334337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036400+0000","last_change":"2024-04-25T10:15:07.331696+0000","last_active":"2024-04-25T10:15:09.036400+0000","last_peered":"2024-04-25T10:15:09.036400+0000","last_clean":"2024-04-25T10:15:09.036400+0000","last_became_active":"2024-04-25T10:15:07.330206+0000","last_became_peered":"2024-04-25T10:15:07.330206+0000","last_unstale":"2024-04-25T10:15:09.036400+0000","last_undegraded":"2024-04-25T10:15:09.036400+0000","last_fullsized":"2024-04-25T10:15:09.036400+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:15.327782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035181+0000","last_change":"2024-04-25T10:15:06.322337+0000","last_active":"2024-04-25T10:15:09.035181+0000","last_peered":"2024-04-25T10:15:09.035181+0000","last_clean":"2024-04-25T10:15:09.035181+0000","last_became_active":"2024-04-25T10:15:06.320077+0000","last_became_peered":"2024-04-25T10:15:06.320077+0000","last_unstale":"2024-04-25T10:15:09.035181+0000","last_undegraded":"2024-04-25T10:15:09.035181+0000","last_fullsized":"2024-04-25T10:15:09.035181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:38.583532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.036738+0000","last_change":"2024-04-25T10:15:07.342309+0000","last_active":"2024-04-25T10:15:09.036738+0000","last_peered":"2024-04-25T10:15:09.036738+0000","last_clean":"2024-04-25T10:15:09.036738+0000","last_became_active":"2024-04-25T10:15:07.342056+0000","last_became_peered":"2024-04-25T10:15:07.342056+0000","last_unstale":"2024-04-25T10:15:09.036738+0000","last_undegraded":"2024-04-25T10:15:09.036738+0000","last_fullsized":"2024-04-25T10:15:09.036738+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:06.298784+0000","last_clean_scrub_stamp":"2024-04-25T10:15:06.298784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:46.920836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:08.341237+0000","last_change":"2024-04-25T10:15:06.319454+0000","last_active":"2024-04-25T10:15:08.341237+0000","last_peered":"2024-04-25T10:15:08.341237+0000","last_clean":"2024-04-25T10:15:08.341237+0000","last_became_active":"2024-04-25T10:15:06.319160+0000","last_became_peered":"2024-04-25T10:15:06.319160+0000","last_unstale":"2024-04-25T10:15:08.341237+0000","last_undegraded":"2024-04-25T10:15:08.341237+0000","last_fullsized":"2024-04-25T10:15:08.341237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:05.293212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:05.293212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:06.120316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:15:09.035787+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:15:09.035787+0000","last_peered":"2024-04-25T10:15:09.035787+0000","last_clean":"2024-04-25T10:15:09.035787+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:15:09.035787+0000","last_undegraded":"2024-04-25T10:15:09.035787+0000","last_fullsized":"2024-04-25T10:15:09.035787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1138688,"data_stored":812615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":548864,"data_stored":222829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":544768,"data_stored":222227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40040,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331800,"statfs":{"total":96636764160,"available":96595763200,"internally_reserved":0,"allocated":552960,"data_stored":223769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40040,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331800,"statfs":{"total":96636764160,"available":96595763200,"internally_reserved":0,"allocated":552960,"data_stored":222795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":223791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40032,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331808,"statfs":{"total":96636764160,"available":96595771392,"internally_reserved":0,"allocated":544768,"data_stored":222227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1138688,"data_stored":812559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:16.289 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:15:16.289 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:15:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3929200640 1 Processor -- start 2024-04-25T10:15:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3929200640 1 -- start start 2024-04-25T10:15:16.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3929200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f392415bbb0 0x7f39240cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3929200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240cf960 0x7f39240cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3929200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392406ca90 con 0x7f39240b4660 2024-04-25T10:15:16.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3929200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392406d050 con 0x7f392415bbb0 2024-04-25T10:15:16.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3929200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392406c830 con 0x7f39240cf960 2024-04-25T10:15:16.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3921e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f392415bbb0 0x7f39240cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.464+0000 7f3921e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f392415bbb0 0x7f39240cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54898/0 (socket says 172.21.15.87:54898) 2024-04-25T10:15:16.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921e00640 1 -- 172.21.15.87:0/2014979510 learned_addr learned my addr 172.21.15.87:0/2014979510 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:16.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3922800640 1 --2- 172.21.15.87:0/2014979510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240cf960 0x7f39240cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921e00640 1 -- 172.21.15.87:0/2014979510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240cf960 msgr2=0x7f39240cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921e00640 1 --2- 172.21.15.87:0/2014979510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240cf960 0x7f39240cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921e00640 1 -- 172.21.15.87:0/2014979510 >> v1:172.21.15.169:6789/0 conn(0x7f39240b4660 legacy=0x7f392406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921e00640 1 -- 172.21.15.87:0/2014979510 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392406cd70 con 0x7f392415bbb0 2024-04-25T10:15:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921e00640 1 --2- 172.21.15.87:0/2014979510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f392415bbb0 0x7f39240cda80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3914009870 tx=0x7f391401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f3ab02f6f40609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921400640 1 -- 172.21.15.87:0/2014979510 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140b2070 con 0x7f392415bbb0 2024-04-25T10:15:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921400640 1 -- 172.21.15.87:0/2014979510 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39140a9070 con 0x7f392415bbb0 2024-04-25T10:15:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.468+0000 7f3921400640 1 -- 172.21.15.87:0/2014979510 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140ad880 con 0x7f392415bbb0 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- 172.21.15.87:0/2014979510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f392415bbb0 msgr2=0x7f39240cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 --2- 172.21.15.87:0/2014979510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f392415bbb0 0x7f39240cda80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3914009870 tx=0x7f391401c560 comp rx=0 tx=0).stop 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- 172.21.15.87:0/2014979510 shutdown_connections 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 --2- 172.21.15.87:0/2014979510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240cf960 0x7f39240cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 --2- 172.21.15.87:0/2014979510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f392415bbb0 0x7f39240cda80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- 172.21.15.87:0/2014979510 >> 172.21.15.87:0/2014979510 conn(0x7f3924066b70 msgr2=0x7f3924066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- 172.21.15.87:0/2014979510 shutdown_connections 2024-04-25T10:15:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- 172.21.15.87:0/2014979510 wait complete. 2024-04-25T10:15:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 Processor -- start 2024-04-25T10:15:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- start start 2024-04-25T10:15:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240b4660 0x7f3924159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39240cf960 0x7f392415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f392415bbb0 0x7f39240aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39240b3a10 con 0x7f39240cf960 2024-04-25T10:15:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39240b3890 con 0x7f392415bbb0 2024-04-25T10:15:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3929200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39240b3b90 con 0x7f39240b4660 2024-04-25T10:15:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39240cf960 0x7f392415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39240cf960 0x7f392415a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54906/0 (socket says 172.21.15.87:54906) 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 -- 172.21.15.87:0/2869700103 learned_addr learned my addr 172.21.15.87:0/2869700103 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3921e00640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240b4660 0x7f3924159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3922800640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f392415bbb0 0x7f39240aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 -- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240b4660 msgr2=0x7f3924159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240b4660 0x7f3924159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 -- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f392415bbb0 msgr2=0x7f39240aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f392415bbb0 0x7f39240aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 -- 172.21.15.87:0/2869700103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392415abe0 con 0x7f39240cf960 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.472+0000 7f3919400640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39240cf960 0x7f392415a6a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f390c00c340 tx=0x7f390c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f391aa00640 1 -- 172.21.15.87:0/2869700103 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0052d0 con 0x7f39240cf960 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f391aa00640 1 -- 172.21.15.87:0/2869700103 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c005470 con 0x7f39240cf960 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f391aa00640 1 -- 172.21.15.87:0/2869700103 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c007260 con 0x7f39240cf960 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392415a320 con 0x7f39240cf960 2024-04-25T10:15:16.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3924158900 con 0x7f39240cf960 2024-04-25T10:15:16.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392404ef30 con 0x7f39240cf960 2024-04-25T10:15:16.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f391aa00640 1 -- 172.21.15.87:0/2869700103 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f390c007400 con 0x7f39240cf960 2024-04-25T10:15:16.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f391aa00640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3900066f60 0x7f3900069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f391aa00640 1 -- 172.21.15.87:0/2869700103 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f390c084430 con 0x7f39240cf960 2024-04-25T10:15:16.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.476+0000 7f3921e00640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3900066f60 0x7f3900069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.480+0000 7f3921e00640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3900066f60 0x7f3900069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f39140af000 tx=0x7f39140a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.480+0000 7f391aa00640 1 -- 172.21.15.87:0/2869700103 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f390c054de0 con 0x7f39240cf960 2024-04-25T10:15:16.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.688+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f392406d980 con 0x7f39240cf960 2024-04-25T10:15:16.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f391aa00640 1 -- 172.21.15.87:0/2869700103 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f390c058e80 con 0x7f39240cf960 2024-04-25T10:15:16.697 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:15:16.697 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:15:16.697 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:15:16.698 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:15:16.698 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:16.698 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:15:16.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3900066f60 msgr2=0x7f3900069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3900066f60 0x7f3900069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f39140af000 tx=0x7f39140a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39240cf960 msgr2=0x7f392415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39240cf960 0x7f392415a6a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f390c00c340 tx=0x7f390c00c800 comp rx=0 tx=0).stop 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 shutdown_connections 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3900066f60 0x7f3900069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f392415bbb0 0x7f39240aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39240cf960 0x7f392415a6a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 --2- 172.21.15.87:0/2869700103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39240b4660 0x7f3924159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.692+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 >> 172.21.15.87:0/2869700103 conn(0x7f3924066b70 msgr2=0x7f3924070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.696+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 shutdown_connections 2024-04-25T10:15:16.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.696+0000 7f3929200640 1 -- 172.21.15.87:0/2869700103 wait complete. 2024-04-25T10:15:16.719 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.848+0000 7f956a400640 1 Processor -- start 2024-04-25T10:15:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.848+0000 7f956a400640 1 -- start start 2024-04-25T10:15:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.848+0000 7f956a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9564066c00 0x7f956415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.848+0000 7f956a400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95640b5970 con 0x7f95640b5560 2024-04-25T10:15:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f956a400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95640b5c20 con 0x7f9564066fe0 2024-04-25T10:15:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f956a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95640af6e0 con 0x7f9564066c00 2024-04-25T10:15:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f95640b5560 0x7f9564157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56246/0 (socket says 172.21.15.87:56246) 2024-04-25T10:15:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 -- 172.21.15.87:0/1592737162 learned_addr learned my addr 172.21.15.87:0/1592737162 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563400640 1 --2- 172.21.15.87:0/1592737162 >> v2:172.21.15.169:3300/0 conn(0x7f9564066c00 0x7f956415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826803490 0 0) 0x7f95640b5970 con 0x7f95640b5560 2024-04-25T10:15:16.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9548003530 con 0x7f95640b5560 2024-04-25T10:15:16.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332613155 0 0) 0x7f95640b5c20 con 0x7f9564066fe0 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95640b5970 con 0x7f9564066fe0 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528973718 0 0) 0x7f9548003530 con 0x7f95640b5560 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 >> v2:172.21.15.169:3300/0 conn(0x7f9564066c00 msgr2=0x7f956415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 --2- 172.21.15.87:0/1592737162 >> v2:172.21.15.169:3300/0 conn(0x7f9564066c00 0x7f956415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 >> v1:172.21.15.169:6789/0 conn(0x7f9564066fe0 legacy=0x7f956406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640af9c0 con 0x7f95640b5560 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f955c001a00 con 0x7f95640b5560 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 >> v1:172.21.15.87:6789/0 conn(0x7f95640b5560 legacy=0x7f9564157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9548003b50 0x7f9548003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9548004810 0x7f9548004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95480051d0 0x7f954800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640af6e0 con 0x7f95480051d0 2024-04-25T10:15:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954800c050 con 0x7f9548003b50 2024-04-25T10:15:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954800c250 con 0x7f9548004810 2024-04-25T10:15:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95480051d0 0x7f954800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 -- 172.21.15.87:0/1592737162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9548004810 msgr2=0x7f9548004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9548004810 0x7f9548004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 -- 172.21.15.87:0/1592737162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9548003b50 msgr2=0x7f9548003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9548003b50 0x7f9548003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.852+0000 7f9563e00640 1 -- 172.21.15.87:0/1592737162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548003530 con 0x7f95480051d0 2024-04-25T10:15:16.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f9563e00640 1 -- 172.21.15.87:0/1592737162 reap_dead start 2024-04-25T10:15:16.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f9563e00640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95480051d0 0x7f954800b9d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f955c00a800 tx=0x7f955c01fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d00f76dc106f87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c00a450 con 0x7f95480051d0 2024-04-25T10:15:16.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c0aa6f0 con 0x7f95480051d0 2024-04-25T10:15:16.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f9562a00640 1 -- 172.21.15.87:0/1592737162 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0aaae0 con 0x7f95480051d0 2024-04-25T10:15:16.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 -- 172.21.15.87:0/1592737162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95480051d0 msgr2=0x7f954800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95480051d0 0x7f954800b9d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f955c00a800 tx=0x7f955c01fa80 comp rx=0 tx=0).stop 2024-04-25T10:15:16.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 -- 172.21.15.87:0/1592737162 shutdown_connections 2024-04-25T10:15:16.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 --2- 172.21.15.87:0/1592737162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95480051d0 0x7f954800b9d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 -- 172.21.15.87:0/1592737162 >> 172.21.15.87:0/1592737162 conn(0x7f95640b24f0 msgr2=0x7f95640b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 -- 172.21.15.87:0/1592737162 shutdown_connections 2024-04-25T10:15:16.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 -- 172.21.15.87:0/1592737162 wait complete. 2024-04-25T10:15:16.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 Processor -- start 2024-04-25T10:15:16.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.856+0000 7f956a400640 1 -- start start 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95480051d0 0x7f95641f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95641483c0 0x7f95641e5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95640c60d0 0x7f95640c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956415e110 con 0x7f95641483c0 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640aff80 con 0x7f95480051d0 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956415e290 con 0x7f95640c60d0 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95641483c0 0x7f95641e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95641483c0 0x7f95641e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54932/0 (socket says 172.21.15.87:54932) 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 -- 172.21.15.87:0/4195758252 learned_addr learned my addr 172.21.15.87:0/4195758252 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f9563e00640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95640c60d0 0x7f95640c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f9563400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95480051d0 0x7f95641f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 -- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95640c60d0 msgr2=0x7f95640c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95640c60d0 0x7f95640c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 -- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95480051d0 msgr2=0x7f95641f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95480051d0 0x7f95641f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 -- 172.21.15.87:0/4195758252 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9564148840 con 0x7f95641483c0 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f9563400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95480051d0 0x7f95641f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955aa00640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95641483c0 0x7f95641e5240 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c500 tx=0x7f954c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955be00640 1 -- 172.21.15.87:0/4195758252 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0052d0 con 0x7f95641483c0 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955be00640 1 -- 172.21.15.87:0/4195758252 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c005470 con 0x7f95641483c0 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955be00640 1 -- 172.21.15.87:0/4195758252 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c012830 con 0x7f95641483c0 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95641e5a90 con 0x7f95641483c0 2024-04-25T10:15:16.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95641e6050 con 0x7f95641483c0 2024-04-25T10:15:16.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956404ef30 con 0x7f95641483c0 2024-04-25T10:15:16.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.860+0000 7f955be00640 1 -- 172.21.15.87:0/4195758252 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f954c0129d0 con 0x7f95641483c0 2024-04-25T10:15:16.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.864+0000 7f955be00640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9540066f90 0x7f9540069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.864+0000 7f955be00640 1 -- 172.21.15.87:0/4195758252 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f954c084dc0 con 0x7f95641483c0 2024-04-25T10:15:16.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.864+0000 7f9563400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9540066f90 0x7f9540069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.864+0000 7f9563400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9540066f90 0x7f9540069450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f95540047c0 tx=0x7f9554008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:16.868+0000 7f955be00640 1 -- 172.21.15.87:0/4195758252 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f954c0182b0 con 0x7f95641483c0 2024-04-25T10:15:17.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9564071c70 con 0x7f95641483c0 2024-04-25T10:15:17.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f955be00640 1 -- 172.21.15.87:0/4195758252 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f954c055d40 con 0x7f95641483c0 2024-04-25T10:15:17.057 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:17.057 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:17.057 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:17.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9540066f90 msgr2=0x7f9540069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9540066f90 0x7f9540069450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f95540047c0 tx=0x7f9554008040 comp rx=0 tx=0).stop 2024-04-25T10:15:17.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95641483c0 msgr2=0x7f95641e5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95641483c0 0x7f95641e5240 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c500 tx=0x7f954c00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 shutdown_connections 2024-04-25T10:15:17.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9540066f90 0x7f9540069450 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95640c60d0 0x7f95640c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95641483c0 0x7f95641e5240 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 --2- 172.21.15.87:0/4195758252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95480051d0 0x7f95641f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 >> 172.21.15.87:0/4195758252 conn(0x7f95640b24f0 msgr2=0x7f956406eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 shutdown_connections 2024-04-25T10:15:17.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.052+0000 7f956a400640 1 -- 172.21.15.87:0/4195758252 wait complete. 2024-04-25T10:15:17.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:15:06.691892+0000', 'modified': '2024-04-25T10:15:09.711913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5631': {'gid': 5631, 'name': 'd', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6834/3236213442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 3236213442}, {'type': 'v1', 'addr': '172.21.15.169:6834', 'nonce': 3236213442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:15:17.482 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:15:17.482 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:15:17.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3761000640 1 Processor -- start 2024-04-25T10:15:17.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3761000640 1 -- start start 2024-04-25T10:15:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3761000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f375c1550b0 0x7f375c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3761000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f375c15b910 0x7f375c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3761000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c071050 con 0x7f375c0cfe70 2024-04-25T10:15:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3761000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c15bd80 con 0x7f375c1550b0 2024-04-25T10:15:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3761000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c15d040 con 0x7f375c15b910 2024-04-25T10:15:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3759e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f375c1550b0 0x7f375c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3759e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f375c1550b0 0x7f375c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37364/0 (socket says 172.21.15.87:37364) 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3759e00640 1 -- 172.21.15.87:0/1660685303 learned_addr learned my addr 172.21.15.87:0/1660685303 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f375a800640 1 --2- 172.21.15.87:0/1660685303 >> v2:172.21.15.169:3300/0 conn(0x7f375c15b910 0x7f375c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3759e00640 1 -- 172.21.15.87:0/1660685303 >> v2:172.21.15.169:3300/0 conn(0x7f375c15b910 msgr2=0x7f375c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3759e00640 1 --2- 172.21.15.87:0/1660685303 >> v2:172.21.15.169:3300/0 conn(0x7f375c15b910 0x7f375c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.616+0000 7f3759e00640 1 -- 172.21.15.87:0/1660685303 >> v1:172.21.15.169:6789/0 conn(0x7f375c0cfe70 legacy=0x7f375c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3759e00640 1 -- 172.21.15.87:0/1660685303 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c0712f0 con 0x7f375c1550b0 2024-04-25T10:15:17.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f375a800640 1 --2- 172.21.15.87:0/1660685303 >> v2:172.21.15.169:3300/0 conn(0x7f375c15b910 0x7f375c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:17.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3759e00640 1 --2- 172.21.15.87:0/1660685303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c0cf930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f374c009e50 tx=0x7f374c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0728229b9b83263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3759400640 1 -- 172.21.15.87:0/1660685303 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0a57a0 con 0x7f375c1550b0 2024-04-25T10:15:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3759400640 1 -- 172.21.15.87:0/1660685303 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c0a5940 con 0x7f375c1550b0 2024-04-25T10:15:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3759400640 1 -- 172.21.15.87:0/1660685303 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0a5d10 con 0x7f375c1550b0 2024-04-25T10:15:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 -- 172.21.15.87:0/1660685303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 msgr2=0x7f375c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 --2- 172.21.15.87:0/1660685303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c0cf930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f374c009e50 tx=0x7f374c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 -- 172.21.15.87:0/1660685303 shutdown_connections 2024-04-25T10:15:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 --2- 172.21.15.87:0/1660685303 >> v2:172.21.15.169:3300/0 conn(0x7f375c15b910 0x7f375c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 --2- 172.21.15.87:0/1660685303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c0cf930 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 -- 172.21.15.87:0/1660685303 >> 172.21.15.87:0/1660685303 conn(0x7f375c066b70 msgr2=0x7f375c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 -- 172.21.15.87:0/1660685303 shutdown_connections 2024-04-25T10:15:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.620+0000 7f3761000640 1 -- 172.21.15.87:0/1660685303 wait complete. 2024-04-25T10:15:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 Processor -- start 2024-04-25T10:15:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 -- start start 2024-04-25T10:15:17.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f375c0cfe70 0x7f375c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f375c15b910 0x7f375c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c15c030 con 0x7f375c1550b0 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0720b0 con 0x7f375c15b910 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c15c1b0 con 0x7f375c0cfe70 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37374/0 (socket says 172.21.15.87:37374) 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 -- 172.21.15.87:0/2370374303 learned_addr learned my addr 172.21.15.87:0/2370374303 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3759e00640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f375c0cfe70 0x7f375c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f375a800640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f375c15b910 0x7f375c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 -- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f375c0cfe70 msgr2=0x7f375c1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f375c0cfe70 0x7f375c1471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 -- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f375c15b910 msgr2=0x7f375c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f375c15b910 0x7f375c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 -- 172.21.15.87:0/2370374303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c14c230 con 0x7f375c1550b0 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f375a800640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f375c15b910 0x7f375c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3751400640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c159790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f374400c450 tx=0x7f374400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3752a00640 1 -- 172.21.15.87:0/2370374303 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744016020 con 0x7f375c1550b0 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3752a00640 1 -- 172.21.15.87:0/2370374303 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37440052d0 con 0x7f375c1550b0 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c14c510 con 0x7f375c1550b0 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3752a00640 1 -- 172.21.15.87:0/2370374303 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744007260 con 0x7f375c1550b0 2024-04-25T10:15:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.624+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c14cdb0 con 0x7f375c1550b0 2024-04-25T10:15:17.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.628+0000 7f3752a00640 1 -- 172.21.15.87:0/2370374303 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3744007400 con 0x7f375c1550b0 2024-04-25T10:15:17.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.628+0000 7f3752a00640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3738066f60 0x7f3738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.628+0000 7f3752a00640 1 -- 172.21.15.87:0/2370374303 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3744084930 con 0x7f375c1550b0 2024-04-25T10:15:17.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.628+0000 7f3759e00640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3738066f60 0x7f3738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.628+0000 7f3764541000 1 Processor -- start 2024-04-25T10:15:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.628+0000 7f3764541000 1 -- start start 2024-04-25T10:15:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.628+0000 7f3759e00640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3738066f60 0x7f3738069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f374c0ae000 tx=0x7f374c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b3c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b3779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b378310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6b25f5f0 con 0x562d6b377f30 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6b1c6da0 con 0x562d6b3c2240 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6b14f150 con 0x562d6b3c1920 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b378310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b378310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37386/0 (socket says 172.21.15.87:37386) 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 -- 172.21.15.87:0/4188125321 learned_addr learned my addr 172.21.15.87:0/4188125321 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3759e00640 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b3c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3751400640 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b3779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 -- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 msgr2=0x562d6b3c1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b3c1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 -- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 msgr2=0x562d6b3779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b3779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 -- 172.21.15.87:0/4188125321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6b168ca0 con 0x562d6b377f30 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f375a800640 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b378310 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f375400b8e0 tx=0x7f375400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff83235bf72953e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f374aa00640 1 -- 172.21.15.87:0/4188125321 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754018020 con 0x562d6b377f30 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f374aa00640 1 -- 172.21.15.87:0/4188125321 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375400fa80 con 0x562d6b377f30 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f374aa00640 1 -- 172.21.15.87:0/4188125321 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375400fe30 con 0x562d6b377f30 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- 172.21.15.87:0/4188125321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 msgr2=0x562d6b378310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b378310 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f375400b8e0 tx=0x7f375400bda0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- 172.21.15.87:0/4188125321 shutdown_connections 2024-04-25T10:15:17.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b378310 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b3779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 --2- 172.21.15.87:0/4188125321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b3c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- 172.21.15.87:0/4188125321 >> 172.21.15.87:0/4188125321 conn(0x562d6b37e2a0 msgr2=0x562d6b3762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- 172.21.15.87:0/4188125321 shutdown_connections 2024-04-25T10:15:17.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- 172.21.15.87:0/4188125321 wait complete. 2024-04-25T10:15:17.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 -- start start 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.632+0000 7f3764541000 1 Processor -- start 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b3c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b3c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b37baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6b181990 con 0x562d6b3c1920 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6b1c2e40 con 0x562d6b3c2240 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6b152900 con 0x562d6b377f30 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 10 client.0 _ll_register_callbacks cb 0x562d6b27a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b3c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b3c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37392/0 (socket says 172.21.15.87:37392) 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 -- 172.21.15.87:0/2648496154 learned_addr learned my addr 172.21.15.87:0/2648496154 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3759e00640 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b3c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f375a800640 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b37baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 -- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 msgr2=0x562d6b3c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b3c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 -- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 msgr2=0x562d6b37baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b37baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 -- 172.21.15.87:0/2648496154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d6b17f630 con 0x562d6b3c1920 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3751400640 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b3c9230 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f374400c420 tx=0x7f37440055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd68cb46a067087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f373f400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f373f400640 1 -- 172.21.15.87:0/2648496154 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374405d940 con 0x562d6b3c1920 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f373f400640 1 -- 172.21.15.87:0/2648496154 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3744082ed0 con 0x562d6b3c1920 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f375400fa80 con 0x562d6b3c1920 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f373f400640 1 -- 172.21.15.87:0/2648496154 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3744073050 con 0x562d6b3c1920 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 10 client.5802 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:17.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562d6b14f3a0 con 0x562d6b3c1920 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f373f400640 1 -- 172.21.15.87:0/2648496154 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375400fa80 con 0x562d6b3c1920 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f373f400640 1 -- 172.21.15.87:0/2648496154 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f37440518a0 con 0x562d6b3c1920 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 10 client.5802 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 10 client.5802 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 20 client.5802 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b37c4b0 0x562d6b37c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 4 client.5802 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d6b162a50 con 0x562d6b37c4b0 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3759e00640 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b37c4b0 0x562d6b37c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f3759e00640 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b37c4b0 0x562d6b37c890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac03f42fbf1ba1dd server_cookie=60cb26446f4f872f in_seq=0 out_seq=0 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.636+0000 7f373f400640 10 client.5802 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f373f400640 1 -- 172.21.15.87:0/2648496154 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d6b162a50 con 0x562d6b37c4b0 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f373f400640 10 client.5802 handle_command_reply: tid=0 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 client.5802 shutdown 2024-04-25T10:15:17.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 msgr2=0x562d6b3c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b3c9230 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f374400c420 tx=0x7f37440055d0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 shutdown_connections 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b37baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b3c9230 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b37c4b0 0x562d6b37c890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 --2- 172.21.15.87:0/2648496154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b3c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f373f400640 0 client.5802 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 >> 172.21.15.87:0/2648496154 conn(0x562d6b37e2a0 msgr2=0x562d6b3769e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 shutdown_connections 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 1 -- 172.21.15.87:0/2648496154 wait complete. 2024-04-25T10:15:17.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.640+0000 7f3764541000 20 client.5802 trim_cache size 0 max 16384 2024-04-25T10:15:17.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 Processor -- start 2024-04-25T10:15:17.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- start start 2024-04-25T10:15:17.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b425850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b425d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b4262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c2023f0 con 0x562d6b3c1920 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c14dbe0 con 0x562d6b3c2240 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c14e1e0 con 0x562d6b377f30 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b425d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b425d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37408/0 (socket says 172.21.15.87:37408) 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 -- 172.21.15.87:0/519356923 learned_addr learned my addr 172.21.15.87:0/519356923 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3759e00640 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b425850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f375a800640 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b4262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 -- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 msgr2=0x562d6b425850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b425850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 -- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 msgr2=0x562d6b4262d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b4262d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 -- 172.21.15.87:0/519356923 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6b3c2070 con 0x562d6b3c1920 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3751400640 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b425d90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3744000c10 tx=0x7f374406c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a444cca0b0bfa2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f374a000640 1 -- 172.21.15.87:0/519356923 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440590e0 con 0x562d6b3c1920 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f374a000640 1 -- 172.21.15.87:0/519356923 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744081070 con 0x562d6b3c1920 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f374a000640 1 -- 172.21.15.87:0/519356923 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440518a0 con 0x562d6b3c1920 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- 172.21.15.87:0/519356923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 msgr2=0x562d6b425d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b425d90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3744000c10 tx=0x7f374406c000 comp rx=0 tx=0).stop 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- 172.21.15.87:0/519356923 shutdown_connections 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b4262d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b3c1920 0x562d6b425d90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 --2- 172.21.15.87:0/519356923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b377f30 0x562d6b425850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- 172.21.15.87:0/519356923 >> 172.21.15.87:0/519356923 conn(0x562d6b37e2a0 msgr2=0x562d6b37b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- 172.21.15.87:0/519356923 shutdown_connections 2024-04-25T10:15:17.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- 172.21.15.87:0/519356923 wait complete. 2024-04-25T10:15:17.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 -- start start 2024-04-25T10:15:17.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.656+0000 7f3764541000 1 Processor -- start 2024-04-25T10:15:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b3ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b426980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b426ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6b1cb3d0 con 0x562d6b377f30 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6b282d50 con 0x562d6b3c2240 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6b161980 con 0x562d6b3c1920 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 10 client.0 _ll_register_callbacks cb 0x7f375400b300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b3ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b3ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37410/0 (socket says 172.21.15.87:37410) 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 -- 172.21.15.87:0/1200790291 learned_addr learned my addr 172.21.15.87:0/1200790291 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f375a800640 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b426ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3751400640 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b426980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 -- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 msgr2=0x562d6b426980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b426980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 -- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 msgr2=0x562d6b426ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b426ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 -- 172.21.15.87:0/1200790291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d6b248600 con 0x562d6b377f30 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3759e00640 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b3ce5f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f374c000f90 tx=0x7f374c01b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553c73cc92bc7ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 1 -- 172.21.15.87:0/1200790291 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0d7430 con 0x562d6b377f30 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 1 -- 172.21.15.87:0/1200790291 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f374c0d75d0 con 0x562d6b377f30 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3744081070 con 0x562d6b377f30 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 1 -- 172.21.15.87:0/1200790291 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f374c0de050 con 0x562d6b377f30 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 10 client.5808 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562d6b15b9b0 con 0x562d6b377f30 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 1 -- 172.21.15.87:0/1200790291 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744081070 con 0x562d6b377f30 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 1 -- 172.21.15.87:0/1200790291 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f374c0e1020 con 0x562d6b377f30 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 10 client.5808 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 10 client.5808 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 20 client.5808 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:17.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b427400 0x562d6b4277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 4 client.5808 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d6b1d42e0 con 0x562d6b427400 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3751400640 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b427400 0x562d6b4277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3751400640 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b427400 0x562d6b4277e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f643c0ffdcb47c server_cookie=5f73919abaebd050 in_seq=0 out_seq=0 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 10 client.5808 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 1 -- 172.21.15.87:0/1200790291 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562d6b1d42e0 con 0x562d6b427400 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 10 client.5808 handle_command_reply: tid=0 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 client.5808 shutdown 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 msgr2=0x562d6b3ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b3ce5f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f374c000f90 tx=0x7f374c01b230 comp rx=0 tx=0).stop 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 shutdown_connections 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d6b3c2240 0x562d6b426ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562d6b427400 0x562d6b4277e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d6b3c1920 0x562d6b426980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 --2- 172.21.15.87:0/1200790291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562d6b377f30 0x562d6b3ce5f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f373f400640 0 client.5808 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 >> 172.21.15.87:0/1200790291 conn(0x562d6b37e2a0 msgr2=0x562d6b3c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 shutdown_connections 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 1 -- 172.21.15.87:0/1200790291 wait complete. 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.660+0000 7f3764541000 20 client.5808 trim_cache size 0 max 16384 2024-04-25T10:15:17.672 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5631, 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 10.336016649999999, 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 46, 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 46, 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 15.206448317 2024-04-25T10:15:17.673 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3738066f60 msgr2=0x7f3738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3738066f60 0x7f3738069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f374c0ae000 tx=0x7f374c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 msgr2=0x7f375c159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c159790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f374400c450 tx=0x7f374400c910 comp rx=0 tx=0).stop 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 shutdown_connections 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3738066f60 0x7f3738069420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f375c15b910 0x7f375c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f375c1550b0 0x7f375c159790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 --2- 172.21.15.87:0/2370374303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f375c0cfe70 0x7f375c1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 >> 172.21.15.87:0/2370374303 conn(0x7f375c066b70 msgr2=0x7f375c15abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 shutdown_connections 2024-04-25T10:15:17.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.664+0000 7f3761000640 1 -- 172.21.15.87:0/2370374303 wait complete. 2024-04-25T10:15:17.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:15:17.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.816+0000 7f858b800640 1 Processor -- start 2024-04-25T10:15:17.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.816+0000 7f858b800640 1 -- start start 2024-04-25T10:15:17.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f858b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85841550b0 0x7f85840cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f858b800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f85840cfe70 0x7f858415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f858b800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f858415b740 0x7f858406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f858b800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8584071050 con 0x7f858415b740 2024-04-25T10:15:17.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f858b800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85840712f0 con 0x7f85840cfe70 2024-04-25T10:15:17.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f858b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85840715d0 con 0x7f85841550b0 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f858415b740 0x7f858406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f858415b740 0x7f858406d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37426/0 (socket says 172.21.15.87:37426) 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 -- 172.21.15.87:0/3383361185 learned_addr learned my addr 172.21.15.87:0/3383361185 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8583e00640 1 --2- 172.21.15.87:0/3383361185 >> v2:172.21.15.169:3300/0 conn(0x7f85840cfe70 0x7f858415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 -- 172.21.15.87:0/3383361185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85841550b0 msgr2=0x7f85840cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8588e00640 1 --2- 172.21.15.87:0/3383361185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85841550b0 0x7f85840cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 --2- 172.21.15.87:0/3383361185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85841550b0 0x7f85840cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 -- 172.21.15.87:0/3383361185 >> v2:172.21.15.169:3300/0 conn(0x7f85840cfe70 msgr2=0x7f858415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 --2- 172.21.15.87:0/3383361185 >> v2:172.21.15.169:3300/0 conn(0x7f85840cfe70 0x7f858415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8589800640 1 -- 172.21.15.87:0/3383361185 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85840718b0 con 0x7f858415b740 2024-04-25T10:15:17.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8583e00640 1 --2- 172.21.15.87:0/3383361185 >> v2:172.21.15.169:3300/0 conn(0x7f85840cfe70 0x7f858415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:17.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.820+0000 7f8588e00640 1 --2- 172.21.15.87:0/3383361185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85841550b0 0x7f85840cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f8589800640 1 --2- 172.21.15.87:0/3383361185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f858415b740 0x7f858406d1a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f858406e290 tx=0x7f857c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db05b8e63d48d748 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f8583400640 1 -- 172.21.15.87:0/3383361185 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0b0070 con 0x7f858415b740 2024-04-25T10:15:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f8583400640 1 -- 172.21.15.87:0/3383361185 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c0ab600 con 0x7f858415b740 2024-04-25T10:15:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f8583400640 1 -- 172.21.15.87:0/3383361185 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0ab9d0 con 0x7f858415b740 2024-04-25T10:15:17.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 -- 172.21.15.87:0/3383361185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f858415b740 msgr2=0x7f858406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 --2- 172.21.15.87:0/3383361185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f858415b740 0x7f858406d1a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f858406e290 tx=0x7f857c01c260 comp rx=0 tx=0).stop 2024-04-25T10:15:17.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 -- 172.21.15.87:0/3383361185 shutdown_connections 2024-04-25T10:15:17.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 --2- 172.21.15.87:0/3383361185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f858415b740 0x7f858406d1a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 --2- 172.21.15.87:0/3383361185 >> v2:172.21.15.169:3300/0 conn(0x7f85840cfe70 0x7f858415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 --2- 172.21.15.87:0/3383361185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85841550b0 0x7f85840cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 -- 172.21.15.87:0/3383361185 >> 172.21.15.87:0/3383361185 conn(0x7f8584066b70 msgr2=0x7f8584066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 -- 172.21.15.87:0/3383361185 shutdown_connections 2024-04-25T10:15:17.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 -- 172.21.15.87:0/3383361185 wait complete. 2024-04-25T10:15:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 Processor -- start 2024-04-25T10:15:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.824+0000 7f858b800640 1 -- start start 2024-04-25T10:15:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85840cfe70 0x7f85841fa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85841550b0 0x7f85840afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858415b740 0x7f858413e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85840b33e0 con 0x7f85841550b0 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85840b3260 con 0x7f858415b740 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85840b3560 con 0x7f85840cfe70 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85841550b0 0x7f85840afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85841550b0 0x7f85840afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37440/0 (socket says 172.21.15.87:37440) 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 -- 172.21.15.87:0/1521502106 learned_addr learned my addr 172.21.15.87:0/1521502106 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8589800640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858415b740 0x7f858413e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8588e00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85840cfe70 0x7f85841fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 -- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85840cfe70 msgr2=0x7f85841fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85840cfe70 0x7f85841fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 -- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858415b740 msgr2=0x7f858413e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858415b740 0x7f858413e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 -- 172.21.15.87:0/1521502106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858414cbb0 con 0x7f85841550b0 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8588e00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85840cfe70 0x7f85841fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8583e00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85841550b0 0x7f85840afba0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f857000c7c0 tx=0x7f857000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8580c00640 1 -- 172.21.15.87:0/1521502106 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570007a30 con 0x7f85841550b0 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8580c00640 1 -- 172.21.15.87:0/1521502106 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8570007bd0 con 0x7f85841550b0 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8580c00640 1 -- 172.21.15.87:0/1521502106 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85700113f0 con 0x7f85841550b0 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858414ce90 con 0x7f85841550b0 2024-04-25T10:15:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858413c680 con 0x7f85841550b0 2024-04-25T10:15:17.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858404ef30 con 0x7f85841550b0 2024-04-25T10:15:17.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.828+0000 7f8580c00640 1 -- 172.21.15.87:0/1521502106 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8570011590 con 0x7f85841550b0 2024-04-25T10:15:17.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.832+0000 7f8580c00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8564066f60 0x7f8564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.832+0000 7f8580c00640 1 -- 172.21.15.87:0/1521502106 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8570085220 con 0x7f85841550b0 2024-04-25T10:15:17.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.832+0000 7f8588e00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8564066f60 0x7f8564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.832+0000 7f8588e00640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8564066f60 0x7f8564069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f85740075b0 tx=0x7f8574007540 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:17.836+0000 7f8580c00640 1 -- 172.21.15.87:0/1521502106 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8570015020 con 0x7f85841550b0 2024-04-25T10:15:18.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8584151c00 con 0x7f85841550b0 2024-04-25T10:15:18.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f8580c00640 1 -- 172.21.15.87:0/1521502106 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8570055b40 con 0x7f85841550b0 2024-04-25T10:15:18.026 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:15:18.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8564066f60 msgr2=0x7f8564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8564066f60 0x7f8564069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f85740075b0 tx=0x7f8574007540 comp rx=0 tx=0).stop 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85841550b0 msgr2=0x7f85840afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85841550b0 0x7f85840afba0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f857000c7c0 tx=0x7f857000cc80 comp rx=0 tx=0).stop 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 shutdown_connections 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8564066f60 0x7f8564069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858415b740 0x7f858413e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85841550b0 0x7f85840afba0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 --2- 172.21.15.87:0/1521502106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85840cfe70 0x7f85841fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 >> 172.21.15.87:0/1521502106 conn(0x7f8584066b70 msgr2=0x7f85840b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 shutdown_connections 2024-04-25T10:15:18.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18.020+0000 7f858b800640 1 -- 172.21.15.87:0/1521502106 wait complete. 2024-04-25T10:15:18.046 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:15:18.046 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:15:18.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:15:18.047 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:15:18.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:18.047 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:15:18.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:15:18.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:15:18.047 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:18.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.047 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:15:18.056 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:15:18.056 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:15:18.056 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:15:18.056 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:15:18.057 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42296sec preferred_lft 42296sec 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:18.058 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:18.059 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:15:18.059 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.059 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:18.059 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:18.059 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:15:18.059 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.060 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:15:18.124 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:15:18.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.126 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:15:18.179 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:15:18.180 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:15:18.180 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:15:18.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.180 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:15:18.180 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:15:18.180 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:15:18.180 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:15:18.180 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:15:18.180 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:15:18.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.271 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:15:18.327 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:15:18.327 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:15:18.327 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:18.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.328 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:15:18.390 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:18.390 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:18.443 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:18.443 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:18.443 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:18.494 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:15:18.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.494 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:18.583 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:18.583 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:18.583 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:15:18.598 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:18.598 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:18.598 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:15:18.598 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:15:18.598 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:15:18.612 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:15:18.616 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.616 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:18.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.683 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:15:18.744 INFO:teuthology.orchestra.run.smithi087.stdout:client5512 2024-04-25T10:15:18.745 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:18.745 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:18.799 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:18.799 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:15:18.799 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:15:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.800 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> import stat 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> import json 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> import sys 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:15:18.860 DEBUG:teuthology.orchestra.run.smithi087:> except OSError as e: 2024-04-25T10:15:18.861 DEBUG:teuthology.orchestra.run.smithi087:> sys.exit(e.errno) 2024-04-25T10:15:18.861 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:15:18.861 DEBUG:teuthology.orchestra.run.smithi087:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:15:18.861 DEBUG:teuthology.orchestra.run.smithi087:> print(json.dumps( 2024-04-25T10:15:18.861 DEBUG:teuthology.orchestra.run.smithi087:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:15:18.861 DEBUG:teuthology.orchestra.run.smithi087:> indent=2)) 2024-04-25T10:15:18.861 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:18 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:15:18.984 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:15:18.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.985 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T10:15:19.039 INFO:teuthology.orchestra.run.smithi087.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T10:15:19.256 INFO:teuthology.orchestra.run.smithi087.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T10:15:21.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:21.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:22.703 INFO:teuthology.orchestra.run.smithi087.stderr:Updating files: 51% (719/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 82% (1149/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T10:15:22.712 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T10:15:22.712 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:15:22.712 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:22.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe127200640 1 Processor -- start 2024-04-25T10:15:22.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe127200640 1 -- start start 2024-04-25T10:15:22.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe127200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 0x7fe12006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe127200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1200b2770 con 0x7fe12015bc40 2024-04-25T10:15:22.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe127200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12006c670 con 0x7fe1200cda10 2024-04-25T10:15:22.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe127200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12006c950 con 0x7fe1200d0040 2024-04-25T10:15:22.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe11fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe12015bc40 0x7fe1200b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44088/0 (socket says 172.21.15.87:44088) 2024-04-25T10:15:22.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.887+0000 7fe11fe00640 1 -- 172.21.15.87:0/636119404 learned_addr learned my addr 172.21.15.87:0/636119404 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:22.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe125200640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 0x7fe12006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052012037 0 0) 0x7fe1200b2770 con 0x7fe12015bc40 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe104003530 con 0x7fe12015bc40 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832080397 0 0) 0x7fe12006c670 con 0x7fe1200cda10 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1200b2770 con 0x7fe1200cda10 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898420981 0 0) 0x7fe104003530 con 0x7fe12015bc40 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 msgr2=0x7fe12006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 0x7fe12006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 >> v1:172.21.15.169:6789/0 conn(0x7fe1200cda10 legacy=0x7fe1200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12006cc30 con 0x7fe12015bc40 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe110001680 con 0x7fe12015bc40 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 >> v1:172.21.15.87:6789/0 conn(0x7fe12015bc40 legacy=0x7fe1200b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe104003b80 0x7fe104003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 0x7fe104004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe104004d50 0x7fe1040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12006c950 con 0x7fe104003b80 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10400bb00 con 0x7fe104004d50 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10400bd00 con 0x7fe1200d0040 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11fe00640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe104003b80 0x7fe104003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe117400640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 0x7fe104004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe125200640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe104004d50 0x7fe1040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11fe00640 1 -- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 msgr2=0x7fe104004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11fe00640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200d0040 0x7fe104004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11fe00640 1 -- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe104004d50 msgr2=0x7fe1040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11fe00640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe104004d50 0x7fe1040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.891+0000 7fe11fe00640 1 -- 172.21.15.87:0/636119404 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104003530 con 0x7fe104003b80 2024-04-25T10:15:22.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe11fe00640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe104003b80 0x7fe104003f80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe11000c190 tx=0x7fe11001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897476dc547f4821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:22.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110007cb0 con 0x7fe104003b80 2024-04-25T10:15:22.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1100a9020 con 0x7fe104003b80 2024-04-25T10:15:22.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe11f400640 1 -- 172.21.15.87:0/636119404 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110005b00 con 0x7fe104003b80 2024-04-25T10:15:22.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- 172.21.15.87:0/636119404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe104003b80 msgr2=0x7fe104003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 --2- 172.21.15.87:0/636119404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe104003b80 0x7fe104003f80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe11000c190 tx=0x7fe11001cc20 comp rx=0 tx=0).stop 2024-04-25T10:15:22.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe125200640 1 -- 172.21.15.87:0/636119404 reap_dead start 2024-04-25T10:15:22.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- 172.21.15.87:0/636119404 shutdown_connections 2024-04-25T10:15:22.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- 172.21.15.87:0/636119404 >> 172.21.15.87:0/636119404 conn(0x7fe120066b70 msgr2=0x7fe120066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:22.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- 172.21.15.87:0/636119404 shutdown_connections 2024-04-25T10:15:22.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- 172.21.15.87:0/636119404 wait complete. 2024-04-25T10:15:22.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 Processor -- start 2024-04-25T10:15:22.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- start start 2024-04-25T10:15:22.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe104003b80 0x7fe1201544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120155bc0 0x7fe120144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120157360 0x7fe1200b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120167560 con 0x7fe120155bc0 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1201673e0 con 0x7fe120157360 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.895+0000 7fe127200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1201676e0 con 0x7fe104003b80 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120155bc0 0x7fe120144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120155bc0 0x7fe120144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37464/0 (socket says 172.21.15.87:37464) 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 -- 172.21.15.87:0/839802089 learned_addr learned my addr 172.21.15.87:0/839802089 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe11fe00640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe104003b80 0x7fe1201544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe125200640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120157360 0x7fe1200b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 -- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe104003b80 msgr2=0x7fe1201544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe104003b80 0x7fe1201544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 -- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120157360 msgr2=0x7fe1200b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120157360 0x7fe1200b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 -- 172.21.15.87:0/839802089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe120156040 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe117400640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120155bc0 0x7fe120144160 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe10800f7d0 tx=0x7fe10800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe11cc00640 1 -- 172.21.15.87:0/839802089 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800fe80 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe11cc00640 1 -- 172.21.15.87:0/839802089 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108018070 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe11cc00640 1 -- 172.21.15.87:0/839802089 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800d730 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1201446a0 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1201565a0 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12004ef30 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe11cc00640 1 -- 172.21.15.87:0/839802089 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe10800d8d0 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe11cc00640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f90 0x7fe0fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.899+0000 7fe11cc00640 1 -- 172.21.15.87:0/839802089 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe108084de0 con 0x7fe120155bc0 2024-04-25T10:15:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.903+0000 7fe11fe00640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f90 0x7fe0fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.903+0000 7fe11fe00640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f90 0x7fe0fc069450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe1100aa000 tx=0x7fe1100b1040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:22.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:22.903+0000 7fe11cc00640 1 -- 172.21.15.87:0/839802089 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe120156040 con 0x7fe120155bc0 2024-04-25T10:15:23.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1200af1c0 con 0x7fe120155bc0 2024-04-25T10:15:23.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe11cc00640 1 -- 172.21.15.87:0/839802089 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fe108055790 con 0x7fe120155bc0 2024-04-25T10:15:23.098 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:23.098 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:23.098 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f90 msgr2=0x7fe0fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f90 0x7fe0fc069450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe1100aa000 tx=0x7fe1100b1040 comp rx=0 tx=0).stop 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120155bc0 msgr2=0x7fe120144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120155bc0 0x7fe120144160 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe10800f7d0 tx=0x7fe10800fc90 comp rx=0 tx=0).stop 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 shutdown_connections 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f90 0x7fe0fc069450 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120157360 0x7fe1200b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120155bc0 0x7fe120144160 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 --2- 172.21.15.87:0/839802089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe104003b80 0x7fe1201544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 >> 172.21.15.87:0/839802089 conn(0x7fe120066b70 msgr2=0x7fe1200ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.091+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 shutdown_connections 2024-04-25T10:15:23.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.095+0000 7fe127200640 1 -- 172.21.15.87:0/839802089 wait complete. 2024-04-25T10:15:23.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:15:23.257 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:23.258 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": -2 2024-04-25T10:15:23.258 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:23.274 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:15:23.274 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:15:23.274 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:23.274 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:15:23.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.411+0000 7f6551000640 1 Processor -- start 2024-04-25T10:15:23.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.411+0000 7f6551000640 1 -- start start 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.411+0000 7f6551000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f654c0aef80 0x7f654c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.411+0000 7f6551000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f654c0b31c0 0x7f654c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.411+0000 7f6551000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654c15b080 con 0x7f654c070350 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.411+0000 7f6551000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654c15adb0 con 0x7f654c0b31c0 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.411+0000 7f6551000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654c15eb60 con 0x7f654c0aef80 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f654c0b31c0 0x7f654c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f654c0b31c0 0x7f654c0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37474/0 (socket says 172.21.15.87:37474) 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 -- 172.21.15.87:0/3286243459 learned_addr learned my addr 172.21.15.87:0/3286243459 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f6549e00640 1 --2- 172.21.15.87:0/3286243459 >> v2:172.21.15.169:3300/0 conn(0x7f654c0aef80 0x7f654c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 -- 172.21.15.87:0/3286243459 >> v2:172.21.15.169:3300/0 conn(0x7f654c0aef80 msgr2=0x7f654c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 --2- 172.21.15.87:0/3286243459 >> v2:172.21.15.169:3300/0 conn(0x7f654c0aef80 0x7f654c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 -- 172.21.15.87:0/3286243459 >> v1:172.21.15.169:6789/0 conn(0x7f654c070350 legacy=0x7f654c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 -- 172.21.15.87:0/3286243459 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c15ee40 con 0x7f654c0b31c0 2024-04-25T10:15:23.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f654a800640 1 --2- 172.21.15.87:0/3286243459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c0b3610 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f654c06e220 tx=0x7f654401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19454df044eaf498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f6549400640 1 -- 172.21.15.87:0/3286243459 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440a57d0 con 0x7f654c0b31c0 2024-04-25T10:15:23.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f6549400640 1 -- 172.21.15.87:0/3286243459 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65440a5970 con 0x7f654c0b31c0 2024-04-25T10:15:23.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.415+0000 7f6549400640 1 -- 172.21.15.87:0/3286243459 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440a5d40 con 0x7f654c0b31c0 2024-04-25T10:15:23.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- 172.21.15.87:0/3286243459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 msgr2=0x7f654c0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 --2- 172.21.15.87:0/3286243459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c0b3610 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f654c06e220 tx=0x7f654401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- 172.21.15.87:0/3286243459 shutdown_connections 2024-04-25T10:15:23.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 --2- 172.21.15.87:0/3286243459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c0b3610 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 --2- 172.21.15.87:0/3286243459 >> v2:172.21.15.169:3300/0 conn(0x7f654c0aef80 0x7f654c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- 172.21.15.87:0/3286243459 >> 172.21.15.87:0/3286243459 conn(0x7f654c0669a0 msgr2=0x7f654c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- 172.21.15.87:0/3286243459 shutdown_connections 2024-04-25T10:15:23.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- 172.21.15.87:0/3286243459 wait complete. 2024-04-25T10:15:23.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 Processor -- start 2024-04-25T10:15:23.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- start start 2024-04-25T10:15:23.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f654c070350 0x7f654c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f654c0aef80 0x7f654c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c1679a0 con 0x7f654c0b31c0 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c167820 con 0x7f654c070350 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6551000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c167b20 con 0x7f654c0aef80 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c1412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37476/0 (socket says 172.21.15.87:37476) 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 -- 172.21.15.87:0/2347118550 learned_addr learned my addr 172.21.15.87:0/2347118550 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6549e00640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f654c070350 0x7f654c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6541400640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f654c0aef80 0x7f654c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 -- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f654c0aef80 msgr2=0x7f654c140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f654c0aef80 0x7f654c140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 -- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f654c070350 msgr2=0x7f654c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f654c070350 0x7f654c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f654a800640 1 -- 172.21.15.87:0/2347118550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c0b0c30 con 0x7f654c0b31c0 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.419+0000 7f6541400640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f654c0aef80 0x7f654c140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f654a800640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c1412e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f65440af040 tx=0x7f6544019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6542a00640 1 -- 172.21.15.87:0/2347118550 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440bc020 con 0x7f654c0b31c0 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6542a00640 1 -- 172.21.15.87:0/2347118550 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65440b8670 con 0x7f654c0b31c0 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c0b0390 con 0x7f654c0b31c0 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6542a00640 1 -- 172.21.15.87:0/2347118550 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544009510 con 0x7f654c0b31c0 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c0b0950 con 0x7f654c0b31c0 2024-04-25T10:15:23.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6542a00640 1 -- 172.21.15.87:0/2347118550 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65440bf020 con 0x7f654c0b31c0 2024-04-25T10:15:23.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6542a00640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6528066f60 0x7f6528069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6542a00640 1 -- 172.21.15.87:0/2347118550 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f654411fe80 con 0x7f654c0b31c0 2024-04-25T10:15:23.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6549e00640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6528066f60 0x7f6528069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f65543b4000 1 Processor -- start 2024-04-25T10:15:23.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f65543b4000 1 -- start start 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.423+0000 7f6549e00640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6528066f60 0x7f6528069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f653c009b60 tx=0x7f653c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be7970 0x55bd62be7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62b9d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62b9dcd0 0x55bd62b9e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd629b38a0 con 0x55bd62be8290 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd629ebd10 con 0x55bd62b9dcd0 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd629742d0 con 0x55bd62be7970 2024-04-25T10:15:23.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6541400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62b9d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be7970 0x55bd62be7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be7970 0x55bd62be7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:55090/0 (socket says 172.21.15.87:55090) 2024-04-25T10:15:23.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 -- 172.21.15.87:0/3483043613 learned_addr learned my addr 172.21.15.87:0/3483043613 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:23.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6541400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62b9d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37484/0 (socket says 172.21.15.87:37484) 2024-04-25T10:15:23.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f654a800640 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62b9dcd0 0x55bd62b9e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 -- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62b9dcd0 msgr2=0x55bd62b9e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62b9dcd0 0x55bd62b9e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 -- 172.21.15.87:0/3483043613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 msgr2=0x55bd62b9d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62b9d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 -- 172.21.15.87:0/3483043613 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd62a32870 con 0x55bd62be7970 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f6549e00640 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be7970 0x55bd62be7d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f653c011ec0 tx=0x7f653c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c79ac939e7d14a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f653aa00640 1 -- 172.21.15.87:0/3483043613 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c01b070 con 0x55bd62be7970 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f653aa00640 1 -- 172.21.15.87:0/3483043613 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c027070 con 0x55bd62be7970 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f653aa00640 1 -- 172.21.15.87:0/3483043613 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c003a30 con 0x55bd62be7970 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be7970 msgr2=0x55bd62be7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be7970 0x55bd62be7d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f653c011ec0 tx=0x7f653c014000 comp rx=0 tx=0).stop 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- 172.21.15.87:0/3483043613 shutdown_connections 2024-04-25T10:15:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62b9dcd0 0x55bd62b9e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62b9d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 --2- 172.21.15.87:0/3483043613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be7970 0x55bd62be7d50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- 172.21.15.87:0/3483043613 >> 172.21.15.87:0/3483043613 conn(0x55bd62ba4540 msgr2=0x55bd62ba4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- 172.21.15.87:0/3483043613 shutdown_connections 2024-04-25T10:15:23.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- 172.21.15.87:0/3483043613 wait complete. 2024-04-25T10:15:23.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 -- start start 2024-04-25T10:15:23.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.427+0000 7f65543b4000 1 Processor -- start 2024-04-25T10:15:23.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62ba2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62ba9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd629a6b50 con 0x55bd62be8290 2024-04-25T10:15:23.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd629e7db0 con 0x55bd62be7970 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd62ba7ad0 con 0x55bd62b9dcd0 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 10 client.0 _ll_register_callbacks cb 0x55bd62a9ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62ba2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6541400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62ba2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:55098/0 (socket says 172.21.15.87:55098) 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 -- 172.21.15.87:0/854261131 learned_addr learned my addr 172.21.15.87:0/854261131 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f654a800640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62ba9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 -- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 msgr2=0x55bd62ba9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 -- 172.21.15.87:0/854261131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 msgr2=0x55bd62ba9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62ba9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 -- 172.21.15.87:0/854261131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd62974520 con 0x55bd62b9dcd0 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f654a800640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62ba9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6541400640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:23.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6549e00640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62ba2c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f653c011750 tx=0x7f653c004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74afe06bbae80c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 1 -- 172.21.15.87:0/854261131 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c01b040 con 0x55bd62b9dcd0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 1 -- 172.21.15.87:0/854261131 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f653c035670 con 0x55bd62b9dcd0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f653c027070 con 0x55bd62b9dcd0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 1 -- 172.21.15.87:0/854261131 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f653c035870 con 0x55bd62b9dcd0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 10 client.5201 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55bd629e6d00 con 0x55bd62b9dcd0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 1 -- 172.21.15.87:0/854261131 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c027070 con 0x55bd62b9dcd0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 1 -- 172.21.15.87:0/854261131 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f653c035a10 con 0x55bd62b9dcd0 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 10 client.5201 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 10 client.5201 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 20 client.5201 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bedda0 0x55bd62bee180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 4 client.5201 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd62987b10 con 0x55bd62bedda0 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6541400640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bedda0 0x55bd62bee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f6541400640 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bedda0 0x55bd62bee180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d6c2be0dbeeca7 server_cookie=2233922446b150c9 in_seq=0 out_seq=0 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 10 client.5201 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 1 -- 172.21.15.87:0/854261131 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd62987b10 con 0x55bd62bedda0 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.431+0000 7f652f400640 10 client.5201 handle_command_reply: tid=0 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 client.5201 shutdown 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 msgr2=0x55bd62ba2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62ba2c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f653c011750 tx=0x7f653c004740 comp rx=0 tx=0).stop 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 shutdown_connections 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be8290 0x55bd62ba9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bedda0 0x55bd62bee180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 --2- 172.21.15.87:0/854261131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62ba2c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f652f400640 0 client.5201 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:23.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 >> 172.21.15.87:0/854261131 conn(0x55bd62ba4540 msgr2=0x55bd62b9c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 shutdown_connections 2024-04-25T10:15:23.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 1 -- 172.21.15.87:0/854261131 wait complete. 2024-04-25T10:15:23.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.435+0000 7f65543b4000 20 client.5201 trim_cache size 0 max 16384 2024-04-25T10:15:23.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 Processor -- start 2024-04-25T10:15:23.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 -- start start 2024-04-25T10:15:23.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62b9dcd0 0x55bd62ba3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be8290 0x55bd62b9bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c202490 con 0x55bd62b9dcd0 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c152350 con 0x55bd62be7970 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.447+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c152950 con 0x55bd62be8290 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62b9dcd0 0x55bd62ba3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62b9dcd0 0x55bd62ba3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37492/0 (socket says 172.21.15.87:37492) 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 -- 172.21.15.87:0/3375619387 learned_addr learned my addr 172.21.15.87:0/3375619387 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6541400640 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f654a800640 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be8290 0x55bd62b9bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 -- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be8290 msgr2=0x55bd62b9bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be8290 0x55bd62b9bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 -- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 msgr2=0x55bd62ba3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 -- 172.21.15.87:0/3375619387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd62a6f920 con 0x55bd62b9dcd0 2024-04-25T10:15:23.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6541400640 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:23.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f654a800640 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be8290 0x55bd62b9bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:23.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62b9dcd0 0x55bd62ba3320 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f653c035ed0 tx=0x7f653c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6966b1a484983b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f653a000640 1 -- 172.21.15.87:0/3375619387 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c047070 con 0x55bd62b9dcd0 2024-04-25T10:15:23.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f653a000640 1 -- 172.21.15.87:0/3375619387 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c027070 con 0x55bd62b9dcd0 2024-04-25T10:15:23.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f653a000640 1 -- 172.21.15.87:0/3375619387 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c01b050 con 0x55bd62b9dcd0 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- 172.21.15.87:0/3375619387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62b9dcd0 msgr2=0x55bd62ba3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62b9dcd0 0x55bd62ba3320 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f653c035ed0 tx=0x7f653c02e070 comp rx=0 tx=0).stop 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- 172.21.15.87:0/3375619387 shutdown_connections 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62be8290 0x55bd62b9bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be7970 0x55bd62ba3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 --2- 172.21.15.87:0/3375619387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62b9dcd0 0x55bd62ba3320 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- 172.21.15.87:0/3375619387 >> 172.21.15.87:0/3375619387 conn(0x55bd62ba4540 msgr2=0x55bd62ba2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- 172.21.15.87:0/3375619387 shutdown_connections 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- 172.21.15.87:0/3375619387 wait complete. 2024-04-25T10:15:23.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- start start 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 Processor -- start 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62c43850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be7970 0x55bd62bebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be8290 0x55bd62bec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd629a2d20 con 0x55bd62be7970 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd629f0340 con 0x55bd62be8290 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd62a6f490 con 0x55bd62b9dcd0 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f65543b4000 10 client.0 _ll_register_callbacks cb 0x7f653c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6541400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be7970 0x55bd62bebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6541400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be7970 0x55bd62bebb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37504/0 (socket says 172.21.15.87:37504) 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6541400640 1 -- 172.21.15.87:0/4221700182 learned_addr learned my addr 172.21.15.87:0/4221700182 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f6549e00640 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62c43850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.451+0000 7f654a800640 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be8290 0x55bd62bec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6541400640 1 -- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 msgr2=0x55bd62c43850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6541400640 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62c43850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6541400640 1 -- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be8290 msgr2=0x55bd62bec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6541400640 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be8290 0x55bd62bec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6541400640 1 -- 172.21.15.87:0/4221700182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd62a45a30 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6541400640 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be7970 0x55bd62bebb30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f654c0ae480 tx=0x7f653401cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64266daa9d3ba1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 1 -- 172.21.15.87:0/4221700182 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340050d0 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 1 -- 172.21.15.87:0/4221700182 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6534005270 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f653c027070 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 1 -- 172.21.15.87:0/4221700182 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6534005470 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 10 client.5838 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55bd62993150 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 1 -- 172.21.15.87:0/4221700182 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c027070 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 1 -- 172.21.15.87:0/4221700182 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6534023020 con 0x55bd62be7970 2024-04-25T10:15:23.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 10 client.5838 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:23.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 10 client.5838 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:23.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 20 client.5838 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:23.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bec5b0 0x55bd62bec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 4 client.5838 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:15:23.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55bd629f9250 con 0x55bd62bec5b0 2024-04-25T10:15:23.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6549e00640 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bec5b0 0x55bd62bec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6549e00640 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bec5b0 0x55bd62bec9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25f6130be51a3db server_cookie=96327ede4186ed6d in_seq=0 out_seq=0 2024-04-25T10:15:23.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 10 client.5838 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:23.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 1 -- 172.21.15.87:0/4221700182 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55bd629f9250 con 0x55bd62bec5b0 2024-04-25T10:15:23.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 10 client.5838 handle_command_reply: tid=0 2024-04-25T10:15:23.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 client.5838 shutdown 2024-04-25T10:15:23.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be7970 msgr2=0x55bd62bebb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be7970 0x55bd62bebb30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f654c0ae480 tx=0x7f653401cf90 comp rx=0 tx=0).stop 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 shutdown_connections 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd62be8290 0x55bd62bec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd62be7970 0x55bd62bebb30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bd62bec5b0 0x55bd62bec9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 --2- 172.21.15.87:0/4221700182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd62b9dcd0 0x55bd62c43850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f652f400640 0 client.5838 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 >> 172.21.15.87:0/4221700182 conn(0x55bd62ba4540 msgr2=0x55bd62be6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 shutdown_connections 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 1 -- 172.21.15.87:0/4221700182 wait complete. 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f65543b4000 20 client.5838 trim_cache size 0 max 16384 2024-04-25T10:15:23.466 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:23.467 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": -2 2024-04-25T10:15:23.467 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:23.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6528066f60 msgr2=0x7f6528069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6528066f60 0x7f6528069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f653c009b60 tx=0x7f653c009190 comp rx=0 tx=0).stop 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 msgr2=0x7f654c1412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c1412e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f65440af040 tx=0x7f6544019790 comp rx=0 tx=0).stop 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 shutdown_connections 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6528066f60 0x7f6528069420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f654c0b31c0 0x7f654c1412e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f654c0aef80 0x7f654c140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 --2- 172.21.15.87:0/2347118550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f654c070350 0x7f654c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.455+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 >> 172.21.15.87:0/2347118550 conn(0x7f654c0669a0 msgr2=0x7f654c06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.459+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 shutdown_connections 2024-04-25T10:15:23.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.459+0000 7f6551000640 1 -- 172.21.15.87:0/2347118550 wait complete. 2024-04-25T10:15:23.482 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:15:23.482 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:15:23.482 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:15:23.482 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:23.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa80e000640 1 Processor -- start 2024-04-25T10:15:23.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa80e000640 1 -- start start 2024-04-25T10:15:23.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa80e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa8080c6a20 0x7fa808166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa80e000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8080ccfb0 con 0x7fa80815b9e0 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa80e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8080cd290 con 0x7fa8080ccb20 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa80e000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8080cd570 con 0x7fa8080c6a20 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa807400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa80815b9e0 0x7fa8080cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44094/0 (socket says 172.21.15.87:44094) 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa807400640 1 -- 172.21.15.87:0/1882665372 learned_addr learned my addr 172.21.15.87:0/1882665372 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa807e00640 1 --2- 172.21.15.87:0/1882665372 >> v2:172.21.15.87:3300/0 conn(0x7fa8080c6a20 0x7fa808166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa806000640 1 -- 172.21.15.87:0/1882665372 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546989765 0 0) 0x7fa8080ccfb0 con 0x7fa80815b9e0 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa806000640 1 -- 172.21.15.87:0/1882665372 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec003530 con 0x7fa80815b9e0 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa807e00640 1 -- 172.21.15.87:0/1882665372 >> v1:172.21.15.169:6789/0 conn(0x7fa8080ccb20 legacy=0x7fa8080d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa807e00640 1 -- 172.21.15.87:0/1882665372 >> v1:172.21.15.87:6789/0 conn(0x7fa80815b9e0 legacy=0x7fa8080cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.611+0000 7fa807e00640 1 -- 172.21.15.87:0/1882665372 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080cd850 con 0x7fa8080c6a20 2024-04-25T10:15:23.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa807e00640 1 --2- 172.21.15.87:0/1882665372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa808166820 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa8000097c0 tx=0x7fa80001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9095f046854de7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa806000640 1 -- 172.21.15.87:0/1882665372 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8000b0070 con 0x7fa8080c6a20 2024-04-25T10:15:23.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa806000640 1 -- 172.21.15.87:0/1882665372 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800004230 con 0x7fa8080c6a20 2024-04-25T10:15:23.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa806000640 1 -- 172.21.15.87:0/1882665372 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800003620 con 0x7fa8080c6a20 2024-04-25T10:15:23.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa80e000640 1 -- 172.21.15.87:0/1882665372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 msgr2=0x7fa808166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa80e000640 1 --2- 172.21.15.87:0/1882665372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa808166820 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa8000097c0 tx=0x7fa80001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa80e000640 1 -- 172.21.15.87:0/1882665372 shutdown_connections 2024-04-25T10:15:23.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa80e000640 1 --2- 172.21.15.87:0/1882665372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa808166820 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa80e000640 1 -- 172.21.15.87:0/1882665372 >> 172.21.15.87:0/1882665372 conn(0x7fa80809a9d0 msgr2=0x7fa80809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa80e000640 1 -- 172.21.15.87:0/1882665372 shutdown_connections 2024-04-25T10:15:23.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.615+0000 7fa80e000640 1 -- 172.21.15.87:0/1882665372 wait complete. 2024-04-25T10:15:23.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 Processor -- start 2024-04-25T10:15:23.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 -- start start 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa80815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80815b9e0 0x7fa8080cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8080ce110 0x7fa80815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa808169910 con 0x7fa8080c6a20 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa808169790 con 0x7fa80815b9e0 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa808169a90 con 0x7fa8080ce110 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa80815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa80815a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37510/0 (socket says 172.21.15.87:37510) 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 -- 172.21.15.87:0/1564820063 learned_addr learned my addr 172.21.15.87:0/1564820063 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807e00640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8080ce110 0x7fa80815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa806a00640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80815b9e0 0x7fa8080cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 -- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8080ce110 msgr2=0x7fa80815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8080ce110 0x7fa80815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 -- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80815b9e0 msgr2=0x7fa8080cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80815b9e0 0x7fa8080cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 -- 172.21.15.87:0/1564820063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080b6730 con 0x7fa8080c6a20 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa806a00640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80815b9e0 0x7fa8080cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa807400640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa80815a0a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800cbf0 tx=0x7fa7f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa7ff400640 1 -- 172.21.15.87:0/1564820063 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8017070 con 0x7fa8080c6a20 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa7ff400640 1 -- 172.21.15.87:0/1564820063 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f800ad40 con 0x7fa8080c6a20 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa7ff400640 1 -- 172.21.15.87:0/1564820063 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8012620 con 0x7fa8080c6a20 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8080b6a10 con 0x7fa8080c6a20 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.619+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8080b6fd0 con 0x7fa8080c6a20 2024-04-25T10:15:23.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.623+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8080ce520 con 0x7fa8080c6a20 2024-04-25T10:15:23.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.623+0000 7fa7ff400640 1 -- 172.21.15.87:0/1564820063 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7f801a070 con 0x7fa8080c6a20 2024-04-25T10:15:23.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.623+0000 7fa7ff400640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e8066f60 0x7fa7e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:23.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.623+0000 7fa7ff400640 1 -- 172.21.15.87:0/1564820063 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7f8084ef0 con 0x7fa8080c6a20 2024-04-25T10:15:23.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.623+0000 7fa806a00640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e8066f60 0x7fa7e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:23.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.623+0000 7fa806a00640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e8066f60 0x7fa7e8069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa8080cb970 tx=0x7fa7f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:23.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.627+0000 7fa7ff400640 1 -- 172.21.15.87:0/1564820063 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8080ce520 con 0x7fa8080c6a20 2024-04-25T10:15:23.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa80804ef30 con 0x7fa8080c6a20 2024-04-25T10:15:23.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa7ff400640 1 -- 172.21.15.87:0/1564820063 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fa7f8055e60 con 0x7fa8080c6a20 2024-04-25T10:15:23.818 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:23.818 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:23.818 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:23.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e8066f60 msgr2=0x7fa7e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e8066f60 0x7fa7e8069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa8080cb970 tx=0x7fa7f4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 msgr2=0x7fa80815a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa80815a0a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800cbf0 tx=0x7fa7f800a430 comp rx=0 tx=0).stop 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 shutdown_connections 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e8066f60 0x7fa7e8069420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8080ce110 0x7fa80815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80815b9e0 0x7fa8080cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.811+0000 7fa80e000640 1 --2- 172.21.15.87:0/1564820063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8080c6a20 0x7fa80815a0a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.815+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 >> 172.21.15.87:0/1564820063 conn(0x7fa80809a9d0 msgr2=0x7fa8080c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.815+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 shutdown_connections 2024-04-25T10:15:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:23.815+0000 7fa80e000640 1 -- 172.21.15.87:0/1564820063 wait complete. 2024-04-25T10:15:23.839 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:15:24.061 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:24.061 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:24.061 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:24.076 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:24.076 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:15:24.077 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:24.077 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:15:24.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5810400640 1 Processor -- start 2024-04-25T10:15:24.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5810400640 1 -- start start 2024-04-25T10:15:24.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5810400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5808159540 0x7f58080ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5810400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f58080721f0 0x7f5808070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5810400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5808159920 con 0x7f58080aebd0 2024-04-25T10:15:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5810400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58080af030 con 0x7f58080721f0 2024-04-25T10:15:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5810400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580815c2f0 con 0x7f5808159540 2024-04-25T10:15:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580d000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f58080aebd0 0x7f5808071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44110/0 (socket says 172.21.15.87:44110) 2024-04-25T10:15:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580d000640 1 -- 172.21.15.87:0/2448391720 learned_addr learned my addr 172.21.15.87:0/2448391720 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580e400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f58080721f0 0x7f5808070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580da00640 1 --2- 172.21.15.87:0/2448391720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5808159540 0x7f58080ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580e400640 1 -- 172.21.15.87:0/2448391720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5808159540 msgr2=0x7f58080ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580e400640 1 --2- 172.21.15.87:0/2448391720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5808159540 0x7f58080ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580e400640 1 -- 172.21.15.87:0/2448391720 >> v1:172.21.15.87:6789/0 conn(0x7f58080aebd0 legacy=0x7f5808071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580e400640 1 -- 172.21.15.87:0/2448391720 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580815c5d0 con 0x7f58080721f0 2024-04-25T10:15:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f580da00640 1 --2- 172.21.15.87:0/2448391720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5808159540 0x7f58080ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.211+0000 7f5807e00640 1 -- 172.21.15.87:0/2448391720 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370910908 0 0) 0x7f5808159920 con 0x7f58080aebd0 2024-04-25T10:15:24.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f580e400640 1 --2- 172.21.15.87:0/2448391720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58080721f0 0x7f5808070350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5800009d90 tx=0x7f580001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f63d6a589b51af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5807e00640 1 -- 172.21.15.87:0/2448391720 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000b2070 con 0x7f58080721f0 2024-04-25T10:15:24.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5807e00640 1 -- 172.21.15.87:0/2448391720 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58000a5a00 con 0x7f58080721f0 2024-04-25T10:15:24.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5807e00640 1 -- 172.21.15.87:0/2448391720 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000a5dd0 con 0x7f58080721f0 2024-04-25T10:15:24.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 -- 172.21.15.87:0/2448391720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58080721f0 msgr2=0x7f5808070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 --2- 172.21.15.87:0/2448391720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58080721f0 0x7f5808070350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5800009d90 tx=0x7f580001c940 comp rx=0 tx=0).stop 2024-04-25T10:15:24.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 -- 172.21.15.87:0/2448391720 shutdown_connections 2024-04-25T10:15:24.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 --2- 172.21.15.87:0/2448391720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58080721f0 0x7f5808070350 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 --2- 172.21.15.87:0/2448391720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5808159540 0x7f58080ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 -- 172.21.15.87:0/2448391720 >> 172.21.15.87:0/2448391720 conn(0x7f58080669a0 msgr2=0x7f5808066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 -- 172.21.15.87:0/2448391720 shutdown_connections 2024-04-25T10:15:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.215+0000 7f5810400640 1 -- 172.21.15.87:0/2448391720 wait complete. 2024-04-25T10:15:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 Processor -- start 2024-04-25T10:15:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 -- start start 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58080721f0 0x7f58081f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58080aebd0 0x7f58080afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5808159540 0x7f58080ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080710a0 con 0x7f5808159540 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580806c0d0 con 0x7f58080aebd0 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5808071220 con 0x7f58080721f0 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5808159540 0x7f58080ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5808159540 0x7f58080ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37524/0 (socket says 172.21.15.87:37524) 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 -- 172.21.15.87:0/297626280 learned_addr learned my addr 172.21.15.87:0/297626280 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580d000640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58080aebd0 0x7f58080afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580da00640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58080721f0 0x7f58081f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 -- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58080721f0 msgr2=0x7f58081f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58080721f0 0x7f58081f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 -- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58080aebd0 msgr2=0x7f58080afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58080aebd0 0x7f58080afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 -- 172.21.15.87:0/297626280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080b03b0 con 0x7f5808159540 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580d000640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58080aebd0 0x7f58080afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580da00640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58080721f0 0x7f58081f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f580e400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5808159540 0x7f58080ad840 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f580001aff0 tx=0x7f58000196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5805600640 1 -- 172.21.15.87:0/297626280 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000b2050 con 0x7f5808159540 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5805600640 1 -- 172.21.15.87:0/297626280 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58000be070 con 0x7f5808159540 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5805600640 1 -- 172.21.15.87:0/297626280 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000adce0 con 0x7f5808159540 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58080b0690 con 0x7f5808159540 2024-04-25T10:15:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.219+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080b0c50 con 0x7f5808159540 2024-04-25T10:15:24.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.223+0000 7f5805600640 1 -- 172.21.15.87:0/297626280 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58000039a0 con 0x7f5808159540 2024-04-25T10:15:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.223+0000 7f5805600640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57e8066f90 0x7f57e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.223+0000 7f5805600640 1 -- 172.21.15.87:0/297626280 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5800120250 con 0x7f5808159540 2024-04-25T10:15:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.223+0000 7f580da00640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57e8066f90 0x7f57e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.223+0000 7f58138cc000 1 Processor -- start 2024-04-25T10:15:24.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.223+0000 7f58138cc000 1 -- start start 2024-04-25T10:15:24.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.223+0000 7f580da00640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57e8066f90 0x7f57e8069450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00a9b0 tx=0x7f57fc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e368280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e3687c0 0x55de5e31e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e31eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5e16cdb0 con 0x55de5e3687c0 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5e0f5230 con 0x55de5e31e710 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de5e168e50 con 0x55de5e367ea0 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e3687c0 0x55de5e31e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e3687c0 0x55de5e31e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37530/0 (socket says 172.21.15.87:37530) 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 -- 172.21.15.87:0/2406070519 learned_addr learned my addr 172.21.15.87:0/2406070519 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580da00640 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e368280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 -- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 msgr2=0x55de5e368280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580e400640 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e31eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e368280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 -- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 msgr2=0x55de5e31eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e31eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 -- 172.21.15.87:0/2406070519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de5e12e750 con 0x55de5e3687c0 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580e400640 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e31eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580da00640 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e368280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f580d000640 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e3687c0 0x55de5e31e1d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f57f40011f0 tx=0x7f57f4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2b56013d8d1136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f57fa000640 1 -- 172.21.15.87:0/2406070519 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4019070 con 0x55de5e3687c0 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f57fa000640 1 -- 172.21.15.87:0/2406070519 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f40018a0 con 0x55de5e3687c0 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f57fa000640 1 -- 172.21.15.87:0/2406070519 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4001c50 con 0x55de5e3687c0 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- 172.21.15.87:0/2406070519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e3687c0 msgr2=0x55de5e31e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e3687c0 0x55de5e31e1d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f57f40011f0 tx=0x7f57f4005bd0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- 172.21.15.87:0/2406070519 shutdown_connections 2024-04-25T10:15:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e31eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e3687c0 0x55de5e31e1d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 --2- 172.21.15.87:0/2406070519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e368280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- 172.21.15.87:0/2406070519 >> 172.21.15.87:0/2406070519 conn(0x55de5e31a6c0 msgr2=0x55de5e31cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- 172.21.15.87:0/2406070519 shutdown_connections 2024-04-25T10:15:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- 172.21.15.87:0/2406070519 wait complete. 2024-04-25T10:15:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 -- start start 2024-04-25T10:15:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.227+0000 7f58138cc000 1 Processor -- start 2024-04-25T10:15:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e34ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e367ea0 0x55de5e34d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e34d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5e127bf0 con 0x55de5e367ea0 2024-04-25T10:15:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5e131770 con 0x55de5e31e710 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de5e0f1990 con 0x55de5e3687c0 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 10 client.0 _ll_register_callbacks cb 0x55de5e309790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e367ea0 0x55de5e34d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e34ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e367ea0 0x55de5e34d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37538/0 (socket says 172.21.15.87:37538) 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 -- 172.21.15.87:0/379916916 learned_addr learned my addr 172.21.15.87:0/379916916 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580e400640 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e34d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e34ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:58440/0 (socket says 172.21.15.87:58440) 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 -- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 msgr2=0x55de5e34d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e34d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 -- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 msgr2=0x55de5e34ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e34ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 -- 172.21.15.87:0/379916916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de5e10f640 con 0x55de5e367ea0 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580d000640 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e367ea0 0x55de5e34d410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f57f4001690 tx=0x7f57f4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c308e7ce6f34c417 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 1 -- 172.21.15.87:0/379916916 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4019030 con 0x55de5e367ea0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 1 -- 172.21.15.87:0/379916916 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f4014480 con 0x55de5e367ea0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57f40018a0 con 0x55de5e367ea0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 1 -- 172.21.15.87:0/379916916 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f402b050 con 0x55de5e367ea0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 10 client.5862 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55de5e0f5480 con 0x55de5e367ea0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 1 -- 172.21.15.87:0/379916916 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f40018a0 con 0x55de5e367ea0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 1 -- 172.21.15.87:0/379916916 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57f402e020 con 0x55de5e367ea0 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 10 client.5862 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 10 client.5862 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 20 client.5862 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e377f40 0x55de5e37a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 4 client.5862 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:24.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de5e108bb0 con 0x55de5e377f40 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580da00640 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e377f40 0x55de5e37a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f580da00640 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e377f40 0x55de5e37a330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8ddb8fec5e95ce server_cookie=a318cbbb556e498a in_seq=0 out_seq=0 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 10 client.5862 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 1 -- 172.21.15.87:0/379916916 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de5e108bb0 con 0x55de5e377f40 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f57eea00640 10 client.5862 handle_command_reply: tid=0 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.231+0000 7f58138cc000 1 client.5862 shutdown 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e367ea0 msgr2=0x55de5e34d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e367ea0 0x55de5e34d410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f57f4001690 tx=0x7f57f4000ee0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 shutdown_connections 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e377f40 0x55de5e37a330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e34d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e367ea0 0x55de5e34d410 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f57eea00640 0 client.5862 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 --2- 172.21.15.87:0/379916916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e31e710 0x55de5e34ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 >> 172.21.15.87:0/379916916 conn(0x55de5e31a6c0 msgr2=0x55de5e31d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 shutdown_connections 2024-04-25T10:15:24.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 1 -- 172.21.15.87:0/379916916 wait complete. 2024-04-25T10:15:24.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.235+0000 7f58138cc000 20 client.5862 trim_cache size 0 max 16384 2024-04-25T10:15:24.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.247+0000 7f58138cc000 1 Processor -- start 2024-04-25T10:15:24.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.247+0000 7f58138cc000 1 -- start start 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.247+0000 7f58138cc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e31cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.247+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e31cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e3687c0 0x55de5e31d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808202780 con 0x55de5e31e710 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b38d0 con 0x55de5e3687c0 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58081fa5f0 con 0x55de5e367ea0 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e31cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e31cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37548/0 (socket says 172.21.15.87:37548) 2024-04-25T10:15:24.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 -- 172.21.15.87:0/1929845174 learned_addr learned my addr 172.21.15.87:0/1929845174 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580d000640 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e31cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580e400640 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e3687c0 0x55de5e31d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 -- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 msgr2=0x55de5e31cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e31cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 -- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e3687c0 msgr2=0x55de5e31d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e3687c0 0x55de5e31d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 -- 172.21.15.87:0/1929845174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de5e1eeef0 con 0x55de5e31e710 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580d000640 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e31cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e31cab0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f57fc006fd0 tx=0x7f57fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74a668f8e00e3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f57f9600640 1 -- 172.21.15.87:0/1929845174 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc015410 con 0x55de5e31e710 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f57f9600640 1 -- 172.21.15.87:0/1929845174 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc02e070 con 0x55de5e31e710 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f57f9600640 1 -- 172.21.15.87:0/1929845174 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0157b0 con 0x55de5e31e710 2024-04-25T10:15:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- 172.21.15.87:0/1929845174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 msgr2=0x55de5e31cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e31cab0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f57fc006fd0 tx=0x7f57fc015000 comp rx=0 tx=0).stop 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- 172.21.15.87:0/1929845174 shutdown_connections 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e3687c0 0x55de5e31d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e367ea0 0x55de5e31cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- 172.21.15.87:0/1929845174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e31cab0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- 172.21.15.87:0/1929845174 >> 172.21.15.87:0/1929845174 conn(0x55de5e31a6c0 msgr2=0x55de5e375490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- 172.21.15.87:0/1929845174 shutdown_connections 2024-04-25T10:15:24.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- 172.21.15.87:0/1929845174 wait complete. 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- start start 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 Processor -- start 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e3672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e367ea0 0x55de5e37a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e37ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5e127bf0 con 0x55de5e31e710 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b32d0 con 0x55de5e367ea0 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de5e16c0c0 con 0x55de5e3687c0 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f58138cc000 10 client.0 _ll_register_callbacks cb 0x7f57f4007e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e3672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e3672b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37558/0 (socket says 172.21.15.87:37558) 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580da00640 1 -- 172.21.15.87:0/4194396408 learned_addr learned my addr 172.21.15.87:0/4194396408 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580e400640 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e37ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.251+0000 7f580d000640 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e367ea0 0x55de5e37a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580da00640 1 -- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 msgr2=0x55de5e37ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580da00640 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e37ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580da00640 1 -- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e367ea0 msgr2=0x55de5e37a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580da00640 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e367ea0 0x55de5e37a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580da00640 1 -- 172.21.15.87:0/4194396408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de5e12c9a0 con 0x55de5e31e710 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580da00640 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e3672b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f57fc008010 tx=0x7f57fc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8ffd0ed0202f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 1 -- 172.21.15.87:0/4194396408 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc034650 con 0x55de5e31e710 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 1 -- 172.21.15.87:0/4194396408 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57fc0347f0 con 0x55de5e31e710 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57fc02e070 con 0x55de5e31e710 2024-04-25T10:15:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 1 -- 172.21.15.87:0/4194396408 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57fc041050 con 0x55de5e31e710 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 10 client.5868 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55de5de295a0 con 0x55de5e31e710 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 1 -- 172.21.15.87:0/4194396408 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc02e070 con 0x55de5e31e710 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 1 -- 172.21.15.87:0/4194396408 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57fc044020 con 0x55de5e31e710 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 10 client.5868 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 10 client.5868 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 20 client.5868 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e37b0c0 0x55de5e3c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 4 client.5868 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55de5e11c080 con 0x55de5e37b0c0 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580d000640 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e37b0c0 0x55de5e3c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f580d000640 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e37b0c0 0x55de5e3c88c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc537f072a4a858 server_cookie=1eb0858c335de13f in_seq=0 out_seq=0 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 10 client.5868 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 1 -- 172.21.15.87:0/4194396408 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55de5e11c080 con 0x55de5e37b0c0 2024-04-25T10:15:24.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 10 client.5868 handle_command_reply: tid=0 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 client.5868 shutdown 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 msgr2=0x55de5e3672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e3672b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f57fc008010 tx=0x7f57fc004740 comp rx=0 tx=0).stop 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 shutdown_connections 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55de5e37b0c0 0x55de5e3c88c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55de5e3687c0 0x55de5e37ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de5e367ea0 0x55de5e37a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f57eea00640 0 client.5868 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 --2- 172.21.15.87:0/4194396408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de5e31e710 0x55de5e3672b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 >> 172.21.15.87:0/4194396408 conn(0x55de5e31a6c0 msgr2=0x55de5e3667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 shutdown_connections 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 1 -- 172.21.15.87:0/4194396408 wait complete. 2024-04-25T10:15:24.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f58138cc000 20 client.5868 trim_cache size 0 max 16384 2024-04-25T10:15:24.267 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:24.267 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:24.267 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "db25cadb-342d-46f1-836e-bd98abb62eaa", 2024-04-25T10:15:24.267 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:24.267 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57e8066f90 msgr2=0x7f57e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57e8066f90 0x7f57e8069450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00a9b0 tx=0x7f57fc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5808159540 msgr2=0x7f58080ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5808159540 0x7f58080ad840 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f580001aff0 tx=0x7f58000196e0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 shutdown_connections 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57e8066f90 0x7f57e8069450 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5808159540 0x7f58080ad840 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.255+0000 7f5810400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58080aebd0 0x7f58080afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.259+0000 7f5810400640 1 --2- 172.21.15.87:0/297626280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58080721f0 0x7f58081f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.259+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 >> 172.21.15.87:0/297626280 conn(0x7f58080669a0 msgr2=0x7f580815d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.259+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 shutdown_connections 2024-04-25T10:15:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.259+0000 7f5810400640 1 -- 172.21.15.87:0/297626280 wait complete. 2024-04-25T10:15:24.281 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'db25cadb-342d-46f1-836e-bd98abb62eaa', 'mode': 'asynchronous'}' 2024-04-25T10:15:24.281 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:15:24.281 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:15:24.282 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:24.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20bc000640 1 Processor -- start 2024-04-25T10:15:24.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20bc000640 1 -- start start 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20bc000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20b41550b0 0x7f20b40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20bc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b415b910 0x7f20b40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20bc000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b4071050 con 0x7f20b40cfe70 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20bc000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b415bda0 con 0x7f20b41550b0 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20bc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b415d040 con 0x7f20b415b910 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20b9600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20b41550b0 0x7f20b40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20b8c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f20b40cfe70 0x7f20b415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44116/0 (socket says 172.21.15.87:44116) 2024-04-25T10:15:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.411+0000 7f20b8c00640 1 -- 172.21.15.87:0/2014518779 learned_addr learned my addr 172.21.15.87:0/2014518779 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20ba000640 1 --2- 172.21.15.87:0/2014518779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b415b910 0x7f20b40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b9600640 1 -- 172.21.15.87:0/2014518779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b415b910 msgr2=0x7f20b40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b9600640 1 --2- 172.21.15.87:0/2014518779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b415b910 0x7f20b40b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b9600640 1 -- 172.21.15.87:0/2014518779 >> v1:172.21.15.87:6789/0 conn(0x7f20b40cfe70 legacy=0x7f20b415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b9600640 1 -- 172.21.15.87:0/2014518779 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40712f0 con 0x7f20b41550b0 2024-04-25T10:15:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b3e00640 1 -- 172.21.15.87:0/2014518779 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689452190 0 0) 0x7f20b4071050 con 0x7f20b40cfe70 2024-04-25T10:15:24.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b9600640 1 --2- 172.21.15.87:0/2014518779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41550b0 0x7f20b40cf930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009dc0 tx=0x7f20a801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c370b9e72dba4abc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b3e00640 1 -- 172.21.15.87:0/2014518779 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80b2070 con 0x7f20b41550b0 2024-04-25T10:15:24.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b3e00640 1 -- 172.21.15.87:0/2014518779 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a80a5810 con 0x7f20b41550b0 2024-04-25T10:15:24.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20b3e00640 1 -- 172.21.15.87:0/2014518779 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80a5be0 con 0x7f20b41550b0 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20bc000640 1 -- 172.21.15.87:0/2014518779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41550b0 msgr2=0x7f20b40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.415+0000 7f20bc000640 1 --2- 172.21.15.87:0/2014518779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41550b0 0x7f20b40cf930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009dc0 tx=0x7f20a801c950 comp rx=0 tx=0).stop 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- 172.21.15.87:0/2014518779 shutdown_connections 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 --2- 172.21.15.87:0/2014518779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b415b910 0x7f20b40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 --2- 172.21.15.87:0/2014518779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41550b0 0x7f20b40cf930 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- 172.21.15.87:0/2014518779 >> 172.21.15.87:0/2014518779 conn(0x7f20b4066b70 msgr2=0x7f20b4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- 172.21.15.87:0/2014518779 shutdown_connections 2024-04-25T10:15:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- 172.21.15.87:0/2014518779 wait complete. 2024-04-25T10:15:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 Processor -- start 2024-04-25T10:15:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- start start 2024-04-25T10:15:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b40cfe70 0x7f20b41382e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b41550b0 0x7f20b41518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b415b910 0x7f20b4151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b415c030 con 0x7f20b415b910 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40720b0 con 0x7f20b40cfe70 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20bc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b415c1b0 con 0x7f20b41550b0 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b415b910 0x7f20b4151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b415b910 0x7f20b4151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37588/0 (socket says 172.21.15.87:37588) 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 -- 172.21.15.87:0/3785114724 learned_addr learned my addr 172.21.15.87:0/3785114724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20b8c00640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b41550b0 0x7f20b41518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20b9600640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b40cfe70 0x7f20b41382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 -- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b41550b0 msgr2=0x7f20b41518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b41550b0 0x7f20b41518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 -- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b40cfe70 msgr2=0x7f20b41382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b40cfe70 0x7f20b41382e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 -- 172.21.15.87:0/3785114724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40cd970 con 0x7f20b415b910 2024-04-25T10:15:24.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20b9600640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b40cfe70 0x7f20b41382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20b8c00640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b41550b0 0x7f20b41518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20ba000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b415b910 0x7f20b4151e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00e7f0 tx=0x7f20ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.419+0000 7f20b1600640 1 -- 172.21.15.87:0/3785114724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac017070 con 0x7f20b415b910 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20b1600640 1 -- 172.21.15.87:0/3785114724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac00c940 con 0x7f20b415b910 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b40cdc50 con 0x7f20b415b910 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20b1600640 1 -- 172.21.15.87:0/3785114724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac00eea0 con 0x7f20b415b910 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b40cca50 con 0x7f20b415b910 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b404ef30 con 0x7f20b415b910 2024-04-25T10:15:24.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20b1600640 1 -- 172.21.15.87:0/3785114724 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20ac01a070 con 0x7f20b415b910 2024-04-25T10:15:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20b1600640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2094066f60 0x7f2094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20b1600640 1 -- 172.21.15.87:0/3785114724 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20ac085280 con 0x7f20b415b910 2024-04-25T10:15:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20b9600640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2094066f60 0x7f2094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.423+0000 7f20b9600640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2094066f60 0x7f2094069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20a80af000 tx=0x7f20a80a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.427+0000 7f20b1600640 1 -- 172.21.15.87:0/3785114724 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20ac056290 con 0x7f20b415b910 2024-04-25T10:15:24.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.615+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20b415e170 con 0x7f20b415b910 2024-04-25T10:15:24.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.615+0000 7f20b1600640 1 -- 172.21.15.87:0/3785114724 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f20ac055b90 con 0x7f20b415b910 2024-04-25T10:15:24.623 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:24.623 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:24.623 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:24.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2094066f60 msgr2=0x7f2094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2094066f60 0x7f2094069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20a80af000 tx=0x7f20a80a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:24.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b415b910 msgr2=0x7f20b4151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b415b910 0x7f20b4151e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00e7f0 tx=0x7f20ac00ecb0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 shutdown_connections 2024-04-25T10:15:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2094066f60 0x7f2094069420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b415b910 0x7f20b4151e10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b41550b0 0x7f20b41518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 --2- 172.21.15.87:0/3785114724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b40cfe70 0x7f20b41382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 >> 172.21.15.87:0/3785114724 conn(0x7f20b4066b70 msgr2=0x7f20b40cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 shutdown_connections 2024-04-25T10:15:24.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.619+0000 7f20bc000640 1 -- 172.21.15.87:0/3785114724 wait complete. 2024-04-25T10:15:24.643 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:15:24.787 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:24.787 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:24.787 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:24.803 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:24.803 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:15:24.803 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:24.803 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:15:24.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104fe00640 1 Processor -- start 2024-04-25T10:15:24.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104fe00640 1 -- start start 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10480aeda0 0x7f1048071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048072020 0x7f1048070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1048159920 con 0x7f1048159540 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104815c2f0 con 0x7f10480aeda0 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104815c5d0 con 0x7f1048072020 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10480aeda0 0x7f1048071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048072020 0x7f1048070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048072020 0x7f1048070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:55194/0 (socket says 172.21.15.87:55194) 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104de00640 1 -- 172.21.15.87:0/1148491610 learned_addr learned my addr 172.21.15.87:0/1148491610 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.935+0000 7f104ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10480aeda0 0x7f1048071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:58486/0 (socket says 172.21.15.87:58486) 2024-04-25T10:15:24.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f104de00640 1 -- 172.21.15.87:0/1148491610 >> v2:172.21.15.169:3300/0 conn(0x7f10480aeda0 msgr2=0x7f1048071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f104de00640 1 --2- 172.21.15.87:0/1148491610 >> v2:172.21.15.169:3300/0 conn(0x7f10480aeda0 0x7f1048071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f104de00640 1 -- 172.21.15.87:0/1148491610 >> v1:172.21.15.169:6789/0 conn(0x7f1048159540 legacy=0x7f10480ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f104de00640 1 -- 172.21.15.87:0/1148491610 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104815c8b0 con 0x7f1048072020 2024-04-25T10:15:24.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f104ca00640 1 --2- 172.21.15.87:0/1148491610 >> v2:172.21.15.169:3300/0 conn(0x7f10480aeda0 0x7f1048071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:24.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f104de00640 1 --2- 172.21.15.87:0/1148491610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048072020 0x7f1048070350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1040009e50 tx=0x7f104001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ddd4e4c610b2e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f1047e00640 1 -- 172.21.15.87:0/1148491610 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400a58b0 con 0x7f1048072020 2024-04-25T10:15:24.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f1047e00640 1 -- 172.21.15.87:0/1148491610 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10400a5a50 con 0x7f1048072020 2024-04-25T10:15:24.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.939+0000 7f1047e00640 1 -- 172.21.15.87:0/1148491610 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400a5e20 con 0x7f1048072020 2024-04-25T10:15:24.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- 172.21.15.87:0/1148491610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048072020 msgr2=0x7f1048070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 --2- 172.21.15.87:0/1148491610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048072020 0x7f1048070350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1040009e50 tx=0x7f104001ca00 comp rx=0 tx=0).stop 2024-04-25T10:15:24.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- 172.21.15.87:0/1148491610 shutdown_connections 2024-04-25T10:15:24.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 --2- 172.21.15.87:0/1148491610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048072020 0x7f1048070350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 --2- 172.21.15.87:0/1148491610 >> v2:172.21.15.169:3300/0 conn(0x7f10480aeda0 0x7f1048071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- 172.21.15.87:0/1148491610 >> 172.21.15.87:0/1148491610 conn(0x7f10480669a0 msgr2=0x7f1048066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- 172.21.15.87:0/1148491610 shutdown_connections 2024-04-25T10:15:24.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- 172.21.15.87:0/1148491610 wait complete. 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 Processor -- start 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- start start 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1048072020 0x7f1048152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10480aeda0 0x7f10481fa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048159540 0x7f1048155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480710a0 con 0x7f10480aeda0 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104806c0d0 con 0x7f1048072020 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1048071220 con 0x7f1048159540 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10480aeda0 0x7f10481fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10480aeda0 0x7f10481fa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37592/0 (socket says 172.21.15.87:37592) 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 -- 172.21.15.87:0/3127357306 learned_addr learned my addr 172.21.15.87:0/3127357306 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 -- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048159540 msgr2=0x7f1048155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104de00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048159540 0x7f1048155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104d400640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1048072020 0x7f1048152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048159540 0x7f1048155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 -- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1048072020 msgr2=0x7f1048152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1048072020 0x7f1048152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 -- 172.21.15.87:0/3127357306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480b3b60 con 0x7f10480aeda0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104de00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048159540 0x7f1048155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104d400640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1048072020 0x7f1048152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.943+0000 7f104ca00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10480aeda0 0x7f10481fa960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f103400e9d0 tx=0x7f103400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f1045600640 1 -- 172.21.15.87:0/3127357306 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034017070 con 0x7f10480aeda0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f1045600640 1 -- 172.21.15.87:0/3127357306 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103400ca60 con 0x7f10480aeda0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10480b3e40 con 0x7f10480aeda0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f1045600640 1 -- 172.21.15.87:0/3127357306 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400ce50 con 0x7f10480aeda0 2024-04-25T10:15:24.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10480b4400 con 0x7f10480aeda0 2024-04-25T10:15:24.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f1045600640 1 -- 172.21.15.87:0/3127357306 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f103401a070 con 0x7f10480aeda0 2024-04-25T10:15:24.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f1045600640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1028067100 0x7f10280695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f1045600640 1 -- 172.21.15.87:0/3127357306 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10340850c0 con 0x7f10480aeda0 2024-04-25T10:15:24.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f104d400640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1028067100 0x7f10280695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.947+0000 7f10533a7000 1 Processor -- start 2024-04-25T10:15:24.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- start start 2024-04-25T10:15:24.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1028067100 0x7f10280695c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f103c001650 tx=0x7f103c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf5b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5bb20 0x5649abf3fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 0x5649abf40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649abdf84a0 con 0x5649abf5b200 2024-04-25T10:15:24.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649abd60210 con 0x5649abf5bb20 2024-04-25T10:15:24.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649abce8980 con 0x5649abf3ff60 2024-04-25T10:15:24.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf5b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf5b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37600/0 (socket says 172.21.15.87:37600) 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 -- 172.21.15.87:0/573229403 learned_addr learned my addr 172.21.15.87:0/573229403 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 -- 172.21.15.87:0/573229403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 msgr2=0x5649abf40340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 --2- 172.21.15.87:0/573229403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 0x5649abf40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 -- 172.21.15.87:0/573229403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5bb20 msgr2=0x5649abf3fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 --2- 172.21.15.87:0/573229403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5bb20 0x5649abf3fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 -- 172.21.15.87:0/573229403 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649abd5b200 con 0x5649abf5b200 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f104d400640 1 --2- 172.21.15.87:0/573229403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf5b5e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f103c00d890 tx=0x7f103c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9242b5d06cd5bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f103a000640 1 -- 172.21.15.87:0/573229403 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c014ba0 con 0x5649abf5b200 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f103a000640 1 -- 172.21.15.87:0/573229403 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c014d40 con 0x5649abf5b200 2024-04-25T10:15:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f103a000640 1 -- 172.21.15.87:0/573229403 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c007220 con 0x5649abf5b200 2024-04-25T10:15:24.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- 172.21.15.87:0/573229403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 msgr2=0x5649abf5b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 --2- 172.21.15.87:0/573229403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf5b5e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f103c00d890 tx=0x7f103c010000 comp rx=0 tx=0).stop 2024-04-25T10:15:24.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- 172.21.15.87:0/573229403 shutdown_connections 2024-04-25T10:15:24.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 --2- 172.21.15.87:0/573229403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 0x5649abf40340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 --2- 172.21.15.87:0/573229403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5bb20 0x5649abf3fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 --2- 172.21.15.87:0/573229403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf5b5e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- 172.21.15.87:0/573229403 >> 172.21.15.87:0/573229403 conn(0x5649abf084f0 msgr2=0x5649abf17a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- 172.21.15.87:0/573229403 shutdown_connections 2024-04-25T10:15:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- 172.21.15.87:0/573229403 wait complete. 2024-04-25T10:15:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 -- start start 2024-04-25T10:15:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.951+0000 7f10533a7000 1 Processor -- start 2024-04-25T10:15:24.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf62980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5b200 0x5649abf62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649abd1ad80 con 0x5649abf5bb20 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649abd5c2b0 con 0x5649abf3ff60 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649abd21830 con 0x5649abf5b200 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 10 client.0 _ll_register_callbacks cb 0x5649abe13070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf40340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37606/0 (socket says 172.21.15.87:37606) 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 -- 172.21.15.87:0/4225077303 learned_addr learned my addr 172.21.15.87:0/4225077303 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104d400640 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf62980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104ca00640 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5b200 0x5649abf62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 -- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5b200 msgr2=0x5649abf62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5b200 0x5649abf62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 -- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 msgr2=0x5649abf62980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf62980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 -- 172.21.15.87:0/4225077303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649abd2c130 con 0x5649abf5bb20 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104de00640 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf40340 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f104001a630 tx=0x7f10400197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f8764006574791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f102ea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f102ea00640 1 -- 172.21.15.87:0/4225077303 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400a58b0 con 0x5649abf5bb20 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f102ea00640 1 -- 172.21.15.87:0/4225077303 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10400b8440 con 0x5649abf5bb20 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f103c014d40 con 0x5649abf5bb20 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f102ea00640 1 -- 172.21.15.87:0/4225077303 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10400c6050 con 0x5649abf5bb20 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 10 client.5886 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5649abde1490 con 0x5649abf5bb20 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f102ea00640 1 -- 172.21.15.87:0/4225077303 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c014d40 con 0x5649abf5bb20 2024-04-25T10:15:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f102ea00640 1 -- 172.21.15.87:0/4225077303 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1040008040 con 0x5649abf5bb20 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 10 client.5886 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 10 client.5886 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 20 client.5886 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf410e0 0x5649abf414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 4 client.5886 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649abcfc020 con 0x5649abf410e0 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104d400640 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf410e0 0x5649abf414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f104d400640 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf410e0 0x5649abf414c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d37f6b4156baeede server_cookie=8c575e7ac69c5a68 in_seq=0 out_seq=0 2024-04-25T10:15:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.955+0000 7f102ea00640 10 client.5886 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f102ea00640 1 -- 172.21.15.87:0/4225077303 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5649abcfc020 con 0x5649abf410e0 2024-04-25T10:15:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f102ea00640 10 client.5886 handle_command_reply: tid=0 2024-04-25T10:15:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 client.5886 shutdown 2024-04-25T10:15:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 msgr2=0x5649abf40340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf40340 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f104001a630 tx=0x7f10400197a0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 shutdown_connections 2024-04-25T10:15:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf40340 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf410e0 0x5649abf414c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5b200 0x5649abf62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 --2- 172.21.15.87:0/4225077303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf62980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f102ea00640 0 client.5886 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 >> 172.21.15.87:0/4225077303 conn(0x5649abf084f0 msgr2=0x5649abf0faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 shutdown_connections 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 1 -- 172.21.15.87:0/4225077303 wait complete. 2024-04-25T10:15:24.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.959+0000 7f10533a7000 20 client.5886 trim_cache size 0 max 16384 2024-04-25T10:15:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.971+0000 7f10533a7000 1 Processor -- start 2024-04-25T10:15:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.971+0000 7f10533a7000 1 -- start start 2024-04-25T10:15:24.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf15f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf16440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5bb20 0x5649abf16980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048202c90 con 0x5649abf5b200 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480b20a0 con 0x5649abf3ff60 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10480b14a0 con 0x5649abf5bb20 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf16440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf16440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37620/0 (socket says 172.21.15.87:37620) 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 -- 172.21.15.87:0/2091363294 learned_addr learned my addr 172.21.15.87:0/2091363294 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104d400640 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf15f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104de00640 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5bb20 0x5649abf16980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 -- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5bb20 msgr2=0x5649abf16980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5bb20 0x5649abf16980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 -- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 msgr2=0x5649abf15f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf15f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 -- 172.21.15.87:0/2091363294 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649abd1fa30 con 0x5649abf5b200 2024-04-25T10:15:24.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104d400640 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf15f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:24.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f104ca00640 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf16440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1034083550 tx=0x7f10340553c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b896f82a9ca44651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f1039600640 1 -- 172.21.15.87:0/2091363294 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034021030 con 0x5649abf5b200 2024-04-25T10:15:24.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f1039600640 1 -- 172.21.15.87:0/2091363294 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034072070 con 0x5649abf5b200 2024-04-25T10:15:24.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f1039600640 1 -- 172.21.15.87:0/2091363294 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034053c30 con 0x5649abf5b200 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- 172.21.15.87:0/2091363294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 msgr2=0x5649abf16440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf16440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1034083550 tx=0x7f10340553c0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- 172.21.15.87:0/2091363294 shutdown_connections 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf5bb20 0x5649abf16980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5b200 0x5649abf16440 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- 172.21.15.87:0/2091363294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf3ff60 0x5649abf15f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- 172.21.15.87:0/2091363294 >> 172.21.15.87:0/2091363294 conn(0x5649abf084f0 msgr2=0x5649abf3e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- 172.21.15.87:0/2091363294 shutdown_connections 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- 172.21.15.87:0/2091363294 wait complete. 2024-04-25T10:15:24.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- start start 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 Processor -- start 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 0x5649abf10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5b200 0x5649abf11010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf11550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649abcec190 con 0x5649abf5bb20 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480b1aa0 con 0x5649abf5b200 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649abd5c2b0 con 0x5649abf3ff60 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.975+0000 7f10533a7000 10 client.0 _ll_register_callbacks cb 0x7f103c001390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf11550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf11550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37634/0 (socket says 172.21.15.87:37634) 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 0x5649abf10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 -- 172.21.15.87:0/2171667527 learned_addr learned my addr 172.21.15.87:0/2171667527 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104ca00640 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5b200 0x5649abf11010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 -- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 msgr2=0x5649abf10650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 0x5649abf10650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 -- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5b200 msgr2=0x5649abf11010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5b200 0x5649abf11010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 -- 172.21.15.87:0/2171667527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649abd28040 con 0x5649abf5bb20 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104de00640 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf11550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f10400985d0 tx=0x7f10400a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249c9d1239a6460e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 1 -- 172.21.15.87:0/2171667527 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400ce070 con 0x5649abf5bb20 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 1 -- 172.21.15.87:0/2171667527 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1040002040 con 0x5649abf5bb20 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1034072070 con 0x5649abf5bb20 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 1 -- 172.21.15.87:0/2171667527 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10400a5590 con 0x5649abf5bb20 2024-04-25T10:15:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 10 client.5892 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5649aba1c900 con 0x5649abf5bb20 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 1 -- 172.21.15.87:0/2171667527 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034072070 con 0x5649abf5bb20 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 1 -- 172.21.15.87:0/2171667527 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10400b8a30 con 0x5649abf5bb20 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 10 client.5892 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 10 client.5892 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 20 client.5892 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf81850 0x5649abf81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 4 client.5892 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x5649abd6d750 con 0x5649abf81850 2024-04-25T10:15:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104d400640 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf81850 0x5649abf81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:24.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f104d400640 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf81850 0x5649abf81c60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4dc0afd413e6105 server_cookie=5fa8aa16f5cc585e in_seq=0 out_seq=0 2024-04-25T10:15:24.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 10 client.5892 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 1 -- 172.21.15.87:0/2171667527 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5649abd6d750 con 0x5649abf81850 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 10 client.5892 handle_command_reply: tid=0 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 client.5892 shutdown 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 msgr2=0x5649abf11550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf11550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f10400985d0 tx=0x7f10400a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 shutdown_connections 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5649abf5bb20 0x5649abf11550 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649abf5b200 0x5649abf11010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5649abf81850 0x5649abf81c60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 --2- 172.21.15.87:0/2171667527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649abf3ff60 0x5649abf10650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 >> 172.21.15.87:0/2171667527 conn(0x5649abf084f0 msgr2=0x5649abf12c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f102ea00640 0 client.5892 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 shutdown_connections 2024-04-25T10:15:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 1 -- 172.21.15.87:0/2171667527 wait complete. 2024-04-25T10:15:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.979+0000 7f10533a7000 20 client.5892 trim_cache size 0 max 16384 2024-04-25T10:15:24.991 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:24.991 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:24.991 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "9e44ab69-08f4-418c-84e1-115af6260f46", 2024-04-25T10:15:24.991 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:24.991 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1028067100 msgr2=0x7f10280695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1028067100 0x7f10280695c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f103c001650 tx=0x7f103c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10480aeda0 msgr2=0x7f10481fa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10480aeda0 0x7f10481fa960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f103400e9d0 tx=0x7f103400ee90 comp rx=0 tx=0).stop 2024-04-25T10:15:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 shutdown_connections 2024-04-25T10:15:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1028067100 0x7f10280695c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1048159540 0x7f1048155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10480aeda0 0x7f10481fa960 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 --2- 172.21.15.87:0/3127357306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1048072020 0x7f1048152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:24.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 >> 172.21.15.87:0/3127357306 conn(0x7f10480669a0 msgr2=0x7f104815fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:24.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 shutdown_connections 2024-04-25T10:15:24.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:24.983+0000 7f104fe00640 1 -- 172.21.15.87:0/3127357306 wait complete. 2024-04-25T10:15:25.007 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '9e44ab69-08f4-418c-84e1-115af6260f46', 'mode': 'asynchronous'}' 2024-04-25T10:15:25.007 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T10:15:25.007 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:15:25.007 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.135+0000 7f1006c00640 1 Processor -- start 2024-04-25T10:15:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.135+0000 7f1006c00640 1 -- start start 2024-04-25T10:15:25.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.135+0000 7f1006c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f10001550b0 0x7f10000cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.135+0000 7f1006c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f10000b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.135+0000 7f1006c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1000071050 con 0x7f10000cfe70 2024-04-25T10:15:25.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.135+0000 7f1006c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100015bda0 con 0x7f10001550b0 2024-04-25T10:15:25.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.135+0000 7f1006c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100015d040 con 0x7f100015b910 2024-04-25T10:15:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0ff7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f10000cfe70 0x7f100015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44120/0 (socket says 172.21.15.87:44120) 2024-04-25T10:15:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0ff7400640 1 -- 172.21.15.87:0/2188084415 learned_addr learned my addr 172.21.15.87:0/2188084415 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fffe00640 1 --2- 172.21.15.87:0/2188084415 >> v2:172.21.15.87:3300/0 conn(0x7f10001550b0 0x7f10000cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f1004c00640 1 --2- 172.21.15.87:0/2188084415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f10000b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fff400640 1 -- 172.21.15.87:0/2188084415 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668482524 0 0) 0x7f1000071050 con 0x7f10000cfe70 2024-04-25T10:15:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fff400640 1 -- 172.21.15.87:0/2188084415 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fe4005740 con 0x7f10000cfe70 2024-04-25T10:15:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fffe00640 1 -- 172.21.15.87:0/2188084415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 msgr2=0x7f10000b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fffe00640 1 --2- 172.21.15.87:0/2188084415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f10000b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fffe00640 1 -- 172.21.15.87:0/2188084415 >> v1:172.21.15.87:6789/0 conn(0x7f10000cfe70 legacy=0x7f100015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fffe00640 1 -- 172.21.15.87:0/2188084415 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000712f0 con 0x7f10001550b0 2024-04-25T10:15:25.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fffe00640 1 --2- 172.21.15.87:0/2188084415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10001550b0 0x7f10000cf930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097f0 tx=0x7f0ff001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3defc3b1989dfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fff400640 1 -- 172.21.15.87:0/2188084415 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00b2070 con 0x7f10001550b0 2024-04-25T10:15:25.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fff400640 1 -- 172.21.15.87:0/2188084415 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff00ad640 con 0x7f10001550b0 2024-04-25T10:15:25.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.139+0000 7f0fff400640 1 -- 172.21.15.87:0/2188084415 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00ad9f0 con 0x7f10001550b0 2024-04-25T10:15:25.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- 172.21.15.87:0/2188084415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10001550b0 msgr2=0x7f10000cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 --2- 172.21.15.87:0/2188084415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10001550b0 0x7f10000cf930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097f0 tx=0x7f0ff001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- 172.21.15.87:0/2188084415 shutdown_connections 2024-04-25T10:15:25.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 --2- 172.21.15.87:0/2188084415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f10000b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 --2- 172.21.15.87:0/2188084415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10001550b0 0x7f10000cf930 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- 172.21.15.87:0/2188084415 >> 172.21.15.87:0/2188084415 conn(0x7f1000066b70 msgr2=0x7f1000066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- 172.21.15.87:0/2188084415 shutdown_connections 2024-04-25T10:15:25.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- 172.21.15.87:0/2188084415 wait complete. 2024-04-25T10:15:25.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 Processor -- start 2024-04-25T10:15:25.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- start start 2024-04-25T10:15:25.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10000cfe70 0x7f100014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10001550b0 0x7f1000156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f1000155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100015c030 con 0x7f10000cfe70 2024-04-25T10:15:25.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10000720b0 con 0x7f10001550b0 2024-04-25T10:15:25.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1006c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100015c1b0 con 0x7f100015b910 2024-04-25T10:15:25.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10000cfe70 0x7f100014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10000cfe70 0x7f100014af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37646/0 (socket says 172.21.15.87:37646) 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 -- 172.21.15.87:0/2015709615 learned_addr learned my addr 172.21.15.87:0/2015709615 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0ff7400640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10001550b0 0x7f1000156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f1004c00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f1000155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 -- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 msgr2=0x7f1000155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f1000155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 -- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10001550b0 msgr2=0x7f1000156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10001550b0 0x7f1000156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.143+0000 7f0fffe00640 1 -- 172.21.15.87:0/2015709615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000155ee0 con 0x7f10000cfe70 2024-04-25T10:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0fffe00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10000cfe70 0x7f100014af80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ff001a0d0 tx=0x7f0ff0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0ffcc00640 1 -- 172.21.15.87:0/2015709615 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00b2040 con 0x7f10000cfe70 2024-04-25T10:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0ffcc00640 1 -- 172.21.15.87:0/2015709615 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff00ad440 con 0x7f10000cfe70 2024-04-25T10:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0ffcc00640 1 -- 172.21.15.87:0/2015709615 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0006e20 con 0x7f10000cfe70 2024-04-25T10:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10001561c0 con 0x7f10000cfe70 2024-04-25T10:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1000156780 con 0x7f10000cfe70 2024-04-25T10:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100004ef30 con 0x7f10000cfe70 2024-04-25T10:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0ffcc00640 1 -- 172.21.15.87:0/2015709615 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ff00bf030 con 0x7f10000cfe70 2024-04-25T10:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0ffcc00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fdc066f60 0x7f0fdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0ffcc00640 1 -- 172.21.15.87:0/2015709615 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ff011e620 con 0x7f10000cfe70 2024-04-25T10:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.147+0000 7f0ff7400640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fdc066f60 0x7f0fdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.151+0000 7f0ff7400640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fdc066f60 0x7f0fdc069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f10000672c0 tx=0x7f0fe8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.151+0000 7f0ffcc00640 1 -- 172.21.15.87:0/2015709615 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ff00c6020 con 0x7f10000cfe70 2024-04-25T10:15:25.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.339+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f100015b5f0 con 0x7f10000cfe70 2024-04-25T10:15:25.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.339+0000 7f0ffcc00640 1 -- 172.21.15.87:0/2015709615 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0ff0016090 con 0x7f10000cfe70 2024-04-25T10:15:25.346 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:25.346 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:25.346 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:25.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.339+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fdc066f60 msgr2=0x7f0fdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.339+0000 7f1006c00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fdc066f60 0x7f0fdc069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f10000672c0 tx=0x7f0fe8009040 comp rx=0 tx=0).stop 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.339+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10000cfe70 msgr2=0x7f100014af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.339+0000 7f1006c00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10000cfe70 0x7f100014af80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ff001a0d0 tx=0x7f0ff0019240 comp rx=0 tx=0).stop 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 shutdown_connections 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fdc066f60 0x7f0fdc069420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100015b910 0x7f1000155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10001550b0 0x7f1000156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 --2- 172.21.15.87:0/2015709615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10000cfe70 0x7f100014af80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 >> 172.21.15.87:0/2015709615 conn(0x7f1000066b70 msgr2=0x7f100006d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 shutdown_connections 2024-04-25T10:15:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.343+0000 7f1006c00640 1 -- 172.21.15.87:0/2015709615 wait complete. 2024-04-25T10:15:25.368 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:15:25.507 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:25.507 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:25.507 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:25.522 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:25.522 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:15:25.522 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:25.523 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:15:25.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 Processor -- start 2024-04-25T10:15:25.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 -- start start 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0638159540 0x7f06380ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f06380aebd0 0x7f0638071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063815f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0638159920 con 0x7f06380aebd0 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063815c2f0 con 0x7f0638159540 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f0640200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063815c5d0 con 0x7f0638072020 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f06380aebd0 0x7f0638071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f06380aebd0 0x7f0638071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37662/0 (socket says 172.21.15.87:37662) 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 -- 172.21.15.87:0/3334272412 learned_addr learned my addr 172.21.15.87:0/3334272412 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063d800640 1 --2- 172.21.15.87:0/3334272412 >> v2:172.21.15.169:3300/0 conn(0x7f0638159540 0x7f06380ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063e200640 1 --2- 172.21.15.87:0/3334272412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 -- 172.21.15.87:0/3334272412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 msgr2=0x7f063815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 --2- 172.21.15.87:0/3334272412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 -- 172.21.15.87:0/3334272412 >> v2:172.21.15.169:3300/0 conn(0x7f0638159540 msgr2=0x7f06380ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 --2- 172.21.15.87:0/3334272412 >> v2:172.21.15.169:3300/0 conn(0x7f0638159540 0x7f06380ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063ce00640 1 -- 172.21.15.87:0/3334272412 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063815c8b0 con 0x7f06380aebd0 2024-04-25T10:15:25.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.655+0000 7f063d800640 1 --2- 172.21.15.87:0/3334272412 >> v2:172.21.15.169:3300/0 conn(0x7f0638159540 0x7f06380ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:25.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f063ce00640 1 --2- 172.21.15.87:0/3334272412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06380aebd0 0x7f0638071ae0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f06380b4c20 tx=0x7f062401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0b692d2620273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0637e00640 1 -- 172.21.15.87:0/3334272412 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240b0070 con 0x7f06380aebd0 2024-04-25T10:15:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0637e00640 1 -- 172.21.15.87:0/3334272412 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06240ab600 con 0x7f06380aebd0 2024-04-25T10:15:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0637e00640 1 -- 172.21.15.87:0/3334272412 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240ab9d0 con 0x7f06380aebd0 2024-04-25T10:15:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 -- 172.21.15.87:0/3334272412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06380aebd0 msgr2=0x7f0638071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 --2- 172.21.15.87:0/3334272412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06380aebd0 0x7f0638071ae0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f06380b4c20 tx=0x7f062401c260 comp rx=0 tx=0).stop 2024-04-25T10:15:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 -- 172.21.15.87:0/3334272412 shutdown_connections 2024-04-25T10:15:25.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 --2- 172.21.15.87:0/3334272412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06380aebd0 0x7f0638071ae0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 --2- 172.21.15.87:0/3334272412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 --2- 172.21.15.87:0/3334272412 >> v2:172.21.15.169:3300/0 conn(0x7f0638159540 0x7f06380ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 -- 172.21.15.87:0/3334272412 >> 172.21.15.87:0/3334272412 conn(0x7f06380669a0 msgr2=0x7f0638066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 -- 172.21.15.87:0/3334272412 shutdown_connections 2024-04-25T10:15:25.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 -- 172.21.15.87:0/3334272412 wait complete. 2024-04-25T10:15:25.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 Processor -- start 2024-04-25T10:15:25.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.659+0000 7f0640200640 1 -- start start 2024-04-25T10:15:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063814b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06380aebd0 0x7f06381572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0638159540 0x7f06380ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638070350 con 0x7f0638159540 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638072430 con 0x7f06380aebd0 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06380704d0 con 0x7f0638072020 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0638159540 0x7f06380ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063814b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0638159540 0x7f06380ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37670/0 (socket says 172.21.15.87:37670) 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 -- 172.21.15.87:0/4055425389 learned_addr learned my addr 172.21.15.87:0/4055425389 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063ce00640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06380aebd0 0x7f06381572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 -- 172.21.15.87:0/4055425389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 msgr2=0x7f063814b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063814b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 -- 172.21.15.87:0/4055425389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06380aebd0 msgr2=0x7f06381572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06380aebd0 0x7f06381572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 -- 172.21.15.87:0/4055425389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380b2e50 con 0x7f0638159540 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f063e200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0638159540 0x7f06380ad840 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f063000d800 tx=0x7f063000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0635600640 1 -- 172.21.15.87:0/4055425389 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630018020 con 0x7f0638159540 2024-04-25T10:15:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0635600640 1 -- 172.21.15.87:0/4055425389 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06300096c0 con 0x7f0638159540 2024-04-25T10:15:25.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0638151f80 con 0x7f0638159540 2024-04-25T10:15:25.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0635600640 1 -- 172.21.15.87:0/4055425389 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630009ab0 con 0x7f0638159540 2024-04-25T10:15:25.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.663+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06380b30c0 con 0x7f0638159540 2024-04-25T10:15:25.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f0635600640 1 -- 172.21.15.87:0/4055425389 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f063001b020 con 0x7f0638159540 2024-04-25T10:15:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f0635600640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0618066f90 0x7f0618069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f0635600640 1 -- 172.21.15.87:0/4055425389 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06300853b0 con 0x7f0638159540 2024-04-25T10:15:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0618066f90 0x7f0618069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 Processor -- start 2024-04-25T10:15:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 -- start start 2024-04-25T10:15:25.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0618066f90 0x7f0618069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06381570d0 tx=0x7f062c009870 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c53b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8920 0x55a9c536ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c536e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9c51bbf40 con 0x55a9c53b8000 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9c5143fc0 con 0x55a9c536e410 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9c51b7fe0 con 0x55a9c53b8920 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c53b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c53b83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37676/0 (socket says 172.21.15.87:37676) 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 -- 172.21.15.87:0/2380008468 learned_addr learned my addr 172.21.15.87:0/2380008468 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063ce00640 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8920 0x55a9c536ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063e200640 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c536e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 -- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8920 msgr2=0x55a9c536ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8920 0x55a9c536ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 -- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 msgr2=0x55a9c536e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c536e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 -- 172.21.15.87:0/2380008468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9c5203a50 con 0x55a9c53b8000 2024-04-25T10:15:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f063d800640 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c53b83e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f062c015540 tx=0x7f062c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb60ce4eb3cbd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f062a000640 1 -- 172.21.15.87:0/2380008468 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c01c050 con 0x55a9c53b8000 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f062a000640 1 -- 172.21.15.87:0/2380008468 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c026070 con 0x55a9c53b8000 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.667+0000 7f062a000640 1 -- 172.21.15.87:0/2380008468 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c021640 con 0x55a9c53b8000 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/2380008468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 msgr2=0x55a9c53b83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c53b83e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f062c015540 tx=0x7f062c013000 comp rx=0 tx=0).stop 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/2380008468 shutdown_connections 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c536e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8920 0x55a9c536ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- 172.21.15.87:0/2380008468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c53b83e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/2380008468 >> 172.21.15.87:0/2380008468 conn(0x55a9c536a3c0 msgr2=0x55a9c536c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/2380008468 shutdown_connections 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/2380008468 wait complete. 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- start start 2024-04-25T10:15:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 Processor -- start 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c5379e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c537a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8920 0x55a9c537a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9c51c0570 con 0x55a9c53b8920 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9c51765f0 con 0x55a9c536e410 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9c5170d60 con 0x55a9c53b8000 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 10 client.0 _ll_register_callbacks cb 0x55a9c52707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8920 0x55a9c537a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8920 0x55a9c537a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37686/0 (socket says 172.21.15.87:37686) 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 -- 172.21.15.87:0/1672539260 learned_addr learned my addr 172.21.15.87:0/1672539260 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063ce00640 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c537a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063d800640 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c5379e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 -- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 msgr2=0x55a9c537a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c537a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 -- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 msgr2=0x55a9c5379e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c5379e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 -- 172.21.15.87:0/1672539260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9c5144210 con 0x55a9c53b8920 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f063e200640 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8920 0x55a9c537a880 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f063000d7d0 tx=0x7f0630059ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497d60a366b2ebbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f061ea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f061ea00640 1 -- 172.21.15.87:0/1672539260 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300523d0 con 0x55a9c53b8920 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f061ea00640 1 -- 172.21.15.87:0/1672539260 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0630009c50 con 0x55a9c53b8920 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f062c026070 con 0x55a9c53b8920 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f061ea00640 1 -- 172.21.15.87:0/1672539260 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0630052570 con 0x55a9c53b8920 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 10 client.5913 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a9c5240400 con 0x55a9c53b8920 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f061ea00640 1 -- 172.21.15.87:0/1672539260 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c026070 con 0x55a9c53b8920 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f061ea00640 1 -- 172.21.15.87:0/1672539260 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0630013050 con 0x55a9c53b8920 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 10 client.5913 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 10 client.5913 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 20 client.5913 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c537b290 0x55a9c53ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 4 client.5913 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.671+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9c5157c90 con 0x55a9c537b290 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f063d800640 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c537b290 0x55a9c53ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f061ea00640 10 client.5913 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f063d800640 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c537b290 0x55a9c53ca2c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ba472bf047d379 server_cookie=6867bf1144c4a140 in_seq=0 out_seq=0 2024-04-25T10:15:25.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f061ea00640 1 -- 172.21.15.87:0/1672539260 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9c5157c90 con 0x55a9c537b290 2024-04-25T10:15:25.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f061ea00640 10 client.5913 handle_command_reply: tid=0 2024-04-25T10:15:25.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 client.5913 shutdown 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8920 msgr2=0x55a9c537a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8920 0x55a9c537a880 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f063000d7d0 tx=0x7f0630059ec0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 shutdown_connections 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8920 0x55a9c537a880 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c537b290 0x55a9c53ca2c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c537a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 --2- 172.21.15.87:0/1672539260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c536e410 0x55a9c5379e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 >> 172.21.15.87:0/1672539260 conn(0x55a9c536a3c0 msgr2=0x55a9c536d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f061ea00640 0 client.5913 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 shutdown_connections 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 1 -- 172.21.15.87:0/1672539260 wait complete. 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.675+0000 7f06437a7000 20 client.5913 trim_cache size 0 max 16384 2024-04-25T10:15:25.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 Processor -- start 2024-04-25T10:15:25.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- start start 2024-04-25T10:15:25.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c536e410 0x55a9c536d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c536dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638203040 con 0x55a9c53b8000 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06381fafc0 con 0x55a9c53b8920 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638151c50 con 0x55a9c536e410 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c536dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c536dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37692/0 (socket says 172.21.15.87:37692) 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 -- 172.21.15.87:0/363625863 learned_addr learned my addr 172.21.15.87:0/363625863 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063d800640 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c536e410 0x55a9c536d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063e200640 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 -- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c536e410 msgr2=0x55a9c536d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c536e410 0x55a9c536d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 -- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 msgr2=0x55a9c53ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 -- 172.21.15.87:0/363625863 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9c5216c10 con 0x55a9c53b8000 2024-04-25T10:15:25.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f063ce00640 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c536dd00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0624008010 tx=0x7f0624004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fd9405ac0bed2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f0629600640 1 -- 172.21.15.87:0/363625863 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240be070 con 0x55a9c53b8000 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f0629600640 1 -- 172.21.15.87:0/363625863 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06240b1040 con 0x55a9c53b8000 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f0629600640 1 -- 172.21.15.87:0/363625863 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240b0040 con 0x55a9c53b8000 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- 172.21.15.87:0/363625863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 msgr2=0x55a9c536dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c536dd00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0624008010 tx=0x7f0624004740 comp rx=0 tx=0).stop 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- 172.21.15.87:0/363625863 shutdown_connections 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c53b8000 0x55a9c536dd00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 --2- 172.21.15.87:0/363625863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c536e410 0x55a9c536d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- 172.21.15.87:0/363625863 >> 172.21.15.87:0/363625863 conn(0x55a9c536a3c0 msgr2=0x55a9c5379ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- 172.21.15.87:0/363625863 shutdown_connections 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.691+0000 7f06437a7000 1 -- 172.21.15.87:0/363625863 wait complete. 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 -- start start 2024-04-25T10:15:25.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 Processor -- start 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c536e410 0x55a9c5419110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c5419650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9c51f9e80 con 0x55a9c536e410 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06381fa9c0 con 0x55a9c53b8920 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9c5188870 con 0x55a9c53b8000 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 10 client.0 _ll_register_callbacks cb 0x55a9c520e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c536e410 0x55a9c5419110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c536e410 0x55a9c5419110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37696/0 (socket says 172.21.15.87:37696) 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 -- 172.21.15.87:0/1749144316 learned_addr learned my addr 172.21.15.87:0/1749144316 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 -- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 msgr2=0x55a9c5419650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063e200640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063ce00640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c5419650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c5419650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 -- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 msgr2=0x55a9c53d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 -- 172.21.15.87:0/1749144316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9c5150bb0 con 0x55a9c536e410 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063e200640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063ce00640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c5419650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063d800640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c536e410 0x55a9c5419110 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f062c023040 tx=0x7f062c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02eee5b518d2f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f061ea00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f061ea00640 1 -- 172.21.15.87:0/1749144316 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c03d060 con 0x55a9c536e410 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f061ea00640 1 -- 172.21.15.87:0/1749144316 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f062c037870 con 0x55a9c536e410 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06240b1040 con 0x55a9c536e410 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f061ea00640 1 -- 172.21.15.87:0/1749144316 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062c01c040 con 0x55a9c536e410 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 10 client.5919 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:25.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a9c5163030 con 0x55a9c536e410 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f061ea00640 1 -- 172.21.15.87:0/1749144316 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06240b1040 con 0x55a9c536e410 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f061ea00640 1 -- 172.21.15.87:0/1749144316 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f062c0214c0 con 0x55a9c536e410 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 10 client.5919 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 10 client.5919 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 20 client.5919 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c53d4ae0 0x55a9c53d4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 4 client.5919 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55a9c51c9480 con 0x55a9c53d4ae0 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063ce00640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c53d4ae0 0x55a9c53d4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f063ce00640 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c53d4ae0 0x55a9c53d4ef0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63871192e30aba2e server_cookie=2a2632253ab41c3a in_seq=0 out_seq=0 2024-04-25T10:15:25.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.695+0000 7f061ea00640 10 client.5919 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:25.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f061ea00640 1 -- 172.21.15.87:0/1749144316 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a9c51c9480 con 0x55a9c53d4ae0 2024-04-25T10:15:25.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f061ea00640 10 client.5919 handle_command_reply: tid=0 2024-04-25T10:15:25.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 client.5919 shutdown 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c536e410 msgr2=0x55a9c5419110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c536e410 0x55a9c5419110 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f062c023040 tx=0x7f062c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 shutdown_connections 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a9c53b8920 0x55a9c53d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a9c53d4ae0 0x55a9c53d4ef0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a9c53b8000 0x55a9c5419650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 --2- 172.21.15.87:0/1749144316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a9c536e410 0x55a9c5419110 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 >> 172.21.15.87:0/1749144316 conn(0x55a9c536a3c0 msgr2=0x55a9c537a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f061ea00640 0 client.5919 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 shutdown_connections 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 1 -- 172.21.15.87:0/1749144316 wait complete. 2024-04-25T10:15:25.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f06437a7000 20 client.5919 trim_cache size 0 max 16384 2024-04-25T10:15:25.708 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:25.708 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:25.708 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "b48d75cc-87d8-4ef1-b523-7a3190703cef", 2024-04-25T10:15:25.708 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:25.708 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0618066f90 msgr2=0x7f0618069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0618066f90 0x7f0618069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06381570d0 tx=0x7f062c009870 comp rx=0 tx=0).stop 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0638159540 msgr2=0x7f06380ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0638159540 0x7f06380ad840 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f063000d800 tx=0x7f063000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 shutdown_connections 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0618066f90 0x7f0618069450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0638159540 0x7f06380ad840 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06380aebd0 0x7f06381572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 --2- 172.21.15.87:0/4055425389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0638072020 0x7f063814b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 >> 172.21.15.87:0/4055425389 conn(0x7f06380669a0 msgr2=0x7f0638066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 shutdown_connections 2024-04-25T10:15:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.699+0000 7f0640200640 1 -- 172.21.15.87:0/4055425389 wait complete. 2024-04-25T10:15:25.724 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'b48d75cc-87d8-4ef1-b523-7a3190703cef', 'mode': 'asynchronous'}' 2024-04-25T10:15:25.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:25.724 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T10:15:25.736 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T10:15:25.736 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:25.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f379de00640 1 Processor -- start 2024-04-25T10:15:25.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f379de00640 1 -- start start 2024-04-25T10:15:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f379de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f379815b9e0 0x7f37980cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f379de00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37980ccfb0 con 0x7f379815bdc0 2024-04-25T10:15:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f379de00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37980cd290 con 0x7f37980d1c30 2024-04-25T10:15:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f379de00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37980cd570 con 0x7f379815b9e0 2024-04-25T10:15:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3796000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f379815bdc0 0x7f37980d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44132/0 (socket says 172.21.15.87:44132) 2024-04-25T10:15:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3796000640 1 -- 172.21.15.87:0/3476532022 learned_addr learned my addr 172.21.15.87:0/3476532022 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3796a00640 1 --2- 172.21.15.87:0/3476532022 >> v2:172.21.15.169:3300/0 conn(0x7f379815b9e0 0x7f37980cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3795600640 1 -- 172.21.15.87:0/3476532022 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923844322 0 0) 0x7f37980ccfb0 con 0x7f379815bdc0 2024-04-25T10:15:25.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3795600640 1 -- 172.21.15.87:0/3476532022 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f377c003530 con 0x7f379815bdc0 2024-04-25T10:15:25.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3796a00640 1 -- 172.21.15.87:0/3476532022 >> v1:172.21.15.169:6789/0 conn(0x7f37980d1c30 legacy=0x7f37980c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3796a00640 1 -- 172.21.15.87:0/3476532022 >> v1:172.21.15.87:6789/0 conn(0x7f379815bdc0 legacy=0x7f37980d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.907+0000 7f3796a00640 1 -- 172.21.15.87:0/3476532022 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37980cd850 con 0x7f379815b9e0 2024-04-25T10:15:25.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f3796a00640 1 --2- 172.21.15.87:0/3476532022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379815b9e0 0x7f37980cc410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f378801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c39ffc656a7e814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f3795600640 1 -- 172.21.15.87:0/3476532022 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880b2070 con 0x7f379815b9e0 2024-04-25T10:15:25.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f3795600640 1 -- 172.21.15.87:0/3476532022 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37880a57a0 con 0x7f379815b9e0 2024-04-25T10:15:25.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f3795600640 1 -- 172.21.15.87:0/3476532022 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880a5b70 con 0x7f379815b9e0 2024-04-25T10:15:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f379de00640 1 -- 172.21.15.87:0/3476532022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379815b9e0 msgr2=0x7f37980cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f379de00640 1 --2- 172.21.15.87:0/3476532022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379815b9e0 0x7f37980cc410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f378801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f379de00640 1 -- 172.21.15.87:0/3476532022 shutdown_connections 2024-04-25T10:15:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f379de00640 1 --2- 172.21.15.87:0/3476532022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379815b9e0 0x7f37980cc410 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f379de00640 1 -- 172.21.15.87:0/3476532022 >> 172.21.15.87:0/3476532022 conn(0x7f379809a9d0 msgr2=0x7f379809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f379de00640 1 -- 172.21.15.87:0/3476532022 shutdown_connections 2024-04-25T10:15:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.911+0000 7f379de00640 1 -- 172.21.15.87:0/3476532022 wait complete. 2024-04-25T10:15:25.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 Processor -- start 2024-04-25T10:15:25.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 -- start start 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37980d1c30 0x7f379815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379815b9e0 0x7f37981545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379815bdc0 0x7f379815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37980c6d40 con 0x7f379815b9e0 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37980c6bc0 con 0x7f37980d1c30 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3798169840 con 0x7f379815bdc0 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379815b9e0 0x7f37981545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379815b9e0 0x7f37981545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37706/0 (socket says 172.21.15.87:37706) 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 -- 172.21.15.87:0/725515282 learned_addr learned my addr 172.21.15.87:0/725515282 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796a00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37980d1c30 0x7f379815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3797400640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379815bdc0 0x7f379815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 -- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379815bdc0 msgr2=0x7f379815a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379815bdc0 0x7f379815a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 -- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37980d1c30 msgr2=0x7f379815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37980d1c30 0x7f379815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 -- 172.21.15.87:0/725515282 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798154af0 con 0x7f379815b9e0 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f3796000640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379815b9e0 0x7f37981545b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f378400c4d0 tx=0x7f378400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f378f400640 1 -- 172.21.15.87:0/725515282 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784017070 con 0x7f379815b9e0 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f378f400640 1 -- 172.21.15.87:0/725515282 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3784005cf0 con 0x7f379815b9e0 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f378f400640 1 -- 172.21.15.87:0/725515282 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784007270 con 0x7f379815b9e0 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379815e230 con 0x7f379815b9e0 2024-04-25T10:15:25.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.915+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379815e7f0 con 0x7f379815b9e0 2024-04-25T10:15:25.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.919+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379804ef30 con 0x7f379815b9e0 2024-04-25T10:15:25.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.919+0000 7f378f400640 1 -- 172.21.15.87:0/725515282 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f378401a070 con 0x7f379815b9e0 2024-04-25T10:15:25.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.919+0000 7f378f400640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3778066f60 0x7f3778069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:25.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.919+0000 7f378f400640 1 -- 172.21.15.87:0/725515282 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37840842d0 con 0x7f379815b9e0 2024-04-25T10:15:25.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.919+0000 7f3796a00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3778066f60 0x7f3778069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:25.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.919+0000 7f3796a00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3778066f60 0x7f3778069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f37880af000 tx=0x7f37880a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:25.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:25.923+0000 7f378f400640 1 -- 172.21.15.87:0/725515282 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3784054c80 con 0x7f379815b9e0 2024-04-25T10:15:26.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.107+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f379806e6c0 con 0x7f379815b9e0 2024-04-25T10:15:26.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.107+0000 7f378f400640 1 -- 172.21.15.87:0/725515282 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3784058d20 con 0x7f379815b9e0 2024-04-25T10:15:26.114 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:26.114 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:26.115 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:26.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3778066f60 msgr2=0x7f3778069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3778066f60 0x7f3778069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f37880af000 tx=0x7f37880a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:26.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379815b9e0 msgr2=0x7f37981545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379815b9e0 0x7f37981545b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f378400c4d0 tx=0x7f378400c990 comp rx=0 tx=0).stop 2024-04-25T10:15:26.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 shutdown_connections 2024-04-25T10:15:26.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3778066f60 0x7f3778069420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379815bdc0 0x7f379815a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379815b9e0 0x7f37981545b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 --2- 172.21.15.87:0/725515282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37980d1c30 0x7f379815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 >> 172.21.15.87:0/725515282 conn(0x7f379809a9d0 msgr2=0x7f379806eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:26.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 shutdown_connections 2024-04-25T10:15:26.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.111+0000 7f379de00640 1 -- 172.21.15.87:0/725515282 wait complete. 2024-04-25T10:15:26.136 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T10:15:26.283 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:26.283 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:26.283 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:26.298 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:26.298 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:26.298 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:26.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:26.307 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:15:26.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:15:26.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:15:26.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206994 s, 50.7 MB/s 2024-04-25T10:15:26.381 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:15:26.381 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:26.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.551+0000 7f7512c00640 1 Processor -- start 2024-04-25T10:15:26.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512c00640 1 -- start start 2024-04-25T10:15:26.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f750c0b2390 0x7f750c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:26.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750c0cf620 con 0x7f750c15a870 2024-04-25T10:15:26.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750c0cf900 con 0x7f750c0b2ff0 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750c0cf3e0 con 0x7f750c0b2390 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f750c0b2390 0x7f750c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7511800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f750c15a870 0x7f750c0b28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44136/0 (socket says 172.21.15.87:44136) 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7511800640 1 -- 172.21.15.87:0/3840189156 learned_addr learned my addr 172.21.15.87:0/3840189156 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512200640 1 -- 172.21.15.87:0/3840189156 >> v1:172.21.15.169:6789/0 conn(0x7f750c0b2ff0 legacy=0x7f750c06daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512200640 1 -- 172.21.15.87:0/3840189156 >> v1:172.21.15.87:6789/0 conn(0x7f750c15a870 legacy=0x7f750c0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f7512200640 1 -- 172.21.15.87:0/3840189156 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c0cfbe0 con 0x7f750c0b2390 2024-04-25T10:15:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.555+0000 7f750be00640 1 -- 172.21.15.87:0/3840189156 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182981666 0 0) 0x7f750c0cf620 con 0x7f750c15a870 2024-04-25T10:15:26.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512200640 1 --2- 172.21.15.87:0/3840189156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c0d00e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7504009dc0 tx=0x7f750401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120f9674b27f6f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:26.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f750be00640 1 -- 172.21.15.87:0/3840189156 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040b2070 con 0x7f750c0b2390 2024-04-25T10:15:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f750be00640 1 -- 172.21.15.87:0/3840189156 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75040a5850 con 0x7f750c0b2390 2024-04-25T10:15:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f750be00640 1 -- 172.21.15.87:0/3840189156 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040a5c20 con 0x7f750c0b2390 2024-04-25T10:15:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 -- 172.21.15.87:0/3840189156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 msgr2=0x7f750c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 --2- 172.21.15.87:0/3840189156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c0d00e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7504009dc0 tx=0x7f750401c880 comp rx=0 tx=0).stop 2024-04-25T10:15:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 -- 172.21.15.87:0/3840189156 shutdown_connections 2024-04-25T10:15:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 --2- 172.21.15.87:0/3840189156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c0d00e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 -- 172.21.15.87:0/3840189156 >> 172.21.15.87:0/3840189156 conn(0x7f750c074310 msgr2=0x7f750c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 -- 172.21.15.87:0/3840189156 shutdown_connections 2024-04-25T10:15:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 -- 172.21.15.87:0/3840189156 wait complete. 2024-04-25T10:15:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 Processor -- start 2024-04-25T10:15:26.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.559+0000 7f7512c00640 1 -- start start 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f750c0b2ff0 0x7f750c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f750c15a870 0x7f750c14d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c168450 con 0x7f750c0b2390 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c06ec50 con 0x7f750c15a870 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1685d0 con 0x7f750c0b2ff0 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37732/0 (socket says 172.21.15.87:37732) 2024-04-25T10:15:26.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 -- 172.21.15.87:0/2619963253 learned_addr learned my addr 172.21.15.87:0/2619963253 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512200640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f750c15a870 0x7f750c14d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7510e00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f750c0b2ff0 0x7f750c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 -- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f750c0b2ff0 msgr2=0x7f750c14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f750c0b2ff0 0x7f750c14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 -- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f750c15a870 msgr2=0x7f750c14d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f750c15a870 0x7f750c14d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 -- 172.21.15.87:0/2619963253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c14f9e0 con 0x7f750c0b2390 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7511800640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c155ec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f750000c520 tx=0x7f750000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7509600640 1 -- 172.21.15.87:0/2619963253 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500018070 con 0x7f750c0b2390 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7509600640 1 -- 172.21.15.87:0/2619963253 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75000074e0 con 0x7f750c0b2390 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c14fca0 con 0x7f750c0b2390 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7509600640 1 -- 172.21.15.87:0/2619963253 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75000049f0 con 0x7f750c0b2390 2024-04-25T10:15:26.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c150540 con 0x7f750c0b2390 2024-04-25T10:15:26.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750c06bc20 con 0x7f750c0b2390 2024-04-25T10:15:26.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.563+0000 7f7509600640 1 -- 172.21.15.87:0/2619963253 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f750001b070 con 0x7f750c0b2390 2024-04-25T10:15:26.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.567+0000 7f7509600640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74ec066f60 0x7f74ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:26.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.567+0000 7f7509600640 1 -- 172.21.15.87:0/2619963253 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7500084570 con 0x7f750c0b2390 2024-04-25T10:15:26.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.567+0000 7f7510e00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74ec066f60 0x7f74ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:26.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.567+0000 7f7510e00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74ec066f60 0x7f74ec069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f750c14a000 tx=0x7f74f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:26.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.571+0000 7f7509600640 1 -- 172.21.15.87:0/2619963253 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75000554e0 con 0x7f750c0b2390 2024-04-25T10:15:26.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.751+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f750c06dd20 con 0x7f750c0b2390 2024-04-25T10:15:26.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7509600640 1 -- 172.21.15.87:0/2619963253 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f7500082e20 con 0x7f750c0b2390 2024-04-25T10:15:26.760 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:26.760 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:26.761 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:26.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74ec066f60 msgr2=0x7f74ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74ec066f60 0x7f74ec069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f750c14a000 tx=0x7f74f8009040 comp rx=0 tx=0).stop 2024-04-25T10:15:26.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 msgr2=0x7f750c155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c155ec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f750000c520 tx=0x7f750000c9e0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 shutdown_connections 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74ec066f60 0x7f74ec069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f750c15a870 0x7f750c14d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f750c0b2ff0 0x7f750c14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 --2- 172.21.15.87:0/2619963253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f750c0b2390 0x7f750c155ec0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 >> 172.21.15.87:0/2619963253 conn(0x7f750c074310 msgr2=0x7f750c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 shutdown_connections 2024-04-25T10:15:26.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:26.755+0000 7f7512c00640 1 -- 172.21.15.87:0/2619963253 wait complete. 2024-04-25T10:15:26.781 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:15:26.926 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:26.926 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:26.926 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:26.941 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:26.941 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:15:26.942 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:15:26.942 DEBUG:teuthology.orchestra.run.smithi087:> import stat 2024-04-25T10:15:26.942 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:15:26.942 DEBUG:teuthology.orchestra.run.smithi087:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T10:15:26.942 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:15:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:1099511630187 2024-04-25T10:15:27.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:15:27.015 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:27.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:15:27.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.191+0000 7f7d95800640 1 Processor -- start 2024-04-25T10:15:27.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.191+0000 7f7d95800640 1 -- start start 2024-04-25T10:15:27.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d95800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7d900aeda0 0x7f7d90071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d95800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d95800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d90159920 con 0x7f7d90159540 2024-04-25T10:15:27.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d95800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9015c2f0 con 0x7f7d900aeda0 2024-04-25T10:15:27.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d95800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9015c5d0 con 0x7f7d90072020 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d8ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7d90159540 0x7f7d900ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44144/0 (socket says 172.21.15.87:44144) 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d8ea00640 1 -- 172.21.15.87:0/4136489536 learned_addr learned my addr 172.21.15.87:0/4136489536 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d86000640 1 --2- 172.21.15.87:0/4136489536 >> v2:172.21.15.87:3300/0 conn(0x7f7d900aeda0 0x7f7d90071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d8f400640 1 --2- 172.21.15.87:0/4136489536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d8e000640 1 -- 172.21.15.87:0/4136489536 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790194845 0 0) 0x7f7d90159920 con 0x7f7d90159540 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d8e000640 1 -- 172.21.15.87:0/4136489536 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d74003530 con 0x7f7d90159540 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d86000640 1 -- 172.21.15.87:0/4136489536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 msgr2=0x7f7d90070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d86000640 1 --2- 172.21.15.87:0/4136489536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d86000640 1 -- 172.21.15.87:0/4136489536 >> v1:172.21.15.87:6789/0 conn(0x7f7d90159540 legacy=0x7f7d900ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d86000640 1 -- 172.21.15.87:0/4136489536 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9015c8b0 con 0x7f7d900aeda0 2024-04-25T10:15:27.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.195+0000 7f7d8f400640 1 --2- 172.21.15.87:0/4136489536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d86000640 1 --2- 172.21.15.87:0/4136489536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d900aeda0 0x7f7d90071ae0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7d780097c0 tx=0x7f7d7801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79ac112a38c068d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d8e000640 1 -- 172.21.15.87:0/4136489536 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780aa5f0 con 0x7f7d900aeda0 2024-04-25T10:15:27.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d8e000640 1 -- 172.21.15.87:0/4136489536 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d780af070 con 0x7f7d900aeda0 2024-04-25T10:15:27.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d8e000640 1 -- 172.21.15.87:0/4136489536 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780aa9b0 con 0x7f7d900aeda0 2024-04-25T10:15:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 -- 172.21.15.87:0/4136489536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d900aeda0 msgr2=0x7f7d90071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 --2- 172.21.15.87:0/4136489536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d900aeda0 0x7f7d90071ae0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7d780097c0 tx=0x7f7d7801c470 comp rx=0 tx=0).stop 2024-04-25T10:15:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 -- 172.21.15.87:0/4136489536 shutdown_connections 2024-04-25T10:15:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 --2- 172.21.15.87:0/4136489536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 --2- 172.21.15.87:0/4136489536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d900aeda0 0x7f7d90071ae0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 -- 172.21.15.87:0/4136489536 >> 172.21.15.87:0/4136489536 conn(0x7f7d900669a0 msgr2=0x7f7d90066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 -- 172.21.15.87:0/4136489536 shutdown_connections 2024-04-25T10:15:27.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 -- 172.21.15.87:0/4136489536 wait complete. 2024-04-25T10:15:27.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 Processor -- start 2024-04-25T10:15:27.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 -- start start 2024-04-25T10:15:27.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.199+0000 7f7d95800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900aeda0 0x7f7d90156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d95800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d90159540 0x7f7d90152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d95800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900710a0 con 0x7f7d90159540 2024-04-25T10:15:27.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d95800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90072430 con 0x7f7d900aeda0 2024-04-25T10:15:27.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d95800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90071220 con 0x7f7d90072020 2024-04-25T10:15:27.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d90159540 0x7f7d90152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d90159540 0x7f7d90152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37750/0 (socket says 172.21.15.87:37750) 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 -- 172.21.15.87:0/202835966 learned_addr learned my addr 172.21.15.87:0/202835966 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d86000640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900aeda0 0x7f7d90156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 -- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 msgr2=0x7f7d90154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 -- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900aeda0 msgr2=0x7f7d90156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900aeda0 0x7f7d90156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 -- 172.21.15.87:0/202835966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90154bb0 con 0x7f7d90159540 2024-04-25T10:15:27.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d8f400640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d90159540 0x7f7d90152ae0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009fa0 tx=0x7f7d8800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d87400640 1 -- 172.21.15.87:0/202835966 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8801a070 con 0x7f7d90159540 2024-04-25T10:15:27.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d87400640 1 -- 172.21.15.87:0/202835966 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d88003bc0 con 0x7f7d90159540 2024-04-25T10:15:27.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d87400640 1 -- 172.21.15.87:0/202835966 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88004030 con 0x7f7d90159540 2024-04-25T10:15:27.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d90153630 con 0x7f7d90159540 2024-04-25T10:15:27.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d90153e70 con 0x7f7d90159540 2024-04-25T10:15:27.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.203+0000 7f7d87400640 1 -- 172.21.15.87:0/202835966 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d8801d070 con 0x7f7d90159540 2024-04-25T10:15:27.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d87400640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d6c066f60 0x7f7d6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d87400640 1 -- 172.21.15.87:0/202835966 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d880852a0 con 0x7f7d90159540 2024-04-25T10:15:27.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d6c066f60 0x7f7d6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 Processor -- start 2024-04-25T10:15:27.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- start start 2024-04-25T10:15:27.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d6c066f60 0x7f7d6c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d80007fd0 tx=0x7f7d80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4955560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c490b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c490b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c4758f20 con 0x55a5c4955180 2024-04-25T10:15:27.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c46e1000 con 0x55a5c490b600 2024-04-25T10:15:27.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5c4754fc0 con 0x55a5c4955aa0 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4955560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4955560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37764/0 (socket says 172.21.15.87:37764) 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 -- 172.21.15.87:0/752939239 learned_addr learned my addr 172.21.15.87:0/752939239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d86000640 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c490b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8f400640 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c490b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 -- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 msgr2=0x55a5c490b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c490b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 -- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 msgr2=0x55a5c490b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c490b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 -- 172.21.15.87:0/752939239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5c4734ba0 con 0x55a5c4955180 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8f400640 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c490b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4955560 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000fa30 tx=0x7f7d80000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166fbc57a9add21b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d7f400640 1 -- 172.21.15.87:0/752939239 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80025070 con 0x55a5c4955180 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d7f400640 1 -- 172.21.15.87:0/752939239 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d80015bf0 con 0x55a5c4955180 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d7f400640 1 -- 172.21.15.87:0/752939239 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80020400 con 0x55a5c4955180 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- 172.21.15.87:0/752939239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 msgr2=0x55a5c4955560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4955560 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000fa30 tx=0x7f7d80000ee0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- 172.21.15.87:0/752939239 shutdown_connections 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c490b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c490b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/752939239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4955560 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- 172.21.15.87:0/752939239 >> 172.21.15.87:0/752939239 conn(0x55a5c4911760 msgr2=0x55a5c49099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- 172.21.15.87:0/752939239 shutdown_connections 2024-04-25T10:15:27.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.207+0000 7f7d98bc8000 1 -- 172.21.15.87:0/752939239 wait complete. 2024-04-25T10:15:27.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 -- start start 2024-04-25T10:15:27.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 Processor -- start 2024-04-25T10:15:27.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c49106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4910c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c4943450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c48169c0 con 0x55a5c4955180 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c47135d0 con 0x55a5c490b600 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5c480e9d0 con 0x55a5c4955aa0 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 10 client.0 _ll_register_callbacks cb 0x55a5c48f6a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d86000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4910c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c49106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c49106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:58668/0 (socket says 172.21.15.87:58668) 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 -- 172.21.15.87:0/977124271 learned_addr learned my addr 172.21.15.87:0/977124271 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8f400640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c4943450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 -- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 msgr2=0x55a5c4943450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c4943450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 -- 172.21.15.87:0/977124271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 msgr2=0x55a5c4910c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4910c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 -- 172.21.15.87:0/977124271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5c479f5a0 con 0x55a5c490b600 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8f400640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c4943450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d86000640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4910c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c49106e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d80000f20 tx=0x7f7d800010b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aab26e92a64d422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d73e00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d73e00640 1 -- 172.21.15.87:0/977124271 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80025050 con 0x55a5c490b600 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d73e00640 1 -- 172.21.15.87:0/977124271 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d800047f0 con 0x55a5c490b600 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d80015bf0 con 0x55a5c490b600 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d73e00640 1 -- 172.21.15.87:0/977124271 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d80037050 con 0x55a5c490b600 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 10 client.5608 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a5c47302b0 con 0x55a5c490b600 2024-04-25T10:15:27.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d73e00640 1 -- 172.21.15.87:0/977124271 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d80015bf0 con 0x55a5c490b600 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d73e00640 1 -- 172.21.15.87:0/977124271 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7d8003a020 con 0x55a5c490b600 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 10 client.5608 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 10 client.5608 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 20 client.5608 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c4943e60 0x55a5c4967440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 4 client.5608 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5c46f4c70 con 0x55a5c4943e60 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.211+0000 7f7d86000640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c4943e60 0x55a5c4967440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d86000640 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c4943e60 0x55a5c4967440 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db905397d28263a5 server_cookie=dd152b523d4d309d in_seq=0 out_seq=0 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d73e00640 10 client.5608 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d73e00640 1 -- 172.21.15.87:0/977124271 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5c46f4c70 con 0x55a5c4943e60 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d73e00640 10 client.5608 handle_command_reply: tid=0 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 client.5608 shutdown 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 msgr2=0x55a5c49106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c49106e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d80000f20 tx=0x7f7d800010b0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 shutdown_connections 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c4943e60 0x55a5c4967440 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955aa0 0x55a5c4943450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955180 0x55a5c4910c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/977124271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c49106e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d73e00640 0 client.5608 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 >> 172.21.15.87:0/977124271 conn(0x55a5c4911760 msgr2=0x55a5c490a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 shutdown_connections 2024-04-25T10:15:27.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 1 -- 172.21.15.87:0/977124271 wait complete. 2024-04-25T10:15:27.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.215+0000 7f7d98bc8000 20 client.5608 trim_cache size 0 max 16384 2024-04-25T10:15:27.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 Processor -- start 2024-04-25T10:15:27.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 -- start start 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c490b600 0x55a5c49625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c4962b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c4955aa0 0x55a5c490a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90202920 con 0x55a5c490b600 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9014e1e0 con 0x55a5c4955aa0 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9014d5e0 con 0x55a5c4955180 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c490b600 0x55a5c49625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c490b600 0x55a5c49625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37786/0 (socket says 172.21.15.87:37786) 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 -- 172.21.15.87:0/2747624002 learned_addr learned my addr 172.21.15.87:0/2747624002 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d86000640 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c4962b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8f400640 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c4955aa0 0x55a5c490a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 -- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 msgr2=0x55a5c4962b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c4962b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 -- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c4955aa0 msgr2=0x55a5c490a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c4955aa0 0x55a5c490a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 -- 172.21.15.87:0/2747624002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5c46e1250 con 0x55a5c490b600 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8f400640 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c4955aa0 0x55a5c490a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c490b600 0x55a5c49625f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000fa30 tx=0x7f7d8001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce08e5401bee206 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d7ea00640 1 -- 172.21.15.87:0/2747624002 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80025070 con 0x55a5c490b600 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d7ea00640 1 -- 172.21.15.87:0/2747624002 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8002f070 con 0x55a5c490b600 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.227+0000 7f7d7ea00640 1 -- 172.21.15.87:0/2747624002 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80015bf0 con 0x55a5c490b600 2024-04-25T10:15:27.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/2747624002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c490b600 msgr2=0x55a5c49625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c490b600 0x55a5c49625f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000fa30 tx=0x7f7d8001d000 comp rx=0 tx=0).stop 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/2747624002 shutdown_connections 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c4955aa0 0x55a5c490a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c4962b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/2747624002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c490b600 0x55a5c49625f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/2747624002 >> 172.21.15.87:0/2747624002 conn(0x55a5c4911760 msgr2=0x55a5c4961ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/2747624002 shutdown_connections 2024-04-25T10:15:27.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/2747624002 wait complete. 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- start start 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 Processor -- start 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c4967fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c49684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955aa0 0x55a5c4968a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c471c7b0 con 0x55a5c4955aa0 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9014dbe0 con 0x55a5c490b600 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5c47042f0 con 0x55a5c4955180 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 10 client.0 _ll_register_callbacks cb 0x55a5c48d7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955aa0 0x55a5c4968a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c4967fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955aa0 0x55a5c4968a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37788/0 (socket says 172.21.15.87:37788) 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 -- 172.21.15.87:0/75592833 learned_addr learned my addr 172.21.15.87:0/75592833 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d86000640 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c49684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 -- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 msgr2=0x55a5c49684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c49684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 -- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 msgr2=0x55a5c4967fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c4967fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 -- 172.21.15.87:0/75592833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5c46edb90 con 0x55a5c4955aa0 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c4967fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d8f400640 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955aa0 0x55a5c4968a20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7d880520b0 tx=0x7f7d88051fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e206c740cdef5420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d73e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d73e00640 1 -- 172.21.15.87:0/75592833 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8805e2b0 con 0x55a5c4955aa0 2024-04-25T10:15:27.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d73e00640 1 -- 172.21.15.87:0/75592833 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d8805de20 con 0x55a5c4955aa0 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d8002f070 con 0x55a5c4955aa0 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d73e00640 1 -- 172.21.15.87:0/75592833 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d8805e020 con 0x55a5c4955aa0 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 10 client.5955 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a5c4700010 con 0x55a5c4955aa0 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d73e00640 1 -- 172.21.15.87:0/75592833 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8002f070 con 0x55a5c4955aa0 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d73e00640 1 -- 172.21.15.87:0/75592833 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7d88069b60 con 0x55a5c4955aa0 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 10 client.5955 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 10 client.5955 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 20 client.5955 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c49bb320 0x55a5c49bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 4 client.5955 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:15:27.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.231+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a5c4766460 con 0x55a5c49bb320 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c49bb320 0x55a5c49bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d8ea00640 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c49bb320 0x55a5c49bd710 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f5f6198a0af9c2 server_cookie=a6442d0072befb62 in_seq=0 out_seq=0 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d73e00640 10 client.5955 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d73e00640 1 -- 172.21.15.87:0/75592833 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a5c4766460 con 0x55a5c49bb320 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d73e00640 10 client.5955 handle_command_reply: tid=0 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 client.5955 shutdown 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955aa0 msgr2=0x55a5c4968a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955aa0 0x55a5c4968a20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7d880520b0 tx=0x7f7d88051fb0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 shutdown_connections 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5c4955aa0 0x55a5c4968a20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55a5c49bb320 0x55a5c49bd710 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a5c4955180 0x55a5c49684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 --2- 172.21.15.87:0/75592833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a5c490b600 0x55a5c4967fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d73e00640 0 client.5955 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 >> 172.21.15.87:0/75592833 conn(0x55a5c4911760 msgr2=0x55a5c49549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 shutdown_connections 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 1 -- 172.21.15.87:0/75592833 wait complete. 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d98bc8000 20 client.5955 trim_cache size 0 max 16384 2024-04-25T10:15:27.247 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-04-25T10:15:27.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d6c066f60 msgr2=0x7f7d6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d6c066f60 0x7f7d6c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d80007fd0 tx=0x7f7d80009040 comp rx=0 tx=0).stop 2024-04-25T10:15:27.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d90159540 msgr2=0x7f7d90152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d90159540 0x7f7d90152ae0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009fa0 tx=0x7f7d8800d560 comp rx=0 tx=0).stop 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 shutdown_connections 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d6c066f60 0x7f7d6c069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d90159540 0x7f7d90152ae0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900aeda0 0x7f7d90156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 --2- 172.21.15.87:0/202835966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d90072020 0x7f7d90154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 >> 172.21.15.87:0/202835966 conn(0x7f7d900669a0 msgr2=0x7f7d9015d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 shutdown_connections 2024-04-25T10:15:27.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.235+0000 7f7d95800640 1 -- 172.21.15.87:0/202835966 wait complete. 2024-04-25T10:15:27.260 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096b.00000000 parent 2024-04-25T10:15:27.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 Processor -- start 2024-04-25T10:15:27.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 -- start start 2024-04-25T10:15:27.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf4b30 0x560f4ccdd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf79a0 0x560f4ccaba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf7d80 0x560f4cd7eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4ccaa830 con 0x560f4ccf79a0 2024-04-25T10:15:27.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4cc509f0 con 0x560f4ccf4b30 2024-04-25T10:15:27.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e9649080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f4ccaa9b0 con 0x560f4ccf7d80 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e5200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf79a0 0x560f4ccaba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e5200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf79a0 0x560f4ccaba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37798/0 (socket says 172.21.15.87:37798) 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e5200640 1 -- 172.21.15.87:0/570370852 learned_addr learned my addr 172.21.15.87:0/570370852 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e5c00640 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf4b30 0x560f4ccdd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e6600640 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf7d80 0x560f4cd7eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.299+0000 7f88e5200640 1 -- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf7d80 msgr2=0x560f4cd7eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5200640 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf7d80 0x560f4cd7eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5200640 1 -- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf4b30 msgr2=0x560f4ccdd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5200640 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf4b30 0x560f4ccdd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5200640 1 -- 172.21.15.87:0/570370852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f4cd7e050 con 0x560f4ccf79a0 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5200640 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf79a0 0x560f4ccaba90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009a10 tx=0x7f88d4005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde41a909d8bb77b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88df400640 1 -- 172.21.15.87:0/570370852 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4008e60 con 0x560f4ccf79a0 2024-04-25T10:15:27.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88df400640 1 -- 172.21.15.87:0/570370852 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4018070 con 0x560f4ccf79a0 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88df400640 1 -- 172.21.15.87:0/570370852 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4013620 con 0x560f4ccf79a0 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- 172.21.15.87:0/570370852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf79a0 msgr2=0x560f4ccaba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf79a0 0x560f4ccaba90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009a10 tx=0x7f88d4005a10 comp rx=0 tx=0).stop 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- 172.21.15.87:0/570370852 shutdown_connections 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf7d80 0x560f4cd7eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf79a0 0x560f4ccaba90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 --2- 172.21.15.87:0/570370852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf4b30 0x560f4ccdd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- 172.21.15.87:0/570370852 >> 172.21.15.87:0/570370852 conn(0x560f4ccd8340 msgr2=0x560f4cd804f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- 172.21.15.87:0/570370852 shutdown_connections 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- 172.21.15.87:0/570370852 wait complete. 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 Processor -- start 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- start start 2024-04-25T10:15:27.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf4b30 0x560f4ce13180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf79a0 0x560f4cd71da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf7d80 0x560f4ce14900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4cd7e590 con 0x560f4ccf4b30 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4cd7e2f0 con 0x560f4ccf7d80 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e9649080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f4cd7e830 con 0x560f4ccf79a0 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf4b30 0x560f4ce13180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf4b30 0x560f4ce13180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37810/0 (socket says 172.21.15.87:37810) 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 -- 172.21.15.87:0/1327336024 learned_addr learned my addr 172.21.15.87:0/1327336024 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e6600640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf7d80 0x560f4ce14900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5200640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf79a0 0x560f4cd71da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 -- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf79a0 msgr2=0x560f4cd71da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf79a0 0x560f4cd71da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 -- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf7d80 msgr2=0x560f4ce14900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf7d80 0x560f4ce14900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 -- 172.21.15.87:0/1327336024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f4ce139e0 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e6600640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f4ccf7d80 0x560f4ce14900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5200640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f4ccf79a0 0x560f4cd71da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.303+0000 7f88e5c00640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf4b30 0x560f4ce13180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f88d80ad040 tx=0x7f88d8018150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88de000640 1 -- 172.21.15.87:0/1327336024 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80b9070 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88de000640 1 -- 172.21.15.87:0/1327336024 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800a030 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88de000640 1 -- 172.21.15.87:0/1327336024 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80af040 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f4ce13c80 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f4ce141c0 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88de000640 1 -- 172.21.15.87:0/1327336024 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88d80bc070 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88de000640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f88c8065940 0x7f88c8067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88de000640 1 -- 172.21.15.87:0/1327336024 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88d811e8e0 con 0x560f4ccf4b30 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88e5200640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f88c8065940 0x7f88c8067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88e9649080 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x560f4cd722e0 0x560f4cd76590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 --> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] -- osd_op(unknown.0.0:1 7.a 7:51f47cfd:::1000000096b.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v9 -- 0x560f4ce15350 con 0x560f4cd722e0 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88e6600640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x560f4cd722e0 0x560f4cd76590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.307+0000 7f88e5200640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f88c8065940 0x7f88c8067e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f88d4005a50 tx=0x7f88d400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e6600640 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x560f4cd722e0 0x560f4cd76590 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e6600640 1 -- 172.21.15.87:0/1327336024 <== osd.3 v2:172.21.15.87:6802/327355267 1 ==== osd_op_reply(1 1000000096b.00000000 [rmxattr] v46'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f88e00015a0 con 0x560f4cd722e0 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x560f4cd722e0 msgr2=0x560f4cd76590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x560f4cd722e0 0x560f4cd76590 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f88c8065940 msgr2=0x7f88c8067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f88c8065940 0x7f88c8067e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f88d4005a50 tx=0x7f88d400b040 comp rx=0 tx=0).stop 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf4b30 msgr2=0x560f4ce13180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 --2- 172.21.15.87:0/1327336024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560f4ccf4b30 0x560f4ce13180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f88d80ad040 tx=0x7f88d8018150 comp rx=0 tx=0).stop 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e6600640 1 -- 172.21.15.87:0/1327336024 reap_dead start 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 shutdown_connections 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 >> 172.21.15.87:0/1327336024 conn(0x560f4ccd8340 msgr2=0x560f4ccdc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 shutdown_connections 2024-04-25T10:15:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.311+0000 7f88e9649080 1 -- 172.21.15.87:0/1327336024 wait complete. 2024-04-25T10:15:27.358 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:15:27.358 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:27.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:15:27.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c61e00640 1 Processor -- start 2024-04-25T10:15:27.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c61e00640 1 -- start start 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c61e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2c5c159540 0x7f2c5c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c61e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2c5c0aebd0 0x7f2c5c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c61e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c071360 con 0x7f2c5c072020 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c61e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c159920 con 0x7f2c5c159540 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c61e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c15c2f0 con 0x7f2c5c0aebd0 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2c5c072020 0x7f2c5c15f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50644/0 (socket says 172.21.15.87:50644) 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5b400640 1 -- 172.21.15.87:0/3026531901 learned_addr learned my addr 172.21.15.87:0/3026531901 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3026531901 >> v2:172.21.15.87:3300/0 conn(0x7f2c5c159540 0x7f2c5c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5a000640 1 --2- 172.21.15.87:0/3026531901 >> v2:172.21.15.169:3300/0 conn(0x7f2c5c0aebd0 0x7f2c5c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5aa00640 1 -- 172.21.15.87:0/3026531901 >> v2:172.21.15.169:3300/0 conn(0x7f2c5c0aebd0 msgr2=0x7f2c5c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3026531901 >> v2:172.21.15.169:3300/0 conn(0x7f2c5c0aebd0 0x7f2c5c071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5aa00640 1 -- 172.21.15.87:0/3026531901 >> v1:172.21.15.87:6789/0 conn(0x7f2c5c072020 legacy=0x7f2c5c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.547+0000 7f2c5aa00640 1 -- 172.21.15.87:0/3026531901 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c15c5d0 con 0x7f2c5c159540 2024-04-25T10:15:27.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3026531901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c159540 0x7f2c5c0ae690 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0cd4e0 tx=0x7f2c4c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb43ffb26db866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c59600640 1 -- 172.21.15.87:0/3026531901 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0b0070 con 0x7f2c5c159540 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c59600640 1 -- 172.21.15.87:0/3026531901 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0ab600 con 0x7f2c5c159540 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c59600640 1 -- 172.21.15.87:0/3026531901 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0ab9d0 con 0x7f2c5c159540 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 -- 172.21.15.87:0/3026531901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c159540 msgr2=0x7f2c5c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 --2- 172.21.15.87:0/3026531901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c159540 0x7f2c5c0ae690 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0cd4e0 tx=0x7f2c4c01c480 comp rx=0 tx=0).stop 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 -- 172.21.15.87:0/3026531901 shutdown_connections 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 --2- 172.21.15.87:0/3026531901 >> v2:172.21.15.169:3300/0 conn(0x7f2c5c0aebd0 0x7f2c5c071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 --2- 172.21.15.87:0/3026531901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c159540 0x7f2c5c0ae690 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 -- 172.21.15.87:0/3026531901 >> 172.21.15.87:0/3026531901 conn(0x7f2c5c0669a0 msgr2=0x7f2c5c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 -- 172.21.15.87:0/3026531901 shutdown_connections 2024-04-25T10:15:27.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.551+0000 7f2c61e00640 1 -- 172.21.15.87:0/3026531901 wait complete. 2024-04-25T10:15:27.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 Processor -- start 2024-04-25T10:15:27.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 -- start start 2024-04-25T10:15:27.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c072020 0x7f2c5c149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c5c0aebd0 0x7f2c5c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c5c159540 0x7f2c5c0b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c072430 con 0x7f2c5c072020 2024-04-25T10:15:27.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c06c0d0 con 0x7f2c5c0aebd0 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c61e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c168930 con 0x7f2c5c159540 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c072020 0x7f2c5c149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c072020 0x7f2c5c149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59024/0 (socket says 172.21.15.87:59024) 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 -- 172.21.15.87:0/2408161413 learned_addr learned my addr 172.21.15.87:0/2408161413 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c5c0aebd0 0x7f2c5c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5b400640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c5c159540 0x7f2c5c0b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 -- 172.21.15.87:0/2408161413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c5c159540 msgr2=0x7f2c5c0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c5c159540 0x7f2c5c0b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 -- 172.21.15.87:0/2408161413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c5c0aebd0 msgr2=0x7f2c5c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c5c0aebd0 0x7f2c5c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.555+0000 7f2c5aa00640 1 -- 172.21.15.87:0/2408161413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c0b3cb0 con 0x7f2c5c072020 2024-04-25T10:15:27.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.559+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c072020 0x7f2c5c149060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0ae040 tx=0x7f2c4c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.559+0000 7f2c53400640 1 -- 172.21.15.87:0/2408161413 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0b8020 con 0x7f2c5c072020 2024-04-25T10:15:27.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.559+0000 7f2c53400640 1 -- 172.21.15.87:0/2408161413 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0b4410 con 0x7f2c5c072020 2024-04-25T10:15:27.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.559+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c0b3f90 con 0x7f2c5c072020 2024-04-25T10:15:27.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.559+0000 7f2c53400640 1 -- 172.21.15.87:0/2408161413 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0b0040 con 0x7f2c5c072020 2024-04-25T10:15:27.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.559+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c0b4550 con 0x7f2c5c072020 2024-04-25T10:15:27.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.559+0000 7f2c53400640 1 -- 172.21.15.87:0/2408161413 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c4c0bb020 con 0x7f2c5c072020 2024-04-25T10:15:27.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 Processor -- start 2024-04-25T10:15:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c53400640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c3c066f60 0x7f2c3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c53400640 1 -- 172.21.15.87:0/2408161413 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c4c11eaa0 con 0x7f2c5c072020 2024-04-25T10:15:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 -- start start 2024-04-25T10:15:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5a000640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c3c066f60 0x7f2c3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5a000640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c3c066f60 0x7f2c3c069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0681e0 tx=0x7f2c48009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd12481f0 0x55ffd12485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd11fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd11fb920 0x55ffd11fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd104bce0 con 0x55ffd12481f0 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd0fd4530 con 0x55ffd1248b10 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffd1047d80 con 0x55ffd11fb920 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd12481f0 0x55ffd12485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd12481f0 0x55ffd12485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59040/0 (socket says 172.21.15.87:59040) 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 -- 172.21.15.87:0/1058685479 learned_addr learned my addr 172.21.15.87:0/1058685479 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5a000640 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd11fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5b400640 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd11fb920 0x55ffd11fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 -- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd11fb920 msgr2=0x55ffd11fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd11fb920 0x55ffd11fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 -- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 msgr2=0x55ffd11fb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd11fb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 -- 172.21.15.87:0/1058685479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd1046cd0 con 0x55ffd12481f0 2024-04-25T10:15:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5b400640 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd11fb920 0x55ffd11fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd12481f0 0x55ffd12485d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c01c9f0 tx=0x7f2c4c0eed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6719fa9698bf2501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c47e00640 1 -- 172.21.15.87:0/1058685479 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0ef3e0 con 0x55ffd12481f0 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c47e00640 1 -- 172.21.15.87:0/1058685479 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0f34f0 con 0x55ffd12481f0 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.563+0000 7f2c47e00640 1 -- 172.21.15.87:0/1058685479 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0f7ab0 con 0x55ffd12481f0 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- 172.21.15.87:0/1058685479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd12481f0 msgr2=0x55ffd12485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd12481f0 0x55ffd12485d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c01c9f0 tx=0x7f2c4c0eed40 comp rx=0 tx=0).stop 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- 172.21.15.87:0/1058685479 shutdown_connections 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd11fb920 0x55ffd11fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd11fb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 --2- 172.21.15.87:0/1058685479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd12481f0 0x55ffd12485d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- 172.21.15.87:0/1058685479 >> 172.21.15.87:0/1058685479 conn(0x55ffd1228f90 msgr2=0x55ffd122b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- 172.21.15.87:0/1058685479 shutdown_connections 2024-04-25T10:15:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- 172.21.15.87:0/1058685479 wait complete. 2024-04-25T10:15:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- start start 2024-04-25T10:15:27.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 Processor -- start 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd1200d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd1201270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd12017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd104bce0 con 0x55ffd1248b10 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd1006bc0 con 0x55ffd11fb920 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffd1050310 con 0x55ffd12481f0 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c65289000 10 client.0 _ll_register_callbacks cb 0x55ffd11e94a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd12017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd1200d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd1200d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:36588/0 (socket says 172.21.15.87:36588) 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c5aa00640 1 -- 172.21.15.87:0/4143282264 learned_addr learned my addr 172.21.15.87:0/4143282264 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.567+0000 7f2c5a000640 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd1201270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5b400640 1 -- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 msgr2=0x55ffd1201270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5b400640 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd1201270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5b400640 1 -- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 msgr2=0x55ffd1200d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5b400640 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd1200d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5b400640 1 -- 172.21.15.87:0/4143282264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffd1059950 con 0x55ffd1248b10 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd1200d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5b400640 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd12017b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2c540012f0 tx=0x7f2c54007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8358edaaa15a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c44c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c44c00640 1 -- 172.21.15.87:0/4143282264 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54018020 con 0x55ffd1248b10 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c44c00640 1 -- 172.21.15.87:0/4143282264 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c54010bd0 con 0x55ffd1248b10 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c4c0f34f0 con 0x55ffd1248b10 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c44c00640 1 -- 172.21.15.87:0/4143282264 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c54022050 con 0x55ffd1248b10 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 10 client.5979 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ffd10c8790 con 0x55ffd1248b10 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c44c00640 1 -- 172.21.15.87:0/4143282264 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0f34f0 con 0x55ffd1248b10 2024-04-25T10:15:27.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c44c00640 1 -- 172.21.15.87:0/4143282264 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c54025020 con 0x55ffd1248b10 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 10 client.5979 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 10 client.5979 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 20 client.5979 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12021c0 0x55ffd1259290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 4 client.5979 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffd0fe7b80 con 0x55ffd12021c0 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12021c0 0x55ffd1259290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12021c0 0x55ffd1259290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc70f5ad66622ed9 server_cookie=933e6026fb3e22da in_seq=0 out_seq=0 2024-04-25T10:15:27.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.571+0000 7f2c44c00640 10 client.5979 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c44c00640 1 -- 172.21.15.87:0/4143282264 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffd0fe7b80 con 0x55ffd12021c0 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c44c00640 10 client.5979 handle_command_reply: tid=0 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 client.5979 shutdown 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 msgr2=0x55ffd12017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd12017b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2c540012f0 tx=0x7f2c54007960 comp rx=0 tx=0).stop 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 shutdown_connections 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd12017b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12021c0 0x55ffd1259290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd1201270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 --2- 172.21.15.87:0/4143282264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd1200d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 >> 172.21.15.87:0/4143282264 conn(0x55ffd1228f90 msgr2=0x55ffd122b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c44c00640 0 client.5979 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 shutdown_connections 2024-04-25T10:15:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 1 -- 172.21.15.87:0/4143282264 wait complete. 2024-04-25T10:15:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.575+0000 7f2c65289000 20 client.5979 trim_cache size 0 max 16384 2024-04-25T10:15:27.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 Processor -- start 2024-04-25T10:15:27.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- start start 2024-04-25T10:15:27.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd11fb920 0x55ffd1258290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd1258d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3400b650 con 0x55ffd11fb920 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3400aa30 con 0x55ffd1248b10 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3400b030 con 0x55ffd12481f0 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd11fb920 0x55ffd1258290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd11fb920 0x55ffd1258290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59046/0 (socket says 172.21.15.87:59046) 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 -- 172.21.15.87:0/3748324184 learned_addr learned my addr 172.21.15.87:0/3748324184 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5b400640 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd1258d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5a000640 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 -- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 msgr2=0x55ffd12587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 -- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 msgr2=0x55ffd1258d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd1258d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 -- 172.21.15.87:0/3748324184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd10116f0 con 0x55ffd11fb920 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5b400640 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd1258d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5a000640 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd11fb920 0x55ffd1258290 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000c10 tx=0x7f2c4c106000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2d44b68542b190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c47400640 1 -- 172.21.15.87:0/3748324184 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0ed6e0 con 0x55ffd11fb920 2024-04-25T10:15:27.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c47400640 1 -- 172.21.15.87:0/3748324184 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0f30d0 con 0x55ffd11fb920 2024-04-25T10:15:27.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c47400640 1 -- 172.21.15.87:0/3748324184 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c10d300 con 0x55ffd11fb920 2024-04-25T10:15:27.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- 172.21.15.87:0/3748324184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd11fb920 msgr2=0x55ffd1258290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd11fb920 0x55ffd1258290 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000c10 tx=0x7f2c4c106000 comp rx=0 tx=0).stop 2024-04-25T10:15:27.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- 172.21.15.87:0/3748324184 shutdown_connections 2024-04-25T10:15:27.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd1248b10 0x55ffd1258d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 --2- 172.21.15.87:0/3748324184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd11fb920 0x55ffd1258290 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- 172.21.15.87:0/3748324184 >> 172.21.15.87:0/3748324184 conn(0x55ffd1228f90 msgr2=0x55ffd1246b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- 172.21.15.87:0/3748324184 shutdown_connections 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.591+0000 7f2c65289000 1 -- 172.21.15.87:0/3748324184 wait complete. 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 -- start start 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 Processor -- start 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd12a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12a8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd1262d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd1029ac0 con 0x55ffd1248b10 2024-04-25T10:15:27.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd0ff28e0 con 0x55ffd11fb920 2024-04-25T10:15:27.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffd1207130 con 0x55ffd12481f0 2024-04-25T10:15:27.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 10 client.0 _ll_register_callbacks cb 0x7f2c4c006ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:27.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd1262d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd1262d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59054/0 (socket says 172.21.15.87:59054) 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 -- 172.21.15.87:0/3694388549 learned_addr learned my addr 172.21.15.87:0/3694388549 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd12a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5a000640 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 -- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 msgr2=0x55ffd12a8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12a8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 -- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 msgr2=0x55ffd12a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd12a7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 -- 172.21.15.87:0/3694388549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffd1007c80 con 0x55ffd1248b10 2024-04-25T10:15:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5b400640 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd1262d10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2c540012f0 tx=0x7f2c54005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5c9b9976682ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c44c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c44c00640 1 -- 172.21.15.87:0/3694388549 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54018050 con 0x55ffd1248b10 2024-04-25T10:15:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c44c00640 1 -- 172.21.15.87:0/3694388549 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c54014980 con 0x55ffd1248b10 2024-04-25T10:15:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c4c0f30d0 con 0x55ffd1248b10 2024-04-25T10:15:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c44c00640 1 -- 172.21.15.87:0/3694388549 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c54038050 con 0x55ffd1248b10 2024-04-25T10:15:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 10 client.5985 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ffd0d08600 con 0x55ffd1248b10 2024-04-25T10:15:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c44c00640 1 -- 172.21.15.87:0/3694388549 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0f30d0 con 0x55ffd1248b10 2024-04-25T10:15:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c44c00640 1 -- 172.21.15.87:0/3694388549 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c5403b020 con 0x55ffd1248b10 2024-04-25T10:15:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 10 client.5985 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:27.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 10 client.5985 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:27.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 20 client.5985 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:27.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12636d0 0x55ffd1263ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 4 client.5985 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T10:15:27.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55ffd1059220 con 0x55ffd12636d0 2024-04-25T10:15:27.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.595+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12636d0 0x55ffd1263ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c5aa00640 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12636d0 0x55ffd1263ae0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5648014149764089 server_cookie=961fa425962708df in_seq=0 out_seq=0 2024-04-25T10:15:27.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c44c00640 10 client.5985 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.611 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T10:15:27.599+0000 7f6ac1a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096b [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v622 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5512=0-4194304@1} caps={5512=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5512 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55df48cccc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1000000096b:[<0x10000000001/new_file_0 v622>,<0x10000000000/ceph-qa-suite v53>,<0x1/scrub_test_path v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c44c00640 1 -- 172.21.15.87:0/3694388549 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ffd1059220 con 0x55ffd12636d0 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c44c00640 10 client.5985 handle_command_reply: tid=0 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 client.5985 shutdown 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 msgr2=0x55ffd1262d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd1262d10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2c540012f0 tx=0x7f2c54005b80 comp rx=0 tx=0).stop 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 shutdown_connections 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffd1248b10 0x55ffd1262d10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffd12636d0 0x55ffd1263ae0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffd12481f0 0x55ffd12a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 --2- 172.21.15.87:0/3694388549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffd11fb920 0x55ffd12a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c44c00640 0 client.5985 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 >> 172.21.15.87:0/3694388549 conn(0x55ffd1228f90 msgr2=0x55ffd1247240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 shutdown_connections 2024-04-25T10:15:27.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 1 -- 172.21.15.87:0/3694388549 wait complete. 2024-04-25T10:15:27.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c65289000 20 client.5985 trim_cache size 0 max 16384 2024-04-25T10:15:27.613 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "7ff2b0ec-c8a8-4c17-837a-67299fbc51ad", 2024-04-25T10:15:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:27.613 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c3c066f60 msgr2=0x7f2c3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c3c066f60 0x7f2c3c069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0681e0 tx=0x7f2c48009190 comp rx=0 tx=0).stop 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c072020 msgr2=0x7f2c5c149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c072020 0x7f2c5c149060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0ae040 tx=0x7f2c4c019220 comp rx=0 tx=0).stop 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 shutdown_connections 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2c3c066f60 0x7f2c3c069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c5c159540 0x7f2c5c0b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c5c0aebd0 0x7f2c5c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 --2- 172.21.15.87:0/2408161413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c5c072020 0x7f2c5c149060 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 >> 172.21.15.87:0/2408161413 conn(0x7f2c5c0669a0 msgr2=0x7f2c5c06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 shutdown_connections 2024-04-25T10:15:27.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.599+0000 7f2c61e00640 1 -- 172.21.15.87:0/2408161413 wait complete. 2024-04-25T10:15:27.623 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '7ff2b0ec-c8a8-4c17-837a-67299fbc51ad', 'mode': 'asynchronous'}' 2024-04-25T10:15:27.623 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:15:27.623 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:15:27.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6328e00640 1 Processor -- start 2024-04-25T10:15:27.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6328e00640 1 -- start start 2024-04-25T10:15:27.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6328e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63240cf980 0x7f63240b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6328e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63240d1830 0x7f63240d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6328e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632415fec0 con 0x7f632415bc10 2024-04-25T10:15:27.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6328e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63241601a0 con 0x7f63240d1830 2024-04-25T10:15:27.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6328e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63240cc5b0 con 0x7f63240cf980 2024-04-25T10:15:27.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6321e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f632415bc10 0x7f632415f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50648/0 (socket says 172.21.15.87:50648) 2024-04-25T10:15:27.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6321e00640 1 -- 172.21.15.87:0/2548819255 learned_addr learned my addr 172.21.15.87:0/2548819255 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:27.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6322800640 1 --2- 172.21.15.87:0/2548819255 >> v2:172.21.15.87:3300/0 conn(0x7f63240d1830 0x7f63240d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6319400640 1 --2- 172.21.15.87:0/2548819255 >> v2:172.21.15.169:3300/0 conn(0x7f63240cf980 0x7f63240b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6321400640 1 -- 172.21.15.87:0/2548819255 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063348191 0 0) 0x7f632415fec0 con 0x7f632415bc10 2024-04-25T10:15:27.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6321400640 1 -- 172.21.15.87:0/2548819255 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6308003530 con 0x7f632415bc10 2024-04-25T10:15:27.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6322800640 1 -- 172.21.15.87:0/2548819255 >> v2:172.21.15.169:3300/0 conn(0x7f63240cf980 msgr2=0x7f63240b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6322800640 1 --2- 172.21.15.87:0/2548819255 >> v2:172.21.15.169:3300/0 conn(0x7f63240cf980 0x7f63240b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6322800640 1 -- 172.21.15.87:0/2548819255 >> v1:172.21.15.87:6789/0 conn(0x7f632415bc10 legacy=0x7f632415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.755+0000 7f6322800640 1 -- 172.21.15.87:0/2548819255 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240cc890 con 0x7f63240d1830 2024-04-25T10:15:27.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6322800640 1 --2- 172.21.15.87:0/2548819255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63240d1830 0x7f63240d1c10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f63240724a0 tx=0x7f631c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46de61ee48666816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6321400640 1 -- 172.21.15.87:0/2548819255 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0b0070 con 0x7f63240d1830 2024-04-25T10:15:27.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6321400640 1 -- 172.21.15.87:0/2548819255 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c0ab600 con 0x7f63240d1830 2024-04-25T10:15:27.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6321400640 1 -- 172.21.15.87:0/2548819255 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0ab9d0 con 0x7f63240d1830 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 -- 172.21.15.87:0/2548819255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63240d1830 msgr2=0x7f63240d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 --2- 172.21.15.87:0/2548819255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63240d1830 0x7f63240d1c10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f63240724a0 tx=0x7f631c01c480 comp rx=0 tx=0).stop 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 -- 172.21.15.87:0/2548819255 shutdown_connections 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 --2- 172.21.15.87:0/2548819255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63240d1830 0x7f63240d1c10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 --2- 172.21.15.87:0/2548819255 >> v2:172.21.15.169:3300/0 conn(0x7f63240cf980 0x7f63240b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 -- 172.21.15.87:0/2548819255 >> 172.21.15.87:0/2548819255 conn(0x7f632409a9d0 msgr2=0x7f632409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 -- 172.21.15.87:0/2548819255 shutdown_connections 2024-04-25T10:15:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.759+0000 7f6328e00640 1 -- 172.21.15.87:0/2548819255 wait complete. 2024-04-25T10:15:27.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 Processor -- start 2024-04-25T10:15:27.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 -- start start 2024-04-25T10:15:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63240cf980 0x7f63240c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63240d1830 0x7f632415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632415bc10 0x7f632415c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63240cd740 con 0x7f632415bc10 2024-04-25T10:15:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63240cd5c0 con 0x7f63240d1830 2024-04-25T10:15:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6324168960 con 0x7f63240cf980 2024-04-25T10:15:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632415bc10 0x7f632415c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632415bc10 0x7f632415c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59064/0 (socket says 172.21.15.87:59064) 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 -- 172.21.15.87:0/2414552433 learned_addr learned my addr 172.21.15.87:0/2414552433 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6319400640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63240d1830 0x7f632415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6321e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63240cf980 0x7f63240c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 -- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63240cf980 msgr2=0x7f63240c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63240cf980 0x7f63240c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 -- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63240d1830 msgr2=0x7f632415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63240d1830 0x7f632415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 -- 172.21.15.87:0/2414552433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240ca4f0 con 0x7f632415bc10 2024-04-25T10:15:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6322800640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632415bc10 0x7f632415c640 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ae040 tx=0x7f631c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f631aa00640 1 -- 172.21.15.87:0/2414552433 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0b0040 con 0x7f632415bc10 2024-04-25T10:15:27.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f631aa00640 1 -- 172.21.15.87:0/2414552433 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c0b9070 con 0x7f632415bc10 2024-04-25T10:15:27.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63240ca7d0 con 0x7f632415bc10 2024-04-25T10:15:27.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f631aa00640 1 -- 172.21.15.87:0/2414552433 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0b4650 con 0x7f632415bc10 2024-04-25T10:15:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.763+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63240cad90 con 0x7f632415bc10 2024-04-25T10:15:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.767+0000 7f631aa00640 1 -- 172.21.15.87:0/2414552433 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f631c0b47f0 con 0x7f632415bc10 2024-04-25T10:15:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.767+0000 7f631aa00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.767+0000 7f631aa00640 1 -- 172.21.15.87:0/2414552433 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f631c11ed00 con 0x7f632415bc10 2024-04-25T10:15:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.767+0000 7f6321e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.767+0000 7f632c223000 1 Processor -- start 2024-04-25T10:15:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.767+0000 7f632c223000 1 -- start start 2024-04-25T10:15:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.767+0000 7f6321e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6300066f60 0x7f6300069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f63240b3c40 tx=0x7f6314000f50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d29d0 0x55f5814d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f581488b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f581489480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5812d5ff0 con 0x55f5814d32f0 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58125e150 con 0x55f5814d29d0 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5812d2090 con 0x55f5814890a0 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f581488b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f581488b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59072/0 (socket says 172.21.15.87:59072) 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 -- 172.21.15.87:0/3144809348 learned_addr learned my addr 172.21.15.87:0/3144809348 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6321e00640 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d29d0 0x55f5814d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6322800640 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f581489480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 -- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 msgr2=0x55f581489480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f581489480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 -- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d29d0 msgr2=0x55f5814d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d29d0 0x55f5814d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 -- 172.21.15.87:0/3144809348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58146c0d0 con 0x55f5814d32f0 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6321e00640 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d29d0 0x55f5814d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6322800640 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f581489480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6319400640 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f581488b60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f630c009b70 tx=0x7f630c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8b053bb7df0164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6312a00640 1 -- 172.21.15.87:0/3144809348 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c019070 con 0x55f5814d32f0 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6312a00640 1 -- 172.21.15.87:0/3144809348 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c004030 con 0x55f5814d32f0 2024-04-25T10:15:27.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f6312a00640 1 -- 172.21.15.87:0/3144809348 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c004400 con 0x55f5814d32f0 2024-04-25T10:15:27.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- 172.21.15.87:0/3144809348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 msgr2=0x55f581488b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f581488b60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f630c009b70 tx=0x7f630c011690 comp rx=0 tx=0).stop 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- 172.21.15.87:0/3144809348 shutdown_connections 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f581489480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f581488b60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 --2- 172.21.15.87:0/3144809348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d29d0 0x55f5814d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- 172.21.15.87:0/3144809348 >> 172.21.15.87:0/3144809348 conn(0x55f581485020 msgr2=0x55f581487440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- 172.21.15.87:0/3144809348 shutdown_connections 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- 172.21.15.87:0/3144809348 wait complete. 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 -- start start 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.771+0000 7f632c223000 1 Processor -- start 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58129a250 con 0x55f5814d32f0 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581290690 con 0x55f5814890a0 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58125ea40 con 0x55f5814d29d0 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 10 client.0 _ll_register_callbacks cb 0x55f581473e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814b8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59076/0 (socket says 172.21.15.87:59076) 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 -- 172.21.15.87:0/557888757 learned_addr learned my addr 172.21.15.87:0/557888757 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6321e00640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6319400640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 -- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 msgr2=0x55f5814b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 -- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 msgr2=0x55f5814b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 -- 172.21.15.87:0/557888757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5813831c0 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6319400640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6322800640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814b8270 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ebb20 tx=0x7f631c0ef780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6722bff318468747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 1 -- 172.21.15.87:0/557888757 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c11b060 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 1 -- 172.21.15.87:0/557888757 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f631c0f7890 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f630c004030 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 1 -- 172.21.15.87:0/557888757 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f631c0f7d20 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 10 client.6000 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f58137f170 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 1 -- 172.21.15.87:0/557888757 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c004030 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 1 -- 172.21.15.87:0/557888757 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f631c0b3070 con 0x55f5814d32f0 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 10 client.6000 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:27.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 10 client.6000 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 20 client.6000 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e2c80 0x55f5814e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 4 client.6000 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f581271c60 con 0x55f5814e2c80 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6321e00640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e2c80 0x55f5814e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6321e00640 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e2c80 0x55f5814e5070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68dab525b9ec4123 server_cookie=addbd9d14205a8f0 in_seq=0 out_seq=0 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 10 client.6000 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 1 -- 172.21.15.87:0/557888757 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f581271c60 con 0x55f5814e2c80 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 10 client.6000 handle_command_reply: tid=0 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 client.6000 shutdown 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 msgr2=0x55f5814b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814b8270 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ebb20 tx=0x7f631c0ef780 comp rx=0 tx=0).stop 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 shutdown_connections 2024-04-25T10:15:27.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814b8270 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e2c80 0x55f5814e5070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 --2- 172.21.15.87:0/557888757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f6307400640 0 client.6000 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.775+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 >> 172.21.15.87:0/557888757 conn(0x55f581485020 msgr2=0x55f581487ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.779+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 shutdown_connections 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.779+0000 7f632c223000 1 -- 172.21.15.87:0/557888757 wait complete. 2024-04-25T10:15:27.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.779+0000 7f632c223000 20 client.6000 trim_cache size 0 max 16384 2024-04-25T10:15:27.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 Processor -- start 2024-04-25T10:15:27.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 -- start start 2024-04-25T10:15:27.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f58152f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d29d0 0x55f58152f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d32f0 0x55f58152fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63242009c0 con 0x55f5814d29d0 2024-04-25T10:15:27.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241ffbb0 con 0x55f5814d32f0 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.791+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6324157f10 con 0x55f5814890a0 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d29d0 0x55f58152f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d29d0 0x55f58152f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59086/0 (socket says 172.21.15.87:59086) 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 -- 172.21.15.87:0/2585452188 learned_addr learned my addr 172.21.15.87:0/2585452188 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6321e00640 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f58152f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6322800640 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d32f0 0x55f58152fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 -- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 msgr2=0x55f58152f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f58152f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 -- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d32f0 msgr2=0x55f58152fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d32f0 0x55f58152fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 -- 172.21.15.87:0/2585452188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58131cde0 con 0x55f5814d29d0 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6322800640 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d32f0 0x55f58152fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6319400640 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d29d0 0x55f58152f630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f630c018000 tx=0x7f630c008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263e4c74ef23aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6312000640 1 -- 172.21.15.87:0/2585452188 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c019040 con 0x55f5814d29d0 2024-04-25T10:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6312000640 1 -- 172.21.15.87:0/2585452188 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c004240 con 0x55f5814d29d0 2024-04-25T10:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6312000640 1 -- 172.21.15.87:0/2585452188 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c006de0 con 0x55f5814d29d0 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- 172.21.15.87:0/2585452188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d29d0 msgr2=0x55f58152f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d29d0 0x55f58152f630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f630c018000 tx=0x7f630c008070 comp rx=0 tx=0).stop 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- 172.21.15.87:0/2585452188 shutdown_connections 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814d32f0 0x55f58152fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d29d0 0x55f58152f630 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 --2- 172.21.15.87:0/2585452188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814890a0 0x55f58152f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- 172.21.15.87:0/2585452188 >> 172.21.15.87:0/2585452188 conn(0x55f581485020 msgr2=0x55f5814dffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- 172.21.15.87:0/2585452188 shutdown_connections 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- 172.21.15.87:0/2585452188 wait complete. 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- start start 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 Processor -- start 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581270e00 con 0x55f5814d32f0 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241ff5e0 con 0x55f5814890a0 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5812d5300 con 0x55f5814d29d0 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f632c223000 10 client.0 _ll_register_callbacks cb 0x55f581454040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.795+0000 7f6322800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814e5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59100/0 (socket says 172.21.15.87:59100) 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 -- 172.21.15.87:0/1439887375 learned_addr learned my addr 172.21.15.87:0/1439887375 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6321e00640 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6319400640 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 -- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 msgr2=0x55f5814e58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814e58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 -- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 msgr2=0x55f5814e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814e5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 -- 172.21.15.87:0/1439887375 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5812778e0 con 0x55f5814d32f0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6322800640 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814e5e00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f631c01ab30 tx=0x7f631c11d100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af326ae3716e46d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 1 -- 172.21.15.87:0/1439887375 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0ef020 con 0x55f5814d32f0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 1 -- 172.21.15.87:0/1439887375 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f631c0b4cf0 con 0x55f5814d32f0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f630c004240 con 0x55f5814d32f0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 1 -- 172.21.15.87:0/1439887375 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f631c0ef200 con 0x55f5814d32f0 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 10 client.6006 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:27.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f58132f8f0 con 0x55f5814d32f0 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 1 -- 172.21.15.87:0/1439887375 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c004240 con 0x55f5814d32f0 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 1 -- 172.21.15.87:0/1439887375 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f631c0b3070 con 0x55f5814d32f0 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 10 client.6006 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 10 client.6006 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 20 client.6006 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e67c0 0x55f58153af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 4 client.6006 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5812e3530 con 0x55f5814e67c0 2024-04-25T10:15:27.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6321e00640 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e67c0 0x55f58153af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:27.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6321e00640 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e67c0 0x55f58153af60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f070c54935b7e3d server_cookie=d4c134475d9d1de4 in_seq=0 out_seq=0 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 10 client.6006 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 1 -- 172.21.15.87:0/1439887375 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x55f5812e3530 con 0x55f5814e67c0 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 10 client.6006 handle_command_reply: tid=0 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 client.6006 shutdown 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 msgr2=0x55f5814e5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814e5e00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f631c01ab30 tx=0x7f631c11d100 comp rx=0 tx=0).stop 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 shutdown_connections 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f5814d32f0 0x55f5814e5e00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55f5814e67c0 0x55f58153af60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f5814d29d0 0x55f5814e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 --2- 172.21.15.87:0/1439887375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f5814890a0 0x55f5814e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6307400640 0 client.6006 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:27.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 >> 172.21.15.87:0/1439887375 conn(0x55f581485020 msgr2=0x55f5814d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 shutdown_connections 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 1 -- 172.21.15.87:0/1439887375 wait complete. 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f632c223000 20 client.6006 trim_cache size 0 max 16384 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stdout: "7ff2b0ec-c8a8-4c17-837a-67299fbc51ad": { 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "7ff2b0ec-c8a8-4c17-837a-67299fbc51ad", 2024-04-25T10:15:27.811 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "" 2024-04-25T10:15:27.812 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:27.812 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:27.812 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:27.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6300066f60 msgr2=0x7f6300069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6300066f60 0x7f6300069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f63240b3c40 tx=0x7f6314000f50 comp rx=0 tx=0).stop 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632415bc10 msgr2=0x7f632415c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632415bc10 0x7f632415c640 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ae040 tx=0x7f631c019220 comp rx=0 tx=0).stop 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 shutdown_connections 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632415bc10 0x7f632415c640 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63240d1830 0x7f632415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 --2- 172.21.15.87:0/2414552433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63240cf980 0x7f63240c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 >> 172.21.15.87:0/2414552433 conn(0x7f632409a9d0 msgr2=0x7f632406f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 shutdown_connections 2024-04-25T10:15:27.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:27.799+0000 7f6328e00640 1 -- 172.21.15.87:0/2414552433 wait complete. 2024-04-25T10:15:32.826 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:15:32.827 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:15:32.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.967+0000 7fd98e000640 1 Processor -- start 2024-04-25T10:15:32.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.967+0000 7fd98e000640 1 -- start start 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd98e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9880722d0 0x7fd9880cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd98e000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880ad840 con 0x7fd9880b3480 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd98e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98815e710 con 0x7fd98815e130 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd98e000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9880d0090 con 0x7fd9880722d0 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd987e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9880722d0 0x7fd9880cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd987400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd9880b3480 0x7fd988070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50664/0 (socket says 172.21.15.87:50664) 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd987400640 1 -- 172.21.15.87:0/3998473931 learned_addr learned my addr 172.21.15.87:0/3998473931 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd986a00640 1 -- 172.21.15.87:0/3998473931 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066076989 0 0) 0x7fd9880ad840 con 0x7fd9880b3480 2024-04-25T10:15:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd987e00640 1 -- 172.21.15.87:0/3998473931 >> v1:172.21.15.169:6789/0 conn(0x7fd98815e130 legacy=0x7fd988071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd987e00640 1 -- 172.21.15.87:0/3998473931 >> v1:172.21.15.87:6789/0 conn(0x7fd9880b3480 legacy=0x7fd988070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.971+0000 7fd987e00640 1 -- 172.21.15.87:0/3998473931 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98815e9f0 con 0x7fd9880722d0 2024-04-25T10:15:32.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd987e00640 1 --2- 172.21.15.87:0/3998473931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd9880cfb50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd98001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611e352d87f35384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd986a00640 1 -- 172.21.15.87:0/3998473931 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800b2070 con 0x7fd9880722d0 2024-04-25T10:15:32.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd986a00640 1 -- 172.21.15.87:0/3998473931 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800a57a0 con 0x7fd9880722d0 2024-04-25T10:15:32.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd986a00640 1 -- 172.21.15.87:0/3998473931 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800a5b70 con 0x7fd9880722d0 2024-04-25T10:15:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 -- 172.21.15.87:0/3998473931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 msgr2=0x7fd9880cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 --2- 172.21.15.87:0/3998473931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd9880cfb50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd98001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 -- 172.21.15.87:0/3998473931 shutdown_connections 2024-04-25T10:15:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 --2- 172.21.15.87:0/3998473931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd9880cfb50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 -- 172.21.15.87:0/3998473931 >> 172.21.15.87:0/3998473931 conn(0x7fd9880669a0 msgr2=0x7fd988066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 -- 172.21.15.87:0/3998473931 shutdown_connections 2024-04-25T10:15:32.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 -- 172.21.15.87:0/3998473931 wait complete. 2024-04-25T10:15:32.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 Processor -- start 2024-04-25T10:15:32.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 -- start start 2024-04-25T10:15:32.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd988155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9880b3480 0x7fd988152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.975+0000 7fd98e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98815e130 0x7fd988153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd98e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98815bb90 con 0x7fd9880722d0 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd98e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880ae7c0 con 0x7fd98815e130 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd98e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98816b8a0 con 0x7fd9880b3480 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd988155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd988155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59128/0 (socket says 172.21.15.87:59128) 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 -- 172.21.15.87:0/4030512157 learned_addr learned my addr 172.21.15.87:0/4030512157 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd97fe00640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9880b3480 0x7fd988152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987e00640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98815e130 0x7fd988153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 -- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9880b3480 msgr2=0x7fd988152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9880b3480 0x7fd988152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 -- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98815e130 msgr2=0x7fd988153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98815e130 0x7fd988153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 -- 172.21.15.87:0/4030512157 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98815f440 con 0x7fd9880722d0 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd97fe00640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9880b3480 0x7fd988152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987e00640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98815e130 0x7fd988153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd987400640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd988155d40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c4a0 tx=0x7fd97800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd97f400640 1 -- 172.21.15.87:0/4030512157 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978018070 con 0x7fd9880722d0 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd97f400640 1 -- 172.21.15.87:0/4030512157 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9780074e0 con 0x7fd9880722d0 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98815f720 con 0x7fd9880722d0 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd97f400640 1 -- 172.21.15.87:0/4030512157 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978004a30 con 0x7fd9880722d0 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98815fce0 con 0x7fd9880722d0 2024-04-25T10:15:32.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.979+0000 7fd97f400640 1 -- 172.21.15.87:0/4030512157 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd97801b070 con 0x7fd9880722d0 2024-04-25T10:15:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd97f400640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd964066f90 0x7fd964069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd97f400640 1 -- 172.21.15.87:0/4030512157 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9780843f0 con 0x7fd9880722d0 2024-04-25T10:15:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 Processor -- start 2024-04-25T10:15:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd97fe00640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd964066f90 0x7fd964069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- start start 2024-04-25T10:15:32.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd97fe00640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd964066f90 0x7fd964069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd988156810 tx=0x7fd970009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f74db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f752f0 0x5579f9f2ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d77ff0 con 0x5579f9f2b0a0 2024-04-25T10:15:32.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d00150 con 0x5579f9f752f0 2024-04-25T10:15:32.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d74090 con 0x5579f9f749d0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59130/0 (socket says 172.21.15.87:59130) 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 -- 172.21.15.87:0/3251192959 learned_addr learned my addr 172.21.15.87:0/3251192959 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd97fe00640 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f752f0 0x5579f9f2ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987400640 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f74db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 -- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 msgr2=0x5579f9f74db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f74db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 -- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f752f0 msgr2=0x5579f9f2ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f752f0 0x5579f9f2ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 -- 172.21.15.87:0/3251192959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f9f0e0d0 con 0x5579f9f2b0a0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd987e00640 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2b480 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd98001c920 tx=0x7fd980004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d483c8b0ddd52ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd977e00640 1 -- 172.21.15.87:0/3251192959 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800b2040 con 0x5579f9f2b0a0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd977e00640 1 -- 172.21.15.87:0/3251192959 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd980003b20 con 0x5579f9f2b0a0 2024-04-25T10:15:32.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd977e00640 1 -- 172.21.15.87:0/3251192959 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980004040 con 0x5579f9f2b0a0 2024-04-25T10:15:32.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- 172.21.15.87:0/3251192959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 msgr2=0x5579f9f2b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2b480 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd98001c920 tx=0x7fd980004ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- 172.21.15.87:0/3251192959 shutdown_connections 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2b480 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f752f0 0x5579f9f2ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 --2- 172.21.15.87:0/3251192959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f74db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- 172.21.15.87:0/3251192959 >> 172.21.15.87:0/3251192959 conn(0x5579f9f27020 msgr2=0x5579f9f29440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- 172.21.15.87:0/3251192959 shutdown_connections 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.983+0000 7fd991589000 1 -- 172.21.15.87:0/3251192959 wait complete. 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 -- start start 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 Processor -- start 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9f59d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f5a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d3c250 con 0x5579f9f2b0a0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d32690 con 0x5579f9f749d0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d00a40 con 0x5579f9f752f0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 10 client.0 _ll_register_callbacks cb 0x5579f9f15e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59144/0 (socket says 172.21.15.87:59144) 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 -- 172.21.15.87:0/3848736967 learned_addr learned my addr 172.21.15.87:0/3848736967 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987e00640 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f5a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd97fe00640 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9f59d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 -- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 msgr2=0x5579f9f5a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f5a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 -- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 msgr2=0x5579f9f59d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9f59d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 -- 172.21.15.87:0/3848736967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579f9e251c0 con 0x5579f9f2b0a0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd987400640 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f597f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c470 tx=0x7fd978054da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea35090cd6fcd95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd974c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd974c00640 1 -- 172.21.15.87:0/3848736967 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97805cee0 con 0x5579f9f2b0a0 2024-04-25T10:15:32.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd974c00640 1 -- 172.21.15.87:0/3848736967 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd97805d080 con 0x5579f9f2b0a0 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd980003b20 con 0x5579f9f2b0a0 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd974c00640 1 -- 172.21.15.87:0/3848736967 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9780540b0 con 0x5579f9f2b0a0 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 10 client.6021 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5579f9e21170 con 0x5579f9f2b0a0 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd974c00640 1 -- 172.21.15.87:0/3848736967 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd980003b20 con 0x5579f9f2b0a0 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd974c00640 1 -- 172.21.15.87:0/3848736967 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd978082ce0 con 0x5579f9f2b0a0 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 10 client.6021 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 10 client.6021 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 20 client.6021 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f84c80 0x5579f9f87070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 4 client.6021 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579f9d13c60 con 0x5579f9f84c80 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.987+0000 7fd97fe00640 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f84c80 0x5579f9f87070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd97fe00640 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f84c80 0x5579f9f87070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8310e2fc070ae2d4 server_cookie=da5a6d9164389106 in_seq=0 out_seq=0 2024-04-25T10:15:32.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd974c00640 10 client.6021 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd974c00640 1 -- 172.21.15.87:0/3848736967 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579f9d13c60 con 0x5579f9f84c80 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd974c00640 10 client.6021 handle_command_reply: tid=0 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 client.6021 shutdown 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 msgr2=0x5579f9f597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f597f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c470 tx=0x7fd978054da0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 shutdown_connections 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f84c80 0x5579f9f87070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f5a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9f59d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 --2- 172.21.15.87:0/3848736967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9f597f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd974c00640 0 client.6021 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 >> 172.21.15.87:0/3848736967 conn(0x5579f9f27020 msgr2=0x5579f9f29ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 shutdown_connections 2024-04-25T10:15:33.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 1 -- 172.21.15.87:0/3848736967 wait complete. 2024-04-25T10:15:33.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:32.991+0000 7fd991589000 20 client.6021 trim_cache size 0 max 16384 2024-04-25T10:15:33.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 Processor -- start 2024-04-25T10:15:33.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- start start 2024-04-25T10:15:33.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f752f0 0x5579f9f73d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988202c10 con 0x5579f9f752f0 2024-04-25T10:15:33.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988148f70 con 0x5579f9f2b0a0 2024-04-25T10:15:33.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd988151fd0 con 0x5579f9f749d0 2024-04-25T10:15:33.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f752f0 0x5579f9f73d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f752f0 0x5579f9f73d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59148/0 (socket says 172.21.15.87:59148) 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 -- 172.21.15.87:0/988568402 learned_addr learned my addr 172.21.15.87:0/988568402 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd97fe00640 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987400640 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 -- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 msgr2=0x5579f9f737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 -- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f2b0a0 msgr2=0x5579f9f2a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 -- 172.21.15.87:0/988568402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f9f75120 con 0x5579f9f752f0 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987400640 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd987e00640 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f752f0 0x5579f9f73d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9800095c0 tx=0x7fd9800a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cac1d2b9caeb441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd977400640 1 -- 172.21.15.87:0/988568402 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800c5070 con 0x5579f9f752f0 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd977400640 1 -- 172.21.15.87:0/988568402 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800a5620 con 0x5579f9f752f0 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd977400640 1 -- 172.21.15.87:0/988568402 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800b2030 con 0x5579f9f752f0 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- 172.21.15.87:0/988568402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f752f0 msgr2=0x5579f9f73d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f752f0 0x5579f9f73d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9800095c0 tx=0x7fd9800a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- 172.21.15.87:0/988568402 shutdown_connections 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f752f0 0x5579f9f73d20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f749d0 0x5579f9f737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 --2- 172.21.15.87:0/988568402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f2b0a0 0x5579f9f2a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- 172.21.15.87:0/988568402 >> 172.21.15.87:0/988568402 conn(0x5579f9f27020 msgr2=0x5579f9f730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- 172.21.15.87:0/988568402 shutdown_connections 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- 172.21.15.87:0/988568402 wait complete. 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 -- start start 2024-04-25T10:15:33.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.007+0000 7fd991589000 1 Processor -- start 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9fd5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9fd5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f87380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d12e00 con 0x5579f9f2b0a0 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988148970 con 0x5579f9f749d0 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579f9d77300 con 0x5579f9f752f0 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 10 client.0 _ll_register_callbacks cb 0x5579f9ef6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9fd5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9fd5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59162/0 (socket says 172.21.15.87:59162) 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 -- 172.21.15.87:0/2855937046 learned_addr learned my addr 172.21.15.87:0/2855937046 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd97fe00640 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9fd5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987e00640 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f87380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 -- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 msgr2=0x5579f9f87380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f87380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 -- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 msgr2=0x5579f9fd5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9fd5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 -- 172.21.15.87:0/2855937046 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579f9dbede0 con 0x5579f9f2b0a0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd987400640 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9fd5520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd978000f70 tx=0x7fd978057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f6794b6753c719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 1 -- 172.21.15.87:0/2855937046 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780547a0 con 0x5579f9f2b0a0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 1 -- 172.21.15.87:0/2855937046 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9780509b0 con 0x5579f9f2b0a0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9800a5620 con 0x5579f9f2b0a0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 1 -- 172.21.15.87:0/2855937046 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd978055360 con 0x5579f9f2b0a0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 10 client.6027 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5579f9d198e0 con 0x5579f9f2b0a0 2024-04-25T10:15:33.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 1 -- 172.21.15.87:0/2855937046 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800a5620 con 0x5579f9f2b0a0 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 1 -- 172.21.15.87:0/2855937046 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd978050bb0 con 0x5579f9f2b0a0 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 10 client.6027 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 10 client.6027 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 20 client.6027 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f87d70 0x5579f9f88150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 4 client.6027 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5579f9d85530 con 0x5579f9f87d70 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd97fe00640 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f87d70 0x5579f9f88150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd97fe00640 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f87d70 0x5579f9f88150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4393f19e4008dd server_cookie=c3daa8e281a4125c in_seq=0 out_seq=0 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 10 client.6027 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 1 -- 172.21.15.87:0/2855937046 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5579f9d85530 con 0x5579f9f87d70 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 10 client.6027 handle_command_reply: tid=0 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 client.6027 shutdown 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 msgr2=0x5579f9fd5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9fd5520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd978000f70 tx=0x7fd978057040 comp rx=0 tx=0).stop 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 shutdown_connections 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5579f9f87d70 0x5579f9f88150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579f9f752f0 0x5579f9f87380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579f9f749d0 0x5579f9fd5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd974c00640 0 client.6027 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 --2- 172.21.15.87:0/2855937046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579f9f2b0a0 0x5579f9fd5520 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 >> 172.21.15.87:0/2855937046 conn(0x5579f9f27020 msgr2=0x5579f9f73960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 shutdown_connections 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.011+0000 7fd991589000 1 -- 172.21.15.87:0/2855937046 wait complete. 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd991589000 20 client.6027 trim_cache size 0 max 16384 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:33.024 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:15:33.024 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:15:33.024 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:33.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd964066f90 msgr2=0x7fd964069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd964066f90 0x7fd964069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd988156810 tx=0x7fd970009040 comp rx=0 tx=0).stop 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 msgr2=0x7fd988155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd988155d40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c4a0 tx=0x7fd97800c960 comp rx=0 tx=0).stop 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 shutdown_connections 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd964066f90 0x7fd964069450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98815e130 0x7fd988153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9880b3480 0x7fd988152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 --2- 172.21.15.87:0/4030512157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9880722d0 0x7fd988155d40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 >> 172.21.15.87:0/4030512157 conn(0x7fd9880669a0 msgr2=0x7fd98806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 shutdown_connections 2024-04-25T10:15:33.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.015+0000 7fd98e000640 1 -- 172.21.15.87:0/4030512157 wait complete. 2024-04-25T10:15:33.039 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:15:33.039 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:33.039 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:15:33.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.171+0000 7f9742200640 1 Processor -- start 2024-04-25T10:15:33.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.171+0000 7f9742200640 1 -- start start 2024-04-25T10:15:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9742200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 0x7f973c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9742200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f973c0b31c0 0x7f973c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9742200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c15b080 con 0x7f973c0aef80 2024-04-25T10:15:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9742200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c15adb0 con 0x7f973c0b31c0 2024-04-25T10:15:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9742200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c15eb60 con 0x7f973c070350 2024-04-25T10:15:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f973c0aef80 0x7f973c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50674/0 (socket says 172.21.15.87:50674) 2024-04-25T10:15:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973b400640 1 -- 172.21.15.87:0/3542508079 learned_addr learned my addr 172.21.15.87:0/3542508079 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332811378 0 0) 0x7f973c15b080 con 0x7f973c0aef80 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9720005740 con 0x7f973c0aef80 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973be00640 1 --2- 172.21.15.87:0/3542508079 >> v2:172.21.15.169:3300/0 conn(0x7f973c0b31c0 0x7f973c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9733e00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 0x7f973c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372524461 0 0) 0x7f9720005740 con 0x7f973c0aef80 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 msgr2=0x7f973c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 0x7f973c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 >> v2:172.21.15.169:3300/0 conn(0x7f973c0b31c0 msgr2=0x7f973c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 --2- 172.21.15.87:0/3542508079 >> v2:172.21.15.169:3300/0 conn(0x7f973c0b31c0 0x7f973c0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c15ee40 con 0x7f973c0aef80 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f972c001680 con 0x7f973c0aef80 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 >> v1:172.21.15.87:6789/0 conn(0x7f973c0aef80 legacy=0x7f973c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 0x7f9720005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9720006150 0x7f9720006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9720006f60 0x7f97200073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c15adb0 con 0x7f9720006150 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c15eb60 con 0x7f9720006f60 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972000dbe0 con 0x7f973c070350 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9733e00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9720006150 0x7f9720006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973b400640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 0x7f9720005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f973be00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9720006f60 0x7f97200073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9733e00640 1 -- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 msgr2=0x7f9720005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9733e00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c070350 0x7f9720005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9733e00640 1 -- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9720006f60 msgr2=0x7f97200073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9733e00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9720006f60 0x7f97200073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.175+0000 7f9733e00640 1 -- 172.21.15.87:0/3542508079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9720005740 con 0x7f9720006150 2024-04-25T10:15:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9733e00640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9720006150 0x7f9720006a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9724009ae0 tx=0x7f972401c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91672ce7d42ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240b1070 con 0x7f9720006150 2024-04-25T10:15:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97240a4b30 con 0x7f9720006150 2024-04-25T10:15:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f973aa00640 1 -- 172.21.15.87:0/3542508079 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240096b0 con 0x7f9720006150 2024-04-25T10:15:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 -- 172.21.15.87:0/3542508079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9720006150 msgr2=0x7f9720006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 --2- 172.21.15.87:0/3542508079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9720006150 0x7f9720006a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9724009ae0 tx=0x7f972401c6f0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f973be00640 1 -- 172.21.15.87:0/3542508079 reap_dead start 2024-04-25T10:15:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 -- 172.21.15.87:0/3542508079 shutdown_connections 2024-04-25T10:15:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 -- 172.21.15.87:0/3542508079 >> 172.21.15.87:0/3542508079 conn(0x7f973c0669a0 msgr2=0x7f973c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 -- 172.21.15.87:0/3542508079 shutdown_connections 2024-04-25T10:15:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 -- 172.21.15.87:0/3542508079 wait complete. 2024-04-25T10:15:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 Processor -- start 2024-04-25T10:15:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.179+0000 7f9742200640 1 -- start start 2024-04-25T10:15:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9720006150 0x7f973c0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f973c0cdc90 0x7f973c0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f973c151eb0 0x7f973c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0cfa70 con 0x7f973c151eb0 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c15bc90 con 0x7f973c0cdc90 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0cfbf0 con 0x7f9720006150 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f973c151eb0 0x7f973c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9733e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f973c0cdc90 0x7f973c0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9720006150 0x7f973c0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f973c151eb0 0x7f973c152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59174/0 (socket says 172.21.15.87:59174) 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 -- 172.21.15.87:0/1064680990 learned_addr learned my addr 172.21.15.87:0/1064680990 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9733e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f973c0cdc90 0x7f973c0ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:36744/0 (socket says 172.21.15.87:36744) 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 -- 172.21.15.87:0/1064680990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9720006150 msgr2=0x7f973c0c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9720006150 0x7f973c0c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 -- 172.21.15.87:0/1064680990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f973c0cdc90 msgr2=0x7f973c0ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f973c0cdc90 0x7f973c0ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 -- 172.21.15.87:0/1064680990 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c152870 con 0x7f973c151eb0 2024-04-25T10:15:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f973be00640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f973c151eb0 0x7f973c152330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f973c0b2f70 tx=0x7f973400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9733400640 1 -- 172.21.15.87:0/1064680990 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400fce0 con 0x7f973c151eb0 2024-04-25T10:15:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9733400640 1 -- 172.21.15.87:0/1064680990 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973401d070 con 0x7f973c151eb0 2024-04-25T10:15:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9733400640 1 -- 172.21.15.87:0/1064680990 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734015670 con 0x7f973c151eb0 2024-04-25T10:15:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c0ce0f0 con 0x7f973c151eb0 2024-04-25T10:15:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0c8150 con 0x7f973c151eb0 2024-04-25T10:15:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.183+0000 7f9733400640 1 -- 172.21.15.87:0/1064680990 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9734015810 con 0x7f973c151eb0 2024-04-25T10:15:33.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f9733400640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9718066f60 0x7f9718069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f9733400640 1 -- 172.21.15.87:0/1064680990 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9734085a30 con 0x7f973c151eb0 2024-04-25T10:15:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9718066f60 0x7f9718069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 Processor -- start 2024-04-25T10:15:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9718066f60 0x7f9718069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f972c0050c0 tx=0x7f972c007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 -- start start 2024-04-25T10:15:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21180 0x565125f21560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21aa0 0x565125ed70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125d24f20 con 0x565125f21180 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125cad000 con 0x565125ed7600 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x565125d20fc0 con 0x565125f21aa0 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21180 0x565125f21560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21180 0x565125f21560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59176/0 (socket says 172.21.15.87:59176) 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 -- 172.21.15.87:0/3312085003 learned_addr learned my addr 172.21.15.87:0/3312085003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f9733e00640 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21aa0 0x565125ed70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973be00640 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 -- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21aa0 msgr2=0x565125ed70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21aa0 0x565125ed70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 -- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 msgr2=0x565125ed79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 -- 172.21.15.87:0/3312085003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565125d00ba0 con 0x565125f21180 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f973b400640 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21180 0x565125f21560 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f972c015ad0 tx=0x7f972c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda521bdb2037f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f972be00640 1 -- 172.21.15.87:0/3312085003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c017a70 con 0x565125f21180 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f972be00640 1 -- 172.21.15.87:0/3312085003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c029070 con 0x565125f21180 2024-04-25T10:15:33.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.187+0000 7f972be00640 1 -- 172.21.15.87:0/3312085003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c017c10 con 0x565125f21180 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- 172.21.15.87:0/3312085003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21180 msgr2=0x565125f21560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21180 0x565125f21560 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f972c015ad0 tx=0x7f972c018000 comp rx=0 tx=0).stop 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- 172.21.15.87:0/3312085003 shutdown_connections 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21aa0 0x565125ed70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 --2- 172.21.15.87:0/3312085003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21180 0x565125f21560 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- 172.21.15.87:0/3312085003 >> 172.21.15.87:0/3312085003 conn(0x565125edd760 msgr2=0x565125ed59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- 172.21.15.87:0/3312085003 shutdown_connections 2024-04-25T10:15:33.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- 172.21.15.87:0/3312085003 wait complete. 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- start start 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 Processor -- start 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125edc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125edcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f0f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125de29c0 con 0x565125f21aa0 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125cdf5d0 con 0x565125ed7600 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x565125cc8360 con 0x565125f21180 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 10 client.0 _ll_register_callbacks cb 0x565125ec2a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:33.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f0f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f0f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59180/0 (socket says 172.21.15.87:59180) 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 -- 172.21.15.87:0/3210644319 learned_addr learned my addr 172.21.15.87:0/3210644319 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f9733e00640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125edcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973b400640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125edc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 -- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 msgr2=0x565125edcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125edcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 -- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 msgr2=0x565125edc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125edc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 -- 172.21.15.87:0/3210644319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565125d6b5a0 con 0x565125f21aa0 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973b400640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125edc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f973be00640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f0f450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9734055f10 tx=0x7f9734056a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edbb5c1d98637fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f9728c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f9728c00640 1 -- 172.21.15.87:0/3210644319 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734082060 con 0x565125f21aa0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f9728c00640 1 -- 172.21.15.87:0/3210644319 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9734055660 con 0x565125f21aa0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f972c029070 con 0x565125f21aa0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f9728c00640 1 -- 172.21.15.87:0/3210644319 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9734055860 con 0x565125f21aa0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 10 client.6039 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.191+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x565125cfc2b0 con 0x565125f21aa0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f9728c00640 1 -- 172.21.15.87:0/3210644319 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c029070 con 0x565125f21aa0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f9728c00640 1 -- 172.21.15.87:0/3210644319 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f97340742a0 con 0x565125f21aa0 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 10 client.6039 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 10 client.6039 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 20 client.6039 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f0ee60 0x565125f0fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 4 client.6039 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565125cc0c70 con 0x565125f0ee60 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f973b400640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f0ee60 0x565125f0fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f973b400640 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f0ee60 0x565125f0fe60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99e7e97029b4c5c server_cookie=cf4e3ffaf75f54b0 in_seq=0 out_seq=0 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f9728c00640 10 client.6039 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f9728c00640 1 -- 172.21.15.87:0/3210644319 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565125cc0c70 con 0x565125f0ee60 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f9728c00640 10 client.6039 handle_command_reply: tid=0 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 client.6039 shutdown 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 msgr2=0x565125f0f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f0f450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9734055f10 tx=0x7f9734056a70 comp rx=0 tx=0).stop 2024-04-25T10:15:33.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 shutdown_connections 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f0f450 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f0ee60 0x565125f0fe60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125edcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 --2- 172.21.15.87:0/3210644319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125edc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f9728c00640 0 client.6039 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 >> 172.21.15.87:0/3210644319 conn(0x565125edd760 msgr2=0x565125ed60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 shutdown_connections 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 1 -- 172.21.15.87:0/3210644319 wait complete. 2024-04-25T10:15:33.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.195+0000 7f97456fc000 20 client.6039 trim_cache size 0 max 16384 2024-04-25T10:15:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 Processor -- start 2024-04-25T10:15:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 -- start start 2024-04-25T10:15:33.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f20b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c201350 con 0x565125f21aa0 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c2004f0 con 0x565125ed7600 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c5da0 con 0x565125f21180 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f20b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f20b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59182/0 (socket says 172.21.15.87:59182) 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 -- 172.21.15.87:0/2194648595 learned_addr learned my addr 172.21.15.87:0/2194648595 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973b400640 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f9733e00640 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 -- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 msgr2=0x565125f205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 -- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 msgr2=0x565125ed6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.207+0000 7f973be00640 1 -- 172.21.15.87:0/2194648595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565125cad250 con 0x565125f21aa0 2024-04-25T10:15:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f20b30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9734000f90 tx=0x7f973406d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcc0d5033cccb68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f972b400640 1 -- 172.21.15.87:0/2194648595 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734056210 con 0x565125f21aa0 2024-04-25T10:15:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f972b400640 1 -- 172.21.15.87:0/2194648595 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734072020 con 0x565125f21aa0 2024-04-25T10:15:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f972b400640 1 -- 172.21.15.87:0/2194648595 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734015d80 con 0x565125f21aa0 2024-04-25T10:15:33.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- 172.21.15.87:0/2194648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 msgr2=0x565125f20b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f20b30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9734000f90 tx=0x7f973406d000 comp rx=0 tx=0).stop 2024-04-25T10:15:33.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- 172.21.15.87:0/2194648595 shutdown_connections 2024-04-25T10:15:33.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f20b30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 --2- 172.21.15.87:0/2194648595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125ed6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- 172.21.15.87:0/2194648595 >> 172.21.15.87:0/2194648595 conn(0x565125edd760 msgr2=0x565125f1fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- 172.21.15.87:0/2194648595 shutdown_connections 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- 172.21.15.87:0/2194648595 wait complete. 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- start start 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 Processor -- start 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125f813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125ce87b0 con 0x565125f21aa0 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c63a0 con 0x565125ed7600 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x565125cdf5d0 con 0x565125f21180 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 10 client.0 _ll_register_callbacks cb 0x565125ea3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f36160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59190/0 (socket says 172.21.15.87:59190) 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 -- 172.21.15.87:0/2520713658 learned_addr learned my addr 172.21.15.87:0/2520713658 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973b400640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125f813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f9733e00640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 -- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 msgr2=0x565125f818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 -- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 msgr2=0x565125f813b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125f813b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 -- 172.21.15.87:0/2520713658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565125cb9b90 con 0x565125f21aa0 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f9733e00640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f973be00640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f36160 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9734055c10 tx=0x7f9734056060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c2a5591ce1a0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f9728c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f9728c00640 1 -- 172.21.15.87:0/2520713658 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734056210 con 0x565125f21aa0 2024-04-25T10:15:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9734072020 con 0x565125f21aa0 2024-04-25T10:15:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f9728c00640 1 -- 172.21.15.87:0/2520713658 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9734055660 con 0x565125f21aa0 2024-04-25T10:15:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.211+0000 7f9728c00640 1 -- 172.21.15.87:0/2520713658 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9734079030 con 0x565125f21aa0 2024-04-25T10:15:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 10 client.6045 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x565125ccc010 con 0x565125f21aa0 2024-04-25T10:15:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f9728c00640 1 -- 172.21.15.87:0/2520713658 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734072020 con 0x565125f21aa0 2024-04-25T10:15:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f9728c00640 1 -- 172.21.15.87:0/2520713658 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f973407d390 con 0x565125f21aa0 2024-04-25T10:15:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 10 client.6045 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 10 client.6045 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 20 client.6045 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f36ac0 0x565125f36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 4 client.6045 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565125f0f7e0 con 0x565125f36ac0 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f973b400640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f36ac0 0x565125f36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f9728c00640 10 client.6045 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f973b400640 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f36ac0 0x565125f36ed0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e58b2dd69538f5 server_cookie=63ee97267a521361 in_seq=0 out_seq=0 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f9728c00640 1 -- 172.21.15.87:0/2520713658 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x565125f0f7e0 con 0x565125f36ac0 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f9728c00640 10 client.6045 handle_command_reply: tid=0 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 client.6045 shutdown 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 msgr2=0x565125f36160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f36160 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9734055c10 tx=0x7f9734056060 comp rx=0 tx=0).stop 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 shutdown_connections 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565125f21aa0 0x565125f36160 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x565125f36ac0 0x565125f36ed0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565125f21180 0x565125f818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 --2- 172.21.15.87:0/2520713658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565125ed7600 0x565125f813b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f9728c00640 0 client.6045 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 >> 172.21.15.87:0/2520713658 conn(0x565125edd760 msgr2=0x565125f20770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 shutdown_connections 2024-04-25T10:15:33.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 1 -- 172.21.15.87:0/2520713658 wait complete. 2024-04-25T10:15:33.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.215+0000 7f97456fc000 20 client.6045 trim_cache size 0 max 16384 2024-04-25T10:15:33.228 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:15:33.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:33.228 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-04-25T10:15:33.228 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 3890914459, 2024-04-25T10:15:33.228 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511630187, 2024-04-25T10:15:33.228 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T10:15:33.228 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:33.229 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:15:33.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9718066f60 msgr2=0x7f9718069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9718066f60 0x7f9718069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f972c0050c0 tx=0x7f972c007900 comp rx=0 tx=0).stop 2024-04-25T10:15:33.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f973c151eb0 msgr2=0x7f973c152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f973c151eb0 0x7f973c152330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f973c0b2f70 tx=0x7f973400f1b0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 shutdown_connections 2024-04-25T10:15:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9718066f60 0x7f9718069420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f973c151eb0 0x7f973c152330 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f973c0cdc90 0x7f973c0ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 --2- 172.21.15.87:0/1064680990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9720006150 0x7f973c0c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 >> 172.21.15.87:0/1064680990 conn(0x7f973c0669a0 msgr2=0x7f973c15a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 shutdown_connections 2024-04-25T10:15:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.219+0000 7f9742200640 1 -- 172.21.15.87:0/1064680990 wait complete. 2024-04-25T10:15:33.245 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:33.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0719e00640 1 Processor -- start 2024-04-25T10:15:33.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0719e00640 1 -- start start 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0719e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07141550b0 0x7f07140cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0719e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f071415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0719e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0714071050 con 0x7f071415b740 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0719e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071415d040 con 0x7f07141550b0 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0719e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07140712f0 con 0x7f07140cfe70 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0712a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07141550b0 0x7f07140cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0712a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07141550b0 0x7f07140cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59192/0 (socket says 172.21.15.87:59192) 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0712a00640 1 -- 172.21.15.87:0/557330689 learned_addr learned my addr 172.21.15.87:0/557330689 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0712a00640 1 -- 172.21.15.87:0/557330689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 msgr2=0x7f071415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f070a000640 1 --2- 172.21.15.87:0/557330689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f071415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0712a00640 1 --2- 172.21.15.87:0/557330689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f071415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0712a00640 1 -- 172.21.15.87:0/557330689 >> v1:172.21.15.169:6789/0 conn(0x7f071415b740 legacy=0x7f071406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f0712a00640 1 -- 172.21.15.87:0/557330689 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07140715d0 con 0x7f07141550b0 2024-04-25T10:15:33.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.375+0000 7f070a000640 1 --2- 172.21.15.87:0/557330689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f071415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0712a00640 1 --2- 172.21.15.87:0/557330689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07141550b0 0x7f07140cf930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f070400bb10 tx=0x7f070401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a397a00de0faa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0712000640 1 -- 172.21.15.87:0/557330689 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07040b4070 con 0x7f07141550b0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0712000640 1 -- 172.21.15.87:0/557330689 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07040a7a90 con 0x7f07141550b0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0712000640 1 -- 172.21.15.87:0/557330689 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07040a7e60 con 0x7f07141550b0 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 -- 172.21.15.87:0/557330689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07141550b0 msgr2=0x7f07140cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 --2- 172.21.15.87:0/557330689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07141550b0 0x7f07140cf930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f070400bb10 tx=0x7f070401ebf0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 -- 172.21.15.87:0/557330689 shutdown_connections 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 --2- 172.21.15.87:0/557330689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f071415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 --2- 172.21.15.87:0/557330689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07141550b0 0x7f07140cf930 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 -- 172.21.15.87:0/557330689 >> 172.21.15.87:0/557330689 conn(0x7f0714066b70 msgr2=0x7f0714066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 -- 172.21.15.87:0/557330689 shutdown_connections 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.379+0000 7f0719e00640 1 -- 172.21.15.87:0/557330689 wait complete. 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 Processor -- start 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 -- start start 2024-04-25T10:15:33.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f07141f9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07141550b0 0x7f07140b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f071415b740 0x7f0714135630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071415bd50 con 0x7f071415b740 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071415bbd0 con 0x7f07141550b0 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071415c030 con 0x7f07140cfe70 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f071415b740 0x7f0714135630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f070a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07141550b0 0x7f07140b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f071415b740 0x7f0714135630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59206/0 (socket says 172.21.15.87:59206) 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 -- 172.21.15.87:0/2834279461 learned_addr learned my addr 172.21.15.87:0/2834279461 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0712a00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f07141f9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 -- 172.21.15.87:0/2834279461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 msgr2=0x7f07141f9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f07141f9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 -- 172.21.15.87:0/2834279461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07141550b0 msgr2=0x7f07140b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07141550b0 0x7f07140b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 -- 172.21.15.87:0/2834279461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071414d120 con 0x7f071415b740 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0713400640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f071415b740 0x7f0714135630 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f070c00b9a0 tx=0x7f070c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f070be00640 1 -- 172.21.15.87:0/2834279461 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c019070 con 0x7f071415b740 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f070be00640 1 -- 172.21.15.87:0/2834279461 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c00cba0 con 0x7f071415b740 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f070be00640 1 -- 172.21.15.87:0/2834279461 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c014440 con 0x7f071415b740 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07141496b0 con 0x7f071415b740 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07141fa410 con 0x7f071415b740 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.383+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071404ef30 con 0x7f071415b740 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.387+0000 7f070be00640 1 -- 172.21.15.87:0/2834279461 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f070c01c070 con 0x7f071415b740 2024-04-25T10:15:33.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.387+0000 7f070be00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f06f0066f60 0x7f06f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.387+0000 7f070be00640 1 -- 172.21.15.87:0/2834279461 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f070c084be0 con 0x7f071415b740 2024-04-25T10:15:33.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.387+0000 7f0712a00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f06f0066f60 0x7f06f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.387+0000 7f0712a00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f06f0066f60 0x7f06f0069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f07040b1000 tx=0x7f07040a9070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.391+0000 7f070be00640 1 -- 172.21.15.87:0/2834279461 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f070c055500 con 0x7f071415b740 2024-04-25T10:15:33.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f071414c290 con 0x7f071415b740 2024-04-25T10:15:33.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f070be00640 1 -- 172.21.15.87:0/2834279461 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f070c0595a0 con 0x7f071415b740 2024-04-25T10:15:33.585 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:33.585 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:33.585 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:33.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f06f0066f60 msgr2=0x7f06f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f06f0066f60 0x7f06f0069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f07040b1000 tx=0x7f07040a9070 comp rx=0 tx=0).stop 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f071415b740 msgr2=0x7f0714135630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f071415b740 0x7f0714135630 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f070c00b9a0 tx=0x7f070c00be60 comp rx=0 tx=0).stop 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 shutdown_connections 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f06f0066f60 0x7f06f0069420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f071415b740 0x7f0714135630 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07141550b0 0x7f07140b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 --2- 172.21.15.87:0/2834279461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07140cfe70 0x7f07141f9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 >> 172.21.15.87:0/2834279461 conn(0x7f0714066b70 msgr2=0x7f071414f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 shutdown_connections 2024-04-25T10:15:33.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.579+0000 7f0719e00640 1 -- 172.21.15.87:0/2834279461 wait complete. 2024-04-25T10:15:33.605 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 3890914459 2024-04-25T10:15:33.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.735+0000 7f220d200640 1 Processor -- start 2024-04-25T10:15:33.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.735+0000 7f220d200640 1 -- start start 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f220d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2208070350 0x7f220815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f220d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22080b31c0 0x7f22080b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f220d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220815b080 con 0x7f22080aef80 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f220d200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220815adb0 con 0x7f2208070350 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f220d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220815eb60 con 0x7f22080b31c0 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f21fd400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2208070350 0x7f220815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f21fd400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2208070350 0x7f220815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59216/0 (socket says 172.21.15.87:59216) 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f21fd400640 1 -- 172.21.15.87:0/1856252811 learned_addr learned my addr 172.21.15.87:0/1856252811 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f2206800640 1 --2- 172.21.15.87:0/1856252811 >> v2:172.21.15.169:3300/0 conn(0x7f22080b31c0 0x7f22080b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f21fd400640 1 -- 172.21.15.87:0/1856252811 >> v2:172.21.15.169:3300/0 conn(0x7f22080b31c0 msgr2=0x7f22080b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f21fd400640 1 --2- 172.21.15.87:0/1856252811 >> v2:172.21.15.169:3300/0 conn(0x7f22080b31c0 0x7f22080b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f21fd400640 1 -- 172.21.15.87:0/1856252811 >> v1:172.21.15.169:6789/0 conn(0x7f22080aef80 legacy=0x7f220815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.739+0000 7f21fd400640 1 -- 172.21.15.87:0/1856252811 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220815ee40 con 0x7f2208070350 2024-04-25T10:15:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f21fd400640 1 --2- 172.21.15.87:0/1856252811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f220815e480 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f22080d0750 tx=0x7f21f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90a970b885dea88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f2205400640 1 -- 172.21.15.87:0/1856252811 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00ab430 con 0x7f2208070350 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f2205400640 1 -- 172.21.15.87:0/1856252811 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f00b0070 con 0x7f2208070350 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f2205400640 1 -- 172.21.15.87:0/1856252811 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00ab7f0 con 0x7f2208070350 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 -- 172.21.15.87:0/1856252811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 msgr2=0x7f220815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 --2- 172.21.15.87:0/1856252811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f220815e480 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f22080d0750 tx=0x7f21f001c520 comp rx=0 tx=0).stop 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 -- 172.21.15.87:0/1856252811 shutdown_connections 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 --2- 172.21.15.87:0/1856252811 >> v2:172.21.15.169:3300/0 conn(0x7f22080b31c0 0x7f22080b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 --2- 172.21.15.87:0/1856252811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f220815e480 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 -- 172.21.15.87:0/1856252811 >> 172.21.15.87:0/1856252811 conn(0x7f22080669a0 msgr2=0x7f2208066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 -- 172.21.15.87:0/1856252811 shutdown_connections 2024-04-25T10:15:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 -- 172.21.15.87:0/1856252811 wait complete. 2024-04-25T10:15:33.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 Processor -- start 2024-04-25T10:15:33.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.743+0000 7f220d200640 1 -- start start 2024-04-25T10:15:33.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f2208146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080aef80 0x7f22080cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22080b31c0 0x7f22080cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22081679a0 con 0x7f2208070350 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2208167820 con 0x7f22080aef80 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2208167b20 con 0x7f22080b31c0 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f2208146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f2208146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59228/0 (socket says 172.21.15.87:59228) 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 -- 172.21.15.87:0/1712530573 learned_addr learned my addr 172.21.15.87:0/1712530573 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f21fd400640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080aef80 0x7f22080cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2206800640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22080b31c0 0x7f22080cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 -- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22080b31c0 msgr2=0x7f22080cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22080b31c0 0x7f22080cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 -- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080aef80 msgr2=0x7f22080cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080aef80 0x7f22080cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 -- 172.21.15.87:0/1712530573 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220815f520 con 0x7f2208070350 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f2205e00640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f2208146b90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c7c0 tx=0x7f21f800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f21fea00640 1 -- 172.21.15.87:0/1712530573 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8016020 con 0x7f2208070350 2024-04-25T10:15:33.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f21fea00640 1 -- 172.21.15.87:0/1712530573 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8006d60 con 0x7f2208070350 2024-04-25T10:15:33.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f21fea00640 1 -- 172.21.15.87:0/1712530573 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8003bc0 con 0x7f2208070350 2024-04-25T10:15:33.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22080b2650 con 0x7f2208070350 2024-04-25T10:15:33.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22081446c0 con 0x7f2208070350 2024-04-25T10:15:33.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.747+0000 7f21fea00640 1 -- 172.21.15.87:0/1712530573 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21f800a850 con 0x7f2208070350 2024-04-25T10:15:33.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21fea00640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e4066f60 0x7f21e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21fea00640 1 -- 172.21.15.87:0/1712530573 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21f8084c20 con 0x7f2208070350 2024-04-25T10:15:33.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21fd400640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e4066f60 0x7f21e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 Processor -- start 2024-04-25T10:15:33.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 -- start start 2024-04-25T10:15:33.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21fd400640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e4066f60 0x7f21e4069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af000 tx=0x7f21f00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd9d070 0x55785fd9d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d990 0x55785fd506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd50be0 0x55785fd50fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fba0f00 con 0x55785fd50be0 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fb294a0 con 0x55785fd9d070 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55785fb9cfa0 con 0x55785fd9d990 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd50be0 0x55785fd50fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd50be0 0x55785fd50fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59236/0 (socket says 172.21.15.87:59236) 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 -- 172.21.15.87:0/2039362447 learned_addr learned my addr 172.21.15.87:0/2039362447 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21fd400640 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d990 0x55785fd506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2205e00640 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd9d070 0x55785fd9d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 -- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d990 msgr2=0x55785fd506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d990 0x55785fd506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 -- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd9d070 msgr2=0x55785fd9d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd9d070 0x55785fd9d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 -- 172.21.15.87:0/2039362447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55785fb26d00 con 0x55785fd50be0 2024-04-25T10:15:33.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f2206800640 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd50be0 0x55785fd50fc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f22000078d0 tx=0x7f2200007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a750324868cb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21f6a00640 1 -- 172.21.15.87:0/2039362447 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200010a40 con 0x55785fd50be0 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21f6a00640 1 -- 172.21.15.87:0/2039362447 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200010be0 con 0x55785fd50be0 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.751+0000 7f21f6a00640 1 -- 172.21.15.87:0/2039362447 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200004030 con 0x55785fd50be0 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- 172.21.15.87:0/2039362447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd50be0 msgr2=0x55785fd50fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd50be0 0x55785fd50fc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f22000078d0 tx=0x7f2200007d90 comp rx=0 tx=0).stop 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- 172.21.15.87:0/2039362447 shutdown_connections 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd50be0 0x55785fd50fc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d990 0x55785fd506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 --2- 172.21.15.87:0/2039362447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd9d070 0x55785fd9d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- 172.21.15.87:0/2039362447 >> 172.21.15.87:0/2039362447 conn(0x55785fd7de10 msgr2=0x55785fd80200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- 172.21.15.87:0/2039362447 shutdown_connections 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- 172.21.15.87:0/2039362447 wait complete. 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- start start 2024-04-25T10:15:33.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 Processor -- start 2024-04-25T10:15:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fda43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fd56a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fba5530 con 0x55785fd9d990 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fb5bae0 con 0x55785fd50be0 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55785fb5fe10 con 0x55785fd9d070 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 10 client.0 _ll_register_callbacks cb 0x55785fd3e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fd56a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fd56a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59242/0 (socket says 172.21.15.87:59242) 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 -- 172.21.15.87:0/3373848010 learned_addr learned my addr 172.21.15.87:0/3373848010 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2205e00640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fda43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f21fd400640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 -- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 msgr2=0x55785fda4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 -- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 msgr2=0x55785fda43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fda43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 -- 172.21.15.87:0/3373848010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55785fb43a10 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2205e00640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fda43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2206800640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fd56a90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2200010620 tx=0x7f2200004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce4e8b7dd42c2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f21eb400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f21eb400640 1 -- 172.21.15.87:0/3373848010 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200010a40 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f21eb400640 1 -- 172.21.15.87:0/3373848010 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f220000f050 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2200010be0 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f21eb400640 1 -- 172.21.15.87:0/3373848010 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2200030050 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 10 client.6063 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55785fb9bef0 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f21eb400640 1 -- 172.21.15.87:0/3373848010 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200010be0 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f21eb400640 1 -- 172.21.15.87:0/3373848010 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2200033020 con 0x55785fd9d990 2024-04-25T10:15:33.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 10 client.6063 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.755+0000 7f2210693000 10 client.6063 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 20 client.6063 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fd57800 0x55785fd57be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 4 client.6063 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55785fb3cb20 con 0x55785fd57800 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2205e00640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fd57800 0x55785fd57be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2205e00640 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fd57800 0x55785fd57be0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9170dcdecbab286 server_cookie=bc9f0528e59e4785 in_seq=0 out_seq=0 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f21eb400640 10 client.6063 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f21eb400640 1 -- 172.21.15.87:0/3373848010 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55785fb3cb20 con 0x55785fd57800 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f21eb400640 10 client.6063 handle_command_reply: tid=0 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 client.6063 shutdown 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 msgr2=0x55785fd56a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fd56a90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2200010620 tx=0x7f2200004740 comp rx=0 tx=0).stop 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 shutdown_connections 2024-04-25T10:15:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fd56a90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fd57800 0x55785fd57be0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 --2- 172.21.15.87:0/3373848010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fda43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 >> 172.21.15.87:0/3373848010 conn(0x55785fd7de10 msgr2=0x55785fd4fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f21eb400640 0 client.6063 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 shutdown_connections 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 1 -- 172.21.15.87:0/3373848010 wait complete. 2024-04-25T10:15:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.759+0000 7f2210693000 20 client.6063 trim_cache size 0 max 16384 2024-04-25T10:15:33.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 Processor -- start 2024-04-25T10:15:33.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 -- start start 2024-04-25T10:15:33.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fdf9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fdfa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fdfa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080b0180 con 0x55785fd9d990 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080af580 con 0x55785fd50be0 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22080afb80 con 0x55785fd9d070 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fdfa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fdfa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59244/0 (socket says 172.21.15.87:59244) 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 -- 172.21.15.87:0/786635555 learned_addr learned my addr 172.21.15.87:0/786635555 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2205e00640 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fdf9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f21fd400640 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fdfa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 -- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 msgr2=0x55785fdfa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fdfa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 -- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 msgr2=0x55785fdf9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fdf9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 -- 172.21.15.87:0/786635555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55785fd9d7c0 con 0x55785fd9d990 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2205e00640 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fdf9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:33.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.771+0000 7f2206800640 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fdfa8f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c10 tx=0x7f2200020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e2d671274e8a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f21f6000640 1 -- 172.21.15.87:0/786635555 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200039070 con 0x55785fd9d990 2024-04-25T10:15:33.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f21f6000640 1 -- 172.21.15.87:0/786635555 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200000e50 con 0x55785fd9d990 2024-04-25T10:15:33.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f21f6000640 1 -- 172.21.15.87:0/786635555 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200004340 con 0x55785fd9d990 2024-04-25T10:15:33.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- 172.21.15.87:0/786635555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 msgr2=0x55785fdfa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fdfa8f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c10 tx=0x7f2200020070 comp rx=0 tx=0).stop 2024-04-25T10:15:33.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- 172.21.15.87:0/786635555 shutdown_connections 2024-04-25T10:15:33.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fdfa8f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fdfa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 --2- 172.21.15.87:0/786635555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fdf9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- 172.21.15.87:0/786635555 >> 172.21.15.87:0/786635555 conn(0x55785fd7de10 msgr2=0x55785fd56870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- 172.21.15.87:0/786635555 shutdown_connections 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- 172.21.15.87:0/786635555 wait complete. 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- start start 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 Processor -- start 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fd4f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fda84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fbe0ca0 con 0x55785fd9d990 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fb6db90 con 0x55785fd50be0 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55785fd5c1c0 con 0x55785fd9d070 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 10 client.0 _ll_register_callbacks cb 0x7f22000074c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fda84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fda84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59254/0 (socket says 172.21.15.87:59254) 2024-04-25T10:15:33.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 -- 172.21.15.87:0/2209325120 learned_addr learned my addr 172.21.15.87:0/2209325120 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2205e00640 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fd4f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f21fd400640 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 -- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 msgr2=0x55785fda7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 -- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 msgr2=0x55785fd4f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fd4f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 -- 172.21.15.87:0/2209325120 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55785fb60790 con 0x55785fd9d990 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2206800640 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fda84e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55785f8c6160 tx=0x7f2200008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5299b3228754010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f21eb400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f21eb400640 1 -- 172.21.15.87:0/2209325120 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200039040 con 0x55785fd9d990 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f21eb400640 1 -- 172.21.15.87:0/2209325120 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f220000f070 con 0x55785fd9d990 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.775+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2200000e50 con 0x55785fd9d990 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f21eb400640 1 -- 172.21.15.87:0/2209325120 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2200048050 con 0x55785fd9d990 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 10 client.6069 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:33.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55785fb68da0 con 0x55785fd9d990 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f21eb400640 1 -- 172.21.15.87:0/2209325120 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200000e50 con 0x55785fd9d990 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f21eb400640 1 -- 172.21.15.87:0/2209325120 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f220004b020 con 0x55785fd9d990 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 10 client.6069 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 10 client.6069 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 20 client.6069 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fdfd4d0 0x55785fdfd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 4 client.6069 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3890914459}] 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3890914459}) v1 -- 0x55785fbae440 con 0x55785fdfd4d0 2024-04-25T10:15:33.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2205e00640 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fdfd4d0 0x55785fdfd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2205e00640 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fdfd4d0 0x55785fdfd8b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5340369b6c82b771 server_cookie=5bf68d13bf7768b6 in_seq=0 out_seq=0 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f21eb400640 10 client.6069 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f21eb400640 1 -- 172.21.15.87:0/2209325120 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55785fbae440 con 0x55785fdfd4d0 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f21eb400640 10 client.6069 handle_command_reply: tid=0 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 client.6069 shutdown 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 msgr2=0x55785fda84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fda84e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55785f8c6160 tx=0x7f2200008040 comp rx=0 tx=0).stop 2024-04-25T10:15:33.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 shutdown_connections 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55785fd9d990 0x55785fda84e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55785fdfd4d0 0x55785fdfd8b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55785fd9d070 0x55785fda7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 --2- 172.21.15.87:0/2209325120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55785fd50be0 0x55785fd4f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 >> 172.21.15.87:0/2209325120 conn(0x55785fd7de10 msgr2=0x55785fd4ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f21eb400640 0 client.6069 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 shutdown_connections 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 1 -- 172.21.15.87:0/2209325120 wait complete. 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f2210693000 20 client.6069 trim_cache size 0 max 16384 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e4066f60 msgr2=0x7f21e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e4066f60 0x7f21e4069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af000 tx=0x7f21f00a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 msgr2=0x7f2208146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f2208146b90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c7c0 tx=0x7f21f800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 shutdown_connections 2024-04-25T10:15:33.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e4066f60 0x7f21e4069420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22080b31c0 0x7f22080cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080aef80 0x7f22080cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 --2- 172.21.15.87:0/1712530573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2208070350 0x7f2208146b90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 >> 172.21.15.87:0/1712530573 conn(0x7f22080669a0 msgr2=0x7f220815d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 shutdown_connections 2024-04-25T10:15:33.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.779+0000 7f220d200640 1 -- 172.21.15.87:0/1712530573 wait complete. 2024-04-25T10:15:33.804 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:15:33.805 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:33.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f0116200640 1 Processor -- start 2024-04-25T10:15:33.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f0116200640 1 -- start start 2024-04-25T10:15:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f0116200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 0x7f01100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f0116200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0110070be0 con 0x7f0110073d10 2024-04-25T10:15:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f0116200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0110074280 con 0x7f0110071960 2024-04-25T10:15:33.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f0116200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011015e270 con 0x7f011006e0a0 2024-04-25T10:15:33.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0110073d10 0x7f011006e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50688/0 (socket says 172.21.15.87:50688) 2024-04-25T10:15:33.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010ea00640 1 -- 172.21.15.87:0/3099686760 learned_addr learned my addr 172.21.15.87:0/3099686760 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:33.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010f400640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 0x7f01100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528620572 0 0) 0x7f0110070be0 con 0x7f0110073d10 2024-04-25T10:15:33.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f4003530 con 0x7f0110073d10 2024-04-25T10:15:33.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1797111498 0 0) 0x7f00f4003530 con 0x7f0110073d10 2024-04-25T10:15:33.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 msgr2=0x7f01100b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 0x7f01100b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 >> v1:172.21.15.169:6789/0 conn(0x7f0110071960 legacy=0x7f0110071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0110070e80 con 0x7f0110073d10 2024-04-25T10:15:33.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f00fc001680 con 0x7f0110073d10 2024-04-25T10:15:33.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 >> v1:172.21.15.87:6789/0 conn(0x7f0110073d10 legacy=0x7f011006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f40039f0 0x7f00f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00f40043a0 0x7f00f4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 0x7f00f40050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f400b9d0 con 0x7f00f40039f0 2024-04-25T10:15:33.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f400bb50 con 0x7f00f40043a0 2024-04-25T10:15:33.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f400bd50 con 0x7f011006e0a0 2024-04-25T10:15:33.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010f400640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f40039f0 0x7f00f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010ea00640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00f40043a0 0x7f00f4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010fe00640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 0x7f00f40050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010f400640 1 -- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 msgr2=0x7f00f40050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010f400640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011006e0a0 0x7f00f40050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010f400640 1 -- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00f40043a0 msgr2=0x7f00f4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010f400640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00f40043a0 0x7f00f4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.935+0000 7f010f400640 1 -- 172.21.15.87:0/3099686760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4003530 con 0x7f00f40039f0 2024-04-25T10:15:33.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f010f400640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f40039f0 0x7f00f4003dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0100009ae0 tx=0x7f010001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea8b0d4cf42dceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01000b2070 con 0x7f00f40039f0 2024-04-25T10:15:33.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01000a5660 con 0x7f00f40039f0 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f010e000640 1 -- 172.21.15.87:0/3099686760 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01000096b0 con 0x7f00f40039f0 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f0116200640 1 -- 172.21.15.87:0/3099686760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f40039f0 msgr2=0x7f00f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f0116200640 1 --2- 172.21.15.87:0/3099686760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f40039f0 0x7f00f4003dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0100009ae0 tx=0x7f010001ca70 comp rx=0 tx=0).stop 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f010fe00640 1 -- 172.21.15.87:0/3099686760 reap_dead start 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f0116200640 1 -- 172.21.15.87:0/3099686760 shutdown_connections 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f0116200640 1 -- 172.21.15.87:0/3099686760 >> 172.21.15.87:0/3099686760 conn(0x7f01100d00a0 msgr2=0x7f01100d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f0116200640 1 -- 172.21.15.87:0/3099686760 shutdown_connections 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.939+0000 7f0116200640 1 -- 172.21.15.87:0/3099686760 wait complete. 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 Processor -- start 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 -- start start 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00f40039f0 0x7f0110160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f011015b0e0 0x7f011014b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f011015b560 0x7f011014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011006f480 con 0x7f011015b560 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100714d0 con 0x7f011015b0e0 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011006f600 con 0x7f00f40039f0 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f011015b560 0x7f011014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f011015b560 0x7f011014bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59274/0 (socket says 172.21.15.87:59274) 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 -- 172.21.15.87:0/1940881714 learned_addr learned my addr 172.21.15.87:0/1940881714 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010f400640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00f40039f0 0x7f0110160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010ea00640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f011015b0e0 0x7f011014b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 -- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00f40039f0 msgr2=0x7f0110160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00f40039f0 0x7f0110160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 -- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f011015b0e0 msgr2=0x7f011014b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f011015b0e0 0x7f011014b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 -- 172.21.15.87:0/1940881714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011015fcd0 con 0x7f011015b560 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f010fe00640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f011015b560 0x7f011014bde0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f011006ddd0 tx=0x7f0108011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0107400640 1 -- 172.21.15.87:0/1940881714 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108003b90 con 0x7f011015b560 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0107400640 1 -- 172.21.15.87:0/1940881714 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010801b070 con 0x7f011015b560 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0107400640 1 -- 172.21.15.87:0/1940881714 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800f7c0 con 0x7f011015b560 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011015ffb0 con 0x7f011015b560 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.943+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0110160570 con 0x7f011015b560 2024-04-25T10:15:33.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.947+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011004ef30 con 0x7f011015b560 2024-04-25T10:15:33.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.947+0000 7f0107400640 1 -- 172.21.15.87:0/1940881714 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0108013040 con 0x7f011015b560 2024-04-25T10:15:33.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.947+0000 7f0107400640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00f0066f60 0x7f00f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.947+0000 7f0107400640 1 -- 172.21.15.87:0/1940881714 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0108087fa0 con 0x7f011015b560 2024-04-25T10:15:33.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.947+0000 7f010f400640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00f0066f60 0x7f00f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.947+0000 7f010f400640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00f0066f60 0x7f00f0069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0100009ae0 tx=0x7f01000046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:33.951+0000 7f0107400640 1 -- 172.21.15.87:0/1940881714 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0108058f10 con 0x7f011015b560 2024-04-25T10:15:34.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.135+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0110072ea0 con 0x7f011015b560 2024-04-25T10:15:34.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.135+0000 7f0107400640 1 -- 172.21.15.87:0/1940881714 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0108058810 con 0x7f011015b560 2024-04-25T10:15:34.143 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:34.144 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:34.144 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00f0066f60 msgr2=0x7f00f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00f0066f60 0x7f00f0069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0100009ae0 tx=0x7f01000046d0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f011015b560 msgr2=0x7f011014bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f011015b560 0x7f011014bde0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f011006ddd0 tx=0x7f0108011f30 comp rx=0 tx=0).stop 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 shutdown_connections 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00f0066f60 0x7f00f0069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f011015b560 0x7f011014bde0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f011015b0e0 0x7f011014b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 --2- 172.21.15.87:0/1940881714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00f40039f0 0x7f0110160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 >> 172.21.15.87:0/1940881714 conn(0x7f01100d00a0 msgr2=0x7f011006e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 shutdown_connections 2024-04-25T10:15:34.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.139+0000 7f0116200640 1 -- 172.21.15.87:0/1940881714 wait complete. 2024-04-25T10:15:34.164 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:15:34.315 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:34.316 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:34.316 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:34.331 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:34.331 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:34.331 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:15:34.331 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:15:34.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:34.331 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> import stat 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> import json 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> import sys 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> except OSError as e: 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> sys.exit(e.errno) 2024-04-25T10:15:34.342 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:15:34.343 DEBUG:teuthology.orchestra.run.smithi087:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:15:34.343 DEBUG:teuthology.orchestra.run.smithi087:> print(json.dumps( 2024-04-25T10:15:34.343 DEBUG:teuthology.orchestra.run.smithi087:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:15:34.343 DEBUG:teuthology.orchestra.run.smithi087:> indent=2)) 2024-04-25T10:15:34.343 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:15:34.450 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_mode": 16893, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_ino": 1099511627777, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_dev": 44, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_nlink": 16, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_uid": 1000, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_gid": 1271, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_size": 18, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_atime": 1714040119.0320895, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_mtime": 1714040126.3518775, 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout: "st_ctime": 1714040126.3518775 2024-04-25T10:15:34.451 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:34.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:15:34.460 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T10:15:34.460 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:15:34.460 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:34.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.631+0000 7f46ef800640 1 Processor -- start 2024-04-25T10:15:34.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.631+0000 7f46ef800640 1 -- start start 2024-04-25T10:15:34.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46ef800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f46e815e050 0x7f46e80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46ef800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e80b1af0 con 0x7f46e80d1820 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46ef800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e80b1dd0 con 0x7f46e80b13e0 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46ef800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e80b20b0 con 0x7f46e815e050 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f46e815e050 0x7f46e80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46ece00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f46e80d1820 0x7f46e80d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50696/0 (socket says 172.21.15.87:50696) 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46ece00640 1 -- 172.21.15.87:0/2337376680 learned_addr learned my addr 172.21.15.87:0/2337376680 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46dfe00640 1 -- 172.21.15.87:0/2337376680 >> v1:172.21.15.169:6789/0 conn(0x7f46e80b13e0 legacy=0x7f46e80cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46dfe00640 1 -- 172.21.15.87:0/2337376680 >> v1:172.21.15.87:6789/0 conn(0x7f46e80d1820 legacy=0x7f46e80d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46dfe00640 1 -- 172.21.15.87:0/2337376680 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80b2390 con 0x7f46e815e050 2024-04-25T10:15:34.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46e7e00640 1 -- 172.21.15.87:0/2337376680 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152095545 0 0) 0x7f46e80b1af0 con 0x7f46e80d1820 2024-04-25T10:15:34.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46dfe00640 1 --2- 172.21.15.87:0/2337376680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e815e050 0x7f46e80b0ea0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f46d00097c0 tx=0x7f46d001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e177dbc782205794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46e7e00640 1 -- 172.21.15.87:0/2337376680 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00b1070 con 0x7f46e815e050 2024-04-25T10:15:34.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46e7e00640 1 -- 172.21.15.87:0/2337376680 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d00ac410 con 0x7f46e815e050 2024-04-25T10:15:34.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.635+0000 7f46e7e00640 1 -- 172.21.15.87:0/2337376680 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00ac7e0 con 0x7f46e815e050 2024-04-25T10:15:34.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- 172.21.15.87:0/2337376680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e815e050 msgr2=0x7f46e80b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 --2- 172.21.15.87:0/2337376680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e815e050 0x7f46e80b0ea0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f46d00097c0 tx=0x7f46d001c410 comp rx=0 tx=0).stop 2024-04-25T10:15:34.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- 172.21.15.87:0/2337376680 shutdown_connections 2024-04-25T10:15:34.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 --2- 172.21.15.87:0/2337376680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e815e050 0x7f46e80b0ea0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- 172.21.15.87:0/2337376680 >> 172.21.15.87:0/2337376680 conn(0x7f46e809a9d0 msgr2=0x7f46e809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:34.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- 172.21.15.87:0/2337376680 shutdown_connections 2024-04-25T10:15:34.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- 172.21.15.87:0/2337376680 wait complete. 2024-04-25T10:15:34.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 Processor -- start 2024-04-25T10:15:34.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- start start 2024-04-25T10:15:34.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46e80b13e0 0x7f46e8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e80d1820 0x7f46e8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46e815e050 0x7f46e815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80d0fa0 con 0x7f46e80d1820 2024-04-25T10:15:34.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80d0e20 con 0x7f46e815e050 2024-04-25T10:15:34.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46ef800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e816b9b0 con 0x7f46e80b13e0 2024-04-25T10:15:34.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e80d1820 0x7f46e8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e80d1820 0x7f46e8155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59294/0 (socket says 172.21.15.87:59294) 2024-04-25T10:15:34.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 -- 172.21.15.87:0/4018685873 learned_addr learned my addr 172.21.15.87:0/4018685873 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:34.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 -- 172.21.15.87:0/4018685873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46e80b13e0 msgr2=0x7f46e8155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46e80b13e0 0x7f46e8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 -- 172.21.15.87:0/4018685873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46e815e050 msgr2=0x7f46e815f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46e815e050 0x7f46e815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.639+0000 7f46dfe00640 1 -- 172.21.15.87:0/4018685873 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80d27a0 con 0x7f46e80d1820 2024-04-25T10:15:34.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46dfe00640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e80d1820 0x7f46e8155fe0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f46d0019020 tx=0x7f46d00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46e5600640 1 -- 172.21.15.87:0/4018685873 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00b9020 con 0x7f46e80d1820 2024-04-25T10:15:34.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46e5600640 1 -- 172.21.15.87:0/4018685873 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d00ace20 con 0x7f46e80d1820 2024-04-25T10:15:34.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e806c7e0 con 0x7f46e80d1820 2024-04-25T10:15:34.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46e5600640 1 -- 172.21.15.87:0/4018685873 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00b1040 con 0x7f46e80d1820 2024-04-25T10:15:34.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e81565e0 con 0x7f46e80d1820 2024-04-25T10:15:34.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e804ef30 con 0x7f46e80d1820 2024-04-25T10:15:34.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46e5600640 1 -- 172.21.15.87:0/4018685873 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46d00bc020 con 0x7f46e80d1820 2024-04-25T10:15:34.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46e5600640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46c4066f60 0x7f46c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46e5600640 1 -- 172.21.15.87:0/4018685873 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46d011e960 con 0x7f46e80d1820 2024-04-25T10:15:34.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.643+0000 7f46ece00640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46c4066f60 0x7f46c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.647+0000 7f46ece00640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46c4066f60 0x7f46c4069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000c10 tx=0x7f46d8007910 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.647+0000 7f46e5600640 1 -- 172.21.15.87:0/4018685873 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46d00c3030 con 0x7f46e80d1820 2024-04-25T10:15:34.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e80d1ff0 con 0x7f46e80d1820 2024-04-25T10:15:34.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46e5600640 1 -- 172.21.15.87:0/4018685873 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f46d0016090 con 0x7f46e80d1820 2024-04-25T10:15:34.841 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:34.841 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:34.842 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46c4066f60 msgr2=0x7f46c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46c4066f60 0x7f46c4069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000c10 tx=0x7f46d8007910 comp rx=0 tx=0).stop 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e80d1820 msgr2=0x7f46e8155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e80d1820 0x7f46e8155fe0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f46d0019020 tx=0x7f46d00191b0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 shutdown_connections 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46c4066f60 0x7f46c4069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46e815e050 0x7f46e815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46e80d1820 0x7f46e8155fe0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 --2- 172.21.15.87:0/4018685873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46e80b13e0 0x7f46e8155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 >> 172.21.15.87:0/4018685873 conn(0x7f46e809a9d0 msgr2=0x7f46e815cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 shutdown_connections 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:34.835+0000 7f46ef800640 1 -- 172.21.15.87:0/4018685873 wait complete. 2024-04-25T10:15:34.862 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:15:35.009 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:35.009 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": -2 2024-04-25T10:15:35.009 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:35.024 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:15:35.024 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:15:35.024 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:35.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:15:35.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5211c00640 1 Processor -- start 2024-04-25T10:15:35.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5211c00640 1 -- start start 2024-04-25T10:15:35.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5211c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f520c159540 0x7f520c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5211c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c159920 con 0x7f520c0aebd0 2024-04-25T10:15:35.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5211c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c15c2f0 con 0x7f520c0721f0 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5211c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520c15c5d0 con 0x7f520c159540 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5202000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f520c0aebd0 0x7f520c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50702/0 (socket says 172.21.15.87:50702) 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f5202000640 1 -- 172.21.15.87:0/1024365722 learned_addr learned my addr 172.21.15.87:0/1024365722 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f520aa00640 1 --2- 172.21.15.87:0/1024365722 >> v2:172.21.15.169:3300/0 conn(0x7f520c159540 0x7f520c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f520a000640 1 -- 172.21.15.87:0/1024365722 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117446466 0 0) 0x7f520c159920 con 0x7f520c0aebd0 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f520a000640 1 -- 172.21.15.87:0/1024365722 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f0003530 con 0x7f520c0aebd0 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f520aa00640 1 -- 172.21.15.87:0/1024365722 >> v1:172.21.15.169:6789/0 conn(0x7f520c0721f0 legacy=0x7f520c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f520aa00640 1 -- 172.21.15.87:0/1024365722 >> v1:172.21.15.87:6789/0 conn(0x7f520c0aebd0 legacy=0x7f520c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.159+0000 7f520aa00640 1 -- 172.21.15.87:0/1024365722 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c15c8b0 con 0x7f520c159540 2024-04-25T10:15:35.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f520aa00640 1 --2- 172.21.15.87:0/1024365722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c159540 0x7f520c0ae690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009d90 tx=0x7f51fc01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765e16305c8efd2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f520a000640 1 -- 172.21.15.87:0/1024365722 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0b2070 con 0x7f520c159540 2024-04-25T10:15:35.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f520a000640 1 -- 172.21.15.87:0/1024365722 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc0a5940 con 0x7f520c159540 2024-04-25T10:15:35.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f520a000640 1 -- 172.21.15.87:0/1024365722 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0a5cf0 con 0x7f520c159540 2024-04-25T10:15:35.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f5211c00640 1 -- 172.21.15.87:0/1024365722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c159540 msgr2=0x7f520c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f5211c00640 1 --2- 172.21.15.87:0/1024365722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c159540 0x7f520c0ae690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009d90 tx=0x7f51fc01c970 comp rx=0 tx=0).stop 2024-04-25T10:15:35.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f5211c00640 1 -- 172.21.15.87:0/1024365722 shutdown_connections 2024-04-25T10:15:35.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f5211c00640 1 --2- 172.21.15.87:0/1024365722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c159540 0x7f520c0ae690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f5211c00640 1 -- 172.21.15.87:0/1024365722 >> 172.21.15.87:0/1024365722 conn(0x7f520c0669a0 msgr2=0x7f520c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f5211c00640 1 -- 172.21.15.87:0/1024365722 shutdown_connections 2024-04-25T10:15:35.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.163+0000 7f5211c00640 1 -- 172.21.15.87:0/1024365722 wait complete. 2024-04-25T10:15:35.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 Processor -- start 2024-04-25T10:15:35.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 -- start start 2024-04-25T10:15:35.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c0721f0 0x7f520c13e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f520c0aebd0 0x7f520c151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520c159540 0x7f520c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c169ac0 con 0x7f520c0aebd0 2024-04-25T10:15:35.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c169940 con 0x7f520c0721f0 2024-04-25T10:15:35.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c169c40 con 0x7f520c159540 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f520c0aebd0 0x7f520c151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f520aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c0721f0 0x7f520c13e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f520c0aebd0 0x7f520c151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59298/0 (socket says 172.21.15.87:59298) 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 -- 172.21.15.87:0/4045270577 learned_addr learned my addr 172.21.15.87:0/4045270577 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f520b400640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520c159540 0x7f520c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 -- 172.21.15.87:0/4045270577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520c159540 msgr2=0x7f520c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520c159540 0x7f520c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 -- 172.21.15.87:0/4045270577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c0721f0 msgr2=0x7f520c13e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c0721f0 0x7f520c13e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 -- 172.21.15.87:0/4045270577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c150770 con 0x7f520c0aebd0 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5202000640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f520c0aebd0 0x7f520c151810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c4d0 tx=0x7f51f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5203400640 1 -- 172.21.15.87:0/4045270577 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4017070 con 0x7f520c0aebd0 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5203400640 1 -- 172.21.15.87:0/4045270577 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f4005cf0 con 0x7f520c0aebd0 2024-04-25T10:15:35.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520c150a50 con 0x7f520c0aebd0 2024-04-25T10:15:35.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5203400640 1 -- 172.21.15.87:0/4045270577 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4007270 con 0x7f520c0aebd0 2024-04-25T10:15:35.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.167+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c151010 con 0x7f520c0aebd0 2024-04-25T10:15:35.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5203400640 1 -- 172.21.15.87:0/4045270577 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51f401a070 con 0x7f520c0aebd0 2024-04-25T10:15:35.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5203400640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51e8066f90 0x7f51e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5203400640 1 -- 172.21.15.87:0/4045270577 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51f4084380 con 0x7f520c0aebd0 2024-04-25T10:15:35.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f520aa00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51e8066f90 0x7f51e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 Processor -- start 2024-04-25T10:15:35.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 -- start start 2024-04-25T10:15:35.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f520aa00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51e8066f90 0x7f51e8069450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0af000 tx=0x7f51fc0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfaf1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfaa7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfaa80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edf8bd8a0 con 0x562edfaf1970 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edf8f5d10 con 0x562edfaf2290 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562edf87e2d0 con 0x562edfaa7cd0 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f520aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfaf1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5202000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfaa7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f520aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfaf1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59300/0 (socket says 172.21.15.87:59300) 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f520aa00640 1 -- 172.21.15.87:0/3758393099 learned_addr learned my addr 172.21.15.87:0/3758393099 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f520b400640 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfaa80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5202000640 1 -- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 msgr2=0x562edfaa80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5202000640 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfaa80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5202000640 1 -- 172.21.15.87:0/3758393099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 msgr2=0x562edfaf1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5202000640 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfaf1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f5202000640 1 -- 172.21.15.87:0/3758393099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562edf93c870 con 0x562edfaf2290 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.171+0000 7f520aa00640 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfaf1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f5202000640 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfaa7790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51f4082810 tx=0x7f51f4054de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aeab1e120df9603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f51fb400640 1 -- 172.21.15.87:0/3758393099 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4052fa0 con 0x562edfaf2290 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f51fb400640 1 -- 172.21.15.87:0/3758393099 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40840c0 con 0x562edfaf2290 2024-04-25T10:15:35.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f51fb400640 1 -- 172.21.15.87:0/3758393099 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f405ce70 con 0x562edfaf2290 2024-04-25T10:15:35.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 msgr2=0x562edfaa7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfaa7790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51f4082810 tx=0x7f51f4054de0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- 172.21.15.87:0/3758393099 shutdown_connections 2024-04-25T10:15:35.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfaa80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfaa7790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 --2- 172.21.15.87:0/3758393099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfaf1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- 172.21.15.87:0/3758393099 >> 172.21.15.87:0/3758393099 conn(0x562edfaae540 msgr2=0x562edfaae920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- 172.21.15.87:0/3758393099 shutdown_connections 2024-04-25T10:15:35.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- 172.21.15.87:0/3758393099 wait complete. 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- start start 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 Processor -- start 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfaf8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf1970 0x562edfaf9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf2290 0x562edfaab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edf8b0b50 con 0x562edfaa7cd0 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edf8f1db0 con 0x562edfaf1970 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562edf8fa340 con 0x562edfaf2290 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f52150b9000 10 client.0 _ll_register_callbacks cb 0x562edf9a9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfaf8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfaf8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59314/0 (socket says 172.21.15.87:59314) 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 -- 172.21.15.87:0/2518814289 learned_addr learned my addr 172.21.15.87:0/2518814289 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f5202000640 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf1970 0x562edfaf9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520b400640 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf2290 0x562edfaab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 -- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf2290 msgr2=0x562edfaab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf2290 0x562edfaab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 -- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf1970 msgr2=0x562edfaf9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf1970 0x562edfaf9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 -- 172.21.15.87:0/2518814289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562edf87e520 con 0x562edfaa7cd0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f520aa00640 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfaf8cd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f51fc01ce50 tx=0x7f51fc01b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a8d568f0062d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.175+0000 7f51efe00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 1 -- 172.21.15.87:0/2518814289 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0cc020 con 0x562edfaa7cd0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 1 -- 172.21.15.87:0/2518814289 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51fc0b9b30 con 0x562edfaa7cd0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51f40840c0 con 0x562edfaa7cd0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 1 -- 172.21.15.87:0/2518814289 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc0b2040 con 0x562edfaa7cd0 2024-04-25T10:15:35.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 10 client.6096 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562edf8f0d00 con 0x562edfaa7cd0 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 1 -- 172.21.15.87:0/2518814289 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40840c0 con 0x562edfaa7cd0 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 1 -- 172.21.15.87:0/2518814289 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f51fc0c8460 con 0x562edfaa7cd0 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 10 client.6096 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 10 client.6096 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 20 client.6096 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaabea0 0x562edfaac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 4 client.6096 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562edf891b10 con 0x562edfaabea0 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f5202000640 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaabea0 0x562edfaac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f5202000640 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaabea0 0x562edfaac2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59190e1f93799598 server_cookie=3aff40222b9af14a in_seq=0 out_seq=0 2024-04-25T10:15:35.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 10 client.6096 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 1 -- 172.21.15.87:0/2518814289 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562edf891b10 con 0x562edfaabea0 2024-04-25T10:15:35.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 10 client.6096 handle_command_reply: tid=0 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 client.6096 shutdown 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 msgr2=0x562edfaf8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfaf8cd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f51fc01ce50 tx=0x7f51fc01b1b0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 shutdown_connections 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaabea0 0x562edfaac2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf2290 0x562edfaab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf1970 0x562edfaf9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 --2- 172.21.15.87:0/2518814289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfaf8cd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f51efe00640 0 client.6096 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 >> 172.21.15.87:0/2518814289 conn(0x562edfaae540 msgr2=0x562edfaa4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 shutdown_connections 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 1 -- 172.21.15.87:0/2518814289 wait complete. 2024-04-25T10:15:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.179+0000 7f52150b9000 20 client.6096 trim_cache size 0 max 16384 2024-04-25T10:15:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 Processor -- start 2024-04-25T10:15:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 -- start start 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfb4dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfb4e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb4ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c202de0 con 0x562edfaf1970 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c15a490 con 0x562edfaf2290 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c14e5a0 con 0x562edfaa7cd0 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfb4e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfb4e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59328/0 (socket says 172.21.15.87:59328) 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 -- 172.21.15.87:0/3936864344 learned_addr learned my addr 172.21.15.87:0/3936864344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f520b400640 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb4ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f520aa00640 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfb4dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 -- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 msgr2=0x562edfb4dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfb4dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 -- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 msgr2=0x562edfb4ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb4ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 -- 172.21.15.87:0/3936864344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562edfaf20c0 con 0x562edfaf1970 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f520b400640 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb4ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f5202000640 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfb4e4f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f51f4082810 tx=0x7f51f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e6d1b3be0ad1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f51faa00640 1 -- 172.21.15.87:0/3936864344 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f407d040 con 0x562edfaf1970 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f51faa00640 1 -- 172.21.15.87:0/3936864344 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40840c0 con 0x562edfaf1970 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.195+0000 7f51faa00640 1 -- 172.21.15.87:0/3936864344 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4082340 con 0x562edfaf1970 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- 172.21.15.87:0/3936864344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 msgr2=0x562edfb4e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfb4e4f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f51f4082810 tx=0x7f51f4013000 comp rx=0 tx=0).stop 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- 172.21.15.87:0/3936864344 shutdown_connections 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb4ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaf1970 0x562edfb4e4f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 --2- 172.21.15.87:0/3936864344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaa7cd0 0x562edfb4dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- 172.21.15.87:0/3936864344 >> 172.21.15.87:0/3936864344 conn(0x562edfaae540 msgr2=0x562edfaac280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- 172.21.15.87:0/3936864344 shutdown_connections 2024-04-25T10:15:35.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- 172.21.15.87:0/3936864344 wait complete. 2024-04-25T10:15:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- start start 2024-04-25T10:15:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 Processor -- start 2024-04-25T10:15:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfb52500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb52a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf69a0 0x562edfaf6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edfab1ad0 con 0x562edfaa7cd0 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c159e90 con 0x562edfaf2290 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562edf8b7700 con 0x562edfaf69a0 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 10 client.0 _ll_register_callbacks cb 0x7f51fc0b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfb52500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfb52500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59334/0 (socket says 172.21.15.87:59334) 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 -- 172.21.15.87:0/88939227 learned_addr learned my addr 172.21.15.87:0/88939227 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f5202000640 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb52a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520b400640 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf69a0 0x562edfaf6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 -- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf69a0 msgr2=0x562edfaf6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf69a0 0x562edfaf6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 -- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 msgr2=0x562edfb52a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb52a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 -- 172.21.15.87:0/88939227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562edf979920 con 0x562edfaa7cd0 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f520aa00640 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfb52500 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f51fc01b1f0 tx=0x7f51fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b67494e465eb9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f51efe00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f51efe00640 1 -- 172.21.15.87:0/88939227 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0df060 con 0x562edfaa7cd0 2024-04-25T10:15:35.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f51efe00640 1 -- 172.21.15.87:0/88939227 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51fc0c8bb0 con 0x562edfaa7cd0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51f40840c0 con 0x562edfaa7cd0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f51efe00640 1 -- 172.21.15.87:0/88939227 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc0bd070 con 0x562edfaa7cd0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 10 client.6102 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562edf94fa30 con 0x562edfaa7cd0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.199+0000 7f51efe00640 1 -- 172.21.15.87:0/88939227 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40840c0 con 0x562edfaa7cd0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f51efe00640 1 -- 172.21.15.87:0/88939227 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f51fc0b9830 con 0x562edfaa7cd0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 10 client.6102 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 10 client.6102 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 20 client.6102 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaf7af0 0x562edfaf7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 4 client.6102 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x562edf903250 con 0x562edfaf7af0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5202000640 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaf7af0 0x562edfaf7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5202000640 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaf7af0 0x562edfaf7f00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e22f0ac97427993 server_cookie=80b1f6e144c8cf97 in_seq=0 out_seq=0 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f51efe00640 10 client.6102 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f51efe00640 1 -- 172.21.15.87:0/88939227 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x562edf903250 con 0x562edfaf7af0 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f51efe00640 10 client.6102 handle_command_reply: tid=0 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 client.6102 shutdown 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 msgr2=0x562edfb52500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfb52500 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f51fc01b1f0 tx=0x7f51fc009750 comp rx=0 tx=0).stop 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 shutdown_connections 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562edfaf7af0 0x562edfaf7f00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562edfaf69a0 0x562edfaf6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562edfaf2290 0x562edfb52a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f51efe00640 0 client.6102 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 --2- 172.21.15.87:0/88939227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562edfaa7cd0 0x562edfb52500 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 >> 172.21.15.87:0/88939227 conn(0x562edfaae540 msgr2=0x562edfaacb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 shutdown_connections 2024-04-25T10:15:35.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 1 -- 172.21.15.87:0/88939227 wait complete. 2024-04-25T10:15:35.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f52150b9000 20 client.6102 trim_cache size 0 max 16384 2024-04-25T10:15:35.214 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:35.214 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": -2 2024-04-25T10:15:35.214 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51e8066f90 msgr2=0x7f51e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51e8066f90 0x7f51e8069450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0af000 tx=0x7f51fc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f520c0aebd0 msgr2=0x7f520c151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f520c0aebd0 0x7f520c151810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c4d0 tx=0x7f51f400c990 comp rx=0 tx=0).stop 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 shutdown_connections 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51e8066f90 0x7f51e8069450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520c159540 0x7f520c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f520c0aebd0 0x7f520c151810 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 --2- 172.21.15.87:0/4045270577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f520c0721f0 0x7f520c13e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 >> 172.21.15.87:0/4045270577 conn(0x7f520c0669a0 msgr2=0x7f520c15b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 shutdown_connections 2024-04-25T10:15:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.203+0000 7f5211c00640 1 -- 172.21.15.87:0/4045270577 wait complete. 2024-04-25T10:15:35.228 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:15:35.228 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:15:35.228 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:35.229 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:15:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.359+0000 7ff20c600640 1 Processor -- start 2024-04-25T10:15:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.359+0000 7ff20c600640 1 -- start start 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff2040b38d0 0x7ff20415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20c600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20415b080 con 0x7ff2040aef80 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20c600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20415adb0 con 0x7ff204070350 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20415eb60 con 0x7ff2040b38d0 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff209c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff2040aef80 0x7ff20415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50716/0 (socket says 172.21.15.87:50716) 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff209c00640 1 -- 172.21.15.87:0/3830559347 learned_addr learned my addr 172.21.15.87:0/3830559347 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20a600640 1 --2- 172.21.15.87:0/3830559347 >> v2:172.21.15.169:3300/0 conn(0x7ff2040b38d0 0x7ff20415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff203e00640 1 -- 172.21.15.87:0/3830559347 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525634076 0 0) 0x7ff20415b080 con 0x7ff2040aef80 2024-04-25T10:15:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff203e00640 1 -- 172.21.15.87:0/3830559347 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1ec003530 con 0x7ff2040aef80 2024-04-25T10:15:35.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20a600640 1 -- 172.21.15.87:0/3830559347 >> v1:172.21.15.169:6789/0 conn(0x7ff204070350 legacy=0x7ff2040b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff203e00640 1 -- 172.21.15.87:0/3830559347 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766686116 0 0) 0x7ff20415adb0 con 0x7ff204070350 2024-04-25T10:15:35.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20a600640 1 -- 172.21.15.87:0/3830559347 >> v1:172.21.15.87:6789/0 conn(0x7ff2040aef80 legacy=0x7ff20415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.363+0000 7ff20a600640 1 -- 172.21.15.87:0/3830559347 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20415ee40 con 0x7ff2040b38d0 2024-04-25T10:15:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20a600640 1 --2- 172.21.15.87:0/3830559347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2040b38d0 0x7ff20415e480 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009d90 tx=0x7ff1fc01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bed1707a56ca94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff203e00640 1 -- 172.21.15.87:0/3830559347 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0b2070 con 0x7ff2040b38d0 2024-04-25T10:15:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff203e00640 1 -- 172.21.15.87:0/3830559347 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc0a5a10 con 0x7ff2040b38d0 2024-04-25T10:15:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff203e00640 1 -- 172.21.15.87:0/3830559347 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0a5dc0 con 0x7ff2040b38d0 2024-04-25T10:15:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 -- 172.21.15.87:0/3830559347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2040b38d0 msgr2=0x7ff20415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 --2- 172.21.15.87:0/3830559347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2040b38d0 0x7ff20415e480 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009d90 tx=0x7ff1fc01c970 comp rx=0 tx=0).stop 2024-04-25T10:15:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 -- 172.21.15.87:0/3830559347 shutdown_connections 2024-04-25T10:15:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 --2- 172.21.15.87:0/3830559347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2040b38d0 0x7ff20415e480 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 -- 172.21.15.87:0/3830559347 >> 172.21.15.87:0/3830559347 conn(0x7ff2040669a0 msgr2=0x7ff204066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 -- 172.21.15.87:0/3830559347 shutdown_connections 2024-04-25T10:15:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 -- 172.21.15.87:0/3830559347 wait complete. 2024-04-25T10:15:35.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 Processor -- start 2024-04-25T10:15:35.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.367+0000 7ff20c600640 1 -- start start 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff204070350 0x7ff204165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2040aef80 0x7ff2040af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2041f65f0 0x7ff204159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041689b0 con 0x7ff204070350 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204168830 con 0x7ff2041f65f0 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff204168b30 con 0x7ff2040aef80 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff204070350 0x7ff204165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2041f65f0 0x7ff204159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff204070350 0x7ff204165260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59346/0 (socket says 172.21.15.87:59346) 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 -- 172.21.15.87:0/2813880293 learned_addr learned my addr 172.21.15.87:0/2813880293 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209200640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2040aef80 0x7ff2040af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 -- 172.21.15.87:0/2813880293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2040aef80 msgr2=0x7ff2040af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2040aef80 0x7ff2040af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 -- 172.21.15.87:0/2813880293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2041f65f0 msgr2=0x7ff204159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2041f65f0 0x7ff204159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 -- 172.21.15.87:0/2813880293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204166420 con 0x7ff204070350 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff209c00640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff204070350 0x7ff204165260 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800c450 tx=0x7ff1f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff201600640 1 -- 172.21.15.87:0/2813880293 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8017070 con 0x7ff204070350 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff201600640 1 -- 172.21.15.87:0/2813880293 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f8005cf0 con 0x7ff204070350 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff201600640 1 -- 172.21.15.87:0/2813880293 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8007290 con 0x7ff204070350 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2041657a0 con 0x7ff204070350 2024-04-25T10:15:35.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2041f6d60 con 0x7ff204070350 2024-04-25T10:15:35.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.371+0000 7ff201600640 1 -- 172.21.15.87:0/2813880293 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1f801a070 con 0x7ff204070350 2024-04-25T10:15:35.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff201600640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1e4066f60 0x7ff1e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff201600640 1 -- 172.21.15.87:0/2813880293 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1f80843a0 con 0x7ff204070350 2024-04-25T10:15:35.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209200640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1e4066f60 0x7ff1e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 Processor -- start 2024-04-25T10:15:35.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- start start 2024-04-25T10:15:35.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209200640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1e4066f60 0x7ff1e4069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b0670 tx=0x7ff1f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc628e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc6244af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc6092db0 con 0x55ffc628dea0 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc601b230 con 0x55ffc6244710 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffc608ee50 con 0x55ffc628e7c0 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc628e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc628e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59356/0 (socket says 172.21.15.87:59356) 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 -- 172.21.15.87:0/1254776338 learned_addr learned my addr 172.21.15.87:0/1254776338 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 -- 172.21.15.87:0/1254776338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 msgr2=0x55ffc62441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 --2- 172.21.15.87:0/1254776338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 -- 172.21.15.87:0/1254776338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 msgr2=0x55ffc6244af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 --2- 172.21.15.87:0/1254776338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc6244af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 -- 172.21.15.87:0/1254776338 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc6054750 con 0x55ffc628dea0 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff209c00640 1 --2- 172.21.15.87:0/1254776338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc628e280 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff1f805ce80 tx=0x7ff1f8058df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d233ac0d3411483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff1f6000640 1 -- 172.21.15.87:0/1254776338 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8053050 con 0x55ffc628dea0 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff1f6000640 1 -- 172.21.15.87:0/1254776338 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f80840e0 con 0x55ffc628dea0 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff1f6000640 1 -- 172.21.15.87:0/1254776338 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f805cf20 con 0x55ffc628dea0 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1254776338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 msgr2=0x55ffc628e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1254776338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc628e280 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff1f805ce80 tx=0x7ff1f8058df0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1254776338 shutdown_connections 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1254776338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc6244af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1254776338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1254776338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc628e280 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1254776338 >> 172.21.15.87:0/1254776338 conn(0x55ffc62406c0 msgr2=0x55ffc6242ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1254776338 shutdown_connections 2024-04-25T10:15:35.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.375+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1254776338 wait complete. 2024-04-25T10:15:35.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 -- start start 2024-04-25T10:15:35.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 Processor -- start 2024-04-25T10:15:35.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc6272ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 0x55ffc6273410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc6273950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc604dbf0 con 0x55ffc6244710 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc6017990 con 0x55ffc628dea0 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffc602b8b0 con 0x55ffc628e7c0 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 10 client.0 _ll_register_callbacks cb 0x55ffc622f790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc6272ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc6272ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59360/0 (socket says 172.21.15.87:59360) 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 -- 172.21.15.87:0/766106236 learned_addr learned my addr 172.21.15.87:0/766106236 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 -- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 msgr2=0x55ffc6273950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc6273950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 -- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 msgr2=0x55ffc6273410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 0x55ffc6273410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 -- 172.21.15.87:0/766106236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffc6035640 con 0x55ffc6244710 2024-04-25T10:15:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209c00640 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc6272ed0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8055fd0 tx=0x7ff1f8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4905ccaf936c4abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff1eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff1eaa00640 1 -- 172.21.15.87:0/766106236 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f805d120 con 0x55ffc6244710 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff1eaa00640 1 -- 172.21.15.87:0/766106236 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1f80541a0 con 0x55ffc6244710 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1f80840e0 con 0x55ffc6244710 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff1eaa00640 1 -- 172.21.15.87:0/766106236 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f80543a0 con 0x55ffc6244710 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 10 client.6114 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ffc601b480 con 0x55ffc6244710 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff1eaa00640 1 -- 172.21.15.87:0/766106236 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f80840e0 con 0x55ffc6244710 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff1eaa00640 1 -- 172.21.15.87:0/766106236 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1f8007430 con 0x55ffc6244710 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 10 client.6114 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 10 client.6114 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 20 client.6114 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc629df40 0x55ffc62a0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 4 client.6114 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffc602ebb0 con 0x55ffc629df40 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209200640 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc629df40 0x55ffc62a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff1eaa00640 10 client.6114 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.379+0000 7ff209200640 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc629df40 0x55ffc62a0330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0bed3513d65a0f server_cookie=a29bf544cf9965b in_seq=0 out_seq=0 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff1eaa00640 1 -- 172.21.15.87:0/766106236 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffc602ebb0 con 0x55ffc629df40 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff1eaa00640 10 client.6114 handle_command_reply: tid=0 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 client.6114 shutdown 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 msgr2=0x55ffc6272ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc6272ed0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8055fd0 tx=0x7ff1f8007f90 comp rx=0 tx=0).stop 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 shutdown_connections 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc629df40 0x55ffc62a0330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc6273950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 0x55ffc6273410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff1eaa00640 0 client.6114 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/766106236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc6272ed0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 >> 172.21.15.87:0/766106236 conn(0x55ffc62406c0 msgr2=0x55ffc6243340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 shutdown_connections 2024-04-25T10:15:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 1 -- 172.21.15.87:0/766106236 wait complete. 2024-04-25T10:15:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.383+0000 7ff20fa7b000 20 client.6114 trim_cache size 0 max 16384 2024-04-25T10:15:35.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.395+0000 7ff20fa7b000 1 Processor -- start 2024-04-25T10:15:35.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.395+0000 7ff20fa7b000 1 -- start start 2024-04-25T10:15:35.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc62ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 0x55ffc62eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204202e60 con 0x55ffc6244710 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041f9c90 con 0x55ffc628dea0 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff204157720 con 0x55ffc628e7c0 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc62ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc62ea9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59372/0 (socket says 172.21.15.87:59372) 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 -- 172.21.15.87:0/46360309 learned_addr learned my addr 172.21.15.87:0/46360309 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20a600640 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209200640 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 0x55ffc62eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 -- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 msgr2=0x55ffc62eb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62eb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 -- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 msgr2=0x55ffc62eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 0x55ffc62eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 -- 172.21.15.87:0/46360309 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc6114ef0 con 0x55ffc6244710 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff209c00640 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc62ea9e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8058e30 tx=0x7ff1f806a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154d6537432b000c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff1f5600640 1 -- 172.21.15.87:0/46360309 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f80840e0 con 0x55ffc6244710 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff1f5600640 1 -- 172.21.15.87:0/46360309 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f806c030 con 0x55ffc6244710 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff1f5600640 1 -- 172.21.15.87:0/46360309 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f805cf20 con 0x55ffc6244710 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- 172.21.15.87:0/46360309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 msgr2=0x55ffc62ea9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc62ea9e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8058e30 tx=0x7ff1f806a070 comp rx=0 tx=0).stop 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- 172.21.15.87:0/46360309 shutdown_connections 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc628dea0 0x55ffc62eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/46360309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc6244710 0x55ffc62ea9e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- 172.21.15.87:0/46360309 >> 172.21.15.87:0/46360309 conn(0x55ffc62406c0 msgr2=0x55ffc629b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- 172.21.15.87:0/46360309 shutdown_connections 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- 172.21.15.87:0/46360309 wait complete. 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- start start 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 Processor -- start 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc62eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc62ef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc604dbf0 con 0x55ffc628dea0 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204164650 con 0x55ffc6244710 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffc60920c0 con 0x55ffc628e7c0 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.399+0000 7ff20fa7b000 10 client.0 _ll_register_callbacks cb 0x7ff1f0007be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc62ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc62eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc62eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:36946/0 (socket says 172.21.15.87:36946) 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209c00640 1 -- 172.21.15.87:0/1119028909 learned_addr learned my addr 172.21.15.87:0/1119028909 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209200640 1 -- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 msgr2=0x55ffc62a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209200640 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209200640 1 -- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 msgr2=0x55ffc62eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209200640 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc62eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209200640 1 -- 172.21.15.87:0/1119028909 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffc60529a0 con 0x55ffc628dea0 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209200640 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc62ef230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b25d0 tx=0x7ff1f0007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1157e6b3fe771f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 1 -- 172.21.15.87:0/1119028909 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0014660 con 0x55ffc628dea0 2024-04-25T10:15:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 1 -- 172.21.15.87:0/1119028909 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1f0010040 con 0x55ffc628dea0 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1f806c030 con 0x55ffc628dea0 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 1 -- 172.21.15.87:0/1119028909 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f0014800 con 0x55ffc628dea0 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 10 client.6120 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ffc5d4f5a0 con 0x55ffc628dea0 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 1 -- 172.21.15.87:0/1119028909 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f806c030 con 0x55ffc628dea0 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 1 -- 172.21.15.87:0/1119028909 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1f0032050 con 0x55ffc628dea0 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 10 client.6120 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 10 client.6120 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 20 client.6120 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc62a1440 0x55ffc62a1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 4 client.6120 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55ffc6042080 con 0x55ffc62a1440 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209c00640 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc62a1440 0x55ffc62a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff209c00640 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc62a1440 0x55ffc62a1850 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e783ac63506e2e server_cookie=27055987845ce2fe in_seq=0 out_seq=0 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 10 client.6120 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 1 -- 172.21.15.87:0/1119028909 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ffc6042080 con 0x55ffc62a1440 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 10 client.6120 handle_command_reply: tid=0 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 client.6120 shutdown 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 msgr2=0x55ffc62ef230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc62ef230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b25d0 tx=0x7ff1f0007d40 comp rx=0 tx=0).stop 2024-04-25T10:15:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 shutdown_connections 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55ffc62a1440 0x55ffc62a1850 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffc628e7c0 0x55ffc62a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffc628dea0 0x55ffc62ef230 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff1eaa00640 0 client.6120 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 --2- 172.21.15.87:0/1119028909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc6244710 0x55ffc62eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 >> 172.21.15.87:0/1119028909 conn(0x55ffc62406c0 msgr2=0x55ffc628c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 shutdown_connections 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 1 -- 172.21.15.87:0/1119028909 wait complete. 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20fa7b000 20 client.6120 trim_cache size 0 max 16384 2024-04-25T10:15:35.419 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "ac415e2d-e139-42e4-9246-88b05c6902df", 2024-04-25T10:15:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:35.420 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1e4066f60 msgr2=0x7ff1e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20c600640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1e4066f60 0x7ff1e4069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b0670 tx=0x7ff1f0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff204070350 msgr2=0x7ff204165260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.403+0000 7ff20c600640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff204070350 0x7ff204165260 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800c450 tx=0x7ff1f800c910 comp rx=0 tx=0).stop 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 shutdown_connections 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1e4066f60 0x7ff1e4069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2041f65f0 0x7ff204159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2040aef80 0x7ff2040af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 --2- 172.21.15.87:0/2813880293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff204070350 0x7ff204165260 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 >> 172.21.15.87:0/2813880293 conn(0x7ff2040669a0 msgr2=0x7ff2040b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 shutdown_connections 2024-04-25T10:15:35.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.407+0000 7ff20c600640 1 -- 172.21.15.87:0/2813880293 wait complete. 2024-04-25T10:15:35.430 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'ac415e2d-e139-42e4-9246-88b05c6902df', 'mode': 'asynchronous'}' 2024-04-25T10:15:35.430 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:15:35.430 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:35.430 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:15:35.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f992d600640 1 Processor -- start 2024-04-25T10:15:35.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f992d600640 1 -- start start 2024-04-25T10:15:35.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f992d600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9928070350 0x7f992815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f992d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280b31c0 0x7f99280b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f992d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992815b080 con 0x7f99280aef80 2024-04-25T10:15:35.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f992d600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992815adb0 con 0x7f9928070350 2024-04-25T10:15:35.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f992d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992815eb60 con 0x7f99280b31c0 2024-04-25T10:15:35.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f991e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9928070350 0x7f992815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f9926a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f99280aef80 0x7f992815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:52220/0 (socket says 172.21.15.87:52220) 2024-04-25T10:15:35.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f9926a00640 1 -- 172.21.15.87:0/1236954620 learned_addr learned my addr 172.21.15.87:0/1236954620 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:35.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.563+0000 7f9927400640 1 --2- 172.21.15.87:0/1236954620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280b31c0 0x7f99280b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f9926000640 1 -- 172.21.15.87:0/1236954620 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014358760 0 0) 0x7f992815b080 con 0x7f99280aef80 2024-04-25T10:15:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f991e000640 1 -- 172.21.15.87:0/1236954620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280b31c0 msgr2=0x7f99280b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f991e000640 1 --2- 172.21.15.87:0/1236954620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280b31c0 0x7f99280b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f991e000640 1 -- 172.21.15.87:0/1236954620 >> v1:172.21.15.169:6789/0 conn(0x7f99280aef80 legacy=0x7f992815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f991e000640 1 -- 172.21.15.87:0/1236954620 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992815ee40 con 0x7f9928070350 2024-04-25T10:15:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f991e000640 1 --2- 172.21.15.87:0/1236954620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9928070350 0x7f992815e480 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f99100097c0 tx=0x7f991001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f878018d4ae94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f9926000640 1 -- 172.21.15.87:0/1236954620 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100b1070 con 0x7f9928070350 2024-04-25T10:15:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f9926000640 1 -- 172.21.15.87:0/1236954620 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99100ac410 con 0x7f9928070350 2024-04-25T10:15:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f9926000640 1 -- 172.21.15.87:0/1236954620 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100ac7e0 con 0x7f9928070350 2024-04-25T10:15:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 -- 172.21.15.87:0/1236954620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9928070350 msgr2=0x7f992815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 --2- 172.21.15.87:0/1236954620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9928070350 0x7f992815e480 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f99100097c0 tx=0x7f991001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 -- 172.21.15.87:0/1236954620 shutdown_connections 2024-04-25T10:15:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 --2- 172.21.15.87:0/1236954620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280b31c0 0x7f99280b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 --2- 172.21.15.87:0/1236954620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9928070350 0x7f992815e480 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 -- 172.21.15.87:0/1236954620 >> 172.21.15.87:0/1236954620 conn(0x7f99280669a0 msgr2=0x7f9928066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 -- 172.21.15.87:0/1236954620 shutdown_connections 2024-04-25T10:15:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.567+0000 7f992d600640 1 -- 172.21.15.87:0/1236954620 wait complete. 2024-04-25T10:15:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 Processor -- start 2024-04-25T10:15:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 -- start start 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928070350 0x7f9928155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280aef80 0x7f9928151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280b31c0 0x7f99280ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281679a0 con 0x7f99280b31c0 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928167820 con 0x7f9928070350 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9928167b20 con 0x7f99280aef80 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280b31c0 0x7f99280ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9926a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928070350 0x7f9928155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280b31c0 0x7f99280ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59416/0 (socket says 172.21.15.87:59416) 2024-04-25T10:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 -- 172.21.15.87:0/3547864098 learned_addr learned my addr 172.21.15.87:0/3547864098 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f991e000640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280aef80 0x7f9928151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 -- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280aef80 msgr2=0x7f9928151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280aef80 0x7f9928151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 -- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928070350 msgr2=0x7f9928155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928070350 0x7f9928155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 -- 172.21.15.87:0/3547864098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928155ac0 con 0x7f99280b31c0 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9926a00640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928070350 0x7f9928155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f9927400640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280b31c0 0x7f99280ccb90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f992000e7f0 tx=0x7f992000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f991f400640 1 -- 172.21.15.87:0/3547864098 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920017070 con 0x7f99280b31c0 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f991f400640 1 -- 172.21.15.87:0/3547864098 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992000c940 con 0x7f99280b31c0 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9928155da0 con 0x7f99280b31c0 2024-04-25T10:15:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f991f400640 1 -- 172.21.15.87:0/3547864098 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000eea0 con 0x7f99280b31c0 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.571+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99280a9e50 con 0x7f99280b31c0 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.575+0000 7f991f400640 1 -- 172.21.15.87:0/3547864098 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f992001a070 con 0x7f99280b31c0 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.575+0000 7f991f400640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9904066f90 0x7f9904069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.575+0000 7f991f400640 1 -- 172.21.15.87:0/3547864098 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99200851d0 con 0x7f99280b31c0 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.575+0000 7f9926a00640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9904066f90 0x7f9904069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.575+0000 7f9930ae7000 1 Processor -- start 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.575+0000 7f9930ae7000 1 -- start start 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.575+0000 7f9926a00640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9904066f90 0x7f9904069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99180012f0 tx=0x7f9918009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e71675830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e7162aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e7162b400 0x564e7162b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7150e4a0 con 0x564e71675d70 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e71476210 con 0x564e7162b400 2024-04-25T10:15:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e713fe980 con 0x564e71675450 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f991e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e7162aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e71675830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e71675830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:44266/0 (socket says 172.21.15.87:44266) 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f991e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e7162aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59424/0 (socket says 172.21.15.87:59424) 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 -- 172.21.15.87:0/3351397743 learned_addr learned my addr 172.21.15.87:0/3351397743 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9927400640 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e7162b400 0x564e7162b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 -- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e7162b400 msgr2=0x564e7162b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e7162b400 0x564e7162b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 -- 172.21.15.87:0/3351397743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 msgr2=0x564e7162aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e7162aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 -- 172.21.15.87:0/3351397743 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e71471200 con 0x564e71675450 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f991e000640 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e7162aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9926a00640 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e71675830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99180147a0 tx=0x7f9918012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9703bf1a921476c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9917400640 1 -- 172.21.15.87:0/3351397743 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918025070 con 0x564e71675450 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9917400640 1 -- 172.21.15.87:0/3351397743 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991801b050 con 0x564e71675450 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9917400640 1 -- 172.21.15.87:0/3351397743 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800fbd0 con 0x564e71675450 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 msgr2=0x564e71675830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e71675830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99180147a0 tx=0x7f9918012000 comp rx=0 tx=0).stop 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- 172.21.15.87:0/3351397743 shutdown_connections 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e7162b400 0x564e7162b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e7162aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- 172.21.15.87:0/3351397743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e71675830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- 172.21.15.87:0/3351397743 >> 172.21.15.87:0/3351397743 conn(0x564e7161e4f0 msgr2=0x564e7162da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- 172.21.15.87:0/3351397743 shutdown_connections 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- 172.21.15.87:0/3351397743 wait complete. 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- start start 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 Processor -- start 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675450 0x564e7167d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e71663560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.579+0000 7f9930ae7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e71430d80 con 0x564e71675d70 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e714722b0 con 0x564e71675450 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e71437830 con 0x564e7162b400 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 10 client.0 _ll_register_callbacks cb 0x564e71529070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e71663560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9926a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f991e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675450 0x564e7167d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e71663560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59432/0 (socket says 172.21.15.87:59432) 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9926a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:44282/0 (socket says 172.21.15.87:44282) 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 -- 172.21.15.87:0/2344115036 learned_addr learned my addr 172.21.15.87:0/2344115036 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 -- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 msgr2=0x564e7167cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 -- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675450 msgr2=0x564e7167d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675450 0x564e7167d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 -- 172.21.15.87:0/2344115036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e71442130 con 0x564e71675d70 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9927400640 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e71663560 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9920055ac0 tx=0x7f9920055c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6047c40e90652a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 1 -- 172.21.15.87:0/2344115036 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992006d020 con 0x564e71675d70 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 1 -- 172.21.15.87:0/2344115036 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99200521f0 con 0x564e71675d70 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f991801b050 con 0x564e71675d70 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 1 -- 172.21.15.87:0/2344115036 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9920021070 con 0x564e71675d70 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 10 client.6132 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564e714f7490 con 0x564e71675d70 2024-04-25T10:15:35.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 1 -- 172.21.15.87:0/2344115036 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991801b050 con 0x564e71675d70 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 1 -- 172.21.15.87:0/2344115036 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99200831b0 con 0x564e71675d70 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 10 client.6132 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 10 client.6132 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 20 client.6132 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e71664300 0x564e716646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 4 client.6132 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e71412020 con 0x564e71664300 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9926a00640 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e71664300 0x564e716646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9926a00640 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e71664300 0x564e716646e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca1183d75fb48f4 server_cookie=79c01c2efff5f7b7 in_seq=0 out_seq=0 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 10 client.6132 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 1 -- 172.21.15.87:0/2344115036 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e71412020 con 0x564e71664300 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 10 client.6132 handle_command_reply: tid=0 2024-04-25T10:15:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 client.6132 shutdown 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 msgr2=0x564e71663560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e71663560 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9920055ac0 tx=0x7f9920055c40 comp rx=0 tx=0).stop 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 shutdown_connections 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675d70 0x564e71663560 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675450 0x564e7167d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e71664300 0x564e716646e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 --2- 172.21.15.87:0/2344115036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f990be00640 0 client.6132 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.583+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 >> 172.21.15.87:0/2344115036 conn(0x564e7161e4f0 msgr2=0x564e7162c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.587+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 shutdown_connections 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.587+0000 7f9930ae7000 1 -- 172.21.15.87:0/2344115036 wait complete. 2024-04-25T10:15:35.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.587+0000 7f9930ae7000 20 client.6132 trim_cache size 0 max 16384 2024-04-25T10:15:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 Processor -- start 2024-04-25T10:15:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 -- start start 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e7162b400 0x564e7167a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e7167bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e7167c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280b1110 con 0x564e7162b400 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280ab510 con 0x564e71675d70 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.599+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9928159b80 con 0x564e71675450 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e7162b400 0x564e7167a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e7162b400 0x564e7167a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59444/0 (socket says 172.21.15.87:59444) 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 -- 172.21.15.87:0/988975078 learned_addr learned my addr 172.21.15.87:0/988975078 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f991e000640 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e7167bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9927400640 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e7167c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 -- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 msgr2=0x564e7167bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e7167bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 -- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 msgr2=0x564e7167c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e7167c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 -- 172.21.15.87:0/988975078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e71435a30 con 0x564e7162b400 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f991e000640 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e7167bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9927400640 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e7167c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e7162b400 0x564e7167a5a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9918002240 tx=0x7f9918015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b159116b57abceae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9916a00640 1 -- 172.21.15.87:0/988975078 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918025040 con 0x564e7162b400 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9916a00640 1 -- 172.21.15.87:0/988975078 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991801b070 con 0x564e7162b400 2024-04-25T10:15:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9916a00640 1 -- 172.21.15.87:0/988975078 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918033b00 con 0x564e7162b400 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- 172.21.15.87:0/988975078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e7162b400 msgr2=0x564e7167a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e7162b400 0x564e7167a5a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9918002240 tx=0x7f9918015000 comp rx=0 tx=0).stop 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- 172.21.15.87:0/988975078 shutdown_connections 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e7167c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e71675450 0x564e7167bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 --2- 172.21.15.87:0/988975078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e7162b400 0x564e7167a5a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- 172.21.15.87:0/988975078 >> 172.21.15.87:0/988975078 conn(0x564e7161e4f0 msgr2=0x564e7167b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- 172.21.15.87:0/988975078 shutdown_connections 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- 172.21.15.87:0/988975078 wait complete. 2024-04-25T10:15:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- start start 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 Processor -- start 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675450 0x564e716922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e71692810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e71402190 con 0x564e71675450 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280aaf10 con 0x564e71675d70 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e714722b0 con 0x564e7162b400 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9930ae7000 10 client.0 _ll_register_callbacks cb 0x7f9918017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f991e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675450 0x564e716922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f991e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675450 0x564e716922d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59454/0 (socket says 172.21.15.87:59454) 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f991e000640 1 -- 172.21.15.87:0/4029911143 learned_addr learned my addr 172.21.15.87:0/4029911143 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9926a00640 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f9927400640 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e71692810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f991e000640 1 -- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 msgr2=0x564e7167a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.603+0000 7f991e000640 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f991e000640 1 -- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 msgr2=0x564e71692810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f991e000640 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e71692810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f991e000640 1 -- 172.21.15.87:0/4029911143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e7143e040 con 0x564e71675450 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f991e000640 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675450 0x564e716922d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f99100097c0 tx=0x7f991001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6768197d29240772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 1 -- 172.21.15.87:0/4029911143 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100b1040 con 0x564e71675450 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 1 -- 172.21.15.87:0/4029911143 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99100b88e0 con 0x564e71675450 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f991801b070 con 0x564e71675450 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 1 -- 172.21.15.87:0/4029911143 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99100cb050 con 0x564e71675450 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 10 client.6138 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564e71132900 con 0x564e71675450 2024-04-25T10:15:35.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 1 -- 172.21.15.87:0/4029911143 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991801b070 con 0x564e71675450 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 1 -- 172.21.15.87:0/4029911143 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99100ce020 con 0x564e71675450 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 10 client.6138 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 10 client.6138 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 20 client.6138 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e716d5910 0x564e716d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 4 client.6138 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x564e71483750 con 0x564e716d5910 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9926a00640 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e716d5910 0x564e716d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9926a00640 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e716d5910 0x564e716d5d20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e08d36ef2da9b2 server_cookie=9a2a722748b19f99 in_seq=0 out_seq=0 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 10 client.6138 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 1 -- 172.21.15.87:0/4029911143 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564e71483750 con 0x564e716d5910 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 10 client.6138 handle_command_reply: tid=0 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 client.6138 shutdown 2024-04-25T10:15:35.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675450 msgr2=0x564e716922d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675450 0x564e716922d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f99100097c0 tx=0x7f991001ace0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 shutdown_connections 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e71675d70 0x564e71692810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564e71675450 0x564e716922d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x564e716d5910 0x564e716d5d20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 --2- 172.21.15.87:0/4029911143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e7162b400 0x564e7167a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f990be00640 0 client.6138 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 >> 172.21.15.87:0/4029911143 conn(0x564e7161e4f0 msgr2=0x564e7167bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 shutdown_connections 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 1 -- 172.21.15.87:0/4029911143 wait complete. 2024-04-25T10:15:35.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f9930ae7000 20 client.6138 trim_cache size 0 max 16384 2024-04-25T10:15:35.623 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:35.623 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:35.623 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "cc352f19-db45-4065-81b0-5844426630f5", 2024-04-25T10:15:35.623 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:35.623 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9904066f90 msgr2=0x7f9904069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9904066f90 0x7f9904069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99180012f0 tx=0x7f9918009040 comp rx=0 tx=0).stop 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280b31c0 msgr2=0x7f99280ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280b31c0 0x7f99280ccb90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f992000e7f0 tx=0x7f992000ecb0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 shutdown_connections 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9904066f90 0x7f9904069450 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280b31c0 0x7f99280ccb90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280aef80 0x7f9928151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 --2- 172.21.15.87:0/3547864098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928070350 0x7f9928155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 >> 172.21.15.87:0/3547864098 conn(0x7f99280669a0 msgr2=0x7f99280b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 shutdown_connections 2024-04-25T10:15:35.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.607+0000 7f992d600640 1 -- 172.21.15.87:0/3547864098 wait complete. 2024-04-25T10:15:35.634 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'cc352f19-db45-4065-81b0-5844426630f5', 'mode': 'asynchronous'}' 2024-04-25T10:15:35.634 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:15:35.634 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:35.634 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:15:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1beb600640 1 Processor -- start 2024-04-25T10:15:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1beb600640 1 -- start start 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1beb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1beb600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1be40b1970 0x7f1be40ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1beb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be40b1e10 con 0x7f1be40aeb50 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1beb600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be40b20f0 con 0x7f1be40b1970 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1beb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be40b23d0 con 0x7f1be415df20 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1bdbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1be40b1970 0x7f1be40ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1bdbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1be40b1970 0x7f1be40ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59462/0 (socket says 172.21.15.87:59462) 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1bdbe00640 1 -- 172.21.15.87:0/502588342 learned_addr learned my addr 172.21.15.87:0/502588342 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1be8c00640 1 --2- 172.21.15.87:0/502588342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1bdbe00640 1 -- 172.21.15.87:0/502588342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 msgr2=0x7f1be40b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/502588342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1bdbe00640 1 -- 172.21.15.87:0/502588342 >> v1:172.21.15.169:6789/0 conn(0x7f1be40aeb50 legacy=0x7f1be40cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.767+0000 7f1bdbe00640 1 -- 172.21.15.87:0/502588342 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be40b26b0 con 0x7f1be40b1970 2024-04-25T10:15:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/502588342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40ae610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009870 tx=0x7f1bcc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd283f8846623c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1be3e00640 1 -- 172.21.15.87:0/502588342 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0b5070 con 0x7f1be40b1970 2024-04-25T10:15:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1be3e00640 1 -- 172.21.15.87:0/502588342 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0a8a50 con 0x7f1be40b1970 2024-04-25T10:15:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1be3e00640 1 -- 172.21.15.87:0/502588342 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0a8e20 con 0x7f1be40b1970 2024-04-25T10:15:35.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 -- 172.21.15.87:0/502588342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 msgr2=0x7f1be40ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 --2- 172.21.15.87:0/502588342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40ae610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009870 tx=0x7f1bcc01ed50 comp rx=0 tx=0).stop 2024-04-25T10:15:35.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 -- 172.21.15.87:0/502588342 shutdown_connections 2024-04-25T10:15:35.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 --2- 172.21.15.87:0/502588342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40ae610 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 --2- 172.21.15.87:0/502588342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 -- 172.21.15.87:0/502588342 >> 172.21.15.87:0/502588342 conn(0x7f1be40d1960 msgr2=0x7f1be40d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 -- 172.21.15.87:0/502588342 shutdown_connections 2024-04-25T10:15:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.771+0000 7f1beb600640 1 -- 172.21.15.87:0/502588342 wait complete. 2024-04-25T10:15:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 Processor -- start 2024-04-25T10:15:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 -- start start 2024-04-25T10:15:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be40aeb50 0x7f1be414dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be406beb0 con 0x7f1be40b1970 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be415e300 con 0x7f1be40aeb50 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be406c030 con 0x7f1be415df20 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be40aeb50 0x7f1be414dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59476/0 (socket says 172.21.15.87:59476) 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 -- 172.21.15.87:0/666473568 learned_addr learned my addr 172.21.15.87:0/666473568 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1be9600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 -- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 msgr2=0x7f1be40d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 -- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be40aeb50 msgr2=0x7f1be414dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be40aeb50 0x7f1be414dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 -- 172.21.15.87:0/666473568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be40d0a00 con 0x7f1be40b1970 2024-04-25T10:15:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1be8c00640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be40aeb50 0x7f1be414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40cfb60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0b3040 tx=0x7f1bcc01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1be1600640 1 -- 172.21.15.87:0/666473568 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0c0070 con 0x7f1be40b1970 2024-04-25T10:15:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1be1600640 1 -- 172.21.15.87:0/666473568 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0ae550 con 0x7f1be40b1970 2024-04-25T10:15:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1be1600640 1 -- 172.21.15.87:0/666473568 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0b5070 con 0x7f1be40b1970 2024-04-25T10:15:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be414e260 con 0x7f1be40b1970 2024-04-25T10:15:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.775+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be40b5820 con 0x7f1be40b1970 2024-04-25T10:15:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1be1600640 1 -- 172.21.15.87:0/666473568 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bcc0c3070 con 0x7f1be40b1970 2024-04-25T10:15:35.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1be1600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1bc0066f60 0x7f1bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1be1600640 1 -- 172.21.15.87:0/666473568 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bcc122850 con 0x7f1be40b1970 2024-04-25T10:15:35.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 Processor -- start 2024-04-25T10:15:35.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1be8c00640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1bc0066f60 0x7f1bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 -- start start 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1be8c00640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1bc0066f60 0x7f1bc0069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4006aa0 tx=0x7f1bd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c8596a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c8591fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8576df40 con 0x562c8596a000 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c856f5fc0 con 0x562c8596a920 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c85769fe0 con 0x562c85920410 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.779+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c8596a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c8596a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59486/0 (socket says 172.21.15.87:59486) 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 -- 172.21.15.87:0/586058159 learned_addr learned my addr 172.21.15.87:0/586058159 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be9600640 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c8591fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 -- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 msgr2=0x562c859207f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859207f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 -- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 msgr2=0x562c8591fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c8591fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 -- 172.21.15.87:0/586058159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c857b5a50 con 0x562c8596a000 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c8596a3e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4010f80 tx=0x7f1bd4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c70ff140b3e6ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1bd9600640 1 -- 172.21.15.87:0/586058159 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd401c050 con 0x562c8596a000 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1bd9600640 1 -- 172.21.15.87:0/586058159 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4026070 con 0x562c8596a000 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1bd9600640 1 -- 172.21.15.87:0/586058159 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4014840 con 0x562c8596a000 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- 172.21.15.87:0/586058159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 msgr2=0x562c8596a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c8596a3e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4010f80 tx=0x7f1bd4013000 comp rx=0 tx=0).stop 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- 172.21.15.87:0/586058159 shutdown_connections 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c8591fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 --2- 172.21.15.87:0/586058159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c8596a3e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- 172.21.15.87:0/586058159 >> 172.21.15.87:0/586058159 conn(0x562c8591c3c0 msgr2=0x562c8591e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- 172.21.15.87:0/586058159 shutdown_connections 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- 172.21.15.87:0/586058159 wait complete. 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- start start 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 Processor -- start 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c85920410 0x562c8592be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a000 0x562c8592c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a920 0x562c8592c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c85772570 con 0x562c8596a920 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c857285f0 con 0x562c85920410 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c85719310 con 0x562c8596a000 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1beea8a000 10 client.0 _ll_register_callbacks cb 0x562c858227a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a920 0x562c8592c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a920 0x562c8592c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59502/0 (socket says 172.21.15.87:59502) 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be9600640 1 -- 172.21.15.87:0/1074106455 learned_addr learned my addr 172.21.15.87:0/1074106455 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a000 0x562c8592c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.783+0000 7f1be8c00640 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c85920410 0x562c8592be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be9600640 1 -- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a000 msgr2=0x562c8592c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be9600640 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a000 0x562c8592c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be9600640 1 -- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c85920410 msgr2=0x562c8592be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be9600640 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c85920410 0x562c8592be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be9600640 1 -- 172.21.15.87:0/1074106455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c856f6210 con 0x562c8596a920 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be9600640 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a920 0x562c8592c880 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0125c0 tx=0x7f1bdc012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7864b12a56c060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 1 -- 172.21.15.87:0/1074106455 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc019020 con 0x562c8596a920 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 1 -- 172.21.15.87:0/1074106455 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bdc003b90 con 0x562c8596a920 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bd4026070 con 0x562c8596a920 2024-04-25T10:15:35.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 1 -- 172.21.15.87:0/1074106455 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc023050 con 0x562c8596a920 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 10 client.6150 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562c857f2400 con 0x562c8596a920 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 1 -- 172.21.15.87:0/1074106455 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4026070 con 0x562c8596a920 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 1 -- 172.21.15.87:0/1074106455 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bdc026020 con 0x562c8596a920 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 10 client.6150 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 10 client.6150 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 20 client.6150 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c8597a2b0 0x562c8597c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 4 client.6150 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c85709c90 con 0x562c8597a2b0 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be8c00640 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c8597a2b0 0x562c8597c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1be8c00640 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c8597a2b0 0x562c8597c6a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc8a9cf9eea5ea2 server_cookie=398824c0cc2a211a in_seq=0 out_seq=0 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 10 client.6150 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 1 -- 172.21.15.87:0/1074106455 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c85709c90 con 0x562c8597a2b0 2024-04-25T10:15:35.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 10 client.6150 handle_command_reply: tid=0 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 client.6150 shutdown 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a920 msgr2=0x562c8592c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a920 0x562c8592c880 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0125c0 tx=0x7f1bdc012a80 comp rx=0 tx=0).stop 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 shutdown_connections 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a920 0x562c8592c880 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c8597a2b0 0x562c8597c6a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a000 0x562c8592c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 --2- 172.21.15.87:0/1074106455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c85920410 0x562c8592be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1bd2000640 0 client.6150 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 >> 172.21.15.87:0/1074106455 conn(0x562c8591c3c0 msgr2=0x562c8591f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 shutdown_connections 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 1 -- 172.21.15.87:0/1074106455 wait complete. 2024-04-25T10:15:35.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.787+0000 7f1beea8a000 20 client.6150 trim_cache size 0 max 16384 2024-04-25T10:15:35.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 Processor -- start 2024-04-25T10:15:35.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 -- start start 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c85920410 0x562c859c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a000 0x562c859c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a920 0x562c859c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be41ffe20 con 0x562c85920410 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be41f4de0 con 0x562c8596a000 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be415efe0 con 0x562c8596a920 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c85920410 0x562c859c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c85920410 0x562c859c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59510/0 (socket says 172.21.15.87:59510) 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 -- 172.21.15.87:0/957749234 learned_addr learned my addr 172.21.15.87:0/957749234 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be9600640 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a920 0x562c859c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a000 0x562c859c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 -- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a920 msgr2=0x562c859c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a920 0x562c859c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 -- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a000 msgr2=0x562c859c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a000 0x562c859c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.803+0000 7f1be8c00640 1 -- 172.21.15.87:0/957749234 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c857c8c10 con 0x562c85920410 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1be8c00640 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c85920410 0x562c859c6b50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4010f20 tx=0x7f1bd4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5cbacb06663b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bd8c00640 1 -- 172.21.15.87:0/957749234 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd401c030 con 0x562c85920410 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bd8c00640 1 -- 172.21.15.87:0/957749234 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4026040 con 0x562c85920410 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bd8c00640 1 -- 172.21.15.87:0/957749234 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4005c00 con 0x562c85920410 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- 172.21.15.87:0/957749234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c85920410 msgr2=0x562c859c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c85920410 0x562c859c6b50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4010f20 tx=0x7f1bd4015000 comp rx=0 tx=0).stop 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- 172.21.15.87:0/957749234 shutdown_connections 2024-04-25T10:15:35.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c8596a920 0x562c859c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a000 0x562c859c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 --2- 172.21.15.87:0/957749234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c85920410 0x562c859c6b50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- 172.21.15.87:0/957749234 >> 172.21.15.87:0/957749234 conn(0x562c8591c3c0 msgr2=0x562c859775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- 172.21.15.87:0/957749234 shutdown_connections 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- 172.21.15.87:0/957749234 wait complete. 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- start start 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 Processor -- start 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c859cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c85986120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c85722d60 con 0x562c8596a000 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be41f47e0 con 0x562c8596a920 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c85772570 con 0x562c85920410 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1beea8a000 10 client.0 _ll_register_callbacks cb 0x562c857c02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c859cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c859cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59520/0 (socket says 172.21.15.87:59520) 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 -- 172.21.15.87:0/487388019 learned_addr learned my addr 172.21.15.87:0/487388019 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1be8c00640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1be9600640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c85986120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 -- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 msgr2=0x562c859cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 -- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 msgr2=0x562c85986120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c85986120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 -- 172.21.15.87:0/487388019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c85702bb0 con 0x562c8596a000 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1be9600640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c85986120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bdbe00640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c859cb1f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc01e7c0 tx=0x7f1bcc0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45491ed5f1ebb987 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bd2000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bd2000640 1 -- 172.21.15.87:0/487388019 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0fb870 con 0x562c8596a000 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.807+0000 7f1bd2000640 1 -- 172.21.15.87:0/487388019 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bcc0fb3e0 con 0x562c8596a000 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bd4026040 con 0x562c8596a000 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1bd2000640 1 -- 172.21.15.87:0/487388019 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bcc117050 con 0x562c8596a000 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 10 client.6156 fetch_fsmap learned FSMap version 25 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562c85715030 con 0x562c8596a000 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1bd2000640 1 -- 172.21.15.87:0/487388019 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4026040 con 0x562c8596a000 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1bd2000640 1 -- 172.21.15.87:0/487388019 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bcc11a020 con 0x562c8596a000 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 10 client.6156 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 10 client.6156 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 20 client.6156 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c85986a50 0x562c85986e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 4 client.6156 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x562c8577b480 con 0x562c85986a50 2024-04-25T10:15:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1be8c00640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c85986a50 0x562c85986e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1be8c00640 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c85986a50 0x562c85986e60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9491bca78846de server_cookie=af402a537e37b797 in_seq=0 out_seq=0 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1bd2000640 10 client.6156 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1bd2000640 1 -- 172.21.15.87:0/487388019 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562c8577b480 con 0x562c85986a50 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1bd2000640 10 client.6156 handle_command_reply: tid=0 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 client.6156 shutdown 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 msgr2=0x562c859cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c859cb1f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc01e7c0 tx=0x7f1bcc0094d0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 shutdown_connections 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c8596a920 0x562c85986120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c8596a000 0x562c859cb1f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x562c85986a50 0x562c85986e60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 --2- 172.21.15.87:0/487388019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562c85920410 0x562c859cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 >> 172.21.15.87:0/487388019 conn(0x562c8591c3c0 msgr2=0x562c8592b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1bd2000640 0 client.6156 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 shutdown_connections 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 1 -- 172.21.15.87:0/487388019 wait complete. 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beea8a000 20 client.6156 trim_cache size 0 max 16384 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "9418855f-d508-4832-a35e-345c69ce5326", 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:35.827 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1bc0066f60 msgr2=0x7f1bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1bc0066f60 0x7f1bc0069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4006aa0 tx=0x7f1bd4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 msgr2=0x7f1be40cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40cfb60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0b3040 tx=0x7f1bcc01baf0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 shutdown_connections 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1bc0066f60 0x7f1bc0069420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1be415df20 0x7f1be40d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be40b1970 0x7f1be40cfb60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 --2- 172.21.15.87:0/666473568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be40aeb50 0x7f1be414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 >> 172.21.15.87:0/666473568 conn(0x7f1be40d1960 msgr2=0x7f1be406efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 shutdown_connections 2024-04-25T10:15:35.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:35.811+0000 7f1beb600640 1 -- 172.21.15.87:0/666473568 wait complete. 2024-04-25T10:15:35.837 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '9418855f-d508-4832-a35e-345c69ce5326', 'mode': 'asynchronous'}' 2024-04-25T10:15:35.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:35.837 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T10:15:35.848 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T10:15:35.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:36.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe100e00640 1 Processor -- start 2024-04-25T10:15:36.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe100e00640 1 -- start start 2024-04-25T10:15:36.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe100e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe0fc1550b0 0x7fe0fc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe100e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc15b910 0x7fe0fc0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe100e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc071050 con 0x7fe0fc0cfe70 2024-04-25T10:15:36.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe100e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc15bda0 con 0x7fe0fc1550b0 2024-04-25T10:15:36.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe100e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc15d040 con 0x7fe0fc15b910 2024-04-25T10:15:36.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe0f9400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe0fc0cfe70 0x7fe0fc15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50720/0 (socket says 172.21.15.87:50720) 2024-04-25T10:15:36.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe0f9400640 1 -- 172.21.15.87:0/4118923996 learned_addr learned my addr 172.21.15.87:0/4118923996 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:36.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe0f9e00640 1 --2- 172.21.15.87:0/4118923996 >> v2:172.21.15.169:3300/0 conn(0x7fe0fc1550b0 0x7fe0fc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe0fa800640 1 --2- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc15b910 0x7fe0fc0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe0f8a00640 1 -- 172.21.15.87:0/4118923996 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157897046 0 0) 0x7fe0fc071050 con 0x7fe0fc0cfe70 2024-04-25T10:15:36.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.019+0000 7fe0f8a00640 1 -- 172.21.15.87:0/4118923996 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e0003530 con 0x7fe0fc0cfe70 2024-04-25T10:15:36.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f9e00640 1 -- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc15b910 msgr2=0x7fe0fc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:36.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f9e00640 1 --2- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc15b910 0x7fe0fc0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f9e00640 1 -- 172.21.15.87:0/4118923996 >> v1:172.21.15.87:6789/0 conn(0x7fe0fc0cfe70 legacy=0x7fe0fc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f9e00640 1 -- 172.21.15.87:0/4118923996 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc0712f0 con 0x7fe0fc1550b0 2024-04-25T10:15:36.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f9e00640 1 --2- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc0cf930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009dc0 tx=0x7fe0ec01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1644daa2abc8f49d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f8a00640 1 -- 172.21.15.87:0/4118923996 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0b2070 con 0x7fe0fc1550b0 2024-04-25T10:15:36.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f8a00640 1 -- 172.21.15.87:0/4118923996 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec0a5900 con 0x7fe0fc1550b0 2024-04-25T10:15:36.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe0f8a00640 1 -- 172.21.15.87:0/4118923996 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0a5cb0 con 0x7fe0fc1550b0 2024-04-25T10:15:36.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe100e00640 1 -- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 msgr2=0x7fe0fc0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe100e00640 1 --2- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc0cf930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009dc0 tx=0x7fe0ec01c950 comp rx=0 tx=0).stop 2024-04-25T10:15:36.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe100e00640 1 -- 172.21.15.87:0/4118923996 shutdown_connections 2024-04-25T10:15:36.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe100e00640 1 --2- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc15b910 0x7fe0fc0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe100e00640 1 --2- 172.21.15.87:0/4118923996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc0cf930 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.023+0000 7fe100e00640 1 -- 172.21.15.87:0/4118923996 >> 172.21.15.87:0/4118923996 conn(0x7fe0fc066b70 msgr2=0x7fe0fc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 -- 172.21.15.87:0/4118923996 shutdown_connections 2024-04-25T10:15:36.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 -- 172.21.15.87:0/4118923996 wait complete. 2024-04-25T10:15:36.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 Processor -- start 2024-04-25T10:15:36.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 -- start start 2024-04-25T10:15:36.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc0cfe70 0x7fe0fc155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0fc15b910 0x7fe0fc0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc15c030 con 0x7fe0fc15b910 2024-04-25T10:15:36.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0720b0 con 0x7fe0fc1550b0 2024-04-25T10:15:36.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe100e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc15c1b0 con 0x7fe0fc0cfe70 2024-04-25T10:15:36.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0fc15b910 0x7fe0fc0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0fc15b910 0x7fe0fc0cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59534/0 (socket says 172.21.15.87:59534) 2024-04-25T10:15:36.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 -- 172.21.15.87:0/10197543 learned_addr learned my addr 172.21.15.87:0/10197543 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:36.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0f9400640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0f9e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc0cfe70 0x7fe0fc155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 -- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc0cfe70 msgr2=0x7fe0fc155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc0cfe70 0x7fe0fc155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 -- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 msgr2=0x7fe0fc159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 -- 172.21.15.87:0/10197543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc1565a0 con 0x7fe0fc15b910 2024-04-25T10:15:36.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0f9400640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:36.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0fa800640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0fc15b910 0x7fe0fc0cafb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400e7f0 tx=0x7fe0f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.027+0000 7fe0f2000640 1 -- 172.21.15.87:0/10197543 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4017070 con 0x7fe0fc15b910 2024-04-25T10:15:36.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc156880 con 0x7fe0fc15b910 2024-04-25T10:15:36.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe0f2000640 1 -- 172.21.15.87:0/10197543 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f400c940 con 0x7fe0fc15b910 2024-04-25T10:15:36.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe0f2000640 1 -- 172.21.15.87:0/10197543 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400eea0 con 0x7fe0fc15b910 2024-04-25T10:15:36.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc156e40 con 0x7fe0fc15b910 2024-04-25T10:15:36.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc04ef30 con 0x7fe0fc15b910 2024-04-25T10:15:36.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe0f2000640 1 -- 172.21.15.87:0/10197543 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0f401a070 con 0x7fe0fc15b910 2024-04-25T10:15:36.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe0f2000640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0dc066f60 0x7fe0dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe0f2000640 1 -- 172.21.15.87:0/10197543 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0f4085280 con 0x7fe0fc15b910 2024-04-25T10:15:36.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe0f9e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0dc066f60 0x7fe0dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.031+0000 7fe0f9e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0dc066f60 0x7fe0dc069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0af000 tx=0x7fe0ec0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.035+0000 7fe0f2000640 1 -- 172.21.15.87:0/10197543 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0f4055bb0 con 0x7fe0fc15b910 2024-04-25T10:15:36.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.223+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0fc0b2a10 con 0x7fe0fc15b910 2024-04-25T10:15:36.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.223+0000 7fe0f2000640 1 -- 172.21.15.87:0/10197543 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fe0f4059c50 con 0x7fe0fc15b910 2024-04-25T10:15:36.231 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:36.231 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:09.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:36.232 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-04-25T10:15:36.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0dc066f60 msgr2=0x7fe0dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0dc066f60 0x7fe0dc069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0af000 tx=0x7fe0ec0a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:36.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0fc15b910 msgr2=0x7fe0fc0cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0fc15b910 0x7fe0fc0cafb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400e7f0 tx=0x7fe0f400ecb0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 shutdown_connections 2024-04-25T10:15:36.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0dc066f60 0x7fe0dc069420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0fc15b910 0x7fe0fc0cafb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0fc1550b0 0x7fe0fc159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 --2- 172.21.15.87:0/10197543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0fc0cfe70 0x7fe0fc155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 >> 172.21.15.87:0/10197543 conn(0x7fe0fc066b70 msgr2=0x7fe0fc15d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 shutdown_connections 2024-04-25T10:15:36.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.227+0000 7fe100e00640 1 -- 172.21.15.87:0/10197543 wait complete. 2024-04-25T10:15:36.235 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:15:36.227+0000 7ff597400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:15:36.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T10:15:36.401 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:36.401 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:36.401 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:36.417 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:36.417 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:36.417 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:36.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:36.427 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:15:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:15:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:15:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213093 s, 49.2 MB/s 2024-04-25T10:15:36.502 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:15:36.502 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:36.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.671+0000 7f7a1d200640 1 Processor -- start 2024-04-25T10:15:36.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.671+0000 7f7a1d200640 1 -- start start 2024-04-25T10:15:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.671+0000 7f7a1d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cfd90 0x7f7a180ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.671+0000 7f7a1d200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1815f3b0 con 0x7f7a180cd110 2024-04-25T10:15:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.671+0000 7f7a1d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a180ca810 con 0x7f7a180b0f70 2024-04-25T10:15:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.671+0000 7f7a1d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1815ef20 con 0x7f7a180cfd90 2024-04-25T10:15:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a16800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7a180cd110 0x7f7a180cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50722/0 (socket says 172.21.15.87:50722) 2024-04-25T10:15:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a16800640 1 -- 172.21.15.87:0/2897406577 learned_addr learned my addr 172.21.15.87:0/2897406577 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a0d400640 1 --2- 172.21.15.87:0/2897406577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cfd90 0x7f7a180ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693232066 0 0) 0x7f7a1815f3b0 con 0x7f7a180cd110 2024-04-25T10:15:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79fc003530 con 0x7f7a180cd110 2024-04-25T10:15:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457939090 0 0) 0x7f7a180ca810 con 0x7f7a180b0f70 2024-04-25T10:15:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a1815f3b0 con 0x7f7a180b0f70 2024-04-25T10:15:36.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885361176 0 0) 0x7f79fc003530 con 0x7f7a180cd110 2024-04-25T10:15:36.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a0d400640 1 -- 172.21.15.87:0/2897406577 >> v1:172.21.15.169:6789/0 conn(0x7f7a180b0f70 legacy=0x7f7a180b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a0d400640 1 -- 172.21.15.87:0/2897406577 >> v1:172.21.15.87:6789/0 conn(0x7f7a180cd110 legacy=0x7f7a180cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a0d400640 1 -- 172.21.15.87:0/2897406577 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1815e5f0 con 0x7f7a180cfd90 2024-04-25T10:15:36.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.675+0000 7f7a0d400640 1 --2- 172.21.15.87:0/2897406577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cfd90 0x7f7a180ccbd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a1815f0a0 tx=0x7f7a0001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2fc7258cba1ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10001680 con 0x7f7a180cfd90 2024-04-25T10:15:36.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a000a6500 con 0x7f7a180cfd90 2024-04-25T10:15:36.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a15400640 1 -- 172.21.15.87:0/2897406577 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a000a68d0 con 0x7f7a180cfd90 2024-04-25T10:15:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- 172.21.15.87:0/2897406577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cfd90 msgr2=0x7f7a180ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 --2- 172.21.15.87:0/2897406577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cfd90 0x7f7a180ccbd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a1815f0a0 tx=0x7f7a0001c8b0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- 172.21.15.87:0/2897406577 shutdown_connections 2024-04-25T10:15:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 --2- 172.21.15.87:0/2897406577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cfd90 0x7f7a180ccbd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- 172.21.15.87:0/2897406577 >> 172.21.15.87:0/2897406577 conn(0x7f7a180b1d50 msgr2=0x7f7a180b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- 172.21.15.87:0/2897406577 shutdown_connections 2024-04-25T10:15:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- 172.21.15.87:0/2897406577 wait complete. 2024-04-25T10:15:36.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 Processor -- start 2024-04-25T10:15:36.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- start start 2024-04-25T10:15:36.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a180b0f70 0x7f7a180b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cd110 0x7f7a180c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a180cfd90 0x7f7a180c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1816bf00 con 0x7f7a180cfd90 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1816bd80 con 0x7f7a180b0f70 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a1d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1816c080 con 0x7f7a180cd110 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a16800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a180cfd90 0x7f7a180c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a16800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a180cfd90 0x7f7a180c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59548/0 (socket says 172.21.15.87:59548) 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a16800640 1 -- 172.21.15.87:0/1787640145 learned_addr learned my addr 172.21.15.87:0/1787640145 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a15e00640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a180b0f70 0x7f7a180b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.679+0000 7f7a0d400640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cd110 0x7f7a180c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a16800640 1 -- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cd110 msgr2=0x7f7a180c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a16800640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cd110 0x7f7a180c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a16800640 1 -- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a180b0f70 msgr2=0x7f7a180b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a16800640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a180b0f70 0x7f7a180b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a16800640 1 -- 172.21.15.87:0/1787640145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180b4860 con 0x7f7a180cfd90 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a16800640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a180cfd90 0x7f7a180c6330 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7a100011f0 tx=0x7f7a1000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a0ea00640 1 -- 172.21.15.87:0/1787640145 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10005ae0 con 0x7f7a180cfd90 2024-04-25T10:15:36.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a0ea00640 1 -- 172.21.15.87:0/1787640145 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a10005ca0 con 0x7f7a180cfd90 2024-04-25T10:15:36.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a0ea00640 1 -- 172.21.15.87:0/1787640145 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100072b0 con 0x7f7a180cfd90 2024-04-25T10:15:36.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a180b4af0 con 0x7f7a180cfd90 2024-04-25T10:15:36.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a180b50b0 con 0x7f7a180cfd90 2024-04-25T10:15:36.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1804ef30 con 0x7f7a180cfd90 2024-04-25T10:15:36.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a0ea00640 1 -- 172.21.15.87:0/1787640145 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a10007450 con 0x7f7a180cfd90 2024-04-25T10:15:36.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a0ea00640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79f4066f90 0x7f79f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a0ea00640 1 -- 172.21.15.87:0/1787640145 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a10084d30 con 0x7f7a180cfd90 2024-04-25T10:15:36.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.683+0000 7f7a15e00640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79f4066f90 0x7f79f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.687+0000 7f7a15e00640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79f4066f90 0x7f79f4069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a080011f0 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.687+0000 7f7a0ea00640 1 -- 172.21.15.87:0/1787640145 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a10055650 con 0x7f7a180cfd90 2024-04-25T10:15:36.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a180cda20 con 0x7f7a180cfd90 2024-04-25T10:15:36.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a0ea00640 1 -- 172.21.15.87:0/1787640145 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f7a100596f0 con 0x7f7a180cfd90 2024-04-25T10:15:36.881 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:36.882 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:35.360341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:36.882 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-04-25T10:15:36.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79f4066f90 msgr2=0x7f79f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79f4066f90 0x7f79f4069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a080011f0 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2024-04-25T10:15:36.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a180cfd90 msgr2=0x7f7a180c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a180cfd90 0x7f7a180c6330 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7a100011f0 tx=0x7f7a1000cbe0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 shutdown_connections 2024-04-25T10:15:36.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79f4066f90 0x7f79f4069450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a180cfd90 0x7f7a180c6330 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a180cd110 0x7f7a180c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 --2- 172.21.15.87:0/1787640145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a180b0f70 0x7f7a180b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.875+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 >> 172.21.15.87:0/1787640145 conn(0x7f7a180b1d50 msgr2=0x7f7a18072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.879+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 shutdown_connections 2024-04-25T10:15:36.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:36.879+0000 7f7a1d200640 1 -- 172.21.15.87:0/1787640145 wait complete. 2024-04-25T10:15:36.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:15:37.046 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:37.046 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:37.046 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:37.061 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:37.061 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:15:37.062 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:15:37.062 DEBUG:teuthology.orchestra.run.smithi087:> import stat 2024-04-25T10:15:37.062 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:15:37.062 DEBUG:teuthology.orchestra.run.smithi087:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T10:15:37.062 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:15:37.126 INFO:teuthology.orchestra.run.smithi087.stdout:1099511630189 2024-04-25T10:15:37.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:15:37.135 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:37.135 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:15:37.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b53200640 1 Processor -- start 2024-04-25T10:15:37.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b53200640 1 -- start start 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b53200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b53200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b4c0cfb30 0x7f5b4c0d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b53200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c06b1d0 con 0x7f5b4c0ce220 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b53200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c15e110 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b53200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0b2940 con 0x7f5b4c15dd30 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b4c0cfb30 0x7f5b4c0d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b4c0cfb30 0x7f5b4c0d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59562/0 (socket says 172.21.15.87:59562) 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4b400640 1 -- 172.21.15.87:0/2974600357 learned_addr learned my addr 172.21.15.87:0/2974600357 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4be00640 1 --2- 172.21.15.87:0/2974600357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4b400640 1 -- 172.21.15.87:0/2974600357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 msgr2=0x7f5b4c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4b400640 1 --2- 172.21.15.87:0/2974600357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4b400640 1 -- 172.21.15.87:0/2974600357 >> v1:172.21.15.169:6789/0 conn(0x7f5b4c0ce220 legacy=0x7f5b4c1617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.311+0000 7f5b4b400640 1 -- 172.21.15.87:0/2974600357 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c06b470 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b4b400640 1 --2- 172.21.15.87:0/2974600357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0d18a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009870 tx=0x7f5b3801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6904ed3360cf9f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b4aa00640 1 -- 172.21.15.87:0/2974600357 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b380b2070 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b4aa00640 1 -- 172.21.15.87:0/2974600357 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b380a9070 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b4aa00640 1 -- 172.21.15.87:0/2974600357 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b380ad880 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 -- 172.21.15.87:0/2974600357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0cfb30 msgr2=0x7f5b4c0d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 --2- 172.21.15.87:0/2974600357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0d18a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009870 tx=0x7f5b3801c560 comp rx=0 tx=0).stop 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 -- 172.21.15.87:0/2974600357 shutdown_connections 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 --2- 172.21.15.87:0/2974600357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0d18a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 --2- 172.21.15.87:0/2974600357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 -- 172.21.15.87:0/2974600357 >> 172.21.15.87:0/2974600357 conn(0x7f5b4c09a9d0 msgr2=0x7f5b4c09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 -- 172.21.15.87:0/2974600357 shutdown_connections 2024-04-25T10:15:37.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.315+0000 7f5b53200640 1 -- 172.21.15.87:0/2974600357 wait complete. 2024-04-25T10:15:37.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 Processor -- start 2024-04-25T10:15:37.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 -- start start 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0ce220 0x7f5b4c1f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c16ace0 con 0x7f5b4c0ce220 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c16ab60 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c16ae60 con 0x7f5b4c15dd30 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0ce220 0x7f5b4c1f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:37084/0 (socket says 172.21.15.87:37084) 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0ce220 0x7f5b4c1f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59576/0 (socket says 172.21.15.87:59576) 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 -- 172.21.15.87:0/279225656 learned_addr learned my addr 172.21.15.87:0/279225656 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b51200640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 -- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 msgr2=0x7f5b4c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 -- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0ce220 msgr2=0x7f5b4c1f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0ce220 0x7f5b4c1f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 -- 172.21.15.87:0/279225656 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c1fa580 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4be00640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0ce220 0x7f5b4c1f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b4b400640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0c7230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b380b0040 tx=0x7f5b38019300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b43e00640 1 -- 172.21.15.87:0/279225656 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b380b2040 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b43e00640 1 -- 172.21.15.87:0/279225656 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b380bb070 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b43e00640 1 -- 172.21.15.87:0/279225656 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b380b6440 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c1fa860 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.319+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c1fae20 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.323+0000 7f5b43e00640 1 -- 172.21.15.87:0/279225656 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b380ab070 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.323+0000 7f5b43e00640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b2c066f60 0x7f5b2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.323+0000 7f5b43e00640 1 -- 172.21.15.87:0/279225656 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b3811e8b0 con 0x7f5b4c0cfb30 2024-04-25T10:15:37.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.323+0000 7f5b566f6000 1 Processor -- start 2024-04-25T10:15:37.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.323+0000 7f5b4be00640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b2c066f60 0x7f5b2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.323+0000 7f5b566f6000 1 -- start start 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b2c066f60 0x7f5b2c069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c15e3c0 tx=0x7f5b3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef84ad180 0x55cef84ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef84630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef8463600 0x55cef84639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef82b0f20 con 0x55cef84ad180 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8239000 con 0x55cef84adaa0 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cef82acfc0 con 0x55cef8463600 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef84ad180 0x55cef84ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef84ad180 0x55cef84ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59584/0 (socket says 172.21.15.87:59584) 2024-04-25T10:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 -- 172.21.15.87:0/707307894 learned_addr learned my addr 172.21.15.87:0/707307894 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b51200640 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef8463600 0x55cef84639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4b400640 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef84630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 -- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef8463600 msgr2=0x55cef84639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef8463600 0x55cef84639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 -- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 msgr2=0x55cef84630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef84630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 -- 172.21.15.87:0/707307894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cef828cba0 con 0x55cef84ad180 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b4be00640 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef84ad180 0x55cef84ad560 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c010ef0 tx=0x7f5b3c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dd8e0689b9bbf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b40c00640 1 -- 172.21.15.87:0/707307894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c01c050 con 0x55cef84ad180 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b40c00640 1 -- 172.21.15.87:0/707307894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c026070 con 0x55cef84ad180 2024-04-25T10:15:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b40c00640 1 -- 172.21.15.87:0/707307894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c007a20 con 0x55cef84ad180 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- 172.21.15.87:0/707307894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef84ad180 msgr2=0x55cef84ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef84ad180 0x55cef84ad560 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c010ef0 tx=0x7f5b3c013000 comp rx=0 tx=0).stop 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- 172.21.15.87:0/707307894 shutdown_connections 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef8463600 0x55cef84639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef84630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 --2- 172.21.15.87:0/707307894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef84ad180 0x55cef84ad560 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- 172.21.15.87:0/707307894 >> 172.21.15.87:0/707307894 conn(0x55cef8469760 msgr2=0x55cef84619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- 172.21.15.87:0/707307894 shutdown_connections 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- 172.21.15.87:0/707307894 wait complete. 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 -- start start 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.327+0000 7f5b566f6000 1 Processor -- start 2024-04-25T10:15:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef84686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8468be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef849b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef836e9c0 con 0x55cef8463600 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef826b5d0 con 0x55cef84adaa0 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cef8257c90 con 0x55cef84ad180 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 10 client.0 _ll_register_callbacks cb 0x55cef844ea60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef84686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef84686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59596/0 (socket says 172.21.15.87:59596) 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 -- 172.21.15.87:0/674505770 learned_addr learned my addr 172.21.15.87:0/674505770 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b51200640 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef849b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4b400640 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8468be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 -- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 msgr2=0x55cef8468be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8468be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 -- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 msgr2=0x55cef849b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef849b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 -- 172.21.15.87:0/674505770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cef82f75a0 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4be00640 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef84686a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55cef82b6f00 tx=0x7f5b3c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742a0d0298d1c99f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 1 -- 172.21.15.87:0/674505770 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c02f070 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 1 -- 172.21.15.87:0/674505770 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b3c0059a0 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b3c026070 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 1 -- 172.21.15.87:0/674505770 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3c01c020 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 10 client.6180 fetch_fsmap learned FSMap version 26 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55cef82882b0 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 1 -- 172.21.15.87:0/674505770 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c026070 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 1 -- 172.21.15.87:0/674505770 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b3c005be0 con 0x55cef8463600 2024-04-25T10:15:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 10 client.6180 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 10 client.6180 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 20 client.6180 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef849ae60 0x55cef849be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 4 client.6180 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cef824cc70 con 0x55cef849ae60 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4b400640 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef849ae60 0x55cef849be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 10 client.6180 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b4b400640 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef849ae60 0x55cef849be60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb0139e9af37acc server_cookie=52f7b53b3d593167 in_seq=0 out_seq=0 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 1 -- 172.21.15.87:0/674505770 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cef824cc70 con 0x55cef849ae60 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b35600640 10 client.6180 handle_command_reply: tid=0 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.331+0000 7f5b566f6000 1 client.6180 shutdown 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 msgr2=0x55cef84686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef84686a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55cef82b6f00 tx=0x7f5b3c004740 comp rx=0 tx=0).stop 2024-04-25T10:15:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 shutdown_connections 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef84adaa0 0x55cef849b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef849ae60 0x55cef849be60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8468be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 --2- 172.21.15.87:0/674505770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef84686a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b35600640 0 client.6180 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 >> 172.21.15.87:0/674505770 conn(0x55cef8469760 msgr2=0x55cef84620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 shutdown_connections 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 1 -- 172.21.15.87:0/674505770 wait complete. 2024-04-25T10:15:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.335+0000 7f5b566f6000 20 client.6180 trim_cache size 0 max 16384 2024-04-25T10:15:37.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 Processor -- start 2024-04-25T10:15:37.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- start start 2024-04-25T10:15:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8469760 0x55cef84c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef8462120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8462660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c2033c0 con 0x55cef8463600 2024-04-25T10:15:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c202580 con 0x55cef8469760 2024-04-25T10:15:37.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c201980 con 0x55cef84ad180 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef8462120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8469760 0x55cef84c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b51200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8462660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8469760 0x55cef84c3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:58840/0 (socket says 172.21.15.87:58840) 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4be00640 1 -- 172.21.15.87:0/802720020 learned_addr learned my addr 172.21.15.87:0/802720020 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4b400640 1 -- 172.21.15.87:0/802720020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 msgr2=0x55cef8462660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4b400640 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8462660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4b400640 1 -- 172.21.15.87:0/802720020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8469760 msgr2=0x55cef84c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4b400640 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8469760 0x55cef84c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4b400640 1 -- 172.21.15.87:0/802720020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cef849c1d0 con 0x55cef8463600 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4be00640 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8469760 0x55cef84c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b4b400640 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef8462120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b3811cec0 tx=0x7f5b380c2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2abab8d7e6bc3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b37e00640 1 -- 172.21.15.87:0/802720020 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b380f7960 con 0x55cef8463600 2024-04-25T10:15:37.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b37e00640 1 -- 172.21.15.87:0/802720020 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b380a9040 con 0x55cef8463600 2024-04-25T10:15:37.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b37e00640 1 -- 172.21.15.87:0/802720020 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b380ed3a0 con 0x55cef8463600 2024-04-25T10:15:37.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- 172.21.15.87:0/802720020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 msgr2=0x55cef8462120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef8462120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b3811cec0 tx=0x7f5b380c2040 comp rx=0 tx=0).stop 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- 172.21.15.87:0/802720020 shutdown_connections 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef8462660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8463600 0x55cef8462120 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 --2- 172.21.15.87:0/802720020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8469760 0x55cef84c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- 172.21.15.87:0/802720020 >> 172.21.15.87:0/802720020 conn(0x55cef849b5b0 msgr2=0x55cef84ba770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- 172.21.15.87:0/802720020 shutdown_connections 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.347+0000 7f5b566f6000 1 -- 172.21.15.87:0/802720020 wait complete. 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 -- start start 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 Processor -- start 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8463600 0x55cef850d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8469760 0x55cef850d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef84c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef82b5550 con 0x55cef8469760 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c201f80 con 0x55cef8463600 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cef836e9c0 con 0x55cef84ad180 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 10 client.0 _ll_register_callbacks cb 0x55cef842f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8469760 0x55cef850d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8469760 0x55cef850d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57966/0 (socket says 172.21.15.87:57966) 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 -- 172.21.15.87:0/3840667764 learned_addr learned my addr 172.21.15.87:0/3840667764 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4be00640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8463600 0x55cef850d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b51200640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef84c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 -- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 msgr2=0x55cef84c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef84c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 -- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8463600 msgr2=0x55cef850d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8463600 0x55cef850d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 -- 172.21.15.87:0/3840667764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cef8239250 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4be00640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8463600 0x55cef850d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4b400640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8469760 0x55cef850d8f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b38097e30 tx=0x7f5b3c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cee4366e96409c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b35600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b35600640 1 -- 172.21.15.87:0/3840667764 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3811ce70 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b35600640 1 -- 172.21.15.87:0/3840667764 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b380eb970 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b380a9040 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b35600640 1 -- 172.21.15.87:0/3840667764 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b380f74d0 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 10 client.6186 fetch_fsmap learned FSMap version 26 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55cef8245b90 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b35600640 1 -- 172.21.15.87:0/3840667764 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b380a9040 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b35600640 1 -- 172.21.15.87:0/3840667764 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b3810d0d0 con 0x55cef8469760 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 10 client.6186 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 10 client.6186 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 20 client.6186 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef84adaa0 0x55cef84c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 4 client.6186 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cef82be460 con 0x55cef84adaa0 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.351+0000 7f5b4be00640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef84adaa0 0x55cef84c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b4be00640 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef84adaa0 0x55cef84c2af0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80d46c27f0ec5b4 server_cookie=9f654939e7c247cf in_seq=0 out_seq=0 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b35600640 10 client.6186 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b35600640 1 -- 172.21.15.87:0/3840667764 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cef82be460 con 0x55cef84adaa0 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b35600640 10 client.6186 handle_command_reply: tid=0 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 client.6186 shutdown 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8469760 msgr2=0x55cef850d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8469760 0x55cef850d8f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b38097e30 tx=0x7f5b3c015000 comp rx=0 tx=0).stop 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 shutdown_connections 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55cef84adaa0 0x55cef84c2af0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cef84ad180 0x55cef84c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cef8469760 0x55cef850d8f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b35600640 0 client.6186 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 --2- 172.21.15.87:0/3840667764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cef8463600 0x55cef850d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 >> 172.21.15.87:0/3840667764 conn(0x55cef849b5b0 msgr2=0x55cef8461110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 shutdown_connections 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 1 -- 172.21.15.87:0/3840667764 wait complete. 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b566f6000 20 client.6186 trim_cache size 0 max 16384 2024-04-25T10:15:37.365 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-04-25T10:15:37.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b2c066f60 msgr2=0x7f5b2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b2c066f60 0x7f5b2c069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c15e3c0 tx=0x7f5b3c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:37.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cfb30 msgr2=0x7f5b4c0c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0c7230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b380b0040 tx=0x7f5b38019300 comp rx=0 tx=0).stop 2024-04-25T10:15:37.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 shutdown_connections 2024-04-25T10:15:37.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b2c066f60 0x7f5b2c069420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15dd30 0x7f5b4c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cfb30 0x7f5b4c0c7230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 --2- 172.21.15.87:0/279225656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b4c0ce220 0x7f5b4c1f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 >> 172.21.15.87:0/279225656 conn(0x7f5b4c09a9d0 msgr2=0x7f5b4c15cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 shutdown_connections 2024-04-25T10:15:37.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.355+0000 7f5b53200640 1 -- 172.21.15.87:0/279225656 wait complete. 2024-04-25T10:15:37.379 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096d.00000000 parent 2024-04-25T10:15:37.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 Processor -- start 2024-04-25T10:15:37.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- start start 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be357d9d00 0x55be357b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be3584ff10 0x55be358721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be35866da0 0x55be35872720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be357daab0 con 0x55be3584ff10 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be357c2620 con 0x55be357d9d00 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be357dad50 con 0x55be35866da0 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be3584ff10 0x55be358721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be3584ff10 0x55be358721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57978/0 (socket says 172.21.15.87:57978) 2024-04-25T10:15:37.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 -- 172.21.15.87:0/3722044565 learned_addr learned my addr 172.21.15.87:0/3722044565 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f672f000640 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be357d9d00 0x55be357b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f672fa00640 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be35866da0 0x55be35872720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 -- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be35866da0 msgr2=0x55be35872720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be35866da0 0x55be35872720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 -- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be357d9d00 msgr2=0x55be357b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be357d9d00 0x55be357b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 -- 172.21.15.87:0/3722044565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be3584f720 con 0x55be3584ff10 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f6727e00640 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be3584ff10 0x55be358721e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f671800a650 tx=0x7f671800ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa69c0df604a309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f672dc00640 1 -- 172.21.15.87:0/3722044565 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718008c00 con 0x55be3584ff10 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f672dc00640 1 -- 172.21.15.87:0/3722044565 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6718008da0 con 0x55be3584ff10 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f672dc00640 1 -- 172.21.15.87:0/3722044565 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67180135f0 con 0x55be3584ff10 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- 172.21.15.87:0/3722044565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be3584ff10 msgr2=0x55be358721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be3584ff10 0x55be358721e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f671800a650 tx=0x7f671800ab10 comp rx=0 tx=0).stop 2024-04-25T10:15:37.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- 172.21.15.87:0/3722044565 shutdown_connections 2024-04-25T10:15:37.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be35866da0 0x55be35872720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be3584ff10 0x55be358721e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 --2- 172.21.15.87:0/3722044565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be357d9d00 0x55be357b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- 172.21.15.87:0/3722044565 >> 172.21.15.87:0/3722044565 conn(0x55be357bc760 msgr2=0x55be3586b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- 172.21.15.87:0/3722044565 shutdown_connections 2024-04-25T10:15:37.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.419+0000 7f673284d080 1 -- 172.21.15.87:0/3722044565 wait complete. 2024-04-25T10:15:37.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 Processor -- start 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 -- start start 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be357d9d00 0x55be3584ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be3584ff10 0x55be3584cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be35866da0 0x55be3586ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3586be70 con 0x55be35866da0 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3586bbd0 con 0x55be3584ff10 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be3586c110 con 0x55be357d9d00 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be35866da0 0x55be3586ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be35866da0 0x55be3586ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57982/0 (socket says 172.21.15.87:57982) 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 -- 172.21.15.87:0/1586294646 learned_addr learned my addr 172.21.15.87:0/1586294646 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f6727e00640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be3584ff10 0x55be3584cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672f000640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be357d9d00 0x55be3584ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 -- 172.21.15.87:0/1586294646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be357d9d00 msgr2=0x55be3584ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55be357d9d00 0x55be3584ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 -- 172.21.15.87:0/1586294646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be3584ff10 msgr2=0x55be3584cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55be3584ff10 0x55be3584cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 -- 172.21.15.87:0/1586294646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be3584b1d0 con 0x55be35866da0 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f672fa00640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be35866da0 0x55be3586ce40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f672800e2b0 tx=0x7f672800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f6727400640 1 -- 172.21.15.87:0/1586294646 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728013870 con 0x55be35866da0 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f6727400640 1 -- 172.21.15.87:0/1586294646 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728013a10 con 0x55be35866da0 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55be357b8690 con 0x55be35866da0 2024-04-25T10:15:37.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f6727400640 1 -- 172.21.15.87:0/1586294646 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728013de0 con 0x55be35866da0 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.423+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be357b8bd0 con 0x55be35866da0 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f6727400640 1 -- 172.21.15.87:0/1586294646 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6728011070 con 0x55be35866da0 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f6727400640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f670c065970 0x7f670c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f6727400640 1 -- 172.21.15.87:0/1586294646 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6728084580 con 0x55be35866da0 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f672f000640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f670c065970 0x7f670c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f673284d080 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x55be357b9ac0 0x55be357b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 --> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] -- osd_op(unknown.0.0:1 7.21 7:844f785e:::1000000096d.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v9 -- 0x55be3584d2f0 con 0x55be357b9ac0 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f6727e00640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x55be357b9ac0 0x55be357b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f672f000640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f670c065970 0x7f670c067e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f67200ac000 tx=0x7f67200a5000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.427+0000 7f6727e00640 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x55be357b9ac0 0x55be357b9ea0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f6727e00640 1 -- 172.21.15.87:0/1586294646 <== osd.3 v2:172.21.15.87:6802/327355267 1 ==== osd_op_reply(1 1000000096d.00000000 [rmxattr] v46'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f671801e420 con 0x55be357b9ac0 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x55be357b9ac0 msgr2=0x55be357b9ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x55be357b9ac0 0x55be357b9ea0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f670c065970 msgr2=0x7f670c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f670c065970 0x7f670c067e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f67200ac000 tx=0x7f67200a5000 comp rx=0 tx=0).stop 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be35866da0 msgr2=0x55be3586ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 --2- 172.21.15.87:0/1586294646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be35866da0 0x55be3586ce40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f672800e2b0 tx=0x7f672800e770 comp rx=0 tx=0).stop 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f672fa00640 1 -- 172.21.15.87:0/1586294646 reap_dead start 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 shutdown_connections 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 >> 172.21.15.87:0/1586294646 conn(0x55be357bc760 msgr2=0x55be3584e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 shutdown_connections 2024-04-25T10:15:37.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.431+0000 7f673284d080 1 -- 172.21.15.87:0/1586294646 wait complete. 2024-04-25T10:15:37.440 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:15:37.440 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:37.440 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:15:37.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7e0400640 1 Processor -- start 2024-04-25T10:15:37.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7e0400640 1 -- start start 2024-04-25T10:15:37.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7e0400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7d815bc10 0x7fd7d815f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7e0400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7d80cf980 0x7fd7d80b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7e0400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d815fec0 con 0x7fd7d80d1830 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7e0400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d81601a0 con 0x7fd7d815bc10 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7e0400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d80cc5b0 con 0x7fd7d80cf980 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dda00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7d815bc10 0x7fd7d815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dda00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7d815bc10 0x7fd7d815f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57986/0 (socket says 172.21.15.87:57986) 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dda00640 1 -- 172.21.15.87:0/118744907 learned_addr learned my addr 172.21.15.87:0/118744907 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dd000640 1 --2- 172.21.15.87:0/118744907 >> v2:172.21.15.169:3300/0 conn(0x7fd7d80cf980 0x7fd7d80b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dda00640 1 -- 172.21.15.87:0/118744907 >> v2:172.21.15.169:3300/0 conn(0x7fd7d80cf980 msgr2=0x7fd7d80b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dda00640 1 --2- 172.21.15.87:0/118744907 >> v2:172.21.15.169:3300/0 conn(0x7fd7d80cf980 0x7fd7d80b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dda00640 1 -- 172.21.15.87:0/118744907 >> v1:172.21.15.169:6789/0 conn(0x7fd7d80d1830 legacy=0x7fd7d80d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dda00640 1 -- 172.21.15.87:0/118744907 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80cc890 con 0x7fd7d815bc10 2024-04-25T10:15:37.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.619+0000 7fd7dd000640 1 --2- 172.21.15.87:0/118744907 >> v2:172.21.15.169:3300/0 conn(0x7fd7d80cf980 0x7fd7d80b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:37.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7dda00640 1 --2- 172.21.15.87:0/118744907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d815bc10 0x7fd7d815f600 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80cdc80 tx=0x7fd7c801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d16b759a88929b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7d7e00640 1 -- 172.21.15.87:0/118744907 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80ab5f0 con 0x7fd7d815bc10 2024-04-25T10:15:37.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7d7e00640 1 -- 172.21.15.87:0/118744907 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c80b0070 con 0x7fd7d815bc10 2024-04-25T10:15:37.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7d7e00640 1 -- 172.21.15.87:0/118744907 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80ab9b0 con 0x7fd7d815bc10 2024-04-25T10:15:37.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 -- 172.21.15.87:0/118744907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d815bc10 msgr2=0x7fd7d815f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 --2- 172.21.15.87:0/118744907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d815bc10 0x7fd7d815f600 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80cdc80 tx=0x7fd7c801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 -- 172.21.15.87:0/118744907 shutdown_connections 2024-04-25T10:15:37.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 --2- 172.21.15.87:0/118744907 >> v2:172.21.15.169:3300/0 conn(0x7fd7d80cf980 0x7fd7d80b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 --2- 172.21.15.87:0/118744907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d815bc10 0x7fd7d815f600 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 -- 172.21.15.87:0/118744907 >> 172.21.15.87:0/118744907 conn(0x7fd7d809a9d0 msgr2=0x7fd7d809adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 -- 172.21.15.87:0/118744907 shutdown_connections 2024-04-25T10:15:37.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.623+0000 7fd7e0400640 1 -- 172.21.15.87:0/118744907 wait complete. 2024-04-25T10:15:37.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 Processor -- start 2024-04-25T10:15:37.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 -- start start 2024-04-25T10:15:37.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d80cf980 0x7fd7d8152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7d80d1830 0x7fd7d815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d815bc10 0x7fd7d815e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80cd5c0 con 0x7fd7d80cf980 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80b17b0 con 0x7fd7d815bc10 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80cd740 con 0x7fd7d80d1830 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d80cf980 0x7fd7d8152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d80cf980 0x7fd7d8152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57994/0 (socket says 172.21.15.87:57994) 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 -- 172.21.15.87:0/4154795995 learned_addr learned my addr 172.21.15.87:0/4154795995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dd000640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7d80d1830 0x7fd7d815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7de400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d815bc10 0x7fd7d815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 -- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7d80d1830 msgr2=0x7fd7d815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7d80d1830 0x7fd7d815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 -- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d815bc10 msgr2=0x7fd7d815e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d815bc10 0x7fd7d815e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 -- 172.21.15.87:0/4154795995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80af640 con 0x7fd7d80cf980 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dd000640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7d80d1830 0x7fd7d815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7de400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d815bc10 0x7fd7d815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7dda00640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d80cf980 0x7fd7d8152eb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80ad040 tx=0x7fd7c801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7d5600640 1 -- 172.21.15.87:0/4154795995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80be5f0 con 0x7fd7d80cf980 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7d5600640 1 -- 172.21.15.87:0/4154795995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c80be7b0 con 0x7fd7d80cf980 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d80af920 con 0x7fd7d80cf980 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7d5600640 1 -- 172.21.15.87:0/4154795995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80beba0 con 0x7fd7d80cf980 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.627+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d80afee0 con 0x7fd7d80cf980 2024-04-25T10:15:37.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.631+0000 7fd7d5600640 1 -- 172.21.15.87:0/4154795995 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7c80b43f0 con 0x7fd7d80cf980 2024-04-25T10:15:37.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.631+0000 7fd7d5600640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7bc066f60 0x7fd7bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.631+0000 7fd7d5600640 1 -- 172.21.15.87:0/4154795995 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7c811fa00 con 0x7fd7d80cf980 2024-04-25T10:15:37.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.631+0000 7fd7dd000640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7bc066f60 0x7fd7bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.631+0000 7fd7e3840000 1 Processor -- start 2024-04-25T10:15:37.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.631+0000 7fd7e3840000 1 -- start start 2024-04-25T10:15:37.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.631+0000 7fd7dd000640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7bc066f60 0x7fd7bc069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8155570 tx=0x7fd7cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc88605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc88133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8813d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc8663ce0 con 0x55bfc88601f0 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc85ec530 con 0x55bfc8813920 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfc865fd80 con 0x55bfc8860b10 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc88605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc88605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57998/0 (socket says 172.21.15.87:57998) 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 -- 172.21.15.87:0/1410036108 learned_addr learned my addr 172.21.15.87:0/1410036108 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7de400640 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8813d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dd000640 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc88133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 -- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 msgr2=0x55bfc88133e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc88133e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 -- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 msgr2=0x55bfc8813d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8813d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 -- 172.21.15.87:0/1410036108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc865ecd0 con 0x55bfc88601f0 2024-04-25T10:15:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7dda00640 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc88605d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd7c801a110 tx=0x7fd7c8019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dbbdff3bc01555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7c6000640 1 -- 172.21.15.87:0/1410036108 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80f8aa0 con 0x55bfc88601f0 2024-04-25T10:15:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7c6000640 1 -- 172.21.15.87:0/1410036108 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c80b0020 con 0x55bfc88601f0 2024-04-25T10:15:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7c6000640 1 -- 172.21.15.87:0/1410036108 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80f4150 con 0x55bfc88601f0 2024-04-25T10:15:37.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- 172.21.15.87:0/1410036108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 msgr2=0x55bfc88605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc88605d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd7c801a110 tx=0x7fd7c8019280 comp rx=0 tx=0).stop 2024-04-25T10:15:37.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- 172.21.15.87:0/1410036108 shutdown_connections 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8813d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc88133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1410036108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc88605d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- 172.21.15.87:0/1410036108 >> 172.21.15.87:0/1410036108 conn(0x55bfc8840f90 msgr2=0x55bfc8843380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- 172.21.15.87:0/1410036108 shutdown_connections 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- 172.21.15.87:0/1410036108 wait complete. 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- start start 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 Processor -- start 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8818ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc8819220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc8819760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.635+0000 7fd7e3840000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc8663ce0 con 0x55bfc88601f0 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc861ebc0 con 0x55bfc8813920 2024-04-25T10:15:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfc881f130 con 0x55bfc8860b10 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 10 client.0 _ll_register_callbacks cb 0x55bfc88014a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc8819220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc8819220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58002/0 (socket says 172.21.15.87:58002) 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 -- 172.21.15.87:0/2487513507 learned_addr learned my addr 172.21.15.87:0/2487513507 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7de400640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc8819760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dda00640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8818ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 -- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 msgr2=0x55bfc8819760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc8819760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 -- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 msgr2=0x55bfc8818ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8818ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 -- 172.21.15.87:0/2487513507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfc8671950 con 0x55bfc88601f0 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dda00640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8818ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7de400640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc8819760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dd000640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc8819220 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0126c0 tx=0x7fd7cc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b37d2186ec1543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 1 -- 172.21.15.87:0/2487513507 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc023070 con 0x55bfc88601f0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 1 -- 172.21.15.87:0/2487513507 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7cc01f800 con 0x55bfc88601f0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7c80b0020 con 0x55bfc88601f0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 1 -- 172.21.15.87:0/2487513507 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7cc02e050 con 0x55bfc88601f0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 10 client.6207 fetch_fsmap learned FSMap version 26 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55bfc86e0790 con 0x55bfc88601f0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 1 -- 172.21.15.87:0/2487513507 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c80b0020 con 0x55bfc88601f0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 1 -- 172.21.15.87:0/2487513507 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7cc031020 con 0x55bfc88601f0 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 10 client.6207 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 10 client.6207 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 20 client.6207 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc881a170 0x55bfc88756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 4 client.6207 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfc85ffb80 con 0x55bfc881a170 2024-04-25T10:15:37.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dda00640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc881a170 0x55bfc88756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7dda00640 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc881a170 0x55bfc88756a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f0cc11dc0a4470 server_cookie=cd77981717691ee5 in_seq=0 out_seq=0 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 10 client.6207 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 1 -- 172.21.15.87:0/2487513507 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfc85ffb80 con 0x55bfc881a170 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 10 client.6207 handle_command_reply: tid=0 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 client.6207 shutdown 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 msgr2=0x55bfc8819220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc8819220 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0126c0 tx=0x7fd7cc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 shutdown_connections 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc881a170 0x55bfc88756a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc8860b10 0x55bfc8819760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc88601f0 0x55bfc8819220 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7baa00640 0 client.6207 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2487513507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc8818ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 >> 172.21.15.87:0/2487513507 conn(0x55bfc8840f90 msgr2=0x55bfc8843380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.639+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 shutdown_connections 2024-04-25T10:15:37.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.643+0000 7fd7e3840000 1 -- 172.21.15.87:0/2487513507 wait complete. 2024-04-25T10:15:37.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.643+0000 7fd7e3840000 20 client.6207 trim_cache size 0 max 16384 2024-04-25T10:15:37.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 Processor -- start 2024-04-25T10:15:37.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 -- start start 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8813920 0x55bfc88c0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8860b10 0x55bfc88c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d806c6d0 con 0x55bfc8813920 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d815aa30 con 0x55bfc8860b10 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80b2fa0 con 0x55bfc88601f0 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7dda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8813920 0x55bfc88c0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7dda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8813920 0x55bfc88c0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58014/0 (socket says 172.21.15.87:58014) 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.655+0000 7fd7dda00640 1 -- 172.21.15.87:0/1260899779 learned_addr learned my addr 172.21.15.87:0/1260899779 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dd000640 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8860b10 0x55bfc88c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dda00640 1 -- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 msgr2=0x55bfc88c0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dda00640 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dda00640 1 -- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8860b10 msgr2=0x55bfc88c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dda00640 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8860b10 0x55bfc88c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dda00640 1 -- 172.21.15.87:0/1260899779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc86296f0 con 0x55bfc8813920 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dda00640 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8813920 0x55bfc88c0530 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80f0550 tx=0x7fd7c80095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae64a4fc6987a690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7c5600640 1 -- 172.21.15.87:0/1260899779 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c811b070 con 0x55bfc8813920 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7c5600640 1 -- 172.21.15.87:0/1260899779 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c811d070 con 0x55bfc8813920 2024-04-25T10:15:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7c5600640 1 -- 172.21.15.87:0/1260899779 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80eca00 con 0x55bfc8813920 2024-04-25T10:15:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- 172.21.15.87:0/1260899779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8813920 msgr2=0x55bfc88c0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8813920 0x55bfc88c0530 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80f0550 tx=0x7fd7c80095f0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- 172.21.15.87:0/1260899779 shutdown_connections 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8860b10 0x55bfc88c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 --2- 172.21.15.87:0/1260899779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8813920 0x55bfc88c0530 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- 172.21.15.87:0/1260899779 >> 172.21.15.87:0/1260899779 conn(0x55bfc8840f90 msgr2=0x55bfc885eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- 172.21.15.87:0/1260899779 shutdown_connections 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- 172.21.15.87:0/1260899779 wait complete. 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- start start 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 Processor -- start 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc88c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8860b10 0x55bfc887f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc8668310 con 0x55bfc8860b10 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d815a430 con 0x55bfc8813920 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfc860a8e0 con 0x55bfc88601f0 2024-04-25T10:15:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7e3840000 10 client.0 _ll_register_callbacks cb 0x7fd7c80ec630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8860b10 0x55bfc887f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8860b10 0x55bfc887f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58028/0 (socket says 172.21.15.87:58028) 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 -- 172.21.15.87:0/2552212771 learned_addr learned my addr 172.21.15.87:0/2552212771 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dd000640 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7dda00640 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc88c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 -- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 msgr2=0x55bfc88c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 -- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 msgr2=0x55bfc88c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc88c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.659+0000 7fd7de400640 1 -- 172.21.15.87:0/2552212771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfc861fc80 con 0x55bfc8860b10 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7de400640 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8860b10 0x55bfc887f120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0011f10 tx=0x7fd7d0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4610a00bc2b5744a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 1 -- 172.21.15.87:0/2552212771 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d001f060 con 0x55bfc8860b10 2024-04-25T10:15:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 1 -- 172.21.15.87:0/2552212771 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7d000faa0 con 0x55bfc8860b10 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7c811d070 con 0x55bfc8860b10 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 1 -- 172.21.15.87:0/2552212771 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d000fca0 con 0x55bfc8860b10 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 10 client.6213 fetch_fsmap learned FSMap version 26 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55bfc8320600 con 0x55bfc8860b10 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 1 -- 172.21.15.87:0/2552212771 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c811d070 con 0x55bfc8860b10 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 1 -- 172.21.15.87:0/2552212771 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7d001a410 con 0x55bfc8860b10 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 10 client.6213 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 10 client.6213 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 20 client.6213 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc887ffb0 0x55bfc88803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 4 client.6213 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55bfc8671220 con 0x55bfc887ffb0 2024-04-25T10:15:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7dda00640 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc887ffb0 0x55bfc88803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.672 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T10:15:37.663+0000 7f6ac1a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096d [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v647 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5512=0-4194304@1} caps={5512=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5512 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55df4712ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1000000096d:[<0x10000000001/new_file_1 v647>,<0x10000000000/ceph-qa-suite v70>,<0x1/scrub_test_path v23>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7dda00640 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc887ffb0 0x55bfc88803c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4555b045e3f9d24e server_cookie=1bd74841b635074a in_seq=0 out_seq=0 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 10 client.6213 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 1 -- 172.21.15.87:0/2552212771 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bfc8671220 con 0x55bfc887ffb0 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 10 client.6213 handle_command_reply: tid=0 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 client.6213 shutdown 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8860b10 msgr2=0x55bfc887f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8860b10 0x55bfc887f120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0011f10 tx=0x7fd7d0007db0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 shutdown_connections 2024-04-25T10:15:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfc8860b10 0x55bfc887f120 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55bfc887ffb0 0x55bfc88803c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfc88601f0 0x55bfc88c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 --2- 172.21.15.87:0/2552212771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfc8813920 0x55bfc88c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7baa00640 0 client.6213 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 >> 172.21.15.87:0/2552212771 conn(0x55bfc8840f90 msgr2=0x55bfc885f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 shutdown_connections 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 1 -- 172.21.15.87:0/2552212771 wait complete. 2024-04-25T10:15:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e3840000 20 client.6213 trim_cache size 0 max 16384 2024-04-25T10:15:37.675 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:37.675 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:15:37.675 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "a5b7569b-455a-41c3-a8aa-2a61ff9ad3e1", 2024-04-25T10:15:37.675 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:15:37.675 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7bc066f60 msgr2=0x7fd7bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7bc066f60 0x7fd7bc069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8155570 tx=0x7fd7cc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d80cf980 msgr2=0x7fd7d8152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d80cf980 0x7fd7d8152eb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80ad040 tx=0x7fd7c801ad20 comp rx=0 tx=0).stop 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 shutdown_connections 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7bc066f60 0x7fd7bc069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d815bc10 0x7fd7d815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7d80d1830 0x7fd7d815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 --2- 172.21.15.87:0/4154795995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7d80cf980 0x7fd7d8152eb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 >> 172.21.15.87:0/4154795995 conn(0x7fd7d809a9d0 msgr2=0x7fd7d806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 shutdown_connections 2024-04-25T10:15:37.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.663+0000 7fd7e0400640 1 -- 172.21.15.87:0/4154795995 wait complete. 2024-04-25T10:15:37.689 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': 'a5b7569b-455a-41c3-a8aa-2a61ff9ad3e1', 'mode': 'asynchronous'}' 2024-04-25T10:15:37.689 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:15:37.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:15:37.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f8522600640 1 Processor -- start 2024-04-25T10:15:37.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f8522600640 1 -- start start 2024-04-25T10:15:37.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f8522600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c0aeda0 0x7f851c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f8522600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c159920 con 0x7f851c072020 2024-04-25T10:15:37.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f8522600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c15c2f0 con 0x7f851c159540 2024-04-25T10:15:37.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f8522600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c15c5d0 con 0x7f851c0aeda0 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f851be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f851c072020 0x7f851c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47724/0 (socket says 172.21.15.87:47724) 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.819+0000 7f851be00640 1 -- 172.21.15.87:0/3816786536 learned_addr learned my addr 172.21.15.87:0/3816786536 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f8512a00640 1 --2- 172.21.15.87:0/3816786536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c0aeda0 0x7f851c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f851aa00640 1 -- 172.21.15.87:0/3816786536 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285311575 0 0) 0x7f851c159920 con 0x7f851c072020 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f851aa00640 1 -- 172.21.15.87:0/3816786536 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8500003530 con 0x7f851c072020 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f851aa00640 1 -- 172.21.15.87:0/3816786536 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725213110 0 0) 0x7f851c15c2f0 con 0x7f851c159540 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f851aa00640 1 -- 172.21.15.87:0/3816786536 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f851c159920 con 0x7f851c159540 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f8512a00640 1 -- 172.21.15.87:0/3816786536 >> v1:172.21.15.169:6789/0 conn(0x7f851c159540 legacy=0x7f851c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f8512a00640 1 -- 172.21.15.87:0/3816786536 >> v1:172.21.15.87:6789/0 conn(0x7f851c072020 legacy=0x7f851c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f8512a00640 1 -- 172.21.15.87:0/3816786536 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c15c8b0 con 0x7f851c0aeda0 2024-04-25T10:15:37.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f8512a00640 1 --2- 172.21.15.87:0/3816786536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c0aeda0 0x7f851c071ae0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85040097f0 tx=0x7f850401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4945f17578c83e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f851aa00640 1 -- 172.21.15.87:0/3816786536 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040aa430 con 0x7f851c0aeda0 2024-04-25T10:15:37.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f851aa00640 1 -- 172.21.15.87:0/3816786536 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85040af070 con 0x7f851c0aeda0 2024-04-25T10:15:37.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.823+0000 7f851aa00640 1 -- 172.21.15.87:0/3816786536 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040aa7f0 con 0x7f851c0aeda0 2024-04-25T10:15:37.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- 172.21.15.87:0/3816786536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c0aeda0 msgr2=0x7f851c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 --2- 172.21.15.87:0/3816786536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c0aeda0 0x7f851c071ae0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85040097f0 tx=0x7f850401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- 172.21.15.87:0/3816786536 shutdown_connections 2024-04-25T10:15:37.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 --2- 172.21.15.87:0/3816786536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c0aeda0 0x7f851c071ae0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- 172.21.15.87:0/3816786536 >> 172.21.15.87:0/3816786536 conn(0x7f851c0669a0 msgr2=0x7f851c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- 172.21.15.87:0/3816786536 shutdown_connections 2024-04-25T10:15:37.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- 172.21.15.87:0/3816786536 wait complete. 2024-04-25T10:15:37.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 Processor -- start 2024-04-25T10:15:37.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- start start 2024-04-25T10:15:37.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f851c072020 0x7f851c140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f851c0aeda0 0x7f851c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c159540 0x7f851c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c169940 con 0x7f851c072020 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c072430 con 0x7f851c0aeda0 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8522600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c169ac0 con 0x7f851c159540 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f851c072020 0x7f851c140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f851c072020 0x7f851c140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58034/0 (socket says 172.21.15.87:58034) 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 -- 172.21.15.87:0/113964159 learned_addr learned my addr 172.21.15.87:0/113964159 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8512a00640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f851c0aeda0 0x7f851c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851be00640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c159540 0x7f851c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 -- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c159540 msgr2=0x7f851c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c159540 0x7f851c15f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 -- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f851c0aeda0 msgr2=0x7f851c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f851c0aeda0 0x7f851c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 -- 172.21.15.87:0/113964159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c150cd0 con 0x7f851c072020 2024-04-25T10:15:37.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f8512a00640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f851c0aeda0 0x7f851c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:37.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.827+0000 7f851b400640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f851c072020 0x7f851c140630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c460 tx=0x7f850c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8513e00640 1 -- 172.21.15.87:0/113964159 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c017070 con 0x7f851c072020 2024-04-25T10:15:37.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8513e00640 1 -- 172.21.15.87:0/113964159 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c0052f0 con 0x7f851c072020 2024-04-25T10:15:37.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8513e00640 1 -- 172.21.15.87:0/113964159 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c003840 con 0x7f851c072020 2024-04-25T10:15:37.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c14ff10 con 0x7f851c072020 2024-04-25T10:15:37.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c1504d0 con 0x7f851c072020 2024-04-25T10:15:37.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8513e00640 1 -- 172.21.15.87:0/113964159 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f850c01a070 con 0x7f851c072020 2024-04-25T10:15:37.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8513e00640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84f8066f60 0x7f84f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8513e00640 1 -- 172.21.15.87:0/113964159 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f850c084350 con 0x7f851c072020 2024-04-25T10:15:37.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8512a00640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84f8066f60 0x7f84f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8525a7a000 1 Processor -- start 2024-04-25T10:15:37.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.831+0000 7f8525a7a000 1 -- start start 2024-04-25T10:15:37.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84f8066f60 0x7f84f8069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85040ae000 tx=0x7f85040a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6ff8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6faeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6faf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a6dfbff0 con 0x5599a6ff92f0 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a6d84150 con 0x5599a6faf0a0 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599a6df8090 con 0x5599a6ff89d0 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6faeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6faeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58044/0 (socket says 172.21.15.87:58044) 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 -- 172.21.15.87:0/1780106309 learned_addr learned my addr 172.21.15.87:0/1780106309 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f851be00640 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6faf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f851b400640 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6ff8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 -- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 msgr2=0x5599a6ff8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6ff8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 -- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 msgr2=0x5599a6faf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6faf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 -- 172.21.15.87:0/1780106309 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a6f920d0 con 0x5599a6ff92f0 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8512a00640 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6faeb60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f85040190c0 tx=0x7f8504004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c508bf347097f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f850be00640 1 -- 172.21.15.87:0/1780106309 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040bb070 con 0x5599a6ff92f0 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f850be00640 1 -- 172.21.15.87:0/1780106309 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85040af040 con 0x5599a6ff92f0 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f850be00640 1 -- 172.21.15.87:0/1780106309 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040aae60 con 0x5599a6ff92f0 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- 172.21.15.87:0/1780106309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 msgr2=0x5599a6faeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6faeb60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f85040190c0 tx=0x7f8504004740 comp rx=0 tx=0).stop 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- 172.21.15.87:0/1780106309 shutdown_connections 2024-04-25T10:15:37.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6faf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6faeb60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1780106309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6ff8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- 172.21.15.87:0/1780106309 >> 172.21.15.87:0/1780106309 conn(0x5599a6fab020 msgr2=0x5599a6fad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- 172.21.15.87:0/1780106309 shutdown_connections 2024-04-25T10:15:37.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- 172.21.15.87:0/1780106309 wait complete. 2024-04-25T10:15:37.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 -- start start 2024-04-25T10:15:37.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.835+0000 7f8525a7a000 1 Processor -- start 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6faf0a0 0x5599a6fdd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6ff89d0 0x5599a6fddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fde270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a6dc0250 con 0x5599a6ff92f0 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a6db6690 con 0x5599a6ff89d0 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599a6d84a40 con 0x5599a6faf0a0 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 10 client.0 _ll_register_callbacks cb 0x5599a6f99e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f851b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6faf0a0 0x5599a6fdd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f851be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fde270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f851b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6faf0a0 0x5599a6fdd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:55896/0 (socket says 172.21.15.87:55896) 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f851b400640 1 -- 172.21.15.87:0/1378997468 learned_addr learned my addr 172.21.15.87:0/1378997468 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8512a00640 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6ff89d0 0x5599a6fddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8512a00640 1 -- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6faf0a0 msgr2=0x5599a6fdd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8512a00640 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6faf0a0 0x5599a6fdd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8512a00640 1 -- 172.21.15.87:0/1378997468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 msgr2=0x5599a6fde270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8512a00640 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fde270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8512a00640 1 -- 172.21.15.87:0/1378997468 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599a6ea91c0 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8512a00640 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6ff89d0 0x5599a6fddd30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8504009740 tx=0x7f850401acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b05af053836d2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8508c00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8508c00640 1 -- 172.21.15.87:0/1378997468 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040bb040 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85040af040 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8508c00640 1 -- 172.21.15.87:0/1378997468 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85040ccbc0 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8508c00640 1 -- 172.21.15.87:0/1378997468 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85040dd050 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 10 client.5851 fetch_fsmap learned FSMap version 26 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5599a6ea5170 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8508c00640 1 -- 172.21.15.87:0/1378997468 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85040af040 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8508c00640 1 -- 172.21.15.87:0/1378997468 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85040e0020 con 0x5599a6ff89d0 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 10 client.5851 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:15:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 10 client.5851 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 20 client.5851 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a7008c80 0x5599a700b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 4 client.5851 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599a6d97c60 con 0x5599a7008c80 2024-04-25T10:15:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f851b400640 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a7008c80 0x5599a700b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f851b400640 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a7008c80 0x5599a700b070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b88578cba37da3f server_cookie=6a9bd7ef108d6daf in_seq=0 out_seq=0 2024-04-25T10:15:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.839+0000 7f8508c00640 10 client.5851 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8508c00640 1 -- 172.21.15.87:0/1378997468 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599a6d97c60 con 0x5599a7008c80 2024-04-25T10:15:37.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8508c00640 10 client.5851 handle_command_reply: tid=0 2024-04-25T10:15:37.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 client.5851 shutdown 2024-04-25T10:15:37.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6ff89d0 msgr2=0x5599a6fddd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6ff89d0 0x5599a6fddd30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8504009740 tx=0x7f850401acf0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 shutdown_connections 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fde270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6ff89d0 0x5599a6fddd30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a7008c80 0x5599a700b070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1378997468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6faf0a0 0x5599a6fdd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 >> 172.21.15.87:0/1378997468 conn(0x5599a6fab020 msgr2=0x5599a6fadca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8508c00640 0 client.5851 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 shutdown_connections 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 1 -- 172.21.15.87:0/1378997468 wait complete. 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.843+0000 7f8525a7a000 20 client.5851 trim_cache size 0 max 16384 2024-04-25T10:15:37.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.855+0000 7f8525a7a000 1 Processor -- start 2024-04-25T10:15:37.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.855+0000 7f8525a7a000 1 -- start start 2024-04-25T10:15:37.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6ff7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6fae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c202d40 con 0x5599a6ff92f0 2024-04-25T10:15:37.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c201f40 con 0x5599a6faf0a0 2024-04-25T10:15:37.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c1531e0 con 0x5599a6ff89d0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58068/0 (socket says 172.21.15.87:58068) 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 -- 172.21.15.87:0/1198772194 learned_addr learned my addr 172.21.15.87:0/1198772194 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851b400640 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6ff7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8512a00640 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6fae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 -- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 msgr2=0x5599a6fae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6fae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 -- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 msgr2=0x5599a6ff7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6ff7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 -- 172.21.15.87:0/1198772194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a6e42de0 con 0x5599a6ff92f0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8512a00640 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6fae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fae960 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8514007120 tx=0x7f85140075e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94de94deee8789b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f850b400640 1 -- 172.21.15.87:0/1198772194 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514016750 con 0x5599a6ff92f0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f850b400640 1 -- 172.21.15.87:0/1198772194 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851401a030 con 0x5599a6ff92f0 2024-04-25T10:15:37.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f850b400640 1 -- 172.21.15.87:0/1198772194 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514016af0 con 0x5599a6ff92f0 2024-04-25T10:15:37.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- 172.21.15.87:0/1198772194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 msgr2=0x5599a6fae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fae960 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8514007120 tx=0x7f85140075e0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- 172.21.15.87:0/1198772194 shutdown_connections 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a6fae960 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a6fae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- 172.21.15.87:0/1198772194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a6ff7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- 172.21.15.87:0/1198772194 >> 172.21.15.87:0/1198772194 conn(0x5599a6fab020 msgr2=0x5599a6ff70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- 172.21.15.87:0/1198772194 shutdown_connections 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- 172.21.15.87:0/1198772194 wait complete. 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- start start 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 Processor -- start 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a7059520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a7059a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a700b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a6d96e00 con 0x5599a6ff92f0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c201940 con 0x5599a6faf0a0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599a6dfb300 con 0x5599a6ff89d0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8525a7a000 10 client.0 _ll_register_callbacks cb 0x5599a6f7a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a700b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8512a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a7059a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f851b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a7059520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8512a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a7059a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:55918/0 (socket says 172.21.15.87:55918) 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.859+0000 7f8512a00640 1 -- 172.21.15.87:0/756311411 learned_addr learned my addr 172.21.15.87:0/756311411 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8512a00640 1 -- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 msgr2=0x5599a7059520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8512a00640 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a7059520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8512a00640 1 -- 172.21.15.87:0/756311411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 msgr2=0x5599a700b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8512a00640 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a700b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8512a00640 1 -- 172.21.15.87:0/756311411 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599a6d9d8e0 con 0x5599a6ff89d0 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f851be00640 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a700b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f851b400640 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a7059520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:37.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8512a00640 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a7059a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f850401c4f0 tx=0x7f85040c2070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba5f48d0a1d87d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 1 -- 172.21.15.87:0/756311411 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040bb070 con 0x5599a6ff89d0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f851401a030 con 0x5599a6ff89d0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 1 -- 172.21.15.87:0/756311411 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85040b76f0 con 0x5599a6ff89d0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 1 -- 172.21.15.87:0/756311411 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85040d5490 con 0x5599a6ff89d0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 10 client.5498 fetch_fsmap learned FSMap version 26 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5599a6e558f0 con 0x5599a6ff89d0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 1 -- 172.21.15.87:0/756311411 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851401a030 con 0x5599a6ff89d0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 1 -- 172.21.15.87:0/756311411 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85040e0040 con 0x5599a6ff89d0 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 10 client.5498 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 10 client.5498 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 20 client.5498 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:37.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a700bcb0 0x5599a700c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 4 client.5498 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599a6e09530 con 0x5599a700bcb0 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f851b400640 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a700bcb0 0x5599a700c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f851b400640 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a700bcb0 0x5599a700c090 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55553ef7c9de79d1 server_cookie=5160aa6d7d28a639 in_seq=0 out_seq=0 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 10 client.5498 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 1 -- 172.21.15.87:0/756311411 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x5599a6e09530 con 0x5599a700bcb0 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 10 client.5498 handle_command_reply: tid=0 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 client.5498 shutdown 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 msgr2=0x5599a7059a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a7059a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f850401c4f0 tx=0x7f85040c2070 comp rx=0 tx=0).stop 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 shutdown_connections 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5599a6ff92f0 0x5599a700b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5599a700bcb0 0x5599a700c090 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5599a6ff89d0 0x5599a7059a60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 --2- 172.21.15.87:0/756311411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a6faf0a0 0x5599a7059520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 >> 172.21.15.87:0/756311411 conn(0x5599a6fab020 msgr2=0x5599a6ff7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8508c00640 0 client.5498 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 shutdown_connections 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 1 -- 172.21.15.87:0/756311411 wait complete. 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8525a7a000 20 client.5498 trim_cache size 0 max 16384 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "a5b7569b-455a-41c3-a8aa-2a61ff9ad3e1": { 2024-04-25T10:15:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T10:15:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "a5b7569b-455a-41c3-a8aa-2a61ff9ad3e1", 2024-04-25T10:15:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "" 2024-04-25T10:15:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:37.875 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84f8066f60 msgr2=0x7f84f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84f8066f60 0x7f84f8069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85040ae000 tx=0x7f85040a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f851c072020 msgr2=0x7f851c140630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f851c072020 0x7f851c140630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c460 tx=0x7f850c00c920 comp rx=0 tx=0).stop 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 shutdown_connections 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84f8066f60 0x7f84f8069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f851c159540 0x7f851c15f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f851c0aeda0 0x7f851c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 --2- 172.21.15.87:0/113964159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f851c072020 0x7f851c140630 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 >> 172.21.15.87:0/113964159 conn(0x7f851c0669a0 msgr2=0x7f851c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 shutdown_connections 2024-04-25T10:15:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:37.863+0000 7f8522600640 1 -- 172.21.15.87:0/113964159 wait complete. 2024-04-25T10:15:42.891 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:15:42.891 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:15:43.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f645e200640 1 Processor -- start 2024-04-25T10:15:43.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f645e200640 1 -- start start 2024-04-25T10:15:43.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f645e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6458070350 0x7f645815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f645e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64580b31c0 0x7f64580b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f645e200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645815b080 con 0x7f64580aef80 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f645e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645815adb0 con 0x7f64580b31c0 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f645e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645815eb60 con 0x7f6458070350 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6457400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f64580aef80 0x7f645815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47736/0 (socket says 172.21.15.87:47736) 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6457400640 1 -- 172.21.15.87:0/1628355587 learned_addr learned my addr 172.21.15.87:0/1628355587 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6457e00640 1 --2- 172.21.15.87:0/1628355587 >> v2:172.21.15.87:3300/0 conn(0x7f64580b31c0 0x7f64580b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6456a00640 1 --2- 172.21.15.87:0/1628355587 >> v2:172.21.15.169:3300/0 conn(0x7f6458070350 0x7f645815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6456000640 1 -- 172.21.15.87:0/1628355587 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811342552 0 0) 0x7f645815b080 con 0x7f64580aef80 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6456000640 1 -- 172.21.15.87:0/1628355587 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c005740 con 0x7f64580aef80 2024-04-25T10:15:43.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6457e00640 1 -- 172.21.15.87:0/1628355587 >> v2:172.21.15.169:3300/0 conn(0x7f6458070350 msgr2=0x7f645815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6457e00640 1 --2- 172.21.15.87:0/1628355587 >> v2:172.21.15.169:3300/0 conn(0x7f6458070350 0x7f645815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6457e00640 1 -- 172.21.15.87:0/1628355587 >> v1:172.21.15.87:6789/0 conn(0x7f64580aef80 legacy=0x7f645815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.035+0000 7f6457e00640 1 -- 172.21.15.87:0/1628355587 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645815ee40 con 0x7f64580b31c0 2024-04-25T10:15:43.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f6457e00640 1 --2- 172.21.15.87:0/1628355587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580b31c0 0x7f64580b3610 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f645806e390 tx=0x7f645001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651e33a3af2a4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f6456000640 1 -- 172.21.15.87:0/1628355587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500b2070 con 0x7f64580b31c0 2024-04-25T10:15:43.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f6456000640 1 -- 172.21.15.87:0/1628355587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64500a5900 con 0x7f64580b31c0 2024-04-25T10:15:43.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f6456000640 1 -- 172.21.15.87:0/1628355587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500a5cd0 con 0x7f64580b31c0 2024-04-25T10:15:43.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 -- 172.21.15.87:0/1628355587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580b31c0 msgr2=0x7f64580b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 --2- 172.21.15.87:0/1628355587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580b31c0 0x7f64580b3610 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f645806e390 tx=0x7f645001c930 comp rx=0 tx=0).stop 2024-04-25T10:15:43.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 -- 172.21.15.87:0/1628355587 shutdown_connections 2024-04-25T10:15:43.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 --2- 172.21.15.87:0/1628355587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580b31c0 0x7f64580b3610 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 --2- 172.21.15.87:0/1628355587 >> v2:172.21.15.169:3300/0 conn(0x7f6458070350 0x7f645815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 -- 172.21.15.87:0/1628355587 >> 172.21.15.87:0/1628355587 conn(0x7f64580669a0 msgr2=0x7f6458066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 -- 172.21.15.87:0/1628355587 shutdown_connections 2024-04-25T10:15:43.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 -- 172.21.15.87:0/1628355587 wait complete. 2024-04-25T10:15:43.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 Processor -- start 2024-04-25T10:15:43.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.039+0000 7f645e200640 1 -- start start 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6458070350 0x7f6458155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64580aef80 0x7f6458147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64580b31c0 0x7f6458157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645806c250 con 0x7f6458070350 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645806c0d0 con 0x7f64580aef80 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645806c3d0 con 0x7f64580b31c0 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6458070350 0x7f6458155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6458070350 0x7f6458155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58092/0 (socket says 172.21.15.87:58092) 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 -- 172.21.15.87:0/1447361469 learned_addr learned my addr 172.21.15.87:0/1447361469 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457e00640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64580b31c0 0x7f6458157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6456a00640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64580aef80 0x7f6458147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 -- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64580b31c0 msgr2=0x7f6458157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64580b31c0 0x7f6458157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 -- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64580aef80 msgr2=0x7f6458147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64580aef80 0x7f6458147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 -- 172.21.15.87:0/1447361469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64581f6980 con 0x7f6458070350 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6456a00640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64580aef80 0x7f6458147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f6457400640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6458070350 0x7f6458155dc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f644000c480 tx=0x7f644000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f644f400640 1 -- 172.21.15.87:0/1447361469 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440017070 con 0x7f6458070350 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f644f400640 1 -- 172.21.15.87:0/1447361469 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6440005cf0 con 0x7f6458070350 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f644f400640 1 -- 172.21.15.87:0/1447361469 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440007290 con 0x7f6458070350 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64581f6bf0 con 0x7f6458070350 2024-04-25T10:15:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.043+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64581f7170 con 0x7f6458070350 2024-04-25T10:15:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f644f400640 1 -- 172.21.15.87:0/1447361469 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f644001a070 con 0x7f6458070350 2024-04-25T10:15:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f644f400640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6434066f60 0x7f6434069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f644f400640 1 -- 172.21.15.87:0/1447361469 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64400844a0 con 0x7f6458070350 2024-04-25T10:15:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6456a00640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6434066f60 0x7f6434069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 Processor -- start 2024-04-25T10:15:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 -- start start 2024-04-25T10:15:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6456a00640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6434066f60 0x7f6434069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64580cc000 tx=0x7f6448008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b49d0 0x5602689b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x56026896ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x56026896b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602687b7ff0 con 0x5602689b49d0 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560268740150 con 0x56026896b0a0 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602687b4090 con 0x5602689b52f0 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b49d0 0x5602689b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b49d0 0x5602689b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58108/0 (socket says 172.21.15.87:58108) 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 -- 172.21.15.87:0/1661639663 learned_addr learned my addr 172.21.15.87:0/1661639663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6456a00640 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x56026896ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457e00640 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x56026896b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 -- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 msgr2=0x56026896ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x56026896ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 -- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 msgr2=0x56026896b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x56026896b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 -- 172.21.15.87:0/1661639663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56026894e0d0 con 0x5602689b49d0 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6457400640 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b49d0 0x5602689b4db0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6440051200 tx=0x7f6440000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436f5bfefa45607b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6447e00640 1 -- 172.21.15.87:0/1661639663 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440058070 con 0x5602689b49d0 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6447e00640 1 -- 172.21.15.87:0/1661639663 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64400530a0 con 0x5602689b49d0 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.047+0000 7f6447e00640 1 -- 172.21.15.87:0/1661639663 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440051410 con 0x5602689b49d0 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- 172.21.15.87:0/1661639663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b49d0 msgr2=0x5602689b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b49d0 0x5602689b4db0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6440051200 tx=0x7f6440000bd0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- 172.21.15.87:0/1661639663 shutdown_connections 2024-04-25T10:15:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x56026896b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x56026896ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 --2- 172.21.15.87:0/1661639663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b49d0 0x5602689b4db0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- 172.21.15.87:0/1661639663 >> 172.21.15.87:0/1661639663 conn(0x560268967020 msgr2=0x560268969440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- 172.21.15.87:0/1661639663 shutdown_connections 2024-04-25T10:15:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- 172.21.15.87:0/1661639663 wait complete. 2024-04-25T10:15:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- start start 2024-04-25T10:15:43.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 Processor -- start 2024-04-25T10:15:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x5602689997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b49d0 0x560268999d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026899a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026877c250 con 0x5602689b52f0 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560268772690 con 0x56026896b0a0 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560268740a40 con 0x5602689b49d0 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 10 client.0 _ll_register_callbacks cb 0x560268955e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6457e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026899a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6457400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x5602689997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6456a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b49d0 0x560268999d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6457e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026899a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58112/0 (socket says 172.21.15.87:58112) 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6457400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x5602689997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:59040/0 (socket says 172.21.15.87:59040) 2024-04-25T10:15:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6457400640 1 -- 172.21.15.87:0/857983074 learned_addr learned my addr 172.21.15.87:0/857983074 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6456a00640 1 -- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 msgr2=0x5602689997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6456a00640 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x5602689997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6456a00640 1 -- 172.21.15.87:0/857983074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 msgr2=0x56026899a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6456a00640 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026899a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6456a00640 1 -- 172.21.15.87:0/857983074 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602688651c0 con 0x5602689b49d0 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6457e00640 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026899a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6457400640 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x5602689997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6456a00640 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b49d0 0x560268999d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6448009fd0 tx=0x7f64480046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb7388c6ee79367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6444c00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2024-04-25T10:15:43.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6444c00640 1 -- 172.21.15.87:0/857983074 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448026070 con 0x5602689b49d0 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6444c00640 1 -- 172.21.15.87:0/857983074 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6448016d80 con 0x5602689b49d0 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64400530a0 con 0x5602689b49d0 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6444c00640 1 -- 172.21.15.87:0/857983074 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6448031050 con 0x5602689b49d0 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 10 client.5507 fetch_fsmap learned FSMap version 27 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.051+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x560268861170 con 0x5602689b49d0 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6444c00640 1 -- 172.21.15.87:0/857983074 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64400530a0 con 0x5602689b49d0 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6444c00640 1 -- 172.21.15.87:0/857983074 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6448034020 con 0x5602689b49d0 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 10 client.5507 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 10 client.5507 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 20 client.5507 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:43.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c4c80 0x5602689c7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 4 client.5507 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560268753c60 con 0x5602689c4c80 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6457400640 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c4c80 0x5602689c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6457400640 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c4c80 0x5602689c7070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2365ce4c53ff2d93 server_cookie=159eb755f7b258b0 in_seq=0 out_seq=0 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6444c00640 10 client.5507 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6444c00640 1 -- 172.21.15.87:0/857983074 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560268753c60 con 0x5602689c4c80 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6444c00640 10 client.5507 handle_command_reply: tid=0 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 client.5507 shutdown 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b49d0 msgr2=0x560268999d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b49d0 0x560268999d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6448009fd0 tx=0x7f64480046d0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 shutdown_connections 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026899a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c4c80 0x5602689c7070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b49d0 0x560268999d30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 --2- 172.21.15.87:0/857983074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56026896b0a0 0x5602689997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 >> 172.21.15.87:0/857983074 conn(0x560268967020 msgr2=0x560268969ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6444c00640 0 client.5507 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 shutdown_connections 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 1 -- 172.21.15.87:0/857983074 wait complete. 2024-04-25T10:15:43.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.055+0000 7f6461655000 20 client.5507 trim_cache size 0 max 16384 2024-04-25T10:15:43.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 Processor -- start 2024-04-25T10:15:43.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 -- start start 2024-04-25T10:15:43.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56026896b0a0 0x5602689b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x56026896a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026896a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458202fc0 con 0x5602689b52f0 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458202360 con 0x5602689b49d0 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64581f82b0 con 0x56026896b0a0 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026896a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026896a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58124/0 (socket says 172.21.15.87:58124) 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 -- 172.21.15.87:0/2762685350 learned_addr learned my addr 172.21.15.87:0/2762685350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6456a00640 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x56026896a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457400640 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56026896b0a0 0x5602689b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 -- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56026896b0a0 msgr2=0x5602689b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56026896b0a0 0x5602689b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 -- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 msgr2=0x56026896a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x56026896a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 -- 172.21.15.87:0/2762685350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602687fede0 con 0x5602689b52f0 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6456a00640 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x56026896a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6457e00640 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026896a960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f645001c970 tx=0x7f6450004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf8bc433e3e08e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6447400640 1 -- 172.21.15.87:0/2762685350 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500b2040 con 0x5602689b52f0 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6447400640 1 -- 172.21.15.87:0/2762685350 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64500a5b50 con 0x5602689b52f0 2024-04-25T10:15:43.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6447400640 1 -- 172.21.15.87:0/2762685350 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500ad440 con 0x5602689b52f0 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 -- 172.21.15.87:0/2762685350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 msgr2=0x56026896a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026896a960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f645001c970 tx=0x7f6450004ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 -- 172.21.15.87:0/2762685350 shutdown_connections 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602689b52f0 0x56026896a960 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x56026896a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 --2- 172.21.15.87:0/2762685350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56026896b0a0 0x5602689b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.071+0000 7f6461655000 1 -- 172.21.15.87:0/2762685350 >> 172.21.15.87:0/2762685350 conn(0x560268967020 msgr2=0x5602689b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- 172.21.15.87:0/2762685350 shutdown_connections 2024-04-25T10:15:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- 172.21.15.87:0/2762685350 wait complete. 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- start start 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 Processor -- start 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56026896b0a0 0x560268a15520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x560268a15a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x5602689c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560268752e00 con 0x56026896b0a0 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64581f88b0 con 0x5602689b49d0 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602687b7300 con 0x5602689b52f0 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 10 client.0 _ll_register_callbacks cb 0x560268936040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56026896b0a0 0x560268a15520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56026896b0a0 0x560268a15520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58128/0 (socket says 172.21.15.87:58128) 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 -- 172.21.15.87:0/3219158519 learned_addr learned my addr 172.21.15.87:0/3219158519 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457e00640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x5602689c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6456a00640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x560268a15a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 -- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 msgr2=0x5602689c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x5602689c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 -- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 msgr2=0x560268a15a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x560268a15a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 -- 172.21.15.87:0/3219158519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602687598e0 con 0x56026896b0a0 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457e00640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x5602689c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6456a00640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x560268a15a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6457400640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56026896b0a0 0x560268a15520 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f644000c450 tx=0x7f644001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c4e88e4fec8a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6444c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6444c00640 1 -- 172.21.15.87:0/3219158519 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440058040 con 0x56026896b0a0 2024-04-25T10:15:43.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6444c00640 1 -- 172.21.15.87:0/3219158519 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6440084070 con 0x56026896b0a0 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64500a5b50 con 0x56026896b0a0 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6444c00640 1 -- 172.21.15.87:0/3219158519 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6440084270 con 0x56026896b0a0 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 10 client.6252 fetch_fsmap learned FSMap version 27 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5602688118f0 con 0x56026896b0a0 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6444c00640 1 -- 172.21.15.87:0/3219158519 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64500a5b50 con 0x56026896b0a0 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6444c00640 1 -- 172.21.15.87:0/3219158519 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f644005d020 con 0x56026896b0a0 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 10 client.6252 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 10 client.6252 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 20 client.6252 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c7d70 0x5602689c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 4 client.6252 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5602687c5530 con 0x5602689c7d70 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.075+0000 7f6456a00640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c7d70 0x5602689c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6456a00640 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c7d70 0x5602689c8150 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449e38064c6890b7 server_cookie=64518ce4baad56b2 in_seq=0 out_seq=0 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6444c00640 10 client.6252 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6444c00640 1 -- 172.21.15.87:0/3219158519 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5602687c5530 con 0x5602689c7d70 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6444c00640 10 client.6252 handle_command_reply: tid=0 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 client.6252 shutdown 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56026896b0a0 msgr2=0x560268a15520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56026896b0a0 0x560268a15520 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f644000c450 tx=0x7f644001e040 comp rx=0 tx=0).stop 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 shutdown_connections 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x5602689c7d70 0x5602689c8150 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5602689b52f0 0x5602689c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5602689b49d0 0x560268a15a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 --2- 172.21.15.87:0/3219158519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56026896b0a0 0x560268a15520 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 >> 172.21.15.87:0/3219158519 conn(0x560268967020 msgr2=0x5602689b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6444c00640 0 client.6252 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 shutdown_connections 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 1 -- 172.21.15.87:0/3219158519 wait complete. 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f6461655000 20 client.6252 trim_cache size 0 max 16384 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:15:43.091 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:15:43.092 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:43.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6434066f60 msgr2=0x7f6434069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6434066f60 0x7f6434069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64580cc000 tx=0x7f6448008040 comp rx=0 tx=0).stop 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6458070350 msgr2=0x7f6458155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6458070350 0x7f6458155dc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f644000c480 tx=0x7f644000c940 comp rx=0 tx=0).stop 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 shutdown_connections 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6434066f60 0x7f6434069420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64580b31c0 0x7f6458157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64580aef80 0x7f6458147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 --2- 172.21.15.87:0/1447361469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6458070350 0x7f6458155dc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 >> 172.21.15.87:0/1447361469 conn(0x7f64580669a0 msgr2=0x7f6458066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 shutdown_connections 2024-04-25T10:15:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.079+0000 7f645e200640 1 -- 172.21.15.87:0/1447361469 wait complete. 2024-04-25T10:15:43.104 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:15:43.104 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:43.104 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:15:43.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.235+0000 7f6a81400640 1 Processor -- start 2024-04-25T10:15:43.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.235+0000 7f6a81400640 1 -- start start 2024-04-25T10:15:43.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a81400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 0x7f6a7c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a81400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6a7c0721f0 0x7f6a7c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a81400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c159920 con 0x7f6a7c0aebd0 2024-04-25T10:15:43.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a81400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0af030 con 0x7f6a7c0721f0 2024-04-25T10:15:43.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a81400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15c2f0 con 0x7f6a7c159540 2024-04-25T10:15:43.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6a7c0aebd0 0x7f6a7c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47738/0 (socket says 172.21.15.87:47738) 2024-04-25T10:15:43.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79e00640 1 -- 172.21.15.87:0/2259955584 learned_addr learned my addr 172.21.15.87:0/2259955584 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:43.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7b200640 1 --2- 172.21.15.87:0/2259955584 >> v2:172.21.15.169:3300/0 conn(0x7f6a7c0721f0 0x7f6a7c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7a800640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 0x7f6a7c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228679103 0 0) 0x7f6a7c159920 con 0x7f6a7c0aebd0 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a60003530 con 0x7f6a7c0aebd0 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913436098 0 0) 0x7f6a60003530 con 0x7f6a7c0aebd0 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 msgr2=0x7f6a7c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 0x7f6a7c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 >> v2:172.21.15.169:3300/0 conn(0x7f6a7c0721f0 msgr2=0x7f6a7c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 --2- 172.21.15.87:0/2259955584 >> v2:172.21.15.169:3300/0 conn(0x7f6a7c0721f0 0x7f6a7c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c15c5d0 con 0x7f6a7c0aebd0 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f6a68001680 con 0x7f6a7c0aebd0 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 >> v1:172.21.15.87:6789/0 conn(0x7f6a7c0aebd0 legacy=0x7f6a7c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a600039c0 0x7f6a60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a60004810 0x7f6a60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 0x7f6a600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15c2f0 con 0x7f6a600039c0 2024-04-25T10:15:43.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6000b9d0 con 0x7f6a60004810 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6000bb50 con 0x7f6a7c159540 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7a800640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a600039c0 0x7f6a60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a79e00640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a60004810 0x7f6a60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7b200640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 0x7f6a600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7a800640 1 -- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 msgr2=0x7f6a600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7a800640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 0x7f6a600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7a800640 1 -- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a60004810 msgr2=0x7f6a60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7a800640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a60004810 0x7f6a60004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7a800640 1 -- 172.21.15.87:0/2259955584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a60003530 con 0x7f6a600039c0 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.239+0000 7f6a7b200640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c159540 0x7f6a600051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a7a800640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a600039c0 0x7f6a60003de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009fc0 tx=0x7f6a6c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7727234a78611580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0b2070 con 0x7f6a600039c0 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c0a5a70 con 0x7f6a600039c0 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a79400640 1 -- 172.21.15.87:0/2259955584 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c009c10 con 0x7f6a600039c0 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 -- 172.21.15.87:0/2259955584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a600039c0 msgr2=0x7f6a60003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 --2- 172.21.15.87:0/2259955584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a600039c0 0x7f6a60003de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009fc0 tx=0x7f6a6c01cbf0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a7b200640 1 -- 172.21.15.87:0/2259955584 reap_dead start 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 -- 172.21.15.87:0/2259955584 shutdown_connections 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 -- 172.21.15.87:0/2259955584 >> 172.21.15.87:0/2259955584 conn(0x7f6a7c0669a0 msgr2=0x7f6a7c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 -- 172.21.15.87:0/2259955584 shutdown_connections 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 -- 172.21.15.87:0/2259955584 wait complete. 2024-04-25T10:15:43.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 Processor -- start 2024-04-25T10:15:43.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.243+0000 7f6a81400640 1 -- start start 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a600039c0 0x7f6a7c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a7c154660 0x7f6a7c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c157060 0x7f6a7c1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0710a0 con 0x7f6a7c154660 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c06c0d0 con 0x7f6a600039c0 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c071220 con 0x7f6a7c157060 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a7c154660 0x7f6a7c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a7c154660 0x7f6a7c06ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58154/0 (socket says 172.21.15.87:58154) 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 -- 172.21.15.87:0/3567297929 learned_addr learned my addr 172.21.15.87:0/3567297929 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a7a800640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a600039c0 0x7f6a7c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a7b200640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c157060 0x7f6a7c1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 -- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c157060 msgr2=0x7f6a7c1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c157060 0x7f6a7c1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 -- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a600039c0 msgr2=0x7f6a7c148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a600039c0 0x7f6a7c148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 -- 172.21.15.87:0/3567297929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c156170 con 0x7f6a7c154660 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a79e00640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a7c154660 0x7f6a7c06ca10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6a68007310 tx=0x7f6a680077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a72a00640 1 -- 172.21.15.87:0/3567297929 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68007e60 con 0x7f6a7c154660 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a72a00640 1 -- 172.21.15.87:0/3567297929 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a68018070 con 0x7f6a7c154660 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c156450 con 0x7f6a7c154660 2024-04-25T10:15:43.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a72a00640 1 -- 172.21.15.87:0/3567297929 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6800fba0 con 0x7f6a7c154660 2024-04-25T10:15:43.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c0b02b0 con 0x7f6a7c154660 2024-04-25T10:15:43.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.247+0000 7f6a72a00640 1 -- 172.21.15.87:0/3567297929 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a6800fd40 con 0x7f6a7c154660 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a72a00640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a5c066f90 0x7f6a5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a72a00640 1 -- 172.21.15.87:0/3567297929 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a68085060 con 0x7f6a7c154660 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7a800640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a5c066f90 0x7f6a5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 Processor -- start 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- start start 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7a800640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a5c066f90 0x7f6a5c069450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0af000 tx=0x7f6a6c004ac0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07180 0x55c699b07560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07aa0 0x55c699abd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699abd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69990af20 con 0x55c699abd600 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c699893000 con 0x55c699b07aa0 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c699906fc0 con 0x55c699b07180 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699abd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699abd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58166/0 (socket says 172.21.15.87:58166) 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 -- 172.21.15.87:0/3531877560 learned_addr learned my addr 172.21.15.87:0/3531877560 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7a800640 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07180 0x55c699b07560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a79e00640 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07aa0 0x55c699abd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 -- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07180 msgr2=0x55c699b07560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07180 0x55c699b07560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 -- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07aa0 msgr2=0x55c699abd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07aa0 0x55c699abd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 -- 172.21.15.87:0/3531877560 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6998e6ba0 con 0x55c699abd600 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a79e00640 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07aa0 0x55c699abd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a7b200640 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699abd9e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c070110 tx=0x7f6a74009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e48d81bc7a58f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a67400640 1 -- 172.21.15.87:0/3531877560 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7401a030 con 0x55c699abd600 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a67400640 1 -- 172.21.15.87:0/3531877560 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a74013810 con 0x55c699abd600 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a67400640 1 -- 172.21.15.87:0/3531877560 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74013be0 con 0x55c699abd600 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- 172.21.15.87:0/3531877560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 msgr2=0x55c699abd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699abd9e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c070110 tx=0x7f6a74009190 comp rx=0 tx=0).stop 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- 172.21.15.87:0/3531877560 shutdown_connections 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699abd9e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07aa0 0x55c699abd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 --2- 172.21.15.87:0/3531877560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07180 0x55c699b07560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- 172.21.15.87:0/3531877560 >> 172.21.15.87:0/3531877560 conn(0x55c699ac3760 msgr2=0x55c699abb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- 172.21.15.87:0/3531877560 shutdown_connections 2024-04-25T10:15:43.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.251+0000 7f6a847d0000 1 -- 172.21.15.87:0/3531877560 wait complete. 2024-04-25T10:15:43.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 -- start start 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 Processor -- start 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699ac26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699ac2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699af5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6999c89c0 con 0x55c699b07aa0 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6998c55d0 con 0x55c699b07180 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6998c7080 con 0x55c699abd600 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 10 client.0 _ll_register_callbacks cb 0x55c699aa8a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699af5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699af5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58174/0 (socket says 172.21.15.87:58174) 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 -- 172.21.15.87:0/1696216334 learned_addr learned my addr 172.21.15.87:0/1696216334 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a79e00640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699ac2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7a800640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699ac26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 -- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 msgr2=0x55c699ac26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699ac26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 -- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 msgr2=0x55c699ac2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699ac2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 -- 172.21.15.87:0/1696216334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6999515a0 con 0x55c699b07aa0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7a800640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699ac26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7b200640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699af5450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cd200 tx=0x7f6a740046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb42cdc8509fe2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a5be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a5be00640 1 -- 172.21.15.87:0/1696216334 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74005ba0 con 0x55c699b07aa0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a5be00640 1 -- 172.21.15.87:0/1696216334 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a74005d40 con 0x55c699b07aa0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a74013810 con 0x55c699b07aa0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a5be00640 1 -- 172.21.15.87:0/1696216334 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a74031050 con 0x55c699b07aa0 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 10 client.6264 fetch_fsmap learned FSMap version 27 2024-04-25T10:15:43.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c6998e22b0 con 0x55c699b07aa0 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a5be00640 1 -- 172.21.15.87:0/1696216334 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a74013810 con 0x55c699b07aa0 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a5be00640 1 -- 172.21.15.87:0/1696216334 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a74034020 con 0x55c699b07aa0 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 10 client.6264 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 10 client.6264 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 20 client.6264 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699af4e60 0x55c699af5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 4 client.6264 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6998a6c70 con 0x55c699af4e60 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.255+0000 7f6a7a800640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699af4e60 0x55c699af5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a7a800640 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699af4e60 0x55c699af5e60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4881758f34ca4cf server_cookie=f8b9c18473bc8f11 in_seq=0 out_seq=0 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a5be00640 10 client.6264 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a5be00640 1 -- 172.21.15.87:0/1696216334 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6998a6c70 con 0x55c699af4e60 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a5be00640 10 client.6264 handle_command_reply: tid=0 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 client.6264 shutdown 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 msgr2=0x55c699af5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699af5450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cd200 tx=0x7f6a740046d0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 shutdown_connections 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699af5450 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699ac2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699af4e60 0x55c699af5e60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 --2- 172.21.15.87:0/1696216334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699ac26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a5be00640 0 client.6264 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 >> 172.21.15.87:0/1696216334 conn(0x55c699ac3760 msgr2=0x55c699abc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 shutdown_connections 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 1 -- 172.21.15.87:0/1696216334 wait complete. 2024-04-25T10:15:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.259+0000 7f6a847d0000 20 client.6264 trim_cache size 0 max 16384 2024-04-25T10:15:43.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 Processor -- start 2024-04-25T10:15:43.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 -- start start 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699b065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b06b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07aa0 0x55c699abc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c2034a0 con 0x55c699abd600 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c2026d0 con 0x55c699b07180 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b1130 con 0x55c699b07aa0 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699b065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699b065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58190/0 (socket says 172.21.15.87:58190) 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 -- 172.21.15.87:0/66391962 learned_addr learned my addr 172.21.15.87:0/66391962 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a79e00640 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b06b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7b200640 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07aa0 0x55c699abc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 -- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07aa0 msgr2=0x55c699abc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07aa0 0x55c699abc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 -- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 msgr2=0x55c699b06b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b06b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 -- 172.21.15.87:0/66391962 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c699893250 con 0x55c699abd600 2024-04-25T10:15:43.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a7a800640 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699b065f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c01ca20 tx=0x7f6a6c01b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7498d14deb26c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a66a00640 1 -- 172.21.15.87:0/66391962 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0b2040 con 0x55c699abd600 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a66a00640 1 -- 172.21.15.87:0/66391962 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c0bb040 con 0x55c699abd600 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.271+0000 7f6a66a00640 1 -- 172.21.15.87:0/66391962 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0c2ad0 con 0x55c699abd600 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- 172.21.15.87:0/66391962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 msgr2=0x55c699b065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699b065f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c01ca20 tx=0x7f6a6c01b490 comp rx=0 tx=0).stop 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- 172.21.15.87:0/66391962 shutdown_connections 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699b07aa0 0x55c699abc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b06b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 --2- 172.21.15.87:0/66391962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699abd600 0x55c699b065f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- 172.21.15.87:0/66391962 >> 172.21.15.87:0/66391962 conn(0x55c699ac3760 msgr2=0x55c699b05ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- 172.21.15.87:0/66391962 shutdown_connections 2024-04-25T10:15:43.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- 172.21.15.87:0/66391962 wait complete. 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- start start 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 Processor -- start 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699b6f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b6fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699b1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69990a230 con 0x55c699b07aa0 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b1700 con 0x55c699b07180 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6998c3e80 con 0x55c699abd600 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 10 client.0 _ll_register_callbacks cb 0x55c699a89600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699b1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699b1b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58194/0 (socket says 172.21.15.87:58194) 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699b6f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 -- 172.21.15.87:0/2180342764 learned_addr learned my addr 172.21.15.87:0/2180342764 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a79e00640 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b6fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 -- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 msgr2=0x55c699b6f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699b6f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 -- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 msgr2=0x55c699b6fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b6fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 -- 172.21.15.87:0/2180342764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c69989fb90 con 0x55c699b07aa0 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a7b200640 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699b1b760 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6a74013b40 tx=0x7f6a74008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93c90f6a9b369a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a5be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a5be00640 1 -- 172.21.15.87:0/2180342764 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74039070 con 0x55c699b07aa0 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a5be00640 1 -- 172.21.15.87:0/2180342764 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a74005fd0 con 0x55c699b07aa0 2024-04-25T10:15:43.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a6c0bb040 con 0x55c699b07aa0 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a5be00640 1 -- 172.21.15.87:0/2180342764 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a74044050 con 0x55c699b07aa0 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 10 client.6270 fetch_fsmap learned FSMap version 27 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.275+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c6998b2010 con 0x55c699b07aa0 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a5be00640 1 -- 172.21.15.87:0/2180342764 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c0bb040 con 0x55c699b07aa0 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a5be00640 1 -- 172.21.15.87:0/2180342764 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a74047020 con 0x55c699b07aa0 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 10 client.6270 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 10 client.6270 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 20 client.6270 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699b1c0f0 0x55c699b1c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 4 client.6270 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c699af57e0 con 0x55c699b1c0f0 2024-04-25T10:15:43.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a7a800640 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699b1c0f0 0x55c699b1c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a5be00640 10 client.6270 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a7a800640 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699b1c0f0 0x55c699b1c500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a46b6bb47f3214e server_cookie=391400933c1462fc in_seq=0 out_seq=0 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a5be00640 1 -- 172.21.15.87:0/2180342764 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55c699af57e0 con 0x55c699b1c0f0 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a5be00640 10 client.6270 handle_command_reply: tid=0 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 client.6270 shutdown 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 msgr2=0x55c699b1b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699b1b760 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6a74013b40 tx=0x7f6a74008040 comp rx=0 tx=0).stop 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 shutdown_connections 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c699b07aa0 0x55c699b1b760 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c699b07180 0x55c699b6fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c699b1c0f0 0x55c699b1c500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 --2- 172.21.15.87:0/2180342764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c699abd600 0x55c699b6f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 >> 172.21.15.87:0/2180342764 conn(0x55c699ac3760 msgr2=0x55c699b06770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a5be00640 0 client.6270 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 shutdown_connections 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 1 -- 172.21.15.87:0/2180342764 wait complete. 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a847d0000 20 client.6270 trim_cache size 0 max 16384 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 3119778829, 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511630189, 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:43.290 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:15:43.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a5c066f90 msgr2=0x7f6a5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a5c066f90 0x7f6a5c069450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0af000 tx=0x7f6a6c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a7c154660 msgr2=0x7f6a7c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a7c154660 0x7f6a7c06ca10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6a68007310 tx=0x7f6a680077d0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 shutdown_connections 2024-04-25T10:15:43.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a5c066f90 0x7f6a5c069450 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a7c157060 0x7f6a7c1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a7c154660 0x7f6a7c06ca10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 --2- 172.21.15.87:0/3567297929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a600039c0 0x7f6a7c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 >> 172.21.15.87:0/3567297929 conn(0x7f6a7c0669a0 msgr2=0x7f6a7c15e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 shutdown_connections 2024-04-25T10:15:43.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.279+0000 7f6a81400640 1 -- 172.21.15.87:0/3567297929 wait complete. 2024-04-25T10:15:43.305 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:43.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 Processor -- start 2024-04-25T10:15:43.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 -- start start 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a40d1820 0x7f08a40d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08a415e050 0x7f08a40b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08a40b13e0 0x7f08a40ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a40b1870 con 0x7f08a415e050 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a40b1af0 con 0x7f08a40b13e0 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40b1dd0 con 0x7f08a40d1820 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08a415e050 0x7f08a40b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08a415e050 0x7f08a40b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58206/0 (socket says 172.21.15.87:58206) 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 -- 172.21.15.87:0/3647978967 learned_addr learned my addr 172.21.15.87:0/3647978967 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a2800640 1 --2- 172.21.15.87:0/3647978967 >> v2:172.21.15.169:3300/0 conn(0x7f08a40b13e0 0x7f08a40ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1e00640 1 --2- 172.21.15.87:0/3647978967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a40d1820 0x7f08a40d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 -- 172.21.15.87:0/3647978967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a40d1820 msgr2=0x7f08a40d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 --2- 172.21.15.87:0/3647978967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a40d1820 0x7f08a40d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 -- 172.21.15.87:0/3647978967 >> v2:172.21.15.169:3300/0 conn(0x7f08a40b13e0 msgr2=0x7f08a40ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 --2- 172.21.15.87:0/3647978967 >> v2:172.21.15.169:3300/0 conn(0x7f08a40b13e0 0x7f08a40ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1400640 1 -- 172.21.15.87:0/3647978967 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40b20b0 con 0x7f08a415e050 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a2800640 1 --2- 172.21.15.87:0/3647978967 >> v2:172.21.15.169:3300/0 conn(0x7f08a40b13e0 0x7f08a40ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.435+0000 7f08a1e00640 1 --2- 172.21.15.87:0/3647978967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a40d1820 0x7f08a40d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:43.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a1400640 1 --2- 172.21.15.87:0/3647978967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a415e050 0x7f08a40b0ea0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f08a406ce10 tx=0x7f089001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e91c2212c7a05aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a0a00640 1 -- 172.21.15.87:0/3647978967 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900b0070 con 0x7f08a415e050 2024-04-25T10:15:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a0a00640 1 -- 172.21.15.87:0/3647978967 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08900ab600 con 0x7f08a415e050 2024-04-25T10:15:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a0a00640 1 -- 172.21.15.87:0/3647978967 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900ab9d0 con 0x7f08a415e050 2024-04-25T10:15:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 -- 172.21.15.87:0/3647978967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a415e050 msgr2=0x7f08a40b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 --2- 172.21.15.87:0/3647978967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a415e050 0x7f08a40b0ea0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f08a406ce10 tx=0x7f089001c260 comp rx=0 tx=0).stop 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 -- 172.21.15.87:0/3647978967 shutdown_connections 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 --2- 172.21.15.87:0/3647978967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a415e050 0x7f08a40b0ea0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 --2- 172.21.15.87:0/3647978967 >> v2:172.21.15.169:3300/0 conn(0x7f08a40b13e0 0x7f08a40ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 --2- 172.21.15.87:0/3647978967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a40d1820 0x7f08a40d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 -- 172.21.15.87:0/3647978967 >> 172.21.15.87:0/3647978967 conn(0x7f08a409a9d0 msgr2=0x7f08a409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 -- 172.21.15.87:0/3647978967 shutdown_connections 2024-04-25T10:15:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.439+0000 7f08a9000640 1 -- 172.21.15.87:0/3647978967 wait complete. 2024-04-25T10:15:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 Processor -- start 2024-04-25T10:15:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 -- start start 2024-04-25T10:15:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08a40b13e0 0x7f08a40cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d1820 0x7f08a4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a415e050 0x7f08a40c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cfdf0 con 0x7f08a40d1820 2024-04-25T10:15:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cfc70 con 0x7f08a40b13e0 2024-04-25T10:15:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40cff70 con 0x7f08a415e050 2024-04-25T10:15:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d1820 0x7f08a4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d1820 0x7f08a4154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58222/0 (socket says 172.21.15.87:58222) 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 -- 172.21.15.87:0/1499626917 learned_addr learned my addr 172.21.15.87:0/1499626917 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1e00640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08a40b13e0 0x7f08a40cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a2800640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a415e050 0x7f08a40c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 -- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a415e050 msgr2=0x7f08a40c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a415e050 0x7f08a40c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 -- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08a40b13e0 msgr2=0x7f08a40cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08a40b13e0 0x7f08a40cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 -- 172.21.15.87:0/1499626917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4154680 con 0x7f08a40d1820 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a2800640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a415e050 0x7f08a40c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a1400640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d1820 0x7f08a4154140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f089001a910 tx=0x7f0890019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f089a000640 1 -- 172.21.15.87:0/1499626917 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900b9070 con 0x7f08a40d1820 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f089a000640 1 -- 172.21.15.87:0/1499626917 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08900b4600 con 0x7f08a40d1820 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f089a000640 1 -- 172.21.15.87:0/1499626917 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900b0040 con 0x7f08a40d1820 2024-04-25T10:15:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a40cc940 con 0x7f08a40d1820 2024-04-25T10:15:43.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.443+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a40ccf00 con 0x7f08a40d1820 2024-04-25T10:15:43.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.447+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a40cd9c0 con 0x7f08a40d1820 2024-04-25T10:15:43.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.447+0000 7f089a000640 1 -- 172.21.15.87:0/1499626917 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08900bc070 con 0x7f08a40d1820 2024-04-25T10:15:43.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.447+0000 7f089a000640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0884066f60 0x7f0884069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.447+0000 7f089a000640 1 -- 172.21.15.87:0/1499626917 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f089011df70 con 0x7f08a40d1820 2024-04-25T10:15:43.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.447+0000 7f08a1e00640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0884066f60 0x7f0884069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.447+0000 7f08a1e00640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0884066f60 0x7f0884069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f089400d540 tx=0x7f0894007810 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.451+0000 7f089a000640 1 -- 172.21.15.87:0/1499626917 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0890016110 con 0x7f08a40d1820 2024-04-25T10:15:43.537 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:15:43.531+0000 7ff597400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:15:43.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.639+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a404ef30 con 0x7f08a40d1820 2024-04-25T10:15:43.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f089a000640 1 -- 172.21.15.87:0/1499626917 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f08900ee9b0 con 0x7f08a40d1820 2024-04-25T10:15:43.648 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:43.648 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:43.370279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:43.648 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 28 2024-04-25T10:15:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0884066f60 msgr2=0x7f0884069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0884066f60 0x7f0884069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f089400d540 tx=0x7f0894007810 comp rx=0 tx=0).stop 2024-04-25T10:15:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d1820 msgr2=0x7f08a4154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d1820 0x7f08a4154140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f089001a910 tx=0x7f0890019000 comp rx=0 tx=0).stop 2024-04-25T10:15:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 shutdown_connections 2024-04-25T10:15:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0884066f60 0x7f0884069420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08a415e050 0x7f08a40c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d1820 0x7f08a4154140 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 --2- 172.21.15.87:0/1499626917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08a40b13e0 0x7f08a40cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 >> 172.21.15.87:0/1499626917 conn(0x7f08a409a9d0 msgr2=0x7f08a406f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 shutdown_connections 2024-04-25T10:15:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.643+0000 7f08a9000640 1 -- 172.21.15.87:0/1499626917 wait complete. 2024-04-25T10:15:43.669 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 3119778829 2024-04-25T10:15:43.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.799+0000 7f3a23200640 1 Processor -- start 2024-04-25T10:15:43.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.799+0000 7f3a23200640 1 -- start start 2024-04-25T10:15:43.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a23200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3a240cdb60 0x7f3a2406aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a23200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a240cbdf0 0x7f3a2406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a23200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a240cdf40 con 0x7f3a240cec70 2024-04-25T10:15:43.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a23200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2406b3e0 con 0x7f3a240cdb60 2024-04-25T10:15:43.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a23200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a240b1230 con 0x7f3a240cbdf0 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a22800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3a240cec70 0x7f3a240b22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47742/0 (socket says 172.21.15.87:47742) 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a22800640 1 -- 172.21.15.87:0/3348274130 learned_addr learned my addr 172.21.15.87:0/3348274130 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3348274130 >> v2:172.21.15.87:3300/0 conn(0x7f3a240cdb60 0x7f3a2406aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a21400640 1 --2- 172.21.15.87:0/3348274130 >> v2:172.21.15.169:3300/0 conn(0x7f3a240cbdf0 0x7f3a2406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a21e00640 1 -- 172.21.15.87:0/3348274130 >> v2:172.21.15.169:3300/0 conn(0x7f3a240cbdf0 msgr2=0x7f3a2406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3348274130 >> v2:172.21.15.169:3300/0 conn(0x7f3a240cbdf0 0x7f3a2406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a21e00640 1 -- 172.21.15.87:0/3348274130 >> v1:172.21.15.87:6789/0 conn(0x7f3a240cec70 legacy=0x7f3a240b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.803+0000 7f3a21e00640 1 -- 172.21.15.87:0/3348274130 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a240b1510 con 0x7f3a240cdb60 2024-04-25T10:15:43.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3348274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cdb60 0x7f3a2406aea0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3a24071c30 tx=0x7f3a1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ccad1789bf747e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a20a00640 1 -- 172.21.15.87:0/3348274130 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140b2070 con 0x7f3a240cdb60 2024-04-25T10:15:43.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a20a00640 1 -- 172.21.15.87:0/3348274130 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a140a9070 con 0x7f3a240cdb60 2024-04-25T10:15:43.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a20a00640 1 -- 172.21.15.87:0/3348274130 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140ad880 con 0x7f3a240cdb60 2024-04-25T10:15:43.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- 172.21.15.87:0/3348274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cdb60 msgr2=0x7f3a2406aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 --2- 172.21.15.87:0/3348274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cdb60 0x7f3a2406aea0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3a24071c30 tx=0x7f3a1401c450 comp rx=0 tx=0).stop 2024-04-25T10:15:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- 172.21.15.87:0/3348274130 shutdown_connections 2024-04-25T10:15:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 --2- 172.21.15.87:0/3348274130 >> v2:172.21.15.169:3300/0 conn(0x7f3a240cbdf0 0x7f3a2406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 --2- 172.21.15.87:0/3348274130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cdb60 0x7f3a2406aea0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- 172.21.15.87:0/3348274130 >> 172.21.15.87:0/3348274130 conn(0x7f3a240d17a0 msgr2=0x7f3a240d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- 172.21.15.87:0/3348274130 shutdown_connections 2024-04-25T10:15:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- 172.21.15.87:0/3348274130 wait complete. 2024-04-25T10:15:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 Processor -- start 2024-04-25T10:15:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- start start 2024-04-25T10:15:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cbdf0 0x7f3a2414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a240cdb60 0x7f3a240c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a240cec70 0x7f3a24159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2415d590 con 0x7f3a240cbdf0 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2415d410 con 0x7f3a240cec70 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a23200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2415d710 con 0x7f3a240cdb60 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cbdf0 0x7f3a2414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cbdf0 0x7f3a2414d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58242/0 (socket says 172.21.15.87:58242) 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.807+0000 7f3a21e00640 1 -- 172.21.15.87:0/3283778680 learned_addr learned my addr 172.21.15.87:0/3283778680 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21400640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a240cdb60 0x7f3a240c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a22800640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a240cec70 0x7f3a24159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21e00640 1 -- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a240cdb60 msgr2=0x7f3a240c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a240cdb60 0x7f3a240c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21e00640 1 -- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a240cec70 msgr2=0x7f3a24159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a240cec70 0x7f3a24159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21e00640 1 -- 172.21.15.87:0/3283778680 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a240b04a0 con 0x7f3a240cbdf0 2024-04-25T10:15:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21400640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a240cdb60 0x7f3a240c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:43.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a22800640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a240cec70 0x7f3a24159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:43.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cbdf0 0x7f3a2414d0c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a140b0040 tx=0x7f3a140191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a1a000640 1 -- 172.21.15.87:0/3283778680 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140bc070 con 0x7f3a240cbdf0 2024-04-25T10:15:43.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a1a000640 1 -- 172.21.15.87:0/3283778680 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a140addb0 con 0x7f3a240cbdf0 2024-04-25T10:15:43.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a1a000640 1 -- 172.21.15.87:0/3283778680 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140b2040 con 0x7f3a240cbdf0 2024-04-25T10:15:43.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a240cc620 con 0x7f3a240cbdf0 2024-04-25T10:15:43.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a240cb5c0 con 0x7f3a240cbdf0 2024-04-25T10:15:43.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a1a000640 1 -- 172.21.15.87:0/3283778680 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a140bf070 con 0x7f3a240cbdf0 2024-04-25T10:15:43.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a1a000640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a04066f90 0x7f3a04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.811+0000 7f3a1a000640 1 -- 172.21.15.87:0/3283778680 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a1411e140 con 0x7f3a240cbdf0 2024-04-25T10:15:43.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a04066f90 0x7f3a04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 Processor -- start 2024-04-25T10:15:43.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- start start 2024-04-25T10:15:43.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a04066f90 0x7f3a04069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3a24159ea0 tx=0x7f3a10008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c507939450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50773cf00 con 0x55c507939990 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5076c54a0 con 0x55c5078ecbe0 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c507738fa0 con 0x55c507939070 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078ec6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58250/0 (socket says 172.21.15.87:58250) 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 -- 172.21.15.87:0/1253227564 learned_addr learned my addr 172.21.15.87:0/1253227564 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21e00640 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c507939450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a22800640 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 -- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 msgr2=0x55c507939450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c507939450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 -- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 msgr2=0x55c5078ecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078ecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 -- 172.21.15.87:0/1253227564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5076c2d00 con 0x55c507939990 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a21400640 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078ec6a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3a10004350 tx=0x7f3a10011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924db50b8e1d9970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a0ea00640 1 -- 172.21.15.87:0/1253227564 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000f040 con 0x55c507939990 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a0ea00640 1 -- 172.21.15.87:0/1253227564 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10004380 con 0x55c507939990 2024-04-25T10:15:43.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a0ea00640 1 -- 172.21.15.87:0/1253227564 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000ee80 con 0x55c507939990 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1253227564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 msgr2=0x55c5078ec6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078ec6a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3a10004350 tx=0x7f3a10011000 comp rx=0 tx=0).stop 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1253227564 shutdown_connections 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078ecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078ec6a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1253227564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c507939450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1253227564 >> 172.21.15.87:0/1253227564 conn(0x55c507919e10 msgr2=0x55c50791c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1253227564 shutdown_connections 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.815+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1253227564 wait complete. 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 -- start start 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 Processor -- start 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c5078f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c507741530 con 0x55c507939990 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5076f7ae0 con 0x55c5078ecbe0 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5076fbe10 con 0x55c507939070 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 10 client.0 _ll_register_callbacks cb 0x55c5078da750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58260/0 (socket says 172.21.15.87:58260) 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 -- 172.21.15.87:0/1650783534 learned_addr learned my addr 172.21.15.87:0/1650783534 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a21400640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c5078f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a21e00640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 -- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 msgr2=0x55c5078f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c5078f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 -- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 msgr2=0x55c5078f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 -- 172.21.15.87:0/1650783534 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5076dfa10 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a21e00640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a22800640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078f30b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00f820 tx=0x7f3a1c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2d892ad05b0910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a03400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a03400640 1 -- 172.21.15.87:0/1650783534 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c017020 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a03400640 1 -- 172.21.15.87:0/1650783534 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a1c00d940 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a10004380 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a03400640 1 -- 172.21.15.87:0/1650783534 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a1c021050 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 10 client.6291 fetch_fsmap learned FSMap version 28 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c507737ef0 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a03400640 1 -- 172.21.15.87:0/1650783534 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10004380 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a03400640 1 -- 172.21.15.87:0/1650783534 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3a1c024020 con 0x55c507939990 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 10 client.6291 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:15:43.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 10 client.6291 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:15:43.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 20 client.6291 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:43.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c507948310 0x55c50794a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 4 client.6291 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:43.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5076d8b20 con 0x55c507948310 2024-04-25T10:15:43.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.819+0000 7f3a21e00640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c507948310 0x55c50794a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a03400640 10 client.6291 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a21e00640 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c507948310 0x55c50794a700 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d63fc86a9217a6b server_cookie=57a0abfa0c7404bf in_seq=0 out_seq=0 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a03400640 1 -- 172.21.15.87:0/1650783534 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5076d8b20 con 0x55c507948310 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a03400640 10 client.6291 handle_command_reply: tid=0 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 client.6291 shutdown 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 msgr2=0x55c5078f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078f30b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00f820 tx=0x7f3a1c00fce0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 shutdown_connections 2024-04-25T10:15:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c507939990 0x55c5078f30b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c507948310 0x55c50794a700 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939070 0x55c5078f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/1650783534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c5078ecbe0 0x55c5078f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 >> 172.21.15.87:0/1650783534 conn(0x55c507919e10 msgr2=0x55c5078ebda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a03400640 0 client.6291 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 shutdown_connections 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/1650783534 wait complete. 2024-04-25T10:15:43.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.823+0000 7f3a2c0a6000 20 client.6291 trim_cache size 0 max 16384 2024-04-25T10:15:43.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 Processor -- start 2024-04-25T10:15:43.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- start start 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50799dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 0x55c50799e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 0x55c50799ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24158b90 con 0x55c5078ecbe0 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24157820 con 0x55c507939070 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24158590 con 0x55c507939990 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50799dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50799dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58264/0 (socket says 172.21.15.87:58264) 2024-04-25T10:15:43.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 -- 172.21.15.87:0/557178388 learned_addr learned my addr 172.21.15.87:0/557178388 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 -- 172.21.15.87:0/557178388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 msgr2=0x55c50799ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 --2- 172.21.15.87:0/557178388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 0x55c50799ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 -- 172.21.15.87:0/557178388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 msgr2=0x55c50799e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 --2- 172.21.15.87:0/557178388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 0x55c50799e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 -- 172.21.15.87:0/557178388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5076fc790 con 0x55c5078ecbe0 2024-04-25T10:15:43.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a21e00640 1 --2- 172.21.15.87:0/557178388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50799dfb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3a1401c9c0 tx=0x7f3a1411c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d2d4937bba4ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a0e000640 1 -- 172.21.15.87:0/557178388 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140ecae0 con 0x55c5078ecbe0 2024-04-25T10:15:43.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a0e000640 1 -- 172.21.15.87:0/557178388 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a140a9040 con 0x55c5078ecbe0 2024-04-25T10:15:43.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a0e000640 1 -- 172.21.15.87:0/557178388 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140eb0b0 con 0x55c5078ecbe0 2024-04-25T10:15:43.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/557178388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 msgr2=0x55c50799dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/557178388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50799dfb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3a1401c9c0 tx=0x7f3a1411c530 comp rx=0 tx=0).stop 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/557178388 shutdown_connections 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/557178388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 0x55c50799ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/557178388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 0x55c50799e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/557178388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50799dfb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/557178388 >> 172.21.15.87:0/557178388 conn(0x55c507919e10 msgr2=0x55c5078f3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/557178388 shutdown_connections 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.835+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/557178388 wait complete. 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 -- start start 2024-04-25T10:15:43.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 Processor -- start 2024-04-25T10:15:43.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50794bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 0x55c5079a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 0x55c5079a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50777cca0 con 0x55c5078ecbe0 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c507709b90 con 0x55c507939070 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5078f81c0 con 0x55c507939990 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 10 client.0 _ll_register_callbacks cb 0x7f3a10009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50794bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50794bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58266/0 (socket says 172.21.15.87:58266) 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 -- 172.21.15.87:0/3354326615 learned_addr learned my addr 172.21.15.87:0/3354326615 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 -- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 msgr2=0x55c5079a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 0x55c5079a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 -- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 msgr2=0x55c5079a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 0x55c5079a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 -- 172.21.15.87:0/3354326615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c507704da0 con 0x55c5078ecbe0 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21e00640 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50794bdc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3a140ee9d0 tx=0x7f3a140a4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86412891269a422b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a03400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a03400640 1 -- 172.21.15.87:0/3354326615 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140eb0b0 con 0x55c5078ecbe0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a03400640 1 -- 172.21.15.87:0/3354326615 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a1411c260 con 0x55c5078ecbe0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a140a9040 con 0x55c5078ecbe0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a03400640 1 -- 172.21.15.87:0/3354326615 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a140eb250 con 0x55c5078ecbe0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 10 client.6297 fetch_fsmap learned FSMap version 28 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c5073f9360 con 0x55c5078ecbe0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a03400640 1 -- 172.21.15.87:0/3354326615 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a140a9040 con 0x55c5078ecbe0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a03400640 1 -- 172.21.15.87:0/3354326615 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3a140b5070 con 0x55c5078ecbe0 2024-04-25T10:15:43.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 10 client.6297 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 10 client.6297 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 20 client.6297 populate_metadata read hostname 'smithi087' 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c50795de60 0x55c50795e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 4 client.6297 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3119778829}] 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 --> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3119778829}) v1 -- 0x55c50774a440 con 0x55c50795de60 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.839+0000 7f3a21400640 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c50795de60 0x55c50795e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a21400640 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c50795de60 0x55c50795e240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778f684ef1d656b8 server_cookie=2473de77d4561dbb in_seq=0 out_seq=0 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a03400640 10 client.6297 ms_handle_connect on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a03400640 1 -- 172.21.15.87:0/3354326615 <== mds.0 v2:172.21.15.169:6832/3236213442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55c50774a440 con 0x55c50795de60 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a03400640 10 client.6297 handle_command_reply: tid=0 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 client.6297 shutdown 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 msgr2=0x55c50794bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50794bdc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3a140ee9d0 tx=0x7f3a140a4f90 comp rx=0 tx=0).stop 2024-04-25T10:15:43.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 shutdown_connections 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:6832/3236213442,v1:172.21.15.169:6834/3236213442] conn(0x55c50795de60 0x55c50795e240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c507939990 0x55c5079a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c507939070 0x55c5079a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 --2- 172.21.15.87:0/3354326615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c5078ecbe0 0x55c50794bdc0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a03400640 0 client.6297 ms_handle_reset on v2:172.21.15.169:6832/3236213442 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 >> 172.21.15.87:0/3354326615 conn(0x55c507919e10 msgr2=0x55c5078f46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 shutdown_connections 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 1 -- 172.21.15.87:0/3354326615 wait complete. 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a2c0a6000 20 client.6297 trim_cache size 0 max 16384 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a04066f90 msgr2=0x7f3a04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a04066f90 0x7f3a04069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3a24159ea0 tx=0x7f3a10008040 comp rx=0 tx=0).stop 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cbdf0 msgr2=0x7f3a2414d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cbdf0 0x7f3a2414d0c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a140b0040 tx=0x7f3a140191f0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 shutdown_connections 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a04066f90 0x7f3a04069450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a240cec70 0x7f3a24159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a240cdb60 0x7f3a240c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 --2- 172.21.15.87:0/3283778680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a240cbdf0 0x7f3a2414d0c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 >> 172.21.15.87:0/3283778680 conn(0x7f3a240d17a0 msgr2=0x7f3a240ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 shutdown_connections 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.843+0000 7f3a23200640 1 -- 172.21.15.87:0/3283778680 wait complete. 2024-04-25T10:15:43.868 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:15:43.868 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.995+0000 7f4c41000640 1 Processor -- start 2024-04-25T10:15:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.995+0000 7f4c41000640 1 -- start start 2024-04-25T10:15:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c41000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4c3c0d0040 0x7f4c3c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c41000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c3c15b740 0x7f4c3c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c41000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c071050 con 0x7f4c3c1550b0 2024-04-25T10:15:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c41000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c15d040 con 0x7f4c3c0d0040 2024-04-25T10:15:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c41000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c0712f0 con 0x7f4c3c15b740 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4c3c0d0040 0x7f4c3c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4c3c0d0040 0x7f4c3c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58274/0 (socket says 172.21.15.87:58274) 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 -- 172.21.15.87:0/281560493 learned_addr learned my addr 172.21.15.87:0/281560493 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c3a800640 1 --2- 172.21.15.87:0/281560493 >> v2:172.21.15.169:3300/0 conn(0x7f4c3c15b740 0x7f4c3c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 -- 172.21.15.87:0/281560493 >> v2:172.21.15.169:3300/0 conn(0x7f4c3c15b740 msgr2=0x7f4c3c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 --2- 172.21.15.87:0/281560493 >> v2:172.21.15.169:3300/0 conn(0x7f4c3c15b740 0x7f4c3c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 -- 172.21.15.87:0/281560493 >> v1:172.21.15.169:6789/0 conn(0x7f4c3c1550b0 legacy=0x7f4c3c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 -- 172.21.15.87:0/281560493 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0715d0 con 0x7f4c3c0d0040 2024-04-25T10:15:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:43.999+0000 7f4c31400640 1 --2- 172.21.15.87:0/281560493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c0d0040 0x7f4c3c15b200 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0d04d0 tx=0x7f4c2401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5730a1a51704911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c39400640 1 -- 172.21.15.87:0/281560493 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240aa650 con 0x7f4c3c0d0040 2024-04-25T10:15:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c39400640 1 -- 172.21.15.87:0/281560493 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c240af070 con 0x7f4c3c0d0040 2024-04-25T10:15:44.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c39400640 1 -- 172.21.15.87:0/281560493 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240aaa10 con 0x7f4c3c0d0040 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- 172.21.15.87:0/281560493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c0d0040 msgr2=0x7f4c3c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 --2- 172.21.15.87:0/281560493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c0d0040 0x7f4c3c15b200 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0d04d0 tx=0x7f4c2401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- 172.21.15.87:0/281560493 shutdown_connections 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 --2- 172.21.15.87:0/281560493 >> v2:172.21.15.169:3300/0 conn(0x7f4c3c15b740 0x7f4c3c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 --2- 172.21.15.87:0/281560493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c0d0040 0x7f4c3c15b200 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- 172.21.15.87:0/281560493 >> 172.21.15.87:0/281560493 conn(0x7f4c3c066b70 msgr2=0x7f4c3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- 172.21.15.87:0/281560493 shutdown_connections 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- 172.21.15.87:0/281560493 wait complete. 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 Processor -- start 2024-04-25T10:15:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- start start 2024-04-25T10:15:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c3c0d0040 0x7f4c3c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c1550b0 0x7f4c3c13f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c15b740 0x7f4c3c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15bcd0 con 0x7f4c3c15b740 2024-04-25T10:15:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15bb50 con 0x7f4c3c1550b0 2024-04-25T10:15:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c41000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15c030 con 0x7f4c3c0d0040 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c3a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c15b740 0x7f4c3c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c3a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c15b740 0x7f4c3c15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58288/0 (socket says 172.21.15.87:58288) 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c3a800640 1 -- 172.21.15.87:0/2630910730 learned_addr learned my addr 172.21.15.87:0/2630910730 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c31400640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c1550b0 0x7f4c3c13f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.003+0000 7f4c39e00640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c3c0d0040 0x7f4c3c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c3a800640 1 -- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c3c0d0040 msgr2=0x7f4c3c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c3a800640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c3c0d0040 0x7f4c3c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c3a800640 1 -- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c1550b0 msgr2=0x7f4c3c13f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c3a800640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c1550b0 0x7f4c3c13f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c3a800640 1 -- 172.21.15.87:0/2630910730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c1fa950 con 0x7f4c3c15b740 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c39e00640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c3c0d0040 0x7f4c3c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c3a800640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c15b740 0x7f4c3c15a190 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400efc0 tx=0x7f4c3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c32a00640 1 -- 172.21.15.87:0/2630910730 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34018020 con 0x7f4c3c15b740 2024-04-25T10:15:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c32a00640 1 -- 172.21.15.87:0/2630910730 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c34014480 con 0x7f4c3c15b740 2024-04-25T10:15:44.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c32a00640 1 -- 172.21.15.87:0/2630910730 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34014870 con 0x7f4c3c15b740 2024-04-25T10:15:44.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c1fac30 con 0x7f4c3c15b740 2024-04-25T10:15:44.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c159990 con 0x7f4c3c15b740 2024-04-25T10:15:44.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c04ef30 con 0x7f4c3c15b740 2024-04-25T10:15:44.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c32a00640 1 -- 172.21.15.87:0/2630910730 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c3401b020 con 0x7f4c3c15b740 2024-04-25T10:15:44.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c32a00640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c18066f60 0x7f4c18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c32a00640 1 -- 172.21.15.87:0/2630910730 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c34085a60 con 0x7f4c3c15b740 2024-04-25T10:15:44.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.007+0000 7f4c39e00640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c18066f60 0x7f4c18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.011+0000 7f4c39e00640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c18066f60 0x7f4c18069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001710 tx=0x7f4c2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.011+0000 7f4c32a00640 1 -- 172.21.15.87:0/2630910730 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c34022070 con 0x7f4c3c15b740 2024-04-25T10:15:44.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c3c0ceed0 con 0x7f4c3c15b740 2024-04-25T10:15:44.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c32a00640 1 -- 172.21.15.87:0/2630910730 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f4c34010040 con 0x7f4c3c15b740 2024-04-25T10:15:44.205 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:44.205 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:43.370279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:44.205 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 28 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c18066f60 msgr2=0x7f4c18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c18066f60 0x7f4c18069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001710 tx=0x7f4c2c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c15b740 msgr2=0x7f4c3c15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c15b740 0x7f4c3c15a190 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400efc0 tx=0x7f4c3400c490 comp rx=0 tx=0).stop 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 shutdown_connections 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c18066f60 0x7f4c18069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c15b740 0x7f4c3c15a190 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c1550b0 0x7f4c3c13f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 --2- 172.21.15.87:0/2630910730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c3c0d0040 0x7f4c3c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 >> 172.21.15.87:0/2630910730 conn(0x7f4c3c066b70 msgr2=0x7f4c3c15c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:44.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 shutdown_connections 2024-04-25T10:15:44.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.199+0000 7f4c41000640 1 -- 172.21.15.87:0/2630910730 wait complete. 2024-04-25T10:15:44.225 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:15:44.377 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T10:15:44.377 INFO:teuthology.orchestra.run.smithi169.stdout: "return_code": 0 2024-04-25T10:15:44.377 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T10:15:44.392 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:15:44.393 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:44.393 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:44.478 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:44.562 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:44.563 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:44.563 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:44.571 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:44.571 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:44.623 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:44.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff655e00640 1 Processor -- start 2024-04-25T10:15:44.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff655e00640 1 -- start start 2024-04-25T10:15:44.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff655e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6500d1820 0x7ff6500d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff655e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff65015e050 0x7ff6500b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff655e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6500b1af0 con 0x7ff6500b13e0 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff655e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6500b1dd0 con 0x7ff6500d1820 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff655e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6500b20b0 con 0x7ff65015e050 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff64f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff6500b13e0 0x7ff6500ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47746/0 (socket says 172.21.15.87:47746) 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff64f400640 1 -- 172.21.15.87:0/793311059 learned_addr learned my addr 172.21.15.87:0/793311059 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff64ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6500d1820 0x7ff6500d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff64e000640 1 --2- 172.21.15.87:0/793311059 >> v2:172.21.15.169:3300/0 conn(0x7ff65015e050 0x7ff6500b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.791+0000 7ff64d600640 1 -- 172.21.15.87:0/793311059 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085331949 0 0) 0x7ff6500b1af0 con 0x7ff6500b13e0 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64d600640 1 -- 172.21.15.87:0/793311059 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff634003530 con 0x7ff6500b13e0 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64ea00640 1 -- 172.21.15.87:0/793311059 >> v2:172.21.15.169:3300/0 conn(0x7ff65015e050 msgr2=0x7ff6500b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64ea00640 1 --2- 172.21.15.87:0/793311059 >> v2:172.21.15.169:3300/0 conn(0x7ff65015e050 0x7ff6500b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64ea00640 1 -- 172.21.15.87:0/793311059 >> v1:172.21.15.87:6789/0 conn(0x7ff6500b13e0 legacy=0x7ff6500ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64ea00640 1 -- 172.21.15.87:0/793311059 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6500b2390 con 0x7ff6500d1820 2024-04-25T10:15:44.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64ea00640 1 --2- 172.21.15.87:0/793311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500d1c00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff6500c5400 tx=0x7ff64001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4815af9fef59452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64d600640 1 -- 172.21.15.87:0/793311059 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400b2070 con 0x7ff6500d1820 2024-04-25T10:15:44.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64d600640 1 -- 172.21.15.87:0/793311059 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6400a5a10 con 0x7ff6500d1820 2024-04-25T10:15:44.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff64d600640 1 -- 172.21.15.87:0/793311059 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400a5de0 con 0x7ff6500d1820 2024-04-25T10:15:44.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 -- 172.21.15.87:0/793311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 msgr2=0x7ff6500d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 --2- 172.21.15.87:0/793311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500d1c00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff6500c5400 tx=0x7ff64001c930 comp rx=0 tx=0).stop 2024-04-25T10:15:44.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 -- 172.21.15.87:0/793311059 shutdown_connections 2024-04-25T10:15:44.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 --2- 172.21.15.87:0/793311059 >> v2:172.21.15.169:3300/0 conn(0x7ff65015e050 0x7ff6500b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 --2- 172.21.15.87:0/793311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500d1c00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 -- 172.21.15.87:0/793311059 >> 172.21.15.87:0/793311059 conn(0x7ff65009a9d0 msgr2=0x7ff65009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:44.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 -- 172.21.15.87:0/793311059 shutdown_connections 2024-04-25T10:15:44.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.795+0000 7ff655e00640 1 -- 172.21.15.87:0/793311059 wait complete. 2024-04-25T10:15:44.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 Processor -- start 2024-04-25T10:15:44.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 -- start start 2024-04-25T10:15:44.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6500b13e0 0x7ff650152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff65015e050 0x7ff6500c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65016ab20 con 0x7ff6500d1820 2024-04-25T10:15:44.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65016a9a0 con 0x7ff6500b13e0 2024-04-25T10:15:44.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65016aca0 con 0x7ff65015e050 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58308/0 (socket says 172.21.15.87:58308) 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 -- 172.21.15.87:0/4261208888 learned_addr learned my addr 172.21.15.87:0/4261208888 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64ea00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6500b13e0 0x7ff650152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64f400640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff65015e050 0x7ff6500c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 -- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff65015e050 msgr2=0x7ff6500c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff65015e050 0x7ff6500c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 -- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6500b13e0 msgr2=0x7ff650152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6500b13e0 0x7ff650152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 -- 172.21.15.87:0/4261208888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6500cbfd0 con 0x7ff6500d1820 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff64e000640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500cd070 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00e870 tx=0x7ff63c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff647400640 1 -- 172.21.15.87:0/4261208888 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c017070 con 0x7ff6500d1820 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff647400640 1 -- 172.21.15.87:0/4261208888 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c00c940 con 0x7ff6500d1820 2024-04-25T10:15:44.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6500cc2b0 con 0x7ff6500d1820 2024-04-25T10:15:44.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff647400640 1 -- 172.21.15.87:0/4261208888 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00cd10 con 0x7ff6500d1820 2024-04-25T10:15:44.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.799+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6500cc870 con 0x7ff6500d1820 2024-04-25T10:15:44.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.803+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65004ef30 con 0x7ff6500d1820 2024-04-25T10:15:44.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.803+0000 7ff647400640 1 -- 172.21.15.87:0/4261208888 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff63c01a070 con 0x7ff6500d1820 2024-04-25T10:15:44.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.803+0000 7ff647400640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff630066f60 0x7ff630069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.803+0000 7ff647400640 1 -- 172.21.15.87:0/4261208888 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff63c085280 con 0x7ff6500d1820 2024-04-25T10:15:44.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.803+0000 7ff64ea00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff630066f60 0x7ff630069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.807+0000 7ff64ea00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff630066f60 0x7ff630069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff6400af000 tx=0x7ff6400a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.807+0000 7ff647400640 1 -- 172.21.15.87:0/4261208888 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff63c055ba0 con 0x7ff6500d1820 2024-04-25T10:15:44.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.963+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6500b0910 con 0x7ff6500d1820 2024-04-25T10:15:44.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.963+0000 7ff647400640 1 -- 172.21.15.87:0/4261208888 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7ff63c059c40 con 0x7ff6500d1820 2024-04-25T10:15:44.970 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:44.971 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:08.327470+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:05.052743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:05.692130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff630066f60 msgr2=0x7ff630069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff630066f60 0x7ff630069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff6400af000 tx=0x7ff6400a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 msgr2=0x7ff6500cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500cd070 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00e870 tx=0x7ff63c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 shutdown_connections 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff630066f60 0x7ff630069420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff65015e050 0x7ff6500c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6500d1820 0x7ff6500cd070 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 --2- 172.21.15.87:0/4261208888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6500b13e0 0x7ff650152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 >> 172.21.15.87:0/4261208888 conn(0x7ff65009a9d0 msgr2=0x7ff6500cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:44.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 shutdown_connections 2024-04-25T10:15:44.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:44.967+0000 7ff655e00640 1 -- 172.21.15.87:0/4261208888 wait complete. 2024-04-25T10:15:44.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:05.052743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:05.692130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:44.991 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:45.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf79400640 1 Processor -- start 2024-04-25T10:15:45.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf79400640 1 -- start start 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf79400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf740aeda0 0x7fbf74071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf79400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf74159920 con 0x7fbf74072020 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf79400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7415c2f0 con 0x7fbf74159540 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf79400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7415c5d0 con 0x7fbf740aeda0 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf73200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbf74072020 0x7fbf74070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47756/0 (socket says 172.21.15.87:47756) 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf73200640 1 -- 172.21.15.87:0/2415326083 learned_addr learned my addr 172.21.15.87:0/2415326083 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf69e00640 1 --2- 172.21.15.87:0/2415326083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf740aeda0 0x7fbf74071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf71e00640 1 -- 172.21.15.87:0/2415326083 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540037628 0 0) 0x7fbf74159920 con 0x7fbf74072020 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf71e00640 1 -- 172.21.15.87:0/2415326083 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf58003530 con 0x7fbf74072020 2024-04-25T10:15:45.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf71e00640 1 -- 172.21.15.87:0/2415326083 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2722115645 0 0) 0x7fbf58003530 con 0x7fbf74072020 2024-04-25T10:15:45.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf69e00640 1 -- 172.21.15.87:0/2415326083 >> v1:172.21.15.169:6789/0 conn(0x7fbf74159540 legacy=0x7fbf740ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf69e00640 1 -- 172.21.15.87:0/2415326083 >> v1:172.21.15.87:6789/0 conn(0x7fbf74072020 legacy=0x7fbf74070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.123+0000 7fbf69e00640 1 -- 172.21.15.87:0/2415326083 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7415c8b0 con 0x7fbf740aeda0 2024-04-25T10:15:45.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf69e00640 1 --2- 172.21.15.87:0/2415326083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf740aeda0 0x7fbf74071ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0097f0 tx=0x7fbf5c01c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d24ab0a1021eb36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf71e00640 1 -- 172.21.15.87:0/2415326083 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0b0070 con 0x7fbf740aeda0 2024-04-25T10:15:45.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf71e00640 1 -- 172.21.15.87:0/2415326083 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c0ab400 con 0x7fbf740aeda0 2024-04-25T10:15:45.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf71e00640 1 -- 172.21.15.87:0/2415326083 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c001680 con 0x7fbf740aeda0 2024-04-25T10:15:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf79400640 1 -- 172.21.15.87:0/2415326083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf740aeda0 msgr2=0x7fbf74071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf79400640 1 --2- 172.21.15.87:0/2415326083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf740aeda0 0x7fbf74071ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0097f0 tx=0x7fbf5c01c280 comp rx=0 tx=0).stop 2024-04-25T10:15:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf79400640 1 -- 172.21.15.87:0/2415326083 shutdown_connections 2024-04-25T10:15:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf79400640 1 --2- 172.21.15.87:0/2415326083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf740aeda0 0x7fbf74071ae0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf79400640 1 -- 172.21.15.87:0/2415326083 >> 172.21.15.87:0/2415326083 conn(0x7fbf740669a0 msgr2=0x7fbf74066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf79400640 1 -- 172.21.15.87:0/2415326083 shutdown_connections 2024-04-25T10:15:45.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.127+0000 7fbf79400640 1 -- 172.21.15.87:0/2415326083 wait complete. 2024-04-25T10:15:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 Processor -- start 2024-04-25T10:15:45.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 -- start start 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf74072020 0x7fbf74152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf740aeda0 0x7fbf7414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf74159540 0x7fbf74154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74169940 con 0x7fbf740aeda0 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7406c0d0 con 0x7fbf74072020 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74169ac0 con 0x7fbf74159540 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf740aeda0 0x7fbf7414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf740aeda0 0x7fbf7414e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58320/0 (socket says 172.21.15.87:58320) 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 -- 172.21.15.87:0/1754929522 learned_addr learned my addr 172.21.15.87:0/1754929522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf73200640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf74159540 0x7fbf74154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf72800640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf74072020 0x7fbf74152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 -- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf74159540 msgr2=0x7fbf74154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf74159540 0x7fbf74154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 -- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf74072020 msgr2=0x7fbf74152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf74072020 0x7fbf74152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 -- 172.21.15.87:0/1754929522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf74157550 con 0x7fbf740aeda0 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf72800640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf74072020 0x7fbf74152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf69e00640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf740aeda0 0x7fbf7414e290 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0ae040 tx=0x7fbf5c01aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf6b400640 1 -- 172.21.15.87:0/1754929522 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0bb020 con 0x7fbf740aeda0 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf741577f0 con 0x7fbf740aeda0 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf6b400640 1 -- 172.21.15.87:0/1754929522 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c0b1040 con 0x7fbf740aeda0 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf6b400640 1 -- 172.21.15.87:0/1754929522 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0b0050 con 0x7fbf740aeda0 2024-04-25T10:15:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.131+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf74157db0 con 0x7fbf740aeda0 2024-04-25T10:15:45.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.135+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7404ef30 con 0x7fbf740aeda0 2024-04-25T10:15:45.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.135+0000 7fbf6b400640 1 -- 172.21.15.87:0/1754929522 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf5c0be070 con 0x7fbf740aeda0 2024-04-25T10:15:45.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.135+0000 7fbf6b400640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf50066f90 0x7fbf50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.135+0000 7fbf6b400640 1 -- 172.21.15.87:0/1754929522 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf5c11e750 con 0x7fbf740aeda0 2024-04-25T10:15:45.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.135+0000 7fbf72800640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf50066f90 0x7fbf50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.135+0000 7fbf72800640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf50066f90 0x7fbf50069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf640012f0 tx=0x7fbf64009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.139+0000 7fbf6b400640 1 -- 172.21.15.87:0/1754929522 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf5c015ff0 con 0x7fbf740aeda0 2024-04-25T10:15:45.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.291+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf7415d200 con 0x7fbf740aeda0 2024-04-25T10:15:45.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.291+0000 7fbf6b400640 1 -- 172.21.15.87:0/1754929522 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf5c0ef6d0 con 0x7fbf740aeda0 2024-04-25T10:15:45.299 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf50066f90 msgr2=0x7fbf50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf50066f90 0x7fbf50069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf640012f0 tx=0x7fbf64009040 comp rx=0 tx=0).stop 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf740aeda0 msgr2=0x7fbf7414e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf740aeda0 0x7fbf7414e290 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0ae040 tx=0x7fbf5c01aac0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 shutdown_connections 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf50066f90 0x7fbf50069450 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf74159540 0x7fbf74154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf740aeda0 0x7fbf7414e290 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 --2- 172.21.15.87:0/1754929522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf74072020 0x7fbf74152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 >> 172.21.15.87:0/1754929522 conn(0x7fbf740669a0 msgr2=0x7fbf74070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 shutdown_connections 2024-04-25T10:15:45.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.295+0000 7fbf79400640 1 -- 172.21.15.87:0/1754929522 wait complete. 2024-04-25T10:15:45.320 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.451+0000 7fc23ea00640 1 Processor -- start 2024-04-25T10:15:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.451+0000 7fc23ea00640 1 -- start start 2024-04-25T10:15:45.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc23ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc238159540 0x7fc2380ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc23ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc2380aebd0 0x7fc238071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc23ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc238071360 con 0x7fc238072020 2024-04-25T10:15:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc23ea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc238159920 con 0x7fc238159540 2024-04-25T10:15:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc23ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23815c2f0 con 0x7fc2380aebd0 2024-04-25T10:15:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc238159540 0x7fc2380ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc238159540 0x7fc2380ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58332/0 (socket says 172.21.15.87:58332) 2024-04-25T10:15:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 -- 172.21.15.87:0/2571918074 learned_addr learned my addr 172.21.15.87:0/2571918074 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237400640 1 --2- 172.21.15.87:0/2571918074 >> v2:172.21.15.169:3300/0 conn(0x7fc2380aebd0 0x7fc238071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 -- 172.21.15.87:0/2571918074 >> v2:172.21.15.169:3300/0 conn(0x7fc2380aebd0 msgr2=0x7fc238071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 --2- 172.21.15.87:0/2571918074 >> v2:172.21.15.169:3300/0 conn(0x7fc2380aebd0 0x7fc238071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 -- 172.21.15.87:0/2571918074 >> v1:172.21.15.169:6789/0 conn(0x7fc238072020 legacy=0x7fc23815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 -- 172.21.15.87:0/2571918074 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23815c5d0 con 0x7fc238159540 2024-04-25T10:15:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237400640 1 --2- 172.21.15.87:0/2571918074 >> v2:172.21.15.169:3300/0 conn(0x7fc2380aebd0 0x7fc238071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc237e00640 1 --2- 172.21.15.87:0/2571918074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238159540 0x7fc2380ae690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc2380cd4e0 tx=0x7fc22801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc91b2b1b0b3762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.455+0000 7fc236a00640 1 -- 172.21.15.87:0/2571918074 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280a5680 con 0x7fc238159540 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc236a00640 1 -- 172.21.15.87:0/2571918074 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2280a5820 con 0x7fc238159540 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc236a00640 1 -- 172.21.15.87:0/2571918074 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280a5bf0 con 0x7fc238159540 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- 172.21.15.87:0/2571918074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238159540 msgr2=0x7fc2380ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 --2- 172.21.15.87:0/2571918074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238159540 0x7fc2380ae690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc2380cd4e0 tx=0x7fc22801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- 172.21.15.87:0/2571918074 shutdown_connections 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 --2- 172.21.15.87:0/2571918074 >> v2:172.21.15.169:3300/0 conn(0x7fc2380aebd0 0x7fc238071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 --2- 172.21.15.87:0/2571918074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238159540 0x7fc2380ae690 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- 172.21.15.87:0/2571918074 >> 172.21.15.87:0/2571918074 conn(0x7fc2380669a0 msgr2=0x7fc238066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- 172.21.15.87:0/2571918074 shutdown_connections 2024-04-25T10:15:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- 172.21.15.87:0/2571918074 wait complete. 2024-04-25T10:15:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 Processor -- start 2024-04-25T10:15:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- start start 2024-04-25T10:15:45.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238072020 0x7fc2380adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2380aebd0 0x7fc23813f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc238159540 0x7fc238152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23806c0d0 con 0x7fc238072020 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc238072450 con 0x7fc238159540 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc238168930 con 0x7fc2380aebd0 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc237e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238072020 0x7fc2380adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc237e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238072020 0x7fc2380adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58340/0 (socket says 172.21.15.87:58340) 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc237e00640 1 -- 172.21.15.87:0/3644673479 learned_addr learned my addr 172.21.15.87:0/3644673479 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc237400640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2380aebd0 0x7fc23813f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.459+0000 7fc23ca00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc238159540 0x7fc238152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc237e00640 1 -- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2380aebd0 msgr2=0x7fc23813f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc237e00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2380aebd0 0x7fc23813f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc237e00640 1 -- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc238159540 msgr2=0x7fc238152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc237e00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc238159540 0x7fc238152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc237e00640 1 -- 172.21.15.87:0/3644673479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238155630 con 0x7fc238072020 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc23ca00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc238159540 0x7fc238152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc237e00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238072020 0x7fc2380adaa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc2280af040 tx=0x7fc228019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc22fe00640 1 -- 172.21.15.87:0/3644673479 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280bc020 con 0x7fc238072020 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc22fe00640 1 -- 172.21.15.87:0/3644673479 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2280b8670 con 0x7fc238072020 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc22fe00640 1 -- 172.21.15.87:0/3644673479 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280b8a60 con 0x7fc238072020 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc238153950 con 0x7fc238072020 2024-04-25T10:15:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23815b350 con 0x7fc238072020 2024-04-25T10:15:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23804ef30 con 0x7fc238072020 2024-04-25T10:15:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc22fe00640 1 -- 172.21.15.87:0/3644673479 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2280bf020 con 0x7fc238072020 2024-04-25T10:15:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.463+0000 7fc22fe00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc218066f60 0x7fc218069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.467+0000 7fc22fe00640 1 -- 172.21.15.87:0/3644673479 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc228120060 con 0x7fc238072020 2024-04-25T10:15:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.467+0000 7fc237400640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc218066f60 0x7fc218069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.467+0000 7fc237400640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc218066f60 0x7fc218069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc23815d7b0 tx=0x7fc224008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.471+0000 7fc22fe00640 1 -- 172.21.15.87:0/3644673479 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2280c6030 con 0x7fc238072020 2024-04-25T10:15:45.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.627+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc238071b70 con 0x7fc238072020 2024-04-25T10:15:45.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.627+0000 7fc22fe00640 1 -- 172.21.15.87:0/3644673479 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2280165a0 con 0x7fc238072020 2024-04-25T10:15:45.634 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:15:45.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc218066f60 msgr2=0x7fc218069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc218066f60 0x7fc218069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc23815d7b0 tx=0x7fc224008040 comp rx=0 tx=0).stop 2024-04-25T10:15:45.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238072020 msgr2=0x7fc2380adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238072020 0x7fc2380adaa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc2280af040 tx=0x7fc228019750 comp rx=0 tx=0).stop 2024-04-25T10:15:45.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 shutdown_connections 2024-04-25T10:15:45.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc218066f60 0x7fc218069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc238159540 0x7fc238152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2380aebd0 0x7fc23813f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 --2- 172.21.15.87:0/3644673479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc238072020 0x7fc2380adaa0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 >> 172.21.15.87:0/3644673479 conn(0x7fc2380669a0 msgr2=0x7fc2380cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 shutdown_connections 2024-04-25T10:15:45.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.631+0000 7fc23ea00640 1 -- 172.21.15.87:0/3644673479 wait complete. 2024-04-25T10:15:45.654 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:45.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe684000640 1 Processor -- start 2024-04-25T10:15:45.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe684000640 1 -- start start 2024-04-25T10:15:45.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe684000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 0x7fe67c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe684000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0cfba0 con 0x7fe67c0d0630 2024-04-25T10:15:45.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe684000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0b4a00 con 0x7fe67c15bbd0 2024-04-25T10:15:45.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe684000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b47b0 con 0x7fe67c0d2450 2024-04-25T10:15:45.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe680c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe67c0d0630 0x7fe67c0b54e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47772/0 (socket says 172.21.15.87:47772) 2024-04-25T10:15:45.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe680c00640 1 -- 172.21.15.87:0/3561669956 learned_addr learned my addr 172.21.15.87:0/3561669956 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe682000640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 0x7fe67c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269911492 0 0) 0x7fe67c0cfba0 con 0x7fe67c0d0630 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe664003530 con 0x7fe67c0d0630 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089515858 0 0) 0x7fe664003530 con 0x7fe67c0d0630 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 msgr2=0x7fe67c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 0x7fe67c0cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 >> v1:172.21.15.169:6789/0 conn(0x7fe67c15bbd0 legacy=0x7fe67c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0b4ce0 con 0x7fe67c0d0630 2024-04-25T10:15:45.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe668001680 con 0x7fe67c0d0630 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 >> v1:172.21.15.87:6789/0 conn(0x7fe67c0d0630 legacy=0x7fe67c0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 0x7fe6640039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664003f30 0x7fe664004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe664004d50 0x7fe6640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b47b0 con 0x7fe664004d50 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66400bae0 con 0x7fe664003f30 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66400bce0 con 0x7fe67c0d2450 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe681600640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 0x7fe6640039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe682000640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe664004d50 0x7fe6640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe680c00640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664003f30 0x7fe664004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe682000640 1 -- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 msgr2=0x7fe6640039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe682000640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0d2450 0x7fe6640039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe682000640 1 -- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664003f30 msgr2=0x7fe664004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe682000640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664003f30 0x7fe664004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe682000640 1 -- 172.21.15.87:0/3561669956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664003530 con 0x7fe664004d50 2024-04-25T10:15:45.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.787+0000 7fe680c00640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664003f30 0x7fe664004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:45.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe682000640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe664004d50 0x7fe6640051d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe674009fc0 tx=0x7fe67401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b562340eaf098095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6740b2070 con 0x7fe664004d50 2024-04-25T10:15:45.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6740a5a70 con 0x7fe664004d50 2024-04-25T10:15:45.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe67be00640 1 -- 172.21.15.87:0/3561669956 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674009c10 con 0x7fe664004d50 2024-04-25T10:15:45.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe684000640 1 -- 172.21.15.87:0/3561669956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe664004d50 msgr2=0x7fe6640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe684000640 1 --2- 172.21.15.87:0/3561669956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe664004d50 0x7fe6640051d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe674009fc0 tx=0x7fe67401cc80 comp rx=0 tx=0).stop 2024-04-25T10:15:45.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe682000640 1 -- 172.21.15.87:0/3561669956 reap_dead start 2024-04-25T10:15:45.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe684000640 1 -- 172.21.15.87:0/3561669956 shutdown_connections 2024-04-25T10:15:45.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe684000640 1 -- 172.21.15.87:0/3561669956 >> 172.21.15.87:0/3561669956 conn(0x7fe67c0d1960 msgr2=0x7fe67c0d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe684000640 1 -- 172.21.15.87:0/3561669956 shutdown_connections 2024-04-25T10:15:45.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.791+0000 7fe684000640 1 -- 172.21.15.87:0/3561669956 wait complete. 2024-04-25T10:15:45.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 Processor -- start 2024-04-25T10:15:45.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 -- start start 2024-04-25T10:15:45.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664004d50 0x7fe67c164590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe67c06bfc0 0x7fe67c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0cb270 0x7fe67c0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b6610 con 0x7fe67c06bfc0 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c15c1c0 con 0x7fe664004d50 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b6790 con 0x7fe67c0cb270 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe67c06bfc0 0x7fe67c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe67c06bfc0 0x7fe67c15e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58362/0 (socket says 172.21.15.87:58362) 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 -- 172.21.15.87:0/2982712739 learned_addr learned my addr 172.21.15.87:0/2982712739 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe681600640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664004d50 0x7fe67c164590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe682000640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0cb270 0x7fe67c0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 -- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0cb270 msgr2=0x7fe67c0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0cb270 0x7fe67c0cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 -- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664004d50 msgr2=0x7fe67c164590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664004d50 0x7fe67c164590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 -- 172.21.15.87:0/2982712739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0cbc60 con 0x7fe67c06bfc0 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe680c00640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe67c06bfc0 0x7fe67c15e190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe6680010f0 tx=0x7fe66800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe679600640 1 -- 172.21.15.87:0/2982712739 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800ee00 con 0x7fe67c06bfc0 2024-04-25T10:15:45.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe679600640 1 -- 172.21.15.87:0/2982712739 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe668018070 con 0x7fe67c06bfc0 2024-04-25T10:15:45.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c164c50 con 0x7fe67c06bfc0 2024-04-25T10:15:45.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe679600640 1 -- 172.21.15.87:0/2982712739 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800cac0 con 0x7fe67c06bfc0 2024-04-25T10:15:45.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.795+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c165210 con 0x7fe67c06bfc0 2024-04-25T10:15:45.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.799+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c04ef30 con 0x7fe67c06bfc0 2024-04-25T10:15:45.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.799+0000 7fe679600640 1 -- 172.21.15.87:0/2982712739 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe66800cc60 con 0x7fe67c06bfc0 2024-04-25T10:15:45.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.799+0000 7fe679600640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe65c066f90 0x7fe65c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.799+0000 7fe679600640 1 -- 172.21.15.87:0/2982712739 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe668084a30 con 0x7fe67c06bfc0 2024-04-25T10:15:45.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.799+0000 7fe681600640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe65c066f90 0x7fe65c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.799+0000 7fe681600640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe65c066f90 0x7fe65c069450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe67c15ebf0 tx=0x7fe670009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.803+0000 7fe679600640 1 -- 172.21.15.87:0/2982712739 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe67c0cbc60 con 0x7fe67c06bfc0 2024-04-25T10:15:45.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe67c06d510 con 0x7fe67c06bfc0 2024-04-25T10:15:45.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe679600640 1 -- 172.21.15.87:0/2982712739 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe668055350 con 0x7fe67c06bfc0 2024-04-25T10:15:45.961 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:15:45.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe65c066f90 msgr2=0x7fe65c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe65c066f90 0x7fe65c069450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe67c15ebf0 tx=0x7fe670009040 comp rx=0 tx=0).stop 2024-04-25T10:15:45.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe67c06bfc0 msgr2=0x7fe67c15e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe67c06bfc0 0x7fe67c15e190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe6680010f0 tx=0x7fe66800ec10 comp rx=0 tx=0).stop 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 shutdown_connections 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe65c066f90 0x7fe65c069450 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe67c0cb270 0x7fe67c0cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe67c06bfc0 0x7fe67c15e190 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 --2- 172.21.15.87:0/2982712739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe664004d50 0x7fe67c164590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 >> 172.21.15.87:0/2982712739 conn(0x7fe67c0d1960 msgr2=0x7fe67c06efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 shutdown_connections 2024-04-25T10:15:45.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:45.955+0000 7fe684000640 1 -- 172.21.15.87:0/2982712739 wait complete. 2024-04-25T10:15:45.982 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:15:46.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.111+0000 7f660c400640 1 Processor -- start 2024-04-25T10:15:46.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.111+0000 7f660c400640 1 -- start start 2024-04-25T10:15:46.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.111+0000 7f660c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66040b0f70 0x7f66040b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f660c400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660415f3b0 con 0x7f66040cd2e0 2024-04-25T10:15:46.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f660c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040ca810 con 0x7f66040cfd90 2024-04-25T10:15:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f660c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660415ef20 con 0x7f66040b0f70 2024-04-25T10:15:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6609a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66040b0f70 0x7f66040b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f660a400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f66040cd2e0 0x7f66040cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47776/0 (socket says 172.21.15.87:47776) 2024-04-25T10:15:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f660a400640 1 -- 172.21.15.87:0/1908056595 learned_addr learned my addr 172.21.15.87:0/1908056595 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6609a00640 1 -- 172.21.15.87:0/1908056595 >> v1:172.21.15.169:6789/0 conn(0x7f66040cfd90 legacy=0x7f66040ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6609a00640 1 -- 172.21.15.87:0/1908056595 >> v1:172.21.15.87:6789/0 conn(0x7f66040cd2e0 legacy=0x7f66040cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6609a00640 1 -- 172.21.15.87:0/1908056595 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040b1890 con 0x7f66040b0f70 2024-04-25T10:15:46.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6609a00640 1 --2- 172.21.15.87:0/1908056595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040b0f70 0x7f66040b1350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009d90 tx=0x7f65f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d678717fa10c1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6603e00640 1 -- 172.21.15.87:0/1908056595 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80b2070 con 0x7f66040b0f70 2024-04-25T10:15:46.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6603e00640 1 -- 172.21.15.87:0/1908056595 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f80a57f0 con 0x7f66040b0f70 2024-04-25T10:15:46.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.115+0000 7f6603e00640 1 -- 172.21.15.87:0/1908056595 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80a5bc0 con 0x7f66040b0f70 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- 172.21.15.87:0/1908056595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040b0f70 msgr2=0x7f66040b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 --2- 172.21.15.87:0/1908056595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040b0f70 0x7f66040b1350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009d90 tx=0x7f65f801c930 comp rx=0 tx=0).stop 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- 172.21.15.87:0/1908056595 shutdown_connections 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 --2- 172.21.15.87:0/1908056595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040b0f70 0x7f66040b1350 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- 172.21.15.87:0/1908056595 >> 172.21.15.87:0/1908056595 conn(0x7f66040b1d50 msgr2=0x7f66040b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- 172.21.15.87:0/1908056595 shutdown_connections 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- 172.21.15.87:0/1908056595 wait complete. 2024-04-25T10:15:46.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 Processor -- start 2024-04-25T10:15:46.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- start start 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66040b0f70 0x7f6604157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040cd2e0 0x7f66040b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66040cfd90 0x7f66040b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660416bf00 con 0x7f66040cd2e0 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660416bd80 con 0x7f66040cfd90 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660416c080 con 0x7f66040b0f70 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040cd2e0 0x7f66040b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040cd2e0 0x7f66040b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58394/0 (socket says 172.21.15.87:58394) 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 -- 172.21.15.87:0/1546451044 learned_addr learned my addr 172.21.15.87:0/1546451044 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609a00640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66040b0f70 0x7f6604157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660a400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66040cfd90 0x7f66040b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 -- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66040b0f70 msgr2=0x7f6604157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66040b0f70 0x7f6604157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 -- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66040cfd90 msgr2=0x7f66040b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66040cfd90 0x7f66040b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 -- 172.21.15.87:0/1546451044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604158350 con 0x7f66040cd2e0 2024-04-25T10:15:46.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f660a400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66040cfd90 0x7f66040b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.119+0000 7f6609000640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040cd2e0 0x7f66040b50d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c410 tx=0x7f65f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6601600640 1 -- 172.21.15.87:0/1546451044 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0011790 con 0x7f66040cd2e0 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6601600640 1 -- 172.21.15.87:0/1546451044 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f0017070 con 0x7f66040cd2e0 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6601600640 1 -- 172.21.15.87:0/1546451044 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0011bb0 con 0x7f66040cd2e0 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040b5c40 con 0x7f66040cd2e0 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66040c7570 con 0x7f66040cd2e0 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660404ef30 con 0x7f66040cd2e0 2024-04-25T10:15:46.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6601600640 1 -- 172.21.15.87:0/1546451044 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65f001c460 con 0x7f66040cd2e0 2024-04-25T10:15:46.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6601600640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f65e4066f60 0x7f65e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6601600640 1 -- 172.21.15.87:0/1546451044 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65f0084b80 con 0x7f66040cd2e0 2024-04-25T10:15:46.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6609a00640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f65e4066f60 0x7f65e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.123+0000 7f6609a00640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f65e4066f60 0x7f65e4069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65f80af000 tx=0x7f65f8004ac0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.127+0000 7f6601600640 1 -- 172.21.15.87:0/1546451044 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65f0021020 con 0x7f66040cd2e0 2024-04-25T10:15:46.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.295+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f66040cda20 con 0x7f66040cd2e0 2024-04-25T10:15:46.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.295+0000 7f6601600640 1 -- 172.21.15.87:0/1546451044 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f65f0055c20 con 0x7f66040cd2e0 2024-04-25T10:15:46.302 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f65e4066f60 msgr2=0x7f65e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f65e4066f60 0x7f65e4069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65f80af000 tx=0x7f65f8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040cd2e0 msgr2=0x7f66040b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040cd2e0 0x7f66040b50d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c410 tx=0x7f65f000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 shutdown_connections 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f65e4066f60 0x7f65e4069420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66040cfd90 0x7f66040b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66040cd2e0 0x7f66040b50d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 --2- 172.21.15.87:0/1546451044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66040b0f70 0x7f6604157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 >> 172.21.15.87:0/1546451044 conn(0x7f66040b1d50 msgr2=0x7f66040b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:46.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 shutdown_connections 2024-04-25T10:15:46.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:46.299+0000 7f660c400640 1 -- 172.21.15.87:0/1546451044 wait complete. 2024-04-25T10:15:46.323 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:15:46.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:46.323 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:47.248 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:15:47.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:47.249 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:15:47.257 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:15:47.257 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:47.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:47.267 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:47.267 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:47.311 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:47.311 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:15:47.311 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:47.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:47.320 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:47.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa56e400640 1 Processor -- start 2024-04-25T10:15:47.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa56e400640 1 -- start start 2024-04-25T10:15:47.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa56e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 0x7fa5680cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa56e400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5680cfe70 0x7fa56815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa56e400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa568071050 con 0x7fa56815b740 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa56e400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56815d040 con 0x7fa5680cfe70 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa56e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5680712f0 con 0x7fa5681550b0 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa567e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa56815b740 0x7fa56806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40158/0 (socket says 172.21.15.87:40158) 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa567e00640 1 -- 172.21.15.87:0/3414721942 learned_addr learned my addr 172.21.15.87:0/3414721942 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605107899 0 0) 0x7fa568071050 con 0x7fa56815b740 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.487+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa54c003530 con 0x7fa56815b740 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566a00640 1 --2- 172.21.15.87:0/3414721942 >> v2:172.21.15.169:3300/0 conn(0x7fa5680cfe70 0x7fa56815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 0x7fa5680cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 419455601 0 0) 0x7fa54c003530 con 0x7fa56815b740 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 msgr2=0x7fa5680cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 0x7fa5680cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 >> v2:172.21.15.169:3300/0 conn(0x7fa5680cfe70 msgr2=0x7fa56815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 --2- 172.21.15.87:0/3414721942 >> v2:172.21.15.169:3300/0 conn(0x7fa5680cfe70 0x7fa56815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680715d0 con 0x7fa56815b740 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fa560001a00 con 0x7fa56815b740 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 >> v1:172.21.15.87:6789/0 conn(0x7fa56815b740 legacy=0x7fa56806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa54c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 0x7fa54c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa54c004d50 0x7fa54c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56815d040 con 0x7fa54c0039c0 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5680712f0 con 0x7fa54c004d50 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c00ba80 con 0x7fa5681550b0 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa54c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567e00640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa54c004d50 0x7fa54c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa566a00640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 0x7fa54c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 -- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 msgr2=0x7fa54c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5681550b0 0x7fa54c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 -- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa54c004d50 msgr2=0x7fa54c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa54c004d50 0x7fa54c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 -- 172.21.15.87:0/3414721942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c003530 con 0x7fa54c0039c0 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567e00640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa54c004d50 0x7fa54c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.491+0000 7fa567400640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa54c003de0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa55800ba60 tx=0x7fa55801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b6bea32261de88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580b3070 con 0x7fa54c0039c0 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5580ae410 con 0x7fa54c0039c0 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa566000640 1 -- 172.21.15.87:0/3414721942 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55800b600 con 0x7fa54c0039c0 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- 172.21.15.87:0/3414721942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 msgr2=0x7fa54c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 --2- 172.21.15.87:0/3414721942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa54c003de0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa55800ba60 tx=0x7fa55801c680 comp rx=0 tx=0).stop 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa567e00640 1 -- 172.21.15.87:0/3414721942 reap_dead start 2024-04-25T10:15:47.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- 172.21.15.87:0/3414721942 shutdown_connections 2024-04-25T10:15:47.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- 172.21.15.87:0/3414721942 >> 172.21.15.87:0/3414721942 conn(0x7fa568066b70 msgr2=0x7fa568066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:47.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- 172.21.15.87:0/3414721942 shutdown_connections 2024-04-25T10:15:47.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- 172.21.15.87:0/3414721942 wait complete. 2024-04-25T10:15:47.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 Processor -- start 2024-04-25T10:15:47.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- start start 2024-04-25T10:15:47.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa56815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa568156220 0x7fa568155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5681566a0 0x7fa568145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56815bcd0 con 0x7fa54c0039c0 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56815bb50 con 0x7fa5681566a0 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa56e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56815c030 con 0x7fa568156220 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa567400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa56815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa567400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa56815a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54262/0 (socket says 172.21.15.87:54262) 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.495+0000 7fa567400640 1 -- 172.21.15.87:0/1412459337 learned_addr learned my addr 172.21.15.87:0/1412459337 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa567e00640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5681566a0 0x7fa568145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa566a00640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa568156220 0x7fa568155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa567400640 1 -- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa568156220 msgr2=0x7fa568155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa567400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa568156220 0x7fa568155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa567400640 1 -- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5681566a0 msgr2=0x7fa568145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa567400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5681566a0 0x7fa568145d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa567400640 1 -- 172.21.15.87:0/1412459337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56815aae0 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa567400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa56815a5a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa5580b1040 tx=0x7fa54c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa55f400640 1 -- 172.21.15.87:0/1412459337 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580b3040 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa55f400640 1 -- 172.21.15.87:0/1412459337 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5580ae5c0 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa55f400640 1 -- 172.21.15.87:0/1412459337 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580aee60 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa568159500 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa568159ac0 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56804ef30 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa55f400640 1 -- 172.21.15.87:0/1412459337 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5580c0020 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa55f400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f30 0x7fa5480693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa55f400640 1 -- 172.21.15.87:0/1412459337 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa55811eff0 con 0x7fa54c0039c0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.499+0000 7fa566a00640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f30 0x7fa5480693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.503+0000 7fa566a00640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f30 0x7fa5480693f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa568159040 tx=0x7fa554008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.503+0000 7fa55f400640 1 -- 172.21.15.87:0/1412459337 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa558016390 con 0x7fa54c0039c0 2024-04-25T10:15:47.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5680cf4c0 con 0x7fa54c0039c0 2024-04-25T10:15:47.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa55f400640 1 -- 172.21.15.87:0/1412459337 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fa5580f00a0 con 0x7fa54c0039c0 2024-04-25T10:15:47.700 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:47.700 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:43.370279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:47.700 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 28 2024-04-25T10:15:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f30 msgr2=0x7fa5480693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f30 0x7fa5480693f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa568159040 tx=0x7fa554008040 comp rx=0 tx=0).stop 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 msgr2=0x7fa56815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa56815a5a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa5580b1040 tx=0x7fa54c005840 comp rx=0 tx=0).stop 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 shutdown_connections 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f30 0x7fa5480693f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5681566a0 0x7fa568145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa568156220 0x7fa568155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 --2- 172.21.15.87:0/1412459337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54c0039c0 0x7fa56815a5a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 >> 172.21.15.87:0/1412459337 conn(0x7fa568066b70 msgr2=0x7fa56815d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 shutdown_connections 2024-04-25T10:15:47.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.695+0000 7fa56e400640 1 -- 172.21.15.87:0/1412459337 wait complete. 2024-04-25T10:15:47.722 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a7400640 1 Processor -- start 2024-04-25T10:15:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a7400640 1 -- start start 2024-04-25T10:15:47.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a01550b0 0x7f64a00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a7400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64a015b910 0x7f64a00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a7400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a0071050 con 0x7f64a00cfe70 2024-04-25T10:15:47.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a7400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a015bda0 con 0x7f64a015b910 2024-04-25T10:15:47.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a7400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a015d040 con 0x7f64a01550b0 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f649fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f64a00cfe70 0x7f64a015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40164/0 (socket says 172.21.15.87:40164) 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f649fe00640 1 -- 172.21.15.87:0/2216949886 learned_addr learned my addr 172.21.15.87:0/2216949886 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a5400640 1 --2- 172.21.15.87:0/2216949886 >> v2:172.21.15.169:3300/0 conn(0x7f64a015b910 0x7f64a00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f64a4a00640 1 --2- 172.21.15.87:0/2216949886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a01550b0 0x7f64a00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.851+0000 7f649f400640 1 -- 172.21.15.87:0/2216949886 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843842478 0 0) 0x7f64a0071050 con 0x7f64a00cfe70 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f649f400640 1 -- 172.21.15.87:0/2216949886 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6484005740 con 0x7f64a00cfe70 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f649f400640 1 -- 172.21.15.87:0/2216949886 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1539171912 0 0) 0x7f6484005740 con 0x7f64a00cfe70 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f64a4a00640 1 -- 172.21.15.87:0/2216949886 >> v2:172.21.15.169:3300/0 conn(0x7f64a015b910 msgr2=0x7f64a00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f64a4a00640 1 --2- 172.21.15.87:0/2216949886 >> v2:172.21.15.169:3300/0 conn(0x7f64a015b910 0x7f64a00b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f64a4a00640 1 -- 172.21.15.87:0/2216949886 >> v1:172.21.15.87:6789/0 conn(0x7f64a00cfe70 legacy=0x7f64a015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f64a4a00640 1 -- 172.21.15.87:0/2216949886 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00712f0 con 0x7f64a01550b0 2024-04-25T10:15:47.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f64a4a00640 1 --2- 172.21.15.87:0/2216949886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a01550b0 0x7f64a00cf930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6490009d90 tx=0x7f649001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f98f371c717bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f649f400640 1 -- 172.21.15.87:0/2216949886 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64900b2070 con 0x7f64a01550b0 2024-04-25T10:15:47.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f649f400640 1 -- 172.21.15.87:0/2216949886 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64900a5580 con 0x7f64a01550b0 2024-04-25T10:15:47.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f649f400640 1 -- 172.21.15.87:0/2216949886 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c001680 con 0x7f64a01550b0 2024-04-25T10:15:47.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f64a7400640 1 -- 172.21.15.87:0/2216949886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a01550b0 msgr2=0x7f64a00cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.855+0000 7f64a7400640 1 --2- 172.21.15.87:0/2216949886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a01550b0 0x7f64a00cf930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6490009d90 tx=0x7f649001c890 comp rx=0 tx=0).stop 2024-04-25T10:15:47.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- 172.21.15.87:0/2216949886 shutdown_connections 2024-04-25T10:15:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 --2- 172.21.15.87:0/2216949886 >> v2:172.21.15.169:3300/0 conn(0x7f64a015b910 0x7f64a00b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 --2- 172.21.15.87:0/2216949886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a01550b0 0x7f64a00cf930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- 172.21.15.87:0/2216949886 >> 172.21.15.87:0/2216949886 conn(0x7f64a0066b70 msgr2=0x7f64a0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- 172.21.15.87:0/2216949886 shutdown_connections 2024-04-25T10:15:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- 172.21.15.87:0/2216949886 wait complete. 2024-04-25T10:15:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 Processor -- start 2024-04-25T10:15:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- start start 2024-04-25T10:15:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a00cfe70 0x7f64a014b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a01550b0 0x7f64a0155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a015b910 0x7f64a0157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a015c030 con 0x7f64a00cfe70 2024-04-25T10:15:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00720b0 con 0x7f64a01550b0 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a015c1b0 con 0x7f64a015b910 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a00cfe70 0x7f64a014b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a00cfe70 0x7f64a014b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54276/0 (socket says 172.21.15.87:54276) 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 -- 172.21.15.87:0/1389726384 learned_addr learned my addr 172.21.15.87:0/1389726384 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f649fe00640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a01550b0 0x7f64a0155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a5400640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a015b910 0x7f64a0157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 -- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a015b910 msgr2=0x7f64a0157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a015b910 0x7f64a0157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 -- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a01550b0 msgr2=0x7f64a0155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a01550b0 0x7f64a0155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 -- 172.21.15.87:0/1389726384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00b25b0 con 0x7f64a00cfe70 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a4a00640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a00cfe70 0x7f64a014b460 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f649001af40 tx=0x7f6490019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f649cc00640 1 -- 172.21.15.87:0/1389726384 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64900b2040 con 0x7f64a00cfe70 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f649cc00640 1 -- 172.21.15.87:0/1389726384 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64900bf070 con 0x7f64a00cfe70 2024-04-25T10:15:47.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f649cc00640 1 -- 172.21.15.87:0/1389726384 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490009450 con 0x7f64a00cfe70 2024-04-25T10:15:47.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.859+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a00b4320 con 0x7f64a00cfe70 2024-04-25T10:15:47.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.863+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a014f5b0 con 0x7f64a00cfe70 2024-04-25T10:15:47.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.863+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a004ef30 con 0x7f64a00cfe70 2024-04-25T10:15:47.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.863+0000 7f649cc00640 1 -- 172.21.15.87:0/1389726384 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64900b88f0 con 0x7f64a00cfe70 2024-04-25T10:15:47.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.863+0000 7f649cc00640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6480066f90 0x7f6480069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.863+0000 7f649cc00640 1 -- 172.21.15.87:0/1389726384 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f649011ff90 con 0x7f64a00cfe70 2024-04-25T10:15:47.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.863+0000 7f649fe00640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6480066f90 0x7f6480069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.863+0000 7f649fe00640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6480066f90 0x7f6480069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f648c000fc0 tx=0x7f648c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:47.867+0000 7f649cc00640 1 -- 172.21.15.87:0/1389726384 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6490016480 con 0x7f64a00cfe70 2024-04-25T10:15:48.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.023+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64a006b390 con 0x7f64a00cfe70 2024-04-25T10:15:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.023+0000 7f649cc00640 1 -- 172.21.15.87:0/1389726384 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f64900f0f00 con 0x7f64a00cfe70 2024-04-25T10:15:48.031 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:48.031 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:08.327470+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:05.052743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:05.692130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:48.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6480066f90 msgr2=0x7f6480069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6480066f90 0x7f6480069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f648c000fc0 tx=0x7f648c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:48.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a00cfe70 msgr2=0x7f64a014b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a00cfe70 0x7f64a014b460 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f649001af40 tx=0x7f6490019630 comp rx=0 tx=0).stop 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 shutdown_connections 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6480066f90 0x7f6480069450 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a015b910 0x7f64a0157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a01550b0 0x7f64a0155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 --2- 172.21.15.87:0/1389726384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a00cfe70 0x7f64a014b460 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 >> 172.21.15.87:0/1389726384 conn(0x7f64a0066b70 msgr2=0x7f64a00af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 shutdown_connections 2024-04-25T10:15:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.027+0000 7f64a7400640 1 -- 172.21.15.87:0/1389726384 wait complete. 2024-04-25T10:15:48.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:05.052743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:05.692130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:48.052 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:48.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7849200640 1 Processor -- start 2024-04-25T10:15:48.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7849200640 1 -- start start 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7849200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78440b3480 0x7f784415e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7849200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78440cfb50 0x7f7844070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7849200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78440ad840 con 0x7f7844071bc0 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7849200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784415bb90 con 0x7f78440cfb50 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7849200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784415e710 con 0x7f78440b3480 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7841400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78440cfb50 0x7f7844070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7842800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7844071bc0 0x7f78440add20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40176/0 (socket says 172.21.15.87:40176) 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7842800640 1 -- 172.21.15.87:0/112734223 learned_addr learned my addr 172.21.15.87:0/112734223 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:48.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7841e00640 1 --2- 172.21.15.87:0/112734223 >> v2:172.21.15.169:3300/0 conn(0x7f78440b3480 0x7f784415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7841400640 1 -- 172.21.15.87:0/112734223 >> v2:172.21.15.169:3300/0 conn(0x7f78440b3480 msgr2=0x7f784415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7841400640 1 --2- 172.21.15.87:0/112734223 >> v2:172.21.15.169:3300/0 conn(0x7f78440b3480 0x7f784415e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7841400640 1 -- 172.21.15.87:0/112734223 >> v1:172.21.15.87:6789/0 conn(0x7f7844071bc0 legacy=0x7f78440add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7841400640 1 -- 172.21.15.87:0/112734223 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784415e9f0 con 0x7f78440cfb50 2024-04-25T10:15:48.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.187+0000 7f7840a00640 1 -- 172.21.15.87:0/112734223 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871402164 0 0) 0x7f78440ad840 con 0x7f7844071bc0 2024-04-25T10:15:48.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7841400640 1 --2- 172.21.15.87:0/112734223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78440cfb50 0x7f7844070350 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f784406e5e0 tx=0x7f783001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4ec8f3b39d70df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7840a00640 1 -- 172.21.15.87:0/112734223 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300b0070 con 0x7f78440cfb50 2024-04-25T10:15:48.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7840a00640 1 -- 172.21.15.87:0/112734223 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78300ab600 con 0x7f78440cfb50 2024-04-25T10:15:48.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7840a00640 1 -- 172.21.15.87:0/112734223 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300ab9d0 con 0x7f78440cfb50 2024-04-25T10:15:48.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 -- 172.21.15.87:0/112734223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78440cfb50 msgr2=0x7f7844070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 --2- 172.21.15.87:0/112734223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78440cfb50 0x7f7844070350 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f784406e5e0 tx=0x7f783001c480 comp rx=0 tx=0).stop 2024-04-25T10:15:48.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 -- 172.21.15.87:0/112734223 shutdown_connections 2024-04-25T10:15:48.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 --2- 172.21.15.87:0/112734223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78440cfb50 0x7f7844070350 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 --2- 172.21.15.87:0/112734223 >> v2:172.21.15.169:3300/0 conn(0x7f78440b3480 0x7f784415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 -- 172.21.15.87:0/112734223 >> 172.21.15.87:0/112734223 conn(0x7f78440669a0 msgr2=0x7f7844066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 -- 172.21.15.87:0/112734223 shutdown_connections 2024-04-25T10:15:48.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.191+0000 7f7849200640 1 -- 172.21.15.87:0/112734223 wait complete. 2024-04-25T10:15:48.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 Processor -- start 2024-04-25T10:15:48.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 -- start start 2024-04-25T10:15:48.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7844071bc0 0x7f784414a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78440b3480 0x7f784414a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440cfb50 0x7f78441591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844072400 con 0x7f7844071bc0 2024-04-25T10:15:48.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78440ae840 con 0x7f78440b3480 2024-04-25T10:15:48.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7844070890 con 0x7f78440cfb50 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7844071bc0 0x7f784414a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7844071bc0 0x7f784414a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54300/0 (socket says 172.21.15.87:54300) 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 -- 172.21.15.87:0/2821465411 learned_addr learned my addr 172.21.15.87:0/2821465411 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7842800640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440cfb50 0x7f78441591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 -- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440cfb50 msgr2=0x7f78441591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841400640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78440b3480 0x7f784414a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440cfb50 0x7f78441591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 -- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78440b3480 msgr2=0x7f784414a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78440b3480 0x7f784414a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 -- 172.21.15.87:0/2821465411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784414aed0 con 0x7f7844071bc0 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7842800640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440cfb50 0x7f78441591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841400640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78440b3480 0x7f784414a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7841e00640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7844071bc0 0x7f784414a450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f783400e930 tx=0x7f783400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f783a000640 1 -- 172.21.15.87:0/2821465411 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834017070 con 0x7f7844071bc0 2024-04-25T10:15:48.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f783a000640 1 -- 172.21.15.87:0/2821465411 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783400ca00 con 0x7f7844071bc0 2024-04-25T10:15:48.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f783a000640 1 -- 172.21.15.87:0/2821465411 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783400cdb0 con 0x7f7844071bc0 2024-04-25T10:15:48.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784415a040 con 0x7f7844071bc0 2024-04-25T10:15:48.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7844156950 con 0x7f7844071bc0 2024-04-25T10:15:48.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.195+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784404ef30 con 0x7f7844071bc0 2024-04-25T10:15:48.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.199+0000 7f783a000640 1 -- 172.21.15.87:0/2821465411 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f783401a070 con 0x7f7844071bc0 2024-04-25T10:15:48.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.199+0000 7f783a000640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7824066f60 0x7f7824069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.199+0000 7f783a000640 1 -- 172.21.15.87:0/2821465411 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7834084fd0 con 0x7f7844071bc0 2024-04-25T10:15:48.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.199+0000 7f7841400640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7824066f60 0x7f7824069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.199+0000 7f7841400640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7824066f60 0x7f7824069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f78300ad000 tx=0x7f78300a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.203+0000 7f783a000640 1 -- 172.21.15.87:0/2821465411 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f784414aed0 con 0x7f7844071bc0 2024-04-25T10:15:48.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.355+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f784406b390 con 0x7f7844071bc0 2024-04-25T10:15:48.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f783a000640 1 -- 172.21.15.87:0/2821465411 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7834056130 con 0x7f7844071bc0 2024-04-25T10:15:48.364 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7824066f60 msgr2=0x7f7824069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7824066f60 0x7f7824069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f78300ad000 tx=0x7f78300a6000 comp rx=0 tx=0).stop 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7844071bc0 msgr2=0x7f784414a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7844071bc0 0x7f784414a450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f783400e930 tx=0x7f783400edf0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 shutdown_connections 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7824066f60 0x7f7824069420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440cfb50 0x7f78441591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78440b3480 0x7f784414a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 --2- 172.21.15.87:0/2821465411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7844071bc0 0x7f784414a450 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 >> 172.21.15.87:0/2821465411 conn(0x7f78440669a0 msgr2=0x7f784415b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 shutdown_connections 2024-04-25T10:15:48.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.359+0000 7f7849200640 1 -- 172.21.15.87:0/2821465411 wait complete. 2024-04-25T10:15:48.384 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.515+0000 7fe265400640 1 Processor -- start 2024-04-25T10:15:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.515+0000 7fe265400640 1 -- start start 2024-04-25T10:15:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.515+0000 7fe265400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe260070350 0x7fe26015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.515+0000 7fe265400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26015b080 con 0x7fe2600b31c0 2024-04-25T10:15:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.515+0000 7fe265400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26015adb0 con 0x7fe2600aef80 2024-04-25T10:15:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.515+0000 7fe265400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26015eb60 con 0x7fe260070350 2024-04-25T10:15:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe255e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe260070350 0x7fe26015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe25f200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe2600b31c0 0x7fe2600cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40192/0 (socket says 172.21.15.87:40192) 2024-04-25T10:15:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe25f200640 1 -- 172.21.15.87:0/366293780 learned_addr learned my addr 172.21.15.87:0/366293780 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:48.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe255e00640 1 -- 172.21.15.87:0/366293780 >> v1:172.21.15.169:6789/0 conn(0x7fe2600aef80 legacy=0x7fe26015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe255e00640 1 -- 172.21.15.87:0/366293780 >> v1:172.21.15.87:6789/0 conn(0x7fe2600b31c0 legacy=0x7fe2600cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe255e00640 1 -- 172.21.15.87:0/366293780 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26015ee40 con 0x7fe260070350 2024-04-25T10:15:48.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe25de00640 1 -- 172.21.15.87:0/366293780 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756979328 0 0) 0x7fe26015b080 con 0x7fe2600b31c0 2024-04-25T10:15:48.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe255e00640 1 --2- 172.21.15.87:0/366293780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015e480 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe2480097c0 tx=0x7fe24801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6fb302fee7196c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe25de00640 1 -- 172.21.15.87:0/366293780 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480b2070 con 0x7fe260070350 2024-04-25T10:15:48.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe25de00640 1 -- 172.21.15.87:0/366293780 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2480ad680 con 0x7fe260070350 2024-04-25T10:15:48.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.519+0000 7fe25de00640 1 -- 172.21.15.87:0/366293780 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480ada50 con 0x7fe260070350 2024-04-25T10:15:48.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- 172.21.15.87:0/366293780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 msgr2=0x7fe26015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 --2- 172.21.15.87:0/366293780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015e480 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe2480097c0 tx=0x7fe24801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- 172.21.15.87:0/366293780 shutdown_connections 2024-04-25T10:15:48.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 --2- 172.21.15.87:0/366293780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015e480 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- 172.21.15.87:0/366293780 >> 172.21.15.87:0/366293780 conn(0x7fe2600669a0 msgr2=0x7fe260066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- 172.21.15.87:0/366293780 shutdown_connections 2024-04-25T10:15:48.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- 172.21.15.87:0/366293780 wait complete. 2024-04-25T10:15:48.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 Processor -- start 2024-04-25T10:15:48.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- start start 2024-04-25T10:15:48.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2600aef80 0x7fe260150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2600b31c0 0x7fe2601509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600cfef0 con 0x7fe260070350 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600cfd70 con 0x7fe2600aef80 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe265400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600b35d0 con 0x7fe2600b31c0 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54318/0 (socket says 172.21.15.87:54318) 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 -- 172.21.15.87:0/2547616565 learned_addr learned my addr 172.21.15.87:0/2547616565 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe255e00640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2600aef80 0x7fe260150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25f200640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2600b31c0 0x7fe2601509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 -- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2600b31c0 msgr2=0x7fe2601509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2600b31c0 0x7fe2601509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 -- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2600aef80 msgr2=0x7fe260150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2600aef80 0x7fe260150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.523+0000 7fe25e800640 1 -- 172.21.15.87:0/2547616565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2601693f0 con 0x7fe260070350 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe25e800640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015db10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c500 tx=0x7fe25000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe257400640 1 -- 172.21.15.87:0/2547616565 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe250016020 con 0x7fe260070350 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe257400640 1 -- 172.21.15.87:0/2547616565 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe250005220 con 0x7fe260070350 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe257400640 1 -- 172.21.15.87:0/2547616565 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe250007030 con 0x7fe260070350 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2601696d0 con 0x7fe260070350 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe260169c90 con 0x7fe260070350 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26004ef30 con 0x7fe260070350 2024-04-25T10:15:48.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe257400640 1 -- 172.21.15.87:0/2547616565 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2500071f0 con 0x7fe260070350 2024-04-25T10:15:48.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe257400640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe23c066f60 0x7fe23c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe257400640 1 -- 172.21.15.87:0/2547616565 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2500849c0 con 0x7fe260070350 2024-04-25T10:15:48.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe255e00640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe23c066f60 0x7fe23c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.527+0000 7fe255e00640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe23c066f60 0x7fe23c069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2480af000 tx=0x7fe2480a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.531+0000 7fe257400640 1 -- 172.21.15.87:0/2547616565 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe250020020 con 0x7fe260070350 2024-04-25T10:15:48.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2600b2fa0 con 0x7fe260070350 2024-04-25T10:15:48.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe257400640 1 -- 172.21.15.87:0/2547616565 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2500559c0 con 0x7fe260070350 2024-04-25T10:15:48.692 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:15:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe23c066f60 msgr2=0x7fe23c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe23c066f60 0x7fe23c069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2480af000 tx=0x7fe2480a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 msgr2=0x7fe26015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015db10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c500 tx=0x7fe25000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 shutdown_connections 2024-04-25T10:15:48.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe23c066f60 0x7fe23c069420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2600b31c0 0x7fe2601509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2600aef80 0x7fe260150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 --2- 172.21.15.87:0/2547616565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe260070350 0x7fe26015db10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 >> 172.21.15.87:0/2547616565 conn(0x7fe2600669a0 msgr2=0x7fe2600cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 shutdown_connections 2024-04-25T10:15:48.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.687+0000 7fe265400640 1 -- 172.21.15.87:0/2547616565 wait complete. 2024-04-25T10:15:48.712 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:48.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.843+0000 7f2541600640 1 Processor -- start 2024-04-25T10:15:48.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.843+0000 7f2541600640 1 -- start start 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f2541600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f253c15b750 0x7f253c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f2541600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c0b4690 0x7f253c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f2541600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c0b2460 con 0x7f253c0cd590 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f2541600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0b2d70 con 0x7f253c15b750 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f2541600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c070fb0 con 0x7f253c0b4690 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f253c15b750 0x7f253c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f253c15b750 0x7f253c0cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54320/0 (socket says 172.21.15.87:54320) 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 -- 172.21.15.87:0/665136310 learned_addr learned my addr 172.21.15.87:0/665136310 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 -- 172.21.15.87:0/665136310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c0b4690 msgr2=0x7f253c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 --2- 172.21.15.87:0/665136310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c0b4690 0x7f253c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 -- 172.21.15.87:0/665136310 >> v1:172.21.15.169:6789/0 conn(0x7f253c0cd590 legacy=0x7f253c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 -- 172.21.15.87:0/665136310 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0b2720 con 0x7f253c15b750 2024-04-25T10:15:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.847+0000 7f253be00640 1 --2- 172.21.15.87:0/665136310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c15b750 0x7f253c0cbc30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f253c067980 tx=0x7f252c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f95cd6e496d4f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253aa00640 1 -- 172.21.15.87:0/665136310 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c0a57e0 con 0x7f253c15b750 2024-04-25T10:15:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253aa00640 1 -- 172.21.15.87:0/665136310 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c0a5980 con 0x7f253c15b750 2024-04-25T10:15:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253aa00640 1 -- 172.21.15.87:0/665136310 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c0a5d50 con 0x7f253c15b750 2024-04-25T10:15:48.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- 172.21.15.87:0/665136310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c15b750 msgr2=0x7f253c0cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 --2- 172.21.15.87:0/665136310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c15b750 0x7f253c0cbc30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f253c067980 tx=0x7f252c01c9a0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- 172.21.15.87:0/665136310 shutdown_connections 2024-04-25T10:15:48.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 --2- 172.21.15.87:0/665136310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c0b4690 0x7f253c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 --2- 172.21.15.87:0/665136310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c15b750 0x7f253c0cbc30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- 172.21.15.87:0/665136310 >> 172.21.15.87:0/665136310 conn(0x7f253c0743f0 msgr2=0x7f253c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- 172.21.15.87:0/665136310 shutdown_connections 2024-04-25T10:15:48.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- 172.21.15.87:0/665136310 wait complete. 2024-04-25T10:15:48.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 Processor -- start 2024-04-25T10:15:48.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- start start 2024-04-25T10:15:48.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f253c0b4690 0x7f253c150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c0cd590 0x7f253c155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c15b750 0x7f253c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c070ce0 con 0x7f253c0cd590 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c070b60 con 0x7f253c0b4690 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2541600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c168760 con 0x7f253c15b750 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c0cd590 0x7f253c155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c0cd590 0x7f253c155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54326/0 (socket says 172.21.15.87:54326) 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 -- 172.21.15.87:0/521174947 learned_addr learned my addr 172.21.15.87:0/521174947 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f2540c00640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c15b750 0x7f253c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253be00640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f253c0b4690 0x7f253c150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 -- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c15b750 msgr2=0x7f253c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c15b750 0x7f253c155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 -- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f253c0b4690 msgr2=0x7f253c150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f253c0b4690 0x7f253c150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.851+0000 7f253b400640 1 -- 172.21.15.87:0/521174947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c14eff0 con 0x7f253c0cd590 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f253b400640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c0cd590 0x7f253c155660 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f252800c4c0 tx=0x7f2528001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2533e00640 1 -- 172.21.15.87:0/521174947 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528017070 con 0x7f253c0cd590 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2533e00640 1 -- 172.21.15.87:0/521174947 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528003600 con 0x7f253c0cd590 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2533e00640 1 -- 172.21.15.87:0/521174947 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280039f0 con 0x7f253c0cd590 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c14f2d0 con 0x7f253c0cd590 2024-04-25T10:15:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c14f890 con 0x7f253c0cd590 2024-04-25T10:15:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c075120 con 0x7f253c0cd590 2024-04-25T10:15:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2533e00640 1 -- 172.21.15.87:0/521174947 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2528003b90 con 0x7f253c0cd590 2024-04-25T10:15:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2533e00640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f2533e00640 1 -- 172.21.15.87:0/521174947 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25280842f0 con 0x7f253c0cd590 2024-04-25T10:15:48.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.855+0000 7f253be00640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.859+0000 7f253be00640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f252c0ae000 tx=0x7f252c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:48.859+0000 7f2533e00640 1 -- 172.21.15.87:0/521174947 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2528055260 con 0x7f253c0cd590 2024-04-25T10:15:49.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.011+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f253c071990 con 0x7f253c0cd590 2024-04-25T10:15:49.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2533e00640 1 -- 172.21.15.87:0/521174947 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2528054b60 con 0x7f253c0cd590 2024-04-25T10:15:49.020 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f251c066f60 msgr2=0x7f251c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f252c0ae000 tx=0x7f252c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c0cd590 msgr2=0x7f253c155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c0cd590 0x7f253c155660 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f252800c4c0 tx=0x7f2528001e80 comp rx=0 tx=0).stop 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 shutdown_connections 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f253c15b750 0x7f253c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f253c0cd590 0x7f253c155660 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 --2- 172.21.15.87:0/521174947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f253c0b4690 0x7f253c150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 >> 172.21.15.87:0/521174947 conn(0x7f253c0743f0 msgr2=0x7f253c15e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:49.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 shutdown_connections 2024-04-25T10:15:49.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.015+0000 7f2541600640 1 -- 172.21.15.87:0/521174947 wait complete. 2024-04-25T10:15:49.041 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2517200640 1 Processor -- start 2024-04-25T10:15:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2517200640 1 -- start start 2024-04-25T10:15:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2517200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251015dc10 0x7f25100d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2517200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f251006bdb0 0x7f25100ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2517200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25100cbff0 con 0x7f251015dff0 2024-04-25T10:15:49.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2517200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25100b45a0 con 0x7f251006bdb0 2024-04-25T10:15:49.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2517200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25100cbda0 con 0x7f251015dc10 2024-04-25T10:15:49.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2515400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f251015dff0 0x7f251006b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40204/0 (socket says 172.21.15.87:40204) 2024-04-25T10:15:49.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2515400640 1 -- 172.21.15.87:0/3791323571 learned_addr learned my addr 172.21.15.87:0/3791323571 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:49.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2515e00640 1 --2- 172.21.15.87:0/3791323571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251015dc10 0x7f25100d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2516800640 1 --2- 172.21.15.87:0/3791323571 >> v2:172.21.15.169:3300/0 conn(0x7f251006bdb0 0x7f25100ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2514a00640 1 -- 172.21.15.87:0/3791323571 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157729003 0 0) 0x7f25100cbff0 con 0x7f251015dff0 2024-04-25T10:15:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2514a00640 1 -- 172.21.15.87:0/3791323571 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f8003530 con 0x7f251015dff0 2024-04-25T10:15:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2515e00640 1 -- 172.21.15.87:0/3791323571 >> v2:172.21.15.169:3300/0 conn(0x7f251006bdb0 msgr2=0x7f25100ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2515e00640 1 --2- 172.21.15.87:0/3791323571 >> v2:172.21.15.169:3300/0 conn(0x7f251006bdb0 0x7f25100ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2515e00640 1 -- 172.21.15.87:0/3791323571 >> v1:172.21.15.87:6789/0 conn(0x7f251015dff0 legacy=0x7f251006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2514a00640 1 -- 172.21.15.87:0/3791323571 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 239802863 0 0) 0x7f24f8003530 con 0x7f251015dff0 2024-04-25T10:15:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.171+0000 7f2515e00640 1 -- 172.21.15.87:0/3791323571 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100cc2d0 con 0x7f251015dc10 2024-04-25T10:15:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2515e00640 1 --2- 172.21.15.87:0/3791323571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251015dc10 0x7f25100d04a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25040097c0 tx=0x7f250401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb9c2e7706f2e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2514a00640 1 -- 172.21.15.87:0/3791323571 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25040b0070 con 0x7f251015dc10 2024-04-25T10:15:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2514a00640 1 -- 172.21.15.87:0/3791323571 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25040ab400 con 0x7f251015dc10 2024-04-25T10:15:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2514a00640 1 -- 172.21.15.87:0/3791323571 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500001680 con 0x7f251015dc10 2024-04-25T10:15:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 -- 172.21.15.87:0/3791323571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251015dc10 msgr2=0x7f25100d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 --2- 172.21.15.87:0/3791323571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251015dc10 0x7f25100d04a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25040097c0 tx=0x7f250401c390 comp rx=0 tx=0).stop 2024-04-25T10:15:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 -- 172.21.15.87:0/3791323571 shutdown_connections 2024-04-25T10:15:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 --2- 172.21.15.87:0/3791323571 >> v2:172.21.15.169:3300/0 conn(0x7f251006bdb0 0x7f25100ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 --2- 172.21.15.87:0/3791323571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251015dc10 0x7f25100d04a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 -- 172.21.15.87:0/3791323571 >> 172.21.15.87:0/3791323571 conn(0x7f25100d1750 msgr2=0x7f25100d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 -- 172.21.15.87:0/3791323571 shutdown_connections 2024-04-25T10:15:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.175+0000 7f2517200640 1 -- 172.21.15.87:0/3791323571 wait complete. 2024-04-25T10:15:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 Processor -- start 2024-04-25T10:15:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 -- start start 2024-04-25T10:15:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251006bdb0 0x7f25100ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251015dc10 0x7f2510147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f251015dff0 0x7f251015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100cdc80 con 0x7f251015dff0 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100cdb00 con 0x7f251015dc10 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25100cde00 con 0x7f251006bdb0 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f251015dff0 0x7f251015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f251015dff0 0x7f251015c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54334/0 (socket says 172.21.15.87:54334) 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 -- 172.21.15.87:0/4079936085 learned_addr learned my addr 172.21.15.87:0/4079936085 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2515e00640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251006bdb0 0x7f25100ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2515400640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251015dc10 0x7f2510147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 -- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251006bdb0 msgr2=0x7f25100ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251006bdb0 0x7f25100ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 -- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251015dc10 msgr2=0x7f2510147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251015dc10 0x7f2510147cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 -- 172.21.15.87:0/4079936085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100caca0 con 0x7f251015dff0 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2515e00640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251006bdb0 0x7f25100ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2516800640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f251015dff0 0x7f251015c000 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f250c00efc0 tx=0x7f250c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f250a000640 1 -- 172.21.15.87:0/4079936085 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c018020 con 0x7f251015dff0 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f250a000640 1 -- 172.21.15.87:0/4079936085 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c014480 con 0x7f251015dff0 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25100ca9c0 con 0x7f251015dff0 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f250a000640 1 -- 172.21.15.87:0/4079936085 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c009e60 con 0x7f251015dff0 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.179+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2510154700 con 0x7f251015dff0 2024-04-25T10:15:49.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.183+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2510075120 con 0x7f251015dff0 2024-04-25T10:15:49.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.183+0000 7f250a000640 1 -- 172.21.15.87:0/4079936085 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f250c01b020 con 0x7f251015dff0 2024-04-25T10:15:49.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.183+0000 7f250a000640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24f4066f60 0x7f24f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.183+0000 7f250a000640 1 -- 172.21.15.87:0/4079936085 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f250c085dd0 con 0x7f251015dff0 2024-04-25T10:15:49.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.183+0000 7f2515e00640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24f4066f60 0x7f24f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.183+0000 7f2515e00640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24f4066f60 0x7f24f4069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f25040ad000 tx=0x7f2504008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.187+0000 7f250a000640 1 -- 172.21.15.87:0/4079936085 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f250c022030 con 0x7f251015dff0 2024-04-25T10:15:49.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.371+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2510156d00 con 0x7f251015dff0 2024-04-25T10:15:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.371+0000 7f250a000640 1 -- 172.21.15.87:0/4079936085 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f250c010040 con 0x7f251015dff0 2024-04-25T10:15:49.379 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:49.380 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:43.370279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:49.382 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 28 2024-04-25T10:15:49.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24f4066f60 msgr2=0x7f24f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24f4066f60 0x7f24f4069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f25040ad000 tx=0x7f2504008040 comp rx=0 tx=0).stop 2024-04-25T10:15:49.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f251015dff0 msgr2=0x7f251015c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f251015dff0 0x7f251015c000 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f250c00efc0 tx=0x7f250c00c490 comp rx=0 tx=0).stop 2024-04-25T10:15:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 shutdown_connections 2024-04-25T10:15:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24f4066f60 0x7f24f4069420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f251015dff0 0x7f251015c000 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251015dc10 0x7f2510147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 --2- 172.21.15.87:0/4079936085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f251006bdb0 0x7f25100ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 >> 172.21.15.87:0/4079936085 conn(0x7f25100d1750 msgr2=0x7f25100d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 shutdown_connections 2024-04-25T10:15:49.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.375+0000 7f2517200640 1 -- 172.21.15.87:0/4079936085 wait complete. 2024-04-25T10:15:49.400 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86fce00640 1 Processor -- start 2024-04-25T10:15:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86fce00640 1 -- start start 2024-04-25T10:15:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86fce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86f815e050 0x7f86f80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86fce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f80cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86fce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f80b1850 con 0x7f86f80d1820 2024-04-25T10:15:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86fce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f80b1af0 con 0x7f86f815e050 2024-04-25T10:15:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86fce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80b1dd0 con 0x7f86f80b13e0 2024-04-25T10:15:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f5e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f86f80d1820 0x7f86f80d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40206/0 (socket says 172.21.15.87:40206) 2024-04-25T10:15:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f5e00640 1 -- 172.21.15.87:0/1446883054 learned_addr learned my addr 172.21.15.87:0/1446883054 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f5400640 1 --2- 172.21.15.87:0/1446883054 >> v2:172.21.15.87:3300/0 conn(0x7f86f815e050 0x7f86f80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f6800640 1 --2- 172.21.15.87:0/1446883054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f80cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f4a00640 1 -- 172.21.15.87:0/1446883054 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558858219 0 0) 0x7f86f80b1850 con 0x7f86f80d1820 2024-04-25T10:15:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f4a00640 1 -- 172.21.15.87:0/1446883054 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86dc003530 con 0x7f86f80d1820 2024-04-25T10:15:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f5400640 1 -- 172.21.15.87:0/1446883054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 msgr2=0x7f86f80cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f5400640 1 --2- 172.21.15.87:0/1446883054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f80cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f5400640 1 -- 172.21.15.87:0/1446883054 >> v1:172.21.15.87:6789/0 conn(0x7f86f80d1820 legacy=0x7f86f80d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.531+0000 7f86f5400640 1 -- 172.21.15.87:0/1446883054 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80b20b0 con 0x7f86f815e050 2024-04-25T10:15:49.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86f5400640 1 --2- 172.21.15.87:0/1446883054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f815e050 0x7f86f80b0ea0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f86e40097c0 tx=0x7f86e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5b976506f53f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86f4a00640 1 -- 172.21.15.87:0/1446883054 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40ab610 con 0x7f86f815e050 2024-04-25T10:15:49.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86f4a00640 1 -- 172.21.15.87:0/1446883054 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e40b0070 con 0x7f86f815e050 2024-04-25T10:15:49.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86f4a00640 1 -- 172.21.15.87:0/1446883054 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40ab9d0 con 0x7f86f815e050 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 -- 172.21.15.87:0/1446883054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f815e050 msgr2=0x7f86f80b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 --2- 172.21.15.87:0/1446883054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f815e050 0x7f86f80b0ea0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f86e40097c0 tx=0x7f86e401c410 comp rx=0 tx=0).stop 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 -- 172.21.15.87:0/1446883054 shutdown_connections 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 --2- 172.21.15.87:0/1446883054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f80cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 --2- 172.21.15.87:0/1446883054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f815e050 0x7f86f80b0ea0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 -- 172.21.15.87:0/1446883054 >> 172.21.15.87:0/1446883054 conn(0x7f86f809a9d0 msgr2=0x7f86f809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 -- 172.21.15.87:0/1446883054 shutdown_connections 2024-04-25T10:15:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.535+0000 7f86fce00640 1 -- 172.21.15.87:0/1446883054 wait complete. 2024-04-25T10:15:49.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 Processor -- start 2024-04-25T10:15:49.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 -- start start 2024-04-25T10:15:49.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f8159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f80d1820 0x7f86f8152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f815e050 0x7f86f8154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80d0dd0 con 0x7f86f80d1820 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80d0c50 con 0x7f86f815e050 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80d0f50 con 0x7f86f80b13e0 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f80d1820 0x7f86f8152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f80d1820 0x7f86f8152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54348/0 (socket says 172.21.15.87:54348) 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 -- 172.21.15.87:0/733277706 learned_addr learned my addr 172.21.15.87:0/733277706 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5e00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f8159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f6800640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f815e050 0x7f86f8154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 -- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 msgr2=0x7f86f8159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f8159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 -- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f815e050 msgr2=0x7f86f8154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f815e050 0x7f86f8154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 -- 172.21.15.87:0/733277706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80b46e0 con 0x7f86f80d1820 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5e00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f8159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86f5400640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f80d1820 0x7f86f8152b20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f86e40ad040 tx=0x7f86e401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86ee000640 1 -- 172.21.15.87:0/733277706 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40bf410 con 0x7f86f80d1820 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86ee000640 1 -- 172.21.15.87:0/733277706 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e40bf5d0 con 0x7f86f80d1820 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86ee000640 1 -- 172.21.15.87:0/733277706 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40bf9c0 con 0x7f86f80d1820 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f80b49c0 con 0x7f86f80d1820 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f80b4f80 con 0x7f86f80d1820 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.539+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f804ef30 con 0x7f86f80d1820 2024-04-25T10:15:49.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.543+0000 7f86ee000640 1 -- 172.21.15.87:0/733277706 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86e40bd020 con 0x7f86f80d1820 2024-04-25T10:15:49.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.543+0000 7f86ee000640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f90 0x7f86d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.543+0000 7f86ee000640 1 -- 172.21.15.87:0/733277706 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86e411f570 con 0x7f86f80d1820 2024-04-25T10:15:49.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.543+0000 7f86f5e00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f90 0x7f86d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.543+0000 7f86f5e00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f90 0x7f86d8069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86f8155410 tx=0x7f86e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.547+0000 7f86ee000640 1 -- 172.21.15.87:0/733277706 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86e40a8050 con 0x7f86f80d1820 2024-04-25T10:15:49.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.703+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86f8159a90 con 0x7f86f80d1820 2024-04-25T10:15:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86ee000640 1 -- 172.21.15.87:0/733277706 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f86e411de20 con 0x7f86f80d1820 2024-04-25T10:15:49.712 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:49.712 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:08.327470+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:05.052743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:05.692130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f90 msgr2=0x7f86d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f90 0x7f86d8069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86f8155410 tx=0x7f86e8009040 comp rx=0 tx=0).stop 2024-04-25T10:15:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f80d1820 msgr2=0x7f86f8152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f80d1820 0x7f86f8152b20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f86e40ad040 tx=0x7f86e401ac50 comp rx=0 tx=0).stop 2024-04-25T10:15:49.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 shutdown_connections 2024-04-25T10:15:49.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f90 0x7f86d8069450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f815e050 0x7f86f8154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86f80d1820 0x7f86f8152b20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 --2- 172.21.15.87:0/733277706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f80b13e0 0x7f86f8159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 >> 172.21.15.87:0/733277706 conn(0x7f86f809a9d0 msgr2=0x7f86f80d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 shutdown_connections 2024-04-25T10:15:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.707+0000 7f86fce00640 1 -- 172.21.15.87:0/733277706 wait complete. 2024-04-25T10:15:49.734 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:15:49.734 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:49.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.863+0000 7f9589c00640 1 Processor -- start 2024-04-25T10:15:49.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.863+0000 7f9589c00640 1 -- start start 2024-04-25T10:15:49.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.863+0000 7f9589c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95841550b0 0x7f95840cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.863+0000 7f9589c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f95840cfe70 0x7f958415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.863+0000 7f9589c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9584071050 con 0x7f958415b740 2024-04-25T10:15:49.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.863+0000 7f9589c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958415d040 con 0x7f95840cfe70 2024-04-25T10:15:49.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.863+0000 7f9589c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95840712f0 con 0x7f95841550b0 2024-04-25T10:15:49.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9583400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f958415b740 0x7f958406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40212/0 (socket says 172.21.15.87:40212) 2024-04-25T10:15:49.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9583400640 1 -- 172.21.15.87:0/318961681 learned_addr learned my addr 172.21.15.87:0/318961681 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:49.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582a00640 1 --2- 172.21.15.87:0/318961681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95841550b0 0x7f95840cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f957a000640 1 --2- 172.21.15.87:0/318961681 >> v2:172.21.15.169:3300/0 conn(0x7f95840cfe70 0x7f958415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582000640 1 -- 172.21.15.87:0/318961681 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352813742 0 0) 0x7f9584071050 con 0x7f958415b740 2024-04-25T10:15:49.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582000640 1 -- 172.21.15.87:0/318961681 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9568003530 con 0x7f958415b740 2024-04-25T10:15:49.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582a00640 1 -- 172.21.15.87:0/318961681 >> v2:172.21.15.169:3300/0 conn(0x7f95840cfe70 msgr2=0x7f958415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:49.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582a00640 1 --2- 172.21.15.87:0/318961681 >> v2:172.21.15.169:3300/0 conn(0x7f95840cfe70 0x7f958415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582a00640 1 -- 172.21.15.87:0/318961681 >> v1:172.21.15.87:6789/0 conn(0x7f958415b740 legacy=0x7f958406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582a00640 1 -- 172.21.15.87:0/318961681 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840715d0 con 0x7f95841550b0 2024-04-25T10:15:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.867+0000 7f9582a00640 1 --2- 172.21.15.87:0/318961681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95841550b0 0x7f95840cf930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f956c0097c0 tx=0x7f956c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ee0dfc51fb5b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9582000640 1 -- 172.21.15.87:0/318961681 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b0070 con 0x7f95841550b0 2024-04-25T10:15:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9582000640 1 -- 172.21.15.87:0/318961681 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c0ab600 con 0x7f95841550b0 2024-04-25T10:15:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9582000640 1 -- 172.21.15.87:0/318961681 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0ab9d0 con 0x7f95841550b0 2024-04-25T10:15:49.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- 172.21.15.87:0/318961681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95841550b0 msgr2=0x7f95840cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 --2- 172.21.15.87:0/318961681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95841550b0 0x7f95840cf930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f956c0097c0 tx=0x7f956c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- 172.21.15.87:0/318961681 shutdown_connections 2024-04-25T10:15:49.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 --2- 172.21.15.87:0/318961681 >> v2:172.21.15.169:3300/0 conn(0x7f95840cfe70 0x7f958415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 --2- 172.21.15.87:0/318961681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95841550b0 0x7f95840cf930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- 172.21.15.87:0/318961681 >> 172.21.15.87:0/318961681 conn(0x7f9584066b70 msgr2=0x7f9584066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:49.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- 172.21.15.87:0/318961681 shutdown_connections 2024-04-25T10:15:49.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- 172.21.15.87:0/318961681 wait complete. 2024-04-25T10:15:49.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 Processor -- start 2024-04-25T10:15:49.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- start start 2024-04-25T10:15:49.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95840cfe70 0x7f958413e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95841550b0 0x7f95840b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f958415b740 0x7f95840b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958415bcd0 con 0x7f958415b740 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958415bb50 con 0x7f95841550b0 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9589c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958415c030 con 0x7f95840cfe70 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9583400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f958415b740 0x7f95840b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9583400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f958415b740 0x7f95840b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54352/0 (socket says 172.21.15.87:54352) 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9583400640 1 -- 172.21.15.87:0/2542236644 learned_addr learned my addr 172.21.15.87:0/2542236644 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f9582a00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95840cfe70 0x7f958413e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.871+0000 7f957a000640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95841550b0 0x7f95840b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9583400640 1 -- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95840cfe70 msgr2=0x7f958413e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9583400640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95840cfe70 0x7f958413e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9583400640 1 -- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95841550b0 msgr2=0x7f95840b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9583400640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95841550b0 0x7f95840b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9583400640 1 -- 172.21.15.87:0/2542236644 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958415afb0 con 0x7f958415b740 2024-04-25T10:15:49.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9583400640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f958415b740 0x7f95840b2a60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f957c00caa0 tx=0x7f957c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f957b400640 1 -- 172.21.15.87:0/2542236644 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c017070 con 0x7f958415b740 2024-04-25T10:15:49.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f957b400640 1 -- 172.21.15.87:0/2542236644 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c00ac00 con 0x7f958415b740 2024-04-25T10:15:49.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9584156310 con 0x7f958415b740 2024-04-25T10:15:49.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f957b400640 1 -- 172.21.15.87:0/2542236644 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c012400 con 0x7f958415b740 2024-04-25T10:15:49.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958414ed60 con 0x7f958415b740 2024-04-25T10:15:49.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958404ef30 con 0x7f958415b740 2024-04-25T10:15:49.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f957b400640 1 -- 172.21.15.87:0/2542236644 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f957c01a070 con 0x7f958415b740 2024-04-25T10:15:49.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f957b400640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9560066f60 0x7f9560069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f957b400640 1 -- 172.21.15.87:0/2542236644 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f957c084d30 con 0x7f958415b740 2024-04-25T10:15:49.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.875+0000 7f9582a00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9560066f60 0x7f9560069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.879+0000 7f9582a00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9560066f60 0x7f9560069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f956c0ad000 tx=0x7f956c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:49.879+0000 7f957b400640 1 -- 172.21.15.87:0/2542236644 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f957c0556f0 con 0x7f958415b740 2024-04-25T10:15:50.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f958406c760 con 0x7f958415b740 2024-04-25T10:15:50.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f957b400640 1 -- 172.21.15.87:0/2542236644 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f957c059790 con 0x7f958415b740 2024-04-25T10:15:50.072 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:50.073 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:43.370279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:50.073 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 28 2024-04-25T10:15:50.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9560066f60 msgr2=0x7f9560069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9560066f60 0x7f9560069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f956c0ad000 tx=0x7f956c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f958415b740 msgr2=0x7f95840b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f958415b740 0x7f95840b2a60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f957c00caa0 tx=0x7f957c00cf60 comp rx=0 tx=0).stop 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 shutdown_connections 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9560066f60 0x7f9560069420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f958415b740 0x7f95840b2a60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95841550b0 0x7f95840b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 --2- 172.21.15.87:0/2542236644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95840cfe70 0x7f958413e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 >> 172.21.15.87:0/2542236644 conn(0x7f9584066b70 msgr2=0x7f9584066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 shutdown_connections 2024-04-25T10:15:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.067+0000 7f9589c00640 1 -- 172.21.15.87:0/2542236644 wait complete. 2024-04-25T10:15:50.094 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:50.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 Processor -- start 2024-04-25T10:15:50.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 -- start start 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc52815b9e0 0x7fc5280cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc52815bdc0 0x7fc5280cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc5280d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280ccfb0 con 0x7fc52815bdc0 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280cd290 con 0x7fc52815b9e0 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc52da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280cd570 con 0x7fc5280d1520 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc52815bdc0 0x7fc5280cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc52815bdc0 0x7fc5280cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54354/0 (socket says 172.21.15.87:54354) 2024-04-25T10:15:50.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 -- 172.21.15.87:0/667462684 learned_addr learned my addr 172.21.15.87:0/667462684 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:50.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc527400640 1 --2- 172.21.15.87:0/667462684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc5280d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526a00640 1 --2- 172.21.15.87:0/667462684 >> v2:172.21.15.169:3300/0 conn(0x7fc52815b9e0 0x7fc5280cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 -- 172.21.15.87:0/667462684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 msgr2=0x7fc5280d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 --2- 172.21.15.87:0/667462684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc5280d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 -- 172.21.15.87:0/667462684 >> v2:172.21.15.169:3300/0 conn(0x7fc52815b9e0 msgr2=0x7fc5280cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 --2- 172.21.15.87:0/667462684 >> v2:172.21.15.169:3300/0 conn(0x7fc52815b9e0 0x7fc5280cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.223+0000 7fc526000640 1 -- 172.21.15.87:0/667462684 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280cd850 con 0x7fc52815bdc0 2024-04-25T10:15:50.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc526000640 1 --2- 172.21.15.87:0/667462684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc5280cc950 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc528070da0 tx=0x7fc51401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79640ae348b6b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc525600640 1 -- 172.21.15.87:0/667462684 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5140b0070 con 0x7fc52815bdc0 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc525600640 1 -- 172.21.15.87:0/667462684 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5140ab600 con 0x7fc52815bdc0 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc525600640 1 -- 172.21.15.87:0/667462684 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5140ab9d0 con 0x7fc52815bdc0 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 -- 172.21.15.87:0/667462684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 msgr2=0x7fc5280cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 --2- 172.21.15.87:0/667462684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc5280cc950 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc528070da0 tx=0x7fc51401c490 comp rx=0 tx=0).stop 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 -- 172.21.15.87:0/667462684 shutdown_connections 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 --2- 172.21.15.87:0/667462684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc5280cc950 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 --2- 172.21.15.87:0/667462684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc5280d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 --2- 172.21.15.87:0/667462684 >> v2:172.21.15.169:3300/0 conn(0x7fc52815b9e0 0x7fc5280cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 -- 172.21.15.87:0/667462684 >> 172.21.15.87:0/667462684 conn(0x7fc52809a9d0 msgr2=0x7fc52809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 -- 172.21.15.87:0/667462684 shutdown_connections 2024-04-25T10:15:50.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.227+0000 7fc52da00640 1 -- 172.21.15.87:0/667462684 wait complete. 2024-04-25T10:15:50.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 Processor -- start 2024-04-25T10:15:50.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 -- start start 2024-04-25T10:15:50.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc528153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52815b9e0 0x7fc5281546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc528154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280cebd0 con 0x7fc52815bdc0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280cea50 con 0x7fc52815b9e0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280ced50 con 0x7fc5280d1520 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc527400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc528154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc526a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc528153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc526a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc528153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:53510/0 (socket says 172.21.15.87:53510) 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc526a00640 1 -- 172.21.15.87:0/3428530126 learned_addr learned my addr 172.21.15.87:0/3428530126 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc526000640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52815b9e0 0x7fc5281546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc527400640 1 -- 172.21.15.87:0/3428530126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 msgr2=0x7fc528153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc527400640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc528153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc527400640 1 -- 172.21.15.87:0/3428530126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52815b9e0 msgr2=0x7fc5281546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc527400640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52815b9e0 0x7fc5281546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc527400640 1 -- 172.21.15.87:0/3428530126 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5281559a0 con 0x7fc52815bdc0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc527400640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc528154c20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc52000efc0 tx=0x7fc52000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc51ea00640 1 -- 172.21.15.87:0/3428530126 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520018020 con 0x7fc52815bdc0 2024-04-25T10:15:50.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc51ea00640 1 -- 172.21.15.87:0/3428530126 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520014480 con 0x7fc52815bdc0 2024-04-25T10:15:50.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc51ea00640 1 -- 172.21.15.87:0/3428530126 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520014870 con 0x7fc52815bdc0 2024-04-25T10:15:50.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52815b4d0 con 0x7fc52815bdc0 2024-04-25T10:15:50.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5280b5260 con 0x7fc52815bdc0 2024-04-25T10:15:50.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.231+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5280c9c50 con 0x7fc52815bdc0 2024-04-25T10:15:50.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.235+0000 7fc51ea00640 1 -- 172.21.15.87:0/3428530126 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc52001b020 con 0x7fc52815bdc0 2024-04-25T10:15:50.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.235+0000 7fc51ea00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc508066f60 0x7fc508069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.235+0000 7fc51ea00640 1 -- 172.21.15.87:0/3428530126 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc520085d30 con 0x7fc52815bdc0 2024-04-25T10:15:50.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.235+0000 7fc526a00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc508066f60 0x7fc508069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.235+0000 7fc526a00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc508066f60 0x7fc508069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc518009ba0 tx=0x7fc518009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.239+0000 7fc51ea00640 1 -- 172.21.15.87:0/3428530126 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc520022030 con 0x7fc52815bdc0 2024-04-25T10:15:50.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc52804ef30 con 0x7fc52815bdc0 2024-04-25T10:15:50.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc51ea00640 1 -- 172.21.15.87:0/3428530126 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fc520010040 con 0x7fc52815bdc0 2024-04-25T10:15:50.432 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:50.433 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:06.691892+0000","modified":"2024-04-25T10:15:43.370279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.169:6834/3236213442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3236213442},{"type":"v1","addr":"172.21.15.169:6834","nonce":3236213442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:15:50.433 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 28 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc508066f60 msgr2=0x7fc508069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc508066f60 0x7fc508069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc518009ba0 tx=0x7fc518009190 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 msgr2=0x7fc528154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc528154c20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc52000efc0 tx=0x7fc52000c490 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 shutdown_connections 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc508066f60 0x7fc508069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc52815bdc0 0x7fc528154c20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52815b9e0 0x7fc5281546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 --2- 172.21.15.87:0/3428530126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5280d1520 0x7fc528153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 >> 172.21.15.87:0/3428530126 conn(0x7fc52809a9d0 msgr2=0x7fc5280c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 shutdown_connections 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.427+0000 7fc52da00640 1 -- 172.21.15.87:0/3428530126 wait complete. 2024-04-25T10:15:50.454 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b6c00640 1 Processor -- start 2024-04-25T10:15:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b6c00640 1 -- start start 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b6c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f55b015bc40 0x7f55b006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b6c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b6c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b00b2770 con 0x7f55b00cf930 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b6c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b006c670 con 0x7f55b015bc40 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b6c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b006c950 con 0x7f55b00cda10 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55afe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f55b015bc40 0x7f55b006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b4c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f55b00cf930 0x7f55b015f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40216/0 (socket says 172.21.15.87:40216) 2024-04-25T10:15:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55a7e00640 1 --2- 172.21.15.87:0/2524084642 >> v2:172.21.15.169:3300/0 conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.583+0000 7f55b4c00640 1 -- 172.21.15.87:0/2524084642 learned_addr learned my addr 172.21.15.87:0/2524084642 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55afe00640 1 -- 172.21.15.87:0/2524084642 >> v2:172.21.15.169:3300/0 conn(0x7f55b00cda10 msgr2=0x7f55b00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55afe00640 1 --2- 172.21.15.87:0/2524084642 >> v2:172.21.15.169:3300/0 conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55afe00640 1 -- 172.21.15.87:0/2524084642 >> v1:172.21.15.87:6789/0 conn(0x7f55b00cf930 legacy=0x7f55b015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55afe00640 1 -- 172.21.15.87:0/2524084642 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b006cc30 con 0x7f55b015bc40 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55afe00640 1 --2- 172.21.15.87:0/2524084642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b015bc40 0x7f55b006c090 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f55b00726f0 tx=0x7f55a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6032f80be8c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55af400640 1 -- 172.21.15.87:0/2524084642 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00b1070 con 0x7f55b015bc40 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55af400640 1 -- 172.21.15.87:0/2524084642 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a00ac5f0 con 0x7f55b015bc40 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55af400640 1 -- 172.21.15.87:0/2524084642 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00ac9c0 con 0x7f55b015bc40 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55b6c00640 1 -- 172.21.15.87:0/2524084642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b015bc40 msgr2=0x7f55b006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55b6c00640 1 --2- 172.21.15.87:0/2524084642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b015bc40 0x7f55b006c090 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f55b00726f0 tx=0x7f55a001c480 comp rx=0 tx=0).stop 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55b6c00640 1 -- 172.21.15.87:0/2524084642 shutdown_connections 2024-04-25T10:15:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55b6c00640 1 --2- 172.21.15.87:0/2524084642 >> v2:172.21.15.169:3300/0 conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55b6c00640 1 --2- 172.21.15.87:0/2524084642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b015bc40 0x7f55b006c090 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.587+0000 7f55b6c00640 1 -- 172.21.15.87:0/2524084642 >> 172.21.15.87:0/2524084642 conn(0x7f55b0066b70 msgr2=0x7f55b0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- 172.21.15.87:0/2524084642 shutdown_connections 2024-04-25T10:15:50.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- 172.21.15.87:0/2524084642 wait complete. 2024-04-25T10:15:50.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 Processor -- start 2024-04-25T10:15:50.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- start start 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b014b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b00afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 0x7f55b00b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0168b50 con 0x7f55b00cf930 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b01689d0 con 0x7f55b015bc40 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0168cd0 con 0x7f55b00cda10 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b00afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b00afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54372/0 (socket says 172.21.15.87:54372) 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 -- 172.21.15.87:0/1261583492 learned_addr learned my addr 172.21.15.87:0/1261583492 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b4c00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 0x7f55b00b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55afe00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b014b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 -- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 msgr2=0x7f55b014b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b014b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 -- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 msgr2=0x7f55b00b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 0x7f55b00b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 -- 172.21.15.87:0/1261583492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0070310 con 0x7f55b00cf930 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55a7e00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b00afdc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f559800e870 tx=0x7f559800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55acc00640 1 -- 172.21.15.87:0/1261583492 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598017070 con 0x7f55b00cf930 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55acc00640 1 -- 172.21.15.87:0/1261583492 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559800c940 con 0x7f55b00cf930 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55acc00640 1 -- 172.21.15.87:0/1261583492 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800ccf0 con 0x7f55b00cf930 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b00b0800 con 0x7f55b00cf930 2024-04-25T10:15:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.591+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b00b0dc0 con 0x7f55b00cf930 2024-04-25T10:15:50.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.595+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b014b780 con 0x7f55b00cf930 2024-04-25T10:15:50.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.595+0000 7f55acc00640 1 -- 172.21.15.87:0/1261583492 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f559801a070 con 0x7f55b00cf930 2024-04-25T10:15:50.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.595+0000 7f55acc00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f558c066f60 0x7f558c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.595+0000 7f55acc00640 1 -- 172.21.15.87:0/1261583492 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5598084dd0 con 0x7f55b00cf930 2024-04-25T10:15:50.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.595+0000 7f55afe00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f558c066f60 0x7f558c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.595+0000 7f55afe00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f558c066f60 0x7f558c069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55a00ae000 tx=0x7f55a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.599+0000 7f55acc00640 1 -- 172.21.15.87:0/1261583492 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5598055820 con 0x7f55b00cf930 2024-04-25T10:15:50.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55b004ef30 con 0x7f55b00cf930 2024-04-25T10:15:50.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55acc00640 1 -- 172.21.15.87:0/1261583492 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f55980598c0 con 0x7f55b00cf930 2024-04-25T10:15:50.761 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:50.761 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:08.327470+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:05.052743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:05.692130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:50.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f558c066f60 msgr2=0x7f558c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f558c066f60 0x7f558c069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55a00ae000 tx=0x7f55a0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:50.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 msgr2=0x7f55b00afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b00afdc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f559800e870 tx=0x7f559800ec70 comp rx=0 tx=0).stop 2024-04-25T10:15:50.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 shutdown_connections 2024-04-25T10:15:50.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f558c066f60 0x7f558c069420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 0x7f55b00b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b00afdc0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 --2- 172.21.15.87:0/1261583492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b014b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 >> 172.21.15.87:0/1261583492 conn(0x7f55b0066b70 msgr2=0x7f55b015a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 shutdown_connections 2024-04-25T10:15:50.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.755+0000 7f55b6c00640 1 -- 172.21.15.87:0/1261583492 wait complete. 2024-04-25T10:15:50.783 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:50.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.911+0000 7f14fa000640 1 Processor -- start 2024-04-25T10:15:50.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14fa000640 1 -- start start 2024-04-25T10:15:50.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14fa000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14f41550b0 0x7f14f40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14fa000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14f40cfe70 0x7f14f415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14fa000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f4071050 con 0x7f14f415b740 2024-04-25T10:15:50.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14fa000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f415d040 con 0x7f14f40cfe70 2024-04-25T10:15:50.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14fa000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f40712f0 con 0x7f14f41550b0 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14f40cfe70 0x7f14f415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14f40cfe70 0x7f14f415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54382/0 (socket says 172.21.15.87:54382) 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f2a00640 1 -- 172.21.15.87:0/2895325734 learned_addr learned my addr 172.21.15.87:0/2895325734 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f3400640 1 --2- 172.21.15.87:0/2895325734 >> v2:172.21.15.169:3300/0 conn(0x7f14f41550b0 0x7f14f40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f2a00640 1 -- 172.21.15.87:0/2895325734 >> v2:172.21.15.169:3300/0 conn(0x7f14f41550b0 msgr2=0x7f14f40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f2a00640 1 --2- 172.21.15.87:0/2895325734 >> v2:172.21.15.169:3300/0 conn(0x7f14f41550b0 0x7f14f40cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f2a00640 1 -- 172.21.15.87:0/2895325734 >> v1:172.21.15.169:6789/0 conn(0x7f14f415b740 legacy=0x7f14f406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.915+0000 7f14f2a00640 1 -- 172.21.15.87:0/2895325734 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f40715d0 con 0x7f14f40cfe70 2024-04-25T10:15:50.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14f2a00640 1 --2- 172.21.15.87:0/2895325734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f40cfe70 0x7f14f415b200 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009870 tx=0x7f14e001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9930666e0c95a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14f2000640 1 -- 172.21.15.87:0/2895325734 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e00b0070 con 0x7f14f40cfe70 2024-04-25T10:15:50.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14f2000640 1 -- 172.21.15.87:0/2895325734 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e00ab600 con 0x7f14f40cfe70 2024-04-25T10:15:50.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14f2000640 1 -- 172.21.15.87:0/2895325734 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e00ab9d0 con 0x7f14f40cfe70 2024-04-25T10:15:50.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 -- 172.21.15.87:0/2895325734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f40cfe70 msgr2=0x7f14f415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 --2- 172.21.15.87:0/2895325734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f40cfe70 0x7f14f415b200 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009870 tx=0x7f14e001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 -- 172.21.15.87:0/2895325734 shutdown_connections 2024-04-25T10:15:50.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 --2- 172.21.15.87:0/2895325734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f40cfe70 0x7f14f415b200 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 --2- 172.21.15.87:0/2895325734 >> v2:172.21.15.169:3300/0 conn(0x7f14f41550b0 0x7f14f40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 -- 172.21.15.87:0/2895325734 >> 172.21.15.87:0/2895325734 conn(0x7f14f4066b70 msgr2=0x7f14f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 -- 172.21.15.87:0/2895325734 shutdown_connections 2024-04-25T10:15:50.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 -- 172.21.15.87:0/2895325734 wait complete. 2024-04-25T10:15:50.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 Processor -- start 2024-04-25T10:15:50.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.919+0000 7f14fa000640 1 -- start start 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f40cfe70 0x7f14f4146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f41550b0 0x7f14f40afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f415b740 0x7f14f40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f415bd30 con 0x7f14f415b740 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f415bbb0 con 0x7f14f40cfe70 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f415c030 con 0x7f14f41550b0 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f415b740 0x7f14f40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f415b740 0x7f14f40b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54390/0 (socket says 172.21.15.87:54390) 2024-04-25T10:15:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 -- 172.21.15.87:0/3837515758 learned_addr learned my addr 172.21.15.87:0/3837515758 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:50.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3400640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f40cfe70 0x7f14f4146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f2a00640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f41550b0 0x7f14f40afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 -- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f41550b0 msgr2=0x7f14f40afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f41550b0 0x7f14f40afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 -- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f40cfe70 msgr2=0x7f14f4146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f40cfe70 0x7f14f4146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 -- 172.21.15.87:0/3837515758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f414a4c0 con 0x7f14f415b740 2024-04-25T10:15:50.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14f3e00640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f415b740 0x7f14f40b0040 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b9a0 tx=0x7f14ec00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14eb400640 1 -- 172.21.15.87:0/3837515758 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec019070 con 0x7f14f415b740 2024-04-25T10:15:50.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14eb400640 1 -- 172.21.15.87:0/3837515758 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec00cba0 con 0x7f14f415b740 2024-04-25T10:15:50.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14eb400640 1 -- 172.21.15.87:0/3837515758 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec014440 con 0x7f14f415b740 2024-04-25T10:15:50.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f4147510 con 0x7f14f415b740 2024-04-25T10:15:50.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f414aa10 con 0x7f14f415b740 2024-04-25T10:15:50.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f404ef30 con 0x7f14f415b740 2024-04-25T10:15:50.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.923+0000 7f14eb400640 1 -- 172.21.15.87:0/3837515758 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14ec01c070 con 0x7f14f415b740 2024-04-25T10:15:50.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.927+0000 7f14eb400640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14d4066f60 0x7f14d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.927+0000 7f14eb400640 1 -- 172.21.15.87:0/3837515758 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14ec084c90 con 0x7f14f415b740 2024-04-25T10:15:50.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.927+0000 7f14f3400640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14d4066f60 0x7f14d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.927+0000 7f14f3400640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14d4066f60 0x7f14d4069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f14e4007a60 tx=0x7f14e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:50.931+0000 7f14eb400640 1 -- 172.21.15.87:0/3837515758 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14f4147510 con 0x7f14f415b740 2024-04-25T10:15:51.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.115+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f14f40cf120 con 0x7f14f415b740 2024-04-25T10:15:51.122 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:15:51.115+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:15:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.119+0000 7f14eb400640 1 -- 172.21.15.87:0/3837515758 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f14ec0555b0 con 0x7f14f415b740 2024-04-25T10:15:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14d4066f60 msgr2=0x7f14d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14d4066f60 0x7f14d4069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f14e4007a60 tx=0x7f14e4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f415b740 msgr2=0x7f14f40b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f415b740 0x7f14f40b0040 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b9a0 tx=0x7f14ec00be60 comp rx=0 tx=0).stop 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 shutdown_connections 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14d4066f60 0x7f14d4069420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14f415b740 0x7f14f40b0040 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f41550b0 0x7f14f40afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 --2- 172.21.15.87:0/3837515758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f40cfe70 0x7f14f4146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 >> 172.21.15.87:0/3837515758 conn(0x7f14f4066b70 msgr2=0x7f14f40b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 shutdown_connections 2024-04-25T10:15:51.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.123+0000 7f14fa000640 1 -- 172.21.15.87:0/3837515758 wait complete. 2024-04-25T10:15:51.147 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:15:51.198 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:15:51.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7607800640 1 Processor -- start 2024-04-25T10:15:51.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7607800640 1 -- start start 2024-04-25T10:15:51.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7607800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76000721f0 0x7f7600166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7607800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7600071360 con 0x7f76000aeda0 2024-04-25T10:15:51.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7607800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7600159920 con 0x7f7600159540 2024-04-25T10:15:51.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7607800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760015c2f0 con 0x7f76000721f0 2024-04-25T10:15:51.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f75ffe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f76000aeda0 0x7f7600071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40224/0 (socket says 172.21.15.87:40224) 2024-04-25T10:15:51.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7605800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76000721f0 0x7f7600166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f75ffe00640 1 -- 172.21.15.87:0/98018686 learned_addr learned my addr 172.21.15.87:0/98018686 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:51.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f75ff400640 1 -- 172.21.15.87:0/98018686 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559777953 0 0) 0x7f7600071360 con 0x7f76000aeda0 2024-04-25T10:15:51.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7605800640 1 -- 172.21.15.87:0/98018686 >> v1:172.21.15.169:6789/0 conn(0x7f7600159540 legacy=0x7f76000ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7605800640 1 -- 172.21.15.87:0/98018686 >> v1:172.21.15.87:6789/0 conn(0x7f76000aeda0 legacy=0x7f7600071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.279+0000 7f7605800640 1 -- 172.21.15.87:0/98018686 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760015c5d0 con 0x7f76000721f0 2024-04-25T10:15:51.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7605800640 1 --2- 172.21.15.87:0/98018686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000721f0 0x7f7600166920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009740 tx=0x7f75f801c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f390137b21b6a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f75ff400640 1 -- 172.21.15.87:0/98018686 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80b0070 con 0x7f76000721f0 2024-04-25T10:15:51.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f75ff400640 1 -- 172.21.15.87:0/98018686 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f80ab600 con 0x7f76000721f0 2024-04-25T10:15:51.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f75ff400640 1 -- 172.21.15.87:0/98018686 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80ab9d0 con 0x7f76000721f0 2024-04-25T10:15:51.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7607800640 1 -- 172.21.15.87:0/98018686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000721f0 msgr2=0x7f7600166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7607800640 1 --2- 172.21.15.87:0/98018686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000721f0 0x7f7600166920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009740 tx=0x7f75f801c2a0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7607800640 1 -- 172.21.15.87:0/98018686 shutdown_connections 2024-04-25T10:15:51.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7607800640 1 --2- 172.21.15.87:0/98018686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000721f0 0x7f7600166920 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7607800640 1 -- 172.21.15.87:0/98018686 >> 172.21.15.87:0/98018686 conn(0x7f76000669a0 msgr2=0x7f7600066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7607800640 1 -- 172.21.15.87:0/98018686 shutdown_connections 2024-04-25T10:15:51.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.283+0000 7f7607800640 1 -- 172.21.15.87:0/98018686 wait complete. 2024-04-25T10:15:51.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 Processor -- start 2024-04-25T10:15:51.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 -- start start 2024-04-25T10:15:51.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76000721f0 0x7f7600149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000aeda0 0x7f760015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7600159540 0x7f7600153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7600169680 con 0x7f76000aeda0 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760006c0d0 con 0x7f7600159540 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7600169800 con 0x7f76000721f0 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000aeda0 0x7f760015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000aeda0 0x7f760015b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54412/0 (socket says 172.21.15.87:54412) 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 -- 172.21.15.87:0/33749628 learned_addr learned my addr 172.21.15.87:0/33749628 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7605800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7600159540 0x7f7600153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7604e00640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76000721f0 0x7f7600149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 -- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76000721f0 msgr2=0x7f7600149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76000721f0 0x7f7600149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 -- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7600159540 msgr2=0x7f7600153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7600159540 0x7f7600153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 -- 172.21.15.87:0/33749628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760006de20 con 0x7f76000aeda0 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7605800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7600159540 0x7f7600153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75ffe00640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000aeda0 0x7f760015b650 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c4b0 tx=0x7f75ec00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75fcc00640 1 -- 172.21.15.87:0/33749628 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec018070 con 0x7f76000aeda0 2024-04-25T10:15:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75fcc00640 1 -- 172.21.15.87:0/33749628 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec007500 con 0x7f76000aeda0 2024-04-25T10:15:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7600158dc0 con 0x7f76000aeda0 2024-04-25T10:15:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f75fcc00640 1 -- 172.21.15.87:0/33749628 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec004a30 con 0x7f76000aeda0 2024-04-25T10:15:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760014fc10 con 0x7f76000aeda0 2024-04-25T10:15:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.287+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760004ef30 con 0x7f76000aeda0 2024-04-25T10:15:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.291+0000 7f75fcc00640 1 -- 172.21.15.87:0/33749628 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75ec01b070 con 0x7f76000aeda0 2024-04-25T10:15:51.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.291+0000 7f75fcc00640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f75e0066f60 0x7f75e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.291+0000 7f75fcc00640 1 -- 172.21.15.87:0/33749628 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f75ec084380 con 0x7f76000aeda0 2024-04-25T10:15:51.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.291+0000 7f7604e00640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f75e0066f60 0x7f75e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.291+0000 7f7604e00640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f75e0066f60 0x7f75e0069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f75f0007f30 tx=0x7f75f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.295+0000 7f75fcc00640 1 -- 172.21.15.87:0/33749628 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75ec0553e0 con 0x7f76000aeda0 2024-04-25T10:15:51.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:51.483+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f760006d7f0 con 0x7f76000aeda0 2024-04-25T10:15:51.891 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:51.895 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:52.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.123+0000 7f75fcc00640 1 -- 172.21.15.87:0/33749628 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f75ec054ce0 con 0x7f76000aeda0 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f75e0066f60 msgr2=0x7f75e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f75e0066f60 0x7f75e0069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f75f0007f30 tx=0x7f75f0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000aeda0 msgr2=0x7f760015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000aeda0 0x7f760015b650 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c4b0 tx=0x7f75ec00c970 comp rx=0 tx=0).stop 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 shutdown_connections 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f75e0066f60 0x7f75e0069420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7600159540 0x7f7600153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000aeda0 0x7f760015b650 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 --2- 172.21.15.87:0/33749628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76000721f0 0x7f7600149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 >> 172.21.15.87:0/33749628 conn(0x7f76000669a0 msgr2=0x7f7600066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 shutdown_connections 2024-04-25T10:15:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.127+0000 7f7607800640 1 -- 172.21.15.87:0/33749628 wait complete. 2024-04-25T10:15:52.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:15:52.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd308a00640 1 Processor -- start 2024-04-25T10:15:52.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd308a00640 1 -- start start 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd308a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd308a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd30415bee0 0x7fd3040711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd308a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30415c340 con 0x7fd304071700 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd308a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3040cda60 con 0x7fd30415bee0 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd308a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30415ce80 con 0x7fd304145220 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd2fbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd30415bee0 0x7fd3040711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd2fbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd30415bee0 0x7fd3040711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54422/0 (socket says 172.21.15.87:54422) 2024-04-25T10:15:52.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd2fbe00640 1 -- 172.21.15.87:0/2120325215 learned_addr learned my addr 172.21.15.87:0/2120325215 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:52.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd303400640 1 --2- 172.21.15.87:0/2120325215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd2fbe00640 1 -- 172.21.15.87:0/2120325215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 msgr2=0x7fd3040cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd2fbe00640 1 --2- 172.21.15.87:0/2120325215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd2fbe00640 1 -- 172.21.15.87:0/2120325215 >> v1:172.21.15.169:6789/0 conn(0x7fd304071700 legacy=0x7fd30415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd2fbe00640 1 -- 172.21.15.87:0/2120325215 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304145600 con 0x7fd30415bee0 2024-04-25T10:15:52.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.295+0000 7fd303400640 1 --2- 172.21.15.87:0/2120325215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd2fbe00640 1 --2- 172.21.15.87:0/2120325215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd3040711c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd3040d1700 tx=0x7fd2ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78014e955b98fb0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd302a00640 1 -- 172.21.15.87:0/2120325215 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0b1070 con 0x7fd30415bee0 2024-04-25T10:15:52.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd302a00640 1 -- 172.21.15.87:0/2120325215 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec0a8050 con 0x7fd30415bee0 2024-04-25T10:15:52.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd302a00640 1 -- 172.21.15.87:0/2120325215 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0ac860 con 0x7fd30415bee0 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 -- 172.21.15.87:0/2120325215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 msgr2=0x7fd3040711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 --2- 172.21.15.87:0/2120325215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd3040711c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd3040d1700 tx=0x7fd2ec01c420 comp rx=0 tx=0).stop 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 -- 172.21.15.87:0/2120325215 shutdown_connections 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 --2- 172.21.15.87:0/2120325215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd3040711c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 --2- 172.21.15.87:0/2120325215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 -- 172.21.15.87:0/2120325215 >> 172.21.15.87:0/2120325215 conn(0x7fd30406ac90 msgr2=0x7fd30406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 -- 172.21.15.87:0/2120325215 shutdown_connections 2024-04-25T10:15:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.299+0000 7fd308a00640 1 -- 172.21.15.87:0/2120325215 wait complete. 2024-04-25T10:15:52.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 Processor -- start 2024-04-25T10:15:52.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 -- start start 2024-04-25T10:15:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd304071700 0x7fd30413da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd304164c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304160210 con 0x7fd30415bee0 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30415f600 con 0x7fd304071700 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd304160390 con 0x7fd304145220 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd304164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd304071700 0x7fd30413da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd304164c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54434/0 (socket says 172.21.15.87:54434) 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 -- 172.21.15.87:0/3660807150 learned_addr learned my addr 172.21.15.87:0/3660807150 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd2fbe00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 -- 172.21.15.87:0/3660807150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 msgr2=0x7fd3040b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 -- 172.21.15.87:0/3660807150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd304071700 msgr2=0x7fd30413da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd304071700 0x7fd30413da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 -- 172.21.15.87:0/3660807150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040b3fb0 con 0x7fd30415bee0 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd303e00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd304164c10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00c780 tx=0x7fd2fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd2fb400640 1 -- 172.21.15.87:0/3660807150 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc004b70 con 0x7fd30415bee0 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd2fb400640 1 -- 172.21.15.87:0/3660807150 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc004d10 con 0x7fd30415bee0 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd2fb400640 1 -- 172.21.15.87:0/3660807150 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00aa90 con 0x7fd30415bee0 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3040b4290 con 0x7fd30415bee0 2024-04-25T10:15:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3040b4850 con 0x7fd30415bee0 2024-04-25T10:15:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.303+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd304075120 con 0x7fd30415bee0 2024-04-25T10:15:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.307+0000 7fd2fb400640 1 -- 172.21.15.87:0/3660807150 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2fc00ac30 con 0x7fd30415bee0 2024-04-25T10:15:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.307+0000 7fd2fb400640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2e0066f60 0x7fd2e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.307+0000 7fd2fb400640 1 -- 172.21.15.87:0/3660807150 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd2fc085150 con 0x7fd30415bee0 2024-04-25T10:15:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.307+0000 7fd303400640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2e0066f60 0x7fd2e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.307+0000 7fd303400640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2e0066f60 0x7fd2e0069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400ab60 tx=0x7fd2f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.311+0000 7fd2fb400640 1 -- 172.21.15.87:0/3660807150 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2fc020070 con 0x7fd30415bee0 2024-04-25T10:15:52.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:52.467+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3040cd340 con 0x7fd30415bee0 2024-04-25T10:15:53.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.127+0000 7fd2fb400640 1 -- 172.21.15.87:0/3660807150 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd2fc055b40 con 0x7fd30415bee0 2024-04-25T10:15:53.134 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:15:53.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2e0066f60 msgr2=0x7fd2e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2e0066f60 0x7fd2e0069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400ab60 tx=0x7fd2f4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 msgr2=0x7fd304164c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd304164c10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00c780 tx=0x7fd2fc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 shutdown_connections 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2e0066f60 0x7fd2e0069420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd30415bee0 0x7fd304164c10 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd304145220 0x7fd3040b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 --2- 172.21.15.87:0/3660807150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd304071700 0x7fd30413da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 >> 172.21.15.87:0/3660807150 conn(0x7fd30406ac90 msgr2=0x7fd304071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 shutdown_connections 2024-04-25T10:15:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.131+0000 7fd308a00640 1 -- 172.21.15.87:0/3660807150 wait complete. 2024-04-25T10:15:53.158 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:15:53.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10fc00640 1 Processor -- start 2024-04-25T10:15:53.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10fc00640 1 -- start start 2024-04-25T10:15:53.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 0x7fe10815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10fc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10815e9b0 con 0x7fe108162310 2024-04-25T10:15:53.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10fc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10815ec90 con 0x7fe1081618a0 2024-04-25T10:15:53.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10815f250 con 0x7fe1080b4f50 2024-04-25T10:15:53.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10d200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe108162310 0x7fe1080b4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40228/0 (socket says 172.21.15.87:40228) 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10d200640 1 -- 172.21.15.87:0/3262791651 learned_addr learned my addr 172.21.15.87:0/3262791651 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10dc00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 0x7fe10815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187956302 0 0) 0x7fe10815e9b0 con 0x7fe108162310 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ec003530 con 0x7fe108162310 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135054725 0 0) 0x7fe10815ec90 con 0x7fe1081618a0 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10815e9b0 con 0x7fe1081618a0 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3954690538 0 0) 0x7fe0ec003530 con 0x7fe108162310 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 msgr2=0x7fe10815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 0x7fe10815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 >> v1:172.21.15.169:6789/0 conn(0x7fe1081618a0 legacy=0x7fe1080b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10815ef70 con 0x7fe108162310 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe0f80016a0 con 0x7fe108162310 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10dc00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 0x7fe10815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 >> v1:172.21.15.87:6789/0 conn(0x7fe108162310 legacy=0x7fe1080b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0ec003b80 0x7fe0ec003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0ec004810 0x7fe0ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 0x7fe0ec005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10815f250 con 0x7fe0ec003b80 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec00bbc0 con 0x7fe0ec004810 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec00bdc0 con 0x7fe1080b4f50 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10d200640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0ec003b80 0x7fe0ec003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.295+0000 7fe10dc00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 0x7fe0ec005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe0ffe00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0ec004810 0x7fe0ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10d200640 1 -- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 msgr2=0x7fe0ec005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10d200640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1080b4f50 0x7fe0ec005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10d200640 1 -- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0ec004810 msgr2=0x7fe0ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10d200640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0ec004810 0x7fe0ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10d200640 1 -- 172.21.15.87:0/3262791651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec003530 con 0x7fe0ec003b80 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10d200640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0ec003b80 0x7fe0ec003f80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe1080676d0 tx=0x7fe0f801cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b61cb05754af92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8007c10 con 0x7fe0ec003b80 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80a9020 con 0x7fe0ec003b80 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe107e00640 1 -- 172.21.15.87:0/3262791651 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8005b20 con 0x7fe0ec003b80 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10fc00640 1 -- 172.21.15.87:0/3262791651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0ec003b80 msgr2=0x7fe0ec003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10fc00640 1 --2- 172.21.15.87:0/3262791651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0ec003b80 0x7fe0ec003f80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe1080676d0 tx=0x7fe0f801cae0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.299+0000 7fe10dc00640 1 -- 172.21.15.87:0/3262791651 reap_dead start 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- 172.21.15.87:0/3262791651 shutdown_connections 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- 172.21.15.87:0/3262791651 >> 172.21.15.87:0/3262791651 conn(0x7fe1080b5610 msgr2=0x7fe1080b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- 172.21.15.87:0/3262791651 shutdown_connections 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- 172.21.15.87:0/3262791651 wait complete. 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 Processor -- start 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- start start 2024-04-25T10:15:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1081486f0 0x7fe108160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe10815da40 0x7fe10814ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe10814b380 0x7fe108149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080b1210 con 0x7fe10815da40 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080b1090 con 0x7fe1081486f0 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080b1390 con 0x7fe10814b380 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe10815da40 0x7fe10814ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe10815da40 0x7fe10814ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54460/0 (socket says 172.21.15.87:54460) 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 -- 172.21.15.87:0/805391364 learned_addr learned my addr 172.21.15.87:0/805391364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10d200640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1081486f0 0x7fe108160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10dc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe10814b380 0x7fe108149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 -- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe10814b380 msgr2=0x7fe108149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe10814b380 0x7fe108149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 -- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1081486f0 msgr2=0x7fe108160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1081486f0 0x7fe108160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 -- 172.21.15.87:0/805391364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10815a720 con 0x7fe10815da40 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe10dc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe10814b380 0x7fe108149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe0ffe00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe10815da40 0x7fe10814ae40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000f770 tx=0x7fe0f000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe105600640 1 -- 172.21.15.87:0/805391364 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00050d0 con 0x7fe10815da40 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe105600640 1 -- 172.21.15.87:0/805391364 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0018070 con 0x7fe10815da40 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.303+0000 7fe105600640 1 -- 172.21.15.87:0/805391364 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000d730 con 0x7fe10815da40 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10815aa00 con 0x7fe10815da40 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10815afc0 con 0x7fe10815da40 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10804ef30 con 0x7fe10815da40 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe105600640 1 -- 172.21.15.87:0/805391364 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0f000d8d0 con 0x7fe10815da40 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe105600640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0e4066f90 0x7fe0e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe105600640 1 -- 172.21.15.87:0/805391364 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe0f0084de0 con 0x7fe10815da40 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe10d200640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0e4066f90 0x7fe0e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.307+0000 7fe10d200640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0e4066f90 0x7fe0e4069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80aa000 tx=0x7fe1080afc50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.311+0000 7fe105600640 1 -- 172.21.15.87:0/805391364 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0f0055eb0 con 0x7fe10815da40 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:53.467+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe10814b790 con 0x7fe10815da40 2024-04-25T10:15:54.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe105600640 1 -- 172.21.15.87:0/805391364 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe0f00557b0 con 0x7fe10815da40 2024-04-25T10:15:54.140 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0e4066f90 msgr2=0x7fe0e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0e4066f90 0x7fe0e4069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80aa000 tx=0x7fe1080afc50 comp rx=0 tx=0).stop 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe10815da40 msgr2=0x7fe10814ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe10815da40 0x7fe10814ae40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000f770 tx=0x7fe0f000fc30 comp rx=0 tx=0).stop 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 shutdown_connections 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0e4066f90 0x7fe0e4069450 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe10814b380 0x7fe108149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe10815da40 0x7fe10814ae40 secure :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000f770 tx=0x7fe0f000fc30 comp rx=0 tx=0).stop 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 --2- 172.21.15.87:0/805391364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1081486f0 0x7fe108160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.135+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 >> 172.21.15.87:0/805391364 conn(0x7fe1080b5610 msgr2=0x7fe1081606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.139+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 shutdown_connections 2024-04-25T10:15:54.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.139+0000 7fe10fc00640 1 -- 172.21.15.87:0/805391364 wait complete. 2024-04-25T10:15:54.170 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:15:54.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ceaa00640 1 Processor -- start 2024-04-25T10:15:54.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ceaa00640 1 -- start start 2024-04-25T10:15:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ceaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce415bc10 0x7f7ce415f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ceaa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce40b1770 con 0x7f7ce40d1830 2024-04-25T10:15:54.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ceaa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce415fec0 con 0x7f7ce40cf980 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ceaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce41601a0 con 0x7f7ce415bc10 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce8a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7ce40d1830 0x7f7ce40d1c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40236/0 (socket says 172.21.15.87:40236) 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce8a00640 1 -- 172.21.15.87:0/3156794538 learned_addr learned my addr 172.21.15.87:0/3156794538 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce3e00640 1 --2- 172.21.15.87:0/3156794538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce415bc10 0x7f7ce415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce2a00640 1 -- 172.21.15.87:0/3156794538 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218599634 0 0) 0x7f7ce40b1770 con 0x7f7ce40d1830 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce2a00640 1 -- 172.21.15.87:0/3156794538 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cc8003530 con 0x7f7ce40d1830 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce3e00640 1 -- 172.21.15.87:0/3156794538 >> v1:172.21.15.169:6789/0 conn(0x7f7ce40cf980 legacy=0x7f7ce40b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce2a00640 1 -- 172.21.15.87:0/3156794538 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223669111 0 0) 0x7f7ce415fec0 con 0x7f7ce40cf980 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce3e00640 1 -- 172.21.15.87:0/3156794538 >> v1:172.21.15.87:6789/0 conn(0x7f7ce40d1830 legacy=0x7f7ce40d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.323+0000 7f7ce3e00640 1 -- 172.21.15.87:0/3156794538 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40cc5b0 con 0x7f7ce415bc10 2024-04-25T10:15:54.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ce3e00640 1 --2- 172.21.15.87:0/3156794538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce415bc10 0x7f7ce415f600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097f0 tx=0x7f7cd401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64280ead7318b5a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ce2a00640 1 -- 172.21.15.87:0/3156794538 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40b0070 con 0x7f7ce415bc10 2024-04-25T10:15:54.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ce2a00640 1 -- 172.21.15.87:0/3156794538 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd40ab600 con 0x7f7ce415bc10 2024-04-25T10:15:54.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ce2a00640 1 -- 172.21.15.87:0/3156794538 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40ab9b0 con 0x7f7ce415bc10 2024-04-25T10:15:54.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ceaa00640 1 -- 172.21.15.87:0/3156794538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce415bc10 msgr2=0x7f7ce415f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/3156794538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce415bc10 0x7f7ce415f600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097f0 tx=0x7f7cd401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ceaa00640 1 -- 172.21.15.87:0/3156794538 shutdown_connections 2024-04-25T10:15:54.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/3156794538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce415bc10 0x7f7ce415f600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ceaa00640 1 -- 172.21.15.87:0/3156794538 >> 172.21.15.87:0/3156794538 conn(0x7f7ce409a9d0 msgr2=0x7f7ce409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ceaa00640 1 -- 172.21.15.87:0/3156794538 shutdown_connections 2024-04-25T10:15:54.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.327+0000 7f7ceaa00640 1 -- 172.21.15.87:0/3156794538 wait complete. 2024-04-25T10:15:54.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 Processor -- start 2024-04-25T10:15:54.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 -- start start 2024-04-25T10:15:54.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ce40cf980 0x7f7ce40b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce40d1830 0x7f7ce414b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ce415bc10 0x7f7ce415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce40cd740 con 0x7f7ce40cf980 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce40cd5c0 con 0x7f7ce415bc10 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce4169970 con 0x7f7ce40d1830 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ce40cf980 0x7f7ce40b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ce40cf980 0x7f7ce40b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54462/0 (socket says 172.21.15.87:54462) 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 -- 172.21.15.87:0/2027586383 learned_addr learned my addr 172.21.15.87:0/2027586383 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3400640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce40d1830 0x7f7ce414b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce8a00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ce415bc10 0x7f7ce415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 -- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce40d1830 msgr2=0x7f7ce414b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce40d1830 0x7f7ce414b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 -- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ce415bc10 msgr2=0x7f7ce415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ce415bc10 0x7f7ce415ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 -- 172.21.15.87:0/2027586383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce414bc90 con 0x7f7ce40cf980 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ce3e00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ce40cf980 0x7f7ce40b65f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097f0 tx=0x7f7cd401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7cdbe00640 1 -- 172.21.15.87:0/2027586383 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40b0050 con 0x7f7ce40cf980 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7cdbe00640 1 -- 172.21.15.87:0/2027586383 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd40b1040 con 0x7f7ce40cf980 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7cdbe00640 1 -- 172.21.15.87:0/2027586383 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40b5610 con 0x7f7ce40cf980 2024-04-25T10:15:54.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce40b6cd0 con 0x7f7ce40cf980 2024-04-25T10:15:54.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.331+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce40b7290 con 0x7f7ce40cf980 2024-04-25T10:15:54.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.335+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce404ef30 con 0x7f7ce40cf980 2024-04-25T10:15:54.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.335+0000 7f7cdbe00640 1 -- 172.21.15.87:0/2027586383 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cd40be030 con 0x7f7ce40cf980 2024-04-25T10:15:54.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.335+0000 7f7cdbe00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7cc4066f60 0x7f7cc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.335+0000 7f7cdbe00640 1 -- 172.21.15.87:0/2027586383 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7cd411ea70 con 0x7f7ce40cf980 2024-04-25T10:15:54.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.335+0000 7f7ce3400640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7cc4066f60 0x7f7cc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.335+0000 7f7ce3400640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7cc4066f60 0x7f7cc4069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ce41fa1b0 tx=0x7f7cd0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.339+0000 7f7cdbe00640 1 -- 172.21.15.87:0/2027586383 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cd40c5020 con 0x7f7ce40cf980 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.495+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7ce40cf760 con 0x7f7ce40cf980 2024-04-25T10:15:54.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7cdbe00640 1 -- 172.21.15.87:0/2027586383 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ce414bc90 con 0x7f7ce40cf980 2024-04-25T10:15:54.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7cc4066f60 msgr2=0x7f7cc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7cc4066f60 0x7f7cc4069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ce41fa1b0 tx=0x7f7cd0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ce40cf980 msgr2=0x7f7ce40b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ce40cf980 0x7f7ce40b65f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097f0 tx=0x7f7cd401ad20 comp rx=0 tx=0).stop 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 shutdown_connections 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7cc4066f60 0x7f7cc4069420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ce415bc10 0x7f7ce415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ce40d1830 0x7f7ce414b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 --2- 172.21.15.87:0/2027586383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ce40cf980 0x7f7ce40b65f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 >> 172.21.15.87:0/2027586383 conn(0x7f7ce409a9d0 msgr2=0x7f7ce40b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 shutdown_connections 2024-04-25T10:15:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.499+0000 7f7ceaa00640 1 -- 172.21.15.87:0/2027586383 wait complete. 2024-04-25T10:15:54.526 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:15:54.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.655+0000 7fca3b000640 1 Processor -- start 2024-04-25T10:15:54.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.655+0000 7fca3b000640 1 -- start start 2024-04-25T10:15:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca3b000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca3415b910 0x7fca3415c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca3b000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3415bd60 con 0x7fca341550b0 2024-04-25T10:15:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca3b000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca34071050 con 0x7fca340d0040 2024-04-25T10:15:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca3b000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca340712f0 con 0x7fca3415b910 2024-04-25T10:15:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca33e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fca341550b0 0x7fca340cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40248/0 (socket says 172.21.15.87:40248) 2024-04-25T10:15:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca33e00640 1 -- 172.21.15.87:0/1163348910 learned_addr learned my addr 172.21.15.87:0/1163348910 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:54.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca39000640 1 --2- 172.21.15.87:0/1163348910 >> v2:172.21.15.169:3300/0 conn(0x7fca3415b910 0x7fca3415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670721685 0 0) 0x7fca3415bd60 con 0x7fca341550b0 2024-04-25T10:15:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca18003530 con 0x7fca341550b0 2024-04-25T10:15:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081528260 0 0) 0x7fca34071050 con 0x7fca340d0040 2024-04-25T10:15:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3415bd60 con 0x7fca340d0040 2024-04-25T10:15:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248354288 0 0) 0x7fca18003530 con 0x7fca341550b0 2024-04-25T10:15:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca39000640 1 -- 172.21.15.87:0/1163348910 >> v1:172.21.15.169:6789/0 conn(0x7fca340d0040 legacy=0x7fca3415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca39000640 1 -- 172.21.15.87:0/1163348910 >> v1:172.21.15.87:6789/0 conn(0x7fca341550b0 legacy=0x7fca340cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.659+0000 7fca39000640 1 -- 172.21.15.87:0/1163348910 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340715d0 con 0x7fca3415b910 2024-04-25T10:15:54.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca39000640 1 --2- 172.21.15.87:0/1163348910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca3415b910 0x7fca3415c030 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d90 tx=0x7fca2c01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2635b04e88ae60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0b2070 con 0x7fca3415b910 2024-04-25T10:15:54.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c0a56a0 con 0x7fca3415b910 2024-04-25T10:15:54.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca32a00640 1 -- 172.21.15.87:0/1163348910 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24001680 con 0x7fca3415b910 2024-04-25T10:15:54.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- 172.21.15.87:0/1163348910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca3415b910 msgr2=0x7fca3415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 --2- 172.21.15.87:0/1163348910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca3415b910 0x7fca3415c030 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d90 tx=0x7fca2c01c860 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- 172.21.15.87:0/1163348910 shutdown_connections 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 --2- 172.21.15.87:0/1163348910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca3415b910 0x7fca3415c030 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- 172.21.15.87:0/1163348910 >> 172.21.15.87:0/1163348910 conn(0x7fca34066b70 msgr2=0x7fca34066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- 172.21.15.87:0/1163348910 shutdown_connections 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- 172.21.15.87:0/1163348910 wait complete. 2024-04-25T10:15:54.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 Processor -- start 2024-04-25T10:15:54.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- start start 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca340d0040 0x7fca34157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca341550b0 0x7fca340cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca3415b910 0x7fca340cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3415cf00 con 0x7fca341550b0 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3415cd80 con 0x7fca340d0040 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.663+0000 7fca3b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3415d080 con 0x7fca3415b910 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca341550b0 0x7fca340cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca341550b0 0x7fca340cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54468/0 (socket says 172.21.15.87:54468) 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 -- 172.21.15.87:0/2979238870 learned_addr learned my addr 172.21.15.87:0/2979238870 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33e00640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca340d0040 0x7fca34157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca39000640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca3415b910 0x7fca340cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 -- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca3415b910 msgr2=0x7fca340cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca3415b910 0x7fca340cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 -- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca340d0040 msgr2=0x7fca34157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca340d0040 0x7fca34157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 -- 172.21.15.87:0/2979238870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34135d00 con 0x7fca341550b0 2024-04-25T10:15:54.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca33400640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca341550b0 0x7fca340cda40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c430 tx=0x7fca2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca2be00640 1 -- 172.21.15.87:0/2979238870 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20017070 con 0x7fca341550b0 2024-04-25T10:15:54.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca2be00640 1 -- 172.21.15.87:0/2979238870 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca200052f0 con 0x7fca341550b0 2024-04-25T10:15:54.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca34135f70 con 0x7fca341550b0 2024-04-25T10:15:54.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca2be00640 1 -- 172.21.15.87:0/2979238870 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20003840 con 0x7fca341550b0 2024-04-25T10:15:54.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34136530 con 0x7fca341550b0 2024-04-25T10:15:54.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3404ef30 con 0x7fca341550b0 2024-04-25T10:15:54.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.667+0000 7fca2be00640 1 -- 172.21.15.87:0/2979238870 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca2001a070 con 0x7fca341550b0 2024-04-25T10:15:54.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.671+0000 7fca2be00640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca14066f90 0x7fca14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.671+0000 7fca2be00640 1 -- 172.21.15.87:0/2979238870 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fca200840c0 con 0x7fca341550b0 2024-04-25T10:15:54.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.671+0000 7fca33e00640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca14066f90 0x7fca14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.671+0000 7fca33e00640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca14066f90 0x7fca14069450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fca240011f0 tx=0x7fca24001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.675+0000 7fca2be00640 1 -- 172.21.15.87:0/2979238870 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca20054ce0 con 0x7fca341550b0 2024-04-25T10:15:54.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:54.879+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7fca34066f50 con 0x7fca341550b0 2024-04-25T10:15:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.135+0000 7fca2be00640 1 -- 172.21.15.87:0/2979238870 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v192) v1 ==== 143+0+0 (secure 0 0 0) 0x7fca20058d80 con 0x7fca341550b0 2024-04-25T10:15:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca14066f90 msgr2=0x7fca14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca14066f90 0x7fca14069450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fca240011f0 tx=0x7fca24001700 comp rx=0 tx=0).stop 2024-04-25T10:15:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca341550b0 msgr2=0x7fca340cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca341550b0 0x7fca340cda40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c430 tx=0x7fca2000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 shutdown_connections 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca14066f90 0x7fca14069450 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca3415b910 0x7fca340cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca341550b0 0x7fca340cda40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 --2- 172.21.15.87:0/2979238870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca340d0040 0x7fca34157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 >> 172.21.15.87:0/2979238870 conn(0x7fca34066b70 msgr2=0x7fca340b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 shutdown_connections 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.139+0000 7fca3b000640 1 -- 172.21.15.87:0/2979238870 wait complete. 2024-04-25T10:15:55.164 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:15:55.164 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:15:55.165 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:55.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef32a00640 1 Processor -- start 2024-04-25T10:15:55.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef32a00640 1 -- start start 2024-04-25T10:15:55.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef32a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef2c0d1820 0x7fef2c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef32a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2c0b1af0 con 0x7fef2c15e050 2024-04-25T10:15:55.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef32a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2c0b1dd0 con 0x7fef2c0b15b0 2024-04-25T10:15:55.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef32a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c0b20b0 con 0x7fef2c0d1820 2024-04-25T10:15:55.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fef2c15e050 0x7fef2c0b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40254/0 (socket says 172.21.15.87:40254) 2024-04-25T10:15:55.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2b400640 1 -- 172.21.15.87:0/3889462807 learned_addr learned my addr 172.21.15.87:0/3889462807 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:55.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2be00640 1 --2- 172.21.15.87:0/3889462807 >> v2:172.21.15.169:3300/0 conn(0x7fef2c0d1820 0x7fef2c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632082828 0 0) 0x7fef2c0b1af0 con 0x7fef2c15e050 2024-04-25T10:15:55.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef10003530 con 0x7fef2c15e050 2024-04-25T10:15:55.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810236531 0 0) 0x7fef2c0b1dd0 con 0x7fef2c0b15b0 2024-04-25T10:15:55.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef2c0b1af0 con 0x7fef2c0b15b0 2024-04-25T10:15:55.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2be00640 1 -- 172.21.15.87:0/3889462807 >> v1:172.21.15.169:6789/0 conn(0x7fef2c0b15b0 legacy=0x7fef2c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2be00640 1 -- 172.21.15.87:0/3889462807 >> v1:172.21.15.87:6789/0 conn(0x7fef2c15e050 legacy=0x7fef2c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2349233651 0 0) 0x7fef10003530 con 0x7fef2c15e050 2024-04-25T10:15:55.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.295+0000 7fef2be00640 1 -- 172.21.15.87:0/3889462807 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c0b2390 con 0x7fef2c0d1820 2024-04-25T10:15:55.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef2be00640 1 --2- 172.21.15.87:0/3889462807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0d1820 0x7fef2c0d1c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009d90 tx=0x7fef1c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938dc58cb314a4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0b2070 con 0x7fef2c0d1820 2024-04-25T10:15:55.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c0a56f0 con 0x7fef2c0d1820 2024-04-25T10:15:55.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef2aa00640 1 -- 172.21.15.87:0/3889462807 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18001680 con 0x7fef2c0d1820 2024-04-25T10:15:55.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef32a00640 1 -- 172.21.15.87:0/3889462807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0d1820 msgr2=0x7fef2c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef32a00640 1 --2- 172.21.15.87:0/3889462807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0d1820 0x7fef2c0d1c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009d90 tx=0x7fef1c01c890 comp rx=0 tx=0).stop 2024-04-25T10:15:55.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef32a00640 1 -- 172.21.15.87:0/3889462807 shutdown_connections 2024-04-25T10:15:55.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef32a00640 1 --2- 172.21.15.87:0/3889462807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0d1820 0x7fef2c0d1c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef32a00640 1 -- 172.21.15.87:0/3889462807 >> 172.21.15.87:0/3889462807 conn(0x7fef2c09a9d0 msgr2=0x7fef2c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef32a00640 1 -- 172.21.15.87:0/3889462807 shutdown_connections 2024-04-25T10:15:55.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.299+0000 7fef32a00640 1 -- 172.21.15.87:0/3889462807 wait complete. 2024-04-25T10:15:55.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 Processor -- start 2024-04-25T10:15:55.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 -- start start 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0b15b0 0x7fef2c152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef2c0d1820 0x7fef2c0c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2c15e050 0x7fef2c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0d0f60 con 0x7fef2c0d1820 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0d0de0 con 0x7fef2c0b15b0 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c16b9b0 con 0x7fef2c15e050 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef2c0d1820 0x7fef2c0c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef2c0d1820 0x7fef2c0c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54478/0 (socket says 172.21.15.87:54478) 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 -- 172.21.15.87:0/2490213626 learned_addr learned my addr 172.21.15.87:0/2490213626 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2be00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0b15b0 0x7fef2c152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef30a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2c15e050 0x7fef2c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 -- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2c15e050 msgr2=0x7fef2c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2c15e050 0x7fef2c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 -- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0b15b0 msgr2=0x7fef2c152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0b15b0 0x7fef2c152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 -- 172.21.15.87:0/2490213626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c0ca3f0 con 0x7fef2c0d1820 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef30a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2c15e050 0x7fef2c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef2b400640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef2c0d1820 0x7fef2c0c5430 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fef18000fe0 tx=0x7fef1800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef23e00640 1 -- 172.21.15.87:0/2490213626 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18017070 con 0x7fef2c0d1820 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef23e00640 1 -- 172.21.15.87:0/2490213626 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef18005ae0 con 0x7fef2c0d1820 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2c15fcd0 con 0x7fef2c0d1820 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef23e00640 1 -- 172.21.15.87:0/2490213626 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18007070 con 0x7fef2c0d1820 2024-04-25T10:15:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.303+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c160290 con 0x7fef2c0d1820 2024-04-25T10:15:55.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.307+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c04ef30 con 0x7fef2c0d1820 2024-04-25T10:15:55.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.307+0000 7fef23e00640 1 -- 172.21.15.87:0/2490213626 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef1801a070 con 0x7fef2c0d1820 2024-04-25T10:15:55.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.307+0000 7fef23e00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef0c066f60 0x7fef0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.307+0000 7fef23e00640 1 -- 172.21.15.87:0/2490213626 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fef18084140 con 0x7fef2c0d1820 2024-04-25T10:15:55.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.307+0000 7fef2be00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef0c066f60 0x7fef0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.307+0000 7fef2be00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef0c066f60 0x7fef0c069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0af000 tx=0x7fef1c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.311+0000 7fef23e00640 1 -- 172.21.15.87:0/2490213626 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef18054d60 con 0x7fef2c0d1820 2024-04-25T10:15:55.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.467+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef2c06b480 con 0x7fef2c0d1820 2024-04-25T10:15:55.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.467+0000 7fef23e00640 1 -- 172.21.15.87:0/2490213626 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+13010 (secure 0 0 0) 0x7fef18058e00 con 0x7fef2c0d1820 2024-04-25T10:15:55.474 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:55.474 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":49,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:54.132054+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/3236213442":"2024-04-26T10:15:51.120446+0000","172.21.15.169:6832/3236213442":"2024-04-26T10:15:51.120446+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:55.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef0c066f60 msgr2=0x7fef0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef0c066f60 0x7fef0c069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0af000 tx=0x7fef1c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:55.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef2c0d1820 msgr2=0x7fef2c0c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef2c0d1820 0x7fef2c0c5430 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fef18000fe0 tx=0x7fef1800cbe0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 shutdown_connections 2024-04-25T10:15:55.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef0c066f60 0x7fef0c069420 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2c15e050 0x7fef2c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef2c0d1820 0x7fef2c0c5430 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 --2- 172.21.15.87:0/2490213626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef2c0b15b0 0x7fef2c152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 >> 172.21.15.87:0/2490213626 conn(0x7fef2c09a9d0 msgr2=0x7fef2c158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 shutdown_connections 2024-04-25T10:15:55.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.471+0000 7fef32a00640 1 -- 172.21.15.87:0/2490213626 wait complete. 2024-04-25T10:15:55.495 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:55.495 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:55.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.627+0000 7f19cc800640 1 Processor -- start 2024-04-25T10:15:55.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.627+0000 7f19cc800640 1 -- start start 2024-04-25T10:15:55.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19cc800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19c40aef80 0x7f19c415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19cc800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19c40b31c0 0x7f19c40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19cc800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c415b080 con 0x7f19c4070350 2024-04-25T10:15:55.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19cc800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c415adb0 con 0x7f19c40aef80 2024-04-25T10:15:55.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19cc800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c415eb60 con 0x7f19c40b31c0 2024-04-25T10:15:55.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19c40aef80 0x7f19c415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c9400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f19c4070350 0x7f19c415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40268/0 (socket says 172.21.15.87:40268) 2024-04-25T10:15:55.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c9400640 1 -- 172.21.15.87:0/1826156834 learned_addr learned my addr 172.21.15.87:0/1826156834 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:55.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c9e00640 1 -- 172.21.15.87:0/1826156834 >> v2:172.21.15.169:3300/0 conn(0x7f19c40b31c0 msgr2=0x7f19c40b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19ca800640 1 --2- 172.21.15.87:0/1826156834 >> v2:172.21.15.169:3300/0 conn(0x7f19c40b31c0 0x7f19c40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c8a00640 1 -- 172.21.15.87:0/1826156834 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102943705 0 0) 0x7f19c415b080 con 0x7f19c4070350 2024-04-25T10:15:55.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c9e00640 1 --2- 172.21.15.87:0/1826156834 >> v2:172.21.15.169:3300/0 conn(0x7f19c40b31c0 0x7f19c40b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c9e00640 1 -- 172.21.15.87:0/1826156834 >> v1:172.21.15.87:6789/0 conn(0x7f19c4070350 legacy=0x7f19c415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19c9e00640 1 -- 172.21.15.87:0/1826156834 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c415ee40 con 0x7f19c40aef80 2024-04-25T10:15:55.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.631+0000 7f19ca800640 1 --2- 172.21.15.87:0/1826156834 >> v2:172.21.15.169:3300/0 conn(0x7f19c40b31c0 0x7f19c40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:55.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19c9e00640 1 --2- 172.21.15.87:0/1826156834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c40aef80 0x7f19c415e480 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f19c40d0750 tx=0x7f19b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a561dfc4e4a200 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19c8a00640 1 -- 172.21.15.87:0/1826156834 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80b1070 con 0x7f19c40aef80 2024-04-25T10:15:55.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19c8a00640 1 -- 172.21.15.87:0/1826156834 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b80ac5f0 con 0x7f19c40aef80 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19c8a00640 1 -- 172.21.15.87:0/1826156834 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80ac9c0 con 0x7f19c40aef80 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- 172.21.15.87:0/1826156834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c40aef80 msgr2=0x7f19c415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 --2- 172.21.15.87:0/1826156834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c40aef80 0x7f19c415e480 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f19c40d0750 tx=0x7f19b801c480 comp rx=0 tx=0).stop 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- 172.21.15.87:0/1826156834 shutdown_connections 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 --2- 172.21.15.87:0/1826156834 >> v2:172.21.15.169:3300/0 conn(0x7f19c40b31c0 0x7f19c40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 --2- 172.21.15.87:0/1826156834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c40aef80 0x7f19c415e480 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- 172.21.15.87:0/1826156834 >> 172.21.15.87:0/1826156834 conn(0x7f19c40669a0 msgr2=0x7f19c4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- 172.21.15.87:0/1826156834 shutdown_connections 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- 172.21.15.87:0/1826156834 wait complete. 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 Processor -- start 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- start start 2024-04-25T10:15:55.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c4070350 0x7f19c4152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19c40aef80 0x7f19c4167850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19c40b31c0 0x7f19c4167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4165990 con 0x7f19c4070350 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4165810 con 0x7f19c40aef80 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.635+0000 7f19cc800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4165b10 con 0x7f19c40b31c0 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c4070350 0x7f19c4152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c4070350 0x7f19c4152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54502/0 (socket says 172.21.15.87:54502) 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 -- 172.21.15.87:0/1334818430 learned_addr learned my addr 172.21.15.87:0/1334818430 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19ca800640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19c40b31c0 0x7f19c4167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9400640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19c40aef80 0x7f19c4167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 -- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19c40b31c0 msgr2=0x7f19c4167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19c40b31c0 0x7f19c4167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 -- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19c40aef80 msgr2=0x7f19c4167850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19c40aef80 0x7f19c4167850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 -- 172.21.15.87:0/1334818430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c41527d0 con 0x7f19c4070350 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19c9e00640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c4070350 0x7f19c4152290 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f19b801c960 tx=0x7f19b8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19be000640 1 -- 172.21.15.87:0/1334818430 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80b9020 con 0x7f19c4070350 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19be000640 1 -- 172.21.15.87:0/1334818430 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b80b5410 con 0x7f19c4070350 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c40b0b90 con 0x7f19c4070350 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19be000640 1 -- 172.21.15.87:0/1334818430 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80b1040 con 0x7f19c4070350 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c40b1150 con 0x7f19c4070350 2024-04-25T10:15:55.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c404ef30 con 0x7f19c4070350 2024-04-25T10:15:55.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.639+0000 7f19be000640 1 -- 172.21.15.87:0/1334818430 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19b80bc020 con 0x7f19c4070350 2024-04-25T10:15:55.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.643+0000 7f19be000640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19a8066f60 0x7f19a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.643+0000 7f19be000640 1 -- 172.21.15.87:0/1334818430 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f19b811e890 con 0x7f19c4070350 2024-04-25T10:15:55.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.643+0000 7f19c9400640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19a8066f60 0x7f19a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.643+0000 7f19c9400640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19a8066f60 0x7f19a8069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19c40720b0 tx=0x7f19b4007b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.647+0000 7f19be000640 1 -- 172.21.15.87:0/1334818430 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19b80c3030 con 0x7f19c4070350 2024-04-25T10:15:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.799+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19c415a840 con 0x7f19c4070350 2024-04-25T10:15:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19be000640 1 -- 172.21.15.87:0/1334818430 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19b80a9070 con 0x7f19c4070350 2024-04-25T10:15:55.807 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:15:55.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19a8066f60 msgr2=0x7f19a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19a8066f60 0x7f19a8069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19c40720b0 tx=0x7f19b4007b60 comp rx=0 tx=0).stop 2024-04-25T10:15:55.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c4070350 msgr2=0x7f19c4152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c4070350 0x7f19c4152290 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f19b801c960 tx=0x7f19b8019220 comp rx=0 tx=0).stop 2024-04-25T10:15:55.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 shutdown_connections 2024-04-25T10:15:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19a8066f60 0x7f19a8069420 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19c40b31c0 0x7f19c4167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19c40aef80 0x7f19c4167850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 --2- 172.21.15.87:0/1334818430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19c4070350 0x7f19c4152290 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 >> 172.21.15.87:0/1334818430 conn(0x7f19c40669a0 msgr2=0x7f19c40cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 shutdown_connections 2024-04-25T10:15:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.803+0000 7f19cc800640 1 -- 172.21.15.87:0/1334818430 wait complete. 2024-04-25T10:15:55.828 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:15:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.955+0000 7f84abe00640 1 Processor -- start 2024-04-25T10:15:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.955+0000 7f84abe00640 1 -- start start 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84abe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4146ed0 0x7f84a41472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84abe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84a40726b0 0x7f84a4071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84abe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a406c600 con 0x7f84a4071c80 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84abe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a40720f0 con 0x7f84a40726b0 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84abe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4161980 con 0x7f84a4146ed0 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84a40726b0 0x7f84a4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84a40726b0 0x7f84a4071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54516/0 (socket says 172.21.15.87:54516) 2024-04-25T10:15:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a8a00640 1 -- 172.21.15.87:0/1286445293 learned_addr learned my addr 172.21.15.87:0/1286445293 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a9400640 1 --2- 172.21.15.87:0/1286445293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4146ed0 0x7f84a41472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a8a00640 1 -- 172.21.15.87:0/1286445293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4146ed0 msgr2=0x7f84a41472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a8a00640 1 --2- 172.21.15.87:0/1286445293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4146ed0 0x7f84a41472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a8a00640 1 -- 172.21.15.87:0/1286445293 >> v1:172.21.15.169:6789/0 conn(0x7f84a4071c80 legacy=0x7f84a406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.959+0000 7f84a8a00640 1 -- 172.21.15.87:0/1286445293 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4161bd0 con 0x7f84a40726b0 2024-04-25T10:15:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84a8a00640 1 --2- 172.21.15.87:0/1286445293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a4071740 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f84a4161b00 tx=0x7f849801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c59f8da1c6e4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84a3e00640 1 -- 172.21.15.87:0/1286445293 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980aa5f0 con 0x7f84a40726b0 2024-04-25T10:15:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84a3e00640 1 -- 172.21.15.87:0/1286445293 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84980af070 con 0x7f84a40726b0 2024-04-25T10:15:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84a3e00640 1 -- 172.21.15.87:0/1286445293 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980aa9b0 con 0x7f84a40726b0 2024-04-25T10:15:55.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- 172.21.15.87:0/1286445293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 msgr2=0x7f84a4071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 --2- 172.21.15.87:0/1286445293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a4071740 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f84a4161b00 tx=0x7f849801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- 172.21.15.87:0/1286445293 shutdown_connections 2024-04-25T10:15:55.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 --2- 172.21.15.87:0/1286445293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a4071740 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 --2- 172.21.15.87:0/1286445293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4146ed0 0x7f84a41472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- 172.21.15.87:0/1286445293 >> 172.21.15.87:0/1286445293 conn(0x7f84a40669a0 msgr2=0x7f84a4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- 172.21.15.87:0/1286445293 shutdown_connections 2024-04-25T10:15:55.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- 172.21.15.87:0/1286445293 wait complete. 2024-04-25T10:15:55.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 Processor -- start 2024-04-25T10:15:55.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- start start 2024-04-25T10:15:55.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4071c80 0x7f84a40b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4146ed0 0x7f84a415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4147bf0 con 0x7f84a40726b0 2024-04-25T10:15:55.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4147a70 con 0x7f84a4146ed0 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84abe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40ae850 con 0x7f84a4071c80 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84a8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84a8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a40afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54530/0 (socket says 172.21.15.87:54530) 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.963+0000 7f84a8a00640 1 -- 172.21.15.87:0/3371371067 learned_addr learned my addr 172.21.15.87:0/3371371067 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a9400640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4071c80 0x7f84a40b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a9e00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4146ed0 0x7f84a415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a8a00640 1 -- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4071c80 msgr2=0x7f84a40b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a8a00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4071c80 0x7f84a40b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a8a00640 1 -- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4146ed0 msgr2=0x7f84a415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a8a00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4146ed0 0x7f84a415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a8a00640 1 -- 172.21.15.87:0/3371371067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a415d930 con 0x7f84a40726b0 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a8a00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a40afe80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f849801ab80 tx=0x7f849801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a1600640 1 -- 172.21.15.87:0/3371371067 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980bd5f0 con 0x7f84a40726b0 2024-04-25T10:15:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a1600640 1 -- 172.21.15.87:0/3371371067 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84980bd790 con 0x7f84a40726b0 2024-04-25T10:15:55.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a415ce50 con 0x7f84a40726b0 2024-04-25T10:15:55.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a1600640 1 -- 172.21.15.87:0/3371371067 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980bdb40 con 0x7f84a40726b0 2024-04-25T10:15:55.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a4141ee0 con 0x7f84a40726b0 2024-04-25T10:15:55.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a404ef30 con 0x7f84a40726b0 2024-04-25T10:15:55.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a1600640 1 -- 172.21.15.87:0/3371371067 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84980b33f0 con 0x7f84a40726b0 2024-04-25T10:15:55.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a1600640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8488066f60 0x7f8488069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.967+0000 7f84a1600640 1 -- 172.21.15.87:0/3371371067 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f849811f2d0 con 0x7f84a40726b0 2024-04-25T10:15:55.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.971+0000 7f84a9400640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8488066f60 0x7f8488069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.971+0000 7f84a9400640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8488066f60 0x7f8488069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f84940079c0 tx=0x7f8494008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:55.971+0000 7f84a1600640 1 -- 172.21.15.87:0/3371371067 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84980c3060 con 0x7f84a40726b0 2024-04-25T10:15:56.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:56.183+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f84a4072490 con 0x7f84a40726b0 2024-04-25T10:15:57.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.143+0000 7f84a1600640 1 -- 172.21.15.87:0/3371371067 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v194) v1 ==== 146+0+0 (secure 0 0 0) 0x7f84980160f0 con 0x7f84a40726b0 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8488066f60 msgr2=0x7f8488069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8488066f60 0x7f8488069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f84940079c0 tx=0x7f8494008040 comp rx=0 tx=0).stop 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 msgr2=0x7f84a40afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a40afe80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f849801ab80 tx=0x7f849801ad10 comp rx=0 tx=0).stop 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 shutdown_connections 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8488066f60 0x7f8488069420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4146ed0 0x7f84a415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a40726b0 0x7f84a40afe80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 --2- 172.21.15.87:0/3371371067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84a4071c80 0x7f84a40b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 >> 172.21.15.87:0/3371371067 conn(0x7f84a40669a0 msgr2=0x7f84a4140f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 shutdown_connections 2024-04-25T10:15:57.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.147+0000 7f84abe00640 1 -- 172.21.15.87:0/3371371067 wait complete. 2024-04-25T10:15:57.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:15:57.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.307+0000 7fcc3ce00640 1 Processor -- start 2024-04-25T10:15:57.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.307+0000 7fcc3ce00640 1 -- start start 2024-04-25T10:15:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.307+0000 7fcc3ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc38159540 0x7fcc380ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.307+0000 7fcc3ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc380aebd0 0x7fcc38071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.307+0000 7fcc3ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc38071360 con 0x7fcc38072020 2024-04-25T10:15:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.307+0000 7fcc3ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc38159920 con 0x7fcc38159540 2024-04-25T10:15:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.307+0000 7fcc3ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3815c2f0 con 0x7fcc380aebd0 2024-04-25T10:15:57.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc35e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc38159540 0x7fcc380ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc35400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc380aebd0 0x7fcc38071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc36800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcc38072020 0x7fcc3815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:34888/0 (socket says 172.21.15.87:34888) 2024-04-25T10:15:57.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc36800640 1 -- 172.21.15.87:0/2288428509 learned_addr learned my addr 172.21.15.87:0/2288428509 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:57.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc35e00640 1 -- 172.21.15.87:0/2288428509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc380aebd0 msgr2=0x7fcc38071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc35e00640 1 --2- 172.21.15.87:0/2288428509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc380aebd0 0x7fcc38071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc35e00640 1 -- 172.21.15.87:0/2288428509 >> v1:172.21.15.169:6789/0 conn(0x7fcc38072020 legacy=0x7fcc3815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc35e00640 1 -- 172.21.15.87:0/2288428509 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3815c5d0 con 0x7fcc38159540 2024-04-25T10:15:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.311+0000 7fcc34a00640 1 -- 172.21.15.87:0/2288428509 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456400692 0 0) 0x7fcc38071360 con 0x7fcc38072020 2024-04-25T10:15:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc35e00640 1 --2- 172.21.15.87:0/2288428509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc38159540 0x7fcc380ae690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009d90 tx=0x7fcc2801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215125a727b9fcbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc34a00640 1 -- 172.21.15.87:0/2288428509 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc280b2070 con 0x7fcc38159540 2024-04-25T10:15:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc34a00640 1 -- 172.21.15.87:0/2288428509 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc280a5840 con 0x7fcc38159540 2024-04-25T10:15:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc34a00640 1 -- 172.21.15.87:0/2288428509 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc280a5c10 con 0x7fcc38159540 2024-04-25T10:15:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- 172.21.15.87:0/2288428509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc38159540 msgr2=0x7fcc380ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/2288428509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc38159540 0x7fcc380ae690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009d90 tx=0x7fcc2801c970 comp rx=0 tx=0).stop 2024-04-25T10:15:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- 172.21.15.87:0/2288428509 shutdown_connections 2024-04-25T10:15:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/2288428509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc380aebd0 0x7fcc38071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/2288428509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc38159540 0x7fcc380ae690 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- 172.21.15.87:0/2288428509 >> 172.21.15.87:0/2288428509 conn(0x7fcc380669a0 msgr2=0x7fcc38066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- 172.21.15.87:0/2288428509 shutdown_connections 2024-04-25T10:15:57.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- 172.21.15.87:0/2288428509 wait complete. 2024-04-25T10:15:57.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 Processor -- start 2024-04-25T10:15:57.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- start start 2024-04-25T10:15:57.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc38072020 0x7fcc38154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc380aebd0 0x7fcc38159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc38159540 0x7fcc38141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38072430 con 0x7fcc38072020 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3806c0d0 con 0x7fcc380aebd0 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc3ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38168930 con 0x7fcc38159540 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc35e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc38072020 0x7fcc38154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc35e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc38072020 0x7fcc38154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54538/0 (socket says 172.21.15.87:54538) 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc35e00640 1 -- 172.21.15.87:0/707837592 learned_addr learned my addr 172.21.15.87:0/707837592 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc36800640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc38159540 0x7fcc38141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.315+0000 7fcc35400640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc380aebd0 0x7fcc38159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc35e00640 1 -- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc38159540 msgr2=0x7fcc38141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc35e00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc38159540 0x7fcc38141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc35e00640 1 -- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc380aebd0 msgr2=0x7fcc38159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc35e00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc380aebd0 0x7fcc38159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc35e00640 1 -- 172.21.15.87:0/707837592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38153340 con 0x7fcc38072020 2024-04-25T10:15:57.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc35e00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc38072020 0x7fcc38154e60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009d90 tx=0x7fcc28019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc2e000640 1 -- 172.21.15.87:0/707837592 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc280be020 con 0x7fcc38072020 2024-04-25T10:15:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc2e000640 1 -- 172.21.15.87:0/707837592 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc280b8470 con 0x7fcc38072020 2024-04-25T10:15:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc2e000640 1 -- 172.21.15.87:0/707837592 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc280b2040 con 0x7fcc38072020 2024-04-25T10:15:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc381535b0 con 0x7fcc38072020 2024-04-25T10:15:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc38153b70 con 0x7fcc38072020 2024-04-25T10:15:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3804ef30 con 0x7fcc38072020 2024-04-25T10:15:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc2e000640 1 -- 172.21.15.87:0/707837592 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc280c1070 con 0x7fcc38072020 2024-04-25T10:15:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc2e000640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc18066f60 0x7fcc18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc2e000640 1 -- 172.21.15.87:0/707837592 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcc2811f360 con 0x7fcc38072020 2024-04-25T10:15:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.319+0000 7fcc35400640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc18066f60 0x7fcc18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.323+0000 7fcc35400640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc18066f60 0x7fcc18069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcc381f9fa0 tx=0x7fcc24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.323+0000 7fcc2e000640 1 -- 172.21.15.87:0/707837592 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc28016560 con 0x7fcc38072020 2024-04-25T10:15:57.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.479+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcc380ae460 con 0x7fcc38072020 2024-04-25T10:15:57.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc2e000640 1 -- 172.21.15.87:0/707837592 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcc280f05d0 con 0x7fcc38072020 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc18066f60 msgr2=0x7fcc18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc18066f60 0x7fcc18069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcc381f9fa0 tx=0x7fcc24008040 comp rx=0 tx=0).stop 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc38072020 msgr2=0x7fcc38154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc38072020 0x7fcc38154e60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009d90 tx=0x7fcc28019710 comp rx=0 tx=0).stop 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 shutdown_connections 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc18066f60 0x7fcc18069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc38159540 0x7fcc38141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc380aebd0 0x7fcc38159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 --2- 172.21.15.87:0/707837592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc38072020 0x7fcc38154e60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 >> 172.21.15.87:0/707837592 conn(0x7fcc380669a0 msgr2=0x7fcc38066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 shutdown_connections 2024-04-25T10:15:57.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.483+0000 7fcc3ce00640 1 -- 172.21.15.87:0/707837592 wait complete. 2024-04-25T10:15:57.508 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:57.509 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:57.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:57.662 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:57.662 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:57.663 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:57.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:57.671 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:15:57.671 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:57.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:57.680 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:15:57.680 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:57.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:57.723 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:15:57.724 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:57.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:57.736 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:15:57.736 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:57.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 Processor -- start 2024-04-25T10:15:57.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 -- start start 2024-04-25T10:15:57.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f144c156150 0x7f144c071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c15c080 0x7f144c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f144c066ee0 0x7f144c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c067350 con 0x7f144c066ee0 2024-04-25T10:15:57.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c156530 con 0x7f144c156150 2024-04-25T10:15:57.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f1451800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c15d220 con 0x7f144c15c080 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f144c066ee0 0x7f144c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f144c066ee0 0x7f144c0af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44418/0 (socket says 172.21.15.87:44418) 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 -- 172.21.15.87:0/2557072982 learned_addr learned my addr 172.21.15.87:0/2557072982 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144a000640 1 --2- 172.21.15.87:0/2557072982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c15c080 0x7f144c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144aa00640 1 --2- 172.21.15.87:0/2557072982 >> v2:172.21.15.169:3300/0 conn(0x7f144c156150 0x7f144c071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 -- 172.21.15.87:0/2557072982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c15c080 msgr2=0x7f144c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 --2- 172.21.15.87:0/2557072982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c15c080 0x7f144c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 -- 172.21.15.87:0/2557072982 >> v2:172.21.15.169:3300/0 conn(0x7f144c156150 msgr2=0x7f144c071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 --2- 172.21.15.87:0/2557072982 >> v2:172.21.15.169:3300/0 conn(0x7f144c156150 0x7f144c071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.895+0000 7f144b400640 1 -- 172.21.15.87:0/2557072982 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c15c640 con 0x7f144c066ee0 2024-04-25T10:15:57.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f144b400640 1 --2- 172.21.15.87:0/2557072982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c066ee0 0x7f144c0af490 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f144c070ab0 tx=0x7f144401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5a41c781c1d18e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1449600640 1 -- 172.21.15.87:0/2557072982 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440a5750 con 0x7f144c066ee0 2024-04-25T10:15:57.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1449600640 1 -- 172.21.15.87:0/2557072982 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14440a58f0 con 0x7f144c066ee0 2024-04-25T10:15:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1449600640 1 -- 172.21.15.87:0/2557072982 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440a5cc0 con 0x7f144c066ee0 2024-04-25T10:15:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 -- 172.21.15.87:0/2557072982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c066ee0 msgr2=0x7f144c0af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 --2- 172.21.15.87:0/2557072982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c066ee0 0x7f144c0af490 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f144c070ab0 tx=0x7f144401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 -- 172.21.15.87:0/2557072982 shutdown_connections 2024-04-25T10:15:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 --2- 172.21.15.87:0/2557072982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c066ee0 0x7f144c0af490 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 --2- 172.21.15.87:0/2557072982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c15c080 0x7f144c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 --2- 172.21.15.87:0/2557072982 >> v2:172.21.15.169:3300/0 conn(0x7f144c156150 0x7f144c071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 -- 172.21.15.87:0/2557072982 >> 172.21.15.87:0/2557072982 conn(0x7f144c06ac90 msgr2=0x7f144c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 -- 172.21.15.87:0/2557072982 shutdown_connections 2024-04-25T10:15:57.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.899+0000 7f1451800640 1 -- 172.21.15.87:0/2557072982 wait complete. 2024-04-25T10:15:57.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 Processor -- start 2024-04-25T10:15:57.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 -- start start 2024-04-25T10:15:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c066ee0 0x7f144c1414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c156150 0x7f144c14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c15c080 0x7f144c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c06d430 con 0x7f144c15c080 2024-04-25T10:15:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c072470 con 0x7f144c066ee0 2024-04-25T10:15:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c06d5b0 con 0x7f144c156150 2024-04-25T10:15:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c15c080 0x7f144c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c15c080 0x7f144c0b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44424/0 (socket says 172.21.15.87:44424) 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 -- 172.21.15.87:0/372606524 learned_addr learned my addr 172.21.15.87:0/372606524 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144a000640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c156150 0x7f144c14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144aa00640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c066ee0 0x7f144c1414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 -- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c156150 msgr2=0x7f144c14d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c156150 0x7f144c14d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 -- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c066ee0 msgr2=0x7f144c1414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c066ee0 0x7f144c1414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 -- 172.21.15.87:0/372606524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c144a30 con 0x7f144c15c080 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f144b400640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c15c080 0x7f144c0b17b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f14440af040 tx=0x7f1444019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1442a00640 1 -- 172.21.15.87:0/372606524 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440b8670 con 0x7f144c15c080 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1442a00640 1 -- 172.21.15.87:0/372606524 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14440be070 con 0x7f144c15c080 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c0b3a70 con 0x7f144c15c080 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1442a00640 1 -- 172.21.15.87:0/372606524 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440b8a70 con 0x7f144c15c080 2024-04-25T10:15:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c14da80 con 0x7f144c15c080 2024-04-25T10:15:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.903+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c04ef30 con 0x7f144c15c080 2024-04-25T10:15:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.907+0000 7f1442a00640 1 -- 172.21.15.87:0/372606524 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14440b8c10 con 0x7f144c15c080 2024-04-25T10:15:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.907+0000 7f1442a00640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1428066f60 0x7f1428069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.907+0000 7f1442a00640 1 -- 172.21.15.87:0/372606524 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1444120060 con 0x7f144c15c080 2024-04-25T10:15:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.907+0000 7f144aa00640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1428066f60 0x7f1428069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.907+0000 7f144aa00640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1428066f60 0x7f1428069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1434009710 tx=0x7f1434008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:57.911+0000 7f1442a00640 1 -- 172.21.15.87:0/372606524 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14440165e0 con 0x7f144c15c080 2024-04-25T10:15:58.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f144c06c3f0 con 0x7f144c15c080 2024-04-25T10:15:58.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1442a00640 1 -- 172.21.15.87:0/372606524 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f14440f0db0 con 0x7f144c15c080 2024-04-25T10:15:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:58.105 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/432143519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":432143519},{"type":"v1","addr":"172.21.15.169:6835","nonce":432143519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1603456809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1603456809},{"type":"v1","addr":"172.21.15.169:6834","nonce":1603456809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4184691479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4184691479},{"type":"v1","addr":"172.21.15.87:6836","nonce":4184691479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2742997755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2742997755},{"type":"v1","addr":"172.21.15.87:6837","nonce":2742997755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[]} 2024-04-25T10:15:58.105 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 30 2024-04-25T10:15:58.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1428066f60 msgr2=0x7f1428069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1428066f60 0x7f1428069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1434009710 tx=0x7f1434008040 comp rx=0 tx=0).stop 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c15c080 msgr2=0x7f144c0b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c15c080 0x7f144c0b17b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f14440af040 tx=0x7f1444019760 comp rx=0 tx=0).stop 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 shutdown_connections 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1428066f60 0x7f1428069420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c15c080 0x7f144c0b17b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f144c156150 0x7f144c14d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 --2- 172.21.15.87:0/372606524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f144c066ee0 0x7f144c1414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 >> 172.21.15.87:0/372606524 conn(0x7f144c06ac90 msgr2=0x7f144c0d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 shutdown_connections 2024-04-25T10:15:58.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.099+0000 7f1451800640 1 -- 172.21.15.87:0/372606524 wait complete. 2024-04-25T10:15:58.130 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:15:58.130 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:15:58.131 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:15:58.131 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:58.131 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:15:58.131 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:15:58.131 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:15:58.131 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:58.132 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:15:58.132 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:15:58.132 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:58.132 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:15:58.132 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:15:58.133 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:15:58.133 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:58.143 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:15:58.143 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:15:58.145 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:15:58.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:15:58.146 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:15:58.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:15:58.148 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:15:58.148 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:15:58.189 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:15:58.189 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:58.191 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:15:58.191 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:58.193 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:15:58.193 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:58.195 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:15:58.195 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:58.196 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:15:58.275 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:15:58.276 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:15:58.284 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:15:58.284 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:15:58.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6c00640 1 Processor -- start 2024-04-25T10:15:58.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6c00640 1 -- start start 2024-04-25T10:15:58.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5dd0152a40 0x7f5dd015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd006c7e0 con 0x7f5dd0072190 2024-04-25T10:15:58.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd00728c0 con 0x7f5dd006c370 2024-04-25T10:15:58.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd006d2f0 con 0x7f5dd0152a40 2024-04-25T10:15:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5dd0072190 0x7f5dd015a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36488/0 (socket says 172.21.15.87:36488) 2024-04-25T10:15:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd6200640 1 -- 172.21.15.87:0/3910904412 learned_addr learned my addr 172.21.15.87:0/3910904412 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd5800640 1 --2- 172.21.15.87:0/3910904412 >> v2:172.21.15.87:3300/0 conn(0x7f5dd0152a40 0x7f5dd015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dcfe00640 1 -- 172.21.15.87:0/3910904412 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738267899 0 0) 0x7f5dd006c7e0 con 0x7f5dd0072190 2024-04-25T10:15:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dcfe00640 1 -- 172.21.15.87:0/3910904412 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db8003530 con 0x7f5dd0072190 2024-04-25T10:15:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd5800640 1 -- 172.21.15.87:0/3910904412 >> v1:172.21.15.169:6789/0 conn(0x7f5dd006c370 legacy=0x7f5dd0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd5800640 1 -- 172.21.15.87:0/3910904412 >> v1:172.21.15.87:6789/0 conn(0x7f5dd0072190 legacy=0x7f5dd015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.335+0000 7f5dd5800640 1 -- 172.21.15.87:0/3910904412 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd006ca50 con 0x7f5dd0152a40 2024-04-25T10:15:58.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd5800640 1 --2- 172.21.15.87:0/3910904412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0152a40 0x7f5dd015bf50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40097c0 tx=0x7f5dc401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5003dc971b2bf99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dcfe00640 1 -- 172.21.15.87:0/3910904412 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40b1070 con 0x7f5dd0152a40 2024-04-25T10:15:58.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dcfe00640 1 -- 172.21.15.87:0/3910904412 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc40ac640 con 0x7f5dd0152a40 2024-04-25T10:15:58.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dcfe00640 1 -- 172.21.15.87:0/3910904412 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40aca10 con 0x7f5dd0152a40 2024-04-25T10:15:58.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd6c00640 1 -- 172.21.15.87:0/3910904412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0152a40 msgr2=0x7f5dd015bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/3910904412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0152a40 0x7f5dd015bf50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40097c0 tx=0x7f5dc401c310 comp rx=0 tx=0).stop 2024-04-25T10:15:58.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd6c00640 1 -- 172.21.15.87:0/3910904412 shutdown_connections 2024-04-25T10:15:58.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/3910904412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0152a40 0x7f5dd015bf50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd6c00640 1 -- 172.21.15.87:0/3910904412 >> 172.21.15.87:0/3910904412 conn(0x7f5dd00b16a0 msgr2=0x7f5dd00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd6c00640 1 -- 172.21.15.87:0/3910904412 shutdown_connections 2024-04-25T10:15:58.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.339+0000 7f5dd6c00640 1 -- 172.21.15.87:0/3910904412 wait complete. 2024-04-25T10:15:58.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 Processor -- start 2024-04-25T10:15:58.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 -- start start 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd006c370 0x7f5dd0157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0072190 0x7f5dd0147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd0152a40 0x7f5dd015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd015b130 con 0x7f5dd0072190 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0152e20 con 0x7f5dd006c370 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd015b2b0 con 0x7f5dd0152a40 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0072190 0x7f5dd0147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0072190 0x7f5dd0147460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44478/0 (socket says 172.21.15.87:44478) 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 -- 172.21.15.87:0/458599045 learned_addr learned my addr 172.21.15.87:0/458599045 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6200640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd0152a40 0x7f5dd015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd5800640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd006c370 0x7f5dd0157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 -- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd0152a40 msgr2=0x7f5dd015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd0152a40 0x7f5dd015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 -- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd006c370 msgr2=0x7f5dd0157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd006c370 0x7f5dd0157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 -- 172.21.15.87:0/458599045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd015f5a0 con 0x7f5dd0072190 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd4e00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0072190 0x7f5dd0147460 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c450 tx=0x7f5dbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dcd600640 1 -- 172.21.15.87:0/458599045 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc012830 con 0x7f5dd0072190 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dcd600640 1 -- 172.21.15.87:0/458599045 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc017070 con 0x7f5dd0072190 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dcd600640 1 -- 172.21.15.87:0/458599045 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc012c30 con 0x7f5dd0072190 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd015f880 con 0x7f5dd0072190 2024-04-25T10:15:58.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.343+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd01479a0 con 0x7f5dd0072190 2024-04-25T10:15:58.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:15:58.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:15:58.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.347+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd0075120 con 0x7f5dd0072190 2024-04-25T10:15:58.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.347+0000 7f5dcd600640 1 -- 172.21.15.87:0/458599045 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dbc01c460 con 0x7f5dd0072190 2024-04-25T10:15:58.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.347+0000 7f5dcd600640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5db0066f60 0x7f5db0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.347+0000 7f5dcd600640 1 -- 172.21.15.87:0/458599045 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5dbc084b80 con 0x7f5dd0072190 2024-04-25T10:15:58.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.347+0000 7f5dd5800640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5db0066f60 0x7f5db0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.347+0000 7f5dd5800640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5db0066f60 0x7f5db0069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40ae000 tx=0x7f5dc4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.351+0000 7f5dcd600640 1 -- 172.21.15.87:0/458599045 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dbc055e80 con 0x7f5dd0072190 2024-04-25T10:15:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:58.515+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5dd006d620 con 0x7f5dd0072190 2024-04-25T10:15:59.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.167+0000 7f5dcd600640 1 -- 172.21.15.87:0/458599045 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5dbc055780 con 0x7f5dd0072190 2024-04-25T10:15:59.174 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:15:59.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5db0066f60 msgr2=0x7f5db0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5db0066f60 0x7f5db0069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40ae000 tx=0x7f5dc4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0072190 msgr2=0x7f5dd0147460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0072190 0x7f5dd0147460 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c450 tx=0x7f5dbc00c910 comp rx=0 tx=0).stop 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 shutdown_connections 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5db0066f60 0x7f5db0069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd0152a40 0x7f5dd015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dd0072190 0x7f5dd0147460 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 --2- 172.21.15.87:0/458599045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd006c370 0x7f5dd0157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 >> 172.21.15.87:0/458599045 conn(0x7f5dd00b16a0 msgr2=0x7f5dd0149740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 shutdown_connections 2024-04-25T10:15:59.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.171+0000 7f5dd6c00640 1 -- 172.21.15.87:0/458599045 wait complete. 2024-04-25T10:15:59.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:15:59.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc889e00640 1 Processor -- start 2024-04-25T10:15:59.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc889e00640 1 -- start start 2024-04-25T10:15:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc889e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc88415dd20 0x7fc8840b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc889e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8840b22a0 con 0x7fc88406bcb0 2024-04-25T10:15:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc889e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8840b2580 con 0x7fc88406b0d0 2024-04-25T10:15:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc889e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8840b2860 con 0x7fc88415dd20 2024-04-25T10:15:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc883400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc88406bcb0 0x7fc8840ae610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36494/0 (socket says 172.21.15.87:36494) 2024-04-25T10:15:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc883400640 1 -- 172.21.15.87:0/3403738036 learned_addr learned my addr 172.21.15.87:0/3403738036 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:15:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.331+0000 7fc882a00640 1 --2- 172.21.15.87:0/3403738036 >> v2:172.21.15.87:3300/0 conn(0x7fc88415dd20 0x7fc8840b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882000640 1 -- 172.21.15.87:0/3403738036 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786389200 0 0) 0x7fc8840b22a0 con 0x7fc88406bcb0 2024-04-25T10:15:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882000640 1 -- 172.21.15.87:0/3403738036 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc868003530 con 0x7fc88406bcb0 2024-04-25T10:15:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882a00640 1 -- 172.21.15.87:0/3403738036 >> v1:172.21.15.169:6789/0 conn(0x7fc88406b0d0 legacy=0x7fc88406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882a00640 1 -- 172.21.15.87:0/3403738036 >> v1:172.21.15.87:6789/0 conn(0x7fc88406bcb0 legacy=0x7fc8840ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882a00640 1 -- 172.21.15.87:0/3403738036 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840b2b40 con 0x7fc88415dd20 2024-04-25T10:15:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882a00640 1 --2- 172.21.15.87:0/3403738036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88415dd20 0x7fc8840b19e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc874009d90 tx=0x7fc87401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9800b7d5ce4a0d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882000640 1 -- 172.21.15.87:0/3403738036 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740b2070 con 0x7fc88415dd20 2024-04-25T10:15:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882000640 1 -- 172.21.15.87:0/3403738036 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8740a5a20 con 0x7fc88415dd20 2024-04-25T10:15:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc882000640 1 -- 172.21.15.87:0/3403738036 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740a5df0 con 0x7fc88415dd20 2024-04-25T10:15:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc889e00640 1 -- 172.21.15.87:0/3403738036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88415dd20 msgr2=0x7fc8840b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.335+0000 7fc889e00640 1 --2- 172.21.15.87:0/3403738036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88415dd20 0x7fc8840b19e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc874009d90 tx=0x7fc87401c940 comp rx=0 tx=0).stop 2024-04-25T10:15:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- 172.21.15.87:0/3403738036 shutdown_connections 2024-04-25T10:15:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 --2- 172.21.15.87:0/3403738036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88415dd20 0x7fc8840b19e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- 172.21.15.87:0/3403738036 >> 172.21.15.87:0/3403738036 conn(0x7fc8840d1900 msgr2=0x7fc8840d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- 172.21.15.87:0/3403738036 shutdown_connections 2024-04-25T10:15:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- 172.21.15.87:0/3403738036 wait complete. 2024-04-25T10:15:59.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 Processor -- start 2024-04-25T10:15:59.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- start start 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88406b0d0 0x7fc884155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc88406bcb0 0x7fc88415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc88415dd20 0x7fc8841547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88416bb20 con 0x7fc88406b0d0 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840af780 con 0x7fc88415dd20 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88416bca0 con 0x7fc88406bcb0 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88406b0d0 0x7fc884155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88406b0d0 0x7fc884155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44492/0 (socket says 172.21.15.87:44492) 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 -- 172.21.15.87:0/2297876988 learned_addr learned my addr 172.21.15.87:0/2297876988 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc883400640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc88415dd20 0x7fc8841547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc87a000640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc88406bcb0 0x7fc88415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 -- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc88406bcb0 msgr2=0x7fc88415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc88406bcb0 0x7fc88415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 -- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc88415dd20 msgr2=0x7fc8841547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc88415dd20 0x7fc8841547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 -- 172.21.15.87:0/2297876988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884158690 con 0x7fc88406b0d0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc882a00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88406b0d0 0x7fc884155320 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc8740b0040 tx=0x7fc8740196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc87be00640 1 -- 172.21.15.87:0/2297876988 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740b2040 con 0x7fc88406b0d0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc87be00640 1 -- 172.21.15.87:0/2297876988 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8740bf070 con 0x7fc88406b0d0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc884158970 con 0x7fc88406b0d0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.339+0000 7fc87be00640 1 -- 172.21.15.87:0/2297876988 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740b8870 con 0x7fc88406b0d0 2024-04-25T10:15:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.343+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8840b48d0 con 0x7fc88406b0d0 2024-04-25T10:15:59.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.343+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88404ef30 con 0x7fc88406b0d0 2024-04-25T10:15:59.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.343+0000 7fc87be00640 1 -- 172.21.15.87:0/2297876988 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8740b8a10 con 0x7fc88406b0d0 2024-04-25T10:15:59.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.343+0000 7fc87be00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc860066f90 0x7fc860069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.343+0000 7fc87be00640 1 -- 172.21.15.87:0/2297876988 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc87411fbf0 con 0x7fc88406b0d0 2024-04-25T10:15:59.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.343+0000 7fc87a000640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc860066f90 0x7fc860069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.343+0000 7fc87a000640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc860066f90 0x7fc860069450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc88415d740 tx=0x7fc86c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.347+0000 7fc87be00640 1 -- 172.21.15.87:0/2297876988 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc874016560 con 0x7fc88406b0d0 2024-04-25T10:15:59.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc8840b7780 con 0x7fc88406b0d0 2024-04-25T10:15:59.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc87be00640 1 -- 172.21.15.87:0/2297876988 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc8740b8d00 con 0x7fc88406b0d0 2024-04-25T10:15:59.560 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:59.560 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:15:59.560 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.561 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:59.562 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:59.563 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:59.565 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.566 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.567 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:15:59.568 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:15:59.570 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:15:59.572 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:15:59.573 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.574 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:15:59.575 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:15:59.576 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:15:59.576 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:59.576 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:15:59.576 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.576 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:15:59.576 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:15:59.576 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:15:59.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc860066f90 msgr2=0x7fc860069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc860066f90 0x7fc860069450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc88415d740 tx=0x7fc86c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:59.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88406b0d0 msgr2=0x7fc884155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88406b0d0 0x7fc884155320 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc8740b0040 tx=0x7fc8740196e0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 shutdown_connections 2024-04-25T10:15:59.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc860066f90 0x7fc860069450 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc88415dd20 0x7fc8841547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc88406bcb0 0x7fc88415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 --2- 172.21.15.87:0/2297876988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc88406b0d0 0x7fc884155320 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 >> 172.21.15.87:0/2297876988 conn(0x7fc8840d1900 msgr2=0x7fc88406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 shutdown_connections 2024-04-25T10:15:59.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.555+0000 7fc889e00640 1 -- 172.21.15.87:0/2297876988 wait complete. 2024-04-25T10:15:59.581 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:59.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b83a00640 1 Processor -- start 2024-04-25T10:15:59.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b83a00640 1 -- start start 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b83a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b7c15bc40 0x7f9b7c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b83a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cf930 0x7f9b7c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b83a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b7c0b2770 con 0x7f9b7c0cda10 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b83a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b7c06c670 con 0x7f9b7c15bc40 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b83a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c06c950 con 0x7f9b7c0cf930 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b81000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b7c15bc40 0x7f9b7c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b81000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b7c15bc40 0x7f9b7c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44502/0 (socket says 172.21.15.87:44502) 2024-04-25T10:15:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.711+0000 7f9b81000640 1 -- 172.21.15.87:0/3461766894 learned_addr learned my addr 172.21.15.87:0/3461766894 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b81000640 1 -- 172.21.15.87:0/3461766894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cf930 msgr2=0x7f9b7c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b81a00640 1 --2- 172.21.15.87:0/3461766894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cf930 0x7f9b7c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b81000640 1 --2- 172.21.15.87:0/3461766894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cf930 0x7f9b7c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b81000640 1 -- 172.21.15.87:0/3461766894 >> v1:172.21.15.169:6789/0 conn(0x7f9b7c0cda10 legacy=0x7f9b7c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b81000640 1 -- 172.21.15.87:0/3461766894 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c06cc30 con 0x7f9b7c15bc40 2024-04-25T10:15:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b81a00640 1 --2- 172.21.15.87:0/3461766894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cf930 0x7f9b7c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:59.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b81000640 1 --2- 172.21.15.87:0/3461766894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c06c090 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009870 tx=0x7f9b6c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806df669e4e38646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b7be00640 1 -- 172.21.15.87:0/3461766894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0b0070 con 0x7f9b7c15bc40 2024-04-25T10:15:59.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b7be00640 1 -- 172.21.15.87:0/3461766894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0ab600 con 0x7f9b7c15bc40 2024-04-25T10:15:59.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b7be00640 1 -- 172.21.15.87:0/3461766894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0ab9d0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b83a00640 1 -- 172.21.15.87:0/3461766894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 msgr2=0x7f9b7c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.715+0000 7f9b83a00640 1 --2- 172.21.15.87:0/3461766894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c06c090 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009870 tx=0x7f9b6c01c490 comp rx=0 tx=0).stop 2024-04-25T10:15:59.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- 172.21.15.87:0/3461766894 shutdown_connections 2024-04-25T10:15:59.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 --2- 172.21.15.87:0/3461766894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cf930 0x7f9b7c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 --2- 172.21.15.87:0/3461766894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c06c090 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- 172.21.15.87:0/3461766894 >> 172.21.15.87:0/3461766894 conn(0x7f9b7c066b70 msgr2=0x7f9b7c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- 172.21.15.87:0/3461766894 shutdown_connections 2024-04-25T10:15:59.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- 172.21.15.87:0/3461766894 wait complete. 2024-04-25T10:15:59.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 Processor -- start 2024-04-25T10:15:59.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- start start 2024-04-25T10:15:59.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cda10 0x7f9b7c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c0cf930 0x7f9b7c15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c0b3890 con 0x7f9b7c15bc40 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c0b3710 con 0x7f9b7c0cf930 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b83a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c0b3a10 con 0x7f9b7c0cda10 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44506/0 (socket says 172.21.15.87:44506) 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 -- 172.21.15.87:0/1762753427 learned_addr learned my addr 172.21.15.87:0/1762753427 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81000640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cda10 0x7f9b7c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b73e00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c0cf930 0x7f9b7c15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 -- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cda10 msgr2=0x7f9b7c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cda10 0x7f9b7c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 -- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c0cf930 msgr2=0x7f9b7c15ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c0cf930 0x7f9b7c15ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 -- 172.21.15.87:0/1762753427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0b1af0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b81a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c156e00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400cd80 tx=0x7f9b7400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.719+0000 7f9b79600640 1 -- 172.21.15.87:0/1762753427 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400f680 con 0x7f9b7c15bc40 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b79600640 1 -- 172.21.15.87:0/1762753427 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7400f840 con 0x7f9b7c15bc40 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7c15f2b0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b79600640 1 -- 172.21.15.87:0/1762753427 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400fc30 con 0x7f9b7c15bc40 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c152dc0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7c152be0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b79600640 1 -- 172.21.15.87:0/1762753427 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b740213f0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b79600640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b58066f90 0x7f9b58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b79600640 1 -- 172.21.15.87:0/1762753427 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b740858e0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b81000640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b58066f90 0x7f9b58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.723+0000 7f9b81000640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b58066f90 0x7f9b58069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0ad000 tx=0x7f9b6c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.727+0000 7f9b79600640 1 -- 172.21.15.87:0/1762753427 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b7c152be0 con 0x7f9b7c15bc40 2024-04-25T10:15:59.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.883+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b7c0aef80 con 0x7f9b7c15bc40 2024-04-25T10:15:59.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b79600640 1 -- 172.21.15.87:0/1762753427 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f9b74014030 con 0x7f9b7c15bc40 2024-04-25T10:15:59.892 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:15:59.893 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":53,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:15:59.181229+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b58066f90 msgr2=0x7f9b58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b58066f90 0x7f9b58069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0ad000 tx=0x7f9b6c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 msgr2=0x7f9b7c156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c156e00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400cd80 tx=0x7f9b7400a650 comp rx=0 tx=0).stop 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 shutdown_connections 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b58066f90 0x7f9b58069450 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b7c15bc40 0x7f9b7c156e00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7c0cf930 0x7f9b7c15ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 --2- 172.21.15.87:0/1762753427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7c0cda10 0x7f9b7c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 >> 172.21.15.87:0/1762753427 conn(0x7f9b7c066b70 msgr2=0x7f9b7c159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 shutdown_connections 2024-04-25T10:15:59.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:15:59.887+0000 7f9b83a00640 1 -- 172.21.15.87:0/1762753427 wait complete. 2024-04-25T10:15:59.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:59.913 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:00.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb42600640 1 Processor -- start 2024-04-25T10:16:00.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb42600640 1 -- start start 2024-04-25T10:16:00.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb42600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb42600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb3c0aebd0 0x7fcb3c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb42600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c071360 con 0x7fcb3c072020 2024-04-25T10:16:00.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb42600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c0af030 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb42600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c159920 con 0x7fcb3c159540 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb33e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb3c0aebd0 0x7fcb3c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb3b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb33e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb3c0aebd0 0x7fcb3c071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:51272/0 (socket says 172.21.15.87:51272) 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb3be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcb3c072020 0x7fcb3c15f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:44340/0 (socket says 172.21.15.87:44340) 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb33e00640 1 -- 172.21.15.87:0/3436421202 learned_addr learned my addr 172.21.15.87:0/3436421202 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb33e00640 1 -- 172.21.15.87:0/3436421202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 msgr2=0x7fcb3c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb33e00640 1 --2- 172.21.15.87:0/3436421202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb33e00640 1 -- 172.21.15.87:0/3436421202 >> v1:172.21.15.169:6789/0 conn(0x7fcb3c072020 legacy=0x7fcb3c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.071+0000 7fcb33e00640 1 -- 172.21.15.87:0/3436421202 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c15c2f0 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb33e00640 1 --2- 172.21.15.87:0/3436421202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c071ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009870 tx=0x7fcb2401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=279dca77b23858fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb3aa00640 1 -- 172.21.15.87:0/3436421202 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240ab650 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb3aa00640 1 -- 172.21.15.87:0/3436421202 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb240b0070 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb3aa00640 1 -- 172.21.15.87:0/3436421202 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240aba10 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb42600640 1 -- 172.21.15.87:0/3436421202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c0aebd0 msgr2=0x7fcb3c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb42600640 1 --2- 172.21.15.87:0/3436421202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c071ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009870 tx=0x7fcb2401c530 comp rx=0 tx=0).stop 2024-04-25T10:16:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb42600640 1 -- 172.21.15.87:0/3436421202 shutdown_connections 2024-04-25T10:16:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb42600640 1 --2- 172.21.15.87:0/3436421202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c071ae0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb42600640 1 --2- 172.21.15.87:0/3436421202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb42600640 1 -- 172.21.15.87:0/3436421202 >> 172.21.15.87:0/3436421202 conn(0x7fcb3c0669a0 msgr2=0x7fcb3c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.075+0000 7fcb42600640 1 -- 172.21.15.87:0/3436421202 shutdown_connections 2024-04-25T10:16:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.079+0000 7fcb42600640 1 -- 172.21.15.87:0/3436421202 wait complete. 2024-04-25T10:16:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.079+0000 7fcb42600640 1 Processor -- start 2024-04-25T10:16:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.079+0000 7fcb42600640 1 -- start start 2024-04-25T10:16:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c072020 0x7fcb3c13e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c06c0d0 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c15bfa0 con 0x7fcb3c072020 2024-04-25T10:16:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c072430 con 0x7fcb3c159540 2024-04-25T10:16:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c0b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44510/0 (socket says 172.21.15.87:44510) 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 -- 172.21.15.87:0/1100868999 learned_addr learned my addr 172.21.15.87:0/1100868999 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb3b400640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c072020 0x7fcb3c13e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb3be00640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 -- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 msgr2=0x7fcb3c15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c15acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 -- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c072020 msgr2=0x7fcb3c13e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c072020 0x7fcb3c13e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 -- 172.21.15.87:0/1100868999 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c13e640 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33e00640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c0b2870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401abe0 tx=0x7fcb2401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33400640 1 -- 172.21.15.87:0/1100868999 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240be5f0 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33400640 1 -- 172.21.15.87:0/1100868999 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb240be790 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33400640 1 -- 172.21.15.87:0/1100868999 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240beb80 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c0b3250 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c0b3810 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c04ef30 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.083+0000 7fcb33400640 1 -- 172.21.15.87:0/1100868999 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb240b43f0 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.087+0000 7fcb33400640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb18066f60 0x7fcb18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.087+0000 7fcb33400640 1 -- 172.21.15.87:0/1100868999 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcb2411f710 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.087+0000 7fcb3b400640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb18066f60 0x7fcb18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.087+0000 7fcb3b400640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb18066f60 0x7fcb18069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009a50 tx=0x7fcb2c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.091+0000 7fcb33400640 1 -- 172.21.15.87:0/1100868999 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb24016150 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.247+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb3c0cf470 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb33400640 1 -- 172.21.15.87:0/1100868999 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb240f0380 con 0x7fcb3c0aebd0 2024-04-25T10:16:00.256 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:16:00.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb18066f60 msgr2=0x7fcb18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb18066f60 0x7fcb18069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009a50 tx=0x7fcb2c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb3c0aebd0 msgr2=0x7fcb3c0b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c0b2870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401abe0 tx=0x7fcb2401ad70 comp rx=0 tx=0).stop 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 shutdown_connections 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb18066f60 0x7fcb18069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3c159540 0x7fcb3c15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb3c0aebd0 0x7fcb3c0b2870 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 --2- 172.21.15.87:0/1100868999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb3c072020 0x7fcb3c13e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 >> 172.21.15.87:0/1100868999 conn(0x7fcb3c0669a0 msgr2=0x7fcb3c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 shutdown_connections 2024-04-25T10:16:00.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.251+0000 7fcb42600640 1 -- 172.21.15.87:0/1100868999 wait complete. 2024-04-25T10:16:00.276 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:16:00.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9da400640 1 Processor -- start 2024-04-25T10:16:00.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9da400640 1 -- start start 2024-04-25T10:16:00.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9da400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa9d4158170 0x7fa9d415e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9da400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa9d406e380 0x7fa9d40b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9da400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40b1e30 con 0x7fa9d40b2ed0 2024-04-25T10:16:00.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9da400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d40b1c00 con 0x7fa9d406e380 2024-04-25T10:16:00.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9da400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d4158550 con 0x7fa9d4158170 2024-04-25T10:16:00.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9d9a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa9d406e380 0x7fa9d40b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9d3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa9d40b2ed0 0x7fa9d406dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36508/0 (socket says 172.21.15.87:36508) 2024-04-25T10:16:00.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9d3e00640 1 -- 172.21.15.87:0/1440546996 learned_addr learned my addr 172.21.15.87:0/1440546996 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:00.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.407+0000 7fa9d9000640 1 --2- 172.21.15.87:0/1440546996 >> v2:172.21.15.169:3300/0 conn(0x7fa9d4158170 0x7fa9d415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d9a00640 1 -- 172.21.15.87:0/1440546996 >> v2:172.21.15.169:3300/0 conn(0x7fa9d4158170 msgr2=0x7fa9d415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:00.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d9a00640 1 --2- 172.21.15.87:0/1440546996 >> v2:172.21.15.169:3300/0 conn(0x7fa9d4158170 0x7fa9d415e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d9a00640 1 -- 172.21.15.87:0/1440546996 >> v1:172.21.15.87:6789/0 conn(0x7fa9d40b2ed0 legacy=0x7fa9d406dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d9a00640 1 -- 172.21.15.87:0/1440546996 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40b2110 con 0x7fa9d406e380 2024-04-25T10:16:00.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d3400640 1 -- 172.21.15.87:0/1440546996 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933165268 0 0) 0x7fa9d40b1e30 con 0x7fa9d40b2ed0 2024-04-25T10:16:00.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d9a00640 1 --2- 172.21.15.87:0/1440546996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d40b16c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa9d406eaa0 tx=0x7fa9cc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e3ad7be9990130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d3400640 1 -- 172.21.15.87:0/1440546996 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0b2070 con 0x7fa9d406e380 2024-04-25T10:16:00.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d3400640 1 -- 172.21.15.87:0/1440546996 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0a5900 con 0x7fa9d406e380 2024-04-25T10:16:00.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9d3400640 1 -- 172.21.15.87:0/1440546996 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0a5cd0 con 0x7fa9d406e380 2024-04-25T10:16:00.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9da400640 1 -- 172.21.15.87:0/1440546996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 msgr2=0x7fa9d40b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9da400640 1 --2- 172.21.15.87:0/1440546996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d40b16c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa9d406eaa0 tx=0x7fa9cc01c930 comp rx=0 tx=0).stop 2024-04-25T10:16:00.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9da400640 1 -- 172.21.15.87:0/1440546996 shutdown_connections 2024-04-25T10:16:00.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9da400640 1 --2- 172.21.15.87:0/1440546996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d40b16c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9da400640 1 --2- 172.21.15.87:0/1440546996 >> v2:172.21.15.169:3300/0 conn(0x7fa9d4158170 0x7fa9d415e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.411+0000 7fa9da400640 1 -- 172.21.15.87:0/1440546996 >> 172.21.15.87:0/1440546996 conn(0x7fa9d4073d10 msgr2=0x7fa9d40740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 -- 172.21.15.87:0/1440546996 shutdown_connections 2024-04-25T10:16:00.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 -- 172.21.15.87:0/1440546996 wait complete. 2024-04-25T10:16:00.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 Processor -- start 2024-04-25T10:16:00.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 -- start start 2024-04-25T10:16:00.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9d40b2ed0 0x7fa9d41f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d4158170 0x7fa9d41fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415fba0 con 0x7fa9d406e380 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415fa20 con 0x7fa9d4158170 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9da400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415fd20 con 0x7fa9d40b2ed0 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44526/0 (socket says 172.21.15.87:44526) 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 -- 172.21.15.87:0/3828643966 learned_addr learned my addr 172.21.15.87:0/3828643966 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9a00640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d4158170 0x7fa9d41fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d3e00640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9d40b2ed0 0x7fa9d41f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 -- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9d40b2ed0 msgr2=0x7fa9d41f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9d40b2ed0 0x7fa9d41f9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 -- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d4158170 msgr2=0x7fa9d41fa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d4158170 0x7fa9d41fa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 -- 172.21.15.87:0/3828643966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40cb520 con 0x7fa9d406e380 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9a00640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d4158170 0x7fa9d41fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d3e00640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9d40b2ed0 0x7fa9d41f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d9000640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d414ef70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d5f0 tx=0x7fa9c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d0c00640 1 -- 172.21.15.87:0/3828643966 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80125f0 con 0x7fa9d406e380 2024-04-25T10:16:00.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d0c00640 1 -- 172.21.15.87:0/3828643966 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8019070 con 0x7fa9d406e380 2024-04-25T10:16:00.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.415+0000 7fa9d0c00640 1 -- 172.21.15.87:0/3828643966 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80129b0 con 0x7fa9d406e380 2024-04-25T10:16:00.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d41509f0 con 0x7fa9d406e380 2024-04-25T10:16:00.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d4151230 con 0x7fa9d406e380 2024-04-25T10:16:00.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d406bca0 con 0x7fa9d406e380 2024-04-25T10:16:00.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9d0c00640 1 -- 172.21.15.87:0/3828643966 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9c8012b50 con 0x7fa9d406e380 2024-04-25T10:16:00.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9d0c00640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9b4066f60 0x7fa9b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9d0c00640 1 -- 172.21.15.87:0/3828643966 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9c80851c0 con 0x7fa9d406e380 2024-04-25T10:16:00.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9d3e00640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9b4066f60 0x7fa9b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.419+0000 7fa9d3e00640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9b4066f60 0x7fa9b4069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4154fc0 tx=0x7fa9c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.427+0000 7fa9d0c00640 1 -- 172.21.15.87:0/3828643966 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9d41509f0 con 0x7fa9d406e380 2024-04-25T10:16:00.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.583+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa9d4075120 con 0x7fa9d406e380 2024-04-25T10:16:00.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.583+0000 7fa9d0c00640 1 -- 172.21.15.87:0/3828643966 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa9c8056480 con 0x7fa9d406e380 2024-04-25T10:16:00.591 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:00.591 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:16:00.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:00.591 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:16:00.591 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:16:00.591 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:00.591 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:16:00.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9b4066f60 msgr2=0x7fa9b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9b4066f60 0x7fa9b4069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4154fc0 tx=0x7fa9c0009040 comp rx=0 tx=0).stop 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 msgr2=0x7fa9d414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d414ef70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d5f0 tx=0x7fa9c800dab0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 shutdown_connections 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9b4066f60 0x7fa9b4069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d4158170 0x7fa9d41fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9d40b2ed0 0x7fa9d41f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 --2- 172.21.15.87:0/3828643966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9d406e380 0x7fa9d414ef70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 >> 172.21.15.87:0/3828643966 conn(0x7fa9d4073d10 msgr2=0x7fa9d40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 shutdown_connections 2024-04-25T10:16:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.587+0000 7fa9da400640 1 -- 172.21.15.87:0/3828643966 wait complete. 2024-04-25T10:16:00.611 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:16:00.611 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:16:00.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.743+0000 7f50fba00640 1 Processor -- start 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.743+0000 7f50fba00640 1 -- start start 2024-04-25T10:16:00.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50fba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f41618a0 0x7f50f415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50fba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f415e9b0 con 0x7f50f40b4f50 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50fba00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f415ec90 con 0x7f50f4162310 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50fba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f415f250 con 0x7f50f41618a0 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50f9a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f50f40b4f50 0x7f50f40b0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36520/0 (socket says 172.21.15.87:36520) 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50f9a00640 1 -- 172.21.15.87:0/999485242 learned_addr learned my addr 172.21.15.87:0/999485242 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50ebe00640 1 --2- 172.21.15.87:0/999485242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f41618a0 0x7f50f415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834472981 0 0) 0x7f50f415e9b0 con 0x7f50f40b4f50 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d8003530 con 0x7f50f40b4f50 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196899744 0 0) 0x7f50f415ec90 con 0x7f50f4162310 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f415e9b0 con 0x7f50f4162310 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3762795884 0 0) 0x7f50d8003530 con 0x7f50f40b4f50 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50ebe00640 1 -- 172.21.15.87:0/999485242 >> v1:172.21.15.169:6789/0 conn(0x7f50f4162310 legacy=0x7f50f40b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50ebe00640 1 -- 172.21.15.87:0/999485242 >> v1:172.21.15.87:6789/0 conn(0x7f50f40b4f50 legacy=0x7f50f40b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.747+0000 7f50ebe00640 1 -- 172.21.15.87:0/999485242 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f415ef70 con 0x7f50f41618a0 2024-04-25T10:16:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50ebe00640 1 --2- 172.21.15.87:0/999485242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f41618a0 0x7f50f415e2b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0097f0 tx=0x7f50dc01c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fdba47ffaac6583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0b0070 con 0x7f50f41618a0 2024-04-25T10:16:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc0ab400 con 0x7f50f41618a0 2024-04-25T10:16:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50f3e00640 1 -- 172.21.15.87:0/999485242 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec001a20 con 0x7f50f41618a0 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 -- 172.21.15.87:0/999485242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f41618a0 msgr2=0x7f50f415e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 --2- 172.21.15.87:0/999485242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f41618a0 0x7f50f415e2b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0097f0 tx=0x7f50dc01c2e0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 -- 172.21.15.87:0/999485242 shutdown_connections 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 --2- 172.21.15.87:0/999485242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f41618a0 0x7f50f415e2b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 -- 172.21.15.87:0/999485242 >> 172.21.15.87:0/999485242 conn(0x7f50f40b5610 msgr2=0x7f50f40b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 -- 172.21.15.87:0/999485242 shutdown_connections 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 -- 172.21.15.87:0/999485242 wait complete. 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 Processor -- start 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.751+0000 7f50fba00640 1 -- start start 2024-04-25T10:16:00.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f40b4f50 0x7f50f4147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f41618a0 0x7f50f414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50f4162310 0x7f50f415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40b1460 con 0x7f50f41618a0 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40b12e0 con 0x7f50f4162310 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f416ba40 con 0x7f50f40b4f50 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f41618a0 0x7f50f414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f41618a0 0x7f50f414aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44528/0 (socket says 172.21.15.87:44528) 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 -- 172.21.15.87:0/3767840670 learned_addr learned my addr 172.21.15.87:0/3767840670 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50f9000640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f40b4f50 0x7f50f4147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50f9a00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50f4162310 0x7f50f415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 -- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f40b4f50 msgr2=0x7f50f4147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f40b4f50 0x7f50f4147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 -- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50f4162310 msgr2=0x7f50f415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50f4162310 0x7f50f415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 -- 172.21.15.87:0/3767840670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f414b980 con 0x7f50f41618a0 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50ebe00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f41618a0 0x7f50f414aed0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0ae040 tx=0x7f50dc019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50f1600640 1 -- 172.21.15.87:0/3767840670 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0b0040 con 0x7f50f41618a0 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50f1600640 1 -- 172.21.15.87:0/3767840670 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc0a4e00 con 0x7f50f41618a0 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50f1600640 1 -- 172.21.15.87:0/3767840670 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc003820 con 0x7f50f41618a0 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f4153280 con 0x7f50f41618a0 2024-04-25T10:16:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f4152a80 con 0x7f50f41618a0 2024-04-25T10:16:00.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f404ef30 con 0x7f50f41618a0 2024-04-25T10:16:00.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50f1600640 1 -- 172.21.15.87:0/3767840670 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50dc0bb030 con 0x7f50f41618a0 2024-04-25T10:16:00.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.755+0000 7f50f1600640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f50d0066f90 0x7f50d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.759+0000 7f50f1600640 1 -- 172.21.15.87:0/3767840670 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f50dc11dc90 con 0x7f50f41618a0 2024-04-25T10:16:00.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.759+0000 7f50f9000640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f50d0066f90 0x7f50d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.759+0000 7f50f9000640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f50d0066f90 0x7f50d0069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f50e40011f0 tx=0x7f50e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.759+0000 7f50f1600640 1 -- 172.21.15.87:0/3767840670 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50dc0c2060 con 0x7f50f41618a0 2024-04-25T10:16:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:00.915+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f50f406bea0 con 0x7f50f41618a0 2024-04-25T10:16:01.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.175+0000 7f50f1600640 1 -- 172.21.15.87:0/3767840670 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) v1 ==== 138+0+0 (secure 0 0 0) 0x7f50dc015ff0 con 0x7f50f41618a0 2024-04-25T10:16:01.183 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:16:01.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f50d0066f90 msgr2=0x7f50d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f50d0066f90 0x7f50d0069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f50e40011f0 tx=0x7f50e4009040 comp rx=0 tx=0).stop 2024-04-25T10:16:01.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f41618a0 msgr2=0x7f50f414aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f41618a0 0x7f50f414aed0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0ae040 tx=0x7f50dc019080 comp rx=0 tx=0).stop 2024-04-25T10:16:01.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 shutdown_connections 2024-04-25T10:16:01.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f50d0066f90 0x7f50d0069450 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50f4162310 0x7f50f415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f41618a0 0x7f50f414aed0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 --2- 172.21.15.87:0/3767840670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50f40b4f50 0x7f50f4147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 >> 172.21.15.87:0/3767840670 conn(0x7f50f40b5610 msgr2=0x7f50f415f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 shutdown_connections 2024-04-25T10:16:01.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.179+0000 7f50fba00640 1 -- 172.21.15.87:0/3767840670 wait complete. 2024-04-25T10:16:01.210 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:16:01.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63b5200640 1 Processor -- start 2024-04-25T10:16:01.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63b5200640 1 -- start start 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63b5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63b0154f70 0x7f63b00cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63b5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b00cfea0 0x7f63b00b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63b5200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b015e6e0 con 0x7f63b00b39d0 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63b5200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b00d0300 con 0x7f63b0154f70 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63b5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b015e9c0 con 0x7f63b00cfea0 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ae800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f63b00b39d0 0x7f63b00ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36528/0 (socket says 172.21.15.87:36528) 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ae800640 1 -- 172.21.15.87:0/4223531407 learned_addr learned my addr 172.21.15.87:0/4223531407 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b00cfea0 0x7f63b00b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ade00640 1 --2- 172.21.15.87:0/4223531407 >> v2:172.21.15.169:3300/0 conn(0x7f63b0154f70 0x7f63b00cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316421595 0 0) 0x7f63b015e6e0 con 0x7f63b00b39d0 2024-04-25T10:16:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6394005740 con 0x7f63b00b39d0 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2424015547 0 0) 0x7f6394005740 con 0x7f63b00b39d0 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b00cfea0 msgr2=0x7f63b00b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b00cfea0 0x7f63b00b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 >> v2:172.21.15.169:3300/0 conn(0x7f63b0154f70 msgr2=0x7f63b00cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 --2- 172.21.15.87:0/4223531407 >> v2:172.21.15.169:3300/0 conn(0x7f63b0154f70 0x7f63b00cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b015eca0 con 0x7f63b00b39d0 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f63a8001680 con 0x7f63b00b39d0 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 >> v1:172.21.15.87:6789/0 conn(0x7f63b00b39d0 legacy=0x7f63b00ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6394005bd0 0x7f6394005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f6394006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940073e0 0x7f639400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b015e9c0 con 0x7f6394006a20 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639400e120 con 0x7f6394005bd0 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639400e2a0 con 0x7f63940073e0 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b00cfea0 0x7f63b00b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f6394006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ade00640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6394005bd0 0x7f6394005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ae800640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940073e0 0x7f639400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 -- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940073e0 msgr2=0x7f639400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940073e0 0x7f639400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 -- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6394005bd0 msgr2=0x7f6394005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6394005bd0 0x7f6394005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.347+0000 7f63ad400640 1 -- 172.21.15.87:0/4223531407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394005740 con 0x7f6394006a20 2024-04-25T10:16:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63ad400640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f6394006ea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f639c009b20 tx=0x7f639c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ac83128af1b953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c0097f0 con 0x7f6394006a20 2024-04-25T10:16:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c004030 con 0x7f6394006a20 2024-04-25T10:16:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63aca00640 1 -- 172.21.15.87:0/4223531407 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c004400 con 0x7f6394006a20 2024-04-25T10:16:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63b5200640 1 -- 172.21.15.87:0/4223531407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 msgr2=0x7f6394006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63b5200640 1 --2- 172.21.15.87:0/4223531407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f6394006ea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f639c009b20 tx=0x7f639c01ca50 comp rx=0 tx=0).stop 2024-04-25T10:16:01.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63ae800640 1 -- 172.21.15.87:0/4223531407 reap_dead start 2024-04-25T10:16:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63b5200640 1 -- 172.21.15.87:0/4223531407 shutdown_connections 2024-04-25T10:16:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63b5200640 1 -- 172.21.15.87:0/4223531407 >> 172.21.15.87:0/4223531407 conn(0x7f63b0066b70 msgr2=0x7f63b0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63b5200640 1 -- 172.21.15.87:0/4223531407 shutdown_connections 2024-04-25T10:16:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.351+0000 7f63b5200640 1 -- 172.21.15.87:0/4223531407 wait complete. 2024-04-25T10:16:01.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 Processor -- start 2024-04-25T10:16:01.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 -- start start 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f63b00c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b0149250 0x7f63b00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b01554d0 0x7f63b00ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b0155350 con 0x7f6394006a20 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00b3e60 con 0x7f63b0149250 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00b3490 con 0x7f63b01554d0 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f63b00c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f63b00c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44548/0 (socket says 172.21.15.87:44548) 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 -- 172.21.15.87:0/1212588946 learned_addr learned my addr 172.21.15.87:0/1212588946 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ad400640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b0149250 0x7f63b00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ae800640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b01554d0 0x7f63b00ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 -- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b01554d0 msgr2=0x7f63b00ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b01554d0 0x7f63b00ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 -- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b0149250 msgr2=0x7f63b00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b0149250 0x7f63b00cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 -- 172.21.15.87:0/1212588946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00cd4c0 con 0x7f6394006a20 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ae800640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b01554d0 0x7f63b00ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63ade00640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f63b00c9500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009fc0 tx=0x7f63a000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63a6000640 1 -- 172.21.15.87:0/1212588946 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0015430 con 0x7f6394006a20 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63a6000640 1 -- 172.21.15.87:0/1212588946 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a00155d0 con 0x7f6394006a20 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63a6000640 1 -- 172.21.15.87:0/1212588946 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a00159c0 con 0x7f6394006a20 2024-04-25T10:16:01.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b00cd7a0 con 0x7f6394006a20 2024-04-25T10:16:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.355+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b00c9ae0 con 0x7f6394006a20 2024-04-25T10:16:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.359+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b004ef30 con 0x7f6394006a20 2024-04-25T10:16:01.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.359+0000 7f63a6000640 1 -- 172.21.15.87:0/1212588946 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63a0015b60 con 0x7f6394006a20 2024-04-25T10:16:01.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.359+0000 7f63a6000640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6390066f60 0x7f6390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.359+0000 7f63a6000640 1 -- 172.21.15.87:0/1212588946 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f63a0085fd0 con 0x7f6394006a20 2024-04-25T10:16:01.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.359+0000 7f63ad400640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6390066f60 0x7f6390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.359+0000 7f63ad400640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6390066f60 0x7f6390069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f639c0ad000 tx=0x7f639c0af040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.363+0000 7f63a6000640 1 -- 172.21.15.87:0/1212588946 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63a0019030 con 0x7f6394006a20 2024-04-25T10:16:01.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:01.519+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f63b0067300 con 0x7f6394006a20 2024-04-25T10:16:02.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.187+0000 7f63a6000640 1 -- 172.21.15.87:0/1212588946 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) v1 ==== 170+0+0 (secure 0 0 0) 0x7f63a0010040 con 0x7f6394006a20 2024-04-25T10:16:02.194 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6390066f60 msgr2=0x7f6390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6390066f60 0x7f6390069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f639c0ad000 tx=0x7f639c0af040 comp rx=0 tx=0).stop 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 msgr2=0x7f63b00c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f63b00c9500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009fc0 tx=0x7f63a000f580 comp rx=0 tx=0).stop 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 shutdown_connections 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6390066f60 0x7f6390069420 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f639c0ad000 tx=0x7f639c0af040 comp rx=0 tx=0).stop 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b01554d0 0x7f63b00ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b0149250 0x7f63b00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 --2- 172.21.15.87:0/1212588946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6394006a20 0x7f63b00c9500 secure :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009fc0 tx=0x7f63a000f580 comp rx=0 tx=0).stop 2024-04-25T10:16:02.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 >> 172.21.15.87:0/1212588946 conn(0x7f63b0066b70 msgr2=0x7f63b006c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 shutdown_connections 2024-04-25T10:16:02.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.191+0000 7f63b5200640 1 -- 172.21.15.87:0/1212588946 wait complete. 2024-04-25T10:16:02.234 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:16:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb1400640 1 Processor -- start 2024-04-25T10:16:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb1400640 1 -- start start 2024-04-25T10:16:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faeac15a840 0x7faeac071510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb1400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faeac070ae0 0x7faeac071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb1400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac0cc2e0 con 0x7faeac071f90 2024-04-25T10:16:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb1400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeac0cc5c0 con 0x7faeac15a840 2024-04-25T10:16:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb1400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeac073db0 con 0x7faeac070ae0 2024-04-25T10:16:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeabe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faeac15a840 0x7faeac071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb0a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7faeac071f90 0x7faeac15e3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36532/0 (socket says 172.21.15.87:36532) 2024-04-25T10:16:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeb0a00640 1 -- 172.21.15.87:0/3355588918 learned_addr learned my addr 172.21.15.87:0/3355588918 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeab400640 1 --2- 172.21.15.87:0/3355588918 >> v2:172.21.15.169:3300/0 conn(0x7faeac070ae0 0x7faeac071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeabe00640 1 -- 172.21.15.87:0/3355588918 >> v2:172.21.15.169:3300/0 conn(0x7faeac070ae0 msgr2=0x7faeac071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeabe00640 1 --2- 172.21.15.87:0/3355588918 >> v2:172.21.15.169:3300/0 conn(0x7faeac070ae0 0x7faeac071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeabe00640 1 -- 172.21.15.87:0/3355588918 >> v1:172.21.15.87:6789/0 conn(0x7faeac071f90 legacy=0x7faeac15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeabe00640 1 -- 172.21.15.87:0/3355588918 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac074090 con 0x7faeac15a840 2024-04-25T10:16:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.367+0000 7faeaaa00640 1 -- 172.21.15.87:0/3355588918 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4044820014 0 0) 0x7faeac0cc2e0 con 0x7faeac071f90 2024-04-25T10:16:02.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeabe00640 1 --2- 172.21.15.87:0/3355588918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac071510 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7faeac06da80 tx=0x7fae9c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca38a6bb71c08b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeaaa00640 1 -- 172.21.15.87:0/3355588918 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b0070 con 0x7faeac15a840 2024-04-25T10:16:02.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeaaa00640 1 -- 172.21.15.87:0/3355588918 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c0ab600 con 0x7faeac15a840 2024-04-25T10:16:02.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeaaa00640 1 -- 172.21.15.87:0/3355588918 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0ab9d0 con 0x7faeac15a840 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 -- 172.21.15.87:0/3355588918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 msgr2=0x7faeac071510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 --2- 172.21.15.87:0/3355588918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac071510 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7faeac06da80 tx=0x7fae9c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 -- 172.21.15.87:0/3355588918 shutdown_connections 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 --2- 172.21.15.87:0/3355588918 >> v2:172.21.15.169:3300/0 conn(0x7faeac070ae0 0x7faeac071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 --2- 172.21.15.87:0/3355588918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac071510 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 -- 172.21.15.87:0/3355588918 >> 172.21.15.87:0/3355588918 conn(0x7faeac0b1bc0 msgr2=0x7faeac0b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 -- 172.21.15.87:0/3355588918 shutdown_connections 2024-04-25T10:16:02.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.371+0000 7faeb1400640 1 -- 172.21.15.87:0/3355588918 wait complete. 2024-04-25T10:16:02.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 Processor -- start 2024-04-25T10:16:02.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 -- start start 2024-04-25T10:16:02.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeac070ae0 0x7faeac13a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac071f90 0x7faeac15cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac167910 con 0x7faeac15a840 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac167790 con 0x7faeac071f90 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac167a90 con 0x7faeac070ae0 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac15d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44568/0 (socket says 172.21.15.87:44568) 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 -- 172.21.15.87:0/965911904 learned_addr learned my addr 172.21.15.87:0/965911904 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 -- 172.21.15.87:0/965911904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeac070ae0 msgr2=0x7faeac13a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeac070ae0 0x7faeac13a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 -- 172.21.15.87:0/965911904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac071f90 msgr2=0x7faeac15cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac071f90 0x7faeac15cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 -- 172.21.15.87:0/965911904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac151010 con 0x7faeac15a840 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb0a00640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac15d0b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faea400c7a0 tx=0x7faea400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faea3e00640 1 -- 172.21.15.87:0/965911904 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4018070 con 0x7faeac15a840 2024-04-25T10:16:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faea3e00640 1 -- 172.21.15.87:0/965911904 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea400a640 con 0x7faeac15a840 2024-04-25T10:16:02.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac149a40 con 0x7faeac15a840 2024-04-25T10:16:02.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faea3e00640 1 -- 172.21.15.87:0/965911904 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400aab0 con 0x7faeac15a840 2024-04-25T10:16:02.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.375+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac14a000 con 0x7faeac15a840 2024-04-25T10:16:02.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.379+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac074520 con 0x7faeac15a840 2024-04-25T10:16:02.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.379+0000 7faea3e00640 1 -- 172.21.15.87:0/965911904 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faea401b070 con 0x7faeac15a840 2024-04-25T10:16:02.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.379+0000 7faea3e00640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae8c066f60 0x7fae8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.379+0000 7faea3e00640 1 -- 172.21.15.87:0/965911904 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5609+0+0 (secure 0 0 0) 0x7faea4084940 con 0x7faeac15a840 2024-04-25T10:16:02.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.379+0000 7faeabe00640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae8c066f60 0x7fae8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.379+0000 7faeabe00640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae8c066f60 0x7fae8c069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0ad000 tx=0x7fae9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.383+0000 7faea3e00640 1 -- 172.21.15.87:0/965911904 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faea4055a20 con 0x7faeac15a840 2024-04-25T10:16:02.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:02.571+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faeac159f00 con 0x7faeac15a840 2024-04-25T10:16:03.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.199+0000 7faea3e00640 1 -- 172.21.15.87:0/965911904 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) v1 ==== 172+0+0 (secure 0 0 0) 0x7faea4055320 con 0x7faeac15a840 2024-04-25T10:16:03.213 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T10:16:03.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae8c066f60 msgr2=0x7fae8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae8c066f60 0x7fae8c069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0ad000 tx=0x7fae9c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 msgr2=0x7faeac15d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac15d0b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faea400c7a0 tx=0x7faea400cc60 comp rx=0 tx=0).stop 2024-04-25T10:16:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 shutdown_connections 2024-04-25T10:16:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fae8c066f60 0x7fae8c069420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeac15a840 0x7faeac15d0b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac071f90 0x7faeac15cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 --2- 172.21.15.87:0/965911904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faeac070ae0 0x7faeac13a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 >> 172.21.15.87:0/965911904 conn(0x7faeac0b1bc0 msgr2=0x7faeac14f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 shutdown_connections 2024-04-25T10:16:03.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.215+0000 7faeb1400640 1 -- 172.21.15.87:0/965911904 wait complete. 2024-04-25T10:16:03.258 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:03.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.387+0000 7f1ac4e00640 1 Processor -- start 2024-04-25T10:16:03.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.387+0000 7f1ac4e00640 1 -- start start 2024-04-25T10:16:03.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ac4e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ac015bb50 0x7f1ac00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ac4e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1ac00d24d0 0x7f1ac00cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ac4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac00d0490 0x7f1ac00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ac4e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac00b1070 con 0x7f1ac00d24d0 2024-04-25T10:16:03.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ac4e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac00ce430 con 0x7f1ac015bb50 2024-04-25T10:16:03.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ac4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00ce6d0 con 0x7f1ac00d0490 2024-04-25T10:16:03.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1ac00d24d0 0x7f1ac00cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1ac00d24d0 0x7f1ac00cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44578/0 (socket says 172.21.15.87:44578) 2024-04-25T10:16:03.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 -- 172.21.15.87:0/3254300682 learned_addr learned my addr 172.21.15.87:0/3254300682 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:03.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1abde00640 1 --2- 172.21.15.87:0/3254300682 >> v2:172.21.15.169:3300/0 conn(0x7f1ac015bb50 0x7f1ac00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1abe800640 1 --2- 172.21.15.87:0/3254300682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac00d0490 0x7f1ac00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 -- 172.21.15.87:0/3254300682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac00d0490 msgr2=0x7f1ac00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 --2- 172.21.15.87:0/3254300682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac00d0490 0x7f1ac00cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 -- 172.21.15.87:0/3254300682 >> v2:172.21.15.169:3300/0 conn(0x7f1ac015bb50 msgr2=0x7f1ac00b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 --2- 172.21.15.87:0/3254300682 >> v2:172.21.15.169:3300/0 conn(0x7f1ac015bb50 0x7f1ac00b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.391+0000 7f1ab5400640 1 -- 172.21.15.87:0/3254300682 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00ce9b0 con 0x7f1ac00d24d0 2024-04-25T10:16:03.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ab5400640 1 --2- 172.21.15.87:0/3254300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac00cff50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009870 tx=0x7f1aa801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdefa251a75803c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1abd400640 1 -- 172.21.15.87:0/3254300682 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80b0070 con 0x7f1ac00d24d0 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1abd400640 1 -- 172.21.15.87:0/3254300682 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa80ab600 con 0x7f1ac00d24d0 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1abd400640 1 -- 172.21.15.87:0/3254300682 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80ab9d0 con 0x7f1ac00d24d0 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- 172.21.15.87:0/3254300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 msgr2=0x7f1ac00cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/3254300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac00cff50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009870 tx=0x7f1aa801c520 comp rx=0 tx=0).stop 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- 172.21.15.87:0/3254300682 shutdown_connections 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/3254300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac00cff50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/3254300682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac00d0490 0x7f1ac00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/3254300682 >> v2:172.21.15.169:3300/0 conn(0x7f1ac015bb50 0x7f1ac00b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- 172.21.15.87:0/3254300682 >> 172.21.15.87:0/3254300682 conn(0x7f1ac00d1970 msgr2=0x7f1ac00d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- 172.21.15.87:0/3254300682 shutdown_connections 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- 172.21.15.87:0/3254300682 wait complete. 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 Processor -- start 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- start start 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac00d0490 0x7f1ac015e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac0154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac015bb50 0x7f1ac015a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00ae880 con 0x7f1ac00d24d0 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac015c140 con 0x7f1ac00d0490 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.395+0000 7f1ac4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00aea00 con 0x7f1ac015bb50 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac0154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac0154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44580/0 (socket says 172.21.15.87:44580) 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 -- 172.21.15.87:0/1639929885 learned_addr learned my addr 172.21.15.87:0/1639929885 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1abde00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac00d0490 0x7f1ac015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1abe800640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac015bb50 0x7f1ac015a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 -- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac015bb50 msgr2=0x7f1ac015a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac015bb50 0x7f1ac015a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 -- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac00d0490 msgr2=0x7f1ac015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac00d0490 0x7f1ac015e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 -- 172.21.15.87:0/1639929885 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac01546d0 con 0x7f1ac00d24d0 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab5400640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac0154190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ae040 tx=0x7f1aa80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab6a00640 1 -- 172.21.15.87:0/1639929885 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80b0040 con 0x7f1ac00d24d0 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab6a00640 1 -- 172.21.15.87:0/1639929885 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa80b9070 con 0x7f1ac00d24d0 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab6a00640 1 -- 172.21.15.87:0/1639929885 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80b4680 con 0x7f1ac00d24d0 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac00c6ed0 con 0x7f1ac00d24d0 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac00c7490 con 0x7f1ac00d24d0 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac004ef30 con 0x7f1ac00d24d0 2024-04-25T10:16:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.399+0000 7f1ab6a00640 1 -- 172.21.15.87:0/1639929885 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1aa80b4820 con 0x7f1ac00d24d0 2024-04-25T10:16:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.403+0000 7f1ab6a00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a9c066f90 0x7f1a9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.403+0000 7f1ab6a00640 1 -- 172.21.15.87:0/1639929885 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aa811ebc0 con 0x7f1ac00d24d0 2024-04-25T10:16:03.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.403+0000 7f1abde00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a9c066f90 0x7f1a9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.403+0000 7f1abde00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a9c066f90 0x7f1a9c069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0003ab0 tx=0x7f1ab0009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.407+0000 7f1ab6a00640 1 -- 172.21.15.87:0/1639929885 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1aa8016110 con 0x7f1ac00d24d0 2024-04-25T10:16:03.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.563+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ac00b2a00 con 0x7f1ac00d24d0 2024-04-25T10:16:03.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.563+0000 7f1ab6a00640 1 -- 172.21.15.87:0/1639929885 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f1aa80ef4e0 con 0x7f1ac00d24d0 2024-04-25T10:16:03.571 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:03.571 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":56,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:02.575877+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a9c066f90 msgr2=0x7f1a9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a9c066f90 0x7f1a9c069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0003ab0 tx=0x7f1ab0009830 comp rx=0 tx=0).stop 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 msgr2=0x7f1ac0154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac0154190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ae040 tx=0x7f1aa80192c0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 shutdown_connections 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a9c066f90 0x7f1a9c069450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ac015bb50 0x7f1ac015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ac00d24d0 0x7f1ac0154190 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 --2- 172.21.15.87:0/1639929885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac00d0490 0x7f1ac015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 >> 172.21.15.87:0/1639929885 conn(0x7f1ac00d1970 msgr2=0x7f1ac006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 shutdown_connections 2024-04-25T10:16:03.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.567+0000 7f1ac4e00640 1 -- 172.21.15.87:0/1639929885 wait complete. 2024-04-25T10:16:03.593 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f7172200640 1 Processor -- start 2024-04-25T10:16:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f7172200640 1 -- start start 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f7172200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c15a9a0 0x7f716c0af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f7172200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f716c06b3a0 0x7f716c15e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f7172200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c0aff80 con 0x7f716c0b37f0 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f7172200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c06e230 con 0x7f716c06b3a0 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f7172200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c06efb0 con 0x7f716c15a9a0 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f716c06b3a0 0x7f716c15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f716c06b3a0 0x7f716c15e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44590/0 (socket says 172.21.15.87:44590) 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716be00640 1 -- 172.21.15.87:0/350188159 learned_addr learned my addr 172.21.15.87:0/350188159 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716b400640 1 --2- 172.21.15.87:0/350188159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c15a9a0 0x7f716c0af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716be00640 1 -- 172.21.15.87:0/350188159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c15a9a0 msgr2=0x7f716c0af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716be00640 1 --2- 172.21.15.87:0/350188159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c15a9a0 0x7f716c0af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716be00640 1 -- 172.21.15.87:0/350188159 >> v1:172.21.15.169:6789/0 conn(0x7f716c0b37f0 legacy=0x7f716c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716be00640 1 -- 172.21.15.87:0/350188159 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c06e4d0 con 0x7f716c06b3a0 2024-04-25T10:16:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.723+0000 7f716b400640 1 --2- 172.21.15.87:0/350188159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c15a9a0 0x7f716c0af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f716be00640 1 --2- 172.21.15.87:0/350188159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c06b3a0 0x7f716c15e630 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7164009e50 tx=0x7f716401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5dc255438e0611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f716a000640 1 -- 172.21.15.87:0/350188159 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640a5830 con 0x7f716c06b3a0 2024-04-25T10:16:03.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f716a000640 1 -- 172.21.15.87:0/350188159 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640a59d0 con 0x7f716c06b3a0 2024-04-25T10:16:03.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f716a000640 1 -- 172.21.15.87:0/350188159 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640a5da0 con 0x7f716c06b3a0 2024-04-25T10:16:03.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 -- 172.21.15.87:0/350188159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c06b3a0 msgr2=0x7f716c15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 --2- 172.21.15.87:0/350188159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c06b3a0 0x7f716c15e630 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7164009e50 tx=0x7f716401c970 comp rx=0 tx=0).stop 2024-04-25T10:16:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 -- 172.21.15.87:0/350188159 shutdown_connections 2024-04-25T10:16:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 --2- 172.21.15.87:0/350188159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c06b3a0 0x7f716c15e630 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 --2- 172.21.15.87:0/350188159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c15a9a0 0x7f716c0af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 -- 172.21.15.87:0/350188159 >> 172.21.15.87:0/350188159 conn(0x7f716c0b24f0 msgr2=0x7f716c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 -- 172.21.15.87:0/350188159 shutdown_connections 2024-04-25T10:16:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 -- 172.21.15.87:0/350188159 wait complete. 2024-04-25T10:16:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.727+0000 7f7172200640 1 Processor -- start 2024-04-25T10:16:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 -- start start 2024-04-25T10:16:03.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c06b3a0 0x7f716c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f716c0b37f0 0x7f716c14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c15a9a0 0x7f716c13e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0afd60 con 0x7f716c15a9a0 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0afbe0 con 0x7f716c0b37f0 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c15ad80 con 0x7f716c06b3a0 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c15a9a0 0x7f716c13e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c15a9a0 0x7f716c13e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44600/0 (socket says 172.21.15.87:44600) 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 -- 172.21.15.87:0/818292809 learned_addr learned my addr 172.21.15.87:0/818292809 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716b400640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c06b3a0 0x7f716c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716aa00640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f716c0b37f0 0x7f716c14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 -- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c06b3a0 msgr2=0x7f716c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c06b3a0 0x7f716c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 -- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f716c0b37f0 msgr2=0x7f716c14a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f716c0b37f0 0x7f716c14a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 -- 172.21.15.87:0/818292809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c067530 con 0x7f716c15a9a0 2024-04-25T10:16:03.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716aa00640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f716c0b37f0 0x7f716c14a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f716be00640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c15a9a0 0x7f716c13e060 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f716401a5a0 tx=0x7f7164019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7163400640 1 -- 172.21.15.87:0/818292809 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640bc020 con 0x7f716c15a9a0 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7163400640 1 -- 172.21.15.87:0/818292809 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640b8670 con 0x7f716c15a9a0 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7163400640 1 -- 172.21.15.87:0/818292809 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640b8a60 con 0x7f716c15a9a0 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c1580d0 con 0x7f716c15a9a0 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c158690 con 0x7f716c15a9a0 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.731+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c04ef30 con 0x7f716c15a9a0 2024-04-25T10:16:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.735+0000 7f7163400640 1 -- 172.21.15.87:0/818292809 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71640bf020 con 0x7f716c15a9a0 2024-04-25T10:16:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.735+0000 7f7163400640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f714c066f60 0x7f714c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.735+0000 7f7163400640 1 -- 172.21.15.87:0/818292809 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71641201e0 con 0x7f716c15a9a0 2024-04-25T10:16:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.735+0000 7f716b400640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f714c066f60 0x7f714c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.735+0000 7f716b400640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f714c066f60 0x7f714c069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f715c006e90 tx=0x7f715c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.739+0000 7f7163400640 1 -- 172.21.15.87:0/818292809 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71640c6030 con 0x7f716c15a9a0 2024-04-25T10:16:03.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.891+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f716c06c420 con 0x7f716c15a9a0 2024-04-25T10:16:03.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.891+0000 7f7163400640 1 -- 172.21.15.87:0/818292809 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f7164016620 con 0x7f716c15a9a0 2024-04-25T10:16:03.899 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:03.899 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":56,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:02.575877+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f714c066f60 msgr2=0x7f714c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f714c066f60 0x7f714c069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f715c006e90 tx=0x7f715c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c15a9a0 msgr2=0x7f716c13e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c15a9a0 0x7f716c13e060 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f716401a5a0 tx=0x7f7164019710 comp rx=0 tx=0).stop 2024-04-25T10:16:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 shutdown_connections 2024-04-25T10:16:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f714c066f60 0x7f714c069420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f716c15a9a0 0x7f716c13e060 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f716c0b37f0 0x7f716c14a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 --2- 172.21.15.87:0/818292809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f716c06b3a0 0x7f716c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 >> 172.21.15.87:0/818292809 conn(0x7f716c0b24f0 msgr2=0x7f716c0b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 shutdown_connections 2024-04-25T10:16:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:03.895+0000 7f7172200640 1 -- 172.21.15.87:0/818292809 wait complete. 2024-04-25T10:16:03.921 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:16:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.051+0000 7f17ec600640 1 Processor -- start 2024-04-25T10:16:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.051+0000 7f17ec600640 1 -- start start 2024-04-25T10:16:04.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ec600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e40cc200 0x7f17e40ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ec600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17e406cf40 0x7f17e406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ec600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e40cdab0 con 0x7f17e40d0d00 2024-04-25T10:16:04.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ec600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e40cd210 con 0x7f17e406cf40 2024-04-25T10:16:04.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ec600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40cd4f0 con 0x7f17e40cc200 2024-04-25T10:16:04.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ea600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17e406cf40 0x7f17e406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ea600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17e406cf40 0x7f17e406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44606/0 (socket says 172.21.15.87:44606) 2024-04-25T10:16:04.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ea600640 1 -- 172.21.15.87:0/792932517 learned_addr learned my addr 172.21.15.87:0/792932517 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:04.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17e9200640 1 --2- 172.21.15.87:0/792932517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e40cc200 0x7f17e40ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ea600640 1 -- 172.21.15.87:0/792932517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e40cc200 msgr2=0x7f17e40ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ea600640 1 --2- 172.21.15.87:0/792932517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e40cc200 0x7f17e40ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ea600640 1 -- 172.21.15.87:0/792932517 >> v1:172.21.15.169:6789/0 conn(0x7f17e40d0d00 legacy=0x7f17e4162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.055+0000 7f17ea600640 1 -- 172.21.15.87:0/792932517 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e40cd7d0 con 0x7f17e406cf40 2024-04-25T10:16:04.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ea600640 1 --2- 172.21.15.87:0/792932517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e406cf40 0x7f17e406d3d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009e50 tx=0x7f17dc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5701b5270636494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17e3e00640 1 -- 172.21.15.87:0/792932517 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0a5660 con 0x7f17e406cf40 2024-04-25T10:16:04.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17e3e00640 1 -- 172.21.15.87:0/792932517 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc0a5800 con 0x7f17e406cf40 2024-04-25T10:16:04.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17e3e00640 1 -- 172.21.15.87:0/792932517 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0a5bd0 con 0x7f17e406cf40 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 -- 172.21.15.87:0/792932517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e406cf40 msgr2=0x7f17e406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 --2- 172.21.15.87:0/792932517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e406cf40 0x7f17e406d3d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009e50 tx=0x7f17dc01c9a0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 -- 172.21.15.87:0/792932517 shutdown_connections 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 --2- 172.21.15.87:0/792932517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e406cf40 0x7f17e406d3d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 --2- 172.21.15.87:0/792932517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e40cc200 0x7f17e40ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 -- 172.21.15.87:0/792932517 >> 172.21.15.87:0/792932517 conn(0x7f17e4072290 msgr2=0x7f17e40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 -- 172.21.15.87:0/792932517 shutdown_connections 2024-04-25T10:16:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 -- 172.21.15.87:0/792932517 wait complete. 2024-04-25T10:16:04.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 Processor -- start 2024-04-25T10:16:04.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 -- start start 2024-04-25T10:16:04.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.059+0000 7f17ec600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e406cf40 0x7f17e40c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e40cc200 0x7f17e41fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e40d0d00 0x7f17e4160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e415e040 con 0x7f17e40d0d00 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4162aa0 con 0x7f17e40cc200 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e415e1c0 con 0x7f17e406cf40 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e40d0d00 0x7f17e4160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e40d0d00 0x7f17e4160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44614/0 (socket says 172.21.15.87:44614) 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 -- 172.21.15.87:0/4161965467 learned_addr learned my addr 172.21.15.87:0/4161965467 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17e9c00640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e406cf40 0x7f17e40c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17e9200640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e40cc200 0x7f17e41fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 -- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e406cf40 msgr2=0x7f17e40c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e406cf40 0x7f17e40c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 -- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e40cc200 msgr2=0x7f17e41fadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e40cc200 0x7f17e41fadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 -- 172.21.15.87:0/4161965467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e40c9420 con 0x7f17e40d0d00 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17e9200640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e40cc200 0x7f17e41fadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ea600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e40d0d00 0x7f17e4160500 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0af040 tx=0x7f17dc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17e1600640 1 -- 172.21.15.87:0/4161965467 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0bc020 con 0x7f17e40d0d00 2024-04-25T10:16:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17e1600640 1 -- 172.21.15.87:0/4161965467 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc0b8690 con 0x7f17e40d0d00 2024-04-25T10:16:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17e1600640 1 -- 172.21.15.87:0/4161965467 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc009510 con 0x7f17e40d0d00 2024-04-25T10:16:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e40c9700 con 0x7f17e40d0d00 2024-04-25T10:16:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e40c9cc0 con 0x7f17e40d0d00 2024-04-25T10:16:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e40c9ff0 con 0x7f17e40d0d00 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.063+0000 7f17e1600640 1 -- 172.21.15.87:0/4161965467 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17dc0bf020 con 0x7f17e40d0d00 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.067+0000 7f17e1600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.067+0000 7f17e1600640 1 -- 172.21.15.87:0/4161965467 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17dc11fef0 con 0x7f17e40d0d00 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.067+0000 7f17e9c00640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.067+0000 7f17e9c00640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f17d8001770 tx=0x7f17d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.071+0000 7f17e1600640 1 -- 172.21.15.87:0/4161965467 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17dc0c6030 con 0x7f17e40d0d00 2024-04-25T10:16:04.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:04.263+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f17e404ef30 con 0x7f17e40d0d00 2024-04-25T10:16:05.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:16:05.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:16:05.182 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-04-25T10:16:05.175+0000 7f7104e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:05.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.219+0000 7f17e1600640 1 -- 172.21.15.87:0/4161965467 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7f17dc016620 con 0x7f17e40d0d00 2024-04-25T10:16:05.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 msgr2=0x7f17c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f17d8001770 tx=0x7f17d8009040 comp rx=0 tx=0).stop 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e40d0d00 msgr2=0x7f17e4160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e40d0d00 0x7f17e4160500 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0af040 tx=0x7f17dc019740 comp rx=0 tx=0).stop 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 shutdown_connections 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e40d0d00 0x7f17e4160500 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e40cc200 0x7f17e41fadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 --2- 172.21.15.87:0/4161965467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e406cf40 0x7f17e40c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 >> 172.21.15.87:0/4161965467 conn(0x7f17e4072290 msgr2=0x7f17e41596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 shutdown_connections 2024-04-25T10:16:05.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.223+0000 7f17ec600640 1 -- 172.21.15.87:0/4161965467 wait complete. 2024-04-25T10:16:05.253 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:16:05.253 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:16:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f0492c00640 1 Processor -- start 2024-04-25T10:16:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f0492c00640 1 -- start start 2024-04-25T10:16:05.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f0492c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f048c0aef80 0x7f048c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f0492c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c15b080 con 0x7f048c070350 2024-04-25T10:16:05.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f0492c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c15adb0 con 0x7f048c0b31c0 2024-04-25T10:16:05.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f0492c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c15eb60 con 0x7f048c0aef80 2024-04-25T10:16:05.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f048be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f048c0aef80 0x7f048c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f048b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f048c070350 0x7f048c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36536/0 (socket says 172.21.15.87:36536) 2024-04-25T10:16:05.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f048b400640 1 -- 172.21.15.87:0/1854347052 learned_addr learned my addr 172.21.15.87:0/1854347052 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:05.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f048be00640 1 -- 172.21.15.87:0/1854347052 >> v1:172.21.15.169:6789/0 conn(0x7f048c0b31c0 legacy=0x7f048c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f048be00640 1 -- 172.21.15.87:0/1854347052 >> v1:172.21.15.87:6789/0 conn(0x7f048c070350 legacy=0x7f048c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f048be00640 1 -- 172.21.15.87:0/1854347052 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c15ee40 con 0x7f048c0aef80 2024-04-25T10:16:05.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.391+0000 7f048aa00640 1 -- 172.21.15.87:0/1854347052 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810025198 0 0) 0x7f048c15b080 con 0x7f048c070350 2024-04-25T10:16:05.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f048be00640 1 --2- 172.21.15.87:0/1854347052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c0aef80 0x7f048c15e480 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f047c0097c0 tx=0x7f047c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142f551e0e3d4b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f048aa00640 1 -- 172.21.15.87:0/1854347052 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0ab680 con 0x7f048c0aef80 2024-04-25T10:16:05.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f048aa00640 1 -- 172.21.15.87:0/1854347052 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c0b0070 con 0x7f048c0aef80 2024-04-25T10:16:05.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f048aa00640 1 -- 172.21.15.87:0/1854347052 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0aba40 con 0x7f048c0aef80 2024-04-25T10:16:05.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f0492c00640 1 -- 172.21.15.87:0/1854347052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c0aef80 msgr2=0x7f048c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f0492c00640 1 --2- 172.21.15.87:0/1854347052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c0aef80 0x7f048c15e480 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f047c0097c0 tx=0x7f047c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f0492c00640 1 -- 172.21.15.87:0/1854347052 shutdown_connections 2024-04-25T10:16:05.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f0492c00640 1 --2- 172.21.15.87:0/1854347052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c0aef80 0x7f048c15e480 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f0492c00640 1 -- 172.21.15.87:0/1854347052 >> 172.21.15.87:0/1854347052 conn(0x7f048c0669a0 msgr2=0x7f048c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f0492c00640 1 -- 172.21.15.87:0/1854347052 shutdown_connections 2024-04-25T10:16:05.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.395+0000 7f0492c00640 1 -- 172.21.15.87:0/1854347052 wait complete. 2024-04-25T10:16:05.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 Processor -- start 2024-04-25T10:16:05.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 -- start start 2024-04-25T10:16:05.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c070350 0x7f048c14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c0aef80 0x7f048c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f048c0b31c0 0x7f048c14a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0cfef0 con 0x7f048c070350 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0cfd70 con 0x7f048c0aef80 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0b35d0 con 0x7f048c0b31c0 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c070350 0x7f048c14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c070350 0x7f048c14a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44624/0 (socket says 172.21.15.87:44624) 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 -- 172.21.15.87:0/4082015699 learned_addr learned my addr 172.21.15.87:0/4082015699 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048b400640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c0aef80 0x7f048c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 -- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f048c0b31c0 msgr2=0x7f048c14a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f048c0b31c0 0x7f048c14a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 -- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c0aef80 msgr2=0x7f048c150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c0aef80 0x7f048c150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 -- 172.21.15.87:0/4082015699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0b00e0 con 0x7f048c070350 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048b400640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c0aef80 0x7f048c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f048be00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c070350 0x7f048c14a130 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f047c0ad040 tx=0x7f047c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0483e00640 1 -- 172.21.15.87:0/4082015699 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c004230 con 0x7f048c070350 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0483e00640 1 -- 172.21.15.87:0/4082015699 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c0a7070 con 0x7f048c070350 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0483e00640 1 -- 172.21.15.87:0/4082015699 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c003620 con 0x7f048c070350 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c0b03c0 con 0x7f048c070350 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c0b0980 con 0x7f048c070350 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.399+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c04ef30 con 0x7f048c070350 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.403+0000 7f0483e00640 1 -- 172.21.15.87:0/4082015699 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f047c0037c0 con 0x7f048c070350 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.403+0000 7f0483e00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f046c066f60 0x7f046c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.403+0000 7f0483e00640 1 -- 172.21.15.87:0/4082015699 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f047c11fd30 con 0x7f048c070350 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.403+0000 7f048b400640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f046c066f60 0x7f046c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.403+0000 7f048b400640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f046c066f60 0x7f046c069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0478000f70 tx=0x7f0478009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.407+0000 7f0483e00640 1 -- 172.21.15.87:0/4082015699 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f047c016090 con 0x7f048c070350 2024-04-25T10:16:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:05.595+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f048c0cc0f0 con 0x7f048c070350 2024-04-25T10:16:06.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.227+0000 7f0483e00640 1 -- 172.21.15.87:0/4082015699 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f047c0f0dc0 con 0x7f048c070350 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f046c066f60 msgr2=0x7f046c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f046c066f60 0x7f046c069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0478000f70 tx=0x7f0478009040 comp rx=0 tx=0).stop 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c070350 msgr2=0x7f048c14a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c070350 0x7f048c14a130 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f047c0ad040 tx=0x7f047c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 shutdown_connections 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f046c066f60 0x7f046c069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f048c0b31c0 0x7f048c14a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c0aef80 0x7f048c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 --2- 172.21.15.87:0/4082015699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f048c070350 0x7f048c14a130 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 >> 172.21.15.87:0/4082015699 conn(0x7f048c0669a0 msgr2=0x7f048c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 shutdown_connections 2024-04-25T10:16:06.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.231+0000 7f0492c00640 1 -- 172.21.15.87:0/4082015699 wait complete. 2024-04-25T10:16:06.256 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:16:06.256 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:06.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.387+0000 7fd4e2800640 1 Processor -- start 2024-04-25T10:16:06.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.387+0000 7fd4e2800640 1 -- start start 2024-04-25T10:16:06.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4e2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4dc0cd2e0 0x7fd4dc0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4e2800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc15f3b0 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4e2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc0ca810 con 0x7fd4dc0b0f70 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4e2800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc15ef20 con 0x7fd4dc0cd2e0 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4d2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd4dc0cfd90 0x7fd4dc0ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36548/0 (socket says 172.21.15.87:36548) 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4d2a00640 1 -- 172.21.15.87:0/2335369737 learned_addr learned my addr 172.21.15.87:0/2335369737 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4dbe00640 1 --2- 172.21.15.87:0/2335369737 >> v2:172.21.15.87:3300/0 conn(0x7fd4dc0cd2e0 0x7fd4dc0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4daa00640 1 -- 172.21.15.87:0/2335369737 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211267828 0 0) 0x7fd4dc15f3b0 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4daa00640 1 -- 172.21.15.87:0/2335369737 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c0003530 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4dbe00640 1 -- 172.21.15.87:0/2335369737 >> v1:172.21.15.169:6789/0 conn(0x7fd4dc0b0f70 legacy=0x7fd4dc0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4dbe00640 1 -- 172.21.15.87:0/2335369737 >> v1:172.21.15.87:6789/0 conn(0x7fd4dc0cfd90 legacy=0x7fd4dc0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4dbe00640 1 -- 172.21.15.87:0/2335369737 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc15e5f0 con 0x7fd4dc0cd2e0 2024-04-25T10:16:06.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.391+0000 7fd4dbe00640 1 --2- 172.21.15.87:0/2335369737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cd2e0 0x7fd4dc0cba20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc15f0a0 tx=0x7fd4d401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ac4020d09f3ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4daa00640 1 -- 172.21.15.87:0/2335369737 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40b2070 con 0x7fd4dc0cd2e0 2024-04-25T10:16:06.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4daa00640 1 -- 172.21.15.87:0/2335369737 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d40a5810 con 0x7fd4dc0cd2e0 2024-04-25T10:16:06.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4daa00640 1 -- 172.21.15.87:0/2335369737 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40a5be0 con 0x7fd4dc0cd2e0 2024-04-25T10:16:06.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- 172.21.15.87:0/2335369737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cd2e0 msgr2=0x7fd4dc0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 --2- 172.21.15.87:0/2335369737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cd2e0 0x7fd4dc0cba20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc15f0a0 tx=0x7fd4d401c950 comp rx=0 tx=0).stop 2024-04-25T10:16:06.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- 172.21.15.87:0/2335369737 shutdown_connections 2024-04-25T10:16:06.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 --2- 172.21.15.87:0/2335369737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cd2e0 0x7fd4dc0cba20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- 172.21.15.87:0/2335369737 >> 172.21.15.87:0/2335369737 conn(0x7fd4dc0b1d50 msgr2=0x7fd4dc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- 172.21.15.87:0/2335369737 shutdown_connections 2024-04-25T10:16:06.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- 172.21.15.87:0/2335369737 wait complete. 2024-04-25T10:16:06.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 Processor -- start 2024-04-25T10:16:06.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- start start 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4dc0b0f70 0x7fd4dc1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc0cd2e0 0x7fd4dc099350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cfd90 0x7fd4dc1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc16c110 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc16bf90 con 0x7fd4dc0cd2e0 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4e2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc16c290 con 0x7fd4dc0b0f70 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cfd90 0x7fd4dc1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cfd90 0x7fd4dc1568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44644/0 (socket says 172.21.15.87:44644) 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4dbe00640 1 -- 172.21.15.87:0/507525939 learned_addr learned my addr 172.21.15.87:0/507525939 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4dbe00640 1 -- 172.21.15.87:0/507525939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4dc0b0f70 msgr2=0x7fd4dc1521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4dbe00640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4dc0b0f70 0x7fd4dc1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4d2a00640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc0cd2e0 0x7fd4dc099350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4dbe00640 1 -- 172.21.15.87:0/507525939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc0cd2e0 msgr2=0x7fd4dc099350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.395+0000 7fd4dbe00640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc0cd2e0 0x7fd4dc099350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4dbe00640 1 -- 172.21.15.87:0/507525939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc099890 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4dbe00640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cfd90 0x7fd4dc1568b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd4d401a580 tx=0x7fd4d40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4d3e00640 1 -- 172.21.15.87:0/507525939 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40b2050 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4d3e00640 1 -- 172.21.15.87:0/507525939 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d40be070 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4d3e00640 1 -- 172.21.15.87:0/507525939 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40ad890 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc098ab0 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc099070 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc04ef30 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4d3e00640 1 -- 172.21.15.87:0/507525939 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4d40ada30 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4d3e00640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4b8066f60 0x7fd4b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4d3e00640 1 -- 172.21.15.87:0/507525939 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4d41201c0 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.399+0000 7fd4db400640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4b8066f60 0x7fd4b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.403+0000 7fd4db400640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4b8066f60 0x7fd4b8069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc000f70 tx=0x7fd4cc0079c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.403+0000 7fd4d3e00640 1 -- 172.21.15.87:0/507525939 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4d4016540 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4dc0c7830 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4d3e00640 1 -- 172.21.15.87:0/507525939 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fd4d40f1140 con 0x7fd4dc0cfd90 2024-04-25T10:16:06.597 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:06.597 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/243216881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":243216881},{"type":"v1","addr":"172.21.15.169:6835","nonce":243216881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6073,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/250986881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":250986881},{"type":"v1","addr":"172.21.15.169:6834","nonce":250986881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6510,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2928764815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2928764815},{"type":"v1","addr":"172.21.15.87:6836","nonce":2928764815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:05.597920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6513":{"gid":6513,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3387238920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3387238920},{"type":"v1","addr":"172.21.15.87:6837","nonce":3387238920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:06.597 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 37 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4b8066f60 msgr2=0x7fd4b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4b8066f60 0x7fd4b8069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc000f70 tx=0x7fd4cc0079c0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cfd90 msgr2=0x7fd4dc1568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cfd90 0x7fd4dc1568b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd4d401a580 tx=0x7fd4d40196f0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 shutdown_connections 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4b8066f60 0x7fd4b8069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4dc0cfd90 0x7fd4dc1568b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc0cd2e0 0x7fd4dc099350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 --2- 172.21.15.87:0/507525939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4dc0b0f70 0x7fd4dc1521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 >> 172.21.15.87:0/507525939 conn(0x7fd4dc0b1d50 msgr2=0x7fd4dc0b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 shutdown_connections 2024-04-25T10:16:06.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.591+0000 7fd4e2800640 1 -- 172.21.15.87:0/507525939 wait complete. 2024-04-25T10:16:06.618 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.747+0000 7f6718a00640 1 Processor -- start 2024-04-25T10:16:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.747+0000 7f6718a00640 1 -- start start 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6718a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67140b1f50 0x7f67140b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6718a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67140b3dd0 0x7f671406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6718a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f671406b1d0 0x7f67140cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6718a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140d0b50 con 0x7f67140b1f50 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6718a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671406b680 con 0x7f671406b1d0 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6718a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671406be90 con 0x7f67140b3dd0 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67140b1f50 0x7f67140b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67140b1f50 0x7f67140b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44658/0 (socket says 172.21.15.87:44658) 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 -- 172.21.15.87:0/3966578291 learned_addr learned my addr 172.21.15.87:0/3966578291 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6713200640 1 --2- 172.21.15.87:0/3966578291 >> v2:172.21.15.169:3300/0 conn(0x7f671406b1d0 0x7f67140cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6711e00640 1 --2- 172.21.15.87:0/3966578291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67140b3dd0 0x7f671406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 -- 172.21.15.87:0/3966578291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67140b3dd0 msgr2=0x7f671406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 --2- 172.21.15.87:0/3966578291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67140b3dd0 0x7f671406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 -- 172.21.15.87:0/3966578291 >> v2:172.21.15.169:3300/0 conn(0x7f671406b1d0 msgr2=0x7f67140cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 --2- 172.21.15.87:0/3966578291 >> v2:172.21.15.169:3300/0 conn(0x7f671406b1d0 0x7f67140cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 -- 172.21.15.87:0/3966578291 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671406b8d0 con 0x7f67140b1f50 2024-04-25T10:16:06.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.751+0000 7f6712800640 1 --2- 172.21.15.87:0/3966578291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b23c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f67140cbd40 tx=0x7f670401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7441dcb44efbe92c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711400640 1 -- 172.21.15.87:0/3966578291 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040b0070 con 0x7f67140b1f50 2024-04-25T10:16:06.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711400640 1 -- 172.21.15.87:0/3966578291 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040a4e50 con 0x7f67140b1f50 2024-04-25T10:16:06.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711400640 1 -- 172.21.15.87:0/3966578291 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040ab820 con 0x7f67140b1f50 2024-04-25T10:16:06.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- 172.21.15.87:0/3966578291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 msgr2=0x7f67140b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 --2- 172.21.15.87:0/3966578291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b23c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f67140cbd40 tx=0x7f670401c400 comp rx=0 tx=0).stop 2024-04-25T10:16:06.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- 172.21.15.87:0/3966578291 shutdown_connections 2024-04-25T10:16:06.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 --2- 172.21.15.87:0/3966578291 >> v2:172.21.15.169:3300/0 conn(0x7f671406b1d0 0x7f67140cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 --2- 172.21.15.87:0/3966578291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67140b3dd0 0x7f671406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 --2- 172.21.15.87:0/3966578291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b23c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- 172.21.15.87:0/3966578291 >> 172.21.15.87:0/3966578291 conn(0x7f67140b32b0 msgr2=0x7f67140b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- 172.21.15.87:0/3966578291 shutdown_connections 2024-04-25T10:16:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- 172.21.15.87:0/3966578291 wait complete. 2024-04-25T10:16:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 Processor -- start 2024-04-25T10:16:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- start start 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671406b1d0 0x7f67140b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67140b3dd0 0x7f67140b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140cbdb0 con 0x7f67140b1f50 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140d0450 con 0x7f67140b3dd0 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6718a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67140cbf30 con 0x7f671406b1d0 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44670/0 (socket says 172.21.15.87:44670) 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711e00640 1 -- 172.21.15.87:0/534292730 learned_addr learned my addr 172.21.15.87:0/534292730 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6712800640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671406b1d0 0x7f67140b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6713200640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67140b3dd0 0x7f67140b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711e00640 1 -- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671406b1d0 msgr2=0x7f67140b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.755+0000 7f6711e00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671406b1d0 0x7f67140b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6711e00640 1 -- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67140b3dd0 msgr2=0x7f67140b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6711e00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67140b3dd0 0x7f67140b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6711e00640 1 -- 172.21.15.87:0/534292730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67141553a0 con 0x7f67140b1f50 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6711e00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b68c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f670000d5f0 tx=0x7f670000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f670aa00640 1 -- 172.21.15.87:0/534292730 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000125f0 con 0x7f67140b1f50 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f670aa00640 1 -- 172.21.15.87:0/534292730 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700019070 con 0x7f67140b1f50 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6714155680 con 0x7f67140b1f50 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f670aa00640 1 -- 172.21.15.87:0/534292730 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000129b0 con 0x7f67140b1f50 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6714156010 con 0x7f67140b1f50 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714075120 con 0x7f67140b1f50 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f670aa00640 1 -- 172.21.15.87:0/534292730 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6700012b50 con 0x7f67140b1f50 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f670aa00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f4066f60 0x7f66f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f670aa00640 1 -- 172.21.15.87:0/534292730 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6700085390 con 0x7f67140b1f50 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.759+0000 7f6712800640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f4066f60 0x7f66f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.763+0000 7f6712800640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f4066f60 0x7f66f4069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f67040ad000 tx=0x7f67040a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.763+0000 7f670aa00640 1 -- 172.21.15.87:0/534292730 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6700055cb0 con 0x7f67140b1f50 2024-04-25T10:16:06.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.919+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67140cd4d0 con 0x7f67140b1f50 2024-04-25T10:16:06.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.919+0000 7f670aa00640 1 -- 172.21.15.87:0/534292730 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f6700059d50 con 0x7f67140b1f50 2024-04-25T10:16:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:06.927 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":57,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:04.204866+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:06.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f4066f60 msgr2=0x7f66f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f4066f60 0x7f66f4069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f67040ad000 tx=0x7f67040a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:06.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 msgr2=0x7f67140b68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b68c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f670000d5f0 tx=0x7f670000dab0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 shutdown_connections 2024-04-25T10:16:06.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66f4066f60 0x7f66f4069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67140b3dd0 0x7f67140b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67140b1f50 0x7f67140b68c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 --2- 172.21.15.87:0/534292730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671406b1d0 0x7f67140b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 >> 172.21.15.87:0/534292730 conn(0x7f67140b32b0 msgr2=0x7f671415dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 shutdown_connections 2024-04-25T10:16:06.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:06.923+0000 7f6718a00640 1 -- 172.21.15.87:0/534292730 wait complete. 2024-04-25T10:16:06.951 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:16:06.951 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:07.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc07e000640 1 Processor -- start 2024-04-25T10:16:07.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc07e000640 1 -- start start 2024-04-25T10:16:07.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc07e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0781550b0 0x7fc0780cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc07e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0780cfe70 0x7fc07815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc07e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc078071050 con 0x7fc07815b740 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc07e000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07815d040 con 0x7fc0781550b0 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc07e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0780712f0 con 0x7fc0780cfe70 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc077400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0781550b0 0x7fc0780cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc077400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0781550b0 0x7fc0780cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44672/0 (socket says 172.21.15.87:44672) 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc077400640 1 -- 172.21.15.87:0/1933864501 learned_addr learned my addr 172.21.15.87:0/1933864501 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc076a00640 1 --2- 172.21.15.87:0/1933864501 >> v2:172.21.15.169:3300/0 conn(0x7fc0780cfe70 0x7fc07815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc077400640 1 -- 172.21.15.87:0/1933864501 >> v2:172.21.15.169:3300/0 conn(0x7fc0780cfe70 msgr2=0x7fc07815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc077400640 1 --2- 172.21.15.87:0/1933864501 >> v2:172.21.15.169:3300/0 conn(0x7fc0780cfe70 0x7fc07815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc077400640 1 -- 172.21.15.87:0/1933864501 >> v1:172.21.15.169:6789/0 conn(0x7fc07815b740 legacy=0x7fc07806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.079+0000 7fc077400640 1 -- 172.21.15.87:0/1933864501 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0780715d0 con 0x7fc0781550b0 2024-04-25T10:16:07.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc077400640 1 --2- 172.21.15.87:0/1933864501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0781550b0 0x7fc0780cf930 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc068009870 tx=0x7fc06801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cf8de7e19e1ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc076000640 1 -- 172.21.15.87:0/1933864501 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680b0070 con 0x7fc0781550b0 2024-04-25T10:16:07.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc076000640 1 -- 172.21.15.87:0/1933864501 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0680ab600 con 0x7fc0781550b0 2024-04-25T10:16:07.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc076000640 1 -- 172.21.15.87:0/1933864501 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680ab9d0 con 0x7fc0781550b0 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 -- 172.21.15.87:0/1933864501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0781550b0 msgr2=0x7fc0780cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 --2- 172.21.15.87:0/1933864501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0781550b0 0x7fc0780cf930 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc068009870 tx=0x7fc06801c520 comp rx=0 tx=0).stop 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 -- 172.21.15.87:0/1933864501 shutdown_connections 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 --2- 172.21.15.87:0/1933864501 >> v2:172.21.15.169:3300/0 conn(0x7fc0780cfe70 0x7fc07815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 --2- 172.21.15.87:0/1933864501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0781550b0 0x7fc0780cf930 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 -- 172.21.15.87:0/1933864501 >> 172.21.15.87:0/1933864501 conn(0x7fc078066b70 msgr2=0x7fc078066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 -- 172.21.15.87:0/1933864501 shutdown_connections 2024-04-25T10:16:07.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.083+0000 7fc07e000640 1 -- 172.21.15.87:0/1933864501 wait complete. 2024-04-25T10:16:07.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 Processor -- start 2024-04-25T10:16:07.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 -- start start 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0780cfe70 0x7fc078146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0781550b0 0x7fc078157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc07815b740 0x7fc07814a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07815bcd0 con 0x7fc0780cfe70 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07815bb50 con 0x7fc0781550b0 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07815c030 con 0x7fc07815b740 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0780cfe70 0x7fc078146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0780cfe70 0x7fc078146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44686/0 (socket says 172.21.15.87:44686) 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 -- 172.21.15.87:0/1821992801 learned_addr learned my addr 172.21.15.87:0/1821992801 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077e00640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc07815b740 0x7fc07814a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc076a00640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0781550b0 0x7fc078157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 -- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc07815b740 msgr2=0x7fc07814a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc07815b740 0x7fc07814a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 -- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0781550b0 msgr2=0x7fc078157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0781550b0 0x7fc078157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 -- 172.21.15.87:0/1821992801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078155da0 con 0x7fc0780cfe70 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc077400640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0780cfe70 0x7fc078146a80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc0680ae040 tx=0x7fc06801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc06f400640 1 -- 172.21.15.87:0/1821992801 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680b0040 con 0x7fc0780cfe70 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc06f400640 1 -- 172.21.15.87:0/1821992801 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0680bc070 con 0x7fc0780cfe70 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc078156060 con 0x7fc0780cfe70 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc06f400640 1 -- 172.21.15.87:0/1821992801 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680b6680 con 0x7fc0780cfe70 2024-04-25T10:16:07.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.087+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc078156620 con 0x7fc0780cfe70 2024-04-25T10:16:07.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.091+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07804ef30 con 0x7fc0780cfe70 2024-04-25T10:16:07.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.091+0000 7fc06f400640 1 -- 172.21.15.87:0/1821992801 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0680b6820 con 0x7fc0780cfe70 2024-04-25T10:16:07.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.091+0000 7fc06f400640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc058066f90 0x7fc058069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.091+0000 7fc06f400640 1 -- 172.21.15.87:0/1821992801 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc06811f300 con 0x7fc0780cfe70 2024-04-25T10:16:07.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.091+0000 7fc076a00640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc058066f90 0x7fc058069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.091+0000 7fc076a00640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc058066f90 0x7fc058069450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc0780cc4d0 tx=0x7fc064008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.095+0000 7fc06f400640 1 -- 172.21.15.87:0/1821992801 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc068016110 con 0x7fc0780cfe70 2024-04-25T10:16:07.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.247+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc07806e3a0 con 0x7fc058066f90 2024-04-25T10:16:07.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.255+0000 7fc06f400640 1 -- 172.21.15.87:0/1821992801 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251955 (secure 0 0 0) 0x7fc07806e3a0 con 0x7fc058066f90 2024-04-25T10:16:07.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:07.266 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc058066f90 msgr2=0x7fc058069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc058066f90 0x7fc058069450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc0780cc4d0 tx=0x7fc064008040 comp rx=0 tx=0).stop 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0780cfe70 msgr2=0x7fc078146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0780cfe70 0x7fc078146a80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc0680ae040 tx=0x7fc06801ad60 comp rx=0 tx=0).stop 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 shutdown_connections 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc058066f90 0x7fc058069450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc07815b740 0x7fc07814a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0781550b0 0x7fc078157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 --2- 172.21.15.87:0/1821992801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0780cfe70 0x7fc078146a80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 >> 172.21.15.87:0/1821992801 conn(0x7fc078066b70 msgr2=0x7fc0780b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 shutdown_connections 2024-04-25T10:16:07.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.259+0000 7fc07e000640 1 -- 172.21.15.87:0/1821992801 wait complete. 2024-04-25T10:16:07.284 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-04-25T10:16:07.172314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324004,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":317299,"kb_avail":754650716,"statfs":{"total":773094113280,"available":772762333184,"internally_reserved":0,"allocated":6725632,"data_stored":3468348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324914737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.990587"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110527+0000","last_change":"2024-04-25T10:16:03.207648+0000","last_active":"2024-04-25T10:16:05.110527+0000","last_peered":"2024-04-25T10:16:05.110527+0000","last_clean":"2024-04-25T10:16:05.110527+0000","last_became_active":"2024-04-25T10:16:03.207366+0000","last_became_peered":"2024-04-25T10:16:03.207366+0000","last_unstale":"2024-04-25T10:16:05.110527+0000","last_undegraded":"2024-04-25T10:16:05.110527+0000","last_fullsized":"2024-04-25T10:16:05.110527+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:21.635103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110961+0000","last_change":"2024-04-25T10:16:03.220908+0000","last_active":"2024-04-25T10:16:05.110961+0000","last_peered":"2024-04-25T10:16:05.110961+0000","last_clean":"2024-04-25T10:16:05.110961+0000","last_became_active":"2024-04-25T10:16:03.220561+0000","last_became_peered":"2024-04-25T10:16:03.220561+0000","last_unstale":"2024-04-25T10:16:05.110961+0000","last_undegraded":"2024-04-25T10:16:05.110961+0000","last_fullsized":"2024-04-25T10:16:05.110961+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:52.987198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112487+0000","last_change":"2024-04-25T10:16:03.218434+0000","last_active":"2024-04-25T10:16:05.112487+0000","last_peered":"2024-04-25T10:16:05.112487+0000","last_clean":"2024-04-25T10:16:05.112487+0000","last_became_active":"2024-04-25T10:16:03.218154+0000","last_became_peered":"2024-04-25T10:16:03.218154+0000","last_unstale":"2024-04-25T10:16:05.112487+0000","last_undegraded":"2024-04-25T10:16:05.112487+0000","last_fullsized":"2024-04-25T10:16:05.112487+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:08.979302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218972+0000","last_change":"2024-04-25T10:16:03.221513+0000","last_active":"2024-04-25T10:16:04.218972+0000","last_peered":"2024-04-25T10:16:04.218972+0000","last_clean":"2024-04-25T10:16:04.218972+0000","last_became_active":"2024-04-25T10:16:03.221151+0000","last_became_peered":"2024-04-25T10:16:03.221151+0000","last_unstale":"2024-04-25T10:16:04.218972+0000","last_undegraded":"2024-04-25T10:16:04.218972+0000","last_fullsized":"2024-04-25T10:16:04.218972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:34.106239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110175+0000","last_change":"2024-04-25T10:16:03.216215+0000","last_active":"2024-04-25T10:16:05.110175+0000","last_peered":"2024-04-25T10:16:05.110175+0000","last_clean":"2024-04-25T10:16:05.110175+0000","last_became_active":"2024-04-25T10:16:03.215966+0000","last_became_peered":"2024-04-25T10:16:03.215966+0000","last_unstale":"2024-04-25T10:16:05.110175+0000","last_undegraded":"2024-04-25T10:16:05.110175+0000","last_fullsized":"2024-04-25T10:16:05.110175+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:41.370363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099432+0000","last_change":"2024-04-25T10:16:03.233166+0000","last_active":"2024-04-25T10:16:05.099432+0000","last_peered":"2024-04-25T10:16:05.099432+0000","last_clean":"2024-04-25T10:16:05.099432+0000","last_became_active":"2024-04-25T10:16:03.229236+0000","last_became_peered":"2024-04-25T10:16:03.229236+0000","last_unstale":"2024-04-25T10:16:05.099432+0000","last_undegraded":"2024-04-25T10:16:05.099432+0000","last_fullsized":"2024-04-25T10:16:05.099432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.241014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219133+0000","last_change":"2024-04-25T10:16:03.210282+0000","last_active":"2024-04-25T10:16:04.219133+0000","last_peered":"2024-04-25T10:16:04.219133+0000","last_clean":"2024-04-25T10:16:04.219133+0000","last_became_active":"2024-04-25T10:16:03.209656+0000","last_became_peered":"2024-04-25T10:16:03.209656+0000","last_unstale":"2024-04-25T10:16:04.219133+0000","last_undegraded":"2024-04-25T10:16:04.219133+0000","last_fullsized":"2024-04-25T10:16:04.219133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:16.346104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110422+0000","last_change":"2024-04-25T10:16:03.216789+0000","last_active":"2024-04-25T10:16:05.110422+0000","last_peered":"2024-04-25T10:16:05.110422+0000","last_clean":"2024-04-25T10:16:05.110422+0000","last_became_active":"2024-04-25T10:16:03.216523+0000","last_became_peered":"2024-04-25T10:16:03.216523+0000","last_unstale":"2024-04-25T10:16:05.110422+0000","last_undegraded":"2024-04-25T10:16:05.110422+0000","last_fullsized":"2024-04-25T10:16:05.110422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:56.223109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218743+0000","last_change":"2024-04-25T10:16:03.228972+0000","last_active":"2024-04-25T10:16:04.218743+0000","last_peered":"2024-04-25T10:16:04.218743+0000","last_clean":"2024-04-25T10:16:04.218743+0000","last_became_active":"2024-04-25T10:16:03.228486+0000","last_became_peered":"2024-04-25T10:16:03.228486+0000","last_unstale":"2024-04-25T10:16:04.218743+0000","last_undegraded":"2024-04-25T10:16:04.218743+0000","last_fullsized":"2024-04-25T10:16:04.218743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:41.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218275+0000","last_change":"2024-04-25T10:16:03.217474+0000","last_active":"2024-04-25T10:16:04.218275+0000","last_peered":"2024-04-25T10:16:04.218275+0000","last_clean":"2024-04-25T10:16:04.218275+0000","last_became_active":"2024-04-25T10:16:03.216092+0000","last_became_peered":"2024-04-25T10:16:03.216092+0000","last_unstale":"2024-04-25T10:16:04.218275+0000","last_undegraded":"2024-04-25T10:16:04.218275+0000","last_fullsized":"2024-04-25T10:16:04.218275+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:52.179510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218196+0000","last_change":"2024-04-25T10:16:03.207581+0000","last_active":"2024-04-25T10:16:04.218196+0000","last_peered":"2024-04-25T10:16:04.218196+0000","last_clean":"2024-04-25T10:16:04.218196+0000","last_became_active":"2024-04-25T10:16:03.206833+0000","last_became_peered":"2024-04-25T10:16:03.206833+0000","last_unstale":"2024-04-25T10:16:04.218196+0000","last_undegraded":"2024-04-25T10:16:04.218196+0000","last_fullsized":"2024-04-25T10:16:04.218196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:38.087836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219176+0000","last_change":"2024-04-25T10:16:03.222688+0000","last_active":"2024-04-25T10:16:04.219176+0000","last_peered":"2024-04-25T10:16:04.219176+0000","last_clean":"2024-04-25T10:16:04.219176+0000","last_became_active":"2024-04-25T10:16:03.222423+0000","last_became_peered":"2024-04-25T10:16:03.222423+0000","last_unstale":"2024-04-25T10:16:04.219176+0000","last_undegraded":"2024-04-25T10:16:04.219176+0000","last_fullsized":"2024-04-25T10:16:04.219176+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:42.808262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112092+0000","last_change":"2024-04-25T10:16:03.227329+0000","last_active":"2024-04-25T10:16:05.112092+0000","last_peered":"2024-04-25T10:16:05.112092+0000","last_clean":"2024-04-25T10:16:05.112092+0000","last_became_active":"2024-04-25T10:16:03.226979+0000","last_became_peered":"2024-04-25T10:16:03.226979+0000","last_unstale":"2024-04-25T10:16:05.112092+0000","last_undegraded":"2024-04-25T10:16:05.112092+0000","last_fullsized":"2024-04-25T10:16:05.112092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:03.053320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218874+0000","last_change":"2024-04-25T10:16:03.209188+0000","last_active":"2024-04-25T10:16:04.218874+0000","last_peered":"2024-04-25T10:16:04.218874+0000","last_clean":"2024-04-25T10:16:04.218874+0000","last_became_active":"2024-04-25T10:16:03.208911+0000","last_became_peered":"2024-04-25T10:16:03.208911+0000","last_unstale":"2024-04-25T10:16:04.218874+0000","last_undegraded":"2024-04-25T10:16:04.218874+0000","last_fullsized":"2024-04-25T10:16:04.218874+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:32.609230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218875+0000","last_change":"2024-04-25T10:16:03.213952+0000","last_active":"2024-04-25T10:16:04.218875+0000","last_peered":"2024-04-25T10:16:04.218875+0000","last_clean":"2024-04-25T10:16:04.218875+0000","last_became_active":"2024-04-25T10:16:03.213659+0000","last_became_peered":"2024-04-25T10:16:03.213659+0000","last_unstale":"2024-04-25T10:16:04.218875+0000","last_undegraded":"2024-04-25T10:16:04.218875+0000","last_fullsized":"2024-04-25T10:16:04.218875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:13.579259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099201+0000","last_change":"2024-04-25T10:16:03.226462+0000","last_active":"2024-04-25T10:16:05.099201+0000","last_peered":"2024-04-25T10:16:05.099201+0000","last_clean":"2024-04-25T10:16:05.099201+0000","last_became_active":"2024-04-25T10:16:03.225260+0000","last_became_peered":"2024-04-25T10:16:03.225260+0000","last_unstale":"2024-04-25T10:16:05.099201+0000","last_undegraded":"2024-04-25T10:16:05.099201+0000","last_fullsized":"2024-04-25T10:16:05.099201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:42.808262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111156+0000","last_change":"2024-04-25T10:16:03.212568+0000","last_active":"2024-04-25T10:16:05.111156+0000","last_peered":"2024-04-25T10:16:05.111156+0000","last_clean":"2024-04-25T10:16:05.111156+0000","last_became_active":"2024-04-25T10:16:03.212277+0000","last_became_peered":"2024-04-25T10:16:03.212277+0000","last_unstale":"2024-04-25T10:16:05.111156+0000","last_undegraded":"2024-04-25T10:16:05.111156+0000","last_fullsized":"2024-04-25T10:16:05.111156+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:58.625880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112846+0000","last_change":"2024-04-25T10:16:03.225756+0000","last_active":"2024-04-25T10:16:05.112846+0000","last_peered":"2024-04-25T10:16:05.112846+0000","last_clean":"2024-04-25T10:16:05.112846+0000","last_became_active":"2024-04-25T10:16:03.225419+0000","last_became_peered":"2024-04-25T10:16:03.225419+0000","last_unstale":"2024-04-25T10:16:05.112846+0000","last_undegraded":"2024-04-25T10:16:05.112846+0000","last_fullsized":"2024-04-25T10:16:05.112846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:28.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110518+0000","last_change":"2024-04-25T10:16:03.215676+0000","last_active":"2024-04-25T10:16:05.110518+0000","last_peered":"2024-04-25T10:16:05.110518+0000","last_clean":"2024-04-25T10:16:05.110518+0000","last_became_active":"2024-04-25T10:16:03.215329+0000","last_became_peered":"2024-04-25T10:16:03.215329+0000","last_unstale":"2024-04-25T10:16:05.110518+0000","last_undegraded":"2024-04-25T10:16:05.110518+0000","last_fullsized":"2024-04-25T10:16:05.110518+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:21.687178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110630+0000","last_change":"2024-04-25T10:16:03.233825+0000","last_active":"2024-04-25T10:16:05.110630+0000","last_peered":"2024-04-25T10:16:05.110630+0000","last_clean":"2024-04-25T10:16:05.110630+0000","last_became_active":"2024-04-25T10:16:03.233552+0000","last_became_peered":"2024-04-25T10:16:03.233552+0000","last_unstale":"2024-04-25T10:16:05.110630+0000","last_undegraded":"2024-04-25T10:16:05.110630+0000","last_fullsized":"2024-04-25T10:16:05.110630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:01.566191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112057+0000","last_change":"2024-04-25T10:16:03.217052+0000","last_active":"2024-04-25T10:16:05.112057+0000","last_peered":"2024-04-25T10:16:05.112057+0000","last_clean":"2024-04-25T10:16:05.112057+0000","last_became_active":"2024-04-25T10:16:03.216781+0000","last_became_peered":"2024-04-25T10:16:03.216781+0000","last_unstale":"2024-04-25T10:16:05.112057+0000","last_undegraded":"2024-04-25T10:16:05.112057+0000","last_fullsized":"2024-04-25T10:16:05.112057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:50.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.109977+0000","last_change":"2024-04-25T10:16:03.210711+0000","last_active":"2024-04-25T10:16:05.109977+0000","last_peered":"2024-04-25T10:16:05.109977+0000","last_clean":"2024-04-25T10:16:05.109977+0000","last_became_active":"2024-04-25T10:16:03.210257+0000","last_became_peered":"2024-04-25T10:16:03.210257+0000","last_unstale":"2024-04-25T10:16:05.109977+0000","last_undegraded":"2024-04-25T10:16:05.109977+0000","last_fullsized":"2024-04-25T10:16:05.109977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:28.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112585+0000","last_change":"2024-04-25T10:16:03.224992+0000","last_active":"2024-04-25T10:16:05.112585+0000","last_peered":"2024-04-25T10:16:05.112585+0000","last_clean":"2024-04-25T10:16:05.112585+0000","last_became_active":"2024-04-25T10:16:03.224677+0000","last_became_peered":"2024-04-25T10:16:03.224677+0000","last_unstale":"2024-04-25T10:16:05.112585+0000","last_undegraded":"2024-04-25T10:16:05.112585+0000","last_fullsized":"2024-04-25T10:16:05.112585+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:26.464110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111718+0000","last_change":"2024-04-25T10:16:03.226568+0000","last_active":"2024-04-25T10:16:05.111718+0000","last_peered":"2024-04-25T10:16:05.111718+0000","last_clean":"2024-04-25T10:16:05.111718+0000","last_became_active":"2024-04-25T10:16:03.226222+0000","last_became_peered":"2024-04-25T10:16:03.226222+0000","last_unstale":"2024-04-25T10:16:05.111718+0000","last_undegraded":"2024-04-25T10:16:05.111718+0000","last_fullsized":"2024-04-25T10:16:05.111718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.928358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110711+0000","last_change":"2024-04-25T10:16:03.213966+0000","last_active":"2024-04-25T10:16:05.110711+0000","last_peered":"2024-04-25T10:16:05.110711+0000","last_clean":"2024-04-25T10:16:05.110711+0000","last_became_active":"2024-04-25T10:16:03.207641+0000","last_became_peered":"2024-04-25T10:16:03.207641+0000","last_unstale":"2024-04-25T10:16:05.110711+0000","last_undegraded":"2024-04-25T10:16:05.110711+0000","last_fullsized":"2024-04-25T10:16:05.110711+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:28.515844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219478+0000","last_change":"2024-04-25T10:16:03.213365+0000","last_active":"2024-04-25T10:16:04.219478+0000","last_peered":"2024-04-25T10:16:04.219478+0000","last_clean":"2024-04-25T10:16:04.219478+0000","last_became_active":"2024-04-25T10:16:03.212977+0000","last_became_peered":"2024-04-25T10:16:03.212977+0000","last_unstale":"2024-04-25T10:16:04.219478+0000","last_undegraded":"2024-04-25T10:16:04.219478+0000","last_fullsized":"2024-04-25T10:16:04.219478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:25.592075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099198+0000","last_change":"2024-04-25T10:16:03.238000+0000","last_active":"2024-04-25T10:16:05.099198+0000","last_peered":"2024-04-25T10:16:05.099198+0000","last_clean":"2024-04-25T10:16:05.099198+0000","last_became_active":"2024-04-25T10:16:03.234047+0000","last_became_peered":"2024-04-25T10:16:03.234047+0000","last_unstale":"2024-04-25T10:16:05.099198+0000","last_undegraded":"2024-04-25T10:16:05.099198+0000","last_fullsized":"2024-04-25T10:16:05.099198+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:34.106239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110655+0000","last_change":"2024-04-25T10:16:03.213779+0000","last_active":"2024-04-25T10:16:05.110655+0000","last_peered":"2024-04-25T10:16:05.110655+0000","last_clean":"2024-04-25T10:16:05.110655+0000","last_became_active":"2024-04-25T10:16:03.213471+0000","last_became_peered":"2024-04-25T10:16:03.213471+0000","last_unstale":"2024-04-25T10:16:05.110655+0000","last_undegraded":"2024-04-25T10:16:05.110655+0000","last_fullsized":"2024-04-25T10:16:05.110655+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:56.118359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110609+0000","last_change":"2024-04-25T10:16:03.208290+0000","last_active":"2024-04-25T10:16:05.110609+0000","last_peered":"2024-04-25T10:16:05.110609+0000","last_clean":"2024-04-25T10:16:05.110609+0000","last_became_active":"2024-04-25T10:16:03.207975+0000","last_became_peered":"2024-04-25T10:16:03.207975+0000","last_unstale":"2024-04-25T10:16:05.110609+0000","last_undegraded":"2024-04-25T10:16:05.110609+0000","last_fullsized":"2024-04-25T10:16:05.110609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.936275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112131+0000","last_change":"2024-04-25T10:16:03.219261+0000","last_active":"2024-04-25T10:16:05.112131+0000","last_peered":"2024-04-25T10:16:05.112131+0000","last_clean":"2024-04-25T10:16:05.112131+0000","last_became_active":"2024-04-25T10:16:03.218985+0000","last_became_peered":"2024-04-25T10:16:03.218985+0000","last_unstale":"2024-04-25T10:16:05.112131+0000","last_undegraded":"2024-04-25T10:16:05.112131+0000","last_fullsized":"2024-04-25T10:16:05.112131+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:17.232248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.109815+0000","last_change":"2024-04-25T10:16:03.208169+0000","last_active":"2024-04-25T10:16:05.109815+0000","last_peered":"2024-04-25T10:16:05.109815+0000","last_clean":"2024-04-25T10:16:05.109815+0000","last_became_active":"2024-04-25T10:16:03.207918+0000","last_became_peered":"2024-04-25T10:16:03.207918+0000","last_unstale":"2024-04-25T10:16:05.109815+0000","last_undegraded":"2024-04-25T10:16:05.109815+0000","last_fullsized":"2024-04-25T10:16:05.109815+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:20.948558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099633+0000","last_change":"2024-04-25T10:16:03.233507+0000","last_active":"2024-04-25T10:16:05.099633+0000","last_peered":"2024-04-25T10:16:05.099633+0000","last_clean":"2024-04-25T10:16:05.099633+0000","last_became_active":"2024-04-25T10:16:03.229273+0000","last_became_peered":"2024-04-25T10:16:03.229273+0000","last_unstale":"2024-04-25T10:16:05.099633+0000","last_undegraded":"2024-04-25T10:16:05.099633+0000","last_fullsized":"2024-04-25T10:16:05.099633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.035601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099657+0000","last_change":"2024-04-25T10:16:03.223259+0000","last_active":"2024-04-25T10:16:05.099657+0000","last_peered":"2024-04-25T10:16:05.099657+0000","last_clean":"2024-04-25T10:16:05.099657+0000","last_became_active":"2024-04-25T10:16:03.222866+0000","last_became_peered":"2024-04-25T10:16:03.222866+0000","last_unstale":"2024-04-25T10:16:05.099657+0000","last_undegraded":"2024-04-25T10:16:05.099657+0000","last_fullsized":"2024-04-25T10:16:05.099657+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:26.464110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099458+0000","last_change":"2024-04-25T10:16:02.218713+0000","last_active":"2024-04-25T10:16:05.099458+0000","last_peered":"2024-04-25T10:16:05.099458+0000","last_clean":"2024-04-25T10:16:05.099458+0000","last_became_active":"2024-04-25T10:16:02.218399+0000","last_became_peered":"2024-04-25T10:16:02.218399+0000","last_unstale":"2024-04-25T10:16:05.099458+0000","last_undegraded":"2024-04-25T10:16:05.099458+0000","last_fullsized":"2024-04-25T10:16:05.099458+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:39.549154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218422+0000","last_change":"2024-04-25T10:16:03.214792+0000","last_active":"2024-04-25T10:16:04.218422+0000","last_peered":"2024-04-25T10:16:04.218422+0000","last_clean":"2024-04-25T10:16:04.218422+0000","last_became_active":"2024-04-25T10:16:03.214327+0000","last_became_peered":"2024-04-25T10:16:03.214327+0000","last_unstale":"2024-04-25T10:16:04.218422+0000","last_undegraded":"2024-04-25T10:16:04.218422+0000","last_fullsized":"2024-04-25T10:16:04.218422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:45.029732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110947+0000","last_change":"2024-04-25T10:16:02.214084+0000","last_active":"2024-04-25T10:16:05.110947+0000","last_peered":"2024-04-25T10:16:05.110947+0000","last_clean":"2024-04-25T10:16:05.110947+0000","last_became_active":"2024-04-25T10:16:02.213799+0000","last_became_peered":"2024-04-25T10:16:02.213799+0000","last_unstale":"2024-04-25T10:16:05.110947+0000","last_undegraded":"2024-04-25T10:16:05.110947+0000","last_fullsized":"2024-04-25T10:16:05.110947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:25.456644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099290+0000","last_change":"2024-04-25T10:16:03.237727+0000","last_active":"2024-04-25T10:16:05.099290+0000","last_peered":"2024-04-25T10:16:05.099290+0000","last_clean":"2024-04-25T10:16:05.099290+0000","last_became_active":"2024-04-25T10:16:03.237242+0000","last_became_peered":"2024-04-25T10:16:03.237242+0000","last_unstale":"2024-04-25T10:16:05.099290+0000","last_undegraded":"2024-04-25T10:16:05.099290+0000","last_fullsized":"2024-04-25T10:16:05.099290+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:50.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099503+0000","last_change":"2024-04-25T10:16:02.220200+0000","last_active":"2024-04-25T10:16:05.099503+0000","last_peered":"2024-04-25T10:16:05.099503+0000","last_clean":"2024-04-25T10:16:05.099503+0000","last_became_active":"2024-04-25T10:16:02.219926+0000","last_became_peered":"2024-04-25T10:16:02.219926+0000","last_unstale":"2024-04-25T10:16:05.099503+0000","last_undegraded":"2024-04-25T10:16:05.099503+0000","last_fullsized":"2024-04-25T10:16:05.099503+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:29.393094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099325+0000","last_change":"2024-04-25T10:16:03.238205+0000","last_active":"2024-04-25T10:16:05.099325+0000","last_peered":"2024-04-25T10:16:05.099325+0000","last_clean":"2024-04-25T10:16:05.099325+0000","last_became_active":"2024-04-25T10:16:03.237802+0000","last_became_peered":"2024-04-25T10:16:03.237802+0000","last_unstale":"2024-04-25T10:16:05.099325+0000","last_undegraded":"2024-04-25T10:16:05.099325+0000","last_fullsized":"2024-04-25T10:16:05.099325+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:17.232248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219213+0000","last_change":"2024-04-25T10:16:02.211448+0000","last_active":"2024-04-25T10:16:04.219213+0000","last_peered":"2024-04-25T10:16:04.219213+0000","last_clean":"2024-04-25T10:16:04.219213+0000","last_became_active":"2024-04-25T10:16:02.211164+0000","last_became_peered":"2024-04-25T10:16:02.211164+0000","last_unstale":"2024-04-25T10:16:04.219213+0000","last_undegraded":"2024-04-25T10:16:04.219213+0000","last_fullsized":"2024-04-25T10:16:04.219213+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.601764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110179+0000","last_change":"2024-04-25T10:16:03.208197+0000","last_active":"2024-04-25T10:16:05.110179+0000","last_peered":"2024-04-25T10:16:05.110179+0000","last_clean":"2024-04-25T10:16:05.110179+0000","last_became_active":"2024-04-25T10:16:03.207905+0000","last_became_peered":"2024-04-25T10:16:03.207905+0000","last_unstale":"2024-04-25T10:16:05.110179+0000","last_undegraded":"2024-04-25T10:16:05.110179+0000","last_fullsized":"2024-04-25T10:16:05.110179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:10.563085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218648+0000","last_change":"2024-04-25T10:16:02.212831+0000","last_active":"2024-04-25T10:16:04.218648+0000","last_peered":"2024-04-25T10:16:04.218648+0000","last_clean":"2024-04-25T10:16:04.218648+0000","last_became_active":"2024-04-25T10:16:02.212574+0000","last_became_peered":"2024-04-25T10:16:02.212574+0000","last_unstale":"2024-04-25T10:16:04.218648+0000","last_undegraded":"2024-04-25T10:16:04.218648+0000","last_fullsized":"2024-04-25T10:16:04.218648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:51.172043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110163+0000","last_change":"2024-04-25T10:16:03.225025+0000","last_active":"2024-04-25T10:16:05.110163+0000","last_peered":"2024-04-25T10:16:05.110163+0000","last_clean":"2024-04-25T10:16:05.110163+0000","last_became_active":"2024-04-25T10:16:03.224421+0000","last_became_peered":"2024-04-25T10:16:03.224421+0000","last_unstale":"2024-04-25T10:16:05.110163+0000","last_undegraded":"2024-04-25T10:16:05.110163+0000","last_fullsized":"2024-04-25T10:16:05.110163+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:11.037843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099072+0000","last_change":"2024-04-25T10:16:02.213326+0000","last_active":"2024-04-25T10:16:05.099072+0000","last_peered":"2024-04-25T10:16:05.099072+0000","last_clean":"2024-04-25T10:16:05.099072+0000","last_became_active":"2024-04-25T10:16:02.213019+0000","last_became_peered":"2024-04-25T10:16:02.213019+0000","last_unstale":"2024-04-25T10:16:05.099072+0000","last_undegraded":"2024-04-25T10:16:05.099072+0000","last_fullsized":"2024-04-25T10:16:05.099072+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:46.109348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111735+0000","last_change":"2024-04-25T10:16:03.217188+0000","last_active":"2024-04-25T10:16:05.111735+0000","last_peered":"2024-04-25T10:16:05.111735+0000","last_clean":"2024-04-25T10:16:05.111735+0000","last_became_active":"2024-04-25T10:16:03.216911+0000","last_became_peered":"2024-04-25T10:16:03.216911+0000","last_unstale":"2024-04-25T10:16:05.111735+0000","last_undegraded":"2024-04-25T10:16:05.111735+0000","last_fullsized":"2024-04-25T10:16:05.111735+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.975892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219191+0000","last_change":"2024-04-25T10:16:02.208298+0000","last_active":"2024-04-25T10:16:04.219191+0000","last_peered":"2024-04-25T10:16:04.219191+0000","last_clean":"2024-04-25T10:16:04.219191+0000","last_became_active":"2024-04-25T10:16:02.207764+0000","last_became_peered":"2024-04-25T10:16:02.207764+0000","last_unstale":"2024-04-25T10:16:04.219191+0000","last_undegraded":"2024-04-25T10:16:04.219191+0000","last_fullsized":"2024-04-25T10:16:04.219191+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:23.211303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110053+0000","last_change":"2024-04-25T10:16:03.214179+0000","last_active":"2024-04-25T10:16:05.110053+0000","last_peered":"2024-04-25T10:16:05.110053+0000","last_clean":"2024-04-25T10:16:05.110053+0000","last_became_active":"2024-04-25T10:16:03.213237+0000","last_became_peered":"2024-04-25T10:16:03.213237+0000","last_unstale":"2024-04-25T10:16:05.110053+0000","last_undegraded":"2024-04-25T10:16:05.110053+0000","last_fullsized":"2024-04-25T10:16:05.110053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:55.174050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219205+0000","last_change":"2024-04-25T10:16:02.216008+0000","last_active":"2024-04-25T10:16:04.219205+0000","last_peered":"2024-04-25T10:16:04.219205+0000","last_clean":"2024-04-25T10:16:04.219205+0000","last_became_active":"2024-04-25T10:16:02.215473+0000","last_became_peered":"2024-04-25T10:16:02.215473+0000","last_unstale":"2024-04-25T10:16:04.219205+0000","last_undegraded":"2024-04-25T10:16:04.219205+0000","last_fullsized":"2024-04-25T10:16:04.219205+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:02.872977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218965+0000","last_change":"2024-04-25T10:16:03.214412+0000","last_active":"2024-04-25T10:16:04.218965+0000","last_peered":"2024-04-25T10:16:04.218965+0000","last_clean":"2024-04-25T10:16:04.218965+0000","last_became_active":"2024-04-25T10:16:03.214127+0000","last_became_peered":"2024-04-25T10:16:03.214127+0000","last_unstale":"2024-04-25T10:16:04.218965+0000","last_undegraded":"2024-04-25T10:16:04.218965+0000","last_fullsized":"2024-04-25T10:16:04.218965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:08.108219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112856+0000","last_change":"2024-04-25T10:16:02.214697+0000","last_active":"2024-04-25T10:16:05.112856+0000","last_peered":"2024-04-25T10:16:05.112856+0000","last_clean":"2024-04-25T10:16:05.112856+0000","last_became_active":"2024-04-25T10:16:02.214228+0000","last_became_peered":"2024-04-25T10:16:02.214228+0000","last_unstale":"2024-04-25T10:16:05.112856+0000","last_undegraded":"2024-04-25T10:16:05.112856+0000","last_fullsized":"2024-04-25T10:16:05.112856+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:17.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112875+0000","last_change":"2024-04-25T10:16:03.222574+0000","last_active":"2024-04-25T10:16:05.112875+0000","last_peered":"2024-04-25T10:16:05.112875+0000","last_clean":"2024-04-25T10:16:05.112875+0000","last_became_active":"2024-04-25T10:16:03.222375+0000","last_became_peered":"2024-04-25T10:16:03.222375+0000","last_unstale":"2024-04-25T10:16:05.112875+0000","last_undegraded":"2024-04-25T10:16:05.112875+0000","last_fullsized":"2024-04-25T10:16:05.112875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:57.698290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111037+0000","last_change":"2024-04-25T10:16:02.210766+0000","last_active":"2024-04-25T10:16:05.111037+0000","last_peered":"2024-04-25T10:16:05.111037+0000","last_clean":"2024-04-25T10:16:05.111037+0000","last_became_active":"2024-04-25T10:16:02.210506+0000","last_became_peered":"2024-04-25T10:16:02.210506+0000","last_unstale":"2024-04-25T10:16:05.111037+0000","last_undegraded":"2024-04-25T10:16:05.111037+0000","last_fullsized":"2024-04-25T10:16:05.111037+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:46.709344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217802+0000","last_change":"2024-04-25T10:16:03.212578+0000","last_active":"2024-04-25T10:16:04.217802+0000","last_peered":"2024-04-25T10:16:04.217802+0000","last_clean":"2024-04-25T10:16:04.217802+0000","last_became_active":"2024-04-25T10:16:03.212035+0000","last_became_peered":"2024-04-25T10:16:03.212035+0000","last_unstale":"2024-04-25T10:16:04.217802+0000","last_undegraded":"2024-04-25T10:16:04.217802+0000","last_fullsized":"2024-04-25T10:16:04.217802+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:10.845809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112694+0000","last_change":"2024-04-25T10:16:02.209148+0000","last_active":"2024-04-25T10:16:05.112694+0000","last_peered":"2024-04-25T10:16:05.112694+0000","last_clean":"2024-04-25T10:16:05.112694+0000","last_became_active":"2024-04-25T10:16:02.208808+0000","last_became_peered":"2024-04-25T10:16:02.208808+0000","last_unstale":"2024-04-25T10:16:05.112694+0000","last_undegraded":"2024-04-25T10:16:05.112694+0000","last_fullsized":"2024-04-25T10:16:05.112694+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:29.393094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112597+0000","last_change":"2024-04-25T10:16:03.221479+0000","last_active":"2024-04-25T10:16:05.112597+0000","last_peered":"2024-04-25T10:16:05.112597+0000","last_clean":"2024-04-25T10:16:05.112597+0000","last_became_active":"2024-04-25T10:16:03.221191+0000","last_became_peered":"2024-04-25T10:16:03.221191+0000","last_unstale":"2024-04-25T10:16:05.112597+0000","last_undegraded":"2024-04-25T10:16:05.112597+0000","last_fullsized":"2024-04-25T10:16:05.112597+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:47.716811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218930+0000","last_change":"2024-04-25T10:16:02.212991+0000","last_active":"2024-04-25T10:16:04.218930+0000","last_peered":"2024-04-25T10:16:04.218930+0000","last_clean":"2024-04-25T10:16:04.218930+0000","last_became_active":"2024-04-25T10:16:02.212705+0000","last_became_peered":"2024-04-25T10:16:02.212705+0000","last_unstale":"2024-04-25T10:16:04.218930+0000","last_undegraded":"2024-04-25T10:16:04.218930+0000","last_fullsized":"2024-04-25T10:16:04.218930+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:44.022265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219303+0000","last_change":"2024-04-25T10:16:03.222814+0000","last_active":"2024-04-25T10:16:04.219303+0000","last_peered":"2024-04-25T10:16:04.219303+0000","last_clean":"2024-04-25T10:16:04.219303+0000","last_became_active":"2024-04-25T10:16:03.221689+0000","last_became_peered":"2024-04-25T10:16:03.221689+0000","last_unstale":"2024-04-25T10:16:04.219303+0000","last_undegraded":"2024-04-25T10:16:04.219303+0000","last_fullsized":"2024-04-25T10:16:04.219303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.241014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110237+0000","last_change":"2024-04-25T10:16:02.211496+0000","last_active":"2024-04-25T10:16:05.110237+0000","last_peered":"2024-04-25T10:16:05.110237+0000","last_clean":"2024-04-25T10:16:05.110237+0000","last_became_active":"2024-04-25T10:16:02.211239+0000","last_became_peered":"2024-04-25T10:16:02.211239+0000","last_unstale":"2024-04-25T10:16:05.110237+0000","last_undegraded":"2024-04-25T10:16:05.110237+0000","last_fullsized":"2024-04-25T10:16:05.110237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.971835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112427+0000","last_change":"2024-04-25T10:16:03.218556+0000","last_active":"2024-04-25T10:16:05.112427+0000","last_peered":"2024-04-25T10:16:05.112427+0000","last_clean":"2024-04-25T10:16:05.112427+0000","last_became_active":"2024-04-25T10:16:03.218197+0000","last_became_peered":"2024-04-25T10:16:03.218197+0000","last_unstale":"2024-04-25T10:16:05.112427+0000","last_undegraded":"2024-04-25T10:16:05.112427+0000","last_fullsized":"2024-04-25T10:16:05.112427+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:25.592075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111813+0000","last_change":"2024-04-25T10:16:02.214051+0000","last_active":"2024-04-25T10:16:05.111813+0000","last_peered":"2024-04-25T10:16:05.111813+0000","last_clean":"2024-04-25T10:16:05.111813+0000","last_became_active":"2024-04-25T10:16:02.213461+0000","last_became_peered":"2024-04-25T10:16:02.213461+0000","last_unstale":"2024-04-25T10:16:05.111813+0000","last_undegraded":"2024-04-25T10:16:05.111813+0000","last_fullsized":"2024-04-25T10:16:05.111813+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.425480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218473+0000","last_change":"2024-04-25T10:16:03.215030+0000","last_active":"2024-04-25T10:16:04.218473+0000","last_peered":"2024-04-25T10:16:04.218473+0000","last_clean":"2024-04-25T10:16:04.218473+0000","last_became_active":"2024-04-25T10:16:03.214773+0000","last_became_peered":"2024-04-25T10:16:03.214773+0000","last_unstale":"2024-04-25T10:16:04.218473+0000","last_undegraded":"2024-04-25T10:16:04.218473+0000","last_fullsized":"2024-04-25T10:16:04.218473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:32.957206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219333+0000","last_change":"2024-04-25T10:16:02.212260+0000","last_active":"2024-04-25T10:16:04.219333+0000","last_peered":"2024-04-25T10:16:04.219333+0000","last_clean":"2024-04-25T10:16:04.219333+0000","last_became_active":"2024-04-25T10:16:02.210462+0000","last_became_peered":"2024-04-25T10:16:02.210462+0000","last_unstale":"2024-04-25T10:16:04.219333+0000","last_undegraded":"2024-04-25T10:16:04.219333+0000","last_fullsized":"2024-04-25T10:16:04.219333+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:05.615221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112494+0000","last_change":"2024-04-25T10:16:03.224990+0000","last_active":"2024-04-25T10:16:05.112494+0000","last_peered":"2024-04-25T10:16:05.112494+0000","last_clean":"2024-04-25T10:16:05.112494+0000","last_became_active":"2024-04-25T10:16:03.224772+0000","last_became_peered":"2024-04-25T10:16:03.224772+0000","last_unstale":"2024-04-25T10:16:05.112494+0000","last_undegraded":"2024-04-25T10:16:05.112494+0000","last_fullsized":"2024-04-25T10:16:05.112494+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.035601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":22,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111953+0000","last_change":"2024-04-25T10:16:02.211589+0000","last_active":"2024-04-25T10:16:05.111953+0000","last_peered":"2024-04-25T10:16:05.111953+0000","last_clean":"2024-04-25T10:16:05.111953+0000","last_became_active":"2024-04-25T10:16:02.211326+0000","last_became_peered":"2024-04-25T10:16:02.211326+0000","last_unstale":"2024-04-25T10:16:05.111953+0000","last_undegraded":"2024-04-25T10:16:05.111953+0000","last_fullsized":"2024-04-25T10:16:05.111953+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:05.615221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112861+0000","last_change":"2024-04-25T10:16:03.221514+0000","last_active":"2024-04-25T10:16:05.112861+0000","last_peered":"2024-04-25T10:16:05.112861+0000","last_clean":"2024-04-25T10:16:05.112861+0000","last_became_active":"2024-04-25T10:16:03.221191+0000","last_became_peered":"2024-04-25T10:16:03.221191+0000","last_unstale":"2024-04-25T10:16:05.112861+0000","last_undegraded":"2024-04-25T10:16:05.112861+0000","last_fullsized":"2024-04-25T10:16:05.112861+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:23.623322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111092+0000","last_change":"2024-04-25T10:16:02.210822+0000","last_active":"2024-04-25T10:16:05.111092+0000","last_peered":"2024-04-25T10:16:05.111092+0000","last_clean":"2024-04-25T10:16:05.111092+0000","last_became_active":"2024-04-25T10:16:02.210601+0000","last_became_peered":"2024-04-25T10:16:02.210601+0000","last_unstale":"2024-04-25T10:16:05.111092+0000","last_undegraded":"2024-04-25T10:16:05.111092+0000","last_fullsized":"2024-04-25T10:16:05.111092+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:24.584609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219214+0000","last_change":"2024-04-25T10:16:03.222519+0000","last_active":"2024-04-25T10:16:04.219214+0000","last_peered":"2024-04-25T10:16:04.219214+0000","last_clean":"2024-04-25T10:16:04.219214+0000","last_became_active":"2024-04-25T10:16:03.222223+0000","last_became_peered":"2024-04-25T10:16:03.222223+0000","last_unstale":"2024-04-25T10:16:04.219214+0000","last_undegraded":"2024-04-25T10:16:04.219214+0000","last_fullsized":"2024-04-25T10:16:04.219214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:50.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099601+0000","last_change":"2024-04-25T10:16:02.215451+0000","last_active":"2024-04-25T10:16:05.099601+0000","last_peered":"2024-04-25T10:16:05.099601+0000","last_clean":"2024-04-25T10:16:05.099601+0000","last_became_active":"2024-04-25T10:16:02.215077+0000","last_became_peered":"2024-04-25T10:16:02.215077+0000","last_unstale":"2024-04-25T10:16:05.099601+0000","last_undegraded":"2024-04-25T10:16:05.099601+0000","last_fullsized":"2024-04-25T10:16:05.099601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:17.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110795+0000","last_change":"2024-04-25T10:16:03.213272+0000","last_active":"2024-04-25T10:16:05.110795+0000","last_peered":"2024-04-25T10:16:05.110795+0000","last_clean":"2024-04-25T10:16:05.110795+0000","last_became_active":"2024-04-25T10:16:03.212960+0000","last_became_peered":"2024-04-25T10:16:03.212960+0000","last_unstale":"2024-04-25T10:16:05.110795+0000","last_undegraded":"2024-04-25T10:16:05.110795+0000","last_fullsized":"2024-04-25T10:16:05.110795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:44.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218979+0000","last_change":"2024-04-25T10:16:02.210609+0000","last_active":"2024-04-25T10:16:04.218979+0000","last_peered":"2024-04-25T10:16:04.218979+0000","last_clean":"2024-04-25T10:16:04.218979+0000","last_became_active":"2024-04-25T10:16:02.210255+0000","last_became_peered":"2024-04-25T10:16:02.210255+0000","last_unstale":"2024-04-25T10:16:04.218979+0000","last_undegraded":"2024-04-25T10:16:04.218979+0000","last_fullsized":"2024-04-25T10:16:04.218979+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:17.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112681+0000","last_change":"2024-04-25T10:16:03.223120+0000","last_active":"2024-04-25T10:16:05.112681+0000","last_peered":"2024-04-25T10:16:05.112681+0000","last_clean":"2024-04-25T10:16:05.112681+0000","last_became_active":"2024-04-25T10:16:03.222910+0000","last_became_peered":"2024-04-25T10:16:03.222910+0000","last_unstale":"2024-04-25T10:16:05.112681+0000","last_undegraded":"2024-04-25T10:16:05.112681+0000","last_fullsized":"2024-04-25T10:16:05.112681+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:07.700085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099284+0000","last_change":"2024-04-25T10:16:02.220183+0000","last_active":"2024-04-25T10:16:05.099284+0000","last_peered":"2024-04-25T10:16:05.099284+0000","last_clean":"2024-04-25T10:16:05.099284+0000","last_became_active":"2024-04-25T10:16:02.219686+0000","last_became_peered":"2024-04-25T10:16:02.219686+0000","last_unstale":"2024-04-25T10:16:05.099284+0000","last_undegraded":"2024-04-25T10:16:05.099284+0000","last_fullsized":"2024-04-25T10:16:05.099284+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.949740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219223+0000","last_change":"2024-04-25T10:16:03.221708+0000","last_active":"2024-04-25T10:16:04.219223+0000","last_peered":"2024-04-25T10:16:04.219223+0000","last_clean":"2024-04-25T10:16:04.219223+0000","last_became_active":"2024-04-25T10:16:03.221301+0000","last_became_peered":"2024-04-25T10:16:03.221301+0000","last_unstale":"2024-04-25T10:16:04.219223+0000","last_undegraded":"2024-04-25T10:16:04.219223+0000","last_fullsized":"2024-04-25T10:16:04.219223+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:17.232248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110667+0000","last_change":"2024-04-25T10:16:02.208859+0000","last_active":"2024-04-25T10:16:05.110667+0000","last_peered":"2024-04-25T10:16:05.110667+0000","last_clean":"2024-04-25T10:16:05.110667+0000","last_became_active":"2024-04-25T10:16:02.208611+0000","last_became_peered":"2024-04-25T10:16:02.208611+0000","last_unstale":"2024-04-25T10:16:05.110667+0000","last_undegraded":"2024-04-25T10:16:05.110667+0000","last_fullsized":"2024-04-25T10:16:05.110667+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:58.028135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099677+0000","last_change":"2024-04-25T10:16:03.229087+0000","last_active":"2024-04-25T10:16:05.099677+0000","last_peered":"2024-04-25T10:16:05.099677+0000","last_clean":"2024-04-25T10:16:05.099677+0000","last_became_active":"2024-04-25T10:16:03.226116+0000","last_became_peered":"2024-04-25T10:16:03.226116+0000","last_unstale":"2024-04-25T10:16:05.099677+0000","last_undegraded":"2024-04-25T10:16:05.099677+0000","last_fullsized":"2024-04-25T10:16:05.099677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:47.716811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217713+0000","last_change":"2024-04-25T10:16:02.217945+0000","last_active":"2024-04-25T10:16:04.217713+0000","last_peered":"2024-04-25T10:16:04.217713+0000","last_clean":"2024-04-25T10:16:04.217713+0000","last_became_active":"2024-04-25T10:16:02.217603+0000","last_became_peered":"2024-04-25T10:16:02.217603+0000","last_unstale":"2024-04-25T10:16:04.217713+0000","last_undegraded":"2024-04-25T10:16:04.217713+0000","last_fullsized":"2024-04-25T10:16:04.217713+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:38.125275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099582+0000","last_change":"2024-04-25T10:16:03.228414+0000","last_active":"2024-04-25T10:16:05.099582+0000","last_peered":"2024-04-25T10:16:05.099582+0000","last_clean":"2024-04-25T10:16:05.099582+0000","last_became_active":"2024-04-25T10:16:03.228138+0000","last_became_peered":"2024-04-25T10:16:03.228138+0000","last_unstale":"2024-04-25T10:16:05.099582+0000","last_undegraded":"2024-04-25T10:16:05.099582+0000","last_fullsized":"2024-04-25T10:16:05.099582+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:25.592075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218297+0000","last_change":"2024-04-25T10:16:02.211646+0000","last_active":"2024-04-25T10:16:04.218297+0000","last_peered":"2024-04-25T10:16:04.218297+0000","last_clean":"2024-04-25T10:16:04.218297+0000","last_became_active":"2024-04-25T10:16:02.210617+0000","last_became_peered":"2024-04-25T10:16:02.210617+0000","last_unstale":"2024-04-25T10:16:04.218297+0000","last_undegraded":"2024-04-25T10:16:04.218297+0000","last_fullsized":"2024-04-25T10:16:04.218297+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:29.393094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219555+0000","last_change":"2024-04-25T10:16:03.209355+0000","last_active":"2024-04-25T10:16:04.219555+0000","last_peered":"2024-04-25T10:16:04.219555+0000","last_clean":"2024-04-25T10:16:04.219555+0000","last_became_active":"2024-04-25T10:16:03.209063+0000","last_became_peered":"2024-04-25T10:16:03.209063+0000","last_unstale":"2024-04-25T10:16:04.219555+0000","last_undegraded":"2024-04-25T10:16:04.219555+0000","last_fullsized":"2024-04-25T10:16:04.219555+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:08.979302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218345+0000","last_change":"2024-04-25T10:16:02.213084+0000","last_active":"2024-04-25T10:16:04.218345+0000","last_peered":"2024-04-25T10:16:04.218345+0000","last_clean":"2024-04-25T10:16:04.218345+0000","last_became_active":"2024-04-25T10:16:02.212783+0000","last_became_peered":"2024-04-25T10:16:02.212783+0000","last_unstale":"2024-04-25T10:16:04.218345+0000","last_undegraded":"2024-04-25T10:16:04.218345+0000","last_fullsized":"2024-04-25T10:16:04.218345+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:04.001291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219719+0000","last_change":"2024-04-25T10:16:03.216940+0000","last_active":"2024-04-25T10:16:04.219719+0000","last_peered":"2024-04-25T10:16:04.219719+0000","last_clean":"2024-04-25T10:16:04.219719+0000","last_became_active":"2024-04-25T10:16:03.216076+0000","last_became_peered":"2024-04-25T10:16:03.216076+0000","last_unstale":"2024-04-25T10:16:04.219719+0000","last_undegraded":"2024-04-25T10:16:04.219719+0000","last_fullsized":"2024-04-25T10:16:04.219719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.035601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099725+0000","last_change":"2024-04-25T10:16:02.212115+0000","last_active":"2024-04-25T10:16:05.099725+0000","last_peered":"2024-04-25T10:16:05.099725+0000","last_clean":"2024-04-25T10:16:05.099725+0000","last_became_active":"2024-04-25T10:16:02.211825+0000","last_became_peered":"2024-04-25T10:16:02.211825+0000","last_unstale":"2024-04-25T10:16:05.099725+0000","last_undegraded":"2024-04-25T10:16:05.099725+0000","last_fullsized":"2024-04-25T10:16:05.099725+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.601764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217896+0000","last_change":"2024-04-25T10:16:03.212335+0000","last_active":"2024-04-25T10:16:04.217896+0000","last_peered":"2024-04-25T10:16:04.217896+0000","last_clean":"2024-04-25T10:16:04.217896+0000","last_became_active":"2024-04-25T10:16:03.211685+0000","last_became_peered":"2024-04-25T10:16:03.211685+0000","last_unstale":"2024-04-25T10:16:04.217896+0000","last_undegraded":"2024-04-25T10:16:04.217896+0000","last_fullsized":"2024-04-25T10:16:04.217896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:03.880444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111761+0000","last_change":"2024-04-25T10:16:02.213981+0000","last_active":"2024-04-25T10:16:05.111761+0000","last_peered":"2024-04-25T10:16:05.111761+0000","last_clean":"2024-04-25T10:16:05.111761+0000","last_became_active":"2024-04-25T10:16:02.213716+0000","last_became_peered":"2024-04-25T10:16:02.213716+0000","last_unstale":"2024-04-25T10:16:05.111761+0000","last_undegraded":"2024-04-25T10:16:05.111761+0000","last_fullsized":"2024-04-25T10:16:05.111761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:31.444807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099618+0000","last_change":"2024-04-25T10:16:03.238129+0000","last_active":"2024-04-25T10:16:05.099618+0000","last_peered":"2024-04-25T10:16:05.099618+0000","last_clean":"2024-04-25T10:16:05.099618+0000","last_became_active":"2024-04-25T10:16:03.237730+0000","last_became_peered":"2024-04-25T10:16:03.237730+0000","last_unstale":"2024-04-25T10:16:05.099618+0000","last_undegraded":"2024-04-25T10:16:05.099618+0000","last_fullsized":"2024-04-25T10:16:05.099618+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:08.979302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219240+0000","last_change":"2024-04-25T10:16:02.210677+0000","last_active":"2024-04-25T10:16:04.219240+0000","last_peered":"2024-04-25T10:16:04.219240+0000","last_clean":"2024-04-25T10:16:04.219240+0000","last_became_active":"2024-04-25T10:16:02.209577+0000","last_became_peered":"2024-04-25T10:16:02.209577+0000","last_unstale":"2024-04-25T10:16:04.219240+0000","last_undegraded":"2024-04-25T10:16:04.219240+0000","last_fullsized":"2024-04-25T10:16:04.219240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.949740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110366+0000","last_change":"2024-04-25T10:16:03.220903+0000","last_active":"2024-04-25T10:16:05.110366+0000","last_peered":"2024-04-25T10:16:05.110366+0000","last_clean":"2024-04-25T10:16:05.110366+0000","last_became_active":"2024-04-25T10:16:03.220460+0000","last_became_peered":"2024-04-25T10:16:03.220460+0000","last_unstale":"2024-04-25T10:16:05.110366+0000","last_undegraded":"2024-04-25T10:16:05.110366+0000","last_fullsized":"2024-04-25T10:16:05.110366+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:20.939254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217546+0000","last_change":"2024-04-25T10:16:02.213146+0000","last_active":"2024-04-25T10:16:04.217546+0000","last_peered":"2024-04-25T10:16:04.217546+0000","last_clean":"2024-04-25T10:16:04.217546+0000","last_became_active":"2024-04-25T10:16:02.212835+0000","last_became_peered":"2024-04-25T10:16:02.212835+0000","last_unstale":"2024-04-25T10:16:04.217546+0000","last_undegraded":"2024-04-25T10:16:04.217546+0000","last_fullsized":"2024-04-25T10:16:04.217546+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:31.444807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112226+0000","last_change":"2024-04-25T10:16:03.217188+0000","last_active":"2024-04-25T10:16:05.112226+0000","last_peered":"2024-04-25T10:16:05.112226+0000","last_clean":"2024-04-25T10:16:05.112226+0000","last_became_active":"2024-04-25T10:16:03.216919+0000","last_became_peered":"2024-04-25T10:16:03.216919+0000","last_unstale":"2024-04-25T10:16:05.112226+0000","last_undegraded":"2024-04-25T10:16:05.112226+0000","last_fullsized":"2024-04-25T10:16:05.112226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.241014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218617+0000","last_change":"2024-04-25T10:16:02.215591+0000","last_active":"2024-04-25T10:16:04.218617+0000","last_peered":"2024-04-25T10:16:04.218617+0000","last_clean":"2024-04-25T10:16:04.218617+0000","last_became_active":"2024-04-25T10:16:02.215314+0000","last_became_peered":"2024-04-25T10:16:02.215314+0000","last_unstale":"2024-04-25T10:16:04.218617+0000","last_undegraded":"2024-04-25T10:16:04.218617+0000","last_fullsized":"2024-04-25T10:16:04.218617+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.080370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217546+0000","last_change":"2024-04-25T10:16:03.215126+0000","last_active":"2024-04-25T10:16:04.217546+0000","last_peered":"2024-04-25T10:16:04.217546+0000","last_clean":"2024-04-25T10:16:04.217546+0000","last_became_active":"2024-04-25T10:16:03.214854+0000","last_became_peered":"2024-04-25T10:16:03.214854+0000","last_unstale":"2024-04-25T10:16:04.217546+0000","last_undegraded":"2024-04-25T10:16:04.217546+0000","last_fullsized":"2024-04-25T10:16:04.217546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:40.918814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217542+0000","last_change":"2024-04-25T10:16:02.215889+0000","last_active":"2024-04-25T10:16:04.217542+0000","last_peered":"2024-04-25T10:16:04.217542+0000","last_clean":"2024-04-25T10:16:04.217542+0000","last_became_active":"2024-04-25T10:16:02.215576+0000","last_became_peered":"2024-04-25T10:16:02.215576+0000","last_unstale":"2024-04-25T10:16:04.217542+0000","last_undegraded":"2024-04-25T10:16:04.217542+0000","last_fullsized":"2024-04-25T10:16:04.217542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.425480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110234+0000","last_change":"2024-04-25T10:16:03.214468+0000","last_active":"2024-04-25T10:16:05.110234+0000","last_peered":"2024-04-25T10:16:05.110234+0000","last_clean":"2024-04-25T10:16:05.110234+0000","last_became_active":"2024-04-25T10:16:03.213627+0000","last_became_peered":"2024-04-25T10:16:03.213627+0000","last_unstale":"2024-04-25T10:16:05.110234+0000","last_undegraded":"2024-04-25T10:16:05.110234+0000","last_fullsized":"2024-04-25T10:16:05.110234+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:06.011341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110241+0000","last_change":"2024-04-25T10:16:02.209475+0000","last_active":"2024-04-25T10:16:05.110241+0000","last_peered":"2024-04-25T10:16:05.110241+0000","last_clean":"2024-04-25T10:16:05.110241+0000","last_became_active":"2024-04-25T10:16:02.209056+0000","last_became_peered":"2024-04-25T10:16:02.209056+0000","last_unstale":"2024-04-25T10:16:05.110241+0000","last_undegraded":"2024-04-25T10:16:05.110241+0000","last_fullsized":"2024-04-25T10:16:05.110241+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:58.233547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110172+0000","last_change":"2024-04-25T10:16:03.219221+0000","last_active":"2024-04-25T10:16:05.110172+0000","last_peered":"2024-04-25T10:16:05.110172+0000","last_clean":"2024-04-25T10:16:05.110172+0000","last_became_active":"2024-04-25T10:16:03.218975+0000","last_became_peered":"2024-04-25T10:16:03.218975+0000","last_unstale":"2024-04-25T10:16:05.110172+0000","last_undegraded":"2024-04-25T10:16:05.110172+0000","last_fullsized":"2024-04-25T10:16:05.110172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:42.563939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111659+0000","last_change":"2024-04-25T10:16:02.213664+0000","last_active":"2024-04-25T10:16:05.111659+0000","last_peered":"2024-04-25T10:16:05.111659+0000","last_clean":"2024-04-25T10:16:05.111659+0000","last_became_active":"2024-04-25T10:16:02.213276+0000","last_became_peered":"2024-04-25T10:16:02.213276+0000","last_unstale":"2024-04-25T10:16:05.111659+0000","last_undegraded":"2024-04-25T10:16:05.111659+0000","last_fullsized":"2024-04-25T10:16:05.111659+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:38.125275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111966+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:16:05.111966+0000","last_peered":"2024-04-25T10:16:05.111966+0000","last_clean":"2024-04-25T10:16:05.111966+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:16:05.111966+0000","last_undegraded":"2024-04-25T10:16:05.111966+0000","last_fullsized":"2024-04-25T10:16:05.111966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40756,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331084,"statfs":{"total":96636764160,"available":96595030016,"internally_reserved":0,"allocated":1286144,"data_stored":875877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":285349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":286001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":286961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58224,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313616,"statfs":{"total":96636764160,"available":96577142784,"internally_reserved":0,"allocated":692224,"data_stored":285405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":285405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40180,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331660,"statfs":{"total":96636764160,"available":96595619840,"internally_reserved":0,"allocated":688128,"data_stored":285439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1294336,"data_stored":877911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:07.289 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:07.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.455+0000 7feb56200640 1 Processor -- start 2024-04-25T10:16:07.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.455+0000 7feb56200640 1 -- start start 2024-04-25T10:16:07.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.455+0000 7feb56200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feb500d0040 0x7feb5015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.455+0000 7feb56200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb5015b740 0x7feb500b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.455+0000 7feb56200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb50071050 con 0x7feb501550b0 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.455+0000 7feb56200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5015d040 con 0x7feb500d0040 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.455+0000 7feb56200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb500712f0 con 0x7feb5015b740 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feb500d0040 0x7feb5015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feb500d0040 0x7feb5015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48758/0 (socket says 172.21.15.87:48758) 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 -- 172.21.15.87:0/1233376244 learned_addr learned my addr 172.21.15.87:0/1233376244 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4fe00640 1 --2- 172.21.15.87:0/1233376244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb5015b740 0x7feb500b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 -- 172.21.15.87:0/1233376244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb5015b740 msgr2=0x7feb500b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 --2- 172.21.15.87:0/1233376244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb5015b740 0x7feb500b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 -- 172.21.15.87:0/1233376244 >> v1:172.21.15.169:6789/0 conn(0x7feb501550b0 legacy=0x7feb500cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 -- 172.21.15.87:0/1233376244 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500715d0 con 0x7feb500d0040 2024-04-25T10:16:07.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4ea00640 1 --2- 172.21.15.87:0/1233376244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb5015b200 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009870 tx=0x7feb3c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9c5c954694fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4e000640 1 -- 172.21.15.87:0/1233376244 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0aa610 con 0x7feb500d0040 2024-04-25T10:16:07.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4e000640 1 -- 172.21.15.87:0/1233376244 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c0af070 con 0x7feb500d0040 2024-04-25T10:16:07.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.459+0000 7feb4e000640 1 -- 172.21.15.87:0/1233376244 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0aa9d0 con 0x7feb500d0040 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- 172.21.15.87:0/1233376244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 msgr2=0x7feb5015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 --2- 172.21.15.87:0/1233376244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb5015b200 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009870 tx=0x7feb3c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- 172.21.15.87:0/1233376244 shutdown_connections 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 --2- 172.21.15.87:0/1233376244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb5015b740 0x7feb500b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 --2- 172.21.15.87:0/1233376244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb5015b200 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- 172.21.15.87:0/1233376244 >> 172.21.15.87:0/1233376244 conn(0x7feb50066b70 msgr2=0x7feb50066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- 172.21.15.87:0/1233376244 shutdown_connections 2024-04-25T10:16:07.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- 172.21.15.87:0/1233376244 wait complete. 2024-04-25T10:16:07.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 Processor -- start 2024-04-25T10:16:07.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- start start 2024-04-25T10:16:07.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb501526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb501550b0 0x7feb50142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb5015b740 0x7feb500b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5015bcd0 con 0x7feb500d0040 2024-04-25T10:16:07.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5015bb50 con 0x7feb5015b740 2024-04-25T10:16:07.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb56200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5015c030 con 0x7feb501550b0 2024-04-25T10:16:07.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb501526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb501526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48760/0 (socket says 172.21.15.87:48760) 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 -- 172.21.15.87:0/982029904 learned_addr learned my addr 172.21.15.87:0/982029904 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4fe00640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb5015b740 0x7feb500b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4ea00640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb501550b0 0x7feb50142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 -- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb501550b0 msgr2=0x7feb50142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb501550b0 0x7feb50142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 -- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb5015b740 msgr2=0x7feb500b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb5015b740 0x7feb500b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 -- 172.21.15.87:0/982029904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500af7c0 con 0x7feb500d0040 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4fe00640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb5015b740 0x7feb500b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.463+0000 7feb4f400640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb501526e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7feb4000c7c0 tx=0x7feb4000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb47400640 1 -- 172.21.15.87:0/982029904 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40016020 con 0x7feb500d0040 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb47400640 1 -- 172.21.15.87:0/982029904 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4000aa50 con 0x7feb500d0040 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb47400640 1 -- 172.21.15.87:0/982029904 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40012420 con 0x7feb500d0040 2024-04-25T10:16:07.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb500afaa0 con 0x7feb500d0040 2024-04-25T10:16:07.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb500b0060 con 0x7feb500d0040 2024-04-25T10:16:07.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5004ef30 con 0x7feb500d0040 2024-04-25T10:16:07.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb47400640 1 -- 172.21.15.87:0/982029904 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb40019020 con 0x7feb500d0040 2024-04-25T10:16:07.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb47400640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feb30066f60 0x7feb30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb47400640 1 -- 172.21.15.87:0/982029904 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb400850d0 con 0x7feb500d0040 2024-04-25T10:16:07.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb4ea00640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feb30066f60 0x7feb30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.467+0000 7feb4ea00640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feb30066f60 0x7feb30069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0ae000 tx=0x7feb3c0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.471+0000 7feb47400640 1 -- 172.21.15.87:0/982029904 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb40020030 con 0x7feb500d0040 2024-04-25T10:16:07.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.623+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb50143010 con 0x7feb30066f60 2024-04-25T10:16:07.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.627+0000 7feb47400640 1 -- 172.21.15.87:0/982029904 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251955 (secure 0 0 0) 0x7feb50143010 con 0x7feb30066f60 2024-04-25T10:16:07.637 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feb30066f60 msgr2=0x7feb30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feb30066f60 0x7feb30069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0ae000 tx=0x7feb3c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 msgr2=0x7feb501526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb501526e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7feb4000c7c0 tx=0x7feb4000cc80 comp rx=0 tx=0).stop 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 shutdown_connections 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feb30066f60 0x7feb30069420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb5015b740 0x7feb500b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb501550b0 0x7feb50142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 --2- 172.21.15.87:0/982029904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb500d0040 0x7feb501526e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 >> 172.21.15.87:0/982029904 conn(0x7feb50066b70 msgr2=0x7feb500b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 shutdown_connections 2024-04-25T10:16:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.631+0000 7feb56200640 1 -- 172.21.15.87:0/982029904 wait complete. 2024-04-25T10:16:07.657 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-04-25T10:16:07.172314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324004,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":317299,"kb_avail":754650716,"statfs":{"total":773094113280,"available":772762333184,"internally_reserved":0,"allocated":6725632,"data_stored":3468348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324914737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.990587"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110527+0000","last_change":"2024-04-25T10:16:03.207648+0000","last_active":"2024-04-25T10:16:05.110527+0000","last_peered":"2024-04-25T10:16:05.110527+0000","last_clean":"2024-04-25T10:16:05.110527+0000","last_became_active":"2024-04-25T10:16:03.207366+0000","last_became_peered":"2024-04-25T10:16:03.207366+0000","last_unstale":"2024-04-25T10:16:05.110527+0000","last_undegraded":"2024-04-25T10:16:05.110527+0000","last_fullsized":"2024-04-25T10:16:05.110527+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:21.635103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110961+0000","last_change":"2024-04-25T10:16:03.220908+0000","last_active":"2024-04-25T10:16:05.110961+0000","last_peered":"2024-04-25T10:16:05.110961+0000","last_clean":"2024-04-25T10:16:05.110961+0000","last_became_active":"2024-04-25T10:16:03.220561+0000","last_became_peered":"2024-04-25T10:16:03.220561+0000","last_unstale":"2024-04-25T10:16:05.110961+0000","last_undegraded":"2024-04-25T10:16:05.110961+0000","last_fullsized":"2024-04-25T10:16:05.110961+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:52.987198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112487+0000","last_change":"2024-04-25T10:16:03.218434+0000","last_active":"2024-04-25T10:16:05.112487+0000","last_peered":"2024-04-25T10:16:05.112487+0000","last_clean":"2024-04-25T10:16:05.112487+0000","last_became_active":"2024-04-25T10:16:03.218154+0000","last_became_peered":"2024-04-25T10:16:03.218154+0000","last_unstale":"2024-04-25T10:16:05.112487+0000","last_undegraded":"2024-04-25T10:16:05.112487+0000","last_fullsized":"2024-04-25T10:16:05.112487+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:08.979302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218972+0000","last_change":"2024-04-25T10:16:03.221513+0000","last_active":"2024-04-25T10:16:04.218972+0000","last_peered":"2024-04-25T10:16:04.218972+0000","last_clean":"2024-04-25T10:16:04.218972+0000","last_became_active":"2024-04-25T10:16:03.221151+0000","last_became_peered":"2024-04-25T10:16:03.221151+0000","last_unstale":"2024-04-25T10:16:04.218972+0000","last_undegraded":"2024-04-25T10:16:04.218972+0000","last_fullsized":"2024-04-25T10:16:04.218972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:34.106239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110175+0000","last_change":"2024-04-25T10:16:03.216215+0000","last_active":"2024-04-25T10:16:05.110175+0000","last_peered":"2024-04-25T10:16:05.110175+0000","last_clean":"2024-04-25T10:16:05.110175+0000","last_became_active":"2024-04-25T10:16:03.215966+0000","last_became_peered":"2024-04-25T10:16:03.215966+0000","last_unstale":"2024-04-25T10:16:05.110175+0000","last_undegraded":"2024-04-25T10:16:05.110175+0000","last_fullsized":"2024-04-25T10:16:05.110175+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:41.370363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099432+0000","last_change":"2024-04-25T10:16:03.233166+0000","last_active":"2024-04-25T10:16:05.099432+0000","last_peered":"2024-04-25T10:16:05.099432+0000","last_clean":"2024-04-25T10:16:05.099432+0000","last_became_active":"2024-04-25T10:16:03.229236+0000","last_became_peered":"2024-04-25T10:16:03.229236+0000","last_unstale":"2024-04-25T10:16:05.099432+0000","last_undegraded":"2024-04-25T10:16:05.099432+0000","last_fullsized":"2024-04-25T10:16:05.099432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.241014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219133+0000","last_change":"2024-04-25T10:16:03.210282+0000","last_active":"2024-04-25T10:16:04.219133+0000","last_peered":"2024-04-25T10:16:04.219133+0000","last_clean":"2024-04-25T10:16:04.219133+0000","last_became_active":"2024-04-25T10:16:03.209656+0000","last_became_peered":"2024-04-25T10:16:03.209656+0000","last_unstale":"2024-04-25T10:16:04.219133+0000","last_undegraded":"2024-04-25T10:16:04.219133+0000","last_fullsized":"2024-04-25T10:16:04.219133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:16.346104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110422+0000","last_change":"2024-04-25T10:16:03.216789+0000","last_active":"2024-04-25T10:16:05.110422+0000","last_peered":"2024-04-25T10:16:05.110422+0000","last_clean":"2024-04-25T10:16:05.110422+0000","last_became_active":"2024-04-25T10:16:03.216523+0000","last_became_peered":"2024-04-25T10:16:03.216523+0000","last_unstale":"2024-04-25T10:16:05.110422+0000","last_undegraded":"2024-04-25T10:16:05.110422+0000","last_fullsized":"2024-04-25T10:16:05.110422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:56.223109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218743+0000","last_change":"2024-04-25T10:16:03.228972+0000","last_active":"2024-04-25T10:16:04.218743+0000","last_peered":"2024-04-25T10:16:04.218743+0000","last_clean":"2024-04-25T10:16:04.218743+0000","last_became_active":"2024-04-25T10:16:03.228486+0000","last_became_peered":"2024-04-25T10:16:03.228486+0000","last_unstale":"2024-04-25T10:16:04.218743+0000","last_undegraded":"2024-04-25T10:16:04.218743+0000","last_fullsized":"2024-04-25T10:16:04.218743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:41.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218275+0000","last_change":"2024-04-25T10:16:03.217474+0000","last_active":"2024-04-25T10:16:04.218275+0000","last_peered":"2024-04-25T10:16:04.218275+0000","last_clean":"2024-04-25T10:16:04.218275+0000","last_became_active":"2024-04-25T10:16:03.216092+0000","last_became_peered":"2024-04-25T10:16:03.216092+0000","last_unstale":"2024-04-25T10:16:04.218275+0000","last_undegraded":"2024-04-25T10:16:04.218275+0000","last_fullsized":"2024-04-25T10:16:04.218275+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:52.179510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218196+0000","last_change":"2024-04-25T10:16:03.207581+0000","last_active":"2024-04-25T10:16:04.218196+0000","last_peered":"2024-04-25T10:16:04.218196+0000","last_clean":"2024-04-25T10:16:04.218196+0000","last_became_active":"2024-04-25T10:16:03.206833+0000","last_became_peered":"2024-04-25T10:16:03.206833+0000","last_unstale":"2024-04-25T10:16:04.218196+0000","last_undegraded":"2024-04-25T10:16:04.218196+0000","last_fullsized":"2024-04-25T10:16:04.218196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:38.087836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219176+0000","last_change":"2024-04-25T10:16:03.222688+0000","last_active":"2024-04-25T10:16:04.219176+0000","last_peered":"2024-04-25T10:16:04.219176+0000","last_clean":"2024-04-25T10:16:04.219176+0000","last_became_active":"2024-04-25T10:16:03.222423+0000","last_became_peered":"2024-04-25T10:16:03.222423+0000","last_unstale":"2024-04-25T10:16:04.219176+0000","last_undegraded":"2024-04-25T10:16:04.219176+0000","last_fullsized":"2024-04-25T10:16:04.219176+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:42.808262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112092+0000","last_change":"2024-04-25T10:16:03.227329+0000","last_active":"2024-04-25T10:16:05.112092+0000","last_peered":"2024-04-25T10:16:05.112092+0000","last_clean":"2024-04-25T10:16:05.112092+0000","last_became_active":"2024-04-25T10:16:03.226979+0000","last_became_peered":"2024-04-25T10:16:03.226979+0000","last_unstale":"2024-04-25T10:16:05.112092+0000","last_undegraded":"2024-04-25T10:16:05.112092+0000","last_fullsized":"2024-04-25T10:16:05.112092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:03.053320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218874+0000","last_change":"2024-04-25T10:16:03.209188+0000","last_active":"2024-04-25T10:16:04.218874+0000","last_peered":"2024-04-25T10:16:04.218874+0000","last_clean":"2024-04-25T10:16:04.218874+0000","last_became_active":"2024-04-25T10:16:03.208911+0000","last_became_peered":"2024-04-25T10:16:03.208911+0000","last_unstale":"2024-04-25T10:16:04.218874+0000","last_undegraded":"2024-04-25T10:16:04.218874+0000","last_fullsized":"2024-04-25T10:16:04.218874+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:32.609230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218875+0000","last_change":"2024-04-25T10:16:03.213952+0000","last_active":"2024-04-25T10:16:04.218875+0000","last_peered":"2024-04-25T10:16:04.218875+0000","last_clean":"2024-04-25T10:16:04.218875+0000","last_became_active":"2024-04-25T10:16:03.213659+0000","last_became_peered":"2024-04-25T10:16:03.213659+0000","last_unstale":"2024-04-25T10:16:04.218875+0000","last_undegraded":"2024-04-25T10:16:04.218875+0000","last_fullsized":"2024-04-25T10:16:04.218875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:13.579259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099201+0000","last_change":"2024-04-25T10:16:03.226462+0000","last_active":"2024-04-25T10:16:05.099201+0000","last_peered":"2024-04-25T10:16:05.099201+0000","last_clean":"2024-04-25T10:16:05.099201+0000","last_became_active":"2024-04-25T10:16:03.225260+0000","last_became_peered":"2024-04-25T10:16:03.225260+0000","last_unstale":"2024-04-25T10:16:05.099201+0000","last_undegraded":"2024-04-25T10:16:05.099201+0000","last_fullsized":"2024-04-25T10:16:05.099201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:42.808262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111156+0000","last_change":"2024-04-25T10:16:03.212568+0000","last_active":"2024-04-25T10:16:05.111156+0000","last_peered":"2024-04-25T10:16:05.111156+0000","last_clean":"2024-04-25T10:16:05.111156+0000","last_became_active":"2024-04-25T10:16:03.212277+0000","last_became_peered":"2024-04-25T10:16:03.212277+0000","last_unstale":"2024-04-25T10:16:05.111156+0000","last_undegraded":"2024-04-25T10:16:05.111156+0000","last_fullsized":"2024-04-25T10:16:05.111156+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:58.625880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112846+0000","last_change":"2024-04-25T10:16:03.225756+0000","last_active":"2024-04-25T10:16:05.112846+0000","last_peered":"2024-04-25T10:16:05.112846+0000","last_clean":"2024-04-25T10:16:05.112846+0000","last_became_active":"2024-04-25T10:16:03.225419+0000","last_became_peered":"2024-04-25T10:16:03.225419+0000","last_unstale":"2024-04-25T10:16:05.112846+0000","last_undegraded":"2024-04-25T10:16:05.112846+0000","last_fullsized":"2024-04-25T10:16:05.112846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:28.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110518+0000","last_change":"2024-04-25T10:16:03.215676+0000","last_active":"2024-04-25T10:16:05.110518+0000","last_peered":"2024-04-25T10:16:05.110518+0000","last_clean":"2024-04-25T10:16:05.110518+0000","last_became_active":"2024-04-25T10:16:03.215329+0000","last_became_peered":"2024-04-25T10:16:03.215329+0000","last_unstale":"2024-04-25T10:16:05.110518+0000","last_undegraded":"2024-04-25T10:16:05.110518+0000","last_fullsized":"2024-04-25T10:16:05.110518+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:21.687178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110630+0000","last_change":"2024-04-25T10:16:03.233825+0000","last_active":"2024-04-25T10:16:05.110630+0000","last_peered":"2024-04-25T10:16:05.110630+0000","last_clean":"2024-04-25T10:16:05.110630+0000","last_became_active":"2024-04-25T10:16:03.233552+0000","last_became_peered":"2024-04-25T10:16:03.233552+0000","last_unstale":"2024-04-25T10:16:05.110630+0000","last_undegraded":"2024-04-25T10:16:05.110630+0000","last_fullsized":"2024-04-25T10:16:05.110630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:01.566191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112057+0000","last_change":"2024-04-25T10:16:03.217052+0000","last_active":"2024-04-25T10:16:05.112057+0000","last_peered":"2024-04-25T10:16:05.112057+0000","last_clean":"2024-04-25T10:16:05.112057+0000","last_became_active":"2024-04-25T10:16:03.216781+0000","last_became_peered":"2024-04-25T10:16:03.216781+0000","last_unstale":"2024-04-25T10:16:05.112057+0000","last_undegraded":"2024-04-25T10:16:05.112057+0000","last_fullsized":"2024-04-25T10:16:05.112057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:50.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.109977+0000","last_change":"2024-04-25T10:16:03.210711+0000","last_active":"2024-04-25T10:16:05.109977+0000","last_peered":"2024-04-25T10:16:05.109977+0000","last_clean":"2024-04-25T10:16:05.109977+0000","last_became_active":"2024-04-25T10:16:03.210257+0000","last_became_peered":"2024-04-25T10:16:03.210257+0000","last_unstale":"2024-04-25T10:16:05.109977+0000","last_undegraded":"2024-04-25T10:16:05.109977+0000","last_fullsized":"2024-04-25T10:16:05.109977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:28.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112585+0000","last_change":"2024-04-25T10:16:03.224992+0000","last_active":"2024-04-25T10:16:05.112585+0000","last_peered":"2024-04-25T10:16:05.112585+0000","last_clean":"2024-04-25T10:16:05.112585+0000","last_became_active":"2024-04-25T10:16:03.224677+0000","last_became_peered":"2024-04-25T10:16:03.224677+0000","last_unstale":"2024-04-25T10:16:05.112585+0000","last_undegraded":"2024-04-25T10:16:05.112585+0000","last_fullsized":"2024-04-25T10:16:05.112585+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:26.464110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111718+0000","last_change":"2024-04-25T10:16:03.226568+0000","last_active":"2024-04-25T10:16:05.111718+0000","last_peered":"2024-04-25T10:16:05.111718+0000","last_clean":"2024-04-25T10:16:05.111718+0000","last_became_active":"2024-04-25T10:16:03.226222+0000","last_became_peered":"2024-04-25T10:16:03.226222+0000","last_unstale":"2024-04-25T10:16:05.111718+0000","last_undegraded":"2024-04-25T10:16:05.111718+0000","last_fullsized":"2024-04-25T10:16:05.111718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.928358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110711+0000","last_change":"2024-04-25T10:16:03.213966+0000","last_active":"2024-04-25T10:16:05.110711+0000","last_peered":"2024-04-25T10:16:05.110711+0000","last_clean":"2024-04-25T10:16:05.110711+0000","last_became_active":"2024-04-25T10:16:03.207641+0000","last_became_peered":"2024-04-25T10:16:03.207641+0000","last_unstale":"2024-04-25T10:16:05.110711+0000","last_undegraded":"2024-04-25T10:16:05.110711+0000","last_fullsized":"2024-04-25T10:16:05.110711+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:28.515844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219478+0000","last_change":"2024-04-25T10:16:03.213365+0000","last_active":"2024-04-25T10:16:04.219478+0000","last_peered":"2024-04-25T10:16:04.219478+0000","last_clean":"2024-04-25T10:16:04.219478+0000","last_became_active":"2024-04-25T10:16:03.212977+0000","last_became_peered":"2024-04-25T10:16:03.212977+0000","last_unstale":"2024-04-25T10:16:04.219478+0000","last_undegraded":"2024-04-25T10:16:04.219478+0000","last_fullsized":"2024-04-25T10:16:04.219478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:25.592075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099198+0000","last_change":"2024-04-25T10:16:03.238000+0000","last_active":"2024-04-25T10:16:05.099198+0000","last_peered":"2024-04-25T10:16:05.099198+0000","last_clean":"2024-04-25T10:16:05.099198+0000","last_became_active":"2024-04-25T10:16:03.234047+0000","last_became_peered":"2024-04-25T10:16:03.234047+0000","last_unstale":"2024-04-25T10:16:05.099198+0000","last_undegraded":"2024-04-25T10:16:05.099198+0000","last_fullsized":"2024-04-25T10:16:05.099198+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:34.106239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110655+0000","last_change":"2024-04-25T10:16:03.213779+0000","last_active":"2024-04-25T10:16:05.110655+0000","last_peered":"2024-04-25T10:16:05.110655+0000","last_clean":"2024-04-25T10:16:05.110655+0000","last_became_active":"2024-04-25T10:16:03.213471+0000","last_became_peered":"2024-04-25T10:16:03.213471+0000","last_unstale":"2024-04-25T10:16:05.110655+0000","last_undegraded":"2024-04-25T10:16:05.110655+0000","last_fullsized":"2024-04-25T10:16:05.110655+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:56.118359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110609+0000","last_change":"2024-04-25T10:16:03.208290+0000","last_active":"2024-04-25T10:16:05.110609+0000","last_peered":"2024-04-25T10:16:05.110609+0000","last_clean":"2024-04-25T10:16:05.110609+0000","last_became_active":"2024-04-25T10:16:03.207975+0000","last_became_peered":"2024-04-25T10:16:03.207975+0000","last_unstale":"2024-04-25T10:16:05.110609+0000","last_undegraded":"2024-04-25T10:16:05.110609+0000","last_fullsized":"2024-04-25T10:16:05.110609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.936275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112131+0000","last_change":"2024-04-25T10:16:03.219261+0000","last_active":"2024-04-25T10:16:05.112131+0000","last_peered":"2024-04-25T10:16:05.112131+0000","last_clean":"2024-04-25T10:16:05.112131+0000","last_became_active":"2024-04-25T10:16:03.218985+0000","last_became_peered":"2024-04-25T10:16:03.218985+0000","last_unstale":"2024-04-25T10:16:05.112131+0000","last_undegraded":"2024-04-25T10:16:05.112131+0000","last_fullsized":"2024-04-25T10:16:05.112131+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:17.232248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.109815+0000","last_change":"2024-04-25T10:16:03.208169+0000","last_active":"2024-04-25T10:16:05.109815+0000","last_peered":"2024-04-25T10:16:05.109815+0000","last_clean":"2024-04-25T10:16:05.109815+0000","last_became_active":"2024-04-25T10:16:03.207918+0000","last_became_peered":"2024-04-25T10:16:03.207918+0000","last_unstale":"2024-04-25T10:16:05.109815+0000","last_undegraded":"2024-04-25T10:16:05.109815+0000","last_fullsized":"2024-04-25T10:16:05.109815+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:20.948558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099633+0000","last_change":"2024-04-25T10:16:03.233507+0000","last_active":"2024-04-25T10:16:05.099633+0000","last_peered":"2024-04-25T10:16:05.099633+0000","last_clean":"2024-04-25T10:16:05.099633+0000","last_became_active":"2024-04-25T10:16:03.229273+0000","last_became_peered":"2024-04-25T10:16:03.229273+0000","last_unstale":"2024-04-25T10:16:05.099633+0000","last_undegraded":"2024-04-25T10:16:05.099633+0000","last_fullsized":"2024-04-25T10:16:05.099633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.035601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099657+0000","last_change":"2024-04-25T10:16:03.223259+0000","last_active":"2024-04-25T10:16:05.099657+0000","last_peered":"2024-04-25T10:16:05.099657+0000","last_clean":"2024-04-25T10:16:05.099657+0000","last_became_active":"2024-04-25T10:16:03.222866+0000","last_became_peered":"2024-04-25T10:16:03.222866+0000","last_unstale":"2024-04-25T10:16:05.099657+0000","last_undegraded":"2024-04-25T10:16:05.099657+0000","last_fullsized":"2024-04-25T10:16:05.099657+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:26.464110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099458+0000","last_change":"2024-04-25T10:16:02.218713+0000","last_active":"2024-04-25T10:16:05.099458+0000","last_peered":"2024-04-25T10:16:05.099458+0000","last_clean":"2024-04-25T10:16:05.099458+0000","last_became_active":"2024-04-25T10:16:02.218399+0000","last_became_peered":"2024-04-25T10:16:02.218399+0000","last_unstale":"2024-04-25T10:16:05.099458+0000","last_undegraded":"2024-04-25T10:16:05.099458+0000","last_fullsized":"2024-04-25T10:16:05.099458+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:39.549154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218422+0000","last_change":"2024-04-25T10:16:03.214792+0000","last_active":"2024-04-25T10:16:04.218422+0000","last_peered":"2024-04-25T10:16:04.218422+0000","last_clean":"2024-04-25T10:16:04.218422+0000","last_became_active":"2024-04-25T10:16:03.214327+0000","last_became_peered":"2024-04-25T10:16:03.214327+0000","last_unstale":"2024-04-25T10:16:04.218422+0000","last_undegraded":"2024-04-25T10:16:04.218422+0000","last_fullsized":"2024-04-25T10:16:04.218422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:45.029732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110947+0000","last_change":"2024-04-25T10:16:02.214084+0000","last_active":"2024-04-25T10:16:05.110947+0000","last_peered":"2024-04-25T10:16:05.110947+0000","last_clean":"2024-04-25T10:16:05.110947+0000","last_became_active":"2024-04-25T10:16:02.213799+0000","last_became_peered":"2024-04-25T10:16:02.213799+0000","last_unstale":"2024-04-25T10:16:05.110947+0000","last_undegraded":"2024-04-25T10:16:05.110947+0000","last_fullsized":"2024-04-25T10:16:05.110947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:25.456644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099290+0000","last_change":"2024-04-25T10:16:03.237727+0000","last_active":"2024-04-25T10:16:05.099290+0000","last_peered":"2024-04-25T10:16:05.099290+0000","last_clean":"2024-04-25T10:16:05.099290+0000","last_became_active":"2024-04-25T10:16:03.237242+0000","last_became_peered":"2024-04-25T10:16:03.237242+0000","last_unstale":"2024-04-25T10:16:05.099290+0000","last_undegraded":"2024-04-25T10:16:05.099290+0000","last_fullsized":"2024-04-25T10:16:05.099290+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:50.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099503+0000","last_change":"2024-04-25T10:16:02.220200+0000","last_active":"2024-04-25T10:16:05.099503+0000","last_peered":"2024-04-25T10:16:05.099503+0000","last_clean":"2024-04-25T10:16:05.099503+0000","last_became_active":"2024-04-25T10:16:02.219926+0000","last_became_peered":"2024-04-25T10:16:02.219926+0000","last_unstale":"2024-04-25T10:16:05.099503+0000","last_undegraded":"2024-04-25T10:16:05.099503+0000","last_fullsized":"2024-04-25T10:16:05.099503+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:29.393094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099325+0000","last_change":"2024-04-25T10:16:03.238205+0000","last_active":"2024-04-25T10:16:05.099325+0000","last_peered":"2024-04-25T10:16:05.099325+0000","last_clean":"2024-04-25T10:16:05.099325+0000","last_became_active":"2024-04-25T10:16:03.237802+0000","last_became_peered":"2024-04-25T10:16:03.237802+0000","last_unstale":"2024-04-25T10:16:05.099325+0000","last_undegraded":"2024-04-25T10:16:05.099325+0000","last_fullsized":"2024-04-25T10:16:05.099325+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:17.232248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219213+0000","last_change":"2024-04-25T10:16:02.211448+0000","last_active":"2024-04-25T10:16:04.219213+0000","last_peered":"2024-04-25T10:16:04.219213+0000","last_clean":"2024-04-25T10:16:04.219213+0000","last_became_active":"2024-04-25T10:16:02.211164+0000","last_became_peered":"2024-04-25T10:16:02.211164+0000","last_unstale":"2024-04-25T10:16:04.219213+0000","last_undegraded":"2024-04-25T10:16:04.219213+0000","last_fullsized":"2024-04-25T10:16:04.219213+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.601764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110179+0000","last_change":"2024-04-25T10:16:03.208197+0000","last_active":"2024-04-25T10:16:05.110179+0000","last_peered":"2024-04-25T10:16:05.110179+0000","last_clean":"2024-04-25T10:16:05.110179+0000","last_became_active":"2024-04-25T10:16:03.207905+0000","last_became_peered":"2024-04-25T10:16:03.207905+0000","last_unstale":"2024-04-25T10:16:05.110179+0000","last_undegraded":"2024-04-25T10:16:05.110179+0000","last_fullsized":"2024-04-25T10:16:05.110179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:10.563085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218648+0000","last_change":"2024-04-25T10:16:02.212831+0000","last_active":"2024-04-25T10:16:04.218648+0000","last_peered":"2024-04-25T10:16:04.218648+0000","last_clean":"2024-04-25T10:16:04.218648+0000","last_became_active":"2024-04-25T10:16:02.212574+0000","last_became_peered":"2024-04-25T10:16:02.212574+0000","last_unstale":"2024-04-25T10:16:04.218648+0000","last_undegraded":"2024-04-25T10:16:04.218648+0000","last_fullsized":"2024-04-25T10:16:04.218648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:51.172043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110163+0000","last_change":"2024-04-25T10:16:03.225025+0000","last_active":"2024-04-25T10:16:05.110163+0000","last_peered":"2024-04-25T10:16:05.110163+0000","last_clean":"2024-04-25T10:16:05.110163+0000","last_became_active":"2024-04-25T10:16:03.224421+0000","last_became_peered":"2024-04-25T10:16:03.224421+0000","last_unstale":"2024-04-25T10:16:05.110163+0000","last_undegraded":"2024-04-25T10:16:05.110163+0000","last_fullsized":"2024-04-25T10:16:05.110163+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:11.037843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099072+0000","last_change":"2024-04-25T10:16:02.213326+0000","last_active":"2024-04-25T10:16:05.099072+0000","last_peered":"2024-04-25T10:16:05.099072+0000","last_clean":"2024-04-25T10:16:05.099072+0000","last_became_active":"2024-04-25T10:16:02.213019+0000","last_became_peered":"2024-04-25T10:16:02.213019+0000","last_unstale":"2024-04-25T10:16:05.099072+0000","last_undegraded":"2024-04-25T10:16:05.099072+0000","last_fullsized":"2024-04-25T10:16:05.099072+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:46.109348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111735+0000","last_change":"2024-04-25T10:16:03.217188+0000","last_active":"2024-04-25T10:16:05.111735+0000","last_peered":"2024-04-25T10:16:05.111735+0000","last_clean":"2024-04-25T10:16:05.111735+0000","last_became_active":"2024-04-25T10:16:03.216911+0000","last_became_peered":"2024-04-25T10:16:03.216911+0000","last_unstale":"2024-04-25T10:16:05.111735+0000","last_undegraded":"2024-04-25T10:16:05.111735+0000","last_fullsized":"2024-04-25T10:16:05.111735+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.975892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219191+0000","last_change":"2024-04-25T10:16:02.208298+0000","last_active":"2024-04-25T10:16:04.219191+0000","last_peered":"2024-04-25T10:16:04.219191+0000","last_clean":"2024-04-25T10:16:04.219191+0000","last_became_active":"2024-04-25T10:16:02.207764+0000","last_became_peered":"2024-04-25T10:16:02.207764+0000","last_unstale":"2024-04-25T10:16:04.219191+0000","last_undegraded":"2024-04-25T10:16:04.219191+0000","last_fullsized":"2024-04-25T10:16:04.219191+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:23.211303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110053+0000","last_change":"2024-04-25T10:16:03.214179+0000","last_active":"2024-04-25T10:16:05.110053+0000","last_peered":"2024-04-25T10:16:05.110053+0000","last_clean":"2024-04-25T10:16:05.110053+0000","last_became_active":"2024-04-25T10:16:03.213237+0000","last_became_peered":"2024-04-25T10:16:03.213237+0000","last_unstale":"2024-04-25T10:16:05.110053+0000","last_undegraded":"2024-04-25T10:16:05.110053+0000","last_fullsized":"2024-04-25T10:16:05.110053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:55.174050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219205+0000","last_change":"2024-04-25T10:16:02.216008+0000","last_active":"2024-04-25T10:16:04.219205+0000","last_peered":"2024-04-25T10:16:04.219205+0000","last_clean":"2024-04-25T10:16:04.219205+0000","last_became_active":"2024-04-25T10:16:02.215473+0000","last_became_peered":"2024-04-25T10:16:02.215473+0000","last_unstale":"2024-04-25T10:16:04.219205+0000","last_undegraded":"2024-04-25T10:16:04.219205+0000","last_fullsized":"2024-04-25T10:16:04.219205+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:02.872977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218965+0000","last_change":"2024-04-25T10:16:03.214412+0000","last_active":"2024-04-25T10:16:04.218965+0000","last_peered":"2024-04-25T10:16:04.218965+0000","last_clean":"2024-04-25T10:16:04.218965+0000","last_became_active":"2024-04-25T10:16:03.214127+0000","last_became_peered":"2024-04-25T10:16:03.214127+0000","last_unstale":"2024-04-25T10:16:04.218965+0000","last_undegraded":"2024-04-25T10:16:04.218965+0000","last_fullsized":"2024-04-25T10:16:04.218965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:08.108219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112856+0000","last_change":"2024-04-25T10:16:02.214697+0000","last_active":"2024-04-25T10:16:05.112856+0000","last_peered":"2024-04-25T10:16:05.112856+0000","last_clean":"2024-04-25T10:16:05.112856+0000","last_became_active":"2024-04-25T10:16:02.214228+0000","last_became_peered":"2024-04-25T10:16:02.214228+0000","last_unstale":"2024-04-25T10:16:05.112856+0000","last_undegraded":"2024-04-25T10:16:05.112856+0000","last_fullsized":"2024-04-25T10:16:05.112856+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:17.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112875+0000","last_change":"2024-04-25T10:16:03.222574+0000","last_active":"2024-04-25T10:16:05.112875+0000","last_peered":"2024-04-25T10:16:05.112875+0000","last_clean":"2024-04-25T10:16:05.112875+0000","last_became_active":"2024-04-25T10:16:03.222375+0000","last_became_peered":"2024-04-25T10:16:03.222375+0000","last_unstale":"2024-04-25T10:16:05.112875+0000","last_undegraded":"2024-04-25T10:16:05.112875+0000","last_fullsized":"2024-04-25T10:16:05.112875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:57.698290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111037+0000","last_change":"2024-04-25T10:16:02.210766+0000","last_active":"2024-04-25T10:16:05.111037+0000","last_peered":"2024-04-25T10:16:05.111037+0000","last_clean":"2024-04-25T10:16:05.111037+0000","last_became_active":"2024-04-25T10:16:02.210506+0000","last_became_peered":"2024-04-25T10:16:02.210506+0000","last_unstale":"2024-04-25T10:16:05.111037+0000","last_undegraded":"2024-04-25T10:16:05.111037+0000","last_fullsized":"2024-04-25T10:16:05.111037+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:46.709344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217802+0000","last_change":"2024-04-25T10:16:03.212578+0000","last_active":"2024-04-25T10:16:04.217802+0000","last_peered":"2024-04-25T10:16:04.217802+0000","last_clean":"2024-04-25T10:16:04.217802+0000","last_became_active":"2024-04-25T10:16:03.212035+0000","last_became_peered":"2024-04-25T10:16:03.212035+0000","last_unstale":"2024-04-25T10:16:04.217802+0000","last_undegraded":"2024-04-25T10:16:04.217802+0000","last_fullsized":"2024-04-25T10:16:04.217802+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:10.845809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112694+0000","last_change":"2024-04-25T10:16:02.209148+0000","last_active":"2024-04-25T10:16:05.112694+0000","last_peered":"2024-04-25T10:16:05.112694+0000","last_clean":"2024-04-25T10:16:05.112694+0000","last_became_active":"2024-04-25T10:16:02.208808+0000","last_became_peered":"2024-04-25T10:16:02.208808+0000","last_unstale":"2024-04-25T10:16:05.112694+0000","last_undegraded":"2024-04-25T10:16:05.112694+0000","last_fullsized":"2024-04-25T10:16:05.112694+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:29.393094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112597+0000","last_change":"2024-04-25T10:16:03.221479+0000","last_active":"2024-04-25T10:16:05.112597+0000","last_peered":"2024-04-25T10:16:05.112597+0000","last_clean":"2024-04-25T10:16:05.112597+0000","last_became_active":"2024-04-25T10:16:03.221191+0000","last_became_peered":"2024-04-25T10:16:03.221191+0000","last_unstale":"2024-04-25T10:16:05.112597+0000","last_undegraded":"2024-04-25T10:16:05.112597+0000","last_fullsized":"2024-04-25T10:16:05.112597+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:47.716811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218930+0000","last_change":"2024-04-25T10:16:02.212991+0000","last_active":"2024-04-25T10:16:04.218930+0000","last_peered":"2024-04-25T10:16:04.218930+0000","last_clean":"2024-04-25T10:16:04.218930+0000","last_became_active":"2024-04-25T10:16:02.212705+0000","last_became_peered":"2024-04-25T10:16:02.212705+0000","last_unstale":"2024-04-25T10:16:04.218930+0000","last_undegraded":"2024-04-25T10:16:04.218930+0000","last_fullsized":"2024-04-25T10:16:04.218930+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:44.022265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219303+0000","last_change":"2024-04-25T10:16:03.222814+0000","last_active":"2024-04-25T10:16:04.219303+0000","last_peered":"2024-04-25T10:16:04.219303+0000","last_clean":"2024-04-25T10:16:04.219303+0000","last_became_active":"2024-04-25T10:16:03.221689+0000","last_became_peered":"2024-04-25T10:16:03.221689+0000","last_unstale":"2024-04-25T10:16:04.219303+0000","last_undegraded":"2024-04-25T10:16:04.219303+0000","last_fullsized":"2024-04-25T10:16:04.219303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.241014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110237+0000","last_change":"2024-04-25T10:16:02.211496+0000","last_active":"2024-04-25T10:16:05.110237+0000","last_peered":"2024-04-25T10:16:05.110237+0000","last_clean":"2024-04-25T10:16:05.110237+0000","last_became_active":"2024-04-25T10:16:02.211239+0000","last_became_peered":"2024-04-25T10:16:02.211239+0000","last_unstale":"2024-04-25T10:16:05.110237+0000","last_undegraded":"2024-04-25T10:16:05.110237+0000","last_fullsized":"2024-04-25T10:16:05.110237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.971835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112427+0000","last_change":"2024-04-25T10:16:03.218556+0000","last_active":"2024-04-25T10:16:05.112427+0000","last_peered":"2024-04-25T10:16:05.112427+0000","last_clean":"2024-04-25T10:16:05.112427+0000","last_became_active":"2024-04-25T10:16:03.218197+0000","last_became_peered":"2024-04-25T10:16:03.218197+0000","last_unstale":"2024-04-25T10:16:05.112427+0000","last_undegraded":"2024-04-25T10:16:05.112427+0000","last_fullsized":"2024-04-25T10:16:05.112427+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:25.592075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111813+0000","last_change":"2024-04-25T10:16:02.214051+0000","last_active":"2024-04-25T10:16:05.111813+0000","last_peered":"2024-04-25T10:16:05.111813+0000","last_clean":"2024-04-25T10:16:05.111813+0000","last_became_active":"2024-04-25T10:16:02.213461+0000","last_became_peered":"2024-04-25T10:16:02.213461+0000","last_unstale":"2024-04-25T10:16:05.111813+0000","last_undegraded":"2024-04-25T10:16:05.111813+0000","last_fullsized":"2024-04-25T10:16:05.111813+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.425480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218473+0000","last_change":"2024-04-25T10:16:03.215030+0000","last_active":"2024-04-25T10:16:04.218473+0000","last_peered":"2024-04-25T10:16:04.218473+0000","last_clean":"2024-04-25T10:16:04.218473+0000","last_became_active":"2024-04-25T10:16:03.214773+0000","last_became_peered":"2024-04-25T10:16:03.214773+0000","last_unstale":"2024-04-25T10:16:04.218473+0000","last_undegraded":"2024-04-25T10:16:04.218473+0000","last_fullsized":"2024-04-25T10:16:04.218473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:32.957206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219333+0000","last_change":"2024-04-25T10:16:02.212260+0000","last_active":"2024-04-25T10:16:04.219333+0000","last_peered":"2024-04-25T10:16:04.219333+0000","last_clean":"2024-04-25T10:16:04.219333+0000","last_became_active":"2024-04-25T10:16:02.210462+0000","last_became_peered":"2024-04-25T10:16:02.210462+0000","last_unstale":"2024-04-25T10:16:04.219333+0000","last_undegraded":"2024-04-25T10:16:04.219333+0000","last_fullsized":"2024-04-25T10:16:04.219333+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:05.615221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112494+0000","last_change":"2024-04-25T10:16:03.224990+0000","last_active":"2024-04-25T10:16:05.112494+0000","last_peered":"2024-04-25T10:16:05.112494+0000","last_clean":"2024-04-25T10:16:05.112494+0000","last_became_active":"2024-04-25T10:16:03.224772+0000","last_became_peered":"2024-04-25T10:16:03.224772+0000","last_unstale":"2024-04-25T10:16:05.112494+0000","last_undegraded":"2024-04-25T10:16:05.112494+0000","last_fullsized":"2024-04-25T10:16:05.112494+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.035601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":22,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111953+0000","last_change":"2024-04-25T10:16:02.211589+0000","last_active":"2024-04-25T10:16:05.111953+0000","last_peered":"2024-04-25T10:16:05.111953+0000","last_clean":"2024-04-25T10:16:05.111953+0000","last_became_active":"2024-04-25T10:16:02.211326+0000","last_became_peered":"2024-04-25T10:16:02.211326+0000","last_unstale":"2024-04-25T10:16:05.111953+0000","last_undegraded":"2024-04-25T10:16:05.111953+0000","last_fullsized":"2024-04-25T10:16:05.111953+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:05.615221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112861+0000","last_change":"2024-04-25T10:16:03.221514+0000","last_active":"2024-04-25T10:16:05.112861+0000","last_peered":"2024-04-25T10:16:05.112861+0000","last_clean":"2024-04-25T10:16:05.112861+0000","last_became_active":"2024-04-25T10:16:03.221191+0000","last_became_peered":"2024-04-25T10:16:03.221191+0000","last_unstale":"2024-04-25T10:16:05.112861+0000","last_undegraded":"2024-04-25T10:16:05.112861+0000","last_fullsized":"2024-04-25T10:16:05.112861+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:23.623322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111092+0000","last_change":"2024-04-25T10:16:02.210822+0000","last_active":"2024-04-25T10:16:05.111092+0000","last_peered":"2024-04-25T10:16:05.111092+0000","last_clean":"2024-04-25T10:16:05.111092+0000","last_became_active":"2024-04-25T10:16:02.210601+0000","last_became_peered":"2024-04-25T10:16:02.210601+0000","last_unstale":"2024-04-25T10:16:05.111092+0000","last_undegraded":"2024-04-25T10:16:05.111092+0000","last_fullsized":"2024-04-25T10:16:05.111092+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:24.584609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219214+0000","last_change":"2024-04-25T10:16:03.222519+0000","last_active":"2024-04-25T10:16:04.219214+0000","last_peered":"2024-04-25T10:16:04.219214+0000","last_clean":"2024-04-25T10:16:04.219214+0000","last_became_active":"2024-04-25T10:16:03.222223+0000","last_became_peered":"2024-04-25T10:16:03.222223+0000","last_unstale":"2024-04-25T10:16:04.219214+0000","last_undegraded":"2024-04-25T10:16:04.219214+0000","last_fullsized":"2024-04-25T10:16:04.219214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:50.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099601+0000","last_change":"2024-04-25T10:16:02.215451+0000","last_active":"2024-04-25T10:16:05.099601+0000","last_peered":"2024-04-25T10:16:05.099601+0000","last_clean":"2024-04-25T10:16:05.099601+0000","last_became_active":"2024-04-25T10:16:02.215077+0000","last_became_peered":"2024-04-25T10:16:02.215077+0000","last_unstale":"2024-04-25T10:16:05.099601+0000","last_undegraded":"2024-04-25T10:16:05.099601+0000","last_fullsized":"2024-04-25T10:16:05.099601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:17.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110795+0000","last_change":"2024-04-25T10:16:03.213272+0000","last_active":"2024-04-25T10:16:05.110795+0000","last_peered":"2024-04-25T10:16:05.110795+0000","last_clean":"2024-04-25T10:16:05.110795+0000","last_became_active":"2024-04-25T10:16:03.212960+0000","last_became_peered":"2024-04-25T10:16:03.212960+0000","last_unstale":"2024-04-25T10:16:05.110795+0000","last_undegraded":"2024-04-25T10:16:05.110795+0000","last_fullsized":"2024-04-25T10:16:05.110795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:44.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218979+0000","last_change":"2024-04-25T10:16:02.210609+0000","last_active":"2024-04-25T10:16:04.218979+0000","last_peered":"2024-04-25T10:16:04.218979+0000","last_clean":"2024-04-25T10:16:04.218979+0000","last_became_active":"2024-04-25T10:16:02.210255+0000","last_became_peered":"2024-04-25T10:16:02.210255+0000","last_unstale":"2024-04-25T10:16:04.218979+0000","last_undegraded":"2024-04-25T10:16:04.218979+0000","last_fullsized":"2024-04-25T10:16:04.218979+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:17.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112681+0000","last_change":"2024-04-25T10:16:03.223120+0000","last_active":"2024-04-25T10:16:05.112681+0000","last_peered":"2024-04-25T10:16:05.112681+0000","last_clean":"2024-04-25T10:16:05.112681+0000","last_became_active":"2024-04-25T10:16:03.222910+0000","last_became_peered":"2024-04-25T10:16:03.222910+0000","last_unstale":"2024-04-25T10:16:05.112681+0000","last_undegraded":"2024-04-25T10:16:05.112681+0000","last_fullsized":"2024-04-25T10:16:05.112681+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:07.700085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099284+0000","last_change":"2024-04-25T10:16:02.220183+0000","last_active":"2024-04-25T10:16:05.099284+0000","last_peered":"2024-04-25T10:16:05.099284+0000","last_clean":"2024-04-25T10:16:05.099284+0000","last_became_active":"2024-04-25T10:16:02.219686+0000","last_became_peered":"2024-04-25T10:16:02.219686+0000","last_unstale":"2024-04-25T10:16:05.099284+0000","last_undegraded":"2024-04-25T10:16:05.099284+0000","last_fullsized":"2024-04-25T10:16:05.099284+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.949740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219223+0000","last_change":"2024-04-25T10:16:03.221708+0000","last_active":"2024-04-25T10:16:04.219223+0000","last_peered":"2024-04-25T10:16:04.219223+0000","last_clean":"2024-04-25T10:16:04.219223+0000","last_became_active":"2024-04-25T10:16:03.221301+0000","last_became_peered":"2024-04-25T10:16:03.221301+0000","last_unstale":"2024-04-25T10:16:04.219223+0000","last_undegraded":"2024-04-25T10:16:04.219223+0000","last_fullsized":"2024-04-25T10:16:04.219223+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:17.232248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110667+0000","last_change":"2024-04-25T10:16:02.208859+0000","last_active":"2024-04-25T10:16:05.110667+0000","last_peered":"2024-04-25T10:16:05.110667+0000","last_clean":"2024-04-25T10:16:05.110667+0000","last_became_active":"2024-04-25T10:16:02.208611+0000","last_became_peered":"2024-04-25T10:16:02.208611+0000","last_unstale":"2024-04-25T10:16:05.110667+0000","last_undegraded":"2024-04-25T10:16:05.110667+0000","last_fullsized":"2024-04-25T10:16:05.110667+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:58.028135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099677+0000","last_change":"2024-04-25T10:16:03.229087+0000","last_active":"2024-04-25T10:16:05.099677+0000","last_peered":"2024-04-25T10:16:05.099677+0000","last_clean":"2024-04-25T10:16:05.099677+0000","last_became_active":"2024-04-25T10:16:03.226116+0000","last_became_peered":"2024-04-25T10:16:03.226116+0000","last_unstale":"2024-04-25T10:16:05.099677+0000","last_undegraded":"2024-04-25T10:16:05.099677+0000","last_fullsized":"2024-04-25T10:16:05.099677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:47.716811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217713+0000","last_change":"2024-04-25T10:16:02.217945+0000","last_active":"2024-04-25T10:16:04.217713+0000","last_peered":"2024-04-25T10:16:04.217713+0000","last_clean":"2024-04-25T10:16:04.217713+0000","last_became_active":"2024-04-25T10:16:02.217603+0000","last_became_peered":"2024-04-25T10:16:02.217603+0000","last_unstale":"2024-04-25T10:16:04.217713+0000","last_undegraded":"2024-04-25T10:16:04.217713+0000","last_fullsized":"2024-04-25T10:16:04.217713+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:38.125275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099582+0000","last_change":"2024-04-25T10:16:03.228414+0000","last_active":"2024-04-25T10:16:05.099582+0000","last_peered":"2024-04-25T10:16:05.099582+0000","last_clean":"2024-04-25T10:16:05.099582+0000","last_became_active":"2024-04-25T10:16:03.228138+0000","last_became_peered":"2024-04-25T10:16:03.228138+0000","last_unstale":"2024-04-25T10:16:05.099582+0000","last_undegraded":"2024-04-25T10:16:05.099582+0000","last_fullsized":"2024-04-25T10:16:05.099582+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:25.592075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218297+0000","last_change":"2024-04-25T10:16:02.211646+0000","last_active":"2024-04-25T10:16:04.218297+0000","last_peered":"2024-04-25T10:16:04.218297+0000","last_clean":"2024-04-25T10:16:04.218297+0000","last_became_active":"2024-04-25T10:16:02.210617+0000","last_became_peered":"2024-04-25T10:16:02.210617+0000","last_unstale":"2024-04-25T10:16:04.218297+0000","last_undegraded":"2024-04-25T10:16:04.218297+0000","last_fullsized":"2024-04-25T10:16:04.218297+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:29.393094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219555+0000","last_change":"2024-04-25T10:16:03.209355+0000","last_active":"2024-04-25T10:16:04.219555+0000","last_peered":"2024-04-25T10:16:04.219555+0000","last_clean":"2024-04-25T10:16:04.219555+0000","last_became_active":"2024-04-25T10:16:03.209063+0000","last_became_peered":"2024-04-25T10:16:03.209063+0000","last_unstale":"2024-04-25T10:16:04.219555+0000","last_undegraded":"2024-04-25T10:16:04.219555+0000","last_fullsized":"2024-04-25T10:16:04.219555+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:08.979302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218345+0000","last_change":"2024-04-25T10:16:02.213084+0000","last_active":"2024-04-25T10:16:04.218345+0000","last_peered":"2024-04-25T10:16:04.218345+0000","last_clean":"2024-04-25T10:16:04.218345+0000","last_became_active":"2024-04-25T10:16:02.212783+0000","last_became_peered":"2024-04-25T10:16:02.212783+0000","last_unstale":"2024-04-25T10:16:04.218345+0000","last_undegraded":"2024-04-25T10:16:04.218345+0000","last_fullsized":"2024-04-25T10:16:04.218345+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:04.001291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219719+0000","last_change":"2024-04-25T10:16:03.216940+0000","last_active":"2024-04-25T10:16:04.219719+0000","last_peered":"2024-04-25T10:16:04.219719+0000","last_clean":"2024-04-25T10:16:04.219719+0000","last_became_active":"2024-04-25T10:16:03.216076+0000","last_became_peered":"2024-04-25T10:16:03.216076+0000","last_unstale":"2024-04-25T10:16:04.219719+0000","last_undegraded":"2024-04-25T10:16:04.219719+0000","last_fullsized":"2024-04-25T10:16:04.219719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.035601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099725+0000","last_change":"2024-04-25T10:16:02.212115+0000","last_active":"2024-04-25T10:16:05.099725+0000","last_peered":"2024-04-25T10:16:05.099725+0000","last_clean":"2024-04-25T10:16:05.099725+0000","last_became_active":"2024-04-25T10:16:02.211825+0000","last_became_peered":"2024-04-25T10:16:02.211825+0000","last_unstale":"2024-04-25T10:16:05.099725+0000","last_undegraded":"2024-04-25T10:16:05.099725+0000","last_fullsized":"2024-04-25T10:16:05.099725+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.601764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217896+0000","last_change":"2024-04-25T10:16:03.212335+0000","last_active":"2024-04-25T10:16:04.217896+0000","last_peered":"2024-04-25T10:16:04.217896+0000","last_clean":"2024-04-25T10:16:04.217896+0000","last_became_active":"2024-04-25T10:16:03.211685+0000","last_became_peered":"2024-04-25T10:16:03.211685+0000","last_unstale":"2024-04-25T10:16:04.217896+0000","last_undegraded":"2024-04-25T10:16:04.217896+0000","last_fullsized":"2024-04-25T10:16:04.217896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:03.880444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111761+0000","last_change":"2024-04-25T10:16:02.213981+0000","last_active":"2024-04-25T10:16:05.111761+0000","last_peered":"2024-04-25T10:16:05.111761+0000","last_clean":"2024-04-25T10:16:05.111761+0000","last_became_active":"2024-04-25T10:16:02.213716+0000","last_became_peered":"2024-04-25T10:16:02.213716+0000","last_unstale":"2024-04-25T10:16:05.111761+0000","last_undegraded":"2024-04-25T10:16:05.111761+0000","last_fullsized":"2024-04-25T10:16:05.111761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:31.444807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.099618+0000","last_change":"2024-04-25T10:16:03.238129+0000","last_active":"2024-04-25T10:16:05.099618+0000","last_peered":"2024-04-25T10:16:05.099618+0000","last_clean":"2024-04-25T10:16:05.099618+0000","last_became_active":"2024-04-25T10:16:03.237730+0000","last_became_peered":"2024-04-25T10:16:03.237730+0000","last_unstale":"2024-04-25T10:16:05.099618+0000","last_undegraded":"2024-04-25T10:16:05.099618+0000","last_fullsized":"2024-04-25T10:16:05.099618+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:08.979302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.219240+0000","last_change":"2024-04-25T10:16:02.210677+0000","last_active":"2024-04-25T10:16:04.219240+0000","last_peered":"2024-04-25T10:16:04.219240+0000","last_clean":"2024-04-25T10:16:04.219240+0000","last_became_active":"2024-04-25T10:16:02.209577+0000","last_became_peered":"2024-04-25T10:16:02.209577+0000","last_unstale":"2024-04-25T10:16:04.219240+0000","last_undegraded":"2024-04-25T10:16:04.219240+0000","last_fullsized":"2024-04-25T10:16:04.219240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:31.949740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110366+0000","last_change":"2024-04-25T10:16:03.220903+0000","last_active":"2024-04-25T10:16:05.110366+0000","last_peered":"2024-04-25T10:16:05.110366+0000","last_clean":"2024-04-25T10:16:05.110366+0000","last_became_active":"2024-04-25T10:16:03.220460+0000","last_became_peered":"2024-04-25T10:16:03.220460+0000","last_unstale":"2024-04-25T10:16:05.110366+0000","last_undegraded":"2024-04-25T10:16:05.110366+0000","last_fullsized":"2024-04-25T10:16:05.110366+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:20.939254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217546+0000","last_change":"2024-04-25T10:16:02.213146+0000","last_active":"2024-04-25T10:16:04.217546+0000","last_peered":"2024-04-25T10:16:04.217546+0000","last_clean":"2024-04-25T10:16:04.217546+0000","last_became_active":"2024-04-25T10:16:02.212835+0000","last_became_peered":"2024-04-25T10:16:02.212835+0000","last_unstale":"2024-04-25T10:16:04.217546+0000","last_undegraded":"2024-04-25T10:16:04.217546+0000","last_fullsized":"2024-04-25T10:16:04.217546+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:31.444807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.112226+0000","last_change":"2024-04-25T10:16:03.217188+0000","last_active":"2024-04-25T10:16:05.112226+0000","last_peered":"2024-04-25T10:16:05.112226+0000","last_clean":"2024-04-25T10:16:05.112226+0000","last_became_active":"2024-04-25T10:16:03.216919+0000","last_became_peered":"2024-04-25T10:16:03.216919+0000","last_unstale":"2024-04-25T10:16:05.112226+0000","last_undegraded":"2024-04-25T10:16:05.112226+0000","last_fullsized":"2024-04-25T10:16:05.112226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.241014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.218617+0000","last_change":"2024-04-25T10:16:02.215591+0000","last_active":"2024-04-25T10:16:04.218617+0000","last_peered":"2024-04-25T10:16:04.218617+0000","last_clean":"2024-04-25T10:16:04.218617+0000","last_became_active":"2024-04-25T10:16:02.215314+0000","last_became_peered":"2024-04-25T10:16:02.215314+0000","last_unstale":"2024-04-25T10:16:04.218617+0000","last_undegraded":"2024-04-25T10:16:04.218617+0000","last_fullsized":"2024-04-25T10:16:04.218617+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.080370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217546+0000","last_change":"2024-04-25T10:16:03.215126+0000","last_active":"2024-04-25T10:16:04.217546+0000","last_peered":"2024-04-25T10:16:04.217546+0000","last_clean":"2024-04-25T10:16:04.217546+0000","last_became_active":"2024-04-25T10:16:03.214854+0000","last_became_peered":"2024-04-25T10:16:03.214854+0000","last_unstale":"2024-04-25T10:16:04.217546+0000","last_undegraded":"2024-04-25T10:16:04.217546+0000","last_fullsized":"2024-04-25T10:16:04.217546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:40.918814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:04.217542+0000","last_change":"2024-04-25T10:16:02.215889+0000","last_active":"2024-04-25T10:16:04.217542+0000","last_peered":"2024-04-25T10:16:04.217542+0000","last_clean":"2024-04-25T10:16:04.217542+0000","last_became_active":"2024-04-25T10:16:02.215576+0000","last_became_peered":"2024-04-25T10:16:02.215576+0000","last_unstale":"2024-04-25T10:16:04.217542+0000","last_undegraded":"2024-04-25T10:16:04.217542+0000","last_fullsized":"2024-04-25T10:16:04.217542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.425480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110234+0000","last_change":"2024-04-25T10:16:03.214468+0000","last_active":"2024-04-25T10:16:05.110234+0000","last_peered":"2024-04-25T10:16:05.110234+0000","last_clean":"2024-04-25T10:16:05.110234+0000","last_became_active":"2024-04-25T10:16:03.213627+0000","last_became_peered":"2024-04-25T10:16:03.213627+0000","last_unstale":"2024-04-25T10:16:05.110234+0000","last_undegraded":"2024-04-25T10:16:05.110234+0000","last_fullsized":"2024-04-25T10:16:05.110234+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:06.011341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110241+0000","last_change":"2024-04-25T10:16:02.209475+0000","last_active":"2024-04-25T10:16:05.110241+0000","last_peered":"2024-04-25T10:16:05.110241+0000","last_clean":"2024-04-25T10:16:05.110241+0000","last_became_active":"2024-04-25T10:16:02.209056+0000","last_became_peered":"2024-04-25T10:16:02.209056+0000","last_unstale":"2024-04-25T10:16:05.110241+0000","last_undegraded":"2024-04-25T10:16:05.110241+0000","last_fullsized":"2024-04-25T10:16:05.110241+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:58.233547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.110172+0000","last_change":"2024-04-25T10:16:03.219221+0000","last_active":"2024-04-25T10:16:05.110172+0000","last_peered":"2024-04-25T10:16:05.110172+0000","last_clean":"2024-04-25T10:16:05.110172+0000","last_became_active":"2024-04-25T10:16:03.218975+0000","last_became_peered":"2024-04-25T10:16:03.218975+0000","last_unstale":"2024-04-25T10:16:05.110172+0000","last_undegraded":"2024-04-25T10:16:05.110172+0000","last_fullsized":"2024-04-25T10:16:05.110172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:02.181305+0000","last_clean_scrub_stamp":"2024-04-25T10:16:02.181305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:42.563939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111659+0000","last_change":"2024-04-25T10:16:02.213664+0000","last_active":"2024-04-25T10:16:05.111659+0000","last_peered":"2024-04-25T10:16:05.111659+0000","last_clean":"2024-04-25T10:16:05.111659+0000","last_became_active":"2024-04-25T10:16:02.213276+0000","last_became_peered":"2024-04-25T10:16:02.213276+0000","last_unstale":"2024-04-25T10:16:05.111659+0000","last_undegraded":"2024-04-25T10:16:05.111659+0000","last_fullsized":"2024-04-25T10:16:05.111659+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:01.173838+0000","last_clean_scrub_stamp":"2024-04-25T10:16:01.173838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:38.125275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:16:05.111966+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:16:05.111966+0000","last_peered":"2024-04-25T10:16:05.111966+0000","last_clean":"2024-04-25T10:16:05.111966+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:16:05.111966+0000","last_undegraded":"2024-04-25T10:16:05.111966+0000","last_fullsized":"2024-04-25T10:16:05.111966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40756,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331084,"statfs":{"total":96636764160,"available":96595030016,"internally_reserved":0,"allocated":1286144,"data_stored":875877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":285349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":286001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":286961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58224,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313616,"statfs":{"total":96636764160,"available":96577142784,"internally_reserved":0,"allocated":692224,"data_stored":285405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":285405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40180,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331660,"statfs":{"total":96636764160,"available":96595619840,"internally_reserved":0,"allocated":688128,"data_stored":285439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1294336,"data_stored":877911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:07.661 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:16:07.661 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:16:07.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9fa00640 1 Processor -- start 2024-04-25T10:16:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9fa00640 1 -- start start 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9fa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa980d0070 0x7ffa980b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9fa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9815e6e0 con 0x7ffa980b39d0 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9fa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9815e9c0 con 0x7ffa98154f70 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9fa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9815eca0 con 0x7ffa980d0070 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9da00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ffa980b39d0 0x7ffa980ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60038/0 (socket says 172.21.15.87:60038) 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa9da00640 1 -- 172.21.15.87:0/2733399663 learned_addr learned my addr 172.21.15.87:0/2733399663 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa97e00640 1 --2- 172.21.15.87:0/2733399663 >> v2:172.21.15.169:3300/0 conn(0x7ffa980d0070 0x7ffa980b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa97400640 1 -- 172.21.15.87:0/2733399663 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240639073 0 0) 0x7ffa9815e6e0 con 0x7ffa980b39d0 2024-04-25T10:16:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa97400640 1 -- 172.21.15.87:0/2733399663 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa80003530 con 0x7ffa980b39d0 2024-04-25T10:16:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa97e00640 1 -- 172.21.15.87:0/2733399663 >> v1:172.21.15.169:6789/0 conn(0x7ffa98154f70 legacy=0x7ffa980cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa97e00640 1 -- 172.21.15.87:0/2733399663 >> v1:172.21.15.87:6789/0 conn(0x7ffa980b39d0 legacy=0x7ffa980ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.839+0000 7ffa97e00640 1 -- 172.21.15.87:0/2733399663 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9815ef80 con 0x7ffa980d0070 2024-04-25T10:16:07.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa97e00640 1 --2- 172.21.15.87:0/2733399663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa980d0070 0x7ffa980b3490 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa840097c0 tx=0x7ffa8401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204b1e75c64d92e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa97400640 1 -- 172.21.15.87:0/2733399663 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840b0070 con 0x7ffa980d0070 2024-04-25T10:16:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa97400640 1 -- 172.21.15.87:0/2733399663 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa840ab600 con 0x7ffa980d0070 2024-04-25T10:16:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa97400640 1 -- 172.21.15.87:0/2733399663 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840ab9d0 con 0x7ffa980d0070 2024-04-25T10:16:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa9fa00640 1 -- 172.21.15.87:0/2733399663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa980d0070 msgr2=0x7ffa980b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/2733399663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa980d0070 0x7ffa980b3490 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa840097c0 tx=0x7ffa8401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa9fa00640 1 -- 172.21.15.87:0/2733399663 shutdown_connections 2024-04-25T10:16:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/2733399663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa980d0070 0x7ffa980b3490 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa9fa00640 1 -- 172.21.15.87:0/2733399663 >> 172.21.15.87:0/2733399663 conn(0x7ffa98066b70 msgr2=0x7ffa98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa9fa00640 1 -- 172.21.15.87:0/2733399663 shutdown_connections 2024-04-25T10:16:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.843+0000 7ffa9fa00640 1 -- 172.21.15.87:0/2733399663 wait complete. 2024-04-25T10:16:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 Processor -- start 2024-04-25T10:16:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 -- start start 2024-04-25T10:16:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa980b39d0 0x7ffa980adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa980d0070 0x7ffa98157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa98154f70 0x7ffa98143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa98168a70 con 0x7ffa980b39d0 2024-04-25T10:16:07.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa981688f0 con 0x7ffa98154f70 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa98168bf0 con 0x7ffa980d0070 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa980b39d0 0x7ffa980adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa980b39d0 0x7ffa980adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48772/0 (socket says 172.21.15.87:48772) 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 -- 172.21.15.87:0/1631385333 learned_addr learned my addr 172.21.15.87:0/1631385333 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa97e00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa980d0070 0x7ffa98157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9da00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa98154f70 0x7ffa98143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 -- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa980d0070 msgr2=0x7ffa98157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa980d0070 0x7ffa98157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 -- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa98154f70 msgr2=0x7ffa98143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa98154f70 0x7ffa98143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 -- 172.21.15.87:0/1631385333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa98158160 con 0x7ffa980b39d0 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9d000640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa980b39d0 0x7ffa980adaa0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00c500 tx=0x7ffa8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa94c00640 1 -- 172.21.15.87:0/1631385333 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c016020 con 0x7ffa980b39d0 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa94c00640 1 -- 172.21.15.87:0/1631385333 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c005220 con 0x7ffa980b39d0 2024-04-25T10:16:07.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa94c00640 1 -- 172.21.15.87:0/1631385333 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c007030 con 0x7ffa980b39d0 2024-04-25T10:16:07.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa98158440 con 0x7ffa980b39d0 2024-04-25T10:16:07.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa98158f40 con 0x7ffa980b39d0 2024-04-25T10:16:07.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.847+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9804ef30 con 0x7ffa980b39d0 2024-04-25T10:16:07.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.851+0000 7ffa94c00640 1 -- 172.21.15.87:0/1631385333 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa8c0071f0 con 0x7ffa980b39d0 2024-04-25T10:16:07.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.851+0000 7ffa94c00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa78066f60 0x7ffa78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.851+0000 7ffa94c00640 1 -- 172.21.15.87:0/1631385333 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa8c0848f0 con 0x7ffa980b39d0 2024-04-25T10:16:07.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.851+0000 7ffa97e00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa78066f60 0x7ffa78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.851+0000 7ffa97e00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa78066f60 0x7ffa78069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffa840ad000 tx=0x7ffa84008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:07.855+0000 7ffa94c00640 1 -- 172.21.15.87:0/1631385333 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa8c020020 con 0x7ffa980b39d0 2024-04-25T10:16:08.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.059+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffa98070cf0 con 0x7ffa980b39d0 2024-04-25T10:16:08.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa94c00640 1 -- 172.21.15.87:0/1631385333 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7ffa8c0552c0 con 0x7ffa980b39d0 2024-04-25T10:16:08.068 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:16:08.069 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:16:08.069 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:16:08.069 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:16:08.069 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:08.069 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:16:08.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa78066f60 msgr2=0x7ffa78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa78066f60 0x7ffa78069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffa840ad000 tx=0x7ffa84008040 comp rx=0 tx=0).stop 2024-04-25T10:16:08.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa980b39d0 msgr2=0x7ffa980adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa980b39d0 0x7ffa980adaa0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00c500 tx=0x7ffa8c00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 shutdown_connections 2024-04-25T10:16:08.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa78066f60 0x7ffa78069420 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa98154f70 0x7ffa98143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa980d0070 0x7ffa98157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 --2- 172.21.15.87:0/1631385333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa980b39d0 0x7ffa980adaa0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 >> 172.21.15.87:0/1631385333 conn(0x7ffa98066b70 msgr2=0x7ffa980ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 shutdown_connections 2024-04-25T10:16:08.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.063+0000 7ffa9fa00640 1 -- 172.21.15.87:0/1631385333 wait complete. 2024-04-25T10:16:08.089 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:08.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd77c00640 1 Processor -- start 2024-04-25T10:16:08.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd77c00640 1 -- start start 2024-04-25T10:16:08.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd77c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd700b0f70 0x7fcd700b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd77c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7015f3b0 con 0x7fcd700cfd90 2024-04-25T10:16:08.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd77c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700ca810 con 0x7fcd700cd2e0 2024-04-25T10:16:08.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd77c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7015ef20 con 0x7fcd700b0f70 2024-04-25T10:16:08.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd75200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd700b0f70 0x7fcd700b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd6fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcd700cfd90 0x7fcd700ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60046/0 (socket says 172.21.15.87:60046) 2024-04-25T10:16:08.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd6fe00640 1 -- 172.21.15.87:0/4175091693 learned_addr learned my addr 172.21.15.87:0/4175091693 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:08.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd75200640 1 -- 172.21.15.87:0/4175091693 >> v1:172.21.15.169:6789/0 conn(0x7fcd700cd2e0 legacy=0x7fcd700cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd75200640 1 -- 172.21.15.87:0/4175091693 >> v1:172.21.15.87:6789/0 conn(0x7fcd700cfd90 legacy=0x7fcd700ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.219+0000 7fcd75200640 1 -- 172.21.15.87:0/4175091693 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700b1890 con 0x7fcd700b0f70 2024-04-25T10:16:08.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd75200640 1 --2- 172.21.15.87:0/4175091693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd700b1350 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcd600097c0 tx=0x7fcd6001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec23ad8484333e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd6f400640 1 -- 172.21.15.87:0/4175091693 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600b0070 con 0x7fcd700b0f70 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd6f400640 1 -- 172.21.15.87:0/4175091693 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd600ab600 con 0x7fcd700b0f70 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd6f400640 1 -- 172.21.15.87:0/4175091693 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600ab9d0 con 0x7fcd700b0f70 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd77c00640 1 -- 172.21.15.87:0/4175091693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 msgr2=0x7fcd700b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd77c00640 1 --2- 172.21.15.87:0/4175091693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd700b1350 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcd600097c0 tx=0x7fcd6001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd77c00640 1 -- 172.21.15.87:0/4175091693 shutdown_connections 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd77c00640 1 --2- 172.21.15.87:0/4175091693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd700b1350 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd77c00640 1 -- 172.21.15.87:0/4175091693 >> 172.21.15.87:0/4175091693 conn(0x7fcd700b1d50 msgr2=0x7fcd700b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd77c00640 1 -- 172.21.15.87:0/4175091693 shutdown_connections 2024-04-25T10:16:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.223+0000 7fcd77c00640 1 -- 172.21.15.87:0/4175091693 wait complete. 2024-04-25T10:16:08.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 Processor -- start 2024-04-25T10:16:08.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 -- start start 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd701519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd700cd2e0 0x7fcd700997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd700cfd90 0x7fcd70157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7016bf00 con 0x7fcd700b0f70 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7016bd80 con 0x7fcd700cfd90 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7016c080 con 0x7fcd700cd2e0 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd701519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd701519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48796/0 (socket says 172.21.15.87:48796) 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 -- 172.21.15.87:0/1178951172 learned_addr learned my addr 172.21.15.87:0/1178951172 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75c00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd700cfd90 0x7fcd70157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd6fe00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd700cd2e0 0x7fcd700997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 -- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd700cd2e0 msgr2=0x7fcd700997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd700cd2e0 0x7fcd700997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 -- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd700cfd90 msgr2=0x7fcd70157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd700cfd90 0x7fcd70157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 -- 172.21.15.87:0/1178951172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700c48e0 con 0x7fcd700b0f70 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd6fe00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd700cd2e0 0x7fcd700997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd75200640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd701519b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcd600190b0 tx=0x7fcd700cc940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd6cc00640 1 -- 172.21.15.87:0/1178951172 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600b0040 con 0x7fcd700b0f70 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd6cc00640 1 -- 172.21.15.87:0/1178951172 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd600ba070 con 0x7fcd700b0f70 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd700c4b80 con 0x7fcd700b0f70 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd6cc00640 1 -- 172.21.15.87:0/1178951172 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600b4830 con 0x7fcd700b0f70 2024-04-25T10:16:08.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd700c5140 con 0x7fcd700b0f70 2024-04-25T10:16:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.227+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7004ef30 con 0x7fcd700b0f70 2024-04-25T10:16:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.231+0000 7fcd6cc00640 1 -- 172.21.15.87:0/1178951172 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd600ab4c0 con 0x7fcd700b0f70 2024-04-25T10:16:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.231+0000 7fcd6cc00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd50066f60 0x7fcd50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.231+0000 7fcd6cc00640 1 -- 172.21.15.87:0/1178951172 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd6011edc0 con 0x7fcd700b0f70 2024-04-25T10:16:08.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.231+0000 7fcd6fe00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd50066f60 0x7fcd50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.231+0000 7fcd6fe00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd50066f60 0x7fcd50069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcd70074490 tx=0x7fcd5c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.235+0000 7fcd6cc00640 1 -- 172.21.15.87:0/1178951172 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd60016090 con 0x7fcd700b0f70 2024-04-25T10:16:08.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.423+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd700cda20 con 0x7fcd700b0f70 2024-04-25T10:16:08.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd6cc00640 1 -- 172.21.15.87:0/1178951172 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fcd600efd30 con 0x7fcd700b0f70 2024-04-25T10:16:08.431 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:08.432 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/243216881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":243216881},{"type":"v1","addr":"172.21.15.169:6835","nonce":243216881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6073,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/250986881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":250986881},{"type":"v1","addr":"172.21.15.169:6834","nonce":250986881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6510,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2928764815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2928764815},{"type":"v1","addr":"172.21.15.87:6836","nonce":2928764815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:05.597920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6513":{"gid":6513,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3387238920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3387238920},{"type":"v1","addr":"172.21.15.87:6837","nonce":3387238920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:08.432 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 37 2024-04-25T10:16:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd50066f60 msgr2=0x7fcd50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd50066f60 0x7fcd50069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcd70074490 tx=0x7fcd5c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 msgr2=0x7fcd701519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd701519b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcd600190b0 tx=0x7fcd700cc940 comp rx=0 tx=0).stop 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 shutdown_connections 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd50066f60 0x7fcd50069420 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd700cfd90 0x7fcd70157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd700cd2e0 0x7fcd700997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 --2- 172.21.15.87:0/1178951172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd700b0f70 0x7fcd701519b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 >> 172.21.15.87:0/1178951172 conn(0x7fcd700b1d50 msgr2=0x7fcd701600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 shutdown_connections 2024-04-25T10:16:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.427+0000 7fcd77c00640 1 -- 172.21.15.87:0/1178951172 wait complete. 2024-04-25T10:16:08.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:02.576203+0000', 'modified': '2024-04-25T10:16:05.597920+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6513}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6513': {'gid': 6513, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/3387238920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 3387238920}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 3387238920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:08.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:08.855 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:16:08.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd871400640 1 Processor -- start 2024-04-25T10:16:08.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd871400640 1 -- start start 2024-04-25T10:16:08.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd871400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd86c070c20 0x7fd86c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd871400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd86c0ae020 0x7fd86c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd871400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c0ad840 con 0x7fd86c0cf930 2024-04-25T10:16:08.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd871400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86c15bb90 con 0x7fd86c070c20 2024-04-25T10:16:08.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd871400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86c15e9f0 con 0x7fd86c0ae020 2024-04-25T10:16:08.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd869e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd86c070c20 0x7fd86c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd869e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd86c070c20 0x7fd86c15e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48798/0 (socket says 172.21.15.87:48798) 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd869e00640 1 -- 172.21.15.87:0/2845366186 learned_addr learned my addr 172.21.15.87:0/2845366186 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd869e00640 1 -- 172.21.15.87:0/2845366186 >> v2:172.21.15.169:3300/0 conn(0x7fd86c0ae020 msgr2=0x7fd86c0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd86b200640 1 --2- 172.21.15.87:0/2845366186 >> v2:172.21.15.169:3300/0 conn(0x7fd86c0ae020 0x7fd86c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd869e00640 1 --2- 172.21.15.87:0/2845366186 >> v2:172.21.15.169:3300/0 conn(0x7fd86c0ae020 0x7fd86c0ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd869e00640 1 -- 172.21.15.87:0/2845366186 >> v1:172.21.15.169:6789/0 conn(0x7fd86c0cf930 legacy=0x7fd86c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd869e00640 1 -- 172.21.15.87:0/2845366186 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c15ecd0 con 0x7fd86c070c20 2024-04-25T10:16:08.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.991+0000 7fd86b200640 1 --2- 172.21.15.87:0/2845366186 >> v2:172.21.15.169:3300/0 conn(0x7fd86c0ae020 0x7fd86c0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:09.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd869e00640 1 --2- 172.21.15.87:0/2845366186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c070c20 0x7fd86c15e170 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0b2820 tx=0x7fd85801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65575122f9fbb31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd869400640 1 -- 172.21.15.87:0/2845366186 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580aa610 con 0x7fd86c070c20 2024-04-25T10:16:09.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd869400640 1 -- 172.21.15.87:0/2845366186 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8580af070 con 0x7fd86c070c20 2024-04-25T10:16:09.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd869400640 1 -- 172.21.15.87:0/2845366186 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580aa9d0 con 0x7fd86c070c20 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 -- 172.21.15.87:0/2845366186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c070c20 msgr2=0x7fd86c15e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 --2- 172.21.15.87:0/2845366186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c070c20 0x7fd86c15e170 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0b2820 tx=0x7fd85801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 -- 172.21.15.87:0/2845366186 shutdown_connections 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 --2- 172.21.15.87:0/2845366186 >> v2:172.21.15.169:3300/0 conn(0x7fd86c0ae020 0x7fd86c0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 --2- 172.21.15.87:0/2845366186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c070c20 0x7fd86c15e170 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 -- 172.21.15.87:0/2845366186 >> 172.21.15.87:0/2845366186 conn(0x7fd86c066b70 msgr2=0x7fd86c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 -- 172.21.15.87:0/2845366186 shutdown_connections 2024-04-25T10:16:09.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 -- 172.21.15.87:0/2845366186 wait complete. 2024-04-25T10:16:09.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 Processor -- start 2024-04-25T10:16:09.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.995+0000 7fd871400640 1 -- start start 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86c070c20 0x7fd86c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0ae020 0x7fd86c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd86c0cf930 0x7fd86c154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c16a8e0 con 0x7fd86c0ae020 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c15e6b0 con 0x7fd86c0cf930 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c16aa60 con 0x7fd86c070c20 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0ae020 0x7fd86c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0ae020 0x7fd86c0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48806/0 (socket says 172.21.15.87:48806) 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 -- 172.21.15.87:0/751000562 learned_addr learned my addr 172.21.15.87:0/751000562 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd86a800640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86c070c20 0x7fd86c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd86b200640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd86c0cf930 0x7fd86c154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 -- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86c070c20 msgr2=0x7fd86c1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86c070c20 0x7fd86c1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 -- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd86c0cf930 msgr2=0x7fd86c154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd86c0cf930 0x7fd86c154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 -- 172.21.15.87:0/751000562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c0af480 con 0x7fd86c0ae020 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd869e00640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0ae020 0x7fd86c0ccb00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd85801a0e0 tx=0x7fd85801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd862a00640 1 -- 172.21.15.87:0/751000562 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580bd3f0 con 0x7fd86c0ae020 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd862a00640 1 -- 172.21.15.87:0/751000562 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8580bd590 con 0x7fd86c0ae020 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd862a00640 1 -- 172.21.15.87:0/751000562 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580bd980 con 0x7fd86c0ae020 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c0af760 con 0x7fd86c0ae020 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c0afd20 con 0x7fd86c0ae020 2024-04-25T10:16:09.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:08.999+0000 7fd862a00640 1 -- 172.21.15.87:0/751000562 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8580b33f0 con 0x7fd86c0ae020 2024-04-25T10:16:09.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd862a00640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd84c066f90 0x7fd84c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd862a00640 1 -- 172.21.15.87:0/751000562 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd85811f350 con 0x7fd86c0ae020 2024-04-25T10:16:09.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86a800640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd84c066f90 0x7fd84c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 Processor -- start 2024-04-25T10:16:09.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 -- start start 2024-04-25T10:16:09.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86a800640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd84c066f90 0x7fd84c069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd85c004e30 tx=0x7fd85c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af84123920 0x55af84123d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840d99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af840d9f30 0x55af840da310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af83fc15f0 con 0x55af840d9f30 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af83f28da0 con 0x55af84123920 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af83eb1150 con 0x55af84124240 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af840d9f30 0x55af840da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af840d9f30 0x55af840da310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48808/0 (socket says 172.21.15.87:48808) 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 -- 172.21.15.87:0/2744584188 learned_addr learned my addr 172.21.15.87:0/2744584188 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd869e00640 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840d99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86a800640 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af84123920 0x55af84123d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 -- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 msgr2=0x55af840d99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840d99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 -- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af84123920 msgr2=0x55af84123d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af84123920 0x55af84123d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 -- 172.21.15.87:0/2744584188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af83ecaca0 con 0x55af840d9f30 2024-04-25T10:16:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd86b200640 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af840d9f30 0x55af840da310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd86400e4c0 tx=0x7fd86400e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ccaf838940b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd857400640 1 -- 172.21.15.87:0/2744584188 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400c9d0 con 0x55af840d9f30 2024-04-25T10:16:09.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd857400640 1 -- 172.21.15.87:0/2744584188 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86400cb70 con 0x55af840d9f30 2024-04-25T10:16:09.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd857400640 1 -- 172.21.15.87:0/2744584188 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400a410 con 0x55af840d9f30 2024-04-25T10:16:09.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 -- 172.21.15.87:0/2744584188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af840d9f30 msgr2=0x55af840da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.003+0000 7fd874928000 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af840d9f30 0x55af840da310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd86400e4c0 tx=0x7fd86400e980 comp rx=0 tx=0).stop 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- 172.21.15.87:0/2744584188 shutdown_connections 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af840d9f30 0x55af840da310 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840d99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 --2- 172.21.15.87:0/2744584188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af84123920 0x55af84123d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- 172.21.15.87:0/2744584188 >> 172.21.15.87:0/2744584188 conn(0x55af840e02a0 msgr2=0x55af840d82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- 172.21.15.87:0/2744584188 shutdown_connections 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- 172.21.15.87:0/2744584188 wait complete. 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- start start 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 Processor -- start 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af8412add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af83ee3990 con 0x55af84123920 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af83f24e40 con 0x55af840d9f30 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af83eb4900 con 0x55af84124240 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 10 client.0 _ll_register_callbacks cb 0x55af83fdc1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48824/0 (socket says 172.21.15.87:48824) 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 -- 172.21.15.87:0/1380214287 learned_addr learned my addr 172.21.15.87:0/1380214287 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd86b200640 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd86a800640 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af8412add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 -- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 msgr2=0x55af840ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 -- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 msgr2=0x55af8412add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af8412add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 -- 172.21.15.87:0/1380214287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af83ee1630 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd869e00640 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412b310 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd8580ec020 tx=0x7fd8580ef7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7877af94119b002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd84be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd84be00640 1 -- 172.21.15.87:0/1380214287 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580f3d10 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd84be00640 1 -- 172.21.15.87:0/1380214287 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8580f7e40 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd86400cb70 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd84be00640 1 -- 172.21.15.87:0/1380214287 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8580f82d0 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 10 client.6657 fetch_fsmap learned FSMap version 37 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55af83eb13a0 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd84be00640 1 -- 172.21.15.87:0/1380214287 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86400cb70 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd84be00640 1 -- 172.21.15.87:0/1380214287 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd858103b80 con 0x55af84123920 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 10 client.6657 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T10:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 10 client.6657 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:09.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 20 client.6657 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:09.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af840df160 0x55af84134e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 4 client.6657 mds_command: new command op to 6513 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:09.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 --> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af83ec4a50 con 0x55af840df160 2024-04-25T10:16:09.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.007+0000 7fd86a800640 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af840df160 0x55af84134e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd84be00640 10 client.6657 ms_handle_connect on v2:172.21.15.87:6835/3387238920 2024-04-25T10:16:09.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd86a800640 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af840df160 0x55af84134e30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0111b36a5e1830b server_cookie=9c367868075e71d7 in_seq=0 out_seq=0 2024-04-25T10:16:09.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd84be00640 1 -- 172.21.15.87:0/1380214287 <== mds.0 v2:172.21.15.87:6835/3387238920 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af83ec4a50 con 0x55af840df160 2024-04-25T10:16:09.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd84be00640 10 client.6657 handle_command_reply: tid=0 2024-04-25T10:16:09.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 client.6657 shutdown 2024-04-25T10:16:09.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 msgr2=0x55af8412b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412b310 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd8580ec020 tx=0x7fd8580ef7c0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 shutdown_connections 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af840df160 0x55af84134e30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af84124240 0x55af840ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412b310 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd84be00640 0 client.6657 ms_handle_reset on v2:172.21.15.87:6835/3387238920 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 --2- 172.21.15.87:0/1380214287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af8412add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 >> 172.21.15.87:0/1380214287 conn(0x55af840e02a0 msgr2=0x55af840d89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 shutdown_connections 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 1 -- 172.21.15.87:0/1380214287 wait complete. 2024-04-25T10:16:09.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.011+0000 7fd874928000 20 client.6657 trim_cache size 0 max 16384 2024-04-25T10:16:09.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 Processor -- start 2024-04-25T10:16:09.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- start start 2024-04-25T10:16:09.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840e02a0 0x55af840dd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840d9f30 0x55af840dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c1590b0 con 0x55af84123920 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c1584b0 con 0x55af840e02a0 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c158ab0 con 0x55af840d9f30 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48836/0 (socket says 172.21.15.87:48836) 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 -- 172.21.15.87:0/3731437108 learned_addr learned my addr 172.21.15.87:0/3731437108 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd869e00640 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840d9f30 0x55af840dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86a800640 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840e02a0 0x55af840dd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 -- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840d9f30 msgr2=0x55af840dd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840d9f30 0x55af840dd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 -- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840e02a0 msgr2=0x55af840dd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840e02a0 0x55af840dd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 -- 172.21.15.87:0/3731437108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af83faa600 con 0x55af84123920 2024-04-25T10:16:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd86b200640 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412da00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd864007fa0 tx=0x7fd864011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0da69af634e1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd856a00640 1 -- 172.21.15.87:0/3731437108 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640039a0 con 0x55af84123920 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd856a00640 1 -- 172.21.15.87:0/3731437108 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864003b40 con 0x55af84123920 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd856a00640 1 -- 172.21.15.87:0/3731437108 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400c7d0 con 0x55af84123920 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- 172.21.15.87:0/3731437108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 msgr2=0x55af8412da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412da00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd864007fa0 tx=0x7fd864011000 comp rx=0 tx=0).stop 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- 172.21.15.87:0/3731437108 shutdown_connections 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af8412da00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840d9f30 0x55af840dd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 --2- 172.21.15.87:0/3731437108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840e02a0 0x55af840dd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- 172.21.15.87:0/3731437108 >> 172.21.15.87:0/3731437108 conn(0x55af8412b190 msgr2=0x55af84110170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- 172.21.15.87:0/3731437108 shutdown_connections 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.027+0000 7fd874928000 1 -- 172.21.15.87:0/3731437108 wait complete. 2024-04-25T10:16:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 -- start start 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 Processor -- start 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af84130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840e02a0 0x55af841671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af84167700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af83f2d3d0 con 0x55af84123920 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af83fe4d50 con 0x55af840d9f30 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af83ec3980 con 0x55af840e02a0 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 10 client.0 _ll_register_callbacks cb 0x7fd864007810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af84167700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af84167700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48842/0 (socket says 172.21.15.87:48842) 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 -- 172.21.15.87:0/3666207849 learned_addr learned my addr 172.21.15.87:0/3666207849 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd869e00640 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840e02a0 0x55af841671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86a800640 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af84130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 -- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840e02a0 msgr2=0x55af841671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840e02a0 0x55af841671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 -- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 msgr2=0x55af84130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af84130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 -- 172.21.15.87:0/3666207849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af83ebd9b0 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86b200640 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af84167700 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd864010010 tx=0x7fd864004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e220ecda36c121c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd84be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd84be00640 1 -- 172.21.15.87:0/3666207849 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640039a0 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd84be00640 1 -- 172.21.15.87:0/3666207849 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd864021650 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd864003b40 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd84be00640 1 -- 172.21.15.87:0/3666207849 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd864039050 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 10 client.6663 fetch_fsmap learned FSMap version 37 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55af83ed00b0 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd84be00640 1 -- 172.21.15.87:0/3666207849 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864003b40 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd84be00640 1 -- 172.21.15.87:0/3666207849 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd86403c020 con 0x55af84123920 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 10 client.6663 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 10 client.6663 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 20 client.6663 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af84124240 0x55af84167f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 4 client.6663 mds_command: new command op to 6513 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 --> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af83f362e0 con 0x55af84124240 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.031+0000 7fd86a800640 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af84124240 0x55af84167f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd86a800640 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af84124240 0x55af84167f60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7942663c07b3fb server_cookie=9df5d03e72a09b23 in_seq=0 out_seq=0 2024-04-25T10:16:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd84be00640 10 client.6663 ms_handle_connect on v2:172.21.15.87:6835/3387238920 2024-04-25T10:16:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd84be00640 1 -- 172.21.15.87:0/3666207849 <== mds.0 v2:172.21.15.87:6835/3387238920 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55af83f362e0 con 0x55af84124240 2024-04-25T10:16:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd84be00640 10 client.6663 handle_command_reply: tid=0 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 client.6663 shutdown 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 msgr2=0x55af84167700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af84167700 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd864010010 tx=0x7fd864004ac0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 shutdown_connections 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:6835/3387238920,v1:172.21.15.87:6837/3387238920] conn(0x55af84124240 0x55af84167f60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af84123920 0x55af84167700 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af840e02a0 0x55af841671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd84be00640 0 client.6663 ms_handle_reset on v2:172.21.15.87:6835/3387238920 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 --2- 172.21.15.87:0/3666207849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af840d9f30 0x55af84130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 >> 172.21.15.87:0/3666207849 conn(0x55af8412b190 msgr2=0x55af840dc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 shutdown_connections 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 1 -- 172.21.15.87:0/3666207849 wait complete. 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd874928000 20 client.6663 trim_cache size 0 max 16384 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 6513, 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.831387082, 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 37, 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 57, 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 57, 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.758570623000001 2024-04-25T10:16:09.046 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:16:09.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd84c066f90 msgr2=0x7fd84c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd84c066f90 0x7fd84c069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd85c004e30 tx=0x7fd85c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:09.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0ae020 msgr2=0x7fd86c0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0ae020 0x7fd86c0ccb00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd85801a0e0 tx=0x7fd85801acf0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 shutdown_connections 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd84c066f90 0x7fd84c069450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd86c0cf930 0x7fd86c154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0ae020 0x7fd86c0ccb00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 --2- 172.21.15.87:0/751000562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86c070c20 0x7fd86c1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 >> 172.21.15.87:0/751000562 conn(0x7fd86c066b70 msgr2=0x7fd86c0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 shutdown_connections 2024-04-25T10:16:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.035+0000 7fd871400640 1 -- 172.21.15.87:0/751000562 wait complete. 2024-04-25T10:16:09.059 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:09.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f867ae00640 1 Processor -- start 2024-04-25T10:16:09.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f867ae00640 1 -- start start 2024-04-25T10:16:09.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f867ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86740cda10 0x7f867406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f867ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86740cf930 0x7f86740cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f867ae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b2770 con 0x7f867415bc40 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f867ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867406c670 con 0x7f86740cda10 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f867ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867406c950 con 0x7f86740cf930 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8673e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f867415bc40 0x7f86740b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60050/0 (socket says 172.21.15.87:60050) 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8673e00640 1 -- 172.21.15.87:0/1380858098 learned_addr learned my addr 172.21.15.87:0/1380858098 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8673400640 1 --2- 172.21.15.87:0/1380858098 >> v2:172.21.15.169:3300/0 conn(0x7f86740cda10 0x7f867406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8678e00640 1 --2- 172.21.15.87:0/1380858098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86740cf930 0x7f86740cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8672a00640 1 -- 172.21.15.87:0/1380858098 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586900545 0 0) 0x7f86740b2770 con 0x7f867415bc40 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8672a00640 1 -- 172.21.15.87:0/1380858098 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8658003530 con 0x7f867415bc40 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8678e00640 1 -- 172.21.15.87:0/1380858098 >> v2:172.21.15.169:3300/0 conn(0x7f86740cda10 msgr2=0x7f867406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8678e00640 1 --2- 172.21.15.87:0/1380858098 >> v2:172.21.15.169:3300/0 conn(0x7f86740cda10 0x7f867406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8678e00640 1 -- 172.21.15.87:0/1380858098 >> v1:172.21.15.87:6789/0 conn(0x7f867415bc40 legacy=0x7f86740b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8678e00640 1 -- 172.21.15.87:0/1380858098 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867406cc30 con 0x7f86740cf930 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.191+0000 7f8673400640 1 --2- 172.21.15.87:0/1380858098 >> v2:172.21.15.169:3300/0 conn(0x7f86740cda10 0x7f867406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f8678e00640 1 --2- 172.21.15.87:0/1380858098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86740cf930 0x7f86740cfde0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f866c0097c0 tx=0x7f866c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=590e632c1ae990aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f8672a00640 1 -- 172.21.15.87:0/1380858098 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0b0070 con 0x7f86740cf930 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f8672a00640 1 -- 172.21.15.87:0/1380858098 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c0ab600 con 0x7f86740cf930 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f8672a00640 1 -- 172.21.15.87:0/1380858098 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0ab9d0 con 0x7f86740cf930 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 -- 172.21.15.87:0/1380858098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86740cf930 msgr2=0x7f86740cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 --2- 172.21.15.87:0/1380858098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86740cf930 0x7f86740cfde0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f866c0097c0 tx=0x7f866c01c410 comp rx=0 tx=0).stop 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 -- 172.21.15.87:0/1380858098 shutdown_connections 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 --2- 172.21.15.87:0/1380858098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86740cf930 0x7f86740cfde0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 --2- 172.21.15.87:0/1380858098 >> v2:172.21.15.169:3300/0 conn(0x7f86740cda10 0x7f867406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 -- 172.21.15.87:0/1380858098 >> 172.21.15.87:0/1380858098 conn(0x7f8674066b70 msgr2=0x7f8674066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 -- 172.21.15.87:0/1380858098 shutdown_connections 2024-04-25T10:16:09.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.195+0000 7f867ae00640 1 -- 172.21.15.87:0/1380858098 wait complete. 2024-04-25T10:16:09.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 Processor -- start 2024-04-25T10:16:09.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 -- start start 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86740cda10 0x7f86741659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86740cf930 0x7f86740b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867415bc40 0x7f86741f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b3890 con 0x7f86740cda10 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b3710 con 0x7f86740cf930 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b3a10 con 0x7f867415bc40 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86740cda10 0x7f86741659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86740cda10 0x7f86741659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48856/0 (socket says 172.21.15.87:48856) 2024-04-25T10:16:09.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 -- 172.21.15.87:0/2584944002 learned_addr learned my addr 172.21.15.87:0/2584944002 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8678e00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867415bc40 0x7f86741f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673400640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86740cf930 0x7f86740b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 -- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867415bc40 msgr2=0x7f86741f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867415bc40 0x7f86741f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 -- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86740cf930 msgr2=0x7f86740b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86740cf930 0x7f86740b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 -- 172.21.15.87:0/2584944002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8674166080 con 0x7f86740cda10 2024-04-25T10:16:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f8673e00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86740cda10 0x7f86741659c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f866400caa0 tx=0x7f866400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f866be00640 1 -- 172.21.15.87:0/2584944002 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664017070 con 0x7f86740cda10 2024-04-25T10:16:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f866be00640 1 -- 172.21.15.87:0/2584944002 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866400ac00 con 0x7f86740cda10 2024-04-25T10:16:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f866be00640 1 -- 172.21.15.87:0/2584944002 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664012400 con 0x7f86740cda10 2024-04-25T10:16:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8674166360 con 0x7f86740cda10 2024-04-25T10:16:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.199+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8674156140 con 0x7f86740cda10 2024-04-25T10:16:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.203+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867404ef30 con 0x7f86740cda10 2024-04-25T10:16:09.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.203+0000 7f866be00640 1 -- 172.21.15.87:0/2584944002 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f866401a070 con 0x7f86740cda10 2024-04-25T10:16:09.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.203+0000 7f866be00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8654066f60 0x7f8654069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.203+0000 7f866be00640 1 -- 172.21.15.87:0/2584944002 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8664084df0 con 0x7f86740cda10 2024-04-25T10:16:09.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.203+0000 7f8673400640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8654066f60 0x7f8654069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.203+0000 7f8673400640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8654066f60 0x7f8654069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f86740b4310 tx=0x7f8660008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.207+0000 7f866be00640 1 -- 172.21.15.87:0/2584944002 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8664055df0 con 0x7f86740cda10 2024-04-25T10:16:09.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8674166530 con 0x7f86740cda10 2024-04-25T10:16:09.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f866be00640 1 -- 172.21.15.87:0/2584944002 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f86640556f0 con 0x7f86740cda10 2024-04-25T10:16:09.405 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8654066f60 msgr2=0x7f8654069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8654066f60 0x7f8654069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f86740b4310 tx=0x7f8660008040 comp rx=0 tx=0).stop 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86740cda10 msgr2=0x7f86741659c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86740cda10 0x7f86741659c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f866400caa0 tx=0x7f866400cf60 comp rx=0 tx=0).stop 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 shutdown_connections 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8654066f60 0x7f8654069420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867415bc40 0x7f86741f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86740cf930 0x7f86740b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 --2- 172.21.15.87:0/2584944002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86740cda10 0x7f86741659c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 >> 172.21.15.87:0/2584944002 conn(0x7f8674066b70 msgr2=0x7f8674066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 shutdown_connections 2024-04-25T10:16:09.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:09.399+0000 7f867ae00640 1 -- 172.21.15.87:0/2584944002 wait complete. 2024-04-25T10:16:09.426 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:16:09.426 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:09.426 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:16:09.426 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:09.426 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:09.426 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:16:09.426 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:16:09.427 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:09.427 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:09.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.427 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:16:09.435 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:09.436 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42244sec preferred_lft 42244sec 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.437 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:09.438 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:09.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.439 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:09.504 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:16:09.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.507 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:16:09.562 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:16:09.562 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:09.562 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:16:09.562 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:16:09.562 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:16:09.563 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:09.563 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:16:09.563 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:09.563 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:09.563 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:09.563 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:09.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.563 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:16:09.563 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:16:09.563 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:16:09.564 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:16:09.564 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:16:09.564 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:16:09.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.651 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:09.703 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:16:09.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:09.766 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:09.815 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:09.815 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:09.862 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:16:09.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.863 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:09.955 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:09.955 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:09.955 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:09.970 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:09.970 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:09.970 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:09.970 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:16:09.970 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:16:09.982 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:16:09.982 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:16:09.985 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:16:09.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.985 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:10.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:10.050 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:16:10.106 INFO:teuthology.orchestra.run.smithi087.stdout:client6196 2024-04-25T10:16:10.106 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:10.106 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:10.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:10.155 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T10:16:10.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:10.213 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T10:16:10.270 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:16:10.270 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:16:10.270 DEBUG:teuthology.orchestra.run.smithi087:> import stat 2024-04-25T10:16:10.271 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:16:10.271 DEBUG:teuthology.orchestra.run.smithi087:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T10:16:10.271 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:16:10.373 INFO:teuthology.orchestra.run.smithi087.stdout:1099511627776 2024-04-25T10:16:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:10.383 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:10.383 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:10.435 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:10.435 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:10.491 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:10.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa520e00640 1 Processor -- start 2024-04-25T10:16:10.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa520e00640 1 -- start start 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa520e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa51c1580d0 0x7fa51c06df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa520e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c161a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa520e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c15e4f0 con 0x7fa51c0af1b0 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa520e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c15f100 con 0x7fa51c1580d0 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa520e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c06e4a0 con 0x7fa51c0d04b0 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa513e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa51c0af1b0 0x7fa51c0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60052/0 (socket says 172.21.15.87:60052) 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa513e00640 1 -- 172.21.15.87:0/393831323 learned_addr learned my addr 172.21.15.87:0/393831323 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51b400640 1 --2- 172.21.15.87:0/393831323 >> v2:172.21.15.169:3300/0 conn(0x7fa51c1580d0 0x7fa51c06df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51be00640 1 --2- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51aa00640 1 -- 172.21.15.87:0/393831323 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446038775 0 0) 0x7fa51c15e4f0 con 0x7fa51c0af1b0 2024-04-25T10:16:10.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51aa00640 1 -- 172.21.15.87:0/393831323 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa500003530 con 0x7fa51c0af1b0 2024-04-25T10:16:10.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51b400640 1 -- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 msgr2=0x7fa51c161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51b400640 1 --2- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51b400640 1 -- 172.21.15.87:0/393831323 >> v1:172.21.15.87:6789/0 conn(0x7fa51c0af1b0 legacy=0x7fa51c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.659+0000 7fa51b400640 1 -- 172.21.15.87:0/393831323 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c15e7d0 con 0x7fa51c1580d0 2024-04-25T10:16:10.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa51b400640 1 --2- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c1580d0 0x7fa51c06df60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0097c0 tx=0x7fa50c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce60b6e525caefd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa51aa00640 1 -- 172.21.15.87:0/393831323 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0b0070 con 0x7fa51c1580d0 2024-04-25T10:16:10.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa51aa00640 1 -- 172.21.15.87:0/393831323 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c0ab600 con 0x7fa51c1580d0 2024-04-25T10:16:10.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa51aa00640 1 -- 172.21.15.87:0/393831323 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0ab9d0 con 0x7fa51c1580d0 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 -- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c1580d0 msgr2=0x7fa51c06df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 --2- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c1580d0 0x7fa51c06df60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0097c0 tx=0x7fa50c01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 -- 172.21.15.87:0/393831323 shutdown_connections 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 --2- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 --2- 172.21.15.87:0/393831323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c1580d0 0x7fa51c06df60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 -- 172.21.15.87:0/393831323 >> 172.21.15.87:0/393831323 conn(0x7fa51c06b4e0 msgr2=0x7fa51c0b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 -- 172.21.15.87:0/393831323 shutdown_connections 2024-04-25T10:16:10.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.663+0000 7fa520e00640 1 -- 172.21.15.87:0/393831323 wait complete. 2024-04-25T10:16:10.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 Processor -- start 2024-04-25T10:16:10.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 -- start start 2024-04-25T10:16:10.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c0af1b0 0x7fa51c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c13f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa51c1580d0 0x7fa51c13c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c15e260 con 0x7fa51c1580d0 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c15e0e0 con 0x7fa51c0af1b0 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c16aa70 con 0x7fa51c0d04b0 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa51c1580d0 0x7fa51c13c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa51c1580d0 0x7fa51c13c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48868/0 (socket says 172.21.15.87:48868) 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 -- 172.21.15.87:0/2040766346 learned_addr learned my addr 172.21.15.87:0/2040766346 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa513e00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c13f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51b400640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c0af1b0 0x7fa51c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 -- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 msgr2=0x7fa51c13f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c13f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 -- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c0af1b0 msgr2=0x7fa51c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c0af1b0 0x7fa51c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 -- 172.21.15.87:0/2040766346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c13fc10 con 0x7fa51c1580d0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa51be00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa51c1580d0 0x7fa51c13c3a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa51400ef90 tx=0x7fa51400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa513400640 1 -- 172.21.15.87:0/2040766346 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514018020 con 0x7fa51c1580d0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa513400640 1 -- 172.21.15.87:0/2040766346 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa514014480 con 0x7fa51c1580d0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c0ceb80 con 0x7fa51c1580d0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa513400640 1 -- 172.21.15.87:0/2040766346 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514014870 con 0x7fa51c1580d0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.667+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c0cf140 con 0x7fa51c1580d0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.671+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c0cf800 con 0x7fa51c1580d0 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.671+0000 7fa513400640 1 -- 172.21.15.87:0/2040766346 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa51401b020 con 0x7fa51c1580d0 2024-04-25T10:16:10.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.671+0000 7fa513400640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4f8066f60 0x7fa4f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.671+0000 7fa513400640 1 -- 172.21.15.87:0/2040766346 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa514085d00 con 0x7fa51c1580d0 2024-04-25T10:16:10.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.671+0000 7fa51b400640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4f8066f60 0x7fa4f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.671+0000 7fa51b400640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4f8066f60 0x7fa4f8069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0ad000 tx=0x7fa50c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.675+0000 7fa513400640 1 -- 172.21.15.87:0/2040766346 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa51c0cf800 con 0x7fa51c1580d0 2024-04-25T10:16:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.831+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa51c06df60 con 0x7fa51c1580d0 2024-04-25T10:16:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.831+0000 7fa513400640 1 -- 172.21.15.87:0/2040766346 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16653 (secure 0 0 0) 0x7fa514022030 con 0x7fa51c1580d0 2024-04-25T10:16:10.838 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:10.838 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":57,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:04.204866+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4f8066f60 msgr2=0x7fa4f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4f8066f60 0x7fa4f8069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0ad000 tx=0x7fa50c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa51c1580d0 msgr2=0x7fa51c13c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa51c1580d0 0x7fa51c13c3a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa51400ef90 tx=0x7fa51400c490 comp rx=0 tx=0).stop 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 shutdown_connections 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4f8066f60 0x7fa4f8069420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa51c1580d0 0x7fa51c13c3a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa51c0d04b0 0x7fa51c13f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 --2- 172.21.15.87:0/2040766346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa51c0af1b0 0x7fa51c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 >> 172.21.15.87:0/2040766346 conn(0x7fa51c06b4e0 msgr2=0x7fa51c0cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 shutdown_connections 2024-04-25T10:16:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.835+0000 7fa520e00640 1 -- 172.21.15.87:0/2040766346 wait complete. 2024-04-25T10:16:10.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:00.962606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:01.567628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:10.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.991+0000 7f5365200640 1 Processor -- start 2024-04-25T10:16:10.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.991+0000 7f5365200640 1 -- start start 2024-04-25T10:16:11.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5365200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5360070350 0x7f536015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5365200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536015b080 con 0x7f53600aef80 2024-04-25T10:16:11.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5365200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536015adb0 con 0x7f53600b31c0 2024-04-25T10:16:11.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5365200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536015eb60 con 0x7f5360070350 2024-04-25T10:16:11.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f535de00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f53600aef80 0x7f536015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60056/0 (socket says 172.21.15.87:60056) 2024-04-25T10:16:11.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f535de00640 1 -- 172.21.15.87:0/1150643367 learned_addr learned my addr 172.21.15.87:0/1150643367 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:11.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5355400640 1 --2- 172.21.15.87:0/1150643367 >> v2:172.21.15.169:3300/0 conn(0x7f5360070350 0x7f536015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f535d400640 1 -- 172.21.15.87:0/1150643367 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727366033 0 0) 0x7f536015b080 con 0x7f53600aef80 2024-04-25T10:16:11.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f535d400640 1 -- 172.21.15.87:0/1150643367 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5344005740 con 0x7f53600aef80 2024-04-25T10:16:11.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5355400640 1 -- 172.21.15.87:0/1150643367 >> v1:172.21.15.169:6789/0 conn(0x7f53600b31c0 legacy=0x7f53600cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5355400640 1 -- 172.21.15.87:0/1150643367 >> v1:172.21.15.87:6789/0 conn(0x7f53600aef80 legacy=0x7f536015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.995+0000 7f5355400640 1 -- 172.21.15.87:0/1150643367 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536015ee40 con 0x7f5360070350 2024-04-25T10:16:11.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5355400640 1 --2- 172.21.15.87:0/1150643367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5360070350 0x7f536015e480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f53480097c0 tx=0x7f534801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601db2457ad081c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f535d400640 1 -- 172.21.15.87:0/1150643367 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53480b2070 con 0x7f5360070350 2024-04-25T10:16:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f535d400640 1 -- 172.21.15.87:0/1150643367 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53480ad680 con 0x7f5360070350 2024-04-25T10:16:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f535d400640 1 -- 172.21.15.87:0/1150643367 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53480ada50 con 0x7f5360070350 2024-04-25T10:16:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 -- 172.21.15.87:0/1150643367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5360070350 msgr2=0x7f536015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 --2- 172.21.15.87:0/1150643367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5360070350 0x7f536015e480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f53480097c0 tx=0x7f534801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 -- 172.21.15.87:0/1150643367 shutdown_connections 2024-04-25T10:16:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 --2- 172.21.15.87:0/1150643367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5360070350 0x7f536015e480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 -- 172.21.15.87:0/1150643367 >> 172.21.15.87:0/1150643367 conn(0x7f53600669a0 msgr2=0x7f5360066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 -- 172.21.15.87:0/1150643367 shutdown_connections 2024-04-25T10:16:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 -- 172.21.15.87:0/1150643367 wait complete. 2024-04-25T10:16:11.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 Processor -- start 2024-04-25T10:16:11.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:10.999+0000 7f5365200640 1 -- start start 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5360070350 0x7f53601553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53600aef80 0x7f5360159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53600b31c0 0x7f536015fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600cfef0 con 0x7f5360070350 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600cfd70 con 0x7f53600b31c0 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53600b35d0 con 0x7f53600aef80 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5360070350 0x7f53601553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5360070350 0x7f53601553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48884/0 (socket says 172.21.15.87:48884) 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 -- 172.21.15.87:0/3167835500 learned_addr learned my addr 172.21.15.87:0/3167835500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5355400640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53600aef80 0x7f5360159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535e800640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53600b31c0 0x7f536015fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 -- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53600aef80 msgr2=0x7f5360159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53600aef80 0x7f5360159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 -- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53600b31c0 msgr2=0x7f536015fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53600b31c0 0x7f536015fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 -- 172.21.15.87:0/3167835500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360155930 con 0x7f5360070350 2024-04-25T10:16:11.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f535de00640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5360070350 0x7f53601553f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f535000c4d0 tx=0x7f535000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5356a00640 1 -- 172.21.15.87:0/3167835500 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350017070 con 0x7f5360070350 2024-04-25T10:16:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5356a00640 1 -- 172.21.15.87:0/3167835500 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350005cf0 con 0x7f5360070350 2024-04-25T10:16:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5356a00640 1 -- 172.21.15.87:0/3167835500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350007270 con 0x7f5360070350 2024-04-25T10:16:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53601f6990 con 0x7f5360070350 2024-04-25T10:16:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5360156f30 con 0x7f5360070350 2024-04-25T10:16:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536004ef30 con 0x7f5360070350 2024-04-25T10:16:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.003+0000 7f5356a00640 1 -- 172.21.15.87:0/3167835500 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f535001a070 con 0x7f5360070350 2024-04-25T10:16:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.007+0000 7f5356a00640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f533c066f60 0x7f533c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.007+0000 7f5356a00640 1 -- 172.21.15.87:0/3167835500 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5350084420 con 0x7f5360070350 2024-04-25T10:16:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.007+0000 7f5355400640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f533c066f60 0x7f533c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.007+0000 7f5355400640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f533c066f60 0x7f533c069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f53480af000 tx=0x7f53480a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.011+0000 7f5356a00640 1 -- 172.21.15.87:0/3167835500 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5360155930 con 0x7f5360070350 2024-04-25T10:16:11.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.163+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53600b2fa0 con 0x7f5360070350 2024-04-25T10:16:11.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.163+0000 7f5356a00640 1 -- 172.21.15.87:0/3167835500 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5350054d40 con 0x7f5360070350 2024-04-25T10:16:11.171 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:16:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f533c066f60 msgr2=0x7f533c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f533c066f60 0x7f533c069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f53480af000 tx=0x7f53480a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5360070350 msgr2=0x7f53601553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5360070350 0x7f53601553f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f535000c4d0 tx=0x7f535000c990 comp rx=0 tx=0).stop 2024-04-25T10:16:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 shutdown_connections 2024-04-25T10:16:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f533c066f60 0x7f533c069420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53600b31c0 0x7f536015fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53600aef80 0x7f5360159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 --2- 172.21.15.87:0/3167835500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5360070350 0x7f53601553f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 >> 172.21.15.87:0/3167835500 conn(0x7f53600669a0 msgr2=0x7f5360068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 shutdown_connections 2024-04-25T10:16:11.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.167+0000 7f5365200640 1 -- 172.21.15.87:0/3167835500 wait complete. 2024-04-25T10:16:11.192 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:11.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29f000640 1 Processor -- start 2024-04-25T10:16:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29f000640 1 -- start start 2024-04-25T10:16:11.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29f000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff2980b3480 0x7ff29815e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2980cfb50 0x7ff298070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29f000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2980ad840 con 0x7ff298071bc0 2024-04-25T10:16:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29f000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29815bb90 con 0x7ff2980b3480 2024-04-25T10:16:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29815e710 con 0x7ff2980cfb50 2024-04-25T10:16:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff297e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff2980b3480 0x7ff29815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29d000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff298071bc0 0x7ff2980add20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60062/0 (socket says 172.21.15.87:60062) 2024-04-25T10:16:11.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff29d000640 1 -- 172.21.15.87:0/3317819096 learned_addr learned my addr 172.21.15.87:0/3317819096 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff297400640 1 --2- 172.21.15.87:0/3317819096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2980cfb50 0x7ff298070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff297e00640 1 -- 172.21.15.87:0/3317819096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2980cfb50 msgr2=0x7ff298070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff297e00640 1 --2- 172.21.15.87:0/3317819096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2980cfb50 0x7ff298070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff297e00640 1 -- 172.21.15.87:0/3317819096 >> v1:172.21.15.87:6789/0 conn(0x7ff298071bc0 legacy=0x7ff2980add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff297e00640 1 -- 172.21.15.87:0/3317819096 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29815e9f0 con 0x7ff2980b3480 2024-04-25T10:16:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.331+0000 7ff296a00640 1 -- 172.21.15.87:0/3317819096 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877118204 0 0) 0x7ff2980ad840 con 0x7ff298071bc0 2024-04-25T10:16:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff297e00640 1 --2- 172.21.15.87:0/3317819096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff29815e130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff2880097c0 tx=0x7ff28801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307c948a44068492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff296a00640 1 -- 172.21.15.87:0/3317819096 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880b0070 con 0x7ff2980b3480 2024-04-25T10:16:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff296a00640 1 -- 172.21.15.87:0/3317819096 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2880ab600 con 0x7ff2980b3480 2024-04-25T10:16:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff296a00640 1 -- 172.21.15.87:0/3317819096 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880ab9d0 con 0x7ff2980b3480 2024-04-25T10:16:11.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 -- 172.21.15.87:0/3317819096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 msgr2=0x7ff29815e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 --2- 172.21.15.87:0/3317819096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff29815e130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff2880097c0 tx=0x7ff28801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 -- 172.21.15.87:0/3317819096 shutdown_connections 2024-04-25T10:16:11.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 --2- 172.21.15.87:0/3317819096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2980cfb50 0x7ff298070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 --2- 172.21.15.87:0/3317819096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff29815e130 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 -- 172.21.15.87:0/3317819096 >> 172.21.15.87:0/3317819096 conn(0x7ff2980669a0 msgr2=0x7ff298066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 -- 172.21.15.87:0/3317819096 shutdown_connections 2024-04-25T10:16:11.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 -- 172.21.15.87:0/3317819096 wait complete. 2024-04-25T10:16:11.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.335+0000 7ff29f000640 1 Processor -- start 2024-04-25T10:16:11.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 -- start start 2024-04-25T10:16:11.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff298071bc0 0x7ff2981431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff298162850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2980cfb50 0x7ff29813fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980cff30 con 0x7ff2980b3480 2024-04-25T10:16:11.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980ae840 con 0x7ff2980cfb50 2024-04-25T10:16:11.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980d00b0 con 0x7ff298071bc0 2024-04-25T10:16:11.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff298162850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff298162850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48908/0 (socket says 172.21.15.87:48908) 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 -- 172.21.15.87:0/4237138359 learned_addr learned my addr 172.21.15.87:0/4237138359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297e00640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff298071bc0 0x7ff2981431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29d000640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2980cfb50 0x7ff29813fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 -- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff298071bc0 msgr2=0x7ff2981431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff298071bc0 0x7ff2981431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 -- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2980cfb50 msgr2=0x7ff29813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2980cfb50 0x7ff29813fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 -- 172.21.15.87:0/4237138359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298156f00 con 0x7ff2980b3480 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff297400640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff298162850 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff28400d800 tx=0x7ff28400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff28fe00640 1 -- 172.21.15.87:0/4237138359 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28400b6c0 con 0x7ff2980b3480 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff28fe00640 1 -- 172.21.15.87:0/4237138359 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284019070 con 0x7ff2980b3480 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff28fe00640 1 -- 172.21.15.87:0/4237138359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28400ba60 con 0x7ff2980b3480 2024-04-25T10:16:11.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff298156940 con 0x7ff2980b3480 2024-04-25T10:16:11.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2981547e0 con 0x7ff2980b3480 2024-04-25T10:16:11.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.339+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29804ef30 con 0x7ff2980b3480 2024-04-25T10:16:11.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.343+0000 7ff28fe00640 1 -- 172.21.15.87:0/4237138359 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff28400bc00 con 0x7ff2980b3480 2024-04-25T10:16:11.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.343+0000 7ff28fe00640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff278066f60 0x7ff278069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.343+0000 7ff28fe00640 1 -- 172.21.15.87:0/4237138359 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2840854f0 con 0x7ff2980b3480 2024-04-25T10:16:11.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.343+0000 7ff297e00640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff278066f60 0x7ff278069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.343+0000 7ff297e00640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff278066f60 0x7ff278069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff2880ad000 tx=0x7ff2880a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.347+0000 7ff28fe00640 1 -- 172.21.15.87:0/4237138359 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff284056460 con 0x7ff2980b3480 2024-04-25T10:16:11.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2980709b0 con 0x7ff2980b3480 2024-04-25T10:16:11.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff28fe00640 1 -- 172.21.15.87:0/4237138359 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff284055d60 con 0x7ff2980b3480 2024-04-25T10:16:11.505 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff278066f60 msgr2=0x7ff278069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff278066f60 0x7ff278069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff2880ad000 tx=0x7ff2880a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 msgr2=0x7ff298162850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff298162850 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff28400d800 tx=0x7ff28400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 shutdown_connections 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff278066f60 0x7ff278069420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2980cfb50 0x7ff29813fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2980b3480 0x7ff298162850 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 --2- 172.21.15.87:0/4237138359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff298071bc0 0x7ff2981431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 >> 172.21.15.87:0/4237138359 conn(0x7ff2980669a0 msgr2=0x7ff2980b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 shutdown_connections 2024-04-25T10:16:11.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.499+0000 7ff29f000640 1 -- 172.21.15.87:0/4237138359 wait complete. 2024-04-25T10:16:11.525 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:11.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fac400640 1 Processor -- start 2024-04-25T10:16:11.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fac400640 1 -- start start 2024-04-25T10:16:11.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fac400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7fa40722d0 0x7f7fa40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fac400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40ad840 con 0x7f7fa40b3480 2024-04-25T10:16:11.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fac400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa415e710 con 0x7f7fa415e130 2024-04-25T10:16:11.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fac400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa40d0090 con 0x7f7fa40722d0 2024-04-25T10:16:11.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7fa40b3480 0x7f7fa4070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60072/0 (socket says 172.21.15.87:60072) 2024-04-25T10:16:11.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9a00640 1 -- 172.21.15.87:0/3795349716 learned_addr learned my addr 172.21.15.87:0/3795349716 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:11.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7faa400640 1 --2- 172.21.15.87:0/3795349716 >> v2:172.21.15.169:3300/0 conn(0x7f7fa40722d0 0x7f7fa40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401569642 0 0) 0x7f7fa40ad840 con 0x7f7fa40b3480 2024-04-25T10:16:11.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f8c003530 con 0x7f7fa40b3480 2024-04-25T10:16:11.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2968511389 0 0) 0x7f7f8c003530 con 0x7f7fa40b3480 2024-04-25T10:16:11.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 >> v2:172.21.15.169:3300/0 conn(0x7f7fa40722d0 msgr2=0x7f7fa40cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 --2- 172.21.15.87:0/3795349716 >> v2:172.21.15.169:3300/0 conn(0x7f7fa40722d0 0x7f7fa40cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 >> v1:172.21.15.169:6789/0 conn(0x7f7fa415e130 legacy=0x7f7fa4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa415e9f0 con 0x7f7fa40b3480 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f7f98001a00 con 0x7f7fa40b3480 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 >> v1:172.21.15.87:6789/0 conn(0x7f7fa40b3480 legacy=0x7f7fa4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f8c0039c0 0x7f7f8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f8c004810 0x7f7f8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c0051d0 0x7f7f8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40d0090 con 0x7f7f8c004810 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c00c170 con 0x7f7f8c0051d0 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c00c370 con 0x7f7f8c0039c0 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9000640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f8c004810 0x7f7f8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9a00640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f8c0039c0 0x7f7f8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7faa400640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c0051d0 0x7f7f8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9000640 1 -- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f8c0039c0 msgr2=0x7f7f8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9000640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f8c0039c0 0x7f7f8c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9000640 1 -- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c0051d0 msgr2=0x7f7f8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9000640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c0051d0 0x7f7f8c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7fa9000640 1 -- 172.21.15.87:0/3795349716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c003530 con 0x7f7f8c004810 2024-04-25T10:16:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7faa400640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c0051d0 0x7f7f8c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.659+0000 7f7faa400640 1 -- 172.21.15.87:0/3795349716 reap_dead start 2024-04-25T10:16:11.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fa9000640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f8c004810 0x7f7f8c004c90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7f90001780 tx=0x7f7f9001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0295d7ff6cff6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900013a0 con 0x7f7f8c004810 2024-04-25T10:16:11.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f900052f0 con 0x7f7f8c004810 2024-04-25T10:16:11.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fa3e00640 1 -- 172.21.15.87:0/3795349716 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003840 con 0x7f7f8c004810 2024-04-25T10:16:11.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fac400640 1 -- 172.21.15.87:0/3795349716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f8c004810 msgr2=0x7f7f8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fac400640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f8c004810 0x7f7f8c004c90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7f90001780 tx=0x7f7f9001c400 comp rx=0 tx=0).stop 2024-04-25T10:16:11.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fac400640 1 -- 172.21.15.87:0/3795349716 shutdown_connections 2024-04-25T10:16:11.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fac400640 1 --2- 172.21.15.87:0/3795349716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f8c004810 0x7f7f8c004c90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fac400640 1 -- 172.21.15.87:0/3795349716 >> 172.21.15.87:0/3795349716 conn(0x7f7fa40669a0 msgr2=0x7f7fa4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fac400640 1 -- 172.21.15.87:0/3795349716 shutdown_connections 2024-04-25T10:16:11.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.663+0000 7f7fac400640 1 -- 172.21.15.87:0/3795349716 wait complete. 2024-04-25T10:16:11.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 Processor -- start 2024-04-25T10:16:11.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 -- start start 2024-04-25T10:16:11.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c004810 0x7f7fa41422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa4142ba0 0x7f7fa4159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa4143020 0x7f7fa41596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40add20 con 0x7f7fa4142ba0 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa415bb70 con 0x7f7f8c004810 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40adea0 con 0x7f7fa4143020 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa4142ba0 0x7f7fa4159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa4142ba0 0x7f7fa4159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48918/0 (socket says 172.21.15.87:48918) 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 -- 172.21.15.87:0/2890387185 learned_addr learned my addr 172.21.15.87:0/2890387185 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 -- 172.21.15.87:0/2890387185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa4143020 msgr2=0x7f7fa41596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa4143020 0x7f7fa41596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 -- 172.21.15.87:0/2890387185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c004810 msgr2=0x7f7fa41422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c004810 0x7f7fa41422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 -- 172.21.15.87:0/2890387185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa41534b0 con 0x7f7fa4142ba0 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa9000640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa4142ba0 0x7f7fa4159190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7f9001c230 tx=0x7f7f8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa1600640 1 -- 172.21.15.87:0/2890387185 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90007a40 con 0x7f7fa4142ba0 2024-04-25T10:16:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa1600640 1 -- 172.21.15.87:0/2890387185 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f900ba070 con 0x7f7fa4142ba0 2024-04-25T10:16:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa4153790 con 0x7f7fa4142ba0 2024-04-25T10:16:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fa1600640 1 -- 172.21.15.87:0/2890387185 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900054a0 con 0x7f7fa4142ba0 2024-04-25T10:16:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.667+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa4153d50 con 0x7f7fa4142ba0 2024-04-25T10:16:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.671+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa404ef30 con 0x7f7fa4142ba0 2024-04-25T10:16:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.671+0000 7f7fa1600640 1 -- 172.21.15.87:0/2890387185 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f90007030 con 0x7f7fa4142ba0 2024-04-25T10:16:11.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.671+0000 7f7fa1600640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f84066f60 0x7f7f84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.671+0000 7f7fa1600640 1 -- 172.21.15.87:0/2890387185 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f9011ef70 con 0x7f7fa4142ba0 2024-04-25T10:16:11.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.671+0000 7f7fa9a00640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f84066f60 0x7f7f84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.671+0000 7f7fa9a00640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f84066f60 0x7f7f84069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000f90 tx=0x7f7f98001f80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.675+0000 7f7fa1600640 1 -- 172.21.15.87:0/2890387185 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f90016110 con 0x7f7fa4142ba0 2024-04-25T10:16:11.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.827+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fa406d560 con 0x7f7fa4142ba0 2024-04-25T10:16:11.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.827+0000 7f7fa1600640 1 -- 172.21.15.87:0/2890387185 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f900efee0 con 0x7f7fa4142ba0 2024-04-25T10:16:11.835 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:16:11.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f84066f60 msgr2=0x7f7f84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f84066f60 0x7f7f84069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000f90 tx=0x7f7f98001f80 comp rx=0 tx=0).stop 2024-04-25T10:16:11.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa4142ba0 msgr2=0x7f7fa4159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa4142ba0 0x7f7fa4159190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7f9001c230 tx=0x7f7f8c005840 comp rx=0 tx=0).stop 2024-04-25T10:16:11.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 shutdown_connections 2024-04-25T10:16:11.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f84066f60 0x7f7f84069420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa4143020 0x7f7fa41596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa4142ba0 0x7f7fa4159190 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 --2- 172.21.15.87:0/2890387185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c004810 0x7f7fa41422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 >> 172.21.15.87:0/2890387185 conn(0x7f7fa40669a0 msgr2=0x7f7fa40af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 shutdown_connections 2024-04-25T10:16:11.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.831+0000 7f7fac400640 1 -- 172.21.15.87:0/2890387185 wait complete. 2024-04-25T10:16:11.857 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:11.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.987+0000 7fb883600640 1 Processor -- start 2024-04-25T10:16:11.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.987+0000 7fb883600640 1 -- start start 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb883600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb87c0d0040 0x7fb87c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb883600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0b2770 con 0x7fb87c0cda10 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb883600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c06c670 con 0x7fb87c15bc40 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb883600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87c06c950 con 0x7fb87c0d0040 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb87be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb87c0cda10 0x7fb87c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60074/0 (socket says 172.21.15.87:60074) 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb87be00640 1 -- 172.21.15.87:0/1778752294 learned_addr learned my addr 172.21.15.87:0/1778752294 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb881600640 1 --2- 172.21.15.87:0/1778752294 >> v2:172.21.15.87:3300/0 conn(0x7fb87c0d0040 0x7fb87c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb87b400640 1 -- 172.21.15.87:0/1778752294 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667603744 0 0) 0x7fb87c0b2770 con 0x7fb87c0cda10 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb87b400640 1 -- 172.21.15.87:0/1778752294 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb860003530 con 0x7fb87c0cda10 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb881600640 1 -- 172.21.15.87:0/1778752294 >> v1:172.21.15.169:6789/0 conn(0x7fb87c15bc40 legacy=0x7fb87c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb881600640 1 -- 172.21.15.87:0/1778752294 >> v1:172.21.15.87:6789/0 conn(0x7fb87c0cda10 legacy=0x7fb87c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb881600640 1 -- 172.21.15.87:0/1778752294 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c06cc30 con 0x7fb87c0d0040 2024-04-25T10:16:11.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb881600640 1 --2- 172.21.15.87:0/1778752294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c0d0040 0x7fb87c06c090 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb8740097c0 tx=0x7fb87401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119367cb02b9355c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.991+0000 7fb87b400640 1 -- 172.21.15.87:0/1778752294 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740b0070 con 0x7fb87c0d0040 2024-04-25T10:16:11.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb87b400640 1 -- 172.21.15.87:0/1778752294 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8740ab600 con 0x7fb87c0d0040 2024-04-25T10:16:11.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb87b400640 1 -- 172.21.15.87:0/1778752294 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740ab9d0 con 0x7fb87c0d0040 2024-04-25T10:16:12.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- 172.21.15.87:0/1778752294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c0d0040 msgr2=0x7fb87c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 --2- 172.21.15.87:0/1778752294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c0d0040 0x7fb87c06c090 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb8740097c0 tx=0x7fb87401c410 comp rx=0 tx=0).stop 2024-04-25T10:16:12.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- 172.21.15.87:0/1778752294 shutdown_connections 2024-04-25T10:16:12.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 --2- 172.21.15.87:0/1778752294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c0d0040 0x7fb87c06c090 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- 172.21.15.87:0/1778752294 >> 172.21.15.87:0/1778752294 conn(0x7fb87c066b70 msgr2=0x7fb87c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- 172.21.15.87:0/1778752294 shutdown_connections 2024-04-25T10:16:12.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- 172.21.15.87:0/1778752294 wait complete. 2024-04-25T10:16:12.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 Processor -- start 2024-04-25T10:16:12.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- start start 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb87c0cda10 0x7fb87c148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb87c0d0040 0x7fb87c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c15bc40 0x7fb87c15d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c169980 con 0x7fb87c15bc40 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c169800 con 0x7fb87c0d0040 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb883600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c169b00 con 0x7fb87c0cda10 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb881600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c15bc40 0x7fb87c15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb881600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c15bc40 0x7fb87c15d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48940/0 (socket says 172.21.15.87:48940) 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb881600640 1 -- 172.21.15.87:0/1458984365 learned_addr learned my addr 172.21.15.87:0/1458984365 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb880c00640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb87c0cda10 0x7fb87c148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.995+0000 7fb87be00640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb87c0d0040 0x7fb87c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb881600640 1 -- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb87c0cda10 msgr2=0x7fb87c148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb881600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb87c0cda10 0x7fb87c148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb881600640 1 -- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb87c0d0040 msgr2=0x7fb87c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb881600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb87c0d0040 0x7fb87c0b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb881600640 1 -- 172.21.15.87:0/1458984365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0b09e0 con 0x7fb87c15bc40 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb881600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c15bc40 0x7fb87c15d010 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb8740ae040 tx=0x7fb8740191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb878c00640 1 -- 172.21.15.87:0/1458984365 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740b8020 con 0x7fb87c15bc40 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb878c00640 1 -- 172.21.15.87:0/1458984365 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8740b4610 con 0x7fb87c15bc40 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb878c00640 1 -- 172.21.15.87:0/1458984365 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740b0040 con 0x7fb87c15bc40 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c0b0cc0 con 0x7fb87c15bc40 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c0b1280 con 0x7fb87c15bc40 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c04ef30 con 0x7fb87c15bc40 2024-04-25T10:16:12.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb878c00640 1 -- 172.21.15.87:0/1458984365 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8740bb020 con 0x7fb87c15bc40 2024-04-25T10:16:12.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb878c00640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb85c066f60 0x7fb85c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb878c00640 1 -- 172.21.15.87:0/1458984365 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb87411ed20 con 0x7fb87c15bc40 2024-04-25T10:16:12.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:11.999+0000 7fb880c00640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb85c066f60 0x7fb85c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.003+0000 7fb880c00640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb85c066f60 0x7fb85c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb86c001d50 tx=0x7fb86c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.003+0000 7fb878c00640 1 -- 172.21.15.87:0/1458984365 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8740c2030 con 0x7fb87c15bc40 2024-04-25T10:16:12.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.171+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb87c06d960 con 0x7fb87c15bc40 2024-04-25T10:16:12.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.171+0000 7fb878c00640 1 -- 172.21.15.87:0/1458984365 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb874016090 con 0x7fb87c15bc40 2024-04-25T10:16:12.179 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:16:12.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb85c066f60 msgr2=0x7fb85c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb85c066f60 0x7fb85c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb86c001d50 tx=0x7fb86c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:12.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c15bc40 msgr2=0x7fb87c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c15bc40 0x7fb87c15d010 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb8740ae040 tx=0x7fb8740191b0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 shutdown_connections 2024-04-25T10:16:12.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb85c066f60 0x7fb85c069420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb87c15bc40 0x7fb87c15d010 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb87c0d0040 0x7fb87c0b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 --2- 172.21.15.87:0/1458984365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb87c0cda10 0x7fb87c148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 >> 172.21.15.87:0/1458984365 conn(0x7fb87c066b70 msgr2=0x7fb87c0ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 shutdown_connections 2024-04-25T10:16:12.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.175+0000 7fb883600640 1 -- 172.21.15.87:0/1458984365 wait complete. 2024-04-25T10:16:12.200 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:16:12.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:12.200 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:12.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:16:12.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:12.252 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:12.275 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.447+0000 7fc002600640 1 Processor -- start 2024-04-25T10:16:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.447+0000 7fc002600640 1 -- start start 2024-04-25T10:16:12.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fc002600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fc002600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbffc0cda10 0x7fbffc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fc002600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc0b2770 con 0x7fbffc0cf930 2024-04-25T10:16:12.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fc002600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc06c670 con 0x7fbffc0cda10 2024-04-25T10:16:12.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fc002600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc06c950 con 0x7fbffc15bc40 2024-04-25T10:16:12.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbffc0cf930 0x7fbffc15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60084/0 (socket says 172.21.15.87:60084) 2024-04-25T10:16:12.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffbe00640 1 -- 172.21.15.87:0/4022823544 learned_addr learned my addr 172.21.15.87:0/4022823544 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:12.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffb400640 1 --2- 172.21.15.87:0/4022823544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffaa00640 1 --2- 172.21.15.87:0/4022823544 >> v2:172.21.15.169:3300/0 conn(0x7fbffc0cda10 0x7fbffc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffa000640 1 -- 172.21.15.87:0/4022823544 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794617116 0 0) 0x7fbffc0b2770 con 0x7fbffc0cf930 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffa000640 1 -- 172.21.15.87:0/4022823544 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe0003530 con 0x7fbffc0cf930 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffb400640 1 -- 172.21.15.87:0/4022823544 >> v2:172.21.15.169:3300/0 conn(0x7fbffc0cda10 msgr2=0x7fbffc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffb400640 1 --2- 172.21.15.87:0/4022823544 >> v2:172.21.15.169:3300/0 conn(0x7fbffc0cda10 0x7fbffc0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffb400640 1 -- 172.21.15.87:0/4022823544 >> v1:172.21.15.87:6789/0 conn(0x7fbffc0cf930 legacy=0x7fbffc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffb400640 1 -- 172.21.15.87:0/4022823544 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc06cc30 con 0x7fbffc15bc40 2024-04-25T10:16:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.451+0000 7fbffaa00640 1 --2- 172.21.15.87:0/4022823544 >> v2:172.21.15.169:3300/0 conn(0x7fbffc0cda10 0x7fbffc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:12.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fbffb400640 1 --2- 172.21.15.87:0/4022823544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc06c090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009d90 tx=0x7fbfec01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a4e8dcb0d698b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fbffa000640 1 -- 172.21.15.87:0/4022823544 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0b2070 con 0x7fbffc15bc40 2024-04-25T10:16:12.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fbffa000640 1 -- 172.21.15.87:0/4022823544 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0a5940 con 0x7fbffc15bc40 2024-04-25T10:16:12.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fbffa000640 1 -- 172.21.15.87:0/4022823544 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0a5d10 con 0x7fbffc15bc40 2024-04-25T10:16:12.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- 172.21.15.87:0/4022823544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 msgr2=0x7fbffc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 --2- 172.21.15.87:0/4022823544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc06c090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009d90 tx=0x7fbfec01c970 comp rx=0 tx=0).stop 2024-04-25T10:16:12.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- 172.21.15.87:0/4022823544 shutdown_connections 2024-04-25T10:16:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 --2- 172.21.15.87:0/4022823544 >> v2:172.21.15.169:3300/0 conn(0x7fbffc0cda10 0x7fbffc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 --2- 172.21.15.87:0/4022823544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc06c090 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- 172.21.15.87:0/4022823544 >> 172.21.15.87:0/4022823544 conn(0x7fbffc066b70 msgr2=0x7fbffc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- 172.21.15.87:0/4022823544 shutdown_connections 2024-04-25T10:16:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- 172.21.15.87:0/4022823544 wait complete. 2024-04-25T10:16:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 Processor -- start 2024-04-25T10:16:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- start start 2024-04-25T10:16:12.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbffc0cda10 0x7fbffc14b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbffc0cf930 0x7fbffc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc168b50 con 0x7fbffc0cda10 2024-04-25T10:16:12.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1689d0 con 0x7fbffc0cf930 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.455+0000 7fc002600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc168cd0 con 0x7fbffc15bc40 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbffc0cda10 0x7fbffc14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbffc0cda10 0x7fbffc14b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48950/0 (socket says 172.21.15.87:48950) 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 -- 172.21.15.87:0/1640260023 learned_addr learned my addr 172.21.15.87:0/1640260023 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffaa00640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbffc0cf930 0x7fbffc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffbe00640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 -- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 msgr2=0x7fbffc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 -- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbffc0cf930 msgr2=0x7fbffc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbffc0cf930 0x7fbffc0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 -- 172.21.15.87:0/1640260023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc155d80 con 0x7fbffc0cda10 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffbe00640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbffb400640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbffc0cda10 0x7fbffc14b520 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbfec01a5a0 tx=0x7fbfec019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbff3400640 1 -- 172.21.15.87:0/1640260023 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0b2070 con 0x7fbffc0cda10 2024-04-25T10:16:12.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbff3400640 1 -- 172.21.15.87:0/1640260023 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0b3040 con 0x7fbffc0cda10 2024-04-25T10:16:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc156060 con 0x7fbffc0cda10 2024-04-25T10:16:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbff3400640 1 -- 172.21.15.87:0/1640260023 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0b9890 con 0x7fbffc0cda10 2024-04-25T10:16:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc156620 con 0x7fbffc0cda10 2024-04-25T10:16:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc04ef30 con 0x7fbffc0cda10 2024-04-25T10:16:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.459+0000 7fbff3400640 1 -- 172.21.15.87:0/1640260023 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfec0c1030 con 0x7fbffc0cda10 2024-04-25T10:16:12.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.463+0000 7fbff3400640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.463+0000 7fbff3400640 1 -- 172.21.15.87:0/1640260023 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfec11f900 con 0x7fbffc0cda10 2024-04-25T10:16:12.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.463+0000 7fbffaa00640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.463+0000 7fbffaa00640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfdc066f60 0x7fbfdc069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009580 tx=0x7fbfe8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.467+0000 7fbff3400640 1 -- 172.21.15.87:0/1640260023 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfec0c8020 con 0x7fbffc0cda10 2024-04-25T10:16:12.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.651+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbffc06d960 con 0x7fbffc0cda10 2024-04-25T10:16:12.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.651+0000 7fbff3400640 1 -- 172.21.15.87:0/1640260023 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fbfec11f070 con 0x7fbffc0cda10 2024-04-25T10:16:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:12.658 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/243216881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":243216881},{"type":"v1","addr":"172.21.15.169:6835","nonce":243216881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6073,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/250986881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":250986881},{"type":"v1","addr":"172.21.15.169:6834","nonce":250986881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6510,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2928764815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2928764815},{"type":"v1","addr":"172.21.15.87:6836","nonce":2928764815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:05.597920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6513":{"gid":6513,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3387238920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3387238920},{"type":"v1","addr":"172.21.15.87:6837","nonce":3387238920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:12.658 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 37 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfdc066f60 msgr2=0x7fbfdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfdc066f60 0x7fbfdc069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009580 tx=0x7fbfe8009190 comp rx=0 tx=0).stop 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbffc0cda10 msgr2=0x7fbffc14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbffc0cda10 0x7fbffc14b520 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbfec01a5a0 tx=0x7fbfec019710 comp rx=0 tx=0).stop 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 shutdown_connections 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbffc15bc40 0x7fbffc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbffc0cf930 0x7fbffc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 --2- 172.21.15.87:0/1640260023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbffc0cda10 0x7fbffc14b520 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 >> 172.21.15.87:0/1640260023 conn(0x7fbffc066b70 msgr2=0x7fbffc1546b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 shutdown_connections 2024-04-25T10:16:12.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.655+0000 7fc002600640 1 -- 172.21.15.87:0/1640260023 wait complete. 2024-04-25T10:16:12.679 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:16:12.847 INFO:teuthology.orchestra.run.smithi087.stdout:{"message":"","return_code":0} 2024-04-25T10:16:12.847 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:16:12.848 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:16:12.848 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:12.848 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:16:12.848 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:12.849 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:16:12.849 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:12.849 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:16:12.849 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:12.861 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:16:12.861 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:12.863 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:16:12.863 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:12.864 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:16:12.864 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:12.876 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:16:12.876 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:12.877 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T10:16:12.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 Processor -- start 2024-04-25T10:16:12.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- start start 2024-04-25T10:16:12.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b504820 0x55f18b4fd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b4e6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b5922b0 0x55f18b5936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18b4e9a50 con 0x55f18b504820 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18b4619f0 con 0x55f18b504c00 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f18b4e9bd0 con 0x55f18b5922b0 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b504820 0x55f18b4fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b504820 0x55f18b4fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48982/0 (socket says 172.21.15.87:48982) 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 -- 172.21.15.87:0/1943005694 learned_addr learned my addr 172.21.15.87:0/1943005694 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467e00640 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b5922b0 0x55f18b5936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f445fe00640 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b4e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 -- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b5922b0 msgr2=0x55f18b5936f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b5922b0 0x55f18b5936f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 -- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 msgr2=0x55f18b4e6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b4e6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 -- 172.21.15.87:0/1943005694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18b5927d0 con 0x55f18b504820 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4467400640 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b504820 0x55f18b4fd430 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f445800b350 tx=0x7f445800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0585c4c30c8540b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4466000640 1 -- 172.21.15.87:0/1943005694 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800d5d0 con 0x55f18b504820 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4466000640 1 -- 172.21.15.87:0/1943005694 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458008070 con 0x55f18b504820 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f4466000640 1 -- 172.21.15.87:0/1943005694 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800d970 con 0x55f18b504820 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- 172.21.15.87:0/1943005694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b504820 msgr2=0x55f18b4fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b504820 0x55f18b4fd430 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f445800b350 tx=0x7f445800b810 comp rx=0 tx=0).stop 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- 172.21.15.87:0/1943005694 shutdown_connections 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b5922b0 0x55f18b5936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b4e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 --2- 172.21.15.87:0/1943005694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b504820 0x55f18b4fd430 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- 172.21.15.87:0/1943005694 >> 172.21.15.87:0/1943005694 conn(0x55f18b4e5760 msgr2=0x55f18b4fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- 172.21.15.87:0/1943005694 shutdown_connections 2024-04-25T10:16:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.947+0000 7f446a1c7080 1 -- 172.21.15.87:0/1943005694 wait complete. 2024-04-25T10:16:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 Processor -- start 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 -- start start 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b504820 0x55f18b4fe440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b59ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b5922b0 0x55f18b5904a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18b592d10 con 0x55f18b5922b0 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18b592a70 con 0x55f18b504c00 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f18b592fb0 con 0x55f18b504820 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b5922b0 0x55f18b5904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b5922b0 0x55f18b5904a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48992/0 (socket says 172.21.15.87:48992) 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 -- 172.21.15.87:0/2474796415 learned_addr learned my addr 172.21.15.87:0/2474796415 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f445fe00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b59ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467400640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b504820 0x55f18b4fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 -- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b504820 msgr2=0x55f18b4fe440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b504820 0x55f18b4fe440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 -- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 msgr2=0x55f18b59ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b59ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 -- 172.21.15.87:0/2474796415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18b59b270 con 0x55f18b5922b0 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467400640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f18b504820 0x55f18b4fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f445fe00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f18b504c00 0x55f18b59ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4467e00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b5922b0 0x55f18b5904a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f446000b4e0 tx=0x7f446000b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4464c00640 1 -- 172.21.15.87:0/2474796415 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460017020 con 0x55f18b5922b0 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4464c00640 1 -- 172.21.15.87:0/2474796415 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4460012660 con 0x55f18b5922b0 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f18b593f10 con 0x55f18b5922b0 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f4464c00640 1 -- 172.21.15.87:0/2474796415 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460012a50 con 0x55f18b5922b0 2024-04-25T10:16:12.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.951+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f18b503060 con 0x55f18b5922b0 2024-04-25T10:16:12.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f4464c00640 1 -- 172.21.15.87:0/2474796415 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f446001a020 con 0x55f18b5922b0 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f4464c00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4444065970 0x7f4444067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f4464c00640 1 -- 172.21.15.87:0/2474796415 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4460084250 con 0x55f18b5922b0 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f4467400640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4444065970 0x7f4444067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f446a1c7080 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x55f18b573ad0 0x55f18b5037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 --> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] -- osd_op(unknown.0.0:1 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57) v9 -- 0x55f18b503ce0 con 0x55f18b573ad0 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f445fe00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x55f18b573ad0 0x55f18b5037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f4467400640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4444065970 0x7f4444067e30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4458000c10 tx=0x7f4458008000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.955+0000 7f445fe00640 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x55f18b573ad0 0x55f18b5037a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f445fe00640 1 -- 172.21.15.87:0/2474796415 <== osd.7 v2:172.21.15.169:6824/2370587194 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v57'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f44500ac9b0 con 0x55f18b573ad0 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x55f18b573ad0 msgr2=0x55f18b5037a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x55f18b573ad0 0x55f18b5037a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4444065970 msgr2=0x7f4444067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4444065970 0x7f4444067e30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4458000c10 tx=0x7f4458008000 comp rx=0 tx=0).stop 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b5922b0 msgr2=0x55f18b5904a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 --2- 172.21.15.87:0/2474796415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f18b5922b0 0x55f18b5904a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f446000b4e0 tx=0x7f446000b9a0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f4467e00640 1 -- 172.21.15.87:0/2474796415 reap_dead start 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 shutdown_connections 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 >> 172.21.15.87:0/2474796415 conn(0x55f18b4e5760 msgr2=0x55f18b4e6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 shutdown_connections 2024-04-25T10:16:12.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:12.959+0000 7f446a1c7080 1 -- 172.21.15.87:0/2474796415 wait complete. 2024-04-25T10:16:12.968 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T10:16:12.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T10:16:12.969 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T10:16:12.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T10:16:12.969 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T10:16:12.970 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T10:16:12.970 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T10:16:12.970 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T10:16:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992f800640 1 Processor -- start 2024-04-25T10:16:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992f800640 1 -- start start 2024-04-25T10:16:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b7e00640 1 Processor -- start 2024-04-25T10:16:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f0e000640 1 Processor -- start 2024-04-25T10:16:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b7e00640 1 -- start start 2024-04-25T10:16:13.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f0e000640 1 -- start start 2024-04-25T10:16:13.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f8600640 1 Processor -- start 2024-04-25T10:16:13.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f8600640 1 -- start start 2024-04-25T10:16:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f0e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5f0806af30 0x7f5f080b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280ae020 0x7f99280ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992f800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99280ad840 con 0x7f99280cf930 2024-04-25T10:16:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992f800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992815e9f0 con 0x7f9928070c20 2024-04-25T10:16:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992815bb90 con 0x7f99280ae020 2024-04-25T10:16:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55b015bc40 0x7f55b006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f0e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f080b0b80 0x7f5f080ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f0e000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f080cec70 con 0x7f5f080aeb50 2024-04-25T10:16:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f0e000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f080d27b0 con 0x7f5f0806af30 2024-04-25T10:16:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f0e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f080b26d0 con 0x7f5f080b0b80 2024-04-25T10:16:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b00b2770 con 0x7f55b00cf930 2024-04-25T10:16:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b006c670 con 0x7f55b015bc40 2024-04-25T10:16:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b006c950 con 0x7f55b00cda10 2024-04-25T10:16:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f8600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f91f0072260 0x7f91f0073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f8600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f006f450 con 0x7f91f006e480 2024-04-25T10:16:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f8600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00cca70 con 0x7f91f0072260 2024-04-25T10:16:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00cccc0 con 0x7f91f006e0a0 2024-04-25T10:16:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280ae020 0x7f99280ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992ce00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f99280cf930 0x7f99280cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60102/0 (socket says 172.21.15.87:60102) 2024-04-25T10:16:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f992ce00640 1 -- 172.21.15.87:0/2066251585 learned_addr learned my addr 172.21.15.87:0/2066251585 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f07e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5f080aeb50 0x7f5f0806b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60114/0 (socket says 172.21.15.87:60114) 2024-04-25T10:16:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f07e00640 1 -- 172.21.15.87:0/2843181889 learned_addr learned my addr 172.21.15.87:0/2843181889 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f06a00640 1 --2- 172.21.15.87:0/2843181889 >> v2:172.21.15.169:3300/0 conn(0x7f5f080b0b80 0x7f5f080ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f5f07400640 1 --2- 172.21.15.87:0/2843181889 >> v2:172.21.15.87:3300/0 conn(0x7f5f0806af30 0x7f5f080b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55b015bc40 0x7f55b006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f91f0072260 0x7f91f0073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:43032/0 (socket says 172.21.15.87:43032) 2024-04-25T10:16:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b4a00640 1 -- 172.21.15.87:0/623717003 learned_addr learned my addr 172.21.15.87:0/623717003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f55b5e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f55b00cf930 0x7f55b015f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:33858/0 (socket says 172.21.15.87:33858) 2024-04-25T10:16:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:43042/0 (socket says 172.21.15.87:43042) 2024-04-25T10:16:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.110+0000 7f91f5c00640 1 -- 172.21.15.87:0/3730503526 learned_addr learned my addr 172.21.15.87:0/3730503526 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f992d800640 1 -- 172.21.15.87:0/2066251585 >> v1:172.21.15.169:6789/0 conn(0x7f9928070c20 legacy=0x7f992815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f992d800640 1 -- 172.21.15.87:0/2066251585 >> v1:172.21.15.87:6789/0 conn(0x7f99280cf930 legacy=0x7f99280cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f9927400640 1 -- 172.21.15.87:0/2066251585 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251200356 0 0) 0x7f992815e9f0 con 0x7f9928070c20 2024-04-25T10:16:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f5f06a00640 1 -- 172.21.15.87:0/2843181889 >> v2:172.21.15.87:3300/0 conn(0x7f5f0806af30 msgr2=0x7f5f080b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f992d800640 1 -- 172.21.15.87:0/2066251585 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992815ecd0 con 0x7f99280ae020 2024-04-25T10:16:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f5f06a00640 1 --2- 172.21.15.87:0/2843181889 >> v2:172.21.15.87:3300/0 conn(0x7f5f0806af30 0x7f5f080b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f5f06a00640 1 -- 172.21.15.87:0/2843181889 >> v1:172.21.15.87:6789/0 conn(0x7f5f080aeb50 legacy=0x7f5f0806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f5f06a00640 1 -- 172.21.15.87:0/2843181889 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080b1da0 con 0x7f5f080b0b80 2024-04-25T10:16:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f5f07400640 1 --2- 172.21.15.87:0/2843181889 >> v2:172.21.15.87:3300/0 conn(0x7f5f0806af30 0x7f5f080b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f55afe00640 1 -- 172.21.15.87:0/623717003 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679401784 0 0) 0x7f55b00b2770 con 0x7f55b00cf930 2024-04-25T10:16:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f55afe00640 1 -- 172.21.15.87:0/623717003 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5598003530 con 0x7f55b00cf930 2024-04-25T10:16:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f91f5c00640 1 -- 172.21.15.87:0/3730503526 >> v2:172.21.15.169:3300/0 conn(0x7f91f0072260 msgr2=0x7f91f0073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f91f5c00640 1 --2- 172.21.15.87:0/3730503526 >> v2:172.21.15.169:3300/0 conn(0x7f91f0072260 0x7f91f0073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f91f5c00640 1 -- 172.21.15.87:0/3730503526 >> v1:172.21.15.87:6789/0 conn(0x7f91f006e480 legacy=0x7f91f0070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f91f5c00640 1 -- 172.21.15.87:0/3730503526 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00ccfa0 con 0x7f91f006e0a0 2024-04-25T10:16:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f55b5400640 1 -- 172.21.15.87:0/623717003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 msgr2=0x7f55b00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f55b5400640 1 --2- 172.21.15.87:0/623717003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f55b5400640 1 -- 172.21.15.87:0/623717003 >> v1:172.21.15.169:6789/0 conn(0x7f55b00cf930 legacy=0x7f55b015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f55b5400640 1 -- 172.21.15.87:0/623717003 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b006cc30 con 0x7f55b015bc40 2024-04-25T10:16:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f91f5c00640 1 --2- 172.21.15.87:0/3730503526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f015c070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009870 tx=0x7f91e401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dc585abce816090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f55b5400640 1 --2- 172.21.15.87:0/623717003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 0x7f55b006c090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55a400f370 tx=0x7f55a4020d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfba6583cdd6b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.114+0000 7f91efe00640 1 -- 172.21.15.87:0/3730503526 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e40b0070 con 0x7f91f006e0a0 2024-04-25T10:16:13.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91efe00640 1 -- 172.21.15.87:0/3730503526 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e40ab400 con 0x7f91f006e0a0 2024-04-25T10:16:13.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91efe00640 1 -- 172.21.15.87:0/3730503526 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e40ab7d0 con 0x7f91f006e0a0 2024-04-25T10:16:13.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55afe00640 1 -- 172.21.15.87:0/623717003 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a40a9b50 con 0x7f55b015bc40 2024-04-25T10:16:13.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55afe00640 1 -- 172.21.15.87:0/623717003 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a40a9cf0 con 0x7f55b015bc40 2024-04-25T10:16:13.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55afe00640 1 -- 172.21.15.87:0/623717003 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400d2d0 con 0x7f55b015bc40 2024-04-25T10:16:13.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 -- 172.21.15.87:0/3730503526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 msgr2=0x7f91f015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 --2- 172.21.15.87:0/3730503526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f015c070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009870 tx=0x7f91e401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 -- 172.21.15.87:0/623717003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 msgr2=0x7f55b006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 --2- 172.21.15.87:0/623717003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 0x7f55b006c090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55a400f370 tx=0x7f55a4020d70 comp rx=0 tx=0).stop 2024-04-25T10:16:13.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 -- 172.21.15.87:0/623717003 shutdown_connections 2024-04-25T10:16:13.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 --2- 172.21.15.87:0/623717003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b00cda10 0x7f55b00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 --2- 172.21.15.87:0/623717003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b015bc40 0x7f55b006c090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 -- 172.21.15.87:0/3730503526 shutdown_connections 2024-04-25T10:16:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 --2- 172.21.15.87:0/3730503526 >> v2:172.21.15.169:3300/0 conn(0x7f91f0072260 0x7f91f0073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 --2- 172.21.15.87:0/3730503526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f015c070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 -- 172.21.15.87:0/3730503526 >> 172.21.15.87:0/3730503526 conn(0x7f91f00d00a0 msgr2=0x7f91f00d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992d800640 1 --2- 172.21.15.87:0/2066251585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280ae020 0x7f99280ae4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9920009dc0 tx=0x7f992001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805103c80c5cbefa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f06a00640 1 --2- 172.21.15.87:0/2843181889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080ae610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f08073390 tx=0x7f5ef401bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7a8670a236aa00e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 -- 172.21.15.87:0/623717003 >> 172.21.15.87:0/623717003 conn(0x7f55b0066b70 msgr2=0x7f55b0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 -- 172.21.15.87:0/623717003 shutdown_connections 2024-04-25T10:16:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 -- 172.21.15.87:0/3730503526 shutdown_connections 2024-04-25T10:16:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 -- 172.21.15.87:0/3730503526 wait complete. 2024-04-25T10:16:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 -- 172.21.15.87:0/623717003 wait complete. 2024-04-25T10:16:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f9927400640 1 -- 172.21.15.87:0/2066251585 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99200b2070 con 0x7f99280ae020 2024-04-25T10:16:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f06000640 1 -- 172.21.15.87:0/2843181889 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40a85f0 con 0x7f5f080b0b80 2024-04-25T10:16:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f06000640 1 -- 172.21.15.87:0/2843181889 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef40a8790 con 0x7f5f080b0b80 2024-04-25T10:16:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f06000640 1 -- 172.21.15.87:0/2843181889 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40a8b60 con 0x7f5f080b0b80 2024-04-25T10:16:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f9927400640 1 -- 172.21.15.87:0/2066251585 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99200a5a20 con 0x7f99280ae020 2024-04-25T10:16:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f9927400640 1 -- 172.21.15.87:0/2066251585 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99200a5df0 con 0x7f99280ae020 2024-04-25T10:16:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992f800640 1 -- 172.21.15.87:0/2066251585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280ae020 msgr2=0x7f99280ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992f800640 1 --2- 172.21.15.87:0/2066251585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280ae020 0x7f99280ae4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9920009dc0 tx=0x7f992001c960 comp rx=0 tx=0).stop 2024-04-25T10:16:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 -- 172.21.15.87:0/2843181889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 msgr2=0x7f5f080ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 --2- 172.21.15.87:0/2843181889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080ae610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f08073390 tx=0x7f5ef401bf00 comp rx=0 tx=0).stop 2024-04-25T10:16:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992f800640 1 -- 172.21.15.87:0/2066251585 shutdown_connections 2024-04-25T10:16:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992f800640 1 --2- 172.21.15.87:0/2066251585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99280ae020 0x7f99280ae4b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992f800640 1 -- 172.21.15.87:0/2066251585 >> 172.21.15.87:0/2066251585 conn(0x7f9928066b70 msgr2=0x7f9928066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992f800640 1 -- 172.21.15.87:0/2066251585 shutdown_connections 2024-04-25T10:16:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 -- 172.21.15.87:0/2843181889 shutdown_connections 2024-04-25T10:16:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 --2- 172.21.15.87:0/2843181889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080ae610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 --2- 172.21.15.87:0/2843181889 >> v2:172.21.15.87:3300/0 conn(0x7f5f0806af30 0x7f5f080b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 -- 172.21.15.87:0/2843181889 >> 172.21.15.87:0/2843181889 conn(0x7f5f080d1900 msgr2=0x7f5f080d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f992f800640 1 -- 172.21.15.87:0/2066251585 wait complete. 2024-04-25T10:16:13.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 -- 172.21.15.87:0/2843181889 shutdown_connections 2024-04-25T10:16:13.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 Processor -- start 2024-04-25T10:16:13.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 Processor -- start 2024-04-25T10:16:13.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f55b7e00640 1 -- start start 2024-04-25T10:16:13.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f91f8600640 1 -- start start 2024-04-25T10:16:13.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.118+0000 7f5f0e000640 1 -- 172.21.15.87:0/2843181889 wait complete. 2024-04-25T10:16:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00cda10 0x7f55b00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b01fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b015bc40 0x7f55b01fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0166b40 con 0x7f55b00cf930 2024-04-25T10:16:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b01669c0 con 0x7f55b00cda10 2024-04-25T10:16:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0166cc0 con 0x7f55b015bc40 2024-04-25T10:16:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f01584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f006e480 0x7f91f01470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0072260 0x7f91f0147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0166930 con 0x7f91f0072260 2024-04-25T10:16:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01667b0 con 0x7f91f006e480 2024-04-25T10:16:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0166ab0 con 0x7f91f006e0a0 2024-04-25T10:16:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00cda10 0x7f55b00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b01fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b01fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49000/0 (socket says 172.21.15.87:49000) 2024-04-25T10:16:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 -- 172.21.15.87:0/3171196777 learned_addr learned my addr 172.21.15.87:0/3171196777 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b5e00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b015bc40 0x7f55b01fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f006e480 0x7f91f01470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f006e480 0x7f91f01470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:36874/0 (socket says 172.21.15.87:36874) 2024-04-25T10:16:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 -- 172.21.15.87:0/666084024 learned_addr learned my addr 172.21.15.87:0/666084024 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:13.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 Processor -- start 2024-04-25T10:16:13.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f6600640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0072260 0x7f91f0147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 Processor -- start 2024-04-25T10:16:13.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5c00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f01584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 -- start start 2024-04-25T10:16:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 -- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 msgr2=0x7f91f01584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f01584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 -- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0072260 msgr2=0x7f91f0147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0072260 0x7f91f0147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 -- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b015bc40 msgr2=0x7f55b01fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b015bc40 0x7f55b01fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 -- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00cda10 msgr2=0x7f55b00ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00cda10 0x7f55b00ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 -- 172.21.15.87:0/666084024 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f01366a0 con 0x7f91f006e480 2024-04-25T10:16:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 -- 172.21.15.87:0/3171196777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b01fb210 con 0x7f55b00cf930 2024-04-25T10:16:13.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b5400640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00cda10 0x7f55b00ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:13.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 -- start start 2024-04-25T10:16:13.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b4a00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b01fe800 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f559c00e820 tx=0x7f559c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f5200640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f006e480 0x7f91f01470c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f91dc012f90 tx=0x7f91dc010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55ad600640 1 -- 172.21.15.87:0/3171196777 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c017070 con 0x7f55b00cf930 2024-04-25T10:16:13.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55ad600640 1 -- 172.21.15.87:0/3171196777 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559c00c940 con 0x7f55b00cf930 2024-04-25T10:16:13.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55ad600640 1 -- 172.21.15.87:0/3171196777 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c00cd30 con 0x7f55b00cf930 2024-04-25T10:16:13.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928070c20 0x7f99281506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280ae020 0x7f992815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99280cf930 0x7f99281573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992816ba70 con 0x7f99280ae020 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992816b8f0 con 0x7f99280cf930 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992816bbf0 con 0x7f9928070c20 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f992d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99280cf930 0x7f99281573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99280cf930 0x7f99281573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:36880/0 (socket says 172.21.15.87:36880) 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928070c20 0x7f99281506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992d800640 1 -- 172.21.15.87:0/2284643144 learned_addr learned my addr 172.21.15.87:0/2284643144 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f0806af30 0x7f5f080c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f080aeb50 0x7f5f0814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f080b3220 con 0x7f5f0806af30 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f080b30a0 con 0x7f5f080b0b80 2024-04-25T10:16:13.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f5f0e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f080d0800 con 0x7f5f080aeb50 2024-04-25T10:16:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b01fb4d0 con 0x7f55b00cf930 2024-04-25T10:16:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b01fba90 con 0x7f55b00cf930 2024-04-25T10:16:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91ed600640 1 -- 172.21.15.87:0/666084024 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc010da0 con 0x7f91f006e480 2024-04-25T10:16:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91ed600640 1 -- 172.21.15.87:0/666084024 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc01d070 con 0x7f91f006e480 2024-04-25T10:16:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91ed600640 1 -- 172.21.15.87:0/666084024 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0186c0 con 0x7f91f006e480 2024-04-25T10:16:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f9927e00640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280ae020 0x7f992815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f07400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f0806af30 0x7f5f080c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f07400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f0806af30 0x7f5f080c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49028/0 (socket says 172.21.15.87:49028) 2024-04-25T10:16:13.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f07400640 1 -- 172.21.15.87:0/1591016895 learned_addr learned my addr 172.21.15.87:0/1591016895 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:13.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f06a00640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f080aeb50 0x7f5f0814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 -- 172.21.15.87:0/666084024 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f0136980 con 0x7f91f006e480 2024-04-25T10:16:13.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.122+0000 7f91f8600640 1 -- 172.21.15.87:0/666084024 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f0136f40 con 0x7f91f006e480 2024-04-25T10:16:13.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b004ef30 con 0x7f55b00cf930 2024-04-25T10:16:13.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f07e00640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992ce00640 1 -- 172.21.15.87:0/2284643144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99280cf930 msgr2=0x7f99281573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992ce00640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99280cf930 0x7f99281573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992ce00640 1 -- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280ae020 msgr2=0x7f992815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992ce00640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280ae020 0x7f992815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992ce00640 1 -- 172.21.15.87:0/2284643144 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928150c30 con 0x7f9928070c20 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f004ef30 con 0x7f91f006e480 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f06a00640 1 -- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 msgr2=0x7f5f080b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f06a00640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f06a00640 1 -- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f0806af30 msgr2=0x7f5f080c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f06a00640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f0806af30 0x7f5f080c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f06a00640 1 -- 172.21.15.87:0/1591016895 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f081611c0 con 0x7f5f080aeb50 2024-04-25T10:16:13.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f07e00640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:13.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992ce00640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928070c20 0x7f99281506f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c390 tx=0x7f991c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f9924c00640 1 -- 172.21.15.87:0/2284643144 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c017070 con 0x7f9928070c20 2024-04-25T10:16:13.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f9924c00640 1 -- 172.21.15.87:0/2284643144 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c005cf0 con 0x7f9928070c20 2024-04-25T10:16:13.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f9924c00640 1 -- 172.21.15.87:0/2284643144 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0072d0 con 0x7f9928070c20 2024-04-25T10:16:13.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f06a00640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f080aeb50 0x7f5f0814cdd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40a9000 tx=0x7f5ef4018ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f55ad600640 1 -- 172.21.15.87:0/3171196777 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f559c01a070 con 0x7f55b00cf930 2024-04-25T10:16:13.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9928151a40 con 0x7f9928070c20 2024-04-25T10:16:13.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9928152000 con 0x7f9928070c20 2024-04-25T10:16:13.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5eff400640 1 -- 172.21.15.87:0/1591016895 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40c0070 con 0x7f5f080aeb50 2024-04-25T10:16:13.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5eff400640 1 -- 172.21.15.87:0/1591016895 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef40a8530 con 0x7f5f080aeb50 2024-04-25T10:16:13.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5eff400640 1 -- 172.21.15.87:0/1591016895 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40bb660 con 0x7f5f080aeb50 2024-04-25T10:16:13.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f080c5400 con 0x7f5f080aeb50 2024-04-25T10:16:13.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f080c59c0 con 0x7f5f080aeb50 2024-04-25T10:16:13.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992804ef30 con 0x7f9928070c20 2024-04-25T10:16:13.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f55ad600640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5590066f60 0x7f5590069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.126+0000 7f55ad600640 1 -- 172.21.15.87:0/3171196777 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f559c085120 con 0x7f55b00cf930 2024-04-25T10:16:13.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0804ef30 con 0x7f5f080aeb50 2024-04-25T10:16:13.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f91ed600640 1 -- 172.21.15.87:0/666084024 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91dc016050 con 0x7f91f006e480 2024-04-25T10:16:13.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f55b5400640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5590066f60 0x7f5590069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f9924c00640 1 -- 172.21.15.87:0/2284643144 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f991c01a070 con 0x7f9928070c20 2024-04-25T10:16:13.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f55b5400640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5590066f60 0x7f5590069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f55a40b2000 tx=0x7f55a40ab070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f91ed600640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91d0066fb0 0x7f91d0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f9924c00640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f9924c00640 1 -- 172.21.15.87:0/2284643144 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f991c084210 con 0x7f9928070c20 2024-04-25T10:16:13.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f91ed600640 1 -- 172.21.15.87:0/666084024 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91dc089880 con 0x7f91f006e480 2024-04-25T10:16:13.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f9927e00640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f91f5c00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91d0066fb0 0x7f91d0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f5eff400640 1 -- 172.21.15.87:0/1591016895 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ef40c3070 con 0x7f5f080aeb50 2024-04-25T10:16:13.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f5eff400640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ee8066fb0 0x7f5ee8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f5eff400640 1 -- 172.21.15.87:0/1591016895 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ef411e3d0 con 0x7f5f080aeb50 2024-04-25T10:16:13.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f5f07400640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ee8066fb0 0x7f5ee8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f91f5c00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91d0066fb0 0x7f91d0069470 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f91e40ad000 tx=0x7f91e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f9927e00640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f992815a190 tx=0x7f9914009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.130+0000 7f5f07400640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ee8066fb0 0x7f5ee8069470 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009580 tx=0x7f5ef8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.134+0000 7f91ed600640 1 -- 172.21.15.87:0/666084024 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91dc026020 con 0x7f91f006e480 2024-04-25T10:16:13.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.134+0000 7f55ad600640 1 -- 172.21.15.87:0/3171196777 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f559c055b70 con 0x7f55b00cf930 2024-04-25T10:16:13.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.134+0000 7f9924c00640 1 -- 172.21.15.87:0/2284643144 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f991c054b30 con 0x7f9928070c20 2024-04-25T10:16:13.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.138+0000 7f5eff400640 1 -- 172.21.15.87:0/1591016895 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ef40ac020 con 0x7f5f080aeb50 2024-04-25T10:16:13.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.358+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f99280aee10 con 0x7f9928070c20 2024-04-25T10:16:13.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.358+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f55b006d960 con 0x7f55b00cf930 2024-04-25T10:16:13.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.366+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f91f0074380 con 0x7f91f006e480 2024-04-25T10:16:13.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:13.366+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f5f080b70e0 con 0x7f5f080aeb50 2024-04-25T10:16:14.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.278+0000 7f9924c00640 1 -- 172.21.15.87:0/2284643144 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6510 v38) v1 ==== 95+0+0 (secure 0 0 0) 0x7f991c058bd0 con 0x7f9928070c20 2024-04-25T10:16:14.286 INFO:teuthology.orchestra.run.smithi087.stderr:failed mds gid 6510 2024-04-25T10:16:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 msgr2=0x7f9908069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f992815a190 tx=0x7f9914009040 comp rx=0 tx=0).stop 2024-04-25T10:16:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928070c20 msgr2=0x7f99281506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928070c20 0x7f99281506f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c390 tx=0x7f991c00c850 comp rx=0 tx=0).stop 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 shutdown_connections 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99280cf930 0x7f99281573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99280ae020 0x7f992815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 --2- 172.21.15.87:0/2284643144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928070c20 0x7f99281506f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 >> 172.21.15.87:0/2284643144 conn(0x7f9928066b70 msgr2=0x7f992806c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 shutdown_connections 2024-04-25T10:16:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.282+0000 7f992f800640 1 -- 172.21.15.87:0/2284643144 wait complete. 2024-04-25T10:16:14.299 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:16:14.294+0000 7ff591000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:16:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.294+0000 7f55ad600640 1 -- 172.21.15.87:0/3171196777 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6073 v39) v1 ==== 95+0+0 (secure 0 0 0) 0x7f559c059c10 con 0x7f55b00cf930 2024-04-25T10:16:14.304 INFO:teuthology.orchestra.run.smithi087.stderr:failed mds gid 6073 2024-04-25T10:16:14.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.298+0000 7f5eff400640 1 -- 172.21.15.87:0/1591016895 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6513 v39) v1 ==== 95+0+0 (secure 0 0 0) 0x7f5ef4015f90 con 0x7f5f080aeb50 2024-04-25T10:16:14.306 INFO:teuthology.orchestra.run.smithi087.stderr:failed mds gid 6513 2024-04-25T10:16:14.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.298+0000 7f91ed600640 1 -- 172.21.15.87:0/666084024 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5660 v39) v1 ==== 95+0+0 (secure 0 0 0) 0x7f91dc014040 con 0x7f91f006e480 2024-04-25T10:16:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:failed mds gid 5660 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5590066f60 msgr2=0x7f5590069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5590066f60 0x7f5590069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f55a40b2000 tx=0x7f55a40ab070 comp rx=0 tx=0).stop 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 msgr2=0x7f55b01fe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b01fe800 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f559c00e820 tx=0x7f559c00ece0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 shutdown_connections 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5590066f60 0x7f5590069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b015bc40 0x7f55b01fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55b00cf930 0x7f55b01fe800 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 --2- 172.21.15.87:0/3171196777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00cda10 0x7f55b00ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 >> 172.21.15.87:0/3171196777 conn(0x7f55b0066b70 msgr2=0x7f55b015ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 shutdown_connections 2024-04-25T10:16:14.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f55b7e00640 1 -- 172.21.15.87:0/3171196777 wait complete. 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91d0066fb0 msgr2=0x7f91d0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f91e2a00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91d0066fb0 0x7f91d0069470 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f91e40ad000 tx=0x7f91e4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f006e480 msgr2=0x7f91f01470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f91e2a00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f006e480 0x7f91f01470c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f91dc012f90 tx=0x7f91dc010490 comp rx=0 tx=0).stop 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ee8066fb0 msgr2=0x7f5ee8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f5f0e000640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ee8066fb0 0x7f5ee8069470 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009580 tx=0x7f5ef8009190 comp rx=0 tx=0).stop 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f080aeb50 msgr2=0x7f5f0814cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.302+0000 7f5f0e000640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f080aeb50 0x7f5f0814cdd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40a9000 tx=0x7f5ef4018ff0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 shutdown_connections 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ee8066fb0 0x7f5ee8069470 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f080b0b80 0x7f5f080b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f080aeb50 0x7f5f0814cdd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 --2- 172.21.15.87:0/1591016895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f0806af30 0x7f5f080c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 >> 172.21.15.87:0/1591016895 conn(0x7f5f080d1900 msgr2=0x7f5f0806b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 shutdown_connections 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f5f0e000640 1 -- 172.21.15.87:0/1591016895 wait complete. 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 shutdown_connections 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91d0066fb0 0x7f91d0069470 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0072260 0x7f91f0147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f006e480 0x7f91f01470c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 --2- 172.21.15.87:0/666084024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f006e0a0 0x7f91f01584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 >> 172.21.15.87:0/666084024 conn(0x7f91f00d00a0 msgr2=0x7f91f01595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 shutdown_connections 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.306+0000 7f91e2a00640 1 -- 172.21.15.87:0/666084024 wait complete. 2024-04-25T10:16:14.321 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:16:14.321 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:16:14.324 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:16:14.325 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:14.337 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:16:14.338 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:16:14.340 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:16:14.340 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:16:14.341 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:16:14.341 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:14.342 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:16:14.342 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:16:14.346 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:16:14.346 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:14.389 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:16:14.389 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:14.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:14.412 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:16:14.432 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:16:14.432 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:16:14.476 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:16:14.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfb4c00640 1 Processor -- start 2024-04-25T10:16:14.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfb4c00640 1 -- start start 2024-04-25T10:16:14.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfb4c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcfb01550b0 0x7fcfb00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfb4c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfb015b910 0x7fcfb00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfb4c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb0071050 con 0x7fcfb00cfe70 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfb4c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb015bd80 con 0x7fcfb015b910 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfb4c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb015d040 con 0x7fcfb01550b0 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfad400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcfb00cfe70 0x7fcfb015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60144/0 (socket says 172.21.15.87:60144) 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfad400640 1 -- 172.21.15.87:0/893939634 learned_addr learned my addr 172.21.15.87:0/893939634 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfae800640 1 --2- 172.21.15.87:0/893939634 >> v2:172.21.15.87:3300/0 conn(0x7fcfb015b910 0x7fcfb00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfade00640 1 --2- 172.21.15.87:0/893939634 >> v2:172.21.15.169:3300/0 conn(0x7fcfb01550b0 0x7fcfb00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfae800640 1 -- 172.21.15.87:0/893939634 >> v2:172.21.15.169:3300/0 conn(0x7fcfb01550b0 msgr2=0x7fcfb00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfae800640 1 --2- 172.21.15.87:0/893939634 >> v2:172.21.15.169:3300/0 conn(0x7fcfb01550b0 0x7fcfb00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfae800640 1 -- 172.21.15.87:0/893939634 >> v1:172.21.15.87:6789/0 conn(0x7fcfb00cfe70 legacy=0x7fcfb015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfae800640 1 -- 172.21.15.87:0/893939634 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb00712f0 con 0x7fcfb015b910 2024-04-25T10:16:14.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.526+0000 7fcfaca00640 1 -- 172.21.15.87:0/893939634 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811800903 0 0) 0x7fcfb0071050 con 0x7fcfb00cfe70 2024-04-25T10:16:14.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfae800640 1 --2- 172.21.15.87:0/893939634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb015b910 0x7fcfb00b3260 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009d90 tx=0x7fcfa801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f337099f77f14d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfaca00640 1 -- 172.21.15.87:0/893939634 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80b2070 con 0x7fcfb015b910 2024-04-25T10:16:14.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfaca00640 1 -- 172.21.15.87:0/893939634 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa80a58e0 con 0x7fcfb015b910 2024-04-25T10:16:14.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfaca00640 1 -- 172.21.15.87:0/893939634 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80a5cb0 con 0x7fcfb015b910 2024-04-25T10:16:14.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 -- 172.21.15.87:0/893939634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb015b910 msgr2=0x7fcfb00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/893939634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb015b910 0x7fcfb00b3260 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009d90 tx=0x7fcfa801c930 comp rx=0 tx=0).stop 2024-04-25T10:16:14.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 -- 172.21.15.87:0/893939634 shutdown_connections 2024-04-25T10:16:14.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/893939634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb015b910 0x7fcfb00b3260 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/893939634 >> v2:172.21.15.169:3300/0 conn(0x7fcfb01550b0 0x7fcfb00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 -- 172.21.15.87:0/893939634 >> 172.21.15.87:0/893939634 conn(0x7fcfb0066b70 msgr2=0x7fcfb0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 -- 172.21.15.87:0/893939634 shutdown_connections 2024-04-25T10:16:14.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.530+0000 7fcfb4c00640 1 -- 172.21.15.87:0/893939634 wait complete. 2024-04-25T10:16:14.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 Processor -- start 2024-04-25T10:16:14.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 -- start start 2024-04-25T10:16:14.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb00cfe70 0x7fcfb013c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfb01550b0 0x7fcfb014f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfb015b910 0x7fcfb00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb015c030 con 0x7fcfb00cfe70 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00720b0 con 0x7fcfb015b910 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb015c1b0 con 0x7fcfb01550b0 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb00cfe70 0x7fcfb013c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb00cfe70 0x7fcfb013c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49090/0 (socket says 172.21.15.87:49090) 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 -- 172.21.15.87:0/4042085696 learned_addr learned my addr 172.21.15.87:0/4042085696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfae800640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfb015b910 0x7fcfb00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfad400640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfb01550b0 0x7fcfb014f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 -- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfb01550b0 msgr2=0x7fcfb014f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfb01550b0 0x7fcfb014f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 -- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfb015b910 msgr2=0x7fcfb00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfb015b910 0x7fcfb00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 -- 172.21.15.87:0/4042085696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb013c620 con 0x7fcfb00cfe70 2024-04-25T10:16:14.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfade00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb00cfe70 0x7fcfb013c0e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e930 tx=0x7fcfa000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfa6000640 1 -- 172.21.15.87:0/4042085696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0017070 con 0x7fcfb00cfe70 2024-04-25T10:16:14.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfa6000640 1 -- 172.21.15.87:0/4042085696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa000ca00 con 0x7fcfb00cfe70 2024-04-25T10:16:14.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb014fce0 con 0x7fcfb00cfe70 2024-04-25T10:16:14.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfa6000640 1 -- 172.21.15.87:0/4042085696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000cdb0 con 0x7fcfb00cfe70 2024-04-25T10:16:14.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb01502a0 con 0x7fcfb00cfe70 2024-04-25T10:16:14.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.534+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb004ef30 con 0x7fcfb00cfe70 2024-04-25T10:16:14.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.538+0000 7fcfa6000640 1 -- 172.21.15.87:0/4042085696 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfa001a070 con 0x7fcfb00cfe70 2024-04-25T10:16:14.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.538+0000 7fcfa6000640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.538+0000 7fcfa6000640 1 -- 172.21.15.87:0/4042085696 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcfa0085130 con 0x7fcfb00cfe70 2024-04-25T10:16:14.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.538+0000 7fcfad400640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.538+0000 7fcfad400640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0149510 tx=0x7fcf98007d60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.542+0000 7fcfa6000640 1 -- 172.21.15.87:0/4042085696 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfa0055a20 con 0x7fcfb00cfe70 2024-04-25T10:16:14.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.726+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfb00670f0 con 0x7fcfb00cfe70 2024-04-25T10:16:14.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.726+0000 7fcfa6000640 1 -- 172.21.15.87:0/4042085696 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+1767 (secure 0 0 0) 0x7fcfa0059ac0 con 0x7fcfb00cfe70 2024-04-25T10:16:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:14.735 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:14.297709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:14.736 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 39 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 msgr2=0x7fcf8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0149510 tx=0x7fcf98007d60 comp rx=0 tx=0).stop 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb00cfe70 msgr2=0x7fcfb013c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb00cfe70 0x7fcfb013c0e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e930 tx=0x7fcfa000edf0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 shutdown_connections 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfb015b910 0x7fcfb00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfb01550b0 0x7fcfb014f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 --2- 172.21.15.87:0/4042085696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb00cfe70 0x7fcfb013c0e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 >> 172.21.15.87:0/4042085696 conn(0x7fcfb0066b70 msgr2=0x7fcfb00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 shutdown_connections 2024-04-25T10:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:14.730+0000 7fcfb4c00640 1 -- 172.21.15.87:0/4042085696 wait complete. 2024-04-25T10:16:15.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:02.576203+0000', 'modified': '2024-04-25T10:16:14.297709+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:15.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-25T10:16:16.159 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:16.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92d6800640 1 Processor -- start 2024-04-25T10:16:16.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92d6800640 1 -- start start 2024-04-25T10:16:16.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92d6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92d01550b0 0x7f92d00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92d6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f92d00cfe70 0x7f92d015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92d6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d0071050 con 0x7f92d015b740 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92d6800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d015d040 con 0x7f92d01550b0 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92d6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d00712f0 con 0x7f92d00cfe70 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92d01550b0 0x7f92d00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92d01550b0 0x7f92d00cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49104/0 (socket says 172.21.15.87:49104) 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cf400640 1 -- 172.21.15.87:0/2473518053 learned_addr learned my addr 172.21.15.87:0/2473518053 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cea00640 1 --2- 172.21.15.87:0/2473518053 >> v2:172.21.15.169:3300/0 conn(0x7f92d00cfe70 0x7f92d015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cf400640 1 -- 172.21.15.87:0/2473518053 >> v2:172.21.15.169:3300/0 conn(0x7f92d00cfe70 msgr2=0x7f92d015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cf400640 1 --2- 172.21.15.87:0/2473518053 >> v2:172.21.15.169:3300/0 conn(0x7f92d00cfe70 0x7f92d015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cf400640 1 -- 172.21.15.87:0/2473518053 >> v1:172.21.15.169:6789/0 conn(0x7f92d015b740 legacy=0x7f92d006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:16.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.302+0000 7f92cf400640 1 -- 172.21.15.87:0/2473518053 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d00715d0 con 0x7f92d01550b0 2024-04-25T10:16:16.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92cf400640 1 --2- 172.21.15.87:0/2473518053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d00cf930 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009870 tx=0x7f92b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afecd8390d40ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:16.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92ce000640 1 -- 172.21.15.87:0/2473518053 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80b0070 con 0x7f92d01550b0 2024-04-25T10:16:16.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92ce000640 1 -- 172.21.15.87:0/2473518053 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b80ab600 con 0x7f92d01550b0 2024-04-25T10:16:16.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92ce000640 1 -- 172.21.15.87:0/2473518053 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80ab9d0 con 0x7f92d01550b0 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 -- 172.21.15.87:0/2473518053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 msgr2=0x7f92d00cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 --2- 172.21.15.87:0/2473518053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d00cf930 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009870 tx=0x7f92b801c520 comp rx=0 tx=0).stop 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 -- 172.21.15.87:0/2473518053 shutdown_connections 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 --2- 172.21.15.87:0/2473518053 >> v2:172.21.15.169:3300/0 conn(0x7f92d00cfe70 0x7f92d015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 --2- 172.21.15.87:0/2473518053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d00cf930 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 -- 172.21.15.87:0/2473518053 >> 172.21.15.87:0/2473518053 conn(0x7f92d0066b70 msgr2=0x7f92d0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 -- 172.21.15.87:0/2473518053 shutdown_connections 2024-04-25T10:16:16.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.306+0000 7f92d6800640 1 -- 172.21.15.87:0/2473518053 wait complete. 2024-04-25T10:16:16.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 Processor -- start 2024-04-25T10:16:16.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 -- start start 2024-04-25T10:16:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92d00cfe70 0x7f92d0146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d0157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92d015b740 0x7f92d014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d015bcd0 con 0x7f92d01550b0 2024-04-25T10:16:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d015bb50 con 0x7f92d00cfe70 2024-04-25T10:16:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d015c030 con 0x7f92d015b740 2024-04-25T10:16:16.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d0157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d0157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49116/0 (socket says 172.21.15.87:49116) 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 -- 172.21.15.87:0/1712228891 learned_addr learned my addr 172.21.15.87:0/1712228891 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cf400640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92d00cfe70 0x7f92d0146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cfe00640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92d015b740 0x7f92d014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 -- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92d015b740 msgr2=0x7f92d014a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92d015b740 0x7f92d014a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 -- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92d00cfe70 msgr2=0x7f92d0146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92d00cfe70 0x7f92d0146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 -- 172.21.15.87:0/1712228891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d0155da0 con 0x7f92d01550b0 2024-04-25T10:16:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92cea00640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d0157b10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d830 tx=0x7f92c000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92c7400640 1 -- 172.21.15.87:0/1712228891 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c000b6c0 con 0x7f92d01550b0 2024-04-25T10:16:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92c7400640 1 -- 172.21.15.87:0/1712228891 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c0019070 con 0x7f92d01550b0 2024-04-25T10:16:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d0156060 con 0x7f92d01550b0 2024-04-25T10:16:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92c7400640 1 -- 172.21.15.87:0/1712228891 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c000ba60 con 0x7f92d01550b0 2024-04-25T10:16:16.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d0156620 con 0x7f92d01550b0 2024-04-25T10:16:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.310+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d004ef30 con 0x7f92d01550b0 2024-04-25T10:16:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.314+0000 7f92c7400640 1 -- 172.21.15.87:0/1712228891 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92c001e460 con 0x7f92d01550b0 2024-04-25T10:16:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.314+0000 7f92c7400640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92ac066f90 0x7f92ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:16.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.314+0000 7f92c7400640 1 -- 172.21.15.87:0/1712228891 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f92c0085620 con 0x7f92d01550b0 2024-04-25T10:16:16.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.314+0000 7f92cf400640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92ac066f90 0x7f92ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:16.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.314+0000 7f92cf400640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92ac066f90 0x7f92ac069450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f92b80ad000 tx=0x7f92b80a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:16.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.318+0000 7f92c7400640 1 -- 172.21.15.87:0/1712228891 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92c0055f80 con 0x7f92d01550b0 2024-04-25T10:16:16.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.502+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92d006e3a0 con 0x7f92d01550b0 2024-04-25T10:16:16.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92c7400640 1 -- 172.21.15.87:0/1712228891 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f92c005a020 con 0x7f92d01550b0 2024-04-25T10:16:16.512 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:16.512 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:15.324914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":41,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.87:6837/427681613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":427681613},{"type":"v1","addr":"172.21.15.87:6837","nonce":427681613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:16.512 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 42 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92ac066f90 msgr2=0x7f92ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92ac066f90 0x7f92ac069450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f92b80ad000 tx=0x7f92b80a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 msgr2=0x7f92d0157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d0157b10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d830 tx=0x7f92c000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 shutdown_connections 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f92ac066f90 0x7f92ac069450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92d015b740 0x7f92d014a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92d01550b0 0x7f92d0157b10 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 --2- 172.21.15.87:0/1712228891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92d00cfe70 0x7f92d0146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 >> 172.21.15.87:0/1712228891 conn(0x7f92d0066b70 msgr2=0x7f92d00b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 shutdown_connections 2024-04-25T10:16:16.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:16.506+0000 7f92d6800640 1 -- 172.21.15.87:0/1712228891 wait complete. 2024-04-25T10:16:16.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:02.576203+0000', 'modified': '2024-04-25T10:16:15.324914+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6768}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6768': {'gid': 6768, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.87:6837/427681613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6836', 'nonce': 427681613}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 427681613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:16.937 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6768:up:reconnect 2024-04-25T10:16:17.937 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:18.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f83a5e00640 1 Processor -- start 2024-04-25T10:16:18.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f83a5e00640 1 -- start start 2024-04-25T10:16:18.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f83a5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f83a015b9e0 0x7f83a00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f83a5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a00d1c30 0x7f83a00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f83a5e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00ccfb0 con 0x7f83a015bdc0 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f83a5e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a00cd290 con 0x7f83a015b9e0 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f83a5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00cd570 con 0x7f83a00d1c30 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f83a015b9e0 0x7f83a00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f83a015b9e0 0x7f83a00cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35852/0 (socket says 172.21.15.87:35852) 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839ea00640 1 -- 172.21.15.87:0/1470584496 learned_addr learned my addr 172.21.15.87:0/1470584496 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839e000640 1 --1- 172.21.15.87:0/1470584496 >> v1:172.21.15.87:6789/0 conn(0x7f83a015bdc0 0x7f83a00d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47402/0 (socket says 172.21.15.87:47402) 2024-04-25T10:16:18.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839f400640 1 --2- 172.21.15.87:0/1470584496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a00d1c30 0x7f83a00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839ea00640 1 -- 172.21.15.87:0/1470584496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a00d1c30 msgr2=0x7f83a00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839ea00640 1 --2- 172.21.15.87:0/1470584496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a00d1c30 0x7f83a00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839ea00640 1 -- 172.21.15.87:0/1470584496 >> v1:172.21.15.87:6789/0 conn(0x7f83a015bdc0 legacy=0x7f83a00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.078+0000 7f839ea00640 1 -- 172.21.15.87:0/1470584496 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a00cd850 con 0x7f83a015b9e0 2024-04-25T10:16:18.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f839ea00640 1 --2- 172.21.15.87:0/1470584496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a015b9e0 0x7f83a00cc410 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8390009e50 tx=0x7f839001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c107db08d51002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f839d600640 1 -- 172.21.15.87:0/1470584496 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83900a5710 con 0x7f83a015b9e0 2024-04-25T10:16:18.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f839d600640 1 -- 172.21.15.87:0/1470584496 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83900a58b0 con 0x7f83a015b9e0 2024-04-25T10:16:18.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f839d600640 1 -- 172.21.15.87:0/1470584496 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83900a5c80 con 0x7f83a015b9e0 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 -- 172.21.15.87:0/1470584496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a015b9e0 msgr2=0x7f83a00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 --2- 172.21.15.87:0/1470584496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a015b9e0 0x7f83a00cc410 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8390009e50 tx=0x7f839001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 -- 172.21.15.87:0/1470584496 shutdown_connections 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 --2- 172.21.15.87:0/1470584496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a00d1c30 0x7f83a00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 --2- 172.21.15.87:0/1470584496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a015b9e0 0x7f83a00cc410 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 -- 172.21.15.87:0/1470584496 >> 172.21.15.87:0/1470584496 conn(0x7f83a009a9d0 msgr2=0x7f83a009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 -- 172.21.15.87:0/1470584496 shutdown_connections 2024-04-25T10:16:18.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.082+0000 7f83a5e00640 1 -- 172.21.15.87:0/1470584496 wait complete. 2024-04-25T10:16:18.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 Processor -- start 2024-04-25T10:16:18.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 -- start start 2024-04-25T10:16:18.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a00d1c30 0x7f83a0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a015b9e0 0x7f83a00ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a015bdc0 0x7f83a00ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00c6630 con 0x7f83a00d1c30 2024-04-25T10:16:18.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00c64b0 con 0x7f83a015bdc0 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00c67b0 con 0x7f83a015b9e0 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a00d1c30 0x7f83a0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a00d1c30 0x7f83a0154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35858/0 (socket says 172.21.15.87:35858) 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 -- 172.21.15.87:0/2555229994 learned_addr learned my addr 172.21.15.87:0/2555229994 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839f400640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a015bdc0 0x7f83a00ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839e000640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a015b9e0 0x7f83a00ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 -- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a015b9e0 msgr2=0x7f83a00ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a015b9e0 0x7f83a00ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 -- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a015bdc0 msgr2=0x7f83a00ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a015bdc0 0x7f83a00ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 -- 172.21.15.87:0/2555229994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a015c2e0 con 0x7f83a00d1c30 2024-04-25T10:16:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f839ea00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a00d1c30 0x7f83a0154be0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f839001a5d0 tx=0x7f8390019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f8397400640 1 -- 172.21.15.87:0/2555229994 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83900bc020 con 0x7f83a00d1c30 2024-04-25T10:16:18.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f8397400640 1 -- 172.21.15.87:0/2555229994 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83900b8670 con 0x7f83a00d1c30 2024-04-25T10:16:18.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f8397400640 1 -- 172.21.15.87:0/2555229994 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83900b8a40 con 0x7f83a00d1c30 2024-04-25T10:16:18.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a015e6b0 con 0x7f83a00d1c30 2024-04-25T10:16:18.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a015c880 con 0x7f83a00d1c30 2024-04-25T10:16:18.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.086+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a004ef30 con 0x7f83a00d1c30 2024-04-25T10:16:18.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.090+0000 7f8397400640 1 -- 172.21.15.87:0/2555229994 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83900bf020 con 0x7f83a00d1c30 2024-04-25T10:16:18.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.090+0000 7f8397400640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8380066f60 0x7f8380069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.090+0000 7f8397400640 1 -- 172.21.15.87:0/2555229994 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f83901201e0 con 0x7f83a00d1c30 2024-04-25T10:16:18.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.090+0000 7f839e000640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8380066f60 0x7f8380069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.090+0000 7f839e000640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8380066f60 0x7f8380069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f83a015d7d0 tx=0x7f838c007c50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.094+0000 7f8397400640 1 -- 172.21.15.87:0/2555229994 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83900c6030 con 0x7f83a00d1c30 2024-04-25T10:16:18.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.282+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83a0155120 con 0x7f83a00d1c30 2024-04-25T10:16:18.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.282+0000 7f8397400640 1 -- 172.21.15.87:0/2555229994 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f8390016620 con 0x7f83a00d1c30 2024-04-25T10:16:18.290 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:18.290 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:16.314671+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":41,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.87:6837/427681613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":427681613},{"type":"v1","addr":"172.21.15.87:6837","nonce":427681613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:18.290 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 43 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8380066f60 msgr2=0x7f8380069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8380066f60 0x7f8380069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f83a015d7d0 tx=0x7f838c007c50 comp rx=0 tx=0).stop 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a00d1c30 msgr2=0x7f83a0154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a00d1c30 0x7f83a0154be0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f839001a5d0 tx=0x7f8390019740 comp rx=0 tx=0).stop 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 shutdown_connections 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8380066f60 0x7f8380069420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a015bdc0 0x7f83a00ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a015b9e0 0x7f83a00ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 --2- 172.21.15.87:0/2555229994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83a00d1c30 0x7f83a0154be0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 >> 172.21.15.87:0/2555229994 conn(0x7f83a009a9d0 msgr2=0x7f83a00c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 shutdown_connections 2024-04-25T10:16:18.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:18.286+0000 7f83a5e00640 1 -- 172.21.15.87:0/2555229994 wait complete. 2024-04-25T10:16:18.714 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:02.576203+0000', 'modified': '2024-04-25T10:16:16.314671+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6768}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6768': {'gid': 6768, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.87:6837/427681613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6836', 'nonce': 427681613}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 427681613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:18.714 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6768:up:rejoin 2024-04-25T10:16:19.715 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:19.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639c400640 1 Processor -- start 2024-04-25T10:16:19.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639c400640 1 -- start start 2024-04-25T10:16:19.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63941550b0 0x7f63940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63940cfe70 0x7f639415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639c400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6394071050 con 0x7f639415b740 2024-04-25T10:16:19.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639415d040 con 0x7f63940cfe70 2024-04-25T10:16:19.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63940712f0 con 0x7f63941550b0 2024-04-25T10:16:19.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63940cfe70 0x7f639415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63940cfe70 0x7f639415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35864/0 (socket says 172.21.15.87:35864) 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f639a400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f639415b740 0x7f639406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47418/0 (socket says 172.21.15.87:47418) 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399000640 1 -- 172.21.15.87:0/409503207 learned_addr learned my addr 172.21.15.87:0/409503207 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399a00640 1 --2- 172.21.15.87:0/409503207 >> v2:172.21.15.169:3300/0 conn(0x7f63941550b0 0x7f63940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399000640 1 -- 172.21.15.87:0/409503207 >> v2:172.21.15.169:3300/0 conn(0x7f63941550b0 msgr2=0x7f63940cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399000640 1 --2- 172.21.15.87:0/409503207 >> v2:172.21.15.169:3300/0 conn(0x7f63941550b0 0x7f63940cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399000640 1 -- 172.21.15.87:0/409503207 >> v1:172.21.15.87:6789/0 conn(0x7f639415b740 legacy=0x7f639406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399000640 1 -- 172.21.15.87:0/409503207 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940715d0 con 0x7f63940cfe70 2024-04-25T10:16:19.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.854+0000 7f6399a00640 1 --2- 172.21.15.87:0/409503207 >> v2:172.21.15.169:3300/0 conn(0x7f63941550b0 0x7f63940cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:19.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f6399000640 1 --2- 172.21.15.87:0/409503207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63940cfe70 0x7f639415b200 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f638001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ceff00d255fcde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f6393e00640 1 -- 172.21.15.87:0/409503207 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800b0070 con 0x7f63940cfe70 2024-04-25T10:16:19.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f6393e00640 1 -- 172.21.15.87:0/409503207 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63800ab600 con 0x7f63940cfe70 2024-04-25T10:16:19.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f6393e00640 1 -- 172.21.15.87:0/409503207 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800ab9d0 con 0x7f63940cfe70 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 -- 172.21.15.87:0/409503207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63940cfe70 msgr2=0x7f639415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 --2- 172.21.15.87:0/409503207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63940cfe70 0x7f639415b200 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f638001c520 comp rx=0 tx=0).stop 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 -- 172.21.15.87:0/409503207 shutdown_connections 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 --2- 172.21.15.87:0/409503207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63940cfe70 0x7f639415b200 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 --2- 172.21.15.87:0/409503207 >> v2:172.21.15.169:3300/0 conn(0x7f63941550b0 0x7f63940cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 -- 172.21.15.87:0/409503207 >> 172.21.15.87:0/409503207 conn(0x7f6394066b70 msgr2=0x7f6394066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 -- 172.21.15.87:0/409503207 shutdown_connections 2024-04-25T10:16:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.858+0000 7f639c400640 1 -- 172.21.15.87:0/409503207 wait complete. 2024-04-25T10:16:19.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 Processor -- start 2024-04-25T10:16:19.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 -- start start 2024-04-25T10:16:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940cfe70 0x7f6394148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63941550b0 0x7f6394139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639415b740 0x7f639406c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639415bcd0 con 0x7f63941550b0 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639415bb50 con 0x7f639415b740 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639415c030 con 0x7f63940cfe70 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63941550b0 0x7f6394139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63941550b0 0x7f6394139f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35880/0 (socket says 172.21.15.87:35880) 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 -- 172.21.15.87:0/1816926552 learned_addr learned my addr 172.21.15.87:0/1816926552 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639a400640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639415b740 0x7f639406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399a00640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940cfe70 0x7f6394148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 -- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940cfe70 msgr2=0x7f6394148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940cfe70 0x7f6394148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 -- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639415b740 msgr2=0x7f639406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639415b740 0x7f639406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 -- 172.21.15.87:0/1816926552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394148ba0 con 0x7f63941550b0 2024-04-25T10:16:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399a00640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940cfe70 0x7f6394148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6399000640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63941550b0 0x7f6394139f90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f63800ae040 tx=0x7f63800192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6391600640 1 -- 172.21.15.87:0/1816926552 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800b0040 con 0x7f63941550b0 2024-04-25T10:16:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6391600640 1 -- 172.21.15.87:0/1816926552 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63800ab400 con 0x7f63941550b0 2024-04-25T10:16:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6394148e80 con 0x7f63941550b0 2024-04-25T10:16:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f6391600640 1 -- 172.21.15.87:0/1816926552 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800b4680 con 0x7f63941550b0 2024-04-25T10:16:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.862+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6394149440 con 0x7f63941550b0 2024-04-25T10:16:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.866+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6394149680 con 0x7f63941550b0 2024-04-25T10:16:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.866+0000 7f6391600640 1 -- 172.21.15.87:0/1816926552 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63800bb020 con 0x7f63941550b0 2024-04-25T10:16:19.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.866+0000 7f6391600640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6374066f90 0x7f6374069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.866+0000 7f6391600640 1 -- 172.21.15.87:0/1816926552 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f638011e5e0 con 0x7f63941550b0 2024-04-25T10:16:19.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.866+0000 7f6399a00640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6374066f90 0x7f6374069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.866+0000 7f6399a00640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6374066f90 0x7f6374069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6388009980 tx=0x7f6388008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:19.870+0000 7f6391600640 1 -- 172.21.15.87:0/1816926552 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6394149680 con 0x7f63941550b0 2024-04-25T10:16:20.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.058+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63940cf4e0 con 0x7f63941550b0 2024-04-25T10:16:20.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.058+0000 7f6391600640 1 -- 172.21.15.87:0/1816926552 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f63800c2020 con 0x7f63941550b0 2024-04-25T10:16:20.067 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:20.067 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:17.322234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6837/427681613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":427681613},{"type":"v1","addr":"172.21.15.87:6837","nonce":427681613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:20.067 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 44 2024-04-25T10:16:20.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6374066f90 msgr2=0x7f6374069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6374066f90 0x7f6374069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6388009980 tx=0x7f6388008040 comp rx=0 tx=0).stop 2024-04-25T10:16:20.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63941550b0 msgr2=0x7f6394139f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63941550b0 0x7f6394139f90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f63800ae040 tx=0x7f63800192c0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 shutdown_connections 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6374066f90 0x7f6374069450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639415b740 0x7f639406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63941550b0 0x7f6394139f90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 --2- 172.21.15.87:0/1816926552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63940cfe70 0x7f6394148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 >> 172.21.15.87:0/1816926552 conn(0x7f6394066b70 msgr2=0x7f639406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 shutdown_connections 2024-04-25T10:16:20.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.062+0000 7f639c400640 1 -- 172.21.15.87:0/1816926552 wait complete. 2024-04-25T10:16:20.321 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-04-25T10:16:20.314+0000 7f33d9a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:20.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:02.576203+0000', 'modified': '2024-04-25T10:16:17.322234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6768}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6768': {'gid': 6768, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.87:6837/427681613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6836', 'nonce': 427681613}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 427681613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:20.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:20.490 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:16:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.618+0000 7f0eee200640 1 Processor -- start 2024-04-25T10:16:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0eee200640 1 -- start start 2024-04-25T10:16:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0eee200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 0x7f0ee80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0eee200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee8070ed0 con 0x7f0ee80b2fc0 2024-04-25T10:16:20.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0eee200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee80711b0 con 0x7f0ee815a0e0 2024-04-25T10:16:20.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0eee200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8071490 con 0x7f0ee8071f70 2024-04-25T10:16:20.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0ee80b2fc0 0x7f0ee806c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47432/0 (socket says 172.21.15.87:47432) 2024-04-25T10:16:20.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7e00640 1 -- 172.21.15.87:0/2461041797 learned_addr learned my addr 172.21.15.87:0/2461041797 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0edea00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 0x7f0ee80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229100631 0 0) 0x7f0ee8070ed0 con 0x7f0ee80b2fc0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ecc003530 con 0x7f0ee80b2fc0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837786202 0 0) 0x7f0ee80711b0 con 0x7f0ee815a0e0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ee8070ed0 con 0x7f0ee815a0e0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 43671063 0 0) 0x7f0ecc003530 con 0x7f0ee80b2fc0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 msgr2=0x7f0ee80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 0x7f0ee80b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 >> v1:172.21.15.169:6789/0 conn(0x7f0ee815a0e0 legacy=0x7f0ee815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee8071770 con 0x7f0ee80b2fc0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0ee0001680 con 0x7f0ee80b2fc0 2024-04-25T10:16:20.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 >> v1:172.21.15.87:6789/0 conn(0x7f0ee80b2fc0 legacy=0x7f0ee806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ecc003b80 0x7f0ecc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc004810 0x7f0ecc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 0x7f0ecc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8071490 con 0x7f0ecc003b80 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc00ba80 con 0x7f0ecc004810 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc00bc80 con 0x7f0ee8071f70 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ecc003b80 0x7f0ecc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0edea00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc004810 0x7f0ecc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7e00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 0x7f0ecc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7400640 1 -- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 msgr2=0x7f0ecc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee8071f70 0x7f0ecc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7400640 1 -- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc004810 msgr2=0x7f0ecc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc004810 0x7f0ecc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0ee7400640 1 -- 172.21.15.87:0/2461041797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc003530 con 0x7f0ecc003b80 2024-04-25T10:16:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.622+0000 7f0edea00640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc004810 0x7f0ecc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:20.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ecc003b80 0x7f0ecc003f80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8001fd0 tx=0x7f0ed801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef6c08117d8067a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8001c20 con 0x7f0ecc003b80 2024-04-25T10:16:20.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed8003ba0 con 0x7f0ecc003b80 2024-04-25T10:16:20.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0ee6a00640 1 -- 172.21.15.87:0/2461041797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80050d0 con 0x7f0ecc003b80 2024-04-25T10:16:20.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0eee200640 1 -- 172.21.15.87:0/2461041797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ecc003b80 msgr2=0x7f0ecc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0eee200640 1 --2- 172.21.15.87:0/2461041797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ecc003b80 0x7f0ecc003f80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8001fd0 tx=0x7f0ed801c900 comp rx=0 tx=0).stop 2024-04-25T10:16:20.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0ee7e00640 1 -- 172.21.15.87:0/2461041797 reap_dead start 2024-04-25T10:16:20.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0eee200640 1 -- 172.21.15.87:0/2461041797 shutdown_connections 2024-04-25T10:16:20.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0eee200640 1 -- 172.21.15.87:0/2461041797 >> 172.21.15.87:0/2461041797 conn(0x7f0ee8066b70 msgr2=0x7f0ee8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0eee200640 1 -- 172.21.15.87:0/2461041797 shutdown_connections 2024-04-25T10:16:20.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.626+0000 7f0eee200640 1 -- 172.21.15.87:0/2461041797 wait complete. 2024-04-25T10:16:20.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 Processor -- start 2024-04-25T10:16:20.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 -- start start 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc003b80 0x7f0ee815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee81525f0 0x7f0ee8148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee815cfa0 0x7f0ee81489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8070b50 con 0x7f0ee81525f0 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee815a4c0 con 0x7f0ecc003b80 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8072350 con 0x7f0ee815cfa0 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee81525f0 0x7f0ee8148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee81525f0 0x7f0ee8148000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35894/0 (socket says 172.21.15.87:35894) 2024-04-25T10:16:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 -- 172.21.15.87:0/1435359398 learned_addr learned my addr 172.21.15.87:0/1435359398 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0ee7400640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc003b80 0x7f0ee815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0ee7e00640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee815cfa0 0x7f0ee81489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 -- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee815cfa0 msgr2=0x7f0ee81489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee815cfa0 0x7f0ee81489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 -- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc003b80 msgr2=0x7f0ee815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc003b80 0x7f0ee815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 -- 172.21.15.87:0/1435359398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee815b230 con 0x7f0ee81525f0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edea00640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee81525f0 0x7f0ee8148000 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00109f0 tx=0x7f0ed0010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edfe00640 1 -- 172.21.15.87:0/1435359398 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000b040 con 0x7f0ee81525f0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edfe00640 1 -- 172.21.15.87:0/1435359398 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed000ecc0 con 0x7f0ee81525f0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0edfe00640 1 -- 172.21.15.87:0/1435359398 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0004240 con 0x7f0ee81525f0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee815b4d0 con 0x7f0ee81525f0 2024-04-25T10:16:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.630+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee815ba90 con 0x7f0ee81525f0 2024-04-25T10:16:20.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0edfe00640 1 -- 172.21.15.87:0/1435359398 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ed0003620 con 0x7f0ee81525f0 2024-04-25T10:16:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0edfe00640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ec4066f90 0x7f0ec4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0edfe00640 1 -- 172.21.15.87:0/1435359398 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0ed00859b0 con 0x7f0ee81525f0 2024-04-25T10:16:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 Processor -- start 2024-04-25T10:16:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ec4066f90 0x7f0ec4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 -- start start 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ec4066f90 0x7f0ec4069450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80ad000 tx=0x7f0ed80a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5586cd920 0x55b5586cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586ce240 0x55b5586839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 0x55b558684310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55856b5f0 con 0x55b5586cd920 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5584d2da0 con 0x55b558683f30 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b55845b150 con 0x55b5586ce240 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5586cd920 0x55b5586cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5586cd920 0x55b5586cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35900/0 (socket says 172.21.15.87:35900) 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 -- 172.21.15.87:0/1014908250 learned_addr learned my addr 172.21.15.87:0/1014908250 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:20.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 -- 172.21.15.87:0/1014908250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586ce240 msgr2=0x55b5586839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 --2- 172.21.15.87:0/1014908250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586ce240 0x55b5586839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 -- 172.21.15.87:0/1014908250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 msgr2=0x55b558684310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 --2- 172.21.15.87:0/1014908250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 0x55b558684310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.634+0000 7f0ee7400640 1 -- 172.21.15.87:0/1014908250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b558474ca0 con 0x55b5586cd920 2024-04-25T10:16:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 --2- 172.21.15.87:0/1014908250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5586cd920 0x55b5586cdd00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80980f0 tx=0x7f0ed8001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711fde8a8218efd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ed7e00640 1 -- 172.21.15.87:0/1014908250 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80cd070 con 0x55b5586cd920 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ed7e00640 1 -- 172.21.15.87:0/1014908250 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed80c8440 con 0x55b5586cd920 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ed7e00640 1 -- 172.21.15.87:0/1014908250 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80c8810 con 0x55b5586cd920 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- 172.21.15.87:0/1014908250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5586cd920 msgr2=0x55b5586cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/1014908250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5586cd920 0x55b5586cdd00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80980f0 tx=0x7f0ed8001660 comp rx=0 tx=0).stop 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- 172.21.15.87:0/1014908250 shutdown_connections 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/1014908250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 0x55b558684310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/1014908250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586ce240 0x55b5586839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/1014908250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5586cd920 0x55b5586cdd00 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- 172.21.15.87:0/1014908250 >> 172.21.15.87:0/1014908250 conn(0x55b55868a2a0 msgr2=0x55b5586822d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- 172.21.15.87:0/1014908250 shutdown_connections 2024-04-25T10:16:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- 172.21.15.87:0/1014908250 wait complete. 2024-04-25T10:16:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- start start 2024-04-25T10:16:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 Processor -- start 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586d4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b5586d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586ce240 0x55b558687aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55848d990 con 0x55b558683f30 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5584cee40 con 0x55b5586ce240 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b55845e900 con 0x55b5586cd920 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 10 client.0 _ll_register_callbacks cb 0x55b5585861c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586d4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586d4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35908/0 (socket says 172.21.15.87:35908) 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 -- 172.21.15.87:0/2406422659 learned_addr learned my addr 172.21.15.87:0/2406422659 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0edea00640 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b5586d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7e00640 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586ce240 0x55b558687aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 -- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 msgr2=0x55b5586d51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b5586d51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 -- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586ce240 msgr2=0x55b558687aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586ce240 0x55b558687aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 -- 172.21.15.87:0/2406422659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b55848b630 con 0x55b558683f30 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586d4c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80980f0 tx=0x7f0ed80196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf7b016deeb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ed4c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2406422659 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80d5020 con 0x55b558683f30 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2406422659 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0ed80c87f0 con 0x55b558683f30 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.638+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ed80c8440 con 0x55b558683f30 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2406422659 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ed80df050 con 0x55b558683f30 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 10 client.6813 fetch_fsmap learned FSMap version 44 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55b55845b3a0 con 0x55b558683f30 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2406422659 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed80c8440 con 0x55b558683f30 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2406422659 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ed80e2020 con 0x55b558683f30 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 10 client.6813 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 10 client.6813 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 20 client.6813 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586884b0 0x55b558688890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 4 client.6813 mds_command: new command op to 6768 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 --> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b55846ea50 con 0x55b5586884b0 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0edea00640 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586884b0 0x55b558688890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0edea00640 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586884b0 0x55b558688890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee78c3193ac1f6f9 server_cookie=ec90203c222bca22 in_seq=0 out_seq=0 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ed4c00640 10 client.6813 ms_handle_connect on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2406422659 <== mds.0 v2:172.21.15.87:6836/427681613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b55846ea50 con 0x55b5586884b0 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ed4c00640 10 client.6813 handle_command_reply: tid=0 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 client.6813 shutdown 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 msgr2=0x55b5586d4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586d4c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80980f0 tx=0x7f0ed80196a0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 shutdown_connections 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586884b0 0x55b558688890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586ce240 0x55b558687aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ed4c00640 0 client.6813 ms_handle_reset on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b5586d51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2406422659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586d4c80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 >> 172.21.15.87:0/2406422659 conn(0x55b55868a2a0 msgr2=0x55b5586829e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 shutdown_connections 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2406422659 wait complete. 2024-04-25T10:16:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.642+0000 7f0ef16d4000 20 client.6813 trim_cache size 0 max 16384 2024-04-25T10:16:20.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 Processor -- start 2024-04-25T10:16:20.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 -- start start 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b55868a2a0 0x55b558728fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 0x55b558729510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b558729a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8203140 con 0x55b55868a2a0 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee81566f0 con 0x55b558683f30 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8202b90 con 0x55b5586cd920 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b55868a2a0 0x55b558728fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b55868a2a0 0x55b558728fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35920/0 (socket says 172.21.15.87:35920) 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 -- 172.21.15.87:0/2747630242 learned_addr learned my addr 172.21.15.87:0/2747630242 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7e00640 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b558729a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0edea00640 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 0x55b558729510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 -- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 msgr2=0x55b558729a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b558729a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 -- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 msgr2=0x55b558729510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 0x55b558729510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 -- 172.21.15.87:0/2747630242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b558554600 con 0x55b55868a2a0 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b55868a2a0 0x55b558728fd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8005010 tx=0x7f0ed80cf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cc2bbec569dae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ed7400640 1 -- 172.21.15.87:0/2747630242 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80c8e60 con 0x55b55868a2a0 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ed7400640 1 -- 172.21.15.87:0/2747630242 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed8006d60 con 0x55b55868a2a0 2024-04-25T10:16:20.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.658+0000 7f0ed7400640 1 -- 172.21.15.87:0/2747630242 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80d5070 con 0x55b55868a2a0 2024-04-25T10:16:20.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2747630242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b55868a2a0 msgr2=0x55b558728fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b55868a2a0 0x55b558728fd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8005010 tx=0x7f0ed80cf000 comp rx=0 tx=0).stop 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2747630242 shutdown_connections 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b5586cd920 0x55b558729a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b558683f30 0x55b558729510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2747630242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b55868a2a0 0x55b558728fd0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2747630242 >> 172.21.15.87:0/2747630242 conn(0x55b5586d4e20 msgr2=0x55b558687220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2747630242 shutdown_connections 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2747630242 wait complete. 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- start start 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 Processor -- start 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b55868a2a0 0x55b558729a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586cd920 0x55b558729fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5584d73d0 con 0x55b558683f30 2024-04-25T10:16:20.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55858ed50 con 0x55b5586cd920 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b55846d980 con 0x55b55868a2a0 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 10 client.0 _ll_register_callbacks cb 0x7f0ecc003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586da580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35924/0 (socket says 172.21.15.87:35924) 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 -- 172.21.15.87:0/2077837690 learned_addr learned my addr 172.21.15.87:0/2077837690 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0edea00640 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b55868a2a0 0x55b558729a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7e00640 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586cd920 0x55b558729fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 -- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b55868a2a0 msgr2=0x55b558729a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b55868a2a0 0x55b558729a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 -- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586cd920 msgr2=0x55b558729fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586cd920 0x55b558729fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 -- 172.21.15.87:0/2077837690 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5584679b0 con 0x55b558683f30 2024-04-25T10:16:20.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ee7400640 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586da580 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80980f0 tx=0x7f0ed80a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b0f9519ebf2c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ed4c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2077837690 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80e8060 con 0x55b558683f30 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2077837690 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0ed80a8440 con 0x55b558683f30 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ed8006d60 con 0x55b558683f30 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2077837690 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ed80ee050 con 0x55b558683f30 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 10 client.6819 fetch_fsmap learned FSMap version 44 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55b55847a0b0 con 0x55b558683f30 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2077837690 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed8006d60 con 0x55b558683f30 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2077837690 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ed80e2040 con 0x55b558683f30 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 10 client.6819 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 10 client.6819 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 20 client.6819 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586ce240 0x55b55872a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 4 client.6819 mds_command: new command op to 6768 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:20.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.662+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 --> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b5584e02e0 con 0x55b5586ce240 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0edea00640 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586ce240 0x55b55872a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0edea00640 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586ce240 0x55b55872a4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da04874156abad server_cookie=2d6967379049ee6c in_seq=0 out_seq=0 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ed4c00640 10 client.6819 ms_handle_connect on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ed4c00640 1 -- 172.21.15.87:0/2077837690 <== mds.0 v2:172.21.15.87:6836/427681613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b5584e02e0 con 0x55b5586ce240 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ed4c00640 10 client.6819 handle_command_reply: tid=0 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 client.6819 shutdown 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 msgr2=0x55b5586da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586da580 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80980f0 tx=0x7f0ed80a9000 comp rx=0 tx=0).stop 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 shutdown_connections 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55b5586ce240 0x55b55872a4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b5586cd920 0x55b558729fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b55868a2a0 0x55b558729a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ed4c00640 0 client.6819 ms_handle_reset on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:20.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 --2- 172.21.15.87:0/2077837690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b558683f30 0x55b5586da580 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 >> 172.21.15.87:0/2077837690 conn(0x55b5586d4e20 msgr2=0x55b5586d5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 shutdown_connections 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 1 -- 172.21.15.87:0/2077837690 wait complete. 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0ef16d4000 20 client.6819 trim_cache size 0 max 16384 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 6768, 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:16:20.677 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:16:20.678 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:16:20.678 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.3582534610000003, 2024-04-25T10:16:20.678 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 44, 2024-04-25T10:16:20.678 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 61, 2024-04-25T10:16:20.678 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 61, 2024-04-25T10:16:20.678 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 6.1921474859999996 2024-04-25T10:16:20.678 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ec4066f90 msgr2=0x7f0ec4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ec4066f90 0x7f0ec4069450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80ad000 tx=0x7f0ed80a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee81525f0 msgr2=0x7f0ee8148000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee81525f0 0x7f0ee8148000 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00109f0 tx=0x7f0ed0010eb0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 shutdown_connections 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ec4066f90 0x7f0ec4069450 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ee815cfa0 0x7f0ee81489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee81525f0 0x7f0ee8148000 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 --2- 172.21.15.87:0/1435359398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ecc003b80 0x7f0ee815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 >> 172.21.15.87:0/1435359398 conn(0x7f0ee8066b70 msgr2=0x7f0ee806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 shutdown_connections 2024-04-25T10:16:20.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.666+0000 7f0eee200640 1 -- 172.21.15.87:0/1435359398 wait complete. 2024-04-25T10:16:20.691 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:20.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.818+0000 7ff1e1000640 1 Processor -- start 2024-04-25T10:16:20.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.818+0000 7ff1e1000640 1 -- start start 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1e1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc1550b0 0x7ff1dc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1e1000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1dc0cfe70 0x7ff1dc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1e1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc071050 con 0x7ff1dc15b740 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1e1000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc15d040 con 0x7ff1dc0cfe70 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1e1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0712f0 con 0x7ff1dc1550b0 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1dc0cfe70 0x7ff1dc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1dc0cfe70 0x7ff1dc15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35938/0 (socket says 172.21.15.87:35938) 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 -- 172.21.15.87:0/706512836 learned_addr learned my addr 172.21.15.87:0/706512836 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9e00640 1 --2- 172.21.15.87:0/706512836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc1550b0 0x7ff1dc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 -- 172.21.15.87:0/706512836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc1550b0 msgr2=0x7ff1dc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 --2- 172.21.15.87:0/706512836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc1550b0 0x7ff1dc0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 -- 172.21.15.87:0/706512836 >> v1:172.21.15.169:6789/0 conn(0x7ff1dc15b740 legacy=0x7ff1dc06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 -- 172.21.15.87:0/706512836 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0715d0 con 0x7ff1dc0cfe70 2024-04-25T10:16:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9e00640 1 --2- 172.21.15.87:0/706512836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc1550b0 0x7ff1dc0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d9400640 1 --2- 172.21.15.87:0/706512836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc0cfe70 0x7ff1dc15b200 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009870 tx=0x7ff1c801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629ee3eed0929bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.822+0000 7ff1d8a00640 1 -- 172.21.15.87:0/706512836 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80b0070 con 0x7ff1dc0cfe70 2024-04-25T10:16:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1d8a00640 1 -- 172.21.15.87:0/706512836 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c80ab600 con 0x7ff1dc0cfe70 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1d8a00640 1 -- 172.21.15.87:0/706512836 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80ab9d0 con 0x7ff1dc0cfe70 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- 172.21.15.87:0/706512836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc0cfe70 msgr2=0x7ff1dc15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 --2- 172.21.15.87:0/706512836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc0cfe70 0x7ff1dc15b200 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009870 tx=0x7ff1c801c560 comp rx=0 tx=0).stop 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- 172.21.15.87:0/706512836 shutdown_connections 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 --2- 172.21.15.87:0/706512836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc0cfe70 0x7ff1dc15b200 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 --2- 172.21.15.87:0/706512836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc1550b0 0x7ff1dc0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- 172.21.15.87:0/706512836 >> 172.21.15.87:0/706512836 conn(0x7ff1dc066b70 msgr2=0x7ff1dc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- 172.21.15.87:0/706512836 shutdown_connections 2024-04-25T10:16:20.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- 172.21.15.87:0/706512836 wait complete. 2024-04-25T10:16:20.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 Processor -- start 2024-04-25T10:16:20.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- start start 2024-04-25T10:16:20.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1dc0cfe70 0x7ff1dc147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc1550b0 0x7ff1dc14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc15b740 0x7ff1dc14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc15bcd0 con 0x7ff1dc1550b0 2024-04-25T10:16:20.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc15bb50 con 0x7ff1dc0cfe70 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1e1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc15c030 con 0x7ff1dc15b740 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc1550b0 0x7ff1dc14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc1550b0 0x7ff1dc14db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35954/0 (socket says 172.21.15.87:35954) 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1d9400640 1 -- 172.21.15.87:0/698149844 learned_addr learned my addr 172.21.15.87:0/698149844 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1d9e00640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1dc0cfe70 0x7ff1dc147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.826+0000 7ff1da800640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc15b740 0x7ff1dc14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d9400640 1 -- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc15b740 msgr2=0x7ff1dc14f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d9400640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc15b740 0x7ff1dc14f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d9400640 1 -- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1dc0cfe70 msgr2=0x7ff1dc147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d9400640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1dc0cfe70 0x7ff1dc147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d9400640 1 -- 172.21.15.87:0/698149844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc148fc0 con 0x7ff1dc1550b0 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d9400640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc1550b0 0x7ff1dc14db90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80ae040 tx=0x7ff1c801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d2000640 1 -- 172.21.15.87:0/698149844 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80b0040 con 0x7ff1dc1550b0 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d2000640 1 -- 172.21.15.87:0/698149844 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c80bc070 con 0x7ff1dc1550b0 2024-04-25T10:16:20.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d2000640 1 -- 172.21.15.87:0/698149844 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80b6880 con 0x7ff1dc1550b0 2024-04-25T10:16:20.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc14e0d0 con 0x7ff1dc1550b0 2024-04-25T10:16:20.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc148ce0 con 0x7ff1dc1550b0 2024-04-25T10:16:20.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc04ef30 con 0x7ff1dc1550b0 2024-04-25T10:16:20.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d2000640 1 -- 172.21.15.87:0/698149844 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1c80ab480 con 0x7ff1dc1550b0 2024-04-25T10:16:20.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d2000640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1bc066f60 0x7ff1bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.830+0000 7ff1d2000640 1 -- 172.21.15.87:0/698149844 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff1c811fcb0 con 0x7ff1dc1550b0 2024-04-25T10:16:20.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.834+0000 7ff1d9e00640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1bc066f60 0x7ff1bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.834+0000 7ff1d9e00640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1bc066f60 0x7ff1bc069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc003b90 tx=0x7ff1cc009910 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:20.834+0000 7ff1d2000640 1 -- 172.21.15.87:0/698149844 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1c8016150 con 0x7ff1dc1550b0 2024-04-25T10:16:21.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.022+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1dc149370 con 0x7ff1dc1550b0 2024-04-25T10:16:21.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1d2000640 1 -- 172.21.15.87:0/698149844 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff1c80f06f0 con 0x7ff1dc1550b0 2024-04-25T10:16:21.031 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:21.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1bc066f60 msgr2=0x7ff1bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1bc066f60 0x7ff1bc069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc003b90 tx=0x7ff1cc009910 comp rx=0 tx=0).stop 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc1550b0 msgr2=0x7ff1dc14db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc1550b0 0x7ff1dc14db90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80ae040 tx=0x7ff1c801ada0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 shutdown_connections 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff1bc066f60 0x7ff1bc069420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1dc15b740 0x7ff1dc14f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1dc1550b0 0x7ff1dc14db90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 --2- 172.21.15.87:0/698149844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1dc0cfe70 0x7ff1dc147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 >> 172.21.15.87:0/698149844 conn(0x7ff1dc066b70 msgr2=0x7ff1dc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 shutdown_connections 2024-04-25T10:16:21.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.026+0000 7ff1e1000640 1 -- 172.21.15.87:0/698149844 wait complete. 2024-04-25T10:16:21.052 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:21.053 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:21.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.053 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:16:21.062 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:21.063 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42233sec preferred_lft 42233sec 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.064 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.065 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:21.124 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:16:21.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.126 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:16:21.182 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:21.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.183 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:16:21.184 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:16:21.184 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:16:21.184 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:16:21.184 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:16:21.184 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:16:21.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.277 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:16:21.331 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:21.331 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:21.332 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:21.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.332 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:16:21.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:21.394 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:21.447 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:21.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:21.448 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:21.502 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:16:21.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.502 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:21.605 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:21.605 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:21.605 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:21.605 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:16:21.605 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:21.616 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:21.616 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:21.616 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:16:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:16:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:16:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:16:21.620 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:16:21.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.620 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:21.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.640 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:16:21.697 INFO:teuthology.orchestra.run.smithi087.stdout:client5849 2024-04-25T10:16:21.698 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:21.698 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:21.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.747 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:16:21.800 INFO:teuthology.orchestra.run.smithi087.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T10:16:21.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:21.802 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T10:16:21.802 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:16:21.802 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T10:16:21.973 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:21.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:21.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 Processor -- start 2024-04-25T10:16:21.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 -- start start 2024-04-25T10:16:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b800 0x7f0fbc0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fbc06b0d0 0x7f0fbc0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fbc06bbe0 0x7f0fbc0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0cde90 con 0x7f0fbc06bbe0 2024-04-25T10:16:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0cfdb0 con 0x7f0fbc06b0d0 2024-04-25T10:16:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.978+0000 7f0fc1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0d0090 con 0x7f0fbc06b800 2024-04-25T10:16:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fbb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fbc06bbe0 0x7f0fbc0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fba000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fbc06b0d0 0x7f0fbc0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fbaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b800 0x7f0fbc0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fbb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fbc06bbe0 0x7f0fbc0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35966/0 (socket says 172.21.15.87:35966) 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fbb400640 1 -- 172.21.15.87:0/2766237731 learned_addr learned my addr 172.21.15.87:0/2766237731 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fba000640 1 -- 172.21.15.87:0/2766237731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b800 msgr2=0x7f0fbc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fba000640 1 --2- 172.21.15.87:0/2766237731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b800 0x7f0fbc0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fba000640 1 -- 172.21.15.87:0/2766237731 >> v2:172.21.15.87:3300/0 conn(0x7f0fbc06bbe0 msgr2=0x7f0fbc0cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fba000640 1 --2- 172.21.15.87:0/2766237731 >> v2:172.21.15.87:3300/0 conn(0x7f0fbc06bbe0 0x7f0fbc0cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fba000640 1 -- 172.21.15.87:0/2766237731 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0d0370 con 0x7f0fbc06b0d0 2024-04-25T10:16:21.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.982+0000 7f0fba000640 1 --2- 172.21.15.87:0/2766237731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06b0d0 0x7f0fbc0cd410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc074810 tx=0x7f0fa801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b31ba91aed8afb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fb9600640 1 -- 172.21.15.87:0/2766237731 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80ab610 con 0x7f0fbc06b0d0 2024-04-25T10:16:21.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fb9600640 1 -- 172.21.15.87:0/2766237731 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa80b0070 con 0x7f0fbc06b0d0 2024-04-25T10:16:21.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fb9600640 1 -- 172.21.15.87:0/2766237731 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80ab9d0 con 0x7f0fbc06b0d0 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- 172.21.15.87:0/2766237731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06b0d0 msgr2=0x7f0fbc0cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/2766237731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06b0d0 0x7f0fbc0cd410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc074810 tx=0x7f0fa801c450 comp rx=0 tx=0).stop 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- 172.21.15.87:0/2766237731 shutdown_connections 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/2766237731 >> v2:172.21.15.87:3300/0 conn(0x7f0fbc06bbe0 0x7f0fbc0cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/2766237731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06b0d0 0x7f0fbc0cd410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/2766237731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b800 0x7f0fbc0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- 172.21.15.87:0/2766237731 >> 172.21.15.87:0/2766237731 conn(0x7f0fbc0d19d0 msgr2=0x7f0fbc0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- 172.21.15.87:0/2766237731 shutdown_connections 2024-04-25T10:16:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- 172.21.15.87:0/2766237731 wait complete. 2024-04-25T10:16:21.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 Processor -- start 2024-04-25T10:16:21.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- start start 2024-04-25T10:16:21.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b0d0 0x7f0fbc14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fbc06b800 0x7f0fbc0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06bbe0 0x7f0fbc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0cf240 con 0x7f0fbc06b800 2024-04-25T10:16:21.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0cf0c0 con 0x7f0fbc06bbe0 2024-04-25T10:16:21.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.986+0000 7f0fc1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc1623a0 con 0x7f0fbc06b0d0 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fbc06b800 0x7f0fbc0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fbc06b800 0x7f0fbc0b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35968/0 (socket says 172.21.15.87:35968) 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 -- 172.21.15.87:0/4247489365 learned_addr learned my addr 172.21.15.87:0/4247489365 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b0d0 0x7f0fbc14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fbb400640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06bbe0 0x7f0fbc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 -- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b0d0 msgr2=0x7f0fbc14cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b0d0 0x7f0fbc14cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 -- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06bbe0 msgr2=0x7f0fbc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06bbe0 0x7f0fbc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 -- 172.21.15.87:0/4247489365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0b1820 con 0x7f0fbc06b800 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fba000640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fbc06b800 0x7f0fbc0b10b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0fa801ccf0 tx=0x7f0fa801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fb2a00640 1 -- 172.21.15.87:0/4247489365 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80bf410 con 0x7f0fbc06b800 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fb2a00640 1 -- 172.21.15.87:0/4247489365 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa80bf5b0 con 0x7f0fbc06b800 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fb2a00640 1 -- 172.21.15.87:0/4247489365 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80bf980 con 0x7f0fbc06b800 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc0b1b00 con 0x7f0fbc06b800 2024-04-25T10:16:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc0b20c0 con 0x7f0fbc06b800 2024-04-25T10:16:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.990+0000 7f0fb2a00640 1 -- 172.21.15.87:0/4247489365 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fa80bd020 con 0x7f0fbc06b800 2024-04-25T10:16:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fb2a00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f9c066f60 0x7f0f9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fb2a00640 1 -- 172.21.15.87:0/4247489365 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0fa811f290 con 0x7f0fbc06b800 2024-04-25T10:16:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f9c066f60 0x7f0f9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 Processor -- start 2024-04-25T10:16:22.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- start start 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f9c066f60 0x7f0f9c069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009b60 tx=0x7f0fac009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecaea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaea780 0x55f6ecaa2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaa3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec987c70 con 0x55f6ecaa2d80 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec8ee040 con 0x55f6ecae9e60 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec876040 con 0x55f6ecaea780 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaa3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaa3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35982/0 (socket says 172.21.15.87:35982) 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 -- 172.21.15.87:0/2441781292 learned_addr learned my addr 172.21.15.87:0/2441781292 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecaea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fba000640 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaea780 0x55f6ecaa2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 -- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaea780 msgr2=0x55f6ecaa2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaea780 0x55f6ecaa2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 -- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 msgr2=0x55f6ecaea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecaea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 -- 172.21.15.87:0/2441781292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ec8a5240 con 0x55f6ecaa2d80 2024-04-25T10:16:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaa3160 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40113f0 tx=0x7f0fb4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89a8866d07ba1db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fa7400640 1 -- 172.21.15.87:0/2441781292 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4019070 con 0x55f6ecaa2d80 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fa7400640 1 -- 172.21.15.87:0/2441781292 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb4015630 con 0x55f6ecaa2d80 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fa7400640 1 -- 172.21.15.87:0/2441781292 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4015a00 con 0x55f6ecaa2d80 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2441781292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 msgr2=0x55f6ecaa3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaa3160 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40113f0 tx=0x7f0fb4009810 comp rx=0 tx=0).stop 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2441781292 shutdown_connections 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaa3160 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaea780 0x55f6ecaa2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2441781292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecaea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2441781292 >> 172.21.15.87:0/2441781292 conn(0x55f6eca9a100 msgr2=0x55f6ecaa1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2441781292 shutdown_connections 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.994+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2441781292 wait complete. 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 -- start start 2024-04-25T10:16:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 Processor -- start 2024-04-25T10:16:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecae9e60 0x55f6ecaab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaea780 0x55f6ecaabc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec8a8540 con 0x55f6ecaa2d80 2024-04-25T10:16:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec8ea0e0 con 0x55f6ecaea780 2024-04-25T10:16:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec970400 con 0x55f6ecae9e60 2024-04-25T10:16:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 10 client.0 _ll_register_callbacks cb 0x55f6ec9a23a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35996/0 (socket says 172.21.15.87:35996) 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 -- 172.21.15.87:0/951587982 learned_addr learned my addr 172.21.15.87:0/951587982 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbb400640 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaea780 0x55f6ecaabc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fba000640 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecae9e60 0x55f6ecaab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 -- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecae9e60 msgr2=0x55f6ecaab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecae9e60 0x55f6ecaab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 -- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaea780 msgr2=0x55f6ecaabc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaea780 0x55f6ecaabc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 -- 172.21.15.87:0/951587982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6ec8b6410 con 0x55f6ecaa2d80 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaab210 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0036d0 tx=0x7f0fac0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77204fa0f81829fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0f9be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0f9be00640 1 -- 172.21.15.87:0/951587982 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac027070 con 0x55f6ecaa2d80 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0f9be00640 1 -- 172.21.15.87:0/951587982 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0fac014650 con 0x55f6ecaa2d80 2024-04-25T10:16:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fb4015630 con 0x55f6ecaa2d80 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0f9be00640 1 -- 172.21.15.87:0/951587982 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fac032050 con 0x55f6ecaa2d80 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 10 client.6837 fetch_fsmap learned FSMap version 44 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55f6ec876290 con 0x55f6ecaa2d80 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0f9be00640 1 -- 172.21.15.87:0/951587982 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb4015630 con 0x55f6ecaa2d80 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0f9be00640 1 -- 172.21.15.87:0/951587982 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fac035020 con 0x55f6ecaa2d80 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 10 client.6837 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 10 client.6837 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 20 client.6837 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaac530 0x55f6ecafb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 4 client.6837 mds_command: new command op to 6768 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 --> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6ec889d70 con 0x55f6ecaac530 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fba000640 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaac530 0x55f6ecafb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0f9be00640 10 client.6837 ms_handle_connect on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:21.998+0000 7f0fba000640 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaac530 0x55f6ecafb5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7cade395086a54 server_cookie=3066295fc4b4cab2 in_seq=0 out_seq=0 2024-04-25T10:16:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0f9be00640 1 -- 172.21.15.87:0/951587982 <== mds.0 v2:172.21.15.87:6836/427681613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6ec889d70 con 0x55f6ecaac530 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0f9be00640 10 client.6837 handle_command_reply: tid=0 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 client.6837 shutdown 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 msgr2=0x55f6ecaab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaab210 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0036d0 tx=0x7f0fac0046d0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 shutdown_connections 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaac530 0x55f6ecafb5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaea780 0x55f6ecaabc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecae9e60 0x55f6ecaab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/951587982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecaa2d80 0x55f6ecaab210 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0f9be00640 0 client.6837 ms_handle_reset on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 >> 172.21.15.87:0/951587982 conn(0x55f6eca9a100 msgr2=0x55f6ecaa1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 shutdown_connections 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 1 -- 172.21.15.87:0/951587982 wait complete. 2024-04-25T10:16:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.002+0000 7f0fc50a6000 20 client.6837 trim_cache size 0 max 16384 2024-04-25T10:16:22.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 Processor -- start 2024-04-25T10:16:22.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- start start 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6eca9a100 0x55f6ecafd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaa2d80 0x55f6ecafb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecafbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc202e20 con 0x55f6eca9a100 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc202020 con 0x55f6ecae9e60 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc201420 con 0x55f6ecaa2d80 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6eca9a100 0x55f6ecafd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6eca9a100 0x55f6ecafd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36008/0 (socket says 172.21.15.87:36008) 2024-04-25T10:16:22.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 -- 172.21.15.87:0/2105449149 learned_addr learned my addr 172.21.15.87:0/2105449149 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fba000640 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaa2d80 0x55f6ecafb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecafbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 -- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaa2d80 msgr2=0x55f6ecafb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaa2d80 0x55f6ecafb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 -- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 msgr2=0x55f6ecafbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecafbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 -- 172.21.15.87:0/2105449149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ec970890 con 0x55f6eca9a100 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecafbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6eca9a100 0x55f6ecafd960 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0fac000c10 tx=0x7f0fac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef9e547749090a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fa6a00640 1 -- 172.21.15.87:0/2105449149 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac027070 con 0x55f6eca9a100 2024-04-25T10:16:22.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fa6a00640 1 -- 172.21.15.87:0/2105449149 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac03c070 con 0x55f6eca9a100 2024-04-25T10:16:22.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fa6a00640 1 -- 172.21.15.87:0/2105449149 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac014c80 con 0x55f6eca9a100 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2105449149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6eca9a100 msgr2=0x55f6ecafd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6eca9a100 0x55f6ecafd960 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0fac000c10 tx=0x7f0fac01b000 comp rx=0 tx=0).stop 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2105449149 shutdown_connections 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecae9e60 0x55f6ecafbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6ecaa2d80 0x55f6ecafb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2105449149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6eca9a100 0x55f6ecafd960 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2105449149 >> 172.21.15.87:0/2105449149 conn(0x55f6ecaa8400 msgr2=0x55f6ecae7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2105449149 shutdown_connections 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2105449149 wait complete. 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 -- start start 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.018+0000 7f0fc50a6000 1 Processor -- start 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6eca9a100 0x55f6ecb4b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaa2d80 0x55f6ecb4b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecae9e60 0x55f6ecb060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec8ea0e0 con 0x55f6ecae9e60 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc201a20 con 0x55f6ecaa2d80 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6ec8a4d30 con 0x55f6eca9a100 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 10 client.0 _ll_register_callbacks cb 0x7f0fb4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecae9e60 0x55f6ecb060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecae9e60 0x55f6ecb060b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36018/0 (socket says 172.21.15.87:36018) 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 -- 172.21.15.87:0/2231987775 learned_addr learned my addr 172.21.15.87:0/2231987775 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6eca9a100 0x55f6ecb4b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 -- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6eca9a100 msgr2=0x55f6ecb4b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fba000640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaa2d80 0x55f6ecb4b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6eca9a100 0x55f6ecb4b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 -- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaa2d80 msgr2=0x55f6ecb4b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaa2d80 0x55f6ecb4b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 -- 172.21.15.87:0/2231987775 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6ec882e50 con 0x55f6ecae9e60 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fba000640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaa2d80 0x55f6ecb4b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbb400640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecae9e60 0x55f6ecb060b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4018000 tx=0x7f0fb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4034728165fc595 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 1 -- 172.21.15.87:0/2231987775 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb402d060 con 0x55f6ecae9e60 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 1 -- 172.21.15.87:0/2231987775 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0fb4004780 con 0x55f6ecae9e60 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fac03c070 con 0x55f6ecae9e60 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 1 -- 172.21.15.87:0/2231987775 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fb4033050 con 0x55f6ecae9e60 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 10 client.6843 fetch_fsmap learned FSMap version 44 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55f6ec894f80 con 0x55f6ecae9e60 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 1 -- 172.21.15.87:0/2231987775 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac03c070 con 0x55f6ecae9e60 2024-04-25T10:16:22.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 1 -- 172.21.15.87:0/2231987775 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fb4036020 con 0x55f6ecae9e60 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 10 client.6843 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 10 client.6843 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 20 client.6843 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaea780 0x55f6ecb06e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 4 client.6843 mds_command: new command op to 6768 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 --> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55f6ec8fb580 con 0x55f6ecaea780 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaea780 0x55f6ecb06e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fbaa00640 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaea780 0x55f6ecb06e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7232651ccc50cec4 server_cookie=834f51a80bb8e4cc in_seq=0 out_seq=0 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 10 client.6843 ms_handle_connect on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 1 -- 172.21.15.87:0/2231987775 <== mds.0 v2:172.21.15.87:6836/427681613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f6ec8fb580 con 0x55f6ecaea780 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0f9be00640 10 client.6843 handle_command_reply: tid=0 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.022+0000 7f0fc50a6000 1 client.6843 shutdown 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecae9e60 msgr2=0x55f6ecb060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecae9e60 0x55f6ecb060b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4018000 tx=0x7f0fb4014000 comp rx=0 tx=0).stop 2024-04-25T10:16:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 shutdown_connections 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:6836/427681613,v1:172.21.15.87:6837/427681613] conn(0x55f6ecaea780 0x55f6ecb06e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6ecae9e60 0x55f6ecb060b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6ecaa2d80 0x55f6ecb4b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 --2- 172.21.15.87:0/2231987775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f6eca9a100 0x55f6ecb4b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0f9be00640 0 client.6843 ms_handle_reset on v2:172.21.15.87:6836/427681613 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 >> 172.21.15.87:0/2231987775 conn(0x55f6ecaa8400 msgr2=0x55f6ecae8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 shutdown_connections 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 1 -- 172.21.15.87:0/2231987775 wait complete. 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc50a6000 20 client.6843 trim_cache size 0 max 16384 2024-04-25T10:16:22.036 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:16:22.037 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:16:22.037 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "8f22bd6a-3d60-4e77-8002-f3b24964b552", 2024-04-25T10:16:22.037 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:16:22.037 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:16:22.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f9c066f60 msgr2=0x7f0f9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f9c066f60 0x7f0f9c069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009b60 tx=0x7f0fac009190 comp rx=0 tx=0).stop 2024-04-25T10:16:22.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fbc06b800 msgr2=0x7f0fbc0b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fbc06b800 0x7f0fbc0b10b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0fa801ccf0 tx=0x7f0fa801ac90 comp rx=0 tx=0).stop 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 shutdown_connections 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f9c066f60 0x7f0f9c069420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fbc06bbe0 0x7f0fbc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fbc06b800 0x7f0fbc0b10b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 --2- 172.21.15.87:0/4247489365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fbc06b0d0 0x7f0fbc14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 >> 172.21.15.87:0/4247489365 conn(0x7f0fbc0d19d0 msgr2=0x7f0fbc0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 shutdown_connections 2024-04-25T10:16:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:22.026+0000 7f0fc1c00640 1 -- 172.21.15.87:0/4247489365 wait complete. 2024-04-25T10:16:22.052 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '8f22bd6a-3d60-4e77-8002-f3b24964b552', 'mode': 'asynchronous'}' 2024-04-25T10:16:32.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:32.053 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:16:32.066 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:32.066 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:32.214 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:32.306 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:32.306 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:32.306 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:32.316 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:32.316 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:32.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:32.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91c000640 1 Processor -- start 2024-04-25T10:16:32.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91c000640 1 -- start start 2024-04-25T10:16:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 0x7fe9140b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91c000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9140cfd90 0x7fe9140ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91c000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91415f3b0 con 0x7fe9140cd110 2024-04-25T10:16:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91c000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91415ef20 con 0x7fe9140cfd90 2024-04-25T10:16:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140ca810 con 0x7fe9140b0f70 2024-04-25T10:16:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91a000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe9140cd110 0x7fe9140cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50772/0 (socket says 172.21.15.87:50772) 2024-04-25T10:16:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe91a000640 1 -- 172.21.15.87:0/135379654 learned_addr learned my addr 172.21.15.87:0/135379654 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:32.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe918c00640 1 --2- 172.21.15.87:0/135379654 >> v2:172.21.15.169:3300/0 conn(0x7fe9140cfd90 0x7fe9140ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294884212 0 0) 0x7fe91415f3b0 con 0x7fe9140cd110 2024-04-25T10:16:32.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe919600640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 0x7fe9140b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8fc003530 con 0x7fe9140cd110 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226135777 0 0) 0x7fe8fc003530 con 0x7fe9140cd110 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 msgr2=0x7fe9140b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 0x7fe9140b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 >> v2:172.21.15.169:3300/0 conn(0x7fe9140cfd90 msgr2=0x7fe9140ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 --2- 172.21.15.87:0/135379654 >> v2:172.21.15.169:3300/0 conn(0x7fe9140cfd90 0x7fe9140ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140b1890 con 0x7fe9140cd110 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe90c001a00 con 0x7fe9140cd110 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 >> v1:172.21.15.87:6789/0 conn(0x7fe9140cd110 legacy=0x7fe9140cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc0039c0 0x7fe8fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe8fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 0x7fe8fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415ef20 con 0x7fe8fc004810 2024-04-25T10:16:32.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140ca810 con 0x7fe8fc0039c0 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.554+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc00b9d0 con 0x7fe9140b0f70 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe918c00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe8fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe919600640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc0039c0 0x7fe8fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe91a000640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 0x7fe8fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe918c00640 1 -- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 msgr2=0x7fe8fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe918c00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9140b0f70 0x7fe8fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe918c00640 1 -- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc0039c0 msgr2=0x7fe8fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe918c00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc0039c0 0x7fe8fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe918c00640 1 -- 172.21.15.87:0/135379654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc003530 con 0x7fe8fc004810 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe918c00640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe8fc004c90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe900009ab0 tx=0x7fe90001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cf3a1df0a5a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9000b2070 con 0x7fe8fc004810 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9000a55a0 con 0x7fe8fc004810 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.558+0000 7fe913e00640 1 -- 172.21.15.87:0/135379654 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9000096b0 con 0x7fe8fc004810 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- 172.21.15.87:0/135379654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 msgr2=0x7fe8fc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 --2- 172.21.15.87:0/135379654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe8fc004c90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe900009ab0 tx=0x7fe90001ca90 comp rx=0 tx=0).stop 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91a000640 1 -- 172.21.15.87:0/135379654 reap_dead start 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- 172.21.15.87:0/135379654 shutdown_connections 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- 172.21.15.87:0/135379654 >> 172.21.15.87:0/135379654 conn(0x7fe9140b1d50 msgr2=0x7fe9140b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:32.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- 172.21.15.87:0/135379654 shutdown_connections 2024-04-25T10:16:32.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- 172.21.15.87:0/135379654 wait complete. 2024-04-25T10:16:32.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 Processor -- start 2024-04-25T10:16:32.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- start start 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe9141f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe91415db10 0x7fe9140ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9140ad570 0x7fe91415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415ebb0 con 0x7fe8fc004810 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415e8d0 con 0x7fe9140ad570 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140cd580 con 0x7fe91415db10 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe9141f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe9141f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41688/0 (socket says 172.21.15.87:41688) 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 -- 172.21.15.87:0/2746361487 learned_addr learned my addr 172.21.15.87:0/2746361487 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:32.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe918c00640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe91415db10 0x7fe9140ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe91a000640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9140ad570 0x7fe91415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 -- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe91415db10 msgr2=0x7fe9140ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe91415db10 0x7fe9140ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 -- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9140ad570 msgr2=0x7fe91415f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9140ad570 0x7fe91415f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.562+0000 7fe919600640 1 -- 172.21.15.87:0/2746361487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140af040 con 0x7fe8fc004810 2024-04-25T10:16:32.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe919600640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe9141f3cd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe908011a30 tx=0x7fe908011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe911600640 1 -- 172.21.15.87:0/2746361487 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800fb50 con 0x7fe8fc004810 2024-04-25T10:16:32.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe911600640 1 -- 172.21.15.87:0/2746361487 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90800fcf0 con 0x7fe8fc004810 2024-04-25T10:16:32.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9141f4210 con 0x7fe8fc004810 2024-04-25T10:16:32.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe911600640 1 -- 172.21.15.87:0/2746361487 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800fea0 con 0x7fe8fc004810 2024-04-25T10:16:32.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9140aea80 con 0x7fe8fc004810 2024-04-25T10:16:32.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91404ef30 con 0x7fe8fc004810 2024-04-25T10:16:32.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe911600640 1 -- 172.21.15.87:0/2746361487 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe908012040 con 0x7fe8fc004810 2024-04-25T10:16:32.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe911600640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe8f4066f60 0x7fe8f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe911600640 1 -- 172.21.15.87:0/2746361487 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe908085630 con 0x7fe8fc004810 2024-04-25T10:16:32.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.566+0000 7fe918c00640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe8f4066f60 0x7fe8f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.570+0000 7fe918c00640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe8f4066f60 0x7fe8f4069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe9000af000 tx=0x7fe9000046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.570+0000 7fe911600640 1 -- 172.21.15.87:0/2746361487 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe908019030 con 0x7fe8fc004810 2024-04-25T10:16:32.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.726+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9140cda20 con 0x7fe8fc004810 2024-04-25T10:16:32.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.726+0000 7fe911600640 1 -- 172.21.15.87:0/2746361487 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16782 (secure 0 0 0) 0x7fe908056600 con 0x7fe8fc004810 2024-04-25T10:16:32.734 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:32.734 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":61,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:14.297721+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/3387238920":"2024-04-26T10:16:14.297697+0000","172.21.15.87:6835/3387238920":"2024-04-26T10:16:14.297697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:32.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe8f4066f60 msgr2=0x7fe8f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe8f4066f60 0x7fe8f4069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe9000af000 tx=0x7fe9000046d0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 msgr2=0x7fe9141f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe9141f3cd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe908011a30 tx=0x7fe908011ef0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 shutdown_connections 2024-04-25T10:16:32.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe8f4066f60 0x7fe8f4069420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9140ad570 0x7fe91415f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe91415db10 0x7fe9140ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 --2- 172.21.15.87:0/2746361487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004810 0x7fe9141f3cd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 >> 172.21.15.87:0/2746361487 conn(0x7fe9140b1d50 msgr2=0x7fe91406e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:32.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 shutdown_connections 2024-04-25T10:16:32.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.730+0000 7fe91c000640 1 -- 172.21.15.87:0/2746361487 wait complete. 2024-04-25T10:16:32.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:00.962606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:01.567628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:32.755 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:32.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.886+0000 7fc7cb200640 1 Processor -- start 2024-04-25T10:16:32.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7cb200640 1 -- start start 2024-04-25T10:16:32.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c40b38d0 0x7fc7c415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7cb200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c415b080 con 0x7fc7c4070350 2024-04-25T10:16:32.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7cb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c415adb0 con 0x7fc7c40aef80 2024-04-25T10:16:32.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415eb60 con 0x7fc7c40b38d0 2024-04-25T10:16:32.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc7c4070350 0x7fc7c40b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50774/0 (socket says 172.21.15.87:50774) 2024-04-25T10:16:32.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3400640 1 -- 172.21.15.87:0/4004319240 learned_addr learned my addr 172.21.15.87:0/4004319240 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c9200640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c40b38d0 0x7fc7c415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794256609 0 0) 0x7fc7c415b080 con 0x7fc7c4070350 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a8003530 con 0x7fc7c4070350 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67940059 0 0) 0x7fc7a8003530 con 0x7fc7c4070350 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c40b38d0 msgr2=0x7fc7c415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c40b38d0 0x7fc7c415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 >> v1:172.21.15.169:6789/0 conn(0x7fc7c40aef80 legacy=0x7fc7c415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c415ee40 con 0x7fc7c4070350 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc7b0001680 con 0x7fc7c4070350 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 >> v1:172.21.15.87:6789/0 conn(0x7fc7c4070350 legacy=0x7fc7c40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a80039f0 0x7fc7a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80043a0 0x7fc7a8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a8005070 0x7fc7a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415eb60 con 0x7fc7a80039f0 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a800bf10 con 0x7fc7a80043a0 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a800c090 con 0x7fc7a8005070 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3e00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a80039f0 0x7fc7a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3400640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80043a0 0x7fc7a8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c9200640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a8005070 0x7fc7a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3e00640 1 -- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a8005070 msgr2=0x7fc7a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3e00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a8005070 0x7fc7a800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3e00640 1 -- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80043a0 msgr2=0x7fc7a8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3e00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80043a0 0x7fc7a8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.890+0000 7fc7c3e00640 1 -- 172.21.15.87:0/4004319240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a8003530 con 0x7fc7a80039f0 2024-04-25T10:16:32.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7c3e00640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a80039f0 0x7fc7a8003dd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4001f60 tx=0x7fc7b401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ea65f6bfef42d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40b2070 con 0x7fc7a80039f0 2024-04-25T10:16:32.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b40a5770 con 0x7fc7a80039f0 2024-04-25T10:16:32.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7c2a00640 1 -- 172.21.15.87:0/4004319240 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4001b60 con 0x7fc7a80039f0 2024-04-25T10:16:32.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7cb200640 1 -- 172.21.15.87:0/4004319240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a80039f0 msgr2=0x7fc7a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7cb200640 1 --2- 172.21.15.87:0/4004319240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7a80039f0 0x7fc7a8003dd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4001f60 tx=0x7fc7b401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7c9200640 1 -- 172.21.15.87:0/4004319240 reap_dead start 2024-04-25T10:16:32.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7cb200640 1 -- 172.21.15.87:0/4004319240 shutdown_connections 2024-04-25T10:16:32.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7cb200640 1 -- 172.21.15.87:0/4004319240 >> 172.21.15.87:0/4004319240 conn(0x7fc7c40669a0 msgr2=0x7fc7c4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:32.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7cb200640 1 -- 172.21.15.87:0/4004319240 shutdown_connections 2024-04-25T10:16:32.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.894+0000 7fc7cb200640 1 -- 172.21.15.87:0/4004319240 wait complete. 2024-04-25T10:16:32.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 Processor -- start 2024-04-25T10:16:32.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 -- start start 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80039f0 0x7fc7c4150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c40b20c0 0x7fc7c40b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c40b0b20 0x7fc7c415d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c406c0d0 con 0x7fc7c40b20c0 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415bc90 con 0x7fc7c40b0b20 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c406c250 con 0x7fc7a80039f0 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c40b20c0 0x7fc7c40b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c40b20c0 0x7fc7c40b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41706/0 (socket says 172.21.15.87:41706) 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 -- 172.21.15.87:0/1003149766 learned_addr learned my addr 172.21.15.87:0/1003149766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:32.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c9200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c40b0b20 0x7fc7c415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3e00640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80039f0 0x7fc7c4150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 -- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80039f0 msgr2=0x7fc7c4150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80039f0 0x7fc7c4150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 -- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c40b0b20 msgr2=0x7fc7c415d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c40b0b20 0x7fc7c415d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 -- 172.21.15.87:0/1003149766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40b1020 con 0x7fc7c40b20c0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c9200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c40b0b20 0x7fc7c415d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7c3400640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c40b20c0 0x7fc7c40b01f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00017f0 tx=0x7fc7b0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7bbe00640 1 -- 172.21.15.87:0/1003149766 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0015070 con 0x7fc7c40b20c0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7bbe00640 1 -- 172.21.15.87:0/1003149766 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b000fd70 con 0x7fc7c40b20c0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7bbe00640 1 -- 172.21.15.87:0/1003149766 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0011680 con 0x7fc7c40b20c0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c40b1300 con 0x7fc7c40b20c0 2024-04-25T10:16:32.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c40b18c0 con 0x7fc7c40b20c0 2024-04-25T10:16:32.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.898+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c404ef30 con 0x7fc7c40b20c0 2024-04-25T10:16:32.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.902+0000 7fc7bbe00640 1 -- 172.21.15.87:0/1003149766 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7b0018070 con 0x7fc7c40b20c0 2024-04-25T10:16:32.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.902+0000 7fc7bbe00640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc7a4066f60 0x7fc7a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.902+0000 7fc7bbe00640 1 -- 172.21.15.87:0/1003149766 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc7b0085b10 con 0x7fc7c40b20c0 2024-04-25T10:16:32.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.902+0000 7fc7c3e00640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc7a4066f60 0x7fc7a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.902+0000 7fc7c3e00640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc7a4066f60 0x7fc7a4069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40af000 tx=0x7fc7b40a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:32.906+0000 7fc7bbe00640 1 -- 172.21.15.87:0/1003149766 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7b0056ab0 con 0x7fc7c40b20c0 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7c40af610 con 0x7fc7c40b20c0 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7bbe00640 1 -- 172.21.15.87:0/1003149766 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7b00563b0 con 0x7fc7c40b20c0 2024-04-25T10:16:33.069 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc7a4066f60 msgr2=0x7fc7a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc7a4066f60 0x7fc7a4069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40af000 tx=0x7fc7b40a7070 comp rx=0 tx=0).stop 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c40b20c0 msgr2=0x7fc7c40b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c40b20c0 0x7fc7c40b01f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00017f0 tx=0x7fc7b0007780 comp rx=0 tx=0).stop 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 shutdown_connections 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc7a4066f60 0x7fc7a4069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c40b0b20 0x7fc7c415d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c40b20c0 0x7fc7c40b01f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 --2- 172.21.15.87:0/1003149766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80039f0 0x7fc7c4150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 >> 172.21.15.87:0/1003149766 conn(0x7fc7c40669a0 msgr2=0x7fc7c4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 shutdown_connections 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.062+0000 7fc7cb200640 1 -- 172.21.15.87:0/1003149766 wait complete. 2024-04-25T10:16:33.090 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfc5c00640 1 Processor -- start 2024-04-25T10:16:33.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfc5c00640 1 -- start start 2024-04-25T10:16:33.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfc5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00cf980 0x7fcfc00b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfc5c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc015fec0 con 0x7fcfc00d1830 2024-04-25T10:16:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfc5c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc01601a0 con 0x7fcfc015bc10 2024-04-25T10:16:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfc5c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00cc5b0 con 0x7fcfc00cf980 2024-04-25T10:16:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfbf400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcfc00d1830 0x7fcfc00d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50780/0 (socket says 172.21.15.87:50780) 2024-04-25T10:16:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfbf400640 1 -- 172.21.15.87:0/2332062657 learned_addr learned my addr 172.21.15.87:0/2332062657 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.222+0000 7fcfbe000640 1 --2- 172.21.15.87:0/2332062657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00cf980 0x7fcfc00b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055483930 0 0) 0x7fcfc015fec0 con 0x7fcfc00d1830 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa4003530 con 0x7fcfc00d1830 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78509418 0 0) 0x7fcfc01601a0 con 0x7fcfc015bc10 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc015fec0 con 0x7fcfc015bc10 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbe000640 1 -- 172.21.15.87:0/2332062657 >> v1:172.21.15.169:6789/0 conn(0x7fcfc015bc10 legacy=0x7fcfc015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbe000640 1 -- 172.21.15.87:0/2332062657 >> v1:172.21.15.87:6789/0 conn(0x7fcfc00d1830 legacy=0x7fcfc00d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226071698 0 0) 0x7fcfa4003530 con 0x7fcfc00d1830 2024-04-25T10:16:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbe000640 1 -- 172.21.15.87:0/2332062657 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00cc890 con 0x7fcfc00cf980 2024-04-25T10:16:33.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbe000640 1 --2- 172.21.15.87:0/2332062657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00cf980 0x7fcfc00b1060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0097f0 tx=0x7fcfac01c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b226689929f002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0b0070 con 0x7fcfc00cf980 2024-04-25T10:16:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc00b17b0 con 0x7fcfc00cf980 2024-04-25T10:16:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.226+0000 7fcfbd600640 1 -- 172.21.15.87:0/2332062657 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8001680 con 0x7fcfc00cf980 2024-04-25T10:16:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- 172.21.15.87:0/2332062657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00cf980 msgr2=0x7fcfc00b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/2332062657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00cf980 0x7fcfc00b1060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0097f0 tx=0x7fcfac01c2e0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- 172.21.15.87:0/2332062657 shutdown_connections 2024-04-25T10:16:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/2332062657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00cf980 0x7fcfc00b1060 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- 172.21.15.87:0/2332062657 >> 172.21.15.87:0/2332062657 conn(0x7fcfc009a9d0 msgr2=0x7fcfc009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- 172.21.15.87:0/2332062657 shutdown_connections 2024-04-25T10:16:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- 172.21.15.87:0/2332062657 wait complete. 2024-04-25T10:16:33.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 Processor -- start 2024-04-25T10:16:33.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- start start 2024-04-25T10:16:33.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00cf980 0x7fcfc00b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00d1830 0x7fcfc00b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc015bc10 0x7fcfc015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0169970 con 0x7fcfc00cf980 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0169af0 con 0x7fcfc015bc10 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfc5c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0169c70 con 0x7fcfc00d1830 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00cf980 0x7fcfc00b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00cf980 0x7fcfc00b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41710/0 (socket says 172.21.15.87:41710) 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 -- 172.21.15.87:0/3887545882 learned_addr learned my addr 172.21.15.87:0/3887545882 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbe000640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00d1830 0x7fcfc00b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbf400640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc015bc10 0x7fcfc015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 -- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00d1830 msgr2=0x7fcfc00b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00d1830 0x7fcfc00b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 -- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc015bc10 msgr2=0x7fcfc015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc015bc10 0x7fcfc015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.230+0000 7fcfbea00640 1 -- 172.21.15.87:0/3887545882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc0156550 con 0x7fcfc00cf980 2024-04-25T10:16:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfbea00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00cf980 0x7fcfc00b5fa0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c870 tx=0x7fcfb000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfb6a00640 1 -- 172.21.15.87:0/3887545882 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003b80 con 0x7fcfc00cf980 2024-04-25T10:16:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc00b6630 con 0x7fcfc00cf980 2024-04-25T10:16:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfb6a00640 1 -- 172.21.15.87:0/3887545882 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb0003d20 con 0x7fcfc00cf980 2024-04-25T10:16:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfb6a00640 1 -- 172.21.15.87:0/3887545882 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000aa30 con 0x7fcfc00cf980 2024-04-25T10:16:33.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc00b4ad0 con 0x7fcfc00cf980 2024-04-25T10:16:33.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc004ef30 con 0x7fcfc00cf980 2024-04-25T10:16:33.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfb6a00640 1 -- 172.21.15.87:0/3887545882 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfb000abd0 con 0x7fcfc00cf980 2024-04-25T10:16:33.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfb6a00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfa0066f60 0x7fcfa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfb6a00640 1 -- 172.21.15.87:0/3887545882 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcfb00851d0 con 0x7fcfc00cf980 2024-04-25T10:16:33.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.234+0000 7fcfbe000640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfa0066f60 0x7fcfa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.238+0000 7fcfbe000640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfa0066f60 0x7fcfa0069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ad000 tx=0x7fcfac0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.238+0000 7fcfb6a00640 1 -- 172.21.15.87:0/3887545882 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfb0056060 con 0x7fcfc00cf980 2024-04-25T10:16:33.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfc00b78f0 con 0x7fcfc00cf980 2024-04-25T10:16:33.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfb6a00640 1 -- 172.21.15.87:0/3887545882 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfb0055960 con 0x7fcfc00cf980 2024-04-25T10:16:33.401 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfa0066f60 msgr2=0x7fcfa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfa0066f60 0x7fcfa0069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ad000 tx=0x7fcfac0a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00cf980 msgr2=0x7fcfc00b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00cf980 0x7fcfc00b5fa0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c870 tx=0x7fcfb000cd30 comp rx=0 tx=0).stop 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 shutdown_connections 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfa0066f60 0x7fcfa0069420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc015bc10 0x7fcfc015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfc00d1830 0x7fcfc00b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 --2- 172.21.15.87:0/3887545882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00cf980 0x7fcfc00b5fa0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 >> 172.21.15.87:0/3887545882 conn(0x7fcfc009a9d0 msgr2=0x7fcfc015f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 shutdown_connections 2024-04-25T10:16:33.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.394+0000 7fcfc5c00640 1 -- 172.21.15.87:0/3887545882 wait complete. 2024-04-25T10:16:33.422 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:33.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb84400640 1 Processor -- start 2024-04-25T10:16:33.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb84400640 1 -- start start 2024-04-25T10:16:33.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb84400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb7c159540 0x7fbb7c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb84400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c0aebd0 0x7fbb7c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb84400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c071360 con 0x7fbb7c072020 2024-04-25T10:16:33.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb84400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c159920 con 0x7fbb7c159540 2024-04-25T10:16:33.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb84400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c15c2f0 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c0aebd0 0x7fbb7c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb7c159540 0x7fbb7c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c0aebd0 0x7fbb7c071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:52126/0 (socket says 172.21.15.87:52126) 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb82400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbb7c072020 0x7fbb7c15f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:47260/0 (socket says 172.21.15.87:47260) 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81000640 1 -- 172.21.15.87:0/1963809935 learned_addr learned my addr 172.21.15.87:0/1963809935 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81000640 1 -- 172.21.15.87:0/1963809935 >> v2:172.21.15.169:3300/0 conn(0x7fbb7c159540 msgr2=0x7fbb7c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81000640 1 --2- 172.21.15.87:0/1963809935 >> v2:172.21.15.169:3300/0 conn(0x7fbb7c159540 0x7fbb7c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81000640 1 -- 172.21.15.87:0/1963809935 >> v1:172.21.15.169:6789/0 conn(0x7fbb7c072020 legacy=0x7fbb7c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.554+0000 7fbb81000640 1 -- 172.21.15.87:0/1963809935 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c15c5d0 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb81000640 1 --2- 172.21.15.87:0/1963809935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c0aebd0 0x7fbb7c071ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009870 tx=0x7fbb6801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0836d2946dc8cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb7be00640 1 -- 172.21.15.87:0/1963809935 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680b0070 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb7be00640 1 -- 172.21.15.87:0/1963809935 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb680ab600 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb7be00640 1 -- 172.21.15.87:0/1963809935 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680ab9d0 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 -- 172.21.15.87:0/1963809935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c0aebd0 msgr2=0x7fbb7c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 --2- 172.21.15.87:0/1963809935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c0aebd0 0x7fbb7c071ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009870 tx=0x7fbb6801c560 comp rx=0 tx=0).stop 2024-04-25T10:16:33.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 -- 172.21.15.87:0/1963809935 shutdown_connections 2024-04-25T10:16:33.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 --2- 172.21.15.87:0/1963809935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c0aebd0 0x7fbb7c071ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 --2- 172.21.15.87:0/1963809935 >> v2:172.21.15.169:3300/0 conn(0x7fbb7c159540 0x7fbb7c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 -- 172.21.15.87:0/1963809935 >> 172.21.15.87:0/1963809935 conn(0x7fbb7c0669a0 msgr2=0x7fbb7c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 -- 172.21.15.87:0/1963809935 shutdown_connections 2024-04-25T10:16:33.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.558+0000 7fbb84400640 1 -- 172.21.15.87:0/1963809935 wait complete. 2024-04-25T10:16:33.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 Processor -- start 2024-04-25T10:16:33.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 -- start start 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c072020 0x7fbb7c1563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb7c0aebd0 0x7fbb7c14c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7c159540 0x7fbb7c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c072430 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c06c0d0 con 0x7fbb7c159540 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c168930 con 0x7fbb7c072020 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb7c0aebd0 0x7fbb7c14c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb7c0aebd0 0x7fbb7c14c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41712/0 (socket says 172.21.15.87:41712) 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 -- 172.21.15.87:0/2382480719 learned_addr learned my addr 172.21.15.87:0/2382480719 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81a00640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c072020 0x7fbb7c1563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb82400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7c159540 0x7fbb7c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 -- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c072020 msgr2=0x7fbb7c1563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c072020 0x7fbb7c1563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 -- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7c159540 msgr2=0x7fbb7c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7c159540 0x7fbb7c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 -- 172.21.15.87:0/2382480719 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c14ff00 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb82400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7c159540 0x7fbb7c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:33.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb81000640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb7c0aebd0 0x7fbb7c14c8c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbb6801ac10 tx=0x7fbb68019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb79600640 1 -- 172.21.15.87:0/2382480719 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680b0040 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb79600640 1 -- 172.21.15.87:0/2382480719 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb680b4420 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c1501e0 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb79600640 1 -- 172.21.15.87:0/2382480719 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680b4810 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c1507a0 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.562+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c04ef30 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.566+0000 7fbb79600640 1 -- 172.21.15.87:0/2382480719 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb680b49b0 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.566+0000 7fbb79600640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb5c066f60 0x7fbb5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.566+0000 7fbb79600640 1 -- 172.21.15.87:0/2382480719 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbb6811ec70 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.566+0000 7fbb81a00640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb5c066f60 0x7fbb5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.566+0000 7fbb81a00640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb5c066f60 0x7fbb5c069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbb700113e0 tx=0x7fbb7000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.570+0000 7fbb79600640 1 -- 172.21.15.87:0/2382480719 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb68016150 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.722+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb7c0cf470 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.722+0000 7fbb79600640 1 -- 172.21.15.87:0/2382480719 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb680efc10 con 0x7fbb7c0aebd0 2024-04-25T10:16:33.729 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:16:33.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb5c066f60 msgr2=0x7fbb5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb5c066f60 0x7fbb5c069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbb700113e0 tx=0x7fbb7000b040 comp rx=0 tx=0).stop 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb7c0aebd0 msgr2=0x7fbb7c14c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb7c0aebd0 0x7fbb7c14c8c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbb6801ac10 tx=0x7fbb68019300 comp rx=0 tx=0).stop 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 shutdown_connections 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb5c066f60 0x7fbb5c069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7c159540 0x7fbb7c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb7c0aebd0 0x7fbb7c14c8c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 --2- 172.21.15.87:0/2382480719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c072020 0x7fbb7c1563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 >> 172.21.15.87:0/2382480719 conn(0x7fbb7c0669a0 msgr2=0x7fbb7c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 shutdown_connections 2024-04-25T10:16:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.726+0000 7fbb84400640 1 -- 172.21.15.87:0/2382480719 wait complete. 2024-04-25T10:16:33.751 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:33.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.882+0000 7febe1200640 1 Processor -- start 2024-04-25T10:16:33.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.882+0000 7febe1200640 1 -- start start 2024-04-25T10:16:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febe1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febe1200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febdc070ed0 con 0x7febdc0b2fc0 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febe1200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febdc0711b0 con 0x7febdc15a0e0 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febe1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc071490 con 0x7febdc071f70 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febda800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7febdc0b2fc0 0x7febdc06c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50790/0 (socket says 172.21.15.87:50790) 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febda800640 1 -- 172.21.15.87:0/4055209 learned_addr learned my addr 172.21.15.87:0/4055209 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd9400640 1 --2- 172.21.15.87:0/4055209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd8a00640 1 -- 172.21.15.87:0/4055209 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971170597 0 0) 0x7febdc070ed0 con 0x7febdc0b2fc0 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd8a00640 1 -- 172.21.15.87:0/4055209 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc0003530 con 0x7febdc0b2fc0 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd8a00640 1 -- 172.21.15.87:0/4055209 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953558757 0 0) 0x7febdc0711b0 con 0x7febdc15a0e0 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd8a00640 1 -- 172.21.15.87:0/4055209 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febdc070ed0 con 0x7febdc15a0e0 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd9400640 1 -- 172.21.15.87:0/4055209 >> v1:172.21.15.169:6789/0 conn(0x7febdc15a0e0 legacy=0x7febdc15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd9400640 1 -- 172.21.15.87:0/4055209 >> v1:172.21.15.87:6789/0 conn(0x7febdc0b2fc0 legacy=0x7febdc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.886+0000 7febd9400640 1 -- 172.21.15.87:0/4055209 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc071770 con 0x7febdc071f70 2024-04-25T10:16:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febd9400640 1 --2- 172.21.15.87:0/4055209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc0b2a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7febc80097f0 tx=0x7febc801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997ef3c8557687b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febd8a00640 1 -- 172.21.15.87:0/4055209 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80b0070 con 0x7febdc071f70 2024-04-25T10:16:33.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febd8a00640 1 -- 172.21.15.87:0/4055209 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc80ab600 con 0x7febdc071f70 2024-04-25T10:16:33.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febd8a00640 1 -- 172.21.15.87:0/4055209 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80ab9d0 con 0x7febdc071f70 2024-04-25T10:16:33.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 -- 172.21.15.87:0/4055209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 msgr2=0x7febdc0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 --2- 172.21.15.87:0/4055209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc0b2a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7febc80097f0 tx=0x7febc801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 -- 172.21.15.87:0/4055209 shutdown_connections 2024-04-25T10:16:33.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 --2- 172.21.15.87:0/4055209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc0b2a80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 -- 172.21.15.87:0/4055209 >> 172.21.15.87:0/4055209 conn(0x7febdc066b70 msgr2=0x7febdc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 -- 172.21.15.87:0/4055209 shutdown_connections 2024-04-25T10:16:33.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 -- 172.21.15.87:0/4055209 wait complete. 2024-04-25T10:16:33.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 Processor -- start 2024-04-25T10:16:33.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.890+0000 7febe1200640 1 -- start start 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc14a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febdc0b2fc0 0x7febdc1f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febdc15a0e0 0x7febdc1f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc070b50 con 0x7febdc15a0e0 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc15a4c0 con 0x7febdc0b2fc0 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0723f0 con 0x7febdc071f70 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febdc15a0e0 0x7febdc1f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febdc15a0e0 0x7febdc1f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41724/0 (socket says 172.21.15.87:41724) 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 -- 172.21.15.87:0/1556896852 learned_addr learned my addr 172.21.15.87:0/1556896852 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febd9e00640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc14a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febd9400640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febdc0b2fc0 0x7febdc1f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 -- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 msgr2=0x7febdc14a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc14a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 -- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febdc0b2fc0 msgr2=0x7febdc1f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febdc0b2fc0 0x7febdc1f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 -- 172.21.15.87:0/1556896852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc158190 con 0x7febdc15a0e0 2024-04-25T10:16:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febda800640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febdc15a0e0 0x7febdc1f6f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7febd400c450 tx=0x7febd400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febd2000640 1 -- 172.21.15.87:0/1556896852 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4017070 con 0x7febdc15a0e0 2024-04-25T10:16:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febd2000640 1 -- 172.21.15.87:0/1556896852 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd4005cf0 con 0x7febdc15a0e0 2024-04-25T10:16:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febd2000640 1 -- 172.21.15.87:0/1556896852 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4007270 con 0x7febdc15a0e0 2024-04-25T10:16:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc158470 con 0x7febdc15a0e0 2024-04-25T10:16:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc1f9d20 con 0x7febdc15a0e0 2024-04-25T10:16:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc04ef30 con 0x7febdc15a0e0 2024-04-25T10:16:33.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.894+0000 7febd2000640 1 -- 172.21.15.87:0/1556896852 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febd401a070 con 0x7febdc15a0e0 2024-04-25T10:16:33.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.898+0000 7febd2000640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febbc066f60 0x7febbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.898+0000 7febd2000640 1 -- 172.21.15.87:0/1556896852 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7febd4084280 con 0x7febdc15a0e0 2024-04-25T10:16:33.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.898+0000 7febd9e00640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febbc066f60 0x7febbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.898+0000 7febd9e00640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febbc066f60 0x7febbc069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7febcc001570 tx=0x7febcc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:33.902+0000 7febd2000640 1 -- 172.21.15.87:0/1556896852 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7febd4054c70 con 0x7febdc15a0e0 2024-04-25T10:16:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7febdc0671a0 con 0x7febdc15a0e0 2024-04-25T10:16:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febd2000640 1 -- 172.21.15.87:0/1556896852 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v61) v1 ==== 80+0+122 (secure 0 0 0) 0x7febd4058d10 con 0x7febdc15a0e0 2024-04-25T10:16:34.072 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6837/3387238920 2024-04-26T10:16:14.297697+0000 2024-04-25T10:16:34.072 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6835/3387238920 2024-04-26T10:16:14.297697+0000 2024-04-25T10:16:34.073 INFO:teuthology.orchestra.run.smithi087.stderr:listed 2 entries 2024-04-25T10:16:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febbc066f60 msgr2=0x7febbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febbc066f60 0x7febbc069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7febcc001570 tx=0x7febcc009040 comp rx=0 tx=0).stop 2024-04-25T10:16:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febdc15a0e0 msgr2=0x7febdc1f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febdc15a0e0 0x7febdc1f6f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7febd400c450 tx=0x7febd400c910 comp rx=0 tx=0).stop 2024-04-25T10:16:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 shutdown_connections 2024-04-25T10:16:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febbc066f60 0x7febbc069420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febdc15a0e0 0x7febdc1f6f60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febdc0b2fc0 0x7febdc1f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 --2- 172.21.15.87:0/1556896852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febdc071f70 0x7febdc14a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 >> 172.21.15.87:0/1556896852 conn(0x7febdc066b70 msgr2=0x7febdc150020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 shutdown_connections 2024-04-25T10:16:34.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.066+0000 7febe1200640 1 -- 172.21.15.87:0/1556896852 wait complete. 2024-04-25T10:16:34.093 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:16:34.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.093 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:34.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:16:34.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.121 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:34.171 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:16:34.171 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:34.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:34.181 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:34.181 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:34.227 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:34.227 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:16:34.227 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:34.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:34.236 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:34.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f74600640 1 Processor -- start 2024-04-25T10:16:34.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f74600640 1 -- start start 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f74600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 0x7f7f6c166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f74600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0ccfb0 con 0x7f7f6c15b9e0 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f74600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0cd290 con 0x7f7f6c0ccb20 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f74600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cd570 con 0x7f7f6c0c6a20 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f71c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7f6c15b9e0 0x7f7f6c0cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50804/0 (socket says 172.21.15.87:50804) 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f71c00640 1 -- 172.21.15.87:0/3536674492 learned_addr learned my addr 172.21.15.87:0/3536674492 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f72600640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 0x7f7f6c166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024083651 0 0) 0x7f7f6c0ccfb0 con 0x7f7f6c15b9e0 2024-04-25T10:16:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f54003530 con 0x7f7f6c15b9e0 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944139823 0 0) 0x7f7f6c0cd290 con 0x7f7f6c0ccb20 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f6c0ccfb0 con 0x7f7f6c0ccb20 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2482761531 0 0) 0x7f7f54003530 con 0x7f7f6c15b9e0 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 msgr2=0x7f7f6c166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 0x7f7f6c166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 >> v1:172.21.15.169:6789/0 conn(0x7f7f6c0ccb20 legacy=0x7f7f6c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0cd850 con 0x7f7f6c15b9e0 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f7f60001a20 con 0x7f7f6c15b9e0 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f72600640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 0x7f7f6c166820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 >> v1:172.21.15.87:6789/0 conn(0x7f7f6c15b9e0 legacy=0x7f7f6c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.402+0000 7f7f6be00640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f54003b80 0x7f7f54003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 0x7f7f54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cd570 con 0x7f7f54004d50 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400bab0 con 0x7f7f54003b80 2024-04-25T10:16:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400bcb0 con 0x7f7f6c0c6a20 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f72600640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f71c00640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f54003b80 0x7f7f54003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f71200640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 0x7f7f54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f72600640 1 -- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 msgr2=0x7f7f54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f72600640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c0c6a20 0x7f7f54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f72600640 1 -- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f54003b80 msgr2=0x7f7f54003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f72600640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f54003b80 0x7f7f54003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f72600640 1 -- 172.21.15.87:0/3536674492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54003530 con 0x7f7f54004d50 2024-04-25T10:16:34.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f72600640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f540051d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009cb0 tx=0x7f7f6401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cebc4dba1ead7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f640b2070 con 0x7f7f54004d50 2024-04-25T10:16:34.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f640a58d0 con 0x7f7f54004d50 2024-04-25T10:16:34.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.406+0000 7f7f6be00640 1 -- 172.21.15.87:0/3536674492 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64009880 con 0x7f7f54004d50 2024-04-25T10:16:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- 172.21.15.87:0/3536674492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 msgr2=0x7f7f540051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 --2- 172.21.15.87:0/3536674492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f540051d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009cb0 tx=0x7f7f6401cc10 comp rx=0 tx=0).stop 2024-04-25T10:16:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f72600640 1 -- 172.21.15.87:0/3536674492 reap_dead start 2024-04-25T10:16:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- 172.21.15.87:0/3536674492 shutdown_connections 2024-04-25T10:16:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- 172.21.15.87:0/3536674492 >> 172.21.15.87:0/3536674492 conn(0x7f7f6c09a9d0 msgr2=0x7f7f6c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- 172.21.15.87:0/3536674492 shutdown_connections 2024-04-25T10:16:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- 172.21.15.87:0/3536674492 wait complete. 2024-04-25T10:16:34.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 Processor -- start 2024-04-25T10:16:34.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- start start 2024-04-25T10:16:34.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f6c14b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0b4680 0x7f7f6c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c1f7080 0x7f7f6c1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0d1c30 con 0x7f7f54004d50 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cf740 con 0x7f7f6c0b4680 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f74600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0d1db0 con 0x7f7f6c1f7080 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f6c14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f6c14b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41728/0 (socket says 172.21.15.87:41728) 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 -- 172.21.15.87:0/1766069872 learned_addr learned my addr 172.21.15.87:0/1766069872 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f72600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c1f7080 0x7f7f6c1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71200640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0b4680 0x7f7f6c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 -- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c1f7080 msgr2=0x7f7f6c1f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c1f7080 0x7f7f6c1f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 -- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0b4680 msgr2=0x7f7f6c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0b4680 0x7f7f6c0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71c00640 1 -- 172.21.15.87:0/1766069872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0cb8c0 con 0x7f7f54004d50 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f72600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c1f7080 0x7f7f6c1f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.410+0000 7f7f71200640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0b4680 0x7f7f6c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f71c00640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f6c14b850 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7f60007310 tx=0x7f7f600077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f69600640 1 -- 172.21.15.87:0/1766069872 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6001c070 con 0x7f7f54004d50 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f69600640 1 -- 172.21.15.87:0/1766069872 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60004d70 con 0x7f7f54004d50 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f69600640 1 -- 172.21.15.87:0/1766069872 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60010040 con 0x7f7f54004d50 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c0b6ad0 con 0x7f7f54004d50 2024-04-25T10:16:34.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c0b7090 con 0x7f7f54004d50 2024-04-25T10:16:34.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c0c6e80 con 0x7f7f54004d50 2024-04-25T10:16:34.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f69600640 1 -- 172.21.15.87:0/1766069872 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f600119d0 con 0x7f7f54004d50 2024-04-25T10:16:34.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f69600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f4c066f60 0x7f7f4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f69600640 1 -- 172.21.15.87:0/1766069872 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7f60087e40 con 0x7f7f54004d50 2024-04-25T10:16:34.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.414+0000 7f7f71200640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f4c066f60 0x7f7f4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.418+0000 7f7f71200640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f4c066f60 0x7f7f4c069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c15f2f0 tx=0x7f7f58009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.418+0000 7f7f69600640 1 -- 172.21.15.87:0/1766069872 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f60058d50 con 0x7f7f54004d50 2024-04-25T10:16:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.606+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f6c04ef30 con 0x7f7f54004d50 2024-04-25T10:16:34.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.606+0000 7f7f69600640 1 -- 172.21.15.87:0/1766069872 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f7f60058650 con 0x7f7f54004d50 2024-04-25T10:16:34.613 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:34.613 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:17.322234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6837/427681613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":427681613},{"type":"v1","addr":"172.21.15.87:6837","nonce":427681613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:34.614 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 44 2024-04-25T10:16:34.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.606+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f4c066f60 msgr2=0x7f7f4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.606+0000 7f7f74600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f4c066f60 0x7f7f4c069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c15f2f0 tx=0x7f7f58009040 comp rx=0 tx=0).stop 2024-04-25T10:16:34.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.606+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 msgr2=0x7f7f6c14b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.606+0000 7f7f74600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f6c14b850 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7f60007310 tx=0x7f7f600077d0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 shutdown_connections 2024-04-25T10:16:34.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f4c066f60 0x7f7f4c069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f6c1f7080 0x7f7f6c1f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0b4680 0x7f7f6c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 --2- 172.21.15.87:0/1766069872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f54004d50 0x7f7f6c14b850 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 >> 172.21.15.87:0/1766069872 conn(0x7f7f6c09a9d0 msgr2=0x7f7f6c15ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 shutdown_connections 2024-04-25T10:16:34.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.610+0000 7f7f74600640 1 -- 172.21.15.87:0/1766069872 wait complete. 2024-04-25T10:16:34.634 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:34.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9657200640 1 Processor -- start 2024-04-25T10:16:34.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9657200640 1 -- start start 2024-04-25T10:16:34.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9657200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f96501550b0 0x7f96500cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9657200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f96500b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9657200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9650071050 con 0x7f96500cfe70 2024-04-25T10:16:34.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9657200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965015bda0 con 0x7f96501550b0 2024-04-25T10:16:34.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9657200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965015d040 con 0x7f965015b910 2024-04-25T10:16:34.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.762+0000 7f9655200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f96500b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f9655200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f96500b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:52202/0 (socket says 172.21.15.87:52202) 2024-04-25T10:16:34.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f9655200640 1 -- 172.21.15.87:0/33261668 learned_addr learned my addr 172.21.15.87:0/33261668 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:34.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f964fe00640 1 --2- 172.21.15.87:0/33261668 >> v2:172.21.15.169:3300/0 conn(0x7f96501550b0 0x7f96500cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f9655200640 1 -- 172.21.15.87:0/33261668 >> v2:172.21.15.169:3300/0 conn(0x7f96501550b0 msgr2=0x7f96500cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f9655200640 1 --2- 172.21.15.87:0/33261668 >> v2:172.21.15.169:3300/0 conn(0x7f96501550b0 0x7f96500cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f9655200640 1 -- 172.21.15.87:0/33261668 >> v1:172.21.15.169:6789/0 conn(0x7f96500cfe70 legacy=0x7f965015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f9655200640 1 -- 172.21.15.87:0/33261668 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96500712f0 con 0x7f965015b910 2024-04-25T10:16:34.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f964fe00640 1 --2- 172.21.15.87:0/33261668 >> v2:172.21.15.169:3300/0 conn(0x7f96501550b0 0x7f96500cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:34.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f9655200640 1 --2- 172.21.15.87:0/33261668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f96500b3260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96480098a0 tx=0x7f964801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d376628034a3a40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f964f400640 1 -- 172.21.15.87:0/33261668 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96480ab430 con 0x7f965015b910 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f964f400640 1 -- 172.21.15.87:0/33261668 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96480b0070 con 0x7f965015b910 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.766+0000 7f964f400640 1 -- 172.21.15.87:0/33261668 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96480ab7f0 con 0x7f965015b910 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- 172.21.15.87:0/33261668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 msgr2=0x7f96500b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 --2- 172.21.15.87:0/33261668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f96500b3260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96480098a0 tx=0x7f964801c540 comp rx=0 tx=0).stop 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- 172.21.15.87:0/33261668 shutdown_connections 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 --2- 172.21.15.87:0/33261668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f96500b3260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 --2- 172.21.15.87:0/33261668 >> v2:172.21.15.169:3300/0 conn(0x7f96501550b0 0x7f96500cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- 172.21.15.87:0/33261668 >> 172.21.15.87:0/33261668 conn(0x7f9650066b70 msgr2=0x7f9650066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- 172.21.15.87:0/33261668 shutdown_connections 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- 172.21.15.87:0/33261668 wait complete. 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 Processor -- start 2024-04-25T10:16:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- start start 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500cfe70 0x7f9650143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96501550b0 0x7f9650155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f9650156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965015c030 con 0x7f96501550b0 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96500720b0 con 0x7f96500cfe70 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9657200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965015c1b0 con 0x7f965015b910 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96501550b0 0x7f9650155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96501550b0 0x7f9650155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41736/0 (socket says 172.21.15.87:41736) 2024-04-25T10:16:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 -- 172.21.15.87:0/1739098840 learned_addr learned my addr 172.21.15.87:0/1739098840 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f964fe00640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500cfe70 0x7f9650143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9655200640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f9650156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 -- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 msgr2=0x7f9650156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f9650156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 -- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500cfe70 msgr2=0x7f9650143cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500cfe70 0x7f9650143cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.770+0000 7f9647400640 1 -- 172.21.15.87:0/1739098840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96501f9de0 con 0x7f96501550b0 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f9647400640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96501550b0 0x7f9650155490 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f963800c7c0 tx=0x7f963800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f964cc00640 1 -- 172.21.15.87:0/1739098840 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638016020 con 0x7f96501550b0 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f964cc00640 1 -- 172.21.15.87:0/1739098840 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638006d60 con 0x7f96501550b0 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f964cc00640 1 -- 172.21.15.87:0/1739098840 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638003bc0 con 0x7f96501550b0 2024-04-25T10:16:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9650155b20 con 0x7f96501550b0 2024-04-25T10:16:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501560e0 con 0x7f96501550b0 2024-04-25T10:16:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965004ef30 con 0x7f96501550b0 2024-04-25T10:16:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f964cc00640 1 -- 172.21.15.87:0/1739098840 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f963800a850 con 0x7f96501550b0 2024-04-25T10:16:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f964cc00640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f962c066f60 0x7f962c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f964cc00640 1 -- 172.21.15.87:0/1739098840 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9638084c50 con 0x7f96501550b0 2024-04-25T10:16:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.774+0000 7f964fe00640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f962c066f60 0x7f962c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.778+0000 7f964fe00640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f962c066f60 0x7f962c069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9640009580 tx=0x7f9640009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.778+0000 7f964cc00640 1 -- 172.21.15.87:0/1739098840 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9638020020 con 0x7f96501550b0 2024-04-25T10:16:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f965006df70 con 0x7f96501550b0 2024-04-25T10:16:34.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f964cc00640 1 -- 172.21.15.87:0/1739098840 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f96380555a0 con 0x7f96501550b0 2024-04-25T10:16:34.945 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:34.945 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":61,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:14.297721+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/3387238920":"2024-04-26T10:16:14.297697+0000","172.21.15.87:6835/3387238920":"2024-04-26T10:16:14.297697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:34.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f962c066f60 msgr2=0x7f962c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f962c066f60 0x7f962c069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9640009580 tx=0x7f9640009190 comp rx=0 tx=0).stop 2024-04-25T10:16:34.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96501550b0 msgr2=0x7f9650155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96501550b0 0x7f9650155490 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f963800c7c0 tx=0x7f963800cc80 comp rx=0 tx=0).stop 2024-04-25T10:16:34.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 shutdown_connections 2024-04-25T10:16:34.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f962c066f60 0x7f962c069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965015b910 0x7f9650156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96501550b0 0x7f9650155490 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 --2- 172.21.15.87:0/1739098840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500cfe70 0x7f9650143cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 >> 172.21.15.87:0/1739098840 conn(0x7f9650066b70 msgr2=0x7f96500afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 shutdown_connections 2024-04-25T10:16:34.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:34.938+0000 7f9657200640 1 -- 172.21.15.87:0/1739098840 wait complete. 2024-04-25T10:16:34.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:00.962606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:01.567628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:34.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:35.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce3e00640 1 Processor -- start 2024-04-25T10:16:35.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce3e00640 1 -- start start 2024-04-25T10:16:35.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc159540 0x7f5cdc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5cdc0721f0 0x7f5cdc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cdc159920 con 0x7f5cdc0aebd0 2024-04-25T10:16:35.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cdc0af030 con 0x7f5cdc0721f0 2024-04-25T10:16:35.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc15c2f0 con 0x7f5cdc159540 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc159540 0x7f5cdc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc159540 0x7f5cdc0ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:52230/0 (socket says 172.21.15.87:52230) 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1400640 1 -- 172.21.15.87:0/1041869127 learned_addr learned my addr 172.21.15.87:0/1041869127 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1e00640 1 --2- 172.21.15.87:0/1041869127 >> v2:172.21.15.169:3300/0 conn(0x7f5cdc0721f0 0x7f5cdc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1400640 1 -- 172.21.15.87:0/1041869127 >> v2:172.21.15.169:3300/0 conn(0x7f5cdc0721f0 msgr2=0x7f5cdc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1400640 1 --2- 172.21.15.87:0/1041869127 >> v2:172.21.15.169:3300/0 conn(0x7f5cdc0721f0 0x7f5cdc070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1400640 1 -- 172.21.15.87:0/1041869127 >> v1:172.21.15.169:6789/0 conn(0x7f5cdc0aebd0 legacy=0x7f5cdc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.098+0000 7f5ce1400640 1 -- 172.21.15.87:0/1041869127 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc15c5d0 con 0x7f5cdc159540 2024-04-25T10:16:35.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce1400640 1 --2- 172.21.15.87:0/1041869127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc159540 0x7f5cdc0ae690 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009870 tx=0x7f5cd001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f5d2ee12000c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5cdbe00640 1 -- 172.21.15.87:0/1041869127 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00a95f0 con 0x7f5cdc159540 2024-04-25T10:16:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5cdbe00640 1 -- 172.21.15.87:0/1041869127 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd00a9790 con 0x7f5cdc159540 2024-04-25T10:16:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5cdbe00640 1 -- 172.21.15.87:0/1041869127 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00a9b60 con 0x7f5cdc159540 2024-04-25T10:16:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1041869127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc159540 msgr2=0x7f5cdc0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1041869127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc159540 0x7f5cdc0ae690 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009870 tx=0x7f5cd001c530 comp rx=0 tx=0).stop 2024-04-25T10:16:35.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1041869127 shutdown_connections 2024-04-25T10:16:35.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1041869127 >> v2:172.21.15.169:3300/0 conn(0x7f5cdc0721f0 0x7f5cdc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1041869127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc159540 0x7f5cdc0ae690 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1041869127 >> 172.21.15.87:0/1041869127 conn(0x7f5cdc0669a0 msgr2=0x7f5cdc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1041869127 shutdown_connections 2024-04-25T10:16:35.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.102+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1041869127 wait complete. 2024-04-25T10:16:35.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 Processor -- start 2024-04-25T10:16:35.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 -- start start 2024-04-25T10:16:35.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cdc0721f0 0x7f5cdc1fa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc0aebd0 0x7f5cdc14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cdc159540 0x7f5cdc1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc0710a0 con 0x7f5cdc0721f0 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc06c0d0 con 0x7f5cdc159540 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc071220 con 0x7f5cdc0aebd0 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cdc0721f0 0x7f5cdc1fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cdc0721f0 0x7f5cdc1fa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41748/0 (socket says 172.21.15.87:41748) 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 -- 172.21.15.87:0/1843101600 learned_addr learned my addr 172.21.15.87:0/1843101600 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce0a00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc0aebd0 0x7f5cdc14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1e00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cdc159540 0x7f5cdc1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 -- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc0aebd0 msgr2=0x7f5cdc14b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc0aebd0 0x7f5cdc14b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 -- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cdc159540 msgr2=0x7f5cdc1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cdc159540 0x7f5cdc1519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 -- 172.21.15.87:0/1843101600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc152300 con 0x7f5cdc0721f0 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce0a00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc0aebd0 0x7f5cdc14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce1400640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cdc0721f0 0x7f5cdc1fa790 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5cd001abe0 tx=0x7f5cd00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5cd9600640 1 -- 172.21.15.87:0/1843101600 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00a95f0 con 0x7f5cdc0721f0 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5cd9600640 1 -- 172.21.15.87:0/1843101600 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd00b1800 con 0x7f5cdc0721f0 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cdc1525e0 con 0x7f5cdc0721f0 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5cd9600640 1 -- 172.21.15.87:0/1843101600 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00b1bf0 con 0x7f5cdc0721f0 2024-04-25T10:16:35.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.106+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc1fa270 con 0x7f5cdc0721f0 2024-04-25T10:16:35.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.110+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc04ef30 con 0x7f5cdc0721f0 2024-04-25T10:16:35.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.110+0000 7f5cd9600640 1 -- 172.21.15.87:0/1843101600 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cd00c43f0 con 0x7f5cdc0721f0 2024-04-25T10:16:35.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.110+0000 7f5cd9600640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cbc066f60 0x7f5cbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.110+0000 7f5cd9600640 1 -- 172.21.15.87:0/1843101600 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5cd011f8b0 con 0x7f5cdc0721f0 2024-04-25T10:16:35.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.110+0000 7f5ce0a00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cbc066f60 0x7f5cbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.110+0000 7f5ce0a00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cbc066f60 0x7f5cbc069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8001710 tx=0x7f5cc8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.114+0000 7f5cd9600640 1 -- 172.21.15.87:0/1843101600 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cd00ad020 con 0x7f5cdc0721f0 2024-04-25T10:16:35.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.274+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cdc0ae590 con 0x7f5cdc0721f0 2024-04-25T10:16:35.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.274+0000 7f5cd9600640 1 -- 172.21.15.87:0/1843101600 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cd0016150 con 0x7f5cdc0721f0 2024-04-25T10:16:35.282 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:16:35.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cbc066f60 msgr2=0x7f5cbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cbc066f60 0x7f5cbc069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8001710 tx=0x7f5cc8009040 comp rx=0 tx=0).stop 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cdc0721f0 msgr2=0x7f5cdc1fa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cdc0721f0 0x7f5cdc1fa790 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5cd001abe0 tx=0x7f5cd00192d0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 shutdown_connections 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cbc066f60 0x7f5cbc069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cdc159540 0x7f5cdc1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cdc0aebd0 0x7f5cdc14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 --2- 172.21.15.87:0/1843101600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cdc0721f0 0x7f5cdc1fa790 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 >> 172.21.15.87:0/1843101600 conn(0x7f5cdc0669a0 msgr2=0x7f5cdc066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 shutdown_connections 2024-04-25T10:16:35.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.278+0000 7f5ce3e00640 1 -- 172.21.15.87:0/1843101600 wait complete. 2024-04-25T10:16:35.303 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:35.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 Processor -- start 2024-04-25T10:16:35.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 -- start start 2024-04-25T10:16:35.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb8a0159540 0x7fb8a00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00aebd0 0x7fb8a0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8a0072020 0x7fb8a015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a0159920 con 0x7fb8a0072020 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a015c2f0 con 0x7fb8a0159540 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a015c5d0 con 0x7fb8a00aebd0 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8a0072020 0x7fb8a015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8a0072020 0x7fb8a015f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41758/0 (socket says 172.21.15.87:41758) 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.434+0000 7fb8a6000640 1 -- 172.21.15.87:0/1728457088 learned_addr learned my addr 172.21.15.87:0/1728457088 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a5600640 1 --2- 172.21.15.87:0/1728457088 >> v2:172.21.15.169:3300/0 conn(0x7fb8a0159540 0x7fb8a00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a4c00640 1 --2- 172.21.15.87:0/1728457088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00aebd0 0x7fb8a0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a6000640 1 -- 172.21.15.87:0/1728457088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00aebd0 msgr2=0x7fb8a0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a6000640 1 --2- 172.21.15.87:0/1728457088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00aebd0 0x7fb8a0071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a6000640 1 -- 172.21.15.87:0/1728457088 >> v2:172.21.15.169:3300/0 conn(0x7fb8a0159540 msgr2=0x7fb8a00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:35.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a6000640 1 --2- 172.21.15.87:0/1728457088 >> v2:172.21.15.169:3300/0 conn(0x7fb8a0159540 0x7fb8a00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a6000640 1 -- 172.21.15.87:0/1728457088 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a015c8b0 con 0x7fb8a0072020 2024-04-25T10:16:35.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a5600640 1 --2- 172.21.15.87:0/1728457088 >> v2:172.21.15.169:3300/0 conn(0x7fb8a0159540 0x7fb8a00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:35.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a4c00640 1 --2- 172.21.15.87:0/1728457088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00aebd0 0x7fb8a0071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:35.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a6000640 1 --2- 172.21.15.87:0/1728457088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0072020 0x7fb8a015f590 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00b4c20 tx=0x7fb89801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c5db89f68c7d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb89fe00640 1 -- 172.21.15.87:0/1728457088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8980ac020 con 0x7fb8a0072020 2024-04-25T10:16:35.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb89fe00640 1 -- 172.21.15.87:0/1728457088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8980a6450 con 0x7fb8a0072020 2024-04-25T10:16:35.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb89fe00640 1 -- 172.21.15.87:0/1728457088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8980a6820 con 0x7fb8a0072020 2024-04-25T10:16:35.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.438+0000 7fb8a8000640 1 -- 172.21.15.87:0/1728457088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0072020 msgr2=0x7fb8a015f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 --2- 172.21.15.87:0/1728457088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0072020 0x7fb8a015f590 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00b4c20 tx=0x7fb89801c730 comp rx=0 tx=0).stop 2024-04-25T10:16:35.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- 172.21.15.87:0/1728457088 shutdown_connections 2024-04-25T10:16:35.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 --2- 172.21.15.87:0/1728457088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0072020 0x7fb8a015f590 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 --2- 172.21.15.87:0/1728457088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00aebd0 0x7fb8a0071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 --2- 172.21.15.87:0/1728457088 >> v2:172.21.15.169:3300/0 conn(0x7fb8a0159540 0x7fb8a00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- 172.21.15.87:0/1728457088 >> 172.21.15.87:0/1728457088 conn(0x7fb8a00669a0 msgr2=0x7fb8a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- 172.21.15.87:0/1728457088 shutdown_connections 2024-04-25T10:16:35.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- 172.21.15.87:0/1728457088 wait complete. 2024-04-25T10:16:35.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 Processor -- start 2024-04-25T10:16:35.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- start start 2024-04-25T10:16:35.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a0072020 0x7fb8a014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00aebd0 0x7fb8a014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0159540 0x7fb8a00cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a0070350 con 0x7fb8a0159540 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a006c0d0 con 0x7fb8a00aebd0 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a00704d0 con 0x7fb8a0072020 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0159540 0x7fb8a00cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0159540 0x7fb8a00cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41764/0 (socket says 172.21.15.87:41764) 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 -- 172.21.15.87:0/2211633304 learned_addr learned my addr 172.21.15.87:0/2211633304 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a5600640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a0072020 0x7fb8a014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a4c00640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00aebd0 0x7fb8a014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 -- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a0072020 msgr2=0x7fb8a014b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a0072020 0x7fb8a014b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 -- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00aebd0 msgr2=0x7fb8a014f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00aebd0 0x7fb8a014f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 -- 172.21.15.87:0/2211633304 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a014fb40 con 0x7fb8a0159540 2024-04-25T10:16:35.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.442+0000 7fb8a6000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0159540 0x7fb8a00cd960 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb89801cfd0 tx=0x7fb8980194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb89d600640 1 -- 172.21.15.87:0/2211633304 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8980ac040 con 0x7fb8a0159540 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb89d600640 1 -- 172.21.15.87:0/2211633304 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8980b6070 con 0x7fb8a0159540 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a014fe20 con 0x7fb8a0159540 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb89d600640 1 -- 172.21.15.87:0/2211633304 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898009510 con 0x7fb8a0159540 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a01503e0 con 0x7fb8a0159540 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a004ef30 con 0x7fb8a0159540 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb89d600640 1 -- 172.21.15.87:0/2211633304 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8980af020 con 0x7fb8a0159540 2024-04-25T10:16:35.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb89d600640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb880066f60 0x7fb880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb89d600640 1 -- 172.21.15.87:0/2211633304 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb89811f2e0 con 0x7fb8a0159540 2024-04-25T10:16:35.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb8a5600640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb880066f60 0x7fb880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.446+0000 7fb8a5600640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb880066f60 0x7fb880069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb894006fd0 tx=0x7fb894008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.450+0000 7fb89d600640 1 -- 172.21.15.87:0/2211633304 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8980165e0 con 0x7fb8a0159540 2024-04-25T10:16:35.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8a00ccef0 con 0x7fb8a0159540 2024-04-25T10:16:35.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb89d600640 1 -- 172.21.15.87:0/2211633304 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8980f0280 con 0x7fb8a0159540 2024-04-25T10:16:35.612 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:16:35.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb880066f60 msgr2=0x7fb880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb880066f60 0x7fb880069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb894006fd0 tx=0x7fb894008040 comp rx=0 tx=0).stop 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0159540 msgr2=0x7fb8a00cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0159540 0x7fb8a00cd960 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb89801cfd0 tx=0x7fb8980194d0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 shutdown_connections 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb880066f60 0x7fb880069420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8a0159540 0x7fb8a00cd960 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00aebd0 0x7fb8a014f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 --2- 172.21.15.87:0/2211633304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a0072020 0x7fb8a014b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 >> 172.21.15.87:0/2211633304 conn(0x7fb8a00669a0 msgr2=0x7fb8a006de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 shutdown_connections 2024-04-25T10:16:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.606+0000 7fb8a8000640 1 -- 172.21.15.87:0/2211633304 wait complete. 2024-04-25T10:16:35.633 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:35.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.762+0000 7f98c5a00640 1 Processor -- start 2024-04-25T10:16:35.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.762+0000 7f98c5a00640 1 -- start start 2024-04-25T10:16:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98c5a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98c015a0e0 0x7f98c015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98c5a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f98c006c0d0 0x7f98c006c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98c5a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c00b2fd0 con 0x7f98c00aea50 2024-04-25T10:16:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98c5a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c00af1d0 con 0x7f98c006c0d0 2024-04-25T10:16:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98c5a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c00b32b0 con 0x7f98c015a0e0 2024-04-25T10:16:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f98c006c0d0 0x7f98c006c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98be000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f98c00aea50 0x7f98c00b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50816/0 (socket says 172.21.15.87:50816) 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f98c006c0d0 0x7f98c006c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41766/0 (socket says 172.21.15.87:41766) 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98be000640 1 -- 172.21.15.87:0/3321806476 learned_addr learned my addr 172.21.15.87:0/3321806476 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bea00640 1 --2- 172.21.15.87:0/3321806476 >> v2:172.21.15.169:3300/0 conn(0x7f98c015a0e0 0x7f98c015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bf400640 1 -- 172.21.15.87:0/3321806476 >> v2:172.21.15.169:3300/0 conn(0x7f98c015a0e0 msgr2=0x7f98c015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bf400640 1 --2- 172.21.15.87:0/3321806476 >> v2:172.21.15.169:3300/0 conn(0x7f98c015a0e0 0x7f98c015b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bf400640 1 -- 172.21.15.87:0/3321806476 >> v1:172.21.15.87:6789/0 conn(0x7f98c00aea50 legacy=0x7f98c00b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bd600640 1 -- 172.21.15.87:0/3321806476 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036009978 0 0) 0x7f98c00b2fd0 con 0x7f98c00aea50 2024-04-25T10:16:35.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.766+0000 7f98bf400640 1 -- 172.21.15.87:0/3321806476 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00b3590 con 0x7f98c006c0d0 2024-04-25T10:16:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98bf400640 1 --2- 172.21.15.87:0/3321806476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c006c0d0 0x7f98c006c500 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f98c006e4c0 tx=0x7f98b801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9995e1ad97d16dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98bd600640 1 -- 172.21.15.87:0/3321806476 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80b0070 con 0x7f98c006c0d0 2024-04-25T10:16:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98bd600640 1 -- 172.21.15.87:0/3321806476 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b80ab600 con 0x7f98c006c0d0 2024-04-25T10:16:35.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98bd600640 1 -- 172.21.15.87:0/3321806476 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80ab9d0 con 0x7f98c006c0d0 2024-04-25T10:16:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- 172.21.15.87:0/3321806476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c006c0d0 msgr2=0x7f98c006c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 --2- 172.21.15.87:0/3321806476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c006c0d0 0x7f98c006c500 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f98c006e4c0 tx=0x7f98b801c360 comp rx=0 tx=0).stop 2024-04-25T10:16:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- 172.21.15.87:0/3321806476 shutdown_connections 2024-04-25T10:16:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 --2- 172.21.15.87:0/3321806476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c006c0d0 0x7f98c006c500 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 --2- 172.21.15.87:0/3321806476 >> v2:172.21.15.169:3300/0 conn(0x7f98c015a0e0 0x7f98c015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- 172.21.15.87:0/3321806476 >> 172.21.15.87:0/3321806476 conn(0x7f98c00669a0 msgr2=0x7f98c0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- 172.21.15.87:0/3321806476 shutdown_connections 2024-04-25T10:16:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- 172.21.15.87:0/3321806476 wait complete. 2024-04-25T10:16:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 Processor -- start 2024-04-25T10:16:35.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- start start 2024-04-25T10:16:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98c006c0d0 0x7f98c0159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00aea50 0x7f98c00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98c015a0e0 0x7f98c00ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00cf880 con 0x7f98c00aea50 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00cf700 con 0x7f98c015a0e0 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98c5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00cfa00 con 0x7f98c006c0d0 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00aea50 0x7f98c00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00aea50 0x7f98c00adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41774/0 (socket says 172.21.15.87:41774) 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.770+0000 7f98be000640 1 -- 172.21.15.87:0/2648809579 learned_addr learned my addr 172.21.15.87:0/2648809579 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98bea00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98c006c0d0 0x7f98c0159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98bf400640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98c015a0e0 0x7f98c00ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98be000640 1 -- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98c006c0d0 msgr2=0x7f98c0159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98be000640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98c006c0d0 0x7f98c0159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98be000640 1 -- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98c015a0e0 msgr2=0x7f98c00ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98be000640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98c015a0e0 0x7f98c00ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98be000640 1 -- 172.21.15.87:0/2648809579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00cd420 con 0x7f98c00aea50 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98be000640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00aea50 0x7f98c00adaa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0011f0 tx=0x7f98ac00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98b6a00640 1 -- 172.21.15.87:0/2648809579 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac018070 con 0x7f98c00aea50 2024-04-25T10:16:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98b6a00640 1 -- 172.21.15.87:0/2648809579 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac0074e0 con 0x7f98c00aea50 2024-04-25T10:16:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98b6a00640 1 -- 172.21.15.87:0/2648809579 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004a10 con 0x7f98c00aea50 2024-04-25T10:16:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c01584b0 con 0x7f98c00aea50 2024-04-25T10:16:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00cd980 con 0x7f98c00aea50 2024-04-25T10:16:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c004ef30 con 0x7f98c00aea50 2024-04-25T10:16:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98b6a00640 1 -- 172.21.15.87:0/2648809579 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98ac004bb0 con 0x7f98c00aea50 2024-04-25T10:16:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98b6a00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f98a0066f60 0x7f98a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.774+0000 7f98b6a00640 1 -- 172.21.15.87:0/2648809579 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f98ac0843f0 con 0x7f98c00aea50 2024-04-25T10:16:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.778+0000 7f98bea00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f98a0066f60 0x7f98a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.778+0000 7f98bea00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f98a0066f60 0x7f98a0069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98b00073f0 tx=0x7f98b000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.782+0000 7f98b6a00640 1 -- 172.21.15.87:0/2648809579 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98c00cd420 con 0x7f98c00aea50 2024-04-25T10:16:35.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98c00b2790 con 0x7f98c00aea50 2024-04-25T10:16:35.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98b6a00640 1 -- 172.21.15.87:0/2648809579 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98ac055430 con 0x7f98c00aea50 2024-04-25T10:16:35.945 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f98a0066f60 msgr2=0x7f98a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f98a0066f60 0x7f98a0069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98b00073f0 tx=0x7f98b000b040 comp rx=0 tx=0).stop 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00aea50 msgr2=0x7f98c00adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00aea50 0x7f98c00adaa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0011f0 tx=0x7f98ac00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 shutdown_connections 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f98a0066f60 0x7f98a0069420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98c015a0e0 0x7f98c00ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00aea50 0x7f98c00adaa0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 --2- 172.21.15.87:0/2648809579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98c006c0d0 0x7f98c0159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 >> 172.21.15.87:0/2648809579 conn(0x7f98c00669a0 msgr2=0x7f98c015ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 shutdown_connections 2024-04-25T10:16:35.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:35.938+0000 7f98c5a00640 1 -- 172.21.15.87:0/2648809579 wait complete. 2024-04-25T10:16:35.965 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:36.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d75000640 1 Processor -- start 2024-04-25T10:16:36.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d75000640 1 -- start start 2024-04-25T10:16:36.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d75000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d700d1820 0x7f0d700d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d75000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d700b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d75000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700b1870 con 0x7f0d700b13e0 2024-04-25T10:16:36.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d75000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d700b1af0 con 0x7f0d700d1820 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d75000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d700b1dd0 con 0x7f0d7015e050 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d6de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d700d1820 0x7f0d700d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d6de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0d700d1820 0x7f0d700d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41782/0 (socket says 172.21.15.87:41782) 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d6de00640 1 -- 172.21.15.87:0/663953995 learned_addr learned my addr 172.21.15.87:0/663953995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d65400640 1 --2- 172.21.15.87:0/663953995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d700b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d6de00640 1 -- 172.21.15.87:0/663953995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 msgr2=0x7f0d700b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d6de00640 1 --2- 172.21.15.87:0/663953995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d700b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d6de00640 1 -- 172.21.15.87:0/663953995 >> v1:172.21.15.169:6789/0 conn(0x7f0d700b13e0 legacy=0x7f0d700ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.094+0000 7f0d6de00640 1 -- 172.21.15.87:0/663953995 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700b20b0 con 0x7f0d700d1820 2024-04-25T10:16:36.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d6de00640 1 --2- 172.21.15.87:0/663953995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700d1820 0x7f0d700d1c00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009870 tx=0x7f0d6001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06c73c45e997466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d6d400640 1 -- 172.21.15.87:0/663953995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600a9620 con 0x7f0d700d1820 2024-04-25T10:16:36.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d6d400640 1 -- 172.21.15.87:0/663953995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d600a97c0 con 0x7f0d700d1820 2024-04-25T10:16:36.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d6d400640 1 -- 172.21.15.87:0/663953995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600a9b90 con 0x7f0d700d1820 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 -- 172.21.15.87:0/663953995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700d1820 msgr2=0x7f0d700d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 --2- 172.21.15.87:0/663953995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700d1820 0x7f0d700d1c00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009870 tx=0x7f0d6001c560 comp rx=0 tx=0).stop 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 -- 172.21.15.87:0/663953995 shutdown_connections 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 --2- 172.21.15.87:0/663953995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d700b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 --2- 172.21.15.87:0/663953995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700d1820 0x7f0d700d1c00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 -- 172.21.15.87:0/663953995 >> 172.21.15.87:0/663953995 conn(0x7f0d7009a9d0 msgr2=0x7f0d7009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 -- 172.21.15.87:0/663953995 shutdown_connections 2024-04-25T10:16:36.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.098+0000 7f0d75000640 1 -- 172.21.15.87:0/663953995 wait complete. 2024-04-25T10:16:36.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 Processor -- start 2024-04-25T10:16:36.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 -- start start 2024-04-25T10:16:36.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700b13e0 0x7f0d700c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d700d1820 0x7f0d7014f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d7015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7016ab20 con 0x7f0d700b13e0 2024-04-25T10:16:36.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7016a9a0 con 0x7f0d700d1820 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7016aca0 con 0x7f0d7015e050 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700b13e0 0x7f0d700c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700b13e0 0x7f0d700c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41784/0 (socket says 172.21.15.87:41784) 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 -- 172.21.15.87:0/3251733874 learned_addr learned my addr 172.21.15.87:0/3251733874 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d65400640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d700d1820 0x7f0d7014f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6e800640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d7015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 -- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 msgr2=0x7f0d7015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d7015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 -- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d700d1820 msgr2=0x7f0d7014f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d700d1820 0x7f0d7014f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 -- 172.21.15.87:0/3251733874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700c5fb0 con 0x7f0d700b13e0 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d65400640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d700d1820 0x7f0d7014f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:36.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d6de00640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700b13e0 0x7f0d700c7010 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0d600ab040 tx=0x7f0d6001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d66a00640 1 -- 172.21.15.87:0/3251733874 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600bc450 con 0x7f0d700b13e0 2024-04-25T10:16:36.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d66a00640 1 -- 172.21.15.87:0/3251733874 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d600bc610 con 0x7f0d700b13e0 2024-04-25T10:16:36.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d66a00640 1 -- 172.21.15.87:0/3251733874 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600bca00 con 0x7f0d700b13e0 2024-04-25T10:16:36.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d700c6250 con 0x7f0d700b13e0 2024-04-25T10:16:36.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d700c6810 con 0x7f0d700b13e0 2024-04-25T10:16:36.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.102+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7004ef30 con 0x7f0d700b13e0 2024-04-25T10:16:36.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.106+0000 7f0d66a00640 1 -- 172.21.15.87:0/3251733874 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d600b13f0 con 0x7f0d700b13e0 2024-04-25T10:16:36.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.106+0000 7f0d66a00640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d4c066f60 0x7f0d4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.106+0000 7f0d66a00640 1 -- 172.21.15.87:0/3251733874 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0d6011f310 con 0x7f0d700b13e0 2024-04-25T10:16:36.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.106+0000 7f0d65400640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d4c066f60 0x7f0d4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.106+0000 7f0d65400640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d4c066f60 0x7f0d4c069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d58003fd0 tx=0x7f0d580039e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.110+0000 7f0d66a00640 1 -- 172.21.15.87:0/3251733874 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d600c3030 con 0x7f0d700b13e0 2024-04-25T10:16:36.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.298+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d70161480 con 0x7f0d700b13e0 2024-04-25T10:16:36.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.298+0000 7f0d66a00640 1 -- 172.21.15.87:0/3251733874 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f0d600ad020 con 0x7f0d700b13e0 2024-04-25T10:16:36.306 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:36.307 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:17.322234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6837/427681613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":427681613},{"type":"v1","addr":"172.21.15.87:6837","nonce":427681613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:36.307 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 44 2024-04-25T10:16:36.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d4c066f60 msgr2=0x7f0d4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d4c066f60 0x7f0d4c069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d58003fd0 tx=0x7f0d580039e0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700b13e0 msgr2=0x7f0d700c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700b13e0 0x7f0d700c7010 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0d600ab040 tx=0x7f0d6001ada0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 shutdown_connections 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d4c066f60 0x7f0d4c069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d7015e050 0x7f0d7015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d700d1820 0x7f0d7014f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 --2- 172.21.15.87:0/3251733874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d700b13e0 0x7f0d700c7010 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 >> 172.21.15.87:0/3251733874 conn(0x7f0d7009a9d0 msgr2=0x7f0d7006fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 shutdown_connections 2024-04-25T10:16:36.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.302+0000 7f0d75000640 1 -- 172.21.15.87:0/3251733874 wait complete. 2024-04-25T10:16:36.327 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:36.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.454+0000 7fd526200640 1 Processor -- start 2024-04-25T10:16:36.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.454+0000 7fd526200640 1 -- start start 2024-04-25T10:16:36.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd526200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52015a0e0 0x7fd52015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd526200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5200b3190 0x7fd52006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd526200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52015b6e0 con 0x7fd520071f70 2024-04-25T10:16:36.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd526200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd520070ed0 con 0x7fd5200b3190 2024-04-25T10:16:36.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd526200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200711b0 con 0x7fd52015a0e0 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd51f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52015a0e0 0x7fd52015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd51fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5200b3190 0x7fd52006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd516a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd520071f70 0x7fd5200b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:47316/0 (socket says 172.21.15.87:47316) 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd516a00640 1 -- 172.21.15.87:0/2035357206 learned_addr learned my addr 172.21.15.87:0/2035357206 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd51fe00640 1 -- 172.21.15.87:0/2035357206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52015a0e0 msgr2=0x7fd52015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd51fe00640 1 --2- 172.21.15.87:0/2035357206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52015a0e0 0x7fd52015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd51fe00640 1 -- 172.21.15.87:0/2035357206 >> v1:172.21.15.169:6789/0 conn(0x7fd520071f70 legacy=0x7fd5200b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd51fe00640 1 -- 172.21.15.87:0/2035357206 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520071490 con 0x7fd5200b3190 2024-04-25T10:16:36.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.458+0000 7fd51ea00640 1 -- 172.21.15.87:0/2035357206 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658677937 0 0) 0x7fd508001120 con 0x7fd520071f70 2024-04-25T10:16:36.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd51fe00640 1 --2- 172.21.15.87:0/2035357206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200b3190 0x7fd52006c090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd518008330 tx=0x7fd51801c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ca4abef0ba0246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd51ea00640 1 -- 172.21.15.87:0/2035357206 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180a57d0 con 0x7fd5200b3190 2024-04-25T10:16:36.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd51ea00640 1 -- 172.21.15.87:0/2035357206 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5180a5970 con 0x7fd5200b3190 2024-04-25T10:16:36.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd51ea00640 1 -- 172.21.15.87:0/2035357206 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180a5d20 con 0x7fd5200b3190 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 -- 172.21.15.87:0/2035357206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200b3190 msgr2=0x7fd52006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 --2- 172.21.15.87:0/2035357206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200b3190 0x7fd52006c090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd518008330 tx=0x7fd51801c990 comp rx=0 tx=0).stop 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 -- 172.21.15.87:0/2035357206 shutdown_connections 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 --2- 172.21.15.87:0/2035357206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200b3190 0x7fd52006c090 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 --2- 172.21.15.87:0/2035357206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52015a0e0 0x7fd52015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 -- 172.21.15.87:0/2035357206 >> 172.21.15.87:0/2035357206 conn(0x7fd520066b70 msgr2=0x7fd520066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 -- 172.21.15.87:0/2035357206 shutdown_connections 2024-04-25T10:16:36.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 -- 172.21.15.87:0/2035357206 wait complete. 2024-04-25T10:16:36.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 Processor -- start 2024-04-25T10:16:36.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.462+0000 7fd526200640 1 -- start start 2024-04-25T10:16:36.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520071f70 0x7fd520156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5200b3190 0x7fd5200ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52015a0e0 0x7fd5200add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd520070b50 con 0x7fd5200b3190 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52015a4c0 con 0x7fd52015a0e0 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200723b0 con 0x7fd520071f70 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5200b3190 0x7fd5200ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5200b3190 0x7fd5200ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41796/0 (socket says 172.21.15.87:41796) 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 -- 172.21.15.87:0/332423287 learned_addr learned my addr 172.21.15.87:0/332423287 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd51f400640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520071f70 0x7fd520156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd51fe00640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52015a0e0 0x7fd5200add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 -- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520071f70 msgr2=0x7fd520156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520071f70 0x7fd520156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 -- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52015a0e0 msgr2=0x7fd5200add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52015a0e0 0x7fd5200add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 -- 172.21.15.87:0/332423287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520152ae0 con 0x7fd5200b3190 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd516a00640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5200b3190 0x7fd5200ad840 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c780 tx=0x7fd50800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd517e00640 1 -- 172.21.15.87:0/332423287 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508018070 con 0x7fd5200b3190 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd517e00640 1 -- 172.21.15.87:0/332423287 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5080072f0 con 0x7fd5200b3190 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd520152dc0 con 0x7fd5200b3190 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd517e00640 1 -- 172.21.15.87:0/332423287 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080047f0 con 0x7fd5200b3190 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd520153380 con 0x7fd5200b3190 2024-04-25T10:16:36.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52004ef30 con 0x7fd5200b3190 2024-04-25T10:16:36.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.466+0000 7fd517e00640 1 -- 172.21.15.87:0/332423287 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd50801b070 con 0x7fd5200b3190 2024-04-25T10:16:36.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.470+0000 7fd517e00640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.470+0000 7fd517e00640 1 -- 172.21.15.87:0/332423287 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd5080844d0 con 0x7fd5200b3190 2024-04-25T10:16:36.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.470+0000 7fd51f400640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.470+0000 7fd51f400640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4fc066f60 0x7fd4fc069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b240 tx=0x7fd51000b3a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.474+0000 7fd517e00640 1 -- 172.21.15.87:0/332423287 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd52015b6e0 con 0x7fd5200b3190 2024-04-25T10:16:36.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5200703e0 con 0x7fd5200b3190 2024-04-25T10:16:36.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd517e00640 1 -- 172.21.15.87:0/332423287 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16782 (secure 0 0 0) 0x7fd508054d90 con 0x7fd5200b3190 2024-04-25T10:16:36.637 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:36.637 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":61,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:14.297721+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/3387238920":"2024-04-26T10:16:14.297697+0000","172.21.15.87:6835/3387238920":"2024-04-26T10:16:14.297697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4fc066f60 msgr2=0x7fd4fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4fc066f60 0x7fd4fc069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b240 tx=0x7fd51000b3a0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5200b3190 msgr2=0x7fd5200ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5200b3190 0x7fd5200ad840 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c780 tx=0x7fd50800cc40 comp rx=0 tx=0).stop 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 shutdown_connections 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52015a0e0 0x7fd5200add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5200b3190 0x7fd5200ad840 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 --2- 172.21.15.87:0/332423287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520071f70 0x7fd520156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 >> 172.21.15.87:0/332423287 conn(0x7fd520066b70 msgr2=0x7fd5200b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 shutdown_connections 2024-04-25T10:16:36.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.630+0000 7fd526200640 1 -- 172.21.15.87:0/332423287 wait complete. 2024-04-25T10:16:36.658 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:16:36.659 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:36.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.786+0000 7f0bed000640 1 Processor -- start 2024-04-25T10:16:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.786+0000 7f0bed000640 1 -- start start 2024-04-25T10:16:36.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0bed000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0be815f420 0x7f0be8166900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0bed000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80cfde0 con 0x7f0be80b3dd0 2024-04-25T10:16:36.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0bed000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80d0080 con 0x7f0be80ce290 2024-04-25T10:16:36.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0bed000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be80d0360 con 0x7f0be815f420 2024-04-25T10:16:36.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0bdea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0be80b3dd0 0x7f0be815ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50820/0 (socket says 172.21.15.87:50820) 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0bdea00640 1 -- 172.21.15.87:0/3489746490 learned_addr learned my addr 172.21.15.87:0/3489746490 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3489746490 >> v2:172.21.15.169:3300/0 conn(0x7f0be815f420 0x7f0be8166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146054343 0 0) 0x7f0be80cfde0 con 0x7f0be80b3dd0 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bcc003530 con 0x7f0be80b3dd0 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479531108 0 0) 0x7f0be80d0080 con 0x7f0be80ce290 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be80cfde0 con 0x7f0be80ce290 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 670614134 0 0) 0x7f0bcc003530 con 0x7f0be80b3dd0 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 >> v2:172.21.15.169:3300/0 conn(0x7f0be815f420 msgr2=0x7f0be8166900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 --2- 172.21.15.87:0/3489746490 >> v2:172.21.15.169:3300/0 conn(0x7f0be815f420 0x7f0be8166900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 >> v1:172.21.15.169:6789/0 conn(0x7f0be80ce290 legacy=0x7f0be80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80d0640 con 0x7f0be80b3dd0 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0bd0001680 con 0x7f0be80b3dd0 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 >> v1:172.21.15.87:6789/0 conn(0x7f0be80b3dd0 legacy=0x7f0be815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bcc003b50 0x7f0bcc003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc004810 0x7f0bcc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bcc0051d0 0x7f0bcc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80d0360 con 0x7f0bcc0051d0 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc00c050 con 0x7f0bcc003b50 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc00c250 con 0x7f0bcc004810 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bcc0051d0 0x7f0bcc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7400640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bcc003b50 0x7f0bcc003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0bdea00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc004810 0x7f0bcc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7e00640 1 -- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc004810 msgr2=0x7f0bcc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:36.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc004810 0x7f0bcc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7e00640 1 -- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bcc003b50 msgr2=0x7f0bcc003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:36.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bcc003b50 0x7f0bcc003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.790+0000 7f0be7e00640 1 -- 172.21.15.87:0/3489746490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc003530 con 0x7f0bcc0051d0 2024-04-25T10:16:36.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0be7e00640 1 -- 172.21.15.87:0/3489746490 reap_dead start 2024-04-25T10:16:36.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bcc0051d0 0x7f0bcc00b9d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009f90 tx=0x7f0be001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fd7f24f35a0b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00b2070 con 0x7f0bcc0051d0 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00a5c90 con 0x7f0bcc0051d0 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0be6a00640 1 -- 172.21.15.87:0/3489746490 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0009c10 con 0x7f0bcc0051d0 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0bed000640 1 -- 172.21.15.87:0/3489746490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bcc0051d0 msgr2=0x7f0bcc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0bed000640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bcc0051d0 0x7f0bcc00b9d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009f90 tx=0x7f0be001cc80 comp rx=0 tx=0).stop 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0bed000640 1 -- 172.21.15.87:0/3489746490 shutdown_connections 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0bed000640 1 --2- 172.21.15.87:0/3489746490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bcc0051d0 0x7f0bcc00b9d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0bed000640 1 -- 172.21.15.87:0/3489746490 >> 172.21.15.87:0/3489746490 conn(0x7f0be80b32b0 msgr2=0x7f0be80b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0bed000640 1 -- 172.21.15.87:0/3489746490 shutdown_connections 2024-04-25T10:16:36.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.794+0000 7f0bed000640 1 -- 172.21.15.87:0/3489746490 wait complete. 2024-04-25T10:16:36.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 Processor -- start 2024-04-25T10:16:36.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 -- start start 2024-04-25T10:16:36.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc0051d0 0x7f0be815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0be80c8830 0x7f0be80c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0be806ed30 0x7f0be806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be815ee90 con 0x7f0be806ed30 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be815ed10 con 0x7f0be80c8830 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be815f010 con 0x7f0bcc0051d0 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0be806ed30 0x7f0be806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0be806ed30 0x7f0be806f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41826/0 (socket says 172.21.15.87:41826) 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 -- 172.21.15.87:0/3339736913 learned_addr learned my addr 172.21.15.87:0/3339736913 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7400640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc0051d0 0x7f0be815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bdea00640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0be80c8830 0x7f0be80c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 -- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc0051d0 msgr2=0x7f0be815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc0051d0 0x7f0be815bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 -- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0be80c8830 msgr2=0x7f0be80c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0be80c8830 0x7f0be80c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 -- 172.21.15.87:0/3339736913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80c9280 con 0x7f0be806ed30 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7400640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc0051d0 0x7f0be815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0be7e00640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0be806ed30 0x7f0be806f190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0be001cab0 tx=0x7f0be0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bdfe00640 1 -- 172.21.15.87:0/3339736913 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00bb070 con 0x7f0be806ed30 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bdfe00640 1 -- 172.21.15.87:0/3339736913 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00ada60 con 0x7f0be806ed30 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be80c9560 con 0x7f0be806ed30 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bdfe00640 1 -- 172.21.15.87:0/3339736913 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00b2040 con 0x7f0be806ed30 2024-04-25T10:16:36.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be80c9b20 con 0x7f0be806ed30 2024-04-25T10:16:36.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.798+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be806f6d0 con 0x7f0be806ed30 2024-04-25T10:16:36.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.802+0000 7f0bdfe00640 1 -- 172.21.15.87:0/3339736913 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0be00be070 con 0x7f0be806ed30 2024-04-25T10:16:36.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.802+0000 7f0bdfe00640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bc4066f90 0x7f0bc4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.802+0000 7f0bdfe00640 1 -- 172.21.15.87:0/3339736913 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0be011ed30 con 0x7f0be806ed30 2024-04-25T10:16:36.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.802+0000 7f0be7400640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bc4066f90 0x7f0bc4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.802+0000 7f0be7400640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bc4066f90 0x7f0bc4069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c10 tx=0x7f0bd8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.806+0000 7f0bdfe00640 1 -- 172.21.15.87:0/3339736913 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0be0016990 con 0x7f0be806ed30 2024-04-25T10:16:36.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.990+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0be8075120 con 0x7f0be806ed30 2024-04-25T10:16:36.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.990+0000 7f0bdfe00640 1 -- 172.21.15.87:0/3339736913 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f0be00ef680 con 0x7f0be806ed30 2024-04-25T10:16:36.999 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:36.999 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:17.322234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6837/427681613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":427681613},{"type":"v1","addr":"172.21.15.87:6837","nonce":427681613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:36.999 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 44 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bc4066f90 msgr2=0x7f0bc4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bc4066f90 0x7f0bc4069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c10 tx=0x7f0bd8009040 comp rx=0 tx=0).stop 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0be806ed30 msgr2=0x7f0be806f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0be806ed30 0x7f0be806f190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0be001cab0 tx=0x7f0be0009750 comp rx=0 tx=0).stop 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 shutdown_connections 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0bc4066f90 0x7f0bc4069450 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0be806ed30 0x7f0be806f190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0be80c8830 0x7f0be80c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 --2- 172.21.15.87:0/3339736913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bcc0051d0 0x7f0be815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 >> 172.21.15.87:0/3339736913 conn(0x7f0be80b32b0 msgr2=0x7f0be80b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 shutdown_connections 2024-04-25T10:16:37.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:36.994+0000 7f0bed000640 1 -- 172.21.15.87:0/3339736913 wait complete. 2024-04-25T10:16:37.019 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:37.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.146+0000 7fca9fa00640 1 Processor -- start 2024-04-25T10:16:37.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.146+0000 7fca9fa00640 1 -- start start 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9fa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca9815bc40 0x7fca9806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 0x7fca980b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9fa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca980cf930 0x7fca980cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9fa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca980b2770 con 0x7fca9815bc40 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9fa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9806c670 con 0x7fca980cf930 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9806c950 con 0x7fca980cda10 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca9815bc40 0x7fca9806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca9815bc40 0x7fca9806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41842/0 (socket says 172.21.15.87:41842) 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 -- 172.21.15.87:0/3405642653 learned_addr learned my addr 172.21.15.87:0/3405642653 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 -- 172.21.15.87:0/3405642653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 msgr2=0x7fca980b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 --2- 172.21.15.87:0/3405642653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 0x7fca980b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 -- 172.21.15.87:0/3405642653 >> v2:172.21.15.169:3300/0 conn(0x7fca980cf930 msgr2=0x7fca980cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 --2- 172.21.15.87:0/3405642653 >> v2:172.21.15.169:3300/0 conn(0x7fca980cf930 0x7fca980cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.150+0000 7fca9d000640 1 -- 172.21.15.87:0/3405642653 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9806cc30 con 0x7fca9815bc40 2024-04-25T10:16:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9d000640 1 --2- 172.21.15.87:0/3405642653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca9806c090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fca980b3f80 tx=0x7fca8801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e013b344da861b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca97400640 1 -- 172.21.15.87:0/3405642653 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca880a5670 con 0x7fca9815bc40 2024-04-25T10:16:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca97400640 1 -- 172.21.15.87:0/3405642653 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca880a5810 con 0x7fca9815bc40 2024-04-25T10:16:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca97400640 1 -- 172.21.15.87:0/3405642653 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca880a5be0 con 0x7fca9815bc40 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 -- 172.21.15.87:0/3405642653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 msgr2=0x7fca9806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 --2- 172.21.15.87:0/3405642653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca9806c090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fca980b3f80 tx=0x7fca8801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 -- 172.21.15.87:0/3405642653 shutdown_connections 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 --2- 172.21.15.87:0/3405642653 >> v2:172.21.15.169:3300/0 conn(0x7fca980cf930 0x7fca980cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 --2- 172.21.15.87:0/3405642653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 0x7fca980b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 --2- 172.21.15.87:0/3405642653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca9806c090 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 -- 172.21.15.87:0/3405642653 >> 172.21.15.87:0/3405642653 conn(0x7fca98066b70 msgr2=0x7fca98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 -- 172.21.15.87:0/3405642653 shutdown_connections 2024-04-25T10:16:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 -- 172.21.15.87:0/3405642653 wait complete. 2024-04-25T10:16:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 Processor -- start 2024-04-25T10:16:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.154+0000 7fca9fa00640 1 -- start start 2024-04-25T10:16:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 0x7fca9813ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca980cf930 0x7fca9815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca981596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca98160230 con 0x7fca9815bc40 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca981600b0 con 0x7fca980cf930 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca981603b0 con 0x7fca980cda10 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca981596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca981596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41848/0 (socket says 172.21.15.87:41848) 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 -- 172.21.15.87:0/1267019861 learned_addr learned my addr 172.21.15.87:0/1267019861 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9d000640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 0x7fca9813ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca97e00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca980cf930 0x7fca9815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 -- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 msgr2=0x7fca9813ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 0x7fca9813ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 -- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca980cf930 msgr2=0x7fca9815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca980cf930 0x7fca9815c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 -- 172.21.15.87:0/1267019861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9815e4b0 con 0x7fca9815bc40 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca97e00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca980cf930 0x7fca9815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9da00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca981596c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b390 tx=0x7fca9000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca94c00640 1 -- 172.21.15.87:0/1267019861 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90015020 con 0x7fca9815bc40 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca94c00640 1 -- 172.21.15.87:0/1267019861 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90011830 con 0x7fca9815bc40 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca94c00640 1 -- 172.21.15.87:0/1267019861 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011c20 con 0x7fca9815bc40 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9815e790 con 0x7fca9815bc40 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9815f270 con 0x7fca9815bc40 2024-04-25T10:16:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9804ef30 con 0x7fca9815bc40 2024-04-25T10:16:37.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.158+0000 7fca94c00640 1 -- 172.21.15.87:0/1267019861 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca90018020 con 0x7fca9815bc40 2024-04-25T10:16:37.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.162+0000 7fca94c00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca78066f60 0x7fca78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.162+0000 7fca94c00640 1 -- 172.21.15.87:0/1267019861 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fca90084f40 con 0x7fca9815bc40 2024-04-25T10:16:37.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.162+0000 7fca9d000640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca78066f60 0x7fca78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.162+0000 7fca9d000640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca78066f60 0x7fca78069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fca880ae000 tx=0x7fca880a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.166+0000 7fca94c00640 1 -- 172.21.15.87:0/1267019861 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca9001f020 con 0x7fca9815bc40 2024-04-25T10:16:37.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca980af1c0 con 0x7fca9815bc40 2024-04-25T10:16:37.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca94c00640 1 -- 172.21.15.87:0/1267019861 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fca90055ef0 con 0x7fca9815bc40 2024-04-25T10:16:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:37.360 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:02.576203+0000","modified":"2024-04-25T10:16:17.322234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6837/427681613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":427681613},{"type":"v1","addr":"172.21.15.87:6837","nonce":427681613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:16:37.360 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 44 2024-04-25T10:16:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca78066f60 msgr2=0x7fca78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca78066f60 0x7fca78069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fca880ae000 tx=0x7fca880a7070 comp rx=0 tx=0).stop 2024-04-25T10:16:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 msgr2=0x7fca981596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca981596c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b390 tx=0x7fca9000b850 comp rx=0 tx=0).stop 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 shutdown_connections 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca78066f60 0x7fca78069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9815bc40 0x7fca981596c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca980cf930 0x7fca9815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 --2- 172.21.15.87:0/1267019861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca980cda10 0x7fca9813ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 >> 172.21.15.87:0/1267019861 conn(0x7fca98066b70 msgr2=0x7fca98153f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 shutdown_connections 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.354+0000 7fca9fa00640 1 -- 172.21.15.87:0/1267019861 wait complete. 2024-04-25T10:16:37.381 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:37.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dda400640 1 Processor -- start 2024-04-25T10:16:37.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dda400640 1 -- start start 2024-04-25T10:16:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dda400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7dd415bc40 0x7f7dd406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dda400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7dd40cda10 0x7f7dd40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dda400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd40b2770 con 0x7f7dd40cf930 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dda400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd406c670 con 0x7f7dd40cda10 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dda400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd406c950 con 0x7f7dd415bc40 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7dd40cda10 0x7f7dd40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7dd40cf930 0x7f7dd415f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37984/0 (socket says 172.21.15.87:37984) 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd3e00640 1 -- 172.21.15.87:0/1467017205 learned_addr learned my addr 172.21.15.87:0/1467017205 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd3400640 1 --2- 172.21.15.87:0/1467017205 >> v2:172.21.15.169:3300/0 conn(0x7f7dd415bc40 0x7f7dd406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd2a00640 1 -- 172.21.15.87:0/1467017205 >> v2:172.21.15.169:3300/0 conn(0x7f7dd415bc40 msgr2=0x7f7dd406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd2a00640 1 --2- 172.21.15.87:0/1467017205 >> v2:172.21.15.169:3300/0 conn(0x7f7dd415bc40 0x7f7dd406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd2a00640 1 -- 172.21.15.87:0/1467017205 >> v1:172.21.15.87:6789/0 conn(0x7f7dd40cf930 legacy=0x7f7dd415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd2a00640 1 -- 172.21.15.87:0/1467017205 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd406cc30 con 0x7f7dd40cda10 2024-04-25T10:16:37.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.510+0000 7f7dd2000640 1 -- 172.21.15.87:0/1467017205 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411220136 0 0) 0x7f7dd40b2770 con 0x7f7dd40cf930 2024-04-25T10:16:37.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dd2a00640 1 --2- 172.21.15.87:0/1467017205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b4610 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40d0800 tx=0x7f7dc001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d37d2cfc5880e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dd2000640 1 -- 172.21.15.87:0/1467017205 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00b1070 con 0x7f7dd40cda10 2024-04-25T10:16:37.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dd2000640 1 -- 172.21.15.87:0/1467017205 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc00ac5f0 con 0x7f7dd40cda10 2024-04-25T10:16:37.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dd2000640 1 -- 172.21.15.87:0/1467017205 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00ac9c0 con 0x7f7dd40cda10 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 -- 172.21.15.87:0/1467017205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 msgr2=0x7f7dd40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 --2- 172.21.15.87:0/1467017205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b4610 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40d0800 tx=0x7f7dc001c480 comp rx=0 tx=0).stop 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 -- 172.21.15.87:0/1467017205 shutdown_connections 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 --2- 172.21.15.87:0/1467017205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b4610 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 --2- 172.21.15.87:0/1467017205 >> v2:172.21.15.169:3300/0 conn(0x7f7dd415bc40 0x7f7dd406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 -- 172.21.15.87:0/1467017205 >> 172.21.15.87:0/1467017205 conn(0x7f7dd4066b70 msgr2=0x7f7dd4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 -- 172.21.15.87:0/1467017205 shutdown_connections 2024-04-25T10:16:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 -- 172.21.15.87:0/1467017205 wait complete. 2024-04-25T10:16:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.514+0000 7f7dda400640 1 Processor -- start 2024-04-25T10:16:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 -- start start 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dd40cf930 0x7f7dd4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dd415bc40 0x7f7dd4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd4168b50 con 0x7f7dd40cda10 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd41689d0 con 0x7f7dd40cf930 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd4168cd0 con 0x7f7dd415bc40 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45532/0 (socket says 172.21.15.87:45532) 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 -- 172.21.15.87:0/1203546174 learned_addr learned my addr 172.21.15.87:0/1203546174 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 -- 172.21.15.87:0/1203546174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dd415bc40 msgr2=0x7f7dd4153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dd415bc40 0x7f7dd4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 -- 172.21.15.87:0/1203546174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dd40cf930 msgr2=0x7f7dd4155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dd40cf930 0x7f7dd4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 -- 172.21.15.87:0/1203546174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd4157320 con 0x7f7dd40cda10 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dd3400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b3160 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d800 tx=0x7f7dc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dcb400640 1 -- 172.21.15.87:0/1203546174 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400b6c0 con 0x7f7dd40cda10 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dcb400640 1 -- 172.21.15.87:0/1203546174 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc4019070 con 0x7f7dd40cda10 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dcb400640 1 -- 172.21.15.87:0/1203546174 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400ba60 con 0x7f7dd40cda10 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd41566a0 con 0x7f7dd40cda10 2024-04-25T10:16:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd4156c60 con 0x7f7dd40cda10 2024-04-25T10:16:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.518+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd404ef30 con 0x7f7dd40cda10 2024-04-25T10:16:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.522+0000 7f7dcb400640 1 -- 172.21.15.87:0/1203546174 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dc400bc00 con 0x7f7dd40cda10 2024-04-25T10:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.522+0000 7f7dcb400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7db4066f60 0x7f7db4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.522+0000 7f7dcb400640 1 -- 172.21.15.87:0/1203546174 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7dc40855d0 con 0x7f7dd40cda10 2024-04-25T10:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.522+0000 7f7dd2a00640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7db4066f60 0x7f7db4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.522+0000 7f7dd2a00640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7db4066f60 0x7f7db4069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00ae000 tx=0x7f7dc0004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.526+0000 7f7dcb400640 1 -- 172.21.15.87:0/1203546174 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dc40564e0 con 0x7f7dd40cda10 2024-04-25T10:16:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.678+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dd40af1c0 con 0x7f7dd40cda10 2024-04-25T10:16:37.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dcb400640 1 -- 172.21.15.87:0/1203546174 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f7dc4083e20 con 0x7f7dd40cda10 2024-04-25T10:16:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:37.688 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":61,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:14.297721+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:00.962606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:01.567628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/3387238920":"2024-04-26T10:16:14.297697+0000","172.21.15.87:6835/3387238920":"2024-04-26T10:16:14.297697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7db4066f60 msgr2=0x7f7db4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7db4066f60 0x7f7db4069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00ae000 tx=0x7f7dc0004740 comp rx=0 tx=0).stop 2024-04-25T10:16:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 msgr2=0x7f7dd40b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b3160 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d800 tx=0x7f7dc400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 shutdown_connections 2024-04-25T10:16:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7db4066f60 0x7f7db4069420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dd415bc40 0x7f7dd4153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dd40cf930 0x7f7dd4155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 --2- 172.21.15.87:0/1203546174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd40cda10 0x7f7dd40b3160 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 >> 172.21.15.87:0/1203546174 conn(0x7f7dd4066b70 msgr2=0x7f7dd4066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 shutdown_connections 2024-04-25T10:16:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.682+0000 7f7dda400640 1 -- 172.21.15.87:0/1203546174 wait complete. 2024-04-25T10:16:37.708 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:16:37.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.838+0000 7f9d72200640 1 Processor -- start 2024-04-25T10:16:37.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.838+0000 7f9d72200640 1 -- start start 2024-04-25T10:16:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.838+0000 7f9d72200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d6c0d0040 0x7f9d6c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.838+0000 7f9d72200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.838+0000 7f9d72200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c071050 con 0x7f9d6c1550b0 2024-04-25T10:16:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d72200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c15d040 con 0x7f9d6c0d0040 2024-04-25T10:16:37.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d72200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0712f0 con 0x7f9d6c15b740 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d6c0d0040 0x7f9d6c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6b400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f9d6c1550b0 0x7f9d6c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:33556/0 (socket says 172.21.15.87:33556) 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6b400640 1 -- 172.21.15.87:0/2510051427 learned_addr learned my addr 172.21.15.87:0/2510051427 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6be00640 1 -- 172.21.15.87:0/2510051427 >> v2:172.21.15.169:3300/0 conn(0x7f9d6c0d0040 msgr2=0x7f9d6c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6be00640 1 --2- 172.21.15.87:0/2510051427 >> v2:172.21.15.169:3300/0 conn(0x7f9d6c0d0040 0x7f9d6c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6be00640 1 -- 172.21.15.87:0/2510051427 >> v1:172.21.15.169:6789/0 conn(0x7f9d6c1550b0 legacy=0x7f9d6c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6a000640 1 -- 172.21.15.87:0/2510051427 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112068140 0 0) 0x7f9d6c071050 con 0x7f9d6c1550b0 2024-04-25T10:16:37.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.842+0000 7f9d6be00640 1 -- 172.21.15.87:0/2510051427 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0715d0 con 0x7f9d6c15b740 2024-04-25T10:16:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d6be00640 1 --2- 172.21.15.87:0/2510051427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c0b3260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400bfa0 tx=0x7f9d6401eb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8ed5e558bea3c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d6a000640 1 -- 172.21.15.87:0/2510051427 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400b2d0 con 0x7f9d6c15b740 2024-04-25T10:16:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d6a000640 1 -- 172.21.15.87:0/2510051427 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d640a7a10 con 0x7f9d6c15b740 2024-04-25T10:16:37.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d6a000640 1 -- 172.21.15.87:0/2510051427 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d640a7de0 con 0x7f9d6c15b740 2024-04-25T10:16:37.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- 172.21.15.87:0/2510051427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 msgr2=0x7f9d6c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 --2- 172.21.15.87:0/2510051427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c0b3260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400bfa0 tx=0x7f9d6401eb40 comp rx=0 tx=0).stop 2024-04-25T10:16:37.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- 172.21.15.87:0/2510051427 shutdown_connections 2024-04-25T10:16:37.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 --2- 172.21.15.87:0/2510051427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c0b3260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 --2- 172.21.15.87:0/2510051427 >> v2:172.21.15.169:3300/0 conn(0x7f9d6c0d0040 0x7f9d6c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- 172.21.15.87:0/2510051427 >> 172.21.15.87:0/2510051427 conn(0x7f9d6c066b70 msgr2=0x7f9d6c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- 172.21.15.87:0/2510051427 shutdown_connections 2024-04-25T10:16:37.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- 172.21.15.87:0/2510051427 wait complete. 2024-04-25T10:16:37.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 Processor -- start 2024-04-25T10:16:37.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- start start 2024-04-25T10:16:37.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d6c0d0040 0x7f9d6c14b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c1550b0 0x7f9d6c159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c15bd50 con 0x7f9d6c0d0040 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c15bbd0 con 0x7f9d6c1550b0 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.846+0000 7f9d72200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c15c030 con 0x7f9d6c15b740 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d6c0d0040 0x7f9d6c14b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d6c0d0040 0x7f9d6c14b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45546/0 (socket says 172.21.15.87:45546) 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 -- 172.21.15.87:0/4155811792 learned_addr learned my addr 172.21.15.87:0/4155811792 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6be00640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6aa00640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c1550b0 0x7f9d6c159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 -- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 msgr2=0x7f9d6c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c1597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 -- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c1550b0 msgr2=0x7f9d6c159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c1550b0 0x7f9d6c159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 -- 172.21.15.87:0/4155811792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c1556e0 con 0x7f9d6c0d0040 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6be00640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d6b400640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d6c0d0040 0x7f9d6c14b460 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9d540011f0 tx=0x7f9d5400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d63400640 1 -- 172.21.15.87:0/4155811792 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54018070 con 0x7f9d6c0d0040 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d63400640 1 -- 172.21.15.87:0/4155811792 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d540074e0 con 0x7f9d6c0d0040 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d63400640 1 -- 172.21.15.87:0/4155811792 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54004a10 con 0x7f9d6c0d0040 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c150910 con 0x7f9d6c0d0040 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c150ed0 con 0x7f9d6c0d0040 2024-04-25T10:16:37.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c04ef30 con 0x7f9d6c0d0040 2024-04-25T10:16:37.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.850+0000 7f9d63400640 1 -- 172.21.15.87:0/4155811792 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d54004bb0 con 0x7f9d6c0d0040 2024-04-25T10:16:37.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.854+0000 7f9d63400640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d48066f90 0x7f9d48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.854+0000 7f9d63400640 1 -- 172.21.15.87:0/4155811792 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9d54084330 con 0x7f9d6c0d0040 2024-04-25T10:16:37.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.854+0000 7f9d6aa00640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d48066f90 0x7f9d48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.854+0000 7f9d6aa00640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d48066f90 0x7f9d48069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c1fa240 tx=0x7f9d5c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:37.858+0000 7f9d63400640 1 -- 172.21.15.87:0/4155811792 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d54054bf0 con 0x7f9d6c0d0040 2024-04-25T10:16:38.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.042+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9d6c0b3060 con 0x7f9d6c0d0040 2024-04-25T10:16:38.048 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:16:38.042+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:16:38.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.046+0000 7f9d63400640 1 -- 172.21.15.87:0/4155811792 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9d54058c90 con 0x7f9d6c0d0040 2024-04-25T10:16:38.052 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d48066f90 msgr2=0x7f9d48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d48066f90 0x7f9d48069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c1fa240 tx=0x7f9d5c009190 comp rx=0 tx=0).stop 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d6c0d0040 msgr2=0x7f9d6c14b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d6c0d0040 0x7f9d6c14b460 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9d540011f0 tx=0x7f9d5400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 shutdown_connections 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d48066f90 0x7f9d48069450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d6c15b740 0x7f9d6c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c1550b0 0x7f9d6c159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 --2- 172.21.15.87:0/4155811792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d6c0d0040 0x7f9d6c14b460 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 >> 172.21.15.87:0/4155811792 conn(0x7f9d6c066b70 msgr2=0x7f9d6c0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 shutdown_connections 2024-04-25T10:16:38.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.050+0000 7f9d72200640 1 -- 172.21.15.87:0/4155811792 wait complete. 2024-04-25T10:16:38.063 INFO:tasks.ceph.mds.c.smithi087.stderr: -2382> 2024-04-25T10:16:20.314+0000 7f33d9a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:38.077 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:16:38.109 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:16:38.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52b4e00640 1 Processor -- start 2024-04-25T10:16:38.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52b4e00640 1 -- start start 2024-04-25T10:16:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52b4e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f52b0159540 0x7f52b00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52b4e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52b00721f0 0x7f52b0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52b4e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b0159920 con 0x7f52b00aebd0 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52b4e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b015c2f0 con 0x7f52b0159540 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52b4e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b015c5d0 con 0x7f52b00721f0 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52ade00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f52b0159540 0x7f52b00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52ade00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f52b0159540 0x7f52b00ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45568/0 (socket says 172.21.15.87:45568) 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52ade00640 1 -- 172.21.15.87:0/3200259823 learned_addr learned my addr 172.21.15.87:0/3200259823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52ade00640 1 -- 172.21.15.87:0/3200259823 >> v2:172.21.15.169:3300/0 conn(0x7f52b00721f0 msgr2=0x7f52b0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52ade00640 1 --2- 172.21.15.87:0/3200259823 >> v2:172.21.15.169:3300/0 conn(0x7f52b00721f0 0x7f52b0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52ade00640 1 -- 172.21.15.87:0/3200259823 >> v1:172.21.15.169:6789/0 conn(0x7f52b00aebd0 legacy=0x7f52b0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.210+0000 7f52ade00640 1 -- 172.21.15.87:0/3200259823 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b015c8b0 con 0x7f52b0159540 2024-04-25T10:16:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52ade00640 1 --2- 172.21.15.87:0/3200259823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b00ae690 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f52b00cd200 tx=0x7f52a001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba90f9b71db3e21b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52ad400640 1 -- 172.21.15.87:0/3200259823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00ab5f0 con 0x7f52b0159540 2024-04-25T10:16:38.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52ad400640 1 -- 172.21.15.87:0/3200259823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a00b0070 con 0x7f52b0159540 2024-04-25T10:16:38.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52ad400640 1 -- 172.21.15.87:0/3200259823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00ab9b0 con 0x7f52b0159540 2024-04-25T10:16:38.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 -- 172.21.15.87:0/3200259823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 msgr2=0x7f52b00ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3200259823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b00ae690 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f52b00cd200 tx=0x7f52a001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 -- 172.21.15.87:0/3200259823 shutdown_connections 2024-04-25T10:16:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3200259823 >> v2:172.21.15.169:3300/0 conn(0x7f52b00721f0 0x7f52b0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3200259823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b00ae690 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 -- 172.21.15.87:0/3200259823 >> 172.21.15.87:0/3200259823 conn(0x7f52b00669a0 msgr2=0x7f52b0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 -- 172.21.15.87:0/3200259823 shutdown_connections 2024-04-25T10:16:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.214+0000 7f52b4e00640 1 -- 172.21.15.87:0/3200259823 wait complete. 2024-04-25T10:16:38.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 Processor -- start 2024-04-25T10:16:38.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 -- start start 2024-04-25T10:16:38.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b00721f0 0x7f52b0158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b00aebd0 0x7f52b0150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b013c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0071220 con 0x7f52b0159540 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00710a0 con 0x7f52b00721f0 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00713a0 con 0x7f52b00aebd0 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b013c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b013c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45574/0 (socket says 172.21.15.87:45574) 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 -- 172.21.15.87:0/3432690650 learned_addr learned my addr 172.21.15.87:0/3432690650 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52a5400640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b00aebd0 0x7f52b0150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ade00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b00721f0 0x7f52b0158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 -- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b00aebd0 msgr2=0x7f52b0150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b00aebd0 0x7f52b0150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 -- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b00721f0 msgr2=0x7f52b0158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b00721f0 0x7f52b0158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 -- 172.21.15.87:0/3432690650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0150680 con 0x7f52b0159540 2024-04-25T10:16:38.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52a5400640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b00aebd0 0x7f52b0150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:38.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52ae800640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b013c070 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b970 tx=0x7f52a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52a6a00640 1 -- 172.21.15.87:0/3432690650 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8019070 con 0x7f52b0159540 2024-04-25T10:16:38.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52a6a00640 1 -- 172.21.15.87:0/3432690650 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a800cb60 con 0x7f52b0159540 2024-04-25T10:16:38.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b0150920 con 0x7f52b0159540 2024-04-25T10:16:38.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52a6a00640 1 -- 172.21.15.87:0/3432690650 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8014440 con 0x7f52b0159540 2024-04-25T10:16:38.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b0150ee0 con 0x7f52b0159540 2024-04-25T10:16:38.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.218+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b004ef30 con 0x7f52b0159540 2024-04-25T10:16:38.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.222+0000 7f52a6a00640 1 -- 172.21.15.87:0/3432690650 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52a801c070 con 0x7f52b0159540 2024-04-25T10:16:38.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.222+0000 7f52a6a00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f528c066f90 0x7f528c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.222+0000 7f52a6a00640 1 -- 172.21.15.87:0/3432690650 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f52a8084cc0 con 0x7f52b0159540 2024-04-25T10:16:38.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.222+0000 7f52ade00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f528c066f90 0x7f528c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.222+0000 7f52ade00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f528c066f90 0x7f528c069450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f52a00af000 tx=0x7f52a00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.226+0000 7f52a6a00640 1 -- 172.21.15.87:0/3432690650 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52a8055660 con 0x7f52b0159540 2024-04-25T10:16:38.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:38.414+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f52b006de20 con 0x7f52b0159540 2024-04-25T10:16:39.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.050+0000 7f52a6a00640 1 -- 172.21.15.87:0/3432690650 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f52a8059700 con 0x7f52b0159540 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f528c066f90 msgr2=0x7f528c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f528c066f90 0x7f528c069450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f52a00af000 tx=0x7f52a00a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 msgr2=0x7f52b013c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b013c070 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b970 tx=0x7f52a800be30 comp rx=0 tx=0).stop 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 shutdown_connections 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f528c066f90 0x7f528c069450 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52b0159540 0x7f52b013c070 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b00aebd0 0x7f52b0150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 --2- 172.21.15.87:0/3432690650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b00721f0 0x7f52b0158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 >> 172.21.15.87:0/3432690650 conn(0x7f52b00669a0 msgr2=0x7f52b015b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 shutdown_connections 2024-04-25T10:16:39.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.054+0000 7f52b4e00640 1 -- 172.21.15.87:0/3432690650 wait complete. 2024-04-25T10:16:39.083 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:16:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf30400640 1 Processor -- start 2024-04-25T10:16:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf30400640 1 -- start start 2024-04-25T10:16:39.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf30400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 0x7fcf280b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf30400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2806c0e0 con 0x7fcf2815bd10 2024-04-25T10:16:39.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf30400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2815c6a0 con 0x7fcf280cf9e0 2024-04-25T10:16:39.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf30400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2815c980 con 0x7fcf2815c0f0 2024-04-25T10:16:39.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf2da00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcf2815bd10 0x7fcf280b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37996/0 (socket says 172.21.15.87:37996) 2024-04-25T10:16:39.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf2da00640 1 -- 172.21.15.87:0/792310199 learned_addr learned my addr 172.21.15.87:0/792310199 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:39.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf2d000640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 0x7fcf280b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606246308 0 0) 0x7fcf2806c0e0 con 0x7fcf2815bd10 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf10003530 con 0x7fcf2815bd10 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.258+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056676124 0 0) 0x7fcf2815c6a0 con 0x7fcf280cf9e0 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf2806c0e0 con 0x7fcf280cf9e0 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4133354223 0 0) 0x7fcf10003530 con 0x7fcf2815bd10 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 msgr2=0x7fcf280b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 0x7fcf280b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 >> v1:172.21.15.169:6789/0 conn(0x7fcf280cf9e0 legacy=0x7fcf2806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2815cc60 con 0x7fcf2815bd10 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fcf1c001680 con 0x7fcf2815bd10 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 0x7fcf280b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 >> v1:172.21.15.87:6789/0 conn(0x7fcf2815bd10 legacy=0x7fcf280b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf10003b80 0x7fcf10003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 0x7fcf100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2815c980 con 0x7fcf10004810 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000ba00 con 0x7fcf10003b80 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000bc00 con 0x7fcf2815c0f0 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2da00640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf10003b80 0x7fcf10003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 -- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 msgr2=0x7fcf100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2815c0f0 0x7fcf100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 -- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf10003b80 msgr2=0x7fcf10003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf10003b80 0x7fcf10003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 -- 172.21.15.87:0/792310199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10003530 con 0x7fcf10004810 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2da00640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf10003b80 0x7fcf10003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf2d000640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009c80 tx=0x7fcf1401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27269651101391df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140b2070 con 0x7fcf10004810 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf140a58b0 con 0x7fcf10004810 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.262+0000 7fcf27e00640 1 -- 172.21.15.87:0/792310199 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf14009880 con 0x7fcf10004810 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- 172.21.15.87:0/792310199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 msgr2=0x7fcf10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 --2- 172.21.15.87:0/792310199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009c80 tx=0x7fcf1401cbf0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2e400640 1 -- 172.21.15.87:0/792310199 reap_dead start 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- 172.21.15.87:0/792310199 shutdown_connections 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- 172.21.15.87:0/792310199 >> 172.21.15.87:0/792310199 conn(0x7fcf28066b70 msgr2=0x7fcf28066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- 172.21.15.87:0/792310199 shutdown_connections 2024-04-25T10:16:39.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- 172.21.15.87:0/792310199 wait complete. 2024-04-25T10:16:39.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 Processor -- start 2024-04-25T10:16:39.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- start start 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf2815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2815eba0 0x7fcf281e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf281e7e20 0x7fcf2813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2806ce70 con 0x7fcf10004810 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2806ccf0 con 0x7fcf2815eba0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf30400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2806cff0 con 0x7fcf281e7e20 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf2815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf2815e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45594/0 (socket says 172.21.15.87:45594) 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 -- 172.21.15.87:0/1465862140 learned_addr learned my addr 172.21.15.87:0/1465862140 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2d000640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2815eba0 0x7fcf281e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2e400640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf281e7e20 0x7fcf2813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 -- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf281e7e20 msgr2=0x7fcf2813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf281e7e20 0x7fcf2813cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 -- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2815eba0 msgr2=0x7fcf281e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2815eba0 0x7fcf281e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.266+0000 7fcf2da00640 1 -- 172.21.15.87:0/1465862140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28154370 con 0x7fcf10004810 2024-04-25T10:16:39.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf2da00640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf2815e660 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00d810 tx=0x7fcf1c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf25600640 1 -- 172.21.15.87:0/1465862140 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c019070 con 0x7fcf10004810 2024-04-25T10:16:39.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf25600640 1 -- 172.21.15.87:0/1465862140 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0074e0 con 0x7fcf10004810 2024-04-25T10:16:39.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf25600640 1 -- 172.21.15.87:0/1465862140 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0049f0 con 0x7fcf10004810 2024-04-25T10:16:39.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf28151410 con 0x7fcf10004810 2024-04-25T10:16:39.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf28153b70 con 0x7fcf10004810 2024-04-25T10:16:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2804ef30 con 0x7fcf10004810 2024-04-25T10:16:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf25600640 1 -- 172.21.15.87:0/1465862140 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf1c004b90 con 0x7fcf10004810 2024-04-25T10:16:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf25600640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf08066f60 0x7fcf08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf25600640 1 -- 172.21.15.87:0/1465862140 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fcf1c0849f0 con 0x7fcf10004810 2024-04-25T10:16:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.270+0000 7fcf2d000640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf08066f60 0x7fcf08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.274+0000 7fcf2d000640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf08066f60 0x7fcf08069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcf140af000 tx=0x7fcf140046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.274+0000 7fcf25600640 1 -- 172.21.15.87:0/1465862140 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf1c055300 con 0x7fcf10004810 2024-04-25T10:16:39.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:39.430+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcf2806b400 con 0x7fcf10004810 2024-04-25T10:16:40.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf25600640 1 -- 172.21.15.87:0/1465862140 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcf1c0593a0 con 0x7fcf10004810 2024-04-25T10:16:40.065 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:16:40.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf08066f60 msgr2=0x7fcf08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf08066f60 0x7fcf08069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcf140af000 tx=0x7fcf140046d0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 msgr2=0x7fcf2815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf2815e660 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00d810 tx=0x7fcf1c00dcd0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 shutdown_connections 2024-04-25T10:16:40.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf08066f60 0x7fcf08069420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf281e7e20 0x7fcf2813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2815eba0 0x7fcf281e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 --2- 172.21.15.87:0/1465862140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf10004810 0x7fcf2815e660 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 >> 172.21.15.87:0/1465862140 conn(0x7fcf28066b70 msgr2=0x7fcf28068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.058+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 shutdown_connections 2024-04-25T10:16:40.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.062+0000 7fcf30400640 1 -- 172.21.15.87:0/1465862140 wait complete. 2024-04-25T10:16:40.091 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:16:40.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c69600640 1 Processor -- start 2024-04-25T10:16:40.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c69600640 1 -- start start 2024-04-25T10:16:40.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c69600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c6406b0d0 0x7f5c640cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c69600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406bbe0 0x7f5c640cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c69600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c640cf290 con 0x7f5c6406b800 2024-04-25T10:16:40.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c69600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c640cde90 con 0x7f5c6406b0d0 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c69600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640cfdb0 con 0x7f5c6406bbe0 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c62000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c6406b0d0 0x7f5c640cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c62000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c6406b0d0 0x7f5c640cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45598/0 (socket says 172.21.15.87:45598) 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c62000640 1 -- 172.21.15.87:0/1429304753 learned_addr learned my addr 172.21.15.87:0/1429304753 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c63400640 1 --2- 172.21.15.87:0/1429304753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406bbe0 0x7f5c640cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c62000640 1 -- 172.21.15.87:0/1429304753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406bbe0 msgr2=0x7f5c640cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c62000640 1 --2- 172.21.15.87:0/1429304753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406bbe0 0x7f5c640cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c62000640 1 -- 172.21.15.87:0/1429304753 >> v1:172.21.15.169:6789/0 conn(0x7f5c6406b800 legacy=0x7f5c640ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.226+0000 7f5c62000640 1 -- 172.21.15.87:0/1429304753 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640d0090 con 0x7f5c6406b0d0 2024-04-25T10:16:40.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c62000640 1 --2- 172.21.15.87:0/1429304753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640cd410 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009870 tx=0x7f5c5001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4d9c63c377c943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c61600640 1 -- 172.21.15.87:0/1429304753 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500b5070 con 0x7f5c6406b0d0 2024-04-25T10:16:40.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c61600640 1 -- 172.21.15.87:0/1429304753 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c500a89d0 con 0x7f5c6406b0d0 2024-04-25T10:16:40.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c61600640 1 -- 172.21.15.87:0/1429304753 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500a8da0 con 0x7f5c6406b0d0 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 -- 172.21.15.87:0/1429304753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 msgr2=0x7f5c640cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 --2- 172.21.15.87:0/1429304753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640cd410 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009870 tx=0x7f5c5001ed10 comp rx=0 tx=0).stop 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 -- 172.21.15.87:0/1429304753 shutdown_connections 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 --2- 172.21.15.87:0/1429304753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406bbe0 0x7f5c640cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 --2- 172.21.15.87:0/1429304753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640cd410 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 -- 172.21.15.87:0/1429304753 >> 172.21.15.87:0/1429304753 conn(0x7f5c640d19d0 msgr2=0x7f5c640d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 -- 172.21.15.87:0/1429304753 shutdown_connections 2024-04-25T10:16:40.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.230+0000 7f5c69600640 1 -- 172.21.15.87:0/1429304753 wait complete. 2024-04-25T10:16:40.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 Processor -- start 2024-04-25T10:16:40.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 -- start start 2024-04-25T10:16:40.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406b800 0x7f5c640cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6406bbe0 0x7f5c64154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640d0930 con 0x7f5c6406b0d0 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640d0650 con 0x7f5c6406bbe0 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640d0e50 con 0x7f5c6406b800 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45604/0 (socket says 172.21.15.87:45604) 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 -- 172.21.15.87:0/4067863902 learned_addr learned my addr 172.21.15.87:0/4067863902 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62000640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406b800 0x7f5c640cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c63400640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6406bbe0 0x7f5c64154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 -- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406b800 msgr2=0x7f5c640cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406b800 0x7f5c640cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 -- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6406bbe0 msgr2=0x7f5c64154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6406bbe0 0x7f5c64154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 -- 172.21.15.87:0/4067863902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640c92f0 con 0x7f5c6406b0d0 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c63400640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6406bbe0 0x7f5c64154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c62a00640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640c8aa0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c450 tx=0x7f5c5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c5aa00640 1 -- 172.21.15.87:0/4067863902 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54012830 con 0x7f5c6406b0d0 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c5aa00640 1 -- 172.21.15.87:0/4067863902 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54017070 con 0x7f5c6406b0d0 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c5aa00640 1 -- 172.21.15.87:0/4067863902 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54012c30 con 0x7f5c6406b0d0 2024-04-25T10:16:40.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c640c95d0 con 0x7f5c6406b0d0 2024-04-25T10:16:40.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.234+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c640c9b90 con 0x7f5c6406b0d0 2024-04-25T10:16:40.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.238+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6404ef30 con 0x7f5c6406b0d0 2024-04-25T10:16:40.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.238+0000 7f5c5aa00640 1 -- 172.21.15.87:0/4067863902 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c5401c460 con 0x7f5c6406b0d0 2024-04-25T10:16:40.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.238+0000 7f5c5aa00640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c44066f60 0x7f5c44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.238+0000 7f5c5aa00640 1 -- 172.21.15.87:0/4067863902 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f5c54084ca0 con 0x7f5c6406b0d0 2024-04-25T10:16:40.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.238+0000 7f5c62000640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c44066f60 0x7f5c44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.238+0000 7f5c62000640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c44066f60 0x7f5c44069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c500b2000 tx=0x7f5c50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.242+0000 7f5c5aa00640 1 -- 172.21.15.87:0/4067863902 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c54055d10 con 0x7f5c6406b0d0 2024-04-25T10:16:40.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:40.402+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c640b7950 con 0x7f5c6406b0d0 2024-04-25T10:16:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.066+0000 7f5c5aa00640 1 -- 172.21.15.87:0/4067863902 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5c54055610 con 0x7f5c6406b0d0 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c44066f60 msgr2=0x7f5c44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c44066f60 0x7f5c44069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c500b2000 tx=0x7f5c50009040 comp rx=0 tx=0).stop 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 msgr2=0x7f5c640c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640c8aa0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c450 tx=0x7f5c5400c910 comp rx=0 tx=0).stop 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 shutdown_connections 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c44066f60 0x7f5c44069420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6406bbe0 0x7f5c64154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c6406b800 0x7f5c640cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 --2- 172.21.15.87:0/4067863902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c6406b0d0 0x7f5c640c8aa0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 >> 172.21.15.87:0/4067863902 conn(0x7f5c640d19d0 msgr2=0x7f5c640cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 shutdown_connections 2024-04-25T10:16:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.070+0000 7f5c69600640 1 -- 172.21.15.87:0/4067863902 wait complete. 2024-04-25T10:16:41.100 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:16:41.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 Processor -- start 2024-04-25T10:16:41.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 -- start start 2024-04-25T10:16:41.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7818159540 0x7f78180ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78180aebd0 0x7f7818071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f781815f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7818159920 con 0x7f78180aebd0 2024-04-25T10:16:41.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781815c2f0 con 0x7f7818159540 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f7820000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781815c5d0 con 0x7f7818072020 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78180aebd0 0x7f7818071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78180aebd0 0x7f7818071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45608/0 (socket says 172.21.15.87:45608) 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 -- 172.21.15.87:0/3654216313 learned_addr learned my addr 172.21.15.87:0/3654216313 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781d600640 1 --2- 172.21.15.87:0/3654216313 >> v2:172.21.15.169:3300/0 conn(0x7f7818159540 0x7f78180ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781e000640 1 --2- 172.21.15.87:0/3654216313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f781815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 -- 172.21.15.87:0/3654216313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 msgr2=0x7f781815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 --2- 172.21.15.87:0/3654216313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f781815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 -- 172.21.15.87:0/3654216313 >> v2:172.21.15.169:3300/0 conn(0x7f7818159540 msgr2=0x7f78180ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 --2- 172.21.15.87:0/3654216313 >> v2:172.21.15.169:3300/0 conn(0x7f7818159540 0x7f78180ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781cc00640 1 -- 172.21.15.87:0/3654216313 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781815c8b0 con 0x7f78180aebd0 2024-04-25T10:16:41.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.234+0000 7f781d600640 1 --2- 172.21.15.87:0/3654216313 >> v2:172.21.15.169:3300/0 conn(0x7f7818159540 0x7f78180ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:41.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f781cc00640 1 --2- 172.21.15.87:0/3654216313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f7818071ae0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f78180b4c20 tx=0x7f780401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd10506a9fe1211e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7817e00640 1 -- 172.21.15.87:0/3654216313 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040b0070 con 0x7f78180aebd0 2024-04-25T10:16:41.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7817e00640 1 -- 172.21.15.87:0/3654216313 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78040ab600 con 0x7f78180aebd0 2024-04-25T10:16:41.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7817e00640 1 -- 172.21.15.87:0/3654216313 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040ab9d0 con 0x7f78180aebd0 2024-04-25T10:16:41.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 -- 172.21.15.87:0/3654216313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 msgr2=0x7f7818071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 --2- 172.21.15.87:0/3654216313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f7818071ae0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f78180b4c20 tx=0x7f780401c260 comp rx=0 tx=0).stop 2024-04-25T10:16:41.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 -- 172.21.15.87:0/3654216313 shutdown_connections 2024-04-25T10:16:41.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 --2- 172.21.15.87:0/3654216313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f7818071ae0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 --2- 172.21.15.87:0/3654216313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f781815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 --2- 172.21.15.87:0/3654216313 >> v2:172.21.15.169:3300/0 conn(0x7f7818159540 0x7f78180ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 -- 172.21.15.87:0/3654216313 >> 172.21.15.87:0/3654216313 conn(0x7f78180669a0 msgr2=0x7f7818066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 -- 172.21.15.87:0/3654216313 shutdown_connections 2024-04-25T10:16:41.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 -- 172.21.15.87:0/3654216313 wait complete. 2024-04-25T10:16:41.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 Processor -- start 2024-04-25T10:16:41.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.238+0000 7f7820000640 1 -- start start 2024-04-25T10:16:41.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f7818149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f78180cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7818159540 0x7f78180cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7818070350 con 0x7f78180aebd0 2024-04-25T10:16:41.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7818072430 con 0x7f7818159540 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78180704d0 con 0x7f7818072020 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f78180cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f78180cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45620/0 (socket says 172.21.15.87:45620) 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 -- 172.21.15.87:0/3194269236 learned_addr learned my addr 172.21.15.87:0/3194269236 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781d600640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f7818149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781e000640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7818159540 0x7f78180cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 -- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 msgr2=0x7f7818149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f7818149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 -- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7818159540 msgr2=0x7f78180cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7818159540 0x7f78180cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 -- 172.21.15.87:0/3194269236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781815aa20 con 0x7f78180aebd0 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f781cc00640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f78180cd960 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c10 tx=0x7f7804019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7815600640 1 -- 172.21.15.87:0/3194269236 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040b0040 con 0x7f78180aebd0 2024-04-25T10:16:41.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7815600640 1 -- 172.21.15.87:0/3194269236 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78040b9070 con 0x7f78180aebd0 2024-04-25T10:16:41.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7815600640 1 -- 172.21.15.87:0/3194269236 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040b4830 con 0x7f78180aebd0 2024-04-25T10:16:41.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7818159c60 con 0x7f78180aebd0 2024-04-25T10:16:41.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781815a220 con 0x7f78180aebd0 2024-04-25T10:16:41.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781804ef30 con 0x7f78180aebd0 2024-04-25T10:16:41.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.242+0000 7f7815600640 1 -- 172.21.15.87:0/3194269236 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78040ab480 con 0x7f78180aebd0 2024-04-25T10:16:41.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.246+0000 7f7815600640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f8066f60 0x7f77f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.246+0000 7f7815600640 1 -- 172.21.15.87:0/3194269236 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f780411e990 con 0x7f78180aebd0 2024-04-25T10:16:41.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.246+0000 7f781d600640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f8066f60 0x7f77f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.246+0000 7f781d600640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f8066f60 0x7f77f8069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f780c009fd0 tx=0x7f780c009870 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.250+0000 7f7815600640 1 -- 172.21.15.87:0/3194269236 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7804016110 con 0x7f78180aebd0 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.414+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f78180cf470 con 0x7f78180aebd0 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.418+0000 7f7815600640 1 -- 172.21.15.87:0/3194269236 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f78040ef5e0 con 0x7f78180aebd0 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.418+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f8066f60 msgr2=0x7f77f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.418+0000 7f7820000640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f8066f60 0x7f77f8069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f780c009fd0 tx=0x7f780c009870 comp rx=0 tx=0).stop 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.418+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 msgr2=0x7f78180cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.418+0000 7f7820000640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f78180cd960 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c10 tx=0x7f7804019000 comp rx=0 tx=0).stop 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.418+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 shutdown_connections 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.418+0000 7f7820000640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f77f8066f60 0x7f77f8069420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.422+0000 7f7820000640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7818159540 0x7f78180cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.422+0000 7f7820000640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78180aebd0 0x7f78180cd960 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.422+0000 7f7820000640 1 --2- 172.21.15.87:0/3194269236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7818072020 0x7f7818149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.422+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 >> 172.21.15.87:0/3194269236 conn(0x7f78180669a0 msgr2=0x7f78180ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.422+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 shutdown_connections 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.422+0000 7f7820000640 1 -- 172.21.15.87:0/3194269236 wait complete. 2024-04-25T10:16:41.446 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:16:41.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2fc00640 1 Processor -- start 2024-04-25T10:16:41.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2fc00640 1 -- start start 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d2815bc40 0x7f5d2806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d280cf930 0x7f5d280cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2fc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d280b2770 con 0x7f5d280cda10 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2806c670 con 0x7f5d2815bc40 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2806c950 con 0x7f5d280cf930 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d2d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d2815bc40 0x7f5d2806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d1fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5d280cda10 0x7f5d280b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38008/0 (socket says 172.21.15.87:38008) 2024-04-25T10:16:41.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.574+0000 7f5d1fe00640 1 -- 172.21.15.87:0/2480493969 learned_addr learned my addr 172.21.15.87:0/2480493969 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:41.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d2dc00640 1 --2- 172.21.15.87:0/2480493969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d280cf930 0x7f5d280cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d2d200640 1 -- 172.21.15.87:0/2480493969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d280cf930 msgr2=0x7f5d280cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d2d200640 1 --2- 172.21.15.87:0/2480493969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d280cf930 0x7f5d280cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d2d200640 1 -- 172.21.15.87:0/2480493969 >> v1:172.21.15.87:6789/0 conn(0x7f5d280cda10 legacy=0x7f5d280b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d2d200640 1 -- 172.21.15.87:0/2480493969 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2806cc30 con 0x7f5d2815bc40 2024-04-25T10:16:41.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d27e00640 1 -- 172.21.15.87:0/2480493969 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882904232 0 0) 0x7f5d280b2770 con 0x7f5d280cda10 2024-04-25T10:16:41.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d2dc00640 1 --2- 172.21.15.87:0/2480493969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d280cf930 0x7f5d280cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:41.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d2d200640 1 --2- 172.21.15.87:0/2480493969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d2815bc40 0x7f5d2806c090 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5d180097c0 tx=0x7f5d1801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540eed60efca9ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d27e00640 1 -- 172.21.15.87:0/2480493969 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d180b0070 con 0x7f5d2815bc40 2024-04-25T10:16:41.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d27e00640 1 -- 172.21.15.87:0/2480493969 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d180ab600 con 0x7f5d2815bc40 2024-04-25T10:16:41.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.578+0000 7f5d27e00640 1 -- 172.21.15.87:0/2480493969 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d180ab9d0 con 0x7f5d2815bc40 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- 172.21.15.87:0/2480493969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d2815bc40 msgr2=0x7f5d2806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/2480493969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d2815bc40 0x7f5d2806c090 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5d180097c0 tx=0x7f5d1801c410 comp rx=0 tx=0).stop 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- 172.21.15.87:0/2480493969 shutdown_connections 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/2480493969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d280cf930 0x7f5d280cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/2480493969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d2815bc40 0x7f5d2806c090 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- 172.21.15.87:0/2480493969 >> 172.21.15.87:0/2480493969 conn(0x7f5d28066b70 msgr2=0x7f5d28066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- 172.21.15.87:0/2480493969 shutdown_connections 2024-04-25T10:16:41.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- 172.21.15.87:0/2480493969 wait complete. 2024-04-25T10:16:41.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 Processor -- start 2024-04-25T10:16:41.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- start start 2024-04-25T10:16:41.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d280cda10 0x7f5d28152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d280cf930 0x7f5d2815e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d2815bc40 0x7f5d2814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280b3890 con 0x7f5d280cf930 2024-04-25T10:16:41.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280b3710 con 0x7f5d280cda10 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280b3a10 con 0x7f5d2815bc40 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d280cf930 0x7f5d2815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d280cf930 0x7f5d2815e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45638/0 (socket says 172.21.15.87:45638) 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 -- 172.21.15.87:0/199777281 learned_addr learned my addr 172.21.15.87:0/199777281 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2dc00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d2815bc40 0x7f5d2814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d2d200640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d280cda10 0x7f5d28152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 -- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d2815bc40 msgr2=0x7f5d2814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d2815bc40 0x7f5d2814cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 -- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d280cda10 msgr2=0x7f5d28152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d280cda10 0x7f5d28152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 -- 172.21.15.87:0/199777281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2814d670 con 0x7f5d280cf930 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.582+0000 7f5d1fe00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d280cf930 0x7f5d2815e510 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000c480 tx=0x7f5d1000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d25600640 1 -- 172.21.15.87:0/199777281 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10018070 con 0x7f5d280cf930 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d25600640 1 -- 172.21.15.87:0/199777281 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d10007500 con 0x7f5d280cf930 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d25600640 1 -- 172.21.15.87:0/199777281 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10004a30 con 0x7f5d280cf930 2024-04-25T10:16:41.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2815d4b0 con 0x7f5d280cf930 2024-04-25T10:16:41.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2815da30 con 0x7f5d280cf930 2024-04-25T10:16:41.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2804ef30 con 0x7f5d280cf930 2024-04-25T10:16:41.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d25600640 1 -- 172.21.15.87:0/199777281 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d10004bd0 con 0x7f5d280cf930 2024-04-25T10:16:41.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d25600640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d04066f60 0x7f5d04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d25600640 1 -- 172.21.15.87:0/199777281 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5d10083f90 con 0x7f5d280cf930 2024-04-25T10:16:41.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.586+0000 7f5d2d200640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d04066f60 0x7f5d04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.590+0000 7f5d2d200640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d04066f60 0x7f5d04069420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d180ad000 tx=0x7f5d180a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.590+0000 7f5d25600640 1 -- 172.21.15.87:0/199777281 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d10054be0 con 0x7f5d280cf930 2024-04-25T10:16:41.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:41.798+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f5d2806d960 con 0x7f5d280cf930 2024-04-25T10:16:42.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d25600640 1 -- 172.21.15.87:0/199777281 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v235) v1 ==== 143+0+0 (secure 0 0 0) 0x7f5d10058c80 con 0x7f5d280cf930 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d04066f60 msgr2=0x7f5d04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d04066f60 0x7f5d04069420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d180ad000 tx=0x7f5d180a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d280cf930 msgr2=0x7f5d2815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d280cf930 0x7f5d2815e510 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000c480 tx=0x7f5d1000c940 comp rx=0 tx=0).stop 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 shutdown_connections 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d04066f60 0x7f5d04069420 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d2815bc40 0x7f5d2814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d280cf930 0x7f5d2815e510 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 --2- 172.21.15.87:0/199777281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d280cda10 0x7f5d28152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 >> 172.21.15.87:0/199777281 conn(0x7f5d28066b70 msgr2=0x7f5d28159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 shutdown_connections 2024-04-25T10:16:42.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.074+0000 7f5d2fc00640 1 -- 172.21.15.87:0/199777281 wait complete. 2024-04-25T10:16:42.103 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:16:42.103 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T10:16:42.103 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:16:42.104 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:42.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f9c00640 1 Processor -- start 2024-04-25T10:16:42.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f9c00640 1 -- start start 2024-04-25T10:16:42.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f40d0040 0x7f78f415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f9c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78f415b740 0x7f78f40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f9c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f4071050 con 0x7f78f41550b0 2024-04-25T10:16:42.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f9c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f415d040 con 0x7f78f415b740 2024-04-25T10:16:42.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40712f0 con 0x7f78f40d0040 2024-04-25T10:16:42.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78f415b740 0x7f78f40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78f415b740 0x7f78f40b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45648/0 (socket says 172.21.15.87:45648) 2024-04-25T10:16:42.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f3400640 1 -- 172.21.15.87:0/272385772 learned_addr learned my addr 172.21.15.87:0/272385772 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:42.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78ea000640 1 --2- 172.21.15.87:0/272385772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f40d0040 0x7f78f415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f3400640 1 -- 172.21.15.87:0/272385772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f40d0040 msgr2=0x7f78f415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f3400640 1 --2- 172.21.15.87:0/272385772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f40d0040 0x7f78f415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f3400640 1 -- 172.21.15.87:0/272385772 >> v1:172.21.15.87:6789/0 conn(0x7f78f41550b0 legacy=0x7f78f40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78f3400640 1 -- 172.21.15.87:0/272385772 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40715d0 con 0x7f78f415b740 2024-04-25T10:16:42.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.234+0000 7f78ea000640 1 --2- 172.21.15.87:0/272385772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f40d0040 0x7f78f415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:42.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f3400640 1 --2- 172.21.15.87:0/272385772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f415b740 0x7f78f40b3260 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009870 tx=0x7f78ec01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ca07a79c533c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f2000640 1 -- 172.21.15.87:0/272385772 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0a95f0 con 0x7f78f415b740 2024-04-25T10:16:42.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f2000640 1 -- 172.21.15.87:0/272385772 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec0a9790 con 0x7f78f415b740 2024-04-25T10:16:42.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f2000640 1 -- 172.21.15.87:0/272385772 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0a9b60 con 0x7f78f415b740 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 -- 172.21.15.87:0/272385772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f415b740 msgr2=0x7f78f40b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 --2- 172.21.15.87:0/272385772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f415b740 0x7f78f40b3260 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009870 tx=0x7f78ec01c530 comp rx=0 tx=0).stop 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 -- 172.21.15.87:0/272385772 shutdown_connections 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 --2- 172.21.15.87:0/272385772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f415b740 0x7f78f40b3260 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 --2- 172.21.15.87:0/272385772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f40d0040 0x7f78f415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 -- 172.21.15.87:0/272385772 >> 172.21.15.87:0/272385772 conn(0x7f78f4066b70 msgr2=0x7f78f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 -- 172.21.15.87:0/272385772 shutdown_connections 2024-04-25T10:16:42.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.238+0000 7f78f9c00640 1 -- 172.21.15.87:0/272385772 wait complete. 2024-04-25T10:16:42.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 Processor -- start 2024-04-25T10:16:42.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 -- start start 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f40d0040 0x7f78f41fb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f41550b0 0x7f78f414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f415b740 0x7f78f40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f415bd70 con 0x7f78f41550b0 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f415bbf0 con 0x7f78f40d0040 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f415c030 con 0x7f78f415b740 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f41550b0 0x7f78f414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f41550b0 0x7f78f414b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45650/0 (socket says 172.21.15.87:45650) 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 -- 172.21.15.87:0/2701099651 learned_addr learned my addr 172.21.15.87:0/2701099651 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f2a00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f40d0040 0x7f78f41fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f3400640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f415b740 0x7f78f40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 -- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f415b740 msgr2=0x7f78f40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f415b740 0x7f78f40ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 -- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f40d0040 msgr2=0x7f78f41fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f40d0040 0x7f78f41fb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 -- 172.21.15.87:0/2701099651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f414cf80 con 0x7f78f41550b0 2024-04-25T10:16:42.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78ea000640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f41550b0 0x7f78f414b160 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00cf60 tx=0x7f78dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78eb400640 1 -- 172.21.15.87:0/2701099651 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc01a070 con 0x7f78f41550b0 2024-04-25T10:16:42.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78eb400640 1 -- 172.21.15.87:0/2701099651 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc00d840 con 0x7f78f41550b0 2024-04-25T10:16:42.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78eb400640 1 -- 172.21.15.87:0/2701099651 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00dc10 con 0x7f78f41550b0 2024-04-25T10:16:42.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f414d260 con 0x7f78f41550b0 2024-04-25T10:16:42.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f415a800 con 0x7f78f41550b0 2024-04-25T10:16:42.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.242+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f404ef30 con 0x7f78f41550b0 2024-04-25T10:16:42.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.246+0000 7f78eb400640 1 -- 172.21.15.87:0/2701099651 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78dc01d070 con 0x7f78f41550b0 2024-04-25T10:16:42.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.246+0000 7f78eb400640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f78d0066f60 0x7f78d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.246+0000 7f78eb400640 1 -- 172.21.15.87:0/2701099651 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f78dc085610 con 0x7f78f41550b0 2024-04-25T10:16:42.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.246+0000 7f78f2a00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f78d0066f60 0x7f78d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.246+0000 7f78f2a00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f78d0066f60 0x7f78d0069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f78e4001810 tx=0x7f78e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.250+0000 7f78eb400640 1 -- 172.21.15.87:0/2701099651 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78dc0561d0 con 0x7f78f41550b0 2024-04-25T10:16:42.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.402+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78f40b4800 con 0x7f78f41550b0 2024-04-25T10:16:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78eb400640 1 -- 172.21.15.87:0/2701099651 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+13136 (secure 0 0 0) 0x7f78dc05a270 con 0x7f78f41550b0 2024-04-25T10:16:42.411 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:42.411 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":64,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:41.063300+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/427681613":"2024-04-26T10:16:38.046902+0000","172.21.15.87:6836/427681613":"2024-04-26T10:16:38.046902+0000","172.21.15.87:6837/3387238920":"2024-04-26T10:16:14.297697+0000","172.21.15.87:6835/3387238920":"2024-04-26T10:16:14.297697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:42.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f78d0066f60 msgr2=0x7f78d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f78d0066f60 0x7f78d0069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f78e4001810 tx=0x7f78e4009040 comp rx=0 tx=0).stop 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f41550b0 msgr2=0x7f78f414b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f41550b0 0x7f78f414b160 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00cf60 tx=0x7f78dc00cf90 comp rx=0 tx=0).stop 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 shutdown_connections 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f78d0066f60 0x7f78d0069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f415b740 0x7f78f40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78f41550b0 0x7f78f414b160 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 --2- 172.21.15.87:0/2701099651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f40d0040 0x7f78f41fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 >> 172.21.15.87:0/2701099651 conn(0x7f78f4066b70 msgr2=0x7f78f415dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 shutdown_connections 2024-04-25T10:16:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.406+0000 7f78f9c00640 1 -- 172.21.15.87:0/2701099651 wait complete. 2024-04-25T10:16:42.433 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:42.433 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:42.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.562+0000 7f6e60600640 1 Processor -- start 2024-04-25T10:16:42.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.562+0000 7f6e60600640 1 -- start start 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e60600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e580aef80 0x7f6e5815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e60600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 0x7f6e5815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e60600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5815b080 con 0x7f6e5815bac0 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e60600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5815adb0 con 0x7f6e580aef80 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e60600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815eb60 con 0x7f6e58070350 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5e600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6e5815bac0 0x7f6e580b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38026/0 (socket says 172.21.15.87:38026) 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5e600640 1 -- 172.21.15.87:0/1439545795 learned_addr learned my addr 172.21.15.87:0/1439545795 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/1439545795 >> v2:172.21.15.169:3300/0 conn(0x7f6e580aef80 0x7f6e5815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5d200640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 0x7f6e5815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291483662 0 0) 0x7f6e5815b080 con 0x7f6e5815bac0 2024-04-25T10:16:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e40005740 con 0x7f6e5815bac0 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4237129561 0 0) 0x7f6e40005740 con 0x7f6e5815bac0 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 msgr2=0x7f6e5815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 0x7f6e5815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 >> v2:172.21.15.169:3300/0 conn(0x7f6e580aef80 msgr2=0x7f6e5815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 --2- 172.21.15.87:0/1439545795 >> v2:172.21.15.169:3300/0 conn(0x7f6e580aef80 0x7f6e5815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5815ee40 con 0x7f6e5815bac0 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f6e50001a00 con 0x7f6e5815bac0 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/1439545795 >> v2:172.21.15.169:3300/0 conn(0x7f6e580aef80 0x7f6e5815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 >> v1:172.21.15.87:6789/0 conn(0x7f6e5815bac0 legacy=0x7f6e580b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e40005bd0 0x7f6e40005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 0x7f6e40006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40006f60 0x7f6e400073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815adb0 con 0x7f6e40005bd0 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815eb60 con 0x7f6e40006f60 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4000dbe0 con 0x7f6e58070350 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e40005bd0 0x7f6e40005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5d200640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 0x7f6e40006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5e600640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40006f60 0x7f6e400073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 -- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 msgr2=0x7f6e40006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e58070350 0x7f6e40006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 -- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40006f60 msgr2=0x7f6e400073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40006f60 0x7f6e400073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.566+0000 7f6e5dc00640 1 -- 172.21.15.87:0/1439545795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40005740 con 0x7f6e40005bd0 2024-04-25T10:16:42.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e40005bd0 0x7f6e40005ff0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009cb0 tx=0x7f6e4c01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3ab7d2ab0ce93c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0b2070 con 0x7f6e40005bd0 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c0a5840 con 0x7f6e40005bd0 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e57e00640 1 -- 172.21.15.87:0/1439545795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c009880 con 0x7f6e40005bd0 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e60600640 1 -- 172.21.15.87:0/1439545795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e40005bd0 msgr2=0x7f6e40005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e60600640 1 --2- 172.21.15.87:0/1439545795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e40005bd0 0x7f6e40005ff0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009cb0 tx=0x7f6e4c01cc90 comp rx=0 tx=0).stop 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e5e600640 1 -- 172.21.15.87:0/1439545795 reap_dead start 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e60600640 1 -- 172.21.15.87:0/1439545795 shutdown_connections 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e60600640 1 -- 172.21.15.87:0/1439545795 >> 172.21.15.87:0/1439545795 conn(0x7f6e580669a0 msgr2=0x7f6e58066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e60600640 1 -- 172.21.15.87:0/1439545795 shutdown_connections 2024-04-25T10:16:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.570+0000 7f6e60600640 1 -- 172.21.15.87:0/1439545795 wait complete. 2024-04-25T10:16:42.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 Processor -- start 2024-04-25T10:16:42.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 -- start start 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40005bd0 0x7f6e580b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e58153560 0x7f6e581521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e581f6850 0x7f6e58160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5806c0d0 con 0x7f6e58153560 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580b3b20 con 0x7f6e40005bd0 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5806c250 con 0x7f6e581f6850 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e58153560 0x7f6e581521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e58153560 0x7f6e581521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45676/0 (socket says 172.21.15.87:45676) 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 -- 172.21.15.87:0/568912248 learned_addr learned my addr 172.21.15.87:0/568912248 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40005bd0 0x7f6e580b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5e600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e581f6850 0x7f6e58160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 -- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e581f6850 msgr2=0x7f6e58160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e581f6850 0x7f6e58160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 -- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40005bd0 msgr2=0x7f6e580b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40005bd0 0x7f6e580b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 -- 172.21.15.87:0/568912248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580b2840 con 0x7f6e58153560 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e5d200640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e58153560 0x7f6e581521c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6e440115e0 tx=0x7f6e44011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e55600640 1 -- 172.21.15.87:0/568912248 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44015870 con 0x7f6e58153560 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e55600640 1 -- 172.21.15.87:0/568912248 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44015a10 con 0x7f6e58153560 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e580b0e60 con 0x7f6e58153560 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e55600640 1 -- 172.21.15.87:0/568912248 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44015dc0 con 0x7f6e58153560 2024-04-25T10:16:42.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581519c0 con 0x7f6e58153560 2024-04-25T10:16:42.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.574+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5804ef30 con 0x7f6e58153560 2024-04-25T10:16:42.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.578+0000 7f6e55600640 1 -- 172.21.15.87:0/568912248 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e4400f480 con 0x7f6e58153560 2024-04-25T10:16:42.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.578+0000 7f6e55600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e38066f90 0x7f6e38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.578+0000 7f6e55600640 1 -- 172.21.15.87:0/568912248 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6e440855b0 con 0x7f6e58153560 2024-04-25T10:16:42.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.578+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e38066f90 0x7f6e38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.578+0000 7f6e5dc00640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e38066f90 0x7f6e38069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0af000 tx=0x7f6e4c0046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.582+0000 7f6e55600640 1 -- 172.21.15.87:0/568912248 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e44019030 con 0x7f6e58153560 2024-04-25T10:16:42.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.734+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e580b2fa0 con 0x7f6e58153560 2024-04-25T10:16:42.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.734+0000 7f6e55600640 1 -- 172.21.15.87:0/568912248 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e440568f0 con 0x7f6e58153560 2024-04-25T10:16:42.742 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e38066f90 msgr2=0x7f6e38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e38066f90 0x7f6e38069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0af000 tx=0x7f6e4c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e58153560 msgr2=0x7f6e581521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e58153560 0x7f6e581521c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6e440115e0 tx=0x7f6e44011aa0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 shutdown_connections 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6e38066f90 0x7f6e38069450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e581f6850 0x7f6e58160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e58153560 0x7f6e581521c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 --2- 172.21.15.87:0/568912248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e40005bd0 0x7f6e580b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 >> 172.21.15.87:0/568912248 conn(0x7f6e580669a0 msgr2=0x7f6e5815d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 shutdown_connections 2024-04-25T10:16:42.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.738+0000 7f6e60600640 1 -- 172.21.15.87:0/568912248 wait complete. 2024-04-25T10:16:42.762 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:16:42.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.890+0000 7fd999a00640 1 Processor -- start 2024-04-25T10:16:42.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.890+0000 7fd999a00640 1 -- start start 2024-04-25T10:16:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd999a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9940cda10 0x7fd99406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd999a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940b2770 con 0x7fd9940cf930 2024-04-25T10:16:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd999a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99406c670 con 0x7fd99415bc40 2024-04-25T10:16:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd999a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99406c950 con 0x7fd9940cda10 2024-04-25T10:16:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd98a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9940cda10 0x7fd99406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd993400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd9940cf930 0x7fd9940b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38042/0 (socket says 172.21.15.87:38042) 2024-04-25T10:16:42.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd993400640 1 -- 172.21.15.87:0/179394949 learned_addr learned my addr 172.21.15.87:0/179394949 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:42.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd98a000640 1 -- 172.21.15.87:0/179394949 >> v1:172.21.15.169:6789/0 conn(0x7fd99415bc40 legacy=0x7fd9940b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd98a000640 1 -- 172.21.15.87:0/179394949 >> v1:172.21.15.87:6789/0 conn(0x7fd9940cf930 legacy=0x7fd9940b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd992000640 1 -- 172.21.15.87:0/179394949 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799998791 0 0) 0x7fd99406c670 con 0x7fd99415bc40 2024-04-25T10:16:42.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd98a000640 1 -- 172.21.15.87:0/179394949 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99406cc30 con 0x7fd9940cda10 2024-04-25T10:16:42.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd98a000640 1 --2- 172.21.15.87:0/179394949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cda10 0x7fd99406c090 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0097c0 tx=0x7fd97c01c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fc87f7ac94b135 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd992000640 1 -- 172.21.15.87:0/179394949 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b0070 con 0x7fd9940cda10 2024-04-25T10:16:42.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd992000640 1 -- 172.21.15.87:0/179394949 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c0ab600 con 0x7fd9940cda10 2024-04-25T10:16:42.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.894+0000 7fd992000640 1 -- 172.21.15.87:0/179394949 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0ab9d0 con 0x7fd9940cda10 2024-04-25T10:16:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- 172.21.15.87:0/179394949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cda10 msgr2=0x7fd99406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 --2- 172.21.15.87:0/179394949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cda10 0x7fd99406c090 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0097c0 tx=0x7fd97c01c2a0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- 172.21.15.87:0/179394949 shutdown_connections 2024-04-25T10:16:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 --2- 172.21.15.87:0/179394949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cda10 0x7fd99406c090 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- 172.21.15.87:0/179394949 >> 172.21.15.87:0/179394949 conn(0x7fd994066b70 msgr2=0x7fd994066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- 172.21.15.87:0/179394949 shutdown_connections 2024-04-25T10:16:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- 172.21.15.87:0/179394949 wait complete. 2024-04-25T10:16:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 Processor -- start 2024-04-25T10:16:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- start start 2024-04-25T10:16:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9940cda10 0x7fd994154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cf930 0x7fd9941e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd99415bc40 0x7fd994154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940cff40 con 0x7fd9940cf930 2024-04-25T10:16:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940cfdc0 con 0x7fd9940cda10 2024-04-25T10:16:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd999a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940d00c0 con 0x7fd99415bc40 2024-04-25T10:16:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd98a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cf930 0x7fd9941e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd98a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cf930 0x7fd9941e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45692/0 (socket says 172.21.15.87:45692) 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd98a000640 1 -- 172.21.15.87:0/3138074796 learned_addr learned my addr 172.21.15.87:0/3138074796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd992a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9940cda10 0x7fd994154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.898+0000 7fd993400640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd99415bc40 0x7fd994154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98a000640 1 -- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd99415bc40 msgr2=0x7fd994154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98a000640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd99415bc40 0x7fd994154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98a000640 1 -- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9940cda10 msgr2=0x7fd994154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98a000640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9940cda10 0x7fd994154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98a000640 1 -- 172.21.15.87:0/3138074796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9941e5280 con 0x7fd9940cf930 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd992a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9940cda10 0x7fd994154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:42.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98a000640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cf930 0x7fd9941e4d40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd97c018eb0 tx=0x7fd97c01aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98b400640 1 -- 172.21.15.87:0/3138074796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b0050 con 0x7fd9940cf930 2024-04-25T10:16:42.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98b400640 1 -- 172.21.15.87:0/3138074796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c0bc070 con 0x7fd9940cf930 2024-04-25T10:16:42.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9941e5500 con 0x7fd9940cf930 2024-04-25T10:16:42.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98b400640 1 -- 172.21.15.87:0/3138074796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0aada0 con 0x7fd9940cf930 2024-04-25T10:16:42.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9941e5ac0 con 0x7fd9940cf930 2024-04-25T10:16:42.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9940af630 con 0x7fd9940cf930 2024-04-25T10:16:42.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98b400640 1 -- 172.21.15.87:0/3138074796 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd97c0ab4c0 con 0x7fd9940cf930 2024-04-25T10:16:42.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98b400640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd970066f60 0x7fd970069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd98b400640 1 -- 172.21.15.87:0/3138074796 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd97c11fa80 con 0x7fd9940cf930 2024-04-25T10:16:42.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.902+0000 7fd992a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd970066f60 0x7fd970069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.906+0000 7fd992a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd970066f60 0x7fd970069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd9940b1d10 tx=0x7fd984009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:42.906+0000 7fd98b400640 1 -- 172.21.15.87:0/3138074796 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9940af630 con 0x7fd9940cf930 2024-04-25T10:16:43.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:43.118+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7fd99406d960 con 0x7fd9940cf930 2024-04-25T10:16:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.082+0000 7fd98b400640 1 -- 172.21.15.87:0/3138074796 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v237) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd97c121e20 con 0x7fd9940cf930 2024-04-25T10:16:44.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd970066f60 msgr2=0x7fd970069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd970066f60 0x7fd970069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd9940b1d10 tx=0x7fd984009040 comp rx=0 tx=0).stop 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cf930 msgr2=0x7fd9941e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cf930 0x7fd9941e4d40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd97c018eb0 tx=0x7fd97c01aae0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 shutdown_connections 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd970066f60 0x7fd970069420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd99415bc40 0x7fd994154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9940cf930 0x7fd9941e4d40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 --2- 172.21.15.87:0/3138074796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9940cda10 0x7fd994154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 >> 172.21.15.87:0/3138074796 conn(0x7fd994066b70 msgr2=0x7fd994159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 shutdown_connections 2024-04-25T10:16:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.086+0000 7fd999a00640 1 -- 172.21.15.87:0/3138074796 wait complete. 2024-04-25T10:16:44.114 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:16:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.250+0000 7fc0eae00640 1 Processor -- start 2024-04-25T10:16:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.250+0000 7fc0eae00640 1 -- start start 2024-04-25T10:16:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0eae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0e415a0e0 0x7fc0e415b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0eae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 0x7fc0e406c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0eae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e40b2fd0 con 0x7fc0e40aea50 2024-04-25T10:16:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0eae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e40af1d0 con 0x7fc0e415a0e0 2024-04-25T10:16:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0eae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40b32b0 con 0x7fc0e406c0d0 2024-04-25T10:16:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0db400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc0e40aea50 0x7fc0e40b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38052/0 (socket says 172.21.15.87:38052) 2024-04-25T10:16:44.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0db400640 1 -- 172.21.15.87:0/1044859629 learned_addr learned my addr 172.21.15.87:0/1044859629 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:44.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 0x7fc0e406c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3e00640 1 --2- 172.21.15.87:0/1044859629 >> v2:172.21.15.169:3300/0 conn(0x7fc0e415a0e0 0x7fc0e415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649557635 0 0) 0x7fc0e40b2fd0 con 0x7fc0e40aea50 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c8003530 con 0x7fc0e40aea50 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142073044 0 0) 0x7fc0c8003530 con 0x7fc0e40aea50 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 msgr2=0x7fc0e406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 0x7fc0e406c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 >> v2:172.21.15.169:3300/0 conn(0x7fc0e415a0e0 msgr2=0x7fc0e415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 --2- 172.21.15.87:0/1044859629 >> v2:172.21.15.169:3300/0 conn(0x7fc0e415a0e0 0x7fc0e415b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40b3590 con 0x7fc0e40aea50 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc0cc001680 con 0x7fc0e40aea50 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 >> v1:172.21.15.87:6789/0 conn(0x7fc0e40aea50 legacy=0x7fc0e40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 0x7fc0c80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8003f40 0x7fc0c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c8004d50 0x7fc0c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40af1d0 con 0x7fc0c8004d50 2024-04-25T10:16:44.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40b32b0 con 0x7fc0c8003f40 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c800cb30 con 0x7fc0e406c0d0 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c8004d50 0x7fc0c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0db400640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8003f40 0x7fc0c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e3e00640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 0x7fc0c80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e8e00640 1 -- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 msgr2=0x7fc0c80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e406c0d0 0x7fc0c80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e8e00640 1 -- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8003f40 msgr2=0x7fc0c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8003f40 0x7fc0c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0e8e00640 1 -- 172.21.15.87:0/1044859629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c8003530 con 0x7fc0c8004d50 2024-04-25T10:16:44.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.254+0000 7fc0db400640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8003f40 0x7fc0c8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c8004d50 0x7fc0c80051d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e170 tx=0x7fc0dc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c21b72142772dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0b2070 con 0x7fc0c8004d50 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0dc0a5c70 con 0x7fc0c8004d50 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0e3400640 1 -- 172.21.15.87:0/1044859629 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc009c10 con 0x7fc0c8004d50 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0eae00640 1 -- 172.21.15.87:0/1044859629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c8004d50 msgr2=0x7fc0c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0eae00640 1 --2- 172.21.15.87:0/1044859629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c8004d50 0x7fc0c80051d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e170 tx=0x7fc0dc01cc60 comp rx=0 tx=0).stop 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0e8e00640 1 -- 172.21.15.87:0/1044859629 reap_dead start 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0eae00640 1 -- 172.21.15.87:0/1044859629 shutdown_connections 2024-04-25T10:16:44.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0eae00640 1 -- 172.21.15.87:0/1044859629 >> 172.21.15.87:0/1044859629 conn(0x7fc0e40669a0 msgr2=0x7fc0e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0eae00640 1 -- 172.21.15.87:0/1044859629 shutdown_connections 2024-04-25T10:16:44.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.258+0000 7fc0eae00640 1 -- 172.21.15.87:0/1044859629 wait complete. 2024-04-25T10:16:44.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 Processor -- start 2024-04-25T10:16:44.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 -- start start 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8004d50 0x7fc0e40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e41f6d70 0x7fc0e414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0e414fe20 0x7fc0e4157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40b44a0 con 0x7fc0e414fe20 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40aeec0 con 0x7fc0c8004d50 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40b4620 con 0x7fc0e41f6d70 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0e414fe20 0x7fc0e4157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0e414fe20 0x7fc0e4157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45700/0 (socket says 172.21.15.87:45700) 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 -- 172.21.15.87:0/231750520 learned_addr learned my addr 172.21.15.87:0/231750520 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0db400640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e41f6d70 0x7fc0e414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e3e00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8004d50 0x7fc0e40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 -- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e41f6d70 msgr2=0x7fc0e414f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e41f6d70 0x7fc0e414f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 -- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8004d50 msgr2=0x7fc0e40adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8004d50 0x7fc0e40adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 -- 172.21.15.87:0/231750520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4150330 con 0x7fc0e414fe20 2024-04-25T10:16:44.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e8e00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0e414fe20 0x7fc0e4157c10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009f90 tx=0x7fc0dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e0c00640 1 -- 172.21.15.87:0/231750520 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0b2040 con 0x7fc0e414fe20 2024-04-25T10:16:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e0c00640 1 -- 172.21.15.87:0/231750520 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0dc0a5a70 con 0x7fc0e414fe20 2024-04-25T10:16:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e4150610 con 0x7fc0e414fe20 2024-04-25T10:16:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0e0c00640 1 -- 172.21.15.87:0/231750520 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc009c10 con 0x7fc0e414fe20 2024-04-25T10:16:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e4150bd0 con 0x7fc0e414fe20 2024-04-25T10:16:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.262+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e404ef30 con 0x7fc0e414fe20 2024-04-25T10:16:44.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.266+0000 7fc0e0c00640 1 -- 172.21.15.87:0/231750520 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0dc0be020 con 0x7fc0e414fe20 2024-04-25T10:16:44.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.266+0000 7fc0e0c00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0c0066f60 0x7fc0c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.266+0000 7fc0e0c00640 1 -- 172.21.15.87:0/231750520 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc0dc11ef20 con 0x7fc0e414fe20 2024-04-25T10:16:44.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.266+0000 7fc0e3e00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0c0066f60 0x7fc0c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.266+0000 7fc0e3e00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0c0066f60 0x7fc0c0069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009ba0 tx=0x7fc0d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.270+0000 7fc0e0c00640 1 -- 172.21.15.87:0/231750520 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0dc0c5020 con 0x7fc0e414fe20 2024-04-25T10:16:44.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.422+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc0e4154c50 con 0x7fc0e414fe20 2024-04-25T10:16:44.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.426+0000 7fc0e0c00640 1 -- 172.21.15.87:0/231750520 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc0dc0a5e20 con 0x7fc0e414fe20 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0c0066f60 msgr2=0x7fc0c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0c0066f60 0x7fc0c0069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009ba0 tx=0x7fc0d4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0e414fe20 msgr2=0x7fc0e4157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0e414fe20 0x7fc0e4157c10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009f90 tx=0x7fc0dc009750 comp rx=0 tx=0).stop 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 shutdown_connections 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0c0066f60 0x7fc0c0069420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0e414fe20 0x7fc0e4157c10 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0e41f6d70 0x7fc0e414f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 --2- 172.21.15.87:0/231750520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c8004d50 0x7fc0e40adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 >> 172.21.15.87:0/231750520 conn(0x7fc0e40669a0 msgr2=0x7fc0e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 shutdown_connections 2024-04-25T10:16:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.430+0000 7fc0eae00640 1 -- 172.21.15.87:0/231750520 wait complete. 2024-04-25T10:16:44.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:44.456 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:44.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:44.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:44.658 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:44.658 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:44.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:44.667 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:16:44.667 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:44.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:44.676 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:44.676 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:44.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:44.718 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:16:44.718 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:44.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:44.727 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:16:44.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f643dc00640 1 Processor -- start 2024-04-25T10:16:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f643dc00640 1 -- start start 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f643dc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64380cf930 0x7f64380cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f643dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6438070c20 0x7f643815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f643dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64380d0250 con 0x7f64380ae020 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f643dc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643815bb90 con 0x7f64380cf930 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f643dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380ad840 con 0x7f6438070c20 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f6436a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64380cf930 0x7f64380cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f6436a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64380cf930 0x7f64380cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45704/0 (socket says 172.21.15.87:45704) 2024-04-25T10:16:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.890+0000 7f6436a00640 1 -- 172.21.15.87:0/3501228042 learned_addr learned my addr 172.21.15.87:0/3501228042 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:44.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f642e000640 1 --2- 172.21.15.87:0/3501228042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6438070c20 0x7f643815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436a00640 1 -- 172.21.15.87:0/3501228042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6438070c20 msgr2=0x7f643815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436a00640 1 --2- 172.21.15.87:0/3501228042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6438070c20 0x7f643815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436a00640 1 -- 172.21.15.87:0/3501228042 >> v1:172.21.15.169:6789/0 conn(0x7f64380ae020 legacy=0x7f64380b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436a00640 1 -- 172.21.15.87:0/3501228042 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643815e9f0 con 0x7f64380cf930 2024-04-25T10:16:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436a00640 1 --2- 172.21.15.87:0/3501228042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64380cf930 0x7f64380cfd10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6428009e50 tx=0x7f642801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c4bb7144cd0c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436000640 1 -- 172.21.15.87:0/3501228042 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280a5840 con 0x7f64380cf930 2024-04-25T10:16:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436000640 1 -- 172.21.15.87:0/3501228042 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64280a59e0 con 0x7f64380cf930 2024-04-25T10:16:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f6436000640 1 -- 172.21.15.87:0/3501228042 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280a5db0 con 0x7f64380cf930 2024-04-25T10:16:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f643dc00640 1 -- 172.21.15.87:0/3501228042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64380cf930 msgr2=0x7f64380cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f643dc00640 1 --2- 172.21.15.87:0/3501228042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64380cf930 0x7f64380cfd10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6428009e50 tx=0x7f642801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f643dc00640 1 -- 172.21.15.87:0/3501228042 shutdown_connections 2024-04-25T10:16:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f643dc00640 1 --2- 172.21.15.87:0/3501228042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6438070c20 0x7f643815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.894+0000 7f643dc00640 1 --2- 172.21.15.87:0/3501228042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64380cf930 0x7f64380cfd10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- 172.21.15.87:0/3501228042 >> 172.21.15.87:0/3501228042 conn(0x7f6438066b70 msgr2=0x7f6438066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- 172.21.15.87:0/3501228042 shutdown_connections 2024-04-25T10:16:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- 172.21.15.87:0/3501228042 wait complete. 2024-04-25T10:16:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 Processor -- start 2024-04-25T10:16:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- start start 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6438070c20 0x7f64381550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380ae020 0x7f643813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380cf930 0x7f6438159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643815e6b0 con 0x7f6438070c20 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643806e130 con 0x7f64380ae020 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643816b0f0 con 0x7f64380cf930 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6438070c20 0x7f64381550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6438070c20 0x7f64381550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45718/0 (socket says 172.21.15.87:45718) 2024-04-25T10:16:44.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 -- 172.21.15.87:0/3144899771 learned_addr learned my addr 172.21.15.87:0/3144899771 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f642e000640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380ae020 0x7f643813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6437400640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380cf930 0x7f6438159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 -- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380cf930 msgr2=0x7f6438159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380cf930 0x7f6438159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 -- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380ae020 msgr2=0x7f643813ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380ae020 0x7f643813ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 -- 172.21.15.87:0/3144899771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6438155610 con 0x7f6438070c20 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6437400640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380cf930 0x7f6438159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f6436a00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6438070c20 0x7f64381550d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f642801b070 tx=0x7f6428019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f642f400640 1 -- 172.21.15.87:0/3144899771 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280b8670 con 0x7f6438070c20 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f642f400640 1 -- 172.21.15.87:0/3144899771 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64280be070 con 0x7f6438070c20 2024-04-25T10:16:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f642f400640 1 -- 172.21.15.87:0/3144899771 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280b8a50 con 0x7f6438070c20 2024-04-25T10:16:44.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.898+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643815caa0 con 0x7f6438070c20 2024-04-25T10:16:44.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.902+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643815d060 con 0x7f6438070c20 2024-04-25T10:16:44.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.902+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643804ef30 con 0x7f6438070c20 2024-04-25T10:16:44.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.902+0000 7f642f400640 1 -- 172.21.15.87:0/3144899771 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64280b8bf0 con 0x7f6438070c20 2024-04-25T10:16:44.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.902+0000 7f642f400640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6414066f60 0x7f6414069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.902+0000 7f642f400640 1 -- 172.21.15.87:0/3144899771 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6428120030 con 0x7f6438070c20 2024-04-25T10:16:44.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.902+0000 7f642e000640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6414066f60 0x7f6414069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.902+0000 7f642e000640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6414066f60 0x7f6414069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f643815fb90 tx=0x7f6420008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:44.906+0000 7f642f400640 1 -- 172.21.15.87:0/3144899771 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64280165e0 con 0x7f6438070c20 2024-04-25T10:16:45.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.094+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6438067060 con 0x7f6438070c20 2024-04-25T10:16:45.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f642f400640 1 -- 172.21.15.87:0/3144899771 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f64280f1240 con 0x7f6438070c20 2024-04-25T10:16:45.103 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:45.104 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2976358870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2976358870},{"type":"v1","addr":"172.21.15.169:6834","nonce":2976358870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2826441278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2826441278},{"type":"v1","addr":"172.21.15.87:6835","nonce":2826441278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3013040205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3013040205},{"type":"v1","addr":"172.21.15.169:6835","nonce":3013040205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6933,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2475035494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2475035494},{"type":"v1","addr":"172.21.15.87:6837","nonce":2475035494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2024-04-25T10:16:45.104 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 46 2024-04-25T10:16:45.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6414066f60 msgr2=0x7f6414069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6414066f60 0x7f6414069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f643815fb90 tx=0x7f6420008040 comp rx=0 tx=0).stop 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6438070c20 msgr2=0x7f64381550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6438070c20 0x7f64381550d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f642801b070 tx=0x7f6428019760 comp rx=0 tx=0).stop 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 shutdown_connections 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6414066f60 0x7f6414069420 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380cf930 0x7f6438159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380ae020 0x7f643813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 --2- 172.21.15.87:0/3144899771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6438070c20 0x7f64381550d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 >> 172.21.15.87:0/3144899771 conn(0x7f6438066b70 msgr2=0x7f64380b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 shutdown_connections 2024-04-25T10:16:45.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.098+0000 7f643dc00640 1 -- 172.21.15.87:0/3144899771 wait complete. 2024-04-25T10:16:45.125 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:16:45.125 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:16:45.125 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:16:45.125 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:45.126 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:16:45.126 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:16:45.126 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:16:45.126 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:45.126 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:16:45.126 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:16:45.126 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:16:45.126 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:45.127 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:16:45.127 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:16:45.127 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:16:45.127 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:45.138 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:16:45.138 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:16:45.139 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:16:45.139 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:16:45.141 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:16:45.141 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:16:45.143 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:16:45.143 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:16:45.185 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:16:45.186 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:45.187 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:16:45.187 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:45.188 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:16:45.188 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:16:45.188 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:45.189 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:45.189 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:16:45.272 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:16:45.273 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:16:45.278 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:16:45.279 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:16:45.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 Processor -- start 2024-04-25T10:16:45.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 -- start start 2024-04-25T10:16:45.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f51dc1550b0 0x7f51dc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f51dc0cfe70 0x7f51dc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc071050 con 0x7f51dc0cfe70 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0712f0 con 0x7f51dc1550b0 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e4800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0715d0 con 0x7f51dc15b740 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f51dc0cfe70 0x7f51dc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f51dc0cfe70 0x7f51dc15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45742/0 (socket says 172.21.15.87:45742) 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 -- 172.21.15.87:0/1976165180 learned_addr learned my addr 172.21.15.87:0/1976165180 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1e00640 1 --2- 172.21.15.87:0/1976165180 >> v2:172.21.15.169:3300/0 conn(0x7f51dc1550b0 0x7f51dc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e2800640 1 --2- 172.21.15.87:0/1976165180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 -- 172.21.15.87:0/1976165180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 msgr2=0x7f51dc06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 --2- 172.21.15.87:0/1976165180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc06d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 -- 172.21.15.87:0/1976165180 >> v2:172.21.15.169:3300/0 conn(0x7f51dc1550b0 msgr2=0x7f51dc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:45.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 --2- 172.21.15.87:0/1976165180 >> v2:172.21.15.169:3300/0 conn(0x7f51dc1550b0 0x7f51dc0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1400640 1 -- 172.21.15.87:0/1976165180 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0718b0 con 0x7f51dc0cfe70 2024-04-25T10:16:45.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.326+0000 7f51e1e00640 1 --2- 172.21.15.87:0/1976165180 >> v2:172.21.15.169:3300/0 conn(0x7f51dc1550b0 0x7f51dc0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:45.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e1400640 1 --2- 172.21.15.87:0/1976165180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc15b200 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009870 tx=0x7f51cc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c4a22b1dd6a14d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:45.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e0a00640 1 -- 172.21.15.87:0/1976165180 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0b0070 con 0x7f51dc0cfe70 2024-04-25T10:16:45.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e0a00640 1 -- 172.21.15.87:0/1976165180 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0ab600 con 0x7f51dc0cfe70 2024-04-25T10:16:45.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e0a00640 1 -- 172.21.15.87:0/1976165180 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0ab9d0 con 0x7f51dc0cfe70 2024-04-25T10:16:45.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 -- 172.21.15.87:0/1976165180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 msgr2=0x7f51dc15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 --2- 172.21.15.87:0/1976165180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc15b200 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009870 tx=0x7f51cc01c260 comp rx=0 tx=0).stop 2024-04-25T10:16:45.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 -- 172.21.15.87:0/1976165180 shutdown_connections 2024-04-25T10:16:45.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 --2- 172.21.15.87:0/1976165180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc15b200 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 --2- 172.21.15.87:0/1976165180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 --2- 172.21.15.87:0/1976165180 >> v2:172.21.15.169:3300/0 conn(0x7f51dc1550b0 0x7f51dc0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 -- 172.21.15.87:0/1976165180 >> 172.21.15.87:0/1976165180 conn(0x7f51dc066b70 msgr2=0x7f51dc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:45.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 -- 172.21.15.87:0/1976165180 shutdown_connections 2024-04-25T10:16:45.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.330+0000 7f51e4800640 1 -- 172.21.15.87:0/1976165180 wait complete. 2024-04-25T10:16:45.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 Processor -- start 2024-04-25T10:16:45.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 -- start start 2024-04-25T10:16:45.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51dc1550b0 0x7f51dc1f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0b3260 con 0x7f51dc0cfe70 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15bd60 con 0x7f51dc1550b0 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0b33e0 con 0x7f51dc15b740 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45746/0 (socket says 172.21.15.87:45746) 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 -- 172.21.15.87:0/2695668746 learned_addr learned my addr 172.21.15.87:0/2695668746 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e2800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1400640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51dc1550b0 0x7f51dc1f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 -- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 msgr2=0x7f51dc153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 -- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51dc1550b0 msgr2=0x7f51dc1f9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51dc1550b0 0x7f51dc1f9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 -- 172.21.15.87:0/2695668746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc155640 con 0x7f51dc0cfe70 2024-04-25T10:16:45.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e2800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:45.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e1e00640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc153500 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f51d000e810 tx=0x7f51d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51d6000640 1 -- 172.21.15.87:0/2695668746 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0017070 con 0x7f51dc0cfe70 2024-04-25T10:16:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51d6000640 1 -- 172.21.15.87:0/2695668746 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d000c8d0 con 0x7f51dc0cfe70 2024-04-25T10:16:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc1fa5b0 con 0x7f51dc0cfe70 2024-04-25T10:16:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51d6000640 1 -- 172.21.15.87:0/2695668746 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000f040 con 0x7f51dc0cfe70 2024-04-25T10:16:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.334+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc1fab70 con 0x7f51dc0cfe70 2024-04-25T10:16:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.338+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc04ef30 con 0x7f51dc0cfe70 2024-04-25T10:16:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.338+0000 7f51d6000640 1 -- 172.21.15.87:0/2695668746 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51d001a070 con 0x7f51dc0cfe70 2024-04-25T10:16:45.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:16:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.338+0000 7f51d6000640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51c0066f60 0x7f51c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.338+0000 7f51d6000640 1 -- 172.21.15.87:0/2695668746 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f51d0084950 con 0x7f51dc0cfe70 2024-04-25T10:16:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.338+0000 7f51e1400640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51c0066f60 0x7f51c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.338+0000 7f51e1400640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51c0066f60 0x7f51c0069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51cc0ad000 tx=0x7f51cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:45.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.342+0000 7f51d6000640 1 -- 172.21.15.87:0/2695668746 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51d0055510 con 0x7f51dc0cfe70 2024-04-25T10:16:45.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:45.506+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f51dc06c2a0 con 0x7f51dc0cfe70 2024-04-25T10:16:46.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.094+0000 7f51d6000640 1 -- 172.21.15.87:0/2695668746 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7f51d00595b0 con 0x7f51dc0cfe70 2024-04-25T10:16:46.102 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:16:46.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51c0066f60 msgr2=0x7f51c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51c0066f60 0x7f51c0069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51cc0ad000 tx=0x7f51cc008040 comp rx=0 tx=0).stop 2024-04-25T10:16:46.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 msgr2=0x7f51dc153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc153500 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f51d000e810 tx=0x7f51d000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 shutdown_connections 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f51c0066f60 0x7f51c0069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51dc15b740 0x7f51dc153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51dc1550b0 0x7f51dc1f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 --2- 172.21.15.87:0/2695668746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51dc0cfe70 0x7f51dc153500 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 >> 172.21.15.87:0/2695668746 conn(0x7f51dc066b70 msgr2=0x7f51dc0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 shutdown_connections 2024-04-25T10:16:46.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.098+0000 7f51e4800640 1 -- 172.21.15.87:0/2695668746 wait complete. 2024-04-25T10:16:46.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:16:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.258+0000 7f3491a00640 1 Processor -- start 2024-04-25T10:16:46.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.258+0000 7f3491a00640 1 -- start start 2024-04-25T10:16:46.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f3491a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f3491a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f348c071f70 0x7f348c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f3491a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c15b6e0 con 0x7f348c0b2fc0 2024-04-25T10:16:46.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f3491a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c070ed0 con 0x7f348c071f70 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f3491a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0711b0 con 0x7f348c15a0e0 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f348c071f70 0x7f348c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f348c071f70 0x7f348c0b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45762/0 (socket says 172.21.15.87:45762) 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348a000640 1 -- 172.21.15.87:0/1783913445 learned_addr learned my addr 172.21.15.87:0/1783913445 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348aa00640 1 --2- 172.21.15.87:0/1783913445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348a000640 1 -- 172.21.15.87:0/1783913445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 msgr2=0x7f348c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348a000640 1 --2- 172.21.15.87:0/1783913445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348a000640 1 -- 172.21.15.87:0/1783913445 >> v1:172.21.15.169:6789/0 conn(0x7f348c0b2fc0 legacy=0x7f348c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348a000640 1 -- 172.21.15.87:0/1783913445 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c071490 con 0x7f348c071f70 2024-04-25T10:16:46.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.262+0000 7f348aa00640 1 --2- 172.21.15.87:0/1783913445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:46.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f348a000640 1 --2- 172.21.15.87:0/1783913445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c071f70 0x7f348c0b2a80 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3478009870 tx=0x7f347801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d106d20eecdb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3489600640 1 -- 172.21.15.87:0/1783913445 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780b0070 con 0x7f348c071f70 2024-04-25T10:16:46.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3489600640 1 -- 172.21.15.87:0/1783913445 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34780ab600 con 0x7f348c071f70 2024-04-25T10:16:46.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3489600640 1 -- 172.21.15.87:0/1783913445 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780ab9d0 con 0x7f348c071f70 2024-04-25T10:16:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- 172.21.15.87:0/1783913445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c071f70 msgr2=0x7f348c0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 --2- 172.21.15.87:0/1783913445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c071f70 0x7f348c0b2a80 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3478009870 tx=0x7f347801c340 comp rx=0 tx=0).stop 2024-04-25T10:16:46.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- 172.21.15.87:0/1783913445 shutdown_connections 2024-04-25T10:16:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 --2- 172.21.15.87:0/1783913445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c071f70 0x7f348c0b2a80 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 --2- 172.21.15.87:0/1783913445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- 172.21.15.87:0/1783913445 >> 172.21.15.87:0/1783913445 conn(0x7f348c066b70 msgr2=0x7f348c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- 172.21.15.87:0/1783913445 shutdown_connections 2024-04-25T10:16:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- 172.21.15.87:0/1783913445 wait complete. 2024-04-25T10:16:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 Processor -- start 2024-04-25T10:16:46.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- start start 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c071f70 0x7f348c1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c0b2fc0 0x7f348c14bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c15a4c0 con 0x7f348c0b2fc0 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0af030 con 0x7f348c071f70 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f3491a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c070b50 con 0x7f348c15a0e0 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.266+0000 7f348a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c0b2fc0 0x7f348c14bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c0b2fc0 0x7f348c14bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45764/0 (socket says 172.21.15.87:45764) 2024-04-25T10:16:46.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 -- 172.21.15.87:0/237212591 learned_addr learned my addr 172.21.15.87:0/237212591 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348b400640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348aa00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c071f70 0x7f348c1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 -- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 msgr2=0x7f348c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 -- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c071f70 msgr2=0x7f348c1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c071f70 0x7f348c1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 -- 172.21.15.87:0/237212591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c14ac60 con 0x7f348c0b2fc0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348aa00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c071f70 0x7f348c1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f348a000640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c0b2fc0 0x7f348c14bd00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f34780ae040 tx=0x7f34780190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3482a00640 1 -- 172.21.15.87:0/237212591 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780ba070 con 0x7f348c0b2fc0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3482a00640 1 -- 172.21.15.87:0/237212591 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34780b4640 con 0x7f348c0b2fc0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3482a00640 1 -- 172.21.15.87:0/237212591 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780b0040 con 0x7f348c0b2fc0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c14af40 con 0x7f348c0b2fc0 2024-04-25T10:16:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c14b500 con 0x7f348c0b2fc0 2024-04-25T10:16:46.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c04ef30 con 0x7f348c0b2fc0 2024-04-25T10:16:46.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3482a00640 1 -- 172.21.15.87:0/237212591 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34780ab4c0 con 0x7f348c0b2fc0 2024-04-25T10:16:46.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.270+0000 7f3482a00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f346c066f60 0x7f346c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.274+0000 7f3482a00640 1 -- 172.21.15.87:0/237212591 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f347811dff0 con 0x7f348c0b2fc0 2024-04-25T10:16:46.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.274+0000 7f348aa00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f346c066f60 0x7f346c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.274+0000 7f348aa00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f346c066f60 0x7f346c069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f348c15dff0 tx=0x7f347c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.274+0000 7f3482a00640 1 -- 172.21.15.87:0/237212591 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f348c15b6e0 con 0x7f348c0b2fc0 2024-04-25T10:16:46.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f348c0703e0 con 0x7f348c0b2fc0 2024-04-25T10:16:46.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3482a00640 1 -- 172.21.15.87:0/237212591 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f34780ef360 con 0x7f348c0b2fc0 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:16:46.490 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:16:46.491 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.492 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.493 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.494 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:16:46.495 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:16:46.497 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:16:46.498 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:16:46.499 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:16:46.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:16:46.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:16:46.502 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:16:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f346c066f60 msgr2=0x7f346c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f346c066f60 0x7f346c069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f348c15dff0 tx=0x7f347c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c0b2fc0 msgr2=0x7f348c14bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c0b2fc0 0x7f348c14bd00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f34780ae040 tx=0x7f34780190e0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 shutdown_connections 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f346c066f60 0x7f346c069420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c15a0e0 0x7f348c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f348c0b2fc0 0x7f348c14bd00 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 --2- 172.21.15.87:0/237212591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c071f70 0x7f348c1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 >> 172.21.15.87:0/237212591 conn(0x7f348c066b70 msgr2=0x7f348c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 shutdown_connections 2024-04-25T10:16:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.482+0000 7f3491a00640 1 -- 172.21.15.87:0/237212591 wait complete. 2024-04-25T10:16:46.509 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:46.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.638+0000 7efde2e00640 1 Processor -- start 2024-04-25T10:16:46.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.638+0000 7efde2e00640 1 -- start start 2024-04-25T10:16:46.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.638+0000 7efde2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cf930 0x7efddc06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.638+0000 7efde2e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc0b4470 con 0x7efddc0cef90 2024-04-25T10:16:46.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.638+0000 7efde2e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc0b4710 con 0x7efddc070310 2024-04-25T10:16:46.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.638+0000 7efde2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0b4cd0 con 0x7efddc0cf930 2024-04-25T10:16:46.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddb400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efddc0cef90 0x7efddc0cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38070/0 (socket says 172.21.15.87:38070) 2024-04-25T10:16:46.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddb400640 1 -- 172.21.15.87:0/1872592626 learned_addr learned my addr 172.21.15.87:0/1872592626 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddbe00640 1 --2- 172.21.15.87:0/1872592626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cf930 0x7efddc06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43141151 0 0) 0x7efddc0b4470 con 0x7efddc0cef90 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdc0003530 con 0x7efddc0cef90 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018432137 0 0) 0x7efddc0b4710 con 0x7efddc070310 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efddc0b4470 con 0x7efddc070310 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddbe00640 1 -- 172.21.15.87:0/1872592626 >> v1:172.21.15.169:6789/0 conn(0x7efddc070310 legacy=0x7efddc0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddbe00640 1 -- 172.21.15.87:0/1872592626 >> v1:172.21.15.87:6789/0 conn(0x7efddc0cef90 legacy=0x7efddc0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707904131 0 0) 0x7efdc0003530 con 0x7efddc0cef90 2024-04-25T10:16:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddbe00640 1 -- 172.21.15.87:0/1872592626 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0b49f0 con 0x7efddc0cf930 2024-04-25T10:16:46.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddbe00640 1 --2- 172.21.15.87:0/1872592626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cf930 0x7efddc06ca50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009df0 tx=0x7efdcc01c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d7e7b833fd2377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.642+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0b2070 con 0x7efddc0cf930 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc0a55b0 con 0x7efddc0cf930 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efddaa00640 1 -- 172.21.15.87:0/1872592626 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8001680 con 0x7efddc0cf930 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- 172.21.15.87:0/1872592626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cf930 msgr2=0x7efddc06ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 --2- 172.21.15.87:0/1872592626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cf930 0x7efddc06ca50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009df0 tx=0x7efdcc01c820 comp rx=0 tx=0).stop 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- 172.21.15.87:0/1872592626 shutdown_connections 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 --2- 172.21.15.87:0/1872592626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cf930 0x7efddc06ca50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- 172.21.15.87:0/1872592626 >> 172.21.15.87:0/1872592626 conn(0x7efddc066b70 msgr2=0x7efddc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- 172.21.15.87:0/1872592626 shutdown_connections 2024-04-25T10:16:46.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- 172.21.15.87:0/1872592626 wait complete. 2024-04-25T10:16:46.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 Processor -- start 2024-04-25T10:16:46.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- start start 2024-04-25T10:16:46.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efddc070310 0x7efddc1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cef90 0x7efddc15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0cf930 0x7efddc1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc071fc0 con 0x7efddc070310 2024-04-25T10:16:46.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc071e40 con 0x7efddc0cf930 2024-04-25T10:16:46.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efde2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0b2e30 con 0x7efddc0cef90 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efddbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efddc070310 0x7efddc1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efddbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efddc070310 0x7efddc1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45766/0 (socket says 172.21.15.87:45766) 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efddbe00640 1 -- 172.21.15.87:0/4289733263 learned_addr learned my addr 172.21.15.87:0/4289733263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.646+0000 7efddb400640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cef90 0x7efddc15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efde0e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0cf930 0x7efddc1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efddbe00640 1 -- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cef90 msgr2=0x7efddc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efddbe00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cef90 0x7efddc15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efddbe00640 1 -- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0cf930 msgr2=0x7efddc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efddbe00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0cf930 0x7efddc1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efddbe00640 1 -- 172.21.15.87:0/4289733263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc1571c0 con 0x7efddc070310 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efddbe00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efddc070310 0x7efddc1556e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efdcc01aed0 tx=0x7efdcc0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efdd3e00640 1 -- 172.21.15.87:0/4289733263 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0bd020 con 0x7efddc070310 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efdd3e00640 1 -- 172.21.15.87:0/4289733263 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc0b8670 con 0x7efddc070310 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efdd3e00640 1 -- 172.21.15.87:0/4289733263 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0b2040 con 0x7efddc070310 2024-04-25T10:16:46.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc160ee0 con 0x7efddc070310 2024-04-25T10:16:46.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc154ee0 con 0x7efddc070310 2024-04-25T10:16:46.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc04ef30 con 0x7efddc070310 2024-04-25T10:16:46.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efdd3e00640 1 -- 172.21.15.87:0/4289733263 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efdcc0c0020 con 0x7efddc070310 2024-04-25T10:16:46.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efdd3e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efdbc066f60 0x7efdbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.650+0000 7efdd3e00640 1 -- 172.21.15.87:0/4289733263 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efdcc11f9c0 con 0x7efddc070310 2024-04-25T10:16:46.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.654+0000 7efddb400640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efdbc066f60 0x7efdbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.654+0000 7efddb400640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efdbc066f60 0x7efdbc069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efddc0b0f40 tx=0x7efdc8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.658+0000 7efdd3e00640 1 -- 172.21.15.87:0/4289733263 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efdcc121e20 con 0x7efddc070310 2024-04-25T10:16:46.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.810+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efddc15d060 con 0x7efddc070310 2024-04-25T10:16:46.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.810+0000 7efdd3e00640 1 -- 172.21.15.87:0/4289733263 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+12879 (secure 0 0 0) 0x7efdcc0c7070 con 0x7efddc070310 2024-04-25T10:16:46.818 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:46.818 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":68,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:46.107752+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:46.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efdbc066f60 msgr2=0x7efdbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efdbc066f60 0x7efdbc069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efddc0b0f40 tx=0x7efdc8009040 comp rx=0 tx=0).stop 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efddc070310 msgr2=0x7efddc1556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efddc070310 0x7efddc1556e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efdcc01aed0 tx=0x7efdcc0195c0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 shutdown_connections 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efdbc066f60 0x7efdbc069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0cf930 0x7efddc1563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efddc0cef90 0x7efddc15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 --2- 172.21.15.87:0/4289733263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efddc070310 0x7efddc1556e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 >> 172.21.15.87:0/4289733263 conn(0x7efddc066b70 msgr2=0x7efddc15db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 shutdown_connections 2024-04-25T10:16:46.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.814+0000 7efde2e00640 1 -- 172.21.15.87:0/4289733263 wait complete. 2024-04-25T10:16:46.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:46.840 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:46.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.970+0000 7fec59c00640 1 Processor -- start 2024-04-25T10:16:46.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.970+0000 7fec59c00640 1 -- start start 2024-04-25T10:16:46.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec59c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec540721f0 0x7fec54166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec59c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec54071360 con 0x7fec540aeda0 2024-04-25T10:16:46.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec59c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec54159920 con 0x7fec54159540 2024-04-25T10:16:46.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec59c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5415c2f0 con 0x7fec540721f0 2024-04-25T10:16:46.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec53400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec540721f0 0x7fec54166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec53400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec540721f0 0x7fec54166920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45782/0 (socket says 172.21.15.87:45782) 2024-04-25T10:16:46.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec53400640 1 -- 172.21.15.87:0/2303380653 learned_addr learned my addr 172.21.15.87:0/2303380653 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:46.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec4a000640 1 --1- 172.21.15.87:0/2303380653 >> v1:172.21.15.87:6789/0 conn(0x7fec540aeda0 0x7fec54071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38082/0 (socket says 172.21.15.87:38082) 2024-04-25T10:16:46.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec53400640 1 -- 172.21.15.87:0/2303380653 >> v1:172.21.15.169:6789/0 conn(0x7fec54159540 legacy=0x7fec540ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec53400640 1 -- 172.21.15.87:0/2303380653 >> v1:172.21.15.87:6789/0 conn(0x7fec540aeda0 legacy=0x7fec54071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec53400640 1 -- 172.21.15.87:0/2303380653 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5415c5d0 con 0x7fec540721f0 2024-04-25T10:16:46.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec53400640 1 --2- 172.21.15.87:0/2303380653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec54166920 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009e50 tx=0x7fec4c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b33d66663b3d147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec52000640 1 -- 172.21.15.87:0/2303380653 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0a58d0 con 0x7fec540721f0 2024-04-25T10:16:46.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec52000640 1 -- 172.21.15.87:0/2303380653 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c0a5a70 con 0x7fec540721f0 2024-04-25T10:16:46.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.974+0000 7fec52000640 1 -- 172.21.15.87:0/2303380653 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0a5e40 con 0x7fec540721f0 2024-04-25T10:16:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- 172.21.15.87:0/2303380653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 msgr2=0x7fec54166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 --2- 172.21.15.87:0/2303380653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec54166920 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009e50 tx=0x7fec4c01ca00 comp rx=0 tx=0).stop 2024-04-25T10:16:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- 172.21.15.87:0/2303380653 shutdown_connections 2024-04-25T10:16:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 --2- 172.21.15.87:0/2303380653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec54166920 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- 172.21.15.87:0/2303380653 >> 172.21.15.87:0/2303380653 conn(0x7fec540669a0 msgr2=0x7fec54066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- 172.21.15.87:0/2303380653 shutdown_connections 2024-04-25T10:16:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- 172.21.15.87:0/2303380653 wait complete. 2024-04-25T10:16:46.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 Processor -- start 2024-04-25T10:16:46.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- start start 2024-04-25T10:16:46.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec5414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec540aeda0 0x7fec54156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec54159540 0x7fec5415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec54169680 con 0x7fec540721f0 2024-04-25T10:16:46.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5406c0d0 con 0x7fec540aeda0 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec59c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec54169800 con 0x7fec54159540 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec5414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec5414d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45798/0 (socket says 172.21.15.87:45798) 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 -- 172.21.15.87:0/2488147083 learned_addr learned my addr 172.21.15.87:0/2488147083 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec4a000640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec540aeda0 0x7fec54156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec53400640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec54159540 0x7fec5415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 -- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec54159540 msgr2=0x7fec5415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec54159540 0x7fec5415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 -- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec540aeda0 msgr2=0x7fec54156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec540aeda0 0x7fec54156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec52a00640 1 -- 172.21.15.87:0/2488147083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec540cbc40 con 0x7fec540721f0 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.978+0000 7fec53400640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec54159540 0x7fec5415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec52a00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec5414d740 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fec4400c3a0 tx=0x7fec4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4b400640 1 -- 172.21.15.87:0/2488147083 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec440052d0 con 0x7fec540721f0 2024-04-25T10:16:46.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4b400640 1 -- 172.21.15.87:0/2488147083 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec44005470 con 0x7fec540721f0 2024-04-25T10:16:46.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4b400640 1 -- 172.21.15.87:0/2488147083 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44007260 con 0x7fec540721f0 2024-04-25T10:16:46.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec54151720 con 0x7fec540721f0 2024-04-25T10:16:46.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec54151ce0 con 0x7fec540721f0 2024-04-25T10:16:46.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5404ef30 con 0x7fec540721f0 2024-04-25T10:16:46.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4b400640 1 -- 172.21.15.87:0/2488147083 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec44007400 con 0x7fec540721f0 2024-04-25T10:16:46.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4b400640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec30066f60 0x7fec30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4b400640 1 -- 172.21.15.87:0/2488147083 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec44084160 con 0x7fec540721f0 2024-04-25T10:16:46.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4a000640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec30066f60 0x7fec30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.982+0000 7fec4a000640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec30066f60 0x7fec30069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fec5415b5c0 tx=0x7fec3c007e50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:46.986+0000 7fec4b400640 1 -- 172.21.15.87:0/2488147083 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec44054dd0 con 0x7fec540721f0 2024-04-25T10:16:47.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec540ccf50 con 0x7fec540721f0 2024-04-25T10:16:47.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec4b400640 1 -- 172.21.15.87:0/2488147083 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec44058e70 con 0x7fec540721f0 2024-04-25T10:16:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec30066f60 msgr2=0x7fec30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec30066f60 0x7fec30069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fec5415b5c0 tx=0x7fec3c007e50 comp rx=0 tx=0).stop 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 msgr2=0x7fec5414d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec5414d740 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fec4400c3a0 tx=0x7fec4400c860 comp rx=0 tx=0).stop 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 shutdown_connections 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec30066f60 0x7fec30069420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec54159540 0x7fec5415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec540aeda0 0x7fec54156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 --2- 172.21.15.87:0/2488147083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec540721f0 0x7fec5414d740 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 >> 172.21.15.87:0/2488147083 conn(0x7fec540669a0 msgr2=0x7fec540cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 shutdown_connections 2024-04-25T10:16:47.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.146+0000 7fec59c00640 1 -- 172.21.15.87:0/2488147083 wait complete. 2024-04-25T10:16:47.173 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:16:47.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac3400640 1 Processor -- start 2024-04-25T10:16:47.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac3400640 1 -- start start 2024-04-25T10:16:47.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0abc0d1820 0x7f0abc0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac3400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc0b1af0 con 0x7f0abc0b15b0 2024-04-25T10:16:47.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac3400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc0b1dd0 con 0x7f0abc15e050 2024-04-25T10:16:47.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac3400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc0b20b0 con 0x7f0abc0d1820 2024-04-25T10:16:47.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac1400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0abc0b15b0 0x7f0abc0cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38090/0 (socket says 172.21.15.87:38090) 2024-04-25T10:16:47.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac1400640 1 -- 172.21.15.87:0/2471799821 learned_addr learned my addr 172.21.15.87:0/2471799821 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:47.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/2471799821 >> v2:172.21.15.169:3300/0 conn(0x7f0abc0d1820 0x7f0abc0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254759352 0 0) 0x7f0abc0b1af0 con 0x7f0abc0b15b0 2024-04-25T10:16:47.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0003530 con 0x7f0abc0b15b0 2024-04-25T10:16:47.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1689847491 0 0) 0x7f0aa0003530 con 0x7f0abc0b15b0 2024-04-25T10:16:47.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 >> v2:172.21.15.169:3300/0 conn(0x7f0abc0d1820 msgr2=0x7f0abc0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 --2- 172.21.15.87:0/2471799821 >> v2:172.21.15.169:3300/0 conn(0x7f0abc0d1820 0x7f0abc0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 >> v1:172.21.15.169:6789/0 conn(0x7f0abc15e050 legacy=0x7f0abc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc0b2390 con 0x7f0abc0b15b0 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0ab4001a00 con 0x7f0abc0b15b0 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 >> v1:172.21.15.87:6789/0 conn(0x7f0abc0b15b0 legacy=0x7f0abc0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0aa00039c0 0x7f0aa0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa0004810 0x7f0aa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa00051d0 0x7f0aa000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0b20b0 con 0x7f0aa00039c0 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa000c070 con 0x7f0aa00051d0 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa000c270 con 0x7f0aa0004810 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.302+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0aa00039c0 0x7f0aa0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac1400640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa00051d0 0x7f0aa000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ab3e00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa0004810 0x7f0aa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac0a00640 1 -- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa0004810 msgr2=0x7f0aa0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa0004810 0x7f0aa0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac0a00640 1 -- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa00051d0 msgr2=0x7f0aa000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa00051d0 0x7f0aa000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac0a00640 1 -- 172.21.15.87:0/2471799821 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa0003530 con 0x7f0aa00039c0 2024-04-25T10:16:47.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac1400640 1 -- 172.21.15.87:0/2471799821 reap_dead start 2024-04-25T10:16:47.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0aa00039c0 0x7f0aa0003de0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009ae0 tx=0x7f0aac01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f9548e4859bdf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0b2070 con 0x7f0aa00039c0 2024-04-25T10:16:47.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac0a5520 con 0x7f0aa00039c0 2024-04-25T10:16:47.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0abbe00640 1 -- 172.21.15.87:0/2471799821 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0096b0 con 0x7f0aa00039c0 2024-04-25T10:16:47.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac3400640 1 -- 172.21.15.87:0/2471799821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0aa00039c0 msgr2=0x7f0aa0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac3400640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0aa00039c0 0x7f0aa0003de0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009ae0 tx=0x7f0aac01ca70 comp rx=0 tx=0).stop 2024-04-25T10:16:47.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac3400640 1 -- 172.21.15.87:0/2471799821 shutdown_connections 2024-04-25T10:16:47.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac3400640 1 --2- 172.21.15.87:0/2471799821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0aa00039c0 0x7f0aa0003de0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.306+0000 7f0ac3400640 1 -- 172.21.15.87:0/2471799821 >> 172.21.15.87:0/2471799821 conn(0x7f0abc09a9d0 msgr2=0x7f0abc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- 172.21.15.87:0/2471799821 shutdown_connections 2024-04-25T10:16:47.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- 172.21.15.87:0/2471799821 wait complete. 2024-04-25T10:16:47.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 Processor -- start 2024-04-25T10:16:47.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- start start 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa00039c0 0x7f0abc0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc154c90 0x7f0abc1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0abc0b66a0 0x7f0abc0b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0ce990 con 0x7f0abc0b66a0 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0d2220 con 0x7f0abc154c90 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0ceb10 con 0x7f0aa00039c0 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0abc0b66a0 0x7f0abc0b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0abc0b66a0 0x7f0abc0b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45802/0 (socket says 172.21.15.87:45802) 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 -- 172.21.15.87:0/3992372255 learned_addr learned my addr 172.21.15.87:0/3992372255 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:47.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa00039c0 0x7f0abc0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ab3e00640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc154c90 0x7f0abc1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 -- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa00039c0 msgr2=0x7f0abc0b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa00039c0 0x7f0abc0b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 -- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc154c90 msgr2=0x7f0abc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc154c90 0x7f0abc1586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 -- 172.21.15.87:0/3992372255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc158be0 con 0x7f0abc0b66a0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ab3e00640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc154c90 0x7f0abc1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac1400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0abc0b66a0 0x7f0abc0b6b20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0b1a00 tx=0x7f0ab4007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ab9600640 1 -- 172.21.15.87:0/3992372255 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400aaf0 con 0x7f0abc0b66a0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ab9600640 1 -- 172.21.15.87:0/3992372255 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4018070 con 0x7f0abc0b66a0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc0b54f0 con 0x7f0abc0b66a0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ab9600640 1 -- 172.21.15.87:0/3992372255 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40133f0 con 0x7f0abc0b66a0 2024-04-25T10:16:47.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.310+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc0b5ab0 con 0x7f0abc0b66a0 2024-04-25T10:16:47.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.314+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc04ef30 con 0x7f0abc0b66a0 2024-04-25T10:16:47.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.314+0000 7f0ab9600640 1 -- 172.21.15.87:0/3992372255 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ab4013590 con 0x7f0abc0b66a0 2024-04-25T10:16:47.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.314+0000 7f0ab9600640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a98066f90 0x7f0a98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.314+0000 7f0ab9600640 1 -- 172.21.15.87:0/3992372255 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0ab40851c0 con 0x7f0abc0b66a0 2024-04-25T10:16:47.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.314+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a98066f90 0x7f0a98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.314+0000 7f0ac0a00640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a98066f90 0x7f0a98069450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0af000 tx=0x7f0aac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.318+0000 7f0ab9600640 1 -- 172.21.15.87:0/3992372255 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ab4056480 con 0x7f0abc0b66a0 2024-04-25T10:16:47.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.474+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0abc155070 con 0x7f0abc0b66a0 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.474+0000 7f0ab9600640 1 -- 172.21.15.87:0/3992372255 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0ab4055d80 con 0x7f0abc0b66a0 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:16:47.483 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a98066f90 msgr2=0x7f0a98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a98066f90 0x7f0a98069450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0af000 tx=0x7f0aac008040 comp rx=0 tx=0).stop 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0abc0b66a0 msgr2=0x7f0abc0b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0abc0b66a0 0x7f0abc0b6b20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0b1a00 tx=0x7f0ab4007bd0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 shutdown_connections 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a98066f90 0x7f0a98069450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0abc0b66a0 0x7f0abc0b6b20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc154c90 0x7f0abc1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 --2- 172.21.15.87:0/3992372255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0aa00039c0 0x7f0abc0b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 >> 172.21.15.87:0/3992372255 conn(0x7f0abc09a9d0 msgr2=0x7f0abc15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 shutdown_connections 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.478+0000 7f0ac3400640 1 -- 172.21.15.87:0/3992372255 wait complete. 2024-04-25T10:16:47.503 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:16:47.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:16:47.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.634+0000 7f0c8d600640 1 Processor -- start 2024-04-25T10:16:47.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.634+0000 7f0c8d600640 1 -- start start 2024-04-25T10:16:47.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.634+0000 7f0c8d600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c88157cb0 0x7f0c880720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.634+0000 7f0c8d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c880b2af0 0x7f0c880b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.634+0000 7f0c8d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c88071040 con 0x7f0c88070ba0 2024-04-25T10:16:47.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c8d600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880af150 con 0x7f0c88157cb0 2024-04-25T10:16:47.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c8d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88071320 con 0x7f0c880b2af0 2024-04-25T10:16:47.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c87400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c880b2af0 0x7f0c880b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c86a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c88157cb0 0x7f0c880720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c7e000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f0c88070ba0 0x7f0c8815b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:45524/0 (socket says 172.21.15.87:45524) 2024-04-25T10:16:47.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c7e000640 1 -- 172.21.15.87:0/3468228916 learned_addr learned my addr 172.21.15.87:0/3468228916 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:47.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c86000640 1 -- 172.21.15.87:0/3468228916 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030525470 0 0) 0x7f0c88071040 con 0x7f0c88070ba0 2024-04-25T10:16:47.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c87400640 1 -- 172.21.15.87:0/3468228916 >> v2:172.21.15.169:3300/0 conn(0x7f0c88157cb0 msgr2=0x7f0c880720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c87400640 1 --2- 172.21.15.87:0/3468228916 >> v2:172.21.15.169:3300/0 conn(0x7f0c88157cb0 0x7f0c880720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c87400640 1 -- 172.21.15.87:0/3468228916 >> v1:172.21.15.169:6789/0 conn(0x7f0c88070ba0 legacy=0x7f0c8815b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c87400640 1 -- 172.21.15.87:0/3468228916 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c88071600 con 0x7f0c880b2af0 2024-04-25T10:16:47.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.638+0000 7f0c87400640 1 --2- 172.21.15.87:0/3468228916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c880b2af0 0x7f0c880b2f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c800097c0 tx=0x7f0c8001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e13b78968a1ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c86000640 1 -- 172.21.15.87:0/3468228916 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800b0070 con 0x7f0c880b2af0 2024-04-25T10:16:47.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c86000640 1 -- 172.21.15.87:0/3468228916 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c800ab600 con 0x7f0c880b2af0 2024-04-25T10:16:47.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c86000640 1 -- 172.21.15.87:0/3468228916 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800ab9d0 con 0x7f0c880b2af0 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- 172.21.15.87:0/3468228916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c880b2af0 msgr2=0x7f0c880b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 --2- 172.21.15.87:0/3468228916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c880b2af0 0x7f0c880b2f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c800097c0 tx=0x7f0c8001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- 172.21.15.87:0/3468228916 shutdown_connections 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 --2- 172.21.15.87:0/3468228916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c880b2af0 0x7f0c880b2f80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 --2- 172.21.15.87:0/3468228916 >> v2:172.21.15.169:3300/0 conn(0x7f0c88157cb0 0x7f0c880720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- 172.21.15.87:0/3468228916 >> 172.21.15.87:0/3468228916 conn(0x7f0c88066b70 msgr2=0x7f0c88066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- 172.21.15.87:0/3468228916 shutdown_connections 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- 172.21.15.87:0/3468228916 wait complete. 2024-04-25T10:16:47.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 Processor -- start 2024-04-25T10:16:47.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- start start 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c88070ba0 0x7f0c8814e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c880b2af0 0x7f0c8814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c88157cb0 0x7f0c8814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8806c260 con 0x7f0c880b2af0 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8806c0e0 con 0x7f0c88157cb0 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c8d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8806c3e0 con 0x7f0c88070ba0 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c7e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c880b2af0 0x7f0c8814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c7e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c880b2af0 0x7f0c8814eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60298/0 (socket says 172.21.15.87:60298) 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c7e000640 1 -- 172.21.15.87:0/2595561025 learned_addr learned my addr 172.21.15.87:0/2595561025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c86a00640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c88070ba0 0x7f0c8814e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.642+0000 7f0c87400640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c88157cb0 0x7f0c8814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7e000640 1 -- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c88070ba0 msgr2=0x7f0c8814e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7e000640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c88070ba0 0x7f0c8814e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7e000640 1 -- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c88157cb0 msgr2=0x7f0c8814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7e000640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c88157cb0 0x7f0c8814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7e000640 1 -- 172.21.15.87:0/2595561025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8815c590 con 0x7f0c880b2af0 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7e000640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c880b2af0 0x7f0c8814eda0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0c700011f0 tx=0x7f0c7000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7f400640 1 -- 172.21.15.87:0/2595561025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70018070 con 0x7f0c880b2af0 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7f400640 1 -- 172.21.15.87:0/2595561025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c700074e0 con 0x7f0c880b2af0 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8815c870 con 0x7f0c880b2af0 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7f400640 1 -- 172.21.15.87:0/2595561025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004a10 con 0x7f0c880b2af0 2024-04-25T10:16:47.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8815ce30 con 0x7f0c880b2af0 2024-04-25T10:16:47.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8804ef30 con 0x7f0c880b2af0 2024-04-25T10:16:47.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7f400640 1 -- 172.21.15.87:0/2595561025 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c70004bb0 con 0x7f0c880b2af0 2024-04-25T10:16:47.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7f400640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c64066f60 0x7f0c64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c7f400640 1 -- 172.21.15.87:0/2595561025 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0c70084110 con 0x7f0c880b2af0 2024-04-25T10:16:47.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.646+0000 7f0c86a00640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c64066f60 0x7f0c64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.650+0000 7f0c86a00640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c64066f60 0x7f0c64069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800a490 tx=0x7f0c7800a860 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.650+0000 7f0c7f400640 1 -- 172.21.15.87:0/2595561025 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c70054d10 con 0x7f0c880b2af0 2024-04-25T10:16:47.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:47.806+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0c88070430 con 0x7f0c880b2af0 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.102+0000 7f0c7f400640 1 -- 172.21.15.87:0/2595561025 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0c70058db0 con 0x7f0c880b2af0 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c64066f60 msgr2=0x7f0c64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c64066f60 0x7f0c64069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800a490 tx=0x7f0c7800a860 comp rx=0 tx=0).stop 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c880b2af0 msgr2=0x7f0c8814eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c880b2af0 0x7f0c8814eda0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0c700011f0 tx=0x7f0c7000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 shutdown_connections 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c64066f60 0x7f0c64069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c88157cb0 0x7f0c8814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c880b2af0 0x7f0c8814eda0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 --2- 172.21.15.87:0/2595561025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c88070ba0 0x7f0c8814e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 >> 172.21.15.87:0/2595561025 conn(0x7f0c88066b70 msgr2=0x7f0c8806d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 shutdown_connections 2024-04-25T10:16:48.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.106+0000 7f0c8d600640 1 -- 172.21.15.87:0/2595561025 wait complete. 2024-04-25T10:16:48.137 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:16:48.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f274a400640 1 Processor -- start 2024-04-25T10:16:48.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f274a400640 1 -- start start 2024-04-25T10:16:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f274a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f274415bbb0 0x7f27440cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f274a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f27440b4660 0x7f274406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f274a400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274406ca90 con 0x7f27440cf960 2024-04-25T10:16:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f274a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274406d050 con 0x7f27440b4660 2024-04-25T10:16:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f274a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274406cd70 con 0x7f274415bbb0 2024-04-25T10:16:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2743400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f274415bbb0 0x7f27440cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2743400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f274415bbb0 0x7f27440cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:59086/0 (socket says 172.21.15.87:59086) 2024-04-25T10:16:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2743400640 1 -- 172.21.15.87:0/2319421895 learned_addr learned my addr 172.21.15.87:0/2319421895 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2742a00640 1 --2- 172.21.15.87:0/2319421895 >> v2:172.21.15.169:3300/0 conn(0x7f27440b4660 0x7f274406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2743400640 1 -- 172.21.15.87:0/2319421895 >> v2:172.21.15.169:3300/0 conn(0x7f27440b4660 msgr2=0x7f274406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2742000640 1 -- 172.21.15.87:0/2319421895 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444042185 0 0) 0x7f274406ca90 con 0x7f27440cf960 2024-04-25T10:16:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2743400640 1 --2- 172.21.15.87:0/2319421895 >> v2:172.21.15.169:3300/0 conn(0x7f27440b4660 0x7f274406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2743400640 1 -- 172.21.15.87:0/2319421895 >> v1:172.21.15.169:6789/0 conn(0x7f27440cf960 legacy=0x7f27440b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.274+0000 7f2743400640 1 -- 172.21.15.87:0/2319421895 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440b2880 con 0x7f274415bbb0 2024-04-25T10:16:48.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f2743400640 1 --2- 172.21.15.87:0/2319421895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f274415bbb0 0x7f27440cda80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2734009870 tx=0x7f273401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f190f60d01d390ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f2742000640 1 -- 172.21.15.87:0/2319421895 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27340ab650 con 0x7f274415bbb0 2024-04-25T10:16:48.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f2742000640 1 -- 172.21.15.87:0/2319421895 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27340b0070 con 0x7f274415bbb0 2024-04-25T10:16:48.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f2742000640 1 -- 172.21.15.87:0/2319421895 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27340aba10 con 0x7f274415bbb0 2024-04-25T10:16:48.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 -- 172.21.15.87:0/2319421895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f274415bbb0 msgr2=0x7f27440cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 --2- 172.21.15.87:0/2319421895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f274415bbb0 0x7f27440cda80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2734009870 tx=0x7f273401c560 comp rx=0 tx=0).stop 2024-04-25T10:16:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 -- 172.21.15.87:0/2319421895 shutdown_connections 2024-04-25T10:16:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 --2- 172.21.15.87:0/2319421895 >> v2:172.21.15.169:3300/0 conn(0x7f27440b4660 0x7f274406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 --2- 172.21.15.87:0/2319421895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f274415bbb0 0x7f27440cda80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 -- 172.21.15.87:0/2319421895 >> 172.21.15.87:0/2319421895 conn(0x7f2744066b70 msgr2=0x7f2744066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 -- 172.21.15.87:0/2319421895 shutdown_connections 2024-04-25T10:16:48.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.278+0000 7f274a400640 1 -- 172.21.15.87:0/2319421895 wait complete. 2024-04-25T10:16:48.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 Processor -- start 2024-04-25T10:16:48.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 -- start start 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27440b4660 0x7f2744151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27440cf960 0x7f274415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f274415bbb0 0x7f2744150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440ce140 con 0x7f274415bbb0 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440cdfc0 con 0x7f27440b4660 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274406c660 con 0x7f27440cf960 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f274415bbb0 0x7f2744150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f274415bbb0 0x7f2744150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60306/0 (socket says 172.21.15.87:60306) 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 -- 172.21.15.87:0/23350808 learned_addr learned my addr 172.21.15.87:0/23350808 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:48.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2742a00640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27440cf960 0x7f274415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27440b4660 0x7f2744151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 -- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27440cf960 msgr2=0x7f274415d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27440cf960 0x7f274415d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 -- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27440b4660 msgr2=0x7f2744151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27440b4660 0x7f2744151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 -- 172.21.15.87:0/23350808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744072390 con 0x7f274415bbb0 2024-04-25T10:16:48.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f2743e00640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f274415bbb0 0x7f2744150910 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c780 tx=0x7f273c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f273b400640 1 -- 172.21.15.87:0/23350808 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c003db0 con 0x7f274415bbb0 2024-04-25T10:16:48.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f273b400640 1 -- 172.21.15.87:0/23350808 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c00ce30 con 0x7f274415bbb0 2024-04-25T10:16:48.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f273b400640 1 -- 172.21.15.87:0/23350808 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c005290 con 0x7f274415bbb0 2024-04-25T10:16:48.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2744152290 con 0x7f274415bbb0 2024-04-25T10:16:48.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.282+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27440720b0 con 0x7f274415bbb0 2024-04-25T10:16:48.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.286+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274404ef30 con 0x7f274415bbb0 2024-04-25T10:16:48.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.286+0000 7f273b400640 1 -- 172.21.15.87:0/23350808 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f273c005430 con 0x7f274415bbb0 2024-04-25T10:16:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.286+0000 7f273b400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2724066f60 0x7f2724069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.286+0000 7f273b400640 1 -- 172.21.15.87:0/23350808 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f273c085720 con 0x7f274415bbb0 2024-04-25T10:16:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.286+0000 7f2743400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2724066f60 0x7f2724069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.286+0000 7f2743400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2724066f60 0x7f2724069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f27340af000 tx=0x7f27340a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.290+0000 7f273b400640 1 -- 172.21.15.87:0/23350808 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f273c056210 con 0x7f274415bbb0 2024-04-25T10:16:48.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:48.446+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f27440cda40 con 0x7f274415bbb0 2024-04-25T10:16:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.110+0000 7f273b400640 1 -- 172.21.15.87:0/23350808 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f273c05a2b0 con 0x7f274415bbb0 2024-04-25T10:16:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:16:49.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2724066f60 msgr2=0x7f2724069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2724066f60 0x7f2724069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f27340af000 tx=0x7f27340a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:49.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f274415bbb0 msgr2=0x7f2744150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f274415bbb0 0x7f2744150910 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c780 tx=0x7f273c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:16:49.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 shutdown_connections 2024-04-25T10:16:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2724066f60 0x7f2724069420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f274415bbb0 0x7f2744150910 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27440cf960 0x7f274415d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 --2- 172.21.15.87:0/23350808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27440b4660 0x7f2744151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 >> 172.21.15.87:0/23350808 conn(0x7f2744066b70 msgr2=0x7f27440b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 shutdown_connections 2024-04-25T10:16:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.114+0000 7f274a400640 1 -- 172.21.15.87:0/23350808 wait complete. 2024-04-25T10:16:49.158 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.290+0000 7f4d68e00640 1 Processor -- start 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.290+0000 7f4d68e00640 1 -- start start 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.290+0000 7f4d68e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d640d0d00 0x7f4d640ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.290+0000 7f4d68e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d6406cf40 0x7f4d6406d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.290+0000 7f4d68e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d640cdab0 con 0x7f4d640cc200 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.290+0000 7f4d68e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6406d8b0 con 0x7f4d6406cf40 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.290+0000 7f4d68e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640cd210 con 0x7f4d640d0d00 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d62800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d6406cf40 0x7f4d6406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d61400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4d640cc200 0x7f4d64162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36284/0 (socket says 172.21.15.87:36284) 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d62800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d6406cf40 0x7f4d6406d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60318/0 (socket says 172.21.15.87:60318) 2024-04-25T10:16:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d61400640 1 -- 172.21.15.87:0/2299333162 learned_addr learned my addr 172.21.15.87:0/2299333162 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d61e00640 1 --2- 172.21.15.87:0/2299333162 >> v2:172.21.15.169:3300/0 conn(0x7f4d640d0d00 0x7f4d640ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d62800640 1 -- 172.21.15.87:0/2299333162 >> v2:172.21.15.169:3300/0 conn(0x7f4d640d0d00 msgr2=0x7f4d640ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d62800640 1 --2- 172.21.15.87:0/2299333162 >> v2:172.21.15.169:3300/0 conn(0x7f4d640d0d00 0x7f4d640ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d62800640 1 -- 172.21.15.87:0/2299333162 >> v1:172.21.15.87:6789/0 conn(0x7f4d640cc200 legacy=0x7f4d64162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d62800640 1 -- 172.21.15.87:0/2299333162 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640cd4f0 con 0x7f4d6406cf40 2024-04-25T10:16:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d62800640 1 --2- 172.21.15.87:0/2299333162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d6406d370 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4d640cdd40 tx=0x7f4d5c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d683f3f788078bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d60a00640 1 -- 172.21.15.87:0/2299333162 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0b0070 con 0x7f4d6406cf40 2024-04-25T10:16:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d60a00640 1 -- 172.21.15.87:0/2299333162 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c0ab600 con 0x7f4d6406cf40 2024-04-25T10:16:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.294+0000 7f4d60a00640 1 -- 172.21.15.87:0/2299333162 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0ab9d0 con 0x7f4d6406cf40 2024-04-25T10:16:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- 172.21.15.87:0/2299333162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 msgr2=0x7f4d6406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2299333162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d6406d370 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4d640cdd40 tx=0x7f4d5c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- 172.21.15.87:0/2299333162 shutdown_connections 2024-04-25T10:16:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2299333162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d6406d370 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2299333162 >> v2:172.21.15.169:3300/0 conn(0x7f4d640d0d00 0x7f4d640ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- 172.21.15.87:0/2299333162 >> 172.21.15.87:0/2299333162 conn(0x7f4d64072290 msgr2=0x7f4d640726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- 172.21.15.87:0/2299333162 shutdown_connections 2024-04-25T10:16:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- 172.21.15.87:0/2299333162 wait complete. 2024-04-25T10:16:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 Processor -- start 2024-04-25T10:16:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- start start 2024-04-25T10:16:49.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d640996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cc200 0x7f4d64159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d0d00 0x7f4d640c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64162a10 con 0x7f4d6406cf40 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64162890 con 0x7f4d640d0d00 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d68e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6415e040 con 0x7f4d640cc200 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d640996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d640996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60324/0 (socket says 172.21.15.87:60324) 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 -- 172.21.15.87:0/2567368011 learned_addr learned my addr 172.21.15.87:0/2567368011 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d62800640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d0d00 0x7f4d640c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61400640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cc200 0x7f4d64159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 -- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cc200 msgr2=0x7f4d64159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cc200 0x7f4d64159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 -- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d0d00 msgr2=0x7f4d640c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d0d00 0x7f4d640c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 -- 172.21.15.87:0/2567368011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6415a650 con 0x7f4d6406cf40 2024-04-25T10:16:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61400640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cc200 0x7f4d64159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.298+0000 7f4d61e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d640996e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4d54007e40 tx=0x7f4d5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d5a000640 1 -- 172.21.15.87:0/2567368011 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54014620 con 0x7f4d6406cf40 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d5a000640 1 -- 172.21.15.87:0/2567368011 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54019070 con 0x7f4d6406cf40 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d5a000640 1 -- 172.21.15.87:0/2567368011 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54014a00 con 0x7f4d6406cf40 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6415a930 con 0x7f4d6406cf40 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6415aef0 con 0x7f4d6406cf40 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6404ef30 con 0x7f4d6406cf40 2024-04-25T10:16:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d5a000640 1 -- 172.21.15.87:0/2567368011 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d54014ba0 con 0x7f4d6406cf40 2024-04-25T10:16:49.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d5a000640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d44066f60 0x7f4d44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d5a000640 1 -- 172.21.15.87:0/2567368011 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4d54086760 con 0x7f4d6406cf40 2024-04-25T10:16:49.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.302+0000 7f4d61400640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d44066f60 0x7f4d44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.306+0000 7f4d61400640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d44066f60 0x7f4d44069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4d50007fd0 tx=0x7f4d50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.306+0000 7f4d5a000640 1 -- 172.21.15.87:0/2567368011 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d5401b310 con 0x7f4d6406cf40 2024-04-25T10:16:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:49.494+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4d640729f0 con 0x7f4d6406cf40 2024-04-25T10:16:50.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.130+0000 7f4d5a000640 1 -- 172.21.15.87:0/2567368011 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4d540577b0 con 0x7f4d6406cf40 2024-04-25T10:16:50.137 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T10:16:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d44066f60 msgr2=0x7f4d44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d44066f60 0x7f4d44069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4d50007fd0 tx=0x7f4d50009040 comp rx=0 tx=0).stop 2024-04-25T10:16:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 msgr2=0x7f4d640996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d640996e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4d54007e40 tx=0x7f4d5400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 shutdown_connections 2024-04-25T10:16:50.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d44066f60 0x7f4d44069420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d0d00 0x7f4d640c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cc200 0x7f4d64159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 --2- 172.21.15.87:0/2567368011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d6406cf40 0x7f4d640996e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.134+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 >> 172.21.15.87:0/2567368011 conn(0x7f4d64072290 msgr2=0x7f4d640ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.138+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 shutdown_connections 2024-04-25T10:16:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.138+0000 7f4d68e00640 1 -- 172.21.15.87:0/2567368011 wait complete. 2024-04-25T10:16:50.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:50.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.302+0000 7fd216a00640 1 Processor -- start 2024-04-25T10:16:50.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.302+0000 7fd216a00640 1 -- start start 2024-04-25T10:16:50.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd216a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2100d0040 0x7fd21015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd216a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd210071050 con 0x7fd2101550b0 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd216a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21015d040 con 0x7fd21015b740 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd216a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2100712f0 con 0x7fd2100d0040 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2100d0040 0x7fd21015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2100d0040 0x7fd21015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60332/0 (socket says 172.21.15.87:60332) 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20f400640 1 -- 172.21.15.87:0/2927200025 learned_addr learned my addr 172.21.15.87:0/2927200025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20fe00640 1 --1- 172.21.15.87:0/2927200025 >> v1:172.21.15.87:6789/0 conn(0x7fd2101550b0 0x7fd2100cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36288/0 (socket says 172.21.15.87:36288) 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20f400640 1 -- 172.21.15.87:0/2927200025 >> v1:172.21.15.169:6789/0 conn(0x7fd21015b740 legacy=0x7fd2100b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20f400640 1 -- 172.21.15.87:0/2927200025 >> v1:172.21.15.87:6789/0 conn(0x7fd2101550b0 legacy=0x7fd2100cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20ea00640 1 -- 172.21.15.87:0/2927200025 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194487289 0 0) 0x7fd210071050 con 0x7fd2101550b0 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.306+0000 7fd20f400640 1 -- 172.21.15.87:0/2927200025 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2100715d0 con 0x7fd2100d0040 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd20f400640 1 --2- 172.21.15.87:0/2927200025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd21015b200 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00bb10 tx=0x7fd1fc01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6cd7ff69ff93c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd20ea00640 1 -- 172.21.15.87:0/2927200025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0b4070 con 0x7fd2100d0040 2024-04-25T10:16:50.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd20ea00640 1 -- 172.21.15.87:0/2927200025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc0a7bd0 con 0x7fd2100d0040 2024-04-25T10:16:50.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd20ea00640 1 -- 172.21.15.87:0/2927200025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0af440 con 0x7fd2100d0040 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 -- 172.21.15.87:0/2927200025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 msgr2=0x7fd21015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 --2- 172.21.15.87:0/2927200025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd21015b200 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00bb10 tx=0x7fd1fc01ec00 comp rx=0 tx=0).stop 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 -- 172.21.15.87:0/2927200025 shutdown_connections 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 --2- 172.21.15.87:0/2927200025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd21015b200 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 -- 172.21.15.87:0/2927200025 >> 172.21.15.87:0/2927200025 conn(0x7fd210066b70 msgr2=0x7fd210066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 -- 172.21.15.87:0/2927200025 shutdown_connections 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 -- 172.21.15.87:0/2927200025 wait complete. 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 Processor -- start 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.310+0000 7fd216a00640 1 -- start start 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd210158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2101550b0 0x7fd210150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21015b740 0x7fd21006c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015bcd0 con 0x7fd2100d0040 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015bb50 con 0x7fd2101550b0 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21006c090 con 0x7fd21015b740 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd210158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd210158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60340/0 (socket says 172.21.15.87:60340) 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 -- 172.21.15.87:0/1616235616 learned_addr learned my addr 172.21.15.87:0/1616235616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20f400640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2101550b0 0x7fd210150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd214a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21015b740 0x7fd21006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 -- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21015b740 msgr2=0x7fd21006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21015b740 0x7fd21006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 -- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2101550b0 msgr2=0x7fd210150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2101550b0 0x7fd210150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 -- 172.21.15.87:0/1616235616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210151040 con 0x7fd2100d0040 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd214a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21015b740 0x7fd21006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd20fe00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd210158ae0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c4a0 tx=0x7fd20000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd207e00640 1 -- 172.21.15.87:0/1616235616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200018070 con 0x7fd2100d0040 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd207e00640 1 -- 172.21.15.87:0/1616235616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2000074e0 con 0x7fd2100d0040 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd210153b50 con 0x7fd2100d0040 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd207e00640 1 -- 172.21.15.87:0/1616235616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200004a30 con 0x7fd2100d0040 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210154110 con 0x7fd2100d0040 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21014eec0 con 0x7fd2100d0040 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.314+0000 7fd207e00640 1 -- 172.21.15.87:0/1616235616 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd20001b070 con 0x7fd2100d0040 2024-04-25T10:16:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.318+0000 7fd207e00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd1f0066f60 0x7fd1f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.318+0000 7fd207e00640 1 -- 172.21.15.87:0/1616235616 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2000841b0 con 0x7fd2100d0040 2024-04-25T10:16:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.318+0000 7fd20f400640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd1f0066f60 0x7fd1f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.318+0000 7fd20f400640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd1f0066f60 0x7fd1f0069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b1000 tx=0x7fd1fc0a9070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.322+0000 7fd207e00640 1 -- 172.21.15.87:0/1616235616 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd200054c00 con 0x7fd2100d0040 2024-04-25T10:16:50.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.474+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd21004ef30 con 0x7fd2100d0040 2024-04-25T10:16:50.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.474+0000 7fd207e00640 1 -- 172.21.15.87:0/1616235616 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16669 (secure 0 0 0) 0x7fd200058ca0 con 0x7fd2100d0040 2024-04-25T10:16:50.483 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:50.483 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:49.500520+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:47.854894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:48.494317+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd1f0066f60 msgr2=0x7fd1f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd1f0066f60 0x7fd1f0069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b1000 tx=0x7fd1fc0a9070 comp rx=0 tx=0).stop 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 msgr2=0x7fd210158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd210158ae0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c4a0 tx=0x7fd20000c960 comp rx=0 tx=0).stop 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 shutdown_connections 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd1f0066f60 0x7fd1f0069420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21015b740 0x7fd21006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2101550b0 0x7fd210150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 --2- 172.21.15.87:0/1616235616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2100d0040 0x7fd210158ae0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 >> 172.21.15.87:0/1616235616 conn(0x7fd210066b70 msgr2=0x7fd2100cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 shutdown_connections 2024-04-25T10:16:50.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.478+0000 7fd216a00640 1 -- 172.21.15.87:0/1616235616 wait complete. 2024-04-25T10:16:50.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:50.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f498ba00640 1 Processor -- start 2024-04-25T10:16:50.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f498ba00640 1 -- start start 2024-04-25T10:16:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f498ba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f498415bc40 0x7f498406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f498ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f49840b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f498ba00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49840b2770 con 0x7f49840cf930 2024-04-25T10:16:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f498ba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498406c670 con 0x7f498415bc40 2024-04-25T10:16:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f498ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498406c950 con 0x7f49840cda10 2024-04-25T10:16:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f4989000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f498415bc40 0x7f498406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f4989000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f498415bc40 0x7f498406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60346/0 (socket says 172.21.15.87:60346) 2024-04-25T10:16:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.630+0000 7f4989000640 1 -- 172.21.15.87:0/2210211951 learned_addr learned my addr 172.21.15.87:0/2210211951 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f497be00640 1 --2- 172.21.15.87:0/2210211951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f49840b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4989000640 1 -- 172.21.15.87:0/2210211951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 msgr2=0x7f49840b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4989000640 1 --2- 172.21.15.87:0/2210211951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f49840b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4989000640 1 -- 172.21.15.87:0/2210211951 >> v1:172.21.15.169:6789/0 conn(0x7f49840cf930 legacy=0x7f498415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4989000640 1 -- 172.21.15.87:0/2210211951 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498406cc30 con 0x7f498415bc40 2024-04-25T10:16:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f497be00640 1 --2- 172.21.15.87:0/2210211951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f49840b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4989000640 1 --2- 172.21.15.87:0/2210211951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f498415bc40 0x7f498406c090 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4974009870 tx=0x7f497401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92267553ed2a47bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4983e00640 1 -- 172.21.15.87:0/2210211951 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740a7070 con 0x7f498415bc40 2024-04-25T10:16:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4983e00640 1 -- 172.21.15.87:0/2210211951 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49740b0070 con 0x7f498415bc40 2024-04-25T10:16:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f4983e00640 1 -- 172.21.15.87:0/2210211951 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740ab890 con 0x7f498415bc40 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f498ba00640 1 -- 172.21.15.87:0/2210211951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f498415bc40 msgr2=0x7f498406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f498ba00640 1 --2- 172.21.15.87:0/2210211951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f498415bc40 0x7f498406c090 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4974009870 tx=0x7f497401c560 comp rx=0 tx=0).stop 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f498ba00640 1 -- 172.21.15.87:0/2210211951 shutdown_connections 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f498ba00640 1 --2- 172.21.15.87:0/2210211951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f49840b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f498ba00640 1 --2- 172.21.15.87:0/2210211951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f498415bc40 0x7f498406c090 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.634+0000 7f498ba00640 1 -- 172.21.15.87:0/2210211951 >> 172.21.15.87:0/2210211951 conn(0x7f4984066b70 msgr2=0x7f4984066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- 172.21.15.87:0/2210211951 shutdown_connections 2024-04-25T10:16:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- 172.21.15.87:0/2210211951 wait complete. 2024-04-25T10:16:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 Processor -- start 2024-04-25T10:16:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- start start 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f4984146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49840cf930 0x7f49840b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498415bc40 0x7f4984156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4984168b50 con 0x7f49840cf930 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49841689d0 con 0x7f498415bc40 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4984168cd0 con 0x7f49840cda10 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49840cf930 0x7f49840b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49840cf930 0x7f49840b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60348/0 (socket says 172.21.15.87:60348) 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 -- 172.21.15.87:0/1666862818 learned_addr learned my addr 172.21.15.87:0/1666862818 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f4989000640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f4984146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f4989a00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498415bc40 0x7f4984156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 -- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 msgr2=0x7f4984146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f4984146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 -- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498415bc40 msgr2=0x7f4984156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498415bc40 0x7f4984156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 -- 172.21.15.87:0/1666862818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498415d060 con 0x7f49840cf930 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f4989a00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498415bc40 0x7f4984156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f497be00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49840cf930 0x7f49840b0110 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f496c00d800 tx=0x7f496c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f4981600640 1 -- 172.21.15.87:0/1666862818 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c018020 con 0x7f49840cf930 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f4981600640 1 -- 172.21.15.87:0/1666862818 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c0096c0 con 0x7f49840cf930 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f4981600640 1 -- 172.21.15.87:0/1666862818 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c009ab0 con 0x7f49840cf930 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49840af630 con 0x7f49840cf930 2024-04-25T10:16:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.638+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4984153ae0 con 0x7f49840cf930 2024-04-25T10:16:50.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.642+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498404ef30 con 0x7f49840cf930 2024-04-25T10:16:50.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.642+0000 7f4981600640 1 -- 172.21.15.87:0/1666862818 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f496c01b020 con 0x7f49840cf930 2024-04-25T10:16:50.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.642+0000 7f4981600640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4960066f60 0x7f4960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.642+0000 7f4981600640 1 -- 172.21.15.87:0/1666862818 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f496c085550 con 0x7f49840cf930 2024-04-25T10:16:50.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.642+0000 7f4989000640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4960066f60 0x7f4960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.642+0000 7f4989000640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4960066f60 0x7f4960069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49740af000 tx=0x7f49740a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.646+0000 7f4981600640 1 -- 172.21.15.87:0/1666862818 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f496c022020 con 0x7f49840cf930 2024-04-25T10:16:50.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.798+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f498415b130 con 0x7f49840cf930 2024-04-25T10:16:50.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f4981600640 1 -- 172.21.15.87:0/1666862818 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f496c0564c0 con 0x7f49840cf930 2024-04-25T10:16:50.808 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:50.808 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:49.500520+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:47.854894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:48.494317+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:50.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4960066f60 msgr2=0x7f4960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4960066f60 0x7f4960069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49740af000 tx=0x7f49740a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:50.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49840cf930 msgr2=0x7f49840b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49840cf930 0x7f49840b0110 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f496c00d800 tx=0x7f496c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 shutdown_connections 2024-04-25T10:16:50.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4960066f60 0x7f4960069420 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f498415bc40 0x7f4984156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49840cf930 0x7f49840b0110 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 --2- 172.21.15.87:0/1666862818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49840cda10 0x7f4984146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 >> 172.21.15.87:0/1666862818 conn(0x7f4984066b70 msgr2=0x7f49840cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 shutdown_connections 2024-04-25T10:16:50.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.802+0000 7f498ba00640 1 -- 172.21.15.87:0/1666862818 wait complete. 2024-04-25T10:16:50.830 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:16:50.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2491400640 1 Processor -- start 2024-04-25T10:16:50.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2491400640 1 -- start start 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2491400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f248c06d650 0x7f248c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2491400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c0cdab0 con 0x7f248c0cc200 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2491400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c0cd210 con 0x7f248c0d0d00 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2491400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0cd4f0 con 0x7f248c06d650 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f248b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f248c06d650 0x7f248c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2489e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f248c0cc200 0x7f248c06cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36290/0 (socket says 172.21.15.87:36290) 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2489e00640 1 -- 172.21.15.87:0/580515190 learned_addr learned my addr 172.21.15.87:0/580515190 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f248b200640 1 -- 172.21.15.87:0/580515190 >> v1:172.21.15.169:6789/0 conn(0x7f248c0d0d00 legacy=0x7f248c162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f248b200640 1 -- 172.21.15.87:0/580515190 >> v1:172.21.15.87:6789/0 conn(0x7f248c0cc200 legacy=0x7f248c06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f248b200640 1 -- 172.21.15.87:0/580515190 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0cd7d0 con 0x7f248c06d650 2024-04-25T10:16:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.962+0000 7f2489400640 1 -- 172.21.15.87:0/580515190 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048203854 0 0) 0x7f248c0cdab0 con 0x7f248c0cc200 2024-04-25T10:16:50.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f248b200640 1 --2- 172.21.15.87:0/580515190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0ccc30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f24840097c0 tx=0x7f248401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cfab4a6aaffd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2489400640 1 -- 172.21.15.87:0/580515190 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24840b0070 con 0x7f248c06d650 2024-04-25T10:16:50.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2489400640 1 -- 172.21.15.87:0/580515190 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24840ab600 con 0x7f248c06d650 2024-04-25T10:16:50.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2489400640 1 -- 172.21.15.87:0/580515190 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24840ab9d0 con 0x7f248c06d650 2024-04-25T10:16:50.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2491400640 1 -- 172.21.15.87:0/580515190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 msgr2=0x7f248c0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2491400640 1 --2- 172.21.15.87:0/580515190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0ccc30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f24840097c0 tx=0x7f248401c410 comp rx=0 tx=0).stop 2024-04-25T10:16:50.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2491400640 1 -- 172.21.15.87:0/580515190 shutdown_connections 2024-04-25T10:16:50.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2491400640 1 --2- 172.21.15.87:0/580515190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0ccc30 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2491400640 1 -- 172.21.15.87:0/580515190 >> 172.21.15.87:0/580515190 conn(0x7f248c072290 msgr2=0x7f248c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2491400640 1 -- 172.21.15.87:0/580515190 shutdown_connections 2024-04-25T10:16:50.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.966+0000 7f2491400640 1 -- 172.21.15.87:0/580515190 wait complete. 2024-04-25T10:16:50.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 Processor -- start 2024-04-25T10:16:50.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 -- start start 2024-04-25T10:16:50.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f248c0cc200 0x7f248c1f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248c0d0d00 0x7f248c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c16c000 con 0x7f248c06d650 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c16be80 con 0x7f248c0d0d00 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c16c180 con 0x7f248c0cc200 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60370/0 (socket says 172.21.15.87:60370) 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 -- 172.21.15.87:0/2960548942 learned_addr learned my addr 172.21.15.87:0/2960548942 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248b200640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248c0d0d00 0x7f248c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2489e00640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f248c0cc200 0x7f248c1f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 -- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f248c0cc200 msgr2=0x7f248c1f9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f248c0cc200 0x7f248c1f9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 -- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248c0d0d00 msgr2=0x7f248c0c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248c0d0d00 0x7f248c0c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 -- 172.21.15.87:0/2960548942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c1fa800 con 0x7f248c06d650 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f248a800640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0c6710 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c7f0 tx=0x7f247c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2482a00640 1 -- 172.21.15.87:0/2960548942 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c003b80 con 0x7f248c06d650 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2482a00640 1 -- 172.21.15.87:0/2960548942 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c003d20 con 0x7f248c06d650 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2482a00640 1 -- 172.21.15.87:0/2960548942 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00aa90 con 0x7f248c06d650 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c1faae0 con 0x7f248c06d650 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c1fa3d0 con 0x7f248c06d650 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.970+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c04ef30 con 0x7f248c06d650 2024-04-25T10:16:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.974+0000 7f2482a00640 1 -- 172.21.15.87:0/2960548942 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f247c00ac30 con 0x7f248c06d650 2024-04-25T10:16:50.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.974+0000 7f2482a00640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f246c066f60 0x7f246c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.974+0000 7f2482a00640 1 -- 172.21.15.87:0/2960548942 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f247c084ed0 con 0x7f248c06d650 2024-04-25T10:16:50.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.974+0000 7f2489e00640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f246c066f60 0x7f246c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.974+0000 7f2489e00640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f246c066f60 0x7f246c069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f248c0ca3f0 tx=0x7f2478007240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:50.978+0000 7f2482a00640 1 -- 172.21.15.87:0/2960548942 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f247c055880 con 0x7f248c06d650 2024-04-25T10:16:51.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:51.166+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f248c072ff0 con 0x7f248c06d650 2024-04-25T10:16:52.041 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:52.044 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:52.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:16:52.111 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T10:16:52.110+0000 7f6384000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:52.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.138+0000 7f2482a00640 1 -- 172.21.15.87:0/2960548942 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f247c059920 con 0x7f248c06d650 2024-04-25T10:16:52.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f246c066f60 msgr2=0x7f246c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f246c066f60 0x7f246c069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f248c0ca3f0 tx=0x7f2478007240 comp rx=0 tx=0).stop 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 msgr2=0x7f248c0c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0c6710 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c7f0 tx=0x7f247c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 shutdown_connections 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f246c066f60 0x7f246c069420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248c0d0d00 0x7f248c0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f248c0cc200 0x7f248c1f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 --2- 172.21.15.87:0/2960548942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c06d650 0x7f248c0c6710 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 >> 172.21.15.87:0/2960548942 conn(0x7f248c072290 msgr2=0x7f248c0cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 shutdown_connections 2024-04-25T10:16:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.142+0000 7f2491400640 1 -- 172.21.15.87:0/2960548942 wait complete. 2024-04-25T10:16:52.168 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:16:52.168 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:16:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.298+0000 7ff5c7e00640 1 Processor -- start 2024-04-25T10:16:52.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.298+0000 7ff5c7e00640 1 -- start start 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5c8071260 0x7ff5c80716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c8071be0 0x7ff5c80b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c806de60 con 0x7ff5c815a780 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c806cdc0 con 0x7ff5c8071260 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c806d0a0 con 0x7ff5c8071be0 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5c8071260 0x7ff5c80716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5c8071260 0x7ff5c80716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60380/0 (socket says 172.21.15.87:60380) 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff5c815a780 0x7ff5c815edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36306/0 (socket says 172.21.15.87:36306) 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6000640 1 -- 172.21.15.87:0/1044598268 learned_addr learned my addr 172.21.15.87:0/1044598268 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c7400640 1 --2- 172.21.15.87:0/1044598268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c8071be0 0x7ff5c80b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6000640 1 -- 172.21.15.87:0/1044598268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c8071be0 msgr2=0x7ff5c80b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6000640 1 --2- 172.21.15.87:0/1044598268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c8071be0 0x7ff5c80b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6000640 1 -- 172.21.15.87:0/1044598268 >> v1:172.21.15.87:6789/0 conn(0x7ff5c815a780 legacy=0x7ff5c815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c6000640 1 -- 172.21.15.87:0/1044598268 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c806d380 con 0x7ff5c8071260 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.302+0000 7ff5c5600640 1 -- 172.21.15.87:0/1044598268 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679997784 0 0) 0x7ff5c806de60 con 0x7ff5c815a780 2024-04-25T10:16:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c6000640 1 --2- 172.21.15.87:0/1044598268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c80716a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009870 tx=0x7ff5b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c05e4265d7ff1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c5600640 1 -- 172.21.15.87:0/1044598268 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40b2070 con 0x7ff5c8071260 2024-04-25T10:16:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c5600640 1 -- 172.21.15.87:0/1044598268 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40a9070 con 0x7ff5c8071260 2024-04-25T10:16:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c5600640 1 -- 172.21.15.87:0/1044598268 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40ad880 con 0x7ff5c8071260 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- 172.21.15.87:0/1044598268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 msgr2=0x7ff5c80716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/1044598268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c80716a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009870 tx=0x7ff5b401c560 comp rx=0 tx=0).stop 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- 172.21.15.87:0/1044598268 shutdown_connections 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/1044598268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c8071be0 0x7ff5c80b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/1044598268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c80716a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- 172.21.15.87:0/1044598268 >> 172.21.15.87:0/1044598268 conn(0x7ff5c80743f0 msgr2=0x7ff5c80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- 172.21.15.87:0/1044598268 shutdown_connections 2024-04-25T10:16:52.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- 172.21.15.87:0/1044598268 wait complete. 2024-04-25T10:16:52.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 Processor -- start 2024-04-25T10:16:52.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- start start 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c8071be0 0x7ff5c8147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c815a780 0x7ff5c81489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b3330 con 0x7ff5c8071260 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b31b0 con 0x7ff5c8071be0 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b34b0 con 0x7ff5c815a780 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.306+0000 7ff5c6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c814a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60390/0 (socket says 172.21.15.87:60390) 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 -- 172.21.15.87:0/2889016245 learned_addr learned my addr 172.21.15.87:0/2889016245 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c7400640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c815a780 0x7ff5c81489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6000640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c8071be0 0x7ff5c8147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 -- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c815a780 msgr2=0x7ff5c81489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c815a780 0x7ff5c81489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 -- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c8071be0 msgr2=0x7ff5c8147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c8071be0 0x7ff5c8147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 -- 172.21.15.87:0/2889016245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8149340 con 0x7ff5c8071260 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c6a00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c814a3e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80011f0 tx=0x7ff5b800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5bea00640 1 -- 172.21.15.87:0/2889016245 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8018070 con 0x7ff5c8071260 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5bea00640 1 -- 172.21.15.87:0/2889016245 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b80074e0 con 0x7ff5c8071260 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5bea00640 1 -- 172.21.15.87:0/2889016245 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8004a10 con 0x7ff5c8071260 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c8149620 con 0x7ff5c8071260 2024-04-25T10:16:52.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c8149be0 con 0x7ff5c8071260 2024-04-25T10:16:52.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8075120 con 0x7ff5c8071260 2024-04-25T10:16:52.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5bea00640 1 -- 172.21.15.87:0/2889016245 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5b8004bb0 con 0x7ff5c8071260 2024-04-25T10:16:52.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5bea00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5a8066f60 0x7ff5a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.310+0000 7ff5bea00640 1 -- 172.21.15.87:0/2889016245 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5b80844a0 con 0x7ff5c8071260 2024-04-25T10:16:52.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.314+0000 7ff5c6000640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5a8066f60 0x7ff5a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.314+0000 7ff5c6000640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5a8066f60 0x7ff5a8069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40af000 tx=0x7ff5b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.318+0000 7ff5bea00640 1 -- 172.21.15.87:0/2889016245 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5b8055410 con 0x7ff5c8071260 2024-04-25T10:16:52.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:52.502+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff5c8156c20 con 0x7ff5c8071260 2024-04-25T10:16:53.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.142+0000 7ff5bea00640 1 -- 172.21.15.87:0/2889016245 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff5b8054d10 con 0x7ff5c8071260 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5a8066f60 msgr2=0x7ff5a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5a8066f60 0x7ff5a8069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40af000 tx=0x7ff5b4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 msgr2=0x7ff5c814a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c814a3e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80011f0 tx=0x7ff5b800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 shutdown_connections 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5a8066f60 0x7ff5a8069420 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c815a780 0x7ff5c81489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c8071be0 0x7ff5c8147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 --2- 172.21.15.87:0/2889016245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5c8071260 0x7ff5c814a3e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 >> 172.21.15.87:0/2889016245 conn(0x7ff5c80743f0 msgr2=0x7ff5c8159980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 shutdown_connections 2024-04-25T10:16:53.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.146+0000 7ff5c7e00640 1 -- 172.21.15.87:0/2889016245 wait complete. 2024-04-25T10:16:53.174 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:16:53.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523e00640 1 Processor -- start 2024-04-25T10:16:53.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523e00640 1 -- start start 2024-04-25T10:16:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f05240cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05240b2dd0 0x7f052406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052415f5e0 con 0x7f05240ce070 2024-04-25T10:16:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052415f8c0 con 0x7f05240b2dd0 2024-04-25T10:16:53.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052415fba0 con 0x7f05241451d0 2024-04-25T10:16:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05240b2dd0 0x7f052406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05240b2dd0 0x7f052406d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60394/0 (socket says 172.21.15.87:60394) 2024-04-25T10:16:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523400640 1 -- 172.21.15.87:0/2481366117 learned_addr learned my addr 172.21.15.87:0/2481366117 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0522a00640 1 --2- 172.21.15.87:0/2481366117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f05240cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523400640 1 -- 172.21.15.87:0/2481366117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 msgr2=0x7f05240cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523400640 1 --2- 172.21.15.87:0/2481366117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f05240cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523400640 1 -- 172.21.15.87:0/2481366117 >> v1:172.21.15.169:6789/0 conn(0x7f05240ce070 legacy=0x7f05240b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.306+0000 7f0523400640 1 -- 172.21.15.87:0/2481366117 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052415fe80 con 0x7f05240b2dd0 2024-04-25T10:16:53.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523400640 1 --2- 172.21.15.87:0/2481366117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240b2dd0 0x7f052406d2a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f051c009e50 tx=0x7f051c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca50502c144110bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0521600640 1 -- 172.21.15.87:0/2481366117 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0a5660 con 0x7f05240b2dd0 2024-04-25T10:16:53.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0521600640 1 -- 172.21.15.87:0/2481366117 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c0a5800 con 0x7f05240b2dd0 2024-04-25T10:16:53.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0521600640 1 -- 172.21.15.87:0/2481366117 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0a5bd0 con 0x7f05240b2dd0 2024-04-25T10:16:53.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 -- 172.21.15.87:0/2481366117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240b2dd0 msgr2=0x7f052406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 --2- 172.21.15.87:0/2481366117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240b2dd0 0x7f052406d2a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f051c009e50 tx=0x7f051c01c960 comp rx=0 tx=0).stop 2024-04-25T10:16:53.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 -- 172.21.15.87:0/2481366117 shutdown_connections 2024-04-25T10:16:53.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 --2- 172.21.15.87:0/2481366117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240b2dd0 0x7f052406d2a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 --2- 172.21.15.87:0/2481366117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f05240cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 -- 172.21.15.87:0/2481366117 >> 172.21.15.87:0/2481366117 conn(0x7f0524074310 msgr2=0x7f052406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 -- 172.21.15.87:0/2481366117 shutdown_connections 2024-04-25T10:16:53.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 -- 172.21.15.87:0/2481366117 wait complete. 2024-04-25T10:16:53.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 Processor -- start 2024-04-25T10:16:53.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.310+0000 7f0523e00640 1 -- start start 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05240b2dd0 0x7f052415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240ce070 0x7f052413e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f052415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052406e170 con 0x7f05240ce070 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052406dff0 con 0x7f05240b2dd0 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052406e2f0 con 0x7f05241451d0 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240ce070 0x7f052413e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240ce070 0x7f052413e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60396/0 (socket says 172.21.15.87:60396) 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 -- 172.21.15.87:0/65254010 learned_addr learned my addr 172.21.15.87:0/65254010 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523400640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f052415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522a00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05240b2dd0 0x7f052415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 -- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 msgr2=0x7f052415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f052415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 -- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05240b2dd0 msgr2=0x7f052415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05240b2dd0 0x7f052415aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 -- 172.21.15.87:0/65254010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052415e470 con 0x7f05240ce070 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0522000640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240ce070 0x7f052413e440 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f051000c450 tx=0x7f051000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f051aa00640 1 -- 172.21.15.87:0/65254010 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510012830 con 0x7f05240ce070 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f051aa00640 1 -- 172.21.15.87:0/65254010 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0510017070 con 0x7f05240ce070 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f051aa00640 1 -- 172.21.15.87:0/65254010 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510012c30 con 0x7f05240ce070 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052415e750 con 0x7f05240ce070 2024-04-25T10:16:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052413c170 con 0x7f05240ce070 2024-04-25T10:16:53.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.314+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052415e920 con 0x7f05240ce070 2024-04-25T10:16:53.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.318+0000 7f051aa00640 1 -- 172.21.15.87:0/65254010 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f051001c460 con 0x7f05240ce070 2024-04-25T10:16:53.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.318+0000 7f051aa00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0504066f60 0x7f0504069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.318+0000 7f051aa00640 1 -- 172.21.15.87:0/65254010 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0510084b80 con 0x7f05240ce070 2024-04-25T10:16:53.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.318+0000 7f0522a00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0504066f60 0x7f0504069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.318+0000 7f0522a00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0504066f60 0x7f0504069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0514007a60 tx=0x7f0514008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.322+0000 7f051aa00640 1 -- 172.21.15.87:0/65254010 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05100555d0 con 0x7f05240ce070 2024-04-25T10:16:53.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.510+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0524075120 con 0x7f05240ce070 2024-04-25T10:16:53.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f051aa00640 1 -- 172.21.15.87:0/65254010 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f0510059670 con 0x7f05240ce070 2024-04-25T10:16:53.519 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:53.519 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/817977905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":817977905},{"type":"v1","addr":"172.21.15.169:6835","nonce":817977905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/979883306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":979883306},{"type":"v1","addr":"172.21.15.87:6836","nonce":979883306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/796485952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":796485952},{"type":"v1","addr":"172.21.15.87:6837","nonce":796485952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:49.500890+0000","modified":"2024-04-25T10:16:52.508879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"d","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1643726385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1643726385},{"type":"v1","addr":"172.21.15.169:6834","nonce":1643726385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:16:53.519 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 53 2024-04-25T10:16:53.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0504066f60 msgr2=0x7f0504069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0504066f60 0x7f0504069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0514007a60 tx=0x7f0514008040 comp rx=0 tx=0).stop 2024-04-25T10:16:53.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240ce070 msgr2=0x7f052413e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240ce070 0x7f052413e440 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f051000c450 tx=0x7f051000c910 comp rx=0 tx=0).stop 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 shutdown_connections 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0504066f60 0x7f0504069420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05241451d0 0x7f052415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05240ce070 0x7f052413e440 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 --2- 172.21.15.87:0/65254010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05240b2dd0 0x7f052415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 >> 172.21.15.87:0/65254010 conn(0x7f0524074310 msgr2=0x7f05240b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 shutdown_connections 2024-04-25T10:16:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.514+0000 7f0523e00640 1 -- 172.21.15.87:0/65254010 wait complete. 2024-04-25T10:16:53.541 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:53.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.666+0000 7fcfe3e00640 1 Processor -- start 2024-04-25T10:16:53.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe3e00640 1 -- start start 2024-04-25T10:16:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfe4145310 0x7fcfe40cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcfe40d01b0 0x7fcfe40669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe4071440 con 0x7fcfe4066ee0 2024-04-25T10:16:53.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe3e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe4067310 con 0x7fcfe4145310 2024-04-25T10:16:53.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe4071720 con 0x7fcfe40d01b0 2024-04-25T10:16:53.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfe4145310 0x7fcfe40cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfe4145310 0x7fcfe40cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60412/0 (socket says 172.21.15.87:60412) 2024-04-25T10:16:53.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2a00640 1 -- 172.21.15.87:0/1395755070 learned_addr learned my addr 172.21.15.87:0/1395755070 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:53.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2000640 1 --2- 172.21.15.87:0/1395755070 >> v2:172.21.15.169:3300/0 conn(0x7fcfe40d01b0 0x7fcfe40669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2a00640 1 -- 172.21.15.87:0/1395755070 >> v2:172.21.15.169:3300/0 conn(0x7fcfe40d01b0 msgr2=0x7fcfe40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2a00640 1 --2- 172.21.15.87:0/1395755070 >> v2:172.21.15.169:3300/0 conn(0x7fcfe40d01b0 0x7fcfe40669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2a00640 1 -- 172.21.15.87:0/1395755070 >> v1:172.21.15.169:6789/0 conn(0x7fcfe4066ee0 legacy=0x7fcfe40b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.670+0000 7fcfe2a00640 1 -- 172.21.15.87:0/1395755070 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4071a00 con 0x7fcfe4145310 2024-04-25T10:16:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe2a00640 1 --2- 172.21.15.87:0/1395755070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe4145310 0x7fcfe40cfc70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4070f00 tx=0x7fcfd401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b66aed01723a7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe1600640 1 -- 172.21.15.87:0/1395755070 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40b0070 con 0x7fcfe4145310 2024-04-25T10:16:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe1600640 1 -- 172.21.15.87:0/1395755070 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd40ab600 con 0x7fcfe4145310 2024-04-25T10:16:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe1600640 1 -- 172.21.15.87:0/1395755070 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40ab9d0 con 0x7fcfe4145310 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 -- 172.21.15.87:0/1395755070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe4145310 msgr2=0x7fcfe40cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/1395755070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe4145310 0x7fcfe40cfc70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4070f00 tx=0x7fcfd401c520 comp rx=0 tx=0).stop 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 -- 172.21.15.87:0/1395755070 shutdown_connections 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/1395755070 >> v2:172.21.15.169:3300/0 conn(0x7fcfe40d01b0 0x7fcfe40669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/1395755070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe4145310 0x7fcfe40cfc70 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 -- 172.21.15.87:0/1395755070 >> 172.21.15.87:0/1395755070 conn(0x7fcfe40d0690 msgr2=0x7fcfe40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 -- 172.21.15.87:0/1395755070 shutdown_connections 2024-04-25T10:16:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 -- 172.21.15.87:0/1395755070 wait complete. 2024-04-25T10:16:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 Processor -- start 2024-04-25T10:16:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.674+0000 7fcfe3e00640 1 -- start start 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe4066ee0 0x7fcfe4143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe40d01b0 0x7fcfe415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe4145310 0x7fcfe4142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4168680 con 0x7fcfe40d01b0 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4168500 con 0x7fcfe4145310 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4168800 con 0x7fcfe4066ee0 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe40d01b0 0x7fcfe415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe40d01b0 0x7fcfe415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60422/0 (socket says 172.21.15.87:60422) 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 -- 172.21.15.87:0/3997520440 learned_addr learned my addr 172.21.15.87:0/3997520440 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2a00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe4066ee0 0x7fcfe4143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3400640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe4145310 0x7fcfe4142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 -- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe4066ee0 msgr2=0x7fcfe4143f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe4066ee0 0x7fcfe4143f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 -- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe4145310 msgr2=0x7fcfe4142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe4145310 0x7fcfe4142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 -- 172.21.15.87:0/3997520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe41444c0 con 0x7fcfe40d01b0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe2000640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe40d01b0 0x7fcfe415b1e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000d830 tx=0x7fcfd000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfdaa00640 1 -- 172.21.15.87:0/3997520440 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000b6c0 con 0x7fcfe40d01b0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfdaa00640 1 -- 172.21.15.87:0/3997520440 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0019070 con 0x7fcfe40d01b0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe4142ee0 con 0x7fcfe40d01b0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfdaa00640 1 -- 172.21.15.87:0/3997520440 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000bae0 con 0x7fcfe40d01b0 2024-04-25T10:16:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe4143780 con 0x7fcfe40d01b0 2024-04-25T10:16:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe4075120 con 0x7fcfe40d01b0 2024-04-25T10:16:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.678+0000 7fcfdaa00640 1 -- 172.21.15.87:0/3997520440 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfd001e460 con 0x7fcfe40d01b0 2024-04-25T10:16:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.682+0000 7fcfdaa00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfc4066f60 0x7fcfc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.682+0000 7fcfdaa00640 1 -- 172.21.15.87:0/3997520440 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfd00856d0 con 0x7fcfe40d01b0 2024-04-25T10:16:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.682+0000 7fcfe2a00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfc4066f60 0x7fcfc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.682+0000 7fcfe2a00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfc4066f60 0x7fcfc4069420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40ad000 tx=0x7fcfd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.686+0000 7fcfdaa00640 1 -- 172.21.15.87:0/3997520440 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfd00566d0 con 0x7fcfe40d01b0 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.838+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfe4140830 con 0x7fcfe40d01b0 2024-04-25T10:16:53.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.838+0000 7fcfdaa00640 1 -- 172.21.15.87:0/3997520440 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16656 (secure 0 0 0) 0x7fcfd0055fd0 con 0x7fcfe40d01b0 2024-04-25T10:16:53.847 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:53.847 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":72,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:16:51.129602+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:47.854894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:48.494317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfc4066f60 msgr2=0x7fcfc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfc4066f60 0x7fcfc4069420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40ad000 tx=0x7fcfd4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe40d01b0 msgr2=0x7fcfe415b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe40d01b0 0x7fcfe415b1e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000d830 tx=0x7fcfd000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 shutdown_connections 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcfc4066f60 0x7fcfc4069420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe4145310 0x7fcfe4142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfe40d01b0 0x7fcfe415b1e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 --2- 172.21.15.87:0/3997520440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe4066ee0 0x7fcfe4143f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 >> 172.21.15.87:0/3997520440 conn(0x7fcfe40d0690 msgr2=0x7fcfe415c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 shutdown_connections 2024-04-25T10:16:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.842+0000 7fcfe3e00640 1 -- 172.21.15.87:0/3997520440 wait complete. 2024-04-25T10:16:53.868 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:16:53.869 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7c2800640 1 Processor -- start 2024-04-25T10:16:54.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7c2800640 1 -- start start 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7c2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7bc15bc40 0x7fd7bc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7c2800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0b2770 con 0x7fd7bc0cf930 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7c2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc06c670 con 0x7fd7bc15bc40 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7c2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc06c950 con 0x7fd7bc0cda10 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7bbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd7bc0cf930 0x7fd7bc15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36310/0 (socket says 172.21.15.87:36310) 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7bbe00640 1 -- 172.21.15.87:0/3498663771 learned_addr learned my addr 172.21.15.87:0/3498663771 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7b3e00640 1 --2- 172.21.15.87:0/3498663771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7bb400640 1 --2- 172.21.15.87:0/3498663771 >> v2:172.21.15.169:3300/0 conn(0x7fd7bc15bc40 0x7fd7bc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7baa00640 1 -- 172.21.15.87:0/3498663771 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392399843 0 0) 0x7fd7bc0b2770 con 0x7fd7bc0cf930 2024-04-25T10:16:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7baa00640 1 -- 172.21.15.87:0/3498663771 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a0003530 con 0x7fd7bc0cf930 2024-04-25T10:16:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7b3e00640 1 -- 172.21.15.87:0/3498663771 >> v2:172.21.15.169:3300/0 conn(0x7fd7bc15bc40 msgr2=0x7fd7bc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7b3e00640 1 --2- 172.21.15.87:0/3498663771 >> v2:172.21.15.169:3300/0 conn(0x7fd7bc15bc40 0x7fd7bc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7baa00640 1 -- 172.21.15.87:0/3498663771 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4035661860 0 0) 0x7fd7a0003530 con 0x7fd7bc0cf930 2024-04-25T10:16:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7b3e00640 1 -- 172.21.15.87:0/3498663771 >> v1:172.21.15.87:6789/0 conn(0x7fd7bc0cf930 legacy=0x7fd7bc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7b3e00640 1 -- 172.21.15.87:0/3498663771 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc06cc30 con 0x7fd7bc0cda10 2024-04-25T10:16:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:53.998+0000 7fd7bb400640 1 --2- 172.21.15.87:0/3498663771 >> v2:172.21.15.169:3300/0 conn(0x7fd7bc15bc40 0x7fd7bc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:54.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7b3e00640 1 --2- 172.21.15.87:0/3498663771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc0b4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40097c0 tx=0x7fd7a401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4623b6da8d5794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7baa00640 1 -- 172.21.15.87:0/3498663771 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b0070 con 0x7fd7bc0cda10 2024-04-25T10:16:54.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7baa00640 1 -- 172.21.15.87:0/3498663771 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a40ab400 con 0x7fd7bc0cda10 2024-04-25T10:16:54.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7baa00640 1 -- 172.21.15.87:0/3498663771 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4001a00 con 0x7fd7bc0cda10 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7c2800640 1 -- 172.21.15.87:0/3498663771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 msgr2=0x7fd7bc0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7c2800640 1 --2- 172.21.15.87:0/3498663771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc0b4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40097c0 tx=0x7fd7a401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7c2800640 1 -- 172.21.15.87:0/3498663771 shutdown_connections 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7c2800640 1 --2- 172.21.15.87:0/3498663771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc0b4610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7c2800640 1 --2- 172.21.15.87:0/3498663771 >> v2:172.21.15.169:3300/0 conn(0x7fd7bc15bc40 0x7fd7bc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.002+0000 7fd7c2800640 1 -- 172.21.15.87:0/3498663771 >> 172.21.15.87:0/3498663771 conn(0x7fd7bc066b70 msgr2=0x7fd7bc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- 172.21.15.87:0/3498663771 shutdown_connections 2024-04-25T10:16:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- 172.21.15.87:0/3498663771 wait complete. 2024-04-25T10:16:54.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 Processor -- start 2024-04-25T10:16:54.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- start start 2024-04-25T10:16:54.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7bc0cf930 0x7fd7bc14fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7bc15bc40 0x7fd7bc15d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc168b50 con 0x7fd7bc0cf930 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1689d0 con 0x7fd7bc15bc40 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc168cd0 con 0x7fd7bc0cda10 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7bc0cf930 0x7fd7bc14fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7bc0cf930 0x7fd7bc14fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60428/0 (socket says 172.21.15.87:60428) 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 -- 172.21.15.87:0/1919211229 learned_addr learned my addr 172.21.15.87:0/1919211229 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 -- 172.21.15.87:0/1919211229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 msgr2=0x7fd7bc15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 -- 172.21.15.87:0/1919211229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7bc15bc40 msgr2=0x7fd7bc15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7bc15bc40 0x7fd7bc15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 -- 172.21.15.87:0/1919211229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc14ebb0 con 0x7fd7bc0cf930 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3e00640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7bc0cf930 0x7fd7bc14fc50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4019ff0 tx=0x7fd7a4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3400640 1 -- 172.21.15.87:0/1919211229 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b0040 con 0x7fd7bc0cf930 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3400640 1 -- 172.21.15.87:0/1919211229 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a40ba070 con 0x7fd7bc0cf930 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7b3400640 1 -- 172.21.15.87:0/1919211229 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b4900 con 0x7fd7bc0cf930 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc14ee90 con 0x7fd7bc0cf930 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.006+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc14f450 con 0x7fd7bc0cf930 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.010+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc04ef30 con 0x7fd7bc0cf930 2024-04-25T10:16:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.010+0000 7fd7b3400640 1 -- 172.21.15.87:0/1919211229 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7a40b3070 con 0x7fd7bc0cf930 2024-04-25T10:16:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.010+0000 7fd7b3400640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd798066f60 0x7fd798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.010+0000 7fd7b3400640 1 -- 172.21.15.87:0/1919211229 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7a411ea20 con 0x7fd7bc0cf930 2024-04-25T10:16:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.010+0000 7fd7bb400640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd798066f60 0x7fd798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.010+0000 7fd7bb400640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd798066f60 0x7fd798069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac006e90 tx=0x7fd7ac006de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.014+0000 7fd7b3400640 1 -- 172.21.15.87:0/1919211229 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7a4015fb0 con 0x7fd7bc0cf930 2024-04-25T10:16:54.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.170+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7bc0af1c0 con 0x7fd798066f60 2024-04-25T10:16:54.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.178+0000 7fd7b3400640 1 -- 172.21.15.87:0/1919211229 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7fd7bc0af1c0 con 0x7fd798066f60 2024-04-25T10:16:54.186 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:16:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd798066f60 msgr2=0x7fd798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd798066f60 0x7fd798069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac006e90 tx=0x7fd7ac006de0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7bc0cf930 msgr2=0x7fd7bc14fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7bc0cf930 0x7fd7bc14fc50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4019ff0 tx=0x7fd7a4019160 comp rx=0 tx=0).stop 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 shutdown_connections 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd798066f60 0x7fd798069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7bc15bc40 0x7fd7bc15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7bc0cf930 0x7fd7bc14fc50 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 --2- 172.21.15.87:0/1919211229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7bc0cda10 0x7fd7bc15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 >> 172.21.15.87:0/1919211229 conn(0x7fd7bc066b70 msgr2=0x7fd7bc154c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 shutdown_connections 2024-04-25T10:16:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.182+0000 7fd7c2800640 1 -- 172.21.15.87:0/1919211229 wait complete. 2024-04-25T10:16:54.206 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-04-25T10:16:53.182446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":325428,"kb_used_data":7988,"kb_used_omap":12,"kb_used_meta":317299,"kb_avail":754649292,"statfs":{"total":773094113280,"available":772760875008,"internally_reserved":0,"allocated":8179712,"data_stored":4123624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324914737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.072572"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162675+0000","last_change":"2024-04-25T10:16:50.157783+0000","last_active":"2024-04-25T10:16:50.162675+0000","last_peered":"2024-04-25T10:16:50.162675+0000","last_clean":"2024-04-25T10:16:50.162675+0000","last_became_active":"2024-04-25T10:16:50.154676+0000","last_became_peered":"2024-04-25T10:16:50.154676+0000","last_unstale":"2024-04-25T10:16:50.162675+0000","last_undegraded":"2024-04-25T10:16:50.162675+0000","last_fullsized":"2024-04-25T10:16:50.162675+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:45.314672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156657+0000","last_change":"2024-04-25T10:16:50.150575+0000","last_active":"2024-04-25T10:16:50.156657+0000","last_peered":"2024-04-25T10:16:50.156657+0000","last_clean":"2024-04-25T10:16:50.156657+0000","last_became_active":"2024-04-25T10:16:50.150204+0000","last_became_peered":"2024-04-25T10:16:50.150204+0000","last_unstale":"2024-04-25T10:16:50.156657+0000","last_undegraded":"2024-04-25T10:16:50.156657+0000","last_fullsized":"2024-04-25T10:16:50.156657+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:59.928760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.144882+0000","last_change":"2024-04-25T10:16:50.143838+0000","last_active":"2024-04-25T10:16:50.144882+0000","last_peered":"2024-04-25T10:16:50.144882+0000","last_clean":"2024-04-25T10:16:50.144882+0000","last_became_active":"2024-04-25T10:16:50.143537+0000","last_became_peered":"2024-04-25T10:16:50.143537+0000","last_unstale":"2024-04-25T10:16:50.144882+0000","last_undegraded":"2024-04-25T10:16:50.144882+0000","last_fullsized":"2024-04-25T10:16:50.144882+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:32.014814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142803+0000","last_change":"2024-04-25T10:16:50.150537+0000","last_active":"2024-04-25T10:16:51.142803+0000","last_peered":"2024-04-25T10:16:51.142803+0000","last_clean":"2024-04-25T10:16:51.142803+0000","last_became_active":"2024-04-25T10:16:50.150265+0000","last_became_peered":"2024-04-25T10:16:50.150265+0000","last_unstale":"2024-04-25T10:16:51.142803+0000","last_undegraded":"2024-04-25T10:16:51.142803+0000","last_fullsized":"2024-04-25T10:16:51.142803+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:44.626876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.141764+0000","last_change":"2024-04-25T10:16:50.138288+0000","last_active":"2024-04-25T10:16:50.141764+0000","last_peered":"2024-04-25T10:16:50.141764+0000","last_clean":"2024-04-25T10:16:50.141764+0000","last_became_active":"2024-04-25T10:16:50.136764+0000","last_became_peered":"2024-04-25T10:16:50.136764+0000","last_unstale":"2024-04-25T10:16:50.141764+0000","last_undegraded":"2024-04-25T10:16:50.141764+0000","last_fullsized":"2024-04-25T10:16:50.141764+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:11.147356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156462+0000","last_change":"2024-04-25T10:16:50.150517+0000","last_active":"2024-04-25T10:16:50.156462+0000","last_peered":"2024-04-25T10:16:50.156462+0000","last_clean":"2024-04-25T10:16:50.156462+0000","last_became_active":"2024-04-25T10:16:50.150108+0000","last_became_peered":"2024-04-25T10:16:50.150108+0000","last_unstale":"2024-04-25T10:16:50.156462+0000","last_undegraded":"2024-04-25T10:16:50.156462+0000","last_fullsized":"2024-04-25T10:16:50.156462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:19.977687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141285+0000","last_change":"2024-04-25T10:16:50.134190+0000","last_active":"2024-04-25T10:16:51.141285+0000","last_peered":"2024-04-25T10:16:51.141285+0000","last_clean":"2024-04-25T10:16:51.141285+0000","last_became_active":"2024-04-25T10:16:50.133921+0000","last_became_peered":"2024-04-25T10:16:50.133921+0000","last_unstale":"2024-04-25T10:16:51.141285+0000","last_undegraded":"2024-04-25T10:16:51.141285+0000","last_fullsized":"2024-04-25T10:16:51.141285+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156803+0000","last_change":"2024-04-25T10:16:50.152597+0000","last_active":"2024-04-25T10:16:50.156803+0000","last_peered":"2024-04-25T10:16:50.156803+0000","last_clean":"2024-04-25T10:16:50.156803+0000","last_became_active":"2024-04-25T10:16:50.152358+0000","last_became_peered":"2024-04-25T10:16:50.152358+0000","last_unstale":"2024-04-25T10:16:50.156803+0000","last_undegraded":"2024-04-25T10:16:50.156803+0000","last_fullsized":"2024-04-25T10:16:50.156803+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:30.704661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.138954+0000","last_change":"2024-04-25T10:16:50.137672+0000","last_active":"2024-04-25T10:16:50.138954+0000","last_peered":"2024-04-25T10:16:50.138954+0000","last_clean":"2024-04-25T10:16:50.138954+0000","last_became_active":"2024-04-25T10:16:50.135708+0000","last_became_peered":"2024-04-25T10:16:50.135708+0000","last_unstale":"2024-04-25T10:16:50.138954+0000","last_undegraded":"2024-04-25T10:16:50.138954+0000","last_fullsized":"2024-04-25T10:16:50.138954+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:34.045400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142552+0000","last_change":"2024-04-25T10:16:50.136848+0000","last_active":"2024-04-25T10:16:51.142552+0000","last_peered":"2024-04-25T10:16:51.142552+0000","last_clean":"2024-04-25T10:16:51.142552+0000","last_became_active":"2024-04-25T10:16:50.136612+0000","last_became_peered":"2024-04-25T10:16:50.136612+0000","last_unstale":"2024-04-25T10:16:51.142552+0000","last_undegraded":"2024-04-25T10:16:51.142552+0000","last_fullsized":"2024-04-25T10:16:51.142552+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:39.915784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162351+0000","last_change":"2024-04-25T10:16:50.152577+0000","last_active":"2024-04-25T10:16:50.162351+0000","last_peered":"2024-04-25T10:16:50.162351+0000","last_clean":"2024-04-25T10:16:50.162351+0000","last_became_active":"2024-04-25T10:16:50.152303+0000","last_became_peered":"2024-04-25T10:16:50.152303+0000","last_unstale":"2024-04-25T10:16:50.162351+0000","last_undegraded":"2024-04-25T10:16:50.162351+0000","last_fullsized":"2024-04-25T10:16:50.162351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142870+0000","last_change":"2024-04-25T10:16:50.137099+0000","last_active":"2024-04-25T10:16:51.142870+0000","last_peered":"2024-04-25T10:16:51.142870+0000","last_clean":"2024-04-25T10:16:51.142870+0000","last_became_active":"2024-04-25T10:16:50.133428+0000","last_became_peered":"2024-04-25T10:16:50.133428+0000","last_unstale":"2024-04-25T10:16:51.142870+0000","last_undegraded":"2024-04-25T10:16:51.142870+0000","last_fullsized":"2024-04-25T10:16:51.142870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:57.940222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144844+0000","last_change":"2024-04-25T10:16:50.147237+0000","last_active":"2024-04-25T10:16:51.144844+0000","last_peered":"2024-04-25T10:16:51.144844+0000","last_clean":"2024-04-25T10:16:51.144844+0000","last_became_active":"2024-04-25T10:16:50.146933+0000","last_became_peered":"2024-04-25T10:16:50.146933+0000","last_unstale":"2024-04-25T10:16:51.144844+0000","last_undegraded":"2024-04-25T10:16:51.144844+0000","last_fullsized":"2024-04-25T10:16:51.144844+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.133752+0000","last_change":"2024-04-25T10:16:50.132721+0000","last_active":"2024-04-25T10:16:50.133752+0000","last_peered":"2024-04-25T10:16:50.133752+0000","last_clean":"2024-04-25T10:16:50.133752+0000","last_became_active":"2024-04-25T10:16:50.132365+0000","last_became_peered":"2024-04-25T10:16:50.132365+0000","last_unstale":"2024-04-25T10:16:50.133752+0000","last_undegraded":"2024-04-25T10:16:50.133752+0000","last_fullsized":"2024-04-25T10:16:50.133752+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:50.809029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144627+0000","last_change":"2024-04-25T10:16:50.139389+0000","last_active":"2024-04-25T10:16:51.144627+0000","last_peered":"2024-04-25T10:16:51.144627+0000","last_clean":"2024-04-25T10:16:51.144627+0000","last_became_active":"2024-04-25T10:16:50.139084+0000","last_became_peered":"2024-04-25T10:16:50.139084+0000","last_unstale":"2024-04-25T10:16:51.144627+0000","last_undegraded":"2024-04-25T10:16:51.144627+0000","last_fullsized":"2024-04-25T10:16:51.144627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141946+0000","last_change":"2024-04-25T10:16:50.152313+0000","last_active":"2024-04-25T10:16:51.141946+0000","last_peered":"2024-04-25T10:16:51.141946+0000","last_clean":"2024-04-25T10:16:51.141946+0000","last_became_active":"2024-04-25T10:16:50.152050+0000","last_became_peered":"2024-04-25T10:16:50.152050+0000","last_unstale":"2024-04-25T10:16:51.141946+0000","last_undegraded":"2024-04-25T10:16:51.141946+0000","last_fullsized":"2024-04-25T10:16:51.141946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:07.867840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.141430+0000","last_change":"2024-04-25T10:16:50.138414+0000","last_active":"2024-04-25T10:16:50.141430+0000","last_peered":"2024-04-25T10:16:50.141430+0000","last_clean":"2024-04-25T10:16:50.141430+0000","last_became_active":"2024-04-25T10:16:50.137765+0000","last_became_peered":"2024-04-25T10:16:50.137765+0000","last_unstale":"2024-04-25T10:16:50.141430+0000","last_undegraded":"2024-04-25T10:16:50.141430+0000","last_fullsized":"2024-04-25T10:16:50.141430+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:53.551273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142953+0000","last_change":"2024-04-25T10:16:50.146423+0000","last_active":"2024-04-25T10:16:51.142953+0000","last_peered":"2024-04-25T10:16:51.142953+0000","last_clean":"2024-04-25T10:16:51.142953+0000","last_became_active":"2024-04-25T10:16:50.146065+0000","last_became_peered":"2024-04-25T10:16:50.146065+0000","last_unstale":"2024-04-25T10:16:51.142953+0000","last_undegraded":"2024-04-25T10:16:51.142953+0000","last_fullsized":"2024-04-25T10:16:51.142953+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:55.487397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156617+0000","last_change":"2024-04-25T10:16:50.150959+0000","last_active":"2024-04-25T10:16:50.156617+0000","last_peered":"2024-04-25T10:16:50.156617+0000","last_clean":"2024-04-25T10:16:50.156617+0000","last_became_active":"2024-04-25T10:16:50.150482+0000","last_became_peered":"2024-04-25T10:16:50.150482+0000","last_unstale":"2024-04-25T10:16:50.156617+0000","last_undegraded":"2024-04-25T10:16:50.156617+0000","last_fullsized":"2024-04-25T10:16:50.156617+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:21.611590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144765+0000","last_change":"2024-04-25T10:16:50.143763+0000","last_active":"2024-04-25T10:16:51.144765+0000","last_peered":"2024-04-25T10:16:51.144765+0000","last_clean":"2024-04-25T10:16:51.144765+0000","last_became_active":"2024-04-25T10:16:50.143172+0000","last_became_peered":"2024-04-25T10:16:50.143172+0000","last_unstale":"2024-04-25T10:16:51.144765+0000","last_undegraded":"2024-04-25T10:16:51.144765+0000","last_fullsized":"2024-04-25T10:16:51.144765+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:42.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142262+0000","last_change":"2024-04-25T10:16:50.145914+0000","last_active":"2024-04-25T10:16:51.142262+0000","last_peered":"2024-04-25T10:16:51.142262+0000","last_clean":"2024-04-25T10:16:51.142262+0000","last_became_active":"2024-04-25T10:16:50.145642+0000","last_became_peered":"2024-04-25T10:16:50.145642+0000","last_unstale":"2024-04-25T10:16:51.142262+0000","last_undegraded":"2024-04-25T10:16:51.142262+0000","last_fullsized":"2024-04-25T10:16:51.142262+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:24.864861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142243+0000","last_change":"2024-04-25T10:16:50.132627+0000","last_active":"2024-04-25T10:16:51.142243+0000","last_peered":"2024-04-25T10:16:51.142243+0000","last_clean":"2024-04-25T10:16:51.142243+0000","last_became_active":"2024-04-25T10:16:50.132374+0000","last_became_peered":"2024-04-25T10:16:50.132374+0000","last_unstale":"2024-04-25T10:16:51.142243+0000","last_undegraded":"2024-04-25T10:16:51.142243+0000","last_fullsized":"2024-04-25T10:16:51.142243+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:43.151695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.153900+0000","last_change":"2024-04-25T10:16:50.145664+0000","last_active":"2024-04-25T10:16:50.153900+0000","last_peered":"2024-04-25T10:16:50.153900+0000","last_clean":"2024-04-25T10:16:50.153900+0000","last_became_active":"2024-04-25T10:16:50.144920+0000","last_became_peered":"2024-04-25T10:16:50.144920+0000","last_unstale":"2024-04-25T10:16:50.153900+0000","last_undegraded":"2024-04-25T10:16:50.153900+0000","last_fullsized":"2024-04-25T10:16:50.153900+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:31.639211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162382+0000","last_change":"2024-04-25T10:16:50.154737+0000","last_active":"2024-04-25T10:16:50.162382+0000","last_peered":"2024-04-25T10:16:50.162382+0000","last_clean":"2024-04-25T10:16:50.162382+0000","last_became_active":"2024-04-25T10:16:50.154535+0000","last_became_peered":"2024-04-25T10:16:50.154535+0000","last_unstale":"2024-04-25T10:16:50.162382+0000","last_undegraded":"2024-04-25T10:16:50.162382+0000","last_fullsized":"2024-04-25T10:16:50.162382+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:32.249267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142562+0000","last_change":"2024-04-25T10:16:50.141328+0000","last_active":"2024-04-25T10:16:51.142562+0000","last_peered":"2024-04-25T10:16:51.142562+0000","last_clean":"2024-04-25T10:16:51.142562+0000","last_became_active":"2024-04-25T10:16:50.141129+0000","last_became_peered":"2024-04-25T10:16:50.141129+0000","last_unstale":"2024-04-25T10:16:51.142562+0000","last_undegraded":"2024-04-25T10:16:51.142562+0000","last_fullsized":"2024-04-25T10:16:51.142562+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141768+0000","last_change":"2024-04-25T10:16:50.146078+0000","last_active":"2024-04-25T10:16:51.141768+0000","last_peered":"2024-04-25T10:16:51.141768+0000","last_clean":"2024-04-25T10:16:51.141768+0000","last_became_active":"2024-04-25T10:16:50.145808+0000","last_became_peered":"2024-04-25T10:16:50.145808+0000","last_unstale":"2024-04-25T10:16:51.141768+0000","last_undegraded":"2024-04-25T10:16:51.141768+0000","last_fullsized":"2024-04-25T10:16:51.141768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.142569+0000","last_change":"2024-04-25T10:16:50.141629+0000","last_active":"2024-04-25T10:16:50.142569+0000","last_peered":"2024-04-25T10:16:50.142569+0000","last_clean":"2024-04-25T10:16:50.142569+0000","last_became_active":"2024-04-25T10:16:50.140963+0000","last_became_peered":"2024-04-25T10:16:50.140963+0000","last_unstale":"2024-04-25T10:16:50.142569+0000","last_undegraded":"2024-04-25T10:16:50.142569+0000","last_fullsized":"2024-04-25T10:16:50.142569+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:45.475155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142510+0000","last_change":"2024-04-25T10:16:50.139607+0000","last_active":"2024-04-25T10:16:51.142510+0000","last_peered":"2024-04-25T10:16:51.142510+0000","last_clean":"2024-04-25T10:16:51.142510+0000","last_became_active":"2024-04-25T10:16:50.139370+0000","last_became_peered":"2024-04-25T10:16:50.139370+0000","last_unstale":"2024-04-25T10:16:51.142510+0000","last_undegraded":"2024-04-25T10:16:51.142510+0000","last_fullsized":"2024-04-25T10:16:51.142510+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:31.639211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.131684+0000","last_change":"2024-04-25T10:16:50.130241+0000","last_active":"2024-04-25T10:16:50.131684+0000","last_peered":"2024-04-25T10:16:50.131684+0000","last_clean":"2024-04-25T10:16:50.131684+0000","last_became_active":"2024-04-25T10:16:50.129961+0000","last_became_peered":"2024-04-25T10:16:50.129961+0000","last_unstale":"2024-04-25T10:16:50.131684+0000","last_undegraded":"2024-04-25T10:16:50.131684+0000","last_fullsized":"2024-04-25T10:16:50.131684+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:23.158296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141398+0000","last_change":"2024-04-25T10:16:50.146359+0000","last_active":"2024-04-25T10:16:51.141398+0000","last_peered":"2024-04-25T10:16:51.141398+0000","last_clean":"2024-04-25T10:16:51.141398+0000","last_became_active":"2024-04-25T10:16:50.146091+0000","last_became_peered":"2024-04-25T10:16:50.146091+0000","last_unstale":"2024-04-25T10:16:51.141398+0000","last_undegraded":"2024-04-25T10:16:51.141398+0000","last_fullsized":"2024-04-25T10:16:51.141398+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141426+0000","last_change":"2024-04-25T10:16:50.137295+0000","last_active":"2024-04-25T10:16:51.141426+0000","last_peered":"2024-04-25T10:16:51.141426+0000","last_clean":"2024-04-25T10:16:51.141426+0000","last_became_active":"2024-04-25T10:16:50.136975+0000","last_became_peered":"2024-04-25T10:16:50.136975+0000","last_unstale":"2024-04-25T10:16:51.141426+0000","last_undegraded":"2024-04-25T10:16:51.141426+0000","last_fullsized":"2024-04-25T10:16:51.141426+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:46.713781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.155703+0000","last_change":"2024-04-25T10:16:50.149725+0000","last_active":"2024-04-25T10:16:50.155703+0000","last_peered":"2024-04-25T10:16:50.155703+0000","last_clean":"2024-04-25T10:16:50.155703+0000","last_became_active":"2024-04-25T10:16:50.147376+0000","last_became_peered":"2024-04-25T10:16:50.147376+0000","last_unstale":"2024-04-25T10:16:50.155703+0000","last_undegraded":"2024-04-25T10:16:50.155703+0000","last_fullsized":"2024-04-25T10:16:50.155703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:53.392334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162559+0000","last_change":"2024-04-25T10:16:50.154006+0000","last_active":"2024-04-25T10:16:50.162559+0000","last_peered":"2024-04-25T10:16:50.162559+0000","last_clean":"2024-04-25T10:16:50.162559+0000","last_became_active":"2024-04-25T10:16:50.153756+0000","last_became_peered":"2024-04-25T10:16:50.153756+0000","last_unstale":"2024-04-25T10:16:50.162559+0000","last_undegraded":"2024-04-25T10:16:50.162559+0000","last_fullsized":"2024-04-25T10:16:50.162559+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.168212+0000","last_change":"2024-04-25T10:16:49.144515+0000","last_active":"2024-04-25T10:16:50.168212+0000","last_peered":"2024-04-25T10:16:50.168212+0000","last_clean":"2024-04-25T10:16:50.168212+0000","last_became_active":"2024-04-25T10:16:49.144172+0000","last_became_peered":"2024-04-25T10:16:49.144172+0000","last_unstale":"2024-04-25T10:16:50.168212+0000","last_undegraded":"2024-04-25T10:16:50.168212+0000","last_fullsized":"2024-04-25T10:16:50.168212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.488016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.141406+0000","last_change":"2024-04-25T10:16:50.137944+0000","last_active":"2024-04-25T10:16:50.141406+0000","last_peered":"2024-04-25T10:16:50.141406+0000","last_clean":"2024-04-25T10:16:50.141406+0000","last_became_active":"2024-04-25T10:16:50.135661+0000","last_became_peered":"2024-04-25T10:16:50.135661+0000","last_unstale":"2024-04-25T10:16:50.141406+0000","last_undegraded":"2024-04-25T10:16:50.141406+0000","last_fullsized":"2024-04-25T10:16:50.141406+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:57.875349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144968+0000","last_change":"2024-04-25T10:16:49.134932+0000","last_active":"2024-04-25T10:16:51.144968+0000","last_peered":"2024-04-25T10:16:51.144968+0000","last_clean":"2024-04-25T10:16:51.144968+0000","last_became_active":"2024-04-25T10:16:49.134675+0000","last_became_peered":"2024-04-25T10:16:49.134675+0000","last_unstale":"2024-04-25T10:16:51.144968+0000","last_undegraded":"2024-04-25T10:16:51.144968+0000","last_fullsized":"2024-04-25T10:16:51.144968+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:51.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.151541+0000","last_change":"2024-04-25T10:16:50.146874+0000","last_active":"2024-04-25T10:16:50.151541+0000","last_peered":"2024-04-25T10:16:50.151541+0000","last_clean":"2024-04-25T10:16:50.151541+0000","last_became_active":"2024-04-25T10:16:50.146566+0000","last_became_peered":"2024-04-25T10:16:50.146566+0000","last_unstale":"2024-04-25T10:16:50.151541+0000","last_undegraded":"2024-04-25T10:16:50.151541+0000","last_fullsized":"2024-04-25T10:16:50.151541+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:57.940222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142485+0000","last_change":"2024-04-25T10:16:49.150795+0000","last_active":"2024-04-25T10:16:51.142485+0000","last_peered":"2024-04-25T10:16:51.142485+0000","last_clean":"2024-04-25T10:16:51.142485+0000","last_became_active":"2024-04-25T10:16:49.150281+0000","last_became_peered":"2024-04-25T10:16:49.150281+0000","last_unstale":"2024-04-25T10:16:51.142485+0000","last_undegraded":"2024-04-25T10:16:51.142485+0000","last_fullsized":"2024-04-25T10:16:51.142485+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:20.604829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162328+0000","last_change":"2024-04-25T10:16:50.152550+0000","last_active":"2024-04-25T10:16:50.162328+0000","last_peered":"2024-04-25T10:16:50.162328+0000","last_clean":"2024-04-25T10:16:50.162328+0000","last_became_active":"2024-04-25T10:16:50.152081+0000","last_became_peered":"2024-04-25T10:16:50.152081+0000","last_unstale":"2024-04-25T10:16:50.162328+0000","last_undegraded":"2024-04-25T10:16:50.162328+0000","last_fullsized":"2024-04-25T10:16:50.162328+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165876+0000","last_change":"2024-04-25T10:16:49.145670+0000","last_active":"2024-04-25T10:16:50.165876+0000","last_peered":"2024-04-25T10:16:50.165876+0000","last_clean":"2024-04-25T10:16:50.165876+0000","last_became_active":"2024-04-25T10:16:49.143322+0000","last_became_peered":"2024-04-25T10:16:49.143322+0000","last_unstale":"2024-04-25T10:16:50.165876+0000","last_undegraded":"2024-04-25T10:16:50.165876+0000","last_fullsized":"2024-04-25T10:16:50.165876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:03.154073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141769+0000","last_change":"2024-04-25T10:16:50.136771+0000","last_active":"2024-04-25T10:16:51.141769+0000","last_peered":"2024-04-25T10:16:51.141769+0000","last_clean":"2024-04-25T10:16:51.141769+0000","last_became_active":"2024-04-25T10:16:50.136525+0000","last_became_peered":"2024-04-25T10:16:50.136525+0000","last_unstale":"2024-04-25T10:16:51.141769+0000","last_undegraded":"2024-04-25T10:16:51.141769+0000","last_fullsized":"2024-04-25T10:16:51.141769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:32.249267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.158713+0000","last_change":"2024-04-25T10:16:49.141696+0000","last_active":"2024-04-25T10:16:50.158713+0000","last_peered":"2024-04-25T10:16:50.158713+0000","last_clean":"2024-04-25T10:16:50.158713+0000","last_became_active":"2024-04-25T10:16:49.140927+0000","last_became_peered":"2024-04-25T10:16:49.140927+0000","last_unstale":"2024-04-25T10:16:50.158713+0000","last_undegraded":"2024-04-25T10:16:50.158713+0000","last_fullsized":"2024-04-25T10:16:50.158713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:51.475669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142420+0000","last_change":"2024-04-25T10:16:50.140123+0000","last_active":"2024-04-25T10:16:51.142420+0000","last_peered":"2024-04-25T10:16:51.142420+0000","last_clean":"2024-04-25T10:16:51.142420+0000","last_became_active":"2024-04-25T10:16:50.139889+0000","last_became_peered":"2024-04-25T10:16:50.139889+0000","last_unstale":"2024-04-25T10:16:51.142420+0000","last_undegraded":"2024-04-25T10:16:51.142420+0000","last_fullsized":"2024-04-25T10:16:51.142420+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:48.494777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165263+0000","last_change":"2024-04-25T10:16:49.146929+0000","last_active":"2024-04-25T10:16:50.165263+0000","last_peered":"2024-04-25T10:16:50.165263+0000","last_clean":"2024-04-25T10:16:50.165263+0000","last_became_active":"2024-04-25T10:16:49.146313+0000","last_became_peered":"2024-04-25T10:16:49.146313+0000","last_unstale":"2024-04-25T10:16:50.165263+0000","last_undegraded":"2024-04-25T10:16:50.165263+0000","last_fullsized":"2024-04-25T10:16:50.165263+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:44.957426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.142348+0000","last_change":"2024-04-25T10:16:50.139888+0000","last_active":"2024-04-25T10:16:50.142348+0000","last_peered":"2024-04-25T10:16:50.142348+0000","last_clean":"2024-04-25T10:16:50.142348+0000","last_became_active":"2024-04-25T10:16:50.139664+0000","last_became_peered":"2024-04-25T10:16:50.139664+0000","last_unstale":"2024-04-25T10:16:50.142348+0000","last_undegraded":"2024-04-25T10:16:50.142348+0000","last_fullsized":"2024-04-25T10:16:50.142348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.329147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142594+0000","last_change":"2024-04-25T10:16:49.137748+0000","last_active":"2024-04-25T10:16:51.142594+0000","last_peered":"2024-04-25T10:16:51.142594+0000","last_clean":"2024-04-25T10:16:51.142594+0000","last_became_active":"2024-04-25T10:16:49.137447+0000","last_became_peered":"2024-04-25T10:16:49.137447+0000","last_unstale":"2024-04-25T10:16:51.142594+0000","last_undegraded":"2024-04-25T10:16:51.142594+0000","last_fullsized":"2024-04-25T10:16:51.142594+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:54.901126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144677+0000","last_change":"2024-04-25T10:16:50.142248+0000","last_active":"2024-04-25T10:16:51.144677+0000","last_peered":"2024-04-25T10:16:51.144677+0000","last_clean":"2024-04-25T10:16:51.144677+0000","last_became_active":"2024-04-25T10:16:50.142003+0000","last_became_peered":"2024-04-25T10:16:50.142003+0000","last_unstale":"2024-04-25T10:16:51.144677+0000","last_undegraded":"2024-04-25T10:16:51.144677+0000","last_fullsized":"2024-04-25T10:16:51.144677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:46.713781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.126855+0000","last_change":"2024-04-25T10:16:49.127114+0000","last_active":"2024-04-25T10:16:50.126855+0000","last_peered":"2024-04-25T10:16:50.126855+0000","last_clean":"2024-04-25T10:16:50.126855+0000","last_became_active":"2024-04-25T10:16:49.126862+0000","last_became_peered":"2024-04-25T10:16:49.126862+0000","last_unstale":"2024-04-25T10:16:50.126855+0000","last_undegraded":"2024-04-25T10:16:50.126855+0000","last_fullsized":"2024-04-25T10:16:50.126855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:26.840639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142008+0000","last_change":"2024-04-25T10:16:50.139496+0000","last_active":"2024-04-25T10:16:51.142008+0000","last_peered":"2024-04-25T10:16:51.142008+0000","last_clean":"2024-04-25T10:16:51.142008+0000","last_became_active":"2024-04-25T10:16:50.137125+0000","last_became_peered":"2024-04-25T10:16:50.137125+0000","last_unstale":"2024-04-25T10:16:51.142008+0000","last_undegraded":"2024-04-25T10:16:51.142008+0000","last_fullsized":"2024-04-25T10:16:51.142008+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:45.314672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142573+0000","last_change":"2024-04-25T10:16:49.139189+0000","last_active":"2024-04-25T10:16:51.142573+0000","last_peered":"2024-04-25T10:16:51.142573+0000","last_clean":"2024-04-25T10:16:51.142573+0000","last_became_active":"2024-04-25T10:16:49.138867+0000","last_became_peered":"2024-04-25T10:16:49.138867+0000","last_unstale":"2024-04-25T10:16:51.142573+0000","last_undegraded":"2024-04-25T10:16:51.142573+0000","last_fullsized":"2024-04-25T10:16:51.142573+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.921998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162653+0000","last_change":"2024-04-25T10:16:50.154927+0000","last_active":"2024-04-25T10:16:50.162653+0000","last_peered":"2024-04-25T10:16:50.162653+0000","last_clean":"2024-04-25T10:16:50.162653+0000","last_became_active":"2024-04-25T10:16:50.154273+0000","last_became_peered":"2024-04-25T10:16:50.154273+0000","last_unstale":"2024-04-25T10:16:50.162653+0000","last_undegraded":"2024-04-25T10:16:50.162653+0000","last_fullsized":"2024-04-25T10:16:50.162653+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:47.927523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.139107+0000","last_change":"2024-04-25T10:16:49.146653+0000","last_active":"2024-04-25T10:16:50.139107+0000","last_peered":"2024-04-25T10:16:50.139107+0000","last_clean":"2024-04-25T10:16:50.139107+0000","last_became_active":"2024-04-25T10:16:49.145303+0000","last_became_peered":"2024-04-25T10:16:49.145303+0000","last_unstale":"2024-04-25T10:16:50.139107+0000","last_undegraded":"2024-04-25T10:16:50.139107+0000","last_fullsized":"2024-04-25T10:16:50.139107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.513900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156775+0000","last_change":"2024-04-25T10:16:50.151045+0000","last_active":"2024-04-25T10:16:50.156775+0000","last_peered":"2024-04-25T10:16:50.156775+0000","last_clean":"2024-04-25T10:16:50.156775+0000","last_became_active":"2024-04-25T10:16:50.150731+0000","last_became_peered":"2024-04-25T10:16:50.150731+0000","last_unstale":"2024-04-25T10:16:50.156775+0000","last_undegraded":"2024-04-25T10:16:50.156775+0000","last_fullsized":"2024-04-25T10:16:50.156775+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.952185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.158683+0000","last_change":"2024-04-25T10:16:49.143221+0000","last_active":"2024-04-25T10:16:50.158683+0000","last_peered":"2024-04-25T10:16:50.158683+0000","last_clean":"2024-04-25T10:16:50.158683+0000","last_became_active":"2024-04-25T10:16:49.142576+0000","last_became_peered":"2024-04-25T10:16:49.142576+0000","last_unstale":"2024-04-25T10:16:50.158683+0000","last_undegraded":"2024-04-25T10:16:50.158683+0000","last_fullsized":"2024-04-25T10:16:50.158683+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:10.848312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.134405+0000","last_change":"2024-04-25T10:16:50.132211+0000","last_active":"2024-04-25T10:16:50.134405+0000","last_peered":"2024-04-25T10:16:50.134405+0000","last_clean":"2024-04-25T10:16:50.134405+0000","last_became_active":"2024-04-25T10:16:50.131901+0000","last_became_peered":"2024-04-25T10:16:50.131901+0000","last_unstale":"2024-04-25T10:16:50.134405+0000","last_undegraded":"2024-04-25T10:16:50.134405+0000","last_fullsized":"2024-04-25T10:16:50.134405+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:57.774395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142987+0000","last_change":"2024-04-25T10:16:49.125247+0000","last_active":"2024-04-25T10:16:51.142987+0000","last_peered":"2024-04-25T10:16:51.142987+0000","last_clean":"2024-04-25T10:16:51.142987+0000","last_became_active":"2024-04-25T10:16:49.124990+0000","last_became_peered":"2024-04-25T10:16:49.124990+0000","last_unstale":"2024-04-25T10:16:51.142987+0000","last_undegraded":"2024-04-25T10:16:51.142987+0000","last_fullsized":"2024-04-25T10:16:51.142987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.513900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.154167+0000","last_change":"2024-04-25T10:16:50.148212+0000","last_active":"2024-04-25T10:16:50.154167+0000","last_peered":"2024-04-25T10:16:50.154167+0000","last_clean":"2024-04-25T10:16:50.154167+0000","last_became_active":"2024-04-25T10:16:50.148017+0000","last_became_peered":"2024-04-25T10:16:50.148017+0000","last_unstale":"2024-04-25T10:16:50.154167+0000","last_undegraded":"2024-04-25T10:16:50.154167+0000","last_fullsized":"2024-04-25T10:16:50.154167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:39.915784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.143131+0000","last_change":"2024-04-25T10:16:49.138521+0000","last_active":"2024-04-25T10:16:51.143131+0000","last_peered":"2024-04-25T10:16:51.143131+0000","last_clean":"2024-04-25T10:16:51.143131+0000","last_became_active":"2024-04-25T10:16:49.138267+0000","last_became_peered":"2024-04-25T10:16:49.138267+0000","last_unstale":"2024-04-25T10:16:51.143131+0000","last_undegraded":"2024-04-25T10:16:51.143131+0000","last_fullsized":"2024-04-25T10:16:51.143131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:45.162838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141704+0000","last_change":"2024-04-25T10:16:50.140685+0000","last_active":"2024-04-25T10:16:51.141704+0000","last_peered":"2024-04-25T10:16:51.141704+0000","last_clean":"2024-04-25T10:16:51.141704+0000","last_became_active":"2024-04-25T10:16:50.140445+0000","last_became_peered":"2024-04-25T10:16:50.140445+0000","last_unstale":"2024-04-25T10:16:51.141704+0000","last_undegraded":"2024-04-25T10:16:51.141704+0000","last_fullsized":"2024-04-25T10:16:51.141704+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:42.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.143795+0000","last_change":"2024-04-25T10:16:49.147180+0000","last_active":"2024-04-25T10:16:50.143795+0000","last_peered":"2024-04-25T10:16:50.143795+0000","last_clean":"2024-04-25T10:16:50.143795+0000","last_became_active":"2024-04-25T10:16:49.146629+0000","last_became_peered":"2024-04-25T10:16:49.146629+0000","last_unstale":"2024-04-25T10:16:50.143795+0000","last_undegraded":"2024-04-25T10:16:50.143795+0000","last_fullsized":"2024-04-25T10:16:50.143795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:12.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144562+0000","last_change":"2024-04-25T10:16:50.151948+0000","last_active":"2024-04-25T10:16:51.144562+0000","last_peered":"2024-04-25T10:16:51.144562+0000","last_clean":"2024-04-25T10:16:51.144562+0000","last_became_active":"2024-04-25T10:16:50.151646+0000","last_became_peered":"2024-04-25T10:16:50.151646+0000","last_unstale":"2024-04-25T10:16:51.144562+0000","last_undegraded":"2024-04-25T10:16:51.144562+0000","last_fullsized":"2024-04-25T10:16:51.144562+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141276+0000","last_change":"2024-04-25T10:16:49.139505+0000","last_active":"2024-04-25T10:16:51.141276+0000","last_peered":"2024-04-25T10:16:51.141276+0000","last_clean":"2024-04-25T10:16:51.141276+0000","last_became_active":"2024-04-25T10:16:49.139202+0000","last_became_peered":"2024-04-25T10:16:49.139202+0000","last_unstale":"2024-04-25T10:16:51.141276+0000","last_undegraded":"2024-04-25T10:16:51.141276+0000","last_fullsized":"2024-04-25T10:16:51.141276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.095875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.138333+0000","last_change":"2024-04-25T10:16:50.132735+0000","last_active":"2024-04-25T10:16:50.138333+0000","last_peered":"2024-04-25T10:16:50.138333+0000","last_clean":"2024-04-25T10:16:50.138333+0000","last_became_active":"2024-04-25T10:16:50.131670+0000","last_became_peered":"2024-04-25T10:16:50.131670+0000","last_unstale":"2024-04-25T10:16:50.138333+0000","last_undegraded":"2024-04-25T10:16:50.138333+0000","last_fullsized":"2024-04-25T10:16:50.138333+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:31.958318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141996+0000","last_change":"2024-04-25T10:16:49.145069+0000","last_active":"2024-04-25T10:16:51.141996+0000","last_peered":"2024-04-25T10:16:51.141996+0000","last_clean":"2024-04-25T10:16:51.141996+0000","last_became_active":"2024-04-25T10:16:49.144777+0000","last_became_peered":"2024-04-25T10:16:49.144777+0000","last_unstale":"2024-04-25T10:16:51.141996+0000","last_undegraded":"2024-04-25T10:16:51.141996+0000","last_fullsized":"2024-04-25T10:16:51.141996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:52.385573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142603+0000","last_change":"2024-04-25T10:16:50.134330+0000","last_active":"2024-04-25T10:16:51.142603+0000","last_peered":"2024-04-25T10:16:51.142603+0000","last_clean":"2024-04-25T10:16:51.142603+0000","last_became_active":"2024-04-25T10:16:50.134088+0000","last_became_peered":"2024-04-25T10:16:50.134088+0000","last_unstale":"2024-04-25T10:16:51.142603+0000","last_undegraded":"2024-04-25T10:16:51.142603+0000","last_fullsized":"2024-04-25T10:16:51.142603+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165657+0000","last_change":"2024-04-25T10:16:49.145300+0000","last_active":"2024-04-25T10:16:50.165657+0000","last_peered":"2024-04-25T10:16:50.165657+0000","last_clean":"2024-04-25T10:16:50.165657+0000","last_became_active":"2024-04-25T10:16:49.143326+0000","last_became_peered":"2024-04-25T10:16:49.143326+0000","last_unstale":"2024-04-25T10:16:50.165657+0000","last_undegraded":"2024-04-25T10:16:50.165657+0000","last_fullsized":"2024-04-25T10:16:50.165657+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:53.621910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165015+0000","last_change":"2024-04-25T10:16:50.154962+0000","last_active":"2024-04-25T10:16:50.165015+0000","last_peered":"2024-04-25T10:16:50.165015+0000","last_clean":"2024-04-25T10:16:50.165015+0000","last_became_active":"2024-04-25T10:16:50.154571+0000","last_became_peered":"2024-04-25T10:16:50.154571+0000","last_unstale":"2024-04-25T10:16:50.165015+0000","last_undegraded":"2024-04-25T10:16:50.165015+0000","last_fullsized":"2024-04-25T10:16:50.165015+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:53.691264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142634+0000","last_change":"2024-04-25T10:16:49.146023+0000","last_active":"2024-04-25T10:16:51.142634+0000","last_peered":"2024-04-25T10:16:51.142634+0000","last_clean":"2024-04-25T10:16:51.142634+0000","last_became_active":"2024-04-25T10:16:49.145749+0000","last_became_peered":"2024-04-25T10:16:49.145749+0000","last_unstale":"2024-04-25T10:16:51.142634+0000","last_undegraded":"2024-04-25T10:16:51.142634+0000","last_fullsized":"2024-04-25T10:16:51.142634+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:34.945424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142984+0000","last_change":"2024-04-25T10:16:50.139904+0000","last_active":"2024-04-25T10:16:51.142984+0000","last_peered":"2024-04-25T10:16:51.142984+0000","last_clean":"2024-04-25T10:16:51.142984+0000","last_became_active":"2024-04-25T10:16:50.133010+0000","last_became_peered":"2024-04-25T10:16:50.133010+0000","last_unstale":"2024-04-25T10:16:51.142984+0000","last_undegraded":"2024-04-25T10:16:51.142984+0000","last_fullsized":"2024-04-25T10:16:51.142984+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:53.392334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.167998+0000","last_change":"2024-04-25T10:16:49.145395+0000","last_active":"2024-04-25T10:16:50.167998+0000","last_peered":"2024-04-25T10:16:50.167998+0000","last_clean":"2024-04-25T10:16:50.167998+0000","last_became_active":"2024-04-25T10:16:49.144104+0000","last_became_peered":"2024-04-25T10:16:49.144104+0000","last_unstale":"2024-04-25T10:16:50.167998+0000","last_undegraded":"2024-04-25T10:16:50.167998+0000","last_fullsized":"2024-04-25T10:16:50.167998+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:45.162838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.135232+0000","last_change":"2024-04-25T10:16:50.133709+0000","last_active":"2024-04-25T10:16:50.135232+0000","last_peered":"2024-04-25T10:16:50.135232+0000","last_clean":"2024-04-25T10:16:50.135232+0000","last_became_active":"2024-04-25T10:16:50.133060+0000","last_became_peered":"2024-04-25T10:16:50.133060+0000","last_unstale":"2024-04-25T10:16:50.135232+0000","last_undegraded":"2024-04-25T10:16:50.135232+0000","last_fullsized":"2024-04-25T10:16:50.135232+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:25.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.140705+0000","last_change":"2024-04-25T10:16:49.148912+0000","last_active":"2024-04-25T10:16:50.140705+0000","last_peered":"2024-04-25T10:16:50.140705+0000","last_clean":"2024-04-25T10:16:50.140705+0000","last_became_active":"2024-04-25T10:16:49.148500+0000","last_became_peered":"2024-04-25T10:16:49.148500+0000","last_unstale":"2024-04-25T10:16:50.140705+0000","last_undegraded":"2024-04-25T10:16:50.140705+0000","last_fullsized":"2024-04-25T10:16:50.140705+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:23.858100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.131950+0000","last_change":"2024-04-25T10:16:50.131043+0000","last_active":"2024-04-25T10:16:50.131950+0000","last_peered":"2024-04-25T10:16:50.131950+0000","last_clean":"2024-04-25T10:16:50.131950+0000","last_became_active":"2024-04-25T10:16:50.130214+0000","last_became_peered":"2024-04-25T10:16:50.130214+0000","last_unstale":"2024-04-25T10:16:50.131950+0000","last_undegraded":"2024-04-25T10:16:50.131950+0000","last_fullsized":"2024-04-25T10:16:50.131950+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:38.072889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.143920+0000","last_change":"2024-04-25T10:16:49.148262+0000","last_active":"2024-04-25T10:16:50.143920+0000","last_peered":"2024-04-25T10:16:50.143920+0000","last_clean":"2024-04-25T10:16:50.143920+0000","last_became_active":"2024-04-25T10:16:49.146035+0000","last_became_peered":"2024-04-25T10:16:49.146035+0000","last_unstale":"2024-04-25T10:16:50.143920+0000","last_undegraded":"2024-04-25T10:16:50.143920+0000","last_fullsized":"2024-04-25T10:16:50.143920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:33.638636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.147461+0000","last_change":"2024-04-25T10:16:50.146591+0000","last_active":"2024-04-25T10:16:50.147461+0000","last_peered":"2024-04-25T10:16:50.147461+0000","last_clean":"2024-04-25T10:16:50.147461+0000","last_became_active":"2024-04-25T10:16:50.146237+0000","last_became_peered":"2024-04-25T10:16:50.146237+0000","last_unstale":"2024-04-25T10:16:50.147461+0000","last_undegraded":"2024-04-25T10:16:50.147461+0000","last_fullsized":"2024-04-25T10:16:50.147461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:53.856935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141378+0000","last_change":"2024-04-25T10:16:49.140176+0000","last_active":"2024-04-25T10:16:51.141378+0000","last_peered":"2024-04-25T10:16:51.141378+0000","last_clean":"2024-04-25T10:16:51.141378+0000","last_became_active":"2024-04-25T10:16:49.138296+0000","last_became_peered":"2024-04-25T10:16:49.138296+0000","last_unstale":"2024-04-25T10:16:51.141378+0000","last_undegraded":"2024-04-25T10:16:51.141378+0000","last_fullsized":"2024-04-25T10:16:51.141378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:56.959668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.146712+0000","last_change":"2024-04-25T10:16:50.145924+0000","last_active":"2024-04-25T10:16:50.146712+0000","last_peered":"2024-04-25T10:16:50.146712+0000","last_clean":"2024-04-25T10:16:50.146712+0000","last_became_active":"2024-04-25T10:16:50.145593+0000","last_became_peered":"2024-04-25T10:16:50.145593+0000","last_unstale":"2024-04-25T10:16:50.146712+0000","last_undegraded":"2024-04-25T10:16:50.146712+0000","last_fullsized":"2024-04-25T10:16:50.146712+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:18.455543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144511+0000","last_change":"2024-04-25T10:16:49.134741+0000","last_active":"2024-04-25T10:16:51.144511+0000","last_peered":"2024-04-25T10:16:51.144511+0000","last_clean":"2024-04-25T10:16:51.144511+0000","last_became_active":"2024-04-25T10:16:49.134490+0000","last_became_peered":"2024-04-25T10:16:49.134490+0000","last_unstale":"2024-04-25T10:16:51.144511+0000","last_undegraded":"2024-04-25T10:16:51.144511+0000","last_fullsized":"2024-04-25T10:16:51.144511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.485763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144997+0000","last_change":"2024-04-25T10:16:50.144490+0000","last_active":"2024-04-25T10:16:51.144997+0000","last_peered":"2024-04-25T10:16:51.144997+0000","last_clean":"2024-04-25T10:16:51.144997+0000","last_became_active":"2024-04-25T10:16:50.143973+0000","last_became_peered":"2024-04-25T10:16:50.143973+0000","last_unstale":"2024-04-25T10:16:51.144997+0000","last_undegraded":"2024-04-25T10:16:51.144997+0000","last_fullsized":"2024-04-25T10:16:51.144997+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:32.249267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165580+0000","last_change":"2024-04-25T10:16:49.143528+0000","last_active":"2024-04-25T10:16:50.165580+0000","last_peered":"2024-04-25T10:16:50.165580+0000","last_clean":"2024-04-25T10:16:50.165580+0000","last_became_active":"2024-04-25T10:16:49.143028+0000","last_became_peered":"2024-04-25T10:16:49.143028+0000","last_unstale":"2024-04-25T10:16:50.165580+0000","last_undegraded":"2024-04-25T10:16:50.165580+0000","last_fullsized":"2024-04-25T10:16:50.165580+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:14.085054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142003+0000","last_change":"2024-04-25T10:16:50.134048+0000","last_active":"2024-04-25T10:16:51.142003+0000","last_peered":"2024-04-25T10:16:51.142003+0000","last_clean":"2024-04-25T10:16:51.142003+0000","last_became_active":"2024-04-25T10:16:50.133765+0000","last_became_peered":"2024-04-25T10:16:50.133765+0000","last_unstale":"2024-04-25T10:16:51.142003+0000","last_undegraded":"2024-04-25T10:16:51.142003+0000","last_fullsized":"2024-04-25T10:16:51.142003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:47.927523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"72'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.146754+0000","last_change":"2024-04-25T10:16:49.145643+0000","last_active":"2024-04-25T10:16:51.146754+0000","last_peered":"2024-04-25T10:16:51.146754+0000","last_clean":"2024-04-25T10:16:51.146754+0000","last_became_active":"2024-04-25T10:16:49.142261+0000","last_became_peered":"2024-04-25T10:16:49.142261+0000","last_unstale":"2024-04-25T10:16:51.146754+0000","last_undegraded":"2024-04-25T10:16:51.146754+0000","last_fullsized":"2024-04-25T10:16:51.146754+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.485763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162227+0000","last_change":"2024-04-25T10:16:50.156135+0000","last_active":"2024-04-25T10:16:50.162227+0000","last_peered":"2024-04-25T10:16:50.162227+0000","last_clean":"2024-04-25T10:16:50.162227+0000","last_became_active":"2024-04-25T10:16:50.155849+0000","last_became_peered":"2024-04-25T10:16:50.155849+0000","last_unstale":"2024-04-25T10:16:50.162227+0000","last_undegraded":"2024-04-25T10:16:50.162227+0000","last_fullsized":"2024-04-25T10:16:50.162227+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:42.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.126256+0000","last_change":"2024-04-25T10:16:49.131595+0000","last_active":"2024-04-25T10:16:50.126256+0000","last_peered":"2024-04-25T10:16:50.126256+0000","last_clean":"2024-04-25T10:16:50.126256+0000","last_became_active":"2024-04-25T10:16:49.131312+0000","last_became_peered":"2024-04-25T10:16:49.131312+0000","last_unstale":"2024-04-25T10:16:50.126256+0000","last_undegraded":"2024-04-25T10:16:50.126256+0000","last_fullsized":"2024-04-25T10:16:50.126256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:48.975144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162444+0000","last_change":"2024-04-25T10:16:50.152768+0000","last_active":"2024-04-25T10:16:50.162444+0000","last_peered":"2024-04-25T10:16:50.162444+0000","last_clean":"2024-04-25T10:16:50.162444+0000","last_became_active":"2024-04-25T10:16:50.152353+0000","last_became_peered":"2024-04-25T10:16:50.152353+0000","last_unstale":"2024-04-25T10:16:50.162444+0000","last_undegraded":"2024-04-25T10:16:50.162444+0000","last_fullsized":"2024-04-25T10:16:50.162444+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:46.713781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144755+0000","last_change":"2024-04-25T10:16:49.138735+0000","last_active":"2024-04-25T10:16:51.144755+0000","last_peered":"2024-04-25T10:16:51.144755+0000","last_clean":"2024-04-25T10:16:51.144755+0000","last_became_active":"2024-04-25T10:16:49.138480+0000","last_became_peered":"2024-04-25T10:16:49.138480+0000","last_unstale":"2024-04-25T10:16:51.144755+0000","last_undegraded":"2024-04-25T10:16:51.144755+0000","last_fullsized":"2024-04-25T10:16:51.144755+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:56.484910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.138141+0000","last_change":"2024-04-25T10:16:50.136042+0000","last_active":"2024-04-25T10:16:50.138141+0000","last_peered":"2024-04-25T10:16:50.138141+0000","last_clean":"2024-04-25T10:16:50.138141+0000","last_became_active":"2024-04-25T10:16:50.135760+0000","last_became_peered":"2024-04-25T10:16:50.135760+0000","last_unstale":"2024-04-25T10:16:50.138141+0000","last_undegraded":"2024-04-25T10:16:50.138141+0000","last_fullsized":"2024-04-25T10:16:50.138141+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:39.108095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.125298+0000","last_change":"2024-04-25T10:16:49.142208+0000","last_active":"2024-04-25T10:16:50.125298+0000","last_peered":"2024-04-25T10:16:50.125298+0000","last_clean":"2024-04-25T10:16:50.125298+0000","last_became_active":"2024-04-25T10:16:49.141946+0000","last_became_peered":"2024-04-25T10:16:49.141946+0000","last_unstale":"2024-04-25T10:16:50.125298+0000","last_undegraded":"2024-04-25T10:16:50.125298+0000","last_fullsized":"2024-04-25T10:16:50.125298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:47.897717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.153485+0000","last_change":"2024-04-25T10:16:50.148524+0000","last_active":"2024-04-25T10:16:50.153485+0000","last_peered":"2024-04-25T10:16:50.153485+0000","last_clean":"2024-04-25T10:16:50.153485+0000","last_became_active":"2024-04-25T10:16:50.148203+0000","last_became_peered":"2024-04-25T10:16:50.148203+0000","last_unstale":"2024-04-25T10:16:50.153485+0000","last_undegraded":"2024-04-25T10:16:50.153485+0000","last_fullsized":"2024-04-25T10:16:50.153485+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:55.487397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142647+0000","last_change":"2024-04-25T10:16:49.141126+0000","last_active":"2024-04-25T10:16:51.142647+0000","last_peered":"2024-04-25T10:16:51.142647+0000","last_clean":"2024-04-25T10:16:51.142647+0000","last_became_active":"2024-04-25T10:16:49.140841+0000","last_became_peered":"2024-04-25T10:16:49.140841+0000","last_unstale":"2024-04-25T10:16:51.142647+0000","last_undegraded":"2024-04-25T10:16:51.142647+0000","last_fullsized":"2024-04-25T10:16:51.142647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:03.154073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142679+0000","last_change":"2024-04-25T10:16:50.129259+0000","last_active":"2024-04-25T10:16:51.142679+0000","last_peered":"2024-04-25T10:16:51.142679+0000","last_clean":"2024-04-25T10:16:51.142679+0000","last_became_active":"2024-04-25T10:16:50.129001+0000","last_became_peered":"2024-04-25T10:16:50.129001+0000","last_unstale":"2024-04-25T10:16:51.142679+0000","last_undegraded":"2024-04-25T10:16:51.142679+0000","last_fullsized":"2024-04-25T10:16:51.142679+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:10.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141997+0000","last_change":"2024-04-25T10:16:49.146997+0000","last_active":"2024-04-25T10:16:51.141997+0000","last_peered":"2024-04-25T10:16:51.141997+0000","last_clean":"2024-04-25T10:16:51.141997+0000","last_became_active":"2024-04-25T10:16:49.146708+0000","last_became_peered":"2024-04-25T10:16:49.146708+0000","last_unstale":"2024-04-25T10:16:51.141997+0000","last_undegraded":"2024-04-25T10:16:51.141997+0000","last_fullsized":"2024-04-25T10:16:51.141997+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:18.970926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.150422+0000","last_change":"2024-04-25T10:16:50.145463+0000","last_active":"2024-04-25T10:16:50.150422+0000","last_peered":"2024-04-25T10:16:50.150422+0000","last_clean":"2024-04-25T10:16:50.150422+0000","last_became_active":"2024-04-25T10:16:50.145234+0000","last_became_peered":"2024-04-25T10:16:50.145234+0000","last_unstale":"2024-04-25T10:16:50.150422+0000","last_undegraded":"2024-04-25T10:16:50.150422+0000","last_fullsized":"2024-04-25T10:16:50.150422+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:10.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.146586+0000","last_change":"2024-04-25T10:16:49.142778+0000","last_active":"2024-04-25T10:16:50.146586+0000","last_peered":"2024-04-25T10:16:50.146586+0000","last_clean":"2024-04-25T10:16:50.146586+0000","last_became_active":"2024-04-25T10:16:49.142458+0000","last_became_peered":"2024-04-25T10:16:49.142458+0000","last_unstale":"2024-04-25T10:16:50.146586+0000","last_undegraded":"2024-04-25T10:16:50.146586+0000","last_fullsized":"2024-04-25T10:16:50.146586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:43.620115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.147715+0000","last_change":"2024-04-25T10:16:50.146848+0000","last_active":"2024-04-25T10:16:50.147715+0000","last_peered":"2024-04-25T10:16:50.147715+0000","last_clean":"2024-04-25T10:16:50.147715+0000","last_became_active":"2024-04-25T10:16:50.145688+0000","last_became_peered":"2024-04-25T10:16:50.145688+0000","last_unstale":"2024-04-25T10:16:50.147715+0000","last_undegraded":"2024-04-25T10:16:50.147715+0000","last_fullsized":"2024-04-25T10:16:50.147715+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:35.844851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.163369+0000","last_change":"2024-04-25T10:16:49.144093+0000","last_active":"2024-04-25T10:16:50.163369+0000","last_peered":"2024-04-25T10:16:50.163369+0000","last_clean":"2024-04-25T10:16:50.163369+0000","last_became_active":"2024-04-25T10:16:49.142820+0000","last_became_peered":"2024-04-25T10:16:49.142820+0000","last_unstale":"2024-04-25T10:16:50.163369+0000","last_undegraded":"2024-04-25T10:16:50.163369+0000","last_fullsized":"2024-04-25T10:16:50.163369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:54.901126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":261,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.133754+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:16:50.133754+0000","last_peered":"2024-04-25T10:16:50.133754+0000","last_clean":"2024-04-25T10:16:50.133754+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:16:50.133754+0000","last_undegraded":"2024-04-25T10:16:50.133754+0000","last_fullsized":"2024-04-25T10:16:50.133754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40932,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330908,"statfs":{"total":96636764160,"available":96594849792,"internally_reserved":0,"allocated":1466368,"data_stored":957455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":872448,"data_stored":366563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":868352,"data_stored":366561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":880640,"data_stored":372841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58404,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313436,"statfs":{"total":96636764160,"available":96576958464,"internally_reserved":0,"allocated":872448,"data_stored":366583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":876544,"data_stored":368083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40372,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331468,"statfs":{"total":96636764160,"available":96595423232,"internally_reserved":0,"allocated":884736,"data_stored":368699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1458176,"data_stored":956839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:54.210 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:54.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff233c00640 1 Processor -- start 2024-04-25T10:16:54.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff233c00640 1 -- start start 2024-04-25T10:16:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff233c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 0x7ff22c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff233c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff22c0b13e0 0x7ff22c0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff233c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c0b1870 con 0x7ff22c0d1820 2024-04-25T10:16:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff233c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c0b1af0 con 0x7ff22c0b13e0 2024-04-25T10:16:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff233c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0b1dd0 con 0x7ff22c15e050 2024-04-25T10:16:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff231200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff22c0d1820 0x7ff22c0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36324/0 (socket says 172.21.15.87:36324) 2024-04-25T10:16:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff231200640 1 -- 172.21.15.87:0/3398597132 learned_addr learned my addr 172.21.15.87:0/3398597132 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff231c00640 1 --2- 172.21.15.87:0/3398597132 >> v2:172.21.15.169:3300/0 conn(0x7ff22c0b13e0 0x7ff22c0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff22be00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 0x7ff22c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598704266 0 0) 0x7ff21c001120 con 0x7ff22c0d1820 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff210003530 con 0x7ff22c0d1820 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565615773 0 0) 0x7ff210003530 con 0x7ff22c0d1820 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 msgr2=0x7ff22c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.378+0000 7ff22b400640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 0x7ff22c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 >> v2:172.21.15.169:3300/0 conn(0x7ff22c0b13e0 msgr2=0x7ff22c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 --2- 172.21.15.87:0/3398597132 >> v2:172.21.15.169:3300/0 conn(0x7ff22c0b13e0 0x7ff22c0cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c0b20b0 con 0x7ff22c0d1820 2024-04-25T10:16:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ff21c001850 con 0x7ff22c0d1820 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 >> v1:172.21.15.87:6789/0 conn(0x7ff22c0d1820 legacy=0x7ff22c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 0x7ff2100039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff210004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d50 0x7ff2100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0b1af0 con 0x7ff210003f40 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0b1dd0 con 0x7ff210004d50 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21000ba20 con 0x7ff22c15e050 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22be00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff210004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff231200640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 0x7ff2100039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff231c00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d50 0x7ff2100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22be00640 1 -- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 msgr2=0x7ff2100039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22be00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c15e050 0x7ff2100039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22be00640 1 -- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d50 msgr2=0x7ff2100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22be00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d50 0x7ff2100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22be00640 1 -- 172.21.15.87:0/3398597132 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff210003530 con 0x7ff210003f40 2024-04-25T10:16:54.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff231c00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d50 0x7ff2100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22be00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff210004810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff218009ae0 tx=0x7ff21801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41a123a46e0aee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180b2070 con 0x7ff210003f40 2024-04-25T10:16:54.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2180a5510 con 0x7ff210003f40 2024-04-25T10:16:54.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.382+0000 7ff22b400640 1 -- 172.21.15.87:0/3398597132 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180096b0 con 0x7ff210003f40 2024-04-25T10:16:54.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- 172.21.15.87:0/3398597132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 msgr2=0x7ff210004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 --2- 172.21.15.87:0/3398597132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff210004810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff218009ae0 tx=0x7ff21801ca30 comp rx=0 tx=0).stop 2024-04-25T10:16:54.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231c00640 1 -- 172.21.15.87:0/3398597132 reap_dead start 2024-04-25T10:16:54.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- 172.21.15.87:0/3398597132 shutdown_connections 2024-04-25T10:16:54.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- 172.21.15.87:0/3398597132 >> 172.21.15.87:0/3398597132 conn(0x7ff22c09a9d0 msgr2=0x7ff22c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- 172.21.15.87:0/3398597132 shutdown_connections 2024-04-25T10:16:54.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- 172.21.15.87:0/3398597132 wait complete. 2024-04-25T10:16:54.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 Processor -- start 2024-04-25T10:16:54.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- start start 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff22c0cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c156cf0 0x7ff22c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c15be30 0x7ff22c0afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0d0d90 con 0x7ff210003f40 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0d0c10 con 0x7ff22c15be30 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff233c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0d0f10 con 0x7ff22c156cf0 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff22c0cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff22c0cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60454/0 (socket says 172.21.15.87:60454) 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 -- 172.21.15.87:0/3488194903 learned_addr learned my addr 172.21.15.87:0/3488194903 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff22be00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c156cf0 0x7ff22c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c15be30 0x7ff22c0afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 -- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c156cf0 msgr2=0x7ff22c1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c156cf0 0x7ff22c1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 -- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c15be30 msgr2=0x7ff22c0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c15be30 0x7ff22c0afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231200640 1 -- 172.21.15.87:0/3488194903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c1569f0 con 0x7ff210003f40 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.386+0000 7ff231c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c15be30 0x7ff22c0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff231200640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff22c0cbf30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0073a0 tx=0x7ff21c007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff228c00640 1 -- 172.21.15.87:0/3488194903 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c01a070 con 0x7ff210003f40 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff228c00640 1 -- 172.21.15.87:0/3488194903 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c00f930 con 0x7ff210003f40 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c0cec40 con 0x7ff210003f40 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff228c00640 1 -- 172.21.15.87:0/3488194903 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00fd00 con 0x7ff210003f40 2024-04-25T10:16:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c0caf60 con 0x7ff210003f40 2024-04-25T10:16:54.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c04ef30 con 0x7ff210003f40 2024-04-25T10:16:54.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff228c00640 1 -- 172.21.15.87:0/3488194903 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff21c01d070 con 0x7ff210003f40 2024-04-25T10:16:54.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff228c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff20c066f60 0x7ff20c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff228c00640 1 -- 172.21.15.87:0/3488194903 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff21c084a20 con 0x7ff210003f40 2024-04-25T10:16:54.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.390+0000 7ff22be00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff20c066f60 0x7ff20c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.394+0000 7ff22be00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff20c066f60 0x7ff20c069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff2180af000 tx=0x7ff218008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.394+0000 7ff228c00640 1 -- 172.21.15.87:0/3488194903 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff22c0b1870 con 0x7ff210003f40 2024-04-25T10:16:54.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.546+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff22c0ce9f0 con 0x7ff20c066f60 2024-04-25T10:16:54.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.554+0000 7ff228c00640 1 -- 172.21.15.87:0/3488194903 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7ff22c0ce9f0 con 0x7ff20c066f60 2024-04-25T10:16:54.562 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:54.564 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:16:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff20c066f60 msgr2=0x7ff20c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff20c066f60 0x7ff20c069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff2180af000 tx=0x7ff218008040 comp rx=0 tx=0).stop 2024-04-25T10:16:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 msgr2=0x7ff22c0cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff22c0cbf30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0073a0 tx=0x7ff21c007860 comp rx=0 tx=0).stop 2024-04-25T10:16:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 shutdown_connections 2024-04-25T10:16:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff20c066f60 0x7ff20c069420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c15be30 0x7ff22c0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c156cf0 0x7ff22c1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 --2- 172.21.15.87:0/3488194903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff210003f40 0x7ff22c0cbf30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 >> 172.21.15.87:0/3488194903 conn(0x7ff22c09a9d0 msgr2=0x7ff22c15af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 shutdown_connections 2024-04-25T10:16:54.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.558+0000 7ff233c00640 1 -- 172.21.15.87:0/3488194903 wait complete. 2024-04-25T10:16:54.583 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-04-25T10:16:53.182446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":325428,"kb_used_data":7988,"kb_used_omap":12,"kb_used_meta":317299,"kb_avail":754649292,"statfs":{"total":773094113280,"available":772760875008,"internally_reserved":0,"allocated":8179712,"data_stored":4123624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324914737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.072572"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162675+0000","last_change":"2024-04-25T10:16:50.157783+0000","last_active":"2024-04-25T10:16:50.162675+0000","last_peered":"2024-04-25T10:16:50.162675+0000","last_clean":"2024-04-25T10:16:50.162675+0000","last_became_active":"2024-04-25T10:16:50.154676+0000","last_became_peered":"2024-04-25T10:16:50.154676+0000","last_unstale":"2024-04-25T10:16:50.162675+0000","last_undegraded":"2024-04-25T10:16:50.162675+0000","last_fullsized":"2024-04-25T10:16:50.162675+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:45.314672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156657+0000","last_change":"2024-04-25T10:16:50.150575+0000","last_active":"2024-04-25T10:16:50.156657+0000","last_peered":"2024-04-25T10:16:50.156657+0000","last_clean":"2024-04-25T10:16:50.156657+0000","last_became_active":"2024-04-25T10:16:50.150204+0000","last_became_peered":"2024-04-25T10:16:50.150204+0000","last_unstale":"2024-04-25T10:16:50.156657+0000","last_undegraded":"2024-04-25T10:16:50.156657+0000","last_fullsized":"2024-04-25T10:16:50.156657+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:59.928760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.144882+0000","last_change":"2024-04-25T10:16:50.143838+0000","last_active":"2024-04-25T10:16:50.144882+0000","last_peered":"2024-04-25T10:16:50.144882+0000","last_clean":"2024-04-25T10:16:50.144882+0000","last_became_active":"2024-04-25T10:16:50.143537+0000","last_became_peered":"2024-04-25T10:16:50.143537+0000","last_unstale":"2024-04-25T10:16:50.144882+0000","last_undegraded":"2024-04-25T10:16:50.144882+0000","last_fullsized":"2024-04-25T10:16:50.144882+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:32.014814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142803+0000","last_change":"2024-04-25T10:16:50.150537+0000","last_active":"2024-04-25T10:16:51.142803+0000","last_peered":"2024-04-25T10:16:51.142803+0000","last_clean":"2024-04-25T10:16:51.142803+0000","last_became_active":"2024-04-25T10:16:50.150265+0000","last_became_peered":"2024-04-25T10:16:50.150265+0000","last_unstale":"2024-04-25T10:16:51.142803+0000","last_undegraded":"2024-04-25T10:16:51.142803+0000","last_fullsized":"2024-04-25T10:16:51.142803+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:44.626876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.141764+0000","last_change":"2024-04-25T10:16:50.138288+0000","last_active":"2024-04-25T10:16:50.141764+0000","last_peered":"2024-04-25T10:16:50.141764+0000","last_clean":"2024-04-25T10:16:50.141764+0000","last_became_active":"2024-04-25T10:16:50.136764+0000","last_became_peered":"2024-04-25T10:16:50.136764+0000","last_unstale":"2024-04-25T10:16:50.141764+0000","last_undegraded":"2024-04-25T10:16:50.141764+0000","last_fullsized":"2024-04-25T10:16:50.141764+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:11.147356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156462+0000","last_change":"2024-04-25T10:16:50.150517+0000","last_active":"2024-04-25T10:16:50.156462+0000","last_peered":"2024-04-25T10:16:50.156462+0000","last_clean":"2024-04-25T10:16:50.156462+0000","last_became_active":"2024-04-25T10:16:50.150108+0000","last_became_peered":"2024-04-25T10:16:50.150108+0000","last_unstale":"2024-04-25T10:16:50.156462+0000","last_undegraded":"2024-04-25T10:16:50.156462+0000","last_fullsized":"2024-04-25T10:16:50.156462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:19.977687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141285+0000","last_change":"2024-04-25T10:16:50.134190+0000","last_active":"2024-04-25T10:16:51.141285+0000","last_peered":"2024-04-25T10:16:51.141285+0000","last_clean":"2024-04-25T10:16:51.141285+0000","last_became_active":"2024-04-25T10:16:50.133921+0000","last_became_peered":"2024-04-25T10:16:50.133921+0000","last_unstale":"2024-04-25T10:16:51.141285+0000","last_undegraded":"2024-04-25T10:16:51.141285+0000","last_fullsized":"2024-04-25T10:16:51.141285+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156803+0000","last_change":"2024-04-25T10:16:50.152597+0000","last_active":"2024-04-25T10:16:50.156803+0000","last_peered":"2024-04-25T10:16:50.156803+0000","last_clean":"2024-04-25T10:16:50.156803+0000","last_became_active":"2024-04-25T10:16:50.152358+0000","last_became_peered":"2024-04-25T10:16:50.152358+0000","last_unstale":"2024-04-25T10:16:50.156803+0000","last_undegraded":"2024-04-25T10:16:50.156803+0000","last_fullsized":"2024-04-25T10:16:50.156803+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:30.704661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.138954+0000","last_change":"2024-04-25T10:16:50.137672+0000","last_active":"2024-04-25T10:16:50.138954+0000","last_peered":"2024-04-25T10:16:50.138954+0000","last_clean":"2024-04-25T10:16:50.138954+0000","last_became_active":"2024-04-25T10:16:50.135708+0000","last_became_peered":"2024-04-25T10:16:50.135708+0000","last_unstale":"2024-04-25T10:16:50.138954+0000","last_undegraded":"2024-04-25T10:16:50.138954+0000","last_fullsized":"2024-04-25T10:16:50.138954+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:34.045400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142552+0000","last_change":"2024-04-25T10:16:50.136848+0000","last_active":"2024-04-25T10:16:51.142552+0000","last_peered":"2024-04-25T10:16:51.142552+0000","last_clean":"2024-04-25T10:16:51.142552+0000","last_became_active":"2024-04-25T10:16:50.136612+0000","last_became_peered":"2024-04-25T10:16:50.136612+0000","last_unstale":"2024-04-25T10:16:51.142552+0000","last_undegraded":"2024-04-25T10:16:51.142552+0000","last_fullsized":"2024-04-25T10:16:51.142552+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:39.915784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162351+0000","last_change":"2024-04-25T10:16:50.152577+0000","last_active":"2024-04-25T10:16:50.162351+0000","last_peered":"2024-04-25T10:16:50.162351+0000","last_clean":"2024-04-25T10:16:50.162351+0000","last_became_active":"2024-04-25T10:16:50.152303+0000","last_became_peered":"2024-04-25T10:16:50.152303+0000","last_unstale":"2024-04-25T10:16:50.162351+0000","last_undegraded":"2024-04-25T10:16:50.162351+0000","last_fullsized":"2024-04-25T10:16:50.162351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142870+0000","last_change":"2024-04-25T10:16:50.137099+0000","last_active":"2024-04-25T10:16:51.142870+0000","last_peered":"2024-04-25T10:16:51.142870+0000","last_clean":"2024-04-25T10:16:51.142870+0000","last_became_active":"2024-04-25T10:16:50.133428+0000","last_became_peered":"2024-04-25T10:16:50.133428+0000","last_unstale":"2024-04-25T10:16:51.142870+0000","last_undegraded":"2024-04-25T10:16:51.142870+0000","last_fullsized":"2024-04-25T10:16:51.142870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:57.940222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144844+0000","last_change":"2024-04-25T10:16:50.147237+0000","last_active":"2024-04-25T10:16:51.144844+0000","last_peered":"2024-04-25T10:16:51.144844+0000","last_clean":"2024-04-25T10:16:51.144844+0000","last_became_active":"2024-04-25T10:16:50.146933+0000","last_became_peered":"2024-04-25T10:16:50.146933+0000","last_unstale":"2024-04-25T10:16:51.144844+0000","last_undegraded":"2024-04-25T10:16:51.144844+0000","last_fullsized":"2024-04-25T10:16:51.144844+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.133752+0000","last_change":"2024-04-25T10:16:50.132721+0000","last_active":"2024-04-25T10:16:50.133752+0000","last_peered":"2024-04-25T10:16:50.133752+0000","last_clean":"2024-04-25T10:16:50.133752+0000","last_became_active":"2024-04-25T10:16:50.132365+0000","last_became_peered":"2024-04-25T10:16:50.132365+0000","last_unstale":"2024-04-25T10:16:50.133752+0000","last_undegraded":"2024-04-25T10:16:50.133752+0000","last_fullsized":"2024-04-25T10:16:50.133752+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:50.809029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144627+0000","last_change":"2024-04-25T10:16:50.139389+0000","last_active":"2024-04-25T10:16:51.144627+0000","last_peered":"2024-04-25T10:16:51.144627+0000","last_clean":"2024-04-25T10:16:51.144627+0000","last_became_active":"2024-04-25T10:16:50.139084+0000","last_became_peered":"2024-04-25T10:16:50.139084+0000","last_unstale":"2024-04-25T10:16:51.144627+0000","last_undegraded":"2024-04-25T10:16:51.144627+0000","last_fullsized":"2024-04-25T10:16:51.144627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141946+0000","last_change":"2024-04-25T10:16:50.152313+0000","last_active":"2024-04-25T10:16:51.141946+0000","last_peered":"2024-04-25T10:16:51.141946+0000","last_clean":"2024-04-25T10:16:51.141946+0000","last_became_active":"2024-04-25T10:16:50.152050+0000","last_became_peered":"2024-04-25T10:16:50.152050+0000","last_unstale":"2024-04-25T10:16:51.141946+0000","last_undegraded":"2024-04-25T10:16:51.141946+0000","last_fullsized":"2024-04-25T10:16:51.141946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:07.867840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.141430+0000","last_change":"2024-04-25T10:16:50.138414+0000","last_active":"2024-04-25T10:16:50.141430+0000","last_peered":"2024-04-25T10:16:50.141430+0000","last_clean":"2024-04-25T10:16:50.141430+0000","last_became_active":"2024-04-25T10:16:50.137765+0000","last_became_peered":"2024-04-25T10:16:50.137765+0000","last_unstale":"2024-04-25T10:16:50.141430+0000","last_undegraded":"2024-04-25T10:16:50.141430+0000","last_fullsized":"2024-04-25T10:16:50.141430+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:53.551273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142953+0000","last_change":"2024-04-25T10:16:50.146423+0000","last_active":"2024-04-25T10:16:51.142953+0000","last_peered":"2024-04-25T10:16:51.142953+0000","last_clean":"2024-04-25T10:16:51.142953+0000","last_became_active":"2024-04-25T10:16:50.146065+0000","last_became_peered":"2024-04-25T10:16:50.146065+0000","last_unstale":"2024-04-25T10:16:51.142953+0000","last_undegraded":"2024-04-25T10:16:51.142953+0000","last_fullsized":"2024-04-25T10:16:51.142953+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:55.487397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156617+0000","last_change":"2024-04-25T10:16:50.150959+0000","last_active":"2024-04-25T10:16:50.156617+0000","last_peered":"2024-04-25T10:16:50.156617+0000","last_clean":"2024-04-25T10:16:50.156617+0000","last_became_active":"2024-04-25T10:16:50.150482+0000","last_became_peered":"2024-04-25T10:16:50.150482+0000","last_unstale":"2024-04-25T10:16:50.156617+0000","last_undegraded":"2024-04-25T10:16:50.156617+0000","last_fullsized":"2024-04-25T10:16:50.156617+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:21.611590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144765+0000","last_change":"2024-04-25T10:16:50.143763+0000","last_active":"2024-04-25T10:16:51.144765+0000","last_peered":"2024-04-25T10:16:51.144765+0000","last_clean":"2024-04-25T10:16:51.144765+0000","last_became_active":"2024-04-25T10:16:50.143172+0000","last_became_peered":"2024-04-25T10:16:50.143172+0000","last_unstale":"2024-04-25T10:16:51.144765+0000","last_undegraded":"2024-04-25T10:16:51.144765+0000","last_fullsized":"2024-04-25T10:16:51.144765+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:42.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142262+0000","last_change":"2024-04-25T10:16:50.145914+0000","last_active":"2024-04-25T10:16:51.142262+0000","last_peered":"2024-04-25T10:16:51.142262+0000","last_clean":"2024-04-25T10:16:51.142262+0000","last_became_active":"2024-04-25T10:16:50.145642+0000","last_became_peered":"2024-04-25T10:16:50.145642+0000","last_unstale":"2024-04-25T10:16:51.142262+0000","last_undegraded":"2024-04-25T10:16:51.142262+0000","last_fullsized":"2024-04-25T10:16:51.142262+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:24.864861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142243+0000","last_change":"2024-04-25T10:16:50.132627+0000","last_active":"2024-04-25T10:16:51.142243+0000","last_peered":"2024-04-25T10:16:51.142243+0000","last_clean":"2024-04-25T10:16:51.142243+0000","last_became_active":"2024-04-25T10:16:50.132374+0000","last_became_peered":"2024-04-25T10:16:50.132374+0000","last_unstale":"2024-04-25T10:16:51.142243+0000","last_undegraded":"2024-04-25T10:16:51.142243+0000","last_fullsized":"2024-04-25T10:16:51.142243+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:43.151695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.153900+0000","last_change":"2024-04-25T10:16:50.145664+0000","last_active":"2024-04-25T10:16:50.153900+0000","last_peered":"2024-04-25T10:16:50.153900+0000","last_clean":"2024-04-25T10:16:50.153900+0000","last_became_active":"2024-04-25T10:16:50.144920+0000","last_became_peered":"2024-04-25T10:16:50.144920+0000","last_unstale":"2024-04-25T10:16:50.153900+0000","last_undegraded":"2024-04-25T10:16:50.153900+0000","last_fullsized":"2024-04-25T10:16:50.153900+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:31.639211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162382+0000","last_change":"2024-04-25T10:16:50.154737+0000","last_active":"2024-04-25T10:16:50.162382+0000","last_peered":"2024-04-25T10:16:50.162382+0000","last_clean":"2024-04-25T10:16:50.162382+0000","last_became_active":"2024-04-25T10:16:50.154535+0000","last_became_peered":"2024-04-25T10:16:50.154535+0000","last_unstale":"2024-04-25T10:16:50.162382+0000","last_undegraded":"2024-04-25T10:16:50.162382+0000","last_fullsized":"2024-04-25T10:16:50.162382+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:32.249267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142562+0000","last_change":"2024-04-25T10:16:50.141328+0000","last_active":"2024-04-25T10:16:51.142562+0000","last_peered":"2024-04-25T10:16:51.142562+0000","last_clean":"2024-04-25T10:16:51.142562+0000","last_became_active":"2024-04-25T10:16:50.141129+0000","last_became_peered":"2024-04-25T10:16:50.141129+0000","last_unstale":"2024-04-25T10:16:51.142562+0000","last_undegraded":"2024-04-25T10:16:51.142562+0000","last_fullsized":"2024-04-25T10:16:51.142562+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141768+0000","last_change":"2024-04-25T10:16:50.146078+0000","last_active":"2024-04-25T10:16:51.141768+0000","last_peered":"2024-04-25T10:16:51.141768+0000","last_clean":"2024-04-25T10:16:51.141768+0000","last_became_active":"2024-04-25T10:16:50.145808+0000","last_became_peered":"2024-04-25T10:16:50.145808+0000","last_unstale":"2024-04-25T10:16:51.141768+0000","last_undegraded":"2024-04-25T10:16:51.141768+0000","last_fullsized":"2024-04-25T10:16:51.141768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.142569+0000","last_change":"2024-04-25T10:16:50.141629+0000","last_active":"2024-04-25T10:16:50.142569+0000","last_peered":"2024-04-25T10:16:50.142569+0000","last_clean":"2024-04-25T10:16:50.142569+0000","last_became_active":"2024-04-25T10:16:50.140963+0000","last_became_peered":"2024-04-25T10:16:50.140963+0000","last_unstale":"2024-04-25T10:16:50.142569+0000","last_undegraded":"2024-04-25T10:16:50.142569+0000","last_fullsized":"2024-04-25T10:16:50.142569+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:45.475155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142510+0000","last_change":"2024-04-25T10:16:50.139607+0000","last_active":"2024-04-25T10:16:51.142510+0000","last_peered":"2024-04-25T10:16:51.142510+0000","last_clean":"2024-04-25T10:16:51.142510+0000","last_became_active":"2024-04-25T10:16:50.139370+0000","last_became_peered":"2024-04-25T10:16:50.139370+0000","last_unstale":"2024-04-25T10:16:51.142510+0000","last_undegraded":"2024-04-25T10:16:51.142510+0000","last_fullsized":"2024-04-25T10:16:51.142510+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:31.639211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.131684+0000","last_change":"2024-04-25T10:16:50.130241+0000","last_active":"2024-04-25T10:16:50.131684+0000","last_peered":"2024-04-25T10:16:50.131684+0000","last_clean":"2024-04-25T10:16:50.131684+0000","last_became_active":"2024-04-25T10:16:50.129961+0000","last_became_peered":"2024-04-25T10:16:50.129961+0000","last_unstale":"2024-04-25T10:16:50.131684+0000","last_undegraded":"2024-04-25T10:16:50.131684+0000","last_fullsized":"2024-04-25T10:16:50.131684+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:23.158296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141398+0000","last_change":"2024-04-25T10:16:50.146359+0000","last_active":"2024-04-25T10:16:51.141398+0000","last_peered":"2024-04-25T10:16:51.141398+0000","last_clean":"2024-04-25T10:16:51.141398+0000","last_became_active":"2024-04-25T10:16:50.146091+0000","last_became_peered":"2024-04-25T10:16:50.146091+0000","last_unstale":"2024-04-25T10:16:51.141398+0000","last_undegraded":"2024-04-25T10:16:51.141398+0000","last_fullsized":"2024-04-25T10:16:51.141398+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141426+0000","last_change":"2024-04-25T10:16:50.137295+0000","last_active":"2024-04-25T10:16:51.141426+0000","last_peered":"2024-04-25T10:16:51.141426+0000","last_clean":"2024-04-25T10:16:51.141426+0000","last_became_active":"2024-04-25T10:16:50.136975+0000","last_became_peered":"2024-04-25T10:16:50.136975+0000","last_unstale":"2024-04-25T10:16:51.141426+0000","last_undegraded":"2024-04-25T10:16:51.141426+0000","last_fullsized":"2024-04-25T10:16:51.141426+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:46.713781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.155703+0000","last_change":"2024-04-25T10:16:50.149725+0000","last_active":"2024-04-25T10:16:50.155703+0000","last_peered":"2024-04-25T10:16:50.155703+0000","last_clean":"2024-04-25T10:16:50.155703+0000","last_became_active":"2024-04-25T10:16:50.147376+0000","last_became_peered":"2024-04-25T10:16:50.147376+0000","last_unstale":"2024-04-25T10:16:50.155703+0000","last_undegraded":"2024-04-25T10:16:50.155703+0000","last_fullsized":"2024-04-25T10:16:50.155703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:53.392334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162559+0000","last_change":"2024-04-25T10:16:50.154006+0000","last_active":"2024-04-25T10:16:50.162559+0000","last_peered":"2024-04-25T10:16:50.162559+0000","last_clean":"2024-04-25T10:16:50.162559+0000","last_became_active":"2024-04-25T10:16:50.153756+0000","last_became_peered":"2024-04-25T10:16:50.153756+0000","last_unstale":"2024-04-25T10:16:50.162559+0000","last_undegraded":"2024-04-25T10:16:50.162559+0000","last_fullsized":"2024-04-25T10:16:50.162559+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.168212+0000","last_change":"2024-04-25T10:16:49.144515+0000","last_active":"2024-04-25T10:16:50.168212+0000","last_peered":"2024-04-25T10:16:50.168212+0000","last_clean":"2024-04-25T10:16:50.168212+0000","last_became_active":"2024-04-25T10:16:49.144172+0000","last_became_peered":"2024-04-25T10:16:49.144172+0000","last_unstale":"2024-04-25T10:16:50.168212+0000","last_undegraded":"2024-04-25T10:16:50.168212+0000","last_fullsized":"2024-04-25T10:16:50.168212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.488016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.141406+0000","last_change":"2024-04-25T10:16:50.137944+0000","last_active":"2024-04-25T10:16:50.141406+0000","last_peered":"2024-04-25T10:16:50.141406+0000","last_clean":"2024-04-25T10:16:50.141406+0000","last_became_active":"2024-04-25T10:16:50.135661+0000","last_became_peered":"2024-04-25T10:16:50.135661+0000","last_unstale":"2024-04-25T10:16:50.141406+0000","last_undegraded":"2024-04-25T10:16:50.141406+0000","last_fullsized":"2024-04-25T10:16:50.141406+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:57.875349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144968+0000","last_change":"2024-04-25T10:16:49.134932+0000","last_active":"2024-04-25T10:16:51.144968+0000","last_peered":"2024-04-25T10:16:51.144968+0000","last_clean":"2024-04-25T10:16:51.144968+0000","last_became_active":"2024-04-25T10:16:49.134675+0000","last_became_peered":"2024-04-25T10:16:49.134675+0000","last_unstale":"2024-04-25T10:16:51.144968+0000","last_undegraded":"2024-04-25T10:16:51.144968+0000","last_fullsized":"2024-04-25T10:16:51.144968+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:51.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.151541+0000","last_change":"2024-04-25T10:16:50.146874+0000","last_active":"2024-04-25T10:16:50.151541+0000","last_peered":"2024-04-25T10:16:50.151541+0000","last_clean":"2024-04-25T10:16:50.151541+0000","last_became_active":"2024-04-25T10:16:50.146566+0000","last_became_peered":"2024-04-25T10:16:50.146566+0000","last_unstale":"2024-04-25T10:16:50.151541+0000","last_undegraded":"2024-04-25T10:16:50.151541+0000","last_fullsized":"2024-04-25T10:16:50.151541+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:57.940222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142485+0000","last_change":"2024-04-25T10:16:49.150795+0000","last_active":"2024-04-25T10:16:51.142485+0000","last_peered":"2024-04-25T10:16:51.142485+0000","last_clean":"2024-04-25T10:16:51.142485+0000","last_became_active":"2024-04-25T10:16:49.150281+0000","last_became_peered":"2024-04-25T10:16:49.150281+0000","last_unstale":"2024-04-25T10:16:51.142485+0000","last_undegraded":"2024-04-25T10:16:51.142485+0000","last_fullsized":"2024-04-25T10:16:51.142485+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:20.604829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162328+0000","last_change":"2024-04-25T10:16:50.152550+0000","last_active":"2024-04-25T10:16:50.162328+0000","last_peered":"2024-04-25T10:16:50.162328+0000","last_clean":"2024-04-25T10:16:50.162328+0000","last_became_active":"2024-04-25T10:16:50.152081+0000","last_became_peered":"2024-04-25T10:16:50.152081+0000","last_unstale":"2024-04-25T10:16:50.162328+0000","last_undegraded":"2024-04-25T10:16:50.162328+0000","last_fullsized":"2024-04-25T10:16:50.162328+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165876+0000","last_change":"2024-04-25T10:16:49.145670+0000","last_active":"2024-04-25T10:16:50.165876+0000","last_peered":"2024-04-25T10:16:50.165876+0000","last_clean":"2024-04-25T10:16:50.165876+0000","last_became_active":"2024-04-25T10:16:49.143322+0000","last_became_peered":"2024-04-25T10:16:49.143322+0000","last_unstale":"2024-04-25T10:16:50.165876+0000","last_undegraded":"2024-04-25T10:16:50.165876+0000","last_fullsized":"2024-04-25T10:16:50.165876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:03.154073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141769+0000","last_change":"2024-04-25T10:16:50.136771+0000","last_active":"2024-04-25T10:16:51.141769+0000","last_peered":"2024-04-25T10:16:51.141769+0000","last_clean":"2024-04-25T10:16:51.141769+0000","last_became_active":"2024-04-25T10:16:50.136525+0000","last_became_peered":"2024-04-25T10:16:50.136525+0000","last_unstale":"2024-04-25T10:16:51.141769+0000","last_undegraded":"2024-04-25T10:16:51.141769+0000","last_fullsized":"2024-04-25T10:16:51.141769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:32.249267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.158713+0000","last_change":"2024-04-25T10:16:49.141696+0000","last_active":"2024-04-25T10:16:50.158713+0000","last_peered":"2024-04-25T10:16:50.158713+0000","last_clean":"2024-04-25T10:16:50.158713+0000","last_became_active":"2024-04-25T10:16:49.140927+0000","last_became_peered":"2024-04-25T10:16:49.140927+0000","last_unstale":"2024-04-25T10:16:50.158713+0000","last_undegraded":"2024-04-25T10:16:50.158713+0000","last_fullsized":"2024-04-25T10:16:50.158713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:51.475669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142420+0000","last_change":"2024-04-25T10:16:50.140123+0000","last_active":"2024-04-25T10:16:51.142420+0000","last_peered":"2024-04-25T10:16:51.142420+0000","last_clean":"2024-04-25T10:16:51.142420+0000","last_became_active":"2024-04-25T10:16:50.139889+0000","last_became_peered":"2024-04-25T10:16:50.139889+0000","last_unstale":"2024-04-25T10:16:51.142420+0000","last_undegraded":"2024-04-25T10:16:51.142420+0000","last_fullsized":"2024-04-25T10:16:51.142420+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:48.494777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165263+0000","last_change":"2024-04-25T10:16:49.146929+0000","last_active":"2024-04-25T10:16:50.165263+0000","last_peered":"2024-04-25T10:16:50.165263+0000","last_clean":"2024-04-25T10:16:50.165263+0000","last_became_active":"2024-04-25T10:16:49.146313+0000","last_became_peered":"2024-04-25T10:16:49.146313+0000","last_unstale":"2024-04-25T10:16:50.165263+0000","last_undegraded":"2024-04-25T10:16:50.165263+0000","last_fullsized":"2024-04-25T10:16:50.165263+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:44.957426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.142348+0000","last_change":"2024-04-25T10:16:50.139888+0000","last_active":"2024-04-25T10:16:50.142348+0000","last_peered":"2024-04-25T10:16:50.142348+0000","last_clean":"2024-04-25T10:16:50.142348+0000","last_became_active":"2024-04-25T10:16:50.139664+0000","last_became_peered":"2024-04-25T10:16:50.139664+0000","last_unstale":"2024-04-25T10:16:50.142348+0000","last_undegraded":"2024-04-25T10:16:50.142348+0000","last_fullsized":"2024-04-25T10:16:50.142348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.329147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142594+0000","last_change":"2024-04-25T10:16:49.137748+0000","last_active":"2024-04-25T10:16:51.142594+0000","last_peered":"2024-04-25T10:16:51.142594+0000","last_clean":"2024-04-25T10:16:51.142594+0000","last_became_active":"2024-04-25T10:16:49.137447+0000","last_became_peered":"2024-04-25T10:16:49.137447+0000","last_unstale":"2024-04-25T10:16:51.142594+0000","last_undegraded":"2024-04-25T10:16:51.142594+0000","last_fullsized":"2024-04-25T10:16:51.142594+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:54.901126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144677+0000","last_change":"2024-04-25T10:16:50.142248+0000","last_active":"2024-04-25T10:16:51.144677+0000","last_peered":"2024-04-25T10:16:51.144677+0000","last_clean":"2024-04-25T10:16:51.144677+0000","last_became_active":"2024-04-25T10:16:50.142003+0000","last_became_peered":"2024-04-25T10:16:50.142003+0000","last_unstale":"2024-04-25T10:16:51.144677+0000","last_undegraded":"2024-04-25T10:16:51.144677+0000","last_fullsized":"2024-04-25T10:16:51.144677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:46.713781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.126855+0000","last_change":"2024-04-25T10:16:49.127114+0000","last_active":"2024-04-25T10:16:50.126855+0000","last_peered":"2024-04-25T10:16:50.126855+0000","last_clean":"2024-04-25T10:16:50.126855+0000","last_became_active":"2024-04-25T10:16:49.126862+0000","last_became_peered":"2024-04-25T10:16:49.126862+0000","last_unstale":"2024-04-25T10:16:50.126855+0000","last_undegraded":"2024-04-25T10:16:50.126855+0000","last_fullsized":"2024-04-25T10:16:50.126855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:26.840639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142008+0000","last_change":"2024-04-25T10:16:50.139496+0000","last_active":"2024-04-25T10:16:51.142008+0000","last_peered":"2024-04-25T10:16:51.142008+0000","last_clean":"2024-04-25T10:16:51.142008+0000","last_became_active":"2024-04-25T10:16:50.137125+0000","last_became_peered":"2024-04-25T10:16:50.137125+0000","last_unstale":"2024-04-25T10:16:51.142008+0000","last_undegraded":"2024-04-25T10:16:51.142008+0000","last_fullsized":"2024-04-25T10:16:51.142008+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:45.314672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142573+0000","last_change":"2024-04-25T10:16:49.139189+0000","last_active":"2024-04-25T10:16:51.142573+0000","last_peered":"2024-04-25T10:16:51.142573+0000","last_clean":"2024-04-25T10:16:51.142573+0000","last_became_active":"2024-04-25T10:16:49.138867+0000","last_became_peered":"2024-04-25T10:16:49.138867+0000","last_unstale":"2024-04-25T10:16:51.142573+0000","last_undegraded":"2024-04-25T10:16:51.142573+0000","last_fullsized":"2024-04-25T10:16:51.142573+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.921998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162653+0000","last_change":"2024-04-25T10:16:50.154927+0000","last_active":"2024-04-25T10:16:50.162653+0000","last_peered":"2024-04-25T10:16:50.162653+0000","last_clean":"2024-04-25T10:16:50.162653+0000","last_became_active":"2024-04-25T10:16:50.154273+0000","last_became_peered":"2024-04-25T10:16:50.154273+0000","last_unstale":"2024-04-25T10:16:50.162653+0000","last_undegraded":"2024-04-25T10:16:50.162653+0000","last_fullsized":"2024-04-25T10:16:50.162653+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:47.927523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.139107+0000","last_change":"2024-04-25T10:16:49.146653+0000","last_active":"2024-04-25T10:16:50.139107+0000","last_peered":"2024-04-25T10:16:50.139107+0000","last_clean":"2024-04-25T10:16:50.139107+0000","last_became_active":"2024-04-25T10:16:49.145303+0000","last_became_peered":"2024-04-25T10:16:49.145303+0000","last_unstale":"2024-04-25T10:16:50.139107+0000","last_undegraded":"2024-04-25T10:16:50.139107+0000","last_fullsized":"2024-04-25T10:16:50.139107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.513900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.156775+0000","last_change":"2024-04-25T10:16:50.151045+0000","last_active":"2024-04-25T10:16:50.156775+0000","last_peered":"2024-04-25T10:16:50.156775+0000","last_clean":"2024-04-25T10:16:50.156775+0000","last_became_active":"2024-04-25T10:16:50.150731+0000","last_became_peered":"2024-04-25T10:16:50.150731+0000","last_unstale":"2024-04-25T10:16:50.156775+0000","last_undegraded":"2024-04-25T10:16:50.156775+0000","last_fullsized":"2024-04-25T10:16:50.156775+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.952185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.158683+0000","last_change":"2024-04-25T10:16:49.143221+0000","last_active":"2024-04-25T10:16:50.158683+0000","last_peered":"2024-04-25T10:16:50.158683+0000","last_clean":"2024-04-25T10:16:50.158683+0000","last_became_active":"2024-04-25T10:16:49.142576+0000","last_became_peered":"2024-04-25T10:16:49.142576+0000","last_unstale":"2024-04-25T10:16:50.158683+0000","last_undegraded":"2024-04-25T10:16:50.158683+0000","last_fullsized":"2024-04-25T10:16:50.158683+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:10.848312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.134405+0000","last_change":"2024-04-25T10:16:50.132211+0000","last_active":"2024-04-25T10:16:50.134405+0000","last_peered":"2024-04-25T10:16:50.134405+0000","last_clean":"2024-04-25T10:16:50.134405+0000","last_became_active":"2024-04-25T10:16:50.131901+0000","last_became_peered":"2024-04-25T10:16:50.131901+0000","last_unstale":"2024-04-25T10:16:50.134405+0000","last_undegraded":"2024-04-25T10:16:50.134405+0000","last_fullsized":"2024-04-25T10:16:50.134405+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:57.774395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142987+0000","last_change":"2024-04-25T10:16:49.125247+0000","last_active":"2024-04-25T10:16:51.142987+0000","last_peered":"2024-04-25T10:16:51.142987+0000","last_clean":"2024-04-25T10:16:51.142987+0000","last_became_active":"2024-04-25T10:16:49.124990+0000","last_became_peered":"2024-04-25T10:16:49.124990+0000","last_unstale":"2024-04-25T10:16:51.142987+0000","last_undegraded":"2024-04-25T10:16:51.142987+0000","last_fullsized":"2024-04-25T10:16:51.142987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.513900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.154167+0000","last_change":"2024-04-25T10:16:50.148212+0000","last_active":"2024-04-25T10:16:50.154167+0000","last_peered":"2024-04-25T10:16:50.154167+0000","last_clean":"2024-04-25T10:16:50.154167+0000","last_became_active":"2024-04-25T10:16:50.148017+0000","last_became_peered":"2024-04-25T10:16:50.148017+0000","last_unstale":"2024-04-25T10:16:50.154167+0000","last_undegraded":"2024-04-25T10:16:50.154167+0000","last_fullsized":"2024-04-25T10:16:50.154167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:39.915784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.143131+0000","last_change":"2024-04-25T10:16:49.138521+0000","last_active":"2024-04-25T10:16:51.143131+0000","last_peered":"2024-04-25T10:16:51.143131+0000","last_clean":"2024-04-25T10:16:51.143131+0000","last_became_active":"2024-04-25T10:16:49.138267+0000","last_became_peered":"2024-04-25T10:16:49.138267+0000","last_unstale":"2024-04-25T10:16:51.143131+0000","last_undegraded":"2024-04-25T10:16:51.143131+0000","last_fullsized":"2024-04-25T10:16:51.143131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:45.162838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141704+0000","last_change":"2024-04-25T10:16:50.140685+0000","last_active":"2024-04-25T10:16:51.141704+0000","last_peered":"2024-04-25T10:16:51.141704+0000","last_clean":"2024-04-25T10:16:51.141704+0000","last_became_active":"2024-04-25T10:16:50.140445+0000","last_became_peered":"2024-04-25T10:16:50.140445+0000","last_unstale":"2024-04-25T10:16:51.141704+0000","last_undegraded":"2024-04-25T10:16:51.141704+0000","last_fullsized":"2024-04-25T10:16:51.141704+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:42.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.143795+0000","last_change":"2024-04-25T10:16:49.147180+0000","last_active":"2024-04-25T10:16:50.143795+0000","last_peered":"2024-04-25T10:16:50.143795+0000","last_clean":"2024-04-25T10:16:50.143795+0000","last_became_active":"2024-04-25T10:16:49.146629+0000","last_became_peered":"2024-04-25T10:16:49.146629+0000","last_unstale":"2024-04-25T10:16:50.143795+0000","last_undegraded":"2024-04-25T10:16:50.143795+0000","last_fullsized":"2024-04-25T10:16:50.143795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:12.385935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144562+0000","last_change":"2024-04-25T10:16:50.151948+0000","last_active":"2024-04-25T10:16:51.144562+0000","last_peered":"2024-04-25T10:16:51.144562+0000","last_clean":"2024-04-25T10:16:51.144562+0000","last_became_active":"2024-04-25T10:16:50.151646+0000","last_became_peered":"2024-04-25T10:16:50.151646+0000","last_unstale":"2024-04-25T10:16:51.144562+0000","last_undegraded":"2024-04-25T10:16:51.144562+0000","last_fullsized":"2024-04-25T10:16:51.144562+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:45.554466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141276+0000","last_change":"2024-04-25T10:16:49.139505+0000","last_active":"2024-04-25T10:16:51.141276+0000","last_peered":"2024-04-25T10:16:51.141276+0000","last_clean":"2024-04-25T10:16:51.141276+0000","last_became_active":"2024-04-25T10:16:49.139202+0000","last_became_peered":"2024-04-25T10:16:49.139202+0000","last_unstale":"2024-04-25T10:16:51.141276+0000","last_undegraded":"2024-04-25T10:16:51.141276+0000","last_fullsized":"2024-04-25T10:16:51.141276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:41.095875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.138333+0000","last_change":"2024-04-25T10:16:50.132735+0000","last_active":"2024-04-25T10:16:50.138333+0000","last_peered":"2024-04-25T10:16:50.138333+0000","last_clean":"2024-04-25T10:16:50.138333+0000","last_became_active":"2024-04-25T10:16:50.131670+0000","last_became_peered":"2024-04-25T10:16:50.131670+0000","last_unstale":"2024-04-25T10:16:50.138333+0000","last_undegraded":"2024-04-25T10:16:50.138333+0000","last_fullsized":"2024-04-25T10:16:50.138333+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:31.958318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141996+0000","last_change":"2024-04-25T10:16:49.145069+0000","last_active":"2024-04-25T10:16:51.141996+0000","last_peered":"2024-04-25T10:16:51.141996+0000","last_clean":"2024-04-25T10:16:51.141996+0000","last_became_active":"2024-04-25T10:16:49.144777+0000","last_became_peered":"2024-04-25T10:16:49.144777+0000","last_unstale":"2024-04-25T10:16:51.141996+0000","last_undegraded":"2024-04-25T10:16:51.141996+0000","last_fullsized":"2024-04-25T10:16:51.141996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:52.385573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142603+0000","last_change":"2024-04-25T10:16:50.134330+0000","last_active":"2024-04-25T10:16:51.142603+0000","last_peered":"2024-04-25T10:16:51.142603+0000","last_clean":"2024-04-25T10:16:51.142603+0000","last_became_active":"2024-04-25T10:16:50.134088+0000","last_became_peered":"2024-04-25T10:16:50.134088+0000","last_unstale":"2024-04-25T10:16:51.142603+0000","last_undegraded":"2024-04-25T10:16:51.142603+0000","last_fullsized":"2024-04-25T10:16:51.142603+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:55.049962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165657+0000","last_change":"2024-04-25T10:16:49.145300+0000","last_active":"2024-04-25T10:16:50.165657+0000","last_peered":"2024-04-25T10:16:50.165657+0000","last_clean":"2024-04-25T10:16:50.165657+0000","last_became_active":"2024-04-25T10:16:49.143326+0000","last_became_peered":"2024-04-25T10:16:49.143326+0000","last_unstale":"2024-04-25T10:16:50.165657+0000","last_undegraded":"2024-04-25T10:16:50.165657+0000","last_fullsized":"2024-04-25T10:16:50.165657+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:53.621910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165015+0000","last_change":"2024-04-25T10:16:50.154962+0000","last_active":"2024-04-25T10:16:50.165015+0000","last_peered":"2024-04-25T10:16:50.165015+0000","last_clean":"2024-04-25T10:16:50.165015+0000","last_became_active":"2024-04-25T10:16:50.154571+0000","last_became_peered":"2024-04-25T10:16:50.154571+0000","last_unstale":"2024-04-25T10:16:50.165015+0000","last_undegraded":"2024-04-25T10:16:50.165015+0000","last_fullsized":"2024-04-25T10:16:50.165015+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:53.691264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142634+0000","last_change":"2024-04-25T10:16:49.146023+0000","last_active":"2024-04-25T10:16:51.142634+0000","last_peered":"2024-04-25T10:16:51.142634+0000","last_clean":"2024-04-25T10:16:51.142634+0000","last_became_active":"2024-04-25T10:16:49.145749+0000","last_became_peered":"2024-04-25T10:16:49.145749+0000","last_unstale":"2024-04-25T10:16:51.142634+0000","last_undegraded":"2024-04-25T10:16:51.142634+0000","last_fullsized":"2024-04-25T10:16:51.142634+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:34.945424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142984+0000","last_change":"2024-04-25T10:16:50.139904+0000","last_active":"2024-04-25T10:16:51.142984+0000","last_peered":"2024-04-25T10:16:51.142984+0000","last_clean":"2024-04-25T10:16:51.142984+0000","last_became_active":"2024-04-25T10:16:50.133010+0000","last_became_peered":"2024-04-25T10:16:50.133010+0000","last_unstale":"2024-04-25T10:16:51.142984+0000","last_undegraded":"2024-04-25T10:16:51.142984+0000","last_fullsized":"2024-04-25T10:16:51.142984+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:53.392334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.167998+0000","last_change":"2024-04-25T10:16:49.145395+0000","last_active":"2024-04-25T10:16:50.167998+0000","last_peered":"2024-04-25T10:16:50.167998+0000","last_clean":"2024-04-25T10:16:50.167998+0000","last_became_active":"2024-04-25T10:16:49.144104+0000","last_became_peered":"2024-04-25T10:16:49.144104+0000","last_unstale":"2024-04-25T10:16:50.167998+0000","last_undegraded":"2024-04-25T10:16:50.167998+0000","last_fullsized":"2024-04-25T10:16:50.167998+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:45.162838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.135232+0000","last_change":"2024-04-25T10:16:50.133709+0000","last_active":"2024-04-25T10:16:50.135232+0000","last_peered":"2024-04-25T10:16:50.135232+0000","last_clean":"2024-04-25T10:16:50.135232+0000","last_became_active":"2024-04-25T10:16:50.133060+0000","last_became_peered":"2024-04-25T10:16:50.133060+0000","last_unstale":"2024-04-25T10:16:50.135232+0000","last_undegraded":"2024-04-25T10:16:50.135232+0000","last_fullsized":"2024-04-25T10:16:50.135232+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:25.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.140705+0000","last_change":"2024-04-25T10:16:49.148912+0000","last_active":"2024-04-25T10:16:50.140705+0000","last_peered":"2024-04-25T10:16:50.140705+0000","last_clean":"2024-04-25T10:16:50.140705+0000","last_became_active":"2024-04-25T10:16:49.148500+0000","last_became_peered":"2024-04-25T10:16:49.148500+0000","last_unstale":"2024-04-25T10:16:50.140705+0000","last_undegraded":"2024-04-25T10:16:50.140705+0000","last_fullsized":"2024-04-25T10:16:50.140705+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:23.858100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.131950+0000","last_change":"2024-04-25T10:16:50.131043+0000","last_active":"2024-04-25T10:16:50.131950+0000","last_peered":"2024-04-25T10:16:50.131950+0000","last_clean":"2024-04-25T10:16:50.131950+0000","last_became_active":"2024-04-25T10:16:50.130214+0000","last_became_peered":"2024-04-25T10:16:50.130214+0000","last_unstale":"2024-04-25T10:16:50.131950+0000","last_undegraded":"2024-04-25T10:16:50.131950+0000","last_fullsized":"2024-04-25T10:16:50.131950+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:38.072889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.143920+0000","last_change":"2024-04-25T10:16:49.148262+0000","last_active":"2024-04-25T10:16:50.143920+0000","last_peered":"2024-04-25T10:16:50.143920+0000","last_clean":"2024-04-25T10:16:50.143920+0000","last_became_active":"2024-04-25T10:16:49.146035+0000","last_became_peered":"2024-04-25T10:16:49.146035+0000","last_unstale":"2024-04-25T10:16:50.143920+0000","last_undegraded":"2024-04-25T10:16:50.143920+0000","last_fullsized":"2024-04-25T10:16:50.143920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:33.638636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.147461+0000","last_change":"2024-04-25T10:16:50.146591+0000","last_active":"2024-04-25T10:16:50.147461+0000","last_peered":"2024-04-25T10:16:50.147461+0000","last_clean":"2024-04-25T10:16:50.147461+0000","last_became_active":"2024-04-25T10:16:50.146237+0000","last_became_peered":"2024-04-25T10:16:50.146237+0000","last_unstale":"2024-04-25T10:16:50.147461+0000","last_undegraded":"2024-04-25T10:16:50.147461+0000","last_fullsized":"2024-04-25T10:16:50.147461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:53.856935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141378+0000","last_change":"2024-04-25T10:16:49.140176+0000","last_active":"2024-04-25T10:16:51.141378+0000","last_peered":"2024-04-25T10:16:51.141378+0000","last_clean":"2024-04-25T10:16:51.141378+0000","last_became_active":"2024-04-25T10:16:49.138296+0000","last_became_peered":"2024-04-25T10:16:49.138296+0000","last_unstale":"2024-04-25T10:16:51.141378+0000","last_undegraded":"2024-04-25T10:16:51.141378+0000","last_fullsized":"2024-04-25T10:16:51.141378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:56.959668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.146712+0000","last_change":"2024-04-25T10:16:50.145924+0000","last_active":"2024-04-25T10:16:50.146712+0000","last_peered":"2024-04-25T10:16:50.146712+0000","last_clean":"2024-04-25T10:16:50.146712+0000","last_became_active":"2024-04-25T10:16:50.145593+0000","last_became_peered":"2024-04-25T10:16:50.145593+0000","last_unstale":"2024-04-25T10:16:50.146712+0000","last_undegraded":"2024-04-25T10:16:50.146712+0000","last_fullsized":"2024-04-25T10:16:50.146712+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:18.455543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144511+0000","last_change":"2024-04-25T10:16:49.134741+0000","last_active":"2024-04-25T10:16:51.144511+0000","last_peered":"2024-04-25T10:16:51.144511+0000","last_clean":"2024-04-25T10:16:51.144511+0000","last_became_active":"2024-04-25T10:16:49.134490+0000","last_became_peered":"2024-04-25T10:16:49.134490+0000","last_unstale":"2024-04-25T10:16:51.144511+0000","last_undegraded":"2024-04-25T10:16:51.144511+0000","last_fullsized":"2024-04-25T10:16:51.144511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.485763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144997+0000","last_change":"2024-04-25T10:16:50.144490+0000","last_active":"2024-04-25T10:16:51.144997+0000","last_peered":"2024-04-25T10:16:51.144997+0000","last_clean":"2024-04-25T10:16:51.144997+0000","last_became_active":"2024-04-25T10:16:50.143973+0000","last_became_peered":"2024-04-25T10:16:50.143973+0000","last_unstale":"2024-04-25T10:16:51.144997+0000","last_undegraded":"2024-04-25T10:16:51.144997+0000","last_fullsized":"2024-04-25T10:16:51.144997+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:32.249267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.165580+0000","last_change":"2024-04-25T10:16:49.143528+0000","last_active":"2024-04-25T10:16:50.165580+0000","last_peered":"2024-04-25T10:16:50.165580+0000","last_clean":"2024-04-25T10:16:50.165580+0000","last_became_active":"2024-04-25T10:16:49.143028+0000","last_became_peered":"2024-04-25T10:16:49.143028+0000","last_unstale":"2024-04-25T10:16:50.165580+0000","last_undegraded":"2024-04-25T10:16:50.165580+0000","last_fullsized":"2024-04-25T10:16:50.165580+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:14.085054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142003+0000","last_change":"2024-04-25T10:16:50.134048+0000","last_active":"2024-04-25T10:16:51.142003+0000","last_peered":"2024-04-25T10:16:51.142003+0000","last_clean":"2024-04-25T10:16:51.142003+0000","last_became_active":"2024-04-25T10:16:50.133765+0000","last_became_peered":"2024-04-25T10:16:50.133765+0000","last_unstale":"2024-04-25T10:16:51.142003+0000","last_undegraded":"2024-04-25T10:16:51.142003+0000","last_fullsized":"2024-04-25T10:16:51.142003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:47.927523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"72'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.146754+0000","last_change":"2024-04-25T10:16:49.145643+0000","last_active":"2024-04-25T10:16:51.146754+0000","last_peered":"2024-04-25T10:16:51.146754+0000","last_clean":"2024-04-25T10:16:51.146754+0000","last_became_active":"2024-04-25T10:16:49.142261+0000","last_became_peered":"2024-04-25T10:16:49.142261+0000","last_unstale":"2024-04-25T10:16:51.146754+0000","last_undegraded":"2024-04-25T10:16:51.146754+0000","last_fullsized":"2024-04-25T10:16:51.146754+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.485763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162227+0000","last_change":"2024-04-25T10:16:50.156135+0000","last_active":"2024-04-25T10:16:50.162227+0000","last_peered":"2024-04-25T10:16:50.162227+0000","last_clean":"2024-04-25T10:16:50.162227+0000","last_became_active":"2024-04-25T10:16:50.155849+0000","last_became_peered":"2024-04-25T10:16:50.155849+0000","last_unstale":"2024-04-25T10:16:50.162227+0000","last_undegraded":"2024-04-25T10:16:50.162227+0000","last_fullsized":"2024-04-25T10:16:50.162227+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:42.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.126256+0000","last_change":"2024-04-25T10:16:49.131595+0000","last_active":"2024-04-25T10:16:50.126256+0000","last_peered":"2024-04-25T10:16:50.126256+0000","last_clean":"2024-04-25T10:16:50.126256+0000","last_became_active":"2024-04-25T10:16:49.131312+0000","last_became_peered":"2024-04-25T10:16:49.131312+0000","last_unstale":"2024-04-25T10:16:50.126256+0000","last_undegraded":"2024-04-25T10:16:50.126256+0000","last_fullsized":"2024-04-25T10:16:50.126256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:48.975144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.162444+0000","last_change":"2024-04-25T10:16:50.152768+0000","last_active":"2024-04-25T10:16:50.162444+0000","last_peered":"2024-04-25T10:16:50.162444+0000","last_clean":"2024-04-25T10:16:50.162444+0000","last_became_active":"2024-04-25T10:16:50.152353+0000","last_became_peered":"2024-04-25T10:16:50.152353+0000","last_unstale":"2024-04-25T10:16:50.162444+0000","last_undegraded":"2024-04-25T10:16:50.162444+0000","last_fullsized":"2024-04-25T10:16:50.162444+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:46.713781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.144755+0000","last_change":"2024-04-25T10:16:49.138735+0000","last_active":"2024-04-25T10:16:51.144755+0000","last_peered":"2024-04-25T10:16:51.144755+0000","last_clean":"2024-04-25T10:16:51.144755+0000","last_became_active":"2024-04-25T10:16:49.138480+0000","last_became_peered":"2024-04-25T10:16:49.138480+0000","last_unstale":"2024-04-25T10:16:51.144755+0000","last_undegraded":"2024-04-25T10:16:51.144755+0000","last_fullsized":"2024-04-25T10:16:51.144755+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:56.484910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.138141+0000","last_change":"2024-04-25T10:16:50.136042+0000","last_active":"2024-04-25T10:16:50.138141+0000","last_peered":"2024-04-25T10:16:50.138141+0000","last_clean":"2024-04-25T10:16:50.138141+0000","last_became_active":"2024-04-25T10:16:50.135760+0000","last_became_peered":"2024-04-25T10:16:50.135760+0000","last_unstale":"2024-04-25T10:16:50.138141+0000","last_undegraded":"2024-04-25T10:16:50.138141+0000","last_fullsized":"2024-04-25T10:16:50.138141+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:39.108095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.125298+0000","last_change":"2024-04-25T10:16:49.142208+0000","last_active":"2024-04-25T10:16:50.125298+0000","last_peered":"2024-04-25T10:16:50.125298+0000","last_clean":"2024-04-25T10:16:50.125298+0000","last_became_active":"2024-04-25T10:16:49.141946+0000","last_became_peered":"2024-04-25T10:16:49.141946+0000","last_unstale":"2024-04-25T10:16:50.125298+0000","last_undegraded":"2024-04-25T10:16:50.125298+0000","last_fullsized":"2024-04-25T10:16:50.125298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:47.897717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.153485+0000","last_change":"2024-04-25T10:16:50.148524+0000","last_active":"2024-04-25T10:16:50.153485+0000","last_peered":"2024-04-25T10:16:50.153485+0000","last_clean":"2024-04-25T10:16:50.153485+0000","last_became_active":"2024-04-25T10:16:50.148203+0000","last_became_peered":"2024-04-25T10:16:50.148203+0000","last_unstale":"2024-04-25T10:16:50.153485+0000","last_undegraded":"2024-04-25T10:16:50.153485+0000","last_fullsized":"2024-04-25T10:16:50.153485+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:55.487397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142647+0000","last_change":"2024-04-25T10:16:49.141126+0000","last_active":"2024-04-25T10:16:51.142647+0000","last_peered":"2024-04-25T10:16:51.142647+0000","last_clean":"2024-04-25T10:16:51.142647+0000","last_became_active":"2024-04-25T10:16:49.140841+0000","last_became_peered":"2024-04-25T10:16:49.140841+0000","last_unstale":"2024-04-25T10:16:51.142647+0000","last_undegraded":"2024-04-25T10:16:51.142647+0000","last_fullsized":"2024-04-25T10:16:51.142647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:03.154073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.142679+0000","last_change":"2024-04-25T10:16:50.129259+0000","last_active":"2024-04-25T10:16:51.142679+0000","last_peered":"2024-04-25T10:16:51.142679+0000","last_clean":"2024-04-25T10:16:51.142679+0000","last_became_active":"2024-04-25T10:16:50.129001+0000","last_became_peered":"2024-04-25T10:16:50.129001+0000","last_unstale":"2024-04-25T10:16:51.142679+0000","last_undegraded":"2024-04-25T10:16:51.142679+0000","last_fullsized":"2024-04-25T10:16:51.142679+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:10.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:16:51.141997+0000","last_change":"2024-04-25T10:16:49.146997+0000","last_active":"2024-04-25T10:16:51.141997+0000","last_peered":"2024-04-25T10:16:51.141997+0000","last_clean":"2024-04-25T10:16:51.141997+0000","last_became_active":"2024-04-25T10:16:49.146708+0000","last_became_peered":"2024-04-25T10:16:49.146708+0000","last_unstale":"2024-04-25T10:16:51.141997+0000","last_undegraded":"2024-04-25T10:16:51.141997+0000","last_fullsized":"2024-04-25T10:16:51.141997+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:18.970926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.150422+0000","last_change":"2024-04-25T10:16:50.145463+0000","last_active":"2024-04-25T10:16:50.150422+0000","last_peered":"2024-04-25T10:16:50.150422+0000","last_clean":"2024-04-25T10:16:50.150422+0000","last_became_active":"2024-04-25T10:16:50.145234+0000","last_became_peered":"2024-04-25T10:16:50.145234+0000","last_unstale":"2024-04-25T10:16:50.150422+0000","last_undegraded":"2024-04-25T10:16:50.150422+0000","last_fullsized":"2024-04-25T10:16:50.150422+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:10.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.146586+0000","last_change":"2024-04-25T10:16:49.142778+0000","last_active":"2024-04-25T10:16:50.146586+0000","last_peered":"2024-04-25T10:16:50.146586+0000","last_clean":"2024-04-25T10:16:50.146586+0000","last_became_active":"2024-04-25T10:16:49.142458+0000","last_became_peered":"2024-04-25T10:16:49.142458+0000","last_unstale":"2024-04-25T10:16:50.146586+0000","last_undegraded":"2024-04-25T10:16:50.146586+0000","last_fullsized":"2024-04-25T10:16:50.146586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:43.620115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.147715+0000","last_change":"2024-04-25T10:16:50.146848+0000","last_active":"2024-04-25T10:16:50.147715+0000","last_peered":"2024-04-25T10:16:50.147715+0000","last_clean":"2024-04-25T10:16:50.147715+0000","last_became_active":"2024-04-25T10:16:50.145688+0000","last_became_peered":"2024-04-25T10:16:50.145688+0000","last_unstale":"2024-04-25T10:16:50.147715+0000","last_undegraded":"2024-04-25T10:16:50.147715+0000","last_fullsized":"2024-04-25T10:16:50.147715+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:49.109891+0000","last_clean_scrub_stamp":"2024-04-25T10:16:49.109891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:35.844851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.163369+0000","last_change":"2024-04-25T10:16:49.144093+0000","last_active":"2024-04-25T10:16:50.163369+0000","last_peered":"2024-04-25T10:16:50.163369+0000","last_clean":"2024-04-25T10:16:50.163369+0000","last_became_active":"2024-04-25T10:16:49.142820+0000","last_became_peered":"2024-04-25T10:16:49.142820+0000","last_unstale":"2024-04-25T10:16:50.163369+0000","last_undegraded":"2024-04-25T10:16:50.163369+0000","last_fullsized":"2024-04-25T10:16:50.163369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:48.103129+0000","last_clean_scrub_stamp":"2024-04-25T10:16:48.103129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:54.901126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":261,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:16:50.133754+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:16:50.133754+0000","last_peered":"2024-04-25T10:16:50.133754+0000","last_clean":"2024-04-25T10:16:50.133754+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:16:50.133754+0000","last_undegraded":"2024-04-25T10:16:50.133754+0000","last_fullsized":"2024-04-25T10:16:50.133754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40932,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330908,"statfs":{"total":96636764160,"available":96594849792,"internally_reserved":0,"allocated":1466368,"data_stored":957455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":872448,"data_stored":366563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":868352,"data_stored":366561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":880640,"data_stored":372841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58404,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313436,"statfs":{"total":96636764160,"available":96576958464,"internally_reserved":0,"allocated":872448,"data_stored":366583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":876544,"data_stored":368083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40372,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331468,"statfs":{"total":96636764160,"available":96595423232,"internally_reserved":0,"allocated":884736,"data_stored":368699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1458176,"data_stored":956839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:54.587 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:16:54.587 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:16:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2fa200640 1 Processor -- start 2024-04-25T10:16:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2fa200640 1 -- start start 2024-04-25T10:16:54.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2fa200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb2f415bbb0 0x7fb2f40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2fa200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f406ca90 con 0x7fb2f40cf960 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2fa200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f406cd70 con 0x7fb2f40b4660 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2fa200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f406d050 con 0x7fb2f415bbb0 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb2f40cf960 0x7fb2f40b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36332/0 (socket says 172.21.15.87:36332) 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3e00640 1 -- 172.21.15.87:0/3327657835 learned_addr learned my addr 172.21.15.87:0/3327657835 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3400640 1 --2- 172.21.15.87:0/3327657835 >> v2:172.21.15.169:3300/0 conn(0x7fb2f415bbb0 0x7fb2f40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770399283 0 0) 0x7fb2f406ca90 con 0x7fb2f40cf960 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d8003530 con 0x7fb2f40cf960 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966731326 0 0) 0x7fb2f406cd70 con 0x7fb2f40b4660 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2f406ca90 con 0x7fb2f40b4660 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2132101255 0 0) 0x7fb2d8003530 con 0x7fb2f40cf960 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 >> v2:172.21.15.169:3300/0 conn(0x7fb2f415bbb0 msgr2=0x7fb2f40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 --2- 172.21.15.87:0/3327657835 >> v2:172.21.15.169:3300/0 conn(0x7fb2f415bbb0 0x7fb2f40cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 >> v1:172.21.15.169:6789/0 conn(0x7fb2f40b4660 legacy=0x7fb2f406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40b2880 con 0x7fb2f40cf960 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb2ec001a00 con 0x7fb2f40cf960 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3400640 1 --2- 172.21.15.87:0/3327657835 >> v2:172.21.15.169:3300/0 conn(0x7fb2f415bbb0 0x7fb2f40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 >> v1:172.21.15.87:6789/0 conn(0x7fb2f40cf960 legacy=0x7fb2f40b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003b50 0x7fb2d8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2d8004810 0x7fb2d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d80051d0 0x7fb2d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f406d050 con 0x7fb2d8004810 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d800c050 con 0x7fb2d80051d0 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d800c250 con 0x7fb2d8003b50 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2ebe00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2d8004810 0x7fb2d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3e00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d80051d0 0x7fb2d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3400640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003b50 0x7fb2d8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2ebe00640 1 -- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003b50 msgr2=0x7fb2d8003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2ebe00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003b50 0x7fb2d8003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2ebe00640 1 -- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d80051d0 msgr2=0x7fb2d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2ebe00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d80051d0 0x7fb2d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2ebe00640 1 -- 172.21.15.87:0/3327657835 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8003530 con 0x7fb2d8004810 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3e00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d80051d0 0x7fb2d800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.762+0000 7fb2f3e00640 1 -- 172.21.15.87:0/3327657835 reap_dead start 2024-04-25T10:16:54.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2ebe00640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2d8004810 0x7fb2d8004c90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40b4a80 tx=0x7fb2dc01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1fef2f2ea2dce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc001520 con 0x7fb2d8004810 2024-04-25T10:16:54.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc0052f0 con 0x7fb2d8004810 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2f2a00640 1 -- 172.21.15.87:0/3327657835 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0a8910 con 0x7fb2d8004810 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2fa200640 1 -- 172.21.15.87:0/3327657835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2d8004810 msgr2=0x7fb2d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2fa200640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2d8004810 0x7fb2d8004c90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40b4a80 tx=0x7fb2dc01ecb0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2fa200640 1 -- 172.21.15.87:0/3327657835 shutdown_connections 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2fa200640 1 --2- 172.21.15.87:0/3327657835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2d8004810 0x7fb2d8004c90 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2fa200640 1 -- 172.21.15.87:0/3327657835 >> 172.21.15.87:0/3327657835 conn(0x7fb2f4066b70 msgr2=0x7fb2f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2fa200640 1 -- 172.21.15.87:0/3327657835 shutdown_connections 2024-04-25T10:16:54.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.766+0000 7fb2fa200640 1 -- 172.21.15.87:0/3327657835 wait complete. 2024-04-25T10:16:54.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 Processor -- start 2024-04-25T10:16:54.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 -- start start 2024-04-25T10:16:54.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004810 0x7fb2f41481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2f4157960 0x7fb2f4157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f40712c0 0x7fb2f4071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f40cff90 con 0x7fb2f40712c0 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f40cfe10 con 0x7fb2d8004810 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f40d0110 con 0x7fb2f4157960 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f40712c0 0x7fb2f4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f40712c0 0x7fb2f4071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60472/0 (socket says 172.21.15.87:60472) 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 -- 172.21.15.87:0/2752288562 learned_addr learned my addr 172.21.15.87:0/2752288562 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2ebe00640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2f4157960 0x7fb2f4157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3400640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004810 0x7fb2f41481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 -- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2f4157960 msgr2=0x7fb2f4157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2f4157960 0x7fb2f4157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 -- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004810 msgr2=0x7fb2f41481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004810 0x7fb2f41481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 -- 172.21.15.87:0/2752288562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40706e0 con 0x7fb2f40712c0 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2ebe00640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2f4157960 0x7fb2f4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3400640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004810 0x7fb2f41481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2f3e00640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f40712c0 0x7fb2f4071740 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000f90 tx=0x7fb2ec00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2eb400640 1 -- 172.21.15.87:0/2752288562 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00eb30 con 0x7fb2f40712c0 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2eb400640 1 -- 172.21.15.87:0/2752288562 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec00ecd0 con 0x7fb2f40712c0 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2eb400640 1 -- 172.21.15.87:0/2752288562 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec014630 con 0x7fb2f40712c0 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f4148710 con 0x7fb2f40712c0 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.770+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f40c9080 con 0x7fb2f40712c0 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.774+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f404ef30 con 0x7fb2f40712c0 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.774+0000 7fb2eb400640 1 -- 172.21.15.87:0/2752288562 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2ec01b060 con 0x7fb2f40712c0 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.774+0000 7fb2eb400640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.774+0000 7fb2eb400640 1 -- 172.21.15.87:0/2752288562 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2ec084b70 con 0x7fb2f40712c0 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.774+0000 7fb2f3400640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.774+0000 7fb2f3400640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2d0066f60 0x7fb2d0069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009c20 tx=0x7fb2e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.778+0000 7fb2eb400640 1 -- 172.21.15.87:0/2752288562 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2ec01a2c0 con 0x7fb2f40712c0 2024-04-25T10:16:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.986+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb2f406d980 con 0x7fb2f40712c0 2024-04-25T10:16:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.986+0000 7fb2eb400640 1 -- 172.21.15.87:0/2752288562 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb2ec055a50 con 0x7fb2f40712c0 2024-04-25T10:16:54.994 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:16:54.994 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:16:54.994 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:16:54.994 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:16:54.994 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:16:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2d0066f60 msgr2=0x7fb2d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2d0066f60 0x7fb2d0069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009c20 tx=0x7fb2e4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f40712c0 msgr2=0x7fb2f4071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f40712c0 0x7fb2f4071740 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000f90 tx=0x7fb2ec00e530 comp rx=0 tx=0).stop 2024-04-25T10:16:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 shutdown_connections 2024-04-25T10:16:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f40712c0 0x7fb2f4071740 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2f4157960 0x7fb2f4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 --2- 172.21.15.87:0/2752288562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004810 0x7fb2f41481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 >> 172.21.15.87:0/2752288562 conn(0x7fb2f4066b70 msgr2=0x7fb2f415c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 shutdown_connections 2024-04-25T10:16:54.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:54.990+0000 7fb2fa200640 1 -- 172.21.15.87:0/2752288562 wait complete. 2024-04-25T10:16:55.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.142+0000 7f3a3ce00640 1 Processor -- start 2024-04-25T10:16:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.142+0000 7f3a3ce00640 1 -- start start 2024-04-25T10:16:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a3ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a380ce8c0 0x7f3a38166730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a3ce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a380b45f0 con 0x7f3a3815bdf0 2024-04-25T10:16:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a3ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a380d05d0 con 0x7f3a3815ba10 2024-04-25T10:16:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a3ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380b48b0 con 0x7f3a380ce8c0 2024-04-25T10:16:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a2ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3a3815bdf0 0x7f3a380ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36342/0 (socket says 172.21.15.87:36342) 2024-04-25T10:16:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a2ea00640 1 -- 172.21.15.87:0/3800441473 learned_addr learned my addr 172.21.15.87:0/3800441473 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:55.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a37e00640 1 --2- 172.21.15.87:0/3800441473 >> v2:172.21.15.169:3300/0 conn(0x7f3a380ce8c0 0x7f3a38166730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a36a00640 1 -- 172.21.15.87:0/3800441473 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706042875 0 0) 0x7f3a380b45f0 con 0x7f3a3815bdf0 2024-04-25T10:16:55.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a36a00640 1 -- 172.21.15.87:0/3800441473 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a1c003530 con 0x7f3a3815bdf0 2024-04-25T10:16:55.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a36a00640 1 -- 172.21.15.87:0/3800441473 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043662500 0 0) 0x7f3a380d05d0 con 0x7f3a3815ba10 2024-04-25T10:16:55.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a36a00640 1 -- 172.21.15.87:0/3800441473 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a380b45f0 con 0x7f3a3815ba10 2024-04-25T10:16:55.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a37e00640 1 -- 172.21.15.87:0/3800441473 >> v1:172.21.15.169:6789/0 conn(0x7f3a3815ba10 legacy=0x7f3a380cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a37e00640 1 -- 172.21.15.87:0/3800441473 >> v1:172.21.15.87:6789/0 conn(0x7f3a3815bdf0 legacy=0x7f3a380ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.146+0000 7f3a37e00640 1 -- 172.21.15.87:0/3800441473 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380b4b90 con 0x7f3a380ce8c0 2024-04-25T10:16:55.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a37e00640 1 --2- 172.21.15.87:0/3800441473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a380ce8c0 0x7f3a38166730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009d90 tx=0x7f3a3001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6f88d83f477aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a36a00640 1 -- 172.21.15.87:0/3800441473 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300b4070 con 0x7f3a380ce8c0 2024-04-25T10:16:55.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a36a00640 1 -- 172.21.15.87:0/3800441473 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a300af600 con 0x7f3a380ce8c0 2024-04-25T10:16:55.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a36a00640 1 -- 172.21.15.87:0/3800441473 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300af9d0 con 0x7f3a380ce8c0 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- 172.21.15.87:0/3800441473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a380ce8c0 msgr2=0x7f3a38166730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/3800441473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a380ce8c0 0x7f3a38166730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009d90 tx=0x7f3a3001f150 comp rx=0 tx=0).stop 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- 172.21.15.87:0/3800441473 shutdown_connections 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/3800441473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a380ce8c0 0x7f3a38166730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- 172.21.15.87:0/3800441473 >> 172.21.15.87:0/3800441473 conn(0x7f3a380d17a0 msgr2=0x7f3a380d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- 172.21.15.87:0/3800441473 shutdown_connections 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- 172.21.15.87:0/3800441473 wait complete. 2024-04-25T10:16:55.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 Processor -- start 2024-04-25T10:16:55.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- start start 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a380ce8c0 0x7f3a3815e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a3815ba10 0x7f3a3814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a3815bdf0 0x7f3a380cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380b6520 con 0x7f3a380ce8c0 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380b63a0 con 0x7f3a3815ba10 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a3ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380b66a0 con 0x7f3a3815bdf0 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a37400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a380ce8c0 0x7f3a3815e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a37400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a380ce8c0 0x7f3a3815e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60476/0 (socket says 172.21.15.87:60476) 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a37400640 1 -- 172.21.15.87:0/763327017 learned_addr learned my addr 172.21.15.87:0/763327017 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a2ea00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a3815ba10 0x7f3a3814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.150+0000 7f3a37e00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a3815bdf0 0x7f3a380cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a37400640 1 -- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a3815bdf0 msgr2=0x7f3a380cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a37400640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a3815bdf0 0x7f3a380cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a37400640 1 -- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a3815ba10 msgr2=0x7f3a3814fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a37400640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a3815ba10 0x7f3a3814fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a37400640 1 -- 172.21.15.87:0/763327017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380c9f80 con 0x7f3a380ce8c0 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a37400640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a380ce8c0 0x7f3a3815e8e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800c490 tx=0x7f3a2800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a2fe00640 1 -- 172.21.15.87:0/763327017 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a280052d0 con 0x7f3a380ce8c0 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a2fe00640 1 -- 172.21.15.87:0/763327017 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a28005470 con 0x7f3a380ce8c0 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a2fe00640 1 -- 172.21.15.87:0/763327017 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28012870 con 0x7f3a380ce8c0 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a380ca260 con 0x7f3a380ce8c0 2024-04-25T10:16:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a380cab00 con 0x7f3a380ce8c0 2024-04-25T10:16:55.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a38075120 con 0x7f3a380ce8c0 2024-04-25T10:16:55.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a2fe00640 1 -- 172.21.15.87:0/763327017 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a28012a10 con 0x7f3a380ce8c0 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a2fe00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a14066f90 0x7f3a14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a2fe00640 1 -- 172.21.15.87:0/763327017 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a28084bb0 con 0x7f3a380ce8c0 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.154+0000 7f3a2ea00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a14066f90 0x7f3a14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.158+0000 7f3a2ea00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a14066f90 0x7f3a14069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3a3815f1f0 tx=0x7f3a20007d40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.162+0000 7f3a2fe00640 1 -- 172.21.15.87:0/763327017 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a280554e0 con 0x7f3a380ce8c0 2024-04-25T10:16:55.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.346+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a380c7e90 con 0x7f3a380ce8c0 2024-04-25T10:16:55.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.346+0000 7f3a2fe00640 1 -- 172.21.15.87:0/763327017 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f3a28059580 con 0x7f3a380ce8c0 2024-04-25T10:16:55.354 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:16:55.354 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/817977905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":817977905},{"type":"v1","addr":"172.21.15.169:6835","nonce":817977905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/979883306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":979883306},{"type":"v1","addr":"172.21.15.87:6836","nonce":979883306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/796485952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":796485952},{"type":"v1","addr":"172.21.15.87:6837","nonce":796485952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:49.500890+0000","modified":"2024-04-25T10:16:52.508879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"d","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1643726385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1643726385},{"type":"v1","addr":"172.21.15.169:6834","nonce":1643726385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:16:55.354 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 53 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a14066f90 msgr2=0x7f3a14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a14066f90 0x7f3a14069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3a3815f1f0 tx=0x7f3a20007d40 comp rx=0 tx=0).stop 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a380ce8c0 msgr2=0x7f3a3815e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a380ce8c0 0x7f3a3815e8e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800c490 tx=0x7f3a2800c950 comp rx=0 tx=0).stop 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 shutdown_connections 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a14066f90 0x7f3a14069450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a3815bdf0 0x7f3a380cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a3815ba10 0x7f3a3814fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 --2- 172.21.15.87:0/763327017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a380ce8c0 0x7f3a3815e8e0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 >> 172.21.15.87:0/763327017 conn(0x7f3a380d17a0 msgr2=0x7f3a380b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 shutdown_connections 2024-04-25T10:16:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.350+0000 7f3a3ce00640 1 -- 172.21.15.87:0/763327017 wait complete. 2024-04-25T10:16:55.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:49.500890+0000', 'modified': '2024-04-25T10:16:52.508879+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7020}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7020': {'gid': 7020, 'name': 'd', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6834/1643726385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 1643726385}, {'type': 'v1', 'addr': '172.21.15.169:6834', 'nonce': 1643726385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:55.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:55.779 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:16:55.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc9600640 1 Processor -- start 2024-04-25T10:16:55.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc9600640 1 -- start start 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc415df60 0x7f5fc40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc9600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5fc40d0180 0x7f5fc40d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc9600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc40d0b20 con 0x7f5fc40cdb40 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc9600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc415e340 con 0x7f5fc40d0180 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40ccaa0 con 0x7f5fc415df60 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5fc40d0180 0x7f5fc40d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5fc40cdb40 0x7f5fc40cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36348/0 (socket says 172.21.15.87:36348) 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc3400640 1 -- 172.21.15.87:0/2202876557 learned_addr learned my addr 172.21.15.87:0/2202876557 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:55.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2202876557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc415df60 0x7f5fc40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc2000640 1 -- 172.21.15.87:0/2202876557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc415df60 msgr2=0x7f5fc40cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc2000640 1 --2- 172.21.15.87:0/2202876557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc415df60 0x7f5fc40cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc2000640 1 -- 172.21.15.87:0/2202876557 >> v1:172.21.15.87:6789/0 conn(0x7f5fc40cdb40 legacy=0x7f5fc40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc2000640 1 -- 172.21.15.87:0/2202876557 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40ccd80 con 0x7f5fc40d0180 2024-04-25T10:16:55.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.910+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2202876557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc415df60 0x7f5fc40cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:55.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc2000640 1 --2- 172.21.15.87:0/2202876557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc40d0180 0x7f5fc40d05e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00097c0 tx=0x7f5fb001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75576383a35df14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc1600640 1 -- 172.21.15.87:0/2202876557 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00b0070 con 0x7f5fc40d0180 2024-04-25T10:16:55.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc1600640 1 -- 172.21.15.87:0/2202876557 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb00ab600 con 0x7f5fc40d0180 2024-04-25T10:16:55.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc1600640 1 -- 172.21.15.87:0/2202876557 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00ab9d0 con 0x7f5fc40d0180 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 -- 172.21.15.87:0/2202876557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc40d0180 msgr2=0x7f5fc40d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2202876557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc40d0180 0x7f5fc40d05e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00097c0 tx=0x7f5fb001c410 comp rx=0 tx=0).stop 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 -- 172.21.15.87:0/2202876557 shutdown_connections 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2202876557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc40d0180 0x7f5fc40d05e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2202876557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc415df60 0x7f5fc40cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 -- 172.21.15.87:0/2202876557 >> 172.21.15.87:0/2202876557 conn(0x7f5fc40d1910 msgr2=0x7f5fc40d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 -- 172.21.15.87:0/2202876557 shutdown_connections 2024-04-25T10:16:55.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 -- 172.21.15.87:0/2202876557 wait complete. 2024-04-25T10:16:55.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 Processor -- start 2024-04-25T10:16:55.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.914+0000 7f5fc9600640 1 -- start start 2024-04-25T10:16:55.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc40cdb40 0x7f5fc4152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40d0180 0x7f5fc406e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc415df60 0x7f5fc406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40d0ff0 con 0x7f5fc415df60 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40d0e70 con 0x7f5fc40cdb40 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40cead0 con 0x7f5fc40d0180 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc415df60 0x7f5fc406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc415df60 0x7f5fc406eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60504/0 (socket says 172.21.15.87:60504) 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 -- 172.21.15.87:0/2210611013 learned_addr learned my addr 172.21.15.87:0/2210611013 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc2000640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40d0180 0x7f5fc406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc40cdb40 0x7f5fc4152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 -- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40d0180 msgr2=0x7f5fc406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40d0180 0x7f5fc406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 -- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc40cdb40 msgr2=0x7f5fc4152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc40cdb40 0x7f5fc4152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 -- 172.21.15.87:0/2210611013 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc415b2b0 con 0x7f5fc415df60 2024-04-25T10:16:55.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc3400640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc415df60 0x7f5fc406eaa0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00cbc0 tx=0x7f5fbc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fbaa00640 1 -- 172.21.15.87:0/2210611013 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc017070 con 0x7f5fc415df60 2024-04-25T10:16:55.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fbaa00640 1 -- 172.21.15.87:0/2210611013 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fbc00ad40 con 0x7f5fc415df60 2024-04-25T10:16:55.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fbaa00640 1 -- 172.21.15.87:0/2210611013 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc012600 con 0x7f5fc415df60 2024-04-25T10:16:55.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc415b590 con 0x7f5fc415df60 2024-04-25T10:16:55.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.918+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc4152590 con 0x7f5fc415df60 2024-04-25T10:16:55.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fbaa00640 1 -- 172.21.15.87:0/2210611013 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fbc01a070 con 0x7f5fc415df60 2024-04-25T10:16:55.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fbaa00640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5fa4066f60 0x7f5fa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fbaa00640 1 -- 172.21.15.87:0/2210611013 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fbc084ca0 con 0x7f5fc415df60 2024-04-25T10:16:55.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5fa4066f60 0x7f5fa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 Processor -- start 2024-04-25T10:16:55.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 -- start start 2024-04-25T10:16:55.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5fa4066f60 0x7f5fa4069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4006e90 tx=0x7f5fb4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e17fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e180240 0x55e16e1359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e135f30 0x55e16e136310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16e01d5f0 con 0x55e16e17f920 2024-04-25T10:16:55.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16df84da0 con 0x55e16e180240 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16df0d150 con 0x55e16e135f30 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e17fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e17fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60514/0 (socket says 172.21.15.87:60514) 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 -- 172.21.15.87:0/1173947906 learned_addr learned my addr 172.21.15.87:0/1173947906 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2000640 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e180240 0x55e16e1359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc3400640 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e135f30 0x55e16e136310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 -- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e135f30 msgr2=0x55e16e136310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e135f30 0x55e16e136310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 -- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e180240 msgr2=0x55e16e1359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e180240 0x55e16e1359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 -- 172.21.15.87:0/1173947906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16df26ca0 con 0x55e16e17f920 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc3400640 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e135f30 0x55e16e136310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e17fd00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4010ad0 tx=0x7f5fb4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1977fedf441c66e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5faf400640 1 -- 172.21.15.87:0/1173947906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb401c050 con 0x55e16e17f920 2024-04-25T10:16:55.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5faf400640 1 -- 172.21.15.87:0/1173947906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb4026070 con 0x55e16e17f920 2024-04-25T10:16:55.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.922+0000 7f5faf400640 1 -- 172.21.15.87:0/1173947906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4021440 con 0x55e16e17f920 2024-04-25T10:16:55.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- 172.21.15.87:0/1173947906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 msgr2=0x55e16e17fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e17fd00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4010ad0 tx=0x7f5fb4013000 comp rx=0 tx=0).stop 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- 172.21.15.87:0/1173947906 shutdown_connections 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e135f30 0x55e16e136310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e180240 0x55e16e1359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 --2- 172.21.15.87:0/1173947906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e17fd00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- 172.21.15.87:0/1173947906 >> 172.21.15.87:0/1173947906 conn(0x55e16e13c2a0 msgr2=0x55e16e1342d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- 172.21.15.87:0/1173947906 shutdown_connections 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- 172.21.15.87:0/1173947906 wait complete. 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- start start 2024-04-25T10:16:55.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 Processor -- start 2024-04-25T10:16:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e186c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e17f920 0x55e16e1871c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e180240 0x55e16e139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16df3f990 con 0x55e16e180240 2024-04-25T10:16:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16df80e40 con 0x55e16e135f30 2024-04-25T10:16:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16df10900 con 0x55e16e17f920 2024-04-25T10:16:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 10 client.0 _ll_register_callbacks cb 0x55e16e0381c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e180240 0x55e16e139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e180240 0x55e16e139aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60516/0 (socket says 172.21.15.87:60516) 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc2000640 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e17f920 0x55e16e1871c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 -- 172.21.15.87:0/519914249 learned_addr learned my addr 172.21.15.87:0/519914249 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e186c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 -- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e17f920 msgr2=0x55e16e1871c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e17f920 0x55e16e1871c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 -- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 msgr2=0x55e16e186c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e186c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 -- 172.21.15.87:0/519914249 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e16df3d630 con 0x55e16e180240 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fc3400640 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e180240 0x55e16e139aa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005c30 tx=0x7f5fbc055710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd3c1cab1635217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fa3e00640 1 -- 172.21.15.87:0/519914249 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc05da20 con 0x55e16e180240 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fa3e00640 1 -- 172.21.15.87:0/519914249 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fbc004b70 con 0x55e16e180240 2024-04-25T10:16:55.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fb4026070 con 0x55e16e180240 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fa3e00640 1 -- 172.21.15.87:0/519914249 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fbc05dcb0 con 0x55e16e180240 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 10 client.7155 fetch_fsmap learned FSMap version 53 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55e16df0d3a0 con 0x55e16e180240 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fa3e00640 1 -- 172.21.15.87:0/519914249 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb4026070 con 0x55e16e180240 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fa3e00640 1 -- 172.21.15.87:0/519914249 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5fbc054aa0 con 0x55e16e180240 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.926+0000 7f5fcca40000 10 client.7155 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 10 client.7155 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 20 client.7155 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e140af0 0x55e16e13a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 4 client.7155 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e16df20a50 con 0x55e16e140af0 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e140af0 0x55e16e13a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e140af0 0x55e16e13a4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caef4293183dd83 server_cookie=3a9cf5da432ea015 in_seq=0 out_seq=0 2024-04-25T10:16:55.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fa3e00640 10 client.7155 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fa3e00640 1 -- 172.21.15.87:0/519914249 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e16df20a50 con 0x55e16e140af0 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fa3e00640 10 client.7155 handle_command_reply: tid=0 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 client.7155 shutdown 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e180240 msgr2=0x55e16e139aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e180240 0x55e16e139aa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005c30 tx=0x7f5fbc055710 comp rx=0 tx=0).stop 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 shutdown_connections 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e140af0 0x55e16e13a4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e180240 0x55e16e139aa0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e17f920 0x55e16e1871c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 --2- 172.21.15.87:0/519914249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e186c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fa3e00640 0 client.7155 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:16:55.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 >> 172.21.15.87:0/519914249 conn(0x55e16e13c2a0 msgr2=0x55e16e1349e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 shutdown_connections 2024-04-25T10:16:55.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 1 -- 172.21.15.87:0/519914249 wait complete. 2024-04-25T10:16:55.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.930+0000 7f5fcca40000 20 client.7155 trim_cache size 0 max 16384 2024-04-25T10:16:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 Processor -- start 2024-04-25T10:16:55.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 -- start start 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e139470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 0x55e16e1bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc4202870 con 0x55e16e17f920 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40b5cd0 con 0x55e16e135f30 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41535c0 con 0x55e16e180240 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60518/0 (socket says 172.21.15.87:60518) 2024-04-25T10:16:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 -- 172.21.15.87:0/157678344 learned_addr learned my addr 172.21.15.87:0/157678344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 -- 172.21.15.87:0/157678344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 msgr2=0x55e16e1bf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 --2- 172.21.15.87:0/157678344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 0x55e16e1bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 -- 172.21.15.87:0/157678344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 msgr2=0x55e16e139470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 --2- 172.21.15.87:0/157678344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 -- 172.21.15.87:0/157678344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16e180070 con 0x55e16e17f920 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5fc2000640 1 --2- 172.21.15.87:0/157678344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1399b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00097c0 tx=0x7f5fb0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f74b9a232bded85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.946+0000 7f5faea00640 1 -- 172.21.15.87:0/157678344 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00bb020 con 0x55e16e17f920 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5faea00640 1 -- 172.21.15.87:0/157678344 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb00ab8b0 con 0x55e16e17f920 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5faea00640 1 -- 172.21.15.87:0/157678344 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00b0040 con 0x55e16e17f920 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- 172.21.15.87:0/157678344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 msgr2=0x55e16e1399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 --2- 172.21.15.87:0/157678344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1399b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00097c0 tx=0x7f5fb0004740 comp rx=0 tx=0).stop 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- 172.21.15.87:0/157678344 shutdown_connections 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 --2- 172.21.15.87:0/157678344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 0x55e16e1bf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 --2- 172.21.15.87:0/157678344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1399b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 --2- 172.21.15.87:0/157678344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e139470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- 172.21.15.87:0/157678344 >> 172.21.15.87:0/157678344 conn(0x55e16e13c2a0 msgr2=0x55e16e196050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- 172.21.15.87:0/157678344 shutdown_connections 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- 172.21.15.87:0/157678344 wait complete. 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- start start 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 Processor -- start 2024-04-25T10:16:55.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e19b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1e0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 0x55e16e1e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16df893d0 con 0x55e16e17f920 2024-04-25T10:16:55.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16e040d50 con 0x55e16e135f30 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16df1f980 con 0x55e16e180240 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 10 client.0 _ll_register_callbacks cb 0x7f5fb4007630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1e0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60530/0 (socket says 172.21.15.87:60530) 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 -- 172.21.15.87:0/2931843600 learned_addr learned my addr 172.21.15.87:0/2931843600 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc3400640 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 0x55e16e1e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 -- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 msgr2=0x55e16e1e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 0x55e16e1e08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 -- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 msgr2=0x55e16e19b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e19b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 -- 172.21.15.87:0/2931843600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e16e006600 con 0x55e16e17f920 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fc2000640 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1e0360 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0008010 tx=0x7f5fb001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c922f91709a89b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fa3e00640 1 -- 172.21.15.87:0/2931843600 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00bb030 con 0x55e16e17f920 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fa3e00640 1 -- 172.21.15.87:0/2931843600 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fb00abce0 con 0x55e16e17f920 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fb00ab8b0 con 0x55e16e17f920 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fa3e00640 1 -- 172.21.15.87:0/2931843600 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fb00ce050 con 0x55e16e17f920 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 10 client.7161 fetch_fsmap learned FSMap version 53 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.950+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55e16df199b0 con 0x55e16e17f920 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fa3e00640 1 -- 172.21.15.87:0/2931843600 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb00ab8b0 con 0x55e16e17f920 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fa3e00640 1 -- 172.21.15.87:0/2931843600 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5fb00d1020 con 0x55e16e17f920 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 10 client.7161 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 10 client.7161 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 20 client.7161 populate_metadata read hostname 'smithi087' 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e1e0ea0 0x55e16e194880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 4 client.7161 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e16df922e0 con 0x55e16e1e0ea0 2024-04-25T10:16:55.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e1e0ea0 0x55e16e194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fa3e00640 10 client.7161 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc2a00640 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e1e0ea0 0x55e16e194880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5a0ab55182660f server_cookie=206e350144fb79da in_seq=0 out_seq=0 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fa3e00640 1 -- 172.21.15.87:0/2931843600 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e16df922e0 con 0x55e16e1e0ea0 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fa3e00640 10 client.7161 handle_command_reply: tid=0 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 client.7161 shutdown 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 msgr2=0x55e16e1e0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1e0360 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0008010 tx=0x7f5fb001ac50 comp rx=0 tx=0).stop 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 shutdown_connections 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55e16e1e0ea0 0x55e16e194880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e16e180240 0x55e16e1e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e16e17f920 0x55e16e1e0360 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 --2- 172.21.15.87:0/2931843600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e16e135f30 0x55e16e19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 >> 172.21.15.87:0/2931843600 conn(0x55e16e13c2a0 msgr2=0x55e16e18fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fa3e00640 0 client.7161 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 shutdown_connections 2024-04-25T10:16:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 1 -- 172.21.15.87:0/2931843600 wait complete. 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fcca40000 20 client.7161 trim_cache size 0 max 16384 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 7020, 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8244251509999998, 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 53, 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 72, 2024-04-25T10:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 72, 2024-04-25T10:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.678663761999999 2024-04-25T10:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:16:55.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5fa4066f60 msgr2=0x7f5fa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5fa4066f60 0x7f5fa4069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4006e90 tx=0x7f5fb4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc415df60 msgr2=0x7f5fc406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc415df60 0x7f5fc406eaa0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00cbc0 tx=0x7f5fbc00a430 comp rx=0 tx=0).stop 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 shutdown_connections 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5fa4066f60 0x7f5fa4069420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fc415df60 0x7f5fc406eaa0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40d0180 0x7f5fc406e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 --2- 172.21.15.87:0/2210611013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc40cdb40 0x7f5fc4152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 >> 172.21.15.87:0/2210611013 conn(0x7f5fc40d1910 msgr2=0x7f5fc40cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 shutdown_connections 2024-04-25T10:16:55.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:55.954+0000 7f5fc9600640 1 -- 172.21.15.87:0/2210611013 wait complete. 2024-04-25T10:16:55.980 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:56.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.106+0000 7f3794000640 1 Processor -- start 2024-04-25T10:16:56.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.106+0000 7f3794000640 1 -- start start 2024-04-25T10:16:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3794000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3794000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f378c0d0180 0x7f378c0d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3794000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c0d0b20 con 0x7f378c0cdb40 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3794000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c15e340 con 0x7f378c15df60 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3794000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c0ccaa0 con 0x7f378c0d0180 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3791600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3792000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f378c0cdb40 0x7f378c0cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36364/0 (socket says 172.21.15.87:36364) 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3792000640 1 -- 172.21.15.87:0/1423437275 learned_addr learned my addr 172.21.15.87:0/1423437275 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3790c00640 1 --2- 172.21.15.87:0/1423437275 >> v2:172.21.15.169:3300/0 conn(0x7f378c0d0180 0x7f378c0d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3791600640 1 -- 172.21.15.87:0/1423437275 >> v2:172.21.15.169:3300/0 conn(0x7f378c0d0180 msgr2=0x7f378c0d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3791600640 1 --2- 172.21.15.87:0/1423437275 >> v2:172.21.15.169:3300/0 conn(0x7f378c0d0180 0x7f378c0d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3791600640 1 -- 172.21.15.87:0/1423437275 >> v1:172.21.15.87:6789/0 conn(0x7f378c0cdb40 legacy=0x7f378c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f378be00640 1 -- 172.21.15.87:0/1423437275 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747571431 0 0) 0x7f378c0d0b20 con 0x7f378c0cdb40 2024-04-25T10:16:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.110+0000 7f3791600640 1 -- 172.21.15.87:0/1423437275 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0ccd80 con 0x7f378c15df60 2024-04-25T10:16:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3791600640 1 --2- 172.21.15.87:0/1423437275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c0cfc40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f378c075210 tx=0x7f378001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a90e00b10882a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f378be00640 1 -- 172.21.15.87:0/1423437275 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37800b0070 con 0x7f378c15df60 2024-04-25T10:16:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f378be00640 1 -- 172.21.15.87:0/1423437275 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37800ab600 con 0x7f378c15df60 2024-04-25T10:16:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f378be00640 1 -- 172.21.15.87:0/1423437275 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37800ab9d0 con 0x7f378c15df60 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 -- 172.21.15.87:0/1423437275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 msgr2=0x7f378c0cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 --2- 172.21.15.87:0/1423437275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c0cfc40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f378c075210 tx=0x7f378001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 -- 172.21.15.87:0/1423437275 shutdown_connections 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 --2- 172.21.15.87:0/1423437275 >> v2:172.21.15.169:3300/0 conn(0x7f378c0d0180 0x7f378c0d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 --2- 172.21.15.87:0/1423437275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 -- 172.21.15.87:0/1423437275 >> 172.21.15.87:0/1423437275 conn(0x7f378c0d1910 msgr2=0x7f378c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 -- 172.21.15.87:0/1423437275 shutdown_connections 2024-04-25T10:16:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 -- 172.21.15.87:0/1423437275 wait complete. 2024-04-25T10:16:56.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 Processor -- start 2024-04-25T10:16:56.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.114+0000 7f3794000640 1 -- start start 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f378c0cdb40 0x7f378c15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f378c0d0180 0x7f378c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0d0ff0 con 0x7f378c15df60 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0d0e70 con 0x7f378c0cdb40 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c16aa20 con 0x7f378c0d0180 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3791600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f378c0cdb40 0x7f378c15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60538/0 (socket says 172.21.15.87:60538) 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 -- 172.21.15.87:0/1098562145 learned_addr learned my addr 172.21.15.87:0/1098562145 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:56.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3790c00640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f378c0d0180 0x7f378c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 -- 172.21.15.87:0/1098562145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f378c0d0180 msgr2=0x7f378c1538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f378c0d0180 0x7f378c1538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 -- 172.21.15.87:0/1098562145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f378c0cdb40 msgr2=0x7f378c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f378c0cdb40 0x7f378c15cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 -- 172.21.15.87:0/1098562145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0b77a0 con 0x7f378c15df60 2024-04-25T10:16:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3792000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c15ba90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f378400c7a0 tx=0x7f378400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3789600640 1 -- 172.21.15.87:0/1098562145 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784018070 con 0x7f378c15df60 2024-04-25T10:16:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3789600640 1 -- 172.21.15.87:0/1098562145 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400a640 con 0x7f378c15df60 2024-04-25T10:16:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3789600640 1 -- 172.21.15.87:0/1098562145 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400aa70 con 0x7f378c15df60 2024-04-25T10:16:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c0b6e70 con 0x7f378c15df60 2024-04-25T10:16:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c15b290 con 0x7f378c15df60 2024-04-25T10:16:56.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.118+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c04ef30 con 0x7f378c15df60 2024-04-25T10:16:56.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.122+0000 7f3789600640 1 -- 172.21.15.87:0/1098562145 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f378401b070 con 0x7f378c15df60 2024-04-25T10:16:56.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.122+0000 7f3789600640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f376c066f60 0x7f376c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:56.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.122+0000 7f3789600640 1 -- 172.21.15.87:0/1098562145 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3784084550 con 0x7f378c15df60 2024-04-25T10:16:56.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.122+0000 7f3791600640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f376c066f60 0x7f376c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:56.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.122+0000 7f3791600640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f376c066f60 0x7f376c069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f37800ad000 tx=0x7f37800a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:56.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.126+0000 7f3789600640 1 -- 172.21.15.87:0/1098562145 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3784054f10 con 0x7f378c15df60 2024-04-25T10:16:56.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f378c15d310 con 0x7f378c15df60 2024-04-25T10:16:56.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3789600640 1 -- 172.21.15.87:0/1098562145 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3784058fb0 con 0x7f378c15df60 2024-04-25T10:16:56.324 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f376c066f60 msgr2=0x7f376c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f376c066f60 0x7f376c069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f37800ad000 tx=0x7f37800a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 msgr2=0x7f378c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c15ba90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f378400c7a0 tx=0x7f378400cc60 comp rx=0 tx=0).stop 2024-04-25T10:16:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 shutdown_connections 2024-04-25T10:16:56.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f376c066f60 0x7f376c069420 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f378c15df60 0x7f378c15ba90 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f378c0d0180 0x7f378c1538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 --2- 172.21.15.87:0/1098562145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f378c0cdb40 0x7f378c15cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 >> 172.21.15.87:0/1098562145 conn(0x7f378c0d1910 msgr2=0x7f378c15a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:56.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 shutdown_connections 2024-04-25T10:16:56.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:56.318+0000 7f3794000640 1 -- 172.21.15.87:0/1098562145 wait complete. 2024-04-25T10:16:56.345 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:16:56.345 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:56.345 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:16:56.345 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:56.345 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:56.345 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:16:56.345 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:16:56.346 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:56.346 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:56.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.346 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:16:56.355 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:56.355 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:56.356 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.357 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:56.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.358 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:56.415 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:16:56.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.418 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:56.479 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:56.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.480 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:16:56.480 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:16:56.480 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:16:56.480 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:16:56.480 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:16:56.480 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:16:56.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.571 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:16:56.579 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:56.579 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:56.580 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:56.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.580 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:16:56.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:56.642 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:56.690 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:56.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:56.690 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:56.737 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:16:56.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.738 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.825 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.825 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:56.825 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:56.840 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.840 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:56.840 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:56.840 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:16:56.840 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:56.848 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.849 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:56.849 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:16:56.849 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:16:56.849 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:56.849 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:16:56.849 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:16:56.849 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:16:56.852 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:16:56.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.853 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:56.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.873 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:16:56.937 INFO:teuthology.orchestra.run.smithi087.stdout:client7173 2024-04-25T10:16:56.937 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:56.938 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:56.986 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:16:57.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71efc00640 1 Processor -- start 2024-04-25T10:16:57.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71efc00640 1 -- start start 2024-04-25T10:16:57.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71efc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71e806b800 0x7f71e80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71efc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e80cfdb0 con 0x7f71e806bbe0 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71efc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e80d0090 con 0x7f71e806b0d0 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71efc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e80cd030 con 0x7f71e806b800 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71edc00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f71e806bbe0 0x7f71e815d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36370/0 (socket says 172.21.15.87:36370) 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71edc00640 1 -- 172.21.15.87:0/2789321423 learned_addr learned my addr 172.21.15.87:0/2789321423 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71ed200640 1 --2- 172.21.15.87:0/2789321423 >> v2:172.21.15.169:3300/0 conn(0x7f71e806b800 0x7f71e80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71e7400640 1 -- 172.21.15.87:0/2789321423 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732121220 0 0) 0x7f71e80cfdb0 con 0x7f71e806bbe0 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71e7400640 1 -- 172.21.15.87:0/2789321423 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d0003530 con 0x7f71e806bbe0 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71ed200640 1 -- 172.21.15.87:0/2789321423 >> v1:172.21.15.169:6789/0 conn(0x7f71e806b0d0 legacy=0x7f71e80cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71ed200640 1 -- 172.21.15.87:0/2789321423 >> v1:172.21.15.87:6789/0 conn(0x7f71e806bbe0 legacy=0x7f71e815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.166+0000 7f71ed200640 1 -- 172.21.15.87:0/2789321423 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80d0370 con 0x7f71e806b800 2024-04-25T10:16:57.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71ed200640 1 --2- 172.21.15.87:0/2789321423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 0x7f71e80ceb80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71e80cd1b0 tx=0x7f71dc01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb4f819a748dade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:57.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71e7400640 1 -- 172.21.15.87:0/2789321423 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0b5070 con 0x7f71e806b800 2024-04-25T10:16:57.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71e7400640 1 -- 172.21.15.87:0/2789321423 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71dc0a8990 con 0x7f71e806b800 2024-04-25T10:16:57.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71e7400640 1 -- 172.21.15.87:0/2789321423 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0a8d60 con 0x7f71e806b800 2024-04-25T10:16:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71efc00640 1 -- 172.21.15.87:0/2789321423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 msgr2=0x7f71e80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/2789321423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 0x7f71e80ceb80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71e80cd1b0 tx=0x7f71dc01ec90 comp rx=0 tx=0).stop 2024-04-25T10:16:57.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71efc00640 1 -- 172.21.15.87:0/2789321423 shutdown_connections 2024-04-25T10:16:57.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/2789321423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 0x7f71e80ceb80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71efc00640 1 -- 172.21.15.87:0/2789321423 >> 172.21.15.87:0/2789321423 conn(0x7f71e80d19d0 msgr2=0x7f71e80d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:57.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71efc00640 1 -- 172.21.15.87:0/2789321423 shutdown_connections 2024-04-25T10:16:57.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.170+0000 7f71efc00640 1 -- 172.21.15.87:0/2789321423 wait complete. 2024-04-25T10:16:57.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 Processor -- start 2024-04-25T10:16:57.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 -- start start 2024-04-25T10:16:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e806b0d0 0x7f71e815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 0x7f71e815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71e806bbe0 0x7f71e80c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80d0e50 con 0x7f71e806bbe0 2024-04-25T10:16:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80d0930 con 0x7f71e806b800 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80b0f80 con 0x7f71e806b0d0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71e806bbe0 0x7f71e80c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71e806bbe0 0x7f71e80c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60554/0 (socket says 172.21.15.87:60554) 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 -- 172.21.15.87:0/520091152 learned_addr learned my addr 172.21.15.87:0/520091152 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71e7e00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 0x7f71e815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71ed200640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e806b0d0 0x7f71e815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 -- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e806b0d0 msgr2=0x7f71e815bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e806b0d0 0x7f71e815bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 -- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 msgr2=0x7f71e815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 0x7f71e815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 -- 172.21.15.87:0/520091152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80b1260 con 0x7f71e806bbe0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71edc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71e806bbe0 0x7f71e80c7d70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f71e000cad0 tx=0x7f71e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71e4c00640 1 -- 172.21.15.87:0/520091152 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0017070 con 0x7f71e806bbe0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71e4c00640 1 -- 172.21.15.87:0/520091152 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e000ac30 con 0x7f71e806bbe0 2024-04-25T10:16:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71e4c00640 1 -- 172.21.15.87:0/520091152 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0012400 con 0x7f71e806bbe0 2024-04-25T10:16:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e80b1540 con 0x7f71e806bbe0 2024-04-25T10:16:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.174+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e80b1b00 con 0x7f71e806bbe0 2024-04-25T10:16:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.178+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e804ef30 con 0x7f71e806bbe0 2024-04-25T10:16:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.178+0000 7f71e4c00640 1 -- 172.21.15.87:0/520091152 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71e001a070 con 0x7f71e806bbe0 2024-04-25T10:16:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.178+0000 7f71e4c00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f71c8066f60 0x7f71c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.178+0000 7f71e4c00640 1 -- 172.21.15.87:0/520091152 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71e0084d20 con 0x7f71e806bbe0 2024-04-25T10:16:57.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.178+0000 7f71ed200640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f71c8066f60 0x7f71c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.178+0000 7f71ed200640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f71c8066f60 0x7f71c8069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b2000 tx=0x7f71dc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.182+0000 7f71e4c00640 1 -- 172.21.15.87:0/520091152 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71e00556d0 con 0x7f71e806bbe0 2024-04-25T10:16:57.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:57.370+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71e80b7950 con 0x7f71e806bbe0 2024-04-25T10:16:58.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.166+0000 7f71e4c00640 1 -- 172.21.15.87:0/520091152 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v54) v1 ==== 167+0+0 (secure 0 0 0) 0x7f71e0059770 con 0x7f71e806bbe0 2024-04-25T10:16:58.174 INFO:teuthology.orchestra.run.smithi087.stderr:enabled new snapshots 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f71c8066f60 msgr2=0x7f71c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f71c8066f60 0x7f71c8069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b2000 tx=0x7f71dc008040 comp rx=0 tx=0).stop 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71e806bbe0 msgr2=0x7f71e80c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71e806bbe0 0x7f71e80c7d70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f71e000cad0 tx=0x7f71e000cf90 comp rx=0 tx=0).stop 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 shutdown_connections 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f71c8066f60 0x7f71c8069420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71e806bbe0 0x7f71e80c7d70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e806b800 0x7f71e815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 --2- 172.21.15.87:0/520091152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e806b0d0 0x7f71e815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 >> 172.21.15.87:0/520091152 conn(0x7f71e80d19d0 msgr2=0x7f71e80ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 shutdown_connections 2024-04-25T10:16:58.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:16:58.170+0000 7f71efc00640 1 -- 172.21.15.87:0/520091152 wait complete. 2024-04-25T10:16:58.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:58.198 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:16:58.215 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:16:58.215 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> import uuid 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> n = 2000 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> create_hard_links = False 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> if 3 > 0: 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> create_hard_links = True 2024-04-25T10:16:58.267 DEBUG:teuthology.orchestra.run.smithi087:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> dpath = os.path.dirname(path) 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> fnameprefix = os.path.basename(path) 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> for i in range(n): 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:16:58.268 DEBUG:teuthology.orchestra.run.smithi087:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> f.write(f"{i}") 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> f.flush() 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(f.fileno()) 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> os.unlink(fpath) 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(dirfd) 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> if create_hard_links: 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> for j in range(3): 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(dirfd) 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> finally: 2024-04-25T10:16:58.269 DEBUG:teuthology.orchestra.run.smithi087:> os.close(dirfd) 2024-04-25T10:16:58.270 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:17:21.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:21.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:21.105 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:17:21.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:21.165 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:17:22.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:22.348 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:49.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:49.663 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:17:49.706 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:17:49.706 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:17:49.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7067600640 1 Processor -- start 2024-04-25T10:17:49.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7067600640 1 -- start start 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7067600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f706006bd10 0x7f7060161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7067600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7060161e50 0x7f706015d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7067600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706006ac90 con 0x7f70600cd9a0 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7067600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70601622a0 con 0x7f706006bd10 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7067600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706006b880 con 0x7f7060161e50 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f705fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f706006bd10 0x7f7060161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f705fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f706006bd10 0x7f7060161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36294/0 (socket says 172.21.15.87:36294) 2024-04-25T10:17:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f705fe00640 1 -- 172.21.15.87:0/139375161 learned_addr learned my addr 172.21.15.87:0/139375161 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:49.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f705fe00640 1 -- 172.21.15.87:0/139375161 >> v2:172.21.15.169:3300/0 conn(0x7f7060161e50 msgr2=0x7f706015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7065600640 1 --2- 172.21.15.87:0/139375161 >> v2:172.21.15.169:3300/0 conn(0x7f7060161e50 0x7f706015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f705fe00640 1 --2- 172.21.15.87:0/139375161 >> v2:172.21.15.169:3300/0 conn(0x7f7060161e50 0x7f706015d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f705fe00640 1 -- 172.21.15.87:0/139375161 >> v1:172.21.15.169:6789/0 conn(0x7f70600cd9a0 legacy=0x7f70600d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f705fe00640 1 -- 172.21.15.87:0/139375161 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706006af50 con 0x7f706006bd10 2024-04-25T10:17:49.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.901+0000 7f7065600640 1 --2- 172.21.15.87:0/139375161 >> v2:172.21.15.169:3300/0 conn(0x7f7060161e50 0x7f706015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:49.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f705fe00640 1 --2- 172.21.15.87:0/139375161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f706006bd10 0x7f7060161910 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f70601629b0 tx=0x7f704c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf8224c6f496abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f705f400640 1 -- 172.21.15.87:0/139375161 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0b5070 con 0x7f706006bd10 2024-04-25T10:17:49.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f705f400640 1 -- 172.21.15.87:0/139375161 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c0a89d0 con 0x7f706006bd10 2024-04-25T10:17:49.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f705f400640 1 -- 172.21.15.87:0/139375161 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0a8da0 con 0x7f706006bd10 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 -- 172.21.15.87:0/139375161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f706006bd10 msgr2=0x7f7060161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 --2- 172.21.15.87:0/139375161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f706006bd10 0x7f7060161910 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f70601629b0 tx=0x7f704c01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 -- 172.21.15.87:0/139375161 shutdown_connections 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 --2- 172.21.15.87:0/139375161 >> v2:172.21.15.169:3300/0 conn(0x7f7060161e50 0x7f706015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 --2- 172.21.15.87:0/139375161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f706006bd10 0x7f7060161910 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 -- 172.21.15.87:0/139375161 >> 172.21.15.87:0/139375161 conn(0x7f70600d19a0 msgr2=0x7f70600d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 -- 172.21.15.87:0/139375161 shutdown_connections 2024-04-25T10:17:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.905+0000 7f7067600640 1 -- 172.21.15.87:0/139375161 wait complete. 2024-04-25T10:17:49.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 Processor -- start 2024-04-25T10:17:49.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 -- start start 2024-04-25T10:17:49.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f706006bd10 0x7f70600c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70600cd9a0 0x7f7060160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7060161e50 0x7f70600af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600cdd80 con 0x7f7060161e50 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706006b510 con 0x7f70600cd9a0 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7060162660 con 0x7f706006bd10 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7060161e50 0x7f70600af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f705fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70600cd9a0 0x7f7060160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7060161e50 0x7f70600af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36304/0 (socket says 172.21.15.87:36304) 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f705fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70600cd9a0 0x7f7060160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:37434/0 (socket says 172.21.15.87:37434) 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 -- 172.21.15.87:0/3603014371 learned_addr learned my addr 172.21.15.87:0/3603014371 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7064c00640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f706006bd10 0x7f70600c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 -- 172.21.15.87:0/3603014371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f706006bd10 msgr2=0x7f70600c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f706006bd10 0x7f70600c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 -- 172.21.15.87:0/3603014371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70600cd9a0 msgr2=0x7f7060160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70600cd9a0 0x7f7060160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 -- 172.21.15.87:0/3603014371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600affd0 con 0x7f7060161e50 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7065600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7060161e50 0x7f70600af3f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f705800cf20 tx=0x7f705800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f705cc00640 1 -- 172.21.15.87:0/3603014371 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705801a070 con 0x7f7060161e50 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f705cc00640 1 -- 172.21.15.87:0/3603014371 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705800d840 con 0x7f7060161e50 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f705cc00640 1 -- 172.21.15.87:0/3603014371 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705800dc30 con 0x7f7060161e50 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600afcf0 con 0x7f7060161e50 2024-04-25T10:17:49.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.909+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70600c7670 con 0x7f7060161e50 2024-04-25T10:17:49.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.913+0000 7f705cc00640 1 -- 172.21.15.87:0/3603014371 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f705801d070 con 0x7f7060161e50 2024-04-25T10:17:49.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.913+0000 7f706ab92000 1 Processor -- start 2024-04-25T10:17:49.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.913+0000 7f706ab92000 1 -- start start 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.913+0000 7f705cc00640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7040066f60 0x7f7040069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.913+0000 7f705cc00640 1 -- 172.21.15.87:0/3603014371 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7058085840 con 0x7f7060161e50 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.913+0000 7f705fe00640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7040066f60 0x7f7040069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ebe1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80eb9c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80eb9d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ea7f1f0 con 0x55a80ebe2360 2024-04-25T10:17:49.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80e9e5e90 con 0x55a80ebe1a40 2024-04-25T10:17:49.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a80e96df50 con 0x55a80eb9cd90 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ebe1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f705fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80eb9c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ebe1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:37440/0 (socket says 172.21.15.87:37440) 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 -- 172.21.15.87:0/1096941082 learned_addr learned my addr 172.21.15.87:0/1096941082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7065600640 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80eb9d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f705fe00640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7040066f60 0x7f7040069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f704c0b2000 tx=0x7f704c0aa070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 -- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 msgr2=0x55a80eb9d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80eb9d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 -- 172.21.15.87:0/1096941082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 msgr2=0x55a80eb9c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80eb9c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 -- 172.21.15.87:0/1096941082 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a80e9bd8b0 con 0x55a80ebe1a40 2024-04-25T10:17:49.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f705fe00640 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80eb9c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7064c00640 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ebe1e20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7054004010 tx=0x7f70540044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56c5438006d203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7051600640 1 -- 172.21.15.87:0/1096941082 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054018020 con 0x55a80ebe1a40 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7051600640 1 -- 172.21.15.87:0/1096941082 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400fab0 con 0x55a80ebe1a40 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f7051600640 1 -- 172.21.15.87:0/1096941082 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400fe60 con 0x55a80ebe1a40 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 msgr2=0x55a80ebe1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ebe1e20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7054004010 tx=0x7f70540044d0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- 172.21.15.87:0/1096941082 shutdown_connections 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80eb9d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80eb9c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- 172.21.15.87:0/1096941082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ebe1e20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- 172.21.15.87:0/1096941082 >> 172.21.15.87:0/1096941082 conn(0x55a80eb98d40 msgr2=0x55a80eb9b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- 172.21.15.87:0/1096941082 shutdown_connections 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- 172.21.15.87:0/1096941082 wait complete. 2024-04-25T10:17:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 -- start start 2024-04-25T10:17:49.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 Processor -- start 2024-04-25T10:17:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80eb9cd90 0x55a80ebe91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe1a40 0x55a80ebe9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.917+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80ebe2360 0x55a80eba2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80e9a0540 con 0x55a80ebe1a40 2024-04-25T10:17:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80e9e1f30 con 0x55a80eb9cd90 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a80eb835b0 con 0x55a80ebe2360 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 10 client.0 _ll_register_callbacks cb 0x55a80ea9a180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe1a40 0x55a80ebe9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7064c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80eb9cd90 0x55a80ebe91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe1a40 0x55a80ebe9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36316/0 (socket says 172.21.15.87:36316) 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 -- 172.21.15.87:0/3574384235 learned_addr learned my addr 172.21.15.87:0/3574384235 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7065600640 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80ebe2360 0x55a80eba2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 -- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80ebe2360 msgr2=0x55a80eba2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80ebe2360 0x55a80eba2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 -- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80eb9cd90 msgr2=0x55a80ebe91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80eb9cd90 0x55a80ebe91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 -- 172.21.15.87:0/3574384235 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a80e96e1a0 con 0x55a80ebe1a40 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f705fe00640 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe1a40 0x55a80ebe9700 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f704c009930 tx=0x7f704c01d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5564fdfcdd2b4053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:17:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 1 -- 172.21.15.87:0/3574384235 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0ca060 con 0x55a80ebe1a40 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f705400fab0 con 0x55a80ebe1a40 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 1 -- 172.21.15.87:0/3574384235 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f704c0b1c60 con 0x55a80ebe1a40 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 1 -- 172.21.15.87:0/3574384235 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f704c0d0050 con 0x55a80ebe1a40 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 10 client.7191 fetch_fsmap learned FSMap version 54 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55a80ea68840 con 0x55a80ebe1a40 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 1 -- 172.21.15.87:0/3574384235 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400fab0 con 0x55a80ebe1a40 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 1 -- 172.21.15.87:0/3574384235 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f704c0d3020 con 0x55a80ebe1a40 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 10 client.7191 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 10 client.7191 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 20 client.7191 populate_metadata read hostname 'smithi087' 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80eba3b70 0x55a80eba3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 4 client.7191 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a80e981be0 con 0x55a80eba3b70 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7064c00640 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80eba3b70 0x55a80eba3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7064c00640 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80eba3b70 0x55a80eba3f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=377104eadc306919 server_cookie=66d73d965ebd4117 in_seq=0 out_seq=0 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 10 client.7191 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 1 -- 172.21.15.87:0/3574384235 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a80e981be0 con 0x55a80eba3b70 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f7046000640 10 client.7191 handle_command_reply: tid=0 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.921+0000 7f706ab92000 1 client.7191 shutdown 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe1a40 msgr2=0x55a80ebe9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe1a40 0x55a80ebe9700 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f704c009930 tx=0x7f704c01d510 comp rx=0 tx=0).stop 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 shutdown_connections 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80eba3b70 0x55a80eba3f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80ebe2360 0x55a80eba2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe1a40 0x55a80ebe9700 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f7046000640 0 client.7191 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 --2- 172.21.15.87:0/3574384235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80eb9cd90 0x55a80ebe91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 >> 172.21.15.87:0/3574384235 conn(0x55a80eb98d40 msgr2=0x55a80eb9b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 shutdown_connections 2024-04-25T10:17:49.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 1 -- 172.21.15.87:0/3574384235 wait complete. 2024-04-25T10:17:49.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.925+0000 7f706ab92000 20 client.7191 trim_cache size 0 max 16384 2024-04-25T10:17:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 Processor -- start 2024-04-25T10:17:49.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- start start 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec3e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec3e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ec3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7060200050 con 0x55a80ebe2360 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600c61a0 con 0x55a80ebe1a40 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600c55a0 con 0x55a80eb9cd90 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ec3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ec3eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36330/0 (socket says 172.21.15.87:36330) 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 -- 172.21.15.87:0/4251951359 learned_addr learned my addr 172.21.15.87:0/4251951359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec3e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7064c00640 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec3e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 -- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 msgr2=0x55a80ec3e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec3e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 -- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 msgr2=0x55a80ec3e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec3e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 -- 172.21.15.87:0/4251951359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a80ea3fa10 con 0x55a80ebe2360 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:49.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ec3eb00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f70580070b0 tx=0x7f7058074040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610e8139fa5372eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7050c00640 1 -- 172.21.15.87:0/4251951359 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058072070 con 0x55a80ebe2360 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7050c00640 1 -- 172.21.15.87:0/4251951359 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705806c030 con 0x55a80ebe2360 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7050c00640 1 -- 172.21.15.87:0/4251951359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058072e60 con 0x55a80ebe2360 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- 172.21.15.87:0/4251951359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 msgr2=0x55a80ec3eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ec3eb00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f70580070b0 tx=0x7f7058074040 comp rx=0 tx=0).stop 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- 172.21.15.87:0/4251951359 shutdown_connections 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ec3eb00 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- 172.21.15.87:0/4251951359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec3e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- 172.21.15.87:0/4251951359 >> 172.21.15.87:0/4251951359 conn(0x55a80eb98d40 msgr2=0x55a80ebef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- 172.21.15.87:0/4251951359 shutdown_connections 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- 172.21.15.87:0/4251951359 wait complete. 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- start start 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 Processor -- start 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ebe66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80e9e51a0 con 0x55a80ebe2360 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600c5ba0 con 0x55a80ebe1a40 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a80e9892d0 con 0x55a80eb9cd90 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f706ab92000 10 client.0 _ll_register_callbacks cb 0x55a80e9a8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:49.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:37468/0 (socket says 172.21.15.87:37468) 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 -- 172.21.15.87:0/1138084347 learned_addr learned my addr 172.21.15.87:0/1138084347 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7064c00640 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7065600640 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ebe66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 -- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 msgr2=0x55a80ec42090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec42090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 -- 172.21.15.87:0/1138084347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 msgr2=0x55a80ebe66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ebe66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 -- 172.21.15.87:0/1138084347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a80e97ab00 con 0x55a80ebe1a40 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f705fe00640 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec425d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f704c01d380 tx=0x7f704c0bf000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11f62a1a28dd9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.941+0000 7f7046000640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 1 -- 172.21.15.87:0/1138084347 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0d8070 con 0x55a80ebe1a40 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 1 -- 172.21.15.87:0/1138084347 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f704c0b1c60 con 0x55a80ebe1a40 2024-04-25T10:17:49.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f705806c030 con 0x55a80ebe1a40 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 1 -- 172.21.15.87:0/1138084347 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f704c0e2050 con 0x55a80ebe1a40 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 10 client.6676 fetch_fsmap learned FSMap version 54 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55a80e98cf80 con 0x55a80ebe1a40 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 1 -- 172.21.15.87:0/1138084347 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705806c030 con 0x55a80ebe1a40 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 1 -- 172.21.15.87:0/1138084347 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f704c0e5020 con 0x55a80ebe1a40 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 10 client.6676 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 10 client.6676 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 20 client.6676 populate_metadata read hostname 'smithi087' 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80ebe7410 0x55a80ebe7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 4 client.6676 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55a80e9f33d0 con 0x55a80ebe7410 2024-04-25T10:17:49.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7064c00640 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80ebe7410 0x55a80ebe7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7064c00640 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80ebe7410 0x55a80ebe7820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c28e2178f850812 server_cookie=cf8e2d0ac028c9ec in_seq=0 out_seq=0 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 10 client.6676 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 1 -- 172.21.15.87:0/1138084347 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17925 (crc 0 0 0) 0x55a80e9f33d0 con 0x55a80ebe7410 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 10 client.6676 handle_command_reply: tid=0 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 client.6676 shutdown 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 msgr2=0x55a80ec425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec425d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f704c01d380 tx=0x7f704c0bf000 comp rx=0 tx=0).stop 2024-04-25T10:17:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 shutdown_connections 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55a80ebe7410 0x55a80ebe7820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a80ebe2360 0x55a80ebe66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a80ebe1a40 0x55a80ec425d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7046000640 0 client.6676 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 --2- 172.21.15.87:0/1138084347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a80eb9cd90 0x55a80ec42090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 >> 172.21.15.87:0/1138084347 conn(0x55a80eb98d40 msgr2=0x55a80ebe0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 shutdown_connections 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 1 -- 172.21.15.87:0/1138084347 wait complete. 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f706ab92000 20 client.6676 trim_cache size 0 max 16384 2024-04-25T10:17:49.957 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 2055, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2056, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 713793, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 32446977, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 9, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.250926314, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.125719379, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.046972190, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.060780695, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2055, 2024-04-25T10:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.320130244, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000155781 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000195, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000097 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 29598, 2024-04-25T10:17:49.959 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 50400, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 10312644, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 54895609, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 7, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 2.713435709, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 1.659199600, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 1.277437517, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.062754761, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 50399, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.567199472, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000031095 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.960 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 44805, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.009256322, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000206 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 114672, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 18848 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 2014, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2027, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 502412, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 19704827, 2024-04-25T10:17:49.961 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.207578842, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.093572983, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.045918077, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.053846051, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2026, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.272115926, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000134311 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.962 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 108816 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 1, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-MDSRank": { 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4180, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 12.847887683, 2024-04-25T10:17:49.963 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003073657 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-PurgeQueue": { 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1893, 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.893396289, 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000471947 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": { 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "request": 25657, 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "reply": 25657, 2024-04-25T10:17:49.964 INFO:teuthology.orchestra.run.smithi087.stdout: "reply_latency": { 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 25657, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 379.823873679, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.014803908 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_reply": 0, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "forward": 0, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_complete": 0, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_keys": 0, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_commit": 12, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_split": 0, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_merge": 0, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes": 19844, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_top": 0, 2024-04-25T10:17:49.965 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_bottom": 19844, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pin_tail": 0, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pinned": 19844, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_expired": 0, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_with_caps": 146, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": 146, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "subtrees": 2, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse": 47667, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_hit": 47262, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_forward": 0, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_discover": 0, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_remote_ino": 0, 2024-04-25T10:17:49.966 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_lock": 123, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "load_cent": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "q": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "exported": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "exported_inodes": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "imported": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "imported_inodes": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_dir_fetch": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_peer_discover": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rfiles": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rbytes": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rsnaps": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_set_tag": 0, 2024-04-25T10:17:49.967 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_file_inodes": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_revoke": 681, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_grant": 4013, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps": 3776, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:17:49.968 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_cap_release": 63, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "process_request_cap_release": 13953 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_cache": { 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update": 0, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update_receipt": 0, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_try_discover": 0, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_send_discover": 0, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_handle_discover": 0, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays": 1690, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_delayed": 0, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_created": 4120, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_enqueued": 1856, 2024-04-25T10:17:49.969 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_reintegrated": 2201, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_migrated": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_processing": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_started": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_completed": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_exportdir": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_flush": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragstats": 0, 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_inodestats": 0 2024-04-25T10:17:49.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_log": { 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "evlrg": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "evadd": 24275, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "evex": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "evtrm": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "ev": 24275, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "evexg": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "evexd": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "segadd": 32, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "segex": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "segtrm": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "seg": 32, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "segmjr": 3, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "segexg": 0, 2024-04-25T10:17:49.971 INFO:teuthology.orchestra.run.smithi087.stdout: "segexd": 0, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "expos": 4194304, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "wrpos": 88119730, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "rdpos": 0, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "jlat": { 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4081, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 113.441700873, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.027797525 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "replayed": 0 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_mem": { 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 2011, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "ino+": 2014, 2024-04-25T10:17:49.972 INFO:teuthology.orchestra.run.smithi087.stdout: "ino-": 3, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dir": 13, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dir+": 13, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dir-": 0, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dn": 19841, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dn+": 19845, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dn-": 4, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "cap": 146, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "cap+": 8730, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "cap-": 8584, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "rss": 106444, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "heap": 215340 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_server": { 2024-04-25T10:17:49.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_client_request": 29250, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_server_request": 0, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_request": 25657, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_session": 3, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_peer_request": 0, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "req_create_latency": { 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2000, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.657542829, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000828771 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getattr_latency": { 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 28, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.005018376, 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000179227 2024-04-25T10:17:49.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getfilelock_latency": { 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "req_link_latency": { 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6000, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 7.556804886, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001259467 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookup_latency": { 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 7415, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6.289161878, 2024-04-25T10:17:49.975 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000848167 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookuphash_latency": { 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupino_latency": { 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupname_latency": { 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.976 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupparent_latency": { 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupsnap_latency": { 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000040942, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000040942 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lssnap_latency": { 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.977 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mkdir_latency": { 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.002957234, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002957234 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mknod_latency": { 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mksnap_latency": { 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:17:49.978 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.005736507, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.005736507 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "req_open_latency": { 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "req_readdir_latency": { 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.089484665, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.011185583 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rename_latency": { 2024-04-25T10:17:49.979 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2201, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 348.324184301, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.158257239 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "req_renamesnap_latency": { 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "req_snapdiff_latency": { 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmdir_latency": { 2024-04-25T10:17:49.980 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmsnap_latency": { 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.032703495, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.032703495 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmxattr_latency": { 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setattr_latency": { 2024-04-25T10:17:49.981 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000427728, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000427728 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setdirlayout_latency": { 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setfilelock_latency": { 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.982 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setlayout_latency": { 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setxattr_latency": { 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "req_symlink_latency": { 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "req_unlink_latency": { 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8000, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 15.859810838, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001982476 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getvxattr_latency": { 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.984 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_sessions": { 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "session_count": 1, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "session_add": 1, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "session_remove": 0, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_open": 1, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_stale": 0, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "total_load": 18224, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "average_load": 18224, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "avg_session_uptime": 48, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "mdthresh_evicted": 0 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-04-25T10:17:49.985 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-04-25T10:17:49.986 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-04-25T10:17:49.987 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 17425334, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 4417, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-04-25T10:17:49.988 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 5504, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 79, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 19984823, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 793068, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-04-25T10:17:49.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "objecter": { 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_active": 0, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_laggy": 0, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send": 6197, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send_bytes": 84297604, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_resend": 37, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_reply": 6197, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_latency": { 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6197, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 23.043774877, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003718537 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "op_inflight": 0, 2024-04-25T10:17:49.990 INFO:teuthology.orchestra.run.smithi087.stdout: "oplen_avg": { 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6197, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6289 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "op": 6197, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "op_r": 58, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "op_w": 6139, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "op_rmw": 0, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "op_pg": 0, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_stat": 0, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_create": 11, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_read": 58, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_write": 4139, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writefull": 36, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writesame": 0, 2024-04-25T10:17:49.991 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_append": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_zero": 1546, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_truncate": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_delete": 342, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_mapext": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_sparse_read": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_clonerange": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_getxattr": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_setxattr": 22, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_rmxattr": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_call": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_watch": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_notify": 0, 2024-04-25T10:17:49.992 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_other": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_active": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_send": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_resend": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_ping": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_active": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_send": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_resend": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_active": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_send": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_resend": 0, 2024-04-25T10:17:49.993 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_active": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_send": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_resend": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "command_active": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "command_send": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "command_resend": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "map_epoch": 74, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "map_full": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "map_inc": 3, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_sessions": 8, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_open": 8, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_close": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_laggy": 0, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_wr": 115, 2024-04-25T10:17:49.994 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_rd": 0, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_del": 20 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "oft": { 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_objs": 1, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_updates": 0, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_removes": 0 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "osd-slow-ops": { 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_ops_count": 0 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "purge_queue": { 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops": 0, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:17:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing": 0, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_high_water": 32, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed_ops": 1856, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed": 1856, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_item_in_journal": 0 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 33662, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 8932410, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 33662, 2024-04-25T10:17:49.996 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 33662, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 8932410, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_bytes": { 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:17:49.997 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 6197, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 84497428, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 6197, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 84497428, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:49.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_ops": { 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 1024, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 6197, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 6197, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 6197, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 6197, 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:17:49.999 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 1856, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 199824, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 1856, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:17:50.000 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 58, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 199824, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle-0x5626e2a4c960": { 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 39346, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 24275, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 83925426, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:17:50.001 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 24275, 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 4081, 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 83886080, 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:50.002 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7040066f60 msgr2=0x7f7040069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7040066f60 0x7f7040069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f704c0b2000 tx=0x7f704c0aa070 comp rx=0 tx=0).stop 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7060161e50 msgr2=0x7f70600af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7060161e50 0x7f70600af3f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f705800cf20 tx=0x7f705800a7d0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 shutdown_connections 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7040066f60 0x7f7040069420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7060161e50 0x7f70600af3f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70600cd9a0 0x7f7060160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 --2- 172.21.15.87:0/3603014371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f706006bd10 0x7f70600c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 >> 172.21.15.87:0/3603014371 conn(0x7f70600d19a0 msgr2=0x7f70600b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 shutdown_connections 2024-04-25T10:17:50.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:49.945+0000 7f7067600640 1 -- 172.21.15.87:0/3603014371 wait complete. 2024-04-25T10:17:50.005 INFO:tasks.cephfs.test_scrub_checks:num of strays: 1690 2024-04-25T10:17:50.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:50.005 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T10:17:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53bba00640 1 Processor -- start 2024-04-25T10:17:50.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53bba00640 1 -- start start 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53bba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f53b40d00f0 0x7f53b40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53bba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53b415bc00 0x7f53b40718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53bba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b4071f80 con 0x7f53b40cf040 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53bba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b4072260 con 0x7f53b415bc00 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53bba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40b3700 con 0x7f53b40d00f0 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53b9000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f53b40cf040 0x7f53b40cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34200/0 (socket says 172.21.15.87:34200) 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53b9000640 1 -- 172.21.15.87:0/4018642458 learned_addr learned my addr 172.21.15.87:0/4018642458 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:17:50.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.137+0000 7f53b9a00640 1 --2- 172.21.15.87:0/4018642458 >> v2:172.21.15.87:3300/0 conn(0x7f53b415bc00 0x7f53b40718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53abe00640 1 --2- 172.21.15.87:0/4018642458 >> v2:172.21.15.169:3300/0 conn(0x7f53b40d00f0 0x7f53b40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b9a00640 1 -- 172.21.15.87:0/4018642458 >> v2:172.21.15.169:3300/0 conn(0x7f53b40d00f0 msgr2=0x7f53b40713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b9a00640 1 --2- 172.21.15.87:0/4018642458 >> v2:172.21.15.169:3300/0 conn(0x7f53b40d00f0 0x7f53b40713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b9a00640 1 -- 172.21.15.87:0/4018642458 >> v1:172.21.15.87:6789/0 conn(0x7f53b40cf040 legacy=0x7f53b40cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b9a00640 1 -- 172.21.15.87:0/4018642458 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b415b120 con 0x7f53b415bc00 2024-04-25T10:17:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b3e00640 1 -- 172.21.15.87:0/4018642458 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419722720 0 0) 0x7f53b4071f80 con 0x7f53b40cf040 2024-04-25T10:17:50.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b9a00640 1 --2- 172.21.15.87:0/4018642458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b40718e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f53b406e840 tx=0x7f53ac01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ee0fe92af61f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b3e00640 1 -- 172.21.15.87:0/4018642458 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0b2070 con 0x7f53b415bc00 2024-04-25T10:17:50.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b3e00640 1 -- 172.21.15.87:0/4018642458 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac0a58e0 con 0x7f53b415bc00 2024-04-25T10:17:50.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.141+0000 7f53b3e00640 1 -- 172.21.15.87:0/4018642458 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0a5cb0 con 0x7f53b415bc00 2024-04-25T10:17:50.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- 172.21.15.87:0/4018642458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 msgr2=0x7f53b40718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 --2- 172.21.15.87:0/4018642458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b40718e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f53b406e840 tx=0x7f53ac01c930 comp rx=0 tx=0).stop 2024-04-25T10:17:50.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- 172.21.15.87:0/4018642458 shutdown_connections 2024-04-25T10:17:50.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 --2- 172.21.15.87:0/4018642458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b40718e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 --2- 172.21.15.87:0/4018642458 >> v2:172.21.15.169:3300/0 conn(0x7f53b40d00f0 0x7f53b40713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- 172.21.15.87:0/4018642458 >> 172.21.15.87:0/4018642458 conn(0x7f53b4066b70 msgr2=0x7f53b4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- 172.21.15.87:0/4018642458 shutdown_connections 2024-04-25T10:17:50.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- 172.21.15.87:0/4018642458 wait complete. 2024-04-25T10:17:50.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 Processor -- start 2024-04-25T10:17:50.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- start start 2024-04-25T10:17:50.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b40cf040 0x7f53b4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53b40d00f0 0x7f53b415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40aec90 con 0x7f53b415bc00 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40cf630 con 0x7f53b40cf040 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53bba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40aee10 con 0x7f53b40d00f0 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b415ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36352/0 (socket says 172.21.15.87:36352) 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 -- 172.21.15.87:0/4240191500 learned_addr learned my addr 172.21.15.87:0/4240191500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53abe00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53b40d00f0 0x7f53b415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9000640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b40cf040 0x7f53b4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 -- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53b40d00f0 msgr2=0x7f53b415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53b40d00f0 0x7f53b415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 -- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b40cf040 msgr2=0x7f53b4154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b40cf040 0x7f53b4154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 -- 172.21.15.87:0/4240191500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4153380 con 0x7f53b415bc00 2024-04-25T10:17:50.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b9a00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b415ccc0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f53ac019540 tx=0x7f53ac0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.145+0000 7f53b1600640 1 -- 172.21.15.87:0/4240191500 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0b2050 con 0x7f53b415bc00 2024-04-25T10:17:50.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53b1600640 1 -- 172.21.15.87:0/4240191500 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac0b3040 con 0x7f53b415bc00 2024-04-25T10:17:50.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b4153660 con 0x7f53b415bc00 2024-04-25T10:17:50.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53b1600640 1 -- 172.21.15.87:0/4240191500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0ad850 con 0x7f53b415bc00 2024-04-25T10:17:50.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b414baf0 con 0x7f53b415bc00 2024-04-25T10:17:50.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53b1600640 1 -- 172.21.15.87:0/4240191500 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53ac0c0020 con 0x7f53b415bc00 2024-04-25T10:17:50.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53b1600640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5390066f90 0x7f5390069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53b1600640 1 -- 172.21.15.87:0/4240191500 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f53ac11f770 con 0x7f53b415bc00 2024-04-25T10:17:50.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.149+0000 7f53b9000640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5390066f90 0x7f5390069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 Processor -- start 2024-04-25T10:17:50.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- start start 2024-04-25T10:17:50.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5390066f90 0x7f5390069450 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000f70 tx=0x7f53a4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0d4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4d1e0 0x55f1d0d02110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d02a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0b50dd0 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0ad9010 con 0x55f1d0d02650 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0b4ce70 con 0x55f1d0d4d1e0 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0d4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0d4cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36354/0 (socket says 172.21.15.87:36354) 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 -- 172.21.15.87:0/3547813577 learned_addr learned my addr 172.21.15.87:0/3547813577 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53abe00640 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4d1e0 0x55f1d0d02110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 -- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4d1e0 msgr2=0x55f1d0d02110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9a00640 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d02a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4d1e0 0x55f1d0d02110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 -- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 msgr2=0x55f1d0d02a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d02a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 -- 172.21.15.87:0/3547813577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d0b099c0 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53abe00640 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4d1e0 0x55f1d0d02110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:50.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9a00640 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d02a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53b9000640 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0d4cca0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f53a400fb70 tx=0x7f53a4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25da0241002c93ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53a9600640 1 -- 172.21.15.87:0/3547813577 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4025070 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53a9600640 1 -- 172.21.15.87:0/3547813577 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a401b050 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53a9600640 1 -- 172.21.15.87:0/3547813577 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4014e00 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- 172.21.15.87:0/3547813577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 msgr2=0x55f1d0d4cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0d4cca0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f53a400fb70 tx=0x7f53a4012000 comp rx=0 tx=0).stop 2024-04-25T10:17:50.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- 172.21.15.87:0/3547813577 shutdown_connections 2024-04-25T10:17:50.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d02a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4d1e0 0x55f1d0d02110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 --2- 172.21.15.87:0/3547813577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0d4cca0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- 172.21.15.87:0/3547813577 >> 172.21.15.87:0/3547813577 conn(0x55f1d0bea870 msgr2=0x55f1d0d009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- 172.21.15.87:0/3547813577 shutdown_connections 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- 172.21.15.87:0/3547813577 wait complete. 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.153+0000 7f53bee94000 1 -- start start 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 Processor -- start 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d02650 0x55f1d0d07c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d4d1e0 0x55f1d0d30de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0b0b420 con 0x55f1d0d02650 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0bd3470 con 0x55f1d0d4d1e0 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0d0bc20 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 10 client.0 _ll_register_callbacks cb 0x55f1d0cedf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d02650 0x55f1d0d07c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d02650 0x55f1d0d07c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36366/0 (socket says 172.21.15.87:36366) 2024-04-25T10:17:50.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 -- 172.21.15.87:0/1964039240 learned_addr learned my addr 172.21.15.87:0/1964039240 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53abe00640 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9a00640 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d4d1e0 0x55f1d0d30de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 -- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 msgr2=0x55f1d0d308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 -- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d4d1e0 msgr2=0x55f1d0d30de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d4d1e0 0x55f1d0d30de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 -- 172.21.15.87:0/1964039240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1d0ad9260 con 0x55f1d0d02650 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53b9000640 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d02650 0x55f1d0d07c90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x55f1d0b948f0 tx=0x7f53a4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8613aef4d43a053f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53a2000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53a2000640 1 -- 172.21.15.87:0/1964039240 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4025040 con 0x55f1d0d02650 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53a2000640 1 -- 172.21.15.87:0/1964039240 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f53a4020b00 con 0x55f1d0d02650 2024-04-25T10:17:50.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53a401b050 con 0x55f1d0d02650 2024-04-25T10:17:50.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53a2000640 1 -- 172.21.15.87:0/1964039240 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a4037050 con 0x55f1d0d02650 2024-04-25T10:17:50.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 10 client.7212 fetch_fsmap learned FSMap version 54 2024-04-25T10:17:50.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55f1d0b0bb30 con 0x55f1d0d02650 2024-04-25T10:17:50.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53a2000640 1 -- 172.21.15.87:0/1964039240 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a401b050 con 0x55f1d0d02650 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53a2000640 1 -- 172.21.15.87:0/1964039240 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53a403a020 con 0x55f1d0d02650 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 10 client.7212 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 10 client.7212 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 20 client.7212 populate_metadata read hostname 'smithi087' 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d53af0 0x55f1d0d53f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 4 client.7212 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1d0aecc80 con 0x55f1d0d53af0 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53abe00640 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d53af0 0x55f1d0d53f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53abe00640 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d53af0 0x55f1d0d53f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812d75a41a593d74 server_cookie=58b393790860ed8f in_seq=0 out_seq=0 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.157+0000 7f53a2000640 10 client.7212 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53a2000640 1 -- 172.21.15.87:0/1964039240 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1d0aecc80 con 0x55f1d0d53af0 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53a2000640 10 client.7212 handle_command_reply: tid=0 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 client.7212 shutdown 2024-04-25T10:17:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d02650 msgr2=0x55f1d0d07c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d02650 0x55f1d0d07c90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x55f1d0b948f0 tx=0x7f53a4005800 comp rx=0 tx=0).stop 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 shutdown_connections 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d53af0 0x55f1d0d53f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d4d1e0 0x55f1d0d30de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 --2- 172.21.15.87:0/1964039240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d02650 0x55f1d0d07c90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53a2000640 0 client.7212 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 >> 172.21.15.87:0/1964039240 conn(0x55f1d0bea870 msgr2=0x55f1d0d009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 shutdown_connections 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 1 -- 172.21.15.87:0/1964039240 wait complete. 2024-04-25T10:17:50.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.161+0000 7f53bee94000 20 client.7212 trim_cache size 0 max 16384 2024-04-25T10:17:50.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.173+0000 7f53bee94000 1 Processor -- start 2024-04-25T10:17:50.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- start start 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0bea870 0x55f1d0da84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d02650 0x55f1d0da89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0da8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4200a00 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41ffd10 con 0x55f1d0bea870 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4152380 con 0x55f1d0d02650 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0da8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0da8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36368/0 (socket says 172.21.15.87:36368) 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 -- 172.21.15.87:0/403372531 learned_addr learned my addr 172.21.15.87:0/403372531 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53abe00640 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d02650 0x55f1d0da89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 -- 172.21.15.87:0/403372531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d02650 msgr2=0x55f1d0da89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d02650 0x55f1d0da89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 -- 172.21.15.87:0/403372531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0bea870 msgr2=0x55f1d0da84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0bea870 0x55f1d0da84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 -- 172.21.15.87:0/403372531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d0bd66d0 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53b9a00640 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0da8f30 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0a55f0 tx=0x7f53ac0ec5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33c72c71e37b727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53a8c00640 1 -- 172.21.15.87:0/403372531 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac11b020 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53a8c00640 1 -- 172.21.15.87:0/403372531 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac0f8900 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53a8c00640 1 -- 172.21.15.87:0/403372531 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0ec910 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- 172.21.15.87:0/403372531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 msgr2=0x55f1d0da8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0da8f30 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0a55f0 tx=0x7f53ac0ec5c0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- 172.21.15.87:0/403372531 shutdown_connections 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0d4c8c0 0x55f1d0da8f30 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d02650 0x55f1d0da89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 --2- 172.21.15.87:0/403372531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0bea870 0x55f1d0da84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- 172.21.15.87:0/403372531 >> 172.21.15.87:0/403372531 conn(0x55f1d0d30c10 msgr2=0x55f1d0d06410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- 172.21.15.87:0/403372531 shutdown_connections 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- 172.21.15.87:0/403372531 wait complete. 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 -- start start 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.177+0000 7f53bee94000 1 Processor -- start 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0bea870 0x55f1d0dac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d50a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d50fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0b55400 con 0x55f1d0bea870 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4152980 con 0x55f1d0d02650 2024-04-25T10:17:50.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1d0b500e0 con 0x55f1d0d4c8c0 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 10 client.0 _ll_register_callbacks cb 0x55f1d0aed200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0bea870 0x55f1d0dac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0bea870 0x55f1d0dac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36382/0 (socket says 172.21.15.87:36382) 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 -- 172.21.15.87:0/1701282008 learned_addr learned my addr 172.21.15.87:0/1701282008 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53abe00640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d50a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9a00640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d50fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 -- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 msgr2=0x55f1d0d50fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d50fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 -- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 msgr2=0x55f1d0d50a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d50a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 -- 172.21.15.87:0/1701282008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1d0bd3900 con 0x55f1d0bea870 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9a00640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d50fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:50.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53abe00640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d50a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53b9000640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0bea870 0x55f1d0dac4b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f53a4035040 tx=0x7f53a4007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598b5e1ab3726741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53a2000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53a2000640 1 -- 172.21.15.87:0/1701282008 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4025070 con 0x55f1d0bea870 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53a2000640 1 -- 172.21.15.87:0/1701282008 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f53a4020d10 con 0x55f1d0bea870 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53ac0f8900 con 0x55f1d0bea870 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53a2000640 1 -- 172.21.15.87:0/1701282008 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a4041070 con 0x55f1d0bea870 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 10 client.7218 fetch_fsmap learned FSMap version 54 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55f1d0af7f20 con 0x55f1d0bea870 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53a2000640 1 -- 172.21.15.87:0/1701282008 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac0f8900 con 0x55f1d0bea870 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53a2000640 1 -- 172.21.15.87:0/1701282008 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53a403a070 con 0x55f1d0bea870 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 10 client.7218 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 10 client.7218 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 20 client.7218 populate_metadata read hostname 'smithi087' 2024-04-25T10:17:50.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d4d1e0 0x55f1d0d51500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 4 client.7218 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:17:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55f1d0b5e310 con 0x55f1d0d4d1e0 2024-04-25T10:17:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53abe00640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d4d1e0 0x55f1d0d51500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53abe00640 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d4d1e0 0x55f1d0d51500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ea6a8122452d81 server_cookie=3f6aabb76ab54de4 in_seq=0 out_seq=0 2024-04-25T10:17:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.181+0000 7f53a2000640 10 client.7218 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53a2000640 1 -- 172.21.15.87:0/1701282008 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f1d0b5e310 con 0x55f1d0d4d1e0 2024-04-25T10:17:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53a2000640 10 client.7218 handle_command_reply: tid=0 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 client.7218 shutdown 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0bea870 msgr2=0x55f1d0dac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0bea870 0x55f1d0dac4b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f53a4035040 tx=0x7f53a4007e90 comp rx=0 tx=0).stop 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 shutdown_connections 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x55f1d0d4d1e0 0x55f1d0d51500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f1d0d4c8c0 0x55f1d0d50fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1d0d02650 0x55f1d0d50a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53a2000640 0 client.7218 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 --2- 172.21.15.87:0/1701282008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f1d0bea870 0x55f1d0dac4b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 >> 172.21.15.87:0/1701282008 conn(0x55f1d0d30c10 msgr2=0x55f1d0d08340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 shutdown_connections 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 1 -- 172.21.15.87:0/1701282008 wait complete. 2024-04-25T10:17:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bee94000 20 client.7218 trim_cache size 0 max 16384 2024-04-25T10:17:50.204 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:17:50.204 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:17:50.204 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "b2c6b03a-daa0-440f-add0-c87f4edb8700", 2024-04-25T10:17:50.204 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:17:50.204 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5390066f90 msgr2=0x7f5390069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5390066f90 0x7f5390069450 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000f70 tx=0x7f53a4009040 comp rx=0 tx=0).stop 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 msgr2=0x7f53b415ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b415ccc0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f53ac019540 tx=0x7f53ac0196d0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 shutdown_connections 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5390066f90 0x7f5390069450 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53b415bc00 0x7f53b415ccc0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53b40d00f0 0x7f53b415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 --2- 172.21.15.87:0/4240191500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53b40cf040 0x7f53b4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 >> 172.21.15.87:0/4240191500 conn(0x7f53b4066b70 msgr2=0x7f53b4066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 shutdown_connections 2024-04-25T10:17:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.197+0000 7f53bba00640 1 -- 172.21.15.87:0/4240191500 wait complete. 2024-04-25T10:17:50.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:50.222 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa085a00640 1 Processor -- start 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa085a00640 1 -- start start 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa085a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa08015dc90 0x7fa0800cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa085a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0800cca70 0x7fa0800ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa085a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0800cd440 con 0x7fa0800cdf50 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa085a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08006ac90 con 0x7fa08015dc90 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa085a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0800cff30 con 0x7fa0800cca70 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa08015dc90 0x7fa0800cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa08015dc90 0x7fa0800cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36396/0 (socket says 172.21.15.87:36396) 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07fe00640 1 -- 172.21.15.87:0/2614492297 learned_addr learned my addr 172.21.15.87:0/2614492297 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07f400640 1 --2- 172.21.15.87:0/2614492297 >> v2:172.21.15.169:3300/0 conn(0x7fa0800cca70 0x7fa0800ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07fe00640 1 -- 172.21.15.87:0/2614492297 >> v2:172.21.15.169:3300/0 conn(0x7fa0800cca70 msgr2=0x7fa0800ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07fe00640 1 --2- 172.21.15.87:0/2614492297 >> v2:172.21.15.169:3300/0 conn(0x7fa0800cca70 0x7fa0800ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07fe00640 1 -- 172.21.15.87:0/2614492297 >> v1:172.21.15.169:6789/0 conn(0x7fa0800cdf50 legacy=0x7fa0800ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.381+0000 7fa07fe00640 1 -- 172.21.15.87:0/2614492297 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800d0210 con 0x7fa08015dc90 2024-04-25T10:17:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.385+0000 7fa07fe00640 1 --2- 172.21.15.87:0/2614492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa0800cc530 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa080071940 tx=0x7fa07001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a932966fcf6d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.385+0000 7fa07ea00640 1 -- 172.21.15.87:0/2614492297 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700ab660 con 0x7fa08015dc90 2024-04-25T10:17:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.385+0000 7fa07ea00640 1 -- 172.21.15.87:0/2614492297 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0700b0070 con 0x7fa08015dc90 2024-04-25T10:17:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.385+0000 7fa07ea00640 1 -- 172.21.15.87:0/2614492297 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700aba20 con 0x7fa08015dc90 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.385+0000 7fa085a00640 1 -- 172.21.15.87:0/2614492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 msgr2=0x7fa0800cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.385+0000 7fa085a00640 1 --2- 172.21.15.87:0/2614492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa0800cc530 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa080071940 tx=0x7fa07001c460 comp rx=0 tx=0).stop 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- 172.21.15.87:0/2614492297 shutdown_connections 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 --2- 172.21.15.87:0/2614492297 >> v2:172.21.15.169:3300/0 conn(0x7fa0800cca70 0x7fa0800ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 --2- 172.21.15.87:0/2614492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa0800cc530 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- 172.21.15.87:0/2614492297 >> 172.21.15.87:0/2614492297 conn(0x7fa0800d1740 msgr2=0x7fa0800d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- 172.21.15.87:0/2614492297 shutdown_connections 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- 172.21.15.87:0/2614492297 wait complete. 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 Processor -- start 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- start start 2024-04-25T10:17:50.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0800cca70 0x7fa0800cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0800cdf50 0x7fa08015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa08014bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800cef00 con 0x7fa08015dc90 2024-04-25T10:17:50.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800cd8e0 con 0x7fa0800cca70 2024-04-25T10:17:50.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800cf080 con 0x7fa0800cdf50 2024-04-25T10:17:50.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa08014bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa07fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0800cca70 0x7fa0800cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa08014bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36412/0 (socket says 172.21.15.87:36412) 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa07f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0800cdf50 0x7fa08015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.389+0000 7fa085000640 1 -- 172.21.15.87:0/1922135912 learned_addr learned my addr 172.21.15.87:0/1922135912 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085000640 1 -- 172.21.15.87:0/1922135912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0800cdf50 msgr2=0x7fa08015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085000640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0800cdf50 0x7fa08015bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085000640 1 -- 172.21.15.87:0/1922135912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0800cca70 msgr2=0x7fa0800cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085000640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0800cca70 0x7fa0800cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085000640 1 -- 172.21.15.87:0/1922135912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800cb6e0 con 0x7fa08015dc90 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085000640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa08014bf60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c770 tx=0x7fa07800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa077e00640 1 -- 172.21.15.87:0/1922135912 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078003b80 con 0x7fa08015dc90 2024-04-25T10:17:50.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa077e00640 1 -- 172.21.15.87:0/1922135912 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa078003d20 con 0x7fa08015dc90 2024-04-25T10:17:50.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa077e00640 1 -- 172.21.15.87:0/1922135912 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07800aa90 con 0x7fa08015dc90 2024-04-25T10:17:50.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0800b1190 con 0x7fa08015dc90 2024-04-25T10:17:50.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0800b1430 con 0x7fa08015dc90 2024-04-25T10:17:50.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.393+0000 7fa077e00640 1 -- 172.21.15.87:0/1922135912 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa07800ac30 con 0x7fa08015dc90 2024-04-25T10:17:50.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa077e00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa060066f60 0x7fa060069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa077e00640 1 -- 172.21.15.87:0/1922135912 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa078084e70 con 0x7fa08015dc90 2024-04-25T10:17:50.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 Processor -- start 2024-04-25T10:17:50.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa07fe00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa060066f60 0x7fa060069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- start start 2024-04-25T10:17:50.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa07fe00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa060066f60 0x7fa060069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0700af000 tx=0x7fa0700a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc884db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8852f0 0x561cdc83ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc83b0a0 0x561cdc83b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cdc687ff0 con 0x561cdc83b0a0 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cdc610150 con 0x561cdc8849d0 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cdc684090 con 0x561cdc8852f0 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc83b0a0 0x561cdc83b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc83b0a0 0x561cdc83b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36414/0 (socket says 172.21.15.87:36414) 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 -- 172.21.15.87:0/257950971 learned_addr learned my addr 172.21.15.87:0/257950971 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa07fe00640 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc884db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa07f400640 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8852f0 0x561cdc83ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 -- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8852f0 msgr2=0x561cdc83ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8852f0 0x561cdc83ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 -- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 msgr2=0x561cdc884db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc884db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 -- 172.21.15.87:0/257950971 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cdc81e0d0 con 0x561cdc83b0a0 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa085000640 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc83b0a0 0x561cdc83b480 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa07805d910 tx=0x7fa078055140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add6cc9f2ad1d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa074c00640 1 -- 172.21.15.87:0/257950971 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07805de40 con 0x561cdc83b0a0 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa074c00640 1 -- 172.21.15.87:0/257950971 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa078053ae0 con 0x561cdc83b0a0 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa074c00640 1 -- 172.21.15.87:0/257950971 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078051e50 con 0x561cdc83b0a0 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- 172.21.15.87:0/257950971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc83b0a0 msgr2=0x561cdc83b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc83b0a0 0x561cdc83b480 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa07805d910 tx=0x7fa078055140 comp rx=0 tx=0).stop 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- 172.21.15.87:0/257950971 shutdown_connections 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc83b0a0 0x561cdc83b480 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8852f0 0x561cdc83ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/257950971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc884db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- 172.21.15.87:0/257950971 >> 172.21.15.87:0/257950971 conn(0x561cdc837020 msgr2=0x561cdc839440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- 172.21.15.87:0/257950971 shutdown_connections 2024-04-25T10:17:50.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.397+0000 7fa08a4c8000 1 -- 172.21.15.87:0/257950971 wait complete. 2024-04-25T10:17:50.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 -- start start 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 Processor -- start 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc869d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc86a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cdc64c250 con 0x561cdc8852f0 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cdc642690 con 0x561cdc83b0a0 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cdc610a40 con 0x561cdc8849d0 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 10 client.0 _ll_register_callbacks cb 0x561cdc825e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc86a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc86a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36418/0 (socket says 172.21.15.87:36418) 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 -- 172.21.15.87:0/1616043247 learned_addr learned my addr 172.21.15.87:0/1616043247 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa07fe00640 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa07f400640 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc869d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 -- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 msgr2=0x561cdc869d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc869d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 -- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 msgr2=0x561cdc8697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 -- 172.21.15.87:0/1616043247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cdc7351c0 con 0x561cdc8852f0 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa085000640 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc86a270 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa07800cfd0 tx=0x7fa078001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aeaa1ab98a5d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa069600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa069600640 1 -- 172.21.15.87:0/1616043247 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07805de40 con 0x561cdc8852f0 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa069600640 1 -- 172.21.15.87:0/1616043247 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa078051e50 con 0x561cdc8852f0 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa078053ae0 con 0x561cdc8852f0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa069600640 1 -- 172.21.15.87:0/1616043247 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa078073b70 con 0x561cdc8852f0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 10 client.7230 fetch_fsmap learned FSMap version 54 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x561cdc731170 con 0x561cdc8852f0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa069600640 1 -- 172.21.15.87:0/1616043247 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa078053ae0 con 0x561cdc8852f0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa069600640 1 -- 172.21.15.87:0/1616043247 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa078054c30 con 0x561cdc8852f0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 10 client.7230 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 10 client.7230 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 20 client.7230 populate_metadata read hostname 'smithi087' 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc894c80 0x561cdc897070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 4 client.7230 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cdc623c60 con 0x561cdc894c80 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.401+0000 7fa07fe00640 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc894c80 0x561cdc897070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa07fe00640 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc894c80 0x561cdc897070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2029b52aa44f85 server_cookie=9dd77ba3e49116db in_seq=0 out_seq=0 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa069600640 10 client.7230 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa069600640 1 -- 172.21.15.87:0/1616043247 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cdc623c60 con 0x561cdc894c80 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa069600640 10 client.7230 handle_command_reply: tid=0 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 client.7230 shutdown 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 msgr2=0x561cdc86a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc86a270 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa07800cfd0 tx=0x7fa078001260 comp rx=0 tx=0).stop 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 shutdown_connections 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc894c80 0x561cdc897070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc86a270 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc869d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1616043247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 >> 172.21.15.87:0/1616043247 conn(0x561cdc837020 msgr2=0x561cdc839ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa069600640 0 client.7230 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 shutdown_connections 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1616043247 wait complete. 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.405+0000 7fa08a4c8000 20 client.7230 trim_cache size 0 max 16384 2024-04-25T10:17:50.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 Processor -- start 2024-04-25T10:17:50.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- start start 2024-04-25T10:17:50.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc8e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc8e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa080203280 con 0x561cdc8852f0 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800b2ce0 con 0x561cdc83b0a0 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800c93b0 con 0x561cdc8849d0 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc8e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc8e1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36434/0 (socket says 172.21.15.87:36434) 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 -- 172.21.15.87:0/1615054063 learned_addr learned my addr 172.21.15.87:0/1615054063 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa07fe00640 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa07f400640 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc8e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 -- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 msgr2=0x561cdc8e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc8e1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 -- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 msgr2=0x561cdc8e10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8e10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 -- 172.21.15.87:0/1615054063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cdc6cede0 con 0x561cdc8852f0 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa07f400640 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc8e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa085000640 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc8e1b70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa078001340 tx=0x7fa07807e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79327fe9600f3a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa06be00640 1 -- 172.21.15.87:0/1615054063 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07805de40 con 0x561cdc8852f0 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa06be00640 1 -- 172.21.15.87:0/1615054063 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa078054c30 con 0x561cdc8852f0 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa06be00640 1 -- 172.21.15.87:0/1615054063 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078051530 con 0x561cdc8852f0 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1615054063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 msgr2=0x561cdc8e1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc8e1b70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa078001340 tx=0x7fa07807e040 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1615054063 shutdown_connections 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc8e1b70 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc8849d0 0x561cdc8e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/1615054063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc83b0a0 0x561cdc8e10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1615054063 >> 172.21.15.87:0/1615054063 conn(0x561cdc837020 msgr2=0x561cdc8830a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1615054063 shutdown_connections 2024-04-25T10:17:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- 172.21.15.87:0/1615054063 wait complete. 2024-04-25T10:17:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 -- start start 2024-04-25T10:17:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 Processor -- start 2024-04-25T10:17:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.421+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc83b0a0 0x561cdc897380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc8978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc897e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cdc622e00 con 0x561cdc8852f0 2024-04-25T10:17:50.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800b26e0 con 0x561cdc8849d0 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cdc687300 con 0x561cdc83b0a0 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 10 client.0 _ll_register_callbacks cb 0x561cdc806040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc897e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc897e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36450/0 (socket says 172.21.15.87:36450) 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 -- 172.21.15.87:0/2939253420 learned_addr learned my addr 172.21.15.87:0/2939253420 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa07fe00640 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc83b0a0 0x561cdc897380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa07f400640 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc8978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 -- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc83b0a0 msgr2=0x561cdc897380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc83b0a0 0x561cdc897380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 -- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 msgr2=0x561cdc8978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc8978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 -- 172.21.15.87:0/2939253420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cdc6298e0 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085000640 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc897e00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa078083350 tx=0x7fa0780014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7471de3bef2933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 1 -- 172.21.15.87:0/2939253420 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07805de40 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 1 -- 172.21.15.87:0/2939253420 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa078051740 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa078054c30 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 1 -- 172.21.15.87:0/2939253420 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa078067050 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 10 client.7236 fetch_fsmap learned FSMap version 54 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x561cdc6e18f0 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 1 -- 172.21.15.87:0/2939253420 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa078054c30 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 1 -- 172.21.15.87:0/2939253420 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa07807a070 con 0x561cdc8852f0 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 10 client.7236 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 10 client.7236 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:17:50.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 20 client.7236 populate_metadata read hostname 'smithi087' 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc898790 0x561cdc8ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 4 client.7236 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561cdc695530 con 0x561cdc898790 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa07fe00640 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc898790 0x561cdc8ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 10 client.7236 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa07fe00640 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc898790 0x561cdc8ecf60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4c2cd494f71625 server_cookie=7ebcf6ee54fd36b0 in_seq=0 out_seq=0 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 1 -- 172.21.15.87:0/2939253420 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x561cdc695530 con 0x561cdc898790 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 10 client.7236 handle_command_reply: tid=0 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 client.7236 shutdown 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 msgr2=0x561cdc897e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc897e00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa078083350 tx=0x7fa0780014b0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 shutdown_connections 2024-04-25T10:17:50.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x561cdc898790 0x561cdc8ecf60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561cdc8852f0 0x561cdc897e00 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561cdc8849d0 0x561cdc8978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa069600640 0 client.7236 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 --2- 172.21.15.87:0/2939253420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561cdc83b0a0 0x561cdc897380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 >> 172.21.15.87:0/2939253420 conn(0x561cdc837020 msgr2=0x561cdc883930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 shutdown_connections 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 1 -- 172.21.15.87:0/2939253420 wait complete. 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa08a4c8000 20 client.7236 trim_cache size 0 max 16384 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:17:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "b2c6b03a-daa0-440f-add0-c87f4edb8700": { 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "~mds0", 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "b2c6b03a-daa0-440f-add0-c87f4edb8700", 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive" 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:17:50.440 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa060066f60 msgr2=0x7fa060069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa060066f60 0x7fa060069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0700af000 tx=0x7fa0700a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 msgr2=0x7fa08014bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa08014bf60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c770 tx=0x7fa07800cc30 comp rx=0 tx=0).stop 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 shutdown_connections 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa060066f60 0x7fa060069420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa08015dc90 0x7fa08014bf60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0800cdf50 0x7fa08015bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 --2- 172.21.15.87:0/1922135912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0800cca70 0x7fa0800cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 >> 172.21.15.87:0/1922135912 conn(0x7fa0800d1740 msgr2=0x7fa0800b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 shutdown_connections 2024-04-25T10:17:50.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:17:50.425+0000 7fa085a00640 1 -- 172.21.15.87:0/1922135912 wait complete. 2024-04-25T10:17:50.452 INFO:tasks.cephfs.filesystem:scrub status for tag:b2c6b03a-daa0-440f-add0-c87f4edb8700 - {'path': '~mds0', 'tag': 'b2c6b03a-daa0-440f-add0-c87f4edb8700', 'options': 'recursive'} 2024-04-25T10:17:52.406 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:52.410 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:20.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.454 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:18:20.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 Processor -- start 2024-04-25T10:18:20.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 -- start start 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b3480 0x7f7e7015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e700cfb50 0x7f7e70070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e70071bc0 0x7f7e70072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7015bb90 con 0x7f7e70071bc0 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e700ad840 con 0x7f7e700cfb50 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e74a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7015e710 con 0x7f7e700b3480 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e70071bc0 0x7f7e70072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e70071bc0 0x7f7e70072050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54430/0 (socket says 172.21.15.87:54430) 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 -- 172.21.15.87:0/3846957701 learned_addr learned my addr 172.21.15.87:0/3846957701 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e65400640 1 --2- 172.21.15.87:0/3846957701 >> v2:172.21.15.169:3300/0 conn(0x7f7e700cfb50 0x7f7e70070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6de00640 1 --2- 172.21.15.87:0/3846957701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b3480 0x7f7e7015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 -- 172.21.15.87:0/3846957701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b3480 msgr2=0x7f7e7015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 --2- 172.21.15.87:0/3846957701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b3480 0x7f7e7015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 -- 172.21.15.87:0/3846957701 >> v2:172.21.15.169:3300/0 conn(0x7f7e700cfb50 msgr2=0x7f7e70070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 --2- 172.21.15.87:0/3846957701 >> v2:172.21.15.169:3300/0 conn(0x7f7e700cfb50 0x7f7e70070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.601+0000 7f7e6e800640 1 -- 172.21.15.87:0/3846957701 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7015e9f0 con 0x7f7e70071bc0 2024-04-25T10:18:20.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e6e800640 1 --2- 172.21.15.87:0/3846957701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e70071bc0 0x7f7e70072050 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7e7006e4f0 tx=0x7f7e6801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f838153276009d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e6d400640 1 -- 172.21.15.87:0/3846957701 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680a5670 con 0x7f7e70071bc0 2024-04-25T10:18:20.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e6d400640 1 -- 172.21.15.87:0/3846957701 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e680a5810 con 0x7f7e70071bc0 2024-04-25T10:18:20.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e6d400640 1 -- 172.21.15.87:0/3846957701 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680a5be0 con 0x7f7e70071bc0 2024-04-25T10:18:20.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 -- 172.21.15.87:0/3846957701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e70071bc0 msgr2=0x7f7e70072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 --2- 172.21.15.87:0/3846957701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e70071bc0 0x7f7e70072050 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7e7006e4f0 tx=0x7f7e6801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 -- 172.21.15.87:0/3846957701 shutdown_connections 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 --2- 172.21.15.87:0/3846957701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e70071bc0 0x7f7e70072050 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 --2- 172.21.15.87:0/3846957701 >> v2:172.21.15.169:3300/0 conn(0x7f7e700cfb50 0x7f7e70070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 --2- 172.21.15.87:0/3846957701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b3480 0x7f7e7015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 -- 172.21.15.87:0/3846957701 >> 172.21.15.87:0/3846957701 conn(0x7f7e700669a0 msgr2=0x7f7e70066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 -- 172.21.15.87:0/3846957701 shutdown_connections 2024-04-25T10:18:20.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.605+0000 7f7e74a00640 1 -- 172.21.15.87:0/3846957701 wait complete. 2024-04-25T10:18:20.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 Processor -- start 2024-04-25T10:18:20.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 -- start start 2024-04-25T10:18:20.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e70071bc0 0x7f7e701507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700b3480 0x7f7e7014aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb50 0x7f7e7014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70070a10 con 0x7f7e700b3480 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70070890 con 0x7f7e70071bc0 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700ae530 con 0x7f7e700cfb50 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700b3480 0x7f7e7014aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700b3480 0x7f7e7014aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54442/0 (socket says 172.21.15.87:54442) 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 -- 172.21.15.87:0/2388749161 learned_addr learned my addr 172.21.15.87:0/2388749161 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e6de00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e70071bc0 0x7f7e701507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e6e800640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb50 0x7f7e7014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 -- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb50 msgr2=0x7f7e7014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb50 0x7f7e7014f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 -- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e70071bc0 msgr2=0x7f7e701507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e70071bc0 0x7f7e701507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 -- 172.21.15.87:0/2388749161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7014b980 con 0x7f7e700b3480 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e65400640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700b3480 0x7f7e7014aec0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800d5f0 tx=0x7f7e5800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e66a00640 1 -- 172.21.15.87:0/2388749161 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580125f0 con 0x7f7e700b3480 2024-04-25T10:18:20.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e66a00640 1 -- 172.21.15.87:0/2388749161 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58019070 con 0x7f7e700b3480 2024-04-25T10:18:20.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e66a00640 1 -- 172.21.15.87:0/2388749161 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580129b0 con 0x7f7e700b3480 2024-04-25T10:18:20.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7014bc60 con 0x7f7e700b3480 2024-04-25T10:18:20.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.609+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7014c6b0 con 0x7f7e700b3480 2024-04-25T10:18:20.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e66a00640 1 -- 172.21.15.87:0/2388749161 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e58012b50 con 0x7f7e700b3480 2024-04-25T10:18:20.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e66a00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e4c066f60 0x7f7e4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e66a00640 1 -- 172.21.15.87:0/2388749161 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7e58085360 con 0x7f7e700b3480 2024-04-25T10:18:20.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e6de00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e4c066f60 0x7f7e4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 Processor -- start 2024-04-25T10:18:20.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 -- start start 2024-04-25T10:18:20.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e6de00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e4c066f60 0x7f7e4c069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7e600045f0 tx=0x7f7e60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292a9d0 0x563c0292adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292b2f0 0x563c028e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 0x563c028e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0272dff0 con 0x563c0292b2f0 2024-04-25T10:18:20.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c026b6150 con 0x563c0292a9d0 2024-04-25T10:18:20.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c0272a090 con 0x563c028e10a0 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292b2f0 0x563c028e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292b2f0 0x563c028e0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54444/0 (socket says 172.21.15.87:54444) 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 -- 172.21.15.87:0/2779232073 learned_addr learned my addr 172.21.15.87:0/2779232073 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e6e800640 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 0x563c028e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e6de00640 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292a9d0 0x563c0292adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 -- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 msgr2=0x563c028e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 0x563c028e1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 -- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292a9d0 msgr2=0x563c0292adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292a9d0 0x563c0292adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.613+0000 7f7e65400640 1 -- 172.21.15.87:0/2779232073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c028c40d0 con 0x563c0292b2f0 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292b2f0 0x563c028e0b60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7e58083790 tx=0x7f7e58055bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504516c98ea30bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e5ea00640 1 -- 172.21.15.87:0/2779232073 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5805e310 con 0x563c0292b2f0 2024-04-25T10:18:20.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e5ea00640 1 -- 172.21.15.87:0/2779232073 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5807e020 con 0x563c0292b2f0 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e5ea00640 1 -- 172.21.15.87:0/2779232073 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58053d50 con 0x563c0292b2f0 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- 172.21.15.87:0/2779232073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292b2f0 msgr2=0x563c028e0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292b2f0 0x563c028e0b60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7e58083790 tx=0x7f7e58055bb0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- 172.21.15.87:0/2779232073 shutdown_connections 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 0x563c028e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292b2f0 0x563c028e0b60 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/2779232073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292a9d0 0x563c0292adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- 172.21.15.87:0/2779232073 >> 172.21.15.87:0/2779232073 conn(0x563c028dd020 msgr2=0x563c028df440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- 172.21.15.87:0/2779232073 shutdown_connections 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- 172.21.15.87:0/2779232073 wait complete. 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- start start 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 Processor -- start 2024-04-25T10:18:20.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c028e10a0 0x563c0290f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c0290fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292b2f0 0x563c02910270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c026f2250 con 0x563c0292a9d0 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c026e8690 con 0x563c028e10a0 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c026b6a40 con 0x563c0292b2f0 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e77ebf000 10 client.0 _ll_register_callbacks cb 0x563c028cbe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c0290fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e6de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c028e10a0 0x563c0290f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c0290fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54450/0 (socket says 172.21.15.87:54450) 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 -- 172.21.15.87:0/1518526796 learned_addr learned my addr 172.21.15.87:0/1518526796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e6e800640 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292b2f0 0x563c02910270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 -- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292b2f0 msgr2=0x563c02910270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292b2f0 0x563c02910270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 -- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c028e10a0 msgr2=0x563c0290f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c028e10a0 0x563c0290f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 -- 172.21.15.87:0/1518526796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c027db1c0 con 0x563c0292a9d0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e65400640 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c0290fd30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800dfb0 tx=0x7f7e58085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e15298a866355e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.617+0000 7f7e53400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 1 -- 172.21.15.87:0/1518526796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58073070 con 0x563c0292a9d0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 1 -- 172.21.15.87:0/1518526796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7e580850b0 con 0x563c0292a9d0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e5807e020 con 0x563c0292a9d0 2024-04-25T10:18:20.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 1 -- 172.21.15.87:0/1518526796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e58053fb0 con 0x563c0292a9d0 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 10 client.7248 fetch_fsmap learned FSMap version 54 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x563c027d7170 con 0x563c0292a9d0 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 1 -- 172.21.15.87:0/1518526796 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5807e020 con 0x563c0292a9d0 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 1 -- 172.21.15.87:0/1518526796 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e58013070 con 0x563c0292a9d0 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 10 client.7248 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 10 client.7248 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 20 client.7248 populate_metadata read hostname 'smithi087' 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c0293ac80 0x563c0293d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 4 client.7248 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c026c9c60 con 0x563c0293ac80 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e6de00640 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c0293ac80 0x563c0293d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e6de00640 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c0293ac80 0x563c0293d070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d76fb37c702bd9 server_cookie=ea21d4c4fdac123e in_seq=0 out_seq=0 2024-04-25T10:18:20.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 10 client.7248 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 1 -- 172.21.15.87:0/1518526796 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c026c9c60 con 0x563c0293ac80 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 10 client.7248 handle_command_reply: tid=0 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 client.7248 shutdown 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 msgr2=0x563c0290fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c0290fd30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800dfb0 tx=0x7f7e58085040 comp rx=0 tx=0).stop 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 shutdown_connections 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c0293ac80 0x563c0293d070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292b2f0 0x563c02910270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c0290fd30 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1518526796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c028e10a0 0x563c0290f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e53400640 0 client.7248 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 >> 172.21.15.87:0/1518526796 conn(0x563c028dd020 msgr2=0x563c028dfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 shutdown_connections 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1518526796 wait complete. 2024-04-25T10:18:20.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.621+0000 7f7e77ebf000 20 client.7248 trim_cache size 0 max 16384 2024-04-25T10:18:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 Processor -- start 2024-04-25T10:18:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 -- start start 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 0x563c02987300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c02987840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c02987d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70202df0 con 0x563c0292a9d0 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70201ff0 con 0x563c0292b2f0 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7015d4b0 con 0x563c028e10a0 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c02987840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c02987840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54460/0 (socket says 172.21.15.87:54460) 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 -- 172.21.15.87:0/3573701071 learned_addr learned my addr 172.21.15.87:0/3573701071 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 -- 172.21.15.87:0/3573701071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 msgr2=0x563c02987300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 --2- 172.21.15.87:0/3573701071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 0x563c02987300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 -- 172.21.15.87:0/3573701071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 msgr2=0x563c02987d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 --2- 172.21.15.87:0/3573701071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c02987d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 -- 172.21.15.87:0/3573701071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c02774de0 con 0x563c0292a9d0 2024-04-25T10:18:20.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e65400640 1 --2- 172.21.15.87:0/3573701071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c02987840 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7e58059f10 tx=0x7f7e58070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909662f6c022327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e5e000640 1 -- 172.21.15.87:0/3573701071 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58073070 con 0x563c0292a9d0 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e5e000640 1 -- 172.21.15.87:0/3573701071 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5807e020 con 0x563c0292a9d0 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.637+0000 7f7e5e000640 1 -- 172.21.15.87:0/3573701071 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580850b0 con 0x563c0292a9d0 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/3573701071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 msgr2=0x563c02987840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/3573701071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c02987840 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7e58059f10 tx=0x7f7e58070000 comp rx=0 tx=0).stop 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/3573701071 shutdown_connections 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/3573701071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c02987d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/3573701071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c0292a9d0 0x563c02987840 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/3573701071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c028e10a0 0x563c02987300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/3573701071 >> 172.21.15.87:0/3573701071 conn(0x563c028dd020 msgr2=0x563c02937fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/3573701071 shutdown_connections 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/3573701071 wait complete. 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- start start 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 Processor -- start 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c028e10a0 0x563c0293d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292a9d0 0x563c0293dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c0293e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c026c8e00 con 0x563c028e10a0 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7015dab0 con 0x563c0292b2f0 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c0272d300 con 0x563c0292a9d0 2024-04-25T10:18:20.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 10 client.0 _ll_register_callbacks cb 0x563c028ac040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c028e10a0 0x563c0293d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c028e10a0 0x563c0293d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54462/0 (socket says 172.21.15.87:54462) 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 -- 172.21.15.87:0/1441911818 learned_addr learned my addr 172.21.15.87:0/1441911818 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6e800640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c0293e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e65400640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292a9d0 0x563c0293dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 -- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292a9d0 msgr2=0x563c0293dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292a9d0 0x563c0293dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 -- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 msgr2=0x563c0293e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c0293e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 -- 172.21.15.87:0/1441911818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c026cf8e0 con 0x563c028e10a0 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6e800640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c0293e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e6de00640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c028e10a0 0x563c0293d770 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000ba20 tx=0x7f7e600046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e000c473251e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e53400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e53400640 1 -- 172.21.15.87:0/1441911818 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6001c040 con 0x563c028e10a0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e53400640 1 -- 172.21.15.87:0/1441911818 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7e60016d10 con 0x563c028e10a0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e5807e020 con 0x563c028e10a0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e53400640 1 -- 172.21.15.87:0/1441911818 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e60034050 con 0x563c028e10a0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 10 client.7254 fetch_fsmap learned FSMap version 54 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x563c027878f0 con 0x563c028e10a0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e53400640 1 -- 172.21.15.87:0/1441911818 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5807e020 con 0x563c028e10a0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e53400640 1 -- 172.21.15.87:0/1441911818 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e60037020 con 0x563c028e10a0 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 10 client.7254 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 10 client.7254 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 20 client.7254 populate_metadata read hostname 'smithi087' 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c02990f50 0x563c02993340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 4 client.7254 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563c0273b530 con 0x563c02990f50 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.641+0000 7f7e65400640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c02990f50 0x563c02993340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e65400640 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c02990f50 0x563c02993340 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc3e2f56dc490ce server_cookie=428c1e6e701460a7 in_seq=0 out_seq=0 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e53400640 10 client.7254 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e53400640 1 -- 172.21.15.87:0/1441911818 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563c0273b530 con 0x563c02990f50 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e53400640 10 client.7254 handle_command_reply: tid=0 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 client.7254 shutdown 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c028e10a0 msgr2=0x563c0293d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c028e10a0 0x563c0293d770 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000ba20 tx=0x7f7e600046d0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 shutdown_connections 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x563c02990f50 0x563c02993340 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c0292b2f0 0x563c0293e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563c0292a9d0 0x563c0293dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 --2- 172.21.15.87:0/1441911818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563c028e10a0 0x563c0293d770 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e53400640 0 client.7254 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 >> 172.21.15.87:0/1441911818 conn(0x563c028dd020 msgr2=0x563c029290d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 shutdown_connections 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 1 -- 172.21.15.87:0/1441911818 wait complete. 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e77ebf000 20 client.7254 trim_cache size 0 max 16384 2024-04-25T10:18:20.651 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:18:20.652 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:18:20.652 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:18:20.652 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:18:20.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e4c066f60 msgr2=0x7f7e4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e4c066f60 0x7f7e4c069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7e600045f0 tx=0x7f7e60008040 comp rx=0 tx=0).stop 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700b3480 msgr2=0x7f7e7014aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700b3480 0x7f7e7014aec0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800d5f0 tx=0x7f7e5800dab0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 shutdown_connections 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e4c066f60 0x7f7e4c069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb50 0x7f7e7014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700b3480 0x7f7e7014aec0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 --2- 172.21.15.87:0/2388749161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e70071bc0 0x7f7e701507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 >> 172.21.15.87:0/2388749161 conn(0x7f7e700669a0 msgr2=0x7f7e70158010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 shutdown_connections 2024-04-25T10:18:20.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.645+0000 7f7e74a00640 1 -- 172.21.15.87:0/2388749161 wait complete. 2024-04-25T10:18:20.666 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:18:20.666 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:18:20.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:18:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.801+0000 7fa225400640 1 Processor -- start 2024-04-25T10:18:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.801+0000 7fa225400640 1 -- start start 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa225400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa220070350 0x7fa22015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa225400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200b31c0 0x7fa2200b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa225400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22015b080 con 0x7fa2200aef80 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa225400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22015adb0 con 0x7fa220070350 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa225400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22015eb60 con 0x7fa2200b31c0 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa220070350 0x7fa22015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa220070350 0x7fa22015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54476/0 (socket says 172.21.15.87:54476) 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21de00640 1 -- 172.21.15.87:0/2659530813 learned_addr learned my addr 172.21.15.87:0/2659530813 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21f200640 1 --2- 172.21.15.87:0/2659530813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200b31c0 0x7fa2200b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21de00640 1 -- 172.21.15.87:0/2659530813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200b31c0 msgr2=0x7fa2200b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21de00640 1 --2- 172.21.15.87:0/2659530813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200b31c0 0x7fa2200b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21de00640 1 -- 172.21.15.87:0/2659530813 >> v1:172.21.15.169:6789/0 conn(0x7fa2200aef80 legacy=0x7fa22015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.805+0000 7fa21de00640 1 -- 172.21.15.87:0/2659530813 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22015ee40 con 0x7fa220070350 2024-04-25T10:18:20.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa21de00640 1 --2- 172.21.15.87:0/2659530813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa220070350 0x7fa22015e480 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009870 tx=0x7fa20c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1537c3172af38110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa21d400640 1 -- 172.21.15.87:0/2659530813 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0a7070 con 0x7fa220070350 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa21d400640 1 -- 172.21.15.87:0/2659530813 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c0b0070 con 0x7fa220070350 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa21d400640 1 -- 172.21.15.87:0/2659530813 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0ab890 con 0x7fa220070350 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- 172.21.15.87:0/2659530813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa220070350 msgr2=0x7fa22015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 --2- 172.21.15.87:0/2659530813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa220070350 0x7fa22015e480 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009870 tx=0x7fa20c01c560 comp rx=0 tx=0).stop 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- 172.21.15.87:0/2659530813 shutdown_connections 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 --2- 172.21.15.87:0/2659530813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200b31c0 0x7fa2200b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 --2- 172.21.15.87:0/2659530813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa220070350 0x7fa22015e480 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- 172.21.15.87:0/2659530813 >> 172.21.15.87:0/2659530813 conn(0x7fa2200669a0 msgr2=0x7fa220066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- 172.21.15.87:0/2659530813 shutdown_connections 2024-04-25T10:18:20.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- 172.21.15.87:0/2659530813 wait complete. 2024-04-25T10:18:20.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 Processor -- start 2024-04-25T10:18:20.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- start start 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa220070350 0x7fa2201440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200aef80 0x7fa2200cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2200b31c0 0x7fa2200cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22006c250 con 0x7fa2200b31c0 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22006c0d0 con 0x7fa220070350 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.809+0000 7fa225400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22006c3d0 con 0x7fa2200aef80 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa220070350 0x7fa2201440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2200b31c0 0x7fa2200cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa220070350 0x7fa2201440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:45308/0 (socket says 172.21.15.87:45308) 2024-04-25T10:18:20.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21e800640 1 -- 172.21.15.87:0/1096083890 learned_addr learned my addr 172.21.15.87:0/1096083890 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21de00640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200aef80 0x7fa2200cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21f200640 1 -- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200aef80 msgr2=0x7fa2200cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21f200640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200aef80 0x7fa2200cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21f200640 1 -- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa220070350 msgr2=0x7fa2201440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21f200640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa220070350 0x7fa2201440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21f200640 1 -- 172.21.15.87:0/1096083890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2200af5d0 con 0x7fa2200b31c0 2024-04-25T10:18:20.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21e800640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa220070350 0x7fa2201440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:20.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa21f200640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2200b31c0 0x7fa2200cdf40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa21800e7f0 tx=0x7fa21800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa216a00640 1 -- 172.21.15.87:0/1096083890 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218017070 con 0x7fa2200b31c0 2024-04-25T10:18:20.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa216a00640 1 -- 172.21.15.87:0/1096083890 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21800c730 con 0x7fa2200b31c0 2024-04-25T10:18:20.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa216a00640 1 -- 172.21.15.87:0/1096083890 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21800cb00 con 0x7fa2200b31c0 2024-04-25T10:18:20.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2200af8b0 con 0x7fa2200b31c0 2024-04-25T10:18:20.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.813+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2200afe70 con 0x7fa2200b31c0 2024-04-25T10:18:20.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa216a00640 1 -- 172.21.15.87:0/1096083890 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa21801a070 con 0x7fa2200b31c0 2024-04-25T10:18:20.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa216a00640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa200066f90 0x7fa200069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 Processor -- start 2024-04-25T10:18:20.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa216a00640 1 -- 172.21.15.87:0/1096083890 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa218084e80 con 0x7fa2200b31c0 2024-04-25T10:18:20.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 -- start start 2024-04-25T10:18:20.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21e800640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa200066f90 0x7fa200069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21e800640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa200066f90 0x7fa200069450 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa21000f290 tx=0x7fa210009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694c1ba40 0x559694c1be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bd6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694bd6d90 0x559694bd7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559694ab91f0 con 0x559694c1c360 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x559694a1fe90 con 0x559694bd6d90 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596949a7f50 con 0x559694c1ba40 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bd6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bd6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54492/0 (socket says 172.21.15.87:54492) 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 -- 172.21.15.87:0/3801123812 learned_addr learned my addr 172.21.15.87:0/3801123812 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21e800640 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694c1ba40 0x559694c1be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21f200640 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694bd6d90 0x559694bd7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 -- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694c1ba40 msgr2=0x559694c1be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694c1ba40 0x559694c1be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 -- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694bd6d90 msgr2=0x559694bd7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694bd6d90 0x559694bd7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 -- 172.21.15.87:0/3801123812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596949f78b0 con 0x559694c1c360 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa21de00640 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bd6850 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa20c01c5a0 tx=0x7fa20c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6bc379e830ab44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa20b400640 1 -- 172.21.15.87:0/3801123812 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0aba70 con 0x559694c1c360 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa20b400640 1 -- 172.21.15.87:0/3801123812 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c0b0040 con 0x559694c1c360 2024-04-25T10:18:20.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.817+0000 7fa20b400640 1 -- 172.21.15.87:0/3801123812 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0abe50 con 0x559694c1c360 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- 172.21.15.87:0/3801123812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 msgr2=0x559694bd6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bd6850 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa20c01c5a0 tx=0x7fa20c004740 comp rx=0 tx=0).stop 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- 172.21.15.87:0/3801123812 shutdown_connections 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694bd6d90 0x559694bd7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bd6850 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3801123812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694c1ba40 0x559694c1be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- 172.21.15.87:0/3801123812 >> 172.21.15.87:0/3801123812 conn(0x559694bd2d40 msgr2=0x559694bd5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- 172.21.15.87:0/3801123812 shutdown_connections 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- 172.21.15.87:0/3801123812 wait complete. 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- start start 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 Processor -- start 2024-04-25T10:18:20.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c231c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1ba40 0x559694c23700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bdcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596949da540 con 0x559694c1c360 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x559694a1bf30 con 0x559694c1ba40 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596949cb260 con 0x559694bd6d90 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 10 client.0 _ll_register_callbacks cb 0x559694ad4180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bdcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bdcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54506/0 (socket says 172.21.15.87:54506) 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 -- 172.21.15.87:0/720251147 learned_addr learned my addr 172.21.15.87:0/720251147 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21e800640 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21de00640 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1ba40 0x559694c23700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 -- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 msgr2=0x559694c231c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c231c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 -- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1ba40 msgr2=0x559694c23700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1ba40 0x559694c23700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 -- 172.21.15.87:0/720251147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596949a81a0 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa21f200640 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bdcdc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa218055440 tx=0x7fa2180598f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42abed78403857ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa1ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa1ffe00640 1 -- 172.21.15.87:0/720251147 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218081060 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa1ffe00640 1 -- 172.21.15.87:0/720251147 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa218054d90 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa20c0b0040 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa1ffe00640 1 -- 172.21.15.87:0/720251147 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa218072050 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 10 client.7266 fetch_fsmap learned FSMap version 54 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x559694aa2840 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa1ffe00640 1 -- 172.21.15.87:0/720251147 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c0b0040 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa1ffe00640 1 -- 172.21.15.87:0/720251147 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa218024060 con 0x559694c1c360 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 10 client.7266 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 10 client.7266 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:18:20.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 20 client.7266 populate_metadata read hostname 'smithi087' 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.821+0000 7fa2287b8000 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694bddb00 0x559694bddee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 4 client.7266 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596949bbbe0 con 0x559694bddb00 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa21e800640 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694bddb00 0x559694bddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa21e800640 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694bddb00 0x559694bddee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace40571a8b565d7 server_cookie=8191a1f079f93178 in_seq=0 out_seq=0 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa1ffe00640 10 client.7266 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa1ffe00640 1 -- 172.21.15.87:0/720251147 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596949bbbe0 con 0x559694bddb00 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa1ffe00640 10 client.7266 handle_command_reply: tid=0 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 client.7266 shutdown 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 msgr2=0x559694bdcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bdcdc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa218055440 tx=0x7fa2180598f0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 shutdown_connections 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694bddb00 0x559694bddee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1c360 0x559694bdcdc0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1ba40 0x559694c23700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 --2- 172.21.15.87:0/720251147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa1ffe00640 0 client.7266 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 >> 172.21.15.87:0/720251147 conn(0x559694bd2d40 msgr2=0x559694bd59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 shutdown_connections 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 1 -- 172.21.15.87:0/720251147 wait complete. 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.825+0000 7fa2287b8000 20 client.7266 trim_cache size 0 max 16384 2024-04-25T10:18:20.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.837+0000 7fa2287b8000 1 Processor -- start 2024-04-25T10:18:20.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.837+0000 7fa2287b8000 1 -- start start 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694bd5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694bd5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694bd6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa220203220 con 0x559694c1ba40 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa220202470 con 0x559694c1c360 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201527b0 con 0x559694bd6d90 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694bd5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694bd5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54514/0 (socket says 172.21.15.87:54514) 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 -- 172.21.15.87:0/3701104904 learned_addr learned my addr 172.21.15.87:0/3701104904 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21e800640 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694bd5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21f200640 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694bd6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 -- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 msgr2=0x559694bd5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694bd5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 -- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 msgr2=0x559694bd6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694bd6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 -- 172.21.15.87:0/3701104904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559694a79a10 con 0x559694c1ba40 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa21de00640 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694bd5670 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa20c01a190 tx=0x7fa20c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8832be72d621fbd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa20aa00640 1 -- 172.21.15.87:0/3701104904 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004240 con 0x559694c1ba40 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa20aa00640 1 -- 172.21.15.87:0/3701104904 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c0c3070 con 0x559694c1ba40 2024-04-25T10:18:20.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa20aa00640 1 -- 172.21.15.87:0/3701104904 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0ab870 con 0x559694c1ba40 2024-04-25T10:18:20.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- 172.21.15.87:0/3701104904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 msgr2=0x559694bd5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694bd5670 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa20c01a190 tx=0x7fa20c009240 comp rx=0 tx=0).stop 2024-04-25T10:18:20.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- 172.21.15.87:0/3701104904 shutdown_connections 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694bd6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694bd5670 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3701104904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694bd5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- 172.21.15.87:0/3701104904 >> 172.21.15.87:0/3701104904 conn(0x559694bd2d40 msgr2=0x559694c298a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- 172.21.15.87:0/3701104904 shutdown_connections 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- 172.21.15.87:0/3701104904 wait complete. 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 -- start start 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 Processor -- start 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.841+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c20640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694c20b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694c210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559694a1f1a0 con 0x559694c1ba40 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa220158a40 con 0x559694c1c360 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596949e1e50 con 0x559694bd6d90 2024-04-25T10:18:20.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 10 client.0 _ll_register_callbacks cb 0x5596949e2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694c20b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694c20b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54524/0 (socket says 172.21.15.87:54524) 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 -- 172.21.15.87:0/3096921038 learned_addr learned my addr 172.21.15.87:0/3096921038 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21e800640 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c20640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21f200640 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694c210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 -- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 msgr2=0x559694c20640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c20640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 -- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 msgr2=0x559694c210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694c210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 -- 172.21.15.87:0/3096921038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596949b4b00 con 0x559694c1ba40 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21de00640 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694c20b80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa20c01a190 tx=0x7fa20c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6bad053caa30d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 1 -- 172.21.15.87:0/3096921038 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004300 con 0x559694c1ba40 2024-04-25T10:18:20.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 1 -- 172.21.15.87:0/3096921038 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa20c0b5b00 con 0x559694c1ba40 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa20c0c3070 con 0x559694c1ba40 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 1 -- 172.21.15.87:0/3096921038 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa20c0044a0 con 0x559694c1ba40 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 10 client.7272 fetch_fsmap learned FSMap version 54 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5596949c6f80 con 0x559694c1ba40 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 1 -- 172.21.15.87:0/3096921038 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c0c3070 con 0x559694c1ba40 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 1 -- 172.21.15.87:0/3096921038 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2200cfe70 con 0x559694c1ba40 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 10 client.7272 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 10 client.7272 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 20 client.7272 populate_metadata read hostname 'smithi087' 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694c81cd0 0x559694c840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 4 client.7272 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:18:20.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 --> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x559694a2d3d0 con 0x559694c81cd0 2024-04-25T10:18:20.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21e800640 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694c81cd0 0x559694c840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 10 client.7272 ms_handle_connect on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa21e800640 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694c81cd0 0x559694c840c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df66762a02191bc8 server_cookie=d4955e71e2850d80 in_seq=0 out_seq=0 2024-04-25T10:18:20.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 1 -- 172.21.15.87:0/3096921038 <== mds.0 v2:172.21.15.169:6832/1643726385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17931 (crc 0 0 0) 0x559694a2d3d0 con 0x559694c81cd0 2024-04-25T10:18:20.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 10 client.7272 handle_command_reply: tid=0 2024-04-25T10:18:20.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 client.7272 shutdown 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 msgr2=0x559694c20b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694c20b80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa20c01a190 tx=0x7fa20c01ada0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 shutdown_connections 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:6832/1643726385,v1:172.21.15.169:6834/1643726385] conn(0x559694c81cd0 0x559694c840c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559694c1c360 0x559694c210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559694c1ba40 0x559694c20b80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 --2- 172.21.15.87:0/3096921038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559694bd6d90 0x559694c20640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa1ffe00640 0 client.7272 ms_handle_reset on v2:172.21.15.169:6832/1643726385 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 >> 172.21.15.87:0/3096921038 conn(0x559694bd2d40 msgr2=0x559694c1a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 shutdown_connections 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.845+0000 7fa2287b8000 1 -- 172.21.15.87:0/3096921038 wait complete. 2024-04-25T10:18:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa2287b8000 20 client.7272 trim_cache size 0 max 16384 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 3261, 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 3262, 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1190291, 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 34176316, 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 17, 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 6, 2024-04-25T10:18:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.486826406, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.178241572, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.073181587, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.209606448, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3261, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.566698614, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000173780 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 10, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000706, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000070 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.855 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 30402, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 51172, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 10525480, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 55151625, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 7, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 2.788777100, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 1.689925301, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 1.294721171, 2024-04-25T10:18:20.856 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.082388016, 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 51171, 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.669271419, 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000032621 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 44807, 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.009257809, 2024-04-25T10:18:20.857 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000206 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 118816, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 25712 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 3475, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 3494, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 874086, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 20196291, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.325078728, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.139527785, 2024-04-25T10:18:20.858 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.073959619, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.086204104, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3493, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.523628417, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000149907 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 119616 2024-04-25T10:18:20.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 1, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-MDSRank": { 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 5513, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 13.120858970, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002379985 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-PurgeQueue": { 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-04-25T10:18:20.860 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3159, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.206450728, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000381909 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": { 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "request": 25657, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "reply": 25657, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "reply_latency": { 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 25657, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 379.823873679, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.014803908 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.861 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_reply": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "forward": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_complete": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_keys": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_commit": 12, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_split": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_merge": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes": 17844, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_top": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_bottom": 17844, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pin_tail": 0, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pinned": 17844, 2024-04-25T10:18:20.862 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_expired": 0, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_with_caps": 2, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": 2, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "subtrees": 2, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse": 47668, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_hit": 47263, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_forward": 0, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_discover": 0, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_remote_ino": 0, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_lock": 123, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "load_cent": 0, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "q": 0, 2024-04-25T10:18:20.863 INFO:teuthology.orchestra.run.smithi087.stdout: "exported": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "exported_inodes": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "imported": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "imported_inodes": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_dir_fetch": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_backtrace_fetch": 1701, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_peer_discover": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rfiles": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rbytes": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rsnaps": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_fetch": 1701, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_set_tag": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:18:20.864 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_inodes": 11, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_file_inodes": 1690, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_revoke": 681, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_grant": 4013, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps": 3776, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_cap_release": 64, 2024-04-25T10:18:20.865 INFO:teuthology.orchestra.run.smithi087.stdout: "process_request_cap_release": 13953 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_cache": { 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update_receipt": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_try_discover": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_send_discover": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_handle_discover": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_delayed": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_created": 4120, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_enqueued": 3546, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_reintegrated": 2201, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_migrated": 0, 2024-04-25T10:18:20.866 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_processing": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_started": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_completed": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_exportdir": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_flush": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragstats": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_inodestats": 0 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_log": { 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "evlrg": 0, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "evadd": 25971, 2024-04-25T10:18:20.867 INFO:teuthology.orchestra.run.smithi087.stdout: "evex": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "evtrm": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "ev": 25971, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "evexg": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "evexd": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "segadd": 34, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "segex": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "segtrm": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "seg": 34, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "segmjr": 4, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "segexg": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "segexd": 0, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "expos": 4194304, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "wrpos": 89149314, 2024-04-25T10:18:20.868 INFO:teuthology.orchestra.run.smithi087.stdout: "rdpos": 0, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "jlat": { 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4084, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 113.472734743, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.027784704 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "replayed": 0 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_mem": { 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 14, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "ino+": 2015, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "ino-": 2001, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "dir": 13, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "dir+": 13, 2024-04-25T10:18:20.869 INFO:teuthology.orchestra.run.smithi087.stdout: "dir-": 0, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "dn": 17844, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "dn+": 19845, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "dn-": 2001, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "cap": 2, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "cap+": 8730, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "cap-": 8728, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "rss": 97624, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "heap": 215340 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_server": { 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_client_request": 29250, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_server_request": 0, 2024-04-25T10:18:20.870 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_request": 25657, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_session": 5, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_peer_request": 0, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "req_create_latency": { 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2000, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1.657542829, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000828771 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getattr_latency": { 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 28, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.005018376, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000179227 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getfilelock_latency": { 2024-04-25T10:18:20.871 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "req_link_latency": { 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 6000, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 7.556804886, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001259467 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookup_latency": { 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 7415, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 6.289161878, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000848167 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookuphash_latency": { 2024-04-25T10:18:20.872 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupino_latency": { 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupname_latency": { 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupparent_latency": { 2024-04-25T10:18:20.873 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupsnap_latency": { 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000040942, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000040942 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lssnap_latency": { 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.874 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mkdir_latency": { 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.002957234, 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002957234 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mknod_latency": { 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.875 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mksnap_latency": { 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.005736507, 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.005736507 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "req_open_latency": { 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "req_readdir_latency": { 2024-04-25T10:18:20.876 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.089484665, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.011185583 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rename_latency": { 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2201, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 348.324184301, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.158257239 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "req_renamesnap_latency": { 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "req_snapdiff_latency": { 2024-04-25T10:18:20.877 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmdir_latency": { 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmsnap_latency": { 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.032703495, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.032703495 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.878 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmxattr_latency": { 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setattr_latency": { 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1, 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000427728, 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000427728 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setdirlayout_latency": { 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.879 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setfilelock_latency": { 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setlayout_latency": { 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setxattr_latency": { 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.880 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "req_symlink_latency": { 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "req_unlink_latency": { 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 8000, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 15.859810838, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001982476 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:18:20.881 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getvxattr_latency": { 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_sessions": { 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "session_count": 1, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "session_add": 1, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "session_remove": 0, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_open": 1, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_stale": 0, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "total_load": 18224, 2024-04-25T10:18:20.882 INFO:teuthology.orchestra.run.smithi087.stdout: "average_load": 18224, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "avg_session_uptime": 83, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "mdthresh_evicted": 0 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:18:20.883 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:18:20.884 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:18:20.885 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 250610, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 86, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 5608, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 81, 2024-04-25T10:18:20.886 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 13435764, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 660915, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "objecter": { 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "op_active": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "op_laggy": 0, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send": 9612, 2024-04-25T10:18:20.887 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send_bytes": 85548018, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "op_resend": 37, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "op_reply": 9612, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "op_latency": { 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 9612, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 34.087667390, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003546365 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "op_inflight": 0, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "oplen_avg": { 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 9612, 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 9706 2024-04-25T10:18:20.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "op": 9612, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "op_r": 1767, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "op_w": 7845, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "op_rmw": 0, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "op_pg": 0, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_stat": 0, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_create": 11, 2024-04-25T10:18:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_read": 65, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_write": 4148, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writefull": 40, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writesame": 0, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_append": 0, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_zero": 1546, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_truncate": 0, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_delete": 2032, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_mapext": 0, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_sparse_read": 0, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_clonerange": 0, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_getxattr": 1702, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_setxattr": 22, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:18:20.890 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_rmxattr": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_call": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_watch": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_notify": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_other": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_active": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_send": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_resend": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_ping": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_active": 0, 2024-04-25T10:18:20.891 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_send": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_resend": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_active": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_send": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_resend": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_active": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_send": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_resend": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "command_active": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "command_send": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "command_resend": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "map_epoch": 76, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "map_full": 0, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "map_inc": 5, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_sessions": 8, 2024-04-25T10:18:20.892 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_open": 8, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_close": 0, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_laggy": 0, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_wr": 119, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_rd": 0, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_del": 21 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "oft": { 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_objs": 1, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_updates": 0, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_removes": 0 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "osd-slow-ops": { 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_ops_count": 0 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "purge_queue": { 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops": 0, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing": 0, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_high_water": 32, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed_ops": 3546, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed": 3546, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_item_in_journal": 0 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 37133, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 9680899, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 37133, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 37133, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 9680899, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_bytes": { 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 9612, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 85918532, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 9612, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 85918532, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.896 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_ops": { 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 1024, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 9612, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 9612, 2024-04-25T10:18:20.897 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 9612, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 9612, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 3546, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 370514, 2024-04-25T10:18:20.898 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 3546, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 64, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 370514, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle-0x5626e2a4c960": { 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-04-25T10:18:20.899 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 25971, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 84955010, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 25971, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 4084, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 84955010, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:20.900 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa200066f90 msgr2=0x7fa200069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa200066f90 0x7fa200069450 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa21000f290 tx=0x7fa210009040 comp rx=0 tx=0).stop 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2200b31c0 msgr2=0x7fa2200cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2200b31c0 0x7fa2200cdf40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa21800e7f0 tx=0x7fa21800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 shutdown_connections 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa200066f90 0x7fa200069450 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2200b31c0 0x7fa2200cdf40 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2200aef80 0x7fa2200cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 --2- 172.21.15.87:0/1096083890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa220070350 0x7fa2201440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 >> 172.21.15.87:0/1096083890 conn(0x7fa2200669a0 msgr2=0x7fa2201562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 shutdown_connections 2024-04-25T10:18:20.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:20.849+0000 7fa225400640 1 -- 172.21.15.87:0/1096083890 wait complete. 2024-04-25T10:18:20.903 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:18:20.904 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:21.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:21.092 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:18:21.092 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:18:21.092 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:21.101 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:18:21.101 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:21.154 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:21.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.333+0000 7fe127400640 1 Processor -- start 2024-04-25T10:18:21.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.333+0000 7fe127400640 1 -- start start 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe127400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe1200cf930 0x7fe1200cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe127400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe120070c20 0x7fe12015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe127400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe127400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12015bb90 con 0x7fe120070c20 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe127400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1200d0250 con 0x7fe1200cf930 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe127400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1200ad840 con 0x7fe1200ae020 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe120070c20 0x7fe12015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe120070c20 0x7fe12015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54530/0 (socket says 172.21.15.87:54530) 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 -- 172.21.15.87:0/130615410 learned_addr learned my addr 172.21.15.87:0/130615410 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe125400640 1 --2- 172.21.15.87:0/130615410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe124a00640 1 --2- 172.21.15.87:0/130615410 >> v2:172.21.15.169:3300/0 conn(0x7fe1200cf930 0x7fe1200cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 -- 172.21.15.87:0/130615410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 msgr2=0x7fe1200ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 --2- 172.21.15.87:0/130615410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 -- 172.21.15.87:0/130615410 >> v2:172.21.15.169:3300/0 conn(0x7fe1200cf930 msgr2=0x7fe1200cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 --2- 172.21.15.87:0/130615410 >> v2:172.21.15.169:3300/0 conn(0x7fe1200cf930 0x7fe1200cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 -- 172.21.15.87:0/130615410 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12015e9f0 con 0x7fe120070c20 2024-04-25T10:18:21.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe124a00640 1 --2- 172.21.15.87:0/130615410 >> v2:172.21.15.169:3300/0 conn(0x7fe1200cf930 0x7fe1200cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:21.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.337+0000 7fe117e00640 1 --2- 172.21.15.87:0/130615410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120070c20 0x7fe12015e170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe1200aec50 tx=0x7fe10801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd40d9668401ae28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe11fe00640 1 -- 172.21.15.87:0/130615410 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080aa610 con 0x7fe120070c20 2024-04-25T10:18:21.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe11fe00640 1 -- 172.21.15.87:0/130615410 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1080af070 con 0x7fe120070c20 2024-04-25T10:18:21.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe11fe00640 1 -- 172.21.15.87:0/130615410 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080aa9d0 con 0x7fe120070c20 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- 172.21.15.87:0/130615410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120070c20 msgr2=0x7fe12015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 --2- 172.21.15.87:0/130615410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120070c20 0x7fe12015e170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe1200aec50 tx=0x7fe10801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- 172.21.15.87:0/130615410 shutdown_connections 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 --2- 172.21.15.87:0/130615410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe120070c20 0x7fe12015e170 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 --2- 172.21.15.87:0/130615410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 --2- 172.21.15.87:0/130615410 >> v2:172.21.15.169:3300/0 conn(0x7fe1200cf930 0x7fe1200cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- 172.21.15.87:0/130615410 >> 172.21.15.87:0/130615410 conn(0x7fe120066b70 msgr2=0x7fe120066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- 172.21.15.87:0/130615410 shutdown_connections 2024-04-25T10:18:21.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- 172.21.15.87:0/130615410 wait complete. 2024-04-25T10:18:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 Processor -- start 2024-04-25T10:18:21.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- start start 2024-04-25T10:18:21.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120070c20 0x7fe120155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1200cf930 0x7fe120161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12015e6b0 con 0x7fe1200cf930 2024-04-25T10:18:21.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12006e130 con 0x7fe120070c20 2024-04-25T10:18:21.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe127400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe120161790 con 0x7fe1200ae020 2024-04-25T10:18:21.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe125400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1200cf930 0x7fe120161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe125400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1200cf930 0x7fe120161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54542/0 (socket says 172.21.15.87:54542) 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe125400640 1 -- 172.21.15.87:0/2959687434 learned_addr learned my addr 172.21.15.87:0/2959687434 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe117e00640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.341+0000 7fe124a00640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120070c20 0x7fe120155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe125400640 1 -- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 msgr2=0x7fe1200cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe125400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe125400640 1 -- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120070c20 msgr2=0x7fe120155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe125400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120070c20 0x7fe120155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe125400640 1 -- 172.21.15.87:0/2959687434 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe120157d90 con 0x7fe1200cf930 2024-04-25T10:18:21.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe125400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1200cf930 0x7fe120161d80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe118007e90 tx=0x7fe11800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe11d600640 1 -- 172.21.15.87:0/2959687434 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118014420 con 0x7fe1200cf930 2024-04-25T10:18:21.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe11d600640 1 -- 172.21.15.87:0/2959687434 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118019070 con 0x7fe1200cf930 2024-04-25T10:18:21.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe11d600640 1 -- 172.21.15.87:0/2959687434 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118014820 con 0x7fe1200cf930 2024-04-25T10:18:21.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe120158070 con 0x7fe1200cf930 2024-04-25T10:18:21.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe120158630 con 0x7fe1200cf930 2024-04-25T10:18:21.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12004ef30 con 0x7fe1200cf930 2024-04-25T10:18:21.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe11d600640 1 -- 172.21.15.87:0/2959687434 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1180149c0 con 0x7fe1200cf930 2024-04-25T10:18:21.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe11d600640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f60 0x7fe0fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.345+0000 7fe11d600640 1 -- 172.21.15.87:0/2959687434 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe118085820 con 0x7fe1200cf930 2024-04-25T10:18:21.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.349+0000 7fe124a00640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f60 0x7fe0fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.349+0000 7fe124a00640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f60 0x7fe0fc069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe120157790 tx=0x7fe110008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.353+0000 7fe11d600640 1 -- 172.21.15.87:0/2959687434 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1180560e0 con 0x7fe1200cf930 2024-04-25T10:18:21.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.505+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe120068bc0 con 0x7fe1200cf930 2024-04-25T10:18:21.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.505+0000 7fe11d600640 1 -- 172.21.15.87:0/2959687434 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16819 (secure 0 0 0) 0x7fe11805a180 con 0x7fe1200cf930 2024-04-25T10:18:21.510 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:21.510 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":76,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:17:57.908660+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:47.854894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:48.494317+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f60 msgr2=0x7fe0fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f60 0x7fe0fc069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe120157790 tx=0x7fe110008040 comp rx=0 tx=0).stop 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1200cf930 msgr2=0x7fe120161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1200cf930 0x7fe120161d80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe118007e90 tx=0x7fe11800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 shutdown_connections 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe0fc066f60 0x7fe0fc069420 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1200cf930 0x7fe120161d80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1200ae020 0x7fe1200cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 --2- 172.21.15.87:0/2959687434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe120070c20 0x7fe120155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 >> 172.21.15.87:0/2959687434 conn(0x7fe120066b70 msgr2=0x7fe1200ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 shutdown_connections 2024-04-25T10:18:21.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.509+0000 7fe127400640 1 -- 172.21.15.87:0/2959687434 wait complete. 2024-04-25T10:18:21.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:47.854894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:48.494317+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:21.531 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:21.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.665+0000 7f220ea00640 1 Processor -- start 2024-04-25T10:18:21.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.665+0000 7f220ea00640 1 -- start start 2024-04-25T10:18:21.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22080b2b70 0x7f220815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f220806dfb0 0x7f2208070150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220ea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2208157e20 con 0x7f2208157a40 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220ea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220806c7c0 con 0x7f22080b2b70 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220806caa0 con 0x7f220806dfb0 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22080b2b70 0x7f220815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22080b2b70 0x7f220815aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54558/0 (socket says 172.21.15.87:54558) 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 -- 172.21.15.87:0/2005002620 learned_addr learned my addr 172.21.15.87:0/2005002620 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220d600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2208157a40 0x7f22080b2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35664/0 (socket says 172.21.15.87:35664) 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220e000640 1 --2- 172.21.15.87:0/2005002620 >> v2:172.21.15.169:3300/0 conn(0x7f220806dfb0 0x7f2208070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 -- 172.21.15.87:0/2005002620 >> v2:172.21.15.169:3300/0 conn(0x7f220806dfb0 msgr2=0x7f2208070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 --2- 172.21.15.87:0/2005002620 >> v2:172.21.15.169:3300/0 conn(0x7f220806dfb0 0x7f2208070150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 -- 172.21.15.87:0/2005002620 >> v1:172.21.15.87:6789/0 conn(0x7f2208157a40 legacy=0x7f22080b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 -- 172.21.15.87:0/2005002620 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220806cd80 con 0x7f22080b2b70 2024-04-25T10:18:21.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220e000640 1 --2- 172.21.15.87:0/2005002620 >> v2:172.21.15.169:3300/0 conn(0x7f220806dfb0 0x7f2208070150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:21.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.669+0000 7f220cc00640 1 --2- 172.21.15.87:0/2005002620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22080b2b70 0x7f220815aeb0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2208067910 tx=0x7f21f401d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13dd0ee9dcbd78f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f2207e00640 1 -- 172.21.15.87:0/2005002620 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f40b3070 con 0x7f22080b2b70 2024-04-25T10:18:21.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f2207e00640 1 -- 172.21.15.87:0/2005002620 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f40a67d0 con 0x7f22080b2b70 2024-04-25T10:18:21.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f2207e00640 1 -- 172.21.15.87:0/2005002620 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f400b040 con 0x7f22080b2b70 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- 172.21.15.87:0/2005002620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22080b2b70 msgr2=0x7f220815aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 --2- 172.21.15.87:0/2005002620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22080b2b70 0x7f220815aeb0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2208067910 tx=0x7f21f401d8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- 172.21.15.87:0/2005002620 shutdown_connections 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 --2- 172.21.15.87:0/2005002620 >> v2:172.21.15.169:3300/0 conn(0x7f220806dfb0 0x7f2208070150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 --2- 172.21.15.87:0/2005002620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22080b2b70 0x7f220815aeb0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- 172.21.15.87:0/2005002620 >> 172.21.15.87:0/2005002620 conn(0x7f22080743f0 msgr2=0x7f22080669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- 172.21.15.87:0/2005002620 shutdown_connections 2024-04-25T10:18:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- 172.21.15.87:0/2005002620 wait complete. 2024-04-25T10:18:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 Processor -- start 2024-04-25T10:18:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- start start 2024-04-25T10:18:21.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220806dfb0 0x7f220813e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080b2b70 0x7f220815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208157a40 0x7f2208070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2208070810 con 0x7f220806dfb0 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2208070690 con 0x7f22080b2b70 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2208070990 con 0x7f2208157a40 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220806dfb0 0x7f220813e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220806dfb0 0x7f220813e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54570/0 (socket says 172.21.15.87:54570) 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.673+0000 7f220d600640 1 -- 172.21.15.87:0/377474225 learned_addr learned my addr 172.21.15.87:0/377474225 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220cc00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080b2b70 0x7f220815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220e000640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208157a40 0x7f2208070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220d600640 1 -- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208157a40 msgr2=0x7f2208070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220d600640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208157a40 0x7f2208070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220d600640 1 -- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080b2b70 msgr2=0x7f220815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220d600640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080b2b70 0x7f220815b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220d600640 1 -- 172.21.15.87:0/377474225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2208162f60 con 0x7f220806dfb0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220d600640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220806dfb0 0x7f220813e050 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7c0 tx=0x7f21fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f2205600640 1 -- 172.21.15.87:0/377474225 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc017070 con 0x7f220806dfb0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f2205600640 1 -- 172.21.15.87:0/377474225 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc00a640 con 0x7f220806dfb0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f2205600640 1 -- 172.21.15.87:0/377474225 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00a9f0 con 0x7f220806dfb0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2208163240 con 0x7f220806dfb0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2208163800 con 0x7f220806dfb0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2208075120 con 0x7f220806dfb0 2024-04-25T10:18:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.677+0000 7f2205600640 1 -- 172.21.15.87:0/377474225 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21fc01a070 con 0x7f220806dfb0 2024-04-25T10:18:21.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.681+0000 7f2205600640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e8066f60 0x7f21e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.681+0000 7f2205600640 1 -- 172.21.15.87:0/377474225 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f21fc0846c0 con 0x7f220806dfb0 2024-04-25T10:18:21.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.681+0000 7f220cc00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e8066f60 0x7f21e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.681+0000 7f220cc00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e8066f60 0x7f21e8069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21f40b0000 tx=0x7f21f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.685+0000 7f2205600640 1 -- 172.21.15.87:0/377474225 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21fc054f80 con 0x7f220806dfb0 2024-04-25T10:18:21.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.837+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2208159520 con 0x7f220806dfb0 2024-04-25T10:18:21.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.837+0000 7f2205600640 1 -- 172.21.15.87:0/377474225 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21fc059020 con 0x7f220806dfb0 2024-04-25T10:18:21.840 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:18:21.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e8066f60 msgr2=0x7f21e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e8066f60 0x7f21e8069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21f40b0000 tx=0x7f21f4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:21.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220806dfb0 msgr2=0x7f220813e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220806dfb0 0x7f220813e050 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7c0 tx=0x7f21fc00cc80 comp rx=0 tx=0).stop 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 shutdown_connections 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e8066f60 0x7f21e8069420 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208157a40 0x7f2208070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22080b2b70 0x7f220815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 --2- 172.21.15.87:0/377474225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220806dfb0 0x7f220813e050 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 >> 172.21.15.87:0/377474225 conn(0x7f22080743f0 msgr2=0x7f22080b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 shutdown_connections 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.841+0000 7f220ea00640 1 -- 172.21.15.87:0/377474225 wait complete. 2024-04-25T10:18:21.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:22.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.997+0000 7f4329000640 1 Processor -- start 2024-04-25T10:18:22.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.997+0000 7f4329000640 1 -- start start 2024-04-25T10:18:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.997+0000 7f4329000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4324159540 0x7f43240ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.997+0000 7f4329000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f43240721f0 0x7f4324070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:21.997+0000 7f4329000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4324159920 con 0x7f43240aebd0 2024-04-25T10:18:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4329000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432415c2f0 con 0x7f4324159540 2024-04-25T10:18:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4329000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432415c5d0 con 0x7f43240721f0 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4324159540 0x7f43240ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4324159540 0x7f43240ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54582/0 (socket says 172.21.15.87:54582) 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 -- 172.21.15.87:0/4233380813 learned_addr learned my addr 172.21.15.87:0/4233380813 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4319400640 1 --1- 172.21.15.87:0/4233380813 >> v1:172.21.15.87:6789/0 conn(0x7f43240aebd0 0x7f4324071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35674/0 (socket says 172.21.15.87:35674) 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4322800640 1 --2- 172.21.15.87:0/4233380813 >> v2:172.21.15.169:3300/0 conn(0x7f43240721f0 0x7f4324070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 -- 172.21.15.87:0/4233380813 >> v2:172.21.15.169:3300/0 conn(0x7f43240721f0 msgr2=0x7f4324070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 --2- 172.21.15.87:0/4233380813 >> v2:172.21.15.169:3300/0 conn(0x7f43240721f0 0x7f4324070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 -- 172.21.15.87:0/4233380813 >> v1:172.21.15.87:6789/0 conn(0x7f43240aebd0 legacy=0x7f4324071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 -- 172.21.15.87:0/4233380813 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432415c8b0 con 0x7f4324159540 2024-04-25T10:18:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321400640 1 -- 172.21.15.87:0/4233380813 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831546110 0 0) 0x7f4324159920 con 0x7f43240aebd0 2024-04-25T10:18:22.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321e00640 1 --2- 172.21.15.87:0/4233380813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324159540 0x7f43240ae690 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f43240cd200 tx=0x7f431401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520da635b400c6ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321400640 1 -- 172.21.15.87:0/4233380813 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140a57d0 con 0x7f4324159540 2024-04-25T10:18:22.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321400640 1 -- 172.21.15.87:0/4233380813 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43140a5970 con 0x7f4324159540 2024-04-25T10:18:22.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.001+0000 7f4321400640 1 -- 172.21.15.87:0/4233380813 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140a5d40 con 0x7f4324159540 2024-04-25T10:18:22.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- 172.21.15.87:0/4233380813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324159540 msgr2=0x7f43240ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 --2- 172.21.15.87:0/4233380813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324159540 0x7f43240ae690 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f43240cd200 tx=0x7f431401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- 172.21.15.87:0/4233380813 shutdown_connections 2024-04-25T10:18:22.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 --2- 172.21.15.87:0/4233380813 >> v2:172.21.15.169:3300/0 conn(0x7f43240721f0 0x7f4324070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 --2- 172.21.15.87:0/4233380813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324159540 0x7f43240ae690 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- 172.21.15.87:0/4233380813 >> 172.21.15.87:0/4233380813 conn(0x7f43240669a0 msgr2=0x7f4324066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- 172.21.15.87:0/4233380813 shutdown_connections 2024-04-25T10:18:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- 172.21.15.87:0/4233380813 wait complete. 2024-04-25T10:18:22.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 Processor -- start 2024-04-25T10:18:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- start start 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43240721f0 0x7f432414ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43240aebd0 0x7f432414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324159540 0x7f4324157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324071220 con 0x7f43240721f0 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240710a0 con 0x7f4324159540 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4329000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240713a0 con 0x7f43240aebd0 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43240721f0 0x7f432414ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43240721f0 0x7f432414ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54596/0 (socket says 172.21.15.87:54596) 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 -- 172.21.15.87:0/4062823152 learned_addr learned my addr 172.21.15.87:0/4062823152 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4322800640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324159540 0x7f4324157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4319400640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43240aebd0 0x7f432414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 -- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43240aebd0 msgr2=0x7f432414e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43240aebd0 0x7f432414e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 -- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324159540 msgr2=0x7f4324157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324159540 0x7f4324157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 -- 172.21.15.87:0/4062823152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432414f7e0 con 0x7f43240721f0 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.005+0000 7f4321e00640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43240721f0 0x7f432414ef90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f431401b0a0 tx=0x7f4314019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f431aa00640 1 -- 172.21.15.87:0/4062823152 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140b8470 con 0x7f43240721f0 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f431aa00640 1 -- 172.21.15.87:0/4062823152 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43140b8610 con 0x7f43240721f0 2024-04-25T10:18:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f431aa00640 1 -- 172.21.15.87:0/4062823152 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140b89e0 con 0x7f43240721f0 2024-04-25T10:18:22.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432414fac0 con 0x7f43240721f0 2024-04-25T10:18:22.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4324150080 con 0x7f43240721f0 2024-04-25T10:18:22.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43240cc3b0 con 0x7f43240721f0 2024-04-25T10:18:22.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f431aa00640 1 -- 172.21.15.87:0/4062823152 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43140c0020 con 0x7f43240721f0 2024-04-25T10:18:22.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f431aa00640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4300066f90 0x7f4300069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f431aa00640 1 -- 172.21.15.87:0/4062823152 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f431411fb40 con 0x7f43240721f0 2024-04-25T10:18:22.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f4319400640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4300066f90 0x7f4300069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.009+0000 7f4319400640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4300066f90 0x7f4300069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4324153130 tx=0x7f430c007b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.013+0000 7f431aa00640 1 -- 172.21.15.87:0/4062823152 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43140c7020 con 0x7f43240721f0 2024-04-25T10:18:22.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.165+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f432404ef30 con 0x7f43240721f0 2024-04-25T10:18:22.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f431aa00640 1 -- 172.21.15.87:0/4062823152 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43140165e0 con 0x7f43240721f0 2024-04-25T10:18:22.170 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:18:22.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4300066f90 msgr2=0x7f4300069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4300066f90 0x7f4300069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4324153130 tx=0x7f430c007b60 comp rx=0 tx=0).stop 2024-04-25T10:18:22.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43240721f0 msgr2=0x7f432414ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43240721f0 0x7f432414ef90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f431401b0a0 tx=0x7f4314019790 comp rx=0 tx=0).stop 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 shutdown_connections 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4300066f90 0x7f4300069450 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324159540 0x7f4324157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43240aebd0 0x7f432414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 --2- 172.21.15.87:0/4062823152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43240721f0 0x7f432414ef90 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 >> 172.21.15.87:0/4062823152 conn(0x7f43240669a0 msgr2=0x7f43240cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 shutdown_connections 2024-04-25T10:18:22.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.169+0000 7f4329000640 1 -- 172.21.15.87:0/4062823152 wait complete. 2024-04-25T10:18:22.190 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:22.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b5b800640 1 Processor -- start 2024-04-25T10:18:22.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b5b800640 1 -- start start 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b5b800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4b54159540 0x7f4b540ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b5b800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4b540721f0 0x7f4b54070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b5b800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b54159920 con 0x7f4b540aebd0 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b5b800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5415c2f0 con 0x7f4b540721f0 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b5b800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5415c5d0 con 0x7f4b54159540 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b59800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4b540721f0 0x7f4b54070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.325+0000 7f4b59800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4b540721f0 0x7f4b54070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54604/0 (socket says 172.21.15.87:54604) 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b59800640 1 -- 172.21.15.87:0/2470717405 learned_addr learned my addr 172.21.15.87:0/2470717405 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b4be00640 1 --1- 172.21.15.87:0/2470717405 >> v1:172.21.15.87:6789/0 conn(0x7f4b540aebd0 0x7f4b54071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35682/0 (socket says 172.21.15.87:35682) 2024-04-25T10:18:22.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b58e00640 1 --2- 172.21.15.87:0/2470717405 >> v2:172.21.15.169:3300/0 conn(0x7f4b54159540 0x7f4b540ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b59800640 1 -- 172.21.15.87:0/2470717405 >> v2:172.21.15.169:3300/0 conn(0x7f4b54159540 msgr2=0x7f4b540ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:22.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b59800640 1 --2- 172.21.15.87:0/2470717405 >> v2:172.21.15.169:3300/0 conn(0x7f4b54159540 0x7f4b540ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b59800640 1 -- 172.21.15.87:0/2470717405 >> v1:172.21.15.87:6789/0 conn(0x7f4b540aebd0 legacy=0x7f4b54071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b59800640 1 -- 172.21.15.87:0/2470717405 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5415c8b0 con 0x7f4b540721f0 2024-04-25T10:18:22.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b59800640 1 --2- 172.21.15.87:0/2470717405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540721f0 0x7f4b54070350 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b4c20 tx=0x7f4b4c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdf7c41d34fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b53e00640 1 -- 172.21.15.87:0/2470717405 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0a57d0 con 0x7f4b540721f0 2024-04-25T10:18:22.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b53e00640 1 -- 172.21.15.87:0/2470717405 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c0a5970 con 0x7f4b540721f0 2024-04-25T10:18:22.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.329+0000 7f4b53e00640 1 -- 172.21.15.87:0/2470717405 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0a5d40 con 0x7f4b540721f0 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- 172.21.15.87:0/2470717405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540721f0 msgr2=0x7f4b54070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 --2- 172.21.15.87:0/2470717405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540721f0 0x7f4b54070350 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b4c20 tx=0x7f4b4c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- 172.21.15.87:0/2470717405 shutdown_connections 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 --2- 172.21.15.87:0/2470717405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540721f0 0x7f4b54070350 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 --2- 172.21.15.87:0/2470717405 >> v2:172.21.15.169:3300/0 conn(0x7f4b54159540 0x7f4b540ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- 172.21.15.87:0/2470717405 >> 172.21.15.87:0/2470717405 conn(0x7f4b540669a0 msgr2=0x7f4b54066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- 172.21.15.87:0/2470717405 shutdown_connections 2024-04-25T10:18:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- 172.21.15.87:0/2470717405 wait complete. 2024-04-25T10:18:22.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 Processor -- start 2024-04-25T10:18:22.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- start start 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b540721f0 0x7f4b54158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540aebd0 0x7f4b541f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b54159540 0x7f4b54157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54071220 con 0x7f4b540aebd0 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540710a0 con 0x7f4b54159540 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b5b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540713a0 con 0x7f4b540721f0 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540aebd0 0x7f4b541f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540aebd0 0x7f4b541f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54608/0 (socket says 172.21.15.87:54608) 2024-04-25T10:18:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 -- 172.21.15.87:0/744512733 learned_addr learned my addr 172.21.15.87:0/744512733 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b59800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b54159540 0x7f4b54157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b58e00640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b540721f0 0x7f4b54158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 -- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b540721f0 msgr2=0x7f4b54158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b540721f0 0x7f4b54158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 -- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b54159540 msgr2=0x7f4b54157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b54159540 0x7f4b54157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 -- 172.21.15.87:0/744512733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b541580a0 con 0x7f4b540aebd0 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b58e00640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b540721f0 0x7f4b54158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b59800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b54159540 0x7f4b54157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.333+0000 7f4b4be00640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540aebd0 0x7f4b541f6c10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c420 tx=0x7f4b3c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b51600640 1 -- 172.21.15.87:0/744512733 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0074e0 con 0x7f4b540aebd0 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b51600640 1 -- 172.21.15.87:0/744512733 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c007680 con 0x7f4b540aebd0 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b54157dc0 con 0x7f4b540aebd0 2024-04-25T10:18:22.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b51600640 1 -- 172.21.15.87:0/744512733 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c004a30 con 0x7f4b540aebd0 2024-04-25T10:18:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b54151970 con 0x7f4b540aebd0 2024-04-25T10:18:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5404ef30 con 0x7f4b540aebd0 2024-04-25T10:18:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b51600640 1 -- 172.21.15.87:0/744512733 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b3c01a020 con 0x7f4b540aebd0 2024-04-25T10:18:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b51600640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b30066f60 0x7f4b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b51600640 1 -- 172.21.15.87:0/744512733 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4b3c084980 con 0x7f4b540aebd0 2024-04-25T10:18:22.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b58e00640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b30066f60 0x7f4b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.337+0000 7f4b58e00640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b30066f60 0x7f4b30069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400b9f0 tx=0x7f4b44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.341+0000 7f4b51600640 1 -- 172.21.15.87:0/744512733 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b3c055890 con 0x7f4b540aebd0 2024-04-25T10:18:22.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b540cf180 con 0x7f4b540aebd0 2024-04-25T10:18:22.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b51600640 1 -- 172.21.15.87:0/744512733 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b3c055190 con 0x7f4b540aebd0 2024-04-25T10:18:22.499 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:18:22.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b30066f60 msgr2=0x7f4b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b30066f60 0x7f4b30069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400b9f0 tx=0x7f4b44008040 comp rx=0 tx=0).stop 2024-04-25T10:18:22.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540aebd0 msgr2=0x7f4b541f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540aebd0 0x7f4b541f6c10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c420 tx=0x7f4b3c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 shutdown_connections 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4b30066f60 0x7f4b30069420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b54159540 0x7f4b54157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b540aebd0 0x7f4b541f6c10 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 --2- 172.21.15.87:0/744512733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b540721f0 0x7f4b54158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 >> 172.21.15.87:0/744512733 conn(0x7f4b540669a0 msgr2=0x7f4b540adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 shutdown_connections 2024-04-25T10:18:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.497+0000 7f4b5b800640 1 -- 172.21.15.87:0/744512733 wait complete. 2024-04-25T10:18:22.519 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:22.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4374e00640 1 Processor -- start 2024-04-25T10:18:22.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4374e00640 1 -- start start 2024-04-25T10:18:22.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4374e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 0x7f437015c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4374e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4370071050 con 0x7f43700d0040 2024-04-25T10:18:22.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4374e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700712f0 con 0x7f43701550b0 2024-04-25T10:18:22.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4374e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437015bda0 con 0x7f437015b910 2024-04-25T10:18:22.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4365400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f43700d0040 0x7f437015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35692/0 (socket says 172.21.15.87:35692) 2024-04-25T10:18:22.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f4365400640 1 -- 172.21.15.87:0/4070257090 learned_addr learned my addr 172.21.15.87:0/4070257090 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f436e800640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 0x7f437015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673909011 0 0) 0x7f4370071050 con 0x7f43700d0040 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4354005740 con 0x7f43700d0040 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969715314 0 0) 0x7f43700712f0 con 0x7f43701550b0 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4370071050 con 0x7f43701550b0 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.653+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4025463809 0 0) 0x7f4354005740 con 0x7f43700d0040 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 msgr2=0x7f437015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 0x7f437015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 >> v1:172.21.15.169:6789/0 conn(0x7f43701550b0 legacy=0x7f43700cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700715d0 con 0x7f43700d0040 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f4358001a00 con 0x7f43700d0040 2024-04-25T10:18:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 >> v1:172.21.15.87:6789/0 conn(0x7f43700d0040 legacy=0x7f437015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354005d90 0x7f4354006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4354006a20 0x7f4354006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 0x7f43540073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435400dc90 con 0x7f4354006a20 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435400de10 con 0x7f4354005d90 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435400dfe0 con 0x7f437015b910 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f4365400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4354006a20 0x7f4354006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436de00640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354005d90 0x7f4354006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436e800640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 0x7f43540073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f4365400640 1 -- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 msgr2=0x7f43540073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f4365400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 0x7f43540073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f4365400640 1 -- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354005d90 msgr2=0x7f4354006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f4365400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354005d90 0x7f4354006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f4365400640 1 -- 172.21.15.87:0/4070257090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4354005740 con 0x7f4354006a20 2024-04-25T10:18:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436e800640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437015b910 0x7f43540073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f4365400640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4354006a20 0x7f4354006ea0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f435800a800 tx=0x7f435801d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206f88fdf357f658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800a430 con 0x7f4354006a20 2024-04-25T10:18:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43580072d0 con 0x7f4354006a20 2024-04-25T10:18:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.657+0000 7f436d400640 1 -- 172.21.15.87:0/4070257090 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580076c0 con 0x7f4354006a20 2024-04-25T10:18:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- 172.21.15.87:0/4070257090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4354006a20 msgr2=0x7f4354006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 --2- 172.21.15.87:0/4070257090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4354006a20 0x7f4354006ea0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f435800a800 tx=0x7f435801d2a0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f436e800640 1 -- 172.21.15.87:0/4070257090 reap_dead start 2024-04-25T10:18:22.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- 172.21.15.87:0/4070257090 shutdown_connections 2024-04-25T10:18:22.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- 172.21.15.87:0/4070257090 >> 172.21.15.87:0/4070257090 conn(0x7f4370066b70 msgr2=0x7f4370066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- 172.21.15.87:0/4070257090 shutdown_connections 2024-04-25T10:18:22.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- 172.21.15.87:0/4070257090 wait complete. 2024-04-25T10:18:22.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 Processor -- start 2024-04-25T10:18:22.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- start start 2024-04-25T10:18:22.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354006a20 0x7f4370155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43701f4ae0 0x7f43701f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4370153570 0x7f4370157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700720b0 con 0x7f43701f4ae0 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4370071b90 con 0x7f4354006a20 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4374e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437006c090 con 0x7f4370153570 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43701f4ae0 0x7f43701f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43701f4ae0 0x7f43701f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54618/0 (socket says 172.21.15.87:54618) 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 -- 172.21.15.87:0/799801133 learned_addr learned my addr 172.21.15.87:0/799801133 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f436de00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354006a20 0x7f4370155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 -- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4370153570 msgr2=0x7f4370157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4370153570 0x7f4370157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 -- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354006a20 msgr2=0x7f4370155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354006a20 0x7f4370155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f4365400640 1 -- 172.21.15.87:0/799801133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700cc890 con 0x7f43701f4ae0 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.661+0000 7f436de00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354006a20 0x7f4370155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4365400640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43701f4ae0 0x7f43701f43c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4358007040 tx=0x7f4354007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4366a00640 1 -- 172.21.15.87:0/799801133 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580070d0 con 0x7f43701f4ae0 2024-04-25T10:18:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4366a00640 1 -- 172.21.15.87:0/799801133 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358001800 con 0x7f43701f4ae0 2024-04-25T10:18:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43700ccb70 con 0x7f43701f4ae0 2024-04-25T10:18:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4366a00640 1 -- 172.21.15.87:0/799801133 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580019f0 con 0x7f43701f4ae0 2024-04-25T10:18:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43700cd130 con 0x7f43701f4ae0 2024-04-25T10:18:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437004ef30 con 0x7f43701f4ae0 2024-04-25T10:18:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4366a00640 1 -- 172.21.15.87:0/799801133 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43580bd020 con 0x7f43701f4ae0 2024-04-25T10:18:22.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4366a00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f434c067100 0x7f434c0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f4366a00640 1 -- 172.21.15.87:0/799801133 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f435811f1e0 con 0x7f43701f4ae0 2024-04-25T10:18:22.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.665+0000 7f436de00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f434c067100 0x7f434c0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.669+0000 7f436de00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f434c067100 0x7f434c0695c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4360000f70 tx=0x7f4360000f00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.669+0000 7f4366a00640 1 -- 172.21.15.87:0/799801133 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43580c4020 con 0x7f43701f4ae0 2024-04-25T10:18:22.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.833+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f437006d0e0 con 0x7f43701f4ae0 2024-04-25T10:18:22.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.833+0000 7f4366a00640 1 -- 172.21.15.87:0/799801133 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v76) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4358016fb0 con 0x7f43701f4ae0 2024-04-25T10:18:22.837 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:18:22.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f434c067100 msgr2=0x7f434c0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f434c067100 0x7f434c0695c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4360000f70 tx=0x7f4360000f00 comp rx=0 tx=0).stop 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43701f4ae0 msgr2=0x7f43701f43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43701f4ae0 0x7f43701f43c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4358007040 tx=0x7f4354007a50 comp rx=0 tx=0).stop 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 shutdown_connections 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f434c067100 0x7f434c0695c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4370153570 0x7f4370157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43701f4ae0 0x7f43701f43c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 --2- 172.21.15.87:0/799801133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4354006a20 0x7f4370155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 >> 172.21.15.87:0/799801133 conn(0x7f4370066b70 msgr2=0x7f437006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 shutdown_connections 2024-04-25T10:18:22.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:22.837+0000 7f4374e00640 1 -- 172.21.15.87:0/799801133 wait complete. 2024-04-25T10:18:22.856 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:18:22.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.857 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:22.893 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:18:22.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.893 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:18:22.942 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:18:22.942 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:22.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:22.951 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:18:22.951 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:22.987 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:22.990 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:22.995 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:22.995 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:18:22.995 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:23.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:23.006 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:23.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff11bc00640 1 Processor -- start 2024-04-25T10:18:23.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff11bc00640 1 -- start start 2024-04-25T10:18:23.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff11bc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff1140d1820 0x7ff1140d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff11bc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1140b1af0 con 0x7ff1140b15b0 2024-04-25T10:18:23.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff11bc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1140b1dd0 con 0x7ff11415e050 2024-04-25T10:18:23.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff11bc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1140b20b0 con 0x7ff1140d1820 2024-04-25T10:18:23.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff119c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff1140b15b0 0x7ff1140cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35694/0 (socket says 172.21.15.87:35694) 2024-04-25T10:18:23.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff119c00640 1 -- 172.21.15.87:0/1755307320 learned_addr learned my addr 172.21.15.87:0/1755307320 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:23.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff119200640 1 --2- 172.21.15.87:0/1755307320 >> v2:172.21.15.169:3300/0 conn(0x7ff1140d1820 0x7ff1140d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813305249 0 0) 0x7ff1140b1af0 con 0x7ff1140b15b0 2024-04-25T10:18:23.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f8003530 con 0x7ff1140b15b0 2024-04-25T10:18:23.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335792870 0 0) 0x7ff1140b1dd0 con 0x7ff11415e050 2024-04-25T10:18:23.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1140b1af0 con 0x7ff11415e050 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272689243 0 0) 0x7ff0f8003530 con 0x7ff1140b15b0 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 >> v2:172.21.15.169:3300/0 conn(0x7ff1140d1820 msgr2=0x7ff1140d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 --2- 172.21.15.87:0/1755307320 >> v2:172.21.15.169:3300/0 conn(0x7ff1140d1820 0x7ff1140d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 >> v1:172.21.15.169:6789/0 conn(0x7ff11415e050 legacy=0x7ff1140b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1140b2390 con 0x7ff1140b15b0 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ff10c001600 con 0x7ff1140b15b0 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 >> v1:172.21.15.87:6789/0 conn(0x7ff1140b15b0 legacy=0x7ff1140cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff0f8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0f8004810 0x7ff0f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f80051d0 0x7ff0f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140b20b0 con 0x7ff0f8003ba0 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f800c0a0 con 0x7ff0f80051d0 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f800c2a0 con 0x7ff0f8004810 2024-04-25T10:18:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff119200640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff0f8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff119c00640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f80051d0 0x7ff0f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.157+0000 7ff113e00640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0f8004810 0x7ff0f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff119200640 1 -- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0f8004810 msgr2=0x7ff0f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff119200640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0f8004810 0x7ff0f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff119200640 1 -- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f80051d0 msgr2=0x7ff0f800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff119200640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f80051d0 0x7ff0f800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff119200640 1 -- 172.21.15.87:0/1755307320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8003530 con 0x7ff0f8003ba0 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff119c00640 1 -- 172.21.15.87:0/1755307320 reap_dead start 2024-04-25T10:18:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff113e00640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0f8004810 0x7ff0f8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff119200640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff0f8003fa0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff104009fc0 tx=0x7ff10401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ea86458c810cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1040b2070 con 0x7ff0f8003ba0 2024-04-25T10:18:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1040a5b50 con 0x7ff0f8003ba0 2024-04-25T10:18:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff113400640 1 -- 172.21.15.87:0/1755307320 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104009c10 con 0x7ff0f8003ba0 2024-04-25T10:18:23.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff11bc00640 1 -- 172.21.15.87:0/1755307320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 msgr2=0x7ff0f8003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff11bc00640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff0f8003fa0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff104009fc0 tx=0x7ff10401cc80 comp rx=0 tx=0).stop 2024-04-25T10:18:23.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff11bc00640 1 -- 172.21.15.87:0/1755307320 shutdown_connections 2024-04-25T10:18:23.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff11bc00640 1 --2- 172.21.15.87:0/1755307320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff0f8003fa0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff11bc00640 1 -- 172.21.15.87:0/1755307320 >> 172.21.15.87:0/1755307320 conn(0x7ff11409a9d0 msgr2=0x7ff11409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff11bc00640 1 -- 172.21.15.87:0/1755307320 shutdown_connections 2024-04-25T10:18:23.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.161+0000 7ff11bc00640 1 -- 172.21.15.87:0/1755307320 wait complete. 2024-04-25T10:18:23.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 Processor -- start 2024-04-25T10:18:23.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 -- start start 2024-04-25T10:18:23.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff11415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1141fa0b0 0x7ff11406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff11406fd10 0x7ff114160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140ce990 con 0x7ff0f8003ba0 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140d2220 con 0x7ff11406fd10 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff11bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140ceb10 con 0x7ff1141fa0b0 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff11415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff11415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54622/0 (socket says 172.21.15.87:54622) 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 -- 172.21.15.87:0/3368735372 learned_addr learned my addr 172.21.15.87:0/3368735372 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff113e00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1141fa0b0 0x7ff11406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119c00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff11406fd10 0x7ff114160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 -- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1141fa0b0 msgr2=0x7ff11406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1141fa0b0 0x7ff11406f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 -- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff11406fd10 msgr2=0x7ff114160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff11406fd10 0x7ff114160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 -- 172.21.15.87:0/3368735372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff114158190 con 0x7ff0f8003ba0 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119c00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff11406fd10 0x7ff114160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff113e00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1141fa0b0 0x7ff11406f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff119200640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff11415ac00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff1040b0040 tx=0x7ff104009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff110c00640 1 -- 172.21.15.87:0/3368735372 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1040ba070 con 0x7ff0f8003ba0 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.165+0000 7ff110c00640 1 -- 172.21.15.87:0/3368735372 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1040ada40 con 0x7ff0f8003ba0 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff110c00640 1 -- 172.21.15.87:0/3368735372 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1040b2040 con 0x7ff0f8003ba0 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1141585e0 con 0x7ff0f8003ba0 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1141544e0 con 0x7ff0f8003ba0 2024-04-25T10:18:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11404ef30 con 0x7ff0f8003ba0 2024-04-25T10:18:23.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff110c00640 1 -- 172.21.15.87:0/3368735372 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1040b6410 con 0x7ff0f8003ba0 2024-04-25T10:18:23.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff110c00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff0f4066f60 0x7ff0f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff110c00640 1 -- 172.21.15.87:0/3368735372 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff10411f6c0 con 0x7ff0f8003ba0 2024-04-25T10:18:23.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff113e00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff0f4066f60 0x7ff0f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.169+0000 7ff113e00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff0f4066f60 0x7ff0f4069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff1140b0980 tx=0x7ff100009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.173+0000 7ff110c00640 1 -- 172.21.15.87:0/3368735372 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff104016990 con 0x7ff0f8003ba0 2024-04-25T10:18:23.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.361+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1140700f0 con 0x7ff0f8003ba0 2024-04-25T10:18:23.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.361+0000 7ff110c00640 1 -- 172.21.15.87:0/3368735372 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4937 (secure 0 0 0) 0x7ff1040f0010 con 0x7ff0f8003ba0 2024-04-25T10:18:23.365 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:23.365 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/817977905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":817977905},{"type":"v1","addr":"172.21.15.169:6835","nonce":817977905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/979883306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":979883306},{"type":"v1","addr":"172.21.15.87:6836","nonce":979883306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/796485952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":796485952},{"type":"v1","addr":"172.21.15.87:6837","nonce":796485952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:16:49.500890+0000","modified":"2024-04-25T10:16:57.376333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"d","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1643726385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1643726385},{"type":"v1","addr":"172.21.15.169:6834","nonce":1643726385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:18:23.365 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 54 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff0f4066f60 msgr2=0x7ff0f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff0f4066f60 0x7ff0f4069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff1140b0980 tx=0x7ff100009040 comp rx=0 tx=0).stop 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 msgr2=0x7ff11415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff11415ac00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff1040b0040 tx=0x7ff104009750 comp rx=0 tx=0).stop 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 shutdown_connections 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff0f4066f60 0x7ff0f4069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff11406fd10 0x7ff114160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1141fa0b0 0x7ff11406f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 --2- 172.21.15.87:0/3368735372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0f8003ba0 0x7ff11415ac00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 >> 172.21.15.87:0/3368735372 conn(0x7ff11409a9d0 msgr2=0x7ff11415bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 shutdown_connections 2024-04-25T10:18:23.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.365+0000 7ff11bc00640 1 -- 172.21.15.87:0/3368735372 wait complete. 2024-04-25T10:18:23.386 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:23.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.517+0000 7f7148600640 1 Processor -- start 2024-04-25T10:18:23.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.517+0000 7f7148600640 1 -- start start 2024-04-25T10:18:23.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7148600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f71400cf930 0x7f71400cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7148600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400ae020 0x7f71400ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7148600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71400d0250 con 0x7f7140070c20 2024-04-25T10:18:23.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7148600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714015bb90 con 0x7f71400cf930 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7148600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400ad840 con 0x7f71400ae020 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7145c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f71400cf930 0x7f71400cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7145c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f71400cf930 0x7f71400cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54636/0 (socket says 172.21.15.87:54636) 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7145c00640 1 -- 172.21.15.87:0/2769747223 learned_addr learned my addr 172.21.15.87:0/2769747223 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7146600640 1 --2- 172.21.15.87:0/2769747223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400ae020 0x7f71400ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7145c00640 1 -- 172.21.15.87:0/2769747223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400ae020 msgr2=0x7f71400ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7145c00640 1 --2- 172.21.15.87:0/2769747223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400ae020 0x7f71400ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7145c00640 1 -- 172.21.15.87:0/2769747223 >> v1:172.21.15.169:6789/0 conn(0x7f7140070c20 legacy=0x7f714015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.521+0000 7f7145c00640 1 -- 172.21.15.87:0/2769747223 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714015e9f0 con 0x7f71400cf930 2024-04-25T10:18:23.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7145c00640 1 --2- 172.21.15.87:0/2769747223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400cf930 0x7f71400cfd10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7134009e50 tx=0x7f713401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbd606b2d18b5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f713fe00640 1 -- 172.21.15.87:0/2769747223 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340a58d0 con 0x7f71400cf930 2024-04-25T10:18:23.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f713fe00640 1 -- 172.21.15.87:0/2769747223 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340a5a70 con 0x7f71400cf930 2024-04-25T10:18:23.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f713fe00640 1 -- 172.21.15.87:0/2769747223 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340a5e40 con 0x7f71400cf930 2024-04-25T10:18:23.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 -- 172.21.15.87:0/2769747223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400cf930 msgr2=0x7f71400cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 --2- 172.21.15.87:0/2769747223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400cf930 0x7f71400cfd10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7134009e50 tx=0x7f713401ca00 comp rx=0 tx=0).stop 2024-04-25T10:18:23.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 -- 172.21.15.87:0/2769747223 shutdown_connections 2024-04-25T10:18:23.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 --2- 172.21.15.87:0/2769747223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400ae020 0x7f71400ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 --2- 172.21.15.87:0/2769747223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400cf930 0x7f71400cfd10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 -- 172.21.15.87:0/2769747223 >> 172.21.15.87:0/2769747223 conn(0x7f7140066b70 msgr2=0x7f7140066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 -- 172.21.15.87:0/2769747223 shutdown_connections 2024-04-25T10:18:23.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 -- 172.21.15.87:0/2769747223 wait complete. 2024-04-25T10:18:23.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 Processor -- start 2024-04-25T10:18:23.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.525+0000 7f7148600640 1 -- start start 2024-04-25T10:18:23.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7140070c20 0x7f7140152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400ae020 0x7f7140160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400cf930 0x7f7140160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714016a8e0 con 0x7f71400ae020 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714006e130 con 0x7f7140070c20 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714016aa60 con 0x7f71400cf930 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400ae020 0x7f7140160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400ae020 0x7f7140160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54652/0 (socket says 172.21.15.87:54652) 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 -- 172.21.15.87:0/2468935937 learned_addr learned my addr 172.21.15.87:0/2468935937 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145c00640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7140070c20 0x7f7140152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7146600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400cf930 0x7f7140160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 -- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400cf930 msgr2=0x7f7140160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400cf930 0x7f7140160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 -- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7140070c20 msgr2=0x7f7140152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7140070c20 0x7f7140152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 -- 172.21.15.87:0/2468935937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140153d40 con 0x7f71400ae020 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7145200640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400ae020 0x7f7140160330 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c420 tx=0x7f712c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f713d600640 1 -- 172.21.15.87:0/2468935937 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0052d0 con 0x7f71400ae020 2024-04-25T10:18:23.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f713d600640 1 -- 172.21.15.87:0/2468935937 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c005470 con 0x7f71400ae020 2024-04-25T10:18:23.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7140154020 con 0x7f71400ae020 2024-04-25T10:18:23.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f713d600640 1 -- 172.21.15.87:0/2468935937 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c007260 con 0x7f71400ae020 2024-04-25T10:18:23.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7140152ff0 con 0x7f71400ae020 2024-04-25T10:18:23.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714004ef30 con 0x7f71400ae020 2024-04-25T10:18:23.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.529+0000 7f713d600640 1 -- 172.21.15.87:0/2468935937 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f712c007400 con 0x7f71400ae020 2024-04-25T10:18:23.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.533+0000 7f713d600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7120066f60 0x7f7120069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.533+0000 7f713d600640 1 -- 172.21.15.87:0/2468935937 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f712c084580 con 0x7f71400ae020 2024-04-25T10:18:23.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.533+0000 7f7145c00640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7120066f60 0x7f7120069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.533+0000 7f7145c00640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7120066f60 0x7f7120069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71340ae000 tx=0x7f71340a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.537+0000 7f713d600640 1 -- 172.21.15.87:0/2468935937 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f712c054dc0 con 0x7f71400ae020 2024-04-25T10:18:23.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.689+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71401541f0 con 0x7f71400ae020 2024-04-25T10:18:23.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.689+0000 7f713d600640 1 -- 172.21.15.87:0/2468935937 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16819 (secure 0 0 0) 0x7f712c058e60 con 0x7f71400ae020 2024-04-25T10:18:23.693 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:23.693 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":76,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:17:57.908660+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:47.854894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:48.494317+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:23.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7120066f60 msgr2=0x7f7120069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7120066f60 0x7f7120069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71340ae000 tx=0x7f71340a7070 comp rx=0 tx=0).stop 2024-04-25T10:18:23.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400ae020 msgr2=0x7f7140160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400ae020 0x7f7140160330 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c420 tx=0x7f712c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 shutdown_connections 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7120066f60 0x7f7120069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71400cf930 0x7f7140160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71400ae020 0x7f7140160330 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 --2- 172.21.15.87:0/2468935937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7140070c20 0x7f7140152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 >> 172.21.15.87:0/2468935937 conn(0x7f7140066b70 msgr2=0x7f7140161500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 shutdown_connections 2024-04-25T10:18:23.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.693+0000 7f7148600640 1 -- 172.21.15.87:0/2468935937 wait complete. 2024-04-25T10:18:23.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:47.854894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:48.494317+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:23.715 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3373a00640 1 Processor -- start 2024-04-25T10:18:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3373a00640 1 -- start start 2024-04-25T10:18:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3373a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f336c0aef80 0x7f336c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3373a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3373a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c15b080 con 0x7f336c15bac0 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3373a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c15adb0 con 0x7f336c0aef80 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3373a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c15eb60 con 0x7f336c070350 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.849+0000 7f3371000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f336c0aef80 0x7f336c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f3371000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f336c0aef80 0x7f336c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54662/0 (socket says 172.21.15.87:54662) 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f3371000640 1 -- 172.21.15.87:0/2797397147 learned_addr learned my addr 172.21.15.87:0/2797397147 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f336be00640 1 --2- 172.21.15.87:0/2797397147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f3371000640 1 -- 172.21.15.87:0/2797397147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 msgr2=0x7f336c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:23.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f3371000640 1 --2- 172.21.15.87:0/2797397147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f3371000640 1 -- 172.21.15.87:0/2797397147 >> v1:172.21.15.169:6789/0 conn(0x7f336c15bac0 legacy=0x7f336c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f3371000640 1 -- 172.21.15.87:0/2797397147 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c15ee40 con 0x7f336c0aef80 2024-04-25T10:18:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f3371000640 1 --2- 172.21.15.87:0/2797397147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c15e480 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f335c009e50 tx=0x7f335c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664abcc850369afb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f336b400640 1 -- 172.21.15.87:0/2797397147 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c0a58e0 con 0x7f336c0aef80 2024-04-25T10:18:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f336b400640 1 -- 172.21.15.87:0/2797397147 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c0a5a80 con 0x7f336c0aef80 2024-04-25T10:18:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.853+0000 7f336b400640 1 -- 172.21.15.87:0/2797397147 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c0a5e50 con 0x7f336c0aef80 2024-04-25T10:18:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- 172.21.15.87:0/2797397147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 msgr2=0x7f336c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 --2- 172.21.15.87:0/2797397147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c15e480 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f335c009e50 tx=0x7f335c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:18:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- 172.21.15.87:0/2797397147 shutdown_connections 2024-04-25T10:18:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 --2- 172.21.15.87:0/2797397147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 --2- 172.21.15.87:0/2797397147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c15e480 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- 172.21.15.87:0/2797397147 >> 172.21.15.87:0/2797397147 conn(0x7f336c0669a0 msgr2=0x7f336c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- 172.21.15.87:0/2797397147 shutdown_connections 2024-04-25T10:18:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- 172.21.15.87:0/2797397147 wait complete. 2024-04-25T10:18:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 Processor -- start 2024-04-25T10:18:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- start start 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f336c15bac0 0x7f336c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c1679a0 con 0x7f336c0aef80 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c167820 con 0x7f336c15bac0 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3373a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c167b20 con 0x7f336c070350 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54672/0 (socket says 172.21.15.87:54672) 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 -- 172.21.15.87:0/1523998851 learned_addr learned my addr 172.21.15.87:0/1523998851 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3371000640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3371a00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f336c15bac0 0x7f336c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 -- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 msgr2=0x7f336c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 -- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f336c15bac0 msgr2=0x7f336c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f336c15bac0 0x7f336c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f336be00640 1 -- 172.21.15.87:0/1523998851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c1580b0 con 0x7f336c0aef80 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.857+0000 7f3371000640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f336be00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c0ad840 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f335800e930 tx=0x7f335800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3368c00640 1 -- 172.21.15.87:0/1523998851 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358017070 con 0x7f336c0aef80 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3368c00640 1 -- 172.21.15.87:0/1523998851 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335800c9a0 con 0x7f336c0aef80 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c158390 con 0x7f336c0aef80 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3368c00640 1 -- 172.21.15.87:0/1523998851 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800cd50 con 0x7f336c0aef80 2024-04-25T10:18:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c158950 con 0x7f336c0aef80 2024-04-25T10:18:23.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336c04ef30 con 0x7f336c0aef80 2024-04-25T10:18:23.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3368c00640 1 -- 172.21.15.87:0/1523998851 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f335801a070 con 0x7f336c0aef80 2024-04-25T10:18:23.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3368c00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f334c066f60 0x7f334c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3368c00640 1 -- 172.21.15.87:0/1523998851 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f33580854c0 con 0x7f336c0aef80 2024-04-25T10:18:23.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.861+0000 7f3371000640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f334c066f60 0x7f334c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.865+0000 7f3371000640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f334c066f60 0x7f334c069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f335c0ae000 tx=0x7f335c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:23.865+0000 7f3368c00640 1 -- 172.21.15.87:0/1523998851 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3358056490 con 0x7f336c0aef80 2024-04-25T10:18:24.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f336c0b2fa0 con 0x7f336c0aef80 2024-04-25T10:18:24.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3368c00640 1 -- 172.21.15.87:0/1523998851 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3358055d90 con 0x7f336c0aef80 2024-04-25T10:18:24.027 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f334c066f60 msgr2=0x7f334c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f334c066f60 0x7f334c069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f335c0ae000 tx=0x7f335c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 msgr2=0x7f336c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c0ad840 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f335800e930 tx=0x7f335800edf0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 shutdown_connections 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f334c066f60 0x7f334c069420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f336c15bac0 0x7f336c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f336c0aef80 0x7f336c0ad840 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 --2- 172.21.15.87:0/1523998851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f336c070350 0x7f336c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 >> 172.21.15.87:0/1523998851 conn(0x7f336c0669a0 msgr2=0x7f336c0cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 shutdown_connections 2024-04-25T10:18:24.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.025+0000 7f3373a00640 1 -- 172.21.15.87:0/1523998851 wait complete. 2024-04-25T10:18:24.048 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:24.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd8800640 1 Processor -- start 2024-04-25T10:18:24.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd8800640 1 -- start start 2024-04-25T10:18:24.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd8800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7febd00d0d00 0x7febd00ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd8800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7febd006cf40 0x7febd006d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd8800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd00cdab0 con 0x7febd00cc200 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd8800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd006d8b0 con 0x7febd006cf40 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd8800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd00cd210 con 0x7febd00d0d00 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7febd006cf40 0x7febd006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd5400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7febd00cc200 0x7febd0162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35706/0 (socket says 172.21.15.87:35706) 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd5400640 1 -- 172.21.15.87:0/2212001669 learned_addr learned my addr 172.21.15.87:0/2212001669 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd5e00640 1 --2- 172.21.15.87:0/2212001669 >> v2:172.21.15.169:3300/0 conn(0x7febd00d0d00 0x7febd00ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd6800640 1 -- 172.21.15.87:0/2212001669 >> v2:172.21.15.169:3300/0 conn(0x7febd00d0d00 msgr2=0x7febd00ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd6800640 1 --2- 172.21.15.87:0/2212001669 >> v2:172.21.15.169:3300/0 conn(0x7febd00d0d00 0x7febd00ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd6800640 1 -- 172.21.15.87:0/2212001669 >> v1:172.21.15.87:6789/0 conn(0x7febd00cc200 legacy=0x7febd0162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd6800640 1 -- 172.21.15.87:0/2212001669 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00cd4f0 con 0x7febd006cf40 2024-04-25T10:18:24.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.185+0000 7febd4a00640 1 -- 172.21.15.87:0/2212001669 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222876920 0 0) 0x7febd00cdab0 con 0x7febd00cc200 2024-04-25T10:18:24.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd6800640 1 --2- 172.21.15.87:0/2212001669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd006cf40 0x7febd006d370 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7febd00cdd40 tx=0x7febcc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe75714ea6f204fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd4a00640 1 -- 172.21.15.87:0/2212001669 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc0b2070 con 0x7febd006cf40 2024-04-25T10:18:24.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd4a00640 1 -- 172.21.15.87:0/2212001669 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc0a5850 con 0x7febd006cf40 2024-04-25T10:18:24.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd4a00640 1 -- 172.21.15.87:0/2212001669 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc0a5c20 con 0x7febd006cf40 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 -- 172.21.15.87:0/2212001669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd006cf40 msgr2=0x7febd006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 --2- 172.21.15.87:0/2212001669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd006cf40 0x7febd006d370 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7febd00cdd40 tx=0x7febcc01c8a0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 -- 172.21.15.87:0/2212001669 shutdown_connections 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 --2- 172.21.15.87:0/2212001669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd006cf40 0x7febd006d370 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 --2- 172.21.15.87:0/2212001669 >> v2:172.21.15.169:3300/0 conn(0x7febd00d0d00 0x7febd00ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 -- 172.21.15.87:0/2212001669 >> 172.21.15.87:0/2212001669 conn(0x7febd0072290 msgr2=0x7febd00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 -- 172.21.15.87:0/2212001669 shutdown_connections 2024-04-25T10:18:24.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 -- 172.21.15.87:0/2212001669 wait complete. 2024-04-25T10:18:24.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 Processor -- start 2024-04-25T10:18:24.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.189+0000 7febd8800640 1 -- start start 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febd006cf40 0x7febd009dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd00cc200 0x7febd00ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febd00d0d00 0x7febd0159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd0162a10 con 0x7febd00cc200 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd0162890 con 0x7febd00d0d00 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd015e040 con 0x7febd006cf40 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd00cc200 0x7febd00ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd00cc200 0x7febd00ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54692/0 (socket says 172.21.15.87:54692) 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 -- 172.21.15.87:0/1146499176 learned_addr learned my addr 172.21.15.87:0/1146499176 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd6800640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febd00d0d00 0x7febd0159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5e00640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febd006cf40 0x7febd009dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 -- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febd006cf40 msgr2=0x7febd009dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febd006cf40 0x7febd009dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 -- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febd00d0d00 msgr2=0x7febd0159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febd00d0d00 0x7febd0159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 -- 172.21.15.87:0/1146499176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00c66c0 con 0x7febd00cc200 2024-04-25T10:18:24.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5e00640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febd006cf40 0x7febd009dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:24.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd5400640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd00cc200 0x7febd00ca020 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7febc000c4a0 tx=0x7febc000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febca000640 1 -- 172.21.15.87:0/1146499176 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0018070 con 0x7febd00cc200 2024-04-25T10:18:24.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febca000640 1 -- 172.21.15.87:0/1146499176 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc00074e0 con 0x7febd00cc200 2024-04-25T10:18:24.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd009e1b0 con 0x7febd00cc200 2024-04-25T10:18:24.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febca000640 1 -- 172.21.15.87:0/1146499176 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0004a30 con 0x7febd00cc200 2024-04-25T10:18:24.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd0157990 con 0x7febd00cc200 2024-04-25T10:18:24.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd004ef30 con 0x7febd00cc200 2024-04-25T10:18:24.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.193+0000 7febca000640 1 -- 172.21.15.87:0/1146499176 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febc001b070 con 0x7febd00cc200 2024-04-25T10:18:24.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.197+0000 7febca000640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febb4066f90 0x7febb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.197+0000 7febca000640 1 -- 172.21.15.87:0/1146499176 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7febc0084330 con 0x7febd00cc200 2024-04-25T10:18:24.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.197+0000 7febd5e00640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febb4066f90 0x7febb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.197+0000 7febd5e00640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febb4066f90 0x7febb4069450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7febc40097a0 tx=0x7febc4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.201+0000 7febca000640 1 -- 172.21.15.87:0/1146499176 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7febd00c66c0 con 0x7febd00cc200 2024-04-25T10:18:24.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.353+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7febd0072ff0 con 0x7febd00cc200 2024-04-25T10:18:24.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febca000640 1 -- 172.21.15.87:0/1146499176 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7febc00552d0 con 0x7febd00cc200 2024-04-25T10:18:24.358 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febb4066f90 msgr2=0x7febb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febb4066f90 0x7febb4069450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7febc40097a0 tx=0x7febc4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd00cc200 msgr2=0x7febd00ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd00cc200 0x7febd00ca020 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7febc000c4a0 tx=0x7febc000c960 comp rx=0 tx=0).stop 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 shutdown_connections 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7febb4066f90 0x7febb4069450 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febd00d0d00 0x7febd0159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febd00cc200 0x7febd00ca020 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 --2- 172.21.15.87:0/1146499176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febd006cf40 0x7febd009dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 >> 172.21.15.87:0/1146499176 conn(0x7febd0072290 msgr2=0x7febd015d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 shutdown_connections 2024-04-25T10:18:24.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.357+0000 7febd8800640 1 -- 172.21.15.87:0/1146499176 wait complete. 2024-04-25T10:18:24.378 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:24.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3d1c00640 1 Processor -- start 2024-04-25T10:18:24.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3d1c00640 1 -- start start 2024-04-25T10:18:24.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3d1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3d1c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3cc072020 0x7fb3cc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3d1c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc159920 con 0x7fb3cc159540 2024-04-25T10:18:24.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3d1c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc15c2f0 con 0x7fb3cc072020 2024-04-25T10:18:24.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3d1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15c5d0 con 0x7fb3cc0aeda0 2024-04-25T10:18:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3ca000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3ca000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:34124/0 (socket says 172.21.15.87:34124) 2024-04-25T10:18:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3ca000640 1 -- 172.21.15.87:0/2485277998 learned_addr learned my addr 172.21.15.87:0/2485277998 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.513+0000 7fb3cb400640 1 --2- 172.21.15.87:0/2485277998 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc072020 0x7fb3cc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3c9600640 1 -- 172.21.15.87:0/2485277998 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753145909 0 0) 0x7fb3cc159920 con 0x7fb3cc159540 2024-04-25T10:18:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3c9600640 1 -- 172.21.15.87:0/2485277998 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b0003530 con 0x7fb3cc159540 2024-04-25T10:18:24.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3ca000640 1 -- 172.21.15.87:0/2485277998 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc072020 msgr2=0x7fb3cc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3ca000640 1 --2- 172.21.15.87:0/2485277998 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc072020 0x7fb3cc070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3ca000640 1 -- 172.21.15.87:0/2485277998 >> v1:172.21.15.169:6789/0 conn(0x7fb3cc159540 legacy=0x7fb3cc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3ca000640 1 -- 172.21.15.87:0/2485277998 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc15c8b0 con 0x7fb3cc0aeda0 2024-04-25T10:18:24.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3cb400640 1 --2- 172.21.15.87:0/2485277998 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc072020 0x7fb3cc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:24.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3ca000640 1 --2- 172.21.15.87:0/2485277998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc071ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009870 tx=0x7fb3b801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1c77c49b86eb0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3c9600640 1 -- 172.21.15.87:0/2485277998 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a9610 con 0x7fb3cc0aeda0 2024-04-25T10:18:24.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3c9600640 1 -- 172.21.15.87:0/2485277998 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80a97b0 con 0x7fb3cc0aeda0 2024-04-25T10:18:24.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.517+0000 7fb3c9600640 1 -- 172.21.15.87:0/2485277998 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a9b60 con 0x7fb3cc0aeda0 2024-04-25T10:18:24.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2485277998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 msgr2=0x7fb3cc071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2485277998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc071ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009870 tx=0x7fb3b801c530 comp rx=0 tx=0).stop 2024-04-25T10:18:24.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2485277998 shutdown_connections 2024-04-25T10:18:24.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2485277998 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc072020 0x7fb3cc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2485277998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc071ae0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2485277998 >> 172.21.15.87:0/2485277998 conn(0x7fb3cc0669a0 msgr2=0x7fb3cc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2485277998 shutdown_connections 2024-04-25T10:18:24.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2485277998 wait complete. 2024-04-25T10:18:24.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 Processor -- start 2024-04-25T10:18:24.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- start start 2024-04-25T10:18:24.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc072020 0x7fb3cc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc159540 0x7fb3cc15e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0710a0 con 0x7fb3cc159540 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc072430 con 0x7fb3cc072020 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3d1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc071220 con 0x7fb3cc0aeda0 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc159540 0x7fb3cc15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc159540 0x7fb3cc15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54702/0 (socket says 172.21.15.87:54702) 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 -- 172.21.15.87:0/2863308657 learned_addr learned my addr 172.21.15.87:0/2863308657 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3ca000640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3caa00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc072020 0x7fb3cc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 -- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 msgr2=0x7fb3cc1f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc1f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 -- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc072020 msgr2=0x7fb3cc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc072020 0x7fb3cc153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.521+0000 7fb3cb400640 1 -- 172.21.15.87:0/2863308657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc150c80 con 0x7fb3cc159540 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3cb400640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc159540 0x7fb3cc15e080 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e950 tx=0x7fb3c400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3c2a00640 1 -- 172.21.15.87:0/2863308657 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4017070 con 0x7fb3cc159540 2024-04-25T10:18:24.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3c2a00640 1 -- 172.21.15.87:0/2863308657 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c400ca00 con 0x7fb3cc159540 2024-04-25T10:18:24.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3c2a00640 1 -- 172.21.15.87:0/2863308657 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400cdd0 con 0x7fb3cc159540 2024-04-25T10:18:24.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc154060 con 0x7fb3cc159540 2024-04-25T10:18:24.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc0b0f20 con 0x7fb3cc159540 2024-04-25T10:18:24.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc04ef30 con 0x7fb3cc159540 2024-04-25T10:18:24.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3c2a00640 1 -- 172.21.15.87:0/2863308657 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3c4013470 con 0x7fb3cc159540 2024-04-25T10:18:24.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3c2a00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f90 0x7fb3ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3c2a00640 1 -- 172.21.15.87:0/2863308657 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb3c4084ba0 con 0x7fb3cc159540 2024-04-25T10:18:24.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.525+0000 7fb3caa00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f90 0x7fb3ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.529+0000 7fb3caa00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f90 0x7fb3ac069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc007fd0 tx=0x7fb3bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.529+0000 7fb3c2a00640 1 -- 172.21.15.87:0/2863308657 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3cc150c80 con 0x7fb3cc159540 2024-04-25T10:18:24.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3cc06d7f0 con 0x7fb3cc159540 2024-04-25T10:18:24.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3c2a00640 1 -- 172.21.15.87:0/2863308657 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3c4055b40 con 0x7fb3cc159540 2024-04-25T10:18:24.687 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f90 msgr2=0x7fb3ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f90 0x7fb3ac069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc007fd0 tx=0x7fb3bc009040 comp rx=0 tx=0).stop 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc159540 msgr2=0x7fb3cc15e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc159540 0x7fb3cc15e080 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e950 tx=0x7fb3c400ee10 comp rx=0 tx=0).stop 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 shutdown_connections 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f90 0x7fb3ac069450 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc159540 0x7fb3cc15e080 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc0aeda0 0x7fb3cc1f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 --2- 172.21.15.87:0/2863308657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc072020 0x7fb3cc153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 >> 172.21.15.87:0/2863308657 conn(0x7fb3cc0669a0 msgr2=0x7fb3cc06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 shutdown_connections 2024-04-25T10:18:24.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.685+0000 7fb3d1c00640 1 -- 172.21.15.87:0/2863308657 wait complete. 2024-04-25T10:18:24.707 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:24.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf36600640 1 Processor -- start 2024-04-25T10:18:24.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf36600640 1 -- start start 2024-04-25T10:18:24.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf36600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf3015bc40 0x7faf3006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf36600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf300b2770 con 0x7faf300cda10 2024-04-25T10:18:24.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf36600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3006c670 con 0x7faf300cf930 2024-04-25T10:18:24.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf36600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3006c950 con 0x7faf3015bc40 2024-04-25T10:18:24.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7faf300cda10 0x7faf300b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35708/0 (socket says 172.21.15.87:35708) 2024-04-25T10:18:24.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 -- 172.21.15.87:0/3867752423 learned_addr learned my addr 172.21.15.87:0/3867752423 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:24.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2f400640 1 --2- 172.21.15.87:0/3867752423 >> v2:172.21.15.169:3300/0 conn(0x7faf3015bc40 0x7faf3006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538475331 0 0) 0x7faf300b2770 con 0x7faf300cda10 2024-04-25T10:18:24.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf14003530 con 0x7faf300cda10 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044315305 0 0) 0x7faf14003530 con 0x7faf300cda10 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 >> v2:172.21.15.169:3300/0 conn(0x7faf3015bc40 msgr2=0x7faf3006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 --2- 172.21.15.87:0/3867752423 >> v2:172.21.15.169:3300/0 conn(0x7faf3015bc40 0x7faf3006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 >> v1:172.21.15.169:6789/0 conn(0x7faf300cf930 legacy=0x7faf300b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3006cc30 con 0x7faf300cda10 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7faf1c001680 con 0x7faf300cda10 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 >> v1:172.21.15.87:6789/0 conn(0x7faf300cda10 legacy=0x7faf300b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14003a10 0x7faf14003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf14005220 0x7faf1400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3006c950 con 0x7faf14004810 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1400c170 con 0x7faf14003a10 2024-04-25T10:18:24.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1400c370 con 0x7faf14005220 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2f400640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14003a10 0x7faf14003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2fe00640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf14005220 0x7faf1400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 -- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf14005220 msgr2=0x7faf1400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf14005220 0x7faf1400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 -- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14003a10 msgr2=0x7faf14003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14003a10 0x7faf14003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2ea00640 1 -- 172.21.15.87:0/3867752423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf14003530 con 0x7faf14004810 2024-04-25T10:18:24.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.841+0000 7faf2fe00640 1 -- 172.21.15.87:0/3867752423 reap_dead start 2024-04-25T10:18:24.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf2ea00640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf14004c90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c0e0 tx=0x7faf1c01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ecfa683aa44443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c007c50 con 0x7faf14004810 2024-04-25T10:18:24.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c0a9070 con 0x7faf14004810 2024-04-25T10:18:24.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf2e000640 1 -- 172.21.15.87:0/3867752423 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c005d80 con 0x7faf14004810 2024-04-25T10:18:24.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf36600640 1 -- 172.21.15.87:0/3867752423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 msgr2=0x7faf14004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf36600640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf14004c90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c0e0 tx=0x7faf1c01cb80 comp rx=0 tx=0).stop 2024-04-25T10:18:24.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf36600640 1 -- 172.21.15.87:0/3867752423 shutdown_connections 2024-04-25T10:18:24.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf36600640 1 --2- 172.21.15.87:0/3867752423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf14004c90 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf36600640 1 -- 172.21.15.87:0/3867752423 >> 172.21.15.87:0/3867752423 conn(0x7faf30066b70 msgr2=0x7faf30066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf36600640 1 -- 172.21.15.87:0/3867752423 shutdown_connections 2024-04-25T10:18:24.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.845+0000 7faf36600640 1 -- 172.21.15.87:0/3867752423 wait complete. 2024-04-25T10:18:24.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 Processor -- start 2024-04-25T10:18:24.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 -- start start 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf301fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf30070c30 0x7faf30153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf3015c370 0x7faf3015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300d00d0 con 0x7faf14004810 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300cff50 con 0x7faf30070c30 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf300d0250 con 0x7faf3015c370 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf301fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf301fafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54722/0 (socket says 172.21.15.87:54722) 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 -- 172.21.15.87:0/868496494 learned_addr learned my addr 172.21.15.87:0/868496494 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2ea00640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf30070c30 0x7faf30153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2fe00640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf3015c370 0x7faf3015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 -- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf3015c370 msgr2=0x7faf3015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf3015c370 0x7faf3015c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 -- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf30070c30 msgr2=0x7faf30153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf30070c30 0x7faf30153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 -- 172.21.15.87:0/868496494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf301fbe40 con 0x7faf14004810 2024-04-25T10:18:24.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf2f400640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf301fafc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faf20011aa0 tx=0x7faf20011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf27400640 1 -- 172.21.15.87:0/868496494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000fb80 con 0x7faf14004810 2024-04-25T10:18:24.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf27400640 1 -- 172.21.15.87:0/868496494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2000fd20 con 0x7faf14004810 2024-04-25T10:18:24.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf27400640 1 -- 172.21.15.87:0/868496494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20014680 con 0x7faf14004810 2024-04-25T10:18:24.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf301fb5a0 con 0x7faf14004810 2024-04-25T10:18:24.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf301fbb60 con 0x7faf14004810 2024-04-25T10:18:24.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.849+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3004ef30 con 0x7faf14004810 2024-04-25T10:18:24.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.853+0000 7faf27400640 1 -- 172.21.15.87:0/868496494 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf20012040 con 0x7faf14004810 2024-04-25T10:18:24.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.853+0000 7faf27400640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf10066f60 0x7faf10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.853+0000 7faf27400640 1 -- 172.21.15.87:0/868496494 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faf200858c0 con 0x7faf14004810 2024-04-25T10:18:24.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.853+0000 7faf2ea00640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf10066f60 0x7faf10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.853+0000 7faf2ea00640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf10066f60 0x7faf10069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0ab000 tx=0x7faf1c0b0040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:24.857+0000 7faf27400640 1 -- 172.21.15.87:0/868496494 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf20019030 con 0x7faf14004810 2024-04-25T10:18:25.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf300b2770 con 0x7faf14004810 2024-04-25T10:18:25.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf27400640 1 -- 172.21.15.87:0/868496494 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4937 (secure 0 0 0) 0x7faf200567d0 con 0x7faf14004810 2024-04-25T10:18:25.047 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:25.047 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/817977905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":817977905},{"type":"v1","addr":"172.21.15.169:6835","nonce":817977905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/979883306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":979883306},{"type":"v1","addr":"172.21.15.87:6836","nonce":979883306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/796485952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":796485952},{"type":"v1","addr":"172.21.15.87:6837","nonce":796485952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:16:49.500890+0000","modified":"2024-04-25T10:16:57.376333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"d","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1643726385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1643726385},{"type":"v1","addr":"172.21.15.169:6834","nonce":1643726385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:18:25.047 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 54 2024-04-25T10:18:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf10066f60 msgr2=0x7faf10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf10066f60 0x7faf10069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0ab000 tx=0x7faf1c0b0040 comp rx=0 tx=0).stop 2024-04-25T10:18:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 msgr2=0x7faf301fafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf301fafc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faf20011aa0 tx=0x7faf20011f60 comp rx=0 tx=0).stop 2024-04-25T10:18:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 shutdown_connections 2024-04-25T10:18:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf10066f60 0x7faf10069420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf3015c370 0x7faf3015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf30070c30 0x7faf30153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 --2- 172.21.15.87:0/868496494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf14004810 0x7faf301fafc0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 >> 172.21.15.87:0/868496494 conn(0x7faf30066b70 msgr2=0x7faf30155330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 shutdown_connections 2024-04-25T10:18:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.045+0000 7faf36600640 1 -- 172.21.15.87:0/868496494 wait complete. 2024-04-25T10:18:25.069 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:25.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc73600640 1 Processor -- start 2024-04-25T10:18:25.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc73600640 1 -- start start 2024-04-25T10:18:25.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc73600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc6c15ddc0 0x7fcc6c06b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc73600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c0d2460 0x7fcc6c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc73600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c0cc060 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc73600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6c0cf2f0 con 0x7fcc6c15ddc0 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc73600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c06af30 con 0x7fcc6c0d2460 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc71600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c0d2460 0x7fcc6c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc6be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcc6c15e1a0 0x7fcc6c0cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:38328/0 (socket says 172.21.15.87:38328) 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc6be00640 1 -- 172.21.15.87:0/650190067 learned_addr learned my addr 172.21.15.87:0/650190067 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc71600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c0d2460 0x7fcc6c0d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:34172/0 (socket says 172.21.15.87:34172) 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc70c00640 1 --2- 172.21.15.87:0/650190067 >> v2:172.21.15.169:3300/0 conn(0x7fcc6c15ddc0 0x7fcc6c06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc6b400640 1 -- 172.21.15.87:0/650190067 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608666622 0 0) 0x7fcc6c0cc060 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc71600640 1 -- 172.21.15.87:0/650190067 >> v2:172.21.15.169:3300/0 conn(0x7fcc6c15ddc0 msgr2=0x7fcc6c06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc71600640 1 --2- 172.21.15.87:0/650190067 >> v2:172.21.15.169:3300/0 conn(0x7fcc6c15ddc0 0x7fcc6c06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc71600640 1 -- 172.21.15.87:0/650190067 >> v1:172.21.15.169:6789/0 conn(0x7fcc6c15e1a0 legacy=0x7fcc6c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.205+0000 7fcc71600640 1 -- 172.21.15.87:0/650190067 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0cc340 con 0x7fcc6c0d2460 2024-04-25T10:18:25.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc71600640 1 --2- 172.21.15.87:0/650190067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c0d2460 0x7fcc6c0d0cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc6401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce20a0064767b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc6b400640 1 -- 172.21.15.87:0/650190067 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640b2070 con 0x7fcc6c0d2460 2024-04-25T10:18:25.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc6b400640 1 -- 172.21.15.87:0/650190067 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc640a5830 con 0x7fcc6c0d2460 2024-04-25T10:18:25.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc6b400640 1 -- 172.21.15.87:0/650190067 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640a5c00 con 0x7fcc6c0d2460 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 -- 172.21.15.87:0/650190067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c0d2460 msgr2=0x7fcc6c0d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 --2- 172.21.15.87:0/650190067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c0d2460 0x7fcc6c0d0cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc6401c970 comp rx=0 tx=0).stop 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 -- 172.21.15.87:0/650190067 shutdown_connections 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 --2- 172.21.15.87:0/650190067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c0d2460 0x7fcc6c0d0cf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 --2- 172.21.15.87:0/650190067 >> v2:172.21.15.169:3300/0 conn(0x7fcc6c15ddc0 0x7fcc6c06b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 -- 172.21.15.87:0/650190067 >> 172.21.15.87:0/650190067 conn(0x7fcc6c0d1970 msgr2=0x7fcc6c0d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 -- 172.21.15.87:0/650190067 shutdown_connections 2024-04-25T10:18:25.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.209+0000 7fcc73600640 1 -- 172.21.15.87:0/650190067 wait complete. 2024-04-25T10:18:25.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 Processor -- start 2024-04-25T10:18:25.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 -- start start 2024-04-25T10:18:25.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc6c0d2460 0x7fcc6c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c15ddc0 0x7fcc6c1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc6c15e1a0 0x7fcc6c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c16ac90 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c16ab10 con 0x7fcc6c0d2460 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c16ae10 con 0x7fcc6c15ddc0 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc6c15e1a0 0x7fcc6c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c15ddc0 0x7fcc6c1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc6c15e1a0 0x7fcc6c0b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54732/0 (socket says 172.21.15.87:54732) 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 -- 172.21.15.87:0/267548602 learned_addr learned my addr 172.21.15.87:0/267548602 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc70c00640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc6c0d2460 0x7fcc6c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 -- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c15ddc0 msgr2=0x7fcc6c1533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c15ddc0 0x7fcc6c1533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 -- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc6c0d2460 msgr2=0x7fcc6c15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc6c0d2460 0x7fcc6c15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 -- 172.21.15.87:0/267548602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c15c1e0 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc6be00640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c15ddc0 0x7fcc6c1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:25.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc71600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc6c15e1a0 0x7fcc6c0b5a50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc64019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc68c00640 1 -- 172.21.15.87:0/267548602 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640b2050 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc68c00640 1 -- 172.21.15.87:0/267548602 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc640be070 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc68c00640 1 -- 172.21.15.87:0/267548602 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640ad870 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c15c4c0 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.213+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c15ca80 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.217+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c06ce20 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.217+0000 7fcc68c00640 1 -- 172.21.15.87:0/267548602 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc640ada10 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.217+0000 7fcc68c00640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc4c066f60 0x7fcc4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.217+0000 7fcc68c00640 1 -- 172.21.15.87:0/267548602 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcc64120140 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.217+0000 7fcc70c00640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc4c066f60 0x7fcc4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.217+0000 7fcc70c00640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc4c066f60 0x7fcc4c069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0b77a0 tx=0x7fcc60009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.221+0000 7fcc68c00640 1 -- 172.21.15.87:0/267548602 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc64016560 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.377+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc6c06d4d0 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.377+0000 7fcc68c00640 1 -- 172.21.15.87:0/267548602 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16819 (secure 0 0 0) 0x7fcc640add00 con 0x7fcc6c15e1a0 2024-04-25T10:18:25.380 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:25.380 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":76,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:17:57.908660+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:47.854894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:48.494317+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc4c066f60 msgr2=0x7fcc4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc4c066f60 0x7fcc4c069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0b77a0 tx=0x7fcc60009190 comp rx=0 tx=0).stop 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc6c15e1a0 msgr2=0x7fcc6c0b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc6c15e1a0 0x7fcc6c0b5a50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc64019710 comp rx=0 tx=0).stop 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 shutdown_connections 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc4c066f60 0x7fcc4c069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc6c15e1a0 0x7fcc6c0b5a50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc6c15ddc0 0x7fcc6c1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 --2- 172.21.15.87:0/267548602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc6c0d2460 0x7fcc6c15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 >> 172.21.15.87:0/267548602 conn(0x7fcc6c0d1970 msgr2=0x7fcc6c0d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 shutdown_connections 2024-04-25T10:18:25.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.381+0000 7fcc73600640 1 -- 172.21.15.87:0/267548602 wait complete. 2024-04-25T10:18:25.402 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:18:25.402 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.533+0000 7f193be00640 1 Processor -- start 2024-04-25T10:18:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f193be00640 1 -- start start 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f193be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f193415e050 0x7f19340b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f193be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19340b13e0 0x7f19340cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f193be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19340b1af0 con 0x7f19340d1820 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f193be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19340b1dd0 con 0x7f19340b13e0 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f193be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19340b20b0 con 0x7f193415e050 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1939e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19340b13e0 0x7f19340cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1939e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19340b13e0 0x7f19340cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54740/0 (socket says 172.21.15.87:54740) 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1939e00640 1 -- 172.21.15.87:0/900667158 learned_addr learned my addr 172.21.15.87:0/900667158 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1938a00640 1 --2- 172.21.15.87:0/900667158 >> v2:172.21.15.169:3300/0 conn(0x7f193415e050 0x7f19340b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1939e00640 1 -- 172.21.15.87:0/900667158 >> v2:172.21.15.169:3300/0 conn(0x7f193415e050 msgr2=0x7f19340b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1939e00640 1 --2- 172.21.15.87:0/900667158 >> v2:172.21.15.169:3300/0 conn(0x7f193415e050 0x7f19340b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1939e00640 1 -- 172.21.15.87:0/900667158 >> v1:172.21.15.87:6789/0 conn(0x7f19340d1820 legacy=0x7f19340d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1939e00640 1 -- 172.21.15.87:0/900667158 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19340b2390 con 0x7f19340b13e0 2024-04-25T10:18:25.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1938a00640 1 --2- 172.21.15.87:0/900667158 >> v2:172.21.15.169:3300/0 conn(0x7f193415e050 0x7f19340b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:25.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.537+0000 7f1933e00640 1 -- 172.21.15.87:0/900667158 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009329351 0 0) 0x7f19340b1af0 con 0x7f19340d1820 2024-04-25T10:18:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f1939e00640 1 --2- 172.21.15.87:0/900667158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19340b13e0 0x7f19340cfc70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f19340c5400 tx=0x7f192c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8602a9061af977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f1933e00640 1 -- 172.21.15.87:0/900667158 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0a7070 con 0x7f19340b13e0 2024-04-25T10:18:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f1933e00640 1 -- 172.21.15.87:0/900667158 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c0b0070 con 0x7f19340b13e0 2024-04-25T10:18:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f1933e00640 1 -- 172.21.15.87:0/900667158 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0ab890 con 0x7f19340b13e0 2024-04-25T10:18:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 -- 172.21.15.87:0/900667158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19340b13e0 msgr2=0x7f19340cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 --2- 172.21.15.87:0/900667158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19340b13e0 0x7f19340cfc70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f19340c5400 tx=0x7f192c01c490 comp rx=0 tx=0).stop 2024-04-25T10:18:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 -- 172.21.15.87:0/900667158 shutdown_connections 2024-04-25T10:18:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 --2- 172.21.15.87:0/900667158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19340b13e0 0x7f19340cfc70 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 --2- 172.21.15.87:0/900667158 >> v2:172.21.15.169:3300/0 conn(0x7f193415e050 0x7f19340b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 -- 172.21.15.87:0/900667158 >> 172.21.15.87:0/900667158 conn(0x7f193409a9d0 msgr2=0x7f193409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 -- 172.21.15.87:0/900667158 shutdown_connections 2024-04-25T10:18:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 -- 172.21.15.87:0/900667158 wait complete. 2024-04-25T10:18:25.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 Processor -- start 2024-04-25T10:18:25.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.541+0000 7f193be00640 1 -- start start 2024-04-25T10:18:25.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19340b13e0 0x7f193415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19340d1820 0x7f193415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f193415e050 0x7f1934148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19340d0d90 con 0x7f193415e050 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19340d0c10 con 0x7f19340b13e0 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19340d0f10 con 0x7f19340d1820 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f193415e050 0x7f1934148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f193415e050 0x7f1934148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54746/0 (socket says 172.21.15.87:54746) 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 -- 172.21.15.87:0/1143750632 learned_addr learned my addr 172.21.15.87:0/1143750632 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1938a00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19340d1820 0x7f193415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 -- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19340d1820 msgr2=0x7f193415a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939400640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19340b13e0 0x7f193415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19340d1820 0x7f193415a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 -- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19340b13e0 msgr2=0x7f193415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19340b13e0 0x7f193415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 -- 172.21.15.87:0/1143750632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19341488d0 con 0x7f193415e050 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1938a00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19340d1820 0x7f193415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939400640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19340b13e0 0x7f193415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:25.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1939e00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f193415e050 0x7f1934148010 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f192c0190a0 tx=0x7f192c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1931600640 1 -- 172.21.15.87:0/1143750632 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0ba070 con 0x7f193415e050 2024-04-25T10:18:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1931600640 1 -- 172.21.15.87:0/1143750632 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c0abc30 con 0x7f193415e050 2024-04-25T10:18:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1934148b70 con 0x7f193415e050 2024-04-25T10:18:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1931600640 1 -- 172.21.15.87:0/1143750632 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c004030 con 0x7f193415e050 2024-04-25T10:18:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1934149130 con 0x7f193415e050 2024-04-25T10:18:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193404ef30 con 0x7f193415e050 2024-04-25T10:18:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.545+0000 7f1931600640 1 -- 172.21.15.87:0/1143750632 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f192c0041d0 con 0x7f193415e050 2024-04-25T10:18:25.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.549+0000 7f1931600640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1914066f60 0x7f1914069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.549+0000 7f1931600640 1 -- 172.21.15.87:0/1143750632 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f192c11f2d0 con 0x7f193415e050 2024-04-25T10:18:25.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.549+0000 7f1939400640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1914066f60 0x7f1914069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.549+0000 7f1939400640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1914066f60 0x7f1914069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1928007fd0 tx=0x7f1928009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.553+0000 7f1931600640 1 -- 172.21.15.87:0/1143750632 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f192c016110 con 0x7f193415e050 2024-04-25T10:18:25.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19340cef40 con 0x7f193415e050 2024-04-25T10:18:25.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f1931600640 1 -- 172.21.15.87:0/1143750632 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f192c0c63a0 con 0x7f193415e050 2024-04-25T10:18:25.739 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:25.739 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/817977905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":817977905},{"type":"v1","addr":"172.21.15.169:6835","nonce":817977905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/979883306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":979883306},{"type":"v1","addr":"172.21.15.87:6836","nonce":979883306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/796485952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":796485952},{"type":"v1","addr":"172.21.15.87:6837","nonce":796485952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:16:49.500890+0000","modified":"2024-04-25T10:16:57.376333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"d","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1643726385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1643726385},{"type":"v1","addr":"172.21.15.169:6834","nonce":1643726385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:18:25.739 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 54 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1914066f60 msgr2=0x7f1914069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1914066f60 0x7f1914069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1928007fd0 tx=0x7f1928009040 comp rx=0 tx=0).stop 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f193415e050 msgr2=0x7f1934148010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f193415e050 0x7f1934148010 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f192c0190a0 tx=0x7f192c01acd0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 shutdown_connections 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1914066f60 0x7f1914069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f193415e050 0x7f1934148010 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19340d1820 0x7f193415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 --2- 172.21.15.87:0/1143750632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19340b13e0 0x7f193415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 >> 172.21.15.87:0/1143750632 conn(0x7f193409a9d0 msgr2=0x7f19340b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 shutdown_connections 2024-04-25T10:18:25.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.737+0000 7f193be00640 1 -- 172.21.15.87:0/1143750632 wait complete. 2024-04-25T10:18:25.760 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:25.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3c200640 1 Processor -- start 2024-04-25T10:18:25.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3c200640 1 -- start start 2024-04-25T10:18:25.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3c200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 0x7fbd34166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3c200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd340ccfb0 con 0x7fbd3415b9e0 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3c200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd340cd290 con 0x7fbd340ccb20 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3c200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340cd570 con 0x7fbd340c6a20 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd39800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbd3415b9e0 0x7fbd340cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35732/0 (socket says 172.21.15.87:35732) 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd39800640 1 -- 172.21.15.87:0/4046966023 learned_addr learned my addr 172.21.15.87:0/4046966023 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3a200640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 0x7fbd34166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765318525 0 0) 0x7fbd340ccfb0 con 0x7fbd3415b9e0 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd1c003530 con 0x7fbd3415b9e0 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2415027521 0 0) 0x7fbd1c003530 con 0x7fbd3415b9e0 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 msgr2=0x7fbd34166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 0x7fbd34166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 >> v1:172.21.15.169:6789/0 conn(0x7fbd340ccb20 legacy=0x7fbd340d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd340cd850 con 0x7fbd3415b9e0 2024-04-25T10:18:25.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fbd280016a0 con 0x7fbd3415b9e0 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 >> v1:172.21.15.87:6789/0 conn(0x7fbd3415b9e0 legacy=0x7fbd340cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c0039f0 0x7fbd1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd1c004810 0x7fbd1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 0x7fbd1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340cd570 con 0x7fbd1c004810 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c00bab0 con 0x7fbd1c0039f0 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c00bcb0 con 0x7fbd340c6a20 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd38e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd1c004810 0x7fbd1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd39800640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c0039f0 0x7fbd1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd3a200640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 0x7fbd1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd38e00640 1 -- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 msgr2=0x7fbd1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd38e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd340c6a20 0x7fbd1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd38e00640 1 -- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c0039f0 msgr2=0x7fbd1c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd38e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c0039f0 0x7fbd1c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.893+0000 7fbd38e00640 1 -- 172.21.15.87:0/4046966023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c003530 con 0x7fbd1c004810 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd38e00640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd1c004810 0x7fbd1c004c90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fbd340d2a00 tx=0x7fbd2001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516ac2692a5bd80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200013a0 con 0x7fbd1c004810 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd200052f0 con 0x7fbd1c004810 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd33e00640 1 -- 172.21.15.87:0/4046966023 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20003840 con 0x7fbd1c004810 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd3c200640 1 -- 172.21.15.87:0/4046966023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd1c004810 msgr2=0x7fbd1c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd3c200640 1 --2- 172.21.15.87:0/4046966023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd1c004810 0x7fbd1c004c90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fbd340d2a00 tx=0x7fbd2001c3a0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd3a200640 1 -- 172.21.15.87:0/4046966023 reap_dead start 2024-04-25T10:18:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd3c200640 1 -- 172.21.15.87:0/4046966023 shutdown_connections 2024-04-25T10:18:25.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd3c200640 1 -- 172.21.15.87:0/4046966023 >> 172.21.15.87:0/4046966023 conn(0x7fbd3409a9d0 msgr2=0x7fbd3409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd3c200640 1 -- 172.21.15.87:0/4046966023 shutdown_connections 2024-04-25T10:18:25.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.897+0000 7fbd3c200640 1 -- 172.21.15.87:0/4046966023 wait complete. 2024-04-25T10:18:25.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 Processor -- start 2024-04-25T10:18:25.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 -- start start 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd1c004810 0x7fbd34156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd340b7120 0x7fbd3415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd3415a870 0x7fbd34157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340d1c30 con 0x7fbd340b7120 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340cf740 con 0x7fbd3415a870 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340d1db0 con 0x7fbd1c004810 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd340b7120 0x7fbd3415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd340b7120 0x7fbd3415a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54766/0 (socket says 172.21.15.87:54766) 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 -- 172.21.15.87:0/879628003 learned_addr learned my addr 172.21.15.87:0/879628003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3a200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd3415a870 0x7fbd34157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd39800640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd1c004810 0x7fbd34156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 -- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd1c004810 msgr2=0x7fbd34156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd1c004810 0x7fbd34156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 -- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd3415a870 msgr2=0x7fbd34157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd3415a870 0x7fbd34157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 -- 172.21.15.87:0/879628003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd340b60c0 con 0x7fbd340b7120 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3a200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd3415a870 0x7fbd34157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd38e00640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd340b7120 0x7fbd3415a330 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbd2001c880 tx=0x7fbd1c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd31600640 1 -- 172.21.15.87:0/879628003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200013a0 con 0x7fbd340b7120 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd31600640 1 -- 172.21.15.87:0/879628003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd200bb070 con 0x7fbd340b7120 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd31600640 1 -- 172.21.15.87:0/879628003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20007a40 con 0x7fbd340b7120 2024-04-25T10:18:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd340b6360 con 0x7fbd340b7120 2024-04-25T10:18:25.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.901+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd340b6920 con 0x7fbd340b7120 2024-04-25T10:18:25.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.905+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3404ef30 con 0x7fbd340b7120 2024-04-25T10:18:25.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.905+0000 7fbd31600640 1 -- 172.21.15.87:0/879628003 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd200b4040 con 0x7fbd340b7120 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.905+0000 7fbd31600640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd14066f90 0x7fbd14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.905+0000 7fbd31600640 1 -- 172.21.15.87:0/879628003 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd2011eff0 con 0x7fbd340b7120 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.905+0000 7fbd39800640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd14066f90 0x7fbd14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.905+0000 7fbd39800640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd14066f90 0x7fbd14069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbd28001060 tx=0x7fbd28009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:25.909+0000 7fbd31600640 1 -- 172.21.15.87:0/879628003 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd200160b0 con 0x7fbd340b7120 2024-04-25T10:18:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd340c7eb0 con 0x7fbd340b7120 2024-04-25T10:18:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd31600640 1 -- 172.21.15.87:0/879628003 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fbd200eff20 con 0x7fbd340b7120 2024-04-25T10:18:26.103 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:26.103 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/817977905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":817977905},{"type":"v1","addr":"172.21.15.169:6835","nonce":817977905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/979883306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":979883306},{"type":"v1","addr":"172.21.15.87:6836","nonce":979883306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/796485952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":796485952},{"type":"v1","addr":"172.21.15.87:6837","nonce":796485952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:16:49.500890+0000","modified":"2024-04-25T10:16:57.376333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"d","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1643726385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1643726385},{"type":"v1","addr":"172.21.15.169:6834","nonce":1643726385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:18:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 54 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd14066f90 msgr2=0x7fbd14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd14066f90 0x7fbd14069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbd28001060 tx=0x7fbd28009040 comp rx=0 tx=0).stop 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd340b7120 msgr2=0x7fbd3415a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd340b7120 0x7fbd3415a330 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbd2001c880 tx=0x7fbd1c005840 comp rx=0 tx=0).stop 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 shutdown_connections 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd14066f90 0x7fbd14069450 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd3415a870 0x7fbd34157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd340b7120 0x7fbd3415a330 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 --2- 172.21.15.87:0/879628003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd1c004810 0x7fbd34156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 >> 172.21.15.87:0/879628003 conn(0x7fbd3409a9d0 msgr2=0x7fbd3406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 shutdown_connections 2024-04-25T10:18:26.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.101+0000 7fbd3c200640 1 -- 172.21.15.87:0/879628003 wait complete. 2024-04-25T10:18:26.123 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:26.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdb0800640 1 Processor -- start 2024-04-25T10:18:26.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdb0800640 1 -- start start 2024-04-25T10:18:26.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdb0800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efda815dab0 0x7efda8162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdb0800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efda80b57e0 0x7efda80af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdb0800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80b0210 con 0x7efda80669a0 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdb0800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80aff30 con 0x7efda80b57e0 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdb0800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda8161840 con 0x7efda815dab0 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdad400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efda80b57e0 0x7efda80af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdad400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efda80b57e0 0x7efda80af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54778/0 (socket says 172.21.15.87:54778) 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdae800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efda80669a0 0x7efda8066dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35748/0 (socket says 172.21.15.87:35748) 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdad400640 1 -- 172.21.15.87:0/3312917231 learned_addr learned my addr 172.21.15.87:0/3312917231 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdade00640 1 --2- 172.21.15.87:0/3312917231 >> v2:172.21.15.169:3300/0 conn(0x7efda815dab0 0x7efda8162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdad400640 1 -- 172.21.15.87:0/3312917231 >> v2:172.21.15.169:3300/0 conn(0x7efda815dab0 msgr2=0x7efda8162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdad400640 1 --2- 172.21.15.87:0/3312917231 >> v2:172.21.15.169:3300/0 conn(0x7efda815dab0 0x7efda8162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdad400640 1 -- 172.21.15.87:0/3312917231 >> v1:172.21.15.87:6789/0 conn(0x7efda80669a0 legacy=0x7efda8066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdad400640 1 -- 172.21.15.87:0/3312917231 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda8161b20 con 0x7efda80b57e0 2024-04-25T10:18:26.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.261+0000 7efdaca00640 1 -- 172.21.15.87:0/3312917231 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222901224 0 0) 0x7efda80b0210 con 0x7efda80669a0 2024-04-25T10:18:26.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdad400640 1 --2- 172.21.15.87:0/3312917231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80b57e0 0x7efda80af690 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7efda8070b00 tx=0x7efd9801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96868064d6973224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdaca00640 1 -- 172.21.15.87:0/3312917231 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980ab680 con 0x7efda80b57e0 2024-04-25T10:18:26.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdaca00640 1 -- 172.21.15.87:0/3312917231 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980b0070 con 0x7efda80b57e0 2024-04-25T10:18:26.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdaca00640 1 -- 172.21.15.87:0/3312917231 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980aba40 con 0x7efda80b57e0 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 -- 172.21.15.87:0/3312917231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80b57e0 msgr2=0x7efda80af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 --2- 172.21.15.87:0/3312917231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80b57e0 0x7efda80af690 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7efda8070b00 tx=0x7efd9801c460 comp rx=0 tx=0).stop 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 -- 172.21.15.87:0/3312917231 shutdown_connections 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 --2- 172.21.15.87:0/3312917231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80b57e0 0x7efda80af690 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 --2- 172.21.15.87:0/3312917231 >> v2:172.21.15.169:3300/0 conn(0x7efda815dab0 0x7efda8162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 -- 172.21.15.87:0/3312917231 >> 172.21.15.87:0/3312917231 conn(0x7efda806ac90 msgr2=0x7efda806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 -- 172.21.15.87:0/3312917231 shutdown_connections 2024-04-25T10:18:26.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.265+0000 7efdb0800640 1 -- 172.21.15.87:0/3312917231 wait complete. 2024-04-25T10:18:26.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 Processor -- start 2024-04-25T10:18:26.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 -- start start 2024-04-25T10:18:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80669a0 0x7efda80b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80b57e0 0x7efda80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda815dab0 0x7efda80cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80d0ed0 con 0x7efda80669a0 2024-04-25T10:18:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80d0d50 con 0x7efda815dab0 2024-04-25T10:18:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80d1050 con 0x7efda80b57e0 2024-04-25T10:18:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80669a0 0x7efda80b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80669a0 0x7efda80b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54788/0 (socket says 172.21.15.87:54788) 2024-04-25T10:18:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 -- 172.21.15.87:0/1434451348 learned_addr learned my addr 172.21.15.87:0/1434451348 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdad400640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80b57e0 0x7efda80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdae800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda815dab0 0x7efda80cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 -- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80b57e0 msgr2=0x7efda80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80b57e0 0x7efda80ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 -- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda815dab0 msgr2=0x7efda80cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda815dab0 0x7efda80cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 -- 172.21.15.87:0/1434451348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80b09d0 con 0x7efda80669a0 2024-04-25T10:18:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdad400640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80b57e0 0x7efda80ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdae800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda815dab0 0x7efda80cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdade00640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80669a0 0x7efda80b1a70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00d5f0 tx=0x7efd9c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efda2000640 1 -- 172.21.15.87:0/1434451348 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0125f0 con 0x7efda80669a0 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efda2000640 1 -- 172.21.15.87:0/1434451348 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c019070 con 0x7efda80669a0 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efda2000640 1 -- 172.21.15.87:0/1434451348 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0129b0 con 0x7efda80669a0 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda80b0cb0 con 0x7efda80669a0 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.269+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda80b1270 con 0x7efda80669a0 2024-04-25T10:18:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.273+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda804ef30 con 0x7efda80669a0 2024-04-25T10:18:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.273+0000 7efda2000640 1 -- 172.21.15.87:0/1434451348 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd9c012b50 con 0x7efda80669a0 2024-04-25T10:18:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.273+0000 7efda2000640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd8c066f60 0x7efd8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.273+0000 7efda2000640 1 -- 172.21.15.87:0/1434451348 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efd9c0854e0 con 0x7efda80669a0 2024-04-25T10:18:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.273+0000 7efdad400640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd8c066f60 0x7efd8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.273+0000 7efdad400640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd8c066f60 0x7efd8c069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efd980af000 tx=0x7efd98008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.277+0000 7efda2000640 1 -- 172.21.15.87:0/1434451348 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd9c056480 con 0x7efda80669a0 2024-04-25T10:18:26.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.433+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efda80aebf0 con 0x7efda80669a0 2024-04-25T10:18:26.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.433+0000 7efda2000640 1 -- 172.21.15.87:0/1434451348 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16819 (secure 0 0 0) 0x7efd9c055d80 con 0x7efda80669a0 2024-04-25T10:18:26.436 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:26.437 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":76,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:17:57.908660+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:47.854894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:48.494317+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:26.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd8c066f60 msgr2=0x7efd8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd8c066f60 0x7efd8c069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efd980af000 tx=0x7efd98008040 comp rx=0 tx=0).stop 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80669a0 msgr2=0x7efda80b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80669a0 0x7efda80b1a70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00d5f0 tx=0x7efd9c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 shutdown_connections 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd8c066f60 0x7efd8c069420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda815dab0 0x7efda80cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80b57e0 0x7efda80ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 --2- 172.21.15.87:0/1434451348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efda80669a0 0x7efda80b1a70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 >> 172.21.15.87:0/1434451348 conn(0x7efda806ac90 msgr2=0x7efda8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 shutdown_connections 2024-04-25T10:18:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.437+0000 7efdb0800640 1 -- 172.21.15.87:0/1434451348 wait complete. 2024-04-25T10:18:26.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:18:26.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe977400640 1 Processor -- start 2024-04-25T10:18:26.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe977400640 1 -- start start 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe977400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9700d0040 0x7fe97015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe977400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015b740 0x7fe9700b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe977400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe970071050 con 0x7fe9701550b0 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe977400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97015d040 con 0x7fe9700d0040 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe977400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700712f0 con 0x7fe97015b740 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe96fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9700d0040 0x7fe97015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe96fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9700d0040 0x7fe97015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54794/0 (socket says 172.21.15.87:54794) 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe96fe00640 1 -- 172.21.15.87:0/626914089 learned_addr learned my addr 172.21.15.87:0/626914089 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe975400640 1 --2- 172.21.15.87:0/626914089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015b740 0x7fe9700b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe96fe00640 1 -- 172.21.15.87:0/626914089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015b740 msgr2=0x7fe9700b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:26.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe96fe00640 1 --2- 172.21.15.87:0/626914089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015b740 0x7fe9700b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe96fe00640 1 -- 172.21.15.87:0/626914089 >> v1:172.21.15.169:6789/0 conn(0x7fe9701550b0 legacy=0x7fe9700cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.593+0000 7fe96fe00640 1 -- 172.21.15.87:0/626914089 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9700715d0 con 0x7fe9700d0040 2024-04-25T10:18:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe96fe00640 1 --2- 172.21.15.87:0/626914089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700d0040 0x7fe97015b200 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00bb10 tx=0x7fe95c01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8724d50ae9d44be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe96f400640 1 -- 172.21.15.87:0/626914089 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0b4070 con 0x7fe9700d0040 2024-04-25T10:18:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe96f400640 1 -- 172.21.15.87:0/626914089 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c0a7cd0 con 0x7fe9700d0040 2024-04-25T10:18:26.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe96f400640 1 -- 172.21.15.87:0/626914089 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0af660 con 0x7fe9700d0040 2024-04-25T10:18:26.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 -- 172.21.15.87:0/626914089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700d0040 msgr2=0x7fe97015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 --2- 172.21.15.87:0/626914089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700d0040 0x7fe97015b200 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00bb10 tx=0x7fe95c01ec00 comp rx=0 tx=0).stop 2024-04-25T10:18:26.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 -- 172.21.15.87:0/626914089 shutdown_connections 2024-04-25T10:18:26.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 --2- 172.21.15.87:0/626914089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015b740 0x7fe9700b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 --2- 172.21.15.87:0/626914089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700d0040 0x7fe97015b200 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 -- 172.21.15.87:0/626914089 >> 172.21.15.87:0/626914089 conn(0x7fe970066b70 msgr2=0x7fe970066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 -- 172.21.15.87:0/626914089 shutdown_connections 2024-04-25T10:18:26.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.597+0000 7fe977400640 1 -- 172.21.15.87:0/626914089 wait complete. 2024-04-25T10:18:26.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 Processor -- start 2024-04-25T10:18:26.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 -- start start 2024-04-25T10:18:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9700d0040 0x7fe970147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9701550b0 0x7fe9700ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015b740 0x7fe9700ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97015bcd0 con 0x7fe97015b740 2024-04-25T10:18:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97015bb50 con 0x7fe9700d0040 2024-04-25T10:18:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97015c030 con 0x7fe9701550b0 2024-04-25T10:18:26.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015b740 0x7fe9700ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015b740 0x7fe9700ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54808/0 (socket says 172.21.15.87:54808) 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 -- 172.21.15.87:0/1875549743 learned_addr learned my addr 172.21.15.87:0/1875549743 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe974a00640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9700d0040 0x7fe970147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe96fe00640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9701550b0 0x7fe9700ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 -- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9701550b0 msgr2=0x7fe9700ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9701550b0 0x7fe9700ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 -- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9700d0040 msgr2=0x7fe970147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9700d0040 0x7fe970147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 -- 172.21.15.87:0/1875549743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97015b190 con 0x7fe97015b740 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe975400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015b740 0x7fe9700ad840 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe968007e60 tx=0x7fe96800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe96cc00640 1 -- 172.21.15.87:0/1875549743 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968014420 con 0x7fe97015b740 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe96cc00640 1 -- 172.21.15.87:0/1875549743 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968019070 con 0x7fe97015b740 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe96cc00640 1 -- 172.21.15.87:0/1875549743 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968014820 con 0x7fe97015b740 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe970154770 con 0x7fe97015b740 2024-04-25T10:18:26.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97015aeb0 con 0x7fe97015b740 2024-04-25T10:18:26.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.601+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97015a9c0 con 0x7fe97015b740 2024-04-25T10:18:26.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.605+0000 7fe96cc00640 1 -- 172.21.15.87:0/1875549743 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9680149c0 con 0x7fe97015b740 2024-04-25T10:18:26.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.605+0000 7fe96cc00640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.605+0000 7fe96cc00640 1 -- 172.21.15.87:0/1875549743 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe968085710 con 0x7fe97015b740 2024-04-25T10:18:26.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.605+0000 7fe974a00640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.605+0000 7fe974a00640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe97014cac0 tx=0x7fe964009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.609+0000 7fe96cc00640 1 -- 172.21.15.87:0/1875549743 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe968056100 con 0x7fe97015b740 2024-04-25T10:18:26.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.793+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe97004ef30 con 0x7fe97015b740 2024-04-25T10:18:26.800 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:18:26.793+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:18:26.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.797+0000 7fe96cc00640 1 -- 172.21.15.87:0/1875549743 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe96805a1a0 con 0x7fe97015b740 2024-04-25T10:18:26.800 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 msgr2=0x7fe950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe97014cac0 tx=0x7fe964009040 comp rx=0 tx=0).stop 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015b740 msgr2=0x7fe9700ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015b740 0x7fe9700ad840 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe968007e60 tx=0x7fe96800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 shutdown_connections 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015b740 0x7fe9700ad840 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9701550b0 0x7fe9700ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 --2- 172.21.15.87:0/1875549743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9700d0040 0x7fe970147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 >> 172.21.15.87:0/1875549743 conn(0x7fe970066b70 msgr2=0x7fe97006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 shutdown_connections 2024-04-25T10:18:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.801+0000 7fe977400640 1 -- 172.21.15.87:0/1875549743 wait complete. 2024-04-25T10:18:26.822 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:18:26.876 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:18:26.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e29c00640 1 Processor -- start 2024-04-25T10:18:26.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e29c00640 1 -- start start 2024-04-25T10:18:26.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e29c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e2406d650 0x7f0e240ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e29c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cdab0 con 0x7f0e240cc200 2024-04-25T10:18:26.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e29c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cd210 con 0x7f0e240d0d00 2024-04-25T10:18:26.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e29c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e240cd4f0 con 0x7f0e2406d650 2024-04-25T10:18:26.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e1a000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0e240cc200 0x7f0e2406cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:35764/0 (socket says 172.21.15.87:35764) 2024-04-25T10:18:26.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.957+0000 7f0e1a000640 1 -- 172.21.15.87:0/2207744486 learned_addr learned my addr 172.21.15.87:0/2207744486 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:26.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e23400640 1 --2- 172.21.15.87:0/2207744486 >> v2:172.21.15.169:3300/0 conn(0x7f0e2406d650 0x7f0e240ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e22000640 1 -- 172.21.15.87:0/2207744486 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833461397 0 0) 0x7f0e240cdab0 con 0x7f0e240cc200 2024-04-25T10:18:26.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e22000640 1 -- 172.21.15.87:0/2207744486 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e08003530 con 0x7f0e240cc200 2024-04-25T10:18:26.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e22000640 1 -- 172.21.15.87:0/2207744486 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549658903 0 0) 0x7f0e240cd210 con 0x7f0e240d0d00 2024-04-25T10:18:26.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e22000640 1 -- 172.21.15.87:0/2207744486 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e240cdab0 con 0x7f0e240d0d00 2024-04-25T10:18:26.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e23400640 1 -- 172.21.15.87:0/2207744486 >> v1:172.21.15.169:6789/0 conn(0x7f0e240d0d00 legacy=0x7f0e24162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e23400640 1 -- 172.21.15.87:0/2207744486 >> v1:172.21.15.87:6789/0 conn(0x7f0e240cc200 legacy=0x7f0e2406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e23400640 1 -- 172.21.15.87:0/2207744486 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240cd7d0 con 0x7f0e2406d650 2024-04-25T10:18:26.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e23400640 1 --2- 172.21.15.87:0/2207744486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2406d650 0x7f0e240ccc30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009d90 tx=0x7f0e1c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9861b8cd0dd2ba1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e22000640 1 -- 172.21.15.87:0/2207744486 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0b2070 con 0x7f0e2406d650 2024-04-25T10:18:26.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e22000640 1 -- 172.21.15.87:0/2207744486 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0a58b0 con 0x7f0e2406d650 2024-04-25T10:18:26.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.961+0000 7f0e22000640 1 -- 172.21.15.87:0/2207744486 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0a5c80 con 0x7f0e2406d650 2024-04-25T10:18:26.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- 172.21.15.87:0/2207744486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2406d650 msgr2=0x7f0e240ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 --2- 172.21.15.87:0/2207744486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2406d650 0x7f0e240ccc30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009d90 tx=0x7f0e1c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- 172.21.15.87:0/2207744486 shutdown_connections 2024-04-25T10:18:26.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 --2- 172.21.15.87:0/2207744486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2406d650 0x7f0e240ccc30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- 172.21.15.87:0/2207744486 >> 172.21.15.87:0/2207744486 conn(0x7f0e24072290 msgr2=0x7f0e240726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- 172.21.15.87:0/2207744486 shutdown_connections 2024-04-25T10:18:26.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- 172.21.15.87:0/2207744486 wait complete. 2024-04-25T10:18:26.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 Processor -- start 2024-04-25T10:18:26.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- start start 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2406d650 0x7f0e240c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e240cc200 0x7f0e24158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e240d0d00 0x7f0e24154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2416c000 con 0x7f0e240d0d00 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2416be80 con 0x7f0e240cc200 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e29c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2416c180 con 0x7f0e2406d650 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e240d0d00 0x7f0e24154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e240d0d00 0x7f0e24154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54810/0 (socket says 172.21.15.87:54810) 2024-04-25T10:18:26.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 -- 172.21.15.87:0/1937855078 learned_addr learned my addr 172.21.15.87:0/1937855078 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e1a000640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e240cc200 0x7f0e24158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e22a00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2406d650 0x7f0e240c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 -- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2406d650 msgr2=0x7f0e240c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2406d650 0x7f0e240c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 -- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e240cc200 msgr2=0x7f0e24158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e240cc200 0x7f0e24158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.965+0000 7f0e23400640 1 -- 172.21.15.87:0/1937855078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240c9780 con 0x7f0e240d0d00 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e23400640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e240d0d00 0x7f0e24154360 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0b0040 tx=0x7f0e2415f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e1b400640 1 -- 172.21.15.87:0/1937855078 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0bc030 con 0x7f0e240d0d00 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e1b400640 1 -- 172.21.15.87:0/1937855078 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0a56b0 con 0x7f0e240d0d00 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e240c76b0 con 0x7f0e240d0d00 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e1b400640 1 -- 172.21.15.87:0/1937855078 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0b2040 con 0x7f0e240d0d00 2024-04-25T10:18:26.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e240c7fe0 con 0x7f0e240d0d00 2024-04-25T10:18:26.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2404ef30 con 0x7f0e240d0d00 2024-04-25T10:18:26.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e1b400640 1 -- 172.21.15.87:0/1937855078 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e1c0bf070 con 0x7f0e240d0d00 2024-04-25T10:18:26.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e1b400640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e00066f60 0x7f0e00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e1b400640 1 -- 172.21.15.87:0/1937855078 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0e1c11f2b0 con 0x7f0e240d0d00 2024-04-25T10:18:26.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.969+0000 7f0e22a00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e00066f60 0x7f0e00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.973+0000 7f0e22a00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e00066f60 0x7f0e00069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0e24074870 tx=0x7f0e14009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:26.973+0000 7f0e1b400640 1 -- 172.21.15.87:0/1937855078 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e1c0c7020 con 0x7f0e240d0d00 2024-04-25T10:18:27.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.161+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0e240b61a0 con 0x7f0e240d0d00 2024-04-25T10:18:27.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.797+0000 7f0e1b400640 1 -- 172.21.15.87:0/1937855078 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e1c016560 con 0x7f0e240d0d00 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e00066f60 msgr2=0x7f0e00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e00066f60 0x7f0e00069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0e24074870 tx=0x7f0e14009040 comp rx=0 tx=0).stop 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e240d0d00 msgr2=0x7f0e24154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e240d0d00 0x7f0e24154360 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0b0040 tx=0x7f0e2415f050 comp rx=0 tx=0).stop 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 shutdown_connections 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e00066f60 0x7f0e00069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e240d0d00 0x7f0e24154360 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e240cc200 0x7f0e24158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 --2- 172.21.15.87:0/1937855078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2406d650 0x7f0e240c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 >> 172.21.15.87:0/1937855078 conn(0x7f0e24072290 msgr2=0x7f0e240aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 shutdown_connections 2024-04-25T10:18:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.801+0000 7f0e29c00640 1 -- 172.21.15.87:0/1937855078 wait complete. 2024-04-25T10:18:27.824 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:18:27.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.961+0000 7f8b50200640 1 Processor -- start 2024-04-25T10:18:27.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.961+0000 7f8b50200640 1 -- start start 2024-04-25T10:18:27.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b50200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b480d00f0 0x7f8b480713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b50200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b4815bc00 0x7f8b480718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b50200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b48071f80 con 0x7f8b480cf040 2024-04-25T10:18:27.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b50200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b48072260 con 0x7f8b4815bc00 2024-04-25T10:18:27.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b50200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b480b3700 con 0x7f8b480d00f0 2024-04-25T10:18:27.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b4815bc00 0x7f8b480718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4d800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8b480cf040 0x7f8b480cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47238/0 (socket says 172.21.15.87:47238) 2024-04-25T10:18:27.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4d800640 1 -- 172.21.15.87:0/1664634436 learned_addr learned my addr 172.21.15.87:0/1664634436 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:27.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4ce00640 1 --2- 172.21.15.87:0/1664634436 >> v2:172.21.15.169:3300/0 conn(0x7f8b480d00f0 0x7f8b480713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4e200640 1 -- 172.21.15.87:0/1664634436 >> v2:172.21.15.169:3300/0 conn(0x7f8b480d00f0 msgr2=0x7f8b480713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4e200640 1 --2- 172.21.15.87:0/1664634436 >> v2:172.21.15.169:3300/0 conn(0x7f8b480d00f0 0x7f8b480713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4e200640 1 -- 172.21.15.87:0/1664634436 >> v1:172.21.15.87:6789/0 conn(0x7f8b480cf040 legacy=0x7f8b480cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.965+0000 7f8b4e200640 1 -- 172.21.15.87:0/1664634436 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4815b120 con 0x7f8b4815bc00 2024-04-25T10:18:27.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b4e200640 1 --2- 172.21.15.87:0/1664634436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b480718e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8b4806e840 tx=0x7f8b4001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac7c8a5d96f4be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b47e00640 1 -- 172.21.15.87:0/1664634436 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b2070 con 0x7f8b4815bc00 2024-04-25T10:18:27.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b47e00640 1 -- 172.21.15.87:0/1664634436 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b400a58e0 con 0x7f8b4815bc00 2024-04-25T10:18:27.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b47e00640 1 -- 172.21.15.87:0/1664634436 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400a5cb0 con 0x7f8b4815bc00 2024-04-25T10:18:27.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 -- 172.21.15.87:0/1664634436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 msgr2=0x7f8b480718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 --2- 172.21.15.87:0/1664634436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b480718e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8b4806e840 tx=0x7f8b4001c930 comp rx=0 tx=0).stop 2024-04-25T10:18:27.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 -- 172.21.15.87:0/1664634436 shutdown_connections 2024-04-25T10:18:27.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 --2- 172.21.15.87:0/1664634436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b480718e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 --2- 172.21.15.87:0/1664634436 >> v2:172.21.15.169:3300/0 conn(0x7f8b480d00f0 0x7f8b480713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 -- 172.21.15.87:0/1664634436 >> 172.21.15.87:0/1664634436 conn(0x7f8b48066b70 msgr2=0x7f8b48066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 -- 172.21.15.87:0/1664634436 shutdown_connections 2024-04-25T10:18:27.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 -- 172.21.15.87:0/1664634436 wait complete. 2024-04-25T10:18:27.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 Processor -- start 2024-04-25T10:18:27.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 -- start start 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b480cf040 0x7f8b48161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b480d00f0 0x7f8b481fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.969+0000 7f8b50200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b48164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b50200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480aec90 con 0x7f8b4815bc00 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b50200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480cf630 con 0x7f8b480cf040 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b50200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480aee10 con 0x7f8b480d00f0 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b48164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b480cf040 0x7f8b48161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b48164ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44738/0 (socket says 172.21.15.87:44738) 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 -- 172.21.15.87:0/3854033496 learned_addr learned my addr 172.21.15.87:0/3854033496 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4d800640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b480cf040 0x7f8b48161ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:59922/0 (socket says 172.21.15.87:59922) 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4ce00640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b480d00f0 0x7f8b481fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 -- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b480d00f0 msgr2=0x7f8b481fa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b480d00f0 0x7f8b481fa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 -- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b480cf040 msgr2=0x7f8b48161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b480cf040 0x7f8b48161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 -- 172.21.15.87:0/3854033496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b481fac30 con 0x7f8b4815bc00 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b4e200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b48164ff0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8b40019540 tx=0x7f8b400196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b45600640 1 -- 172.21.15.87:0/3854033496 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b2050 con 0x7f8b4815bc00 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b45600640 1 -- 172.21.15.87:0/3854033496 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b400be070 con 0x7f8b4815bc00 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b45600640 1 -- 172.21.15.87:0/3854033496 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400ada90 con 0x7f8b4815bc00 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b481faed0 con 0x7f8b4815bc00 2024-04-25T10:18:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b481fb490 con 0x7f8b4815bc00 2024-04-25T10:18:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48164c50 con 0x7f8b4815bc00 2024-04-25T10:18:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.973+0000 7f8b45600640 1 -- 172.21.15.87:0/3854033496 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b400adc30 con 0x7f8b4815bc00 2024-04-25T10:18:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.977+0000 7f8b45600640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b28066f60 0x7f8b28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.977+0000 7f8b45600640 1 -- 172.21.15.87:0/3854033496 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8b401201c0 con 0x7f8b4815bc00 2024-04-25T10:18:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.977+0000 7f8b4d800640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b28066f60 0x7f8b28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.977+0000 7f8b4d800640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b28066f60 0x7f8b28069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000f70 tx=0x7f8b3c007e90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:27.981+0000 7f8b45600640 1 -- 172.21.15.87:0/3854033496 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b40016520 con 0x7f8b4815bc00 2024-04-25T10:18:28.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.137+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b4804ef30 con 0x7f8b4815bc00 2024-04-25T10:18:28.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.993+0000 7f8b45600640 1 -- 172.21.15.87:0/3854033496 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b400f0ae0 con 0x7f8b4815bc00 2024-04-25T10:18:28.995 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:18:28.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.993+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b28066f60 msgr2=0x7f8b28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.993+0000 7f8b50200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b28066f60 0x7f8b28069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000f70 tx=0x7f8b3c007e90 comp rx=0 tx=0).stop 2024-04-25T10:18:28.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.993+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 msgr2=0x7f8b48164ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.993+0000 7f8b50200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b48164ff0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8b40019540 tx=0x7f8b400196d0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 shutdown_connections 2024-04-25T10:18:28.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b28066f60 0x7f8b28069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b4815bc00 0x7f8b48164ff0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b480d00f0 0x7f8b481fa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 --2- 172.21.15.87:0/3854033496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b480cf040 0x7f8b48161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 >> 172.21.15.87:0/3854033496 conn(0x7f8b48066b70 msgr2=0x7f8b4806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 shutdown_connections 2024-04-25T10:18:28.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:28.997+0000 7f8b50200640 1 -- 172.21.15.87:0/3854033496 wait complete. 2024-04-25T10:18:29.020 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:18:29.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa369400640 1 Processor -- start 2024-04-25T10:18:29.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa369400640 1 -- start start 2024-04-25T10:18:29.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa369400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa36406b6a0 0x7fa36406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa369400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa3640b3fc0 0x7fa3640b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa369400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3640b2f50 con 0x7fa3640b26b0 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa369400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36406bfc0 con 0x7fa3640b3fc0 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa369400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36406b040 con 0x7fa36406b6a0 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa363200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa3640b3fc0 0x7fa3640b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa363200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa3640b3fc0 0x7fa3640b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44740/0 (socket says 172.21.15.87:44740) 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa363200640 1 -- 172.21.15.87:0/1297784997 learned_addr learned my addr 172.21.15.87:0/1297784997 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:29.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa362800640 1 --2- 172.21.15.87:0/1297784997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa36406b6a0 0x7fa36406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa363200640 1 -- 172.21.15.87:0/1297784997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa36406b6a0 msgr2=0x7fa36406ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:29.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa363200640 1 --2- 172.21.15.87:0/1297784997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa36406b6a0 0x7fa36406ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa363200640 1 -- 172.21.15.87:0/1297784997 >> v1:172.21.15.169:6789/0 conn(0x7fa3640b26b0 legacy=0x7fa3640cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa363200640 1 -- 172.21.15.87:0/1297784997 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36415f360 con 0x7fa3640b3fc0 2024-04-25T10:18:29.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.165+0000 7fa362800640 1 --2- 172.21.15.87:0/1297784997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa36406b6a0 0x7fa36406ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:29.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa363200640 1 --2- 172.21.15.87:0/1297784997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa3640b1890 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009e50 tx=0x7fa35c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a1fe6530033bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa361e00640 1 -- 172.21.15.87:0/1297784997 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0a5700 con 0x7fa3640b3fc0 2024-04-25T10:18:29.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa361e00640 1 -- 172.21.15.87:0/1297784997 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c0a58a0 con 0x7fa3640b3fc0 2024-04-25T10:18:29.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa361e00640 1 -- 172.21.15.87:0/1297784997 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0a5c70 con 0x7fa3640b3fc0 2024-04-25T10:18:29.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 -- 172.21.15.87:0/1297784997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 msgr2=0x7fa3640b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 --2- 172.21.15.87:0/1297784997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa3640b1890 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009e50 tx=0x7fa35c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:18:29.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 -- 172.21.15.87:0/1297784997 shutdown_connections 2024-04-25T10:18:29.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 --2- 172.21.15.87:0/1297784997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa3640b1890 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 --2- 172.21.15.87:0/1297784997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa36406b6a0 0x7fa36406ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 -- 172.21.15.87:0/1297784997 >> 172.21.15.87:0/1297784997 conn(0x7fa3640b34a0 msgr2=0x7fa3640b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 -- 172.21.15.87:0/1297784997 shutdown_connections 2024-04-25T10:18:29.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.169+0000 7fa369400640 1 -- 172.21.15.87:0/1297784997 wait complete. 2024-04-25T10:18:29.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 Processor -- start 2024-04-25T10:18:29.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 -- start start 2024-04-25T10:18:29.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36406b6a0 0x7fa3640c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3640b26b0 0x7fa3640b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa36414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36416b380 con 0x7fa3640b3fc0 2024-04-25T10:18:29.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36416b200 con 0x7fa36406b6a0 2024-04-25T10:18:29.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36416b500 con 0x7fa3640b26b0 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa36414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa36414d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44752/0 (socket says 172.21.15.87:44752) 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 -- 172.21.15.87:0/2763499193 learned_addr learned my addr 172.21.15.87:0/2763499193 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa359e00640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3640b26b0 0x7fa3640b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa362800640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36406b6a0 0x7fa3640c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 -- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3640b26b0 msgr2=0x7fa3640b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3640b26b0 0x7fa3640b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 -- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36406b6a0 msgr2=0x7fa3640c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36406b6a0 0x7fa3640c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 -- 172.21.15.87:0/2763499193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364160210 con 0x7fa3640b3fc0 2024-04-25T10:18:29.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa363200640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa36414d350 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0af040 tx=0x7fa35c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa35b400640 1 -- 172.21.15.87:0/2763499193 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c009510 con 0x7fa3640b3fc0 2024-04-25T10:18:29.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa35b400640 1 -- 172.21.15.87:0/2763499193 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c0b8440 con 0x7fa3640b3fc0 2024-04-25T10:18:29.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa35b400640 1 -- 172.21.15.87:0/2763499193 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0b8850 con 0x7fa3640b3fc0 2024-04-25T10:18:29.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36406fec0 con 0x7fa3640b3fc0 2024-04-25T10:18:29.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.173+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3641604b0 con 0x7fa3640b3fc0 2024-04-25T10:18:29.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.177+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36404ef30 con 0x7fa3640b3fc0 2024-04-25T10:18:29.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.177+0000 7fa35b400640 1 -- 172.21.15.87:0/2763499193 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa35c0c1020 con 0x7fa3640b3fc0 2024-04-25T10:18:29.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.177+0000 7fa35b400640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa340066f60 0x7fa340069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.177+0000 7fa35b400640 1 -- 172.21.15.87:0/2763499193 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fa35c11f270 con 0x7fa3640b3fc0 2024-04-25T10:18:29.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.177+0000 7fa362800640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa340066f60 0x7fa340069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.177+0000 7fa362800640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa340066f60 0x7fa340069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa36415e3a0 tx=0x7fa354008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.181+0000 7fa35b400640 1 -- 172.21.15.87:0/2763499193 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa35c0c8020 con 0x7fa3640b3fc0 2024-04-25T10:18:29.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.337+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa36406b480 con 0x7fa3640b3fc0 2024-04-25T10:18:30.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:29.997+0000 7fa35b400640 1 -- 172.21.15.87:0/2763499193 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa35c016620 con 0x7fa3640b3fc0 2024-04-25T10:18:30.000 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:18:30.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa340066f60 msgr2=0x7fa340069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa340066f60 0x7fa340069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa36415e3a0 tx=0x7fa354008040 comp rx=0 tx=0).stop 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 msgr2=0x7fa36414d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa36414d350 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0af040 tx=0x7fa35c0197d0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 shutdown_connections 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa340066f60 0x7fa340069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3640b3fc0 0x7fa36414d350 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3640b26b0 0x7fa3640b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 --2- 172.21.15.87:0/2763499193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36406b6a0 0x7fa3640c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 >> 172.21.15.87:0/2763499193 conn(0x7fa3640b34a0 msgr2=0x7fa3640afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 shutdown_connections 2024-04-25T10:18:30.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.001+0000 7fa369400640 1 -- 172.21.15.87:0/2763499193 wait complete. 2024-04-25T10:18:30.030 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:18:30.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.165+0000 7f33e9e00640 1 Processor -- start 2024-04-25T10:18:30.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.165+0000 7f33e9e00640 1 -- start start 2024-04-25T10:18:30.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e9e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33e40aeda0 0x7f33e4071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e4072020 0x7f33e4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e9e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e4159920 con 0x7f33e4159540 2024-04-25T10:18:30.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e9e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e415c2f0 con 0x7f33e40aeda0 2024-04-25T10:18:30.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e415c5d0 con 0x7f33e4072020 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e4072020 0x7f33e4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33e40aeda0 0x7f33e4071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33e40aeda0 0x7f33e4071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:59932/0 (socket says 172.21.15.87:59932) 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e2000640 1 -- 172.21.15.87:0/2139741575 learned_addr learned my addr 172.21.15.87:0/2139741575 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e2000640 1 -- 172.21.15.87:0/2139741575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e4072020 msgr2=0x7f33e4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:30.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e2000640 1 --2- 172.21.15.87:0/2139741575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e4072020 0x7f33e4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e2000640 1 -- 172.21.15.87:0/2139741575 >> v1:172.21.15.169:6789/0 conn(0x7f33e4159540 legacy=0x7f33e40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.169+0000 7f33e2000640 1 -- 172.21.15.87:0/2139741575 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e415c8b0 con 0x7f33e40aeda0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e2000640 1 --2- 172.21.15.87:0/2139741575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40aeda0 0x7f33e4071ae0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009870 tx=0x7f33d001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630799098385d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e1600640 1 -- 172.21.15.87:0/2139741575 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00a9610 con 0x7f33e40aeda0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e1600640 1 -- 172.21.15.87:0/2139741575 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d00a97b0 con 0x7f33e40aeda0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e1600640 1 -- 172.21.15.87:0/2139741575 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00a9b80 con 0x7f33e40aeda0 2024-04-25T10:18:30.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- 172.21.15.87:0/2139741575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40aeda0 msgr2=0x7f33e4071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 --2- 172.21.15.87:0/2139741575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40aeda0 0x7f33e4071ae0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009870 tx=0x7f33d001c530 comp rx=0 tx=0).stop 2024-04-25T10:18:30.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- 172.21.15.87:0/2139741575 shutdown_connections 2024-04-25T10:18:30.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 --2- 172.21.15.87:0/2139741575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e4072020 0x7f33e4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 --2- 172.21.15.87:0/2139741575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e40aeda0 0x7f33e4071ae0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- 172.21.15.87:0/2139741575 >> 172.21.15.87:0/2139741575 conn(0x7f33e40669a0 msgr2=0x7f33e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- 172.21.15.87:0/2139741575 shutdown_connections 2024-04-25T10:18:30.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- 172.21.15.87:0/2139741575 wait complete. 2024-04-25T10:18:30.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 Processor -- start 2024-04-25T10:18:30.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- start start 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4072020 0x7f33e414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40aeda0 0x7f33e4153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e4159540 0x7f33e415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40710a0 con 0x7f33e4159540 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4072430 con 0x7f33e4072020 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.173+0000 7f33e9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4071220 con 0x7f33e40aeda0 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e4159540 0x7f33e415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40aeda0 0x7f33e4153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4072020 0x7f33e414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40aeda0 0x7f33e4153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:50218/0 (socket says 172.21.15.87:50218) 2024-04-25T10:18:30.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e2000640 1 -- 172.21.15.87:0/1842332712 learned_addr learned my addr 172.21.15.87:0/1842332712 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e3400640 1 -- 172.21.15.87:0/1842332712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40aeda0 msgr2=0x7f33e4153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e3400640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40aeda0 0x7f33e4153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e3400640 1 -- 172.21.15.87:0/1842332712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4072020 msgr2=0x7f33e414e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e3400640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4072020 0x7f33e414e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e3400640 1 -- 172.21.15.87:0/1842332712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e41fa050 con 0x7f33e4159540 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e2000640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40aeda0 0x7f33e4153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e3400640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e4159540 0x7f33e415a660 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e800 tx=0x7f33dc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33db400640 1 -- 172.21.15.87:0/1842332712 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc017070 con 0x7f33e4159540 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33db400640 1 -- 172.21.15.87:0/1842332712 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc00c940 con 0x7f33e4159540 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e4150ef0 con 0x7f33e4159540 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33db400640 1 -- 172.21.15.87:0/1842332712 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00cd30 con 0x7f33e4159540 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e41fa2f0 con 0x7f33e4159540 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e404ef30 con 0x7f33e4159540 2024-04-25T10:18:30.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.177+0000 7f33db400640 1 -- 172.21.15.87:0/1842332712 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33dc01a070 con 0x7f33e4159540 2024-04-25T10:18:30.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.181+0000 7f33db400640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33c4066f60 0x7f33c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.181+0000 7f33db400640 1 -- 172.21.15.87:0/1842332712 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f33dc084fe0 con 0x7f33e4159540 2024-04-25T10:18:30.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.181+0000 7f33e2a00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33c4066f60 0x7f33c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.181+0000 7f33e2a00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33c4066f60 0x7f33c4069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f33e41544b0 tx=0x7f33d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.185+0000 7f33db400640 1 -- 172.21.15.87:0/1842332712 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33dc055c40 con 0x7f33e4159540 2024-04-25T10:18:30.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.337+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f33e406d7f0 con 0x7f33e4159540 2024-04-25T10:18:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.341+0000 7f33db400640 1 -- 172.21.15.87:0/1842332712 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f33dc059ce0 con 0x7f33e4159540 2024-04-25T10:18:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33c4066f60 msgr2=0x7f33c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33c4066f60 0x7f33c4069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f33e41544b0 tx=0x7f33d4009040 comp rx=0 tx=0).stop 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e4159540 msgr2=0x7f33e415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e4159540 0x7f33e415a660 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e800 tx=0x7f33dc00ecc0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 shutdown_connections 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33c4066f60 0x7f33c4069420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e4159540 0x7f33e415a660 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33e40aeda0 0x7f33e4153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 --2- 172.21.15.87:0/1842332712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4072020 0x7f33e414e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 >> 172.21.15.87:0/1842332712 conn(0x7f33e40669a0 msgr2=0x7f33e415d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 shutdown_connections 2024-04-25T10:18:30.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.345+0000 7f33e9e00640 1 -- 172.21.15.87:0/1842332712 wait complete. 2024-04-25T10:18:30.366 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:18:30.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.497+0000 7fa172a00640 1 Processor -- start 2024-04-25T10:18:30.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.497+0000 7fa172a00640 1 -- start start 2024-04-25T10:18:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa172a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa16c0d0040 0x7fa16c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa172a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c071050 con 0x7fa16c1550b0 2024-04-25T10:18:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa172a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c15d040 con 0x7fa16c15b740 2024-04-25T10:18:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa172a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c0712f0 con 0x7fa16c0d0040 2024-04-25T10:18:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa16be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa16c1550b0 0x7fa16c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47254/0 (socket says 172.21.15.87:47254) 2024-04-25T10:18:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa16be00640 1 -- 172.21.15.87:0/2046465888 learned_addr learned my addr 172.21.15.87:0/2046465888 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa163e00640 1 --2- 172.21.15.87:0/2046465888 >> v2:172.21.15.169:3300/0 conn(0x7fa16c0d0040 0x7fa16c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056568459 0 0) 0x7fa16c071050 con 0x7fa16c1550b0 2024-04-25T10:18:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa150003530 con 0x7fa16c1550b0 2024-04-25T10:18:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992326825 0 0) 0x7fa16c15d040 con 0x7fa16c15b740 2024-04-25T10:18:30.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa16c071050 con 0x7fa16c15b740 2024-04-25T10:18:30.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa163e00640 1 -- 172.21.15.87:0/2046465888 >> v1:172.21.15.169:6789/0 conn(0x7fa16c15b740 legacy=0x7fa16c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa163e00640 1 -- 172.21.15.87:0/2046465888 >> v1:172.21.15.87:6789/0 conn(0x7fa16c1550b0 legacy=0x7fa16c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216020652 0 0) 0x7fa150003530 con 0x7fa16c1550b0 2024-04-25T10:18:30.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.501+0000 7fa163e00640 1 -- 172.21.15.87:0/2046465888 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0715d0 con 0x7fa16c0d0040 2024-04-25T10:18:30.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa163e00640 1 --2- 172.21.15.87:0/2046465888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 0x7fa16c15b200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa1540097c0 tx=0x7fa15401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=292aa64a396c9cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540b0070 con 0x7fa16c0d0040 2024-04-25T10:18:30.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1540ab400 con 0x7fa16c0d0040 2024-04-25T10:18:30.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa16b400640 1 -- 172.21.15.87:0/2046465888 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c001680 con 0x7fa16c0d0040 2024-04-25T10:18:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 -- 172.21.15.87:0/2046465888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 msgr2=0x7fa16c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 --2- 172.21.15.87:0/2046465888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 0x7fa16c15b200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa1540097c0 tx=0x7fa15401c390 comp rx=0 tx=0).stop 2024-04-25T10:18:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 -- 172.21.15.87:0/2046465888 shutdown_connections 2024-04-25T10:18:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 --2- 172.21.15.87:0/2046465888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 0x7fa16c15b200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 -- 172.21.15.87:0/2046465888 >> 172.21.15.87:0/2046465888 conn(0x7fa16c066b70 msgr2=0x7fa16c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 -- 172.21.15.87:0/2046465888 shutdown_connections 2024-04-25T10:18:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 -- 172.21.15.87:0/2046465888 wait complete. 2024-04-25T10:18:30.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 Processor -- start 2024-04-25T10:18:30.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 -- start start 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 0x7fa16c14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.505+0000 7fa172a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c1550b0 0x7fa16c14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa172a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa16c15b740 0x7fa16c13d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa172a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c15bcd0 con 0x7fa16c1550b0 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa172a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c15bb50 con 0x7fa16c0d0040 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa172a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c06c090 con 0x7fa16c15b740 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c1550b0 0x7fa16c14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c1550b0 0x7fa16c14ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44764/0 (socket says 172.21.15.87:44764) 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 -- 172.21.15.87:0/2902810004 learned_addr learned my addr 172.21.15.87:0/2902810004 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa16be00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 0x7fa16c14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa170a00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa16c15b740 0x7fa16c13d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 -- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa16c15b740 msgr2=0x7fa16c13d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa16c15b740 0x7fa16c13d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 -- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 msgr2=0x7fa16c14d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 0x7fa16c14d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 -- 172.21.15.87:0/2902810004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c14f470 con 0x7fa16c1550b0 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa163e00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c1550b0 0x7fa16c14ef30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa1540ae040 tx=0x7fa15401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa168c00640 1 -- 172.21.15.87:0/2902810004 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540bb020 con 0x7fa16c1550b0 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa168c00640 1 -- 172.21.15.87:0/2902810004 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1540b1040 con 0x7fa16c1550b0 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa168c00640 1 -- 172.21.15.87:0/2902810004 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540b0050 con 0x7fa16c1550b0 2024-04-25T10:18:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c14f690 con 0x7fa16c1550b0 2024-04-25T10:18:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c14cf00 con 0x7fa16c1550b0 2024-04-25T10:18:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c14cd30 con 0x7fa16c1550b0 2024-04-25T10:18:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.509+0000 7fa168c00640 1 -- 172.21.15.87:0/2902810004 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1540be070 con 0x7fa16c1550b0 2024-04-25T10:18:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.513+0000 7fa168c00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa148066f90 0x7fa148069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.513+0000 7fa168c00640 1 -- 172.21.15.87:0/2902810004 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa15411e200 con 0x7fa16c1550b0 2024-04-25T10:18:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.513+0000 7fa16be00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa148066f90 0x7fa148069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.513+0000 7fa16be00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa148066f90 0x7fa148069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa15c000fe0 tx=0x7fa15c001160 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.517+0000 7fa168c00640 1 -- 172.21.15.87:0/2902810004 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa154015fb0 con 0x7fa16c1550b0 2024-04-25T10:18:30.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:30.721+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7fa16c0b2750 con 0x7fa16c1550b0 2024-04-25T10:18:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa168c00640 1 -- 172.21.15.87:0/2902810004 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v305) v1 ==== 158+0+0 (secure 0 0 0) 0x7fa1540eeea0 con 0x7fa16c1550b0 2024-04-25T10:18:31.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa148066f90 msgr2=0x7fa148069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa148066f90 0x7fa148069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa15c000fe0 tx=0x7fa15c001160 comp rx=0 tx=0).stop 2024-04-25T10:18:31.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c1550b0 msgr2=0x7fa16c14ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c1550b0 0x7fa16c14ef30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa1540ae040 tx=0x7fa15401abd0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 shutdown_connections 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa148066f90 0x7fa148069450 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa16c15b740 0x7fa16c13d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c1550b0 0x7fa16c14ef30 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 --2- 172.21.15.87:0/2902810004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa16c0d0040 0x7fa16c14d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 >> 172.21.15.87:0/2902810004 conn(0x7fa16c066b70 msgr2=0x7fa16c0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 shutdown_connections 2024-04-25T10:18:31.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.000+0000 7fa172a00640 1 -- 172.21.15.87:0/2902810004 wait complete. 2024-04-25T10:18:31.024 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T10:18:31.024 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:18:31.025 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:31.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e96c00640 1 Processor -- start 2024-04-25T10:18:31.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e96c00640 1 -- start start 2024-04-25T10:18:31.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e96c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e90158260 0x7f0e9015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e96c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e900b5c20 con 0x7f0e90066fe0 2024-04-25T10:18:31.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e96c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e900af6e0 con 0x7f0e90066c00 2024-04-25T10:18:31.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e96c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e900af9c0 con 0x7f0e90158260 2024-04-25T10:18:31.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e87400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0e90066fe0 0x7f0e90157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47258/0 (socket says 172.21.15.87:47258) 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.156+0000 7f0e87400640 1 -- 172.21.15.87:0/2358642014 learned_addr learned my addr 172.21.15.87:0/2358642014 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e94c00640 1 --2- 172.21.15.87:0/2358642014 >> v2:172.21.15.169:3300/0 conn(0x7f0e90158260 0x7f0e9015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e8f400640 1 -- 172.21.15.87:0/2358642014 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204004985 0 0) 0x7f0e900b5c20 con 0x7f0e90066fe0 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e8f400640 1 -- 172.21.15.87:0/2358642014 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e74003530 con 0x7f0e90066fe0 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e8f400640 1 -- 172.21.15.87:0/2358642014 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090683284 0 0) 0x7f0e900af6e0 con 0x7f0e90066c00 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e8f400640 1 -- 172.21.15.87:0/2358642014 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e900b5c20 con 0x7f0e90066c00 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e94c00640 1 -- 172.21.15.87:0/2358642014 >> v1:172.21.15.169:6789/0 conn(0x7f0e90066c00 legacy=0x7f0e9006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e94c00640 1 -- 172.21.15.87:0/2358642014 >> v1:172.21.15.87:6789/0 conn(0x7f0e90066fe0 legacy=0x7f0e90157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e94c00640 1 -- 172.21.15.87:0/2358642014 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e900afca0 con 0x7f0e90158260 2024-04-25T10:18:31.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e94c00640 1 --2- 172.21.15.87:0/2358642014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90158260 0x7f0e9015e110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009d90 tx=0x7f0e8801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70da57ec5e4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e8f400640 1 -- 172.21.15.87:0/2358642014 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880b2070 con 0x7f0e90158260 2024-04-25T10:18:31.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e8f400640 1 -- 172.21.15.87:0/2358642014 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e880a5940 con 0x7f0e90158260 2024-04-25T10:18:31.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.160+0000 7f0e8f400640 1 -- 172.21.15.87:0/2358642014 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880a5cf0 con 0x7f0e90158260 2024-04-25T10:18:31.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- 172.21.15.87:0/2358642014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90158260 msgr2=0x7f0e9015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 --2- 172.21.15.87:0/2358642014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90158260 0x7f0e9015e110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009d90 tx=0x7f0e8801c970 comp rx=0 tx=0).stop 2024-04-25T10:18:31.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- 172.21.15.87:0/2358642014 shutdown_connections 2024-04-25T10:18:31.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 --2- 172.21.15.87:0/2358642014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90158260 0x7f0e9015e110 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- 172.21.15.87:0/2358642014 >> 172.21.15.87:0/2358642014 conn(0x7f0e900b24f0 msgr2=0x7f0e900b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- 172.21.15.87:0/2358642014 shutdown_connections 2024-04-25T10:18:31.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- 172.21.15.87:0/2358642014 wait complete. 2024-04-25T10:18:31.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 Processor -- start 2024-04-25T10:18:31.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- start start 2024-04-25T10:18:31.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90066c00 0x7f0e90144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90066fe0 0x7f0e9015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e90158260 0x7f0e9015f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9016bb00 con 0x7f0e90158260 2024-04-25T10:18:31.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e900af270 con 0x7f0e90066c00 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e96c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9016bc80 con 0x7f0e90066fe0 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e90158260 0x7f0e9015f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e90158260 0x7f0e9015f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44766/0 (socket says 172.21.15.87:44766) 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 -- 172.21.15.87:0/757498147 learned_addr learned my addr 172.21.15.87:0/757498147 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e8fe00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90066c00 0x7f0e90144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e87400640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90066fe0 0x7f0e9015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 -- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90066fe0 msgr2=0x7f0e9015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90066fe0 0x7f0e9015f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 -- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90066c00 msgr2=0x7f0e90144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90066c00 0x7f0e90144490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.164+0000 7f0e94c00640 1 -- 172.21.15.87:0/757498147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e901449d0 con 0x7f0e90158260 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e94c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e90158260 0x7f0e9015f950 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009d90 tx=0x7f0e88019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e8cc00640 1 -- 172.21.15.87:0/757498147 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880b2050 con 0x7f0e90158260 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e8cc00640 1 -- 172.21.15.87:0/757498147 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e880be070 con 0x7f0e90158260 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e8cc00640 1 -- 172.21.15.87:0/757498147 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88004040 con 0x7f0e90158260 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9014c7d0 con 0x7f0e90158260 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9014cd90 con 0x7f0e90158260 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9004ef30 con 0x7f0e90158260 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e8cc00640 1 -- 172.21.15.87:0/757498147 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e880041e0 con 0x7f0e90158260 2024-04-25T10:18:31.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e8cc00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e6c066f60 0x7f0e6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e8cc00640 1 -- 172.21.15.87:0/757498147 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0e8811fc30 con 0x7f0e90158260 2024-04-25T10:18:31.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.168+0000 7f0e8fe00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e6c066f60 0x7f0e6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.172+0000 7f0e8fe00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e6c066f60 0x7f0e6c069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e800011f0 tx=0x7f0e80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.172+0000 7f0e8cc00640 1 -- 172.21.15.87:0/757498147 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e88016560 con 0x7f0e90158260 2024-04-25T10:18:31.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.328+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e900cff00 con 0x7f0e90158260 2024-04-25T10:18:31.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e8cc00640 1 -- 172.21.15.87:0/757498147 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13011 (secure 0 0 0) 0x7f0e880f08e0 con 0x7f0e90158260 2024-04-25T10:18:31.334 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:31.334 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":79,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:18:29.994697+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/1643726385":"2024-04-26T10:18:26.796900+0000","172.21.15.169:6832/1643726385":"2024-04-26T10:18:26.796900+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e6c066f60 msgr2=0x7f0e6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e6c066f60 0x7f0e6c069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e800011f0 tx=0x7f0e80009040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e90158260 msgr2=0x7f0e9015f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e90158260 0x7f0e9015f950 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009d90 tx=0x7f0e88019710 comp rx=0 tx=0).stop 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 shutdown_connections 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0e6c066f60 0x7f0e6c069420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e90158260 0x7f0e9015f950 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90066fe0 0x7f0e9015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 --2- 172.21.15.87:0/757498147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90066c00 0x7f0e90144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 >> 172.21.15.87:0/757498147 conn(0x7f0e900b24f0 msgr2=0x7f0e900b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 shutdown_connections 2024-04-25T10:18:31.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.332+0000 7f0e96c00640 1 -- 172.21.15.87:0/757498147 wait complete. 2024-04-25T10:18:31.355 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:31.355 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:31.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.488+0000 7f68ffa00640 1 Processor -- start 2024-04-25T10:18:31.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.488+0000 7f68ffa00640 1 -- start start 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.488+0000 7f68ffa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f68f815dbe0 0x7f68f80ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68ffa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f68f80d1160 0x7f68f80b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68ffa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f815dfc0 con 0x7f68f815ee90 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68ffa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f80d1fc0 con 0x7f68f80d1160 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68ffa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f80d22a0 con 0x7f68f815dbe0 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68f7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f68f815ee90 0x7f68f80d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47268/0 (socket says 172.21.15.87:47268) 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68f7e00640 1 -- 172.21.15.87:0/1981317232 learned_addr learned my addr 172.21.15.87:0/1981317232 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:31.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fda00640 1 --2- 172.21.15.87:0/1981317232 >> v2:172.21.15.87:3300/0 conn(0x7f68f80d1160 0x7f68f80b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fd000640 1 --2- 172.21.15.87:0/1981317232 >> v2:172.21.15.169:3300/0 conn(0x7f68f815dbe0 0x7f68f80ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68f7400640 1 -- 172.21.15.87:0/1981317232 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771815739 0 0) 0x7f68f815dfc0 con 0x7f68f815ee90 2024-04-25T10:18:31.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68f7400640 1 -- 172.21.15.87:0/1981317232 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e0003530 con 0x7f68f815ee90 2024-04-25T10:18:31.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fda00640 1 -- 172.21.15.87:0/1981317232 >> v2:172.21.15.169:3300/0 conn(0x7f68f815dbe0 msgr2=0x7f68f80ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fda00640 1 --2- 172.21.15.87:0/1981317232 >> v2:172.21.15.169:3300/0 conn(0x7f68f815dbe0 0x7f68f80ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fda00640 1 -- 172.21.15.87:0/1981317232 >> v1:172.21.15.87:6789/0 conn(0x7f68f815ee90 legacy=0x7f68f80d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fda00640 1 -- 172.21.15.87:0/1981317232 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80d2580 con 0x7f68f80d1160 2024-04-25T10:18:31.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fd000640 1 --2- 172.21.15.87:0/1981317232 >> v2:172.21.15.169:3300/0 conn(0x7f68f815dbe0 0x7f68f80ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:31.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68fda00640 1 --2- 172.21.15.87:0/1981317232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f80d1160 0x7f68f80b8c90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009d90 tx=0x7f68f001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13122f64518275c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.492+0000 7f68f7400640 1 -- 172.21.15.87:0/1981317232 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00b2070 con 0x7f68f80d1160 2024-04-25T10:18:31.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68f7400640 1 -- 172.21.15.87:0/1981317232 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f00a57f0 con 0x7f68f80d1160 2024-04-25T10:18:31.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68f7400640 1 -- 172.21.15.87:0/1981317232 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00a5bc0 con 0x7f68f80d1160 2024-04-25T10:18:31.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- 172.21.15.87:0/1981317232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f80d1160 msgr2=0x7f68f80b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 --2- 172.21.15.87:0/1981317232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f80d1160 0x7f68f80b8c90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009d90 tx=0x7f68f001c930 comp rx=0 tx=0).stop 2024-04-25T10:18:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- 172.21.15.87:0/1981317232 shutdown_connections 2024-04-25T10:18:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 --2- 172.21.15.87:0/1981317232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f80d1160 0x7f68f80b8c90 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 --2- 172.21.15.87:0/1981317232 >> v2:172.21.15.169:3300/0 conn(0x7f68f815dbe0 0x7f68f80ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- 172.21.15.87:0/1981317232 >> 172.21.15.87:0/1981317232 conn(0x7f68f806b090 msgr2=0x7f68f806b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- 172.21.15.87:0/1981317232 shutdown_connections 2024-04-25T10:18:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- 172.21.15.87:0/1981317232 wait complete. 2024-04-25T10:18:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 Processor -- start 2024-04-25T10:18:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- start start 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80d1160 0x7f68f81544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f815dbe0 0x7f68f81fa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f815ee90 0x7f68f81faa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80d33f0 con 0x7f68f815ee90 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80d3270 con 0x7f68f80d1160 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68ffa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80d3570 con 0x7f68f815dbe0 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68fda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f815ee90 0x7f68f81faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68fda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f815ee90 0x7f68f81faa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44782/0 (socket says 172.21.15.87:44782) 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68fda00640 1 -- 172.21.15.87:0/406122297 learned_addr learned my addr 172.21.15.87:0/406122297 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.496+0000 7f68fd000640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80d1160 0x7f68f81544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68f7e00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f815dbe0 0x7f68f81fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fda00640 1 -- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f815dbe0 msgr2=0x7f68f81fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fda00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f815dbe0 0x7f68f81fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fda00640 1 -- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80d1160 msgr2=0x7f68f81544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fda00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80d1160 0x7f68f81544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fda00640 1 -- 172.21.15.87:0/406122297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8154fb0 con 0x7f68f815ee90 2024-04-25T10:18:31.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fd000640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80d1160 0x7f68f81544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:31.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fda00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f815ee90 0x7f68f81faa40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f68f001a560 tx=0x7f68f00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68f4c00640 1 -- 172.21.15.87:0/406122297 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00b2050 con 0x7f68f815ee90 2024-04-25T10:18:31.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68f4c00640 1 -- 172.21.15.87:0/406122297 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f00be070 con 0x7f68f815ee90 2024-04-25T10:18:31.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f8155290 con 0x7f68f815ee90 2024-04-25T10:18:31.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68f4c00640 1 -- 172.21.15.87:0/406122297 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00ad850 con 0x7f68f815ee90 2024-04-25T10:18:31.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f81571d0 con 0x7f68f815ee90 2024-04-25T10:18:31.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f8150550 con 0x7f68f815ee90 2024-04-25T10:18:31.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68f4c00640 1 -- 172.21.15.87:0/406122297 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68f00ad9f0 con 0x7f68f815ee90 2024-04-25T10:18:31.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68f4c00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68f4c00640 1 -- 172.21.15.87:0/406122297 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f68f011fea0 con 0x7f68f815ee90 2024-04-25T10:18:31.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.500+0000 7f68fd000640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.504+0000 7f68fd000640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d8066f90 0x7f68d8069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009970 tx=0x7f68ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.504+0000 7f68f4c00640 1 -- 172.21.15.87:0/406122297 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68f0016520 con 0x7f68f815ee90 2024-04-25T10:18:31.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68f80d18f0 con 0x7f68f815ee90 2024-04-25T10:18:31.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68f4c00640 1 -- 172.21.15.87:0/406122297 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68f00f1240 con 0x7f68f815ee90 2024-04-25T10:18:31.663 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d8066f90 msgr2=0x7f68d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d8066f90 0x7f68d8069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009970 tx=0x7f68ec008040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f815ee90 msgr2=0x7f68f81faa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f815ee90 0x7f68f81faa40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f68f001a560 tx=0x7f68f00196d0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 shutdown_connections 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f815ee90 0x7f68f81faa40 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f815dbe0 0x7f68f81fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 --2- 172.21.15.87:0/406122297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80d1160 0x7f68f81544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 >> 172.21.15.87:0/406122297 conn(0x7f68f806b090 msgr2=0x7f68f80b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 shutdown_connections 2024-04-25T10:18:31.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.660+0000 7f68ffa00640 1 -- 172.21.15.87:0/406122297 wait complete. 2024-04-25T10:18:31.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:18:31.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 Processor -- start 2024-04-25T10:18:31.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 -- start start 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b541550b0 0x7f7b540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7b540cfe70 0x7f7b5415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b5406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b54071050 con 0x7f7b541550b0 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540712f0 con 0x7f7b540cfe70 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b59200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540715d0 con 0x7f7b5415b740 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b541550b0 0x7f7b540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b541550b0 0x7f7b540cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44790/0 (socket says 172.21.15.87:44790) 2024-04-25T10:18:31.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 -- 172.21.15.87:0/3290789504 learned_addr learned my addr 172.21.15.87:0/3290789504 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b52800640 1 --2- 172.21.15.87:0/3290789504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b5406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 -- 172.21.15.87:0/3290789504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 msgr2=0x7f7b5406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51400640 1 --2- 172.21.15.87:0/3290789504 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cfe70 0x7f7b5415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 --2- 172.21.15.87:0/3290789504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b5406d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 -- 172.21.15.87:0/3290789504 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cfe70 msgr2=0x7f7b5415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 --2- 172.21.15.87:0/3290789504 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cfe70 0x7f7b5415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51e00640 1 -- 172.21.15.87:0/3290789504 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540718b0 con 0x7f7b541550b0 2024-04-25T10:18:31.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.816+0000 7f7b51400640 1 --2- 172.21.15.87:0/3290789504 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cfe70 0x7f7b5415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b51e00640 1 --2- 172.21.15.87:0/3290789504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541550b0 0x7f7b540cf930 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009870 tx=0x7f7b3c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c0852f21d19e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b50a00640 1 -- 172.21.15.87:0/3290789504 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0b0070 con 0x7f7b541550b0 2024-04-25T10:18:31.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b50a00640 1 -- 172.21.15.87:0/3290789504 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c0ab600 con 0x7f7b541550b0 2024-04-25T10:18:31.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b50a00640 1 -- 172.21.15.87:0/3290789504 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0ab9d0 con 0x7f7b541550b0 2024-04-25T10:18:31.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 -- 172.21.15.87:0/3290789504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541550b0 msgr2=0x7f7b540cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 --2- 172.21.15.87:0/3290789504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541550b0 0x7f7b540cf930 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009870 tx=0x7f7b3c01c260 comp rx=0 tx=0).stop 2024-04-25T10:18:31.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 -- 172.21.15.87:0/3290789504 shutdown_connections 2024-04-25T10:18:31.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 --2- 172.21.15.87:0/3290789504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b5406d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 --2- 172.21.15.87:0/3290789504 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cfe70 0x7f7b5415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 --2- 172.21.15.87:0/3290789504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541550b0 0x7f7b540cf930 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 -- 172.21.15.87:0/3290789504 >> 172.21.15.87:0/3290789504 conn(0x7f7b54066b70 msgr2=0x7f7b54066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 -- 172.21.15.87:0/3290789504 shutdown_connections 2024-04-25T10:18:31.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.820+0000 7f7b59200640 1 -- 172.21.15.87:0/3290789504 wait complete. 2024-04-25T10:18:31.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 Processor -- start 2024-04-25T10:18:31.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 -- start start 2024-04-25T10:18:31.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b540cfe70 0x7f7b54158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541550b0 0x7f7b5415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b54149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540b3260 con 0x7f7b540cfe70 2024-04-25T10:18:31.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5415bd60 con 0x7f7b541550b0 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540b33e0 con 0x7f7b5415b740 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b540cfe70 0x7f7b54158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b540cfe70 0x7f7b54158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44792/0 (socket says 172.21.15.87:44792) 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 -- 172.21.15.87:0/757505274 learned_addr learned my addr 172.21.15.87:0/757505274 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51400640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541550b0 0x7f7b5415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b52800640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b54149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 -- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 msgr2=0x7f7b54149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b54149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 -- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541550b0 msgr2=0x7f7b5415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541550b0 0x7f7b5415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 -- 172.21.15.87:0/757505274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5415ae70 con 0x7f7b540cfe70 2024-04-25T10:18:31.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b51e00640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b540cfe70 0x7f7b54158be0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0ae040 tx=0x7f7b3c01aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b4a000640 1 -- 172.21.15.87:0/757505274 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0b0040 con 0x7f7b540cfe70 2024-04-25T10:18:31.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b4a000640 1 -- 172.21.15.87:0/757505274 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c0bc070 con 0x7f7b540cfe70 2024-04-25T10:18:31.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b4a000640 1 -- 172.21.15.87:0/757505274 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0b6880 con 0x7f7b540cfe70 2024-04-25T10:18:31.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b54159330 con 0x7f7b540cfe70 2024-04-25T10:18:31.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.824+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b541598f0 con 0x7f7b540cfe70 2024-04-25T10:18:31.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.828+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5404ef30 con 0x7f7b540cfe70 2024-04-25T10:18:31.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.828+0000 7f7b4a000640 1 -- 172.21.15.87:0/757505274 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b3c0b6a20 con 0x7f7b540cfe70 2024-04-25T10:18:31.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.828+0000 7f7b4a000640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.828+0000 7f7b4a000640 1 -- 172.21.15.87:0/757505274 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7b3c11f2c0 con 0x7f7b540cfe70 2024-04-25T10:18:31.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.828+0000 7f7b51400640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.828+0000 7f7b51400640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b5414ee80 tx=0x7f7b44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:31.832+0000 7f7b4a000640 1 -- 172.21.15.87:0/757505274 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b3c016110 con 0x7f7b540cfe70 2024-04-25T10:18:32.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:32.040+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f7b54154430 con 0x7f7b540cfe70 2024-04-25T10:18:33.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.008+0000 7f7b4a000640 1 -- 172.21.15.87:0/757505274 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v307) v1 ==== 147+0+0 (secure 0 0 0) 0x7f7b3c0f05f0 con 0x7f7b540cfe70 2024-04-25T10:18:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 msgr2=0x7f7b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b5414ee80 tx=0x7f7b44008040 comp rx=0 tx=0).stop 2024-04-25T10:18:33.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b540cfe70 msgr2=0x7f7b54158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b540cfe70 0x7f7b54158be0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0ae040 tx=0x7f7b3c01aaa0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 shutdown_connections 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b5415b740 0x7f7b54149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541550b0 0x7f7b5415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 --2- 172.21.15.87:0/757505274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b540cfe70 0x7f7b54158be0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 >> 172.21.15.87:0/757505274 conn(0x7f7b54066b70 msgr2=0x7f7b540afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 shutdown_connections 2024-04-25T10:18:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.012+0000 7f7b59200640 1 -- 172.21.15.87:0/757505274 wait complete. 2024-04-25T10:18:33.037 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:18:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.176+0000 7f738bc00640 1 Processor -- start 2024-04-25T10:18:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.176+0000 7f738bc00640 1 -- start start 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.176+0000 7f738bc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73840aef80 0x7f738415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.176+0000 7f738bc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f73840b31c0 0x7f73840b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.176+0000 7f738bc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738415b080 con 0x7f7384070350 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.176+0000 7f738bc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738415adb0 con 0x7f73840aef80 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.176+0000 7f738bc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738415eb60 con 0x7f73840b31c0 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73840aef80 0x7f738415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73840aef80 0x7f738415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44796/0 (socket says 172.21.15.87:44796) 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 -- 172.21.15.87:0/1355144974 learned_addr learned my addr 172.21.15.87:0/1355144974 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:33.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389c00640 1 --2- 172.21.15.87:0/1355144974 >> v2:172.21.15.169:3300/0 conn(0x7f73840b31c0 0x7f73840b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 -- 172.21.15.87:0/1355144974 >> v2:172.21.15.169:3300/0 conn(0x7f73840b31c0 msgr2=0x7f73840b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 --2- 172.21.15.87:0/1355144974 >> v2:172.21.15.169:3300/0 conn(0x7f73840b31c0 0x7f73840b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 -- 172.21.15.87:0/1355144974 >> v1:172.21.15.169:6789/0 conn(0x7f7384070350 legacy=0x7f738415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 -- 172.21.15.87:0/1355144974 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738415ee40 con 0x7f73840aef80 2024-04-25T10:18:33.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7389200640 1 --2- 172.21.15.87:0/1355144974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f738415e480 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f73840d0750 tx=0x7f737401ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed3056fc4ca406c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7383400640 1 -- 172.21.15.87:0/1355144974 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740b4070 con 0x7f73840aef80 2024-04-25T10:18:33.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7383400640 1 -- 172.21.15.87:0/1355144974 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73740a7b90 con 0x7f73840aef80 2024-04-25T10:18:33.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.180+0000 7f7383400640 1 -- 172.21.15.87:0/1355144974 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740af440 con 0x7f73840aef80 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- 172.21.15.87:0/1355144974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 msgr2=0x7f738415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 --2- 172.21.15.87:0/1355144974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f738415e480 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f73840d0750 tx=0x7f737401ebb0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- 172.21.15.87:0/1355144974 shutdown_connections 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 --2- 172.21.15.87:0/1355144974 >> v2:172.21.15.169:3300/0 conn(0x7f73840b31c0 0x7f73840b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 --2- 172.21.15.87:0/1355144974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f738415e480 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- 172.21.15.87:0/1355144974 >> 172.21.15.87:0/1355144974 conn(0x7f73840669a0 msgr2=0x7f7384066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- 172.21.15.87:0/1355144974 shutdown_connections 2024-04-25T10:18:33.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- 172.21.15.87:0/1355144974 wait complete. 2024-04-25T10:18:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 Processor -- start 2024-04-25T10:18:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- start start 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7384070350 0x7f73841420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f7384159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73840b31c0 0x7f73841572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738406c250 con 0x7f73840aef80 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738406c0d0 con 0x7f73840b31c0 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f738bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738406c3d0 con 0x7f7384070350 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f7384159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f7384159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44798/0 (socket says 172.21.15.87:44798) 2024-04-25T10:18:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 -- 172.21.15.87:0/4068112418 learned_addr learned my addr 172.21.15.87:0/4068112418 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7389c00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73840b31c0 0x7f73841572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7389200640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7384070350 0x7f73841420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 -- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7384070350 msgr2=0x7f73841420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7384070350 0x7f73841420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 -- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73840b31c0 msgr2=0x7f73841572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73840b31c0 0x7f73841572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7383e00640 1 -- 172.21.15.87:0/4068112418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384159bb0 con 0x7f73840aef80 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.184+0000 7f7389200640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7384070350 0x7f73841420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7383e00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f7384159260 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c480 tx=0x7f736c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7380c00640 1 -- 172.21.15.87:0/4068112418 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c012830 con 0x7f73840aef80 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7380c00640 1 -- 172.21.15.87:0/4068112418 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c017070 con 0x7f73840aef80 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7380c00640 1 -- 172.21.15.87:0/4068112418 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c012c30 con 0x7f73840aef80 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7384159e90 con 0x7f73840aef80 2024-04-25T10:18:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738415a450 con 0x7f73840aef80 2024-04-25T10:18:33.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738404ef30 con 0x7f73840aef80 2024-04-25T10:18:33.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7380c00640 1 -- 172.21.15.87:0/4068112418 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f736c01c460 con 0x7f73840aef80 2024-04-25T10:18:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7380c00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7360066f60 0x7f7360069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7380c00640 1 -- 172.21.15.87:0/4068112418 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f736c084b10 con 0x7f73840aef80 2024-04-25T10:18:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.188+0000 7f7389200640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7360066f60 0x7f7360069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.192+0000 7f7389200640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7360066f60 0x7f7360069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f73740b1000 tx=0x7f7374008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.192+0000 7f7380c00640 1 -- 172.21.15.87:0/4068112418 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f736c055e10 con 0x7f73840aef80 2024-04-25T10:18:33.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.348+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f73840b2fa0 con 0x7f73840aef80 2024-04-25T10:18:33.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.352+0000 7f7380c00640 1 -- 172.21.15.87:0/4068112418 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f736c055710 con 0x7f73840aef80 2024-04-25T10:18:33.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7360066f60 msgr2=0x7f7360069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7360066f60 0x7f7360069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f73740b1000 tx=0x7f7374008040 comp rx=0 tx=0).stop 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 msgr2=0x7f7384159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f7384159260 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c480 tx=0x7f736c00c940 comp rx=0 tx=0).stop 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 shutdown_connections 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7360066f60 0x7f7360069420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73840b31c0 0x7f73841572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73840aef80 0x7f7384159260 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 --2- 172.21.15.87:0/4068112418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7384070350 0x7f73841420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 >> 172.21.15.87:0/4068112418 conn(0x7f73840669a0 msgr2=0x7f7384152c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 shutdown_connections 2024-04-25T10:18:33.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.356+0000 7f738bc00640 1 -- 172.21.15.87:0/4068112418 wait complete. 2024-04-25T10:18:33.379 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:18:33.379 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:33.459 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:33.560 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:18:33.560 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:18:33.560 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:33.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:33.568 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:18:33.568 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:33.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:33.578 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:18:33.578 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:33.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:33.617 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:18:33.617 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:33.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:33.633 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:18:33.633 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:33.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.796+0000 7f844e800640 1 Processor -- start 2024-04-25T10:18:33.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.796+0000 7f844e800640 1 -- start start 2024-04-25T10:18:33.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f844e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844815bc40 0x7f844806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f844e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84480cda10 0x7f84480b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f844e800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84480cf930 0x7f84480cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f844e800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84480b2770 con 0x7f84480cda10 2024-04-25T10:18:33.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f844e800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844806c670 con 0x7f84480cf930 2024-04-25T10:18:33.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f844e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844806c950 con 0x7f844815bc40 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84480cda10 0x7f84480b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84480cda10 0x7f84480b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44812/0 (socket says 172.21.15.87:44812) 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 -- 172.21.15.87:0/4126419478 learned_addr learned my addr 172.21.15.87:0/4126419478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f8447e00640 1 --2- 172.21.15.87:0/4126419478 >> v2:172.21.15.169:3300/0 conn(0x7f84480cf930 0x7f84480cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f8447400640 1 --2- 172.21.15.87:0/4126419478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844815bc40 0x7f844806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 -- 172.21.15.87:0/4126419478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844815bc40 msgr2=0x7f844806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 --2- 172.21.15.87:0/4126419478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844815bc40 0x7f844806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 -- 172.21.15.87:0/4126419478 >> v2:172.21.15.169:3300/0 conn(0x7f84480cf930 msgr2=0x7f84480cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 --2- 172.21.15.87:0/4126419478 >> v2:172.21.15.169:3300/0 conn(0x7f84480cf930 0x7f84480cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 -- 172.21.15.87:0/4126419478 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844806cc30 con 0x7f84480cda10 2024-04-25T10:18:33.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f843ea00640 1 --2- 172.21.15.87:0/4126419478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84480cda10 0x7f84480b4610 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f84480b3f80 tx=0x7f843001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34098fd76f5f9378 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f8446a00640 1 -- 172.21.15.87:0/4126419478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300b0070 con 0x7f84480cda10 2024-04-25T10:18:33.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f8446a00640 1 -- 172.21.15.87:0/4126419478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84300ab600 con 0x7f84480cda10 2024-04-25T10:18:33.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.800+0000 7f8446a00640 1 -- 172.21.15.87:0/4126419478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300ab9d0 con 0x7f84480cda10 2024-04-25T10:18:33.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- 172.21.15.87:0/4126419478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84480cda10 msgr2=0x7f84480b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 --2- 172.21.15.87:0/4126419478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84480cda10 0x7f84480b4610 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f84480b3f80 tx=0x7f843001c520 comp rx=0 tx=0).stop 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- 172.21.15.87:0/4126419478 shutdown_connections 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 --2- 172.21.15.87:0/4126419478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84480cda10 0x7f84480b4610 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 --2- 172.21.15.87:0/4126419478 >> v2:172.21.15.169:3300/0 conn(0x7f84480cf930 0x7f84480cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 --2- 172.21.15.87:0/4126419478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844815bc40 0x7f844806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- 172.21.15.87:0/4126419478 >> 172.21.15.87:0/4126419478 conn(0x7f8448066b70 msgr2=0x7f8448066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- 172.21.15.87:0/4126419478 shutdown_connections 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- 172.21.15.87:0/4126419478 wait complete. 2024-04-25T10:18:33.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 Processor -- start 2024-04-25T10:18:33.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- start start 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84480cda10 0x7f8448157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84480cf930 0x7f8448146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844815bc40 0x7f844815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8448160230 con 0x7f844815bc40 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481600b0 con 0x7f84480cf930 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f844e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84481603b0 con 0x7f84480cda10 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844815bc40 0x7f844815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844815bc40 0x7f844815cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44826/0 (socket says 172.21.15.87:44826) 2024-04-25T10:18:33.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 -- 172.21.15.87:0/351391755 learned_addr learned my addr 172.21.15.87:0/351391755 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447400640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84480cda10 0x7f8448157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f843ea00640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84480cf930 0x7f8448146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 -- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84480cda10 msgr2=0x7f8448157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84480cda10 0x7f8448157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 -- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84480cf930 msgr2=0x7f8448146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84480cf930 0x7f8448146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447e00640 1 -- 172.21.15.87:0/351391755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844815ebf0 con 0x7f844815bc40 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f8447400640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84480cda10 0x7f8448157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.804+0000 7f843ea00640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84480cf930 0x7f8448146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f8447e00640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844815bc40 0x7f844815cb50 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8440007d20 tx=0x7f844000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f843fe00640 1 -- 172.21.15.87:0/351391755 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440014420 con 0x7f844815bc40 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f843fe00640 1 -- 172.21.15.87:0/351391755 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440019070 con 0x7f844815bc40 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f843fe00640 1 -- 172.21.15.87:0/351391755 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440014820 con 0x7f844815bc40 2024-04-25T10:18:33.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844815eed0 con 0x7f844815bc40 2024-04-25T10:18:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8448157050 con 0x7f844815bc40 2024-04-25T10:18:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8448156b80 con 0x7f844815bc40 2024-04-25T10:18:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f843fe00640 1 -- 172.21.15.87:0/351391755 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84400149c0 con 0x7f844815bc40 2024-04-25T10:18:33.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f843fe00640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8424066f60 0x7f8424069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f843fe00640 1 -- 172.21.15.87:0/351391755 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8440085550 con 0x7f844815bc40 2024-04-25T10:18:33.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f8447400640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8424066f60 0x7f8424069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.808+0000 7f8447400640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8424066f60 0x7f8424069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8438007a70 tx=0x7f8438008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:33.812+0000 7f843fe00640 1 -- 172.21.15.87:0/351391755 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8440056850 con 0x7f844815bc40 2024-04-25T10:18:34.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f844804ef30 con 0x7f844815bc40 2024-04-25T10:18:34.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f843fe00640 1 -- 172.21.15.87:0/351391755 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f8440056150 con 0x7f844815bc40 2024-04-25T10:18:34.003 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:34.003 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/817977905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":817977905},{"type":"v1","addr":"172.21.15.169:6835","nonce":817977905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3943072134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3943072134},{"type":"v1","addr":"172.21.15.169:6834","nonce":3943072134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/979883306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":979883306},{"type":"v1","addr":"172.21.15.87:6836","nonce":979883306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/796485952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":796485952},{"type":"v1","addr":"172.21.15.87:6837","nonce":796485952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[]} 2024-04-25T10:18:34.003 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 56 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8424066f60 msgr2=0x7f8424069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8424066f60 0x7f8424069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8438007a70 tx=0x7f8438008040 comp rx=0 tx=0).stop 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844815bc40 msgr2=0x7f844815cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844815bc40 0x7f844815cb50 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8440007d20 tx=0x7f844000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 shutdown_connections 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8424066f60 0x7f8424069420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844815bc40 0x7f844815cb50 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84480cf930 0x7f8448146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 --2- 172.21.15.87:0/351391755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84480cda10 0x7f8448157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 >> 172.21.15.87:0/351391755 conn(0x7f8448066b70 msgr2=0x7f844815a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 shutdown_connections 2024-04-25T10:18:34.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.000+0000 7f844e800640 1 -- 172.21.15.87:0/351391755 wait complete. 2024-04-25T10:18:34.025 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:18:34.025 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:18:34.025 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:18:34.026 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:34.026 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:18:34.026 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:18:34.026 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:18:34.026 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:34.026 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:18:34.026 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:18:34.027 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:18:34.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:34.027 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:18:34.027 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:18:34.027 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:18:34.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:34.038 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:18:34.038 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:18:34.039 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:18:34.039 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:18:34.041 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:18:34.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:18:34.042 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:18:34.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:18:34.084 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:18:34.084 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:34.086 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:18:34.086 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:34.088 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:18:34.088 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:34.089 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:18:34.089 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:34.090 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:18:34.171 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:18:34.171 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:18:34.179 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:18:34.180 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:18:34.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2693200640 1 Processor -- start 2024-04-25T10:18:34.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2693200640 1 -- start start 2024-04-25T10:18:34.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2693200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f269415ddc0 0x7f26940b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2693200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26940b37e0 con 0x7f26940ccf10 2024-04-25T10:18:34.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2693200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26940b3a60 con 0x7f2694162290 2024-04-25T10:18:34.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2693200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269415f1b0 con 0x7f269415ddc0 2024-04-25T10:18:34.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2692800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f26940ccf10 0x7f269406c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47288/0 (socket says 172.21.15.87:47288) 2024-04-25T10:18:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2692800640 1 -- 172.21.15.87:0/1912570165 learned_addr learned my addr 172.21.15.87:0/1912570165 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2691e00640 1 --2- 172.21.15.87:0/1912570165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f269415ddc0 0x7f26940b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2690a00640 1 -- 172.21.15.87:0/1912570165 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288042668 0 0) 0x7f268c0014a0 con 0x7f26940ccf10 2024-04-25T10:18:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2690a00640 1 -- 172.21.15.87:0/1912570165 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2678003530 con 0x7f26940ccf10 2024-04-25T10:18:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2690a00640 1 -- 172.21.15.87:0/1912570165 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641664786 0 0) 0x7f26940b3a60 con 0x7f2694162290 2024-04-25T10:18:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2690a00640 1 -- 172.21.15.87:0/1912570165 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f268c0014a0 con 0x7f2694162290 2024-04-25T10:18:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2691e00640 1 -- 172.21.15.87:0/1912570165 >> v1:172.21.15.169:6789/0 conn(0x7f2694162290 legacy=0x7f26940cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2691e00640 1 -- 172.21.15.87:0/1912570165 >> v1:172.21.15.87:6789/0 conn(0x7f26940ccf10 legacy=0x7f269406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.232+0000 7f2691e00640 1 -- 172.21.15.87:0/1912570165 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940b3d40 con 0x7f269415ddc0 2024-04-25T10:18:34.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2691e00640 1 --2- 172.21.15.87:0/1912570165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f269415ddc0 0x7f26940b4820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26840097f0 tx=0x7f269406d680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95f7ecd1b7dc5014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2690a00640 1 -- 172.21.15.87:0/1912570165 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840b1070 con 0x7f269415ddc0 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2690a00640 1 -- 172.21.15.87:0/1912570165 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26840ac630 con 0x7f269415ddc0 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2690a00640 1 -- 172.21.15.87:0/1912570165 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840a8050 con 0x7f269415ddc0 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2693200640 1 -- 172.21.15.87:0/1912570165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f269415ddc0 msgr2=0x7f26940b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2693200640 1 --2- 172.21.15.87:0/1912570165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f269415ddc0 0x7f26940b4820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26840097f0 tx=0x7f269406d680 comp rx=0 tx=0).stop 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2693200640 1 -- 172.21.15.87:0/1912570165 shutdown_connections 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2693200640 1 --2- 172.21.15.87:0/1912570165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f269415ddc0 0x7f26940b4820 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2693200640 1 -- 172.21.15.87:0/1912570165 >> 172.21.15.87:0/1912570165 conn(0x7f26940b1a20 msgr2=0x7f26940b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2693200640 1 -- 172.21.15.87:0/1912570165 shutdown_connections 2024-04-25T10:18:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.236+0000 7f2693200640 1 -- 172.21.15.87:0/1912570165 wait complete. 2024-04-25T10:18:34.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 Processor -- start 2024-04-25T10:18:34.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 -- start start 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26940ccf10 0x7f2694160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f269415ddc0 0x7f2694152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2694162290 0x7f269415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269416bb80 con 0x7f2694162290 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269416ba00 con 0x7f269415ddc0 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269416bd00 con 0x7f26940ccf10 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2694162290 0x7f269415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2694162290 0x7f269415bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44874/0 (socket says 172.21.15.87:44874) 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 -- 172.21.15.87:0/3127156266 learned_addr learned my addr 172.21.15.87:0/3127156266 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2691e00640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26940ccf10 0x7f2694160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2691400640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f269415ddc0 0x7f2694152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 -- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26940ccf10 msgr2=0x7f2694160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26940ccf10 0x7f2694160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 -- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f269415ddc0 msgr2=0x7f2694152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f269415ddc0 0x7f2694152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 -- 172.21.15.87:0/3127156266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694160450 con 0x7f2694162290 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2692800640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2694162290 0x7f269415bf30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f268c001f80 tx=0x7f268c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f268a000640 1 -- 172.21.15.87:0/3127156266 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c017070 con 0x7f2694162290 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f268a000640 1 -- 172.21.15.87:0/3127156266 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c012400 con 0x7f2694162290 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2694155f00 con 0x7f2694162290 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f268a000640 1 -- 172.21.15.87:0/3127156266 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c012830 con 0x7f2694162290 2024-04-25T10:18:34.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.240+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26941606f0 con 0x7f2694162290 2024-04-25T10:18:34.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.244+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694075120 con 0x7f2694162290 2024-04-25T10:18:34.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.244+0000 7f268a000640 1 -- 172.21.15.87:0/3127156266 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f268c01a070 con 0x7f2694162290 2024-04-25T10:18:34.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.244+0000 7f268a000640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2674066f60 0x7f2674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.244+0000 7f268a000640 1 -- 172.21.15.87:0/3127156266 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f268c084d10 con 0x7f2694162290 2024-04-25T10:18:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.244+0000 7f2691e00640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2674066f60 0x7f2674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.244+0000 7f2691e00640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2674066f60 0x7f2674069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f26840ae000 tx=0x7f2684004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.248+0000 7f268a000640 1 -- 172.21.15.87:0/3127156266 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f268c08a020 con 0x7f2694162290 2024-04-25T10:18:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:34.412+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2694074a80 con 0x7f2694162290 2024-04-25T10:18:35.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f268a000640 1 -- 172.21.15.87:0/3127156266 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7f26940b37e0 con 0x7f2694162290 2024-04-25T10:18:35.030 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2674066f60 msgr2=0x7f2674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2674066f60 0x7f2674069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f26840ae000 tx=0x7f2684004740 comp rx=0 tx=0).stop 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2694162290 msgr2=0x7f269415bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2694162290 0x7f269415bf30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f268c001f80 tx=0x7f268c00a430 comp rx=0 tx=0).stop 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 shutdown_connections 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2674066f60 0x7f2674069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2694162290 0x7f269415bf30 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f269415ddc0 0x7f2694152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 --2- 172.21.15.87:0/3127156266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26940ccf10 0x7f2694160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 >> 172.21.15.87:0/3127156266 conn(0x7f26940b1a20 msgr2=0x7f26941617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 shutdown_connections 2024-04-25T10:18:35.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.028+0000 7f2693200640 1 -- 172.21.15.87:0/3127156266 wait complete. 2024-04-25T10:18:35.055 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:18:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357f800640 1 Processor -- start 2024-04-25T10:18:35.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357f800640 1 -- start start 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357f800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f35780d0040 0x7f357815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f35780b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357f800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3578071050 con 0x7f35781550b0 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357f800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357815d040 con 0x7f35780d0040 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35780712f0 con 0x7f357815b740 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f356fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f35780d0040 0x7f357815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f356fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f35780d0040 0x7f357815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44876/0 (socket says 172.21.15.87:44876) 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f356fe00640 1 -- 172.21.15.87:0/697596111 learned_addr learned my addr 172.21.15.87:0/697596111 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:35.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f357d800640 1 --2- 172.21.15.87:0/697596111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f35780b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f356fe00640 1 -- 172.21.15.87:0/697596111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 msgr2=0x7f35780b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f356fe00640 1 --2- 172.21.15.87:0/697596111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f35780b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f356fe00640 1 -- 172.21.15.87:0/697596111 >> v1:172.21.15.169:6789/0 conn(0x7f35781550b0 legacy=0x7f35780cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.188+0000 7f356fe00640 1 -- 172.21.15.87:0/697596111 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780715d0 con 0x7f35780d0040 2024-04-25T10:18:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f356fe00640 1 --2- 172.21.15.87:0/697596111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35780d0040 0x7f357815b200 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3560009870 tx=0x7f356001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfc116fddcac05a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f3577e00640 1 -- 172.21.15.87:0/697596111 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600a95f0 con 0x7f35780d0040 2024-04-25T10:18:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f3577e00640 1 -- 172.21.15.87:0/697596111 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35600a9790 con 0x7f35780d0040 2024-04-25T10:18:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f3577e00640 1 -- 172.21.15.87:0/697596111 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600a9b60 con 0x7f35780d0040 2024-04-25T10:18:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 -- 172.21.15.87:0/697596111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35780d0040 msgr2=0x7f357815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 --2- 172.21.15.87:0/697596111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35780d0040 0x7f357815b200 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3560009870 tx=0x7f356001c530 comp rx=0 tx=0).stop 2024-04-25T10:18:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 -- 172.21.15.87:0/697596111 shutdown_connections 2024-04-25T10:18:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 --2- 172.21.15.87:0/697596111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f35780b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 --2- 172.21.15.87:0/697596111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35780d0040 0x7f357815b200 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 -- 172.21.15.87:0/697596111 >> 172.21.15.87:0/697596111 conn(0x7f3578066b70 msgr2=0x7f3578066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 -- 172.21.15.87:0/697596111 shutdown_connections 2024-04-25T10:18:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.192+0000 7f357f800640 1 -- 172.21.15.87:0/697596111 wait complete. 2024-04-25T10:18:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 Processor -- start 2024-04-25T10:18:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 -- start start 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35780d0040 0x7f357814d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35781550b0 0x7f357814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f357814b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357815bcd0 con 0x7f35781550b0 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357815bb50 con 0x7f35780d0040 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357815c030 con 0x7f357815b740 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35781550b0 0x7f357814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35781550b0 0x7f357814ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44892/0 (socket says 172.21.15.87:44892) 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 -- 172.21.15.87:0/3560751114 learned_addr learned my addr 172.21.15.87:0/3560751114 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357d800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f357814b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357ce00640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35780d0040 0x7f357814d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 -- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 msgr2=0x7f357814b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f357814b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 -- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35780d0040 msgr2=0x7f357814d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35780d0040 0x7f357814d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 -- 172.21.15.87:0/3560751114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578157010 con 0x7f35781550b0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357ce00640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35780d0040 0x7f357814d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357d800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f357814b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f356fe00640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35781550b0 0x7f357814ad40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f35600ab040 tx=0x7f356001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f3575600640 1 -- 172.21.15.87:0/3560751114 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600b1800 con 0x7f35781550b0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f3575600640 1 -- 172.21.15.87:0/3560751114 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35600b19c0 con 0x7f35781550b0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f3575600640 1 -- 172.21.15.87:0/3560751114 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600b1db0 con 0x7f35781550b0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35781572f0 con 0x7f35781550b0 2024-04-25T10:18:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.196+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35781578b0 con 0x7f35781550b0 2024-04-25T10:18:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.200+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357804ef30 con 0x7f35781550b0 2024-04-25T10:18:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.200+0000 7f3575600640 1 -- 172.21.15.87:0/3560751114 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35600a8350 con 0x7f35781550b0 2024-04-25T10:18:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.200+0000 7f3575600640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3554066f60 0x7f3554069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.200+0000 7f3575600640 1 -- 172.21.15.87:0/3560751114 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f356011f1c0 con 0x7f35781550b0 2024-04-25T10:18:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.200+0000 7f357ce00640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3554066f60 0x7f3554069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.200+0000 7f357ce00640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3554066f60 0x7f3554069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f35680016f0 tx=0x7f3568009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.204+0000 7f3575600640 1 -- 172.21.15.87:0/3560751114 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35600c3070 con 0x7f35781550b0 2024-04-25T10:18:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.412+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f357815dd90 con 0x7f35781550b0 2024-04-25T10:18:35.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.412+0000 7f3575600640 1 -- 172.21.15.87:0/3560751114 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f35600ad020 con 0x7f35781550b0 2024-04-25T10:18:35.416 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:35.416 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:18:35.416 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:18:35.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.416 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:18:35.417 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:18:35.418 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:18:35.420 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.422 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.423 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:18:35.425 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.426 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.428 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:35.429 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:18:35.429 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:18:35.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3554066f60 msgr2=0x7f3554069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3554066f60 0x7f3554069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f35680016f0 tx=0x7f3568009040 comp rx=0 tx=0).stop 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35781550b0 msgr2=0x7f357814ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35781550b0 0x7f357814ad40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f35600ab040 tx=0x7f356001ad70 comp rx=0 tx=0).stop 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 shutdown_connections 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3554066f60 0x7f3554069420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357815b740 0x7f357814b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35781550b0 0x7f357814ad40 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 --2- 172.21.15.87:0/3560751114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35780d0040 0x7f357814d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 >> 172.21.15.87:0/3560751114 conn(0x7f3578066b70 msgr2=0x7f35780cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 shutdown_connections 2024-04-25T10:18:35.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.416+0000 7f357f800640 1 -- 172.21.15.87:0/3560751114 wait complete. 2024-04-25T10:18:35.437 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:35.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 Processor -- start 2024-04-25T10:18:35.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 -- start start 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c15bc40 0x7f715c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f715c0cda10 0x7f715c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f715c0cf930 0x7f715c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c0b2770 con 0x7f715c0cda10 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c06c670 con 0x7f715c0cf930 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f7163a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c06c950 con 0x7f715c15bc40 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f715be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f715c0cda10 0x7f715c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f715be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f715c0cda10 0x7f715c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44906/0 (socket says 172.21.15.87:44906) 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.568+0000 7f715be00640 1 -- 172.21.15.87:0/307889195 learned_addr learned my addr 172.21.15.87:0/307889195 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f7161000640 1 --2- 172.21.15.87:0/307889195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c15bc40 0x7f715c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715be00640 1 -- 172.21.15.87:0/307889195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c15bc40 msgr2=0x7f715c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715be00640 1 --2- 172.21.15.87:0/307889195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c15bc40 0x7f715c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715be00640 1 -- 172.21.15.87:0/307889195 >> v2:172.21.15.169:3300/0 conn(0x7f715c0cf930 msgr2=0x7f715c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f7161a00640 1 --2- 172.21.15.87:0/307889195 >> v2:172.21.15.169:3300/0 conn(0x7f715c0cf930 0x7f715c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715be00640 1 --2- 172.21.15.87:0/307889195 >> v2:172.21.15.169:3300/0 conn(0x7f715c0cf930 0x7f715c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715be00640 1 -- 172.21.15.87:0/307889195 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c06cc30 con 0x7f715c0cda10 2024-04-25T10:18:35.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f7161a00640 1 --2- 172.21.15.87:0/307889195 >> v2:172.21.15.169:3300/0 conn(0x7f715c0cf930 0x7f715c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715be00640 1 --2- 172.21.15.87:0/307889195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c0cda10 0x7f715c0b4610 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f715c0b3f80 tx=0x7f714c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5577a92370256d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715b400640 1 -- 172.21.15.87:0/307889195 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0b0070 con 0x7f715c0cda10 2024-04-25T10:18:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715b400640 1 -- 172.21.15.87:0/307889195 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c0ab600 con 0x7f715c0cda10 2024-04-25T10:18:35.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.572+0000 7f715b400640 1 -- 172.21.15.87:0/307889195 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0ab9d0 con 0x7f715c0cda10 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- 172.21.15.87:0/307889195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c0cda10 msgr2=0x7f715c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 --2- 172.21.15.87:0/307889195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c0cda10 0x7f715c0b4610 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f715c0b3f80 tx=0x7f714c01c520 comp rx=0 tx=0).stop 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- 172.21.15.87:0/307889195 shutdown_connections 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 --2- 172.21.15.87:0/307889195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c0cda10 0x7f715c0b4610 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 --2- 172.21.15.87:0/307889195 >> v2:172.21.15.169:3300/0 conn(0x7f715c0cf930 0x7f715c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 --2- 172.21.15.87:0/307889195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c15bc40 0x7f715c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- 172.21.15.87:0/307889195 >> 172.21.15.87:0/307889195 conn(0x7f715c066b70 msgr2=0x7f715c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- 172.21.15.87:0/307889195 shutdown_connections 2024-04-25T10:18:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- 172.21.15.87:0/307889195 wait complete. 2024-04-25T10:18:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 Processor -- start 2024-04-25T10:18:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- start start 2024-04-25T10:18:35.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c0cda10 0x7f715c147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f715c0cf930 0x7f715c155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c15bc40 0x7f715c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c160230 con 0x7f715c15bc40 2024-04-25T10:18:35.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1600b0 con 0x7f715c0cf930 2024-04-25T10:18:35.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7163a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1603b0 con 0x7f715c0cda10 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c15bc40 0x7f715c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f715be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f715c0cf930 0x7f715c155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c15bc40 0x7f715c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44914/0 (socket says 172.21.15.87:44914) 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f715be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f715c0cf930 0x7f715c155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:60092/0 (socket says 172.21.15.87:60092) 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 -- 172.21.15.87:0/1835481083 learned_addr learned my addr 172.21.15.87:0/1835481083 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161000640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c0cda10 0x7f715c147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 -- 172.21.15.87:0/1835481083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c0cda10 msgr2=0x7f715c147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c0cda10 0x7f715c147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 -- 172.21.15.87:0/1835481083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f715c0cf930 msgr2=0x7f715c155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f715c0cf930 0x7f715c155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.576+0000 7f7161a00640 1 -- 172.21.15.87:0/1835481083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c147760 con 0x7f715c15bc40 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7161a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c15bc40 0x7f715c0af3e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f715400cd10 tx=0x7f715400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7158c00640 1 -- 172.21.15.87:0/1835481083 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400f680 con 0x7f715c15bc40 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7158c00640 1 -- 172.21.15.87:0/1835481083 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400f840 con 0x7f715c15bc40 2024-04-25T10:18:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7158c00640 1 -- 172.21.15.87:0/1835481083 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400fc30 con 0x7f715c15bc40 2024-04-25T10:18:35.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c0afd80 con 0x7f715c15bc40 2024-04-25T10:18:35.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c0707d0 con 0x7f715c15bc40 2024-04-25T10:18:35.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c070310 con 0x7f715c15bc40 2024-04-25T10:18:35.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7158c00640 1 -- 172.21.15.87:0/1835481083 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71540213f0 con 0x7f715c15bc40 2024-04-25T10:18:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7158c00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7138066f60 0x7f7138069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7158c00640 1 -- 172.21.15.87:0/1835481083 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f71540858b0 con 0x7f715c15bc40 2024-04-25T10:18:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.580+0000 7f7161000640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7138066f60 0x7f7138069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.584+0000 7f7161000640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7138066f60 0x7f7138069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7144007a60 tx=0x7f7144008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.584+0000 7f7158c00640 1 -- 172.21.15.87:0/1835481083 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f715c070310 con 0x7f715c15bc40 2024-04-25T10:18:35.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f715c0af1c0 con 0x7f715c15bc40 2024-04-25T10:18:35.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7158c00640 1 -- 172.21.15.87:0/1835481083 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+12880 (secure 0 0 0) 0x7f7154014030 con 0x7f715c15bc40 2024-04-25T10:18:35.743 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:35.743 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":83,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:18:35.039401+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7138066f60 msgr2=0x7f7138069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7138066f60 0x7f7138069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7144007a60 tx=0x7f7144008040 comp rx=0 tx=0).stop 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c15bc40 msgr2=0x7f715c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c15bc40 0x7f715c0af3e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f715400cd10 tx=0x7f715400a650 comp rx=0 tx=0).stop 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 shutdown_connections 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7138066f60 0x7f7138069420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c15bc40 0x7f715c0af3e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.740+0000 7f7163a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f715c0cf930 0x7f715c155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.744+0000 7f7163a00640 1 --2- 172.21.15.87:0/1835481083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f715c0cda10 0x7f715c147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.744+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 >> 172.21.15.87:0/1835481083 conn(0x7f715c066b70 msgr2=0x7f715c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.744+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 shutdown_connections 2024-04-25T10:18:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.744+0000 7f7163a00640 1 -- 172.21.15.87:0/1835481083 wait complete. 2024-04-25T10:18:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:35.765 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:35.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb6ba00640 1 Processor -- start 2024-04-25T10:18:35.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb6ba00640 1 -- start start 2024-04-25T10:18:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb6ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb64070350 0x7fbb6415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb6ba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb640b31c0 0x7fbb640b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb6ba00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6415b080 con 0x7fbb640aef80 2024-04-25T10:18:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb6ba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6415adb0 con 0x7fbb640b31c0 2024-04-25T10:18:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb6ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6415eb60 con 0x7fbb64070350 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb69a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb640b31c0 0x7fbb640b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb69a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb640b31c0 0x7fbb640b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44918/0 (socket says 172.21.15.87:44918) 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb69a00640 1 -- 172.21.15.87:0/996880159 learned_addr learned my addr 172.21.15.87:0/996880159 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb63e00640 1 --2- 172.21.15.87:0/996880159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb64070350 0x7fbb6415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb69a00640 1 -- 172.21.15.87:0/996880159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb64070350 msgr2=0x7fbb6415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb69a00640 1 --2- 172.21.15.87:0/996880159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb64070350 0x7fbb6415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb69a00640 1 -- 172.21.15.87:0/996880159 >> v1:172.21.15.169:6789/0 conn(0x7fbb640aef80 legacy=0x7fbb6415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.900+0000 7fbb69a00640 1 -- 172.21.15.87:0/996880159 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6415ee40 con 0x7fbb640b31c0 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb69a00640 1 --2- 172.21.15.87:0/996880159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb640b31c0 0x7fbb640b3610 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009e50 tx=0x7fbb5c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fe120656acfb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb63400640 1 -- 172.21.15.87:0/996880159 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0a5810 con 0x7fbb640b31c0 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb63400640 1 -- 172.21.15.87:0/996880159 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c0a59b0 con 0x7fbb640b31c0 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb63400640 1 -- 172.21.15.87:0/996880159 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0a5d80 con 0x7fbb640b31c0 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 -- 172.21.15.87:0/996880159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb640b31c0 msgr2=0x7fbb640b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/996880159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb640b31c0 0x7fbb640b3610 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009e50 tx=0x7fbb5c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 -- 172.21.15.87:0/996880159 shutdown_connections 2024-04-25T10:18:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/996880159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb640b31c0 0x7fbb640b3610 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/996880159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb64070350 0x7fbb6415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 -- 172.21.15.87:0/996880159 >> 172.21.15.87:0/996880159 conn(0x7fbb640669a0 msgr2=0x7fbb64066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 -- 172.21.15.87:0/996880159 shutdown_connections 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.904+0000 7fbb6ba00640 1 -- 172.21.15.87:0/996880159 wait complete. 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 Processor -- start 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 -- start start 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64070350 0x7fbb6414a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb640aef80 0x7fbb640b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb640b31c0 0x7fbb640ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb641679a0 con 0x7fbb64070350 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64167820 con 0x7fbb640b31c0 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64167b20 con 0x7fbb640aef80 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64070350 0x7fbb6414a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64070350 0x7fbb6414a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44934/0 (socket says 172.21.15.87:44934) 2024-04-25T10:18:35.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 -- 172.21.15.87:0/2523245476 learned_addr learned my addr 172.21.15.87:0/2523245476 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb63e00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb640aef80 0x7fbb640b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69a00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb640b31c0 0x7fbb640ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 -- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb640aef80 msgr2=0x7fbb640b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb640aef80 0x7fbb640b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 -- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb640b31c0 msgr2=0x7fbb640ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb640b31c0 0x7fbb640ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 -- 172.21.15.87:0/2523245476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640ae5e0 con 0x7fbb64070350 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb63e00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb640aef80 0x7fbb640b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb69000640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64070350 0x7fbb6414a2c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c420 tx=0x7fbb5800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb60c00640 1 -- 172.21.15.87:0/2523245476 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580052d0 con 0x7fbb64070350 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb60c00640 1 -- 172.21.15.87:0/2523245476 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58005470 con 0x7fbb64070350 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6415f5c0 con 0x7fbb64070350 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb60c00640 1 -- 172.21.15.87:0/2523245476 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58007260 con 0x7fbb64070350 2024-04-25T10:18:35.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6415fb80 con 0x7fbb64070350 2024-04-25T10:18:35.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.908+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6404ef30 con 0x7fbb64070350 2024-04-25T10:18:35.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.912+0000 7fbb60c00640 1 -- 172.21.15.87:0/2523245476 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb58007400 con 0x7fbb64070350 2024-04-25T10:18:35.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.912+0000 7fbb60c00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb44066f60 0x7fbb44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.912+0000 7fbb60c00640 1 -- 172.21.15.87:0/2523245476 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb58084220 con 0x7fbb64070350 2024-04-25T10:18:35.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.912+0000 7fbb63e00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb44066f60 0x7fbb44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.912+0000 7fbb63e00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb44066f60 0x7fbb44069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb6415d890 tx=0x7fbb50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:35.916+0000 7fbb60c00640 1 -- 172.21.15.87:0/2523245476 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb58055570 con 0x7fbb64070350 2024-04-25T10:18:36.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.076+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb640cb960 con 0x7fbb64070350 2024-04-25T10:18:36.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.076+0000 7fbb60c00640 1 -- 172.21.15.87:0/2523245476 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb58082e20 con 0x7fbb64070350 2024-04-25T10:18:36.080 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:18:36.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb44066f60 msgr2=0x7fbb44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb44066f60 0x7fbb44069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb6415d890 tx=0x7fbb50008040 comp rx=0 tx=0).stop 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64070350 msgr2=0x7fbb6414a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64070350 0x7fbb6414a2c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c420 tx=0x7fbb5800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 shutdown_connections 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb44066f60 0x7fbb44069420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb640b31c0 0x7fbb640ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb640aef80 0x7fbb640b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 --2- 172.21.15.87:0/2523245476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64070350 0x7fbb6414a2c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 >> 172.21.15.87:0/2523245476 conn(0x7fbb640669a0 msgr2=0x7fbb6415d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 shutdown_connections 2024-04-25T10:18:36.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.080+0000 7fbb6ba00640 1 -- 172.21.15.87:0/2523245476 wait complete. 2024-04-25T10:18:36.101 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:18:36.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.232+0000 7fdc05c00640 1 Processor -- start 2024-04-25T10:18:36.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.232+0000 7fdc05c00640 1 -- start start 2024-04-25T10:18:36.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdc05c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdc0015bc40 0x7fdc0006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdc05c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000b2770 con 0x7fdc000cf930 2024-04-25T10:18:36.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdc05c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0006c670 con 0x7fdc000cda10 2024-04-25T10:18:36.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdc05c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0006c950 con 0x7fdc0015bc40 2024-04-25T10:18:36.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbff400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fdc000cf930 0x7fdc000b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47290/0 (socket says 172.21.15.87:47290) 2024-04-25T10:18:36.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbff400640 1 -- 172.21.15.87:0/212283558 learned_addr learned my addr 172.21.15.87:0/212283558 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741129551 0 0) 0x7fdc000b2770 con 0x7fdc000cf930 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe4003530 con 0x7fdc000cf930 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfea00640 1 --2- 172.21.15.87:0/212283558 >> v2:172.21.15.169:3300/0 conn(0x7fdc0015bc40 0x7fdc0006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2543532542 0 0) 0x7fdbe4003530 con 0x7fdc000cf930 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 >> v2:172.21.15.169:3300/0 conn(0x7fdc0015bc40 msgr2=0x7fdc0006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 --2- 172.21.15.87:0/212283558 >> v2:172.21.15.169:3300/0 conn(0x7fdc0015bc40 0x7fdc0006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 >> v1:172.21.15.169:6789/0 conn(0x7fdc000cda10 legacy=0x7fdc000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0006cc30 con 0x7fdc000cf930 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fdbf8001a00 con 0x7fdc000cf930 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 >> v1:172.21.15.87:6789/0 conn(0x7fdc000cf930 legacy=0x7fdc000b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbe40039c0 0x7fdbe4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe4004810 0x7fdbe4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe40052f0 0x7fdbe400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0006c950 con 0x7fdbe40039c0 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe400c1c0 con 0x7fdbe40052f0 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe400c3c0 con 0x7fdbe4004810 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfea00640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbe40039c0 0x7fdbe4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbff400640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe40052f0 0x7fdbe400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbf6000640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe4004810 0x7fdbe4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfea00640 1 -- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe4004810 msgr2=0x7fdbe4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfea00640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe4004810 0x7fdbe4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfea00640 1 -- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe40052f0 msgr2=0x7fdbe400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfea00640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe40052f0 0x7fdbe400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbfea00640 1 -- 172.21.15.87:0/212283558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4003530 con 0x7fdbe40039c0 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.236+0000 7fdbff400640 1 -- 172.21.15.87:0/212283558 reap_dead start 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdbfea00640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbe40039c0 0x7fdbe4003de0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000ba00 tx=0x7fdbf001c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a951add83d94834e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00b3070 con 0x7fdbe40039c0 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf00a9070 con 0x7fdbe40039c0 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdbfe000640 1 -- 172.21.15.87:0/212283558 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf000b600 con 0x7fdbe40039c0 2024-04-25T10:18:36.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdc05c00640 1 -- 172.21.15.87:0/212283558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbe40039c0 msgr2=0x7fdbe4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdc05c00640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbe40039c0 0x7fdbe4003de0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000ba00 tx=0x7fdbf001c660 comp rx=0 tx=0).stop 2024-04-25T10:18:36.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdc05c00640 1 -- 172.21.15.87:0/212283558 shutdown_connections 2024-04-25T10:18:36.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdc05c00640 1 --2- 172.21.15.87:0/212283558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbe40039c0 0x7fdbe4003de0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdc05c00640 1 -- 172.21.15.87:0/212283558 >> 172.21.15.87:0/212283558 conn(0x7fdc00066b70 msgr2=0x7fdc00066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdc05c00640 1 -- 172.21.15.87:0/212283558 shutdown_connections 2024-04-25T10:18:36.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.240+0000 7fdc05c00640 1 -- 172.21.15.87:0/212283558 wait complete. 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 Processor -- start 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 -- start start 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe40039c0 0x7fdc0015d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc00151cb0 0x7fdc0014df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0014f3a0 0x7fdc00157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000d00d0 con 0x7fdc00151cb0 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000cff50 con 0x7fdc0014f3a0 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000d0250 con 0x7fdbe40039c0 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc00151cb0 0x7fdc0014df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc00151cb0 0x7fdc0014df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44944/0 (socket says 172.21.15.87:44944) 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 -- 172.21.15.87:0/1038722181 learned_addr learned my addr 172.21.15.87:0/1038722181 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbfea00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe40039c0 0x7fdc0015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbff400640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0014f3a0 0x7fdc00157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 -- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe40039c0 msgr2=0x7fdc0015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe40039c0 0x7fdc0015d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 -- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0014f3a0 msgr2=0x7fdc00157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0014f3a0 0x7fdc00157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 -- 172.21.15.87:0/1038722181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0015e4b0 con 0x7fdc00151cb0 2024-04-25T10:18:36.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf6000640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc00151cb0 0x7fdc0014df60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e410 tx=0x7fdbe800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf7400640 1 -- 172.21.15.87:0/1038722181 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8019070 con 0x7fdc00151cb0 2024-04-25T10:18:36.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf7400640 1 -- 172.21.15.87:0/1038722181 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe80052e0 con 0x7fdc00151cb0 2024-04-25T10:18:36.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0015def0 con 0x7fdc00151cb0 2024-04-25T10:18:36.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdbf7400640 1 -- 172.21.15.87:0/1038722181 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8003820 con 0x7fdc00151cb0 2024-04-25T10:18:36.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0015c330 con 0x7fdc00151cb0 2024-04-25T10:18:36.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.244+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0004ef30 con 0x7fdc00151cb0 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.248+0000 7fdbf7400640 1 -- 172.21.15.87:0/1038722181 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbe801c070 con 0x7fdc00151cb0 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.248+0000 7fdbf7400640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdbdc066f60 0x7fdbdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.248+0000 7fdbf7400640 1 -- 172.21.15.87:0/1038722181 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdbe80842f0 con 0x7fdc00151cb0 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.248+0000 7fdbfea00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdbdc066f60 0x7fdbdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.248+0000 7fdbfea00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdbdc066f60 0x7fdbdc069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00b0000 tx=0x7fdbf00046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.252+0000 7fdbf7400640 1 -- 172.21.15.87:0/1038722181 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdbe8054f60 con 0x7fdc00151cb0 2024-04-25T10:18:36.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.408+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdc0015f430 con 0x7fdc00151cb0 2024-04-25T10:18:36.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdbf7400640 1 -- 172.21.15.87:0/1038722181 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdbe8059000 con 0x7fdc00151cb0 2024-04-25T10:18:36.414 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:36.414 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:18:36.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:18:36.414 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:18:36.414 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:18:36.414 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:18:36.414 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdbdc066f60 msgr2=0x7fdbdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdbdc066f60 0x7fdbdc069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00b0000 tx=0x7fdbf00046d0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc00151cb0 msgr2=0x7fdc0014df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc00151cb0 0x7fdc0014df60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e410 tx=0x7fdbe800e8d0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 shutdown_connections 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdbdc066f60 0x7fdbdc069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0014f3a0 0x7fdc00157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc00151cb0 0x7fdc0014df60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 --2- 172.21.15.87:0/1038722181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbe40039c0 0x7fdc0015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 >> 172.21.15.87:0/1038722181 conn(0x7fdc00066b70 msgr2=0x7fdc000ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 shutdown_connections 2024-04-25T10:18:36.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.412+0000 7fdc05c00640 1 -- 172.21.15.87:0/1038722181 wait complete. 2024-04-25T10:18:36.435 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:18:36.435 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:18:36.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fe0a00640 1 Processor -- start 2024-04-25T10:18:36.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fe0a00640 1 -- start start 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fe0a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8fdc0cf040 0x7f8fdc0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fe0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8fdc071ab0 0x7f8fdc0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fe0a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fdc071f80 con 0x7f8fdc15bc00 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fe0a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fdc072260 con 0x7f8fdc071ab0 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fe0a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fdc0b3700 con 0x7f8fdc0cf040 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fd9400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8fdc15bc00 0x7f8fdc0713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47298/0 (socket says 172.21.15.87:47298) 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fd9400640 1 -- 172.21.15.87:0/2579989526 learned_addr learned my addr 172.21.15.87:0/2579989526 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fda800640 1 --2- 172.21.15.87:0/2579989526 >> v2:172.21.15.87:3300/0 conn(0x7f8fdc071ab0 0x7f8fdc0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fd9e00640 1 --2- 172.21.15.87:0/2579989526 >> v2:172.21.15.169:3300/0 conn(0x7f8fdc0cf040 0x7f8fdc0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fd8a00640 1 -- 172.21.15.87:0/2579989526 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243593162 0 0) 0x7f8fdc071f80 con 0x7f8fdc15bc00 2024-04-25T10:18:36.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fd8a00640 1 -- 172.21.15.87:0/2579989526 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc0005740 con 0x7f8fdc15bc00 2024-04-25T10:18:36.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fda800640 1 -- 172.21.15.87:0/2579989526 >> v2:172.21.15.169:3300/0 conn(0x7f8fdc0cf040 msgr2=0x7f8fdc0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fda800640 1 --2- 172.21.15.87:0/2579989526 >> v2:172.21.15.169:3300/0 conn(0x7f8fdc0cf040 0x7f8fdc0cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fda800640 1 -- 172.21.15.87:0/2579989526 >> v1:172.21.15.87:6789/0 conn(0x7f8fdc15bc00 legacy=0x7f8fdc0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.572+0000 7f8fda800640 1 -- 172.21.15.87:0/2579989526 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc15b120 con 0x7f8fdc071ab0 2024-04-25T10:18:36.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fda800640 1 --2- 172.21.15.87:0/2579989526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc071ab0 0x7f8fdc0cff20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0d05d0 tx=0x7f8fd401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1933b628bf64b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fd8a00640 1 -- 172.21.15.87:0/2579989526 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40b2070 con 0x7f8fdc071ab0 2024-04-25T10:18:36.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fd8a00640 1 -- 172.21.15.87:0/2579989526 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd40a58e0 con 0x7f8fdc071ab0 2024-04-25T10:18:36.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fd8a00640 1 -- 172.21.15.87:0/2579989526 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40a5cb0 con 0x7f8fdc071ab0 2024-04-25T10:18:36.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 -- 172.21.15.87:0/2579989526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc071ab0 msgr2=0x7f8fdc0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/2579989526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc071ab0 0x7f8fdc0cff20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0d05d0 tx=0x7f8fd401c930 comp rx=0 tx=0).stop 2024-04-25T10:18:36.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 -- 172.21.15.87:0/2579989526 shutdown_connections 2024-04-25T10:18:36.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/2579989526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc071ab0 0x7f8fdc0cff20 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/2579989526 >> v2:172.21.15.169:3300/0 conn(0x7f8fdc0cf040 0x7f8fdc0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 -- 172.21.15.87:0/2579989526 >> 172.21.15.87:0/2579989526 conn(0x7f8fdc066b70 msgr2=0x7f8fdc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 -- 172.21.15.87:0/2579989526 shutdown_connections 2024-04-25T10:18:36.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.576+0000 7f8fe0a00640 1 -- 172.21.15.87:0/2579989526 wait complete. 2024-04-25T10:18:36.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 Processor -- start 2024-04-25T10:18:36.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 -- start start 2024-04-25T10:18:36.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fdc071ab0 0x7f8fdc0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fdc0cf040 0x7f8fdc14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc15bc00 0x7f8fdc14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0aee10 con 0x7f8fdc15bc00 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0aec90 con 0x7f8fdc0cf040 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0aef90 con 0x7f8fdc071ab0 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc15bc00 0x7f8fdc14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc15bc00 0x7f8fdc14ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44964/0 (socket says 172.21.15.87:44964) 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 -- 172.21.15.87:0/3835889172 learned_addr learned my addr 172.21.15.87:0/3835889172 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fd9400640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fdc0cf040 0x7f8fdc14e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fd9e00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fdc071ab0 0x7f8fdc0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 -- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fdc071ab0 msgr2=0x7f8fdc0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fdc071ab0 0x7f8fdc0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 -- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fdc0cf040 msgr2=0x7f8fdc14e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fdc0cf040 0x7f8fdc14e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 -- 172.21.15.87:0/3835889172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc136da0 con 0x7f8fdc15bc00 2024-04-25T10:18:36.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fda800640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc15bc00 0x7f8fdc14ee70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4019540 tx=0x7f8fd40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fd2000640 1 -- 172.21.15.87:0/3835889172 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40b2050 con 0x7f8fdc15bc00 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fd2000640 1 -- 172.21.15.87:0/3835889172 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd40be070 con 0x7f8fdc15bc00 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fd2000640 1 -- 172.21.15.87:0/3835889172 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40ad890 con 0x7f8fdc15bc00 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc1399e0 con 0x7f8fdc15bc00 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.580+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc138cb0 con 0x7f8fdc15bc00 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.584+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc04ef30 con 0x7f8fdc15bc00 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.584+0000 7f8fd2000640 1 -- 172.21.15.87:0/3835889172 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fd40ada30 con 0x7f8fdc15bc00 2024-04-25T10:18:36.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.584+0000 7f8fd2000640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fbc066f90 0x7f8fbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.584+0000 7f8fd2000640 1 -- 172.21.15.87:0/3835889172 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8fd411f8b0 con 0x7f8fdc15bc00 2024-04-25T10:18:36.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.584+0000 7f8fd9e00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fbc066f90 0x7f8fbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.584+0000 7f8fd9e00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fbc066f90 0x7f8fbc069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009af0 tx=0x7f8fcc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.588+0000 7f8fd2000640 1 -- 172.21.15.87:0/3835889172 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fd40c7020 con 0x7f8fdc15bc00 2024-04-25T10:18:36.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:36.744+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8fdc0b2a10 con 0x7f8fdc15bc00 2024-04-25T10:18:37.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fd2000640 1 -- 172.21.15.87:0/3835889172 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8fd4016520 con 0x7f8fdc15bc00 2024-04-25T10:18:37.051 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fbc066f90 msgr2=0x7f8fbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fbc066f90 0x7f8fbc069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009af0 tx=0x7f8fcc008040 comp rx=0 tx=0).stop 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc15bc00 msgr2=0x7f8fdc14ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc15bc00 0x7f8fdc14ee70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4019540 tx=0x7f8fd40196d0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 shutdown_connections 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8fbc066f90 0x7f8fbc069450 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fdc15bc00 0x7f8fdc14ee70 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fdc0cf040 0x7f8fdc14e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 --2- 172.21.15.87:0/3835889172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fdc071ab0 0x7f8fdc0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 >> 172.21.15.87:0/3835889172 conn(0x7f8fdc066b70 msgr2=0x7f8fdc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 shutdown_connections 2024-04-25T10:18:37.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.048+0000 7f8fe0a00640 1 -- 172.21.15.87:0/3835889172 wait complete. 2024-04-25T10:18:37.078 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.216+0000 7f4ec1000640 1 Processor -- start 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ec1000640 1 -- start start 2024-04-25T10:18:37.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ec1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4ebc15c4d0 0x7f4ebc0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ec1000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ebc15ca90 con 0x7f4ebc0b06b0 2024-04-25T10:18:37.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ec1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ebc15cd70 con 0x7f4ebc144f80 2024-04-25T10:18:37.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ec1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ebc0cea10 con 0x7f4ebc15c4d0 2024-04-25T10:18:37.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4ebc0b06b0 0x7f4ebc15bdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47312/0 (socket says 172.21.15.87:47312) 2024-04-25T10:18:37.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2662590051 learned_addr learned my addr 172.21.15.87:0/2662590051 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebbe00640 1 --2- 172.21.15.87:0/2662590051 >> v2:172.21.15.169:3300/0 conn(0x7f4ebc15c4d0 0x7f4ebc0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13799075 0 0) 0x7f4ebc15ca90 con 0x7f4ebc0b06b0 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea0003530 con 0x7f4ebc0b06b0 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444442050 0 0) 0x7f4ebc15cd70 con 0x7f4ebc144f80 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ebc15ca90 con 0x7f4ebc144f80 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1254784316 0 0) 0x7f4ea0003530 con 0x7f4ebc0b06b0 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 >> v2:172.21.15.169:3300/0 conn(0x7f4ebc15c4d0 msgr2=0x7f4ebc0b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 --2- 172.21.15.87:0/2662590051 >> v2:172.21.15.169:3300/0 conn(0x7f4ebc15c4d0 0x7f4ebc0b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 >> v1:172.21.15.169:6789/0 conn(0x7f4ebc144f80 legacy=0x7f4ebc06c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0cc910 con 0x7f4ebc0b06b0 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f4ea8001680 con 0x7f4ebc0b06b0 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 >> v1:172.21.15.87:6789/0 conn(0x7f4ebc0b06b0 legacy=0x7f4ebc15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea0003b50 0x7f4ea0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea0004810 0x7f4ea0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00051d0 0x7f4ea000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea000c020 con 0x7f4ea0004810 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea000c1d0 con 0x7f4ea00051d0 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea000c3d0 con 0x7f4ea0003b50 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea0004810 0x7f4ea0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebb400640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea0003b50 0x7f4ea0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebbe00640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00051d0 0x7f4ea000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea0003b50 msgr2=0x7f4ea0003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea0003b50 0x7f4ea0003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00051d0 msgr2=0x7f4ea000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00051d0 0x7f4ea000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2662590051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0003530 con 0x7f4ea0004810 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebbe00640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00051d0 0x7f4ea000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.220+0000 7f4ebbe00640 1 -- 172.21.15.87:0/2662590051 reap_dead start 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea0004810 0x7f4ea0004c90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c160 tx=0x7f4ea801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e894ba54565a98fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8007cd0 con 0x7f4ea0004810 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea80a9020 con 0x7f4ea0004810 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4eba000640 1 -- 172.21.15.87:0/2662590051 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8005ae0 con 0x7f4ea0004810 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ec1000640 1 -- 172.21.15.87:0/2662590051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea0004810 msgr2=0x7f4ea0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea0004810 0x7f4ea0004c90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c160 tx=0x7f4ea801cbe0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ec1000640 1 -- 172.21.15.87:0/2662590051 shutdown_connections 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2662590051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea0004810 0x7f4ea0004c90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ec1000640 1 -- 172.21.15.87:0/2662590051 >> 172.21.15.87:0/2662590051 conn(0x7f4ebc0743a0 msgr2=0x7f4ebc0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ec1000640 1 -- 172.21.15.87:0/2662590051 shutdown_connections 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.224+0000 7f4ec1000640 1 -- 172.21.15.87:0/2662590051 wait complete. 2024-04-25T10:18:37.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 Processor -- start 2024-04-25T10:18:37.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 -- start start 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea0004810 0x7f4ebc1400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc0c5ae0 0x7f4ebc06c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ebc0c5f40 0x7f4ebc06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc145360 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0ccf60 con 0x7f4ea0004810 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc1454e0 con 0x7f4ebc0c5f40 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc0c5ae0 0x7f4ebc06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc0c5ae0 0x7f4ebc06c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44984/0 (socket says 172.21.15.87:44984) 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2201310583 learned_addr learned my addr 172.21.15.87:0/2201310583 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebbe00640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ebc0c5f40 0x7f4ebc06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebb400640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea0004810 0x7f4ebc1400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ebc0c5f40 msgr2=0x7f4ebc06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ebc0c5f40 0x7f4ebc06cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea0004810 msgr2=0x7f4ebc1400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea0004810 0x7f4ebc1400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 -- 172.21.15.87:0/2201310583 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0c6450 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebbe00640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ebc0c5f40 0x7f4ebc06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebb400640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea0004810 0x7f4ebc1400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ebaa00640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc0c5ae0 0x7f4ebc06c830 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80ab040 tx=0x7f4ea0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4eb3400640 1 -- 172.21.15.87:0/2201310583 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80049f0 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4eb3400640 1 -- 172.21.15.87:0/2201310583 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea80b2070 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4eb3400640 1 -- 172.21.15.87:0/2201310583 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8004d90 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc0c6730 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.228+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc0c6cf0 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.232+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc075120 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.232+0000 7f4eb3400640 1 -- 172.21.15.87:0/2201310583 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ea80ae430 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.232+0000 7f4eb3400640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e9c066f60 0x7f4e9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.232+0000 7f4eb3400640 1 -- 172.21.15.87:0/2201310583 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4ea811eb10 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.232+0000 7f4ebb400640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e9c066f60 0x7f4e9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.232+0000 7f4ebb400640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e9c066f60 0x7f4e9c069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4eac000c10 tx=0x7f4eac0077f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.236+0000 7f4eb3400640 1 -- 172.21.15.87:0/2201310583 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ea80168f0 con 0x7f4ebc0c5ae0 2024-04-25T10:18:37.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:37.392+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4ebc06dbd0 con 0x7f4ebc0c5ae0 2024-04-25T10:18:38.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.056+0000 7f4eb3400640 1 -- 172.21.15.87:0/2201310583 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4ea80efc60 con 0x7f4ebc0c5ae0 2024-04-25T10:18:38.059 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:18:38.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.056+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e9c066f60 msgr2=0x7f4e9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.056+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e9c066f60 0x7f4e9c069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4eac000c10 tx=0x7f4eac0077f0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.056+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc0c5ae0 msgr2=0x7f4ebc06c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.056+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc0c5ae0 0x7f4ebc06c830 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80ab040 tx=0x7f4ea0005840 comp rx=0 tx=0).stop 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.056+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 shutdown_connections 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.056+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4e9c066f60 0x7f4e9c069420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.060+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ebc0c5f40 0x7f4ebc06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.060+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc0c5ae0 0x7f4ebc06c830 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.060+0000 7f4ec1000640 1 --2- 172.21.15.87:0/2201310583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea0004810 0x7f4ebc1400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.060+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 >> 172.21.15.87:0/2201310583 conn(0x7f4ebc0743a0 msgr2=0x7f4ebc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.060+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 shutdown_connections 2024-04-25T10:18:38.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.060+0000 7f4ec1000640 1 -- 172.21.15.87:0/2201310583 wait complete. 2024-04-25T10:18:38.098 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:18:38.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.232+0000 7f6bc2600640 1 Processor -- start 2024-04-25T10:18:38.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bc2600640 1 -- start start 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bc2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6bbc159540 0x7f6bbc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bc2600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6bbc0aebd0 0x7f6bbc071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bc2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc071360 con 0x7f6bbc072020 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bc2600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc159920 con 0x7f6bbc159540 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bc2600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc15c2f0 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bbb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6bbc159540 0x7f6bbc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bbb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6bbc159540 0x7f6bbc0ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39310/0 (socket says 172.21.15.87:39310) 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bbb400640 1 -- 172.21.15.87:0/594860056 learned_addr learned my addr 172.21.15.87:0/594860056 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:38.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bb2a00640 1 --2- 172.21.15.87:0/594860056 >> v2:172.21.15.169:3300/0 conn(0x7f6bbc0aebd0 0x7f6bbc071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bbb400640 1 -- 172.21.15.87:0/594860056 >> v2:172.21.15.169:3300/0 conn(0x7f6bbc0aebd0 msgr2=0x7f6bbc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bbb400640 1 --2- 172.21.15.87:0/594860056 >> v2:172.21.15.169:3300/0 conn(0x7f6bbc0aebd0 0x7f6bbc071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bbb400640 1 -- 172.21.15.87:0/594860056 >> v1:172.21.15.169:6789/0 conn(0x7f6bbc072020 legacy=0x7f6bbc15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bbb400640 1 -- 172.21.15.87:0/594860056 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc15c5d0 con 0x7f6bbc159540 2024-04-25T10:18:38.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.236+0000 7f6bb2a00640 1 --2- 172.21.15.87:0/594860056 >> v2:172.21.15.169:3300/0 conn(0x7f6bbc0aebd0 0x7f6bbc071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:38.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bbb400640 1 --2- 172.21.15.87:0/594860056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc159540 0x7f6bbc0ae690 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0cd4e0 tx=0x7f6bac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e374b8067dae6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bbaa00640 1 -- 172.21.15.87:0/594860056 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0b0070 con 0x7f6bbc159540 2024-04-25T10:18:38.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bbaa00640 1 -- 172.21.15.87:0/594860056 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac0ab600 con 0x7f6bbc159540 2024-04-25T10:18:38.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bbaa00640 1 -- 172.21.15.87:0/594860056 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0ab9d0 con 0x7f6bbc159540 2024-04-25T10:18:38.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 -- 172.21.15.87:0/594860056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc159540 msgr2=0x7f6bbc0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 --2- 172.21.15.87:0/594860056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc159540 0x7f6bbc0ae690 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0cd4e0 tx=0x7f6bac01c520 comp rx=0 tx=0).stop 2024-04-25T10:18:38.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 -- 172.21.15.87:0/594860056 shutdown_connections 2024-04-25T10:18:38.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 --2- 172.21.15.87:0/594860056 >> v2:172.21.15.169:3300/0 conn(0x7f6bbc0aebd0 0x7f6bbc071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 --2- 172.21.15.87:0/594860056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc159540 0x7f6bbc0ae690 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 -- 172.21.15.87:0/594860056 >> 172.21.15.87:0/594860056 conn(0x7f6bbc0669a0 msgr2=0x7f6bbc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 -- 172.21.15.87:0/594860056 shutdown_connections 2024-04-25T10:18:38.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 -- 172.21.15.87:0/594860056 wait complete. 2024-04-25T10:18:38.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 Processor -- start 2024-04-25T10:18:38.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.240+0000 7f6bc2600640 1 -- start start 2024-04-25T10:18:38.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bbc072020 0x7f6bbc143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc0aebd0 0x7f6bbc1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bbc159540 0x7f6bbc1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc072430 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc06c0d0 con 0x7f6bbc159540 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc168930 con 0x7f6bbc072020 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc0aebd0 0x7f6bbc1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc0aebd0 0x7f6bbc1585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39326/0 (socket says 172.21.15.87:39326) 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bbc072020 0x7f6bbc143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 -- 172.21.15.87:0/724280607 learned_addr learned my addr 172.21.15.87:0/724280607 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bbbe00640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bbc159540 0x7f6bbc1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 -- 172.21.15.87:0/724280607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bbc072020 msgr2=0x7f6bbc143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bbc072020 0x7f6bbc143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 -- 172.21.15.87:0/724280607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bbc159540 msgr2=0x7f6bbc1534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bbc159540 0x7f6bbc1534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 -- 172.21.15.87:0/724280607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0b2ac0 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb2a00640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc0aebd0 0x7f6bbc1585a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e810 tx=0x7f6ba400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb3e00640 1 -- 172.21.15.87:0/724280607 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4017070 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb3e00640 1 -- 172.21.15.87:0/724280607 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba400c8d0 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bb3e00640 1 -- 172.21.15.87:0/724280607 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400f040 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc0b2da0 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc0b3360 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.244+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc04ef30 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.248+0000 7f6bb3e00640 1 -- 172.21.15.87:0/724280607 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ba401a070 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.248+0000 7f6bb3e00640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b98066f60 0x7f6b98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.248+0000 7f6bb3e00640 1 -- 172.21.15.87:0/724280607 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6ba4084ff0 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.248+0000 7f6bbb400640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b98066f60 0x7f6b98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.248+0000 7f6bbb400640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b98066f60 0x7f6b98069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0ad000 tx=0x7f6bac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.252+0000 7f6bb3e00640 1 -- 172.21.15.87:0/724280607 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ba4055a10 con 0x7f6bbc0aebd0 2024-04-25T10:18:38.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:38.436+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6bbc06d7f0 con 0x7f6bbc0aebd0 2024-04-25T10:18:39.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.056+0000 7f6bb3e00640 1 -- 172.21.15.87:0/724280607 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v59) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6ba4059ab0 con 0x7f6bbc0aebd0 2024-04-25T10:18:39.061 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T10:18:39.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.060+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b98066f60 msgr2=0x7f6b98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.060+0000 7f6bc2600640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b98066f60 0x7f6b98069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0ad000 tx=0x7f6bac008040 comp rx=0 tx=0).stop 2024-04-25T10:18:39.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.060+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc0aebd0 msgr2=0x7f6bbc1585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.060+0000 7f6bc2600640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc0aebd0 0x7f6bbc1585a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e810 tx=0x7f6ba400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 shutdown_connections 2024-04-25T10:18:39.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b98066f60 0x7f6b98069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bbc159540 0x7f6bbc1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bbc0aebd0 0x7f6bbc1585a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 --2- 172.21.15.87:0/724280607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bbc072020 0x7f6bbc143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 >> 172.21.15.87:0/724280607 conn(0x7f6bbc0669a0 msgr2=0x7f6bbc0cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 shutdown_connections 2024-04-25T10:18:39.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.064+0000 7f6bc2600640 1 -- 172.21.15.87:0/724280607 wait complete. 2024-04-25T10:18:39.107 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:39.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.240+0000 7f475d400640 1 Processor -- start 2024-04-25T10:18:39.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.240+0000 7f475d400640 1 -- start start 2024-04-25T10:18:39.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.240+0000 7f475d400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f47580d0040 0x7f475815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.240+0000 7f475d400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4758071050 con 0x7f475815b740 2024-04-25T10:18:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.240+0000 7f475d400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475815d040 con 0x7f47581550b0 2024-04-25T10:18:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.240+0000 7f475d400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47580712f0 con 0x7f47580d0040 2024-04-25T10:18:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4757200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f475815b740 0x7f47580b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34286/0 (socket says 172.21.15.87:34286) 2024-04-25T10:18:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4757200640 1 -- 172.21.15.87:0/890207370 learned_addr learned my addr 172.21.15.87:0/890207370 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755e00640 1 --2- 172.21.15.87:0/890207370 >> v2:172.21.15.169:3300/0 conn(0x7f47580d0040 0x7f475815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755400640 1 -- 172.21.15.87:0/890207370 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640290546 0 0) 0x7f4758071050 con 0x7f475815b740 2024-04-25T10:18:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755400640 1 -- 172.21.15.87:0/890207370 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f473c003530 con 0x7f475815b740 2024-04-25T10:18:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755400640 1 -- 172.21.15.87:0/890207370 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168205636 0 0) 0x7f475815d040 con 0x7f47581550b0 2024-04-25T10:18:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755400640 1 -- 172.21.15.87:0/890207370 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4758071050 con 0x7f47581550b0 2024-04-25T10:18:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755e00640 1 -- 172.21.15.87:0/890207370 >> v1:172.21.15.169:6789/0 conn(0x7f47581550b0 legacy=0x7f47580cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755e00640 1 -- 172.21.15.87:0/890207370 >> v1:172.21.15.87:6789/0 conn(0x7f475815b740 legacy=0x7f47580b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755e00640 1 -- 172.21.15.87:0/890207370 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47580715d0 con 0x7f47580d0040 2024-04-25T10:18:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.244+0000 7f4755e00640 1 --2- 172.21.15.87:0/890207370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 0x7f475815b200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f47440097c0 tx=0x7f474401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7204cfe6ebc90af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4755400640 1 -- 172.21.15.87:0/890207370 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47440b0070 con 0x7f47580d0040 2024-04-25T10:18:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4755400640 1 -- 172.21.15.87:0/890207370 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47440ab600 con 0x7f47580d0040 2024-04-25T10:18:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4755400640 1 -- 172.21.15.87:0/890207370 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47440ab9d0 con 0x7f47580d0040 2024-04-25T10:18:39.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- 172.21.15.87:0/890207370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 msgr2=0x7f475815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 --2- 172.21.15.87:0/890207370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 0x7f475815b200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f47440097c0 tx=0x7f474401c470 comp rx=0 tx=0).stop 2024-04-25T10:18:39.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- 172.21.15.87:0/890207370 shutdown_connections 2024-04-25T10:18:39.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 --2- 172.21.15.87:0/890207370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 0x7f475815b200 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- 172.21.15.87:0/890207370 >> 172.21.15.87:0/890207370 conn(0x7f4758066b70 msgr2=0x7f4758066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- 172.21.15.87:0/890207370 shutdown_connections 2024-04-25T10:18:39.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- 172.21.15.87:0/890207370 wait complete. 2024-04-25T10:18:39.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 Processor -- start 2024-04-25T10:18:39.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- start start 2024-04-25T10:18:39.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 0x7f47581400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47581550b0 0x7f47580adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475815b740 0x7f47580b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475815bcd0 con 0x7f475815b740 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475815bb50 con 0x7f47580d0040 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f475d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47581684c0 con 0x7f47581550b0 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4757200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475815b740 0x7f47580b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4757200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475815b740 0x7f47580b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39330/0 (socket says 172.21.15.87:39330) 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4757200640 1 -- 172.21.15.87:0/682007771 learned_addr learned my addr 172.21.15.87:0/682007771 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4756800640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 0x7f47581400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.248+0000 7f4755e00640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47581550b0 0x7f47580adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f4757200640 1 -- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47581550b0 msgr2=0x7f47580adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f4757200640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47581550b0 0x7f47580adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f4757200640 1 -- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 msgr2=0x7f47581400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f4757200640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 0x7f47581400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f4757200640 1 -- 172.21.15.87:0/682007771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4758154060 con 0x7f475815b740 2024-04-25T10:18:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f4757200640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475815b740 0x7f47580b0560 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f475000cb20 tx=0x7f475000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f474ea00640 1 -- 172.21.15.87:0/682007771 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750017070 con 0x7f475815b740 2024-04-25T10:18:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f474ea00640 1 -- 172.21.15.87:0/682007771 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475000aca0 con 0x7f475815b740 2024-04-25T10:18:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f474ea00640 1 -- 172.21.15.87:0/682007771 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750012600 con 0x7f475815b740 2024-04-25T10:18:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4758154340 con 0x7f475815b740 2024-04-25T10:18:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47581405f0 con 0x7f475815b740 2024-04-25T10:18:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475804ef30 con 0x7f475815b740 2024-04-25T10:18:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f474ea00640 1 -- 172.21.15.87:0/682007771 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f475001a070 con 0x7f475815b740 2024-04-25T10:18:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f474ea00640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4738066f60 0x7f4738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f474ea00640 1 -- 172.21.15.87:0/682007771 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4750084c30 con 0x7f475815b740 2024-04-25T10:18:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.252+0000 7f4756800640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4738066f60 0x7f4738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.256+0000 7f4756800640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4738066f60 0x7f4738069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47581f9f90 tx=0x7f4748009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.256+0000 7f474ea00640 1 -- 172.21.15.87:0/682007771 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f475008b080 con 0x7f475815b740 2024-04-25T10:18:39.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.412+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f475814e740 con 0x7f475815b740 2024-04-25T10:18:39.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.412+0000 7f474ea00640 1 -- 172.21.15.87:0/682007771 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16661 (secure 0 0 0) 0x7f4750055660 con 0x7f475815b740 2024-04-25T10:18:39.416 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:39.416 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":86,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:18:38.441796+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:36.789308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:37.439968+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:39.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4738066f60 msgr2=0x7f4738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4738066f60 0x7f4738069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47581f9f90 tx=0x7f4748009040 comp rx=0 tx=0).stop 2024-04-25T10:18:39.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475815b740 msgr2=0x7f47580b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475815b740 0x7f47580b0560 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f475000cb20 tx=0x7f475000a060 comp rx=0 tx=0).stop 2024-04-25T10:18:39.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 shutdown_connections 2024-04-25T10:18:39.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4738066f60 0x7f4738069420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475815b740 0x7f47580b0560 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47581550b0 0x7f47580adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 --2- 172.21.15.87:0/682007771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47580d0040 0x7f47581400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 >> 172.21.15.87:0/682007771 conn(0x7f4758066b70 msgr2=0x7f475815bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 shutdown_connections 2024-04-25T10:18:39.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.416+0000 7f475d400640 1 -- 172.21.15.87:0/682007771 wait complete. 2024-04-25T10:18:39.438 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:39.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.568+0000 7fe07f800640 1 Processor -- start 2024-04-25T10:18:39.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.568+0000 7fe07f800640 1 -- start start 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe0780cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07f800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe07815b910 0x7fe0780b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07f800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe078071050 con 0x7fe0780cfe70 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07f800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07815bda0 con 0x7fe07815b910 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07815d040 con 0x7fe0781550b0 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07d800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe07815b910 0x7fe0780b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07d800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe07815b910 0x7fe0780b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39342/0 (socket says 172.21.15.87:39342) 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07d800640 1 -- 172.21.15.87:0/3445863250 learned_addr learned my addr 172.21.15.87:0/3445863250 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:39.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07ce00640 1 --2- 172.21.15.87:0/3445863250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe0780cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07d800640 1 -- 172.21.15.87:0/3445863250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 msgr2=0x7fe0780cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07d800640 1 --2- 172.21.15.87:0/3445863250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe0780cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07d800640 1 -- 172.21.15.87:0/3445863250 >> v1:172.21.15.169:6789/0 conn(0x7fe0780cfe70 legacy=0x7fe07815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07d800640 1 -- 172.21.15.87:0/3445863250 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780712f0 con 0x7fe07815b910 2024-04-25T10:18:39.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.572+0000 7fe07ce00640 1 --2- 172.21.15.87:0/3445863250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe0780cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:39.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07d800640 1 --2- 172.21.15.87:0/3445863250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe07815b910 0x7fe0780b3260 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe070009e50 tx=0x7fe07001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf462fe354cea75d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe077400640 1 -- 172.21.15.87:0/3445863250 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0700a5760 con 0x7fe07815b910 2024-04-25T10:18:39.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe077400640 1 -- 172.21.15.87:0/3445863250 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0700a5900 con 0x7fe07815b910 2024-04-25T10:18:39.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe077400640 1 -- 172.21.15.87:0/3445863250 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0700a5cd0 con 0x7fe07815b910 2024-04-25T10:18:39.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 -- 172.21.15.87:0/3445863250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe07815b910 msgr2=0x7fe0780b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 --2- 172.21.15.87:0/3445863250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe07815b910 0x7fe0780b3260 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe070009e50 tx=0x7fe07001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 -- 172.21.15.87:0/3445863250 shutdown_connections 2024-04-25T10:18:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 --2- 172.21.15.87:0/3445863250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe07815b910 0x7fe0780b3260 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 --2- 172.21.15.87:0/3445863250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe0780cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 -- 172.21.15.87:0/3445863250 >> 172.21.15.87:0/3445863250 conn(0x7fe078066b70 msgr2=0x7fe078066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 -- 172.21.15.87:0/3445863250 shutdown_connections 2024-04-25T10:18:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 -- 172.21.15.87:0/3445863250 wait complete. 2024-04-25T10:18:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 Processor -- start 2024-04-25T10:18:39.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.576+0000 7fe07f800640 1 -- start start 2024-04-25T10:18:39.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0780cfe70 0x7fe078159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe07815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe07815b910 0x7fe078151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07815c030 con 0x7fe0780cfe70 2024-04-25T10:18:39.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780720b0 con 0x7fe07815b910 2024-04-25T10:18:39.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07815c1b0 con 0x7fe0781550b0 2024-04-25T10:18:39.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0780cfe70 0x7fe078159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0780cfe70 0x7fe078159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39350/0 (socket says 172.21.15.87:39350) 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 -- 172.21.15.87:0/1980397612 learned_addr learned my addr 172.21.15.87:0/1980397612 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe077e00640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe07815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07d800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe07815b910 0x7fe078151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 -- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 msgr2=0x7fe07815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe07815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 -- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe07815b910 msgr2=0x7fe078151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe07815b910 0x7fe078151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 -- 172.21.15.87:0/1980397612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07814d080 con 0x7fe0780cfe70 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07d800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe07815b910 0x7fe078151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07ce00640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0780cfe70 0x7fe078159b80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c7c0 tx=0x7fe06c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe074c00640 1 -- 172.21.15.87:0/1980397612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c007a30 con 0x7fe0780cfe70 2024-04-25T10:18:39.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe074c00640 1 -- 172.21.15.87:0/1980397612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c007bd0 con 0x7fe0780cfe70 2024-04-25T10:18:39.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe078152940 con 0x7fe0780cfe70 2024-04-25T10:18:39.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe074c00640 1 -- 172.21.15.87:0/1980397612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0113f0 con 0x7fe0780cfe70 2024-04-25T10:18:39.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe078152f00 con 0x7fe0780cfe70 2024-04-25T10:18:39.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07804ef30 con 0x7fe0780cfe70 2024-04-25T10:18:39.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.580+0000 7fe074c00640 1 -- 172.21.15.87:0/1980397612 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe06c011590 con 0x7fe0780cfe70 2024-04-25T10:18:39.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.584+0000 7fe074c00640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe058066f60 0x7fe058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.584+0000 7fe074c00640 1 -- 172.21.15.87:0/1980397612 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe06c085220 con 0x7fe0780cfe70 2024-04-25T10:18:39.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.584+0000 7fe077e00640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe058066f60 0x7fe058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.584+0000 7fe077e00640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe058066f60 0x7fe058069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe0780cbc90 tx=0x7fe064007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.588+0000 7fe074c00640 1 -- 172.21.15.87:0/1980397612 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe07814d080 con 0x7fe0780cfe70 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.740+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe078153ee0 con 0x7fe0780cfe70 2024-04-25T10:18:39.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe074c00640 1 -- 172.21.15.87:0/1980397612 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16661 (secure 0 0 0) 0x7fe06c015020 con 0x7fe0780cfe70 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":86,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:18:38.441796+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:36.789308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:37.439968+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe058066f60 msgr2=0x7fe058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe058066f60 0x7fe058069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe0780cbc90 tx=0x7fe064007e40 comp rx=0 tx=0).stop 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0780cfe70 msgr2=0x7fe078159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0780cfe70 0x7fe078159b80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c7c0 tx=0x7fe06c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 shutdown_connections 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe058066f60 0x7fe058069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe07815b910 0x7fe078151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0781550b0 0x7fe07815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 --2- 172.21.15.87:0/1980397612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0780cfe70 0x7fe078159b80 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 >> 172.21.15.87:0/1980397612 conn(0x7fe078066b70 msgr2=0x7fe07814d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 shutdown_connections 2024-04-25T10:18:39.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.744+0000 7fe07f800640 1 -- 172.21.15.87:0/1980397612 wait complete. 2024-04-25T10:18:39.766 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:18:39.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.900+0000 7f7e31200640 1 Processor -- start 2024-04-25T10:18:39.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.900+0000 7f7e31200640 1 -- start start 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e31200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e2c15a0e0 0x7f7e2c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e31200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e2c06c0d0 0x7f7e2c06c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e31200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0b2fd0 con 0x7f7e2c0aea50 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e31200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c0af1d0 con 0x7f7e2c06c0d0 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e31200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c0b32b0 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e2a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e2c06c0d0 0x7f7e2c06c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e2a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e2c06c0d0 0x7f7e2c06c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39360/0 (socket says 172.21.15.87:39360) 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e2a800640 1 -- 172.21.15.87:0/184220293 learned_addr learned my addr 172.21.15.87:0/184220293 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e29e00640 1 --2- 172.21.15.87:0/184220293 >> v2:172.21.15.169:3300/0 conn(0x7f7e2c15a0e0 0x7f7e2c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e2a800640 1 -- 172.21.15.87:0/184220293 >> v2:172.21.15.169:3300/0 conn(0x7f7e2c15a0e0 msgr2=0x7f7e2c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e2a800640 1 --2- 172.21.15.87:0/184220293 >> v2:172.21.15.169:3300/0 conn(0x7f7e2c15a0e0 0x7f7e2c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e2a800640 1 -- 172.21.15.87:0/184220293 >> v1:172.21.15.169:6789/0 conn(0x7f7e2c0aea50 legacy=0x7f7e2c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.904+0000 7f7e2a800640 1 -- 172.21.15.87:0/184220293 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0b3590 con 0x7f7e2c06c0d0 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e2a800640 1 --2- 172.21.15.87:0/184220293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c06c0d0 0x7f7e2c06c500 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06e4c0 tx=0x7f7e2401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662044eb0899ddee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e29400640 1 -- 172.21.15.87:0/184220293 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240b0070 con 0x7f7e2c06c0d0 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e29400640 1 -- 172.21.15.87:0/184220293 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e240ab600 con 0x7f7e2c06c0d0 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e29400640 1 -- 172.21.15.87:0/184220293 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240ab9d0 con 0x7f7e2c06c0d0 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- 172.21.15.87:0/184220293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c06c0d0 msgr2=0x7f7e2c06c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 --2- 172.21.15.87:0/184220293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c06c0d0 0x7f7e2c06c500 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06e4c0 tx=0x7f7e2401c400 comp rx=0 tx=0).stop 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- 172.21.15.87:0/184220293 shutdown_connections 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 --2- 172.21.15.87:0/184220293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c06c0d0 0x7f7e2c06c500 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 --2- 172.21.15.87:0/184220293 >> v2:172.21.15.169:3300/0 conn(0x7f7e2c15a0e0 0x7f7e2c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- 172.21.15.87:0/184220293 >> 172.21.15.87:0/184220293 conn(0x7f7e2c0669a0 msgr2=0x7f7e2c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- 172.21.15.87:0/184220293 shutdown_connections 2024-04-25T10:18:39.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- 172.21.15.87:0/184220293 wait complete. 2024-04-25T10:18:39.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 Processor -- start 2024-04-25T10:18:39.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- start start 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c06c0d0 0x7f7e2c14d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c0aea50 0x7f7e2c0af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c15a0e0 0x7f7e2c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c070770 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0705f0 con 0x7f7e2c06c0d0 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e31200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0708f0 con 0x7f7e2c0aea50 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.908+0000 7f7e2a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c15a0e0 0x7f7e2c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c15a0e0 0x7f7e2c1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39366/0 (socket says 172.21.15.87:39366) 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 -- 172.21.15.87:0/625079247 learned_addr learned my addr 172.21.15.87:0/625079247 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e29e00640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c06c0d0 0x7f7e2c14d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e21400640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c0aea50 0x7f7e2c0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 -- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c0aea50 msgr2=0x7f7e2c0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c0aea50 0x7f7e2c0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 -- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c06c0d0 msgr2=0x7f7e2c14d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c06c0d0 0x7f7e2c14d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 -- 172.21.15.87:0/625079247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0afcd0 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e2a800640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c15a0e0 0x7f7e2c1576e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7e2401a030 tx=0x7f7e240191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e22a00640 1 -- 172.21.15.87:0/625079247 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240b0040 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e22a00640 1 -- 172.21.15.87:0/625079247 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e240b4430 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e22a00640 1 -- 172.21.15.87:0/625079247 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240b4820 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c0affb0 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c0b0570 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c04ef30 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.912+0000 7f7e22a00640 1 -- 172.21.15.87:0/625079247 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e240b49c0 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.916+0000 7f7e22a00640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e08066f60 0x7f7e08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.916+0000 7f7e22a00640 1 -- 172.21.15.87:0/625079247 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e2411e9b0 con 0x7f7e2c15a0e0 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.916+0000 7f7e29e00640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e08066f60 0x7f7e08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.916+0000 7f7e29e00640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e08066f60 0x7f7e08069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0ce4c0 tx=0x7f7e1c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:39.920+0000 7f7e22a00640 1 -- 172.21.15.87:0/625079247 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e24016110 con 0x7f7e2c15a0e0 2024-04-25T10:18:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:40.112+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7e2c0ad9e0 con 0x7f7e2c15a0e0 2024-04-25T10:18:41.036 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-04-25T10:18:41.032+0000 7fdfb4800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:18:41.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.088+0000 7f7e22a00640 1 -- 172.21.15.87:0/625079247 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7e240ef470 con 0x7f7e2c15a0e0 2024-04-25T10:18:41.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e08066f60 msgr2=0x7f7e08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e08066f60 0x7f7e08069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0ce4c0 tx=0x7f7e1c00b040 comp rx=0 tx=0).stop 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c15a0e0 msgr2=0x7f7e2c1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c15a0e0 0x7f7e2c1576e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7e2401a030 tx=0x7f7e240191a0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 shutdown_connections 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e08066f60 0x7f7e08069420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e2c15a0e0 0x7f7e2c1576e0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c0aea50 0x7f7e2c0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 --2- 172.21.15.87:0/625079247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c06c0d0 0x7f7e2c14d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 >> 172.21.15.87:0/625079247 conn(0x7f7e2c0669a0 msgr2=0x7f7e2c159010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 shutdown_connections 2024-04-25T10:18:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.092+0000 7f7e31200640 1 -- 172.21.15.87:0/625079247 wait complete. 2024-04-25T10:18:41.119 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:18:41.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:18:41.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 Processor -- start 2024-04-25T10:18:41.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 -- start start 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9cc0aef80 0x7fe9cc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9cc070350 0x7fe9cc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc15b080 con 0x7fe9cc0aef80 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc15adb0 con 0x7fe9cc070350 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc15eb60 con 0x7fe9cc15bac0 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9cc0aef80 0x7fe9cc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9cc0aef80 0x7fe9cc15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39382/0 (socket says 172.21.15.87:39382) 2024-04-25T10:18:41.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 -- 172.21.15.87:0/3895103073 learned_addr learned my addr 172.21.15.87:0/3895103073 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:41.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d0e00640 1 --2- 172.21.15.87:0/3895103073 >> v2:172.21.15.169:3300/0 conn(0x7fe9cc070350 0x7fe9cc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d2200640 1 --2- 172.21.15.87:0/3895103073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 -- 172.21.15.87:0/3895103073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 msgr2=0x7fe9cc0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 --2- 172.21.15.87:0/3895103073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 -- 172.21.15.87:0/3895103073 >> v2:172.21.15.169:3300/0 conn(0x7fe9cc070350 msgr2=0x7fe9cc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 --2- 172.21.15.87:0/3895103073 >> v2:172.21.15.169:3300/0 conn(0x7fe9cc070350 0x7fe9cc15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d1800640 1 -- 172.21.15.87:0/3895103073 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc15ee40 con 0x7fe9cc0aef80 2024-04-25T10:18:41.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d0e00640 1 --2- 172.21.15.87:0/3895103073 >> v2:172.21.15.169:3300/0 conn(0x7fe9cc070350 0x7fe9cc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:41.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.260+0000 7fe9d2200640 1 --2- 172.21.15.87:0/3895103073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:41.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d1800640 1 --2- 172.21.15.87:0/3895103073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc15e480 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc06e390 tx=0x7fe9c001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353c045b9994ceda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9cbe00640 1 -- 172.21.15.87:0/3895103073 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000b2d0 con 0x7fe9cc0aef80 2024-04-25T10:18:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9cbe00640 1 -- 172.21.15.87:0/3895103073 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c000b470 con 0x7fe9cc0aef80 2024-04-25T10:18:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9cbe00640 1 -- 172.21.15.87:0/3895103073 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00a7aa0 con 0x7fe9cc0aef80 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 -- 172.21.15.87:0/3895103073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 msgr2=0x7fe9cc15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 --2- 172.21.15.87:0/3895103073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc15e480 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc06e390 tx=0x7fe9c001ebf0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 -- 172.21.15.87:0/3895103073 shutdown_connections 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 --2- 172.21.15.87:0/3895103073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 --2- 172.21.15.87:0/3895103073 >> v2:172.21.15.169:3300/0 conn(0x7fe9cc070350 0x7fe9cc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 --2- 172.21.15.87:0/3895103073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc15e480 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 -- 172.21.15.87:0/3895103073 >> 172.21.15.87:0/3895103073 conn(0x7fe9cc0669a0 msgr2=0x7fe9cc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 -- 172.21.15.87:0/3895103073 shutdown_connections 2024-04-25T10:18:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.264+0000 7fe9d4200640 1 -- 172.21.15.87:0/3895103073 wait complete. 2024-04-25T10:18:41.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 Processor -- start 2024-04-25T10:18:41.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 -- start start 2024-04-25T10:18:41.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc070350 0x7fe9cc156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc165ba0 con 0x7fe9cc0aef80 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc165a20 con 0x7fe9cc070350 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc165d20 con 0x7fe9cc15bac0 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc1543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39392/0 (socket says 172.21.15.87:39392) 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 -- 172.21.15.87:0/1388031085 learned_addr learned my addr 172.21.15.87:0/1388031085 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d1800640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc070350 0x7fe9cc156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d2200640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 -- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 msgr2=0x7fe9cc0ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0ab530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 -- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc070350 msgr2=0x7fe9cc156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc070350 0x7fe9cc156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 -- 172.21.15.87:0/1388031085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0a9870 con 0x7fe9cc0aef80 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d0e00640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc1543d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c7f0 tx=0x7fe9b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9c9600640 1 -- 172.21.15.87:0/1388031085 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8015070 con 0x7fe9cc0aef80 2024-04-25T10:18:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9c9600640 1 -- 172.21.15.87:0/1388031085 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8007820 con 0x7fe9cc0aef80 2024-04-25T10:18:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9c9600640 1 -- 172.21.15.87:0/1388031085 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8007c10 con 0x7fe9cc0aef80 2024-04-25T10:18:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc156fe0 con 0x7fe9cc0aef80 2024-04-25T10:18:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.268+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc1575a0 con 0x7fe9cc0aef80 2024-04-25T10:18:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.272+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc04ef30 con 0x7fe9cc0aef80 2024-04-25T10:18:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.272+0000 7fe9c9600640 1 -- 172.21.15.87:0/1388031085 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9b8018070 con 0x7fe9cc0aef80 2024-04-25T10:18:41.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.272+0000 7fe9c9600640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9ac066f60 0x7fe9ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.272+0000 7fe9c9600640 1 -- 172.21.15.87:0/1388031085 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9b8085160 con 0x7fe9cc0aef80 2024-04-25T10:18:41.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.272+0000 7fe9d1800640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9ac066f60 0x7fe9ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.272+0000 7fe9d1800640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9ac066f60 0x7fe9ac069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00b0000 tx=0x7fe9c00a9070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.276+0000 7fe9c9600640 1 -- 172.21.15.87:0/1388031085 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9b80560d0 con 0x7fe9cc0aef80 2024-04-25T10:18:41.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:41.464+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe9cc0b2fa0 con 0x7fe9cc0aef80 2024-04-25T10:18:42.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9c9600640 1 -- 172.21.15.87:0/1388031085 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe9b80559d0 con 0x7fe9cc0aef80 2024-04-25T10:18:42.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9ac066f60 msgr2=0x7fe9ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9ac066f60 0x7fe9ac069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00b0000 tx=0x7fe9c00a9070 comp rx=0 tx=0).stop 2024-04-25T10:18:42.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 msgr2=0x7fe9cc1543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc1543d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c7f0 tx=0x7fe9b800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 shutdown_connections 2024-04-25T10:18:42.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe9ac066f60 0x7fe9ac069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9cc15bac0 0x7fe9cc0ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9cc0aef80 0x7fe9cc1543d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 --2- 172.21.15.87:0/1388031085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9cc070350 0x7fe9cc156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 >> 172.21.15.87:0/1388031085 conn(0x7fe9cc0669a0 msgr2=0x7fe9cc0ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 shutdown_connections 2024-04-25T10:18:42.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.100+0000 7fe9d4200640 1 -- 172.21.15.87:0/1388031085 wait complete. 2024-04-25T10:18:42.126 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:18:42.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:42.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e9600640 1 Processor -- start 2024-04-25T10:18:42.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e9600640 1 -- start start 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f49e4070c20 0x7f49e415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e9600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f49e40ae020 0x7f49e40ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e9600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e40ad840 con 0x7f49e40cf930 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e9600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e415bb90 con 0x7f49e40ae020 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e9600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e415e9f0 con 0x7f49e4070c20 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f49e40ae020 0x7f49e40ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f49e40ae020 0x7f49e40ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39396/0 (socket says 172.21.15.87:39396) 2024-04-25T10:18:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e3400640 1 -- 172.21.15.87:0/1474513384 learned_addr learned my addr 172.21.15.87:0/1474513384 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e3400640 1 -- 172.21.15.87:0/1474513384 >> v2:172.21.15.169:3300/0 conn(0x7f49e4070c20 msgr2=0x7f49e415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e3400640 1 --2- 172.21.15.87:0/1474513384 >> v2:172.21.15.169:3300/0 conn(0x7f49e4070c20 0x7f49e415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e3400640 1 -- 172.21.15.87:0/1474513384 >> v1:172.21.15.169:6789/0 conn(0x7f49e40cf930 legacy=0x7f49e40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.260+0000 7f49e3400640 1 -- 172.21.15.87:0/1474513384 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e415ecd0 con 0x7f49e40ae020 2024-04-25T10:18:42.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e3400640 1 --2- 172.21.15.87:0/1474513384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e40ae020 0x7f49e40ae4b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f49e40d0630 tx=0x7f49dc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3f1c2c3faa509c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e2000640 1 -- 172.21.15.87:0/1474513384 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0aa610 con 0x7f49e40ae020 2024-04-25T10:18:42.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e2000640 1 -- 172.21.15.87:0/1474513384 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc0af070 con 0x7f49e40ae020 2024-04-25T10:18:42.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e2000640 1 -- 172.21.15.87:0/1474513384 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0aa9d0 con 0x7f49e40ae020 2024-04-25T10:18:42.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 -- 172.21.15.87:0/1474513384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e40ae020 msgr2=0x7f49e40ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 --2- 172.21.15.87:0/1474513384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e40ae020 0x7f49e40ae4b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f49e40d0630 tx=0x7f49dc01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 -- 172.21.15.87:0/1474513384 shutdown_connections 2024-04-25T10:18:42.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 --2- 172.21.15.87:0/1474513384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e40ae020 0x7f49e40ae4b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 --2- 172.21.15.87:0/1474513384 >> v2:172.21.15.169:3300/0 conn(0x7f49e4070c20 0x7f49e415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 -- 172.21.15.87:0/1474513384 >> 172.21.15.87:0/1474513384 conn(0x7f49e4066b70 msgr2=0x7f49e4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 -- 172.21.15.87:0/1474513384 shutdown_connections 2024-04-25T10:18:42.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.264+0000 7f49e9600640 1 -- 172.21.15.87:0/1474513384 wait complete. 2024-04-25T10:18:42.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 Processor -- start 2024-04-25T10:18:42.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 -- start start 2024-04-25T10:18:42.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e4070c20 0x7f49e4157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49e40ae020 0x7f49e40b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e40cf930 0x7f49e40b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e416a8e0 con 0x7f49e4070c20 2024-04-25T10:18:42.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e415e6b0 con 0x7f49e40cf930 2024-04-25T10:18:42.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e416aa60 con 0x7f49e40ae020 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e4070c20 0x7f49e4157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e4070c20 0x7f49e4157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39410/0 (socket says 172.21.15.87:39410) 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 -- 172.21.15.87:0/90528324 learned_addr learned my addr 172.21.15.87:0/90528324 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49da000640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49e40ae020 0x7f49e40b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e3400640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e40cf930 0x7f49e40b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 -- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49e40ae020 msgr2=0x7f49e40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49e40ae020 0x7f49e40b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 -- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e40cf930 msgr2=0x7f49e40b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e40cf930 0x7f49e40b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 -- 172.21.15.87:0/90528324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4158270 con 0x7f49e4070c20 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e3400640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e40cf930 0x7f49e40b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e2a00640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e4070c20 0x7f49e4157890 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f49d400ba50 tx=0x7f49d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49db400640 1 -- 172.21.15.87:0/90528324 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4019070 con 0x7f49e4070c20 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49db400640 1 -- 172.21.15.87:0/90528324 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d400cc10 con 0x7f49e4070c20 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e4158550 con 0x7f49e4070c20 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49db400640 1 -- 172.21.15.87:0/90528324 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4014440 con 0x7f49e4070c20 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.268+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e4158b10 con 0x7f49e4070c20 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.272+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e404ef30 con 0x7f49e4070c20 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.272+0000 7f49db400640 1 -- 172.21.15.87:0/90528324 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49d401c070 con 0x7f49e4070c20 2024-04-25T10:18:42.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.272+0000 7f49db400640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f49c0066f60 0x7f49c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.272+0000 7f49db400640 1 -- 172.21.15.87:0/90528324 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49d4084ee0 con 0x7f49e4070c20 2024-04-25T10:18:42.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.272+0000 7f49da000640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f49c0066f60 0x7f49c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.272+0000 7f49da000640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f49c0066f60 0x7f49c0069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f49e4157820 tx=0x7f49cc009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.276+0000 7f49db400640 1 -- 172.21.15.87:0/90528324 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49d40556f0 con 0x7f49e4070c20 2024-04-25T10:18:42.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.460+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49e40b3fb0 con 0x7f49e4070c20 2024-04-25T10:18:42.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49db400640 1 -- 172.21.15.87:0/90528324 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f49d4059790 con 0x7f49e4070c20 2024-04-25T10:18:42.466 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:42.466 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:18:42.466 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f49c0066f60 msgr2=0x7f49c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f49c0066f60 0x7f49c0069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f49e4157820 tx=0x7f49cc009110 comp rx=0 tx=0).stop 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e4070c20 msgr2=0x7f49e4157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e4070c20 0x7f49e4157890 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f49d400ba50 tx=0x7f49d400bf10 comp rx=0 tx=0).stop 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 shutdown_connections 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f49c0066f60 0x7f49c0069420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e40cf930 0x7f49e40b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49e40ae020 0x7f49e40b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 --2- 172.21.15.87:0/90528324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49e4070c20 0x7f49e4157890 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 >> 172.21.15.87:0/90528324 conn(0x7f49e4066b70 msgr2=0x7f49e415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 shutdown_connections 2024-04-25T10:18:42.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.464+0000 7f49e9600640 1 -- 172.21.15.87:0/90528324 wait complete. 2024-04-25T10:18:42.487 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:42.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a8600640 1 Processor -- start 2024-04-25T10:18:42.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a8600640 1 -- start start 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a8600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6a00ccb20 0x7fa6a00d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a015bdc0 0x7fa6a00d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a8600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00ccfb0 con 0x7fa6a015b9e0 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a8600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a00cd290 con 0x7fa6a00ccb20 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00cd570 con 0x7fa6a015bdc0 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a5c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa6a015b9e0 0x7fa6a00cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34302/0 (socket says 172.21.15.87:34302) 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a5c00640 1 -- 172.21.15.87:0/2658335471 learned_addr learned my addr 172.21.15.87:0/2658335471 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a5200640 1 --2- 172.21.15.87:0/2658335471 >> v2:172.21.15.87:3300/0 conn(0x7fa6a00ccb20 0x7fa6a00d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a6600640 1 --2- 172.21.15.87:0/2658335471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a015bdc0 0x7fa6a00d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a5200640 1 -- 172.21.15.87:0/2658335471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a015bdc0 msgr2=0x7fa6a00d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a5200640 1 --2- 172.21.15.87:0/2658335471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a015bdc0 0x7fa6a00d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a5200640 1 -- 172.21.15.87:0/2658335471 >> v1:172.21.15.87:6789/0 conn(0x7fa6a015b9e0 legacy=0x7fa6a00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa69fe00640 1 -- 172.21.15.87:0/2658335471 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691807137 0 0) 0x7fa6a00ccfb0 con 0x7fa6a015b9e0 2024-04-25T10:18:42.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.620+0000 7fa6a5200640 1 -- 172.21.15.87:0/2658335471 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00cd850 con 0x7fa6a00ccb20 2024-04-25T10:18:42.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a5200640 1 --2- 172.21.15.87:0/2658335471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ccb20 0x7fa6a00d1520 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0097c0 tx=0x7fa68c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d97b2204e2b62a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa69fe00640 1 -- 172.21.15.87:0/2658335471 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0b0070 con 0x7fa6a00ccb20 2024-04-25T10:18:42.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa69fe00640 1 -- 172.21.15.87:0/2658335471 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c0ab600 con 0x7fa6a00ccb20 2024-04-25T10:18:42.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa69fe00640 1 -- 172.21.15.87:0/2658335471 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0ab9d0 con 0x7fa6a00ccb20 2024-04-25T10:18:42.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 -- 172.21.15.87:0/2658335471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ccb20 msgr2=0x7fa6a00d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2658335471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ccb20 0x7fa6a00d1520 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0097c0 tx=0x7fa68c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 -- 172.21.15.87:0/2658335471 shutdown_connections 2024-04-25T10:18:42.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2658335471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a015bdc0 0x7fa6a00d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2658335471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ccb20 0x7fa6a00d1520 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 -- 172.21.15.87:0/2658335471 >> 172.21.15.87:0/2658335471 conn(0x7fa6a009a9d0 msgr2=0x7fa6a009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 -- 172.21.15.87:0/2658335471 shutdown_connections 2024-04-25T10:18:42.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 -- 172.21.15.87:0/2658335471 wait complete. 2024-04-25T10:18:42.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 Processor -- start 2024-04-25T10:18:42.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.624+0000 7fa6a8600640 1 -- start start 2024-04-25T10:18:42.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a015b9e0 0x7fa6a01643f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a015bdc0 0x7fa6a0153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a00c6b80 0x7fa6a00c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00c65f0 con 0x7fa6a015b9e0 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00c6470 con 0x7fa6a015bdc0 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00c6770 con 0x7fa6a00c6b80 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a015b9e0 0x7fa6a01643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a015b9e0 0x7fa6a01643f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39436/0 (socket says 172.21.15.87:39436) 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 -- 172.21.15.87:0/2446001220 learned_addr learned my addr 172.21.15.87:0/2446001220 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5200640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a015bdc0 0x7fa6a0153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a6600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a00c6b80 0x7fa6a00c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 -- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a00c6b80 msgr2=0x7fa6a00c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a00c6b80 0x7fa6a00c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 -- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a015bdc0 msgr2=0x7fa6a0153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a015bdc0 0x7fa6a0153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 -- 172.21.15.87:0/2446001220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a0165580 con 0x7fa6a015b9e0 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a6600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a00c6b80 0x7fa6a00c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5200640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a015bdc0 0x7fa6a0153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a5c00640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a015b9e0 0x7fa6a01643f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c580 tx=0x7fa69400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa69d600640 1 -- 172.21.15.87:0/2446001220 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694017070 con 0x7fa6a015b9e0 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa69d600640 1 -- 172.21.15.87:0/2446001220 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa694005d10 con 0x7fa6a015b9e0 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a0165860 con 0x7fa6a015b9e0 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa69d600640 1 -- 172.21.15.87:0/2446001220 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694007330 con 0x7fa6a015b9e0 2024-04-25T10:18:42.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a0165e20 con 0x7fa6a015b9e0 2024-04-25T10:18:42.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a004ef30 con 0x7fa6a015b9e0 2024-04-25T10:18:42.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.628+0000 7fa69d600640 1 -- 172.21.15.87:0/2446001220 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa69401a070 con 0x7fa6a015b9e0 2024-04-25T10:18:42.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.632+0000 7fa69d600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa680066f90 0x7fa680069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.632+0000 7fa69d600640 1 -- 172.21.15.87:0/2446001220 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa694084500 con 0x7fa6a015b9e0 2024-04-25T10:18:42.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.632+0000 7fa6a5200640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa680066f90 0x7fa680069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.632+0000 7fa6a5200640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa680066f90 0x7fa680069450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0ad000 tx=0x7fa68c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.636+0000 7fa69d600640 1 -- 172.21.15.87:0/2446001220 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa694055480 con 0x7fa6a015b9e0 2024-04-25T10:18:42.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.788+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6a00c98e0 con 0x7fa6a015b9e0 2024-04-25T10:18:42.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa69d600640 1 -- 172.21.15.87:0/2446001220 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16648 (secure 0 0 0) 0x7fa694054d80 con 0x7fa6a015b9e0 2024-04-25T10:18:42.794 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:42.794 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":87,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:18:40.073458+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:36.789308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:37.439968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa680066f90 msgr2=0x7fa680069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa680066f90 0x7fa680069450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0ad000 tx=0x7fa68c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a015b9e0 msgr2=0x7fa6a01643f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a015b9e0 0x7fa6a01643f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c580 tx=0x7fa69400ca40 comp rx=0 tx=0).stop 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 shutdown_connections 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa680066f90 0x7fa680069450 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a00c6b80 0x7fa6a00c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a015bdc0 0x7fa6a0153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 --2- 172.21.15.87:0/2446001220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a015b9e0 0x7fa6a01643f0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 >> 172.21.15.87:0/2446001220 conn(0x7fa6a009a9d0 msgr2=0x7fa6a0154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 shutdown_connections 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.792+0000 7fa6a8600640 1 -- 172.21.15.87:0/2446001220 wait complete. 2024-04-25T10:18:42.815 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:18:42.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:42.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5da00640 1 Processor -- start 2024-04-25T10:18:42.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5da00640 1 -- start start 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca5806ac90 0x7fca580b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5806b2c0 0x7fca5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5da00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580b5780 con 0x7fca5815db40 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580b5550 con 0x7fca5806ac90 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b5a60 con 0x7fca5806b2c0 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca57e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fca5815db40 0x7fca5815df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34310/0 (socket says 172.21.15.87:34310) 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca57e00640 1 -- 172.21.15.87:0/3566392249 learned_addr learned my addr 172.21.15.87:0/3566392249 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca57400640 1 --2- 172.21.15.87:0/3566392249 >> v2:172.21.15.169:3300/0 conn(0x7fca5806ac90 0x7fca580b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca5d000640 1 --2- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5806b2c0 0x7fca5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca56a00640 1 -- 172.21.15.87:0/3566392249 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300078142 0 0) 0x7fca580b5780 con 0x7fca5815db40 2024-04-25T10:18:42.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca56a00640 1 -- 172.21.15.87:0/3566392249 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3c003530 con 0x7fca5815db40 2024-04-25T10:18:42.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca57400640 1 -- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5806b2c0 msgr2=0x7fca5806b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca57400640 1 --2- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5806b2c0 0x7fca5806b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca57400640 1 -- 172.21.15.87:0/3566392249 >> v1:172.21.15.87:6789/0 conn(0x7fca5815db40 legacy=0x7fca5815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.948+0000 7fca57400640 1 -- 172.21.15.87:0/3566392249 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580b5d40 con 0x7fca5806ac90 2024-04-25T10:18:42.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca57400640 1 --2- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b5010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca440097c0 tx=0x7fca4401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c31f3ee566b7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca56a00640 1 -- 172.21.15.87:0/3566392249 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca440b5070 con 0x7fca5806ac90 2024-04-25T10:18:42.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca56a00640 1 -- 172.21.15.87:0/3566392249 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca440a8990 con 0x7fca5806ac90 2024-04-25T10:18:42.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca56a00640 1 -- 172.21.15.87:0/3566392249 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca440a8d60 con 0x7fca5806ac90 2024-04-25T10:18:42.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 -- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 msgr2=0x7fca580b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 --2- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b5010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca440097c0 tx=0x7fca4401ec90 comp rx=0 tx=0).stop 2024-04-25T10:18:42.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 -- 172.21.15.87:0/3566392249 shutdown_connections 2024-04-25T10:18:42.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 --2- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5806b2c0 0x7fca5806b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 --2- 172.21.15.87:0/3566392249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b5010 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 -- 172.21.15.87:0/3566392249 >> 172.21.15.87:0/3566392249 conn(0x7fca580d17b0 msgr2=0x7fca580d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 -- 172.21.15.87:0/3566392249 shutdown_connections 2024-04-25T10:18:42.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.952+0000 7fca5da00640 1 -- 172.21.15.87:0/3566392249 wait complete. 2024-04-25T10:18:42.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 Processor -- start 2024-04-25T10:18:42.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 -- start start 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca5806b2c0 0x7fca580c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5815db40 0x7fca580ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b6820 con 0x7fca5806b2c0 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b6300 con 0x7fca5806ac90 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b69a0 con 0x7fca5815db40 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca5806b2c0 0x7fca580c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca5806b2c0 0x7fca580c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39448/0 (socket says 172.21.15.87:39448) 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 -- 172.21.15.87:0/242017557 learned_addr learned my addr 172.21.15.87:0/242017557 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57e00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5d000640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5815db40 0x7fca580ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 -- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5815db40 msgr2=0x7fca580ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5815db40 0x7fca580ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 -- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 msgr2=0x7fca580b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 -- 172.21.15.87:0/242017557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580c71a0 con 0x7fca5806b2c0 2024-04-25T10:18:42.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57e00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:42.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca57400640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca5806b2c0 0x7fca580c6a50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fca440b3040 tx=0x7fca4401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca4fe00640 1 -- 172.21.15.87:0/242017557 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca440bf070 con 0x7fca5806b2c0 2024-04-25T10:18:42.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca4fe00640 1 -- 172.21.15.87:0/242017557 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca440b0700 con 0x7fca5806b2c0 2024-04-25T10:18:42.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca580c7480 con 0x7fca5806b2c0 2024-04-25T10:18:42.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca4fe00640 1 -- 172.21.15.87:0/242017557 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca440b5040 con 0x7fca5806b2c0 2024-04-25T10:18:42.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca580c55b0 con 0x7fca5806b2c0 2024-04-25T10:18:42.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.956+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca58075120 con 0x7fca5806b2c0 2024-04-25T10:18:42.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.960+0000 7fca4fe00640 1 -- 172.21.15.87:0/242017557 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca440c2070 con 0x7fca5806b2c0 2024-04-25T10:18:42.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.960+0000 7fca4fe00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca38066f60 0x7fca38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.960+0000 7fca4fe00640 1 -- 172.21.15.87:0/242017557 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca44121700 con 0x7fca5806b2c0 2024-04-25T10:18:42.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.960+0000 7fca57e00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca38066f60 0x7fca38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.960+0000 7fca57e00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca38066f60 0x7fca38069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fca48005890 tx=0x7fca48009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:42.964+0000 7fca4fe00640 1 -- 172.21.15.87:0/242017557 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca440187f0 con 0x7fca5806b2c0 2024-04-25T10:18:43.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.116+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca5806d380 con 0x7fca38066f60 2024-04-25T10:18:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.120+0000 7fca4fe00640 1 -- 172.21.15.87:0/242017557 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237296 (secure 0 0 0) 0x7fca5806d380 con 0x7fca38066f60 2024-04-25T10:18:43.128 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:18:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca38066f60 msgr2=0x7fca38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca38066f60 0x7fca38069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fca48005890 tx=0x7fca48009040 comp rx=0 tx=0).stop 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca5806b2c0 msgr2=0x7fca580c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca5806b2c0 0x7fca580c6a50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fca440b3040 tx=0x7fca4401ba30 comp rx=0 tx=0).stop 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 shutdown_connections 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca38066f60 0x7fca38069420 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5815db40 0x7fca580ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca5806b2c0 0x7fca580c6a50 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 --2- 172.21.15.87:0/242017557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5806ac90 0x7fca580b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 >> 172.21.15.87:0/242017557 conn(0x7fca580d17b0 msgr2=0x7fca5806f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 shutdown_connections 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.124+0000 7fca5da00640 1 -- 172.21.15.87:0/242017557 wait complete. 2024-04-25T10:18:43.148 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-04-25T10:18:41.206039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":397424,"kb_used_data":9128,"kb_used_omap":12,"kb_used_meta":388147,"kb_avail":754577296,"statfs":{"total":773094113280,"available":772687151104,"internally_reserved":0,"allocated":9347072,"data_stored":4664152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":397463089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.153174"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:38.050835+0000","last_change":"2024-04-25T10:18:38.050835+0000","last_active":"2024-04-25T10:18:38.050835+0000","last_peered":"2024-04-25T10:18:38.050835+0000","last_clean":"2024-04-25T10:18:38.050835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:38.050835+0000","last_undegraded":"2024-04-25T10:18:38.050835+0000","last_fullsized":"2024-04-25T10:18:38.050835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:37.046397+0000","last_change":"2024-04-25T10:18:37.046397+0000","last_active":"2024-04-25T10:18:37.046397+0000","last_peered":"2024-04-25T10:18:37.046397+0000","last_clean":"2024-04-25T10:18:37.046397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:37.046397+0000","last_undegraded":"2024-04-25T10:18:37.046397+0000","last_fullsized":"2024-04-25T10:18:37.046397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T10:18:35.050802+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:18:35.050802+0000","last_peered":"2024-04-25T10:18:35.050802+0000","last_clean":"2024-04-25T10:18:35.050802+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:18:35.050802+0000","last_undegraded":"2024-04-25T10:18:35.050802+0000","last_fullsized":"2024-04-25T10:18:35.050802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50804,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1613824,"data_stored":1025795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50224,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321616,"statfs":{"total":96636764160,"available":96585334784,"internally_reserved":0,"allocated":1019904,"data_stored":435427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40504,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331336,"statfs":{"total":96636764160,"available":96595288064,"internally_reserved":0,"allocated":1019904,"data_stored":435427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50236,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321604,"statfs":{"total":96636764160,"available":96585322496,"internally_reserved":0,"allocated":1019904,"data_stored":435427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46064,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325776,"statfs":{"total":96636764160,"available":96589594624,"internally_reserved":0,"allocated":1019904,"data_stored":435427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40500,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331340,"statfs":{"total":96636764160,"available":96595292160,"internally_reserved":0,"allocated":1019904,"data_stored":435427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72444,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299396,"statfs":{"total":96636764160,"available":96562581504,"internally_reserved":0,"allocated":1019904,"data_stored":435427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46648,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325192,"statfs":{"total":96636764160,"available":96588996608,"internally_reserved":0,"allocated":1613824,"data_stored":1025795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:43.152 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:43.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.320+0000 7fc3b3200640 1 Processor -- start 2024-04-25T10:18:43.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.320+0000 7fc3b3200640 1 -- start start 2024-04-25T10:18:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.320+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.320+0000 7fc3b3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3b40670b0 0x7fc3b4161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.320+0000 7fc3b3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b40ad840 con 0x7fc3b40af380 2024-04-25T10:18:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.320+0000 7fc3b3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b406e7a0 con 0x7fc3b40670b0 2024-04-25T10:18:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.320+0000 7fc3b3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40af9b0 con 0x7fc3b4157f40 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3b40670b0 0x7fc3b4161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3a9400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc3b40af380 0x7fc3b40669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:55386/0 (socket says 172.21.15.87:55386) 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3a9400640 1 -- 172.21.15.87:0/3575320316 learned_addr learned my addr 172.21.15.87:0/3575320316 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1e00640 1 -- 172.21.15.87:0/3575320316 >> v2:172.21.15.169:3300/0 conn(0x7fc3b40670b0 msgr2=0x7fc3b4161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1e00640 1 --2- 172.21.15.87:0/3575320316 >> v2:172.21.15.169:3300/0 conn(0x7fc3b40670b0 0x7fc3b4161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1e00640 1 -- 172.21.15.87:0/3575320316 >> v1:172.21.15.169:6789/0 conn(0x7fc3b40af380 legacy=0x7fc3b40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1400640 1 -- 172.21.15.87:0/3575320316 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188068843 0 0) 0x7fc39c001120 con 0x7fc3b40af380 2024-04-25T10:18:43.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1e00640 1 -- 172.21.15.87:0/3575320316 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b415e120 con 0x7fc3b4157f40 2024-04-25T10:18:43.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1e00640 1 --2- 172.21.15.87:0/3575320316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40aee40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097c0 tx=0x7fc3a401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8ca08c42cebb69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1400640 1 -- 172.21.15.87:0/3575320316 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40b2070 con 0x7fc3b4157f40 2024-04-25T10:18:43.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1400640 1 -- 172.21.15.87:0/3575320316 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a40ad640 con 0x7fc3b4157f40 2024-04-25T10:18:43.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.324+0000 7fc3b1400640 1 -- 172.21.15.87:0/3575320316 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40ada10 con 0x7fc3b4157f40 2024-04-25T10:18:43.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- 172.21.15.87:0/3575320316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 msgr2=0x7fc3b40aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 --2- 172.21.15.87:0/3575320316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40aee40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097c0 tx=0x7fc3a401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- 172.21.15.87:0/3575320316 shutdown_connections 2024-04-25T10:18:43.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 --2- 172.21.15.87:0/3575320316 >> v2:172.21.15.169:3300/0 conn(0x7fc3b40670b0 0x7fc3b4161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 --2- 172.21.15.87:0/3575320316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40aee40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- 172.21.15.87:0/3575320316 >> 172.21.15.87:0/3575320316 conn(0x7fc3b4074310 msgr2=0x7fc3b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- 172.21.15.87:0/3575320316 shutdown_connections 2024-04-25T10:18:43.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- 172.21.15.87:0/3575320316 wait complete. 2024-04-25T10:18:43.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 Processor -- start 2024-04-25T10:18:43.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- start start 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b40670b0 0x7fc3b40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3b40af380 0x7fc3b414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4158320 con 0x7fc3b40af380 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b415eee0 con 0x7fc3b40670b0 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b41584a0 con 0x7fc3b4157f40 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3b40af380 0x7fc3b414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3b40af380 0x7fc3b414d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39464/0 (socket says 172.21.15.87:39464) 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 -- 172.21.15.87:0/69558624 learned_addr learned my addr 172.21.15.87:0/69558624 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b1e00640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b40670b0 0x7fc3b40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3b2800640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 -- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 msgr2=0x7fc3b40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 -- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b40670b0 msgr2=0x7fc3b40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b40670b0 0x7fc3b40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.328+0000 7fc3a9400640 1 -- 172.21.15.87:0/69558624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b4152850 con 0x7fc3b40af380 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3a9400640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3b40af380 0x7fc3b414d2c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c7b0 tx=0x7fc39c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3aaa00640 1 -- 172.21.15.87:0/69558624 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c018070 con 0x7fc3b40af380 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3aaa00640 1 -- 172.21.15.87:0/69558624 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c0072f0 con 0x7fc3b40af380 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3aaa00640 1 -- 172.21.15.87:0/69558624 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0047f0 con 0x7fc3b40af380 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b4152b30 con 0x7fc3b40af380 2024-04-25T10:18:43.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b41530f0 con 0x7fc3b40af380 2024-04-25T10:18:43.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b4075120 con 0x7fc3b40af380 2024-04-25T10:18:43.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3aaa00640 1 -- 172.21.15.87:0/69558624 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc39c004990 con 0x7fc3b40af380 2024-04-25T10:18:43.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3aaa00640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc390066f60 0x7fc390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3aaa00640 1 -- 172.21.15.87:0/69558624 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc39c084440 con 0x7fc3b40af380 2024-04-25T10:18:43.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.332+0000 7fc3b1e00640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc390066f60 0x7fc390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.336+0000 7fc3b1e00640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc390066f60 0x7fc390069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40af000 tx=0x7fc3a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.336+0000 7fc3aaa00640 1 -- 172.21.15.87:0/69558624 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3b40ad840 con 0x7fc3b40af380 2024-04-25T10:18:43.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.488+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3b40b4e30 con 0x7fc390066f60 2024-04-25T10:18:43.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.492+0000 7fc3aaa00640 1 -- 172.21.15.87:0/69558624 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252084 (secure 0 0 0) 0x7fc3b40b4e30 con 0x7fc390066f60 2024-04-25T10:18:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:43.500 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:18:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc390066f60 msgr2=0x7fc390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc390066f60 0x7fc390069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40af000 tx=0x7fc3a4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3b40af380 msgr2=0x7fc3b414d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3b40af380 0x7fc3b414d2c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c7b0 tx=0x7fc39c00cc70 comp rx=0 tx=0).stop 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 shutdown_connections 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc390066f60 0x7fc390069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b4157f40 0x7fc3b40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3b40af380 0x7fc3b414d2c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 --2- 172.21.15.87:0/69558624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b40670b0 0x7fc3b40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 >> 172.21.15.87:0/69558624 conn(0x7fc3b4074310 msgr2=0x7fc3b40b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 shutdown_connections 2024-04-25T10:18:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.496+0000 7fc3b3200640 1 -- 172.21.15.87:0/69558624 wait complete. 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-04-25T10:18:43.206350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402008,"kb_used_data":9576,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754572712,"statfs":{"total":773094113280,"available":772682457088,"internally_reserved":0,"allocated":9805824,"data_stored":4870308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":401722929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000920"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266856+0000","last_change":"2024-04-25T10:18:39.082048+0000","last_active":"2024-04-25T10:18:40.266856+0000","last_peered":"2024-04-25T10:18:40.266856+0000","last_clean":"2024-04-25T10:18:40.266856+0000","last_became_active":"2024-04-25T10:18:39.081280+0000","last_became_peered":"2024-04-25T10:18:39.081280+0000","last_unstale":"2024-04-25T10:18:40.266856+0000","last_undegraded":"2024-04-25T10:18:40.266856+0000","last_fullsized":"2024-04-25T10:18:40.266856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:16.919229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566454+0000","last_change":"2024-04-25T10:18:39.088766+0000","last_active":"2024-04-25T10:18:40.566454+0000","last_peered":"2024-04-25T10:18:40.566454+0000","last_clean":"2024-04-25T10:18:40.566454+0000","last_became_active":"2024-04-25T10:18:39.088510+0000","last_became_peered":"2024-04-25T10:18:39.088510+0000","last_unstale":"2024-04-25T10:18:40.566454+0000","last_undegraded":"2024-04-25T10:18:40.566454+0000","last_fullsized":"2024-04-25T10:18:40.566454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:55.419836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.089944+0000","last_change":"2024-04-25T10:18:39.095788+0000","last_active":"2024-04-25T10:18:40.089944+0000","last_peered":"2024-04-25T10:18:40.089944+0000","last_clean":"2024-04-25T10:18:40.089944+0000","last_became_active":"2024-04-25T10:18:39.094281+0000","last_became_peered":"2024-04-25T10:18:39.094281+0000","last_unstale":"2024-04-25T10:18:40.089944+0000","last_undegraded":"2024-04-25T10:18:40.089944+0000","last_fullsized":"2024-04-25T10:18:40.089944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:37.282328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567066+0000","last_change":"2024-04-25T10:18:39.089078+0000","last_active":"2024-04-25T10:18:40.567066+0000","last_peered":"2024-04-25T10:18:40.567066+0000","last_clean":"2024-04-25T10:18:40.567066+0000","last_became_active":"2024-04-25T10:18:39.087346+0000","last_became_peered":"2024-04-25T10:18:39.087346+0000","last_unstale":"2024-04-25T10:18:40.567066+0000","last_undegraded":"2024-04-25T10:18:40.567066+0000","last_fullsized":"2024-04-25T10:18:40.567066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:36.839684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567727+0000","last_change":"2024-04-25T10:18:39.084405+0000","last_active":"2024-04-25T10:18:40.567727+0000","last_peered":"2024-04-25T10:18:40.567727+0000","last_clean":"2024-04-25T10:18:40.567727+0000","last_became_active":"2024-04-25T10:18:39.083609+0000","last_became_peered":"2024-04-25T10:18:39.083609+0000","last_unstale":"2024-04-25T10:18:40.567727+0000","last_undegraded":"2024-04-25T10:18:40.567727+0000","last_fullsized":"2024-04-25T10:18:40.567727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:45.920495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090139+0000","last_change":"2024-04-25T10:18:39.079836+0000","last_active":"2024-04-25T10:18:40.090139+0000","last_peered":"2024-04-25T10:18:40.090139+0000","last_clean":"2024-04-25T10:18:40.090139+0000","last_became_active":"2024-04-25T10:18:39.079524+0000","last_became_peered":"2024-04-25T10:18:39.079524+0000","last_unstale":"2024-04-25T10:18:40.090139+0000","last_undegraded":"2024-04-25T10:18:40.090139+0000","last_fullsized":"2024-04-25T10:18:40.090139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:22.583587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567025+0000","last_change":"2024-04-25T10:18:39.087812+0000","last_active":"2024-04-25T10:18:40.567025+0000","last_peered":"2024-04-25T10:18:40.567025+0000","last_clean":"2024-04-25T10:18:40.567025+0000","last_became_active":"2024-04-25T10:18:39.087484+0000","last_became_peered":"2024-04-25T10:18:39.087484+0000","last_unstale":"2024-04-25T10:18:40.567025+0000","last_undegraded":"2024-04-25T10:18:40.567025+0000","last_fullsized":"2024-04-25T10:18:40.567025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:26.154416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090124+0000","last_change":"2024-04-25T10:18:39.083317+0000","last_active":"2024-04-25T10:18:40.090124+0000","last_peered":"2024-04-25T10:18:40.090124+0000","last_clean":"2024-04-25T10:18:40.090124+0000","last_became_active":"2024-04-25T10:18:39.081012+0000","last_became_peered":"2024-04-25T10:18:39.081012+0000","last_unstale":"2024-04-25T10:18:40.090124+0000","last_undegraded":"2024-04-25T10:18:40.090124+0000","last_fullsized":"2024-04-25T10:18:40.090124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:58.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090223+0000","last_change":"2024-04-25T10:18:39.091182+0000","last_active":"2024-04-25T10:18:40.090223+0000","last_peered":"2024-04-25T10:18:40.090223+0000","last_clean":"2024-04-25T10:18:40.090223+0000","last_became_active":"2024-04-25T10:18:39.090913+0000","last_became_peered":"2024-04-25T10:18:39.090913+0000","last_unstale":"2024-04-25T10:18:40.090223+0000","last_undegraded":"2024-04-25T10:18:40.090223+0000","last_fullsized":"2024-04-25T10:18:40.090223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:46.535043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267226+0000","last_change":"2024-04-25T10:18:39.089263+0000","last_active":"2024-04-25T10:18:40.267226+0000","last_peered":"2024-04-25T10:18:40.267226+0000","last_clean":"2024-04-25T10:18:40.267226+0000","last_became_active":"2024-04-25T10:18:39.088802+0000","last_became_peered":"2024-04-25T10:18:39.088802+0000","last_unstale":"2024-04-25T10:18:40.267226+0000","last_undegraded":"2024-04-25T10:18:40.267226+0000","last_fullsized":"2024-04-25T10:18:40.267226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:46.040542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566314+0000","last_change":"2024-04-25T10:18:39.085869+0000","last_active":"2024-04-25T10:18:40.566314+0000","last_peered":"2024-04-25T10:18:40.566314+0000","last_clean":"2024-04-25T10:18:40.566314+0000","last_became_active":"2024-04-25T10:18:39.085576+0000","last_became_peered":"2024-04-25T10:18:39.085576+0000","last_unstale":"2024-04-25T10:18:40.566314+0000","last_undegraded":"2024-04-25T10:18:40.566314+0000","last_fullsized":"2024-04-25T10:18:40.566314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:09.407907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266726+0000","last_change":"2024-04-25T10:18:39.083857+0000","last_active":"2024-04-25T10:18:40.266726+0000","last_peered":"2024-04-25T10:18:40.266726+0000","last_clean":"2024-04-25T10:18:40.266726+0000","last_became_active":"2024-04-25T10:18:39.082641+0000","last_became_peered":"2024-04-25T10:18:39.082641+0000","last_unstale":"2024-04-25T10:18:40.266726+0000","last_undegraded":"2024-04-25T10:18:40.266726+0000","last_fullsized":"2024-04-25T10:18:40.266726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:22.583587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090493+0000","last_change":"2024-04-25T10:18:39.086579+0000","last_active":"2024-04-25T10:18:40.090493+0000","last_peered":"2024-04-25T10:18:40.090493+0000","last_clean":"2024-04-25T10:18:40.090493+0000","last_became_active":"2024-04-25T10:18:39.086267+0000","last_became_peered":"2024-04-25T10:18:39.086267+0000","last_unstale":"2024-04-25T10:18:40.090493+0000","last_undegraded":"2024-04-25T10:18:40.090493+0000","last_fullsized":"2024-04-25T10:18:40.090493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:48.929567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566909+0000","last_change":"2024-04-25T10:18:39.089688+0000","last_active":"2024-04-25T10:18:40.566909+0000","last_peered":"2024-04-25T10:18:40.566909+0000","last_clean":"2024-04-25T10:18:40.566909+0000","last_became_active":"2024-04-25T10:18:39.089430+0000","last_became_peered":"2024-04-25T10:18:39.089430+0000","last_unstale":"2024-04-25T10:18:40.566909+0000","last_undegraded":"2024-04-25T10:18:40.566909+0000","last_fullsized":"2024-04-25T10:18:40.566909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:07.396487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.080870+0000","last_change":"2024-04-25T10:18:39.086450+0000","last_active":"2024-04-25T10:18:40.080870+0000","last_peered":"2024-04-25T10:18:40.080870+0000","last_clean":"2024-04-25T10:18:40.080870+0000","last_became_active":"2024-04-25T10:18:39.085887+0000","last_became_peered":"2024-04-25T10:18:39.085887+0000","last_unstale":"2024-04-25T10:18:40.080870+0000","last_undegraded":"2024-04-25T10:18:40.080870+0000","last_fullsized":"2024-04-25T10:18:40.080870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:43.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567736+0000","last_change":"2024-04-25T10:18:39.077110+0000","last_active":"2024-04-25T10:18:40.567736+0000","last_peered":"2024-04-25T10:18:40.567736+0000","last_clean":"2024-04-25T10:18:40.567736+0000","last_became_active":"2024-04-25T10:18:39.076811+0000","last_became_peered":"2024-04-25T10:18:39.076811+0000","last_unstale":"2024-04-25T10:18:40.567736+0000","last_undegraded":"2024-04-25T10:18:40.567736+0000","last_fullsized":"2024-04-25T10:18:40.567736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:25.769701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.568095+0000","last_change":"2024-04-25T10:18:39.084516+0000","last_active":"2024-04-25T10:18:40.568095+0000","last_peered":"2024-04-25T10:18:40.568095+0000","last_clean":"2024-04-25T10:18:40.568095+0000","last_became_active":"2024-04-25T10:18:39.082697+0000","last_became_peered":"2024-04-25T10:18:39.082697+0000","last_unstale":"2024-04-25T10:18:40.568095+0000","last_undegraded":"2024-04-25T10:18:40.568095+0000","last_fullsized":"2024-04-25T10:18:40.568095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:49.470703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566734+0000","last_change":"2024-04-25T10:18:39.072632+0000","last_active":"2024-04-25T10:18:40.566734+0000","last_peered":"2024-04-25T10:18:40.566734+0000","last_clean":"2024-04-25T10:18:40.566734+0000","last_became_active":"2024-04-25T10:18:39.072394+0000","last_became_peered":"2024-04-25T10:18:39.072394+0000","last_unstale":"2024-04-25T10:18:40.566734+0000","last_undegraded":"2024-04-25T10:18:40.566734+0000","last_fullsized":"2024-04-25T10:18:40.566734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:34.416099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567331+0000","last_change":"2024-04-25T10:18:39.084087+0000","last_active":"2024-04-25T10:18:40.567331+0000","last_peered":"2024-04-25T10:18:40.567331+0000","last_clean":"2024-04-25T10:18:40.567331+0000","last_became_active":"2024-04-25T10:18:39.083822+0000","last_became_peered":"2024-04-25T10:18:39.083822+0000","last_unstale":"2024-04-25T10:18:40.567331+0000","last_undegraded":"2024-04-25T10:18:40.567331+0000","last_fullsized":"2024-04-25T10:18:40.567331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:31.145883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566925+0000","last_change":"2024-04-25T10:18:39.090452+0000","last_active":"2024-04-25T10:18:40.566925+0000","last_peered":"2024-04-25T10:18:40.566925+0000","last_clean":"2024-04-25T10:18:40.566925+0000","last_became_active":"2024-04-25T10:18:39.090171+0000","last_became_peered":"2024-04-25T10:18:39.090171+0000","last_unstale":"2024-04-25T10:18:40.566925+0000","last_undegraded":"2024-04-25T10:18:40.566925+0000","last_fullsized":"2024-04-25T10:18:40.566925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:42.797879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566828+0000","last_change":"2024-04-25T10:18:39.088062+0000","last_active":"2024-04-25T10:18:40.566828+0000","last_peered":"2024-04-25T10:18:40.566828+0000","last_clean":"2024-04-25T10:18:40.566828+0000","last_became_active":"2024-04-25T10:18:39.087779+0000","last_became_peered":"2024-04-25T10:18:39.087779+0000","last_unstale":"2024-04-25T10:18:40.566828+0000","last_undegraded":"2024-04-25T10:18:40.566828+0000","last_fullsized":"2024-04-25T10:18:40.566828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:20.955758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565583+0000","last_change":"2024-04-25T10:18:39.095120+0000","last_active":"2024-04-25T10:18:40.565583+0000","last_peered":"2024-04-25T10:18:40.565583+0000","last_clean":"2024-04-25T10:18:40.565583+0000","last_became_active":"2024-04-25T10:18:39.094868+0000","last_became_peered":"2024-04-25T10:18:39.094868+0000","last_unstale":"2024-04-25T10:18:40.565583+0000","last_undegraded":"2024-04-25T10:18:40.565583+0000","last_fullsized":"2024-04-25T10:18:40.565583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:08.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266755+0000","last_change":"2024-04-25T10:18:39.090904+0000","last_active":"2024-04-25T10:18:40.266755+0000","last_peered":"2024-04-25T10:18:40.266755+0000","last_clean":"2024-04-25T10:18:40.266755+0000","last_became_active":"2024-04-25T10:18:39.090572+0000","last_became_peered":"2024-04-25T10:18:39.090572+0000","last_unstale":"2024-04-25T10:18:40.266755+0000","last_undegraded":"2024-04-25T10:18:40.266755+0000","last_fullsized":"2024-04-25T10:18:40.266755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:21.204031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566047+0000","last_change":"2024-04-25T10:18:39.088519+0000","last_active":"2024-04-25T10:18:40.566047+0000","last_peered":"2024-04-25T10:18:40.566047+0000","last_clean":"2024-04-25T10:18:40.566047+0000","last_became_active":"2024-04-25T10:18:39.088221+0000","last_became_peered":"2024-04-25T10:18:39.088221+0000","last_unstale":"2024-04-25T10:18:40.566047+0000","last_undegraded":"2024-04-25T10:18:40.566047+0000","last_fullsized":"2024-04-25T10:18:40.566047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:12.099240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566424+0000","last_change":"2024-04-25T10:18:39.093639+0000","last_active":"2024-04-25T10:18:40.566424+0000","last_peered":"2024-04-25T10:18:40.566424+0000","last_clean":"2024-04-25T10:18:40.566424+0000","last_became_active":"2024-04-25T10:18:39.093199+0000","last_became_peered":"2024-04-25T10:18:39.093199+0000","last_unstale":"2024-04-25T10:18:40.566424+0000","last_undegraded":"2024-04-25T10:18:40.566424+0000","last_fullsized":"2024-04-25T10:18:40.566424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090059+0000","last_change":"2024-04-25T10:18:39.089345+0000","last_active":"2024-04-25T10:18:40.090059+0000","last_peered":"2024-04-25T10:18:40.090059+0000","last_clean":"2024-04-25T10:18:40.090059+0000","last_became_active":"2024-04-25T10:18:39.088840+0000","last_became_peered":"2024-04-25T10:18:39.088840+0000","last_unstale":"2024-04-25T10:18:40.090059+0000","last_undegraded":"2024-04-25T10:18:40.090059+0000","last_fullsized":"2024-04-25T10:18:40.090059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:25.148354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567672+0000","last_change":"2024-04-25T10:18:39.083059+0000","last_active":"2024-04-25T10:18:40.567672+0000","last_peered":"2024-04-25T10:18:40.567672+0000","last_clean":"2024-04-25T10:18:40.567672+0000","last_became_active":"2024-04-25T10:18:39.082743+0000","last_became_peered":"2024-04-25T10:18:39.082743+0000","last_unstale":"2024-04-25T10:18:40.567672+0000","last_undegraded":"2024-04-25T10:18:40.567672+0000","last_fullsized":"2024-04-25T10:18:40.567672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:31.145883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266734+0000","last_change":"2024-04-25T10:18:39.074841+0000","last_active":"2024-04-25T10:18:40.266734+0000","last_peered":"2024-04-25T10:18:40.266734+0000","last_clean":"2024-04-25T10:18:40.266734+0000","last_became_active":"2024-04-25T10:18:39.074536+0000","last_became_peered":"2024-04-25T10:18:39.074536+0000","last_unstale":"2024-04-25T10:18:40.266734+0000","last_undegraded":"2024-04-25T10:18:40.266734+0000","last_fullsized":"2024-04-25T10:18:40.266734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:12.424932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565958+0000","last_change":"2024-04-25T10:18:39.085902+0000","last_active":"2024-04-25T10:18:40.565958+0000","last_peered":"2024-04-25T10:18:40.565958+0000","last_clean":"2024-04-25T10:18:40.565958+0000","last_became_active":"2024-04-25T10:18:39.085633+0000","last_became_peered":"2024-04-25T10:18:39.085633+0000","last_unstale":"2024-04-25T10:18:40.565958+0000","last_undegraded":"2024-04-25T10:18:40.565958+0000","last_fullsized":"2024-04-25T10:18:40.565958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:07.052101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565714+0000","last_change":"2024-04-25T10:18:39.099331+0000","last_active":"2024-04-25T10:18:40.565714+0000","last_peered":"2024-04-25T10:18:40.565714+0000","last_clean":"2024-04-25T10:18:40.565714+0000","last_became_active":"2024-04-25T10:18:39.099081+0000","last_became_peered":"2024-04-25T10:18:39.099081+0000","last_unstale":"2024-04-25T10:18:40.565714+0000","last_undegraded":"2024-04-25T10:18:40.565714+0000","last_fullsized":"2024-04-25T10:18:40.565714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:56.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567517+0000","last_change":"2024-04-25T10:18:39.092357+0000","last_active":"2024-04-25T10:18:40.567517+0000","last_peered":"2024-04-25T10:18:40.567517+0000","last_clean":"2024-04-25T10:18:40.567517+0000","last_became_active":"2024-04-25T10:18:39.092062+0000","last_became_peered":"2024-04-25T10:18:39.092062+0000","last_unstale":"2024-04-25T10:18:40.567517+0000","last_undegraded":"2024-04-25T10:18:40.567517+0000","last_fullsized":"2024-04-25T10:18:40.567517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:16.919229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266677+0000","last_change":"2024-04-25T10:18:39.079342+0000","last_active":"2024-04-25T10:18:40.266677+0000","last_peered":"2024-04-25T10:18:40.266677+0000","last_clean":"2024-04-25T10:18:40.266677+0000","last_became_active":"2024-04-25T10:18:39.078848+0000","last_became_peered":"2024-04-25T10:18:39.078848+0000","last_unstale":"2024-04-25T10:18:40.266677+0000","last_undegraded":"2024-04-25T10:18:40.266677+0000","last_fullsized":"2024-04-25T10:18:40.266677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:46.535043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.568042+0000","last_change":"2024-04-25T10:18:39.076392+0000","last_active":"2024-04-25T10:18:40.568042+0000","last_peered":"2024-04-25T10:18:40.568042+0000","last_clean":"2024-04-25T10:18:40.568042+0000","last_became_active":"2024-04-25T10:18:39.076101+0000","last_became_peered":"2024-04-25T10:18:39.076101+0000","last_unstale":"2024-04-25T10:18:40.568042+0000","last_undegraded":"2024-04-25T10:18:40.568042+0000","last_fullsized":"2024-04-25T10:18:40.568042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.208973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267228+0000","last_change":"2024-04-25T10:18:38.086072+0000","last_active":"2024-04-25T10:18:40.267228+0000","last_peered":"2024-04-25T10:18:40.267228+0000","last_clean":"2024-04-25T10:18:40.267228+0000","last_became_active":"2024-04-25T10:18:38.085621+0000","last_became_peered":"2024-04-25T10:18:38.085621+0000","last_unstale":"2024-04-25T10:18:40.267228+0000","last_undegraded":"2024-04-25T10:18:40.267228+0000","last_fullsized":"2024-04-25T10:18:40.267228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:08.141253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267059+0000","last_change":"2024-04-25T10:18:39.090151+0000","last_active":"2024-04-25T10:18:40.267059+0000","last_peered":"2024-04-25T10:18:40.267059+0000","last_clean":"2024-04-25T10:18:40.267059+0000","last_became_active":"2024-04-25T10:18:39.089894+0000","last_became_peered":"2024-04-25T10:18:39.089894+0000","last_unstale":"2024-04-25T10:18:40.267059+0000","last_undegraded":"2024-04-25T10:18:40.267059+0000","last_fullsized":"2024-04-25T10:18:40.267059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:25.769701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.568140+0000","last_change":"2024-04-25T10:18:38.080653+0000","last_active":"2024-04-25T10:18:40.568140+0000","last_peered":"2024-04-25T10:18:40.568140+0000","last_clean":"2024-04-25T10:18:40.568140+0000","last_became_active":"2024-04-25T10:18:38.080368+0000","last_became_peered":"2024-04-25T10:18:38.080368+0000","last_unstale":"2024-04-25T10:18:40.568140+0000","last_undegraded":"2024-04-25T10:18:40.568140+0000","last_fullsized":"2024-04-25T10:18:40.568140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:16.640385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266949+0000","last_change":"2024-04-25T10:18:39.082655+0000","last_active":"2024-04-25T10:18:40.266949+0000","last_peered":"2024-04-25T10:18:40.266949+0000","last_clean":"2024-04-25T10:18:40.266949+0000","last_became_active":"2024-04-25T10:18:39.082356+0000","last_became_peered":"2024-04-25T10:18:39.082356+0000","last_unstale":"2024-04-25T10:18:40.266949+0000","last_undegraded":"2024-04-25T10:18:40.266949+0000","last_fullsized":"2024-04-25T10:18:40.266949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:31.145883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567220+0000","last_change":"2024-04-25T10:18:38.075055+0000","last_active":"2024-04-25T10:18:40.567220+0000","last_peered":"2024-04-25T10:18:40.567220+0000","last_clean":"2024-04-25T10:18:40.567220+0000","last_became_active":"2024-04-25T10:18:38.074776+0000","last_became_peered":"2024-04-25T10:18:38.074776+0000","last_unstale":"2024-04-25T10:18:40.567220+0000","last_undegraded":"2024-04-25T10:18:40.567220+0000","last_fullsized":"2024-04-25T10:18:40.567220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:48.316078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090101+0000","last_change":"2024-04-25T10:18:39.085841+0000","last_active":"2024-04-25T10:18:40.090101+0000","last_peered":"2024-04-25T10:18:40.090101+0000","last_clean":"2024-04-25T10:18:40.090101+0000","last_became_active":"2024-04-25T10:18:39.085463+0000","last_became_peered":"2024-04-25T10:18:39.085463+0000","last_unstale":"2024-04-25T10:18:40.090101+0000","last_undegraded":"2024-04-25T10:18:40.090101+0000","last_fullsized":"2024-04-25T10:18:40.090101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:43.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567005+0000","last_change":"2024-04-25T10:18:38.075514+0000","last_active":"2024-04-25T10:18:40.567005+0000","last_peered":"2024-04-25T10:18:40.567005+0000","last_clean":"2024-04-25T10:18:40.567005+0000","last_became_active":"2024-04-25T10:18:38.075013+0000","last_became_peered":"2024-04-25T10:18:38.075013+0000","last_unstale":"2024-04-25T10:18:40.567005+0000","last_undegraded":"2024-04-25T10:18:40.567005+0000","last_fullsized":"2024-04-25T10:18:40.567005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:09.481057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565850+0000","last_change":"2024-04-25T10:18:39.090152+0000","last_active":"2024-04-25T10:18:40.565850+0000","last_peered":"2024-04-25T10:18:40.565850+0000","last_clean":"2024-04-25T10:18:40.565850+0000","last_became_active":"2024-04-25T10:18:39.089817+0000","last_became_peered":"2024-04-25T10:18:39.089817+0000","last_unstale":"2024-04-25T10:18:40.565850+0000","last_undegraded":"2024-04-25T10:18:40.565850+0000","last_fullsized":"2024-04-25T10:18:40.565850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:47.630158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567893+0000","last_change":"2024-04-25T10:18:38.075808+0000","last_active":"2024-04-25T10:18:40.567893+0000","last_peered":"2024-04-25T10:18:40.567893+0000","last_clean":"2024-04-25T10:18:40.567893+0000","last_became_active":"2024-04-25T10:18:38.075508+0000","last_became_peered":"2024-04-25T10:18:38.075508+0000","last_unstale":"2024-04-25T10:18:40.567893+0000","last_undegraded":"2024-04-25T10:18:40.567893+0000","last_fullsized":"2024-04-25T10:18:40.567893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:48.316078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567693+0000","last_change":"2024-04-25T10:18:39.082487+0000","last_active":"2024-04-25T10:18:40.567693+0000","last_peered":"2024-04-25T10:18:40.567693+0000","last_clean":"2024-04-25T10:18:40.567693+0000","last_became_active":"2024-04-25T10:18:39.082223+0000","last_became_peered":"2024-04-25T10:18:39.082223+0000","last_unstale":"2024-04-25T10:18:40.567693+0000","last_undegraded":"2024-04-25T10:18:40.567693+0000","last_fullsized":"2024-04-25T10:18:40.567693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:30.783354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567581+0000","last_change":"2024-04-25T10:18:38.082112+0000","last_active":"2024-04-25T10:18:40.567581+0000","last_peered":"2024-04-25T10:18:40.567581+0000","last_clean":"2024-04-25T10:18:40.567581+0000","last_became_active":"2024-04-25T10:18:38.080784+0000","last_became_peered":"2024-04-25T10:18:38.080784+0000","last_unstale":"2024-04-25T10:18:40.567581+0000","last_undegraded":"2024-04-25T10:18:40.567581+0000","last_fullsized":"2024-04-25T10:18:40.567581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:40.854691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.089401+0000","last_change":"2024-04-25T10:18:39.090109+0000","last_active":"2024-04-25T10:18:40.089401+0000","last_peered":"2024-04-25T10:18:40.089401+0000","last_clean":"2024-04-25T10:18:40.089401+0000","last_became_active":"2024-04-25T10:18:39.089839+0000","last_became_peered":"2024-04-25T10:18:39.089839+0000","last_unstale":"2024-04-25T10:18:40.089401+0000","last_undegraded":"2024-04-25T10:18:40.089401+0000","last_fullsized":"2024-04-25T10:18:40.089401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.208973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565751+0000","last_change":"2024-04-25T10:18:38.075723+0000","last_active":"2024-04-25T10:18:40.565751+0000","last_peered":"2024-04-25T10:18:40.565751+0000","last_clean":"2024-04-25T10:18:40.565751+0000","last_became_active":"2024-04-25T10:18:38.074419+0000","last_became_peered":"2024-04-25T10:18:38.074419+0000","last_unstale":"2024-04-25T10:18:40.565751+0000","last_undegraded":"2024-04-25T10:18:40.565751+0000","last_fullsized":"2024-04-25T10:18:40.565751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:16.640385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.080931+0000","last_change":"2024-04-25T10:18:39.086692+0000","last_active":"2024-04-25T10:18:40.080931+0000","last_peered":"2024-04-25T10:18:40.080931+0000","last_clean":"2024-04-25T10:18:40.080931+0000","last_became_active":"2024-04-25T10:18:39.086188+0000","last_became_peered":"2024-04-25T10:18:39.086188+0000","last_unstale":"2024-04-25T10:18:40.080931+0000","last_undegraded":"2024-04-25T10:18:40.080931+0000","last_fullsized":"2024-04-25T10:18:40.080931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:41.090603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565371+0000","last_change":"2024-04-25T10:18:38.066159+0000","last_active":"2024-04-25T10:18:40.565371+0000","last_peered":"2024-04-25T10:18:40.565371+0000","last_clean":"2024-04-25T10:18:40.565371+0000","last_became_active":"2024-04-25T10:18:38.065416+0000","last_became_peered":"2024-04-25T10:18:38.065416+0000","last_unstale":"2024-04-25T10:18:40.565371+0000","last_undegraded":"2024-04-25T10:18:40.565371+0000","last_fullsized":"2024-04-25T10:18:40.565371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:23.781356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.081297+0000","last_change":"2024-04-25T10:18:39.079614+0000","last_active":"2024-04-25T10:18:40.081297+0000","last_peered":"2024-04-25T10:18:40.081297+0000","last_clean":"2024-04-25T10:18:40.081297+0000","last_became_active":"2024-04-25T10:18:39.079346+0000","last_became_peered":"2024-04-25T10:18:39.079346+0000","last_unstale":"2024-04-25T10:18:40.081297+0000","last_undegraded":"2024-04-25T10:18:40.081297+0000","last_fullsized":"2024-04-25T10:18:40.081297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:48.929567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267205+0000","last_change":"2024-04-25T10:18:38.080750+0000","last_active":"2024-04-25T10:18:40.267205+0000","last_peered":"2024-04-25T10:18:40.267205+0000","last_clean":"2024-04-25T10:18:40.267205+0000","last_became_active":"2024-04-25T10:18:38.080329+0000","last_became_peered":"2024-04-25T10:18:38.080329+0000","last_unstale":"2024-04-25T10:18:40.267205+0000","last_undegraded":"2024-04-25T10:18:40.267205+0000","last_fullsized":"2024-04-25T10:18:40.267205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:54.119752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090375+0000","last_change":"2024-04-25T10:18:39.079910+0000","last_active":"2024-04-25T10:18:40.090375+0000","last_peered":"2024-04-25T10:18:40.090375+0000","last_clean":"2024-04-25T10:18:40.090375+0000","last_became_active":"2024-04-25T10:18:39.079577+0000","last_became_peered":"2024-04-25T10:18:39.079577+0000","last_unstale":"2024-04-25T10:18:40.090375+0000","last_undegraded":"2024-04-25T10:18:40.090375+0000","last_fullsized":"2024-04-25T10:18:40.090375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:21.204031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267177+0000","last_change":"2024-04-25T10:18:38.082716+0000","last_active":"2024-04-25T10:18:40.267177+0000","last_peered":"2024-04-25T10:18:40.267177+0000","last_clean":"2024-04-25T10:18:40.267177+0000","last_became_active":"2024-04-25T10:18:38.082415+0000","last_became_peered":"2024-04-25T10:18:38.082415+0000","last_unstale":"2024-04-25T10:18:40.267177+0000","last_undegraded":"2024-04-25T10:18:40.267177+0000","last_fullsized":"2024-04-25T10:18:40.267177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:54.415398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566901+0000","last_change":"2024-04-25T10:18:39.092318+0000","last_active":"2024-04-25T10:18:40.566901+0000","last_peered":"2024-04-25T10:18:40.566901+0000","last_clean":"2024-04-25T10:18:40.566901+0000","last_became_active":"2024-04-25T10:18:39.091810+0000","last_became_peered":"2024-04-25T10:18:39.091810+0000","last_unstale":"2024-04-25T10:18:40.566901+0000","last_undegraded":"2024-04-25T10:18:40.566901+0000","last_fullsized":"2024-04-25T10:18:40.566901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:16.919229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266914+0000","last_change":"2024-04-25T10:18:38.074968+0000","last_active":"2024-04-25T10:18:40.266914+0000","last_peered":"2024-04-25T10:18:40.266914+0000","last_clean":"2024-04-25T10:18:40.266914+0000","last_became_active":"2024-04-25T10:18:38.074752+0000","last_became_peered":"2024-04-25T10:18:38.074752+0000","last_unstale":"2024-04-25T10:18:40.266914+0000","last_undegraded":"2024-04-25T10:18:40.266914+0000","last_fullsized":"2024-04-25T10:18:40.266914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.166443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567805+0000","last_change":"2024-04-25T10:18:39.092787+0000","last_active":"2024-04-25T10:18:40.567805+0000","last_peered":"2024-04-25T10:18:40.567805+0000","last_clean":"2024-04-25T10:18:40.567805+0000","last_became_active":"2024-04-25T10:18:39.091673+0000","last_became_peered":"2024-04-25T10:18:39.091673+0000","last_unstale":"2024-04-25T10:18:40.567805+0000","last_undegraded":"2024-04-25T10:18:40.567805+0000","last_fullsized":"2024-04-25T10:18:40.567805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:02.140427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.089902+0000","last_change":"2024-04-25T10:18:38.083984+0000","last_active":"2024-04-25T10:18:40.089902+0000","last_peered":"2024-04-25T10:18:40.089902+0000","last_clean":"2024-04-25T10:18:40.089902+0000","last_became_active":"2024-04-25T10:18:38.082615+0000","last_became_peered":"2024-04-25T10:18:38.082615+0000","last_unstale":"2024-04-25T10:18:40.089902+0000","last_undegraded":"2024-04-25T10:18:40.089902+0000","last_fullsized":"2024-04-25T10:18:40.089902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:00.827630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565983+0000","last_change":"2024-04-25T10:18:39.083095+0000","last_active":"2024-04-25T10:18:40.565983+0000","last_peered":"2024-04-25T10:18:40.565983+0000","last_clean":"2024-04-25T10:18:40.565983+0000","last_became_active":"2024-04-25T10:18:39.082840+0000","last_became_peered":"2024-04-25T10:18:39.082840+0000","last_unstale":"2024-04-25T10:18:40.565983+0000","last_undegraded":"2024-04-25T10:18:40.565983+0000","last_fullsized":"2024-04-25T10:18:40.565983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:12.424932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566065+0000","last_change":"2024-04-25T10:18:38.075602+0000","last_active":"2024-04-25T10:18:40.566065+0000","last_peered":"2024-04-25T10:18:40.566065+0000","last_clean":"2024-04-25T10:18:40.566065+0000","last_became_active":"2024-04-25T10:18:38.075327+0000","last_became_peered":"2024-04-25T10:18:38.075327+0000","last_unstale":"2024-04-25T10:18:40.566065+0000","last_undegraded":"2024-04-25T10:18:40.566065+0000","last_fullsized":"2024-04-25T10:18:40.566065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:35.835246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090591+0000","last_change":"2024-04-25T10:18:39.093600+0000","last_active":"2024-04-25T10:18:40.090591+0000","last_peered":"2024-04-25T10:18:40.090591+0000","last_clean":"2024-04-25T10:18:40.090591+0000","last_became_active":"2024-04-25T10:18:39.091276+0000","last_became_peered":"2024-04-25T10:18:39.091276+0000","last_unstale":"2024-04-25T10:18:40.090591+0000","last_undegraded":"2024-04-25T10:18:40.090591+0000","last_fullsized":"2024-04-25T10:18:40.090591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:18.437549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.080983+0000","last_change":"2024-04-25T10:18:38.083951+0000","last_active":"2024-04-25T10:18:40.080983+0000","last_peered":"2024-04-25T10:18:40.080983+0000","last_clean":"2024-04-25T10:18:40.080983+0000","last_became_active":"2024-04-25T10:18:38.083458+0000","last_became_peered":"2024-04-25T10:18:38.083458+0000","last_unstale":"2024-04-25T10:18:40.080983+0000","last_undegraded":"2024-04-25T10:18:40.080983+0000","last_fullsized":"2024-04-25T10:18:40.080983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.466265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565396+0000","last_change":"2024-04-25T10:18:39.099119+0000","last_active":"2024-04-25T10:18:40.565396+0000","last_peered":"2024-04-25T10:18:40.565396+0000","last_clean":"2024-04-25T10:18:40.565396+0000","last_became_active":"2024-04-25T10:18:39.098848+0000","last_became_peered":"2024-04-25T10:18:39.098848+0000","last_unstale":"2024-04-25T10:18:40.565396+0000","last_undegraded":"2024-04-25T10:18:40.565396+0000","last_fullsized":"2024-04-25T10:18:40.565396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:51.736693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090216+0000","last_change":"2024-04-25T10:18:38.071136+0000","last_active":"2024-04-25T10:18:40.090216+0000","last_peered":"2024-04-25T10:18:40.090216+0000","last_clean":"2024-04-25T10:18:40.090216+0000","last_became_active":"2024-04-25T10:18:38.070771+0000","last_became_peered":"2024-04-25T10:18:38.070771+0000","last_unstale":"2024-04-25T10:18:40.090216+0000","last_undegraded":"2024-04-25T10:18:40.090216+0000","last_fullsized":"2024-04-25T10:18:40.090216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:11.677147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565468+0000","last_change":"2024-04-25T10:18:39.093547+0000","last_active":"2024-04-25T10:18:40.565468+0000","last_peered":"2024-04-25T10:18:40.565468+0000","last_clean":"2024-04-25T10:18:40.565468+0000","last_became_active":"2024-04-25T10:18:39.093163+0000","last_became_peered":"2024-04-25T10:18:39.093163+0000","last_unstale":"2024-04-25T10:18:40.565468+0000","last_undegraded":"2024-04-25T10:18:40.565468+0000","last_fullsized":"2024-04-25T10:18:40.565468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:36.671278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.565603+0000","last_change":"2024-04-25T10:18:38.077629+0000","last_active":"2024-04-25T10:18:40.565603+0000","last_peered":"2024-04-25T10:18:40.565603+0000","last_clean":"2024-04-25T10:18:40.565603+0000","last_became_active":"2024-04-25T10:18:38.077302+0000","last_became_peered":"2024-04-25T10:18:38.077302+0000","last_unstale":"2024-04-25T10:18:40.565603+0000","last_undegraded":"2024-04-25T10:18:40.565603+0000","last_fullsized":"2024-04-25T10:18:40.565603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:52.995297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090214+0000","last_change":"2024-04-25T10:18:39.089135+0000","last_active":"2024-04-25T10:18:40.090214+0000","last_peered":"2024-04-25T10:18:40.090214+0000","last_clean":"2024-04-25T10:18:40.090214+0000","last_became_active":"2024-04-25T10:18:39.088466+0000","last_became_peered":"2024-04-25T10:18:39.088466+0000","last_unstale":"2024-04-25T10:18:40.090214+0000","last_undegraded":"2024-04-25T10:18:40.090214+0000","last_fullsized":"2024-04-25T10:18:40.090214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:41.090603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267354+0000","last_change":"2024-04-25T10:18:38.081066+0000","last_active":"2024-04-25T10:18:40.267354+0000","last_peered":"2024-04-25T10:18:40.267354+0000","last_clean":"2024-04-25T10:18:40.267354+0000","last_became_active":"2024-04-25T10:18:38.079254+0000","last_became_peered":"2024-04-25T10:18:38.079254+0000","last_unstale":"2024-04-25T10:18:40.267354+0000","last_undegraded":"2024-04-25T10:18:40.267354+0000","last_fullsized":"2024-04-25T10:18:40.267354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:12.724048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267084+0000","last_change":"2024-04-25T10:18:39.087321+0000","last_active":"2024-04-25T10:18:40.267084+0000","last_peered":"2024-04-25T10:18:40.267084+0000","last_clean":"2024-04-25T10:18:40.267084+0000","last_became_active":"2024-04-25T10:18:39.087021+0000","last_became_peered":"2024-04-25T10:18:39.087021+0000","last_unstale":"2024-04-25T10:18:40.267084+0000","last_undegraded":"2024-04-25T10:18:40.267084+0000","last_fullsized":"2024-04-25T10:18:40.267084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:02.140427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567165+0000","last_change":"2024-04-25T10:18:38.080528+0000","last_active":"2024-04-25T10:18:40.567165+0000","last_peered":"2024-04-25T10:18:40.567165+0000","last_clean":"2024-04-25T10:18:40.567165+0000","last_became_active":"2024-04-25T10:18:38.080276+0000","last_became_peered":"2024-04-25T10:18:38.080276+0000","last_unstale":"2024-04-25T10:18:40.567165+0000","last_undegraded":"2024-04-25T10:18:40.567165+0000","last_fullsized":"2024-04-25T10:18:40.567165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:11.677147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567959+0000","last_change":"2024-04-25T10:18:39.099778+0000","last_active":"2024-04-25T10:18:40.567959+0000","last_peered":"2024-04-25T10:18:40.567959+0000","last_clean":"2024-04-25T10:18:40.567959+0000","last_became_active":"2024-04-25T10:18:39.099462+0000","last_became_peered":"2024-04-25T10:18:39.099462+0000","last_unstale":"2024-04-25T10:18:40.567959+0000","last_undegraded":"2024-04-25T10:18:40.567959+0000","last_fullsized":"2024-04-25T10:18:40.567959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:46.040542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567063+0000","last_change":"2024-04-25T10:18:38.074672+0000","last_active":"2024-04-25T10:18:40.567063+0000","last_peered":"2024-04-25T10:18:40.567063+0000","last_clean":"2024-04-25T10:18:40.567063+0000","last_became_active":"2024-04-25T10:18:38.074401+0000","last_became_peered":"2024-04-25T10:18:38.074401+0000","last_unstale":"2024-04-25T10:18:40.567063+0000","last_undegraded":"2024-04-25T10:18:40.567063+0000","last_fullsized":"2024-04-25T10:18:40.567063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:57.734615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566962+0000","last_change":"2024-04-25T10:18:39.088967+0000","last_active":"2024-04-25T10:18:40.566962+0000","last_peered":"2024-04-25T10:18:40.566962+0000","last_clean":"2024-04-25T10:18:40.566962+0000","last_became_active":"2024-04-25T10:18:39.086998+0000","last_became_peered":"2024-04-25T10:18:39.086998+0000","last_unstale":"2024-04-25T10:18:40.566962+0000","last_undegraded":"2024-04-25T10:18:40.566962+0000","last_fullsized":"2024-04-25T10:18:40.566962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:24.785794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.081132+0000","last_change":"2024-04-25T10:18:38.073948+0000","last_active":"2024-04-25T10:18:40.081132+0000","last_peered":"2024-04-25T10:18:40.081132+0000","last_clean":"2024-04-25T10:18:40.081132+0000","last_became_active":"2024-04-25T10:18:38.073697+0000","last_became_peered":"2024-04-25T10:18:38.073697+0000","last_unstale":"2024-04-25T10:18:40.081132+0000","last_undegraded":"2024-04-25T10:18:40.081132+0000","last_fullsized":"2024-04-25T10:18:40.081132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:19.204535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090410+0000","last_change":"2024-04-25T10:18:39.082813+0000","last_active":"2024-04-25T10:18:40.090410+0000","last_peered":"2024-04-25T10:18:40.090410+0000","last_clean":"2024-04-25T10:18:40.090410+0000","last_became_active":"2024-04-25T10:18:39.082547+0000","last_became_peered":"2024-04-25T10:18:39.082547+0000","last_unstale":"2024-04-25T10:18:40.090410+0000","last_undegraded":"2024-04-25T10:18:40.090410+0000","last_fullsized":"2024-04-25T10:18:40.090410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:16.919229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567280+0000","last_change":"2024-04-25T10:18:38.080404+0000","last_active":"2024-04-25T10:18:40.567280+0000","last_peered":"2024-04-25T10:18:40.567280+0000","last_clean":"2024-04-25T10:18:40.567280+0000","last_became_active":"2024-04-25T10:18:38.080141+0000","last_became_peered":"2024-04-25T10:18:38.080141+0000","last_unstale":"2024-04-25T10:18:40.567280+0000","last_undegraded":"2024-04-25T10:18:40.567280+0000","last_fullsized":"2024-04-25T10:18:40.567280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:41.627770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566344+0000","last_change":"2024-04-25T10:18:39.075698+0000","last_active":"2024-04-25T10:18:40.566344+0000","last_peered":"2024-04-25T10:18:40.566344+0000","last_clean":"2024-04-25T10:18:40.566344+0000","last_became_active":"2024-04-25T10:18:39.075439+0000","last_became_peered":"2024-04-25T10:18:39.075439+0000","last_unstale":"2024-04-25T10:18:40.566344+0000","last_undegraded":"2024-04-25T10:18:40.566344+0000","last_fullsized":"2024-04-25T10:18:40.566344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:27.013833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566793+0000","last_change":"2024-04-25T10:18:38.079919+0000","last_active":"2024-04-25T10:18:40.566793+0000","last_peered":"2024-04-25T10:18:40.566793+0000","last_clean":"2024-04-25T10:18:40.566793+0000","last_became_active":"2024-04-25T10:18:38.079668+0000","last_became_peered":"2024-04-25T10:18:38.079668+0000","last_unstale":"2024-04-25T10:18:40.566793+0000","last_undegraded":"2024-04-25T10:18:40.566793+0000","last_fullsized":"2024-04-25T10:18:40.566793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:33.251178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.089510+0000","last_change":"2024-04-25T10:18:39.086812+0000","last_active":"2024-04-25T10:18:40.089510+0000","last_peered":"2024-04-25T10:18:40.089510+0000","last_clean":"2024-04-25T10:18:40.089510+0000","last_became_active":"2024-04-25T10:18:39.086494+0000","last_became_peered":"2024-04-25T10:18:39.086494+0000","last_unstale":"2024-04-25T10:18:40.089510+0000","last_undegraded":"2024-04-25T10:18:40.089510+0000","last_fullsized":"2024-04-25T10:18:40.089510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:10.485495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.080804+0000","last_change":"2024-04-25T10:18:38.072959+0000","last_active":"2024-04-25T10:18:40.080804+0000","last_peered":"2024-04-25T10:18:40.080804+0000","last_clean":"2024-04-25T10:18:40.080804+0000","last_became_active":"2024-04-25T10:18:38.072676+0000","last_became_peered":"2024-04-25T10:18:38.072676+0000","last_unstale":"2024-04-25T10:18:40.080804+0000","last_undegraded":"2024-04-25T10:18:40.080804+0000","last_fullsized":"2024-04-25T10:18:40.080804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:35.865602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.568028+0000","last_change":"2024-04-25T10:18:39.085568+0000","last_active":"2024-04-25T10:18:40.568028+0000","last_peered":"2024-04-25T10:18:40.568028+0000","last_clean":"2024-04-25T10:18:40.568028+0000","last_became_active":"2024-04-25T10:18:39.085268+0000","last_became_peered":"2024-04-25T10:18:39.085268+0000","last_unstale":"2024-04-25T10:18:40.568028+0000","last_undegraded":"2024-04-25T10:18:40.568028+0000","last_fullsized":"2024-04-25T10:18:40.568028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:36.870040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566377+0000","last_change":"2024-04-25T10:18:38.080760+0000","last_active":"2024-04-25T10:18:40.566377+0000","last_peered":"2024-04-25T10:18:40.566377+0000","last_clean":"2024-04-25T10:18:40.566377+0000","last_became_active":"2024-04-25T10:18:38.080504+0000","last_became_peered":"2024-04-25T10:18:38.080504+0000","last_unstale":"2024-04-25T10:18:40.566377+0000","last_undegraded":"2024-04-25T10:18:40.566377+0000","last_fullsized":"2024-04-25T10:18:40.566377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:25.149978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.089785+0000","last_change":"2024-04-25T10:18:39.095903+0000","last_active":"2024-04-25T10:18:40.089785+0000","last_peered":"2024-04-25T10:18:40.089785+0000","last_clean":"2024-04-25T10:18:40.089785+0000","last_became_active":"2024-04-25T10:18:39.094571+0000","last_became_peered":"2024-04-25T10:18:39.094571+0000","last_unstale":"2024-04-25T10:18:40.089785+0000","last_undegraded":"2024-04-25T10:18:40.089785+0000","last_fullsized":"2024-04-25T10:18:40.089785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:07.964897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:41.069269+0000","last_change":"2024-04-25T10:18:38.076632+0000","last_active":"2024-04-25T10:18:41.069269+0000","last_peered":"2024-04-25T10:18:41.069269+0000","last_clean":"2024-04-25T10:18:41.069269+0000","last_became_active":"2024-04-25T10:18:38.076242+0000","last_became_peered":"2024-04-25T10:18:38.076242+0000","last_unstale":"2024-04-25T10:18:41.069269+0000","last_undegraded":"2024-04-25T10:18:41.069269+0000","last_fullsized":"2024-04-25T10:18:41.069269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:41.793441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266966+0000","last_change":"2024-04-25T10:18:39.076475+0000","last_active":"2024-04-25T10:18:40.266966+0000","last_peered":"2024-04-25T10:18:40.266966+0000","last_clean":"2024-04-25T10:18:40.266966+0000","last_became_active":"2024-04-25T10:18:39.076190+0000","last_became_peered":"2024-04-25T10:18:39.076190+0000","last_unstale":"2024-04-25T10:18:40.266966+0000","last_undegraded":"2024-04-25T10:18:40.266966+0000","last_fullsized":"2024-04-25T10:18:40.266966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:45.920495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266908+0000","last_change":"2024-04-25T10:18:38.086992+0000","last_active":"2024-04-25T10:18:40.266908+0000","last_peered":"2024-04-25T10:18:40.266908+0000","last_clean":"2024-04-25T10:18:40.266908+0000","last_became_active":"2024-04-25T10:18:38.086760+0000","last_became_peered":"2024-04-25T10:18:38.086760+0000","last_unstale":"2024-04-25T10:18:40.266908+0000","last_undegraded":"2024-04-25T10:18:40.266908+0000","last_fullsized":"2024-04-25T10:18:40.266908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:11.214828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.568027+0000","last_change":"2024-04-25T10:18:39.093326+0000","last_active":"2024-04-25T10:18:40.568027+0000","last_peered":"2024-04-25T10:18:40.568027+0000","last_clean":"2024-04-25T10:18:40.568027+0000","last_became_active":"2024-04-25T10:18:39.092822+0000","last_became_peered":"2024-04-25T10:18:39.092822+0000","last_unstale":"2024-04-25T10:18:40.568027+0000","last_undegraded":"2024-04-25T10:18:40.568027+0000","last_fullsized":"2024-04-25T10:18:40.568027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:43.138703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.267091+0000","last_change":"2024-04-25T10:18:38.079358+0000","last_active":"2024-04-25T10:18:40.267091+0000","last_peered":"2024-04-25T10:18:40.267091+0000","last_clean":"2024-04-25T10:18:40.267091+0000","last_became_active":"2024-04-25T10:18:38.077511+0000","last_became_peered":"2024-04-25T10:18:38.077511+0000","last_unstale":"2024-04-25T10:18:40.267091+0000","last_undegraded":"2024-04-25T10:18:40.267091+0000","last_fullsized":"2024-04-25T10:18:40.267091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:43.378416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090293+0000","last_change":"2024-04-25T10:18:39.084067+0000","last_active":"2024-04-25T10:18:40.090293+0000","last_peered":"2024-04-25T10:18:40.090293+0000","last_clean":"2024-04-25T10:18:40.090293+0000","last_became_active":"2024-04-25T10:18:39.083787+0000","last_became_peered":"2024-04-25T10:18:39.083787+0000","last_unstale":"2024-04-25T10:18:40.090293+0000","last_undegraded":"2024-04-25T10:18:40.090293+0000","last_fullsized":"2024-04-25T10:18:40.090293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:12.424932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566756+0000","last_change":"2024-04-25T10:18:38.078522+0000","last_active":"2024-04-25T10:18:40.566756+0000","last_peered":"2024-04-25T10:18:40.566756+0000","last_clean":"2024-04-25T10:18:40.566756+0000","last_became_active":"2024-04-25T10:18:38.078186+0000","last_became_peered":"2024-04-25T10:18:38.078186+0000","last_unstale":"2024-04-25T10:18:40.566756+0000","last_undegraded":"2024-04-25T10:18:40.566756+0000","last_fullsized":"2024-04-25T10:18:40.566756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:03.344601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566467+0000","last_change":"2024-04-25T10:18:39.083089+0000","last_active":"2024-04-25T10:18:40.566467+0000","last_peered":"2024-04-25T10:18:40.566467+0000","last_clean":"2024-04-25T10:18:40.566467+0000","last_became_active":"2024-04-25T10:18:39.082789+0000","last_became_peered":"2024-04-25T10:18:39.082789+0000","last_unstale":"2024-04-25T10:18:40.566467+0000","last_undegraded":"2024-04-25T10:18:40.566467+0000","last_fullsized":"2024-04-25T10:18:40.566467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:21.204031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567707+0000","last_change":"2024-04-25T10:18:38.084340+0000","last_active":"2024-04-25T10:18:40.567707+0000","last_peered":"2024-04-25T10:18:40.567707+0000","last_clean":"2024-04-25T10:18:40.567707+0000","last_became_active":"2024-04-25T10:18:38.080419+0000","last_became_peered":"2024-04-25T10:18:38.080419+0000","last_unstale":"2024-04-25T10:18:40.567707+0000","last_undegraded":"2024-04-25T10:18:40.567707+0000","last_fullsized":"2024-04-25T10:18:40.567707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:11.677147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.089566+0000","last_change":"2024-04-25T10:18:39.099958+0000","last_active":"2024-04-25T10:18:40.089566+0000","last_peered":"2024-04-25T10:18:40.089566+0000","last_clean":"2024-04-25T10:18:40.089566+0000","last_became_active":"2024-04-25T10:18:39.099752+0000","last_became_peered":"2024-04-25T10:18:39.099752+0000","last_unstale":"2024-04-25T10:18:40.089566+0000","last_undegraded":"2024-04-25T10:18:40.089566+0000","last_fullsized":"2024-04-25T10:18:40.089566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:49.470703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266994+0000","last_change":"2024-04-25T10:18:38.076667+0000","last_active":"2024-04-25T10:18:40.266994+0000","last_peered":"2024-04-25T10:18:40.266994+0000","last_clean":"2024-04-25T10:18:40.266994+0000","last_became_active":"2024-04-25T10:18:38.076227+0000","last_became_peered":"2024-04-25T10:18:38.076227+0000","last_unstale":"2024-04-25T10:18:40.266994+0000","last_undegraded":"2024-04-25T10:18:40.266994+0000","last_fullsized":"2024-04-25T10:18:40.266994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:11.776009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.266968+0000","last_change":"2024-04-25T10:18:39.076166+0000","last_active":"2024-04-25T10:18:40.266968+0000","last_peered":"2024-04-25T10:18:40.266968+0000","last_clean":"2024-04-25T10:18:40.266968+0000","last_became_active":"2024-04-25T10:18:39.075426+0000","last_became_peered":"2024-04-25T10:18:39.075426+0000","last_unstale":"2024-04-25T10:18:40.266968+0000","last_undegraded":"2024-04-25T10:18:40.266968+0000","last_fullsized":"2024-04-25T10:18:40.266968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:30.783354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.567440+0000","last_change":"2024-04-25T10:18:38.081731+0000","last_active":"2024-04-25T10:18:40.567440+0000","last_peered":"2024-04-25T10:18:40.567440+0000","last_clean":"2024-04-25T10:18:40.567440+0000","last_became_active":"2024-04-25T10:18:38.081433+0000","last_became_peered":"2024-04-25T10:18:38.081433+0000","last_unstale":"2024-04-25T10:18:40.567440+0000","last_undegraded":"2024-04-25T10:18:40.567440+0000","last_fullsized":"2024-04-25T10:18:40.567440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:00.827630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.566154+0000","last_change":"2024-04-25T10:18:39.092306+0000","last_active":"2024-04-25T10:18:40.566154+0000","last_peered":"2024-04-25T10:18:40.566154+0000","last_clean":"2024-04-25T10:18:40.566154+0000","last_became_active":"2024-04-25T10:18:39.092000+0000","last_became_peered":"2024-04-25T10:18:39.092000+0000","last_unstale":"2024-04-25T10:18:40.566154+0000","last_undegraded":"2024-04-25T10:18:40.566154+0000","last_fullsized":"2024-04-25T10:18:40.566154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:38.050835+0000","last_clean_scrub_stamp":"2024-04-25T10:18:38.050835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:32.674674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.090198+0000","last_change":"2024-04-25T10:18:38.085578+0000","last_active":"2024-04-25T10:18:40.090198+0000","last_peered":"2024-04-25T10:18:40.090198+0000","last_clean":"2024-04-25T10:18:40.090198+0000","last_became_active":"2024-04-25T10:18:38.085056+0000","last_became_peered":"2024-04-25T10:18:38.085056+0000","last_unstale":"2024-04-25T10:18:40.090198+0000","last_undegraded":"2024-04-25T10:18:40.090198+0000","last_fullsized":"2024-04-25T10:18:40.090198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:37.046397+0000","last_clean_scrub_stamp":"2024-04-25T10:18:37.046397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:20.199593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":291,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:18:40.089620+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:18:40.089620+0000","last_peered":"2024-04-25T10:18:40.089620+0000","last_clean":"2024-04-25T10:18:40.089620+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:18:40.089620+0000","last_undegraded":"2024-04-25T10:18:40.089620+0000","last_fullsized":"2024-04-25T10:18:40.089620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50856,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320984,"statfs":{"total":96636764160,"available":96584687616,"internally_reserved":0,"allocated":1667072,"data_stored":1051032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50292,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321548,"statfs":{"total":96636764160,"available":96585265152,"internally_reserved":0,"allocated":1081344,"data_stored":461288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40564,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331276,"statfs":{"total":96636764160,"available":96595226624,"internally_reserved":0,"allocated":1085440,"data_stored":462228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50272,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321568,"statfs":{"total":96636764160,"available":96585285632,"internally_reserved":0,"allocated":1069056,"data_stored":460594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46124,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325716,"statfs":{"total":96636764160,"available":96589533184,"internally_reserved":0,"allocated":1081344,"data_stored":462206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44708,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327132,"statfs":{"total":96636764160,"available":96590983168,"internally_reserved":0,"allocated":1073152,"data_stored":461176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72496,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299344,"statfs":{"total":96636764160,"available":96562528256,"internally_reserved":0,"allocated":1081344,"data_stored":460788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46696,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325144,"statfs":{"total":96636764160,"available":96588947456,"internally_reserved":0,"allocated":1667072,"data_stored":1050996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:43.523 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:18:43.524 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:18:43.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc450a00640 1 Processor -- start 2024-04-25T10:18:43.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc450a00640 1 -- start start 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc450a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc44c15ba40 0x7fc44c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc450a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0cf080 con 0x7fc44c15be20 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc450a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0cdfe0 con 0x7fc44c0d0450 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc450a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0cdd90 con 0x7fc44c15ba40 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc449e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc44c15be20 0x7fc44c0cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34318/0 (socket says 172.21.15.87:34318) 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc449e00640 1 -- 172.21.15.87:0/941206343 learned_addr learned my addr 172.21.15.87:0/941206343 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc44a800640 1 --2- 172.21.15.87:0/941206343 >> v2:172.21.15.169:3300/0 conn(0x7fc44c15ba40 0x7fc44c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc449400640 1 -- 172.21.15.87:0/941206343 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607413112 0 0) 0x7fc44c0cf080 con 0x7fc44c15be20 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc449400640 1 -- 172.21.15.87:0/941206343 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc430003530 con 0x7fc44c15be20 2024-04-25T10:18:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc44a800640 1 -- 172.21.15.87:0/941206343 >> v1:172.21.15.169:6789/0 conn(0x7fc44c0d0450 legacy=0x7fc44c0b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc44a800640 1 -- 172.21.15.87:0/941206343 >> v1:172.21.15.87:6789/0 conn(0x7fc44c15be20 legacy=0x7fc44c0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.700+0000 7fc44a800640 1 -- 172.21.15.87:0/941206343 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c0ce2c0 con 0x7fc44c15ba40 2024-04-25T10:18:43.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc44a800640 1 --2- 172.21.15.87:0/941206343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c15ba40 0x7fc44c0ccd80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c01f120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79d991b98ea0d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc449400640 1 -- 172.21.15.87:0/941206343 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0b4070 con 0x7fc44c15ba40 2024-04-25T10:18:43.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc449400640 1 -- 172.21.15.87:0/941206343 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c0af600 con 0x7fc44c15ba40 2024-04-25T10:18:43.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc449400640 1 -- 172.21.15.87:0/941206343 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0af9d0 con 0x7fc44c15ba40 2024-04-25T10:18:43.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc450a00640 1 -- 172.21.15.87:0/941206343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c15ba40 msgr2=0x7fc44c0ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc450a00640 1 --2- 172.21.15.87:0/941206343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c15ba40 0x7fc44c0ccd80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c01f120 comp rx=0 tx=0).stop 2024-04-25T10:18:43.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc450a00640 1 -- 172.21.15.87:0/941206343 shutdown_connections 2024-04-25T10:18:43.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc450a00640 1 --2- 172.21.15.87:0/941206343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c15ba40 0x7fc44c0ccd80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc450a00640 1 -- 172.21.15.87:0/941206343 >> 172.21.15.87:0/941206343 conn(0x7fc44c0d1760 msgr2=0x7fc44c0d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc450a00640 1 -- 172.21.15.87:0/941206343 shutdown_connections 2024-04-25T10:18:43.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.704+0000 7fc450a00640 1 -- 172.21.15.87:0/941206343 wait complete. 2024-04-25T10:18:43.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 Processor -- start 2024-04-25T10:18:43.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 -- start start 2024-04-25T10:18:43.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c0d0450 0x7fc44c1f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc44c15ba40 0x7fc44c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc44c15be20 0x7fc44c1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0ceb60 con 0x7fc44c15be20 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0ce880 con 0x7fc44c0d0450 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0b55c0 con 0x7fc44c15ba40 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc44c15be20 0x7fc44c1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc44c15be20 0x7fc44c1f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39478/0 (socket says 172.21.15.87:39478) 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 -- 172.21.15.87:0/1118607103 learned_addr learned my addr 172.21.15.87:0/1118607103 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc449e00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc44c15ba40 0x7fc44c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44a800640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c0d0450 0x7fc44c1f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 -- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc44c15ba40 msgr2=0x7fc44c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc44c15ba40 0x7fc44c15de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 -- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c0d0450 msgr2=0x7fc44c1f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c0d0450 0x7fc44c1f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 -- 172.21.15.87:0/1118607103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c1f79a0 con 0x7fc44c15be20 2024-04-25T10:18:43.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc44b200640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc44c15be20 0x7fc44c1f4390 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc44400b410 tx=0x7fc44400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc442a00640 1 -- 172.21.15.87:0/1118607103 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444015020 con 0x7fc44c15be20 2024-04-25T10:18:43.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc442a00640 1 -- 172.21.15.87:0/1118607103 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc444011850 con 0x7fc44c15be20 2024-04-25T10:18:43.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c1f7ce0 con 0x7fc44c15be20 2024-04-25T10:18:43.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc442a00640 1 -- 172.21.15.87:0/1118607103 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444011c40 con 0x7fc44c15be20 2024-04-25T10:18:43.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.708+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c1f3b60 con 0x7fc44c15be20 2024-04-25T10:18:43.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.712+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c075120 con 0x7fc44c15be20 2024-04-25T10:18:43.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.712+0000 7fc442a00640 1 -- 172.21.15.87:0/1118607103 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc444018020 con 0x7fc44c15be20 2024-04-25T10:18:43.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.712+0000 7fc442a00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc42c066f60 0x7fc42c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.712+0000 7fc442a00640 1 -- 172.21.15.87:0/1118607103 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc444084f20 con 0x7fc44c15be20 2024-04-25T10:18:43.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.712+0000 7fc44a800640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc42c066f60 0x7fc42c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.712+0000 7fc44a800640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc42c066f60 0x7fc42c069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0b1000 tx=0x7fc43c0aa000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.716+0000 7fc442a00640 1 -- 172.21.15.87:0/1118607103 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc44401f020 con 0x7fc44c15be20 2024-04-25T10:18:43.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.924+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc44c0725c0 con 0x7fc44c15be20 2024-04-25T10:18:43.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.924+0000 7fc442a00640 1 -- 172.21.15.87:0/1118607103 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc44c1f7ce0 con 0x7fc44c15be20 2024-04-25T10:18:43.928 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:18:43.928 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:18:43.928 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:18:43.928 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:18:43.929 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:18:43.929 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:18:43.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc42c066f60 msgr2=0x7fc42c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc42c066f60 0x7fc42c069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0b1000 tx=0x7fc43c0aa000 comp rx=0 tx=0).stop 2024-04-25T10:18:43.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc44c15be20 msgr2=0x7fc44c1f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc44c15be20 0x7fc44c1f4390 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc44400b410 tx=0x7fc44400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 shutdown_connections 2024-04-25T10:18:43.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc42c066f60 0x7fc42c069420 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc44c15be20 0x7fc44c1f4390 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc44c15ba40 0x7fc44c15de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 --2- 172.21.15.87:0/1118607103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc44c0d0450 0x7fc44c1f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 >> 172.21.15.87:0/1118607103 conn(0x7fc44c0d1760 msgr2=0x7fc44c0b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 shutdown_connections 2024-04-25T10:18:43.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:43.928+0000 7fc450a00640 1 -- 172.21.15.87:0/1118607103 wait complete. 2024-04-25T10:18:43.951 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:44.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe158a00640 1 Processor -- start 2024-04-25T10:18:44.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe158a00640 1 -- start start 2024-04-25T10:18:44.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe158a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe1540670b0 0x7fe154071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe158a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1540b1ec0 0x7fe1540cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe158a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540b5850 con 0x7fe15415dca0 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe158a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1540b5af0 con 0x7fe1540b1ec0 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe158a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1540b5dd0 con 0x7fe1540670b0 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe153e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1540b1ec0 0x7fe1540cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe153e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1540b1ec0 0x7fe1540cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39492/0 (socket says 172.21.15.87:39492) 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe153e00640 1 -- 172.21.15.87:0/763537476 learned_addr learned my addr 172.21.15.87:0/763537476 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe152a00640 1 --2- 172.21.15.87:0/763537476 >> v2:172.21.15.169:3300/0 conn(0x7fe1540670b0 0x7fe154071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe153e00640 1 -- 172.21.15.87:0/763537476 >> v2:172.21.15.169:3300/0 conn(0x7fe1540670b0 msgr2=0x7fe154071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe153e00640 1 --2- 172.21.15.87:0/763537476 >> v2:172.21.15.169:3300/0 conn(0x7fe1540670b0 0x7fe154071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe153e00640 1 -- 172.21.15.87:0/763537476 >> v1:172.21.15.169:6789/0 conn(0x7fe15415dca0 legacy=0x7fe1540669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.084+0000 7fe153e00640 1 -- 172.21.15.87:0/763537476 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540b60b0 con 0x7fe1540b1ec0 2024-04-25T10:18:44.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe153e00640 1 --2- 172.21.15.87:0/763537476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540b1ec0 0x7fe1540cea40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe1540679a0 tx=0x7fe14c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be6b1ff9dc6d8de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe152000640 1 -- 172.21.15.87:0/763537476 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0ab660 con 0x7fe1540b1ec0 2024-04-25T10:18:44.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe152000640 1 -- 172.21.15.87:0/763537476 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c0b0070 con 0x7fe1540b1ec0 2024-04-25T10:18:44.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe152000640 1 -- 172.21.15.87:0/763537476 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0aba20 con 0x7fe1540b1ec0 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 -- 172.21.15.87:0/763537476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540b1ec0 msgr2=0x7fe1540cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 --2- 172.21.15.87:0/763537476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540b1ec0 0x7fe1540cea40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe1540679a0 tx=0x7fe14c01c460 comp rx=0 tx=0).stop 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 -- 172.21.15.87:0/763537476 shutdown_connections 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 --2- 172.21.15.87:0/763537476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540b1ec0 0x7fe1540cea40 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 --2- 172.21.15.87:0/763537476 >> v2:172.21.15.169:3300/0 conn(0x7fe1540670b0 0x7fe154071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 -- 172.21.15.87:0/763537476 >> 172.21.15.87:0/763537476 conn(0x7fe1540b2440 msgr2=0x7fe1540b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 -- 172.21.15.87:0/763537476 shutdown_connections 2024-04-25T10:18:44.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.088+0000 7fe158a00640 1 -- 172.21.15.87:0/763537476 wait complete. 2024-04-25T10:18:44.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 Processor -- start 2024-04-25T10:18:44.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 -- start start 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540670b0 0x7fe15414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1540b1ec0 0x7fe154157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe15415dca0 0x7fe154150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540cfb20 con 0x7fe1540670b0 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540cf9a0 con 0x7fe1540b1ec0 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540cfca0 con 0x7fe15415dca0 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540670b0 0x7fe15414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540670b0 0x7fe15414d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39508/0 (socket says 172.21.15.87:39508) 2024-04-25T10:18:44.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 -- 172.21.15.87:0/2190024868 learned_addr learned my addr 172.21.15.87:0/2190024868 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153e00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe15415dca0 0x7fe154150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe152a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1540b1ec0 0x7fe154157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 -- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe15415dca0 msgr2=0x7fe154150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe15415dca0 0x7fe154150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 -- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1540b1ec0 msgr2=0x7fe154157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1540b1ec0 0x7fe154157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 -- 172.21.15.87:0/2190024868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154159110 con 0x7fe1540670b0 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153e00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe15415dca0 0x7fe154150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe152a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1540b1ec0 0x7fe154157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe153400640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540670b0 0x7fe15414d6c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c760 tx=0x7fe14400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe14b400640 1 -- 172.21.15.87:0/2190024868 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144003ba0 con 0x7fe1540670b0 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe14b400640 1 -- 172.21.15.87:0/2190024868 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe144003d60 con 0x7fe1540670b0 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe14b400640 1 -- 172.21.15.87:0/2190024868 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400aa30 con 0x7fe1540670b0 2024-04-25T10:18:44.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1541593f0 con 0x7fe1540670b0 2024-04-25T10:18:44.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.092+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1541599b0 con 0x7fe1540670b0 2024-04-25T10:18:44.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.096+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe154075120 con 0x7fe1540670b0 2024-04-25T10:18:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.096+0000 7fe14b400640 1 -- 172.21.15.87:0/2190024868 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe14400abd0 con 0x7fe1540670b0 2024-04-25T10:18:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.096+0000 7fe14b400640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe134066f60 0x7fe134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.096+0000 7fe14b400640 1 -- 172.21.15.87:0/2190024868 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe144084f00 con 0x7fe1540670b0 2024-04-25T10:18:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.096+0000 7fe152a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe134066f60 0x7fe134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.096+0000 7fe152a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe134066f60 0x7fe134069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe15415e130 tx=0x7fe140008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.100+0000 7fe14b400640 1 -- 172.21.15.87:0/2190024868 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe144055f00 con 0x7fe1540670b0 2024-04-25T10:18:44.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.288+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe15415c070 con 0x7fe1540670b0 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe14b400640 1 -- 172.21.15.87:0/2190024868 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe144055800 con 0x7fe1540670b0 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe134066f60 msgr2=0x7fe134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe134066f60 0x7fe134069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe15415e130 tx=0x7fe140008040 comp rx=0 tx=0).stop 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540670b0 msgr2=0x7fe15414d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540670b0 0x7fe15414d6c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c760 tx=0x7fe14400cc20 comp rx=0 tx=0).stop 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 shutdown_connections 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe134066f60 0x7fe134069420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe15415dca0 0x7fe154150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1540b1ec0 0x7fe154157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 --2- 172.21.15.87:0/2190024868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1540670b0 0x7fe15414d6c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 >> 172.21.15.87:0/2190024868 conn(0x7fe1540b2440 msgr2=0x7fe1540669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 shutdown_connections 2024-04-25T10:18:44.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.292+0000 7fe158a00640 1 -- 172.21.15.87:0/2190024868 wait complete. 2024-04-25T10:18:44.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:18:38.442126+0000', 'modified': '2024-04-25T10:18:41.469242+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7467': {'gid': 7467, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/3032837514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 3032837514}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 3032837514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:44.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:18:44.718 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:18:44.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1838e00640 1 Processor -- start 2024-04-25T10:18:44.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1838e00640 1 -- start start 2024-04-25T10:18:44.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1838e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f183415e050 0x7f18340b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1838e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340b1af0 con 0x7f18340b13e0 2024-04-25T10:18:44.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1838e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340b1dd0 con 0x7f18340d1820 2024-04-25T10:18:44.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1838e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340b20b0 con 0x7f183415e050 2024-04-25T10:18:44.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1832800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f18340b13e0 0x7f18340cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:34326/0 (socket says 172.21.15.87:34326) 2024-04-25T10:18:44.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1832800640 1 -- 172.21.15.87:0/1933352297 learned_addr learned my addr 172.21.15.87:0/1933352297 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:18:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429653565 0 0) 0x7f18340b1af0 con 0x7f18340b13e0 2024-04-25T10:18:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1818003530 con 0x7f18340b13e0 2024-04-25T10:18:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1831400640 1 --2- 172.21.15.87:0/1933352297 >> v2:172.21.15.169:3300/0 conn(0x7f183415e050 0x7f18340b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433014750 0 0) 0x7f1818003530 con 0x7f18340b13e0 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 >> v2:172.21.15.169:3300/0 conn(0x7f183415e050 msgr2=0x7f18340b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 --2- 172.21.15.87:0/1933352297 >> v2:172.21.15.169:3300/0 conn(0x7f183415e050 0x7f18340b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 >> v1:172.21.15.169:6789/0 conn(0x7f18340d1820 legacy=0x7f18340d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340b2390 con 0x7f18340b13e0 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f182c001680 con 0x7f18340b13e0 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 >> v1:172.21.15.87:6789/0 conn(0x7f18340b13e0 legacy=0x7f18340cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18180039c0 0x7f1818003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1818004810 0x7f1818004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18180051d0 0x7f181800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340b20b0 con 0x7f18180039c0 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181800c170 con 0x7f18180051d0 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181800c370 con 0x7f1818004810 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1831e00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18180039c0 0x7f1818003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1832800640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18180051d0 0x7f181800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.852+0000 7f1831400640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1818004810 0x7f1818004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1831e00640 1 -- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1818004810 msgr2=0x7f1818004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1831e00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1818004810 0x7f1818004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1831e00640 1 -- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18180051d0 msgr2=0x7f181800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1831e00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18180051d0 0x7f181800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1831e00640 1 -- 172.21.15.87:0/1933352297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1818003530 con 0x7f18180039c0 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1832800640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18180051d0 0x7f181800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1832800640 1 -- 172.21.15.87:0/1933352297 reap_dead start 2024-04-25T10:18:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1831e00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18180039c0 0x7f1818003de0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f1824001e70 tx=0x7f182401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e153a221ee27bb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824001a90 con 0x7f18180039c0 2024-04-25T10:18:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824003b80 con 0x7f18180039c0 2024-04-25T10:18:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1830a00640 1 -- 172.21.15.87:0/1933352297 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824005080 con 0x7f18180039c0 2024-04-25T10:18:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1838e00640 1 -- 172.21.15.87:0/1933352297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18180039c0 msgr2=0x7f1818003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1838e00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18180039c0 0x7f1818003de0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f1824001e70 tx=0x7f182401c760 comp rx=0 tx=0).stop 2024-04-25T10:18:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1838e00640 1 -- 172.21.15.87:0/1933352297 shutdown_connections 2024-04-25T10:18:44.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1838e00640 1 --2- 172.21.15.87:0/1933352297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18180039c0 0x7f1818003de0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1838e00640 1 -- 172.21.15.87:0/1933352297 >> 172.21.15.87:0/1933352297 conn(0x7f183409a9d0 msgr2=0x7f183409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1838e00640 1 -- 172.21.15.87:0/1933352297 shutdown_connections 2024-04-25T10:18:44.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.856+0000 7f1838e00640 1 -- 172.21.15.87:0/1933352297 wait complete. 2024-04-25T10:18:44.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 Processor -- start 2024-04-25T10:18:44.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 -- start start 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18180039c0 0x7f18341fac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18340c72a0 0x7f18340c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183409ea10 0x7f183409ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340ceb10 con 0x7f183409ea10 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340ce990 con 0x7f18340c72a0 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340cec90 con 0x7f18180039c0 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183409ea10 0x7f183409ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183409ea10 0x7f183409ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39530/0 (socket says 172.21.15.87:39530) 2024-04-25T10:18:44.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 -- 172.21.15.87:0/4199023 learned_addr learned my addr 172.21.15.87:0/4199023 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1831400640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18340c72a0 0x7f18340c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1831e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18180039c0 0x7f18341fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 -- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18180039c0 msgr2=0x7f18341fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18180039c0 0x7f18341fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 -- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18340c72a0 msgr2=0x7f18340c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18340c72a0 0x7f18340c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 -- 172.21.15.87:0/4199023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834151a00 con 0x7f183409ea10 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1832800640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183409ea10 0x7f183409ee90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f182c001ec0 tx=0x7f182c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f182a000640 1 -- 172.21.15.87:0/4199023 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00c690 con 0x7f183409ea10 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f182a000640 1 -- 172.21.15.87:0/4199023 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c018070 con 0x7f183409ea10 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f182a000640 1 -- 172.21.15.87:0/4199023 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00ca90 con 0x7f183409ea10 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18340c8460 con 0x7f183409ea10 2024-04-25T10:18:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.860+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18340c8180 con 0x7f183409ea10 2024-04-25T10:18:44.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.864+0000 7f182a000640 1 -- 172.21.15.87:0/4199023 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f182c00cc30 con 0x7f183409ea10 2024-04-25T10:18:44.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.864+0000 7f182a000640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1814066f60 0x7f1814069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.864+0000 7f182a000640 1 -- 172.21.15.87:0/4199023 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f182c084c00 con 0x7f183409ea10 2024-04-25T10:18:44.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.864+0000 7f1831e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1814066f60 0x7f1814069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.864+0000 7f183c3b0000 1 Processor -- start 2024-04-25T10:18:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.864+0000 7f183c3b0000 1 -- start start 2024-04-25T10:18:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.864+0000 7f1831e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1814066f60 0x7f1814069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1824005000 tx=0x7f1824009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccad9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca8f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595cca90310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cc9775f0 con 0x5595ccad9920 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cc8deda0 con 0x5595cca8ff30 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595cc867150 con 0x5595ccada240 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccad9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccad9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39538/0 (socket says 172.21.15.87:39538) 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 -- 172.21.15.87:0/3551484133 learned_addr learned my addr 172.21.15.87:0/3551484133 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831400640 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca8f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1832800640 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595cca90310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 -- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 msgr2=0x5595cca8f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca8f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 -- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 msgr2=0x5595cca90310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595cca90310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 -- 172.21.15.87:0/3551484133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595cc880ca0 con 0x5595ccad9920 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1832800640 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595cca90310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f1831e00640 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccad9d00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1824098030 tx=0x7f1824001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ac40fcb50d4a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f181ea00640 1 -- 172.21.15.87:0/3551484133 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240057c0 con 0x5595ccad9920 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f181ea00640 1 -- 172.21.15.87:0/3551484133 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240c3070 con 0x5595ccad9920 2024-04-25T10:18:44.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f181ea00640 1 -- 172.21.15.87:0/3551484133 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240a9640 con 0x5595ccad9920 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- 172.21.15.87:0/3551484133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 msgr2=0x5595ccad9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccad9d00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1824098030 tx=0x7f1824001520 comp rx=0 tx=0).stop 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- 172.21.15.87:0/3551484133 shutdown_connections 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595cca90310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca8f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3551484133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccad9d00 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- 172.21.15.87:0/3551484133 >> 172.21.15.87:0/3551484133 conn(0x5595cca962a0 msgr2=0x5595cca8e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- 172.21.15.87:0/3551484133 shutdown_connections 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- 172.21.15.87:0/3551484133 wait complete. 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- start start 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 Processor -- start 2024-04-25T10:18:44.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595cca8ff30 0x5595ccae0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccad9920 0x5595ccae12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca93aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cc899990 con 0x5595cca8ff30 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cc8dae40 con 0x5595ccad9920 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595cc86a900 con 0x5595ccada240 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.868+0000 7f183c3b0000 10 client.0 _ll_register_callbacks cb 0x5595cc9921c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595cca8ff30 0x5595ccae0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595cca8ff30 0x5595ccae0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39550/0 (socket says 172.21.15.87:39550) 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 -- 172.21.15.87:0/3357788149 learned_addr learned my addr 172.21.15.87:0/3357788149 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831400640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccad9920 0x5595ccae12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1832800640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca93aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 -- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 msgr2=0x5595cca93aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca93aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 -- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccad9920 msgr2=0x5595ccae12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccad9920 0x5595ccae12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 -- 172.21.15.87:0/3357788149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595cc897630 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831400640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccad9920 0x5595ccae12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1832800640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca93aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831e00640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595cca8ff30 0x5595ccae0d60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1824001e40 tx=0x7f1824019500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b3cdc26647d2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 1 -- 172.21.15.87:0/3357788149 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240d2610 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 1 -- 172.21.15.87:0/3357788149 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18240d27b0 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18240c3070 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 1 -- 172.21.15.87:0/3357788149 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18240da050 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 10 client.7596 fetch_fsmap learned FSMap version 63 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5595cc8673a0 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 1 -- 172.21.15.87:0/3357788149 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240c3070 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 1 -- 172.21.15.87:0/3357788149 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f18240dd020 con 0x5595cca8ff30 2024-04-25T10:18:44.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 10 client.7596 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 10 client.7596 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 20 client.7596 populate_metadata read hostname 'smithi087' 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595cca95160 0x5595ccaeadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 4 client.7596 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595cc87aa50 con 0x5595cca95160 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831400640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595cca95160 0x5595ccaeadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1831400640 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595cca95160 0x5595ccaeadc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d601ca0fc6ee04a2 server_cookie=8d3e9a965d360f6e in_seq=0 out_seq=0 2024-04-25T10:18:44.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 10 client.7596 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:18:44.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 1 -- 172.21.15.87:0/3357788149 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595cc87aa50 con 0x5595cca95160 2024-04-25T10:18:44.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 10 client.7596 handle_command_reply: tid=0 2024-04-25T10:18:44.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 client.7596 shutdown 2024-04-25T10:18:44.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595cca8ff30 msgr2=0x5595ccae0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595cca8ff30 0x5595ccae0d60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1824001e40 tx=0x7f1824019500 comp rx=0 tx=0).stop 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 shutdown_connections 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595cca95160 0x5595ccaeadc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595cca93aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccad9920 0x5595ccae12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 --2- 172.21.15.87:0/3357788149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595cca8ff30 0x5595ccae0d60 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 >> 172.21.15.87:0/3357788149 conn(0x5595cca962a0 msgr2=0x5595cca8e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f1813400640 0 client.7596 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 shutdown_connections 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.872+0000 7f183c3b0000 1 -- 172.21.15.87:0/3357788149 wait complete. 2024-04-25T10:18:44.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.876+0000 7f183c3b0000 20 client.7596 trim_cache size 0 max 16384 2024-04-25T10:18:44.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 Processor -- start 2024-04-25T10:18:44.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 -- start start 2024-04-25T10:18:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595ccb34ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb35400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595ccb35940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834202f50 con 0x5595ccad9920 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340b6700 con 0x5595cca8ff30 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.888+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340b6d00 con 0x5595ccada240 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb35400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb35400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39556/0 (socket says 172.21.15.87:39556) 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 -- 172.21.15.87:0/307018569 learned_addr learned my addr 172.21.15.87:0/307018569 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831e00640 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595ccb34ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1832800640 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595ccb35940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 -- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 msgr2=0x5595ccb35940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595ccb35940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 -- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 msgr2=0x5595ccb34ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595ccb34ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 -- 172.21.15.87:0/307018569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595cc960600 con 0x5595ccad9920 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb35400 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1834069870 tx=0x7f1820009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262dffac9321d7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f181e000640 1 -- 172.21.15.87:0/307018569 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820009cc0 con 0x5595ccad9920 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f181e000640 1 -- 172.21.15.87:0/307018569 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1820026430 con 0x5595ccad9920 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f181e000640 1 -- 172.21.15.87:0/307018569 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820009e60 con 0x5595ccad9920 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- 172.21.15.87:0/307018569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 msgr2=0x5595ccb35400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb35400 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1834069870 tx=0x7f1820009190 comp rx=0 tx=0).stop 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- 172.21.15.87:0/307018569 shutdown_connections 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595ccada240 0x5595ccb35940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb35400 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 --2- 172.21.15.87:0/307018569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595cca8ff30 0x5595ccb34ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- 172.21.15.87:0/307018569 >> 172.21.15.87:0/307018569 conn(0x5595cca962a0 msgr2=0x5595ccae1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- 172.21.15.87:0/307018569 shutdown_connections 2024-04-25T10:18:44.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- 172.21.15.87:0/307018569 wait complete. 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- start start 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 Processor -- start 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595cca8ff30 0x5595ccae7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb1d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccada240 0x5595ccb1d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cc8e33d0 con 0x5595ccad9920 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cc99ad50 con 0x5595ccada240 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595cc879980 con 0x5595cca8ff30 2024-04-25T10:18:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f183c3b0000 10 client.0 _ll_register_callbacks cb 0x7f1818003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb1d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39558/0 (socket says 172.21.15.87:39558) 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.892+0000 7f1831400640 1 -- 172.21.15.87:0/462720685 learned_addr learned my addr 172.21.15.87:0/462720685 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831e00640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595cca8ff30 0x5595ccae7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1832800640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccada240 0x5595ccb1d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831400640 1 -- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595cca8ff30 msgr2=0x5595ccae7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831400640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595cca8ff30 0x5595ccae7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831400640 1 -- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccada240 msgr2=0x5595ccb1d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831400640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccada240 0x5595ccb1d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831400640 1 -- 172.21.15.87:0/462720685 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595cc8739b0 con 0x5595ccad9920 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831e00640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595cca8ff30 0x5595ccae7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831400640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb1d340 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1820008010 tx=0x7f18200046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bf80b32637696d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 1 -- 172.21.15.87:0/462720685 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820004030 con 0x5595ccad9920 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 1 -- 172.21.15.87:0/462720685 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18200041d0 con 0x5595ccad9920 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1820026430 con 0x5595ccad9920 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 1 -- 172.21.15.87:0/462720685 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1820038050 con 0x5595ccad9920 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 10 client.7602 fetch_fsmap learned FSMap version 63 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5595cc8860b0 con 0x5595ccad9920 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 1 -- 172.21.15.87:0/462720685 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1820026430 con 0x5595ccad9920 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 1 -- 172.21.15.87:0/462720685 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f182003b020 con 0x5595ccad9920 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 10 client.7602 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 10 client.7602 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 20 client.7602 populate_metadata read hostname 'smithi087' 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595ccb1e100 0x5595ccb1e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 4 client.7602 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5595cc8ec2e0 con 0x5595ccb1e100 2024-04-25T10:18:44.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831e00640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595ccb1e100 0x5595ccb1e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1831e00640 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595ccb1e100 0x5595ccb1e4e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d9bac3dd84aad5 server_cookie=78c026b73b5cdba6 in_seq=0 out_seq=0 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 10 client.7602 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 1 -- 172.21.15.87:0/462720685 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5595cc8ec2e0 con 0x5595ccb1e100 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 10 client.7602 handle_command_reply: tid=0 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 client.7602 shutdown 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 msgr2=0x5595ccb1d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb1d340 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1820008010 tx=0x7f18200046d0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 shutdown_connections 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5595ccb1e100 0x5595ccb1e4e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5595ccada240 0x5595ccb1d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595ccad9920 0x5595ccb1d340 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 --2- 172.21.15.87:0/462720685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5595cca8ff30 0x5595ccae7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1813400640 0 client.7602 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:18:44.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 >> 172.21.15.87:0/462720685 conn(0x5595cca962a0 msgr2=0x5595ccad7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 shutdown_connections 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 1 -- 172.21.15.87:0/462720685 wait complete. 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f183c3b0000 20 client.7602 trim_cache size 0 max 16384 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 7467, 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:18:44.906 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8384394979999996, 2024-04-25T10:18:44.907 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 63, 2024-04-25T10:18:44.907 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 87, 2024-04-25T10:18:44.907 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T10:18:44.907 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.724815356000001 2024-04-25T10:18:44.907 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1814066f60 msgr2=0x7f1814069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1814066f60 0x7f1814069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1824005000 tx=0x7f1824009040 comp rx=0 tx=0).stop 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183409ea10 msgr2=0x7f183409ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183409ea10 0x7f183409ee90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f182c001ec0 tx=0x7f182c00eb90 comp rx=0 tx=0).stop 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 shutdown_connections 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1814066f60 0x7f1814069420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183409ea10 0x7f183409ee90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18340c72a0 0x7f18340c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 --2- 172.21.15.87:0/4199023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18180039c0 0x7f18341fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 >> 172.21.15.87:0/4199023 conn(0x7f183409a9d0 msgr2=0x7f183415be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 shutdown_connections 2024-04-25T10:18:44.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:44.896+0000 7f1838e00640 1 -- 172.21.15.87:0/4199023 wait complete. 2024-04-25T10:18:44.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:45.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24cd800640 1 Processor -- start 2024-04-25T10:18:45.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24cd800640 1 -- start start 2024-04-25T10:18:45.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24cd800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24c8070c20 0x7f24c815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24cd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c80ae020 0x7f24c80ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24cd800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c80ad840 con 0x7f24c80cf930 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24cd800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c815bb90 con 0x7f24c8070c20 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24cd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c815e9f0 con 0x7f24c80ae020 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24c8070c20 0x7f24c815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24c8070c20 0x7f24c815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39570/0 (socket says 172.21.15.87:39570) 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c6000640 1 -- 172.21.15.87:0/1977622457 learned_addr learned my addr 172.21.15.87:0/1977622457 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:45.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c7400640 1 --2- 172.21.15.87:0/1977622457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c80ae020 0x7f24c80ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c6000640 1 -- 172.21.15.87:0/1977622457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c80ae020 msgr2=0x7f24c80ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:45.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c6000640 1 --2- 172.21.15.87:0/1977622457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c80ae020 0x7f24c80ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c6000640 1 -- 172.21.15.87:0/1977622457 >> v1:172.21.15.169:6789/0 conn(0x7f24c80cf930 legacy=0x7f24c80cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c6000640 1 -- 172.21.15.87:0/1977622457 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c815ecd0 con 0x7f24c8070c20 2024-04-25T10:18:45.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.052+0000 7f24c5600640 1 -- 172.21.15.87:0/1977622457 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984068069 0 0) 0x7f24c80ad840 con 0x7f24c80cf930 2024-04-25T10:18:45.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24c6000640 1 --2- 172.21.15.87:0/1977622457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c8070c20 0x7f24c815e170 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009870 tx=0x7f24b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53469ecab5ad23b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24c5600640 1 -- 172.21.15.87:0/1977622457 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40a95f0 con 0x7f24c8070c20 2024-04-25T10:18:45.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24c5600640 1 -- 172.21.15.87:0/1977622457 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b40a9790 con 0x7f24c8070c20 2024-04-25T10:18:45.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24c5600640 1 -- 172.21.15.87:0/1977622457 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40a9b60 con 0x7f24c8070c20 2024-04-25T10:18:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 -- 172.21.15.87:0/1977622457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c8070c20 msgr2=0x7f24c815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 --2- 172.21.15.87:0/1977622457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c8070c20 0x7f24c815e170 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009870 tx=0x7f24b401c530 comp rx=0 tx=0).stop 2024-04-25T10:18:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 -- 172.21.15.87:0/1977622457 shutdown_connections 2024-04-25T10:18:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 --2- 172.21.15.87:0/1977622457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c80ae020 0x7f24c80ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 --2- 172.21.15.87:0/1977622457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c8070c20 0x7f24c815e170 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 -- 172.21.15.87:0/1977622457 >> 172.21.15.87:0/1977622457 conn(0x7f24c8066b70 msgr2=0x7f24c8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 -- 172.21.15.87:0/1977622457 shutdown_connections 2024-04-25T10:18:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.056+0000 7f24cd800640 1 -- 172.21.15.87:0/1977622457 wait complete. 2024-04-25T10:18:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 Processor -- start 2024-04-25T10:18:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 -- start start 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c8070c20 0x7f24c8152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80ae020 0x7f24c815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c80cf930 0x7f24c815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c816a8e0 con 0x7f24c80ae020 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c815e6b0 con 0x7f24c80cf930 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c816aa60 con 0x7f24c8070c20 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80ae020 0x7f24c815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80ae020 0x7f24c815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39586/0 (socket says 172.21.15.87:39586) 2024-04-25T10:18:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 -- 172.21.15.87:0/3295962795 learned_addr learned my addr 172.21.15.87:0/3295962795 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c7400640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c80cf930 0x7f24c815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6a00640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c8070c20 0x7f24c8152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 -- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c8070c20 msgr2=0x7f24c8152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c8070c20 0x7f24c8152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 -- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c80cf930 msgr2=0x7f24c815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c80cf930 0x7f24c815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 -- 172.21.15.87:0/3295962795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8152880 con 0x7f24c80ae020 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c7400640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c80cf930 0x7f24c815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24c6000640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80ae020 0x7f24c815ac30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f24b401abe0 tx=0x7f24b40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24bea00640 1 -- 172.21.15.87:0/3295962795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40a9e50 con 0x7f24c80ae020 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24bea00640 1 -- 172.21.15.87:0/3295962795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b40b9420 con 0x7f24c80ae020 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c8160af0 con 0x7f24c80ae020 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24bea00640 1 -- 172.21.15.87:0/3295962795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b97f0 con 0x7f24c80ae020 2024-04-25T10:18:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c81538e0 con 0x7f24c80ae020 2024-04-25T10:18:45.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.060+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c804ef30 con 0x7f24c80ae020 2024-04-25T10:18:45.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.064+0000 7f24bea00640 1 -- 172.21.15.87:0/3295962795 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24b40b13f0 con 0x7f24c80ae020 2024-04-25T10:18:45.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.064+0000 7f24bea00640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8067130 0x7f24a80695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.064+0000 7f24bea00640 1 -- 172.21.15.87:0/3295962795 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24b411ea80 con 0x7f24c80ae020 2024-04-25T10:18:45.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.064+0000 7f24c6a00640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8067130 0x7f24a80695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.064+0000 7f24c6a00640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8067130 0x7f24a80695f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f24b80012f0 tx=0x7f24b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.068+0000 7f24bea00640 1 -- 172.21.15.87:0/3295962795 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24b4120e20 con 0x7f24c80ae020 2024-04-25T10:18:45.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24c8155780 con 0x7f24c80ae020 2024-04-25T10:18:45.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24bea00640 1 -- 172.21.15.87:0/3295962795 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7f24b40ad020 con 0x7f24c80ae020 2024-04-25T10:18:45.262 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:45.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8067130 msgr2=0x7f24a80695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8067130 0x7f24a80695f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f24b80012f0 tx=0x7f24b8009040 comp rx=0 tx=0).stop 2024-04-25T10:18:45.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80ae020 msgr2=0x7f24c815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80ae020 0x7f24c815ac30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f24b401abe0 tx=0x7f24b40192d0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 shutdown_connections 2024-04-25T10:18:45.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8067130 0x7f24a80695f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c80cf930 0x7f24c815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80ae020 0x7f24c815ac30 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 --2- 172.21.15.87:0/3295962795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24c8070c20 0x7f24c8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 >> 172.21.15.87:0/3295962795 conn(0x7f24c8066b70 msgr2=0x7f24c80b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:45.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 shutdown_connections 2024-04-25T10:18:45.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:18:45.260+0000 7f24cd800640 1 -- 172.21.15.87:0/3295962795 wait complete. 2024-04-25T10:18:45.283 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:18:45.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:45.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:18:45.283 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:18:45.283 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:45.283 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:18:45.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:18:45.284 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:45.284 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:45.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.284 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:45.293 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42088sec preferred_lft 42088sec 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:18:45.294 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:18:45.295 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:18:45.296 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:45.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.297 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:45.358 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:18:45.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.361 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:45.413 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:45.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.414 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:18:45.414 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:45.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:18:45.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:18:45.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:18:45.414 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:45.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.501 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:18:45.553 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:18:45.553 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:18:45.553 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:45.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.554 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:18:45.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:45.616 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:45.664 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:45.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:45.665 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:45.712 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:18:45.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.712 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:45.800 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:45.800 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:45.800 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:18:45.815 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:45.815 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:45.815 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:18:45.815 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:18:45.815 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:45.823 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:45.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:45.824 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:18:45.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:18:45.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:45.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:18:45.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:18:45.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:18:45.828 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:45.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.828 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:45.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.847 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:45.910 INFO:teuthology.orchestra.run.smithi087.stdout:client7057 2024-04-25T10:18:45.910 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:18:45.910 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:45.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:45.961 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:45.961 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:45.961 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:18:45.961 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:45.961 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:45.962 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:45.962 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:46.049 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.049 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.049 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233367 s, 44.9 MB/s 2024-04-25T10:18:46.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.072 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020346 s, 51.5 MB/s 2024-04-25T10:18:46.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219321 s, 47.8 MB/s 2024-04-25T10:18:46.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236359 s, 44.4 MB/s 2024-04-25T10:18:46.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232932 s, 45.0 MB/s 2024-04-25T10:18:46.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.166 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184838 s, 56.7 MB/s 2024-04-25T10:18:46.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.188 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201994 s, 51.9 MB/s 2024-04-25T10:18:46.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.210 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198314 s, 52.9 MB/s 2024-04-25T10:18:46.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.231 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198943 s, 52.7 MB/s 2024-04-25T10:18:46.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.254 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203929 s, 51.4 MB/s 2024-04-25T10:18:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.277 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216803 s, 48.4 MB/s 2024-04-25T10:18:46.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200628 s, 52.3 MB/s 2024-04-25T10:18:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219968 s, 47.7 MB/s 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225014 s, 46.6 MB/s 2024-04-25T10:18:46.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181659 s, 57.7 MB/s 2024-04-25T10:18:46.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.386 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174977 s, 59.9 MB/s 2024-04-25T10:18:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157577 s, 66.5 MB/s 2024-04-25T10:18:46.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185505 s, 56.5 MB/s 2024-04-25T10:18:46.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.445 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190007 s, 55.2 MB/s 2024-04-25T10:18:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216703 s, 48.4 MB/s 2024-04-25T10:18:46.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.486 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158045 s, 66.3 MB/s 2024-04-25T10:18:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171453 s, 61.2 MB/s 2024-04-25T10:18:46.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.521 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142355 s, 73.7 MB/s 2024-04-25T10:18:46.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.544 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211602 s, 49.6 MB/s 2024-04-25T10:18:46.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.565 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019267 s, 54.4 MB/s 2024-04-25T10:18:46.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205914 s, 50.9 MB/s 2024-04-25T10:18:46.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.609 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196948 s, 53.2 MB/s 2024-04-25T10:18:46.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.629 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184276 s, 56.9 MB/s 2024-04-25T10:18:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.649 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186402 s, 56.3 MB/s 2024-04-25T10:18:46.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.669 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175807 s, 59.6 MB/s 2024-04-25T10:18:46.684 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.685 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140557 s, 74.6 MB/s 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179949 s, 58.3 MB/s 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:46.706 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:46.706 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:46.706 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:18:46.706 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:46.706 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:46.706 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:46.706 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:46.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.785 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192232 s, 54.5 MB/s 2024-04-25T10:18:46.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.804 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173755 s, 60.3 MB/s 2024-04-25T10:18:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.823 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171383 s, 61.2 MB/s 2024-04-25T10:18:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187337 s, 56.0 MB/s 2024-04-25T10:18:46.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.869 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234297 s, 44.8 MB/s 2024-04-25T10:18:46.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166898 s, 62.8 MB/s 2024-04-25T10:18:46.906 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.906 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.906 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173549 s, 60.4 MB/s 2024-04-25T10:18:46.927 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.927 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.928 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194662 s, 53.9 MB/s 2024-04-25T10:18:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.947 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173705 s, 60.4 MB/s 2024-04-25T10:18:46.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.966 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176061 s, 59.6 MB/s 2024-04-25T10:18:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:46.984 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159847 s, 65.6 MB/s 2024-04-25T10:18:47.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.004 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182233 s, 57.5 MB/s 2024-04-25T10:18:47.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.024 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179994 s, 58.3 MB/s 2024-04-25T10:18:47.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.044 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018772 s, 55.9 MB/s 2024-04-25T10:18:47.061 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154168 s, 68.0 MB/s 2024-04-25T10:18:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151729 s, 69.1 MB/s 2024-04-25T10:18:47.103 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228513 s, 45.9 MB/s 2024-04-25T10:18:47.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156907 s, 66.8 MB/s 2024-04-25T10:18:47.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157293 s, 66.7 MB/s 2024-04-25T10:18:47.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.164 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233277 s, 44.9 MB/s 2024-04-25T10:18:47.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.190 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237567 s, 44.1 MB/s 2024-04-25T10:18:47.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014456 s, 72.5 MB/s 2024-04-25T10:18:47.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.230 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214373 s, 48.9 MB/s 2024-04-25T10:18:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202943 s, 51.7 MB/s 2024-04-25T10:18:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152156 s, 68.9 MB/s 2024-04-25T10:18:47.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173951 s, 60.3 MB/s 2024-04-25T10:18:47.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.308 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018082 s, 58.0 MB/s 2024-04-25T10:18:47.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.324 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146415 s, 71.6 MB/s 2024-04-25T10:18:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167855 s, 62.5 MB/s 2024-04-25T10:18:47.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.359 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142617 s, 73.5 MB/s 2024-04-25T10:18:47.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.375 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146734 s, 71.5 MB/s 2024-04-25T10:18:47.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.392 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148551 s, 70.6 MB/s 2024-04-25T10:18:47.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:47.393 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:47.394 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:47.394 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:18:47.394 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:47.394 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:47.394 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:47.394 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:47.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.468 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164542 s, 63.7 MB/s 2024-04-25T10:18:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156411 s, 67.0 MB/s 2024-04-25T10:18:47.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200166 s, 52.4 MB/s 2024-04-25T10:18:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209592 s, 50.0 MB/s 2024-04-25T10:18:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160159 s, 65.5 MB/s 2024-04-25T10:18:47.562 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.562 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.562 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125246 s, 83.7 MB/s 2024-04-25T10:18:47.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.586 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228119 s, 46.0 MB/s 2024-04-25T10:18:47.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.603 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01465 s, 71.6 MB/s 2024-04-25T10:18:47.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.622 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.622 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169168 s, 62.0 MB/s 2024-04-25T10:18:47.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.641 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179134 s, 58.5 MB/s 2024-04-25T10:18:47.661 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.661 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.661 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177869 s, 59.0 MB/s 2024-04-25T10:18:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189114 s, 55.4 MB/s 2024-04-25T10:18:47.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.700 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016278 s, 64.4 MB/s 2024-04-25T10:18:47.719 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.719 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.719 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176927 s, 59.3 MB/s 2024-04-25T10:18:47.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.742 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210678 s, 49.8 MB/s 2024-04-25T10:18:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176468 s, 59.4 MB/s 2024-04-25T10:18:47.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150485 s, 69.7 MB/s 2024-04-25T10:18:47.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.795 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151033 s, 69.4 MB/s 2024-04-25T10:18:47.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189591 s, 55.3 MB/s 2024-04-25T10:18:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014484 s, 72.4 MB/s 2024-04-25T10:18:47.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204066 s, 51.4 MB/s 2024-04-25T10:18:47.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151163 s, 69.4 MB/s 2024-04-25T10:18:47.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.888 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147564 s, 71.1 MB/s 2024-04-25T10:18:47.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.906 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.906 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158944 s, 66.0 MB/s 2024-04-25T10:18:47.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168111 s, 62.4 MB/s 2024-04-25T10:18:47.944 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.944 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.944 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181831 s, 57.7 MB/s 2024-04-25T10:18:47.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.967 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209373 s, 50.1 MB/s 2024-04-25T10:18:47.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:47.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:47.986 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017001 s, 61.7 MB/s 2024-04-25T10:18:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202933 s, 51.7 MB/s 2024-04-25T10:18:48.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.024 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146196 s, 71.7 MB/s 2024-04-25T10:18:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172676 s, 60.7 MB/s 2024-04-25T10:18:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142656 s, 73.5 MB/s 2024-04-25T10:18:48.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:48.061 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:48.061 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:48.061 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:18:48.061 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:48.061 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:48.061 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:48.061 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:48.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166835 s, 62.9 MB/s 2024-04-25T10:18:48.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.156 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.156 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152511 s, 68.8 MB/s 2024-04-25T10:18:48.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.178 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203759 s, 51.5 MB/s 2024-04-25T10:18:48.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.200 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201388 s, 52.1 MB/s 2024-04-25T10:18:48.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206156 s, 50.9 MB/s 2024-04-25T10:18:48.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.237 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130388 s, 80.4 MB/s 2024-04-25T10:18:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162757 s, 64.4 MB/s 2024-04-25T10:18:48.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175655 s, 59.7 MB/s 2024-04-25T10:18:48.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231541 s, 45.3 MB/s 2024-04-25T10:18:48.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.321 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203018 s, 51.6 MB/s 2024-04-25T10:18:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.337 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.338 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146889 s, 71.4 MB/s 2024-04-25T10:18:48.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189624 s, 55.3 MB/s 2024-04-25T10:18:48.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.381 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207309 s, 50.6 MB/s 2024-04-25T10:18:48.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.398 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150117 s, 69.9 MB/s 2024-04-25T10:18:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225709 s, 46.5 MB/s 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153471 s, 68.3 MB/s 2024-04-25T10:18:48.462 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.463 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217001 s, 48.3 MB/s 2024-04-25T10:18:48.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132732 s, 79.0 MB/s 2024-04-25T10:18:48.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.498 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181573 s, 57.7 MB/s 2024-04-25T10:18:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149222 s, 70.3 MB/s 2024-04-25T10:18:48.537 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.537 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.537 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208863 s, 50.2 MB/s 2024-04-25T10:18:48.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015571 s, 67.3 MB/s 2024-04-25T10:18:48.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.575 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182493 s, 57.5 MB/s 2024-04-25T10:18:48.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178474 s, 58.8 MB/s 2024-04-25T10:18:48.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.617 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.617 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207284 s, 50.6 MB/s 2024-04-25T10:18:48.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012328 s, 85.1 MB/s 2024-04-25T10:18:48.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.651 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188279 s, 55.7 MB/s 2024-04-25T10:18:48.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.674 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206954 s, 50.7 MB/s 2024-04-25T10:18:48.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.690 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145649 s, 72.0 MB/s 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020219 s, 51.9 MB/s 2024-04-25T10:18:48.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.727 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125562 s, 83.5 MB/s 2024-04-25T10:18:48.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.744 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164027 s, 63.9 MB/s 2024-04-25T10:18:48.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:48.746 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:48.746 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:48.746 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:18:48.746 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:48.746 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:48.746 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:48.746 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:48.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.820 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173768 s, 60.3 MB/s 2024-04-25T10:18:48.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177855 s, 59.0 MB/s 2024-04-25T10:18:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198289 s, 52.9 MB/s 2024-04-25T10:18:48.878 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.878 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.878 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154026 s, 68.1 MB/s 2024-04-25T10:18:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.895 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155264 s, 67.5 MB/s 2024-04-25T10:18:48.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212036 s, 49.5 MB/s 2024-04-25T10:18:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154572 s, 67.8 MB/s 2024-04-25T10:18:48.959 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.959 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.959 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219067 s, 47.9 MB/s 2024-04-25T10:18:48.975 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.975 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.975 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144938 s, 72.3 MB/s 2024-04-25T10:18:48.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:48.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:48.993 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154808 s, 67.7 MB/s 2024-04-25T10:18:49.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203894 s, 51.4 MB/s 2024-04-25T10:18:49.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.037 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200922 s, 52.2 MB/s 2024-04-25T10:18:49.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.058 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193347 s, 54.2 MB/s 2024-04-25T10:18:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151829 s, 69.1 MB/s 2024-04-25T10:18:49.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.091 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142001 s, 73.8 MB/s 2024-04-25T10:18:49.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209305 s, 50.1 MB/s 2024-04-25T10:18:49.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.133 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174656 s, 60.0 MB/s 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153643 s, 68.2 MB/s 2024-04-25T10:18:49.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.167 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149185 s, 70.3 MB/s 2024-04-25T10:18:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156405 s, 67.0 MB/s 2024-04-25T10:18:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230656 s, 45.5 MB/s 2024-04-25T10:18:49.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.228 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168912 s, 62.1 MB/s 2024-04-25T10:18:49.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206313 s, 50.8 MB/s 2024-04-25T10:18:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151086 s, 69.4 MB/s 2024-04-25T10:18:49.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.290 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197236 s, 53.2 MB/s 2024-04-25T10:18:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165061 s, 63.5 MB/s 2024-04-25T10:18:49.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.324 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144515 s, 72.6 MB/s 2024-04-25T10:18:49.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.341 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147058 s, 71.3 MB/s 2024-04-25T10:18:49.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.355 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125055 s, 83.8 MB/s 2024-04-25T10:18:49.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022709 s, 46.2 MB/s 2024-04-25T10:18:49.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.397 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156601 s, 67.0 MB/s 2024-04-25T10:18:49.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.416 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169971 s, 61.7 MB/s 2024-04-25T10:18:49.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:49.418 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:49.418 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:49.418 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:18:49.418 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:49.418 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:49.418 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:49.418 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:49.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.489 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155995 s, 67.2 MB/s 2024-04-25T10:18:49.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.511 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196585 s, 53.3 MB/s 2024-04-25T10:18:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163755 s, 64.0 MB/s 2024-04-25T10:18:49.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152285 s, 68.9 MB/s 2024-04-25T10:18:49.568 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.568 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.568 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207283 s, 50.6 MB/s 2024-04-25T10:18:49.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.585 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152687 s, 68.7 MB/s 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161504 s, 64.9 MB/s 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151558 s, 69.2 MB/s 2024-04-25T10:18:49.640 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.640 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.640 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179823 s, 58.3 MB/s 2024-04-25T10:18:49.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202774 s, 51.7 MB/s 2024-04-25T10:18:49.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141638 s, 74.0 MB/s 2024-04-25T10:18:49.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.697 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170357 s, 61.6 MB/s 2024-04-25T10:18:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147251 s, 71.2 MB/s 2024-04-25T10:18:49.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.727 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124017 s, 84.6 MB/s 2024-04-25T10:18:49.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.744 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151232 s, 69.3 MB/s 2024-04-25T10:18:49.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.764 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017199 s, 61.0 MB/s 2024-04-25T10:18:49.780 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.780 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.780 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144077 s, 72.8 MB/s 2024-04-25T10:18:49.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181285 s, 57.8 MB/s 2024-04-25T10:18:49.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146799 s, 71.4 MB/s 2024-04-25T10:18:49.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153074 s, 68.5 MB/s 2024-04-25T10:18:49.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177384 s, 59.1 MB/s 2024-04-25T10:18:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204178 s, 51.4 MB/s 2024-04-25T10:18:49.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.897 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203599 s, 51.5 MB/s 2024-04-25T10:18:49.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.916 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163863 s, 64.0 MB/s 2024-04-25T10:18:49.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.937 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201625 s, 52.0 MB/s 2024-04-25T10:18:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.956 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171337 s, 61.2 MB/s 2024-04-25T10:18:49.975 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.975 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.975 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169244 s, 62.0 MB/s 2024-04-25T10:18:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153775 s, 68.2 MB/s 2024-04-25T10:18:50.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.009 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148835 s, 70.5 MB/s 2024-04-25T10:18:50.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.029 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176497 s, 59.4 MB/s 2024-04-25T10:18:50.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.048 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179825 s, 58.3 MB/s 2024-04-25T10:18:50.068 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.068 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.068 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177781 s, 59.0 MB/s 2024-04-25T10:18:50.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:50.069 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:50.070 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:50.070 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:18:50.070 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:50.070 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:50.070 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:50.070 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.148 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169442 s, 61.9 MB/s 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128704 s, 81.5 MB/s 2024-04-25T10:18:50.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.179 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145861 s, 71.9 MB/s 2024-04-25T10:18:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204564 s, 51.3 MB/s 2024-04-25T10:18:50.220 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.220 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.220 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171587 s, 61.1 MB/s 2024-04-25T10:18:50.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.237 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152172 s, 68.9 MB/s 2024-04-25T10:18:50.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.251 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126819 s, 82.7 MB/s 2024-04-25T10:18:50.266 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.266 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.266 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124462 s, 84.2 MB/s 2024-04-25T10:18:50.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.284 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169942 s, 61.7 MB/s 2024-04-25T10:18:50.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.301 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151435 s, 69.2 MB/s 2024-04-25T10:18:50.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.318 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153479 s, 68.3 MB/s 2024-04-25T10:18:50.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.336 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154094 s, 68.0 MB/s 2024-04-25T10:18:50.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.354 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016545 s, 63.4 MB/s 2024-04-25T10:18:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156184 s, 67.1 MB/s 2024-04-25T10:18:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020233 s, 51.8 MB/s 2024-04-25T10:18:50.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.417 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202985 s, 51.7 MB/s 2024-04-25T10:18:50.432 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152533 s, 68.7 MB/s 2024-04-25T10:18:50.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197776 s, 53.0 MB/s 2024-04-25T10:18:50.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149349 s, 70.2 MB/s 2024-04-25T10:18:50.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.487 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145819 s, 71.9 MB/s 2024-04-25T10:18:50.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.504 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155873 s, 67.3 MB/s 2024-04-25T10:18:50.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020542 s, 51.0 MB/s 2024-04-25T10:18:50.547 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.547 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.547 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018636 s, 56.3 MB/s 2024-04-25T10:18:50.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165599 s, 63.3 MB/s 2024-04-25T10:18:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208198 s, 50.4 MB/s 2024-04-25T10:18:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174866 s, 60.0 MB/s 2024-04-25T10:18:50.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.630 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205385 s, 51.1 MB/s 2024-04-25T10:18:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.647 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153237 s, 68.4 MB/s 2024-04-25T10:18:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148171 s, 70.8 MB/s 2024-04-25T10:18:50.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.679 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137469 s, 76.3 MB/s 2024-04-25T10:18:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171225 s, 61.2 MB/s 2024-04-25T10:18:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.721 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208769 s, 50.2 MB/s 2024-04-25T10:18:50.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:50.722 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:50.722 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:50.722 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:18:50.722 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:50.722 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:50.722 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:50.723 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:50.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.795 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169809 s, 61.8 MB/s 2024-04-25T10:18:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020574 s, 51.0 MB/s 2024-04-25T10:18:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180079 s, 58.2 MB/s 2024-04-25T10:18:50.855 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.856 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170175 s, 61.6 MB/s 2024-04-25T10:18:50.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.875 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177458 s, 59.1 MB/s 2024-04-25T10:18:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.892 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153825 s, 68.2 MB/s 2024-04-25T10:18:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.917 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225615 s, 46.5 MB/s 2024-04-25T10:18:50.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015239 s, 68.8 MB/s 2024-04-25T10:18:50.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.952 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170849 s, 61.4 MB/s 2024-04-25T10:18:50.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.970 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153148 s, 68.5 MB/s 2024-04-25T10:18:50.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:50.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:50.992 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203812 s, 51.4 MB/s 2024-04-25T10:18:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151181 s, 69.4 MB/s 2024-04-25T10:18:51.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.030 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200693 s, 52.2 MB/s 2024-04-25T10:18:51.049 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.049 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.049 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016835 s, 62.3 MB/s 2024-04-25T10:18:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180637 s, 58.0 MB/s 2024-04-25T10:18:51.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.087 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155957 s, 67.2 MB/s 2024-04-25T10:18:51.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.101 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126711 s, 82.8 MB/s 2024-04-25T10:18:51.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168677 s, 62.2 MB/s 2024-04-25T10:18:51.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.142 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209993 s, 49.9 MB/s 2024-04-25T10:18:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125876 s, 83.3 MB/s 2024-04-25T10:18:51.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.176 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178101 s, 58.9 MB/s 2024-04-25T10:18:51.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.196 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017807 s, 58.9 MB/s 2024-04-25T10:18:51.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163083 s, 64.3 MB/s 2024-04-25T10:18:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204395 s, 51.3 MB/s 2024-04-25T10:18:51.255 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.255 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.255 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172709 s, 60.7 MB/s 2024-04-25T10:18:51.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.278 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021074 s, 49.8 MB/s 2024-04-25T10:18:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.295 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152066 s, 69.0 MB/s 2024-04-25T10:18:51.312 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.312 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156591 s, 67.0 MB/s 2024-04-25T10:18:51.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.330 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154691 s, 67.8 MB/s 2024-04-25T10:18:51.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.353 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218771 s, 47.9 MB/s 2024-04-25T10:18:51.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118799 s, 88.3 MB/s 2024-04-25T10:18:51.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.384 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152962 s, 68.6 MB/s 2024-04-25T10:18:51.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:51.386 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:51.386 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:51.386 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:18:51.386 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:51.386 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:51.386 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:51.386 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:51.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171544 s, 61.1 MB/s 2024-04-25T10:18:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.481 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198233 s, 52.9 MB/s 2024-04-25T10:18:51.500 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.501 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179475 s, 58.4 MB/s 2024-04-25T10:18:51.517 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.517 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.517 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144851 s, 72.4 MB/s 2024-04-25T10:18:51.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.535 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165993 s, 63.2 MB/s 2024-04-25T10:18:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199269 s, 52.6 MB/s 2024-04-25T10:18:51.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.575 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163926 s, 64.0 MB/s 2024-04-25T10:18:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158746 s, 66.1 MB/s 2024-04-25T10:18:51.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.616 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206661 s, 50.7 MB/s 2024-04-25T10:18:51.632 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.633 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151915 s, 69.0 MB/s 2024-04-25T10:18:51.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203672 s, 51.5 MB/s 2024-04-25T10:18:51.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.672 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.672 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151375 s, 69.3 MB/s 2024-04-25T10:18:51.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.689 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151249 s, 69.3 MB/s 2024-04-25T10:18:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174812 s, 60.0 MB/s 2024-04-25T10:18:51.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.733 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.733 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227963 s, 46.0 MB/s 2024-04-25T10:18:51.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.752 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178769 s, 58.7 MB/s 2024-04-25T10:18:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146976 s, 71.3 MB/s 2024-04-25T10:18:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187721 s, 55.9 MB/s 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175255 s, 59.8 MB/s 2024-04-25T10:18:51.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.830 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200138 s, 52.4 MB/s 2024-04-25T10:18:51.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.846 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144529 s, 72.6 MB/s 2024-04-25T10:18:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206798 s, 50.7 MB/s 2024-04-25T10:18:51.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.890 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199394 s, 52.6 MB/s 2024-04-25T10:18:51.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149244 s, 70.3 MB/s 2024-04-25T10:18:51.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.929 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202983 s, 51.7 MB/s 2024-04-25T10:18:51.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.948 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175534 s, 59.7 MB/s 2024-04-25T10:18:51.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.970 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020224 s, 51.8 MB/s 2024-04-25T10:18:51.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:51.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:51.987 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145232 s, 72.2 MB/s 2024-04-25T10:18:52.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.009 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206514 s, 50.8 MB/s 2024-04-25T10:18:52.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.025 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142665 s, 73.5 MB/s 2024-04-25T10:18:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122201 s, 85.8 MB/s 2024-04-25T10:18:52.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.057 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157452 s, 66.6 MB/s 2024-04-25T10:18:52.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:52.058 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:52.059 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:52.059 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:18:52.059 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:52.059 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:52.059 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:52.059 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:52.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.142 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216177 s, 48.5 MB/s 2024-04-25T10:18:52.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147678 s, 71.0 MB/s 2024-04-25T10:18:52.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209124 s, 50.1 MB/s 2024-04-25T10:18:52.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.202 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199691 s, 52.5 MB/s 2024-04-25T10:18:52.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203943 s, 51.4 MB/s 2024-04-25T10:18:52.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.241 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014996 s, 69.9 MB/s 2024-04-25T10:18:52.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.259 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156946 s, 66.8 MB/s 2024-04-25T10:18:52.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.279 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181018 s, 57.9 MB/s 2024-04-25T10:18:52.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186497 s, 56.2 MB/s 2024-04-25T10:18:52.320 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.320 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.321 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196242 s, 53.4 MB/s 2024-04-25T10:18:52.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173912 s, 60.3 MB/s 2024-04-25T10:18:52.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.521 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186895 s, 56.1 MB/s 2024-04-25T10:18:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147672 s, 71.0 MB/s 2024-04-25T10:18:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209398 s, 50.1 MB/s 2024-04-25T10:18:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124756 s, 84.0 MB/s 2024-04-25T10:18:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128176 s, 81.8 MB/s 2024-04-25T10:18:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214386 s, 48.9 MB/s 2024-04-25T10:18:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176492 s, 59.4 MB/s 2024-04-25T10:18:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181003 s, 57.9 MB/s 2024-04-25T10:18:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145019 s, 72.3 MB/s 2024-04-25T10:18:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173013 s, 60.6 MB/s 2024-04-25T10:18:52.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.553 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.553 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214317 s, 48.9 MB/s 2024-04-25T10:18:52.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.575 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201285 s, 52.1 MB/s 2024-04-25T10:18:52.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161033 s, 65.1 MB/s 2024-04-25T10:18:52.607 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.607 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.607 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148945 s, 70.4 MB/s 2024-04-25T10:18:52.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.629 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014815 s, 70.8 MB/s 2024-04-25T10:18:52.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.647 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204629 s, 51.2 MB/s 2024-04-25T10:18:52.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.664 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170491 s, 61.5 MB/s 2024-04-25T10:18:52.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.686 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160131 s, 65.5 MB/s 2024-04-25T10:18:52.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.703 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201132 s, 52.1 MB/s 2024-04-25T10:18:52.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.721 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156427 s, 67.0 MB/s 2024-04-25T10:18:52.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.742 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198672 s, 52.8 MB/s 2024-04-25T10:18:52.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:52.744 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:52.744 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:52.744 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:18:52.744 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:52.744 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:52.744 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:52.744 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:52.819 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.819 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.819 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162055 s, 64.7 MB/s 2024-04-25T10:18:52.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.836 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154475 s, 67.9 MB/s 2024-04-25T10:18:52.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143939 s, 72.8 MB/s 2024-04-25T10:18:52.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.877 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219828 s, 47.7 MB/s 2024-04-25T10:18:52.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.895 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170612 s, 61.5 MB/s 2024-04-25T10:18:52.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.909 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116264 s, 90.2 MB/s 2024-04-25T10:18:52.925 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.925 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.925 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143739 s, 72.9 MB/s 2024-04-25T10:18:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145163 s, 72.2 MB/s 2024-04-25T10:18:52.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.964 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209848 s, 50.0 MB/s 2024-04-25T10:18:52.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:52.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:52.986 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198462 s, 52.8 MB/s 2024-04-25T10:18:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.005 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170146 s, 61.6 MB/s 2024-04-25T10:18:53.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150499 s, 69.7 MB/s 2024-04-25T10:18:53.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156324 s, 67.1 MB/s 2024-04-25T10:18:53.042 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:53.045 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:53.059 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.060 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188916 s, 55.5 MB/s 2024-04-25T10:18:53.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170047 s, 61.7 MB/s 2024-04-25T10:18:53.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.094 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141707 s, 74.0 MB/s 2024-04-25T10:18:53.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145798 s, 71.9 MB/s 2024-04-25T10:18:53.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.128 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156157 s, 67.1 MB/s 2024-04-25T10:18:53.153 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.153 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.153 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233867 s, 44.8 MB/s 2024-04-25T10:18:53.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115886 s, 90.5 MB/s 2024-04-25T10:18:53.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.186 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176377 s, 59.5 MB/s 2024-04-25T10:18:53.203 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.203 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.203 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015377 s, 68.2 MB/s 2024-04-25T10:18:53.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172104 s, 60.9 MB/s 2024-04-25T10:18:53.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.242 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174187 s, 60.2 MB/s 2024-04-25T10:18:53.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.258 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152264 s, 68.9 MB/s 2024-04-25T10:18:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143525 s, 73.1 MB/s 2024-04-25T10:18:53.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.291 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141813 s, 73.9 MB/s 2024-04-25T10:18:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146644 s, 71.5 MB/s 2024-04-25T10:18:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128089 s, 81.9 MB/s 2024-04-25T10:18:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188517 s, 55.6 MB/s 2024-04-25T10:18:53.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.361 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174431 s, 60.1 MB/s 2024-04-25T10:18:53.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.378 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143851 s, 72.9 MB/s 2024-04-25T10:18:53.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:53.379 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:53.379 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:53.379 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:18:53.379 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:53.379 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:53.379 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:53.379 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:53.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.448 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143262 s, 73.2 MB/s 2024-04-25T10:18:53.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.465 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155749 s, 67.3 MB/s 2024-04-25T10:18:53.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.482 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147476 s, 71.1 MB/s 2024-04-25T10:18:53.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015464 s, 67.8 MB/s 2024-04-25T10:18:53.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.516 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155327 s, 67.5 MB/s 2024-04-25T10:18:53.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.535 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166253 s, 63.1 MB/s 2024-04-25T10:18:53.557 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.557 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.557 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206887 s, 50.7 MB/s 2024-04-25T10:18:53.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.574 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015199 s, 69.0 MB/s 2024-04-25T10:18:53.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.592 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161944 s, 64.7 MB/s 2024-04-25T10:18:53.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.611 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175739 s, 59.7 MB/s 2024-04-25T10:18:53.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217195 s, 48.3 MB/s 2024-04-25T10:18:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140854 s, 74.4 MB/s 2024-04-25T10:18:53.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.670 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172558 s, 60.8 MB/s 2024-04-25T10:18:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168846 s, 62.1 MB/s 2024-04-25T10:18:53.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.707 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172975 s, 60.6 MB/s 2024-04-25T10:18:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174241 s, 60.2 MB/s 2024-04-25T10:18:53.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.739 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011216 s, 93.5 MB/s 2024-04-25T10:18:53.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.755 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144347 s, 72.6 MB/s 2024-04-25T10:18:53.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.772 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150651 s, 69.6 MB/s 2024-04-25T10:18:53.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.790 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158037 s, 66.3 MB/s 2024-04-25T10:18:53.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.811 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019563 s, 53.6 MB/s 2024-04-25T10:18:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141123 s, 74.3 MB/s 2024-04-25T10:18:53.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.846 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176242 s, 59.5 MB/s 2024-04-25T10:18:53.867 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.867 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.867 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186701 s, 56.2 MB/s 2024-04-25T10:18:53.886 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.886 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.886 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175364 s, 59.8 MB/s 2024-04-25T10:18:53.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.909 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213547 s, 49.1 MB/s 2024-04-25T10:18:53.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.931 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198681 s, 52.8 MB/s 2024-04-25T10:18:53.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.950 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175054 s, 59.9 MB/s 2024-04-25T10:18:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176541 s, 59.4 MB/s 2024-04-25T10:18:53.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:53.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:53.991 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201731 s, 52.0 MB/s 2024-04-25T10:18:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143667 s, 73.0 MB/s 2024-04-25T10:18:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202009 s, 51.9 MB/s 2024-04-25T10:18:54.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:54.031 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:54.031 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:54.031 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:18:54.031 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:54.031 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:54.032 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:54.032 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:54.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.106 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156911 s, 66.8 MB/s 2024-04-25T10:18:54.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.126 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175419 s, 59.8 MB/s 2024-04-25T10:18:54.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018041 s, 58.1 MB/s 2024-04-25T10:18:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143796 s, 72.9 MB/s 2024-04-25T10:18:54.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178668 s, 58.7 MB/s 2024-04-25T10:18:54.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156602 s, 67.0 MB/s 2024-04-25T10:18:54.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.217 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160046 s, 65.5 MB/s 2024-04-25T10:18:54.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.237 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177906 s, 58.9 MB/s 2024-04-25T10:18:54.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.257 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183368 s, 57.2 MB/s 2024-04-25T10:18:54.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.278 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182172 s, 57.6 MB/s 2024-04-25T10:18:54.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.296 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.296 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165398 s, 63.4 MB/s 2024-04-25T10:18:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145474 s, 72.1 MB/s 2024-04-25T10:18:54.328 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.328 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.328 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143157 s, 73.2 MB/s 2024-04-25T10:18:54.346 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.347 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.347 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164475 s, 63.8 MB/s 2024-04-25T10:18:54.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181683 s, 57.7 MB/s 2024-04-25T10:18:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146326 s, 71.7 MB/s 2024-04-25T10:18:54.402 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.402 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.402 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171347 s, 61.2 MB/s 2024-04-25T10:18:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.418 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142443 s, 73.6 MB/s 2024-04-25T10:18:54.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.434 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.434 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142303 s, 73.7 MB/s 2024-04-25T10:18:54.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168941 s, 62.1 MB/s 2024-04-25T10:18:54.475 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.475 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.475 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209501 s, 50.1 MB/s 2024-04-25T10:18:54.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.495 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017563 s, 59.7 MB/s 2024-04-25T10:18:54.517 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.517 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.517 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210619 s, 49.8 MB/s 2024-04-25T10:18:54.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.537 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.537 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173546 s, 60.4 MB/s 2024-04-25T10:18:54.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021658 s, 48.4 MB/s 2024-04-25T10:18:54.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.584 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212443 s, 49.4 MB/s 2024-04-25T10:18:54.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.600 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155592 s, 67.4 MB/s 2024-04-25T10:18:54.617 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.618 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154088 s, 68.1 MB/s 2024-04-25T10:18:54.637 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.637 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.637 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018043 s, 58.1 MB/s 2024-04-25T10:18:54.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129811 s, 80.8 MB/s 2024-04-25T10:18:54.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.671 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169672 s, 61.8 MB/s 2024-04-25T10:18:54.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.690 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170893 s, 61.4 MB/s 2024-04-25T10:18:54.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:54.691 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:54.691 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:54.691 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:18:54.691 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:54.692 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:54.692 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:54.692 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142281 s, 73.7 MB/s 2024-04-25T10:18:54.782 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.782 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.782 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197527 s, 53.1 MB/s 2024-04-25T10:18:54.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.798 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144876 s, 72.4 MB/s 2024-04-25T10:18:54.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.820 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200111 s, 52.4 MB/s 2024-04-25T10:18:54.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.837 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015775 s, 66.5 MB/s 2024-04-25T10:18:54.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229753 s, 45.6 MB/s 2024-04-25T10:18:54.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.881 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168194 s, 62.3 MB/s 2024-04-25T10:18:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162004 s, 64.7 MB/s 2024-04-25T10:18:54.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.917 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164813 s, 63.6 MB/s 2024-04-25T10:18:54.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210786 s, 49.7 MB/s 2024-04-25T10:18:54.955 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.955 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.955 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139361 s, 75.2 MB/s 2024-04-25T10:18:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113886 s, 92.1 MB/s 2024-04-25T10:18:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115564 s, 90.7 MB/s 2024-04-25T10:18:55.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.006 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225395 s, 46.5 MB/s 2024-04-25T10:18:55.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.019 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011935 s, 87.9 MB/s 2024-04-25T10:18:55.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170218 s, 61.6 MB/s 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145362 s, 72.1 MB/s 2024-04-25T10:18:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154123 s, 68.0 MB/s 2024-04-25T10:18:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.093 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187427 s, 55.9 MB/s 2024-04-25T10:18:55.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153373 s, 68.4 MB/s 2024-04-25T10:18:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142446 s, 73.6 MB/s 2024-04-25T10:18:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177183 s, 59.2 MB/s 2024-04-25T10:18:55.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154779 s, 67.7 MB/s 2024-04-25T10:18:55.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.182 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165649 s, 63.3 MB/s 2024-04-25T10:18:55.204 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.205 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213687 s, 49.1 MB/s 2024-04-25T10:18:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169885 s, 61.7 MB/s 2024-04-25T10:18:55.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.240 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014482 s, 72.4 MB/s 2024-04-25T10:18:55.255 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.256 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.256 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140767 s, 74.5 MB/s 2024-04-25T10:18:55.276 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.277 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192791 s, 54.4 MB/s 2024-04-25T10:18:55.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.293 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144774 s, 72.4 MB/s 2024-04-25T10:18:55.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.309 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139889 s, 75.0 MB/s 2024-04-25T10:18:55.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.327 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164968 s, 63.6 MB/s 2024-04-25T10:18:55.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:55.328 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:55.328 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:55.328 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:18:55.328 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:55.328 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:55.329 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:55.329 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:55.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.399 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163346 s, 64.2 MB/s 2024-04-25T10:18:55.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.418 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183156 s, 57.3 MB/s 2024-04-25T10:18:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.442 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214903 s, 48.8 MB/s 2024-04-25T10:18:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155403 s, 67.5 MB/s 2024-04-25T10:18:55.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.485 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235804 s, 44.5 MB/s 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155734 s, 67.3 MB/s 2024-04-25T10:18:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206676 s, 50.7 MB/s 2024-04-25T10:18:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173005 s, 60.6 MB/s 2024-04-25T10:18:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147143 s, 71.3 MB/s 2024-04-25T10:18:55.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.577 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149449 s, 70.2 MB/s 2024-04-25T10:18:55.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.595 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170132 s, 61.6 MB/s 2024-04-25T10:18:55.612 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.612 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.612 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014698 s, 71.3 MB/s 2024-04-25T10:18:55.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.628 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141955 s, 73.9 MB/s 2024-04-25T10:18:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175622 s, 59.7 MB/s 2024-04-25T10:18:55.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.664 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146185 s, 71.7 MB/s 2024-04-25T10:18:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173836 s, 60.3 MB/s 2024-04-25T10:18:55.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.699 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146743 s, 71.5 MB/s 2024-04-25T10:18:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149904 s, 69.9 MB/s 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172194 s, 60.9 MB/s 2024-04-25T10:18:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169519 s, 61.9 MB/s 2024-04-25T10:18:55.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.773 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173672 s, 60.4 MB/s 2024-04-25T10:18:55.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.790 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151069 s, 69.4 MB/s 2024-04-25T10:18:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158779 s, 66.0 MB/s 2024-04-25T10:18:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186332 s, 56.3 MB/s 2024-04-25T10:18:55.847 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.847 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.847 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172099 s, 60.9 MB/s 2024-04-25T10:18:55.866 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.867 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.867 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177502 s, 59.1 MB/s 2024-04-25T10:18:55.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.884 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157241 s, 66.7 MB/s 2024-04-25T10:18:55.901 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.901 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.901 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152997 s, 68.5 MB/s 2024-04-25T10:18:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199525 s, 52.6 MB/s 2024-04-25T10:18:55.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.939 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141622 s, 74.0 MB/s 2024-04-25T10:18:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.954 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141075 s, 74.3 MB/s 2024-04-25T10:18:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:55.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167444 s, 62.6 MB/s 2024-04-25T10:18:55.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:55.975 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:55.975 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:55.975 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:18:55.975 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:55.975 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:55.975 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:55.975 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:56.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.048 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017468 s, 60.0 MB/s 2024-04-25T10:18:56.067 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.067 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.067 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173846 s, 60.3 MB/s 2024-04-25T10:18:56.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.089 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198089 s, 52.9 MB/s 2024-04-25T10:18:56.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.105 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145315 s, 72.2 MB/s 2024-04-25T10:18:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142625 s, 73.5 MB/s 2024-04-25T10:18:56.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.141 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174454 s, 60.1 MB/s 2024-04-25T10:18:56.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015871 s, 66.1 MB/s 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186551 s, 56.2 MB/s 2024-04-25T10:18:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174478 s, 60.1 MB/s 2024-04-25T10:18:56.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.212 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.212 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123931 s, 84.6 MB/s 2024-04-25T10:18:56.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.231 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177881 s, 58.9 MB/s 2024-04-25T10:18:56.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.248 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153581 s, 68.3 MB/s 2024-04-25T10:18:56.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.268 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017489 s, 60.0 MB/s 2024-04-25T10:18:56.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181676 s, 57.7 MB/s 2024-04-25T10:18:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149997 s, 69.9 MB/s 2024-04-25T10:18:56.320 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.321 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144851 s, 72.4 MB/s 2024-04-25T10:18:56.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174839 s, 60.0 MB/s 2024-04-25T10:18:56.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018594 s, 56.4 MB/s 2024-04-25T10:18:56.376 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.376 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.376 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140042 s, 74.9 MB/s 2024-04-25T10:18:56.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.400 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221999 s, 47.2 MB/s 2024-04-25T10:18:56.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.418 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016463 s, 63.7 MB/s 2024-04-25T10:18:56.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.437 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170519 s, 61.5 MB/s 2024-04-25T10:18:56.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.459 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196642 s, 53.3 MB/s 2024-04-25T10:18:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020252 s, 51.8 MB/s 2024-04-25T10:18:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167132 s, 62.7 MB/s 2024-04-25T10:18:56.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.515 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147679 s, 71.0 MB/s 2024-04-25T10:18:56.533 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.533 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.533 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155359 s, 67.5 MB/s 2024-04-25T10:18:56.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.558 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239246 s, 43.8 MB/s 2024-04-25T10:18:56.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015688 s, 66.8 MB/s 2024-04-25T10:18:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141506 s, 74.1 MB/s 2024-04-25T10:18:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169544 s, 61.8 MB/s 2024-04-25T10:18:56.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.634 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214612 s, 48.9 MB/s 2024-04-25T10:18:56.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:56.635 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:56.635 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:56.635 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:18:56.635 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:56.635 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:56.635 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:56.635 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:56.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.704 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146109 s, 71.8 MB/s 2024-04-25T10:18:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014727 s, 71.2 MB/s 2024-04-25T10:18:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.738 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149235 s, 70.3 MB/s 2024-04-25T10:18:56.764 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.764 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.764 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247129 s, 42.4 MB/s 2024-04-25T10:18:56.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.781 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154699 s, 67.8 MB/s 2024-04-25T10:18:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170864 s, 61.4 MB/s 2024-04-25T10:18:56.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147922 s, 70.9 MB/s 2024-04-25T10:18:56.832 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.833 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014248 s, 73.6 MB/s 2024-04-25T10:18:56.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182302 s, 57.5 MB/s 2024-04-25T10:18:56.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.872 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178655 s, 58.7 MB/s 2024-04-25T10:18:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165468 s, 63.4 MB/s 2024-04-25T10:18:56.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.912 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193263 s, 54.3 MB/s 2024-04-25T10:18:56.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.928 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146268 s, 71.7 MB/s 2024-04-25T10:18:56.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.954 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236793 s, 44.3 MB/s 2024-04-25T10:18:56.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.970 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146202 s, 71.7 MB/s 2024-04-25T10:18:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148197 s, 70.8 MB/s 2024-04-25T10:18:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174788 s, 60.0 MB/s 2024-04-25T10:18:57.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.022 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143786 s, 72.9 MB/s 2024-04-25T10:18:57.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170177 s, 61.6 MB/s 2024-04-25T10:18:57.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.059 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156359 s, 67.1 MB/s 2024-04-25T10:18:57.076 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.076 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.076 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163122 s, 64.3 MB/s 2024-04-25T10:18:57.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176044 s, 59.6 MB/s 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014457 s, 72.5 MB/s 2024-04-25T10:18:57.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169209 s, 62.0 MB/s 2024-04-25T10:18:57.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.148 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156534 s, 67.0 MB/s 2024-04-25T10:18:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144462 s, 72.6 MB/s 2024-04-25T10:18:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.182 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158484 s, 66.2 MB/s 2024-04-25T10:18:57.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236479 s, 44.3 MB/s 2024-04-25T10:18:57.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.224 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144458 s, 72.6 MB/s 2024-04-25T10:18:57.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.246 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202127 s, 51.9 MB/s 2024-04-25T10:18:57.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.267 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199131 s, 52.7 MB/s 2024-04-25T10:18:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.285 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152028 s, 69.0 MB/s 2024-04-25T10:18:57.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:57.286 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:57.286 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:57.286 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:18:57.286 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:57.287 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:57.287 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:57.287 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143665 s, 73.0 MB/s 2024-04-25T10:18:57.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.374 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159267 s, 65.8 MB/s 2024-04-25T10:18:57.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.391 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151418 s, 69.3 MB/s 2024-04-25T10:18:57.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.410 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172282 s, 60.9 MB/s 2024-04-25T10:18:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.431 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190267 s, 55.1 MB/s 2024-04-25T10:18:57.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.447 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014496 s, 72.3 MB/s 2024-04-25T10:18:57.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.465 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160294 s, 65.4 MB/s 2024-04-25T10:18:57.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.484 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170684 s, 61.4 MB/s 2024-04-25T10:18:57.500 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.500 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.500 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148459 s, 70.6 MB/s 2024-04-25T10:18:57.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.517 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143837 s, 72.9 MB/s 2024-04-25T10:18:57.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.538 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195857 s, 53.5 MB/s 2024-04-25T10:18:57.555 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.555 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.555 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158664 s, 66.1 MB/s 2024-04-25T10:18:57.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015143 s, 69.2 MB/s 2024-04-25T10:18:57.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.591 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167039 s, 62.8 MB/s 2024-04-25T10:18:57.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.615 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221616 s, 47.3 MB/s 2024-04-25T10:18:57.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.634 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176111 s, 59.5 MB/s 2024-04-25T10:18:57.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.657 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210092 s, 49.9 MB/s 2024-04-25T10:18:57.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.673 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014647 s, 71.6 MB/s 2024-04-25T10:18:57.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.689 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144132 s, 72.8 MB/s 2024-04-25T10:18:57.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016415 s, 63.9 MB/s 2024-04-25T10:18:57.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.724 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141912 s, 73.9 MB/s 2024-04-25T10:18:57.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.739 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139392 s, 75.2 MB/s 2024-04-25T10:18:57.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.762 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204767 s, 51.2 MB/s 2024-04-25T10:18:57.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.781 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174761 s, 60.0 MB/s 2024-04-25T10:18:57.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.803 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200673 s, 52.3 MB/s 2024-04-25T10:18:57.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.819 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.819 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142313 s, 73.7 MB/s 2024-04-25T10:18:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146398 s, 71.6 MB/s 2024-04-25T10:18:57.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171946 s, 61.0 MB/s 2024-04-25T10:18:57.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147085 s, 71.3 MB/s 2024-04-25T10:18:57.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146482 s, 71.6 MB/s 2024-04-25T10:18:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.903 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146418 s, 71.6 MB/s 2024-04-25T10:18:57.925 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.926 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.926 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202555 s, 51.8 MB/s 2024-04-25T10:18:57.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:57.927 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:57.927 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:57.927 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:18:57.927 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:57.927 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:57.927 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:57.927 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:57.998 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:57.998 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:57.999 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154919 s, 67.7 MB/s 2024-04-25T10:18:58.020 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.021 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203358 s, 51.6 MB/s 2024-04-25T10:18:58.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.038 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152163 s, 68.9 MB/s 2024-04-25T10:18:58.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.063 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239095 s, 43.9 MB/s 2024-04-25T10:18:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01983 s, 52.9 MB/s 2024-04-25T10:18:58.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.101 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145706 s, 72.0 MB/s 2024-04-25T10:18:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179203 s, 58.5 MB/s 2024-04-25T10:18:58.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.141 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186475 s, 56.2 MB/s 2024-04-25T10:18:58.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202089 s, 51.9 MB/s 2024-04-25T10:18:58.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.182 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170325 s, 61.6 MB/s 2024-04-25T10:18:58.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.201 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169709 s, 61.8 MB/s 2024-04-25T10:18:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168629 s, 62.2 MB/s 2024-04-25T10:18:58.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141213 s, 74.3 MB/s 2024-04-25T10:18:58.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.260 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229359 s, 45.7 MB/s 2024-04-25T10:18:58.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.278 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165991 s, 63.2 MB/s 2024-04-25T10:18:58.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.295 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152136 s, 68.9 MB/s 2024-04-25T10:18:58.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.313 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016347 s, 64.1 MB/s 2024-04-25T10:18:58.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.330 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155334 s, 67.5 MB/s 2024-04-25T10:18:58.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156788 s, 66.9 MB/s 2024-04-25T10:18:58.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169306 s, 61.9 MB/s 2024-04-25T10:18:58.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.385 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164039 s, 63.9 MB/s 2024-04-25T10:18:58.399 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.400 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126817 s, 82.7 MB/s 2024-04-25T10:18:58.421 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.422 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203381 s, 51.6 MB/s 2024-04-25T10:18:58.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.438 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142202 s, 73.7 MB/s 2024-04-25T10:18:58.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.463 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232796 s, 45.0 MB/s 2024-04-25T10:18:58.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.486 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217094 s, 48.3 MB/s 2024-04-25T10:18:58.505 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.506 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176941 s, 59.3 MB/s 2024-04-25T10:18:58.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212911 s, 49.2 MB/s 2024-04-25T10:18:58.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014774 s, 71.0 MB/s 2024-04-25T10:18:58.567 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.567 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.567 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198009 s, 53.0 MB/s 2024-04-25T10:18:58.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.589 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208062 s, 50.4 MB/s 2024-04-25T10:18:58.605 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.606 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.606 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147211 s, 71.2 MB/s 2024-04-25T10:18:58.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:58.607 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:58.607 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:58.607 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:18:58.607 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:58.607 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:58.607 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:58.607 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:58.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.680 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173809 s, 60.3 MB/s 2024-04-25T10:18:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214323 s, 48.9 MB/s 2024-04-25T10:18:58.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.721 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165513 s, 63.4 MB/s 2024-04-25T10:18:58.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.740 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171005 s, 61.3 MB/s 2024-04-25T10:18:58.756 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.757 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150359 s, 69.7 MB/s 2024-04-25T10:18:58.775 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.776 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.776 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171723 s, 61.1 MB/s 2024-04-25T10:18:58.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.799 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217195 s, 48.3 MB/s 2024-04-25T10:18:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205681 s, 51.0 MB/s 2024-04-25T10:18:58.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.838 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146493 s, 71.6 MB/s 2024-04-25T10:18:58.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.861 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208107 s, 50.4 MB/s 2024-04-25T10:18:58.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.877 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143458 s, 73.1 MB/s 2024-04-25T10:18:58.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.896 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177379 s, 59.1 MB/s 2024-04-25T10:18:58.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.915 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172909 s, 60.6 MB/s 2024-04-25T10:18:58.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.932 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151125 s, 69.4 MB/s 2024-04-25T10:18:58.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.951 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168128 s, 62.4 MB/s 2024-04-25T10:18:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208088 s, 50.4 MB/s 2024-04-25T10:18:58.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:58.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:58.989 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142556 s, 73.6 MB/s 2024-04-25T10:18:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171809 s, 61.0 MB/s 2024-04-25T10:18:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148726 s, 70.5 MB/s 2024-04-25T10:18:59.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014281 s, 73.4 MB/s 2024-04-25T10:18:59.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.060 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171552 s, 61.1 MB/s 2024-04-25T10:18:59.077 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.077 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.077 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152951 s, 68.6 MB/s 2024-04-25T10:18:59.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.094 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148536 s, 70.6 MB/s 2024-04-25T10:18:59.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.112 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159947 s, 65.6 MB/s 2024-04-25T10:18:59.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.129 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153078 s, 68.5 MB/s 2024-04-25T10:18:59.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.147 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159162 s, 65.9 MB/s 2024-04-25T10:18:59.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.164 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153166 s, 68.5 MB/s 2024-04-25T10:18:59.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.183 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171948 s, 61.0 MB/s 2024-04-25T10:18:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242719 s, 43.2 MB/s 2024-04-25T10:18:59.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181224 s, 57.9 MB/s 2024-04-25T10:18:59.253 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.253 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.253 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222187 s, 47.2 MB/s 2024-04-25T10:18:59.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.275 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.275 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197203 s, 53.2 MB/s 2024-04-25T10:18:59.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:59.276 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:59.276 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:59.276 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:18:59.276 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:59.277 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:59.277 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:59.277 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018294 s, 57.3 MB/s 2024-04-25T10:18:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207985 s, 50.4 MB/s 2024-04-25T10:18:59.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167031 s, 62.8 MB/s 2024-04-25T10:18:59.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.407 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153216 s, 68.4 MB/s 2024-04-25T10:18:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.432 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024185 s, 43.4 MB/s 2024-04-25T10:18:59.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204638 s, 51.2 MB/s 2024-04-25T10:18:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170536 s, 61.5 MB/s 2024-04-25T10:18:59.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.490 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153386 s, 68.4 MB/s 2024-04-25T10:18:59.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.509 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169381 s, 61.9 MB/s 2024-04-25T10:18:59.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151242 s, 69.3 MB/s 2024-04-25T10:18:59.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.542 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148436 s, 70.6 MB/s 2024-04-25T10:18:59.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216864 s, 48.4 MB/s 2024-04-25T10:18:59.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.582 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143292 s, 73.2 MB/s 2024-04-25T10:18:59.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.599 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154591 s, 67.8 MB/s 2024-04-25T10:18:59.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.619 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175506 s, 59.7 MB/s 2024-04-25T10:18:59.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142052 s, 73.8 MB/s 2024-04-25T10:18:59.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151309 s, 69.3 MB/s 2024-04-25T10:18:59.667 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.668 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014044 s, 74.7 MB/s 2024-04-25T10:18:59.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173613 s, 60.4 MB/s 2024-04-25T10:18:59.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164355 s, 63.8 MB/s 2024-04-25T10:18:59.722 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.722 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.722 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015428 s, 68.0 MB/s 2024-04-25T10:18:59.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.737 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127383 s, 82.3 MB/s 2024-04-25T10:18:59.756 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.756 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.756 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180092 s, 58.2 MB/s 2024-04-25T10:18:59.776 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.776 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.776 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182141 s, 57.6 MB/s 2024-04-25T10:18:59.796 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.796 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.796 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176467 s, 59.4 MB/s 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151567 s, 69.2 MB/s 2024-04-25T10:18:59.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.829 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146119 s, 71.8 MB/s 2024-04-25T10:18:59.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.846 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149029 s, 70.4 MB/s 2024-04-25T10:18:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236861 s, 44.3 MB/s 2024-04-25T10:18:59.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.888 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150582 s, 69.6 MB/s 2024-04-25T10:18:59.906 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.906 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.906 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160335 s, 65.4 MB/s 2024-04-25T10:18:59.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.922 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140886 s, 74.4 MB/s 2024-04-25T10:18:59.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:59.923 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:18:59.924 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:18:59.924 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:18:59.924 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:18:59.924 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:18:59.924 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:18:59.924 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:18:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:18:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:18:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200167 s, 52.4 MB/s 2024-04-25T10:19:00.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152041 s, 69.0 MB/s 2024-04-25T10:19:00.033 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.033 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.033 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157373 s, 66.6 MB/s 2024-04-25T10:19:00.049 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.050 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.050 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149701 s, 70.0 MB/s 2024-04-25T10:19:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220671 s, 47.5 MB/s 2024-04-25T10:19:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217364 s, 48.2 MB/s 2024-04-25T10:19:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155425 s, 67.5 MB/s 2024-04-25T10:19:00.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143529 s, 73.1 MB/s 2024-04-25T10:19:00.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.151 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182098 s, 57.6 MB/s 2024-04-25T10:19:00.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.174 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212479 s, 49.3 MB/s 2024-04-25T10:19:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199004 s, 52.7 MB/s 2024-04-25T10:19:00.212 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.212 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.212 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150391 s, 69.7 MB/s 2024-04-25T10:19:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197598 s, 53.1 MB/s 2024-04-25T10:19:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144046 s, 72.8 MB/s 2024-04-25T10:19:00.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.267 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015216 s, 68.9 MB/s 2024-04-25T10:19:00.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.284 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152728 s, 68.7 MB/s 2024-04-25T10:19:00.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.306 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.306 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198412 s, 52.8 MB/s 2024-04-25T10:19:00.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146875 s, 71.4 MB/s 2024-04-25T10:19:00.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147211 s, 71.2 MB/s 2024-04-25T10:19:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017001 s, 61.7 MB/s 2024-04-25T10:19:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207786 s, 50.5 MB/s 2024-04-25T10:19:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147366 s, 71.2 MB/s 2024-04-25T10:19:00.413 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.413 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.413 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014661 s, 71.5 MB/s 2024-04-25T10:19:00.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.439 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236563 s, 44.3 MB/s 2024-04-25T10:19:00.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.458 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174722 s, 60.0 MB/s 2024-04-25T10:19:00.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.480 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199995 s, 52.4 MB/s 2024-04-25T10:19:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157092 s, 66.7 MB/s 2024-04-25T10:19:00.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.516 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172402 s, 60.8 MB/s 2024-04-25T10:19:00.533 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.533 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.533 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153189 s, 68.4 MB/s 2024-04-25T10:19:00.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.556 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211132 s, 49.7 MB/s 2024-04-25T10:19:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154485 s, 67.9 MB/s 2024-04-25T10:19:00.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.596 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.596 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204327 s, 51.3 MB/s 2024-04-25T10:19:00.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:00.597 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:00.597 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:00.597 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:19:00.597 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:00.598 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:00.598 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:00.598 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:00.672 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.672 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.672 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179507 s, 58.4 MB/s 2024-04-25T10:19:00.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176362 s, 59.5 MB/s 2024-04-25T10:19:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170229 s, 61.6 MB/s 2024-04-25T10:19:00.728 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.728 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.728 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156547 s, 67.0 MB/s 2024-04-25T10:19:00.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.750 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201418 s, 52.1 MB/s 2024-04-25T10:19:00.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.766 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148844 s, 70.4 MB/s 2024-04-25T10:19:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157703 s, 66.5 MB/s 2024-04-25T10:19:00.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143376 s, 73.1 MB/s 2024-04-25T10:19:00.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150953 s, 69.5 MB/s 2024-04-25T10:19:00.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200258 s, 52.4 MB/s 2024-04-25T10:19:00.859 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.859 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.859 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184688 s, 56.8 MB/s 2024-04-25T10:19:00.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.877 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01583 s, 66.2 MB/s 2024-04-25T10:19:00.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199214 s, 52.6 MB/s 2024-04-25T10:19:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170208 s, 61.6 MB/s 2024-04-25T10:19:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200793 s, 52.2 MB/s 2024-04-25T10:19:00.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.958 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175706 s, 59.7 MB/s 2024-04-25T10:19:00.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.974 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014171 s, 74.0 MB/s 2024-04-25T10:19:00.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:00.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:00.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204296 s, 51.3 MB/s 2024-04-25T10:19:01.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.016 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181989 s, 57.6 MB/s 2024-04-25T10:19:01.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.037 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018427 s, 56.9 MB/s 2024-04-25T10:19:01.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.056 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175492 s, 59.8 MB/s 2024-04-25T10:19:01.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.078 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205373 s, 51.1 MB/s 2024-04-25T10:19:01.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161644 s, 64.9 MB/s 2024-04-25T10:19:01.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206336 s, 50.8 MB/s 2024-04-25T10:19:01.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.141 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021108 s, 49.7 MB/s 2024-04-25T10:19:01.160 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.160 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.161 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172185 s, 60.9 MB/s 2024-04-25T10:19:01.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.177 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151794 s, 69.1 MB/s 2024-04-25T10:19:01.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.197 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173435 s, 60.5 MB/s 2024-04-25T10:19:01.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.213 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148606 s, 70.6 MB/s 2024-04-25T10:19:01.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.233 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175354 s, 59.8 MB/s 2024-04-25T10:19:01.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.251 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154307 s, 68.0 MB/s 2024-04-25T10:19:01.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.267 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150691 s, 69.6 MB/s 2024-04-25T10:19:01.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:01.268 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:01.268 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:01.268 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:19:01.268 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:01.269 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:01.269 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:01.269 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:01.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.337 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.337 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147722 s, 71.0 MB/s 2024-04-25T10:19:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.355 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162707 s, 64.4 MB/s 2024-04-25T10:19:01.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151164 s, 69.4 MB/s 2024-04-25T10:19:01.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.391 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178143 s, 58.9 MB/s 2024-04-25T10:19:01.408 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.408 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.408 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146223 s, 71.7 MB/s 2024-04-25T10:19:01.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.428 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185184 s, 56.6 MB/s 2024-04-25T10:19:01.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.444 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01413 s, 74.2 MB/s 2024-04-25T10:19:01.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.463 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173491 s, 60.4 MB/s 2024-04-25T10:19:01.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.480 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150223 s, 69.8 MB/s 2024-04-25T10:19:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169442 s, 61.9 MB/s 2024-04-25T10:19:01.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.514 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140315 s, 74.7 MB/s 2024-04-25T10:19:01.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.535 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019041 s, 55.1 MB/s 2024-04-25T10:19:01.551 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.551 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.552 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146254 s, 71.7 MB/s 2024-04-25T10:19:01.571 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.571 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.571 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017489 s, 60.0 MB/s 2024-04-25T10:19:01.590 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.590 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.590 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172329 s, 60.8 MB/s 2024-04-25T10:19:01.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.613 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211053 s, 49.7 MB/s 2024-04-25T10:19:01.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.630 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148598 s, 70.6 MB/s 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014883 s, 70.5 MB/s 2024-04-25T10:19:01.667 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.668 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194859 s, 53.8 MB/s 2024-04-25T10:19:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.684 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144234 s, 72.7 MB/s 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020576 s, 51.0 MB/s 2024-04-25T10:19:01.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178198 s, 58.8 MB/s 2024-04-25T10:19:01.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.749 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214497 s, 48.9 MB/s 2024-04-25T10:19:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154548 s, 67.8 MB/s 2024-04-25T10:19:01.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.784 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154404 s, 67.9 MB/s 2024-04-25T10:19:01.806 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.806 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.806 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207005 s, 50.7 MB/s 2024-04-25T10:19:01.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.822 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014393 s, 72.9 MB/s 2024-04-25T10:19:01.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.836 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012371 s, 84.8 MB/s 2024-04-25T10:19:01.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146337 s, 71.7 MB/s 2024-04-25T10:19:01.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.870 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149794 s, 70.0 MB/s 2024-04-25T10:19:01.886 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.886 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.886 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014928 s, 70.2 MB/s 2024-04-25T10:19:01.908 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.909 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205882 s, 50.9 MB/s 2024-04-25T10:19:01.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:01.910 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:01.910 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:01.910 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:19:01.910 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:01.911 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:01.911 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:01.911 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:01.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.983 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.983 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167807 s, 62.5 MB/s 2024-04-25T10:19:01.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:01.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:01.999 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146895 s, 71.4 MB/s 2024-04-25T10:19:02.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.018 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169821 s, 61.7 MB/s 2024-04-25T10:19:02.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.038 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187237 s, 56.0 MB/s 2024-04-25T10:19:02.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.056 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154759 s, 67.8 MB/s 2024-04-25T10:19:02.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165977 s, 63.2 MB/s 2024-04-25T10:19:02.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.092 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158984 s, 66.0 MB/s 2024-04-25T10:19:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013961 s, 75.1 MB/s 2024-04-25T10:19:02.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.125 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156733 s, 66.9 MB/s 2024-04-25T10:19:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229352 s, 45.7 MB/s 2024-04-25T10:19:02.171 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.172 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199821 s, 52.5 MB/s 2024-04-25T10:19:02.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191568 s, 54.7 MB/s 2024-04-25T10:19:02.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.215 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212058 s, 49.4 MB/s 2024-04-25T10:19:02.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172819 s, 60.7 MB/s 2024-04-25T10:19:02.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.251 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149747 s, 70.0 MB/s 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150584 s, 69.6 MB/s 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173151 s, 60.6 MB/s 2024-04-25T10:19:02.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183469 s, 57.2 MB/s 2024-04-25T10:19:02.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.325 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160643 s, 65.3 MB/s 2024-04-25T10:19:02.344 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.344 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.344 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016913 s, 62.0 MB/s 2024-04-25T10:19:02.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.361 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161345 s, 65.0 MB/s 2024-04-25T10:19:02.378 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.378 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.378 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149048 s, 70.4 MB/s 2024-04-25T10:19:02.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.392 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120074 s, 87.3 MB/s 2024-04-25T10:19:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154435 s, 67.9 MB/s 2024-04-25T10:19:02.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.429 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180146 s, 58.2 MB/s 2024-04-25T10:19:02.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.445 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148625 s, 70.6 MB/s 2024-04-25T10:19:02.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.465 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173179 s, 60.5 MB/s 2024-04-25T10:19:02.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.484 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01791 s, 58.5 MB/s 2024-04-25T10:19:02.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.504 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177696 s, 59.0 MB/s 2024-04-25T10:19:02.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201074 s, 52.1 MB/s 2024-04-25T10:19:02.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.542 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145161 s, 72.2 MB/s 2024-04-25T10:19:02.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.561 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172786 s, 60.7 MB/s 2024-04-25T10:19:02.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:02.563 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:02.563 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:02.563 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:19:02.563 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:02.563 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:02.563 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:02.563 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:02.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206825 s, 50.7 MB/s 2024-04-25T10:19:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190351 s, 55.1 MB/s 2024-04-25T10:19:02.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.682 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170588 s, 61.5 MB/s 2024-04-25T10:19:02.701 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.701 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.701 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173964 s, 60.3 MB/s 2024-04-25T10:19:02.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.720 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172129 s, 60.9 MB/s 2024-04-25T10:19:02.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.740 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179434 s, 58.4 MB/s 2024-04-25T10:19:02.756 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.757 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149526 s, 70.1 MB/s 2024-04-25T10:19:02.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179293 s, 58.5 MB/s 2024-04-25T10:19:02.793 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.793 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.793 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143279 s, 73.2 MB/s 2024-04-25T10:19:02.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.812 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173124 s, 60.6 MB/s 2024-04-25T10:19:02.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145062 s, 72.3 MB/s 2024-04-25T10:19:02.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.846 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.847 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165208 s, 63.5 MB/s 2024-04-25T10:19:02.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.866 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.866 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176329 s, 59.5 MB/s 2024-04-25T10:19:02.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.884 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164772 s, 63.6 MB/s 2024-04-25T10:19:02.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.904 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186029 s, 56.4 MB/s 2024-04-25T10:19:02.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.921 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152882 s, 68.6 MB/s 2024-04-25T10:19:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170043 s, 61.7 MB/s 2024-04-25T10:19:02.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.956 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142862 s, 73.4 MB/s 2024-04-25T10:19:02.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015184 s, 69.1 MB/s 2024-04-25T10:19:02.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:02.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:02.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203978 s, 51.4 MB/s 2024-04-25T10:19:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168155 s, 62.4 MB/s 2024-04-25T10:19:03.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.031 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154458 s, 67.9 MB/s 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142478 s, 73.6 MB/s 2024-04-25T10:19:03.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.069 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205294 s, 51.1 MB/s 2024-04-25T10:19:03.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156093 s, 67.2 MB/s 2024-04-25T10:19:03.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.105 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015462 s, 67.8 MB/s 2024-04-25T10:19:03.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.126 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193399 s, 54.2 MB/s 2024-04-25T10:19:03.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.143 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015847 s, 66.2 MB/s 2024-04-25T10:19:03.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225136 s, 46.6 MB/s 2024-04-25T10:19:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.188 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017873 s, 58.7 MB/s 2024-04-25T10:19:03.204 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.204 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.204 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143832 s, 72.9 MB/s 2024-04-25T10:19:03.220 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.221 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151862 s, 69.0 MB/s 2024-04-25T10:19:03.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:03.222 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:03.222 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:03.222 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:19:03.222 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:03.222 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:03.222 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:03.222 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:03.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.298 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199389 s, 52.6 MB/s 2024-04-25T10:19:03.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.315 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015338 s, 68.4 MB/s 2024-04-25T10:19:03.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.332 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146139 s, 71.8 MB/s 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014126 s, 74.2 MB/s 2024-04-25T10:19:03.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.365 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150887 s, 69.5 MB/s 2024-04-25T10:19:03.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.381 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147315 s, 71.2 MB/s 2024-04-25T10:19:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.399 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155142 s, 67.6 MB/s 2024-04-25T10:19:03.421 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.421 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.421 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205832 s, 50.9 MB/s 2024-04-25T10:19:03.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.437 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147915 s, 70.9 MB/s 2024-04-25T10:19:03.455 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.455 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.455 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159507 s, 65.7 MB/s 2024-04-25T10:19:03.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.475 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.475 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177138 s, 59.2 MB/s 2024-04-25T10:19:03.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.494 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176465 s, 59.4 MB/s 2024-04-25T10:19:03.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.510 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145158 s, 72.2 MB/s 2024-04-25T10:19:03.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176354 s, 59.5 MB/s 2024-04-25T10:19:03.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145459 s, 72.1 MB/s 2024-04-25T10:19:03.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175105 s, 59.9 MB/s 2024-04-25T10:19:03.588 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.588 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.588 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209519 s, 50.0 MB/s 2024-04-25T10:19:03.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.609 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188496 s, 55.6 MB/s 2024-04-25T10:19:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015092 s, 69.5 MB/s 2024-04-25T10:19:03.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170544 s, 61.5 MB/s 2024-04-25T10:19:03.660 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.660 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.661 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01442 s, 72.7 MB/s 2024-04-25T10:19:03.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.681 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018518 s, 56.6 MB/s 2024-04-25T10:19:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150592 s, 69.6 MB/s 2024-04-25T10:19:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189514 s, 55.3 MB/s 2024-04-25T10:19:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147153 s, 71.3 MB/s 2024-04-25T10:19:03.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.753 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159602 s, 65.7 MB/s 2024-04-25T10:19:03.775 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.775 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.775 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208022 s, 50.4 MB/s 2024-04-25T10:19:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.792 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142348 s, 73.7 MB/s 2024-04-25T10:19:03.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.812 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185917 s, 56.4 MB/s 2024-04-25T10:19:03.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140881 s, 74.4 MB/s 2024-04-25T10:19:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150104 s, 69.9 MB/s 2024-04-25T10:19:03.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147014 s, 71.3 MB/s 2024-04-25T10:19:03.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:03.864 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:03.864 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:03.864 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:19:03.864 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:03.865 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:03.865 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:03.865 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:03.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.936 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173534 s, 60.4 MB/s 2024-04-25T10:19:03.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.955 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.955 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171062 s, 61.3 MB/s 2024-04-25T10:19:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180033 s, 58.2 MB/s 2024-04-25T10:19:03.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:03.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:03.991 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146534 s, 71.6 MB/s 2024-04-25T10:19:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156909 s, 66.8 MB/s 2024-04-25T10:19:04.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.027 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170084 s, 61.7 MB/s 2024-04-25T10:19:04.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.047 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018379 s, 57.1 MB/s 2024-04-25T10:19:04.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.070 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208377 s, 50.3 MB/s 2024-04-25T10:19:04.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.088 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169033 s, 62.0 MB/s 2024-04-25T10:19:04.107 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.107 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.107 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168185 s, 62.3 MB/s 2024-04-25T10:19:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.126 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168684 s, 62.2 MB/s 2024-04-25T10:19:04.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.148 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204106 s, 51.4 MB/s 2024-04-25T10:19:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.166 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166487 s, 63.0 MB/s 2024-04-25T10:19:04.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.185 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166483 s, 63.0 MB/s 2024-04-25T10:19:04.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.207 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206429 s, 50.8 MB/s 2024-04-25T10:19:04.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.231 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216438 s, 48.4 MB/s 2024-04-25T10:19:04.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195976 s, 53.5 MB/s 2024-04-25T10:19:04.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.270 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159695 s, 65.7 MB/s 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154636 s, 67.8 MB/s 2024-04-25T10:19:04.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174546 s, 60.1 MB/s 2024-04-25T10:19:04.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207777 s, 50.5 MB/s 2024-04-25T10:19:04.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.352 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208177 s, 50.4 MB/s 2024-04-25T10:19:04.370 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169016 s, 62.0 MB/s 2024-04-25T10:19:04.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204769 s, 51.2 MB/s 2024-04-25T10:19:04.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.410 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148675 s, 70.5 MB/s 2024-04-25T10:19:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155415 s, 67.5 MB/s 2024-04-25T10:19:04.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.444 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149354 s, 70.2 MB/s 2024-04-25T10:19:04.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.461 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149629 s, 70.1 MB/s 2024-04-25T10:19:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146346 s, 71.7 MB/s 2024-04-25T10:19:04.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.493 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014281 s, 73.4 MB/s 2024-04-25T10:19:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.514 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172827 s, 60.7 MB/s 2024-04-25T10:19:04.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165021 s, 63.5 MB/s 2024-04-25T10:19:04.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:04.532 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:04.532 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:04.532 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:19:04.532 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:04.532 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:04.532 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:04.532 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:04.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232451 s, 45.1 MB/s 2024-04-25T10:19:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192692 s, 54.4 MB/s 2024-04-25T10:19:04.650 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.651 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181684 s, 57.7 MB/s 2024-04-25T10:19:04.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.669 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169176 s, 62.0 MB/s 2024-04-25T10:19:04.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.691 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198616 s, 52.8 MB/s 2024-04-25T10:19:04.711 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.711 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.711 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182028 s, 57.6 MB/s 2024-04-25T10:19:04.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124419 s, 84.3 MB/s 2024-04-25T10:19:04.747 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.748 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.748 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209178 s, 50.1 MB/s 2024-04-25T10:19:04.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.766 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171458 s, 61.2 MB/s 2024-04-25T10:19:04.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.780 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113742 s, 92.2 MB/s 2024-04-25T10:19:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147044 s, 71.3 MB/s 2024-04-25T10:19:04.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.812 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014383 s, 72.9 MB/s 2024-04-25T10:19:04.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201126 s, 52.1 MB/s 2024-04-25T10:19:04.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.851 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149774 s, 70.0 MB/s 2024-04-25T10:19:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152411 s, 68.8 MB/s 2024-04-25T10:19:04.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.884 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143924 s, 72.9 MB/s 2024-04-25T10:19:04.902 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.902 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.902 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167775 s, 62.5 MB/s 2024-04-25T10:19:04.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.922 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163971 s, 63.9 MB/s 2024-04-25T10:19:04.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.937 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149716 s, 70.0 MB/s 2024-04-25T10:19:04.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.953 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143096 s, 73.3 MB/s 2024-04-25T10:19:04.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.979 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235685 s, 44.5 MB/s 2024-04-25T10:19:04.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:04.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:04.992 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119525 s, 87.7 MB/s 2024-04-25T10:19:05.012 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.012 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.012 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182078 s, 57.6 MB/s 2024-04-25T10:19:05.028 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.028 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.028 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014489 s, 72.4 MB/s 2024-04-25T10:19:05.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.048 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174049 s, 60.2 MB/s 2024-04-25T10:19:05.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.064 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147694 s, 71.0 MB/s 2024-04-25T10:19:05.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.081 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149912 s, 69.9 MB/s 2024-04-25T10:19:05.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.101 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176771 s, 59.3 MB/s 2024-04-25T10:19:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172317 s, 60.9 MB/s 2024-04-25T10:19:05.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248229 s, 42.2 MB/s 2024-04-25T10:19:05.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.164 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160161 s, 65.5 MB/s 2024-04-25T10:19:05.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.186 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201476 s, 52.0 MB/s 2024-04-25T10:19:05.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:05.188 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:05.188 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:05.188 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:19:05.188 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:05.188 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:05.188 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:05.188 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:05.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.260 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177128 s, 59.2 MB/s 2024-04-25T10:19:05.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.281 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196857 s, 53.3 MB/s 2024-04-25T10:19:05.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.300 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172315 s, 60.9 MB/s 2024-04-25T10:19:05.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.321 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192831 s, 54.4 MB/s 2024-04-25T10:19:05.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166784 s, 62.9 MB/s 2024-04-25T10:19:05.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.355 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128929 s, 81.3 MB/s 2024-04-25T10:19:05.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147028 s, 71.3 MB/s 2024-04-25T10:19:05.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.391 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179139 s, 58.5 MB/s 2024-04-25T10:19:05.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.410 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170792 s, 61.4 MB/s 2024-04-25T10:19:05.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149043 s, 70.4 MB/s 2024-04-25T10:19:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189126 s, 55.4 MB/s 2024-04-25T10:19:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142811 s, 73.4 MB/s 2024-04-25T10:19:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.480 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150497 s, 69.7 MB/s 2024-04-25T10:19:05.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.497 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014932 s, 70.2 MB/s 2024-04-25T10:19:05.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.515 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158177 s, 66.3 MB/s 2024-04-25T10:19:05.531 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150105 s, 69.9 MB/s 2024-04-25T10:19:05.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.548 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146227 s, 71.7 MB/s 2024-04-25T10:19:05.564 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.564 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.564 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145806 s, 71.9 MB/s 2024-04-25T10:19:05.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.581 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149631 s, 70.1 MB/s 2024-04-25T10:19:05.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.602 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187471 s, 55.9 MB/s 2024-04-25T10:19:05.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.618 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149647 s, 70.1 MB/s 2024-04-25T10:19:05.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.646 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254749 s, 41.2 MB/s 2024-04-25T10:19:05.661 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141604 s, 74.0 MB/s 2024-04-25T10:19:05.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.676 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125065 s, 83.8 MB/s 2024-04-25T10:19:05.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.696 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018308 s, 57.3 MB/s 2024-04-25T10:19:05.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.721 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233962 s, 44.8 MB/s 2024-04-25T10:19:05.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.741 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180782 s, 58.0 MB/s 2024-04-25T10:19:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.764 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210695 s, 49.8 MB/s 2024-04-25T10:19:05.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.783 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172339 s, 60.8 MB/s 2024-04-25T10:19:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014581 s, 71.9 MB/s 2024-04-25T10:19:05.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.818 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016925 s, 62.0 MB/s 2024-04-25T10:19:05.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.836 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159596 s, 65.7 MB/s 2024-04-25T10:19:05.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:05.837 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:05.837 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:05.837 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:19:05.838 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:05.838 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:05.838 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:05.838 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:05.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.914 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165553 s, 63.3 MB/s 2024-04-25T10:19:05.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.932 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154317 s, 67.9 MB/s 2024-04-25T10:19:05.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.950 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165637 s, 63.3 MB/s 2024-04-25T10:19:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141643 s, 74.0 MB/s 2024-04-25T10:19:05.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:05.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:05.988 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202478 s, 51.8 MB/s 2024-04-25T10:19:06.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.004 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146501 s, 71.6 MB/s 2024-04-25T10:19:06.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.023 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164195 s, 63.9 MB/s 2024-04-25T10:19:06.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.044 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199351 s, 52.6 MB/s 2024-04-25T10:19:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164066 s, 63.9 MB/s 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016959 s, 61.8 MB/s 2024-04-25T10:19:06.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.100 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017655 s, 59.4 MB/s 2024-04-25T10:19:06.118 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.118 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.118 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158227 s, 66.3 MB/s 2024-04-25T10:19:06.137 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.137 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.137 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176031 s, 59.6 MB/s 2024-04-25T10:19:06.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.161 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216165 s, 48.5 MB/s 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167379 s, 62.6 MB/s 2024-04-25T10:19:06.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.201 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186702 s, 56.2 MB/s 2024-04-25T10:19:06.221 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.221 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184122 s, 56.9 MB/s 2024-04-25T10:19:06.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.241 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171414 s, 61.2 MB/s 2024-04-25T10:19:06.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.258 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154483 s, 67.9 MB/s 2024-04-25T10:19:06.276 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.276 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.276 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165477 s, 63.4 MB/s 2024-04-25T10:19:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144134 s, 72.7 MB/s 2024-04-25T10:19:06.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151005 s, 69.4 MB/s 2024-04-25T10:19:06.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.336 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242649 s, 43.2 MB/s 2024-04-25T10:19:06.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.355 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169987 s, 61.7 MB/s 2024-04-25T10:19:06.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141169 s, 74.3 MB/s 2024-04-25T10:19:06.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.387 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148537 s, 70.6 MB/s 2024-04-25T10:19:06.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.409 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200141 s, 52.4 MB/s 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160776 s, 65.2 MB/s 2024-04-25T10:19:06.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.446 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173698 s, 60.4 MB/s 2024-04-25T10:19:06.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.463 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154929 s, 67.7 MB/s 2024-04-25T10:19:06.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.486 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210286 s, 49.9 MB/s 2024-04-25T10:19:06.505 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.506 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174062 s, 60.2 MB/s 2024-04-25T10:19:06.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:06.507 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:06.507 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:06.507 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:19:06.507 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:06.508 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:06.508 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:06.508 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:06.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.574 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117033 s, 89.6 MB/s 2024-04-25T10:19:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161753 s, 64.8 MB/s 2024-04-25T10:19:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167039 s, 62.8 MB/s 2024-04-25T10:19:06.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.627 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153523 s, 68.3 MB/s 2024-04-25T10:19:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165548 s, 63.3 MB/s 2024-04-25T10:19:06.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184102 s, 57.0 MB/s 2024-04-25T10:19:06.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.688 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201817 s, 52.0 MB/s 2024-04-25T10:19:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.707 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173348 s, 60.5 MB/s 2024-04-25T10:19:06.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.732 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229949 s, 45.6 MB/s 2024-04-25T10:19:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204871 s, 51.2 MB/s 2024-04-25T10:19:06.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.773 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171604 s, 61.1 MB/s 2024-04-25T10:19:06.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.795 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199017 s, 52.7 MB/s 2024-04-25T10:19:06.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196786 s, 53.3 MB/s 2024-04-25T10:19:06.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.833 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157239 s, 66.7 MB/s 2024-04-25T10:19:06.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.852 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166537 s, 63.0 MB/s 2024-04-25T10:19:06.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.872 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181607 s, 57.7 MB/s 2024-04-25T10:19:06.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.891 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172872 s, 60.7 MB/s 2024-04-25T10:19:06.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.913 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201319 s, 52.1 MB/s 2024-04-25T10:19:06.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192196 s, 54.6 MB/s 2024-04-25T10:19:06.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.954 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181288 s, 57.8 MB/s 2024-04-25T10:19:06.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176388 s, 59.4 MB/s 2024-04-25T10:19:06.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:06.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:06.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201961 s, 51.9 MB/s 2024-04-25T10:19:07.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.014 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170918 s, 61.3 MB/s 2024-04-25T10:19:07.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.038 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222277 s, 47.2 MB/s 2024-04-25T10:19:07.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.057 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168059 s, 62.4 MB/s 2024-04-25T10:19:07.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203149 s, 51.6 MB/s 2024-04-25T10:19:07.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.098 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168453 s, 62.2 MB/s 2024-04-25T10:19:07.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156386 s, 67.1 MB/s 2024-04-25T10:19:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180006 s, 58.3 MB/s 2024-04-25T10:19:07.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.155 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185105 s, 56.6 MB/s 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166379 s, 63.0 MB/s 2024-04-25T10:19:07.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:07.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:07.197 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210506 s, 49.8 MB/s 2024-04-25T10:19:07.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.198 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:19:07.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.376+0000 7f57e3200640 1 Processor -- start 2024-04-25T10:19:07.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.376+0000 7f57e3200640 1 -- start start 2024-04-25T10:19:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57e3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f57dc0cf040 0x7f57dc0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57e3200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57dc15bc00 0x7f57dc0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57e3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0713a0 0x7f57dc071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57e3200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc072260 con 0x7f57dc15bc00 2024-04-25T10:19:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57e3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc0b3700 con 0x7f57dc0cf040 2024-04-25T10:19:07.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57e3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc071d40 con 0x7f57dc0713a0 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57dc15bc00 0x7f57dc0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57dc15bc00 0x7f57dc0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41900/0 (socket says 172.21.15.87:41900) 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 -- 172.21.15.87:0/118804990 learned_addr learned my addr 172.21.15.87:0/118804990 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 -- 172.21.15.87:0/118804990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0713a0 msgr2=0x7f57dc071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 --2- 172.21.15.87:0/118804990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0713a0 0x7f57dc071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 -- 172.21.15.87:0/118804990 >> v2:172.21.15.169:3300/0 conn(0x7f57dc0cf040 msgr2=0x7f57dc0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 --2- 172.21.15.87:0/118804990 >> v2:172.21.15.169:3300/0 conn(0x7f57dc0cf040 0x7f57dc0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.380+0000 7f57d3400640 1 -- 172.21.15.87:0/118804990 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc071f80 con 0x7f57dc15bc00 2024-04-25T10:19:07.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57d3400640 1 --2- 172.21.15.87:0/118804990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc15bc00 0x7f57dc0cff20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0d05d0 tx=0x7f57c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0c2ab9616b719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57db400640 1 -- 172.21.15.87:0/118804990 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40b1070 con 0x7f57dc15bc00 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57db400640 1 -- 172.21.15.87:0/118804990 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c40ac630 con 0x7f57dc15bc00 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57db400640 1 -- 172.21.15.87:0/118804990 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40a8050 con 0x7f57dc15bc00 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- 172.21.15.87:0/118804990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc15bc00 msgr2=0x7f57dc0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 --2- 172.21.15.87:0/118804990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc15bc00 0x7f57dc0cff20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0d05d0 tx=0x7f57c401c520 comp rx=0 tx=0).stop 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- 172.21.15.87:0/118804990 shutdown_connections 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 --2- 172.21.15.87:0/118804990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc15bc00 0x7f57dc0cff20 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 --2- 172.21.15.87:0/118804990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0713a0 0x7f57dc071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 --2- 172.21.15.87:0/118804990 >> v2:172.21.15.169:3300/0 conn(0x7f57dc0cf040 0x7f57dc0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- 172.21.15.87:0/118804990 >> 172.21.15.87:0/118804990 conn(0x7f57dc066b70 msgr2=0x7f57dc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- 172.21.15.87:0/118804990 shutdown_connections 2024-04-25T10:19:07.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- 172.21.15.87:0/118804990 wait complete. 2024-04-25T10:19:07.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 Processor -- start 2024-04-25T10:19:07.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- start start 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0713a0 0x7f57dc1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0cf040 0x7f57dc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57dc15bc00 0x7f57dc1f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0cf5a0 con 0x7f57dc0713a0 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0cf420 con 0x7f57dc15bc00 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57e3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0aec90 con 0x7f57dc0cf040 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0713a0 0x7f57dc1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0713a0 0x7f57dc1536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41904/0 (socket says 172.21.15.87:41904) 2024-04-25T10:19:07.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.384+0000 7f57dbe00640 1 -- 172.21.15.87:0/3260256532 learned_addr learned my addr 172.21.15.87:0/3260256532 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57e1200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57dc15bc00 0x7f57dc1f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57d3400640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0cf040 0x7f57dc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57dbe00640 1 -- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0cf040 msgr2=0x7f57dc0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57dbe00640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0cf040 0x7f57dc0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57dbe00640 1 -- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57dc15bc00 msgr2=0x7f57dc1f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57dbe00640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57dc15bc00 0x7f57dc1f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57dbe00640 1 -- 172.21.15.87:0/3260256532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc154020 con 0x7f57dc0713a0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57e1200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57dc15bc00 0x7f57dc1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57dbe00640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0713a0 0x7f57dc1536d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00ba80 tx=0x7f57cc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57d8c00640 1 -- 172.21.15.87:0/3260256532 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc019070 con 0x7f57dc0713a0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57d8c00640 1 -- 172.21.15.87:0/3260256532 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc00cc20 con 0x7f57dc0713a0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57d8c00640 1 -- 172.21.15.87:0/3260256532 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc014440 con 0x7f57dc0713a0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc154300 con 0x7f57dc0713a0 2024-04-25T10:19:07.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc1548c0 con 0x7f57dc0713a0 2024-04-25T10:19:07.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57d8c00640 1 -- 172.21.15.87:0/3260256532 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57cc01c070 con 0x7f57dc0713a0 2024-04-25T10:19:07.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57d8c00640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57b8066f90 0x7f57b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.388+0000 7f57d8c00640 1 -- 172.21.15.87:0/3260256532 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f57cc084e50 con 0x7f57dc0713a0 2024-04-25T10:19:07.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57d3400640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57b8066f90 0x7f57b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 Processor -- start 2024-04-25T10:19:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- start start 2024-04-25T10:19:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57d3400640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57b8066f90 0x7f57b8069450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f57c40ae000 tx=0x7f57c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3e817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e37fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3e38900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c85d50 con 0x55aee3e813f0 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c0de40 con 0x55aee3e81d10 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c81df0 con 0x55aee3e38520 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3e817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3e817d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41910/0 (socket says 172.21.15.87:41910) 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 -- 172.21.15.87:0/1868656399 learned_addr learned my addr 172.21.15.87:0/1868656399 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57d3400640 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e37fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e1200640 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3e38900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 -- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 msgr2=0x55aee3e38900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3e38900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 -- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 msgr2=0x55aee3e37fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e37fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 -- 172.21.15.87:0/1868656399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aee3c22d90 con 0x55aee3e813f0 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57dbe00640 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3e817d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f57cc051b00 tx=0x7f57cc0556a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1980ca5cc6c871b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57d0c00640 1 -- 172.21.15.87:0/1868656399 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc06d170 con 0x55aee3e813f0 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57d0c00640 1 -- 172.21.15.87:0/1868656399 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0539a0 con 0x55aee3e813f0 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57d0c00640 1 -- 172.21.15.87:0/1868656399 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc05d920 con 0x55aee3e813f0 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- 172.21.15.87:0/1868656399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 msgr2=0x55aee3e817d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3e817d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f57cc051b00 tx=0x7f57cc0556a0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- 172.21.15.87:0/1868656399 shutdown_connections 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3e38900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e37fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 --2- 172.21.15.87:0/1868656399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3e817d0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- 172.21.15.87:0/1868656399 >> 172.21.15.87:0/1868656399 conn(0x55aee3e344d0 msgr2=0x55aee3e368c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- 172.21.15.87:0/1868656399 shutdown_connections 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.392+0000 7f57e6623000 1 -- 172.21.15.87:0/1868656399 wait complete. 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 -- start start 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 Processor -- start 2024-04-25T10:19:07.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e813f0 0x55aee3e734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c2b610 con 0x55aee3e38520 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c409a0 con 0x55aee3e81d10 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c8a380 con 0x55aee3e813f0 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 10 client.0 _ll_register_callbacks cb 0x55aee3d3a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e3e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41916/0 (socket says 172.21.15.87:41916) 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 -- 172.21.15.87:0/2419241322 learned_addr learned my addr 172.21.15.87:0/2419241322 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e1200640 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57d3400640 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e813f0 0x55aee3e734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 -- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e813f0 msgr2=0x55aee3e734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e813f0 0x55aee3e734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 -- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 msgr2=0x55aee3e73a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e73a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 -- 172.21.15.87:0/2419241322 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aee3c80d40 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57dbe00640 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e3e020 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00ba50 tx=0x7f57cc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d9bc9ef2784730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57c9600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57c9600640 1 -- 172.21.15.87:0/2419241322 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc072060 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57c9600640 1 -- 172.21.15.87:0/2419241322 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f57cc05db20 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57cc0539a0 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57c9600640 1 -- 172.21.15.87:0/2419241322 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57cc06d230 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 10 client.7620 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55aee3d08f00 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57c9600640 1 -- 172.21.15.87:0/2419241322 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0539a0 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57c9600640 1 -- 172.21.15.87:0/2419241322 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57cc054af0 con 0x55aee3e38520 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 10 client.7620 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:07.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 10 client.7620 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 20 client.7620 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e8c7c0 0x55aee3e8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 4 client.7620 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aee3c21a20 con 0x55aee3e8c7c0 2024-04-25T10:19:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57d3400640 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e8c7c0 0x55aee3e8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57d3400640 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e8c7c0 0x55aee3e8cba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233f5446186ed5b0 server_cookie=48d74adab98ac3f9 in_seq=0 out_seq=0 2024-04-25T10:19:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.396+0000 7f57c9600640 10 client.7620 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57c9600640 1 -- 172.21.15.87:0/2419241322 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aee3c21a20 con 0x55aee3e8c7c0 2024-04-25T10:19:07.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57c9600640 10 client.7620 handle_command_reply: tid=0 2024-04-25T10:19:07.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 client.7620 shutdown 2024-04-25T10:19:07.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 msgr2=0x55aee3e3e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e3e020 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00ba50 tx=0x7f57cc004a50 comp rx=0 tx=0).stop 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 shutdown_connections 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e8c7c0 0x55aee3e8cba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e73a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e813f0 0x55aee3e734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 --2- 172.21.15.87:0/2419241322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e3e020 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57c9600640 0 client.7620 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 >> 172.21.15.87:0/2419241322 conn(0x55aee3e344d0 msgr2=0x55aee3e37150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 shutdown_connections 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 1 -- 172.21.15.87:0/2419241322 wait complete. 2024-04-25T10:19:07.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.400+0000 7f57e6623000 20 client.7620 trim_cache size 0 max 16384 2024-04-25T10:19:07.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.412+0000 7f57e6623000 1 Processor -- start 2024-04-25T10:19:07.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.412+0000 7f57e6623000 1 -- start start 2024-04-25T10:19:07.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e9bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e813f0 0x55aee3e378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e81d10 0x55aee3e37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc146550 con 0x55aee3e38520 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc145790 con 0x55aee3e813f0 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0b07f0 con 0x55aee3e81d10 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e9bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e9bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41928/0 (socket says 172.21.15.87:41928) 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 -- 172.21.15.87:0/522414695 learned_addr learned my addr 172.21.15.87:0/522414695 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57d3400640 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e813f0 0x55aee3e378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e1200640 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e81d10 0x55aee3e37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 -- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e81d10 msgr2=0x55aee3e37e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e81d10 0x55aee3e37e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 -- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e813f0 msgr2=0x55aee3e378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e813f0 0x55aee3e378d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 -- 172.21.15.87:0/522414695 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aee3e81b40 con 0x55aee3e38520 2024-04-25T10:19:07.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57dbe00640 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e9bf70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f57cc053950 tx=0x7f57cc07e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472459d5aa7b87c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57cbe00640 1 -- 172.21.15.87:0/522414695 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc073ae0 con 0x55aee3e38520 2024-04-25T10:19:07.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57cbe00640 1 -- 172.21.15.87:0/522414695 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc054af0 con 0x55aee3e38520 2024-04-25T10:19:07.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57cbe00640 1 -- 172.21.15.87:0/522414695 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc05db60 con 0x55aee3e38520 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- 172.21.15.87:0/522414695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 msgr2=0x55aee3e9bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e9bf70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f57cc053950 tx=0x7f57cc07e070 comp rx=0 tx=0).stop 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- 172.21.15.87:0/522414695 shutdown_connections 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e81d10 0x55aee3e37e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e813f0 0x55aee3e378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 --2- 172.21.15.87:0/522414695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e38520 0x55aee3e9bf70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- 172.21.15.87:0/522414695 >> 172.21.15.87:0/522414695 conn(0x55aee3e344d0 msgr2=0x55aee3e3cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- 172.21.15.87:0/522414695 shutdown_connections 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- 172.21.15.87:0/522414695 wait complete. 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 -- start start 2024-04-25T10:19:07.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.416+0000 7f57e6623000 1 Processor -- start 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3ee5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3ee61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e9a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c3cdc0 con 0x55aee3e813f0 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0b0df0 con 0x55aee3e81d10 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aee3c85060 con 0x55aee3e38520 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 10 client.0 _ll_register_callbacks cb 0x7f57cc055470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3ee61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3ee61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41940/0 (socket says 172.21.15.87:41940) 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 -- 172.21.15.87:0/2388057281 learned_addr learned my addr 172.21.15.87:0/2388057281 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e1200640 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e9a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57dbe00640 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3ee5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 -- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 msgr2=0x55aee3ee5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3ee5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 -- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 msgr2=0x55aee3e9a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e9a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 -- 172.21.15.87:0/2388057281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aee3cdf330 con 0x55aee3e813f0 2024-04-25T10:19:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57d3400640 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3ee61a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f57c401ca00 tx=0x7f57c40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7461d905b6a53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57c9600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57c9600640 1 -- 172.21.15.87:0/2388057281 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40b1040 con 0x55aee3e813f0 2024-04-25T10:19:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57c9600640 1 -- 172.21.15.87:0/2388057281 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f57c40b6bf0 con 0x55aee3e813f0 2024-04-25T10:19:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57cc054af0 con 0x55aee3e813f0 2024-04-25T10:19:07.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57c9600640 1 -- 172.21.15.87:0/2388057281 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57c40d8050 con 0x55aee3e813f0 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 10 client.7626 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55aee3c525f0 con 0x55aee3e813f0 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57c9600640 1 -- 172.21.15.87:0/2388057281 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc054af0 con 0x55aee3e813f0 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57c9600640 1 -- 172.21.15.87:0/2388057281 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57c40db020 con 0x55aee3e813f0 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 10 client.7626 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 10 client.7626 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 20 client.7626 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e9b340 0x55aee3e9b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 4 client.7626 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55aee3c93290 con 0x55aee3e9b340 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57dbe00640 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e9b340 0x55aee3e9b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57dbe00640 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e9b340 0x55aee3e9b750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5887649edfd6a80 server_cookie=70dace1396b54824 in_seq=0 out_seq=0 2024-04-25T10:19:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.420+0000 7f57c9600640 10 client.7626 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57c9600640 1 -- 172.21.15.87:0/2388057281 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55aee3c93290 con 0x55aee3e9b340 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57c9600640 10 client.7626 handle_command_reply: tid=0 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 client.7626 shutdown 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 msgr2=0x55aee3ee61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3ee61a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f57c401ca00 tx=0x7f57c40046d0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 shutdown_connections 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x55aee3e9b340 0x55aee3e9b750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aee3e81d10 0x55aee3e9a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aee3e813f0 0x55aee3ee61a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 --2- 172.21.15.87:0/2388057281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aee3e38520 0x55aee3ee5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 >> 172.21.15.87:0/2388057281 conn(0x55aee3e344d0 msgr2=0x55aee3e932e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57c9600640 0 client.7626 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 shutdown_connections 2024-04-25T10:19:07.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 1 -- 172.21.15.87:0/2388057281 wait complete. 2024-04-25T10:19:07.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e6623000 20 client.7626 trim_cache size 0 max 16384 2024-04-25T10:19:07.429 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:19:07.429 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:19:07.429 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "2b3df92c-1c29-4b5d-8fc9-88d045c49ade", 2024-04-25T10:19:07.429 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:19:07.429 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57b8066f90 msgr2=0x7f57b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57b8066f90 0x7f57b8069450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f57c40ae000 tx=0x7f57c4008040 comp rx=0 tx=0).stop 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0713a0 msgr2=0x7f57dc1536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0713a0 0x7f57dc1536d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00ba80 tx=0x7f57cc00bf40 comp rx=0 tx=0).stop 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 shutdown_connections 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f57b8066f90 0x7f57b8069450 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57dc15bc00 0x7f57dc1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57dc0cf040 0x7f57dc0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 --2- 172.21.15.87:0/3260256532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0713a0 0x7f57dc1536d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 >> 172.21.15.87:0/3260256532 conn(0x7f57dc066b70 msgr2=0x7f57dc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 shutdown_connections 2024-04-25T10:19:07.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.424+0000 7f57e3200640 1 -- 172.21.15.87:0/3260256532 wait complete. 2024-04-25T10:19:07.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.456 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T10:19:07.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322be00640 1 Processor -- start 2024-04-25T10:19:07.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322be00640 1 -- start start 2024-04-25T10:19:07.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f322c0cd530 0x7f322c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f322c0cf490 0x7f322c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0cc490 con 0x7f322c15b950 2024-04-25T10:19:07.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0cd0a0 con 0x7f322c0cd530 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0cc770 con 0x7f322c0cf490 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f322c0cd530 0x7f322c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f322c15b950 0x7f322c0ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60548/0 (socket says 172.21.15.87:60548) 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322aa00640 1 -- 172.21.15.87:0/3515761235 learned_addr learned my addr 172.21.15.87:0/3515761235 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322a000640 1 -- 172.21.15.87:0/3515761235 >> v2:172.21.15.169:3300/0 conn(0x7f322c0cf490 msgr2=0x7f322c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322b400640 1 --2- 172.21.15.87:0/3515761235 >> v2:172.21.15.169:3300/0 conn(0x7f322c0cf490 0x7f322c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322a000640 1 --2- 172.21.15.87:0/3515761235 >> v2:172.21.15.169:3300/0 conn(0x7f322c0cf490 0x7f322c0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322a000640 1 -- 172.21.15.87:0/3515761235 >> v1:172.21.15.87:6789/0 conn(0x7f322c15b950 legacy=0x7f322c0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322a000640 1 -- 172.21.15.87:0/3515761235 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0cca50 con 0x7f322c0cd530 2024-04-25T10:19:07.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.624+0000 7f322b400640 1 --2- 172.21.15.87:0/3515761235 >> v2:172.21.15.169:3300/0 conn(0x7f322c0cf490 0x7f322c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:07.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322a000640 1 --2- 172.21.15.87:0/3515761235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cd530 0x7f322c0cf9e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f322c070b50 tx=0x7f321c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10159cabb5a4cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f3229600640 1 -- 172.21.15.87:0/3515761235 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0a7070 con 0x7f322c0cd530 2024-04-25T10:19:07.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f3229600640 1 -- 172.21.15.87:0/3515761235 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0b0070 con 0x7f322c0cd530 2024-04-25T10:19:07.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f3229600640 1 -- 172.21.15.87:0/3515761235 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0ab890 con 0x7f322c0cd530 2024-04-25T10:19:07.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 -- 172.21.15.87:0/3515761235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cd530 msgr2=0x7f322c0cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 --2- 172.21.15.87:0/3515761235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cd530 0x7f322c0cf9e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f322c070b50 tx=0x7f321c01c450 comp rx=0 tx=0).stop 2024-04-25T10:19:07.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 -- 172.21.15.87:0/3515761235 shutdown_connections 2024-04-25T10:19:07.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 --2- 172.21.15.87:0/3515761235 >> v2:172.21.15.169:3300/0 conn(0x7f322c0cf490 0x7f322c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 --2- 172.21.15.87:0/3515761235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cd530 0x7f322c0cf9e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 -- 172.21.15.87:0/3515761235 >> 172.21.15.87:0/3515761235 conn(0x7f322c0d17e0 msgr2=0x7f322c0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 -- 172.21.15.87:0/3515761235 shutdown_connections 2024-04-25T10:19:07.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.628+0000 7f322be00640 1 -- 172.21.15.87:0/3515761235 wait complete. 2024-04-25T10:19:07.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 Processor -- start 2024-04-25T10:19:07.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 -- start start 2024-04-25T10:19:07.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f322c0cd530 0x7f322c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cf490 0x7f322c159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f322c15b950 0x7f322c149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b17e0 con 0x7f322c0cf490 2024-04-25T10:19:07.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b1660 con 0x7f322c15b950 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b1960 con 0x7f322c0cd530 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cf490 0x7f322c159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cf490 0x7f322c159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41944/0 (socket says 172.21.15.87:41944) 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 -- 172.21.15.87:0/952936582 learned_addr learned my addr 172.21.15.87:0/952936582 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 -- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f322c0cd530 msgr2=0x7f322c14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322aa00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f322c0cd530 0x7f322c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322b400640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f322c15b950 0x7f322c149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f322c0cd530 0x7f322c14ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 -- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f322c15b950 msgr2=0x7f322c149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f322c15b950 0x7f322c149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 -- 172.21.15.87:0/952936582 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0ce910 con 0x7f322c0cf490 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322b400640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f322c15b950 0x7f322c149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322aa00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f322c0cd530 0x7f322c14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322a000640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cf490 0x7f322c159d70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f321c019060 tx=0x7f321c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f3222a00640 1 -- 172.21.15.87:0/952936582 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0b5810 con 0x7f322c0cf490 2024-04-25T10:19:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f3222a00640 1 -- 172.21.15.87:0/952936582 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0b59d0 con 0x7f322c0cf490 2024-04-25T10:19:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f3222a00640 1 -- 172.21.15.87:0/952936582 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0b5dc0 con 0x7f322c0cf490 2024-04-25T10:19:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c166380 con 0x7f322c0cf490 2024-04-25T10:19:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.632+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c06ae10 con 0x7f322c0cf490 2024-04-25T10:19:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3222a00640 1 -- 172.21.15.87:0/952936582 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f321c0bd020 con 0x7f322c0cf490 2024-04-25T10:19:07.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3222a00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3208066f60 0x7f3208069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3222a00640 1 -- 172.21.15.87:0/952936582 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f321c11f640 con 0x7f322c0cf490 2024-04-25T10:19:07.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f322aa00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3208066f60 0x7f3208069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 Processor -- start 2024-04-25T10:19:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 -- start start 2024-04-25T10:19:07.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f322aa00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3208066f60 0x7f3208069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f322c074190 tx=0x7f3214009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x5557725fae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557725b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557724981f0 con 0x5557725b5d90 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557723fee90 con 0x5557725faa40 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.636+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x555772386f50 con 0x5557725fb360 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557725b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557725b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41954/0 (socket says 172.21.15.87:41954) 2024-04-25T10:19:07.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 -- 172.21.15.87:0/3509511156 learned_addr learned my addr 172.21.15.87:0/3509511156 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322aa00640 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x5557725fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322a000640 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 -- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 msgr2=0x5557725b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 -- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 msgr2=0x5557725fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x5557725fae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 -- 172.21.15.87:0/3509511156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557723d68b0 con 0x5557725b5d90 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557725b6170 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f322400d460 tx=0x7f322400d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33865452a236090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f321b400640 1 -- 172.21.15.87:0/3509511156 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322401a070 con 0x5557725b5d90 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f321b400640 1 -- 172.21.15.87:0/3509511156 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400b740 con 0x5557725b5d90 2024-04-25T10:19:07.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f321b400640 1 -- 172.21.15.87:0/3509511156 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322400bb50 con 0x5557725b5d90 2024-04-25T10:19:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- 172.21.15.87:0/3509511156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 msgr2=0x5557725b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557725b6170 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f322400d460 tx=0x7f322400d920 comp rx=0 tx=0).stop 2024-04-25T10:19:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- 172.21.15.87:0/3509511156 shutdown_connections 2024-04-25T10:19:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557725b6170 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3509511156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x5557725fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- 172.21.15.87:0/3509511156 >> 172.21.15.87:0/3509511156 conn(0x5557725b1d40 msgr2=0x5557725b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- 172.21.15.87:0/3509511156 shutdown_connections 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- 172.21.15.87:0/3509511156 wait complete. 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- start start 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 Processor -- start 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557726021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x555772602700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557723b9540 con 0x5557725b5d90 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557723faf30 con 0x5557725faa40 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557723aa260 con 0x5557725fb360 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f3234d4d000 10 client.0 _ll_register_callbacks cb 0x5557724b3180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557726021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557726021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41960/0 (socket says 172.21.15.87:41960) 2024-04-25T10:19:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322aa00640 1 -- 172.21.15.87:0/3097452430 learned_addr learned my addr 172.21.15.87:0/3097452430 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322a000640 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x555772602700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.640+0000 7f322b400640 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322aa00640 1 -- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 msgr2=0x5557725bbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322aa00640 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725bbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322aa00640 1 -- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 msgr2=0x555772602700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322aa00640 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x555772602700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322aa00640 1 -- 172.21.15.87:0/3097452430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557723871a0 con 0x5557725b5d90 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322aa00640 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557726021c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f321400d710 tx=0x7f3214000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2ba3b97968b273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 1 -- 172.21.15.87:0/3097452430 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214026060 con 0x5557725b5d90 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 1 -- 172.21.15.87:0/3097452430 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3214021ab0 con 0x5557725b5d90 2024-04-25T10:19:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f322400b740 con 0x5557725b5d90 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 1 -- 172.21.15.87:0/3097452430 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3214021cf0 con 0x5557725b5d90 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 10 client.7641 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x555772481840 con 0x5557725b5d90 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 1 -- 172.21.15.87:0/3097452430 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400b740 con 0x5557725b5d90 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 1 -- 172.21.15.87:0/3097452430 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3214012410 con 0x5557725b5d90 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 10 client.7641 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 10 client.7641 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 20 client.7641 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557725bcb90 0x5557725bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 4 client.7641 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55577239abe0 con 0x5557725bcb90 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322a000640 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557725bcb90 0x5557725bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f322a000640 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557725bcb90 0x5557725bcfa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2015690c3d570a server_cookie=eb86b0e4017791a2 in_seq=0 out_seq=0 2024-04-25T10:19:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 10 client.7641 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 1 -- 172.21.15.87:0/3097452430 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55577239abe0 con 0x5557725bcb90 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 10 client.7641 handle_command_reply: tid=0 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 client.7641 shutdown 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 msgr2=0x5557726021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557726021c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f321400d710 tx=0x7f3214000bd0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 shutdown_connections 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557725bcb90 0x5557725bcfa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x5557725bbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725faa40 0x555772602700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 --2- 172.21.15.87:0/3097452430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725b5d90 0x5557726021c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f320fe00640 0 client.7641 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 >> 172.21.15.87:0/3097452430 conn(0x5557725b1d40 msgr2=0x5557725b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 shutdown_connections 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 1 -- 172.21.15.87:0/3097452430 wait complete. 2024-04-25T10:19:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.644+0000 7f3234d4d000 20 client.7641 trim_cache size 0 max 16384 2024-04-25T10:19:07.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 Processor -- start 2024-04-25T10:19:07.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 -- start start 2024-04-25T10:19:07.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725b5d90 0x5557726574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x5557726579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x555772657f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c202d00 con 0x5557725faa40 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c06c6a0 con 0x5557725b5d90 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1657d0 con 0x5557725fb360 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x5557726579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x5557726579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41968/0 (socket says 172.21.15.87:41968) 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 -- 172.21.15.87:0/1809744518 learned_addr learned my addr 172.21.15.87:0/1809744518 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322aa00640 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725b5d90 0x5557726574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322b400640 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x555772657f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 -- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 msgr2=0x555772657f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x555772657f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 -- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725b5d90 msgr2=0x5557726574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725b5d90 0x5557726574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 -- 172.21.15.87:0/1809744518 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557725bbd90 con 0x5557725faa40 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322aa00640 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725b5d90 0x5557726574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f322a000640 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x5557726579e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f321c01c9c0 tx=0x7f321c10e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa17e22a83a6f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f321aa00640 1 -- 172.21.15.87:0/1809744518 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0ee220 con 0x5557725faa40 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f321aa00640 1 -- 172.21.15.87:0/1809744518 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0b0070 con 0x5557725faa40 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.660+0000 7f321aa00640 1 -- 172.21.15.87:0/1809744518 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0ec590 con 0x5557725faa40 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/1809744518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 msgr2=0x5557726579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x5557726579e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f321c01c9c0 tx=0x7f321c10e040 comp rx=0 tx=0).stop 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/1809744518 shutdown_connections 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725fb360 0x555772657f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x5557726579e0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- 172.21.15.87:0/1809744518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725b5d90 0x5557726574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/1809744518 >> 172.21.15.87:0/1809744518 conn(0x5557725b1d40 msgr2=0x5557725f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/1809744518 shutdown_connections 2024-04-25T10:19:07.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/1809744518 wait complete. 2024-04-25T10:19:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- start start 2024-04-25T10:19:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 Processor -- start 2024-04-25T10:19:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725b5d90 0x55577265b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x55577265bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725fb360 0x5557725ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c06c0a0 con 0x5557725faa40 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557723faf30 con 0x5557725fb360 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557723fe1a0 con 0x5557725b5d90 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 10 client.0 _ll_register_callbacks cb 0x5557723c1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x55577265bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x55577265bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41984/0 (socket says 172.21.15.87:41984) 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 -- 172.21.15.87:0/2845089109 learned_addr learned my addr 172.21.15.87:0/2845089109 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322b400640 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725fb360 0x5557725ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322aa00640 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725b5d90 0x55577265b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 -- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725b5d90 msgr2=0x55577265b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725b5d90 0x55577265b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 -- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725fb360 msgr2=0x5557725ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725fb360 0x5557725ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 -- 172.21.15.87:0/2845089109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555772458a10 con 0x5557725faa40 2024-04-25T10:19:07.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f322a000640 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x55577265bc00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f321c01a080 tx=0x7f321c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d283347c00218a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f320fe00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f320fe00640 1 -- 172.21.15.87:0/2845089109 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c10d070 con 0x5557725faa40 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f320fe00640 1 -- 172.21.15.87:0/2845089109 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f321c0ec590 con 0x5557725faa40 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f321c0b0070 con 0x5557725faa40 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f320fe00640 1 -- 172.21.15.87:0/2845089109 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f321c0f60b0 con 0x5557725faa40 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 10 client.7647 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x555772393b00 con 0x5557725faa40 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f320fe00640 1 -- 172.21.15.87:0/2845089109 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0b0070 con 0x5557725faa40 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f320fe00640 1 -- 172.21.15.87:0/2845089109 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f321c0ef370 con 0x5557725faa40 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 10 client.7647 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 10 client.7647 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 20 client.7647 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557726008c0 0x555772600cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 4 client.7647 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.664+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55577240c3d0 con 0x5557726008c0 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322aa00640 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557726008c0 0x555772600cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322aa00640 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557726008c0 0x555772600cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8347bfdbaa1eda0 server_cookie=5cea55384559f1bd in_seq=0 out_seq=0 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f320fe00640 10 client.7647 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f320fe00640 1 -- 172.21.15.87:0/2845089109 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55577240c3d0 con 0x5557726008c0 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f320fe00640 10 client.7647 handle_command_reply: tid=0 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 client.7647 shutdown 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 msgr2=0x55577265bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x55577265bc00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f321c01a080 tx=0x7f321c0191f0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 shutdown_connections 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5557726008c0 0x555772600cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557725fb360 0x5557725ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5557725faa40 0x55577265bc00 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 --2- 172.21.15.87:0/2845089109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557725b5d90 0x55577265b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f320fe00640 0 client.7647 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 >> 172.21.15.87:0/2845089109 conn(0x5557725b1d40 msgr2=0x5557725f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 shutdown_connections 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 1 -- 172.21.15.87:0/2845089109 wait complete. 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f3234d4d000 20 client.7647 trim_cache size 0 max 16384 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:19:07.675 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0 2024-04-25T10:19:07.676 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:19:07.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3208066f60 msgr2=0x7f3208069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3208066f60 0x7f3208069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f322c074190 tx=0x7f3214009040 comp rx=0 tx=0).stop 2024-04-25T10:19:07.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cf490 msgr2=0x7f322c159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cf490 0x7f322c159d70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f321c019060 tx=0x7f321c01ac90 comp rx=0 tx=0).stop 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 shutdown_connections 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3208066f60 0x7f3208069420 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f322c15b950 0x7f322c149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322c0cf490 0x7f322c159d70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 --2- 172.21.15.87:0/952936582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f322c0cd530 0x7f322c14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 >> 172.21.15.87:0/952936582 conn(0x7f322c0d17e0 msgr2=0x7f322c154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 shutdown_connections 2024-04-25T10:19:07.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.668+0000 7f322be00640 1 -- 172.21.15.87:0/952936582 wait complete. 2024-04-25T10:19:07.689 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:19:07.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:19:07.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714f400640 1 Processor -- start 2024-04-25T10:19:07.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714f400640 1 -- start start 2024-04-25T10:19:07.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7148159540 0x7f71480ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71480721f0 0x7f7148070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7148159920 con 0x7f71480aebd0 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714f400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714815c2f0 con 0x7f7148159540 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714815c5d0 con 0x7f71480721f0 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714ca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7148159540 0x7f71480ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714ca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7148159540 0x7f71480ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41990/0 (socket says 172.21.15.87:41990) 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714ca00640 1 -- 172.21.15.87:0/3198156853 learned_addr learned my addr 172.21.15.87:0/3198156853 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714d400640 1 --2- 172.21.15.87:0/3198156853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71480721f0 0x7f7148070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714ca00640 1 -- 172.21.15.87:0/3198156853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71480721f0 msgr2=0x7f7148070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714ca00640 1 --2- 172.21.15.87:0/3198156853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71480721f0 0x7f7148070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714ca00640 1 -- 172.21.15.87:0/3198156853 >> v1:172.21.15.169:6789/0 conn(0x7f71480aebd0 legacy=0x7f7148071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.824+0000 7f714ca00640 1 -- 172.21.15.87:0/3198156853 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714815c8b0 con 0x7f7148159540 2024-04-25T10:19:07.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714ca00640 1 --2- 172.21.15.87:0/3198156853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7148159540 0x7f71480ae690 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7138009870 tx=0x7f713801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6f6b34ba129539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f7147400640 1 -- 172.21.15.87:0/3198156853 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380a7070 con 0x7f7148159540 2024-04-25T10:19:07.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f7147400640 1 -- 172.21.15.87:0/3198156853 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71380b0070 con 0x7f7148159540 2024-04-25T10:19:07.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f7147400640 1 -- 172.21.15.87:0/3198156853 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380ab890 con 0x7f7148159540 2024-04-25T10:19:07.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 -- 172.21.15.87:0/3198156853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7148159540 msgr2=0x7f71480ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 --2- 172.21.15.87:0/3198156853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7148159540 0x7f71480ae690 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7138009870 tx=0x7f713801c560 comp rx=0 tx=0).stop 2024-04-25T10:19:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 -- 172.21.15.87:0/3198156853 shutdown_connections 2024-04-25T10:19:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 --2- 172.21.15.87:0/3198156853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71480721f0 0x7f7148070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 --2- 172.21.15.87:0/3198156853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7148159540 0x7f71480ae690 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 -- 172.21.15.87:0/3198156853 >> 172.21.15.87:0/3198156853 conn(0x7f71480669a0 msgr2=0x7f7148066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 -- 172.21.15.87:0/3198156853 shutdown_connections 2024-04-25T10:19:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.828+0000 7f714f400640 1 -- 172.21.15.87:0/3198156853 wait complete. 2024-04-25T10:19:07.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 Processor -- start 2024-04-25T10:19:07.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 -- start start 2024-04-25T10:19:07.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71480721f0 0x7f7148134b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480aebd0 0x7f7148135350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7148159540 0x7f7148152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148071220 con 0x7f71480aebd0 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480710a0 con 0x7f71480721f0 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480713a0 con 0x7f7148159540 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480aebd0 0x7f7148135350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480aebd0 0x7f7148135350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41996/0 (socket says 172.21.15.87:41996) 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 -- 172.21.15.87:0/2832310239 learned_addr learned my addr 172.21.15.87:0/2832310239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714ca00640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71480721f0 0x7f7148134b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714d400640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7148159540 0x7f7148152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 -- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7148159540 msgr2=0x7f7148152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7148159540 0x7f7148152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 -- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71480721f0 msgr2=0x7f7148134b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71480721f0 0x7f7148134b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 -- 172.21.15.87:0/2832310239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714815a290 con 0x7f71480aebd0 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7147e00640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480aebd0 0x7f7148135350 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f713400c420 tx=0x7f713400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7144c00640 1 -- 172.21.15.87:0/2832310239 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340052d0 con 0x7f71480aebd0 2024-04-25T10:19:07.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7144c00640 1 -- 172.21.15.87:0/2832310239 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7134005470 con 0x7f71480aebd0 2024-04-25T10:19:07.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714815a570 con 0x7f71480aebd0 2024-04-25T10:19:07.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f7144c00640 1 -- 172.21.15.87:0/2832310239 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134007260 con 0x7f71480aebd0 2024-04-25T10:19:07.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.832+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714815ab30 con 0x7f71480aebd0 2024-04-25T10:19:07.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f7144c00640 1 -- 172.21.15.87:0/2832310239 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7134007400 con 0x7f71480aebd0 2024-04-25T10:19:07.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f7144c00640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7128066f60 0x7f7128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f7144c00640 1 -- 172.21.15.87:0/2832310239 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f71340844e0 con 0x7f71480aebd0 2024-04-25T10:19:07.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f714ca00640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7128066f60 0x7f7128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 Processor -- start 2024-04-25T10:19:07.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f714ca00640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7128066f60 0x7f7128069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f71380af000 tx=0x7f71380a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 -- start start 2024-04-25T10:19:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb39210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beaec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beaecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be937ff0 con 0x5579beb38e30 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be8c0150 con 0x5579beaec980 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.836+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579be934090 con 0x5579beb39750 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb39210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb39210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42006/0 (socket says 172.21.15.87:42006) 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 -- 172.21.15.87:0/2401550140 learned_addr learned my addr 172.21.15.87:0/2401550140 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f7147e00640 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beaec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714d400640 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beaecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 -- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 msgr2=0x5579beaec440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beaec440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 -- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 msgr2=0x5579beaecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beaecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 -- 172.21.15.87:0/2401550140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579beace0d0 con 0x5579beb38e30 2024-04-25T10:19:07.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714d400640 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beaecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:07.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb39210 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f713801c5a0 tx=0x7f7138004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bfbb122e2a0b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f713d600640 1 -- 172.21.15.87:0/2401550140 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380c9630 con 0x5579beb38e30 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f713d600640 1 -- 172.21.15.87:0/2401550140 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71380b0040 con 0x5579beb38e30 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f713d600640 1 -- 172.21.15.87:0/2401550140 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380c99d0 con 0x5579beb38e30 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- 172.21.15.87:0/2401550140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 msgr2=0x5579beb39210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb39210 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f713801c5a0 tx=0x7f7138004740 comp rx=0 tx=0).stop 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- 172.21.15.87:0/2401550140 shutdown_connections 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beaecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beaec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 --2- 172.21.15.87:0/2401550140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb39210 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- 172.21.15.87:0/2401550140 >> 172.21.15.87:0/2401550140 conn(0x5579beb199c0 msgr2=0x5579beb1bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- 172.21.15.87:0/2401550140 shutdown_connections 2024-04-25T10:19:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- 172.21.15.87:0/2401550140 wait complete. 2024-04-25T10:19:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- start start 2024-04-25T10:19:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 Processor -- start 2024-04-25T10:19:07.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb1f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beb1fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be8fc250 con 0x5579beb38e30 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be8f2690 con 0x5579beaec980 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579be8c0a40 con 0x5579beb39750 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f715288e000 10 client.0 _ll_register_callbacks cb 0x5579bead5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f7147e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb1f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f7147e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb1f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42014/0 (socket says 172.21.15.87:42014) 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f7147e00640 1 -- 172.21.15.87:0/2079080551 learned_addr learned my addr 172.21.15.87:0/2079080551 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714ca00640 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.840+0000 7f714d400640 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beb1fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7147e00640 1 -- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 msgr2=0x5579beb1fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7147e00640 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beb1fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7147e00640 1 -- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 msgr2=0x5579beb1f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7147e00640 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb1f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7147e00640 1 -- 172.21.15.87:0/2079080551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579be9e51c0 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7147e00640 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb1f660 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f713400cde0 tx=0x7f7134054fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e7ae0a01093127 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 1 -- 172.21.15.87:0/2079080551 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713406c020 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 1 -- 172.21.15.87:0/2079080551 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f71340514e0 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71380b0040 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 1 -- 172.21.15.87:0/2079080551 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f713405d040 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 10 client.7659 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5579be9e1170 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 1 -- 172.21.15.87:0/2079080551 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71380b0040 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 1 -- 172.21.15.87:0/2079080551 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f713406cb50 con 0x5579beb38e30 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 10 client.7659 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 10 client.7659 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:07.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 20 client.7659 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb48ed0 0x5579beb4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 4 client.7659 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579be8d3c60 con 0x5579beb48ed0 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f714ca00640 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb48ed0 0x5579beb4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f714ca00640 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb48ed0 0x5579beb4b2c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498409c7d12d3c2c server_cookie=16ec2100752524cf in_seq=0 out_seq=0 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 10 client.7659 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 1 -- 172.21.15.87:0/2079080551 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579be8d3c60 con 0x5579beb48ed0 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 10 client.7659 handle_command_reply: tid=0 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 client.7659 shutdown 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 msgr2=0x5579beb1f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb1f660 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f713400cde0 tx=0x7f7134054fb0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 shutdown_connections 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb48ed0 0x5579beb4b2c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb39750 0x5579beb1fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beb38e30 0x5579beb1f660 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f7132000640 0 client.7659 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 --2- 172.21.15.87:0/2079080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb1f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 >> 172.21.15.87:0/2079080551 conn(0x5579beb199c0 msgr2=0x5579beb1bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 shutdown_connections 2024-04-25T10:19:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 1 -- 172.21.15.87:0/2079080551 wait complete. 2024-04-25T10:19:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.844+0000 7f715288e000 20 client.7659 trim_cache size 0 max 16384 2024-04-25T10:19:07.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 Processor -- start 2024-04-25T10:19:07.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 -- start start 2024-04-25T10:19:07.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec050 0x5579beb96db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b4230 con 0x5579beaec050 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148149e90 con 0x5579beaec980 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714815ba80 con 0x5579beb199c0 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec050 0x5579beb96db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec050 0x5579beb96db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42024/0 (socket says 172.21.15.87:42024) 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 -- 172.21.15.87:0/2326383152 learned_addr learned my addr 172.21.15.87:0/2326383152 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f7147e00640 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714d400640 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 -- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 msgr2=0x5579beb972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 -- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 msgr2=0x5579beb97830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb97830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 -- 172.21.15.87:0/2326383152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579be97ede0 con 0x5579beaec050 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f714ca00640 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec050 0x5579beb96db0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f713801ca40 tx=0x7f7138008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344f0e67133997ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f713cc00640 1 -- 172.21.15.87:0/2326383152 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380d7120 con 0x5579beaec050 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f713cc00640 1 -- 172.21.15.87:0/2326383152 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71380b0040 con 0x5579beaec050 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.860+0000 7f713cc00640 1 -- 172.21.15.87:0/2326383152 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380d74e0 con 0x5579beaec050 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- 172.21.15.87:0/2326383152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec050 msgr2=0x5579beb96db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec050 0x5579beb96db0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f713801ca40 tx=0x7f7138008040 comp rx=0 tx=0).stop 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- 172.21.15.87:0/2326383152 shutdown_connections 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec980 0x5579beb97830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 --2- 172.21.15.87:0/2326383152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec050 0x5579beb96db0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- 172.21.15.87:0/2326383152 >> 172.21.15.87:0/2326383152 conn(0x5579beaef0e0 msgr2=0x5579beb492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- 172.21.15.87:0/2326383152 shutdown_connections 2024-04-25T10:19:07.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- 172.21.15.87:0/2326383152 wait complete. 2024-04-25T10:19:07.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- start start 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 Processor -- start 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec050 0x5579beb9a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec980 0x5579beb9ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb55d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be8d2e00 con 0x5579beaec980 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148149890 con 0x5579beaec050 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579be937300 con 0x5579beb199c0 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 10 client.0 _ll_register_callbacks cb 0x5579beab6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec980 0x5579beb9ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec980 0x5579beb9ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42034/0 (socket says 172.21.15.87:42034) 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f714ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec050 0x5579beb9a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f714d400640 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb55d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 -- 172.21.15.87:0/3923065448 learned_addr learned my addr 172.21.15.87:0/3923065448 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:07.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 -- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 msgr2=0x5579beb55d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb55d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 -- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec050 msgr2=0x5579beb9a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec050 0x5579beb9a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 -- 172.21.15.87:0/3923065448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579be8d98e0 con 0x5579beaec980 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f714ca00640 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec050 0x5579beb9a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7147e00640 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec980 0x5579beb9ae60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f713400c3f0 tx=0x7f71340828b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfac65649fdb1fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7132000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7132000640 1 -- 172.21.15.87:0/3923065448 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713406c040 con 0x5579beaec980 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7132000640 1 -- 172.21.15.87:0/3923065448 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f71340542c0 con 0x5579beaec980 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71380b0040 con 0x5579beaec980 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7132000640 1 -- 172.21.15.87:0/3923065448 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71340544c0 con 0x5579beaec980 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 10 client.7665 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5579be9918f0 con 0x5579beaec980 2024-04-25T10:19:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.864+0000 7f7132000640 1 -- 172.21.15.87:0/3923065448 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71380b0040 con 0x5579beaec980 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f7132000640 1 -- 172.21.15.87:0/3923065448 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7134050bc0 con 0x5579beaec980 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 10 client.7665 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 10 client.7665 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 20 client.7665 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb38e30 0x5579beb56780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 4 client.7665 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5579be945530 con 0x5579beb38e30 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714ca00640 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb38e30 0x5579beb56780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714ca00640 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb38e30 0x5579beb56780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84cca4ce9b7fa2f4 server_cookie=d0e7d245839fbf5d in_seq=0 out_seq=0 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f7132000640 10 client.7665 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f7132000640 1 -- 172.21.15.87:0/3923065448 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x5579be945530 con 0x5579beb38e30 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f7132000640 10 client.7665 handle_command_reply: tid=0 2024-04-25T10:19:07.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 client.7665 shutdown 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec980 msgr2=0x5579beb9ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec980 0x5579beb9ae60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f713400c3f0 tx=0x7f71340828b0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 shutdown_connections 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x5579beb38e30 0x5579beb56780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579beb199c0 0x5579beb55d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5579beaec980 0x5579beb9ae60 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f7132000640 0 client.7665 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 --2- 172.21.15.87:0/3923065448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579beaec050 0x5579beb9a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 >> 172.21.15.87:0/3923065448 conn(0x5579beaef0e0 msgr2=0x5579beb17d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 shutdown_connections 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 1 -- 172.21.15.87:0/3923065448 wait complete. 2024-04-25T10:19:07.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f715288e000 20 client.7665 trim_cache size 0 max 16384 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: "2b3df92c-1c29-4b5d-8fc9-88d045c49ade": { 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "2b3df92c-1c29-4b5d-8fc9-88d045c49ade", 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive" 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:07.876 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:19:07.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7128066f60 msgr2=0x7f7128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7128066f60 0x7f7128069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f71380af000 tx=0x7f71380a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:07.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480aebd0 msgr2=0x7f7148135350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480aebd0 0x7f7148135350 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f713400c420 tx=0x7f713400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 shutdown_connections 2024-04-25T10:19:07.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7128066f60 0x7f7128069420 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7148159540 0x7f7148152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480aebd0 0x7f7148135350 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 --2- 172.21.15.87:0/2832310239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71480721f0 0x7f7148134b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 >> 172.21.15.87:0/2832310239 conn(0x7f71480669a0 msgr2=0x7f714815d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 shutdown_connections 2024-04-25T10:19:07.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:07.868+0000 7f714f400640 1 -- 172.21.15.87:0/2832310239 wait complete. 2024-04-25T10:19:12.889 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:19:12.890 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:19:13.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d7e000640 1 Processor -- start 2024-04-25T10:19:13.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d7e000640 1 -- start start 2024-04-25T10:19:13.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d7e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 0x7f4d780ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d7e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d780aebd0 0x7f4d78071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d7e000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d78071360 con 0x7f4d78072020 2024-04-25T10:19:13.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d7e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d780af030 con 0x7f4d780aebd0 2024-04-25T10:19:13.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d7e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78159920 con 0x7f4d78159540 2024-04-25T10:19:13.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d77e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4d78072020 0x7f4d7815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60552/0 (socket says 172.21.15.87:60552) 2024-04-25T10:19:13.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d77e00640 1 -- 172.21.15.87:0/1635050256 learned_addr learned my addr 172.21.15.87:0/1635050256 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:13.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1635050256 >> v2:172.21.15.169:3300/0 conn(0x7f4d780aebd0 0x7f4d78071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d77400640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 0x7f4d780ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508868588 0 0) 0x7f4d78071360 con 0x7f4d78072020 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.036+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d5c003530 con 0x7f4d78072020 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3764696643 0 0) 0x7f4d5c003530 con 0x7f4d78072020 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 msgr2=0x7f4d780ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 0x7f4d780ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 >> v2:172.21.15.169:3300/0 conn(0x7f4d780aebd0 msgr2=0x7f4d78071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 --2- 172.21.15.87:0/1635050256 >> v2:172.21.15.169:3300/0 conn(0x7f4d780aebd0 0x7f4d78071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7815c2f0 con 0x7f4d78072020 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f4d70001a00 con 0x7f4d78072020 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 >> v1:172.21.15.87:6789/0 conn(0x7f4d78072020 legacy=0x7f4d7815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d5c0039c0 0x7f4d5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c004810 0x7f4d5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 0x7f4d5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78159920 con 0x7f4d5c0039c0 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c00b9d0 con 0x7f4d5c004810 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c00bb50 con 0x7f4d78159540 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77400640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d5c0039c0 0x7f4d5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c004810 0x7f4d5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 0x7f4d5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77400640 1 -- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 msgr2=0x7f4d5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77400640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 0x7f4d5c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77400640 1 -- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c004810 msgr2=0x7f4d5c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77400640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c004810 0x7f4d5c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77400640 1 -- 172.21.15.87:0/1635050256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c003530 con 0x7f4d5c0039c0 2024-04-25T10:19:13.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d78159540 0x7f4d5c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:13.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c004810 0x7f4d5c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d77400640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d5c0039c0 0x7f4d5c003de0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009ae0 tx=0x7f4d6801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead6e39fd89c6f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d680096b0 con 0x7f4d5c0039c0 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d680a6450 con 0x7f4d5c0039c0 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.040+0000 7f4d76000640 1 -- 172.21.15.87:0/1635050256 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d680a6840 con 0x7f4d5c0039c0 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- 172.21.15.87:0/1635050256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d5c0039c0 msgr2=0x7f4d5c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 --2- 172.21.15.87:0/1635050256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d5c0039c0 0x7f4d5c003de0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009ae0 tx=0x7f4d6801c9e0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d77e00640 1 -- 172.21.15.87:0/1635050256 reap_dead start 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- 172.21.15.87:0/1635050256 shutdown_connections 2024-04-25T10:19:13.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- 172.21.15.87:0/1635050256 >> 172.21.15.87:0/1635050256 conn(0x7f4d780669a0 msgr2=0x7f4d78066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- 172.21.15.87:0/1635050256 shutdown_connections 2024-04-25T10:19:13.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- 172.21.15.87:0/1635050256 wait complete. 2024-04-25T10:19:13.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 Processor -- start 2024-04-25T10:19:13.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- start start 2024-04-25T10:19:13.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c0039c0 0x7f4d780b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d780b2010 0x7f4d7815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7815d550 0x7f4d78144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7806c0d0 con 0x7f4d780b2010 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7815bfa0 con 0x7f4d5c0039c0 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d7e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78072430 con 0x7f4d7815d550 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d780b2010 0x7f4d7815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d780b2010 0x7f4d7815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42046/0 (socket says 172.21.15.87:42046) 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 -- 172.21.15.87:0/1528847191 learned_addr learned my addr 172.21.15.87:0/1528847191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7815d550 0x7f4d78144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d77400640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c0039c0 0x7f4d780b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 -- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7815d550 msgr2=0x7f4d78144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7815d550 0x7f4d78144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 -- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c0039c0 msgr2=0x7f4d780b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c0039c0 0x7f4d780b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.044+0000 7f4d76a00640 1 -- 172.21.15.87:0/1528847191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d78147880 con 0x7f4d780b2010 2024-04-25T10:19:13.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d780b2010 0x7f4d7815d010 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4d640118d0 tx=0x7f4d64011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d6f400640 1 -- 172.21.15.87:0/1528847191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64004220 con 0x7f4d780b2010 2024-04-25T10:19:13.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d6f400640 1 -- 172.21.15.87:0/1528847191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d640043c0 con 0x7f4d780b2010 2024-04-25T10:19:13.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d78147b60 con 0x7f4d780b2010 2024-04-25T10:19:13.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d6f400640 1 -- 172.21.15.87:0/1528847191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400fbd0 con 0x7f4d780b2010 2024-04-25T10:19:13.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d78148120 con 0x7f4d780b2010 2024-04-25T10:19:13.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d6f400640 1 -- 172.21.15.87:0/1528847191 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d64003620 con 0x7f4d780b2010 2024-04-25T10:19:13.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.048+0000 7f4d6f400640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d58066f90 0x7f4d58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d6f400640 1 -- 172.21.15.87:0/1528847191 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4d64085680 con 0x7f4d780b2010 2024-04-25T10:19:13.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d58066f90 0x7f4d58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 Processor -- start 2024-04-25T10:19:13.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- start start 2024-04-25T10:19:13.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d58066f90 0x7f4d58069450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ad000 tx=0x7f4d680b2040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f075b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f076480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004eec2ff0 con 0x56004f0bf9d0 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004ee4b150 con 0x56004f0c02f0 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56004eebf090 con 0x56004f0760a0 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42060/0 (socket says 172.21.15.87:42060) 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 -- 172.21.15.87:0/1201676403 learned_addr learned my addr 172.21.15.87:0/1201676403 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f076480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 -- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 msgr2=0x56004f076480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f076480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 -- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 msgr2=0x56004f075b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f075b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 -- 172.21.15.87:0/1201676403 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56004f0590d0 con 0x56004f0bf9d0 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d77400640 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0bfdb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4d6801d280 tx=0x7f4d68004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99095d8b8795d558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d63e00640 1 -- 172.21.15.87:0/1201676403 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d680cb070 con 0x56004f0bf9d0 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d63e00640 1 -- 172.21.15.87:0/1201676403 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d680a74f0 con 0x56004f0bf9d0 2024-04-25T10:19:13.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d63e00640 1 -- 172.21.15.87:0/1201676403 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d680a78a0 con 0x56004f0bf9d0 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- 172.21.15.87:0/1201676403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 msgr2=0x56004f0bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0bfdb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4d6801d280 tx=0x7f4d68004740 comp rx=0 tx=0).stop 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- 172.21.15.87:0/1201676403 shutdown_connections 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 --2- 172.21.15.87:0/1201676403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0bfdb0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- 172.21.15.87:0/1201676403 >> 172.21.15.87:0/1201676403 conn(0x56004f072020 msgr2=0x56004f074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- 172.21.15.87:0/1201676403 shutdown_connections 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.052+0000 7f4d813de000 1 -- 172.21.15.87:0/1201676403 wait complete. 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 -- start start 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 Processor -- start 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004ee87250 con 0x56004f0c02f0 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004ee7d690 con 0x56004f0760a0 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56004ee4ba40 con 0x56004f0bf9d0 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 10 client.0 _ll_register_callbacks cb 0x56004f060e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f0a5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42076/0 (socket says 172.21.15.87:42076) 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 -- 172.21.15.87:0/229631236 learned_addr learned my addr 172.21.15.87:0/229631236 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d76a00640 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77400640 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 -- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 msgr2=0x56004f0a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f0a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 -- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 msgr2=0x56004f0a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f0a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 -- 172.21.15.87:0/229631236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56004ef701c0 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77e00640 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f0a5270 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4d78070110 tx=0x7f4d70007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61559644eedb0a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d60c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d60c00640 1 -- 172.21.15.87:0/229631236 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700119f0 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d60c00640 1 -- 172.21.15.87:0/229631236 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4d70011b90 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d680a74f0 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d60c00640 1 -- 172.21.15.87:0/229631236 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d70022050 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 10 client.7677 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56004ef6c170 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d60c00640 1 -- 172.21.15.87:0/229631236 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d680a74f0 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d60c00640 1 -- 172.21.15.87:0/229631236 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d70025020 con 0x56004f0c02f0 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 10 client.7677 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:13.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 10 client.7677 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:13.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 20 client.7677 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:13.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f0cfc80 0x56004f0d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 4 client.7677 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:13.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56004ee5ec60 con 0x56004f0cfc80 2024-04-25T10:19:13.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77400640 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f0cfc80 0x56004f0d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d60c00640 10 client.7677 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:13.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.056+0000 7f4d77400640 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f0cfc80 0x56004f0d2070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd91c2d0e303fa2 server_cookie=469ad74924ede51f in_seq=0 out_seq=0 2024-04-25T10:19:13.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d60c00640 1 -- 172.21.15.87:0/229631236 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56004ee5ec60 con 0x56004f0cfc80 2024-04-25T10:19:13.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d60c00640 10 client.7677 handle_command_reply: tid=0 2024-04-25T10:19:13.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 client.7677 shutdown 2024-04-25T10:19:13.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 msgr2=0x56004f0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f0a5270 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4d78070110 tx=0x7f4d70007a90 comp rx=0 tx=0).stop 2024-04-25T10:19:13.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 shutdown_connections 2024-04-25T10:19:13.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f0cfc80 0x56004f0d2070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f0a5270 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f0a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d60c00640 0 client.7677 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 --2- 172.21.15.87:0/229631236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f0a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 >> 172.21.15.87:0/229631236 conn(0x56004f072020 msgr2=0x56004f074ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 shutdown_connections 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 1 -- 172.21.15.87:0/229631236 wait complete. 2024-04-25T10:19:13.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.060+0000 7f4d813de000 20 client.7677 trim_cache size 0 max 16384 2024-04-25T10:19:13.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.072+0000 7f4d813de000 1 Processor -- start 2024-04-25T10:19:13.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- start start 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f074410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f074950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f074e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d782037f0 con 0x56004f0c02f0 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d782029e0 con 0x56004f0760a0 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780b0cd0 con 0x56004f0bf9d0 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42082/0 (socket says 172.21.15.87:42082) 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 -- 172.21.15.87:0/1625426251 learned_addr learned my addr 172.21.15.87:0/1625426251 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77400640 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f074410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 -- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 msgr2=0x56004f074950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f074950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 -- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 msgr2=0x56004f074410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f074410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 -- 172.21.15.87:0/1625426251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56004ef09de0 con 0x56004f0c02f0 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77400640 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f074e90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f4d700013b0 tx=0x7f4d7000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402ad3e476d03452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d63400640 1 -- 172.21.15.87:0/1625426251 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700119f0 con 0x56004f0c02f0 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d63400640 1 -- 172.21.15.87:0/1625426251 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7001a070 con 0x56004f0c02f0 2024-04-25T10:19:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d63400640 1 -- 172.21.15.87:0/1625426251 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70015540 con 0x56004f0c02f0 2024-04-25T10:19:13.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- 172.21.15.87:0/1625426251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 msgr2=0x56004f074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f074e90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f4d700013b0 tx=0x7f4d7000a000 comp rx=0 tx=0).stop 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- 172.21.15.87:0/1625426251 shutdown_connections 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0c02f0 0x56004f074e90 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0bf9d0 0x56004f074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 --2- 172.21.15.87:0/1625426251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0760a0 0x56004f074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- 172.21.15.87:0/1625426251 >> 172.21.15.87:0/1625426251 conn(0x56004f072020 msgr2=0x56004f0be0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- 172.21.15.87:0/1625426251 shutdown_connections 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- 172.21.15.87:0/1625426251 wait complete. 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 -- start start 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.076+0000 7f4d813de000 1 Processor -- start 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f0d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004ee5de00 con 0x56004f0bf9d0 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78202440 con 0x56004f0c02f0 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56004eec2300 con 0x56004f0760a0 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 10 client.0 _ll_register_callbacks cb 0x56004f041040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42096/0 (socket says 172.21.15.87:42096) 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 -- 172.21.15.87:0/1770179326 learned_addr learned my addr 172.21.15.87:0/1770179326 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d77400640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d77e00640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f0d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 -- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 msgr2=0x56004f0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 -- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 msgr2=0x56004f0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f0d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 -- 172.21.15.87:0/1770179326 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56004ee648e0 con 0x56004f0bf9d0 2024-04-25T10:19:13.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d77400640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d76a00640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0d28c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400f5b0 tx=0x7f4d64006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef33d43b2a99518d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 1 -- 172.21.15.87:0/1770179326 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64022040 con 0x56004f0bf9d0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 1 -- 172.21.15.87:0/1770179326 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4d6405c0b0 con 0x56004f0bf9d0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d7001a070 con 0x56004f0bf9d0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 1 -- 172.21.15.87:0/1770179326 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d64074050 con 0x56004f0bf9d0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 10 client.7683 fetch_fsmap learned FSMap version 63 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56004ef1c8f0 con 0x56004f0bf9d0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 1 -- 172.21.15.87:0/1770179326 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7001a070 con 0x56004f0bf9d0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 1 -- 172.21.15.87:0/1770179326 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d640553c0 con 0x56004f0bf9d0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 10 client.7683 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 10 client.7683 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 20 client.7683 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f125f50 0x56004f128340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 4 client.7683 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 --> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56004eed0530 con 0x56004f125f50 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d77400640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f125f50 0x56004f128340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d77400640 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f125f50 0x56004f128340 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4652c890b0913a00 server_cookie=d1809feee108ef79 in_seq=0 out_seq=0 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 10 client.7683 ms_handle_connect on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 1 -- 172.21.15.87:0/1770179326 <== mds.0 v2:172.21.15.87:6835/3032837514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56004eed0530 con 0x56004f125f50 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 10 client.7683 handle_command_reply: tid=0 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 client.7683 shutdown 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 msgr2=0x56004f0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0d28c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400f5b0 tx=0x7f4d64006d60 comp rx=0 tx=0).stop 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 shutdown_connections 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:6835/3032837514,v1:172.21.15.87:6837/3032837514] conn(0x56004f125f50 0x56004f128340 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56004f0c02f0 0x56004f0d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56004f0bf9d0 0x56004f0d28c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d60c00640 0 client.7683 ms_handle_reset on v2:172.21.15.87:6835/3032837514 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 --2- 172.21.15.87:0/1770179326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56004f0760a0 0x56004f0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 >> 172.21.15.87:0/1770179326 conn(0x56004f072020 msgr2=0x56004f0be930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 shutdown_connections 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 1 -- 172.21.15.87:0/1770179326 wait complete. 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d813de000 20 client.7683 trim_cache size 0 max 16384 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d58066f90 msgr2=0x7f4d58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d7e000640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d58066f90 0x7f4d58069450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ad000 tx=0x7f4d680b2040 comp rx=0 tx=0).stop 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d780b2010 msgr2=0x7f4d7815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.080+0000 7f4d7e000640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d780b2010 0x7f4d7815d010 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4d640118d0 tx=0x7f4d64011d90 comp rx=0 tx=0).stop 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 shutdown_connections 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d58066f90 0x7f4d58069450 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7815d550 0x7f4d78144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d780b2010 0x7f4d7815d010 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 --2- 172.21.15.87:0/1528847191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d5c0039c0 0x7f4d780b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 >> 172.21.15.87:0/1528847191 conn(0x7f4d780669a0 msgr2=0x7f4d7815e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 shutdown_connections 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.084+0000 7f4d7e000640 1 -- 172.21.15.87:0/1528847191 wait complete. 2024-04-25T10:19:13.103 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:19:13.104 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:13.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cc6600640 1 Processor -- start 2024-04-25T10:19:13.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cc6600640 1 -- start start 2024-04-25T10:19:13.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cc6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5cc01550b0 0x7f5cc00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cc6600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc0071050 con 0x7f5cc015b910 2024-04-25T10:19:13.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cc6600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc015d040 con 0x7f5cc00cfe70 2024-04-25T10:19:13.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cc6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc015bd80 con 0x7f5cc01550b0 2024-04-25T10:19:13.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbfe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5cc015b910 0x7f5cc00b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60558/0 (socket says 172.21.15.87:60558) 2024-04-25T10:19:13.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbfe00640 1 -- 172.21.15.87:0/222104272 learned_addr learned my addr 172.21.15.87:0/222104272 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbf400640 1 --2- 172.21.15.87:0/222104272 >> v2:172.21.15.169:3300/0 conn(0x7f5cc01550b0 0x7f5cc00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741675385 0 0) 0x7f5cc0071050 con 0x7f5cc015b910 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca4003530 con 0x7f5cc015b910 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467543103 0 0) 0x7f5cc015d040 con 0x7f5cc00cfe70 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc0071050 con 0x7f5cc00cfe70 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3528072174 0 0) 0x7f5ca4003530 con 0x7f5cc015b910 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 >> v2:172.21.15.169:3300/0 conn(0x7f5cc01550b0 msgr2=0x7f5cc00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 --2- 172.21.15.87:0/222104272 >> v2:172.21.15.169:3300/0 conn(0x7f5cc01550b0 0x7f5cc00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 >> v1:172.21.15.169:6789/0 conn(0x7f5cc00cfe70 legacy=0x7f5cc015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc00712f0 con 0x7f5cc015b910 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5cb8001a00 con 0x7f5cc015b910 2024-04-25T10:19:13.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 >> v1:172.21.15.87:6789/0 conn(0x7f5cc015b910 legacy=0x7f5cc00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4003b50 0x7f5ca4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca4004810 0x7f5ca4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ca40051d0 0x7f5ca400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc015bd80 con 0x7f5ca4004810 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca400c050 con 0x7f5ca4003b50 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca400e460 con 0x7f5ca40051d0 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbea00640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca4004810 0x7f5ca4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbf400640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4003b50 0x7f5ca4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbfe00640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ca40051d0 0x7f5ca400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbea00640 1 -- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ca40051d0 msgr2=0x7f5ca400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbea00640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ca40051d0 0x7f5ca400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbea00640 1 -- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4003b50 msgr2=0x7f5ca4003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbea00640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4003b50 0x7f5ca4003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbea00640 1 -- 172.21.15.87:0/222104272 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca4003530 con 0x7f5ca4004810 2024-04-25T10:19:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.240+0000 7f5cbfe00640 1 -- 172.21.15.87:0/222104272 reap_dead start 2024-04-25T10:19:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cbea00640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca4004810 0x7f5ca4004c90 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0018d0 tx=0x7f5cac01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8835bfa6e5555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac001520 con 0x7f5ca4004810 2024-04-25T10:19:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac0052f0 con 0x7f5ca4004810 2024-04-25T10:19:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cbe000640 1 -- 172.21.15.87:0/222104272 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0a8930 con 0x7f5ca4004810 2024-04-25T10:19:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cc6600640 1 -- 172.21.15.87:0/222104272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca4004810 msgr2=0x7f5ca4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cc6600640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca4004810 0x7f5ca4004c90 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0018d0 tx=0x7f5cac01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cc6600640 1 -- 172.21.15.87:0/222104272 shutdown_connections 2024-04-25T10:19:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cc6600640 1 --2- 172.21.15.87:0/222104272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca4004810 0x7f5ca4004c90 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cc6600640 1 -- 172.21.15.87:0/222104272 >> 172.21.15.87:0/222104272 conn(0x7f5cc0066b70 msgr2=0x7f5cc0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cc6600640 1 -- 172.21.15.87:0/222104272 shutdown_connections 2024-04-25T10:19:13.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.244+0000 7f5cc6600640 1 -- 172.21.15.87:0/222104272 wait complete. 2024-04-25T10:19:13.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 Processor -- start 2024-04-25T10:19:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 -- start start 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4004810 0x7f5cc0149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cc0147b20 0x7f5cc01482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cc0148830 0x7f5cc00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0071b90 con 0x7f5cc0147b20 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00718b0 con 0x7f5ca4004810 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00720b0 con 0x7f5cc0148830 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cc0147b20 0x7f5cc01482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cc0147b20 0x7f5cc01482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42110/0 (socket says 172.21.15.87:42110) 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 -- 172.21.15.87:0/392096400 learned_addr learned my addr 172.21.15.87:0/392096400 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbf400640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4004810 0x7f5cc0149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbfe00640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cc0148830 0x7f5cc00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 -- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cc0148830 msgr2=0x7f5cc00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cc0148830 0x7f5cc00ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 -- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4004810 msgr2=0x7f5cc0149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4004810 0x7f5cc0149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 -- 172.21.15.87:0/392096400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0148cd0 con 0x7f5cc0147b20 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cbea00640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cc0147b20 0x7f5cc01482f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5cac01eb00 tx=0x7f5ca4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cb7400640 1 -- 172.21.15.87:0/392096400 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0be070 con 0x7f5cc0147b20 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cb7400640 1 -- 172.21.15.87:0/392096400 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac0054a0 con 0x7f5cc0147b20 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cb7400640 1 -- 172.21.15.87:0/392096400 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac001500 con 0x7f5cc0147b20 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc0148fb0 con 0x7f5cc0147b20 2024-04-25T10:19:13.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.248+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc0149570 con 0x7f5cc0147b20 2024-04-25T10:19:13.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.252+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc004ef30 con 0x7f5cc0147b20 2024-04-25T10:19:13.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.252+0000 7f5cb7400640 1 -- 172.21.15.87:0/392096400 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cac0a8c90 con 0x7f5cc0147b20 2024-04-25T10:19:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.252+0000 7f5cb7400640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ca0066f60 0x7f5ca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.252+0000 7f5cb7400640 1 -- 172.21.15.87:0/392096400 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5cac121590 con 0x7f5cc0147b20 2024-04-25T10:19:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.252+0000 7f5cbf400640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ca0066f60 0x7f5ca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.252+0000 7f5cbf400640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ca0066f60 0x7f5ca0069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5cc01f6eb0 tx=0x7f5cb0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.256+0000 7f5cb7400640 1 -- 172.21.15.87:0/392096400 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cac0189e0 con 0x7f5cc0147b20 2024-04-25T10:19:13.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.444+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc00b40b0 con 0x7f5cc0147b20 2024-04-25T10:19:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.444+0000 7f5cb7400640 1 -- 172.21.15.87:0/392096400 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5cac0f1f20 con 0x7f5cc0147b20 2024-04-25T10:19:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:13.448 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ca0066f60 msgr2=0x7f5ca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ca0066f60 0x7f5ca0069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5cc01f6eb0 tx=0x7f5cb0009190 comp rx=0 tx=0).stop 2024-04-25T10:19:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cc0147b20 msgr2=0x7f5cc01482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cc0147b20 0x7f5cc01482f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5cac01eb00 tx=0x7f5ca4005840 comp rx=0 tx=0).stop 2024-04-25T10:19:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 shutdown_connections 2024-04-25T10:19:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ca0066f60 0x7f5ca0069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cc0148830 0x7f5cc00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cc0147b20 0x7f5cc01482f0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 --2- 172.21.15.87:0/392096400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ca4004810 0x7f5cc0149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 >> 172.21.15.87:0/392096400 conn(0x7f5cc0066b70 msgr2=0x7f5cc015c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 shutdown_connections 2024-04-25T10:19:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.448+0000 7f5cc6600640 1 -- 172.21.15.87:0/392096400 wait complete. 2024-04-25T10:19:13.469 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:19:13.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:19:13.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.600+0000 7fd840200640 1 Processor -- start 2024-04-25T10:19:13.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.600+0000 7fd840200640 1 -- start start 2024-04-25T10:19:13.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd840200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd83815a0e0 0x7fd83815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd840200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd8380b3190 0x7fd83806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd840200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83815b6e0 con 0x7fd838071f70 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd840200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd838070ed0 con 0x7fd8380b3190 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd840200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380711b0 con 0x7fd83815a0e0 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd8380b3190 0x7fd83806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd8380b3190 0x7fd83806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42126/0 (socket says 172.21.15.87:42126) 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 -- 172.21.15.87:0/2249485255 learned_addr learned my addr 172.21.15.87:0/2249485255 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83d800640 1 --2- 172.21.15.87:0/2249485255 >> v2:172.21.15.169:3300/0 conn(0x7fd83815a0e0 0x7fd83815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 -- 172.21.15.87:0/2249485255 >> v2:172.21.15.169:3300/0 conn(0x7fd83815a0e0 msgr2=0x7fd83815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 --2- 172.21.15.87:0/2249485255 >> v2:172.21.15.169:3300/0 conn(0x7fd83815a0e0 0x7fd83815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 -- 172.21.15.87:0/2249485255 >> v1:172.21.15.169:6789/0 conn(0x7fd838071f70 legacy=0x7fd8380b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 -- 172.21.15.87:0/2249485255 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd838071490 con 0x7fd8380b3190 2024-04-25T10:19:13.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.604+0000 7fd83e200640 1 --2- 172.21.15.87:0/2249485255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380b3190 0x7fd83806c090 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd8380d0800 tx=0x7fd83001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d7e900caa85c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd837e00640 1 -- 172.21.15.87:0/2249485255 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300a5770 con 0x7fd8380b3190 2024-04-25T10:19:13.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd837e00640 1 -- 172.21.15.87:0/2249485255 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8300a5910 con 0x7fd8380b3190 2024-04-25T10:19:13.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd837e00640 1 -- 172.21.15.87:0/2249485255 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300a5ce0 con 0x7fd8380b3190 2024-04-25T10:19:13.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- 172.21.15.87:0/2249485255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380b3190 msgr2=0x7fd83806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 --2- 172.21.15.87:0/2249485255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380b3190 0x7fd83806c090 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd8380d0800 tx=0x7fd83001c930 comp rx=0 tx=0).stop 2024-04-25T10:19:13.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- 172.21.15.87:0/2249485255 shutdown_connections 2024-04-25T10:19:13.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 --2- 172.21.15.87:0/2249485255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380b3190 0x7fd83806c090 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 --2- 172.21.15.87:0/2249485255 >> v2:172.21.15.169:3300/0 conn(0x7fd83815a0e0 0x7fd83815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- 172.21.15.87:0/2249485255 >> 172.21.15.87:0/2249485255 conn(0x7fd838066b70 msgr2=0x7fd838066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- 172.21.15.87:0/2249485255 shutdown_connections 2024-04-25T10:19:13.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- 172.21.15.87:0/2249485255 wait complete. 2024-04-25T10:19:13.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 Processor -- start 2024-04-25T10:19:13.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- start start 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838071f70 0x7fd83814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380b3190 0x7fd838148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd83815a0e0 0x7fd83814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd838070b50 con 0x7fd83815a0e0 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83815a4c0 con 0x7fd8380b3190 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd840200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd838072370 con 0x7fd838071f70 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd83e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd83815a0e0 0x7fd83814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd83e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd83815a0e0 0x7fd83814f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42142/0 (socket says 172.21.15.87:42142) 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd83e200640 1 -- 172.21.15.87:0/2837251662 learned_addr learned my addr 172.21.15.87:0/2837251662 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:13.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.608+0000 7fd83ce00640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380b3190 0x7fd838148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83d800640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838071f70 0x7fd83814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83e200640 1 -- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838071f70 msgr2=0x7fd83814d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83e200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838071f70 0x7fd83814d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83e200640 1 -- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380b3190 msgr2=0x7fd838148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83e200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380b3190 0x7fd838148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83e200640 1 -- 172.21.15.87:0/2837251662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83814bff0 con 0x7fd83815a0e0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83d800640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838071f70 0x7fd83814d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd83e200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd83815a0e0 0x7fd83814f740 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd830019540 tx=0x7fd8300196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd835600640 1 -- 172.21.15.87:0/2837251662 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300bc020 con 0x7fd83815a0e0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd835600640 1 -- 172.21.15.87:0/2837251662 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8300b8670 con 0x7fd83815a0e0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83814c2d0 con 0x7fd83815a0e0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd835600640 1 -- 172.21.15.87:0/2837251662 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300b8a40 con 0x7fd83815a0e0 2024-04-25T10:19:13.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83814c890 con 0x7fd83815a0e0 2024-04-25T10:19:13.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83804ef30 con 0x7fd83815a0e0 2024-04-25T10:19:13.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.612+0000 7fd835600640 1 -- 172.21.15.87:0/2837251662 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8300bf020 con 0x7fd83815a0e0 2024-04-25T10:19:13.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.616+0000 7fd835600640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd818066f60 0x7fd818069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.616+0000 7fd835600640 1 -- 172.21.15.87:0/2837251662 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd8301200f0 con 0x7fd83815a0e0 2024-04-25T10:19:13.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.616+0000 7fd83d800640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd818066f60 0x7fd818069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.616+0000 7fd83d800640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd818066f60 0x7fd818069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009b10 tx=0x7fd82c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.620+0000 7fd835600640 1 -- 172.21.15.87:0/2837251662 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8300c6030 con 0x7fd83815a0e0 2024-04-25T10:19:13.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.824+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd8380b3c30 con 0x7fd83815a0e0 2024-04-25T10:19:13.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.824+0000 7fd835600640 1 -- 172.21.15.87:0/2837251662 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1811 (secure 0 0 0) 0x7fd8300165e0 con 0x7fd83815a0e0 2024-04-25T10:19:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:13.829 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":435,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1001,"data_bytes":1027017426,"bytes_used":2449494016,"bytes_avail":770644619264,"bytes_total":773094113280,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":1310410,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":10485760,"num_keys_recovered":0,"write_bytes_sec":32046183,"read_op_per_sec":0,"write_op_per_sec":32},"fsmap":{"epoch":63,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7467}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:19:11.212307+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:19:13.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd818066f60 msgr2=0x7fd818069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd818066f60 0x7fd818069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009b10 tx=0x7fd82c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:13.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd83815a0e0 msgr2=0x7fd83814f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd83815a0e0 0x7fd83814f740 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd830019540 tx=0x7fd8300196d0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 shutdown_connections 2024-04-25T10:19:13.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd818066f60 0x7fd818069420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd83815a0e0 0x7fd83814f740 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380b3190 0x7fd838148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 --2- 172.21.15.87:0/2837251662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838071f70 0x7fd83814d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 >> 172.21.15.87:0/2837251662 conn(0x7fd838066b70 msgr2=0x7fd83815cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 shutdown_connections 2024-04-25T10:19:13.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:13.828+0000 7fd840200640 1 -- 172.21.15.87:0/2837251662 wait complete. 2024-04-25T10:19:13.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:19:14.850 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:14.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.988+0000 7fcef8a00640 1 Processor -- start 2024-04-25T10:19:14.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.988+0000 7fcef8a00640 1 -- start start 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcef4072200 0x7fcef40725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcef40cd380 0x7fcef40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef8a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef40b1330 con 0x7fcef4072b20 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef8a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40cbee0 con 0x7fcef4072200 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40b1610 con 0x7fcef40cd380 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef1400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcef4072b20 0x7fcef40b0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60562/0 (socket says 172.21.15.87:60562) 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef1400640 1 -- 172.21.15.87:0/2986274108 learned_addr learned my addr 172.21.15.87:0/2986274108 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:14.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef1e00640 1 --2- 172.21.15.87:0/2986274108 >> v2:172.21.15.87:3300/0 conn(0x7fcef4072200 0x7fcef40725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef2800640 1 --2- 172.21.15.87:0/2986274108 >> v2:172.21.15.169:3300/0 conn(0x7fcef40cd380 0x7fcef40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef1e00640 1 -- 172.21.15.87:0/2986274108 >> v2:172.21.15.169:3300/0 conn(0x7fcef40cd380 msgr2=0x7fcef40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:14.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef1e00640 1 --2- 172.21.15.87:0/2986274108 >> v2:172.21.15.169:3300/0 conn(0x7fcef40cd380 0x7fcef40b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef1e00640 1 -- 172.21.15.87:0/2986274108 >> v1:172.21.15.87:6789/0 conn(0x7fcef4072b20 legacy=0x7fcef40b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef1e00640 1 -- 172.21.15.87:0/2986274108 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40b18f0 con 0x7fcef4072200 2024-04-25T10:19:14.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.992+0000 7fcef0a00640 1 -- 172.21.15.87:0/2986274108 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317658104 0 0) 0x7fcef40b1330 con 0x7fcef4072b20 2024-04-25T10:19:14.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef1e00640 1 --2- 172.21.15.87:0/2986274108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072200 0x7fcef40725e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fcef4070da0 tx=0x7fcee401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62db207734f5f50b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef0a00640 1 -- 172.21.15.87:0/2986274108 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee40b1070 con 0x7fcef4072200 2024-04-25T10:19:14.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef0a00640 1 -- 172.21.15.87:0/2986274108 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee40ac5f0 con 0x7fcef4072200 2024-04-25T10:19:14.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef0a00640 1 -- 172.21.15.87:0/2986274108 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee40ac9c0 con 0x7fcef4072200 2024-04-25T10:19:14.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 -- 172.21.15.87:0/2986274108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072200 msgr2=0x7fcef40725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2986274108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072200 0x7fcef40725e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fcef4070da0 tx=0x7fcee401c480 comp rx=0 tx=0).stop 2024-04-25T10:19:14.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 -- 172.21.15.87:0/2986274108 shutdown_connections 2024-04-25T10:19:14.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2986274108 >> v2:172.21.15.169:3300/0 conn(0x7fcef40cd380 0x7fcef40b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2986274108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072200 0x7fcef40725e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 -- 172.21.15.87:0/2986274108 >> 172.21.15.87:0/2986274108 conn(0x7fcef40b1d50 msgr2=0x7fcef40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:14.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 -- 172.21.15.87:0/2986274108 shutdown_connections 2024-04-25T10:19:14.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 -- 172.21.15.87:0/2986274108 wait complete. 2024-04-25T10:19:15.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 Processor -- start 2024-04-25T10:19:15.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:14.996+0000 7fcef8a00640 1 -- start start 2024-04-25T10:19:15.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcef4072200 0x7fcef4164df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072b20 0x7fcef40cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcef40cd380 0x7fcef40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4168310 con 0x7fcef4072b20 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4168190 con 0x7fcef4072200 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4168490 con 0x7fcef40cd380 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072b20 0x7fcef40cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072b20 0x7fcef40cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42160/0 (socket says 172.21.15.87:42160) 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 -- 172.21.15.87:0/2729901589 learned_addr learned my addr 172.21.15.87:0/2729901589 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1e00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcef4072200 0x7fcef4164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef2800640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcef40cd380 0x7fcef40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 -- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcef40cd380 msgr2=0x7fcef40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcef40cd380 0x7fcef40ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 -- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcef4072200 msgr2=0x7fcef4164df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcef4072200 0x7fcef4164df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 -- 172.21.15.87:0/2729901589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40ceb70 con 0x7fcef4072b20 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef1400640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072b20 0x7fcef40cde00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0011f0 tx=0x7fcedc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fceea000640 1 -- 172.21.15.87:0/2729901589 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc018070 con 0x7fcef4072b20 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fceea000640 1 -- 172.21.15.87:0/2729901589 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc0074e0 con 0x7fcef4072b20 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fceea000640 1 -- 172.21.15.87:0/2729901589 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc004a10 con 0x7fcef4072b20 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40cee50 con 0x7fcef4072b20 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef40cf410 con 0x7fcef4072b20 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef40ce8e0 con 0x7fcef4072b20 2024-04-25T10:19:15.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.000+0000 7fceea000640 1 -- 172.21.15.87:0/2729901589 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcedc01b070 con 0x7fcef4072b20 2024-04-25T10:19:15.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.004+0000 7fceea000640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fced0066f60 0x7fced0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.004+0000 7fceea000640 1 -- 172.21.15.87:0/2729901589 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcedc084370 con 0x7fcef4072b20 2024-04-25T10:19:15.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.004+0000 7fcef1e00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fced0066f60 0x7fced0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.004+0000 7fcef1e00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fced0066f60 0x7fced0069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcee40ae000 tx=0x7fcee40a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.008+0000 7fceea000640 1 -- 172.21.15.87:0/2729901589 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcedc054d00 con 0x7fcef4072b20 2024-04-25T10:19:15.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef404ef30 con 0x7fcef4072b20 2024-04-25T10:19:15.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fceea000640 1 -- 172.21.15.87:0/2729901589 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fcedc058da0 con 0x7fcef4072b20 2024-04-25T10:19:15.198 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:15.198 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:15.198 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:15.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fced0066f60 msgr2=0x7fced0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fced0066f60 0x7fced0069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcee40ae000 tx=0x7fcee40a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:15.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072b20 msgr2=0x7fcef40cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072b20 0x7fcef40cde00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0011f0 tx=0x7fcedc00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 shutdown_connections 2024-04-25T10:19:15.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fced0066f60 0x7fced0069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcef40cd380 0x7fcef40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcef4072b20 0x7fcef40cde00 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 --2- 172.21.15.87:0/2729901589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcef4072200 0x7fcef4164df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 >> 172.21.15.87:0/2729901589 conn(0x7fcef40b1d50 msgr2=0x7fcef4154d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 shutdown_connections 2024-04-25T10:19:15.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.196+0000 7fcef8a00640 1 -- 172.21.15.87:0/2729901589 wait complete. 2024-04-25T10:19:15.219 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:19:15.219 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:19:15.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.348+0000 7f35ea400640 1 Processor -- start 2024-04-25T10:19:15.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35ea400640 1 -- start start 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35ea400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f35e40af9c0 0x7f35e406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35ea400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e40cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35ea400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e406f200 con 0x7f35e4071f50 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35ea400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e406d380 con 0x7f35e40af9c0 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35ea400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40ccc10 con 0x7f35e40afda0 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f35e40af9c0 0x7f35e406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f35e40af9c0 0x7f35e406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42176/0 (socket says 172.21.15.87:42176) 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e3400640 1 -- 172.21.15.87:0/2122256655 learned_addr learned my addr 172.21.15.87:0/2122256655 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e2a00640 1 --2- 172.21.15.87:0/2122256655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e40cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e3400640 1 -- 172.21.15.87:0/2122256655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 msgr2=0x7f35e40cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e3400640 1 --2- 172.21.15.87:0/2122256655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e40cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e3400640 1 -- 172.21.15.87:0/2122256655 >> v1:172.21.15.169:6789/0 conn(0x7f35e4071f50 legacy=0x7f35e40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.352+0000 7f35e3400640 1 -- 172.21.15.87:0/2122256655 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40ccef0 con 0x7f35e40af9c0 2024-04-25T10:19:15.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35e3400640 1 --2- 172.21.15.87:0/2122256655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40af9c0 0x7f35e406ddd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009870 tx=0x7f35cc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac451febab537a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35e2000640 1 -- 172.21.15.87:0/2122256655 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0ab610 con 0x7f35e40af9c0 2024-04-25T10:19:15.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35e2000640 1 -- 172.21.15.87:0/2122256655 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc0b0070 con 0x7f35e40af9c0 2024-04-25T10:19:15.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35e2000640 1 -- 172.21.15.87:0/2122256655 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0ab9d0 con 0x7f35e40af9c0 2024-04-25T10:19:15.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 -- 172.21.15.87:0/2122256655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40af9c0 msgr2=0x7f35e406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 --2- 172.21.15.87:0/2122256655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40af9c0 0x7f35e406ddd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009870 tx=0x7f35cc01c410 comp rx=0 tx=0).stop 2024-04-25T10:19:15.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 -- 172.21.15.87:0/2122256655 shutdown_connections 2024-04-25T10:19:15.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 --2- 172.21.15.87:0/2122256655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e40cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 --2- 172.21.15.87:0/2122256655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40af9c0 0x7f35e406ddd0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 -- 172.21.15.87:0/2122256655 >> 172.21.15.87:0/2122256655 conn(0x7f35e40b2410 msgr2=0x7f35e40b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 -- 172.21.15.87:0/2122256655 shutdown_connections 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 -- 172.21.15.87:0/2122256655 wait complete. 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 Processor -- start 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.356+0000 7f35ea400640 1 -- start start 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4071f50 0x7f35e40c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40af9c0 0x7f35e414df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e41591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4067230 con 0x7f35e4071f50 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40670b0 con 0x7f35e40af9c0 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e41600d0 con 0x7f35e40afda0 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4071f50 0x7f35e40c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4071f50 0x7f35e40c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42178/0 (socket says 172.21.15.87:42178) 2024-04-25T10:19:15.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 -- 172.21.15.87:0/3943357082 learned_addr learned my addr 172.21.15.87:0/3943357082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e2a00640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40af9c0 0x7f35e414df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3e00640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e41591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 -- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 msgr2=0x7f35e41591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e41591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 -- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40af9c0 msgr2=0x7f35e414df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40af9c0 0x7f35e414df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 -- 172.21.15.87:0/3943357082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40c6970 con 0x7f35e4071f50 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3e00640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e41591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35e3400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4071f50 0x7f35e40c6390 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0ad040 tx=0x7f35cc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35db400640 1 -- 172.21.15.87:0/3943357082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0be5f0 con 0x7f35e4071f50 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35db400640 1 -- 172.21.15.87:0/3943357082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc0be790 con 0x7f35e4071f50 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35db400640 1 -- 172.21.15.87:0/3943357082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0beb80 con 0x7f35e4071f50 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e40c6c50 con 0x7f35e4071f50 2024-04-25T10:19:15.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e414ec20 con 0x7f35e4071f50 2024-04-25T10:19:15.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e414e730 con 0x7f35e4071f50 2024-04-25T10:19:15.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.360+0000 7f35db400640 1 -- 172.21.15.87:0/3943357082 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35cc0b43f0 con 0x7f35e4071f50 2024-04-25T10:19:15.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.364+0000 7f35db400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35c0066f60 0x7f35c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.364+0000 7f35db400640 1 -- 172.21.15.87:0/3943357082 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f35cc11fb60 con 0x7f35e4071f50 2024-04-25T10:19:15.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.364+0000 7f35e2a00640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35c0066f60 0x7f35c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.364+0000 7f35e2a00640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35c0066f60 0x7f35c0069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f35e415e770 tx=0x7f35d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.368+0000 7f35db400640 1 -- 172.21.15.87:0/3943357082 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35cc0a8050 con 0x7f35e4071f50 2024-04-25T10:19:15.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f35e404ef30 con 0x7f35e4071f50 2024-04-25T10:19:15.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35db400640 1 -- 172.21.15.87:0/3943357082 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1811 (secure 0 0 0) 0x7f35cc016150 con 0x7f35e4071f50 2024-04-25T10:19:15.578 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:15.578 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":437,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076615520,"bytes_used":2582343680,"bytes_avail":770511769600,"bytes_total":773094113280,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":1300270,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":10485760,"num_keys_recovered":0,"write_bytes_sec":37950508,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":63,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7467}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:19:11.212307+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:19:15.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35c0066f60 msgr2=0x7f35c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35c0066f60 0x7f35c0069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f35e415e770 tx=0x7f35d4008040 comp rx=0 tx=0).stop 2024-04-25T10:19:15.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4071f50 msgr2=0x7f35e40c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4071f50 0x7f35e40c6390 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0ad040 tx=0x7f35cc01ac50 comp rx=0 tx=0).stop 2024-04-25T10:19:15.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 shutdown_connections 2024-04-25T10:19:15.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35c0066f60 0x7f35c0069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40afda0 0x7f35e41591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40af9c0 0x7f35e414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 --2- 172.21.15.87:0/3943357082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4071f50 0x7f35e40c6390 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 >> 172.21.15.87:0/3943357082 conn(0x7f35e40b2410 msgr2=0x7f35e406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 shutdown_connections 2024-04-25T10:19:15.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:15.576+0000 7f35ea400640 1 -- 172.21.15.87:0/3943357082 wait complete. 2024-04-25T10:19:15.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:19:16.599 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7443c00640 1 Processor -- start 2024-04-25T10:19:16.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7443c00640 1 -- start start 2024-04-25T10:19:16.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7443c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c1550b0 0x7f743c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7443c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f743c0cfe70 0x7f743c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7443c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c071050 con 0x7f743c15b740 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7443c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c15d040 con 0x7f743c0cfe70 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7443c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0712f0 con 0x7f743c1550b0 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f743be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f743c0cfe70 0x7f743c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f7441200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c1550b0 0x7f743c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f743be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f743c0cfe70 0x7f743c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:44798/0 (socket says 172.21.15.87:44798) 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.740+0000 7f743be00640 1 -- 172.21.15.87:0/3821560007 learned_addr learned my addr 172.21.15.87:0/3821560007 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743be00640 1 -- 172.21.15.87:0/3821560007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c1550b0 msgr2=0x7f743c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743be00640 1 --2- 172.21.15.87:0/3821560007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c1550b0 0x7f743c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743be00640 1 -- 172.21.15.87:0/3821560007 >> v1:172.21.15.169:6789/0 conn(0x7f743c15b740 legacy=0x7f743c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743be00640 1 -- 172.21.15.87:0/3821560007 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0715d0 con 0x7f743c0cfe70 2024-04-25T10:19:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743be00640 1 --2- 172.21.15.87:0/3821560007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c15b200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f742800bb10 tx=0x7f742801ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57cda3fd9c5d8070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743b400640 1 -- 172.21.15.87:0/3821560007 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280b4070 con 0x7f743c0cfe70 2024-04-25T10:19:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743b400640 1 -- 172.21.15.87:0/3821560007 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74280a7b10 con 0x7f743c0cfe70 2024-04-25T10:19:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f743b400640 1 -- 172.21.15.87:0/3821560007 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280af440 con 0x7f743c0cfe70 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 -- 172.21.15.87:0/3821560007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 msgr2=0x7f743c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 --2- 172.21.15.87:0/3821560007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c15b200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f742800bb10 tx=0x7f742801ec30 comp rx=0 tx=0).stop 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 -- 172.21.15.87:0/3821560007 shutdown_connections 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 --2- 172.21.15.87:0/3821560007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c15b200 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 --2- 172.21.15.87:0/3821560007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c1550b0 0x7f743c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 -- 172.21.15.87:0/3821560007 >> 172.21.15.87:0/3821560007 conn(0x7f743c066b70 msgr2=0x7f743c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 -- 172.21.15.87:0/3821560007 shutdown_connections 2024-04-25T10:19:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.744+0000 7f7443c00640 1 -- 172.21.15.87:0/3821560007 wait complete. 2024-04-25T10:19:16.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 Processor -- start 2024-04-25T10:19:16.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 -- start start 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c13b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f743c1550b0 0x7f743c15a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c15b740 0x7f743c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c15bcd0 con 0x7f743c1550b0 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c15bb50 con 0x7f743c0cfe70 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c15c030 con 0x7f743c15b740 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f743c1550b0 0x7f743c15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f743c1550b0 0x7f743c15a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42182/0 (socket says 172.21.15.87:42182) 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 -- 172.21.15.87:0/2383862488 learned_addr learned my addr 172.21.15.87:0/2383862488 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7441c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c15b740 0x7f743c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7441200640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c13b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 -- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c15b740 msgr2=0x7f743c158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c15b740 0x7f743c158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 -- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 msgr2=0x7f743c13b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c13b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 -- 172.21.15.87:0/2383862488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c15ce70 con 0x7f743c1550b0 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7441200640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c13b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f743be00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f743c1550b0 0x7f743c15a410 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f742800b2c0 tx=0x7f742801d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7438c00640 1 -- 172.21.15.87:0/2383862488 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280bf070 con 0x7f743c1550b0 2024-04-25T10:19:16.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7438c00640 1 -- 172.21.15.87:0/2383862488 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74280afe40 con 0x7f743c1550b0 2024-04-25T10:19:16.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7438c00640 1 -- 172.21.15.87:0/2383862488 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280b4070 con 0x7f743c1550b0 2024-04-25T10:19:16.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.748+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c159370 con 0x7f743c1550b0 2024-04-25T10:19:16.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.752+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c159930 con 0x7f743c1550b0 2024-04-25T10:19:16.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.752+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c04ef30 con 0x7f743c1550b0 2024-04-25T10:19:16.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.752+0000 7f7438c00640 1 -- 172.21.15.87:0/2383862488 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74280c2070 con 0x7f743c1550b0 2024-04-25T10:19:16.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.752+0000 7f7438c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f741c066f60 0x7f741c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.752+0000 7f7438c00640 1 -- 172.21.15.87:0/2383862488 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f742811f590 con 0x7f743c1550b0 2024-04-25T10:19:16.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.752+0000 7f7441200640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f741c066f60 0x7f741c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.752+0000 7f7441200640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f741c066f60 0x7f741c069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f742c0074c0 tx=0x7f742c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.756+0000 7f7438c00640 1 -- 172.21.15.87:0/2383862488 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7428018820 con 0x7f743c1550b0 2024-04-25T10:19:16.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f743c15dea0 con 0x7f743c1550b0 2024-04-25T10:19:16.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7438c00640 1 -- 172.21.15.87:0/2383862488 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f742811de20 con 0x7f743c1550b0 2024-04-25T10:19:16.950 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:16.950 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:16.950 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:16.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f741c066f60 msgr2=0x7f741c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f741c066f60 0x7f741c069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f742c0074c0 tx=0x7f742c00b040 comp rx=0 tx=0).stop 2024-04-25T10:19:16.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f743c1550b0 msgr2=0x7f743c15a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f743c1550b0 0x7f743c15a410 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f742800b2c0 tx=0x7f742801d470 comp rx=0 tx=0).stop 2024-04-25T10:19:16.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 shutdown_connections 2024-04-25T10:19:16.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f741c066f60 0x7f741c069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f743c15b740 0x7f743c158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f743c1550b0 0x7f743c15a410 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 --2- 172.21.15.87:0/2383862488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0cfe70 0x7f743c13b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 >> 172.21.15.87:0/2383862488 conn(0x7f743c066b70 msgr2=0x7f743c0af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 shutdown_connections 2024-04-25T10:19:16.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:16.948+0000 7f7443c00640 1 -- 172.21.15.87:0/2383862488 wait complete. 2024-04-25T10:19:16.971 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:19:16.971 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:19:17.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f7582e00640 1 Processor -- start 2024-04-25T10:19:17.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f7582e00640 1 -- start start 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f7582e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f757c0d0040 0x7f757c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f7582e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c071050 con 0x7f757c1550b0 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f7582e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c15d040 con 0x7f757c15b740 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f7582e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c0712f0 con 0x7f757c0d0040 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f757be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f757c1550b0 0x7f757c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60574/0 (socket says 172.21.15.87:60574) 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f757b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f757c0d0040 0x7f757c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f757be00640 1 -- 172.21.15.87:0/1633030871 learned_addr learned my addr 172.21.15.87:0/1633030871 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f757b400640 1 -- 172.21.15.87:0/1633030871 >> v1:172.21.15.169:6789/0 conn(0x7f757c15b740 legacy=0x7f757c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f757b400640 1 -- 172.21.15.87:0/1633030871 >> v1:172.21.15.87:6789/0 conn(0x7f757c1550b0 legacy=0x7f757c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.104+0000 7f757b400640 1 -- 172.21.15.87:0/1633030871 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0715d0 con 0x7f757c0d0040 2024-04-25T10:19:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f757b400640 1 --2- 172.21.15.87:0/1633030871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c15b200 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f75680097c0 tx=0x7f756801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7230860c56a0e27d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f757aa00640 1 -- 172.21.15.87:0/1633030871 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680b0070 con 0x7f757c0d0040 2024-04-25T10:19:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f757aa00640 1 -- 172.21.15.87:0/1633030871 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75680ab600 con 0x7f757c0d0040 2024-04-25T10:19:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f757aa00640 1 -- 172.21.15.87:0/1633030871 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680ab9d0 con 0x7f757c0d0040 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 -- 172.21.15.87:0/1633030871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 msgr2=0x7f757c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 --2- 172.21.15.87:0/1633030871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c15b200 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f75680097c0 tx=0x7f756801c430 comp rx=0 tx=0).stop 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 -- 172.21.15.87:0/1633030871 shutdown_connections 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 --2- 172.21.15.87:0/1633030871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c15b200 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 -- 172.21.15.87:0/1633030871 >> 172.21.15.87:0/1633030871 conn(0x7f757c066b70 msgr2=0x7f757c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 -- 172.21.15.87:0/1633030871 shutdown_connections 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 -- 172.21.15.87:0/1633030871 wait complete. 2024-04-25T10:19:17.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 Processor -- start 2024-04-25T10:19:17.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.108+0000 7f7582e00640 1 -- start start 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c14e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757c1550b0 0x7f757c149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c15b740 0x7f757c151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c15bd10 con 0x7f757c0d0040 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c15bb90 con 0x7f757c15b740 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c1684c0 con 0x7f757c1550b0 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c14e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c14e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42194/0 (socket says 172.21.15.87:42194) 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 -- 172.21.15.87:0/3609459212 learned_addr learned my addr 172.21.15.87:0/3609459212 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757b400640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757c1550b0 0x7f757c149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7580e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c15b740 0x7f757c151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 -- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757c1550b0 msgr2=0x7f757c149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757c1550b0 0x7f757c149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 -- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c15b740 msgr2=0x7f757c151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c15b740 0x7f757c151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 -- 172.21.15.87:0/3609459212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c14e7e0 con 0x7f757c0d0040 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7580e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c15b740 0x7f757c151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f757be00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c14e200 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f756c00c450 tx=0x7f756c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7573e00640 1 -- 172.21.15.87:0/3609459212 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c017070 con 0x7f757c0d0040 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7573e00640 1 -- 172.21.15.87:0/3609459212 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c005cf0 con 0x7f757c0d0040 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7573e00640 1 -- 172.21.15.87:0/3609459212 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c007290 con 0x7f757c0d0040 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c14eac0 con 0x7f757c0d0040 2024-04-25T10:19:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c14f080 con 0x7f757c0d0040 2024-04-25T10:19:17.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c04ef30 con 0x7f757c0d0040 2024-04-25T10:19:17.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.112+0000 7f7573e00640 1 -- 172.21.15.87:0/3609459212 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f756c01a070 con 0x7f757c0d0040 2024-04-25T10:19:17.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.116+0000 7f7573e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f755c066f60 0x7f755c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.116+0000 7f7573e00640 1 -- 172.21.15.87:0/3609459212 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f756c084330 con 0x7f757c0d0040 2024-04-25T10:19:17.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.116+0000 7f757b400640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f755c066f60 0x7f755c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.116+0000 7f757b400640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f755c066f60 0x7f755c069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f75680ad000 tx=0x7f7568008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.120+0000 7f7573e00640 1 -- 172.21.15.87:0/3609459212 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f756c054c30 con 0x7f757c0d0040 2024-04-25T10:19:17.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f757c0b3250 con 0x7f757c0d0040 2024-04-25T10:19:17.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7573e00640 1 -- 172.21.15.87:0/3609459212 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1811 (secure 0 0 0) 0x7f756c058cd0 con 0x7f757c0d0040 2024-04-25T10:19:17.334 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:17.334 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":439,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076615520,"bytes_used":2582343680,"bytes_avail":770511769600,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":1048330,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":10485760,"num_keys_recovered":0,"write_bytes_sec":30597226,"read_op_per_sec":0,"write_op_per_sec":31},"fsmap":{"epoch":63,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7467}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:19:11.212307+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:19:17.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f755c066f60 msgr2=0x7f755c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f755c066f60 0x7f755c069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f75680ad000 tx=0x7f7568008040 comp rx=0 tx=0).stop 2024-04-25T10:19:17.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 msgr2=0x7f757c14e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c14e200 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f756c00c450 tx=0x7f756c00c910 comp rx=0 tx=0).stop 2024-04-25T10:19:17.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 shutdown_connections 2024-04-25T10:19:17.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f755c066f60 0x7f755c069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c15b740 0x7f757c151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757c1550b0 0x7f757c149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 --2- 172.21.15.87:0/3609459212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f757c0d0040 0x7f757c14e200 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 >> 172.21.15.87:0/3609459212 conn(0x7f757c066b70 msgr2=0x7f757c0b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 shutdown_connections 2024-04-25T10:19:17.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:17.332+0000 7f7582e00640 1 -- 172.21.15.87:0/3609459212 wait complete. 2024-04-25T10:19:17.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:19:18.357 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:18.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.496+0000 7fd934000640 1 Processor -- start 2024-04-25T10:19:18.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.496+0000 7fd934000640 1 -- start start 2024-04-25T10:19:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd934000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd92c15bc40 0x7fd92c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd934000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c0b2770 con 0x7fd92c0cda10 2024-04-25T10:19:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd934000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c06c670 con 0x7fd92c0cf930 2024-04-25T10:19:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd934000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92c06c950 con 0x7fd92c15bc40 2024-04-25T10:19:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd931600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd92c15bc40 0x7fd92c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd930c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd92c0cda10 0x7fd92c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58560/0 (socket says 172.21.15.87:58560) 2024-04-25T10:19:18.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd930c00640 1 -- 172.21.15.87:0/1561836100 learned_addr learned my addr 172.21.15.87:0/1561836100 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd92be00640 1 -- 172.21.15.87:0/1561836100 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208767883 0 0) 0x7fd92c0b2770 con 0x7fd92c0cda10 2024-04-25T10:19:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd92be00640 1 -- 172.21.15.87:0/1561836100 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd914003530 con 0x7fd92c0cda10 2024-04-25T10:19:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd931600640 1 -- 172.21.15.87:0/1561836100 >> v1:172.21.15.169:6789/0 conn(0x7fd92c0cf930 legacy=0x7fd92c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd931600640 1 -- 172.21.15.87:0/1561836100 >> v1:172.21.15.87:6789/0 conn(0x7fd92c0cda10 legacy=0x7fd92c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.500+0000 7fd931600640 1 -- 172.21.15.87:0/1561836100 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c06cc30 con 0x7fd92c15bc40 2024-04-25T10:19:18.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd931600640 1 --2- 172.21.15.87:0/1561836100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c15bc40 0x7fd92c06c090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd9200097c0 tx=0x7fd92001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4498c5d601983b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd92be00640 1 -- 172.21.15.87:0/1561836100 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200b0070 con 0x7fd92c15bc40 2024-04-25T10:19:18.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd92be00640 1 -- 172.21.15.87:0/1561836100 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9200ab600 con 0x7fd92c15bc40 2024-04-25T10:19:18.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd92be00640 1 -- 172.21.15.87:0/1561836100 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200ab9d0 con 0x7fd92c15bc40 2024-04-25T10:19:18.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 -- 172.21.15.87:0/1561836100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c15bc40 msgr2=0x7fd92c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 --2- 172.21.15.87:0/1561836100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c15bc40 0x7fd92c06c090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd9200097c0 tx=0x7fd92001c410 comp rx=0 tx=0).stop 2024-04-25T10:19:18.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 -- 172.21.15.87:0/1561836100 shutdown_connections 2024-04-25T10:19:18.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 --2- 172.21.15.87:0/1561836100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c15bc40 0x7fd92c06c090 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 -- 172.21.15.87:0/1561836100 >> 172.21.15.87:0/1561836100 conn(0x7fd92c066b70 msgr2=0x7fd92c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:18.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 -- 172.21.15.87:0/1561836100 shutdown_connections 2024-04-25T10:19:18.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 -- 172.21.15.87:0/1561836100 wait complete. 2024-04-25T10:19:18.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 Processor -- start 2024-04-25T10:19:18.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.504+0000 7fd934000640 1 -- start start 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c0cda10 0x7fd92c0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92c0cf930 0x7fd92c15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd92c15bc40 0x7fd92c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c169d70 con 0x7fd92c0cda10 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c169bf0 con 0x7fd92c0cf930 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c169ef0 con 0x7fd92c15bc40 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c0cda10 0x7fd92c0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c0cda10 0x7fd92c0ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36940/0 (socket says 172.21.15.87:36940) 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 -- 172.21.15.87:0/448309836 learned_addr learned my addr 172.21.15.87:0/448309836 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 -- 172.21.15.87:0/448309836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd92c15bc40 msgr2=0x7fd92c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd92c15bc40 0x7fd92c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 -- 172.21.15.87:0/448309836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92c0cf930 msgr2=0x7fd92c15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92c0cf930 0x7fd92c15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 -- 172.21.15.87:0/448309836 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c0afe90 con 0x7fd92c0cda10 2024-04-25T10:19:18.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd931600640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c0cda10 0x7fd92c0ae890 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fd9200ae040 tx=0x7fd92001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd929600640 1 -- 172.21.15.87:0/448309836 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200b0050 con 0x7fd92c0cda10 2024-04-25T10:19:18.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd929600640 1 -- 172.21.15.87:0/448309836 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9200bc070 con 0x7fd92c0cda10 2024-04-25T10:19:18.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd929600640 1 -- 172.21.15.87:0/448309836 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200b76b0 con 0x7fd92c0cda10 2024-04-25T10:19:18.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c0b0170 con 0x7fd92c0cda10 2024-04-25T10:19:18.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c0b0730 con 0x7fd92c0cda10 2024-04-25T10:19:18.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c04ef30 con 0x7fd92c0cda10 2024-04-25T10:19:18.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.508+0000 7fd929600640 1 -- 172.21.15.87:0/448309836 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9200b7850 con 0x7fd92c0cda10 2024-04-25T10:19:18.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.512+0000 7fd929600640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd90c066f60 0x7fd90c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.512+0000 7fd929600640 1 -- 172.21.15.87:0/448309836 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd92011f710 con 0x7fd92c0cda10 2024-04-25T10:19:18.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.512+0000 7fd930c00640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd90c066f60 0x7fd90c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.512+0000 7fd930c00640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd90c066f60 0x7fd90c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd918007fd0 tx=0x7fd918007940 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.516+0000 7fd929600640 1 -- 172.21.15.87:0/448309836 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd920016090 con 0x7fd92c0cda10 2024-04-25T10:19:18.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd92c06d960 con 0x7fd92c0cda10 2024-04-25T10:19:18.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd929600640 1 -- 172.21.15.87:0/448309836 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd9200f0660 con 0x7fd92c0cda10 2024-04-25T10:19:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:18.707 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:18.707 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:18.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd90c066f60 msgr2=0x7fd90c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd90c066f60 0x7fd90c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd918007fd0 tx=0x7fd918007940 comp rx=0 tx=0).stop 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c0cda10 msgr2=0x7fd92c0ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c0cda10 0x7fd92c0ae890 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fd9200ae040 tx=0x7fd92001ac50 comp rx=0 tx=0).stop 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 shutdown_connections 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd90c066f60 0x7fd90c069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd92c15bc40 0x7fd92c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92c0cf930 0x7fd92c15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 --2- 172.21.15.87:0/448309836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd92c0cda10 0x7fd92c0ae890 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 >> 172.21.15.87:0/448309836 conn(0x7fd92c066b70 msgr2=0x7fd92c1564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 shutdown_connections 2024-04-25T10:19:18.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.704+0000 7fd934000640 1 -- 172.21.15.87:0/448309836 wait complete. 2024-04-25T10:19:18.728 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:19:18.728 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:19:18.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342f000640 1 Processor -- start 2024-04-25T10:19:18.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342f000640 1 -- start start 2024-04-25T10:19:18.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342f000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f34281550b0 0x7f34280cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f342815b910 0x7f34280b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342f000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3428071050 con 0x7f34280cfe70 2024-04-25T10:19:18.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342f000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342815bda0 con 0x7f34281550b0 2024-04-25T10:19:18.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342815d040 con 0x7f342815b910 2024-04-25T10:19:18.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f3427e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f34281550b0 0x7f34280cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f3427400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f34280cfe70 0x7f342815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58562/0 (socket says 172.21.15.87:58562) 2024-04-25T10:19:18.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f3427400640 1 -- 172.21.15.87:0/3472713675 learned_addr learned my addr 172.21.15.87:0/3472713675 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:18.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.860+0000 7f342d000640 1 --2- 172.21.15.87:0/3472713675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f342815b910 0x7f34280b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3427e00640 1 -- 172.21.15.87:0/3472713675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f342815b910 msgr2=0x7f34280b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:18.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3427e00640 1 --2- 172.21.15.87:0/3472713675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f342815b910 0x7f34280b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3427e00640 1 -- 172.21.15.87:0/3472713675 >> v1:172.21.15.87:6789/0 conn(0x7f34280cfe70 legacy=0x7f342815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3427e00640 1 -- 172.21.15.87:0/3472713675 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280712f0 con 0x7f34281550b0 2024-04-25T10:19:18.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3426a00640 1 -- 172.21.15.87:0/3472713675 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903081659 0 0) 0x7f3428071050 con 0x7f34280cfe70 2024-04-25T10:19:18.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3427e00640 1 --2- 172.21.15.87:0/3472713675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281550b0 0x7f34280cf930 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f34180097f0 tx=0x7f341801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557c0ca03053262e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3426a00640 1 -- 172.21.15.87:0/3472713675 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34180b2070 con 0x7f34281550b0 2024-04-25T10:19:18.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3426a00640 1 -- 172.21.15.87:0/3472713675 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34180ad640 con 0x7f34281550b0 2024-04-25T10:19:18.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f3426a00640 1 -- 172.21.15.87:0/3472713675 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34180ada10 con 0x7f34281550b0 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 -- 172.21.15.87:0/3472713675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281550b0 msgr2=0x7f34280cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 --2- 172.21.15.87:0/3472713675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281550b0 0x7f34280cf930 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f34180097f0 tx=0x7f341801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 -- 172.21.15.87:0/3472713675 shutdown_connections 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 --2- 172.21.15.87:0/3472713675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f342815b910 0x7f34280b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 --2- 172.21.15.87:0/3472713675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281550b0 0x7f34280cf930 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 -- 172.21.15.87:0/3472713675 >> 172.21.15.87:0/3472713675 conn(0x7f3428066b70 msgr2=0x7f3428066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 -- 172.21.15.87:0/3472713675 shutdown_connections 2024-04-25T10:19:18.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.864+0000 7f342f000640 1 -- 172.21.15.87:0/3472713675 wait complete. 2024-04-25T10:19:18.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 Processor -- start 2024-04-25T10:19:18.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 -- start start 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34280cfe70 0x7f3428153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34281550b0 0x7f3428154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f342815b910 0x7f3428159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342815c030 con 0x7f342815b910 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280720b0 con 0x7f34280cfe70 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342815c1b0 con 0x7f34281550b0 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f342815b910 0x7f3428159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f342815b910 0x7f3428159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36966/0 (socket says 172.21.15.87:36966) 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 -- 172.21.15.87:0/2576524584 learned_addr learned my addr 172.21.15.87:0/2576524584 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f3427400640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34281550b0 0x7f3428154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f3427e00640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34280cfe70 0x7f3428153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 -- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34281550b0 msgr2=0x7f3428154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34281550b0 0x7f3428154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 -- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34280cfe70 msgr2=0x7f3428153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34280cfe70 0x7f3428153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 -- 172.21.15.87:0/2576524584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342815a140 con 0x7f342815b910 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f3427e00640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34280cfe70 0x7f3428153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342d000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f342815b910 0x7f3428159410 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3420007e60 tx=0x7f342000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f341fe00640 1 -- 172.21.15.87:0/2576524584 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420014420 con 0x7f342815b910 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f341fe00640 1 -- 172.21.15.87:0/2576524584 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420019070 con 0x7f342815b910 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342815a420 con 0x7f342815b910 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.868+0000 7f341fe00640 1 -- 172.21.15.87:0/2576524584 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420014820 con 0x7f342815b910 2024-04-25T10:19:18.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.872+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342815a9e0 con 0x7f342815b910 2024-04-25T10:19:18.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.872+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3428155490 con 0x7f342815b910 2024-04-25T10:19:18.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.872+0000 7f341fe00640 1 -- 172.21.15.87:0/2576524584 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34200149c0 con 0x7f342815b910 2024-04-25T10:19:18.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.872+0000 7f341fe00640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3408066f60 0x7f3408069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.872+0000 7f341fe00640 1 -- 172.21.15.87:0/2576524584 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f34200858b0 con 0x7f342815b910 2024-04-25T10:19:18.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.872+0000 7f3427e00640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3408066f60 0x7f3408069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.872+0000 7f3427e00640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3408066f60 0x7f3408069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f34180af000 tx=0x7f3418008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:18.876+0000 7f341fe00640 1 -- 172.21.15.87:0/2576524584 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3420056800 con 0x7f342815b910 2024-04-25T10:19:19.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.088+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f342815dec0 con 0x7f342815b910 2024-04-25T10:19:19.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.088+0000 7f341fe00640 1 -- 172.21.15.87:0/2576524584 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1748 (secure 0 0 0) 0x7f3420056100 con 0x7f342815b910 2024-04-25T10:19:19.093 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:19.093 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":441,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076619745,"bytes_used":2582351872,"bytes_avail":770511761408,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":948701,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":10485760,"num_keys_recovered":0,"write_bytes_sec":27689955,"read_op_per_sec":0,"write_op_per_sec":28},"fsmap":{"epoch":63,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7467}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:19:17.213750+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:19:19.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3408066f60 msgr2=0x7f3408069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3408066f60 0x7f3408069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f34180af000 tx=0x7f3418008040 comp rx=0 tx=0).stop 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f342815b910 msgr2=0x7f3428159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f342815b910 0x7f3428159410 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3420007e60 tx=0x7f342000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 shutdown_connections 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3408066f60 0x7f3408069420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f342815b910 0x7f3428159410 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34281550b0 0x7f3428154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 --2- 172.21.15.87:0/2576524584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34280cfe70 0x7f3428153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 >> 172.21.15.87:0/2576524584 conn(0x7f3428066b70 msgr2=0x7f342806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 shutdown_connections 2024-04-25T10:19:19.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.092+0000 7f342f000640 1 -- 172.21.15.87:0/2576524584 wait complete. 2024-04-25T10:19:19.114 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:19:19.114 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:19:19.115 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:19.211 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:19.303 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:19:19.303 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:19:19.303 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:19.312 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:19:19.312 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:19.360 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:19.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f059ac00640 1 Processor -- start 2024-04-25T10:19:19.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f059ac00640 1 -- start start 2024-04-25T10:19:19.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f059ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059415dce0 0x7f05940669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f059ac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f059406eec0 0x7f059406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f059ac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059415e0c0 con 0x7f05940ad850 2024-04-25T10:19:19.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f059ac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05940674d0 con 0x7f059406eec0 2024-04-25T10:19:19.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f059ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05940671f0 con 0x7f059415dce0 2024-04-25T10:19:19.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f058b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f059406eec0 0x7f059406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f058b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f059406eec0 0x7f059406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36980/0 (socket says 172.21.15.87:36980) 2024-04-25T10:19:19.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f058b400640 1 -- 172.21.15.87:0/2330669972 learned_addr learned my addr 172.21.15.87:0/2330669972 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:19.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f0593e00640 1 --2- 172.21.15.87:0/2330669972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059415dce0 0x7f05940669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f058b400640 1 -- 172.21.15.87:0/2330669972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059415dce0 msgr2=0x7f05940669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f058b400640 1 --2- 172.21.15.87:0/2330669972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059415dce0 0x7f05940669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f058b400640 1 -- 172.21.15.87:0/2330669972 >> v1:172.21.15.169:6789/0 conn(0x7f05940ad850 legacy=0x7f05940adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.532+0000 7f058b400640 1 -- 172.21.15.87:0/2330669972 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059406d4b0 con 0x7f059406eec0 2024-04-25T10:19:19.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f058b400640 1 --2- 172.21.15.87:0/2330669972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059406eec0 0x7f059406df90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f057c009870 tx=0x7f057c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53208ca7af8f116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:19.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f0593400640 1 -- 172.21.15.87:0/2330669972 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c0b0070 con 0x7f059406eec0 2024-04-25T10:19:19.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f0593400640 1 -- 172.21.15.87:0/2330669972 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c0ab600 con 0x7f059406eec0 2024-04-25T10:19:19.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f0593400640 1 -- 172.21.15.87:0/2330669972 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c0ab9d0 con 0x7f059406eec0 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 -- 172.21.15.87:0/2330669972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059406eec0 msgr2=0x7f059406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 --2- 172.21.15.87:0/2330669972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059406eec0 0x7f059406df90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f057c009870 tx=0x7f057c01c440 comp rx=0 tx=0).stop 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 -- 172.21.15.87:0/2330669972 shutdown_connections 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 --2- 172.21.15.87:0/2330669972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059406eec0 0x7f059406df90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 --2- 172.21.15.87:0/2330669972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059415dce0 0x7f05940669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 -- 172.21.15.87:0/2330669972 >> 172.21.15.87:0/2330669972 conn(0x7f05940b2410 msgr2=0x7f05940b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 -- 172.21.15.87:0/2330669972 shutdown_connections 2024-04-25T10:19:19.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.536+0000 7f059ac00640 1 -- 172.21.15.87:0/2330669972 wait complete. 2024-04-25T10:19:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 Processor -- start 2024-04-25T10:19:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 -- start start 2024-04-25T10:19:19.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059406eec0 0x7f059415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05940ad850 0x7f0594157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059415dce0 0x7f05941485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05940d0320 con 0x7f059415dce0 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05940d04a0 con 0x7f05940ad850 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059416abc0 con 0x7f059406eec0 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059415dce0 0x7f05941485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059415dce0 0x7f05941485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36986/0 (socket says 172.21.15.87:36986) 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0593e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059406eec0 0x7f059415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 -- 172.21.15.87:0/561090011 learned_addr learned my addr 172.21.15.87:0/561090011 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f058b400640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05940ad850 0x7f0594157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 -- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059406eec0 msgr2=0x7f059415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059406eec0 0x7f059415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 -- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05940ad850 msgr2=0x7f0594157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05940ad850 0x7f0594157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 -- 172.21.15.87:0/561090011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940c8bb0 con 0x7f059415dce0 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f058b400640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05940ad850 0x7f0594157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0598c00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059415dce0 0x7f05941485e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f058c00c730 tx=0x7f058c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0590c00640 1 -- 172.21.15.87:0/561090011 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c003b80 con 0x7f059415dce0 2024-04-25T10:19:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0590c00640 1 -- 172.21.15.87:0/561090011 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c003d20 con 0x7f059415dce0 2024-04-25T10:19:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f0590c00640 1 -- 172.21.15.87:0/561090011 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00aa30 con 0x7f059415dce0 2024-04-25T10:19:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05940c8e90 con 0x7f059415dce0 2024-04-25T10:19:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05940c5b40 con 0x7f059415dce0 2024-04-25T10:19:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.540+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059404ef30 con 0x7f059415dce0 2024-04-25T10:19:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.544+0000 7f0590c00640 1 -- 172.21.15.87:0/561090011 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f058c00abd0 con 0x7f059415dce0 2024-04-25T10:19:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.544+0000 7f0590c00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0570066f60 0x7f0570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.544+0000 7f0590c00640 1 -- 172.21.15.87:0/561090011 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f058c084f50 con 0x7f059415dce0 2024-04-25T10:19:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.544+0000 7f0593e00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0570066f60 0x7f0570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.544+0000 7f0593e00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0570066f60 0x7f0570069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0584009b60 tx=0x7f0584009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:19.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.548+0000 7f0590c00640 1 -- 172.21.15.87:0/561090011 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f058c055e10 con 0x7f059415dce0 2024-04-25T10:19:19.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.700+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0594071630 con 0x7f059415dce0 2024-04-25T10:19:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.700+0000 7f0590c00640 1 -- 172.21.15.87:0/561090011 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16698 (secure 0 0 0) 0x7f058c055710 con 0x7f059415dce0 2024-04-25T10:19:19.705 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:19.705 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:06.154692+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:36.789308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:37.439968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0570066f60 msgr2=0x7f0570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0570066f60 0x7f0570069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0584009b60 tx=0x7f0584009190 comp rx=0 tx=0).stop 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059415dce0 msgr2=0x7f05941485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059415dce0 0x7f05941485e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f058c00c730 tx=0x7f058c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 shutdown_connections 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0570066f60 0x7f0570069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f059415dce0 0x7f05941485e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05940ad850 0x7f0594157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 --2- 172.21.15.87:0/561090011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f059406eec0 0x7f059415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 >> 172.21.15.87:0/561090011 conn(0x7f05940b2410 msgr2=0x7f05941603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:19.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 shutdown_connections 2024-04-25T10:19:19.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.704+0000 7f059ac00640 1 -- 172.21.15.87:0/561090011 wait complete. 2024-04-25T10:19:19.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:18:36.789308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:37.439968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:19.726 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:19.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.860+0000 7fa7f0800640 1 Processor -- start 2024-04-25T10:19:19.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7f0800640 1 -- start start 2024-04-25T10:19:19.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7f0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 0x7fa7e8166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7f0800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e8071360 con 0x7fa7e80aeda0 2024-04-25T10:19:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7f0800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e8159920 con 0x7fa7e8159540 2024-04-25T10:19:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7f0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e815c2f0 con 0x7fa7e80721f0 2024-04-25T10:19:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ed400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa7e80aeda0 0x7fa7e8071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58574/0 (socket says 172.21.15.87:58574) 2024-04-25T10:19:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ed400640 1 -- 172.21.15.87:0/1215953653 learned_addr learned my addr 172.21.15.87:0/1215953653 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:19.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ee800640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 0x7fa7e8166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135595804 0 0) 0x7fa7e8071360 con 0x7fa7e80aeda0 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d0003530 con 0x7fa7e80aeda0 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515484947 0 0) 0x7fa7e8159920 con 0x7fa7e8159540 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7e8071360 con 0x7fa7e8159540 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54122109 0 0) 0x7fa7d0003530 con 0x7fa7e80aeda0 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 msgr2=0x7fa7e8166920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 0x7fa7e8166920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 >> v1:172.21.15.169:6789/0 conn(0x7fa7e8159540 legacy=0x7fa7e80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e815c5d0 con 0x7fa7e80aeda0 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fa7d80016a0 con 0x7fa7e80aeda0 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 >> v1:172.21.15.87:6789/0 conn(0x7fa7e80aeda0 legacy=0x7fa7e8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7d0003b80 0x7fa7d0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0004810 0x7fa7d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 0x7fa7d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e815c2f0 con 0x7fa7d0003b80 2024-04-25T10:19:19.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000ba80 con 0x7fa7d0004810 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000bc50 con 0x7fa7e80721f0 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ede00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7d0003b80 0x7fa7d0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ed400640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0004810 0x7fa7d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ee800640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 0x7fa7d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ede00640 1 -- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 msgr2=0x7fa7d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ede00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e80721f0 0x7fa7d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ede00640 1 -- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0004810 msgr2=0x7fa7d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ede00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0004810 0x7fa7d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ede00640 1 -- 172.21.15.87:0/1215953653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d0003530 con 0x7fa7d0003b80 2024-04-25T10:19:19.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.864+0000 7fa7ed400640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0004810 0x7fa7d0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7ede00640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7d0003b80 0x7fa7d0003f80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0018d0 tx=0x7fa7dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72ee008b034cad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0b0070 con 0x7fa7d0003b80 2024-04-25T10:19:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc0052f0 con 0x7fa7d0003b80 2024-04-25T10:19:19.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7eca00640 1 -- 172.21.15.87:0/1215953653 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc001520 con 0x7fa7d0003b80 2024-04-25T10:19:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7f0800640 1 -- 172.21.15.87:0/1215953653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7d0003b80 msgr2=0x7fa7d0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7f0800640 1 --2- 172.21.15.87:0/1215953653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7d0003b80 0x7fa7d0003f80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0018d0 tx=0x7fa7dc01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7ee800640 1 -- 172.21.15.87:0/1215953653 reap_dead start 2024-04-25T10:19:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7f0800640 1 -- 172.21.15.87:0/1215953653 shutdown_connections 2024-04-25T10:19:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7f0800640 1 -- 172.21.15.87:0/1215953653 >> 172.21.15.87:0/1215953653 conn(0x7fa7e80669a0 msgr2=0x7fa7e8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7f0800640 1 -- 172.21.15.87:0/1215953653 shutdown_connections 2024-04-25T10:19:19.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.868+0000 7fa7f0800640 1 -- 172.21.15.87:0/1215953653 wait complete. 2024-04-25T10:19:19.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 Processor -- start 2024-04-25T10:19:19.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 -- start start 2024-04-25T10:19:19.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0003b80 0x7fa7e814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e8163e80 0x7fa7e8164300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e81f5e20 0x7fa7e81f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e8167130 con 0x7fa7e8163e80 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e806c0d0 con 0x7fa7d0003b80 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e81672b0 con 0x7fa7e81f5e20 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e8163e80 0x7fa7e8164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e8163e80 0x7fa7e8164300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37004/0 (socket says 172.21.15.87:37004) 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 -- 172.21.15.87:0/1036110750 learned_addr learned my addr 172.21.15.87:0/1036110750 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ee800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e81f5e20 0x7fa7e81f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ede00640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0003b80 0x7fa7e814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 -- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e81f5e20 msgr2=0x7fa7e81f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e81f5e20 0x7fa7e81f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 -- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0003b80 msgr2=0x7fa7e814f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:19.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0003b80 0x7fa7e814f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 -- 172.21.15.87:0/1036110750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e814f7e0 con 0x7fa7e8163e80 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ee800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e81f5e20 0x7fa7e81f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7ed400640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e8163e80 0x7fa7e8164300 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e7e0 tx=0x7fa7d800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7e2000640 1 -- 172.21.15.87:0/1036110750 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800ee90 con 0x7fa7e8163e80 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7e2000640 1 -- 172.21.15.87:0/1036110750 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8018070 con 0x7fa7e8163e80 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e81630c0 con 0x7fa7e8163e80 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7e2000640 1 -- 172.21.15.87:0/1036110750 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800cad0 con 0x7fa7e8163e80 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e8163680 con 0x7fa7e8163e80 2024-04-25T10:19:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.872+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e804ef30 con 0x7fa7e8163e80 2024-04-25T10:19:19.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.876+0000 7fa7e2000640 1 -- 172.21.15.87:0/1036110750 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7d800cc70 con 0x7fa7e8163e80 2024-04-25T10:19:19.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.876+0000 7fa7e2000640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc067130 0x7fa7cc0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:19.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.876+0000 7fa7e2000640 1 -- 172.21.15.87:0/1036110750 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa7d8084ba0 con 0x7fa7e8163e80 2024-04-25T10:19:19.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.876+0000 7fa7ede00640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc067130 0x7fa7cc0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:19.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.876+0000 7fa7ede00640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc067130 0x7fa7cc0695f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0ad000 tx=0x7fa7dc0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:19.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:19.880+0000 7fa7e2000640 1 -- 172.21.15.87:0/1036110750 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7d8055af0 con 0x7fa7e8163e80 2024-04-25T10:19:20.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.032+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7e806d7f0 con 0x7fa7e8163e80 2024-04-25T10:19:20.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.032+0000 7fa7e2000640 1 -- 172.21.15.87:0/1036110750 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7d80553f0 con 0x7fa7e8163e80 2024-04-25T10:19:20.037 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc067130 msgr2=0x7fa7cc0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc067130 0x7fa7cc0695f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0ad000 tx=0x7fa7dc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e8163e80 msgr2=0x7fa7e8164300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e8163e80 0x7fa7e8164300 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e7e0 tx=0x7fa7d800eca0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 shutdown_connections 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc067130 0x7fa7cc0695f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7e81f5e20 0x7fa7e81f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e8163e80 0x7fa7e8164300 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 --2- 172.21.15.87:0/1036110750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7d0003b80 0x7fa7e814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 >> 172.21.15.87:0/1036110750 conn(0x7fa7e80669a0 msgr2=0x7fa7e80cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 shutdown_connections 2024-04-25T10:19:20.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.036+0000 7fa7f0800640 1 -- 172.21.15.87:0/1036110750 wait complete. 2024-04-25T10:19:20.058 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:19:20.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38fb800640 1 Processor -- start 2024-04-25T10:19:20.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38fb800640 1 -- start start 2024-04-25T10:19:20.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38fb800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f38f4159540 0x7f38f40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38fb800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 0x7f38f4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38fb800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f4159920 con 0x7f38f40aebd0 2024-04-25T10:19:20.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38fb800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f415c2f0 con 0x7f38f4159540 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38fb800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f415c5d0 con 0x7f38f40721f0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38ebe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f38f40aebd0 0x7f38f4071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58586/0 (socket says 172.21.15.87:58586) 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38ebe00640 1 -- 172.21.15.87:0/3877863060 learned_addr learned my addr 172.21.15.87:0/3877863060 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f8e00640 1 --2- 172.21.15.87:0/3877863060 >> v2:172.21.15.169:3300/0 conn(0x7f38f4159540 0x7f38f40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f9800640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 0x7f38f4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526154855 0 0) 0x7f38f4159920 con 0x7f38f40aebd0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8003530 con 0x7f38f40aebd0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851934785 0 0) 0x7f38d8003530 con 0x7f38f40aebd0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 msgr2=0x7f38f4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 0x7f38f4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 >> v2:172.21.15.169:3300/0 conn(0x7f38f4159540 msgr2=0x7f38f40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 --2- 172.21.15.87:0/3877863060 >> v2:172.21.15.169:3300/0 conn(0x7f38f4159540 0x7f38f40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f415c8b0 con 0x7f38f40aebd0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f38dc001680 con 0x7f38f40aebd0 2024-04-25T10:19:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 >> v1:172.21.15.87:6789/0 conn(0x7f38f40aebd0 legacy=0x7f38f4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d80039c0 0x7f38d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 0x7f38d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d8004d50 0x7f38d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f415c2f0 con 0x7f38d8004d50 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f415c5d0 con 0x7f38d80039c0 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d800b9d0 con 0x7f38f40721f0 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f9800640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d8004d50 0x7f38d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f8e00640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d80039c0 0x7f38d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38ebe00640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 0x7f38d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f9800640 1 -- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 msgr2=0x7f38d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f9800640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 0x7f38d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.196+0000 7f38f9800640 1 -- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d80039c0 msgr2=0x7f38d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.200+0000 7f38f9800640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d80039c0 0x7f38d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.200+0000 7f38f9800640 1 -- 172.21.15.87:0/3877863060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8003530 con 0x7f38d8004d50 2024-04-25T10:19:20.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.200+0000 7f38ebe00640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40721f0 0x7f38d8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:20.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.200+0000 7f38f9800640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d8004d50 0x7f38d80051d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009fc0 tx=0x7f38ec01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75e3c181a016666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0b2070 con 0x7f38d8004d50 2024-04-25T10:19:20.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec0a5c90 con 0x7f38d8004d50 2024-04-25T10:19:20.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38f3e00640 1 -- 172.21.15.87:0/3877863060 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec009c10 con 0x7f38d8004d50 2024-04-25T10:19:20.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- 172.21.15.87:0/3877863060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d8004d50 msgr2=0x7f38d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 --2- 172.21.15.87:0/3877863060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d8004d50 0x7f38d80051d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009fc0 tx=0x7f38ec01cc80 comp rx=0 tx=0).stop 2024-04-25T10:19:20.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38f9800640 1 -- 172.21.15.87:0/3877863060 reap_dead start 2024-04-25T10:19:20.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- 172.21.15.87:0/3877863060 shutdown_connections 2024-04-25T10:19:20.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- 172.21.15.87:0/3877863060 >> 172.21.15.87:0/3877863060 conn(0x7f38f40669a0 msgr2=0x7f38f4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:20.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- 172.21.15.87:0/3877863060 shutdown_connections 2024-04-25T10:19:20.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- 172.21.15.87:0/3877863060 wait complete. 2024-04-25T10:19:20.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 Processor -- start 2024-04-25T10:19:20.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- start start 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d8004d50 0x7f38f40cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40ad840 0x7f38f40adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38f4147ae0 0x7f38f4147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f406c0d0 con 0x7f38f4147ae0 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f415bfa0 con 0x7f38d8004d50 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.204+0000 7f38fb800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4070350 con 0x7f38f40ad840 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38f4147ae0 0x7f38f4147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38f4147ae0 0x7f38f4147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37018/0 (socket says 172.21.15.87:37018) 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 -- 172.21.15.87:0/3988083677 learned_addr learned my addr 172.21.15.87:0/3988083677 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38ebe00640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40ad840 0x7f38f40adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f8e00640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d8004d50 0x7f38f40cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 -- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40ad840 msgr2=0x7f38f40adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40ad840 0x7f38f40adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 -- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d8004d50 msgr2=0x7f38f40cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d8004d50 0x7f38f40cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 -- 172.21.15.87:0/3988083677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4148970 con 0x7f38f4147ae0 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f9800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38f4147ae0 0x7f38f4147f60 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0b0040 tx=0x7f38ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f1600640 1 -- 172.21.15.87:0/3988083677 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0bd070 con 0x7f38f4147ae0 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f1600640 1 -- 172.21.15.87:0/3988083677 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec0a5a90 con 0x7f38f4147ae0 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f41f42f0 con 0x7f38f4147ae0 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f1600640 1 -- 172.21.15.87:0/3988083677 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0b2040 con 0x7f38f4147ae0 2024-04-25T10:19:20.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f41f48b0 con 0x7f38f4147ae0 2024-04-25T10:19:20.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f40ae200 con 0x7f38f4147ae0 2024-04-25T10:19:20.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f1600640 1 -- 172.21.15.87:0/3988083677 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38ec0c0070 con 0x7f38f4147ae0 2024-04-25T10:19:20.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.208+0000 7f38f1600640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38d0066f90 0x7f38d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.212+0000 7f38f1600640 1 -- 172.21.15.87:0/3988083677 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f38ec11ef90 con 0x7f38f4147ae0 2024-04-25T10:19:20.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.212+0000 7f38f8e00640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38d0066f90 0x7f38d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.212+0000 7f38f8e00640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38d0066f90 0x7f38d0069450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009be0 tx=0x7f38e4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.212+0000 7f38f1600640 1 -- 172.21.15.87:0/3988083677 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38ec016990 con 0x7f38f4147ae0 2024-04-25T10:19:20.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38f404ef30 con 0x7f38f4147ae0 2024-04-25T10:19:20.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38f1600640 1 -- 172.21.15.87:0/3988083677 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38ec0ef800 con 0x7f38f4147ae0 2024-04-25T10:19:20.375 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38d0066f90 msgr2=0x7f38d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38d0066f90 0x7f38d0069450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009be0 tx=0x7f38e4009190 comp rx=0 tx=0).stop 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38f4147ae0 msgr2=0x7f38f4147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38f4147ae0 0x7f38f4147f60 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0b0040 tx=0x7f38ec009750 comp rx=0 tx=0).stop 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 shutdown_connections 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38d0066f90 0x7f38d0069450 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38f4147ae0 0x7f38f4147f60 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f40ad840 0x7f38f40adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 --2- 172.21.15.87:0/3988083677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d8004d50 0x7f38f40cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.372+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 >> 172.21.15.87:0/3988083677 conn(0x7f38f40669a0 msgr2=0x7f38f415d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:20.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.376+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 shutdown_connections 2024-04-25T10:19:20.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.376+0000 7f38fb800640 1 -- 172.21.15.87:0/3988083677 wait complete. 2024-04-25T10:19:20.396 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:19:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.528+0000 7f992b200640 1 Processor -- start 2024-04-25T10:19:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.528+0000 7f992b200640 1 -- start start 2024-04-25T10:19:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 0x7f99240cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99240b4690 0x7f99240cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99240b2460 con 0x7f99240cd590 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99240b2d70 con 0x7f99240b4690 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9924070fb0 con 0x7f992415b750 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f99240cd590 0x7f99240b3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58594/0 (socket says 172.21.15.87:58594) 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929400640 1 -- 172.21.15.87:0/3156917580 learned_addr learned my addr 172.21.15.87:0/3156917580 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929e00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 0x7f99240cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992a800640 1 --2- 172.21.15.87:0/3156917580 >> v2:172.21.15.169:3300/0 conn(0x7f99240b4690 0x7f99240cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377060643 0 0) 0x7f99240b2460 con 0x7f99240cd590 2024-04-25T10:19:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f990c003530 con 0x7f99240cd590 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3662180131 0 0) 0x7f990c003530 con 0x7f99240cd590 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 msgr2=0x7f99240cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 0x7f99240cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 >> v2:172.21.15.169:3300/0 conn(0x7f99240b4690 msgr2=0x7f99240cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 --2- 172.21.15.87:0/3156917580 >> v2:172.21.15.169:3300/0 conn(0x7f99240b4690 0x7f99240cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240b2720 con 0x7f99240cd590 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f9914001680 con 0x7f99240cd590 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992a800640 1 --2- 172.21.15.87:0/3156917580 >> v2:172.21.15.169:3300/0 conn(0x7f99240b4690 0x7f99240cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 >> v1:172.21.15.87:6789/0 conn(0x7f99240cd590 legacy=0x7f99240b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f990c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 0x7f990c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f990c004d50 0x7f990c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b2d70 con 0x7f990c003a30 2024-04-25T10:19:20.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9924070fb0 con 0x7f990c004d50 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c00b9d0 con 0x7f992415b750 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929e00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f990c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f992a800640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f990c004d50 0x7f990c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929400640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 0x7f990c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929e00640 1 -- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 msgr2=0x7f990c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929e00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 0x7f990c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929e00640 1 -- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f990c004d50 msgr2=0x7f990c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929e00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f990c004d50 0x7f990c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929e00640 1 -- 172.21.15.87:0/3156917580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c003530 con 0x7f990c003a30 2024-04-25T10:19:20.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.532+0000 7f9929400640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992415b750 0x7f990c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:20.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f9929e00640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f990c003e50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9918009ae0 tx=0x7f991801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1688e0c68ec8a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180b2070 con 0x7f990c003a30 2024-04-25T10:19:20.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99180a5520 con 0x7f990c003a30 2024-04-25T10:19:20.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f9928a00640 1 -- 172.21.15.87:0/3156917580 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180096b0 con 0x7f990c003a30 2024-04-25T10:19:20.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 -- 172.21.15.87:0/3156917580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 msgr2=0x7f990c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 --2- 172.21.15.87:0/3156917580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f990c003e50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9918009ae0 tx=0x7f991801ca40 comp rx=0 tx=0).stop 2024-04-25T10:19:20.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992a800640 1 -- 172.21.15.87:0/3156917580 reap_dead start 2024-04-25T10:19:20.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 -- 172.21.15.87:0/3156917580 shutdown_connections 2024-04-25T10:19:20.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 -- 172.21.15.87:0/3156917580 >> 172.21.15.87:0/3156917580 conn(0x7f99240743f0 msgr2=0x7f99240669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:20.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 -- 172.21.15.87:0/3156917580 shutdown_connections 2024-04-25T10:19:20.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 -- 172.21.15.87:0/3156917580 wait complete. 2024-04-25T10:19:20.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 Processor -- start 2024-04-25T10:19:20.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.536+0000 7f992b200640 1 -- start start 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f99241fae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992414bfe0 0x7f992413fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99240ad840 0x7f99240adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9924070ce0 con 0x7f990c003a30 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9924070b60 con 0x7f99240ad840 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b39f0 con 0x7f992414bfe0 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f99241fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f99241fae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37026/0 (socket says 172.21.15.87:37026) 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 -- 172.21.15.87:0/3123243369 learned_addr learned my addr 172.21.15.87:0/3123243369 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992a800640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99240ad840 0x7f99240adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929400640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992414bfe0 0x7f992413fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 -- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992414bfe0 msgr2=0x7f992413fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992414bfe0 0x7f992413fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 -- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99240ad840 msgr2=0x7f99240adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99240ad840 0x7f99240adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 -- 172.21.15.87:0/3123243369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240b1960 con 0x7f990c003a30 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929400640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992414bfe0 0x7f992413fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f9929e00640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f99241fae50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9918009ae0 tx=0x7f990c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f991e000640 1 -- 172.21.15.87:0/3123243369 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180b2040 con 0x7f990c003a30 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f991e000640 1 -- 172.21.15.87:0/3123243369 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991800f000 con 0x7f990c003a30 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99240b1c40 con 0x7f990c003a30 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f991e000640 1 -- 172.21.15.87:0/3123243369 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918004020 con 0x7f990c003a30 2024-04-25T10:19:20.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9924140310 con 0x7f990c003a30 2024-04-25T10:19:20.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9924075120 con 0x7f990c003a30 2024-04-25T10:19:20.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.540+0000 7f991e000640 1 -- 172.21.15.87:0/3123243369 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99180c0030 con 0x7f990c003a30 2024-04-25T10:19:20.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.544+0000 7f991e000640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.544+0000 7f991e000640 1 -- 172.21.15.87:0/3123243369 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f991811ebf0 con 0x7f990c003a30 2024-04-25T10:19:20.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.544+0000 7f9929400640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.544+0000 7f9929400640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9924140670 tx=0x7f9914009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.548+0000 7f991e000640 1 -- 172.21.15.87:0/3123243369 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99180c7060 con 0x7f990c003a30 2024-04-25T10:19:20.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9924151790 con 0x7f990c003a30 2024-04-25T10:19:20.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f991e000640 1 -- 172.21.15.87:0/3123243369 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9918016780 con 0x7f990c003a30 2024-04-25T10:19:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:19:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 msgr2=0x7f9908069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9924140670 tx=0x7f9914009040 comp rx=0 tx=0).stop 2024-04-25T10:19:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 msgr2=0x7f99241fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f99241fae50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9918009ae0 tx=0x7f990c005840 comp rx=0 tx=0).stop 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 shutdown_connections 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9908066f60 0x7f9908069420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99240ad840 0x7f99240adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f992414bfe0 0x7f992413fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 --2- 172.21.15.87:0/3123243369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f990c003a30 0x7f99241fae50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 >> 172.21.15.87:0/3123243369 conn(0x7f99240743f0 msgr2=0x7f992415c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 shutdown_connections 2024-04-25T10:19:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.700+0000 7f992b200640 1 -- 172.21.15.87:0/3123243369 wait complete. 2024-04-25T10:19:20.724 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:19:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 Processor -- start 2024-04-25T10:19:20.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 -- start start 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa8040cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8040cfe70 0x7fa80415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa80415b740 0x7fa80406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa804071050 con 0x7fa80415b740 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8040712f0 con 0x7fa8040cfe70 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.856+0000 7fa80c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040715d0 con 0x7fa8041550b0 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa80415b740 0x7fa80406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa809a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa8040cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa80415b740 0x7fa80406d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37028/0 (socket says 172.21.15.87:37028) 2024-04-25T10:19:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 -- 172.21.15.87:0/2982826112 learned_addr learned my addr 172.21.15.87:0/2982826112 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa809000640 1 --2- 172.21.15.87:0/2982826112 >> v2:172.21.15.169:3300/0 conn(0x7fa8040cfe70 0x7fa80415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 -- 172.21.15.87:0/2982826112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 msgr2=0x7fa8040cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 --2- 172.21.15.87:0/2982826112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa8040cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 -- 172.21.15.87:0/2982826112 >> v2:172.21.15.169:3300/0 conn(0x7fa8040cfe70 msgr2=0x7fa80415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 --2- 172.21.15.87:0/2982826112 >> v2:172.21.15.169:3300/0 conn(0x7fa8040cfe70 0x7fa80415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 -- 172.21.15.87:0/2982826112 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8040718b0 con 0x7fa80415b740 2024-04-25T10:19:20.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa80a400640 1 --2- 172.21.15.87:0/2982826112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80415b740 0x7fa80406d1a0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa80406e290 tx=0x7fa7fc01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b7f401ab09f3d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa803e00640 1 -- 172.21.15.87:0/2982826112 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0ac020 con 0x7fa80415b740 2024-04-25T10:19:20.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa803e00640 1 -- 172.21.15.87:0/2982826112 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc0a6440 con 0x7fa80415b740 2024-04-25T10:19:20.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.860+0000 7fa803e00640 1 -- 172.21.15.87:0/2982826112 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0a6810 con 0x7fa80415b740 2024-04-25T10:19:20.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- 172.21.15.87:0/2982826112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80415b740 msgr2=0x7fa80406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 --2- 172.21.15.87:0/2982826112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80415b740 0x7fa80406d1a0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa80406e290 tx=0x7fa7fc01c730 comp rx=0 tx=0).stop 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- 172.21.15.87:0/2982826112 shutdown_connections 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 --2- 172.21.15.87:0/2982826112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80415b740 0x7fa80406d1a0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 --2- 172.21.15.87:0/2982826112 >> v2:172.21.15.169:3300/0 conn(0x7fa8040cfe70 0x7fa80415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 --2- 172.21.15.87:0/2982826112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa8040cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- 172.21.15.87:0/2982826112 >> 172.21.15.87:0/2982826112 conn(0x7fa804066b70 msgr2=0x7fa804066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- 172.21.15.87:0/2982826112 shutdown_connections 2024-04-25T10:19:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- 172.21.15.87:0/2982826112 wait complete. 2024-04-25T10:19:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 Processor -- start 2024-04-25T10:19:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- start start 2024-04-25T10:19:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8040cfe70 0x7fa80413a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa80414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80415b740 0x7fa80415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040b33e0 con 0x7fa8040cfe70 2024-04-25T10:19:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040b3260 con 0x7fa80415b740 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040b3560 con 0x7fa8041550b0 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8040cfe70 0x7fa80413a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8040cfe70 0x7fa80413a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37032/0 (socket says 172.21.15.87:37032) 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 -- 172.21.15.87:0/4028435229 learned_addr learned my addr 172.21.15.87:0/4028435229 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa80a400640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80415b740 0x7fa80415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809000640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa80414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 -- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 msgr2=0x7fa80414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa80414a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 -- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80415b740 msgr2=0x7fa80415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80415b740 0x7fa80415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809a00640 1 -- 172.21.15.87:0/4028435229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa804159560 con 0x7fa8040cfe70 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.864+0000 7fa809000640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa80414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa809a00640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8040cfe70 0x7fa80413a130 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e960 tx=0x7fa7f800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa801600640 1 -- 172.21.15.87:0/4028435229 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8017070 con 0x7fa8040cfe70 2024-04-25T10:19:20.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa801600640 1 -- 172.21.15.87:0/4028435229 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f800ca00 con 0x7fa8040cfe70 2024-04-25T10:19:20.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa801600640 1 -- 172.21.15.87:0/4028435229 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800cdb0 con 0x7fa8040cfe70 2024-04-25T10:19:20.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80414b040 con 0x7fa8040cfe70 2024-04-25T10:19:20.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa804159ae0 con 0x7fa8040cfe70 2024-04-25T10:19:20.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80404ef30 con 0x7fa8040cfe70 2024-04-25T10:19:20.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa801600640 1 -- 172.21.15.87:0/4028435229 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7f801a070 con 0x7fa8040cfe70 2024-04-25T10:19:20.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa801600640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e4066f60 0x7fa7e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:20.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa801600640 1 -- 172.21.15.87:0/4028435229 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa7f8084ff0 con 0x7fa8040cfe70 2024-04-25T10:19:20.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.868+0000 7fa809000640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e4066f60 0x7fa7e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:20.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.872+0000 7fa809000640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e4066f60 0x7fa7e4069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa80414f550 tx=0x7fa7f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:20.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:20.872+0000 7fa801600640 1 -- 172.21.15.87:0/4028435229 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7f8055930 con 0x7fa8040cfe70 2024-04-25T10:19:21.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa8040aed20 con 0x7fa8040cfe70 2024-04-25T10:19:21.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa801600640 1 -- 172.21.15.87:0/4028435229 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa7f80599d0 con 0x7fa8040cfe70 2024-04-25T10:19:21.043 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e4066f60 msgr2=0x7fa7e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e4066f60 0x7fa7e4069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa80414f550 tx=0x7fa7f0008040 comp rx=0 tx=0).stop 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8040cfe70 msgr2=0x7fa80413a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8040cfe70 0x7fa80413a130 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e960 tx=0x7fa7f800ee20 comp rx=0 tx=0).stop 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 shutdown_connections 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7e4066f60 0x7fa7e4069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa80415b740 0x7fa80415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8041550b0 0x7fa80414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 --2- 172.21.15.87:0/4028435229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8040cfe70 0x7fa80413a130 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.040+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 >> 172.21.15.87:0/4028435229 conn(0x7fa804066b70 msgr2=0x7fa8040ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:21.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.044+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 shutdown_connections 2024-04-25T10:19:21.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.044+0000 7fa80c400640 1 -- 172.21.15.87:0/4028435229 wait complete. 2024-04-25T10:19:21.063 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:19:21.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:21.063 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:21.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:19:21.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:21.751 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:19:21.760 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:19:21.760 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:21.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:21.769 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:19:21.769 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:21.808 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:21.809 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:19:21.809 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:21.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:21.825 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f008cc00640 1 Processor -- start 2024-04-25T10:19:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f008cc00640 1 -- start start 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f008cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00881550b0 0x7f00880cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f008cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f00880b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f008cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0088071050 con 0x7f00880cfe70 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f008cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008815bda0 con 0x7f00881550b0 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f008cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008815d040 con 0x7f008815b910 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0085e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00881550b0 0x7f00880cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0085400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f00880cfe70 0x7f008815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58604/0 (socket says 172.21.15.87:58604) 2024-04-25T10:19:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0085400640 1 -- 172.21.15.87:0/1165774212 learned_addr learned my addr 172.21.15.87:0/1165774212 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:21.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0086800640 1 --2- 172.21.15.87:0/1165774212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f00880b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:21.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0085e00640 1 -- 172.21.15.87:0/1165774212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 msgr2=0x7f00880b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:21.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0085e00640 1 --2- 172.21.15.87:0/1165774212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f00880b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0085e00640 1 -- 172.21.15.87:0/1165774212 >> v1:172.21.15.87:6789/0 conn(0x7f00880cfe70 legacy=0x7f008815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:21.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0085e00640 1 -- 172.21.15.87:0/1165774212 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880712f0 con 0x7f00881550b0 2024-04-25T10:19:21.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.984+0000 7f0084a00640 1 -- 172.21.15.87:0/1165774212 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560436626 0 0) 0x7f0088071050 con 0x7f00880cfe70 2024-04-25T10:19:21.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f0085e00640 1 --2- 172.21.15.87:0/1165774212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00881550b0 0x7f00880cf930 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0078009dc0 tx=0x7f007801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67754c7aee0cbeec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f0084a00640 1 -- 172.21.15.87:0/1165774212 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780b2070 con 0x7f00881550b0 2024-04-25T10:19:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f0084a00640 1 -- 172.21.15.87:0/1165774212 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00780a5900 con 0x7f00881550b0 2024-04-25T10:19:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f0084a00640 1 -- 172.21.15.87:0/1165774212 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780a5cd0 con 0x7f00881550b0 2024-04-25T10:19:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 -- 172.21.15.87:0/1165774212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00881550b0 msgr2=0x7f00880cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 --2- 172.21.15.87:0/1165774212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00881550b0 0x7f00880cf930 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0078009dc0 tx=0x7f007801c950 comp rx=0 tx=0).stop 2024-04-25T10:19:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 -- 172.21.15.87:0/1165774212 shutdown_connections 2024-04-25T10:19:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 --2- 172.21.15.87:0/1165774212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f00880b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 --2- 172.21.15.87:0/1165774212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00881550b0 0x7f00880cf930 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 -- 172.21.15.87:0/1165774212 >> 172.21.15.87:0/1165774212 conn(0x7f0088066b70 msgr2=0x7f0088066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 -- 172.21.15.87:0/1165774212 shutdown_connections 2024-04-25T10:19:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.988+0000 7f008cc00640 1 -- 172.21.15.87:0/1165774212 wait complete. 2024-04-25T10:19:21.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 Processor -- start 2024-04-25T10:19:21.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 -- start start 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00880cfe70 0x7f0088155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00881550b0 0x7f0088148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f008815aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008815c030 con 0x7f00880cfe70 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880720b0 con 0x7f00881550b0 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008815c1b0 con 0x7f008815b910 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00880cfe70 0x7f0088155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00880cfe70 0x7f0088155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37054/0 (socket says 172.21.15.87:37054) 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 -- 172.21.15.87:0/1140934522 learned_addr learned my addr 172.21.15.87:0/1140934522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085400640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00881550b0 0x7f0088148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0086800640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f008815aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 -- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 msgr2=0x7f008815aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:21.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f008815aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 -- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00881550b0 msgr2=0x7f0088148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00881550b0 0x7f0088148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 -- 172.21.15.87:0/1140934522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880673c0 con 0x7f00880cfe70 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085400640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00881550b0 0x7f0088148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f0085e00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00880cfe70 0x7f0088155490 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0078009dc0 tx=0x7f00780196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f007e000640 1 -- 172.21.15.87:0/1140934522 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780b2040 con 0x7f00880cfe70 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f007e000640 1 -- 172.21.15.87:0/1140934522 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0078003ba0 con 0x7f00880cfe70 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f007e000640 1 -- 172.21.15.87:0/1140934522 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078004020 con 0x7f00880cfe70 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00881559d0 con 0x7f00880cfe70 2024-04-25T10:19:21.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.992+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00880b1e50 con 0x7f00880cfe70 2024-04-25T10:19:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.996+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008804ef30 con 0x7f00880cfe70 2024-04-25T10:19:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.996+0000 7f007e000640 1 -- 172.21.15.87:0/1140934522 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00780c2060 con 0x7f00880cfe70 2024-04-25T10:19:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.996+0000 7f007e000640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0068066f60 0x7f0068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.996+0000 7f007e000640 1 -- 172.21.15.87:0/1140934522 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f007811f580 con 0x7f00880cfe70 2024-04-25T10:19:21.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.996+0000 7f0085400640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0068066f60 0x7f0068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:21.996+0000 7f0085400640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0068066f60 0x7f0068069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f008815e2c0 tx=0x7f0074007c20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.000+0000 7f007e000640 1 -- 172.21.15.87:0/1140934522 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0078016540 con 0x7f00880cfe70 2024-04-25T10:19:22.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.188+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f008814fb20 con 0x7f00880cfe70 2024-04-25T10:19:22.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.188+0000 7f007e000640 1 -- 172.21.15.87:0/1140934522 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f00780f04d0 con 0x7f00880cfe70 2024-04-25T10:19:22.192 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:22.193 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:22.193 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:22.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0068066f60 msgr2=0x7f0068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0068066f60 0x7f0068069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f008815e2c0 tx=0x7f0074007c20 comp rx=0 tx=0).stop 2024-04-25T10:19:22.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00880cfe70 msgr2=0x7f0088155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00880cfe70 0x7f0088155490 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0078009dc0 tx=0x7f00780196f0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 shutdown_connections 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0068066f60 0x7f0068069420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008815b910 0x7f008815aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00881550b0 0x7f0088148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 --2- 172.21.15.87:0/1140934522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00880cfe70 0x7f0088155490 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 >> 172.21.15.87:0/1140934522 conn(0x7f0088066b70 msgr2=0x7f008815d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 shutdown_connections 2024-04-25T10:19:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.192+0000 7f008cc00640 1 -- 172.21.15.87:0/1140934522 wait complete. 2024-04-25T10:19:22.214 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:22.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b57400640 1 Processor -- start 2024-04-25T10:19:22.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b57400640 1 -- start start 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b57400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b5015b9e0 0x7f8b500cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b57400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b500d1c30 0x7f8b500cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b57400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b500ccfb0 con 0x7f8b5015bdc0 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b57400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500cd290 con 0x7f8b5015b9e0 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b57400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500cd570 con 0x7f8b500d1c30 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b47e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8b5015bdc0 0x7f8b500d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58606/0 (socket says 172.21.15.87:58606) 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b54a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b5015b9e0 0x7f8b500cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b47e00640 1 -- 172.21.15.87:0/3925753716 learned_addr learned my addr 172.21.15.87:0/3925753716 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:22.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b55400640 1 --2- 172.21.15.87:0/3925753716 >> v2:172.21.15.169:3300/0 conn(0x7f8b500d1c30 0x7f8b500cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b54a00640 1 -- 172.21.15.87:0/3925753716 >> v2:172.21.15.169:3300/0 conn(0x7f8b500d1c30 msgr2=0x7f8b500cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:22.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b54a00640 1 --2- 172.21.15.87:0/3925753716 >> v2:172.21.15.169:3300/0 conn(0x7f8b500d1c30 0x7f8b500cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b54a00640 1 -- 172.21.15.87:0/3925753716 >> v1:172.21.15.87:6789/0 conn(0x7f8b5015bdc0 legacy=0x7f8b500d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b54a00640 1 -- 172.21.15.87:0/3925753716 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500cd850 con 0x7f8b5015b9e0 2024-04-25T10:19:22.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b4fe00640 1 -- 172.21.15.87:0/3925753716 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159897149 0 0) 0x7f8b500ccfb0 con 0x7f8b5015bdc0 2024-04-25T10:19:22.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.348+0000 7f8b55400640 1 --2- 172.21.15.87:0/3925753716 >> v2:172.21.15.169:3300/0 conn(0x7f8b500d1c30 0x7f8b500cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:22.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b54a00640 1 --2- 172.21.15.87:0/3925753716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5015b9e0 0x7f8b500cc410 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8b50070e60 tx=0x7f8b4001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5baf33512b012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:22.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b4fe00640 1 -- 172.21.15.87:0/3925753716 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b0070 con 0x7f8b5015b9e0 2024-04-25T10:19:22.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b4fe00640 1 -- 172.21.15.87:0/3925753716 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b400ab600 con 0x7f8b5015b9e0 2024-04-25T10:19:22.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b4fe00640 1 -- 172.21.15.87:0/3925753716 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400ab9d0 con 0x7f8b5015b9e0 2024-04-25T10:19:22.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 -- 172.21.15.87:0/3925753716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5015b9e0 msgr2=0x7f8b500cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 --2- 172.21.15.87:0/3925753716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5015b9e0 0x7f8b500cc410 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8b50070e60 tx=0x7f8b4001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 -- 172.21.15.87:0/3925753716 shutdown_connections 2024-04-25T10:19:22.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 --2- 172.21.15.87:0/3925753716 >> v2:172.21.15.169:3300/0 conn(0x7f8b500d1c30 0x7f8b500cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 --2- 172.21.15.87:0/3925753716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5015b9e0 0x7f8b500cc410 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 -- 172.21.15.87:0/3925753716 >> 172.21.15.87:0/3925753716 conn(0x7f8b5009a9d0 msgr2=0x7f8b5009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:22.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 -- 172.21.15.87:0/3925753716 shutdown_connections 2024-04-25T10:19:22.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.352+0000 7f8b57400640 1 -- 172.21.15.87:0/3925753716 wait complete. 2024-04-25T10:19:22.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 Processor -- start 2024-04-25T10:19:22.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 -- start start 2024-04-25T10:19:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b500d1c30 0x7f8b501f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b5015b9e0 0x7f8b501f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5015bdc0 0x7f8b500b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500c6630 con 0x7f8b500d1c30 2024-04-25T10:19:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500c64b0 con 0x7f8b5015bdc0 2024-04-25T10:19:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500c67b0 con 0x7f8b5015b9e0 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b500d1c30 0x7f8b501f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b500d1c30 0x7f8b501f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37070/0 (socket says 172.21.15.87:37070) 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 -- 172.21.15.87:0/935391985 learned_addr learned my addr 172.21.15.87:0/935391985 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b47e00640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b5015b9e0 0x7f8b501f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b55400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5015bdc0 0x7f8b500b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 -- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b5015b9e0 msgr2=0x7f8b501f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b5015b9e0 0x7f8b501f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 -- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5015bdc0 msgr2=0x7f8b500b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5015bdc0 0x7f8b500b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 -- 172.21.15.87:0/935391985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500b4ee0 con 0x7f8b500d1c30 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b47e00640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b5015b9e0 0x7f8b501f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b55400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5015bdc0 0x7f8b500b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b54a00640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b500d1c30 0x7f8b501f7620 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8b400ae040 tx=0x7f8b40019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b4d600640 1 -- 172.21.15.87:0/935391985 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b8020 con 0x7f8b500d1c30 2024-04-25T10:19:22.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b4d600640 1 -- 172.21.15.87:0/935391985 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b400b4630 con 0x7f8b500d1c30 2024-04-25T10:19:22.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b4d600640 1 -- 172.21.15.87:0/935391985 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b0040 con 0x7f8b500d1c30 2024-04-25T10:19:22.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b50158db0 con 0x7f8b500d1c30 2024-04-25T10:19:22.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b50159090 con 0x7f8b500d1c30 2024-04-25T10:19:22.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.356+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b500b53e0 con 0x7f8b500d1c30 2024-04-25T10:19:22.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.360+0000 7f8b4d600640 1 -- 172.21.15.87:0/935391985 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b400bb020 con 0x7f8b500d1c30 2024-04-25T10:19:22.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.360+0000 7f8b4d600640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b2c066f90 0x7f8b2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.360+0000 7f8b4d600640 1 -- 172.21.15.87:0/935391985 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8b4011ee40 con 0x7f8b500d1c30 2024-04-25T10:19:22.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.360+0000 7f8b47e00640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b2c066f90 0x7f8b2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.360+0000 7f8b47e00640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b2c066f90 0x7f8b2c069450 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8b38007fd0 tx=0x7f8b38009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:22.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.364+0000 7f8b4d600640 1 -- 172.21.15.87:0/935391985 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b400c2030 con 0x7f8b500d1c30 2024-04-25T10:19:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.520+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b500cf520 con 0x7f8b500d1c30 2024-04-25T10:19:22.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.520+0000 7f8b4d600640 1 -- 172.21.15.87:0/935391985 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16698 (secure 0 0 0) 0x7f8b40016070 con 0x7f8b500d1c30 2024-04-25T10:19:22.523 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:22.523 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:06.154692+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:36.789308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:37.439968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:22.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.520+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b2c066f90 msgr2=0x7f8b2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.520+0000 7f8b57400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b2c066f90 0x7f8b2c069450 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8b38007fd0 tx=0x7f8b38009040 comp rx=0 tx=0).stop 2024-04-25T10:19:22.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.520+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b500d1c30 msgr2=0x7f8b501f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.520+0000 7f8b57400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b500d1c30 0x7f8b501f7620 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8b400ae040 tx=0x7f8b40019190 comp rx=0 tx=0).stop 2024-04-25T10:19:22.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 shutdown_connections 2024-04-25T10:19:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b2c066f90 0x7f8b2c069450 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5015bdc0 0x7f8b500b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b5015b9e0 0x7f8b501f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 --2- 172.21.15.87:0/935391985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b500d1c30 0x7f8b501f7620 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 >> 172.21.15.87:0/935391985 conn(0x7f8b5009a9d0 msgr2=0x7f8b5006fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 shutdown_connections 2024-04-25T10:19:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.524+0000 7f8b57400640 1 -- 172.21.15.87:0/935391985 wait complete. 2024-04-25T10:19:22.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:18:36.789308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:37.439968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:22.545 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:22.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.680+0000 7f9e00600640 1 Processor -- start 2024-04-25T10:19:22.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.680+0000 7f9e00600640 1 -- start start 2024-04-25T10:19:22.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9e00600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9df80b3480 0x7f9df815e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9e00600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df80ad840 con 0x7f9df8071bc0 2024-04-25T10:19:22.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9e00600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df815e710 con 0x7f9df80cfb50 2024-04-25T10:19:22.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9e00600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df815bb90 con 0x7f9df80b3480 2024-04-25T10:19:22.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9dfe600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9df8071bc0 0x7f9df80add20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58618/0 (socket says 172.21.15.87:58618) 2024-04-25T10:19:22.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9dfe600640 1 -- 172.21.15.87:0/114631499 learned_addr learned my addr 172.21.15.87:0/114631499 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:22.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9dfdc00640 1 --2- 172.21.15.87:0/114631499 >> v2:172.21.15.169:3300/0 conn(0x7f9df80b3480 0x7f9df815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9df7e00640 1 -- 172.21.15.87:0/114631499 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406188247 0 0) 0x7f9df80ad840 con 0x7f9df8071bc0 2024-04-25T10:19:22.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9df7e00640 1 -- 172.21.15.87:0/114631499 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de0003530 con 0x7f9df8071bc0 2024-04-25T10:19:22.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9df7e00640 1 -- 172.21.15.87:0/114631499 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319211844 0 0) 0x7f9df815e710 con 0x7f9df80cfb50 2024-04-25T10:19:22.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9df7e00640 1 -- 172.21.15.87:0/114631499 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df80ad840 con 0x7f9df80cfb50 2024-04-25T10:19:22.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9dfdc00640 1 -- 172.21.15.87:0/114631499 >> v1:172.21.15.169:6789/0 conn(0x7f9df80cfb50 legacy=0x7f9df8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9dfdc00640 1 -- 172.21.15.87:0/114631499 >> v1:172.21.15.87:6789/0 conn(0x7f9df8071bc0 legacy=0x7f9df80add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.684+0000 7f9dfdc00640 1 -- 172.21.15.87:0/114631499 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df815e9f0 con 0x7f9df80b3480 2024-04-25T10:19:22.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9dfdc00640 1 --2- 172.21.15.87:0/114631499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815e130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009d90 tx=0x7f9dec01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f68837daf80f54c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:22.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9df7e00640 1 -- 172.21.15.87:0/114631499 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0b2070 con 0x7f9df80b3480 2024-04-25T10:19:22.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9df7e00640 1 -- 172.21.15.87:0/114631499 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec0a5830 con 0x7f9df80b3480 2024-04-25T10:19:22.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9df7e00640 1 -- 172.21.15.87:0/114631499 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0a5c00 con 0x7f9df80b3480 2024-04-25T10:19:22.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 -- 172.21.15.87:0/114631499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 msgr2=0x7f9df815e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 --2- 172.21.15.87:0/114631499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815e130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009d90 tx=0x7f9dec01c970 comp rx=0 tx=0).stop 2024-04-25T10:19:22.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 -- 172.21.15.87:0/114631499 shutdown_connections 2024-04-25T10:19:22.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 --2- 172.21.15.87:0/114631499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815e130 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 -- 172.21.15.87:0/114631499 >> 172.21.15.87:0/114631499 conn(0x7f9df80669a0 msgr2=0x7f9df8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:22.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 -- 172.21.15.87:0/114631499 shutdown_connections 2024-04-25T10:19:22.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 -- 172.21.15.87:0/114631499 wait complete. 2024-04-25T10:19:22.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 Processor -- start 2024-04-25T10:19:22.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 -- start start 2024-04-25T10:19:22.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9df8071bc0 0x7f9df80af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.688+0000 7f9e00600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9e00600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df80cfb50 0x7f9df815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9e00600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80723e0 con 0x7f9df8071bc0 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9e00600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80ae840 con 0x7f9df80b3480 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9e00600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8152960 con 0x7f9df80cfb50 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9df8071bc0 0x7f9df80af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9df8071bc0 0x7f9df80af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37072/0 (socket says 172.21.15.87:37072) 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 -- 172.21.15.87:0/2056361667 learned_addr learned my addr 172.21.15.87:0/2056361667 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 -- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df80cfb50 msgr2=0x7f9df815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df80cfb50 0x7f9df815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 -- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 msgr2=0x7f9df815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfd200640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 -- 172.21.15.87:0/2056361667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8152e60 con 0x7f9df8071bc0 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfd200640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9dfdc00640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9df8071bc0 0x7f9df80af540 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9dec01a5a0 tx=0x7f9dec019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:22.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9df5600640 1 -- 172.21.15.87:0/2056361667 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0bd020 con 0x7f9df8071bc0 2024-04-25T10:19:22.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9df5600640 1 -- 172.21.15.87:0/2056361667 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec0b3040 con 0x7f9df8071bc0 2024-04-25T10:19:22.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df8153140 con 0x7f9df8071bc0 2024-04-25T10:19:22.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9df5600640 1 -- 172.21.15.87:0/2056361667 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0b2070 con 0x7f9df8071bc0 2024-04-25T10:19:22.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df8153700 con 0x7f9df8071bc0 2024-04-25T10:19:22.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df804ef30 con 0x7f9df8071bc0 2024-04-25T10:19:22.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.692+0000 7f9df5600640 1 -- 172.21.15.87:0/2056361667 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dec0c0020 con 0x7f9df8071bc0 2024-04-25T10:19:22.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.696+0000 7f9df5600640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dd8066f60 0x7f9dd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:22.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.696+0000 7f9df5600640 1 -- 172.21.15.87:0/2056361667 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9dec11fde0 con 0x7f9df8071bc0 2024-04-25T10:19:22.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.696+0000 7f9dfd200640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dd8066f60 0x7f9dd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:22.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.696+0000 7f9dfd200640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dd8066f60 0x7f9dd8069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9df80b0810 tx=0x7f9de4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:22.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.700+0000 7f9df5600640 1 -- 172.21.15.87:0/2056361667 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dec0c7030 con 0x7f9df8071bc0 2024-04-25T10:19:22.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.852+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9df806cca0 con 0x7f9df8071bc0 2024-04-25T10:19:22.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.852+0000 7f9df5600640 1 -- 172.21.15.87:0/2056361667 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9dec016560 con 0x7f9df8071bc0 2024-04-25T10:19:22.856 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dd8066f60 msgr2=0x7f9dd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dd8066f60 0x7f9dd8069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9df80b0810 tx=0x7f9de4009040 comp rx=0 tx=0).stop 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9df8071bc0 msgr2=0x7f9df80af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9df8071bc0 0x7f9df80af540 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9dec01a5a0 tx=0x7f9dec019710 comp rx=0 tx=0).stop 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 shutdown_connections 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9dd8066f60 0x7f9dd8069420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9df80cfb50 0x7f9df815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df80b3480 0x7f9df815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 --2- 172.21.15.87:0/2056361667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9df8071bc0 0x7f9df80af540 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 >> 172.21.15.87:0/2056361667 conn(0x7f9df80669a0 msgr2=0x7f9df806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 shutdown_connections 2024-04-25T10:19:22.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:22.856+0000 7f9e00600640 1 -- 172.21.15.87:0/2056361667 wait complete. 2024-04-25T10:19:22.877 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:19:23.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f303d200640 1 Processor -- start 2024-04-25T10:19:23.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f303d200640 1 -- start start 2024-04-25T10:19:23.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f303d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30380aef80 0x7f303815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f303d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f30380b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f303d200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303815b080 con 0x7f3038070350 2024-04-25T10:19:23.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f303d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303815adb0 con 0x7f30380aef80 2024-04-25T10:19:23.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f303d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303815eb60 con 0x7f30380b31c0 2024-04-25T10:19:23.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f302d400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3038070350 0x7f303815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58622/0 (socket says 172.21.15.87:58622) 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.012+0000 7f302d400640 1 -- 172.21.15.87:0/2417396536 learned_addr learned my addr 172.21.15.87:0/2417396536 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3036800640 1 --2- 172.21.15.87:0/2417396536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f30380b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3035e00640 1 --2- 172.21.15.87:0/2417396536 >> v2:172.21.15.169:3300/0 conn(0x7f30380aef80 0x7f303815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3035400640 1 -- 172.21.15.87:0/2417396536 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032195394 0 0) 0x7f303815b080 con 0x7f3038070350 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3035400640 1 -- 172.21.15.87:0/2417396536 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f301c003530 con 0x7f3038070350 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3036800640 1 -- 172.21.15.87:0/2417396536 >> v2:172.21.15.169:3300/0 conn(0x7f30380aef80 msgr2=0x7f303815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3036800640 1 --2- 172.21.15.87:0/2417396536 >> v2:172.21.15.169:3300/0 conn(0x7f30380aef80 0x7f303815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3036800640 1 -- 172.21.15.87:0/2417396536 >> v1:172.21.15.87:6789/0 conn(0x7f3038070350 legacy=0x7f303815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3036800640 1 -- 172.21.15.87:0/2417396536 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303815ee40 con 0x7f30380b31c0 2024-04-25T10:19:23.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3036800640 1 --2- 172.21.15.87:0/2417396536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f30380b3610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3030009d90 tx=0x7f303001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8eeba6cde84be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3035400640 1 -- 172.21.15.87:0/2417396536 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300b2070 con 0x7f30380b31c0 2024-04-25T10:19:23.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3035400640 1 -- 172.21.15.87:0/2417396536 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30300a5940 con 0x7f30380b31c0 2024-04-25T10:19:23.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.016+0000 7f3035400640 1 -- 172.21.15.87:0/2417396536 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300a5d10 con 0x7f30380b31c0 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- 172.21.15.87:0/2417396536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 msgr2=0x7f30380b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 --2- 172.21.15.87:0/2417396536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f30380b3610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3030009d90 tx=0x7f303001c970 comp rx=0 tx=0).stop 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- 172.21.15.87:0/2417396536 shutdown_connections 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 --2- 172.21.15.87:0/2417396536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f30380b3610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 --2- 172.21.15.87:0/2417396536 >> v2:172.21.15.169:3300/0 conn(0x7f30380aef80 0x7f303815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- 172.21.15.87:0/2417396536 >> 172.21.15.87:0/2417396536 conn(0x7f30380669a0 msgr2=0x7f3038066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- 172.21.15.87:0/2417396536 shutdown_connections 2024-04-25T10:19:23.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- 172.21.15.87:0/2417396536 wait complete. 2024-04-25T10:19:23.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 Processor -- start 2024-04-25T10:19:23.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- start start 2024-04-25T10:19:23.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3038070350 0x7f30380b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30380aef80 0x7f3038154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f303815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381679a0 con 0x7f3038070350 2024-04-25T10:19:23.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3038167820 con 0x7f30380aef80 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f303d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3038167b20 con 0x7f30380b31c0 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3038070350 0x7f30380b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3038070350 0x7f30380b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37074/0 (socket says 172.21.15.87:37074) 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 -- 172.21.15.87:0/2508690366 learned_addr learned my addr 172.21.15.87:0/2508690366 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 -- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 msgr2=0x7f303815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3036800640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f303815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f302d400640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30380aef80 0x7f3038154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f303815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 -- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30380aef80 msgr2=0x7f3038154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30380aef80 0x7f3038154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3035e00640 1 -- 172.21.15.87:0/2508690366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038154db0 con 0x7f3038070350 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f3036800640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f303815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:23.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.020+0000 7f302d400640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30380aef80 0x7f3038154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f3035e00640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3038070350 0x7f30380b1760 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f302800e960 tx=0x7f302800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f302ea00640 1 -- 172.21.15.87:0/2508690366 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028017070 con 0x7f3038070350 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f302ea00640 1 -- 172.21.15.87:0/2508690366 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302800ca00 con 0x7f3038070350 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f302ea00640 1 -- 172.21.15.87:0/2508690366 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302800cdb0 con 0x7f3038070350 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3038155090 con 0x7f3038070350 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3038151d40 con 0x7f3038070350 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303804ef30 con 0x7f3038070350 2024-04-25T10:19:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f302ea00640 1 -- 172.21.15.87:0/2508690366 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f302801a070 con 0x7f3038070350 2024-04-25T10:19:23.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f302ea00640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3014066f90 0x7f3014069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f302ea00640 1 -- 172.21.15.87:0/2508690366 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f30280853a0 con 0x7f3038070350 2024-04-25T10:19:23.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.024+0000 7f302d400640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3014066f90 0x7f3014069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.028+0000 7f302d400640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3014066f90 0x7f3014069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f303815e1b0 tx=0x7f3020009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.028+0000 7f302ea00640 1 -- 172.21.15.87:0/2508690366 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3028056420 con 0x7f3038070350 2024-04-25T10:19:23.107 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:23.110 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:23.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.184+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30380b2fa0 con 0x7f3038070350 2024-04-25T10:19:23.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.184+0000 7f302ea00640 1 -- 172.21.15.87:0/2508690366 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3028055d20 con 0x7f3038070350 2024-04-25T10:19:23.189 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3014066f90 msgr2=0x7f3014069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3014066f90 0x7f3014069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f303815e1b0 tx=0x7f3020009040 comp rx=0 tx=0).stop 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3038070350 msgr2=0x7f30380b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3038070350 0x7f30380b1760 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f302800e960 tx=0x7f302800ee20 comp rx=0 tx=0).stop 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 shutdown_connections 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3014066f90 0x7f3014069450 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30380b31c0 0x7f303815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30380aef80 0x7f3038154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 --2- 172.21.15.87:0/2508690366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3038070350 0x7f30380b1760 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 >> 172.21.15.87:0/2508690366 conn(0x7f30380669a0 msgr2=0x7f30380cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:23.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 shutdown_connections 2024-04-25T10:19:23.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.188+0000 7f303d200640 1 -- 172.21.15.87:0/2508690366 wait complete. 2024-04-25T10:19:23.210 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:19:23.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc520200640 1 Processor -- start 2024-04-25T10:19:23.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc520200640 1 -- start start 2024-04-25T10:19:23.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc520200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5180aef80 0x7fc51815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc520200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51815b080 con 0x7fc5180b31c0 2024-04-25T10:19:23.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc520200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51815adb0 con 0x7fc518070350 2024-04-25T10:19:23.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc520200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51815eb60 con 0x7fc5180aef80 2024-04-25T10:19:23.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51e200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc5180b31c0 0x7fc5180cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58638/0 (socket says 172.21.15.87:58638) 2024-04-25T10:19:23.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51e200640 1 -- 172.21.15.87:0/3820582184 learned_addr learned my addr 172.21.15.87:0/3820582184 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:23.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51d800640 1 --2- 172.21.15.87:0/3820582184 >> v2:172.21.15.169:3300/0 conn(0x7fc5180aef80 0x7fc51815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781484229 0 0) 0x7fc51815b080 con 0x7fc5180b31c0 2024-04-25T10:19:23.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc500003530 con 0x7fc5180b31c0 2024-04-25T10:19:23.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3916661837 0 0) 0x7fc500003530 con 0x7fc5180b31c0 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 >> v2:172.21.15.169:3300/0 conn(0x7fc5180aef80 msgr2=0x7fc51815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 --2- 172.21.15.87:0/3820582184 >> v2:172.21.15.169:3300/0 conn(0x7fc5180aef80 0x7fc51815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 >> v1:172.21.15.169:6789/0 conn(0x7fc518070350 legacy=0x7fc51815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51815ee40 con 0x7fc5180b31c0 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc510001a00 con 0x7fc5180b31c0 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 >> v1:172.21.15.87:6789/0 conn(0x7fc5180b31c0 legacy=0x7fc5180cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5000039c0 0x7fc500003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc500004810 0x7fc500004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000051d0 0x7fc50000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51815eb60 con 0x7fc5000039c0 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50000c070 con 0x7fc5000051d0 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50000c270 con 0x7fc500004810 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51d800640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5000039c0 0x7fc500003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51ce00640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc500004810 0x7fc500004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51e200640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000051d0 0x7fc50000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51d800640 1 -- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc500004810 msgr2=0x7fc500004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51d800640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc500004810 0x7fc500004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51d800640 1 -- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000051d0 msgr2=0x7fc50000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51d800640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000051d0 0x7fc50000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51d800640 1 -- 172.21.15.87:0/3820582184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500003530 con 0x7fc5000039c0 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51e200640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000051d0 0x7fc50000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.388+0000 7fc51e200640 1 -- 172.21.15.87:0/3820582184 reap_dead start 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc51d800640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5000039c0 0x7fc500003de0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009670 tx=0x7fc50c01eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251649121fd5fe8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b4070 con 0x7fc5000039c0 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c0a7770 con 0x7fc5000039c0 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc517e00640 1 -- 172.21.15.87:0/3820582184 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0096b0 con 0x7fc5000039c0 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc520200640 1 -- 172.21.15.87:0/3820582184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5000039c0 msgr2=0x7fc500003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc520200640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5000039c0 0x7fc500003de0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009670 tx=0x7fc50c01eca0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc520200640 1 -- 172.21.15.87:0/3820582184 shutdown_connections 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc520200640 1 --2- 172.21.15.87:0/3820582184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5000039c0 0x7fc500003de0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc520200640 1 -- 172.21.15.87:0/3820582184 >> 172.21.15.87:0/3820582184 conn(0x7fc5180669a0 msgr2=0x7fc518066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc520200640 1 -- 172.21.15.87:0/3820582184 shutdown_connections 2024-04-25T10:19:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.392+0000 7fc520200640 1 -- 172.21.15.87:0/3820582184 wait complete. 2024-04-25T10:19:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 Processor -- start 2024-04-25T10:19:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 -- start start 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000039c0 0x7fc5180a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5181512a0 0x7fc5181f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5180cc8c0 0x7fc5180ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180b39e0 con 0x7fc5180cc8c0 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51815bd90 con 0x7fc5000039c0 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180cf660 con 0x7fc5181512a0 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5180cc8c0 0x7fc5180ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5180cc8c0 0x7fc5180ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37096/0 (socket says 172.21.15.87:37096) 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 -- 172.21.15.87:0/2247684937 learned_addr learned my addr 172.21.15.87:0/2247684937 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51d800640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000039c0 0x7fc5180a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51ce00640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5181512a0 0x7fc5181f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 -- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5181512a0 msgr2=0x7fc5181f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5181512a0 0x7fc5181f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 -- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000039c0 msgr2=0x7fc5180a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000039c0 0x7fc5180a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 -- 172.21.15.87:0/2247684937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5181f43c0 con 0x7fc5180cc8c0 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51d800640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000039c0 0x7fc5180a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51ce00640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5181512a0 0x7fc5181f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc51e200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5180cc8c0 0x7fc5180ccd40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc5180b35a0 tx=0x7fc51000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc515600640 1 -- 172.21.15.87:0/2247684937 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000de50 con 0x7fc5180cc8c0 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc515600640 1 -- 172.21.15.87:0/2247684937 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000bb80 con 0x7fc5180cc8c0 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5180aa460 con 0x7fc5180cc8c0 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc515600640 1 -- 172.21.15.87:0/2247684937 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510007110 con 0x7fc5180cc8c0 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.396+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc518157050 con 0x7fc5180cc8c0 2024-04-25T10:19:23.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.400+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51804ef30 con 0x7fc5180cc8c0 2024-04-25T10:19:23.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.400+0000 7fc515600640 1 -- 172.21.15.87:0/2247684937 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc51001b020 con 0x7fc5180cc8c0 2024-04-25T10:19:23.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.400+0000 7fc515600640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4f8066f60 0x7fc4f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.400+0000 7fc515600640 1 -- 172.21.15.87:0/2247684937 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc510085360 con 0x7fc5180cc8c0 2024-04-25T10:19:23.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.400+0000 7fc51d800640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4f8066f60 0x7fc4f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.400+0000 7fc51d800640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4f8066f60 0x7fc4f8069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0b1000 tx=0x7fc50c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.404+0000 7fc515600640 1 -- 172.21.15.87:0/2247684937 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc510056340 con 0x7fc5180cc8c0 2024-04-25T10:19:23.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.556+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc51815b080 con 0x7fc5180cc8c0 2024-04-25T10:19:23.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.556+0000 7fc515600640 1 -- 172.21.15.87:0/2247684937 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc510055c40 con 0x7fc5180cc8c0 2024-04-25T10:19:23.561 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:19:23.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4f8066f60 msgr2=0x7fc4f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4f8066f60 0x7fc4f8069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0b1000 tx=0x7fc50c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5180cc8c0 msgr2=0x7fc5180ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5180cc8c0 0x7fc5180ccd40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc5180b35a0 tx=0x7fc51000b4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 shutdown_connections 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4f8066f60 0x7fc4f8069420 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5180cc8c0 0x7fc5180ccd40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5181512a0 0x7fc5181f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 --2- 172.21.15.87:0/2247684937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000039c0 0x7fc5180a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 >> 172.21.15.87:0/2247684937 conn(0x7fc5180669a0 msgr2=0x7fc5181554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:23.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 shutdown_connections 2024-04-25T10:19:23.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.560+0000 7fc520200640 1 -- 172.21.15.87:0/2247684937 wait complete. 2024-04-25T10:19:23.583 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:23.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.716+0000 7f94c6600640 1 Processor -- start 2024-04-25T10:19:23.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.716+0000 7f94c6600640 1 -- start start 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.716+0000 7f94c6600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f94c015bc40 0x7f94c006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.716+0000 7f94c6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f94c00cda10 0x7f94c00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94c6600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c00b2770 con 0x7f94c00cf930 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94c6600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006c670 con 0x7f94c015bc40 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94c6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006c950 con 0x7f94c00cda10 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bfe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f94c00cf930 0x7f94c015f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58654/0 (socket says 172.21.15.87:58654) 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f94c015bc40 0x7f94c006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bfe00640 1 -- 172.21.15.87:0/1676767290 learned_addr learned my addr 172.21.15.87:0/1676767290 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bea00640 1 --2- 172.21.15.87:0/1676767290 >> v2:172.21.15.169:3300/0 conn(0x7f94c00cda10 0x7f94c00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94be000640 1 -- 172.21.15.87:0/1676767290 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524072588 0 0) 0x7f94c00b2770 con 0x7f94c00cf930 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94be000640 1 -- 172.21.15.87:0/1676767290 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a4003530 con 0x7f94c00cf930 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bf400640 1 -- 172.21.15.87:0/1676767290 >> v2:172.21.15.169:3300/0 conn(0x7f94c00cda10 msgr2=0x7f94c00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bf400640 1 --2- 172.21.15.87:0/1676767290 >> v2:172.21.15.169:3300/0 conn(0x7f94c00cda10 0x7f94c00b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bf400640 1 -- 172.21.15.87:0/1676767290 >> v1:172.21.15.87:6789/0 conn(0x7f94c00cf930 legacy=0x7f94c015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bf400640 1 -- 172.21.15.87:0/1676767290 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c006cc30 con 0x7f94c015bc40 2024-04-25T10:19:23.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bea00640 1 --2- 172.21.15.87:0/1676767290 >> v2:172.21.15.169:3300/0 conn(0x7f94c00cda10 0x7f94c00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:23.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94bf400640 1 --2- 172.21.15.87:0/1676767290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c015bc40 0x7f94c006c090 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f94c00726f0 tx=0x7f94b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fcf3e78df0abcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94be000640 1 -- 172.21.15.87:0/1676767290 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00b2070 con 0x7f94c015bc40 2024-04-25T10:19:23.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.720+0000 7f94be000640 1 -- 172.21.15.87:0/1676767290 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b00a5900 con 0x7f94c015bc40 2024-04-25T10:19:23.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94be000640 1 -- 172.21.15.87:0/1676767290 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00a5cd0 con 0x7f94c015bc40 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- 172.21.15.87:0/1676767290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c015bc40 msgr2=0x7f94c006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 --2- 172.21.15.87:0/1676767290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c015bc40 0x7f94c006c090 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f94c00726f0 tx=0x7f94b001c930 comp rx=0 tx=0).stop 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- 172.21.15.87:0/1676767290 shutdown_connections 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 --2- 172.21.15.87:0/1676767290 >> v2:172.21.15.169:3300/0 conn(0x7f94c00cda10 0x7f94c00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 --2- 172.21.15.87:0/1676767290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c015bc40 0x7f94c006c090 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- 172.21.15.87:0/1676767290 >> 172.21.15.87:0/1676767290 conn(0x7f94c0066b70 msgr2=0x7f94c0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- 172.21.15.87:0/1676767290 shutdown_connections 2024-04-25T10:19:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- 172.21.15.87:0/1676767290 wait complete. 2024-04-25T10:19:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 Processor -- start 2024-04-25T10:19:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- start start 2024-04-25T10:19:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94c00cda10 0x7f94c0156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c00cf930 0x7f94c015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94c015bc40 0x7f94c0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0168b50 con 0x7f94c00cf930 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01689d0 con 0x7f94c015bc40 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94c6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0168cd0 con 0x7f94c00cda10 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c00cf930 0x7f94c015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c00cf930 0x7f94c015e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37110/0 (socket says 172.21.15.87:37110) 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bea00640 1 -- 172.21.15.87:0/1778262940 learned_addr learned my addr 172.21.15.87:0/1778262940 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bf400640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94c00cda10 0x7f94c0156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bfe00640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94c015bc40 0x7f94c0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bea00640 1 -- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94c00cda10 msgr2=0x7f94c0156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bea00640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94c00cda10 0x7f94c0156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bea00640 1 -- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94c015bc40 msgr2=0x7f94c0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.724+0000 7f94bea00640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94c015bc40 0x7f94c0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94bea00640 1 -- 172.21.15.87:0/1778262940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c015f340 con 0x7f94c00cf930 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94bf400640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94c00cda10 0x7f94c0156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94bea00640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c00cf930 0x7f94c015e050 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00d830 tx=0x7f94ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94b7400640 1 -- 172.21.15.87:0/1778262940 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac018020 con 0x7f94c00cf930 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94b7400640 1 -- 172.21.15.87:0/1778262940 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac009660 con 0x7f94c00cf930 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94b7400640 1 -- 172.21.15.87:0/1778262940 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac009a50 con 0x7f94c00cf930 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c015eaa0 con 0x7f94c00cf930 2024-04-25T10:19:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c015f060 con 0x7f94c00cf930 2024-04-25T10:19:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c004ef30 con 0x7f94c00cf930 2024-04-25T10:19:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94b7400640 1 -- 172.21.15.87:0/1778262940 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94ac01b020 con 0x7f94c00cf930 2024-04-25T10:19:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94b7400640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94a0066f60 0x7f94a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94b7400640 1 -- 172.21.15.87:0/1778262940 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f94ac085810 con 0x7f94c00cf930 2024-04-25T10:19:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.728+0000 7f94bf400640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94a0066f60 0x7f94a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.732+0000 7f94bf400640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94a0066f60 0x7f94a0069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f94b00af000 tx=0x7f94b00a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:23.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.732+0000 7f94b7400640 1 -- 172.21.15.87:0/1778262940 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94ac022020 con 0x7f94c00cf930 2024-04-25T10:19:23.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94c00af1c0 con 0x7f94c00cf930 2024-04-25T10:19:23.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94b7400640 1 -- 172.21.15.87:0/1778262940 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f94ac011340 con 0x7f94c00cf930 2024-04-25T10:19:23.922 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:23.922 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:23.922 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94a0066f60 msgr2=0x7f94a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94a0066f60 0x7f94a0069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f94b00af000 tx=0x7f94b00a7070 comp rx=0 tx=0).stop 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c00cf930 msgr2=0x7f94c015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c00cf930 0x7f94c015e050 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00d830 tx=0x7f94ac00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 shutdown_connections 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94a0066f60 0x7f94a0069420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94c015bc40 0x7f94c0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c00cf930 0x7f94c015e050 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 --2- 172.21.15.87:0/1778262940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94c00cda10 0x7f94c0156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 >> 172.21.15.87:0/1778262940 conn(0x7f94c0066b70 msgr2=0x7f94c00ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 shutdown_connections 2024-04-25T10:19:23.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:23.920+0000 7f94c6600640 1 -- 172.21.15.87:0/1778262940 wait complete. 2024-04-25T10:19:23.946 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:24.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.076+0000 7f9d54400640 1 Processor -- start 2024-04-25T10:19:24.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d54400640 1 -- start start 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d54400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d4c15bc40 0x7f9d4c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d54400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9d4c0cda10 0x7f9d4c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d54400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4c0b2770 con 0x7f9d4c0cf930 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d54400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4c06c670 con 0x7f9d4c0cda10 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d54400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4c06c950 con 0x7f9d4c15bc40 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d52400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9d4c0cf930 0x7f9d4c15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58660/0 (socket says 172.21.15.87:58660) 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d51000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9d4c0cda10 0x7f9d4c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d52400640 1 -- 172.21.15.87:0/2892775213 learned_addr learned my addr 172.21.15.87:0/2892775213 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:24.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d51a00640 1 --2- 172.21.15.87:0/2892775213 >> v2:172.21.15.169:3300/0 conn(0x7f9d4c15bc40 0x7f9d4c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d51000640 1 -- 172.21.15.87:0/2892775213 >> v2:172.21.15.169:3300/0 conn(0x7f9d4c15bc40 msgr2=0x7f9d4c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:24.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d51000640 1 --2- 172.21.15.87:0/2892775213 >> v2:172.21.15.169:3300/0 conn(0x7f9d4c15bc40 0x7f9d4c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d51000640 1 -- 172.21.15.87:0/2892775213 >> v1:172.21.15.87:6789/0 conn(0x7f9d4c0cf930 legacy=0x7f9d4c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d51000640 1 -- 172.21.15.87:0/2892775213 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c06cc30 con 0x7f9d4c0cda10 2024-04-25T10:19:24.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d4be00640 1 -- 172.21.15.87:0/2892775213 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855241787 0 0) 0x7f9d4c0b2770 con 0x7f9d4c0cf930 2024-04-25T10:19:24.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.080+0000 7f9d51a00640 1 --2- 172.21.15.87:0/2892775213 >> v2:172.21.15.169:3300/0 conn(0x7f9d4c15bc40 0x7f9d4c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:24.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d51000640 1 --2- 172.21.15.87:0/2892775213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c0cda10 0x7f9d4c0b4610 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0d0800 tx=0x7f9d3801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa0ac013c779390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d4be00640 1 -- 172.21.15.87:0/2892775213 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d380b1070 con 0x7f9d4c0cda10 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d4be00640 1 -- 172.21.15.87:0/2892775213 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d380ac5f0 con 0x7f9d4c0cda10 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d4be00640 1 -- 172.21.15.87:0/2892775213 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d380ac9c0 con 0x7f9d4c0cda10 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 -- 172.21.15.87:0/2892775213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c0cda10 msgr2=0x7f9d4c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 --2- 172.21.15.87:0/2892775213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c0cda10 0x7f9d4c0b4610 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0d0800 tx=0x7f9d3801c480 comp rx=0 tx=0).stop 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 -- 172.21.15.87:0/2892775213 shutdown_connections 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 --2- 172.21.15.87:0/2892775213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c0cda10 0x7f9d4c0b4610 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 --2- 172.21.15.87:0/2892775213 >> v2:172.21.15.169:3300/0 conn(0x7f9d4c15bc40 0x7f9d4c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 -- 172.21.15.87:0/2892775213 >> 172.21.15.87:0/2892775213 conn(0x7f9d4c066b70 msgr2=0x7f9d4c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 -- 172.21.15.87:0/2892775213 shutdown_connections 2024-04-25T10:19:24.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 -- 172.21.15.87:0/2892775213 wait complete. 2024-04-25T10:19:24.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 Processor -- start 2024-04-25T10:19:24.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.084+0000 7f9d54400640 1 -- start start 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0cda10 0x7f9d4c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c0cf930 0x7f9d4c0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c15bc40 0x7f9d4c145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c168b50 con 0x7f9d4c15bc40 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c1689d0 con 0x7f9d4c0cf930 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c168cd0 con 0x7f9d4c0cda10 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c15bc40 0x7f9d4c145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c15bc40 0x7f9d4c145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37132/0 (socket says 172.21.15.87:37132) 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 -- 172.21.15.87:0/3939020989 learned_addr learned my addr 172.21.15.87:0/3939020989 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:24.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d51000640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c0cf930 0x7f9d4c0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 -- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0cda10 msgr2=0x7f9d4c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d51a00640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0cda10 0x7f9d4c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0cda10 0x7f9d4c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 -- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c0cf930 msgr2=0x7f9d4c0cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c0cf930 0x7f9d4c0cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 -- 172.21.15.87:0/3939020989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c157340 con 0x7f9d4c15bc40 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d51000640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c0cf930 0x7f9d4c0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d51a00640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0cda10 0x7f9d4c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d52400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c15bc40 0x7f9d4c145570 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c730 tx=0x7f9d4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d49600640 1 -- 172.21.15.87:0/3939020989 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44018070 con 0x7f9d4c15bc40 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d49600640 1 -- 172.21.15.87:0/3939020989 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4400ce40 con 0x7f9d4c15bc40 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c1470a0 con 0x7f9d4c15bc40 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d49600640 1 -- 172.21.15.87:0/3939020989 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400a860 con 0x7f9d4c15bc40 2024-04-25T10:19:24.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c156580 con 0x7f9d4c15bc40 2024-04-25T10:19:24.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c04ef30 con 0x7f9d4c15bc40 2024-04-25T10:19:24.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.088+0000 7f9d49600640 1 -- 172.21.15.87:0/3939020989 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d4401b070 con 0x7f9d4c15bc40 2024-04-25T10:19:24.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.092+0000 7f9d49600640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d2c066f60 0x7f9d2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.092+0000 7f9d49600640 1 -- 172.21.15.87:0/3939020989 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9d44084690 con 0x7f9d4c15bc40 2024-04-25T10:19:24.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.092+0000 7f9d51a00640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d2c066f60 0x7f9d2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.092+0000 7f9d51a00640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d2c066f60 0x7f9d2c069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9d40008930 tx=0x7f9d40008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.096+0000 7f9d49600640 1 -- 172.21.15.87:0/3939020989 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d440555e0 con 0x7f9d4c15bc40 2024-04-25T10:19:24.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.252+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d4c0af1c0 con 0x7f9d4c15bc40 2024-04-25T10:19:24.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d49600640 1 -- 172.21.15.87:0/3939020989 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16698 (secure 0 0 0) 0x7f9d44054ee0 con 0x7f9d4c15bc40 2024-04-25T10:19:24.258 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:24.258 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:06.154692+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:36.789308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:37.439968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d2c066f60 msgr2=0x7f9d2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d2c066f60 0x7f9d2c069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9d40008930 tx=0x7f9d40008040 comp rx=0 tx=0).stop 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c15bc40 msgr2=0x7f9d4c145570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c15bc40 0x7f9d4c145570 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c730 tx=0x7f9d4400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 shutdown_connections 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d2c066f60 0x7f9d2c069420 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d4c15bc40 0x7f9d4c145570 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c0cf930 0x7f9d4c0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 --2- 172.21.15.87:0/3939020989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0cda10 0x7f9d4c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 >> 172.21.15.87:0/3939020989 conn(0x7f9d4c066b70 msgr2=0x7f9d4c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:24.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 shutdown_connections 2024-04-25T10:19:24.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.256+0000 7f9d54400640 1 -- 172.21.15.87:0/3939020989 wait complete. 2024-04-25T10:19:24.279 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:19:24.279 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:24.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2ebbc00640 1 Processor -- start 2024-04-25T10:19:24.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2ebbc00640 1 -- start start 2024-04-25T10:19:24.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2ebbc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2eb415dce0 0x7f2eb40669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2ebbc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb406eec0 0x7f2eb406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2ebbc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb415e0c0 con 0x7f2eb40ad850 2024-04-25T10:19:24.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2ebbc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb40674d0 con 0x7f2eb415dce0 2024-04-25T10:19:24.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2ebbc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb40671f0 con 0x7f2eb406eec0 2024-04-25T10:19:24.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb9c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2eb40ad850 0x7f2eb40adc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58662/0 (socket says 172.21.15.87:58662) 2024-04-25T10:19:24.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb9200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2eb415dce0 0x7f2eb40669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb9c00640 1 -- 172.21.15.87:0/150614992 learned_addr learned my addr 172.21.15.87:0/150614992 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eabe00640 1 --2- 172.21.15.87:0/150614992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb406eec0 0x7f2eb406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb9200640 1 -- 172.21.15.87:0/150614992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb406eec0 msgr2=0x7f2eb406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb9200640 1 --2- 172.21.15.87:0/150614992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb406eec0 0x7f2eb406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb9200640 1 -- 172.21.15.87:0/150614992 >> v1:172.21.15.87:6789/0 conn(0x7f2eb40ad850 legacy=0x7f2eb40adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb3e00640 1 -- 172.21.15.87:0/150614992 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512101498 0 0) 0x7f2eb415e0c0 con 0x7f2eb40ad850 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eb9200640 1 -- 172.21.15.87:0/150614992 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb406d4b0 con 0x7f2eb415dce0 2024-04-25T10:19:24.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.412+0000 7f2eabe00640 1 --2- 172.21.15.87:0/150614992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb406eec0 0x7f2eb406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2eb9200640 1 --2- 172.21.15.87:0/150614992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb415dce0 0x7f2eb40669a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40097c0 tx=0x7f2ea401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c4ad8a51aace19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2eb3e00640 1 -- 172.21.15.87:0/150614992 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40b0070 con 0x7f2eb415dce0 2024-04-25T10:19:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2eb3e00640 1 -- 172.21.15.87:0/150614992 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea40ab600 con 0x7f2eb415dce0 2024-04-25T10:19:24.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2eb3e00640 1 -- 172.21.15.87:0/150614992 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40ab9d0 con 0x7f2eb415dce0 2024-04-25T10:19:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 -- 172.21.15.87:0/150614992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb415dce0 msgr2=0x7f2eb40669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/150614992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb415dce0 0x7f2eb40669a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40097c0 tx=0x7f2ea401c380 comp rx=0 tx=0).stop 2024-04-25T10:19:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 -- 172.21.15.87:0/150614992 shutdown_connections 2024-04-25T10:19:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/150614992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb406eec0 0x7f2eb406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/150614992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb415dce0 0x7f2eb40669a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 -- 172.21.15.87:0/150614992 >> 172.21.15.87:0/150614992 conn(0x7f2eb40b2410 msgr2=0x7f2eb40b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 -- 172.21.15.87:0/150614992 shutdown_connections 2024-04-25T10:19:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.416+0000 7f2ebbc00640 1 -- 172.21.15.87:0/150614992 wait complete. 2024-04-25T10:19:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 Processor -- start 2024-04-25T10:19:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 -- start start 2024-04-25T10:19:24.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb406eec0 0x7f2eb4154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb40ad850 0x7f2eb4151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb415dce0 0x7f2eb41f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb40d0320 con 0x7f2eb406eec0 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb40d04a0 con 0x7f2eb415dce0 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb416abc0 con 0x7f2eb40ad850 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb406eec0 0x7f2eb4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb406eec0 0x7f2eb4154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37156/0 (socket says 172.21.15.87:37156) 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 -- 172.21.15.87:0/51390189 learned_addr learned my addr 172.21.15.87:0/51390189 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9c00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb415dce0 0x7f2eb41f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eabe00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb40ad850 0x7f2eb4151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 -- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb40ad850 msgr2=0x7f2eb4151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb40ad850 0x7f2eb4151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 -- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb415dce0 msgr2=0x7f2eb41f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb415dce0 0x7f2eb41f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 -- 172.21.15.87:0/51390189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb4155f10 con 0x7f2eb406eec0 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb9200640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb406eec0 0x7f2eb4154950 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4019fb0 tx=0x7f2ea401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb1600640 1 -- 172.21.15.87:0/51390189 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40b0050 con 0x7f2eb406eec0 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb1600640 1 -- 172.21.15.87:0/51390189 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea40bc070 con 0x7f2eb406eec0 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb41552e0 con 0x7f2eb406eec0 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2eb1600640 1 -- 172.21.15.87:0/51390189 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40b5a30 con 0x7f2eb406eec0 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb41561d0 con 0x7f2eb406eec0 2024-04-25T10:19:24.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.420+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb404ef30 con 0x7f2eb406eec0 2024-04-25T10:19:24.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.424+0000 7f2eb1600640 1 -- 172.21.15.87:0/51390189 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ea40ab4c0 con 0x7f2eb406eec0 2024-04-25T10:19:24.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.424+0000 7f2eb1600640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e90066f60 0x7f2e90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.424+0000 7f2eb1600640 1 -- 172.21.15.87:0/51390189 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2ea411fa00 con 0x7f2eb406eec0 2024-04-25T10:19:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.424+0000 7f2eabe00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e90066f60 0x7f2e90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.424+0000 7f2eabe00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e90066f60 0x7f2e90069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c006fd0 tx=0x7f2e9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.428+0000 7f2eb1600640 1 -- 172.21.15.87:0/51390189 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2eb41552e0 con 0x7f2eb406eec0 2024-04-25T10:19:24.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2eb40af770 con 0x7f2eb406eec0 2024-04-25T10:19:24.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2eb1600640 1 -- 172.21.15.87:0/51390189 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2ea4016090 con 0x7f2eb406eec0 2024-04-25T10:19:24.618 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:24.618 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:24.618 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:24.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e90066f60 msgr2=0x7f2e90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e90066f60 0x7f2e90069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c006fd0 tx=0x7f2e9c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:24.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb406eec0 msgr2=0x7f2eb4154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb406eec0 0x7f2eb4154950 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4019fb0 tx=0x7f2ea401abc0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 shutdown_connections 2024-04-25T10:19:24.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e90066f60 0x7f2e90069420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb415dce0 0x7f2eb41f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2eb40ad850 0x7f2eb4151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 --2- 172.21.15.87:0/51390189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2eb406eec0 0x7f2eb4154950 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 >> 172.21.15.87:0/51390189 conn(0x7f2eb40b2410 msgr2=0x7f2eb40b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:24.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 shutdown_connections 2024-04-25T10:19:24.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.616+0000 7f2ebbc00640 1 -- 172.21.15.87:0/51390189 wait complete. 2024-04-25T10:19:24.639 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:24.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.768+0000 7fc460c00640 1 Processor -- start 2024-04-25T10:19:24.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.768+0000 7fc460c00640 1 -- start start 2024-04-25T10:19:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc460c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc45c15bc40 0x7fc45c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc460c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c0cf930 0x7fc45c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc460c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45c0b2770 con 0x7fc45c0cda10 2024-04-25T10:19:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc460c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c06c670 con 0x7fc45c15bc40 2024-04-25T10:19:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc460c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c06c950 con 0x7fc45c0cf930 2024-04-25T10:19:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc451400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc45c0cda10 0x7fc45c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58672/0 (socket says 172.21.15.87:58672) 2024-04-25T10:19:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc451400640 1 -- 172.21.15.87:0/2766708510 learned_addr learned my addr 172.21.15.87:0/2766708510 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:24.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc45a800640 1 --2- 172.21.15.87:0/2766708510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c0cf930 0x7fc45c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc459e00640 1 --2- 172.21.15.87:0/2766708510 >> v2:172.21.15.169:3300/0 conn(0x7fc45c15bc40 0x7fc45c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc459400640 1 -- 172.21.15.87:0/2766708510 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210956691 0 0) 0x7fc45c0b2770 con 0x7fc45c0cda10 2024-04-25T10:19:24.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc459400640 1 -- 172.21.15.87:0/2766708510 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc440003530 con 0x7fc45c0cda10 2024-04-25T10:19:24.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc45a800640 1 -- 172.21.15.87:0/2766708510 >> v2:172.21.15.169:3300/0 conn(0x7fc45c15bc40 msgr2=0x7fc45c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:24.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc45a800640 1 --2- 172.21.15.87:0/2766708510 >> v2:172.21.15.169:3300/0 conn(0x7fc45c15bc40 0x7fc45c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc45a800640 1 -- 172.21.15.87:0/2766708510 >> v1:172.21.15.87:6789/0 conn(0x7fc45c0cda10 legacy=0x7fc45c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.772+0000 7fc45a800640 1 -- 172.21.15.87:0/2766708510 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c06cc30 con 0x7fc45c0cf930 2024-04-25T10:19:24.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc45a800640 1 --2- 172.21.15.87:0/2766708510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c0cf930 0x7fc45c0cfde0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc454009d90 tx=0x7fc45401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767ead5513525b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc459400640 1 -- 172.21.15.87:0/2766708510 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540b2070 con 0x7fc45c0cf930 2024-04-25T10:19:24.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc459400640 1 -- 172.21.15.87:0/2766708510 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4540a57a0 con 0x7fc45c0cf930 2024-04-25T10:19:24.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc459400640 1 -- 172.21.15.87:0/2766708510 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540a5b50 con 0x7fc45c0cf930 2024-04-25T10:19:24.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 -- 172.21.15.87:0/2766708510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c0cf930 msgr2=0x7fc45c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 --2- 172.21.15.87:0/2766708510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c0cf930 0x7fc45c0cfde0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc454009d90 tx=0x7fc45401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 -- 172.21.15.87:0/2766708510 shutdown_connections 2024-04-25T10:19:24.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 --2- 172.21.15.87:0/2766708510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c0cf930 0x7fc45c0cfde0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 --2- 172.21.15.87:0/2766708510 >> v2:172.21.15.169:3300/0 conn(0x7fc45c15bc40 0x7fc45c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 -- 172.21.15.87:0/2766708510 >> 172.21.15.87:0/2766708510 conn(0x7fc45c066b70 msgr2=0x7fc45c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:24.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 -- 172.21.15.87:0/2766708510 shutdown_connections 2024-04-25T10:19:24.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 -- 172.21.15.87:0/2766708510 wait complete. 2024-04-25T10:19:24.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 Processor -- start 2024-04-25T10:19:24.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.776+0000 7fc460c00640 1 -- start start 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc45c0cda10 0x7fc45c0ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45c0cf930 0x7fc45c1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c15bc40 0x7fc45c1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0b3890 con 0x7fc45c0cf930 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0b3710 con 0x7fc45c0cda10 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0b3a10 con 0x7fc45c15bc40 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45c0cf930 0x7fc45c1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45c0cf930 0x7fc45c1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37158/0 (socket says 172.21.15.87:37158) 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 -- 172.21.15.87:0/3451177316 learned_addr learned my addr 172.21.15.87:0/3451177316 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc459e00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc45c0cda10 0x7fc45c0ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc45a800640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c15bc40 0x7fc45c1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 -- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c15bc40 msgr2=0x7fc45c1584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c15bc40 0x7fc45c1584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 -- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc45c0cda10 msgr2=0x7fc45c0ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc45c0cda10 0x7fc45c0ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 -- 172.21.15.87:0/3451177316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c1568a0 con 0x7fc45c0cf930 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc459e00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc45c0cda10 0x7fc45c0ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc451400640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45c0cf930 0x7fc45c1534e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c3f0 tx=0x7fc44400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc452a00640 1 -- 172.21.15.87:0/3451177316 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444017070 con 0x7fc45c0cf930 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc452a00640 1 -- 172.21.15.87:0/3451177316 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc444005cf0 con 0x7fc45c0cf930 2024-04-25T10:19:24.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc452a00640 1 -- 172.21.15.87:0/3451177316 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440072f0 con 0x7fc45c0cf930 2024-04-25T10:19:24.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c06d750 con 0x7fc45c0cf930 2024-04-25T10:19:24.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c0ae030 con 0x7fc45c0cf930 2024-04-25T10:19:24.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.780+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c04ef30 con 0x7fc45c0cf930 2024-04-25T10:19:24.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.784+0000 7fc452a00640 1 -- 172.21.15.87:0/3451177316 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc44401a070 con 0x7fc45c0cf930 2024-04-25T10:19:24.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.784+0000 7fc452a00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc438066f60 0x7fc438069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:24.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.784+0000 7fc452a00640 1 -- 172.21.15.87:0/3451177316 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc444084470 con 0x7fc45c0cf930 2024-04-25T10:19:24.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.784+0000 7fc459e00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc438066f60 0x7fc438069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:24.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.784+0000 7fc459e00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc438066f60 0x7fc438069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009b10 tx=0x7fc44c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:24.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.788+0000 7fc452a00640 1 -- 172.21.15.87:0/3451177316 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc444055450 con 0x7fc45c0cf930 2024-04-25T10:19:24.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.972+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc45c06d960 con 0x7fc45c0cf930 2024-04-25T10:19:24.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.972+0000 7fc452a00640 1 -- 172.21.15.87:0/3451177316 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc444054d50 con 0x7fc45c0cf930 2024-04-25T10:19:24.976 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:24.977 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:38.442126+0000","modified":"2024-04-25T10:18:41.469242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3032837514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3032837514},{"type":"v1","addr":"172.21.15.87:6837","nonce":3032837514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:19:24.977 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 63 2024-04-25T10:19:24.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc438066f60 msgr2=0x7fc438069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc438066f60 0x7fc438069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009b10 tx=0x7fc44c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:24.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45c0cf930 msgr2=0x7fc45c1534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45c0cf930 0x7fc45c1534e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c3f0 tx=0x7fc44400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 shutdown_connections 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc438066f60 0x7fc438069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc45c15bc40 0x7fc45c1584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45c0cf930 0x7fc45c1534e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 --2- 172.21.15.87:0/3451177316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc45c0cda10 0x7fc45c0ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 >> 172.21.15.87:0/3451177316 conn(0x7fc45c066b70 msgr2=0x7fc45c068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 shutdown_connections 2024-04-25T10:19:24.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:24.976+0000 7fc460c00640 1 -- 172.21.15.87:0/3451177316 wait complete. 2024-04-25T10:19:24.997 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:25.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.128+0000 7fb624600640 1 Processor -- start 2024-04-25T10:19:25.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.128+0000 7fb624600640 1 -- start start 2024-04-25T10:19:25.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb624600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c1550b0 0x7fb61c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb624600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c071050 con 0x7fb61c15b910 2024-04-25T10:19:25.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb624600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c15d040 con 0x7fb61c0cfe70 2024-04-25T10:19:25.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb624600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c15bda0 con 0x7fb61c1550b0 2024-04-25T10:19:25.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb622600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb61c15b910 0x7fb61c0b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58682/0 (socket says 172.21.15.87:58682) 2024-04-25T10:19:25.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb622600640 1 -- 172.21.15.87:0/4189457631 learned_addr learned my addr 172.21.15.87:0/4189457631 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:25.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb621c00640 1 --2- 172.21.15.87:0/4189457631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c1550b0 0x7fb61c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230254262 0 0) 0x7fb61c071050 con 0x7fb61c15b910 2024-04-25T10:19:25.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb604005740 con 0x7fb61c15b910 2024-04-25T10:19:25.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751093190 0 0) 0x7fb61c15d040 con 0x7fb61c0cfe70 2024-04-25T10:19:25.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb61c071050 con 0x7fb61c0cfe70 2024-04-25T10:19:25.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb621c00640 1 -- 172.21.15.87:0/4189457631 >> v1:172.21.15.169:6789/0 conn(0x7fb61c0cfe70 legacy=0x7fb61c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb621c00640 1 -- 172.21.15.87:0/4189457631 >> v1:172.21.15.87:6789/0 conn(0x7fb61c15b910 legacy=0x7fb61c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013522099 0 0) 0x7fb604005740 con 0x7fb61c15b910 2024-04-25T10:19:25.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.132+0000 7fb621c00640 1 -- 172.21.15.87:0/4189457631 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0712f0 con 0x7fb61c1550b0 2024-04-25T10:19:25.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb621c00640 1 --2- 172.21.15.87:0/4189457631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c1550b0 0x7fb61c0cf930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb610009dc0 tx=0x7fb61001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cd7953f47b1129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614001680 con 0x7fb61c1550b0 2024-04-25T10:19:25.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6100a64f0 con 0x7fb61c1550b0 2024-04-25T10:19:25.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb61be00640 1 -- 172.21.15.87:0/4189457631 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6100a68c0 con 0x7fb61c1550b0 2024-04-25T10:19:25.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 -- 172.21.15.87:0/4189457631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c1550b0 msgr2=0x7fb61c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 --2- 172.21.15.87:0/4189457631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c1550b0 0x7fb61c0cf930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb610009dc0 tx=0x7fb61001c8b0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 -- 172.21.15.87:0/4189457631 shutdown_connections 2024-04-25T10:19:25.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 --2- 172.21.15.87:0/4189457631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c1550b0 0x7fb61c0cf930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 -- 172.21.15.87:0/4189457631 >> 172.21.15.87:0/4189457631 conn(0x7fb61c066b70 msgr2=0x7fb61c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:25.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 -- 172.21.15.87:0/4189457631 shutdown_connections 2024-04-25T10:19:25.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 -- 172.21.15.87:0/4189457631 wait complete. 2024-04-25T10:19:25.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 Processor -- start 2024-04-25T10:19:25.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 -- start start 2024-04-25T10:19:25.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c0cfe70 0x7fb61c13e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.136+0000 7fb624600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb61c1550b0 0x7fb61c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb624600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb61c15b910 0x7fb61c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb624600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c06c090 con 0x7fb61c15b910 2024-04-25T10:19:25.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb624600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0720b0 con 0x7fb61c1550b0 2024-04-25T10:19:25.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb624600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c06c210 con 0x7fb61c0cfe70 2024-04-25T10:19:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb61c15b910 0x7fb61c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb61c15b910 0x7fb61c15c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37166/0 (socket says 172.21.15.87:37166) 2024-04-25T10:19:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 -- 172.21.15.87:0/3827969239 learned_addr learned my addr 172.21.15.87:0/3827969239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:25.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 -- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c0cfe70 msgr2=0x7fb61c13e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb621c00640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c0cfe70 0x7fb61c13e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb621200640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb61c1550b0 0x7fb61c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c0cfe70 0x7fb61c13e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 -- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb61c1550b0 msgr2=0x7fb61c150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb61c1550b0 0x7fb61c150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 -- 172.21.15.87:0/3827969239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c13e6d0 con 0x7fb61c15b910 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb621c00640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c0cfe70 0x7fb61c13e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb621200640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb61c1550b0 0x7fb61c150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb622600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb61c15b910 0x7fb61c15c010 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb61400c730 tx=0x7fb61400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb619600640 1 -- 172.21.15.87:0/3827969239 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614017070 con 0x7fb61c15b910 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb619600640 1 -- 172.21.15.87:0/3827969239 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614005ae0 con 0x7fb61c15b910 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c0cdf20 con 0x7fb61c15b910 2024-04-25T10:19:25.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb619600640 1 -- 172.21.15.87:0/3827969239 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614007070 con 0x7fb61c15b910 2024-04-25T10:19:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c14f3b0 con 0x7fb61c15b910 2024-04-25T10:19:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c0ce0f0 con 0x7fb61c15b910 2024-04-25T10:19:25.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.140+0000 7fb619600640 1 -- 172.21.15.87:0/3827969239 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb61401a070 con 0x7fb61c15b910 2024-04-25T10:19:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.144+0000 7fb619600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5fc066f60 0x7fb5fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.144+0000 7fb619600640 1 -- 172.21.15.87:0/3827969239 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb614084540 con 0x7fb61c15b910 2024-04-25T10:19:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.144+0000 7fb621c00640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5fc066f60 0x7fb5fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.144+0000 7fb621c00640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5fc066f60 0x7fb5fc069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb6100af000 tx=0x7fb6100a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.148+0000 7fb619600640 1 -- 172.21.15.87:0/3827969239 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6140555e0 con 0x7fb61c15b910 2024-04-25T10:19:25.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.304+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb61c0b0d00 con 0x7fb61c15b910 2024-04-25T10:19:25.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.304+0000 7fb619600640 1 -- 172.21.15.87:0/3827969239 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16698 (secure 0 0 0) 0x7fb614054ee0 con 0x7fb61c15b910 2024-04-25T10:19:25.308 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:25.308 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:06.154692+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:36.789308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:37.439968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:25.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5fc066f60 msgr2=0x7fb5fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5fc066f60 0x7fb5fc069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb6100af000 tx=0x7fb6100a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:25.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb61c15b910 msgr2=0x7fb61c15c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb61c15b910 0x7fb61c15c010 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb61400c730 tx=0x7fb61400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 shutdown_connections 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5fc066f60 0x7fb5fc069420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb61c15b910 0x7fb61c15c010 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb61c1550b0 0x7fb61c150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 --2- 172.21.15.87:0/3827969239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb61c0cfe70 0x7fb61c13e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 >> 172.21.15.87:0/3827969239 conn(0x7fb61c066b70 msgr2=0x7fb61c15b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 shutdown_connections 2024-04-25T10:19:25.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.308+0000 7fb624600640 1 -- 172.21.15.87:0/3827969239 wait complete. 2024-04-25T10:19:25.330 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:19:25.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cdb600640 1 Processor -- start 2024-04-25T10:19:25.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cdb600640 1 -- start start 2024-04-25T10:19:25.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cdb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd415bc40 0x7f5cd406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cdb600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5cd40cda10 0x7f5cd40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cdb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd40b2770 con 0x7f5cd40cf930 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cdb600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd406c670 con 0x7f5cd40cda10 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cdb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406c950 con 0x7f5cd415bc40 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5ccbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5cd40cda10 0x7f5cd40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5ccbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5cd40cda10 0x7f5cd40b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37174/0 (socket says 172.21.15.87:37174) 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5ccbe00640 1 -- 172.21.15.87:0/3115732827 learned_addr learned my addr 172.21.15.87:0/3115732827 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cd8c00640 1 --2- 172.21.15.87:0/3115732827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd415bc40 0x7f5cd406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5ccbe00640 1 -- 172.21.15.87:0/3115732827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd415bc40 msgr2=0x7f5cd406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:25.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5ccbe00640 1 --2- 172.21.15.87:0/3115732827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd415bc40 0x7f5cd406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5ccbe00640 1 -- 172.21.15.87:0/3115732827 >> v1:172.21.15.169:6789/0 conn(0x7f5cd40cf930 legacy=0x7f5cd415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5ccbe00640 1 -- 172.21.15.87:0/3115732827 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd406cc30 con 0x7f5cd40cda10 2024-04-25T10:19:25.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.464+0000 7f5cd8c00640 1 --2- 172.21.15.87:0/3115732827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd415bc40 0x7f5cd406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:25.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5ccbe00640 1 --2- 172.21.15.87:0/3115732827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd40cda10 0x7f5cd40b4610 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009870 tx=0x7f5cbc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958889da98050d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cd3e00640 1 -- 172.21.15.87:0/3115732827 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0a7070 con 0x7f5cd40cda10 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cd3e00640 1 -- 172.21.15.87:0/3115732827 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc0b0070 con 0x7f5cd40cda10 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cd3e00640 1 -- 172.21.15.87:0/3115732827 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0ab890 con 0x7f5cd40cda10 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 -- 172.21.15.87:0/3115732827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd40cda10 msgr2=0x7f5cd40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 --2- 172.21.15.87:0/3115732827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd40cda10 0x7f5cd40b4610 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009870 tx=0x7f5cbc01c560 comp rx=0 tx=0).stop 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 -- 172.21.15.87:0/3115732827 shutdown_connections 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 --2- 172.21.15.87:0/3115732827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd40cda10 0x7f5cd40b4610 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 --2- 172.21.15.87:0/3115732827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd415bc40 0x7f5cd406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 -- 172.21.15.87:0/3115732827 >> 172.21.15.87:0/3115732827 conn(0x7f5cd4066b70 msgr2=0x7f5cd4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 -- 172.21.15.87:0/3115732827 shutdown_connections 2024-04-25T10:19:25.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.468+0000 7f5cdb600640 1 -- 172.21.15.87:0/3115732827 wait complete. 2024-04-25T10:19:25.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 Processor -- start 2024-04-25T10:19:25.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 -- start start 2024-04-25T10:19:25.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd40cda10 0x7f5cd41489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd40cf930 0x7f5cd415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd415bc40 0x7f5cd415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4168b50 con 0x7f5cd415bc40 2024-04-25T10:19:25.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41689d0 con 0x7f5cd40cf930 2024-04-25T10:19:25.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4168cd0 con 0x7f5cd40cda10 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd415bc40 0x7f5cd415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd415bc40 0x7f5cd415d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37180/0 (socket says 172.21.15.87:37180) 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 -- 172.21.15.87:0/2807444419 learned_addr learned my addr 172.21.15.87:0/2807444419 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd8c00640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd40cda10 0x7f5cd41489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5ccbe00640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd40cf930 0x7f5cd415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 -- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd40cda10 msgr2=0x7f5cd41489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd40cda10 0x7f5cd41489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 -- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd40cf930 msgr2=0x7f5cd415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd40cf930 0x7f5cd415cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 -- 172.21.15.87:0/2807444419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd415d7d0 con 0x7f5cd415bc40 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5ccbe00640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd40cf930 0x7f5cd415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd9600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd415bc40 0x7f5cd415d1e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c7a0 tx=0x7f5ccc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd1600640 1 -- 172.21.15.87:0/2807444419 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc003b80 con 0x7f5cd415bc40 2024-04-25T10:19:25.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd1600640 1 -- 172.21.15.87:0/2807444419 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc003d20 con 0x7f5cd415bc40 2024-04-25T10:19:25.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cd1600640 1 -- 172.21.15.87:0/2807444419 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00aa90 con 0x7f5cd415bc40 2024-04-25T10:19:25.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd415dab0 con 0x7f5cd415bc40 2024-04-25T10:19:25.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd4155660 con 0x7f5cd415bc40 2024-04-25T10:19:25.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.472+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd404ef30 con 0x7f5cd415bc40 2024-04-25T10:19:25.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.476+0000 7f5cd1600640 1 -- 172.21.15.87:0/2807444419 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ccc00ac30 con 0x7f5cd415bc40 2024-04-25T10:19:25.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.476+0000 7f5cd1600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cb0066f60 0x7f5cb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.476+0000 7f5cd1600640 1 -- 172.21.15.87:0/2807444419 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5ccc084fb0 con 0x7f5cd415bc40 2024-04-25T10:19:25.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.476+0000 7f5cd8c00640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cb0066f60 0x7f5cb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.476+0000 7f5cd8c00640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cb0066f60 0x7f5cb0069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c900 tx=0x7f5cc4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.480+0000 7f5cd1600640 1 -- 172.21.15.87:0/2807444419 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ccc055f90 con 0x7f5cd415bc40 2024-04-25T10:19:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.668+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5cd406d960 con 0x7f5cd415bc40 2024-04-25T10:19:25.670 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:19:25.668+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:19:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.668+0000 7f5cd1600640 1 -- 172.21.15.87:0/2807444419 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5ccc055890 con 0x7f5cd415bc40 2024-04-25T10:19:25.672 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cb0066f60 msgr2=0x7f5cb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cb0066f60 0x7f5cb0069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c900 tx=0x7f5cc4008040 comp rx=0 tx=0).stop 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd415bc40 msgr2=0x7f5cd415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd415bc40 0x7f5cd415d1e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c7a0 tx=0x7f5ccc00cc60 comp rx=0 tx=0).stop 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 shutdown_connections 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5cb0066f60 0x7f5cb0069420 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cd415bc40 0x7f5cd415d1e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd40cf930 0x7f5cd415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 --2- 172.21.15.87:0/2807444419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5cd40cda10 0x7f5cd41489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 >> 172.21.15.87:0/2807444419 conn(0x7f5cd4066b70 msgr2=0x7f5cd415a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 shutdown_connections 2024-04-25T10:19:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.672+0000 7f5cdb600640 1 -- 172.21.15.87:0/2807444419 wait complete. 2024-04-25T10:19:25.696 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:19:25.740 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:19:25.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2b2800640 1 Processor -- start 2024-04-25T10:19:25.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2b2800640 1 -- start start 2024-04-25T10:19:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2b2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2b2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb2ac0b31c0 0x7fb2ac0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2b2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ac15b080 con 0x7fb2ac0aef80 2024-04-25T10:19:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2b2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ac15adb0 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2b2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac15eb60 con 0x7fb2ac070350 2024-04-25T10:19:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.832+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:55106/0 (socket says 172.21.15.87:55106) 2024-04-25T10:19:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2a2a00640 1 -- 172.21.15.87:0/4083106745 learned_addr learned my addr 172.21.15.87:0/4083106745 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2abe00640 1 --2- 172.21.15.87:0/4083106745 >> v2:172.21.15.169:3300/0 conn(0x7fb2ac0b31c0 0x7fb2ac0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2a2a00640 1 -- 172.21.15.87:0/4083106745 >> v2:172.21.15.169:3300/0 conn(0x7fb2ac0b31c0 msgr2=0x7fb2ac0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2a2a00640 1 --2- 172.21.15.87:0/4083106745 >> v2:172.21.15.169:3300/0 conn(0x7fb2ac0b31c0 0x7fb2ac0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2a2a00640 1 -- 172.21.15.87:0/4083106745 >> v1:172.21.15.169:6789/0 conn(0x7fb2ac0aef80 legacy=0x7fb2ac15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2a2a00640 1 -- 172.21.15.87:0/4083106745 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac15ee40 con 0x7fb2ac070350 2024-04-25T10:19:25.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2a2a00640 1 --2- 172.21.15.87:0/4083106745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac15e480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb294009870 tx=0x7fb29401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d66d1467cd608e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2aaa00640 1 -- 172.21.15.87:0/4083106745 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2940a7070 con 0x7fb2ac070350 2024-04-25T10:19:25.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2aaa00640 1 -- 172.21.15.87:0/4083106745 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2940b0070 con 0x7fb2ac070350 2024-04-25T10:19:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.836+0000 7fb2aaa00640 1 -- 172.21.15.87:0/4083106745 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2940ab890 con 0x7fb2ac070350 2024-04-25T10:19:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- 172.21.15.87:0/4083106745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 msgr2=0x7fb2ac15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 --2- 172.21.15.87:0/4083106745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac15e480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb294009870 tx=0x7fb29401c560 comp rx=0 tx=0).stop 2024-04-25T10:19:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- 172.21.15.87:0/4083106745 shutdown_connections 2024-04-25T10:19:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 --2- 172.21.15.87:0/4083106745 >> v2:172.21.15.169:3300/0 conn(0x7fb2ac0b31c0 0x7fb2ac0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 --2- 172.21.15.87:0/4083106745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac15e480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- 172.21.15.87:0/4083106745 >> 172.21.15.87:0/4083106745 conn(0x7fb2ac0669a0 msgr2=0x7fb2ac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- 172.21.15.87:0/4083106745 shutdown_connections 2024-04-25T10:19:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- 172.21.15.87:0/4083106745 wait complete. 2024-04-25T10:19:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 Processor -- start 2024-04-25T10:19:25.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- start start 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ac0aef80 0x7fb2ac06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac0b31c0 0x7fb2ac06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac1679a0 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac167820 con 0x7fb2ac0aef80 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2b2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac167b20 con 0x7fb2ac070350 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac0b31c0 0x7fb2ac06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac0b31c0 0x7fb2ac06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37192/0 (socket says 172.21.15.87:37192) 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 -- 172.21.15.87:0/3719881657 learned_addr learned my addr 172.21.15.87:0/3719881657 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:25.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2ab400640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2a2a00640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ac0aef80 0x7fb2ac06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 -- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 msgr2=0x7fb2ac14a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac14a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 -- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ac0aef80 msgr2=0x7fb2ac06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ac0aef80 0x7fb2ac06c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 -- 172.21.15.87:0/3719881657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac14c380 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.840+0000 7fb2abe00640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac0b31c0 0x7fb2ac06c610 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009d30 tx=0x7fb2a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2a3e00640 1 -- 172.21.15.87:0/3719881657 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4018020 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2a3e00640 1 -- 172.21.15.87:0/3719881657 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a4014480 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2a3e00640 1 -- 172.21.15.87:0/3719881657 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4014870 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac14acc0 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac14bdc0 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac04ef30 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2a3e00640 1 -- 172.21.15.87:0/3719881657 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2a401b020 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2a3e00640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb288066f60 0x7fb288069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:25.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2a3e00640 1 -- 172.21.15.87:0/3719881657 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb2a40859f0 con 0x7fb2ac0b31c0 2024-04-25T10:19:25.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.844+0000 7fb2ab400640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb288066f60 0x7fb288069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:25.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.848+0000 7fb2ab400640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb288066f60 0x7fb288069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb29c001810 tx=0x7fb29c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:25.848+0000 7fb2a3e00640 1 -- 172.21.15.87:0/3719881657 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2a4022070 con 0x7fb2ac0b31c0 2024-04-25T10:19:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.036+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2ac15c050 con 0x7fb2ac0b31c0 2024-04-25T10:19:26.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2a3e00640 1 -- 172.21.15.87:0/3719881657 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2a4010040 con 0x7fb2ac0b31c0 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb288066f60 msgr2=0x7fb288069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb288066f60 0x7fb288069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb29c001810 tx=0x7fb29c009040 comp rx=0 tx=0).stop 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac0b31c0 msgr2=0x7fb2ac06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac0b31c0 0x7fb2ac06c610 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009d30 tx=0x7fb2a400c490 comp rx=0 tx=0).stop 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 shutdown_connections 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb288066f60 0x7fb288069420 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac0b31c0 0x7fb2ac06c610 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ac0aef80 0x7fb2ac06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 --2- 172.21.15.87:0/3719881657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ac070350 0x7fb2ac14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 >> 172.21.15.87:0/3719881657 conn(0x7fb2ac0669a0 msgr2=0x7fb2ac0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:26.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.672+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 shutdown_connections 2024-04-25T10:19:26.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.676+0000 7fb2b2800640 1 -- 172.21.15.87:0/3719881657 wait complete. 2024-04-25T10:19:26.699 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:19:26.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.876+0000 7fb5e3000640 1 Processor -- start 2024-04-25T10:19:26.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.876+0000 7fb5e3000640 1 -- start start 2024-04-25T10:19:26.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5e3000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5dc162310 0x7fb5dc15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:26.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5e3000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc15e9b0 con 0x7fb5dc0b4f50 2024-04-25T10:19:26.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5e3000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc15ec90 con 0x7fb5dc1618a0 2024-04-25T10:19:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5e3000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5dc15f250 con 0x7fb5dc162310 2024-04-25T10:19:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5dbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5dc162310 0x7fb5dc15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5e1000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb5dc0b4f50 0x7fb5dc0b0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58704/0 (socket says 172.21.15.87:58704) 2024-04-25T10:19:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5e1000640 1 -- 172.21.15.87:0/2890735051 learned_addr learned my addr 172.21.15.87:0/2890735051 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5dbe00640 1 -- 172.21.15.87:0/2890735051 >> v1:172.21.15.169:6789/0 conn(0x7fb5dc1618a0 legacy=0x7fb5dc0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5dbe00640 1 -- 172.21.15.87:0/2890735051 >> v1:172.21.15.87:6789/0 conn(0x7fb5dc0b4f50 legacy=0x7fb5dc0b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5db400640 1 -- 172.21.15.87:0/2890735051 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879423327 0 0) 0x7fb5dc15e9b0 con 0x7fb5dc0b4f50 2024-04-25T10:19:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.880+0000 7fb5dbe00640 1 -- 172.21.15.87:0/2890735051 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc15ef70 con 0x7fb5dc162310 2024-04-25T10:19:26.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5dbe00640 1 --2- 172.21.15.87:0/2890735051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc15e2b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc15f3d0 tx=0x7fb5cc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fde245262ce2048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:26.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5db400640 1 -- 172.21.15.87:0/2890735051 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0b0070 con 0x7fb5dc162310 2024-04-25T10:19:26.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5db400640 1 -- 172.21.15.87:0/2890735051 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc0ab600 con 0x7fb5dc162310 2024-04-25T10:19:26.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5db400640 1 -- 172.21.15.87:0/2890735051 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0ab9d0 con 0x7fb5dc162310 2024-04-25T10:19:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- 172.21.15.87:0/2890735051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 msgr2=0x7fb5dc15e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2890735051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc15e2b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc15f3d0 tx=0x7fb5cc01c380 comp rx=0 tx=0).stop 2024-04-25T10:19:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- 172.21.15.87:0/2890735051 shutdown_connections 2024-04-25T10:19:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2890735051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc15e2b0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- 172.21.15.87:0/2890735051 >> 172.21.15.87:0/2890735051 conn(0x7fb5dc0b5610 msgr2=0x7fb5dc0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- 172.21.15.87:0/2890735051 shutdown_connections 2024-04-25T10:19:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- 172.21.15.87:0/2890735051 wait complete. 2024-04-25T10:19:26.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 Processor -- start 2024-04-25T10:19:26.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- start start 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5dc0b4f50 0x7fb5dc158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5dc1618a0 0x7fb5dc152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0b1460 con 0x7fb5dc162310 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0b12e0 con 0x7fb5dc1618a0 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0afce0 con 0x7fb5dc0b4f50 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37208/0 (socket says 172.21.15.87:37208) 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.884+0000 7fb5e1000640 1 -- 172.21.15.87:0/2902015963 learned_addr learned my addr 172.21.15.87:0/2902015963 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5d3400640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5dc1618a0 0x7fb5dc152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5dbe00640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5dc0b4f50 0x7fb5dc158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e1000640 1 -- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5dc0b4f50 msgr2=0x7fb5dc158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e1000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5dc0b4f50 0x7fb5dc158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e1000640 1 -- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5dc1618a0 msgr2=0x7fb5dc152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e1000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5dc1618a0 0x7fb5dc152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e1000640 1 -- 172.21.15.87:0/2902015963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc0adb80 con 0x7fb5dc162310 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e1000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc153530 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400c7a0 tx=0x7fb5d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5d8c00640 1 -- 172.21.15.87:0/2902015963 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4018070 con 0x7fb5dc162310 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5d8c00640 1 -- 172.21.15.87:0/2902015963 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d400a640 con 0x7fb5dc162310 2024-04-25T10:19:26.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5d8c00640 1 -- 172.21.15.87:0/2902015963 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400aab0 con 0x7fb5dc162310 2024-04-25T10:19:26.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc14e360 con 0x7fb5dc162310 2024-04-25T10:19:26.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc0af5b0 con 0x7fb5dc162310 2024-04-25T10:19:26.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5dc04ef30 con 0x7fb5dc162310 2024-04-25T10:19:26.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5d8c00640 1 -- 172.21.15.87:0/2902015963 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5d400ac50 con 0x7fb5dc162310 2024-04-25T10:19:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5d8c00640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5b8066f60 0x7fb5b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.888+0000 7fb5d8c00640 1 -- 172.21.15.87:0/2902015963 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb5d4084630 con 0x7fb5dc162310 2024-04-25T10:19:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.892+0000 7fb5dbe00640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5b8066f60 0x7fb5b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.892+0000 7fb5dbe00640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5b8066f60 0x7fb5b8069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0ad000 tx=0x7fb5cc0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:26.892+0000 7fb5d8c00640 1 -- 172.21.15.87:0/2902015963 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5d4054f90 con 0x7fb5dc162310 2024-04-25T10:19:27.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.048+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb5dc0b02f0 con 0x7fb5dc162310 2024-04-25T10:19:27.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.676+0000 7fb5d8c00640 1 -- 172.21.15.87:0/2902015963 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb5d4059030 con 0x7fb5dc162310 2024-04-25T10:19:27.680 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5b8066f60 msgr2=0x7fb5b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5b8066f60 0x7fb5b8069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0ad000 tx=0x7fb5cc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 msgr2=0x7fb5dc153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc153530 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400c7a0 tx=0x7fb5d400cc60 comp rx=0 tx=0).stop 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 shutdown_connections 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5b8066f60 0x7fb5b8069420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5dc162310 0x7fb5dc153530 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5dc1618a0 0x7fb5dc152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 --2- 172.21.15.87:0/2902015963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5dc0b4f50 0x7fb5dc158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 >> 172.21.15.87:0/2902015963 conn(0x7fb5dc0b5610 msgr2=0x7fb5dc15f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 shutdown_connections 2024-04-25T10:19:27.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.680+0000 7fb5e3000640 1 -- 172.21.15.87:0/2902015963 wait complete. 2024-04-25T10:19:27.707 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:19:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b45200640 1 Processor -- start 2024-04-25T10:19:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b45200640 1 -- start start 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b45200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b400d00f0 0x7f6b400713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b45200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b4015bc00 0x7f6b400718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b45200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b40071f80 con 0x7f6b400cf040 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b45200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b40072260 con 0x7f6b400d00f0 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b45200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b400b3700 con 0x7f6b4015bc00 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b35400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b400d00f0 0x7f6b400713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b3de00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6b400cf040 0x7f6b400cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50400/0 (socket says 172.21.15.87:50400) 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b3de00640 1 -- 172.21.15.87:0/23605226 learned_addr learned my addr 172.21.15.87:0/23605226 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b3e800640 1 --2- 172.21.15.87:0/23605226 >> v2:172.21.15.169:3300/0 conn(0x7f6b4015bc00 0x7f6b400718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b35400640 1 -- 172.21.15.87:0/23605226 >> v2:172.21.15.169:3300/0 conn(0x7f6b4015bc00 msgr2=0x7f6b400718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b35400640 1 --2- 172.21.15.87:0/23605226 >> v2:172.21.15.169:3300/0 conn(0x7f6b4015bc00 0x7f6b400718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b35400640 1 -- 172.21.15.87:0/23605226 >> v1:172.21.15.87:6789/0 conn(0x7f6b400cf040 legacy=0x7f6b400cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:27.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b35400640 1 -- 172.21.15.87:0/23605226 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4015b120 con 0x7f6b400d00f0 2024-04-25T10:19:27.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.848+0000 7f6b3d400640 1 -- 172.21.15.87:0/23605226 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740054927 0 0) 0x7f6b40071f80 con 0x7f6b400cf040 2024-04-25T10:19:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b35400640 1 --2- 172.21.15.87:0/23605226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400d00f0 0x7f6b400713a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6b400b4e40 tx=0x7f6b2801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2578cce725c61982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b3d400640 1 -- 172.21.15.87:0/23605226 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280b0070 con 0x7f6b400d00f0 2024-04-25T10:19:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b3d400640 1 -- 172.21.15.87:0/23605226 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b280ab600 con 0x7f6b400d00f0 2024-04-25T10:19:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b3d400640 1 -- 172.21.15.87:0/23605226 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280ab9d0 con 0x7f6b400d00f0 2024-04-25T10:19:27.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 -- 172.21.15.87:0/23605226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400d00f0 msgr2=0x7f6b400713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:27.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 --2- 172.21.15.87:0/23605226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400d00f0 0x7f6b400713a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6b400b4e40 tx=0x7f6b2801c480 comp rx=0 tx=0).stop 2024-04-25T10:19:27.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 -- 172.21.15.87:0/23605226 shutdown_connections 2024-04-25T10:19:27.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 --2- 172.21.15.87:0/23605226 >> v2:172.21.15.169:3300/0 conn(0x7f6b4015bc00 0x7f6b400718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 --2- 172.21.15.87:0/23605226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400d00f0 0x7f6b400713a0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 -- 172.21.15.87:0/23605226 >> 172.21.15.87:0/23605226 conn(0x7f6b40066b70 msgr2=0x7f6b40066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:27.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 -- 172.21.15.87:0/23605226 shutdown_connections 2024-04-25T10:19:27.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.852+0000 7f6b45200640 1 -- 172.21.15.87:0/23605226 wait complete. 2024-04-25T10:19:27.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 Processor -- start 2024-04-25T10:19:27.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 -- start start 2024-04-25T10:19:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b400cf040 0x7f6b4014ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b400d00f0 0x7f6b401464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b4015bc00 0x7f6b4014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400aec90 con 0x7f6b4015bc00 2024-04-25T10:19:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400cf630 con 0x7f6b400d00f0 2024-04-25T10:19:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400aee10 con 0x7f6b400cf040 2024-04-25T10:19:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b4015bc00 0x7f6b4014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b4015bc00 0x7f6b4014b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41442/0 (socket says 172.21.15.87:41442) 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 -- 172.21.15.87:0/4002017548 learned_addr learned my addr 172.21.15.87:0/4002017548 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3de00640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b400cf040 0x7f6b4014ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b35400640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b400d00f0 0x7f6b401464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 -- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b400cf040 msgr2=0x7f6b4014ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b400cf040 0x7f6b4014ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 -- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b400d00f0 msgr2=0x7f6b401464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b400d00f0 0x7f6b401464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 -- 172.21.15.87:0/4002017548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4014bff0 con 0x7f6b4015bc00 2024-04-25T10:19:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b3e800640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b4015bc00 0x7f6b4014b530 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800d5a0 tx=0x7f6b3800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b36a00640 1 -- 172.21.15.87:0/4002017548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380125f0 con 0x7f6b4015bc00 2024-04-25T10:19:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b36a00640 1 -- 172.21.15.87:0/4002017548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b38019070 con 0x7f6b4015bc00 2024-04-25T10:19:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b36a00640 1 -- 172.21.15.87:0/4002017548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380129f0 con 0x7f6b4015bc00 2024-04-25T10:19:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4014a0a0 con 0x7f6b4015bc00 2024-04-25T10:19:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4014a660 con 0x7f6b4015bc00 2024-04-25T10:19:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.856+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4004ef30 con 0x7f6b4015bc00 2024-04-25T10:19:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.860+0000 7f6b36a00640 1 -- 172.21.15.87:0/4002017548 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b38012b90 con 0x7f6b4015bc00 2024-04-25T10:19:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.860+0000 7f6b36a00640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b1c066f60 0x7f6b1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.860+0000 7f6b36a00640 1 -- 172.21.15.87:0/4002017548 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f6b38085280 con 0x7f6b4015bc00 2024-04-25T10:19:27.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.860+0000 7f6b3de00640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b1c066f60 0x7f6b1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:27.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.860+0000 7f6b3de00640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b1c066f60 0x7f6b1c069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6b300012f0 tx=0x7f6b30009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:27.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:27.864+0000 7f6b36a00640 1 -- 172.21.15.87:0/4002017548 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b380563c0 con 0x7f6b4015bc00 2024-04-25T10:19:28.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.024+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b400b2a10 con 0x7f6b4015bc00 2024-04-25T10:19:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b36a00640 1 -- 172.21.15.87:0/4002017548 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6b38055cc0 con 0x7f6b4015bc00 2024-04-25T10:19:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:19:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b1c066f60 msgr2=0x7f6b1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b1c066f60 0x7f6b1c069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6b300012f0 tx=0x7f6b30009040 comp rx=0 tx=0).stop 2024-04-25T10:19:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b4015bc00 msgr2=0x7f6b4014b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b4015bc00 0x7f6b4014b530 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800d5a0 tx=0x7f6b3800da60 comp rx=0 tx=0).stop 2024-04-25T10:19:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 shutdown_connections 2024-04-25T10:19:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b1c066f60 0x7f6b1c069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b4015bc00 0x7f6b4014b530 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b400d00f0 0x7f6b401464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 --2- 172.21.15.87:0/4002017548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b400cf040 0x7f6b4014ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 >> 172.21.15.87:0/4002017548 conn(0x7f6b40066b70 msgr2=0x7f6b40066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 shutdown_connections 2024-04-25T10:19:28.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.680+0000 7f6b45200640 1 -- 172.21.15.87:0/4002017548 wait complete. 2024-04-25T10:19:28.713 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:19:28.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7238600640 1 Processor -- start 2024-04-25T10:19:28.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7238600640 1 -- start start 2024-04-25T10:19:28.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7238600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f723006cf40 0x7f723015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:28.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7238600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006d320 0x7f723015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:28.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7238600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72300cddf0 con 0x7f723006db90 2024-04-25T10:19:28.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7238600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723015de80 con 0x7f723006cf40 2024-04-25T10:19:28.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7238600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723015e440 con 0x7f723006d320 2024-04-25T10:19:28.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7236600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006d320 0x7f723015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7235200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f723006db90 0x7f723006ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:57928/0 (socket says 172.21.15.87:57928) 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7235c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f723006cf40 0x7f723015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.884+0000 7f7235200640 1 -- 172.21.15.87:0/3008832697 learned_addr learned my addr 172.21.15.87:0/3008832697 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f7236600640 1 -- 172.21.15.87:0/3008832697 >> v2:172.21.15.169:3300/0 conn(0x7f723006cf40 msgr2=0x7f723015e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f722fe00640 1 -- 172.21.15.87:0/3008832697 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877797528 0 0) 0x7f72300cddf0 con 0x7f723006db90 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f7236600640 1 --2- 172.21.15.87:0/3008832697 >> v2:172.21.15.169:3300/0 conn(0x7f723006cf40 0x7f723015e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f7236600640 1 -- 172.21.15.87:0/3008832697 >> v1:172.21.15.169:6789/0 conn(0x7f723006db90 legacy=0x7f723006ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:28.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f7236600640 1 -- 172.21.15.87:0/3008832697 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72300ce090 con 0x7f723006d320 2024-04-25T10:19:28.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f7236600640 1 --2- 172.21.15.87:0/3008832697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006d320 0x7f723015a7b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72280097c0 tx=0x7f722801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a2b0ae91707c570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:28.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f722fe00640 1 -- 172.21.15.87:0/3008832697 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280b0070 con 0x7f723006d320 2024-04-25T10:19:28.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f722fe00640 1 -- 172.21.15.87:0/3008832697 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72280ab600 con 0x7f723006d320 2024-04-25T10:19:28.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f722fe00640 1 -- 172.21.15.87:0/3008832697 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280ab9d0 con 0x7f723006d320 2024-04-25T10:19:28.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f7238600640 1 -- 172.21.15.87:0/3008832697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006d320 msgr2=0x7f723015a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:28.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.888+0000 7f7238600640 1 --2- 172.21.15.87:0/3008832697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006d320 0x7f723015a7b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72280097c0 tx=0x7f722801c410 comp rx=0 tx=0).stop 2024-04-25T10:19:28.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- 172.21.15.87:0/3008832697 shutdown_connections 2024-04-25T10:19:28.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 --2- 172.21.15.87:0/3008832697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006d320 0x7f723015a7b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 --2- 172.21.15.87:0/3008832697 >> v2:172.21.15.169:3300/0 conn(0x7f723006cf40 0x7f723015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- 172.21.15.87:0/3008832697 >> 172.21.15.87:0/3008832697 conn(0x7f7230072290 msgr2=0x7f72300726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:28.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- 172.21.15.87:0/3008832697 shutdown_connections 2024-04-25T10:19:28.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- 172.21.15.87:0/3008832697 wait complete. 2024-04-25T10:19:28.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 Processor -- start 2024-04-25T10:19:28.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- start start 2024-04-25T10:19:28.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723006cf40 0x7f7230152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:28.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f723006d320 0x7f72300c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:28.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006db90 0x7f723015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:28.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723015ae70 con 0x7f723006d320 2024-04-25T10:19:28.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723015acf0 con 0x7f723006cf40 2024-04-25T10:19:28.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7238600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723015aff0 con 0x7f723006db90 2024-04-25T10:19:28.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f723006d320 0x7f72300c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:28.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f723006d320 0x7f72300c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41448/0 (socket says 172.21.15.87:41448) 2024-04-25T10:19:28.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 -- 172.21.15.87:0/340158253 learned_addr learned my addr 172.21.15.87:0/340158253 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:28.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235c00640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723006cf40 0x7f7230152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:28.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7236600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006db90 0x7f723015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:28.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 -- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006db90 msgr2=0x7f723015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006db90 0x7f723015cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 -- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723006cf40 msgr2=0x7f7230152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723006cf40 0x7f7230152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 -- 172.21.15.87:0/340158253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723015ab20 con 0x7f723006d320 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235c00640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723006cf40 0x7f7230152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.892+0000 7f7235200640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f723006d320 0x7f72300c7d00 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f721c00c480 tx=0x7f721c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f722d600640 1 -- 172.21.15.87:0/340158253 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c018070 con 0x7f723006d320 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f722d600640 1 -- 172.21.15.87:0/340158253 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0074e0 con 0x7f723006d320 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f722d600640 1 -- 172.21.15.87:0/340158253 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0049f0 con 0x7f723006d320 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72300c17a0 con 0x7f723006d320 2024-04-25T10:19:28.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72300c1d60 con 0x7f723006d320 2024-04-25T10:19:28.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723004ef30 con 0x7f723006d320 2024-04-25T10:19:28.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f722d600640 1 -- 172.21.15.87:0/340158253 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f721c004b90 con 0x7f723006d320 2024-04-25T10:19:28.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f722d600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7210066f60 0x7f7210069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:28.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f722d600640 1 -- 172.21.15.87:0/340158253 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f721c083fc0 con 0x7f723006d320 2024-04-25T10:19:28.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.896+0000 7f7235c00640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7210066f60 0x7f7210069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:28.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.900+0000 7f7235c00640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7210066f60 0x7f7210069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f723009dd30 tx=0x7f722400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:28.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:28.900+0000 7f722d600640 1 -- 172.21.15.87:0/340158253 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f721c054c70 con 0x7f723006d320 2024-04-25T10:19:29.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.056+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7230072ff0 con 0x7f723006d320 2024-04-25T10:19:29.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.060+0000 7f722d600640 1 -- 172.21.15.87:0/340158253 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f721c058d10 con 0x7f723006d320 2024-04-25T10:19:29.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7210066f60 msgr2=0x7f7210069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7210066f60 0x7f7210069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f723009dd30 tx=0x7f722400b040 comp rx=0 tx=0).stop 2024-04-25T10:19:29.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f723006d320 msgr2=0x7f72300c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f723006d320 0x7f72300c7d00 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f721c00c480 tx=0x7f721c00c940 comp rx=0 tx=0).stop 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 shutdown_connections 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7210066f60 0x7f7210069420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723006db90 0x7f723015cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f723006d320 0x7f72300c7d00 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 --2- 172.21.15.87:0/340158253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723006cf40 0x7f7230152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 >> 172.21.15.87:0/340158253 conn(0x7f7230072290 msgr2=0x7f72300af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 shutdown_connections 2024-04-25T10:19:29.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.064+0000 7f7238600640 1 -- 172.21.15.87:0/340158253 wait complete. 2024-04-25T10:19:29.084 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:19:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.216+0000 7f68fdc00640 1 Processor -- start 2024-04-25T10:19:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.216+0000 7f68fdc00640 1 -- start start 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68fdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68fdc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f80d0250 con 0x7f68f80ae020 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68fdc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f80ad840 con 0x7f68f8070c20 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68fdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f815bb90 con 0x7f68f80cf930 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f68f80ae020 0x7f68f80b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50402/0 (socket says 172.21.15.87:50402) 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f7400640 1 -- 172.21.15.87:0/1876383347 learned_addr learned my addr 172.21.15.87:0/1876383347 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6a00640 1 --2- 172.21.15.87:0/1876383347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633374049 0 0) 0x7f68f80d0250 con 0x7f68f80ae020 2024-04-25T10:19:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68dc003530 con 0x7f68f80ae020 2024-04-25T10:19:29.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15390395 0 0) 0x7f68f80ad840 con 0x7f68f8070c20 2024-04-25T10:19:29.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f80d0250 con 0x7f68f8070c20 2024-04-25T10:19:29.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6a00640 1 -- 172.21.15.87:0/1876383347 >> v1:172.21.15.169:6789/0 conn(0x7f68f8070c20 legacy=0x7f68f815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6a00640 1 -- 172.21.15.87:0/1876383347 >> v1:172.21.15.87:6789/0 conn(0x7f68f80ae020 legacy=0x7f68f80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2458113531 0 0) 0x7f68dc003530 con 0x7f68f80ae020 2024-04-25T10:19:29.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.220+0000 7f68f6a00640 1 -- 172.21.15.87:0/1876383347 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f815e9f0 con 0x7f68f80cf930 2024-04-25T10:19:29.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68f6a00640 1 --2- 172.21.15.87:0/1876383347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f80cfd10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009dc0 tx=0x7f68e801c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc204016e22119cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:29.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80b2070 con 0x7f68f80cf930 2024-04-25T10:19:29.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80a5a20 con 0x7f68f80cf930 2024-04-25T10:19:29.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68f6000640 1 -- 172.21.15.87:0/1876383347 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80a5dd0 con 0x7f68f80cf930 2024-04-25T10:19:29.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- 172.21.15.87:0/1876383347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 msgr2=0x7f68f80cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 --2- 172.21.15.87:0/1876383347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f80cfd10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009dc0 tx=0x7f68e801c960 comp rx=0 tx=0).stop 2024-04-25T10:19:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- 172.21.15.87:0/1876383347 shutdown_connections 2024-04-25T10:19:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 --2- 172.21.15.87:0/1876383347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f80cfd10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- 172.21.15.87:0/1876383347 >> 172.21.15.87:0/1876383347 conn(0x7f68f8066b70 msgr2=0x7f68f8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- 172.21.15.87:0/1876383347 shutdown_connections 2024-04-25T10:19:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- 172.21.15.87:0/1876383347 wait complete. 2024-04-25T10:19:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 Processor -- start 2024-04-25T10:19:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- start start 2024-04-25T10:19:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f8070c20 0x7f68f8160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80ae020 0x7f68f8153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f8154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806e130 con 0x7f68f8070c20 2024-04-25T10:19:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f816c100 con 0x7f68f80ae020 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68fdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f816c280 con 0x7f68f80cf930 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.224+0000 7f68f6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f8070c20 0x7f68f8160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f8154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ee000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80ae020 0x7f68f8153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f8070c20 0x7f68f8160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41462/0 (socket says 172.21.15.87:41462) 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f8154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:44140/0 (socket says 172.21.15.87:44140) 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 -- 172.21.15.87:0/811789706 learned_addr learned my addr 172.21.15.87:0/811789706 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 -- 172.21.15.87:0/811789706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 msgr2=0x7f68f8154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f8154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 -- 172.21.15.87:0/811789706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80ae020 msgr2=0x7f68f8153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80ae020 0x7f68f8153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 -- 172.21.15.87:0/811789706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8152470 con 0x7f68f8070c20 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ee000640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80ae020 0x7f68f8153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68f6a00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f8070c20 0x7f68f8160700 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f68e80b0040 tx=0x7f68e8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ef400640 1 -- 172.21.15.87:0/811789706 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80b2050 con 0x7f68f8070c20 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ef400640 1 -- 172.21.15.87:0/811789706 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80be070 con 0x7f68f8070c20 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ef400640 1 -- 172.21.15.87:0/811789706 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80ada90 con 0x7f68f8070c20 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f8152750 con 0x7f68f8070c20 2024-04-25T10:19:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f8152d10 con 0x7f68f8070c20 2024-04-25T10:19:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f804ef30 con 0x7f68f8070c20 2024-04-25T10:19:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ef400640 1 -- 172.21.15.87:0/811789706 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68e80adc30 con 0x7f68f8070c20 2024-04-25T10:19:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ef400640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d4066f90 0x7f68d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.228+0000 7f68ef400640 1 -- 172.21.15.87:0/811789706 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f68e811f6f0 con 0x7f68f8070c20 2024-04-25T10:19:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.232+0000 7f68ee000640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d4066f90 0x7f68d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.232+0000 7f68ee000640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d4066f90 0x7f68d4069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f68f815b210 tx=0x7f68e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:29.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.232+0000 7f68ef400640 1 -- 172.21.15.87:0/811789706 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68e80c7020 con 0x7f68f8070c20 2024-04-25T10:19:29.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.440+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f68f806d7f0 con 0x7f68f8070c20 2024-04-25T10:19:29.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.688+0000 7f68ef400640 1 -- 172.21.15.87:0/811789706 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v351) v1 ==== 144+0+0 (secure 0 0 0) 0x7f68e8016580 con 0x7f68f8070c20 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d4066f90 msgr2=0x7f68d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d4066f90 0x7f68d4069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f68f815b210 tx=0x7f68e0009040 comp rx=0 tx=0).stop 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f8070c20 msgr2=0x7f68f8160700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f8070c20 0x7f68f8160700 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f68e80b0040 tx=0x7f68e8019700 comp rx=0 tx=0).stop 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 shutdown_connections 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f68d4066f90 0x7f68d4069450 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f80cf930 0x7f68f8154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f80ae020 0x7f68f8153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 --2- 172.21.15.87:0/811789706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68f8070c20 0x7f68f8160700 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 >> 172.21.15.87:0/811789706 conn(0x7f68f8066b70 msgr2=0x7f68f80b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 shutdown_connections 2024-04-25T10:19:29.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.692+0000 7f68fdc00640 1 -- 172.21.15.87:0/811789706 wait complete. 2024-04-25T10:19:29.714 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:19:29.714 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:19:29.715 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:29.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f2205800640 1 Processor -- start 2024-04-25T10:19:29.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f2205800640 1 -- start start 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f2205800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f220015bc40 0x7f220006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f2205800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22000cf930 0x7f22000cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f2205800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22000b2770 con 0x7f22000cda10 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f2205800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220006c670 con 0x7f22000cf930 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f2205800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220006c950 con 0x7f220015bc40 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21ff400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22000cf930 0x7f22000cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21fe000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f22000cda10 0x7f22000b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50418/0 (socket says 172.21.15.87:50418) 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21fe000640 1 -- 172.21.15.87:0/2897787720 learned_addr learned my addr 172.21.15.87:0/2897787720 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:29.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21fea00640 1 --2- 172.21.15.87:0/2897787720 >> v2:172.21.15.169:3300/0 conn(0x7f220015bc40 0x7f220006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21ff400640 1 -- 172.21.15.87:0/2897787720 >> v2:172.21.15.169:3300/0 conn(0x7f220015bc40 msgr2=0x7f220006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21ff400640 1 --2- 172.21.15.87:0/2897787720 >> v2:172.21.15.169:3300/0 conn(0x7f220015bc40 0x7f220006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21ff400640 1 -- 172.21.15.87:0/2897787720 >> v1:172.21.15.87:6789/0 conn(0x7f22000cda10 legacy=0x7f22000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21ff400640 1 -- 172.21.15.87:0/2897787720 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220006cc30 con 0x7f22000cf930 2024-04-25T10:19:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.848+0000 7f21fd600640 1 -- 172.21.15.87:0/2897787720 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105150597 0 0) 0x7f22000b2770 con 0x7f22000cda10 2024-04-25T10:19:29.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f21ff400640 1 --2- 172.21.15.87:0/2897787720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22000cf930 0x7f22000cfdc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f22000d0800 tx=0x7f21f801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc439cb024bcbce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:29.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f21fd600640 1 -- 172.21.15.87:0/2897787720 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80b0070 con 0x7f22000cf930 2024-04-25T10:19:29.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f21fd600640 1 -- 172.21.15.87:0/2897787720 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f80ab600 con 0x7f22000cf930 2024-04-25T10:19:29.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f21fd600640 1 -- 172.21.15.87:0/2897787720 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80ab9d0 con 0x7f22000cf930 2024-04-25T10:19:29.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 -- 172.21.15.87:0/2897787720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22000cf930 msgr2=0x7f22000cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 --2- 172.21.15.87:0/2897787720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22000cf930 0x7f22000cfdc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f22000d0800 tx=0x7f21f801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 -- 172.21.15.87:0/2897787720 shutdown_connections 2024-04-25T10:19:29.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 --2- 172.21.15.87:0/2897787720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22000cf930 0x7f22000cfdc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 --2- 172.21.15.87:0/2897787720 >> v2:172.21.15.169:3300/0 conn(0x7f220015bc40 0x7f220006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 -- 172.21.15.87:0/2897787720 >> 172.21.15.87:0/2897787720 conn(0x7f2200066b70 msgr2=0x7f2200066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:29.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 -- 172.21.15.87:0/2897787720 shutdown_connections 2024-04-25T10:19:29.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.852+0000 7f2205800640 1 -- 172.21.15.87:0/2897787720 wait complete. 2024-04-25T10:19:29.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 Processor -- start 2024-04-25T10:19:29.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 -- start start 2024-04-25T10:19:29.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22000cda10 0x7f220014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22000cf930 0x7f22000b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220015bc40 0x7f22000b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000b3850 con 0x7f220015bc40 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000b36d0 con 0x7f22000cda10 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000b39d0 con 0x7f22000cf930 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220015bc40 0x7f22000b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220015bc40 0x7f22000b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41486/0 (socket says 172.21.15.87:41486) 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 -- 172.21.15.87:0/3867595382 learned_addr learned my addr 172.21.15.87:0/3867595382 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21fea00640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22000cda10 0x7f220014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21fe000640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22000cf930 0x7f22000b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 -- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22000cf930 msgr2=0x7f22000b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22000cf930 0x7f22000b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 -- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22000cda10 msgr2=0x7f220014a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22000cda10 0x7f220014a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:29.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 -- 172.21.15.87:0/3867595382 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22000af8e0 con 0x7f220015bc40 2024-04-25T10:19:29.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21ff400640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220015bc40 0x7f22000b0f20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f21f8019000 tx=0x7f21f8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:29.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21f6a00640 1 -- 172.21.15.87:0/3867595382 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80b8020 con 0x7f220015bc40 2024-04-25T10:19:29.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21f6a00640 1 -- 172.21.15.87:0/3867595382 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f80b4610 con 0x7f220015bc40 2024-04-25T10:19:29.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f21f6a00640 1 -- 172.21.15.87:0/3867595382 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80b0040 con 0x7f220015bc40 2024-04-25T10:19:29.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22000afbc0 con 0x7f220015bc40 2024-04-25T10:19:29.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.856+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22000b0180 con 0x7f220015bc40 2024-04-25T10:19:29.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.860+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220004ef30 con 0x7f220015bc40 2024-04-25T10:19:29.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.860+0000 7f21f6a00640 1 -- 172.21.15.87:0/3867595382 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21f80bb020 con 0x7f220015bc40 2024-04-25T10:19:29.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.860+0000 7f21f6a00640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e0066f90 0x7f21e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:29.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.860+0000 7f21f6a00640 1 -- 172.21.15.87:0/3867595382 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f21f811e900 con 0x7f220015bc40 2024-04-25T10:19:29.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.860+0000 7f21fea00640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e0066f90 0x7f21e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:29.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.860+0000 7f21fea00640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e0066f90 0x7f21e0069450 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2200158620 tx=0x7f21f000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:29.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:29.864+0000 7f21f6a00640 1 -- 172.21.15.87:0/3867595382 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21f80c2030 con 0x7f220015bc40 2024-04-25T10:19:30.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.016+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22000b2770 con 0x7f220015bc40 2024-04-25T10:19:30.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f21f6a00640 1 -- 172.21.15.87:0/3867595382 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13009 (secure 0 0 0) 0x7f21f8016070 con 0x7f220015bc40 2024-04-25T10:19:30.021 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:30.021 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":92,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:28.679810+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/3032837514":"2024-04-26T10:19:25.670169+0000","172.21.15.87:6835/3032837514":"2024-04-26T10:19:25.670169+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e0066f90 msgr2=0x7f21e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e0066f90 0x7f21e0069450 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2200158620 tx=0x7f21f000b040 comp rx=0 tx=0).stop 2024-04-25T10:19:30.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220015bc40 msgr2=0x7f22000b0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220015bc40 0x7f22000b0f20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f21f8019000 tx=0x7f21f8019190 comp rx=0 tx=0).stop 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 shutdown_connections 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f21e0066f90 0x7f21e0069450 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220015bc40 0x7f22000b0f20 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22000cf930 0x7f22000b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 --2- 172.21.15.87:0/3867595382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22000cda10 0x7f220014a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 >> 172.21.15.87:0/3867595382 conn(0x7f2200066b70 msgr2=0x7f2200066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 shutdown_connections 2024-04-25T10:19:30.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.020+0000 7f2205800640 1 -- 172.21.15.87:0/3867595382 wait complete. 2024-04-25T10:19:30.042 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:30.042 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:30.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12df200640 1 Processor -- start 2024-04-25T10:19:30.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12df200640 1 -- start start 2024-04-25T10:19:30.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12df200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f12e0071db0 0x7f12e0165730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12df200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00b3470 con 0x7f12e0071260 2024-04-25T10:19:30.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12df200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e006de60 con 0x7f12e015a780 2024-04-25T10:19:30.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12df200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e006cdc0 con 0x7f12e0071db0 2024-04-25T10:19:30.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12dd400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f12e0071260 0x7f12e00716a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50434/0 (socket says 172.21.15.87:50434) 2024-04-25T10:19:30.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12de800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f12e0071db0 0x7f12e0165730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12dd400640 1 -- 172.21.15.87:0/2281737899 learned_addr learned my addr 172.21.15.87:0/2281737899 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:30.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12de800640 1 -- 172.21.15.87:0/2281737899 >> v1:172.21.15.169:6789/0 conn(0x7f12e015a780 legacy=0x7f12e015edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12de800640 1 -- 172.21.15.87:0/2281737899 >> v1:172.21.15.87:6789/0 conn(0x7f12e0071260 legacy=0x7f12e00716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12dca00640 1 -- 172.21.15.87:0/2281737899 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458956244 0 0) 0x7f12cc001120 con 0x7f12e0071260 2024-04-25T10:19:30.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.180+0000 7f12de800640 1 -- 172.21.15.87:0/2281737899 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e006d0a0 con 0x7f12e0071db0 2024-04-25T10:19:30.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12de800640 1 --2- 172.21.15.87:0/2281737899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e0165730 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009d90 tx=0x7f12d801eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cc9a07bff965fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:30.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12dca00640 1 -- 172.21.15.87:0/2281737899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80b4070 con 0x7f12e0071db0 2024-04-25T10:19:30.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12dca00640 1 -- 172.21.15.87:0/2281737899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80a7950 con 0x7f12e0071db0 2024-04-25T10:19:30.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12dca00640 1 -- 172.21.15.87:0/2281737899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a7d20 con 0x7f12e0071db0 2024-04-25T10:19:30.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 -- 172.21.15.87:0/2281737899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 msgr2=0x7f12e0165730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 --2- 172.21.15.87:0/2281737899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e0165730 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009d90 tx=0x7f12d801eb80 comp rx=0 tx=0).stop 2024-04-25T10:19:30.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 -- 172.21.15.87:0/2281737899 shutdown_connections 2024-04-25T10:19:30.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 --2- 172.21.15.87:0/2281737899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e0165730 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 -- 172.21.15.87:0/2281737899 >> 172.21.15.87:0/2281737899 conn(0x7f12e00743f0 msgr2=0x7f12e00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:30.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 -- 172.21.15.87:0/2281737899 shutdown_connections 2024-04-25T10:19:30.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 -- 172.21.15.87:0/2281737899 wait complete. 2024-04-25T10:19:30.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.184+0000 7f12df200640 1 Processor -- start 2024-04-25T10:19:30.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 -- start start 2024-04-25T10:19:30.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0071260 0x7f12e0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12e015a780 0x7f12e01f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0070ae0 con 0x7f12e0071db0 2024-04-25T10:19:30.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0070960 con 0x7f12e0071260 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0070c60 con 0x7f12e015a780 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e015d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41496/0 (socket says 172.21.15.87:41496) 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 -- 172.21.15.87:0/2205153558 learned_addr learned my addr 172.21.15.87:0/2205153558 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dde00640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0071260 0x7f12e0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12de800640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12e015a780 0x7f12e01f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 -- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12e015a780 msgr2=0x7f12e01f9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12e015a780 0x7f12e01f9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 -- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0071260 msgr2=0x7f12e0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0071260 0x7f12e0156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 -- 172.21.15.87:0/2205153558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00c52c0 con 0x7f12e0071db0 2024-04-25T10:19:30.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12dd400640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e015d5a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00c7b0 tx=0x7f12cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:30.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12d6000640 1 -- 172.21.15.87:0/2205153558 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc018070 con 0x7f12e0071db0 2024-04-25T10:19:30.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12d6000640 1 -- 172.21.15.87:0/2205153558 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc0072d0 con 0x7f12e0071db0 2024-04-25T10:19:30.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e00c55a0 con 0x7f12e0071db0 2024-04-25T10:19:30.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12d6000640 1 -- 172.21.15.87:0/2205153558 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0047f0 con 0x7f12e0071db0 2024-04-25T10:19:30.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e01566a0 con 0x7f12e0071db0 2024-04-25T10:19:30.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.188+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e0075120 con 0x7f12e0071db0 2024-04-25T10:19:30.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.192+0000 7f12d6000640 1 -- 172.21.15.87:0/2205153558 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12cc01b070 con 0x7f12e0071db0 2024-04-25T10:19:30.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.192+0000 7f12d6000640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12c0066f90 0x7f12c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.192+0000 7f12d6000640 1 -- 172.21.15.87:0/2205153558 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f12cc083f80 con 0x7f12e0071db0 2024-04-25T10:19:30.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.192+0000 7f12dde00640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12c0066f90 0x7f12c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.192+0000 7f12dde00640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12c0066f90 0x7f12c0069450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f12e01f9aa0 tx=0x7f12d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.196+0000 7f12d6000640 1 -- 172.21.15.87:0/2205153558 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12e00b3470 con 0x7f12e0071db0 2024-04-25T10:19:30.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.348+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12e006c8b0 con 0x7f12e0071db0 2024-04-25T10:19:30.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12d6000640 1 -- 172.21.15.87:0/2205153558 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12cc0076c0 con 0x7f12e0071db0 2024-04-25T10:19:30.353 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12c0066f90 msgr2=0x7f12c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12c0066f90 0x7f12c0069450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f12e01f9aa0 tx=0x7f12d0009040 comp rx=0 tx=0).stop 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 msgr2=0x7f12e015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e015d5a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00c7b0 tx=0x7f12cc00cc70 comp rx=0 tx=0).stop 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 shutdown_connections 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12c0066f90 0x7f12c0069450 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12e015a780 0x7f12e01f9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12e0071db0 0x7f12e015d5a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 --2- 172.21.15.87:0/2205153558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0071260 0x7f12e0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 >> 172.21.15.87:0/2205153558 conn(0x7f12e00743f0 msgr2=0x7f12e00afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:30.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 shutdown_connections 2024-04-25T10:19:30.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.352+0000 7f12df200640 1 -- 172.21.15.87:0/2205153558 wait complete. 2024-04-25T10:19:30.373 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:19:30.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.504+0000 7f8d67e00640 1 Processor -- start 2024-04-25T10:19:30.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67e00640 1 -- start start 2024-04-25T10:19:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d6815b9b0 0x7f8d680ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d680ce700 0x7f8d680ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d680cd120 con 0x7f8d680cf380 2024-04-25T10:19:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d680d2600 con 0x7f8d680ce700 2024-04-25T10:19:30.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d680cd400 con 0x7f8d6815b9b0 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8d680cf380 0x7f8d680cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50450/0 (socket says 172.21.15.87:50450) 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d5e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d680ce700 0x7f8d680ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d67400640 1 -- 172.21.15.87:0/3616744258 learned_addr learned my addr 172.21.15.87:0/3616744258 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d66a00640 1 --2- 172.21.15.87:0/3616744258 >> v2:172.21.15.169:3300/0 conn(0x7f8d6815b9b0 0x7f8d680ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d5e000640 1 -- 172.21.15.87:0/3616744258 >> v2:172.21.15.169:3300/0 conn(0x7f8d6815b9b0 msgr2=0x7f8d680ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d5e000640 1 --2- 172.21.15.87:0/3616744258 >> v2:172.21.15.169:3300/0 conn(0x7f8d6815b9b0 0x7f8d680ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d66000640 1 -- 172.21.15.87:0/3616744258 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221676710 0 0) 0x7f8d680cd120 con 0x7f8d680cf380 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d5e000640 1 -- 172.21.15.87:0/3616744258 >> v1:172.21.15.87:6789/0 conn(0x7f8d680cf380 legacy=0x7f8d680cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.508+0000 7f8d5e000640 1 -- 172.21.15.87:0/3616744258 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680b47a0 con 0x7f8d680ce700 2024-04-25T10:19:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d5e000640 1 --2- 172.21.15.87:0/3616744258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680ce700 0x7f8d680ceb90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f8d68070b60 tx=0x7f8d5001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d830f9138db6b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d66000640 1 -- 172.21.15.87:0/3616744258 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500b5070 con 0x7f8d680ce700 2024-04-25T10:19:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d66000640 1 -- 172.21.15.87:0/3616744258 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d500a8970 con 0x7f8d680ce700 2024-04-25T10:19:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d66000640 1 -- 172.21.15.87:0/3616744258 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500a8d40 con 0x7f8d680ce700 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 -- 172.21.15.87:0/3616744258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680ce700 msgr2=0x7f8d680ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 --2- 172.21.15.87:0/3616744258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680ce700 0x7f8d680ceb90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f8d68070b60 tx=0x7f8d5001ec70 comp rx=0 tx=0).stop 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 -- 172.21.15.87:0/3616744258 shutdown_connections 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 --2- 172.21.15.87:0/3616744258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680ce700 0x7f8d680ceb90 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 --2- 172.21.15.87:0/3616744258 >> v2:172.21.15.169:3300/0 conn(0x7f8d6815b9b0 0x7f8d680ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 -- 172.21.15.87:0/3616744258 >> 172.21.15.87:0/3616744258 conn(0x7f8d680d17b0 msgr2=0x7f8d680d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 -- 172.21.15.87:0/3616744258 shutdown_connections 2024-04-25T10:19:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 -- 172.21.15.87:0/3616744258 wait complete. 2024-04-25T10:19:30.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 Processor -- start 2024-04-25T10:19:30.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.512+0000 7f8d67e00640 1 -- start start 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d680ce700 0x7f8d680cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680cf380 0x7f8d680c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d6815b9b0 0x7f8d6814da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815bf10 con 0x7f8d680cf380 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815bd90 con 0x7f8d680ce700 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680ccd30 con 0x7f8d6815b9b0 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680cf380 0x7f8d680c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680cf380 0x7f8d680c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41526/0 (socket says 172.21.15.87:41526) 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 -- 172.21.15.87:0/2209860640 learned_addr learned my addr 172.21.15.87:0/2209860640 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:30.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d66a00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d680ce700 0x7f8d680cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67400640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d6815b9b0 0x7f8d6814da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 -- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d6815b9b0 msgr2=0x7f8d6814da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d6815b9b0 0x7f8d6814da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 -- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d680ce700 msgr2=0x7f8d680cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d680ce700 0x7f8d680cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 -- 172.21.15.87:0/2209860640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68153f80 con 0x7f8d680cf380 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67400640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d6815b9b0 0x7f8d6814da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:30.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5e000640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680cf380 0x7f8d680c8a50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8d500b3040 tx=0x7f8d5001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:30.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5f400640 1 -- 172.21.15.87:0/2209860640 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500bf070 con 0x7f8d680cf380 2024-04-25T10:19:30.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5f400640 1 -- 172.21.15.87:0/2209860640 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d500b0640 con 0x7f8d680cf380 2024-04-25T10:19:30.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5f400640 1 -- 172.21.15.87:0/2209860640 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500b5040 con 0x7f8d680cf380 2024-04-25T10:19:30.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d680cb870 con 0x7f8d680cf380 2024-04-25T10:19:30.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d681568f0 con 0x7f8d680cf380 2024-04-25T10:19:30.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d68075120 con 0x7f8d680cf380 2024-04-25T10:19:30.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.516+0000 7f8d5f400640 1 -- 172.21.15.87:0/2209860640 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d500b0a30 con 0x7f8d680cf380 2024-04-25T10:19:30.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.520+0000 7f8d5f400640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d44066f60 0x7f8d44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:30.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.520+0000 7f8d5f400640 1 -- 172.21.15.87:0/2209860640 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d50121640 con 0x7f8d680cf380 2024-04-25T10:19:30.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.520+0000 7f8d66a00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d44066f60 0x7f8d44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:30.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.520+0000 7f8d66a00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d44066f60 0x7f8d44069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009a50 tx=0x7f8d58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:30.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.524+0000 7f8d5f400640 1 -- 172.21.15.87:0/2209860640 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d500187d0 con 0x7f8d680cf380 2024-04-25T10:19:30.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:30.732+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f8d6815a690 con 0x7f8d680cf380 2024-04-25T10:19:31.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d5f400640 1 -- 172.21.15.87:0/2209860640 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v353) v1 ==== 158+0+0 (secure 0 0 0) 0x7f8d500f2940 con 0x7f8d680cf380 2024-04-25T10:19:31.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d44066f60 msgr2=0x7f8d44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:31.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d44066f60 0x7f8d44069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009a50 tx=0x7f8d58008040 comp rx=0 tx=0).stop 2024-04-25T10:19:31.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680cf380 msgr2=0x7f8d680c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:31.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680cf380 0x7f8d680c8a50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8d500b3040 tx=0x7f8d5001ba10 comp rx=0 tx=0).stop 2024-04-25T10:19:31.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 shutdown_connections 2024-04-25T10:19:31.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d44066f60 0x7f8d44069420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d6815b9b0 0x7f8d6814da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d680cf380 0x7f8d680c8a50 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 --2- 172.21.15.87:0/2209860640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d680ce700 0x7f8d680cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 >> 172.21.15.87:0/2209860640 conn(0x7f8d680d17b0 msgr2=0x7f8d680d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:31.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 shutdown_connections 2024-04-25T10:19:31.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.700+0000 7f8d67e00640 1 -- 172.21.15.87:0/2209860640 wait complete. 2024-04-25T10:19:31.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:19:31.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.864+0000 7f4396600640 1 Processor -- start 2024-04-25T10:19:31.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.864+0000 7f4396600640 1 -- start start 2024-04-25T10:19:31.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4396600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f43900aeda0 0x7f4390071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:31.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4396600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4390072020 0x7f4390070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:31.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4396600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4390159920 con 0x7f4390159540 2024-04-25T10:19:31.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4396600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439015c2f0 con 0x7f43900aeda0 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4396600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439015c5d0 con 0x7f4390072020 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4386a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f43900aeda0 0x7f4390071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f438f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4390159540 0x7f43900ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50460/0 (socket says 172.21.15.87:50460) 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f438f400640 1 -- 172.21.15.87:0/1578657503 learned_addr learned my addr 172.21.15.87:0/1578657503 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f438fe00640 1 --2- 172.21.15.87:0/1578657503 >> v2:172.21.15.169:3300/0 conn(0x7f4390072020 0x7f4390070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4386a00640 1 -- 172.21.15.87:0/1578657503 >> v2:172.21.15.169:3300/0 conn(0x7f4390072020 msgr2=0x7f4390070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4386a00640 1 --2- 172.21.15.87:0/1578657503 >> v2:172.21.15.169:3300/0 conn(0x7f4390072020 0x7f4390070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4386a00640 1 -- 172.21.15.87:0/1578657503 >> v1:172.21.15.87:6789/0 conn(0x7f4390159540 legacy=0x7f43900ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f438ea00640 1 -- 172.21.15.87:0/1578657503 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212001924 0 0) 0x7f4390159920 con 0x7f4390159540 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f4386a00640 1 -- 172.21.15.87:0/1578657503 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439015c8b0 con 0x7f43900aeda0 2024-04-25T10:19:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.868+0000 7f438fe00640 1 --2- 172.21.15.87:0/1578657503 >> v2:172.21.15.169:3300/0 conn(0x7f4390072020 0x7f4390070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4386a00640 1 --2- 172.21.15.87:0/1578657503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f4390071ae0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f43900cd230 tx=0x7f437801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad06014a16951e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f438ea00640 1 -- 172.21.15.87:0/1578657503 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43780a7070 con 0x7f43900aeda0 2024-04-25T10:19:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f438ea00640 1 -- 172.21.15.87:0/1578657503 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43780b0070 con 0x7f43900aeda0 2024-04-25T10:19:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f438ea00640 1 -- 172.21.15.87:0/1578657503 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43780ab890 con 0x7f43900aeda0 2024-04-25T10:19:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 -- 172.21.15.87:0/1578657503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 msgr2=0x7f4390071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 --2- 172.21.15.87:0/1578657503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f4390071ae0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f43900cd230 tx=0x7f437801c450 comp rx=0 tx=0).stop 2024-04-25T10:19:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 -- 172.21.15.87:0/1578657503 shutdown_connections 2024-04-25T10:19:31.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 --2- 172.21.15.87:0/1578657503 >> v2:172.21.15.169:3300/0 conn(0x7f4390072020 0x7f4390070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 --2- 172.21.15.87:0/1578657503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f4390071ae0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 -- 172.21.15.87:0/1578657503 >> 172.21.15.87:0/1578657503 conn(0x7f43900669a0 msgr2=0x7f4390066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:31.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 -- 172.21.15.87:0/1578657503 shutdown_connections 2024-04-25T10:19:31.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 -- 172.21.15.87:0/1578657503 wait complete. 2024-04-25T10:19:31.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 Processor -- start 2024-04-25T10:19:31.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.872+0000 7f4396600640 1 -- start start 2024-04-25T10:19:31.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4390072020 0x7f43901586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f439014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4390159540 0x7f43901571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43900710a0 con 0x7f43900aeda0 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4390072430 con 0x7f4390159540 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4390071220 con 0x7f4390072020 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f439014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f439014c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41534/0 (socket says 172.21.15.87:41534) 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 -- 172.21.15.87:0/344748455 learned_addr learned my addr 172.21.15.87:0/344748455 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f438f400640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4390072020 0x7f43901586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f438fe00640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4390159540 0x7f43901571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:31.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 -- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4390072020 msgr2=0x7f43901586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:31.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4390072020 0x7f43901586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 -- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4390159540 msgr2=0x7f43901571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:31.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4390159540 0x7f43901571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:31.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 -- 172.21.15.87:0/344748455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439014d990 con 0x7f43900aeda0 2024-04-25T10:19:31.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f438fe00640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4390159540 0x7f43901571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:31.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f438f400640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4390072020 0x7f43901586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:31.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4386a00640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f439014c400 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f4378019060 tx=0x7f437801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:31.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4387e00640 1 -- 172.21.15.87:0/344748455 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378004220 con 0x7f43900aeda0 2024-04-25T10:19:31.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4387e00640 1 -- 172.21.15.87:0/344748455 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43780043c0 con 0x7f43900aeda0 2024-04-25T10:19:31.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439014dc70 con 0x7f43900aeda0 2024-04-25T10:19:31.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4387e00640 1 -- 172.21.15.87:0/344748455 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378003820 con 0x7f43900aeda0 2024-04-25T10:19:31.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43901524e0 con 0x7f43900aeda0 2024-04-25T10:19:31.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439004ef30 con 0x7f43900aeda0 2024-04-25T10:19:31.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.876+0000 7f4387e00640 1 -- 172.21.15.87:0/344748455 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43780039c0 con 0x7f43900aeda0 2024-04-25T10:19:31.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.880+0000 7f4387e00640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f436c066f90 0x7f436c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:31.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.880+0000 7f4387e00640 1 -- 172.21.15.87:0/344748455 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f437811f620 con 0x7f43900aeda0 2024-04-25T10:19:31.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.880+0000 7f438f400640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f436c066f90 0x7f436c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:31.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.880+0000 7f438f400640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f436c066f90 0x7f436c069450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4380007fd0 tx=0x7f4380009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:31.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:31.884+0000 7f4387e00640 1 -- 172.21.15.87:0/344748455 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4378016070 con 0x7f43900aeda0 2024-04-25T10:19:32.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.036+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f43900cf470 con 0x7f43900aeda0 2024-04-25T10:19:32.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.040+0000 7f4387e00640 1 -- 172.21.15.87:0/344748455 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f43780f0920 con 0x7f43900aeda0 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f436c066f90 msgr2=0x7f436c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f436c066f90 0x7f436c069450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4380007fd0 tx=0x7f4380009040 comp rx=0 tx=0).stop 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 msgr2=0x7f439014c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f439014c400 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f4378019060 tx=0x7f437801ac90 comp rx=0 tx=0).stop 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 shutdown_connections 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f436c066f90 0x7f436c069450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4390159540 0x7f43901571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43900aeda0 0x7f439014c400 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 --2- 172.21.15.87:0/344748455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4390072020 0x7f43901586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 >> 172.21.15.87:0/344748455 conn(0x7f43900669a0 msgr2=0x7f4390066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 shutdown_connections 2024-04-25T10:19:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.044+0000 7f4396600640 1 -- 172.21.15.87:0/344748455 wait complete. 2024-04-25T10:19:32.066 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:19:32.066 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:32.163 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:32.255 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:19:32.255 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:19:32.255 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:32.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:32.264 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:19:32.264 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:32.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:32.274 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:19:32.274 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:32.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:32.312 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:19:32.312 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:32.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:32.329 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:19:32.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:32.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.492+0000 7fad52800640 1 Processor -- start 2024-04-25T10:19:32.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.492+0000 7fad52800640 1 -- start start 2024-04-25T10:19:32.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad52800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad52800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad4c0cf930 0x7fad4c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad52800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4c0b2770 con 0x7fad4c15bc40 2024-04-25T10:19:32.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad52800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4c06c670 con 0x7fad4c0cf930 2024-04-25T10:19:32.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad52800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c06c950 con 0x7fad4c0cda10 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad4c0cf930 0x7fad4c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad4c0cf930 0x7fad4c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41546/0 (socket says 172.21.15.87:41546) 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 -- 172.21.15.87:0/1494792304 learned_addr learned my addr 172.21.15.87:0/1494792304 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4aa00640 1 --2- 172.21.15.87:0/1494792304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 -- 172.21.15.87:0/1494792304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 msgr2=0x7fad4c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 --2- 172.21.15.87:0/1494792304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 -- 172.21.15.87:0/1494792304 >> v1:172.21.15.169:6789/0 conn(0x7fad4c15bc40 legacy=0x7fad4c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 -- 172.21.15.87:0/1494792304 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c06cc30 con 0x7fad4c0cf930 2024-04-25T10:19:32.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4be00640 1 --2- 172.21.15.87:0/1494792304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c0cf930 0x7fad4c0cfde0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fad44009870 tx=0x7fad4401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357bce1cd043258b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:32.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.496+0000 7fad4a000640 1 -- 172.21.15.87:0/1494792304 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440b0070 con 0x7fad4c0cf930 2024-04-25T10:19:32.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad4a000640 1 -- 172.21.15.87:0/1494792304 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad440ab600 con 0x7fad4c0cf930 2024-04-25T10:19:32.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad4a000640 1 -- 172.21.15.87:0/1494792304 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440ab9d0 con 0x7fad4c0cf930 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- 172.21.15.87:0/1494792304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c0cf930 msgr2=0x7fad4c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 --2- 172.21.15.87:0/1494792304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c0cf930 0x7fad4c0cfde0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fad44009870 tx=0x7fad4401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- 172.21.15.87:0/1494792304 shutdown_connections 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 --2- 172.21.15.87:0/1494792304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c0cf930 0x7fad4c0cfde0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 --2- 172.21.15.87:0/1494792304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- 172.21.15.87:0/1494792304 >> 172.21.15.87:0/1494792304 conn(0x7fad4c066b70 msgr2=0x7fad4c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- 172.21.15.87:0/1494792304 shutdown_connections 2024-04-25T10:19:32.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- 172.21.15.87:0/1494792304 wait complete. 2024-04-25T10:19:32.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 Processor -- start 2024-04-25T10:19:32.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- start start 2024-04-25T10:19:32.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c13f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0cf930 0x7fad4c141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c15bc40 0x7fad4c13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0b3850 con 0x7fad4c15bc40 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0b36d0 con 0x7fad4c0cf930 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad52800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0b39d0 con 0x7fad4c0cda10 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c15bc40 0x7fad4c13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad4b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c13f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad4be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c15bc40 0x7fad4c13ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41560/0 (socket says 172.21.15.87:41560) 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad4be00640 1 -- 172.21.15.87:0/310303283 learned_addr learned my addr 172.21.15.87:0/310303283 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.500+0000 7fad4aa00640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0cf930 0x7fad4c141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad4be00640 1 -- 172.21.15.87:0/310303283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 msgr2=0x7fad4c13f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad4be00640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c13f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad4be00640 1 -- 172.21.15.87:0/310303283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0cf930 msgr2=0x7fad4c141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad4be00640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0cf930 0x7fad4c141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad4be00640 1 -- 172.21.15.87:0/310303283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c13f830 con 0x7fad4c15bc40 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad4be00640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c15bc40 0x7fad4c13ea90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fad440ae040 tx=0x7fad44019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad43400640 1 -- 172.21.15.87:0/310303283 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440b0040 con 0x7fad4c15bc40 2024-04-25T10:19:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad43400640 1 -- 172.21.15.87:0/310303283 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad440ba070 con 0x7fad4c15bc40 2024-04-25T10:19:32.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad43400640 1 -- 172.21.15.87:0/310303283 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440b4850 con 0x7fad4c15bc40 2024-04-25T10:19:32.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c0b0c10 con 0x7fad4c15bc40 2024-04-25T10:19:32.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c13efd0 con 0x7fad4c15bc40 2024-04-25T10:19:32.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4c04ef30 con 0x7fad4c15bc40 2024-04-25T10:19:32.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad43400640 1 -- 172.21.15.87:0/310303283 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad440b49f0 con 0x7fad4c15bc40 2024-04-25T10:19:32.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad43400640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad2c066f60 0x7fad2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad43400640 1 -- 172.21.15.87:0/310303283 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad4411e340 con 0x7fad4c15bc40 2024-04-25T10:19:32.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.504+0000 7fad4b400640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad2c066f60 0x7fad2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.508+0000 7fad4b400640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad2c066f60 0x7fad2c069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001e00 tx=0x7fad3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:32.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.508+0000 7fad43400640 1 -- 172.21.15.87:0/310303283 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad440c3020 con 0x7fad4c15bc40 2024-04-25T10:19:32.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.692+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad4c1585d0 con 0x7fad4c15bc40 2024-04-25T10:19:32.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.692+0000 7fad43400640 1 -- 172.21.15.87:0/310303283 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fad4c0b0c10 con 0x7fad4c15bc40 2024-04-25T10:19:32.696 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:32.696 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2451052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2451052721},{"type":"v1","addr":"172.21.15.169:6834","nonce":2451052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/988712853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":988712853},{"type":"v1","addr":"172.21.15.169:6835","nonce":988712853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/2178548331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2178548331},{"type":"v1","addr":"172.21.15.87:6836","nonce":2178548331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3969249507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3969249507},{"type":"v1","addr":"172.21.15.87:6837","nonce":3969249507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[]} 2024-04-25T10:19:32.696 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 65 2024-04-25T10:19:32.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad2c066f60 msgr2=0x7fad2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad2c066f60 0x7fad2c069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001e00 tx=0x7fad3c009040 comp rx=0 tx=0).stop 2024-04-25T10:19:32.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c15bc40 msgr2=0x7fad4c13ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c15bc40 0x7fad4c13ea90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fad440ae040 tx=0x7fad44019270 comp rx=0 tx=0).stop 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 shutdown_connections 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad2c066f60 0x7fad2c069420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad4c15bc40 0x7fad4c13ea90 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad4c0cf930 0x7fad4c141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 --2- 172.21.15.87:0/310303283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad4c0cda10 0x7fad4c13f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 >> 172.21.15.87:0/310303283 conn(0x7fad4c066b70 msgr2=0x7fad4c159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 shutdown_connections 2024-04-25T10:19:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.696+0000 7fad52800640 1 -- 172.21.15.87:0/310303283 wait complete. 2024-04-25T10:19:32.718 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:19:32.718 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:19:32.719 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:19:32.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:32.719 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:19:32.719 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:19:32.719 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:19:32.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:32.720 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:19:32.720 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:19:32.720 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:19:32.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:32.720 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:19:32.721 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:19:32.721 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:19:32.721 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:32.730 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:19:32.731 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:19:32.733 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:19:32.733 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:19:32.735 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:19:32.735 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:19:32.736 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:19:32.736 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:19:32.778 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:19:32.779 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:32.781 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:19:32.781 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:32.783 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:19:32.783 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:32.784 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:19:32.785 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:32.785 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:19:32.864 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:19:32.865 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:19:32.871 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:19:32.872 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:19:32.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.924+0000 7f30b6e00640 1 Processor -- start 2024-04-25T10:19:32.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.924+0000 7f30b6e00640 1 -- start start 2024-04-25T10:19:32.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30b6e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30b006eec0 0x7f30b006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30b6e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30b00ad850 0x7f30b00adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30b6e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b015e0c0 con 0x7f30b015dce0 2024-04-25T10:19:32.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30b6e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b00674d0 con 0x7f30b006eec0 2024-04-25T10:19:32.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30b6e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b00671f0 con 0x7f30b00ad850 2024-04-25T10:19:32.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30b006eec0 0x7f30b006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30b006eec0 0x7f30b006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41594/0 (socket says 172.21.15.87:41594) 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30afe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f30b015dce0 0x7f30b00669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50496/0 (socket says 172.21.15.87:50496) 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 -- 172.21.15.87:0/3847853203 learned_addr learned my addr 172.21.15.87:0/3847853203 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30b4e00640 1 --2- 172.21.15.87:0/3847853203 >> v2:172.21.15.169:3300/0 conn(0x7f30b00ad850 0x7f30b00adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 -- 172.21.15.87:0/3847853203 >> v2:172.21.15.169:3300/0 conn(0x7f30b00ad850 msgr2=0x7f30b00adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 --2- 172.21.15.87:0/3847853203 >> v2:172.21.15.169:3300/0 conn(0x7f30b00ad850 0x7f30b00adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 -- 172.21.15.87:0/3847853203 >> v1:172.21.15.87:6789/0 conn(0x7f30b015dce0 legacy=0x7f30b00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 -- 172.21.15.87:0/3847853203 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b006d4b0 con 0x7f30b006eec0 2024-04-25T10:19:32.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30af400640 1 -- 172.21.15.87:0/3847853203 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604259587 0 0) 0x7f30b015e0c0 con 0x7f30b015dce0 2024-04-25T10:19:32.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.928+0000 7f30a7e00640 1 --2- 172.21.15.87:0/3847853203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b006df90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f30b00715d0 tx=0x7f309801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5616abd07cafcff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:32.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30af400640 1 -- 172.21.15.87:0/3847853203 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980b0070 con 0x7f30b006eec0 2024-04-25T10:19:32.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30af400640 1 -- 172.21.15.87:0/3847853203 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30980ab600 con 0x7f30b006eec0 2024-04-25T10:19:32.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30af400640 1 -- 172.21.15.87:0/3847853203 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980ab9d0 con 0x7f30b006eec0 2024-04-25T10:19:32.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- 172.21.15.87:0/3847853203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 msgr2=0x7f30b006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 --2- 172.21.15.87:0/3847853203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b006df90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f30b00715d0 tx=0x7f309801c400 comp rx=0 tx=0).stop 2024-04-25T10:19:32.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- 172.21.15.87:0/3847853203 shutdown_connections 2024-04-25T10:19:32.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 --2- 172.21.15.87:0/3847853203 >> v2:172.21.15.169:3300/0 conn(0x7f30b00ad850 0x7f30b00adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 --2- 172.21.15.87:0/3847853203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b006df90 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- 172.21.15.87:0/3847853203 >> 172.21.15.87:0/3847853203 conn(0x7f30b00b2410 msgr2=0x7f30b00b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:32.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- 172.21.15.87:0/3847853203 shutdown_connections 2024-04-25T10:19:32.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- 172.21.15.87:0/3847853203 wait complete. 2024-04-25T10:19:32.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 Processor -- start 2024-04-25T10:19:32.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- start start 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b0159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30b00ad850 0x7f30b00c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30b015dce0 0x7f30b01592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00d0320 con 0x7f30b006eec0 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00d04a0 con 0x7f30b015dce0 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b016abc0 con 0x7f30b00ad850 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b0159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b0159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41604/0 (socket says 172.21.15.87:41604) 2024-04-25T10:19:32.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30afe00640 1 -- 172.21.15.87:0/2015130391 learned_addr learned my addr 172.21.15.87:0/2015130391 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30a7e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30b00ad850 0x7f30b00c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.932+0000 7f30b4e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30b015dce0 0x7f30b01592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30afe00640 1 -- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30b00ad850 msgr2=0x7f30b00c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30afe00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30b00ad850 0x7f30b00c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30afe00640 1 -- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30b015dce0 msgr2=0x7f30b01592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30afe00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30b015dce0 0x7f30b01592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30afe00640 1 -- 172.21.15.87:0/2015130391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b015a100 con 0x7f30b006eec0 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30a7e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30b00ad850 0x7f30b00c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:32.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30afe00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b0159b20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f30a000c480 tx=0x7f30a000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:32.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:19:32.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:19:32.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:19:32.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:19:32.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30acc00640 1 -- 172.21.15.87:0/2015130391 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0018070 con 0x7f30b006eec0 2024-04-25T10:19:32.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30acc00640 1 -- 172.21.15.87:0/2015130391 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0007500 con 0x7f30b006eec0 2024-04-25T10:19:32.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30acc00640 1 -- 172.21.15.87:0/2015130391 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0004a30 con 0x7f30b006eec0 2024-04-25T10:19:32.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b015a3e0 con 0x7f30b006eec0 2024-04-25T10:19:32.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b015a9a0 con 0x7f30b006eec0 2024-04-25T10:19:32.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b004ef30 con 0x7f30b006eec0 2024-04-25T10:19:32.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30acc00640 1 -- 172.21.15.87:0/2015130391 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30a0004bd0 con 0x7f30b006eec0 2024-04-25T10:19:32.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.936+0000 7f30acc00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f308c066f60 0x7f308c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:32.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.940+0000 7f30acc00640 1 -- 172.21.15.87:0/2015130391 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f30a0083f70 con 0x7f30b006eec0 2024-04-25T10:19:32.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.940+0000 7f30a7e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f308c066f60 0x7f308c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:32.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.940+0000 7f30a7e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f308c066f60 0x7f308c069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f30980ad000 tx=0x7f30980a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:32.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:32.944+0000 7f30acc00640 1 -- 172.21.15.87:0/2015130391 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30a0054c20 con 0x7f30b006eec0 2024-04-25T10:19:33.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.108+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f30b00af770 con 0x7f30b006eec0 2024-04-25T10:19:33.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.716+0000 7f30acc00640 1 -- 172.21.15.87:0/2015130391 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) v1 ==== 97+0+0 (secure 0 0 0) 0x7f30a0058cc0 con 0x7f30b006eec0 2024-04-25T10:19:33.721 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:19:33.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f308c066f60 msgr2=0x7f308c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:33.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f308c066f60 0x7f308c069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f30980ad000 tx=0x7f30980a6000 comp rx=0 tx=0).stop 2024-04-25T10:19:33.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 msgr2=0x7f30b0159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:33.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b0159b20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f30a000c480 tx=0x7f30a000c940 comp rx=0 tx=0).stop 2024-04-25T10:19:33.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 shutdown_connections 2024-04-25T10:19:33.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f308c066f60 0x7f308c069420 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30b015dce0 0x7f30b01592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30b00ad850 0x7f30b00c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 --2- 172.21.15.87:0/2015130391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b006eec0 0x7f30b0159b20 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 >> 172.21.15.87:0/2015130391 conn(0x7f30b00b2410 msgr2=0x7f30b015ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:33.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 shutdown_connections 2024-04-25T10:19:33.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.720+0000 7f30b6e00640 1 -- 172.21.15.87:0/2015130391 wait complete. 2024-04-25T10:19:33.749 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:19:33.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.884+0000 7f5aaee00640 1 Processor -- start 2024-04-25T10:19:33.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.884+0000 7f5aaee00640 1 -- start start 2024-04-25T10:19:33.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aaee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 0x7f5aa806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aaee00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa80b2770 con 0x7f5aa80cda10 2024-04-25T10:19:33.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aaee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa806c670 con 0x7f5aa815bc40 2024-04-25T10:19:33.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aaee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa806c950 con 0x7f5aa80d0040 2024-04-25T10:19:33.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5aa80cda10 0x7f5aa80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50508/0 (socket says 172.21.15.87:50508) 2024-04-25T10:19:33.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7400640 1 -- 172.21.15.87:0/2522467051 learned_addr learned my addr 172.21.15.87:0/2522467051 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aace00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 0x7f5aa806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783944632 0 0) 0x7f5aa80b2770 con 0x7f5aa80cda10 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a8c003530 con 0x7f5aa80cda10 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028696477 0 0) 0x7f5a8c003530 con 0x7f5aa80cda10 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 msgr2=0x7f5aa806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 0x7f5aa806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 >> v1:172.21.15.169:6789/0 conn(0x7f5aa815bc40 legacy=0x7f5aa80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa806cc30 con 0x7f5aa80cda10 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5a90001680 con 0x7f5aa80cda10 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 >> v1:172.21.15.87:6789/0 conn(0x7f5aa80cda10 legacy=0x7f5aa80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:33.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a8c0039f0 0x7f5a8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c004810 0x7f5a8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 0x7f5a8c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa806c950 con 0x7f5a8c0039f0 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c00bb30 con 0x7f5a8c004810 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c00bd30 con 0x7f5aa80d0040 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a8c0039f0 0x7f5a8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7400640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c004810 0x7f5a8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aace00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 0x7f5a8c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7e00640 1 -- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 msgr2=0x7f5a8c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa80d0040 0x7f5a8c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7e00640 1 -- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c004810 msgr2=0x7f5a8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c004810 0x7f5a8c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7e00640 1 -- 172.21.15.87:0/2522467051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c003530 con 0x7f5a8c0039f0 2024-04-25T10:19:33.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.888+0000 7f5aa7400640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c004810 0x7f5a8c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a8c0039f0 0x7f5a8c003e10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5a98001780 tx=0x7f5a9801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931f821f26b3dc6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980013a0 con 0x7f5a8c0039f0 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a980052f0 con 0x7f5a8c0039f0 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aa6a00640 1 -- 172.21.15.87:0/2522467051 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98003840 con 0x7f5a8c0039f0 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 -- 172.21.15.87:0/2522467051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a8c0039f0 msgr2=0x7f5a8c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 --2- 172.21.15.87:0/2522467051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a8c0039f0 0x7f5a8c003e10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5a98001780 tx=0x7f5a9801c420 comp rx=0 tx=0).stop 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aace00640 1 -- 172.21.15.87:0/2522467051 reap_dead start 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 -- 172.21.15.87:0/2522467051 shutdown_connections 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 -- 172.21.15.87:0/2522467051 >> 172.21.15.87:0/2522467051 conn(0x7f5aa8066b70 msgr2=0x7f5aa8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 -- 172.21.15.87:0/2522467051 shutdown_connections 2024-04-25T10:19:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 -- 172.21.15.87:0/2522467051 wait complete. 2024-04-25T10:19:33.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 Processor -- start 2024-04-25T10:19:33.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.892+0000 7f5aaee00640 1 -- start start 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c0039f0 0x7f5aa8151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aa8154570 0x7f5aa81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa8150ea0 0x7f5aa8151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8167560 con 0x7f5aa8154570 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa81673e0 con 0x7f5a8c0039f0 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa81676e0 con 0x7f5aa8150ea0 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aa8154570 0x7f5aa81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aa8154570 0x7f5aa81550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41624/0 (socket says 172.21.15.87:41624) 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 -- 172.21.15.87:0/1042714494 learned_addr learned my addr 172.21.15.87:0/1042714494 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c0039f0 0x7f5aa8151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aace00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa8150ea0 0x7f5aa8151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 -- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa8150ea0 msgr2=0x7f5aa8151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa8150ea0 0x7f5aa8151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 -- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c0039f0 msgr2=0x7f5aa8151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c0039f0 0x7f5aa8151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 -- 172.21.15.87:0/1042714494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa8152320 con 0x7f5aa8154570 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c0039f0 0x7f5aa8151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aa7400640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aa8154570 0x7f5aa81550b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000e780 tx=0x7f5a9000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5a9fe00640 1 -- 172.21.15.87:0/1042714494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000c660 con 0x7f5aa8154570 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5a9fe00640 1 -- 172.21.15.87:0/1042714494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a90018070 con 0x7f5aa8154570 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5a9fe00640 1 -- 172.21.15.87:0/1042714494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000ca00 con 0x7f5aa8154570 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa8152600 con 0x7f5aa8154570 2024-04-25T10:19:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa8152bc0 con 0x7f5aa8154570 2024-04-25T10:19:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa804ef30 con 0x7f5aa8154570 2024-04-25T10:19:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.896+0000 7f5a9fe00640 1 -- 172.21.15.87:0/1042714494 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a9000cba0 con 0x7f5aa8154570 2024-04-25T10:19:33.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.900+0000 7f5a9fe00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a84066f60 0x7f5a84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.900+0000 7f5a9fe00640 1 -- 172.21.15.87:0/1042714494 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5a90084b00 con 0x7f5aa8154570 2024-04-25T10:19:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.900+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a84066f60 0x7f5a84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:33.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.900+0000 7f5aa7e00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a84066f60 0x7f5a84069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5a98001750 tx=0x7f5a980011a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:33.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:33.904+0000 7f5a9fe00640 1 -- 172.21.15.87:0/1042714494 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a90055dc0 con 0x7f5aa8154570 2024-04-25T10:19:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.108+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5aa80af1c0 con 0x7f5aa8154570 2024-04-25T10:19:34.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.108+0000 7f5a9fe00640 1 -- 172.21.15.87:0/1042714494 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5a90084e20 con 0x7f5aa8154570 2024-04-25T10:19:34.112 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:34.112 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:19:34.112 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:19:34.113 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:19:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.115 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.116 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:19:34.117 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.118 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:19:34.119 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.121 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:19:34.122 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:19:34.123 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.124 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:19:34.125 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:19:34.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a84066f60 msgr2=0x7f5a84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a84066f60 0x7f5a84069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5a98001750 tx=0x7f5a980011a0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aa8154570 msgr2=0x7f5aa81550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aa8154570 0x7f5aa81550b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000e780 tx=0x7f5a9000ec40 comp rx=0 tx=0).stop 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 shutdown_connections 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5a84066f60 0x7f5a84069420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5aa8150ea0 0x7f5aa8151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aa8154570 0x7f5aa81550b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 --2- 172.21.15.87:0/1042714494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a8c0039f0 0x7f5aa8151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 >> 172.21.15.87:0/1042714494 conn(0x7f5aa8066b70 msgr2=0x7f5aa80ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 shutdown_connections 2024-04-25T10:19:34.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.112+0000 7f5aaee00640 1 -- 172.21.15.87:0/1042714494 wait complete. 2024-04-25T10:19:34.133 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:34.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa911000640 1 Processor -- start 2024-04-25T10:19:34.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa911000640 1 -- start start 2024-04-25T10:19:34.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa911000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa90c15bc40 0x7fa90c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa911000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa90c0cda10 0x7fa90c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa911000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c0b2770 con 0x7fa90c0cf930 2024-04-25T10:19:34.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa911000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90c06c670 con 0x7fa90c15bc40 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa911000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90c06c950 con 0x7fa90c0cda10 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa90c15bc40 0x7fa90c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa90c15bc40 0x7fa90c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41640/0 (socket says 172.21.15.87:41640) 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909e00640 1 -- 172.21.15.87:0/2139417211 learned_addr learned my addr 172.21.15.87:0/2139417211 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909400640 1 --2- 172.21.15.87:0/2139417211 >> v2:172.21.15.169:3300/0 conn(0x7fa90c0cda10 0x7fa90c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909e00640 1 -- 172.21.15.87:0/2139417211 >> v2:172.21.15.169:3300/0 conn(0x7fa90c0cda10 msgr2=0x7fa90c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909e00640 1 --2- 172.21.15.87:0/2139417211 >> v2:172.21.15.169:3300/0 conn(0x7fa90c0cda10 0x7fa90c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909e00640 1 -- 172.21.15.87:0/2139417211 >> v1:172.21.15.169:6789/0 conn(0x7fa90c0cf930 legacy=0x7fa90c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909e00640 1 -- 172.21.15.87:0/2139417211 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c06cc30 con 0x7fa90c15bc40 2024-04-25T10:19:34.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.268+0000 7fa909400640 1 --2- 172.21.15.87:0/2139417211 >> v2:172.21.15.169:3300/0 conn(0x7fa90c0cda10 0x7fa90c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:34.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa909e00640 1 --2- 172.21.15.87:0/2139417211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c15bc40 0x7fa90c06c090 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0726f0 tx=0x7fa8fc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb68fba6852252c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa908a00640 1 -- 172.21.15.87:0/2139417211 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0ab5f0 con 0x7fa90c15bc40 2024-04-25T10:19:34.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa908a00640 1 -- 172.21.15.87:0/2139417211 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc0b0070 con 0x7fa90c15bc40 2024-04-25T10:19:34.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa908a00640 1 -- 172.21.15.87:0/2139417211 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0ab9b0 con 0x7fa90c15bc40 2024-04-25T10:19:34.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 -- 172.21.15.87:0/2139417211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c15bc40 msgr2=0x7fa90c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 --2- 172.21.15.87:0/2139417211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c15bc40 0x7fa90c06c090 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0726f0 tx=0x7fa8fc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 -- 172.21.15.87:0/2139417211 shutdown_connections 2024-04-25T10:19:34.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 --2- 172.21.15.87:0/2139417211 >> v2:172.21.15.169:3300/0 conn(0x7fa90c0cda10 0x7fa90c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 --2- 172.21.15.87:0/2139417211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c15bc40 0x7fa90c06c090 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 -- 172.21.15.87:0/2139417211 >> 172.21.15.87:0/2139417211 conn(0x7fa90c066b70 msgr2=0x7fa90c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:34.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 -- 172.21.15.87:0/2139417211 shutdown_connections 2024-04-25T10:19:34.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 -- 172.21.15.87:0/2139417211 wait complete. 2024-04-25T10:19:34.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.272+0000 7fa911000640 1 Processor -- start 2024-04-25T10:19:34.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 -- start start 2024-04-25T10:19:34.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c0cda10 0x7fa90c14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa90c0cf930 0x7fa90c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90c15bc40 0x7fa90c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c168b50 con 0x7fa90c0cda10 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c1689d0 con 0x7fa90c0cf930 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c168cd0 con 0x7fa90c15bc40 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c0cda10 0x7fa90c14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c0cda10 0x7fa90c14b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41644/0 (socket says 172.21.15.87:41644) 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 -- 172.21.15.87:0/1584879724 learned_addr learned my addr 172.21.15.87:0/1584879724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909400640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa90c0cf930 0x7fa90c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa90a800640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90c15bc40 0x7fa90c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 -- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90c15bc40 msgr2=0x7fa90c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90c15bc40 0x7fa90c158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 -- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa90c0cf930 msgr2=0x7fa90c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa90c0cf930 0x7fa90c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 -- 172.21.15.87:0/1584879724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c1574e0 con 0x7fa90c0cda10 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa90a800640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90c15bc40 0x7fa90c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa909e00640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c0cda10 0x7fa90c14b240 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0ad040 tx=0x7fa8fc01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa902000640 1 -- 172.21.15.87:0/1584879724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0be5f0 con 0x7fa90c0cda10 2024-04-25T10:19:34.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa902000640 1 -- 172.21.15.87:0/1584879724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc0be790 con 0x7fa90c0cda10 2024-04-25T10:19:34.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa902000640 1 -- 172.21.15.87:0/1584879724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0beb80 con 0x7fa90c0cda10 2024-04-25T10:19:34.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c14b780 con 0x7fa90c0cda10 2024-04-25T10:19:34.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c157a30 con 0x7fa90c0cda10 2024-04-25T10:19:34.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.276+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c04ef30 con 0x7fa90c0cda10 2024-04-25T10:19:34.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.280+0000 7fa902000640 1 -- 172.21.15.87:0/1584879724 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8fc0b43f0 con 0x7fa90c0cda10 2024-04-25T10:19:34.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.280+0000 7fa902000640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa8ec066f60 0x7fa8ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.280+0000 7fa902000640 1 -- 172.21.15.87:0/1584879724 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8fc11ee30 con 0x7fa90c0cda10 2024-04-25T10:19:34.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.280+0000 7fa909400640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa8ec066f60 0x7fa8ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.280+0000 7fa909400640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa8ec066f60 0x7fa8ec069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa90c1592b0 tx=0x7fa8f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.284+0000 7fa902000640 1 -- 172.21.15.87:0/1584879724 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8fc0160d0 con 0x7fa90c0cda10 2024-04-25T10:19:34.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.436+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa90c06d960 con 0x7fa90c0cda10 2024-04-25T10:19:34.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa902000640 1 -- 172.21.15.87:0/1584879724 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+12880 (secure 0 0 0) 0x7fa8fc0efc50 con 0x7fa90c0cda10 2024-04-25T10:19:34.441 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:34.441 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":96,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:33.730401+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:34.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa8ec066f60 msgr2=0x7fa8ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa8ec066f60 0x7fa8ec069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa90c1592b0 tx=0x7fa8f8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:34.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c0cda10 msgr2=0x7fa90c14b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c0cda10 0x7fa90c14b240 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0ad040 tx=0x7fa8fc01ad20 comp rx=0 tx=0).stop 2024-04-25T10:19:34.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 shutdown_connections 2024-04-25T10:19:34.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa8ec066f60 0x7fa8ec069420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90c15bc40 0x7fa90c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa90c0cf930 0x7fa90c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 --2- 172.21.15.87:0/1584879724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa90c0cda10 0x7fa90c14b240 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 >> 172.21.15.87:0/1584879724 conn(0x7fa90c066b70 msgr2=0x7fa90c15a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:34.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 shutdown_connections 2024-04-25T10:19:34.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.440+0000 7fa911000640 1 -- 172.21.15.87:0/1584879724 wait complete. 2024-04-25T10:19:34.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:34.463 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:34.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.596+0000 7f0091600640 1 Processor -- start 2024-04-25T10:19:34.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.596+0000 7f0091600640 1 -- start start 2024-04-25T10:19:34.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f0091600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f008c159540 0x7f008c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f0091600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f0091600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008c159920 con 0x7f008c0aebd0 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f0091600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c15c2f0 con 0x7f008c159540 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f0091600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c15c5d0 con 0x7f008c0721f0 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f008c159540 0x7f008c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f008c159540 0x7f008c0ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41654/0 (socket says 172.21.15.87:41654) 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008aa00640 1 -- 172.21.15.87:0/3742659037 learned_addr learned my addr 172.21.15.87:0/3742659037 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008b400640 1 --2- 172.21.15.87:0/3742659037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008aa00640 1 -- 172.21.15.87:0/3742659037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 msgr2=0x7f008c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008aa00640 1 --2- 172.21.15.87:0/3742659037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008aa00640 1 -- 172.21.15.87:0/3742659037 >> v1:172.21.15.169:6789/0 conn(0x7f008c0aebd0 legacy=0x7f008c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.600+0000 7f008aa00640 1 -- 172.21.15.87:0/3742659037 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c15c8b0 con 0x7f008c159540 2024-04-25T10:19:34.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f008aa00640 1 --2- 172.21.15.87:0/3742659037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ae690 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f007c009e50 tx=0x7f007c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b87e0583d407bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0089600640 1 -- 172.21.15.87:0/3742659037 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0a5810 con 0x7f008c159540 2024-04-25T10:19:34.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0089600640 1 -- 172.21.15.87:0/3742659037 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c0a59b0 con 0x7f008c159540 2024-04-25T10:19:34.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0089600640 1 -- 172.21.15.87:0/3742659037 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0a5d80 con 0x7f008c159540 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 -- 172.21.15.87:0/3742659037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 msgr2=0x7f008c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 --2- 172.21.15.87:0/3742659037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ae690 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f007c009e50 tx=0x7f007c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 -- 172.21.15.87:0/3742659037 shutdown_connections 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 --2- 172.21.15.87:0/3742659037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 --2- 172.21.15.87:0/3742659037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ae690 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 -- 172.21.15.87:0/3742659037 >> 172.21.15.87:0/3742659037 conn(0x7f008c0669a0 msgr2=0x7f008c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 -- 172.21.15.87:0/3742659037 shutdown_connections 2024-04-25T10:19:34.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 -- 172.21.15.87:0/3742659037 wait complete. 2024-04-25T10:19:34.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 Processor -- start 2024-04-25T10:19:34.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.604+0000 7f0091600640 1 -- start start 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f008c0aebd0 0x7f008c161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c071220 con 0x7f008c159540 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0710a0 con 0x7f008c0aebd0 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0713a0 con 0x7f008c0721f0 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41664/0 (socket says 172.21.15.87:41664) 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 -- 172.21.15.87:0/444269794 learned_addr learned my addr 172.21.15.87:0/444269794 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008aa00640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008a000640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f008c0aebd0 0x7f008c161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 -- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 msgr2=0x7f008c146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 -- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f008c0aebd0 msgr2=0x7f008c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f008c0aebd0 0x7f008c161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 -- 172.21.15.87:0/444269794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c156830 con 0x7f008c159540 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f008b400640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ad840 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f008400d830 tx=0x7f008400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0082a00640 1 -- 172.21.15.87:0/444269794 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008400b6c0 con 0x7f008c159540 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0082a00640 1 -- 172.21.15.87:0/444269794 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084019070 con 0x7f008c159540 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c0b34d0 con 0x7f008c159540 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0082a00640 1 -- 172.21.15.87:0/444269794 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008400baa0 con 0x7f008c159540 2024-04-25T10:19:34.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c0b3a90 con 0x7f008c159540 2024-04-25T10:19:34.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c04ef30 con 0x7f008c159540 2024-04-25T10:19:34.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.608+0000 7f0082a00640 1 -- 172.21.15.87:0/444269794 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f008401e460 con 0x7f008c159540 2024-04-25T10:19:34.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.612+0000 7f0082a00640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f006c066f60 0x7f006c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.612+0000 7f0082a00640 1 -- 172.21.15.87:0/444269794 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f00840853f0 con 0x7f008c159540 2024-04-25T10:19:34.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.612+0000 7f008aa00640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f006c066f60 0x7f006c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.612+0000 7f008aa00640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f006c066f60 0x7f006c069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f007c0ae000 tx=0x7f007c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.616+0000 7f0082a00640 1 -- 172.21.15.87:0/444269794 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00840566b0 con 0x7f008c159540 2024-04-25T10:19:34.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.772+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f008c0cf470 con 0x7f008c159540 2024-04-25T10:19:34.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0082a00640 1 -- 172.21.15.87:0/444269794 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0084055fb0 con 0x7f008c159540 2024-04-25T10:19:34.777 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:19:34.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f006c066f60 msgr2=0x7f006c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f006c066f60 0x7f006c069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f007c0ae000 tx=0x7f007c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:19:34.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 msgr2=0x7f008c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ad840 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f008400d830 tx=0x7f008400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 shutdown_connections 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f006c066f60 0x7f006c069420 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f008c159540 0x7f008c0ad840 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f008c0aebd0 0x7f008c161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 --2- 172.21.15.87:0/444269794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f008c0721f0 0x7f008c146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 >> 172.21.15.87:0/444269794 conn(0x7f008c0669a0 msgr2=0x7f008c0cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 shutdown_connections 2024-04-25T10:19:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.776+0000 7f0091600640 1 -- 172.21.15.87:0/444269794 wait complete. 2024-04-25T10:19:34.797 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:19:34.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.928+0000 7f79ef200640 1 Processor -- start 2024-04-25T10:19:34.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ef200640 1 -- start start 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ef200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f00669a0 0x7f79f0066e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ef200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f79f0161600 0x7f79f0161a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ef200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00ad840 con 0x7f79f0157f40 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ef200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f006e7a0 con 0x7f79f0161600 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ef200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00af9b0 con 0x7f79f00669a0 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ee800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f79f0161600 0x7f79f0161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ee800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f79f0161600 0x7f79f0161a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41670/0 (socket says 172.21.15.87:41670) 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ee800640 1 -- 172.21.15.87:0/280920046 learned_addr learned my addr 172.21.15.87:0/280920046 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ed400640 1 --2- 172.21.15.87:0/280920046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f00669a0 0x7f79f0066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ee800640 1 -- 172.21.15.87:0/280920046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f00669a0 msgr2=0x7f79f0066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:34.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ee800640 1 --2- 172.21.15.87:0/280920046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f00669a0 0x7f79f0066e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ee800640 1 -- 172.21.15.87:0/280920046 >> v1:172.21.15.169:6789/0 conn(0x7f79f0157f40 legacy=0x7f79f00aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.932+0000 7f79ee800640 1 -- 172.21.15.87:0/280920046 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f015e120 con 0x7f79f0161600 2024-04-25T10:19:34.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ee800640 1 --2- 172.21.15.87:0/280920046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0161600 0x7f79f0161a70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009e50 tx=0x7f79e801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd46fa8b48ddd3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79eca00640 1 -- 172.21.15.87:0/280920046 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80a57f0 con 0x7f79f0161600 2024-04-25T10:19:34.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79eca00640 1 -- 172.21.15.87:0/280920046 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e80a5990 con 0x7f79f0161600 2024-04-25T10:19:34.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79eca00640 1 -- 172.21.15.87:0/280920046 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80a5d60 con 0x7f79f0161600 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 -- 172.21.15.87:0/280920046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0161600 msgr2=0x7f79f0161a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 --2- 172.21.15.87:0/280920046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0161600 0x7f79f0161a70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009e50 tx=0x7f79e801ca30 comp rx=0 tx=0).stop 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 -- 172.21.15.87:0/280920046 shutdown_connections 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 --2- 172.21.15.87:0/280920046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0161600 0x7f79f0161a70 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 --2- 172.21.15.87:0/280920046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f00669a0 0x7f79f0066e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 -- 172.21.15.87:0/280920046 >> 172.21.15.87:0/280920046 conn(0x7f79f0074310 msgr2=0x7f79f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 -- 172.21.15.87:0/280920046 shutdown_connections 2024-04-25T10:19:34.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 -- 172.21.15.87:0/280920046 wait complete. 2024-04-25T10:19:34.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 Processor -- start 2024-04-25T10:19:34.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.936+0000 7f79ef200640 1 -- start start 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00669a0 0x7f79f0151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0157f40 0x7f79f0156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0161600 0x7f79f013ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01623c0 con 0x7f79f0157f40 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f015eee0 con 0x7f79f00669a0 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0162540 con 0x7f79f0161600 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0157f40 0x7f79f0156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0157f40 0x7f79f0156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41674/0 (socket says 172.21.15.87:41674) 2024-04-25T10:19:34.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 -- 172.21.15.87:0/198429993 learned_addr learned my addr 172.21.15.87:0/198429993 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ede00640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00669a0 0x7f79f0151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ee800640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0161600 0x7f79f013ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 -- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0161600 msgr2=0x7f79f013ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0161600 0x7f79f013ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 -- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00669a0 msgr2=0x7f79f0151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00669a0 0x7f79f0151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 -- 172.21.15.87:0/198429993 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0153b00 con 0x7f79f0157f40 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ed400640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0157f40 0x7f79f0156ac0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e960 tx=0x7f79dc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79e6000640 1 -- 172.21.15.87:0/198429993 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc017070 con 0x7f79f0157f40 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79e6000640 1 -- 172.21.15.87:0/198429993 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc00ca00 con 0x7f79f0157f40 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79e6000640 1 -- 172.21.15.87:0/198429993 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00cdb0 con 0x7f79f0157f40 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f0150540 con 0x7f79f0157f40 2024-04-25T10:19:34.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f0150b00 con 0x7f79f0157f40 2024-04-25T10:19:34.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0075120 con 0x7f79f0157f40 2024-04-25T10:19:34.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.940+0000 7f79e6000640 1 -- 172.21.15.87:0/198429993 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79dc01a070 con 0x7f79f0157f40 2024-04-25T10:19:34.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.944+0000 7f79e6000640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0066f90 0x7f79d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:34.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.944+0000 7f79e6000640 1 -- 172.21.15.87:0/198429993 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f79dc084d30 con 0x7f79f0157f40 2024-04-25T10:19:34.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.944+0000 7f79ede00640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0066f90 0x7f79d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:34.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.944+0000 7f79ede00640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0066f90 0x7f79d0069450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f79e0000f70 tx=0x7f79e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:34.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:34.948+0000 7f79e6000640 1 -- 172.21.15.87:0/198429993 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79dc055ff0 con 0x7f79f0157f40 2024-04-25T10:19:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.104+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f79f015f300 con 0x7f79f0157f40 2024-04-25T10:19:35.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.104+0000 7f79e6000640 1 -- 172.21.15.87:0/198429993 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+65 (secure 0 0 0) 0x7f79dc0558f0 con 0x7f79f0157f40 2024-04-25T10:19:35.108 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:35.108 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:19:35.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:19:35.108 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:19:35.108 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:19:35.108 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:19:35.108 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0066f90 msgr2=0x7f79d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0066f90 0x7f79d0069450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f79e0000f70 tx=0x7f79e0009040 comp rx=0 tx=0).stop 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0157f40 msgr2=0x7f79f0156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0157f40 0x7f79f0156ac0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e960 tx=0x7f79dc00ee20 comp rx=0 tx=0).stop 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 shutdown_connections 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f79d0066f90 0x7f79d0069450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f0161600 0x7f79f013ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79f0157f40 0x7f79f0156ac0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 --2- 172.21.15.87:0/198429993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f00669a0 0x7f79f0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 >> 172.21.15.87:0/198429993 conn(0x7f79f0074310 msgr2=0x7f79f00b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 shutdown_connections 2024-04-25T10:19:35.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.108+0000 7f79ef200640 1 -- 172.21.15.87:0/198429993 wait complete. 2024-04-25T10:19:35.129 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:19:35.129 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:19:35.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa570a00640 1 Processor -- start 2024-04-25T10:19:35.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa570a00640 1 -- start start 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa570a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa56c0cfbb0 0x7fa56c0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa570a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa56c0b06f0 0x7fa56c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa570a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c0cce10 con 0x7fa56c0cff90 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa570a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c0cd700 con 0x7fa56c0b06f0 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa570a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c0cd0b0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa561e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa56c0b06f0 0x7fa56c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa561e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa56c0b06f0 0x7fa56c15ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41688/0 (socket says 172.21.15.87:41688) 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa561e00640 1 -- 172.21.15.87:0/3961751742 learned_addr learned my addr 172.21.15.87:0/3961751742 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa56a800640 1 --2- 172.21.15.87:0/3961751742 >> v2:172.21.15.169:3300/0 conn(0x7fa56c0cfbb0 0x7fa56c0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa561e00640 1 -- 172.21.15.87:0/3961751742 >> v2:172.21.15.169:3300/0 conn(0x7fa56c0cfbb0 msgr2=0x7fa56c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:35.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa561e00640 1 --2- 172.21.15.87:0/3961751742 >> v2:172.21.15.169:3300/0 conn(0x7fa56c0cfbb0 0x7fa56c0cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa561e00640 1 -- 172.21.15.87:0/3961751742 >> v1:172.21.15.169:6789/0 conn(0x7fa56c0cff90 legacy=0x7fa56c0ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:35.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.268+0000 7fa561e00640 1 -- 172.21.15.87:0/3961751742 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c0cd390 con 0x7fa56c0b06f0 2024-04-25T10:19:35.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa561e00640 1 --2- 172.21.15.87:0/3961751742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0b06f0 0x7fa56c15ebe0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0b4250 tx=0x7fa55401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9156f5cb82ec8ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:35.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa569e00640 1 -- 172.21.15.87:0/3961751742 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540ab430 con 0x7fa56c0b06f0 2024-04-25T10:19:35.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa569e00640 1 -- 172.21.15.87:0/3961751742 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5540b0070 con 0x7fa56c0b06f0 2024-04-25T10:19:35.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa569e00640 1 -- 172.21.15.87:0/3961751742 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540ab7f0 con 0x7fa56c0b06f0 2024-04-25T10:19:35.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 -- 172.21.15.87:0/3961751742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0b06f0 msgr2=0x7fa56c15ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 --2- 172.21.15.87:0/3961751742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0b06f0 0x7fa56c15ebe0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0b4250 tx=0x7fa55401c520 comp rx=0 tx=0).stop 2024-04-25T10:19:35.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 -- 172.21.15.87:0/3961751742 shutdown_connections 2024-04-25T10:19:35.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 --2- 172.21.15.87:0/3961751742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0b06f0 0x7fa56c15ebe0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 --2- 172.21.15.87:0/3961751742 >> v2:172.21.15.169:3300/0 conn(0x7fa56c0cfbb0 0x7fa56c0cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 -- 172.21.15.87:0/3961751742 >> 172.21.15.87:0/3961751742 conn(0x7fa56c0d1750 msgr2=0x7fa56c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:35.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 -- 172.21.15.87:0/3961751742 shutdown_connections 2024-04-25T10:19:35.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.272+0000 7fa570a00640 1 -- 172.21.15.87:0/3961751742 wait complete. 2024-04-25T10:19:35.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 Processor -- start 2024-04-25T10:19:35.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 -- start start 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c0b06f0 0x7fa56c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0cfbb0 0x7fa56c165190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa56c0cff90 0x7fa56c0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1689a0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c168820 con 0x7fa56c0b06f0 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c168b20 con 0x7fa56c0cff90 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0cfbb0 0x7fa56c165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0cfbb0 0x7fa56c165190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41704/0 (socket says 172.21.15.87:41704) 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 -- 172.21.15.87:0/3231924525 learned_addr learned my addr 172.21.15.87:0/3231924525 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:35.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa56a800640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c0b06f0 0x7fa56c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa56b200640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa56c0cff90 0x7fa56c0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 -- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa56c0cff90 msgr2=0x7fa56c0c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa56c0cff90 0x7fa56c0c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 -- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c0b06f0 msgr2=0x7fa56c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c0b06f0 0x7fa56c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 -- 172.21.15.87:0/3231924525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c1571b0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa561e00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0cfbb0 0x7fa56c165190 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa55401a150 tx=0x7fa5540192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa563400640 1 -- 172.21.15.87:0/3231924525 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540be5f0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa563400640 1 -- 172.21.15.87:0/3231924525 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5540be790 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa563400640 1 -- 172.21.15.87:0/3231924525 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540beb40 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c0c5620 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.276+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c0c5be0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.280+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c075120 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.280+0000 7fa563400640 1 -- 172.21.15.87:0/3231924525 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5540b43f0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.280+0000 7fa563400640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f60 0x7fa548069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.280+0000 7fa563400640 1 -- 172.21.15.87:0/3231924525 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa5540a8070 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.280+0000 7fa56a800640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f60 0x7fa548069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.280+0000 7fa56a800640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f60 0x7fa548069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa55c007a60 tx=0x7fa55c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:35.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.284+0000 7fa563400640 1 -- 172.21.15.87:0/3231924525 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5540c6020 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.436+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa56c06e6b0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa563400640 1 -- 172.21.15.87:0/3231924525 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa56c1571b0 con 0x7fa56c0cfbb0 2024-04-25T10:19:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:19:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f60 msgr2=0x7fa548069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f60 0x7fa548069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa55c007a60 tx=0x7fa55c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0cfbb0 msgr2=0x7fa56c165190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0cfbb0 0x7fa56c165190 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa55401a150 tx=0x7fa5540192c0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 shutdown_connections 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa548066f60 0x7fa548069420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa56c0cff90 0x7fa56c0c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa56c0cfbb0 0x7fa56c165190 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 --2- 172.21.15.87:0/3231924525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c0b06f0 0x7fa56c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 >> 172.21.15.87:0/3231924525 conn(0x7fa56c0d1750 msgr2=0x7fa56c06ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 shutdown_connections 2024-04-25T10:19:35.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.724+0000 7fa570a00640 1 -- 172.21.15.87:0/3231924525 wait complete. 2024-04-25T10:19:35.753 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:19:35.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb42b400640 1 Processor -- start 2024-04-25T10:19:35.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb42b400640 1 -- start start 2024-04-25T10:19:35.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb42b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb42415bbb0 0x7fb4240cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb42b400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42406ca90 con 0x7fb4240b4660 2024-04-25T10:19:35.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb42b400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42406cd70 con 0x7fb4240cf960 2024-04-25T10:19:35.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb42b400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42406d050 con 0x7fb42415bbb0 2024-04-25T10:19:35.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb428a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb42415bbb0 0x7fb4240cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb41be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb4240b4660 0x7fb42406c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:50514/0 (socket says 172.21.15.87:50514) 2024-04-25T10:19:35.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.892+0000 7fb41be00640 1 -- 172.21.15.87:0/4002099379 learned_addr learned my addr 172.21.15.87:0/4002099379 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:35.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb428a00640 1 -- 172.21.15.87:0/4002099379 >> v1:172.21.15.169:6789/0 conn(0x7fb4240cf960 legacy=0x7fb4240b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:35.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb428a00640 1 -- 172.21.15.87:0/4002099379 >> v1:172.21.15.87:6789/0 conn(0x7fb4240b4660 legacy=0x7fb42406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb428a00640 1 -- 172.21.15.87:0/4002099379 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240b2880 con 0x7fb42415bbb0 2024-04-25T10:19:35.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb423e00640 1 -- 172.21.15.87:0/4002099379 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518812287 0 0) 0x7fb42406ca90 con 0x7fb4240b4660 2024-04-25T10:19:35.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb428a00640 1 --2- 172.21.15.87:0/4002099379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240cda80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb4140097c0 tx=0x7fb41401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6ac97a4ffa4a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:35.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb423e00640 1 -- 172.21.15.87:0/4002099379 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140b0070 con 0x7fb42415bbb0 2024-04-25T10:19:35.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb423e00640 1 -- 172.21.15.87:0/4002099379 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4140ab600 con 0x7fb42415bbb0 2024-04-25T10:19:35.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.896+0000 7fb423e00640 1 -- 172.21.15.87:0/4002099379 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140ab9d0 con 0x7fb42415bbb0 2024-04-25T10:19:35.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- 172.21.15.87:0/4002099379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 msgr2=0x7fb4240cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 --2- 172.21.15.87:0/4002099379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240cda80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb4140097c0 tx=0x7fb41401c460 comp rx=0 tx=0).stop 2024-04-25T10:19:35.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- 172.21.15.87:0/4002099379 shutdown_connections 2024-04-25T10:19:35.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 --2- 172.21.15.87:0/4002099379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240cda80 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- 172.21.15.87:0/4002099379 >> 172.21.15.87:0/4002099379 conn(0x7fb424066b70 msgr2=0x7fb424066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:35.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- 172.21.15.87:0/4002099379 shutdown_connections 2024-04-25T10:19:35.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- 172.21.15.87:0/4002099379 wait complete. 2024-04-25T10:19:35.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 Processor -- start 2024-04-25T10:19:35.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- start start 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4240b4660 0x7fb42414f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4240cf960 0x7fb42415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240ce140 con 0x7fb42415bbb0 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240cdfc0 con 0x7fb4240cf960 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb42b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42416a180 con 0x7fb4240b4660 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41724/0 (socket says 172.21.15.87:41724) 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 -- 172.21.15.87:0/3085075311 learned_addr learned my addr 172.21.15.87:0/3085075311 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb428a00640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4240b4660 0x7fb42414f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb41be00640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4240cf960 0x7fb42415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 -- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4240b4660 msgr2=0x7fb42414f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4240b4660 0x7fb42414f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 -- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4240cf960 msgr2=0x7fb42415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4240cf960 0x7fb42415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:35.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 -- 172.21.15.87:0/3085075311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42415d120 con 0x7fb42415bbb0 2024-04-25T10:19:35.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.900+0000 7fb429400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240adde0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c780 tx=0x7fb41c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:35.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb421600640 1 -- 172.21.15.87:0/3085075311 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c004b70 con 0x7fb42415bbb0 2024-04-25T10:19:35.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb421600640 1 -- 172.21.15.87:0/3085075311 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c004d10 con 0x7fb42415bbb0 2024-04-25T10:19:35.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb421600640 1 -- 172.21.15.87:0/3085075311 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00aa90 con 0x7fb42415bbb0 2024-04-25T10:19:35.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42415d400 con 0x7fb42415bbb0 2024-04-25T10:19:35.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42415d9c0 con 0x7fb42415bbb0 2024-04-25T10:19:35.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42404ef30 con 0x7fb42415bbb0 2024-04-25T10:19:35.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb421600640 1 -- 172.21.15.87:0/3085075311 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb41c00ac30 con 0x7fb42415bbb0 2024-04-25T10:19:35.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb421600640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb400066f60 0x7fb400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:35.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb421600640 1 -- 172.21.15.87:0/3085075311 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb41c084fb0 con 0x7fb42415bbb0 2024-04-25T10:19:35.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.904+0000 7fb428a00640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb400066f60 0x7fb400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:35.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.908+0000 7fb428a00640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb400066f60 0x7fb400069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb4140ad000 tx=0x7fb414008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:35.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:35.912+0000 7fb421600640 1 -- 172.21.15.87:0/3085075311 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb41c020070 con 0x7fb42415bbb0 2024-04-25T10:19:36.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.064+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb42406d980 con 0x7fb42415bbb0 2024-04-25T10:19:36.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb421600640 1 -- 172.21.15.87:0/3085075311 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb41c055b40 con 0x7fb42415bbb0 2024-04-25T10:19:36.729 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:19:36.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb400066f60 msgr2=0x7fb400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:36.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb400066f60 0x7fb400069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb4140ad000 tx=0x7fb414008040 comp rx=0 tx=0).stop 2024-04-25T10:19:36.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 msgr2=0x7fb4240adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:36.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240adde0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c780 tx=0x7fb41c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 shutdown_connections 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb400066f60 0x7fb400069420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb42415bbb0 0x7fb4240adde0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4240cf960 0x7fb42415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 --2- 172.21.15.87:0/3085075311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4240b4660 0x7fb42414f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 >> 172.21.15.87:0/3085075311 conn(0x7fb424066b70 msgr2=0x7fb424152210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 shutdown_connections 2024-04-25T10:19:36.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.728+0000 7fb42b400640 1 -- 172.21.15.87:0/3085075311 wait complete. 2024-04-25T10:19:36.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:19:36.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.908+0000 7f209f400640 1 Processor -- start 2024-04-25T10:19:36.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.908+0000 7f209f400640 1 -- start start 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.908+0000 7f209f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980aef80 0x7f209815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.908+0000 7f209f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20980b31c0 0x7f20980b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.908+0000 7f209f400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209815b080 con 0x7f2098070350 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.908+0000 7f209f400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209815adb0 con 0x7f20980b31c0 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.908+0000 7f209f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209815eb60 con 0x7f20980aef80 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20980b31c0 0x7f20980b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20980b31c0 0x7f20980b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41728/0 (socket says 172.21.15.87:41728) 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 -- 172.21.15.87:0/3594710587 learned_addr learned my addr 172.21.15.87:0/3594710587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:36.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209ca00640 1 --2- 172.21.15.87:0/3594710587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980aef80 0x7f209815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:36.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 -- 172.21.15.87:0/3594710587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980aef80 msgr2=0x7f209815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:36.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 --2- 172.21.15.87:0/3594710587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980aef80 0x7f209815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 -- 172.21.15.87:0/3594710587 >> v1:172.21.15.87:6789/0 conn(0x7f2098070350 legacy=0x7f209815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:36.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 -- 172.21.15.87:0/3594710587 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209815ee40 con 0x7f20980b31c0 2024-04-25T10:19:36.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f2097e00640 1 -- 172.21.15.87:0/3594710587 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005625598 0 0) 0x7f209815b080 con 0x7f2098070350 2024-04-25T10:19:36.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f209d400640 1 --2- 172.21.15.87:0/3594710587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20980b31c0 0x7f20980b3610 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2090009e50 tx=0x7f209001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7110635e130db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:36.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f2097e00640 1 -- 172.21.15.87:0/3594710587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900a5810 con 0x7f20980b31c0 2024-04-25T10:19:36.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f2097e00640 1 -- 172.21.15.87:0/3594710587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900a59b0 con 0x7f20980b31c0 2024-04-25T10:19:36.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.912+0000 7f2097e00640 1 -- 172.21.15.87:0/3594710587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900a5d80 con 0x7f20980b31c0 2024-04-25T10:19:36.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- 172.21.15.87:0/3594710587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20980b31c0 msgr2=0x7f20980b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:36.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 --2- 172.21.15.87:0/3594710587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20980b31c0 0x7f20980b3610 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2090009e50 tx=0x7f209001ca30 comp rx=0 tx=0).stop 2024-04-25T10:19:36.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- 172.21.15.87:0/3594710587 shutdown_connections 2024-04-25T10:19:36.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 --2- 172.21.15.87:0/3594710587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20980b31c0 0x7f20980b3610 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 --2- 172.21.15.87:0/3594710587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980aef80 0x7f209815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- 172.21.15.87:0/3594710587 >> 172.21.15.87:0/3594710587 conn(0x7f20980669a0 msgr2=0x7f2098066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:36.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- 172.21.15.87:0/3594710587 shutdown_connections 2024-04-25T10:19:36.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- 172.21.15.87:0/3594710587 wait complete. 2024-04-25T10:19:36.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 Processor -- start 2024-04-25T10:19:36.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- start start 2024-04-25T10:19:36.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2098070350 0x7f20981544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:36.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20980aef80 0x7f20980b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:36.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980b31c0 0x7f20980cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:36.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20981679a0 con 0x7f2098070350 2024-04-25T10:19:36.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098167820 con 0x7f20980aef80 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2098167b20 con 0x7f20980b31c0 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2098070350 0x7f20981544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2098070350 0x7f20981544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41730/0 (socket says 172.21.15.87:41730) 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 -- 172.21.15.87:0/1620061857 learned_addr learned my addr 172.21.15.87:0/1620061857 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209d400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980b31c0 0x7f20980cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f208fe00640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20980aef80 0x7f20980b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 -- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980b31c0 msgr2=0x7f20980cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980b31c0 0x7f20980cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 -- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20980aef80 msgr2=0x7f20980b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20980aef80 0x7f20980b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209ca00640 1 -- 172.21.15.87:0/1620061857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209814f870 con 0x7f2098070350 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.916+0000 7f209d400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980b31c0 0x7f20980cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f209ca00640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2098070350 0x7f20981544f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f208800e930 tx=0x7f208800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:36.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f2095600640 1 -- 172.21.15.87:0/1620061857 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088017070 con 0x7f2098070350 2024-04-25T10:19:36.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f2095600640 1 -- 172.21.15.87:0/1620061857 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800ca00 con 0x7f2098070350 2024-04-25T10:19:36.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209814fb50 con 0x7f2098070350 2024-04-25T10:19:36.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f2095600640 1 -- 172.21.15.87:0/1620061857 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800cdb0 con 0x7f2098070350 2024-04-25T10:19:36.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098150110 con 0x7f2098070350 2024-04-25T10:19:36.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209804ef30 con 0x7f2098070350 2024-04-25T10:19:36.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f2095600640 1 -- 172.21.15.87:0/1620061857 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f208801a070 con 0x7f2098070350 2024-04-25T10:19:36.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f2095600640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2074066f90 0x7f2074069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:36.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f2095600640 1 -- 172.21.15.87:0/1620061857 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f20880853c0 con 0x7f2098070350 2024-04-25T10:19:36.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.920+0000 7f208fe00640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2074066f90 0x7f2074069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:36.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.924+0000 7f208fe00640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2074066f90 0x7f2074069450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f20981f9c30 tx=0x7f2080009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:36.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:36.928+0000 7f2095600640 1 -- 172.21.15.87:0/1620061857 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20880564e0 con 0x7f2098070350 2024-04-25T10:19:37.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.116+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f20980b2fa0 con 0x7f2098070350 2024-04-25T10:19:37.765 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T10:19:37.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.760+0000 7f2095600640 1 -- 172.21.15.87:0/1620061857 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2088055de0 con 0x7f2098070350 2024-04-25T10:19:37.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.768+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2074066f90 msgr2=0x7f2074069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:37.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.768+0000 7f209f400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2074066f90 0x7f2074069450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f20981f9c30 tx=0x7f2080009040 comp rx=0 tx=0).stop 2024-04-25T10:19:37.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.768+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2098070350 msgr2=0x7f20981544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:37.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.768+0000 7f209f400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2098070350 0x7f20981544f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f208800e930 tx=0x7f208800edf0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.776+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 shutdown_connections 2024-04-25T10:19:37.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.776+0000 7f209f400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2074066f90 0x7f2074069450 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.776+0000 7f209f400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20980b31c0 0x7f20980cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.776+0000 7f209f400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20980aef80 0x7f20980b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.776+0000 7f209f400640 1 --2- 172.21.15.87:0/1620061857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2098070350 0x7f20981544f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.776+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 >> 172.21.15.87:0/1620061857 conn(0x7f20980669a0 msgr2=0x7f209815bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:37.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.780+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 shutdown_connections 2024-04-25T10:19:37.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.780+0000 7f209f400640 1 -- 172.21.15.87:0/1620061857 wait complete. 2024-04-25T10:19:37.813 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:37.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.948+0000 7f39e4c00640 1 Processor -- start 2024-04-25T10:19:37.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.948+0000 7f39e4c00640 1 -- start start 2024-04-25T10:19:37.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cda10 0x7f39e006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:37.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39e4c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39e00cf930 0x7f39e00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:37.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39e4c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00b2770 con 0x7f39e015bc40 2024-04-25T10:19:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39e4c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e006c670 con 0x7f39e00cf930 2024-04-25T10:19:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39e4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e006c950 con 0x7f39e00cda10 2024-04-25T10:19:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39dde00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f39e015bc40 0x7f39e00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39836/0 (socket says 172.21.15.87:39836) 2024-04-25T10:19:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39dde00640 1 -- 172.21.15.87:0/1325955544 learned_addr learned my addr 172.21.15.87:0/1325955544 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39de800640 1 --2- 172.21.15.87:0/1325955544 >> v2:172.21.15.87:3300/0 conn(0x7f39e00cf930 0x7f39e00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39d5400640 1 --2- 172.21.15.87:0/1325955544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cda10 0x7f39e006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39de800640 1 -- 172.21.15.87:0/1325955544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cda10 msgr2=0x7f39e006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39de800640 1 --2- 172.21.15.87:0/1325955544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cda10 0x7f39e006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39de800640 1 -- 172.21.15.87:0/1325955544 >> v1:172.21.15.87:6789/0 conn(0x7f39e015bc40 legacy=0x7f39e00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39de800640 1 -- 172.21.15.87:0/1325955544 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e006cc30 con 0x7f39e00cf930 2024-04-25T10:19:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.952+0000 7f39dd400640 1 -- 172.21.15.87:0/1325955544 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378168461 0 0) 0x7f39e00b2770 con 0x7f39e015bc40 2024-04-25T10:19:37.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39de800640 1 --2- 172.21.15.87:0/1325955544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cf930 0x7f39e00cfde0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009d90 tx=0x7f39d801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7047a4e11e168774 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:37.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39dd400640 1 -- 172.21.15.87:0/1325955544 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80b2070 con 0x7f39e00cf930 2024-04-25T10:19:37.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39dd400640 1 -- 172.21.15.87:0/1325955544 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d80a57a0 con 0x7f39e00cf930 2024-04-25T10:19:37.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39dd400640 1 -- 172.21.15.87:0/1325955544 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80a5b70 con 0x7f39e00cf930 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- 172.21.15.87:0/1325955544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cf930 msgr2=0x7f39e00cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 --2- 172.21.15.87:0/1325955544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cf930 0x7f39e00cfde0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009d90 tx=0x7f39d801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- 172.21.15.87:0/1325955544 shutdown_connections 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 --2- 172.21.15.87:0/1325955544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cf930 0x7f39e00cfde0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 --2- 172.21.15.87:0/1325955544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cda10 0x7f39e006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- 172.21.15.87:0/1325955544 >> 172.21.15.87:0/1325955544 conn(0x7f39e0066b70 msgr2=0x7f39e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- 172.21.15.87:0/1325955544 shutdown_connections 2024-04-25T10:19:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- 172.21.15.87:0/1325955544 wait complete. 2024-04-25T10:19:37.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 Processor -- start 2024-04-25T10:19:37.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- start start 2024-04-25T10:19:37.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cda10 0x7f39e0158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:37.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cf930 0x7f39e014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e015bc40 0x7f39e00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00b3850 con 0x7f39e00cda10 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00b36d0 con 0x7f39e015bc40 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39e4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00b39d0 con 0x7f39e00cf930 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39dde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cda10 0x7f39e0158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39dde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cda10 0x7f39e0158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54544/0 (socket says 172.21.15.87:54544) 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.956+0000 7f39dde00640 1 -- 172.21.15.87:0/2102989786 learned_addr learned my addr 172.21.15.87:0/2102989786 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39d5400640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cf930 0x7f39e014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39de800640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e015bc40 0x7f39e00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39dde00640 1 -- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cf930 msgr2=0x7f39e014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39dde00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cf930 0x7f39e014bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39dde00640 1 -- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e015bc40 msgr2=0x7f39e00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39dde00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e015bc40 0x7f39e00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39dde00640 1 -- 172.21.15.87:0/2102989786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0154d30 con 0x7f39e00cda10 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39d5400640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cf930 0x7f39e014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39de800640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e015bc40 0x7f39e00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39dde00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cda10 0x7f39e0158990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c480 tx=0x7f39d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39d6a00640 1 -- 172.21.15.87:0/2102989786 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0018070 con 0x7f39e00cda10 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39d6a00640 1 -- 172.21.15.87:0/2102989786 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0007500 con 0x7f39e00cda10 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39d6a00640 1 -- 172.21.15.87:0/2102989786 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0004a30 con 0x7f39e00cda10 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e0155010 con 0x7f39e00cda10 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e01555d0 con 0x7f39e00cda10 2024-04-25T10:19:37.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e0155a10 con 0x7f39e00cda10 2024-04-25T10:19:37.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.960+0000 7f39d6a00640 1 -- 172.21.15.87:0/2102989786 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39d0004bd0 con 0x7f39e00cda10 2024-04-25T10:19:37.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.964+0000 7f39d6a00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39bc066f60 0x7f39bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:37.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.964+0000 7f39d6a00640 1 -- 172.21.15.87:0/2102989786 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39d0084280 con 0x7f39e00cda10 2024-04-25T10:19:37.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.964+0000 7f39d5400640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39bc066f60 0x7f39bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:37.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.964+0000 7f39d5400640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39bc066f60 0x7f39bc069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39e00d01c0 tx=0x7f39c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:37.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:37.968+0000 7f39d6a00640 1 -- 172.21.15.87:0/2102989786 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39d0055320 con 0x7f39e00cda10 2024-04-25T10:19:38.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.120+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39e004ef30 con 0x7f39e00cda10 2024-04-25T10:19:38.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39d6a00640 1 -- 172.21.15.87:0/2102989786 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f39d0054c20 con 0x7f39e00cda10 2024-04-25T10:19:38.125 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:38.126 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":99,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:37.118346+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:35.484354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:19:36.111171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:38.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39bc066f60 msgr2=0x7f39bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39bc066f60 0x7f39bc069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39e00d01c0 tx=0x7f39c8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:38.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cda10 msgr2=0x7f39e0158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cda10 0x7f39e0158990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c480 tx=0x7f39d000c940 comp rx=0 tx=0).stop 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 shutdown_connections 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39bc066f60 0x7f39bc069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e015bc40 0x7f39e00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cf930 0x7f39e014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 --2- 172.21.15.87:0/2102989786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39e00cda10 0x7f39e0158990 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 >> 172.21.15.87:0/2102989786 conn(0x7f39e0066b70 msgr2=0x7f39e015feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 shutdown_connections 2024-04-25T10:19:38.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.124+0000 7f39e4c00640 1 -- 172.21.15.87:0/2102989786 wait complete. 2024-04-25T10:19:38.147 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:38.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 Processor -- start 2024-04-25T10:19:38.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 -- start start 2024-04-25T10:19:38.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e7015bc40 0x7f2e7006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e700b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e700cf930 0x7f2e700cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e700b2770 con 0x7f2e700cf930 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e7006c670 con 0x7f2e7015bc40 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e76c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7006c950 con 0x7f2e700cda10 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e700cf930 0x7f2e700cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e6fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e7015bc40 0x7f2e7006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e700cf930 0x7f2e700cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54560/0 (socket says 172.21.15.87:54560) 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 -- 172.21.15.87:0/4211336268 learned_addr learned my addr 172.21.15.87:0/4211336268 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e67400640 1 --2- 172.21.15.87:0/4211336268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e700b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 -- 172.21.15.87:0/4211336268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 msgr2=0x7f2e700b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 --2- 172.21.15.87:0/4211336268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e700b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 -- 172.21.15.87:0/4211336268 >> v2:172.21.15.169:3300/0 conn(0x7f2e7015bc40 msgr2=0x7f2e7006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 --2- 172.21.15.87:0/4211336268 >> v2:172.21.15.169:3300/0 conn(0x7f2e7015bc40 0x7f2e7006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.280+0000 7f2e74c00640 1 -- 172.21.15.87:0/4211336268 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7006cc30 con 0x7f2e700cf930 2024-04-25T10:19:38.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e74c00640 1 --2- 172.21.15.87:0/4211336268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700cf930 0x7f2e700cfde0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2e700d0800 tx=0x7f2e6801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f7f4c45beebd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e6f400640 1 -- 172.21.15.87:0/4211336268 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680a5670 con 0x7f2e700cf930 2024-04-25T10:19:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e6f400640 1 -- 172.21.15.87:0/4211336268 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680a5810 con 0x7f2e700cf930 2024-04-25T10:19:38.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e6f400640 1 -- 172.21.15.87:0/4211336268 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680a5be0 con 0x7f2e700cf930 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 -- 172.21.15.87:0/4211336268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700cf930 msgr2=0x7f2e700cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 --2- 172.21.15.87:0/4211336268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700cf930 0x7f2e700cfde0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2e700d0800 tx=0x7f2e6801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 -- 172.21.15.87:0/4211336268 shutdown_connections 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 --2- 172.21.15.87:0/4211336268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700cf930 0x7f2e700cfde0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 --2- 172.21.15.87:0/4211336268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e700b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 --2- 172.21.15.87:0/4211336268 >> v2:172.21.15.169:3300/0 conn(0x7f2e7015bc40 0x7f2e7006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 -- 172.21.15.87:0/4211336268 >> 172.21.15.87:0/4211336268 conn(0x7f2e70066b70 msgr2=0x7f2e70066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 -- 172.21.15.87:0/4211336268 shutdown_connections 2024-04-25T10:19:38.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.284+0000 7f2e76c00640 1 -- 172.21.15.87:0/4211336268 wait complete. 2024-04-25T10:19:38.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 Processor -- start 2024-04-25T10:19:38.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 -- start start 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e7014e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e700cf930 0x7f2e7014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7015bc40 0x7f2e70152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70160230 con 0x7f2e7015bc40 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701600b0 con 0x7f2e700cf930 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701603b0 con 0x7f2e700cda10 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7015bc40 0x7f2e70152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7015bc40 0x7f2e70152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54564/0 (socket says 172.21.15.87:54564) 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 -- 172.21.15.87:0/782825542 learned_addr learned my addr 172.21.15.87:0/782825542 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e7014e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e67400640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e700cf930 0x7f2e7014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 -- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 msgr2=0x7f2e7014e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e7014e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 -- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e700cf930 msgr2=0x7f2e7014adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e700cf930 0x7f2e7014adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 -- 172.21.15.87:0/782825542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e70152ec0 con 0x7f2e7015bc40 2024-04-25T10:19:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e74c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7015bc40 0x7f2e70152400 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2e6801b0a0 tx=0x7f2e68019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e6cc00640 1 -- 172.21.15.87:0/782825542 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68009510 con 0x7f2e7015bc40 2024-04-25T10:19:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e6cc00640 1 -- 172.21.15.87:0/782825542 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680be070 con 0x7f2e7015bc40 2024-04-25T10:19:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e6cc00640 1 -- 172.21.15.87:0/782825542 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680b88f0 con 0x7f2e7015bc40 2024-04-25T10:19:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e701531a0 con 0x7f2e7015bc40 2024-04-25T10:19:38.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.288+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e70155bf0 con 0x7f2e7015bc40 2024-04-25T10:19:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.292+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7004ef30 con 0x7f2e7015bc40 2024-04-25T10:19:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.292+0000 7f2e6cc00640 1 -- 172.21.15.87:0/782825542 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e680b8a90 con 0x7f2e7015bc40 2024-04-25T10:19:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.292+0000 7f2e6cc00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e4c066f60 0x7f2e4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.292+0000 7f2e6cc00640 1 -- 172.21.15.87:0/782825542 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e68120150 con 0x7f2e7015bc40 2024-04-25T10:19:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.292+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e4c066f60 0x7f2e4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.292+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e4c066f60 0x7f2e4c069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2e701f8260 tx=0x7f2e60009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.296+0000 7f2e6cc00640 1 -- 172.21.15.87:0/782825542 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e680165e0 con 0x7f2e7015bc40 2024-04-25T10:19:38.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.452+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e700af1c0 con 0x7f2e7015bc40 2024-04-25T10:19:38.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.452+0000 7f2e6cc00640 1 -- 172.21.15.87:0/782825542 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f2e680f0b90 con 0x7f2e7015bc40 2024-04-25T10:19:38.456 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:38.456 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":99,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:37.118346+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:35.484354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:19:36.111171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:38.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e4c066f60 msgr2=0x7f2e4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e4c066f60 0x7f2e4c069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2e701f8260 tx=0x7f2e60009190 comp rx=0 tx=0).stop 2024-04-25T10:19:38.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7015bc40 msgr2=0x7f2e70152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7015bc40 0x7f2e70152400 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2e6801b0a0 tx=0x7f2e68019790 comp rx=0 tx=0).stop 2024-04-25T10:19:38.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 shutdown_connections 2024-04-25T10:19:38.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e4c066f60 0x7f2e4c069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7015bc40 0x7f2e70152400 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e700cf930 0x7f2e7014adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 --2- 172.21.15.87:0/782825542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700cda10 0x7f2e7014e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 >> 172.21.15.87:0/782825542 conn(0x7f2e70066b70 msgr2=0x7f2e7015af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 shutdown_connections 2024-04-25T10:19:38.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.456+0000 7f2e76c00640 1 -- 172.21.15.87:0/782825542 wait complete. 2024-04-25T10:19:38.477 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:19:38.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a97400640 1 Processor -- start 2024-04-25T10:19:38.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a97400640 1 -- start start 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a97400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a9015a0e0 0x7f1a9015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a97400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a900b2fd0 con 0x7f1a900b3d90 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a97400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a900b32b0 con 0x7f1a900aea50 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a97400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a900cfe10 con 0x7f1a9015a0e0 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a94a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a9015a0e0 0x7f1a9015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a94a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a9015a0e0 0x7f1a9015b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54580/0 (socket says 172.21.15.87:54580) 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a94a00640 1 -- 172.21.15.87:0/1420926290 learned_addr learned my addr 172.21.15.87:0/1420926290 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:38.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.612+0000 7f1a95400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1a900b3d90 0x7f1a9006c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39850/0 (socket says 172.21.15.87:39850) 2024-04-25T10:19:38.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a94a00640 1 -- 172.21.15.87:0/1420926290 >> v1:172.21.15.169:6789/0 conn(0x7f1a900aea50 legacy=0x7f1a900cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a94a00640 1 -- 172.21.15.87:0/1420926290 >> v1:172.21.15.87:6789/0 conn(0x7f1a900b3d90 legacy=0x7f1a9006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a94a00640 1 -- 172.21.15.87:0/1420926290 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a900b3590 con 0x7f1a9015a0e0 2024-04-25T10:19:38.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a8f400640 1 -- 172.21.15.87:0/1420926290 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347478834 0 0) 0x7f1a900b2fd0 con 0x7f1a900b3d90 2024-04-25T10:19:38.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a94a00640 1 --2- 172.21.15.87:0/1420926290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9015a0e0 0x7f1a9015b150 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009e50 tx=0x7f1a8001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a0aca416cb9762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a8f400640 1 -- 172.21.15.87:0/1420926290 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800a55d0 con 0x7f1a9015a0e0 2024-04-25T10:19:38.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a8f400640 1 -- 172.21.15.87:0/1420926290 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a800a5770 con 0x7f1a9015a0e0 2024-04-25T10:19:38.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a8f400640 1 -- 172.21.15.87:0/1420926290 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800a5b40 con 0x7f1a9015a0e0 2024-04-25T10:19:38.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a97400640 1 -- 172.21.15.87:0/1420926290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9015a0e0 msgr2=0x7f1a9015b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a97400640 1 --2- 172.21.15.87:0/1420926290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9015a0e0 0x7f1a9015b150 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009e50 tx=0x7f1a8001c910 comp rx=0 tx=0).stop 2024-04-25T10:19:38.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a97400640 1 -- 172.21.15.87:0/1420926290 shutdown_connections 2024-04-25T10:19:38.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a97400640 1 --2- 172.21.15.87:0/1420926290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9015a0e0 0x7f1a9015b150 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a97400640 1 -- 172.21.15.87:0/1420926290 >> 172.21.15.87:0/1420926290 conn(0x7f1a900669a0 msgr2=0x7f1a90066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a97400640 1 -- 172.21.15.87:0/1420926290 shutdown_connections 2024-04-25T10:19:38.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.616+0000 7f1a97400640 1 -- 172.21.15.87:0/1420926290 wait complete. 2024-04-25T10:19:38.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 Processor -- start 2024-04-25T10:19:38.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 -- start start 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a900aea50 0x7f1a90150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a900b3d90 0x7f1a900adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9015a0e0 0x7f1a900a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9006c7e0 con 0x7f1a900aea50 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900af1d0 con 0x7f1a9015a0e0 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9006c960 con 0x7f1a900b3d90 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a900aea50 0x7f1a90150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a900aea50 0x7f1a90150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54594/0 (socket says 172.21.15.87:54594) 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 -- 172.21.15.87:0/3039200250 learned_addr learned my addr 172.21.15.87:0/3039200250 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a95400640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9015a0e0 0x7f1a900a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a8fe00640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a900b3d90 0x7f1a900adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 -- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a900b3d90 msgr2=0x7f1a900adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a900b3d90 0x7f1a900adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 -- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9015a0e0 msgr2=0x7f1a900a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9015a0e0 0x7f1a900a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 -- 172.21.15.87:0/3039200250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a900aa570 con 0x7f1a900aea50 2024-04-25T10:19:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a94a00640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a900aea50 0x7f1a90150de0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f1a80019520 tx=0x7f1a800196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a8cc00640 1 -- 172.21.15.87:0/3039200250 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800bc020 con 0x7f1a900aea50 2024-04-25T10:19:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a8cc00640 1 -- 172.21.15.87:0/3039200250 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a800b8880 con 0x7f1a900aea50 2024-04-25T10:19:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a900aa850 con 0x7f1a900aea50 2024-04-25T10:19:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a8cc00640 1 -- 172.21.15.87:0/3039200250 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80009510 con 0x7f1a900aea50 2024-04-25T10:19:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.620+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a900ae210 con 0x7f1a900aea50 2024-04-25T10:19:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.624+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9004ef30 con 0x7f1a900aea50 2024-04-25T10:19:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.624+0000 7f1a8cc00640 1 -- 172.21.15.87:0/3039200250 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a800bf020 con 0x7f1a900aea50 2024-04-25T10:19:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.624+0000 7f1a8cc00640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a70066f60 0x7f1a70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.624+0000 7f1a8cc00640 1 -- 172.21.15.87:0/3039200250 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a801205e0 con 0x7f1a900aea50 2024-04-25T10:19:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.624+0000 7f1a8fe00640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a70066f60 0x7f1a70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.624+0000 7f1a8fe00640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a70066f60 0x7f1a70069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1a90150350 tx=0x7f1a7c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.628+0000 7f1a8cc00640 1 -- 172.21.15.87:0/3039200250 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a800c6030 con 0x7f1a900aea50 2024-04-25T10:19:38.723 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:19:38.720+0000 7f90eaa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:19:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:38.820+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1a9015b8f0 con 0x7f1a900aea50 2024-04-25T10:19:39.651 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:19:39.651 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:19:39.651 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:19:39.651 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:19:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.764+0000 7f1a8cc00640 1 -- 172.21.15.87:0/3039200250 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1a80016620 con 0x7f1a900aea50 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a70066f60 msgr2=0x7f1a70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a70066f60 0x7f1a70069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1a90150350 tx=0x7f1a7c009040 comp rx=0 tx=0).stop 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a900aea50 msgr2=0x7f1a90150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a900aea50 0x7f1a90150de0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f1a80019520 tx=0x7f1a800196b0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 shutdown_connections 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a70066f60 0x7f1a70069420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a9015a0e0 0x7f1a900a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a900b3d90 0x7f1a900adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 --2- 172.21.15.87:0/3039200250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a900aea50 0x7f1a90150de0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 >> 172.21.15.87:0/3039200250 conn(0x7f1a900669a0 msgr2=0x7f1a900ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:39.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 shutdown_connections 2024-04-25T10:19:39.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.768+0000 7f1a97400640 1 -- 172.21.15.87:0/3039200250 wait complete. 2024-04-25T10:19:39.792 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:19:39.793 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:19:39.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.928+0000 7f67d5200640 1 Processor -- start 2024-04-25T10:19:39.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.928+0000 7f67d5200640 1 -- start start 2024-04-25T10:19:39.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67d5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f67d00cf9b0 0x7f67d015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:39.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67d5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d00b3360 0x7f67d0070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:39.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67d5200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67d0070f50 0x7f67d01540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:39.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67d5200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d015ef10 con 0x7f67d0070f50 2024-04-25T10:19:39.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67d5200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d00b5200 con 0x7f67d00cf9b0 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67d5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00b38b0 con 0x7f67d00b3360 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67d0070f50 0x7f67d01540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67d0070f50 0x7f67d01540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54598/0 (socket says 172.21.15.87:54598) 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 -- 172.21.15.87:0/1900057117 learned_addr learned my addr 172.21.15.87:0/1900057117 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67c6a00640 1 --2- 172.21.15.87:0/1900057117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d00b3360 0x7f67d0070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 -- 172.21.15.87:0/1900057117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d00b3360 msgr2=0x7f67d0070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 --2- 172.21.15.87:0/1900057117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d00b3360 0x7f67d0070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 -- 172.21.15.87:0/1900057117 >> v2:172.21.15.169:3300/0 conn(0x7f67d00cf9b0 msgr2=0x7f67d015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cf400640 1 --2- 172.21.15.87:0/1900057117 >> v2:172.21.15.169:3300/0 conn(0x7f67d00cf9b0 0x7f67d015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 --2- 172.21.15.87:0/1900057117 >> v2:172.21.15.169:3300/0 conn(0x7f67d00cf9b0 0x7f67d015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cfe00640 1 -- 172.21.15.87:0/1900057117 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00b3b90 con 0x7f67d0070f50 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67c6a00640 1 --2- 172.21.15.87:0/1900057117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d00b3360 0x7f67d0070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:39.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.932+0000 7f67cf400640 1 --2- 172.21.15.87:0/1900057117 >> v2:172.21.15.169:3300/0 conn(0x7f67d00cf9b0 0x7f67d015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:39.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67cfe00640 1 --2- 172.21.15.87:0/1900057117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d0070f50 0x7f67d01540c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f67d0072ec0 tx=0x7f67c801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8608bccb3a0b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:39.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67cea00640 1 -- 172.21.15.87:0/1900057117 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80a5780 con 0x7f67d0070f50 2024-04-25T10:19:39.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67cea00640 1 -- 172.21.15.87:0/1900057117 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c80a5920 con 0x7f67d0070f50 2024-04-25T10:19:39.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67cea00640 1 -- 172.21.15.87:0/1900057117 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80a5cf0 con 0x7f67d0070f50 2024-04-25T10:19:39.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- 172.21.15.87:0/1900057117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d0070f50 msgr2=0x7f67d01540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:39.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 --2- 172.21.15.87:0/1900057117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d0070f50 0x7f67d01540c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f67d0072ec0 tx=0x7f67c801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- 172.21.15.87:0/1900057117 shutdown_connections 2024-04-25T10:19:39.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 --2- 172.21.15.87:0/1900057117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d0070f50 0x7f67d01540c0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 --2- 172.21.15.87:0/1900057117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d00b3360 0x7f67d0070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 --2- 172.21.15.87:0/1900057117 >> v2:172.21.15.169:3300/0 conn(0x7f67d00cf9b0 0x7f67d015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- 172.21.15.87:0/1900057117 >> 172.21.15.87:0/1900057117 conn(0x7f67d00cfe80 msgr2=0x7f67d00d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:39.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- 172.21.15.87:0/1900057117 shutdown_connections 2024-04-25T10:19:39.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- 172.21.15.87:0/1900057117 wait complete. 2024-04-25T10:19:39.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 Processor -- start 2024-04-25T10:19:39.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- start start 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d0070f50 0x7f67d01406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67d00b3360 0x7f67d01600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d00cf9b0 0x7f67d0160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0161920 con 0x7f67d00cf9b0 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d01617a0 con 0x7f67d00b3360 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.936+0000 7f67d5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0161aa0 con 0x7f67d0070f50 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d00cf9b0 0x7f67d0160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d0070f50 0x7f67d01406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d00cf9b0 0x7f67d0160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54606/0 (socket says 172.21.15.87:54606) 2024-04-25T10:19:39.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 -- 172.21.15.87:0/2486597587 learned_addr learned my addr 172.21.15.87:0/2486597587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:39.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67c6a00640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67d00b3360 0x7f67d01600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:39.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 -- 172.21.15.87:0/2486597587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d0070f50 msgr2=0x7f67d01406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:39.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d0070f50 0x7f67d01406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 -- 172.21.15.87:0/2486597587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67d00b3360 msgr2=0x7f67d01600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:39.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67d00b3360 0x7f67d01600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 -- 172.21.15.87:0/2486597587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d01550d0 con 0x7f67d00cf9b0 2024-04-25T10:19:39.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67cfe00640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d00cf9b0 0x7f67d0160620 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f67c801b0a0 tx=0x7f67c8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:39.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67c7e00640 1 -- 172.21.15.87:0/2486597587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80b8670 con 0x7f67d00cf9b0 2024-04-25T10:19:39.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67c7e00640 1 -- 172.21.15.87:0/2486597587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c80be070 con 0x7f67d00cf9b0 2024-04-25T10:19:39.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67c7e00640 1 -- 172.21.15.87:0/2486597587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80b8a70 con 0x7f67d00cf9b0 2024-04-25T10:19:39.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d01553b0 con 0x7f67d00cf9b0 2024-04-25T10:19:39.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d0155970 con 0x7f67d00cf9b0 2024-04-25T10:19:39.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d0075120 con 0x7f67d00cf9b0 2024-04-25T10:19:39.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67c7e00640 1 -- 172.21.15.87:0/2486597587 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67c80b8c10 con 0x7f67d00cf9b0 2024-04-25T10:19:39.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.940+0000 7f67c7e00640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f67ac066f60 0x7f67ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:39.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.944+0000 7f67c7e00640 1 -- 172.21.15.87:0/2486597587 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67c8120250 con 0x7f67d00cf9b0 2024-04-25T10:19:39.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.944+0000 7f67cf400640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f67ac066f60 0x7f67ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:39.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.944+0000 7f67cf400640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f67ac066f60 0x7f67ac069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f67c0006fd0 tx=0x7f67c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:39.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:39.944+0000 7f67c7e00640 1 -- 172.21.15.87:0/2486597587 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67c80165e0 con 0x7f67d00cf9b0 2024-04-25T10:19:40.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.132+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f67d00740e0 con 0x7f67d00cf9b0 2024-04-25T10:19:40.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.772+0000 7f67c7e00640 1 -- 172.21.15.87:0/2486597587 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f67c80f0ae0 con 0x7f67d00cf9b0 2024-04-25T10:19:40.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f67ac066f60 msgr2=0x7f67ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f67ac066f60 0x7f67ac069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f67c0006fd0 tx=0x7f67c0008040 comp rx=0 tx=0).stop 2024-04-25T10:19:40.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d00cf9b0 msgr2=0x7f67d0160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d00cf9b0 0x7f67d0160620 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f67c801b0a0 tx=0x7f67c8019790 comp rx=0 tx=0).stop 2024-04-25T10:19:40.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 shutdown_connections 2024-04-25T10:19:40.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f67ac066f60 0x7f67ac069420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67d00cf9b0 0x7f67d0160620 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67d00b3360 0x7f67d01600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 --2- 172.21.15.87:0/2486597587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67d0070f50 0x7f67d01406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 >> 172.21.15.87:0/2486597587 conn(0x7f67d00cfe80 msgr2=0x7f67d00adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 shutdown_connections 2024-04-25T10:19:40.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.776+0000 7f67d5200640 1 -- 172.21.15.87:0/2486597587 wait complete. 2024-04-25T10:19:40.800 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:19:40.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f82a00640 1 Processor -- start 2024-04-25T10:19:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f82a00640 1 -- start start 2024-04-25T10:19:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f82a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 0x7f7f7c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f82a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7f7c15bdc0 0x7f7f7c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f82a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c0ccfb0 con 0x7f7f7c0d1520 2024-04-25T10:19:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f82a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c0cd290 con 0x7f7f7c15bdc0 2024-04-25T10:19:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f82a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0cd570 con 0x7f7f7c15b9e0 2024-04-25T10:19:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f80a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7f7c0d1520 0x7f7f7c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39862/0 (socket says 172.21.15.87:39862) 2024-04-25T10:19:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f80a00640 1 -- 172.21.15.87:0/3992158794 learned_addr learned my addr 172.21.15.87:0/3992158794 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f73400640 1 --2- 172.21.15.87:0/3992158794 >> v2:172.21.15.169:3300/0 conn(0x7f7f7c15bdc0 0x7f7f7c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 0x7f7f7c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884892912 0 0) 0x7f7f7c0ccfb0 con 0x7f7f7c0d1520 2024-04-25T10:19:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f60003530 con 0x7f7f7c0d1520 2024-04-25T10:19:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878409854 0 0) 0x7f7f60003530 con 0x7f7f7c0d1520 2024-04-25T10:19:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 msgr2=0x7f7f7c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 0x7f7f7c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 >> v2:172.21.15.169:3300/0 conn(0x7f7f7c15bdc0 msgr2=0x7f7f7c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 --2- 172.21.15.87:0/3992158794 >> v2:172.21.15.169:3300/0 conn(0x7f7f7c15bdc0 0x7f7f7c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0cd850 con 0x7f7f7c0d1520 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f7f74001a00 con 0x7f7f7c0d1520 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 0x7f7f7c0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f73400640 1 --2- 172.21.15.87:0/3992158794 >> v2:172.21.15.169:3300/0 conn(0x7f7f7c15bdc0 0x7f7f7c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 >> v1:172.21.15.87:6789/0 conn(0x7f7f7c0d1520 legacy=0x7f7f7c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f600039c0 0x7f7f60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60004810 0x7f7f60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 0x7f7f600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0cd290 con 0x7f7f600039c0 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0cd570 con 0x7f7f60004810 2024-04-25T10:19:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6000b9d0 con 0x7f7f7c15b9e0 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f600039c0 0x7f7f60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f73400640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60004810 0x7f7f60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f80a00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 0x7f7f600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 -- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 msgr2=0x7f7f600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f7c15b9e0 0x7f7f600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 -- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60004810 msgr2=0x7f7f60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60004810 0x7f7f60004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f7be00640 1 -- 172.21.15.87:0/3992158794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f60003530 con 0x7f7f600039c0 2024-04-25T10:19:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.936+0000 7f7f73400640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60004810 0x7f7f60004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f7be00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f600039c0 0x7f7f60003de0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009cb0 tx=0x7f7f6c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4ea23e57c34b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0b2070 con 0x7f7f600039c0 2024-04-25T10:19:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c0a55d0 con 0x7f7f600039c0 2024-04-25T10:19:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f7b400640 1 -- 172.21.15.87:0/3992158794 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c009880 con 0x7f7f600039c0 2024-04-25T10:19:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f82a00640 1 -- 172.21.15.87:0/3992158794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f600039c0 msgr2=0x7f7f60003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f82a00640 1 --2- 172.21.15.87:0/3992158794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f600039c0 0x7f7f60003de0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009cb0 tx=0x7f7f6c01cc10 comp rx=0 tx=0).stop 2024-04-25T10:19:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f80a00640 1 -- 172.21.15.87:0/3992158794 reap_dead start 2024-04-25T10:19:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f82a00640 1 -- 172.21.15.87:0/3992158794 shutdown_connections 2024-04-25T10:19:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f82a00640 1 -- 172.21.15.87:0/3992158794 >> 172.21.15.87:0/3992158794 conn(0x7f7f7c09a9d0 msgr2=0x7f7f7c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f82a00640 1 -- 172.21.15.87:0/3992158794 shutdown_connections 2024-04-25T10:19:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.940+0000 7f7f82a00640 1 -- 172.21.15.87:0/3992158794 wait complete. 2024-04-25T10:19:40.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 Processor -- start 2024-04-25T10:19:40.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 -- start start 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f600039c0 0x7f7f7c14b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c15e470 0x7f7f7c15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f7c09d660 0x7f7f7c09dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c65f0 con 0x7f7f7c09d660 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c6470 con 0x7f7f7c15e470 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c6770 con 0x7f7f600039c0 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f7c09d660 0x7f7f7c09dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f7c09d660 0x7f7f7c09dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54620/0 (socket says 172.21.15.87:54620) 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 -- 172.21.15.87:0/2555251996 learned_addr learned my addr 172.21.15.87:0/2555251996 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f7be00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f600039c0 0x7f7f7c14b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f73400640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c15e470 0x7f7f7c15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 -- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f600039c0 msgr2=0x7f7f7c14b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f600039c0 0x7f7f7c14b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 -- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c15e470 msgr2=0x7f7f7c15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c15e470 0x7f7f7c15e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 -- 172.21.15.87:0/2555251996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c14bc10 con 0x7f7f7c09d660 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f73400640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c15e470 0x7f7f7c15e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f80a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f7c09d660 0x7f7f7c09dae0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0732f0 tx=0x7f7f7400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f78c00640 1 -- 172.21.15.87:0/2555251996 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74007ae0 con 0x7f7f7c09d660 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f78c00640 1 -- 172.21.15.87:0/2555251996 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400eda0 con 0x7f7f7c09d660 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f78c00640 1 -- 172.21.15.87:0/2555251996 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74014650 con 0x7f7f7c09d660 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c14bef0 con 0x7f7f7c09d660 2024-04-25T10:19:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.944+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c14c4b0 con 0x7f7f7c09d660 2024-04-25T10:19:40.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.948+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c04ef30 con 0x7f7f7c09d660 2024-04-25T10:19:40.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.948+0000 7f7f78c00640 1 -- 172.21.15.87:0/2555251996 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f740147f0 con 0x7f7f7c09d660 2024-04-25T10:19:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.948+0000 7f7f78c00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f58066f90 0x7f7f58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.948+0000 7f7f78c00640 1 -- 172.21.15.87:0/2555251996 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f74084e00 con 0x7f7f7c09d660 2024-04-25T10:19:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.948+0000 7f7f7be00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f58066f90 0x7f7f58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.948+0000 7f7f7be00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f58066f90 0x7f7f58069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0af000 tx=0x7f7f6c0046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:40.952+0000 7f7f78c00640 1 -- 172.21.15.87:0/2555251996 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f74055720 con 0x7f7f7c09d660 2024-04-25T10:19:41.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f7c06d4d0 con 0x7f7f7c09d660 2024-04-25T10:19:41.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f78c00640 1 -- 172.21.15.87:0/2555251996 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7f740597c0 con 0x7f7f7c09d660 2024-04-25T10:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:19:41.143 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f58066f90 msgr2=0x7f7f58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f58066f90 0x7f7f58069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0af000 tx=0x7f7f6c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f7c09d660 msgr2=0x7f7f7c09dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f7c09d660 0x7f7f7c09dae0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0732f0 tx=0x7f7f7400e530 comp rx=0 tx=0).stop 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 shutdown_connections 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f58066f90 0x7f7f58069450 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f7c09d660 0x7f7f7c09dae0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f7c15e470 0x7f7f7c15e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 --2- 172.21.15.87:0/2555251996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f600039c0 0x7f7f7c14b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.140+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 >> 172.21.15.87:0/2555251996 conn(0x7f7f7c09a9d0 msgr2=0x7f7f7c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.144+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 shutdown_connections 2024-04-25T10:19:41.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.144+0000 7f7f82a00640 1 -- 172.21.15.87:0/2555251996 wait complete. 2024-04-25T10:19:41.163 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:41.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0dac600640 1 Processor -- start 2024-04-25T10:19:41.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0dac600640 1 -- start start 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0dac600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da41550b0 0x7f0da40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0dac600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0da415b910 0x7f0da40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0dac600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da4071050 con 0x7f0da40cfe70 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0dac600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da415bda0 con 0x7f0da415b910 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0dac600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415d040 con 0x7f0da41550b0 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0da9200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0da40cfe70 0x7f0da415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39876/0 (socket says 172.21.15.87:39876) 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0da9200640 1 -- 172.21.15.87:0/3658362768 learned_addr learned my addr 172.21.15.87:0/3658362768 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0da9c00640 1 --2- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da41550b0 0x7f0da40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.296+0000 7f0daa600640 1 --2- 172.21.15.87:0/3658362768 >> v2:172.21.15.169:3300/0 conn(0x7f0da415b910 0x7f0da40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0da3e00640 1 -- 172.21.15.87:0/3658362768 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069916241 0 0) 0x7f0da4071050 con 0x7f0da40cfe70 2024-04-25T10:19:41.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0da3e00640 1 -- 172.21.15.87:0/3658362768 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d8c003530 con 0x7f0da40cfe70 2024-04-25T10:19:41.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0daa600640 1 -- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da41550b0 msgr2=0x7f0da40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0daa600640 1 --2- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da41550b0 0x7f0da40cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0daa600640 1 -- 172.21.15.87:0/3658362768 >> v1:172.21.15.87:6789/0 conn(0x7f0da40cfe70 legacy=0x7f0da415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0daa600640 1 -- 172.21.15.87:0/3658362768 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40712f0 con 0x7f0da415b910 2024-04-25T10:19:41.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0da9c00640 1 --2- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da41550b0 0x7f0da40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:41.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0daa600640 1 --2- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 0x7f0da40b3260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009d90 tx=0x7f0d9c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f0733f2a7e4f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0da3e00640 1 -- 172.21.15.87:0/3658362768 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0b2070 con 0x7f0da415b910 2024-04-25T10:19:41.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0da3e00640 1 -- 172.21.15.87:0/3658362768 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0a5920 con 0x7f0da415b910 2024-04-25T10:19:41.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.300+0000 7f0da3e00640 1 -- 172.21.15.87:0/3658362768 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0a5cf0 con 0x7f0da415b910 2024-04-25T10:19:41.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 msgr2=0x7f0da40b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 --2- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 0x7f0da40b3260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009d90 tx=0x7f0d9c01c970 comp rx=0 tx=0).stop 2024-04-25T10:19:41.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- 172.21.15.87:0/3658362768 shutdown_connections 2024-04-25T10:19:41.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 --2- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 0x7f0da40b3260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 --2- 172.21.15.87:0/3658362768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da41550b0 0x7f0da40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- 172.21.15.87:0/3658362768 >> 172.21.15.87:0/3658362768 conn(0x7f0da4066b70 msgr2=0x7f0da4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- 172.21.15.87:0/3658362768 shutdown_connections 2024-04-25T10:19:41.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- 172.21.15.87:0/3658362768 wait complete. 2024-04-25T10:19:41.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 Processor -- start 2024-04-25T10:19:41.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- start start 2024-04-25T10:19:41.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da40cfe70 0x7f0da413fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0da41550b0 0x7f0da41f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 0x7f0da406c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415c030 con 0x7f0da41550b0 2024-04-25T10:19:41.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40720b0 con 0x7f0da415b910 2024-04-25T10:19:41.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0dac600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415c1b0 con 0x7f0da40cfe70 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0da41550b0 0x7f0da41f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0da41550b0 0x7f0da41f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54624/0 (socket says 172.21.15.87:54624) 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 -- 172.21.15.87:0/37727470 learned_addr learned my addr 172.21.15.87:0/37727470 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9c00640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da40cfe70 0x7f0da413fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0daa600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 0x7f0da406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 -- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da40cfe70 msgr2=0x7f0da413fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da40cfe70 0x7f0da413fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 -- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 msgr2=0x7f0da406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 0x7f0da406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 -- 172.21.15.87:0/37727470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da41fa580 con 0x7f0da41550b0 2024-04-25T10:19:41.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.304+0000 7f0da9200640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0da41550b0 0x7f0da41f9c30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c450 tx=0x7f0d9000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da1600640 1 -- 172.21.15.87:0/37727470 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90017070 con 0x7f0da41550b0 2024-04-25T10:19:41.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da1600640 1 -- 172.21.15.87:0/37727470 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d90005cf0 con 0x7f0da41550b0 2024-04-25T10:19:41.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da1600640 1 -- 172.21.15.87:0/37727470 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90007290 con 0x7f0da41550b0 2024-04-25T10:19:41.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da41fa860 con 0x7f0da41550b0 2024-04-25T10:19:41.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da4140430 con 0x7f0da41550b0 2024-04-25T10:19:41.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da404ef30 con 0x7f0da41550b0 2024-04-25T10:19:41.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da1600640 1 -- 172.21.15.87:0/37727470 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d9001a070 con 0x7f0da41550b0 2024-04-25T10:19:41.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da1600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da1600640 1 -- 172.21.15.87:0/37727470 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d90084330 con 0x7f0da41550b0 2024-04-25T10:19:41.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da9c00640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.308+0000 7f0da9c00640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d84066f60 0x7f0d84069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009580 tx=0x7f0d98009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.312+0000 7f0da1600640 1 -- 172.21.15.87:0/37727470 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d90054cf0 con 0x7f0da41550b0 2024-04-25T10:19:41.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.468+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0da4159280 con 0x7f0da41550b0 2024-04-25T10:19:41.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.468+0000 7f0da1600640 1 -- 172.21.15.87:0/37727470 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16598 (secure 0 0 0) 0x7f0d90058d90 con 0x7f0da41550b0 2024-04-25T10:19:41.470 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:41.471 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:38.756098+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:35.484354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:19:36.111171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.468+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d84066f60 msgr2=0x7f0d84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.468+0000 7f0dac600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d84066f60 0x7f0d84069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009580 tx=0x7f0d98009190 comp rx=0 tx=0).stop 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.468+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0da41550b0 msgr2=0x7f0da41f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.468+0000 7f0dac600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0da41550b0 0x7f0da41f9c30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c450 tx=0x7f0d9000c910 comp rx=0 tx=0).stop 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 shutdown_connections 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da415b910 0x7f0da406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0da41550b0 0x7f0da41f9c30 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 --2- 172.21.15.87:0/37727470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da40cfe70 0x7f0da413fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 >> 172.21.15.87:0/37727470 conn(0x7f0da4066b70 msgr2=0x7f0da4153d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 shutdown_connections 2024-04-25T10:19:41.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.472+0000 7f0dac600640 1 -- 172.21.15.87:0/37727470 wait complete. 2024-04-25T10:19:41.491 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:19:41.491 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:41.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad60c00640 1 Processor -- start 2024-04-25T10:19:41.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad60c00640 1 -- start start 2024-04-25T10:19:41.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad60c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad5c15e050 0x7fad5c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad60c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c0b1af0 con 0x7fad5c0d1820 2024-04-25T10:19:41.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad60c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c0b1dd0 con 0x7fad5c0b13e0 2024-04-25T10:19:41.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad60c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c0b20b0 con 0x7fad5c15e050 2024-04-25T10:19:41.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad59e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fad5c0d1820 0x7fad5c0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39882/0 (socket says 172.21.15.87:39882) 2024-04-25T10:19:41.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad59e00640 1 -- 172.21.15.87:0/1051232623 learned_addr learned my addr 172.21.15.87:0/1051232623 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:41.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad59400640 1 --2- 172.21.15.87:0/1051232623 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15e050 0x7fad5c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921330806 0 0) 0x7fad5c0b1af0 con 0x7fad5c0d1820 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad40003530 con 0x7fad5c0d1820 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2994582790 0 0) 0x7fad40003530 con 0x7fad5c0d1820 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15e050 msgr2=0x7fad5c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 --2- 172.21.15.87:0/1051232623 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15e050 0x7fad5c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 >> v1:172.21.15.169:6789/0 conn(0x7fad5c0b13e0 legacy=0x7fad5c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0b2390 con 0x7fad5c0d1820 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fad4c001680 con 0x7fad5c0d1820 2024-04-25T10:19:41.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad59400640 1 --2- 172.21.15.87:0/1051232623 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15e050 0x7fad5c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.624+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 >> v1:172.21.15.87:6789/0 conn(0x7fad5c0d1820 legacy=0x7fad5c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400039c0 0x7fad40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad40004810 0x7fad40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad400051d0 0x7fad4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0b20b0 con 0x7fad400051d0 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4000c0d0 con 0x7fad400039c0 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4000c2d0 con 0x7fad40004810 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad400051d0 0x7fad4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad59e00640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400039c0 0x7fad40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad59400640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad40004810 0x7fad40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 -- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad40004810 msgr2=0x7fad40004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad40004810 0x7fad40004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 -- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400039c0 msgr2=0x7fad40003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400039c0 0x7fad40003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 -- 172.21.15.87:0/1051232623 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40003530 con 0x7fad400051d0 2024-04-25T10:19:41.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 -- 172.21.15.87:0/1051232623 reap_dead start 2024-04-25T10:19:41.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad5a800640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad400051d0 0x7fad4000b9d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fad54001f60 tx=0x7fad5401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985651f3b4e5f456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54001b60 con 0x7fad400051d0 2024-04-25T10:19:41.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad54003b80 con 0x7fad400051d0 2024-04-25T10:19:41.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.628+0000 7fad58a00640 1 -- 172.21.15.87:0/1051232623 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540050d0 con 0x7fad400051d0 2024-04-25T10:19:41.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- 172.21.15.87:0/1051232623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad400051d0 msgr2=0x7fad4000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad400051d0 0x7fad4000b9d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fad54001f60 tx=0x7fad5401c880 comp rx=0 tx=0).stop 2024-04-25T10:19:41.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- 172.21.15.87:0/1051232623 shutdown_connections 2024-04-25T10:19:41.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 --2- 172.21.15.87:0/1051232623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad400051d0 0x7fad4000b9d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- 172.21.15.87:0/1051232623 >> 172.21.15.87:0/1051232623 conn(0x7fad5c09a9d0 msgr2=0x7fad5c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- 172.21.15.87:0/1051232623 shutdown_connections 2024-04-25T10:19:41.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- 172.21.15.87:0/1051232623 wait complete. 2024-04-25T10:19:41.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 Processor -- start 2024-04-25T10:19:41.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- start start 2024-04-25T10:19:41.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400051d0 0x7fad5c0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad5c0b55f0 0x7fad5c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c1582f0 0x7fad5c150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0b1230 con 0x7fad5c0b55f0 2024-04-25T10:19:41.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0b10b0 con 0x7fad400051d0 2024-04-25T10:19:41.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad60c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0d0b90 con 0x7fad5c1582f0 2024-04-25T10:19:41.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad5c0b55f0 0x7fad5c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad5c0b55f0 0x7fad5c0b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54636/0 (socket says 172.21.15.87:54636) 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 -- 172.21.15.87:0/2727761576 learned_addr learned my addr 172.21.15.87:0/2727761576 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59e00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400051d0 0x7fad5c0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad5a800640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c1582f0 0x7fad5c150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 -- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c1582f0 msgr2=0x7fad5c150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c1582f0 0x7fad5c150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 -- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400051d0 msgr2=0x7fad5c0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400051d0 0x7fad5c0c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad59400640 1 -- 172.21.15.87:0/2727761576 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0b7120 con 0x7fad5c0b55f0 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.632+0000 7fad5a800640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c1582f0 0x7fad5c150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad59400640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad5c0b55f0 0x7fad5c0b6be0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fad48011a50 tx=0x7fad48011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad52000640 1 -- 172.21.15.87:0/2727761576 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800fc50 con 0x7fad5c0b55f0 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad52000640 1 -- 172.21.15.87:0/2727761576 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4800fdf0 con 0x7fad5c0b55f0 2024-04-25T10:19:41.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad52000640 1 -- 172.21.15.87:0/2727761576 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48014680 con 0x7fad5c0b55f0 2024-04-25T10:19:41.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c0b5b40 con 0x7fad5c0b55f0 2024-04-25T10:19:41.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c0b6100 con 0x7fad5c0b55f0 2024-04-25T10:19:41.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c04ef30 con 0x7fad5c0b55f0 2024-04-25T10:19:41.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad52000640 1 -- 172.21.15.87:0/2727761576 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad48012040 con 0x7fad5c0b55f0 2024-04-25T10:19:41.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad52000640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad3c066f90 0x7fad3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad52000640 1 -- 172.21.15.87:0/2727761576 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad48085720 con 0x7fad5c0b55f0 2024-04-25T10:19:41.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.636+0000 7fad59e00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad3c066f90 0x7fad3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.640+0000 7fad59e00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad3c066f90 0x7fad3c069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fad4c001970 tx=0x7fad4c001de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.640+0000 7fad52000640 1 -- 172.21.15.87:0/2727761576 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad48019030 con 0x7fad5c0b55f0 2024-04-25T10:19:41.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.796+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad5c0d0f20 con 0x7fad3c066f90 2024-04-25T10:19:41.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.800+0000 7fad52000640 1 -- 172.21.15.87:0/2727761576 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240160 (secure 0 0 0) 0x7fad5c0d0f20 con 0x7fad3c066f90 2024-04-25T10:19:41.806 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:41.808 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad3c066f90 msgr2=0x7fad3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad3c066f90 0x7fad3c069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fad4c001970 tx=0x7fad4c001de0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad5c0b55f0 msgr2=0x7fad5c0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad5c0b55f0 0x7fad5c0b6be0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fad48011a50 tx=0x7fad48011f10 comp rx=0 tx=0).stop 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 shutdown_connections 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fad3c066f90 0x7fad3c069450 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c1582f0 0x7fad5c150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad5c0b55f0 0x7fad5c0b6be0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 --2- 172.21.15.87:0/2727761576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad400051d0 0x7fad5c0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 >> 172.21.15.87:0/2727761576 conn(0x7fad5c09a9d0 msgr2=0x7fad5c0b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 shutdown_connections 2024-04-25T10:19:41.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:41.804+0000 7fad60c00640 1 -- 172.21.15.87:0/2727761576 wait complete. 2024-04-25T10:19:41.826 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-25T10:19:41.218078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425180,"kb_used_data":10500,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754549540,"statfs":{"total":773094113280,"available":772658728960,"internally_reserved":0,"allocated":10752000,"data_stored":5294741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.490911"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734216+0000","last_change":"2024-04-25T10:19:35.734216+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734216+0000","last_undegraded":"2024-04-25T10:19:35.734216+0000","last_fullsized":"2024-04-25T10:19:35.734216+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736633+0000","last_change":"2024-04-25T10:19:35.736633+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736633+0000","last_undegraded":"2024-04-25T10:19:35.736633+0000","last_fullsized":"2024-04-25T10:19:35.736633+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733352+0000","last_change":"2024-04-25T10:19:35.733352+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733352+0000","last_undegraded":"2024-04-25T10:19:35.733352+0000","last_fullsized":"2024-04-25T10:19:35.733352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734255+0000","last_change":"2024-04-25T10:19:35.734255+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734255+0000","last_undegraded":"2024-04-25T10:19:35.734255+0000","last_fullsized":"2024-04-25T10:19:35.734255+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730843+0000","last_change":"2024-04-25T10:19:35.730843+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730843+0000","last_undegraded":"2024-04-25T10:19:35.730843+0000","last_fullsized":"2024-04-25T10:19:35.730843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.732997+0000","last_change":"2024-04-25T10:19:35.732997+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.732997+0000","last_undegraded":"2024-04-25T10:19:35.732997+0000","last_fullsized":"2024-04-25T10:19:35.732997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736455+0000","last_change":"2024-04-25T10:19:35.736455+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736455+0000","last_undegraded":"2024-04-25T10:19:35.736455+0000","last_fullsized":"2024-04-25T10:19:35.736455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.732151+0000","last_change":"2024-04-25T10:19:35.732151+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.732151+0000","last_undegraded":"2024-04-25T10:19:35.732151+0000","last_fullsized":"2024-04-25T10:19:35.732151+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730843+0000","last_change":"2024-04-25T10:19:35.730843+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730843+0000","last_undegraded":"2024-04-25T10:19:35.730843+0000","last_fullsized":"2024-04-25T10:19:35.730843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734465+0000","last_change":"2024-04-25T10:19:35.734465+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734465+0000","last_undegraded":"2024-04-25T10:19:35.734465+0000","last_fullsized":"2024-04-25T10:19:35.734465+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736959+0000","last_change":"2024-04-25T10:19:35.736959+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736959+0000","last_undegraded":"2024-04-25T10:19:35.736959+0000","last_fullsized":"2024-04-25T10:19:35.736959+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.735956+0000","last_change":"2024-04-25T10:19:35.735956+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.735956+0000","last_undegraded":"2024-04-25T10:19:35.735956+0000","last_fullsized":"2024-04-25T10:19:35.735956+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736591+0000","last_change":"2024-04-25T10:19:35.736591+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736591+0000","last_undegraded":"2024-04-25T10:19:35.736591+0000","last_fullsized":"2024-04-25T10:19:35.736591+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736668+0000","last_change":"2024-04-25T10:19:35.736668+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736668+0000","last_undegraded":"2024-04-25T10:19:35.736668+0000","last_fullsized":"2024-04-25T10:19:35.736668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.735490+0000","last_change":"2024-04-25T10:19:35.735490+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.735490+0000","last_undegraded":"2024-04-25T10:19:35.735490+0000","last_fullsized":"2024-04-25T10:19:35.735490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730838+0000","last_change":"2024-04-25T10:19:35.730838+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730838+0000","last_undegraded":"2024-04-25T10:19:35.730838+0000","last_fullsized":"2024-04-25T10:19:35.730838+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736052+0000","last_change":"2024-04-25T10:19:35.736052+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736052+0000","last_undegraded":"2024-04-25T10:19:35.736052+0000","last_fullsized":"2024-04-25T10:19:35.736052+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737398+0000","last_change":"2024-04-25T10:19:35.737398+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737398+0000","last_undegraded":"2024-04-25T10:19:35.737398+0000","last_fullsized":"2024-04-25T10:19:35.737398+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733018+0000","last_change":"2024-04-25T10:19:35.733018+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733018+0000","last_undegraded":"2024-04-25T10:19:35.733018+0000","last_fullsized":"2024-04-25T10:19:35.733018+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733720+0000","last_change":"2024-04-25T10:19:35.733720+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733720+0000","last_undegraded":"2024-04-25T10:19:35.733720+0000","last_fullsized":"2024-04-25T10:19:35.733720+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738416+0000","last_change":"2024-04-25T10:19:35.738416+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738416+0000","last_undegraded":"2024-04-25T10:19:35.738416+0000","last_fullsized":"2024-04-25T10:19:35.738416+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738476+0000","last_change":"2024-04-25T10:19:35.738476+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738476+0000","last_undegraded":"2024-04-25T10:19:35.738476+0000","last_fullsized":"2024-04-25T10:19:35.738476+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733021+0000","last_change":"2024-04-25T10:19:35.733021+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733021+0000","last_undegraded":"2024-04-25T10:19:35.733021+0000","last_fullsized":"2024-04-25T10:19:35.733021+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738638+0000","last_change":"2024-04-25T10:19:35.738638+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738638+0000","last_undegraded":"2024-04-25T10:19:35.738638+0000","last_fullsized":"2024-04-25T10:19:35.738638+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733038+0000","last_change":"2024-04-25T10:19:35.733038+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733038+0000","last_undegraded":"2024-04-25T10:19:35.733038+0000","last_fullsized":"2024-04-25T10:19:35.733038+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734969+0000","last_change":"2024-04-25T10:19:35.734969+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734969+0000","last_undegraded":"2024-04-25T10:19:35.734969+0000","last_fullsized":"2024-04-25T10:19:35.734969+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.739016+0000","last_change":"2024-04-25T10:19:35.739016+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.739016+0000","last_undegraded":"2024-04-25T10:19:35.739016+0000","last_fullsized":"2024-04-25T10:19:35.739016+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733342+0000","last_change":"2024-04-25T10:19:35.733342+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733342+0000","last_undegraded":"2024-04-25T10:19:35.733342+0000","last_fullsized":"2024-04-25T10:19:35.733342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736673+0000","last_change":"2024-04-25T10:19:35.736673+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736673+0000","last_undegraded":"2024-04-25T10:19:35.736673+0000","last_fullsized":"2024-04-25T10:19:35.736673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734323+0000","last_change":"2024-04-25T10:19:35.734323+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734323+0000","last_undegraded":"2024-04-25T10:19:35.734323+0000","last_fullsized":"2024-04-25T10:19:35.734323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737221+0000","last_change":"2024-04-25T10:19:35.737221+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737221+0000","last_undegraded":"2024-04-25T10:19:35.737221+0000","last_fullsized":"2024-04-25T10:19:35.737221+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737320+0000","last_change":"2024-04-25T10:19:35.737320+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737320+0000","last_undegraded":"2024-04-25T10:19:35.737320+0000","last_fullsized":"2024-04-25T10:19:35.737320+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:19:35.737283+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:19:35.737283+0000","last_peered":"2024-04-25T10:19:35.737283+0000","last_clean":"2024-04-25T10:19:35.737283+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:19:35.737283+0000","last_undegraded":"2024-04-25T10:19:35.737283+0000","last_fullsized":"2024-04-25T10:19:35.737283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50984,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320856,"statfs":{"total":96636764160,"available":96584556544,"internally_reserved":0,"allocated":1794048,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54548,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317292,"statfs":{"total":96636764160,"available":96580907008,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44824,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327016,"statfs":{"total":96636764160,"available":96590864384,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50404,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321436,"statfs":{"total":96636764160,"available":96585150464,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76764,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295076,"statfs":{"total":96636764160,"available":96558157824,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46832,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325008,"statfs":{"total":96636764160,"available":96588808192,"internally_reserved":0,"allocated":1794048,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:41.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:42.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc19200640 1 Processor -- start 2024-04-25T10:19:42.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc19200640 1 -- start start 2024-04-25T10:19:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc19200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 0x7efc1415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc19200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc140ad840 con 0x7efc140cf930 2024-04-25T10:19:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc19200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1415e9f0 con 0x7efc140ae020 2024-04-25T10:19:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc19200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1415bb90 con 0x7efc14070c20 2024-04-25T10:19:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc11e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efc140cf930 0x7efc140cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39894/0 (socket says 172.21.15.87:39894) 2024-04-25T10:19:42.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.004+0000 7efc11e00640 1 -- 172.21.15.87:0/3745751168 learned_addr learned my addr 172.21.15.87:0/3745751168 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:42.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc0be00640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 0x7efc1415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789460605 0 0) 0x7efc140ad840 con 0x7efc140cf930 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf8003530 con 0x7efc140cf930 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560692930 0 0) 0x7efbf8003530 con 0x7efc140cf930 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 msgr2=0x7efc1415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 0x7efc1415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 >> v1:172.21.15.169:6789/0 conn(0x7efc140ae020 legacy=0x7efc140b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1415ecd0 con 0x7efc140cf930 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7efc04001680 con 0x7efc140cf930 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 >> v1:172.21.15.87:6789/0 conn(0x7efc140cf930 legacy=0x7efc140cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf80039f0 0x7efbf8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbf8004810 0x7efbf8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 0x7efbf80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1415bb90 con 0x7efbf8004810 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf800bab0 con 0x7efbf80039f0 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf800bcb0 con 0x7efc14070c20 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc0be00640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbf8004810 0x7efbf8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc11e00640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf80039f0 0x7efbf8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc12800640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 0x7efbf80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc0be00640 1 -- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 msgr2=0x7efbf80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc0be00640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc14070c20 0x7efbf80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc0be00640 1 -- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf80039f0 msgr2=0x7efbf8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc0be00640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf80039f0 0x7efbf8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.008+0000 7efc0be00640 1 -- 172.21.15.87:0/3745751168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8003530 con 0x7efbf8004810 2024-04-25T10:19:42.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc0be00640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbf8004810 0x7efbf8004c90 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009ab0 tx=0x7efbfc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e4ac550c4fe1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0b2070 con 0x7efbf8004810 2024-04-25T10:19:42.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0a5620 con 0x7efbf8004810 2024-04-25T10:19:42.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc11400640 1 -- 172.21.15.87:0/3745751168 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0096b0 con 0x7efbf8004810 2024-04-25T10:19:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- 172.21.15.87:0/3745751168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbf8004810 msgr2=0x7efbf8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 --2- 172.21.15.87:0/3745751168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbf8004810 0x7efbf8004c90 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009ab0 tx=0x7efbfc01ca30 comp rx=0 tx=0).stop 2024-04-25T10:19:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc12800640 1 -- 172.21.15.87:0/3745751168 reap_dead start 2024-04-25T10:19:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- 172.21.15.87:0/3745751168 shutdown_connections 2024-04-25T10:19:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- 172.21.15.87:0/3745751168 >> 172.21.15.87:0/3745751168 conn(0x7efc14066b70 msgr2=0x7efc14066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- 172.21.15.87:0/3745751168 shutdown_connections 2024-04-25T10:19:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- 172.21.15.87:0/3745751168 wait complete. 2024-04-25T10:19:42.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 Processor -- start 2024-04-25T10:19:42.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- start start 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8004810 0x7efc141f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc140af2e0 0x7efc14144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc14159e10 0x7efc1415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1406e130 con 0x7efc14159e10 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140adb00 con 0x7efc140af2e0 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.012+0000 7efc19200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b30c0 con 0x7efbf8004810 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc14159e10 0x7efc1415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc0be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc140af2e0 0x7efc14144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc0be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc140af2e0 0x7efc14144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:37456/0 (socket says 172.21.15.87:37456) 2024-04-25T10:19:42.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc14159e10 0x7efc1415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54648/0 (socket says 172.21.15.87:54648) 2024-04-25T10:19:42.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc0be00640 1 -- 172.21.15.87:0/3919119445 learned_addr learned my addr 172.21.15.87:0/3919119445 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:42.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc11e00640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8004810 0x7efc141f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 -- 172.21.15.87:0/3919119445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8004810 msgr2=0x7efc141f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8004810 0x7efc141f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 -- 172.21.15.87:0/3919119445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc140af2e0 msgr2=0x7efc14144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc140af2e0 0x7efc14144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 -- 172.21.15.87:0/3919119445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc141447a0 con 0x7efc14159e10 2024-04-25T10:19:42.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc12800640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc14159e10 0x7efc1415a290 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7efc140af190 tx=0x7efc0c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc0a000640 1 -- 172.21.15.87:0/3919119445 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c01a070 con 0x7efc14159e10 2024-04-25T10:19:42.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc0a000640 1 -- 172.21.15.87:0/3919119445 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c006d70 con 0x7efc14159e10 2024-04-25T10:19:42.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc0a000640 1 -- 172.21.15.87:0/3919119445 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c003ba0 con 0x7efc14159e10 2024-04-25T10:19:42.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc141f41c0 con 0x7efc14159e10 2024-04-25T10:19:42.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc141f4780 con 0x7efc14159e10 2024-04-25T10:19:42.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1404ef30 con 0x7efc14159e10 2024-04-25T10:19:42.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.016+0000 7efc0a000640 1 -- 172.21.15.87:0/3919119445 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc0c005070 con 0x7efc14159e10 2024-04-25T10:19:42.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.020+0000 7efc0a000640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efbf0066f60 0x7efbf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.020+0000 7efc0a000640 1 -- 172.21.15.87:0/3919119445 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc0c0846f0 con 0x7efc14159e10 2024-04-25T10:19:42.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.020+0000 7efc11e00640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efbf0066f60 0x7efbf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.020+0000 7efc11e00640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efbf0066f60 0x7efbf0069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efc04005180 tx=0x7efc04001dc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.024+0000 7efc0a000640 1 -- 172.21.15.87:0/3919119445 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc0c055010 con 0x7efc14159e10 2024-04-25T10:19:42.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.172+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc14067470 con 0x7efbf0066f60 2024-04-25T10:19:42.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.180+0000 7efc0a000640 1 -- 172.21.15.87:0/3919119445 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240160 (secure 0 0 0) 0x7efc14067470 con 0x7efbf0066f60 2024-04-25T10:19:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:42.186 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efbf0066f60 msgr2=0x7efbf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efbf0066f60 0x7efbf0069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efc04005180 tx=0x7efc04001dc0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc14159e10 msgr2=0x7efc1415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc14159e10 0x7efc1415a290 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7efc140af190 tx=0x7efc0c00fc90 comp rx=0 tx=0).stop 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 shutdown_connections 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efbf0066f60 0x7efbf0069420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc14159e10 0x7efc1415a290 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc140af2e0 0x7efc14144260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 --2- 172.21.15.87:0/3919119445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8004810 0x7efc141f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 >> 172.21.15.87:0/3919119445 conn(0x7efc14066b70 msgr2=0x7efc141606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 shutdown_connections 2024-04-25T10:19:42.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.184+0000 7efc19200640 1 -- 172.21.15.87:0/3919119445 wait complete. 2024-04-25T10:19:42.204 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-25T10:19:41.218078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425180,"kb_used_data":10500,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754549540,"statfs":{"total":773094113280,"available":772658728960,"internally_reserved":0,"allocated":10752000,"data_stored":5294741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.490911"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734216+0000","last_change":"2024-04-25T10:19:35.734216+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734216+0000","last_undegraded":"2024-04-25T10:19:35.734216+0000","last_fullsized":"2024-04-25T10:19:35.734216+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736633+0000","last_change":"2024-04-25T10:19:35.736633+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736633+0000","last_undegraded":"2024-04-25T10:19:35.736633+0000","last_fullsized":"2024-04-25T10:19:35.736633+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733352+0000","last_change":"2024-04-25T10:19:35.733352+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733352+0000","last_undegraded":"2024-04-25T10:19:35.733352+0000","last_fullsized":"2024-04-25T10:19:35.733352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734255+0000","last_change":"2024-04-25T10:19:35.734255+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734255+0000","last_undegraded":"2024-04-25T10:19:35.734255+0000","last_fullsized":"2024-04-25T10:19:35.734255+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730843+0000","last_change":"2024-04-25T10:19:35.730843+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730843+0000","last_undegraded":"2024-04-25T10:19:35.730843+0000","last_fullsized":"2024-04-25T10:19:35.730843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.732997+0000","last_change":"2024-04-25T10:19:35.732997+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.732997+0000","last_undegraded":"2024-04-25T10:19:35.732997+0000","last_fullsized":"2024-04-25T10:19:35.732997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736455+0000","last_change":"2024-04-25T10:19:35.736455+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736455+0000","last_undegraded":"2024-04-25T10:19:35.736455+0000","last_fullsized":"2024-04-25T10:19:35.736455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.732151+0000","last_change":"2024-04-25T10:19:35.732151+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.732151+0000","last_undegraded":"2024-04-25T10:19:35.732151+0000","last_fullsized":"2024-04-25T10:19:35.732151+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730843+0000","last_change":"2024-04-25T10:19:35.730843+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730843+0000","last_undegraded":"2024-04-25T10:19:35.730843+0000","last_fullsized":"2024-04-25T10:19:35.730843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734465+0000","last_change":"2024-04-25T10:19:35.734465+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734465+0000","last_undegraded":"2024-04-25T10:19:35.734465+0000","last_fullsized":"2024-04-25T10:19:35.734465+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736959+0000","last_change":"2024-04-25T10:19:35.736959+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736959+0000","last_undegraded":"2024-04-25T10:19:35.736959+0000","last_fullsized":"2024-04-25T10:19:35.736959+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.735956+0000","last_change":"2024-04-25T10:19:35.735956+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.735956+0000","last_undegraded":"2024-04-25T10:19:35.735956+0000","last_fullsized":"2024-04-25T10:19:35.735956+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736591+0000","last_change":"2024-04-25T10:19:35.736591+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736591+0000","last_undegraded":"2024-04-25T10:19:35.736591+0000","last_fullsized":"2024-04-25T10:19:35.736591+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736668+0000","last_change":"2024-04-25T10:19:35.736668+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736668+0000","last_undegraded":"2024-04-25T10:19:35.736668+0000","last_fullsized":"2024-04-25T10:19:35.736668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.735490+0000","last_change":"2024-04-25T10:19:35.735490+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.735490+0000","last_undegraded":"2024-04-25T10:19:35.735490+0000","last_fullsized":"2024-04-25T10:19:35.735490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730838+0000","last_change":"2024-04-25T10:19:35.730838+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730838+0000","last_undegraded":"2024-04-25T10:19:35.730838+0000","last_fullsized":"2024-04-25T10:19:35.730838+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736052+0000","last_change":"2024-04-25T10:19:35.736052+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736052+0000","last_undegraded":"2024-04-25T10:19:35.736052+0000","last_fullsized":"2024-04-25T10:19:35.736052+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737398+0000","last_change":"2024-04-25T10:19:35.737398+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737398+0000","last_undegraded":"2024-04-25T10:19:35.737398+0000","last_fullsized":"2024-04-25T10:19:35.737398+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733018+0000","last_change":"2024-04-25T10:19:35.733018+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733018+0000","last_undegraded":"2024-04-25T10:19:35.733018+0000","last_fullsized":"2024-04-25T10:19:35.733018+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733720+0000","last_change":"2024-04-25T10:19:35.733720+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733720+0000","last_undegraded":"2024-04-25T10:19:35.733720+0000","last_fullsized":"2024-04-25T10:19:35.733720+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738416+0000","last_change":"2024-04-25T10:19:35.738416+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738416+0000","last_undegraded":"2024-04-25T10:19:35.738416+0000","last_fullsized":"2024-04-25T10:19:35.738416+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738476+0000","last_change":"2024-04-25T10:19:35.738476+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738476+0000","last_undegraded":"2024-04-25T10:19:35.738476+0000","last_fullsized":"2024-04-25T10:19:35.738476+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733021+0000","last_change":"2024-04-25T10:19:35.733021+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733021+0000","last_undegraded":"2024-04-25T10:19:35.733021+0000","last_fullsized":"2024-04-25T10:19:35.733021+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738638+0000","last_change":"2024-04-25T10:19:35.738638+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738638+0000","last_undegraded":"2024-04-25T10:19:35.738638+0000","last_fullsized":"2024-04-25T10:19:35.738638+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733038+0000","last_change":"2024-04-25T10:19:35.733038+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733038+0000","last_undegraded":"2024-04-25T10:19:35.733038+0000","last_fullsized":"2024-04-25T10:19:35.733038+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734969+0000","last_change":"2024-04-25T10:19:35.734969+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734969+0000","last_undegraded":"2024-04-25T10:19:35.734969+0000","last_fullsized":"2024-04-25T10:19:35.734969+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.739016+0000","last_change":"2024-04-25T10:19:35.739016+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.739016+0000","last_undegraded":"2024-04-25T10:19:35.739016+0000","last_fullsized":"2024-04-25T10:19:35.739016+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733342+0000","last_change":"2024-04-25T10:19:35.733342+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733342+0000","last_undegraded":"2024-04-25T10:19:35.733342+0000","last_fullsized":"2024-04-25T10:19:35.733342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736673+0000","last_change":"2024-04-25T10:19:35.736673+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736673+0000","last_undegraded":"2024-04-25T10:19:35.736673+0000","last_fullsized":"2024-04-25T10:19:35.736673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734323+0000","last_change":"2024-04-25T10:19:35.734323+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734323+0000","last_undegraded":"2024-04-25T10:19:35.734323+0000","last_fullsized":"2024-04-25T10:19:35.734323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737221+0000","last_change":"2024-04-25T10:19:35.737221+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737221+0000","last_undegraded":"2024-04-25T10:19:35.737221+0000","last_fullsized":"2024-04-25T10:19:35.737221+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737320+0000","last_change":"2024-04-25T10:19:35.737320+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737320+0000","last_undegraded":"2024-04-25T10:19:35.737320+0000","last_fullsized":"2024-04-25T10:19:35.737320+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:19:35.737283+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:19:35.737283+0000","last_peered":"2024-04-25T10:19:35.737283+0000","last_clean":"2024-04-25T10:19:35.737283+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:19:35.737283+0000","last_undegraded":"2024-04-25T10:19:35.737283+0000","last_fullsized":"2024-04-25T10:19:35.737283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50984,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320856,"statfs":{"total":96636764160,"available":96584556544,"internally_reserved":0,"allocated":1794048,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54548,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317292,"statfs":{"total":96636764160,"available":96580907008,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44824,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327016,"statfs":{"total":96636764160,"available":96590864384,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50404,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321436,"statfs":{"total":96636764160,"available":96585150464,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76764,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295076,"statfs":{"total":96636764160,"available":96558157824,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46832,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325008,"statfs":{"total":96636764160,"available":96588808192,"internally_reserved":0,"allocated":1794048,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:42.207 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54c1400640 1 Processor -- start 2024-04-25T10:19:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54c1400640 1 -- start start 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54c1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54bc15bc40 0x7f54bc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54c1400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f54bc0cda10 0x7f54bc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54c1400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54bc0b2770 con 0x7f54bc0cf930 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54c1400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc06c670 con 0x7f54bc15bc40 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54c1400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc06c950 con 0x7f54bc0cda10 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54ba800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54bc15bc40 0x7f54bc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54ba800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54bc15bc40 0x7f54bc06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54664/0 (socket says 172.21.15.87:54664) 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.384+0000 7f54ba800640 1 -- 172.21.15.87:0/3731404614 learned_addr learned my addr 172.21.15.87:0/3731404614 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54b1e00640 1 --2- 172.21.15.87:0/3731404614 >> v2:172.21.15.169:3300/0 conn(0x7f54bc0cda10 0x7f54bc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54ba800640 1 -- 172.21.15.87:0/3731404614 >> v2:172.21.15.169:3300/0 conn(0x7f54bc0cda10 msgr2=0x7f54bc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54ba800640 1 --2- 172.21.15.87:0/3731404614 >> v2:172.21.15.169:3300/0 conn(0x7f54bc0cda10 0x7f54bc0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54ba800640 1 -- 172.21.15.87:0/3731404614 >> v1:172.21.15.87:6789/0 conn(0x7f54bc0cf930 legacy=0x7f54bc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54ba800640 1 -- 172.21.15.87:0/3731404614 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc06cc30 con 0x7f54bc15bc40 2024-04-25T10:19:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54ba800640 1 --2- 172.21.15.87:0/3731404614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc06c090 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0726f0 tx=0x7f54ac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd893f7c055cda0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54b9e00640 1 -- 172.21.15.87:0/3731404614 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0a57d0 con 0x7f54bc15bc40 2024-04-25T10:19:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54b9e00640 1 -- 172.21.15.87:0/3731404614 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac0a5970 con 0x7f54bc15bc40 2024-04-25T10:19:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54b9e00640 1 -- 172.21.15.87:0/3731404614 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0a5d40 con 0x7f54bc15bc40 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54c1400640 1 -- 172.21.15.87:0/3731404614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 msgr2=0x7f54bc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54c1400640 1 --2- 172.21.15.87:0/3731404614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc06c090 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0726f0 tx=0x7f54ac01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54c1400640 1 -- 172.21.15.87:0/3731404614 shutdown_connections 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54c1400640 1 --2- 172.21.15.87:0/3731404614 >> v2:172.21.15.169:3300/0 conn(0x7f54bc0cda10 0x7f54bc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54c1400640 1 --2- 172.21.15.87:0/3731404614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc06c090 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.388+0000 7f54c1400640 1 -- 172.21.15.87:0/3731404614 >> 172.21.15.87:0/3731404614 conn(0x7f54bc066b70 msgr2=0x7f54bc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 -- 172.21.15.87:0/3731404614 shutdown_connections 2024-04-25T10:19:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 -- 172.21.15.87:0/3731404614 wait complete. 2024-04-25T10:19:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 Processor -- start 2024-04-25T10:19:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 -- start start 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54bc0cda10 0x7f54bc14b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54bc0cf930 0x7f54bc15e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc168b50 con 0x7f54bc15bc40 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc1689d0 con 0x7f54bc0cda10 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54c1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc168cd0 con 0x7f54bc0cf930 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc15ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54680/0 (socket says 172.21.15.87:54680) 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54b1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54bc0cf930 0x7f54bc15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 -- 172.21.15.87:0/2163293509 learned_addr learned my addr 172.21.15.87:0/2163293509 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:42.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54ba800640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54bc0cda10 0x7f54bc14b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 -- 172.21.15.87:0/2163293509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54bc0cf930 msgr2=0x7f54bc15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54bc0cf930 0x7f54bc15e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 -- 172.21.15.87:0/2163293509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54bc0cda10 msgr2=0x7f54bc14b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54bc0cda10 0x7f54bc14b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 -- 172.21.15.87:0/2163293509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc15c3a0 con 0x7f54bc15bc40 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54bb200640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc15ed50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f54b400c340 tx=0x7f54b400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.392+0000 7f54b3400640 1 -- 172.21.15.87:0/2163293509 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4005220 con 0x7f54bc15bc40 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54b3400640 1 -- 172.21.15.87:0/2163293509 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b40053c0 con 0x7f54bc15bc40 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54b3400640 1 -- 172.21.15.87:0/2163293509 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4007270 con 0x7f54bc15bc40 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc15c680 con 0x7f54bc15bc40 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc15cf20 con 0x7f54bc15bc40 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc15c850 con 0x7f54bc15bc40 2024-04-25T10:19:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54b3400640 1 -- 172.21.15.87:0/2163293509 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54b4019020 con 0x7f54bc15bc40 2024-04-25T10:19:42.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54b3400640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5498066f60 0x7f5498069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54b3400640 1 -- 172.21.15.87:0/2163293509 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54b40841e0 con 0x7f54bc15bc40 2024-04-25T10:19:42.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54ba800640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5498066f60 0x7f5498069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.396+0000 7f54ba800640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5498066f60 0x7f5498069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0ae000 tx=0x7f54ac0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.400+0000 7f54b3400640 1 -- 172.21.15.87:0/2163293509 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54b4054ba0 con 0x7f54bc15bc40 2024-04-25T10:19:42.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.552+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54bc04ef30 con 0x7f5498066f60 2024-04-25T10:19:42.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.556+0000 7f54b3400640 1 -- 172.21.15.87:0/2163293509 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240160 (secure 0 0 0) 0x7f54bc04ef30 con 0x7f5498066f60 2024-04-25T10:19:42.561 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:19:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5498066f60 msgr2=0x7f5498069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5498066f60 0x7f5498069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0ae000 tx=0x7f54ac0a7070 comp rx=0 tx=0).stop 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 msgr2=0x7f54bc15ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc15ed50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f54b400c340 tx=0x7f54b400c800 comp rx=0 tx=0).stop 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 shutdown_connections 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5498066f60 0x7f5498069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54bc15bc40 0x7f54bc15ed50 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54bc0cf930 0x7f54bc15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 --2- 172.21.15.87:0/2163293509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54bc0cda10 0x7f54bc14b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 >> 172.21.15.87:0/2163293509 conn(0x7f54bc066b70 msgr2=0x7f54bc0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 shutdown_connections 2024-04-25T10:19:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.560+0000 7f54c1400640 1 -- 172.21.15.87:0/2163293509 wait complete. 2024-04-25T10:19:42.582 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-25T10:19:41.218078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425180,"kb_used_data":10500,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754549540,"statfs":{"total":773094113280,"available":772658728960,"internally_reserved":0,"allocated":10752000,"data_stored":5294741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.490911"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734216+0000","last_change":"2024-04-25T10:19:35.734216+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734216+0000","last_undegraded":"2024-04-25T10:19:35.734216+0000","last_fullsized":"2024-04-25T10:19:35.734216+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736633+0000","last_change":"2024-04-25T10:19:35.736633+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736633+0000","last_undegraded":"2024-04-25T10:19:35.736633+0000","last_fullsized":"2024-04-25T10:19:35.736633+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733352+0000","last_change":"2024-04-25T10:19:35.733352+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733352+0000","last_undegraded":"2024-04-25T10:19:35.733352+0000","last_fullsized":"2024-04-25T10:19:35.733352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734255+0000","last_change":"2024-04-25T10:19:35.734255+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734255+0000","last_undegraded":"2024-04-25T10:19:35.734255+0000","last_fullsized":"2024-04-25T10:19:35.734255+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730843+0000","last_change":"2024-04-25T10:19:35.730843+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730843+0000","last_undegraded":"2024-04-25T10:19:35.730843+0000","last_fullsized":"2024-04-25T10:19:35.730843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.732997+0000","last_change":"2024-04-25T10:19:35.732997+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.732997+0000","last_undegraded":"2024-04-25T10:19:35.732997+0000","last_fullsized":"2024-04-25T10:19:35.732997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736455+0000","last_change":"2024-04-25T10:19:35.736455+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736455+0000","last_undegraded":"2024-04-25T10:19:35.736455+0000","last_fullsized":"2024-04-25T10:19:35.736455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.732151+0000","last_change":"2024-04-25T10:19:35.732151+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.732151+0000","last_undegraded":"2024-04-25T10:19:35.732151+0000","last_fullsized":"2024-04-25T10:19:35.732151+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730843+0000","last_change":"2024-04-25T10:19:35.730843+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730843+0000","last_undegraded":"2024-04-25T10:19:35.730843+0000","last_fullsized":"2024-04-25T10:19:35.730843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734465+0000","last_change":"2024-04-25T10:19:35.734465+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734465+0000","last_undegraded":"2024-04-25T10:19:35.734465+0000","last_fullsized":"2024-04-25T10:19:35.734465+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736959+0000","last_change":"2024-04-25T10:19:35.736959+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736959+0000","last_undegraded":"2024-04-25T10:19:35.736959+0000","last_fullsized":"2024-04-25T10:19:35.736959+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.735956+0000","last_change":"2024-04-25T10:19:35.735956+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.735956+0000","last_undegraded":"2024-04-25T10:19:35.735956+0000","last_fullsized":"2024-04-25T10:19:35.735956+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736591+0000","last_change":"2024-04-25T10:19:35.736591+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736591+0000","last_undegraded":"2024-04-25T10:19:35.736591+0000","last_fullsized":"2024-04-25T10:19:35.736591+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736668+0000","last_change":"2024-04-25T10:19:35.736668+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736668+0000","last_undegraded":"2024-04-25T10:19:35.736668+0000","last_fullsized":"2024-04-25T10:19:35.736668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.735490+0000","last_change":"2024-04-25T10:19:35.735490+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.735490+0000","last_undegraded":"2024-04-25T10:19:35.735490+0000","last_fullsized":"2024-04-25T10:19:35.735490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.730838+0000","last_change":"2024-04-25T10:19:35.730838+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.730838+0000","last_undegraded":"2024-04-25T10:19:35.730838+0000","last_fullsized":"2024-04-25T10:19:35.730838+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736052+0000","last_change":"2024-04-25T10:19:35.736052+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736052+0000","last_undegraded":"2024-04-25T10:19:35.736052+0000","last_fullsized":"2024-04-25T10:19:35.736052+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737398+0000","last_change":"2024-04-25T10:19:35.737398+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737398+0000","last_undegraded":"2024-04-25T10:19:35.737398+0000","last_fullsized":"2024-04-25T10:19:35.737398+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733018+0000","last_change":"2024-04-25T10:19:35.733018+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733018+0000","last_undegraded":"2024-04-25T10:19:35.733018+0000","last_fullsized":"2024-04-25T10:19:35.733018+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733720+0000","last_change":"2024-04-25T10:19:35.733720+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733720+0000","last_undegraded":"2024-04-25T10:19:35.733720+0000","last_fullsized":"2024-04-25T10:19:35.733720+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738416+0000","last_change":"2024-04-25T10:19:35.738416+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738416+0000","last_undegraded":"2024-04-25T10:19:35.738416+0000","last_fullsized":"2024-04-25T10:19:35.738416+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738476+0000","last_change":"2024-04-25T10:19:35.738476+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738476+0000","last_undegraded":"2024-04-25T10:19:35.738476+0000","last_fullsized":"2024-04-25T10:19:35.738476+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733021+0000","last_change":"2024-04-25T10:19:35.733021+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733021+0000","last_undegraded":"2024-04-25T10:19:35.733021+0000","last_fullsized":"2024-04-25T10:19:35.733021+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.738638+0000","last_change":"2024-04-25T10:19:35.738638+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.738638+0000","last_undegraded":"2024-04-25T10:19:35.738638+0000","last_fullsized":"2024-04-25T10:19:35.738638+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733038+0000","last_change":"2024-04-25T10:19:35.733038+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733038+0000","last_undegraded":"2024-04-25T10:19:35.733038+0000","last_fullsized":"2024-04-25T10:19:35.733038+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734969+0000","last_change":"2024-04-25T10:19:35.734969+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734969+0000","last_undegraded":"2024-04-25T10:19:35.734969+0000","last_fullsized":"2024-04-25T10:19:35.734969+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.739016+0000","last_change":"2024-04-25T10:19:35.739016+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.739016+0000","last_undegraded":"2024-04-25T10:19:35.739016+0000","last_fullsized":"2024-04-25T10:19:35.739016+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.733342+0000","last_change":"2024-04-25T10:19:35.733342+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.733342+0000","last_undegraded":"2024-04-25T10:19:35.733342+0000","last_fullsized":"2024-04-25T10:19:35.733342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.736673+0000","last_change":"2024-04-25T10:19:35.736673+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.736673+0000","last_undegraded":"2024-04-25T10:19:35.736673+0000","last_fullsized":"2024-04-25T10:19:35.736673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.734323+0000","last_change":"2024-04-25T10:19:35.734323+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.734323+0000","last_undegraded":"2024-04-25T10:19:35.734323+0000","last_fullsized":"2024-04-25T10:19:35.734323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737221+0000","last_change":"2024-04-25T10:19:35.737221+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737221+0000","last_undegraded":"2024-04-25T10:19:35.737221+0000","last_fullsized":"2024-04-25T10:19:35.737221+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:19:36.726514+0000","last_change":"2024-04-25T10:19:36.726514+0000","last_active":"2024-04-25T10:19:36.726514+0000","last_peered":"2024-04-25T10:19:36.726514+0000","last_clean":"2024-04-25T10:19:36.726514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:36.726514+0000","last_undegraded":"2024-04-25T10:19:36.726514+0000","last_fullsized":"2024-04-25T10:19:36.726514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:19:35.737320+0000","last_change":"2024-04-25T10:19:35.737320+0000","last_active":"2024-04-25T10:19:35.720477+0000","last_peered":"2024-04-25T10:19:35.720477+0000","last_clean":"2024-04-25T10:19:35.720477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:19:35.737320+0000","last_undegraded":"2024-04-25T10:19:35.737320+0000","last_fullsized":"2024-04-25T10:19:35.737320+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:19:35.737283+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:19:35.737283+0000","last_peered":"2024-04-25T10:19:35.737283+0000","last_clean":"2024-04-25T10:19:35.737283+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:19:35.737283+0000","last_undegraded":"2024-04-25T10:19:35.737283+0000","last_fullsized":"2024-04-25T10:19:35.737283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50984,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320856,"statfs":{"total":96636764160,"available":96584556544,"internally_reserved":0,"allocated":1794048,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54548,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317292,"statfs":{"total":96636764160,"available":96580907008,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44824,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327016,"statfs":{"total":96636764160,"available":96590864384,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50404,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321436,"statfs":{"total":96636764160,"available":96585150464,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76764,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295076,"statfs":{"total":96636764160,"available":96558157824,"internally_reserved":0,"allocated":1187840,"data_stored":510615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46832,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325008,"statfs":{"total":96636764160,"available":96588808192,"internally_reserved":0,"allocated":1794048,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-04-25T10:19:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-04-25T10:19:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-04-25T10:19:42.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.734216+0000', 'last_change': '2024-04-25T10:19:35.734216+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.734216+0000', 'last_undegraded': '2024-04-25T10:19:35.734216+0000', 'last_fullsized': '2024-04-25T10:19:35.734216+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.736633+0000', 'last_change': '2024-04-25T10:19:35.736633+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.736633+0000', 'last_undegraded': '2024-04-25T10:19:35.736633+0000', 'last_fullsized': '2024-04-25T10:19:35.736633+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-04-25T10:19:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.733352+0000', 'last_change': '2024-04-25T10:19:35.733352+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.733352+0000', 'last_undegraded': '2024-04-25T10:19:35.733352+0000', 'last_fullsized': '2024-04-25T10:19:35.733352+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.734255+0000', 'last_change': '2024-04-25T10:19:35.734255+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.734255+0000', 'last_undegraded': '2024-04-25T10:19:35.734255+0000', 'last_fullsized': '2024-04-25T10:19:35.734255+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.730843+0000', 'last_change': '2024-04-25T10:19:35.730843+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.730843+0000', 'last_undegraded': '2024-04-25T10:19:35.730843+0000', 'last_fullsized': '2024-04-25T10:19:35.730843+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-04-25T10:19:42.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.732997+0000', 'last_change': '2024-04-25T10:19:35.732997+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.732997+0000', 'last_undegraded': '2024-04-25T10:19:35.732997+0000', 'last_fullsized': '2024-04-25T10:19:35.732997+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.736455+0000', 'last_change': '2024-04-25T10:19:35.736455+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.736455+0000', 'last_undegraded': '2024-04-25T10:19:35.736455+0000', 'last_fullsized': '2024-04-25T10:19:35.736455+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.732151+0000', 'last_change': '2024-04-25T10:19:35.732151+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.732151+0000', 'last_undegraded': '2024-04-25T10:19:35.732151+0000', 'last_fullsized': '2024-04-25T10:19:35.732151+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-04-25T10:19:42.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.730843+0000', 'last_change': '2024-04-25T10:19:35.730843+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.730843+0000', 'last_undegraded': '2024-04-25T10:19:35.730843+0000', 'last_fullsized': '2024-04-25T10:19:35.730843+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.734465+0000', 'last_change': '2024-04-25T10:19:35.734465+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.734465+0000', 'last_undegraded': '2024-04-25T10:19:35.734465+0000', 'last_fullsized': '2024-04-25T10:19:35.734465+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.736959+0000', 'last_change': '2024-04-25T10:19:35.736959+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.736959+0000', 'last_undegraded': '2024-04-25T10:19:35.736959+0000', 'last_fullsized': '2024-04-25T10:19:35.736959+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:19:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.735956+0000', 'last_change': '2024-04-25T10:19:35.735956+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.735956+0000', 'last_undegraded': '2024-04-25T10:19:35.735956+0000', 'last_fullsized': '2024-04-25T10:19:35.735956+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.736591+0000', 'last_change': '2024-04-25T10:19:35.736591+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.736591+0000', 'last_undegraded': '2024-04-25T10:19:35.736591+0000', 'last_fullsized': '2024-04-25T10:19:35.736591+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-25T10:19:42.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.736668+0000', 'last_change': '2024-04-25T10:19:35.736668+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.736668+0000', 'last_undegraded': '2024-04-25T10:19:35.736668+0000', 'last_fullsized': '2024-04-25T10:19:35.736668+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.735490+0000', 'last_change': '2024-04-25T10:19:35.735490+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.735490+0000', 'last_undegraded': '2024-04-25T10:19:35.735490+0000', 'last_fullsized': '2024-04-25T10:19:35.735490+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.730838+0000', 'last_change': '2024-04-25T10:19:35.730838+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.730838+0000', 'last_undegraded': '2024-04-25T10:19:35.730838+0000', 'last_fullsized': '2024-04-25T10:19:35.730838+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-25T10:19:42.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.736052+0000', 'last_change': '2024-04-25T10:19:35.736052+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.736052+0000', 'last_undegraded': '2024-04-25T10:19:35.736052+0000', 'last_fullsized': '2024-04-25T10:19:35.736052+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.737398+0000', 'last_change': '2024-04-25T10:19:35.737398+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.737398+0000', 'last_undegraded': '2024-04-25T10:19:35.737398+0000', 'last_fullsized': '2024-04-25T10:19:35.737398+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.733018+0000', 'last_change': '2024-04-25T10:19:35.733018+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.733018+0000', 'last_undegraded': '2024-04-25T10:19:35.733018+0000', 'last_fullsized': '2024-04-25T10:19:35.733018+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T10:19:42.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.733720+0000', 'last_change': '2024-04-25T10:19:35.733720+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.733720+0000', 'last_undegraded': '2024-04-25T10:19:35.733720+0000', 'last_fullsized': '2024-04-25T10:19:35.733720+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.738416+0000', 'last_change': '2024-04-25T10:19:35.738416+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.738416+0000', 'last_undegraded': '2024-04-25T10:19:35.738416+0000', 'last_fullsized': '2024-04-25T10:19:35.738416+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.738476+0000', 'last_change': '2024-04-25T10:19:35.738476+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.738476+0000', 'last_undegraded': '2024-04-25T10:19:35.738476+0000', 'last_fullsized': '2024-04-25T10:19:35.738476+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T10:19:42.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.733021+0000', 'last_change': '2024-04-25T10:19:35.733021+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.733021+0000', 'last_undegraded': '2024-04-25T10:19:35.733021+0000', 'last_fullsized': '2024-04-25T10:19:35.733021+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.738638+0000', 'last_change': '2024-04-25T10:19:35.738638+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.738638+0000', 'last_undegraded': '2024-04-25T10:19:35.738638+0000', 'last_fullsized': '2024-04-25T10:19:35.738638+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.733038+0000', 'last_change': '2024-04-25T10:19:35.733038+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.733038+0000', 'last_undegraded': '2024-04-25T10:19:35.733038+0000', 'last_fullsized': '2024-04-25T10:19:35.733038+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T10:19:42.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.734969+0000', 'last_change': '2024-04-25T10:19:35.734969+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.734969+0000', 'last_undegraded': '2024-04-25T10:19:35.734969+0000', 'last_fullsized': '2024-04-25T10:19:35.734969+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.739016+0000', 'last_change': '2024-04-25T10:19:35.739016+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.739016+0000', 'last_undegraded': '2024-04-25T10:19:35.739016+0000', 'last_fullsized': '2024-04-25T10:19:35.739016+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.733342+0000', 'last_change': '2024-04-25T10:19:35.733342+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.733342+0000', 'last_undegraded': '2024-04-25T10:19:35.733342+0000', 'last_fullsized': '2024-04-25T10:19:35.733342+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.736673+0000', 'last_change': '2024-04-25T10:19:35.736673+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.736673+0000', 'last_undegraded': '2024-04-25T10:19:35.736673+0000', 'last_fullsized': '2024-04-25T10:19:35.736673+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.734323+0000', 'last_change': '2024-04-25T10:19:35.734323+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.734323+0000', 'last_undegraded': '2024-04-25T10:19:35.734323+0000', 'last_fullsized': '2024-04-25T10:19:35.734323+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.737221+0000', 'last_change': '2024-04-25T10:19:35.737221+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.737221+0000', 'last_undegraded': '2024-04-25T10:19:35.737221+0000', 'last_fullsized': '2024-04-25T10:19:35.737221+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:19:36.726514+0000', 'last_change': '2024-04-25T10:19:36.726514+0000', 'last_active': '2024-04-25T10:19:36.726514+0000', 'last_peered': '2024-04-25T10:19:36.726514+0000', 'last_clean': '2024-04-25T10:19:36.726514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:36.726514+0000', 'last_undegraded': '2024-04-25T10:19:36.726514+0000', 'last_fullsized': '2024-04-25T10:19:36.726514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:36.726514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:19:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T10:19:42.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:19:35.737320+0000', 'last_change': '2024-04-25T10:19:35.737320+0000', 'last_active': '2024-04-25T10:19:35.720477+0000', 'last_peered': '2024-04-25T10:19:35.720477+0000', 'last_clean': '2024-04-25T10:19:35.720477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:19:35.737320+0000', 'last_undegraded': '2024-04-25T10:19:35.737320+0000', 'last_fullsized': '2024-04-25T10:19:35.737320+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'last_clean_scrub_stamp': '2024-04-25T10:19:35.720477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:19:42.602 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:19:42.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91c600640 1 Processor -- start 2024-04-25T10:19:42.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91c600640 1 -- start start 2024-04-25T10:19:42.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 0x7fc9140cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91c600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9140cc2c0 con 0x7fc9140cbc80 2024-04-25T10:19:42.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91c600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9140cc5a0 con 0x7fc9140cf310 2024-04-25T10:19:42.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9140cf770 con 0x7fc91415a9f0 2024-04-25T10:19:42.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91a600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc9140cbc80 0x7fc91406d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39904/0 (socket says 172.21.15.87:39904) 2024-04-25T10:19:42.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91a600640 1 -- 172.21.15.87:0/673438440 learned_addr learned my addr 172.21.15.87:0/673438440 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:42.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc919c00640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 0x7fc9140cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64312366 0 0) 0x7fc9140cc2c0 con 0x7fc9140cbc80 2024-04-25T10:19:42.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8fc005740 con 0x7fc9140cbc80 2024-04-25T10:19:42.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 948877105 0 0) 0x7fc8fc005740 con 0x7fc9140cbc80 2024-04-25T10:19:42.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 msgr2=0x7fc9140cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 0x7fc9140cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 >> v1:172.21.15.169:6789/0 conn(0x7fc9140cf310 legacy=0x7fc9140cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140cc880 con 0x7fc9140cbc80 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc90c001a00 con 0x7fc9140cbc80 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 >> v1:172.21.15.87:6789/0 conn(0x7fc9140cbc80 legacy=0x7fc91406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 0x7fc8fc005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc8fc006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8fc006f60 0x7fc8fc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc00dc10 con 0x7fc8fc006140 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc00ddc0 con 0x7fc8fc006f60 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc00dfc0 con 0x7fc91415a9f0 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc919200640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc8fc006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc91a600640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8fc006f60 0x7fc8fc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.760+0000 7fc919c00640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 0x7fc8fc005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc919200640 1 -- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 msgr2=0x7fc8fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc919200640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 0x7fc8fc005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc919200640 1 -- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8fc006f60 msgr2=0x7fc8fc0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc919200640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8fc006f60 0x7fc8fc0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc919200640 1 -- 172.21.15.87:0/673438440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc005740 con 0x7fc8fc006140 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc919c00640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91415a9f0 0x7fc8fc005c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc91a600640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8fc006f60 0x7fc8fc0073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc919200640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc8fc006a20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc900001860 tx=0x7fc90001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d88731e7c5a9452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000b0070 con 0x7fc8fc006140 2024-04-25T10:19:42.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9000ab600 con 0x7fc8fc006140 2024-04-25T10:19:42.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc913e00640 1 -- 172.21.15.87:0/673438440 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900001460 con 0x7fc8fc006140 2024-04-25T10:19:42.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc91c600640 1 -- 172.21.15.87:0/673438440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 msgr2=0x7fc8fc006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc91c600640 1 --2- 172.21.15.87:0/673438440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc8fc006a20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc900001860 tx=0x7fc90001c440 comp rx=0 tx=0).stop 2024-04-25T10:19:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc91a600640 1 -- 172.21.15.87:0/673438440 reap_dead start 2024-04-25T10:19:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc91c600640 1 -- 172.21.15.87:0/673438440 shutdown_connections 2024-04-25T10:19:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc91c600640 1 -- 172.21.15.87:0/673438440 >> 172.21.15.87:0/673438440 conn(0x7fc9140b2410 msgr2=0x7fc9140b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.764+0000 7fc91c600640 1 -- 172.21.15.87:0/673438440 shutdown_connections 2024-04-25T10:19:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 -- 172.21.15.87:0/673438440 wait complete. 2024-04-25T10:19:42.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 Processor -- start 2024-04-25T10:19:42.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 -- start start 2024-04-25T10:19:42.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc9141f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9141f4dd0 0x7fc9140b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9141f5230 0x7fc9140b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91406e500 con 0x7fc8fc006140 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91406e380 con 0x7fc9141f4dd0 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91406e680 con 0x7fc9141f5230 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc9141f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc9141f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54688/0 (socket says 172.21.15.87:54688) 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 -- 172.21.15.87:0/4034407385 learned_addr learned my addr 172.21.15.87:0/4034407385 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919200640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9141f4dd0 0x7fc9140b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91a600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9141f5230 0x7fc9140b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 -- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9141f5230 msgr2=0x7fc9140b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9141f5230 0x7fc9140b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 -- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9141f4dd0 msgr2=0x7fc9140b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9141f4dd0 0x7fc9140b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 -- 172.21.15.87:0/4034407385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91415c3b0 con 0x7fc8fc006140 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91a600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9141f5230 0x7fc9140b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc919c00640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc9141f2130 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc9080116a0 tx=0x7fc908011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc911600640 1 -- 172.21.15.87:0/4034407385 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908004220 con 0x7fc8fc006140 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc911600640 1 -- 172.21.15.87:0/4034407385 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9080043c0 con 0x7fc8fc006140 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91415c690 con 0x7fc8fc006140 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc911600640 1 -- 172.21.15.87:0/4034407385 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90800f9e0 con 0x7fc8fc006140 2024-04-25T10:19:42.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.768+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91415cc50 con 0x7fc8fc006140 2024-04-25T10:19:42.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.772+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91404ef30 con 0x7fc8fc006140 2024-04-25T10:19:42.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.772+0000 7fc911600640 1 -- 172.21.15.87:0/4034407385 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc90800fb80 con 0x7fc8fc006140 2024-04-25T10:19:42.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.772+0000 7fc911600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8f4066f90 0x7fc8f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.772+0000 7fc911600640 1 -- 172.21.15.87:0/4034407385 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9080857a0 con 0x7fc8fc006140 2024-04-25T10:19:42.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.772+0000 7fc919200640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8f4066f90 0x7fc8f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.772+0000 7fc919200640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8f4066f90 0x7fc8f4069450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc9000ad000 tx=0x7fc900005040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.776+0000 7fc911600640 1 -- 172.21.15.87:0/4034407385 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc908019030 con 0x7fc8fc006140 2024-04-25T10:19:42.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.984+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc9140b0390 con 0x7fc8fc006140 2024-04-25T10:19:42.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.984+0000 7fc911600640 1 -- 172.21.15.87:0/4034407385 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1659 (secure 0 0 0) 0x7fc908056710 con 0x7fc8fc006140 2024-04-25T10:19:42.987 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:42.987 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":464,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":435384320,"bytes_avail":772658728960,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:19:17.213750+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:19:42.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8f4066f90 msgr2=0x7fc8f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8f4066f90 0x7fc8f4069450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc9000ad000 tx=0x7fc900005040 comp rx=0 tx=0).stop 2024-04-25T10:19:42.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 msgr2=0x7fc9141f2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc9141f2130 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc9080116a0 tx=0x7fc908011b60 comp rx=0 tx=0).stop 2024-04-25T10:19:42.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 shutdown_connections 2024-04-25T10:19:42.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8f4066f90 0x7fc8f4069450 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9141f5230 0x7fc9140b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9141f4dd0 0x7fc9140b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 --2- 172.21.15.87:0/4034407385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8fc006140 0x7fc9141f2130 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 >> 172.21.15.87:0/4034407385 conn(0x7fc9140b2410 msgr2=0x7fc914067040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 shutdown_connections 2024-04-25T10:19:42.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:42.988+0000 7fc91c600640 1 -- 172.21.15.87:0/4034407385 wait complete. 2024-04-25T10:19:43.008 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:19:43.008 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:43.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.140+0000 7fe2be000640 1 Processor -- start 2024-04-25T10:19:43.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.140+0000 7fe2be000640 1 -- start start 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2be000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2be000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2b80cda10 0x7fe2b80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2be000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b80b2770 con 0x7fe2b80cf930 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2be000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b806c670 con 0x7fe2b80cda10 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2be000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806c950 con 0x7fe2b815bc40 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2afe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2b80cda10 0x7fe2b80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2afe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2b80cda10 0x7fe2b80b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54696/0 (socket says 172.21.15.87:54696) 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2afe00640 1 -- 172.21.15.87:0/1782342182 learned_addr learned my addr 172.21.15.87:0/1782342182 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:43.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2b7400640 1 --2- 172.21.15.87:0/1782342182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2afe00640 1 -- 172.21.15.87:0/1782342182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 msgr2=0x7fe2b806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:43.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2afe00640 1 --2- 172.21.15.87:0/1782342182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2afe00640 1 -- 172.21.15.87:0/1782342182 >> v1:172.21.15.169:6789/0 conn(0x7fe2b80cf930 legacy=0x7fe2b815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.144+0000 7fe2afe00640 1 -- 172.21.15.87:0/1782342182 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b806cc30 con 0x7fe2b80cda10 2024-04-25T10:19:43.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2afe00640 1 --2- 172.21.15.87:0/1782342182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cda10 0x7fe2b80b4610 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009870 tx=0x7fe2a001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18f461de44f061e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2b6a00640 1 -- 172.21.15.87:0/1782342182 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00a7070 con 0x7fe2b80cda10 2024-04-25T10:19:43.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2b6a00640 1 -- 172.21.15.87:0/1782342182 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a00b0070 con 0x7fe2b80cda10 2024-04-25T10:19:43.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2b6a00640 1 -- 172.21.15.87:0/1782342182 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00ab890 con 0x7fe2b80cda10 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- 172.21.15.87:0/1782342182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cda10 msgr2=0x7fe2b80b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 --2- 172.21.15.87:0/1782342182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cda10 0x7fe2b80b4610 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009870 tx=0x7fe2a001c560 comp rx=0 tx=0).stop 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- 172.21.15.87:0/1782342182 shutdown_connections 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 --2- 172.21.15.87:0/1782342182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cda10 0x7fe2b80b4610 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 --2- 172.21.15.87:0/1782342182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- 172.21.15.87:0/1782342182 >> 172.21.15.87:0/1782342182 conn(0x7fe2b8066b70 msgr2=0x7fe2b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- 172.21.15.87:0/1782342182 shutdown_connections 2024-04-25T10:19:43.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- 172.21.15.87:0/1782342182 wait complete. 2024-04-25T10:19:43.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 Processor -- start 2024-04-25T10:19:43.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- start start 2024-04-25T10:19:43.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b80cda10 0x7fe2b81489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cf930 0x7fe2b815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8168b50 con 0x7fe2b80cf930 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81689d0 con 0x7fe2b80cda10 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.148+0000 7fe2be000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8168cd0 con 0x7fe2b815bc40 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cf930 0x7fe2b815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cf930 0x7fe2b815cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54702/0 (socket says 172.21.15.87:54702) 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 -- 172.21.15.87:0/2535529450 learned_addr learned my addr 172.21.15.87:0/2535529450 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2b7400640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b80cda10 0x7fe2b81489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2b7e00640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 -- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 msgr2=0x7fe2b815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 -- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b80cda10 msgr2=0x7fe2b81489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b80cda10 0x7fe2b81489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 -- 172.21.15.87:0/2535529450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b815d7d0 con 0x7fe2b80cf930 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2afe00640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cf930 0x7fe2b815cca0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe2a001ac10 tx=0x7fe2a001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2af400640 1 -- 172.21.15.87:0/2535529450 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00be5f0 con 0x7fe2b80cf930 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2af400640 1 -- 172.21.15.87:0/2535529450 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a00be7b0 con 0x7fe2b80cf930 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2af400640 1 -- 172.21.15.87:0/2535529450 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00beba0 con 0x7fe2b80cf930 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b815dab0 con 0x7fe2b80cf930 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8155660 con 0x7fe2b80cf930 2024-04-25T10:19:43.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b804ef30 con 0x7fe2b80cf930 2024-04-25T10:19:43.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2af400640 1 -- 172.21.15.87:0/2535529450 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2a00b43f0 con 0x7fe2b80cf930 2024-04-25T10:19:43.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2af400640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe29406b370 0x7fe29406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.152+0000 7fe2af400640 1 -- 172.21.15.87:0/2535529450 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2a011f9c0 con 0x7fe2b80cf930 2024-04-25T10:19:43.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.156+0000 7fe2b7400640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe29406b370 0x7fe29406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.156+0000 7fe2b7400640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe29406b370 0x7fe29406d830 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8007a60 tx=0x7fe2a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.160+0000 7fe2af400640 1 -- 172.21.15.87:0/2535529450 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2a0016150 con 0x7fe2b80cf930 2024-04-25T10:19:43.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.312+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2b80af1c0 con 0x7fe29406b370 2024-04-25T10:19:43.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.316+0000 7fe2af400640 1 -- 172.21.15.87:0/2535529450 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252182 (secure 0 0 0) 0x7fe2b80af1c0 con 0x7fe29406b370 2024-04-25T10:19:43.322 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:19:43.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe29406b370 msgr2=0x7fe29406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe29406b370 0x7fe29406d830 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8007a60 tx=0x7fe2a8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:43.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cf930 msgr2=0x7fe2b815cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cf930 0x7fe2b815cca0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe2a001ac10 tx=0x7fe2a001ada0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 shutdown_connections 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe29406b370 0x7fe29406d830 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b815bc40 0x7fe2b815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2b80cf930 0x7fe2b815cca0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 --2- 172.21.15.87:0/2535529450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b80cda10 0x7fe2b81489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 >> 172.21.15.87:0/2535529450 conn(0x7fe2b8066b70 msgr2=0x7fe2b815a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 shutdown_connections 2024-04-25T10:19:43.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:43.320+0000 7fe2be000640 1 -- 172.21.15.87:0/2535529450 wait complete. 2024-04-25T10:19:43.342 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-04-25T10:19:43.218386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413072,"kb_used_data":10888,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754561648,"statfs":{"total":773094113280,"available":772671127552,"internally_reserved":0,"allocated":11149312,"data_stored":5471812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":411684401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.485913"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839751+0000","last_change":"2024-04-25T10:19:37.781818+0000","last_active":"2024-04-25T10:19:38.839751+0000","last_peered":"2024-04-25T10:19:38.839751+0000","last_clean":"2024-04-25T10:19:38.839751+0000","last_became_active":"2024-04-25T10:19:37.779903+0000","last_became_peered":"2024-04-25T10:19:37.779903+0000","last_unstale":"2024-04-25T10:19:38.839751+0000","last_undegraded":"2024-04-25T10:19:38.839751+0000","last_fullsized":"2024-04-25T10:19:38.839751+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:39.766282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768990+0000","last_change":"2024-04-25T10:19:37.772495+0000","last_active":"2024-04-25T10:19:38.768990+0000","last_peered":"2024-04-25T10:19:38.768990+0000","last_clean":"2024-04-25T10:19:38.768990+0000","last_became_active":"2024-04-25T10:19:37.772073+0000","last_became_peered":"2024-04-25T10:19:37.772073+0000","last_unstale":"2024-04-25T10:19:38.768990+0000","last_undegraded":"2024-04-25T10:19:38.768990+0000","last_fullsized":"2024-04-25T10:19:38.768990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.849204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839526+0000","last_change":"2024-04-25T10:19:37.779548+0000","last_active":"2024-04-25T10:19:38.839526+0000","last_peered":"2024-04-25T10:19:38.839526+0000","last_clean":"2024-04-25T10:19:38.839526+0000","last_became_active":"2024-04-25T10:19:37.779212+0000","last_became_peered":"2024-04-25T10:19:37.779212+0000","last_unstale":"2024-04-25T10:19:38.839526+0000","last_undegraded":"2024-04-25T10:19:38.839526+0000","last_fullsized":"2024-04-25T10:19:38.839526+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:35.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958170+0000","last_change":"2024-04-25T10:19:37.773021+0000","last_active":"2024-04-25T10:19:38.958170+0000","last_peered":"2024-04-25T10:19:38.958170+0000","last_clean":"2024-04-25T10:19:38.958170+0000","last_became_active":"2024-04-25T10:19:37.772738+0000","last_became_peered":"2024-04-25T10:19:37.772738+0000","last_unstale":"2024-04-25T10:19:38.958170+0000","last_undegraded":"2024-04-25T10:19:38.958170+0000","last_fullsized":"2024-04-25T10:19:38.958170+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:31.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125010+0000","last_change":"2024-04-25T10:19:37.790648+0000","last_active":"2024-04-25T10:19:40.125010+0000","last_peered":"2024-04-25T10:19:40.125010+0000","last_clean":"2024-04-25T10:19:40.125010+0000","last_became_active":"2024-04-25T10:19:37.790336+0000","last_became_peered":"2024-04-25T10:19:37.790336+0000","last_unstale":"2024-04-25T10:19:40.125010+0000","last_undegraded":"2024-04-25T10:19:40.125010+0000","last_fullsized":"2024-04-25T10:19:40.125010+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:14.153645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125605+0000","last_change":"2024-04-25T10:19:37.771284+0000","last_active":"2024-04-25T10:19:40.125605+0000","last_peered":"2024-04-25T10:19:40.125605+0000","last_clean":"2024-04-25T10:19:40.125605+0000","last_became_active":"2024-04-25T10:19:37.770967+0000","last_became_peered":"2024-04-25T10:19:37.770967+0000","last_unstale":"2024-04-25T10:19:40.125605+0000","last_undegraded":"2024-04-25T10:19:40.125605+0000","last_fullsized":"2024-04-25T10:19:40.125605+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:54.184769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125555+0000","last_change":"2024-04-25T10:19:37.771217+0000","last_active":"2024-04-25T10:19:40.125555+0000","last_peered":"2024-04-25T10:19:40.125555+0000","last_clean":"2024-04-25T10:19:40.125555+0000","last_became_active":"2024-04-25T10:19:37.770914+0000","last_became_peered":"2024-04-25T10:19:37.770914+0000","last_unstale":"2024-04-25T10:19:40.125555+0000","last_undegraded":"2024-04-25T10:19:40.125555+0000","last_fullsized":"2024-04-25T10:19:40.125555+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.645465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768746+0000","last_change":"2024-04-25T10:19:37.779008+0000","last_active":"2024-04-25T10:19:38.768746+0000","last_peered":"2024-04-25T10:19:38.768746+0000","last_clean":"2024-04-25T10:19:38.768746+0000","last_became_active":"2024-04-25T10:19:37.778578+0000","last_became_peered":"2024-04-25T10:19:37.778578+0000","last_unstale":"2024-04-25T10:19:38.768746+0000","last_undegraded":"2024-04-25T10:19:38.768746+0000","last_fullsized":"2024-04-25T10:19:38.768746+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768211+0000","last_change":"2024-04-25T10:19:37.768155+0000","last_active":"2024-04-25T10:19:38.768211+0000","last_peered":"2024-04-25T10:19:38.768211+0000","last_clean":"2024-04-25T10:19:38.768211+0000","last_became_active":"2024-04-25T10:19:37.766771+0000","last_became_peered":"2024-04-25T10:19:37.766771+0000","last_unstale":"2024-04-25T10:19:38.768211+0000","last_undegraded":"2024-04-25T10:19:38.768211+0000","last_fullsized":"2024-04-25T10:19:38.768211+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:35.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125286+0000","last_change":"2024-04-25T10:19:37.779119+0000","last_active":"2024-04-25T10:19:40.125286+0000","last_peered":"2024-04-25T10:19:40.125286+0000","last_clean":"2024-04-25T10:19:40.125286+0000","last_became_active":"2024-04-25T10:19:37.778834+0000","last_became_peered":"2024-04-25T10:19:37.778834+0000","last_unstale":"2024-04-25T10:19:40.125286+0000","last_undegraded":"2024-04-25T10:19:40.125286+0000","last_fullsized":"2024-04-25T10:19:40.125286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125229+0000","last_change":"2024-04-25T10:19:37.770142+0000","last_active":"2024-04-25T10:19:40.125229+0000","last_peered":"2024-04-25T10:19:40.125229+0000","last_clean":"2024-04-25T10:19:40.125229+0000","last_became_active":"2024-04-25T10:19:37.769871+0000","last_became_peered":"2024-04-25T10:19:37.769871+0000","last_unstale":"2024-04-25T10:19:40.125229+0000","last_undegraded":"2024-04-25T10:19:40.125229+0000","last_fullsized":"2024-04-25T10:19:40.125229+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:06.128960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768117+0000","last_change":"2024-04-25T10:19:37.769809+0000","last_active":"2024-04-25T10:19:38.768117+0000","last_peered":"2024-04-25T10:19:38.768117+0000","last_clean":"2024-04-25T10:19:38.768117+0000","last_became_active":"2024-04-25T10:19:37.768260+0000","last_became_peered":"2024-04-25T10:19:37.768260+0000","last_unstale":"2024-04-25T10:19:38.768117+0000","last_undegraded":"2024-04-25T10:19:38.768117+0000","last_fullsized":"2024-04-25T10:19:38.768117+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.629408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957969+0000","last_change":"2024-04-25T10:19:37.780096+0000","last_active":"2024-04-25T10:19:38.957969+0000","last_peered":"2024-04-25T10:19:38.957969+0000","last_clean":"2024-04-25T10:19:38.957969+0000","last_became_active":"2024-04-25T10:19:37.769941+0000","last_became_peered":"2024-04-25T10:19:37.769941+0000","last_unstale":"2024-04-25T10:19:38.957969+0000","last_undegraded":"2024-04-25T10:19:38.957969+0000","last_fullsized":"2024-04-25T10:19:38.957969+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764342+0000","last_change":"2024-04-25T10:19:37.775131+0000","last_active":"2024-04-25T10:19:38.764342+0000","last_peered":"2024-04-25T10:19:38.764342+0000","last_clean":"2024-04-25T10:19:38.764342+0000","last_became_active":"2024-04-25T10:19:37.774866+0000","last_became_peered":"2024-04-25T10:19:37.774866+0000","last_unstale":"2024-04-25T10:19:38.764342+0000","last_undegraded":"2024-04-25T10:19:38.764342+0000","last_fullsized":"2024-04-25T10:19:38.764342+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:19.879710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957910+0000","last_change":"2024-04-25T10:19:37.774477+0000","last_active":"2024-04-25T10:19:38.957910+0000","last_peered":"2024-04-25T10:19:38.957910+0000","last_clean":"2024-04-25T10:19:38.957910+0000","last_became_active":"2024-04-25T10:19:37.773941+0000","last_became_peered":"2024-04-25T10:19:37.773941+0000","last_unstale":"2024-04-25T10:19:38.957910+0000","last_undegraded":"2024-04-25T10:19:38.957910+0000","last_fullsized":"2024-04-25T10:19:38.957910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125346+0000","last_change":"2024-04-25T10:19:37.781811+0000","last_active":"2024-04-25T10:19:40.125346+0000","last_peered":"2024-04-25T10:19:40.125346+0000","last_clean":"2024-04-25T10:19:40.125346+0000","last_became_active":"2024-04-25T10:19:37.781277+0000","last_became_peered":"2024-04-25T10:19:37.781277+0000","last_unstale":"2024-04-25T10:19:40.125346+0000","last_undegraded":"2024-04-25T10:19:40.125346+0000","last_fullsized":"2024-04-25T10:19:40.125346+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:31.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769165+0000","last_change":"2024-04-25T10:19:37.774791+0000","last_active":"2024-04-25T10:19:38.769165+0000","last_peered":"2024-04-25T10:19:38.769165+0000","last_clean":"2024-04-25T10:19:38.769165+0000","last_became_active":"2024-04-25T10:19:37.774510+0000","last_became_peered":"2024-04-25T10:19:37.774510+0000","last_unstale":"2024-04-25T10:19:38.769165+0000","last_undegraded":"2024-04-25T10:19:38.769165+0000","last_fullsized":"2024-04-25T10:19:38.769165+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.807600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958231+0000","last_change":"2024-04-25T10:19:37.788665+0000","last_active":"2024-04-25T10:19:38.958231+0000","last_peered":"2024-04-25T10:19:38.958231+0000","last_clean":"2024-04-25T10:19:38.958231+0000","last_became_active":"2024-04-25T10:19:37.788349+0000","last_became_peered":"2024-04-25T10:19:37.788349+0000","last_unstale":"2024-04-25T10:19:38.958231+0000","last_undegraded":"2024-04-25T10:19:38.958231+0000","last_fullsized":"2024-04-25T10:19:38.958231+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.849204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764638+0000","last_change":"2024-04-25T10:19:37.772490+0000","last_active":"2024-04-25T10:19:38.764638+0000","last_peered":"2024-04-25T10:19:38.764638+0000","last_clean":"2024-04-25T10:19:38.764638+0000","last_became_active":"2024-04-25T10:19:37.772163+0000","last_became_peered":"2024-04-25T10:19:37.772163+0000","last_unstale":"2024-04-25T10:19:38.764638+0000","last_undegraded":"2024-04-25T10:19:38.764638+0000","last_fullsized":"2024-04-25T10:19:38.764638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:24.445381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768296+0000","last_change":"2024-04-25T10:19:37.760459+0000","last_active":"2024-04-25T10:19:38.768296+0000","last_peered":"2024-04-25T10:19:38.768296+0000","last_clean":"2024-04-25T10:19:38.768296+0000","last_became_active":"2024-04-25T10:19:37.760170+0000","last_became_peered":"2024-04-25T10:19:37.760170+0000","last_unstale":"2024-04-25T10:19:38.768296+0000","last_undegraded":"2024-04-25T10:19:38.768296+0000","last_fullsized":"2024-04-25T10:19:38.768296+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:48.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764490+0000","last_change":"2024-04-25T10:19:37.784933+0000","last_active":"2024-04-25T10:19:38.764490+0000","last_peered":"2024-04-25T10:19:38.764490+0000","last_clean":"2024-04-25T10:19:38.764490+0000","last_became_active":"2024-04-25T10:19:37.784659+0000","last_became_peered":"2024-04-25T10:19:37.784659+0000","last_unstale":"2024-04-25T10:19:38.764490+0000","last_undegraded":"2024-04-25T10:19:38.764490+0000","last_fullsized":"2024-04-25T10:19:38.764490+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:15.594908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768870+0000","last_change":"2024-04-25T10:19:37.770191+0000","last_active":"2024-04-25T10:19:38.768870+0000","last_peered":"2024-04-25T10:19:38.768870+0000","last_clean":"2024-04-25T10:19:38.768870+0000","last_became_active":"2024-04-25T10:19:37.769851+0000","last_became_peered":"2024-04-25T10:19:37.769851+0000","last_unstale":"2024-04-25T10:19:38.768870+0000","last_undegraded":"2024-04-25T10:19:38.768870+0000","last_fullsized":"2024-04-25T10:19:38.768870+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:31.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768715+0000","last_change":"2024-04-25T10:19:37.769040+0000","last_active":"2024-04-25T10:19:38.768715+0000","last_peered":"2024-04-25T10:19:38.768715+0000","last_clean":"2024-04-25T10:19:38.768715+0000","last_became_active":"2024-04-25T10:19:37.768733+0000","last_became_peered":"2024-04-25T10:19:37.768733+0000","last_unstale":"2024-04-25T10:19:38.768715+0000","last_undegraded":"2024-04-25T10:19:38.768715+0000","last_fullsized":"2024-04-25T10:19:38.768715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125143+0000","last_change":"2024-04-25T10:19:37.782354+0000","last_active":"2024-04-25T10:19:40.125143+0000","last_peered":"2024-04-25T10:19:40.125143+0000","last_clean":"2024-04-25T10:19:40.125143+0000","last_became_active":"2024-04-25T10:19:37.782102+0000","last_became_peered":"2024-04-25T10:19:37.782102+0000","last_unstale":"2024-04-25T10:19:40.125143+0000","last_undegraded":"2024-04-25T10:19:40.125143+0000","last_fullsized":"2024-04-25T10:19:40.125143+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125707+0000","last_change":"2024-04-25T10:19:37.781507+0000","last_active":"2024-04-25T10:19:40.125707+0000","last_peered":"2024-04-25T10:19:40.125707+0000","last_clean":"2024-04-25T10:19:40.125707+0000","last_became_active":"2024-04-25T10:19:37.780601+0000","last_became_peered":"2024-04-25T10:19:37.780601+0000","last_unstale":"2024-04-25T10:19:40.125707+0000","last_undegraded":"2024-04-25T10:19:40.125707+0000","last_fullsized":"2024-04-25T10:19:40.125707+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769057+0000","last_change":"2024-04-25T10:19:37.771467+0000","last_active":"2024-04-25T10:19:38.769057+0000","last_peered":"2024-04-25T10:19:38.769057+0000","last_clean":"2024-04-25T10:19:38.769057+0000","last_became_active":"2024-04-25T10:19:37.769290+0000","last_became_peered":"2024-04-25T10:19:37.769290+0000","last_unstale":"2024-04-25T10:19:38.769057+0000","last_undegraded":"2024-04-25T10:19:38.769057+0000","last_fullsized":"2024-04-25T10:19:38.769057+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.914760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125390+0000","last_change":"2024-04-25T10:19:37.788074+0000","last_active":"2024-04-25T10:19:40.125390+0000","last_peered":"2024-04-25T10:19:40.125390+0000","last_clean":"2024-04-25T10:19:40.125390+0000","last_became_active":"2024-04-25T10:19:37.787748+0000","last_became_peered":"2024-04-25T10:19:37.787748+0000","last_unstale":"2024-04-25T10:19:40.125390+0000","last_undegraded":"2024-04-25T10:19:40.125390+0000","last_fullsized":"2024-04-25T10:19:40.125390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.849204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125053+0000","last_change":"2024-04-25T10:19:37.775285+0000","last_active":"2024-04-25T10:19:40.125053+0000","last_peered":"2024-04-25T10:19:40.125053+0000","last_clean":"2024-04-25T10:19:40.125053+0000","last_became_active":"2024-04-25T10:19:37.774991+0000","last_became_peered":"2024-04-25T10:19:37.774991+0000","last_unstale":"2024-04-25T10:19:40.125053+0000","last_undegraded":"2024-04-25T10:19:40.125053+0000","last_fullsized":"2024-04-25T10:19:40.125053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.629408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768803+0000","last_change":"2024-04-25T10:19:37.780736+0000","last_active":"2024-04-25T10:19:38.768803+0000","last_peered":"2024-04-25T10:19:38.768803+0000","last_clean":"2024-04-25T10:19:38.768803+0000","last_became_active":"2024-04-25T10:19:37.780473+0000","last_became_peered":"2024-04-25T10:19:37.780473+0000","last_unstale":"2024-04-25T10:19:38.768803+0000","last_undegraded":"2024-04-25T10:19:38.768803+0000","last_fullsized":"2024-04-25T10:19:38.768803+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:06.128960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764315+0000","last_change":"2024-04-25T10:19:37.775511+0000","last_active":"2024-04-25T10:19:38.764315+0000","last_peered":"2024-04-25T10:19:38.764315+0000","last_clean":"2024-04-25T10:19:38.764315+0000","last_became_active":"2024-04-25T10:19:37.768009+0000","last_became_peered":"2024-04-25T10:19:37.768009+0000","last_unstale":"2024-04-25T10:19:38.764315+0000","last_undegraded":"2024-04-25T10:19:38.764315+0000","last_fullsized":"2024-04-25T10:19:38.764315+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:11.100611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957958+0000","last_change":"2024-04-25T10:19:37.774229+0000","last_active":"2024-04-25T10:19:38.957958+0000","last_peered":"2024-04-25T10:19:38.957958+0000","last_clean":"2024-04-25T10:19:38.957958+0000","last_became_active":"2024-04-25T10:19:37.773529+0000","last_became_peered":"2024-04-25T10:19:37.773529+0000","last_unstale":"2024-04-25T10:19:38.957958+0000","last_undegraded":"2024-04-25T10:19:38.957958+0000","last_fullsized":"2024-04-25T10:19:38.957958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:48.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839433+0000","last_change":"2024-04-25T10:19:37.768567+0000","last_active":"2024-04-25T10:19:38.839433+0000","last_peered":"2024-04-25T10:19:38.839433+0000","last_clean":"2024-04-25T10:19:38.839433+0000","last_became_active":"2024-04-25T10:19:37.768294+0000","last_became_peered":"2024-04-25T10:19:37.768294+0000","last_unstale":"2024-04-25T10:19:38.839433+0000","last_undegraded":"2024-04-25T10:19:38.839433+0000","last_fullsized":"2024-04-25T10:19:38.839433+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:06.640576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764738+0000","last_change":"2024-04-25T10:19:37.788308+0000","last_active":"2024-04-25T10:19:38.764738+0000","last_peered":"2024-04-25T10:19:38.764738+0000","last_clean":"2024-04-25T10:19:38.764738+0000","last_became_active":"2024-04-25T10:19:37.786260+0000","last_became_peered":"2024-04-25T10:19:37.786260+0000","last_unstale":"2024-04-25T10:19:38.764738+0000","last_undegraded":"2024-04-25T10:19:38.764738+0000","last_fullsized":"2024-04-25T10:19:38.764738+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:00.816107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769120+0000","last_change":"2024-04-25T10:19:36.754304+0000","last_active":"2024-04-25T10:19:38.769120+0000","last_peered":"2024-04-25T10:19:38.769120+0000","last_clean":"2024-04-25T10:19:38.769120+0000","last_became_active":"2024-04-25T10:19:36.754033+0000","last_became_peered":"2024-04-25T10:19:36.754033+0000","last_unstale":"2024-04-25T10:19:38.769120+0000","last_undegraded":"2024-04-25T10:19:38.769120+0000","last_fullsized":"2024-04-25T10:19:38.769120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:16.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958344+0000","last_change":"2024-04-25T10:19:37.772229+0000","last_active":"2024-04-25T10:19:38.958344+0000","last_peered":"2024-04-25T10:19:38.958344+0000","last_clean":"2024-04-25T10:19:38.958344+0000","last_became_active":"2024-04-25T10:19:37.771908+0000","last_became_peered":"2024-04-25T10:19:37.771908+0000","last_unstale":"2024-04-25T10:19:38.958344+0000","last_undegraded":"2024-04-25T10:19:38.958344+0000","last_fullsized":"2024-04-25T10:19:38.958344+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.914760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125423+0000","last_change":"2024-04-25T10:19:36.757825+0000","last_active":"2024-04-25T10:19:40.125423+0000","last_peered":"2024-04-25T10:19:40.125423+0000","last_clean":"2024-04-25T10:19:40.125423+0000","last_became_active":"2024-04-25T10:19:36.757506+0000","last_became_peered":"2024-04-25T10:19:36.757506+0000","last_unstale":"2024-04-25T10:19:40.125423+0000","last_undegraded":"2024-04-25T10:19:40.125423+0000","last_fullsized":"2024-04-25T10:19:40.125423+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:10.876366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839412+0000","last_change":"2024-04-25T10:19:37.777602+0000","last_active":"2024-04-25T10:19:38.839412+0000","last_peered":"2024-04-25T10:19:38.839412+0000","last_clean":"2024-04-25T10:19:38.839412+0000","last_became_active":"2024-04-25T10:19:37.769547+0000","last_became_peered":"2024-04-25T10:19:37.769547+0000","last_unstale":"2024-04-25T10:19:38.839412+0000","last_undegraded":"2024-04-25T10:19:38.839412+0000","last_fullsized":"2024-04-25T10:19:38.839412+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:48.146382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768982+0000","last_change":"2024-04-25T10:19:36.761223+0000","last_active":"2024-04-25T10:19:38.768982+0000","last_peered":"2024-04-25T10:19:38.768982+0000","last_clean":"2024-04-25T10:19:38.768982+0000","last_became_active":"2024-04-25T10:19:36.761008+0000","last_became_peered":"2024-04-25T10:19:36.761008+0000","last_unstale":"2024-04-25T10:19:38.768982+0000","last_undegraded":"2024-04-25T10:19:38.768982+0000","last_fullsized":"2024-04-25T10:19:38.768982+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:11.139422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768063+0000","last_change":"2024-04-25T10:19:37.762533+0000","last_active":"2024-04-25T10:19:38.768063+0000","last_peered":"2024-04-25T10:19:38.768063+0000","last_clean":"2024-04-25T10:19:38.768063+0000","last_became_active":"2024-04-25T10:19:37.762211+0000","last_became_peered":"2024-04-25T10:19:37.762211+0000","last_unstale":"2024-04-25T10:19:38.768063+0000","last_undegraded":"2024-04-25T10:19:38.768063+0000","last_fullsized":"2024-04-25T10:19:38.768063+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:14.153645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768824+0000","last_change":"2024-04-25T10:19:36.755923+0000","last_active":"2024-04-25T10:19:38.768824+0000","last_peered":"2024-04-25T10:19:38.768824+0000","last_clean":"2024-04-25T10:19:38.768824+0000","last_became_active":"2024-04-25T10:19:36.755237+0000","last_became_peered":"2024-04-25T10:19:36.755237+0000","last_unstale":"2024-04-25T10:19:38.768824+0000","last_undegraded":"2024-04-25T10:19:38.768824+0000","last_fullsized":"2024-04-25T10:19:38.768824+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:23.420555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767851+0000","last_change":"2024-04-25T10:19:37.763156+0000","last_active":"2024-04-25T10:19:38.767851+0000","last_peered":"2024-04-25T10:19:38.767851+0000","last_clean":"2024-04-25T10:19:38.767851+0000","last_became_active":"2024-04-25T10:19:37.762875+0000","last_became_peered":"2024-04-25T10:19:37.762875+0000","last_unstale":"2024-04-25T10:19:38.767851+0000","last_undegraded":"2024-04-25T10:19:38.767851+0000","last_fullsized":"2024-04-25T10:19:38.767851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:13.864917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839308+0000","last_change":"2024-04-25T10:19:36.752703+0000","last_active":"2024-04-25T10:19:38.839308+0000","last_peered":"2024-04-25T10:19:38.839308+0000","last_clean":"2024-04-25T10:19:38.839308+0000","last_became_active":"2024-04-25T10:19:36.751221+0000","last_became_peered":"2024-04-25T10:19:36.751221+0000","last_unstale":"2024-04-25T10:19:38.839308+0000","last_undegraded":"2024-04-25T10:19:38.839308+0000","last_fullsized":"2024-04-25T10:19:38.839308+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:14.588871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125027+0000","last_change":"2024-04-25T10:19:37.770422+0000","last_active":"2024-04-25T10:19:40.125027+0000","last_peered":"2024-04-25T10:19:40.125027+0000","last_clean":"2024-04-25T10:19:40.125027+0000","last_became_active":"2024-04-25T10:19:37.769530+0000","last_became_peered":"2024-04-25T10:19:37.769530+0000","last_unstale":"2024-04-25T10:19:40.125027+0000","last_undegraded":"2024-04-25T10:19:40.125027+0000","last_fullsized":"2024-04-25T10:19:40.125027+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768626+0000","last_change":"2024-04-25T10:19:36.758029+0000","last_active":"2024-04-25T10:19:38.768626+0000","last_peered":"2024-04-25T10:19:38.768626+0000","last_clean":"2024-04-25T10:19:38.768626+0000","last_became_active":"2024-04-25T10:19:36.757771+0000","last_became_peered":"2024-04-25T10:19:36.757771+0000","last_unstale":"2024-04-25T10:19:38.768626+0000","last_undegraded":"2024-04-25T10:19:38.768626+0000","last_fullsized":"2024-04-25T10:19:38.768626+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:27.647159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957988+0000","last_change":"2024-04-25T10:19:37.786017+0000","last_active":"2024-04-25T10:19:38.957988+0000","last_peered":"2024-04-25T10:19:38.957988+0000","last_clean":"2024-04-25T10:19:38.957988+0000","last_became_active":"2024-04-25T10:19:37.785742+0000","last_became_peered":"2024-04-25T10:19:37.785742+0000","last_unstale":"2024-04-25T10:19:38.957988+0000","last_undegraded":"2024-04-25T10:19:38.957988+0000","last_fullsized":"2024-04-25T10:19:38.957988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124876+0000","last_change":"2024-04-25T10:19:36.759897+0000","last_active":"2024-04-25T10:19:40.124876+0000","last_peered":"2024-04-25T10:19:40.124876+0000","last_clean":"2024-04-25T10:19:40.124876+0000","last_became_active":"2024-04-25T10:19:36.759589+0000","last_became_peered":"2024-04-25T10:19:36.759589+0000","last_unstale":"2024-04-25T10:19:40.124876+0000","last_undegraded":"2024-04-25T10:19:40.124876+0000","last_fullsized":"2024-04-25T10:19:40.124876+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:23.420555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767347+0000","last_change":"2024-04-25T10:19:37.758007+0000","last_active":"2024-04-25T10:19:38.767347+0000","last_peered":"2024-04-25T10:19:38.767347+0000","last_clean":"2024-04-25T10:19:38.767347+0000","last_became_active":"2024-04-25T10:19:37.757720+0000","last_became_peered":"2024-04-25T10:19:37.757720+0000","last_unstale":"2024-04-25T10:19:38.767347+0000","last_undegraded":"2024-04-25T10:19:38.767347+0000","last_fullsized":"2024-04-25T10:19:38.767347+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:15.750695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839301+0000","last_change":"2024-04-25T10:19:36.753819+0000","last_active":"2024-04-25T10:19:38.839301+0000","last_peered":"2024-04-25T10:19:38.839301+0000","last_clean":"2024-04-25T10:19:38.839301+0000","last_became_active":"2024-04-25T10:19:36.753582+0000","last_became_peered":"2024-04-25T10:19:36.753582+0000","last_unstale":"2024-04-25T10:19:38.839301+0000","last_undegraded":"2024-04-25T10:19:38.839301+0000","last_fullsized":"2024-04-25T10:19:38.839301+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:44.204686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769145+0000","last_change":"2024-04-25T10:19:37.772310+0000","last_active":"2024-04-25T10:19:38.769145+0000","last_peered":"2024-04-25T10:19:38.769145+0000","last_clean":"2024-04-25T10:19:38.769145+0000","last_became_active":"2024-04-25T10:19:37.771136+0000","last_became_peered":"2024-04-25T10:19:37.771136+0000","last_unstale":"2024-04-25T10:19:38.769145+0000","last_undegraded":"2024-04-25T10:19:38.769145+0000","last_fullsized":"2024-04-25T10:19:38.769145+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.645465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839778+0000","last_change":"2024-04-25T10:19:36.747385+0000","last_active":"2024-04-25T10:19:38.839778+0000","last_peered":"2024-04-25T10:19:38.839778+0000","last_clean":"2024-04-25T10:19:38.839778+0000","last_became_active":"2024-04-25T10:19:36.746924+0000","last_became_peered":"2024-04-25T10:19:36.746924+0000","last_unstale":"2024-04-25T10:19:38.839778+0000","last_undegraded":"2024-04-25T10:19:38.839778+0000","last_fullsized":"2024-04-25T10:19:38.839778+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:18.873673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768271+0000","last_change":"2024-04-25T10:19:37.762256+0000","last_active":"2024-04-25T10:19:38.768271+0000","last_peered":"2024-04-25T10:19:38.768271+0000","last_clean":"2024-04-25T10:19:38.768271+0000","last_became_active":"2024-04-25T10:19:37.761921+0000","last_became_peered":"2024-04-25T10:19:37.761921+0000","last_unstale":"2024-04-25T10:19:38.768271+0000","last_undegraded":"2024-04-25T10:19:38.768271+0000","last_fullsized":"2024-04-25T10:19:38.768271+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958180+0000","last_change":"2024-04-25T10:19:36.759744+0000","last_active":"2024-04-25T10:19:38.958180+0000","last_peered":"2024-04-25T10:19:38.958180+0000","last_clean":"2024-04-25T10:19:38.958180+0000","last_became_active":"2024-04-25T10:19:36.759477+0000","last_became_peered":"2024-04-25T10:19:36.759477+0000","last_unstale":"2024-04-25T10:19:38.958180+0000","last_undegraded":"2024-04-25T10:19:38.958180+0000","last_fullsized":"2024-04-25T10:19:38.958180+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.473366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125088+0000","last_change":"2024-04-25T10:19:37.777332+0000","last_active":"2024-04-25T10:19:40.125088+0000","last_peered":"2024-04-25T10:19:40.125088+0000","last_clean":"2024-04-25T10:19:40.125088+0000","last_became_active":"2024-04-25T10:19:37.777052+0000","last_became_peered":"2024-04-25T10:19:37.777052+0000","last_unstale":"2024-04-25T10:19:40.125088+0000","last_undegraded":"2024-04-25T10:19:40.125088+0000","last_fullsized":"2024-04-25T10:19:40.125088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:35.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125472+0000","last_change":"2024-04-25T10:19:36.754486+0000","last_active":"2024-04-25T10:19:40.125472+0000","last_peered":"2024-04-25T10:19:40.125472+0000","last_clean":"2024-04-25T10:19:40.125472+0000","last_became_active":"2024-04-25T10:19:36.754183+0000","last_became_peered":"2024-04-25T10:19:36.754183+0000","last_unstale":"2024-04-25T10:19:40.125472+0000","last_undegraded":"2024-04-25T10:19:40.125472+0000","last_fullsized":"2024-04-25T10:19:40.125472+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.473366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125416+0000","last_change":"2024-04-25T10:19:37.782113+0000","last_active":"2024-04-25T10:19:40.125416+0000","last_peered":"2024-04-25T10:19:40.125416+0000","last_clean":"2024-04-25T10:19:40.125416+0000","last_became_active":"2024-04-25T10:19:37.781847+0000","last_became_peered":"2024-04-25T10:19:37.781847+0000","last_unstale":"2024-04-25T10:19:40.125416+0000","last_undegraded":"2024-04-25T10:19:40.125416+0000","last_fullsized":"2024-04-25T10:19:40.125416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.914760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768199+0000","last_change":"2024-04-25T10:19:36.738569+0000","last_active":"2024-04-25T10:19:38.768199+0000","last_peered":"2024-04-25T10:19:38.768199+0000","last_clean":"2024-04-25T10:19:38.768199+0000","last_became_active":"2024-04-25T10:19:36.738280+0000","last_became_peered":"2024-04-25T10:19:36.738280+0000","last_unstale":"2024-04-25T10:19:38.768199+0000","last_undegraded":"2024-04-25T10:19:38.768199+0000","last_fullsized":"2024-04-25T10:19:38.768199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:55.564589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958101+0000","last_change":"2024-04-25T10:19:37.763128+0000","last_active":"2024-04-25T10:19:38.958101+0000","last_peered":"2024-04-25T10:19:38.958101+0000","last_clean":"2024-04-25T10:19:38.958101+0000","last_became_active":"2024-04-25T10:19:37.762821+0000","last_became_peered":"2024-04-25T10:19:37.762821+0000","last_unstale":"2024-04-25T10:19:38.958101+0000","last_undegraded":"2024-04-25T10:19:38.958101+0000","last_fullsized":"2024-04-25T10:19:38.958101+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:06.128960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764543+0000","last_change":"2024-04-25T10:19:36.753686+0000","last_active":"2024-04-25T10:19:38.764543+0000","last_peered":"2024-04-25T10:19:38.764543+0000","last_clean":"2024-04-25T10:19:38.764543+0000","last_became_active":"2024-04-25T10:19:36.753151+0000","last_became_peered":"2024-04-25T10:19:36.753151+0000","last_unstale":"2024-04-25T10:19:38.764543+0000","last_undegraded":"2024-04-25T10:19:38.764543+0000","last_fullsized":"2024-04-25T10:19:38.764543+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:46.990158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839629+0000","last_change":"2024-04-25T10:19:37.788705+0000","last_active":"2024-04-25T10:19:38.839629+0000","last_peered":"2024-04-25T10:19:38.839629+0000","last_clean":"2024-04-25T10:19:38.839629+0000","last_became_active":"2024-04-25T10:19:37.788149+0000","last_became_peered":"2024-04-25T10:19:37.788149+0000","last_unstale":"2024-04-25T10:19:38.839629+0000","last_undegraded":"2024-04-25T10:19:38.839629+0000","last_fullsized":"2024-04-25T10:19:38.839629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:42.482477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764560+0000","last_change":"2024-04-25T10:19:36.755955+0000","last_active":"2024-04-25T10:19:38.764560+0000","last_peered":"2024-04-25T10:19:38.764560+0000","last_clean":"2024-04-25T10:19:38.764560+0000","last_became_active":"2024-04-25T10:19:36.755468+0000","last_became_peered":"2024-04-25T10:19:36.755468+0000","last_unstale":"2024-04-25T10:19:38.764560+0000","last_undegraded":"2024-04-25T10:19:38.764560+0000","last_fullsized":"2024-04-25T10:19:38.764560+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:52.793832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767003+0000","last_change":"2024-04-25T10:19:37.761881+0000","last_active":"2024-04-25T10:19:38.767003+0000","last_peered":"2024-04-25T10:19:38.767003+0000","last_clean":"2024-04-25T10:19:38.767003+0000","last_became_active":"2024-04-25T10:19:37.761503+0000","last_became_peered":"2024-04-25T10:19:37.761503+0000","last_unstale":"2024-04-25T10:19:38.767003+0000","last_undegraded":"2024-04-25T10:19:38.767003+0000","last_fullsized":"2024-04-25T10:19:38.767003+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:11.113877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767070+0000","last_change":"2024-04-25T10:19:36.745660+0000","last_active":"2024-04-25T10:19:38.767070+0000","last_peered":"2024-04-25T10:19:38.767070+0000","last_clean":"2024-04-25T10:19:38.767070+0000","last_became_active":"2024-04-25T10:19:36.745382+0000","last_became_peered":"2024-04-25T10:19:36.745382+0000","last_unstale":"2024-04-25T10:19:38.767070+0000","last_undegraded":"2024-04-25T10:19:38.767070+0000","last_fullsized":"2024-04-25T10:19:38.767070+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:23.420555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764127+0000","last_change":"2024-04-25T10:19:37.770742+0000","last_active":"2024-04-25T10:19:38.764127+0000","last_peered":"2024-04-25T10:19:38.764127+0000","last_clean":"2024-04-25T10:19:38.764127+0000","last_became_active":"2024-04-25T10:19:37.770255+0000","last_became_peered":"2024-04-25T10:19:37.770255+0000","last_unstale":"2024-04-25T10:19:38.764127+0000","last_undegraded":"2024-04-25T10:19:38.764127+0000","last_fullsized":"2024-04-25T10:19:38.764127+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:45.210723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839419+0000","last_change":"2024-04-25T10:19:36.748763+0000","last_active":"2024-04-25T10:19:38.839419+0000","last_peered":"2024-04-25T10:19:38.839419+0000","last_clean":"2024-04-25T10:19:38.839419+0000","last_became_active":"2024-04-25T10:19:36.748478+0000","last_became_peered":"2024-04-25T10:19:36.748478+0000","last_unstale":"2024-04-25T10:19:38.839419+0000","last_undegraded":"2024-04-25T10:19:38.839419+0000","last_fullsized":"2024-04-25T10:19:38.839419+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:10.094574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125753+0000","last_change":"2024-04-25T10:19:37.773281+0000","last_active":"2024-04-25T10:19:40.125753+0000","last_peered":"2024-04-25T10:19:40.125753+0000","last_clean":"2024-04-25T10:19:40.125753+0000","last_became_active":"2024-04-25T10:19:37.772880+0000","last_became_peered":"2024-04-25T10:19:37.772880+0000","last_unstale":"2024-04-25T10:19:40.125753+0000","last_undegraded":"2024-04-25T10:19:40.125753+0000","last_fullsized":"2024-04-25T10:19:40.125753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767884+0000","last_change":"2024-04-25T10:19:36.755567+0000","last_active":"2024-04-25T10:19:38.767884+0000","last_peered":"2024-04-25T10:19:38.767884+0000","last_clean":"2024-04-25T10:19:38.767884+0000","last_became_active":"2024-04-25T10:19:36.755294+0000","last_became_peered":"2024-04-25T10:19:36.755294+0000","last_unstale":"2024-04-25T10:19:38.767884+0000","last_undegraded":"2024-04-25T10:19:38.767884+0000","last_fullsized":"2024-04-25T10:19:38.767884+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:20.728540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767755+0000","last_change":"2024-04-25T10:19:37.767526+0000","last_active":"2024-04-25T10:19:38.767755+0000","last_peered":"2024-04-25T10:19:38.767755+0000","last_clean":"2024-04-25T10:19:38.767755+0000","last_became_active":"2024-04-25T10:19:37.767229+0000","last_became_peered":"2024-04-25T10:19:37.767229+0000","last_unstale":"2024-04-25T10:19:38.767755+0000","last_undegraded":"2024-04-25T10:19:38.767755+0000","last_fullsized":"2024-04-25T10:19:38.767755+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:03.838106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125564+0000","last_change":"2024-04-25T10:19:36.748121+0000","last_active":"2024-04-25T10:19:40.125564+0000","last_peered":"2024-04-25T10:19:40.125564+0000","last_clean":"2024-04-25T10:19:40.125564+0000","last_became_active":"2024-04-25T10:19:36.747810+0000","last_became_peered":"2024-04-25T10:19:36.747810+0000","last_unstale":"2024-04-25T10:19:40.125564+0000","last_undegraded":"2024-04-25T10:19:40.125564+0000","last_fullsized":"2024-04-25T10:19:40.125564+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:30.325508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125571+0000","last_change":"2024-04-25T10:19:37.773693+0000","last_active":"2024-04-25T10:19:40.125571+0000","last_peered":"2024-04-25T10:19:40.125571+0000","last_clean":"2024-04-25T10:19:40.125571+0000","last_became_active":"2024-04-25T10:19:37.772443+0000","last_became_peered":"2024-04-25T10:19:37.772443+0000","last_unstale":"2024-04-25T10:19:40.125571+0000","last_undegraded":"2024-04-25T10:19:40.125571+0000","last_fullsized":"2024-04-25T10:19:40.125571+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.807600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768902+0000","last_change":"2024-04-25T10:19:36.755521+0000","last_active":"2024-04-25T10:19:38.768902+0000","last_peered":"2024-04-25T10:19:38.768902+0000","last_clean":"2024-04-25T10:19:38.768902+0000","last_became_active":"2024-04-25T10:19:36.755228+0000","last_became_peered":"2024-04-25T10:19:36.755228+0000","last_unstale":"2024-04-25T10:19:38.768902+0000","last_undegraded":"2024-04-25T10:19:38.768902+0000","last_fullsized":"2024-04-25T10:19:38.768902+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:55.564589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764586+0000","last_change":"2024-04-25T10:19:37.768378+0000","last_active":"2024-04-25T10:19:38.764586+0000","last_peered":"2024-04-25T10:19:38.764586+0000","last_clean":"2024-04-25T10:19:38.764586+0000","last_became_active":"2024-04-25T10:19:37.767376+0000","last_became_peered":"2024-04-25T10:19:37.767376+0000","last_unstale":"2024-04-25T10:19:38.764586+0000","last_undegraded":"2024-04-25T10:19:38.764586+0000","last_fullsized":"2024-04-25T10:19:38.764586+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:29.459034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768741+0000","last_change":"2024-04-25T10:19:36.755373+0000","last_active":"2024-04-25T10:19:38.768741+0000","last_peered":"2024-04-25T10:19:38.768741+0000","last_clean":"2024-04-25T10:19:38.768741+0000","last_became_active":"2024-04-25T10:19:36.754566+0000","last_became_peered":"2024-04-25T10:19:36.754566+0000","last_unstale":"2024-04-25T10:19:38.768741+0000","last_undegraded":"2024-04-25T10:19:38.768741+0000","last_fullsized":"2024-04-25T10:19:38.768741+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:19.654181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764619+0000","last_change":"2024-04-25T10:19:37.779539+0000","last_active":"2024-04-25T10:19:38.764619+0000","last_peered":"2024-04-25T10:19:38.764619+0000","last_clean":"2024-04-25T10:19:38.764619+0000","last_became_active":"2024-04-25T10:19:37.779252+0000","last_became_peered":"2024-04-25T10:19:37.779252+0000","last_unstale":"2024-04-25T10:19:38.764619+0000","last_undegraded":"2024-04-25T10:19:38.764619+0000","last_fullsized":"2024-04-25T10:19:38.764619+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:44.596174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767904+0000","last_change":"2024-04-25T10:19:36.743832+0000","last_active":"2024-04-25T10:19:38.767904+0000","last_peered":"2024-04-25T10:19:38.767904+0000","last_clean":"2024-04-25T10:19:38.767904+0000","last_became_active":"2024-04-25T10:19:36.743555+0000","last_became_peered":"2024-04-25T10:19:36.743555+0000","last_unstale":"2024-04-25T10:19:38.767904+0000","last_undegraded":"2024-04-25T10:19:38.767904+0000","last_fullsized":"2024-04-25T10:19:38.767904+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:19.678873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764078+0000","last_change":"2024-04-25T10:19:37.776658+0000","last_active":"2024-04-25T10:19:38.764078+0000","last_peered":"2024-04-25T10:19:38.764078+0000","last_clean":"2024-04-25T10:19:38.764078+0000","last_became_active":"2024-04-25T10:19:37.768021+0000","last_became_peered":"2024-04-25T10:19:37.768021+0000","last_unstale":"2024-04-25T10:19:38.764078+0000","last_undegraded":"2024-04-25T10:19:38.764078+0000","last_fullsized":"2024-04-25T10:19:38.764078+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:21.259266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767509+0000","last_change":"2024-04-25T10:19:36.738856+0000","last_active":"2024-04-25T10:19:38.767509+0000","last_peered":"2024-04-25T10:19:38.767509+0000","last_clean":"2024-04-25T10:19:38.767509+0000","last_became_active":"2024-04-25T10:19:36.738521+0000","last_became_peered":"2024-04-25T10:19:36.738521+0000","last_unstale":"2024-04-25T10:19:38.767509+0000","last_undegraded":"2024-04-25T10:19:38.767509+0000","last_fullsized":"2024-04-25T10:19:38.767509+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:25.506297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124724+0000","last_change":"2024-04-25T10:19:37.779235+0000","last_active":"2024-04-25T10:19:40.124724+0000","last_peered":"2024-04-25T10:19:40.124724+0000","last_clean":"2024-04-25T10:19:40.124724+0000","last_became_active":"2024-04-25T10:19:37.778972+0000","last_became_peered":"2024-04-25T10:19:37.778972+0000","last_unstale":"2024-04-25T10:19:40.124724+0000","last_undegraded":"2024-04-25T10:19:40.124724+0000","last_fullsized":"2024-04-25T10:19:40.124724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:15.750695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768655+0000","last_change":"2024-04-25T10:19:36.753235+0000","last_active":"2024-04-25T10:19:38.768655+0000","last_peered":"2024-04-25T10:19:38.768655+0000","last_clean":"2024-04-25T10:19:38.768655+0000","last_became_active":"2024-04-25T10:19:36.752968+0000","last_became_peered":"2024-04-25T10:19:36.752968+0000","last_unstale":"2024-04-25T10:19:38.768655+0000","last_undegraded":"2024-04-25T10:19:38.768655+0000","last_fullsized":"2024-04-25T10:19:38.768655+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:10.876366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769199+0000","last_change":"2024-04-25T10:19:37.772521+0000","last_active":"2024-04-25T10:19:38.769199+0000","last_peered":"2024-04-25T10:19:38.769199+0000","last_clean":"2024-04-25T10:19:38.769199+0000","last_became_active":"2024-04-25T10:19:37.771768+0000","last_became_peered":"2024-04-25T10:19:37.771768+0000","last_unstale":"2024-04-25T10:19:38.769199+0000","last_undegraded":"2024-04-25T10:19:38.769199+0000","last_fullsized":"2024-04-25T10:19:38.769199+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:54.184769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768013+0000","last_change":"2024-04-25T10:19:36.745047+0000","last_active":"2024-04-25T10:19:38.768013+0000","last_peered":"2024-04-25T10:19:38.768013+0000","last_clean":"2024-04-25T10:19:38.768013+0000","last_became_active":"2024-04-25T10:19:36.744128+0000","last_became_peered":"2024-04-25T10:19:36.744128+0000","last_unstale":"2024-04-25T10:19:38.768013+0000","last_undegraded":"2024-04-25T10:19:38.768013+0000","last_fullsized":"2024-04-25T10:19:38.768013+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:55.951430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839785+0000","last_change":"2024-04-25T10:19:37.788920+0000","last_active":"2024-04-25T10:19:38.839785+0000","last_peered":"2024-04-25T10:19:38.839785+0000","last_clean":"2024-04-25T10:19:38.839785+0000","last_became_active":"2024-04-25T10:19:37.788544+0000","last_became_peered":"2024-04-25T10:19:37.788544+0000","last_unstale":"2024-04-25T10:19:38.839785+0000","last_undegraded":"2024-04-25T10:19:38.839785+0000","last_fullsized":"2024-04-25T10:19:38.839785+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.605247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"100'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769270+0000","last_change":"2024-04-25T10:19:36.758600+0000","last_active":"2024-04-25T10:19:38.769270+0000","last_peered":"2024-04-25T10:19:38.769270+0000","last_clean":"2024-04-25T10:19:38.769270+0000","last_became_active":"2024-04-25T10:19:36.757301+0000","last_became_peered":"2024-04-25T10:19:36.757301+0000","last_unstale":"2024-04-25T10:19:38.769270+0000","last_undegraded":"2024-04-25T10:19:38.769270+0000","last_fullsized":"2024-04-25T10:19:38.769270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.473366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839573+0000","last_change":"2024-04-25T10:19:37.777741+0000","last_active":"2024-04-25T10:19:38.839573+0000","last_peered":"2024-04-25T10:19:38.839573+0000","last_clean":"2024-04-25T10:19:38.839573+0000","last_became_active":"2024-04-25T10:19:37.777439+0000","last_became_peered":"2024-04-25T10:19:37.777439+0000","last_unstale":"2024-04-25T10:19:38.839573+0000","last_undegraded":"2024-04-25T10:19:38.839573+0000","last_fullsized":"2024-04-25T10:19:38.839573+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:23.824033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768942+0000","last_change":"2024-04-25T10:19:36.758075+0000","last_active":"2024-04-25T10:19:38.768942+0000","last_peered":"2024-04-25T10:19:38.768942+0000","last_clean":"2024-04-25T10:19:38.768942+0000","last_became_active":"2024-04-25T10:19:36.757829+0000","last_became_peered":"2024-04-25T10:19:36.757829+0000","last_unstale":"2024-04-25T10:19:38.768942+0000","last_undegraded":"2024-04-25T10:19:38.768942+0000","last_fullsized":"2024-04-25T10:19:38.768942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:20.728540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125502+0000","last_change":"2024-04-25T10:19:37.786005+0000","last_active":"2024-04-25T10:19:40.125502+0000","last_peered":"2024-04-25T10:19:40.125502+0000","last_clean":"2024-04-25T10:19:40.125502+0000","last_became_active":"2024-04-25T10:19:37.785695+0000","last_became_peered":"2024-04-25T10:19:37.785695+0000","last_unstale":"2024-04-25T10:19:40.125502+0000","last_undegraded":"2024-04-25T10:19:40.125502+0000","last_fullsized":"2024-04-25T10:19:40.125502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:48.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125116+0000","last_change":"2024-04-25T10:19:36.747869+0000","last_active":"2024-04-25T10:19:40.125116+0000","last_peered":"2024-04-25T10:19:40.125116+0000","last_clean":"2024-04-25T10:19:40.125116+0000","last_became_active":"2024-04-25T10:19:36.747566+0000","last_became_peered":"2024-04-25T10:19:36.747566+0000","last_unstale":"2024-04-25T10:19:40.125116+0000","last_undegraded":"2024-04-25T10:19:40.125116+0000","last_fullsized":"2024-04-25T10:19:40.125116+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:12.858880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764504+0000","last_change":"2024-04-25T10:19:37.774497+0000","last_active":"2024-04-25T10:19:38.764504+0000","last_peered":"2024-04-25T10:19:38.764504+0000","last_clean":"2024-04-25T10:19:38.764504+0000","last_became_active":"2024-04-25T10:19:37.770481+0000","last_became_peered":"2024-04-25T10:19:37.770481+0000","last_unstale":"2024-04-25T10:19:38.764504+0000","last_undegraded":"2024-04-25T10:19:38.764504+0000","last_fullsized":"2024-04-25T10:19:38.764504+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:29.821562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768859+0000","last_change":"2024-04-25T10:19:36.755590+0000","last_active":"2024-04-25T10:19:38.768859+0000","last_peered":"2024-04-25T10:19:38.768859+0000","last_clean":"2024-04-25T10:19:38.768859+0000","last_became_active":"2024-04-25T10:19:36.754895+0000","last_became_peered":"2024-04-25T10:19:36.754895+0000","last_unstale":"2024-04-25T10:19:38.768859+0000","last_undegraded":"2024-04-25T10:19:38.768859+0000","last_fullsized":"2024-04-25T10:19:38.768859+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:28.718247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767805+0000","last_change":"2024-04-25T10:19:37.766788+0000","last_active":"2024-04-25T10:19:38.767805+0000","last_peered":"2024-04-25T10:19:38.767805+0000","last_clean":"2024-04-25T10:19:38.767805+0000","last_became_active":"2024-04-25T10:19:37.766484+0000","last_became_peered":"2024-04-25T10:19:37.766484+0000","last_unstale":"2024-04-25T10:19:38.767805+0000","last_undegraded":"2024-04-25T10:19:38.767805+0000","last_fullsized":"2024-04-25T10:19:38.767805+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:55.149429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124739+0000","last_change":"2024-04-25T10:19:36.747810+0000","last_active":"2024-04-25T10:19:40.124739+0000","last_peered":"2024-04-25T10:19:40.124739+0000","last_clean":"2024-04-25T10:19:40.124739+0000","last_became_active":"2024-04-25T10:19:36.747502+0000","last_became_peered":"2024-04-25T10:19:36.747502+0000","last_unstale":"2024-04-25T10:19:40.124739+0000","last_undegraded":"2024-04-25T10:19:40.124739+0000","last_fullsized":"2024-04-25T10:19:40.124739+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:02.832069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764077+0000","last_change":"2024-04-25T10:19:37.771355+0000","last_active":"2024-04-25T10:19:38.764077+0000","last_peered":"2024-04-25T10:19:38.764077+0000","last_clean":"2024-04-25T10:19:38.764077+0000","last_became_active":"2024-04-25T10:19:37.771035+0000","last_became_peered":"2024-04-25T10:19:37.771035+0000","last_unstale":"2024-04-25T10:19:38.764077+0000","last_undegraded":"2024-04-25T10:19:38.764077+0000","last_fullsized":"2024-04-25T10:19:38.764077+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:57.414732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958013+0000","last_change":"2024-04-25T10:19:36.744987+0000","last_active":"2024-04-25T10:19:38.958013+0000","last_peered":"2024-04-25T10:19:38.958013+0000","last_clean":"2024-04-25T10:19:38.958013+0000","last_became_active":"2024-04-25T10:19:36.744728+0000","last_became_peered":"2024-04-25T10:19:36.744728+0000","last_unstale":"2024-04-25T10:19:38.958013+0000","last_undegraded":"2024-04-25T10:19:38.958013+0000","last_fullsized":"2024-04-25T10:19:38.958013+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:16.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124990+0000","last_change":"2024-04-25T10:19:37.785995+0000","last_active":"2024-04-25T10:19:40.124990+0000","last_peered":"2024-04-25T10:19:40.124990+0000","last_clean":"2024-04-25T10:19:40.124990+0000","last_became_active":"2024-04-25T10:19:37.785705+0000","last_became_peered":"2024-04-25T10:19:37.785705+0000","last_unstale":"2024-04-25T10:19:40.124990+0000","last_undegraded":"2024-04-25T10:19:40.124990+0000","last_fullsized":"2024-04-25T10:19:40.124990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:55.149429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764112+0000","last_change":"2024-04-25T10:19:36.759979+0000","last_active":"2024-04-25T10:19:38.764112+0000","last_peered":"2024-04-25T10:19:38.764112+0000","last_clean":"2024-04-25T10:19:38.764112+0000","last_became_active":"2024-04-25T10:19:36.759706+0000","last_became_peered":"2024-04-25T10:19:36.759706+0000","last_unstale":"2024-04-25T10:19:38.764112+0000","last_undegraded":"2024-04-25T10:19:38.764112+0000","last_fullsized":"2024-04-25T10:19:38.764112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:06.815334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764916+0000","last_change":"2024-04-25T10:19:37.790840+0000","last_active":"2024-04-25T10:19:38.764916+0000","last_peered":"2024-04-25T10:19:38.764916+0000","last_clean":"2024-04-25T10:19:38.764916+0000","last_became_active":"2024-04-25T10:19:37.785918+0000","last_became_peered":"2024-04-25T10:19:37.785918+0000","last_unstale":"2024-04-25T10:19:38.764916+0000","last_undegraded":"2024-04-25T10:19:38.764916+0000","last_fullsized":"2024-04-25T10:19:38.764916+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:44.716221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125108+0000","last_change":"2024-04-25T10:19:36.758326+0000","last_active":"2024-04-25T10:19:40.125108+0000","last_peered":"2024-04-25T10:19:40.125108+0000","last_clean":"2024-04-25T10:19:40.125108+0000","last_became_active":"2024-04-25T10:19:36.758021+0000","last_became_peered":"2024-04-25T10:19:36.758021+0000","last_unstale":"2024-04-25T10:19:40.125108+0000","last_undegraded":"2024-04-25T10:19:40.125108+0000","last_fullsized":"2024-04-25T10:19:40.125108+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:27.647159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":314,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125204+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:19:40.125204+0000","last_peered":"2024-04-25T10:19:40.125204+0000","last_clean":"2024-04-25T10:19:40.125204+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:19:40.125204+0000","last_undegraded":"2024-04-25T10:19:40.125204+0000","last_fullsized":"2024-04-25T10:19:40.125204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51024,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320816,"statfs":{"total":96636764160,"available":96584515584,"internally_reserved":0,"allocated":1839104,"data_stored":1126262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54604,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317236,"statfs":{"total":96636764160,"available":96580849664,"internally_reserved":0,"allocated":1245184,"data_stored":536434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1241088,"data_stored":535852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54624,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317216,"statfs":{"total":96636764160,"available":96580829184,"internally_reserved":0,"allocated":1257472,"data_stored":537518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50460,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321380,"statfs":{"total":96636764160,"available":96585093120,"internally_reserved":0,"allocated":1253376,"data_stored":537416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46288,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325552,"statfs":{"total":96636764160,"available":96589365248,"internally_reserved":0,"allocated":1241088,"data_stored":535816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76804,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295036,"statfs":{"total":96636764160,"available":96558116864,"internally_reserved":0,"allocated":1236992,"data_stored":535782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51028,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320812,"statfs":{"total":96636764160,"available":96584511488,"internally_reserved":0,"allocated":1835008,"data_stored":1126732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:46.346 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:46.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f03600640 1 Processor -- start 2024-04-25T10:19:46.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f03600640 1 -- start start 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f03600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5efc0cf930 0x7f5efc0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f03600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 0x7f5efc15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f03600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0d0250 con 0x7f5efc0ae020 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f03600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc15bb90 con 0x7f5efc0cf930 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f03600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0ad840 con 0x7f5efc070c20 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f00c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5efc0cf930 0x7f5efc0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f00c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5efc0cf930 0x7f5efc0cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54716/0 (socket says 172.21.15.87:54716) 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f00c00640 1 -- 172.21.15.87:0/324058030 learned_addr learned my addr 172.21.15.87:0/324058030 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f00c00640 1 -- 172.21.15.87:0/324058030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 msgr2=0x7f5efc15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f00c00640 1 --2- 172.21.15.87:0/324058030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 0x7f5efc15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f00c00640 1 -- 172.21.15.87:0/324058030 >> v1:172.21.15.169:6789/0 conn(0x7f5efc0ae020 legacy=0x7f5efc0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.492+0000 7f5f00c00640 1 -- 172.21.15.87:0/324058030 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc15e9f0 con 0x7f5efc0cf930 2024-04-25T10:19:46.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f00c00640 1 --2- 172.21.15.87:0/324058030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0cf930 0x7f5efc0cfd10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009e50 tx=0x7f5ef001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3575235a64f690b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5efb400640 1 -- 172.21.15.87:0/324058030 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00a5840 con 0x7f5efc0cf930 2024-04-25T10:19:46.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5efb400640 1 -- 172.21.15.87:0/324058030 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef00a59e0 con 0x7f5efc0cf930 2024-04-25T10:19:46.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5efb400640 1 -- 172.21.15.87:0/324058030 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00a5db0 con 0x7f5efc0cf930 2024-04-25T10:19:46.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 -- 172.21.15.87:0/324058030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0cf930 msgr2=0x7f5efc0cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 --2- 172.21.15.87:0/324058030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0cf930 0x7f5efc0cfd10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009e50 tx=0x7f5ef001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 -- 172.21.15.87:0/324058030 shutdown_connections 2024-04-25T10:19:46.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 --2- 172.21.15.87:0/324058030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 0x7f5efc15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 --2- 172.21.15.87:0/324058030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0cf930 0x7f5efc0cfd10 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 -- 172.21.15.87:0/324058030 >> 172.21.15.87:0/324058030 conn(0x7f5efc066b70 msgr2=0x7f5efc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:46.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 -- 172.21.15.87:0/324058030 shutdown_connections 2024-04-25T10:19:46.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.496+0000 7f5f03600640 1 -- 172.21.15.87:0/324058030 wait complete. 2024-04-25T10:19:46.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 Processor -- start 2024-04-25T10:19:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 -- start start 2024-04-25T10:19:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 0x7f5efc153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0ae020 0x7f5efc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0cf930 0x7f5efc15a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc15e6b0 con 0x7f5efc0ae020 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc06e130 con 0x7f5efc0cf930 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc16b0f0 con 0x7f5efc070c20 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0ae020 0x7f5efc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0ae020 0x7f5efc159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54730/0 (socket says 172.21.15.87:54730) 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 -- 172.21.15.87:0/2936876840 learned_addr learned my addr 172.21.15.87:0/2936876840 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f00c00640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 0x7f5efc153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f01600640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0cf930 0x7f5efc15a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 -- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 msgr2=0x7f5efc153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 0x7f5efc153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 -- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0cf930 msgr2=0x7f5efc15a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0cf930 0x7f5efc15a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 -- 172.21.15.87:0/2936876840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0afcf0 con 0x7f5efc0ae020 2024-04-25T10:19:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5efbe00640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0ae020 0x7f5efc159bd0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b550 tx=0x7f5ee800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5ef8c00640 1 -- 172.21.15.87:0/2936876840 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8014070 con 0x7f5efc0ae020 2024-04-25T10:19:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5ef8c00640 1 -- 172.21.15.87:0/2936876840 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8010890 con 0x7f5efc0ae020 2024-04-25T10:19:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5ef8c00640 1 -- 172.21.15.87:0/2936876840 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8010c80 con 0x7f5efc0ae020 2024-04-25T10:19:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc0affd0 con 0x7f5efc0ae020 2024-04-25T10:19:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc0b0590 con 0x7f5efc0ae020 2024-04-25T10:19:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.500+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc04ef30 con 0x7f5efc0ae020 2024-04-25T10:19:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.504+0000 7f5ef8c00640 1 -- 172.21.15.87:0/2936876840 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ee8017070 con 0x7f5efc0ae020 2024-04-25T10:19:46.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.504+0000 7f5ef8c00640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5edc066f90 0x7f5edc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.504+0000 7f5ef8c00640 1 -- 172.21.15.87:0/2936876840 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ee8084d40 con 0x7f5efc0ae020 2024-04-25T10:19:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.504+0000 7f5f00c00640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5edc066f90 0x7f5edc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.504+0000 7f5f00c00640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5edc066f90 0x7f5edc069450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00ae000 tx=0x7f5ef00a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.508+0000 7f5ef8c00640 1 -- 172.21.15.87:0/2936876840 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ee80556f0 con 0x7f5efc0ae020 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.660+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5efc167b40 con 0x7f5edc066f90 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.664+0000 7f5ef8c00640 1 -- 172.21.15.87:0/2936876840 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252182 (secure 0 0 0) 0x7f5efc167b40 con 0x7f5edc066f90 2024-04-25T10:19:46.670 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:19:46.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5edc066f90 msgr2=0x7f5edc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5edc066f90 0x7f5edc069450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00ae000 tx=0x7f5ef00a7070 comp rx=0 tx=0).stop 2024-04-25T10:19:46.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0ae020 msgr2=0x7f5efc159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0ae020 0x7f5efc159bd0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b550 tx=0x7f5ee800ba10 comp rx=0 tx=0).stop 2024-04-25T10:19:46.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 shutdown_connections 2024-04-25T10:19:46.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5edc066f90 0x7f5edc069450 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0cf930 0x7f5efc15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5efc0ae020 0x7f5efc159bd0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 --2- 172.21.15.87:0/2936876840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5efc070c20 0x7f5efc153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 >> 172.21.15.87:0/2936876840 conn(0x7f5efc066b70 msgr2=0x7f5efc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 shutdown_connections 2024-04-25T10:19:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.668+0000 7f5f03600640 1 -- 172.21.15.87:0/2936876840 wait complete. 2024-04-25T10:19:46.691 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-04-25T10:19:45.218843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413072,"kb_used_data":10888,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754561648,"statfs":{"total":773094113280,"available":772671127552,"internally_reserved":0,"allocated":11149312,"data_stored":5471812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":411684401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001427"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839751+0000","last_change":"2024-04-25T10:19:37.781818+0000","last_active":"2024-04-25T10:19:38.839751+0000","last_peered":"2024-04-25T10:19:38.839751+0000","last_clean":"2024-04-25T10:19:38.839751+0000","last_became_active":"2024-04-25T10:19:37.779903+0000","last_became_peered":"2024-04-25T10:19:37.779903+0000","last_unstale":"2024-04-25T10:19:38.839751+0000","last_undegraded":"2024-04-25T10:19:38.839751+0000","last_fullsized":"2024-04-25T10:19:38.839751+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:39.766282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768990+0000","last_change":"2024-04-25T10:19:37.772495+0000","last_active":"2024-04-25T10:19:38.768990+0000","last_peered":"2024-04-25T10:19:38.768990+0000","last_clean":"2024-04-25T10:19:38.768990+0000","last_became_active":"2024-04-25T10:19:37.772073+0000","last_became_peered":"2024-04-25T10:19:37.772073+0000","last_unstale":"2024-04-25T10:19:38.768990+0000","last_undegraded":"2024-04-25T10:19:38.768990+0000","last_fullsized":"2024-04-25T10:19:38.768990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.849204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839526+0000","last_change":"2024-04-25T10:19:37.779548+0000","last_active":"2024-04-25T10:19:38.839526+0000","last_peered":"2024-04-25T10:19:38.839526+0000","last_clean":"2024-04-25T10:19:38.839526+0000","last_became_active":"2024-04-25T10:19:37.779212+0000","last_became_peered":"2024-04-25T10:19:37.779212+0000","last_unstale":"2024-04-25T10:19:38.839526+0000","last_undegraded":"2024-04-25T10:19:38.839526+0000","last_fullsized":"2024-04-25T10:19:38.839526+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:35.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958170+0000","last_change":"2024-04-25T10:19:37.773021+0000","last_active":"2024-04-25T10:19:38.958170+0000","last_peered":"2024-04-25T10:19:38.958170+0000","last_clean":"2024-04-25T10:19:38.958170+0000","last_became_active":"2024-04-25T10:19:37.772738+0000","last_became_peered":"2024-04-25T10:19:37.772738+0000","last_unstale":"2024-04-25T10:19:38.958170+0000","last_undegraded":"2024-04-25T10:19:38.958170+0000","last_fullsized":"2024-04-25T10:19:38.958170+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:31.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125010+0000","last_change":"2024-04-25T10:19:37.790648+0000","last_active":"2024-04-25T10:19:40.125010+0000","last_peered":"2024-04-25T10:19:40.125010+0000","last_clean":"2024-04-25T10:19:40.125010+0000","last_became_active":"2024-04-25T10:19:37.790336+0000","last_became_peered":"2024-04-25T10:19:37.790336+0000","last_unstale":"2024-04-25T10:19:40.125010+0000","last_undegraded":"2024-04-25T10:19:40.125010+0000","last_fullsized":"2024-04-25T10:19:40.125010+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:14.153645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125605+0000","last_change":"2024-04-25T10:19:37.771284+0000","last_active":"2024-04-25T10:19:40.125605+0000","last_peered":"2024-04-25T10:19:40.125605+0000","last_clean":"2024-04-25T10:19:40.125605+0000","last_became_active":"2024-04-25T10:19:37.770967+0000","last_became_peered":"2024-04-25T10:19:37.770967+0000","last_unstale":"2024-04-25T10:19:40.125605+0000","last_undegraded":"2024-04-25T10:19:40.125605+0000","last_fullsized":"2024-04-25T10:19:40.125605+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:54.184769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125555+0000","last_change":"2024-04-25T10:19:37.771217+0000","last_active":"2024-04-25T10:19:40.125555+0000","last_peered":"2024-04-25T10:19:40.125555+0000","last_clean":"2024-04-25T10:19:40.125555+0000","last_became_active":"2024-04-25T10:19:37.770914+0000","last_became_peered":"2024-04-25T10:19:37.770914+0000","last_unstale":"2024-04-25T10:19:40.125555+0000","last_undegraded":"2024-04-25T10:19:40.125555+0000","last_fullsized":"2024-04-25T10:19:40.125555+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.645465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768746+0000","last_change":"2024-04-25T10:19:37.779008+0000","last_active":"2024-04-25T10:19:38.768746+0000","last_peered":"2024-04-25T10:19:38.768746+0000","last_clean":"2024-04-25T10:19:38.768746+0000","last_became_active":"2024-04-25T10:19:37.778578+0000","last_became_peered":"2024-04-25T10:19:37.778578+0000","last_unstale":"2024-04-25T10:19:38.768746+0000","last_undegraded":"2024-04-25T10:19:38.768746+0000","last_fullsized":"2024-04-25T10:19:38.768746+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768211+0000","last_change":"2024-04-25T10:19:37.768155+0000","last_active":"2024-04-25T10:19:38.768211+0000","last_peered":"2024-04-25T10:19:38.768211+0000","last_clean":"2024-04-25T10:19:38.768211+0000","last_became_active":"2024-04-25T10:19:37.766771+0000","last_became_peered":"2024-04-25T10:19:37.766771+0000","last_unstale":"2024-04-25T10:19:38.768211+0000","last_undegraded":"2024-04-25T10:19:38.768211+0000","last_fullsized":"2024-04-25T10:19:38.768211+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:35.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125286+0000","last_change":"2024-04-25T10:19:37.779119+0000","last_active":"2024-04-25T10:19:40.125286+0000","last_peered":"2024-04-25T10:19:40.125286+0000","last_clean":"2024-04-25T10:19:40.125286+0000","last_became_active":"2024-04-25T10:19:37.778834+0000","last_became_peered":"2024-04-25T10:19:37.778834+0000","last_unstale":"2024-04-25T10:19:40.125286+0000","last_undegraded":"2024-04-25T10:19:40.125286+0000","last_fullsized":"2024-04-25T10:19:40.125286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125229+0000","last_change":"2024-04-25T10:19:37.770142+0000","last_active":"2024-04-25T10:19:40.125229+0000","last_peered":"2024-04-25T10:19:40.125229+0000","last_clean":"2024-04-25T10:19:40.125229+0000","last_became_active":"2024-04-25T10:19:37.769871+0000","last_became_peered":"2024-04-25T10:19:37.769871+0000","last_unstale":"2024-04-25T10:19:40.125229+0000","last_undegraded":"2024-04-25T10:19:40.125229+0000","last_fullsized":"2024-04-25T10:19:40.125229+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:06.128960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768117+0000","last_change":"2024-04-25T10:19:37.769809+0000","last_active":"2024-04-25T10:19:38.768117+0000","last_peered":"2024-04-25T10:19:38.768117+0000","last_clean":"2024-04-25T10:19:38.768117+0000","last_became_active":"2024-04-25T10:19:37.768260+0000","last_became_peered":"2024-04-25T10:19:37.768260+0000","last_unstale":"2024-04-25T10:19:38.768117+0000","last_undegraded":"2024-04-25T10:19:38.768117+0000","last_fullsized":"2024-04-25T10:19:38.768117+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.629408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957969+0000","last_change":"2024-04-25T10:19:37.780096+0000","last_active":"2024-04-25T10:19:38.957969+0000","last_peered":"2024-04-25T10:19:38.957969+0000","last_clean":"2024-04-25T10:19:38.957969+0000","last_became_active":"2024-04-25T10:19:37.769941+0000","last_became_peered":"2024-04-25T10:19:37.769941+0000","last_unstale":"2024-04-25T10:19:38.957969+0000","last_undegraded":"2024-04-25T10:19:38.957969+0000","last_fullsized":"2024-04-25T10:19:38.957969+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764342+0000","last_change":"2024-04-25T10:19:37.775131+0000","last_active":"2024-04-25T10:19:38.764342+0000","last_peered":"2024-04-25T10:19:38.764342+0000","last_clean":"2024-04-25T10:19:38.764342+0000","last_became_active":"2024-04-25T10:19:37.774866+0000","last_became_peered":"2024-04-25T10:19:37.774866+0000","last_unstale":"2024-04-25T10:19:38.764342+0000","last_undegraded":"2024-04-25T10:19:38.764342+0000","last_fullsized":"2024-04-25T10:19:38.764342+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:19.879710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957910+0000","last_change":"2024-04-25T10:19:37.774477+0000","last_active":"2024-04-25T10:19:38.957910+0000","last_peered":"2024-04-25T10:19:38.957910+0000","last_clean":"2024-04-25T10:19:38.957910+0000","last_became_active":"2024-04-25T10:19:37.773941+0000","last_became_peered":"2024-04-25T10:19:37.773941+0000","last_unstale":"2024-04-25T10:19:38.957910+0000","last_undegraded":"2024-04-25T10:19:38.957910+0000","last_fullsized":"2024-04-25T10:19:38.957910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125346+0000","last_change":"2024-04-25T10:19:37.781811+0000","last_active":"2024-04-25T10:19:40.125346+0000","last_peered":"2024-04-25T10:19:40.125346+0000","last_clean":"2024-04-25T10:19:40.125346+0000","last_became_active":"2024-04-25T10:19:37.781277+0000","last_became_peered":"2024-04-25T10:19:37.781277+0000","last_unstale":"2024-04-25T10:19:40.125346+0000","last_undegraded":"2024-04-25T10:19:40.125346+0000","last_fullsized":"2024-04-25T10:19:40.125346+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:31.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769165+0000","last_change":"2024-04-25T10:19:37.774791+0000","last_active":"2024-04-25T10:19:38.769165+0000","last_peered":"2024-04-25T10:19:38.769165+0000","last_clean":"2024-04-25T10:19:38.769165+0000","last_became_active":"2024-04-25T10:19:37.774510+0000","last_became_peered":"2024-04-25T10:19:37.774510+0000","last_unstale":"2024-04-25T10:19:38.769165+0000","last_undegraded":"2024-04-25T10:19:38.769165+0000","last_fullsized":"2024-04-25T10:19:38.769165+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.807600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958231+0000","last_change":"2024-04-25T10:19:37.788665+0000","last_active":"2024-04-25T10:19:38.958231+0000","last_peered":"2024-04-25T10:19:38.958231+0000","last_clean":"2024-04-25T10:19:38.958231+0000","last_became_active":"2024-04-25T10:19:37.788349+0000","last_became_peered":"2024-04-25T10:19:37.788349+0000","last_unstale":"2024-04-25T10:19:38.958231+0000","last_undegraded":"2024-04-25T10:19:38.958231+0000","last_fullsized":"2024-04-25T10:19:38.958231+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.849204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764638+0000","last_change":"2024-04-25T10:19:37.772490+0000","last_active":"2024-04-25T10:19:38.764638+0000","last_peered":"2024-04-25T10:19:38.764638+0000","last_clean":"2024-04-25T10:19:38.764638+0000","last_became_active":"2024-04-25T10:19:37.772163+0000","last_became_peered":"2024-04-25T10:19:37.772163+0000","last_unstale":"2024-04-25T10:19:38.764638+0000","last_undegraded":"2024-04-25T10:19:38.764638+0000","last_fullsized":"2024-04-25T10:19:38.764638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:24.445381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768296+0000","last_change":"2024-04-25T10:19:37.760459+0000","last_active":"2024-04-25T10:19:38.768296+0000","last_peered":"2024-04-25T10:19:38.768296+0000","last_clean":"2024-04-25T10:19:38.768296+0000","last_became_active":"2024-04-25T10:19:37.760170+0000","last_became_peered":"2024-04-25T10:19:37.760170+0000","last_unstale":"2024-04-25T10:19:38.768296+0000","last_undegraded":"2024-04-25T10:19:38.768296+0000","last_fullsized":"2024-04-25T10:19:38.768296+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:48.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764490+0000","last_change":"2024-04-25T10:19:37.784933+0000","last_active":"2024-04-25T10:19:38.764490+0000","last_peered":"2024-04-25T10:19:38.764490+0000","last_clean":"2024-04-25T10:19:38.764490+0000","last_became_active":"2024-04-25T10:19:37.784659+0000","last_became_peered":"2024-04-25T10:19:37.784659+0000","last_unstale":"2024-04-25T10:19:38.764490+0000","last_undegraded":"2024-04-25T10:19:38.764490+0000","last_fullsized":"2024-04-25T10:19:38.764490+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:15.594908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768870+0000","last_change":"2024-04-25T10:19:37.770191+0000","last_active":"2024-04-25T10:19:38.768870+0000","last_peered":"2024-04-25T10:19:38.768870+0000","last_clean":"2024-04-25T10:19:38.768870+0000","last_became_active":"2024-04-25T10:19:37.769851+0000","last_became_peered":"2024-04-25T10:19:37.769851+0000","last_unstale":"2024-04-25T10:19:38.768870+0000","last_undegraded":"2024-04-25T10:19:38.768870+0000","last_fullsized":"2024-04-25T10:19:38.768870+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:31.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768715+0000","last_change":"2024-04-25T10:19:37.769040+0000","last_active":"2024-04-25T10:19:38.768715+0000","last_peered":"2024-04-25T10:19:38.768715+0000","last_clean":"2024-04-25T10:19:38.768715+0000","last_became_active":"2024-04-25T10:19:37.768733+0000","last_became_peered":"2024-04-25T10:19:37.768733+0000","last_unstale":"2024-04-25T10:19:38.768715+0000","last_undegraded":"2024-04-25T10:19:38.768715+0000","last_fullsized":"2024-04-25T10:19:38.768715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125143+0000","last_change":"2024-04-25T10:19:37.782354+0000","last_active":"2024-04-25T10:19:40.125143+0000","last_peered":"2024-04-25T10:19:40.125143+0000","last_clean":"2024-04-25T10:19:40.125143+0000","last_became_active":"2024-04-25T10:19:37.782102+0000","last_became_peered":"2024-04-25T10:19:37.782102+0000","last_unstale":"2024-04-25T10:19:40.125143+0000","last_undegraded":"2024-04-25T10:19:40.125143+0000","last_fullsized":"2024-04-25T10:19:40.125143+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125707+0000","last_change":"2024-04-25T10:19:37.781507+0000","last_active":"2024-04-25T10:19:40.125707+0000","last_peered":"2024-04-25T10:19:40.125707+0000","last_clean":"2024-04-25T10:19:40.125707+0000","last_became_active":"2024-04-25T10:19:37.780601+0000","last_became_peered":"2024-04-25T10:19:37.780601+0000","last_unstale":"2024-04-25T10:19:40.125707+0000","last_undegraded":"2024-04-25T10:19:40.125707+0000","last_fullsized":"2024-04-25T10:19:40.125707+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769057+0000","last_change":"2024-04-25T10:19:37.771467+0000","last_active":"2024-04-25T10:19:38.769057+0000","last_peered":"2024-04-25T10:19:38.769057+0000","last_clean":"2024-04-25T10:19:38.769057+0000","last_became_active":"2024-04-25T10:19:37.769290+0000","last_became_peered":"2024-04-25T10:19:37.769290+0000","last_unstale":"2024-04-25T10:19:38.769057+0000","last_undegraded":"2024-04-25T10:19:38.769057+0000","last_fullsized":"2024-04-25T10:19:38.769057+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.914760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125390+0000","last_change":"2024-04-25T10:19:37.788074+0000","last_active":"2024-04-25T10:19:40.125390+0000","last_peered":"2024-04-25T10:19:40.125390+0000","last_clean":"2024-04-25T10:19:40.125390+0000","last_became_active":"2024-04-25T10:19:37.787748+0000","last_became_peered":"2024-04-25T10:19:37.787748+0000","last_unstale":"2024-04-25T10:19:40.125390+0000","last_undegraded":"2024-04-25T10:19:40.125390+0000","last_fullsized":"2024-04-25T10:19:40.125390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.849204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125053+0000","last_change":"2024-04-25T10:19:37.775285+0000","last_active":"2024-04-25T10:19:40.125053+0000","last_peered":"2024-04-25T10:19:40.125053+0000","last_clean":"2024-04-25T10:19:40.125053+0000","last_became_active":"2024-04-25T10:19:37.774991+0000","last_became_peered":"2024-04-25T10:19:37.774991+0000","last_unstale":"2024-04-25T10:19:40.125053+0000","last_undegraded":"2024-04-25T10:19:40.125053+0000","last_fullsized":"2024-04-25T10:19:40.125053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.629408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768803+0000","last_change":"2024-04-25T10:19:37.780736+0000","last_active":"2024-04-25T10:19:38.768803+0000","last_peered":"2024-04-25T10:19:38.768803+0000","last_clean":"2024-04-25T10:19:38.768803+0000","last_became_active":"2024-04-25T10:19:37.780473+0000","last_became_peered":"2024-04-25T10:19:37.780473+0000","last_unstale":"2024-04-25T10:19:38.768803+0000","last_undegraded":"2024-04-25T10:19:38.768803+0000","last_fullsized":"2024-04-25T10:19:38.768803+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:06.128960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764315+0000","last_change":"2024-04-25T10:19:37.775511+0000","last_active":"2024-04-25T10:19:38.764315+0000","last_peered":"2024-04-25T10:19:38.764315+0000","last_clean":"2024-04-25T10:19:38.764315+0000","last_became_active":"2024-04-25T10:19:37.768009+0000","last_became_peered":"2024-04-25T10:19:37.768009+0000","last_unstale":"2024-04-25T10:19:38.764315+0000","last_undegraded":"2024-04-25T10:19:38.764315+0000","last_fullsized":"2024-04-25T10:19:38.764315+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:11.100611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957958+0000","last_change":"2024-04-25T10:19:37.774229+0000","last_active":"2024-04-25T10:19:38.957958+0000","last_peered":"2024-04-25T10:19:38.957958+0000","last_clean":"2024-04-25T10:19:38.957958+0000","last_became_active":"2024-04-25T10:19:37.773529+0000","last_became_peered":"2024-04-25T10:19:37.773529+0000","last_unstale":"2024-04-25T10:19:38.957958+0000","last_undegraded":"2024-04-25T10:19:38.957958+0000","last_fullsized":"2024-04-25T10:19:38.957958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:48.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839433+0000","last_change":"2024-04-25T10:19:37.768567+0000","last_active":"2024-04-25T10:19:38.839433+0000","last_peered":"2024-04-25T10:19:38.839433+0000","last_clean":"2024-04-25T10:19:38.839433+0000","last_became_active":"2024-04-25T10:19:37.768294+0000","last_became_peered":"2024-04-25T10:19:37.768294+0000","last_unstale":"2024-04-25T10:19:38.839433+0000","last_undegraded":"2024-04-25T10:19:38.839433+0000","last_fullsized":"2024-04-25T10:19:38.839433+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:06.640576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764738+0000","last_change":"2024-04-25T10:19:37.788308+0000","last_active":"2024-04-25T10:19:38.764738+0000","last_peered":"2024-04-25T10:19:38.764738+0000","last_clean":"2024-04-25T10:19:38.764738+0000","last_became_active":"2024-04-25T10:19:37.786260+0000","last_became_peered":"2024-04-25T10:19:37.786260+0000","last_unstale":"2024-04-25T10:19:38.764738+0000","last_undegraded":"2024-04-25T10:19:38.764738+0000","last_fullsized":"2024-04-25T10:19:38.764738+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:00.816107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769120+0000","last_change":"2024-04-25T10:19:36.754304+0000","last_active":"2024-04-25T10:19:38.769120+0000","last_peered":"2024-04-25T10:19:38.769120+0000","last_clean":"2024-04-25T10:19:38.769120+0000","last_became_active":"2024-04-25T10:19:36.754033+0000","last_became_peered":"2024-04-25T10:19:36.754033+0000","last_unstale":"2024-04-25T10:19:38.769120+0000","last_undegraded":"2024-04-25T10:19:38.769120+0000","last_fullsized":"2024-04-25T10:19:38.769120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:16.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958344+0000","last_change":"2024-04-25T10:19:37.772229+0000","last_active":"2024-04-25T10:19:38.958344+0000","last_peered":"2024-04-25T10:19:38.958344+0000","last_clean":"2024-04-25T10:19:38.958344+0000","last_became_active":"2024-04-25T10:19:37.771908+0000","last_became_peered":"2024-04-25T10:19:37.771908+0000","last_unstale":"2024-04-25T10:19:38.958344+0000","last_undegraded":"2024-04-25T10:19:38.958344+0000","last_fullsized":"2024-04-25T10:19:38.958344+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.914760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125423+0000","last_change":"2024-04-25T10:19:36.757825+0000","last_active":"2024-04-25T10:19:40.125423+0000","last_peered":"2024-04-25T10:19:40.125423+0000","last_clean":"2024-04-25T10:19:40.125423+0000","last_became_active":"2024-04-25T10:19:36.757506+0000","last_became_peered":"2024-04-25T10:19:36.757506+0000","last_unstale":"2024-04-25T10:19:40.125423+0000","last_undegraded":"2024-04-25T10:19:40.125423+0000","last_fullsized":"2024-04-25T10:19:40.125423+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:10.876366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839412+0000","last_change":"2024-04-25T10:19:37.777602+0000","last_active":"2024-04-25T10:19:38.839412+0000","last_peered":"2024-04-25T10:19:38.839412+0000","last_clean":"2024-04-25T10:19:38.839412+0000","last_became_active":"2024-04-25T10:19:37.769547+0000","last_became_peered":"2024-04-25T10:19:37.769547+0000","last_unstale":"2024-04-25T10:19:38.839412+0000","last_undegraded":"2024-04-25T10:19:38.839412+0000","last_fullsized":"2024-04-25T10:19:38.839412+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:48.146382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768982+0000","last_change":"2024-04-25T10:19:36.761223+0000","last_active":"2024-04-25T10:19:38.768982+0000","last_peered":"2024-04-25T10:19:38.768982+0000","last_clean":"2024-04-25T10:19:38.768982+0000","last_became_active":"2024-04-25T10:19:36.761008+0000","last_became_peered":"2024-04-25T10:19:36.761008+0000","last_unstale":"2024-04-25T10:19:38.768982+0000","last_undegraded":"2024-04-25T10:19:38.768982+0000","last_fullsized":"2024-04-25T10:19:38.768982+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:11.139422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768063+0000","last_change":"2024-04-25T10:19:37.762533+0000","last_active":"2024-04-25T10:19:38.768063+0000","last_peered":"2024-04-25T10:19:38.768063+0000","last_clean":"2024-04-25T10:19:38.768063+0000","last_became_active":"2024-04-25T10:19:37.762211+0000","last_became_peered":"2024-04-25T10:19:37.762211+0000","last_unstale":"2024-04-25T10:19:38.768063+0000","last_undegraded":"2024-04-25T10:19:38.768063+0000","last_fullsized":"2024-04-25T10:19:38.768063+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:14.153645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768824+0000","last_change":"2024-04-25T10:19:36.755923+0000","last_active":"2024-04-25T10:19:38.768824+0000","last_peered":"2024-04-25T10:19:38.768824+0000","last_clean":"2024-04-25T10:19:38.768824+0000","last_became_active":"2024-04-25T10:19:36.755237+0000","last_became_peered":"2024-04-25T10:19:36.755237+0000","last_unstale":"2024-04-25T10:19:38.768824+0000","last_undegraded":"2024-04-25T10:19:38.768824+0000","last_fullsized":"2024-04-25T10:19:38.768824+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:23.420555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767851+0000","last_change":"2024-04-25T10:19:37.763156+0000","last_active":"2024-04-25T10:19:38.767851+0000","last_peered":"2024-04-25T10:19:38.767851+0000","last_clean":"2024-04-25T10:19:38.767851+0000","last_became_active":"2024-04-25T10:19:37.762875+0000","last_became_peered":"2024-04-25T10:19:37.762875+0000","last_unstale":"2024-04-25T10:19:38.767851+0000","last_undegraded":"2024-04-25T10:19:38.767851+0000","last_fullsized":"2024-04-25T10:19:38.767851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:13.864917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839308+0000","last_change":"2024-04-25T10:19:36.752703+0000","last_active":"2024-04-25T10:19:38.839308+0000","last_peered":"2024-04-25T10:19:38.839308+0000","last_clean":"2024-04-25T10:19:38.839308+0000","last_became_active":"2024-04-25T10:19:36.751221+0000","last_became_peered":"2024-04-25T10:19:36.751221+0000","last_unstale":"2024-04-25T10:19:38.839308+0000","last_undegraded":"2024-04-25T10:19:38.839308+0000","last_fullsized":"2024-04-25T10:19:38.839308+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:14.588871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125027+0000","last_change":"2024-04-25T10:19:37.770422+0000","last_active":"2024-04-25T10:19:40.125027+0000","last_peered":"2024-04-25T10:19:40.125027+0000","last_clean":"2024-04-25T10:19:40.125027+0000","last_became_active":"2024-04-25T10:19:37.769530+0000","last_became_peered":"2024-04-25T10:19:37.769530+0000","last_unstale":"2024-04-25T10:19:40.125027+0000","last_undegraded":"2024-04-25T10:19:40.125027+0000","last_fullsized":"2024-04-25T10:19:40.125027+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:31.871757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768626+0000","last_change":"2024-04-25T10:19:36.758029+0000","last_active":"2024-04-25T10:19:38.768626+0000","last_peered":"2024-04-25T10:19:38.768626+0000","last_clean":"2024-04-25T10:19:38.768626+0000","last_became_active":"2024-04-25T10:19:36.757771+0000","last_became_peered":"2024-04-25T10:19:36.757771+0000","last_unstale":"2024-04-25T10:19:38.768626+0000","last_undegraded":"2024-04-25T10:19:38.768626+0000","last_fullsized":"2024-04-25T10:19:38.768626+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:27.647159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.957988+0000","last_change":"2024-04-25T10:19:37.786017+0000","last_active":"2024-04-25T10:19:38.957988+0000","last_peered":"2024-04-25T10:19:38.957988+0000","last_clean":"2024-04-25T10:19:38.957988+0000","last_became_active":"2024-04-25T10:19:37.785742+0000","last_became_peered":"2024-04-25T10:19:37.785742+0000","last_unstale":"2024-04-25T10:19:38.957988+0000","last_undegraded":"2024-04-25T10:19:38.957988+0000","last_fullsized":"2024-04-25T10:19:38.957988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124876+0000","last_change":"2024-04-25T10:19:36.759897+0000","last_active":"2024-04-25T10:19:40.124876+0000","last_peered":"2024-04-25T10:19:40.124876+0000","last_clean":"2024-04-25T10:19:40.124876+0000","last_became_active":"2024-04-25T10:19:36.759589+0000","last_became_peered":"2024-04-25T10:19:36.759589+0000","last_unstale":"2024-04-25T10:19:40.124876+0000","last_undegraded":"2024-04-25T10:19:40.124876+0000","last_fullsized":"2024-04-25T10:19:40.124876+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:23.420555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767347+0000","last_change":"2024-04-25T10:19:37.758007+0000","last_active":"2024-04-25T10:19:38.767347+0000","last_peered":"2024-04-25T10:19:38.767347+0000","last_clean":"2024-04-25T10:19:38.767347+0000","last_became_active":"2024-04-25T10:19:37.757720+0000","last_became_peered":"2024-04-25T10:19:37.757720+0000","last_unstale":"2024-04-25T10:19:38.767347+0000","last_undegraded":"2024-04-25T10:19:38.767347+0000","last_fullsized":"2024-04-25T10:19:38.767347+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:15.750695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839301+0000","last_change":"2024-04-25T10:19:36.753819+0000","last_active":"2024-04-25T10:19:38.839301+0000","last_peered":"2024-04-25T10:19:38.839301+0000","last_clean":"2024-04-25T10:19:38.839301+0000","last_became_active":"2024-04-25T10:19:36.753582+0000","last_became_peered":"2024-04-25T10:19:36.753582+0000","last_unstale":"2024-04-25T10:19:38.839301+0000","last_undegraded":"2024-04-25T10:19:38.839301+0000","last_fullsized":"2024-04-25T10:19:38.839301+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:44.204686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769145+0000","last_change":"2024-04-25T10:19:37.772310+0000","last_active":"2024-04-25T10:19:38.769145+0000","last_peered":"2024-04-25T10:19:38.769145+0000","last_clean":"2024-04-25T10:19:38.769145+0000","last_became_active":"2024-04-25T10:19:37.771136+0000","last_became_peered":"2024-04-25T10:19:37.771136+0000","last_unstale":"2024-04-25T10:19:38.769145+0000","last_undegraded":"2024-04-25T10:19:38.769145+0000","last_fullsized":"2024-04-25T10:19:38.769145+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.645465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839778+0000","last_change":"2024-04-25T10:19:36.747385+0000","last_active":"2024-04-25T10:19:38.839778+0000","last_peered":"2024-04-25T10:19:38.839778+0000","last_clean":"2024-04-25T10:19:38.839778+0000","last_became_active":"2024-04-25T10:19:36.746924+0000","last_became_peered":"2024-04-25T10:19:36.746924+0000","last_unstale":"2024-04-25T10:19:38.839778+0000","last_undegraded":"2024-04-25T10:19:38.839778+0000","last_fullsized":"2024-04-25T10:19:38.839778+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:18.873673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768271+0000","last_change":"2024-04-25T10:19:37.762256+0000","last_active":"2024-04-25T10:19:38.768271+0000","last_peered":"2024-04-25T10:19:38.768271+0000","last_clean":"2024-04-25T10:19:38.768271+0000","last_became_active":"2024-04-25T10:19:37.761921+0000","last_became_peered":"2024-04-25T10:19:37.761921+0000","last_unstale":"2024-04-25T10:19:38.768271+0000","last_undegraded":"2024-04-25T10:19:38.768271+0000","last_fullsized":"2024-04-25T10:19:38.768271+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958180+0000","last_change":"2024-04-25T10:19:36.759744+0000","last_active":"2024-04-25T10:19:38.958180+0000","last_peered":"2024-04-25T10:19:38.958180+0000","last_clean":"2024-04-25T10:19:38.958180+0000","last_became_active":"2024-04-25T10:19:36.759477+0000","last_became_peered":"2024-04-25T10:19:36.759477+0000","last_unstale":"2024-04-25T10:19:38.958180+0000","last_undegraded":"2024-04-25T10:19:38.958180+0000","last_fullsized":"2024-04-25T10:19:38.958180+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.473366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125088+0000","last_change":"2024-04-25T10:19:37.777332+0000","last_active":"2024-04-25T10:19:40.125088+0000","last_peered":"2024-04-25T10:19:40.125088+0000","last_clean":"2024-04-25T10:19:40.125088+0000","last_became_active":"2024-04-25T10:19:37.777052+0000","last_became_peered":"2024-04-25T10:19:37.777052+0000","last_unstale":"2024-04-25T10:19:40.125088+0000","last_undegraded":"2024-04-25T10:19:40.125088+0000","last_fullsized":"2024-04-25T10:19:40.125088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:35.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125472+0000","last_change":"2024-04-25T10:19:36.754486+0000","last_active":"2024-04-25T10:19:40.125472+0000","last_peered":"2024-04-25T10:19:40.125472+0000","last_clean":"2024-04-25T10:19:40.125472+0000","last_became_active":"2024-04-25T10:19:36.754183+0000","last_became_peered":"2024-04-25T10:19:36.754183+0000","last_unstale":"2024-04-25T10:19:40.125472+0000","last_undegraded":"2024-04-25T10:19:40.125472+0000","last_fullsized":"2024-04-25T10:19:40.125472+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.473366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125416+0000","last_change":"2024-04-25T10:19:37.782113+0000","last_active":"2024-04-25T10:19:40.125416+0000","last_peered":"2024-04-25T10:19:40.125416+0000","last_clean":"2024-04-25T10:19:40.125416+0000","last_became_active":"2024-04-25T10:19:37.781847+0000","last_became_peered":"2024-04-25T10:19:37.781847+0000","last_unstale":"2024-04-25T10:19:40.125416+0000","last_undegraded":"2024-04-25T10:19:40.125416+0000","last_fullsized":"2024-04-25T10:19:40.125416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.914760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768199+0000","last_change":"2024-04-25T10:19:36.738569+0000","last_active":"2024-04-25T10:19:38.768199+0000","last_peered":"2024-04-25T10:19:38.768199+0000","last_clean":"2024-04-25T10:19:38.768199+0000","last_became_active":"2024-04-25T10:19:36.738280+0000","last_became_peered":"2024-04-25T10:19:36.738280+0000","last_unstale":"2024-04-25T10:19:38.768199+0000","last_undegraded":"2024-04-25T10:19:38.768199+0000","last_fullsized":"2024-04-25T10:19:38.768199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:55.564589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958101+0000","last_change":"2024-04-25T10:19:37.763128+0000","last_active":"2024-04-25T10:19:38.958101+0000","last_peered":"2024-04-25T10:19:38.958101+0000","last_clean":"2024-04-25T10:19:38.958101+0000","last_became_active":"2024-04-25T10:19:37.762821+0000","last_became_peered":"2024-04-25T10:19:37.762821+0000","last_unstale":"2024-04-25T10:19:38.958101+0000","last_undegraded":"2024-04-25T10:19:38.958101+0000","last_fullsized":"2024-04-25T10:19:38.958101+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:06.128960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764543+0000","last_change":"2024-04-25T10:19:36.753686+0000","last_active":"2024-04-25T10:19:38.764543+0000","last_peered":"2024-04-25T10:19:38.764543+0000","last_clean":"2024-04-25T10:19:38.764543+0000","last_became_active":"2024-04-25T10:19:36.753151+0000","last_became_peered":"2024-04-25T10:19:36.753151+0000","last_unstale":"2024-04-25T10:19:38.764543+0000","last_undegraded":"2024-04-25T10:19:38.764543+0000","last_fullsized":"2024-04-25T10:19:38.764543+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:46.990158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839629+0000","last_change":"2024-04-25T10:19:37.788705+0000","last_active":"2024-04-25T10:19:38.839629+0000","last_peered":"2024-04-25T10:19:38.839629+0000","last_clean":"2024-04-25T10:19:38.839629+0000","last_became_active":"2024-04-25T10:19:37.788149+0000","last_became_peered":"2024-04-25T10:19:37.788149+0000","last_unstale":"2024-04-25T10:19:38.839629+0000","last_undegraded":"2024-04-25T10:19:38.839629+0000","last_fullsized":"2024-04-25T10:19:38.839629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:42.482477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764560+0000","last_change":"2024-04-25T10:19:36.755955+0000","last_active":"2024-04-25T10:19:38.764560+0000","last_peered":"2024-04-25T10:19:38.764560+0000","last_clean":"2024-04-25T10:19:38.764560+0000","last_became_active":"2024-04-25T10:19:36.755468+0000","last_became_peered":"2024-04-25T10:19:36.755468+0000","last_unstale":"2024-04-25T10:19:38.764560+0000","last_undegraded":"2024-04-25T10:19:38.764560+0000","last_fullsized":"2024-04-25T10:19:38.764560+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:52.793832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767003+0000","last_change":"2024-04-25T10:19:37.761881+0000","last_active":"2024-04-25T10:19:38.767003+0000","last_peered":"2024-04-25T10:19:38.767003+0000","last_clean":"2024-04-25T10:19:38.767003+0000","last_became_active":"2024-04-25T10:19:37.761503+0000","last_became_peered":"2024-04-25T10:19:37.761503+0000","last_unstale":"2024-04-25T10:19:38.767003+0000","last_undegraded":"2024-04-25T10:19:38.767003+0000","last_fullsized":"2024-04-25T10:19:38.767003+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:11.113877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767070+0000","last_change":"2024-04-25T10:19:36.745660+0000","last_active":"2024-04-25T10:19:38.767070+0000","last_peered":"2024-04-25T10:19:38.767070+0000","last_clean":"2024-04-25T10:19:38.767070+0000","last_became_active":"2024-04-25T10:19:36.745382+0000","last_became_peered":"2024-04-25T10:19:36.745382+0000","last_unstale":"2024-04-25T10:19:38.767070+0000","last_undegraded":"2024-04-25T10:19:38.767070+0000","last_fullsized":"2024-04-25T10:19:38.767070+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:23.420555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764127+0000","last_change":"2024-04-25T10:19:37.770742+0000","last_active":"2024-04-25T10:19:38.764127+0000","last_peered":"2024-04-25T10:19:38.764127+0000","last_clean":"2024-04-25T10:19:38.764127+0000","last_became_active":"2024-04-25T10:19:37.770255+0000","last_became_peered":"2024-04-25T10:19:37.770255+0000","last_unstale":"2024-04-25T10:19:38.764127+0000","last_undegraded":"2024-04-25T10:19:38.764127+0000","last_fullsized":"2024-04-25T10:19:38.764127+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:45.210723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839419+0000","last_change":"2024-04-25T10:19:36.748763+0000","last_active":"2024-04-25T10:19:38.839419+0000","last_peered":"2024-04-25T10:19:38.839419+0000","last_clean":"2024-04-25T10:19:38.839419+0000","last_became_active":"2024-04-25T10:19:36.748478+0000","last_became_peered":"2024-04-25T10:19:36.748478+0000","last_unstale":"2024-04-25T10:19:38.839419+0000","last_undegraded":"2024-04-25T10:19:38.839419+0000","last_fullsized":"2024-04-25T10:19:38.839419+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:10.094574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125753+0000","last_change":"2024-04-25T10:19:37.773281+0000","last_active":"2024-04-25T10:19:40.125753+0000","last_peered":"2024-04-25T10:19:40.125753+0000","last_clean":"2024-04-25T10:19:40.125753+0000","last_became_active":"2024-04-25T10:19:37.772880+0000","last_became_peered":"2024-04-25T10:19:37.772880+0000","last_unstale":"2024-04-25T10:19:40.125753+0000","last_undegraded":"2024-04-25T10:19:40.125753+0000","last_fullsized":"2024-04-25T10:19:40.125753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:52.423560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767884+0000","last_change":"2024-04-25T10:19:36.755567+0000","last_active":"2024-04-25T10:19:38.767884+0000","last_peered":"2024-04-25T10:19:38.767884+0000","last_clean":"2024-04-25T10:19:38.767884+0000","last_became_active":"2024-04-25T10:19:36.755294+0000","last_became_peered":"2024-04-25T10:19:36.755294+0000","last_unstale":"2024-04-25T10:19:38.767884+0000","last_undegraded":"2024-04-25T10:19:38.767884+0000","last_fullsized":"2024-04-25T10:19:38.767884+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:20.728540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767755+0000","last_change":"2024-04-25T10:19:37.767526+0000","last_active":"2024-04-25T10:19:38.767755+0000","last_peered":"2024-04-25T10:19:38.767755+0000","last_clean":"2024-04-25T10:19:38.767755+0000","last_became_active":"2024-04-25T10:19:37.767229+0000","last_became_peered":"2024-04-25T10:19:37.767229+0000","last_unstale":"2024-04-25T10:19:38.767755+0000","last_undegraded":"2024-04-25T10:19:38.767755+0000","last_fullsized":"2024-04-25T10:19:38.767755+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:03.838106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125564+0000","last_change":"2024-04-25T10:19:36.748121+0000","last_active":"2024-04-25T10:19:40.125564+0000","last_peered":"2024-04-25T10:19:40.125564+0000","last_clean":"2024-04-25T10:19:40.125564+0000","last_became_active":"2024-04-25T10:19:36.747810+0000","last_became_peered":"2024-04-25T10:19:36.747810+0000","last_unstale":"2024-04-25T10:19:40.125564+0000","last_undegraded":"2024-04-25T10:19:40.125564+0000","last_fullsized":"2024-04-25T10:19:40.125564+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:30.325508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125571+0000","last_change":"2024-04-25T10:19:37.773693+0000","last_active":"2024-04-25T10:19:40.125571+0000","last_peered":"2024-04-25T10:19:40.125571+0000","last_clean":"2024-04-25T10:19:40.125571+0000","last_became_active":"2024-04-25T10:19:37.772443+0000","last_became_peered":"2024-04-25T10:19:37.772443+0000","last_unstale":"2024-04-25T10:19:40.125571+0000","last_undegraded":"2024-04-25T10:19:40.125571+0000","last_fullsized":"2024-04-25T10:19:40.125571+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.807600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768902+0000","last_change":"2024-04-25T10:19:36.755521+0000","last_active":"2024-04-25T10:19:38.768902+0000","last_peered":"2024-04-25T10:19:38.768902+0000","last_clean":"2024-04-25T10:19:38.768902+0000","last_became_active":"2024-04-25T10:19:36.755228+0000","last_became_peered":"2024-04-25T10:19:36.755228+0000","last_unstale":"2024-04-25T10:19:38.768902+0000","last_undegraded":"2024-04-25T10:19:38.768902+0000","last_fullsized":"2024-04-25T10:19:38.768902+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:55.564589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764586+0000","last_change":"2024-04-25T10:19:37.768378+0000","last_active":"2024-04-25T10:19:38.764586+0000","last_peered":"2024-04-25T10:19:38.764586+0000","last_clean":"2024-04-25T10:19:38.764586+0000","last_became_active":"2024-04-25T10:19:37.767376+0000","last_became_peered":"2024-04-25T10:19:37.767376+0000","last_unstale":"2024-04-25T10:19:38.764586+0000","last_undegraded":"2024-04-25T10:19:38.764586+0000","last_fullsized":"2024-04-25T10:19:38.764586+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:29.459034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768741+0000","last_change":"2024-04-25T10:19:36.755373+0000","last_active":"2024-04-25T10:19:38.768741+0000","last_peered":"2024-04-25T10:19:38.768741+0000","last_clean":"2024-04-25T10:19:38.768741+0000","last_became_active":"2024-04-25T10:19:36.754566+0000","last_became_peered":"2024-04-25T10:19:36.754566+0000","last_unstale":"2024-04-25T10:19:38.768741+0000","last_undegraded":"2024-04-25T10:19:38.768741+0000","last_fullsized":"2024-04-25T10:19:38.768741+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:19.654181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764619+0000","last_change":"2024-04-25T10:19:37.779539+0000","last_active":"2024-04-25T10:19:38.764619+0000","last_peered":"2024-04-25T10:19:38.764619+0000","last_clean":"2024-04-25T10:19:38.764619+0000","last_became_active":"2024-04-25T10:19:37.779252+0000","last_became_peered":"2024-04-25T10:19:37.779252+0000","last_unstale":"2024-04-25T10:19:38.764619+0000","last_undegraded":"2024-04-25T10:19:38.764619+0000","last_fullsized":"2024-04-25T10:19:38.764619+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:44.596174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767904+0000","last_change":"2024-04-25T10:19:36.743832+0000","last_active":"2024-04-25T10:19:38.767904+0000","last_peered":"2024-04-25T10:19:38.767904+0000","last_clean":"2024-04-25T10:19:38.767904+0000","last_became_active":"2024-04-25T10:19:36.743555+0000","last_became_peered":"2024-04-25T10:19:36.743555+0000","last_unstale":"2024-04-25T10:19:38.767904+0000","last_undegraded":"2024-04-25T10:19:38.767904+0000","last_fullsized":"2024-04-25T10:19:38.767904+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:19.678873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764078+0000","last_change":"2024-04-25T10:19:37.776658+0000","last_active":"2024-04-25T10:19:38.764078+0000","last_peered":"2024-04-25T10:19:38.764078+0000","last_clean":"2024-04-25T10:19:38.764078+0000","last_became_active":"2024-04-25T10:19:37.768021+0000","last_became_peered":"2024-04-25T10:19:37.768021+0000","last_unstale":"2024-04-25T10:19:38.764078+0000","last_undegraded":"2024-04-25T10:19:38.764078+0000","last_fullsized":"2024-04-25T10:19:38.764078+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:21.259266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767509+0000","last_change":"2024-04-25T10:19:36.738856+0000","last_active":"2024-04-25T10:19:38.767509+0000","last_peered":"2024-04-25T10:19:38.767509+0000","last_clean":"2024-04-25T10:19:38.767509+0000","last_became_active":"2024-04-25T10:19:36.738521+0000","last_became_peered":"2024-04-25T10:19:36.738521+0000","last_unstale":"2024-04-25T10:19:38.767509+0000","last_undegraded":"2024-04-25T10:19:38.767509+0000","last_fullsized":"2024-04-25T10:19:38.767509+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:25.506297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124724+0000","last_change":"2024-04-25T10:19:37.779235+0000","last_active":"2024-04-25T10:19:40.124724+0000","last_peered":"2024-04-25T10:19:40.124724+0000","last_clean":"2024-04-25T10:19:40.124724+0000","last_became_active":"2024-04-25T10:19:37.778972+0000","last_became_peered":"2024-04-25T10:19:37.778972+0000","last_unstale":"2024-04-25T10:19:40.124724+0000","last_undegraded":"2024-04-25T10:19:40.124724+0000","last_fullsized":"2024-04-25T10:19:40.124724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:15.750695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768655+0000","last_change":"2024-04-25T10:19:36.753235+0000","last_active":"2024-04-25T10:19:38.768655+0000","last_peered":"2024-04-25T10:19:38.768655+0000","last_clean":"2024-04-25T10:19:38.768655+0000","last_became_active":"2024-04-25T10:19:36.752968+0000","last_became_peered":"2024-04-25T10:19:36.752968+0000","last_unstale":"2024-04-25T10:19:38.768655+0000","last_undegraded":"2024-04-25T10:19:38.768655+0000","last_fullsized":"2024-04-25T10:19:38.768655+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:10.876366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769199+0000","last_change":"2024-04-25T10:19:37.772521+0000","last_active":"2024-04-25T10:19:38.769199+0000","last_peered":"2024-04-25T10:19:38.769199+0000","last_clean":"2024-04-25T10:19:38.769199+0000","last_became_active":"2024-04-25T10:19:37.771768+0000","last_became_peered":"2024-04-25T10:19:37.771768+0000","last_unstale":"2024-04-25T10:19:38.769199+0000","last_undegraded":"2024-04-25T10:19:38.769199+0000","last_fullsized":"2024-04-25T10:19:38.769199+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:54.184769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768013+0000","last_change":"2024-04-25T10:19:36.745047+0000","last_active":"2024-04-25T10:19:38.768013+0000","last_peered":"2024-04-25T10:19:38.768013+0000","last_clean":"2024-04-25T10:19:38.768013+0000","last_became_active":"2024-04-25T10:19:36.744128+0000","last_became_peered":"2024-04-25T10:19:36.744128+0000","last_unstale":"2024-04-25T10:19:38.768013+0000","last_undegraded":"2024-04-25T10:19:38.768013+0000","last_fullsized":"2024-04-25T10:19:38.768013+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:55.951430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839785+0000","last_change":"2024-04-25T10:19:37.788920+0000","last_active":"2024-04-25T10:19:38.839785+0000","last_peered":"2024-04-25T10:19:38.839785+0000","last_clean":"2024-04-25T10:19:38.839785+0000","last_became_active":"2024-04-25T10:19:37.788544+0000","last_became_peered":"2024-04-25T10:19:37.788544+0000","last_unstale":"2024-04-25T10:19:38.839785+0000","last_undegraded":"2024-04-25T10:19:38.839785+0000","last_fullsized":"2024-04-25T10:19:38.839785+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.605247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"100'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.769270+0000","last_change":"2024-04-25T10:19:36.758600+0000","last_active":"2024-04-25T10:19:38.769270+0000","last_peered":"2024-04-25T10:19:38.769270+0000","last_clean":"2024-04-25T10:19:38.769270+0000","last_became_active":"2024-04-25T10:19:36.757301+0000","last_became_peered":"2024-04-25T10:19:36.757301+0000","last_unstale":"2024-04-25T10:19:38.769270+0000","last_undegraded":"2024-04-25T10:19:38.769270+0000","last_fullsized":"2024-04-25T10:19:38.769270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.473366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.839573+0000","last_change":"2024-04-25T10:19:37.777741+0000","last_active":"2024-04-25T10:19:38.839573+0000","last_peered":"2024-04-25T10:19:38.839573+0000","last_clean":"2024-04-25T10:19:38.839573+0000","last_became_active":"2024-04-25T10:19:37.777439+0000","last_became_peered":"2024-04-25T10:19:37.777439+0000","last_unstale":"2024-04-25T10:19:38.839573+0000","last_undegraded":"2024-04-25T10:19:38.839573+0000","last_fullsized":"2024-04-25T10:19:38.839573+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:23.824033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768942+0000","last_change":"2024-04-25T10:19:36.758075+0000","last_active":"2024-04-25T10:19:38.768942+0000","last_peered":"2024-04-25T10:19:38.768942+0000","last_clean":"2024-04-25T10:19:38.768942+0000","last_became_active":"2024-04-25T10:19:36.757829+0000","last_became_peered":"2024-04-25T10:19:36.757829+0000","last_unstale":"2024-04-25T10:19:38.768942+0000","last_undegraded":"2024-04-25T10:19:38.768942+0000","last_fullsized":"2024-04-25T10:19:38.768942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:20.728540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125502+0000","last_change":"2024-04-25T10:19:37.786005+0000","last_active":"2024-04-25T10:19:40.125502+0000","last_peered":"2024-04-25T10:19:40.125502+0000","last_clean":"2024-04-25T10:19:40.125502+0000","last_became_active":"2024-04-25T10:19:37.785695+0000","last_became_peered":"2024-04-25T10:19:37.785695+0000","last_unstale":"2024-04-25T10:19:40.125502+0000","last_undegraded":"2024-04-25T10:19:40.125502+0000","last_fullsized":"2024-04-25T10:19:40.125502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:48.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125116+0000","last_change":"2024-04-25T10:19:36.747869+0000","last_active":"2024-04-25T10:19:40.125116+0000","last_peered":"2024-04-25T10:19:40.125116+0000","last_clean":"2024-04-25T10:19:40.125116+0000","last_became_active":"2024-04-25T10:19:36.747566+0000","last_became_peered":"2024-04-25T10:19:36.747566+0000","last_unstale":"2024-04-25T10:19:40.125116+0000","last_undegraded":"2024-04-25T10:19:40.125116+0000","last_fullsized":"2024-04-25T10:19:40.125116+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:12.858880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764504+0000","last_change":"2024-04-25T10:19:37.774497+0000","last_active":"2024-04-25T10:19:38.764504+0000","last_peered":"2024-04-25T10:19:38.764504+0000","last_clean":"2024-04-25T10:19:38.764504+0000","last_became_active":"2024-04-25T10:19:37.770481+0000","last_became_peered":"2024-04-25T10:19:37.770481+0000","last_unstale":"2024-04-25T10:19:38.764504+0000","last_undegraded":"2024-04-25T10:19:38.764504+0000","last_fullsized":"2024-04-25T10:19:38.764504+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:29.821562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.768859+0000","last_change":"2024-04-25T10:19:36.755590+0000","last_active":"2024-04-25T10:19:38.768859+0000","last_peered":"2024-04-25T10:19:38.768859+0000","last_clean":"2024-04-25T10:19:38.768859+0000","last_became_active":"2024-04-25T10:19:36.754895+0000","last_became_peered":"2024-04-25T10:19:36.754895+0000","last_unstale":"2024-04-25T10:19:38.768859+0000","last_undegraded":"2024-04-25T10:19:38.768859+0000","last_fullsized":"2024-04-25T10:19:38.768859+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:28.718247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.767805+0000","last_change":"2024-04-25T10:19:37.766788+0000","last_active":"2024-04-25T10:19:38.767805+0000","last_peered":"2024-04-25T10:19:38.767805+0000","last_clean":"2024-04-25T10:19:38.767805+0000","last_became_active":"2024-04-25T10:19:37.766484+0000","last_became_peered":"2024-04-25T10:19:37.766484+0000","last_unstale":"2024-04-25T10:19:38.767805+0000","last_undegraded":"2024-04-25T10:19:38.767805+0000","last_fullsized":"2024-04-25T10:19:38.767805+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:55.149429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124739+0000","last_change":"2024-04-25T10:19:36.747810+0000","last_active":"2024-04-25T10:19:40.124739+0000","last_peered":"2024-04-25T10:19:40.124739+0000","last_clean":"2024-04-25T10:19:40.124739+0000","last_became_active":"2024-04-25T10:19:36.747502+0000","last_became_peered":"2024-04-25T10:19:36.747502+0000","last_unstale":"2024-04-25T10:19:40.124739+0000","last_undegraded":"2024-04-25T10:19:40.124739+0000","last_fullsized":"2024-04-25T10:19:40.124739+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:02.832069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764077+0000","last_change":"2024-04-25T10:19:37.771355+0000","last_active":"2024-04-25T10:19:38.764077+0000","last_peered":"2024-04-25T10:19:38.764077+0000","last_clean":"2024-04-25T10:19:38.764077+0000","last_became_active":"2024-04-25T10:19:37.771035+0000","last_became_peered":"2024-04-25T10:19:37.771035+0000","last_unstale":"2024-04-25T10:19:38.764077+0000","last_undegraded":"2024-04-25T10:19:38.764077+0000","last_fullsized":"2024-04-25T10:19:38.764077+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:57.414732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.958013+0000","last_change":"2024-04-25T10:19:36.744987+0000","last_active":"2024-04-25T10:19:38.958013+0000","last_peered":"2024-04-25T10:19:38.958013+0000","last_clean":"2024-04-25T10:19:38.958013+0000","last_became_active":"2024-04-25T10:19:36.744728+0000","last_became_peered":"2024-04-25T10:19:36.744728+0000","last_unstale":"2024-04-25T10:19:38.958013+0000","last_undegraded":"2024-04-25T10:19:38.958013+0000","last_fullsized":"2024-04-25T10:19:38.958013+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:16.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.124990+0000","last_change":"2024-04-25T10:19:37.785995+0000","last_active":"2024-04-25T10:19:40.124990+0000","last_peered":"2024-04-25T10:19:40.124990+0000","last_clean":"2024-04-25T10:19:40.124990+0000","last_became_active":"2024-04-25T10:19:37.785705+0000","last_became_peered":"2024-04-25T10:19:37.785705+0000","last_unstale":"2024-04-25T10:19:40.124990+0000","last_undegraded":"2024-04-25T10:19:40.124990+0000","last_fullsized":"2024-04-25T10:19:40.124990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:55.149429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764112+0000","last_change":"2024-04-25T10:19:36.759979+0000","last_active":"2024-04-25T10:19:38.764112+0000","last_peered":"2024-04-25T10:19:38.764112+0000","last_clean":"2024-04-25T10:19:38.764112+0000","last_became_active":"2024-04-25T10:19:36.759706+0000","last_became_peered":"2024-04-25T10:19:36.759706+0000","last_unstale":"2024-04-25T10:19:38.764112+0000","last_undegraded":"2024-04-25T10:19:38.764112+0000","last_fullsized":"2024-04-25T10:19:38.764112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:06.815334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:38.764916+0000","last_change":"2024-04-25T10:19:37.790840+0000","last_active":"2024-04-25T10:19:38.764916+0000","last_peered":"2024-04-25T10:19:38.764916+0000","last_clean":"2024-04-25T10:19:38.764916+0000","last_became_active":"2024-04-25T10:19:37.785918+0000","last_became_peered":"2024-04-25T10:19:37.785918+0000","last_unstale":"2024-04-25T10:19:38.764916+0000","last_undegraded":"2024-04-25T10:19:38.764916+0000","last_fullsized":"2024-04-25T10:19:38.764916+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:36.726514+0000","last_clean_scrub_stamp":"2024-04-25T10:19:36.726514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:44.716221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125108+0000","last_change":"2024-04-25T10:19:36.758326+0000","last_active":"2024-04-25T10:19:40.125108+0000","last_peered":"2024-04-25T10:19:40.125108+0000","last_clean":"2024-04-25T10:19:40.125108+0000","last_became_active":"2024-04-25T10:19:36.758021+0000","last_became_peered":"2024-04-25T10:19:36.758021+0000","last_unstale":"2024-04-25T10:19:40.125108+0000","last_undegraded":"2024-04-25T10:19:40.125108+0000","last_fullsized":"2024-04-25T10:19:40.125108+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:19:35.720477+0000","last_clean_scrub_stamp":"2024-04-25T10:19:35.720477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:27.647159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":314,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:19:40.125204+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:19:40.125204+0000","last_peered":"2024-04-25T10:19:40.125204+0000","last_clean":"2024-04-25T10:19:40.125204+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:19:40.125204+0000","last_undegraded":"2024-04-25T10:19:40.125204+0000","last_fullsized":"2024-04-25T10:19:40.125204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51024,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320816,"statfs":{"total":96636764160,"available":96584515584,"internally_reserved":0,"allocated":1839104,"data_stored":1126262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54604,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317236,"statfs":{"total":96636764160,"available":96580849664,"internally_reserved":0,"allocated":1245184,"data_stored":536434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1241088,"data_stored":535852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54624,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317216,"statfs":{"total":96636764160,"available":96580829184,"internally_reserved":0,"allocated":1257472,"data_stored":537518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50460,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321380,"statfs":{"total":96636764160,"available":96585093120,"internally_reserved":0,"allocated":1253376,"data_stored":537416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46288,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325552,"statfs":{"total":96636764160,"available":96589365248,"internally_reserved":0,"allocated":1241088,"data_stored":535816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76804,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295036,"statfs":{"total":96636764160,"available":96558116864,"internally_reserved":0,"allocated":1236992,"data_stored":535782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51028,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320812,"statfs":{"total":96636764160,"available":96584511488,"internally_reserved":0,"allocated":1835008,"data_stored":1126732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:46.694 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:19:46.694 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:19:46.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.872+0000 7f89fbc00640 1 Processor -- start 2024-04-25T10:19:46.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.872+0000 7f89fbc00640 1 -- start start 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.872+0000 7f89fbc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f89f415bbb0 0x7f89f40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.872+0000 7f89fbc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89f40b4660 0x7f89f406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.872+0000 7f89fbc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f406ca90 con 0x7f89f40cf960 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.872+0000 7f89fbc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f406d050 con 0x7f89f40b4660 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.872+0000 7f89fbc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f406cd70 con 0x7f89f415bbb0 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89f40b4660 0x7f89f406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f9c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f89f40cf960 0x7f89f40b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39920/0 (socket says 172.21.15.87:39920) 2024-04-25T10:19:46.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f9c00640 1 -- 172.21.15.87:0/4036163233 learned_addr learned my addr 172.21.15.87:0/4036163233 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:46.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f9200640 1 --2- 172.21.15.87:0/4036163233 >> v2:172.21.15.169:3300/0 conn(0x7f89f415bbb0 0x7f89f40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3e00640 1 -- 172.21.15.87:0/4036163233 >> v2:172.21.15.169:3300/0 conn(0x7f89f415bbb0 msgr2=0x7f89f40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3e00640 1 --2- 172.21.15.87:0/4036163233 >> v2:172.21.15.169:3300/0 conn(0x7f89f415bbb0 0x7f89f40cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3e00640 1 -- 172.21.15.87:0/4036163233 >> v1:172.21.15.87:6789/0 conn(0x7f89f40cf960 legacy=0x7f89f40b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3400640 1 -- 172.21.15.87:0/4036163233 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621544239 0 0) 0x7f89f406ca90 con 0x7f89f40cf960 2024-04-25T10:19:46.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3e00640 1 -- 172.21.15.87:0/4036163233 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f40b2880 con 0x7f89f40b4660 2024-04-25T10:19:46.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3e00640 1 --2- 172.21.15.87:0/4036163233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f40b4660 0x7f89f406c120 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f89f406e8b0 tx=0x7f89e001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af93466f1db90bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3400640 1 -- 172.21.15.87:0/4036163233 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00b0070 con 0x7f89f40b4660 2024-04-25T10:19:46.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3400640 1 -- 172.21.15.87:0/4036163233 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e00ab600 con 0x7f89f40b4660 2024-04-25T10:19:46.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.876+0000 7f89f3400640 1 -- 172.21.15.87:0/4036163233 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00ab9d0 con 0x7f89f40b4660 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- 172.21.15.87:0/4036163233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f40b4660 msgr2=0x7f89f406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 --2- 172.21.15.87:0/4036163233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f40b4660 0x7f89f406c120 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f89f406e8b0 tx=0x7f89e001c480 comp rx=0 tx=0).stop 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- 172.21.15.87:0/4036163233 shutdown_connections 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 --2- 172.21.15.87:0/4036163233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f40b4660 0x7f89f406c120 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 --2- 172.21.15.87:0/4036163233 >> v2:172.21.15.169:3300/0 conn(0x7f89f415bbb0 0x7f89f40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- 172.21.15.87:0/4036163233 >> 172.21.15.87:0/4036163233 conn(0x7f89f4066b70 msgr2=0x7f89f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- 172.21.15.87:0/4036163233 shutdown_connections 2024-04-25T10:19:46.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- 172.21.15.87:0/4036163233 wait complete. 2024-04-25T10:19:46.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 Processor -- start 2024-04-25T10:19:46.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- start start 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f40b4660 0x7f89f413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f40cf960 0x7f89f414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f415bbb0 0x7f89f413a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40ce140 con 0x7f89f415bbb0 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40cdfc0 con 0x7f89f40b4660 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89fbc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f406c660 con 0x7f89f40cf960 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f415bbb0 0x7f89f413a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f415bbb0 0x7f89f413a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54744/0 (socket says 172.21.15.87:54744) 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 -- 172.21.15.87:0/3942556653 learned_addr learned my addr 172.21.15.87:0/3942556653 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 -- 172.21.15.87:0/3942556653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f40cf960 msgr2=0x7f89f414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f40cf960 0x7f89f414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 -- 172.21.15.87:0/3942556653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f40b4660 msgr2=0x7f89f413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f40b4660 0x7f89f413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 -- 172.21.15.87:0/3942556653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4150650 con 0x7f89f415bbb0 2024-04-25T10:19:46.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f9c00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f415bbb0 0x7f89f413a2f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c410 tx=0x7f89ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f0c00640 1 -- 172.21.15.87:0/3942556653 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0074e0 con 0x7f89f415bbb0 2024-04-25T10:19:46.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.880+0000 7f89f0c00640 1 -- 172.21.15.87:0/3942556653 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec0047f0 con 0x7f89f415bbb0 2024-04-25T10:19:46.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f4151d60 con 0x7f89f415bbb0 2024-04-25T10:19:46.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89f0c00640 1 -- 172.21.15.87:0/3942556653 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec007680 con 0x7f89f415bbb0 2024-04-25T10:19:46.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f4152320 con 0x7f89f415bbb0 2024-04-25T10:19:46.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f404ef30 con 0x7f89f415bbb0 2024-04-25T10:19:46.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89f0c00640 1 -- 172.21.15.87:0/3942556653 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89ec011070 con 0x7f89f415bbb0 2024-04-25T10:19:46.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89f0c00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d4066f90 0x7f89d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89f0c00640 1 -- 172.21.15.87:0/3942556653 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89ec084bb0 con 0x7f89f415bbb0 2024-04-25T10:19:46.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89f9200640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d4066f90 0x7f89d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.884+0000 7f89f9200640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d4066f90 0x7f89d4069450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89e400e190 tx=0x7f89e400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:46.888+0000 7f89f0c00640 1 -- 172.21.15.87:0/3942556653 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89ec055bb0 con 0x7f89f415bbb0 2024-04-25T10:19:47.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.096+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f89f4166730 con 0x7f89f415bbb0 2024-04-25T10:19:47.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89f0c00640 1 -- 172.21.15.87:0/3942556653 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f89ec0554b0 con 0x7f89f415bbb0 2024-04-25T10:19:47.101 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:19:47.101 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:19:47.101 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:19:47.101 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:19:47.101 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:19:47.102 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:19:47.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d4066f90 msgr2=0x7f89d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d4066f90 0x7f89d4069450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89e400e190 tx=0x7f89e400b040 comp rx=0 tx=0).stop 2024-04-25T10:19:47.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f415bbb0 msgr2=0x7f89f413a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f415bbb0 0x7f89f413a2f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c410 tx=0x7f89ec00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 shutdown_connections 2024-04-25T10:19:47.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d4066f90 0x7f89d4069450 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f415bbb0 0x7f89f413a2f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f40cf960 0x7f89f414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 --2- 172.21.15.87:0/3942556653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f40b4660 0x7f89f413cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 >> 172.21.15.87:0/3942556653 conn(0x7f89f4066b70 msgr2=0x7f89f40b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 shutdown_connections 2024-04-25T10:19:47.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.100+0000 7f89fbc00640 1 -- 172.21.15.87:0/3942556653 wait complete. 2024-04-25T10:19:47.122 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:47.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 Processor -- start 2024-04-25T10:19:47.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 -- start start 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9bcc0d1820 0x7f9bcc0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc15e050 0x7f9bcc0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9bcc0b13e0 0x7f9bcc0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b1870 con 0x7f9bcc0b13e0 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b1af0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bd1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b1dd0 con 0x7f9bcc15e050 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9bcc0b13e0 0x7f9bcc0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9bcc0b13e0 0x7f9bcc0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54754/0 (socket says 172.21.15.87:54754) 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 -- 172.21.15.87:0/1260663514 learned_addr learned my addr 172.21.15.87:0/1260663514 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:47.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bc2000640 1 --2- 172.21.15.87:0/1260663514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc15e050 0x7f9bcc0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcaa00640 1 --2- 172.21.15.87:0/1260663514 >> v2:172.21.15.169:3300/0 conn(0x7f9bcc0d1820 0x7f9bcc0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 -- 172.21.15.87:0/1260663514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc15e050 msgr2=0x7f9bcc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 --2- 172.21.15.87:0/1260663514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc15e050 0x7f9bcc0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 -- 172.21.15.87:0/1260663514 >> v2:172.21.15.169:3300/0 conn(0x7f9bcc0d1820 msgr2=0x7f9bcc0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 --2- 172.21.15.87:0/1260663514 >> v2:172.21.15.169:3300/0 conn(0x7f9bcc0d1820 0x7f9bcc0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.256+0000 7f9bcb400640 1 -- 172.21.15.87:0/1260663514 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0b20b0 con 0x7f9bcc0b13e0 2024-04-25T10:19:47.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bcb400640 1 --2- 172.21.15.87:0/1260663514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0b13e0 0x7f9bcc0ce180 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc070e90 tx=0x7f9bc401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81c6259be8b460f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bca000640 1 -- 172.21.15.87:0/1260663514 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40b0070 con 0x7f9bcc0b13e0 2024-04-25T10:19:47.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bca000640 1 -- 172.21.15.87:0/1260663514 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc40ab600 con 0x7f9bcc0b13e0 2024-04-25T10:19:47.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bca000640 1 -- 172.21.15.87:0/1260663514 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40ab9d0 con 0x7f9bcc0b13e0 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1260663514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0b13e0 msgr2=0x7f9bcc0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1260663514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0b13e0 0x7f9bcc0ce180 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc070e90 tx=0x7f9bc401c260 comp rx=0 tx=0).stop 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1260663514 shutdown_connections 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1260663514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0b13e0 0x7f9bcc0ce180 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1260663514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc15e050 0x7f9bcc0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1260663514 >> v2:172.21.15.169:3300/0 conn(0x7f9bcc0d1820 0x7f9bcc0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1260663514 >> 172.21.15.87:0/1260663514 conn(0x7f9bcc09a9d0 msgr2=0x7f9bcc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:47.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1260663514 shutdown_connections 2024-04-25T10:19:47.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.260+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1260663514 wait complete. 2024-04-25T10:19:47.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 Processor -- start 2024-04-25T10:19:47.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 -- start start 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc0b13e0 0x7f9bcc151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0d1820 0x7f9bcc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc15e050 0x7f9bcc0b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0cfdf0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0cfc70 con 0x7f9bcc15e050 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0cff70 con 0x7f9bcc0b13e0 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0d1820 0x7f9bcc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0d1820 0x7f9bcc0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54770/0 (socket says 172.21.15.87:54770) 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 -- 172.21.15.87:0/1927293930 learned_addr learned my addr 172.21.15.87:0/1927293930 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bcaa00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc0b13e0 0x7f9bcc151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bcb400640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc15e050 0x7f9bcc0b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 -- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc0b13e0 msgr2=0x7f9bcc151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc0b13e0 0x7f9bcc151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 -- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc15e050 msgr2=0x7f9bcc0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc15e050 0x7f9bcc0b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 -- 172.21.15.87:0/1927293930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0b5bd0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc2000640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0d1820 0x7f9bcc0b5050 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400d5d0 tx=0x7f9bb400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc3400640 1 -- 172.21.15.87:0/1927293930 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4004240 con 0x7f9bcc0d1820 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc3400640 1 -- 172.21.15.87:0/1927293930 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb40043e0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc0b5eb0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bc3400640 1 -- 172.21.15.87:0/1927293930 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4003840 con 0x7f9bcc0d1820 2024-04-25T10:19:47.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.264+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc0b6750 con 0x7f9bcc0d1820 2024-04-25T10:19:47.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.268+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc04ef30 con 0x7f9bcc0d1820 2024-04-25T10:19:47.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.268+0000 7f9bc3400640 1 -- 172.21.15.87:0/1927293930 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bb40039e0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.268+0000 7f9bc3400640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9ba8066f60 0x7f9ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.268+0000 7f9bc3400640 1 -- 172.21.15.87:0/1927293930 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bb4085200 con 0x7f9bcc0d1820 2024-04-25T10:19:47.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.268+0000 7f9bcaa00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9ba8066f60 0x7f9ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.268+0000 7f9bcaa00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9ba8066f60 0x7f9ba8069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc003b90 tx=0x7f9bbc0098d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.272+0000 7f9bc3400640 1 -- 172.21.15.87:0/1927293930 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bb4055b20 con 0x7f9bcc0d1820 2024-04-25T10:19:47.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.456+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bcc15d4e0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.456+0000 7f9bc3400640 1 -- 172.21.15.87:0/1927293930 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9bb4059bc0 con 0x7f9bcc0d1820 2024-04-25T10:19:47.460 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:19:47.460 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:19:47.460 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9ba8066f60 msgr2=0x7f9ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9ba8066f60 0x7f9ba8069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc003b90 tx=0x7f9bbc0098d0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0d1820 msgr2=0x7f9bcc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0d1820 0x7f9bcc0b5050 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400d5d0 tx=0x7f9bb400da90 comp rx=0 tx=0).stop 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 shutdown_connections 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9ba8066f60 0x7f9ba8069420 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc15e050 0x7f9bcc0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bcc0d1820 0x7f9bcc0b5050 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 --2- 172.21.15.87:0/1927293930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bcc0b13e0 0x7f9bcc151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 >> 172.21.15.87:0/1927293930 conn(0x7f9bcc09a9d0 msgr2=0x7f9bcc0ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:47.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 shutdown_connections 2024-04-25T10:19:47.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:47.460+0000 7f9bd1a00640 1 -- 172.21.15.87:0/1927293930 wait complete. 2024-04-25T10:19:47.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:19:37.118679+0000', 'modified': '2024-04-25T10:19:40.136830+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7947}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7947': {'gid': 7947, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6836/1167245113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 1167245113}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 1167245113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:19:47.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:19:47.883 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:19:48.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.016+0000 7f5963e00640 1 Processor -- start 2024-04-25T10:19:48.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.016+0000 7f5963e00640 1 -- start start 2024-04-25T10:19:48.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f59640cf9b0 0x7f596415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f59641540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640b5200 con 0x7f59640b3360 2024-04-25T10:19:48.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596415ef10 con 0x7f59640cf9b0 2024-04-25T10:19:48.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640b38b0 con 0x7f5964071120 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5962a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f59640cf9b0 0x7f596415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f59641540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f59641540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:32768/0 (socket says 172.21.15.87:32768) 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f595a000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f59640b3360 0x7f5964070a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:52314/0 (socket says 172.21.15.87:52314) 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963400640 1 -- 172.21.15.87:0/3727899561 learned_addr learned my addr 172.21.15.87:0/3727899561 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5962a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f59640cf9b0 0x7f596415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:55658/0 (socket says 172.21.15.87:55658) 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5962a00640 1 -- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 msgr2=0x7f59641540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5962a00640 1 --2- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f59641540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5962a00640 1 -- 172.21.15.87:0/3727899561 >> v1:172.21.15.169:6789/0 conn(0x7f59640b3360 legacy=0x7f5964070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5962a00640 1 -- 172.21.15.87:0/3727899561 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640b3b90 con 0x7f59640cf9b0 2024-04-25T10:19:48.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5962000640 1 -- 172.21.15.87:0/3727899561 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739631255 0 0) 0x7f59640b5200 con 0x7f59640b3360 2024-04-25T10:19:48.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.020+0000 7f5963400640 1 --2- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f59641540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:48.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5962a00640 1 --2- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 0x7f596415e9d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5954009d90 tx=0x7f595401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e03acdbce9220f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5962000640 1 -- 172.21.15.87:0/3727899561 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540b2070 con 0x7f59640cf9b0 2024-04-25T10:19:48.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5962000640 1 -- 172.21.15.87:0/3727899561 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540a5940 con 0x7f59640cf9b0 2024-04-25T10:19:48.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5962000640 1 -- 172.21.15.87:0/3727899561 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540a5d10 con 0x7f59640cf9b0 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 -- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 msgr2=0x7f596415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 --2- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 0x7f596415e9d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5954009d90 tx=0x7f595401c970 comp rx=0 tx=0).stop 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 -- 172.21.15.87:0/3727899561 shutdown_connections 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 --2- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f59641540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 --2- 172.21.15.87:0/3727899561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 0x7f596415e9d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 -- 172.21.15.87:0/3727899561 >> 172.21.15.87:0/3727899561 conn(0x7f59640cfe80 msgr2=0x7f59640d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 -- 172.21.15.87:0/3727899561 shutdown_connections 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 -- 172.21.15.87:0/3727899561 wait complete. 2024-04-25T10:19:48.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 Processor -- start 2024-04-25T10:19:48.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.024+0000 7f5963e00640 1 -- start start 2024-04-25T10:19:48.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f596414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59640b3360 0x7f59640b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 0x7f596406e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640adf70 con 0x7f59640b3360 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640addf0 con 0x7f59640cf9b0 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640ae0f0 con 0x7f5964071120 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59640b3360 0x7f59640b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59640b3360 0x7f59640b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48402/0 (socket says 172.21.15.87:48402) 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 -- 172.21.15.87:0/3507510727 learned_addr learned my addr 172.21.15.87:0/3507510727 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963400640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 0x7f596406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5962a00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f596414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 -- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 msgr2=0x7f596414b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f596414b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 -- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 msgr2=0x7f596406e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 0x7f596406e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 -- 172.21.15.87:0/3507510727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640b0d30 con 0x7f59640b3360 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595a000640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59640b3360 0x7f59640b0430 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f594c0011f0 tx=0x7f594c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595b400640 1 -- 172.21.15.87:0/3507510727 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c018070 con 0x7f59640b3360 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595b400640 1 -- 172.21.15.87:0/3507510727 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c0074e0 con 0x7f59640b3360 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f595b400640 1 -- 172.21.15.87:0/3507510727 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c004a10 con 0x7f59640b3360 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640b1010 con 0x7f59640b3360 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.028+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596414bee0 con 0x7f59640b3360 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595b400640 1 -- 172.21.15.87:0/3507510727 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f594c004bb0 con 0x7f59640b3360 2024-04-25T10:19:48.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595b400640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5940066f60 0x7f5940069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595b400640 1 -- 172.21.15.87:0/3507510727 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f594c084410 con 0x7f59640b3360 2024-04-25T10:19:48.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f5962a00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5940066f60 0x7f5940069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 Processor -- start 2024-04-25T10:19:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 -- start start 2024-04-25T10:19:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f5962a00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5940066f60 0x7f5940069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f59540af000 tx=0x7f59540a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d665920 0x561b3d665d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d61b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d61c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b3d5035f0 con 0x561b3d666240 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b3d46ada0 con 0x561b3d61bf30 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b3d3f3150 con 0x561b3d665920 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d61b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d61b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48410/0 (socket says 172.21.15.87:48410) 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 -- 172.21.15.87:0/1853586544 learned_addr learned my addr 172.21.15.87:0/1853586544 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f5962a00640 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d665920 0x561b3d665d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f5963400640 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d61c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 -- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d665920 msgr2=0x561b3d665d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d665920 0x561b3d665d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 -- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 msgr2=0x561b3d61c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d61c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 -- 172.21.15.87:0/1853586544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b3d40cca0 con 0x561b3d666240 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f595a000640 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d61b9f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f594c001130 tx=0x7f594c054770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8785db2fabdd00c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f5953400640 1 -- 172.21.15.87:0/1853586544 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c05d420 con 0x561b3d666240 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f5953400640 1 -- 172.21.15.87:0/1853586544 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c052db0 con 0x561b3d666240 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f5953400640 1 -- 172.21.15.87:0/1853586544 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0840c0 con 0x561b3d666240 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 -- 172.21.15.87:0/1853586544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 msgr2=0x561b3d61b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.032+0000 7f596cacc000 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d61b9f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f594c001130 tx=0x7f594c054770 comp rx=0 tx=0).stop 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- 172.21.15.87:0/1853586544 shutdown_connections 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d61c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d61b9f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 --2- 172.21.15.87:0/1853586544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d665920 0x561b3d665d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- 172.21.15.87:0/1853586544 >> 172.21.15.87:0/1853586544 conn(0x561b3d6222a0 msgr2=0x561b3d61a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- 172.21.15.87:0/1853586544 shutdown_connections 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- 172.21.15.87:0/1853586544 wait complete. 2024-04-25T10:19:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- start start 2024-04-25T10:19:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 Processor -- start 2024-04-25T10:19:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d66ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d665920 0x561b3d66d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d61faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b3d425990 con 0x561b3d665920 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b3d466e40 con 0x561b3d61bf30 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b3d3f6900 con 0x561b3d666240 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 10 client.0 _ll_register_callbacks cb 0x561b3d51e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d665920 0x561b3d66d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d665920 0x561b3d66d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48424/0 (socket says 172.21.15.87:48424) 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 -- 172.21.15.87:0/3234670612 learned_addr learned my addr 172.21.15.87:0/3234670612 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5962a00640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d66ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5963400640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d61faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 -- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 msgr2=0x561b3d61faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d61faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 -- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 msgr2=0x561b3d66ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d66ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 -- 172.21.15.87:0/3234670612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b3d423630 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5963400640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d61faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f595a000640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d665920 0x561b3d66d380 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f594c05cef0 tx=0x7f594c057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321b37b3e95fecba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5947e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5947e00640 1 -- 172.21.15.87:0/3234670612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c054160 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5947e00640 1 -- 172.21.15.87:0/3234670612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f594c054300 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f594c052db0 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5947e00640 1 -- 172.21.15.87:0/3234670612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f594c050a60 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 10 client.8118 fetch_fsmap learned FSMap version 71 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x561b3d3f33a0 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5947e00640 1 -- 172.21.15.87:0/3234670612 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c052db0 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f5947e00640 1 -- 172.21.15.87:0/3234670612 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f594c050c00 con 0x561b3d665920 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 10 client.8118 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 10 client.8118 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:19:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.036+0000 7f596cacc000 20 client.8118 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621160 0x561b3d676ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 4 client.8118 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b3d406a50 con 0x561b3d621160 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f5962a00640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621160 0x561b3d676ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f5962a00640 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621160 0x561b3d676ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ee1b1470b2cab0 server_cookie=c53345d13d8f7c2e in_seq=0 out_seq=0 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f5947e00640 10 client.8118 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f5947e00640 1 -- 172.21.15.87:0/3234670612 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b3d406a50 con 0x561b3d621160 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f5947e00640 10 client.8118 handle_command_reply: tid=0 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 client.8118 shutdown 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d665920 msgr2=0x561b3d66d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d665920 0x561b3d66d380 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f594c05cef0 tx=0x7f594c057040 comp rx=0 tx=0).stop 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 shutdown_connections 2024-04-25T10:19:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621160 0x561b3d676ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d61faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d665920 0x561b3d66d380 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f5947e00640 0 client.8118 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 --2- 172.21.15.87:0/3234670612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d61bf30 0x561b3d66ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 >> 172.21.15.87:0/3234670612 conn(0x561b3d6222a0 msgr2=0x561b3d61a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 shutdown_connections 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 1 -- 172.21.15.87:0/3234670612 wait complete. 2024-04-25T10:19:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.040+0000 7f596cacc000 20 client.8118 trim_cache size 0 max 16384 2024-04-25T10:19:48.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 Processor -- start 2024-04-25T10:19:48.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- start start 2024-04-25T10:19:48.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d61bf30 0x561b3d6c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d6c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d6c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5964202080 con 0x561b3d61bf30 2024-04-25T10:19:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596415aa10 con 0x561b3d665920 2024-04-25T10:19:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596415b010 con 0x561b3d666240 2024-04-25T10:19:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d61bf30 0x561b3d6c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d61bf30 0x561b3d6c0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48434/0 (socket says 172.21.15.87:48434) 2024-04-25T10:19:48.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 -- 172.21.15.87:0/4145121762 learned_addr learned my addr 172.21.15.87:0/4145121762 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f595a000640 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d6c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5963400640 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d6c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 -- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 msgr2=0x561b3d6c11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d6c11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 -- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 msgr2=0x561b3d6c0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d6c0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 -- 172.21.15.87:0/4145121762 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b3d666070 con 0x561b3d61bf30 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f595a000640 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d6c0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:48.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5963400640 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d6c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5962a00640 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d61bf30 0x561b3d6c0760 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f595401a5a0 tx=0x7f59540a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab621fde40ae8d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5952a00640 1 -- 172.21.15.87:0/4145121762 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540b2040 con 0x561b3d61bf30 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5952a00640 1 -- 172.21.15.87:0/4145121762 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540c8110 con 0x561b3d61bf30 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f5952a00640 1 -- 172.21.15.87:0/4145121762 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540c84c0 con 0x561b3d61bf30 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- 172.21.15.87:0/4145121762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d61bf30 msgr2=0x561b3d6c0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d61bf30 0x561b3d6c0760 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f595401a5a0 tx=0x7f59540a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- 172.21.15.87:0/4145121762 shutdown_connections 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d666240 0x561b3d6c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d6c0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 --2- 172.21.15.87:0/4145121762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d61bf30 0x561b3d6c0760 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- 172.21.15.87:0/4145121762 >> 172.21.15.87:0/4145121762 conn(0x561b3d6222a0 msgr2=0x561b3d61f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- 172.21.15.87:0/4145121762 shutdown_connections 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.056+0000 7f596cacc000 1 -- 172.21.15.87:0/4145121762 wait complete. 2024-04-25T10:19:48.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 -- start start 2024-04-25T10:19:48.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 Processor -- start 2024-04-25T10:19:48.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d61bf30 0x561b3d6ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d672260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d6727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b3d46f3d0 con 0x561b3d666240 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b3d526d50 con 0x561b3d665920 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b3d405980 con 0x561b3d61bf30 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 10 client.0 _ll_register_callbacks cb 0x7f594c007690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d6727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d6727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48436/0 (socket says 172.21.15.87:48436) 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 -- 172.21.15.87:0/4218816075 learned_addr learned my addr 172.21.15.87:0/4218816075 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f595a000640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d672260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5962a00640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d61bf30 0x561b3d6ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 -- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d61bf30 msgr2=0x561b3d6ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d61bf30 0x561b3d6ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 -- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 msgr2=0x561b3d672260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d672260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 -- 172.21.15.87:0/4218816075 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b3d4ec600 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5962a00640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d61bf30 0x561b3d6ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5963400640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d6727a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f595c009f70 tx=0x7f595c016800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b05e65fd4b27ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5947e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5947e00640 1 -- 172.21.15.87:0/4218816075 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c018860 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5947e00640 1 -- 172.21.15.87:0/4218816075 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f595c022b30 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59540c8110 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5947e00640 1 -- 172.21.15.87:0/4218816075 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f595c022d30 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 10 client.8124 fetch_fsmap learned FSMap version 71 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x561b3d3ff9b0 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5947e00640 1 -- 172.21.15.87:0/4218816075 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540c8110 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5947e00640 1 -- 172.21.15.87:0/4218816075 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f595c015040 con 0x561b3d666240 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 10 client.8124 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:19:48.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 10 client.8124 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 20 client.8124 populate_metadata read hostname 'smithi087' 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621e80 0x561b3d6c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 4 client.8124 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561b3d4782e0 con 0x561b3d621e80 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.060+0000 7f5962a00640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621e80 0x561b3d6c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5962a00640 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621e80 0x561b3d6c2460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa6642c8b5fd461 server_cookie=9c5d685bff69aa87 in_seq=0 out_seq=0 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5947e00640 10 client.8124 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5947e00640 1 -- 172.21.15.87:0/4218816075 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x561b3d4782e0 con 0x561b3d621e80 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5947e00640 10 client.8124 handle_command_reply: tid=0 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 client.8124 shutdown 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 msgr2=0x561b3d6727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d6727a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f595c009f70 tx=0x7f595c016800 comp rx=0 tx=0).stop 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 shutdown_connections 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x561b3d621e80 0x561b3d6c2460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561b3d666240 0x561b3d6727a0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561b3d665920 0x561b3d672260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 --2- 172.21.15.87:0/4218816075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561b3d61bf30 0x561b3d6ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5947e00640 0 client.8124 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 >> 172.21.15.87:0/4218816075 conn(0x561b3d6222a0 msgr2=0x561b3d66d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 shutdown_connections 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 1 -- 172.21.15.87:0/4218816075 wait complete. 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f596cacc000 20 client.8124 trim_cache size 0 max 16384 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:19:48.070 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 7947, 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 10.308516721, 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 71, 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 100, 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 100, 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 15.197104691 2024-04-25T10:19:48.071 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5940066f60 msgr2=0x7f5940069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5940066f60 0x7f5940069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f59540af000 tx=0x7f59540a7070 comp rx=0 tx=0).stop 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59640b3360 msgr2=0x7f59640b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59640b3360 0x7f59640b0430 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f594c0011f0 tx=0x7f594c00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 shutdown_connections 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5940066f60 0x7f5940069420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59640cf9b0 0x7f596406e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59640b3360 0x7f59640b0430 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 --2- 172.21.15.87:0/3507510727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5964071120 0x7f596414b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 >> 172.21.15.87:0/3507510727 conn(0x7f59640cfe80 msgr2=0x7f5964161b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 shutdown_connections 2024-04-25T10:19:48.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.064+0000 7f5963e00640 1 -- 172.21.15.87:0/3507510727 wait complete. 2024-04-25T10:19:48.085 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:19:48.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067e600640 1 Processor -- start 2024-04-25T10:19:48.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067e600640 1 -- start start 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067e600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0678161270 0x7f06780cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f067806f100 0x7f06780cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067e600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780d0a10 con 0x7f06780b0a90 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067e600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780d1550 con 0x7f0678161270 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780d0cb0 con 0x7f067806f100 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0678161270 0x7f06780cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f0677e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f06780b0a90 0x7f067806e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52680/0 (socket says 172.21.15.87:52680) 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f0677e00640 1 -- 172.21.15.87:0/2104330285 learned_addr learned my addr 172.21.15.87:0/2104330285 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067dc00640 1 --2- 172.21.15.87:0/2104330285 >> v2:172.21.15.169:3300/0 conn(0x7f067806f100 0x7f06780cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067d200640 1 -- 172.21.15.87:0/2104330285 >> v2:172.21.15.169:3300/0 conn(0x7f067806f100 msgr2=0x7f06780cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067d200640 1 --2- 172.21.15.87:0/2104330285 >> v2:172.21.15.169:3300/0 conn(0x7f067806f100 0x7f06780cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067d200640 1 -- 172.21.15.87:0/2104330285 >> v1:172.21.15.87:6789/0 conn(0x7f06780b0a90 legacy=0x7f067806e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f067d200640 1 -- 172.21.15.87:0/2104330285 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780d0f90 con 0x7f0678161270 2024-04-25T10:19:48.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.220+0000 7f0677400640 1 -- 172.21.15.87:0/2104330285 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364200673 0 0) 0x7f06780d0a10 con 0x7f06780b0a90 2024-04-25T10:19:48.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067d200640 1 --2- 172.21.15.87:0/2104330285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0678161270 0x7f06780cbdf0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f06780d16d0 tx=0x7f066c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5882160e26e52d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f0677400640 1 -- 172.21.15.87:0/2104330285 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0b0070 con 0x7f0678161270 2024-04-25T10:19:48.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f0677400640 1 -- 172.21.15.87:0/2104330285 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c0ab600 con 0x7f0678161270 2024-04-25T10:19:48.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f0677400640 1 -- 172.21.15.87:0/2104330285 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0ab9d0 con 0x7f0678161270 2024-04-25T10:19:48.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 -- 172.21.15.87:0/2104330285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0678161270 msgr2=0x7f06780cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 --2- 172.21.15.87:0/2104330285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0678161270 0x7f06780cbdf0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f06780d16d0 tx=0x7f066c01c340 comp rx=0 tx=0).stop 2024-04-25T10:19:48.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 -- 172.21.15.87:0/2104330285 shutdown_connections 2024-04-25T10:19:48.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 --2- 172.21.15.87:0/2104330285 >> v2:172.21.15.169:3300/0 conn(0x7f067806f100 0x7f06780cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 --2- 172.21.15.87:0/2104330285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0678161270 0x7f06780cbdf0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 -- 172.21.15.87:0/2104330285 >> 172.21.15.87:0/2104330285 conn(0x7f0678066d70 msgr2=0x7f0678067180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 -- 172.21.15.87:0/2104330285 shutdown_connections 2024-04-25T10:19:48.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.224+0000 7f067e600640 1 -- 172.21.15.87:0/2104330285 wait complete. 2024-04-25T10:19:48.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 Processor -- start 2024-04-25T10:19:48.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 -- start start 2024-04-25T10:19:48.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f067806f100 0x7f06780af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06780b0a90 0x7f0678151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0678161270 0x7f0678151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067806bc70 con 0x7f067806f100 2024-04-25T10:19:48.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067806baf0 con 0x7f06780b0a90 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067806bdf0 con 0x7f0678161270 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f067806f100 0x7f06780af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f067806f100 0x7f06780af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48448/0 (socket says 172.21.15.87:48448) 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 -- 172.21.15.87:0/3678540373 learned_addr learned my addr 172.21.15.87:0/3678540373 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067dc00640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0678161270 0x7f0678151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f0677e00640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06780b0a90 0x7f0678151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 -- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0678161270 msgr2=0x7f0678151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0678161270 0x7f0678151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 -- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06780b0a90 msgr2=0x7f0678151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06780b0a90 0x7f0678151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 -- 172.21.15.87:0/3678540373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780ad980 con 0x7f067806f100 2024-04-25T10:19:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067d200640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f067806f100 0x7f06780af620 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f066c019f70 tx=0x7f066c01ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f0674c00640 1 -- 172.21.15.87:0/3678540373 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0b0040 con 0x7f067806f100 2024-04-25T10:19:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f0674c00640 1 -- 172.21.15.87:0/3678540373 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c004030 con 0x7f067806f100 2024-04-25T10:19:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f0674c00640 1 -- 172.21.15.87:0/3678540373 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0043e0 con 0x7f067806f100 2024-04-25T10:19:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06780adc60 con 0x7f067806f100 2024-04-25T10:19:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.228+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06780ae220 con 0x7f067806f100 2024-04-25T10:19:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.232+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0678075120 con 0x7f067806f100 2024-04-25T10:19:48.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.232+0000 7f0674c00640 1 -- 172.21.15.87:0/3678540373 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f066c003620 con 0x7f067806f100 2024-04-25T10:19:48.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.232+0000 7f0674c00640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0658066f60 0x7f0658069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.232+0000 7f0674c00640 1 -- 172.21.15.87:0/3678540373 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f066c11fca0 con 0x7f067806f100 2024-04-25T10:19:48.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.232+0000 7f0677e00640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0658066f60 0x7f0658069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.232+0000 7f0677e00640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0658066f60 0x7f0658069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f06780b0a60 tx=0x7f0664007b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.236+0000 7f0674c00640 1 -- 172.21.15.87:0/3678540373 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f066c016050 con 0x7f067806f100 2024-04-25T10:19:48.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06780cd570 con 0x7f067806f100 2024-04-25T10:19:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f0674c00640 1 -- 172.21.15.87:0/3678540373 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f066c0f0c10 con 0x7f067806f100 2024-04-25T10:19:48.426 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:19:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0658066f60 msgr2=0x7f0658069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0658066f60 0x7f0658069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f06780b0a60 tx=0x7f0664007b60 comp rx=0 tx=0).stop 2024-04-25T10:19:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f067806f100 msgr2=0x7f06780af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f067806f100 0x7f06780af620 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f066c019f70 tx=0x7f066c01ab80 comp rx=0 tx=0).stop 2024-04-25T10:19:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 shutdown_connections 2024-04-25T10:19:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0658066f60 0x7f0658069420 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0678161270 0x7f0678151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06780b0a90 0x7f0678151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 --2- 172.21.15.87:0/3678540373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f067806f100 0x7f06780af620 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 >> 172.21.15.87:0/3678540373 conn(0x7f0678066d70 msgr2=0x7f067806c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 shutdown_connections 2024-04-25T10:19:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:19:48.424+0000 7f067e600640 1 -- 172.21.15.87:0/3678540373 wait complete. 2024-04-25T10:19:48.447 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:19:48.448 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:48.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.448 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:19:48.457 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:19:48.457 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:19:48.457 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:19:48.457 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.457 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:19:48.457 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:48.458 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42025sec preferred_lft 42025sec 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:19:48.459 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:19:48.460 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:19:48.460 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.460 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:19:48.460 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:19:48.460 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:19:48.460 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:48.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.461 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:19:48.521 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:19:48.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.524 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:19:48.575 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:19:48.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.576 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:19:48.576 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:48.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:19:48.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:19:48.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:19:48.576 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:48.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.665 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:19:48.673 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:19:48.674 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:19:48.674 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:48.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.674 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:19:48.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:19:48.735 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:48.788 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:48.788 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:19:48.788 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:48.839 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:19:48.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.839 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:48.931 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:48.931 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:19:48.931 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:19:48.956 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:19:48.956 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:19:48.956 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:19:48.959 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:19:48.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.959 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:48.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:48.979 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:19:49.039 INFO:teuthology.orchestra.run.smithi087.stdout:client7471 2024-04-25T10:19:49.039 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:19:49.039 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:49.092 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:49.092 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T10:19:49.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:49.092 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:49.092 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:49.092 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:19:49.092 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:49.092 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:49.093 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:49.093 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227832 s, 46.0 MB/s 2024-04-25T10:19:49.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.207 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02131 s, 49.2 MB/s 2024-04-25T10:19:49.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.234 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257424 s, 40.7 MB/s 2024-04-25T10:19:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228145 s, 46.0 MB/s 2024-04-25T10:19:49.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.285 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240528 s, 43.6 MB/s 2024-04-25T10:19:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.308 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215975 s, 48.6 MB/s 2024-04-25T10:19:49.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174904 s, 60.0 MB/s 2024-04-25T10:19:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204378 s, 51.3 MB/s 2024-04-25T10:19:49.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157236 s, 66.7 MB/s 2024-04-25T10:19:49.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.388 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186285 s, 56.3 MB/s 2024-04-25T10:19:49.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.404 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142025 s, 73.8 MB/s 2024-04-25T10:19:49.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168257 s, 62.3 MB/s 2024-04-25T10:19:49.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.442 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175392 s, 59.8 MB/s 2024-04-25T10:19:49.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.463 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190596 s, 55.0 MB/s 2024-04-25T10:19:49.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.479 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137413 s, 76.3 MB/s 2024-04-25T10:19:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128664 s, 81.5 MB/s 2024-04-25T10:19:49.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.515 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199499 s, 52.6 MB/s 2024-04-25T10:19:49.531 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.532 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015265 s, 68.7 MB/s 2024-04-25T10:19:49.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204596 s, 51.3 MB/s 2024-04-25T10:19:49.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.573 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178189 s, 58.8 MB/s 2024-04-25T10:19:49.590 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.590 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.590 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01514 s, 69.3 MB/s 2024-04-25T10:19:49.608 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.608 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.608 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155399 s, 67.5 MB/s 2024-04-25T10:19:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175724 s, 59.7 MB/s 2024-04-25T10:19:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191434 s, 54.8 MB/s 2024-04-25T10:19:49.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.670 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206188 s, 50.9 MB/s 2024-04-25T10:19:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150013 s, 69.9 MB/s 2024-04-25T10:19:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153213 s, 68.4 MB/s 2024-04-25T10:19:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190355 s, 55.1 MB/s 2024-04-25T10:19:49.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.742 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156536 s, 67.0 MB/s 2024-04-25T10:19:49.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.761 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169928 s, 61.7 MB/s 2024-04-25T10:19:49.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.779 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157819 s, 66.4 MB/s 2024-04-25T10:19:49.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191584 s, 54.7 MB/s 2024-04-25T10:19:49.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:49.801 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:49.802 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:49.802 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:19:49.802 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:49.802 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:49.802 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:49.802 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.875 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018233 s, 57.5 MB/s 2024-04-25T10:19:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.896 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018561 s, 56.5 MB/s 2024-04-25T10:19:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160666 s, 65.3 MB/s 2024-04-25T10:19:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.931 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156861 s, 66.8 MB/s 2024-04-25T10:19:49.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.950 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175418 s, 59.8 MB/s 2024-04-25T10:19:49.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.969 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164158 s, 63.9 MB/s 2024-04-25T10:19:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155101 s, 67.6 MB/s 2024-04-25T10:19:50.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203981 s, 51.4 MB/s 2024-04-25T10:19:50.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.025 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157 s, 66.8 MB/s 2024-04-25T10:19:50.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142667 s, 73.5 MB/s 2024-04-25T10:19:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142598 s, 73.5 MB/s 2024-04-25T10:19:50.077 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.077 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.077 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176962 s, 59.3 MB/s 2024-04-25T10:19:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140293 s, 74.7 MB/s 2024-04-25T10:19:50.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01554 s, 67.5 MB/s 2024-04-25T10:19:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.127 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153655 s, 68.2 MB/s 2024-04-25T10:19:50.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.150 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205551 s, 51.0 MB/s 2024-04-25T10:19:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.169 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178069 s, 58.9 MB/s 2024-04-25T10:19:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.185 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141103 s, 74.3 MB/s 2024-04-25T10:19:50.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.212 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248821 s, 42.1 MB/s 2024-04-25T10:19:50.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.238 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.238 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240875 s, 43.5 MB/s 2024-04-25T10:19:50.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.259 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196901 s, 53.3 MB/s 2024-04-25T10:19:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159677 s, 65.7 MB/s 2024-04-25T10:19:50.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149395 s, 70.2 MB/s 2024-04-25T10:19:50.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144414 s, 72.6 MB/s 2024-04-25T10:19:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185761 s, 56.4 MB/s 2024-04-25T10:19:50.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165811 s, 63.2 MB/s 2024-04-25T10:19:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185853 s, 56.4 MB/s 2024-04-25T10:19:50.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235306 s, 44.6 MB/s 2024-04-25T10:19:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.415 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188953 s, 55.5 MB/s 2024-04-25T10:19:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.441 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239133 s, 43.8 MB/s 2024-04-25T10:19:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.458 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149938 s, 69.9 MB/s 2024-04-25T10:19:50.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178877 s, 58.6 MB/s 2024-04-25T10:19:50.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:50.479 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:50.479 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:50.479 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:19:50.479 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:50.479 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:50.479 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:50.479 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:50.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.556 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188524 s, 55.6 MB/s 2024-04-25T10:19:50.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.572 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146879 s, 71.4 MB/s 2024-04-25T10:19:50.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.593 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185853 s, 56.4 MB/s 2024-04-25T10:19:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150333 s, 69.8 MB/s 2024-04-25T10:19:50.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.628 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171487 s, 61.1 MB/s 2024-04-25T10:19:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.649 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185398 s, 56.6 MB/s 2024-04-25T10:19:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142571 s, 73.5 MB/s 2024-04-25T10:19:50.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.681 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149573 s, 70.1 MB/s 2024-04-25T10:19:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156109 s, 67.2 MB/s 2024-04-25T10:19:50.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.716 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150709 s, 69.6 MB/s 2024-04-25T10:19:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.732 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014275 s, 73.5 MB/s 2024-04-25T10:19:50.748 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.748 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.748 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141495 s, 74.1 MB/s 2024-04-25T10:19:50.764 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.764 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.764 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148283 s, 70.7 MB/s 2024-04-25T10:19:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.784 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178616 s, 58.7 MB/s 2024-04-25T10:19:50.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.801 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148573 s, 70.6 MB/s 2024-04-25T10:19:50.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.818 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151141 s, 69.4 MB/s 2024-04-25T10:19:50.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.835 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156841 s, 66.9 MB/s 2024-04-25T10:19:50.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174231 s, 60.2 MB/s 2024-04-25T10:19:50.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.874 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185086 s, 56.7 MB/s 2024-04-25T10:19:50.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.891 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146997 s, 71.3 MB/s 2024-04-25T10:19:50.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148285 s, 70.7 MB/s 2024-04-25T10:19:50.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014729 s, 71.2 MB/s 2024-04-25T10:19:50.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157669 s, 66.5 MB/s 2024-04-25T10:19:50.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.961 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174731 s, 60.0 MB/s 2024-04-25T10:19:50.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.980 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171367 s, 61.2 MB/s 2024-04-25T10:19:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140842 s, 74.5 MB/s 2024-04-25T10:19:51.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142293 s, 73.7 MB/s 2024-04-25T10:19:51.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.035 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213231 s, 49.2 MB/s 2024-04-25T10:19:51.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173919 s, 60.3 MB/s 2024-04-25T10:19:51.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.070 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014737 s, 71.2 MB/s 2024-04-25T10:19:51.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.088 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157481 s, 66.6 MB/s 2024-04-25T10:19:51.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.107 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.107 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169474 s, 61.9 MB/s 2024-04-25T10:19:51.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:51.108 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:51.108 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:51.108 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:19:51.108 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:51.108 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:51.108 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:51.108 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:51.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.177 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151983 s, 69.0 MB/s 2024-04-25T10:19:51.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.191 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124687 s, 84.1 MB/s 2024-04-25T10:19:51.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.207 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144929 s, 72.4 MB/s 2024-04-25T10:19:51.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.230 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.230 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210522 s, 49.8 MB/s 2024-04-25T10:19:51.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210631 s, 49.8 MB/s 2024-04-25T10:19:51.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.270 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156025 s, 67.2 MB/s 2024-04-25T10:19:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242714 s, 43.2 MB/s 2024-04-25T10:19:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.313 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150623 s, 69.6 MB/s 2024-04-25T10:19:51.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.335 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205767 s, 51.0 MB/s 2024-04-25T10:19:51.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.352 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151215 s, 69.3 MB/s 2024-04-25T10:19:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165429 s, 63.4 MB/s 2024-04-25T10:19:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.388 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158694 s, 66.1 MB/s 2024-04-25T10:19:51.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.407 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172575 s, 60.8 MB/s 2024-04-25T10:19:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148177 s, 70.8 MB/s 2024-04-25T10:19:51.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174958 s, 59.9 MB/s 2024-04-25T10:19:51.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014909 s, 70.3 MB/s 2024-04-25T10:19:51.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157021 s, 66.8 MB/s 2024-04-25T10:19:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020477 s, 51.2 MB/s 2024-04-25T10:19:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.513 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011943 s, 87.8 MB/s 2024-04-25T10:19:51.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149316 s, 70.2 MB/s 2024-04-25T10:19:51.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145036 s, 72.3 MB/s 2024-04-25T10:19:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152995 s, 68.5 MB/s 2024-04-25T10:19:51.580 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.580 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.580 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147788 s, 71.0 MB/s 2024-04-25T10:19:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144023 s, 72.8 MB/s 2024-04-25T10:19:51.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165175 s, 63.5 MB/s 2024-04-25T10:19:51.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.632 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.632 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157849 s, 66.4 MB/s 2024-04-25T10:19:51.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.653 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196592 s, 53.3 MB/s 2024-04-25T10:19:51.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.670 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150077 s, 69.9 MB/s 2024-04-25T10:19:51.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151084 s, 69.4 MB/s 2024-04-25T10:19:51.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169695 s, 61.8 MB/s 2024-04-25T10:19:51.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.723 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155602 s, 67.4 MB/s 2024-04-25T10:19:51.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.739 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145308 s, 72.2 MB/s 2024-04-25T10:19:51.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:51.741 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:51.741 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:51.741 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:19:51.741 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:51.741 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:51.741 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:51.741 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:51.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.820 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231944 s, 45.2 MB/s 2024-04-25T10:19:51.840 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.840 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.840 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183004 s, 57.3 MB/s 2024-04-25T10:19:51.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.863 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204658 s, 51.2 MB/s 2024-04-25T10:19:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.881 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170651 s, 61.4 MB/s 2024-04-25T10:19:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200374 s, 52.3 MB/s 2024-04-25T10:19:51.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.919 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147471 s, 71.1 MB/s 2024-04-25T10:19:51.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.936 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148138 s, 70.8 MB/s 2024-04-25T10:19:51.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.954 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163553 s, 64.1 MB/s 2024-04-25T10:19:51.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161122 s, 65.1 MB/s 2024-04-25T10:19:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:51.988 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145527 s, 72.1 MB/s 2024-04-25T10:19:52.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.005 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014569 s, 72.0 MB/s 2024-04-25T10:19:52.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.024 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175547 s, 59.7 MB/s 2024-04-25T10:19:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01652 s, 63.5 MB/s 2024-04-25T10:19:52.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182399 s, 57.5 MB/s 2024-04-25T10:19:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141029 s, 74.4 MB/s 2024-04-25T10:19:52.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.094 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143079 s, 73.3 MB/s 2024-04-25T10:19:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186496 s, 56.2 MB/s 2024-04-25T10:19:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160433 s, 65.4 MB/s 2024-04-25T10:19:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180782 s, 58.0 MB/s 2024-04-25T10:19:52.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.176 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209516 s, 50.0 MB/s 2024-04-25T10:19:52.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.190 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129176 s, 81.2 MB/s 2024-04-25T10:19:52.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016101 s, 65.1 MB/s 2024-04-25T10:19:52.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159312 s, 65.8 MB/s 2024-04-25T10:19:52.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.243 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154837 s, 67.7 MB/s 2024-04-25T10:19:52.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.260 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146758 s, 71.4 MB/s 2024-04-25T10:19:52.280 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.280 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.280 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189131 s, 55.4 MB/s 2024-04-25T10:19:52.296 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.296 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.297 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144975 s, 72.3 MB/s 2024-04-25T10:19:52.314 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.314 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.314 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154479 s, 67.9 MB/s 2024-04-25T10:19:52.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.331 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149859 s, 70.0 MB/s 2024-04-25T10:19:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180764 s, 58.0 MB/s 2024-04-25T10:19:52.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.370 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.370 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173911 s, 60.3 MB/s 2024-04-25T10:19:52.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175689 s, 59.7 MB/s 2024-04-25T10:19:52.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:52.391 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:52.391 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:52.391 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:19:52.391 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:52.391 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:52.391 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:52.391 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:52.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.461 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164114 s, 63.9 MB/s 2024-04-25T10:19:52.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141474 s, 74.1 MB/s 2024-04-25T10:19:52.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.494 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015117 s, 69.4 MB/s 2024-04-25T10:19:52.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.511 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151539 s, 69.2 MB/s 2024-04-25T10:19:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148229 s, 70.7 MB/s 2024-04-25T10:19:52.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016129 s, 65.0 MB/s 2024-04-25T10:19:52.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.563 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157957 s, 66.4 MB/s 2024-04-25T10:19:52.580 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.580 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.580 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149057 s, 70.3 MB/s 2024-04-25T10:19:52.596 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.596 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.596 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151216 s, 69.3 MB/s 2024-04-25T10:19:52.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.619 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211815 s, 49.5 MB/s 2024-04-25T10:19:52.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141161 s, 74.3 MB/s 2024-04-25T10:19:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.653 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01558 s, 67.3 MB/s 2024-04-25T10:19:52.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.669 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146191 s, 71.7 MB/s 2024-04-25T10:19:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213634 s, 49.1 MB/s 2024-04-25T10:19:52.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.706 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121628 s, 86.2 MB/s 2024-04-25T10:19:52.722 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.722 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.722 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014298 s, 73.3 MB/s 2024-04-25T10:19:52.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.739 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151815 s, 69.1 MB/s 2024-04-25T10:19:52.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.757 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160704 s, 65.2 MB/s 2024-04-25T10:19:52.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.774 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150904 s, 69.5 MB/s 2024-04-25T10:19:52.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.794 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179218 s, 58.5 MB/s 2024-04-25T10:19:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204368 s, 51.3 MB/s 2024-04-25T10:19:52.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.837 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188526 s, 55.6 MB/s 2024-04-25T10:19:52.855 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.855 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.855 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160424 s, 65.4 MB/s 2024-04-25T10:19:52.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147097 s, 71.3 MB/s 2024-04-25T10:19:52.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.899 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026186 s, 40.0 MB/s 2024-04-25T10:19:52.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166256 s, 63.1 MB/s 2024-04-25T10:19:52.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.935 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147688 s, 71.0 MB/s 2024-04-25T10:19:52.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.951 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013915 s, 75.4 MB/s 2024-04-25T10:19:52.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.967 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146485 s, 71.6 MB/s 2024-04-25T10:19:52.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:52.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:52.990 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020552 s, 51.0 MB/s 2024-04-25T10:19:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.006 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01473 s, 71.2 MB/s 2024-04-25T10:19:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140524 s, 74.6 MB/s 2024-04-25T10:19:53.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:53.023 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:53.023 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:53.024 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:19:53.024 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:53.024 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:53.024 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:53.024 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:53.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.095 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171046 s, 61.3 MB/s 2024-04-25T10:19:53.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.112 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154573 s, 67.8 MB/s 2024-04-25T10:19:53.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.128 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150224 s, 69.8 MB/s 2024-04-25T10:19:53.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.145 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148893 s, 70.4 MB/s 2024-04-25T10:19:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148495 s, 70.6 MB/s 2024-04-25T10:19:53.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:53.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:53.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.178 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146213 s, 71.7 MB/s 2024-04-25T10:19:53.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.198 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175667 s, 59.7 MB/s 2024-04-25T10:19:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155394 s, 67.5 MB/s 2024-04-25T10:19:53.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178195 s, 58.8 MB/s 2024-04-25T10:19:53.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015634 s, 67.1 MB/s 2024-04-25T10:19:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200258 s, 52.4 MB/s 2024-04-25T10:19:53.292 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.293 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169794 s, 61.8 MB/s 2024-04-25T10:19:53.309 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.309 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.309 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144546 s, 72.5 MB/s 2024-04-25T10:19:53.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.326 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149094 s, 70.3 MB/s 2024-04-25T10:19:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144134 s, 72.8 MB/s 2024-04-25T10:19:53.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166974 s, 62.8 MB/s 2024-04-25T10:19:53.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.377 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148063 s, 70.8 MB/s 2024-04-25T10:19:53.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.394 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157354 s, 66.6 MB/s 2024-04-25T10:19:53.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.411 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148411 s, 70.7 MB/s 2024-04-25T10:19:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014596 s, 71.8 MB/s 2024-04-25T10:19:53.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.448 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196166 s, 53.5 MB/s 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.468 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174173 s, 60.2 MB/s 2024-04-25T10:19:53.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.485 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152196 s, 68.9 MB/s 2024-04-25T10:19:53.505 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.505 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.505 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189633 s, 55.3 MB/s 2024-04-25T10:19:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.522 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146218 s, 71.7 MB/s 2024-04-25T10:19:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.539 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148735 s, 70.5 MB/s 2024-04-25T10:19:53.555 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.556 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155846 s, 67.3 MB/s 2024-04-25T10:19:53.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.572 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146315 s, 71.7 MB/s 2024-04-25T10:19:53.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.589 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154311 s, 68.0 MB/s 2024-04-25T10:19:53.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.611 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200037 s, 52.4 MB/s 2024-04-25T10:19:53.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.632 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.632 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018862 s, 55.6 MB/s 2024-04-25T10:19:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020701 s, 50.7 MB/s 2024-04-25T10:19:53.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:53.656 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:53.656 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:53.656 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:19:53.656 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:53.656 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:53.656 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:53.656 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167656 s, 62.5 MB/s 2024-04-25T10:19:53.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.743 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152621 s, 68.7 MB/s 2024-04-25T10:19:53.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.760 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150888 s, 69.5 MB/s 2024-04-25T10:19:53.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.784 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021915 s, 47.8 MB/s 2024-04-25T10:19:53.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.801 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015133 s, 69.3 MB/s 2024-04-25T10:19:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167759 s, 62.5 MB/s 2024-04-25T10:19:53.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174836 s, 60.0 MB/s 2024-04-25T10:19:53.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.856 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160091 s, 65.5 MB/s 2024-04-25T10:19:53.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.873 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147635 s, 71.0 MB/s 2024-04-25T10:19:53.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.893 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180934 s, 58.0 MB/s 2024-04-25T10:19:53.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.911 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168383 s, 62.3 MB/s 2024-04-25T10:19:53.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.930 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167578 s, 62.6 MB/s 2024-04-25T10:19:53.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.952 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201638 s, 52.0 MB/s 2024-04-25T10:19:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165088 s, 63.5 MB/s 2024-04-25T10:19:53.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:53.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:53.987 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153205 s, 68.4 MB/s 2024-04-25T10:19:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.007 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183992 s, 57.0 MB/s 2024-04-25T10:19:54.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.161 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160809 s, 65.2 MB/s 2024-04-25T10:19:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191017 s, 54.9 MB/s 2024-04-25T10:19:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017828 s, 58.8 MB/s 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169051 s, 62.0 MB/s 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147423 s, 71.1 MB/s 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179839 s, 58.3 MB/s 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161264 s, 65.0 MB/s 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152761 s, 68.6 MB/s 2024-04-25T10:19:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.172 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145664 s, 72.0 MB/s 2024-04-25T10:19:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142835 s, 73.4 MB/s 2024-04-25T10:19:54.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160289 s, 65.4 MB/s 2024-04-25T10:19:54.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143031 s, 73.3 MB/s 2024-04-25T10:19:54.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.239 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148051 s, 70.8 MB/s 2024-04-25T10:19:54.256 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.256 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.256 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150695 s, 69.6 MB/s 2024-04-25T10:19:54.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160596 s, 65.3 MB/s 2024-04-25T10:19:54.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.290 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143114 s, 73.3 MB/s 2024-04-25T10:19:54.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:54.292 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:54.292 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:54.292 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:19:54.292 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:54.292 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:54.293 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:54.293 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:54.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.364 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142961 s, 73.3 MB/s 2024-04-25T10:19:54.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148257 s, 70.7 MB/s 2024-04-25T10:19:54.399 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.399 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.399 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171905 s, 61.0 MB/s 2024-04-25T10:19:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.419 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179695 s, 58.4 MB/s 2024-04-25T10:19:54.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.436 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149255 s, 70.3 MB/s 2024-04-25T10:19:54.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139147 s, 75.4 MB/s 2024-04-25T10:19:54.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207798 s, 50.5 MB/s 2024-04-25T10:19:54.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.490 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143946 s, 72.8 MB/s 2024-04-25T10:19:54.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144573 s, 72.5 MB/s 2024-04-25T10:19:54.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.523 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142009 s, 73.8 MB/s 2024-04-25T10:19:54.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203253 s, 51.6 MB/s 2024-04-25T10:19:54.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.563 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164092 s, 63.9 MB/s 2024-04-25T10:19:54.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.585 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202677 s, 51.7 MB/s 2024-04-25T10:19:54.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.601 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146232 s, 71.7 MB/s 2024-04-25T10:19:54.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227827 s, 46.0 MB/s 2024-04-25T10:19:54.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015063 s, 69.6 MB/s 2024-04-25T10:19:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203899 s, 51.4 MB/s 2024-04-25T10:19:54.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.681 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014606 s, 71.8 MB/s 2024-04-25T10:19:54.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.703 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02007 s, 52.2 MB/s 2024-04-25T10:19:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.720 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152013 s, 69.0 MB/s 2024-04-25T10:19:54.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.741 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191647 s, 54.7 MB/s 2024-04-25T10:19:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173449 s, 60.5 MB/s 2024-04-25T10:19:54.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.777 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153933 s, 68.1 MB/s 2024-04-25T10:19:54.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.799 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194423 s, 53.9 MB/s 2024-04-25T10:19:54.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163715 s, 64.0 MB/s 2024-04-25T10:19:54.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015455 s, 67.8 MB/s 2024-04-25T10:19:54.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.851 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154452 s, 67.9 MB/s 2024-04-25T10:19:54.868 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.868 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.868 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152658 s, 68.7 MB/s 2024-04-25T10:19:54.885 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.885 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.885 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148251 s, 70.7 MB/s 2024-04-25T10:19:54.902 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.902 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.902 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150835 s, 69.5 MB/s 2024-04-25T10:19:54.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201737 s, 52.0 MB/s 2024-04-25T10:19:54.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:54.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:54.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159628 s, 65.7 MB/s 2024-04-25T10:19:54.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:55.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.016 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139648 s, 75.1 MB/s 2024-04-25T10:19:55.032 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.033 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.033 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151457 s, 69.2 MB/s 2024-04-25T10:19:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176159 s, 59.5 MB/s 2024-04-25T10:19:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181406 s, 57.8 MB/s 2024-04-25T10:19:55.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.091 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017077 s, 61.4 MB/s 2024-04-25T10:19:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165471 s, 63.4 MB/s 2024-04-25T10:19:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149634 s, 70.1 MB/s 2024-04-25T10:19:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170887 s, 61.4 MB/s 2024-04-25T10:19:55.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.165 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183267 s, 57.2 MB/s 2024-04-25T10:19:55.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.182 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150082 s, 69.9 MB/s 2024-04-25T10:19:55.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.201 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169247 s, 62.0 MB/s 2024-04-25T10:19:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208057 s, 50.4 MB/s 2024-04-25T10:19:55.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.245 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201913 s, 51.9 MB/s 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175762 s, 59.7 MB/s 2024-04-25T10:19:55.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.281 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144823 s, 72.4 MB/s 2024-04-25T10:19:55.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.297 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144381 s, 72.6 MB/s 2024-04-25T10:19:55.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.316 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175707 s, 59.7 MB/s 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159927 s, 65.6 MB/s 2024-04-25T10:19:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.356 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200137 s, 52.4 MB/s 2024-04-25T10:19:55.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.372 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014605 s, 71.8 MB/s 2024-04-25T10:19:55.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.392 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175774 s, 59.7 MB/s 2024-04-25T10:19:55.408 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.408 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.408 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148413 s, 70.7 MB/s 2024-04-25T10:19:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158135 s, 66.3 MB/s 2024-04-25T10:19:55.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.446 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176081 s, 59.6 MB/s 2024-04-25T10:19:55.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.464 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168208 s, 62.3 MB/s 2024-04-25T10:19:55.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.483 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173985 s, 60.3 MB/s 2024-04-25T10:19:55.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014668 s, 71.5 MB/s 2024-04-25T10:19:55.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.516 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145757 s, 71.9 MB/s 2024-04-25T10:19:55.533 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.533 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.533 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151788 s, 69.1 MB/s 2024-04-25T10:19:55.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.550 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154329 s, 67.9 MB/s 2024-04-25T10:19:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168612 s, 62.2 MB/s 2024-04-25T10:19:55.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.585 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148865 s, 70.4 MB/s 2024-04-25T10:19:55.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:55.587 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:55.587 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:55.587 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:19:55.587 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:55.587 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:55.587 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:55.587 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:55.659 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.659 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.659 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177322 s, 59.1 MB/s 2024-04-25T10:19:55.677 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.677 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.677 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162157 s, 64.7 MB/s 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125105 s, 83.8 MB/s 2024-04-25T10:19:55.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148111 s, 70.8 MB/s 2024-04-25T10:19:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181515 s, 57.8 MB/s 2024-04-25T10:19:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158005 s, 66.4 MB/s 2024-04-25T10:19:55.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.771 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242088 s, 43.3 MB/s 2024-04-25T10:19:55.791 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.792 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183945 s, 57.0 MB/s 2024-04-25T10:19:55.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.808 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151976 s, 69.0 MB/s 2024-04-25T10:19:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195604 s, 53.6 MB/s 2024-04-25T10:19:55.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.848 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168881 s, 62.1 MB/s 2024-04-25T10:19:55.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.865 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145751 s, 71.9 MB/s 2024-04-25T10:19:55.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.890 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232309 s, 45.1 MB/s 2024-04-25T10:19:55.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160362 s, 65.4 MB/s 2024-04-25T10:19:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143923 s, 72.9 MB/s 2024-04-25T10:19:55.942 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.942 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.943 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172981 s, 60.6 MB/s 2024-04-25T10:19:55.959 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.959 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.959 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152127 s, 68.9 MB/s 2024-04-25T10:19:55.976 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.976 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.976 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151308 s, 69.3 MB/s 2024-04-25T10:19:55.994 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:55.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:55.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166452 s, 63.0 MB/s 2024-04-25T10:19:56.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.011 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147002 s, 71.3 MB/s 2024-04-25T10:19:56.028 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.028 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.028 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150464 s, 69.7 MB/s 2024-04-25T10:19:56.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.043 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130394 s, 80.4 MB/s 2024-04-25T10:19:56.061 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173301 s, 60.5 MB/s 2024-04-25T10:19:56.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155131 s, 67.6 MB/s 2024-04-25T10:19:56.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.100 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191552 s, 54.7 MB/s 2024-04-25T10:19:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141998 s, 73.8 MB/s 2024-04-25T10:19:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.134 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163066 s, 64.3 MB/s 2024-04-25T10:19:56.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.154 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180496 s, 58.1 MB/s 2024-04-25T10:19:56.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.170 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144764 s, 72.4 MB/s 2024-04-25T10:19:56.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.189 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170045 s, 61.7 MB/s 2024-04-25T10:19:56.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150143 s, 69.8 MB/s 2024-04-25T10:19:56.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145674 s, 72.0 MB/s 2024-04-25T10:19:56.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:56.224 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:56.224 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:56.224 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:19:56.224 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:56.224 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:56.224 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:56.224 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170006 s, 61.7 MB/s 2024-04-25T10:19:56.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014677 s, 71.4 MB/s 2024-04-25T10:19:56.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.335 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224922 s, 46.6 MB/s 2024-04-25T10:19:56.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.351 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148405 s, 70.7 MB/s 2024-04-25T10:19:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156695 s, 66.9 MB/s 2024-04-25T10:19:56.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.387 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164524 s, 63.7 MB/s 2024-04-25T10:19:56.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146756 s, 71.5 MB/s 2024-04-25T10:19:56.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.419 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139598 s, 75.1 MB/s 2024-04-25T10:19:56.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.437 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165092 s, 63.5 MB/s 2024-04-25T10:19:56.453 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.453 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.453 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141666 s, 74.0 MB/s 2024-04-25T10:19:56.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163902 s, 64.0 MB/s 2024-04-25T10:19:56.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.490 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171084 s, 61.3 MB/s 2024-04-25T10:19:56.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.510 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184699 s, 56.8 MB/s 2024-04-25T10:19:56.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140086 s, 74.9 MB/s 2024-04-25T10:19:56.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.542 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147002 s, 71.3 MB/s 2024-04-25T10:19:56.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.559 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146162 s, 71.7 MB/s 2024-04-25T10:19:56.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148302 s, 70.7 MB/s 2024-04-25T10:19:56.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.597 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195706 s, 53.6 MB/s 2024-04-25T10:19:56.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154527 s, 67.9 MB/s 2024-04-25T10:19:56.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147223 s, 71.2 MB/s 2024-04-25T10:19:56.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.649 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167319 s, 62.7 MB/s 2024-04-25T10:19:56.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.666 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154419 s, 67.9 MB/s 2024-04-25T10:19:56.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.688 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197916 s, 53.0 MB/s 2024-04-25T10:19:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202164 s, 51.9 MB/s 2024-04-25T10:19:56.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.727 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150713 s, 69.6 MB/s 2024-04-25T10:19:56.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.744 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149446 s, 70.2 MB/s 2024-04-25T10:19:56.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.760 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.760 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149445 s, 70.2 MB/s 2024-04-25T10:19:56.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.783 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210151 s, 49.9 MB/s 2024-04-25T10:19:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181309 s, 57.8 MB/s 2024-04-25T10:19:56.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.829 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239256 s, 43.8 MB/s 2024-04-25T10:19:56.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.851 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207728 s, 50.5 MB/s 2024-04-25T10:19:56.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.872 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184656 s, 56.8 MB/s 2024-04-25T10:19:56.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:56.873 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:56.873 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:56.873 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:19:56.873 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:56.873 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:56.873 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:56.873 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:56.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011185 s, 93.7 MB/s 2024-04-25T10:19:56.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.957 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146077 s, 71.8 MB/s 2024-04-25T10:19:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169786 s, 61.8 MB/s 2024-04-25T10:19:56.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:56.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:56.991 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141492 s, 74.1 MB/s 2024-04-25T10:19:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157872 s, 66.4 MB/s 2024-04-25T10:19:57.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.025 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143515 s, 73.1 MB/s 2024-04-25T10:19:57.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.045 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01802 s, 58.2 MB/s 2024-04-25T10:19:57.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155455 s, 67.5 MB/s 2024-04-25T10:19:57.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.081 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172163 s, 60.9 MB/s 2024-04-25T10:19:57.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.098 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149816 s, 70.0 MB/s 2024-04-25T10:19:57.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152309 s, 68.8 MB/s 2024-04-25T10:19:57.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.134 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172531 s, 60.8 MB/s 2024-04-25T10:19:57.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.150 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148571 s, 70.6 MB/s 2024-04-25T10:19:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155311 s, 67.5 MB/s 2024-04-25T10:19:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173945 s, 60.3 MB/s 2024-04-25T10:19:57.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.205 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157438 s, 66.6 MB/s 2024-04-25T10:19:57.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184481 s, 56.8 MB/s 2024-04-25T10:19:57.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.243 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167827 s, 62.5 MB/s 2024-04-25T10:19:57.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.265 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202865 s, 51.7 MB/s 2024-04-25T10:19:57.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.286 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184351 s, 56.9 MB/s 2024-04-25T10:19:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205417 s, 51.0 MB/s 2024-04-25T10:19:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159742 s, 65.6 MB/s 2024-04-25T10:19:57.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.342 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014238 s, 73.6 MB/s 2024-04-25T10:19:57.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.362 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01773 s, 59.1 MB/s 2024-04-25T10:19:57.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.377 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140504 s, 74.6 MB/s 2024-04-25T10:19:57.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.394 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148447 s, 70.6 MB/s 2024-04-25T10:19:57.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.414 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181028 s, 57.9 MB/s 2024-04-25T10:19:57.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.436 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198402 s, 52.9 MB/s 2024-04-25T10:19:57.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014468 s, 72.5 MB/s 2024-04-25T10:19:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190259 s, 55.1 MB/s 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150115 s, 69.9 MB/s 2024-04-25T10:19:57.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.511 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198805 s, 52.7 MB/s 2024-04-25T10:19:57.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:57.512 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:57.512 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:57.513 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:19:57.513 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:57.513 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:57.513 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:57.513 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:57.584 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.584 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.584 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150865 s, 69.5 MB/s 2024-04-25T10:19:57.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.601 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152 s, 69.0 MB/s 2024-04-25T10:19:57.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.618 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153645 s, 68.2 MB/s 2024-04-25T10:19:57.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.638 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181251 s, 57.9 MB/s 2024-04-25T10:19:57.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156297 s, 67.1 MB/s 2024-04-25T10:19:57.672 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.672 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.672 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148458 s, 70.6 MB/s 2024-04-25T10:19:57.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.688 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141327 s, 74.2 MB/s 2024-04-25T10:19:57.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.704 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140975 s, 74.4 MB/s 2024-04-25T10:19:57.722 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.722 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.722 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164145 s, 63.9 MB/s 2024-04-25T10:19:57.738 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.738 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.738 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143125 s, 73.3 MB/s 2024-04-25T10:19:57.759 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.759 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.759 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195257 s, 53.7 MB/s 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154958 s, 67.7 MB/s 2024-04-25T10:19:57.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.798 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194568 s, 53.9 MB/s 2024-04-25T10:19:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011386 s, 92.1 MB/s 2024-04-25T10:19:57.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.831 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179557 s, 58.4 MB/s 2024-04-25T10:19:57.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206099 s, 50.9 MB/s 2024-04-25T10:19:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151968 s, 69.0 MB/s 2024-04-25T10:19:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021967 s, 47.7 MB/s 2024-04-25T10:19:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168297 s, 62.3 MB/s 2024-04-25T10:19:57.943 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.943 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.943 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278718 s, 37.6 MB/s 2024-04-25T10:19:57.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.962 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017164 s, 61.1 MB/s 2024-04-25T10:19:57.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.978 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145502 s, 72.1 MB/s 2024-04-25T10:19:57.994 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:57.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:57.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144343 s, 72.6 MB/s 2024-04-25T10:19:58.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.011 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142138 s, 73.8 MB/s 2024-04-25T10:19:58.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.028 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148755 s, 70.5 MB/s 2024-04-25T10:19:58.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.053 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233823 s, 44.8 MB/s 2024-04-25T10:19:58.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.070 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152735 s, 68.7 MB/s 2024-04-25T10:19:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150377 s, 69.7 MB/s 2024-04-25T10:19:58.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197965 s, 53.0 MB/s 2024-04-25T10:19:58.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.128 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182145 s, 57.6 MB/s 2024-04-25T10:19:58.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.145 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149899 s, 70.0 MB/s 2024-04-25T10:19:58.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.161 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014335 s, 73.1 MB/s 2024-04-25T10:19:58.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:58.163 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:58.163 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:58.163 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:19:58.163 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:58.163 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:58.163 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:58.163 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:58.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.232 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014188 s, 73.9 MB/s 2024-04-25T10:19:58.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.248 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148394 s, 70.7 MB/s 2024-04-25T10:19:58.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.271 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209123 s, 50.1 MB/s 2024-04-25T10:19:58.289 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.289 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.289 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164522 s, 63.7 MB/s 2024-04-25T10:19:58.312 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.313 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217352 s, 48.2 MB/s 2024-04-25T10:19:58.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145213 s, 72.2 MB/s 2024-04-25T10:19:58.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.355 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247132 s, 42.4 MB/s 2024-04-25T10:19:58.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.372 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148869 s, 70.4 MB/s 2024-04-25T10:19:58.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.390 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016276 s, 64.4 MB/s 2024-04-25T10:19:58.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145278 s, 72.2 MB/s 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.423 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146167 s, 71.7 MB/s 2024-04-25T10:19:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.439 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01462 s, 71.7 MB/s 2024-04-25T10:19:58.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019208 s, 54.6 MB/s 2024-04-25T10:19:58.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118274 s, 88.7 MB/s 2024-04-25T10:19:58.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.490 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148565 s, 70.6 MB/s 2024-04-25T10:19:58.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153725 s, 68.2 MB/s 2024-04-25T10:19:58.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150769 s, 69.5 MB/s 2024-04-25T10:19:58.540 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.541 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149298 s, 70.2 MB/s 2024-04-25T10:19:58.557 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.557 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.557 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149616 s, 70.1 MB/s 2024-04-25T10:19:58.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.581 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220475 s, 47.6 MB/s 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152479 s, 68.8 MB/s 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144688 s, 72.5 MB/s 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146648 s, 71.5 MB/s 2024-04-25T10:19:58.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.647 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146771 s, 71.4 MB/s 2024-04-25T10:19:58.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.663 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140888 s, 74.4 MB/s 2024-04-25T10:19:58.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.680 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146083 s, 71.8 MB/s 2024-04-25T10:19:58.701 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.701 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.701 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198467 s, 52.8 MB/s 2024-04-25T10:19:58.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.717 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140908 s, 74.4 MB/s 2024-04-25T10:19:58.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.735 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159364 s, 65.8 MB/s 2024-04-25T10:19:58.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.752 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152134 s, 68.9 MB/s 2024-04-25T10:19:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.769 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151632 s, 69.2 MB/s 2024-04-25T10:19:58.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.787 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160309 s, 65.4 MB/s 2024-04-25T10:19:58.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:58.788 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:58.788 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:58.788 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:19:58.788 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:58.789 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:58.789 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:58.789 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:58.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172914 s, 60.6 MB/s 2024-04-25T10:19:58.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233219 s, 45.0 MB/s 2024-04-25T10:19:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.904 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153572 s, 68.3 MB/s 2024-04-25T10:19:58.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.921 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147592 s, 71.0 MB/s 2024-04-25T10:19:58.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.939 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016819 s, 62.3 MB/s 2024-04-25T10:19:58.960 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.960 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.960 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192772 s, 54.4 MB/s 2024-04-25T10:19:58.976 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.976 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146807 s, 71.4 MB/s 2024-04-25T10:19:58.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:58.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:58.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174325 s, 60.2 MB/s 2024-04-25T10:19:59.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177563 s, 59.1 MB/s 2024-04-25T10:19:59.032 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.032 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.032 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151936 s, 69.0 MB/s 2024-04-25T10:19:59.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204392 s, 51.3 MB/s 2024-04-25T10:19:59.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.071 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.071 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142223 s, 73.7 MB/s 2024-04-25T10:19:59.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.093 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200612 s, 52.3 MB/s 2024-04-25T10:19:59.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156303 s, 67.1 MB/s 2024-04-25T10:19:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202481 s, 51.8 MB/s 2024-04-25T10:19:59.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.149 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148398 s, 70.7 MB/s 2024-04-25T10:19:59.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.169 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178179 s, 58.8 MB/s 2024-04-25T10:19:59.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.185 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145997 s, 71.8 MB/s 2024-04-25T10:19:59.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219983 s, 47.7 MB/s 2024-04-25T10:19:59.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.228 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179444 s, 58.4 MB/s 2024-04-25T10:19:59.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.247 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171629 s, 61.1 MB/s 2024-04-25T10:19:59.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.263 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144909 s, 72.4 MB/s 2024-04-25T10:19:59.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.283 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173677 s, 60.4 MB/s 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143159 s, 73.2 MB/s 2024-04-25T10:19:59.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.317 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164202 s, 63.9 MB/s 2024-04-25T10:19:59.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.333 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142834 s, 73.4 MB/s 2024-04-25T10:19:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166083 s, 63.1 MB/s 2024-04-25T10:19:59.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.373 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019617 s, 53.5 MB/s 2024-04-25T10:19:59.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147591 s, 71.0 MB/s 2024-04-25T10:19:59.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147713 s, 71.0 MB/s 2024-04-25T10:19:59.430 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.430 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.430 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223142 s, 47.0 MB/s 2024-04-25T10:19:59.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.446 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142625 s, 73.5 MB/s 2024-04-25T10:19:59.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:59.448 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:19:59.448 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:19:59.448 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:19:59.448 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:19:59.448 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:19:59.448 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:19:59.448 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:19:59.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.516 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146276 s, 71.7 MB/s 2024-04-25T10:19:59.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.532 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143156 s, 73.2 MB/s 2024-04-25T10:19:59.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.550 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165856 s, 63.2 MB/s 2024-04-25T10:19:59.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.572 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020271 s, 51.7 MB/s 2024-04-25T10:19:59.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.589 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156453 s, 67.0 MB/s 2024-04-25T10:19:59.605 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.605 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.605 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139647 s, 75.1 MB/s 2024-04-25T10:19:59.624 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.624 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.624 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016859 s, 62.2 MB/s 2024-04-25T10:19:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186961 s, 56.1 MB/s 2024-04-25T10:19:59.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185179 s, 56.6 MB/s 2024-04-25T10:19:59.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.681 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.681 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014652 s, 71.6 MB/s 2024-04-25T10:19:59.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.698 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150858 s, 69.5 MB/s 2024-04-25T10:19:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020114 s, 52.1 MB/s 2024-04-25T10:19:59.738 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.738 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.738 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016946 s, 61.9 MB/s 2024-04-25T10:19:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.756 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01519 s, 69.0 MB/s 2024-04-25T10:19:59.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.775 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.775 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173216 s, 60.5 MB/s 2024-04-25T10:19:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174006 s, 60.3 MB/s 2024-04-25T10:19:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142954 s, 73.4 MB/s 2024-04-25T10:19:59.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.827 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155175 s, 67.6 MB/s 2024-04-25T10:19:59.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.849 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198646 s, 52.8 MB/s 2024-04-25T10:19:59.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.873 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222979 s, 47.0 MB/s 2024-04-25T10:19:59.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.889 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143332 s, 73.2 MB/s 2024-04-25T10:19:59.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.909 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185363 s, 56.6 MB/s 2024-04-25T10:19:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145654 s, 72.0 MB/s 2024-04-25T10:19:59.943 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.943 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.943 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150129 s, 69.8 MB/s 2024-04-25T10:19:59.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200232 s, 52.4 MB/s 2024-04-25T10:19:59.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.982 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154896 s, 67.7 MB/s 2024-04-25T10:19:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:19:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:19:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142191 s, 73.7 MB/s 2024-04-25T10:20:00.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.017 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177643 s, 59.0 MB/s 2024-04-25T10:20:00.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.038 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188054 s, 55.8 MB/s 2024-04-25T10:20:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152308 s, 68.8 MB/s 2024-04-25T10:20:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156836 s, 66.9 MB/s 2024-04-25T10:20:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154248 s, 68.0 MB/s 2024-04-25T10:20:00.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:00.091 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:00.091 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:00.091 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:20:00.091 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:00.091 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:00.091 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:00.091 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:00.160 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.160 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.160 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143962 s, 72.8 MB/s 2024-04-25T10:20:00.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.179 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178251 s, 58.8 MB/s 2024-04-25T10:20:00.204 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.204 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.204 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234227 s, 44.8 MB/s 2024-04-25T10:20:00.221 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154806 s, 67.7 MB/s 2024-04-25T10:20:00.238 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.238 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.238 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014983 s, 70.0 MB/s 2024-04-25T10:20:00.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116344 s, 90.1 MB/s 2024-04-25T10:20:00.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.271 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172279 s, 60.9 MB/s 2024-04-25T10:20:00.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156127 s, 67.2 MB/s 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144427 s, 72.6 MB/s 2024-04-25T10:20:00.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166058 s, 63.1 MB/s 2024-04-25T10:20:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149558 s, 70.1 MB/s 2024-04-25T10:20:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172572 s, 60.8 MB/s 2024-04-25T10:20:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148567 s, 70.6 MB/s 2024-04-25T10:20:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168029 s, 62.4 MB/s 2024-04-25T10:20:00.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.409 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144762 s, 72.4 MB/s 2024-04-25T10:20:00.432 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.432 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.432 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203566 s, 51.5 MB/s 2024-04-25T10:20:00.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202319 s, 51.8 MB/s 2024-04-25T10:20:00.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157764 s, 66.5 MB/s 2024-04-25T10:20:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211282 s, 49.6 MB/s 2024-04-25T10:20:00.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.511 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153172 s, 68.5 MB/s 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146112 s, 71.8 MB/s 2024-04-25T10:20:00.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.547 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.547 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171045 s, 61.3 MB/s 2024-04-25T10:20:00.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209887 s, 50.0 MB/s 2024-04-25T10:20:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158931 s, 66.0 MB/s 2024-04-25T10:20:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142898 s, 73.4 MB/s 2024-04-25T10:20:00.622 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.623 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.623 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175525 s, 59.7 MB/s 2024-04-25T10:20:00.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.639 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.639 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143951 s, 72.8 MB/s 2024-04-25T10:20:00.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144749 s, 72.4 MB/s 2024-04-25T10:20:00.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.671 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142788 s, 73.4 MB/s 2024-04-25T10:20:00.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.690 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168362 s, 62.3 MB/s 2024-04-25T10:20:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144869 s, 72.4 MB/s 2024-04-25T10:20:00.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.723 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153844 s, 68.2 MB/s 2024-04-25T10:20:00.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:00.725 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:00.725 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:00.725 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:20:00.725 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:00.725 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:00.725 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:00.725 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:00.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.801 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165677 s, 63.3 MB/s 2024-04-25T10:20:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.818 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149873 s, 70.0 MB/s 2024-04-25T10:20:00.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145349 s, 72.1 MB/s 2024-04-25T10:20:00.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179411 s, 58.4 MB/s 2024-04-25T10:20:00.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.874 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180657 s, 58.0 MB/s 2024-04-25T10:20:00.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.891 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148433 s, 70.6 MB/s 2024-04-25T10:20:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.912 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192132 s, 54.6 MB/s 2024-04-25T10:20:00.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.928 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149776 s, 70.0 MB/s 2024-04-25T10:20:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145775 s, 71.9 MB/s 2024-04-25T10:20:00.965 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.965 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.965 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181138 s, 57.9 MB/s 2024-04-25T10:20:00.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:00.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:00.981 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146527 s, 71.6 MB/s 2024-04-25T10:20:01.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.003 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203006 s, 51.7 MB/s 2024-04-25T10:20:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.020 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144512 s, 72.6 MB/s 2024-04-25T10:20:01.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.042 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.042 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202768 s, 51.7 MB/s 2024-04-25T10:20:01.059 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.059 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.060 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160858 s, 65.2 MB/s 2024-04-25T10:20:01.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.082 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205572 s, 51.0 MB/s 2024-04-25T10:20:01.099 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.099 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.099 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151147 s, 69.4 MB/s 2024-04-25T10:20:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.118 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017926 s, 58.5 MB/s 2024-04-25T10:20:01.135 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.135 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.135 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148793 s, 70.5 MB/s 2024-04-25T10:20:01.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206246 s, 50.8 MB/s 2024-04-25T10:20:01.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201221 s, 52.1 MB/s 2024-04-25T10:20:01.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.202 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203415 s, 51.5 MB/s 2024-04-25T10:20:01.219 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.219 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.219 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149025 s, 70.4 MB/s 2024-04-25T10:20:01.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.242 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207626 s, 50.5 MB/s 2024-04-25T10:20:01.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.258 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144102 s, 72.8 MB/s 2024-04-25T10:20:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145404 s, 72.1 MB/s 2024-04-25T10:20:01.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.291 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151917 s, 69.0 MB/s 2024-04-25T10:20:01.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148083 s, 70.8 MB/s 2024-04-25T10:20:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.324 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146294 s, 71.7 MB/s 2024-04-25T10:20:01.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.341 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153076 s, 68.5 MB/s 2024-04-25T10:20:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148013 s, 70.8 MB/s 2024-04-25T10:20:01.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.372 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121937 s, 86.0 MB/s 2024-04-25T10:20:01.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:01.373 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:01.373 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:01.373 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:20:01.373 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:01.373 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:01.373 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:01.373 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:01.450 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.450 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.450 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209675 s, 50.0 MB/s 2024-04-25T10:20:01.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.470 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187084 s, 56.0 MB/s 2024-04-25T10:20:01.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.493 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212013 s, 49.5 MB/s 2024-04-25T10:20:01.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.510 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014372 s, 73.0 MB/s 2024-04-25T10:20:01.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.528 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162623 s, 64.5 MB/s 2024-04-25T10:20:01.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151497 s, 69.2 MB/s 2024-04-25T10:20:01.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197325 s, 53.1 MB/s 2024-04-25T10:20:01.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195257 s, 53.7 MB/s 2024-04-25T10:20:01.605 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.606 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.606 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165154 s, 63.5 MB/s 2024-04-25T10:20:01.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.627 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193849 s, 54.1 MB/s 2024-04-25T10:20:01.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146529 s, 71.6 MB/s 2024-04-25T10:20:01.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168318 s, 62.3 MB/s 2024-04-25T10:20:01.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151275 s, 69.3 MB/s 2024-04-25T10:20:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145754 s, 71.9 MB/s 2024-04-25T10:20:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.711 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141383 s, 74.2 MB/s 2024-04-25T10:20:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160218 s, 65.4 MB/s 2024-04-25T10:20:01.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.750 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195309 s, 53.7 MB/s 2024-04-25T10:20:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156424 s, 67.0 MB/s 2024-04-25T10:20:01.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.786 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.786 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159831 s, 65.6 MB/s 2024-04-25T10:20:01.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.806 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182766 s, 57.4 MB/s 2024-04-25T10:20:01.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.826 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183141 s, 57.3 MB/s 2024-04-25T10:20:01.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.841 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133278 s, 78.7 MB/s 2024-04-25T10:20:01.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.857 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143482 s, 73.1 MB/s 2024-04-25T10:20:01.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.874 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152784 s, 68.6 MB/s 2024-04-25T10:20:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.894 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188126 s, 55.7 MB/s 2024-04-25T10:20:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155501 s, 67.4 MB/s 2024-04-25T10:20:01.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.928 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143672 s, 73.0 MB/s 2024-04-25T10:20:01.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.945 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152043 s, 69.0 MB/s 2024-04-25T10:20:01.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.964 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171325 s, 61.2 MB/s 2024-04-25T10:20:01.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.981 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153737 s, 68.2 MB/s 2024-04-25T10:20:01.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:01.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:01.997 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141767 s, 74.0 MB/s 2024-04-25T10:20:02.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.019 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203812 s, 51.4 MB/s 2024-04-25T10:20:02.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:02.020 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:02.020 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:02.021 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:20:02.021 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:02.021 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:02.021 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:02.021 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:02.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147739 s, 71.0 MB/s 2024-04-25T10:20:02.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.116 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017836 s, 58.8 MB/s 2024-04-25T10:20:02.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.133 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151734 s, 69.1 MB/s 2024-04-25T10:20:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177815 s, 59.0 MB/s 2024-04-25T10:20:02.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142462 s, 73.6 MB/s 2024-04-25T10:20:02.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143696 s, 73.0 MB/s 2024-04-25T10:20:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162115 s, 64.7 MB/s 2024-04-25T10:20:02.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.218 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138045 s, 76.0 MB/s 2024-04-25T10:20:02.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149809 s, 70.0 MB/s 2024-04-25T10:20:02.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.251 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147939 s, 70.9 MB/s 2024-04-25T10:20:02.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.267 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142564 s, 73.6 MB/s 2024-04-25T10:20:02.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.284 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144101 s, 72.8 MB/s 2024-04-25T10:20:02.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.304 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186578 s, 56.2 MB/s 2024-04-25T10:20:02.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.323 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01682 s, 62.3 MB/s 2024-04-25T10:20:02.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.341 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164372 s, 63.8 MB/s 2024-04-25T10:20:02.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.357 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147351 s, 71.2 MB/s 2024-04-25T10:20:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146518 s, 71.6 MB/s 2024-04-25T10:20:02.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.390 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141493 s, 74.1 MB/s 2024-04-25T10:20:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.409 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177498 s, 59.1 MB/s 2024-04-25T10:20:02.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.429 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174033 s, 60.3 MB/s 2024-04-25T10:20:02.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.444 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141313 s, 74.2 MB/s 2024-04-25T10:20:02.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.464 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177776 s, 59.0 MB/s 2024-04-25T10:20:02.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115448 s, 90.8 MB/s 2024-04-25T10:20:02.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.495 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165855 s, 63.2 MB/s 2024-04-25T10:20:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.513 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156562 s, 67.0 MB/s 2024-04-25T10:20:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180147 s, 58.2 MB/s 2024-04-25T10:20:02.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.550 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153405 s, 68.4 MB/s 2024-04-25T10:20:02.571 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.571 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.571 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192821 s, 54.4 MB/s 2024-04-25T10:20:02.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.591 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186218 s, 56.3 MB/s 2024-04-25T10:20:02.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166272 s, 63.1 MB/s 2024-04-25T10:20:02.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148869 s, 70.4 MB/s 2024-04-25T10:20:02.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147342 s, 71.2 MB/s 2024-04-25T10:20:02.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:02.644 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:02.644 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:02.644 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:20:02.644 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:02.644 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:02.644 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:02.644 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:02.719 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.719 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.719 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140649 s, 74.6 MB/s 2024-04-25T10:20:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152066 s, 69.0 MB/s 2024-04-25T10:20:02.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.755 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171125 s, 61.3 MB/s 2024-04-25T10:20:02.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.772 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153281 s, 68.4 MB/s 2024-04-25T10:20:02.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.790 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157675 s, 66.5 MB/s 2024-04-25T10:20:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.807 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151138 s, 69.4 MB/s 2024-04-25T10:20:02.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.824 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152103 s, 68.9 MB/s 2024-04-25T10:20:02.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183008 s, 57.3 MB/s 2024-04-25T10:20:02.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.861 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157526 s, 66.6 MB/s 2024-04-25T10:20:02.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.878 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145328 s, 72.2 MB/s 2024-04-25T10:20:02.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204187 s, 51.4 MB/s 2024-04-25T10:20:02.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153725 s, 68.2 MB/s 2024-04-25T10:20:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209944 s, 49.9 MB/s 2024-04-25T10:20:02.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.958 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160652 s, 65.3 MB/s 2024-04-25T10:20:02.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:02.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:02.981 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215217 s, 48.7 MB/s 2024-04-25T10:20:03.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017566 s, 59.7 MB/s 2024-04-25T10:20:03.020 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.020 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.020 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179171 s, 58.5 MB/s 2024-04-25T10:20:03.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.038 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156789 s, 66.9 MB/s 2024-04-25T10:20:03.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145446 s, 72.1 MB/s 2024-04-25T10:20:03.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.070 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142329 s, 73.7 MB/s 2024-04-25T10:20:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162583 s, 64.5 MB/s 2024-04-25T10:20:03.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.105 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149401 s, 70.2 MB/s 2024-04-25T10:20:03.122 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.122 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.122 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154291 s, 68.0 MB/s 2024-04-25T10:20:03.144 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.144 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.144 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202956 s, 51.7 MB/s 2024-04-25T10:20:03.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174961 s, 59.9 MB/s 2024-04-25T10:20:03.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114387 s, 91.7 MB/s 2024-04-25T10:20:03.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.191 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121186 s, 86.5 MB/s 2024-04-25T10:20:03.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165817 s, 63.2 MB/s 2024-04-25T10:20:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148794 s, 70.5 MB/s 2024-04-25T10:20:03.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.248 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203081 s, 51.6 MB/s 2024-04-25T10:20:03.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.264 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147167 s, 71.3 MB/s 2024-04-25T10:20:03.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.283 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016617 s, 63.1 MB/s 2024-04-25T10:20:03.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:03.284 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:03.284 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:03.284 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:20:03.285 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:03.285 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:03.285 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:03.285 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:03.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161364 s, 65.0 MB/s 2024-04-25T10:20:03.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204024 s, 51.4 MB/s 2024-04-25T10:20:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.399 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017256 s, 60.8 MB/s 2024-04-25T10:20:03.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.416 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155586 s, 67.4 MB/s 2024-04-25T10:20:03.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.438 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199594 s, 52.5 MB/s 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145284 s, 72.2 MB/s 2024-04-25T10:20:03.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153271 s, 68.4 MB/s 2024-04-25T10:20:03.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.488 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144336 s, 72.6 MB/s 2024-04-25T10:20:03.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177385 s, 59.1 MB/s 2024-04-25T10:20:03.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155625 s, 67.4 MB/s 2024-04-25T10:20:03.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.547 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205246 s, 51.1 MB/s 2024-04-25T10:20:03.564 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.564 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.564 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159872 s, 65.6 MB/s 2024-04-25T10:20:03.580 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.581 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146203 s, 71.7 MB/s 2024-04-25T10:20:03.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.597 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148251 s, 70.7 MB/s 2024-04-25T10:20:03.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145117 s, 72.3 MB/s 2024-04-25T10:20:03.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151877 s, 69.0 MB/s 2024-04-25T10:20:03.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.648 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015953 s, 65.7 MB/s 2024-04-25T10:20:03.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144414 s, 72.6 MB/s 2024-04-25T10:20:03.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.683 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170867 s, 61.4 MB/s 2024-04-25T10:20:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139621 s, 75.1 MB/s 2024-04-25T10:20:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174488 s, 60.1 MB/s 2024-04-25T10:20:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.735 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147789 s, 71.0 MB/s 2024-04-25T10:20:03.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.749 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120646 s, 86.9 MB/s 2024-04-25T10:20:03.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.767 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160931 s, 65.2 MB/s 2024-04-25T10:20:03.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.790 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212402 s, 49.4 MB/s 2024-04-25T10:20:03.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.809 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017191 s, 61.0 MB/s 2024-04-25T10:20:03.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.827 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167764 s, 62.5 MB/s 2024-04-25T10:20:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166465 s, 63.0 MB/s 2024-04-25T10:20:03.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01539 s, 68.1 MB/s 2024-04-25T10:20:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181175 s, 57.9 MB/s 2024-04-25T10:20:03.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.899 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147221 s, 71.2 MB/s 2024-04-25T10:20:03.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01701 s, 61.6 MB/s 2024-04-25T10:20:03.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:03.920 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:03.920 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:03.920 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:20:03.920 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:03.920 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:03.920 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:03.920 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:03.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:03.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:03.990 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014119 s, 74.3 MB/s 2024-04-25T10:20:04.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.011 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209047 s, 50.2 MB/s 2024-04-25T10:20:04.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.029 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169486 s, 61.9 MB/s 2024-04-25T10:20:04.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.046 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143321 s, 73.2 MB/s 2024-04-25T10:20:04.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.063 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154675 s, 67.8 MB/s 2024-04-25T10:20:04.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143895 s, 72.9 MB/s 2024-04-25T10:20:04.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.098 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168506 s, 62.2 MB/s 2024-04-25T10:20:04.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220086 s, 47.6 MB/s 2024-04-25T10:20:04.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.138 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152874 s, 68.6 MB/s 2024-04-25T10:20:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.161 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020472 s, 51.2 MB/s 2024-04-25T10:20:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.178 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151275 s, 69.3 MB/s 2024-04-25T10:20:04.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.196 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168314 s, 62.3 MB/s 2024-04-25T10:20:04.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.213 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147508 s, 71.1 MB/s 2024-04-25T10:20:04.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144853 s, 72.4 MB/s 2024-04-25T10:20:04.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.246 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014593 s, 71.9 MB/s 2024-04-25T10:20:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168292 s, 62.3 MB/s 2024-04-25T10:20:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220172 s, 47.6 MB/s 2024-04-25T10:20:04.309 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.309 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.309 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187577 s, 55.9 MB/s 2024-04-25T10:20:04.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.325 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015006 s, 69.9 MB/s 2024-04-25T10:20:04.347 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.347 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.347 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199405 s, 52.6 MB/s 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169841 s, 61.7 MB/s 2024-04-25T10:20:04.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.385 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174977 s, 59.9 MB/s 2024-04-25T10:20:04.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164728 s, 63.7 MB/s 2024-04-25T10:20:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197057 s, 53.2 MB/s 2024-04-25T10:20:04.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.441 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142658 s, 73.5 MB/s 2024-04-25T10:20:04.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.458 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160001 s, 65.5 MB/s 2024-04-25T10:20:04.475 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.475 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.475 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147165 s, 71.3 MB/s 2024-04-25T10:20:04.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143485 s, 73.1 MB/s 2024-04-25T10:20:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020149 s, 52.0 MB/s 2024-04-25T10:20:04.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.535 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020048 s, 52.3 MB/s 2024-04-25T10:20:04.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.552 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015204 s, 69.0 MB/s 2024-04-25T10:20:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168586 s, 62.2 MB/s 2024-04-25T10:20:04.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:04.572 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:04.572 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:04.572 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:20:04.572 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:04.572 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:04.573 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:04.573 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242192 s, 43.3 MB/s 2024-04-25T10:20:04.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.671 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147111 s, 71.3 MB/s 2024-04-25T10:20:04.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142972 s, 73.3 MB/s 2024-04-25T10:20:04.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.706 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171178 s, 61.3 MB/s 2024-04-25T10:20:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152242 s, 68.9 MB/s 2024-04-25T10:20:04.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.742 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170718 s, 61.4 MB/s 2024-04-25T10:20:04.759 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.759 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.759 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152799 s, 68.6 MB/s 2024-04-25T10:20:04.776 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.776 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.776 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016037 s, 65.4 MB/s 2024-04-25T10:20:04.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.798 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195318 s, 53.7 MB/s 2024-04-25T10:20:04.814 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.815 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149748 s, 70.0 MB/s 2024-04-25T10:20:04.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171499 s, 61.1 MB/s 2024-04-25T10:20:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203367 s, 51.6 MB/s 2024-04-25T10:20:04.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.873 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158866 s, 66.0 MB/s 2024-04-25T10:20:04.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.890 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149223 s, 70.3 MB/s 2024-04-25T10:20:04.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152626 s, 68.7 MB/s 2024-04-25T10:20:04.926 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.926 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.926 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175672 s, 59.7 MB/s 2024-04-25T10:20:04.942 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.942 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.943 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143417 s, 73.1 MB/s 2024-04-25T10:20:04.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.961 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166456 s, 63.0 MB/s 2024-04-25T10:20:04.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.978 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152195 s, 68.9 MB/s 2024-04-25T10:20:04.994 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:04.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:04.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146643 s, 71.5 MB/s 2024-04-25T10:20:05.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.011 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145438 s, 72.1 MB/s 2024-04-25T10:20:05.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.030 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170475 s, 61.5 MB/s 2024-04-25T10:20:05.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.046 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147924 s, 70.9 MB/s 2024-04-25T10:20:05.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.063 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146269 s, 71.7 MB/s 2024-04-25T10:20:05.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.085 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203218 s, 51.6 MB/s 2024-04-25T10:20:05.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.102 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147481 s, 71.1 MB/s 2024-04-25T10:20:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156599 s, 67.0 MB/s 2024-04-25T10:20:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183782 s, 57.1 MB/s 2024-04-25T10:20:05.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.157 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155504 s, 67.4 MB/s 2024-04-25T10:20:05.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.177 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179771 s, 58.3 MB/s 2024-04-25T10:20:05.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147289 s, 71.2 MB/s 2024-04-25T10:20:05.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.210 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151891 s, 69.0 MB/s 2024-04-25T10:20:05.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:05.211 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:05.212 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:05.212 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:20:05.212 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:05.212 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:05.212 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:05.212 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:05.280 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.281 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149629 s, 70.1 MB/s 2024-04-25T10:20:05.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01683 s, 62.3 MB/s 2024-04-25T10:20:05.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.316 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149296 s, 70.2 MB/s 2024-04-25T10:20:05.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.334 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168545 s, 62.2 MB/s 2024-04-25T10:20:05.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.356 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01974 s, 53.1 MB/s 2024-04-25T10:20:05.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.373 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151331 s, 69.3 MB/s 2024-04-25T10:20:05.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.391 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165505 s, 63.4 MB/s 2024-04-25T10:20:05.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.410 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169678 s, 61.8 MB/s 2024-04-25T10:20:05.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.426 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147741 s, 71.0 MB/s 2024-04-25T10:20:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170606 s, 61.5 MB/s 2024-04-25T10:20:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144515 s, 72.6 MB/s 2024-04-25T10:20:05.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.482 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189621 s, 55.3 MB/s 2024-04-25T10:20:05.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147944 s, 70.9 MB/s 2024-04-25T10:20:05.518 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.518 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.518 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176866 s, 59.3 MB/s 2024-04-25T10:20:05.540 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.540 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.540 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199247 s, 52.6 MB/s 2024-04-25T10:20:05.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.556 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149161 s, 70.3 MB/s 2024-04-25T10:20:05.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.574 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154746 s, 67.8 MB/s 2024-04-25T10:20:05.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018637 s, 56.3 MB/s 2024-04-25T10:20:05.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145975 s, 71.8 MB/s 2024-04-25T10:20:05.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.627 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148996 s, 70.4 MB/s 2024-04-25T10:20:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151903 s, 69.0 MB/s 2024-04-25T10:20:05.661 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.661 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.661 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149478 s, 70.1 MB/s 2024-04-25T10:20:05.677 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.677 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150958 s, 69.5 MB/s 2024-04-25T10:20:05.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.700 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205823 s, 50.9 MB/s 2024-04-25T10:20:05.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118118 s, 88.8 MB/s 2024-04-25T10:20:05.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.730 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014262 s, 73.5 MB/s 2024-04-25T10:20:05.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.746 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145572 s, 72.0 MB/s 2024-04-25T10:20:05.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.763 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015024 s, 69.8 MB/s 2024-04-25T10:20:05.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.779 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146985 s, 71.3 MB/s 2024-04-25T10:20:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.799 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184119 s, 57.0 MB/s 2024-04-25T10:20:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136574 s, 76.8 MB/s 2024-04-25T10:20:05.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.835 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01867 s, 56.2 MB/s 2024-04-25T10:20:05.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:05.837 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:05.837 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:05.837 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:20:05.837 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:05.837 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:05.837 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:05.837 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:05.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.905 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121484 s, 86.3 MB/s 2024-04-25T10:20:05.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.922 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153546 s, 68.3 MB/s 2024-04-25T10:20:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168677 s, 62.2 MB/s 2024-04-25T10:20:05.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.957 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144065 s, 72.8 MB/s 2024-04-25T10:20:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195686 s, 53.6 MB/s 2024-04-25T10:20:05.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:05.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:05.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147945 s, 70.9 MB/s 2024-04-25T10:20:06.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.017 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019928 s, 52.6 MB/s 2024-04-25T10:20:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180545 s, 58.1 MB/s 2024-04-25T10:20:06.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.059 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204282 s, 51.3 MB/s 2024-04-25T10:20:06.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.082 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220924 s, 47.5 MB/s 2024-04-25T10:20:06.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.104 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200688 s, 52.2 MB/s 2024-04-25T10:20:06.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.120 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144503 s, 72.6 MB/s 2024-04-25T10:20:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169554 s, 61.8 MB/s 2024-04-25T10:20:06.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.157 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164787 s, 63.6 MB/s 2024-04-25T10:20:06.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.176 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167726 s, 62.5 MB/s 2024-04-25T10:20:06.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.195 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181233 s, 57.9 MB/s 2024-04-25T10:20:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167125 s, 62.7 MB/s 2024-04-25T10:20:06.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.231 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157833 s, 66.4 MB/s 2024-04-25T10:20:06.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.247 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143897 s, 72.9 MB/s 2024-04-25T10:20:06.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.267 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174956 s, 59.9 MB/s 2024-04-25T10:20:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147935 s, 70.9 MB/s 2024-04-25T10:20:06.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.302 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016793 s, 62.4 MB/s 2024-04-25T10:20:06.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.319 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150857 s, 69.5 MB/s 2024-04-25T10:20:06.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.338 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175725 s, 59.7 MB/s 2024-04-25T10:20:06.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.354 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143724 s, 73.0 MB/s 2024-04-25T10:20:06.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.373 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172558 s, 60.8 MB/s 2024-04-25T10:20:06.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.391 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.391 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016046 s, 65.3 MB/s 2024-04-25T10:20:06.413 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.413 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.413 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202548 s, 51.8 MB/s 2024-04-25T10:20:06.434 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.435 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197137 s, 53.2 MB/s 2024-04-25T10:20:06.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.451 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150614 s, 69.6 MB/s 2024-04-25T10:20:06.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.467 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013943 s, 75.2 MB/s 2024-04-25T10:20:06.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.483 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146307 s, 71.7 MB/s 2024-04-25T10:20:06.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:06.485 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:06.485 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:06.485 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:20:06.485 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:06.485 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:06.485 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:06.485 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:06.564 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.564 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.564 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196798 s, 53.3 MB/s 2024-04-25T10:20:06.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.581 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151991 s, 69.0 MB/s 2024-04-25T10:20:06.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.601 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175832 s, 59.6 MB/s 2024-04-25T10:20:06.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.622 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190648 s, 55.0 MB/s 2024-04-25T10:20:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192615 s, 54.4 MB/s 2024-04-25T10:20:06.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.664 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201988 s, 51.9 MB/s 2024-04-25T10:20:06.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.682 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158635 s, 66.1 MB/s 2024-04-25T10:20:06.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.699 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153774 s, 68.2 MB/s 2024-04-25T10:20:06.716 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.716 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.716 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148717 s, 70.5 MB/s 2024-04-25T10:20:06.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.733 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148063 s, 70.8 MB/s 2024-04-25T10:20:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.754 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195437 s, 53.7 MB/s 2024-04-25T10:20:06.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.771 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152592 s, 68.7 MB/s 2024-04-25T10:20:06.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.788 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150082 s, 69.9 MB/s 2024-04-25T10:20:06.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151003 s, 69.4 MB/s 2024-04-25T10:20:06.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.821 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148611 s, 70.6 MB/s 2024-04-25T10:20:06.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.838 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152161 s, 68.9 MB/s 2024-04-25T10:20:06.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140554 s, 74.6 MB/s 2024-04-25T10:20:06.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.870 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141576 s, 74.1 MB/s 2024-04-25T10:20:06.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.888 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153905 s, 68.1 MB/s 2024-04-25T10:20:06.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175086 s, 59.9 MB/s 2024-04-25T10:20:06.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.923 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143447 s, 73.1 MB/s 2024-04-25T10:20:06.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.939 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145871 s, 71.9 MB/s 2024-04-25T10:20:06.960 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193114 s, 54.3 MB/s 2024-04-25T10:20:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173422 s, 60.5 MB/s 2024-04-25T10:20:06.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:06.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:06.999 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176513 s, 59.4 MB/s 2024-04-25T10:20:07.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.022 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021788 s, 48.1 MB/s 2024-04-25T10:20:07.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149585 s, 70.1 MB/s 2024-04-25T10:20:07.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.056 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151736 s, 69.1 MB/s 2024-04-25T10:20:07.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.072 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145118 s, 72.3 MB/s 2024-04-25T10:20:07.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.094 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197304 s, 53.1 MB/s 2024-04-25T10:20:07.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.112 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165681 s, 63.3 MB/s 2024-04-25T10:20:07.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.129 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148799 s, 70.5 MB/s 2024-04-25T10:20:07.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:07.130 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:07.130 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:07.130 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:20:07.130 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:07.130 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:07.130 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:07.130 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:07.204 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.204 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.205 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157881 s, 66.4 MB/s 2024-04-25T10:20:07.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154276 s, 68.0 MB/s 2024-04-25T10:20:07.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.236 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120663 s, 86.9 MB/s 2024-04-25T10:20:07.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.254 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168314 s, 62.3 MB/s 2024-04-25T10:20:07.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.270 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143683 s, 73.0 MB/s 2024-04-25T10:20:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159707 s, 65.7 MB/s 2024-04-25T10:20:07.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.304 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143846 s, 72.9 MB/s 2024-04-25T10:20:07.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.321 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147298 s, 71.2 MB/s 2024-04-25T10:20:07.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.342 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194107 s, 54.0 MB/s 2024-04-25T10:20:07.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163184 s, 64.3 MB/s 2024-04-25T10:20:07.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.378 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015983 s, 65.6 MB/s 2024-04-25T10:20:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.394 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143672 s, 73.0 MB/s 2024-04-25T10:20:07.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.418 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022345 s, 46.9 MB/s 2024-04-25T10:20:07.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.437 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173539 s, 60.4 MB/s 2024-04-25T10:20:07.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.458 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180334 s, 58.1 MB/s 2024-04-25T10:20:07.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149841 s, 70.0 MB/s 2024-04-25T10:20:07.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153212 s, 68.4 MB/s 2024-04-25T10:20:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155093 s, 67.6 MB/s 2024-04-25T10:20:07.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198515 s, 52.8 MB/s 2024-04-25T10:20:07.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150411 s, 69.7 MB/s 2024-04-25T10:20:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.563 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146082 s, 71.8 MB/s 2024-04-25T10:20:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167335 s, 62.7 MB/s 2024-04-25T10:20:07.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.597 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145467 s, 72.1 MB/s 2024-04-25T10:20:07.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146694 s, 71.5 MB/s 2024-04-25T10:20:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.639 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227666 s, 46.1 MB/s 2024-04-25T10:20:07.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147038 s, 71.3 MB/s 2024-04-25T10:20:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.671 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146724 s, 71.5 MB/s 2024-04-25T10:20:07.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.688 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147096 s, 71.3 MB/s 2024-04-25T10:20:07.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.707 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174752 s, 60.0 MB/s 2024-04-25T10:20:07.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172655 s, 60.7 MB/s 2024-04-25T10:20:07.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.744 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015832 s, 66.2 MB/s 2024-04-25T10:20:07.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.763 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165843 s, 63.2 MB/s 2024-04-25T10:20:07.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:07.764 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:07.764 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:07.764 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:20:07.764 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:07.764 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:07.764 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:07.764 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:07.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.836 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01474 s, 71.1 MB/s 2024-04-25T10:20:07.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151137 s, 69.4 MB/s 2024-04-25T10:20:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.873 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180879 s, 58.0 MB/s 2024-04-25T10:20:07.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.899 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239433 s, 43.8 MB/s 2024-04-25T10:20:07.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.921 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200007 s, 52.4 MB/s 2024-04-25T10:20:07.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.937 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143879 s, 72.9 MB/s 2024-04-25T10:20:07.955 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.955 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.955 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159423 s, 65.8 MB/s 2024-04-25T10:20:07.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198816 s, 52.7 MB/s 2024-04-25T10:20:07.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:07.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:07.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168155 s, 62.4 MB/s 2024-04-25T10:20:08.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170532 s, 61.5 MB/s 2024-04-25T10:20:08.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.037 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203144 s, 51.6 MB/s 2024-04-25T10:20:08.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155148 s, 67.6 MB/s 2024-04-25T10:20:08.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.075 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186985 s, 56.1 MB/s 2024-04-25T10:20:08.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.091 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149774 s, 70.0 MB/s 2024-04-25T10:20:08.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223798 s, 46.9 MB/s 2024-04-25T10:20:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.132 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141948 s, 73.9 MB/s 2024-04-25T10:20:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.148 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151591 s, 69.2 MB/s 2024-04-25T10:20:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174804 s, 60.0 MB/s 2024-04-25T10:20:08.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144461 s, 72.6 MB/s 2024-04-25T10:20:08.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.201 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147521 s, 71.1 MB/s 2024-04-25T10:20:08.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.217 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144888 s, 72.4 MB/s 2024-04-25T10:20:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171488 s, 61.1 MB/s 2024-04-25T10:20:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014817 s, 70.8 MB/s 2024-04-25T10:20:08.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.268 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144507 s, 72.6 MB/s 2024-04-25T10:20:08.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.282 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116 s, 90.4 MB/s 2024-04-25T10:20:08.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150988 s, 69.4 MB/s 2024-04-25T10:20:08.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.315 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146992 s, 71.3 MB/s 2024-04-25T10:20:08.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.332 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151449 s, 69.2 MB/s 2024-04-25T10:20:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181929 s, 57.6 MB/s 2024-04-25T10:20:08.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148957 s, 70.4 MB/s 2024-04-25T10:20:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198026 s, 53.0 MB/s 2024-04-25T10:20:08.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144398 s, 72.6 MB/s 2024-04-25T10:20:08.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:08.408 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:08.408 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:08.408 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:20:08.408 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:08.408 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:08.408 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:08.408 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:08.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.486 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207275 s, 50.6 MB/s 2024-04-25T10:20:08.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.506 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181014 s, 57.9 MB/s 2024-04-25T10:20:08.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.522 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150671 s, 69.6 MB/s 2024-04-25T10:20:08.539 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.539 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.539 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146446 s, 71.6 MB/s 2024-04-25T10:20:08.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.559 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184065 s, 57.0 MB/s 2024-04-25T10:20:08.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.583 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225824 s, 46.4 MB/s 2024-04-25T10:20:08.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.604 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.604 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185502 s, 56.5 MB/s 2024-04-25T10:20:08.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208062 s, 50.4 MB/s 2024-04-25T10:20:08.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150224 s, 69.8 MB/s 2024-04-25T10:20:08.659 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.660 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.660 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147661 s, 71.0 MB/s 2024-04-25T10:20:08.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.676 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144698 s, 72.5 MB/s 2024-04-25T10:20:08.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141615 s, 74.0 MB/s 2024-04-25T10:20:08.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143847 s, 72.9 MB/s 2024-04-25T10:20:08.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148059 s, 70.8 MB/s 2024-04-25T10:20:08.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.743 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161688 s, 64.9 MB/s 2024-04-25T10:20:08.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.761 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161814 s, 64.8 MB/s 2024-04-25T10:20:08.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015552 s, 67.4 MB/s 2024-04-25T10:20:08.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199878 s, 52.5 MB/s 2024-04-25T10:20:08.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144189 s, 72.7 MB/s 2024-04-25T10:20:08.832 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.833 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145719 s, 72.0 MB/s 2024-04-25T10:20:08.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.849 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147398 s, 71.1 MB/s 2024-04-25T10:20:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196813 s, 53.3 MB/s 2024-04-25T10:20:08.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.890 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146206 s, 71.7 MB/s 2024-04-25T10:20:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146145 s, 71.7 MB/s 2024-04-25T10:20:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236574 s, 44.3 MB/s 2024-04-25T10:20:08.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.947 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164817 s, 63.6 MB/s 2024-04-25T10:20:08.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.964 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151835 s, 69.1 MB/s 2024-04-25T10:20:08.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:08.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:08.980 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142166 s, 73.8 MB/s 2024-04-25T10:20:08.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.000 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179432 s, 58.4 MB/s 2024-04-25T10:20:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.016 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147728 s, 71.0 MB/s 2024-04-25T10:20:09.033 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.033 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.033 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146049 s, 71.8 MB/s 2024-04-25T10:20:09.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193653 s, 54.1 MB/s 2024-04-25T10:20:09.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:09.055 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:09.055 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:09.055 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:20:09.055 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:09.056 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:09.056 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:09.056 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:09.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.126 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.126 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017367 s, 60.4 MB/s 2024-04-25T10:20:09.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.143 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150968 s, 69.5 MB/s 2024-04-25T10:20:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163166 s, 64.3 MB/s 2024-04-25T10:20:09.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.177 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.178 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144018 s, 72.8 MB/s 2024-04-25T10:20:09.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.194 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141865 s, 73.9 MB/s 2024-04-25T10:20:09.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.210 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142447 s, 73.6 MB/s 2024-04-25T10:20:09.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145594 s, 72.0 MB/s 2024-04-25T10:20:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176623 s, 59.4 MB/s 2024-04-25T10:20:09.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.265 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177036 s, 59.2 MB/s 2024-04-25T10:20:09.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.281 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146671 s, 71.5 MB/s 2024-04-25T10:20:09.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161881 s, 64.8 MB/s 2024-04-25T10:20:09.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.317 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156462 s, 67.0 MB/s 2024-04-25T10:20:09.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.334 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154971 s, 67.7 MB/s 2024-04-25T10:20:09.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.351 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150462 s, 69.7 MB/s 2024-04-25T10:20:09.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168294 s, 62.3 MB/s 2024-04-25T10:20:09.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.386 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144756 s, 72.4 MB/s 2024-04-25T10:20:09.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157144 s, 66.7 MB/s 2024-04-25T10:20:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172472 s, 60.8 MB/s 2024-04-25T10:20:09.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.442 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177511 s, 59.1 MB/s 2024-04-25T10:20:09.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167852 s, 62.5 MB/s 2024-04-25T10:20:09.476 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.476 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.476 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142504 s, 73.6 MB/s 2024-04-25T10:20:09.496 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.496 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.497 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183084 s, 57.3 MB/s 2024-04-25T10:20:09.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.516 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.516 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174749 s, 60.0 MB/s 2024-04-25T10:20:09.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.532 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.532 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142274 s, 73.7 MB/s 2024-04-25T10:20:09.549 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.549 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.549 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154566 s, 67.8 MB/s 2024-04-25T10:20:09.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188026 s, 55.8 MB/s 2024-04-25T10:20:09.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.592 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202596 s, 51.8 MB/s 2024-04-25T10:20:09.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161501 s, 64.9 MB/s 2024-04-25T10:20:09.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.629 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017847 s, 58.8 MB/s 2024-04-25T10:20:09.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.653 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215702 s, 48.6 MB/s 2024-04-25T10:20:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191959 s, 54.6 MB/s 2024-04-25T10:20:09.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:09.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:09.696 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205207 s, 51.1 MB/s 2024-04-25T10:20:09.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:09.697 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:20:09.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853e00640 1 Processor -- start 2024-04-25T10:20:09.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853e00640 1 -- start start 2024-04-25T10:20:09.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8540ad840 0x7fc8540cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85406ca20 con 0x7fc8540cf7f0 2024-04-25T10:20:09.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8540ade00 con 0x7fc854146660 2024-04-25T10:20:09.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8540cfc20 con 0x7fc8540ad840 2024-04-25T10:20:09.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc8540cf7f0 0x7fc85406cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52972/0 (socket says 172.21.15.87:52972) 2024-04-25T10:20:09.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc853400640 1 -- 172.21.15.87:0/2925549638 learned_addr learned my addr 172.21.15.87:0/2925549638 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:09.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc84a000640 1 --2- 172.21.15.87:0/2925549638 >> v2:172.21.15.169:3300/0 conn(0x7fc8540ad840 0x7fc8540cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.879+0000 7fc852000640 1 -- 172.21.15.87:0/2925549638 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465634706 0 0) 0x7fc85406ca20 con 0x7fc8540cf7f0 2024-04-25T10:20:09.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc852000640 1 -- 172.21.15.87:0/2925549638 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc838003530 con 0x7fc8540cf7f0 2024-04-25T10:20:09.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc84a000640 1 -- 172.21.15.87:0/2925549638 >> v1:172.21.15.169:6789/0 conn(0x7fc854146660 legacy=0x7fc854070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc84a000640 1 -- 172.21.15.87:0/2925549638 >> v1:172.21.15.87:6789/0 conn(0x7fc8540cf7f0 legacy=0x7fc85406cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc84a000640 1 -- 172.21.15.87:0/2925549638 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540ae0e0 con 0x7fc8540ad840 2024-04-25T10:20:09.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc84a000640 1 --2- 172.21.15.87:0/2925549638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540ad840 0x7fc8540cf2b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097c0 tx=0x7fc83c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c056b9b37a1b9c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:09.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc852000640 1 -- 172.21.15.87:0/2925549638 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0a7070 con 0x7fc8540ad840 2024-04-25T10:20:09.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc852000640 1 -- 172.21.15.87:0/2925549638 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c0b0070 con 0x7fc8540ad840 2024-04-25T10:20:09.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.883+0000 7fc852000640 1 -- 172.21.15.87:0/2925549638 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0ab890 con 0x7fc8540ad840 2024-04-25T10:20:09.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- 172.21.15.87:0/2925549638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540ad840 msgr2=0x7fc8540cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 --2- 172.21.15.87:0/2925549638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540ad840 0x7fc8540cf2b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097c0 tx=0x7fc83c01c470 comp rx=0 tx=0).stop 2024-04-25T10:20:09.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- 172.21.15.87:0/2925549638 shutdown_connections 2024-04-25T10:20:09.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 --2- 172.21.15.87:0/2925549638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540ad840 0x7fc8540cf2b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- 172.21.15.87:0/2925549638 >> 172.21.15.87:0/2925549638 conn(0x7fc854074450 msgr2=0x7fc8540669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:09.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- 172.21.15.87:0/2925549638 shutdown_connections 2024-04-25T10:20:09.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- 172.21.15.87:0/2925549638 wait complete. 2024-04-25T10:20:09.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 Processor -- start 2024-04-25T10:20:09.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- start start 2024-04-25T10:20:09.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8540ad840 0x7fc85415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540cf7f0 0x7fc85415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc854146660 0x7fc85415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85406dfc0 con 0x7fc854146660 2024-04-25T10:20:09.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85406de40 con 0x7fc8540cf7f0 2024-04-25T10:20:09.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85406e140 con 0x7fc8540ad840 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc854146660 0x7fc85415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc854146660 0x7fc85415c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57426/0 (socket says 172.21.15.87:57426) 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc84a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540cf7f0 0x7fc85415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 -- 172.21.15.87:0/2759346605 learned_addr learned my addr 172.21.15.87:0/2759346605 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc852a00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8540ad840 0x7fc85415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 -- 172.21.15.87:0/2759346605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8540ad840 msgr2=0x7fc85415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8540ad840 0x7fc85415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 -- 172.21.15.87:0/2759346605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540cf7f0 msgr2=0x7fc85415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540cf7f0 0x7fc85415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.887+0000 7fc853400640 1 -- 172.21.15.87:0/2759346605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85415c010 con 0x7fc854146660 2024-04-25T10:20:09.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.891+0000 7fc853400640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc854146660 0x7fc85415c800 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00cb70 tx=0x7fc84c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:09.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.891+0000 7fc84b400640 1 -- 172.21.15.87:0/2759346605 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c017070 con 0x7fc854146660 2024-04-25T10:20:09.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.891+0000 7fc84b400640 1 -- 172.21.15.87:0/2759346605 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00ace0 con 0x7fc854146660 2024-04-25T10:20:09.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.891+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85415d260 con 0x7fc854146660 2024-04-25T10:20:09.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.891+0000 7fc84b400640 1 -- 172.21.15.87:0/2759346605 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c012620 con 0x7fc854146660 2024-04-25T10:20:09.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.891+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85415d820 con 0x7fc854146660 2024-04-25T10:20:09.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.891+0000 7fc84b400640 1 -- 172.21.15.87:0/2759346605 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc84c01a070 con 0x7fc854146660 2024-04-25T10:20:09.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc84b400640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc830066f90 0x7fc830069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc84b400640 1 -- 172.21.15.87:0/2759346605 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc84c084d50 con 0x7fc854146660 2024-04-25T10:20:09.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc852a00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc830066f90 0x7fc830069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 Processor -- start 2024-04-25T10:20:09.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- start start 2024-04-25T10:20:09.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc852a00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc830066f90 0x7fc830069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc844007fd0 tx=0x7fc844009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfcc3f0 0x55779bfcc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bf82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779bf83900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55779bdd0d50 con 0x55779bf83520 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55779bd58e40 con 0x55779bfcc3f0 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55779bdccdf0 con 0x55779bfccd10 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779bf83900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc852a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfcc3f0 0x55779bfcc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779bf83900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57434/0 (socket says 172.21.15.87:57434) 2024-04-25T10:20:09.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 -- 172.21.15.87:0/2767017446 learned_addr learned my addr 172.21.15.87:0/2767017446 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc84a000640 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bf82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 -- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 msgr2=0x55779bf82fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bf82fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 -- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfcc3f0 msgr2=0x55779bfcc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfcc3f0 0x55779bfcc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 -- 172.21.15.87:0/2767017446 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55779bd6dd90 con 0x55779bf83520 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc852a00640 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfcc3f0 0x55779bfcc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc853400640 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779bf83900 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc84c051b60 tx=0x7fc84c0557c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2845d3cc5f918b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc843400640 1 -- 172.21.15.87:0/2767017446 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c06d1c0 con 0x55779bf83520 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc843400640 1 -- 172.21.15.87:0/2767017446 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c053a00 con 0x55779bf83520 2024-04-25T10:20:09.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc843400640 1 -- 172.21.15.87:0/2767017446 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c05d8d0 con 0x55779bf83520 2024-04-25T10:20:09.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- 172.21.15.87:0/2767017446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 msgr2=0x55779bf83900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779bf83900 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc84c051b60 tx=0x7fc84c0557c0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- 172.21.15.87:0/2767017446 shutdown_connections 2024-04-25T10:20:09.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779bf83900 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bf82fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2767017446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfcc3f0 0x55779bfcc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- 172.21.15.87:0/2767017446 >> 172.21.15.87:0/2767017446 conn(0x55779bf7f4d0 msgr2=0x55779bf818c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- 172.21.15.87:0/2767017446 shutdown_connections 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.895+0000 7fc85d06a000 1 -- 172.21.15.87:0/2767017446 wait complete. 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 -- start start 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 Processor -- start 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779bf89020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779bfba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bfba600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55779bd76610 con 0x55779bfcc3f0 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55779bd8b9a0 con 0x55779bf83520 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55779bdd5380 con 0x55779bfccd10 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 10 client.0 _ll_register_callbacks cb 0x55779be850d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779bfba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779bfba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57448/0 (socket says 172.21.15.87:57448) 2024-04-25T10:20:09.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 -- 172.21.15.87:0/1097711300 learned_addr learned my addr 172.21.15.87:0/1097711300 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc852a00640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779bf89020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc853400640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bfba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 -- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 msgr2=0x55779bfba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bfba600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 -- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 msgr2=0x55779bf89020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779bf89020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 -- 172.21.15.87:0/1097711300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55779bdcbd40 con 0x55779bfcc3f0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc852a00640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779bf89020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc84a000640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779bfba0c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097c0 tx=0x7fc83c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db6a5cecda6a408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc837e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc837e00640 1 -- 172.21.15.87:0/1097711300 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0abbd0 con 0x55779bfcc3f0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc837e00640 1 -- 172.21.15.87:0/1097711300 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc83c0abd70 con 0x55779bfcc3f0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc84c053a00 con 0x55779bfcc3f0 2024-04-25T10:20:09.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc837e00640 1 -- 172.21.15.87:0/1097711300 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc83c0c1420 con 0x55779bfcc3f0 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 10 client.8145 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55779be53f00 con 0x55779bfcc3f0 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc837e00640 1 -- 172.21.15.87:0/1097711300 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c053a00 con 0x55779bfcc3f0 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc837e00640 1 -- 172.21.15.87:0/1097711300 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc83c0c9020 con 0x55779bfcc3f0 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 10 client.8145 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 10 client.8145 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 20 client.8145 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd3570 0x55779bfd3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 4 client.8145 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55779bd6ca20 con 0x55779bfd3570 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.899+0000 7fc852a00640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd3570 0x55779bfd3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc852a00640 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd3570 0x55779bfd3980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb389f809817419 server_cookie=54f24f838cbca43e in_seq=0 out_seq=0 2024-04-25T10:20:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc837e00640 10 client.8145 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:09.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc837e00640 1 -- 172.21.15.87:0/1097711300 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55779bd6ca20 con 0x55779bfd3570 2024-04-25T10:20:09.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc837e00640 10 client.8145 handle_command_reply: tid=0 2024-04-25T10:20:09.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 client.8145 shutdown 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 msgr2=0x55779bfba0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779bfba0c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097c0 tx=0x7fc83c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 shutdown_connections 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd3570 0x55779bfd3980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779bfba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779bfba0c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 --2- 172.21.15.87:0/1097711300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779bf89020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc837e00640 0 client.8145 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 >> 172.21.15.87:0/1097711300 conn(0x55779bf7f4d0 msgr2=0x55779bf82150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 shutdown_connections 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 1 -- 172.21.15.87:0/1097711300 wait complete. 2024-04-25T10:20:09.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.903+0000 7fc85d06a000 20 client.8145 trim_cache size 0 max 16384 2024-04-25T10:20:09.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 Processor -- start 2024-04-25T10:20:09.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- start start 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779c031450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779c031990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779c031ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854202900 con 0x55779bfcc3f0 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854156ef0 con 0x55779bf83520 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8541562f0 con 0x55779bfccd10 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779c031990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779c031990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57462/0 (socket says 172.21.15.87:57462) 2024-04-25T10:20:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 -- 172.21.15.87:0/3630878580 learned_addr learned my addr 172.21.15.87:0/3630878580 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc853400640 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779c031ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc852a00640 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779c031450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 -- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 msgr2=0x55779c031ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779c031ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 -- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 msgr2=0x55779c031450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779c031450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 -- 172.21.15.87:0/3630878580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55779bfccb40 con 0x55779bfcc3f0 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc852a00640 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779c031450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc84a000640 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779c031990 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009160 tx=0x7fc83c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6855a25dfad8bc70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc842a00640 1 -- 172.21.15.87:0/3630878580 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c004140 con 0x55779bfcc3f0 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc842a00640 1 -- 172.21.15.87:0/3630878580 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c000e50 con 0x55779bfcc3f0 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc842a00640 1 -- 172.21.15.87:0/3630878580 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0c18f0 con 0x55779bfcc3f0 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- 172.21.15.87:0/3630878580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 msgr2=0x55779c031990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779c031990 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009160 tx=0x7fc83c009190 comp rx=0 tx=0).stop 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- 172.21.15.87:0/3630878580 shutdown_connections 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfccd10 0x55779c031ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bfcc3f0 0x55779c031990 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 --2- 172.21.15.87:0/3630878580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bf83520 0x55779c031450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- 172.21.15.87:0/3630878580 >> 172.21.15.87:0/3630878580 conn(0x55779bf7f4d0 msgr2=0x55779bf88470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:09.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- 172.21.15.87:0/3630878580 shutdown_connections 2024-04-25T10:20:09.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- 172.21.15.87:0/3630878580 wait complete. 2024-04-25T10:20:09.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 -- start start 2024-04-25T10:20:09.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.919+0000 7fc85d06a000 1 Processor -- start 2024-04-25T10:20:09.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779c0363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfcc3f0 0x55779c036920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfccd10 0x55779bfd1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55779bd87dc0 con 0x55779bf83520 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8541568f0 con 0x55779bfccd10 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55779bdd0060 con 0x55779bfcc3f0 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 10 client.0 _ll_register_callbacks cb 0x7fc844007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779c0363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779c0363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57464/0 (socket says 172.21.15.87:57464) 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 -- 172.21.15.87:0/2629982136 learned_addr learned my addr 172.21.15.87:0/2629982136 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc84a000640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfcc3f0 0x55779c036920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc853400640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfccd10 0x55779bfd1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 -- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfcc3f0 msgr2=0x55779c036920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfcc3f0 0x55779c036920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 -- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfccd10 msgr2=0x55779bfd1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:09.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfccd10 0x55779bfd1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 -- 172.21.15.87:0/2629982136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55779be2a330 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc853400640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfccd10 0x55779bfd1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc852a00640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779c0363e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc8440015e0 tx=0x7fc844001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9682dddc7ad5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc837e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc837e00640 1 -- 172.21.15.87:0/2629982136 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844014720 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc837e00640 1 -- 172.21.15.87:0/2629982136 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8440148c0 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc83c000e50 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc837e00640 1 -- 172.21.15.87:0/2629982136 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc844038050 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 10 client.8151 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55779bd9d5f0 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc837e00640 1 -- 172.21.15.87:0/2629982136 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c000e50 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc837e00640 1 -- 172.21.15.87:0/2629982136 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc84403b020 con 0x55779bf83520 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 10 client.8151 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 10 client.8151 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:09.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 20 client.8151 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd1960 0x55779bfd1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 4 client.8151 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:20:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55779bdde290 con 0x55779bfd1960 2024-04-25T10:20:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc84a000640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd1960 0x55779bfd1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc837e00640 10 client.8151 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.923+0000 7fc84a000640 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd1960 0x55779bfd1d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2485a3cef19bea3 server_cookie=f5b4090e26b09636 in_seq=0 out_seq=0 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc837e00640 1 -- 172.21.15.87:0/2629982136 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55779bdde290 con 0x55779bfd1960 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc837e00640 10 client.8151 handle_command_reply: tid=0 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 client.8151 shutdown 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 msgr2=0x55779c0363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779c0363e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc8440015e0 tx=0x7fc844001090 comp rx=0 tx=0).stop 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 shutdown_connections 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55779bfd1960 0x55779bfd1d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55779bfccd10 0x55779bfd1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55779bfcc3f0 0x55779c036920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 --2- 172.21.15.87:0/2629982136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55779bf83520 0x55779c0363e0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 >> 172.21.15.87:0/2629982136 conn(0x55779bf7f4d0 msgr2=0x55779bfcb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc837e00640 0 client.8151 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:09.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 shutdown_connections 2024-04-25T10:20:09.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 1 -- 172.21.15.87:0/2629982136 wait complete. 2024-04-25T10:20:09.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.927+0000 7fc85d06a000 20 client.8151 trim_cache size 0 max 16384 2024-04-25T10:20:09.933 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:20:09.933 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:20:09.933 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "f8a4fa23-2d0c-4cca-b622-5347410cf725", 2024-04-25T10:20:09.933 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:20:09.933 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:20:09.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc830066f90 msgr2=0x7fc830069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc830066f90 0x7fc830069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc844007fd0 tx=0x7fc844009040 comp rx=0 tx=0).stop 2024-04-25T10:20:09.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc854146660 msgr2=0x7fc85415c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:09.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc854146660 0x7fc85415c800 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00cb70 tx=0x7fc84c00a430 comp rx=0 tx=0).stop 2024-04-25T10:20:09.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 shutdown_connections 2024-04-25T10:20:09.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc830066f90 0x7fc830069450 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc854146660 0x7fc85415c800 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8540cf7f0 0x7fc85415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 --2- 172.21.15.87:0/2759346605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8540ad840 0x7fc85415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:09.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 >> 172.21.15.87:0/2759346605 conn(0x7fc854074450 msgr2=0x7fc8541615f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:09.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 shutdown_connections 2024-04-25T10:20:09.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:09.931+0000 7fc853e00640 1 -- 172.21.15.87:0/2759346605 wait complete. 2024-04-25T10:20:09.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:09.955 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T10:20:10.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f4092600640 1 Processor -- start 2024-04-25T10:20:10.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f4092600640 1 -- start start 2024-04-25T10:20:10.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f4092600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f408c0b38d0 0x7f408c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f4092600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c15b080 con 0x7f408c0aef80 2024-04-25T10:20:10.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f4092600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c15adb0 con 0x7f408c070350 2024-04-25T10:20:10.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f4092600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c15eb60 con 0x7f408c0b38d0 2024-04-25T10:20:10.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f408b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f408c0aef80 0x7f408c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52980/0 (socket says 172.21.15.87:52980) 2024-04-25T10:20:10.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f408b400640 1 -- 172.21.15.87:0/2730064894 learned_addr learned my addr 172.21.15.87:0/2730064894 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:10.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f408be00640 1 --2- 172.21.15.87:0/2730064894 >> v2:172.21.15.87:3300/0 conn(0x7f408c0b38d0 0x7f408c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f408be00640 1 -- 172.21.15.87:0/2730064894 >> v1:172.21.15.169:6789/0 conn(0x7f408c070350 legacy=0x7f408c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f408a000640 1 -- 172.21.15.87:0/2730064894 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266068806 0 0) 0x7f408c15b080 con 0x7f408c0aef80 2024-04-25T10:20:10.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f408be00640 1 -- 172.21.15.87:0/2730064894 >> v1:172.21.15.87:6789/0 conn(0x7f408c0aef80 legacy=0x7f408c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.127+0000 7f408be00640 1 -- 172.21.15.87:0/2730064894 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c15ee40 con 0x7f408c0b38d0 2024-04-25T10:20:10.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f408be00640 1 --2- 172.21.15.87:0/2730064894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c0b38d0 0x7f408c15e480 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f4084009d90 tx=0x7f408401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6c0cb835be881a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f408a000640 1 -- 172.21.15.87:0/2730064894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840b2070 con 0x7f408c0b38d0 2024-04-25T10:20:10.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f408a000640 1 -- 172.21.15.87:0/2730064894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40840a5a10 con 0x7f408c0b38d0 2024-04-25T10:20:10.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f408a000640 1 -- 172.21.15.87:0/2730064894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840a5de0 con 0x7f408c0b38d0 2024-04-25T10:20:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f4092600640 1 -- 172.21.15.87:0/2730064894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c0b38d0 msgr2=0x7f408c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f4092600640 1 --2- 172.21.15.87:0/2730064894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c0b38d0 0x7f408c15e480 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f4084009d90 tx=0x7f408401c970 comp rx=0 tx=0).stop 2024-04-25T10:20:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f4092600640 1 -- 172.21.15.87:0/2730064894 shutdown_connections 2024-04-25T10:20:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f4092600640 1 --2- 172.21.15.87:0/2730064894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c0b38d0 0x7f408c15e480 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f4092600640 1 -- 172.21.15.87:0/2730064894 >> 172.21.15.87:0/2730064894 conn(0x7f408c0669a0 msgr2=0x7f408c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f4092600640 1 -- 172.21.15.87:0/2730064894 shutdown_connections 2024-04-25T10:20:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.131+0000 7f4092600640 1 -- 172.21.15.87:0/2730064894 wait complete. 2024-04-25T10:20:10.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 Processor -- start 2024-04-25T10:20:10.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 -- start start 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c070350 0x7f408c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408c0aef80 0x7f408c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0cce90 0x7f408c15ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c1689b0 con 0x7f408c070350 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c168830 con 0x7f408c0cce90 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c168b30 con 0x7f408c0aef80 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c070350 0x7f408c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c070350 0x7f408c1563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57492/0 (socket says 172.21.15.87:57492) 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 -- 172.21.15.87:0/3291149024 learned_addr learned my addr 172.21.15.87:0/3291149024 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408be00640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0cce90 0x7f408c15ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408aa00640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408c0aef80 0x7f408c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 -- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408c0aef80 msgr2=0x7f408c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408c0aef80 0x7f408c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 -- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0cce90 msgr2=0x7f408c15ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0cce90 0x7f408c15ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 -- 172.21.15.87:0/3291149024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c162c60 con 0x7f408c070350 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408aa00640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408c0aef80 0x7f408c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f408b400640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c070350 0x7f408c1563f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f407400c4a0 tx=0x7f407400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4083400640 1 -- 172.21.15.87:0/3291149024 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074018070 con 0x7f408c070350 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4083400640 1 -- 172.21.15.87:0/3291149024 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40740074e0 con 0x7f408c070350 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c160bf0 con 0x7f408c070350 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4083400640 1 -- 172.21.15.87:0/3291149024 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074004a30 con 0x7f408c070350 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.135+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c1611b0 con 0x7f408c070350 2024-04-25T10:20:10.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4083400640 1 -- 172.21.15.87:0/3291149024 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f407401b070 con 0x7f408c070350 2024-04-25T10:20:10.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4083400640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4068066f90 0x7f4068069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4083400640 1 -- 172.21.15.87:0/3291149024 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4074084280 con 0x7f408c070350 2024-04-25T10:20:10.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408aa00640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4068066f90 0x7f4068069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 Processor -- start 2024-04-25T10:20:10.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408aa00640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4068066f90 0x7f4068069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f408c15e7d0 tx=0x7f407c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 -- start start 2024-04-25T10:20:10.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add048e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add003850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add003d90 0x557add004170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557adcee61f0 con 0x557add048a40 2024-04-25T10:20:10.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557adce4ce90 con 0x557add049360 2024-04-25T10:20:10.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557adcdd4f50 con 0x557add003d90 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add048e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add048e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57506/0 (socket says 172.21.15.87:57506) 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 -- 172.21.15.87:0/3271732880 learned_addr learned my addr 172.21.15.87:0/3271732880 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408be00640 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add003d90 0x557add004170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408aa00640 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add003850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 -- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add003d90 msgr2=0x557add004170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add003d90 0x557add004170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 -- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 msgr2=0x557add003850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add003850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 -- 172.21.15.87:0/3271732880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557adce248b0 con 0x557add048a40 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.139+0000 7f408b400640 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add048e20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4074050fe0 tx=0x7f4074055340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8e0d00b1ba13e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f407be00640 1 -- 172.21.15.87:0/3271732880 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740840c0 con 0x557add048a40 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f407be00640 1 -- 172.21.15.87:0/3271732880 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407405d290 con 0x557add048a40 2024-04-25T10:20:10.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f407be00640 1 -- 172.21.15.87:0/3271732880 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740511f0 con 0x557add048a40 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- 172.21.15.87:0/3271732880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 msgr2=0x557add048e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add048e20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4074050fe0 tx=0x7f4074055340 comp rx=0 tx=0).stop 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- 172.21.15.87:0/3271732880 shutdown_connections 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add003d90 0x557add004170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add003850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 --2- 172.21.15.87:0/3271732880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add048e20 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- 172.21.15.87:0/3271732880 >> 172.21.15.87:0/3271732880 conn(0x557adcfffd40 msgr2=0x557add002130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- 172.21.15.87:0/3271732880 shutdown_connections 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- 172.21.15.87:0/3271732880 wait complete. 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- start start 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 Processor -- start 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add003d90 0x557add0501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add050700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add009dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557adce07540 con 0x557add003d90 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557adce48f30 con 0x557add049360 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557adce197c0 con 0x557add048a40 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 10 client.0 _ll_register_callbacks cb 0x557adcf01180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add003d90 0x557add0501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add003d90 0x557add0501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57510/0 (socket says 172.21.15.87:57510) 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 -- 172.21.15.87:0/1420921257 learned_addr learned my addr 172.21.15.87:0/1420921257 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408aa00640 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add050700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408be00640 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add009dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 -- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 msgr2=0x557add050700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add050700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 -- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 msgr2=0x557add009dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add009dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 -- 172.21.15.87:0/1420921257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557adcdd51a0 con 0x557add003d90 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f408b400640 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add003d90 0x557add0501c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4074054d40 tx=0x7f4074057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dfed90d34c4871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4078c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4078c00640 1 -- 172.21.15.87:0/1420921257 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407405ce00 con 0x557add003d90 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4078c00640 1 -- 172.21.15.87:0/1420921257 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f407405cfa0 con 0x557add003d90 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f407405d290 con 0x557add003d90 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4078c00640 1 -- 172.21.15.87:0/1420921257 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4074068b40 con 0x557add003d90 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 10 client.8166 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.143+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x557adcecf840 con 0x557add003d90 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4078c00640 1 -- 172.21.15.87:0/1420921257 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407405d290 con 0x557add003d90 2024-04-25T10:20:10.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4078c00640 1 -- 172.21.15.87:0/1420921257 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4074068ce0 con 0x557add003d90 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 10 client.8166 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 10 client.8166 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 20 client.8166 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add00ab60 0x557add00af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 4 client.8166 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557adcde8be0 con 0x557add00ab60 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f408aa00640 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add00ab60 0x557add00af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f408aa00640 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add00ab60 0x557add00af70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d8ac5312d2070a server_cookie=926ded0e141711ca in_seq=0 out_seq=0 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4078c00640 10 client.8166 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4078c00640 1 -- 172.21.15.87:0/1420921257 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557adcde8be0 con 0x557add00ab60 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4078c00640 10 client.8166 handle_command_reply: tid=0 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 client.8166 shutdown 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add003d90 msgr2=0x557add0501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add003d90 0x557add0501c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4074054d40 tx=0x7f4074057040 comp rx=0 tx=0).stop 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 shutdown_connections 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add00ab60 0x557add00af70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add049360 0x557add009dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add050700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 --2- 172.21.15.87:0/1420921257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add003d90 0x557add0501c0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 >> 172.21.15.87:0/1420921257 conn(0x557adcfffd40 msgr2=0x557add0029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4078c00640 0 client.8166 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 shutdown_connections 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 1 -- 172.21.15.87:0/1420921257 wait complete. 2024-04-25T10:20:10.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.147+0000 7f4095af6000 20 client.8166 trim_cache size 0 max 16384 2024-04-25T10:20:10.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 Processor -- start 2024-04-25T10:20:10.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 -- start start 2024-04-25T10:20:10.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add003290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add002130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add049360 0x557add002670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c202fb0 con 0x557add048a40 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c164330 con 0x557add003d90 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c158790 con 0x557add049360 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f408aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add002130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f408aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add002130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57512/0 (socket says 172.21.15.87:57512) 2024-04-25T10:20:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f408aa00640 1 -- 172.21.15.87:0/744728925 learned_addr learned my addr 172.21.15.87:0/744728925 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f408aa00640 1 -- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add049360 msgr2=0x557add002670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f408aa00640 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add049360 0x557add002670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f408b400640 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add003290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.163+0000 7f408aa00640 1 -- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 msgr2=0x557add003290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408aa00640 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add003290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408aa00640 1 -- 172.21.15.87:0/744728925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557adcea6a10 con 0x557add048a40 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408b400640 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add003290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408aa00640 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add002130 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f407c000f70 tx=0x7f407c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92dfdee103efbef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f407b400640 1 -- 172.21.15.87:0/744728925 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c007030 con 0x557add048a40 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f407b400640 1 -- 172.21.15.87:0/744728925 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c02b070 con 0x557add048a40 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f407b400640 1 -- 172.21.15.87:0/744728925 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0073d0 con 0x557add048a40 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- 172.21.15.87:0/744728925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 msgr2=0x557add002130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add002130 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f407c000f70 tx=0x7f407c013000 comp rx=0 tx=0).stop 2024-04-25T10:20:10.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- 172.21.15.87:0/744728925 shutdown_connections 2024-04-25T10:20:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add049360 0x557add002670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add048a40 0x557add002130 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 --2- 172.21.15.87:0/744728925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add003290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- 172.21.15.87:0/744728925 >> 172.21.15.87:0/744728925 conn(0x557adcfffd40 msgr2=0x557add047150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- 172.21.15.87:0/744728925 shutdown_connections 2024-04-25T10:20:10.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- 172.21.15.87:0/744728925 wait complete. 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- start start 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 Processor -- start 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add0a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add0a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add049360 0x557add04ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557adce48f30 con 0x557add049360 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c163d30 con 0x557add003d90 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557adce10720 con 0x557add048a40 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f4095af6000 10 client.0 _ll_register_callbacks cb 0x557adce0fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add049360 0x557add04ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add049360 0x557add04ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57524/0 (socket says 172.21.15.87:57524) 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 -- 172.21.15.87:0/2219098977 learned_addr learned my addr 172.21.15.87:0/2219098977 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408b400640 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add0a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408aa00640 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add0a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 -- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 msgr2=0x557add0a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add0a99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 -- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 msgr2=0x557add0a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add0a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.167+0000 7f408be00640 1 -- 172.21.15.87:0/2219098977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557adcde1b00 con 0x557add049360 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f408be00640 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add049360 0x557add04ded0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f4084009d90 tx=0x7f408401b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc432e9ecc7e78e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 1 -- 172.21.15.87:0/2219098977 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840b2040 con 0x557add049360 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 1 -- 172.21.15.87:0/2219098977 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40840c3470 con 0x557add049360 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f407c02b070 con 0x557add049360 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 1 -- 172.21.15.87:0/2219098977 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40840cb050 con 0x557add049360 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 10 client.8172 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x557adcdf3f80 con 0x557add049360 2024-04-25T10:20:10.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 1 -- 172.21.15.87:0/2219098977 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c02b070 con 0x557add049360 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 1 -- 172.21.15.87:0/2219098977 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f40840ce020 con 0x557add049360 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 10 client.8172 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 10 client.8172 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 20 client.8172 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add04eb50 0x557add04ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 4 client.8172 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x557adce5a3d0 con 0x557add04eb50 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f408b400640 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add04eb50 0x557add04ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f408b400640 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add04eb50 0x557add04ef60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132b9194561767be server_cookie=709cc684c6e92d34 in_seq=0 out_seq=0 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 10 client.8172 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 1 -- 172.21.15.87:0/2219098977 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x557adce5a3d0 con 0x557add04eb50 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 10 client.8172 handle_command_reply: tid=0 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 client.8172 shutdown 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add049360 msgr2=0x557add04ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add049360 0x557add04ded0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f4084009d90 tx=0x7f408401b1b0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 shutdown_connections 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x557add04eb50 0x557add04ef60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557add049360 0x557add04ded0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557add048a40 0x557add0a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 --2- 172.21.15.87:0/2219098977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557add003d90 0x557add0a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4078c00640 0 client.8172 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 >> 172.21.15.87:0/2219098977 conn(0x557adcfffd40 msgr2=0x557add0479e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 shutdown_connections 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 1 -- 172.21.15.87:0/2219098977 wait complete. 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4095af6000 20 client.8172 trim_cache size 0 max 16384 2024-04-25T10:20:10.178 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:20:10.179 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0 2024-04-25T10:20:10.179 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:20:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4068066f90 msgr2=0x7f4068069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4068066f90 0x7f4068069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f408c15e7d0 tx=0x7f407c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c070350 msgr2=0x7f408c1563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c070350 0x7f408c1563f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f407400c4a0 tx=0x7f407400c960 comp rx=0 tx=0).stop 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 shutdown_connections 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4068066f90 0x7f4068069450 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0cce90 0x7f408c15ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408c0aef80 0x7f408c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 --2- 172.21.15.87:0/3291149024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408c070350 0x7f408c1563f0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 >> 172.21.15.87:0/3291149024 conn(0x7f408c0669a0 msgr2=0x7f408c15b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 shutdown_connections 2024-04-25T10:20:10.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.171+0000 7f4092600640 1 -- 172.21.15.87:0/3291149024 wait complete. 2024-04-25T10:20:10.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:10.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:20:10.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.327+0000 7f513de00640 1 Processor -- start 2024-04-25T10:20:10.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.327+0000 7f513de00640 1 -- start start 2024-04-25T10:20:10.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.327+0000 7f513de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5138070350 0x7f513815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.327+0000 7f513de00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513815b080 con 0x7f51380aef80 2024-04-25T10:20:10.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.327+0000 7f513de00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513815adb0 con 0x7f51380b31c0 2024-04-25T10:20:10.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.327+0000 7f513de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513815eb60 con 0x7f5138070350 2024-04-25T10:20:10.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5136a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f51380aef80 0x7f513815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52996/0 (socket says 172.21.15.87:52996) 2024-04-25T10:20:10.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5136a00640 1 -- 172.21.15.87:0/3141133315 learned_addr learned my addr 172.21.15.87:0/3141133315 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:10.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5136000640 1 --2- 172.21.15.87:0/3141133315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5138070350 0x7f513815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5135600640 1 -- 172.21.15.87:0/3141133315 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752822955 0 0) 0x7f513815b080 con 0x7f51380aef80 2024-04-25T10:20:10.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5135600640 1 -- 172.21.15.87:0/3141133315 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f511c003530 con 0x7f51380aef80 2024-04-25T10:20:10.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5135600640 1 -- 172.21.15.87:0/3141133315 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536047726 0 0) 0x7f513815adb0 con 0x7f51380b31c0 2024-04-25T10:20:10.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5136000640 1 -- 172.21.15.87:0/3141133315 >> v1:172.21.15.169:6789/0 conn(0x7f51380b31c0 legacy=0x7f51380cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5136000640 1 -- 172.21.15.87:0/3141133315 >> v1:172.21.15.87:6789/0 conn(0x7f51380aef80 legacy=0x7f513815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5136000640 1 -- 172.21.15.87:0/3141133315 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513815ee40 con 0x7f5138070350 2024-04-25T10:20:10.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5136000640 1 --2- 172.21.15.87:0/3141133315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5138070350 0x7f513815e480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51240097f0 tx=0x7f512401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9df2572eabe64f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.331+0000 7f5135600640 1 -- 172.21.15.87:0/3141133315 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240aa620 con 0x7f5138070350 2024-04-25T10:20:10.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f5135600640 1 -- 172.21.15.87:0/3141133315 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240af070 con 0x7f5138070350 2024-04-25T10:20:10.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f5135600640 1 -- 172.21.15.87:0/3141133315 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240aa9e0 con 0x7f5138070350 2024-04-25T10:20:10.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- 172.21.15.87:0/3141133315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5138070350 msgr2=0x7f513815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 --2- 172.21.15.87:0/3141133315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5138070350 0x7f513815e480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51240097f0 tx=0x7f512401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- 172.21.15.87:0/3141133315 shutdown_connections 2024-04-25T10:20:10.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 --2- 172.21.15.87:0/3141133315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5138070350 0x7f513815e480 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- 172.21.15.87:0/3141133315 >> 172.21.15.87:0/3141133315 conn(0x7f51380669a0 msgr2=0x7f5138066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- 172.21.15.87:0/3141133315 shutdown_connections 2024-04-25T10:20:10.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- 172.21.15.87:0/3141133315 wait complete. 2024-04-25T10:20:10.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 Processor -- start 2024-04-25T10:20:10.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- start start 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5138070350 0x7f513814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51380aef80 0x7f513814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51380b31c0 0x7f513814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380cfef0 con 0x7f5138070350 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380cfd70 con 0x7f51380aef80 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f513de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51380b35d0 con 0x7f51380b31c0 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f5136a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5138070350 0x7f513814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f5136a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5138070350 0x7f513814c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57536/0 (socket says 172.21.15.87:57536) 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f5136a00640 1 -- 172.21.15.87:0/582637729 learned_addr learned my addr 172.21.15.87:0/582637729 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f5136000640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51380aef80 0x7f513814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.335+0000 7f5137400640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51380b31c0 0x7f513814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f5136a00640 1 -- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51380b31c0 msgr2=0x7f513814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f5136a00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51380b31c0 0x7f513814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f5136a00640 1 -- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51380aef80 msgr2=0x7f513814cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f5136a00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51380aef80 0x7f513814cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f5136a00640 1 -- 172.21.15.87:0/582637729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513814fee0 con 0x7f5138070350 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f5136a00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5138070350 0x7f513814c370 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f512800cb40 tx=0x7f512800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f512f400640 1 -- 172.21.15.87:0/582637729 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128017070 con 0x7f5138070350 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f512f400640 1 -- 172.21.15.87:0/582637729 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512800aca0 con 0x7f5138070350 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51381f6950 con 0x7f5138070350 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f512f400640 1 -- 172.21.15.87:0/582637729 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128012620 con 0x7f5138070350 2024-04-25T10:20:10.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51381f6f10 con 0x7f5138070350 2024-04-25T10:20:10.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.339+0000 7f512f400640 1 -- 172.21.15.87:0/582637729 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f512801a070 con 0x7f5138070350 2024-04-25T10:20:10.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f512f400640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5118066f90 0x7f5118069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f512f400640 1 -- 172.21.15.87:0/582637729 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5128084d40 con 0x7f5138070350 2024-04-25T10:20:10.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5118066f90 0x7f5118069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 Processor -- start 2024-04-25T10:20:10.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- start start 2024-04-25T10:20:10.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5118066f90 0x7f5118069450 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae000 tx=0x7f51240a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751cd0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751c86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751c87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e751ad3ff0 con 0x55e751cd12f0 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e751a5c150 con 0x55e751c870a0 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e751ad0090 con 0x55e751cd09d0 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751c86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751c86b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57540/0 (socket says 172.21.15.87:57540) 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 -- 172.21.15.87:0/1019786588 learned_addr learned my addr 172.21.15.87:0/1019786588 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5137400640 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751c87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136a00640 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751cd0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 -- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 msgr2=0x55e751cd0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751cd0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 -- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 msgr2=0x55e751c87480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751c87480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 -- 172.21.15.87:0/1019786588 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e751c6a0d0 con 0x55e751cd12f0 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5136000640 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751c86b60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f51240ac040 tx=0x7f5124004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9895931025805c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5123e00640 1 -- 172.21.15.87:0/1019786588 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240c5070 con 0x55e751cd12f0 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5123e00640 1 -- 172.21.15.87:0/1019786588 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240af040 con 0x55e751cd12f0 2024-04-25T10:20:10.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5123e00640 1 -- 172.21.15.87:0/1019786588 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240ca460 con 0x55e751cd12f0 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- 172.21.15.87:0/1019786588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 msgr2=0x55e751c86b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751c86b60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f51240ac040 tx=0x7f5124004740 comp rx=0 tx=0).stop 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- 172.21.15.87:0/1019786588 shutdown_connections 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751c87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751c86b60 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 --2- 172.21.15.87:0/1019786588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751cd0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- 172.21.15.87:0/1019786588 >> 172.21.15.87:0/1019786588 conn(0x55e751c83020 msgr2=0x55e751c85440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- 172.21.15.87:0/1019786588 shutdown_connections 2024-04-25T10:20:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.343+0000 7f5141234000 1 -- 172.21.15.87:0/1019786588 wait complete. 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 -- start start 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 Processor -- start 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751cb57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751cb5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd12f0 0x55e751cb6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e751a98250 con 0x55e751cd09d0 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e751a8e690 con 0x55e751c870a0 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e751a5ca40 con 0x55e751cd12f0 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 10 client.0 _ll_register_callbacks cb 0x55e751c71e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751cb5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751cb5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57548/0 (socket says 172.21.15.87:57548) 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 -- 172.21.15.87:0/1803586440 learned_addr learned my addr 172.21.15.87:0/1803586440 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136a00640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751cb57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5137400640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd12f0 0x55e751cb6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 -- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd12f0 msgr2=0x55e751cb6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd12f0 0x55e751cb6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 -- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 msgr2=0x55e751cb57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751cb57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 -- 172.21.15.87:0/1803586440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e751b811c0 con 0x55e751cd09d0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5137400640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd12f0 0x55e751cb6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136000640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751cb5d30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f512401c9c0 tx=0x7f512401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fad395c8ca5964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5120c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5120c00640 1 -- 172.21.15.87:0/1803586440 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240d7060 con 0x55e751cd09d0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5120c00640 1 -- 172.21.15.87:0/1803586440 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5124003df0 con 0x55e751cd09d0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51240af040 con 0x55e751cd09d0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5120c00640 1 -- 172.21.15.87:0/1803586440 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51240dd050 con 0x55e751cd09d0 2024-04-25T10:20:10.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 10 client.8184 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55e751b7d170 con 0x55e751cd09d0 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5120c00640 1 -- 172.21.15.87:0/1803586440 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240af040 con 0x55e751cd09d0 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5120c00640 1 -- 172.21.15.87:0/1803586440 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f51240e0020 con 0x55e751cd09d0 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 10 client.8184 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 10 client.8184 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 20 client.8184 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce0c80 0x55e751ce3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 4 client.8184 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e751a6fc60 con 0x55e751ce0c80 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.347+0000 7f5136a00640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce0c80 0x55e751ce3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5136a00640 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce0c80 0x55e751ce3070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b98f3bdbef2137a server_cookie=1a3afff646097d50 in_seq=0 out_seq=0 2024-04-25T10:20:10.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5120c00640 10 client.8184 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5120c00640 1 -- 172.21.15.87:0/1803586440 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e751a6fc60 con 0x55e751ce0c80 2024-04-25T10:20:10.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5120c00640 10 client.8184 handle_command_reply: tid=0 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 client.8184 shutdown 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 msgr2=0x55e751cb5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751cb5d30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f512401c9c0 tx=0x7f512401ad20 comp rx=0 tx=0).stop 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 shutdown_connections 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce0c80 0x55e751ce3070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd12f0 0x55e751cb6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751cb5d30 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5120c00640 0 client.8184 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 --2- 172.21.15.87:0/1803586440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751cb57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 >> 172.21.15.87:0/1803586440 conn(0x55e751c83020 msgr2=0x55e751c85ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 shutdown_connections 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 1 -- 172.21.15.87:0/1803586440 wait complete. 2024-04-25T10:20:10.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.351+0000 7f5141234000 20 client.8184 trim_cache size 0 max 16384 2024-04-25T10:20:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.363+0000 7f5141234000 1 Processor -- start 2024-04-25T10:20:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.363+0000 7f5141234000 1 -- start start 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751c870a0 0x55e751ccfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751c86420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751cd12f0 0x55e751c86960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5138202830 con 0x55e751cd09d0 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5138201a30 con 0x55e751cd12f0 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5138200ec0 con 0x55e751c870a0 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751c86420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751c86420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57560/0 (socket says 172.21.15.87:57560) 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 -- 172.21.15.87:0/745579789 learned_addr learned my addr 172.21.15.87:0/745579789 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 -- 172.21.15.87:0/745579789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751c870a0 msgr2=0x55e751ccfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 --2- 172.21.15.87:0/745579789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751c870a0 0x55e751ccfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 -- 172.21.15.87:0/745579789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751cd12f0 msgr2=0x55e751c86960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 --2- 172.21.15.87:0/745579789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751cd12f0 0x55e751c86960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 -- 172.21.15.87:0/745579789 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e751b1ade0 con 0x55e751cd09d0 2024-04-25T10:20:10.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5136000640 1 --2- 172.21.15.87:0/745579789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751c86420 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5124004110 tx=0x7f51240cf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad13be47a21655e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5123400640 1 -- 172.21.15.87:0/745579789 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240cf070 con 0x55e751cd09d0 2024-04-25T10:20:10.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5123400640 1 -- 172.21.15.87:0/745579789 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240af040 con 0x55e751cd09d0 2024-04-25T10:20:10.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5123400640 1 -- 172.21.15.87:0/745579789 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240c5040 con 0x55e751cd09d0 2024-04-25T10:20:10.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- 172.21.15.87:0/745579789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 msgr2=0x55e751c86420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 --2- 172.21.15.87:0/745579789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751c86420 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5124004110 tx=0x7f51240cf000 comp rx=0 tx=0).stop 2024-04-25T10:20:10.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- 172.21.15.87:0/745579789 shutdown_connections 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 --2- 172.21.15.87:0/745579789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751cd12f0 0x55e751c86960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 --2- 172.21.15.87:0/745579789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd09d0 0x55e751c86420 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 --2- 172.21.15.87:0/745579789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751c870a0 0x55e751ccfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- 172.21.15.87:0/745579789 >> 172.21.15.87:0/745579789 conn(0x55e751c83020 msgr2=0x55e751ccf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- 172.21.15.87:0/745579789 shutdown_connections 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- 172.21.15.87:0/745579789 wait complete. 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 -- start start 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.367+0000 7f5141234000 1 Processor -- start 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751d39d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751d3a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751ce3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e751a6ee00 con 0x55e751cd12f0 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5138201430 con 0x55e751c870a0 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e751ad3300 con 0x55e751cd09d0 2024-04-25T10:20:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 10 client.0 _ll_register_callbacks cb 0x55e751c52040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751ce3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751ce3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57562/0 (socket says 172.21.15.87:57562) 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 -- 172.21.15.87:0/3754907045 learned_addr learned my addr 172.21.15.87:0/3754907045 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5136a00640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751d39d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5136000640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751d3a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 -- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 msgr2=0x55e751d3a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751d3a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 -- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 msgr2=0x55e751d39d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751d39d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 -- 172.21.15.87:0/3754907045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e751a758e0 con 0x55e751cd12f0 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5136000640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751d3a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5136a00640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751d39d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5137400640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751ce3380 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5130003bf0 tx=0x7f5130001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72162cbc534f2142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 1 -- 172.21.15.87:0/3754907045 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513001a070 con 0x55e751cd12f0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 1 -- 172.21.15.87:0/3754907045 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5130016420 con 0x55e751cd12f0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51240af040 con 0x55e751cd12f0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 1 -- 172.21.15.87:0/3754907045 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5130025050 con 0x55e751cd12f0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 10 client.8190 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55e751b2d8f0 con 0x55e751cd12f0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 1 -- 172.21.15.87:0/3754907045 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240af040 con 0x55e751cd12f0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 1 -- 172.21.15.87:0/3754907045 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5130028020 con 0x55e751cd12f0 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 10 client.8190 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 10 client.8190 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 20 client.8190 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce4310 0x55e751ce4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 4 client.8190 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:20:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e751ae1530 con 0x55e751ce4310 2024-04-25T10:20:10.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5136a00640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce4310 0x55e751ce4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5136a00640 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce4310 0x55e751ce4720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe1c4e6b708a238 server_cookie=67301496ee70eabd in_seq=0 out_seq=0 2024-04-25T10:20:10.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 10 client.8190 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 1 -- 172.21.15.87:0/3754907045 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55e751ae1530 con 0x55e751ce4310 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 10 client.8190 handle_command_reply: tid=0 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 client.8190 shutdown 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 msgr2=0x55e751ce3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751ce3380 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5130003bf0 tx=0x7f5130001690 comp rx=0 tx=0).stop 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 shutdown_connections 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55e751ce4310 0x55e751ce4720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e751cd12f0 0x55e751ce3380 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e751cd09d0 0x55e751d3a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 --2- 172.21.15.87:0/3754907045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e751c870a0 0x55e751d39d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5120c00640 0 client.8190 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 >> 172.21.15.87:0/3754907045 conn(0x55e751c83020 msgr2=0x55e751ccf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 shutdown_connections 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 1 -- 172.21.15.87:0/3754907045 wait complete. 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f5141234000 20 client.8190 trim_cache size 0 max 16384 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stdout: "f8a4fa23-2d0c-4cca-b622-5347410cf725": { 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:20:10.380 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "f8a4fa23-2d0c-4cca-b622-5347410cf725", 2024-04-25T10:20:10.381 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive" 2024-04-25T10:20:10.381 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:10.381 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:10.381 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5118066f90 msgr2=0x7f5118069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5118066f90 0x7f5118069450 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae000 tx=0x7f51240a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5138070350 msgr2=0x7f513814c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5138070350 0x7f513814c370 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f512800cb40 tx=0x7f512800a060 comp rx=0 tx=0).stop 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 shutdown_connections 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5118066f90 0x7f5118069450 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51380b31c0 0x7f513814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51380aef80 0x7f513814cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 --2- 172.21.15.87:0/582637729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5138070350 0x7f513814c370 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.371+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 >> 172.21.15.87:0/582637729 conn(0x7f51380669a0 msgr2=0x7f51380b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.375+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 shutdown_connections 2024-04-25T10:20:10.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.375+0000 7f513de00640 1 -- 172.21.15.87:0/582637729 wait complete. 2024-04-25T10:20:10.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:10.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faac2000640 1 Processor -- start 2024-04-25T10:20:10.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faac2000640 1 -- start start 2024-04-25T10:20:10.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faac2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc0ccb20 0x7faabc0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faac2000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc0ccfb0 con 0x7faabc15b9e0 2024-04-25T10:20:10.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faac2000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc0cd290 con 0x7faabc15bdc0 2024-04-25T10:20:10.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faac2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0cd570 con 0x7faabc0ccb20 2024-04-25T10:20:10.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faabb400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7faabc15b9e0 0x7faabc0cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53012/0 (socket says 172.21.15.87:53012) 2024-04-25T10:20:10.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faabb400640 1 -- 172.21.15.87:0/392183585 learned_addr learned my addr 172.21.15.87:0/392183585 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:10.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faabaa00640 1 --2- 172.21.15.87:0/392183585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc0ccb20 0x7faabc0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faaba000640 1 -- 172.21.15.87:0/392183585 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273462787 0 0) 0x7faabc0ccfb0 con 0x7faabc15b9e0 2024-04-25T10:20:10.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faaba000640 1 -- 172.21.15.87:0/392183585 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa0003530 con 0x7faabc15b9e0 2024-04-25T10:20:10.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faaba000640 1 -- 172.21.15.87:0/392183585 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174525298 0 0) 0x7faabc0cd290 con 0x7faabc15bdc0 2024-04-25T10:20:10.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faaba000640 1 -- 172.21.15.87:0/392183585 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc0ccfb0 con 0x7faabc15bdc0 2024-04-25T10:20:10.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faabaa00640 1 -- 172.21.15.87:0/392183585 >> v1:172.21.15.169:6789/0 conn(0x7faabc15bdc0 legacy=0x7faabc0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faabaa00640 1 -- 172.21.15.87:0/392183585 >> v1:172.21.15.87:6789/0 conn(0x7faabc15b9e0 legacy=0x7faabc0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.527+0000 7faabaa00640 1 -- 172.21.15.87:0/392183585 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0cd850 con 0x7faabc0ccb20 2024-04-25T10:20:10.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faabaa00640 1 --2- 172.21.15.87:0/392183585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc0ccb20 0x7faabc0d1520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faaa80097f0 tx=0x7faaa801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc291172a7f64a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faaba000640 1 -- 172.21.15.87:0/392183585 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80b0070 con 0x7faabc0ccb20 2024-04-25T10:20:10.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faaba000640 1 -- 172.21.15.87:0/392183585 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa80ab600 con 0x7faabc0ccb20 2024-04-25T10:20:10.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faaba000640 1 -- 172.21.15.87:0/392183585 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80ab9d0 con 0x7faabc0ccb20 2024-04-25T10:20:10.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faac2000640 1 -- 172.21.15.87:0/392183585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc0ccb20 msgr2=0x7faabc0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faac2000640 1 --2- 172.21.15.87:0/392183585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc0ccb20 0x7faabc0d1520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faaa80097f0 tx=0x7faaa801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faac2000640 1 -- 172.21.15.87:0/392183585 shutdown_connections 2024-04-25T10:20:10.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faac2000640 1 --2- 172.21.15.87:0/392183585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc0ccb20 0x7faabc0d1520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faac2000640 1 -- 172.21.15.87:0/392183585 >> 172.21.15.87:0/392183585 conn(0x7faabc09a9d0 msgr2=0x7faabc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faac2000640 1 -- 172.21.15.87:0/392183585 shutdown_connections 2024-04-25T10:20:10.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.531+0000 7faac2000640 1 -- 172.21.15.87:0/392183585 wait complete. 2024-04-25T10:20:10.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 Processor -- start 2024-04-25T10:20:10.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 -- start start 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc15b9e0 0x7faabc1549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faabc15bdc0 0x7faabc153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faabc15ee70 0x7faabc15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0c6ab0 con 0x7faabc15bdc0 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0c6930 con 0x7faabc15ee70 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0c6c30 con 0x7faabc15b9e0 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faabc15bdc0 0x7faabc153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faabc15bdc0 0x7faabc153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57578/0 (socket says 172.21.15.87:57578) 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 -- 172.21.15.87:0/3848010796 learned_addr learned my addr 172.21.15.87:0/3848010796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabbe00640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faabc15ee70 0x7faabc15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 -- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc15b9e0 msgr2=0x7faabc1549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabb400640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc15b9e0 0x7faabc1549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc15b9e0 0x7faabc1549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 -- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faabc15ee70 msgr2=0x7faabc15ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faabc15ee70 0x7faabc15ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 -- 172.21.15.87:0/3848010796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc155210 con 0x7faabc15bdc0 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabbe00640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faabc15ee70 0x7faabc15ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabb400640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc15b9e0 0x7faabc1549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faabaa00640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faabc15bdc0 0x7faabc153db0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7faaa80190f0 tx=0x7faaa8019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faab3400640 1 -- 172.21.15.87:0/3848010796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80abe60 con 0x7faabc15bdc0 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faab3400640 1 -- 172.21.15.87:0/3848010796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa80b9070 con 0x7faabc15bdc0 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faab3400640 1 -- 172.21.15.87:0/3848010796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80b0040 con 0x7faabc15bdc0 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc1554f0 con 0x7faabc15bdc0 2024-04-25T10:20:10.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.535+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc0c8f00 con 0x7faabc15bdc0 2024-04-25T10:20:10.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.539+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc04ef30 con 0x7faabc15bdc0 2024-04-25T10:20:10.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.539+0000 7faab3400640 1 -- 172.21.15.87:0/3848010796 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faaa80b4690 con 0x7faabc15bdc0 2024-04-25T10:20:10.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.539+0000 7faab3400640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faa9c066f60 0x7faa9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.539+0000 7faab3400640 1 -- 172.21.15.87:0/3848010796 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faaa811eaa0 con 0x7faabc15bdc0 2024-04-25T10:20:10.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.539+0000 7faabb400640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faa9c066f60 0x7faa9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.539+0000 7faabb400640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faa9c066f60 0x7faa9c069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faaac000f70 tx=0x7faaac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.543+0000 7faab3400640 1 -- 172.21.15.87:0/3848010796 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faaa80160d0 con 0x7faabc15bdc0 2024-04-25T10:20:10.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.727+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faabc0cbd10 con 0x7faabc15bdc0 2024-04-25T10:20:10.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.727+0000 7faab3400640 1 -- 172.21.15.87:0/3848010796 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7faaa80bb310 con 0x7faabc15bdc0 2024-04-25T10:20:10.731 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:10.731 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:10.732 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:10.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faa9c066f60 msgr2=0x7faa9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faa9c066f60 0x7faa9c069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faaac000f70 tx=0x7faaac009040 comp rx=0 tx=0).stop 2024-04-25T10:20:10.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faabc15bdc0 msgr2=0x7faabc153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faabc15bdc0 0x7faabc153db0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7faaa80190f0 tx=0x7faaa8019280 comp rx=0 tx=0).stop 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 shutdown_connections 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faa9c066f60 0x7faa9c069420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faabc15ee70 0x7faabc15ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faabc15bdc0 0x7faabc153db0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 --2- 172.21.15.87:0/3848010796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faabc15b9e0 0x7faabc1549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 >> 172.21.15.87:0/3848010796 conn(0x7faabc09a9d0 msgr2=0x7faabc068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 shutdown_connections 2024-04-25T10:20:10.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.731+0000 7faac2000640 1 -- 172.21.15.87:0/3848010796 wait complete. 2024-04-25T10:20:10.751 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:20:10.752 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:10.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe435c00640 1 Processor -- start 2024-04-25T10:20:10.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe435c00640 1 -- start start 2024-04-25T10:20:10.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe435c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 0x7fe430168990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe435c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4300b1af0 con 0x7fe4300d1820 2024-04-25T10:20:10.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe435c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4300b1dd0 con 0x7fe43015e050 2024-04-25T10:20:10.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe435c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300b20b0 con 0x7fe4300b15b0 2024-04-25T10:20:10.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe4300d1820 0x7fe4300d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53024/0 (socket says 172.21.15.87:53024) 2024-04-25T10:20:10.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42ea00640 1 -- 172.21.15.87:0/1361667564 learned_addr learned my addr 172.21.15.87:0/1361667564 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42f400640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 0x7fe430168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041661145 0 0) 0x7fe4300b1af0 con 0x7fe4300d1820 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe414003530 con 0x7fe4300d1820 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744102418 0 0) 0x7fe414003530 con 0x7fe4300d1820 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 msgr2=0x7fe430168990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 0x7fe430168990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 >> v1:172.21.15.169:6789/0 conn(0x7fe43015e050 legacy=0x7fe4300b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4300b2390 con 0x7fe4300d1820 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe418001680 con 0x7fe4300d1820 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 >> v1:172.21.15.87:6789/0 conn(0x7fe4300d1820 legacy=0x7fe4300d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4140039f0 0x7fe414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe414004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 0x7fe414005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300b20b0 con 0x7fe4140043a0 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41400ba80 con 0x7fe4140039f0 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.883+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41400bc80 con 0x7fe4300b15b0 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42e000640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe414004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42ea00640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4140039f0 0x7fe414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42f400640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 0x7fe414005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42e000640 1 -- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 msgr2=0x7fe414005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42e000640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4300b15b0 0x7fe414005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42e000640 1 -- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4140039f0 msgr2=0x7fe414003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:10.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42e000640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4140039f0 0x7fe414003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42e000640 1 -- 172.21.15.87:0/1361667564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414003530 con 0x7fe4140043a0 2024-04-25T10:20:10.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42e000640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe414004870 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe430070e60 tx=0x7fe42001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64166eef69420cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420001460 con 0x7fe4140043a0 2024-04-25T10:20:10.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4200b0070 con 0x7fe4140043a0 2024-04-25T10:20:10.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.887+0000 7fe42d600640 1 -- 172.21.15.87:0/1361667564 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200ab840 con 0x7fe4140043a0 2024-04-25T10:20:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- 172.21.15.87:0/1361667564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 msgr2=0x7fe414004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 --2- 172.21.15.87:0/1361667564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe414004870 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe430070e60 tx=0x7fe42001c4c0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42f400640 1 -- 172.21.15.87:0/1361667564 reap_dead start 2024-04-25T10:20:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- 172.21.15.87:0/1361667564 shutdown_connections 2024-04-25T10:20:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- 172.21.15.87:0/1361667564 >> 172.21.15.87:0/1361667564 conn(0x7fe43009a9d0 msgr2=0x7fe43009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- 172.21.15.87:0/1361667564 shutdown_connections 2024-04-25T10:20:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- 172.21.15.87:0/1361667564 wait complete. 2024-04-25T10:20:10.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 Processor -- start 2024-04-25T10:20:10.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- start start 2024-04-25T10:20:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe4301fac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe43015bf20 0x7fe43014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe43015c590 0x7fe43015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe430169530 con 0x7fe4140043a0 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301693b0 con 0x7fe43015c590 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe435c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301696b0 con 0x7fe43015bf20 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe4301fac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe4301fac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57590/0 (socket says 172.21.15.87:57590) 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 -- 172.21.15.87:0/4096401057 learned_addr learned my addr 172.21.15.87:0/4096401057 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42f400640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe43015c590 0x7fe43015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42e000640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe43015bf20 0x7fe43014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 -- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe43015bf20 msgr2=0x7fe43014e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe43015bf20 0x7fe43014e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 -- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe43015c590 msgr2=0x7fe43015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe43015c590 0x7fe43015ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 -- 172.21.15.87:0/4096401057 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe430072760 con 0x7fe4140043a0 2024-04-25T10:20:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe42ea00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe4301fac20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe4180017f0 tx=0x7fe4180074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.891+0000 7fe426a00640 1 -- 172.21.15.87:0/4096401057 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41800f7b0 con 0x7fe4140043a0 2024-04-25T10:20:10.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe426a00640 1 -- 172.21.15.87:0/4096401057 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418018070 con 0x7fe4140043a0 2024-04-25T10:20:10.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe426a00640 1 -- 172.21.15.87:0/4096401057 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41800fbd0 con 0x7fe4140043a0 2024-04-25T10:20:10.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4300aea00 con 0x7fe4140043a0 2024-04-25T10:20:10.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4300b69f0 con 0x7fe4140043a0 2024-04-25T10:20:10.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43004ef30 con 0x7fe4140043a0 2024-04-25T10:20:10.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe426a00640 1 -- 172.21.15.87:0/4096401057 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe41801d460 con 0x7fe4140043a0 2024-04-25T10:20:10.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe426a00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe40c066f90 0x7fe40c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:10.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe426a00640 1 -- 172.21.15.87:0/4096401057 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe418085220 con 0x7fe4140043a0 2024-04-25T10:20:10.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe42e000640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe40c066f90 0x7fe40c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:10.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.895+0000 7fe42e000640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe40c066f90 0x7fe40c069450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe4200af000 tx=0x7fe4200a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:10.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:10.899+0000 7fe426a00640 1 -- 172.21.15.87:0/4096401057 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe418056190 con 0x7fe4140043a0 2024-04-25T10:20:11.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.107+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe4301f6ca0 con 0x7fe4140043a0 2024-04-25T10:20:11.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.107+0000 7fe426a00640 1 -- 172.21.15.87:0/4096401057 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7fe418055a90 con 0x7fe4140043a0 2024-04-25T10:20:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:11.112 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":873,"data_bytes":892359450,"bytes_used":2167250944,"bytes_avail":770926862336,"bytes_total":773094113280,"write_bytes_sec":43517440,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:19:17.213750+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:11.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe40c066f90 msgr2=0x7fe40c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe40c066f90 0x7fe40c069450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe4200af000 tx=0x7fe4200a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 msgr2=0x7fe4301fac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe4301fac20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe4180017f0 tx=0x7fe4180074f0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 shutdown_connections 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe40c066f90 0x7fe40c069450 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe43015c590 0x7fe43015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe43015bf20 0x7fe43014e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 --2- 172.21.15.87:0/4096401057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4140043a0 0x7fe4301fac20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 >> 172.21.15.87:0/4096401057 conn(0x7fe43009a9d0 msgr2=0x7fe4300aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 shutdown_connections 2024-04-25T10:20:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:11.111+0000 7fe435c00640 1 -- 172.21.15.87:0/4096401057 wait complete. 2024-04-25T10:20:11.132 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:20:12.133 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:12.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.271+0000 7f893ca00640 1 Processor -- start 2024-04-25T10:20:12.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.271+0000 7f893ca00640 1 -- start start 2024-04-25T10:20:12.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.271+0000 7f893ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f89381550b0 0x7f89380cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.271+0000 7f893ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f893815b910 0x7f89380b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.271+0000 7f893ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8938071050 con 0x7f89380cfe70 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.271+0000 7f893ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893815bda0 con 0x7f89381550b0 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.271+0000 7f893ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893815d040 con 0x7f893815b910 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8935e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f89381550b0 0x7f89380cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8936800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f893815b910 0x7f89380b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8935e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f89381550b0 0x7f89380cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:52774/0 (socket says 172.21.15.87:52774) 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f892d400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f89380cfe70 0x7f893815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:54578/0 (socket says 172.21.15.87:54578) 2024-04-25T10:20:12.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8935e00640 1 -- 172.21.15.87:0/11720021 learned_addr learned my addr 172.21.15.87:0/11720021 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8936800640 1 -- 172.21.15.87:0/11720021 >> v2:172.21.15.169:3300/0 conn(0x7f89381550b0 msgr2=0x7f89380cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8936800640 1 --2- 172.21.15.87:0/11720021 >> v2:172.21.15.169:3300/0 conn(0x7f89381550b0 0x7f89380cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8936800640 1 -- 172.21.15.87:0/11720021 >> v1:172.21.15.169:6789/0 conn(0x7f89380cfe70 legacy=0x7f893815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8936800640 1 -- 172.21.15.87:0/11720021 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380712f0 con 0x7f893815b910 2024-04-25T10:20:12.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.275+0000 7f8936800640 1 --2- 172.21.15.87:0/11720021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f893815b910 0x7f89380b3260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8930009dc0 tx=0x7f893001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4cba421d7fa46b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f8935400640 1 -- 172.21.15.87:0/11720021 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300b2070 con 0x7f893815b910 2024-04-25T10:20:12.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f8935400640 1 -- 172.21.15.87:0/11720021 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300a5900 con 0x7f893815b910 2024-04-25T10:20:12.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f8935400640 1 -- 172.21.15.87:0/11720021 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300a5cd0 con 0x7f893815b910 2024-04-25T10:20:12.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- 172.21.15.87:0/11720021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f893815b910 msgr2=0x7f89380b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 --2- 172.21.15.87:0/11720021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f893815b910 0x7f89380b3260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8930009dc0 tx=0x7f893001c950 comp rx=0 tx=0).stop 2024-04-25T10:20:12.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- 172.21.15.87:0/11720021 shutdown_connections 2024-04-25T10:20:12.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 --2- 172.21.15.87:0/11720021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f893815b910 0x7f89380b3260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 --2- 172.21.15.87:0/11720021 >> v2:172.21.15.169:3300/0 conn(0x7f89381550b0 0x7f89380cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- 172.21.15.87:0/11720021 >> 172.21.15.87:0/11720021 conn(0x7f8938066b70 msgr2=0x7f8938066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- 172.21.15.87:0/11720021 shutdown_connections 2024-04-25T10:20:12.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- 172.21.15.87:0/11720021 wait complete. 2024-04-25T10:20:12.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 Processor -- start 2024-04-25T10:20:12.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- start start 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89380cfe70 0x7f89381440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89381550b0 0x7f8938153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815b910 0x7f89381543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893815c030 con 0x7f893815b910 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380720b0 con 0x7f89381550b0 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f893ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893815c1b0 con 0x7f89380cfe70 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f8936800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815b910 0x7f89381543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f892d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89381550b0 0x7f8938153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f8936800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815b910 0x7f89381543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57596/0 (socket says 172.21.15.87:57596) 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f8936800640 1 -- 172.21.15.87:0/3321612651 learned_addr learned my addr 172.21.15.87:0/3321612651 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.279+0000 7f8935e00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89380cfe70 0x7f89381440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f8936800640 1 -- 172.21.15.87:0/3321612651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89380cfe70 msgr2=0x7f89381440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f8936800640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89380cfe70 0x7f89381440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f8936800640 1 -- 172.21.15.87:0/3321612651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89381550b0 msgr2=0x7f8938153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f8936800640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89381550b0 0x7f8938153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f8936800640 1 -- 172.21.15.87:0/3321612651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380ccc70 con 0x7f893815b910 2024-04-25T10:20:12.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f8936800640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815b910 0x7f89381543d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f89300b0040 tx=0x7f89300196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f892ea00640 1 -- 172.21.15.87:0/3321612651 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300b2050 con 0x7f893815b910 2024-04-25T10:20:12.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f892ea00640 1 -- 172.21.15.87:0/3321612651 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300be070 con 0x7f893815b910 2024-04-25T10:20:12.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f892ea00640 1 -- 172.21.15.87:0/3321612651 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300ada90 con 0x7f893815b910 2024-04-25T10:20:12.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8938156150 con 0x7f893815b910 2024-04-25T10:20:12.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89380cd1f0 con 0x7f893815b910 2024-04-25T10:20:12.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893804ef30 con 0x7f893815b910 2024-04-25T10:20:12.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f892ea00640 1 -- 172.21.15.87:0/3321612651 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89300adc30 con 0x7f893815b910 2024-04-25T10:20:12.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f892ea00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8914066f60 0x7f8914069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.283+0000 7f892ea00640 1 -- 172.21.15.87:0/3321612651 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89301200b0 con 0x7f893815b910 2024-04-25T10:20:12.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.287+0000 7f8935e00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8914066f60 0x7f8914069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.287+0000 7f8935e00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8914066f60 0x7f8914069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8928009b50 tx=0x7f8928009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.291+0000 7f892ea00640 1 -- 172.21.15.87:0/3321612651 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8930016540 con 0x7f893815b910 2024-04-25T10:20:12.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.475+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8938144630 con 0x7f893815b910 2024-04-25T10:20:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.475+0000 7f892ea00640 1 -- 172.21.15.87:0/3321612651 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f89300f09d0 con 0x7f893815b910 2024-04-25T10:20:12.479 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:12.479 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:12.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8914066f60 msgr2=0x7f8914069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8914066f60 0x7f8914069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8928009b50 tx=0x7f8928009240 comp rx=0 tx=0).stop 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815b910 msgr2=0x7f89381543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815b910 0x7f89381543d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f89300b0040 tx=0x7f89300196f0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 shutdown_connections 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8914066f60 0x7f8914069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815b910 0x7f89381543d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89381550b0 0x7f8938153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 --2- 172.21.15.87:0/3321612651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89380cfe70 0x7f89381440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 >> 172.21.15.87:0/3321612651 conn(0x7f8938066b70 msgr2=0x7f893806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 shutdown_connections 2024-04-25T10:20:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.479+0000 7f893ca00640 1 -- 172.21.15.87:0/3321612651 wait complete. 2024-04-25T10:20:12.500 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:20:12.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:12.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f2e00640 1 Processor -- start 2024-04-25T10:20:12.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f2e00640 1 -- start start 2024-04-25T10:20:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f2e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f74ec0cda10 0x7f74ec06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f2e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec0b2770 con 0x7f74ec0cf930 2024-04-25T10:20:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f2e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec06c670 con 0x7f74ec15bc40 2024-04-25T10:20:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f2e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec06c950 con 0x7f74ec0cda10 2024-04-25T10:20:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f0e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f74ec0cf930 0x7f74ec0b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53034/0 (socket says 172.21.15.87:53034) 2024-04-25T10:20:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74f0e00640 1 -- 172.21.15.87:0/1065164865 learned_addr learned my addr 172.21.15.87:0/1065164865 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.631+0000 7f74e3400640 1 --2- 172.21.15.87:0/1065164865 >> v2:172.21.15.169:3300/0 conn(0x7f74ec0cda10 0x7f74ec06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74eb400640 1 -- 172.21.15.87:0/1065164865 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542044182 0 0) 0x7f74ec0b2770 con 0x7f74ec0cf930 2024-04-25T10:20:12.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74eb400640 1 -- 172.21.15.87:0/1065164865 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74d0003530 con 0x7f74ec0cf930 2024-04-25T10:20:12.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74eb400640 1 -- 172.21.15.87:0/1065164865 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926024092 0 0) 0x7f74ec06c670 con 0x7f74ec15bc40 2024-04-25T10:20:12.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74e3400640 1 -- 172.21.15.87:0/1065164865 >> v1:172.21.15.169:6789/0 conn(0x7f74ec15bc40 legacy=0x7f74ec0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74e3400640 1 -- 172.21.15.87:0/1065164865 >> v1:172.21.15.87:6789/0 conn(0x7f74ec0cf930 legacy=0x7f74ec0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74e3400640 1 -- 172.21.15.87:0/1065164865 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec06cc30 con 0x7f74ec0cda10 2024-04-25T10:20:12.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74e3400640 1 --2- 172.21.15.87:0/1065164865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 0x7f74ec06c090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097c0 tx=0x7f74d401c250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f472e2b176d7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74eb400640 1 -- 172.21.15.87:0/1065164865 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d40b0070 con 0x7f74ec0cda10 2024-04-25T10:20:12.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74eb400640 1 -- 172.21.15.87:0/1065164865 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d40ab400 con 0x7f74ec0cda10 2024-04-25T10:20:12.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.635+0000 7f74eb400640 1 -- 172.21.15.87:0/1065164865 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4001980 con 0x7f74ec0cda10 2024-04-25T10:20:12.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- 172.21.15.87:0/1065164865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 msgr2=0x7f74ec06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 --2- 172.21.15.87:0/1065164865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 0x7f74ec06c090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097c0 tx=0x7f74d401c250 comp rx=0 tx=0).stop 2024-04-25T10:20:12.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- 172.21.15.87:0/1065164865 shutdown_connections 2024-04-25T10:20:12.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 --2- 172.21.15.87:0/1065164865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 0x7f74ec06c090 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- 172.21.15.87:0/1065164865 >> 172.21.15.87:0/1065164865 conn(0x7f74ec066b70 msgr2=0x7f74ec066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- 172.21.15.87:0/1065164865 shutdown_connections 2024-04-25T10:20:12.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- 172.21.15.87:0/1065164865 wait complete. 2024-04-25T10:20:12.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 Processor -- start 2024-04-25T10:20:12.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- start start 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 0x7f74ec0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74ec0cf930 0x7f74ec1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74ec15bc40 0x7f74ec0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec169d70 con 0x7f74ec0cf930 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec169bf0 con 0x7f74ec0cda10 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec169ef0 con 0x7f74ec15bc40 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74ec0cf930 0x7f74ec1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74ec0cf930 0x7f74ec1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57598/0 (socket says 172.21.15.87:57598) 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 -- 172.21.15.87:0/973026666 learned_addr learned my addr 172.21.15.87:0/973026666 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74f0e00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74ec15bc40 0x7f74ec0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74ebe00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 0x7f74ec0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 -- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74ec15bc40 msgr2=0x7f74ec0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74ec15bc40 0x7f74ec0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 -- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 msgr2=0x7f74ec0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 0x7f74ec0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.639+0000 7f74e3400640 1 -- 172.21.15.87:0/973026666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec0b0970 con 0x7f74ec0cf930 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74e3400640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74ec0cf930 0x7f74ec1597d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097c0 tx=0x7f74d401aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74e8c00640 1 -- 172.21.15.87:0/973026666 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d40bb020 con 0x7f74ec0cf930 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74e8c00640 1 -- 172.21.15.87:0/973026666 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d40b1040 con 0x7f74ec0cf930 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74e8c00640 1 -- 172.21.15.87:0/973026666 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d40b0050 con 0x7f74ec0cf930 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec0b0f30 con 0x7f74ec0cf930 2024-04-25T10:20:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec0b14f0 con 0x7f74ec0cf930 2024-04-25T10:20:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec04ef30 con 0x7f74ec0cf930 2024-04-25T10:20:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74e8c00640 1 -- 172.21.15.87:0/973026666 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74d40be070 con 0x7f74ec0cf930 2024-04-25T10:20:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74e8c00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74c8066f60 0x7f74c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74e8c00640 1 -- 172.21.15.87:0/973026666 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74d411e560 con 0x7f74ec0cf930 2024-04-25T10:20:12.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.643+0000 7f74ebe00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74c8066f60 0x7f74c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.647+0000 7f74ebe00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74c8066f60 0x7f74c8069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f74ec15f2b0 tx=0x7f74dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.647+0000 7f74e8c00640 1 -- 172.21.15.87:0/973026666 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74d4015f90 con 0x7f74ec0cf930 2024-04-25T10:20:12.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f74ec06d960 con 0x7f74ec0cf930 2024-04-25T10:20:12.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74e8c00640 1 -- 172.21.15.87:0/973026666 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f74d40ef4d0 con 0x7f74ec0cf930 2024-04-25T10:20:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:12.857 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":494,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":873,"data_bytes":892359450,"bytes_used":2167250944,"bytes_avail":770926862336,"bytes_total":773094113280,"write_bytes_sec":43518131,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:19:17.213750+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:12.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74c8066f60 msgr2=0x7f74c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74c8066f60 0x7f74c8069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f74ec15f2b0 tx=0x7f74dc009040 comp rx=0 tx=0).stop 2024-04-25T10:20:12.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74ec0cf930 msgr2=0x7f74ec1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74ec0cf930 0x7f74ec1597d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097c0 tx=0x7f74d401aa90 comp rx=0 tx=0).stop 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 shutdown_connections 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f74c8066f60 0x7f74c8069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74ec15bc40 0x7f74ec0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74ec0cf930 0x7f74ec1597d0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 --2- 172.21.15.87:0/973026666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74ec0cda10 0x7f74ec0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 >> 172.21.15.87:0/973026666 conn(0x7f74ec066b70 msgr2=0x7f74ec066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 shutdown_connections 2024-04-25T10:20:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:12.855+0000 7f74f2e00640 1 -- 172.21.15.87:0/973026666 wait complete. 2024-04-25T10:20:12.879 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:20:13.881 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.023+0000 7f8054600640 1 Processor -- start 2024-04-25T10:20:14.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.023+0000 7f8054600640 1 -- start start 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.023+0000 7f8054600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f804c1550b0 0x7f804c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8054600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8054600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c071050 con 0x7f804c0cfe70 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8054600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c15bda0 con 0x7f804c1550b0 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8054600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c15d040 con 0x7f804c15b910 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f804c1550b0 0x7f804c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f804c1550b0 0x7f804c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57614/0 (socket says 172.21.15.87:57614) 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 -- 172.21.15.87:0/1565404301 learned_addr learned my addr 172.21.15.87:0/1565404301 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:14.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051200640 1 --1- 172.21.15.87:0/1565404301 >> v1:172.21.15.87:6789/0 conn(0x7f804c0cfe70 0x7f804c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53036/0 (socket says 172.21.15.87:53036) 2024-04-25T10:20:14.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8052600640 1 --2- 172.21.15.87:0/1565404301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 -- 172.21.15.87:0/1565404301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 msgr2=0x7f804c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:14.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 --2- 172.21.15.87:0/1565404301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 -- 172.21.15.87:0/1565404301 >> v1:172.21.15.87:6789/0 conn(0x7f804c0cfe70 legacy=0x7f804c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 -- 172.21.15.87:0/1565404301 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0712f0 con 0x7f804c1550b0 2024-04-25T10:20:14.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f8051c00640 1 --2- 172.21.15.87:0/1565404301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c0cf930 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f80400098a0 tx=0x7f804001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed328ea76fa0b85c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f804be00640 1 -- 172.21.15.87:0/1565404301 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400ab430 con 0x7f804c1550b0 2024-04-25T10:20:14.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f804be00640 1 -- 172.21.15.87:0/1565404301 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80400b0070 con 0x7f804c1550b0 2024-04-25T10:20:14.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.027+0000 7f804be00640 1 -- 172.21.15.87:0/1565404301 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400ab7f0 con 0x7f804c1550b0 2024-04-25T10:20:14.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- 172.21.15.87:0/1565404301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 msgr2=0x7f804c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 --2- 172.21.15.87:0/1565404301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c0cf930 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f80400098a0 tx=0x7f804001c540 comp rx=0 tx=0).stop 2024-04-25T10:20:14.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- 172.21.15.87:0/1565404301 shutdown_connections 2024-04-25T10:20:14.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 --2- 172.21.15.87:0/1565404301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 --2- 172.21.15.87:0/1565404301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c0cf930 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- 172.21.15.87:0/1565404301 >> 172.21.15.87:0/1565404301 conn(0x7f804c066b70 msgr2=0x7f804c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:14.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- 172.21.15.87:0/1565404301 shutdown_connections 2024-04-25T10:20:14.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- 172.21.15.87:0/1565404301 wait complete. 2024-04-25T10:20:14.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 Processor -- start 2024-04-25T10:20:14.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- start start 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0cfe70 0x7f804c147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c15c030 con 0x7f804c1550b0 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0720b0 con 0x7f804c0cfe70 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8054600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c15c1b0 con 0x7f804c15b910 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8051200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8051200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57622/0 (socket says 172.21.15.87:57622) 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8051200640 1 -- 172.21.15.87:0/3530723536 learned_addr learned my addr 172.21.15.87:0/3530723536 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8051200640 1 -- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 msgr2=0x7f804c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8052600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.031+0000 7f8051c00640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0cfe70 0x7f804c147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8051200640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8051200640 1 -- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0cfe70 msgr2=0x7f804c147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8051200640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0cfe70 0x7f804c147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8051200640 1 -- 172.21.15.87:0/3530723536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c157fc0 con 0x7f804c1550b0 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8052600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8051c00640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0cfe70 0x7f804c147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8051200640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c157440 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f803800c450 tx=0x7f803800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8049600640 1 -- 172.21.15.87:0/3530723536 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038018070 con 0x7f804c1550b0 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8049600640 1 -- 172.21.15.87:0/3530723536 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8038007480 con 0x7f804c1550b0 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c1582a0 con 0x7f804c1550b0 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8049600640 1 -- 172.21.15.87:0/3530723536 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380047f0 con 0x7f804c1550b0 2024-04-25T10:20:14.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c0af910 con 0x7f804c1550b0 2024-04-25T10:20:14.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c04ef30 con 0x7f804c1550b0 2024-04-25T10:20:14.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8049600640 1 -- 172.21.15.87:0/3530723536 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f803801b070 con 0x7f804c1550b0 2024-04-25T10:20:14.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8049600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f802c066f60 0x7f802c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8049600640 1 -- 172.21.15.87:0/3530723536 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8038084270 con 0x7f804c1550b0 2024-04-25T10:20:14.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.035+0000 7f8051c00640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f802c066f60 0x7f802c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.039+0000 7f8051c00640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f802c066f60 0x7f802c069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80400af000 tx=0x7f80400a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.039+0000 7f8049600640 1 -- 172.21.15.87:0/3530723536 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8038055270 con 0x7f804c1550b0 2024-04-25T10:20:14.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.227+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f804c06cd50 con 0x7f804c1550b0 2024-04-25T10:20:14.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.227+0000 7f8049600640 1 -- 172.21.15.87:0/3530723536 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8038054b70 con 0x7f804c1550b0 2024-04-25T10:20:14.231 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:14.231 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:14.231 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:14.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f802c066f60 msgr2=0x7f802c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f802c066f60 0x7f802c069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80400af000 tx=0x7f80400a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:14.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 msgr2=0x7f804c157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c157440 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f803800c450 tx=0x7f803800c910 comp rx=0 tx=0).stop 2024-04-25T10:20:14.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 shutdown_connections 2024-04-25T10:20:14.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f802c066f60 0x7f802c069420 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f804c15b910 0x7f804c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f804c1550b0 0x7f804c157440 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 --2- 172.21.15.87:0/3530723536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0cfe70 0x7f804c147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 >> 172.21.15.87:0/3530723536 conn(0x7f804c066b70 msgr2=0x7f804c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:14.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 shutdown_connections 2024-04-25T10:20:14.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.231+0000 7f8054600640 1 -- 172.21.15.87:0/3530723536 wait complete. 2024-04-25T10:20:14.251 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:20:14.251 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:14.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff883800640 1 Processor -- start 2024-04-25T10:20:14.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff883800640 1 -- start start 2024-04-25T10:20:14.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff883800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c15b9e0 0x7ff87c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff883800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c0ccfb0 con 0x7ff87c15bdc0 2024-04-25T10:20:14.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff883800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c0cd290 con 0x7ff87c0d1c30 2024-04-25T10:20:14.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff883800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0cd570 con 0x7ff87c15b9e0 2024-04-25T10:20:14.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff87be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff87c15bdc0 0x7ff87c0d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53050/0 (socket says 172.21.15.87:53050) 2024-04-25T10:20:14.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff87be00640 1 -- 172.21.15.87:0/4031578622 learned_addr learned my addr 172.21.15.87:0/4031578622 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:14.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff880e00640 1 --2- 172.21.15.87:0/4031578622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c15b9e0 0x7ff87c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff87b400640 1 -- 172.21.15.87:0/4031578622 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843732798 0 0) 0x7ff87c0ccfb0 con 0x7ff87c15bdc0 2024-04-25T10:20:14.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff87b400640 1 -- 172.21.15.87:0/4031578622 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff860003530 con 0x7ff87c15bdc0 2024-04-25T10:20:14.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff87b400640 1 -- 172.21.15.87:0/4031578622 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719662983 0 0) 0x7ff87c0cd290 con 0x7ff87c0d1c30 2024-04-25T10:20:14.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff87b400640 1 -- 172.21.15.87:0/4031578622 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff87c0ccfb0 con 0x7ff87c0d1c30 2024-04-25T10:20:14.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff880e00640 1 -- 172.21.15.87:0/4031578622 >> v1:172.21.15.169:6789/0 conn(0x7ff87c0d1c30 legacy=0x7ff87c0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff880e00640 1 -- 172.21.15.87:0/4031578622 >> v1:172.21.15.87:6789/0 conn(0x7ff87c15bdc0 legacy=0x7ff87c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.383+0000 7ff880e00640 1 -- 172.21.15.87:0/4031578622 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0cd850 con 0x7ff87c15b9e0 2024-04-25T10:20:14.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.387+0000 7ff880e00640 1 --2- 172.21.15.87:0/4031578622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c15b9e0 0x7ff87c0cc410 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009dc0 tx=0x7ff86c01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee669faf6d64bde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.387+0000 7ff87b400640 1 -- 172.21.15.87:0/4031578622 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0b2070 con 0x7ff87c15b9e0 2024-04-25T10:20:14.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.387+0000 7ff87b400640 1 -- 172.21.15.87:0/4031578622 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c0a5850 con 0x7ff87c15b9e0 2024-04-25T10:20:14.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.387+0000 7ff87b400640 1 -- 172.21.15.87:0/4031578622 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0a5c20 con 0x7ff87c15b9e0 2024-04-25T10:20:14.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.387+0000 7ff883800640 1 -- 172.21.15.87:0/4031578622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c15b9e0 msgr2=0x7ff87c0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.387+0000 7ff883800640 1 --2- 172.21.15.87:0/4031578622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c15b9e0 0x7ff87c0cc410 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009dc0 tx=0x7ff86c01c990 comp rx=0 tx=0).stop 2024-04-25T10:20:14.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.387+0000 7ff883800640 1 -- 172.21.15.87:0/4031578622 shutdown_connections 2024-04-25T10:20:14.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 --2- 172.21.15.87:0/4031578622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c15b9e0 0x7ff87c0cc410 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- 172.21.15.87:0/4031578622 >> 172.21.15.87:0/4031578622 conn(0x7ff87c09a9d0 msgr2=0x7ff87c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:14.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- 172.21.15.87:0/4031578622 shutdown_connections 2024-04-25T10:20:14.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- 172.21.15.87:0/4031578622 wait complete. 2024-04-25T10:20:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 Processor -- start 2024-04-25T10:20:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- start start 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c0d1c30 0x7ff87c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff87c15b9e0 0x7ff87c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87c15bdc0 0x7ff87c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0ccd60 con 0x7ff87c15b9e0 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0c6c00 con 0x7ff87c15bdc0 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c169840 con 0x7ff87c0d1c30 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff87c15b9e0 0x7ff87c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff87c15b9e0 0x7ff87c158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57624/0 (socket says 172.21.15.87:57624) 2024-04-25T10:20:14.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 -- 172.21.15.87:0/1210438206 learned_addr learned my addr 172.21.15.87:0/1210438206 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff881800640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87c15bdc0 0x7ff87c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff880e00640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c0d1c30 0x7ff87c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 -- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c0d1c30 msgr2=0x7ff87c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c0d1c30 0x7ff87c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 -- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87c15bdc0 msgr2=0x7ff87c153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87c15bdc0 0x7ff87c153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 -- 172.21.15.87:0/1210438206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c155680 con 0x7ff87c15b9e0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff87be00640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff87c15b9e0 0x7ff87c158be0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ff86800c420 tx=0x7ff86800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff878c00640 1 -- 172.21.15.87:0/1210438206 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868017070 con 0x7ff87c15b9e0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff878c00640 1 -- 172.21.15.87:0/1210438206 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868005cf0 con 0x7ff87c15b9e0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff878c00640 1 -- 172.21.15.87:0/1210438206 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868007310 con 0x7ff87c15b9e0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c1559c0 con 0x7ff87c15b9e0 2024-04-25T10:20:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.391+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c156200 con 0x7ff87c15b9e0 2024-04-25T10:20:14.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.395+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c156400 con 0x7ff87c15b9e0 2024-04-25T10:20:14.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.395+0000 7ff878c00640 1 -- 172.21.15.87:0/1210438206 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff86801a070 con 0x7ff87c15b9e0 2024-04-25T10:20:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.395+0000 7ff878c00640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff85c066f60 0x7ff85c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.395+0000 7ff878c00640 1 -- 172.21.15.87:0/1210438206 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff868084500 con 0x7ff87c15b9e0 2024-04-25T10:20:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.395+0000 7ff880e00640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff85c066f60 0x7ff85c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.395+0000 7ff880e00640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff85c066f60 0x7ff85c069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0af000 tx=0x7ff86c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.399+0000 7ff878c00640 1 -- 172.21.15.87:0/1210438206 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff87c156400 con 0x7ff87c15b9e0 2024-04-25T10:20:14.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff87c04ef30 con 0x7ff87c15b9e0 2024-04-25T10:20:14.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff878c00640 1 -- 172.21.15.87:0/1210438206 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7ff87c1559c0 con 0x7ff87c15b9e0 2024-04-25T10:20:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:14.610 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":496,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076434023,"bytes_used":2593071104,"bytes_avail":770501042176,"bytes_total":773094113280,"write_bytes_sec":58856713,"read_op_per_sec":0,"write_op_per_sec":59},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:19:17.213750+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:14.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff85c066f60 msgr2=0x7ff85c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff85c066f60 0x7ff85c069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0af000 tx=0x7ff86c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff87c15b9e0 msgr2=0x7ff87c158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff87c15b9e0 0x7ff87c158be0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ff86800c420 tx=0x7ff86800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 shutdown_connections 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff85c066f60 0x7ff85c069420 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87c15bdc0 0x7ff87c153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff87c15b9e0 0x7ff87c158be0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 --2- 172.21.15.87:0/1210438206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87c0d1c30 0x7ff87c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 >> 172.21.15.87:0/1210438206 conn(0x7ff87c09a9d0 msgr2=0x7ff87c06f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 shutdown_connections 2024-04-25T10:20:14.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:14.607+0000 7ff883800640 1 -- 172.21.15.87:0/1210438206 wait complete. 2024-04-25T10:20:14.630 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:20:15.634 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:15.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.775+0000 7fc2b8c00640 1 Processor -- start 2024-04-25T10:20:15.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.775+0000 7fc2b8c00640 1 -- start start 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b8c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc2b40d0040 0x7fc2b406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b8c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b40b2770 con 0x7fc2b40cda10 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b8c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b406c670 con 0x7fc2b415bc40 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b8c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b406c950 con 0x7fc2b40d0040 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc2b40d0040 0x7fc2b406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc2b40d0040 0x7fc2b406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57626/0 (socket says 172.21.15.87:57626) 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b2800640 1 -- 172.21.15.87:0/3157141787 learned_addr learned my addr 172.21.15.87:0/3157141787 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b1400640 1 --1- 172.21.15.87:0/3157141787 >> v1:172.21.15.87:6789/0 conn(0x7fc2b40cda10 0x7fc2b40cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53052/0 (socket says 172.21.15.87:53052) 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b2800640 1 -- 172.21.15.87:0/3157141787 >> v1:172.21.15.169:6789/0 conn(0x7fc2b415bc40 legacy=0x7fc2b40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b2800640 1 -- 172.21.15.87:0/3157141787 >> v1:172.21.15.87:6789/0 conn(0x7fc2b40cda10 legacy=0x7fc2b40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:15.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.779+0000 7fc2b2800640 1 -- 172.21.15.87:0/3157141787 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b406cc30 con 0x7fc2b40d0040 2024-04-25T10:20:15.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b2800640 1 --2- 172.21.15.87:0/3157141787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b406c090 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009870 tx=0x7fc2ac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c6146b9fb6961c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:15.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b0a00640 1 -- 172.21.15.87:0/3157141787 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0b1070 con 0x7fc2b40d0040 2024-04-25T10:20:15.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b0a00640 1 -- 172.21.15.87:0/3157141787 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac0a8050 con 0x7fc2b40d0040 2024-04-25T10:20:15.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b0a00640 1 -- 172.21.15.87:0/3157141787 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0ac860 con 0x7fc2b40d0040 2024-04-25T10:20:15.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 -- 172.21.15.87:0/3157141787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 msgr2=0x7fc2b406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:15.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/3157141787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b406c090 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009870 tx=0x7fc2ac01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 -- 172.21.15.87:0/3157141787 shutdown_connections 2024-04-25T10:20:15.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/3157141787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b406c090 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 -- 172.21.15.87:0/3157141787 >> 172.21.15.87:0/3157141787 conn(0x7fc2b4066b70 msgr2=0x7fc2b4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:15.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 -- 172.21.15.87:0/3157141787 shutdown_connections 2024-04-25T10:20:15.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 -- 172.21.15.87:0/3157141787 wait complete. 2024-04-25T10:20:15.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 Processor -- start 2024-04-25T10:20:15.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.783+0000 7fc2b8c00640 1 -- start start 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b40cda10 0x7fc2b41491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b4071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b415bc40 0x7fc2b4155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4169980 con 0x7fc2b40d0040 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4169800 con 0x7fc2b40cda10 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4169b00 con 0x7fc2b415bc40 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b4071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b4071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57634/0 (socket says 172.21.15.87:57634) 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 -- 172.21.15.87:0/1916201197 learned_addr learned my addr 172.21.15.87:0/1916201197 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1e00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b40cda10 0x7fc2b41491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b2800640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b415bc40 0x7fc2b4155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:15.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 -- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b415bc40 msgr2=0x7fc2b4155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b415bc40 0x7fc2b4155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 -- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b40cda10 msgr2=0x7fc2b41491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b40cda10 0x7fc2b41491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 -- 172.21.15.87:0/1916201197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b41509b0 con 0x7fc2b40d0040 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1e00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b40cda10 0x7fc2b41491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b1400640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b4071610 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c460 tx=0x7fc2a000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2aa000640 1 -- 172.21.15.87:0/1916201197 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0018070 con 0x7fc2b40d0040 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2aa000640 1 -- 172.21.15.87:0/1916201197 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0007480 con 0x7fc2b40d0040 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b4150c90 con 0x7fc2b40d0040 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2aa000640 1 -- 172.21.15.87:0/1916201197 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00047f0 con 0x7fc2b40d0040 2024-04-25T10:20:15.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b41496f0 con 0x7fc2b40d0040 2024-04-25T10:20:15.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b404ef30 con 0x7fc2b40d0040 2024-04-25T10:20:15.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.787+0000 7fc2aa000640 1 -- 172.21.15.87:0/1916201197 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2a001b070 con 0x7fc2b40d0040 2024-04-25T10:20:15.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.791+0000 7fc2aa000640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc294066f60 0x7fc294069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:15.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.791+0000 7fc2aa000640 1 -- 172.21.15.87:0/1916201197 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2a0084160 con 0x7fc2b40d0040 2024-04-25T10:20:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.791+0000 7fc2b1e00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc294066f60 0x7fc294069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.791+0000 7fc2b1e00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc294066f60 0x7fc294069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001e10 tx=0x7fc2a4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:15.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.795+0000 7fc2aa000640 1 -- 172.21.15.87:0/1916201197 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2a0054b20 con 0x7fc2b40d0040 2024-04-25T10:20:15.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.979+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2b406d960 con 0x7fc2b40d0040 2024-04-25T10:20:15.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.979+0000 7fc2aa000640 1 -- 172.21.15.87:0/1916201197 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc2a0058bc0 con 0x7fc2b40d0040 2024-04-25T10:20:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:15.983 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:15.983 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:15.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.979+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc294066f60 msgr2=0x7fc294069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:15.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.979+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc294066f60 0x7fc294069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001e10 tx=0x7fc2a4009040 comp rx=0 tx=0).stop 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 msgr2=0x7fc2b4071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b4071610 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c460 tx=0x7fc2a000c920 comp rx=0 tx=0).stop 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 shutdown_connections 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc294066f60 0x7fc294069420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b415bc40 0x7fc2b4155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40d0040 0x7fc2b4071610 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 --2- 172.21.15.87:0/1916201197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b40cda10 0x7fc2b41491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 >> 172.21.15.87:0/1916201197 conn(0x7fc2b4066b70 msgr2=0x7fc2b4157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 shutdown_connections 2024-04-25T10:20:15.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:15.983+0000 7fc2b8c00640 1 -- 172.21.15.87:0/1916201197 wait complete. 2024-04-25T10:20:16.004 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:20:16.004 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:16.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9523400640 1 Processor -- start 2024-04-25T10:20:16.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9523400640 1 -- start start 2024-04-25T10:20:16.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9523400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c071f70 0x7f951c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9523400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c070ed0 con 0x7f951c0b2fc0 2024-04-25T10:20:16.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9523400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c0711b0 con 0x7f951c15a0e0 2024-04-25T10:20:16.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9523400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c071490 con 0x7f951c071f70 2024-04-25T10:20:16.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9521400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f951c0b2fc0 0x7f951c06c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53060/0 (socket says 172.21.15.87:53060) 2024-04-25T10:20:16.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.135+0000 7f9521400640 1 -- 172.21.15.87:0/861086756 learned_addr learned my addr 172.21.15.87:0/861086756 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:16.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951be00640 1 --2- 172.21.15.87:0/861086756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c071f70 0x7f951c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951b400640 1 -- 172.21.15.87:0/861086756 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555908203 0 0) 0x7f951c070ed0 con 0x7f951c0b2fc0 2024-04-25T10:20:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951b400640 1 -- 172.21.15.87:0/861086756 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9500003450 con 0x7f951c0b2fc0 2024-04-25T10:20:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951b400640 1 -- 172.21.15.87:0/861086756 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967783920 0 0) 0x7f951c0711b0 con 0x7f951c15a0e0 2024-04-25T10:20:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951b400640 1 -- 172.21.15.87:0/861086756 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f951c070ed0 con 0x7f951c15a0e0 2024-04-25T10:20:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951be00640 1 -- 172.21.15.87:0/861086756 >> v1:172.21.15.169:6789/0 conn(0x7f951c15a0e0 legacy=0x7f951c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951be00640 1 -- 172.21.15.87:0/861086756 >> v1:172.21.15.87:6789/0 conn(0x7f951c0b2fc0 legacy=0x7f951c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951be00640 1 -- 172.21.15.87:0/861086756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c071770 con 0x7f951c071f70 2024-04-25T10:20:16.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951be00640 1 --2- 172.21.15.87:0/861086756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c071f70 0x7f951c0b2a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95080097f0 tx=0x7f950801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c90e205258ae784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.139+0000 7f951b400640 1 -- 172.21.15.87:0/861086756 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080b0070 con 0x7f951c071f70 2024-04-25T10:20:16.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f951b400640 1 -- 172.21.15.87:0/861086756 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080ab600 con 0x7f951c071f70 2024-04-25T10:20:16.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f951b400640 1 -- 172.21.15.87:0/861086756 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080ab9d0 con 0x7f951c071f70 2024-04-25T10:20:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- 172.21.15.87:0/861086756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c071f70 msgr2=0x7f951c0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 --2- 172.21.15.87:0/861086756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c071f70 0x7f951c0b2a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95080097f0 tx=0x7f950801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- 172.21.15.87:0/861086756 shutdown_connections 2024-04-25T10:20:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 --2- 172.21.15.87:0/861086756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c071f70 0x7f951c0b2a80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- 172.21.15.87:0/861086756 >> 172.21.15.87:0/861086756 conn(0x7f951c066b70 msgr2=0x7f951c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- 172.21.15.87:0/861086756 shutdown_connections 2024-04-25T10:20:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- 172.21.15.87:0/861086756 wait complete. 2024-04-25T10:20:16.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 Processor -- start 2024-04-25T10:20:16.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- start start 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c071f70 0x7f951c1f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0b2fc0 0x7f951c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15a0e0 0x7f951c14ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c070b50 con 0x7f951c071f70 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c15a4c0 con 0x7f951c15a0e0 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9523400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0723f0 con 0x7f951c0b2fc0 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9520a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c071f70 0x7f951c1f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9520a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c071f70 0x7f951c1f7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57650/0 (socket says 172.21.15.87:57650) 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9520a00640 1 -- 172.21.15.87:0/320410452 learned_addr learned my addr 172.21.15.87:0/320410452 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f951be00640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0b2fc0 0x7f951c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.143+0000 7f9521400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15a0e0 0x7f951c14ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9520a00640 1 -- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0b2fc0 msgr2=0x7f951c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9520a00640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0b2fc0 0x7f951c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9520a00640 1 -- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15a0e0 msgr2=0x7f951c14ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9520a00640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15a0e0 0x7f951c14ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9520a00640 1 -- 172.21.15.87:0/320410452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c14fa50 con 0x7f951c071f70 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9521400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15a0e0 0x7f951c14ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9520a00640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c071f70 0x7f951c1f7de0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c770 tx=0x7f950c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9518c00640 1 -- 172.21.15.87:0/320410452 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c003b80 con 0x7f951c071f70 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9518c00640 1 -- 172.21.15.87:0/320410452 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c003d20 con 0x7f951c071f70 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c154ad0 con 0x7f951c071f70 2024-04-25T10:20:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9518c00640 1 -- 172.21.15.87:0/320410452 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c00aa90 con 0x7f951c071f70 2024-04-25T10:20:16.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c155090 con 0x7f951c071f70 2024-04-25T10:20:16.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c04ef30 con 0x7f951c071f70 2024-04-25T10:20:16.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9518c00640 1 -- 172.21.15.87:0/320410452 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f950c00ac30 con 0x7f951c071f70 2024-04-25T10:20:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9518c00640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94fc066f60 0x7f94fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f9518c00640 1 -- 172.21.15.87:0/320410452 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f950c084fe0 con 0x7f951c071f70 2024-04-25T10:20:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.147+0000 7f951be00640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94fc066f60 0x7f94fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.151+0000 7f951be00640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94fc066f60 0x7f94fc069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f95080ad000 tx=0x7f9508008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.151+0000 7f9518c00640 1 -- 172.21.15.87:0/320410452 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f950c055f50 con 0x7f951c071f70 2024-04-25T10:20:16.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.363+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f951c0703e0 con 0x7f951c071f70 2024-04-25T10:20:16.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.363+0000 7f9518c00640 1 -- 172.21.15.87:0/320410452 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f950c055850 con 0x7f951c071f70 2024-04-25T10:20:16.368 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:16.368 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":498,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076434023,"bytes_used":2593071104,"bytes_avail":770501042176,"bytes_total":773094113280,"write_bytes_sec":37226941,"read_op_per_sec":0,"write_op_per_sec":37},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:20:15.225075+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:16.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94fc066f60 msgr2=0x7f94fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94fc066f60 0x7f94fc069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f95080ad000 tx=0x7f9508008040 comp rx=0 tx=0).stop 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c071f70 msgr2=0x7f951c1f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c071f70 0x7f951c1f7de0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c770 tx=0x7f950c00cc30 comp rx=0 tx=0).stop 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 shutdown_connections 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94fc066f60 0x7f94fc069420 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15a0e0 0x7f951c14ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0b2fc0 0x7f951c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 --2- 172.21.15.87:0/320410452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c071f70 0x7f951c1f7de0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 >> 172.21.15.87:0/320410452 conn(0x7f951c066b70 msgr2=0x7f951c1579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 shutdown_connections 2024-04-25T10:20:16.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.367+0000 7f9523400640 1 -- 172.21.15.87:0/320410452 wait complete. 2024-04-25T10:20:16.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:20:16.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:16.388 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T10:20:16.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b64a00640 1 Processor -- start 2024-04-25T10:20:16.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b64a00640 1 -- start start 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b64a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2b600aeda0 0x7f2b60071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b64a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b60072020 0x7f2b60070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b64a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b60159920 con 0x7f2b60159540 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b64a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6015c2f0 con 0x7f2b60072020 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b64a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6015c5d0 con 0x7f2b600aeda0 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5de00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2b60159540 0x7f2b600ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53066/0 (socket says 172.21.15.87:53066) 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5de00640 1 -- 172.21.15.87:0/2042557948 learned_addr learned my addr 172.21.15.87:0/2042557948 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:16.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2042557948 >> v2:172.21.15.87:3300/0 conn(0x7f2b60072020 0x7f2b60070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b55400640 1 --2- 172.21.15.87:0/2042557948 >> v2:172.21.15.169:3300/0 conn(0x7f2b600aeda0 0x7f2b60071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5d400640 1 -- 172.21.15.87:0/2042557948 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079856048 0 0) 0x7f2b60159920 con 0x7f2b60159540 2024-04-25T10:20:16.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5d400640 1 -- 172.21.15.87:0/2042557948 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b44003530 con 0x7f2b60159540 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5e800640 1 -- 172.21.15.87:0/2042557948 >> v2:172.21.15.169:3300/0 conn(0x7f2b600aeda0 msgr2=0x7f2b60071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2042557948 >> v2:172.21.15.169:3300/0 conn(0x7f2b600aeda0 0x7f2b60071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5e800640 1 -- 172.21.15.87:0/2042557948 >> v1:172.21.15.87:6789/0 conn(0x7f2b60159540 legacy=0x7f2b600ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.523+0000 7f2b5e800640 1 -- 172.21.15.87:0/2042557948 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6015c8b0 con 0x7f2b60072020 2024-04-25T10:20:16.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2042557948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60070350 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2b600cd510 tx=0x7f2b5801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67581ba6759fe3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b5d400640 1 -- 172.21.15.87:0/2042557948 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580b2070 con 0x7f2b60072020 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b5d400640 1 -- 172.21.15.87:0/2042557948 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b580a59a0 con 0x7f2b60072020 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b5d400640 1 -- 172.21.15.87:0/2042557948 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580a5d70 con 0x7f2b60072020 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 -- 172.21.15.87:0/2042557948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 msgr2=0x7f2b60070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2042557948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60070350 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2b600cd510 tx=0x7f2b5801c900 comp rx=0 tx=0).stop 2024-04-25T10:20:16.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 -- 172.21.15.87:0/2042557948 shutdown_connections 2024-04-25T10:20:16.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2042557948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60070350 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2042557948 >> v2:172.21.15.169:3300/0 conn(0x7f2b600aeda0 0x7f2b60071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 -- 172.21.15.87:0/2042557948 >> 172.21.15.87:0/2042557948 conn(0x7f2b600669a0 msgr2=0x7f2b60066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 -- 172.21.15.87:0/2042557948 shutdown_connections 2024-04-25T10:20:16.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.527+0000 7f2b64a00640 1 -- 172.21.15.87:0/2042557948 wait complete. 2024-04-25T10:20:16.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 Processor -- start 2024-04-25T10:20:16.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 -- start start 2024-04-25T10:20:16.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b600aeda0 0x7f2b60152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b60159540 0x7f2b60155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600710a0 con 0x7f2b60072020 2024-04-25T10:20:16.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6006c0d0 con 0x7f2b60159540 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60071220 con 0x7f2b600aeda0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57664/0 (socket says 172.21.15.87:57664) 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 -- 172.21.15.87:0/2290261447 learned_addr learned my addr 172.21.15.87:0/2290261447 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b60159540 0x7f2b60155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b55400640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b600aeda0 0x7f2b60152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 -- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b600aeda0 msgr2=0x7f2b60152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b600aeda0 0x7f2b60152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 -- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b60159540 msgr2=0x7f2b60155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b60159540 0x7f2b60155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 -- 172.21.15.87:0/2290261447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6014ef30 con 0x7f2b60072020 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60148d40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000c450 tx=0x7f2b5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b56a00640 1 -- 172.21.15.87:0/2290261447 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50017070 con 0x7f2b60072020 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b56a00640 1 -- 172.21.15.87:0/2290261447 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50005cf0 con 0x7f2b60072020 2024-04-25T10:20:16.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6014f210 con 0x7f2b60072020 2024-04-25T10:20:16.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b56a00640 1 -- 172.21.15.87:0/2290261447 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50007290 con 0x7f2b60072020 2024-04-25T10:20:16.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.531+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6014f7d0 con 0x7f2b60072020 2024-04-25T10:20:16.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b56a00640 1 -- 172.21.15.87:0/2290261447 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b5001a070 con 0x7f2b60072020 2024-04-25T10:20:16.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b56a00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b3c066f90 0x7f2b3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b56a00640 1 -- 172.21.15.87:0/2290261447 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b50084380 con 0x7f2b60072020 2024-04-25T10:20:16.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b55400640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b3c066f90 0x7f2b3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 Processor -- start 2024-04-25T10:20:16.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b55400640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b3c066f90 0x7f2b3c069450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2b60070130 tx=0x7f2b48008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 -- start start 2024-04-25T10:20:16.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b254db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2552f0 0x55691b20ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b20b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b057ff0 con 0x55691b2549d0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691afe0150 con 0x55691b2552f0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691b054090 con 0x55691b20b0a0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b254db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b254db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57676/0 (socket says 172.21.15.87:57676) 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 -- 172.21.15.87:0/3208558548 learned_addr learned my addr 172.21.15.87:0/3208558548 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b55400640 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2552f0 0x55691b20ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5e800640 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b20b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 -- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 msgr2=0x55691b20b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b20b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 -- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2552f0 msgr2=0x55691b20ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2552f0 0x55691b20ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.535+0000 7f2b5de00640 1 -- 172.21.15.87:0/3208558548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691b1ee0d0 con 0x55691b2549d0 2024-04-25T10:20:16.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5de00640 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b254db0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2b500510e0 tx=0x7f2b500548f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84725ceeced492c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b4ea00640 1 -- 172.21.15.87:0/3208558548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5005b0c0 con 0x55691b2549d0 2024-04-25T10:20:16.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b4ea00640 1 -- 172.21.15.87:0/3208558548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50052f80 con 0x55691b2549d0 2024-04-25T10:20:16.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b4ea00640 1 -- 172.21.15.87:0/3208558548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b500512f0 con 0x55691b2549d0 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- 172.21.15.87:0/3208558548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 msgr2=0x55691b254db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b254db0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2b500510e0 tx=0x7f2b500548f0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- 172.21.15.87:0/3208558548 shutdown_connections 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b20b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2552f0 0x55691b20ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 --2- 172.21.15.87:0/3208558548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b254db0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- 172.21.15.87:0/3208558548 >> 172.21.15.87:0/3208558548 conn(0x55691b207020 msgr2=0x55691b209440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- 172.21.15.87:0/3208558548 shutdown_connections 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- 172.21.15.87:0/3208558548 wait complete. 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- start start 2024-04-25T10:20:16.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 Processor -- start 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b2397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2549d0 0x55691b239d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2552f0 0x55691b23a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b01c250 con 0x55691b2552f0 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b012690 con 0x55691b2549d0 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691afe0a40 con 0x55691b20b0a0 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b67e11000 10 client.0 _ll_register_callbacks cb 0x55691b1f5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2552f0 0x55691b23a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b55400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2549d0 0x55691b239d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b2397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2552f0 0x55691b23a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57680/0 (socket says 172.21.15.87:57680) 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b55400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2549d0 0x55691b239d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:52854/0 (socket says 172.21.15.87:52854) 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 -- 172.21.15.87:0/2429898034 learned_addr learned my addr 172.21.15.87:0/2429898034 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 -- 172.21.15.87:0/2429898034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 msgr2=0x55691b2397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b2397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 -- 172.21.15.87:0/2429898034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2549d0 msgr2=0x55691b239d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2549d0 0x55691b239d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.539+0000 7f2b5e800640 1 -- 172.21.15.87:0/2429898034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55691b1051c0 con 0x55691b2552f0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2552f0 0x55691b23a270 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2b580b0040 tx=0x7f2b5801b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb980f35a9d98a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 1 -- 172.21.15.87:0/2429898034 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580b2040 con 0x55691b2552f0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 1 -- 172.21.15.87:0/2429898034 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b580ad440 con 0x55691b2552f0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b50052f80 con 0x55691b2552f0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 1 -- 172.21.15.87:0/2429898034 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b580c9050 con 0x55691b2552f0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 10 client.8256 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55691b101170 con 0x55691b2552f0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 1 -- 172.21.15.87:0/2429898034 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50052f80 con 0x55691b2552f0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 1 -- 172.21.15.87:0/2429898034 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b580cc020 con 0x55691b2552f0 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 10 client.8256 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 10 client.8256 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 20 client.8256 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b264c80 0x55691b267070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 4 client.8256 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55691aff3c60 con 0x55691b264c80 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b264c80 0x55691b267070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b264c80 0x55691b267070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf4f857afe79f6d server_cookie=dd4b399f763eba16 in_seq=0 out_seq=0 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 10 client.8256 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 1 -- 172.21.15.87:0/2429898034 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55691aff3c60 con 0x55691b264c80 2024-04-25T10:20:16.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 10 client.8256 handle_command_reply: tid=0 2024-04-25T10:20:16.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 client.8256 shutdown 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2552f0 msgr2=0x55691b23a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2552f0 0x55691b23a270 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2b580b0040 tx=0x7f2b5801b140 comp rx=0 tx=0).stop 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 shutdown_connections 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b264c80 0x55691b267070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2552f0 0x55691b23a270 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b2549d0 0x55691b239d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b43400640 0 client.8256 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2429898034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b20b0a0 0x55691b2397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 >> 172.21.15.87:0/2429898034 conn(0x55691b207020 msgr2=0x55691b209ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 shutdown_connections 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 1 -- 172.21.15.87:0/2429898034 wait complete. 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.543+0000 7f2b67e11000 20 client.8256 trim_cache size 0 max 16384 2024-04-25T10:20:16.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 Processor -- start 2024-04-25T10:20:16.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 -- start start 2024-04-25T10:20:16.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b2530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b253610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b253b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60202a90 con 0x55691b2549d0 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60157b70 con 0x55691b20b0a0 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601515e0 con 0x55691b2552f0 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b253610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b253610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57690/0 (socket says 172.21.15.87:57690) 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 -- 172.21.15.87:0/1175893268 learned_addr learned my addr 172.21.15.87:0/1175893268 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b5de00640 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b2530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b5e800640 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b253b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 -- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 msgr2=0x55691b253b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b253b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 -- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 msgr2=0x55691b2530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b2530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 -- 172.21.15.87:0/1175893268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691b09ede0 con 0x55691b2549d0 2024-04-25T10:20:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b55400640 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b253610 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800fc20 tx=0x7f2b48012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83425ddb6738847c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b4e000640 1 -- 172.21.15.87:0/1175893268 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801b040 con 0x55691b2549d0 2024-04-25T10:20:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b4e000640 1 -- 172.21.15.87:0/1175893268 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4801c040 con 0x55691b2549d0 2024-04-25T10:20:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.559+0000 7f2b4e000640 1 -- 172.21.15.87:0/1175893268 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48023440 con 0x55691b2549d0 2024-04-25T10:20:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/1175893268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 msgr2=0x55691b253610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b253610 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800fc20 tx=0x7f2b48012000 comp rx=0 tx=0).stop 2024-04-25T10:20:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/1175893268 shutdown_connections 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b253b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b253610 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- 172.21.15.87:0/1175893268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b2530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/1175893268 >> 172.21.15.87:0/1175893268 conn(0x55691b207020 msgr2=0x55691b261fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/1175893268 shutdown_connections 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/1175893268 wait complete. 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- start start 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 Processor -- start 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b20a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b2680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b268600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691aff2e00 con 0x55691b2549d0 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60157570 con 0x55691b20b0a0 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691b057300 con 0x55691b2552f0 2024-04-25T10:20:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 10 client.0 _ll_register_callbacks cb 0x55691b1d6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b2680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b2680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57702/0 (socket says 172.21.15.87:57702) 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 -- 172.21.15.87:0/2363259858 learned_addr learned my addr 172.21.15.87:0/2363259858 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b20a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b268600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 -- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 msgr2=0x55691b268600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b268600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 -- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 msgr2=0x55691b20a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b20a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 -- 172.21.15.87:0/2363259858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55691aff98e0 con 0x55691b2549d0 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b20a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b5e800640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b268600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b55400640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b2680c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x55691b0aff20 tx=0x7f2b48004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bd5acbee36712c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b43400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b43400640 1 -- 172.21.15.87:0/2363259858 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801b040 con 0x55691b2549d0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b43400640 1 -- 172.21.15.87:0/2363259858 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b48036ad0 con 0x55691b2549d0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b4801c040 con 0x55691b2549d0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b43400640 1 -- 172.21.15.87:0/2363259858 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b4803d050 con 0x55691b2549d0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 10 client.8262 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55691b0b18f0 con 0x55691b2549d0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b43400640 1 -- 172.21.15.87:0/2363259858 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4801c040 con 0x55691b2549d0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b43400640 1 -- 172.21.15.87:0/2363259858 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b48040020 con 0x55691b2549d0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 10 client.8262 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 10 client.8262 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 20 client.8262 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b2b4f20 0x55691b2b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 4 client.8262 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.563+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55691b065530 con 0x55691b2b4f20 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b2b4f20 0x55691b2b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b5de00640 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b2b4f20 0x55691b2b5330 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14dc646acf4baa6 server_cookie=16ca47b7192bb8ed in_seq=0 out_seq=0 2024-04-25T10:20:16.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b43400640 10 client.8262 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b43400640 1 -- 172.21.15.87:0/2363259858 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55691b065530 con 0x55691b2b4f20 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b43400640 10 client.8262 handle_command_reply: tid=0 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 client.8262 shutdown 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 msgr2=0x55691b2680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b2680c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x55691b0aff20 tx=0x7f2b48004740 comp rx=0 tx=0).stop 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 shutdown_connections 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x55691b2b4f20 0x55691b2b5330 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55691b2552f0 0x55691b268600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55691b2549d0 0x55691b2680c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b43400640 0 client.8262 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 --2- 172.21.15.87:0/2363259858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55691b20b0a0 0x55691b20a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 >> 172.21.15.87:0/2363259858 conn(0x55691b207020 msgr2=0x55691b2530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 shutdown_connections 2024-04-25T10:20:16.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 1 -- 172.21.15.87:0/2363259858 wait complete. 2024-04-25T10:20:16.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b67e11000 20 client.8262 trim_cache size 0 max 16384 2024-04-25T10:20:16.574 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:20:16.574 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0 2024-04-25T10:20:16.574 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b3c066f90 msgr2=0x7f2b3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b3c066f90 0x7f2b3c069450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2b60070130 tx=0x7f2b48008040 comp rx=0 tx=0).stop 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 msgr2=0x7f2b60148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60148d40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000c450 tx=0x7f2b5000c910 comp rx=0 tx=0).stop 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 shutdown_connections 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2b3c066f90 0x7f2b3c069450 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b60159540 0x7f2b60155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b600aeda0 0x7f2b60152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 --2- 172.21.15.87:0/2290261447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b60072020 0x7f2b60148d40 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 >> 172.21.15.87:0/2290261447 conn(0x7f2b600669a0 msgr2=0x7f2b600cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 shutdown_connections 2024-04-25T10:20:16.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.567+0000 7f2b64a00640 1 -- 172.21.15.87:0/2290261447 wait complete. 2024-04-25T10:20:16.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:16.588 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:20:16.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ea00640 1 Processor -- start 2024-04-25T10:20:16.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ea00640 1 -- start start 2024-04-25T10:20:16.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63080aeda0 0x7f6308071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 0x7f6308070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6308159920 con 0x7f6308159540 2024-04-25T10:20:16.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630815c2f0 con 0x7f63080aeda0 2024-04-25T10:20:16.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630815c5d0 con 0x7f6308072020 2024-04-25T10:20:16.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6308159540 0x7f63080ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53078/0 (socket says 172.21.15.87:53078) 2024-04-25T10:20:16.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 -- 172.21.15.87:0/1985376652 learned_addr learned my addr 172.21.15.87:0/1985376652 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:16.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307400640 1 --2- 172.21.15.87:0/1985376652 >> v2:172.21.15.169:3300/0 conn(0x7f63080aeda0 0x7f6308071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091542907 0 0) 0x7f6308159920 con 0x7f6308159540 2024-04-25T10:20:16.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ec003530 con 0x7f6308159540 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ca00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 0x7f6308070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208313546 0 0) 0x7f62ec003530 con 0x7f6308159540 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 msgr2=0x7f6308070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 0x7f6308070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 >> v2:172.21.15.169:3300/0 conn(0x7f63080aeda0 msgr2=0x7f6308071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 --2- 172.21.15.87:0/1985376652 >> v2:172.21.15.169:3300/0 conn(0x7f63080aeda0 0x7f6308071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630815c8b0 con 0x7f6308159540 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f62f8001680 con 0x7f6308159540 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 >> v1:172.21.15.87:6789/0 conn(0x7f6308159540 legacy=0x7f63080ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62ec0039c0 0x7f62ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec004810 0x7f62ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 0x7f62ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630815c2f0 con 0x7f62ec0039c0 2024-04-25T10:20:16.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630815c5d0 con 0x7f62ec004810 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec00b9d0 con 0x7f6308072020 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62ec0039c0 0x7f62ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307400640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec004810 0x7f62ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ca00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 0x7f62ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 -- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 msgr2=0x7f62ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 0x7f62ec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 -- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec004810 msgr2=0x7f62ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec004810 0x7f62ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f6307e00640 1 -- 172.21.15.87:0/1985376652 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec003530 con 0x7f62ec0039c0 2024-04-25T10:20:16.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.723+0000 7f630ca00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6308072020 0x7f62ec0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f6307e00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62ec0039c0 0x7f62ec003de0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c2c0 tx=0x7f62f801cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd26f76c1de6f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8007d60 con 0x7f62ec0039c0 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f80a9020 con 0x7f62ec0039c0 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f6306a00640 1 -- 172.21.15.87:0/1985376652 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80058e0 con 0x7f62ec0039c0 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f630ea00640 1 -- 172.21.15.87:0/1985376652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62ec0039c0 msgr2=0x7f62ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f630ea00640 1 --2- 172.21.15.87:0/1985376652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62ec0039c0 0x7f62ec003de0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c2c0 tx=0x7f62f801cd10 comp rx=0 tx=0).stop 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f630ca00640 1 -- 172.21.15.87:0/1985376652 reap_dead start 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f630ea00640 1 -- 172.21.15.87:0/1985376652 shutdown_connections 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f630ea00640 1 -- 172.21.15.87:0/1985376652 >> 172.21.15.87:0/1985376652 conn(0x7f63080669a0 msgr2=0x7f6308066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f630ea00640 1 -- 172.21.15.87:0/1985376652 shutdown_connections 2024-04-25T10:20:16.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.727+0000 7f630ea00640 1 -- 172.21.15.87:0/1985376652 wait complete. 2024-04-25T10:20:16.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 Processor -- start 2024-04-25T10:20:16.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 -- start start 2024-04-25T10:20:16.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec0039c0 0x7f63080b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630814fb30 0x7f630815a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630814a9a0 0x7f630814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308070b60 con 0x7f630814fb30 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308072430 con 0x7f62ec0039c0 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6308070ce0 con 0x7f630814a9a0 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630814fb30 0x7f630815a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630814fb30 0x7f630815a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57722/0 (socket says 172.21.15.87:57722) 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 -- 172.21.15.87:0/2699178936 learned_addr learned my addr 172.21.15.87:0/2699178936 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ca00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630814a9a0 0x7f630814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307e00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec0039c0 0x7f63080b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 -- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630814a9a0 msgr2=0x7f630814ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630814a9a0 0x7f630814ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 -- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec0039c0 msgr2=0x7f63080b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec0039c0 0x7f63080b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 -- 172.21.15.87:0/2699178936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630815a620 con 0x7f630814fb30 2024-04-25T10:20:16.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ca00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630814a9a0 0x7f630814ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307e00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec0039c0 0x7f63080b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f6307400640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630814fb30 0x7f630815a0e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f62f4011940 tx=0x7f62f4011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f62ffe00640 1 -- 172.21.15.87:0/2699178936 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4019070 con 0x7f630814fb30 2024-04-25T10:20:16.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f62ffe00640 1 -- 172.21.15.87:0/2699178936 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4004240 con 0x7f630814fb30 2024-04-25T10:20:16.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630814a370 con 0x7f630814fb30 2024-04-25T10:20:16.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f62ffe00640 1 -- 172.21.15.87:0/2699178936 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400f8e0 con 0x7f630814fb30 2024-04-25T10:20:16.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.731+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63080b3280 con 0x7f630814fb30 2024-04-25T10:20:16.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.735+0000 7f62ffe00640 1 -- 172.21.15.87:0/2699178936 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62f401c070 con 0x7f630814fb30 2024-04-25T10:20:16.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.735+0000 7f62ffe00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62e8066f90 0x7f62e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.735+0000 7f62ffe00640 1 -- 172.21.15.87:0/2699178936 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f4085480 con 0x7f630814fb30 2024-04-25T10:20:16.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.735+0000 7f6307e00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62e8066f90 0x7f62e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.735+0000 7f6311fa4000 1 Processor -- start 2024-04-25T10:20:16.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.735+0000 7f6311fa4000 1 -- start start 2024-04-25T10:20:16.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.735+0000 7f6307e00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62e8066f90 0x7f62e8069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f62f80aa000 tx=0x7f62f8001a50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140169d0 0x563214016db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140172f0 0x563213fccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563213fcd0a0 0x563213fcd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563213e19ff0 con 0x5632140169d0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x563213da2150 con 0x563213fcd0a0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x563213e16090 con 0x5632140172f0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140169d0 0x563214016db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140169d0 0x563214016db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57730/0 (socket says 172.21.15.87:57730) 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 -- 172.21.15.87:0/2450234604 learned_addr learned my addr 172.21.15.87:0/2450234604 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f630ca00640 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563213fcd0a0 0x563213fcd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307400640 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140172f0 0x563213fccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 -- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140172f0 msgr2=0x563213fccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140172f0 0x563213fccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 -- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563213fcd0a0 msgr2=0x563213fcd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563213fcd0a0 0x563213fcd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 -- 172.21.15.87:0/2450234604 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563213fb00d0 con 0x5632140169d0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6307e00640 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140169d0 0x563214016db0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f62f80ab040 tx=0x7f630815ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e58fea9100c2358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f62fcc00640 1 -- 172.21.15.87:0/2450234604 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8004c30 con 0x5632140169d0 2024-04-25T10:20:16.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f62fcc00640 1 -- 172.21.15.87:0/2450234604 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8004dd0 con 0x5632140169d0 2024-04-25T10:20:16.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f62fcc00640 1 -- 172.21.15.87:0/2450234604 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80072d0 con 0x5632140169d0 2024-04-25T10:20:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- 172.21.15.87:0/2450234604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140169d0 msgr2=0x563214016db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140169d0 0x563214016db0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f62f80ab040 tx=0x7f630815ca80 comp rx=0 tx=0).stop 2024-04-25T10:20:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- 172.21.15.87:0/2450234604 shutdown_connections 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563213fcd0a0 0x563213fcd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140172f0 0x563213fccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2450234604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140169d0 0x563214016db0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- 172.21.15.87:0/2450234604 >> 172.21.15.87:0/2450234604 conn(0x563213fc9020 msgr2=0x563213fcb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- 172.21.15.87:0/2450234604 shutdown_connections 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- 172.21.15.87:0/2450234604 wait complete. 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- start start 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 Processor -- start 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563213fcd0a0 0x563213ffb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140169d0 0x563213ffbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140172f0 0x563213ffc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.739+0000 7f6311fa4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563213dde250 con 0x5632140172f0 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x563213dd4690 con 0x5632140169d0 2024-04-25T10:20:16.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x563213da2a40 con 0x563213fcd0a0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 10 client.0 _ll_register_callbacks cb 0x563213fb7e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140172f0 0x563213ffc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140172f0 0x563213ffc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57740/0 (socket says 172.21.15.87:57740) 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 -- 172.21.15.87:0/2977440394 learned_addr learned my addr 172.21.15.87:0/2977440394 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6307400640 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140169d0 0x563213ffbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6307e00640 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563213fcd0a0 0x563213ffb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 -- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563213fcd0a0 msgr2=0x563213ffb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563213fcd0a0 0x563213ffb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 -- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140169d0 msgr2=0x563213ffbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140169d0 0x563213ffbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 -- 172.21.15.87:0/2977440394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563213ec71c0 con 0x5632140172f0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f630ca00640 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140172f0 0x563213ffc270 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6308072800 tx=0x7f6300004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df6fd7f2d1df628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 1 -- 172.21.15.87:0/2977440394 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630001b070 con 0x5632140172f0 2024-04-25T10:20:16.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62f8004dd0 con 0x5632140172f0 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 1 -- 172.21.15.87:0/2977440394 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6300007c70 con 0x5632140172f0 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 1 -- 172.21.15.87:0/2977440394 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6300026050 con 0x5632140172f0 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 10 client.8274 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x563213ec3170 con 0x5632140172f0 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 1 -- 172.21.15.87:0/2977440394 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8004dd0 con 0x5632140172f0 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 1 -- 172.21.15.87:0/2977440394 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6300029020 con 0x5632140172f0 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 10 client.8274 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 10 client.8274 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 20 client.8274 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x563214026c80 0x563214029070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 4 client.8274 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563213db5c60 con 0x563214026c80 2024-04-25T10:20:16.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6307e00640 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x563214026c80 0x563214029070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6307e00640 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x563214026c80 0x563214029070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3507ecc46273b7ff server_cookie=304067e2e7c7e569 in_seq=0 out_seq=0 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 10 client.8274 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 1 -- 172.21.15.87:0/2977440394 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563213db5c60 con 0x563214026c80 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 10 client.8274 handle_command_reply: tid=0 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 client.8274 shutdown 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140172f0 msgr2=0x563213ffc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140172f0 0x563213ffc270 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6308072800 tx=0x7f6300004700 comp rx=0 tx=0).stop 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 shutdown_connections 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x563214026c80 0x563214029070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5632140172f0 0x563213ffc270 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140169d0 0x563213ffbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 --2- 172.21.15.87:0/2977440394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563213fcd0a0 0x563213ffb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f62f1600640 0 client.8274 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 >> 172.21.15.87:0/2977440394 conn(0x563213fc9020 msgr2=0x563213fcbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 shutdown_connections 2024-04-25T10:20:16.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.743+0000 7f6311fa4000 1 -- 172.21.15.87:0/2977440394 wait complete. 2024-04-25T10:20:16.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.747+0000 7f6311fa4000 20 client.8274 trim_cache size 0 max 16384 2024-04-25T10:20:16.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 Processor -- start 2024-04-25T10:20:16.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- start start 2024-04-25T10:20:16.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x5632140730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214073630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x563214073b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63082032a0 con 0x563213fcd0a0 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63082025f0 con 0x5632140172f0 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630814ddb0 con 0x5632140169d0 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x5632140730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x5632140730f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57756/0 (socket says 172.21.15.87:57756) 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 -- 172.21.15.87:0/3583771090 learned_addr learned my addr 172.21.15.87:0/3583771090 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f630ca00640 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x563214073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 -- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 msgr2=0x563214073630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307400640 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214073630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 -- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 msgr2=0x563214073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x563214073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 -- 172.21.15.87:0/3583771090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563213e60de0 con 0x563213fcd0a0 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f630ca00640 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x563214073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307400640 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6307e00640 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x5632140730f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f62f80ab040 tx=0x7f62f8007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedb94d3e8a5486d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f62f3e00640 1 -- 172.21.15.87:0/3583771090 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80cc3a0 con 0x563213fcd0a0 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f62f3e00640 1 -- 172.21.15.87:0/3583771090 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f80d2070 con 0x563213fcd0a0 2024-04-25T10:20:16.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f62f3e00640 1 -- 172.21.15.87:0/3583771090 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80cc740 con 0x563213fcd0a0 2024-04-25T10:20:16.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- 172.21.15.87:0/3583771090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 msgr2=0x5632140730f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x5632140730f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f62f80ab040 tx=0x7f62f8007900 comp rx=0 tx=0).stop 2024-04-25T10:20:16.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- 172.21.15.87:0/3583771090 shutdown_connections 2024-04-25T10:20:16.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x563214073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3583771090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x5632140730f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- 172.21.15.87:0/3583771090 >> 172.21.15.87:0/3583771090 conn(0x563213fc9020 msgr2=0x563214015070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- 172.21.15.87:0/3583771090 shutdown_connections 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- 172.21.15.87:0/3583771090 wait complete. 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.763+0000 7f6311fa4000 1 -- start start 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 Processor -- start 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x563214029770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214029cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x56321402a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563213db4e00 con 0x563213fcd0a0 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630815b390 con 0x5632140172f0 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x563213e19300 con 0x5632140169d0 2024-04-25T10:20:16.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 10 client.0 _ll_register_callbacks cb 0x563213f98040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x563214029770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x563214029770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57768/0 (socket says 172.21.15.87:57768) 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 -- 172.21.15.87:0/3910646926 learned_addr learned my addr 172.21.15.87:0/3910646926 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f630ca00640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x56321402a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307400640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214029cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 -- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 msgr2=0x563214029cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214029cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 -- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 msgr2=0x56321402a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x56321402a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 -- 172.21.15.87:0/3910646926 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563213dbb8e0 con 0x563213fcd0a0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f630ca00640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x56321402a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307e00640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x563214029770 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009010 tx=0x7f62f801b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b3540975d77d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 1 -- 172.21.15.87:0/3910646926 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80dd060 con 0x563213fcd0a0 2024-04-25T10:20:16.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 1 -- 172.21.15.87:0/3910646926 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f80cc660 con 0x563213fcd0a0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62f80d2070 con 0x563213fcd0a0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 1 -- 172.21.15.87:0/3910646926 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62f80e3050 con 0x563213fcd0a0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 10 client.8280 fetch_fsmap learned FSMap version 71 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x563213e738f0 con 0x563213fcd0a0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 1 -- 172.21.15.87:0/3910646926 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f80d2070 con 0x563213fcd0a0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 1 -- 172.21.15.87:0/3910646926 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f62f80e6020 con 0x563213fcd0a0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 10 client.8280 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 10 client.8280 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 20 client.8280 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x56321407cf50 0x56321407f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 4 client.8280 mds_command: new command op to 7947 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 --> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563213e27530 con 0x56321407cf50 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307400640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x56321407cf50 0x56321407f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6307400640 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x56321407cf50 0x56321407f340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0da80b40f08a657 server_cookie=94d6cb67702f26a2 in_seq=0 out_seq=0 2024-04-25T10:20:16.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 10 client.8280 ms_handle_connect on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 1 -- 172.21.15.87:0/3910646926 <== mds.0 v2:172.21.15.87:6834/1167245113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563213e27530 con 0x56321407cf50 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 10 client.8280 handle_command_reply: tid=0 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 client.8280 shutdown 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 msgr2=0x563214029770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x563214029770 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009010 tx=0x7f62f801b550 comp rx=0 tx=0).stop 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 shutdown_connections 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:6834/1167245113,v1:172.21.15.87:6836/1167245113] conn(0x56321407cf50 0x56321407f340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5632140172f0 0x56321402a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5632140169d0 0x563214029cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f62f1600640 0 client.8280 ms_handle_reset on v2:172.21.15.87:6834/1167245113 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 --2- 172.21.15.87:0/3910646926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563213fcd0a0 0x563214029770 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.767+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 >> 172.21.15.87:0/3910646926 conn(0x563213fc9020 msgr2=0x563214015900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 shutdown_connections 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f6311fa4000 1 -- 172.21.15.87:0/3910646926 wait complete. 2024-04-25T10:20:16.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f6311fa4000 20 client.8280 trim_cache size 0 max 16384 2024-04-25T10:20:16.777 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:20:16.777 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:20:16.777 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:20:16.777 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:20:16.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62e8066f90 msgr2=0x7f62e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62e8066f90 0x7f62e8069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f62f80aa000 tx=0x7f62f8001a50 comp rx=0 tx=0).stop 2024-04-25T10:20:16.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630814fb30 msgr2=0x7f630815a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630814fb30 0x7f630815a0e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f62f4011940 tx=0x7f62f4011e00 comp rx=0 tx=0).stop 2024-04-25T10:20:16.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 shutdown_connections 2024-04-25T10:20:16.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62e8066f90 0x7f62e8069450 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630814a9a0 0x7f630814ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630814fb30 0x7f630815a0e0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 --2- 172.21.15.87:0/2699178936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62ec0039c0 0x7f63080b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 >> 172.21.15.87:0/2699178936 conn(0x7f63080669a0 msgr2=0x7f63080cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 shutdown_connections 2024-04-25T10:20:16.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.771+0000 7f630ea00640 1 -- 172.21.15.87:0/2699178936 wait complete. 2024-04-25T10:20:16.791 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:16.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.923+0000 7fc3cd600640 1 Processor -- start 2024-04-25T10:20:16.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.923+0000 7fc3cd600640 1 -- start start 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3cd600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3c80b0f70 0x7fc3c80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3cd600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3c80cfd90 0x7fc3c80ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3cd600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c815f3b0 con 0x7fc3c80cd110 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3cd600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c815ef20 con 0x7fc3c80cfd90 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3cd600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c80ca810 con 0x7fc3c80b0f70 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3c80cfd90 0x7fc3c80ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3c80cfd90 0x7fc3c80ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57782/0 (socket says 172.21.15.87:57782) 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6000640 1 -- 172.21.15.87:0/3898365449 learned_addr learned my addr 172.21.15.87:0/3898365449 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c7400640 1 --1- 172.21.15.87:0/3898365449 >> v1:172.21.15.87:6789/0 conn(0x7fc3c80cd110 0x7fc3c80cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53088/0 (socket says 172.21.15.87:53088) 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6a00640 1 --2- 172.21.15.87:0/3898365449 >> v2:172.21.15.169:3300/0 conn(0x7fc3c80b0f70 0x7fc3c80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6000640 1 -- 172.21.15.87:0/3898365449 >> v2:172.21.15.169:3300/0 conn(0x7fc3c80b0f70 msgr2=0x7fc3c80b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6000640 1 --2- 172.21.15.87:0/3898365449 >> v2:172.21.15.169:3300/0 conn(0x7fc3c80b0f70 0x7fc3c80b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6000640 1 -- 172.21.15.87:0/3898365449 >> v1:172.21.15.87:6789/0 conn(0x7fc3c80cd110 legacy=0x7fc3c80cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c6000640 1 -- 172.21.15.87:0/3898365449 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80b1890 con 0x7fc3c80cfd90 2024-04-25T10:20:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.927+0000 7fc3c5600640 1 -- 172.21.15.87:0/3898365449 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179143376 0 0) 0x7fc3c815f3b0 con 0x7fc3c80cd110 2024-04-25T10:20:16.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3c6000640 1 --2- 172.21.15.87:0/3898365449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cfd90 0x7fc3c80ccbd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8071230 tx=0x7fc3b401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca64085cb1352054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3c5600640 1 -- 172.21.15.87:0/3898365449 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40b0070 con 0x7fc3c80cfd90 2024-04-25T10:20:16.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3c5600640 1 -- 172.21.15.87:0/3898365449 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b40ab600 con 0x7fc3c80cfd90 2024-04-25T10:20:16.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3c5600640 1 -- 172.21.15.87:0/3898365449 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40ab9d0 con 0x7fc3c80cfd90 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- 172.21.15.87:0/3898365449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cfd90 msgr2=0x7fc3c80ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 --2- 172.21.15.87:0/3898365449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cfd90 0x7fc3c80ccbd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8071230 tx=0x7fc3b401c520 comp rx=0 tx=0).stop 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- 172.21.15.87:0/3898365449 shutdown_connections 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 --2- 172.21.15.87:0/3898365449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cfd90 0x7fc3c80ccbd0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 --2- 172.21.15.87:0/3898365449 >> v2:172.21.15.169:3300/0 conn(0x7fc3c80b0f70 0x7fc3c80b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- 172.21.15.87:0/3898365449 >> 172.21.15.87:0/3898365449 conn(0x7fc3c80b1d50 msgr2=0x7fc3c80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- 172.21.15.87:0/3898365449 shutdown_connections 2024-04-25T10:20:16.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- 172.21.15.87:0/3898365449 wait complete. 2024-04-25T10:20:16.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 Processor -- start 2024-04-25T10:20:16.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- start start 2024-04-25T10:20:16.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c80b0f70 0x7fc3c8156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cd110 0x7fc3c80c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cfd90 0x7fc3c80c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c81619d0 con 0x7fc3c80cd110 2024-04-25T10:20:16.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80cd520 con 0x7fc3c80b0f70 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3cd600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c8161b50 con 0x7fc3c80cfd90 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.931+0000 7fc3c6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cd110 0x7fc3c80c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cd110 0x7fc3c80c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57784/0 (socket says 172.21.15.87:57784) 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 -- 172.21.15.87:0/221639820 learned_addr learned my addr 172.21.15.87:0/221639820 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c7400640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cfd90 0x7fc3c80c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6a00640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c80b0f70 0x7fc3c8156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 -- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cfd90 msgr2=0x7fc3c80c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cfd90 0x7fc3c80c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 -- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c80b0f70 msgr2=0x7fc3c8156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c80b0f70 0x7fc3c8156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 -- 172.21.15.87:0/221639820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c81565c0 con 0x7fc3c80cd110 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c7400640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cfd90 0x7fc3c80c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3c6000640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cd110 0x7fc3c80c5df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c10 tx=0x7fc3b40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3bea00640 1 -- 172.21.15.87:0/221639820 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40b0040 con 0x7fc3c80cd110 2024-04-25T10:20:16.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3bea00640 1 -- 172.21.15.87:0/221639820 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b4004020 con 0x7fc3c80cd110 2024-04-25T10:20:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c80b57e0 con 0x7fc3c80cd110 2024-04-25T10:20:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3bea00640 1 -- 172.21.15.87:0/221639820 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40043d0 con 0x7fc3c80cd110 2024-04-25T10:20:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c80adec0 con 0x7fc3c80cd110 2024-04-25T10:20:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c80ad9f0 con 0x7fc3c80cd110 2024-04-25T10:20:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.935+0000 7fc3bea00640 1 -- 172.21.15.87:0/221639820 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3b40be060 con 0x7fc3c80cd110 2024-04-25T10:20:16.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.939+0000 7fc3bea00640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3a8066f60 0x7fc3a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.939+0000 7fc3bea00640 1 -- 172.21.15.87:0/221639820 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3b411e670 con 0x7fc3c80cd110 2024-04-25T10:20:16.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.939+0000 7fc3c6a00640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3a8066f60 0x7fc3a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.939+0000 7fc3c6a00640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3a8066f60 0x7fc3a8069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800b9d0 tx=0x7fc3b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:16.943+0000 7fc3bea00640 1 -- 172.21.15.87:0/221639820 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3b4016110 con 0x7fc3c80cd110 2024-04-25T10:20:17.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3c804ef30 con 0x7fc3c80cd110 2024-04-25T10:20:17.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3bea00640 1 -- 172.21.15.87:0/221639820 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc3b40b3350 con 0x7fc3c80cd110 2024-04-25T10:20:17.137 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:17.137 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:17.138 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3a8066f60 msgr2=0x7fc3a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3a8066f60 0x7fc3a8069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800b9d0 tx=0x7fc3b8008040 comp rx=0 tx=0).stop 2024-04-25T10:20:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cd110 msgr2=0x7fc3c80c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cd110 0x7fc3c80c5df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c10 tx=0x7fc3b40192c0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 shutdown_connections 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3a8066f60 0x7fc3a8069420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cfd90 0x7fc3c80c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3c80cd110 0x7fc3c80c5df0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 --2- 172.21.15.87:0/221639820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c80b0f70 0x7fc3c8156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 >> 172.21.15.87:0/221639820 conn(0x7fc3c80b1d50 msgr2=0x7fc3c80caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 shutdown_connections 2024-04-25T10:20:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.135+0000 7fc3cd600640 1 -- 172.21.15.87:0/221639820 wait complete. 2024-04-25T10:20:17.158 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:20:17.158 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:17.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.291+0000 7fbedd200640 1 Processor -- start 2024-04-25T10:20:17.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.291+0000 7fbedd200640 1 -- start start 2024-04-25T10:20:17.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbedd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 0x7fbed80709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbedd200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed8162470 con 0x7fbed815e060 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbedd200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed8071770 con 0x7fbed815ee20 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbedd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b3610 con 0x7fbed815ea40 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbed815e060 0x7fbed8161d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53094/0 (socket says 172.21.15.87:53094) 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7e00640 1 -- 172.21.15.87:0/336223317 learned_addr learned my addr 172.21.15.87:0/336223317 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7400640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 0x7fbed80709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175419902 0 0) 0x7fbed8162470 con 0x7fbed815e060 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbebc003530 con 0x7fbed815e060 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309952082 0 0) 0x7fbed8071770 con 0x7fbed815ee20 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed8162470 con 0x7fbed815ee20 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260781192 0 0) 0x7fbebc003530 con 0x7fbed815e060 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 msgr2=0x7fbed80709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 0x7fbed80709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 >> v1:172.21.15.169:6789/0 conn(0x7fbed815ee20 legacy=0x7fbed8161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed80b3860 con 0x7fbed815e060 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fbed0001a00 con 0x7fbed815e060 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 >> v1:172.21.15.87:6789/0 conn(0x7fbed815e060 legacy=0x7fbed8161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbebc003bf0 0x7fbebc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbebc004810 0x7fbebc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 0x7fbebc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b3610 con 0x7fbebc003bf0 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc00bb90 con 0x7fbebc004810 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc00bd90 con 0x7fbed815ea40 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7400640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbebc003bf0 0x7fbebc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbecea00640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbebc004810 0x7fbebc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7e00640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 0x7fbebc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7400640 1 -- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 msgr2=0x7fbebc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:17.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7400640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed815ea40 0x7fbebc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7400640 1 -- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbebc004810 msgr2=0x7fbebc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:17.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7400640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbebc004810 0x7fbebc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbed7400640 1 -- 172.21.15.87:0/336223317 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc003530 con 0x7fbebc003bf0 2024-04-25T10:20:17.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.295+0000 7fbecea00640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbebc004810 0x7fbebc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbed7400640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbebc003bf0 0x7fbebc004010 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009ae0 tx=0x7fbec801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a8f04a7fcdfb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80096b0 con 0x7fbebc003bf0 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec80a6450 con 0x7fbebc003bf0 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbed6a00640 1 -- 172.21.15.87:0/336223317 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80a6840 con 0x7fbebc003bf0 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 -- 172.21.15.87:0/336223317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbebc003bf0 msgr2=0x7fbebc004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 --2- 172.21.15.87:0/336223317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbebc003bf0 0x7fbebc004010 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009ae0 tx=0x7fbec801ca70 comp rx=0 tx=0).stop 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbed7e00640 1 -- 172.21.15.87:0/336223317 reap_dead start 2024-04-25T10:20:17.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 -- 172.21.15.87:0/336223317 shutdown_connections 2024-04-25T10:20:17.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 -- 172.21.15.87:0/336223317 >> 172.21.15.87:0/336223317 conn(0x7fbed806b6d0 msgr2=0x7fbed80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:17.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 -- 172.21.15.87:0/336223317 shutdown_connections 2024-04-25T10:20:17.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 -- 172.21.15.87:0/336223317 wait complete. 2024-04-25T10:20:17.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 Processor -- start 2024-04-25T10:20:17.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.299+0000 7fbedd200640 1 -- start start 2024-04-25T10:20:17.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbebc003bf0 0x7fbed8143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbed8160250 0x7fbed81606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed8144d40 0x7fbed81451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b3300 con 0x7fbed8160250 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b3180 con 0x7fbed8144d40 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b50d0 con 0x7fbebc003bf0 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbed8160250 0x7fbed81606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbed8160250 0x7fbed81606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57794/0 (socket says 172.21.15.87:57794) 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 -- 172.21.15.87:0/945280106 learned_addr learned my addr 172.21.15.87:0/945280106 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbed7400640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbebc003bf0 0x7fbed8143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 -- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbebc003bf0 msgr2=0x7fbed8143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbed7e00640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed8144d40 0x7fbed81451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbebc003bf0 0x7fbed8143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 -- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed8144d40 msgr2=0x7fbed81451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed8144d40 0x7fbed81451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 -- 172.21.15.87:0/945280106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8145af0 con 0x7fbed8160250 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbed7400640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbebc003bf0 0x7fbed8143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbed7e00640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed8144d40 0x7fbed81451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecea00640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbed8160250 0x7fbed81606b0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fbec000e810 tx=0x7fbec000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecfe00640 1 -- 172.21.15.87:0/945280106 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00050d0 con 0x7fbed8160250 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecfe00640 1 -- 172.21.15.87:0/945280106 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0005270 con 0x7fbed8160250 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed81472a0 con 0x7fbed8160250 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbecfe00640 1 -- 172.21.15.87:0/945280106 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0005460 con 0x7fbed8160250 2024-04-25T10:20:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed8147860 con 0x7fbed8160250 2024-04-25T10:20:17.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.303+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed8160bf0 con 0x7fbed8160250 2024-04-25T10:20:17.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.307+0000 7fbecfe00640 1 -- 172.21.15.87:0/945280106 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbec001a030 con 0x7fbed8160250 2024-04-25T10:20:17.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.307+0000 7fbecfe00640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbeb4066f90 0x7fbeb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.307+0000 7fbecfe00640 1 -- 172.21.15.87:0/945280106 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbec00845e0 con 0x7fbed8160250 2024-04-25T10:20:17.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.307+0000 7fbed7400640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbeb4066f90 0x7fbeb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.307+0000 7fbed7400640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbeb4066f90 0x7fbeb4069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbec80ad000 tx=0x7fbec80046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.311+0000 7fbecfe00640 1 -- 172.21.15.87:0/945280106 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbec0021020 con 0x7fbed8160250 2024-04-25T10:20:17.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbed8075120 con 0x7fbed8160250 2024-04-25T10:20:17.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbecfe00640 1 -- 172.21.15.87:0/945280106 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7fbec00555e0 con 0x7fbed8160250 2024-04-25T10:20:17.521 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:17.521 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076434023,"bytes_used":2593071104,"bytes_avail":770501042176,"bytes_total":773094113280,"write_bytes_sec":37226941,"read_op_per_sec":0,"write_op_per_sec":37},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:20:15.225075+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:17.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbeb4066f90 msgr2=0x7fbeb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbeb4066f90 0x7fbeb4069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbec80ad000 tx=0x7fbec80046d0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbed8160250 msgr2=0x7fbed81606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbed8160250 0x7fbed81606b0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fbec000e810 tx=0x7fbec000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 shutdown_connections 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbeb4066f90 0x7fbeb4069450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed8144d40 0x7fbed81451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbed8160250 0x7fbed81606b0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 --2- 172.21.15.87:0/945280106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbebc003bf0 0x7fbed8143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 >> 172.21.15.87:0/945280106 conn(0x7fbed806b6d0 msgr2=0x7fbed8154960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 shutdown_connections 2024-04-25T10:20:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:17.519+0000 7fbedd200640 1 -- 172.21.15.87:0/945280106 wait complete. 2024-04-25T10:20:17.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:20:18.544 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:18.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.675+0000 7f5da1400640 1 Processor -- start 2024-04-25T10:20:18.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.675+0000 7f5da1400640 1 -- start start 2024-04-25T10:20:18.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5da1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d9c0cda10 0x7f5d9c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5da1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cf930 0x7f5d9c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5da1400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0b2770 con 0x7f5d9c15bc40 2024-04-25T10:20:18.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5da1400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c06c670 con 0x7f5d9c0cda10 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5da1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06c950 con 0x7f5d9c0cf930 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d91e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d9c0cda10 0x7f5d9c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d9a800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5d9c15bc40 0x7f5d9c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60298/0 (socket says 172.21.15.87:60298) 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d9a800640 1 -- 172.21.15.87:0/2407748788 learned_addr learned my addr 172.21.15.87:0/2407748788 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d9b200640 1 --2- 172.21.15.87:0/2407748788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cf930 0x7f5d9c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d91e00640 1 -- 172.21.15.87:0/2407748788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cf930 msgr2=0x7f5d9c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d91e00640 1 --2- 172.21.15.87:0/2407748788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cf930 0x7f5d9c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d91e00640 1 -- 172.21.15.87:0/2407748788 >> v1:172.21.15.87:6789/0 conn(0x7f5d9c15bc40 legacy=0x7f5d9c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d91e00640 1 -- 172.21.15.87:0/2407748788 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c06cc30 con 0x7f5d9c0cda10 2024-04-25T10:20:18.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.679+0000 7f5d99e00640 1 -- 172.21.15.87:0/2407748788 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318987551 0 0) 0x7f5d9c0b2770 con 0x7f5d9c15bc40 2024-04-25T10:20:18.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5d91e00640 1 --2- 172.21.15.87:0/2407748788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c0cda10 0x7f5d9c06c090 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5d840097c0 tx=0x7f5d8401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ba6b6a98ddf5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5d99e00640 1 -- 172.21.15.87:0/2407748788 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840b0070 con 0x7f5d9c0cda10 2024-04-25T10:20:18.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5d99e00640 1 -- 172.21.15.87:0/2407748788 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840ab600 con 0x7f5d9c0cda10 2024-04-25T10:20:18.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5d99e00640 1 -- 172.21.15.87:0/2407748788 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840ab9d0 con 0x7f5d9c0cda10 2024-04-25T10:20:18.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- 172.21.15.87:0/2407748788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c0cda10 msgr2=0x7f5d9c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 --2- 172.21.15.87:0/2407748788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c0cda10 0x7f5d9c06c090 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5d840097c0 tx=0x7f5d8401c410 comp rx=0 tx=0).stop 2024-04-25T10:20:18.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- 172.21.15.87:0/2407748788 shutdown_connections 2024-04-25T10:20:18.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 --2- 172.21.15.87:0/2407748788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cf930 0x7f5d9c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 --2- 172.21.15.87:0/2407748788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c0cda10 0x7f5d9c06c090 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- 172.21.15.87:0/2407748788 >> 172.21.15.87:0/2407748788 conn(0x7f5d9c066b70 msgr2=0x7f5d9c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:18.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- 172.21.15.87:0/2407748788 shutdown_connections 2024-04-25T10:20:18.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- 172.21.15.87:0/2407748788 wait complete. 2024-04-25T10:20:18.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 Processor -- start 2024-04-25T10:20:18.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- start start 2024-04-25T10:20:18.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cda10 0x7f5d9c1475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d9c0cf930 0x7f5d9c0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c15bc40 0x7f5d9c165630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b3890 con 0x7f5d9c15bc40 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b3710 con 0x7f5d9c0cf930 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5da1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b3a10 con 0x7f5d9c0cda10 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5d9b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c15bc40 0x7f5d9c165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5d9b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c15bc40 0x7f5d9c165630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34246/0 (socket says 172.21.15.87:34246) 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.683+0000 7f5d9b200640 1 -- 172.21.15.87:0/2454501328 learned_addr learned my addr 172.21.15.87:0/2454501328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9a800640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cda10 0x7f5d9c1475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d91e00640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d9c0cf930 0x7f5d9c0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9b200640 1 -- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cda10 msgr2=0x7f5d9c1475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9b200640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cda10 0x7f5d9c1475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9b200640 1 -- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d9c0cf930 msgr2=0x7f5d9c0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9b200640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d9c0cf930 0x7f5d9c0b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9b200640 1 -- 172.21.15.87:0/2454501328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c15f8e0 con 0x7f5d9c15bc40 2024-04-25T10:20:18.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9b200640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c15bc40 0x7f5d9c165630 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400d560 tx=0x7f5d9400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d93400640 1 -- 172.21.15.87:0/2454501328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d940125f0 con 0x7f5d9c15bc40 2024-04-25T10:20:18.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d93400640 1 -- 172.21.15.87:0/2454501328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94019070 con 0x7f5d9c15bc40 2024-04-25T10:20:18.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c146510 con 0x7f5d9c15bc40 2024-04-25T10:20:18.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d93400640 1 -- 172.21.15.87:0/2454501328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d940129f0 con 0x7f5d9c15bc40 2024-04-25T10:20:18.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c146ad0 con 0x7f5d9c15bc40 2024-04-25T10:20:18.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c04ef30 con 0x7f5d9c15bc40 2024-04-25T10:20:18.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d93400640 1 -- 172.21.15.87:0/2454501328 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d94012b90 con 0x7f5d9c15bc40 2024-04-25T10:20:18.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d93400640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d78066f60 0x7f5d78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d93400640 1 -- 172.21.15.87:0/2454501328 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d940854c0 con 0x7f5d9c15bc40 2024-04-25T10:20:18.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.687+0000 7f5d9a800640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d78066f60 0x7f5d78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.691+0000 7f5d9a800640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d78066f60 0x7f5d78069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005fd0 tx=0x7f5d8c005b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.695+0000 7f5d93400640 1 -- 172.21.15.87:0/2454501328 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d94056430 con 0x7f5d9c15bc40 2024-04-25T10:20:18.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.879+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d9c15f400 con 0x7f5d9c15bc40 2024-04-25T10:20:18.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.879+0000 7f5d93400640 1 -- 172.21.15.87:0/2454501328 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5d94055d30 con 0x7f5d9c15bc40 2024-04-25T10:20:18.883 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:18.883 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:18.883 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d78066f60 msgr2=0x7f5d78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d78066f60 0x7f5d78069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005fd0 tx=0x7f5d8c005b80 comp rx=0 tx=0).stop 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c15bc40 msgr2=0x7f5d9c165630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c15bc40 0x7f5d9c165630 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400d560 tx=0x7f5d9400da20 comp rx=0 tx=0).stop 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 shutdown_connections 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d78066f60 0x7f5d78069420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d9c15bc40 0x7f5d9c165630 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d9c0cf930 0x7f5d9c0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 --2- 172.21.15.87:0/2454501328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d9c0cda10 0x7f5d9c1475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 >> 172.21.15.87:0/2454501328 conn(0x7f5d9c066b70 msgr2=0x7f5d9c15a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:18.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 shutdown_connections 2024-04-25T10:20:18.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:18.883+0000 7f5da1400640 1 -- 172.21.15.87:0/2454501328 wait complete. 2024-04-25T10:20:18.903 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:20:18.904 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:19.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8e5a00640 1 Processor -- start 2024-04-25T10:20:19.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8e5a00640 1 -- start start 2024-04-25T10:20:19.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8e5a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd8e006eec0 0x7fd8e006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8e5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e00ad850 0x7fd8e00adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8e5a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e015e0c0 con 0x7fd8e015dce0 2024-04-25T10:20:19.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8e5a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00674d0 con 0x7fd8e006eec0 2024-04-25T10:20:19.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8e5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00671f0 con 0x7fd8e00ad850 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8df400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e00ad850 0x7fd8e00adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8dea00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd8e015dce0 0x7fd8e00669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:59284/0 (socket says 172.21.15.87:59284) 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8dea00640 1 -- 172.21.15.87:0/1024721521 learned_addr learned my addr 172.21.15.87:0/1024721521 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8d6000640 1 --2- 172.21.15.87:0/1024721521 >> v2:172.21.15.169:3300/0 conn(0x7fd8e006eec0 0x7fd8e006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8df400640 1 -- 172.21.15.87:0/1024721521 >> v2:172.21.15.169:3300/0 conn(0x7fd8e006eec0 msgr2=0x7fd8e006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8df400640 1 --2- 172.21.15.87:0/1024721521 >> v2:172.21.15.169:3300/0 conn(0x7fd8e006eec0 0x7fd8e006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8df400640 1 -- 172.21.15.87:0/1024721521 >> v1:172.21.15.169:6789/0 conn(0x7fd8e015dce0 legacy=0x7fd8e00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.035+0000 7fd8df400640 1 -- 172.21.15.87:0/1024721521 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e006d4b0 con 0x7fd8e00ad850 2024-04-25T10:20:19.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8df400640 1 --2- 172.21.15.87:0/1024721521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e00ad850 0x7fd8e00adcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009d90 tx=0x7fd8d801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33836ff326486d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:19.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8de000640 1 -- 172.21.15.87:0/1024721521 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80b2070 con 0x7fd8e00ad850 2024-04-25T10:20:19.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8de000640 1 -- 172.21.15.87:0/1024721521 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d80a57e0 con 0x7fd8e00ad850 2024-04-25T10:20:19.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8de000640 1 -- 172.21.15.87:0/1024721521 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80a5bb0 con 0x7fd8e00ad850 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1024721521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e00ad850 msgr2=0x7fd8e00adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1024721521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e00ad850 0x7fd8e00adcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009d90 tx=0x7fd8d801c850 comp rx=0 tx=0).stop 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1024721521 shutdown_connections 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1024721521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e00ad850 0x7fd8e00adcb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1024721521 >> v2:172.21.15.169:3300/0 conn(0x7fd8e006eec0 0x7fd8e006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1024721521 >> 172.21.15.87:0/1024721521 conn(0x7fd8e00b2410 msgr2=0x7fd8e00b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1024721521 shutdown_connections 2024-04-25T10:20:19.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.039+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1024721521 wait complete. 2024-04-25T10:20:19.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 Processor -- start 2024-04-25T10:20:19.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 -- start start 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8e006eec0 0x7fd8e0155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8e00ad850 0x7fd8e00c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e015dce0 0x7fd8e01f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00d0320 con 0x7fd8e00ad850 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00d04a0 con 0x7fd8e006eec0 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e016abc0 con 0x7fd8e015dce0 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8e00ad850 0x7fd8e00c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8e00ad850 0x7fd8e00c3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34248/0 (socket says 172.21.15.87:34248) 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 -- 172.21.15.87:0/1554200419 learned_addr learned my addr 172.21.15.87:0/1554200419 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8dea00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8e006eec0 0x7fd8e0155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8df400640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e015dce0 0x7fd8e01f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 -- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e015dce0 msgr2=0x7fd8e01f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e015dce0 0x7fd8e01f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 -- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8e006eec0 msgr2=0x7fd8e0155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8e006eec0 0x7fd8e0155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 -- 172.21.15.87:0/1554200419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e01556d0 con 0x7fd8e00ad850 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d6000640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8e00ad850 0x7fd8e00c3f10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800e8a0 tx=0x7fd8c800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d7400640 1 -- 172.21.15.87:0/1554200419 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8017070 con 0x7fd8e00ad850 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d7400640 1 -- 172.21.15.87:0/1554200419 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800c940 con 0x7fd8e00ad850 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8d7400640 1 -- 172.21.15.87:0/1554200419 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800ccf0 con 0x7fd8e00ad850 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e00c4760 con 0x7fd8e00ad850 2024-04-25T10:20:19.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.043+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e01f7420 con 0x7fd8e00ad850 2024-04-25T10:20:19.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.047+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e004ef30 con 0x7fd8e00ad850 2024-04-25T10:20:19.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.047+0000 7fd8d7400640 1 -- 172.21.15.87:0/1554200419 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8c801a070 con 0x7fd8e00ad850 2024-04-25T10:20:19.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.047+0000 7fd8d7400640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8bc066f60 0x7fd8bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.047+0000 7fd8d7400640 1 -- 172.21.15.87:0/1554200419 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8c8084f20 con 0x7fd8e00ad850 2024-04-25T10:20:19.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.047+0000 7fd8dea00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8bc066f60 0x7fd8bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.047+0000 7fd8dea00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8bc066f60 0x7fd8bc069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0000f70 tx=0x7fd8d0007e90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:19.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.051+0000 7fd8d7400640 1 -- 172.21.15.87:0/1554200419 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8c8055840 con 0x7fd8e00ad850 2024-04-25T10:20:19.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd8e00cdb40 con 0x7fd8e00ad850 2024-04-25T10:20:19.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8d7400640 1 -- 172.21.15.87:0/1554200419 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7fd8c80598e0 con 0x7fd8e00ad850 2024-04-25T10:20:19.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:19.266 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":501,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076635813,"bytes_used":2593480704,"bytes_avail":770500632576,"bytes_total":773094113280,"write_bytes_sec":37245006,"read_op_per_sec":0,"write_op_per_sec":37},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:20:15.225075+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8bc066f60 msgr2=0x7fd8bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8bc066f60 0x7fd8bc069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0000f70 tx=0x7fd8d0007e90 comp rx=0 tx=0).stop 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8e00ad850 msgr2=0x7fd8e00c3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8e00ad850 0x7fd8e00c3f10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800e8a0 tx=0x7fd8c800eca0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 shutdown_connections 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd8bc066f60 0x7fd8bc069420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8e015dce0 0x7fd8e01f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8e00ad850 0x7fd8e00c3f10 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 --2- 172.21.15.87:0/1554200419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8e006eec0 0x7fd8e0155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 >> 172.21.15.87:0/1554200419 conn(0x7fd8e00b2410 msgr2=0x7fd8e00b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 shutdown_connections 2024-04-25T10:20:19.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:19.263+0000 7fd8e5a00640 1 -- 172.21.15.87:0/1554200419 wait complete. 2024-04-25T10:20:19.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:20:20.286 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:20.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e14e00640 1 Processor -- start 2024-04-25T10:20:20.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e14e00640 1 -- start start 2024-04-25T10:20:20.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e14e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e14e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e10073d10 0x7f7e1006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e14e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e10070be0 con 0x7f7e10161900 2024-04-25T10:20:20.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e14e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1015e270 con 0x7f7e10073d10 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e14e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e100714d0 con 0x7f7e1006e0a0 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e10073d10 0x7f7e1006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e10073d10 0x7f7e1006e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34250/0 (socket says 172.21.15.87:34250) 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0d400640 1 -- 172.21.15.87:0/2978376222 learned_addr learned my addr 172.21.15.87:0/2978376222 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0de00640 1 --2- 172.21.15.87:0/2978376222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0d400640 1 -- 172.21.15.87:0/2978376222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 msgr2=0x7f7e100b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0d400640 1 --2- 172.21.15.87:0/2978376222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e100b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0d400640 1 -- 172.21.15.87:0/2978376222 >> v1:172.21.15.169:6789/0 conn(0x7f7e10161900 legacy=0x7f7e10161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.431+0000 7f7e0d400640 1 -- 172.21.15.87:0/2978376222 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1006e9c0 con 0x7f7e10073d10 2024-04-25T10:20:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e0d400640 1 --2- 172.21.15.87:0/2978376222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10073d10 0x7f7e1006e480 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009870 tx=0x7f7dfc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8aedb383cbcc6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e0ca00640 1 -- 172.21.15.87:0/2978376222 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0b1070 con 0x7f7e10073d10 2024-04-25T10:20:20.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e0ca00640 1 -- 172.21.15.87:0/2978376222 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc0ac630 con 0x7f7e10073d10 2024-04-25T10:20:20.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e0ca00640 1 -- 172.21.15.87:0/2978376222 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0a8050 con 0x7f7e10073d10 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 -- 172.21.15.87:0/2978376222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10073d10 msgr2=0x7f7e1006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 --2- 172.21.15.87:0/2978376222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10073d10 0x7f7e1006e480 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009870 tx=0x7f7dfc01c520 comp rx=0 tx=0).stop 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 -- 172.21.15.87:0/2978376222 shutdown_connections 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 --2- 172.21.15.87:0/2978376222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10073d10 0x7f7e1006e480 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 --2- 172.21.15.87:0/2978376222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e100b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 -- 172.21.15.87:0/2978376222 >> 172.21.15.87:0/2978376222 conn(0x7f7e100d00a0 msgr2=0x7f7e100d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 -- 172.21.15.87:0/2978376222 shutdown_connections 2024-04-25T10:20:20.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.435+0000 7f7e14e00640 1 -- 172.21.15.87:0/2978376222 wait complete. 2024-04-25T10:20:20.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 Processor -- start 2024-04-25T10:20:20.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 -- start start 2024-04-25T10:20:20.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e1014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e10073d10 0x7f7e1014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10161900 0x7f7e1014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1006f600 con 0x7f7e10161900 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1006f480 con 0x7f7e10073d10 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e100ad840 con 0x7f7e1006e0a0 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10161900 0x7f7e1014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10161900 0x7f7e1014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34254/0 (socket says 172.21.15.87:34254) 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 -- 172.21.15.87:0/1712603686 learned_addr learned my addr 172.21.15.87:0/1712603686 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0de00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e1014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0d400640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e10073d10 0x7f7e1014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 -- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 msgr2=0x7f7e1014f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e1014f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 -- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e10073d10 msgr2=0x7f7e1014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e10073d10 0x7f7e1014c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 -- 172.21.15.87:0/1712603686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1015cb30 con 0x7f7e10161900 2024-04-25T10:20:20.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e0e800640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10161900 0x7f7e1014fea0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c780 tx=0x7f7e0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e06000640 1 -- 172.21.15.87:0/1712603686 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08004b70 con 0x7f7e10161900 2024-04-25T10:20:20.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e06000640 1 -- 172.21.15.87:0/1712603686 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e08004d10 con 0x7f7e10161900 2024-04-25T10:20:20.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1015ce10 con 0x7f7e10161900 2024-04-25T10:20:20.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e06000640 1 -- 172.21.15.87:0/1712603686 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800aa90 con 0x7f7e10161900 2024-04-25T10:20:20.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.439+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1014cbc0 con 0x7f7e10161900 2024-04-25T10:20:20.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.443+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1014fcd0 con 0x7f7e10161900 2024-04-25T10:20:20.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.443+0000 7f7e06000640 1 -- 172.21.15.87:0/1712603686 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e0800ac30 con 0x7f7e10161900 2024-04-25T10:20:20.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.443+0000 7f7e06000640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7df0066f60 0x7f7df0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.443+0000 7f7e06000640 1 -- 172.21.15.87:0/1712603686 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e08085220 con 0x7f7e10161900 2024-04-25T10:20:20.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.443+0000 7f7e0de00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7df0066f60 0x7f7df0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.443+0000 7f7e0de00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7df0066f60 0x7f7df0069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7e100b39f0 tx=0x7f7e00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.447+0000 7f7e06000640 1 -- 172.21.15.87:0/1712603686 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e08020070 con 0x7f7e10161900 2024-04-25T10:20:20.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e1004ef30 con 0x7f7e10161900 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e06000640 1 -- 172.21.15.87:0/1712603686 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7e08055b40 con 0x7f7e10161900 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:20.639 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:20.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7df0066f60 msgr2=0x7f7df0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7df0066f60 0x7f7df0069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7e100b39f0 tx=0x7f7e00008040 comp rx=0 tx=0).stop 2024-04-25T10:20:20.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10161900 msgr2=0x7f7e1014fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10161900 0x7f7e1014fea0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c780 tx=0x7f7e0800cc40 comp rx=0 tx=0).stop 2024-04-25T10:20:20.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 shutdown_connections 2024-04-25T10:20:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7df0066f60 0x7f7df0069420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e10161900 0x7f7e1014fea0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e10073d10 0x7f7e1014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 --2- 172.21.15.87:0/1712603686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e1006e0a0 0x7f7e1014f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.635+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 >> 172.21.15.87:0/1712603686 conn(0x7f7e100d00a0 msgr2=0x7f7e100af310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.639+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 shutdown_connections 2024-04-25T10:20:20.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.639+0000 7f7e14e00640 1 -- 172.21.15.87:0/1712603686 wait complete. 2024-04-25T10:20:20.659 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:20:20.659 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:20.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6687c00640 1 Processor -- start 2024-04-25T10:20:20.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6687c00640 1 -- start start 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6687c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66800ae020 0x7f66800ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6687c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66800ad840 con 0x7f66800cf930 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6687c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668015e9f0 con 0x7f6680070c20 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6687c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668015bb90 con 0x7f66800ae020 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6685200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f66800cf930 0x7f66800cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60310/0 (socket says 172.21.15.87:60310) 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6685200640 1 -- 172.21.15.87:0/4050612186 learned_addr learned my addr 172.21.15.87:0/4050612186 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6685c00640 1 --2- 172.21.15.87:0/4050612186 >> v2:172.21.15.87:3300/0 conn(0x7f66800ae020 0x7f66800ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f667fe00640 1 -- 172.21.15.87:0/4050612186 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437551385 0 0) 0x7f66800ad840 con 0x7f66800cf930 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f667fe00640 1 -- 172.21.15.87:0/4050612186 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6664003530 con 0x7f66800cf930 2024-04-25T10:20:20.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6685c00640 1 -- 172.21.15.87:0/4050612186 >> v1:172.21.15.169:6789/0 conn(0x7f6680070c20 legacy=0x7f668015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6685c00640 1 -- 172.21.15.87:0/4050612186 >> v1:172.21.15.87:6789/0 conn(0x7f66800cf930 legacy=0x7f66800cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.791+0000 7f6685c00640 1 -- 172.21.15.87:0/4050612186 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668015ecd0 con 0x7f66800ae020 2024-04-25T10:20:20.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6685c00640 1 --2- 172.21.15.87:0/4050612186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66800ae020 0x7f66800ae4b0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6678009d90 tx=0x7f667801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679a3e34e06c8b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f667fe00640 1 -- 172.21.15.87:0/4050612186 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66780b2070 con 0x7f66800ae020 2024-04-25T10:20:20.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f667fe00640 1 -- 172.21.15.87:0/4050612186 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66780a59c0 con 0x7f66800ae020 2024-04-25T10:20:20.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f667fe00640 1 -- 172.21.15.87:0/4050612186 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66780a5d90 con 0x7f66800ae020 2024-04-25T10:20:20.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6687c00640 1 -- 172.21.15.87:0/4050612186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66800ae020 msgr2=0x7f66800ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6687c00640 1 --2- 172.21.15.87:0/4050612186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66800ae020 0x7f66800ae4b0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6678009d90 tx=0x7f667801c900 comp rx=0 tx=0).stop 2024-04-25T10:20:20.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6687c00640 1 -- 172.21.15.87:0/4050612186 shutdown_connections 2024-04-25T10:20:20.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6687c00640 1 --2- 172.21.15.87:0/4050612186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66800ae020 0x7f66800ae4b0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6687c00640 1 -- 172.21.15.87:0/4050612186 >> 172.21.15.87:0/4050612186 conn(0x7f6680066b70 msgr2=0x7f6680066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:20.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6687c00640 1 -- 172.21.15.87:0/4050612186 shutdown_connections 2024-04-25T10:20:20.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.795+0000 7f6687c00640 1 -- 172.21.15.87:0/4050612186 wait complete. 2024-04-25T10:20:20.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 Processor -- start 2024-04-25T10:20:20.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 -- start start 2024-04-25T10:20:20.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6680070c20 0x7f66801515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800ae020 0x7f6680159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66800cf930 0x7f66801598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668016ba70 con 0x7f6680070c20 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668016b8f0 con 0x7f66800cf930 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668016bbf0 con 0x7f66800ae020 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6680070c20 0x7f66801515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6680070c20 0x7f66801515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34280/0 (socket says 172.21.15.87:34280) 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 -- 172.21.15.87:0/1241244090 learned_addr learned my addr 172.21.15.87:0/1241244090 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685c00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66800cf930 0x7f66801598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6677e00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800ae020 0x7f6680159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 -- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800ae020 msgr2=0x7f6680159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800ae020 0x7f6680159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 -- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66800cf930 msgr2=0x7f66801598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66800cf930 0x7f66801598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 -- 172.21.15.87:0/1241244090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680151b00 con 0x7f6680070c20 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6685200640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6680070c20 0x7f66801515c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f667000c3c0 tx=0x7f667000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f667d600640 1 -- 172.21.15.87:0/1241244090 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670017070 con 0x7f6680070c20 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f667d600640 1 -- 172.21.15.87:0/1241244090 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6670005cf0 con 0x7f6680070c20 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66800ce340 con 0x7f6680070c20 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f667d600640 1 -- 172.21.15.87:0/1241244090 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700072f0 con 0x7f6680070c20 2024-04-25T10:20:20.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.799+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6680151f90 con 0x7f6680070c20 2024-04-25T10:20:20.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.803+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6680151da0 con 0x7f6680070c20 2024-04-25T10:20:20.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.803+0000 7f667d600640 1 -- 172.21.15.87:0/1241244090 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f667001a070 con 0x7f6680070c20 2024-04-25T10:20:20.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.803+0000 7f667d600640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f665c066f60 0x7f665c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.803+0000 7f667d600640 1 -- 172.21.15.87:0/1241244090 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6670084390 con 0x7f6680070c20 2024-04-25T10:20:20.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.803+0000 7f6677e00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f665c066f60 0x7f665c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.803+0000 7f6677e00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f665c066f60 0x7f665c069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f66801f9f40 tx=0x7f6668007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:20.807+0000 7f667d600640 1 -- 172.21.15.87:0/1241244090 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6670054d40 con 0x7f6680070c20 2024-04-25T10:20:21.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.015+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f668004ef30 con 0x7f6680070c20 2024-04-25T10:20:21.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.015+0000 7f667d600640 1 -- 172.21.15.87:0/1241244090 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f6670058de0 con 0x7f6680070c20 2024-04-25T10:20:21.019 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:21.019 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":502,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076635813,"bytes_used":2593480704,"bytes_avail":770500632576,"bytes_total":773094113280,"write_bytes_sec":15355807,"read_op_per_sec":0,"write_op_per_sec":15},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T10:20:19.226077+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f665c066f60 msgr2=0x7f665c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f665c066f60 0x7f665c069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f66801f9f40 tx=0x7f6668007e40 comp rx=0 tx=0).stop 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6680070c20 msgr2=0x7f66801515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6680070c20 0x7f66801515c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f667000c3c0 tx=0x7f667000c880 comp rx=0 tx=0).stop 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 shutdown_connections 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f665c066f60 0x7f665c069420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66800cf930 0x7f66801598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800ae020 0x7f6680159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 --2- 172.21.15.87:0/1241244090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6680070c20 0x7f66801515c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 >> 172.21.15.87:0/1241244090 conn(0x7f6680066b70 msgr2=0x7f668015f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 shutdown_connections 2024-04-25T10:20:21.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.019+0000 7f6687c00640 1 -- 172.21.15.87:0/1241244090 wait complete. 2024-04-25T10:20:21.039 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:20:21.039 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:20:21.039 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:21.127 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:21.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:20:21.223 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:20:21.223 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:21.233 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:20:21.233 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:21.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:21.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf9ce00640 1 Processor -- start 2024-04-25T10:20:21.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf9ce00640 1 -- start start 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf9ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 0x7fbf9806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf9ce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf980b2770 con 0x7fbf980cf930 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf9ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9806c670 con 0x7fbf980cda10 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf9ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9806c950 con 0x7fbf9815bc40 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf96800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbf980cf930 0x7fbf980b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60316/0 (socket says 172.21.15.87:60316) 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf96800640 1 -- 172.21.15.87:0/3254519901 learned_addr learned my addr 172.21.15.87:0/3254519901 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:21.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95e00640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 0x7fbf9806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820993863 0 0) 0x7fbf980b2770 con 0x7fbf980cf930 2024-04-25T10:20:21.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf7c003530 con 0x7fbf980cf930 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000039317 0 0) 0x7fbf9806c670 con 0x7fbf980cda10 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf980b2770 con 0x7fbf980cda10 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3319110730 0 0) 0x7fbf7c003530 con 0x7fbf980cf930 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 msgr2=0x7fbf9806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 0x7fbf9806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 >> v1:172.21.15.169:6789/0 conn(0x7fbf980cda10 legacy=0x7fbf980b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9806cc30 con 0x7fbf980cf930 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.459+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fbf90001a00 con 0x7fbf980cf930 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 >> v1:172.21.15.87:6789/0 conn(0x7fbf980cf930 legacy=0x7fbf980b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf7c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 0x7fbf7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf7c004d50 0x7fbf7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9806c950 con 0x7fbf7c003b80 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c00bad0 con 0x7fbf7c004d50 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c00bcd0 con 0x7fbf9815bc40 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95e00640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf7c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf8fe00640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 0x7fbf7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf96800640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf7c004d50 0x7fbf7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95e00640 1 -- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 msgr2=0x7fbf7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95e00640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf9815bc40 0x7fbf7c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95e00640 1 -- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf7c004d50 msgr2=0x7fbf7c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95e00640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf7c004d50 0x7fbf7c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95e00640 1 -- 172.21.15.87:0/3254519901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c003530 con 0x7fbf7c003b80 2024-04-25T10:20:21.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95e00640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf7c003f80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009ae0 tx=0x7fbf8801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2c076f0a8baa24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880096b0 con 0x7fbf7c003b80 2024-04-25T10:20:21.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf880a6450 con 0x7fbf7c003b80 2024-04-25T10:20:21.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.463+0000 7fbf95400640 1 -- 172.21.15.87:0/3254519901 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880a6840 con 0x7fbf7c003b80 2024-04-25T10:20:21.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- 172.21.15.87:0/3254519901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 msgr2=0x7fbf7c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 --2- 172.21.15.87:0/3254519901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf7c003f80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009ae0 tx=0x7fbf8801ca70 comp rx=0 tx=0).stop 2024-04-25T10:20:21.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf96800640 1 -- 172.21.15.87:0/3254519901 reap_dead start 2024-04-25T10:20:21.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- 172.21.15.87:0/3254519901 shutdown_connections 2024-04-25T10:20:21.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- 172.21.15.87:0/3254519901 >> 172.21.15.87:0/3254519901 conn(0x7fbf98066b70 msgr2=0x7fbf98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- 172.21.15.87:0/3254519901 shutdown_connections 2024-04-25T10:20:21.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- 172.21.15.87:0/3254519901 wait complete. 2024-04-25T10:20:21.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 Processor -- start 2024-04-25T10:20:21.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- start start 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf980b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf98071f40 0x7fbf981f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf98070bb0 0x7fbf980af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf98167b50 con 0x7fbf7c003b80 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981679d0 con 0x7fbf98071f40 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf9ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf98167cd0 con 0x7fbf98070bb0 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf980b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf980b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34286/0 (socket says 172.21.15.87:34286) 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 -- 172.21.15.87:0/2782210005 learned_addr learned my addr 172.21.15.87:0/2782210005 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf96800640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf98070bb0 0x7fbf980af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf8fe00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf98071f40 0x7fbf981f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 -- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf98070bb0 msgr2=0x7fbf980af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf98070bb0 0x7fbf980af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 -- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf98071f40 msgr2=0x7fbf981f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf98071f40 0x7fbf981f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf95e00640 1 -- 172.21.15.87:0/2782210005 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf981f8050 con 0x7fbf7c003b80 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf8fe00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf98071f40 0x7fbf981f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.467+0000 7fbf96800640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf98070bb0 0x7fbf980af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf95e00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf980b1880 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009ae0 tx=0x7fbf7c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8e000640 1 -- 172.21.15.87:0/2782210005 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880c0070 con 0x7fbf7c003b80 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8e000640 1 -- 172.21.15.87:0/2782210005 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8800f000 con 0x7fbf7c003b80 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf981f8330 con 0x7fbf7c003b80 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8e000640 1 -- 172.21.15.87:0/2782210005 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880a6e20 con 0x7fbf7c003b80 2024-04-25T10:20:21.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf980703b0 con 0x7fbf7c003b80 2024-04-25T10:20:21.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9804ef30 con 0x7fbf7c003b80 2024-04-25T10:20:21.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8e000640 1 -- 172.21.15.87:0/2782210005 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf880c3070 con 0x7fbf7c003b80 2024-04-25T10:20:21.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8e000640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f90 0x7fbf74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8e000640 1 -- 172.21.15.87:0/2782210005 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf8811ed60 con 0x7fbf7c003b80 2024-04-25T10:20:21.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8fe00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f90 0x7fbf74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.471+0000 7fbf8fe00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f90 0x7fbf74069450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbf98072380 tx=0x7fbf80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.475+0000 7fbf8e000640 1 -- 172.21.15.87:0/2782210005 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf880ac020 con 0x7fbf7c003b80 2024-04-25T10:20:21.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.631+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf980af1c0 con 0x7fbf7c003b80 2024-04-25T10:20:21.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.631+0000 7fbf8e000640 1 -- 172.21.15.87:0/2782210005 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16598 (secure 0 0 0) 0x7fbf88016780 con 0x7fbf7c003b80 2024-04-25T10:20:21.634 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:21.634 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:38.756098+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:35.484354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:19:36.111171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:21.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.631+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f90 msgr2=0x7fbf74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.631+0000 7fbf9ce00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f90 0x7fbf74069450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbf98072380 tx=0x7fbf80009040 comp rx=0 tx=0).stop 2024-04-25T10:20:21.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.631+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 msgr2=0x7fbf980b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.631+0000 7fbf9ce00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf980b1880 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009ae0 tx=0x7fbf7c005840 comp rx=0 tx=0).stop 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 shutdown_connections 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f90 0x7fbf74069450 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf98070bb0 0x7fbf980af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf98071f40 0x7fbf981f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 --2- 172.21.15.87:0/2782210005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c003b80 0x7fbf980b1880 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 >> 172.21.15.87:0/2782210005 conn(0x7fbf98066b70 msgr2=0x7fbf98068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 shutdown_connections 2024-04-25T10:20:21.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.635+0000 7fbf9ce00640 1 -- 172.21.15.87:0/2782210005 wait complete. 2024-04-25T10:20:21.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:19:35.484354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:19:36.111171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:21.655 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:21.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc57e400640 1 Processor -- start 2024-04-25T10:20:21.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc57e400640 1 -- start start 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc57e400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5780aef80 0x7fc57815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc57e400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc578070350 0x7fc57815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc57e400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57815b080 con 0x7fc57815bac0 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc57e400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57815adb0 con 0x7fc578070350 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc57e400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57815eb60 con 0x7fc5780aef80 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc576a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc578070350 0x7fc57815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc577e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc57815bac0 0x7fc5780b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60332/0 (socket says 172.21.15.87:60332) 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc576a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc578070350 0x7fc57815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34300/0 (socket says 172.21.15.87:34300) 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc577e00640 1 -- 172.21.15.87:0/1284115340 learned_addr learned my addr 172.21.15.87:0/1284115340 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc577400640 1 --2- 172.21.15.87:0/1284115340 >> v2:172.21.15.169:3300/0 conn(0x7fc5780aef80 0x7fc57815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc576a00640 1 -- 172.21.15.87:0/1284115340 >> v2:172.21.15.169:3300/0 conn(0x7fc5780aef80 msgr2=0x7fc57815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:21.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc576a00640 1 --2- 172.21.15.87:0/1284115340 >> v2:172.21.15.169:3300/0 conn(0x7fc5780aef80 0x7fc57815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc576a00640 1 -- 172.21.15.87:0/1284115340 >> v1:172.21.15.87:6789/0 conn(0x7fc57815bac0 legacy=0x7fc5780b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.791+0000 7fc576a00640 1 -- 172.21.15.87:0/1284115340 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57815ee40 con 0x7fc578070350 2024-04-25T10:20:21.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc576a00640 1 --2- 172.21.15.87:0/1284115340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc578070350 0x7fc57815b580 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc57806e220 tx=0x7fc56401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca65d2589c15772e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc576000640 1 -- 172.21.15.87:0/1284115340 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5640ab660 con 0x7fc578070350 2024-04-25T10:20:21.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc576000640 1 -- 172.21.15.87:0/1284115340 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5640b0070 con 0x7fc578070350 2024-04-25T10:20:21.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc576000640 1 -- 172.21.15.87:0/1284115340 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5640aba20 con 0x7fc578070350 2024-04-25T10:20:21.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 -- 172.21.15.87:0/1284115340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc578070350 msgr2=0x7fc57815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 --2- 172.21.15.87:0/1284115340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc578070350 0x7fc57815b580 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc57806e220 tx=0x7fc56401c480 comp rx=0 tx=0).stop 2024-04-25T10:20:21.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 -- 172.21.15.87:0/1284115340 shutdown_connections 2024-04-25T10:20:21.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 --2- 172.21.15.87:0/1284115340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc578070350 0x7fc57815b580 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 --2- 172.21.15.87:0/1284115340 >> v2:172.21.15.169:3300/0 conn(0x7fc5780aef80 0x7fc57815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 -- 172.21.15.87:0/1284115340 >> 172.21.15.87:0/1284115340 conn(0x7fc5780669a0 msgr2=0x7fc578066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 -- 172.21.15.87:0/1284115340 shutdown_connections 2024-04-25T10:20:21.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.795+0000 7fc57e400640 1 -- 172.21.15.87:0/1284115340 wait complete. 2024-04-25T10:20:21.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 Processor -- start 2024-04-25T10:20:21.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 -- start start 2024-04-25T10:20:21.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc578070350 0x7fc5780afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5780aef80 0x7fc57815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc57815bac0 0x7fc5781590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5781679a0 con 0x7fc57815bac0 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc578167820 con 0x7fc578070350 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc578167b20 con 0x7fc5780aef80 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc57815bac0 0x7fc5781590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc57815bac0 0x7fc5781590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34314/0 (socket says 172.21.15.87:34314) 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 -- 172.21.15.87:0/3503640080 learned_addr learned my addr 172.21.15.87:0/3503640080 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc576a00640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5780aef80 0x7fc57815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc578070350 0x7fc5780afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 -- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5780aef80 msgr2=0x7fc57815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5780aef80 0x7fc57815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 -- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc578070350 msgr2=0x7fc5780afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc578070350 0x7fc5780afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 -- 172.21.15.87:0/3503640080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780b06b0 con 0x7fc57815bac0 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc577e00640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc57815bac0 0x7fc5781590b0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c3c0 tx=0x7fc57000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc56f400640 1 -- 172.21.15.87:0/3503640080 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc570017070 con 0x7fc57815bac0 2024-04-25T10:20:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc56f400640 1 -- 172.21.15.87:0/3503640080 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc570005cf0 con 0x7fc57815bac0 2024-04-25T10:20:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc56f400640 1 -- 172.21.15.87:0/3503640080 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700072f0 con 0x7fc57815bac0 2024-04-25T10:20:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5780b0990 con 0x7fc57815bac0 2024-04-25T10:20:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.799+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5780b0f50 con 0x7fc57815bac0 2024-04-25T10:20:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.803+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57804ef30 con 0x7fc57815bac0 2024-04-25T10:20:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.803+0000 7fc56f400640 1 -- 172.21.15.87:0/3503640080 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc57001a070 con 0x7fc57815bac0 2024-04-25T10:20:21.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.803+0000 7fc56f400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc558066f60 0x7fc558069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.803+0000 7fc56f400640 1 -- 172.21.15.87:0/3503640080 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5700842e0 con 0x7fc57815bac0 2024-04-25T10:20:21.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.803+0000 7fc577400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc558066f60 0x7fc558069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.803+0000 7fc577400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc558066f60 0x7fc558069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5680113e0 tx=0x7fc56800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.807+0000 7fc56f400640 1 -- 172.21.15.87:0/3503640080 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc570054c90 con 0x7fc57815bac0 2024-04-25T10:20:21.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.963+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5780b2fa0 con 0x7fc57815bac0 2024-04-25T10:20:21.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc56f400640 1 -- 172.21.15.87:0/3503640080 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc570058d30 con 0x7fc57815bac0 2024-04-25T10:20:21.969 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:20:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc558066f60 msgr2=0x7fc558069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc558066f60 0x7fc558069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5680113e0 tx=0x7fc56800b040 comp rx=0 tx=0).stop 2024-04-25T10:20:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc57815bac0 msgr2=0x7fc5781590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc57815bac0 0x7fc5781590b0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c3c0 tx=0x7fc57000c880 comp rx=0 tx=0).stop 2024-04-25T10:20:21.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 shutdown_connections 2024-04-25T10:20:21.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc558066f60 0x7fc558069420 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc57815bac0 0x7fc5781590b0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5780aef80 0x7fc57815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 --2- 172.21.15.87:0/3503640080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc578070350 0x7fc5780afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 >> 172.21.15.87:0/3503640080 conn(0x7fc5780669a0 msgr2=0x7fc5780b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 shutdown_connections 2024-04-25T10:20:21.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:21.967+0000 7fc57e400640 1 -- 172.21.15.87:0/3503640080 wait complete. 2024-04-25T10:20:21.990 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:20:22.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd542e00640 1 Processor -- start 2024-04-25T10:20:22.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd542e00640 1 -- start start 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd542e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 0x7fd53c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd542e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd53c070350 0x7fd53c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd542e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53c15b080 con 0x7fd53c15bac0 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd542e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53c15adb0 con 0x7fd53c070350 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd542e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c15eb60 con 0x7fd53c0aef80 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd53c15bac0 0x7fd53c0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60334/0 (socket says 172.21.15.87:60334) 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 -- 172.21.15.87:0/1035635926 learned_addr learned my addr 172.21.15.87:0/1035635926 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53be00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 0x7fd53c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53b400640 1 --2- 172.21.15.87:0/1035635926 >> v2:172.21.15.169:3300/0 conn(0x7fd53c070350 0x7fd53c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717064179 0 0) 0x7fd53c15b080 con 0x7fd53c15bac0 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd520003530 con 0x7fd53c15bac0 2024-04-25T10:20:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041436348 0 0) 0x7fd520003530 con 0x7fd53c15bac0 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 msgr2=0x7fd53c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 0x7fd53c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 >> v2:172.21.15.169:3300/0 conn(0x7fd53c070350 msgr2=0x7fd53c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 --2- 172.21.15.87:0/1035635926 >> v2:172.21.15.169:3300/0 conn(0x7fd53c070350 0x7fd53c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c15ee40 con 0x7fd53c15bac0 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd534001a00 con 0x7fd53c15bac0 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 >> v1:172.21.15.87:6789/0 conn(0x7fd53c15bac0 legacy=0x7fd53c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200039c0 0x7fd520003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 0x7fd520004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd520004d50 0x7fd5200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c15adb0 con 0x7fd520004d50 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c15eb60 con 0x7fd5200039c0 2024-04-25T10:20:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52000b9d0 con 0x7fd53c0aef80 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd520004d50 0x7fd5200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53be00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200039c0 0x7fd520003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd53b400640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 0x7fd520004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 -- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 msgr2=0x7fd520004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53c0aef80 0x7fd520004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 -- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200039c0 msgr2=0x7fd520003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5200039c0 0x7fd520003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.127+0000 7fd540e00640 1 -- 172.21.15.87:0/1035635926 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520003530 con 0x7fd520004d50 2024-04-25T10:20:22.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd540e00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd520004d50 0x7fd5200051d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd53400a920 tx=0x7fd53401d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0e55bcde06e933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53400a4c0 con 0x7fd520004d50 2024-04-25T10:20:22.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5340072f0 con 0x7fd520004d50 2024-04-25T10:20:22.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd53aa00640 1 -- 172.21.15.87:0/1035635926 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340076e0 con 0x7fd520004d50 2024-04-25T10:20:22.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd542e00640 1 -- 172.21.15.87:0/1035635926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd520004d50 msgr2=0x7fd5200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd542e00640 1 --2- 172.21.15.87:0/1035635926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd520004d50 0x7fd5200051d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd53400a920 tx=0x7fd53401d440 comp rx=0 tx=0).stop 2024-04-25T10:20:22.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd540e00640 1 -- 172.21.15.87:0/1035635926 reap_dead start 2024-04-25T10:20:22.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd542e00640 1 -- 172.21.15.87:0/1035635926 shutdown_connections 2024-04-25T10:20:22.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd542e00640 1 -- 172.21.15.87:0/1035635926 >> 172.21.15.87:0/1035635926 conn(0x7fd53c0669a0 msgr2=0x7fd53c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd542e00640 1 -- 172.21.15.87:0/1035635926 shutdown_connections 2024-04-25T10:20:22.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.131+0000 7fd542e00640 1 -- 172.21.15.87:0/1035635926 wait complete. 2024-04-25T10:20:22.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 Processor -- start 2024-04-25T10:20:22.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 -- start start 2024-04-25T10:20:22.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520004d50 0x7fd53c158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd53c157220 0x7fd53c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd53c0cfb80 0x7fd53c13bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c0b38d0 con 0x7fd53c157220 2024-04-25T10:20:22.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c071a00 con 0x7fd53c0cfb80 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c0b3a50 con 0x7fd520004d50 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd53c157220 0x7fd53c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd53c157220 0x7fd53c0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34328/0 (socket says 172.21.15.87:34328) 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 -- 172.21.15.87:0/2117265051 learned_addr learned my addr 172.21.15.87:0/2117265051 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53be00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520004d50 0x7fd53c158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd540e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd53c0cfb80 0x7fd53c13bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 -- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520004d50 msgr2=0x7fd53c158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520004d50 0x7fd53c158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 -- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd53c0cfb80 msgr2=0x7fd53c13bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd53c0cfb80 0x7fd53c13bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 -- 172.21.15.87:0/2117265051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c159830 con 0x7fd53c157220 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd540e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd53c0cfb80 0x7fd53c13bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd53b400640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd53c157220 0x7fd53c0cf640 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd528011ad0 tx=0x7fd528011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd533e00640 1 -- 172.21.15.87:0/2117265051 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52800fbb0 con 0x7fd53c157220 2024-04-25T10:20:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd533e00640 1 -- 172.21.15.87:0/2117265051 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52800fd50 con 0x7fd53c157220 2024-04-25T10:20:22.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c159b10 con 0x7fd53c157220 2024-04-25T10:20:22.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd533e00640 1 -- 172.21.15.87:0/2117265051 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528014680 con 0x7fd53c157220 2024-04-25T10:20:22.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c15a0d0 con 0x7fd53c157220 2024-04-25T10:20:22.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.135+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53c04ef30 con 0x7fd53c157220 2024-04-25T10:20:22.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.139+0000 7fd533e00640 1 -- 172.21.15.87:0/2117265051 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd528012040 con 0x7fd53c157220 2024-04-25T10:20:22.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.139+0000 7fd533e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd51c066f60 0x7fd51c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.139+0000 7fd533e00640 1 -- 172.21.15.87:0/2117265051 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd528085860 con 0x7fd53c157220 2024-04-25T10:20:22.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.139+0000 7fd53be00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd51c066f60 0x7fd51c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.139+0000 7fd53be00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd51c066f60 0x7fd51c069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0d0200 tx=0x7fd52c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.143+0000 7fd533e00640 1 -- 172.21.15.87:0/2117265051 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd528019030 con 0x7fd53c157220 2024-04-25T10:20:22.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd53c072590 con 0x7fd53c157220 2024-04-25T10:20:22.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd533e00640 1 -- 172.21.15.87:0/2117265051 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5280567d0 con 0x7fd53c157220 2024-04-25T10:20:22.306 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd51c066f60 msgr2=0x7fd51c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd51c066f60 0x7fd51c069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0d0200 tx=0x7fd52c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd53c157220 msgr2=0x7fd53c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd53c157220 0x7fd53c0cf640 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd528011ad0 tx=0x7fd528011f90 comp rx=0 tx=0).stop 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 shutdown_connections 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd51c066f60 0x7fd51c069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd53c0cfb80 0x7fd53c13bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd53c157220 0x7fd53c0cf640 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 --2- 172.21.15.87:0/2117265051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd520004d50 0x7fd53c158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 >> 172.21.15.87:0/2117265051 conn(0x7fd53c0669a0 msgr2=0x7fd53c15bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 shutdown_connections 2024-04-25T10:20:22.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.303+0000 7fd542e00640 1 -- 172.21.15.87:0/2117265051 wait complete. 2024-04-25T10:20:22.327 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:20:22.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b7600640 1 Processor -- start 2024-04-25T10:20:22.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b7600640 1 -- start start 2024-04-25T10:20:22.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b7600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b0159540 0x7f81b00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b7600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81b00aebd0 0x7f81b0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b7600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b0071360 con 0x7f81b0072020 2024-04-25T10:20:22.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b7600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b00af030 con 0x7f81b00aebd0 2024-04-25T10:20:22.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b7600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0159920 con 0x7f81b0159540 2024-04-25T10:20:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81afe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81b00aebd0 0x7f81b0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81afe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81b00aebd0 0x7f81b0071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34334/0 (socket says 172.21.15.87:34334) 2024-04-25T10:20:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81afe00640 1 -- 172.21.15.87:0/2257869325 learned_addr learned my addr 172.21.15.87:0/2257869325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b4c00640 1 --2- 172.21.15.87:0/2257869325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b0159540 0x7f81b00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81afe00640 1 -- 172.21.15.87:0/2257869325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b0159540 msgr2=0x7f81b00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81afe00640 1 --2- 172.21.15.87:0/2257869325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b0159540 0x7f81b00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81afe00640 1 -- 172.21.15.87:0/2257869325 >> v1:172.21.15.169:6789/0 conn(0x7f81b0072020 legacy=0x7f81b015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81afe00640 1 -- 172.21.15.87:0/2257869325 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b015c2f0 con 0x7f81b00aebd0 2024-04-25T10:20:22.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.463+0000 7f81b4c00640 1 --2- 172.21.15.87:0/2257869325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b0159540 0x7f81b00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:22.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81afe00640 1 --2- 172.21.15.87:0/2257869325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b00aebd0 0x7f81b0071ae0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f819c009870 tx=0x7f819c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b594596969c3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81af400640 1 -- 172.21.15.87:0/2257869325 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0ab650 con 0x7f81b00aebd0 2024-04-25T10:20:22.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81af400640 1 -- 172.21.15.87:0/2257869325 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c0b0070 con 0x7f81b00aebd0 2024-04-25T10:20:22.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81af400640 1 -- 172.21.15.87:0/2257869325 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0aba10 con 0x7f81b00aebd0 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 -- 172.21.15.87:0/2257869325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b00aebd0 msgr2=0x7f81b0071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 --2- 172.21.15.87:0/2257869325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b00aebd0 0x7f81b0071ae0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f819c009870 tx=0x7f819c01c530 comp rx=0 tx=0).stop 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 -- 172.21.15.87:0/2257869325 shutdown_connections 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 --2- 172.21.15.87:0/2257869325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b00aebd0 0x7f81b0071ae0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 --2- 172.21.15.87:0/2257869325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b0159540 0x7f81b00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 -- 172.21.15.87:0/2257869325 >> 172.21.15.87:0/2257869325 conn(0x7f81b00669a0 msgr2=0x7f81b0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 -- 172.21.15.87:0/2257869325 shutdown_connections 2024-04-25T10:20:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.467+0000 7f81b7600640 1 -- 172.21.15.87:0/2257869325 wait complete. 2024-04-25T10:20:22.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 Processor -- start 2024-04-25T10:20:22.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 -- start start 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81b0072020 0x7f81b014cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b00aebd0 0x7f81b01509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0159540 0x7f81b0156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b006c0d0 con 0x7f81b0159540 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015bfa0 con 0x7f81b0072020 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0072430 con 0x7f81b00aebd0 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0159540 0x7f81b0156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0159540 0x7f81b0156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34350/0 (socket says 172.21.15.87:34350) 2024-04-25T10:20:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 -- 172.21.15.87:0/3548490797 learned_addr learned my addr 172.21.15.87:0/3548490797 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b4c00640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81b0072020 0x7f81b014cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81afe00640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b00aebd0 0x7f81b01509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 -- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b00aebd0 msgr2=0x7f81b01509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b00aebd0 0x7f81b01509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 -- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81b0072020 msgr2=0x7f81b014cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81b0072020 0x7f81b014cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 -- 172.21.15.87:0/3548490797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b014fc30 con 0x7f81b0159540 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b5600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0159540 0x7f81b0156270 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c7a0 tx=0x7f81a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81acc00640 1 -- 172.21.15.87:0/3548490797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8003b80 con 0x7f81b0159540 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81acc00640 1 -- 172.21.15.87:0/3548490797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8003d20 con 0x7f81b0159540 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81acc00640 1 -- 172.21.15.87:0/3548490797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800aa30 con 0x7f81b0159540 2024-04-25T10:20:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b014ff10 con 0x7f81b0159540 2024-04-25T10:20:22.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b01504d0 con 0x7f81b0159540 2024-04-25T10:20:22.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.471+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b004ef30 con 0x7f81b0159540 2024-04-25T10:20:22.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.475+0000 7f81acc00640 1 -- 172.21.15.87:0/3548490797 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81a800abd0 con 0x7f81b0159540 2024-04-25T10:20:22.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.475+0000 7f81acc00640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8190066f90 0x7f8190069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.475+0000 7f81acc00640 1 -- 172.21.15.87:0/3548490797 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81a8084f30 con 0x7f81b0159540 2024-04-25T10:20:22.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.475+0000 7f81b4c00640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8190066f90 0x7f8190069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.475+0000 7f81b4c00640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8190066f90 0x7f8190069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f81b014f730 tx=0x7f81a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.479+0000 7f81acc00640 1 -- 172.21.15.87:0/3548490797 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81a8055f30 con 0x7f81b0159540 2024-04-25T10:20:22.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.631+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81b0166700 con 0x7f81b0159540 2024-04-25T10:20:22.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.631+0000 7f81acc00640 1 -- 172.21.15.87:0/3548490797 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81a8055830 con 0x7f81b0159540 2024-04-25T10:20:22.636 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8190066f90 msgr2=0x7f8190069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8190066f90 0x7f8190069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f81b014f730 tx=0x7f81a4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0159540 msgr2=0x7f81b0156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0159540 0x7f81b0156270 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c7a0 tx=0x7f81a800cc60 comp rx=0 tx=0).stop 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 shutdown_connections 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8190066f90 0x7f8190069450 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0159540 0x7f81b0156270 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81b00aebd0 0x7f81b01509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 --2- 172.21.15.87:0/3548490797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81b0072020 0x7f81b014cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 >> 172.21.15.87:0/3548490797 conn(0x7f81b00669a0 msgr2=0x7f81b0066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 shutdown_connections 2024-04-25T10:20:22.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.635+0000 7f81b7600640 1 -- 172.21.15.87:0/3548490797 wait complete. 2024-04-25T10:20:22.656 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:20:22.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e9000640 1 Processor -- start 2024-04-25T10:20:22.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e9000640 1 -- start start 2024-04-25T10:20:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05e41550b0 0x7f05e40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e40cfe70 0x7f05e415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e9000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e4071050 con 0x7f05e415b740 2024-04-25T10:20:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e9000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e415d040 con 0x7f05e41550b0 2024-04-25T10:20:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40712f0 con 0x7f05e40cfe70 2024-04-25T10:20:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05e41550b0 0x7f05e40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e2800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f05e415b740 0x7f05e406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60340/0 (socket says 172.21.15.87:60340) 2024-04-25T10:20:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e2800640 1 -- 172.21.15.87:0/1943350530 learned_addr learned my addr 172.21.15.87:0/1943350530 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05dbe00640 1 --2- 172.21.15.87:0/1943350530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e40cfe70 0x7f05e415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e1e00640 1 -- 172.21.15.87:0/1943350530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e40cfe70 msgr2=0x7f05e415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e1e00640 1 --2- 172.21.15.87:0/1943350530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e40cfe70 0x7f05e415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e1e00640 1 -- 172.21.15.87:0/1943350530 >> v1:172.21.15.87:6789/0 conn(0x7f05e415b740 legacy=0x7f05e406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e1400640 1 -- 172.21.15.87:0/1943350530 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195532475 0 0) 0x7f05e4071050 con 0x7f05e415b740 2024-04-25T10:20:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.791+0000 7f05e1e00640 1 -- 172.21.15.87:0/1943350530 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40715d0 con 0x7f05e41550b0 2024-04-25T10:20:22.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e1e00640 1 --2- 172.21.15.87:0/1943350530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e40cf930 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f05d40097c0 tx=0x7f05d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6186ceb7acc153d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e1400640 1 -- 172.21.15.87:0/1943350530 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40b0070 con 0x7f05e41550b0 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e1400640 1 -- 172.21.15.87:0/1943350530 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40ab600 con 0x7f05e41550b0 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e1400640 1 -- 172.21.15.87:0/1943350530 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40ab9d0 con 0x7f05e41550b0 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 -- 172.21.15.87:0/1943350530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 msgr2=0x7f05e40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 --2- 172.21.15.87:0/1943350530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e40cf930 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f05d40097c0 tx=0x7f05d401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 -- 172.21.15.87:0/1943350530 shutdown_connections 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 --2- 172.21.15.87:0/1943350530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e40cfe70 0x7f05e415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 --2- 172.21.15.87:0/1943350530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e40cf930 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 -- 172.21.15.87:0/1943350530 >> 172.21.15.87:0/1943350530 conn(0x7f05e4066b70 msgr2=0x7f05e4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 -- 172.21.15.87:0/1943350530 shutdown_connections 2024-04-25T10:20:22.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 -- 172.21.15.87:0/1943350530 wait complete. 2024-04-25T10:20:22.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.795+0000 7f05e9000640 1 Processor -- start 2024-04-25T10:20:22.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 -- start start 2024-04-25T10:20:22.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e40cfe70 0x7f05e4159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e413e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e415b740 0x7f05e40b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e415bd10 con 0x7f05e41550b0 2024-04-25T10:20:22.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e415bb90 con 0x7f05e40cfe70 2024-04-25T10:20:22.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e415c030 con 0x7f05e415b740 2024-04-25T10:20:22.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e413e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e413e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34372/0 (socket says 172.21.15.87:34372) 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 -- 172.21.15.87:0/3822658498 learned_addr learned my addr 172.21.15.87:0/3822658498 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e1e00640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e40cfe70 0x7f05e4159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e2800640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e415b740 0x7f05e40b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 -- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e415b740 msgr2=0x7f05e40b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e415b740 0x7f05e40b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 -- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e40cfe70 msgr2=0x7f05e4159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e40cfe70 0x7f05e4159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 -- 172.21.15.87:0/3822658498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e41599f0 con 0x7f05e41550b0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05dbe00640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e413e170 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e960 tx=0x7f05cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05da000640 1 -- 172.21.15.87:0/3822658498 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc017070 con 0x7f05e41550b0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05da000640 1 -- 172.21.15.87:0/3822658498 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc00ca00 con 0x7f05e41550b0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05da000640 1 -- 172.21.15.87:0/3822658498 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00cdb0 con 0x7f05e41550b0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e4159cd0 con 0x7f05e41550b0 2024-04-25T10:20:22.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e415a290 con 0x7f05e41550b0 2024-04-25T10:20:22.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.799+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e404ef30 con 0x7f05e41550b0 2024-04-25T10:20:22.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.803+0000 7f05da000640 1 -- 172.21.15.87:0/3822658498 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05cc01a070 con 0x7f05e41550b0 2024-04-25T10:20:22.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.803+0000 7f05da000640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05c0066f60 0x7f05c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.803+0000 7f05da000640 1 -- 172.21.15.87:0/3822658498 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05cc085450 con 0x7f05e41550b0 2024-04-25T10:20:22.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.803+0000 7f05e1e00640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05c0066f60 0x7f05c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.803+0000 7f05e1e00640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05c0066f60 0x7f05c0069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f05d40ad000 tx=0x7f05d40a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.807+0000 7f05da000640 1 -- 172.21.15.87:0/3822658498 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05cc056450 con 0x7f05e41550b0 2024-04-25T10:20:22.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f05e4151c80 con 0x7f05e41550b0 2024-04-25T10:20:22.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05da000640 1 -- 172.21.15.87:0/3822658498 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) v1 ==== 80+0+0 (secure 0 0 0) 0x7f05cc055d50 con 0x7f05e41550b0 2024-04-25T10:20:22.978 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05c0066f60 msgr2=0x7f05c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05c0066f60 0x7f05c0069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f05d40ad000 tx=0x7f05d40a6000 comp rx=0 tx=0).stop 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 msgr2=0x7f05e413e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e413e170 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e960 tx=0x7f05cc00ee20 comp rx=0 tx=0).stop 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 shutdown_connections 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05c0066f60 0x7f05c0069420 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e415b740 0x7f05e40b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05e41550b0 0x7f05e413e170 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 --2- 172.21.15.87:0/3822658498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e40cfe70 0x7f05e4159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 >> 172.21.15.87:0/3822658498 conn(0x7f05e4066b70 msgr2=0x7f05e4150fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 shutdown_connections 2024-04-25T10:20:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:22.975+0000 7f05e9000640 1 -- 172.21.15.87:0/3822658498 wait complete. 2024-04-25T10:20:22.998 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:20:22.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:22.999 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:23.230 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:23.234 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:23.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:20:23.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:23.680 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:20:23.689 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:20:23.689 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:23.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:23.699 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:20:23.699 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:23.744 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:23.744 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:20:23.744 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:23.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:23.753 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:23.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.923+0000 7f7064200640 1 Processor -- start 2024-04-25T10:20:23.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.923+0000 7f7064200640 1 -- start start 2024-04-25T10:20:23.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.923+0000 7f7064200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0cf930 0x7f705c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.923+0000 7f7064200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f705c0ae020 0x7f705c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.923+0000 7f7064200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c0d0250 con 0x7f705c070c20 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.923+0000 7f7064200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c15bb90 con 0x7f705c0ae020 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.923+0000 7f7064200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0ad840 con 0x7f705c0cf930 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7062200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f705c0ae020 0x7f705c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7060e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f705c070c20 0x7f705c15e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60356/0 (socket says 172.21.15.87:60356) 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7060e00640 1 -- 172.21.15.87:0/3522743096 learned_addr learned my addr 172.21.15.87:0/3522743096 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7062200640 1 -- 172.21.15.87:0/3522743096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0cf930 msgr2=0x7f705c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7062200640 1 --2- 172.21.15.87:0/3522743096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0cf930 0x7f705c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7062200640 1 -- 172.21.15.87:0/3522743096 >> v1:172.21.15.87:6789/0 conn(0x7f705c070c20 legacy=0x7f705c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7062200640 1 -- 172.21.15.87:0/3522743096 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c15e9f0 con 0x7f705c0ae020 2024-04-25T10:20:23.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f705be00640 1 -- 172.21.15.87:0/3522743096 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608225582 0 0) 0x7f705c0d0250 con 0x7f705c070c20 2024-04-25T10:20:23.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f7062200640 1 --2- 172.21.15.87:0/3522743096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0ae020 0x7f705c0ae4b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f70540097c0 tx=0x7f705401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ad9c82f347a13f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f705be00640 1 -- 172.21.15.87:0/3522743096 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540a7070 con 0x7f705c0ae020 2024-04-25T10:20:23.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f705be00640 1 -- 172.21.15.87:0/3522743096 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70540b0070 con 0x7f705c0ae020 2024-04-25T10:20:23.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.927+0000 7f705be00640 1 -- 172.21.15.87:0/3522743096 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540ab890 con 0x7f705c0ae020 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- 172.21.15.87:0/3522743096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0ae020 msgr2=0x7f705c0ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 --2- 172.21.15.87:0/3522743096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0ae020 0x7f705c0ae4b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f70540097c0 tx=0x7f705401c470 comp rx=0 tx=0).stop 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- 172.21.15.87:0/3522743096 shutdown_connections 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 --2- 172.21.15.87:0/3522743096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0ae020 0x7f705c0ae4b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 --2- 172.21.15.87:0/3522743096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0cf930 0x7f705c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- 172.21.15.87:0/3522743096 >> 172.21.15.87:0/3522743096 conn(0x7f705c066b70 msgr2=0x7f705c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- 172.21.15.87:0/3522743096 shutdown_connections 2024-04-25T10:20:23.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- 172.21.15.87:0/3522743096 wait complete. 2024-04-25T10:20:23.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 Processor -- start 2024-04-25T10:20:23.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- start start 2024-04-25T10:20:23.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c070c20 0x7f705c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0ae020 0x7f705c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0cf930 0x7f705c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c16a8e0 con 0x7f705c0cf930 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c06e130 con 0x7f705c070c20 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7064200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c16aa60 con 0x7f705c0ae020 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7062200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0cf930 0x7f705c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7062200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0cf930 0x7f705c1530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34400/0 (socket says 172.21.15.87:34400) 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7062200640 1 -- 172.21.15.87:0/3713705831 learned_addr learned my addr 172.21.15.87:0/3713705831 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7060e00640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0ae020 0x7f705c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7061800640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c070c20 0x7f705c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7062200640 1 -- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0ae020 msgr2=0x7f705c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.931+0000 7f7062200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0ae020 0x7f705c15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7062200640 1 -- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c070c20 msgr2=0x7f705c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7062200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c070c20 0x7f705c0cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7062200640 1 -- 172.21.15.87:0/3713705831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c0cca90 con 0x7f705c0cf930 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7060e00640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0ae020 0x7f705c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7062200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0cf930 0x7f705c1530d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f70540097c0 tx=0x7f705401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7059600640 1 -- 172.21.15.87:0/3713705831 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540abd40 con 0x7f705c0cf930 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7059600640 1 -- 172.21.15.87:0/3713705831 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054004030 con 0x7f705c0cf930 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7059600640 1 -- 172.21.15.87:0/3713705831 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054004420 con 0x7f705c0cf930 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c0ccd70 con 0x7f705c0cf930 2024-04-25T10:20:23.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c0cc490 con 0x7f705c0cf930 2024-04-25T10:20:23.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c04ef30 con 0x7f705c0cf930 2024-04-25T10:20:23.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7059600640 1 -- 172.21.15.87:0/3713705831 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7054003620 con 0x7f705c0cf930 2024-04-25T10:20:23.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7059600640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f703c066f60 0x7f703c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.935+0000 7f7059600640 1 -- 172.21.15.87:0/3713705831 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f705411f900 con 0x7f705c0cf930 2024-04-25T10:20:23.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.939+0000 7f7061800640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f703c066f60 0x7f703c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.939+0000 7f7061800640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f703c066f60 0x7f703c069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f705000a400 tx=0x7f705000d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:23.943+0000 7f7059600640 1 -- 172.21.15.87:0/3713705831 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7054016090 con 0x7f705c0cf930 2024-04-25T10:20:24.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.127+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f705c158b70 con 0x7f705c0cf930 2024-04-25T10:20:24.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7059600640 1 -- 172.21.15.87:0/3713705831 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f705411f030 con 0x7f705c0cf930 2024-04-25T10:20:24.132 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:24.132 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:24.133 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f703c066f60 msgr2=0x7f703c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f703c066f60 0x7f703c069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f705000a400 tx=0x7f705000d040 comp rx=0 tx=0).stop 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0cf930 msgr2=0x7f705c1530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0cf930 0x7f705c1530d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f70540097c0 tx=0x7f705401acb0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 shutdown_connections 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f703c066f60 0x7f703c069420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f705c0cf930 0x7f705c1530d0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f705c0ae020 0x7f705c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 --2- 172.21.15.87:0/3713705831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c070c20 0x7f705c0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 >> 172.21.15.87:0/3713705831 conn(0x7f705c066b70 msgr2=0x7f705c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 shutdown_connections 2024-04-25T10:20:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.131+0000 7f7064200640 1 -- 172.21.15.87:0/3713705831 wait complete. 2024-04-25T10:20:24.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:24.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 Processor -- start 2024-04-25T10:20:24.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 -- start start 2024-04-25T10:20:24.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad140cf930 0x7fad140cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad1415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad140ae020 0x7fad140ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1415bb90 con 0x7fad140cf930 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad140d0250 con 0x7fad140ae020 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad1a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad140ad840 con 0x7fad14070c20 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad140cf930 0x7fad140cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad140cf930 0x7fad140cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34416/0 (socket says 172.21.15.87:34416) 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13400640 1 -- 172.21.15.87:0/691445377 learned_addr learned my addr 172.21.15.87:0/691445377 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13e00640 1 --2- 172.21.15.87:0/691445377 >> v2:172.21.15.169:3300/0 conn(0x7fad140ae020 0x7fad140ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad12a00640 1 --2- 172.21.15.87:0/691445377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad1415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13400640 1 -- 172.21.15.87:0/691445377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 msgr2=0x7fad1415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:24.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13400640 1 --2- 172.21.15.87:0/691445377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad1415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13400640 1 -- 172.21.15.87:0/691445377 >> v2:172.21.15.169:3300/0 conn(0x7fad140ae020 msgr2=0x7fad140ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:24.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.287+0000 7fad13400640 1 --2- 172.21.15.87:0/691445377 >> v2:172.21.15.169:3300/0 conn(0x7fad140ae020 0x7fad140ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad13400640 1 -- 172.21.15.87:0/691445377 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1415e9f0 con 0x7fad140cf930 2024-04-25T10:20:24.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad13400640 1 --2- 172.21.15.87:0/691445377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140cf930 0x7fad140cfd10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fad140d05c0 tx=0x7fad0401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca49c29d37ebf85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad12000640 1 -- 172.21.15.87:0/691445377 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040a5840 con 0x7fad140cf930 2024-04-25T10:20:24.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad12000640 1 -- 172.21.15.87:0/691445377 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad040a59e0 con 0x7fad140cf930 2024-04-25T10:20:24.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad12000640 1 -- 172.21.15.87:0/691445377 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040a5db0 con 0x7fad140cf930 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 -- 172.21.15.87:0/691445377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140cf930 msgr2=0x7fad140cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 --2- 172.21.15.87:0/691445377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140cf930 0x7fad140cfd10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fad140d05c0 tx=0x7fad0401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 -- 172.21.15.87:0/691445377 shutdown_connections 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 --2- 172.21.15.87:0/691445377 >> v2:172.21.15.169:3300/0 conn(0x7fad140ae020 0x7fad140ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 --2- 172.21.15.87:0/691445377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad1415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 --2- 172.21.15.87:0/691445377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140cf930 0x7fad140cfd10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 -- 172.21.15.87:0/691445377 >> 172.21.15.87:0/691445377 conn(0x7fad14066b70 msgr2=0x7fad14066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 -- 172.21.15.87:0/691445377 shutdown_connections 2024-04-25T10:20:24.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.291+0000 7fad1a000640 1 -- 172.21.15.87:0/691445377 wait complete. 2024-04-25T10:20:24.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 Processor -- start 2024-04-25T10:20:24.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 -- start start 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad14156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140ae020 0x7fad1415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad140cf930 0x7fad14157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1415e6b0 con 0x7fad140ae020 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1406e130 con 0x7fad140cf930 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad14161790 con 0x7fad14070c20 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140ae020 0x7fad1415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140ae020 0x7fad1415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34432/0 (socket says 172.21.15.87:34432) 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 -- 172.21.15.87:0/766953546 learned_addr learned my addr 172.21.15.87:0/766953546 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad13400640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad14156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad13e00640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad140cf930 0x7fad14157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 -- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 msgr2=0x7fad14156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad14156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 -- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad140cf930 msgr2=0x7fad14157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad140cf930 0x7fad14157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 -- 172.21.15.87:0/766953546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1415c440 con 0x7fad140ae020 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad12a00640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140ae020 0x7fad1415bd10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e8b0 tx=0x7fad0000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad0b400640 1 -- 172.21.15.87:0/766953546 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00017070 con 0x7fad140ae020 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad0b400640 1 -- 172.21.15.87:0/766953546 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0000c940 con 0x7fad140ae020 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1415c720 con 0x7fad140ae020 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad0b400640 1 -- 172.21.15.87:0/766953546 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0000cd10 con 0x7fad140ae020 2024-04-25T10:20:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.295+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad140cc490 con 0x7fad140ae020 2024-04-25T10:20:24.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.299+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1404ef30 con 0x7fad140ae020 2024-04-25T10:20:24.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.299+0000 7fad0b400640 1 -- 172.21.15.87:0/766953546 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad0001a070 con 0x7fad140ae020 2024-04-25T10:20:24.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.299+0000 7fad0b400640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.299+0000 7fad0b400640 1 -- 172.21.15.87:0/766953546 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad00085330 con 0x7fad140ae020 2024-04-25T10:20:24.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.299+0000 7fad13400640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.299+0000 7fad13400640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fad040ae000 tx=0x7fad040a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.303+0000 7fad0b400640 1 -- 172.21.15.87:0/766953546 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad000563d0 con 0x7fad140ae020 2024-04-25T10:20:24.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad140cfdb0 con 0x7fad140ae020 2024-04-25T10:20:24.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad0b400640 1 -- 172.21.15.87:0/766953546 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16598 (secure 0 0 0) 0x7fad0005c0d0 con 0x7fad140ae020 2024-04-25T10:20:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:24.462 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:38.756098+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:35.484354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:19:36.111171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facf4066f60 msgr2=0x7facf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fad040ae000 tx=0x7fad040a7070 comp rx=0 tx=0).stop 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140ae020 msgr2=0x7fad1415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140ae020 0x7fad1415bd10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e8b0 tx=0x7fad0000ed70 comp rx=0 tx=0).stop 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 shutdown_connections 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad140cf930 0x7fad14157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad140ae020 0x7fad1415bd10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 --2- 172.21.15.87:0/766953546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad14070c20 0x7fad14156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 >> 172.21.15.87:0/766953546 conn(0x7fad14066b70 msgr2=0x7fad140ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 shutdown_connections 2024-04-25T10:20:24.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.459+0000 7fad1a000640 1 -- 172.21.15.87:0/766953546 wait complete. 2024-04-25T10:20:24.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:19:35.484354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:19:36.111171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:24.483 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:24.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f051a800640 1 Processor -- start 2024-04-25T10:20:24.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f051a800640 1 -- start start 2024-04-25T10:20:24.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f051a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f051a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05140d1fb0 0x7f05140b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f051a800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051406b1d0 con 0x7f05140cfb30 2024-04-25T10:20:24.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f051a800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051415e110 con 0x7f05140d1fb0 2024-04-25T10:20:24.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f051a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051406b470 con 0x7f051415dd30 2024-04-25T10:20:24.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05140d1fb0 0x7f05140b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05140d1fb0 0x7f05140b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34440/0 (socket says 172.21.15.87:34440) 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513e00640 1 -- 172.21.15.87:0/3120350313 learned_addr learned my addr 172.21.15.87:0/3120350313 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0512a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f05140cfb30 0x7f05140d18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60370/0 (socket says 172.21.15.87:60370) 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513400640 1 --2- 172.21.15.87:0/3120350313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513e00640 1 -- 172.21.15.87:0/3120350313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 msgr2=0x7f051406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513e00640 1 --2- 172.21.15.87:0/3120350313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513e00640 1 -- 172.21.15.87:0/3120350313 >> v1:172.21.15.87:6789/0 conn(0x7f05140cfb30 legacy=0x7f05140d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0513e00640 1 -- 172.21.15.87:0/3120350313 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051406b750 con 0x7f05140d1fb0 2024-04-25T10:20:24.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.619+0000 7f0512000640 1 -- 172.21.15.87:0/3120350313 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963006653 0 0) 0x7f051406b1d0 con 0x7f05140cfb30 2024-04-25T10:20:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f0513e00640 1 --2- 172.21.15.87:0/3120350313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140d1fb0 0x7f05140b1930 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f050c009e50 tx=0x7f050c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dca74979665645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f0512000640 1 -- 172.21.15.87:0/3120350313 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c0a57f0 con 0x7f05140d1fb0 2024-04-25T10:20:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f0512000640 1 -- 172.21.15.87:0/3120350313 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c0a5990 con 0x7f05140d1fb0 2024-04-25T10:20:24.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f0512000640 1 -- 172.21.15.87:0/3120350313 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c0a5d60 con 0x7f05140d1fb0 2024-04-25T10:20:24.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 -- 172.21.15.87:0/3120350313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140d1fb0 msgr2=0x7f05140b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 --2- 172.21.15.87:0/3120350313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140d1fb0 0x7f05140b1930 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f050c009e50 tx=0x7f050c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:20:24.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 -- 172.21.15.87:0/3120350313 shutdown_connections 2024-04-25T10:20:24.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 --2- 172.21.15.87:0/3120350313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140d1fb0 0x7f05140b1930 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 --2- 172.21.15.87:0/3120350313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 -- 172.21.15.87:0/3120350313 >> 172.21.15.87:0/3120350313 conn(0x7f051409a9d0 msgr2=0x7f051409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 -- 172.21.15.87:0/3120350313 shutdown_connections 2024-04-25T10:20:24.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.623+0000 7f051a800640 1 -- 172.21.15.87:0/3120350313 wait complete. 2024-04-25T10:20:24.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 Processor -- start 2024-04-25T10:20:24.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 -- start start 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140cfb30 0x7f0514155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05140d1fb0 0x7f051415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051416ab60 con 0x7f05140cfb30 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b2890 con 0x7f05140d1fb0 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051416ace0 con 0x7f051415dd30 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140cfb30 0x7f0514155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140cfb30 0x7f0514155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34448/0 (socket says 172.21.15.87:34448) 2024-04-25T10:20:24.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 -- 172.21.15.87:0/2848638185 learned_addr learned my addr 172.21.15.87:0/2848638185 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0512a00640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05140d1fb0 0x7f051415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513e00640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 -- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 msgr2=0x7f051415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051415c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 -- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05140d1fb0 msgr2=0x7f051415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05140d1fb0 0x7f051415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 -- 172.21.15.87:0/2848638185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051415b8a0 con 0x7f05140cfb30 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f0513400640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140cfb30 0x7f0514155ea0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f050400d5f0 tx=0x7f050400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f050b400640 1 -- 172.21.15.87:0/2848638185 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05040125f0 con 0x7f05140cfb30 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f050b400640 1 -- 172.21.15.87:0/2848638185 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0504019070 con 0x7f05140cfb30 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051415bbe0 con 0x7f05140cfb30 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f050b400640 1 -- 172.21.15.87:0/2848638185 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05040129d0 con 0x7f05140cfb30 2024-04-25T10:20:24.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.627+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051415c420 con 0x7f05140cfb30 2024-04-25T10:20:24.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.631+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051404ef30 con 0x7f05140cfb30 2024-04-25T10:20:24.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.631+0000 7f050b400640 1 -- 172.21.15.87:0/2848638185 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0504012b70 con 0x7f05140cfb30 2024-04-25T10:20:24.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.631+0000 7f050b400640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f4066f60 0x7f04f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.631+0000 7f050b400640 1 -- 172.21.15.87:0/2848638185 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0504085550 con 0x7f05140cfb30 2024-04-25T10:20:24.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.631+0000 7f0512a00640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f4066f60 0x7f04f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.631+0000 7f0512a00640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f4066f60 0x7f04f4069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f05140b0310 tx=0x7f0500007b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.635+0000 7f050b400640 1 -- 172.21.15.87:0/2848638185 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f051415bbe0 con 0x7f05140cfb30 2024-04-25T10:20:24.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f051406b180 con 0x7f05140cfb30 2024-04-25T10:20:24.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f050b400640 1 -- 172.21.15.87:0/2848638185 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05040564d0 con 0x7f05140cfb30 2024-04-25T10:20:24.818 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:20:24.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f4066f60 msgr2=0x7f04f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f4066f60 0x7f04f4069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f05140b0310 tx=0x7f0500007b60 comp rx=0 tx=0).stop 2024-04-25T10:20:24.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140cfb30 msgr2=0x7f0514155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140cfb30 0x7f0514155ea0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f050400d5f0 tx=0x7f050400dab0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 shutdown_connections 2024-04-25T10:20:24.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f4066f60 0x7f04f4069420 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f051415dd30 0x7f051415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05140d1fb0 0x7f051415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 --2- 172.21.15.87:0/2848638185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05140cfb30 0x7f0514155ea0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 >> 172.21.15.87:0/2848638185 conn(0x7f051409a9d0 msgr2=0x7f05140cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 shutdown_connections 2024-04-25T10:20:24.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.795+0000 7f051a800640 1 -- 172.21.15.87:0/2848638185 wait complete. 2024-04-25T10:20:24.820 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:20:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62eec00640 1 Processor -- start 2024-04-25T10:20:24.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62eec00640 1 -- start start 2024-04-25T10:20:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62eec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62eec00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62e8070350 0x7f62e815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62eec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e815b080 con 0x7f62e815bac0 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62eec00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e815adb0 con 0x7f62e8070350 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62eec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e815eb60 con 0x7f62e80aef80 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62e7400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62e8070350 0x7f62e815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62e7400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62e8070350 0x7f62e815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34454/0 (socket says 172.21.15.87:34454) 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.955+0000 7f62e7400640 1 -- 172.21.15.87:0/3130143505 learned_addr learned my addr 172.21.15.87:0/3130143505 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e7e00640 1 --2- 172.21.15.87:0/3130143505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e7400640 1 -- 172.21.15.87:0/3130143505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 msgr2=0x7f62e815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e7400640 1 --2- 172.21.15.87:0/3130143505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e7400640 1 -- 172.21.15.87:0/3130143505 >> v1:172.21.15.169:6789/0 conn(0x7f62e815bac0 legacy=0x7f62e80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e7400640 1 -- 172.21.15.87:0/3130143505 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e815ee40 con 0x7f62e8070350 2024-04-25T10:20:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e7400640 1 --2- 172.21.15.87:0/3130143505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e8070350 0x7f62e815b580 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009870 tx=0x7f62d801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c08c756789c5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e6a00640 1 -- 172.21.15.87:0/3130143505 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80a9400 con 0x7f62e8070350 2024-04-25T10:20:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e6a00640 1 -- 172.21.15.87:0/3130143505 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d80a95a0 con 0x7f62e8070350 2024-04-25T10:20:24.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.959+0000 7f62e6a00640 1 -- 172.21.15.87:0/3130143505 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80a9970 con 0x7f62e8070350 2024-04-25T10:20:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- 172.21.15.87:0/3130143505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e8070350 msgr2=0x7f62e815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 --2- 172.21.15.87:0/3130143505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e8070350 0x7f62e815b580 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009870 tx=0x7f62d801c560 comp rx=0 tx=0).stop 2024-04-25T10:20:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- 172.21.15.87:0/3130143505 shutdown_connections 2024-04-25T10:20:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 --2- 172.21.15.87:0/3130143505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e8070350 0x7f62e815b580 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 --2- 172.21.15.87:0/3130143505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- 172.21.15.87:0/3130143505 >> 172.21.15.87:0/3130143505 conn(0x7f62e80669a0 msgr2=0x7f62e8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- 172.21.15.87:0/3130143505 shutdown_connections 2024-04-25T10:20:24.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- 172.21.15.87:0/3130143505 wait complete. 2024-04-25T10:20:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 Processor -- start 2024-04-25T10:20:24.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- start start 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e8070350 0x7f62e8159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e80af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e815bac0 0x7f62e806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8167820 con 0x7f62e815bac0 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80b3ae0 con 0x7f62e8070350 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62eec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e81679a0 con 0x7f62e80aef80 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e815bac0 0x7f62e806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e815bac0 0x7f62e806c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34462/0 (socket says 172.21.15.87:34462) 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 -- 172.21.15.87:0/2841074370 learned_addr learned my addr 172.21.15.87:0/2841074370 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:24.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62e7e00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e8070350 0x7f62e8159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62e7400640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e80af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 -- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 msgr2=0x7f62e80af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e80af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 -- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e8070350 msgr2=0x7f62e8159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e8070350 0x7f62e8159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 -- 172.21.15.87:0/2841074370 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e815deb0 con 0x7f62e815bac0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62e7400640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e80af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62ecc00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e815bac0 0x7f62e806c9f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c760 tx=0x7f62e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62dfe00640 1 -- 172.21.15.87:0/2841074370 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0003b80 con 0x7f62e815bac0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62dfe00640 1 -- 172.21.15.87:0/2841074370 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e0003d20 con 0x7f62e815bac0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.963+0000 7f62dfe00640 1 -- 172.21.15.87:0/2841074370 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e000aa30 con 0x7f62e815bac0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e81595f0 con 0x7f62e815bac0 2024-04-25T10:20:24.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e815dbd0 con 0x7f62e815bac0 2024-04-25T10:20:24.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e804ef30 con 0x7f62e815bac0 2024-04-25T10:20:24.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62dfe00640 1 -- 172.21.15.87:0/2841074370 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62e000abd0 con 0x7f62e815bac0 2024-04-25T10:20:24.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62dfe00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62c4066f60 0x7f62c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62dfe00640 1 -- 172.21.15.87:0/2841074370 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62e0084e70 con 0x7f62e815bac0 2024-04-25T10:20:24.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62e7e00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62c4066f60 0x7f62c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.967+0000 7f62e7e00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62c4066f60 0x7f62c4069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009b60 tx=0x7f62d0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:24.971+0000 7f62dfe00640 1 -- 172.21.15.87:0/2841074370 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62e0055de0 con 0x7f62e815bac0 2024-04-25T10:20:25.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.123+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62e81598d0 con 0x7f62e815bac0 2024-04-25T10:20:25.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.123+0000 7f62dfe00640 1 -- 172.21.15.87:0/2841074370 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62e00556e0 con 0x7f62e815bac0 2024-04-25T10:20:25.127 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62c4066f60 msgr2=0x7f62c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62c4066f60 0x7f62c4069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009b60 tx=0x7f62d0009190 comp rx=0 tx=0).stop 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e815bac0 msgr2=0x7f62e806c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e815bac0 0x7f62e806c9f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c760 tx=0x7f62e000cc20 comp rx=0 tx=0).stop 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 shutdown_connections 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f62c4066f60 0x7f62c4069420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62e815bac0 0x7f62e806c9f0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62e80aef80 0x7f62e80af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 --2- 172.21.15.87:0/2841074370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e8070350 0x7f62e8159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 >> 172.21.15.87:0/2841074370 conn(0x7f62e80669a0 msgr2=0x7f62e80ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 shutdown_connections 2024-04-25T10:20:25.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.127+0000 7f62eec00640 1 -- 172.21.15.87:0/2841074370 wait complete. 2024-04-25T10:20:25.148 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:20:25.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.283+0000 7f0072e00640 1 Processor -- start 2024-04-25T10:20:25.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.283+0000 7f0072e00640 1 -- start start 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0072e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f006c070350 0x7f006c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0072e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f006c0b31c0 0x7f006c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0072e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c15b080 con 0x7f006c0aef80 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0072e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c15adb0 con 0x7f006c070350 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0072e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c15eb60 con 0x7f006c0b31c0 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0063400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f006c070350 0x7f006c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0063400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f006c070350 0x7f006c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34468/0 (socket says 172.21.15.87:34468) 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0063400640 1 -- 172.21.15.87:0/3347549161 learned_addr learned my addr 172.21.15.87:0/3347549161 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f006be00640 1 --1- 172.21.15.87:0/3347549161 >> v1:172.21.15.87:6789/0 conn(0x7f006c0aef80 0x7f006c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60384/0 (socket says 172.21.15.87:60384) 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0070e00640 1 --2- 172.21.15.87:0/3347549161 >> v2:172.21.15.169:3300/0 conn(0x7f006c0b31c0 0x7f006c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0063400640 1 -- 172.21.15.87:0/3347549161 >> v2:172.21.15.169:3300/0 conn(0x7f006c0b31c0 msgr2=0x7f006c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0063400640 1 --2- 172.21.15.87:0/3347549161 >> v2:172.21.15.169:3300/0 conn(0x7f006c0b31c0 0x7f006c0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0063400640 1 -- 172.21.15.87:0/3347549161 >> v1:172.21.15.87:6789/0 conn(0x7f006c0aef80 legacy=0x7f006c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.287+0000 7f0063400640 1 -- 172.21.15.87:0/3347549161 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c15ee40 con 0x7f006c070350 2024-04-25T10:20:25.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0063400640 1 --2- 172.21.15.87:0/3347549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c070350 0x7f006c15e480 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f006c0d0750 tx=0x7f005401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58921509daff9457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f006b400640 1 -- 172.21.15.87:0/3347549161 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00540ab430 con 0x7f006c070350 2024-04-25T10:20:25.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f006b400640 1 -- 172.21.15.87:0/3347549161 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00540b0070 con 0x7f006c070350 2024-04-25T10:20:25.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f006b400640 1 -- 172.21.15.87:0/3347549161 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00540ab7f0 con 0x7f006c070350 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- 172.21.15.87:0/3347549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c070350 msgr2=0x7f006c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 --2- 172.21.15.87:0/3347549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c070350 0x7f006c15e480 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f006c0d0750 tx=0x7f005401c520 comp rx=0 tx=0).stop 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- 172.21.15.87:0/3347549161 shutdown_connections 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 --2- 172.21.15.87:0/3347549161 >> v2:172.21.15.169:3300/0 conn(0x7f006c0b31c0 0x7f006c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 --2- 172.21.15.87:0/3347549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c070350 0x7f006c15e480 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- 172.21.15.87:0/3347549161 >> 172.21.15.87:0/3347549161 conn(0x7f006c0669a0 msgr2=0x7f006c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- 172.21.15.87:0/3347549161 shutdown_connections 2024-04-25T10:20:25.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- 172.21.15.87:0/3347549161 wait complete. 2024-04-25T10:20:25.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 Processor -- start 2024-04-25T10:20:25.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- start start 2024-04-25T10:20:25.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f006c070350 0x7f006c14b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f006c0aef80 0x7f006c0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c0b31c0 0x7f006c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c1679a0 con 0x7f006c0b31c0 2024-04-25T10:20:25.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c167820 con 0x7f006c0aef80 2024-04-25T10:20:25.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.291+0000 7f0072e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c167b20 con 0x7f006c070350 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c0b31c0 0x7f006c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c0b31c0 0x7f006c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34470/0 (socket says 172.21.15.87:34470) 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 -- 172.21.15.87:0/2778530949 learned_addr learned my addr 172.21.15.87:0/2778530949 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f006be00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f006c070350 0x7f006c14b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0063400640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f006c0aef80 0x7f006c0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 -- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f006c070350 msgr2=0x7f006c14b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f006c070350 0x7f006c14b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 -- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f006c0aef80 msgr2=0x7f006c0b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f006c0aef80 0x7f006c0b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 -- 172.21.15.87:0/2778530949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c14b890 con 0x7f006c0b31c0 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0063400640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f006c0aef80 0x7f006c0b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f006be00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f006c070350 0x7f006c14b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:25.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0070e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c0b31c0 0x7f006c0ad840 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0064007e60 tx=0x7f006400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0068c00640 1 -- 172.21.15.87:0/2778530949 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064014420 con 0x7f006c0b31c0 2024-04-25T10:20:25.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0068c00640 1 -- 172.21.15.87:0/2778530949 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0064019070 con 0x7f006c0b31c0 2024-04-25T10:20:25.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0068c00640 1 -- 172.21.15.87:0/2778530949 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064014820 con 0x7f006c0b31c0 2024-04-25T10:20:25.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c0b1200 con 0x7f006c0b31c0 2024-04-25T10:20:25.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c1560d0 con 0x7f006c0b31c0 2024-04-25T10:20:25.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c04ef30 con 0x7f006c0b31c0 2024-04-25T10:20:25.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0068c00640 1 -- 172.21.15.87:0/2778530949 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00640149c0 con 0x7f006c0b31c0 2024-04-25T10:20:25.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.295+0000 7f0068c00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0048066f60 0x7f0048069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.299+0000 7f0068c00640 1 -- 172.21.15.87:0/2778530949 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00640858e0 con 0x7f006c0b31c0 2024-04-25T10:20:25.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.299+0000 7f006be00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0048066f60 0x7f0048069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.299+0000 7f006be00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0048066f60 0x7f0048069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f006c1fa700 tx=0x7f005c007d90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.303+0000 7f0068c00640 1 -- 172.21.15.87:0/2778530949 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00640568e0 con 0x7f006c0b31c0 2024-04-25T10:20:25.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f006c0b2fa0 con 0x7f006c0b31c0 2024-04-25T10:20:25.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0068c00640 1 -- 172.21.15.87:0/2778530949 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00640561e0 con 0x7f006c0b31c0 2024-04-25T10:20:25.458 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:20:25.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0048066f60 msgr2=0x7f0048069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0048066f60 0x7f0048069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f006c1fa700 tx=0x7f005c007d90 comp rx=0 tx=0).stop 2024-04-25T10:20:25.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c0b31c0 msgr2=0x7f006c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c0b31c0 0x7f006c0ad840 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0064007e60 tx=0x7f006400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 shutdown_connections 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0048066f60 0x7f0048069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c0b31c0 0x7f006c0ad840 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f006c0aef80 0x7f006c0b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 --2- 172.21.15.87:0/2778530949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f006c070350 0x7f006c14b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 >> 172.21.15.87:0/2778530949 conn(0x7f006c0669a0 msgr2=0x7f006c154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 shutdown_connections 2024-04-25T10:20:25.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.455+0000 7f0072e00640 1 -- 172.21.15.87:0/2778530949 wait complete. 2024-04-25T10:20:25.479 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:25.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.611+0000 7fe229400640 1 Processor -- start 2024-04-25T10:20:25.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.611+0000 7fe229400640 1 -- start start 2024-04-25T10:20:25.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe229400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2240cf930 0x7fe2240cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe229400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe2240ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe229400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2240d0250 con 0x7fe224070c20 2024-04-25T10:20:25.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe229400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22415bb90 con 0x7fe2240cf930 2024-04-25T10:20:25.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe229400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240ad840 con 0x7fe2240ae020 2024-04-25T10:20:25.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe222800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2240cf930 0x7fe2240cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe222800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2240cf930 0x7fe2240cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34472/0 (socket says 172.21.15.87:34472) 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe222800640 1 -- 172.21.15.87:0/4236052218 learned_addr learned my addr 172.21.15.87:0/4236052218 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe223200640 1 --2- 172.21.15.87:0/4236052218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe2240ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe222800640 1 -- 172.21.15.87:0/4236052218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 msgr2=0x7fe2240ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe222800640 1 --2- 172.21.15.87:0/4236052218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe2240ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe222800640 1 -- 172.21.15.87:0/4236052218 >> v1:172.21.15.169:6789/0 conn(0x7fe224070c20 legacy=0x7fe22415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe222800640 1 -- 172.21.15.87:0/4236052218 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22415e9f0 con 0x7fe2240cf930 2024-04-25T10:20:25.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.615+0000 7fe223200640 1 --2- 172.21.15.87:0/4236052218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe2240ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:25.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe222800640 1 --2- 172.21.15.87:0/4236052218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2240cf930 0x7fe2240cfd10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fe214009e50 tx=0x7fe21401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9882a544f4488d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe221e00640 1 -- 172.21.15.87:0/4236052218 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140a58d0 con 0x7fe2240cf930 2024-04-25T10:20:25.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe221e00640 1 -- 172.21.15.87:0/4236052218 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2140a5a70 con 0x7fe2240cf930 2024-04-25T10:20:25.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe221e00640 1 -- 172.21.15.87:0/4236052218 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140a5e40 con 0x7fe2240cf930 2024-04-25T10:20:25.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- 172.21.15.87:0/4236052218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2240cf930 msgr2=0x7fe2240cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 --2- 172.21.15.87:0/4236052218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2240cf930 0x7fe2240cfd10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fe214009e50 tx=0x7fe21401ca00 comp rx=0 tx=0).stop 2024-04-25T10:20:25.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- 172.21.15.87:0/4236052218 shutdown_connections 2024-04-25T10:20:25.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 --2- 172.21.15.87:0/4236052218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe2240ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 --2- 172.21.15.87:0/4236052218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2240cf930 0x7fe2240cfd10 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- 172.21.15.87:0/4236052218 >> 172.21.15.87:0/4236052218 conn(0x7fe224066b70 msgr2=0x7fe224066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- 172.21.15.87:0/4236052218 shutdown_connections 2024-04-25T10:20:25.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- 172.21.15.87:0/4236052218 wait complete. 2024-04-25T10:20:25.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 Processor -- start 2024-04-25T10:20:25.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- start start 2024-04-25T10:20:25.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe224070c20 0x7fe224160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe224160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2240cf930 0x7fe2241412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22416a8e0 con 0x7fe224070c20 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22406e130 con 0x7fe2240cf930 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe229400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22416aa60 con 0x7fe2240ae020 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.619+0000 7fe222800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe224070c20 0x7fe224160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe224070c20 0x7fe224160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34480/0 (socket says 172.21.15.87:34480) 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 -- 172.21.15.87:0/1562590452 learned_addr learned my addr 172.21.15.87:0/1562590452 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe223200640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2240cf930 0x7fe2241412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe219e00640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe224160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 -- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 msgr2=0x7fe224160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe224160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 -- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2240cf930 msgr2=0x7fe2241412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2240cf930 0x7fe2241412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 -- 172.21.15.87:0/1562590452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22415de10 con 0x7fe224070c20 2024-04-25T10:20:25.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe222800640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe224070c20 0x7fe224160330 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fe2140af040 tx=0x7fe2140197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe21b400640 1 -- 172.21.15.87:0/1562590452 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140b8880 con 0x7fe224070c20 2024-04-25T10:20:25.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe21b400640 1 -- 172.21.15.87:0/1562590452 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2140be070 con 0x7fe224070c20 2024-04-25T10:20:25.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe21b400640 1 -- 172.21.15.87:0/1562590452 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140b8c80 con 0x7fe224070c20 2024-04-25T10:20:25.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe224153180 con 0x7fe224070c20 2024-04-25T10:20:25.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe224153740 con 0x7fe224070c20 2024-04-25T10:20:25.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22404ef30 con 0x7fe224070c20 2024-04-25T10:20:25.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe21b400640 1 -- 172.21.15.87:0/1562590452 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2140c3460 con 0x7fe224070c20 2024-04-25T10:20:25.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.623+0000 7fe21b400640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe200066f60 0x7fe200069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.627+0000 7fe21b400640 1 -- 172.21.15.87:0/1562590452 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe214120320 con 0x7fe224070c20 2024-04-25T10:20:25.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.627+0000 7fe219e00640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe200066f60 0x7fe200069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.627+0000 7fe219e00640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe200066f60 0x7fe200069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe2240b13d0 tx=0x7fe20c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.627+0000 7fe21b400640 1 -- 172.21.15.87:0/1562590452 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe214016620 con 0x7fe224070c20 2024-04-25T10:20:25.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.819+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe22415e080 con 0x7fe224070c20 2024-04-25T10:20:25.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe21b400640 1 -- 172.21.15.87:0/1562590452 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe2140f0d60 con 0x7fe224070c20 2024-04-25T10:20:25.824 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:25.825 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:25.825 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:25.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe200066f60 msgr2=0x7fe200069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe200066f60 0x7fe200069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe2240b13d0 tx=0x7fe20c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:25.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe224070c20 msgr2=0x7fe224160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe224070c20 0x7fe224160330 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fe2140af040 tx=0x7fe2140197a0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 shutdown_connections 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe200066f60 0x7fe200069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2240cf930 0x7fe2241412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2240ae020 0x7fe224160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 --2- 172.21.15.87:0/1562590452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe224070c20 0x7fe224160330 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 >> 172.21.15.87:0/1562590452 conn(0x7fe224066b70 msgr2=0x7fe224154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 shutdown_connections 2024-04-25T10:20:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.823+0000 7fe229400640 1 -- 172.21.15.87:0/1562590452 wait complete. 2024-04-25T10:20:25.845 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:25.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.975+0000 7f6dc2000640 1 Processor -- start 2024-04-25T10:20:25.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc2000640 1 -- start start 2024-04-25T10:20:25.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6dbc0ce070 0x7f6dbc0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc2000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc15f5e0 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc2000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b3060 con 0x7f6dbc0ce070 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc15f8c0 con 0x7f6dbc1451d0 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6dbc0ce070 0x7f6dbc0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6dbc0ce070 0x7f6dbc0b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:48004/0 (socket says 172.21.15.87:48004) 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbbe00640 1 -- 172.21.15.87:0/875556273 learned_addr learned my addr 172.21.15.87:0/875556273 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc1600640 1 --1- 172.21.15.87:0/875556273 >> v1:172.21.15.169:6789/0 conn(0x7f6dbc0b2c00 0x7f6dbc0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:59342/0 (socket says 172.21.15.87:59342) 2024-04-25T10:20:25.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbbe00640 1 -- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 msgr2=0x7f6dbc0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbbe00640 1 --2- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc0cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbbe00640 1 -- 172.21.15.87:0/875556273 >> v1:172.21.15.169:6789/0 conn(0x7f6dbc0b2c00 legacy=0x7f6dbc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbbe00640 1 -- 172.21.15.87:0/875556273 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc15fba0 con 0x7f6dbc0ce070 2024-04-25T10:20:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dbb400640 1 -- 172.21.15.87:0/875556273 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341056452 0 0) 0x7f6dbc15f5e0 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.979+0000 7f6dc0c00640 1 --2- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dbbe00640 1 --2- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 0x7f6dbc0b26c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009870 tx=0x7f6da801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=710241bc524c3227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dbb400640 1 -- 172.21.15.87:0/875556273 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80b0070 con 0x7f6dbc0ce070 2024-04-25T10:20:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dbb400640 1 -- 172.21.15.87:0/875556273 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da80ab600 con 0x7f6dbc0ce070 2024-04-25T10:20:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dbb400640 1 -- 172.21.15.87:0/875556273 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80ab9d0 con 0x7f6dbc0ce070 2024-04-25T10:20:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 -- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 msgr2=0x7f6dbc0b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 --2- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 0x7f6dbc0b26c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009870 tx=0x7f6da801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 -- 172.21.15.87:0/875556273 shutdown_connections 2024-04-25T10:20:25.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 --2- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 0x7f6dbc0b26c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 --2- 172.21.15.87:0/875556273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 -- 172.21.15.87:0/875556273 >> 172.21.15.87:0/875556273 conn(0x7f6dbc074310 msgr2=0x7f6dbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 -- 172.21.15.87:0/875556273 shutdown_connections 2024-04-25T10:20:25.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 -- 172.21.15.87:0/875556273 wait complete. 2024-04-25T10:20:25.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.983+0000 7f6dc2000640 1 Processor -- start 2024-04-25T10:20:25.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 -- start start 2024-04-25T10:20:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dbc0b2c00 0x7f6dbc13e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 0x7f6dbc13d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc06ee60 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc06ece0 con 0x7f6dbc0ce070 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc06e2b0 con 0x7f6dbc1451d0 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dbc0b2c00 0x7f6dbc13e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dbc0b2c00 0x7f6dbc13e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34486/0 (socket says 172.21.15.87:34486) 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 -- 172.21.15.87:0/1942754237 learned_addr learned my addr 172.21.15.87:0/1942754237 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dbbe00640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 0x7f6dbc13d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc1600640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 -- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 msgr2=0x7f6dbc15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc15cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 -- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 msgr2=0x7f6dbc13d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 0x7f6dbc13d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 -- 172.21.15.87:0/1942754237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0aef60 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc1600640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc0c00640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dbc0b2c00 0x7f6dbc13e3d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6db000ec10 tx=0x7f6db000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6db8c00640 1 -- 172.21.15.87:0/1942754237 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0017070 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6db8c00640 1 -- 172.21.15.87:0/1942754237 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db000cc30 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc15dcf0 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6db8c00640 1 -- 172.21.15.87:0/1942754237 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0013470 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc15d460 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.987+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc075120 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.991+0000 7f6db8c00640 1 -- 172.21.15.87:0/1942754237 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6db001a070 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.991+0000 7f6db8c00640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d9c066f60 0x7f6d9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.991+0000 7f6db8c00640 1 -- 172.21.15.87:0/1942754237 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6db0085260 con 0x7f6dbc0b2c00 2024-04-25T10:20:25.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.991+0000 7f6dbbe00640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d9c066f60 0x7f6d9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.991+0000 7f6dbbe00640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d9c066f60 0x7f6d9c069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6da80ad000 tx=0x7f6da8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:25.995+0000 7f6db8c00640 1 -- 172.21.15.87:0/1942754237 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6db0055cb0 con 0x7f6dbc0b2c00 2024-04-25T10:20:26.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.147+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dbc066e40 con 0x7f6dbc0b2c00 2024-04-25T10:20:26.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.147+0000 7f6db8c00640 1 -- 172.21.15.87:0/1942754237 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16598 (secure 0 0 0) 0x7f6db0059d50 con 0x7f6dbc0b2c00 2024-04-25T10:20:26.151 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:26.151 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:38.756098+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:35.484354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:19:36.111171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:26.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d9c066f60 msgr2=0x7f6d9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d9c066f60 0x7f6d9c069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6da80ad000 tx=0x7f6da8008040 comp rx=0 tx=0).stop 2024-04-25T10:20:26.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dbc0b2c00 msgr2=0x7f6dbc13e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dbc0b2c00 0x7f6dbc13e3d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6db000ec10 tx=0x7f6db000c520 comp rx=0 tx=0).stop 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 shutdown_connections 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d9c066f60 0x7f6d9c069420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dbc1451d0 0x7f6dbc15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dbc0ce070 0x7f6dbc13d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 --2- 172.21.15.87:0/1942754237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dbc0b2c00 0x7f6dbc13e3d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 >> 172.21.15.87:0/1942754237 conn(0x7f6dbc074310 msgr2=0x7f6dbc0b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 shutdown_connections 2024-04-25T10:20:26.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.151+0000 7f6dc2000640 1 -- 172.21.15.87:0/1942754237 wait complete. 2024-04-25T10:20:26.173 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:20:26.173 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:26.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9121a00640 1 Processor -- start 2024-04-25T10:20:26.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9121a00640 1 -- start start 2024-04-25T10:20:26.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9121a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f911c15bc40 0x7f911c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9121a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0b2770 con 0x7f911c0cda10 2024-04-25T10:20:26.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9121a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c06c670 con 0x7f911c0cf930 2024-04-25T10:20:26.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9121a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c06c950 con 0x7f911c15bc40 2024-04-25T10:20:26.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9112000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f911c0cda10 0x7f911c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60396/0 (socket says 172.21.15.87:60396) 2024-04-25T10:20:26.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f9112000640 1 -- 172.21.15.87:0/497184491 learned_addr learned my addr 172.21.15.87:0/497184491 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911aa00640 1 --2- 172.21.15.87:0/497184491 >> v2:172.21.15.169:3300/0 conn(0x7f911c15bc40 0x7f911c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942279672 0 0) 0x7f911c0b2770 con 0x7f911c0cda10 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9100003530 con 0x7f911c0cda10 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3503903882 0 0) 0x7f9100003530 con 0x7f911c0cda10 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 >> v2:172.21.15.169:3300/0 conn(0x7f911c15bc40 msgr2=0x7f911c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 --2- 172.21.15.87:0/497184491 >> v2:172.21.15.169:3300/0 conn(0x7f911c15bc40 0x7f911c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 >> v1:172.21.15.169:6789/0 conn(0x7f911c0cf930 legacy=0x7f911c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c06cc30 con 0x7f911c0cda10 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f9104001680 con 0x7f911c0cda10 2024-04-25T10:20:26.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 >> v1:172.21.15.87:6789/0 conn(0x7f911c0cda10 legacy=0x7f911c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91000039c0 0x7f9100003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100004810 0x7f9100004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000051d0 0x7f910000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c06c950 con 0x7f91000039c0 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910000c170 con 0x7f9100004810 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.307+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910000c370 con 0x7f91000051d0 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911aa00640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91000039c0 0x7f9100003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f9112000640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100004810 0x7f9100004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911b400640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000051d0 0x7f910000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911aa00640 1 -- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000051d0 msgr2=0x7f910000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911aa00640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000051d0 0x7f910000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911aa00640 1 -- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100004810 msgr2=0x7f9100004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911aa00640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100004810 0x7f9100004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911aa00640 1 -- 172.21.15.87:0/497184491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100003530 con 0x7f91000039c0 2024-04-25T10:20:26.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911b400640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000051d0 0x7f910000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:26.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911b400640 1 -- 172.21.15.87:0/497184491 reap_dead start 2024-04-25T10:20:26.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911aa00640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91000039c0 0x7f9100003de0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f910c009ae0 tx=0x7f910c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caaaec480c6678a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c0096b0 con 0x7f91000039c0 2024-04-25T10:20:26.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c0a6450 con 0x7f91000039c0 2024-04-25T10:20:26.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f911a000640 1 -- 172.21.15.87:0/497184491 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c0a6840 con 0x7f91000039c0 2024-04-25T10:20:26.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f9121a00640 1 -- 172.21.15.87:0/497184491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91000039c0 msgr2=0x7f9100003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.311+0000 7f9121a00640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91000039c0 0x7f9100003de0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f910c009ae0 tx=0x7f910c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:20:26.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- 172.21.15.87:0/497184491 shutdown_connections 2024-04-25T10:20:26.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 --2- 172.21.15.87:0/497184491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91000039c0 0x7f9100003de0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- 172.21.15.87:0/497184491 >> 172.21.15.87:0/497184491 conn(0x7f911c066b70 msgr2=0x7f911c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:26.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- 172.21.15.87:0/497184491 shutdown_connections 2024-04-25T10:20:26.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- 172.21.15.87:0/497184491 wait complete. 2024-04-25T10:20:26.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 Processor -- start 2024-04-25T10:20:26.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- start start 2024-04-25T10:20:26.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000039c0 0x7f911c1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1580b0 0x7f911c1f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c1f93b0 0x7f911c14a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0d00d0 con 0x7f911c1580b0 2024-04-25T10:20:26.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0cff50 con 0x7f911c1f93b0 2024-04-25T10:20:26.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0d0250 con 0x7f91000039c0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1580b0 0x7f911c1f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1580b0 0x7f911c1f8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34502/0 (socket says 172.21.15.87:34502) 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 -- 172.21.15.87:0/2101949833 learned_addr learned my addr 172.21.15.87:0/2101949833 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f911aa00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000039c0 0x7f911c1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f911b400640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c1f93b0 0x7f911c14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 -- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000039c0 msgr2=0x7f911c1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000039c0 0x7f911c1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 -- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c1f93b0 msgr2=0x7f911c14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c1f93b0 0x7f911c14a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 -- 172.21.15.87:0/2101949833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c156fb0 con 0x7f911c1580b0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f911aa00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000039c0 0x7f911c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9112000640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1580b0 0x7f911c1f8e70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f91040072c0 tx=0x7f9104007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9113400640 1 -- 172.21.15.87:0/2101949833 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910401a070 con 0x7f911c1580b0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9113400640 1 -- 172.21.15.87:0/2101949833 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910400f960 con 0x7f911c1580b0 2024-04-25T10:20:26.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c157290 con 0x7f911c1580b0 2024-04-25T10:20:26.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.315+0000 7f9113400640 1 -- 172.21.15.87:0/2101949833 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400fd50 con 0x7f911c1580b0 2024-04-25T10:20:26.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.319+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c157850 con 0x7f911c1580b0 2024-04-25T10:20:26.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.319+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c04ef30 con 0x7f911c1580b0 2024-04-25T10:20:26.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.319+0000 7f9113400640 1 -- 172.21.15.87:0/2101949833 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f910401d360 con 0x7f911c1580b0 2024-04-25T10:20:26.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.319+0000 7f9113400640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f90f8066f90 0x7f90f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.319+0000 7f9113400640 1 -- 172.21.15.87:0/2101949833 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9104084ab0 con 0x7f911c1580b0 2024-04-25T10:20:26.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.319+0000 7f911aa00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f90f8066f90 0x7f90f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.319+0000 7f911aa00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f90f8066f90 0x7f90f8069450 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f910c0ad000 tx=0x7f910c0046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.323+0000 7f9113400640 1 -- 172.21.15.87:0/2101949833 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9104055a20 con 0x7f911c1580b0 2024-04-25T10:20:26.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.507+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f911c1f9790 con 0x7f911c1580b0 2024-04-25T10:20:26.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.507+0000 7f9113400640 1 -- 172.21.15.87:0/2101949833 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9104055320 con 0x7f911c1580b0 2024-04-25T10:20:26.511 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:26.511 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:26.511 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f90f8066f90 msgr2=0x7f90f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f90f8066f90 0x7f90f8069450 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f910c0ad000 tx=0x7f910c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1580b0 msgr2=0x7f911c1f8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1580b0 0x7f911c1f8e70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f91040072c0 tx=0x7f9104007780 comp rx=0 tx=0).stop 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 shutdown_connections 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f90f8066f90 0x7f90f8069450 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c1f93b0 0x7f911c14a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1580b0 0x7f911c1f8e70 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 --2- 172.21.15.87:0/2101949833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000039c0 0x7f911c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 >> 172.21.15.87:0/2101949833 conn(0x7f911c066b70 msgr2=0x7f911c1553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 shutdown_connections 2024-04-25T10:20:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.511+0000 7f9121a00640 1 -- 172.21.15.87:0/2101949833 wait complete. 2024-04-25T10:20:26.533 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:26.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2e00640 1 Processor -- start 2024-04-25T10:20:26.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2e00640 1 -- start start 2024-04-25T10:20:26.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0d0440 0x7fb4cc0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc15bff0 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cd550 con 0x7fb4cc0d0440 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b0da0 con 0x7fb4cc15bc10 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d1000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb4cc0ccd40 0x7fb4cc0cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60398/0 (socket says 172.21.15.87:60398) 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d1000640 1 -- 172.21.15.87:0/1838782111 learned_addr learned my addr 172.21.15.87:0/1838782111 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d1a00640 1 --2- 172.21.15.87:0/1838782111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d2400640 1 --2- 172.21.15.87:0/1838782111 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0d0440 0x7fb4cc0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4cbe00640 1 -- 172.21.15.87:0/1838782111 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394997276 0 0) 0x7fb4cc15bff0 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4cbe00640 1 -- 172.21.15.87:0/1838782111 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b4003530 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d1a00640 1 -- 172.21.15.87:0/1838782111 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0d0440 msgr2=0x7fb4cc0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d1a00640 1 --2- 172.21.15.87:0/1838782111 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0d0440 0x7fb4cc0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d1a00640 1 -- 172.21.15.87:0/1838782111 >> v1:172.21.15.87:6789/0 conn(0x7fb4cc0ccd40 legacy=0x7fb4cc0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.667+0000 7fb4d1a00640 1 -- 172.21.15.87:0/1838782111 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0b1080 con 0x7fb4cc15bc10 2024-04-25T10:20:26.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d1a00640 1 --2- 172.21.15.87:0/1838782111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc0cf760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00097c0 tx=0x7fb4c001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe87d36a94ad95f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4cbe00640 1 -- 172.21.15.87:0/1838782111 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00aa620 con 0x7fb4cc15bc10 2024-04-25T10:20:26.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4cbe00640 1 -- 172.21.15.87:0/1838782111 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c00af070 con 0x7fb4cc15bc10 2024-04-25T10:20:26.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4cbe00640 1 -- 172.21.15.87:0/1838782111 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00aa9c0 con 0x7fb4cc15bc10 2024-04-25T10:20:26.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 -- 172.21.15.87:0/1838782111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 msgr2=0x7fb4cc0cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/1838782111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc0cf760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00097c0 tx=0x7fb4c001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 -- 172.21.15.87:0/1838782111 shutdown_connections 2024-04-25T10:20:26.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/1838782111 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0d0440 0x7fb4cc0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/1838782111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc0cf760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 -- 172.21.15.87:0/1838782111 >> 172.21.15.87:0/1838782111 conn(0x7fb4cc0d1750 msgr2=0x7fb4cc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:26.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 -- 172.21.15.87:0/1838782111 shutdown_connections 2024-04-25T10:20:26.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 -- 172.21.15.87:0/1838782111 wait complete. 2024-04-25T10:20:26.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 Processor -- start 2024-04-25T10:20:26.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.671+0000 7fb4d2e00640 1 -- start start 2024-04-25T10:20:26.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0ccd40 0x7fb4cc165060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0d0440 0x7fb4cc1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc06c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b2960 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b27e0 con 0x7fb4cc0d0440 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b2ae0 con 0x7fb4cc15bc10 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0ccd40 0x7fb4cc165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0ccd40 0x7fb4cc165060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34512/0 (socket says 172.21.15.87:34512) 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 -- 172.21.15.87:0/2756903597 learned_addr learned my addr 172.21.15.87:0/2756903597 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1000640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0d0440 0x7fb4cc1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2400640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 -- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 msgr2=0x7fb4cc06c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc06c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 -- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0d0440 msgr2=0x7fb4cc1575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0d0440 0x7fb4cc1575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 -- 172.21.15.87:0/2756903597 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0b6ff0 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1000640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0d0440 0x7fb4cc1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d1a00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0ccd40 0x7fb4cc165060 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb4c001ab50 tx=0x7fb4c0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4c9600640 1 -- 172.21.15.87:0/2756903597 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00aaae0 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4c9600640 1 -- 172.21.15.87:0/2756903597 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c00aac80 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4c9600640 1 -- 172.21.15.87:0/2756903597 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00be650 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc164580 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc164b40 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.675+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc075120 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.679+0000 7fb4c9600640 1 -- 172.21.15.87:0/2756903597 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4c00be7f0 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.679+0000 7fb4c9600640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4ac066f60 0x7fb4ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.679+0000 7fb4c9600640 1 -- 172.21.15.87:0/2756903597 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4c011f460 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.679+0000 7fb4d1000640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4ac066f60 0x7fb4ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.679+0000 7fb4d1000640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4ac066f60 0x7fb4ac069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc1655a0 tx=0x7fb4b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.683+0000 7fb4c9600640 1 -- 172.21.15.87:0/2756903597 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4c00c5030 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.871+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4cc158360 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.871+0000 7fb4c9600640 1 -- 172.21.15.87:0/2756903597 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb4c0016090 con 0x7fb4cc0ccd40 2024-04-25T10:20:26.874 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:26.874 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:19:37.118679+0000","modified":"2024-04-25T10:19:40.136830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7947":{"gid":7947,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/1167245113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1167245113},{"type":"v1","addr":"172.21.15.87:6836","nonce":1167245113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:20:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 71 2024-04-25T10:20:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4ac066f60 msgr2=0x7fb4ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4ac066f60 0x7fb4ac069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc1655a0 tx=0x7fb4b8009040 comp rx=0 tx=0).stop 2024-04-25T10:20:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0ccd40 msgr2=0x7fb4cc165060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0ccd40 0x7fb4cc165060 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb4c001ab50 tx=0x7fb4c0019240 comp rx=0 tx=0).stop 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 shutdown_connections 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4ac066f60 0x7fb4ac069420 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc15bc10 0x7fb4cc06c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0d0440 0x7fb4cc1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 --2- 172.21.15.87:0/2756903597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0ccd40 0x7fb4cc165060 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 >> 172.21.15.87:0/2756903597 conn(0x7fb4cc0d1750 msgr2=0x7fb4cc0ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 shutdown_connections 2024-04-25T10:20:26.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:26.875+0000 7fb4d2e00640 1 -- 172.21.15.87:0/2756903597 wait complete. 2024-04-25T10:20:26.895 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:27.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c4200640 1 Processor -- start 2024-04-25T10:20:27.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c4200640 1 -- start start 2024-04-25T10:20:27.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 0x7f97bc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c4200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f97bc0cfe70 0x7f97bc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c4200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97bc071050 con 0x7f97bc15b740 2024-04-25T10:20:27.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c4200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97bc15d040 con 0x7f97bc0cfe70 2024-04-25T10:20:27.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0712f0 con 0x7f97bc1550b0 2024-04-25T10:20:27.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c2200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f97bc15b740 0x7f97bc06d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60402/0 (socket says 172.21.15.87:60402) 2024-04-25T10:20:27.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c2200640 1 -- 172.21.15.87:0/1842936480 learned_addr learned my addr 172.21.15.87:0/1842936480 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c0e00640 1 --2- 172.21.15.87:0/1842936480 >> v2:172.21.15.169:3300/0 conn(0x7f97bc0cfe70 0x7f97bc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c1800640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 0x7f97bc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690486011 0 0) 0x7f97bc071050 con 0x7f97bc15b740 2024-04-25T10:20:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a4003530 con 0x7f97bc15b740 2024-04-25T10:20:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3891421711 0 0) 0x7f97a4003530 con 0x7f97bc15b740 2024-04-25T10:20:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 msgr2=0x7f97bc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 0x7f97bc0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 >> v2:172.21.15.169:3300/0 conn(0x7f97bc0cfe70 msgr2=0x7f97bc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 --2- 172.21.15.87:0/1842936480 >> v2:172.21.15.169:3300/0 conn(0x7f97bc0cfe70 0x7f97bc15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0715d0 con 0x7f97bc15b740 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f97b4001a00 con 0x7f97bc15b740 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 >> v1:172.21.15.87:6789/0 conn(0x7f97bc15b740 legacy=0x7f97bc06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a40039c0 0x7f97a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a4004810 0x7f97a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 0x7f97a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc15d040 con 0x7f97a40039c0 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0712f0 con 0x7f97a4004810 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a400b9d0 con 0x7f97bc1550b0 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c1800640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a40039c0 0x7f97a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c0e00640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a4004810 0x7f97a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c2200640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 0x7f97a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c1800640 1 -- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 msgr2=0x7f97a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c1800640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc1550b0 0x7f97a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c1800640 1 -- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a4004810 msgr2=0x7f97a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c1800640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a4004810 0x7f97a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c1800640 1 -- 172.21.15.87:0/1842936480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4003530 con 0x7f97a40039c0 2024-04-25T10:20:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.027+0000 7f97c0e00640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a4004810 0x7f97a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c1800640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a40039c0 0x7f97a4003de0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f97b000de10 tx=0x7f97b0020a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06ffeed4fd701eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000da60 con 0x7f97a40039c0 2024-04-25T10:20:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b00a95d0 con 0x7f97a40039c0 2024-04-25T10:20:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97bbe00640 1 -- 172.21.15.87:0/1842936480 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000d2d0 con 0x7f97a40039c0 2024-04-25T10:20:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c4200640 1 -- 172.21.15.87:0/1842936480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a40039c0 msgr2=0x7f97a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c4200640 1 --2- 172.21.15.87:0/1842936480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a40039c0 0x7f97a4003de0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f97b000de10 tx=0x7f97b0020a60 comp rx=0 tx=0).stop 2024-04-25T10:20:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c2200640 1 -- 172.21.15.87:0/1842936480 reap_dead start 2024-04-25T10:20:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c4200640 1 -- 172.21.15.87:0/1842936480 shutdown_connections 2024-04-25T10:20:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c4200640 1 -- 172.21.15.87:0/1842936480 >> 172.21.15.87:0/1842936480 conn(0x7f97bc066b70 msgr2=0x7f97bc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c4200640 1 -- 172.21.15.87:0/1842936480 shutdown_connections 2024-04-25T10:20:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.031+0000 7f97c4200640 1 -- 172.21.15.87:0/1842936480 wait complete. 2024-04-25T10:20:27.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 Processor -- start 2024-04-25T10:20:27.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 -- start start 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a40039c0 0x7f97bc1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc14d380 0x7f97bc151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97bc1524c0 0x7f97bc158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc15bcd0 con 0x7f97bc1524c0 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc15bb50 con 0x7f97a40039c0 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc15c030 con 0x7f97bc14d380 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97bc1524c0 0x7f97bc158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97bc1524c0 0x7f97bc158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34516/0 (socket says 172.21.15.87:34516) 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 -- 172.21.15.87:0/1239969387 learned_addr learned my addr 172.21.15.87:0/1239969387 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c1800640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a40039c0 0x7f97bc1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c0e00640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc14d380 0x7f97bc151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 -- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc14d380 msgr2=0x7f97bc151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc14d380 0x7f97bc151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 -- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a40039c0 msgr2=0x7f97bc1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a40039c0 0x7f97bc1580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 -- 172.21.15.87:0/1239969387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0cd200 con 0x7f97bc1524c0 2024-04-25T10:20:27.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c2200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97bc1524c0 0x7f97bc158bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f97b40013e0 tx=0x7f97b400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97b9600640 1 -- 172.21.15.87:0/1239969387 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400eb30 con 0x7f97bc1524c0 2024-04-25T10:20:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97b9600640 1 -- 172.21.15.87:0/1239969387 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b4018070 con 0x7f97bc1524c0 2024-04-25T10:20:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc0cd4e0 con 0x7f97bc1524c0 2024-04-25T10:20:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97b9600640 1 -- 172.21.15.87:0/1239969387 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4013420 con 0x7f97bc1524c0 2024-04-25T10:20:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.035+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc0cdaa0 con 0x7f97bc1524c0 2024-04-25T10:20:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.039+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97bc04ef30 con 0x7f97bc1524c0 2024-04-25T10:20:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.039+0000 7f97b9600640 1 -- 172.21.15.87:0/1239969387 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97b40135c0 con 0x7f97bc1524c0 2024-04-25T10:20:27.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.039+0000 7f97b9600640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f979c066f60 0x7f979c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.039+0000 7f97b9600640 1 -- 172.21.15.87:0/1239969387 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97b4085140 con 0x7f97bc1524c0 2024-04-25T10:20:27.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.039+0000 7f97c1800640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f979c066f60 0x7f979c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.039+0000 7f97c1800640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f979c066f60 0x7f979c069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f97b00b2000 tx=0x7f97b00ab040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.043+0000 7f97b9600640 1 -- 172.21.15.87:0/1239969387 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97b4055ae0 con 0x7f97bc1524c0 2024-04-25T10:20:27.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.199+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97bc14d760 con 0x7f97bc1524c0 2024-04-25T10:20:27.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.199+0000 7f97b9600640 1 -- 172.21.15.87:0/1239969387 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16598 (secure 0 0 0) 0x7f97b4059b80 con 0x7f97bc1524c0 2024-04-25T10:20:27.202 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:27.203 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:19:38.756098+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:35.484354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:19:36.111171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f979c066f60 msgr2=0x7f979c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f979c066f60 0x7f979c069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f97b00b2000 tx=0x7f97b00ab040 comp rx=0 tx=0).stop 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97bc1524c0 msgr2=0x7f97bc158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97bc1524c0 0x7f97bc158bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f97b40013e0 tx=0x7f97b400e530 comp rx=0 tx=0).stop 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 shutdown_connections 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f979c066f60 0x7f979c069420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97bc1524c0 0x7f97bc158bb0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97bc14d380 0x7f97bc151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 --2- 172.21.15.87:0/1239969387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a40039c0 0x7f97bc1580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 >> 172.21.15.87:0/1239969387 conn(0x7f97bc066b70 msgr2=0x7f97bc15f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:27.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 shutdown_connections 2024-04-25T10:20:27.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.203+0000 7f97c4200640 1 -- 172.21.15.87:0/1239969387 wait complete. 2024-04-25T10:20:27.225 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:20:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8aa600640 1 Processor -- start 2024-04-25T10:20:27.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8aa600640 1 -- start start 2024-04-25T10:20:27.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8aa600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe8a415de10 0x7fe8a4073d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8aa600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a41622c0 con 0x7fe8a40cf080 2024-04-25T10:20:27.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8aa600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40ceaa0 con 0x7fe8a406ec60 2024-04-25T10:20:27.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8aa600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40cdf00 con 0x7fe8a415de10 2024-04-25T10:20:27.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a9c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe8a40cf080 0x7fe8a40ca470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59678/0 (socket says 172.21.15.87:59678) 2024-04-25T10:20:27.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a9c00640 1 -- 172.21.15.87:0/3275022097 learned_addr learned my addr 172.21.15.87:0/3275022097 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:27.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a9200640 1 --2- 172.21.15.87:0/3275022097 >> v2:172.21.15.169:3300/0 conn(0x7fe8a415de10 0x7fe8a4073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a3400640 1 -- 172.21.15.87:0/3275022097 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911932493 0 0) 0x7fe8a41622c0 con 0x7fe8a40cf080 2024-04-25T10:20:27.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a3400640 1 -- 172.21.15.87:0/3275022097 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe888003530 con 0x7fe8a40cf080 2024-04-25T10:20:27.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a9200640 1 -- 172.21.15.87:0/3275022097 >> v1:172.21.15.169:6789/0 conn(0x7fe8a406ec60 legacy=0x7fe8a40ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a9200640 1 -- 172.21.15.87:0/3275022097 >> v1:172.21.15.87:6789/0 conn(0x7fe8a40cf080 legacy=0x7fe8a40ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.359+0000 7fe8a9200640 1 -- 172.21.15.87:0/3275022097 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4162560 con 0x7fe8a415de10 2024-04-25T10:20:27.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8a9200640 1 --2- 172.21.15.87:0/3275022097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415de10 0x7fe8a4073d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe894009d90 tx=0x7fe89401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f690a3a2c98f852b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8a3400640 1 -- 172.21.15.87:0/3275022097 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940b2070 con 0x7fe8a415de10 2024-04-25T10:20:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8a3400640 1 -- 172.21.15.87:0/3275022097 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8940a59c0 con 0x7fe8a415de10 2024-04-25T10:20:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8a3400640 1 -- 172.21.15.87:0/3275022097 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940a5d90 con 0x7fe8a415de10 2024-04-25T10:20:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8aa600640 1 -- 172.21.15.87:0/3275022097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415de10 msgr2=0x7fe8a4073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8aa600640 1 --2- 172.21.15.87:0/3275022097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415de10 0x7fe8a4073d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe894009d90 tx=0x7fe89401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8aa600640 1 -- 172.21.15.87:0/3275022097 shutdown_connections 2024-04-25T10:20:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8aa600640 1 --2- 172.21.15.87:0/3275022097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415de10 0x7fe8a4073d80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8aa600640 1 -- 172.21.15.87:0/3275022097 >> 172.21.15.87:0/3275022097 conn(0x7fe8a40cfe20 msgr2=0x7fe8a40d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8aa600640 1 -- 172.21.15.87:0/3275022097 shutdown_connections 2024-04-25T10:20:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.363+0000 7fe8aa600640 1 -- 172.21.15.87:0/3275022097 wait complete. 2024-04-25T10:20:27.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 Processor -- start 2024-04-25T10:20:27.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 -- start start 2024-04-25T10:20:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a406ec60 0x7fe8a40b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a40cf080 0x7fe8a409e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a415de10 0x7fe8a415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a416c530 con 0x7fe8a406ec60 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a416c3b0 con 0x7fe8a40cf080 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a416c6b0 con 0x7fe8a415de10 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a406ec60 0x7fe8a40b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a406ec60 0x7fe8a40b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59490/0 (socket says 172.21.15.87:59490) 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 -- 172.21.15.87:0/522880360 learned_addr learned my addr 172.21.15.87:0/522880360 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a3e00640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a40cf080 0x7fe8a409e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9c00640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a415de10 0x7fe8a415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 -- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a415de10 msgr2=0x7fe8a415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a415de10 0x7fe8a415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 -- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a40cf080 msgr2=0x7fe8a409e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a40cf080 0x7fe8a409e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 -- 172.21.15.87:0/522880360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40b2e20 con 0x7fe8a406ec60 2024-04-25T10:20:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a9200640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a406ec60 0x7fe8a40b3490 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe8940b0040 tx=0x7fe894019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a0c00640 1 -- 172.21.15.87:0/522880360 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940bd020 con 0x7fe8a406ec60 2024-04-25T10:20:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a0c00640 1 -- 172.21.15.87:0/522880360 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8940b8670 con 0x7fe8a406ec60 2024-04-25T10:20:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8a0c00640 1 -- 172.21.15.87:0/522880360 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940b2040 con 0x7fe8a406ec60 2024-04-25T10:20:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a40b3100 con 0x7fe8a406ec60 2024-04-25T10:20:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.367+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a409bf10 con 0x7fe8a406ec60 2024-04-25T10:20:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.371+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4075120 con 0x7fe8a406ec60 2024-04-25T10:20:27.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.371+0000 7fe8a0c00640 1 -- 172.21.15.87:0/522880360 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8940c0020 con 0x7fe8a406ec60 2024-04-25T10:20:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.371+0000 7fe8a0c00640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe884066f90 0x7fe884069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.371+0000 7fe8a0c00640 1 -- 172.21.15.87:0/522880360 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe894120080 con 0x7fe8a406ec60 2024-04-25T10:20:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.371+0000 7fe8a3e00640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe884066f90 0x7fe884069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.371+0000 7fe8a3e00640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe884066f90 0x7fe884069450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe8900015b0 tx=0x7fe890009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.375+0000 7fe8a0c00640 1 -- 172.21.15.87:0/522880360 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe894016560 con 0x7fe8a406ec60 2024-04-25T10:20:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.563+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe8a409da00 con 0x7fe8a406ec60 2024-04-25T10:20:27.566 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:20:27.563+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:20:27.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.563+0000 7fe8a0c00640 1 -- 172.21.15.87:0/522880360 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe8940f1080 con 0x7fe8a406ec60 2024-04-25T10:20:27.567 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:20:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe884066f90 msgr2=0x7fe884069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe884066f90 0x7fe884069450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe8900015b0 tx=0x7fe890009040 comp rx=0 tx=0).stop 2024-04-25T10:20:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a406ec60 msgr2=0x7fe8a40b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a406ec60 0x7fe8a40b3490 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe8940b0040 tx=0x7fe894019680 comp rx=0 tx=0).stop 2024-04-25T10:20:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 shutdown_connections 2024-04-25T10:20:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe884066f90 0x7fe884069450 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a415de10 0x7fe8a415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a40cf080 0x7fe8a409e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 --2- 172.21.15.87:0/522880360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a406ec60 0x7fe8a40b3490 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 >> 172.21.15.87:0/522880360 conn(0x7fe8a40cfe20 msgr2=0x7fe8a406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 shutdown_connections 2024-04-25T10:20:27.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.567+0000 7fe8aa600640 1 -- 172.21.15.87:0/522880360 wait complete. 2024-04-25T10:20:27.592 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:20:27.639 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:20:27.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 Processor -- start 2024-04-25T10:20:27.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 -- start start 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0600aef80 0x7fc06015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 0x7fc06015b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc06015bac0 0x7fc0600b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06015b080 con 0x7fc0600aef80 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06015adb0 con 0x7fc06015bac0 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc065600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06015eb60 con 0x7fc060070350 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0600aef80 0x7fc06015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0600aef80 0x7fc06015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59516/0 (socket says 172.21.15.87:59516) 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 -- 172.21.15.87:0/715842633 learned_addr learned my addr 172.21.15.87:0/715842633 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 -- 172.21.15.87:0/715842633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 msgr2=0x7fc06015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 --2- 172.21.15.87:0/715842633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 0x7fc06015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 -- 172.21.15.87:0/715842633 >> v2:172.21.15.169:3300/0 conn(0x7fc06015bac0 msgr2=0x7fc0600b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:27.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 --2- 172.21.15.87:0/715842633 >> v2:172.21.15.169:3300/0 conn(0x7fc06015bac0 0x7fc0600b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.731+0000 7fc05ea00640 1 -- 172.21.15.87:0/715842633 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06015ee40 con 0x7fc0600aef80 2024-04-25T10:20:27.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc05ea00640 1 --2- 172.21.15.87:0/715842633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0600aef80 0x7fc06015e480 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc0600cc330 tx=0x7fc05001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a1e31ec4074969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc05d600640 1 -- 172.21.15.87:0/715842633 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0500ab5f0 con 0x7fc0600aef80 2024-04-25T10:20:27.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc05d600640 1 -- 172.21.15.87:0/715842633 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0500b0070 con 0x7fc0600aef80 2024-04-25T10:20:27.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc05d600640 1 -- 172.21.15.87:0/715842633 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0500ab9b0 con 0x7fc0600aef80 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 -- 172.21.15.87:0/715842633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0600aef80 msgr2=0x7fc06015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 --2- 172.21.15.87:0/715842633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0600aef80 0x7fc06015e480 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc0600cc330 tx=0x7fc05001c520 comp rx=0 tx=0).stop 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 -- 172.21.15.87:0/715842633 shutdown_connections 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 --2- 172.21.15.87:0/715842633 >> v2:172.21.15.169:3300/0 conn(0x7fc06015bac0 0x7fc0600b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 --2- 172.21.15.87:0/715842633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 0x7fc06015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 --2- 172.21.15.87:0/715842633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0600aef80 0x7fc06015e480 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 -- 172.21.15.87:0/715842633 >> 172.21.15.87:0/715842633 conn(0x7fc0600669a0 msgr2=0x7fc060066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 -- 172.21.15.87:0/715842633 shutdown_connections 2024-04-25T10:20:27.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.735+0000 7fc065600640 1 -- 172.21.15.87:0/715842633 wait complete. 2024-04-25T10:20:27.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 Processor -- start 2024-04-25T10:20:27.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 -- start start 2024-04-25T10:20:27.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 0x7fc0601593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0600aef80 0x7fc060151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc06015bac0 0x7fc060156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06006d5b0 con 0x7fc06015bac0 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06006d430 con 0x7fc0600aef80 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06006d730 con 0x7fc060070350 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc06015bac0 0x7fc060156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc06015bac0 0x7fc060156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59520/0 (socket says 172.21.15.87:59520) 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 -- 172.21.15.87:0/1225954004 learned_addr learned my addr 172.21.15.87:0/1225954004 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05e000640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0600aef80 0x7fc060151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05ea00640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 0x7fc0601593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 -- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 msgr2=0x7fc0601593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 0x7fc0601593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 -- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0600aef80 msgr2=0x7fc060151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0600aef80 0x7fc060151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 -- 172.21.15.87:0/1225954004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060157450 con 0x7fc06015bac0 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc05f400640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc06015bac0 0x7fc060156440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b9a0 tx=0x7fc05800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc056a00640 1 -- 172.21.15.87:0/1225954004 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058019070 con 0x7fc06015bac0 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc056a00640 1 -- 172.21.15.87:0/1225954004 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05800cbc0 con 0x7fc06015bac0 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc060156bb0 con 0x7fc06015bac0 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc056a00640 1 -- 172.21.15.87:0/1225954004 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058014440 con 0x7fc06015bac0 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.739+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc060157170 con 0x7fc06015bac0 2024-04-25T10:20:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.743+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06004ef30 con 0x7fc06015bac0 2024-04-25T10:20:27.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.743+0000 7fc056a00640 1 -- 172.21.15.87:0/1225954004 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc05801c070 con 0x7fc06015bac0 2024-04-25T10:20:27.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.743+0000 7fc056a00640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc040066f90 0x7fc040069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.743+0000 7fc056a00640 1 -- 172.21.15.87:0/1225954004 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc058084ca0 con 0x7fc06015bac0 2024-04-25T10:20:27.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.743+0000 7fc05ea00640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc040066f90 0x7fc040069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.743+0000 7fc05ea00640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc040066f90 0x7fc040069450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc0500af000 tx=0x7fc0500a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.747+0000 7fc056a00640 1 -- 172.21.15.87:0/1225954004 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc058055690 con 0x7fc06015bac0 2024-04-25T10:20:27.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:27.931+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc0600b2fa0 con 0x7fc06015bac0 2024-04-25T10:20:28.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc056a00640 1 -- 172.21.15.87:0/1225954004 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc058059730 con 0x7fc06015bac0 2024-04-25T10:20:28.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc040066f90 msgr2=0x7fc040069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc040066f90 0x7fc040069450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc0500af000 tx=0x7fc0500a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc06015bac0 msgr2=0x7fc060156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc06015bac0 0x7fc060156440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b9a0 tx=0x7fc05800be60 comp rx=0 tx=0).stop 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 shutdown_connections 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc040066f90 0x7fc040069450 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc06015bac0 0x7fc060156440 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0600aef80 0x7fc060151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 --2- 172.21.15.87:0/1225954004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc060070350 0x7fc0601593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 >> 172.21.15.87:0/1225954004 conn(0x7fc0600669a0 msgr2=0x7fc060152520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 shutdown_connections 2024-04-25T10:20:28.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.567+0000 7fc065600640 1 -- 172.21.15.87:0/1225954004 wait complete. 2024-04-25T10:20:28.594 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:20:28.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.731+0000 7fbe3d000640 1 Processor -- start 2024-04-25T10:20:28.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.731+0000 7fbe3d000640 1 -- start start 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe3d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380cf040 0x7fbe380cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe3d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbe3815bc00 0x7fbe380cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe3d000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe38071f80 con 0x7fbe380713a0 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe3d000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe38072260 con 0x7fbe3815bc00 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe3d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380b3700 con 0x7fbe380cf040 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe35400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbe3815bc00 0x7fbe380cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe36800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbe380713a0 0x7fbe380ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59692/0 (socket says 172.21.15.87:59692) 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe36800640 1 -- 172.21.15.87:0/2269655565 learned_addr learned my addr 172.21.15.87:0/2269655565 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe35e00640 1 --2- 172.21.15.87:0/2269655565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380cf040 0x7fbe380cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe35400640 1 -- 172.21.15.87:0/2269655565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380cf040 msgr2=0x7fbe380cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:28.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe35400640 1 --2- 172.21.15.87:0/2269655565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380cf040 0x7fbe380cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe35400640 1 -- 172.21.15.87:0/2269655565 >> v1:172.21.15.87:6789/0 conn(0x7fbe380713a0 legacy=0x7fbe380ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe35400640 1 -- 172.21.15.87:0/2269655565 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3815b120 con 0x7fbe3815bc00 2024-04-25T10:20:28.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.735+0000 7fbe34a00640 1 -- 172.21.15.87:0/2269655565 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292272726 0 0) 0x7fbe38071f80 con 0x7fbe380713a0 2024-04-25T10:20:28.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe35400640 1 --2- 172.21.15.87:0/2269655565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3815bc00 0x7fbe380cff20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fbe240097c0 tx=0x7fbe2401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d63bf9c49f00d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe34a00640 1 -- 172.21.15.87:0/2269655565 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240b2070 con 0x7fbe3815bc00 2024-04-25T10:20:28.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe34a00640 1 -- 172.21.15.87:0/2269655565 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe240ad640 con 0x7fbe3815bc00 2024-04-25T10:20:28.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe34a00640 1 -- 172.21.15.87:0/2269655565 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240ada10 con 0x7fbe3815bc00 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 -- 172.21.15.87:0/2269655565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3815bc00 msgr2=0x7fbe380cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2269655565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3815bc00 0x7fbe380cff20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fbe240097c0 tx=0x7fbe2401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 -- 172.21.15.87:0/2269655565 shutdown_connections 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2269655565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3815bc00 0x7fbe380cff20 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2269655565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380cf040 0x7fbe380cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 -- 172.21.15.87:0/2269655565 >> 172.21.15.87:0/2269655565 conn(0x7fbe38066b70 msgr2=0x7fbe38066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 -- 172.21.15.87:0/2269655565 shutdown_connections 2024-04-25T10:20:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 -- 172.21.15.87:0/2269655565 wait complete. 2024-04-25T10:20:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 Processor -- start 2024-04-25T10:20:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.739+0000 7fbe3d000640 1 -- start start 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380713a0 0x7fbe3814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe380cf040 0x7fbe381f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe3815bc00 0x7fbe38165900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380cf5a0 con 0x7fbe380cf040 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380cf420 con 0x7fbe3815bc00 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3815e500 con 0x7fbe380713a0 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe380cf040 0x7fbe381f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe380cf040 0x7fbe381f6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59534/0 (socket says 172.21.15.87:59534) 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 -- 172.21.15.87:0/2473722144 learned_addr learned my addr 172.21.15.87:0/2473722144 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35e00640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380713a0 0x7fbe3814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe36800640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe3815bc00 0x7fbe38165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 -- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380713a0 msgr2=0x7fbe3814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380713a0 0x7fbe3814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 -- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe3815bc00 msgr2=0x7fbe38165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe3815bc00 0x7fbe38165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 -- 172.21.15.87:0/2473722144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3814f440 con 0x7fbe380cf040 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe36800640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe3815bc00 0x7fbe38165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe35400640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe380cf040 0x7fbe381f6d60 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbe240097c0 tx=0x7fbe24019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe2e000640 1 -- 172.21.15.87:0/2473722144 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240bd070 con 0x7fbe380cf040 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe2e000640 1 -- 172.21.15.87:0/2473722144 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe24004030 con 0x7fbe380cf040 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe2e000640 1 -- 172.21.15.87:0/2473722144 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240b2040 con 0x7fbe380cf040 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3814f720 con 0x7fbe380cf040 2024-04-25T10:20:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3814fce0 con 0x7fbe380cf040 2024-04-25T10:20:28.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3804ef30 con 0x7fbe380cf040 2024-04-25T10:20:28.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.743+0000 7fbe2e000640 1 -- 172.21.15.87:0/2473722144 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe24003620 con 0x7fbe380cf040 2024-04-25T10:20:28.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.747+0000 7fbe2e000640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbe18066f90 0x7fbe18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.747+0000 7fbe2e000640 1 -- 172.21.15.87:0/2473722144 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbe2411e3d0 con 0x7fbe380cf040 2024-04-25T10:20:28.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.747+0000 7fbe35e00640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbe18066f90 0x7fbe18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.747+0000 7fbe35e00640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbe18066f90 0x7fbe18069450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbe38159730 tx=0x7fbe2800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.751+0000 7fbe2e000640 1 -- 172.21.15.87:0/2473722144 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe24016090 con 0x7fbe380cf040 2024-04-25T10:20:28.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:28.907+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe380b2a10 con 0x7fbe380cf040 2024-04-25T10:20:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.567+0000 7fbe2e000640 1 -- 172.21.15.87:0/2473722144 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbe240eec90 con 0x7fbe380cf040 2024-04-25T10:20:29.571 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:20:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbe18066f90 msgr2=0x7fbe18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbe18066f90 0x7fbe18069450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbe38159730 tx=0x7fbe2800b040 comp rx=0 tx=0).stop 2024-04-25T10:20:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe380cf040 msgr2=0x7fbe381f6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe380cf040 0x7fbe381f6d60 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbe240097c0 tx=0x7fbe24019240 comp rx=0 tx=0).stop 2024-04-25T10:20:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 shutdown_connections 2024-04-25T10:20:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbe18066f90 0x7fbe18069450 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe3815bc00 0x7fbe38165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe380cf040 0x7fbe381f6d60 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 --2- 172.21.15.87:0/2473722144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe380713a0 0x7fbe3814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 >> 172.21.15.87:0/2473722144 conn(0x7fbe38066b70 msgr2=0x7fbe38066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 shutdown_connections 2024-04-25T10:20:29.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.571+0000 7fbe3d000640 1 -- 172.21.15.87:0/2473722144 wait complete. 2024-04-25T10:20:29.600 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:20:29.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.735+0000 7fc41c000640 1 Processor -- start 2024-04-25T10:20:29.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.735+0000 7fc41c000640 1 -- start start 2024-04-25T10:20:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.735+0000 7fc41c000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc41415c0f0 0x7fc4140b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc41c000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4140cf9e0 0x7fc4140cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc41c000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41406c0e0 con 0x7fc41415bd10 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc41c000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140d0350 con 0x7fc41415c0f0 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc41c000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41415c6a0 con 0x7fc4140cf9e0 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc41415c0f0 0x7fc4140b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc41415c0f0 0x7fc4140b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59548/0 (socket says 172.21.15.87:59548) 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 -- 172.21.15.87:0/801771459 learned_addr learned my addr 172.21.15.87:0/801771459 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc41a000640 1 --2- 172.21.15.87:0/801771459 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cf9e0 0x7fc4140cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 -- 172.21.15.87:0/801771459 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cf9e0 msgr2=0x7fc4140cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 --2- 172.21.15.87:0/801771459 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cf9e0 0x7fc4140cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 -- 172.21.15.87:0/801771459 >> v1:172.21.15.169:6789/0 conn(0x7fc41415bd10 legacy=0x7fc4140b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 -- 172.21.15.87:0/801771459 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41415c980 con 0x7fc41415c0f0 2024-04-25T10:20:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc41a000640 1 --2- 172.21.15.87:0/801771459 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cf9e0 0x7fc4140cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:29.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc418c00640 1 --2- 172.21.15.87:0/801771459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc41415c0f0 0x7fc4140b27e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc4140702f0 tx=0x7fc40001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fe5164552b62be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc413e00640 1 -- 172.21.15.87:0/801771459 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000b1070 con 0x7fc41415c0f0 2024-04-25T10:20:29.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc413e00640 1 -- 172.21.15.87:0/801771459 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4000ac630 con 0x7fc41415c0f0 2024-04-25T10:20:29.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.739+0000 7fc413e00640 1 -- 172.21.15.87:0/801771459 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000a8050 con 0x7fc41415c0f0 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- 172.21.15.87:0/801771459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc41415c0f0 msgr2=0x7fc4140b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 --2- 172.21.15.87:0/801771459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc41415c0f0 0x7fc4140b27e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc4140702f0 tx=0x7fc40001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- 172.21.15.87:0/801771459 shutdown_connections 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 --2- 172.21.15.87:0/801771459 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cf9e0 0x7fc4140cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 --2- 172.21.15.87:0/801771459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc41415c0f0 0x7fc4140b27e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- 172.21.15.87:0/801771459 >> 172.21.15.87:0/801771459 conn(0x7fc414066b70 msgr2=0x7fc414066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- 172.21.15.87:0/801771459 shutdown_connections 2024-04-25T10:20:29.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- 172.21.15.87:0/801771459 wait complete. 2024-04-25T10:20:29.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 Processor -- start 2024-04-25T10:20:29.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- start start 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4140cf9e0 0x7fc414153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc41415bd10 0x7fc414071230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41415c0f0 0x7fc4140ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41406cf70 con 0x7fc4140cf9e0 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41406cdf0 con 0x7fc41415c0f0 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc414168ac0 con 0x7fc41415bd10 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4140cf9e0 0x7fc414153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4140cf9e0 0x7fc414153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59554/0 (socket says 172.21.15.87:59554) 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 -- 172.21.15.87:0/2824455574 learned_addr learned my addr 172.21.15.87:0/2824455574 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc418c00640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc41415bd10 0x7fc414071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41a000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41415c0f0 0x7fc4140ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 -- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc41415bd10 msgr2=0x7fc414071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc41415bd10 0x7fc414071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 -- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41415c0f0 msgr2=0x7fc4140ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41415c0f0 0x7fc4140ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc419600640 1 -- 172.21.15.87:0/2824455574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414154570 con 0x7fc4140cf9e0 2024-04-25T10:20:29.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc418c00640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc41415bd10 0x7fc414071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.743+0000 7fc41a000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41415c0f0 0x7fc4140ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc419600640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4140cf9e0 0x7fc414153ee0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c7f0 tx=0x7fc40800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc411600640 1 -- 172.21.15.87:0/2824455574 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408016020 con 0x7fc4140cf9e0 2024-04-25T10:20:29.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc411600640 1 -- 172.21.15.87:0/2824455574 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800aa50 con 0x7fc4140cf9e0 2024-04-25T10:20:29.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc411600640 1 -- 172.21.15.87:0/2824455574 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800ae40 con 0x7fc4140cf9e0 2024-04-25T10:20:29.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc414153800 con 0x7fc4140cf9e0 2024-04-25T10:20:29.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc414070750 con 0x7fc4140cf9e0 2024-04-25T10:20:29.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41404ef30 con 0x7fc4140cf9e0 2024-04-25T10:20:29.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc411600640 1 -- 172.21.15.87:0/2824455574 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc408019020 con 0x7fc4140cf9e0 2024-04-25T10:20:29.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc411600640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3f4066f90 0x7fc3f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc411600640 1 -- 172.21.15.87:0/2824455574 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fc408084f30 con 0x7fc4140cf9e0 2024-04-25T10:20:29.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.747+0000 7fc418c00640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3f4066f90 0x7fc3f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.751+0000 7fc418c00640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3f4066f90 0x7fc3f4069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc4000ae000 tx=0x7fc400008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.751+0000 7fc411600640 1 -- 172.21.15.87:0/2824455574 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc408020030 con 0x7fc4140cf9e0 2024-04-25T10:20:29.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:29.907+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc4140cde80 con 0x7fc4140cf9e0 2024-04-25T10:20:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.571+0000 7fc411600640 1 -- 172.21.15.87:0/2824455574 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc4080559b0 con 0x7fc4140cf9e0 2024-04-25T10:20:30.573 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:20:30.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.571+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3f4066f90 msgr2=0x7fc3f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.571+0000 7fc41c000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3f4066f90 0x7fc3f4069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc4000ae000 tx=0x7fc400008040 comp rx=0 tx=0).stop 2024-04-25T10:20:30.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4140cf9e0 msgr2=0x7fc414153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4140cf9e0 0x7fc414153ee0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c7f0 tx=0x7fc40800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 shutdown_connections 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc3f4066f90 0x7fc3f4069450 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41415c0f0 0x7fc4140ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc41415bd10 0x7fc414071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 --2- 172.21.15.87:0/2824455574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4140cf9e0 0x7fc414153ee0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 >> 172.21.15.87:0/2824455574 conn(0x7fc414066b70 msgr2=0x7fc4140ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 shutdown_connections 2024-04-25T10:20:30.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.575+0000 7fc41c000640 1 -- 172.21.15.87:0/2824455574 wait complete. 2024-04-25T10:20:30.613 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:20:30.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.767+0000 7f2458a00640 1 Processor -- start 2024-04-25T10:20:30.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.767+0000 7f2458a00640 1 -- start start 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2458a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24541594d0 0x7f2454071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2458a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245406c1e0 con 0x7f24541598b0 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2458a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245406c4c0 con 0x7f24540aeda0 2024-04-25T10:20:30.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2458a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245415d260 con 0x7f24541594d0 2024-04-25T10:20:30.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2451400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f24541598b0 0x7f24540ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59706/0 (socket says 172.21.15.87:59706) 2024-04-25T10:20:30.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2451400640 1 -- 172.21.15.87:0/1142418814 learned_addr learned my addr 172.21.15.87:0/1142418814 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:30.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2451e00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24541594d0 0x7f2454071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645005702 0 0) 0x7f245406c1e0 con 0x7f24541598b0 2024-04-25T10:20:30.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.771+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2438003530 con 0x7f24541598b0 2024-04-25T10:20:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1289199734 0 0) 0x7f2438003530 con 0x7f24541598b0 2024-04-25T10:20:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24541594d0 msgr2=0x7f2454071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24541594d0 0x7f2454071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 >> v1:172.21.15.169:6789/0 conn(0x7f24540aeda0 legacy=0x7f2454070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245406c7a0 con 0x7f24541598b0 2024-04-25T10:20:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f243c001680 con 0x7f24541598b0 2024-04-25T10:20:30.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 >> v1:172.21.15.87:6789/0 conn(0x7f24541598b0 legacy=0x7f24540ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24380039f0 0x7f2438003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2438004810 0x7f2438004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24380051d0 0x7f243800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243800bfc0 con 0x7f24380039f0 2024-04-25T10:20:30.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438010580 con 0x7f2438004810 2024-04-25T10:20:30.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2438010780 con 0x7f24380051d0 2024-04-25T10:20:30.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2452800640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24380051d0 0x7f243800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2451400640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2438004810 0x7f2438004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.775+0000 7f2451e00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24380039f0 0x7f2438003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.779+0000 7f2451400640 1 -- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24380051d0 msgr2=0x7f243800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.779+0000 7f2451400640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24380051d0 0x7f243800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.779+0000 7f2451400640 1 -- 172.21.15.87:0/1142418814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24380039f0 msgr2=0x7f2438003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.779+0000 7f2451400640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24380039f0 0x7f2438003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.779+0000 7f2451400640 1 -- 172.21.15.87:0/1142418814 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245406c7a0 con 0x7f2438004810 2024-04-25T10:20:30.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.779+0000 7f2451e00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24380039f0 0x7f2438003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:30.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.783+0000 7f2451400640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2438004810 0x7f2438004c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2454159c90 tx=0x7f243c01cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff26a3db070887cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.783+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c007d40 con 0x7f2438004810 2024-04-25T10:20:30.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c0a9030 con 0x7f2438004810 2024-04-25T10:20:30.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2450a00640 1 -- 172.21.15.87:0/1142418814 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0a5d90 con 0x7f2438004810 2024-04-25T10:20:30.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2458a00640 1 -- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2438004810 msgr2=0x7f2438004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2458a00640 1 --2- 172.21.15.87:0/1142418814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2438004810 0x7f2438004c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2454159c90 tx=0x7f243c01cc90 comp rx=0 tx=0).stop 2024-04-25T10:20:30.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2452800640 1 -- 172.21.15.87:0/1142418814 reap_dead start 2024-04-25T10:20:30.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2458a00640 1 -- 172.21.15.87:0/1142418814 shutdown_connections 2024-04-25T10:20:30.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2458a00640 1 -- 172.21.15.87:0/1142418814 >> 172.21.15.87:0/1142418814 conn(0x7f24540b5480 msgr2=0x7f24540669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2458a00640 1 -- 172.21.15.87:0/1142418814 shutdown_connections 2024-04-25T10:20:30.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.787+0000 7f2458a00640 1 -- 172.21.15.87:0/1142418814 wait complete. 2024-04-25T10:20:30.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 Processor -- start 2024-04-25T10:20:30.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 -- start start 2024-04-25T10:20:30.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f245415a490 0x7f24540b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2454204370 0x7f24540b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f245415a870 0x7f2454205b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2454070be0 con 0x7f245415a870 2024-04-25T10:20:30.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2454070a60 con 0x7f2454204370 2024-04-25T10:20:30.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.791+0000 7f2458a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2454070d60 con 0x7f245415a490 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f245415a870 0x7f2454205b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f245415a870 0x7f2454205b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59580/0 (socket says 172.21.15.87:59580) 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 -- 172.21.15.87:0/978529334 learned_addr learned my addr 172.21.15.87:0/978529334 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 -- 172.21.15.87:0/978529334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f245415a490 msgr2=0x7f24540b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f245415a490 0x7f24540b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 -- 172.21.15.87:0/978529334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2454204370 msgr2=0x7f24540b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2454204370 0x7f24540b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 -- 172.21.15.87:0/978529334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24542068d0 con 0x7f245415a870 2024-04-25T10:20:30.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2452800640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f245415a870 0x7f2454205b30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2438004790 tx=0x7f244400f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f244a000640 1 -- 172.21.15.87:0/978529334 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c009bb0 con 0x7f245415a870 2024-04-25T10:20:30.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f244a000640 1 -- 172.21.15.87:0/978529334 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444018070 con 0x7f245415a870 2024-04-25T10:20:30.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f244a000640 1 -- 172.21.15.87:0/978529334 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444003600 con 0x7f245415a870 2024-04-25T10:20:30.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2454206bb0 con 0x7f245415a870 2024-04-25T10:20:30.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2454207170 con 0x7f245415a870 2024-04-25T10:20:30.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.795+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245404ef30 con 0x7f245415a870 2024-04-25T10:20:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.799+0000 7f244a000640 1 -- 172.21.15.87:0/978529334 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24440037a0 con 0x7f245415a870 2024-04-25T10:20:30.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.799+0000 7f244a000640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2430066f90 0x7f2430069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.799+0000 7f244a000640 1 -- 172.21.15.87:0/978529334 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2444084540 con 0x7f245415a870 2024-04-25T10:20:30.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.799+0000 7f2451e00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2430066f90 0x7f2430069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.799+0000 7f2451e00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2430066f90 0x7f2430069450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f24540b4c20 tx=0x7f244c009750 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.803+0000 7f244a000640 1 -- 172.21.15.87:0/978529334 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24440558e0 con 0x7f245415a870 2024-04-25T10:20:30.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.959+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f245406d7f0 con 0x7f245415a870 2024-04-25T10:20:30.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f244a000640 1 -- 172.21.15.87:0/978529334 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f24440551e0 con 0x7f245415a870 2024-04-25T10:20:30.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2430066f90 msgr2=0x7f2430069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2430066f90 0x7f2430069450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f24540b4c20 tx=0x7f244c009750 comp rx=0 tx=0).stop 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f245415a870 msgr2=0x7f2454205b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f245415a870 0x7f2454205b30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2438004790 tx=0x7f244400f910 comp rx=0 tx=0).stop 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 shutdown_connections 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2430066f90 0x7f2430069450 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f245415a870 0x7f2454205b30 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2454204370 0x7f24540b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 --2- 172.21.15.87:0/978529334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f245415a490 0x7f24540b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 >> 172.21.15.87:0/978529334 conn(0x7f24540b5480 msgr2=0x7f24540cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 shutdown_connections 2024-04-25T10:20:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:30.963+0000 7f2458a00640 1 -- 172.21.15.87:0/978529334 wait complete. 2024-04-25T10:20:30.986 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:20:31.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794f800640 1 Processor -- start 2024-04-25T10:20:31.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794f800640 1 -- start start 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794f800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f79481550b0 0x7f79480cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794f800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79480cfe70 0x7f794815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794f800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7948071050 con 0x7f794815b740 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794f800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794815d040 con 0x7f79481550b0 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794f800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79480712f0 con 0x7f79480cfe70 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f79481550b0 0x7f79480cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794d800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f794815b740 0x7f794806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59710/0 (socket says 172.21.15.87:59710) 2024-04-25T10:20:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794d800640 1 -- 172.21.15.87:0/837479751 learned_addr learned my addr 172.21.15.87:0/837479751 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:31.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f7947e00640 1 --2- 172.21.15.87:0/837479751 >> v2:172.21.15.169:3300/0 conn(0x7f79480cfe70 0x7f794815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794ce00640 1 -- 172.21.15.87:0/837479751 >> v2:172.21.15.169:3300/0 conn(0x7f79480cfe70 msgr2=0x7f794815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794ce00640 1 --2- 172.21.15.87:0/837479751 >> v2:172.21.15.169:3300/0 conn(0x7f79480cfe70 0x7f794815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794ce00640 1 -- 172.21.15.87:0/837479751 >> v1:172.21.15.87:6789/0 conn(0x7f794815b740 legacy=0x7f794806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f794ce00640 1 -- 172.21.15.87:0/837479751 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480715d0 con 0x7f79481550b0 2024-04-25T10:20:31.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.119+0000 7f7947400640 1 -- 172.21.15.87:0/837479751 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154528109 0 0) 0x7f7948071050 con 0x7f794815b740 2024-04-25T10:20:31.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794ce00640 1 --2- 172.21.15.87:0/837479751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f79480cf930 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b9ab4972558fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f7947400640 1 -- 172.21.15.87:0/837479751 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380b2070 con 0x7f79481550b0 2024-04-25T10:20:31.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f7947400640 1 -- 172.21.15.87:0/837479751 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380a5800 con 0x7f79481550b0 2024-04-25T10:20:31.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f7947400640 1 -- 172.21.15.87:0/837479751 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380a5bd0 con 0x7f79481550b0 2024-04-25T10:20:31.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 -- 172.21.15.87:0/837479751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 msgr2=0x7f79480cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 --2- 172.21.15.87:0/837479751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f79480cf930 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793801c930 comp rx=0 tx=0).stop 2024-04-25T10:20:31.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 -- 172.21.15.87:0/837479751 shutdown_connections 2024-04-25T10:20:31.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 --2- 172.21.15.87:0/837479751 >> v2:172.21.15.169:3300/0 conn(0x7f79480cfe70 0x7f794815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 --2- 172.21.15.87:0/837479751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f79480cf930 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 -- 172.21.15.87:0/837479751 >> 172.21.15.87:0/837479751 conn(0x7f7948066b70 msgr2=0x7f7948066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 -- 172.21.15.87:0/837479751 shutdown_connections 2024-04-25T10:20:31.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.123+0000 7f794f800640 1 -- 172.21.15.87:0/837479751 wait complete. 2024-04-25T10:20:31.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 Processor -- start 2024-04-25T10:20:31.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 -- start start 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79480cfe70 0x7f79480cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f794814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794815b740 0x7f7948155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794815bcf0 con 0x7f79481550b0 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794815bb70 con 0x7f79480cfe70 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794815c030 con 0x7f794815b740 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f794814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f794814bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59608/0 (socket says 172.21.15.87:59608) 2024-04-25T10:20:31.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 -- 172.21.15.87:0/2809219117 learned_addr learned my addr 172.21.15.87:0/2809219117 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794d800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794815b740 0x7f7948155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794ce00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79480cfe70 0x7f79480cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 -- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794815b740 msgr2=0x7f7948155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794815b740 0x7f7948155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 -- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79480cfe70 msgr2=0x7f79480cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79480cfe70 0x7f79480cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 -- 172.21.15.87:0/2809219117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480b0180 con 0x7f79481550b0 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794ce00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79480cfe70 0x7f79480cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794d800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794815b740 0x7f7948155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7947e00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f794814bb60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f793400d800 tx=0x7f793400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7944c00640 1 -- 172.21.15.87:0/2809219117 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400b6c0 con 0x7f79481550b0 2024-04-25T10:20:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7944c00640 1 -- 172.21.15.87:0/2809219117 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934019070 con 0x7f79481550b0 2024-04-25T10:20:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79480b0460 con 0x7f79481550b0 2024-04-25T10:20:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f7944c00640 1 -- 172.21.15.87:0/2809219117 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400ba60 con 0x7f79481550b0 2024-04-25T10:20:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.127+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79480ce400 con 0x7f79481550b0 2024-04-25T10:20:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.131+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794804ef30 con 0x7f79481550b0 2024-04-25T10:20:31.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.131+0000 7f7944c00640 1 -- 172.21.15.87:0/2809219117 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f793400bc00 con 0x7f79481550b0 2024-04-25T10:20:31.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.131+0000 7f7944c00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7928066f60 0x7f7928069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.131+0000 7f7944c00640 1 -- 172.21.15.87:0/2809219117 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7934085140 con 0x7f79481550b0 2024-04-25T10:20:31.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.131+0000 7f794ce00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7928066f60 0x7f7928069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.131+0000 7f794ce00640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7928066f60 0x7f7928069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f79380af000 tx=0x7f79380a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.135+0000 7f7944c00640 1 -- 172.21.15.87:0/2809219117 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79340564e0 con 0x7f79481550b0 2024-04-25T10:20:31.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.347+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f794815a8b0 con 0x7f79481550b0 2024-04-25T10:20:31.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.583+0000 7f7944c00640 1 -- 172.21.15.87:0/2809219117 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v399) v1 ==== 155+0+0 (secure 0 0 0) 0x7f7934055de0 con 0x7f79481550b0 2024-04-25T10:20:31.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7928066f60 msgr2=0x7f7928069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7928066f60 0x7f7928069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f79380af000 tx=0x7f79380a7070 comp rx=0 tx=0).stop 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 msgr2=0x7f794814bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f794814bb60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f793400d800 tx=0x7f793400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 shutdown_connections 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7928066f60 0x7f7928069420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794815b740 0x7f7948155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79481550b0 0x7f794814bb60 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 --2- 172.21.15.87:0/2809219117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79480cfe70 0x7f79480cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 >> 172.21.15.87:0/2809219117 conn(0x7f7948066b70 msgr2=0x7f794814fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 shutdown_connections 2024-04-25T10:20:31.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.587+0000 7f794f800640 1 -- 172.21.15.87:0/2809219117 wait complete. 2024-04-25T10:20:31.609 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:20:31.609 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:20:31.610 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:31.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f123d800640 1 Processor -- start 2024-04-25T10:20:31.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f123d800640 1 -- start start 2024-04-25T10:20:31.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f123d800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12380ccb20 0x7f12380d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f123d800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380ccfb0 con 0x7f123815b9e0 2024-04-25T10:20:31.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f123d800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380cd290 con 0x7f123815bdc0 2024-04-25T10:20:31.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f123d800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12380cd570 con 0x7f12380ccb20 2024-04-25T10:20:31.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f123815b9e0 0x7f12380cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59722/0 (socket says 172.21.15.87:59722) 2024-04-25T10:20:31.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 -- 172.21.15.87:0/2798130555 learned_addr learned my addr 172.21.15.87:0/2798130555 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:31.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f122fe00640 1 --2- 172.21.15.87:0/2798130555 >> v2:172.21.15.169:3300/0 conn(0x7f12380ccb20 0x7f12380d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040081372 0 0) 0x7f12380ccfb0 con 0x7f123815b9e0 2024-04-25T10:20:31.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f121c003530 con 0x7f123815b9e0 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2847434221 0 0) 0x7f121c003530 con 0x7f123815b9e0 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 >> v2:172.21.15.169:3300/0 conn(0x7f12380ccb20 msgr2=0x7f12380d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 --2- 172.21.15.87:0/2798130555 >> v2:172.21.15.169:3300/0 conn(0x7f12380ccb20 0x7f12380d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 >> v1:172.21.15.169:6789/0 conn(0x7f123815bdc0 legacy=0x7f12380c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380cd850 con 0x7f123815b9e0 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f1228001680 con 0x7f123815b9e0 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 >> v1:172.21.15.87:6789/0 conn(0x7f123815b9e0 legacy=0x7f12380cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c0039c0 0x7f121c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f121c004810 0x7f121c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0051d0 0x7f121c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380cd570 con 0x7f121c0039c0 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c00c070 con 0x7f121c004810 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121c00c270 con 0x7f121c0051d0 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c0039c0 0x7f121c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f122fe00640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f121c004810 0x7f121c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1237400640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0051d0 0x7f121c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 -- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0051d0 msgr2=0x7f121c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0051d0 0x7f121c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 -- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f121c004810 msgr2=0x7f121c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f121c004810 0x7f121c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1236a00640 1 -- 172.21.15.87:0/2798130555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c003530 con 0x7f121c0039c0 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1237400640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0051d0 0x7f121c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:31.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.743+0000 7f1237400640 1 -- 172.21.15.87:0/2798130555 reap_dead start 2024-04-25T10:20:31.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f1236a00640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c0039c0 0x7f121c003de0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f122800c1c0 tx=0x7f122801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe0698a0ce0c3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228007d00 con 0x7f121c0039c0 2024-04-25T10:20:31.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12280a9020 con 0x7f121c0039c0 2024-04-25T10:20:31.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f1236000640 1 -- 172.21.15.87:0/2798130555 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228005ae0 con 0x7f121c0039c0 2024-04-25T10:20:31.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f123d800640 1 -- 172.21.15.87:0/2798130555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c0039c0 msgr2=0x7f121c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f123d800640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c0039c0 0x7f121c003de0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f122800c1c0 tx=0x7f122801cc60 comp rx=0 tx=0).stop 2024-04-25T10:20:31.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f123d800640 1 -- 172.21.15.87:0/2798130555 shutdown_connections 2024-04-25T10:20:31.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f123d800640 1 --2- 172.21.15.87:0/2798130555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c0039c0 0x7f121c003de0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f123d800640 1 -- 172.21.15.87:0/2798130555 >> 172.21.15.87:0/2798130555 conn(0x7f123809a9d0 msgr2=0x7f123809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f123d800640 1 -- 172.21.15.87:0/2798130555 shutdown_connections 2024-04-25T10:20:31.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.747+0000 7f123d800640 1 -- 172.21.15.87:0/2798130555 wait complete. 2024-04-25T10:20:31.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 Processor -- start 2024-04-25T10:20:31.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 -- start start 2024-04-25T10:20:31.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0039c0 0x7f123815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f123815ec50 0x7f123815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12380ca550 0x7f12380ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380ce3c0 con 0x7f12380ca550 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380ce240 con 0x7f123815ec50 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380ce540 con 0x7f121c0039c0 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12380ca550 0x7f12380ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12380ca550 0x7f12380ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59620/0 (socket says 172.21.15.87:59620) 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 -- 172.21.15.87:0/617887263 learned_addr learned my addr 172.21.15.87:0/617887263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f122fe00640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f123815ec50 0x7f123815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1236a00640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0039c0 0x7f123815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 -- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0039c0 msgr2=0x7f123815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0039c0 0x7f123815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 -- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f123815ec50 msgr2=0x7f123815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f123815ec50 0x7f123815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 -- 172.21.15.87:0/617887263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380cb360 con 0x7f12380ca550 2024-04-25T10:20:31.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f1237400640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12380ca550 0x7f12380ca9d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f12380d2590 tx=0x7f123000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f122ea00640 1 -- 172.21.15.87:0/617887263 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230019070 con 0x7f12380ca550 2024-04-25T10:20:31.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f122ea00640 1 -- 172.21.15.87:0/617887263 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1230003b90 con 0x7f12380ca550 2024-04-25T10:20:31.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12380cb640 con 0x7f12380ca550 2024-04-25T10:20:31.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f122ea00640 1 -- 172.21.15.87:0/617887263 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300050d0 con 0x7f12380ca550 2024-04-25T10:20:31.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.751+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123815d220 con 0x7f12380ca550 2024-04-25T10:20:31.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.755+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123806ce20 con 0x7f12380ca550 2024-04-25T10:20:31.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.755+0000 7f122ea00640 1 -- 172.21.15.87:0/617887263 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1230005270 con 0x7f12380ca550 2024-04-25T10:20:31.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.755+0000 7f122ea00640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1214066f90 0x7f1214069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.755+0000 7f122ea00640 1 -- 172.21.15.87:0/617887263 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1230084230 con 0x7f12380ca550 2024-04-25T10:20:31.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.755+0000 7f1236a00640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1214066f90 0x7f1214069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.755+0000 7f1236a00640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1214066f90 0x7f1214069450 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f12280aa000 tx=0x7f12380cd460 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.759+0000 7f122ea00640 1 -- 172.21.15.87:0/617887263 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1230054dc0 con 0x7f12380ca550 2024-04-25T10:20:31.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.915+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f123804ef30 con 0x7f12380ca550 2024-04-25T10:20:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f122ea00640 1 -- 172.21.15.87:0/617887263 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13010 (secure 0 0 0) 0x7f1230058e60 con 0x7f12380ca550 2024-04-25T10:20:31.920 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:31.920 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":103,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:20:30.571806+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6836/1167245113":"2024-04-26T10:20:27.566147+0000","172.21.15.87:6834/1167245113":"2024-04-26T10:20:27.566147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1214066f90 msgr2=0x7f1214069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1214066f90 0x7f1214069450 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f12280aa000 tx=0x7f12380cd460 comp rx=0 tx=0).stop 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12380ca550 msgr2=0x7f12380ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12380ca550 0x7f12380ca9d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f12380d2590 tx=0x7f123000ead0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 shutdown_connections 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1214066f90 0x7f1214069450 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12380ca550 0x7f12380ca9d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f123815ec50 0x7f123815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 --2- 172.21.15.87:0/617887263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f121c0039c0 0x7f123815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 >> 172.21.15.87:0/617887263 conn(0x7f123809a9d0 msgr2=0x7f123806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 shutdown_connections 2024-04-25T10:20:31.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:31.919+0000 7f123d800640 1 -- 172.21.15.87:0/617887263 wait complete. 2024-04-25T10:20:31.940 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:31.941 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.075+0000 7ff887200640 1 Processor -- start 2024-04-25T10:20:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.075+0000 7ff887200640 1 -- start start 2024-04-25T10:20:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff887200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff888071120 0x7ff88815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff887200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8880b5200 con 0x7ff8880cf9b0 2024-04-25T10:20:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff887200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8880b38b0 con 0x7ff8880b3360 2024-04-25T10:20:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff887200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88815ef10 con 0x7ff888071120 2024-04-25T10:20:32.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff8880cf9b0 0x7ff888070a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59732/0 (socket says 172.21.15.87:59732) 2024-04-25T10:20:32.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885e00640 1 -- 172.21.15.87:0/1771112613 learned_addr learned my addr 172.21.15.87:0/1771112613 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:32.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff87d400640 1 --2- 172.21.15.87:0/1771112613 >> v2:172.21.15.169:3300/0 conn(0x7ff888071120 0x7ff88815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880896025 0 0) 0x7ff8880b5200 con 0x7ff8880cf9b0 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff86c003530 con 0x7ff8880cf9b0 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3743225201 0 0) 0x7ff86c003530 con 0x7ff8880cf9b0 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 >> v2:172.21.15.169:3300/0 conn(0x7ff888071120 msgr2=0x7ff88815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 --2- 172.21.15.87:0/1771112613 >> v2:172.21.15.169:3300/0 conn(0x7ff888071120 0x7ff88815e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 >> v1:172.21.15.169:6789/0 conn(0x7ff8880b3360 legacy=0x7ff8881540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880b3b90 con 0x7ff8880cf9b0 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ff878001680 con 0x7ff8880cf9b0 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 >> v1:172.21.15.87:6789/0 conn(0x7ff8880cf9b0 legacy=0x7ff888070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86c003a30 0x7ff86c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff86c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff86c005240 0x7ff86c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88815ef10 con 0x7ff86c004810 2024-04-25T10:20:32.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c00c1a0 con 0x7ff86c003a30 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c00c3a0 con 0x7ff86c005240 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff87d400640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff86c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff886800640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff86c005240 0x7ff86c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff885e00640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86c003a30 0x7ff86c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff87d400640 1 -- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff86c005240 msgr2=0x7ff86c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff87d400640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff86c005240 0x7ff86c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff87d400640 1 -- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86c003a30 msgr2=0x7ff86c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff87d400640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86c003a30 0x7ff86c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff87d400640 1 -- 172.21.15.87:0/1771112613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c003530 con 0x7ff86c004810 2024-04-25T10:20:32.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.079+0000 7ff886800640 1 -- 172.21.15.87:0/1771112613 reap_dead start 2024-04-25T10:20:32.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff87d400640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff86c004c90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7ff870009ae0 tx=0x7ff87001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ca5f189b2d583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700b2070 con 0x7ff86c004810 2024-04-25T10:20:32.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8700a5550 con 0x7ff86c004810 2024-04-25T10:20:32.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff885400640 1 -- 172.21.15.87:0/1771112613 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700096b0 con 0x7ff86c004810 2024-04-25T10:20:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 -- 172.21.15.87:0/1771112613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 msgr2=0x7ff86c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff86c004c90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7ff870009ae0 tx=0x7ff87001ca70 comp rx=0 tx=0).stop 2024-04-25T10:20:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 -- 172.21.15.87:0/1771112613 shutdown_connections 2024-04-25T10:20:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 --2- 172.21.15.87:0/1771112613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff86c004c90 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 -- 172.21.15.87:0/1771112613 >> 172.21.15.87:0/1771112613 conn(0x7ff8880cfe80 msgr2=0x7ff8880d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 -- 172.21.15.87:0/1771112613 shutdown_connections 2024-04-25T10:20:32.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 -- 172.21.15.87:0/1771112613 wait complete. 2024-04-25T10:20:32.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 Processor -- start 2024-04-25T10:20:32.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.083+0000 7ff887200640 1 -- start start 2024-04-25T10:20:32.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff88806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88815ad70 0x7ff88806e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff888158890 0x7ff888158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff888154a50 con 0x7ff86c004810 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881548d0 con 0x7ff88815ad70 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff888154bd0 con 0x7ff888158890 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff88806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff88806e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59636/0 (socket says 172.21.15.87:59636) 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 -- 172.21.15.87:0/644921426 learned_addr learned my addr 172.21.15.87:0/644921426 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff886800640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff888158890 0x7ff888158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff87d400640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88815ad70 0x7ff88806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 -- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff888158890 msgr2=0x7ff888158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff888158890 0x7ff888158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 -- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88815ad70 msgr2=0x7ff88806e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88815ad70 0x7ff88806e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 -- 172.21.15.87:0/644921426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88815b330 con 0x7ff86c004810 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff87d400640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88815ad70 0x7ff88806e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff885e00640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff88806e1f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff8780072c0 tx=0x7ff878007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff87ea00640 1 -- 172.21.15.87:0/644921426 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800f8e0 con 0x7ff86c004810 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff87ea00640 1 -- 172.21.15.87:0/644921426 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878018070 con 0x7ff86c004810 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff87ea00640 1 -- 172.21.15.87:0/644921426 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800fce0 con 0x7ff86c004810 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88815a3e0 con 0x7ff86c004810 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff888159250 con 0x7ff86c004810 2024-04-25T10:20:32.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff888075120 con 0x7ff86c004810 2024-04-25T10:20:32.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.087+0000 7ff87ea00640 1 -- 172.21.15.87:0/644921426 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff87801d460 con 0x7ff86c004810 2024-04-25T10:20:32.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.091+0000 7ff87ea00640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff864066f90 0x7ff864069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.091+0000 7ff87ea00640 1 -- 172.21.15.87:0/644921426 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff878084bb0 con 0x7ff86c004810 2024-04-25T10:20:32.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.091+0000 7ff87d400640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff864066f90 0x7ff864069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.091+0000 7ff87d400640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff864066f90 0x7ff864069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff8700af000 tx=0x7ff870008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.095+0000 7ff87ea00640 1 -- 172.21.15.87:0/644921426 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8780559c0 con 0x7ff86c004810 2024-04-25T10:20:32.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.247+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8880740e0 con 0x7ff86c004810 2024-04-25T10:20:32.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.247+0000 7ff87ea00640 1 -- 172.21.15.87:0/644921426 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff878059a60 con 0x7ff86c004810 2024-04-25T10:20:32.251 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff864066f90 msgr2=0x7ff864069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff864066f90 0x7ff864069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff8700af000 tx=0x7ff870008040 comp rx=0 tx=0).stop 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 msgr2=0x7ff88806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff88806e1f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff8780072c0 tx=0x7ff878007780 comp rx=0 tx=0).stop 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 shutdown_connections 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff864066f90 0x7ff864069450 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff888158890 0x7ff888158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88815ad70 0x7ff88806e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 --2- 172.21.15.87:0/644921426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff86c004810 0x7ff88806e1f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 >> 172.21.15.87:0/644921426 conn(0x7ff8880cfe80 msgr2=0x7ff888160760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 shutdown_connections 2024-04-25T10:20:32.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.251+0000 7ff887200640 1 -- 172.21.15.87:0/644921426 wait complete. 2024-04-25T10:20:32.272 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:20:32.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.403+0000 7f8bb2a00640 1 Processor -- start 2024-04-25T10:20:32.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.403+0000 7f8bb2a00640 1 -- start start 2024-04-25T10:20:32.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.403+0000 7f8bb2a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8bac071f70 0x7f8bac0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.403+0000 7f8bb2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8bac0b2fc0 0x7f8bac06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.403+0000 7f8bb2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bac070ed0 con 0x7f8bac15a0e0 2024-04-25T10:20:32.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb2a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac0b3420 con 0x7f8bac0b2fc0 2024-04-25T10:20:32.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb2a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac0711b0 con 0x7f8bac071f70 2024-04-25T10:20:32.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8bac0b2fc0 0x7f8bac06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8bac0b2fc0 0x7f8bac06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59642/0 (socket says 172.21.15.87:59642) 2024-04-25T10:20:32.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 -- 172.21.15.87:0/1751297174 learned_addr learned my addr 172.21.15.87:0/1751297174 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:32.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bab400640 1 --2- 172.21.15.87:0/1751297174 >> v2:172.21.15.169:3300/0 conn(0x7f8bac071f70 0x7f8bac0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 -- 172.21.15.87:0/1751297174 >> v2:172.21.15.169:3300/0 conn(0x7f8bac071f70 msgr2=0x7f8bac0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 --2- 172.21.15.87:0/1751297174 >> v2:172.21.15.169:3300/0 conn(0x7f8bac071f70 0x7f8bac0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 -- 172.21.15.87:0/1751297174 >> v1:172.21.15.169:6789/0 conn(0x7f8bac15a0e0 legacy=0x7f8bac15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 -- 172.21.15.87:0/1751297174 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac071490 con 0x7f8bac0b2fc0 2024-04-25T10:20:32.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8bb0a00640 1 --2- 172.21.15.87:0/1751297174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac0b2fc0 0x7f8bac06c090 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8bac06e5d0 tx=0x7f8ba401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108bc96dd07bd66c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8baaa00640 1 -- 172.21.15.87:0/1751297174 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40b2070 con 0x7f8bac0b2fc0 2024-04-25T10:20:32.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.407+0000 7f8baaa00640 1 -- 172.21.15.87:0/1751297174 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba40a9070 con 0x7f8bac0b2fc0 2024-04-25T10:20:32.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8baaa00640 1 -- 172.21.15.87:0/1751297174 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40ad880 con 0x7f8bac0b2fc0 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- 172.21.15.87:0/1751297174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac0b2fc0 msgr2=0x7f8bac06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/1751297174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac0b2fc0 0x7f8bac06c090 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8bac06e5d0 tx=0x7f8ba401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- 172.21.15.87:0/1751297174 shutdown_connections 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/1751297174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac0b2fc0 0x7f8bac06c090 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/1751297174 >> v2:172.21.15.169:3300/0 conn(0x7f8bac071f70 0x7f8bac0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- 172.21.15.87:0/1751297174 >> 172.21.15.87:0/1751297174 conn(0x7f8bac066b70 msgr2=0x7f8bac066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- 172.21.15.87:0/1751297174 shutdown_connections 2024-04-25T10:20:32.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- 172.21.15.87:0/1751297174 wait complete. 2024-04-25T10:20:32.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 Processor -- start 2024-04-25T10:20:32.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- start start 2024-04-25T10:20:32.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac071f70 0x7f8bac146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8bac0b2fc0 0x7f8bac155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bac15a0e0 0x7f8bac15b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac070b50 con 0x7f8bac071f70 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac15a4c0 con 0x7f8bac15a0e0 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac072350 con 0x7f8bac0b2fc0 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac071f70 0x7f8bac146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac071f70 0x7f8bac146d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59648/0 (socket says 172.21.15.87:59648) 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 -- 172.21.15.87:0/231633229 learned_addr learned my addr 172.21.15.87:0/231633229 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bab400640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8bac0b2fc0 0x7f8bac155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb0a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bac15a0e0 0x7f8bac15b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 -- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8bac0b2fc0 msgr2=0x7f8bac155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8bac0b2fc0 0x7f8bac155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 -- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bac15a0e0 msgr2=0x7f8bac15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bac15a0e0 0x7f8bac15b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8babe00640 1 -- 172.21.15.87:0/231633229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac150f30 con 0x7f8bac071f70 2024-04-25T10:20:32.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.411+0000 7f8bb0a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bac15a0e0 0x7f8bac15b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:32.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8babe00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac071f70 0x7f8bac146d40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c760 tx=0x7f8b9c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8ba3e00640 1 -- 172.21.15.87:0/231633229 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c016020 con 0x7f8bac071f70 2024-04-25T10:20:32.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8ba3e00640 1 -- 172.21.15.87:0/231633229 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c006d60 con 0x7f8bac071f70 2024-04-25T10:20:32.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac151210 con 0x7f8bac071f70 2024-04-25T10:20:32.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8ba3e00640 1 -- 172.21.15.87:0/231633229 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c003bc0 con 0x7f8bac071f70 2024-04-25T10:20:32.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac1517d0 con 0x7f8bac071f70 2024-04-25T10:20:32.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac04ef30 con 0x7f8bac071f70 2024-04-25T10:20:32.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8ba3e00640 1 -- 172.21.15.87:0/231633229 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b9c00a850 con 0x7f8bac071f70 2024-04-25T10:20:32.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8ba3e00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b8c066f60 0x7f8b8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8ba3e00640 1 -- 172.21.15.87:0/231633229 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8b9c084950 con 0x7f8bac071f70 2024-04-25T10:20:32.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.415+0000 7f8bab400640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b8c066f60 0x7f8b8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.419+0000 7f8bab400640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b8c066f60 0x7f8b8c069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0b0d90 tx=0x7f8b98008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.419+0000 7f8ba3e00640 1 -- 172.21.15.87:0/231633229 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b9c020020 con 0x7f8bac071f70 2024-04-25T10:20:32.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:32.627+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f8bac15d790 con 0x7f8bac071f70 2024-04-25T10:20:33.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.591+0000 7f8ba3e00640 1 -- 172.21.15.87:0/231633229 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v401) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b9c055cf0 con 0x7f8bac071f70 2024-04-25T10:20:33.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b8c066f60 msgr2=0x7f8b8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b8c066f60 0x7f8b8c069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0b0d90 tx=0x7f8b98008040 comp rx=0 tx=0).stop 2024-04-25T10:20:33.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac071f70 msgr2=0x7f8bac146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac071f70 0x7f8bac146d40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c760 tx=0x7f8b9c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 shutdown_connections 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b8c066f60 0x7f8b8c069420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bac15a0e0 0x7f8bac15b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8bac0b2fc0 0x7f8bac155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 --2- 172.21.15.87:0/231633229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bac071f70 0x7f8bac146d40 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 >> 172.21.15.87:0/231633229 conn(0x7f8bac066b70 msgr2=0x7f8bac0ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 shutdown_connections 2024-04-25T10:20:33.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.595+0000 7f8bb2a00640 1 -- 172.21.15.87:0/231633229 wait complete. 2024-04-25T10:20:33.622 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:20:33.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.759+0000 7f2dd6600640 1 Processor -- start 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.759+0000 7f2dd6600640 1 -- start start 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dd6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dd6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2dd0072020 0x7f2dd0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dd6600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd0159920 con 0x7f2dd0159540 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dd6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd015c2f0 con 0x7f2dd0072020 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dd6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd015c5d0 con 0x7f2dd00aeda0 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2dd0072020 0x7f2dd0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:34056/0 (socket says 172.21.15.87:34056) 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcea00640 1 -- 172.21.15.87:0/4120889202 learned_addr learned my addr 172.21.15.87:0/4120889202 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2dd0072020 0x7f2dd0070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:56436/0 (socket says 172.21.15.87:56436) 2024-04-25T10:20:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcea00640 1 -- 172.21.15.87:0/4120889202 >> v2:172.21.15.169:3300/0 conn(0x7f2dd0072020 msgr2=0x7f2dd0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcea00640 1 --2- 172.21.15.87:0/4120889202 >> v2:172.21.15.169:3300/0 conn(0x7f2dd0072020 0x7f2dd0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcea00640 1 -- 172.21.15.87:0/4120889202 >> v1:172.21.15.169:6789/0 conn(0x7f2dd0159540 legacy=0x7f2dd00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.763+0000 7f2dcea00640 1 -- 172.21.15.87:0/4120889202 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd015c8b0 con 0x7f2dd00aeda0 2024-04-25T10:20:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dcea00640 1 --2- 172.21.15.87:0/4120889202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0071ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009870 tx=0x7f2dbc01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8330379b989edef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dce000640 1 -- 172.21.15.87:0/4120889202 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0a9610 con 0x7f2dd00aeda0 2024-04-25T10:20:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dce000640 1 -- 172.21.15.87:0/4120889202 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc0a97b0 con 0x7f2dd00aeda0 2024-04-25T10:20:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dce000640 1 -- 172.21.15.87:0/4120889202 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0a9b80 con 0x7f2dd00aeda0 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 -- 172.21.15.87:0/4120889202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 msgr2=0x7f2dd0071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4120889202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0071ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009870 tx=0x7f2dbc01c530 comp rx=0 tx=0).stop 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 -- 172.21.15.87:0/4120889202 shutdown_connections 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4120889202 >> v2:172.21.15.169:3300/0 conn(0x7f2dd0072020 0x7f2dd0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4120889202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0071ae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 -- 172.21.15.87:0/4120889202 >> 172.21.15.87:0/4120889202 conn(0x7f2dd00669a0 msgr2=0x7f2dd0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 -- 172.21.15.87:0/4120889202 shutdown_connections 2024-04-25T10:20:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 -- 172.21.15.87:0/4120889202 wait complete. 2024-04-25T10:20:33.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 Processor -- start 2024-04-25T10:20:33.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.767+0000 7f2dd6600640 1 -- start start 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dd0072020 0x7f2dd00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dd0159540 0x7f2dd00b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00710a0 con 0x7f2dd0159540 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0072430 con 0x7f2dd0072020 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0071220 con 0x7f2dd00aeda0 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dd0159540 0x7f2dd00b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dd0159540 0x7f2dd00b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59656/0 (socket says 172.21.15.87:59656) 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 -- 172.21.15.87:0/4070737409 learned_addr learned my addr 172.21.15.87:0/4070737409 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcea00640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcf400640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dd0072020 0x7f2dd00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 -- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 msgr2=0x7f2dd0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 -- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dd0072020 msgr2=0x7f2dd00afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dd0072020 0x7f2dd00afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 -- 172.21.15.87:0/4070737409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd00b0490 con 0x7f2dd0159540 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dcfe00640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dd0159540 0x7f2dd00b4290 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800e960 tx=0x7f2dc800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dc7400640 1 -- 172.21.15.87:0/4070737409 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8017070 con 0x7f2dd0159540 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dc7400640 1 -- 172.21.15.87:0/4070737409 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc800c9a0 con 0x7f2dd0159540 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dc7400640 1 -- 172.21.15.87:0/4070737409 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc800cd70 con 0x7f2dd0159540 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd00b0770 con 0x7f2dd0159540 2024-04-25T10:20:33.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd00b47d0 con 0x7f2dd0159540 2024-04-25T10:20:33.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd004ef30 con 0x7f2dd0159540 2024-04-25T10:20:33.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.771+0000 7f2dc7400640 1 -- 172.21.15.87:0/4070737409 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2dc801a070 con 0x7f2dd0159540 2024-04-25T10:20:33.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.775+0000 7f2dc7400640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2db0066f60 0x7f2db0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.775+0000 7f2dc7400640 1 -- 172.21.15.87:0/4070737409 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2dc8084d20 con 0x7f2dd0159540 2024-04-25T10:20:33.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.775+0000 7f2dcf400640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2db0066f60 0x7f2db0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.775+0000 7f2dcf400640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2db0066f60 0x7f2db0069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0001e00 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.779+0000 7f2dc7400640 1 -- 172.21.15.87:0/4070737409 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2dc8055940 con 0x7f2dd0159540 2024-04-25T10:20:33.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.931+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2dd00cf470 con 0x7f2dd0159540 2024-04-25T10:20:33.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.935+0000 7f2dc7400640 1 -- 172.21.15.87:0/4070737409 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2dc80599e0 con 0x7f2dd0159540 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2db0066f60 msgr2=0x7f2db0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2db0066f60 0x7f2db0069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0001e00 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dd0159540 msgr2=0x7f2dd00b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dd0159540 0x7f2dd00b4290 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800e960 tx=0x7f2dc800ee20 comp rx=0 tx=0).stop 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 shutdown_connections 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2db0066f60 0x7f2db0069420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dd0159540 0x7f2dd00b4290 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dd00aeda0 0x7f2dd0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 --2- 172.21.15.87:0/4070737409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dd0072020 0x7f2dd00afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 >> 172.21.15.87:0/4070737409 conn(0x7f2dd00669a0 msgr2=0x7f2dd00cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 shutdown_connections 2024-04-25T10:20:33.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:33.939+0000 7f2dd6600640 1 -- 172.21.15.87:0/4070737409 wait complete. 2024-04-25T10:20:33.960 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:20:33.960 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:34.046 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:34.150 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:20:34.151 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:20:34.151 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:34.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:34.160 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:20:34.160 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:34.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:34.169 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:20:34.169 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:34.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:34.211 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:20:34.212 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:34.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:34.221 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:20:34.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:34.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb12dc00640 1 Processor -- start 2024-04-25T10:20:34.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb12dc00640 1 -- start start 2024-04-25T10:20:34.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb12dc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb1281550b0 0x7fb1280cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb12dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1280cfe70 0x7fb12815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb12dc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb128071050 con 0x7fb12815b740 2024-04-25T10:20:34.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb12dc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12815d040 con 0x7fb1281550b0 2024-04-25T10:20:34.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb12dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1280712f0 con 0x7fb1280cfe70 2024-04-25T10:20:34.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb126a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb1281550b0 0x7fb1280cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb127400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb12815b740 0x7fb12806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59746/0 (socket says 172.21.15.87:59746) 2024-04-25T10:20:34.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb127400640 1 -- 172.21.15.87:0/37945925 learned_addr learned my addr 172.21.15.87:0/37945925 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:34.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb126000640 1 --2- 172.21.15.87:0/37945925 >> v2:172.21.15.169:3300/0 conn(0x7fb1280cfe70 0x7fb12815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.383+0000 7fb125600640 1 -- 172.21.15.87:0/37945925 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239864575 0 0) 0x7fb128071050 con 0x7fb12815b740 2024-04-25T10:20:34.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb125600640 1 -- 172.21.15.87:0/37945925 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb10c003530 con 0x7fb12815b740 2024-04-25T10:20:34.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb126a00640 1 -- 172.21.15.87:0/37945925 >> v2:172.21.15.169:3300/0 conn(0x7fb1280cfe70 msgr2=0x7fb12815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb126a00640 1 --2- 172.21.15.87:0/37945925 >> v2:172.21.15.169:3300/0 conn(0x7fb1280cfe70 0x7fb12815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb126a00640 1 -- 172.21.15.87:0/37945925 >> v1:172.21.15.87:6789/0 conn(0x7fb12815b740 legacy=0x7fb12806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb126a00640 1 -- 172.21.15.87:0/37945925 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280715d0 con 0x7fb1281550b0 2024-04-25T10:20:34.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb126000640 1 --2- 172.21.15.87:0/37945925 >> v2:172.21.15.169:3300/0 conn(0x7fb1280cfe70 0x7fb12815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:34.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb126a00640 1 --2- 172.21.15.87:0/37945925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1281550b0 0x7fb1280cf930 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb118009d90 tx=0x7fb11801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb07ce3e2eceb8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb125600640 1 -- 172.21.15.87:0/37945925 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180b2070 con 0x7fb1281550b0 2024-04-25T10:20:34.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb125600640 1 -- 172.21.15.87:0/37945925 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1180a5800 con 0x7fb1281550b0 2024-04-25T10:20:34.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb125600640 1 -- 172.21.15.87:0/37945925 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180a5bd0 con 0x7fb1281550b0 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 -- 172.21.15.87:0/37945925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1281550b0 msgr2=0x7fb1280cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 --2- 172.21.15.87:0/37945925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1281550b0 0x7fb1280cf930 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb118009d90 tx=0x7fb11801c930 comp rx=0 tx=0).stop 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 -- 172.21.15.87:0/37945925 shutdown_connections 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 --2- 172.21.15.87:0/37945925 >> v2:172.21.15.169:3300/0 conn(0x7fb1280cfe70 0x7fb12815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 --2- 172.21.15.87:0/37945925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1281550b0 0x7fb1280cf930 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 -- 172.21.15.87:0/37945925 >> 172.21.15.87:0/37945925 conn(0x7fb128066b70 msgr2=0x7fb128066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 -- 172.21.15.87:0/37945925 shutdown_connections 2024-04-25T10:20:34.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.387+0000 7fb12dc00640 1 -- 172.21.15.87:0/37945925 wait complete. 2024-04-25T10:20:34.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 Processor -- start 2024-04-25T10:20:34.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 -- start start 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1280cfe70 0x7fb128149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1281550b0 0x7fb1281f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb12815b740 0x7fb1281fa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12815bcd0 con 0x7fb1280cfe70 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12815bb50 con 0x7fb12815b740 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12815c030 con 0x7fb1281550b0 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1280cfe70 0x7fb128149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1280cfe70 0x7fb128149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59678/0 (socket says 172.21.15.87:59678) 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 -- 172.21.15.87:0/279190579 learned_addr learned my addr 172.21.15.87:0/279190579 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:34.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb127400640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb12815b740 0x7fb1281fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126000640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1281550b0 0x7fb1281f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 -- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1281550b0 msgr2=0x7fb1281f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1281550b0 0x7fb1281f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 -- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb12815b740 msgr2=0x7fb1281fa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb12815b740 0x7fb1281fa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 -- 172.21.15.87:0/279190579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb128156020 con 0x7fb1280cfe70 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb127400640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb12815b740 0x7fb1281fa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126000640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1281550b0 0x7fb1281f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb126a00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1280cfe70 0x7fb128149ab0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb118019540 tx=0x7fb1180196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb11ea00640 1 -- 172.21.15.87:0/279190579 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180b2040 con 0x7fb1280cfe70 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb11ea00640 1 -- 172.21.15.87:0/279190579 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1180bf070 con 0x7fb1280cfe70 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb11ea00640 1 -- 172.21.15.87:0/279190579 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180b88d0 con 0x7fb1280cfe70 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1281562e0 con 0x7fb1280cfe70 2024-04-25T10:20:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.391+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb128149ff0 con 0x7fb1280cfe70 2024-04-25T10:20:34.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.395+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12804ef30 con 0x7fb1280cfe70 2024-04-25T10:20:34.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.395+0000 7fb11ea00640 1 -- 172.21.15.87:0/279190579 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1180a5680 con 0x7fb1280cfe70 2024-04-25T10:20:34.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.395+0000 7fb11ea00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb108066f90 0x7fb108069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.395+0000 7fb11ea00640 1 -- 172.21.15.87:0/279190579 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb11811fcd0 con 0x7fb1280cfe70 2024-04-25T10:20:34.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.395+0000 7fb126000640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb108066f90 0x7fb108069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.395+0000 7fb126000640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb108066f90 0x7fb108069450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb1280cc330 tx=0x7fb114008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.399+0000 7fb11ea00640 1 -- 172.21.15.87:0/279190579 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb118016520 con 0x7fb1280cfe70 2024-04-25T10:20:34.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.583+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb128156800 con 0x7fb1280cfe70 2024-04-25T10:20:34.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb11ea00640 1 -- 172.21.15.87:0/279190579 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fb1180f0fe0 con 0x7fb1280cfe70 2024-04-25T10:20:34.589 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:34.589 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6593,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3000408220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3000408220},{"type":"v1","addr":"172.21.15.87:6837","nonce":3000408220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1417854661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1417854661},{"type":"v1","addr":"172.21.15.169:6835","nonce":1417854661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/1587280613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1587280613},{"type":"v1","addr":"172.21.15.169:6833","nonce":1587280613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7717,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4152730978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4152730978},{"type":"v1","addr":"172.21.15.87:6836","nonce":4152730978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[]} 2024-04-25T10:20:34.589 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 73 2024-04-25T10:20:34.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb108066f90 msgr2=0x7fb108069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb108066f90 0x7fb108069450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb1280cc330 tx=0x7fb114008040 comp rx=0 tx=0).stop 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1280cfe70 msgr2=0x7fb128149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1280cfe70 0x7fb128149ab0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb118019540 tx=0x7fb1180196d0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 shutdown_connections 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb108066f90 0x7fb108069450 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb12815b740 0x7fb1281fa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1281550b0 0x7fb1281f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 --2- 172.21.15.87:0/279190579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1280cfe70 0x7fb128149ab0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 >> 172.21.15.87:0/279190579 conn(0x7fb128066b70 msgr2=0x7fb1280b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 shutdown_connections 2024-04-25T10:20:34.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.587+0000 7fb12dc00640 1 -- 172.21.15.87:0/279190579 wait complete. 2024-04-25T10:20:34.610 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:20:34.610 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:20:34.610 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:20:34.610 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:34.610 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:20:34.610 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:20:34.611 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:20:34.611 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:34.611 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:20:34.611 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:20:34.611 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:20:34.611 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:34.611 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:20:34.612 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:20:34.612 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:20:34.612 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:34.623 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:20:34.624 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:20:34.626 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:20:34.626 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:20:34.626 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:20:34.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:20:34.630 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:20:34.630 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:20:34.670 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:20:34.670 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:34.671 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:20:34.671 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:34.671 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:20:34.672 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:34.672 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:20:34.672 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:34.673 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:20:34.756 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:20:34.757 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:20:34.760 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:20:34.760 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:20:34.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9641c00640 1 Processor -- start 2024-04-25T10:20:34.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9641c00640 1 -- start start 2024-04-25T10:20:34.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9641c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f963c0b2ff0 0x7f963c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9641c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963c0cf620 con 0x7f963c15a870 2024-04-25T10:20:34.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9641c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963c0cf900 con 0x7f963c0b2390 2024-04-25T10:20:34.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9641c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963c0cf3e0 con 0x7f963c0b2ff0 2024-04-25T10:20:34.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f963be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f963c15a870 0x7f963c0b28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59762/0 (socket says 172.21.15.87:59762) 2024-04-25T10:20:34.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f963be00640 1 -- 172.21.15.87:0/1012514620 learned_addr learned my addr 172.21.15.87:0/1012514620 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:34.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9633e00640 1 --2- 172.21.15.87:0/1012514620 >> v2:172.21.15.169:3300/0 conn(0x7f963c0b2ff0 0x7f963c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f963b400640 1 -- 172.21.15.87:0/1012514620 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945611975 0 0) 0x7f963c0cf620 con 0x7f963c15a870 2024-04-25T10:20:34.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f963b400640 1 -- 172.21.15.87:0/1012514620 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9620003530 con 0x7f963c15a870 2024-04-25T10:20:34.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9633e00640 1 -- 172.21.15.87:0/1012514620 >> v1:172.21.15.169:6789/0 conn(0x7f963c0b2390 legacy=0x7f963c06daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f963b400640 1 -- 172.21.15.87:0/1012514620 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519687017 0 0) 0x7f963c0cf900 con 0x7f963c0b2390 2024-04-25T10:20:34.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9633e00640 1 -- 172.21.15.87:0/1012514620 >> v1:172.21.15.87:6789/0 conn(0x7f963c15a870 legacy=0x7f963c0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.815+0000 7f9633e00640 1 -- 172.21.15.87:0/1012514620 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0cfbe0 con 0x7f963c0b2ff0 2024-04-25T10:20:34.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9633e00640 1 --2- 172.21.15.87:0/1012514620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 0x7f963c0d00e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f96240097f0 tx=0x7f962401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5e89c7f3d88def server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f963b400640 1 -- 172.21.15.87:0/1012514620 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240b0070 con 0x7f963c0b2ff0 2024-04-25T10:20:34.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f963b400640 1 -- 172.21.15.87:0/1012514620 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96240ab600 con 0x7f963c0b2ff0 2024-04-25T10:20:34.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f963b400640 1 -- 172.21.15.87:0/1012514620 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240ab9d0 con 0x7f963c0b2ff0 2024-04-25T10:20:34.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9641c00640 1 -- 172.21.15.87:0/1012514620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 msgr2=0x7f963c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9641c00640 1 --2- 172.21.15.87:0/1012514620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 0x7f963c0d00e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f96240097f0 tx=0x7f962401c410 comp rx=0 tx=0).stop 2024-04-25T10:20:34.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9641c00640 1 -- 172.21.15.87:0/1012514620 shutdown_connections 2024-04-25T10:20:34.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9641c00640 1 --2- 172.21.15.87:0/1012514620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 0x7f963c0d00e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9641c00640 1 -- 172.21.15.87:0/1012514620 >> 172.21.15.87:0/1012514620 conn(0x7f963c074310 msgr2=0x7f963c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9641c00640 1 -- 172.21.15.87:0/1012514620 shutdown_connections 2024-04-25T10:20:34.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.819+0000 7f9641c00640 1 -- 172.21.15.87:0/1012514620 wait complete. 2024-04-25T10:20:34.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 Processor -- start 2024-04-25T10:20:34.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 -- start start 2024-04-25T10:20:34.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963c0b2390 0x7f963c1f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 0x7f963c0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f963c15a870 0x7f963c150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c168450 con 0x7f963c15a870 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c06ec10 con 0x7f963c0b2ff0 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1685d0 con 0x7f963c0b2390 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f963c15a870 0x7f963c150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f963c15a870 0x7f963c150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59740/0 (socket says 172.21.15.87:59740) 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 -- 172.21.15.87:0/2311535707 learned_addr learned my addr 172.21.15.87:0/2311535707 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9633e00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 0x7f963c0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f963be00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963c0b2390 0x7f963c1f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 -- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963c0b2390 msgr2=0x7f963c1f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963c0b2390 0x7f963c1f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 -- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 msgr2=0x7f963c0aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 0x7f963c0aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 -- 172.21.15.87:0/2311535707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0af380 con 0x7f963c15a870 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f963be00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963c0b2390 0x7f963c1f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641200640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f963c15a870 0x7f963c150030 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f963400c780 tx=0x7f963400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9638c00640 1 -- 172.21.15.87:0/2311535707 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634003db0 con 0x7f963c15a870 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9638c00640 1 -- 172.21.15.87:0/2311535707 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634005070 con 0x7f963c15a870 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c1f6620 con 0x7f963c15a870 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9638c00640 1 -- 172.21.15.87:0/2311535707 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634005460 con 0x7f963c15a870 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.823+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c1f6ec0 con 0x7f963c15a870 2024-04-25T10:20:34.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.827+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c075120 con 0x7f963c15a870 2024-04-25T10:20:34.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.827+0000 7f9638c00640 1 -- 172.21.15.87:0/2311535707 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9634005600 con 0x7f963c15a870 2024-04-25T10:20:34.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.827+0000 7f9638c00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9618066f60 0x7f9618069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.827+0000 7f9638c00640 1 -- 172.21.15.87:0/2311535707 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9634084c20 con 0x7f963c15a870 2024-04-25T10:20:34.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.827+0000 7f963be00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9618066f60 0x7f9618069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.827+0000 7f963be00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9618066f60 0x7f9618069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f962c005890 tx=0x7f962c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.831+0000 7f9638c00640 1 -- 172.21.15.87:0/2311535707 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9634055f20 con 0x7f963c15a870 2024-04-25T10:20:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:34.999+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f963c150570 con 0x7f963c15a870 2024-04-25T10:20:35.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.611+0000 7f9638c00640 1 -- 172.21.15.87:0/2311535707 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9634055820 con 0x7f963c15a870 2024-04-25T10:20:35.615 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:20:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9618066f60 msgr2=0x7f9618069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9618066f60 0x7f9618069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f962c005890 tx=0x7f962c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f963c15a870 msgr2=0x7f963c150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f963c15a870 0x7f963c150030 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f963400c780 tx=0x7f963400cc40 comp rx=0 tx=0).stop 2024-04-25T10:20:35.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 shutdown_connections 2024-04-25T10:20:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9618066f60 0x7f9618069420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f963c15a870 0x7f963c150030 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f963c0b2ff0 0x7f963c0aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 --2- 172.21.15.87:0/2311535707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963c0b2390 0x7f963c1f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 >> 172.21.15.87:0/2311535707 conn(0x7f963c074310 msgr2=0x7f963c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 shutdown_connections 2024-04-25T10:20:35.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.615+0000 7f9641c00640 1 -- 172.21.15.87:0/2311535707 wait complete. 2024-04-25T10:20:35.639 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:20:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff478c00640 1 Processor -- start 2024-04-25T10:20:35.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff478c00640 1 -- start start 2024-04-25T10:20:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff478c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4741550b0 0x7ff4740cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff478c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff47415b910 0x7ff4740b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff478c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff474071050 con 0x7ff4740cfe70 2024-04-25T10:20:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff478c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47415bda0 con 0x7ff47415b910 2024-04-25T10:20:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff478c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47415d040 con 0x7ff4741550b0 2024-04-25T10:20:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.771+0000 7ff472800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff47415b910 0x7ff4740b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff472800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff47415b910 0x7ff4740b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59746/0 (socket says 172.21.15.87:59746) 2024-04-25T10:20:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff472800640 1 -- 172.21.15.87:0/4120166603 learned_addr learned my addr 172.21.15.87:0/4120166603 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff471e00640 1 --2- 172.21.15.87:0/4120166603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4741550b0 0x7ff4740cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff472800640 1 -- 172.21.15.87:0/4120166603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4741550b0 msgr2=0x7ff4740cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff472800640 1 --2- 172.21.15.87:0/4120166603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4741550b0 0x7ff4740cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff472800640 1 -- 172.21.15.87:0/4120166603 >> v1:172.21.15.169:6789/0 conn(0x7ff4740cfe70 legacy=0x7ff47415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff472800640 1 -- 172.21.15.87:0/4120166603 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740712f0 con 0x7ff47415b910 2024-04-25T10:20:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff472800640 1 --2- 172.21.15.87:0/4120166603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff4740b3260 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009e50 tx=0x7ff46c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57851a82693ed9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff471400640 1 -- 172.21.15.87:0/4120166603 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0a57f0 con 0x7ff47415b910 2024-04-25T10:20:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff471400640 1 -- 172.21.15.87:0/4120166603 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c0a5990 con 0x7ff47415b910 2024-04-25T10:20:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.775+0000 7ff471400640 1 -- 172.21.15.87:0/4120166603 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0a5d60 con 0x7ff47415b910 2024-04-25T10:20:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- 172.21.15.87:0/4120166603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 msgr2=0x7ff4740b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 --2- 172.21.15.87:0/4120166603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff4740b3260 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009e50 tx=0x7ff46c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:20:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- 172.21.15.87:0/4120166603 shutdown_connections 2024-04-25T10:20:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 --2- 172.21.15.87:0/4120166603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff4740b3260 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 --2- 172.21.15.87:0/4120166603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4741550b0 0x7ff4740cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- 172.21.15.87:0/4120166603 >> 172.21.15.87:0/4120166603 conn(0x7ff474066b70 msgr2=0x7ff474066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- 172.21.15.87:0/4120166603 shutdown_connections 2024-04-25T10:20:35.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- 172.21.15.87:0/4120166603 wait complete. 2024-04-25T10:20:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 Processor -- start 2024-04-25T10:20:35.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- start start 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4740cfe70 0x7ff47415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4741550b0 0x7ff4740adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff47413dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47415c030 con 0x7ff47415b910 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740720b0 con 0x7ff4741550b0 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff478c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47415c1b0 con 0x7ff4740cfe70 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4740cfe70 0x7ff47415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff472800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff47413dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4740cfe70 0x7ff47415a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:34130/0 (socket says 172.21.15.87:34130) 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 -- 172.21.15.87:0/2674416043 learned_addr learned my addr 172.21.15.87:0/2674416043 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:35.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff469400640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4741550b0 0x7ff4740adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 -- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4741550b0 msgr2=0x7ff4740adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4741550b0 0x7ff4740adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 -- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 msgr2=0x7ff47413dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff47413dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff471e00640 1 -- 172.21.15.87:0/2674416043 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47414cfb0 con 0x7ff4740cfe70 2024-04-25T10:20:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.779+0000 7ff472800640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff47413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:35.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff471e00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4740cfe70 0x7ff47415a440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff46400e930 tx=0x7ff46400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff46aa00640 1 -- 172.21.15.87:0/2674416043 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464017070 con 0x7ff4740cfe70 2024-04-25T10:20:35.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff46aa00640 1 -- 172.21.15.87:0/2674416043 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46400ca00 con 0x7ff4740cfe70 2024-04-25T10:20:35.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47414d290 con 0x7ff4740cfe70 2024-04-25T10:20:35.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff46aa00640 1 -- 172.21.15.87:0/2674416043 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400cdb0 con 0x7ff4740cfe70 2024-04-25T10:20:35.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47414d850 con 0x7ff4740cfe70 2024-04-25T10:20:35.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47404ef30 con 0x7ff4740cfe70 2024-04-25T10:20:35.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff46aa00640 1 -- 172.21.15.87:0/2674416043 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff46401a070 con 0x7ff4740cfe70 2024-04-25T10:20:35.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff46aa00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff450066f90 0x7ff450069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff46aa00640 1 -- 172.21.15.87:0/2674416043 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff464084f80 con 0x7ff4740cfe70 2024-04-25T10:20:35.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff469400640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff450066f90 0x7ff450069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.783+0000 7ff469400640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff450066f90 0x7ff450069450 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff4741f9d70 tx=0x7ff45c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.791+0000 7ff46aa00640 1 -- 172.21.15.87:0/2674416043 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff464055de0 con 0x7ff4740cfe70 2024-04-25T10:20:35.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.995+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff474152280 con 0x7ff4740cfe70 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.995+0000 7ff46aa00640 1 -- 172.21.15.87:0/2674416043 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff464059e80 con 0x7ff4740cfe70 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.000 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.002 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.004 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.006 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:20:36.007 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:20:36.008 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:20:36.009 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:20:36.010 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.011 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:20:36.012 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.013 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.013 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:20:36.013 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff450066f90 msgr2=0x7ff450069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff450066f90 0x7ff450069450 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff4741f9d70 tx=0x7ff45c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4740cfe70 msgr2=0x7ff47415a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4740cfe70 0x7ff47415a440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff46400e930 tx=0x7ff46400edf0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 shutdown_connections 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff450066f90 0x7ff450069450 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47415b910 0x7ff47413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4741550b0 0x7ff4740adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 --2- 172.21.15.87:0/2674416043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4740cfe70 0x7ff47415a440 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 >> 172.21.15.87:0/2674416043 conn(0x7ff474066b70 msgr2=0x7ff474151480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 shutdown_connections 2024-04-25T10:20:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:35.999+0000 7ff478c00640 1 -- 172.21.15.87:0/2674416043 wait complete. 2024-04-25T10:20:36.021 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:36.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f2b600640 1 Processor -- start 2024-04-25T10:20:36.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f2b600640 1 -- start start 2024-04-25T10:20:36.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f2b600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f24156150 0x7f1f24071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f2b600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f240670b0 0x7f1f240af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f2b600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f24156530 con 0x7f1f2415c080 2024-04-25T10:20:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f2b600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2415d220 con 0x7f1f240670b0 2024-04-25T10:20:36.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f2b600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2415c640 con 0x7f1f24156150 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f29600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f240670b0 0x7f1f240af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f29600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f240670b0 0x7f1f240af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59762/0 (socket says 172.21.15.87:59762) 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f29600640 1 -- 172.21.15.87:0/672352724 learned_addr learned my addr 172.21.15.87:0/672352724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f28c00640 1 --2- 172.21.15.87:0/672352724 >> v2:172.21.15.169:3300/0 conn(0x7f1f24156150 0x7f1f24071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f29600640 1 -- 172.21.15.87:0/672352724 >> v2:172.21.15.169:3300/0 conn(0x7f1f24156150 msgr2=0x7f1f24071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f29600640 1 --2- 172.21.15.87:0/672352724 >> v2:172.21.15.169:3300/0 conn(0x7f1f24156150 0x7f1f24071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f29600640 1 -- 172.21.15.87:0/672352724 >> v1:172.21.15.169:6789/0 conn(0x7f1f2415c080 legacy=0x7f1f240669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.155+0000 7f1f29600640 1 -- 172.21.15.87:0/672352724 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2415c920 con 0x7f1f240670b0 2024-04-25T10:20:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f29600640 1 --2- 172.21.15.87:0/672352724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f240670b0 0x7f1f240af490 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f1f24070ab0 tx=0x7f1f1c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc7bca5b1468eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f23400640 1 -- 172.21.15.87:0/672352724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0a5880 con 0x7f1f240670b0 2024-04-25T10:20:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f23400640 1 -- 172.21.15.87:0/672352724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c0a5a20 con 0x7f1f240670b0 2024-04-25T10:20:36.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f23400640 1 -- 172.21.15.87:0/672352724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0a5df0 con 0x7f1f240670b0 2024-04-25T10:20:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 -- 172.21.15.87:0/672352724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f240670b0 msgr2=0x7f1f240af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 --2- 172.21.15.87:0/672352724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f240670b0 0x7f1f240af490 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f1f24070ab0 tx=0x7f1f1c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 -- 172.21.15.87:0/672352724 shutdown_connections 2024-04-25T10:20:36.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 --2- 172.21.15.87:0/672352724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f240670b0 0x7f1f240af490 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 --2- 172.21.15.87:0/672352724 >> v2:172.21.15.169:3300/0 conn(0x7f1f24156150 0x7f1f24071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 -- 172.21.15.87:0/672352724 >> 172.21.15.87:0/672352724 conn(0x7f1f2406ac90 msgr2=0x7f1f2406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 -- 172.21.15.87:0/672352724 shutdown_connections 2024-04-25T10:20:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 -- 172.21.15.87:0/672352724 wait complete. 2024-04-25T10:20:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 Processor -- start 2024-04-25T10:20:36.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.159+0000 7f1f2b600640 1 -- start start 2024-04-25T10:20:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f240670b0 0x7f1f241413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f24156150 0x7f1f240a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f2415c080 0x7f1f240aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2415d880 con 0x7f1f24156150 2024-04-25T10:20:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2415d700 con 0x7f1f240670b0 2024-04-25T10:20:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2415da00 con 0x7f1f2415c080 2024-04-25T10:20:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f24156150 0x7f1f240a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f24156150 0x7f1f240a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59770/0 (socket says 172.21.15.87:59770) 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 -- 172.21.15.87:0/1875366045 learned_addr learned my addr 172.21.15.87:0/1875366045 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f29600640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f2415c080 0x7f1f240aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f28c00640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f240670b0 0x7f1f241413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 -- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f2415c080 msgr2=0x7f1f240aa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f2415c080 0x7f1f240aa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 -- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f240670b0 msgr2=0x7f1f241413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f240670b0 0x7f1f241413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 -- 172.21.15.87:0/1875366045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f24150df0 con 0x7f1f24156150 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f23e00640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f24156150 0x7f1f240a9cb0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c370 tx=0x7f1f1000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f20c00640 1 -- 172.21.15.87:0/1875366045 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100052d0 con 0x7f1f24156150 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f20c00640 1 -- 172.21.15.87:0/1875366045 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10005470 con 0x7f1f24156150 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f20c00640 1 -- 172.21.15.87:0/1875366045 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10007260 con 0x7f1f24156150 2024-04-25T10:20:36.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f241510d0 con 0x7f1f24156150 2024-04-25T10:20:36.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f24151690 con 0x7f1f24156150 2024-04-25T10:20:36.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.163+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2404ef30 con 0x7f1f24156150 2024-04-25T10:20:36.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.167+0000 7f1f20c00640 1 -- 172.21.15.87:0/1875366045 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f10007400 con 0x7f1f24156150 2024-04-25T10:20:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.167+0000 7f1f20c00640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f04066f60 0x7f1f04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.167+0000 7f1f20c00640 1 -- 172.21.15.87:0/1875366045 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f10084140 con 0x7f1f24156150 2024-04-25T10:20:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.167+0000 7f1f28c00640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f04066f60 0x7f1f04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.167+0000 7f1f28c00640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f04066f60 0x7f1f04069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009b60 tx=0x7f1f18009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.171+0000 7f1f20c00640 1 -- 172.21.15.87:0/1875366045 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f10054da0 con 0x7f1f24156150 2024-04-25T10:20:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f24153f10 con 0x7f1f24156150 2024-04-25T10:20:36.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f20c00640 1 -- 172.21.15.87:0/1875366045 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+12881 (secure 0 0 0) 0x7f1f10058e40 con 0x7f1f24156150 2024-04-25T10:20:36.330 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:36.330 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":107,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:20:35.627048+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f04066f60 msgr2=0x7f1f04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f04066f60 0x7f1f04069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009b60 tx=0x7f1f18009190 comp rx=0 tx=0).stop 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f24156150 msgr2=0x7f1f240a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f24156150 0x7f1f240a9cb0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c370 tx=0x7f1f1000c830 comp rx=0 tx=0).stop 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 shutdown_connections 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f04066f60 0x7f1f04069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f2415c080 0x7f1f240aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f24156150 0x7f1f240a9cb0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 --2- 172.21.15.87:0/1875366045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f240670b0 0x7f1f241413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 >> 172.21.15.87:0/1875366045 conn(0x7f1f2406ac90 msgr2=0x7f1f2406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 shutdown_connections 2024-04-25T10:20:36.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.327+0000 7f1f2b600640 1 -- 172.21.15.87:0/1875366045 wait complete. 2024-04-25T10:20:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:36.351 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:36.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.487+0000 7f1ce5000640 1 Processor -- start 2024-04-25T10:20:36.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.487+0000 7f1ce5000640 1 -- start start 2024-04-25T10:20:36.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.487+0000 7f1ce5000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1ce0066c00 0x7f1ce0161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.487+0000 7f1ce5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce00b5560 0x7f1ce00ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.487+0000 7f1ce5000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce00b40c0 con 0x7f1ce0066fe0 2024-04-25T10:20:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.487+0000 7f1ce5000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce0161d60 con 0x7f1ce0066c00 2024-04-25T10:20:36.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.487+0000 7f1ce5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00b5b00 con 0x7f1ce00b5560 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdde00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1ce0066c00 0x7f1ce0161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdd400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1ce0066fe0 0x7f1ce0072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59766/0 (socket says 172.21.15.87:59766) 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdd400640 1 -- 172.21.15.87:0/1949024964 learned_addr learned my addr 172.21.15.87:0/1949024964 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cde800640 1 --2- 172.21.15.87:0/1949024964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce00b5560 0x7f1ce00ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdde00640 1 -- 172.21.15.87:0/1949024964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce00b5560 msgr2=0x7f1ce00ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdde00640 1 --2- 172.21.15.87:0/1949024964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce00b5560 0x7f1ce00ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdde00640 1 -- 172.21.15.87:0/1949024964 >> v1:172.21.15.87:6789/0 conn(0x7f1ce0066fe0 legacy=0x7f1ce0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdde00640 1 -- 172.21.15.87:0/1949024964 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00b3a00 con 0x7f1ce0066c00 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdca00640 1 -- 172.21.15.87:0/1949024964 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802014619 0 0) 0x7f1ce00b40c0 con 0x7f1ce0066fe0 2024-04-25T10:20:36.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cde800640 1 --2- 172.21.15.87:0/1949024964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce00b5560 0x7f1ce00ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:36.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdde00640 1 --2- 172.21.15.87:0/1949024964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce0066c00 0x7f1ce0161820 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00097c0 tx=0x7f1cd001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f60cf1e1c10323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdca00640 1 -- 172.21.15.87:0/1949024964 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00b0070 con 0x7f1ce0066c00 2024-04-25T10:20:36.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdca00640 1 -- 172.21.15.87:0/1949024964 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd00ab600 con 0x7f1ce0066c00 2024-04-25T10:20:36.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.491+0000 7f1cdca00640 1 -- 172.21.15.87:0/1949024964 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00ab9d0 con 0x7f1ce0066c00 2024-04-25T10:20:36.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- 172.21.15.87:0/1949024964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce0066c00 msgr2=0x7f1ce0161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 --2- 172.21.15.87:0/1949024964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce0066c00 0x7f1ce0161820 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00097c0 tx=0x7f1cd001c380 comp rx=0 tx=0).stop 2024-04-25T10:20:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- 172.21.15.87:0/1949024964 shutdown_connections 2024-04-25T10:20:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 --2- 172.21.15.87:0/1949024964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce00b5560 0x7f1ce00ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 --2- 172.21.15.87:0/1949024964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce0066c00 0x7f1ce0161820 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- 172.21.15.87:0/1949024964 >> 172.21.15.87:0/1949024964 conn(0x7f1ce00b2560 msgr2=0x7f1ce00b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- 172.21.15.87:0/1949024964 shutdown_connections 2024-04-25T10:20:36.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- 172.21.15.87:0/1949024964 wait complete. 2024-04-25T10:20:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 Processor -- start 2024-04-25T10:20:36.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- start start 2024-04-25T10:20:36.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ce0066c00 0x7f1ce013f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce0066fe0 0x7f1ce00b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce00b5560 0x7f1ce00b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce015e540 con 0x7f1ce00b5560 2024-04-25T10:20:36.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce01585e0 con 0x7f1ce0066c00 2024-04-25T10:20:36.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1ce5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce015e6c0 con 0x7f1ce0066fe0 2024-04-25T10:20:36.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce00b5560 0x7f1ce00b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce00b5560 0x7f1ce00b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59792/0 (socket says 172.21.15.87:59792) 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 -- 172.21.15.87:0/3436286905 learned_addr learned my addr 172.21.15.87:0/3436286905 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cdd400640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce0066fe0 0x7f1ce00b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cdde00640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ce0066c00 0x7f1ce013f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 -- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce0066fe0 msgr2=0x7f1ce00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce0066fe0 0x7f1ce00b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 -- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ce0066c00 msgr2=0x7f1ce013f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ce0066c00 0x7f1ce013f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.495+0000 7f1cde800640 1 -- 172.21.15.87:0/3436286905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce0155520 con 0x7f1ce00b5560 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cde800640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce00b5560 0x7f1ce00b1a90 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800cd80 tx=0x7f1cd800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cd6000640 1 -- 172.21.15.87:0/3436286905 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800f680 con 0x7f1ce00b5560 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cd6000640 1 -- 172.21.15.87:0/3436286905 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd800f840 con 0x7f1ce00b5560 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce0155800 con 0x7f1ce00b5560 2024-04-25T10:20:36.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cd6000640 1 -- 172.21.15.87:0/3436286905 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800fc30 con 0x7f1ce00b5560 2024-04-25T10:20:36.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce0155dc0 con 0x7f1ce00b5560 2024-04-25T10:20:36.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce004ef30 con 0x7f1ce00b5560 2024-04-25T10:20:36.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cd6000640 1 -- 172.21.15.87:0/3436286905 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cd80213f0 con 0x7f1ce00b5560 2024-04-25T10:20:36.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cd6000640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cc0066f90 0x7f1cc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cd6000640 1 -- 172.21.15.87:0/3436286905 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1cd8085980 con 0x7f1ce00b5560 2024-04-25T10:20:36.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.499+0000 7f1cdde00640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cc0066f90 0x7f1cc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.503+0000 7f1cdde00640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cc0066f90 0x7f1cc0069450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00ad000 tx=0x7f1cd00a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.503+0000 7f1cd6000640 1 -- 172.21.15.87:0/3436286905 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cd8014030 con 0x7f1ce00b5560 2024-04-25T10:20:36.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.663+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ce0071c20 con 0x7f1ce00b5560 2024-04-25T10:20:36.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.663+0000 7f1cd6000640 1 -- 172.21.15.87:0/3436286905 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cd80565f0 con 0x7f1ce00b5560 2024-04-25T10:20:36.666 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:20:36.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.663+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cc0066f90 msgr2=0x7f1cc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.663+0000 7f1ce5000640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cc0066f90 0x7f1cc0069450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00ad000 tx=0x7f1cd00a6000 comp rx=0 tx=0).stop 2024-04-25T10:20:36.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.663+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce00b5560 msgr2=0x7f1ce00b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.663+0000 7f1ce5000640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce00b5560 0x7f1ce00b1a90 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800cd80 tx=0x7f1cd800a650 comp rx=0 tx=0).stop 2024-04-25T10:20:36.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 shutdown_connections 2024-04-25T10:20:36.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1cc0066f90 0x7f1cc0069450 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce00b5560 0x7f1ce00b1a90 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ce0066fe0 0x7f1ce00b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 --2- 172.21.15.87:0/3436286905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ce0066c00 0x7f1ce013f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 >> 172.21.15.87:0/3436286905 conn(0x7f1ce00b2560 msgr2=0x7f1ce00adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 shutdown_connections 2024-04-25T10:20:36.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.667+0000 7f1ce5000640 1 -- 172.21.15.87:0/3436286905 wait complete. 2024-04-25T10:20:36.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:20:36.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592fc00640 1 Processor -- start 2024-04-25T10:20:36.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592fc00640 1 -- start start 2024-04-25T10:20:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f59280d0040 0x7f592806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592fc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59280b2770 con 0x7f592815bc40 2024-04-25T10:20:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592fc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592806c670 con 0x7f59280cda10 2024-04-25T10:20:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592806c950 con 0x7f59280d0040 2024-04-25T10:20:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592d200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f592815bc40 0x7f59280b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59778/0 (socket says 172.21.15.87:59778) 2024-04-25T10:20:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592d200640 1 -- 172.21.15.87:0/3920289976 learned_addr learned my addr 172.21.15.87:0/3920289976 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:36.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592dc00640 1 --2- 172.21.15.87:0/3920289976 >> v2:172.21.15.87:3300/0 conn(0x7f59280d0040 0x7f592806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f5927e00640 1 -- 172.21.15.87:0/3920289976 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502164356 0 0) 0x7f59280b2770 con 0x7f592815bc40 2024-04-25T10:20:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f5927e00640 1 -- 172.21.15.87:0/3920289976 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f590c003530 con 0x7f592815bc40 2024-04-25T10:20:36.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592dc00640 1 -- 172.21.15.87:0/3920289976 >> v1:172.21.15.169:6789/0 conn(0x7f59280cda10 legacy=0x7f59280cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592dc00640 1 -- 172.21.15.87:0/3920289976 >> v1:172.21.15.87:6789/0 conn(0x7f592815bc40 legacy=0x7f59280b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.819+0000 7f592dc00640 1 -- 172.21.15.87:0/3920289976 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592806cc30 con 0x7f59280d0040 2024-04-25T10:20:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592dc00640 1 --2- 172.21.15.87:0/3920289976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592806c090 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d90 tx=0x7f592001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed52f5c28dc1574b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f5927e00640 1 -- 172.21.15.87:0/3920289976 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200b2070 con 0x7f59280d0040 2024-04-25T10:20:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f5927e00640 1 -- 172.21.15.87:0/3920289976 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200a58b0 con 0x7f59280d0040 2024-04-25T10:20:36.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f5927e00640 1 -- 172.21.15.87:0/3920289976 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200a5c80 con 0x7f59280d0040 2024-04-25T10:20:36.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592fc00640 1 -- 172.21.15.87:0/3920289976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 msgr2=0x7f592806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592fc00640 1 --2- 172.21.15.87:0/3920289976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592806c090 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d90 tx=0x7f592001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592fc00640 1 -- 172.21.15.87:0/3920289976 shutdown_connections 2024-04-25T10:20:36.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592fc00640 1 --2- 172.21.15.87:0/3920289976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592806c090 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592fc00640 1 -- 172.21.15.87:0/3920289976 >> 172.21.15.87:0/3920289976 conn(0x7f5928066b70 msgr2=0x7f5928066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592fc00640 1 -- 172.21.15.87:0/3920289976 shutdown_connections 2024-04-25T10:20:36.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.823+0000 7f592fc00640 1 -- 172.21.15.87:0/3920289976 wait complete. 2024-04-25T10:20:36.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 Processor -- start 2024-04-25T10:20:36.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 -- start start 2024-04-25T10:20:36.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59280cda10 0x7f592813e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592813ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592815bc40 0x7f5928159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928169980 con 0x7f59280d0040 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928169800 con 0x7f59280cda10 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5928169b00 con 0x7f592815bc40 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592813ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592813ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59802/0 (socket says 172.21.15.87:59802) 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 -- 172.21.15.87:0/2939038606 learned_addr learned my addr 172.21.15.87:0/2939038606 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592dc00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592815bc40 0x7f5928159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592d200640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59280cda10 0x7f592813e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 -- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592815bc40 msgr2=0x7f5928159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592815bc40 0x7f5928159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 -- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59280cda10 msgr2=0x7f592813e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59280cda10 0x7f592813e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 -- 172.21.15.87:0/2939038606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592813e9c0 con 0x7f59280d0040 2024-04-25T10:20:36.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f591fe00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592813ba60 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f591000c370 tx=0x7f591000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f5925600640 1 -- 172.21.15.87:0/2939038606 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100052d0 con 0x7f59280d0040 2024-04-25T10:20:36.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f5925600640 1 -- 172.21.15.87:0/2939038606 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5910005470 con 0x7f59280d0040 2024-04-25T10:20:36.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f5925600640 1 -- 172.21.15.87:0/2939038606 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910007260 con 0x7f59280d0040 2024-04-25T10:20:36.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592813eca0 con 0x7f59280d0040 2024-04-25T10:20:36.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.827+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59280b0ac0 con 0x7f59280d0040 2024-04-25T10:20:36.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.831+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592804ef30 con 0x7f59280d0040 2024-04-25T10:20:36.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.831+0000 7f5925600640 1 -- 172.21.15.87:0/2939038606 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5910007400 con 0x7f59280d0040 2024-04-25T10:20:36.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.831+0000 7f5925600640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5904066f60 0x7f5904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.831+0000 7f5925600640 1 -- 172.21.15.87:0/2939038606 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5910084180 con 0x7f59280d0040 2024-04-25T10:20:36.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.831+0000 7f592d200640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5904066f60 0x7f5904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.831+0000 7f592d200640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5904066f60 0x7f5904069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f59180011f0 tx=0x7f5918009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.835+0000 7f5925600640 1 -- 172.21.15.87:0/2939038606 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59100554d0 con 0x7f59280d0040 2024-04-25T10:20:36.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f59280b0d80 con 0x7f59280d0040 2024-04-25T10:20:36.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f5925600640 1 -- 172.21.15.87:0/2939038606 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5910054dd0 con 0x7f59280d0040 2024-04-25T10:20:36.994 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:36.994 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:20:36.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:20:36.994 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:20:36.994 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:20:36.995 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:20:36.995 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5904066f60 msgr2=0x7f5904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5904066f60 0x7f5904069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f59180011f0 tx=0x7f5918009040 comp rx=0 tx=0).stop 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 msgr2=0x7f592813ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592813ba60 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f591000c370 tx=0x7f591000c830 comp rx=0 tx=0).stop 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 shutdown_connections 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5904066f60 0x7f5904069420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592815bc40 0x7f5928159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59280d0040 0x7f592813ba60 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 --2- 172.21.15.87:0/2939038606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59280cda10 0x7f592813e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.991+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 >> 172.21.15.87:0/2939038606 conn(0x7f5928066b70 msgr2=0x7f5928066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.995+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 shutdown_connections 2024-04-25T10:20:36.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:36.995+0000 7f592fc00640 1 -- 172.21.15.87:0/2939038606 wait complete. 2024-04-25T10:20:37.015 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:20:37.016 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:20:37.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.151+0000 7fa616800640 1 Processor -- start 2024-04-25T10:20:37.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.151+0000 7fa616800640 1 -- start start 2024-04-25T10:20:37.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa616800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6100ce030 0x7fa6100ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa616800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa616800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6100cf6b0 0x7fa6100cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa616800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100cbe00 con 0x7fa6100cf6b0 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa616800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100cee90 con 0x7fa6100ce030 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa616800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61006c890 con 0x7fa61006bd30 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6100cf6b0 0x7fa6100cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6100cf6b0 0x7fa6100cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59806/0 (socket says 172.21.15.87:59806) 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 -- 172.21.15.87:0/1090391094 learned_addr learned my addr 172.21.15.87:0/1090391094 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa614a00640 1 --2- 172.21.15.87:0/1090391094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615400640 1 --2- 172.21.15.87:0/1090391094 >> v2:172.21.15.169:3300/0 conn(0x7fa6100ce030 0x7fa6100ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 -- 172.21.15.87:0/1090391094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 msgr2=0x7fa6100ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 --2- 172.21.15.87:0/1090391094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 -- 172.21.15.87:0/1090391094 >> v2:172.21.15.169:3300/0 conn(0x7fa6100ce030 msgr2=0x7fa6100ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 --2- 172.21.15.87:0/1090391094 >> v2:172.21.15.169:3300/0 conn(0x7fa6100ce030 0x7fa6100ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 -- 172.21.15.87:0/1090391094 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61006c2d0 con 0x7fa6100cf6b0 2024-04-25T10:20:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa614a00640 1 --2- 172.21.15.87:0/1090391094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615400640 1 --2- 172.21.15.87:0/1090391094 >> v2:172.21.15.169:3300/0 conn(0x7fa6100ce030 0x7fa6100ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:37.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa615e00640 1 --2- 172.21.15.87:0/1090391094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100cf6b0 0x7fa6100cfb40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa6100719d0 tx=0x7fa60801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ea043283cd5de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa60fe00640 1 -- 172.21.15.87:0/1090391094 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080ab5f0 con 0x7fa6100cf6b0 2024-04-25T10:20:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa60fe00640 1 -- 172.21.15.87:0/1090391094 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6080b0070 con 0x7fa6100cf6b0 2024-04-25T10:20:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.155+0000 7fa60fe00640 1 -- 172.21.15.87:0/1090391094 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080ab9b0 con 0x7fa6100cf6b0 2024-04-25T10:20:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- 172.21.15.87:0/1090391094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100cf6b0 msgr2=0x7fa6100cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 --2- 172.21.15.87:0/1090391094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100cf6b0 0x7fa6100cfb40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa6100719d0 tx=0x7fa60801c520 comp rx=0 tx=0).stop 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- 172.21.15.87:0/1090391094 shutdown_connections 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 --2- 172.21.15.87:0/1090391094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100cf6b0 0x7fa6100cfb40 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 --2- 172.21.15.87:0/1090391094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 --2- 172.21.15.87:0/1090391094 >> v2:172.21.15.169:3300/0 conn(0x7fa6100ce030 0x7fa6100ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- 172.21.15.87:0/1090391094 >> 172.21.15.87:0/1090391094 conn(0x7fa61009a9d0 msgr2=0x7fa6100d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- 172.21.15.87:0/1090391094 shutdown_connections 2024-04-25T10:20:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- 172.21.15.87:0/1090391094 wait complete. 2024-04-25T10:20:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 Processor -- start 2024-04-25T10:20:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- start start 2024-04-25T10:20:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100ce030 0x7fa61015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6100cf6b0 0x7fa610152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100d25b0 con 0x7fa6100ce030 2024-04-25T10:20:37.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100b13f0 con 0x7fa6100cf6b0 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa616800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100cdab0 con 0x7fa61006bd30 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100ce030 0x7fa61015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100ce030 0x7fa61015a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59814/0 (socket says 172.21.15.87:59814) 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 -- 172.21.15.87:0/3363786032 learned_addr learned my addr 172.21.15.87:0/3363786032 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa615400640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa615e00640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6100cf6b0 0x7fa610152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 -- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 msgr2=0x7fa6100b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 -- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6100cf6b0 msgr2=0x7fa610152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6100cf6b0 0x7fa610152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa614a00640 1 -- 172.21.15.87:0/3363786032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61015aad0 con 0x7fa6100ce030 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.159+0000 7fa615e00640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6100cf6b0 0x7fa610152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa614a00640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100ce030 0x7fa61015a590 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c7c0 tx=0x7fa5fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa60d600640 1 -- 172.21.15.87:0/3363786032 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc015070 con 0x7fa6100ce030 2024-04-25T10:20:37.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa60d600640 1 -- 172.21.15.87:0/3363786032 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc007a30 con 0x7fa6100ce030 2024-04-25T10:20:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6100b5450 con 0x7fa6100ce030 2024-04-25T10:20:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa60d600640 1 -- 172.21.15.87:0/3363786032 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc007e00 con 0x7fa6100ce030 2024-04-25T10:20:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6100b5ef0 con 0x7fa6100ce030 2024-04-25T10:20:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa610075120 con 0x7fa6100ce030 2024-04-25T10:20:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa60d600640 1 -- 172.21.15.87:0/3363786032 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5fc018070 con 0x7fa6100ce030 2024-04-25T10:20:37.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa60d600640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa60d600640 1 -- 172.21.15.87:0/3363786032 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa5fc084bd0 con 0x7fa6100ce030 2024-04-25T10:20:37.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.163+0000 7fa615400640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.167+0000 7fa615400640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b5f0 tx=0x7fa60400e040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.167+0000 7fa60d600640 1 -- 172.21.15.87:0/3363786032 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5fc055840 con 0x7fa6100ce030 2024-04-25T10:20:37.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.323+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa61015b0d0 con 0x7fa6100ce030 2024-04-25T10:20:37.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa60d600640 1 -- 172.21.15.87:0/3363786032 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v108) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa5fc0598e0 con 0x7fa6100ce030 2024-04-25T10:20:37.617 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:20:37.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 msgr2=0x7fa5f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b5f0 tx=0x7fa60400e040 comp rx=0 tx=0).stop 2024-04-25T10:20:37.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100ce030 msgr2=0x7fa61015a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100ce030 0x7fa61015a590 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c7c0 tx=0x7fa5fc00cc80 comp rx=0 tx=0).stop 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 shutdown_connections 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6100cf6b0 0x7fa610152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100ce030 0x7fa61015a590 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 --2- 172.21.15.87:0/3363786032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61006bd30 0x7fa6100b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 >> 172.21.15.87:0/3363786032 conn(0x7fa61009a9d0 msgr2=0x7fa61006f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 shutdown_connections 2024-04-25T10:20:37.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.615+0000 7fa616800640 1 -- 172.21.15.87:0/3363786032 wait complete. 2024-04-25T10:20:37.644 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:20:37.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd948800640 1 Processor -- start 2024-04-25T10:20:37.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd948800640 1 -- start start 2024-04-25T10:20:37.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd948800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 0x7fd9400cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd948800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9400b3ba0 0x7fd9400ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd948800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94015e6e0 con 0x7fd9400cfea0 2024-04-25T10:20:37.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd948800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9400d0300 con 0x7fd9400b3ba0 2024-04-25T10:20:37.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd948800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94015e9c0 con 0x7fd940154f70 2024-04-25T10:20:37.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd945400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd9400cfea0 0x7fd9400b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47298/0 (socket says 172.21.15.87:47298) 2024-04-25T10:20:37.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.783+0000 7fd945400640 1 -- 172.21.15.87:0/1296207809 learned_addr learned my addr 172.21.15.87:0/1296207809 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd946800640 1 --2- 172.21.15.87:0/1296207809 >> v2:172.21.15.169:3300/0 conn(0x7fd9400b3ba0 0x7fd9400ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd945e00640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 0x7fd9400cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479106500 0 0) 0x7fd94015e6e0 con 0x7fd9400cfea0 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd928003530 con 0x7fd9400cfea0 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 802001511 0 0) 0x7fd928003530 con 0x7fd9400cfea0 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 msgr2=0x7fd9400cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 0x7fd9400cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 >> v2:172.21.15.169:3300/0 conn(0x7fd9400b3ba0 msgr2=0x7fd9400ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 --2- 172.21.15.87:0/1296207809 >> v2:172.21.15.169:3300/0 conn(0x7fd9400b3ba0 0x7fd9400ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94015eca0 con 0x7fd9400cfea0 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd930001680 con 0x7fd9400cfea0 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 >> v1:172.21.15.87:6789/0 conn(0x7fd9400cfea0 legacy=0x7fd9400b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9280039c0 0x7fd928003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 0x7fd928004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928004d50 0x7fd9280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94015e9c0 con 0x7fd928004d50 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92800b9d0 con 0x7fd9280039c0 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92800bb50 con 0x7fd940154f70 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd945e00640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9280039c0 0x7fd928003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd946800640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928004d50 0x7fd9280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd945400640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 0x7fd928004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd946800640 1 -- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 msgr2=0x7fd928004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd946800640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd940154f70 0x7fd928004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd946800640 1 -- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9280039c0 msgr2=0x7fd928003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd946800640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9280039c0 0x7fd928003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.787+0000 7fd946800640 1 -- 172.21.15.87:0/1296207809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd928003530 con 0x7fd928004d50 2024-04-25T10:20:37.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd946800640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928004d50 0x7fd9280051d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009fc0 tx=0x7fd93c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89af70a8935f22fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0b2070 con 0x7fd928004d50 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0a5cd0 con 0x7fd928004d50 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd944a00640 1 -- 172.21.15.87:0/1296207809 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c009c10 con 0x7fd928004d50 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- 172.21.15.87:0/1296207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928004d50 msgr2=0x7fd9280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 --2- 172.21.15.87:0/1296207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928004d50 0x7fd9280051d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009fc0 tx=0x7fd93c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd946800640 1 -- 172.21.15.87:0/1296207809 reap_dead start 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- 172.21.15.87:0/1296207809 shutdown_connections 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- 172.21.15.87:0/1296207809 >> 172.21.15.87:0/1296207809 conn(0x7fd940066b70 msgr2=0x7fd940066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- 172.21.15.87:0/1296207809 shutdown_connections 2024-04-25T10:20:37.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- 172.21.15.87:0/1296207809 wait complete. 2024-04-25T10:20:37.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 Processor -- start 2024-04-25T10:20:37.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- start start 2024-04-25T10:20:37.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd928004d50 0x7fd940157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd940151cb0 0x7fd940152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9400adb00 0x7fd9400adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400cf960 con 0x7fd9400adb00 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940155350 con 0x7fd940151cb0 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.791+0000 7fd948800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400cfae0 con 0x7fd928004d50 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9400adb00 0x7fd9400adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9400adb00 0x7fd9400adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55908/0 (socket says 172.21.15.87:55908) 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 -- 172.21.15.87:0/1446119476 learned_addr learned my addr 172.21.15.87:0/1446119476 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd945e00640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd928004d50 0x7fd940157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd945400640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd940151cb0 0x7fd940152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 -- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd928004d50 msgr2=0x7fd940157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd928004d50 0x7fd940157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 -- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd940151cb0 msgr2=0x7fd940152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd940151cb0 0x7fd940152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 -- 172.21.15.87:0/1446119476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9401584f0 con 0x7fd9400adb00 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd946800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9400adb00 0x7fd9400adfb0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd93c002240 tx=0x7fd93c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd93a000640 1 -- 172.21.15.87:0/1446119476 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0bd070 con 0x7fd9400adb00 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd93a000640 1 -- 172.21.15.87:0/1446119476 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0a5ad0 con 0x7fd9400adb00 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd93a000640 1 -- 172.21.15.87:0/1446119476 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0b2040 con 0x7fd9400adb00 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9400ae8c0 con 0x7fd9400adb00 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9400aee80 con 0x7fd9400adb00 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94004ef30 con 0x7fd9400adb00 2024-04-25T10:20:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.795+0000 7fd93a000640 1 -- 172.21.15.87:0/1446119476 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd93c0c0070 con 0x7fd9400adb00 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.799+0000 7fd93a000640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd924066f60 0x7fd924069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.799+0000 7fd93a000640 1 -- 172.21.15.87:0/1446119476 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd93c11edb0 con 0x7fd9400adb00 2024-04-25T10:20:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.799+0000 7fd945e00640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd924066f60 0x7fd924069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.799+0000 7fd945e00640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd924066f60 0x7fd924069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd9400cef80 tx=0x7fd934009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.803+0000 7fd93a000640 1 -- 172.21.15.87:0/1446119476 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9401584f0 con 0x7fd9400adb00 2024-04-25T10:20:37.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:37.955+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd940070cf0 con 0x7fd9400adb00 2024-04-25T10:20:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.623+0000 7fd93a000640 1 -- 172.21.15.87:0/1446119476 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v109) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd93c0169d0 con 0x7fd9400adb00 2024-04-25T10:20:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:20:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.623+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd924066f60 msgr2=0x7fd924069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.623+0000 7fd948800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd924066f60 0x7fd924069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd9400cef80 tx=0x7fd934009190 comp rx=0 tx=0).stop 2024-04-25T10:20:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.623+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9400adb00 msgr2=0x7fd9400adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.623+0000 7fd948800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9400adb00 0x7fd9400adfb0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd93c002240 tx=0x7fd93c009750 comp rx=0 tx=0).stop 2024-04-25T10:20:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.627+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 shutdown_connections 2024-04-25T10:20:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.627+0000 7fd948800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd924066f60 0x7fd924069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.627+0000 7fd948800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9400adb00 0x7fd9400adfb0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.627+0000 7fd948800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd940151cb0 0x7fd940152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.627+0000 7fd948800640 1 --2- 172.21.15.87:0/1446119476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd928004d50 0x7fd940157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.627+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 >> 172.21.15.87:0/1446119476 conn(0x7fd940066b70 msgr2=0x7fd94006d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.627+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 shutdown_connections 2024-04-25T10:20:38.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.631+0000 7fd948800640 1 -- 172.21.15.87:0/1446119476 wait complete. 2024-04-25T10:20:38.666 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f4000640 1 Processor -- start 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f4000640 1 -- start start 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f4000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f11ec15bbd0 0x7f11ec0d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f4000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f4000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec0cfba0 con 0x7f11ec0cf530 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f4000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec0cf970 con 0x7f11ec15bbd0 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f4000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0b47b0 con 0x7f11ec0d0630 2024-04-25T10:20:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec0b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:56490/0 (socket says 172.21.15.87:56490) 2024-04-25T10:20:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f2000640 1 -- 172.21.15.87:0/379320174 learned_addr learned my addr 172.21.15.87:0/379320174 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f1600640 1 --2- 172.21.15.87:0/379320174 >> v2:172.21.15.169:3300/0 conn(0x7f11ec15bbd0 0x7f11ec0d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11ebe00640 1 -- 172.21.15.87:0/379320174 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438482975 0 0) 0x7f11ec0cfba0 con 0x7f11ec0cf530 2024-04-25T10:20:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11ebe00640 1 -- 172.21.15.87:0/379320174 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d4003530 con 0x7f11ec0cf530 2024-04-25T10:20:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f2000640 1 -- 172.21.15.87:0/379320174 >> v2:172.21.15.169:3300/0 conn(0x7f11ec15bbd0 msgr2=0x7f11ec0d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f2000640 1 --2- 172.21.15.87:0/379320174 >> v2:172.21.15.169:3300/0 conn(0x7f11ec15bbd0 0x7f11ec0d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f2000640 1 -- 172.21.15.87:0/379320174 >> v1:172.21.15.169:6789/0 conn(0x7f11ec0cf530 legacy=0x7f11ec0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f2000640 1 -- 172.21.15.87:0/379320174 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0b4a00 con 0x7f11ec0d0630 2024-04-25T10:20:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.803+0000 7f11f1600640 1 --2- 172.21.15.87:0/379320174 >> v2:172.21.15.169:3300/0 conn(0x7f11ec15bbd0 0x7f11ec0d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f2000640 1 --2- 172.21.15.87:0/379320174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec0b54e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009e50 tx=0x7f11e401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae2065e6e9a9d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11ebe00640 1 -- 172.21.15.87:0/379320174 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40a58d0 con 0x7f11ec0d0630 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11ebe00640 1 -- 172.21.15.87:0/379320174 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e40a5a70 con 0x7f11ec0d0630 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11ebe00640 1 -- 172.21.15.87:0/379320174 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40a5e40 con 0x7f11ec0d0630 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 -- 172.21.15.87:0/379320174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 msgr2=0x7f11ec0b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 --2- 172.21.15.87:0/379320174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec0b54e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009e50 tx=0x7f11e401ca00 comp rx=0 tx=0).stop 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 -- 172.21.15.87:0/379320174 shutdown_connections 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 --2- 172.21.15.87:0/379320174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec0b54e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 --2- 172.21.15.87:0/379320174 >> v2:172.21.15.169:3300/0 conn(0x7f11ec15bbd0 0x7f11ec0d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 -- 172.21.15.87:0/379320174 >> 172.21.15.87:0/379320174 conn(0x7f11ec0d1960 msgr2=0x7f11ec0d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 -- 172.21.15.87:0/379320174 shutdown_connections 2024-04-25T10:20:38.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.807+0000 7f11f4000640 1 -- 172.21.15.87:0/379320174 wait complete. 2024-04-25T10:20:38.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 Processor -- start 2024-04-25T10:20:38.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 -- start start 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11ec0cf530 0x7f11ec15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11ec15bbd0 0x7f11ec156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0b6230 con 0x7f11ec0cf530 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec15c1c0 con 0x7f11ec15bbd0 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0b63b0 con 0x7f11ec0d0630 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11ec0cf530 0x7f11ec15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11ec0cf530 0x7f11ec15d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55918/0 (socket says 172.21.15.87:55918) 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 -- 172.21.15.87:0/1148347008 learned_addr learned my addr 172.21.15.87:0/1148347008 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f0c00640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f2000640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11ec15bbd0 0x7f11ec156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 -- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 msgr2=0x7f11ec156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 -- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11ec15bbd0 msgr2=0x7f11ec156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11ec15bbd0 0x7f11ec156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 -- 172.21.15.87:0/1148347008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec06aea0 con 0x7f11ec0cf530 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f1600640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11ec0cf530 0x7f11ec15d5f0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f11e000e9b0 tx=0x7f11e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11e9600640 1 -- 172.21.15.87:0/1148347008 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0017070 con 0x7f11ec0cf530 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11e9600640 1 -- 172.21.15.87:0/1148347008 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e000ca00 con 0x7f11ec0cf530 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec06b180 con 0x7f11ec0cf530 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11e9600640 1 -- 172.21.15.87:0/1148347008 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000cdd0 con 0x7f11ec0cf530 2024-04-25T10:20:38.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.811+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec06bc60 con 0x7f11ec0cf530 2024-04-25T10:20:38.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.815+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec04ef30 con 0x7f11ec0cf530 2024-04-25T10:20:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.815+0000 7f11e9600640 1 -- 172.21.15.87:0/1148347008 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11e001a070 con 0x7f11ec0cf530 2024-04-25T10:20:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.815+0000 7f11e9600640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11cc066f90 0x7f11cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.815+0000 7f11e9600640 1 -- 172.21.15.87:0/1148347008 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f11e0085600 con 0x7f11ec0cf530 2024-04-25T10:20:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.815+0000 7f11f0c00640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11cc066f90 0x7f11cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.815+0000 7f11f0c00640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11cc066f90 0x7f11cc069450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f11ec1667d0 tx=0x7f11d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:38.819+0000 7f11e9600640 1 -- 172.21.15.87:0/1148347008 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11e0056000 con 0x7f11ec0cf530 2024-04-25T10:20:39.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.007+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f11ec158240 con 0x7f11ec0cf530 2024-04-25T10:20:39.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.631+0000 7f11e9600640 1 -- 172.21.15.87:0/1148347008 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7f11e005a0a0 con 0x7f11ec0cf530 2024-04-25T10:20:39.639 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T10:20:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11cc066f90 msgr2=0x7f11cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11cc066f90 0x7f11cc069450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f11ec1667d0 tx=0x7f11d8009040 comp rx=0 tx=0).stop 2024-04-25T10:20:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11ec0cf530 msgr2=0x7f11ec15d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11ec0cf530 0x7f11ec15d5f0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f11e000e9b0 tx=0x7f11e000ee70 comp rx=0 tx=0).stop 2024-04-25T10:20:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 shutdown_connections 2024-04-25T10:20:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11cc066f90 0x7f11cc069450 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11ec15bbd0 0x7f11ec156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11ec0d0630 0x7f11ec156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 --2- 172.21.15.87:0/1148347008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11ec0cf530 0x7f11ec15d5f0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.643+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 >> 172.21.15.87:0/1148347008 conn(0x7f11ec0d1960 msgr2=0x7f11ec15ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.647+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 shutdown_connections 2024-04-25T10:20:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.647+0000 7f11f4000640 1 -- 172.21.15.87:0/1148347008 wait complete. 2024-04-25T10:20:39.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.815+0000 7fb885400640 1 Processor -- start 2024-04-25T10:20:39.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.815+0000 7fb885400640 1 -- start start 2024-04-25T10:20:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb885400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb8801550b0 0x7fb8800cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb885400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 0x7fb88015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb885400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb880071050 con 0x7fb88015b740 2024-04-25T10:20:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb885400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88015d040 con 0x7fb8801550b0 2024-04-25T10:20:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb885400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800712f0 con 0x7fb8800cfe70 2024-04-25T10:20:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb88015b740 0x7fb88006d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47304/0 (socket says 172.21.15.87:47304) 2024-04-25T10:20:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 -- 172.21.15.87:0/4211905150 learned_addr learned my addr 172.21.15.87:0/4211905150 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399397976 0 0) 0x7fb880071050 con 0x7fb88015b740 2024-04-25T10:20:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87e800640 1 --2- 172.21.15.87:0/4211905150 >> v2:172.21.15.169:3300/0 conn(0x7fb8801550b0 0x7fb8800cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87de00640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 0x7fb88015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb864005740 con 0x7fb88015b740 2024-04-25T10:20:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570459958 0 0) 0x7fb864005740 con 0x7fb88015b740 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 msgr2=0x7fb88015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 0x7fb88015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 >> v2:172.21.15.169:3300/0 conn(0x7fb8801550b0 msgr2=0x7fb8800cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 --2- 172.21.15.87:0/4211905150 >> v2:172.21.15.169:3300/0 conn(0x7fb8801550b0 0x7fb8800cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800715d0 con 0x7fb88015b740 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb878001a00 con 0x7fb88015b740 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 >> v1:172.21.15.87:6789/0 conn(0x7fb88015b740 legacy=0x7fb88006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 0x7fb864005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006150 0x7fb864006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb864006f60 0x7fb8640073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88015d040 con 0x7fb864006f60 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800712f0 con 0x7fb864006150 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86400dbe0 con 0x7fb8800cfe70 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb864006f60 0x7fb8640073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87e800640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 0x7fb864005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87de00640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006150 0x7fb864006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 -- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 msgr2=0x7fb864005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 0x7fb864005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 -- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006150 msgr2=0x7fb864006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006150 0x7fb864006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87f200640 1 -- 172.21.15.87:0/4211905150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb864005740 con 0x7fb864006f60 2024-04-25T10:20:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.819+0000 7fb87e800640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8800cfe70 0x7fb864005bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb87f200640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb864006f60 0x7fb8640073e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fb8780013e0 tx=0x7fb87801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6762f5698894fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800a4e0 con 0x7fb864006f60 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8780072f0 con 0x7fb864006f60 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb87d400640 1 -- 172.21.15.87:0/4211905150 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8780076e0 con 0x7fb864006f60 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb885400640 1 -- 172.21.15.87:0/4211905150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb864006f60 msgr2=0x7fb8640073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb885400640 1 --2- 172.21.15.87:0/4211905150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb864006f60 0x7fb8640073e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fb8780013e0 tx=0x7fb87801d340 comp rx=0 tx=0).stop 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb87f200640 1 -- 172.21.15.87:0/4211905150 reap_dead start 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb885400640 1 -- 172.21.15.87:0/4211905150 shutdown_connections 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb885400640 1 -- 172.21.15.87:0/4211905150 >> 172.21.15.87:0/4211905150 conn(0x7fb880066b70 msgr2=0x7fb880066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb885400640 1 -- 172.21.15.87:0/4211905150 shutdown_connections 2024-04-25T10:20:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.823+0000 7fb885400640 1 -- 172.21.15.87:0/4211905150 wait complete. 2024-04-25T10:20:39.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 Processor -- start 2024-04-25T10:20:39.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 -- start start 2024-04-25T10:20:39.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006f60 0x7fb880156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8801451e0 0x7fb880151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb88006d0e0 0x7fb88006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800720b0 con 0x7fb88006d0e0 2024-04-25T10:20:39.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880071b90 con 0x7fb864006f60 2024-04-25T10:20:39.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88015bbd0 con 0x7fb8801451e0 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb88006d0e0 0x7fb88006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb88006d0e0 0x7fb88006d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55928/0 (socket says 172.21.15.87:55928) 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 -- 172.21.15.87:0/3455325268 learned_addr learned my addr 172.21.15.87:0/3455325268 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87e800640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006f60 0x7fb880156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87de00640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8801451e0 0x7fb880151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 -- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8801451e0 msgr2=0x7fb880151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8801451e0 0x7fb880151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 -- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006f60 msgr2=0x7fb880156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006f60 0x7fb880156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 -- 172.21.15.87:0/3455325268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880157350 con 0x7fb88006d0e0 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87de00640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8801451e0 0x7fb880151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb87f200640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb88006d0e0 0x7fb88006d560 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb878007040 tx=0x7fb864007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb876a00640 1 -- 172.21.15.87:0/3455325268 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8700096b0 con 0x7fb88006d0e0 2024-04-25T10:20:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb876a00640 1 -- 172.21.15.87:0/3455325268 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87800a4c0 con 0x7fb88006d0e0 2024-04-25T10:20:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb876a00640 1 -- 172.21.15.87:0/3455325268 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8780074a0 con 0x7fb88006d0e0 2024-04-25T10:20:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb880157630 con 0x7fb88006d0e0 2024-04-25T10:20:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb880157bf0 con 0x7fb88006d0e0 2024-04-25T10:20:39.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.827+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88004ef30 con 0x7fb88006d0e0 2024-04-25T10:20:39.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.831+0000 7fb876a00640 1 -- 172.21.15.87:0/3455325268 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8780bd020 con 0x7fb88006d0e0 2024-04-25T10:20:39.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.831+0000 7fb876a00640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb860066f90 0x7fb860069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.831+0000 7fb876a00640 1 -- 172.21.15.87:0/3455325268 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb87811ee40 con 0x7fb88006d0e0 2024-04-25T10:20:39.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.831+0000 7fb87e800640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb860066f90 0x7fb860069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.831+0000 7fb87e800640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb860066f90 0x7fb860069450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb870009d60 tx=0x7fb870009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.835+0000 7fb876a00640 1 -- 172.21.15.87:0/3455325268 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8780170b0 con 0x7fb88006d0e0 2024-04-25T10:20:39.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.987+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8801547e0 con 0x7fb88006d0e0 2024-04-25T10:20:39.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb876a00640 1 -- 172.21.15.87:0/3455325268 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16628 (secure 0 0 0) 0x7fb8780f0000 con 0x7fb88006d0e0 2024-04-25T10:20:39.993 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:39.993 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":110,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:20:39.010262+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:37.371368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:38.004409+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb860066f90 msgr2=0x7fb860069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb860066f90 0x7fb860069450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb870009d60 tx=0x7fb870009190 comp rx=0 tx=0).stop 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb88006d0e0 msgr2=0x7fb88006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb88006d0e0 0x7fb88006d560 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb878007040 tx=0x7fb864007a50 comp rx=0 tx=0).stop 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 shutdown_connections 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb860066f90 0x7fb860069450 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb88006d0e0 0x7fb88006d560 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8801451e0 0x7fb880151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 --2- 172.21.15.87:0/3455325268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb864006f60 0x7fb880156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 >> 172.21.15.87:0/3455325268 conn(0x7fb880066b70 msgr2=0x7fb8800afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 shutdown_connections 2024-04-25T10:20:39.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:39.991+0000 7fb885400640 1 -- 172.21.15.87:0/3455325268 wait complete. 2024-04-25T10:20:40.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.147+0000 7f1f8b800640 1 Processor -- start 2024-04-25T10:20:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.147+0000 7f1f8b800640 1 -- start start 2024-04-25T10:20:40.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.147+0000 7f1f8b800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f840d0040 0x7f1f8415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.147+0000 7f1f8b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f8415b740 0x7f1f840b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.147+0000 7f1f8b800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f84071050 con 0x7f1f841550b0 2024-04-25T10:20:40.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.147+0000 7f1f8b800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8415d040 con 0x7f1f840d0040 2024-04-25T10:20:40.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.147+0000 7f1f8b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840712f0 con 0x7f1f8415b740 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f840d0040 0x7f1f8415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f88e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1f841550b0 0x7f1f840cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47312/0 (socket says 172.21.15.87:47312) 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f88e00640 1 -- 172.21.15.87:0/1506084179 learned_addr learned my addr 172.21.15.87:0/1506084179 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f89800640 1 --2- 172.21.15.87:0/1506084179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f8415b740 0x7f1f840b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83e00640 1 -- 172.21.15.87:0/1506084179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f8415b740 msgr2=0x7f1f840b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83e00640 1 --2- 172.21.15.87:0/1506084179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f8415b740 0x7f1f840b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83e00640 1 -- 172.21.15.87:0/1506084179 >> v1:172.21.15.87:6789/0 conn(0x7f1f841550b0 legacy=0x7f1f840cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83400640 1 -- 172.21.15.87:0/1506084179 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331999237 0 0) 0x7f1f84071050 con 0x7f1f841550b0 2024-04-25T10:20:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83e00640 1 -- 172.21.15.87:0/1506084179 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840715d0 con 0x7f1f840d0040 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83e00640 1 --2- 172.21.15.87:0/1506084179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f840d0040 0x7f1f8415b200 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1f740097c0 tx=0x7f1f7401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470e00fcb48ecdf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83400640 1 -- 172.21.15.87:0/1506084179 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740a7070 con 0x7f1f840d0040 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83400640 1 -- 172.21.15.87:0/1506084179 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f740b0070 con 0x7f1f840d0040 2024-04-25T10:20:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.151+0000 7f1f83400640 1 -- 172.21.15.87:0/1506084179 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740ab890 con 0x7f1f840d0040 2024-04-25T10:20:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- 172.21.15.87:0/1506084179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f840d0040 msgr2=0x7f1f8415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 --2- 172.21.15.87:0/1506084179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f840d0040 0x7f1f8415b200 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1f740097c0 tx=0x7f1f7401c470 comp rx=0 tx=0).stop 2024-04-25T10:20:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- 172.21.15.87:0/1506084179 shutdown_connections 2024-04-25T10:20:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 --2- 172.21.15.87:0/1506084179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f8415b740 0x7f1f840b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 --2- 172.21.15.87:0/1506084179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f840d0040 0x7f1f8415b200 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- 172.21.15.87:0/1506084179 >> 172.21.15.87:0/1506084179 conn(0x7f1f84066b70 msgr2=0x7f1f84066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- 172.21.15.87:0/1506084179 shutdown_connections 2024-04-25T10:20:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- 172.21.15.87:0/1506084179 wait complete. 2024-04-25T10:20:40.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 Processor -- start 2024-04-25T10:20:40.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- start start 2024-04-25T10:20:40.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f840d0040 0x7f1f84147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f841550b0 0x7f1f840cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f8415b740 0x7f1f84158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8415bcd0 con 0x7f1f8415b740 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8415bb50 con 0x7f1f841550b0 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f8b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8415c030 con 0x7f1f840d0040 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f8415b740 0x7f1f84158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f8415b740 0x7f1f84158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55936/0 (socket says 172.21.15.87:55936) 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 -- 172.21.15.87:0/79241809 learned_addr learned my addr 172.21.15.87:0/79241809 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f88e00640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f840d0040 0x7f1f84147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f83e00640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f841550b0 0x7f1f840cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 -- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f840d0040 msgr2=0x7f1f84147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f840d0040 0x7f1f84147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 -- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f841550b0 msgr2=0x7f1f840cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f841550b0 0x7f1f840cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.155+0000 7f1f89800640 1 -- 172.21.15.87:0/79241809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840cd1d0 con 0x7f1f8415b740 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f89800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f8415b740 0x7f1f84158c20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009d30 tx=0x7f1f7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f80c00640 1 -- 172.21.15.87:0/79241809 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c018020 con 0x7f1f8415b740 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f80c00640 1 -- 172.21.15.87:0/79241809 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c014480 con 0x7f1f8415b740 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f80c00640 1 -- 172.21.15.87:0/79241809 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c014870 con 0x7f1f8415b740 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f840cd4b0 con 0x7f1f8415b740 2024-04-25T10:20:40.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f841558a0 con 0x7f1f8415b740 2024-04-25T10:20:40.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8404ef30 con 0x7f1f8415b740 2024-04-25T10:20:40.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f80c00640 1 -- 172.21.15.87:0/79241809 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f7c01b020 con 0x7f1f8415b740 2024-04-25T10:20:40.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f80c00640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f60066f60 0x7f1f60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f80c00640 1 -- 172.21.15.87:0/79241809 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f7c085ae0 con 0x7f1f8415b740 2024-04-25T10:20:40.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.159+0000 7f1f88e00640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f60066f60 0x7f1f60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.163+0000 7f1f88e00640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f60066f60 0x7f1f60069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000f70 tx=0x7f1f6c007b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.163+0000 7f1f80c00640 1 -- 172.21.15.87:0/79241809 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f7c022070 con 0x7f1f8415b740 2024-04-25T10:20:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.319+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f8415d9b0 con 0x7f1f8415b740 2024-04-25T10:20:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.319+0000 7f1f80c00640 1 -- 172.21.15.87:0/79241809 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16628 (secure 0 0 0) 0x7f1f7c010040 con 0x7f1f8415b740 2024-04-25T10:20:40.323 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:40.324 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":110,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:20:39.010262+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:37.371368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:38.004409+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f60066f60 msgr2=0x7f1f60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f60066f60 0x7f1f60069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000f70 tx=0x7f1f6c007b60 comp rx=0 tx=0).stop 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f8415b740 msgr2=0x7f1f84158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f8415b740 0x7f1f84158c20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009d30 tx=0x7f1f7c00c490 comp rx=0 tx=0).stop 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 shutdown_connections 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1f60066f60 0x7f1f60069420 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f8415b740 0x7f1f84158c20 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f841550b0 0x7f1f840cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 --2- 172.21.15.87:0/79241809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f840d0040 0x7f1f84147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 >> 172.21.15.87:0/79241809 conn(0x7f1f84066b70 msgr2=0x7f1f84154060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 shutdown_connections 2024-04-25T10:20:40.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.323+0000 7f1f8b800640 1 -- 172.21.15.87:0/79241809 wait complete. 2024-04-25T10:20:40.346 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:20:40.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.479+0000 7f623c400640 1 Processor -- start 2024-04-25T10:20:40.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.479+0000 7f623c400640 1 -- start start 2024-04-25T10:20:40.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f623c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62340aeda0 0x7f6234071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f623c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6234072020 0x7f6234070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f623c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6234159920 con 0x7f6234159540 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f623c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623415c2f0 con 0x7f62340aeda0 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f623c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623415c5d0 con 0x7f6234072020 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f6239000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62340aeda0 0x7f6234071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f6239000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62340aeda0 0x7f6234071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55948/0 (socket says 172.21.15.87:55948) 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f6239000640 1 -- 172.21.15.87:0/2696056593 learned_addr learned my addr 172.21.15.87:0/2696056593 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f623a400640 1 --2- 172.21.15.87:0/2696056593 >> v2:172.21.15.169:3300/0 conn(0x7f6234072020 0x7f6234070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f6239000640 1 -- 172.21.15.87:0/2696056593 >> v2:172.21.15.169:3300/0 conn(0x7f6234072020 msgr2=0x7f6234070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f6239000640 1 --2- 172.21.15.87:0/2696056593 >> v2:172.21.15.169:3300/0 conn(0x7f6234072020 0x7f6234070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f6239000640 1 -- 172.21.15.87:0/2696056593 >> v1:172.21.15.169:6789/0 conn(0x7f6234159540 legacy=0x7f62340ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.483+0000 7f6239000640 1 -- 172.21.15.87:0/2696056593 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623415c8b0 con 0x7f62340aeda0 2024-04-25T10:20:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f6239000640 1 --2- 172.21.15.87:0/2696056593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62340aeda0 0x7f6234071ae0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f62340cd230 tx=0x7f622001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e783f4cd7281ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f6233e00640 1 -- 172.21.15.87:0/2696056593 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200aa430 con 0x7f62340aeda0 2024-04-25T10:20:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f6233e00640 1 -- 172.21.15.87:0/2696056593 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62200af070 con 0x7f62340aeda0 2024-04-25T10:20:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f6233e00640 1 -- 172.21.15.87:0/2696056593 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200aa7f0 con 0x7f62340aeda0 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 -- 172.21.15.87:0/2696056593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62340aeda0 msgr2=0x7f6234071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 --2- 172.21.15.87:0/2696056593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62340aeda0 0x7f6234071ae0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f62340cd230 tx=0x7f622001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 -- 172.21.15.87:0/2696056593 shutdown_connections 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 --2- 172.21.15.87:0/2696056593 >> v2:172.21.15.169:3300/0 conn(0x7f6234072020 0x7f6234070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 --2- 172.21.15.87:0/2696056593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62340aeda0 0x7f6234071ae0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 -- 172.21.15.87:0/2696056593 >> 172.21.15.87:0/2696056593 conn(0x7f62340669a0 msgr2=0x7f6234066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 -- 172.21.15.87:0/2696056593 shutdown_connections 2024-04-25T10:20:40.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 -- 172.21.15.87:0/2696056593 wait complete. 2024-04-25T10:20:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 Processor -- start 2024-04-25T10:20:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.487+0000 7f623c400640 1 -- start start 2024-04-25T10:20:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6234072020 0x7f623414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62340aeda0 0x7f6234150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6234159540 0x7f62340b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340710a0 con 0x7f6234159540 2024-04-25T10:20:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234072430 con 0x7f6234072020 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6234071220 con 0x7f62340aeda0 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6234159540 0x7f62340b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6234159540 0x7f62340b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55964/0 (socket says 172.21.15.87:55964) 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 -- 172.21.15.87:0/3360738878 learned_addr learned my addr 172.21.15.87:0/3360738878 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f6239000640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62340aeda0 0x7f6234150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f6239a00640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6234072020 0x7f623414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 -- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62340aeda0 msgr2=0x7f6234150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62340aeda0 0x7f6234150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 -- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6234072020 msgr2=0x7f623414d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6234072020 0x7f623414d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 -- 172.21.15.87:0/3360738878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234153b00 con 0x7f6234159540 2024-04-25T10:20:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f6239a00640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6234072020 0x7f623414d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623a400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6234159540 0x7f62340b0f80 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f622c007e90 tx=0x7f622c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f6231600640 1 -- 172.21.15.87:0/3360738878 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c014420 con 0x7f6234159540 2024-04-25T10:20:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f6231600640 1 -- 172.21.15.87:0/3360738878 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c019070 con 0x7f6234159540 2024-04-25T10:20:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f6231600640 1 -- 172.21.15.87:0/3360738878 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c014800 con 0x7f6234159540 2024-04-25T10:20:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6234153de0 con 0x7f6234159540 2024-04-25T10:20:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62340af440 con 0x7f6234159540 2024-04-25T10:20:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623404ef30 con 0x7f6234159540 2024-04-25T10:20:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.491+0000 7f6231600640 1 -- 172.21.15.87:0/3360738878 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f622c0149a0 con 0x7f6234159540 2024-04-25T10:20:40.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.495+0000 7f6231600640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6214066f60 0x7f6214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.495+0000 7f6231600640 1 -- 172.21.15.87:0/3360738878 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f622c085850 con 0x7f6234159540 2024-04-25T10:20:40.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.495+0000 7f6239a00640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6214066f60 0x7f6214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.495+0000 7f6239a00640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6214066f60 0x7f6214069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f62340cbc70 tx=0x7f6228009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.499+0000 7f6231600640 1 -- 172.21.15.87:0/3360738878 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f622c0567c0 con 0x7f6234159540 2024-04-25T10:20:40.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:40.687+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f62340cf470 con 0x7f6234159540 2024-04-25T10:20:41.618 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:20:41.615+0000 7f5c9a600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:20:41.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f6231600640 1 -- 172.21.15.87:0/3360738878 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7f622c0560c0 con 0x7f6234159540 2024-04-25T10:20:41.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6214066f60 msgr2=0x7f6214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6214066f60 0x7f6214069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f62340cbc70 tx=0x7f6228009040 comp rx=0 tx=0).stop 2024-04-25T10:20:41.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6234159540 msgr2=0x7f62340b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6234159540 0x7f62340b0f80 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f622c007e90 tx=0x7f622c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 shutdown_connections 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6214066f60 0x7f6214069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6234159540 0x7f62340b0f80 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62340aeda0 0x7f6234150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 --2- 172.21.15.87:0/3360738878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6234072020 0x7f623414d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 >> 172.21.15.87:0/3360738878 conn(0x7f62340669a0 msgr2=0x7f623415d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 shutdown_connections 2024-04-25T10:20:41.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.647+0000 7f623c400640 1 -- 172.21.15.87:0/3360738878 wait complete. 2024-04-25T10:20:41.674 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:20:41.674 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:20:41.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0261000640 1 Processor -- start 2024-04-25T10:20:41.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0261000640 1 -- start start 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0261000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f025c0b38d0 0x7f025c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0261000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c15b080 con 0x7f025c0aef80 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0261000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c15adb0 con 0x7f025c070350 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0261000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c15eb60 con 0x7f025c0b38d0 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f025c0aef80 0x7f025c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47322/0 (socket says 172.21.15.87:47322) 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259e00640 1 -- 172.21.15.87:0/884678088 learned_addr learned my addr 172.21.15.87:0/884678088 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f025a800640 1 --2- 172.21.15.87:0/884678088 >> v2:172.21.15.169:3300/0 conn(0x7f025c0b38d0 0x7f025c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959794969 0 0) 0x7f025c15b080 con 0x7f025c0aef80 2024-04-25T10:20:41.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0240003530 con 0x7f025c0aef80 2024-04-25T10:20:41.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501259581 0 0) 0x7f0240003530 con 0x7f025c0aef80 2024-04-25T10:20:41.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 >> v2:172.21.15.169:3300/0 conn(0x7f025c0b38d0 msgr2=0x7f025c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:41.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 --2- 172.21.15.87:0/884678088 >> v2:172.21.15.169:3300/0 conn(0x7f025c0b38d0 0x7f025c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 >> v1:172.21.15.169:6789/0 conn(0x7f025c070350 legacy=0x7f025c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c15ee40 con 0x7f025c0aef80 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f024c001680 con 0x7f025c0aef80 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 >> v1:172.21.15.87:6789/0 conn(0x7f025c0aef80 legacy=0x7f025c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f0240003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0240004810 0x7f0240004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02400051d0 0x7f024000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15eb60 con 0x7f02400039c0 2024-04-25T10:20:41.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024000c070 con 0x7f02400051d0 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.815+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024000c270 con 0x7f0240004810 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259e00640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f0240003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f025a800640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02400051d0 0x7f024000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0251400640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0240004810 0x7f0240004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259e00640 1 -- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0240004810 msgr2=0x7f0240004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259e00640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0240004810 0x7f0240004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259e00640 1 -- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02400051d0 msgr2=0x7f024000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259e00640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02400051d0 0x7f024000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259e00640 1 -- 172.21.15.87:0/884678088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0240003530 con 0x7f02400039c0 2024-04-25T10:20:41.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f025a800640 1 -- 172.21.15.87:0/884678088 reap_dead start 2024-04-25T10:20:41.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259e00640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f0240003de0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c210 tx=0x7f024c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28d56d9904fbf6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c007d40 con 0x7f02400039c0 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c0a9070 con 0x7f02400039c0 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.819+0000 7f0259400640 1 -- 172.21.15.87:0/884678088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c005d80 con 0x7f02400039c0 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- 172.21.15.87:0/884678088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 msgr2=0x7f0240003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f0240003de0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c210 tx=0x7f024c01cc20 comp rx=0 tx=0).stop 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- 172.21.15.87:0/884678088 shutdown_connections 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 --2- 172.21.15.87:0/884678088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f0240003de0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- 172.21.15.87:0/884678088 >> 172.21.15.87:0/884678088 conn(0x7f025c0669a0 msgr2=0x7f025c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- 172.21.15.87:0/884678088 shutdown_connections 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- 172.21.15.87:0/884678088 wait complete. 2024-04-25T10:20:41.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 Processor -- start 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- start start 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f025c1f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f025c13cc60 0x7f025c13d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c151b80 0x7f025c152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0b35d0 con 0x7f02400039c0 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15bd90 con 0x7f025c151b80 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0261000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0b3750 con 0x7f025c13cc60 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f025c1f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f025c1f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55978/0 (socket says 172.21.15.87:55978) 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 -- 172.21.15.87:0/2552271499 learned_addr learned my addr 172.21.15.87:0/2552271499 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f025a800640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c151b80 0x7f025c152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0251400640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f025c13cc60 0x7f025c13d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 -- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f025c13cc60 msgr2=0x7f025c13d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f025c13cc60 0x7f025c13d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 -- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c151b80 msgr2=0x7f025c152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c151b80 0x7f025c152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 -- 172.21.15.87:0/2552271499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c1f7130 con 0x7f02400039c0 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.823+0000 7f0259e00640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f025c1f6bf0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f024c01ca50 tx=0x7f0240005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0252a00640 1 -- 172.21.15.87:0/2552271499 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0bc070 con 0x7f02400039c0 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0252a00640 1 -- 172.21.15.87:0/2552271499 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c000ee0 con 0x7f02400039c0 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0252a00640 1 -- 172.21.15.87:0/2552271499 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c007d40 con 0x7f02400039c0 2024-04-25T10:20:41.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c150ae0 con 0x7f02400039c0 2024-04-25T10:20:41.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c1510a0 con 0x7f02400039c0 2024-04-25T10:20:41.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c04ef30 con 0x7f02400039c0 2024-04-25T10:20:41.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0252a00640 1 -- 172.21.15.87:0/2552271499 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f024c007140 con 0x7f02400039c0 2024-04-25T10:20:41.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0252a00640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0238066f90 0x7f0238069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0252a00640 1 -- 172.21.15.87:0/2552271499 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f024c11e460 con 0x7f02400039c0 2024-04-25T10:20:41.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0251400640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0238066f90 0x7f0238069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.827+0000 7f0251400640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0238066f90 0x7f0238069450 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0244000c10 tx=0x7f0244009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:41.831+0000 7f0252a00640 1 -- 172.21.15.87:0/2552271499 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f024c016930 con 0x7f02400039c0 2024-04-25T10:20:42.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.023+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f025c15b080 con 0x7f02400039c0 2024-04-25T10:20:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0252a00640 1 -- 172.21.15.87:0/2552271499 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7f024c0ef3d0 con 0x7f02400039c0 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0238066f90 msgr2=0x7f0238069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0238066f90 0x7f0238069450 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0244000c10 tx=0x7f0244009040 comp rx=0 tx=0).stop 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 msgr2=0x7f025c1f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f025c1f6bf0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f024c01ca50 tx=0x7f0240005840 comp rx=0 tx=0).stop 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 shutdown_connections 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0238066f90 0x7f0238069450 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c151b80 0x7f025c152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f025c13cc60 0x7f025c13d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 --2- 172.21.15.87:0/2552271499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02400039c0 0x7f025c1f6bf0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.651+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 >> 172.21.15.87:0/2552271499 conn(0x7f025c0669a0 msgr2=0x7f025c0ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.655+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 shutdown_connections 2024-04-25T10:20:42.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.655+0000 7f0261000640 1 -- 172.21.15.87:0/2552271499 wait complete. 2024-04-25T10:20:42.676 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:20:42.676 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:42.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77aa800640 1 Processor -- start 2024-04-25T10:20:42.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77aa800640 1 -- start start 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77aa800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f77a415b9e0 0x7f77a40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77aa800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40ccfb0 con 0x7f77a415bdc0 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77aa800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40cd290 con 0x7f77a40d1c30 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77aa800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40cd570 con 0x7f77a415b9e0 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77a3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f77a415b9e0 0x7f77a40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77a2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f77a415bdc0 0x7f77a40d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47332/0 (socket says 172.21.15.87:47332) 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77a2a00640 1 -- 172.21.15.87:0/3937075350 learned_addr learned my addr 172.21.15.87:0/3937075350 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:42.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77a3400640 1 -- 172.21.15.87:0/3937075350 >> v1:172.21.15.169:6789/0 conn(0x7f77a40d1c30 legacy=0x7f77a40c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77a3400640 1 -- 172.21.15.87:0/3937075350 >> v1:172.21.15.87:6789/0 conn(0x7f77a415bdc0 legacy=0x7f77a40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77a2000640 1 -- 172.21.15.87:0/3937075350 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000991961 0 0) 0x7f77a40ccfb0 con 0x7f77a415bdc0 2024-04-25T10:20:42.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.811+0000 7f77a3400640 1 -- 172.21.15.87:0/3937075350 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40cd850 con 0x7f77a415b9e0 2024-04-25T10:20:42.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77a3400640 1 --2- 172.21.15.87:0/3937075350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415b9e0 0x7f77a40cc410 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f77940097c0 tx=0x7f779401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a78865e53d71b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77a2000640 1 -- 172.21.15.87:0/3937075350 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940b0070 con 0x7f77a415b9e0 2024-04-25T10:20:42.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77a2000640 1 -- 172.21.15.87:0/3937075350 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77940ab600 con 0x7f77a415b9e0 2024-04-25T10:20:42.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77a2000640 1 -- 172.21.15.87:0/3937075350 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940a4e60 con 0x7f77a415b9e0 2024-04-25T10:20:42.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77aa800640 1 -- 172.21.15.87:0/3937075350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415b9e0 msgr2=0x7f77a40cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77aa800640 1 --2- 172.21.15.87:0/3937075350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415b9e0 0x7f77a40cc410 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f77940097c0 tx=0x7f779401c3d0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77aa800640 1 -- 172.21.15.87:0/3937075350 shutdown_connections 2024-04-25T10:20:42.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77aa800640 1 --2- 172.21.15.87:0/3937075350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415b9e0 0x7f77a40cc410 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77aa800640 1 -- 172.21.15.87:0/3937075350 >> 172.21.15.87:0/3937075350 conn(0x7f77a409a9d0 msgr2=0x7f77a409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77aa800640 1 -- 172.21.15.87:0/3937075350 shutdown_connections 2024-04-25T10:20:42.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.815+0000 7f77aa800640 1 -- 172.21.15.87:0/3937075350 wait complete. 2024-04-25T10:20:42.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 Processor -- start 2024-04-25T10:20:42.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 -- start start 2024-04-25T10:20:42.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a40d1c30 0x7f77a415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a415b9e0 0x7f77a40cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415bdc0 0x7f77a4154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40c6d40 con 0x7f77a415bdc0 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40c6bc0 con 0x7f77a40d1c30 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4169840 con 0x7f77a415b9e0 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415bdc0 0x7f77a4154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415bdc0 0x7f77a4154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55992/0 (socket says 172.21.15.87:55992) 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 -- 172.21.15.87:0/2453702784 learned_addr learned my addr 172.21.15.87:0/2453702784 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3400640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a40d1c30 0x7f77a415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a2a00640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a415b9e0 0x7f77a40cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 -- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a415b9e0 msgr2=0x7f77a40cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a415b9e0 0x7f77a40cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 -- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a40d1c30 msgr2=0x7f77a415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a40d1c30 0x7f77a415ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 -- 172.21.15.87:0/2453702784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40cb810 con 0x7f77a415bdc0 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3400640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a40d1c30 0x7f77a415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a2a00640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a415b9e0 0x7f77a40cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77a3e00640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415bdc0 0x7f77a4154640 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f779c00b990 tx=0x7f779c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f779b400640 1 -- 172.21.15.87:0/2453702784 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c019070 con 0x7f77a415bdc0 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f779b400640 1 -- 172.21.15.87:0/2453702784 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00cba0 con 0x7f77a415bdc0 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a40ca230 con 0x7f77a415bdc0 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f779b400640 1 -- 172.21.15.87:0/2453702784 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c014440 con 0x7f77a415bdc0 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a40ca7f0 con 0x7f77a415bdc0 2024-04-25T10:20:42.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.819+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a404ef30 con 0x7f77a415bdc0 2024-04-25T10:20:42.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.823+0000 7f779b400640 1 -- 172.21.15.87:0/2453702784 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f779c01c070 con 0x7f77a415bdc0 2024-04-25T10:20:42.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.823+0000 7f779b400640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7784066f60 0x7f7784069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.823+0000 7f779b400640 1 -- 172.21.15.87:0/2453702784 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f779c084e30 con 0x7f77a415bdc0 2024-04-25T10:20:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.823+0000 7f77a3400640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7784066f60 0x7f7784069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.823+0000 7f77a3400640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7784066f60 0x7f7784069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f77940ad000 tx=0x7f7794008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:42.827+0000 7f779b400640 1 -- 172.21.15.87:0/2453702784 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f779c055da0 con 0x7f77a415bdc0 2024-04-25T10:20:43.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.015+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77a40b2e90 con 0x7f77a415bdc0 2024-04-25T10:20:43.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.015+0000 7f779b400640 1 -- 172.21.15.87:0/2453702784 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f779c0556a0 con 0x7f77a415bdc0 2024-04-25T10:20:43.018 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:43.019 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:20:43.019 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:20:43.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7784066f60 msgr2=0x7f7784069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7784066f60 0x7f7784069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f77940ad000 tx=0x7f7794008040 comp rx=0 tx=0).stop 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415bdc0 msgr2=0x7f77a4154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415bdc0 0x7f77a4154640 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f779c00b990 tx=0x7f779c00be50 comp rx=0 tx=0).stop 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 shutdown_connections 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7784066f60 0x7f7784069420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a415bdc0 0x7f77a4154640 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a415b9e0 0x7f77a40cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 --2- 172.21.15.87:0/2453702784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a40d1c30 0x7f77a415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 >> 172.21.15.87:0/2453702784 conn(0x7f77a409a9d0 msgr2=0x7f77a4155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 shutdown_connections 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.019+0000 7f77aa800640 1 -- 172.21.15.87:0/2453702784 wait complete. 2024-04-25T10:20:43.039 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:43.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.171+0000 7ffaf4c00640 1 Processor -- start 2024-04-25T10:20:43.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.171+0000 7ffaf4c00640 1 -- start start 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.171+0000 7ffaf4c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffaf015bc40 0x7ffaf006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.171+0000 7ffaf4c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffaf00cda10 0x7ffaf00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaf4c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf00b2770 con 0x7ffaf00cf930 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaf4c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf006c670 con 0x7ffaf00cda10 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaf4c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf006c950 con 0x7ffaf015bc40 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaee800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ffaf00cf930 0x7ffaf015f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47340/0 (socket says 172.21.15.87:47340) 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaee800640 1 -- 172.21.15.87:0/1125005550 learned_addr learned my addr 172.21.15.87:0/1125005550 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:43.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffae7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffaf00cda10 0x7ffaf00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaede00640 1 --2- 172.21.15.87:0/1125005550 >> v2:172.21.15.169:3300/0 conn(0x7ffaf015bc40 0x7ffaf006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffae7e00640 1 -- 172.21.15.87:0/1125005550 >> v2:172.21.15.169:3300/0 conn(0x7ffaf015bc40 msgr2=0x7ffaf006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffae7e00640 1 --2- 172.21.15.87:0/1125005550 >> v2:172.21.15.169:3300/0 conn(0x7ffaf015bc40 0x7ffaf006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffae7e00640 1 -- 172.21.15.87:0/1125005550 >> v1:172.21.15.87:6789/0 conn(0x7ffaf00cf930 legacy=0x7ffaf015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffae7e00640 1 -- 172.21.15.87:0/1125005550 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf006cc30 con 0x7ffaf00cda10 2024-04-25T10:20:43.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffae7e00640 1 --2- 172.21.15.87:0/1125005550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf00b4610 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00d0800 tx=0x7ffad801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c55f407ecb96380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaed400640 1 -- 172.21.15.87:0/1125005550 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80b1070 con 0x7ffaf00cda10 2024-04-25T10:20:43.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaed400640 1 -- 172.21.15.87:0/1125005550 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad80ac5f0 con 0x7ffaf00cda10 2024-04-25T10:20:43.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.175+0000 7ffaed400640 1 -- 172.21.15.87:0/1125005550 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80ac9c0 con 0x7ffaf00cda10 2024-04-25T10:20:43.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1125005550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 msgr2=0x7ffaf00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1125005550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf00b4610 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00d0800 tx=0x7ffad801c480 comp rx=0 tx=0).stop 2024-04-25T10:20:43.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1125005550 shutdown_connections 2024-04-25T10:20:43.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1125005550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf00b4610 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1125005550 >> v2:172.21.15.169:3300/0 conn(0x7ffaf015bc40 0x7ffaf006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1125005550 >> 172.21.15.87:0/1125005550 conn(0x7ffaf0066b70 msgr2=0x7ffaf0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1125005550 shutdown_connections 2024-04-25T10:20:43.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1125005550 wait complete. 2024-04-25T10:20:43.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 Processor -- start 2024-04-25T10:20:43.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- start start 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf014dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf00cf930 0x7ffaf0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffaf015bc40 0x7ffaf00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf0168b50 con 0x7ffaf00cda10 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf01689d0 con 0x7ffaf00cf930 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaf4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf0168cd0 con 0x7ffaf015bc40 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaede00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf014dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffae7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf00cf930 0x7ffaf0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaede00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf014dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56006/0 (socket says 172.21.15.87:56006) 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaede00640 1 -- 172.21.15.87:0/1122913494 learned_addr learned my addr 172.21.15.87:0/1122913494 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.179+0000 7ffaee800640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffaf015bc40 0x7ffaf00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae7e00640 1 -- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffaf015bc40 msgr2=0x7ffaf00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae7e00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffaf015bc40 0x7ffaf00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae7e00640 1 -- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 msgr2=0x7ffaf014dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae7e00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf014dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae7e00640 1 -- 172.21.15.87:0/1122913494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0070dd0 con 0x7ffaf00cf930 2024-04-25T10:20:43.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffaede00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae7e00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf00cf930 0x7ffaf0070310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffad80ac000 tx=0x7ffad8019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae6000640 1 -- 172.21.15.87:0/1122913494 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80b9020 con 0x7ffaf00cf930 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae6000640 1 -- 172.21.15.87:0/1122913494 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad80b5410 con 0x7ffaf00cf930 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae6000640 1 -- 172.21.15.87:0/1122913494 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80b1040 con 0x7ffaf00cf930 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf00710b0 con 0x7ffaf00cf930 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf0071670 con 0x7ffaf00cf930 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf004ef30 con 0x7ffaf00cf930 2024-04-25T10:20:43.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae6000640 1 -- 172.21.15.87:0/1122913494 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffad80bc020 con 0x7ffaf00cf930 2024-04-25T10:20:43.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae6000640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffacc066f60 0x7ffacc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffae6000640 1 -- 172.21.15.87:0/1122913494 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffad811e730 con 0x7ffaf00cf930 2024-04-25T10:20:43.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.183+0000 7ffaede00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffacc066f60 0x7ffacc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.187+0000 7ffaede00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffacc066f60 0x7ffacc069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ffaf01533b0 tx=0x7ffae0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.187+0000 7ffae6000640 1 -- 172.21.15.87:0/1122913494 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffad80c3030 con 0x7ffaf00cf930 2024-04-25T10:20:43.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.343+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaf006d960 con 0x7ffaf00cf930 2024-04-25T10:20:43.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.343+0000 7ffae6000640 1 -- 172.21.15.87:0/1122913494 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16615 (secure 0 0 0) 0x7ffad80a9070 con 0x7ffaf00cf930 2024-04-25T10:20:43.348 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:43.348 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":111,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:20:40.635201+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:37.371368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:38.004409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:43.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffacc066f60 msgr2=0x7ffacc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffacc066f60 0x7ffacc069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ffaf01533b0 tx=0x7ffae0009190 comp rx=0 tx=0).stop 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf00cf930 msgr2=0x7ffaf0070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf00cf930 0x7ffaf0070310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffad80ac000 tx=0x7ffad8019220 comp rx=0 tx=0).stop 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 shutdown_connections 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffacc066f60 0x7ffacc069420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffaf015bc40 0x7ffaf00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf00cf930 0x7ffaf0070310 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 --2- 172.21.15.87:0/1122913494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaf00cda10 0x7ffaf014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 >> 172.21.15.87:0/1122913494 conn(0x7ffaf0066b70 msgr2=0x7ffaf0158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 shutdown_connections 2024-04-25T10:20:43.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.347+0000 7ffaf4c00640 1 -- 172.21.15.87:0/1122913494 wait complete. 2024-04-25T10:20:43.369 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:20:43.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:43.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c7e00640 1 Processor -- start 2024-04-25T10:20:43.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c7e00640 1 -- start start 2024-04-25T10:20:43.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f07c00cda10 0x7f07c006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 0x7f07c00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c00b2770 con 0x7f07c015bc40 2024-04-25T10:20:43.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c006c670 con 0x7f07c00cda10 2024-04-25T10:20:43.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c006c950 con 0x7f07c00cf930 2024-04-25T10:20:43.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c5400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f07c015bc40 0x7f07c00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:47348/0 (socket says 172.21.15.87:47348) 2024-04-25T10:20:43.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c5400640 1 -- 172.21.15.87:0/3022153025 learned_addr learned my addr 172.21.15.87:0/3022153025 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:20:43.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c4a00640 1 --2- 172.21.15.87:0/3022153025 >> v2:172.21.15.169:3300/0 conn(0x7f07c00cda10 0x7f07c006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c5e00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 0x7f07c00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920343832 0 0) 0x7f07c00b2770 con 0x7f07c015bc40 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07a8003530 con 0x7f07c015bc40 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2823533604 0 0) 0x7f07a8003530 con 0x7f07c015bc40 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 msgr2=0x7f07c00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 0x7f07c00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 >> v2:172.21.15.169:3300/0 conn(0x7f07c00cda10 msgr2=0x7f07c006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 --2- 172.21.15.87:0/3022153025 >> v2:172.21.15.169:3300/0 conn(0x7f07c00cda10 0x7f07c006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c006cc30 con 0x7f07c015bc40 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f07b4001680 con 0x7f07c015bc40 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 >> v1:172.21.15.87:6789/0 conn(0x7f07c015bc40 legacy=0x7f07c00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a80039c0 0x7f07a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a8004810 0x7f07a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 0x7f07a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c006c670 con 0x7f07a80039c0 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c006c950 con 0x7f07a8004810 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a800b9d0 con 0x7f07c00cf930 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.503+0000 7f07c5400640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a80039c0 0x7f07a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c4a00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a8004810 0x7f07a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5e00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 0x7f07a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5400640 1 -- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 msgr2=0x7f07a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5400640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 0x7f07a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5400640 1 -- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a8004810 msgr2=0x7f07a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5400640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a8004810 0x7f07a8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5400640 1 -- 172.21.15.87:0/3022153025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8003530 con 0x7f07a80039c0 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c4a00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a8004810 0x7f07a8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:43.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5e00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c00cf930 0x7f07a80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5400640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a80039c0 0x7f07a8003de0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c2f0 tx=0x7f07b401cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ac0b969baa123a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4007d60 con 0x7f07a80039c0 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b40a9020 con 0x7f07a80039c0 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07bfe00640 1 -- 172.21.15.87:0/3022153025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4005ae0 con 0x7f07a80039c0 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c7e00640 1 -- 172.21.15.87:0/3022153025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a80039c0 msgr2=0x7f07a8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c7e00640 1 --2- 172.21.15.87:0/3022153025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07a80039c0 0x7f07a8003de0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c2f0 tx=0x7f07b401cd40 comp rx=0 tx=0).stop 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c5e00640 1 -- 172.21.15.87:0/3022153025 reap_dead start 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c7e00640 1 -- 172.21.15.87:0/3022153025 shutdown_connections 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.507+0000 7f07c7e00640 1 -- 172.21.15.87:0/3022153025 >> 172.21.15.87:0/3022153025 conn(0x7f07c0066b70 msgr2=0x7f07c0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- 172.21.15.87:0/3022153025 shutdown_connections 2024-04-25T10:20:43.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- 172.21.15.87:0/3022153025 wait complete. 2024-04-25T10:20:43.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 Processor -- start 2024-04-25T10:20:43.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- start start 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a80039c0 0x7f07c014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c01540b0 0x7f07c0158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c0154530 0x7f07c01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c00b3850 con 0x7f07c0154530 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c00b36d0 con 0x7f07a80039c0 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c00b39d0 con 0x7f07c01540b0 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c0154530 0x7f07c01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c0154530 0x7f07c01587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56020/0 (socket says 172.21.15.87:56020) 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 -- 172.21.15.87:0/4007549729 learned_addr learned my addr 172.21.15.87:0/4007549729 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5400640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a80039c0 0x7f07c014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c4a00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c01540b0 0x7f07c0158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 -- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c01540b0 msgr2=0x7f07c0158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c01540b0 0x7f07c0158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 -- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a80039c0 msgr2=0x7f07c014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a80039c0 0x7f07c014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 -- 172.21.15.87:0/4007549729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c015dc70 con 0x7f07c0154530 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5400640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a80039c0 0x7f07c014c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c4a00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c01540b0 0x7f07c0158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c5e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c0154530 0x7f07c01587c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f07c0072580 tx=0x7f07b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07bd600640 1 -- 172.21.15.87:0/4007549729 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b801a030 con 0x7f07c0154530 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07bd600640 1 -- 172.21.15.87:0/4007549729 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b8010df0 con 0x7f07c0154530 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c015df50 con 0x7f07c0154530 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07bd600640 1 -- 172.21.15.87:0/4007549729 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003be0 con 0x7f07c0154530 2024-04-25T10:20:43.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.511+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c015e510 con 0x7f07c0154530 2024-04-25T10:20:43.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.515+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c004ef30 con 0x7f07c0154530 2024-04-25T10:20:43.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.515+0000 7f07bd600640 1 -- 172.21.15.87:0/4007549729 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07b801d070 con 0x7f07c0154530 2024-04-25T10:20:43.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.515+0000 7f07bd600640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f07a0066f90 0x7f07a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.515+0000 7f07bd600640 1 -- 172.21.15.87:0/4007549729 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07b8085b10 con 0x7f07c0154530 2024-04-25T10:20:43.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.515+0000 7f07c5400640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f07a0066f90 0x7f07a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.515+0000 7f07c5400640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f07a0066f90 0x7f07a0069450 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f07b40aa000 tx=0x7f07b40b0040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.519+0000 7f07bd600640 1 -- 172.21.15.87:0/4007549729 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07b8056bb0 con 0x7f07c0154530 2024-04-25T10:20:43.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.671+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07c015f400 con 0x7f07a0066f90 2024-04-25T10:20:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.679+0000 7f07bd600640 1 -- 172.21.15.87:0/4007549729 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252446 (secure 0 0 0) 0x7f07c015f400 con 0x7f07a0066f90 2024-04-25T10:20:43.685 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:43.686 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f07a0066f90 msgr2=0x7f07a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f07a0066f90 0x7f07a0069450 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f07b40aa000 tx=0x7f07b40b0040 comp rx=0 tx=0).stop 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c0154530 msgr2=0x7f07c01587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c0154530 0x7f07c01587c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f07c0072580 tx=0x7f07b8009f90 comp rx=0 tx=0).stop 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 shutdown_connections 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f07a0066f90 0x7f07a0069450 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c0154530 0x7f07c01587c0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07c01540b0 0x7f07c0158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 --2- 172.21.15.87:0/4007549729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07a80039c0 0x7f07c014c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 >> 172.21.15.87:0/4007549729 conn(0x7f07c0066b70 msgr2=0x7f07c00af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 shutdown_connections 2024-04-25T10:20:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.683+0000 7f07c7e00640 1 -- 172.21.15.87:0/4007549729 wait complete. 2024-04-25T10:20:43.704 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-04-25T10:20:43.230706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":432184,"kb_used_data":11948,"kb_used_omap":12,"kb_used_meta":420083,"kb_avail":754542536,"statfs":{"total":773094113280,"available":772651556864,"internally_reserved":0,"allocated":12234752,"data_stored":5945912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":430165553},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.609760"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662025+0000","last_change":"2024-04-25T10:20:39.642791+0000","last_active":"2024-04-25T10:20:39.662025+0000","last_peered":"2024-04-25T10:20:39.662025+0000","last_clean":"2024-04-25T10:20:39.662025+0000","last_became_active":"2024-04-25T10:20:39.641400+0000","last_became_peered":"2024-04-25T10:20:39.641400+0000","last_unstale":"2024-04-25T10:20:39.662025+0000","last_undegraded":"2024-04-25T10:20:39.662025+0000","last_fullsized":"2024-04-25T10:20:39.662025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:32.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.648170+0000","last_change":"2024-04-25T10:20:39.659816+0000","last_active":"2024-04-25T10:20:40.648170+0000","last_peered":"2024-04-25T10:20:40.648170+0000","last_clean":"2024-04-25T10:20:40.648170+0000","last_became_active":"2024-04-25T10:20:39.659540+0000","last_became_peered":"2024-04-25T10:20:39.659540+0000","last_unstale":"2024-04-25T10:20:40.648170+0000","last_undegraded":"2024-04-25T10:20:40.648170+0000","last_fullsized":"2024-04-25T10:20:40.648170+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:17.057311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655762+0000","last_change":"2024-04-25T10:20:39.649666+0000","last_active":"2024-04-25T10:20:39.655762+0000","last_peered":"2024-04-25T10:20:39.655762+0000","last_clean":"2024-04-25T10:20:39.655762+0000","last_became_active":"2024-04-25T10:20:39.647660+0000","last_became_peered":"2024-04-25T10:20:39.647660+0000","last_unstale":"2024-04-25T10:20:39.655762+0000","last_undegraded":"2024-04-25T10:20:39.655762+0000","last_fullsized":"2024-04-25T10:20:39.655762+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:20.653656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.663558+0000","last_change":"2024-04-25T10:20:39.641952+0000","last_active":"2024-04-25T10:20:39.663558+0000","last_peered":"2024-04-25T10:20:39.663558+0000","last_clean":"2024-04-25T10:20:39.663558+0000","last_became_active":"2024-04-25T10:20:39.641692+0000","last_became_peered":"2024-04-25T10:20:39.641692+0000","last_unstale":"2024-04-25T10:20:39.663558+0000","last_undegraded":"2024-04-25T10:20:39.663558+0000","last_fullsized":"2024-04-25T10:20:39.663558+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.319099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.664165+0000","last_change":"2024-04-25T10:20:39.652608+0000","last_active":"2024-04-25T10:20:39.664165+0000","last_peered":"2024-04-25T10:20:39.664165+0000","last_clean":"2024-04-25T10:20:39.664165+0000","last_became_active":"2024-04-25T10:20:39.651507+0000","last_became_peered":"2024-04-25T10:20:39.651507+0000","last_unstale":"2024-04-25T10:20:39.664165+0000","last_undegraded":"2024-04-25T10:20:39.664165+0000","last_fullsized":"2024-04-25T10:20:39.664165+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:51.285662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662784+0000","last_change":"2024-04-25T10:20:39.658661+0000","last_active":"2024-04-25T10:20:39.662784+0000","last_peered":"2024-04-25T10:20:39.662784+0000","last_clean":"2024-04-25T10:20:39.662784+0000","last_became_active":"2024-04-25T10:20:39.658385+0000","last_became_peered":"2024-04-25T10:20:39.658385+0000","last_unstale":"2024-04-25T10:20:39.662784+0000","last_undegraded":"2024-04-25T10:20:39.662784+0000","last_fullsized":"2024-04-25T10:20:39.662784+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:51.285662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646201+0000","last_change":"2024-04-25T10:20:39.652735+0000","last_active":"2024-04-25T10:20:40.646201+0000","last_peered":"2024-04-25T10:20:40.646201+0000","last_clean":"2024-04-25T10:20:40.646201+0000","last_became_active":"2024-04-25T10:20:39.651043+0000","last_became_peered":"2024-04-25T10:20:39.651043+0000","last_unstale":"2024-04-25T10:20:40.646201+0000","last_undegraded":"2024-04-25T10:20:40.646201+0000","last_fullsized":"2024-04-25T10:20:40.646201+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:08.589164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.659702+0000","last_change":"2024-04-25T10:20:39.651806+0000","last_active":"2024-04-25T10:20:39.659702+0000","last_peered":"2024-04-25T10:20:39.659702+0000","last_clean":"2024-04-25T10:20:39.659702+0000","last_became_active":"2024-04-25T10:20:39.651475+0000","last_became_peered":"2024-04-25T10:20:39.651475+0000","last_unstale":"2024-04-25T10:20:39.659702+0000","last_undegraded":"2024-04-25T10:20:39.659702+0000","last_fullsized":"2024-04-25T10:20:39.659702+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:54.075960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662089+0000","last_change":"2024-04-25T10:20:39.653738+0000","last_active":"2024-04-25T10:20:39.662089+0000","last_peered":"2024-04-25T10:20:39.662089+0000","last_clean":"2024-04-25T10:20:39.662089+0000","last_became_active":"2024-04-25T10:20:39.653339+0000","last_became_peered":"2024-04-25T10:20:39.653339+0000","last_unstale":"2024-04-25T10:20:39.662089+0000","last_undegraded":"2024-04-25T10:20:39.662089+0000","last_fullsized":"2024-04-25T10:20:39.662089+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:27.430334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.657755+0000","last_change":"2024-04-25T10:20:39.641326+0000","last_active":"2024-04-25T10:20:39.657755+0000","last_peered":"2024-04-25T10:20:39.657755+0000","last_clean":"2024-04-25T10:20:39.657755+0000","last_became_active":"2024-04-25T10:20:39.641003+0000","last_became_peered":"2024-04-25T10:20:39.641003+0000","last_unstale":"2024-04-25T10:20:39.657755+0000","last_undegraded":"2024-04-25T10:20:39.657755+0000","last_fullsized":"2024-04-25T10:20:39.657755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:01.050069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647900+0000","last_change":"2024-04-25T10:20:39.659083+0000","last_active":"2024-04-25T10:20:40.647900+0000","last_peered":"2024-04-25T10:20:40.647900+0000","last_clean":"2024-04-25T10:20:40.647900+0000","last_became_active":"2024-04-25T10:20:39.658785+0000","last_became_peered":"2024-04-25T10:20:39.658785+0000","last_unstale":"2024-04-25T10:20:40.647900+0000","last_undegraded":"2024-04-25T10:20:40.647900+0000","last_fullsized":"2024-04-25T10:20:40.647900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:47.086858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.663581+0000","last_change":"2024-04-25T10:20:39.643645+0000","last_active":"2024-04-25T10:20:39.663581+0000","last_peered":"2024-04-25T10:20:39.663581+0000","last_clean":"2024-04-25T10:20:39.663581+0000","last_became_active":"2024-04-25T10:20:39.641767+0000","last_became_peered":"2024-04-25T10:20:39.641767+0000","last_unstale":"2024-04-25T10:20:39.663581+0000","last_undegraded":"2024-04-25T10:20:39.663581+0000","last_fullsized":"2024-04-25T10:20:39.663581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:47.086858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660483+0000","last_change":"2024-04-25T10:20:39.657710+0000","last_active":"2024-04-25T10:20:39.660483+0000","last_peered":"2024-04-25T10:20:39.660483+0000","last_clean":"2024-04-25T10:20:39.660483+0000","last_became_active":"2024-04-25T10:20:39.657253+0000","last_became_peered":"2024-04-25T10:20:39.657253+0000","last_unstale":"2024-04-25T10:20:39.660483+0000","last_undegraded":"2024-04-25T10:20:39.660483+0000","last_fullsized":"2024-04-25T10:20:39.660483+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:34.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.659968+0000","last_change":"2024-04-25T10:20:39.652347+0000","last_active":"2024-04-25T10:20:39.659968+0000","last_peered":"2024-04-25T10:20:39.659968+0000","last_clean":"2024-04-25T10:20:39.659968+0000","last_became_active":"2024-04-25T10:20:39.652089+0000","last_became_peered":"2024-04-25T10:20:39.652089+0000","last_unstale":"2024-04-25T10:20:39.659968+0000","last_undegraded":"2024-04-25T10:20:39.659968+0000","last_fullsized":"2024-04-25T10:20:39.659968+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:11.074810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661946+0000","last_change":"2024-04-25T10:20:39.659409+0000","last_active":"2024-04-25T10:20:39.661946+0000","last_peered":"2024-04-25T10:20:39.661946+0000","last_clean":"2024-04-25T10:20:39.661946+0000","last_became_active":"2024-04-25T10:20:39.659141+0000","last_became_peered":"2024-04-25T10:20:39.659141+0000","last_unstale":"2024-04-25T10:20:39.661946+0000","last_undegraded":"2024-04-25T10:20:39.661946+0000","last_fullsized":"2024-04-25T10:20:39.661946+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:17.642871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661164+0000","last_change":"2024-04-25T10:20:39.645549+0000","last_active":"2024-04-25T10:20:39.661164+0000","last_peered":"2024-04-25T10:20:39.661164+0000","last_clean":"2024-04-25T10:20:39.661164+0000","last_became_active":"2024-04-25T10:20:39.645230+0000","last_became_peered":"2024-04-25T10:20:39.645230+0000","last_unstale":"2024-04-25T10:20:39.661164+0000","last_undegraded":"2024-04-25T10:20:39.661164+0000","last_fullsized":"2024-04-25T10:20:39.661164+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:17.057311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.648156+0000","last_change":"2024-04-25T10:20:39.659528+0000","last_active":"2024-04-25T10:20:40.648156+0000","last_peered":"2024-04-25T10:20:40.648156+0000","last_clean":"2024-04-25T10:20:40.648156+0000","last_became_active":"2024-04-25T10:20:39.658833+0000","last_became_peered":"2024-04-25T10:20:39.658833+0000","last_unstale":"2024-04-25T10:20:40.648156+0000","last_undegraded":"2024-04-25T10:20:40.648156+0000","last_fullsized":"2024-04-25T10:20:40.648156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:05.384710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650254+0000","last_change":"2024-04-25T10:20:39.649923+0000","last_active":"2024-04-25T10:20:40.650254+0000","last_peered":"2024-04-25T10:20:40.650254+0000","last_clean":"2024-04-25T10:20:40.650254+0000","last_became_active":"2024-04-25T10:20:39.649633+0000","last_became_peered":"2024-04-25T10:20:39.649633+0000","last_unstale":"2024-04-25T10:20:40.650254+0000","last_undegraded":"2024-04-25T10:20:40.650254+0000","last_fullsized":"2024-04-25T10:20:40.650254+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:35.776326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660296+0000","last_change":"2024-04-25T10:20:39.656606+0000","last_active":"2024-04-25T10:20:39.660296+0000","last_peered":"2024-04-25T10:20:39.660296+0000","last_clean":"2024-04-25T10:20:39.660296+0000","last_became_active":"2024-04-25T10:20:39.656258+0000","last_became_peered":"2024-04-25T10:20:39.656258+0000","last_unstale":"2024-04-25T10:20:39.660296+0000","last_undegraded":"2024-04-25T10:20:39.660296+0000","last_fullsized":"2024-04-25T10:20:39.660296+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.264186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647873+0000","last_change":"2024-04-25T10:20:39.648026+0000","last_active":"2024-04-25T10:20:40.647873+0000","last_peered":"2024-04-25T10:20:40.647873+0000","last_clean":"2024-04-25T10:20:40.647873+0000","last_became_active":"2024-04-25T10:20:39.647758+0000","last_became_peered":"2024-04-25T10:20:39.647758+0000","last_unstale":"2024-04-25T10:20:40.647873+0000","last_undegraded":"2024-04-25T10:20:40.647873+0000","last_fullsized":"2024-04-25T10:20:40.647873+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:32.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647837+0000","last_change":"2024-04-25T10:20:39.654142+0000","last_active":"2024-04-25T10:20:40.647837+0000","last_peered":"2024-04-25T10:20:40.647837+0000","last_clean":"2024-04-25T10:20:40.647837+0000","last_became_active":"2024-04-25T10:20:39.653830+0000","last_became_peered":"2024-04-25T10:20:39.653830+0000","last_unstale":"2024-04-25T10:20:40.647837+0000","last_undegraded":"2024-04-25T10:20:40.647837+0000","last_fullsized":"2024-04-25T10:20:40.647837+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.319099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642980+0000","last_change":"2024-04-25T10:20:39.654944+0000","last_active":"2024-04-25T10:20:40.642980+0000","last_peered":"2024-04-25T10:20:40.642980+0000","last_clean":"2024-04-25T10:20:40.642980+0000","last_became_active":"2024-04-25T10:20:39.654489+0000","last_became_peered":"2024-04-25T10:20:39.654489+0000","last_unstale":"2024-04-25T10:20:40.642980+0000","last_undegraded":"2024-04-25T10:20:40.642980+0000","last_fullsized":"2024-04-25T10:20:40.642980+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:17.057311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647276+0000","last_change":"2024-04-25T10:20:39.646775+0000","last_active":"2024-04-25T10:20:40.647276+0000","last_peered":"2024-04-25T10:20:40.647276+0000","last_clean":"2024-04-25T10:20:40.647276+0000","last_became_active":"2024-04-25T10:20:39.646493+0000","last_became_peered":"2024-04-25T10:20:39.646493+0000","last_unstale":"2024-04-25T10:20:40.647276+0000","last_undegraded":"2024-04-25T10:20:40.647276+0000","last_fullsized":"2024-04-25T10:20:40.647276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642725+0000","last_change":"2024-04-25T10:20:39.652343+0000","last_active":"2024-04-25T10:20:40.642725+0000","last_peered":"2024-04-25T10:20:40.642725+0000","last_clean":"2024-04-25T10:20:40.642725+0000","last_became_active":"2024-04-25T10:20:39.651965+0000","last_became_peered":"2024-04-25T10:20:39.651965+0000","last_unstale":"2024-04-25T10:20:40.642725+0000","last_undegraded":"2024-04-25T10:20:40.642725+0000","last_fullsized":"2024-04-25T10:20:40.642725+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:32.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.648127+0000","last_change":"2024-04-25T10:20:39.654597+0000","last_active":"2024-04-25T10:20:40.648127+0000","last_peered":"2024-04-25T10:20:40.648127+0000","last_clean":"2024-04-25T10:20:40.648127+0000","last_became_active":"2024-04-25T10:20:39.654350+0000","last_became_peered":"2024-04-25T10:20:39.654350+0000","last_unstale":"2024-04-25T10:20:40.648127+0000","last_undegraded":"2024-04-25T10:20:40.648127+0000","last_fullsized":"2024-04-25T10:20:40.648127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:04.923819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662283+0000","last_change":"2024-04-25T10:20:39.646234+0000","last_active":"2024-04-25T10:20:39.662283+0000","last_peered":"2024-04-25T10:20:39.662283+0000","last_clean":"2024-04-25T10:20:39.662283+0000","last_became_active":"2024-04-25T10:20:39.645979+0000","last_became_peered":"2024-04-25T10:20:39.645979+0000","last_unstale":"2024-04-25T10:20:39.662283+0000","last_undegraded":"2024-04-25T10:20:39.662283+0000","last_fullsized":"2024-04-25T10:20:39.662283+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.526291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.667719+0000","last_change":"2024-04-25T10:20:39.666655+0000","last_active":"2024-04-25T10:20:39.667719+0000","last_peered":"2024-04-25T10:20:39.667719+0000","last_clean":"2024-04-25T10:20:39.667719+0000","last_became_active":"2024-04-25T10:20:39.666100+0000","last_became_peered":"2024-04-25T10:20:39.666100+0000","last_unstale":"2024-04-25T10:20:39.667719+0000","last_undegraded":"2024-04-25T10:20:39.667719+0000","last_fullsized":"2024-04-25T10:20:39.667719+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:16.045822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643174+0000","last_change":"2024-04-25T10:20:39.656861+0000","last_active":"2024-04-25T10:20:40.643174+0000","last_peered":"2024-04-25T10:20:40.643174+0000","last_clean":"2024-04-25T10:20:40.643174+0000","last_became_active":"2024-04-25T10:20:39.656561+0000","last_became_peered":"2024-04-25T10:20:39.656561+0000","last_unstale":"2024-04-25T10:20:40.643174+0000","last_undegraded":"2024-04-25T10:20:40.643174+0000","last_fullsized":"2024-04-25T10:20:40.643174+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:45.284651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647489+0000","last_change":"2024-04-25T10:20:39.653869+0000","last_active":"2024-04-25T10:20:40.647489+0000","last_peered":"2024-04-25T10:20:40.647489+0000","last_clean":"2024-04-25T10:20:40.647489+0000","last_became_active":"2024-04-25T10:20:39.653373+0000","last_became_peered":"2024-04-25T10:20:39.653373+0000","last_unstale":"2024-04-25T10:20:40.647489+0000","last_undegraded":"2024-04-25T10:20:40.647489+0000","last_fullsized":"2024-04-25T10:20:40.647489+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:34.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646070+0000","last_change":"2024-04-25T10:20:39.652907+0000","last_active":"2024-04-25T10:20:40.646070+0000","last_peered":"2024-04-25T10:20:40.646070+0000","last_clean":"2024-04-25T10:20:40.646070+0000","last_became_active":"2024-04-25T10:20:39.652178+0000","last_became_peered":"2024-04-25T10:20:39.652178+0000","last_unstale":"2024-04-25T10:20:40.646070+0000","last_undegraded":"2024-04-25T10:20:40.646070+0000","last_fullsized":"2024-04-25T10:20:40.646070+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:11.261676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662564+0000","last_change":"2024-04-25T10:20:39.646974+0000","last_active":"2024-04-25T10:20:39.662564+0000","last_peered":"2024-04-25T10:20:39.662564+0000","last_clean":"2024-04-25T10:20:39.662564+0000","last_became_active":"2024-04-25T10:20:39.646728+0000","last_became_peered":"2024-04-25T10:20:39.646728+0000","last_unstale":"2024-04-25T10:20:39.662564+0000","last_undegraded":"2024-04-25T10:20:39.662564+0000","last_fullsized":"2024-04-25T10:20:39.662564+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.534489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.648147+0000","last_change":"2024-04-25T10:20:39.646912+0000","last_active":"2024-04-25T10:20:39.648147+0000","last_peered":"2024-04-25T10:20:39.648147+0000","last_clean":"2024-04-25T10:20:39.648147+0000","last_became_active":"2024-04-25T10:20:39.646641+0000","last_became_peered":"2024-04-25T10:20:39.646641+0000","last_unstale":"2024-04-25T10:20:39.648147+0000","last_undegraded":"2024-04-25T10:20:39.648147+0000","last_fullsized":"2024-04-25T10:20:39.648147+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.774579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.657385+0000","last_change":"2024-04-25T10:20:39.656111+0000","last_active":"2024-04-25T10:20:39.657385+0000","last_peered":"2024-04-25T10:20:39.657385+0000","last_clean":"2024-04-25T10:20:39.657385+0000","last_became_active":"2024-04-25T10:20:39.655781+0000","last_became_peered":"2024-04-25T10:20:39.655781+0000","last_unstale":"2024-04-25T10:20:39.657385+0000","last_undegraded":"2024-04-25T10:20:39.657385+0000","last_fullsized":"2024-04-25T10:20:39.657385+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:30.545372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646511+0000","last_change":"2024-04-25T10:20:38.646916+0000","last_active":"2024-04-25T10:20:40.646511+0000","last_peered":"2024-04-25T10:20:40.646511+0000","last_clean":"2024-04-25T10:20:40.646511+0000","last_became_active":"2024-04-25T10:20:38.646062+0000","last_became_peered":"2024-04-25T10:20:38.646062+0000","last_unstale":"2024-04-25T10:20:40.646511+0000","last_undegraded":"2024-04-25T10:20:40.646511+0000","last_fullsized":"2024-04-25T10:20:40.646511+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650399+0000","last_change":"2024-04-25T10:20:39.652720+0000","last_active":"2024-04-25T10:20:40.650399+0000","last_peered":"2024-04-25T10:20:40.650399+0000","last_clean":"2024-04-25T10:20:40.650399+0000","last_became_active":"2024-04-25T10:20:39.652391+0000","last_became_peered":"2024-04-25T10:20:39.652391+0000","last_unstale":"2024-04-25T10:20:40.650399+0000","last_undegraded":"2024-04-25T10:20:40.650399+0000","last_fullsized":"2024-04-25T10:20:40.650399+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660085+0000","last_change":"2024-04-25T10:20:38.647429+0000","last_active":"2024-04-25T10:20:39.660085+0000","last_peered":"2024-04-25T10:20:39.660085+0000","last_clean":"2024-04-25T10:20:39.660085+0000","last_became_active":"2024-04-25T10:20:38.647153+0000","last_became_peered":"2024-04-25T10:20:38.647153+0000","last_unstale":"2024-04-25T10:20:39.660085+0000","last_undegraded":"2024-04-25T10:20:39.660085+0000","last_fullsized":"2024-04-25T10:20:39.660085+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:20.235562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661936+0000","last_change":"2024-04-25T10:20:39.658933+0000","last_active":"2024-04-25T10:20:39.661936+0000","last_peered":"2024-04-25T10:20:39.661936+0000","last_clean":"2024-04-25T10:20:39.661936+0000","last_became_active":"2024-04-25T10:20:39.658398+0000","last_became_peered":"2024-04-25T10:20:39.658398+0000","last_unstale":"2024-04-25T10:20:39.661936+0000","last_undegraded":"2024-04-25T10:20:39.661936+0000","last_fullsized":"2024-04-25T10:20:39.661936+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:13.006053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.668583+0000","last_change":"2024-04-25T10:20:38.641543+0000","last_active":"2024-04-25T10:20:39.668583+0000","last_peered":"2024-04-25T10:20:39.668583+0000","last_clean":"2024-04-25T10:20:39.668583+0000","last_became_active":"2024-04-25T10:20:38.641213+0000","last_became_peered":"2024-04-25T10:20:38.641213+0000","last_unstale":"2024-04-25T10:20:39.668583+0000","last_undegraded":"2024-04-25T10:20:39.668583+0000","last_fullsized":"2024-04-25T10:20:39.668583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:18.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660316+0000","last_change":"2024-04-25T10:20:39.657179+0000","last_active":"2024-04-25T10:20:39.660316+0000","last_peered":"2024-04-25T10:20:39.660316+0000","last_clean":"2024-04-25T10:20:39.660316+0000","last_became_active":"2024-04-25T10:20:39.656912+0000","last_became_peered":"2024-04-25T10:20:39.656912+0000","last_unstale":"2024-04-25T10:20:39.660316+0000","last_undegraded":"2024-04-25T10:20:39.660316+0000","last_fullsized":"2024-04-25T10:20:39.660316+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.666594+0000","last_change":"2024-04-25T10:20:38.642997+0000","last_active":"2024-04-25T10:20:39.666594+0000","last_peered":"2024-04-25T10:20:39.666594+0000","last_clean":"2024-04-25T10:20:39.666594+0000","last_became_active":"2024-04-25T10:20:38.642700+0000","last_became_peered":"2024-04-25T10:20:38.642700+0000","last_unstale":"2024-04-25T10:20:39.666594+0000","last_undegraded":"2024-04-25T10:20:39.666594+0000","last_fullsized":"2024-04-25T10:20:39.666594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:30.610434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662350+0000","last_change":"2024-04-25T10:20:39.653345+0000","last_active":"2024-04-25T10:20:39.662350+0000","last_peered":"2024-04-25T10:20:39.662350+0000","last_clean":"2024-04-25T10:20:39.662350+0000","last_became_active":"2024-04-25T10:20:39.653078+0000","last_became_peered":"2024-04-25T10:20:39.653078+0000","last_unstale":"2024-04-25T10:20:39.662350+0000","last_undegraded":"2024-04-25T10:20:39.662350+0000","last_fullsized":"2024-04-25T10:20:39.662350+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:55.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.645914+0000","last_change":"2024-04-25T10:20:38.645381+0000","last_active":"2024-04-25T10:20:40.645914+0000","last_peered":"2024-04-25T10:20:40.645914+0000","last_clean":"2024-04-25T10:20:40.645914+0000","last_became_active":"2024-04-25T10:20:38.644984+0000","last_became_peered":"2024-04-25T10:20:38.644984+0000","last_unstale":"2024-04-25T10:20:40.645914+0000","last_undegraded":"2024-04-25T10:20:40.645914+0000","last_fullsized":"2024-04-25T10:20:40.645914+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:52.735354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647511+0000","last_change":"2024-04-25T10:20:39.660362+0000","last_active":"2024-04-25T10:20:40.647511+0000","last_peered":"2024-04-25T10:20:40.647511+0000","last_clean":"2024-04-25T10:20:40.647511+0000","last_became_active":"2024-04-25T10:20:39.659904+0000","last_became_peered":"2024-04-25T10:20:39.659904+0000","last_unstale":"2024-04-25T10:20:40.647511+0000","last_undegraded":"2024-04-25T10:20:40.647511+0000","last_fullsized":"2024-04-25T10:20:40.647511+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:30.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643084+0000","last_change":"2024-04-25T10:20:38.644434+0000","last_active":"2024-04-25T10:20:40.643084+0000","last_peered":"2024-04-25T10:20:40.643084+0000","last_clean":"2024-04-25T10:20:40.643084+0000","last_became_active":"2024-04-25T10:20:38.644168+0000","last_became_peered":"2024-04-25T10:20:38.644168+0000","last_unstale":"2024-04-25T10:20:40.643084+0000","last_undegraded":"2024-04-25T10:20:40.643084+0000","last_fullsized":"2024-04-25T10:20:40.643084+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:23.667467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661189+0000","last_change":"2024-04-25T10:20:39.649821+0000","last_active":"2024-04-25T10:20:39.661189+0000","last_peered":"2024-04-25T10:20:39.661189+0000","last_clean":"2024-04-25T10:20:39.661189+0000","last_became_active":"2024-04-25T10:20:39.649548+0000","last_became_peered":"2024-04-25T10:20:39.649548+0000","last_unstale":"2024-04-25T10:20:39.661189+0000","last_undegraded":"2024-04-25T10:20:39.661189+0000","last_fullsized":"2024-04-25T10:20:39.661189+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:29.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647298+0000","last_change":"2024-04-25T10:20:38.646179+0000","last_active":"2024-04-25T10:20:40.647298+0000","last_peered":"2024-04-25T10:20:40.647298+0000","last_clean":"2024-04-25T10:20:40.647298+0000","last_became_active":"2024-04-25T10:20:38.645911+0000","last_became_peered":"2024-04-25T10:20:38.645911+0000","last_unstale":"2024-04-25T10:20:40.647298+0000","last_undegraded":"2024-04-25T10:20:40.647298+0000","last_fullsized":"2024-04-25T10:20:40.647298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:21.713286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646037+0000","last_change":"2024-04-25T10:20:39.655750+0000","last_active":"2024-04-25T10:20:40.646037+0000","last_peered":"2024-04-25T10:20:40.646037+0000","last_clean":"2024-04-25T10:20:40.646037+0000","last_became_active":"2024-04-25T10:20:39.650566+0000","last_became_peered":"2024-04-25T10:20:39.650566+0000","last_unstale":"2024-04-25T10:20:40.646037+0000","last_undegraded":"2024-04-25T10:20:40.646037+0000","last_fullsized":"2024-04-25T10:20:40.646037+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:18.223411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.635567+0000","last_change":"2024-04-25T10:20:38.647912+0000","last_active":"2024-04-25T10:20:39.635567+0000","last_peered":"2024-04-25T10:20:39.635567+0000","last_clean":"2024-04-25T10:20:39.635567+0000","last_became_active":"2024-04-25T10:20:38.647275+0000","last_became_peered":"2024-04-25T10:20:38.647275+0000","last_unstale":"2024-04-25T10:20:39.635567+0000","last_undegraded":"2024-04-25T10:20:39.635567+0000","last_fullsized":"2024-04-25T10:20:39.635567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:34.770299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646479+0000","last_change":"2024-04-25T10:20:39.659008+0000","last_active":"2024-04-25T10:20:40.646479+0000","last_peered":"2024-04-25T10:20:40.646479+0000","last_clean":"2024-04-25T10:20:40.646479+0000","last_became_active":"2024-04-25T10:20:39.658725+0000","last_became_peered":"2024-04-25T10:20:39.658725+0000","last_unstale":"2024-04-25T10:20:40.646479+0000","last_undegraded":"2024-04-25T10:20:40.646479+0000","last_fullsized":"2024-04-25T10:20:40.646479+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:51.768470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.630483+0000","last_change":"2024-04-25T10:20:38.643558+0000","last_active":"2024-04-25T10:20:39.630483+0000","last_peered":"2024-04-25T10:20:39.630483+0000","last_clean":"2024-04-25T10:20:39.630483+0000","last_became_active":"2024-04-25T10:20:38.643286+0000","last_became_peered":"2024-04-25T10:20:38.643286+0000","last_unstale":"2024-04-25T10:20:39.630483+0000","last_undegraded":"2024-04-25T10:20:39.630483+0000","last_fullsized":"2024-04-25T10:20:39.630483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:12.768553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650452+0000","last_change":"2024-04-25T10:20:39.648670+0000","last_active":"2024-04-25T10:20:40.650452+0000","last_peered":"2024-04-25T10:20:40.650452+0000","last_clean":"2024-04-25T10:20:40.650452+0000","last_became_active":"2024-04-25T10:20:39.647657+0000","last_became_peered":"2024-04-25T10:20:39.647657+0000","last_unstale":"2024-04-25T10:20:40.650452+0000","last_undegraded":"2024-04-25T10:20:40.650452+0000","last_fullsized":"2024-04-25T10:20:40.650452+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:34.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643333+0000","last_change":"2024-04-25T10:20:38.643850+0000","last_active":"2024-04-25T10:20:40.643333+0000","last_peered":"2024-04-25T10:20:40.643333+0000","last_clean":"2024-04-25T10:20:40.643333+0000","last_became_active":"2024-04-25T10:20:38.643602+0000","last_became_peered":"2024-04-25T10:20:38.643602+0000","last_unstale":"2024-04-25T10:20:40.643333+0000","last_undegraded":"2024-04-25T10:20:40.643333+0000","last_fullsized":"2024-04-25T10:20:40.643333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:53.069934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643171+0000","last_change":"2024-04-25T10:20:39.649529+0000","last_active":"2024-04-25T10:20:40.643171+0000","last_peered":"2024-04-25T10:20:40.643171+0000","last_clean":"2024-04-25T10:20:40.643171+0000","last_became_active":"2024-04-25T10:20:39.649233+0000","last_became_peered":"2024-04-25T10:20:39.649233+0000","last_unstale":"2024-04-25T10:20:40.643171+0000","last_undegraded":"2024-04-25T10:20:40.643171+0000","last_fullsized":"2024-04-25T10:20:40.643171+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:29.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"110'3","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.667711+0000","last_change":"2024-04-25T10:20:38.649213+0000","last_active":"2024-04-25T10:20:39.667711+0000","last_peered":"2024-04-25T10:20:39.667711+0000","last_clean":"2024-04-25T10:20:39.667711+0000","last_became_active":"2024-04-25T10:20:38.648950+0000","last_became_peered":"2024-04-25T10:20:38.648950+0000","last_unstale":"2024-04-25T10:20:39.667711+0000","last_undegraded":"2024-04-25T10:20:39.667711+0000","last_fullsized":"2024-04-25T10:20:39.667711+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:33.851385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646337+0000","last_change":"2024-04-25T10:20:39.654796+0000","last_active":"2024-04-25T10:20:40.646337+0000","last_peered":"2024-04-25T10:20:40.646337+0000","last_clean":"2024-04-25T10:20:40.646337+0000","last_became_active":"2024-04-25T10:20:39.650003+0000","last_became_peered":"2024-04-25T10:20:39.650003+0000","last_unstale":"2024-04-25T10:20:40.646337+0000","last_undegraded":"2024-04-25T10:20:40.646337+0000","last_fullsized":"2024-04-25T10:20:40.646337+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:11.221161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"110'6","reported_seq":18,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.671276+0000","last_change":"2024-04-25T10:20:38.649519+0000","last_active":"2024-04-25T10:20:39.671276+0000","last_peered":"2024-04-25T10:20:39.671276+0000","last_clean":"2024-04-25T10:20:39.671276+0000","last_became_active":"2024-04-25T10:20:38.647111+0000","last_became_peered":"2024-04-25T10:20:38.647111+0000","last_unstale":"2024-04-25T10:20:39.671276+0000","last_undegraded":"2024-04-25T10:20:39.671276+0000","last_fullsized":"2024-04-25T10:20:39.671276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:10.068783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643025+0000","last_change":"2024-04-25T10:20:39.648935+0000","last_active":"2024-04-25T10:20:40.643025+0000","last_peered":"2024-04-25T10:20:40.643025+0000","last_clean":"2024-04-25T10:20:40.643025+0000","last_became_active":"2024-04-25T10:20:39.648692+0000","last_became_peered":"2024-04-25T10:20:39.648692+0000","last_unstale":"2024-04-25T10:20:40.643025+0000","last_undegraded":"2024-04-25T10:20:40.643025+0000","last_fullsized":"2024-04-25T10:20:40.643025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.526291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.665826+0000","last_change":"2024-04-25T10:20:38.643464+0000","last_active":"2024-04-25T10:20:39.665826+0000","last_peered":"2024-04-25T10:20:39.665826+0000","last_clean":"2024-04-25T10:20:39.665826+0000","last_became_active":"2024-04-25T10:20:38.643071+0000","last_became_peered":"2024-04-25T10:20:38.643071+0000","last_unstale":"2024-04-25T10:20:39.665826+0000","last_undegraded":"2024-04-25T10:20:39.665826+0000","last_fullsized":"2024-04-25T10:20:39.665826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:30.610434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643088+0000","last_change":"2024-04-25T10:20:39.652808+0000","last_active":"2024-04-25T10:20:40.643088+0000","last_peered":"2024-04-25T10:20:40.643088+0000","last_clean":"2024-04-25T10:20:40.643088+0000","last_became_active":"2024-04-25T10:20:39.652301+0000","last_became_peered":"2024-04-25T10:20:39.652301+0000","last_unstale":"2024-04-25T10:20:40.643088+0000","last_undegraded":"2024-04-25T10:20:40.643088+0000","last_fullsized":"2024-04-25T10:20:40.643088+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.534489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.649968+0000","last_change":"2024-04-25T10:20:38.649707+0000","last_active":"2024-04-25T10:20:40.649968+0000","last_peered":"2024-04-25T10:20:40.649968+0000","last_clean":"2024-04-25T10:20:40.649968+0000","last_became_active":"2024-04-25T10:20:38.649332+0000","last_became_peered":"2024-04-25T10:20:38.649332+0000","last_unstale":"2024-04-25T10:20:40.649968+0000","last_undegraded":"2024-04-25T10:20:40.649968+0000","last_fullsized":"2024-04-25T10:20:40.649968+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:57.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642745+0000","last_change":"2024-04-25T10:20:39.648738+0000","last_active":"2024-04-25T10:20:40.642745+0000","last_peered":"2024-04-25T10:20:40.642745+0000","last_clean":"2024-04-25T10:20:40.642745+0000","last_became_active":"2024-04-25T10:20:39.648422+0000","last_became_peered":"2024-04-25T10:20:39.648422+0000","last_unstale":"2024-04-25T10:20:40.642745+0000","last_undegraded":"2024-04-25T10:20:40.642745+0000","last_fullsized":"2024-04-25T10:20:40.642745+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:04.923819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"110'8","reported_seq":20,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.670166+0000","last_change":"2024-04-25T10:20:38.639055+0000","last_active":"2024-04-25T10:20:39.670166+0000","last_peered":"2024-04-25T10:20:39.670166+0000","last_clean":"2024-04-25T10:20:39.670166+0000","last_became_active":"2024-04-25T10:20:38.636323+0000","last_became_peered":"2024-04-25T10:20:38.636323+0000","last_unstale":"2024-04-25T10:20:39.670166+0000","last_undegraded":"2024-04-25T10:20:39.670166+0000","last_fullsized":"2024-04-25T10:20:39.670166+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.031609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655276+0000","last_change":"2024-04-25T10:20:39.650747+0000","last_active":"2024-04-25T10:20:39.655276+0000","last_peered":"2024-04-25T10:20:39.655276+0000","last_clean":"2024-04-25T10:20:39.655276+0000","last_became_active":"2024-04-25T10:20:39.650506+0000","last_became_peered":"2024-04-25T10:20:39.650506+0000","last_unstale":"2024-04-25T10:20:39.655276+0000","last_undegraded":"2024-04-25T10:20:39.655276+0000","last_fullsized":"2024-04-25T10:20:39.655276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:30.545372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.668104+0000","last_change":"2024-04-25T10:20:38.646455+0000","last_active":"2024-04-25T10:20:39.668104+0000","last_peered":"2024-04-25T10:20:39.668104+0000","last_clean":"2024-04-25T10:20:39.668104+0000","last_became_active":"2024-04-25T10:20:38.646113+0000","last_became_peered":"2024-04-25T10:20:38.646113+0000","last_unstale":"2024-04-25T10:20:39.668104+0000","last_undegraded":"2024-04-25T10:20:39.668104+0000","last_fullsized":"2024-04-25T10:20:39.668104+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:34.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.665051+0000","last_change":"2024-04-25T10:20:39.663863+0000","last_active":"2024-04-25T10:20:39.665051+0000","last_peered":"2024-04-25T10:20:39.665051+0000","last_clean":"2024-04-25T10:20:39.665051+0000","last_became_active":"2024-04-25T10:20:39.663482+0000","last_became_peered":"2024-04-25T10:20:39.663482+0000","last_unstale":"2024-04-25T10:20:39.665051+0000","last_undegraded":"2024-04-25T10:20:39.665051+0000","last_fullsized":"2024-04-25T10:20:39.665051+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:57.041605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.669895+0000","last_change":"2024-04-25T10:20:38.636365+0000","last_active":"2024-04-25T10:20:39.669895+0000","last_peered":"2024-04-25T10:20:39.669895+0000","last_clean":"2024-04-25T10:20:39.669895+0000","last_became_active":"2024-04-25T10:20:38.635394+0000","last_became_peered":"2024-04-25T10:20:38.635394+0000","last_unstale":"2024-04-25T10:20:39.669895+0000","last_undegraded":"2024-04-25T10:20:39.669895+0000","last_fullsized":"2024-04-25T10:20:39.669895+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:20.365553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.653063+0000","last_change":"2024-04-25T10:20:39.648658+0000","last_active":"2024-04-25T10:20:39.653063+0000","last_peered":"2024-04-25T10:20:39.653063+0000","last_clean":"2024-04-25T10:20:39.653063+0000","last_became_active":"2024-04-25T10:20:39.648403+0000","last_became_peered":"2024-04-25T10:20:39.648403+0000","last_unstale":"2024-04-25T10:20:39.653063+0000","last_undegraded":"2024-04-25T10:20:39.653063+0000","last_fullsized":"2024-04-25T10:20:39.653063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:04.956589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650088+0000","last_change":"2024-04-25T10:20:38.647871+0000","last_active":"2024-04-25T10:20:40.650088+0000","last_peered":"2024-04-25T10:20:40.650088+0000","last_clean":"2024-04-25T10:20:40.650088+0000","last_became_active":"2024-04-25T10:20:38.647479+0000","last_became_peered":"2024-04-25T10:20:38.647479+0000","last_unstale":"2024-04-25T10:20:40.650088+0000","last_undegraded":"2024-04-25T10:20:40.650088+0000","last_fullsized":"2024-04-25T10:20:40.650088+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:21.425913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655743+0000","last_change":"2024-04-25T10:20:39.648970+0000","last_active":"2024-04-25T10:20:39.655743+0000","last_peered":"2024-04-25T10:20:39.655743+0000","last_clean":"2024-04-25T10:20:39.655743+0000","last_became_active":"2024-04-25T10:20:39.648013+0000","last_became_peered":"2024-04-25T10:20:39.648013+0000","last_unstale":"2024-04-25T10:20:39.655743+0000","last_undegraded":"2024-04-25T10:20:39.655743+0000","last_fullsized":"2024-04-25T10:20:39.655743+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:24.673494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646058+0000","last_change":"2024-04-25T10:20:38.652355+0000","last_active":"2024-04-25T10:20:40.646058+0000","last_peered":"2024-04-25T10:20:40.646058+0000","last_clean":"2024-04-25T10:20:40.646058+0000","last_became_active":"2024-04-25T10:20:38.652049+0000","last_became_peered":"2024-04-25T10:20:38.652049+0000","last_unstale":"2024-04-25T10:20:40.646058+0000","last_undegraded":"2024-04-25T10:20:40.646058+0000","last_fullsized":"2024-04-25T10:20:40.646058+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:36.693750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655709+0000","last_change":"2024-04-25T10:20:39.649351+0000","last_active":"2024-04-25T10:20:39.655709+0000","last_peered":"2024-04-25T10:20:39.655709+0000","last_clean":"2024-04-25T10:20:39.655709+0000","last_became_active":"2024-04-25T10:20:39.647588+0000","last_became_peered":"2024-04-25T10:20:39.647588+0000","last_unstale":"2024-04-25T10:20:39.655709+0000","last_undegraded":"2024-04-25T10:20:39.655709+0000","last_fullsized":"2024-04-25T10:20:39.655709+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:16.386812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.669743+0000","last_change":"2024-04-25T10:20:38.643742+0000","last_active":"2024-04-25T10:20:39.669743+0000","last_peered":"2024-04-25T10:20:39.669743+0000","last_clean":"2024-04-25T10:20:39.669743+0000","last_became_active":"2024-04-25T10:20:38.643472+0000","last_became_peered":"2024-04-25T10:20:38.643472+0000","last_unstale":"2024-04-25T10:20:39.669743+0000","last_undegraded":"2024-04-25T10:20:39.669743+0000","last_fullsized":"2024-04-25T10:20:39.669743+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:18.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661136+0000","last_change":"2024-04-25T10:20:39.644552+0000","last_active":"2024-04-25T10:20:39.661136+0000","last_peered":"2024-04-25T10:20:39.661136+0000","last_clean":"2024-04-25T10:20:39.661136+0000","last_became_active":"2024-04-25T10:20:39.644274+0000","last_became_peered":"2024-04-25T10:20:39.644274+0000","last_unstale":"2024-04-25T10:20:39.661136+0000","last_undegraded":"2024-04-25T10:20:39.661136+0000","last_fullsized":"2024-04-25T10:20:39.661136+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:05.384710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.629817+0000","last_change":"2024-04-25T10:20:38.641441+0000","last_active":"2024-04-25T10:20:39.629817+0000","last_peered":"2024-04-25T10:20:39.629817+0000","last_clean":"2024-04-25T10:20:39.629817+0000","last_became_active":"2024-04-25T10:20:38.641204+0000","last_became_peered":"2024-04-25T10:20:38.641204+0000","last_unstale":"2024-04-25T10:20:39.629817+0000","last_undegraded":"2024-04-25T10:20:39.629817+0000","last_fullsized":"2024-04-25T10:20:39.629817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:22.620727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660268+0000","last_change":"2024-04-25T10:20:39.656372+0000","last_active":"2024-04-25T10:20:39.660268+0000","last_peered":"2024-04-25T10:20:39.660268+0000","last_clean":"2024-04-25T10:20:39.660268+0000","last_became_active":"2024-04-25T10:20:39.656024+0000","last_became_peered":"2024-04-25T10:20:39.656024+0000","last_unstale":"2024-04-25T10:20:39.660268+0000","last_undegraded":"2024-04-25T10:20:39.660268+0000","last_fullsized":"2024-04-25T10:20:39.660268+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:35.776326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.644296+0000","last_change":"2024-04-25T10:20:38.647179+0000","last_active":"2024-04-25T10:20:39.644296+0000","last_peered":"2024-04-25T10:20:39.644296+0000","last_clean":"2024-04-25T10:20:39.644296+0000","last_became_active":"2024-04-25T10:20:38.646864+0000","last_became_peered":"2024-04-25T10:20:38.646864+0000","last_unstale":"2024-04-25T10:20:39.644296+0000","last_undegraded":"2024-04-25T10:20:39.644296+0000","last_fullsized":"2024-04-25T10:20:39.644296+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:20.365553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642815+0000","last_change":"2024-04-25T10:20:39.644973+0000","last_active":"2024-04-25T10:20:40.642815+0000","last_peered":"2024-04-25T10:20:40.642815+0000","last_clean":"2024-04-25T10:20:40.642815+0000","last_became_active":"2024-04-25T10:20:39.643727+0000","last_became_peered":"2024-04-25T10:20:39.643727+0000","last_unstale":"2024-04-25T10:20:40.642815+0000","last_undegraded":"2024-04-25T10:20:40.642815+0000","last_fullsized":"2024-04-25T10:20:40.642815+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:05.384710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.635708+0000","last_change":"2024-04-25T10:20:38.648235+0000","last_active":"2024-04-25T10:20:39.635708+0000","last_peered":"2024-04-25T10:20:39.635708+0000","last_clean":"2024-04-25T10:20:39.635708+0000","last_became_active":"2024-04-25T10:20:38.647963+0000","last_became_peered":"2024-04-25T10:20:38.647963+0000","last_unstale":"2024-04-25T10:20:39.635708+0000","last_undegraded":"2024-04-25T10:20:39.635708+0000","last_fullsized":"2024-04-25T10:20:39.635708+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:12.258160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650257+0000","last_change":"2024-04-25T10:20:39.649305+0000","last_active":"2024-04-25T10:20:40.650257+0000","last_peered":"2024-04-25T10:20:40.650257+0000","last_clean":"2024-04-25T10:20:40.650257+0000","last_became_active":"2024-04-25T10:20:39.647621+0000","last_became_peered":"2024-04-25T10:20:39.647621+0000","last_unstale":"2024-04-25T10:20:40.650257+0000","last_undegraded":"2024-04-25T10:20:40.650257+0000","last_fullsized":"2024-04-25T10:20:40.650257+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.264186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643510+0000","last_change":"2024-04-25T10:20:38.648026+0000","last_active":"2024-04-25T10:20:40.643510+0000","last_peered":"2024-04-25T10:20:40.643510+0000","last_clean":"2024-04-25T10:20:40.643510+0000","last_became_active":"2024-04-25T10:20:38.647753+0000","last_became_peered":"2024-04-25T10:20:38.647753+0000","last_unstale":"2024-04-25T10:20:40.643510+0000","last_undegraded":"2024-04-25T10:20:40.643510+0000","last_fullsized":"2024-04-25T10:20:40.643510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:15.380785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662051+0000","last_change":"2024-04-25T10:20:39.644409+0000","last_active":"2024-04-25T10:20:39.662051+0000","last_peered":"2024-04-25T10:20:39.662051+0000","last_clean":"2024-04-25T10:20:39.662051+0000","last_became_active":"2024-04-25T10:20:39.644155+0000","last_became_peered":"2024-04-25T10:20:39.644155+0000","last_unstale":"2024-04-25T10:20:39.662051+0000","last_undegraded":"2024-04-25T10:20:39.662051+0000","last_fullsized":"2024-04-25T10:20:39.662051+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:04.923819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"111'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.651675+0000","last_change":"2024-04-25T10:20:38.646629+0000","last_active":"2024-04-25T10:20:40.651675+0000","last_peered":"2024-04-25T10:20:40.651675+0000","last_clean":"2024-04-25T10:20:40.651675+0000","last_became_active":"2024-04-25T10:20:38.646323+0000","last_became_peered":"2024-04-25T10:20:38.646323+0000","last_unstale":"2024-04-25T10:20:40.651675+0000","last_undegraded":"2024-04-25T10:20:40.651675+0000","last_fullsized":"2024-04-25T10:20:40.651675+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:11.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.663602+0000","last_change":"2024-04-25T10:20:39.650067+0000","last_active":"2024-04-25T10:20:39.663602+0000","last_peered":"2024-04-25T10:20:39.663602+0000","last_clean":"2024-04-25T10:20:39.663602+0000","last_became_active":"2024-04-25T10:20:39.649822+0000","last_became_peered":"2024-04-25T10:20:39.649822+0000","last_unstale":"2024-04-25T10:20:39.663602+0000","last_undegraded":"2024-04-25T10:20:39.663602+0000","last_fullsized":"2024-04-25T10:20:39.663602+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:45.284651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.636338+0000","last_change":"2024-04-25T10:20:38.647846+0000","last_active":"2024-04-25T10:20:39.636338+0000","last_peered":"2024-04-25T10:20:39.636338+0000","last_clean":"2024-04-25T10:20:39.636338+0000","last_became_active":"2024-04-25T10:20:38.647577+0000","last_became_peered":"2024-04-25T10:20:38.647577+0000","last_unstale":"2024-04-25T10:20:39.636338+0000","last_undegraded":"2024-04-25T10:20:39.636338+0000","last_fullsized":"2024-04-25T10:20:39.636338+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:29.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650154+0000","last_change":"2024-04-25T10:20:39.643726+0000","last_active":"2024-04-25T10:20:40.650154+0000","last_peered":"2024-04-25T10:20:40.650154+0000","last_clean":"2024-04-25T10:20:40.650154+0000","last_became_active":"2024-04-25T10:20:39.643410+0000","last_became_peered":"2024-04-25T10:20:39.643410+0000","last_unstale":"2024-04-25T10:20:40.650154+0000","last_undegraded":"2024-04-25T10:20:40.650154+0000","last_fullsized":"2024-04-25T10:20:40.650154+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:11.074810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643279+0000","last_change":"2024-04-25T10:20:38.645150+0000","last_active":"2024-04-25T10:20:40.643279+0000","last_peered":"2024-04-25T10:20:40.643279+0000","last_clean":"2024-04-25T10:20:40.643279+0000","last_became_active":"2024-04-25T10:20:38.644775+0000","last_became_peered":"2024-04-25T10:20:38.644775+0000","last_unstale":"2024-04-25T10:20:40.643279+0000","last_undegraded":"2024-04-25T10:20:40.643279+0000","last_fullsized":"2024-04-25T10:20:40.643279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:19.647629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647514+0000","last_change":"2024-04-25T10:20:39.648066+0000","last_active":"2024-04-25T10:20:40.647514+0000","last_peered":"2024-04-25T10:20:40.647514+0000","last_clean":"2024-04-25T10:20:40.647514+0000","last_became_active":"2024-04-25T10:20:39.647804+0000","last_became_peered":"2024-04-25T10:20:39.647804+0000","last_unstale":"2024-04-25T10:20:40.647514+0000","last_undegraded":"2024-04-25T10:20:40.647514+0000","last_fullsized":"2024-04-25T10:20:40.647514+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:35.517703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.645923+0000","last_change":"2024-04-25T10:20:38.645651+0000","last_active":"2024-04-25T10:20:40.645923+0000","last_peered":"2024-04-25T10:20:40.645923+0000","last_clean":"2024-04-25T10:20:40.645923+0000","last_became_active":"2024-04-25T10:20:38.645348+0000","last_became_peered":"2024-04-25T10:20:38.645348+0000","last_unstale":"2024-04-25T10:20:40.645923+0000","last_undegraded":"2024-04-25T10:20:40.645923+0000","last_fullsized":"2024-04-25T10:20:40.645923+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:56.800910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647710+0000","last_change":"2024-04-25T10:20:39.657071+0000","last_active":"2024-04-25T10:20:40.647710+0000","last_peered":"2024-04-25T10:20:40.647710+0000","last_clean":"2024-04-25T10:20:40.647710+0000","last_became_active":"2024-04-25T10:20:39.656741+0000","last_became_peered":"2024-04-25T10:20:39.656741+0000","last_unstale":"2024-04-25T10:20:40.647710+0000","last_undegraded":"2024-04-25T10:20:40.647710+0000","last_fullsized":"2024-04-25T10:20:40.647710+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:01.050069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647442+0000","last_change":"2024-04-25T10:20:38.643710+0000","last_active":"2024-04-25T10:20:40.647442+0000","last_peered":"2024-04-25T10:20:40.647442+0000","last_clean":"2024-04-25T10:20:40.647442+0000","last_became_active":"2024-04-25T10:20:38.643424+0000","last_became_peered":"2024-04-25T10:20:38.643424+0000","last_unstale":"2024-04-25T10:20:40.647442+0000","last_undegraded":"2024-04-25T10:20:40.647442+0000","last_fullsized":"2024-04-25T10:20:40.647442+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:19.758367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660546+0000","last_change":"2024-04-25T10:20:39.659573+0000","last_active":"2024-04-25T10:20:39.660546+0000","last_peered":"2024-04-25T10:20:39.660546+0000","last_clean":"2024-04-25T10:20:39.660546+0000","last_became_active":"2024-04-25T10:20:39.659287+0000","last_became_peered":"2024-04-25T10:20:39.659287+0000","last_unstale":"2024-04-25T10:20:39.660546+0000","last_undegraded":"2024-04-25T10:20:39.660546+0000","last_fullsized":"2024-04-25T10:20:39.660546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:30.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.629849+0000","last_change":"2024-04-25T10:20:38.641564+0000","last_active":"2024-04-25T10:20:39.629849+0000","last_peered":"2024-04-25T10:20:39.629849+0000","last_clean":"2024-04-25T10:20:39.629849+0000","last_became_active":"2024-04-25T10:20:38.641207+0000","last_became_peered":"2024-04-25T10:20:38.641207+0000","last_unstale":"2024-04-25T10:20:39.629849+0000","last_undegraded":"2024-04-25T10:20:39.629849+0000","last_fullsized":"2024-04-25T10:20:39.629849+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:21.546368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662585+0000","last_change":"2024-04-25T10:20:39.653285+0000","last_active":"2024-04-25T10:20:39.662585+0000","last_peered":"2024-04-25T10:20:39.662585+0000","last_clean":"2024-04-25T10:20:39.662585+0000","last_became_active":"2024-04-25T10:20:39.650378+0000","last_became_peered":"2024-04-25T10:20:39.650378+0000","last_unstale":"2024-04-25T10:20:39.662585+0000","last_undegraded":"2024-04-25T10:20:39.662585+0000","last_fullsized":"2024-04-25T10:20:39.662585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:23.610910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.635837+0000","last_change":"2024-04-25T10:20:38.648148+0000","last_active":"2024-04-25T10:20:39.635837+0000","last_peered":"2024-04-25T10:20:39.635837+0000","last_clean":"2024-04-25T10:20:39.635837+0000","last_became_active":"2024-04-25T10:20:38.647635+0000","last_became_peered":"2024-04-25T10:20:38.647635+0000","last_unstale":"2024-04-25T10:20:39.635837+0000","last_undegraded":"2024-04-25T10:20:39.635837+0000","last_fullsized":"2024-04-25T10:20:39.635837+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.987715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642729+0000","last_change":"2024-04-25T10:20:39.646523+0000","last_active":"2024-04-25T10:20:40.642729+0000","last_peered":"2024-04-25T10:20:40.642729+0000","last_clean":"2024-04-25T10:20:40.642729+0000","last_became_active":"2024-04-25T10:20:39.646262+0000","last_became_peered":"2024-04-25T10:20:39.646262+0000","last_unstale":"2024-04-25T10:20:40.642729+0000","last_undegraded":"2024-04-25T10:20:40.642729+0000","last_fullsized":"2024-04-25T10:20:40.642729+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:47.086858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646249+0000","last_change":"2024-04-25T10:20:38.649284+0000","last_active":"2024-04-25T10:20:40.646249+0000","last_peered":"2024-04-25T10:20:40.646249+0000","last_clean":"2024-04-25T10:20:40.646249+0000","last_became_active":"2024-04-25T10:20:38.648314+0000","last_became_peered":"2024-04-25T10:20:38.648314+0000","last_unstale":"2024-04-25T10:20:40.646249+0000","last_undegraded":"2024-04-25T10:20:40.646249+0000","last_fullsized":"2024-04-25T10:20:40.646249+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:55.070919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":331,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.643189+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:20:39.643189+0000","last_peered":"2024-04-25T10:20:39.643189+0000","last_clean":"2024-04-25T10:20:39.643189+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:20:39.643189+0000","last_undegraded":"2024-04-25T10:20:39.643189+0000","last_fullsized":"2024-04-25T10:20:39.643189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51160,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320680,"statfs":{"total":96636764160,"available":96584376320,"internally_reserved":0,"allocated":1970176,"data_stored":1183666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54748,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317092,"statfs":{"total":96636764160,"available":96580702208,"internally_reserved":0,"allocated":1384448,"data_stored":599052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46428,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325412,"statfs":{"total":96636764160,"available":96589221888,"internally_reserved":0,"allocated":1388544,"data_stored":599586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54732,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317108,"statfs":{"total":96636764160,"available":96580718592,"internally_reserved":0,"allocated":1368064,"data_stored":592682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50592,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321248,"statfs":{"total":96636764160,"available":96584957952,"internally_reserved":0,"allocated":1396736,"data_stored":600664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":1368064,"data_stored":592682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76940,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294900,"statfs":{"total":96636764160,"available":96557977600,"internally_reserved":0,"allocated":1376256,"data_stored":592806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51172,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320668,"statfs":{"total":96636764160,"available":96584364032,"internally_reserved":0,"allocated":1982464,"data_stored":1184774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:43.710 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:43.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7faff7600640 1 Processor -- start 2024-04-25T10:20:43.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7faff7600640 1 -- start start 2024-04-25T10:20:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7faff7600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7faff7600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faff00cda10 0x7faff00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7faff7600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff00b2770 con 0x7faff00cf930 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7faff7600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff006c670 con 0x7faff00cda10 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7faff7600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff006c950 con 0x7faff015bc40 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.879+0000 7fafe7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faff00cda10 0x7faff00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafe7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faff00cda10 0x7faff00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56024/0 (socket says 172.21.15.87:56024) 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafe7e00640 1 -- 172.21.15.87:0/1463310411 learned_addr learned my addr 172.21.15.87:0/1463310411 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7faff4c00640 1 --2- 172.21.15.87:0/1463310411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafe7e00640 1 -- 172.21.15.87:0/1463310411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 msgr2=0x7faff006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafe7e00640 1 --2- 172.21.15.87:0/1463310411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafe7e00640 1 -- 172.21.15.87:0/1463310411 >> v1:172.21.15.169:6789/0 conn(0x7faff00cf930 legacy=0x7faff015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafe7e00640 1 -- 172.21.15.87:0/1463310411 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff006cc30 con 0x7faff00cda10 2024-04-25T10:20:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafe7e00640 1 --2- 172.21.15.87:0/1463310411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cda10 0x7faff00b4610 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009870 tx=0x7fafd801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab7d8562cd9de19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafefe00640 1 -- 172.21.15.87:0/1463310411 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80a7070 con 0x7faff00cda10 2024-04-25T10:20:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafefe00640 1 -- 172.21.15.87:0/1463310411 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd80b0070 con 0x7faff00cda10 2024-04-25T10:20:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.883+0000 7fafefe00640 1 -- 172.21.15.87:0/1463310411 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80ab890 con 0x7faff00cda10 2024-04-25T10:20:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- 172.21.15.87:0/1463310411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cda10 msgr2=0x7faff00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 --2- 172.21.15.87:0/1463310411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cda10 0x7faff00b4610 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009870 tx=0x7fafd801c560 comp rx=0 tx=0).stop 2024-04-25T10:20:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- 172.21.15.87:0/1463310411 shutdown_connections 2024-04-25T10:20:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 --2- 172.21.15.87:0/1463310411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cda10 0x7faff00b4610 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 --2- 172.21.15.87:0/1463310411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- 172.21.15.87:0/1463310411 >> 172.21.15.87:0/1463310411 conn(0x7faff0066b70 msgr2=0x7faff0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- 172.21.15.87:0/1463310411 shutdown_connections 2024-04-25T10:20:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- 172.21.15.87:0/1463310411 wait complete. 2024-04-25T10:20:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 Processor -- start 2024-04-25T10:20:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- start start 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff00cda10 0x7faff01489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cf930 0x7faff015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff0168b50 con 0x7faff00cf930 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff01689d0 con 0x7faff00cda10 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff7600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff0168cd0 con 0x7faff015bc40 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cf930 0x7faff015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cf930 0x7faff015cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56034/0 (socket says 172.21.15.87:56034) 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff00cda10 0x7faff01489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 -- 172.21.15.87:0/1588745612 learned_addr learned my addr 172.21.15.87:0/1588745612 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7faff5600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 -- 172.21.15.87:0/1588745612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 msgr2=0x7faff015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 -- 172.21.15.87:0/1588745612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff00cda10 msgr2=0x7faff01489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff00cda10 0x7faff01489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.887+0000 7fafe7e00640 1 -- 172.21.15.87:0/1588745612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff015d7d0 con 0x7faff00cf930 2024-04-25T10:20:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7fafe7e00640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cf930 0x7faff015cca0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fafd801ac10 tx=0x7fafd801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7fafed600640 1 -- 172.21.15.87:0/1588745612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80be5f0 con 0x7faff00cf930 2024-04-25T10:20:43.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7fafed600640 1 -- 172.21.15.87:0/1588745612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd80be7b0 con 0x7faff00cf930 2024-04-25T10:20:43.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7fafed600640 1 -- 172.21.15.87:0/1588745612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80beba0 con 0x7faff00cf930 2024-04-25T10:20:43.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff015dab0 con 0x7faff00cf930 2024-04-25T10:20:43.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff0155660 con 0x7faff00cf930 2024-04-25T10:20:43.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff004ef30 con 0x7faff00cf930 2024-04-25T10:20:43.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7fafed600640 1 -- 172.21.15.87:0/1588745612 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafd80b43f0 con 0x7faff00cf930 2024-04-25T10:20:43.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7fafed600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafcc066f60 0x7fafcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7fafed600640 1 -- 172.21.15.87:0/1588745612 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafd811f940 con 0x7faff00cf930 2024-04-25T10:20:43.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.891+0000 7faff4c00640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafcc066f60 0x7fafcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.895+0000 7faff4c00640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafcc066f60 0x7fafcc069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009b60 tx=0x7fafe0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:43.895+0000 7fafed600640 1 -- 172.21.15.87:0/1588745612 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafd8016150 con 0x7faff00cf930 2024-04-25T10:20:44.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.047+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faff00af1c0 con 0x7fafcc066f60 2024-04-25T10:20:44.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7fafed600640 1 -- 172.21.15.87:0/1588745612 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252446 (secure 0 0 0) 0x7faff00af1c0 con 0x7fafcc066f60 2024-04-25T10:20:44.059 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:44.060 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafcc066f60 msgr2=0x7fafcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafcc066f60 0x7fafcc069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009b60 tx=0x7fafe0009190 comp rx=0 tx=0).stop 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cf930 msgr2=0x7faff015cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cf930 0x7faff015cca0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fafd801ac10 tx=0x7fafd801ada0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 shutdown_connections 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fafcc066f60 0x7fafcc069420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faff015bc40 0x7faff015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff00cf930 0x7faff015cca0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 --2- 172.21.15.87:0/1588745612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faff00cda10 0x7faff01489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.055+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 >> 172.21.15.87:0/1588745612 conn(0x7faff0066b70 msgr2=0x7faff015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.059+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 shutdown_connections 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.059+0000 7faff7600640 1 -- 172.21.15.87:0/1588745612 wait complete. 2024-04-25T10:20:44.079 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-04-25T10:20:43.230706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":432184,"kb_used_data":11948,"kb_used_omap":12,"kb_used_meta":420083,"kb_avail":754542536,"statfs":{"total":773094113280,"available":772651556864,"internally_reserved":0,"allocated":12234752,"data_stored":5945912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":430165553},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.609760"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662025+0000","last_change":"2024-04-25T10:20:39.642791+0000","last_active":"2024-04-25T10:20:39.662025+0000","last_peered":"2024-04-25T10:20:39.662025+0000","last_clean":"2024-04-25T10:20:39.662025+0000","last_became_active":"2024-04-25T10:20:39.641400+0000","last_became_peered":"2024-04-25T10:20:39.641400+0000","last_unstale":"2024-04-25T10:20:39.662025+0000","last_undegraded":"2024-04-25T10:20:39.662025+0000","last_fullsized":"2024-04-25T10:20:39.662025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:32.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.648170+0000","last_change":"2024-04-25T10:20:39.659816+0000","last_active":"2024-04-25T10:20:40.648170+0000","last_peered":"2024-04-25T10:20:40.648170+0000","last_clean":"2024-04-25T10:20:40.648170+0000","last_became_active":"2024-04-25T10:20:39.659540+0000","last_became_peered":"2024-04-25T10:20:39.659540+0000","last_unstale":"2024-04-25T10:20:40.648170+0000","last_undegraded":"2024-04-25T10:20:40.648170+0000","last_fullsized":"2024-04-25T10:20:40.648170+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:17.057311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655762+0000","last_change":"2024-04-25T10:20:39.649666+0000","last_active":"2024-04-25T10:20:39.655762+0000","last_peered":"2024-04-25T10:20:39.655762+0000","last_clean":"2024-04-25T10:20:39.655762+0000","last_became_active":"2024-04-25T10:20:39.647660+0000","last_became_peered":"2024-04-25T10:20:39.647660+0000","last_unstale":"2024-04-25T10:20:39.655762+0000","last_undegraded":"2024-04-25T10:20:39.655762+0000","last_fullsized":"2024-04-25T10:20:39.655762+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:20.653656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.663558+0000","last_change":"2024-04-25T10:20:39.641952+0000","last_active":"2024-04-25T10:20:39.663558+0000","last_peered":"2024-04-25T10:20:39.663558+0000","last_clean":"2024-04-25T10:20:39.663558+0000","last_became_active":"2024-04-25T10:20:39.641692+0000","last_became_peered":"2024-04-25T10:20:39.641692+0000","last_unstale":"2024-04-25T10:20:39.663558+0000","last_undegraded":"2024-04-25T10:20:39.663558+0000","last_fullsized":"2024-04-25T10:20:39.663558+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.319099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.664165+0000","last_change":"2024-04-25T10:20:39.652608+0000","last_active":"2024-04-25T10:20:39.664165+0000","last_peered":"2024-04-25T10:20:39.664165+0000","last_clean":"2024-04-25T10:20:39.664165+0000","last_became_active":"2024-04-25T10:20:39.651507+0000","last_became_peered":"2024-04-25T10:20:39.651507+0000","last_unstale":"2024-04-25T10:20:39.664165+0000","last_undegraded":"2024-04-25T10:20:39.664165+0000","last_fullsized":"2024-04-25T10:20:39.664165+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:51.285662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662784+0000","last_change":"2024-04-25T10:20:39.658661+0000","last_active":"2024-04-25T10:20:39.662784+0000","last_peered":"2024-04-25T10:20:39.662784+0000","last_clean":"2024-04-25T10:20:39.662784+0000","last_became_active":"2024-04-25T10:20:39.658385+0000","last_became_peered":"2024-04-25T10:20:39.658385+0000","last_unstale":"2024-04-25T10:20:39.662784+0000","last_undegraded":"2024-04-25T10:20:39.662784+0000","last_fullsized":"2024-04-25T10:20:39.662784+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:51.285662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646201+0000","last_change":"2024-04-25T10:20:39.652735+0000","last_active":"2024-04-25T10:20:40.646201+0000","last_peered":"2024-04-25T10:20:40.646201+0000","last_clean":"2024-04-25T10:20:40.646201+0000","last_became_active":"2024-04-25T10:20:39.651043+0000","last_became_peered":"2024-04-25T10:20:39.651043+0000","last_unstale":"2024-04-25T10:20:40.646201+0000","last_undegraded":"2024-04-25T10:20:40.646201+0000","last_fullsized":"2024-04-25T10:20:40.646201+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:08.589164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.659702+0000","last_change":"2024-04-25T10:20:39.651806+0000","last_active":"2024-04-25T10:20:39.659702+0000","last_peered":"2024-04-25T10:20:39.659702+0000","last_clean":"2024-04-25T10:20:39.659702+0000","last_became_active":"2024-04-25T10:20:39.651475+0000","last_became_peered":"2024-04-25T10:20:39.651475+0000","last_unstale":"2024-04-25T10:20:39.659702+0000","last_undegraded":"2024-04-25T10:20:39.659702+0000","last_fullsized":"2024-04-25T10:20:39.659702+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:54.075960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662089+0000","last_change":"2024-04-25T10:20:39.653738+0000","last_active":"2024-04-25T10:20:39.662089+0000","last_peered":"2024-04-25T10:20:39.662089+0000","last_clean":"2024-04-25T10:20:39.662089+0000","last_became_active":"2024-04-25T10:20:39.653339+0000","last_became_peered":"2024-04-25T10:20:39.653339+0000","last_unstale":"2024-04-25T10:20:39.662089+0000","last_undegraded":"2024-04-25T10:20:39.662089+0000","last_fullsized":"2024-04-25T10:20:39.662089+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:27.430334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.657755+0000","last_change":"2024-04-25T10:20:39.641326+0000","last_active":"2024-04-25T10:20:39.657755+0000","last_peered":"2024-04-25T10:20:39.657755+0000","last_clean":"2024-04-25T10:20:39.657755+0000","last_became_active":"2024-04-25T10:20:39.641003+0000","last_became_peered":"2024-04-25T10:20:39.641003+0000","last_unstale":"2024-04-25T10:20:39.657755+0000","last_undegraded":"2024-04-25T10:20:39.657755+0000","last_fullsized":"2024-04-25T10:20:39.657755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:01.050069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647900+0000","last_change":"2024-04-25T10:20:39.659083+0000","last_active":"2024-04-25T10:20:40.647900+0000","last_peered":"2024-04-25T10:20:40.647900+0000","last_clean":"2024-04-25T10:20:40.647900+0000","last_became_active":"2024-04-25T10:20:39.658785+0000","last_became_peered":"2024-04-25T10:20:39.658785+0000","last_unstale":"2024-04-25T10:20:40.647900+0000","last_undegraded":"2024-04-25T10:20:40.647900+0000","last_fullsized":"2024-04-25T10:20:40.647900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:47.086858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.663581+0000","last_change":"2024-04-25T10:20:39.643645+0000","last_active":"2024-04-25T10:20:39.663581+0000","last_peered":"2024-04-25T10:20:39.663581+0000","last_clean":"2024-04-25T10:20:39.663581+0000","last_became_active":"2024-04-25T10:20:39.641767+0000","last_became_peered":"2024-04-25T10:20:39.641767+0000","last_unstale":"2024-04-25T10:20:39.663581+0000","last_undegraded":"2024-04-25T10:20:39.663581+0000","last_fullsized":"2024-04-25T10:20:39.663581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:47.086858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660483+0000","last_change":"2024-04-25T10:20:39.657710+0000","last_active":"2024-04-25T10:20:39.660483+0000","last_peered":"2024-04-25T10:20:39.660483+0000","last_clean":"2024-04-25T10:20:39.660483+0000","last_became_active":"2024-04-25T10:20:39.657253+0000","last_became_peered":"2024-04-25T10:20:39.657253+0000","last_unstale":"2024-04-25T10:20:39.660483+0000","last_undegraded":"2024-04-25T10:20:39.660483+0000","last_fullsized":"2024-04-25T10:20:39.660483+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:34.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.659968+0000","last_change":"2024-04-25T10:20:39.652347+0000","last_active":"2024-04-25T10:20:39.659968+0000","last_peered":"2024-04-25T10:20:39.659968+0000","last_clean":"2024-04-25T10:20:39.659968+0000","last_became_active":"2024-04-25T10:20:39.652089+0000","last_became_peered":"2024-04-25T10:20:39.652089+0000","last_unstale":"2024-04-25T10:20:39.659968+0000","last_undegraded":"2024-04-25T10:20:39.659968+0000","last_fullsized":"2024-04-25T10:20:39.659968+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:11.074810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661946+0000","last_change":"2024-04-25T10:20:39.659409+0000","last_active":"2024-04-25T10:20:39.661946+0000","last_peered":"2024-04-25T10:20:39.661946+0000","last_clean":"2024-04-25T10:20:39.661946+0000","last_became_active":"2024-04-25T10:20:39.659141+0000","last_became_peered":"2024-04-25T10:20:39.659141+0000","last_unstale":"2024-04-25T10:20:39.661946+0000","last_undegraded":"2024-04-25T10:20:39.661946+0000","last_fullsized":"2024-04-25T10:20:39.661946+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:17.642871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661164+0000","last_change":"2024-04-25T10:20:39.645549+0000","last_active":"2024-04-25T10:20:39.661164+0000","last_peered":"2024-04-25T10:20:39.661164+0000","last_clean":"2024-04-25T10:20:39.661164+0000","last_became_active":"2024-04-25T10:20:39.645230+0000","last_became_peered":"2024-04-25T10:20:39.645230+0000","last_unstale":"2024-04-25T10:20:39.661164+0000","last_undegraded":"2024-04-25T10:20:39.661164+0000","last_fullsized":"2024-04-25T10:20:39.661164+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:17.057311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.648156+0000","last_change":"2024-04-25T10:20:39.659528+0000","last_active":"2024-04-25T10:20:40.648156+0000","last_peered":"2024-04-25T10:20:40.648156+0000","last_clean":"2024-04-25T10:20:40.648156+0000","last_became_active":"2024-04-25T10:20:39.658833+0000","last_became_peered":"2024-04-25T10:20:39.658833+0000","last_unstale":"2024-04-25T10:20:40.648156+0000","last_undegraded":"2024-04-25T10:20:40.648156+0000","last_fullsized":"2024-04-25T10:20:40.648156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:05.384710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650254+0000","last_change":"2024-04-25T10:20:39.649923+0000","last_active":"2024-04-25T10:20:40.650254+0000","last_peered":"2024-04-25T10:20:40.650254+0000","last_clean":"2024-04-25T10:20:40.650254+0000","last_became_active":"2024-04-25T10:20:39.649633+0000","last_became_peered":"2024-04-25T10:20:39.649633+0000","last_unstale":"2024-04-25T10:20:40.650254+0000","last_undegraded":"2024-04-25T10:20:40.650254+0000","last_fullsized":"2024-04-25T10:20:40.650254+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:35.776326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660296+0000","last_change":"2024-04-25T10:20:39.656606+0000","last_active":"2024-04-25T10:20:39.660296+0000","last_peered":"2024-04-25T10:20:39.660296+0000","last_clean":"2024-04-25T10:20:39.660296+0000","last_became_active":"2024-04-25T10:20:39.656258+0000","last_became_peered":"2024-04-25T10:20:39.656258+0000","last_unstale":"2024-04-25T10:20:39.660296+0000","last_undegraded":"2024-04-25T10:20:39.660296+0000","last_fullsized":"2024-04-25T10:20:39.660296+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.264186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647873+0000","last_change":"2024-04-25T10:20:39.648026+0000","last_active":"2024-04-25T10:20:40.647873+0000","last_peered":"2024-04-25T10:20:40.647873+0000","last_clean":"2024-04-25T10:20:40.647873+0000","last_became_active":"2024-04-25T10:20:39.647758+0000","last_became_peered":"2024-04-25T10:20:39.647758+0000","last_unstale":"2024-04-25T10:20:40.647873+0000","last_undegraded":"2024-04-25T10:20:40.647873+0000","last_fullsized":"2024-04-25T10:20:40.647873+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:32.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647837+0000","last_change":"2024-04-25T10:20:39.654142+0000","last_active":"2024-04-25T10:20:40.647837+0000","last_peered":"2024-04-25T10:20:40.647837+0000","last_clean":"2024-04-25T10:20:40.647837+0000","last_became_active":"2024-04-25T10:20:39.653830+0000","last_became_peered":"2024-04-25T10:20:39.653830+0000","last_unstale":"2024-04-25T10:20:40.647837+0000","last_undegraded":"2024-04-25T10:20:40.647837+0000","last_fullsized":"2024-04-25T10:20:40.647837+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.319099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642980+0000","last_change":"2024-04-25T10:20:39.654944+0000","last_active":"2024-04-25T10:20:40.642980+0000","last_peered":"2024-04-25T10:20:40.642980+0000","last_clean":"2024-04-25T10:20:40.642980+0000","last_became_active":"2024-04-25T10:20:39.654489+0000","last_became_peered":"2024-04-25T10:20:39.654489+0000","last_unstale":"2024-04-25T10:20:40.642980+0000","last_undegraded":"2024-04-25T10:20:40.642980+0000","last_fullsized":"2024-04-25T10:20:40.642980+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:17.057311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647276+0000","last_change":"2024-04-25T10:20:39.646775+0000","last_active":"2024-04-25T10:20:40.647276+0000","last_peered":"2024-04-25T10:20:40.647276+0000","last_clean":"2024-04-25T10:20:40.647276+0000","last_became_active":"2024-04-25T10:20:39.646493+0000","last_became_peered":"2024-04-25T10:20:39.646493+0000","last_unstale":"2024-04-25T10:20:40.647276+0000","last_undegraded":"2024-04-25T10:20:40.647276+0000","last_fullsized":"2024-04-25T10:20:40.647276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642725+0000","last_change":"2024-04-25T10:20:39.652343+0000","last_active":"2024-04-25T10:20:40.642725+0000","last_peered":"2024-04-25T10:20:40.642725+0000","last_clean":"2024-04-25T10:20:40.642725+0000","last_became_active":"2024-04-25T10:20:39.651965+0000","last_became_peered":"2024-04-25T10:20:39.651965+0000","last_unstale":"2024-04-25T10:20:40.642725+0000","last_undegraded":"2024-04-25T10:20:40.642725+0000","last_fullsized":"2024-04-25T10:20:40.642725+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:32.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.648127+0000","last_change":"2024-04-25T10:20:39.654597+0000","last_active":"2024-04-25T10:20:40.648127+0000","last_peered":"2024-04-25T10:20:40.648127+0000","last_clean":"2024-04-25T10:20:40.648127+0000","last_became_active":"2024-04-25T10:20:39.654350+0000","last_became_peered":"2024-04-25T10:20:39.654350+0000","last_unstale":"2024-04-25T10:20:40.648127+0000","last_undegraded":"2024-04-25T10:20:40.648127+0000","last_fullsized":"2024-04-25T10:20:40.648127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:04.923819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662283+0000","last_change":"2024-04-25T10:20:39.646234+0000","last_active":"2024-04-25T10:20:39.662283+0000","last_peered":"2024-04-25T10:20:39.662283+0000","last_clean":"2024-04-25T10:20:39.662283+0000","last_became_active":"2024-04-25T10:20:39.645979+0000","last_became_peered":"2024-04-25T10:20:39.645979+0000","last_unstale":"2024-04-25T10:20:39.662283+0000","last_undegraded":"2024-04-25T10:20:39.662283+0000","last_fullsized":"2024-04-25T10:20:39.662283+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.526291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.667719+0000","last_change":"2024-04-25T10:20:39.666655+0000","last_active":"2024-04-25T10:20:39.667719+0000","last_peered":"2024-04-25T10:20:39.667719+0000","last_clean":"2024-04-25T10:20:39.667719+0000","last_became_active":"2024-04-25T10:20:39.666100+0000","last_became_peered":"2024-04-25T10:20:39.666100+0000","last_unstale":"2024-04-25T10:20:39.667719+0000","last_undegraded":"2024-04-25T10:20:39.667719+0000","last_fullsized":"2024-04-25T10:20:39.667719+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:16.045822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643174+0000","last_change":"2024-04-25T10:20:39.656861+0000","last_active":"2024-04-25T10:20:40.643174+0000","last_peered":"2024-04-25T10:20:40.643174+0000","last_clean":"2024-04-25T10:20:40.643174+0000","last_became_active":"2024-04-25T10:20:39.656561+0000","last_became_peered":"2024-04-25T10:20:39.656561+0000","last_unstale":"2024-04-25T10:20:40.643174+0000","last_undegraded":"2024-04-25T10:20:40.643174+0000","last_fullsized":"2024-04-25T10:20:40.643174+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:45.284651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647489+0000","last_change":"2024-04-25T10:20:39.653869+0000","last_active":"2024-04-25T10:20:40.647489+0000","last_peered":"2024-04-25T10:20:40.647489+0000","last_clean":"2024-04-25T10:20:40.647489+0000","last_became_active":"2024-04-25T10:20:39.653373+0000","last_became_peered":"2024-04-25T10:20:39.653373+0000","last_unstale":"2024-04-25T10:20:40.647489+0000","last_undegraded":"2024-04-25T10:20:40.647489+0000","last_fullsized":"2024-04-25T10:20:40.647489+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:34.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646070+0000","last_change":"2024-04-25T10:20:39.652907+0000","last_active":"2024-04-25T10:20:40.646070+0000","last_peered":"2024-04-25T10:20:40.646070+0000","last_clean":"2024-04-25T10:20:40.646070+0000","last_became_active":"2024-04-25T10:20:39.652178+0000","last_became_peered":"2024-04-25T10:20:39.652178+0000","last_unstale":"2024-04-25T10:20:40.646070+0000","last_undegraded":"2024-04-25T10:20:40.646070+0000","last_fullsized":"2024-04-25T10:20:40.646070+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:11.261676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662564+0000","last_change":"2024-04-25T10:20:39.646974+0000","last_active":"2024-04-25T10:20:39.662564+0000","last_peered":"2024-04-25T10:20:39.662564+0000","last_clean":"2024-04-25T10:20:39.662564+0000","last_became_active":"2024-04-25T10:20:39.646728+0000","last_became_peered":"2024-04-25T10:20:39.646728+0000","last_unstale":"2024-04-25T10:20:39.662564+0000","last_undegraded":"2024-04-25T10:20:39.662564+0000","last_fullsized":"2024-04-25T10:20:39.662564+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.534489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.648147+0000","last_change":"2024-04-25T10:20:39.646912+0000","last_active":"2024-04-25T10:20:39.648147+0000","last_peered":"2024-04-25T10:20:39.648147+0000","last_clean":"2024-04-25T10:20:39.648147+0000","last_became_active":"2024-04-25T10:20:39.646641+0000","last_became_peered":"2024-04-25T10:20:39.646641+0000","last_unstale":"2024-04-25T10:20:39.648147+0000","last_undegraded":"2024-04-25T10:20:39.648147+0000","last_fullsized":"2024-04-25T10:20:39.648147+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:13.774579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.657385+0000","last_change":"2024-04-25T10:20:39.656111+0000","last_active":"2024-04-25T10:20:39.657385+0000","last_peered":"2024-04-25T10:20:39.657385+0000","last_clean":"2024-04-25T10:20:39.657385+0000","last_became_active":"2024-04-25T10:20:39.655781+0000","last_became_peered":"2024-04-25T10:20:39.655781+0000","last_unstale":"2024-04-25T10:20:39.657385+0000","last_undegraded":"2024-04-25T10:20:39.657385+0000","last_fullsized":"2024-04-25T10:20:39.657385+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:30.545372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646511+0000","last_change":"2024-04-25T10:20:38.646916+0000","last_active":"2024-04-25T10:20:40.646511+0000","last_peered":"2024-04-25T10:20:40.646511+0000","last_clean":"2024-04-25T10:20:40.646511+0000","last_became_active":"2024-04-25T10:20:38.646062+0000","last_became_peered":"2024-04-25T10:20:38.646062+0000","last_unstale":"2024-04-25T10:20:40.646511+0000","last_undegraded":"2024-04-25T10:20:40.646511+0000","last_fullsized":"2024-04-25T10:20:40.646511+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650399+0000","last_change":"2024-04-25T10:20:39.652720+0000","last_active":"2024-04-25T10:20:40.650399+0000","last_peered":"2024-04-25T10:20:40.650399+0000","last_clean":"2024-04-25T10:20:40.650399+0000","last_became_active":"2024-04-25T10:20:39.652391+0000","last_became_peered":"2024-04-25T10:20:39.652391+0000","last_unstale":"2024-04-25T10:20:40.650399+0000","last_undegraded":"2024-04-25T10:20:40.650399+0000","last_fullsized":"2024-04-25T10:20:40.650399+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660085+0000","last_change":"2024-04-25T10:20:38.647429+0000","last_active":"2024-04-25T10:20:39.660085+0000","last_peered":"2024-04-25T10:20:39.660085+0000","last_clean":"2024-04-25T10:20:39.660085+0000","last_became_active":"2024-04-25T10:20:38.647153+0000","last_became_peered":"2024-04-25T10:20:38.647153+0000","last_unstale":"2024-04-25T10:20:39.660085+0000","last_undegraded":"2024-04-25T10:20:39.660085+0000","last_fullsized":"2024-04-25T10:20:39.660085+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:20.235562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661936+0000","last_change":"2024-04-25T10:20:39.658933+0000","last_active":"2024-04-25T10:20:39.661936+0000","last_peered":"2024-04-25T10:20:39.661936+0000","last_clean":"2024-04-25T10:20:39.661936+0000","last_became_active":"2024-04-25T10:20:39.658398+0000","last_became_peered":"2024-04-25T10:20:39.658398+0000","last_unstale":"2024-04-25T10:20:39.661936+0000","last_undegraded":"2024-04-25T10:20:39.661936+0000","last_fullsized":"2024-04-25T10:20:39.661936+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:13.006053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.668583+0000","last_change":"2024-04-25T10:20:38.641543+0000","last_active":"2024-04-25T10:20:39.668583+0000","last_peered":"2024-04-25T10:20:39.668583+0000","last_clean":"2024-04-25T10:20:39.668583+0000","last_became_active":"2024-04-25T10:20:38.641213+0000","last_became_peered":"2024-04-25T10:20:38.641213+0000","last_unstale":"2024-04-25T10:20:39.668583+0000","last_undegraded":"2024-04-25T10:20:39.668583+0000","last_fullsized":"2024-04-25T10:20:39.668583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:18.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660316+0000","last_change":"2024-04-25T10:20:39.657179+0000","last_active":"2024-04-25T10:20:39.660316+0000","last_peered":"2024-04-25T10:20:39.660316+0000","last_clean":"2024-04-25T10:20:39.660316+0000","last_became_active":"2024-04-25T10:20:39.656912+0000","last_became_peered":"2024-04-25T10:20:39.656912+0000","last_unstale":"2024-04-25T10:20:39.660316+0000","last_undegraded":"2024-04-25T10:20:39.660316+0000","last_fullsized":"2024-04-25T10:20:39.660316+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.666594+0000","last_change":"2024-04-25T10:20:38.642997+0000","last_active":"2024-04-25T10:20:39.666594+0000","last_peered":"2024-04-25T10:20:39.666594+0000","last_clean":"2024-04-25T10:20:39.666594+0000","last_became_active":"2024-04-25T10:20:38.642700+0000","last_became_peered":"2024-04-25T10:20:38.642700+0000","last_unstale":"2024-04-25T10:20:39.666594+0000","last_undegraded":"2024-04-25T10:20:39.666594+0000","last_fullsized":"2024-04-25T10:20:39.666594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:30.610434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662350+0000","last_change":"2024-04-25T10:20:39.653345+0000","last_active":"2024-04-25T10:20:39.662350+0000","last_peered":"2024-04-25T10:20:39.662350+0000","last_clean":"2024-04-25T10:20:39.662350+0000","last_became_active":"2024-04-25T10:20:39.653078+0000","last_became_peered":"2024-04-25T10:20:39.653078+0000","last_unstale":"2024-04-25T10:20:39.662350+0000","last_undegraded":"2024-04-25T10:20:39.662350+0000","last_fullsized":"2024-04-25T10:20:39.662350+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:55.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.645914+0000","last_change":"2024-04-25T10:20:38.645381+0000","last_active":"2024-04-25T10:20:40.645914+0000","last_peered":"2024-04-25T10:20:40.645914+0000","last_clean":"2024-04-25T10:20:40.645914+0000","last_became_active":"2024-04-25T10:20:38.644984+0000","last_became_peered":"2024-04-25T10:20:38.644984+0000","last_unstale":"2024-04-25T10:20:40.645914+0000","last_undegraded":"2024-04-25T10:20:40.645914+0000","last_fullsized":"2024-04-25T10:20:40.645914+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:52.735354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647511+0000","last_change":"2024-04-25T10:20:39.660362+0000","last_active":"2024-04-25T10:20:40.647511+0000","last_peered":"2024-04-25T10:20:40.647511+0000","last_clean":"2024-04-25T10:20:40.647511+0000","last_became_active":"2024-04-25T10:20:39.659904+0000","last_became_peered":"2024-04-25T10:20:39.659904+0000","last_unstale":"2024-04-25T10:20:40.647511+0000","last_undegraded":"2024-04-25T10:20:40.647511+0000","last_fullsized":"2024-04-25T10:20:40.647511+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:30.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643084+0000","last_change":"2024-04-25T10:20:38.644434+0000","last_active":"2024-04-25T10:20:40.643084+0000","last_peered":"2024-04-25T10:20:40.643084+0000","last_clean":"2024-04-25T10:20:40.643084+0000","last_became_active":"2024-04-25T10:20:38.644168+0000","last_became_peered":"2024-04-25T10:20:38.644168+0000","last_unstale":"2024-04-25T10:20:40.643084+0000","last_undegraded":"2024-04-25T10:20:40.643084+0000","last_fullsized":"2024-04-25T10:20:40.643084+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:23.667467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661189+0000","last_change":"2024-04-25T10:20:39.649821+0000","last_active":"2024-04-25T10:20:39.661189+0000","last_peered":"2024-04-25T10:20:39.661189+0000","last_clean":"2024-04-25T10:20:39.661189+0000","last_became_active":"2024-04-25T10:20:39.649548+0000","last_became_peered":"2024-04-25T10:20:39.649548+0000","last_unstale":"2024-04-25T10:20:39.661189+0000","last_undegraded":"2024-04-25T10:20:39.661189+0000","last_fullsized":"2024-04-25T10:20:39.661189+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:29.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647298+0000","last_change":"2024-04-25T10:20:38.646179+0000","last_active":"2024-04-25T10:20:40.647298+0000","last_peered":"2024-04-25T10:20:40.647298+0000","last_clean":"2024-04-25T10:20:40.647298+0000","last_became_active":"2024-04-25T10:20:38.645911+0000","last_became_peered":"2024-04-25T10:20:38.645911+0000","last_unstale":"2024-04-25T10:20:40.647298+0000","last_undegraded":"2024-04-25T10:20:40.647298+0000","last_fullsized":"2024-04-25T10:20:40.647298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:21.713286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646037+0000","last_change":"2024-04-25T10:20:39.655750+0000","last_active":"2024-04-25T10:20:40.646037+0000","last_peered":"2024-04-25T10:20:40.646037+0000","last_clean":"2024-04-25T10:20:40.646037+0000","last_became_active":"2024-04-25T10:20:39.650566+0000","last_became_peered":"2024-04-25T10:20:39.650566+0000","last_unstale":"2024-04-25T10:20:40.646037+0000","last_undegraded":"2024-04-25T10:20:40.646037+0000","last_fullsized":"2024-04-25T10:20:40.646037+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:18.223411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.635567+0000","last_change":"2024-04-25T10:20:38.647912+0000","last_active":"2024-04-25T10:20:39.635567+0000","last_peered":"2024-04-25T10:20:39.635567+0000","last_clean":"2024-04-25T10:20:39.635567+0000","last_became_active":"2024-04-25T10:20:38.647275+0000","last_became_peered":"2024-04-25T10:20:38.647275+0000","last_unstale":"2024-04-25T10:20:39.635567+0000","last_undegraded":"2024-04-25T10:20:39.635567+0000","last_fullsized":"2024-04-25T10:20:39.635567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:34.770299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646479+0000","last_change":"2024-04-25T10:20:39.659008+0000","last_active":"2024-04-25T10:20:40.646479+0000","last_peered":"2024-04-25T10:20:40.646479+0000","last_clean":"2024-04-25T10:20:40.646479+0000","last_became_active":"2024-04-25T10:20:39.658725+0000","last_became_peered":"2024-04-25T10:20:39.658725+0000","last_unstale":"2024-04-25T10:20:40.646479+0000","last_undegraded":"2024-04-25T10:20:40.646479+0000","last_fullsized":"2024-04-25T10:20:40.646479+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:51.768470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.630483+0000","last_change":"2024-04-25T10:20:38.643558+0000","last_active":"2024-04-25T10:20:39.630483+0000","last_peered":"2024-04-25T10:20:39.630483+0000","last_clean":"2024-04-25T10:20:39.630483+0000","last_became_active":"2024-04-25T10:20:38.643286+0000","last_became_peered":"2024-04-25T10:20:38.643286+0000","last_unstale":"2024-04-25T10:20:39.630483+0000","last_undegraded":"2024-04-25T10:20:39.630483+0000","last_fullsized":"2024-04-25T10:20:39.630483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:12.768553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650452+0000","last_change":"2024-04-25T10:20:39.648670+0000","last_active":"2024-04-25T10:20:40.650452+0000","last_peered":"2024-04-25T10:20:40.650452+0000","last_clean":"2024-04-25T10:20:40.650452+0000","last_became_active":"2024-04-25T10:20:39.647657+0000","last_became_peered":"2024-04-25T10:20:39.647657+0000","last_unstale":"2024-04-25T10:20:40.650452+0000","last_undegraded":"2024-04-25T10:20:40.650452+0000","last_fullsized":"2024-04-25T10:20:40.650452+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:34.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643333+0000","last_change":"2024-04-25T10:20:38.643850+0000","last_active":"2024-04-25T10:20:40.643333+0000","last_peered":"2024-04-25T10:20:40.643333+0000","last_clean":"2024-04-25T10:20:40.643333+0000","last_became_active":"2024-04-25T10:20:38.643602+0000","last_became_peered":"2024-04-25T10:20:38.643602+0000","last_unstale":"2024-04-25T10:20:40.643333+0000","last_undegraded":"2024-04-25T10:20:40.643333+0000","last_fullsized":"2024-04-25T10:20:40.643333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:53.069934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643171+0000","last_change":"2024-04-25T10:20:39.649529+0000","last_active":"2024-04-25T10:20:40.643171+0000","last_peered":"2024-04-25T10:20:40.643171+0000","last_clean":"2024-04-25T10:20:40.643171+0000","last_became_active":"2024-04-25T10:20:39.649233+0000","last_became_peered":"2024-04-25T10:20:39.649233+0000","last_unstale":"2024-04-25T10:20:40.643171+0000","last_undegraded":"2024-04-25T10:20:40.643171+0000","last_fullsized":"2024-04-25T10:20:40.643171+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:29.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"110'3","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.667711+0000","last_change":"2024-04-25T10:20:38.649213+0000","last_active":"2024-04-25T10:20:39.667711+0000","last_peered":"2024-04-25T10:20:39.667711+0000","last_clean":"2024-04-25T10:20:39.667711+0000","last_became_active":"2024-04-25T10:20:38.648950+0000","last_became_peered":"2024-04-25T10:20:38.648950+0000","last_unstale":"2024-04-25T10:20:39.667711+0000","last_undegraded":"2024-04-25T10:20:39.667711+0000","last_fullsized":"2024-04-25T10:20:39.667711+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:33.851385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646337+0000","last_change":"2024-04-25T10:20:39.654796+0000","last_active":"2024-04-25T10:20:40.646337+0000","last_peered":"2024-04-25T10:20:40.646337+0000","last_clean":"2024-04-25T10:20:40.646337+0000","last_became_active":"2024-04-25T10:20:39.650003+0000","last_became_peered":"2024-04-25T10:20:39.650003+0000","last_unstale":"2024-04-25T10:20:40.646337+0000","last_undegraded":"2024-04-25T10:20:40.646337+0000","last_fullsized":"2024-04-25T10:20:40.646337+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:11.221161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"110'6","reported_seq":18,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.671276+0000","last_change":"2024-04-25T10:20:38.649519+0000","last_active":"2024-04-25T10:20:39.671276+0000","last_peered":"2024-04-25T10:20:39.671276+0000","last_clean":"2024-04-25T10:20:39.671276+0000","last_became_active":"2024-04-25T10:20:38.647111+0000","last_became_peered":"2024-04-25T10:20:38.647111+0000","last_unstale":"2024-04-25T10:20:39.671276+0000","last_undegraded":"2024-04-25T10:20:39.671276+0000","last_fullsized":"2024-04-25T10:20:39.671276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:10.068783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643025+0000","last_change":"2024-04-25T10:20:39.648935+0000","last_active":"2024-04-25T10:20:40.643025+0000","last_peered":"2024-04-25T10:20:40.643025+0000","last_clean":"2024-04-25T10:20:40.643025+0000","last_became_active":"2024-04-25T10:20:39.648692+0000","last_became_peered":"2024-04-25T10:20:39.648692+0000","last_unstale":"2024-04-25T10:20:40.643025+0000","last_undegraded":"2024-04-25T10:20:40.643025+0000","last_fullsized":"2024-04-25T10:20:40.643025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.526291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.665826+0000","last_change":"2024-04-25T10:20:38.643464+0000","last_active":"2024-04-25T10:20:39.665826+0000","last_peered":"2024-04-25T10:20:39.665826+0000","last_clean":"2024-04-25T10:20:39.665826+0000","last_became_active":"2024-04-25T10:20:38.643071+0000","last_became_peered":"2024-04-25T10:20:38.643071+0000","last_unstale":"2024-04-25T10:20:39.665826+0000","last_undegraded":"2024-04-25T10:20:39.665826+0000","last_fullsized":"2024-04-25T10:20:39.665826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:30.610434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643088+0000","last_change":"2024-04-25T10:20:39.652808+0000","last_active":"2024-04-25T10:20:40.643088+0000","last_peered":"2024-04-25T10:20:40.643088+0000","last_clean":"2024-04-25T10:20:40.643088+0000","last_became_active":"2024-04-25T10:20:39.652301+0000","last_became_peered":"2024-04-25T10:20:39.652301+0000","last_unstale":"2024-04-25T10:20:40.643088+0000","last_undegraded":"2024-04-25T10:20:40.643088+0000","last_fullsized":"2024-04-25T10:20:40.643088+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.534489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.649968+0000","last_change":"2024-04-25T10:20:38.649707+0000","last_active":"2024-04-25T10:20:40.649968+0000","last_peered":"2024-04-25T10:20:40.649968+0000","last_clean":"2024-04-25T10:20:40.649968+0000","last_became_active":"2024-04-25T10:20:38.649332+0000","last_became_peered":"2024-04-25T10:20:38.649332+0000","last_unstale":"2024-04-25T10:20:40.649968+0000","last_undegraded":"2024-04-25T10:20:40.649968+0000","last_fullsized":"2024-04-25T10:20:40.649968+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:57.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642745+0000","last_change":"2024-04-25T10:20:39.648738+0000","last_active":"2024-04-25T10:20:40.642745+0000","last_peered":"2024-04-25T10:20:40.642745+0000","last_clean":"2024-04-25T10:20:40.642745+0000","last_became_active":"2024-04-25T10:20:39.648422+0000","last_became_peered":"2024-04-25T10:20:39.648422+0000","last_unstale":"2024-04-25T10:20:40.642745+0000","last_undegraded":"2024-04-25T10:20:40.642745+0000","last_fullsized":"2024-04-25T10:20:40.642745+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:04.923819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"110'8","reported_seq":20,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.670166+0000","last_change":"2024-04-25T10:20:38.639055+0000","last_active":"2024-04-25T10:20:39.670166+0000","last_peered":"2024-04-25T10:20:39.670166+0000","last_clean":"2024-04-25T10:20:39.670166+0000","last_became_active":"2024-04-25T10:20:38.636323+0000","last_became_peered":"2024-04-25T10:20:38.636323+0000","last_unstale":"2024-04-25T10:20:39.670166+0000","last_undegraded":"2024-04-25T10:20:39.670166+0000","last_fullsized":"2024-04-25T10:20:39.670166+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.031609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655276+0000","last_change":"2024-04-25T10:20:39.650747+0000","last_active":"2024-04-25T10:20:39.655276+0000","last_peered":"2024-04-25T10:20:39.655276+0000","last_clean":"2024-04-25T10:20:39.655276+0000","last_became_active":"2024-04-25T10:20:39.650506+0000","last_became_peered":"2024-04-25T10:20:39.650506+0000","last_unstale":"2024-04-25T10:20:39.655276+0000","last_undegraded":"2024-04-25T10:20:39.655276+0000","last_fullsized":"2024-04-25T10:20:39.655276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:30.545372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.668104+0000","last_change":"2024-04-25T10:20:38.646455+0000","last_active":"2024-04-25T10:20:39.668104+0000","last_peered":"2024-04-25T10:20:39.668104+0000","last_clean":"2024-04-25T10:20:39.668104+0000","last_became_active":"2024-04-25T10:20:38.646113+0000","last_became_peered":"2024-04-25T10:20:38.646113+0000","last_unstale":"2024-04-25T10:20:39.668104+0000","last_undegraded":"2024-04-25T10:20:39.668104+0000","last_fullsized":"2024-04-25T10:20:39.668104+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:34.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.665051+0000","last_change":"2024-04-25T10:20:39.663863+0000","last_active":"2024-04-25T10:20:39.665051+0000","last_peered":"2024-04-25T10:20:39.665051+0000","last_clean":"2024-04-25T10:20:39.665051+0000","last_became_active":"2024-04-25T10:20:39.663482+0000","last_became_peered":"2024-04-25T10:20:39.663482+0000","last_unstale":"2024-04-25T10:20:39.665051+0000","last_undegraded":"2024-04-25T10:20:39.665051+0000","last_fullsized":"2024-04-25T10:20:39.665051+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:57.041605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.669895+0000","last_change":"2024-04-25T10:20:38.636365+0000","last_active":"2024-04-25T10:20:39.669895+0000","last_peered":"2024-04-25T10:20:39.669895+0000","last_clean":"2024-04-25T10:20:39.669895+0000","last_became_active":"2024-04-25T10:20:38.635394+0000","last_became_peered":"2024-04-25T10:20:38.635394+0000","last_unstale":"2024-04-25T10:20:39.669895+0000","last_undegraded":"2024-04-25T10:20:39.669895+0000","last_fullsized":"2024-04-25T10:20:39.669895+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:20.365553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.653063+0000","last_change":"2024-04-25T10:20:39.648658+0000","last_active":"2024-04-25T10:20:39.653063+0000","last_peered":"2024-04-25T10:20:39.653063+0000","last_clean":"2024-04-25T10:20:39.653063+0000","last_became_active":"2024-04-25T10:20:39.648403+0000","last_became_peered":"2024-04-25T10:20:39.648403+0000","last_unstale":"2024-04-25T10:20:39.653063+0000","last_undegraded":"2024-04-25T10:20:39.653063+0000","last_fullsized":"2024-04-25T10:20:39.653063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:04.956589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650088+0000","last_change":"2024-04-25T10:20:38.647871+0000","last_active":"2024-04-25T10:20:40.650088+0000","last_peered":"2024-04-25T10:20:40.650088+0000","last_clean":"2024-04-25T10:20:40.650088+0000","last_became_active":"2024-04-25T10:20:38.647479+0000","last_became_peered":"2024-04-25T10:20:38.647479+0000","last_unstale":"2024-04-25T10:20:40.650088+0000","last_undegraded":"2024-04-25T10:20:40.650088+0000","last_fullsized":"2024-04-25T10:20:40.650088+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:21.425913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655743+0000","last_change":"2024-04-25T10:20:39.648970+0000","last_active":"2024-04-25T10:20:39.655743+0000","last_peered":"2024-04-25T10:20:39.655743+0000","last_clean":"2024-04-25T10:20:39.655743+0000","last_became_active":"2024-04-25T10:20:39.648013+0000","last_became_peered":"2024-04-25T10:20:39.648013+0000","last_unstale":"2024-04-25T10:20:39.655743+0000","last_undegraded":"2024-04-25T10:20:39.655743+0000","last_fullsized":"2024-04-25T10:20:39.655743+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:24.673494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646058+0000","last_change":"2024-04-25T10:20:38.652355+0000","last_active":"2024-04-25T10:20:40.646058+0000","last_peered":"2024-04-25T10:20:40.646058+0000","last_clean":"2024-04-25T10:20:40.646058+0000","last_became_active":"2024-04-25T10:20:38.652049+0000","last_became_peered":"2024-04-25T10:20:38.652049+0000","last_unstale":"2024-04-25T10:20:40.646058+0000","last_undegraded":"2024-04-25T10:20:40.646058+0000","last_fullsized":"2024-04-25T10:20:40.646058+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:36.693750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.655709+0000","last_change":"2024-04-25T10:20:39.649351+0000","last_active":"2024-04-25T10:20:39.655709+0000","last_peered":"2024-04-25T10:20:39.655709+0000","last_clean":"2024-04-25T10:20:39.655709+0000","last_became_active":"2024-04-25T10:20:39.647588+0000","last_became_peered":"2024-04-25T10:20:39.647588+0000","last_unstale":"2024-04-25T10:20:39.655709+0000","last_undegraded":"2024-04-25T10:20:39.655709+0000","last_fullsized":"2024-04-25T10:20:39.655709+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:16.386812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.669743+0000","last_change":"2024-04-25T10:20:38.643742+0000","last_active":"2024-04-25T10:20:39.669743+0000","last_peered":"2024-04-25T10:20:39.669743+0000","last_clean":"2024-04-25T10:20:39.669743+0000","last_became_active":"2024-04-25T10:20:38.643472+0000","last_became_peered":"2024-04-25T10:20:38.643472+0000","last_unstale":"2024-04-25T10:20:39.669743+0000","last_undegraded":"2024-04-25T10:20:39.669743+0000","last_fullsized":"2024-04-25T10:20:39.669743+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:18.119478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.661136+0000","last_change":"2024-04-25T10:20:39.644552+0000","last_active":"2024-04-25T10:20:39.661136+0000","last_peered":"2024-04-25T10:20:39.661136+0000","last_clean":"2024-04-25T10:20:39.661136+0000","last_became_active":"2024-04-25T10:20:39.644274+0000","last_became_peered":"2024-04-25T10:20:39.644274+0000","last_unstale":"2024-04-25T10:20:39.661136+0000","last_undegraded":"2024-04-25T10:20:39.661136+0000","last_fullsized":"2024-04-25T10:20:39.661136+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:05.384710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.629817+0000","last_change":"2024-04-25T10:20:38.641441+0000","last_active":"2024-04-25T10:20:39.629817+0000","last_peered":"2024-04-25T10:20:39.629817+0000","last_clean":"2024-04-25T10:20:39.629817+0000","last_became_active":"2024-04-25T10:20:38.641204+0000","last_became_peered":"2024-04-25T10:20:38.641204+0000","last_unstale":"2024-04-25T10:20:39.629817+0000","last_undegraded":"2024-04-25T10:20:39.629817+0000","last_fullsized":"2024-04-25T10:20:39.629817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:22.620727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660268+0000","last_change":"2024-04-25T10:20:39.656372+0000","last_active":"2024-04-25T10:20:39.660268+0000","last_peered":"2024-04-25T10:20:39.660268+0000","last_clean":"2024-04-25T10:20:39.660268+0000","last_became_active":"2024-04-25T10:20:39.656024+0000","last_became_peered":"2024-04-25T10:20:39.656024+0000","last_unstale":"2024-04-25T10:20:39.660268+0000","last_undegraded":"2024-04-25T10:20:39.660268+0000","last_fullsized":"2024-04-25T10:20:39.660268+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:35.776326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.644296+0000","last_change":"2024-04-25T10:20:38.647179+0000","last_active":"2024-04-25T10:20:39.644296+0000","last_peered":"2024-04-25T10:20:39.644296+0000","last_clean":"2024-04-25T10:20:39.644296+0000","last_became_active":"2024-04-25T10:20:38.646864+0000","last_became_peered":"2024-04-25T10:20:38.646864+0000","last_unstale":"2024-04-25T10:20:39.644296+0000","last_undegraded":"2024-04-25T10:20:39.644296+0000","last_fullsized":"2024-04-25T10:20:39.644296+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:20.365553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642815+0000","last_change":"2024-04-25T10:20:39.644973+0000","last_active":"2024-04-25T10:20:40.642815+0000","last_peered":"2024-04-25T10:20:40.642815+0000","last_clean":"2024-04-25T10:20:40.642815+0000","last_became_active":"2024-04-25T10:20:39.643727+0000","last_became_peered":"2024-04-25T10:20:39.643727+0000","last_unstale":"2024-04-25T10:20:40.642815+0000","last_undegraded":"2024-04-25T10:20:40.642815+0000","last_fullsized":"2024-04-25T10:20:40.642815+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:05.384710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.635708+0000","last_change":"2024-04-25T10:20:38.648235+0000","last_active":"2024-04-25T10:20:39.635708+0000","last_peered":"2024-04-25T10:20:39.635708+0000","last_clean":"2024-04-25T10:20:39.635708+0000","last_became_active":"2024-04-25T10:20:38.647963+0000","last_became_peered":"2024-04-25T10:20:38.647963+0000","last_unstale":"2024-04-25T10:20:39.635708+0000","last_undegraded":"2024-04-25T10:20:39.635708+0000","last_fullsized":"2024-04-25T10:20:39.635708+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:12.258160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650257+0000","last_change":"2024-04-25T10:20:39.649305+0000","last_active":"2024-04-25T10:20:40.650257+0000","last_peered":"2024-04-25T10:20:40.650257+0000","last_clean":"2024-04-25T10:20:40.650257+0000","last_became_active":"2024-04-25T10:20:39.647621+0000","last_became_peered":"2024-04-25T10:20:39.647621+0000","last_unstale":"2024-04-25T10:20:40.650257+0000","last_undegraded":"2024-04-25T10:20:40.650257+0000","last_fullsized":"2024-04-25T10:20:40.650257+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.264186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643510+0000","last_change":"2024-04-25T10:20:38.648026+0000","last_active":"2024-04-25T10:20:40.643510+0000","last_peered":"2024-04-25T10:20:40.643510+0000","last_clean":"2024-04-25T10:20:40.643510+0000","last_became_active":"2024-04-25T10:20:38.647753+0000","last_became_peered":"2024-04-25T10:20:38.647753+0000","last_unstale":"2024-04-25T10:20:40.643510+0000","last_undegraded":"2024-04-25T10:20:40.643510+0000","last_fullsized":"2024-04-25T10:20:40.643510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:15.380785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662051+0000","last_change":"2024-04-25T10:20:39.644409+0000","last_active":"2024-04-25T10:20:39.662051+0000","last_peered":"2024-04-25T10:20:39.662051+0000","last_clean":"2024-04-25T10:20:39.662051+0000","last_became_active":"2024-04-25T10:20:39.644155+0000","last_became_peered":"2024-04-25T10:20:39.644155+0000","last_unstale":"2024-04-25T10:20:39.662051+0000","last_undegraded":"2024-04-25T10:20:39.662051+0000","last_fullsized":"2024-04-25T10:20:39.662051+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:04.923819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"111'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.651675+0000","last_change":"2024-04-25T10:20:38.646629+0000","last_active":"2024-04-25T10:20:40.651675+0000","last_peered":"2024-04-25T10:20:40.651675+0000","last_clean":"2024-04-25T10:20:40.651675+0000","last_became_active":"2024-04-25T10:20:38.646323+0000","last_became_peered":"2024-04-25T10:20:38.646323+0000","last_unstale":"2024-04-25T10:20:40.651675+0000","last_undegraded":"2024-04-25T10:20:40.651675+0000","last_fullsized":"2024-04-25T10:20:40.651675+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:11.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.663602+0000","last_change":"2024-04-25T10:20:39.650067+0000","last_active":"2024-04-25T10:20:39.663602+0000","last_peered":"2024-04-25T10:20:39.663602+0000","last_clean":"2024-04-25T10:20:39.663602+0000","last_became_active":"2024-04-25T10:20:39.649822+0000","last_became_peered":"2024-04-25T10:20:39.649822+0000","last_unstale":"2024-04-25T10:20:39.663602+0000","last_undegraded":"2024-04-25T10:20:39.663602+0000","last_fullsized":"2024-04-25T10:20:39.663602+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:45.284651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.636338+0000","last_change":"2024-04-25T10:20:38.647846+0000","last_active":"2024-04-25T10:20:39.636338+0000","last_peered":"2024-04-25T10:20:39.636338+0000","last_clean":"2024-04-25T10:20:39.636338+0000","last_became_active":"2024-04-25T10:20:38.647577+0000","last_became_peered":"2024-04-25T10:20:38.647577+0000","last_unstale":"2024-04-25T10:20:39.636338+0000","last_undegraded":"2024-04-25T10:20:39.636338+0000","last_fullsized":"2024-04-25T10:20:39.636338+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:29.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.650154+0000","last_change":"2024-04-25T10:20:39.643726+0000","last_active":"2024-04-25T10:20:40.650154+0000","last_peered":"2024-04-25T10:20:40.650154+0000","last_clean":"2024-04-25T10:20:40.650154+0000","last_became_active":"2024-04-25T10:20:39.643410+0000","last_became_peered":"2024-04-25T10:20:39.643410+0000","last_unstale":"2024-04-25T10:20:40.650154+0000","last_undegraded":"2024-04-25T10:20:40.650154+0000","last_fullsized":"2024-04-25T10:20:40.650154+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:11.074810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.643279+0000","last_change":"2024-04-25T10:20:38.645150+0000","last_active":"2024-04-25T10:20:40.643279+0000","last_peered":"2024-04-25T10:20:40.643279+0000","last_clean":"2024-04-25T10:20:40.643279+0000","last_became_active":"2024-04-25T10:20:38.644775+0000","last_became_peered":"2024-04-25T10:20:38.644775+0000","last_unstale":"2024-04-25T10:20:40.643279+0000","last_undegraded":"2024-04-25T10:20:40.643279+0000","last_fullsized":"2024-04-25T10:20:40.643279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:19.647629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647514+0000","last_change":"2024-04-25T10:20:39.648066+0000","last_active":"2024-04-25T10:20:40.647514+0000","last_peered":"2024-04-25T10:20:40.647514+0000","last_clean":"2024-04-25T10:20:40.647514+0000","last_became_active":"2024-04-25T10:20:39.647804+0000","last_became_peered":"2024-04-25T10:20:39.647804+0000","last_unstale":"2024-04-25T10:20:40.647514+0000","last_undegraded":"2024-04-25T10:20:40.647514+0000","last_fullsized":"2024-04-25T10:20:40.647514+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:35.517703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.645923+0000","last_change":"2024-04-25T10:20:38.645651+0000","last_active":"2024-04-25T10:20:40.645923+0000","last_peered":"2024-04-25T10:20:40.645923+0000","last_clean":"2024-04-25T10:20:40.645923+0000","last_became_active":"2024-04-25T10:20:38.645348+0000","last_became_peered":"2024-04-25T10:20:38.645348+0000","last_unstale":"2024-04-25T10:20:40.645923+0000","last_undegraded":"2024-04-25T10:20:40.645923+0000","last_fullsized":"2024-04-25T10:20:40.645923+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:56.800910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647710+0000","last_change":"2024-04-25T10:20:39.657071+0000","last_active":"2024-04-25T10:20:40.647710+0000","last_peered":"2024-04-25T10:20:40.647710+0000","last_clean":"2024-04-25T10:20:40.647710+0000","last_became_active":"2024-04-25T10:20:39.656741+0000","last_became_peered":"2024-04-25T10:20:39.656741+0000","last_unstale":"2024-04-25T10:20:40.647710+0000","last_undegraded":"2024-04-25T10:20:40.647710+0000","last_fullsized":"2024-04-25T10:20:40.647710+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:01.050069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.647442+0000","last_change":"2024-04-25T10:20:38.643710+0000","last_active":"2024-04-25T10:20:40.647442+0000","last_peered":"2024-04-25T10:20:40.647442+0000","last_clean":"2024-04-25T10:20:40.647442+0000","last_became_active":"2024-04-25T10:20:38.643424+0000","last_became_peered":"2024-04-25T10:20:38.643424+0000","last_unstale":"2024-04-25T10:20:40.647442+0000","last_undegraded":"2024-04-25T10:20:40.647442+0000","last_fullsized":"2024-04-25T10:20:40.647442+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:19.758367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.660546+0000","last_change":"2024-04-25T10:20:39.659573+0000","last_active":"2024-04-25T10:20:39.660546+0000","last_peered":"2024-04-25T10:20:39.660546+0000","last_clean":"2024-04-25T10:20:39.660546+0000","last_became_active":"2024-04-25T10:20:39.659287+0000","last_became_peered":"2024-04-25T10:20:39.659287+0000","last_unstale":"2024-04-25T10:20:39.660546+0000","last_undegraded":"2024-04-25T10:20:39.660546+0000","last_fullsized":"2024-04-25T10:20:39.660546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:30.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.629849+0000","last_change":"2024-04-25T10:20:38.641564+0000","last_active":"2024-04-25T10:20:39.629849+0000","last_peered":"2024-04-25T10:20:39.629849+0000","last_clean":"2024-04-25T10:20:39.629849+0000","last_became_active":"2024-04-25T10:20:38.641207+0000","last_became_peered":"2024-04-25T10:20:38.641207+0000","last_unstale":"2024-04-25T10:20:39.629849+0000","last_undegraded":"2024-04-25T10:20:39.629849+0000","last_fullsized":"2024-04-25T10:20:39.629849+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:21.546368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.662585+0000","last_change":"2024-04-25T10:20:39.653285+0000","last_active":"2024-04-25T10:20:39.662585+0000","last_peered":"2024-04-25T10:20:39.662585+0000","last_clean":"2024-04-25T10:20:39.662585+0000","last_became_active":"2024-04-25T10:20:39.650378+0000","last_became_peered":"2024-04-25T10:20:39.650378+0000","last_unstale":"2024-04-25T10:20:39.662585+0000","last_undegraded":"2024-04-25T10:20:39.662585+0000","last_fullsized":"2024-04-25T10:20:39.662585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:23.610910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.635837+0000","last_change":"2024-04-25T10:20:38.648148+0000","last_active":"2024-04-25T10:20:39.635837+0000","last_peered":"2024-04-25T10:20:39.635837+0000","last_clean":"2024-04-25T10:20:39.635837+0000","last_became_active":"2024-04-25T10:20:38.647635+0000","last_became_peered":"2024-04-25T10:20:38.647635+0000","last_unstale":"2024-04-25T10:20:39.635837+0000","last_undegraded":"2024-04-25T10:20:39.635837+0000","last_fullsized":"2024-04-25T10:20:39.635837+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.987715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.642729+0000","last_change":"2024-04-25T10:20:39.646523+0000","last_active":"2024-04-25T10:20:40.642729+0000","last_peered":"2024-04-25T10:20:40.642729+0000","last_clean":"2024-04-25T10:20:40.642729+0000","last_became_active":"2024-04-25T10:20:39.646262+0000","last_became_peered":"2024-04-25T10:20:39.646262+0000","last_unstale":"2024-04-25T10:20:40.642729+0000","last_undegraded":"2024-04-25T10:20:40.642729+0000","last_fullsized":"2024-04-25T10:20:40.642729+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:38.618690+0000","last_clean_scrub_stamp":"2024-04-25T10:20:38.618690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:47.086858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.646249+0000","last_change":"2024-04-25T10:20:38.649284+0000","last_active":"2024-04-25T10:20:40.646249+0000","last_peered":"2024-04-25T10:20:40.646249+0000","last_clean":"2024-04-25T10:20:40.646249+0000","last_became_active":"2024-04-25T10:20:38.648314+0000","last_became_peered":"2024-04-25T10:20:38.648314+0000","last_unstale":"2024-04-25T10:20:40.646249+0000","last_undegraded":"2024-04-25T10:20:40.646249+0000","last_fullsized":"2024-04-25T10:20:40.646249+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.612664+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.612664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:55.070919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":331,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.643189+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:20:39.643189+0000","last_peered":"2024-04-25T10:20:39.643189+0000","last_clean":"2024-04-25T10:20:39.643189+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:20:39.643189+0000","last_undegraded":"2024-04-25T10:20:39.643189+0000","last_fullsized":"2024-04-25T10:20:39.643189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51160,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320680,"statfs":{"total":96636764160,"available":96584376320,"internally_reserved":0,"allocated":1970176,"data_stored":1183666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54748,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317092,"statfs":{"total":96636764160,"available":96580702208,"internally_reserved":0,"allocated":1384448,"data_stored":599052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46428,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325412,"statfs":{"total":96636764160,"available":96589221888,"internally_reserved":0,"allocated":1388544,"data_stored":599586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54732,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317108,"statfs":{"total":96636764160,"available":96580718592,"internally_reserved":0,"allocated":1368064,"data_stored":592682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50592,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321248,"statfs":{"total":96636764160,"available":96584957952,"internally_reserved":0,"allocated":1396736,"data_stored":600664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":1368064,"data_stored":592682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76940,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294900,"statfs":{"total":96636764160,"available":96557977600,"internally_reserved":0,"allocated":1376256,"data_stored":592806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51172,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320668,"statfs":{"total":96636764160,"available":96584364032,"internally_reserved":0,"allocated":1982464,"data_stored":1184774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:44.124 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:20:44.124 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:20:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c43c00640 1 Processor -- start 2024-04-25T10:20:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c43c00640 1 -- start start 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c43c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c3c0b4660 0x7f6c3c06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c43c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0cf960 0x7f6c3c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c43c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c06ca90 con 0x7f6c3c15bbb0 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c43c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c06d050 con 0x7f6c3c0b4660 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c43c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c06cd70 con 0x7f6c3c0cf960 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c3be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c3c0b4660 0x7f6c3c06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c3be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c3c0b4660 0x7f6c3c06c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56040/0 (socket says 172.21.15.87:56040) 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c3be00640 1 -- 172.21.15.87:0/3883571911 learned_addr learned my addr 172.21.15.87:0/3883571911 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c41c00640 1 --2- 172.21.15.87:0/3883571911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0cf960 0x7f6c3c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c3be00640 1 -- 172.21.15.87:0/3883571911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0cf960 msgr2=0x7f6c3c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c3be00640 1 --2- 172.21.15.87:0/3883571911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0cf960 0x7f6c3c0cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c3be00640 1 -- 172.21.15.87:0/3883571911 >> v1:172.21.15.169:6789/0 conn(0x7f6c3c15bbb0 legacy=0x7f6c3c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c3be00640 1 -- 172.21.15.87:0/3883571911 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0b2880 con 0x7f6c3c0b4660 2024-04-25T10:20:44.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.275+0000 7f6c41c00640 1 --2- 172.21.15.87:0/3883571911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0cf960 0x7f6c3c0cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c3be00640 1 --2- 172.21.15.87:0/3883571911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0b4660 0x7f6c3c06c120 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009870 tx=0x7f6c2801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46da4c42b02161ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c3b400640 1 -- 172.21.15.87:0/3883571911 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280b2070 con 0x7f6c3c0b4660 2024-04-25T10:20:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c3b400640 1 -- 172.21.15.87:0/3883571911 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c280a9070 con 0x7f6c3c0b4660 2024-04-25T10:20:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c3b400640 1 -- 172.21.15.87:0/3883571911 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280ad880 con 0x7f6c3c0b4660 2024-04-25T10:20:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 -- 172.21.15.87:0/3883571911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0b4660 msgr2=0x7f6c3c06c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3883571911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0b4660 0x7f6c3c06c120 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009870 tx=0x7f6c2801c560 comp rx=0 tx=0).stop 2024-04-25T10:20:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 -- 172.21.15.87:0/3883571911 shutdown_connections 2024-04-25T10:20:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3883571911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0cf960 0x7f6c3c0cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3883571911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0b4660 0x7f6c3c06c120 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 -- 172.21.15.87:0/3883571911 >> 172.21.15.87:0/3883571911 conn(0x7f6c3c066b70 msgr2=0x7f6c3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 -- 172.21.15.87:0/3883571911 shutdown_connections 2024-04-25T10:20:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.279+0000 7f6c43c00640 1 -- 172.21.15.87:0/3883571911 wait complete. 2024-04-25T10:20:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 Processor -- start 2024-04-25T10:20:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 -- start start 2024-04-25T10:20:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0b4660 0x7f6c3c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0cf960 0x7f6c3c14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c15bbb0 0x7f6c3c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b3ae0 con 0x7f6c3c0cf960 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c06c870 con 0x7f6c3c15bbb0 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c168960 con 0x7f6c3c0b4660 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0cf960 0x7f6c3c14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0cf960 0x7f6c3c14db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56042/0 (socket says 172.21.15.87:56042) 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 -- 172.21.15.87:0/3891068447 learned_addr learned my addr 172.21.15.87:0/3891068447 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 -- 172.21.15.87:0/3891068447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0b4660 msgr2=0x7f6c3c155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0b4660 0x7f6c3c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 -- 172.21.15.87:0/3891068447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c15bbb0 msgr2=0x7f6c3c070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c15bbb0 0x7f6c3c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 -- 172.21.15.87:0/3891068447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c166290 con 0x7f6c3c0cf960 2024-04-25T10:20:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c3be00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0cf960 0x7f6c3c14db90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6c280b0040 tx=0x7f6c28019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c38c00640 1 -- 172.21.15.87:0/3891068447 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280bb070 con 0x7f6c3c0cf960 2024-04-25T10:20:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c38c00640 1 -- 172.21.15.87:0/3891068447 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c280addd0 con 0x7f6c3c0cf960 2024-04-25T10:20:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c38c00640 1 -- 172.21.15.87:0/3891068447 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280b2040 con 0x7f6c3c0cf960 2024-04-25T10:20:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c153ce0 con 0x7f6c3c0cf960 2024-04-25T10:20:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c1542a0 con 0x7f6c3c0cf960 2024-04-25T10:20:44.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.283+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c04ef30 con 0x7f6c3c0cf960 2024-04-25T10:20:44.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.287+0000 7f6c38c00640 1 -- 172.21.15.87:0/3891068447 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c280ab070 con 0x7f6c3c0cf960 2024-04-25T10:20:44.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.287+0000 7f6c38c00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c1c066f60 0x7f6c1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.287+0000 7f6c38c00640 1 -- 172.21.15.87:0/3891068447 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c2811df30 con 0x7f6c3c0cf960 2024-04-25T10:20:44.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.287+0000 7f6c41200640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c1c066f60 0x7f6c1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.287+0000 7f6c41200640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c1c066f60 0x7f6c1c069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6c30000f70 tx=0x7f6c300079e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.291+0000 7f6c38c00640 1 -- 172.21.15.87:0/3891068447 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c28016150 con 0x7f6c3c0cf960 2024-04-25T10:20:44.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.499+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6c3c06d980 con 0x7f6c3c0cf960 2024-04-25T10:20:44.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c38c00640 1 -- 172.21.15.87:0/3891068447 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6c280eeea0 con 0x7f6c3c0cf960 2024-04-25T10:20:44.505 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:20:44.505 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:20:44.505 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:20:44.505 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:20:44.506 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:20:44.506 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:20:44.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c1c066f60 msgr2=0x7f6c1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c1c066f60 0x7f6c1c069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6c30000f70 tx=0x7f6c300079e0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0cf960 msgr2=0x7f6c3c14db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0cf960 0x7f6c3c14db90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6c280b0040 tx=0x7f6c28019300 comp rx=0 tx=0).stop 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 shutdown_connections 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c1c066f60 0x7f6c1c069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c15bbb0 0x7f6c3c070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c3c0cf960 0x7f6c3c14db90 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 --2- 172.21.15.87:0/3891068447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3c0b4660 0x7f6c3c155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 >> 172.21.15.87:0/3891068447 conn(0x7f6c3c066b70 msgr2=0x7f6c3c1595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 shutdown_connections 2024-04-25T10:20:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.503+0000 7f6c43c00640 1 -- 172.21.15.87:0/3891068447 wait complete. 2024-04-25T10:20:44.528 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:44.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.659+0000 7f404f400640 1 Processor -- start 2024-04-25T10:20:44.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.659+0000 7f404f400640 1 -- start start 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480b0f70 0x7f40480b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40480cfd90 0x7f40480ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40480cd110 0x7f40480cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404f400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404815ef20 con 0x7f40480cfd90 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404f400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404815f3b0 con 0x7f40480cd110 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40480ca810 con 0x7f40480b0f70 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40480cfd90 0x7f40480ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40480cfd90 0x7f40480ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56050/0 (socket says 172.21.15.87:56050) 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 -- 172.21.15.87:0/689831423 learned_addr learned my addr 172.21.15.87:0/689831423 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:44.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404d400640 1 --2- 172.21.15.87:0/689831423 >> v2:172.21.15.169:3300/0 conn(0x7f40480cd110 0x7f40480cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f404ca00640 1 --2- 172.21.15.87:0/689831423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480b0f70 0x7f40480b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 -- 172.21.15.87:0/689831423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480b0f70 msgr2=0x7f40480b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 --2- 172.21.15.87:0/689831423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480b0f70 0x7f40480b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 -- 172.21.15.87:0/689831423 >> v2:172.21.15.169:3300/0 conn(0x7f40480cd110 msgr2=0x7f40480cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 --2- 172.21.15.87:0/689831423 >> v2:172.21.15.169:3300/0 conn(0x7f40480cd110 0x7f40480cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 -- 172.21.15.87:0/689831423 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480b1890 con 0x7f40480cfd90 2024-04-25T10:20:44.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.663+0000 7f403fe00640 1 --2- 172.21.15.87:0/689831423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f40480ccbd0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f40480b2d50 tx=0x7f403001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e781c38452d09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f4047e00640 1 -- 172.21.15.87:0/689831423 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300b0070 con 0x7f40480cfd90 2024-04-25T10:20:44.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f4047e00640 1 -- 172.21.15.87:0/689831423 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40300ab600 con 0x7f40480cfd90 2024-04-25T10:20:44.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f4047e00640 1 -- 172.21.15.87:0/689831423 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300ab9d0 con 0x7f40480cfd90 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- 172.21.15.87:0/689831423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 msgr2=0x7f40480ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 --2- 172.21.15.87:0/689831423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f40480ccbd0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f40480b2d50 tx=0x7f403001c520 comp rx=0 tx=0).stop 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- 172.21.15.87:0/689831423 shutdown_connections 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 --2- 172.21.15.87:0/689831423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f40480ccbd0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 --2- 172.21.15.87:0/689831423 >> v2:172.21.15.169:3300/0 conn(0x7f40480cd110 0x7f40480cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 --2- 172.21.15.87:0/689831423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480b0f70 0x7f40480b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- 172.21.15.87:0/689831423 >> 172.21.15.87:0/689831423 conn(0x7f40480b1d50 msgr2=0x7f40480b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- 172.21.15.87:0/689831423 shutdown_connections 2024-04-25T10:20:44.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- 172.21.15.87:0/689831423 wait complete. 2024-04-25T10:20:44.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 Processor -- start 2024-04-25T10:20:44.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- start start 2024-04-25T10:20:44.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40480b0f70 0x7f404806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480cd110 0x7f40480ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f4048092aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048072200 con 0x7f40480cfd90 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40480cd520 con 0x7f40480b0f70 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4048072380 con 0x7f40480cd110 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f4048092aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f4048092aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56060/0 (socket says 172.21.15.87:56060) 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404d400640 1 -- 172.21.15.87:0/4109239649 learned_addr learned my addr 172.21.15.87:0/4109239649 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.667+0000 7f404ca00640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40480b0f70 0x7f404806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f403fe00640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480cd110 0x7f40480ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404d400640 1 -- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480cd110 msgr2=0x7f40480ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404d400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480cd110 0x7f40480ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404d400640 1 -- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40480b0f70 msgr2=0x7f404806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404d400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40480b0f70 0x7f404806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404d400640 1 -- 172.21.15.87:0/4109239649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480af280 con 0x7f40480cfd90 2024-04-25T10:20:44.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404d400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f4048092aa0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4040007e70 tx=0x7f404000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f4045600640 1 -- 172.21.15.87:0/4109239649 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040014620 con 0x7f40480cfd90 2024-04-25T10:20:44.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f4045600640 1 -- 172.21.15.87:0/4109239649 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040019070 con 0x7f40480cfd90 2024-04-25T10:20:44.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f4045600640 1 -- 172.21.15.87:0/4109239649 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040014a20 con 0x7f40480cfd90 2024-04-25T10:20:44.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40480af560 con 0x7f40480cfd90 2024-04-25T10:20:44.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404806e9f0 con 0x7f40480cfd90 2024-04-25T10:20:44.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404804ef30 con 0x7f40480cfd90 2024-04-25T10:20:44.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f4045600640 1 -- 172.21.15.87:0/4109239649 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4040014bc0 con 0x7f40480cfd90 2024-04-25T10:20:44.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f4045600640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f4045600640 1 -- 172.21.15.87:0/4109239649 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40400859e0 con 0x7f40480cfd90 2024-04-25T10:20:44.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.671+0000 7f404ca00640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.675+0000 7f404ca00640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4024066f60 0x7f4024069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f403800ba30 tx=0x7f4038008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.675+0000 7f4045600640 1 -- 172.21.15.87:0/4109239649 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4040011260 con 0x7f40480cfd90 2024-04-25T10:20:44.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.863+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40480cda20 con 0x7f40480cfd90 2024-04-25T10:20:44.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.863+0000 7f4045600640 1 -- 172.21.15.87:0/4109239649 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4040056950 con 0x7f40480cfd90 2024-04-25T10:20:44.867 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:20:44.867 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:20:44.868 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:20:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4024066f60 msgr2=0x7f4024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4024066f60 0x7f4024069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f403800ba30 tx=0x7f4038008040 comp rx=0 tx=0).stop 2024-04-25T10:20:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 msgr2=0x7f4048092aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f4048092aa0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4040007e70 tx=0x7f404000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 shutdown_connections 2024-04-25T10:20:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40480cfd90 0x7f4048092aa0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40480cd110 0x7f40480ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 --2- 172.21.15.87:0/4109239649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40480b0f70 0x7f404806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 >> 172.21.15.87:0/4109239649 conn(0x7f40480b1d50 msgr2=0x7f404806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 shutdown_connections 2024-04-25T10:20:44.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:44.867+0000 7f404f400640 1 -- 172.21.15.87:0/4109239649 wait complete. 2024-04-25T10:20:45.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:39.010608+0000', 'modified': '2024-04-25T10:20:42.026817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8523}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8523': {'gid': 8523, 'name': 'a', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/3078360268', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 3078360268}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 3078360268}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:20:45.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:20:45.302 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:20:45.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.447+0000 7ff97e200640 1 Processor -- start 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.447+0000 7ff97e200640 1 -- start start 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff97e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97815bc40 0x7ff97806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff97e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9780cda10 0x7ff9780b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff97e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9780b2770 con 0x7ff9780cf930 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff97e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff97806c670 con 0x7ff9780cda10 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff97e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97806c950 con 0x7ff97815bc40 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff96ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9780cda10 0x7ff9780b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff96ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9780cda10 0x7ff9780b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56070/0 (socket says 172.21.15.87:56070) 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff96ea00640 1 -- 172.21.15.87:0/4116042976 learned_addr learned my addr 172.21.15.87:0/4116042976 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff977400640 1 --2- 172.21.15.87:0/4116042976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97815bc40 0x7ff97806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff96ea00640 1 -- 172.21.15.87:0/4116042976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97815bc40 msgr2=0x7ff97806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff96ea00640 1 --2- 172.21.15.87:0/4116042976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97815bc40 0x7ff97806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff96ea00640 1 -- 172.21.15.87:0/4116042976 >> v1:172.21.15.169:6789/0 conn(0x7ff9780cf930 legacy=0x7ff97815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.451+0000 7ff96ea00640 1 -- 172.21.15.87:0/4116042976 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97806cc30 con 0x7ff9780cda10 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff96ea00640 1 --2- 172.21.15.87:0/4116042976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9780cda10 0x7ff9780b4610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff960009870 tx=0x7ff96001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f427cc83f12c76bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff976a00640 1 -- 172.21.15.87:0/4116042976 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600ab5f0 con 0x7ff9780cda10 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff976a00640 1 -- 172.21.15.87:0/4116042976 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9600b0070 con 0x7ff9780cda10 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff976a00640 1 -- 172.21.15.87:0/4116042976 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600ab9b0 con 0x7ff9780cda10 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 -- 172.21.15.87:0/4116042976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9780cda10 msgr2=0x7ff9780b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 --2- 172.21.15.87:0/4116042976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9780cda10 0x7ff9780b4610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff960009870 tx=0x7ff96001c520 comp rx=0 tx=0).stop 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 -- 172.21.15.87:0/4116042976 shutdown_connections 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 --2- 172.21.15.87:0/4116042976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9780cda10 0x7ff9780b4610 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 --2- 172.21.15.87:0/4116042976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97815bc40 0x7ff97806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 -- 172.21.15.87:0/4116042976 >> 172.21.15.87:0/4116042976 conn(0x7ff978066b70 msgr2=0x7ff978066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 -- 172.21.15.87:0/4116042976 shutdown_connections 2024-04-25T10:20:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 -- 172.21.15.87:0/4116042976 wait complete. 2024-04-25T10:20:45.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 Processor -- start 2024-04-25T10:20:45.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.455+0000 7ff97e200640 1 -- start start 2024-04-25T10:20:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9780cda10 0x7ff978070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9780cf930 0x7ff978150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97815bc40 0x7ff978071e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff978168b50 con 0x7ff97815bc40 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781689d0 con 0x7ff9780cda10 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff978168cd0 con 0x7ff9780cf930 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97815bc40 0x7ff978071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97815bc40 0x7ff978071e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56084/0 (socket says 172.21.15.87:56084) 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 -- 172.21.15.87:0/563101548 learned_addr learned my addr 172.21.15.87:0/563101548 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff96ea00640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9780cf930 0x7ff978150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977400640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9780cda10 0x7ff978070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 -- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9780cf930 msgr2=0x7ff978150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9780cf930 0x7ff978150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 -- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9780cda10 msgr2=0x7ff978070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9780cda10 0x7ff978070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 -- 172.21.15.87:0/563101548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff978070d70 con 0x7ff97815bc40 2024-04-25T10:20:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff977e00640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97815bc40 0x7ff978071e10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c780 tx=0x7ff97000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff96fe00640 1 -- 172.21.15.87:0/563101548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970004b70 con 0x7ff97815bc40 2024-04-25T10:20:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff96fe00640 1 -- 172.21.15.87:0/563101548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff970004d10 con 0x7ff97815bc40 2024-04-25T10:20:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff96fe00640 1 -- 172.21.15.87:0/563101548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000aa90 con 0x7ff97815bc40 2024-04-25T10:20:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff978071050 con 0x7ff97815bc40 2024-04-25T10:20:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff978071610 con 0x7ff97815bc40 2024-04-25T10:20:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.459+0000 7ff96fe00640 1 -- 172.21.15.87:0/563101548 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff97000ac30 con 0x7ff97815bc40 2024-04-25T10:20:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96fe00640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff954066f60 0x7ff954069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96fe00640 1 -- 172.21.15.87:0/563101548 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff970085300 con 0x7ff97815bc40 2024-04-25T10:20:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff977400640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff954066f60 0x7ff954069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 Processor -- start 2024-04-25T10:20:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 -- start start 2024-04-25T10:20:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff977400640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff954066f60 0x7ff954069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff9780703c0 tx=0x7ff968008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc90cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc8c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc7aa5f0 con 0x55f4cc90d240 2024-04-25T10:20:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc711da0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc69a150 con 0x55f4cc90c920 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56086/0 (socket says 172.21.15.87:56086) 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 -- 172.21.15.87:0/4106864633 learned_addr learned my addr 172.21.15.87:0/4106864633 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff977400640 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc90cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff977e00640 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc8c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 -- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 msgr2=0x55f4cc90cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc90cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 -- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 msgr2=0x55f4cc8c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc8c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 -- 172.21.15.87:0/4106864633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4cc6b3ca0 con 0x55f4cc90d240 2024-04-25T10:20:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff977400640 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc90cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff977e00640 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc8c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff96ea00640 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c29f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7ff96001abd0 tx=0x7ff960004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf3aa53f89af7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff967e00640 1 -- 172.21.15.87:0/4106864633 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600ba070 con 0x55f4cc90d240 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff967e00640 1 -- 172.21.15.87:0/4106864633 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9600b0040 con 0x55f4cc90d240 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff967e00640 1 -- 172.21.15.87:0/4106864633 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600abc70 con 0x55f4cc90d240 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 -- 172.21.15.87:0/4106864633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 msgr2=0x55f4cc8c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c29f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7ff96001abd0 tx=0x7ff960004740 comp rx=0 tx=0).stop 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 -- 172.21.15.87:0/4106864633 shutdown_connections 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc8c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c29f0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 --2- 172.21.15.87:0/4106864633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc90cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.463+0000 7ff981695000 1 -- 172.21.15.87:0/4106864633 >> 172.21.15.87:0/4106864633 conn(0x55f4cc8c92a0 msgr2=0x55f4cc8c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- 172.21.15.87:0/4106864633 shutdown_connections 2024-04-25T10:20:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- 172.21.15.87:0/4106864633 wait complete. 2024-04-25T10:20:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- start start 2024-04-25T10:20:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 Processor -- start 2024-04-25T10:20:45.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc913c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc6cc990 con 0x55f4cc90d240 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc70de40 con 0x55f4cc8c2f30 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc69d900 con 0x55f4cc90c920 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 10 client.0 _ll_register_callbacks cb 0x55f4cc7c51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56094/0 (socket says 172.21.15.87:56094) 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 -- 172.21.15.87:0/2067030159 learned_addr learned my addr 172.21.15.87:0/2067030159 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff96ea00640 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977400640 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc913c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 -- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 msgr2=0x55f4cc9141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9141c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 -- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 msgr2=0x55f4cc913c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc913c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 -- 172.21.15.87:0/2067030159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4cc6ca630 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977e00640 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c6aa0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff970052060 tx=0x7ff9700563c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298ef672ab708286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff964c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff964c00640 1 -- 172.21.15.87:0/2067030159 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97006d070 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff964c00640 1 -- 172.21.15.87:0/2067030159 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff970053f00 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9600b0040 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff964c00640 1 -- 172.21.15.87:0/2067030159 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97005e310 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 10 client.8661 fetch_fsmap learned FSMap version 80 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55f4cc69a3a0 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff964c00640 1 -- 172.21.15.87:0/2067030159 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9600b0040 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff964c00640 1 -- 172.21.15.87:0/2067030159 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff970052270 con 0x55f4cc90d240 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 10 client.8661 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 10 client.8661 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:20:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 20 client.8661 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc8c74b0 0x55f4cc8c7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 4 client.8661 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4cc6ada50 con 0x55f4cc8c74b0 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.467+0000 7ff977400640 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc8c74b0 0x55f4cc8c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff977400640 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc8c74b0 0x55f4cc8c7890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920aa908ab96974d server_cookie=62deb71f0c17799f in_seq=0 out_seq=0 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff964c00640 10 client.8661 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff964c00640 1 -- 172.21.15.87:0/2067030159 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4cc6ada50 con 0x55f4cc8c74b0 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff964c00640 10 client.8661 handle_command_reply: tid=0 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 client.8661 shutdown 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 msgr2=0x55f4cc8c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c6aa0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff970052060 tx=0x7ff9700563c0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 shutdown_connections 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc8c74b0 0x55f4cc8c7890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc90d240 0x55f4cc8c6aa0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff964c00640 0 client.8661 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:20:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 --2- 172.21.15.87:0/2067030159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc913c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 >> 172.21.15.87:0/2067030159 conn(0x55f4cc8c92a0 msgr2=0x55f4cc8c19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 shutdown_connections 2024-04-25T10:20:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 1 -- 172.21.15.87:0/2067030159 wait complete. 2024-04-25T10:20:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.471+0000 7ff981695000 20 client.8661 trim_cache size 0 max 16384 2024-04-25T10:20:45.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 Processor -- start 2024-04-25T10:20:45.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- start start 2024-04-25T10:20:45.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c92a0 0x55f4cc967f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc9684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9780ae510 con 0x55f4cc8c92a0 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9780ad910 con 0x55f4cc8c2f30 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9780adf10 con 0x55f4cc90c920 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c92a0 0x55f4cc967f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c92a0 0x55f4cc967f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56104/0 (socket says 172.21.15.87:56104) 2024-04-25T10:20:45.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 -- 172.21.15.87:0/1598616531 learned_addr learned my addr 172.21.15.87:0/1598616531 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff96ea00640 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc9684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977e00640 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 -- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 msgr2=0x55f4cc9689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 -- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 msgr2=0x55f4cc9684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc9684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 -- 172.21.15.87:0/1598616531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4cc793600 con 0x55f4cc8c92a0 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff96ea00640 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc9684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977e00640 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff977400640 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c92a0 0x55f4cc967f60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff968004260 tx=0x7ff968014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928a7074b3928108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff967400640 1 -- 172.21.15.87:0/1598616531 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800ece0 con 0x55f4cc8c92a0 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff967400640 1 -- 172.21.15.87:0/1598616531 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96800ee80 con 0x55f4cc8c92a0 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff967400640 1 -- 172.21.15.87:0/1598616531 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968003af0 con 0x55f4cc8c92a0 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- 172.21.15.87:0/1598616531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c92a0 msgr2=0x55f4cc967f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c92a0 0x55f4cc967f60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff968004260 tx=0x7ff968014000 comp rx=0 tx=0).stop 2024-04-25T10:20:45.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- 172.21.15.87:0/1598616531 shutdown_connections 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc9689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c2f30 0x55f4cc9684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 --2- 172.21.15.87:0/1598616531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c92a0 0x55f4cc967f60 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- 172.21.15.87:0/1598616531 >> 172.21.15.87:0/1598616531 conn(0x55f4cc914190 msgr2=0x55f4cc8c6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- 172.21.15.87:0/1598616531 shutdown_connections 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.487+0000 7ff981695000 1 -- 172.21.15.87:0/1598616531 wait complete. 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 -- start start 2024-04-25T10:20:45.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 Processor -- start 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c2f30 0x55f4cc919580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c92a0 0x55f4cc954800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc954d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc7163d0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc7cdd50 con 0x55f4cc8c92a0 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4cc6ac980 con 0x55f4cc90c920 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 10 client.0 _ll_register_callbacks cb 0x7ff9600a6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c2f30 0x55f4cc919580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c2f30 0x55f4cc919580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56116/0 (socket says 172.21.15.87:56116) 2024-04-25T10:20:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 -- 172.21.15.87:0/3041927994 learned_addr learned my addr 172.21.15.87:0/3041927994 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977e00640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc954d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff96ea00640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c92a0 0x55f4cc954800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 -- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 msgr2=0x55f4cc954d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc954d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 -- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c92a0 msgr2=0x55f4cc954800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c92a0 0x55f4cc954800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 -- 172.21.15.87:0/3041927994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4cc6a69b0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977e00640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc954d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff977400640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c2f30 0x55f4cc919580 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x55f4cc790960 tx=0x7ff968004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d42f4e658be1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff964c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff964c00640 1 -- 172.21.15.87:0/3041927994 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9680059a0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff964c00640 1 -- 172.21.15.87:0/3041927994 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff968005b40 con 0x55f4cc8c2f30 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4cc71f2e0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff964c00640 1 -- 172.21.15.87:0/3041927994 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff968025030 con 0x55f4cc8c2f30 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 10 client.8667 fetch_fsmap learned FSMap version 80 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55f4cc6b90b0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff964c00640 1 -- 172.21.15.87:0/3041927994 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f4cc71f2e0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff964c00640 1 -- 172.21.15.87:0/3041927994 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff96800ece0 con 0x55f4cc8c2f30 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 10 client.8667 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 10 client.8667 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 20 client.8667 populate_metadata read hostname 'smithi087' 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc90d240 0x55f4cc955280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 4 client.8667 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f4cc719630 con 0x55f4cc90d240 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff96ea00640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc90d240 0x55f4cc955280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff96ea00640 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc90d240 0x55f4cc955280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8f1542b73f8b85 server_cookie=357b26aa7a6326f1 in_seq=0 out_seq=0 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.491+0000 7ff964c00640 10 client.8667 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff964c00640 1 -- 172.21.15.87:0/3041927994 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x55f4cc719630 con 0x55f4cc90d240 2024-04-25T10:20:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff964c00640 10 client.8667 handle_command_reply: tid=0 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 client.8667 shutdown 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c2f30 msgr2=0x55f4cc919580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c2f30 0x55f4cc919580 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x55f4cc790960 tx=0x7ff968004740 comp rx=0 tx=0).stop 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 shutdown_connections 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55f4cc90d240 0x55f4cc955280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f4cc90c920 0x55f4cc954d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f4cc8c92a0 0x55f4cc954800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 --2- 172.21.15.87:0/3041927994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f4cc8c2f30 0x55f4cc919580 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 >> 172.21.15.87:0/3041927994 conn(0x55f4cc914190 msgr2=0x55f4cc914840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff964c00640 0 client.8667 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 shutdown_connections 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 1 -- 172.21.15.87:0/3041927994 wait complete. 2024-04-25T10:20:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff981695000 20 client.8667 trim_cache size 0 max 16384 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 8523, 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.861193557, 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 80, 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 111, 2024-04-25T10:20:45.502 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 111, 2024-04-25T10:20:45.503 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.734479533 2024-04-25T10:20:45.503 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:20:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff954066f60 msgr2=0x7ff954069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff954066f60 0x7ff954069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff9780703c0 tx=0x7ff968008040 comp rx=0 tx=0).stop 2024-04-25T10:20:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97815bc40 msgr2=0x7ff978071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97815bc40 0x7ff978071e10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c780 tx=0x7ff97000cc40 comp rx=0 tx=0).stop 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 shutdown_connections 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff954066f60 0x7ff954069420 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97815bc40 0x7ff978071e10 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9780cf930 0x7ff978150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 --2- 172.21.15.87:0/563101548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9780cda10 0x7ff978070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 >> 172.21.15.87:0/563101548 conn(0x7ff978066b70 msgr2=0x7ff978066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 shutdown_connections 2024-04-25T10:20:45.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.495+0000 7ff97e200640 1 -- 172.21.15.87:0/563101548 wait complete. 2024-04-25T10:20:45.516 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:20:45.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.647+0000 7f027aa00640 1 Processor -- start 2024-04-25T10:20:45.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.647+0000 7f027aa00640 1 -- start start 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f027415a780 0x7f027415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f02740cf3b0 0x7f02740cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f0274071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027aa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027406de60 con 0x7f027415a780 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027aa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027415f2f0 con 0x7f02740cf3b0 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02740d01c0 con 0x7f02740cfd30 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f027415a780 0x7f027415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f027415a780 0x7f027415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56128/0 (socket says 172.21.15.87:56128) 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 -- 172.21.15.87:0/2157303051 learned_addr learned my addr 172.21.15.87:0/2157303051 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027a000640 1 --2- 172.21.15.87:0/2157303051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f0274071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0278c00640 1 --2- 172.21.15.87:0/2157303051 >> v2:172.21.15.169:3300/0 conn(0x7f02740cf3b0 0x7f02740cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 -- 172.21.15.87:0/2157303051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 msgr2=0x7f0274071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 --2- 172.21.15.87:0/2157303051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f0274071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 -- 172.21.15.87:0/2157303051 >> v2:172.21.15.169:3300/0 conn(0x7f02740cf3b0 msgr2=0x7f02740cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 --2- 172.21.15.87:0/2157303051 >> v2:172.21.15.169:3300/0 conn(0x7f02740cf3b0 0x7f02740cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 -- 172.21.15.87:0/2157303051 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027406cdc0 con 0x7f027415a780 2024-04-25T10:20:45.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0278c00640 1 --2- 172.21.15.87:0/2157303051 >> v2:172.21.15.169:3300/0 conn(0x7f02740cf3b0 0x7f02740cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:45.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f027a000640 1 --2- 172.21.15.87:0/2157303051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f0274071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:45.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.651+0000 7f0279600640 1 --2- 172.21.15.87:0/2157303051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f027415a780 0x7f027415edb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f02740677f0 tx=0x7f026801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87812b734494bec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0273e00640 1 -- 172.21.15.87:0/2157303051 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680b0070 con 0x7f027415a780 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0273e00640 1 -- 172.21.15.87:0/2157303051 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02680ab600 con 0x7f027415a780 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0273e00640 1 -- 172.21.15.87:0/2157303051 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680ab9d0 con 0x7f027415a780 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- 172.21.15.87:0/2157303051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f027415a780 msgr2=0x7f027415edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 --2- 172.21.15.87:0/2157303051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f027415a780 0x7f027415edb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f02740677f0 tx=0x7f026801c520 comp rx=0 tx=0).stop 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- 172.21.15.87:0/2157303051 shutdown_connections 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 --2- 172.21.15.87:0/2157303051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f0274071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 --2- 172.21.15.87:0/2157303051 >> v2:172.21.15.169:3300/0 conn(0x7f02740cf3b0 0x7f02740cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 --2- 172.21.15.87:0/2157303051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f027415a780 0x7f027415edb0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- 172.21.15.87:0/2157303051 >> 172.21.15.87:0/2157303051 conn(0x7f02740743f0 msgr2=0x7f02740669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- 172.21.15.87:0/2157303051 shutdown_connections 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- 172.21.15.87:0/2157303051 wait complete. 2024-04-25T10:20:45.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 Processor -- start 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- start start 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02740cf3b0 0x7f027415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f027413fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f027415a780 0x7f02740ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740b2460 con 0x7f02740cf3b0 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027415ab60 con 0x7f027415a780 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02740b25e0 con 0x7f02740cfd30 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0279600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02740cf3b0 0x7f027415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0279600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02740cf3b0 0x7f027415ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56142/0 (socket says 172.21.15.87:56142) 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0279600640 1 -- 172.21.15.87:0/1265089856 learned_addr learned my addr 172.21.15.87:0/1265089856 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0278c00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f027413fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f027a000640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f027415a780 0x7f02740ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0279600640 1 -- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 msgr2=0x7f027413fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0279600640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f027413fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0279600640 1 -- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f027415a780 msgr2=0x7f02740ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.655+0000 7f0279600640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f027415a780 0x7f02740ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0279600640 1 -- 172.21.15.87:0/1265089856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740cd1e0 con 0x7f02740cf3b0 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0279600640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02740cf3b0 0x7f027415ace0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f026801abd0 tx=0x7f026801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0271600640 1 -- 172.21.15.87:0/1265089856 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680b0040 con 0x7f02740cf3b0 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0271600640 1 -- 172.21.15.87:0/1265089856 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02680bc070 con 0x7f02740cf3b0 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0271600640 1 -- 172.21.15.87:0/1265089856 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680b6880 con 0x7f02740cf3b0 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02740cd4c0 con 0x7f02740cf3b0 2024-04-25T10:20:45.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02740cda80 con 0x7f02740cf3b0 2024-04-25T10:20:45.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274075120 con 0x7f02740cf3b0 2024-04-25T10:20:45.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0271600640 1 -- 172.21.15.87:0/1265089856 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02680ab4c0 con 0x7f02740cf3b0 2024-04-25T10:20:45.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0271600640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0254066f60 0x7f0254069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.659+0000 7f0271600640 1 -- 172.21.15.87:0/1265089856 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f026811f3f0 con 0x7f02740cf3b0 2024-04-25T10:20:45.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.663+0000 7f0278c00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0254066f60 0x7f0254069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.663+0000 7f0278c00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0254066f60 0x7f0254069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0274140700 tx=0x7f0260008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.663+0000 7f0271600640 1 -- 172.21.15.87:0/1265089856 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0268016110 con 0x7f02740cf3b0 2024-04-25T10:20:45.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.851+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f027406c8d0 con 0x7f02740cf3b0 2024-04-25T10:20:45.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.851+0000 7f0271600640 1 -- 172.21.15.87:0/1265089856 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7f02680f0360 con 0x7f02740cf3b0 2024-04-25T10:20:45.855 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:20:45.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0254066f60 msgr2=0x7f0254069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0254066f60 0x7f0254069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0274140700 tx=0x7f0260008040 comp rx=0 tx=0).stop 2024-04-25T10:20:45.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02740cf3b0 msgr2=0x7f027415ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02740cf3b0 0x7f027415ace0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f026801abd0 tx=0x7f026801ad60 comp rx=0 tx=0).stop 2024-04-25T10:20:45.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 shutdown_connections 2024-04-25T10:20:45.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0254066f60 0x7f0254069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f027415a780 0x7f02740ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cfd30 0x7f027413fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 --2- 172.21.15.87:0/1265089856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02740cf3b0 0x7f027415ace0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 >> 172.21.15.87:0/1265089856 conn(0x7f02740743f0 msgr2=0x7f02740ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 shutdown_connections 2024-04-25T10:20:45.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:20:45.855+0000 7f027aa00640 1 -- 172.21.15.87:0/1265089856 wait complete. 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:20:45.876 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:45.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.877 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:20:45.885 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:20:45.886 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41968sec preferred_lft 41968sec 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.887 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:20:45.888 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.889 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:20:45.953 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:20:45.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.956 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:20:46.011 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:20:46.011 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:20:46.011 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:20:46.011 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:20:46.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.012 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:20:46.012 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:46.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:20:46.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:20:46.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:20:46.013 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:46.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.108 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:20:46.116 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:20:46.116 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:20:46.116 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:46.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.117 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:20:46.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:46.179 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:46.227 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:46.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:46.228 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:46.283 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:20:46.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.283 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.370 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.371 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:46.371 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:20:46.385 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.385 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:46.386 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:20:46.386 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:20:46.386 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:20:46.393 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.394 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:46.394 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:20:46.394 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:20:46.394 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:20:46.394 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:20:46.394 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:20:46.394 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:20:46.397 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:20:46.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.398 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:46.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.417 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:20:46.481 INFO:teuthology.orchestra.run.smithi087.stdout:client8676 2024-04-25T10:20:46.481 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:20:46.481 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:46.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.532 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:46.532 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:46.532 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:20:46.532 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:46.532 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:46.532 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:46.532 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:46.620 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.621 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244683 s, 42.9 MB/s 2024-04-25T10:20:46.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.641 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188085 s, 55.8 MB/s 2024-04-25T10:20:46.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.668 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254121 s, 41.3 MB/s 2024-04-25T10:20:46.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.690 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019393 s, 54.1 MB/s 2024-04-25T10:20:46.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.709 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173338 s, 60.5 MB/s 2024-04-25T10:20:46.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.735 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242034 s, 43.3 MB/s 2024-04-25T10:20:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181342 s, 57.8 MB/s 2024-04-25T10:20:46.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.777 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202539 s, 51.8 MB/s 2024-04-25T10:20:46.797 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.797 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.797 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184763 s, 56.8 MB/s 2024-04-25T10:20:46.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.814 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144758 s, 72.4 MB/s 2024-04-25T10:20:46.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229268 s, 45.7 MB/s 2024-04-25T10:20:46.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126821 s, 82.7 MB/s 2024-04-25T10:20:46.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.872 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174486 s, 60.1 MB/s 2024-04-25T10:20:46.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.894 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197183 s, 53.2 MB/s 2024-04-25T10:20:46.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.915 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194968 s, 53.8 MB/s 2024-04-25T10:20:46.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164412 s, 63.8 MB/s 2024-04-25T10:20:46.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.951 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161501 s, 64.9 MB/s 2024-04-25T10:20:46.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202402 s, 51.8 MB/s 2024-04-25T10:20:46.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:46.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:46.991 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157731 s, 66.5 MB/s 2024-04-25T10:20:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183228 s, 57.2 MB/s 2024-04-25T10:20:47.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.031 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182427 s, 57.5 MB/s 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208811 s, 50.2 MB/s 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152469 s, 68.8 MB/s 2024-04-25T10:20:47.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.088 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151262 s, 69.3 MB/s 2024-04-25T10:20:47.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186889 s, 56.1 MB/s 2024-04-25T10:20:47.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02061 s, 50.9 MB/s 2024-04-25T10:20:47.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.150 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172909 s, 60.6 MB/s 2024-04-25T10:20:47.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.172 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204651 s, 51.2 MB/s 2024-04-25T10:20:47.194 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.195 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205427 s, 51.0 MB/s 2024-04-25T10:20:47.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.211 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142303 s, 73.7 MB/s 2024-04-25T10:20:47.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138984 s, 75.4 MB/s 2024-04-25T10:20:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152322 s, 68.8 MB/s 2024-04-25T10:20:47.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:47.245 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:47.245 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:47.245 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:20:47.245 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:47.246 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:47.246 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:47.246 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.313 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123758 s, 84.7 MB/s 2024-04-25T10:20:47.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.330 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153317 s, 68.4 MB/s 2024-04-25T10:20:47.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.355 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224995 s, 46.6 MB/s 2024-04-25T10:20:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017569 s, 59.7 MB/s 2024-04-25T10:20:47.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.392 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166244 s, 63.1 MB/s 2024-04-25T10:20:47.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.414 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199588 s, 52.5 MB/s 2024-04-25T10:20:47.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.438 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227615 s, 46.1 MB/s 2024-04-25T10:20:47.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.456 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164289 s, 63.8 MB/s 2024-04-25T10:20:47.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016033 s, 65.4 MB/s 2024-04-25T10:20:47.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150513 s, 69.7 MB/s 2024-04-25T10:20:47.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151186 s, 69.4 MB/s 2024-04-25T10:20:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192824 s, 54.4 MB/s 2024-04-25T10:20:47.549 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.549 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.549 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184839 s, 56.7 MB/s 2024-04-25T10:20:47.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186672 s, 56.2 MB/s 2024-04-25T10:20:47.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.589 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017241 s, 60.8 MB/s 2024-04-25T10:20:47.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.611 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200255 s, 52.4 MB/s 2024-04-25T10:20:47.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.633 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204109 s, 51.4 MB/s 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015584 s, 67.3 MB/s 2024-04-25T10:20:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.675 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226642 s, 46.3 MB/s 2024-04-25T10:20:47.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148355 s, 70.7 MB/s 2024-04-25T10:20:47.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020288 s, 51.7 MB/s 2024-04-25T10:20:47.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.732 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165697 s, 63.3 MB/s 2024-04-25T10:20:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180668 s, 58.0 MB/s 2024-04-25T10:20:47.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.769 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155564 s, 67.4 MB/s 2024-04-25T10:20:47.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.789 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174737 s, 60.0 MB/s 2024-04-25T10:20:47.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148035 s, 70.8 MB/s 2024-04-25T10:20:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153898 s, 68.1 MB/s 2024-04-25T10:20:47.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204955 s, 51.2 MB/s 2024-04-25T10:20:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178163 s, 58.9 MB/s 2024-04-25T10:20:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151496 s, 69.2 MB/s 2024-04-25T10:20:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176323 s, 59.5 MB/s 2024-04-25T10:20:47.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.919 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166005 s, 63.2 MB/s 2024-04-25T10:20:47.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:47.920 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:47.920 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:47.920 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:20:47.920 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:47.920 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:47.920 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:47.920 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:47.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:47.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:47.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156016 s, 67.2 MB/s 2024-04-25T10:20:48.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.018 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195499 s, 53.6 MB/s 2024-04-25T10:20:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174562 s, 60.1 MB/s 2024-04-25T10:20:48.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.057 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180338 s, 58.1 MB/s 2024-04-25T10:20:48.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.073 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140955 s, 74.4 MB/s 2024-04-25T10:20:48.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.089 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142441 s, 73.6 MB/s 2024-04-25T10:20:48.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.106 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148755 s, 70.5 MB/s 2024-04-25T10:20:48.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.120 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122376 s, 85.7 MB/s 2024-04-25T10:20:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.137 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150097 s, 69.9 MB/s 2024-04-25T10:20:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.157 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184793 s, 56.7 MB/s 2024-04-25T10:20:48.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.175 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169209 s, 62.0 MB/s 2024-04-25T10:20:48.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154176 s, 68.0 MB/s 2024-04-25T10:20:48.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196084 s, 53.5 MB/s 2024-04-25T10:20:48.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.233 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167157 s, 62.7 MB/s 2024-04-25T10:20:48.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.255 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201722 s, 52.0 MB/s 2024-04-25T10:20:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145374 s, 72.1 MB/s 2024-04-25T10:20:48.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.285 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125162 s, 83.8 MB/s 2024-04-25T10:20:48.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.305 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177563 s, 59.1 MB/s 2024-04-25T10:20:48.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.323 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165523 s, 63.3 MB/s 2024-04-25T10:20:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.342 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167946 s, 62.4 MB/s 2024-04-25T10:20:48.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166144 s, 63.1 MB/s 2024-04-25T10:20:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158849 s, 66.0 MB/s 2024-04-25T10:20:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236335 s, 44.4 MB/s 2024-04-25T10:20:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180547 s, 58.1 MB/s 2024-04-25T10:20:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148313 s, 70.7 MB/s 2024-04-25T10:20:48.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.456 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015083 s, 69.5 MB/s 2024-04-25T10:20:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014286 s, 73.4 MB/s 2024-04-25T10:20:48.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.495 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206715 s, 50.7 MB/s 2024-04-25T10:20:48.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.512 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145286 s, 72.2 MB/s 2024-04-25T10:20:48.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165505 s, 63.4 MB/s 2024-04-25T10:20:48.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146231 s, 71.7 MB/s 2024-04-25T10:20:48.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022418 s, 46.8 MB/s 2024-04-25T10:20:48.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:48.572 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:48.572 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:48.572 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:20:48.572 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:48.572 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:48.572 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:48.572 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.646 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168305 s, 62.3 MB/s 2024-04-25T10:20:48.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175419 s, 59.8 MB/s 2024-04-25T10:20:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115492 s, 90.8 MB/s 2024-04-25T10:20:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.694 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145815 s, 71.9 MB/s 2024-04-25T10:20:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014621 s, 71.7 MB/s 2024-04-25T10:20:48.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.727 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150269 s, 69.8 MB/s 2024-04-25T10:20:48.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.744 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144116 s, 72.8 MB/s 2024-04-25T10:20:48.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.763 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171268 s, 61.2 MB/s 2024-04-25T10:20:48.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.779 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139729 s, 75.0 MB/s 2024-04-25T10:20:48.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.794 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139017 s, 75.4 MB/s 2024-04-25T10:20:48.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.813 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168217 s, 62.3 MB/s 2024-04-25T10:20:48.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.831 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163676 s, 64.1 MB/s 2024-04-25T10:20:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209423 s, 50.1 MB/s 2024-04-25T10:20:48.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152941 s, 68.6 MB/s 2024-04-25T10:20:48.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01443 s, 72.7 MB/s 2024-04-25T10:20:48.908 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.909 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198549 s, 52.8 MB/s 2024-04-25T10:20:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183043 s, 57.3 MB/s 2024-04-25T10:20:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197675 s, 53.0 MB/s 2024-04-25T10:20:48.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.967 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150259 s, 69.8 MB/s 2024-04-25T10:20:48.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:48.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:48.987 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180014 s, 58.2 MB/s 2024-04-25T10:20:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194683 s, 53.9 MB/s 2024-04-25T10:20:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.028 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.028 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177535 s, 59.1 MB/s 2024-04-25T10:20:49.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.044 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150147 s, 69.8 MB/s 2024-04-25T10:20:49.061 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154123 s, 68.0 MB/s 2024-04-25T10:20:49.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.081 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173135 s, 60.6 MB/s 2024-04-25T10:20:49.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.100 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170495 s, 61.5 MB/s 2024-04-25T10:20:49.117 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.117 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.117 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158994 s, 66.0 MB/s 2024-04-25T10:20:49.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.136 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016812 s, 62.4 MB/s 2024-04-25T10:20:49.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198747 s, 52.8 MB/s 2024-04-25T10:20:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197982 s, 53.0 MB/s 2024-04-25T10:20:49.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.196 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152142 s, 68.9 MB/s 2024-04-25T10:20:49.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.218 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201311 s, 52.1 MB/s 2024-04-25T10:20:49.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:49.219 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:49.219 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:49.219 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:20:49.220 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:49.220 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:49.220 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:49.220 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204061 s, 51.4 MB/s 2024-04-25T10:20:49.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.315 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160866 s, 65.2 MB/s 2024-04-25T10:20:49.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.331 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142258 s, 73.7 MB/s 2024-04-25T10:20:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017112 s, 61.3 MB/s 2024-04-25T10:20:49.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.364 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127337 s, 82.3 MB/s 2024-04-25T10:20:49.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.381 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147603 s, 71.0 MB/s 2024-04-25T10:20:49.396 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.396 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.396 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140048 s, 74.9 MB/s 2024-04-25T10:20:49.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.414 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154638 s, 67.8 MB/s 2024-04-25T10:20:49.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177465 s, 59.1 MB/s 2024-04-25T10:20:49.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172226 s, 60.9 MB/s 2024-04-25T10:20:49.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167368 s, 62.7 MB/s 2024-04-25T10:20:49.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.487 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144383 s, 72.6 MB/s 2024-04-25T10:20:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157526 s, 66.6 MB/s 2024-04-25T10:20:49.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201391 s, 52.1 MB/s 2024-04-25T10:20:49.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150162 s, 69.8 MB/s 2024-04-25T10:20:49.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151065 s, 69.4 MB/s 2024-04-25T10:20:49.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142954 s, 73.4 MB/s 2024-04-25T10:20:49.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.595 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175721 s, 59.7 MB/s 2024-04-25T10:20:49.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165156 s, 63.5 MB/s 2024-04-25T10:20:49.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015524 s, 67.5 MB/s 2024-04-25T10:20:49.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.649 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016079 s, 65.2 MB/s 2024-04-25T10:20:49.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.671 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200971 s, 52.2 MB/s 2024-04-25T10:20:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143462 s, 73.1 MB/s 2024-04-25T10:20:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146859 s, 71.4 MB/s 2024-04-25T10:20:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204162 s, 51.4 MB/s 2024-04-25T10:20:49.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.746 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188258 s, 55.7 MB/s 2024-04-25T10:20:49.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.765 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168875 s, 62.1 MB/s 2024-04-25T10:20:49.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.793 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02583 s, 40.6 MB/s 2024-04-25T10:20:49.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222904 s, 47.0 MB/s 2024-04-25T10:20:49.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149898 s, 70.0 MB/s 2024-04-25T10:20:49.850 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.850 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.850 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014259 s, 73.5 MB/s 2024-04-25T10:20:49.867 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.867 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.867 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154851 s, 67.7 MB/s 2024-04-25T10:20:49.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:49.869 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:49.869 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:49.869 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:20:49.869 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:49.869 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:49.869 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:49.869 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.946 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171036 s, 61.3 MB/s 2024-04-25T10:20:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.967 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189959 s, 55.2 MB/s 2024-04-25T10:20:49.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172173 s, 60.9 MB/s 2024-04-25T10:20:50.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.006 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179744 s, 58.3 MB/s 2024-04-25T10:20:50.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.023 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154446 s, 67.9 MB/s 2024-04-25T10:20:50.040 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.040 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.040 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015181 s, 69.1 MB/s 2024-04-25T10:20:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148174 s, 70.8 MB/s 2024-04-25T10:20:50.075 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.076 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.076 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174976 s, 59.9 MB/s 2024-04-25T10:20:50.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.098 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209174 s, 50.1 MB/s 2024-04-25T10:20:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.114 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139584 s, 75.1 MB/s 2024-04-25T10:20:50.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.136 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202495 s, 51.8 MB/s 2024-04-25T10:20:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206984 s, 50.7 MB/s 2024-04-25T10:20:50.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.175 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148293 s, 70.7 MB/s 2024-04-25T10:20:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.192 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145885 s, 71.9 MB/s 2024-04-25T10:20:50.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146597 s, 71.5 MB/s 2024-04-25T10:20:50.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154363 s, 67.9 MB/s 2024-04-25T10:20:50.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.246 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185551 s, 56.5 MB/s 2024-04-25T10:20:50.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.265 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174058 s, 60.2 MB/s 2024-04-25T10:20:50.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.287 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199889 s, 52.5 MB/s 2024-04-25T10:20:50.306 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.306 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179464 s, 58.4 MB/s 2024-04-25T10:20:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143944 s, 72.8 MB/s 2024-04-25T10:20:50.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.339 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142143 s, 73.8 MB/s 2024-04-25T10:20:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209095 s, 50.1 MB/s 2024-04-25T10:20:50.382 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.382 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.382 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018756 s, 55.9 MB/s 2024-04-25T10:20:50.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.405 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208929 s, 50.2 MB/s 2024-04-25T10:20:50.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.422 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148836 s, 70.5 MB/s 2024-04-25T10:20:50.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.442 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185495 s, 56.5 MB/s 2024-04-25T10:20:50.462 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.463 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171187 s, 61.3 MB/s 2024-04-25T10:20:50.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.481 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183693 s, 57.1 MB/s 2024-04-25T10:20:50.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.497 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146872 s, 71.4 MB/s 2024-04-25T10:20:50.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.515 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143527 s, 73.1 MB/s 2024-04-25T10:20:50.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.538 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167259 s, 62.7 MB/s 2024-04-25T10:20:50.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:50.539 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:50.539 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:50.539 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:20:50.539 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:50.539 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:50.539 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:50.539 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:50.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.619 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144726 s, 72.5 MB/s 2024-04-25T10:20:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195106 s, 53.7 MB/s 2024-04-25T10:20:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166416 s, 63.0 MB/s 2024-04-25T10:20:50.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.691 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200318 s, 52.3 MB/s 2024-04-25T10:20:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.698 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164076 s, 63.9 MB/s 2024-04-25T10:20:50.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141823 s, 73.9 MB/s 2024-04-25T10:20:50.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.731 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013825 s, 75.8 MB/s 2024-04-25T10:20:50.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.750 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017264 s, 60.7 MB/s 2024-04-25T10:20:50.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.774 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022195 s, 47.2 MB/s 2024-04-25T10:20:50.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.801 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233042 s, 45.0 MB/s 2024-04-25T10:20:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174663 s, 60.0 MB/s 2024-04-25T10:20:50.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184607 s, 56.8 MB/s 2024-04-25T10:20:50.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.856 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142822 s, 73.4 MB/s 2024-04-25T10:20:50.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.870 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154321 s, 67.9 MB/s 2024-04-25T10:20:50.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.889 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167865 s, 62.5 MB/s 2024-04-25T10:20:50.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.905 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145657 s, 72.0 MB/s 2024-04-25T10:20:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.929 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218238 s, 48.0 MB/s 2024-04-25T10:20:50.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.949 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179655 s, 58.4 MB/s 2024-04-25T10:20:50.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.967 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161625 s, 64.9 MB/s 2024-04-25T10:20:50.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:50.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:50.988 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194379 s, 53.9 MB/s 2024-04-25T10:20:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205607 s, 51.0 MB/s 2024-04-25T10:20:51.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.029 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173665 s, 60.4 MB/s 2024-04-25T10:20:51.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.048 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168843 s, 62.1 MB/s 2024-04-25T10:20:51.066 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.066 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.067 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017175 s, 61.1 MB/s 2024-04-25T10:20:51.083 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.083 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.084 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152565 s, 68.7 MB/s 2024-04-25T10:20:51.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.100 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150187 s, 69.8 MB/s 2024-04-25T10:20:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182913 s, 57.3 MB/s 2024-04-25T10:20:51.140 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.141 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184522 s, 56.8 MB/s 2024-04-25T10:20:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147343 s, 71.2 MB/s 2024-04-25T10:20:51.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.174 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152244 s, 68.9 MB/s 2024-04-25T10:20:51.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.191 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148932 s, 70.4 MB/s 2024-04-25T10:20:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165469 s, 63.4 MB/s 2024-04-25T10:20:51.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:51.211 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:51.211 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:51.211 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:20:51.211 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:51.211 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:51.211 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:51.211 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:51.289 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.290 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198068 s, 52.9 MB/s 2024-04-25T10:20:51.306 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.306 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.306 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146415 s, 71.6 MB/s 2024-04-25T10:20:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014756 s, 71.1 MB/s 2024-04-25T10:20:51.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.338 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136563 s, 76.8 MB/s 2024-04-25T10:20:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193766 s, 54.1 MB/s 2024-04-25T10:20:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017573 s, 59.7 MB/s 2024-04-25T10:20:51.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.395 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150229 s, 69.8 MB/s 2024-04-25T10:20:51.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.412 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153793 s, 68.2 MB/s 2024-04-25T10:20:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166263 s, 63.1 MB/s 2024-04-25T10:20:51.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.444 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119862 s, 87.5 MB/s 2024-04-25T10:20:51.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.462 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155549 s, 67.4 MB/s 2024-04-25T10:20:51.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.480 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166068 s, 63.1 MB/s 2024-04-25T10:20:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180609 s, 58.1 MB/s 2024-04-25T10:20:51.518 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.518 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.518 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170262 s, 61.6 MB/s 2024-04-25T10:20:51.534 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.534 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.534 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141876 s, 73.9 MB/s 2024-04-25T10:20:51.551 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.551 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.551 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150119 s, 69.8 MB/s 2024-04-25T10:20:51.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.571 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017513 s, 59.9 MB/s 2024-04-25T10:20:51.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143341 s, 73.2 MB/s 2024-04-25T10:20:51.608 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.609 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201298 s, 52.1 MB/s 2024-04-25T10:20:51.625 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.625 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.625 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148842 s, 70.4 MB/s 2024-04-25T10:20:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144729 s, 72.5 MB/s 2024-04-25T10:20:51.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.658 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146606 s, 71.5 MB/s 2024-04-25T10:20:51.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.675 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148195 s, 70.8 MB/s 2024-04-25T10:20:51.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.691 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149255 s, 70.3 MB/s 2024-04-25T10:20:51.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169297 s, 61.9 MB/s 2024-04-25T10:20:51.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145325 s, 72.2 MB/s 2024-04-25T10:20:51.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.745 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170382 s, 61.5 MB/s 2024-04-25T10:20:51.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.762 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148588 s, 70.6 MB/s 2024-04-25T10:20:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153651 s, 68.2 MB/s 2024-04-25T10:20:51.793 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.793 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.793 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119454 s, 87.8 MB/s 2024-04-25T10:20:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150099 s, 69.9 MB/s 2024-04-25T10:20:51.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166712 s, 62.9 MB/s 2024-04-25T10:20:51.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:51.829 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:51.829 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:51.829 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:20:51.830 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:51.830 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:51.830 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:51.830 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:51.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.905 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166008 s, 63.2 MB/s 2024-04-25T10:20:51.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.923 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166933 s, 62.8 MB/s 2024-04-25T10:20:51.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.940 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150864 s, 69.5 MB/s 2024-04-25T10:20:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.964 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211247 s, 49.6 MB/s 2024-04-25T10:20:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:51.985 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199461 s, 52.6 MB/s 2024-04-25T10:20:52.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.002 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152619 s, 68.7 MB/s 2024-04-25T10:20:52.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.022 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180619 s, 58.1 MB/s 2024-04-25T10:20:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182376 s, 57.5 MB/s 2024-04-25T10:20:52.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187211 s, 56.0 MB/s 2024-04-25T10:20:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144171 s, 72.7 MB/s 2024-04-25T10:20:52.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.095 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01497 s, 70.0 MB/s 2024-04-25T10:20:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01509 s, 69.5 MB/s 2024-04-25T10:20:52.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.129 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154319 s, 67.9 MB/s 2024-04-25T10:20:52.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148919 s, 70.4 MB/s 2024-04-25T10:20:52.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149481 s, 70.1 MB/s 2024-04-25T10:20:52.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150894 s, 69.5 MB/s 2024-04-25T10:20:52.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.196 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150977 s, 69.5 MB/s 2024-04-25T10:20:52.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160664 s, 65.3 MB/s 2024-04-25T10:20:52.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.237 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206416 s, 50.8 MB/s 2024-04-25T10:20:52.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.257 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189283 s, 55.4 MB/s 2024-04-25T10:20:52.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.277 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176357 s, 59.5 MB/s 2024-04-25T10:20:52.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149413 s, 70.2 MB/s 2024-04-25T10:20:52.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152365 s, 68.8 MB/s 2024-04-25T10:20:52.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.330 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174717 s, 60.0 MB/s 2024-04-25T10:20:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165454 s, 63.4 MB/s 2024-04-25T10:20:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229161 s, 45.8 MB/s 2024-04-25T10:20:52.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.394 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201019 s, 52.2 MB/s 2024-04-25T10:20:52.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.411 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146123 s, 71.8 MB/s 2024-04-25T10:20:52.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.430 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.430 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171157 s, 61.3 MB/s 2024-04-25T10:20:52.453 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.453 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.453 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220409 s, 47.6 MB/s 2024-04-25T10:20:52.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.470 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015341 s, 68.4 MB/s 2024-04-25T10:20:52.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142202 s, 73.7 MB/s 2024-04-25T10:20:52.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:52.488 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:52.488 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:52.488 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:20:52.488 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:52.488 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:52.488 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:52.488 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:52.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014747 s, 71.1 MB/s 2024-04-25T10:20:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149261 s, 70.3 MB/s 2024-04-25T10:20:52.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.593 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147127 s, 71.3 MB/s 2024-04-25T10:20:52.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151992 s, 69.0 MB/s 2024-04-25T10:20:52.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147828 s, 70.9 MB/s 2024-04-25T10:20:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144011 s, 72.8 MB/s 2024-04-25T10:20:52.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174446 s, 60.1 MB/s 2024-04-25T10:20:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.681 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168537 s, 62.2 MB/s 2024-04-25T10:20:52.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.700 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176609 s, 59.4 MB/s 2024-04-25T10:20:52.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.717 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149533 s, 70.1 MB/s 2024-04-25T10:20:52.733 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.733 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.733 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143633 s, 73.0 MB/s 2024-04-25T10:20:52.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.755 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206802 s, 50.7 MB/s 2024-04-25T10:20:52.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.772 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150754 s, 69.6 MB/s 2024-04-25T10:20:52.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.792 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180814 s, 58.0 MB/s 2024-04-25T10:20:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194969 s, 53.8 MB/s 2024-04-25T10:20:52.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.829 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145789 s, 71.9 MB/s 2024-04-25T10:20:52.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014206 s, 73.8 MB/s 2024-04-25T10:20:52.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.864 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017406 s, 60.2 MB/s 2024-04-25T10:20:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144337 s, 72.6 MB/s 2024-04-25T10:20:52.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158042 s, 66.3 MB/s 2024-04-25T10:20:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159283 s, 65.8 MB/s 2024-04-25T10:20:52.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164484 s, 63.7 MB/s 2024-04-25T10:20:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183043 s, 57.3 MB/s 2024-04-25T10:20:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167627 s, 62.6 MB/s 2024-04-25T10:20:52.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:52.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:52.991 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170764 s, 61.4 MB/s 2024-04-25T10:20:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015243 s, 68.8 MB/s 2024-04-25T10:20:53.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.027 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167518 s, 62.6 MB/s 2024-04-25T10:20:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.045 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166271 s, 63.1 MB/s 2024-04-25T10:20:53.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015018 s, 69.8 MB/s 2024-04-25T10:20:53.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.078 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142724 s, 73.5 MB/s 2024-04-25T10:20:53.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.100 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020618 s, 50.9 MB/s 2024-04-25T10:20:53.123 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.123 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.123 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208033 s, 50.4 MB/s 2024-04-25T10:20:53.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:53.124 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:53.124 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:53.124 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:20:53.124 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:53.124 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:53.124 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:53.125 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:53.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.195 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146832 s, 71.4 MB/s 2024-04-25T10:20:53.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.217 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200129 s, 52.4 MB/s 2024-04-25T10:20:53.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.233 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144655 s, 72.5 MB/s 2024-04-25T10:20:53.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.251 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150336 s, 69.7 MB/s 2024-04-25T10:20:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147026 s, 71.3 MB/s 2024-04-25T10:20:53.288 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:53.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:53.292 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.292 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.292 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208932 s, 50.2 MB/s 2024-04-25T10:20:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181811 s, 57.7 MB/s 2024-04-25T10:20:53.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.332 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207021 s, 50.7 MB/s 2024-04-25T10:20:53.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152154 s, 68.9 MB/s 2024-04-25T10:20:53.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.363 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122232 s, 85.8 MB/s 2024-04-25T10:20:53.382 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.382 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.382 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175998 s, 59.6 MB/s 2024-04-25T10:20:53.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.408 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232636 s, 45.1 MB/s 2024-04-25T10:20:53.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023935 s, 43.8 MB/s 2024-04-25T10:20:53.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.449 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146666 s, 71.5 MB/s 2024-04-25T10:20:53.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.469 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179073 s, 58.6 MB/s 2024-04-25T10:20:53.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.487 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016538 s, 63.4 MB/s 2024-04-25T10:20:53.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.509 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201029 s, 52.2 MB/s 2024-04-25T10:20:53.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157661 s, 66.5 MB/s 2024-04-25T10:20:53.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176859 s, 59.3 MB/s 2024-04-25T10:20:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151619 s, 69.2 MB/s 2024-04-25T10:20:53.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.580 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.580 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146772 s, 71.4 MB/s 2024-04-25T10:20:53.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128662 s, 81.5 MB/s 2024-04-25T10:20:53.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.616 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199666 s, 52.5 MB/s 2024-04-25T10:20:53.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.634 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165959 s, 63.2 MB/s 2024-04-25T10:20:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014565 s, 72.0 MB/s 2024-04-25T10:20:53.667 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.667 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.667 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147608 s, 71.0 MB/s 2024-04-25T10:20:53.684 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.684 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.684 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149374 s, 70.2 MB/s 2024-04-25T10:20:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166417 s, 63.0 MB/s 2024-04-25T10:20:53.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.718 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141819 s, 73.9 MB/s 2024-04-25T10:20:53.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.735 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.735 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151542 s, 69.2 MB/s 2024-04-25T10:20:53.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.753 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016314 s, 64.3 MB/s 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151769 s, 69.1 MB/s 2024-04-25T10:20:53.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:53.772 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:53.772 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:53.772 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:20:53.772 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:53.772 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:53.772 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:53.772 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167859 s, 62.5 MB/s 2024-04-25T10:20:53.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148771 s, 70.5 MB/s 2024-04-25T10:20:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200041 s, 52.4 MB/s 2024-04-25T10:20:53.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151359 s, 69.3 MB/s 2024-04-25T10:20:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164949 s, 63.6 MB/s 2024-04-25T10:20:53.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.935 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145273 s, 72.2 MB/s 2024-04-25T10:20:53.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.956 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196521 s, 53.4 MB/s 2024-04-25T10:20:53.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188227 s, 55.7 MB/s 2024-04-25T10:20:53.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:53.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:53.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180054 s, 58.2 MB/s 2024-04-25T10:20:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150835 s, 69.5 MB/s 2024-04-25T10:20:54.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.036 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204729 s, 51.2 MB/s 2024-04-25T10:20:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142029 s, 73.8 MB/s 2024-04-25T10:20:54.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.073 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191775 s, 54.7 MB/s 2024-04-25T10:20:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148467 s, 70.6 MB/s 2024-04-25T10:20:54.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168748 s, 62.1 MB/s 2024-04-25T10:20:54.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.126 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162507 s, 64.5 MB/s 2024-04-25T10:20:54.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.142 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147106 s, 71.3 MB/s 2024-04-25T10:20:54.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.166 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226231 s, 46.3 MB/s 2024-04-25T10:20:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.182 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143781 s, 72.9 MB/s 2024-04-25T10:20:54.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145573 s, 72.0 MB/s 2024-04-25T10:20:54.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.215 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143839 s, 72.9 MB/s 2024-04-25T10:20:54.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.234 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171233 s, 61.2 MB/s 2024-04-25T10:20:54.253 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.254 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178332 s, 58.8 MB/s 2024-04-25T10:20:54.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.270 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144819 s, 72.4 MB/s 2024-04-25T10:20:54.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163794 s, 64.0 MB/s 2024-04-25T10:20:54.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.305 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155814 s, 67.3 MB/s 2024-04-25T10:20:54.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145691 s, 72.0 MB/s 2024-04-25T10:20:54.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.338 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143481 s, 73.1 MB/s 2024-04-25T10:20:54.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.357 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175173 s, 59.9 MB/s 2024-04-25T10:20:54.376 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.376 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.376 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175541 s, 59.7 MB/s 2024-04-25T10:20:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.397 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193429 s, 54.2 MB/s 2024-04-25T10:20:54.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.414 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146202 s, 71.7 MB/s 2024-04-25T10:20:54.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.415 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:54.415 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:54.416 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:20:54.416 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:54.416 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:54.416 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:54.416 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:54.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.495 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224756 s, 46.7 MB/s 2024-04-25T10:20:54.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.515 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01841 s, 57.0 MB/s 2024-04-25T10:20:54.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.538 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212846 s, 49.3 MB/s 2024-04-25T10:20:54.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198588 s, 52.8 MB/s 2024-04-25T10:20:54.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143386 s, 73.1 MB/s 2024-04-25T10:20:54.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.597 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199026 s, 52.7 MB/s 2024-04-25T10:20:54.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.616 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170896 s, 61.4 MB/s 2024-04-25T10:20:54.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.633 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157925 s, 66.4 MB/s 2024-04-25T10:20:54.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200015 s, 52.4 MB/s 2024-04-25T10:20:54.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.671 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.671 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144062 s, 72.8 MB/s 2024-04-25T10:20:54.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.691 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172323 s, 60.8 MB/s 2024-04-25T10:20:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.712 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197806 s, 53.0 MB/s 2024-04-25T10:20:54.728 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.728 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.728 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146067 s, 71.8 MB/s 2024-04-25T10:20:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165255 s, 63.5 MB/s 2024-04-25T10:20:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149873 s, 70.0 MB/s 2024-04-25T10:20:54.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.787 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215213 s, 48.7 MB/s 2024-04-25T10:20:54.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.807 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018145 s, 57.8 MB/s 2024-04-25T10:20:54.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.830 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209623 s, 50.0 MB/s 2024-04-25T10:20:54.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.849 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162274 s, 64.6 MB/s 2024-04-25T10:20:54.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.865 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140453 s, 74.7 MB/s 2024-04-25T10:20:54.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203632 s, 51.5 MB/s 2024-04-25T10:20:54.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.903 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141824 s, 73.9 MB/s 2024-04-25T10:20:54.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.919 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143076 s, 73.3 MB/s 2024-04-25T10:20:54.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.936 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149318 s, 70.2 MB/s 2024-04-25T10:20:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155641 s, 67.4 MB/s 2024-04-25T10:20:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150982 s, 69.5 MB/s 2024-04-25T10:20:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168859 s, 62.1 MB/s 2024-04-25T10:20:55.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172493 s, 60.8 MB/s 2024-04-25T10:20:55.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.023 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133987 s, 78.3 MB/s 2024-04-25T10:20:55.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165331 s, 63.4 MB/s 2024-04-25T10:20:55.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.058 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147092 s, 71.3 MB/s 2024-04-25T10:20:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214558 s, 48.9 MB/s 2024-04-25T10:20:55.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:55.083 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:55.083 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:55.083 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:20:55.083 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:55.083 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:55.083 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:55.083 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:55.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.155 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176663 s, 59.4 MB/s 2024-04-25T10:20:55.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.172 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154342 s, 67.9 MB/s 2024-04-25T10:20:55.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.190 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015478 s, 67.7 MB/s 2024-04-25T10:20:55.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.207 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152806 s, 68.6 MB/s 2024-04-25T10:20:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017141 s, 61.2 MB/s 2024-04-25T10:20:55.242 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.242 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.242 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150517 s, 69.7 MB/s 2024-04-25T10:20:55.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.259 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143686 s, 73.0 MB/s 2024-04-25T10:20:55.275 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.275 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.275 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146982 s, 71.3 MB/s 2024-04-25T10:20:55.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170524 s, 61.5 MB/s 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014479 s, 72.4 MB/s 2024-04-25T10:20:55.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.327 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142528 s, 73.6 MB/s 2024-04-25T10:20:55.345 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.345 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.345 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162486 s, 64.5 MB/s 2024-04-25T10:20:55.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.363 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168354 s, 62.3 MB/s 2024-04-25T10:20:55.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.386 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212578 s, 49.3 MB/s 2024-04-25T10:20:55.402 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.402 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.402 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014531 s, 72.2 MB/s 2024-04-25T10:20:55.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.418 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138141 s, 75.9 MB/s 2024-04-25T10:20:55.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.438 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177939 s, 58.9 MB/s 2024-04-25T10:20:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012219 s, 85.8 MB/s 2024-04-25T10:20:55.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.470 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167908 s, 62.4 MB/s 2024-04-25T10:20:55.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.489 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017026 s, 61.6 MB/s 2024-04-25T10:20:55.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.510 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198393 s, 52.9 MB/s 2024-04-25T10:20:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145907 s, 71.9 MB/s 2024-04-25T10:20:55.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015873 s, 66.1 MB/s 2024-04-25T10:20:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.561 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014479 s, 72.4 MB/s 2024-04-25T10:20:55.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.577 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147831 s, 70.9 MB/s 2024-04-25T10:20:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151893 s, 69.0 MB/s 2024-04-25T10:20:55.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.611 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152677 s, 68.7 MB/s 2024-04-25T10:20:55.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.628 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148564 s, 70.6 MB/s 2024-04-25T10:20:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.647 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174493 s, 60.1 MB/s 2024-04-25T10:20:55.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.668 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018274 s, 57.4 MB/s 2024-04-25T10:20:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148224 s, 70.7 MB/s 2024-04-25T10:20:55.701 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.701 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.701 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149449 s, 70.2 MB/s 2024-04-25T10:20:55.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:55.703 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:55.703 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:55.703 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:20:55.703 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:55.703 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:55.703 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:55.703 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:55.775 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.775 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.775 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144244 s, 72.7 MB/s 2024-04-25T10:20:55.796 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.797 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.797 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200824 s, 52.2 MB/s 2024-04-25T10:20:55.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.813 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143078 s, 73.3 MB/s 2024-04-25T10:20:55.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150255 s, 69.8 MB/s 2024-04-25T10:20:55.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.851 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197139 s, 53.2 MB/s 2024-04-25T10:20:55.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01787 s, 58.7 MB/s 2024-04-25T10:20:55.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014825 s, 70.7 MB/s 2024-04-25T10:20:55.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.904 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153166 s, 68.5 MB/s 2024-04-25T10:20:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.923 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165171 s, 63.5 MB/s 2024-04-25T10:20:55.942 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.942 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.942 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174626 s, 60.0 MB/s 2024-04-25T10:20:55.960 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.960 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.960 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159799 s, 65.6 MB/s 2024-04-25T10:20:55.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.979 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171789 s, 61.0 MB/s 2024-04-25T10:20:55.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:55.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:55.997 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165696 s, 63.3 MB/s 2024-04-25T10:20:56.013 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.013 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.013 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141556 s, 74.1 MB/s 2024-04-25T10:20:56.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.027 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123542 s, 84.9 MB/s 2024-04-25T10:20:56.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.046 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168025 s, 62.4 MB/s 2024-04-25T10:20:56.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.063 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152829 s, 68.6 MB/s 2024-04-25T10:20:56.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.085 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205423 s, 51.0 MB/s 2024-04-25T10:20:56.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.101 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014558 s, 72.0 MB/s 2024-04-25T10:20:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173072 s, 60.6 MB/s 2024-04-25T10:20:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.137 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150067 s, 69.9 MB/s 2024-04-25T10:20:56.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.159 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200057 s, 52.4 MB/s 2024-04-25T10:20:56.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.179 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176697 s, 59.3 MB/s 2024-04-25T10:20:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176623 s, 59.4 MB/s 2024-04-25T10:20:56.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.216 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.216 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157095 s, 66.7 MB/s 2024-04-25T10:20:56.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.233 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147186 s, 71.2 MB/s 2024-04-25T10:20:56.249 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.249 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.249 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152082 s, 68.9 MB/s 2024-04-25T10:20:56.272 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.272 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.272 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213694 s, 49.1 MB/s 2024-04-25T10:20:56.289 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.289 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.289 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145205 s, 72.2 MB/s 2024-04-25T10:20:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166777 s, 62.9 MB/s 2024-04-25T10:20:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.326 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170458 s, 61.5 MB/s 2024-04-25T10:20:56.346 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.346 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.346 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180528 s, 58.1 MB/s 2024-04-25T10:20:56.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:56.347 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:56.347 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:56.347 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:20:56.348 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:56.348 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:56.348 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:56.348 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:56.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192366 s, 54.5 MB/s 2024-04-25T10:20:56.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.439 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141723 s, 74.0 MB/s 2024-04-25T10:20:56.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.459 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175006 s, 59.9 MB/s 2024-04-25T10:20:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206263 s, 50.8 MB/s 2024-04-25T10:20:56.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.502 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186716 s, 56.2 MB/s 2024-04-25T10:20:56.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.521 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174648 s, 60.0 MB/s 2024-04-25T10:20:56.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.538 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153041 s, 68.5 MB/s 2024-04-25T10:20:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139175 s, 75.3 MB/s 2024-04-25T10:20:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149194 s, 70.3 MB/s 2024-04-25T10:20:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.593 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201944 s, 51.9 MB/s 2024-04-25T10:20:56.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.609 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151263 s, 69.3 MB/s 2024-04-25T10:20:56.632 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.632 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.632 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207878 s, 50.4 MB/s 2024-04-25T10:20:56.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.648 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143128 s, 73.3 MB/s 2024-04-25T10:20:56.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148519 s, 70.6 MB/s 2024-04-25T10:20:56.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.689 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217716 s, 48.2 MB/s 2024-04-25T10:20:56.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147997 s, 70.9 MB/s 2024-04-25T10:20:56.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017769 s, 59.0 MB/s 2024-04-25T10:20:56.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.742 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150547 s, 69.7 MB/s 2024-04-25T10:20:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.758 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141405 s, 74.2 MB/s 2024-04-25T10:20:56.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.774 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01513 s, 69.3 MB/s 2024-04-25T10:20:56.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.790 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014002 s, 74.9 MB/s 2024-04-25T10:20:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153348 s, 68.4 MB/s 2024-04-25T10:20:56.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186197 s, 56.3 MB/s 2024-04-25T10:20:56.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141806 s, 73.9 MB/s 2024-04-25T10:20:56.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.860 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146511 s, 71.6 MB/s 2024-04-25T10:20:56.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.884 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221928 s, 47.2 MB/s 2024-04-25T10:20:56.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.905 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187456 s, 55.9 MB/s 2024-04-25T10:20:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.925 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179237 s, 58.5 MB/s 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192733 s, 54.4 MB/s 2024-04-25T10:20:56.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.963 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150485 s, 69.7 MB/s 2024-04-25T10:20:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211862 s, 49.5 MB/s 2024-04-25T10:20:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150448 s, 69.7 MB/s 2024-04-25T10:20:57.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:57.004 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:57.004 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:57.004 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:20:57.005 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:57.005 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:57.005 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:57.005 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:57.075 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.075 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.075 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148388 s, 70.7 MB/s 2024-04-25T10:20:57.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193323 s, 54.2 MB/s 2024-04-25T10:20:57.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.113 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014752 s, 71.1 MB/s 2024-04-25T10:20:57.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.134 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194879 s, 53.8 MB/s 2024-04-25T10:20:57.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.151 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148437 s, 70.6 MB/s 2024-04-25T10:20:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151312 s, 69.3 MB/s 2024-04-25T10:20:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172422 s, 60.8 MB/s 2024-04-25T10:20:57.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109654 s, 95.6 MB/s 2024-04-25T10:20:57.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.219 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017448 s, 60.1 MB/s 2024-04-25T10:20:57.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149054 s, 70.3 MB/s 2024-04-25T10:20:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145494 s, 72.1 MB/s 2024-04-25T10:20:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226636 s, 46.3 MB/s 2024-04-25T10:20:57.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162274 s, 64.6 MB/s 2024-04-25T10:20:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.315 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187268 s, 56.0 MB/s 2024-04-25T10:20:57.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.337 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197525 s, 53.1 MB/s 2024-04-25T10:20:57.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.353 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149607 s, 70.1 MB/s 2024-04-25T10:20:57.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.370 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148471 s, 70.6 MB/s 2024-04-25T10:20:57.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.386 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146325 s, 71.7 MB/s 2024-04-25T10:20:57.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.409 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211834 s, 49.5 MB/s 2024-04-25T10:20:57.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.428 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169116 s, 62.0 MB/s 2024-04-25T10:20:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139401 s, 75.2 MB/s 2024-04-25T10:20:57.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147265 s, 71.2 MB/s 2024-04-25T10:20:57.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.479 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174457 s, 60.1 MB/s 2024-04-25T10:20:57.500 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.501 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194643 s, 53.9 MB/s 2024-04-25T10:20:57.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.522 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196602 s, 53.3 MB/s 2024-04-25T10:20:57.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.539 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149162 s, 70.3 MB/s 2024-04-25T10:20:57.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.556 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.556 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159404 s, 65.8 MB/s 2024-04-25T10:20:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153482 s, 68.3 MB/s 2024-04-25T10:20:57.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.592 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.592 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01702 s, 61.6 MB/s 2024-04-25T10:20:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147852 s, 70.9 MB/s 2024-04-25T10:20:57.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159492 s, 65.7 MB/s 2024-04-25T10:20:57.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155676 s, 67.4 MB/s 2024-04-25T10:20:57.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:57.645 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:57.645 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:57.645 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:20:57.645 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:57.645 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:57.645 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:57.645 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:57.719 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.720 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192697 s, 54.4 MB/s 2024-04-25T10:20:57.734 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.734 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.734 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012508 s, 83.8 MB/s 2024-04-25T10:20:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.750 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148327 s, 70.7 MB/s 2024-04-25T10:20:57.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.766 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014019 s, 74.8 MB/s 2024-04-25T10:20:57.782 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.783 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148676 s, 70.5 MB/s 2024-04-25T10:20:57.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.799 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145259 s, 72.2 MB/s 2024-04-25T10:20:57.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148746 s, 70.5 MB/s 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204475 s, 51.3 MB/s 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163706 s, 64.1 MB/s 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146027 s, 71.8 MB/s 2024-04-25T10:20:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155356 s, 67.5 MB/s 2024-04-25T10:20:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188758 s, 55.6 MB/s 2024-04-25T10:20:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176949 s, 59.3 MB/s 2024-04-25T10:20:57.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.950 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179499 s, 58.4 MB/s 2024-04-25T10:20:57.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.968 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165034 s, 63.5 MB/s 2024-04-25T10:20:57.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:57.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:57.985 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151872 s, 69.0 MB/s 2024-04-25T10:20:58.001 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.001 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.002 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145938 s, 71.9 MB/s 2024-04-25T10:20:58.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.018 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151793 s, 69.1 MB/s 2024-04-25T10:20:58.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186028 s, 56.4 MB/s 2024-04-25T10:20:58.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013997 s, 74.9 MB/s 2024-04-25T10:20:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237605 s, 44.1 MB/s 2024-04-25T10:20:58.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.097 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147194 s, 71.2 MB/s 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144898 s, 72.4 MB/s 2024-04-25T10:20:58.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.138 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230205 s, 45.5 MB/s 2024-04-25T10:20:58.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.155 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148358 s, 70.7 MB/s 2024-04-25T10:20:58.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.171 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140113 s, 74.8 MB/s 2024-04-25T10:20:58.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.197 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240695 s, 43.6 MB/s 2024-04-25T10:20:58.212 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.213 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145323 s, 72.2 MB/s 2024-04-25T10:20:58.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206793 s, 50.7 MB/s 2024-04-25T10:20:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143728 s, 73.0 MB/s 2024-04-25T10:20:58.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.271 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183284 s, 57.2 MB/s 2024-04-25T10:20:58.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.288 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151858 s, 69.0 MB/s 2024-04-25T10:20:58.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:58.290 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:58.290 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:58.290 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:20:58.290 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:58.290 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:58.290 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:58.290 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239287 s, 43.8 MB/s 2024-04-25T10:20:58.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.390 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188319 s, 55.7 MB/s 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174393 s, 60.1 MB/s 2024-04-25T10:20:58.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.426 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151085 s, 69.4 MB/s 2024-04-25T10:20:58.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147211 s, 71.2 MB/s 2024-04-25T10:20:58.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152182 s, 68.9 MB/s 2024-04-25T10:20:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157022 s, 66.8 MB/s 2024-04-25T10:20:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.492 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012912 s, 81.2 MB/s 2024-04-25T10:20:58.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.514 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200158 s, 52.4 MB/s 2024-04-25T10:20:58.531 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.531 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.531 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155373 s, 67.5 MB/s 2024-04-25T10:20:58.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.550 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169656 s, 61.8 MB/s 2024-04-25T10:20:58.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014184 s, 73.9 MB/s 2024-04-25T10:20:58.584 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.584 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.584 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161302 s, 65.0 MB/s 2024-04-25T10:20:58.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.600 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141919 s, 73.9 MB/s 2024-04-25T10:20:58.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.617 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.617 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148879 s, 70.4 MB/s 2024-04-25T10:20:58.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017004 s, 61.7 MB/s 2024-04-25T10:20:58.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151769 s, 69.1 MB/s 2024-04-25T10:20:58.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.669 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144899 s, 72.4 MB/s 2024-04-25T10:20:58.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168279 s, 62.3 MB/s 2024-04-25T10:20:58.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.707 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181272 s, 57.8 MB/s 2024-04-25T10:20:58.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156217 s, 67.1 MB/s 2024-04-25T10:20:58.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.741 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142673 s, 73.5 MB/s 2024-04-25T10:20:58.759 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.759 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.759 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164888 s, 63.6 MB/s 2024-04-25T10:20:58.776 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.776 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.776 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150968 s, 69.5 MB/s 2024-04-25T10:20:58.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.793 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149761 s, 70.0 MB/s 2024-04-25T10:20:58.809 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.809 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.809 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146454 s, 71.6 MB/s 2024-04-25T10:20:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144513 s, 72.6 MB/s 2024-04-25T10:20:58.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.843 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162776 s, 64.4 MB/s 2024-04-25T10:20:58.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.860 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150258 s, 69.8 MB/s 2024-04-25T10:20:58.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.876 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144906 s, 72.4 MB/s 2024-04-25T10:20:58.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.897 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184076 s, 57.0 MB/s 2024-04-25T10:20:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.916 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179973 s, 58.3 MB/s 2024-04-25T10:20:58.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:58.918 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:58.918 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:58.918 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:20:58.918 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:58.918 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:58.918 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:58.918 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:58.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:58.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:58.992 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150253 s, 69.8 MB/s 2024-04-25T10:20:59.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.009 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151162 s, 69.4 MB/s 2024-04-25T10:20:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.025 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014058 s, 74.6 MB/s 2024-04-25T10:20:59.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146661 s, 71.5 MB/s 2024-04-25T10:20:59.061 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.061 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.061 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177281 s, 59.1 MB/s 2024-04-25T10:20:59.080 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.080 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.080 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172707 s, 60.7 MB/s 2024-04-25T10:20:59.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.104 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223906 s, 46.8 MB/s 2024-04-25T10:20:59.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147331 s, 71.2 MB/s 2024-04-25T10:20:59.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233157 s, 45.0 MB/s 2024-04-25T10:20:59.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.166 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180639 s, 58.0 MB/s 2024-04-25T10:20:59.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.183 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158317 s, 66.2 MB/s 2024-04-25T10:20:59.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214405 s, 48.9 MB/s 2024-04-25T10:20:59.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173752 s, 60.3 MB/s 2024-04-25T10:20:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167752 s, 62.5 MB/s 2024-04-25T10:20:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147604 s, 71.0 MB/s 2024-04-25T10:20:59.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.278 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155363 s, 67.5 MB/s 2024-04-25T10:20:59.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148293 s, 70.7 MB/s 2024-04-25T10:20:59.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139558 s, 75.1 MB/s 2024-04-25T10:20:59.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174192 s, 60.2 MB/s 2024-04-25T10:20:59.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182132 s, 57.6 MB/s 2024-04-25T10:20:59.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.365 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141415 s, 74.1 MB/s 2024-04-25T10:20:59.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.382 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.382 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145561 s, 72.0 MB/s 2024-04-25T10:20:59.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.398 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.398 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149206 s, 70.3 MB/s 2024-04-25T10:20:59.413 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.413 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.413 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131832 s, 79.5 MB/s 2024-04-25T10:20:59.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.435 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020757 s, 50.5 MB/s 2024-04-25T10:20:59.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.449 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113915 s, 92.0 MB/s 2024-04-25T10:20:59.466 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.466 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.466 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155611 s, 67.4 MB/s 2024-04-25T10:20:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193678 s, 54.1 MB/s 2024-04-25T10:20:59.503 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.503 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.503 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144258 s, 72.7 MB/s 2024-04-25T10:20:59.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197986 s, 53.0 MB/s 2024-04-25T10:20:59.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.541 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146046 s, 71.8 MB/s 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185913 s, 56.4 MB/s 2024-04-25T10:20:59.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:59.563 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:20:59.563 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:20:59.563 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:20:59.563 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:20:59.563 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:20:59.563 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:20:59.564 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:20:59.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.633 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167558 s, 62.6 MB/s 2024-04-25T10:20:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.654 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189849 s, 55.2 MB/s 2024-04-25T10:20:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023295 s, 45.0 MB/s 2024-04-25T10:20:59.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.697 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163701 s, 64.1 MB/s 2024-04-25T10:20:59.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.712 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132068 s, 79.4 MB/s 2024-04-25T10:20:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.729 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145973 s, 71.8 MB/s 2024-04-25T10:20:59.751 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.751 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.751 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020521 s, 51.1 MB/s 2024-04-25T10:20:59.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.771 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018442 s, 56.9 MB/s 2024-04-25T10:20:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.795 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211803 s, 49.5 MB/s 2024-04-25T10:20:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160238 s, 65.4 MB/s 2024-04-25T10:20:59.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.831 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161973 s, 64.7 MB/s 2024-04-25T10:20:59.847 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.847 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.847 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014147 s, 74.1 MB/s 2024-04-25T10:20:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212645 s, 49.3 MB/s 2024-04-25T10:20:59.892 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.892 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.892 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199765 s, 52.5 MB/s 2024-04-25T10:20:59.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.911 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178111 s, 58.9 MB/s 2024-04-25T10:20:59.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.929 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159723 s, 65.6 MB/s 2024-04-25T10:20:59.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.948 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171298 s, 61.2 MB/s 2024-04-25T10:20:59.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.968 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180811 s, 58.0 MB/s 2024-04-25T10:20:59.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:20:59.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:20:59.985 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157425 s, 66.6 MB/s 2024-04-25T10:21:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148159 s, 70.8 MB/s 2024-04-25T10:21:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020174 s, 52.0 MB/s 2024-04-25T10:21:00.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153056 s, 68.5 MB/s 2024-04-25T10:21:00.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.058 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148701 s, 70.5 MB/s 2024-04-25T10:21:00.075 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.075 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.075 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151249 s, 69.3 MB/s 2024-04-25T10:21:00.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.092 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150515 s, 69.7 MB/s 2024-04-25T10:21:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157496 s, 66.6 MB/s 2024-04-25T10:21:00.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.129 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178312 s, 58.8 MB/s 2024-04-25T10:21:00.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.148 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168046 s, 62.4 MB/s 2024-04-25T10:21:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141245 s, 74.2 MB/s 2024-04-25T10:21:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.178 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127425 s, 82.3 MB/s 2024-04-25T10:21:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154738 s, 67.8 MB/s 2024-04-25T10:21:00.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170037 s, 61.7 MB/s 2024-04-25T10:21:00.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:00.216 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:00.216 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:00.216 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:21:00.216 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:00.216 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:00.216 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:00.216 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:00.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.287 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142178 s, 73.8 MB/s 2024-04-25T10:21:00.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.303 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147531 s, 71.1 MB/s 2024-04-25T10:21:00.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.326 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.326 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213048 s, 49.2 MB/s 2024-04-25T10:21:00.345 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.345 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.345 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172122 s, 60.9 MB/s 2024-04-25T10:21:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205277 s, 51.1 MB/s 2024-04-25T10:21:00.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.384 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148633 s, 70.5 MB/s 2024-04-25T10:21:00.402 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168632 s, 62.2 MB/s 2024-04-25T10:21:00.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.419 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146787 s, 71.4 MB/s 2024-04-25T10:21:00.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.438 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176471 s, 59.4 MB/s 2024-04-25T10:21:00.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138651 s, 75.6 MB/s 2024-04-25T10:21:00.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.470 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141475 s, 74.1 MB/s 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198332 s, 52.9 MB/s 2024-04-25T10:21:00.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143693 s, 73.0 MB/s 2024-04-25T10:21:00.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169246 s, 62.0 MB/s 2024-04-25T10:21:00.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.552 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239286 s, 43.8 MB/s 2024-04-25T10:21:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016152 s, 64.9 MB/s 2024-04-25T10:21:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149492 s, 70.1 MB/s 2024-04-25T10:21:00.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.609 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207102 s, 50.6 MB/s 2024-04-25T10:21:00.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.627 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163439 s, 64.2 MB/s 2024-04-25T10:21:00.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143387 s, 73.1 MB/s 2024-04-25T10:21:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016576 s, 63.3 MB/s 2024-04-25T10:21:00.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.680 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169556 s, 61.8 MB/s 2024-04-25T10:21:00.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.703 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.703 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208542 s, 50.3 MB/s 2024-04-25T10:21:00.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207521 s, 50.5 MB/s 2024-04-25T10:21:00.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.743 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152666 s, 68.7 MB/s 2024-04-25T10:21:00.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.762 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170374 s, 61.5 MB/s 2024-04-25T10:21:00.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152839 s, 68.6 MB/s 2024-04-25T10:21:00.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.803 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217285 s, 48.3 MB/s 2024-04-25T10:21:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.821 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168734 s, 62.1 MB/s 2024-04-25T10:21:00.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.837 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142714 s, 73.5 MB/s 2024-04-25T10:21:00.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.856 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.856 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017528 s, 59.8 MB/s 2024-04-25T10:21:00.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.875 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164383 s, 63.8 MB/s 2024-04-25T10:21:00.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:00.876 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:00.876 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:00.876 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:21:00.876 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:00.876 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:00.877 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:00.877 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.957 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205706 s, 51.0 MB/s 2024-04-25T10:21:00.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.974 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155641 s, 67.4 MB/s 2024-04-25T10:21:00.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:00.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:00.990 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142378 s, 73.6 MB/s 2024-04-25T10:21:01.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.007 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149951 s, 69.9 MB/s 2024-04-25T10:21:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.027 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181805 s, 57.7 MB/s 2024-04-25T10:21:01.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.047 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176412 s, 59.4 MB/s 2024-04-25T10:21:01.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.070 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212558 s, 49.3 MB/s 2024-04-25T10:21:01.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.091 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194771 s, 53.8 MB/s 2024-04-25T10:21:01.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.114 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210667 s, 49.8 MB/s 2024-04-25T10:21:01.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150362 s, 69.7 MB/s 2024-04-25T10:21:01.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.147 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142182 s, 73.7 MB/s 2024-04-25T10:21:01.171 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.171 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.171 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227896 s, 46.0 MB/s 2024-04-25T10:21:01.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.190 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166496 s, 63.0 MB/s 2024-04-25T10:21:01.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147812 s, 70.9 MB/s 2024-04-25T10:21:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014846 s, 70.6 MB/s 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169533 s, 61.9 MB/s 2024-04-25T10:21:01.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.261 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173262 s, 60.5 MB/s 2024-04-25T10:21:01.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.283 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201045 s, 52.2 MB/s 2024-04-25T10:21:01.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.300 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161025 s, 65.1 MB/s 2024-04-25T10:21:01.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.317 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145671 s, 72.0 MB/s 2024-04-25T10:21:01.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.334 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152365 s, 68.8 MB/s 2024-04-25T10:21:01.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.351 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149282 s, 70.2 MB/s 2024-04-25T10:21:01.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.372 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01974 s, 53.1 MB/s 2024-04-25T10:21:01.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184893 s, 56.7 MB/s 2024-04-25T10:21:01.408 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.408 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.408 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141224 s, 74.2 MB/s 2024-04-25T10:21:01.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014253 s, 73.6 MB/s 2024-04-25T10:21:01.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173577 s, 60.4 MB/s 2024-04-25T10:21:01.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.463 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.463 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179091 s, 58.5 MB/s 2024-04-25T10:21:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.481 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015829 s, 66.2 MB/s 2024-04-25T10:21:01.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.498 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149608 s, 70.1 MB/s 2024-04-25T10:21:01.519 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.519 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.519 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200849 s, 52.2 MB/s 2024-04-25T10:21:01.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.536 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015126 s, 69.3 MB/s 2024-04-25T10:21:01.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:01.538 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:01.538 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:01.538 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:21:01.538 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:01.538 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:01.538 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:01.539 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:01.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.616 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196674 s, 53.3 MB/s 2024-04-25T10:21:01.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.633 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015064 s, 69.6 MB/s 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146614 s, 71.5 MB/s 2024-04-25T10:21:01.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.666 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015046 s, 69.7 MB/s 2024-04-25T10:21:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208892 s, 50.2 MB/s 2024-04-25T10:21:01.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169841 s, 61.7 MB/s 2024-04-25T10:21:01.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.727 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178876 s, 58.6 MB/s 2024-04-25T10:21:01.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.746 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171577 s, 61.1 MB/s 2024-04-25T10:21:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189307 s, 55.4 MB/s 2024-04-25T10:21:01.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.788 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197313 s, 53.1 MB/s 2024-04-25T10:21:01.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.806 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.806 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154473 s, 67.9 MB/s 2024-04-25T10:21:01.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.824 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166697 s, 62.9 MB/s 2024-04-25T10:21:01.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01847 s, 56.8 MB/s 2024-04-25T10:21:01.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.863 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166568 s, 63.0 MB/s 2024-04-25T10:21:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144022 s, 72.8 MB/s 2024-04-25T10:21:01.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.899 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178067 s, 58.9 MB/s 2024-04-25T10:21:01.920 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.920 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.920 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019767 s, 53.0 MB/s 2024-04-25T10:21:01.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.939 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173534 s, 60.4 MB/s 2024-04-25T10:21:01.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.962 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204869 s, 51.2 MB/s 2024-04-25T10:21:01.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.978 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143584 s, 73.0 MB/s 2024-04-25T10:21:01.994 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:01.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:01.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152173 s, 68.9 MB/s 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144353 s, 72.6 MB/s 2024-04-25T10:21:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172904 s, 60.6 MB/s 2024-04-25T10:21:02.051 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.051 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.051 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194852 s, 53.8 MB/s 2024-04-25T10:21:02.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.071 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.071 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176152 s, 59.5 MB/s 2024-04-25T10:21:02.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.092 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193158 s, 54.3 MB/s 2024-04-25T10:21:02.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.111 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.111 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168991 s, 62.0 MB/s 2024-04-25T10:21:02.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.127 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147057 s, 71.3 MB/s 2024-04-25T10:21:02.144 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.144 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.144 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150429 s, 69.7 MB/s 2024-04-25T10:21:02.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167408 s, 62.6 MB/s 2024-04-25T10:21:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179584 s, 58.4 MB/s 2024-04-25T10:21:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183313 s, 57.2 MB/s 2024-04-25T10:21:02.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:02.204 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:02.204 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:02.204 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:21:02.204 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:02.204 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:02.204 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:02.204 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:02.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.279 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145954 s, 71.8 MB/s 2024-04-25T10:21:02.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.297 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165057 s, 63.5 MB/s 2024-04-25T10:21:02.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.316 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165445 s, 63.4 MB/s 2024-04-25T10:21:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145997 s, 71.8 MB/s 2024-04-25T10:21:02.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158164 s, 66.3 MB/s 2024-04-25T10:21:02.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.366 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146255 s, 71.7 MB/s 2024-04-25T10:21:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.383 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015385 s, 68.2 MB/s 2024-04-25T10:21:02.402 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.402 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.402 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173387 s, 60.5 MB/s 2024-04-25T10:21:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019859 s, 52.8 MB/s 2024-04-25T10:21:02.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.441 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149641 s, 70.1 MB/s 2024-04-25T10:21:02.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.457 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151862 s, 69.0 MB/s 2024-04-25T10:21:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014621 s, 71.7 MB/s 2024-04-25T10:21:02.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.494 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180838 s, 58.0 MB/s 2024-04-25T10:21:02.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.511 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149575 s, 70.1 MB/s 2024-04-25T10:21:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140818 s, 74.5 MB/s 2024-04-25T10:21:02.553 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.553 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.553 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244403 s, 42.9 MB/s 2024-04-25T10:21:02.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156013 s, 67.2 MB/s 2024-04-25T10:21:02.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.590 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.590 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182355 s, 57.5 MB/s 2024-04-25T10:21:02.606 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.607 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.607 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152074 s, 69.0 MB/s 2024-04-25T10:21:02.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180432 s, 58.1 MB/s 2024-04-25T10:21:02.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155729 s, 67.3 MB/s 2024-04-25T10:21:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185439 s, 56.5 MB/s 2024-04-25T10:21:02.677 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.677 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112371 s, 93.3 MB/s 2024-04-25T10:21:02.694 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.694 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.694 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147822 s, 70.9 MB/s 2024-04-25T10:21:02.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142232 s, 73.7 MB/s 2024-04-25T10:21:02.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.731 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187815 s, 55.8 MB/s 2024-04-25T10:21:02.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.749 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165899 s, 63.2 MB/s 2024-04-25T10:21:02.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.766 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156135 s, 67.2 MB/s 2024-04-25T10:21:02.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.788 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191044 s, 54.9 MB/s 2024-04-25T10:21:02.806 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.806 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.806 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169545 s, 61.8 MB/s 2024-04-25T10:21:02.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.827 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196634 s, 53.3 MB/s 2024-04-25T10:21:02.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144358 s, 72.6 MB/s 2024-04-25T10:21:02.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:02.845 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:02.845 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:02.845 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:21:02.845 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:02.846 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:02.846 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:02.846 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:02.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.919 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143443 s, 73.1 MB/s 2024-04-25T10:21:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.940 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195017 s, 53.8 MB/s 2024-04-25T10:21:02.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.962 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205756 s, 51.0 MB/s 2024-04-25T10:21:02.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:02.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:02.981 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173526 s, 60.4 MB/s 2024-04-25T10:21:03.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.003 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201734 s, 52.0 MB/s 2024-04-25T10:21:03.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.021 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160739 s, 65.2 MB/s 2024-04-25T10:21:03.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.045 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206716 s, 50.7 MB/s 2024-04-25T10:21:03.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016628 s, 63.1 MB/s 2024-04-25T10:21:03.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.078 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145544 s, 72.0 MB/s 2024-04-25T10:21:03.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.095 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154202 s, 68.0 MB/s 2024-04-25T10:21:03.117 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.118 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.118 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207717 s, 50.5 MB/s 2024-04-25T10:21:03.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.134 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014211 s, 73.8 MB/s 2024-04-25T10:21:03.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.151 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152858 s, 68.6 MB/s 2024-04-25T10:21:03.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.167 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144325 s, 72.7 MB/s 2024-04-25T10:21:03.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.183 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145476 s, 72.1 MB/s 2024-04-25T10:21:03.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209297 s, 50.1 MB/s 2024-04-25T10:21:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163614 s, 64.1 MB/s 2024-04-25T10:21:03.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182909 s, 57.3 MB/s 2024-04-25T10:21:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192016 s, 54.6 MB/s 2024-04-25T10:21:03.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.283 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160441 s, 65.4 MB/s 2024-04-25T10:21:03.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.302 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173418 s, 60.5 MB/s 2024-04-25T10:21:03.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.318 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143805 s, 72.9 MB/s 2024-04-25T10:21:03.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.341 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208649 s, 50.3 MB/s 2024-04-25T10:21:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164055 s, 63.9 MB/s 2024-04-25T10:21:03.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.375 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143631 s, 73.0 MB/s 2024-04-25T10:21:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169405 s, 61.9 MB/s 2024-04-25T10:21:03.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.411 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154947 s, 67.7 MB/s 2024-04-25T10:21:03.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146775 s, 71.4 MB/s 2024-04-25T10:21:03.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140546 s, 74.6 MB/s 2024-04-25T10:21:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.466 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206737 s, 50.7 MB/s 2024-04-25T10:21:03.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.482 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148302 s, 70.7 MB/s 2024-04-25T10:21:03.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147054 s, 71.3 MB/s 2024-04-25T10:21:03.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:03.500 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:03.500 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:03.500 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:21:03.500 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:03.501 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:03.501 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:03.501 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:03.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.575 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150167 s, 69.8 MB/s 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022834 s, 45.9 MB/s 2024-04-25T10:21:03.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.616 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145058 s, 72.3 MB/s 2024-04-25T10:21:03.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.633 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147511 s, 71.1 MB/s 2024-04-25T10:21:03.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175896 s, 59.6 MB/s 2024-04-25T10:21:03.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.676 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215055 s, 48.8 MB/s 2024-04-25T10:21:03.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140661 s, 74.5 MB/s 2024-04-25T10:21:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140515 s, 74.6 MB/s 2024-04-25T10:21:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143826 s, 72.9 MB/s 2024-04-25T10:21:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178392 s, 58.8 MB/s 2024-04-25T10:21:03.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.765 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198029 s, 53.0 MB/s 2024-04-25T10:21:03.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.785 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180541 s, 58.1 MB/s 2024-04-25T10:21:03.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.804 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172628 s, 60.7 MB/s 2024-04-25T10:21:03.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.826 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019747 s, 53.1 MB/s 2024-04-25T10:21:03.842 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.843 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154933 s, 67.7 MB/s 2024-04-25T10:21:03.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.859 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.859 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142743 s, 73.5 MB/s 2024-04-25T10:21:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199401 s, 52.6 MB/s 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180244 s, 58.2 MB/s 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182289 s, 57.5 MB/s 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150635 s, 69.6 MB/s 2024-04-25T10:21:03.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.956 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177037 s, 59.2 MB/s 2024-04-25T10:21:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014985 s, 70.0 MB/s 2024-04-25T10:21:03.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:03.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:03.993 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182367 s, 57.5 MB/s 2024-04-25T10:21:04.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.016 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212333 s, 49.4 MB/s 2024-04-25T10:21:04.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.035 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016824 s, 62.3 MB/s 2024-04-25T10:21:04.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.058 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206618 s, 50.7 MB/s 2024-04-25T10:21:04.076 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.077 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.077 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174096 s, 60.2 MB/s 2024-04-25T10:21:04.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.093 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149613 s, 70.1 MB/s 2024-04-25T10:21:04.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.113 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177721 s, 59.0 MB/s 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168398 s, 62.3 MB/s 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226923 s, 46.2 MB/s 2024-04-25T10:21:04.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.172 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014442 s, 72.6 MB/s 2024-04-25T10:21:04.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:04.174 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:04.174 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:04.174 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:21:04.174 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:04.174 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:04.174 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:04.174 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:04.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147959 s, 70.9 MB/s 2024-04-25T10:21:04.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.263 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177904 s, 58.9 MB/s 2024-04-25T10:21:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168884 s, 62.1 MB/s 2024-04-25T10:21:04.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238145 s, 44.0 MB/s 2024-04-25T10:21:04.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.324 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150227 s, 69.8 MB/s 2024-04-25T10:21:04.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147826 s, 70.9 MB/s 2024-04-25T10:21:04.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182617 s, 57.4 MB/s 2024-04-25T10:21:04.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.377 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.377 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147171 s, 71.2 MB/s 2024-04-25T10:21:04.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.394 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148162 s, 70.8 MB/s 2024-04-25T10:21:04.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.412 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016179 s, 64.8 MB/s 2024-04-25T10:21:04.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.435 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021604 s, 48.5 MB/s 2024-04-25T10:21:04.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158235 s, 66.3 MB/s 2024-04-25T10:21:04.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.472 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179772 s, 58.3 MB/s 2024-04-25T10:21:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.489 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154854 s, 67.7 MB/s 2024-04-25T10:21:04.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164896 s, 63.6 MB/s 2024-04-25T10:21:04.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016327 s, 64.2 MB/s 2024-04-25T10:21:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180246 s, 58.2 MB/s 2024-04-25T10:21:04.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.558 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110006 s, 95.3 MB/s 2024-04-25T10:21:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153584 s, 68.3 MB/s 2024-04-25T10:21:04.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.591 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141888 s, 73.9 MB/s 2024-04-25T10:21:04.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.609 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165251 s, 63.5 MB/s 2024-04-25T10:21:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167581 s, 62.6 MB/s 2024-04-25T10:21:04.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.646 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016171 s, 64.8 MB/s 2024-04-25T10:21:04.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.665 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.665 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174737 s, 60.0 MB/s 2024-04-25T10:21:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185508 s, 56.5 MB/s 2024-04-25T10:21:04.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.704 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169674 s, 61.8 MB/s 2024-04-25T10:21:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177367 s, 59.1 MB/s 2024-04-25T10:21:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201162 s, 52.1 MB/s 2024-04-25T10:21:04.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.762 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154622 s, 67.8 MB/s 2024-04-25T10:21:04.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.783 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186846 s, 56.1 MB/s 2024-04-25T10:21:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015683 s, 66.9 MB/s 2024-04-25T10:21:04.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.826 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232619 s, 45.1 MB/s 2024-04-25T10:21:04.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:04.827 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:04.827 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:04.828 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:21:04.828 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:04.828 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:04.828 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:04.828 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:04.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.901 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157373 s, 66.6 MB/s 2024-04-25T10:21:04.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160883 s, 65.2 MB/s 2024-04-25T10:21:04.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.935 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153377 s, 68.4 MB/s 2024-04-25T10:21:04.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.958 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206734 s, 50.7 MB/s 2024-04-25T10:21:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149721 s, 70.0 MB/s 2024-04-25T10:21:04.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:04.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:04.991 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147026 s, 71.3 MB/s 2024-04-25T10:21:05.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.017 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241742 s, 43.4 MB/s 2024-04-25T10:21:05.034 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.034 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.034 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150078 s, 69.9 MB/s 2024-04-25T10:21:05.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.053 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173094 s, 60.6 MB/s 2024-04-25T10:21:05.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.070 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.070 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152353 s, 68.8 MB/s 2024-04-25T10:21:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014819 s, 70.8 MB/s 2024-04-25T10:21:05.103 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.103 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.103 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148419 s, 70.6 MB/s 2024-04-25T10:21:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142139 s, 73.8 MB/s 2024-04-25T10:21:05.140 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.140 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.140 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195452 s, 53.6 MB/s 2024-04-25T10:21:05.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.159 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169835 s, 61.7 MB/s 2024-04-25T10:21:05.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.176 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154714 s, 67.8 MB/s 2024-04-25T10:21:05.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.198 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193127 s, 54.3 MB/s 2024-04-25T10:21:05.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.213 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141135 s, 74.3 MB/s 2024-04-25T10:21:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.234 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183113 s, 57.3 MB/s 2024-04-25T10:21:05.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167509 s, 62.6 MB/s 2024-04-25T10:21:05.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.271 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172813 s, 60.7 MB/s 2024-04-25T10:21:05.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.290 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171772 s, 61.0 MB/s 2024-04-25T10:21:05.311 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.311 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.311 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192143 s, 54.6 MB/s 2024-04-25T10:21:05.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.327 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147178 s, 71.2 MB/s 2024-04-25T10:21:05.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197941 s, 53.0 MB/s 2024-04-25T10:21:05.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.366 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148752 s, 70.5 MB/s 2024-04-25T10:21:05.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.383 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159801 s, 65.6 MB/s 2024-04-25T10:21:05.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.400 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150455 s, 69.7 MB/s 2024-04-25T10:21:05.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.416 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145894 s, 71.9 MB/s 2024-04-25T10:21:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145167 s, 72.2 MB/s 2024-04-25T10:21:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017152 s, 61.1 MB/s 2024-04-25T10:21:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152221 s, 68.9 MB/s 2024-04-25T10:21:05.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:05.470 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:05.471 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:05.471 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:21:05.471 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:05.471 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:05.471 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:05.471 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:05.540 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.540 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.540 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152595 s, 68.7 MB/s 2024-04-25T10:21:05.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176912 s, 59.3 MB/s 2024-04-25T10:21:05.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.583 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221919 s, 47.3 MB/s 2024-04-25T10:21:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143238 s, 73.2 MB/s 2024-04-25T10:21:05.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.619 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179539 s, 58.4 MB/s 2024-04-25T10:21:05.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.636 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.636 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146162 s, 71.7 MB/s 2024-04-25T10:21:05.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.653 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151269 s, 69.3 MB/s 2024-04-25T10:21:05.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.668 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142029 s, 73.8 MB/s 2024-04-25T10:21:05.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.685 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151691 s, 69.1 MB/s 2024-04-25T10:21:05.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.709 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217053 s, 48.3 MB/s 2024-04-25T10:21:05.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150613 s, 69.6 MB/s 2024-04-25T10:21:05.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.744 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169573 s, 61.8 MB/s 2024-04-25T10:21:05.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.766 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196466 s, 53.4 MB/s 2024-04-25T10:21:05.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.783 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157772 s, 66.5 MB/s 2024-04-25T10:21:05.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.802 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01728 s, 60.7 MB/s 2024-04-25T10:21:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.820 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166072 s, 63.1 MB/s 2024-04-25T10:21:05.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.837 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153305 s, 68.4 MB/s 2024-04-25T10:21:05.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.858 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192151 s, 54.6 MB/s 2024-04-25T10:21:05.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.877 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168146 s, 62.4 MB/s 2024-04-25T10:21:05.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.894 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158278 s, 66.2 MB/s 2024-04-25T10:21:05.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.913 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167793 s, 62.5 MB/s 2024-04-25T10:21:05.931 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.931 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.931 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165861 s, 63.2 MB/s 2024-04-25T10:21:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016068 s, 65.3 MB/s 2024-04-25T10:21:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115882 s, 90.5 MB/s 2024-04-25T10:21:05.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.980 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160328 s, 65.4 MB/s 2024-04-25T10:21:05.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:05.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:05.997 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157723 s, 66.5 MB/s 2024-04-25T10:21:06.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182137 s, 57.6 MB/s 2024-04-25T10:21:06.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:06.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:06.036 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153116 s, 68.5 MB/s 2024-04-25T10:21:07.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.007 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.969764 s, 1.1 MB/s 2024-04-25T10:21:07.028 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.028 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.028 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187631 s, 55.9 MB/s 2024-04-25T10:21:07.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.048 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179574 s, 58.4 MB/s 2024-04-25T10:21:07.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.072 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220004 s, 47.7 MB/s 2024-04-25T10:21:07.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:07.074 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:07.074 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:07.074 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:21:07.074 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:07.074 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:07.074 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:07.074 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:07.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.143 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146239 s, 71.7 MB/s 2024-04-25T10:21:07.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227891 s, 46.0 MB/s 2024-04-25T10:21:07.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.190 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203644 s, 51.5 MB/s 2024-04-25T10:21:07.212 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.212 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.212 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201157 s, 52.1 MB/s 2024-04-25T10:21:07.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.231 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164464 s, 63.8 MB/s 2024-04-25T10:21:07.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.245 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131123 s, 80.0 MB/s 2024-04-25T10:21:07.261 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.261 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.261 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137896 s, 76.0 MB/s 2024-04-25T10:21:07.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.277 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148609 s, 70.6 MB/s 2024-04-25T10:21:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017602 s, 59.6 MB/s 2024-04-25T10:21:07.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.319 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205819 s, 50.9 MB/s 2024-04-25T10:21:07.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.335 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146224 s, 71.7 MB/s 2024-04-25T10:21:07.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.359 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214048 s, 49.0 MB/s 2024-04-25T10:21:07.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.381 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203804 s, 51.5 MB/s 2024-04-25T10:21:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.404 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212855 s, 49.3 MB/s 2024-04-25T10:21:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.426 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206952 s, 50.7 MB/s 2024-04-25T10:21:07.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148476 s, 70.6 MB/s 2024-04-25T10:21:07.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.461 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016455 s, 63.7 MB/s 2024-04-25T10:21:07.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.478 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146705 s, 71.5 MB/s 2024-04-25T10:21:07.500 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.500 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.500 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203917 s, 51.4 MB/s 2024-04-25T10:21:07.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.523 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204501 s, 51.3 MB/s 2024-04-25T10:21:07.540 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.541 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164568 s, 63.7 MB/s 2024-04-25T10:21:07.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115911 s, 90.5 MB/s 2024-04-25T10:21:07.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146343 s, 71.7 MB/s 2024-04-25T10:21:07.588 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.588 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.589 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162355 s, 64.6 MB/s 2024-04-25T10:21:07.606 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.607 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.607 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163992 s, 63.9 MB/s 2024-04-25T10:21:07.624 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.624 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.625 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162484 s, 64.5 MB/s 2024-04-25T10:21:07.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.646 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196167 s, 53.5 MB/s 2024-04-25T10:21:07.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.668 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203713 s, 51.5 MB/s 2024-04-25T10:21:07.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169585 s, 61.8 MB/s 2024-04-25T10:21:07.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167333 s, 62.7 MB/s 2024-04-25T10:21:07.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.730 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022735 s, 46.1 MB/s 2024-04-25T10:21:07.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.750 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172651 s, 60.7 MB/s 2024-04-25T10:21:07.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:07.750 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:07.751 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:07.751 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:21:07.751 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:07.751 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:07.751 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:07.751 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:07.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.827 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187019 s, 56.1 MB/s 2024-04-25T10:21:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162083 s, 64.7 MB/s 2024-04-25T10:21:07.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155425 s, 67.5 MB/s 2024-04-25T10:21:07.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.880 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162929 s, 64.4 MB/s 2024-04-25T10:21:07.901 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.902 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.902 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196851 s, 53.3 MB/s 2024-04-25T10:21:07.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149094 s, 70.3 MB/s 2024-04-25T10:21:07.944 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.944 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.944 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239675 s, 43.7 MB/s 2024-04-25T10:21:07.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.967 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.967 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213752 s, 49.1 MB/s 2024-04-25T10:21:07.983 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:07.984 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147784 s, 71.0 MB/s 2024-04-25T10:21:08.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.000 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146428 s, 71.6 MB/s 2024-04-25T10:21:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236023 s, 44.4 MB/s 2024-04-25T10:21:08.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.045 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174506 s, 60.1 MB/s 2024-04-25T10:21:08.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.062 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151889 s, 69.0 MB/s 2024-04-25T10:21:08.083 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.083 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.083 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194949 s, 53.8 MB/s 2024-04-25T10:21:08.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.100 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155407 s, 67.5 MB/s 2024-04-25T10:21:08.117 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.117 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.117 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150266 s, 69.8 MB/s 2024-04-25T10:21:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148225 s, 70.7 MB/s 2024-04-25T10:21:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.153 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165681 s, 63.3 MB/s 2024-04-25T10:21:08.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.174 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205619 s, 51.0 MB/s 2024-04-25T10:21:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208186 s, 50.4 MB/s 2024-04-25T10:21:08.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014678 s, 71.4 MB/s 2024-04-25T10:21:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140375 s, 74.7 MB/s 2024-04-25T10:21:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131018 s, 80.0 MB/s 2024-04-25T10:21:08.266 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.266 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.266 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199137 s, 52.7 MB/s 2024-04-25T10:21:08.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.285 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171814 s, 61.0 MB/s 2024-04-25T10:21:08.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.301 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145627 s, 72.0 MB/s 2024-04-25T10:21:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148096 s, 70.8 MB/s 2024-04-25T10:21:08.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.336 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158719 s, 66.1 MB/s 2024-04-25T10:21:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.353 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151454 s, 69.2 MB/s 2024-04-25T10:21:08.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.371 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171533 s, 61.1 MB/s 2024-04-25T10:21:08.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162796 s, 64.4 MB/s 2024-04-25T10:21:08.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:08.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:08.405 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143982 s, 72.8 MB/s 2024-04-25T10:21:08.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:08.407 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:21:08.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.587+0000 7f639d400640 1 Processor -- start 2024-04-25T10:21:08.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.587+0000 7f639d400640 1 -- start start 2024-04-25T10:21:08.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f639d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63980cf040 0x7f63980cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f639d400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6398071f80 con 0x7f639815bc00 2024-04-25T10:21:08.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f639d400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639815b120 con 0x7f6398071ab0 2024-04-25T10:21:08.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f639d400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6398072260 con 0x7f63980cf040 2024-04-25T10:21:08.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6396800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63980cf040 0x7f63980cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6396800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63980cf040 0x7f63980cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36160/0 (socket says 172.21.15.87:36160) 2024-04-25T10:21:08.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6395e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f639815bc00 0x7f63980713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37858/0 (socket says 172.21.15.87:37858) 2024-04-25T10:21:08.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6396800640 1 -- 172.21.15.87:0/2261034573 learned_addr learned my addr 172.21.15.87:0/2261034573 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6396800640 1 -- 172.21.15.87:0/2261034573 >> v1:172.21.15.169:6789/0 conn(0x7f6398071ab0 legacy=0x7f63980ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6396800640 1 -- 172.21.15.87:0/2261034573 >> v1:172.21.15.87:6789/0 conn(0x7f639815bc00 legacy=0x7f63980713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6396800640 1 -- 172.21.15.87:0/2261034573 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639815b400 con 0x7f63980cf040 2024-04-25T10:21:08.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.591+0000 7f6395400640 1 -- 172.21.15.87:0/2261034573 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042489516 0 0) 0x7f6398071f80 con 0x7f639815bc00 2024-04-25T10:21:08.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f6396800640 1 --2- 172.21.15.87:0/2261034573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63980cf9e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6388009e50 tx=0x7f638801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5edb6113ae563f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f6395400640 1 -- 172.21.15.87:0/2261034573 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880a5700 con 0x7f63980cf040 2024-04-25T10:21:08.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f6395400640 1 -- 172.21.15.87:0/2261034573 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63880a58a0 con 0x7f63980cf040 2024-04-25T10:21:08.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f6395400640 1 -- 172.21.15.87:0/2261034573 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880a5c70 con 0x7f63980cf040 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- 172.21.15.87:0/2261034573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 msgr2=0x7f63980cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 --2- 172.21.15.87:0/2261034573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63980cf9e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6388009e50 tx=0x7f638801ca30 comp rx=0 tx=0).stop 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- 172.21.15.87:0/2261034573 shutdown_connections 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 --2- 172.21.15.87:0/2261034573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63980cf9e0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- 172.21.15.87:0/2261034573 >> 172.21.15.87:0/2261034573 conn(0x7f6398066b70 msgr2=0x7f6398066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- 172.21.15.87:0/2261034573 shutdown_connections 2024-04-25T10:21:08.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- 172.21.15.87:0/2261034573 wait complete. 2024-04-25T10:21:08.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 Processor -- start 2024-04-25T10:21:08.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- start start 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6398071ab0 0x7f639815a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63981513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639815bc00 0x7f63981555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980d0330 con 0x7f63980cf040 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980cf630 con 0x7f639815bc00 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.595+0000 7f639d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398168870 con 0x7f6398071ab0 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63981513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6396800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6398071ab0 0x7f639815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63981513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36176/0 (socket says 172.21.15.87:36176) 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6396800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6398071ab0 0x7f639815a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35394/0 (socket says 172.21.15.87:35394) 2024-04-25T10:21:08.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 -- 172.21.15.87:0/3252500851 learned_addr learned my addr 172.21.15.87:0/3252500851 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6397200640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639815bc00 0x7f63981555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 -- 172.21.15.87:0/3252500851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6398071ab0 msgr2=0x7f639815a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6398071ab0 0x7f639815a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 -- 172.21.15.87:0/3252500851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639815bc00 msgr2=0x7f63981555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639815bc00 0x7f63981555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 -- 172.21.15.87:0/3252500851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63981518e0 con 0x7f63980cf040 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f6395e00640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63981513a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f638400c480 tx=0x7f638400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f638ea00640 1 -- 172.21.15.87:0/3252500851 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384018070 con 0x7f63980cf040 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f638ea00640 1 -- 172.21.15.87:0/3252500851 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63840074e0 con 0x7f63980cf040 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f638ea00640 1 -- 172.21.15.87:0/3252500851 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384004a30 con 0x7f63980cf040 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6398159d20 con 0x7f63980cf040 2024-04-25T10:21:08.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6398152560 con 0x7f63980cf040 2024-04-25T10:21:08.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.599+0000 7f638ea00640 1 -- 172.21.15.87:0/3252500851 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6384004bd0 con 0x7f63980cf040 2024-04-25T10:21:08.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f638ea00640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6378066f60 0x7f6378069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f638ea00640 1 -- 172.21.15.87:0/3252500851 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6384084240 con 0x7f63980cf040 2024-04-25T10:21:08.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6396800640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6378066f60 0x7f6378069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 Processor -- start 2024-04-25T10:21:08.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- start start 2024-04-25T10:21:08.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6396800640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6378066f60 0x7f6378069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f63880ae000 tx=0x7f63880a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e2b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477de1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477de2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564477c2fd50 con 0x564477de2520 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564477bb7e40 con 0x564477e2b3f0 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564477c2bdf0 con 0x564477e2bd10 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477de2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477de2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36186/0 (socket says 172.21.15.87:36186) 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 -- 172.21.15.87:0/561531091 learned_addr learned my addr 172.21.15.87:0/561531091 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6395e00640 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477de1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6396800640 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e2b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 -- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 msgr2=0x564477de1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477de1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 -- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 msgr2=0x564477e2b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e2b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 -- 172.21.15.87:0/561531091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564477bccd90 con 0x564477de2520 2024-04-25T10:21:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6397200640 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477de2900 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6390001770 tx=0x7f6390007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b8a2616a793de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6383400640 1 -- 172.21.15.87:0/561531091 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900052e0 con 0x564477de2520 2024-04-25T10:21:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6383400640 1 -- 172.21.15.87:0/561531091 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390018070 con 0x564477de2520 2024-04-25T10:21:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f6383400640 1 -- 172.21.15.87:0/561531091 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000f950 con 0x564477de2520 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- 172.21.15.87:0/561531091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 msgr2=0x564477de2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477de2900 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6390001770 tx=0x7f6390007dc0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- 172.21.15.87:0/561531091 shutdown_connections 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477de2900 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477de1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 --2- 172.21.15.87:0/561531091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e2b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- 172.21.15.87:0/561531091 >> 172.21.15.87:0/561531091 conn(0x564477dde4d0 msgr2=0x564477de08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- 172.21.15.87:0/561531091 shutdown_connections 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.603+0000 7f63a088b000 1 -- 172.21.15.87:0/561531091 wait complete. 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 -- start start 2024-04-25T10:21:08.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 Processor -- start 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e19600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564477bd5610 con 0x564477e2bd10 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564477bea9a0 con 0x564477e2b3f0 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564477c34380 con 0x564477de2520 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 10 client.0 _ll_register_callbacks cb 0x564477ce40d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6397200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35410/0 (socket says 172.21.15.87:35410) 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 -- 172.21.15.87:0/1319349705 learned_addr learned my addr 172.21.15.87:0/1319349705 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6395e00640 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 -- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 msgr2=0x564477e190c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e190c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 -- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 msgr2=0x564477e19600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e19600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 -- 172.21.15.87:0/1319349705 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564477c2ad40 con 0x564477de2520 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6397200640 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6396800640 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de8080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f638801ca70 tx=0x7f638801b270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7c1db75d17488e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6377e00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6377e00640 1 -- 172.21.15.87:0/1319349705 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880b6b40 con 0x564477de2520 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6377e00640 1 -- 172.21.15.87:0/1319349705 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f63880b6ce0 con 0x564477de2520 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6390018070 con 0x564477de2520 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6377e00640 1 -- 172.21.15.87:0/1319349705 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63880d4050 con 0x564477de2520 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 10 client.7073 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x564477cb2f00 con 0x564477de2520 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6377e00640 1 -- 172.21.15.87:0/1319349705 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390018070 con 0x564477de2520 2024-04-25T10:21:08.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6377e00640 1 -- 172.21.15.87:0/1319349705 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6388008030 con 0x564477de2520 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 10 client.7073 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 10 client.7073 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 20 client.7073 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e32830 0x564477e32c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 4 client.7073 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564477bcba20 con 0x564477e32830 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.607+0000 7f6395e00640 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e32830 0x564477e32c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f6395e00640 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e32830 0x564477e32c10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5e14c323d5296f server_cookie=dfef89174e681d7c in_seq=0 out_seq=0 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f6377e00640 10 client.7073 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f6377e00640 1 -- 172.21.15.87:0/1319349705 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564477bcba20 con 0x564477e32830 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f6377e00640 10 client.7073 handle_command_reply: tid=0 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 client.7073 shutdown 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 msgr2=0x564477de8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de8080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f638801ca70 tx=0x7f638801b270 comp rx=0 tx=0).stop 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 shutdown_connections 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e32830 0x564477e32c10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e190c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 --2- 172.21.15.87:0/1319349705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de8080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f6377e00640 0 client.7073 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 >> 172.21.15.87:0/1319349705 conn(0x564477dde4d0 msgr2=0x564477de1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 shutdown_connections 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 1 -- 172.21.15.87:0/1319349705 wait complete. 2024-04-25T10:21:08.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.611+0000 7f63a088b000 20 client.7073 trim_cache size 0 max 16384 2024-04-25T10:21:08.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 Processor -- start 2024-04-25T10:21:08.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- start start 2024-04-25T10:21:08.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477de1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398203230 con 0x564477e2bd10 2024-04-25T10:21:08.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63982023b0 con 0x564477e2b3f0 2024-04-25T10:21:08.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398201900 con 0x564477de2520 2024-04-25T10:21:08.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e2e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36210/0 (socket says 172.21.15.87:36210) 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 -- 172.21.15.87:0/3436762423 learned_addr learned my addr 172.21.15.87:0/3436762423 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 -- 172.21.15.87:0/3436762423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 msgr2=0x564477de18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 --2- 172.21.15.87:0/3436762423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 -- 172.21.15.87:0/3436762423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 msgr2=0x564477de1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 --2- 172.21.15.87:0/3436762423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477de1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 -- 172.21.15.87:0/3436762423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564477e2bb40 con 0x564477e2bd10 2024-04-25T10:21:08.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6397200640 1 --2- 172.21.15.87:0/3436762423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e2e100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f6390000c10 tx=0x7f639000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6e25cd5f3182ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6382a00640 1 -- 172.21.15.87:0/3436762423 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000fb60 con 0x564477e2bd10 2024-04-25T10:21:08.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6382a00640 1 -- 172.21.15.87:0/3436762423 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390018040 con 0x564477e2bd10 2024-04-25T10:21:08.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f6382a00640 1 -- 172.21.15.87:0/3436762423 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390003600 con 0x564477e2bd10 2024-04-25T10:21:08.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- 172.21.15.87:0/3436762423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 msgr2=0x564477e2e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 --2- 172.21.15.87:0/3436762423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e2e100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f6390000c10 tx=0x7f639000a000 comp rx=0 tx=0).stop 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- 172.21.15.87:0/3436762423 shutdown_connections 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 --2- 172.21.15.87:0/3436762423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477e2bd10 0x564477e2e100 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 --2- 172.21.15.87:0/3436762423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477de1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 --2- 172.21.15.87:0/3436762423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477de2520 0x564477de18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- 172.21.15.87:0/3436762423 >> 172.21.15.87:0/3436762423 conn(0x564477dde4d0 msgr2=0x564477e29f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- 172.21.15.87:0/3436762423 shutdown_connections 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- 172.21.15.87:0/3436762423 wait complete. 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 -- start start 2024-04-25T10:21:08.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.627+0000 7f63a088b000 1 Processor -- start 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477e30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e48e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477e49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564477be6dc0 con 0x564477de2520 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398201e40 con 0x564477e2b3f0 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564477c2f060 con 0x564477e2bd10 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 10 client.0 _ll_register_callbacks cb 0x7f6390001210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477e30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477e30c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36218/0 (socket says 172.21.15.87:36218) 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 -- 172.21.15.87:0/3762722888 learned_addr learned my addr 172.21.15.87:0/3762722888 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6397200640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477e49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6395e00640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e48e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 -- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 msgr2=0x564477e49350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477e49350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 -- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 msgr2=0x564477e48e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e48e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 -- 172.21.15.87:0/3762722888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564477c89330 con 0x564477de2520 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6395e00640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6396800640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477e30c70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f63880097e0 tx=0x7f63880197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708d97857d78d8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6377e00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6377e00640 1 -- 172.21.15.87:0/3762722888 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880df060 con 0x564477de2520 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6377e00640 1 -- 172.21.15.87:0/3762722888 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f63880b6bd0 con 0x564477de2520 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6390018040 con 0x564477de2520 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6377e00640 1 -- 172.21.15.87:0/3762722888 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63880e5050 con 0x564477de2520 2024-04-25T10:21:08.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 10 client.8697 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x564477bfc5f0 con 0x564477de2520 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6377e00640 1 -- 172.21.15.87:0/3762722888 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390018040 con 0x564477de2520 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6377e00640 1 -- 172.21.15.87:0/3762722888 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63880e8020 con 0x564477de2520 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 10 client.8697 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 10 client.8697 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 20 client.8697 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e8c780 0x564477e8cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 4 client.8697 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x564477c3d290 con 0x564477e8c780 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6395e00640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e8c780 0x564477e8cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6377e00640 10 client.8697 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.631+0000 7f6395e00640 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e8c780 0x564477e8cb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7396f0e326e7896d server_cookie=5ebedcd78e2fc639 in_seq=0 out_seq=0 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f6377e00640 1 -- 172.21.15.87:0/3762722888 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564477c3d290 con 0x564477e8c780 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f6377e00640 10 client.8697 handle_command_reply: tid=0 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 client.8697 shutdown 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 msgr2=0x564477e30c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477e30c70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f63880097e0 tx=0x7f63880197d0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 shutdown_connections 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x564477e8c780 0x564477e8cb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564477e2bd10 0x564477e49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564477e2b3f0 0x564477e48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 --2- 172.21.15.87:0/3762722888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564477de2520 0x564477e30c70 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 >> 172.21.15.87:0/3762722888 conn(0x564477dde4d0 msgr2=0x564477e2a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f6377e00640 0 client.8697 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 shutdown_connections 2024-04-25T10:21:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 1 -- 172.21.15.87:0/3762722888 wait complete. 2024-04-25T10:21:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f63a088b000 20 client.8697 trim_cache size 0 max 16384 2024-04-25T10:21:08.640 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:08.640 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:21:08.640 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "88487972-2d40-452b-a934-070918dfcc03", 2024-04-25T10:21:08.640 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:21:08.640 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6378066f60 msgr2=0x7f6378069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6378066f60 0x7f6378069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f63880ae000 tx=0x7f63880a7070 comp rx=0 tx=0).stop 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 msgr2=0x7f63981513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63981513a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f638400c480 tx=0x7f638400c940 comp rx=0 tx=0).stop 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 shutdown_connections 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6378066f60 0x7f6378069420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f639815bc00 0x7f63981555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63980cf040 0x7f63981513a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 --2- 172.21.15.87:0/3252500851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6398071ab0 0x7f639815a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 >> 172.21.15.87:0/3252500851 conn(0x7f6398066b70 msgr2=0x7f6398066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 shutdown_connections 2024-04-25T10:21:08.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.635+0000 7f639d400640 1 -- 172.21.15.87:0/3252500851 wait complete. 2024-04-25T10:21:08.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:08.659 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T10:21:08.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91e6400640 1 Processor -- start 2024-04-25T10:21:08.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91e6400640 1 -- start start 2024-04-25T10:21:08.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91e6400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f91e0070350 0x7f91e015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91e6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e015b080 con 0x7f91e00aef80 2024-04-25T10:21:08.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91e6400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e015adb0 con 0x7f91e00b31c0 2024-04-25T10:21:08.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91e6400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e015eb60 con 0x7f91e0070350 2024-04-25T10:21:08.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91df400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f91e00aef80 0x7f91e015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37872/0 (socket says 172.21.15.87:37872) 2024-04-25T10:21:08.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91df400640 1 -- 172.21.15.87:0/585297290 learned_addr learned my addr 172.21.15.87:0/585297290 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:08.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91dea00640 1 --2- 172.21.15.87:0/585297290 >> v2:172.21.15.169:3300/0 conn(0x7f91e0070350 0x7f91e015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91de000640 1 -- 172.21.15.87:0/585297290 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630125662 0 0) 0x7f91e015b080 con 0x7f91e00aef80 2024-04-25T10:21:08.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91de000640 1 -- 172.21.15.87:0/585297290 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c4003530 con 0x7f91e00aef80 2024-04-25T10:21:08.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91de000640 1 -- 172.21.15.87:0/585297290 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316362250 0 0) 0x7f91e015adb0 con 0x7f91e00b31c0 2024-04-25T10:21:08.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91de000640 1 -- 172.21.15.87:0/585297290 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e015b080 con 0x7f91e00b31c0 2024-04-25T10:21:08.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91dea00640 1 -- 172.21.15.87:0/585297290 >> v1:172.21.15.169:6789/0 conn(0x7f91e00b31c0 legacy=0x7f91e00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91dea00640 1 -- 172.21.15.87:0/585297290 >> v1:172.21.15.87:6789/0 conn(0x7f91e00aef80 legacy=0x7f91e015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.831+0000 7f91dea00640 1 -- 172.21.15.87:0/585297290 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e015ee40 con 0x7f91e0070350 2024-04-25T10:21:08.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91dea00640 1 --2- 172.21.15.87:0/585297290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e0070350 0x7f91e015e480 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0097c0 tx=0x7f91cc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b19acad88c8e2f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91de000640 1 -- 172.21.15.87:0/585297290 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0b2070 con 0x7f91e0070350 2024-04-25T10:21:08.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91de000640 1 -- 172.21.15.87:0/585297290 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0ad680 con 0x7f91e0070350 2024-04-25T10:21:08.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91de000640 1 -- 172.21.15.87:0/585297290 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0ada50 con 0x7f91e0070350 2024-04-25T10:21:08.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91e6400640 1 -- 172.21.15.87:0/585297290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e0070350 msgr2=0x7f91e015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91e6400640 1 --2- 172.21.15.87:0/585297290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e0070350 0x7f91e015e480 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0097c0 tx=0x7f91cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91e6400640 1 -- 172.21.15.87:0/585297290 shutdown_connections 2024-04-25T10:21:08.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91e6400640 1 --2- 172.21.15.87:0/585297290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e0070350 0x7f91e015e480 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91e6400640 1 -- 172.21.15.87:0/585297290 >> 172.21.15.87:0/585297290 conn(0x7f91e00669a0 msgr2=0x7f91e0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91e6400640 1 -- 172.21.15.87:0/585297290 shutdown_connections 2024-04-25T10:21:08.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.835+0000 7f91e6400640 1 -- 172.21.15.87:0/585297290 wait complete. 2024-04-25T10:21:08.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 Processor -- start 2024-04-25T10:21:08.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 -- start start 2024-04-25T10:21:08.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e0070350 0x7f91e0157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e00aef80 0x7f91e01512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00b31c0 0x7f91e01545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cfef0 con 0x7f91e00b31c0 2024-04-25T10:21:08.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cfd70 con 0x7f91e00aef80 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00b35d0 con 0x7f91e0070350 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00b31c0 0x7f91e01545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00b31c0 0x7f91e01545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36232/0 (socket says 172.21.15.87:36232) 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 -- 172.21.15.87:0/3414245387 learned_addr learned my addr 172.21.15.87:0/3414245387 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dea00640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e00aef80 0x7f91e01512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91df400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e0070350 0x7f91e0157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 -- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e0070350 msgr2=0x7f91e0157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e0070350 0x7f91e0157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 -- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e00aef80 msgr2=0x7f91e01512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e00aef80 0x7f91e01512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 -- 172.21.15.87:0/3414245387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00b4ac0 con 0x7f91e00b31c0 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91df400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e0070350 0x7f91e0157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91dfe00640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00b31c0 0x7f91e01545e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c730 tx=0x7f91d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91d7400640 1 -- 172.21.15.87:0/3414245387 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8003b80 con 0x7f91e00b31c0 2024-04-25T10:21:08.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91d7400640 1 -- 172.21.15.87:0/3414245387 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8003d20 con 0x7f91e00b31c0 2024-04-25T10:21:08.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91d7400640 1 -- 172.21.15.87:0/3414245387 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d800aa90 con 0x7f91e00b31c0 2024-04-25T10:21:08.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e0157e60 con 0x7f91e00b31c0 2024-04-25T10:21:08.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.839+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0158420 con 0x7f91e00b31c0 2024-04-25T10:21:08.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91d7400640 1 -- 172.21.15.87:0/3414245387 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91d800ac30 con 0x7f91e00b31c0 2024-04-25T10:21:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91d7400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f60 0x7f91c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91d7400640 1 -- 172.21.15.87:0/3414245387 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f91d8084f90 con 0x7f91e00b31c0 2024-04-25T10:21:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f60 0x7f91c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 Processor -- start 2024-04-25T10:21:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 -- start start 2024-04-25T10:21:08.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f60 0x7f91c0069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f91d00011f0 tx=0x7f91d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed0099f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0099f770 0x55ed00959c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0095a1a0 0x55ed0095a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed008381f0 con 0x55ed0099ee50 2024-04-25T10:21:08.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0079ee90 con 0x55ed0095a1a0 2024-04-25T10:21:08.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed00726f50 con 0x55ed0099f770 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed0099f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed0099f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36246/0 (socket says 172.21.15.87:36246) 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 -- 172.21.15.87:0/2796201612 learned_addr learned my addr 172.21.15.87:0/2796201612 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91dfe00640 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0095a1a0 0x55ed0095a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91dea00640 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0099f770 0x55ed00959c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 -- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0099f770 msgr2=0x55ed00959c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0099f770 0x55ed00959c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 -- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0095a1a0 msgr2=0x55ed0095a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0095a1a0 0x55ed0095a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 -- 172.21.15.87:0/2796201612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed007768b0 con 0x55ed0099ee50 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91dfe00640 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0095a1a0 0x55ed0095a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.843+0000 7f91df400640 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed0099f230 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005b70 tx=0x7f91d0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4571b4c64260e110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91cbe00640 1 -- 172.21.15.87:0/2796201612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000dbb0 con 0x55ed0099ee50 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91cbe00640 1 -- 172.21.15.87:0/2796201612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000dd50 con 0x55ed0099ee50 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91cbe00640 1 -- 172.21.15.87:0/2796201612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0020690 con 0x55ed0099ee50 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- 172.21.15.87:0/2796201612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 msgr2=0x55ed0099f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed0099f230 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005b70 tx=0x7f91d0010000 comp rx=0 tx=0).stop 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- 172.21.15.87:0/2796201612 shutdown_connections 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0095a1a0 0x55ed0095a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0099f770 0x55ed00959c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 --2- 172.21.15.87:0/2796201612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed0099f230 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- 172.21.15.87:0/2796201612 >> 172.21.15.87:0/2796201612 conn(0x55ed00951d40 msgr2=0x55ed00954130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- 172.21.15.87:0/2796201612 shutdown_connections 2024-04-25T10:21:08.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- 172.21.15.87:0/2796201612 wait complete. 2024-04-25T10:21:08.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- start start 2024-04-25T10:21:08.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 Processor -- start 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 0x55ed009a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed0095fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed00759540 con 0x55ed0099f770 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0079af30 con 0x55ed0099ee50 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed00745c00 con 0x55ed0095a1a0 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 10 client.0 _ll_register_callbacks cb 0x55ed00853180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed0095fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed0095fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36256/0 (socket says 172.21.15.87:36256) 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 -- 172.21.15.87:0/3534484807 learned_addr learned my addr 172.21.15.87:0/3534484807 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91df400640 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dea00640 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 0x55ed009a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 -- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 msgr2=0x55ed009a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 -- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 msgr2=0x55ed009a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 0x55ed009a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 -- 172.21.15.87:0/3534484807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed007271a0 con 0x55ed0099f770 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91dfe00640 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed0095fda0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f91d80557e0 tx=0x7f91d8055f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862482537540bbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91c8c00640 1 -- 172.21.15.87:0/3534484807 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8081060 con 0x55ed0099f770 2024-04-25T10:21:08.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91c8c00640 1 -- 172.21.15.87:0/3534484807 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f91d806d210 con 0x55ed0099f770 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.847+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91d000dd50 con 0x55ed0099f770 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91c8c00640 1 -- 172.21.15.87:0/3534484807 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d8072050 con 0x55ed0099f770 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 10 client.8709 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55ed00821840 con 0x55ed0099f770 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91c8c00640 1 -- 172.21.15.87:0/3534484807 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000dd50 con 0x55ed0099f770 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91c8c00640 1 -- 172.21.15.87:0/3534484807 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91d805dad0 con 0x55ed0099f770 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 10 client.8709 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 10 client.8709 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 20 client.8709 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed00960ae0 0x55ed00960ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 4 client.8709 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed0073abe0 con 0x55ed00960ae0 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91df400640 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed00960ae0 0x55ed00960ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91c8c00640 10 client.8709 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91df400640 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed00960ae0 0x55ed00960ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ac23cc48171317 server_cookie=75041e3eeeacbb7c in_seq=0 out_seq=0 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91c8c00640 1 -- 172.21.15.87:0/3534484807 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed0073abe0 con 0x55ed00960ae0 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91c8c00640 10 client.8709 handle_command_reply: tid=0 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 client.8709 shutdown 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 msgr2=0x55ed0095fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed0095fda0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f91d80557e0 tx=0x7f91d8055f50 comp rx=0 tx=0).stop 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 shutdown_connections 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed00960ae0 0x55ed00960ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed0095fda0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 0x55ed009a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91c8c00640 0 client.8709 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 --2- 172.21.15.87:0/3534484807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 >> 172.21.15.87:0/3534484807 conn(0x55ed00951d40 msgr2=0x55ed009549c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 shutdown_connections 2024-04-25T10:21:08.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 1 -- 172.21.15.87:0/3534484807 wait complete. 2024-04-25T10:21:08.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.851+0000 7f91e9958000 20 client.8709 trim_cache size 0 max 16384 2024-04-25T10:21:08.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 Processor -- start 2024-04-25T10:21:08.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- start start 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 0x55ed009fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed009fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01fe9f0 con 0x55ed0099f770 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01fdbf0 con 0x55ed0099ee50 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01fcff0 con 0x55ed0095a1a0 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed009fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed009fbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36260/0 (socket says 172.21.15.87:36260) 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 -- 172.21.15.87:0/2929733665 learned_addr learned my addr 172.21.15.87:0/2929733665 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 -- 172.21.15.87:0/2929733665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 msgr2=0x55ed009fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 --2- 172.21.15.87:0/2929733665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 -- 172.21.15.87:0/2929733665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 msgr2=0x55ed009fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 --2- 172.21.15.87:0/2929733665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 0x55ed009fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 -- 172.21.15.87:0/2929733665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0099f5a0 con 0x55ed0099f770 2024-04-25T10:21:08.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91dfe00640 1 --2- 172.21.15.87:0/2929733665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed009fbd00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f91d805deb0 tx=0x7f91d8005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ed15764a85336a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91cb400640 1 -- 172.21.15.87:0/2929733665 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d806d070 con 0x55ed0099f770 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91cb400640 1 -- 172.21.15.87:0/2929733665 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8078070 con 0x55ed0099f770 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91cb400640 1 -- 172.21.15.87:0/2929733665 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8083300 con 0x55ed0099f770 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- 172.21.15.87:0/2929733665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 msgr2=0x55ed009fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 --2- 172.21.15.87:0/2929733665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed009fbd00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f91d805deb0 tx=0x7f91d8005070 comp rx=0 tx=0).stop 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- 172.21.15.87:0/2929733665 shutdown_connections 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 --2- 172.21.15.87:0/2929733665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099f770 0x55ed009fbd00 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 --2- 172.21.15.87:0/2929733665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099ee50 0x55ed009fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 --2- 172.21.15.87:0/2929733665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- 172.21.15.87:0/2929733665 >> 172.21.15.87:0/2929733665 conn(0x55ed00951d40 msgr2=0x55ed00954130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- 172.21.15.87:0/2929733665 shutdown_connections 2024-04-25T10:21:08.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- 172.21.15.87:0/2929733665 wait complete. 2024-04-25T10:21:08.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.867+0000 7f91e9958000 1 -- start start 2024-04-25T10:21:08.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 Processor -- start 2024-04-25T10:21:08.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed009a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099f770 0x55ed009a4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed00951bc0 con 0x55ed0099ee50 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01fd5f0 con 0x55ed0099f770 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed00757df0 con 0x55ed0095a1a0 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 10 client.0 _ll_register_callbacks cb 0x55ed00761a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed009a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91df400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed009a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36274/0 (socket says 172.21.15.87:36274) 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 -- 172.21.15.87:0/2284860516 learned_addr learned my addr 172.21.15.87:0/2284860516 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dfe00640 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099f770 0x55ed009a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 -- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 msgr2=0x55ed009a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 -- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099f770 msgr2=0x55ed009a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099f770 0x55ed009a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 -- 172.21.15.87:0/2284860516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed007f8a10 con 0x55ed0099ee50 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91df400640 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91dea00640 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed009a42e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f91cc01c4e0 tx=0x7f91cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435b9a315862f0da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91c8c00640 1 -- 172.21.15.87:0/2284860516 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0b2040 con 0x55ed0099ee50 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91c8c00640 1 -- 172.21.15.87:0/2284860516 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f91cc004240 con 0x55ed0099ee50 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91d8078070 con 0x55ed0099ee50 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91c8c00640 1 -- 172.21.15.87:0/2284860516 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91cc004440 con 0x55ed0099ee50 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 10 client.8715 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55ed00733b00 con 0x55ed0099ee50 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91c8c00640 1 -- 172.21.15.87:0/2284860516 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8078070 con 0x55ed0099ee50 2024-04-25T10:21:08.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91c8c00640 1 -- 172.21.15.87:0/2284860516 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91cc0cb050 con 0x55ed0099ee50 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 10 client.8715 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 10 client.8715 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 20 client.8715 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed009bd010 0x55ed00a07ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 4 client.8715 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55ed007ac3d0 con 0x55ed009bd010 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91df400640 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed009bd010 0x55ed00a07ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91df400640 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed009bd010 0x55ed00a07ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=facf4d6a948db3b0 server_cookie=3e0f281cb1e93257 in_seq=0 out_seq=0 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.871+0000 7f91c8c00640 10 client.8715 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91c8c00640 1 -- 172.21.15.87:0/2284860516 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55ed007ac3d0 con 0x55ed009bd010 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91c8c00640 10 client.8715 handle_command_reply: tid=0 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 client.8715 shutdown 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 msgr2=0x55ed009a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed009a42e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f91cc01c4e0 tx=0x7f91cc019240 comp rx=0 tx=0).stop 2024-04-25T10:21:08.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 shutdown_connections 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x55ed009bd010 0x55ed00a07ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed0099f770 0x55ed009a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed0099ee50 0x55ed009a42e0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 --2- 172.21.15.87:0/2284860516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ed0095a1a0 0x55ed009a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91c8c00640 0 client.8715 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 >> 172.21.15.87:0/2284860516 conn(0x55ed00951d40 msgr2=0x55ed009549c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 shutdown_connections 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 1 -- 172.21.15.87:0/2284860516 wait complete. 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e9958000 20 client.8715 trim_cache size 0 max 16384 2024-04-25T10:21:08.880 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:08.881 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0 2024-04-25T10:21:08.881 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:08.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f60 msgr2=0x7f91c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f60 0x7f91c0069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f91d00011f0 tx=0x7f91d0009040 comp rx=0 tx=0).stop 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00b31c0 msgr2=0x7f91e01545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00b31c0 0x7f91e01545e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c730 tx=0x7f91d800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 shutdown_connections 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91c0066f60 0x7f91c0069420 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91e00b31c0 0x7f91e01545e0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91e00aef80 0x7f91e01512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 --2- 172.21.15.87:0/3414245387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91e0070350 0x7f91e0157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 >> 172.21.15.87:0/3414245387 conn(0x7f91e00669a0 msgr2=0x7f91e00b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 shutdown_connections 2024-04-25T10:21:08.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:08.875+0000 7f91e6400640 1 -- 172.21.15.87:0/3414245387 wait complete. 2024-04-25T10:21:08.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:08.895 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:21:09.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafb0800640 1 Processor -- start 2024-04-25T10:21:09.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafb0800640 1 -- start start 2024-04-25T10:21:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafb0800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fafa80aef80 0x7fafa815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafb0800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fafa8070350 0x7fafa815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafb0800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa815b080 con 0x7fafa815bac0 2024-04-25T10:21:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafb0800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa815adb0 con 0x7fafa80aef80 2024-04-25T10:21:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafb0800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa815eb60 con 0x7fafa8070350 2024-04-25T10:21:09.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafade00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fafa80aef80 0x7fafa815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafade00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fafa80aef80 0x7fafa815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36290/0 (socket says 172.21.15.87:36290) 2024-04-25T10:21:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafade00640 1 -- 172.21.15.87:0/3057821995 learned_addr learned my addr 172.21.15.87:0/3057821995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafad400640 1 --2- 172.21.15.87:0/3057821995 >> v2:172.21.15.169:3300/0 conn(0x7fafa8070350 0x7fafa815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafade00640 1 -- 172.21.15.87:0/3057821995 >> v2:172.21.15.169:3300/0 conn(0x7fafa8070350 msgr2=0x7fafa815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafade00640 1 --2- 172.21.15.87:0/3057821995 >> v2:172.21.15.169:3300/0 conn(0x7fafa8070350 0x7fafa815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafade00640 1 -- 172.21.15.87:0/3057821995 >> v1:172.21.15.169:6789/0 conn(0x7fafa815bac0 legacy=0x7fafa80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.031+0000 7fafade00640 1 -- 172.21.15.87:0/3057821995 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa815ee40 con 0x7fafa80aef80 2024-04-25T10:21:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafade00640 1 --2- 172.21.15.87:0/3057821995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 0x7fafa815e480 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fafa806e390 tx=0x7faf9c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5011b50c34ecd36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafaca00640 1 -- 172.21.15.87:0/3057821995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0aa660 con 0x7fafa80aef80 2024-04-25T10:21:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafaca00640 1 -- 172.21.15.87:0/3057821995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c0af070 con 0x7fafa80aef80 2024-04-25T10:21:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafaca00640 1 -- 172.21.15.87:0/3057821995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0aaa20 con 0x7fafa80aef80 2024-04-25T10:21:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 -- 172.21.15.87:0/3057821995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 msgr2=0x7fafa815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 --2- 172.21.15.87:0/3057821995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 0x7fafa815e480 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fafa806e390 tx=0x7faf9c01c520 comp rx=0 tx=0).stop 2024-04-25T10:21:09.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 -- 172.21.15.87:0/3057821995 shutdown_connections 2024-04-25T10:21:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 --2- 172.21.15.87:0/3057821995 >> v2:172.21.15.169:3300/0 conn(0x7fafa8070350 0x7fafa815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 --2- 172.21.15.87:0/3057821995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 0x7fafa815e480 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 -- 172.21.15.87:0/3057821995 >> 172.21.15.87:0/3057821995 conn(0x7fafa80669a0 msgr2=0x7fafa8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 -- 172.21.15.87:0/3057821995 shutdown_connections 2024-04-25T10:21:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.035+0000 7fafb0800640 1 -- 172.21.15.87:0/3057821995 wait complete. 2024-04-25T10:21:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 Processor -- start 2024-04-25T10:21:09.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 -- start start 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa8070350 0x7fafa8155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 0x7fafa813b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafa815bac0 0x7fafa8154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8167820 con 0x7fafa80aef80 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80b3b20 con 0x7fafa8070350 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa81679a0 con 0x7fafa815bac0 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa8070350 0x7fafa8155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafad400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 0x7fafa813b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa8070350 0x7fafa8155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:52932/0 (socket says 172.21.15.87:52932) 2024-04-25T10:21:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 -- 172.21.15.87:0/3559130158 learned_addr learned my addr 172.21.15.87:0/3559130158 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafae800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafa815bac0 0x7fafa8154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 -- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafa815bac0 msgr2=0x7fafa8154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafa815bac0 0x7fafa8154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 -- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 msgr2=0x7fafa813b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 0x7fafa813b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 -- 172.21.15.87:0/3559130158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa813c040 con 0x7fafa8070350 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafae800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafa815bac0 0x7fafa8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:09.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafade00640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa8070350 0x7fafa8155f20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faf9c019130 tx=0x7faf9c01ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafa2000640 1 -- 172.21.15.87:0/3559130158 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0bd5f0 con 0x7fafa8070350 2024-04-25T10:21:09.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafa2000640 1 -- 172.21.15.87:0/3559130158 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c0bd790 con 0x7fafa8070350 2024-04-25T10:21:09.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafa2000640 1 -- 172.21.15.87:0/3559130158 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0bdb40 con 0x7fafa8070350 2024-04-25T10:21:09.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa813c320 con 0x7fafa8070350 2024-04-25T10:21:09.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.039+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa813c8e0 con 0x7fafa8070350 2024-04-25T10:21:09.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.043+0000 7fafa2000640 1 -- 172.21.15.87:0/3559130158 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf9c0b33f0 con 0x7fafa8070350 2024-04-25T10:21:09.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.043+0000 7fafa2000640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf8c066f60 0x7faf8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.043+0000 7fafa2000640 1 -- 172.21.15.87:0/3559130158 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faf9c11fa40 con 0x7fafa8070350 2024-04-25T10:21:09.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.043+0000 7fafad400640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf8c066f60 0x7faf8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.043+0000 7fafb3bdf000 1 Processor -- start 2024-04-25T10:21:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.043+0000 7fafb3bdf000 1 -- start start 2024-04-25T10:21:09.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.043+0000 7fafad400640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf8c066f60 0x7faf8c069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fafa80cc230 tx=0x7faf98009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075d9d0 0x5645b075ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075e2f0 0x5645b0713b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b07140a0 0x5645b0714480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b0560ff0 con 0x5645b07140a0 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b04e9150 con 0x5645b075d9d0 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645b055d090 con 0x5645b075e2f0 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b07140a0 0x5645b0714480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b07140a0 0x5645b0714480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36300/0 (socket says 172.21.15.87:36300) 2024-04-25T10:21:09.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 -- 172.21.15.87:0/2098324824 learned_addr learned my addr 172.21.15.87:0/2098324824 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafade00640 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075d9d0 0x5645b075ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafad400640 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075e2f0 0x5645b0713b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 -- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075e2f0 msgr2=0x5645b0713b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075e2f0 0x5645b0713b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 -- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075d9d0 msgr2=0x5645b075ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075d9d0 0x5645b075ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 -- 172.21.15.87:0/2098324824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b06f70d0 con 0x5645b07140a0 2024-04-25T10:21:09.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafae800640 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b07140a0 0x5645b0714480 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fafa4001c90 tx=0x7fafa4002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8dcc9086ad7e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7faf96a00640 1 -- 172.21.15.87:0/2098324824 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4019070 con 0x5645b07140a0 2024-04-25T10:21:09.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7faf96a00640 1 -- 172.21.15.87:0/2098324824 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa4003b90 con 0x5645b07140a0 2024-04-25T10:21:09.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7faf96a00640 1 -- 172.21.15.87:0/2098324824 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa400e8c0 con 0x5645b07140a0 2024-04-25T10:21:09.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- 172.21.15.87:0/2098324824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b07140a0 msgr2=0x5645b0714480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b07140a0 0x5645b0714480 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fafa4001c90 tx=0x7fafa4002150 comp rx=0 tx=0).stop 2024-04-25T10:21:09.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- 172.21.15.87:0/2098324824 shutdown_connections 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b07140a0 0x5645b0714480 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075e2f0 0x5645b0713b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/2098324824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075d9d0 0x5645b075ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- 172.21.15.87:0/2098324824 >> 172.21.15.87:0/2098324824 conn(0x5645b0710020 msgr2=0x5645b0712440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- 172.21.15.87:0/2098324824 shutdown_connections 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- 172.21.15.87:0/2098324824 wait complete. 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 -- start start 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 Processor -- start 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.047+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b0742d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0743270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b0525250 con 0x5645b075e2f0 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b051b690 con 0x5645b07140a0 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645b04e9a40 con 0x5645b075d9d0 2024-04-25T10:21:09.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 10 client.0 _ll_register_callbacks cb 0x5645b06fee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0743270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0743270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36308/0 (socket says 172.21.15.87:36308) 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 -- 172.21.15.87:0/278919214 learned_addr learned my addr 172.21.15.87:0/278919214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafad400640 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b0742d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafade00640 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 -- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 msgr2=0x5645b0742d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b0742d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 -- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 msgr2=0x5645b07427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 -- 172.21.15.87:0/278919214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645b060e1c0 con 0x5645b075e2f0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafae800640 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0743270 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f90 tx=0x7fafa4001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48534dfb89363a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 1 -- 172.21.15.87:0/278919214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4023020 con 0x5645b075e2f0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 1 -- 172.21.15.87:0/278919214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fafa4006b60 con 0x5645b075e2f0 2024-04-25T10:21:09.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafa4003b90 con 0x5645b075e2f0 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 1 -- 172.21.15.87:0/278919214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafa402d050 con 0x5645b075e2f0 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 10 client.8727 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5645b060a170 con 0x5645b075e2f0 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 1 -- 172.21.15.87:0/278919214 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa4003b90 con 0x5645b075e2f0 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 1 -- 172.21.15.87:0/278919214 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafa4030020 con 0x5645b075e2f0 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 10 client.8727 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 10 client.8727 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 20 client.8727 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b076dc80 0x5645b0770070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 4 client.8727 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645b04fcc60 con 0x5645b076dc80 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafade00640 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b076dc80 0x5645b0770070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafade00640 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b076dc80 0x5645b0770070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76899f9eb0a292d server_cookie=262394fcbb86f0e8 in_seq=0 out_seq=0 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 10 client.8727 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 1 -- 172.21.15.87:0/278919214 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645b04fcc60 con 0x5645b076dc80 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 10 client.8727 handle_command_reply: tid=0 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 client.8727 shutdown 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 msgr2=0x5645b0743270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0743270 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f90 tx=0x7fafa4001260 comp rx=0 tx=0).stop 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 shutdown_connections 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b076dc80 0x5645b0770070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0743270 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b0742d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/278919214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7faf8b400640 0 client.8727 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 >> 172.21.15.87:0/278919214 conn(0x5645b0710020 msgr2=0x5645b0712ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 shutdown_connections 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 1 -- 172.21.15.87:0/278919214 wait complete. 2024-04-25T10:21:09.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.051+0000 7fafb3bdf000 20 client.8727 trim_cache size 0 max 16384 2024-04-25T10:21:09.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 Processor -- start 2024-04-25T10:21:09.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 -- start start 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b07140a0 0x5645b075cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075d9d0 0x5645b0713420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075e2f0 0x5645b0713960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8203060 con 0x5645b075d9d0 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8152cd0 con 0x5645b075e2f0 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa813ac50 con 0x5645b07140a0 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075d9d0 0x5645b0713420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075d9d0 0x5645b0713420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36320/0 (socket says 172.21.15.87:36320) 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 -- 172.21.15.87:0/106611355 learned_addr learned my addr 172.21.15.87:0/106611355 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafae800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075e2f0 0x5645b0713960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafade00640 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b07140a0 0x5645b075cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 -- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b07140a0 msgr2=0x5645b075cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b07140a0 0x5645b075cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 -- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075e2f0 msgr2=0x5645b0713960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075e2f0 0x5645b0713960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafad400640 1 -- 172.21.15.87:0/106611355 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b05a7de0 con 0x5645b075d9d0 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.067+0000 7fafae800640 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075e2f0 0x5645b0713960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafad400640 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075d9d0 0x5645b0713420 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fafa81518e0 tx=0x7faf98014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0756d1878c782b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7faf96000640 1 -- 172.21.15.87:0/106611355 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98014d50 con 0x5645b075d9d0 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7faf96000640 1 -- 172.21.15.87:0/106611355 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9801b050 con 0x5645b075d9d0 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7faf96000640 1 -- 172.21.15.87:0/106611355 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98004250 con 0x5645b075d9d0 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- 172.21.15.87:0/106611355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075d9d0 msgr2=0x5645b0713420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075d9d0 0x5645b0713420 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fafa81518e0 tx=0x7faf98014000 comp rx=0 tx=0).stop 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- 172.21.15.87:0/106611355 shutdown_connections 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b075e2f0 0x5645b0713960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075d9d0 0x5645b0713420 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/106611355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b07140a0 0x5645b075cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- 172.21.15.87:0/106611355 >> 172.21.15.87:0/106611355 conn(0x5645b0710020 msgr2=0x5645b075c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- 172.21.15.87:0/106611355 shutdown_connections 2024-04-25T10:21:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- 172.21.15.87:0/106611355 wait complete. 2024-04-25T10:21:09.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- start start 2024-04-25T10:21:09.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 Processor -- start 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b07c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0770380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b0560300 con 0x5645b075e2f0 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b04fbe00 con 0x5645b07140a0 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645b0565620 con 0x5645b075d9d0 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafb3bdf000 10 client.0 _ll_register_callbacks cb 0x5645b06df040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0770380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafade00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b07c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0770380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36322/0 (socket says 172.21.15.87:36322) 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 -- 172.21.15.87:0/1526414485 learned_addr learned my addr 172.21.15.87:0/1526414485 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 -- 172.21.15.87:0/1526414485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 msgr2=0x5645b07c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b07c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 -- 172.21.15.87:0/1526414485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 msgr2=0x5645b07c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 -- 172.21.15.87:0/1526414485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645b05028e0 con 0x5645b075e2f0 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7fafae800640 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0770380 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fafa400ec10 tx=0x7fafa4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7c320f4787282d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7faf8b400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.071+0000 7faf8b400640 1 -- 172.21.15.87:0/1526414485 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4023050 con 0x5645b075e2f0 2024-04-25T10:21:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 1 -- 172.21.15.87:0/1526414485 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fafa40050d0 con 0x5645b075e2f0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf9801b050 con 0x5645b075e2f0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 1 -- 172.21.15.87:0/1526414485 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafa4042050 con 0x5645b075e2f0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 10 client.8733 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5645b05ba8f0 con 0x5645b075e2f0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 1 -- 172.21.15.87:0/1526414485 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9801b050 con 0x5645b075e2f0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 1 -- 172.21.15.87:0/1526414485 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafa4045020 con 0x5645b075e2f0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 10 client.8733 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 10 client.8733 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 20 client.8733 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b0770ce0 0x5645b07710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 4 client.8733 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5645b056e530 con 0x5645b0770ce0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafade00640 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b0770ce0 0x5645b07710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafade00640 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b0770ce0 0x5645b07710f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b91d74e1956ccb8 server_cookie=aa4b86f9dfe9ae23 in_seq=0 out_seq=0 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 10 client.8733 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 1 -- 172.21.15.87:0/1526414485 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x5645b056e530 con 0x5645b0770ce0 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 10 client.8733 handle_command_reply: tid=0 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 client.8733 shutdown 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 msgr2=0x5645b0770380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0770380 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fafa400ec10 tx=0x7fafa4012000 comp rx=0 tx=0).stop 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 shutdown_connections 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5645b0770ce0 0x5645b07710f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645b075e2f0 0x5645b0770380 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5645b075d9d0 0x5645b07c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 --2- 172.21.15.87:0/1526414485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5645b07140a0 0x5645b07c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7faf8b400640 0 client.8733 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 >> 172.21.15.87:0/1526414485 conn(0x5645b0710020 msgr2=0x5645b075c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 shutdown_connections 2024-04-25T10:21:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 1 -- 172.21.15.87:0/1526414485 wait complete. 2024-04-25T10:21:09.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb3bdf000 20 client.8733 trim_cache size 0 max 16384 2024-04-25T10:21:09.082 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:09.082 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:21:09.082 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:21:09.082 INFO:teuthology.orchestra.run.smithi087.stdout: "88487972-2d40-452b-a934-070918dfcc03": { 2024-04-25T10:21:09.082 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:21:09.082 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "88487972-2d40-452b-a934-070918dfcc03", 2024-04-25T10:21:09.083 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive" 2024-04-25T10:21:09.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:09.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:09.083 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf8c066f60 msgr2=0x7faf8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf8c066f60 0x7faf8c069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fafa80cc230 tx=0x7faf98009190 comp rx=0 tx=0).stop 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa8070350 msgr2=0x7fafa8155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa8070350 0x7fafa8155f20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faf9c019130 tx=0x7faf9c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 shutdown_connections 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf8c066f60 0x7faf8c069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafa815bac0 0x7fafa8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafa80aef80 0x7fafa813b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 --2- 172.21.15.87:0/3559130158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa8070350 0x7fafa8155f20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 >> 172.21.15.87:0/3559130158 conn(0x7fafa80669a0 msgr2=0x7fafa8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 shutdown_connections 2024-04-25T10:21:09.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.075+0000 7fafb0800640 1 -- 172.21.15.87:0/3559130158 wait complete. 2024-04-25T10:21:09.096 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:09.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.227+0000 7fb3b8000640 1 Processor -- start 2024-04-25T10:21:09.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.227+0000 7fb3b8000640 1 -- start start 2024-04-25T10:21:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b8000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3b00cfe70 0x7fb3b015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b8000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b0071050 con 0x7fb3b015b740 2024-04-25T10:21:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b8000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b015d040 con 0x7fb3b00cfe70 2024-04-25T10:21:09.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00712f0 con 0x7fb3b01550b0 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b4c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3b00cfe70 0x7fb3b015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b4c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3b00cfe70 0x7fb3b015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:52960/0 (socket says 172.21.15.87:52960) 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b4c00640 1 -- 172.21.15.87:0/1394762946 learned_addr learned my addr 172.21.15.87:0/1394762946 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b5600640 1 --2- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b4c00640 1 -- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 msgr2=0x7fb3b00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b4c00640 1 --2- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b4c00640 1 -- 172.21.15.87:0/1394762946 >> v1:172.21.15.169:6789/0 conn(0x7fb3b015b740 legacy=0x7fb3b006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.231+0000 7fb3b4c00640 1 -- 172.21.15.87:0/1394762946 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00715d0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b4c00640 1 --2- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b00cfe70 0x7fb3b015b200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009870 tx=0x7fb39c01e770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57e5a029e981394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3afe00640 1 -- 172.21.15.87:0/1394762946 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c0ab650 con 0x7fb3b00cfe70 2024-04-25T10:21:09.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3afe00640 1 -- 172.21.15.87:0/1394762946 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c0ab7f0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3afe00640 1 -- 172.21.15.87:0/1394762946 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c0abbc0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 -- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b00cfe70 msgr2=0x7fb3b015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 --2- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b00cfe70 0x7fb3b015b200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009870 tx=0x7fb39c01e770 comp rx=0 tx=0).stop 2024-04-25T10:21:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 -- 172.21.15.87:0/1394762946 shutdown_connections 2024-04-25T10:21:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 --2- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b00cfe70 0x7fb3b015b200 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 --2- 172.21.15.87:0/1394762946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 -- 172.21.15.87:0/1394762946 >> 172.21.15.87:0/1394762946 conn(0x7fb3b0066b70 msgr2=0x7fb3b0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 -- 172.21.15.87:0/1394762946 shutdown_connections 2024-04-25T10:21:09.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.235+0000 7fb3b8000640 1 -- 172.21.15.87:0/1394762946 wait complete. 2024-04-25T10:21:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 Processor -- start 2024-04-25T10:21:09.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 -- start start 2024-04-25T10:21:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b00cfe70 0x7fb3b014b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b0157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b015b740 0x7fb3b00b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b015bcd0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b015bb50 con 0x7fb3b015b740 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b015c030 con 0x7fb3b01550b0 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b00cfe70 0x7fb3b014b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b00cfe70 0x7fb3b014b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36336/0 (socket says 172.21.15.87:36336) 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 -- 172.21.15.87:0/3622403522 learned_addr learned my addr 172.21.15.87:0/3622403522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b4c00640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b0157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b6000640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b015b740 0x7fb3b00b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 -- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 msgr2=0x7fb3b0157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b0157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 -- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b015b740 msgr2=0x7fb3b00b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b015b740 0x7fb3b00b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 -- 172.21.15.87:0/3622403522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b014bac0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b5600640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b00cfe70 0x7fb3b014b580 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e930 tx=0x7fb3a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3ad600640 1 -- 172.21.15.87:0/3622403522 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017070 con 0x7fb3b00cfe70 2024-04-25T10:21:09.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3ad600640 1 -- 172.21.15.87:0/3622403522 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a400ca00 con 0x7fb3b00cfe70 2024-04-25T10:21:09.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3ad600640 1 -- 172.21.15.87:0/3622403522 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a400cdb0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b0155630 con 0x7fb3b00cfe70 2024-04-25T10:21:09.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b0155bf0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.239+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b004ef30 con 0x7fb3b00cfe70 2024-04-25T10:21:09.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.243+0000 7fb3ad600640 1 -- 172.21.15.87:0/3622403522 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3a401a070 con 0x7fb3b00cfe70 2024-04-25T10:21:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.243+0000 7fb3ad600640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb390066f60 0x7fb390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.243+0000 7fb3ad600640 1 -- 172.21.15.87:0/3622403522 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb3a4085080 con 0x7fb3b00cfe70 2024-04-25T10:21:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.243+0000 7fb3b4c00640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb390066f60 0x7fb390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.243+0000 7fb3b4c00640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb390066f60 0x7fb390069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0b0000 tx=0x7fb39c0b5040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.247+0000 7fb3ad600640 1 -- 172.21.15.87:0/3622403522 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3b014bac0 con 0x7fb3b00cfe70 2024-04-25T10:21:09.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.435+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3b00cc180 con 0x7fb3b00cfe70 2024-04-25T10:21:09.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.435+0000 7fb3ad600640 1 -- 172.21.15.87:0/3622403522 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb3a4055a20 con 0x7fb3b00cfe70 2024-04-25T10:21:09.439 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:09.439 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:09.439 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb390066f60 msgr2=0x7fb390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb390066f60 0x7fb390069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0b0000 tx=0x7fb39c0b5040 comp rx=0 tx=0).stop 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b00cfe70 msgr2=0x7fb3b014b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b00cfe70 0x7fb3b014b580 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e930 tx=0x7fb3a400edf0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 shutdown_connections 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb390066f60 0x7fb390069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b015b740 0x7fb3b00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b01550b0 0x7fb3b0157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 --2- 172.21.15.87:0/3622403522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b00cfe70 0x7fb3b014b580 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 >> 172.21.15.87:0/3622403522 conn(0x7fb3b0066b70 msgr2=0x7fb3b015d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 shutdown_connections 2024-04-25T10:21:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.439+0000 7fb3b8000640 1 -- 172.21.15.87:0/3622403522 wait complete. 2024-04-25T10:21:09.460 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:09.460 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:09.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.591+0000 7f9d86a00640 1 Processor -- start 2024-04-25T10:21:09.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.591+0000 7f9d86a00640 1 -- start start 2024-04-25T10:21:09.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.591+0000 7f9d86a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d801550b0 0x7f9d800cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.591+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d8015b910 0x7f9d800b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.591+0000 7f9d86a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d80071050 con 0x7f9d800cfe70 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.591+0000 7f9d86a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8015bda0 con 0x7f9d801550b0 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.591+0000 7f9d86a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015d040 con 0x7f9d8015b910 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d77400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9d800cfe70 0x7f9d8015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37884/0 (socket says 172.21.15.87:37884) 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d77400640 1 -- 172.21.15.87:0/3756161586 learned_addr learned my addr 172.21.15.87:0/3756161586 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7fe00640 1 --2- 172.21.15.87:0/3756161586 >> v2:172.21.15.169:3300/0 conn(0x7f9d801550b0 0x7f9d800cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d84a00640 1 --2- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d8015b910 0x7f9d800b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7f400640 1 -- 172.21.15.87:0/3756161586 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661896002 0 0) 0x7f9d80071050 con 0x7f9d800cfe70 2024-04-25T10:21:09.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7f400640 1 -- 172.21.15.87:0/3756161586 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d64003530 con 0x7f9d800cfe70 2024-04-25T10:21:09.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7fe00640 1 -- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d8015b910 msgr2=0x7f9d800b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7fe00640 1 --2- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d8015b910 0x7f9d800b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7fe00640 1 -- 172.21.15.87:0/3756161586 >> v1:172.21.15.87:6789/0 conn(0x7f9d800cfe70 legacy=0x7f9d8015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7fe00640 1 -- 172.21.15.87:0/3756161586 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800712f0 con 0x7f9d801550b0 2024-04-25T10:21:09.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7fe00640 1 --2- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d801550b0 0x7f9d800cf930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009dc0 tx=0x7f9d7001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e1d25299e9541e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7f400640 1 -- 172.21.15.87:0/3756161586 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d700b2070 con 0x7f9d801550b0 2024-04-25T10:21:09.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7f400640 1 -- 172.21.15.87:0/3756161586 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d700a5900 con 0x7f9d801550b0 2024-04-25T10:21:09.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.595+0000 7f9d7f400640 1 -- 172.21.15.87:0/3756161586 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d700a5cd0 con 0x7f9d801550b0 2024-04-25T10:21:09.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d801550b0 msgr2=0x7f9d800cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 --2- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d801550b0 0x7f9d800cf930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009dc0 tx=0x7f9d7001c950 comp rx=0 tx=0).stop 2024-04-25T10:21:09.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- 172.21.15.87:0/3756161586 shutdown_connections 2024-04-25T10:21:09.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 --2- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d8015b910 0x7f9d800b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 --2- 172.21.15.87:0/3756161586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d801550b0 0x7f9d800cf930 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- 172.21.15.87:0/3756161586 >> 172.21.15.87:0/3756161586 conn(0x7f9d80066b70 msgr2=0x7f9d80066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- 172.21.15.87:0/3756161586 shutdown_connections 2024-04-25T10:21:09.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- 172.21.15.87:0/3756161586 wait complete. 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 Processor -- start 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- start start 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d800cfe70 0x7f9d800cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d801550b0 0x7f9d8014a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d8015b910 0x7f9d8014dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015c030 con 0x7f9d801550b0 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800720b0 con 0x7f9d8015b910 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d86a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015c1b0 con 0x7f9d800cfe70 2024-04-25T10:21:09.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d801550b0 0x7f9d8014a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d801550b0 0x7f9d8014a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36338/0 (socket says 172.21.15.87:36338) 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 -- 172.21.15.87:0/2923357540 learned_addr learned my addr 172.21.15.87:0/2923357540 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d7fe00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d800cfe70 0x7f9d800cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d84a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d8015b910 0x7f9d8014dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 -- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d800cfe70 msgr2=0x7f9d800cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d800cfe70 0x7f9d800cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 -- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d8015b910 msgr2=0x7f9d8014dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d8015b910 0x7f9d8014dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d77400640 1 -- 172.21.15.87:0/2923357540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8014f0b0 con 0x7f9d801550b0 2024-04-25T10:21:09.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.599+0000 7f9d84a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d8015b910 0x7f9d8014dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:09.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d77400640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d801550b0 0x7f9d8014a1b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c450 tx=0x7f9d6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d7cc00640 1 -- 172.21.15.87:0/2923357540 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68017070 con 0x7f9d801550b0 2024-04-25T10:21:09.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d7cc00640 1 -- 172.21.15.87:0/2923357540 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d68005cf0 con 0x7f9d801550b0 2024-04-25T10:21:09.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d800cc7f0 con 0x7f9d801550b0 2024-04-25T10:21:09.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d7cc00640 1 -- 172.21.15.87:0/2923357540 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68007290 con 0x7f9d801550b0 2024-04-25T10:21:09.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8014f630 con 0x7f9d801550b0 2024-04-25T10:21:09.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8004ef30 con 0x7f9d801550b0 2024-04-25T10:21:09.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d7cc00640 1 -- 172.21.15.87:0/2923357540 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d6801a070 con 0x7f9d801550b0 2024-04-25T10:21:09.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d7cc00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f90 0x7f9d5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d7cc00640 1 -- 172.21.15.87:0/2923357540 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9d68084420 con 0x7f9d801550b0 2024-04-25T10:21:09.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.603+0000 7f9d7fe00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f90 0x7f9d5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.607+0000 7f9d7fe00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f90 0x7f9d5c069450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9d700af000 tx=0x7f9d700a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.607+0000 7f9d7cc00640 1 -- 172.21.15.87:0/2923357540 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d68054d00 con 0x7f9d801550b0 2024-04-25T10:21:09.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.823+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9d80067060 con 0x7f9d801550b0 2024-04-25T10:21:09.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d7cc00640 1 -- 172.21.15.87:0/2923357540 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1264 (secure 0 0 0) 0x7f9d68058da0 con 0x7f9d801550b0 2024-04-25T10:21:09.828 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:09.828 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":551,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":980,"data_bytes":1005033250,"bytes_used":2428493824,"bytes_avail":770665619456,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":63441263,"read_op_per_sec":0,"write_op_per_sec":63},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T10:20:35.229274+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f90 msgr2=0x7f9d5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f90 0x7f9d5c069450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9d700af000 tx=0x7f9d700a7070 comp rx=0 tx=0).stop 2024-04-25T10:21:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d801550b0 msgr2=0x7f9d8014a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d801550b0 0x7f9d8014a1b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c450 tx=0x7f9d6800c910 comp rx=0 tx=0).stop 2024-04-25T10:21:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 shutdown_connections 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9d5c066f90 0x7f9d5c069450 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d8015b910 0x7f9d8014dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d801550b0 0x7f9d8014a1b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 --2- 172.21.15.87:0/2923357540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d800cfe70 0x7f9d800cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 >> 172.21.15.87:0/2923357540 conn(0x7f9d80066b70 msgr2=0x7f9d800b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 shutdown_connections 2024-04-25T10:21:09.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:09.827+0000 7f9d86a00640 1 -- 172.21.15.87:0/2923357540 wait complete. 2024-04-25T10:21:09.848 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:21:10.849 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:10.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.991+0000 7f2109000640 1 Processor -- start 2024-04-25T10:21:10.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.991+0000 7f2109000640 1 -- start start 2024-04-25T10:21:10.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2109000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21041580d0 0x7f210406df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2109000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21040d04b0 0x7f2104161a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2109000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210415e4f0 con 0x7f21040af1b0 2024-04-25T10:21:10.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2109000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210415f100 con 0x7f21040d04b0 2024-04-25T10:21:10.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2109000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210406e4a0 con 0x7f21041580d0 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21041580d0 0x7f210406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21040d04b0 0x7f2104161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21040d04b0 0x7f2104161a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:52986/0 (socket says 172.21.15.87:52986) 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103e00640 1 -- 172.21.15.87:0/1830316527 learned_addr learned my addr 172.21.15.87:0/1830316527 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21041580d0 0x7f210406df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35558/0 (socket says 172.21.15.87:35558) 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103e00640 1 -- 172.21.15.87:0/1830316527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21041580d0 msgr2=0x7f210406df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103e00640 1 --2- 172.21.15.87:0/1830316527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21041580d0 0x7f210406df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103e00640 1 -- 172.21.15.87:0/1830316527 >> v1:172.21.15.169:6789/0 conn(0x7f21040af1b0 legacy=0x7f21040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2103e00640 1 -- 172.21.15.87:0/1830316527 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210415e7d0 con 0x7f21040d04b0 2024-04-25T10:21:10.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.995+0000 7f2102a00640 1 -- 172.21.15.87:0/1830316527 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869696600 0 0) 0x7f210415e4f0 con 0x7f21040af1b0 2024-04-25T10:21:11.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2103e00640 1 --2- 172.21.15.87:0/1830316527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040d04b0 0x7f2104161a40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009e50 tx=0x7f20fc01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8c6761c57dc7af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2102a00640 1 -- 172.21.15.87:0/1830316527 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0a57a0 con 0x7f21040d04b0 2024-04-25T10:21:11.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2102a00640 1 -- 172.21.15.87:0/1830316527 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc0a5940 con 0x7f21040d04b0 2024-04-25T10:21:11.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2102a00640 1 -- 172.21.15.87:0/1830316527 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0a5d10 con 0x7f21040d04b0 2024-04-25T10:21:11.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 -- 172.21.15.87:0/1830316527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040d04b0 msgr2=0x7f2104161a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 --2- 172.21.15.87:0/1830316527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040d04b0 0x7f2104161a40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009e50 tx=0x7f20fc01c970 comp rx=0 tx=0).stop 2024-04-25T10:21:11.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 -- 172.21.15.87:0/1830316527 shutdown_connections 2024-04-25T10:21:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 --2- 172.21.15.87:0/1830316527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040d04b0 0x7f2104161a40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 --2- 172.21.15.87:0/1830316527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21041580d0 0x7f210406df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 -- 172.21.15.87:0/1830316527 >> 172.21.15.87:0/1830316527 conn(0x7f210406b4e0 msgr2=0x7f21040b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 -- 172.21.15.87:0/1830316527 shutdown_connections 2024-04-25T10:21:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 -- 172.21.15.87:0/1830316527 wait complete. 2024-04-25T10:21:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:10.999+0000 7f2109000640 1 Processor -- start 2024-04-25T10:21:11.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 -- start start 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040af1b0 0x7f210413b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21040d04b0 0x7f2104157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21041580d0 0x7f21040b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210415e260 con 0x7f21041580d0 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210415e0e0 con 0x7f21040af1b0 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210416aa70 con 0x7f21040d04b0 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21041580d0 0x7f21040b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21041580d0 0x7f21040b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36350/0 (socket says 172.21.15.87:36350) 2024-04-25T10:21:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 -- 172.21.15.87:0/1173977801 learned_addr learned my addr 172.21.15.87:0/1173977801 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f20fbe00640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21040d04b0 0x7f2104157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103400640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040af1b0 0x7f210413b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 -- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21040d04b0 msgr2=0x7f2104157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21040d04b0 0x7f2104157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 -- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040af1b0 msgr2=0x7f210413b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040af1b0 0x7f210413b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 -- 172.21.15.87:0/1173977801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210413b890 con 0x7f21041580d0 2024-04-25T10:21:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2103e00640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21041580d0 0x7f21040b0de0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0af040 tx=0x7f20fc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f20fb400640 1 -- 172.21.15.87:0/1173977801 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0bc030 con 0x7f21041580d0 2024-04-25T10:21:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f20fb400640 1 -- 172.21.15.87:0/1173977801 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc0b8470 con 0x7f21041580d0 2024-04-25T10:21:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f20fb400640 1 -- 172.21.15.87:0/1173977801 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0b8820 con 0x7f21041580d0 2024-04-25T10:21:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21040b1950 con 0x7f21041580d0 2024-04-25T10:21:11.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21040b1f10 con 0x7f21041580d0 2024-04-25T10:21:11.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.003+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2104075120 con 0x7f21041580d0 2024-04-25T10:21:11.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.007+0000 7f20fb400640 1 -- 172.21.15.87:0/1173977801 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20fc0bf070 con 0x7f21041580d0 2024-04-25T10:21:11.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.007+0000 7f20fb400640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.007+0000 7f20fb400640 1 -- 172.21.15.87:0/1173977801 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f20fc11ffa0 con 0x7f21041580d0 2024-04-25T10:21:11.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.007+0000 7f2103400640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.007+0000 7f2103400640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20e0066f60 0x7f20e0069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009b70 tx=0x7f20f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.011+0000 7f20fb400640 1 -- 172.21.15.87:0/1173977801 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20fc016620 con 0x7f21041580d0 2024-04-25T10:21:11.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.195+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21040728a0 con 0x7f21041580d0 2024-04-25T10:21:11.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.195+0000 7f20fb400640 1 -- 172.21.15.87:0/1173977801 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f20fc0f1120 con 0x7f21041580d0 2024-04-25T10:21:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:11.198 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:11.199 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:11.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20e0066f60 msgr2=0x7f20e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20e0066f60 0x7f20e0069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009b70 tx=0x7f20f4008040 comp rx=0 tx=0).stop 2024-04-25T10:21:11.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21041580d0 msgr2=0x7f21040b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21041580d0 0x7f21040b0de0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0af040 tx=0x7f20fc019710 comp rx=0 tx=0).stop 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 shutdown_connections 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21041580d0 0x7f21040b0de0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21040d04b0 0x7f2104157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 --2- 172.21.15.87:0/1173977801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21040af1b0 0x7f210413b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 >> 172.21.15.87:0/1173977801 conn(0x7f210406b4e0 msgr2=0x7f210415f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 shutdown_connections 2024-04-25T10:21:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.199+0000 7f2109000640 1 -- 172.21.15.87:0/1173977801 wait complete. 2024-04-25T10:21:11.219 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:11.219 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:11.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.351+0000 7f79bf000640 1 Processor -- start 2024-04-25T10:21:11.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.351+0000 7f79bf000640 1 -- start start 2024-04-25T10:21:11.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b8071060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bf000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b80b34f0 con 0x7f79b80ad840 2024-04-25T10:21:11.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bf000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b80b37a0 con 0x7f79b81582f0 2024-04-25T10:21:11.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80b3a80 con 0x7f79b8070be0 2024-04-25T10:21:11.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79be600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f79b80ad840 0x7f79b80adcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37892/0 (socket says 172.21.15.87:37892) 2024-04-25T10:21:11.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79be600640 1 -- 172.21.15.87:0/1635343605 learned_addr learned my addr 172.21.15.87:0/1635343605 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:11.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bd200640 1 --2- 172.21.15.87:0/1635343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b8071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257202694 0 0) 0x7f79b80b34f0 con 0x7f79b80ad840 2024-04-25T10:21:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a0003530 con 0x7f79b80ad840 2024-04-25T10:21:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924383047 0 0) 0x7f79b80b37a0 con 0x7f79b81582f0 2024-04-25T10:21:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b80b34f0 con 0x7f79b81582f0 2024-04-25T10:21:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2043980430 0 0) 0x7f79a0003530 con 0x7f79b80ad840 2024-04-25T10:21:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bd200640 1 -- 172.21.15.87:0/1635343605 >> v1:172.21.15.169:6789/0 conn(0x7f79b81582f0 legacy=0x7f79b80b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bd200640 1 -- 172.21.15.87:0/1635343605 >> v1:172.21.15.87:6789/0 conn(0x7f79b80ad840 legacy=0x7f79b80adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.355+0000 7f79bd200640 1 -- 172.21.15.87:0/1635343605 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80b3d60 con 0x7f79b8070be0 2024-04-25T10:21:11.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bd200640 1 --2- 172.21.15.87:0/1635343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b8071060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79a40097f0 tx=0x7f79a401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73b7ac8e70a28d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40b0070 con 0x7f79b8070be0 2024-04-25T10:21:11.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a40ab400 con 0x7f79b8070be0 2024-04-25T10:21:11.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79b7e00640 1 -- 172.21.15.87:0/1635343605 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0001a00 con 0x7f79b8070be0 2024-04-25T10:21:11.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 -- 172.21.15.87:0/1635343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 msgr2=0x7f79b8071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 --2- 172.21.15.87:0/1635343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b8071060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79a40097f0 tx=0x7f79a401c400 comp rx=0 tx=0).stop 2024-04-25T10:21:11.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 -- 172.21.15.87:0/1635343605 shutdown_connections 2024-04-25T10:21:11.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 --2- 172.21.15.87:0/1635343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b8071060 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 -- 172.21.15.87:0/1635343605 >> 172.21.15.87:0/1635343605 conn(0x7f79b80b1ae0 msgr2=0x7f79b80b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 -- 172.21.15.87:0/1635343605 shutdown_connections 2024-04-25T10:21:11.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 -- 172.21.15.87:0/1635343605 wait complete. 2024-04-25T10:21:11.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 Processor -- start 2024-04-25T10:21:11.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.359+0000 7f79bf000640 1 -- start start 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b80ad840 0x7f79b815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b81582f0 0x7f79b814a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b816ba80 con 0x7f79b81582f0 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b815e4a0 con 0x7f79b80ad840 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b816bc00 con 0x7f79b8070be0 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b81582f0 0x7f79b814a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b81582f0 0x7f79b814a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36352/0 (socket says 172.21.15.87:36352) 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 -- 172.21.15.87:0/1621702890 learned_addr learned my addr 172.21.15.87:0/1621702890 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bdc00640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bd200640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b80ad840 0x7f79b815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 -- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 msgr2=0x7f79b81586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b81586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 -- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b80ad840 msgr2=0x7f79b815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b80ad840 0x7f79b815ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 -- 172.21.15.87:0/1621702890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b815b4c0 con 0x7f79b81582f0 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79be600640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b81582f0 0x7f79b814a7e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f79b0001f40 tx=0x7f79b000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79b5600640 1 -- 172.21.15.87:0/1621702890 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0017070 con 0x7f79b81582f0 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79b5600640 1 -- 172.21.15.87:0/1621702890 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0012400 con 0x7f79b81582f0 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79b5600640 1 -- 172.21.15.87:0/1621702890 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0012830 con 0x7f79b81582f0 2024-04-25T10:21:11.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b814e000 con 0x7f79b81582f0 2024-04-25T10:21:11.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b8159450 con 0x7f79b81582f0 2024-04-25T10:21:11.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b8074520 con 0x7f79b81582f0 2024-04-25T10:21:11.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.363+0000 7f79b5600640 1 -- 172.21.15.87:0/1621702890 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79b001a070 con 0x7f79b81582f0 2024-04-25T10:21:11.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.367+0000 7f79b5600640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.367+0000 7f79b5600640 1 -- 172.21.15.87:0/1621702890 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f79b0084fc0 con 0x7f79b81582f0 2024-04-25T10:21:11.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.367+0000 7f79bdc00640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.367+0000 7f79bdc00640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0011f0 tx=0x7f79ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.371+0000 7f79b5600640 1 -- 172.21.15.87:0/1621702890 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79b0056020 con 0x7f79b81582f0 2024-04-25T10:21:11.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.579+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f79b8075120 con 0x7f79b81582f0 2024-04-25T10:21:11.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.579+0000 7f79b5600640 1 -- 172.21.15.87:0/1621702890 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1264 (secure 0 0 0) 0x7f79b0055920 con 0x7f79b81582f0 2024-04-25T10:21:11.583 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:11.583 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":553,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":980,"data_bytes":1005033250,"bytes_used":2428493824,"bytes_avail":770665619456,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":45681195,"read_op_per_sec":0,"write_op_per_sec":46},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T10:20:35.229274+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 msgr2=0x7f7998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0011f0 tx=0x7f79ac009040 comp rx=0 tx=0).stop 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b81582f0 msgr2=0x7f79b814a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b81582f0 0x7f79b814a7e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f79b0001f40 tx=0x7f79b000a430 comp rx=0 tx=0).stop 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 shutdown_connections 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b81582f0 0x7f79b814a7e0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b80ad840 0x7f79b815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 --2- 172.21.15.87:0/1621702890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8070be0 0x7f79b81586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 >> 172.21.15.87:0/1621702890 conn(0x7f79b80b1ae0 msgr2=0x7f79b80b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 shutdown_connections 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:11.583+0000 7f79bf000640 1 -- 172.21.15.87:0/1621702890 wait complete. 2024-04-25T10:21:11.603 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:21:12.604 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:12.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08cf000640 1 Processor -- start 2024-04-25T10:21:12.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08cf000640 1 -- start start 2024-04-25T10:21:12.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08cf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c80d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08cf000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08c80d0750 0x7f08c80d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08cf000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c80d2490 con 0x7f08c80d1ce0 2024-04-25T10:21:12.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08cf000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c80d2770 con 0x7f08c80d0750 2024-04-25T10:21:12.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08cf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80d2a50 con 0x7f08c815df00 2024-04-25T10:21:12.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c80d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08c80d0750 0x7f08c80d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c80d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35578/0 (socket says 172.21.15.87:35578) 2024-04-25T10:21:12.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7e00640 1 -- 172.21.15.87:0/899858816 learned_addr learned my addr 172.21.15.87:0/899858816 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:12.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7e00640 1 -- 172.21.15.87:0/899858816 >> v2:172.21.15.169:3300/0 conn(0x7f08c80d0750 msgr2=0x7f08c80d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7e00640 1 --2- 172.21.15.87:0/899858816 >> v2:172.21.15.169:3300/0 conn(0x7f08c80d0750 0x7f08c80d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7e00640 1 -- 172.21.15.87:0/899858816 >> v1:172.21.15.169:6789/0 conn(0x7f08c80d1ce0 legacy=0x7f08c80b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7e00640 1 -- 172.21.15.87:0/899858816 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80d2d30 con 0x7f08c815df00 2024-04-25T10:21:12.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.751+0000 7f08c7400640 1 --2- 172.21.15.87:0/899858816 >> v2:172.21.15.169:3300/0 conn(0x7f08c80d0750 0x7f08c80d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:12.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08c7e00640 1 --2- 172.21.15.87:0/899858816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c80d0210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc512dddfa1797f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08c6a00640 1 -- 172.21.15.87:0/899858816 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80a9610 con 0x7f08c815df00 2024-04-25T10:21:12.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08c6a00640 1 -- 172.21.15.87:0/899858816 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b80a97b0 con 0x7f08c815df00 2024-04-25T10:21:12.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08c6a00640 1 -- 172.21.15.87:0/899858816 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80a9b80 con 0x7f08c815df00 2024-04-25T10:21:12.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 -- 172.21.15.87:0/899858816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 msgr2=0x7f08c80d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 --2- 172.21.15.87:0/899858816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c80d0210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b801c530 comp rx=0 tx=0).stop 2024-04-25T10:21:12.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 -- 172.21.15.87:0/899858816 shutdown_connections 2024-04-25T10:21:12.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 --2- 172.21.15.87:0/899858816 >> v2:172.21.15.169:3300/0 conn(0x7f08c80d0750 0x7f08c80d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 --2- 172.21.15.87:0/899858816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c80d0210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 -- 172.21.15.87:0/899858816 >> 172.21.15.87:0/899858816 conn(0x7f08c806b290 msgr2=0x7f08c80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 -- 172.21.15.87:0/899858816 shutdown_connections 2024-04-25T10:21:12.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.755+0000 7f08cf000640 1 -- 172.21.15.87:0/899858816 wait complete. 2024-04-25T10:21:12.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 Processor -- start 2024-04-25T10:21:12.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 -- start start 2024-04-25T10:21:12.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c80d0750 0x7f08c8146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c80d1ce0 0x7f08c81534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c8153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80d2270 con 0x7f08c80d1ce0 2024-04-25T10:21:12.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80d20f0 con 0x7f08c80d0750 2024-04-25T10:21:12.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c816ab70 con 0x7f08c815df00 2024-04-25T10:21:12.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c80d1ce0 0x7f08c81534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c80d1ce0 0x7f08c81534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36368/0 (socket says 172.21.15.87:36368) 2024-04-25T10:21:12.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 -- 172.21.15.87:0/2324673548 learned_addr learned my addr 172.21.15.87:0/2324673548 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7e00640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c80d0750 0x7f08c8146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 -- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 msgr2=0x7f08c8153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cd000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c8153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c8153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 -- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c80d0750 msgr2=0x7f08c8146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c80d0750 0x7f08c8146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 -- 172.21.15.87:0/2324673548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c81477b0 con 0x7f08c80d1ce0 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7e00640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c80d0750 0x7f08c8146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cd000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c8153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08c7400640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c80d1ce0 0x7f08c81534f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f08b400ec10 tx=0x7f08b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08bfe00640 1 -- 172.21.15.87:0/2324673548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4017070 con 0x7f08c80d1ce0 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08bfe00640 1 -- 172.21.15.87:0/2324673548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b400cc30 con 0x7f08c80d1ce0 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c8147a90 con 0x7f08c80d1ce0 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08bfe00640 1 -- 172.21.15.87:0/2324673548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4013470 con 0x7f08c80d1ce0 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c8148050 con 0x7f08c80d1ce0 2024-04-25T10:21:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.759+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c8148430 con 0x7f08c80d1ce0 2024-04-25T10:21:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.763+0000 7f08bfe00640 1 -- 172.21.15.87:0/2324673548 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08b401a070 con 0x7f08c80d1ce0 2024-04-25T10:21:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.763+0000 7f08bfe00640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a8066f60 0x7f08a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.763+0000 7f08bfe00640 1 -- 172.21.15.87:0/2324673548 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f08b4085480 con 0x7f08c80d1ce0 2024-04-25T10:21:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.763+0000 7f08c7e00640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a8066f60 0x7f08a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.763+0000 7f08c7e00640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a8066f60 0x7f08a8069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f08b80ae000 tx=0x7f08b80a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.767+0000 7f08bfe00640 1 -- 172.21.15.87:0/2324673548 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08c8148430 con 0x7f08c80d1ce0 2024-04-25T10:21:12.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.951+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08c804ef30 con 0x7f08c80d1ce0 2024-04-25T10:21:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.951+0000 7f08bfe00640 1 -- 172.21.15.87:0/2324673548 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f08b40563b0 con 0x7f08c80d1ce0 2024-04-25T10:21:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:12.955 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:12.955 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:12.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a8066f60 msgr2=0x7f08a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a8066f60 0x7f08a8069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f08b80ae000 tx=0x7f08b80a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:12.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c80d1ce0 msgr2=0x7f08c81534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c80d1ce0 0x7f08c81534f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f08b400ec10 tx=0x7f08b400c520 comp rx=0 tx=0).stop 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 shutdown_connections 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a8066f60 0x7f08a8069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c815df00 0x7f08c8153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c80d1ce0 0x7f08c81534f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 --2- 172.21.15.87:0/2324673548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c80d0750 0x7f08c8146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 >> 172.21.15.87:0/2324673548 conn(0x7f08c806b290 msgr2=0x7f08c815bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 shutdown_connections 2024-04-25T10:21:12.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:12.955+0000 7f08cf000640 1 -- 172.21.15.87:0/2324673548 wait complete. 2024-04-25T10:21:12.975 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:12.976 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.107+0000 7fa425e00640 1 Processor -- start 2024-04-25T10:21:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.107+0000 7fa425e00640 1 -- start start 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa425e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 0x7fa4200cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa425e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200d0250 con 0x7fa420070c20 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa425e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200ad840 con 0x7fa4200ae020 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa425e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42015bb90 con 0x7fa4200cf930 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa416000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa420070c20 0x7fa42015e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37894/0 (socket says 172.21.15.87:37894) 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa416000640 1 -- 172.21.15.87:0/1210849241 learned_addr learned my addr 172.21.15.87:0/1210849241 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41ea00640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 0x7fa4200cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150518689 0 0) 0x7fa4200d0250 con 0x7fa420070c20 2024-04-25T10:21:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa404003530 con 0x7fa420070c20 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465425097 0 0) 0x7fa404003530 con 0x7fa420070c20 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 msgr2=0x7fa4200cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 0x7fa4200cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 >> v1:172.21.15.169:6789/0 conn(0x7fa4200ae020 legacy=0x7fa4200b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42015e9f0 con 0x7fa420070c20 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fa408001680 con 0x7fa420070c20 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 >> v1:172.21.15.87:6789/0 conn(0x7fa420070c20 legacy=0x7fa42015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040039f0 0x7fa404003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4040043a0 0x7fa404004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 0x7fa4040050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42015bb90 con 0x7fa4040039f0 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40400b9d0 con 0x7fa4040043a0 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40400bbd0 con 0x7fa4200cf930 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41ea00640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040039f0 0x7fa404003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa416000640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4040043a0 0x7fa404004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41f400640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 0x7fa4040050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41ea00640 1 -- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 msgr2=0x7fa4040050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41ea00640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4200cf930 0x7fa4040050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41ea00640 1 -- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4040043a0 msgr2=0x7fa404004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41ea00640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4040043a0 0x7fa404004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.111+0000 7fa41ea00640 1 -- 172.21.15.87:0/1210849241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa404003530 con 0x7fa4040039f0 2024-04-25T10:21:13.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa41ea00640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040039f0 0x7fa404003dd0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ae0 tx=0x7fa41001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13064819b6ea896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100b2070 con 0x7fa4040039f0 2024-04-25T10:21:13.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4100a5660 con 0x7fa4040039f0 2024-04-25T10:21:13.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa41e000640 1 -- 172.21.15.87:0/1210849241 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100096b0 con 0x7fa4040039f0 2024-04-25T10:21:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa425e00640 1 -- 172.21.15.87:0/1210849241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040039f0 msgr2=0x7fa404003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa425e00640 1 --2- 172.21.15.87:0/1210849241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040039f0 0x7fa404003dd0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ae0 tx=0x7fa41001ca70 comp rx=0 tx=0).stop 2024-04-25T10:21:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa41f400640 1 -- 172.21.15.87:0/1210849241 reap_dead start 2024-04-25T10:21:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa425e00640 1 -- 172.21.15.87:0/1210849241 shutdown_connections 2024-04-25T10:21:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa425e00640 1 -- 172.21.15.87:0/1210849241 >> 172.21.15.87:0/1210849241 conn(0x7fa420066b70 msgr2=0x7fa420066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa425e00640 1 -- 172.21.15.87:0/1210849241 shutdown_connections 2024-04-25T10:21:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.115+0000 7fa425e00640 1 -- 172.21.15.87:0/1210849241 wait complete. 2024-04-25T10:21:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 Processor -- start 2024-04-25T10:21:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 -- start start 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4040039f0 0x7fa4200b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420150080 0x7fa420146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4200b39c0 0x7fa4200b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200adb00 con 0x7fa4200b39c0 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42015efb0 con 0x7fa420150080 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42006e130 con 0x7fa4040039f0 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4200b39c0 0x7fa4200b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4200b39c0 0x7fa4200b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36372/0 (socket says 172.21.15.87:36372) 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 -- 172.21.15.87:0/2308477361 learned_addr learned my addr 172.21.15.87:0/2308477361 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa416000640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420150080 0x7fa420146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41ea00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4040039f0 0x7fa4200b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 -- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4040039f0 msgr2=0x7fa4200b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4040039f0 0x7fa4200b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 -- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420150080 msgr2=0x7fa420146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420150080 0x7fa420146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 -- 172.21.15.87:0/2308477361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200af2e0 con 0x7fa4200b39c0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa41f400640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4200b39c0 0x7fa4200b3e40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fa42006e790 tx=0x7fa418010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa417e00640 1 -- 172.21.15.87:0/2308477361 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41801b070 con 0x7fa4200b39c0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa417e00640 1 -- 172.21.15.87:0/2308477361 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa418003980 con 0x7fa4200b39c0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4200af5c0 con 0x7fa4200b39c0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa417e00640 1 -- 172.21.15.87:0/2308477361 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418003d70 con 0x7fa4200b39c0 2024-04-25T10:21:13.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4200afb80 con 0x7fa4200b39c0 2024-04-25T10:21:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.119+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42004ef30 con 0x7fa4200b39c0 2024-04-25T10:21:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.123+0000 7fa417e00640 1 -- 172.21.15.87:0/2308477361 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa41801e070 con 0x7fa4200b39c0 2024-04-25T10:21:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.123+0000 7fa417e00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa3fc066f90 0x7fa3fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.123+0000 7fa417e00640 1 -- 172.21.15.87:0/2308477361 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa4180878d0 con 0x7fa4200b39c0 2024-04-25T10:21:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.123+0000 7fa41ea00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa3fc066f90 0x7fa3fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.123+0000 7fa41ea00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa3fc066f90 0x7fa3fc069450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ae0 tx=0x7fa4100046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.127+0000 7fa417e00640 1 -- 172.21.15.87:0/2308477361 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa418058240 con 0x7fa4200b39c0 2024-04-25T10:21:13.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.339+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa420067470 con 0x7fa4200b39c0 2024-04-25T10:21:13.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.339+0000 7fa417e00640 1 -- 172.21.15.87:0/2308477361 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1265 (secure 0 0 0) 0x7fa41805c2e0 con 0x7fa4200b39c0 2024-04-25T10:21:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:13.342 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":555,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":1010,"data_bytes":1036786831,"bytes_used":2483736576,"bytes_avail":770610376704,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":35099841,"read_op_per_sec":0,"write_op_per_sec":35},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T10:20:35.229274+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa3fc066f90 msgr2=0x7fa3fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa3fc066f90 0x7fa3fc069450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ae0 tx=0x7fa4100046d0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4200b39c0 msgr2=0x7fa4200b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4200b39c0 0x7fa4200b3e40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fa42006e790 tx=0x7fa418010ea0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 shutdown_connections 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa3fc066f90 0x7fa3fc069450 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4200b39c0 0x7fa4200b3e40 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa420150080 0x7fa420146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 --2- 172.21.15.87:0/2308477361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4040039f0 0x7fa4200b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 >> 172.21.15.87:0/2308477361 conn(0x7fa420066b70 msgr2=0x7fa42015b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 shutdown_connections 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:13.343+0000 7fa425e00640 1 -- 172.21.15.87:0/2308477361 wait complete. 2024-04-25T10:21:13.363 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:21:14.364 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:14.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0211400640 1 Processor -- start 2024-04-25T10:21:14.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0211400640 1 -- start start 2024-04-25T10:21:14.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0211400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f020c0d0040 0x7f020c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0211400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c15b740 0x7f020c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0211400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c071050 con 0x7f020c1550b0 2024-04-25T10:21:14.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0211400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c15d040 con 0x7f020c0d0040 2024-04-25T10:21:14.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0211400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0712f0 con 0x7f020c15b740 2024-04-25T10:21:14.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0209e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f020c0d0040 0x7f020c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0209e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f020c0d0040 0x7f020c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36380/0 (socket says 172.21.15.87:36380) 2024-04-25T10:21:14.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f0209e00640 1 -- 172.21.15.87:0/3404292931 learned_addr learned my addr 172.21.15.87:0/3404292931 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:14.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.503+0000 7f020b200640 1 --2- 172.21.15.87:0/3404292931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c15b740 0x7f020c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209e00640 1 -- 172.21.15.87:0/3404292931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c15b740 msgr2=0x7f020c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209e00640 1 --2- 172.21.15.87:0/3404292931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c15b740 0x7f020c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209e00640 1 -- 172.21.15.87:0/3404292931 >> v1:172.21.15.169:6789/0 conn(0x7f020c1550b0 legacy=0x7f020c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209e00640 1 -- 172.21.15.87:0/3404292931 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0715d0 con 0x7f020c0d0040 2024-04-25T10:21:14.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209e00640 1 --2- 172.21.15.87:0/3404292931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c0d0040 0x7f020c15b200 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009870 tx=0x7f01f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb492541113ec1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209400640 1 -- 172.21.15.87:0/3404292931 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80aa610 con 0x7f020c0d0040 2024-04-25T10:21:14.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209400640 1 -- 172.21.15.87:0/3404292931 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f80af070 con 0x7f020c0d0040 2024-04-25T10:21:14.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0209400640 1 -- 172.21.15.87:0/3404292931 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80aa9d0 con 0x7f020c0d0040 2024-04-25T10:21:14.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 -- 172.21.15.87:0/3404292931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c0d0040 msgr2=0x7f020c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 --2- 172.21.15.87:0/3404292931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c0d0040 0x7f020c15b200 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009870 tx=0x7f01f801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 -- 172.21.15.87:0/3404292931 shutdown_connections 2024-04-25T10:21:14.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 --2- 172.21.15.87:0/3404292931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c15b740 0x7f020c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 --2- 172.21.15.87:0/3404292931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c0d0040 0x7f020c15b200 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 -- 172.21.15.87:0/3404292931 >> 172.21.15.87:0/3404292931 conn(0x7f020c066b70 msgr2=0x7f020c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 -- 172.21.15.87:0/3404292931 shutdown_connections 2024-04-25T10:21:14.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.507+0000 7f0211400640 1 -- 172.21.15.87:0/3404292931 wait complete. 2024-04-25T10:21:14.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 Processor -- start 2024-04-25T10:21:14.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 -- start start 2024-04-25T10:21:14.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c0d0040 0x7f020c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c1550b0 0x7f020c142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f020c15b740 0x7f020c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15bcd0 con 0x7f020c1550b0 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15bb50 con 0x7f020c15b740 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15c030 con 0x7f020c0d0040 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c1550b0 0x7f020c142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c1550b0 0x7f020c142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36394/0 (socket says 172.21.15.87:36394) 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 -- 172.21.15.87:0/3046153592 learned_addr learned my addr 172.21.15.87:0/3046153592 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f020a800640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c0d0040 0x7f020c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f020b200640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f020c15b740 0x7f020c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 -- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c0d0040 msgr2=0x7f020c1526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c0d0040 0x7f020c1526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 -- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f020c15b740 msgr2=0x7f020c0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f020c15b740 0x7f020c0b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 -- 172.21.15.87:0/3046153592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0af7c0 con 0x7f020c1550b0 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f020b200640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f020c15b740 0x7f020c0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0209e00640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c1550b0 0x7f020c142ad0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f01f80ac040 tx=0x7f01f801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0202a00640 1 -- 172.21.15.87:0/3046153592 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80bd5f0 con 0x7f020c1550b0 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0202a00640 1 -- 172.21.15.87:0/3046153592 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f80bd7b0 con 0x7f020c1550b0 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c0afaa0 con 0x7f020c1550b0 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0202a00640 1 -- 172.21.15.87:0/3046153592 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80bdba0 con 0x7f020c1550b0 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.511+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c0b0060 con 0x7f020c1550b0 2024-04-25T10:21:14.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.515+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c04ef30 con 0x7f020c1550b0 2024-04-25T10:21:14.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.515+0000 7f0202a00640 1 -- 172.21.15.87:0/3046153592 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01f80b33f0 con 0x7f020c1550b0 2024-04-25T10:21:14.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.515+0000 7f0202a00640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.515+0000 7f0202a00640 1 -- 172.21.15.87:0/3046153592 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01f811fc00 con 0x7f020c1550b0 2024-04-25T10:21:14.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.515+0000 7f020a800640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.515+0000 7f020a800640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f01fc005530 tx=0x7f01fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.519+0000 7f0202a00640 1 -- 172.21.15.87:0/3046153592 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01f80c3060 con 0x7f020c1550b0 2024-04-25T10:21:14.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.703+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f020c143010 con 0x7f020c1550b0 2024-04-25T10:21:14.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0202a00640 1 -- 172.21.15.87:0/3046153592 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f01f80a7070 con 0x7f020c1550b0 2024-04-25T10:21:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:14.708 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:14.708 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:14.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01ec066f60 msgr2=0x7f01ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f01fc005530 tx=0x7f01fc009040 comp rx=0 tx=0).stop 2024-04-25T10:21:14.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c1550b0 msgr2=0x7f020c142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c1550b0 0x7f020c142ad0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f01f80ac040 tx=0x7f01f801ad30 comp rx=0 tx=0).stop 2024-04-25T10:21:14.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 shutdown_connections 2024-04-25T10:21:14.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f020c15b740 0x7f020c0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f020c1550b0 0x7f020c142ad0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 --2- 172.21.15.87:0/3046153592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f020c0d0040 0x7f020c1526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 >> 172.21.15.87:0/3046153592 conn(0x7f020c066b70 msgr2=0x7f020c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 shutdown_connections 2024-04-25T10:21:14.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.707+0000 7f0211400640 1 -- 172.21.15.87:0/3046153592 wait complete. 2024-04-25T10:21:14.728 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:14.729 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa681800640 1 Processor -- start 2024-04-25T10:21:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa681800640 1 -- start start 2024-04-25T10:21:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa681800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa67c0d0040 0x7fa67c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa681800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67c071050 con 0x7fa67c1550b0 2024-04-25T10:21:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa681800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67c15d040 con 0x7fa67c15b740 2024-04-25T10:21:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa681800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c0712f0 con 0x7fa67c0d0040 2024-04-25T10:21:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa67aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa67c1550b0 0x7fa67c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37898/0 (socket says 172.21.15.87:37898) 2024-04-25T10:21:14.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa67aa00640 1 -- 172.21.15.87:0/3558411537 learned_addr learned my addr 172.21.15.87:0/3558411537 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:14.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa673e00640 1 --2- 172.21.15.87:0/3558411537 >> v2:172.21.15.87:3300/0 conn(0x7fa67c0d0040 0x7fa67c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa67a000640 1 -- 172.21.15.87:0/3558411537 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025509773 0 0) 0x7fa67c071050 con 0x7fa67c1550b0 2024-04-25T10:21:14.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa67a000640 1 -- 172.21.15.87:0/3558411537 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa660005740 con 0x7fa67c1550b0 2024-04-25T10:21:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa673e00640 1 -- 172.21.15.87:0/3558411537 >> v1:172.21.15.169:6789/0 conn(0x7fa67c15b740 legacy=0x7fa67c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa673e00640 1 -- 172.21.15.87:0/3558411537 >> v1:172.21.15.87:6789/0 conn(0x7fa67c1550b0 legacy=0x7fa67c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.863+0000 7fa673e00640 1 -- 172.21.15.87:0/3558411537 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c0715d0 con 0x7fa67c0d0040 2024-04-25T10:21:14.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa673e00640 1 --2- 172.21.15.87:0/3558411537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c0d0040 0x7fa67c15b200 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fa6640097c0 tx=0x7fa66401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30abb162ed67a4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa67a000640 1 -- 172.21.15.87:0/3558411537 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640b0070 con 0x7fa67c0d0040 2024-04-25T10:21:14.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa67a000640 1 -- 172.21.15.87:0/3558411537 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6640ab600 con 0x7fa67c0d0040 2024-04-25T10:21:14.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa67a000640 1 -- 172.21.15.87:0/3558411537 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640ab9d0 con 0x7fa67c0d0040 2024-04-25T10:21:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 -- 172.21.15.87:0/3558411537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c0d0040 msgr2=0x7fa67c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 --2- 172.21.15.87:0/3558411537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c0d0040 0x7fa67c15b200 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fa6640097c0 tx=0x7fa66401c470 comp rx=0 tx=0).stop 2024-04-25T10:21:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 -- 172.21.15.87:0/3558411537 shutdown_connections 2024-04-25T10:21:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 --2- 172.21.15.87:0/3558411537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c0d0040 0x7fa67c15b200 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 -- 172.21.15.87:0/3558411537 >> 172.21.15.87:0/3558411537 conn(0x7fa67c066b70 msgr2=0x7fa67c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 -- 172.21.15.87:0/3558411537 shutdown_connections 2024-04-25T10:21:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 -- 172.21.15.87:0/3558411537 wait complete. 2024-04-25T10:21:14.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 Processor -- start 2024-04-25T10:21:14.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.867+0000 7fa681800640 1 -- start start 2024-04-25T10:21:14.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa67c0d0040 0x7fa67c157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c1550b0 0x7fa67c0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa67c15b740 0x7fa67c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c15bcd0 con 0x7fa67c1550b0 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c15bb50 con 0x7fa67c15b740 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c06c090 con 0x7fa67c0d0040 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c1550b0 0x7fa67c0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c1550b0 0x7fa67c0cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36410/0 (socket says 172.21.15.87:36410) 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 -- 172.21.15.87:0/3035919790 learned_addr learned my addr 172.21.15.87:0/3035919790 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 -- 172.21.15.87:0/3035919790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa67c0d0040 msgr2=0x7fa67c157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa67c0d0040 0x7fa67c157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 -- 172.21.15.87:0/3035919790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa67c15b740 msgr2=0x7fa67c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa67c15b740 0x7fa67c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 -- 172.21.15.87:0/3035919790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c156aa0 con 0x7fa67c1550b0 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa673e00640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c1550b0 0x7fa67c0cb1d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa664019080 tx=0x7fa664019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa672a00640 1 -- 172.21.15.87:0/3035919790 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640ba070 con 0x7fa67c1550b0 2024-04-25T10:21:14.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa672a00640 1 -- 172.21.15.87:0/3035919790 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6640b43f0 con 0x7fa67c1550b0 2024-04-25T10:21:14.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c155a00 con 0x7fa67c1550b0 2024-04-25T10:21:14.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa672a00640 1 -- 172.21.15.87:0/3035919790 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640b0040 con 0x7fa67c1550b0 2024-04-25T10:21:14.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c155fc0 con 0x7fa67c1550b0 2024-04-25T10:21:14.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c04ef30 con 0x7fa67c1550b0 2024-04-25T10:21:14.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.871+0000 7fa672a00640 1 -- 172.21.15.87:0/3035919790 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6640bd070 con 0x7fa67c1550b0 2024-04-25T10:21:14.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.875+0000 7fa672a00640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa658066f60 0x7fa658069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.875+0000 7fa672a00640 1 -- 172.21.15.87:0/3035919790 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa66411e120 con 0x7fa67c1550b0 2024-04-25T10:21:14.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.875+0000 7fa67aa00640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa658066f60 0x7fa658069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.875+0000 7fa67aa00640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa658066f60 0x7fa658069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa66c007fd0 tx=0x7fa66c007940 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:14.879+0000 7fa672a00640 1 -- 172.21.15.87:0/3035919790 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa664016090 con 0x7fa67c1550b0 2024-04-25T10:21:15.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.087+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa67c0cee90 con 0x7fa67c1550b0 2024-04-25T10:21:15.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.087+0000 7fa672a00640 1 -- 172.21.15.87:0/3035919790 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1806 (secure 0 0 0) 0x7fa6640eefc0 con 0x7fa67c1550b0 2024-04-25T10:21:15.091 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:15.091 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":557,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076632719,"bytes_used":2603368448,"bytes_avail":770490744832,"bytes_total":773094113280,"recovering_objects_per_sec":3,"recovering_bytes_per_sec":3538310,"recovering_keys_per_sec":0,"num_objects_recovered":27,"num_bytes_recovered":28311552,"num_keys_recovered":0,"write_bytes_sec":40081532,"read_op_per_sec":0,"write_op_per_sec":41},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:21:13.237152+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:15.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa658066f60 msgr2=0x7fa658069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa658066f60 0x7fa658069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa66c007fd0 tx=0x7fa66c007940 comp rx=0 tx=0).stop 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c1550b0 msgr2=0x7fa67c0cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c1550b0 0x7fa67c0cb1d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa664019080 tx=0x7fa664019210 comp rx=0 tx=0).stop 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 shutdown_connections 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa658066f60 0x7fa658069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa67c15b740 0x7fa67c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa67c1550b0 0x7fa67c0cb1d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 --2- 172.21.15.87:0/3035919790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa67c0d0040 0x7fa67c157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 >> 172.21.15.87:0/3035919790 conn(0x7fa67c066b70 msgr2=0x7fa67c0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 shutdown_connections 2024-04-25T10:21:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.091+0000 7fa681800640 1 -- 172.21.15.87:0/3035919790 wait complete. 2024-04-25T10:21:15.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:21:15.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:15.112 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T10:21:15.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12ac400640 1 Processor -- start 2024-04-25T10:21:15.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12ac400640 1 -- start start 2024-04-25T10:21:15.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12ac400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415b150 0x7f12a40aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12ac400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f12a406c0d0 0x7f12a406c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12ac400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40b2fd0 con 0x7f12a415a0e0 2024-04-25T10:21:15.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12ac400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40aef90 con 0x7f12a406c0d0 2024-04-25T10:21:15.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12ac400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40af1d0 con 0x7f12a415b150 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12aa400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f12a406c0d0 0x7f12a406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12aa400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f12a406c0d0 0x7f12a406c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36426/0 (socket says 172.21.15.87:36426) 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12aa400640 1 -- 172.21.15.87:0/1785083086 learned_addr learned my addr 172.21.15.87:0/1785083086 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12a9000640 1 --2- 172.21.15.87:0/1785083086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415b150 0x7f12a40aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12aa400640 1 -- 172.21.15.87:0/1785083086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415b150 msgr2=0x7f12a40aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12aa400640 1 --2- 172.21.15.87:0/1785083086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415b150 0x7f12a40aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12aa400640 1 -- 172.21.15.87:0/1785083086 >> v1:172.21.15.169:6789/0 conn(0x7f12a415a0e0 legacy=0x7f12a40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12aa400640 1 -- 172.21.15.87:0/1785083086 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40b32b0 con 0x7f12a406c0d0 2024-04-25T10:21:15.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.247+0000 7f12a9000640 1 --2- 172.21.15.87:0/1785083086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415b150 0x7f12a40aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:15.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12aa400640 1 --2- 172.21.15.87:0/1785083086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a406c520 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e50 tx=0x7f129c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61ed02ec972b087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12a3e00640 1 -- 172.21.15.87:0/1785083086 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0a55e0 con 0x7f12a406c0d0 2024-04-25T10:21:15.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12a3e00640 1 -- 172.21.15.87:0/1785083086 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c0a5780 con 0x7f12a406c0d0 2024-04-25T10:21:15.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12a3e00640 1 -- 172.21.15.87:0/1785083086 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0a5b50 con 0x7f12a406c0d0 2024-04-25T10:21:15.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 -- 172.21.15.87:0/1785083086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 msgr2=0x7f12a406c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 --2- 172.21.15.87:0/1785083086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a406c520 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e50 tx=0x7f129c01c9a0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 -- 172.21.15.87:0/1785083086 shutdown_connections 2024-04-25T10:21:15.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 --2- 172.21.15.87:0/1785083086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a406c520 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 --2- 172.21.15.87:0/1785083086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415b150 0x7f12a40aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 -- 172.21.15.87:0/1785083086 >> 172.21.15.87:0/1785083086 conn(0x7f12a40669a0 msgr2=0x7f12a4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 -- 172.21.15.87:0/1785083086 shutdown_connections 2024-04-25T10:21:15.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.251+0000 7f12ac400640 1 -- 172.21.15.87:0/1785083086 wait complete. 2024-04-25T10:21:15.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 Processor -- start 2024-04-25T10:21:15.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 -- start start 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a4144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415a0e0 0x7f12a414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a415b150 0x7f12a41455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40b4620 con 0x7f12a406c0d0 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40b44a0 con 0x7f12a415b150 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40705f0 con 0x7f12a415a0e0 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a4144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a4144d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36438/0 (socket says 172.21.15.87:36438) 2024-04-25T10:21:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 -- 172.21.15.87:0/65016497 learned_addr learned my addr 172.21.15.87:0/65016497 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12aa400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a415b150 0x7f12a41455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9000640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415a0e0 0x7f12a414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 -- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415a0e0 msgr2=0x7f12a414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415a0e0 0x7f12a414ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 -- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a415b150 msgr2=0x7f12a41455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a415b150 0x7f12a41455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 -- 172.21.15.87:0/65016497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a414d700 con 0x7f12a406c0d0 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12aa400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a415b150 0x7f12a41455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a9a00640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a4144d00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f129800c420 tx=0x7f129800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a1600640 1 -- 172.21.15.87:0/65016497 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12980052d0 con 0x7f12a406c0d0 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a1600640 1 -- 172.21.15.87:0/65016497 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298005470 con 0x7f12a406c0d0 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12a1600640 1 -- 172.21.15.87:0/65016497 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298007260 con 0x7f12a406c0d0 2024-04-25T10:21:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a414f4a0 con 0x7f12a406c0d0 2024-04-25T10:21:15.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.255+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a4146d00 con 0x7f12a406c0d0 2024-04-25T10:21:15.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12a1600640 1 -- 172.21.15.87:0/65016497 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1298007400 con 0x7f12a406c0d0 2024-04-25T10:21:15.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12a1600640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1284066f60 0x7f1284069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12a1600640 1 -- 172.21.15.87:0/65016497 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1298084500 con 0x7f12a406c0d0 2024-04-25T10:21:15.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12a9000640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1284066f60 0x7f1284069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 Processor -- start 2024-04-25T10:21:15.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 -- start start 2024-04-25T10:21:15.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12a9000640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1284066f60 0x7f1284069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f12a415c1c0 tx=0x7f1290008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cbf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbf770 0x556c01c79c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01c7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01b581f0 con 0x556c01c7a1a0 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01abee90 con 0x556c01cbf770 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.259+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c01a46f50 con 0x556c01cbee50 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01c7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01c7a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36444/0 (socket says 172.21.15.87:36444) 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 -- 172.21.15.87:0/4023160303 learned_addr learned my addr 172.21.15.87:0/4023160303 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12a9000640 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbf770 0x556c01c79c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12a9a00640 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cbf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 -- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 msgr2=0x556c01cbf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cbf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 -- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbf770 msgr2=0x556c01c79c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbf770 0x556c01c79c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 -- 172.21.15.87:0/4023160303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c01a968b0 con 0x556c01c7a1a0 2024-04-25T10:21:15.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12aa400640 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01c7a580 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f129c0af040 tx=0x7f129c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6cc95a0c7f289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f1296000640 1 -- 172.21.15.87:0/4023160303 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0bd070 con 0x556c01c7a1a0 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f1296000640 1 -- 172.21.15.87:0/4023160303 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c0a57e0 con 0x556c01c7a1a0 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f1296000640 1 -- 172.21.15.87:0/4023160303 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0a5bb0 con 0x556c01c7a1a0 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- 172.21.15.87:0/4023160303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 msgr2=0x556c01c7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01c7a580 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f129c0af040 tx=0x7f129c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- 172.21.15.87:0/4023160303 shutdown_connections 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01c7a580 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbf770 0x556c01c79c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 --2- 172.21.15.87:0/4023160303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cbf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- 172.21.15.87:0/4023160303 >> 172.21.15.87:0/4023160303 conn(0x556c01c71d40 msgr2=0x556c01c74130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- 172.21.15.87:0/4023160303 shutdown_connections 2024-04-25T10:21:15.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- 172.21.15.87:0/4023160303 wait complete. 2024-04-25T10:21:15.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- start start 2024-04-25T10:21:15.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 Processor -- start 2024-04-25T10:21:15.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cc66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbf770 0x556c01c7fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01a79540 con 0x556c01cbf770 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01abaf30 con 0x556c01c7a1a0 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.263+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c01a8b7c0 con 0x556c01cbee50 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 10 client.0 _ll_register_callbacks cb 0x556c01b73180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbf770 0x556c01c7fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbf770 0x556c01c7fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36454/0 (socket says 172.21.15.87:36454) 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 -- 172.21.15.87:0/4130295328 learned_addr learned my addr 172.21.15.87:0/4130295328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12a9a00640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12a9000640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cc66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 -- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 msgr2=0x556c01cc66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cc66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 -- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 msgr2=0x556c01cc61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 -- 172.21.15.87:0/4130295328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c01a471a0 con 0x556c01cbf770 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12a9000640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cc66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12aa400640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbf770 0x556c01c7fe00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f129c098380 tx=0x7f129c01b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685b3df15b13edf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 1 -- 172.21.15.87:0/4130295328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0ba070 con 0x556c01cbf770 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 1 -- 172.21.15.87:0/4130295328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f129c0a55e0 con 0x556c01cbf770 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f129c0a57e0 con 0x556c01cbf770 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 1 -- 172.21.15.87:0/4130295328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f129c0bd040 con 0x556c01cbf770 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 10 client.8787 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x556c01b41840 con 0x556c01cbf770 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 1 -- 172.21.15.87:0/4130295328 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c0a57e0 con 0x556c01cbf770 2024-04-25T10:21:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 1 -- 172.21.15.87:0/4130295328 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f129c008040 con 0x556c01cbf770 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 10 client.8787 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 10 client.8787 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 20 client.8787 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01c80ba0 0x556c01c80fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 4 client.8787 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c01a5abe0 con 0x556c01c80ba0 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12a9a00640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01c80ba0 0x556c01c80fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12a9a00640 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01c80ba0 0x556c01c80fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35cd450197a6716b server_cookie=213263c1619d3e23 in_seq=0 out_seq=0 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 10 client.8787 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 1 -- 172.21.15.87:0/4130295328 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c01a5abe0 con 0x556c01c80ba0 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 10 client.8787 handle_command_reply: tid=0 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 client.8787 shutdown 2024-04-25T10:21:15.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbf770 msgr2=0x556c01c7fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbf770 0x556c01c7fe00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f129c098380 tx=0x7f129c01b1e0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 shutdown_connections 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01c80ba0 0x556c01c80fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbf770 0x556c01c7fe00 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbee50 0x556c01cc66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f128aa00640 0 client.8787 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 --2- 172.21.15.87:0/4130295328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 >> 172.21.15.87:0/4130295328 conn(0x556c01c71d40 msgr2=0x556c01c749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 shutdown_connections 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 1 -- 172.21.15.87:0/4130295328 wait complete. 2024-04-25T10:21:15.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.267+0000 7f12af986000 20 client.8787 trim_cache size 0 max 16384 2024-04-25T10:21:15.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 Processor -- start 2024-04-25T10:21:15.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 -- start start 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01d1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbee50 0x556c01d1bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01d1c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41596a0 con 0x556c01c7a1a0 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41588f0 con 0x556c01cbee50 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a414e450 con 0x556c01cbf770 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01d1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01d1b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36456/0 (socket says 172.21.15.87:36456) 2024-04-25T10:21:15.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 -- 172.21.15.87:0/1685437913 learned_addr learned my addr 172.21.15.87:0/1685437913 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 -- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 msgr2=0x556c01d1c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12aa400640 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01d1c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9000640 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbee50 0x556c01d1bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01d1c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 -- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbee50 msgr2=0x556c01d1bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbee50 0x556c01d1bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 -- 172.21.15.87:0/1685437913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c01cbf5a0 con 0x556c01c7a1a0 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12aa400640 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01d1c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f12a9a00640 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01d1b6a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1298054b70 tx=0x7f129805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8d3a14438d64a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f1295600640 1 -- 172.21.15.87:0/1685437913 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298053170 con 0x556c01c7a1a0 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f1295600640 1 -- 172.21.15.87:0/1685437913 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129805b0b0 con 0x556c01c7a1a0 2024-04-25T10:21:15.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.283+0000 7f1295600640 1 -- 172.21.15.87:0/1685437913 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12980840c0 con 0x556c01c7a1a0 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/1685437913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 msgr2=0x556c01d1b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01d1b6a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1298054b70 tx=0x7f129805b040 comp rx=0 tx=0).stop 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/1685437913 shutdown_connections 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01d1c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01cbee50 0x556c01d1bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- 172.21.15.87:0/1685437913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01c7a1a0 0x556c01d1b6a0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/1685437913 >> 172.21.15.87:0/1685437913 conn(0x556c01c71d40 msgr2=0x556c01c74130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/1685437913 shutdown_connections 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/1685437913 wait complete. 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- start start 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 Processor -- start 2024-04-25T10:21:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbee50 0x556c01cc4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01cdc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01ac34c0 con 0x556c01cbee50 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a414ea50 con 0x556c01c7a1a0 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c01c71bc0 con 0x556c01cbf770 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 10 client.0 _ll_register_callbacks cb 0x556c01a81a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbee50 0x556c01cc4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbee50 0x556c01cc4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36466/0 (socket says 172.21.15.87:36466) 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 -- 172.21.15.87:0/3824407980 learned_addr learned my addr 172.21.15.87:0/3824407980 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9a00640 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12aa400640 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01cdc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 -- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 msgr2=0x556c01cdc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01cdc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 -- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 msgr2=0x556c01cc42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 -- 172.21.15.87:0/3824407980 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c01b18a10 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9000640 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbee50 0x556c01cc4820 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1290016730 tx=0x7f12900046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48e5ddd2b3292fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f128aa00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f128aa00640 1 -- 172.21.15.87:0/3824407980 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290028070 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f128aa00640 1 -- 172.21.15.87:0/3824407980 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f12900148f0 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f129805b0b0 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f128aa00640 1 -- 172.21.15.87:0/3824407980 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f129001c040 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 10 client.8793 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x556c01a53b00 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f128aa00640 1 -- 172.21.15.87:0/3824407980 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129805b0b0 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f128aa00640 1 -- 172.21.15.87:0/3824407980 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1290036080 con 0x556c01cbee50 2024-04-25T10:21:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 10 client.8793 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 10 client.8793 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 20 client.8793 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01cdcdb0 0x556c01cdd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 4 client.8793 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x556c01acc3d0 con 0x556c01cdcdb0 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.287+0000 7f12a9a00640 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01cdcdb0 0x556c01cdd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12a9a00640 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01cdcdb0 0x556c01cdd1c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41502711e52a8926 server_cookie=92b800609163df6f in_seq=0 out_seq=0 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f128aa00640 10 client.8793 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f128aa00640 1 -- 172.21.15.87:0/3824407980 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x556c01acc3d0 con 0x556c01cdcdb0 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f128aa00640 10 client.8793 handle_command_reply: tid=0 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 client.8793 shutdown 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbee50 msgr2=0x556c01cc4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbee50 0x556c01cc4820 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1290016730 tx=0x7f12900046d0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 shutdown_connections 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x556c01cdcdb0 0x556c01cdd1c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c01cbf770 0x556c01cdc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556c01cbee50 0x556c01cc4820 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f128aa00640 0 client.8793 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 --2- 172.21.15.87:0/3824407980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c01c7a1a0 0x556c01cc42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 >> 172.21.15.87:0/3824407980 conn(0x556c01c71d40 msgr2=0x556c01c749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 shutdown_connections 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 1 -- 172.21.15.87:0/3824407980 wait complete. 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12af986000 20 client.8793 trim_cache size 0 max 16384 2024-04-25T10:21:15.297 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:15.298 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0 2024-04-25T10:21:15.298 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:15.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1284066f60 msgr2=0x7f1284069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1284066f60 0x7f1284069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f12a415c1c0 tx=0x7f1290008040 comp rx=0 tx=0).stop 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 msgr2=0x7f12a4144d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a4144d00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f129800c420 tx=0x7f129800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 shutdown_connections 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1284066f60 0x7f1284069420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a415b150 0x7f12a41455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a415a0e0 0x7f12a414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 --2- 172.21.15.87:0/65016497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12a406c0d0 0x7f12a4144d00 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 >> 172.21.15.87:0/65016497 conn(0x7f12a40669a0 msgr2=0x7f12a40711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 shutdown_connections 2024-04-25T10:21:15.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.291+0000 7f12ac400640 1 -- 172.21.15.87:0/65016497 wait complete. 2024-04-25T10:21:15.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:15.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:21:15.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f838b000640 1 Processor -- start 2024-04-25T10:21:15.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f838b000640 1 -- start start 2024-04-25T10:21:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f838b000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83841610b0 0x7f83840af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f838b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f838406e5e0 0x7f83840669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f838b000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83840b3820 con 0x7f83840b59a0 2024-04-25T10:21:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f838b000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83840b3a90 con 0x7f83841610b0 2024-04-25T10:21:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f838b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b3d70 con 0x7f838406e5e0 2024-04-25T10:21:15.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8383e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83841610b0 0x7f83840af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8383400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f838406e5e0 0x7f83840669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8389000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f83840b59a0 0x7f8384161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:51578/0 (socket says 172.21.15.87:51578) 2024-04-25T10:21:15.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8389000640 1 -- 172.21.15.87:0/664102023 learned_addr learned my addr 172.21.15.87:0/664102023 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:15.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8383e00640 1 -- 172.21.15.87:0/664102023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f838406e5e0 msgr2=0x7f83840669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8383e00640 1 --2- 172.21.15.87:0/664102023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f838406e5e0 0x7f83840669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8383e00640 1 -- 172.21.15.87:0/664102023 >> v1:172.21.15.169:6789/0 conn(0x7f83840b59a0 legacy=0x7f8384161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8383e00640 1 -- 172.21.15.87:0/664102023 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840afee0 con 0x7f83841610b0 2024-04-25T10:21:15.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.447+0000 7f8382a00640 1 -- 172.21.15.87:0/664102023 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175681136 0 0) 0x7f83840b3820 con 0x7f83840b59a0 2024-04-25T10:21:15.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.451+0000 7f8383e00640 1 --2- 172.21.15.87:0/664102023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f83840af600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8374009d90 tx=0x7f837401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af15a676307340e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.451+0000 7f8382a00640 1 -- 172.21.15.87:0/664102023 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740b2070 con 0x7f83841610b0 2024-04-25T10:21:15.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.451+0000 7f8382a00640 1 -- 172.21.15.87:0/664102023 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740a5830 con 0x7f83841610b0 2024-04-25T10:21:15.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.451+0000 7f8382a00640 1 -- 172.21.15.87:0/664102023 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740a5c00 con 0x7f83841610b0 2024-04-25T10:21:15.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- 172.21.15.87:0/664102023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 msgr2=0x7f83840af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 --2- 172.21.15.87:0/664102023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f83840af600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8374009d90 tx=0x7f837401c970 comp rx=0 tx=0).stop 2024-04-25T10:21:15.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- 172.21.15.87:0/664102023 shutdown_connections 2024-04-25T10:21:15.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 --2- 172.21.15.87:0/664102023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f838406e5e0 0x7f83840669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 --2- 172.21.15.87:0/664102023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f83840af600 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- 172.21.15.87:0/664102023 >> 172.21.15.87:0/664102023 conn(0x7f838406ac90 msgr2=0x7f838406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- 172.21.15.87:0/664102023 shutdown_connections 2024-04-25T10:21:15.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- 172.21.15.87:0/664102023 wait complete. 2024-04-25T10:21:15.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 Processor -- start 2024-04-25T10:21:15.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- start start 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f838406e5e0 0x7f8384152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83840b59a0 0x7f83840af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f8384158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838416ad90 con 0x7f838406e5e0 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838416ac10 con 0x7f83841610b0 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f838b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838416af10 con 0x7f83840b59a0 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f838406e5e0 0x7f8384152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f838406e5e0 0x7f8384152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36476/0 (socket says 172.21.15.87:36476) 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 -- 172.21.15.87:0/1080689255 learned_addr learned my addr 172.21.15.87:0/1080689255 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383400640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83840b59a0 0x7f83840af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8389000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f8384158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 -- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83840b59a0 msgr2=0x7f83840af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83840b59a0 0x7f83840af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 -- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 msgr2=0x7f8384158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f8384158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8383e00640 1 -- 172.21.15.87:0/1080689255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384151f10 con 0x7f838406e5e0 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.455+0000 7f8389000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f8384158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f8383e00640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f838406e5e0 0x7f8384152cd0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83740b0040 tx=0x7f8374019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f837be00640 1 -- 172.21.15.87:0/1080689255 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740b2050 con 0x7f838406e5e0 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f837be00640 1 -- 172.21.15.87:0/1080689255 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740be070 con 0x7f838406e5e0 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f837be00640 1 -- 172.21.15.87:0/1080689255 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740ada70 con 0x7f838406e5e0 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83841521f0 con 0x7f838406e5e0 2024-04-25T10:21:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8384158a70 con 0x7f838406e5e0 2024-04-25T10:21:15.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f837be00640 1 -- 172.21.15.87:0/1080689255 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83740a56b0 con 0x7f838406e5e0 2024-04-25T10:21:15.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f837be00640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8364066f60 0x7f8364069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.459+0000 7f837be00640 1 -- 172.21.15.87:0/1080689255 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8374120270 con 0x7f838406e5e0 2024-04-25T10:21:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383400640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8364066f60 0x7f8364069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 Processor -- start 2024-04-25T10:21:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- start start 2024-04-25T10:21:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383400640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8364066f60 0x7f8364069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f838415e1e0 tx=0x7f8370008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b89d0 0x5567ff2b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b92f0 0x5567ff26eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff26f0a0 0x5567ff26f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ff0bbff0 con 0x5567ff2b89d0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ff044150 con 0x5567ff2b92f0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ff0b8090 con 0x5567ff26f0a0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b89d0 0x5567ff2b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b89d0 0x5567ff2b8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36480/0 (socket says 172.21.15.87:36480) 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 -- 172.21.15.87:0/1257823981 learned_addr learned my addr 172.21.15.87:0/1257823981 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8389000640 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff26f0a0 0x5567ff26f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383400640 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b92f0 0x5567ff26eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 -- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff26f0a0 msgr2=0x5567ff26f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff26f0a0 0x5567ff26f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 -- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b92f0 msgr2=0x5567ff26eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b92f0 0x5567ff26eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 -- 172.21.15.87:0/1257823981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ff2520d0 con 0x5567ff2b89d0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383400640 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b92f0 0x5567ff26eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8383e00640 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b89d0 0x5567ff2b8db0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f83740f8da0 tx=0x7f83740f0ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f827866bf64c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8378c00640 1 -- 172.21.15.87:0/1257823981 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740f0bc0 con 0x5567ff2b89d0 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8378c00640 1 -- 172.21.15.87:0/1257823981 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740b3040 con 0x5567ff2b89d0 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f8378c00640 1 -- 172.21.15.87:0/1257823981 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740f4d10 con 0x5567ff2b89d0 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- 172.21.15.87:0/1257823981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b89d0 msgr2=0x5567ff2b8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b89d0 0x5567ff2b8db0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f83740f8da0 tx=0x7f83740f0ac0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- 172.21.15.87:0/1257823981 shutdown_connections 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff26f0a0 0x5567ff26f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b92f0 0x5567ff26eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 --2- 172.21.15.87:0/1257823981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b89d0 0x5567ff2b8db0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- 172.21.15.87:0/1257823981 >> 172.21.15.87:0/1257823981 conn(0x5567ff26b020 msgr2=0x5567ff26d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- 172.21.15.87:0/1257823981 shutdown_connections 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- 172.21.15.87:0/1257823981 wait complete. 2024-04-25T10:21:15.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.463+0000 7f838e52f000 1 -- start start 2024-04-25T10:21:15.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 Processor -- start 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff26f0a0 0x5567ff29d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b89d0 0x5567ff29dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b92f0 0x5567ff29e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ff080250 con 0x5567ff2b92f0 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ff076690 con 0x5567ff26f0a0 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ff044a40 con 0x5567ff2b89d0 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 10 client.0 _ll_register_callbacks cb 0x5567ff259e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b92f0 0x5567ff29e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b92f0 0x5567ff29e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36488/0 (socket says 172.21.15.87:36488) 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 -- 172.21.15.87:0/3972135753 learned_addr learned my addr 172.21.15.87:0/3972135753 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8383e00640 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff26f0a0 0x5567ff29d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8383400640 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b89d0 0x5567ff29dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 -- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b89d0 msgr2=0x5567ff29dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b89d0 0x5567ff29dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 -- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff26f0a0 msgr2=0x5567ff29d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff26f0a0 0x5567ff29d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 -- 172.21.15.87:0/3972135753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567ff1691c0 con 0x5567ff2b92f0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8389000640 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b92f0 0x5567ff29e270 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f837c005ea0 tx=0x7f837c001530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b344972febbd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f836d600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f836d600640 1 -- 172.21.15.87:0/3972135753 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c0133f0 con 0x5567ff2b92f0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f836d600640 1 -- 172.21.15.87:0/3972135753 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f837c013590 con 0x5567ff2b92f0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83740b3040 con 0x5567ff2b92f0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f836d600640 1 -- 172.21.15.87:0/3972135753 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f837c020050 con 0x5567ff2b92f0 2024-04-25T10:21:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 10 client.8802 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5567ff165170 con 0x5567ff2b92f0 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f836d600640 1 -- 172.21.15.87:0/3972135753 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740b3040 con 0x5567ff2b92f0 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f836d600640 1 -- 172.21.15.87:0/3972135753 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f837c023020 con 0x5567ff2b92f0 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 10 client.8802 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 10 client.8802 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 20 client.8802 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff2c8c80 0x5567ff2cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 4 client.8802 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567ff057c60 con 0x5567ff2c8c80 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8383e00640 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff2c8c80 0x5567ff2cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f8383e00640 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff2c8c80 0x5567ff2cb070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631524d575f79418 server_cookie=bb066bc05901742d in_seq=0 out_seq=0 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.467+0000 7f836d600640 10 client.8802 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f836d600640 1 -- 172.21.15.87:0/3972135753 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567ff057c60 con 0x5567ff2c8c80 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f836d600640 10 client.8802 handle_command_reply: tid=0 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 client.8802 shutdown 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b92f0 msgr2=0x5567ff29e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b92f0 0x5567ff29e270 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f837c005ea0 tx=0x7f837c001530 comp rx=0 tx=0).stop 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 shutdown_connections 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff2c8c80 0x5567ff2cb070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff2b92f0 0x5567ff29e270 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b89d0 0x5567ff29dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 --2- 172.21.15.87:0/3972135753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff26f0a0 0x5567ff29d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 >> 172.21.15.87:0/3972135753 conn(0x5567ff26b020 msgr2=0x5567ff26dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f836d600640 0 client.8802 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 shutdown_connections 2024-04-25T10:21:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 1 -- 172.21.15.87:0/3972135753 wait complete. 2024-04-25T10:21:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.471+0000 7f838e52f000 20 client.8802 trim_cache size 0 max 16384 2024-04-25T10:21:15.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.483+0000 7f838e52f000 1 Processor -- start 2024-04-25T10:21:15.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.483+0000 7f838e52f000 1 -- start start 2024-04-25T10:21:15.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff315300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff315840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff315d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8384203120 con 0x5567ff26f0a0 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8384202380 con 0x5567ff2b89d0 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840adc50 con 0x5567ff2b92f0 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff315300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff315300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36494/0 (socket says 172.21.15.87:36494) 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 -- 172.21.15.87:0/2916907219 learned_addr learned my addr 172.21.15.87:0/2916907219 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 -- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 msgr2=0x5567ff315d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8389000640 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff315d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383400640 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff315840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff315d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 -- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 msgr2=0x5567ff315840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff315840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 -- 172.21.15.87:0/2916907219 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ff102de0 con 0x5567ff26f0a0 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8389000640 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff315d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383400640 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff315840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff315300 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8374009d90 tx=0x7f83740ba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca6a7244fb5c628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f836fe00640 1 -- 172.21.15.87:0/2916907219 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740fe070 con 0x5567ff26f0a0 2024-04-25T10:21:15.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f836fe00640 1 -- 172.21.15.87:0/2916907219 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740f4d10 con 0x5567ff26f0a0 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f836fe00640 1 -- 172.21.15.87:0/2916907219 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740f0bc0 con 0x5567ff26f0a0 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- 172.21.15.87:0/2916907219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 msgr2=0x5567ff315300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff315300 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8374009d90 tx=0x7f83740ba000 comp rx=0 tx=0).stop 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- 172.21.15.87:0/2916907219 shutdown_connections 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff315d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff315840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- 172.21.15.87:0/2916907219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff315300 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- 172.21.15.87:0/2916907219 >> 172.21.15.87:0/2916907219 conn(0x5567ff26b020 msgr2=0x5567ff2c5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- 172.21.15.87:0/2916907219 shutdown_connections 2024-04-25T10:21:15.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- 172.21.15.87:0/2916907219 wait complete. 2024-04-25T10:21:15.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- start start 2024-04-25T10:21:15.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 Processor -- start 2024-04-25T10:21:15.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff2cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff2cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff2cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ff056e00 con 0x5567ff26f0a0 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83840ae250 con 0x5567ff2b89d0 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ff0bb300 con 0x5567ff2b92f0 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f838e52f000 10 client.0 _ll_register_callbacks cb 0x5567ff23a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff2cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff2cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36506/0 (socket says 172.21.15.87:36506) 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383e00640 1 -- 172.21.15.87:0/2057825098 learned_addr learned my addr 172.21.15.87:0/2057825098 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8389000640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff2cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.487+0000 7f8383400640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff2cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383e00640 1 -- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 msgr2=0x5567ff2cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383e00640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff2cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383e00640 1 -- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 msgr2=0x5567ff2cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383e00640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff2cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383e00640 1 -- 172.21.15.87:0/2057825098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567ff05d8e0 con 0x5567ff26f0a0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8389000640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff2cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383e00640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff2cb770 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f83740ed020 tx=0x7f8374120040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16beede646ee111d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 1 -- 172.21.15.87:0/2057825098 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740fe070 con 0x5567ff26f0a0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 1 -- 172.21.15.87:0/2057825098 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f83740b6df0 con 0x5567ff26f0a0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83740f4d10 con 0x5567ff26f0a0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 1 -- 172.21.15.87:0/2057825098 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83740f0bc0 con 0x5567ff26f0a0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 10 client.8808 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5567ff1158f0 con 0x5567ff26f0a0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 1 -- 172.21.15.87:0/2057825098 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740f4d10 con 0x5567ff26f0a0 2024-04-25T10:21:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 1 -- 172.21.15.87:0/2057825098 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83740ed230 con 0x5567ff26f0a0 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 10 client.8808 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 10 client.8808 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 20 client.8808 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff31ef50 0x5567ff321340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 4 client.8808 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5567ff0c9530 con 0x5567ff31ef50 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383400640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff31ef50 0x5567ff321340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 10 client.8808 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f8383400640 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff31ef50 0x5567ff321340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f74ca85bb71bd6 server_cookie=68cbf1fea7ce79f in_seq=0 out_seq=0 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 1 -- 172.21.15.87:0/2057825098 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x5567ff0c9530 con 0x5567ff31ef50 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 10 client.8808 handle_command_reply: tid=0 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 client.8808 shutdown 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 msgr2=0x5567ff2cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff2cb770 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f83740ed020 tx=0x7f8374120040 comp rx=0 tx=0).stop 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 shutdown_connections 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x5567ff31ef50 0x5567ff321340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5567ff2b92f0 0x5567ff2cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567ff2b89d0 0x5567ff2cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f836d600640 0 client.8808 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 --2- 172.21.15.87:0/2057825098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567ff26f0a0 0x5567ff2cb770 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 >> 172.21.15.87:0/2057825098 conn(0x5567ff26b020 msgr2=0x5567ff2b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 shutdown_connections 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 1 -- 172.21.15.87:0/2057825098 wait complete. 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838e52f000 20 client.8808 trim_cache size 0 max 16384 2024-04-25T10:21:15.500 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:21:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:21:15.501 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8364066f60 msgr2=0x7f8364069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8364066f60 0x7f8364069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f838415e1e0 tx=0x7f8370008040 comp rx=0 tx=0).stop 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f838406e5e0 msgr2=0x7f8384152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f838406e5e0 0x7f8384152cd0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83740b0040 tx=0x7f8374019710 comp rx=0 tx=0).stop 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 shutdown_connections 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8364066f60 0x7f8364069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83841610b0 0x7f8384158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83840b59a0 0x7f83840af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 --2- 172.21.15.87:0/1080689255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f838406e5e0 0x7f8384152cd0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 >> 172.21.15.87:0/1080689255 conn(0x7f838406ac90 msgr2=0x7f838415a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 shutdown_connections 2024-04-25T10:21:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.491+0000 7f838b000640 1 -- 172.21.15.87:0/1080689255 wait complete. 2024-04-25T10:21:15.513 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9210c00640 1 Processor -- start 2024-04-25T10:21:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9210c00640 1 -- start start 2024-04-25T10:21:15.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9210c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f920c0d0040 0x7f920c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9210c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c071050 con 0x7f920c15b740 2024-04-25T10:21:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9210c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c15d040 con 0x7f920c1550b0 2024-04-25T10:21:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9210c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c0712f0 con 0x7f920c0d0040 2024-04-25T10:21:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f920a800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f920c15b740 0x7f920c0b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37906/0 (socket says 172.21.15.87:37906) 2024-04-25T10:21:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f920a800640 1 -- 172.21.15.87:0/1877292664 learned_addr learned my addr 172.21.15.87:0/1877292664 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209400640 1 --2- 172.21.15.87:0/1877292664 >> v2:172.21.15.169:3300/0 conn(0x7f920c0d0040 0x7f920c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798374437 0 0) 0x7f920c071050 con 0x7f920c15b740 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f0003530 con 0x7f920c15b740 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1752611170 0 0) 0x7f91f0003530 con 0x7f920c15b740 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 >> v2:172.21.15.169:3300/0 conn(0x7f920c0d0040 msgr2=0x7f920c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 --2- 172.21.15.87:0/1877292664 >> v2:172.21.15.169:3300/0 conn(0x7f920c0d0040 0x7f920c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 >> v1:172.21.15.169:6789/0 conn(0x7f920c1550b0 legacy=0x7f920c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0715d0 con 0x7f920c15b740 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f9204001a00 con 0x7f920c15b740 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 >> v1:172.21.15.87:6789/0 conn(0x7f920c15b740 legacy=0x7f920c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0003a10 0x7f91f0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f0004810 0x7f91f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f00051d0 0x7f91f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0712f0 con 0x7f91f0003a10 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000c0d0 con 0x7f91f00051d0 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000c2d0 con 0x7f91f0004810 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209e00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0003a10 0x7f91f0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209400640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f0004810 0x7f91f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f920a800640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f00051d0 0x7f91f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209e00640 1 -- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f0004810 msgr2=0x7f91f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209e00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f0004810 0x7f91f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209e00640 1 -- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f00051d0 msgr2=0x7f91f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209e00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f00051d0 0x7f91f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f9209e00640 1 -- 172.21.15.87:0/1877292664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0003530 con 0x7f91f0003a10 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f920a800640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f00051d0 0x7f91f000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:15.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.646+0000 7f920a800640 1 -- 172.21.15.87:0/1877292664 reap_dead start 2024-04-25T10:21:15.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9209e00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0003a10 0x7f91f0003e30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f91fc001780 tx=0x7f91fc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74c5d1d0fbc912f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc0013a0 con 0x7f91f0003a10 2024-04-25T10:21:15.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc0052f0 con 0x7f91f0003a10 2024-04-25T10:21:15.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9208a00640 1 -- 172.21.15.87:0/1877292664 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc003840 con 0x7f91f0003a10 2024-04-25T10:21:15.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9210c00640 1 -- 172.21.15.87:0/1877292664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0003a10 msgr2=0x7f91f0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9210c00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0003a10 0x7f91f0003e30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f91fc001780 tx=0x7f91fc01c3d0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9210c00640 1 -- 172.21.15.87:0/1877292664 shutdown_connections 2024-04-25T10:21:15.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9210c00640 1 --2- 172.21.15.87:0/1877292664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f0003a10 0x7f91f0003e30 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9210c00640 1 -- 172.21.15.87:0/1877292664 >> 172.21.15.87:0/1877292664 conn(0x7f920c066b70 msgr2=0x7f920c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9210c00640 1 -- 172.21.15.87:0/1877292664 shutdown_connections 2024-04-25T10:21:15.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.650+0000 7f9210c00640 1 -- 172.21.15.87:0/1877292664 wait complete. 2024-04-25T10:21:15.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 Processor -- start 2024-04-25T10:21:15.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 -- start start 2024-04-25T10:21:15.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f0003a10 0x7f920c145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0ad840 0x7f920c0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c0cdde0 0x7f920c0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0cfec0 con 0x7f920c0cdde0 2024-04-25T10:21:15.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0cfd40 con 0x7f91f0003a10 2024-04-25T10:21:15.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c15bb50 con 0x7f920c0ad840 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c0cdde0 0x7f920c0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c0cdde0 0x7f920c0ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36536/0 (socket says 172.21.15.87:36536) 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 -- 172.21.15.87:0/1508522066 learned_addr learned my addr 172.21.15.87:0/1508522066 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9209e00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f0003a10 0x7f920c145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9209400640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0ad840 0x7f920c0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 -- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0ad840 msgr2=0x7f920c0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0ad840 0x7f920c0adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 -- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f0003a10 msgr2=0x7f920c145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f0003a10 0x7f920c145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 -- 172.21.15.87:0/1508522066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0cead0 con 0x7f920c0cdde0 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f920a800640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c0cdde0 0x7f920c0ce260 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f920c15bb20 tx=0x7f920400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9202000640 1 -- 172.21.15.87:0/1508522066 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400eb90 con 0x7f920c0cdde0 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9202000640 1 -- 172.21.15.87:0/1508522066 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9204018070 con 0x7f920c0cdde0 2024-04-25T10:21:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9202000640 1 -- 172.21.15.87:0/1508522066 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204013420 con 0x7f920c0cdde0 2024-04-25T10:21:15.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c145d70 con 0x7f920c0cdde0 2024-04-25T10:21:15.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.654+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c146330 con 0x7f920c0cdde0 2024-04-25T10:21:15.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.658+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920c146530 con 0x7f920c0cdde0 2024-04-25T10:21:15.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.658+0000 7f9202000640 1 -- 172.21.15.87:0/1508522066 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92040135c0 con 0x7f920c0cdde0 2024-04-25T10:21:15.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.658+0000 7f9202000640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91ec066f60 0x7f91ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:15.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.658+0000 7f9202000640 1 -- 172.21.15.87:0/1508522066 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9204085250 con 0x7f920c0cdde0 2024-04-25T10:21:15.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.658+0000 7f9209e00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91ec066f60 0x7f91ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.658+0000 7f9209e00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91ec066f60 0x7f91ec069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f91fc001750 tx=0x7f91fc0011a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:15.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.662+0000 7f9202000640 1 -- 172.21.15.87:0/1508522066 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f920c146530 con 0x7f920c0cdde0 2024-04-25T10:21:15.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.846+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f920c04ef30 con 0x7f920c0cdde0 2024-04-25T10:21:15.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.846+0000 7f9202000640 1 -- 172.21.15.87:0/1508522066 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f92040562e0 con 0x7f920c0cdde0 2024-04-25T10:21:15.850 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:15.850 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:15.851 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91ec066f60 msgr2=0x7f91ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91ec066f60 0x7f91ec069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f91fc001750 tx=0x7f91fc0011a0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c0cdde0 msgr2=0x7f920c0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c0cdde0 0x7f920c0ce260 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f920c15bb20 tx=0x7f920400e530 comp rx=0 tx=0).stop 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 shutdown_connections 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f91ec066f60 0x7f91ec069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c0cdde0 0x7f920c0ce260 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0ad840 0x7f920c0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 --2- 172.21.15.87:0/1508522066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f0003a10 0x7f920c145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:15.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 >> 172.21.15.87:0/1508522066 conn(0x7f920c066b70 msgr2=0x7f920c158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:15.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 shutdown_connections 2024-04-25T10:21:15.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.850+0000 7f9210c00640 1 -- 172.21.15.87:0/1508522066 wait complete. 2024-04-25T10:21:15.870 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:15.871 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:16.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:15.998+0000 7fd36c800640 1 Processor -- start 2024-04-25T10:21:16.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36c800640 1 -- start start 2024-04-25T10:21:16.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36c800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd3640d1820 0x7fd3640d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 0x7fd3640cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36c800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640b1af0 con 0x7fd36415e050 2024-04-25T10:21:16.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36c800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640b1dd0 con 0x7fd3640d1820 2024-04-25T10:21:16.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b20b0 con 0x7fd3640b15b0 2024-04-25T10:21:16.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd369400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd36415e050 0x7fd3640b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37922/0 (socket says 172.21.15.87:37922) 2024-04-25T10:21:16.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd369400640 1 -- 172.21.15.87:0/1823171790 learned_addr learned my addr 172.21.15.87:0/1823171790 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:16.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd369e00640 1 --2- 172.21.15.87:0/1823171790 >> v2:172.21.15.169:3300/0 conn(0x7fd3640d1820 0x7fd3640d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36a800640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 0x7fd3640cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799154796 0 0) 0x7fd3640b1af0 con 0x7fd36415e050 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd34c003530 con 0x7fd36415e050 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 302014776 0 0) 0x7fd34c003530 con 0x7fd36415e050 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 msgr2=0x7fd3640cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 0x7fd3640cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 >> v2:172.21.15.169:3300/0 conn(0x7fd3640d1820 msgr2=0x7fd3640d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 --2- 172.21.15.87:0/1823171790 >> v2:172.21.15.169:3300/0 conn(0x7fd3640d1820 0x7fd3640d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640b2390 con 0x7fd36415e050 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd354001680 con 0x7fd36415e050 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 >> v1:172.21.15.87:6789/0 conn(0x7fd36415e050 legacy=0x7fd3640b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 0x7fd34c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c003f40 0x7fd34c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd34c004d50 0x7fd34c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b1dd0 con 0x7fd34c004d50 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b20b0 con 0x7fd34c003f40 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c00b9d0 con 0x7fd3640b15b0 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36a800640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd34c004d50 0x7fd34c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd369400640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c003f40 0x7fd34c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd369e00640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 0x7fd34c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36a800640 1 -- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 msgr2=0x7fd34c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36a800640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640b15b0 0x7fd34c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36a800640 1 -- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c003f40 msgr2=0x7fd34c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36a800640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c003f40 0x7fd34c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.002+0000 7fd36a800640 1 -- 172.21.15.87:0/1823171790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c003530 con 0x7fd34c004d50 2024-04-25T10:21:16.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36a800640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd34c004d50 0x7fd34c0051d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd360009f90 tx=0x7fd36001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c04b58ae27d1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600b2070 con 0x7fd34c004d50 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3600a5c50 con 0x7fd34c004d50 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd368a00640 1 -- 172.21.15.87:0/1823171790 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360009c10 con 0x7fd34c004d50 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36c800640 1 -- 172.21.15.87:0/1823171790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd34c004d50 msgr2=0x7fd34c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36c800640 1 --2- 172.21.15.87:0/1823171790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd34c004d50 0x7fd34c0051d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd360009f90 tx=0x7fd36001cbe0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36a800640 1 -- 172.21.15.87:0/1823171790 reap_dead start 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36c800640 1 -- 172.21.15.87:0/1823171790 shutdown_connections 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36c800640 1 -- 172.21.15.87:0/1823171790 >> 172.21.15.87:0/1823171790 conn(0x7fd36409a9d0 msgr2=0x7fd36409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36c800640 1 -- 172.21.15.87:0/1823171790 shutdown_connections 2024-04-25T10:21:16.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.006+0000 7fd36c800640 1 -- 172.21.15.87:0/1823171790 wait complete. 2024-04-25T10:21:16.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 Processor -- start 2024-04-25T10:21:16.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 -- start start 2024-04-25T10:21:16.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c004d50 0x7fd3640ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd364155db0 0x7fd3640c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640c8fc0 0x7fd36409cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cfc70 con 0x7fd364155db0 2024-04-25T10:21:16.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640d2140 con 0x7fd34c004d50 2024-04-25T10:21:16.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cfdf0 con 0x7fd3640c8fc0 2024-04-25T10:21:16.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd364155db0 0x7fd3640c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd364155db0 0x7fd3640c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36560/0 (socket says 172.21.15.87:36560) 2024-04-25T10:21:16.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 -- 172.21.15.87:0/1096522766 learned_addr learned my addr 172.21.15.87:0/1096522766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369e00640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c004d50 0x7fd3640ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 -- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640c8fc0 msgr2=0x7fd36409cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36a800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640c8fc0 0x7fd36409cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640c8fc0 0x7fd36409cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 -- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c004d50 msgr2=0x7fd3640ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c004d50 0x7fd3640ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 -- 172.21.15.87:0/1096522766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640c97b0 con 0x7fd364155db0 2024-04-25T10:21:16.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369e00640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c004d50 0x7fd3640ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:16.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36a800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640c8fc0 0x7fd36409cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:16.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd369400640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd364155db0 0x7fd3640c8a80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd354007370 tx=0x7fd354007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd35e000640 1 -- 172.21.15.87:0/1096522766 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35400f960 con 0x7fd364155db0 2024-04-25T10:21:16.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd35e000640 1 -- 172.21.15.87:0/1096522766 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd354018070 con 0x7fd364155db0 2024-04-25T10:21:16.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640c9a90 con 0x7fd364155db0 2024-04-25T10:21:16.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd35e000640 1 -- 172.21.15.87:0/1096522766 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35400fd80 con 0x7fd364155db0 2024-04-25T10:21:16.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3640ca050 con 0x7fd364155db0 2024-04-25T10:21:16.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.010+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36404ef30 con 0x7fd364155db0 2024-04-25T10:21:16.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.014+0000 7fd35e000640 1 -- 172.21.15.87:0/1096522766 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd35401d460 con 0x7fd364155db0 2024-04-25T10:21:16.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.014+0000 7fd35e000640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd348066f60 0x7fd348069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.014+0000 7fd35e000640 1 -- 172.21.15.87:0/1096522766 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd3540850c0 con 0x7fd364155db0 2024-04-25T10:21:16.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.014+0000 7fd369e00640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd348066f60 0x7fd348069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.014+0000 7fd369e00640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd348066f60 0x7fd348069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd358009b50 tx=0x7fd358008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.018+0000 7fd35e000640 1 -- 172.21.15.87:0/1096522766 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd354056120 con 0x7fd364155db0 2024-04-25T10:21:16.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.226+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd3640ae990 con 0x7fd364155db0 2024-04-25T10:21:16.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd35e000640 1 -- 172.21.15.87:0/1096522766 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1806 (secure 0 0 0) 0x7fd354055a20 con 0x7fd364155db0 2024-04-25T10:21:16.288 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:16.289 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":558,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076632719,"bytes_used":2603368448,"bytes_avail":770490744832,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":3059594,"recovering_keys_per_sec":0,"num_objects_recovered":27,"num_bytes_recovered":28311552,"num_keys_recovered":0,"write_bytes_sec":34658691,"read_op_per_sec":0,"write_op_per_sec":35},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:21:13.237152+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd348066f60 msgr2=0x7fd348069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd348066f60 0x7fd348069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd358009b50 tx=0x7fd358008040 comp rx=0 tx=0).stop 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd364155db0 msgr2=0x7fd3640c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd364155db0 0x7fd3640c8a80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd354007370 tx=0x7fd354007830 comp rx=0 tx=0).stop 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 shutdown_connections 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd348066f60 0x7fd348069420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3640c8fc0 0x7fd36409cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd364155db0 0x7fd3640c8a80 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 --2- 172.21.15.87:0/1096522766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c004d50 0x7fd3640ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 >> 172.21.15.87:0/1096522766 conn(0x7fd36409a9d0 msgr2=0x7fd3640b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 shutdown_connections 2024-04-25T10:21:16.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.230+0000 7fd36c800640 1 -- 172.21.15.87:0/1096522766 wait complete. 2024-04-25T10:21:16.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:21:16.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:16.291 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T10:21:16.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16afa00640 1 Processor -- start 2024-04-25T10:21:16.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16afa00640 1 -- start start 2024-04-25T10:21:16.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16afa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16afa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16a8072020 0x7f16a8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16afa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a8159920 con 0x7f16a8159540 2024-04-25T10:21:16.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16afa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a815c2f0 con 0x7f16a8072020 2024-04-25T10:21:16.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16afa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a815c5d0 con 0x7f16a80aeda0 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16ada00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16a8072020 0x7f16a8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16a7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16ada00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16a8072020 0x7f16a8070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:53216/0 (socket says 172.21.15.87:53216) 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16a7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a8071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35814/0 (socket says 172.21.15.87:35814) 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16ada00640 1 -- 172.21.15.87:0/267073948 learned_addr learned my addr 172.21.15.87:0/267073948 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16ada00640 1 -- 172.21.15.87:0/267073948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 msgr2=0x7f16a8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16ada00640 1 --2- 172.21.15.87:0/267073948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16ada00640 1 -- 172.21.15.87:0/267073948 >> v1:172.21.15.169:6789/0 conn(0x7f16a8159540 legacy=0x7f16a80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.426+0000 7f16ada00640 1 -- 172.21.15.87:0/267073948 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a815c8b0 con 0x7f16a8072020 2024-04-25T10:21:16.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16ada00640 1 --2- 172.21.15.87:0/267073948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8072020 0x7f16a8070350 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009e50 tx=0x7f16a001ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f92d81d051bc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16a7400640 1 -- 172.21.15.87:0/267073948 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00a58f0 con 0x7f16a8072020 2024-04-25T10:21:16.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16a7400640 1 -- 172.21.15.87:0/267073948 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a00a5a90 con 0x7f16a8072020 2024-04-25T10:21:16.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16a7400640 1 -- 172.21.15.87:0/267073948 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00a5e60 con 0x7f16a8072020 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 -- 172.21.15.87:0/267073948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8072020 msgr2=0x7f16a8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 --2- 172.21.15.87:0/267073948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8072020 0x7f16a8070350 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009e50 tx=0x7f16a001ca00 comp rx=0 tx=0).stop 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 -- 172.21.15.87:0/267073948 shutdown_connections 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 --2- 172.21.15.87:0/267073948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8072020 0x7f16a8070350 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 --2- 172.21.15.87:0/267073948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 -- 172.21.15.87:0/267073948 >> 172.21.15.87:0/267073948 conn(0x7f16a80669a0 msgr2=0x7f16a8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 -- 172.21.15.87:0/267073948 shutdown_connections 2024-04-25T10:21:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.430+0000 7f16afa00640 1 -- 172.21.15.87:0/267073948 wait complete. 2024-04-25T10:21:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 Processor -- start 2024-04-25T10:21:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 -- start start 2024-04-25T10:21:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a8072020 0x7f16a8141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a814f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8159540 0x7f16a815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80710a0 con 0x7f16a8072020 2024-04-25T10:21:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a806c0d0 con 0x7f16a8159540 2024-04-25T10:21:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8071220 con 0x7f16a80aeda0 2024-04-25T10:21:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a8072020 0x7f16a8141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a8072020 0x7f16a8141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36574/0 (socket says 172.21.15.87:36574) 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 -- 172.21.15.87:0/3758983176 learned_addr learned my addr 172.21.15.87:0/3758983176 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ada00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8159540 0x7f16a815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a814f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 -- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 msgr2=0x7f16a814f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a814f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 -- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8159540 msgr2=0x7f16a815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8159540 0x7f16a815ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 -- 172.21.15.87:0/3758983176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8141670 con 0x7f16a8072020 2024-04-25T10:21:16.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16ad000640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a8072020 0x7f16a8141130 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f169800c500 tx=0x7f169800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16a4c00640 1 -- 172.21.15.87:0/3758983176 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698016020 con 0x7f16a8072020 2024-04-25T10:21:16.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16a4c00640 1 -- 172.21.15.87:0/3758983176 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1698005220 con 0x7f16a8072020 2024-04-25T10:21:16.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16a4c00640 1 -- 172.21.15.87:0/3758983176 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698007030 con 0x7f16a8072020 2024-04-25T10:21:16.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a814fb30 con 0x7f16a8072020 2024-04-25T10:21:16.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.434+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a81500f0 con 0x7f16a8072020 2024-04-25T10:21:16.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16a4c00640 1 -- 172.21.15.87:0/3758983176 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16980071f0 con 0x7f16a8072020 2024-04-25T10:21:16.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16a4c00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1688066f60 0x7f1688069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16a4c00640 1 -- 172.21.15.87:0/3758983176 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1698084470 con 0x7f16a8072020 2024-04-25T10:21:16.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1688066f60 0x7f1688069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16b2e5f000 1 Processor -- start 2024-04-25T10:21:16.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16b2e5f000 1 -- start start 2024-04-25T10:21:16.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1688066f60 0x7f1688069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f16a80aed70 tx=0x7f1694009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bc2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.438+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b77f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113b78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601139c0ff0 con 0x560113bc1e30 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560113949150 con 0x560113b784b0 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601139bd090 con 0x560113bc2750 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bc2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bc2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36580/0 (socket says 172.21.15.87:36580) 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 -- 172.21.15.87:0/3514582479 learned_addr learned my addr 172.21.15.87:0/3514582479 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b77f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ada00640 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113b78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 -- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 msgr2=0x560113b77f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b77f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 -- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 msgr2=0x560113b78890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113b78890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 -- 172.21.15.87:0/3514582479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560113b570d0 con 0x560113bc1e30 2024-04-25T10:21:16.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bc2210 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1698082990 tx=0x7f169800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c1999c5393ee34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f169d600640 1 -- 172.21.15.87:0/3514582479 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169805d460 con 0x560113bc1e30 2024-04-25T10:21:16.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f169d600640 1 -- 172.21.15.87:0/3514582479 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169805b0f0 con 0x560113bc1e30 2024-04-25T10:21:16.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f169d600640 1 -- 172.21.15.87:0/3514582479 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698051470 con 0x560113bc1e30 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3514582479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 msgr2=0x560113bc2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bc2210 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1698082990 tx=0x7f169800f000 comp rx=0 tx=0).stop 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3514582479 shutdown_connections 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113b78890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b77f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3514582479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bc2210 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3514582479 >> 172.21.15.87:0/3514582479 conn(0x560113b70020 msgr2=0x560113b72440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3514582479 shutdown_connections 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3514582479 wait complete. 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- start start 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 Processor -- start 2024-04-25T10:21:16.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113b784b0 0x560113bc1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc1e30 0x560113b7e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b7e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601139c5620 con 0x560113b784b0 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011397b690 con 0x560113bc1e30 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560113949a40 con 0x560113bc2750 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16b2e5f000 10 client.0 _ll_register_callbacks cb 0x560113b5ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113b784b0 0x560113bc1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113b784b0 0x560113bc1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36590/0 (socket says 172.21.15.87:36590) 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.442+0000 7f16ad000640 1 -- 172.21.15.87:0/769979693 learned_addr learned my addr 172.21.15.87:0/769979693 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16a7e00640 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc1e30 0x560113b7e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16ada00640 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b7e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16ad000640 1 -- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 msgr2=0x560113b7e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16ad000640 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b7e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16ad000640 1 -- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc1e30 msgr2=0x560113b7e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16ad000640 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc1e30 0x560113b7e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16ad000640 1 -- 172.21.15.87:0/769979693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560113a6e1c0 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16ad000640 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113b784b0 0x560113bc1790 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1698059060 tx=0x7f169801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393d482767a3cfee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 1 -- 172.21.15.87:0/769979693 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698082060 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 1 -- 172.21.15.87:0/769979693 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f16980840d0 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f169805b0f0 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 1 -- 172.21.15.87:0/769979693 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1698054250 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 10 client.8829 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x560113a6a170 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 1 -- 172.21.15.87:0/769979693 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169805b0f0 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 1 -- 172.21.15.87:0/769979693 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f16980543f0 con 0x560113b784b0 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 10 client.8829 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 10 client.8829 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 20 client.8829 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113b7f640 0x560113bd40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 4 client.8829 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:16.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56011395cc60 con 0x560113b7f640 2024-04-25T10:21:16.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16a7e00640 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113b7f640 0x560113bd40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16a7e00640 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113b7f640 0x560113bd40f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb8940d191db5253 server_cookie=b2e64df110779f5c in_seq=0 out_seq=0 2024-04-25T10:21:16.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 10 client.8829 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 1 -- 172.21.15.87:0/769979693 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56011395cc60 con 0x560113b7f640 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 10 client.8829 handle_command_reply: tid=0 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 client.8829 shutdown 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113b784b0 msgr2=0x560113bc1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113b784b0 0x560113bc1790 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1698059060 tx=0x7f169801c000 comp rx=0 tx=0).stop 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 shutdown_connections 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113b7f640 0x560113bd40f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc2750 0x560113b7e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc1e30 0x560113b7e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/769979693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113b784b0 0x560113bc1790 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 >> 172.21.15.87:0/769979693 conn(0x560113b70020 msgr2=0x560113b79120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f1692000640 0 client.8829 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 shutdown_connections 2024-04-25T10:21:16.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 1 -- 172.21.15.87:0/769979693 wait complete. 2024-04-25T10:21:16.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.446+0000 7f16b2e5f000 20 client.8829 trim_cache size 0 max 16384 2024-04-25T10:21:16.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 Processor -- start 2024-04-25T10:21:16.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 -- start start 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113bd6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc1e30 0x560113bd6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc2750 0x560113bcf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80b4670 con 0x560113bc2750 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a81528b0 con 0x560113b784b0 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80affb0 con 0x560113bc1e30 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc2750 0x560113bcf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc2750 0x560113bcf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36600/0 (socket says 172.21.15.87:36600) 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 -- 172.21.15.87:0/795240976 learned_addr learned my addr 172.21.15.87:0/795240976 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16a7e00640 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc1e30 0x560113bd6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ad000640 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113bd6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 -- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc1e30 msgr2=0x560113bd6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc1e30 0x560113bd6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 -- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 msgr2=0x560113bd6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113bd6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 -- 172.21.15.87:0/795240976 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560113a07de0 con 0x560113bc2750 2024-04-25T10:21:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f16ada00640 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc2750 0x560113bcf2a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f16a001a630 tx=0x7f16a00a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f60321ecb75c264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f169cc00640 1 -- 172.21.15.87:0/795240976 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00a58f0 con 0x560113bc2750 2024-04-25T10:21:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f169cc00640 1 -- 172.21.15.87:0/795240976 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a00c5070 con 0x560113bc2750 2024-04-25T10:21:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.462+0000 7f169cc00640 1 -- 172.21.15.87:0/795240976 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00c0640 con 0x560113bc2750 2024-04-25T10:21:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- 172.21.15.87:0/795240976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc2750 msgr2=0x560113bcf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc2750 0x560113bcf2a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f16a001a630 tx=0x7f16a00a7070 comp rx=0 tx=0).stop 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- 172.21.15.87:0/795240976 shutdown_connections 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc2750 0x560113bcf2a0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113bc1e30 0x560113bd6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/795240976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113b784b0 0x560113bd6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- 172.21.15.87:0/795240976 >> 172.21.15.87:0/795240976 conn(0x560113b70020 msgr2=0x560113bceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- 172.21.15.87:0/795240976 shutdown_connections 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- 172.21.15.87:0/795240976 wait complete. 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- start start 2024-04-25T10:21:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 Processor -- start 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113b784b0 0x560113bd40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bd4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc2750 0x560113bd4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011395be00 con 0x560113bc1e30 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a81522b0 con 0x560113bc2750 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601139c0300 con 0x560113b784b0 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 10 client.0 _ll_register_callbacks cb 0x560113b3f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bd4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bd4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36604/0 (socket says 172.21.15.87:36604) 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 -- 172.21.15.87:0/3444425118 learned_addr learned my addr 172.21.15.87:0/3444425118 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16ad000640 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113b784b0 0x560113bd40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16ada00640 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc2750 0x560113bd4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 -- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113b784b0 msgr2=0x560113bd40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113b784b0 0x560113bd40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 -- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc2750 msgr2=0x560113bd4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc2750 0x560113bd4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 -- 172.21.15.87:0/3444425118 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601139628e0 con 0x560113bc1e30 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bd4630 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f16a8154120 tx=0x7f1694004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb1c2e7c2217b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f1692000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f1692000640 1 -- 172.21.15.87:0/3444425118 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169402a250 con 0x560113bc1e30 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f1692000640 1 -- 172.21.15.87:0/3444425118 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f169400f040 con 0x560113bc1e30 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16a00c5070 con 0x560113bc1e30 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f1692000640 1 -- 172.21.15.87:0/3444425118 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1694038050 con 0x560113bc1e30 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 10 client.8835 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.466+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x560113a1a8f0 con 0x560113bc1e30 2024-04-25T10:21:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f1692000640 1 -- 172.21.15.87:0/3444425118 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a00c5070 con 0x560113bc1e30 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f1692000640 1 -- 172.21.15.87:0/3444425118 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f169403b020 con 0x560113bc1e30 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 10 client.8835 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 10 client.8835 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 20 client.8835 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113c27fd0 0x560113c2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 4 client.8835 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x5601139ce530 con 0x560113c27fd0 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16ad000640 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113c27fd0 0x560113c2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16ad000640 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113c27fd0 0x560113c2a3c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9cbe0cf0905734 server_cookie=c2e921f7a3c8d076 in_seq=0 out_seq=0 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f1692000640 10 client.8835 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f1692000640 1 -- 172.21.15.87:0/3444425118 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5601139ce530 con 0x560113c27fd0 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f1692000640 10 client.8835 handle_command_reply: tid=0 2024-04-25T10:21:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 client.8835 shutdown 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 msgr2=0x560113bd4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bd4630 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f16a8154120 tx=0x7f1694004740 comp rx=0 tx=0).stop 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 shutdown_connections 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560113c27fd0 0x560113c2a3c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560113bc2750 0x560113bd4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560113bc1e30 0x560113bd4630 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f1692000640 0 client.8835 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 --2- 172.21.15.87:0/3444425118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560113b784b0 0x560113bd40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 >> 172.21.15.87:0/3444425118 conn(0x560113b70020 msgr2=0x560113bcf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 shutdown_connections 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 1 -- 172.21.15.87:0/3444425118 wait complete. 2024-04-25T10:21:16.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16b2e5f000 20 client.8835 trim_cache size 0 max 16384 2024-04-25T10:21:16.478 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:16.478 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0 2024-04-25T10:21:16.478 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1688066f60 msgr2=0x7f1688069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1688066f60 0x7f1688069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f16a80aed70 tx=0x7f1694009190 comp rx=0 tx=0).stop 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a8072020 msgr2=0x7f16a8141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a8072020 0x7f16a8141130 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f169800c500 tx=0x7f169800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 shutdown_connections 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1688066f60 0x7f1688069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8159540 0x7f16a815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80aeda0 0x7f16a814f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 --2- 172.21.15.87:0/3758983176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a8072020 0x7f16a8141130 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 >> 172.21.15.87:0/3758983176 conn(0x7f16a80669a0 msgr2=0x7f16a815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 shutdown_connections 2024-04-25T10:21:16.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.470+0000 7f16afa00640 1 -- 172.21.15.87:0/3758983176 wait complete. 2024-04-25T10:21:16.492 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:21:16.492 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:21:16.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 Processor -- start 2024-04-25T10:21:16.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 -- start start 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f97a0161b90 0x7f97a00d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a00d0a70 0x7f97a00b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f97a0161f70 0x7f97a00b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a00b75e0 con 0x7f97a0161f70 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a00b4630 con 0x7f97a0161b90 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00b4880 con 0x7f97a00d0a70 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f97a0161f70 0x7f97a00b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f97a0161f70 0x7f97a00b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36614/0 (socket says 172.21.15.87:36614) 2024-04-25T10:21:16.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.626+0000 7f97a5400640 1 -- 172.21.15.87:0/1576585314 learned_addr learned my addr 172.21.15.87:0/1576585314 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a4a00640 1 --2- 172.21.15.87:0/1576585314 >> v2:172.21.15.169:3300/0 conn(0x7f97a0161b90 0x7f97a00d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f979fe00640 1 --2- 172.21.15.87:0/1576585314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a00d0a70 0x7f97a00b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5400640 1 -- 172.21.15.87:0/1576585314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a00d0a70 msgr2=0x7f97a00b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5400640 1 --2- 172.21.15.87:0/1576585314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a00d0a70 0x7f97a00b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5400640 1 -- 172.21.15.87:0/1576585314 >> v2:172.21.15.169:3300/0 conn(0x7f97a0161b90 msgr2=0x7f97a00d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5400640 1 --2- 172.21.15.87:0/1576585314 >> v2:172.21.15.169:3300/0 conn(0x7f97a0161b90 0x7f97a00d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5400640 1 -- 172.21.15.87:0/1576585314 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00b4b60 con 0x7f97a0161f70 2024-04-25T10:21:16.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f979fe00640 1 --2- 172.21.15.87:0/1576585314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a00d0a70 0x7f97a00b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:16.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5400640 1 --2- 172.21.15.87:0/1576585314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a0161f70 0x7f97a00b2cd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f97a0071940 tx=0x7f979801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1272e468a017d448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f979f400640 1 -- 172.21.15.87:0/1576585314 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980aa610 con 0x7f97a0161f70 2024-04-25T10:21:16.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f979f400640 1 -- 172.21.15.87:0/1576585314 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97980af070 con 0x7f97a0161f70 2024-04-25T10:21:16.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f979f400640 1 -- 172.21.15.87:0/1576585314 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980aa9d0 con 0x7f97a0161f70 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5e00640 1 -- 172.21.15.87:0/1576585314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a0161f70 msgr2=0x7f97a00b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.630+0000 7f97a5e00640 1 --2- 172.21.15.87:0/1576585314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a0161f70 0x7f97a00b2cd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f97a0071940 tx=0x7f979801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- 172.21.15.87:0/1576585314 shutdown_connections 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 --2- 172.21.15.87:0/1576585314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a0161f70 0x7f97a00b2cd0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 --2- 172.21.15.87:0/1576585314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a00d0a70 0x7f97a00b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 --2- 172.21.15.87:0/1576585314 >> v2:172.21.15.169:3300/0 conn(0x7f97a0161b90 0x7f97a00d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- 172.21.15.87:0/1576585314 >> 172.21.15.87:0/1576585314 conn(0x7f97a00d17e0 msgr2=0x7f97a00d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- 172.21.15.87:0/1576585314 shutdown_connections 2024-04-25T10:21:16.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- 172.21.15.87:0/1576585314 wait complete. 2024-04-25T10:21:16.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 Processor -- start 2024-04-25T10:21:16.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- start start 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a00d0a70 0x7f97a0155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a0161b90 0x7f97a00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a0161f70 0x7f97a00cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a015dde0 con 0x7f97a00d0a70 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a015dc60 con 0x7f97a0161f70 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a015df60 con 0x7f97a0161b90 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a00d0a70 0x7f97a0155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a00d0a70 0x7f97a0155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36630/0 (socket says 172.21.15.87:36630) 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 -- 172.21.15.87:0/3905945911 learned_addr learned my addr 172.21.15.87:0/3905945911 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f979fe00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a0161b90 0x7f97a00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5400640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a0161f70 0x7f97a00cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 -- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a0161b90 msgr2=0x7f97a00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a0161b90 0x7f97a00cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 -- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a0161f70 msgr2=0x7f97a00cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a0161f70 0x7f97a00cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 -- 172.21.15.87:0/3905945911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00cc390 con 0x7f97a00d0a70 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5400640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a0161f70 0x7f97a00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a4a00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a00d0a70 0x7f97a0155ff0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9790007f20 tx=0x7f979000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f979cc00640 1 -- 172.21.15.87:0/3905945911 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790014620 con 0x7f97a00d0a70 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f979cc00640 1 -- 172.21.15.87:0/3905945911 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790019070 con 0x7f97a00d0a70 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f979cc00640 1 -- 172.21.15.87:0/3905945911 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790014a00 con 0x7f97a00d0a70 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.634+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a00cc670 con 0x7f97a00d0a70 2024-04-25T10:21:16.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a00ccc30 con 0x7f97a00d0a70 2024-04-25T10:21:16.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f979cc00640 1 -- 172.21.15.87:0/3905945911 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9790014ba0 con 0x7f97a00d0a70 2024-04-25T10:21:16.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f979cc00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f979cc00640 1 -- 172.21.15.87:0/3905945911 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9790085a30 con 0x7f97a00d0a70 2024-04-25T10:21:16.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f979fe00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f97aa8be000 1 Processor -- start 2024-04-25T10:21:16.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f97aa8be000 1 -- start start 2024-04-25T10:21:16.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.638+0000 7f979fe00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9780066f60 0x7f9780069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f97a0161390 tx=0x7f978c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c1cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711c1d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560711a69ff0 con 0x560711c672f0 2024-04-25T10:21:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607119f2150 con 0x560711c669d0 2024-04-25T10:21:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560711a66090 con 0x560711c1d0a0 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c1cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c1cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36636/0 (socket says 172.21.15.87:36636) 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 -- 172.21.15.87:0/1575770267 learned_addr learned my addr 172.21.15.87:0/1575770267 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97a4a00640 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97a5400640 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711c1d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 -- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 msgr2=0x560711c1d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711c1d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 -- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 msgr2=0x560711c66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 -- 172.21.15.87:0/1575770267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560711c000d0 con 0x560711c672f0 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97a5400640 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711c1d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f979fe00640 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c1cb60 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f97a00d0eb0 tx=0x7f978c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38da5109a6944882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f9795600640 1 -- 172.21.15.87:0/1575770267 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c013420 con 0x560711c672f0 2024-04-25T10:21:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f9795600640 1 -- 172.21.15.87:0/1575770267 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c019070 con 0x560711c672f0 2024-04-25T10:21:16.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f9795600640 1 -- 172.21.15.87:0/1575770267 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c013800 con 0x560711c672f0 2024-04-25T10:21:16.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- 172.21.15.87:0/1575770267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 msgr2=0x560711c1cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c1cb60 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f97a00d0eb0 tx=0x7f978c00af90 comp rx=0 tx=0).stop 2024-04-25T10:21:16.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- 172.21.15.87:0/1575770267 shutdown_connections 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711c1d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c1cb60 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 --2- 172.21.15.87:0/1575770267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- 172.21.15.87:0/1575770267 >> 172.21.15.87:0/1575770267 conn(0x560711c19020 msgr2=0x560711c1b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- 172.21.15.87:0/1575770267 shutdown_connections 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- 172.21.15.87:0/1575770267 wait complete. 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 -- start start 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.642+0000 7f97aa8be000 1 Processor -- start 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c1d0a0 0x560711c4b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c4bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c672f0 0x560711c4c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560711a2e250 con 0x560711c1d0a0 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560711a24690 con 0x560711c669d0 2024-04-25T10:21:16.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607119f2a40 con 0x560711c672f0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 10 client.0 _ll_register_callbacks cb 0x560711c07e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c1d0a0 0x560711c4b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c1d0a0 0x560711c4b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36642/0 (socket says 172.21.15.87:36642) 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 -- 172.21.15.87:0/2078353363 learned_addr learned my addr 172.21.15.87:0/2078353363 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f979fe00640 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c4bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a5400640 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c672f0 0x560711c4c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 -- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c672f0 msgr2=0x560711c4c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c672f0 0x560711c4c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 -- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 msgr2=0x560711c4bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c4bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 -- 172.21.15.87:0/2078353363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560711b171c0 con 0x560711c1d0a0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97a4a00640 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c1d0a0 0x560711c4b7f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f979000ea70 tx=0x7f9790056310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2731ce8fd119cf7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 1 -- 172.21.15.87:0/2078353363 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790082060 con 0x560711c1d0a0 2024-04-25T10:21:16.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 1 -- 172.21.15.87:0/2078353363 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f979005e4e0 con 0x560711c1d0a0 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f978c019070 con 0x560711c1d0a0 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 1 -- 172.21.15.87:0/2078353363 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f979005e6e0 con 0x560711c1d0a0 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 10 client.8847 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x560711b13170 con 0x560711c1d0a0 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 1 -- 172.21.15.87:0/2078353363 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c019070 con 0x560711c1d0a0 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 1 -- 172.21.15.87:0/2078353363 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f97900741f0 con 0x560711c1d0a0 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 10 client.8847 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 10 client.8847 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 20 client.8847 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c76c80 0x560711c79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 4 client.8847 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560711a05c60 con 0x560711c76c80 2024-04-25T10:21:16.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f979fe00640 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c76c80 0x560711c79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f979fe00640 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c76c80 0x560711c79070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece5355bc4930414 server_cookie=a9397f0456a07ded in_seq=0 out_seq=0 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 10 client.8847 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 1 -- 172.21.15.87:0/2078353363 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560711a05c60 con 0x560711c76c80 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 10 client.8847 handle_command_reply: tid=0 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 client.8847 shutdown 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c1d0a0 msgr2=0x560711c4b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c1d0a0 0x560711c4b7f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f979000ea70 tx=0x7f9790056310 comp rx=0 tx=0).stop 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 shutdown_connections 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c76c80 0x560711c79070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c672f0 0x560711c4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711c4bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f978a000640 0 client.8847 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 --2- 172.21.15.87:0/2078353363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c1d0a0 0x560711c4b7f0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 >> 172.21.15.87:0/2078353363 conn(0x560711c19020 msgr2=0x560711c1bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 shutdown_connections 2024-04-25T10:21:16.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.646+0000 7f97aa8be000 1 -- 172.21.15.87:0/2078353363 wait complete. 2024-04-25T10:21:16.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.650+0000 7f97aa8be000 20 client.8847 trim_cache size 0 max 16384 2024-04-25T10:21:16.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 Processor -- start 2024-04-25T10:21:16.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 -- start start 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711cc3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711cc3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711cc3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a02028f0 con 0x560711c672f0 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a015a1a0 con 0x560711c669d0 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a01595a0 con 0x560711c1d0a0 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.662+0000 7f97a5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711cc3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711cc3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f979fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711cc3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711cc3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35896/0 (socket says 172.21.15.87:35896) 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711cc3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36658/0 (socket says 172.21.15.87:36658) 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 -- 172.21.15.87:0/385017453 learned_addr learned my addr 172.21.15.87:0/385017453 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 -- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 msgr2=0x560711cc3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711cc3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 -- 172.21.15.87:0/385017453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 msgr2=0x560711cc3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711cc3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 -- 172.21.15.87:0/385017453 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560711ab0de0 con 0x560711c1d0a0 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a5400640 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711cc3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f979fe00640 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711cc3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:16.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711cc3300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9790052630 tx=0x7f9790059000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c86c65e56842fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f9794c00640 1 -- 172.21.15.87:0/385017453 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790055b30 con 0x560711c1d0a0 2024-04-25T10:21:16.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f9794c00640 1 -- 172.21.15.87:0/385017453 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790052840 con 0x560711c1d0a0 2024-04-25T10:21:16.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f9794c00640 1 -- 172.21.15.87:0/385017453 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900741f0 con 0x560711c1d0a0 2024-04-25T10:21:16.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 msgr2=0x560711cc3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711cc3300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9790052630 tx=0x7f9790059000 comp rx=0 tx=0).stop 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- 172.21.15.87:0/385017453 shutdown_connections 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711cc3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c669d0 0x560711cc3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 --2- 172.21.15.87:0/385017453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c1d0a0 0x560711cc3300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- 172.21.15.87:0/385017453 >> 172.21.15.87:0/385017453 conn(0x560711c19020 msgr2=0x560711c73fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- 172.21.15.87:0/385017453 shutdown_connections 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- 172.21.15.87:0/385017453 wait complete. 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- start start 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 Processor -- start 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c1d0a0 0x560711c79380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c669d0 0x560711c798c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c79e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560711a04e00 con 0x560711c672f0 2024-04-25T10:21:16.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0159ba0 con 0x560711c1d0a0 2024-04-25T10:21:16.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560711a69300 con 0x560711c669d0 2024-04-25T10:21:16.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97aa8be000 10 client.0 _ll_register_callbacks cb 0x560711be8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:16.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c79e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c1d0a0 0x560711c79380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c79e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36664/0 (socket says 172.21.15.87:36664) 2024-04-25T10:21:16.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f97a5400640 1 -- 172.21.15.87:0/3965864263 learned_addr learned my addr 172.21.15.87:0/3965864263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.666+0000 7f979fe00640 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c669d0 0x560711c798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5400640 1 -- 172.21.15.87:0/3965864263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c669d0 msgr2=0x560711c798c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5400640 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c669d0 0x560711c798c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5400640 1 -- 172.21.15.87:0/3965864263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c1d0a0 msgr2=0x560711c79380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5400640 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c1d0a0 0x560711c79380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5400640 1 -- 172.21.15.87:0/3965864263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560711a0b8e0 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5400640 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c79e00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f979801cd90 tx=0x7f979801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c2bf705e4551b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 1 -- 172.21.15.87:0/3965864263 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980c3020 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 1 -- 172.21.15.87:0/3965864263 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f97980aacb0 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9790052840 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 1 -- 172.21.15.87:0/3965864263 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97980cd050 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 10 client.8853 fetch_fsmap learned FSMap version 80 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x560711ac38f0 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 1 -- 172.21.15.87:0/3965864263 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790052840 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 1 -- 172.21.15.87:0/3965864263 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f97980d0020 con 0x560711c672f0 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 10 client.8853 fetch_fsmap finished waiting for FSMap version 80 2024-04-25T10:21:16.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 10 client.8853 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 20 client.8853 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c7aae0 0x560711ccef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 4 client.8853 mds_command: new command op to 8523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 --> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560711a77530 con 0x560711c7aae0 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a4a00640 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c7aae0 0x560711ccef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a4a00640 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c7aae0 0x560711ccef60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=534d2d3412720913 server_cookie=a1ac01c1c0a13975 in_seq=0 out_seq=0 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 10 client.8853 ms_handle_connect on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 1 -- 172.21.15.87:0/3965864263 <== mds.0 v2:172.21.15.87:6835/3078360268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560711a77530 con 0x560711c7aae0 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 10 client.8853 handle_command_reply: tid=0 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 client.8853 shutdown 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 msgr2=0x560711c79e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c79e00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f979801cd90 tx=0x7f979801ad30 comp rx=0 tx=0).stop 2024-04-25T10:21:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 shutdown_connections 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:6835/3078360268,v1:172.21.15.87:6837/3078360268] conn(0x560711c7aae0 0x560711ccef60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560711c672f0 0x560711c79e00 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560711c669d0 0x560711c798c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 --2- 172.21.15.87:0/3965864263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560711c1d0a0 0x560711c79380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f978a000640 0 client.8853 ms_handle_reset on v2:172.21.15.87:6835/3078360268 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 >> 172.21.15.87:0/3965864263 conn(0x560711c19020 msgr2=0x560711c650d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 shutdown_connections 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 1 -- 172.21.15.87:0/3965864263 wait complete. 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97aa8be000 20 client.8853 trim_cache size 0 max 16384 2024-04-25T10:21:16.679 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:16.680 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:21:16.680 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:21:16.680 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9780066f60 msgr2=0x7f9780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9780066f60 0x7f9780069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f97a0161390 tx=0x7f978c008040 comp rx=0 tx=0).stop 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a00d0a70 msgr2=0x7f97a0155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a00d0a70 0x7f97a0155ff0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9790007f20 tx=0x7f979000e570 comp rx=0 tx=0).stop 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 shutdown_connections 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97a0161f70 0x7f97a00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97a0161b90 0x7f97a00cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 --2- 172.21.15.87:0/3905945911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97a00d0a70 0x7f97a0155ff0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 >> 172.21.15.87:0/3905945911 conn(0x7f97a00d17e0 msgr2=0x7f97a00b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 shutdown_connections 2024-04-25T10:21:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.670+0000 7f97a5e00640 1 -- 172.21.15.87:0/3905945911 wait complete. 2024-04-25T10:21:16.693 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:21:16.693 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:16.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a84a00640 1 Processor -- start 2024-04-25T10:21:16.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a84a00640 1 -- start start 2024-04-25T10:21:16.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a84a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a8015bc40 0x7f3a8006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a84a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a800cf930 0x7f3a800cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a84a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a800b2770 con 0x7f3a800cda10 2024-04-25T10:21:16.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a84a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8006c670 con 0x7f3a8015bc40 2024-04-25T10:21:16.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a84a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8006c950 con 0x7f3a800cf930 2024-04-25T10:21:16.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7d400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3a800cda10 0x7f3a800b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37930/0 (socket says 172.21.15.87:37930) 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7d400640 1 -- 172.21.15.87:0/2977830612 learned_addr learned my addr 172.21.15.87:0/2977830612 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7e800640 1 --2- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a800cf930 0x7f3a800cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7de00640 1 --2- 172.21.15.87:0/2977830612 >> v2:172.21.15.169:3300/0 conn(0x7f3a8015bc40 0x7f3a8006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7ca00640 1 -- 172.21.15.87:0/2977830612 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071606785 0 0) 0x7f3a800b2770 con 0x7f3a800cda10 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7ca00640 1 -- 172.21.15.87:0/2977830612 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a64003530 con 0x7f3a800cda10 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7de00640 1 -- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a800cf930 msgr2=0x7f3a800cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7de00640 1 --2- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a800cf930 0x7f3a800cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7de00640 1 -- 172.21.15.87:0/2977830612 >> v1:172.21.15.87:6789/0 conn(0x7f3a800cda10 legacy=0x7f3a800b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7de00640 1 -- 172.21.15.87:0/2977830612 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8006cc30 con 0x7f3a8015bc40 2024-04-25T10:21:16.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.826+0000 7f3a7e800640 1 --2- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a800cf930 0x7f3a800cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:16.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a7de00640 1 --2- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a8015bc40 0x7f3a8006c090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3a700097c0 tx=0x7f3a7001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aceea29b1a331061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a7ca00640 1 -- 172.21.15.87:0/2977830612 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700b0070 con 0x7f3a8015bc40 2024-04-25T10:21:16.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a7ca00640 1 -- 172.21.15.87:0/2977830612 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a700ab600 con 0x7f3a8015bc40 2024-04-25T10:21:16.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a7ca00640 1 -- 172.21.15.87:0/2977830612 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700ab9d0 con 0x7f3a8015bc40 2024-04-25T10:21:16.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 -- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a8015bc40 msgr2=0x7f3a8006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 --2- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a8015bc40 0x7f3a8006c090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3a700097c0 tx=0x7f3a7001c410 comp rx=0 tx=0).stop 2024-04-25T10:21:16.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 -- 172.21.15.87:0/2977830612 shutdown_connections 2024-04-25T10:21:16.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 --2- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a800cf930 0x7f3a800cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 --2- 172.21.15.87:0/2977830612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a8015bc40 0x7f3a8006c090 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 -- 172.21.15.87:0/2977830612 >> 172.21.15.87:0/2977830612 conn(0x7f3a80066b70 msgr2=0x7f3a80066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:16.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 -- 172.21.15.87:0/2977830612 shutdown_connections 2024-04-25T10:21:16.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.830+0000 7f3a84a00640 1 -- 172.21.15.87:0/2977830612 wait complete. 2024-04-25T10:21:16.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 Processor -- start 2024-04-25T10:21:16.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 -- start start 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a800cda10 0x7f3a8014cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a800cf930 0x7f3a80070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a8015bc40 0x7f3a80070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800b3890 con 0x7f3a800cda10 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800b3710 con 0x7f3a800cf930 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800b3a10 con 0x7f3a8015bc40 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a800cda10 0x7f3a8014cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a800cda10 0x7f3a8014cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36668/0 (socket says 172.21.15.87:36668) 2024-04-25T10:21:16.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 -- 172.21.15.87:0/1001671760 learned_addr learned my addr 172.21.15.87:0/1001671760 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7e800640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a8015bc40 0x7f3a80070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7d400640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a800cf930 0x7f3a80070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 -- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a8015bc40 msgr2=0x7f3a80070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a8015bc40 0x7f3a80070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 -- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a800cf930 msgr2=0x7f3a80070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a800cf930 0x7f3a80070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 -- 172.21.15.87:0/1001671760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800713c0 con 0x7f3a800cda10 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7d400640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a800cf930 0x7f3a80070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a7de00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a800cda10 0x7f3a8014cb50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f3a700ae040 tx=0x7f3a7001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a76000640 1 -- 172.21.15.87:0/1001671760 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700b0050 con 0x7f3a800cda10 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a76000640 1 -- 172.21.15.87:0/1001671760 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a700bc070 con 0x7f3a800cda10 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a76000640 1 -- 172.21.15.87:0/1001671760 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700b5a50 con 0x7f3a800cda10 2024-04-25T10:21:16.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8015def0 con 0x7f3a800cda10 2024-04-25T10:21:16.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8015e4b0 con 0x7f3a800cda10 2024-04-25T10:21:16.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.834+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8004ef30 con 0x7f3a800cda10 2024-04-25T10:21:16.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.838+0000 7f3a76000640 1 -- 172.21.15.87:0/1001671760 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a700ab4c0 con 0x7f3a800cda10 2024-04-25T10:21:16.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.838+0000 7f3a76000640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a60066f60 0x7f3a60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:16.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.838+0000 7f3a76000640 1 -- 172.21.15.87:0/1001671760 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3a7011f790 con 0x7f3a800cda10 2024-04-25T10:21:16.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.838+0000 7f3a7d400640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a60066f60 0x7f3a60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:16.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.838+0000 7f3a7d400640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a60066f60 0x7f3a60069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3a800aef40 tx=0x7f3a6c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:16.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:16.842+0000 7f3a76000640 1 -- 172.21.15.87:0/1001671760 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a70016090 con 0x7f3a800cda10 2024-04-25T10:21:17.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.026+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a800af690 con 0x7f3a800cda10 2024-04-25T10:21:17.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.026+0000 7f3a76000640 1 -- 172.21.15.87:0/1001671760 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3a700f07f0 con 0x7f3a800cda10 2024-04-25T10:21:17.030 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:17.030 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:17.030 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:17.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a60066f60 msgr2=0x7f3a60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a60066f60 0x7f3a60069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3a800aef40 tx=0x7f3a6c009040 comp rx=0 tx=0).stop 2024-04-25T10:21:17.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a800cda10 msgr2=0x7f3a8014cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a800cda10 0x7f3a8014cb50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f3a700ae040 tx=0x7f3a7001ac50 comp rx=0 tx=0).stop 2024-04-25T10:21:17.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 shutdown_connections 2024-04-25T10:21:17.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3a60066f60 0x7f3a60069420 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a8015bc40 0x7f3a80070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a800cf930 0x7f3a80070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 --2- 172.21.15.87:0/1001671760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a800cda10 0x7f3a8014cb50 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 >> 172.21.15.87:0/1001671760 conn(0x7f3a80066b70 msgr2=0x7f3a80159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:17.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 shutdown_connections 2024-04-25T10:21:17.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.030+0000 7f3a84a00640 1 -- 172.21.15.87:0/1001671760 wait complete. 2024-04-25T10:21:17.052 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:17.052 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:17.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.182+0000 7f08fb000640 1 Processor -- start 2024-04-25T10:21:17.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.182+0000 7f08fb000640 1 -- start start 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.182+0000 7f08fb000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08f40d0040 0x7f08f415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.182+0000 7f08fb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f415b740 0x7f08f40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.182+0000 7f08fb000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f4071050 con 0x7f08f41550b0 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.182+0000 7f08fb000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f415d040 con 0x7f08f40d0040 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.182+0000 7f08fb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40712f0 con 0x7f08f415b740 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08f40d0040 0x7f08f415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08f40d0040 0x7f08f415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36682/0 (socket says 172.21.15.87:36682) 2024-04-25T10:21:17.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 -- 172.21.15.87:0/415183122 learned_addr learned my addr 172.21.15.87:0/415183122 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:17.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f9000640 1 --2- 172.21.15.87:0/415183122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f415b740 0x7f08f40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:17.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 -- 172.21.15.87:0/415183122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f415b740 msgr2=0x7f08f40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:17.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 --2- 172.21.15.87:0/415183122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f415b740 0x7f08f40b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 -- 172.21.15.87:0/415183122 >> v1:172.21.15.169:6789/0 conn(0x7f08f41550b0 legacy=0x7f08f40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 -- 172.21.15.87:0/415183122 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40715d0 con 0x7f08f40d0040 2024-04-25T10:21:17.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f3400640 1 --2- 172.21.15.87:0/415183122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f40d0040 0x7f08f415b200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009870 tx=0x7f08e001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a194182f802015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:17.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f2a00640 1 -- 172.21.15.87:0/415183122 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a95f0 con 0x7f08f40d0040 2024-04-25T10:21:17.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f2a00640 1 -- 172.21.15.87:0/415183122 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e00a9790 con 0x7f08f40d0040 2024-04-25T10:21:17.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.186+0000 7f08f2a00640 1 -- 172.21.15.87:0/415183122 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a9b60 con 0x7f08f40d0040 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- 172.21.15.87:0/415183122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f40d0040 msgr2=0x7f08f415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 --2- 172.21.15.87:0/415183122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f40d0040 0x7f08f415b200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009870 tx=0x7f08e001c530 comp rx=0 tx=0).stop 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- 172.21.15.87:0/415183122 shutdown_connections 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 --2- 172.21.15.87:0/415183122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f415b740 0x7f08f40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 --2- 172.21.15.87:0/415183122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f40d0040 0x7f08f415b200 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- 172.21.15.87:0/415183122 >> 172.21.15.87:0/415183122 conn(0x7f08f4066b70 msgr2=0x7f08f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- 172.21.15.87:0/415183122 shutdown_connections 2024-04-25T10:21:17.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- 172.21.15.87:0/415183122 wait complete. 2024-04-25T10:21:17.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 Processor -- start 2024-04-25T10:21:17.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- start start 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f40d0040 0x7f08f4149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f41550b0 0x7f08f4147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08f415b740 0x7f08f40b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415bcd0 con 0x7f08f41550b0 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415bb50 con 0x7f08f415b740 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08fb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415c030 con 0x7f08f40d0040 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f41550b0 0x7f08f4147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f41550b0 0x7f08f4147af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36686/0 (socket says 172.21.15.87:36686) 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 -- 172.21.15.87:0/2866181914 learned_addr learned my addr 172.21.15.87:0/2866181914 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3e00640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f40d0040 0x7f08f4149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f9000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08f415b740 0x7f08f40b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 -- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f40d0040 msgr2=0x7f08f4149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f40d0040 0x7f08f4149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 -- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08f415b740 msgr2=0x7f08f40b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08f415b740 0x7f08f40b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 -- 172.21.15.87:0/2866181914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f41495e0 con 0x7f08f41550b0 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f9000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08f415b740 0x7f08f40b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.190+0000 7f08f3400640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f41550b0 0x7f08f4147af0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f08e00ab040 tx=0x7f08e001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08ebe00640 1 -- 172.21.15.87:0/2866181914 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00b1800 con 0x7f08f41550b0 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08ebe00640 1 -- 172.21.15.87:0/2866181914 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e00b19c0 con 0x7f08f41550b0 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08ebe00640 1 -- 172.21.15.87:0/2866181914 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00b1db0 con 0x7f08f41550b0 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f40b0480 con 0x7f08f41550b0 2024-04-25T10:21:17.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40b0a40 con 0x7f08f41550b0 2024-04-25T10:21:17.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f404ef30 con 0x7f08f41550b0 2024-04-25T10:21:17.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08ebe00640 1 -- 172.21.15.87:0/2866181914 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08e00a8350 con 0x7f08f41550b0 2024-04-25T10:21:17.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08ebe00640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08d4066f60 0x7f08d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:17.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08ebe00640 1 -- 172.21.15.87:0/2866181914 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f08e011f470 con 0x7f08f41550b0 2024-04-25T10:21:17.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.194+0000 7f08f3e00640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08d4066f60 0x7f08d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:17.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.198+0000 7f08f3e00640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08d4066f60 0x7f08d4069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f08e4001ea0 tx=0x7f08e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:17.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.198+0000 7f08ebe00640 1 -- 172.21.15.87:0/2866181914 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08e00c3070 con 0x7f08f41550b0 2024-04-25T10:21:17.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.406+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f08f40b3160 con 0x7f08f41550b0 2024-04-25T10:21:17.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.406+0000 7f08ebe00640 1 -- 172.21.15.87:0/2866181914 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1806 (secure 0 0 0) 0x7f08e00ad020 con 0x7f08f41550b0 2024-04-25T10:21:17.411 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:17.411 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":559,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076632719,"bytes_used":2603368448,"bytes_avail":770490744832,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":3059594,"recovering_keys_per_sec":0,"num_objects_recovered":27,"num_bytes_recovered":28311552,"num_keys_recovered":0,"write_bytes_sec":34658691,"read_op_per_sec":0,"write_op_per_sec":35},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:21:13.237152+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08d4066f60 msgr2=0x7f08d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08d4066f60 0x7f08d4069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f08e4001ea0 tx=0x7f08e4009040 comp rx=0 tx=0).stop 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f41550b0 msgr2=0x7f08f4147af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f41550b0 0x7f08f4147af0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f08e00ab040 tx=0x7f08e001ad70 comp rx=0 tx=0).stop 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 shutdown_connections 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08d4066f60 0x7f08d4069420 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08f415b740 0x7f08f40b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f41550b0 0x7f08f4147af0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 --2- 172.21.15.87:0/2866181914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08f40d0040 0x7f08f4149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 >> 172.21.15.87:0/2866181914 conn(0x7f08f4066b70 msgr2=0x7f08f415da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:17.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 shutdown_connections 2024-04-25T10:21:17.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:17.410+0000 7f08fb000640 1 -- 172.21.15.87:0/2866181914 wait complete. 2024-04-25T10:21:17.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:21:18.432 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:18.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.574+0000 7f3d81c00640 1 Processor -- start 2024-04-25T10:21:18.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.574+0000 7f3d81c00640 1 -- start start 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d81c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 0x7f3d7c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d81c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0b2770 con 0x7f3d7c0cf930 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d81c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c06c670 con 0x7f3d7c15bc40 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d81c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c06c950 con 0x7f3d7c0cda10 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3d7c0cf930 0x7f3d7c0b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42742/0 (socket says 172.21.15.87:42742) 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7b400640 1 -- 172.21.15.87:0/2036104049 learned_addr learned my addr 172.21.15.87:0/2036104049 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d73e00640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 0x7f3d7c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036811025 0 0) 0x7f3d7c0b2770 con 0x7f3d7c0cf930 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d60003530 con 0x7f3d7c0cf930 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167729657 0 0) 0x7f3d7c06c670 con 0x7f3d7c15bc40 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d7c0b2770 con 0x7f3d7c15bc40 2024-04-25T10:21:18.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2078325241 0 0) 0x7f3d60003530 con 0x7f3d7c0cf930 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 msgr2=0x7f3d7c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 0x7f3d7c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 >> v1:172.21.15.169:6789/0 conn(0x7f3d7c15bc40 legacy=0x7f3d7c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c06cc30 con 0x7f3d7c0cf930 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f3d74001a00 con 0x7f3d7c0cf930 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 >> v1:172.21.15.87:6789/0 conn(0x7f3d7c0cf930 legacy=0x7f3d7c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60003b80 0x7f3d60003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60004810 0x7f3d60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 0x7f3d60005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c06c950 con 0x7f3d60003b80 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6000bb60 con 0x7f3d60004810 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6000bd30 con 0x7f3d7c0cda10 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60003b80 0x7f3d60003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d73e00640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60004810 0x7f3d60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7b400640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 0x7f3d60005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7aa00640 1 -- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 msgr2=0x7f3d60005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:18.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c0cda10 0x7f3d60005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7aa00640 1 -- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60004810 msgr2=0x7f3d60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:18.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60004810 0x7f3d60004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.578+0000 7f3d7aa00640 1 -- 172.21.15.87:0/2036104049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d60003530 con 0x7f3d60003b80 2024-04-25T10:21:18.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60003b80 0x7f3d60003f80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c001900 tx=0x7f3d6c01c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371358d4d20d685d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:18.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c001520 con 0x7f3d60003b80 2024-04-25T10:21:18.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c0052f0 con 0x7f3d60003b80 2024-04-25T10:21:18.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d7a000640 1 -- 172.21.15.87:0/2036104049 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c003840 con 0x7f3d60003b80 2024-04-25T10:21:18.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 -- 172.21.15.87:0/2036104049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60003b80 msgr2=0x7f3d60003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 --2- 172.21.15.87:0/2036104049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60003b80 0x7f3d60003f80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c001900 tx=0x7f3d6c01c580 comp rx=0 tx=0).stop 2024-04-25T10:21:18.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d7b400640 1 -- 172.21.15.87:0/2036104049 reap_dead start 2024-04-25T10:21:18.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 -- 172.21.15.87:0/2036104049 shutdown_connections 2024-04-25T10:21:18.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 -- 172.21.15.87:0/2036104049 >> 172.21.15.87:0/2036104049 conn(0x7f3d7c066b70 msgr2=0x7f3d7c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:18.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 -- 172.21.15.87:0/2036104049 shutdown_connections 2024-04-25T10:21:18.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 -- 172.21.15.87:0/2036104049 wait complete. 2024-04-25T10:21:18.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 Processor -- start 2024-04-25T10:21:18.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.582+0000 7f3d81c00640 1 -- start start 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60003b80 0x7f3d7c15ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7c0b1450 0x7f3d7c1f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c1fb480 0x7f3d7c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c167b50 con 0x7f3d7c0b1450 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1679d0 con 0x7f3d60003b80 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c167cd0 con 0x7f3d7c1fb480 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7c0b1450 0x7f3d7c1f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7c0b1450 0x7f3d7c1f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32812/0 (socket says 172.21.15.87:32812) 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 -- 172.21.15.87:0/3651668728 learned_addr learned my addr 172.21.15.87:0/3651668728 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60003b80 0x7f3d7c15ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d7b400640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c1fb480 0x7f3d7c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 -- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c1fb480 msgr2=0x7f3d7c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c1fb480 0x7f3d7c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 -- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60003b80 msgr2=0x7f3d7c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60003b80 0x7f3d7c15ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 -- 172.21.15.87:0/3651668728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c1f7eb0 con 0x7f3d7c0b1450 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60003b80 0x7f3d7c15ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d73e00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7c0b1450 0x7f3d7c1f7610 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3d640119c0 tx=0x7f3d64011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:18.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d72a00640 1 -- 172.21.15.87:0/3651668728 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400fb50 con 0x7f3d7c0b1450 2024-04-25T10:21:18.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d72a00640 1 -- 172.21.15.87:0/3651668728 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6400fcf0 con 0x7f3d7c0b1450 2024-04-25T10:21:18.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c1f8190 con 0x7f3d7c0b1450 2024-04-25T10:21:18.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d72a00640 1 -- 172.21.15.87:0/3651668728 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400fea0 con 0x7f3d7c0b1450 2024-04-25T10:21:18.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c1fac80 con 0x7f3d7c0b1450 2024-04-25T10:21:18.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c04ef30 con 0x7f3d7c0b1450 2024-04-25T10:21:18.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.586+0000 7f3d72a00640 1 -- 172.21.15.87:0/3651668728 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d64012040 con 0x7f3d7c0b1450 2024-04-25T10:21:18.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.590+0000 7f3d72a00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.590+0000 7f3d72a00640 1 -- 172.21.15.87:0/3651668728 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3d64085770 con 0x7f3d7c0b1450 2024-04-25T10:21:18.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.590+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.590+0000 7f3d7aa00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d58066f60 0x7f3d58069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0ac000 tx=0x7f3d6c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:18.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.594+0000 7f3d72a00640 1 -- 172.21.15.87:0/3651668728 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d64019030 con 0x7f3d7c0b1450 2024-04-25T10:21:18.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.782+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7c06d960 con 0x7f3d7c0b1450 2024-04-25T10:21:18.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.782+0000 7f3d72a00640 1 -- 172.21.15.87:0/3651668728 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3d640566a0 con 0x7f3d7c0b1450 2024-04-25T10:21:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:18.786 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:18.787 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:18.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d58066f60 msgr2=0x7f3d58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d58066f60 0x7f3d58069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0ac000 tx=0x7f3d6c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7c0b1450 msgr2=0x7f3d7c1f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7c0b1450 0x7f3d7c1f7610 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3d640119c0 tx=0x7f3d64011e80 comp rx=0 tx=0).stop 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 shutdown_connections 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d7c1fb480 0x7f3d7c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7c0b1450 0x7f3d7c1f7610 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 --2- 172.21.15.87:0/3651668728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d60003b80 0x7f3d7c15ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 >> 172.21.15.87:0/3651668728 conn(0x7f3d7c066b70 msgr2=0x7f3d7c161f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 shutdown_connections 2024-04-25T10:21:18.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.786+0000 7f3d81c00640 1 -- 172.21.15.87:0/3651668728 wait complete. 2024-04-25T10:21:18.808 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:18.808 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:18.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.938+0000 7fa707e00640 1 Processor -- start 2024-04-25T10:21:18.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.938+0000 7fa707e00640 1 -- start start 2024-04-25T10:21:18.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa707e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7000d0040 0x7fa70015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa707e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa70015b740 0x7fa7000b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa707e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa700071050 con 0x7fa7001550b0 2024-04-25T10:21:18.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa707e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70015d040 con 0x7fa70015b740 2024-04-25T10:21:18.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa707e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7000712f0 con 0x7fa7000d0040 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa70015b740 0x7fa7000b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa70015b740 0x7fa7000b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32824/0 (socket says 172.21.15.87:32824) 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 -- 172.21.15.87:0/3197402520 learned_addr learned my addr 172.21.15.87:0/3197402520 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa704a00640 1 --2- 172.21.15.87:0/3197402520 >> v2:172.21.15.169:3300/0 conn(0x7fa7000d0040 0x7fa70015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 -- 172.21.15.87:0/3197402520 >> v2:172.21.15.169:3300/0 conn(0x7fa7000d0040 msgr2=0x7fa70015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 --2- 172.21.15.87:0/3197402520 >> v2:172.21.15.169:3300/0 conn(0x7fa7000d0040 0x7fa70015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 -- 172.21.15.87:0/3197402520 >> v1:172.21.15.169:6789/0 conn(0x7fa7001550b0 legacy=0x7fa7000cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 -- 172.21.15.87:0/3197402520 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000715d0 con 0x7fa70015b740 2024-04-25T10:21:18.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa705e00640 1 --2- 172.21.15.87:0/3197402520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa70015b740 0x7fa7000b3260 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fa700070160 tx=0x7fa6f801eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628b5e4dcd8c983a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.942+0000 7fa6ffe00640 1 -- 172.21.15.87:0/3197402520 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80a7a60 con 0x7fa70015b740 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa6ffe00640 1 -- 172.21.15.87:0/3197402520 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f80a7c00 con 0x7fa70015b740 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa6ffe00640 1 -- 172.21.15.87:0/3197402520 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80af3f0 con 0x7fa70015b740 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- 172.21.15.87:0/3197402520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa70015b740 msgr2=0x7fa7000b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 --2- 172.21.15.87:0/3197402520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa70015b740 0x7fa7000b3260 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fa700070160 tx=0x7fa6f801eb90 comp rx=0 tx=0).stop 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- 172.21.15.87:0/3197402520 shutdown_connections 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 --2- 172.21.15.87:0/3197402520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa70015b740 0x7fa7000b3260 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 --2- 172.21.15.87:0/3197402520 >> v2:172.21.15.169:3300/0 conn(0x7fa7000d0040 0x7fa70015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- 172.21.15.87:0/3197402520 >> 172.21.15.87:0/3197402520 conn(0x7fa700066b70 msgr2=0x7fa700066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:18.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- 172.21.15.87:0/3197402520 shutdown_connections 2024-04-25T10:21:18.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- 172.21.15.87:0/3197402520 wait complete. 2024-04-25T10:21:18.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 Processor -- start 2024-04-25T10:21:18.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- start start 2024-04-25T10:21:18.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7000d0040 0x7fa700147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7001550b0 0x7fa70014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa70015b740 0x7fa70006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70015bd10 con 0x7fa7001550b0 2024-04-25T10:21:18.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70015bb90 con 0x7fa70015b740 2024-04-25T10:21:18.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa707e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70015c030 con 0x7fa7000d0040 2024-04-25T10:21:18.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7001550b0 0x7fa70014ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7001550b0 0x7fa70014ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32834/0 (socket says 172.21.15.87:32834) 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 -- 172.21.15.87:0/2720794353 learned_addr learned my addr 172.21.15.87:0/2720794353 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa705e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa70015b740 0x7fa70006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa705400640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7000d0040 0x7fa700147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 -- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7000d0040 msgr2=0x7fa700147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7000d0040 0x7fa700147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 -- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa70015b740 msgr2=0x7fa70006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa70015b740 0x7fa70006cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa704a00640 1 -- 172.21.15.87:0/2720794353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa700150a70 con 0x7fa7001550b0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.946+0000 7fa705e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa70015b740 0x7fa70006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa704a00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7001550b0 0x7fa70014ffb0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d800 tx=0x7fa6ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa6fd600640 1 -- 172.21.15.87:0/2720794353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00b6c0 con 0x7fa7001550b0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa6fd600640 1 -- 172.21.15.87:0/2720794353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec019070 con 0x7fa7001550b0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70014b3c0 con 0x7fa7001550b0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa6fd600640 1 -- 172.21.15.87:0/2720794353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00ba60 con 0x7fa7001550b0 2024-04-25T10:21:18.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70014b940 con 0x7fa7001550b0 2024-04-25T10:21:18.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70004ef30 con 0x7fa7001550b0 2024-04-25T10:21:18.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa6fd600640 1 -- 172.21.15.87:0/2720794353 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6ec00bc00 con 0x7fa7001550b0 2024-04-25T10:21:18.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa6fd600640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa6e0066f60 0x7fa6e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:18.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa6fd600640 1 -- 172.21.15.87:0/2720794353 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa6ec085440 con 0x7fa7001550b0 2024-04-25T10:21:18.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.950+0000 7fa705400640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa6e0066f60 0x7fa6e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:18.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.954+0000 7fa705400640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa6e0066f60 0x7fa6e0069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa70006d820 tx=0x7fa6f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:18.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:18.954+0000 7fa6fd600640 1 -- 172.21.15.87:0/2720794353 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6ec0564a0 con 0x7fa7001550b0 2024-04-25T10:21:19.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.170+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa7000cee40 con 0x7fa7001550b0 2024-04-25T10:21:19.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.170+0000 7fa6fd600640 1 -- 172.21.15.87:0/2720794353 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1771 (secure 0 0 0) 0x7fa6ec055da0 con 0x7fa7001550b0 2024-04-25T10:21:19.174 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:19.174 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":561,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076636944,"bytes_used":2603368448,"bytes_avail":770490744832,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2763850,"recovering_keys_per_sec":0,"num_objects_recovered":27,"num_bytes_recovered":28311552,"num_keys_recovered":0,"write_bytes_sec":6991996,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:21:17.237973+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:19.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa6e0066f60 msgr2=0x7fa6e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa6e0066f60 0x7fa6e0069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa70006d820 tx=0x7fa6f4009040 comp rx=0 tx=0).stop 2024-04-25T10:21:19.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7001550b0 msgr2=0x7fa70014ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7001550b0 0x7fa70014ffb0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d800 tx=0x7fa6ec00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 shutdown_connections 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa6e0066f60 0x7fa6e0069420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa70015b740 0x7fa70006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7001550b0 0x7fa70014ffb0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 --2- 172.21.15.87:0/2720794353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7000d0040 0x7fa700147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 >> 172.21.15.87:0/2720794353 conn(0x7fa700066b70 msgr2=0x7fa7000cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 shutdown_connections 2024-04-25T10:21:19.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:19.174+0000 7fa707e00640 1 -- 172.21.15.87:0/2720794353 wait complete. 2024-04-25T10:21:19.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:21:20.198 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:20.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a4200640 1 Processor -- start 2024-04-25T10:21:20.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a4200640 1 -- start start 2024-04-25T10:21:20.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 0x7f559c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a4200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c071050 con 0x7f559c0cfe70 2024-04-25T10:21:20.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a4200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c15d040 con 0x7f559c15b910 2024-04-25T10:21:20.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c15bda0 con 0x7f559c1550b0 2024-04-25T10:21:20.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f559c0cfe70 0x7f559c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42756/0 (socket says 172.21.15.87:42756) 2024-04-25T10:21:20.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 -- 172.21.15.87:0/2292228326 learned_addr learned my addr 172.21.15.87:0/2292228326 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:20.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a1800640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 0x7f559c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875028501 0 0) 0x7f559c071050 con 0x7f559c0cfe70 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5584003530 con 0x7f559c0cfe70 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2211325999 0 0) 0x7f5584003530 con 0x7f559c0cfe70 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 msgr2=0x7f559c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 0x7f559c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 >> v1:172.21.15.169:6789/0 conn(0x7f559c15b910 legacy=0x7f559c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0712f0 con 0x7f559c0cfe70 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5588001680 con 0x7f559c0cfe70 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 >> v1:172.21.15.87:6789/0 conn(0x7f559c0cfe70 legacy=0x7f559c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 0x7f55840039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f5584004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5584004d50 0x7f55840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558400ba00 con 0x7f5584003f30 2024-04-25T10:21:20.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558400bbb0 con 0x7f5584004d50 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558400dfc0 con 0x7f559c1550b0 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f5584004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a1800640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 0x7f55840039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a2200640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5584004d50 0x7f55840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 -- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 msgr2=0x7f55840039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c1550b0 0x7f55840039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 -- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5584004d50 msgr2=0x7f55840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5584004d50 0x7f55840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.342+0000 7f55a0e00640 1 -- 172.21.15.87:0/2292228326 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5584003530 con 0x7f5584003f30 2024-04-25T10:21:20.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a0e00640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f5584004810 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f558800c2c0 tx=0x7f558801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f061d5598ff427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588007d60 con 0x7f5584003f30 2024-04-25T10:21:20.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55880a9070 con 0x7f5584003f30 2024-04-25T10:21:20.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f559be00640 1 -- 172.21.15.87:0/2292228326 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588005d80 con 0x7f5584003f30 2024-04-25T10:21:20.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a4200640 1 -- 172.21.15.87:0/2292228326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 msgr2=0x7f5584004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a4200640 1 --2- 172.21.15.87:0/2292228326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f5584004810 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f558800c2c0 tx=0x7f558801cc60 comp rx=0 tx=0).stop 2024-04-25T10:21:20.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a2200640 1 -- 172.21.15.87:0/2292228326 reap_dead start 2024-04-25T10:21:20.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a4200640 1 -- 172.21.15.87:0/2292228326 shutdown_connections 2024-04-25T10:21:20.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a4200640 1 -- 172.21.15.87:0/2292228326 >> 172.21.15.87:0/2292228326 conn(0x7f559c066b70 msgr2=0x7f559c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a4200640 1 -- 172.21.15.87:0/2292228326 shutdown_connections 2024-04-25T10:21:20.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.346+0000 7f55a4200640 1 -- 172.21.15.87:0/2292228326 wait complete. 2024-04-25T10:21:20.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 Processor -- start 2024-04-25T10:21:20.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 -- start start 2024-04-25T10:21:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f559c157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c0a9430 0x7f559c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f559c1f6380 0x7f559c1f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c06c090 con 0x7f5584003f30 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0720b0 con 0x7f559c1f6380 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c06c210 con 0x7f559c0a9430 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f559c157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f559c157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32866/0 (socket says 172.21.15.87:32866) 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 -- 172.21.15.87:0/1025921287 learned_addr learned my addr 172.21.15.87:0/1025921287 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 -- 172.21.15.87:0/1025921287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c0a9430 msgr2=0x7f559c1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c0a9430 0x7f559c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 -- 172.21.15.87:0/1025921287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f559c1f6380 msgr2=0x7f559c1f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f559c1f6380 0x7f559c1f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 -- 172.21.15.87:0/1025921287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c1545b0 con 0x7f5584003f30 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a1800640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f559c157130 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f5590011a00 tx=0x7f5590011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f5599600640 1 -- 172.21.15.87:0/1025921287 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559000faf0 con 0x7f5584003f30 2024-04-25T10:21:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f5599600640 1 -- 172.21.15.87:0/1025921287 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559000fc90 con 0x7f5584003f30 2024-04-25T10:21:20.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f5599600640 1 -- 172.21.15.87:0/1025921287 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590014440 con 0x7f5584003f30 2024-04-25T10:21:20.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c0a98b0 con 0x7f5584003f30 2024-04-25T10:21:20.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.350+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c1582f0 con 0x7f5584003f30 2024-04-25T10:21:20.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.354+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559c04ef30 con 0x7f5584003f30 2024-04-25T10:21:20.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.354+0000 7f5599600640 1 -- 172.21.15.87:0/1025921287 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5590012040 con 0x7f5584003f30 2024-04-25T10:21:20.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.354+0000 7f5599600640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f557c066f60 0x7f557c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.354+0000 7f5599600640 1 -- 172.21.15.87:0/1025921287 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f55900856f0 con 0x7f5584003f30 2024-04-25T10:21:20.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.354+0000 7f55a0e00640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f557c066f60 0x7f557c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.354+0000 7f55a0e00640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f557c066f60 0x7f557c069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f55880ab000 tx=0x7f5588001a50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.358+0000 7f5599600640 1 -- 172.21.15.87:0/1025921287 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5590019030 con 0x7f5584003f30 2024-04-25T10:21:20.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.542+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f559c15cc20 con 0x7f5584003f30 2024-04-25T10:21:20.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.542+0000 7f5599600640 1 -- 172.21.15.87:0/1025921287 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f559001c360 con 0x7f5584003f30 2024-04-25T10:21:20.547 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:20.547 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:20.547 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f557c066f60 msgr2=0x7f557c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f557c066f60 0x7f557c069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f55880ab000 tx=0x7f5588001a50 comp rx=0 tx=0).stop 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 msgr2=0x7f559c157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f559c157130 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f5590011a00 tx=0x7f5590011ec0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 shutdown_connections 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f557c066f60 0x7f557c069420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f559c1f6380 0x7f559c1f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f559c0a9430 0x7f559c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 --2- 172.21.15.87:0/1025921287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5584003f30 0x7f559c157130 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 >> 172.21.15.87:0/1025921287 conn(0x7f559c066b70 msgr2=0x7f559c0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 shutdown_connections 2024-04-25T10:21:20.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.546+0000 7f55a4200640 1 -- 172.21.15.87:0/1025921287 wait complete. 2024-04-25T10:21:20.568 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:20.568 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.698+0000 7f1646a00640 1 Processor -- start 2024-04-25T10:21:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.698+0000 7f1646a00640 1 -- start start 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1646a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f164015b9e0 0x7f16400cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1646a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16400d1c30 0x7f16400cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1646a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400ccfb0 con 0x7f164015bdc0 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1646a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16400cd290 con 0x7f164015b9e0 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1646a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400cd570 con 0x7f16400d1c30 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f163fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f164015b9e0 0x7f16400cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1644a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16400d1c30 0x7f16400cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1644a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16400d1c30 0x7f16400cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:56254/0 (socket says 172.21.15.87:56254) 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f163f400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f164015bdc0 0x7f16400d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:52600/0 (socket says 172.21.15.87:52600) 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1644a00640 1 -- 172.21.15.87:0/3172720670 learned_addr learned my addr 172.21.15.87:0/3172720670 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f163fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f164015b9e0 0x7f16400cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:55920/0 (socket says 172.21.15.87:55920) 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1644a00640 1 -- 172.21.15.87:0/3172720670 >> v2:172.21.15.169:3300/0 conn(0x7f164015b9e0 msgr2=0x7f16400cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1644a00640 1 --2- 172.21.15.87:0/3172720670 >> v2:172.21.15.169:3300/0 conn(0x7f164015b9e0 0x7f16400cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1644a00640 1 -- 172.21.15.87:0/3172720670 >> v1:172.21.15.169:6789/0 conn(0x7f164015bdc0 legacy=0x7f16400d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.702+0000 7f1644a00640 1 -- 172.21.15.87:0/3172720670 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16400cd850 con 0x7f16400d1c30 2024-04-25T10:21:20.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1644a00640 1 --2- 172.21.15.87:0/3172720670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16400d1c30 0x7f16400cc950 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1638009e50 tx=0x7f163801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ce6316be5f2cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f163ea00640 1 -- 172.21.15.87:0/3172720670 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380a5660 con 0x7f16400d1c30 2024-04-25T10:21:20.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f163ea00640 1 -- 172.21.15.87:0/3172720670 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16380a5800 con 0x7f16400d1c30 2024-04-25T10:21:20.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f163ea00640 1 -- 172.21.15.87:0/3172720670 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380a5bd0 con 0x7f16400d1c30 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 -- 172.21.15.87:0/3172720670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16400d1c30 msgr2=0x7f16400cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 --2- 172.21.15.87:0/3172720670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16400d1c30 0x7f16400cc950 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1638009e50 tx=0x7f163801c9a0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 -- 172.21.15.87:0/3172720670 shutdown_connections 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 --2- 172.21.15.87:0/3172720670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16400d1c30 0x7f16400cc950 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 --2- 172.21.15.87:0/3172720670 >> v2:172.21.15.169:3300/0 conn(0x7f164015b9e0 0x7f16400cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 -- 172.21.15.87:0/3172720670 >> 172.21.15.87:0/3172720670 conn(0x7f164009a9d0 msgr2=0x7f164009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 -- 172.21.15.87:0/3172720670 shutdown_connections 2024-04-25T10:21:20.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 -- 172.21.15.87:0/3172720670 wait complete. 2024-04-25T10:21:20.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 Processor -- start 2024-04-25T10:21:20.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.706+0000 7f1646a00640 1 -- start start 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16400d1c30 0x7f16400b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f164015b9e0 0x7f16400b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164015bdc0 0x7f16401519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16400c65b0 con 0x7f164015bdc0 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16400c6430 con 0x7f16400d1c30 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400c6730 con 0x7f164015b9e0 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164015bdc0 0x7f16401519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164015bdc0 0x7f16401519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32868/0 (socket says 172.21.15.87:32868) 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 -- 172.21.15.87:0/1623737866 learned_addr learned my addr 172.21.15.87:0/1623737866 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f163fe00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16400d1c30 0x7f16400b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f163f400640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f164015b9e0 0x7f16400b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 -- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f164015b9e0 msgr2=0x7f16400b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f164015b9e0 0x7f16400b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 -- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16400d1c30 msgr2=0x7f16400b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16400d1c30 0x7f16400b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 -- 172.21.15.87:0/1623737866 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16400b5420 con 0x7f164015bdc0 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f163f400640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f164015b9e0 0x7f16400b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1644a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164015bdc0 0x7f16401519e0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f163801a5d0 tx=0x7f1638019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1637e00640 1 -- 172.21.15.87:0/1623737866 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380bc020 con 0x7f164015bdc0 2024-04-25T10:21:20.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1637e00640 1 -- 172.21.15.87:0/1623737866 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16380b8690 con 0x7f164015bdc0 2024-04-25T10:21:20.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16400b56c0 con 0x7f164015bdc0 2024-04-25T10:21:20.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1637e00640 1 -- 172.21.15.87:0/1623737866 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638009510 con 0x7f164015bdc0 2024-04-25T10:21:20.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16400c8c20 con 0x7f164015bdc0 2024-04-25T10:21:20.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164004ef30 con 0x7f164015bdc0 2024-04-25T10:21:20.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.710+0000 7f1637e00640 1 -- 172.21.15.87:0/1623737866 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16380bf020 con 0x7f164015bdc0 2024-04-25T10:21:20.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.714+0000 7f1637e00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1620066f60 0x7f1620069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.714+0000 7f1637e00640 1 -- 172.21.15.87:0/1623737866 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f163811fe60 con 0x7f164015bdc0 2024-04-25T10:21:20.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.714+0000 7f163fe00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1620066f60 0x7f1620069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.714+0000 7f163fe00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1620066f60 0x7f1620069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1630015230 tx=0x7f163000f040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.718+0000 7f1637e00640 1 -- 172.21.15.87:0/1623737866 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16380c6030 con 0x7f164015bdc0 2024-04-25T10:21:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.922+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f16400d19e0 con 0x7f164015bdc0 2024-04-25T10:21:20.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.922+0000 7f1637e00640 1 -- 172.21.15.87:0/1623737866 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1771 (secure 0 0 0) 0x7f1638016620 con 0x7f164015bdc0 2024-04-25T10:21:20.926 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:20.926 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":562,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076636944,"bytes_used":2603368448,"bytes_avail":770490744832,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2358850,"recovering_keys_per_sec":0,"num_objects_recovered":27,"num_bytes_recovered":28311552,"num_keys_recovered":0,"write_bytes_sec":5967427,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:21:17.237973+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:20.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1620066f60 msgr2=0x7f1620069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1620066f60 0x7f1620069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1630015230 tx=0x7f163000f040 comp rx=0 tx=0).stop 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164015bdc0 msgr2=0x7f16401519e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164015bdc0 0x7f16401519e0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f163801a5d0 tx=0x7f1638019740 comp rx=0 tx=0).stop 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 shutdown_connections 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1620066f60 0x7f1620069420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164015bdc0 0x7f16401519e0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f164015b9e0 0x7f16400b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 --2- 172.21.15.87:0/1623737866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16400d1c30 0x7f16400b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 >> 172.21.15.87:0/1623737866 conn(0x7f164009a9d0 msgr2=0x7f164006e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 shutdown_connections 2024-04-25T10:21:20.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:20.926+0000 7f1646a00640 1 -- 172.21.15.87:0/1623737866 wait complete. 2024-04-25T10:21:20.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:21:21.947 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b59200640 1 Processor -- start 2024-04-25T10:21:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b59200640 1 -- start start 2024-04-25T10:21:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b59200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7b5415dab0 0x7f7b54162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b59200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b540b0210 con 0x7f7b540b57e0 2024-04-25T10:21:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b59200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b540aff30 con 0x7f7b540669a0 2024-04-25T10:21:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b59200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b54161840 con 0x7f7b5415dab0 2024-04-25T10:21:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b49400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7b540b57e0 0x7f7b540af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42762/0 (socket says 172.21.15.87:42762) 2024-04-25T10:21:22.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b49400640 1 -- 172.21.15.87:0/1104344683 learned_addr learned my addr 172.21.15.87:0/1104344683 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51e00640 1 --2- 172.21.15.87:0/1104344683 >> v2:172.21.15.169:3300/0 conn(0x7f7b5415dab0 0x7f7b54162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228829310 0 0) 0x7f7b540b0210 con 0x7f7b540b57e0 2024-04-25T10:21:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b38003530 con 0x7f7b540b57e0 2024-04-25T10:21:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377442401 0 0) 0x7f7b540aff30 con 0x7f7b540669a0 2024-04-25T10:21:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b540b0210 con 0x7f7b540669a0 2024-04-25T10:21:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 419499620 0 0) 0x7f7b38003530 con 0x7f7b540b57e0 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 >> v2:172.21.15.169:3300/0 conn(0x7f7b5415dab0 msgr2=0x7f7b54162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 --2- 172.21.15.87:0/1104344683 >> v2:172.21.15.169:3300/0 conn(0x7f7b5415dab0 0x7f7b54162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 >> v1:172.21.15.169:6789/0 conn(0x7f7b540669a0 legacy=0x7f7b54066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b54161b20 con 0x7f7b540b57e0 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f7b3c001680 con 0x7f7b540b57e0 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 >> v1:172.21.15.87:6789/0 conn(0x7f7b540b57e0 legacy=0x7f7b540af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b38003ba0 0x7f7b38003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b38004810 0x7f7b38004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b3800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b54161840 con 0x7f7b38005220 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3800c150 con 0x7f7b38004810 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3800c350 con 0x7f7b38003ba0 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b51e00640 1 --2- 172.21.15.87:0/1104344683 >> v2:172.21.15.169:3300/0 conn(0x7f7b5415dab0 0x7f7b54162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.090+0000 7f7b52800640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b3800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b49400640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b38004810 0x7f7b38004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b51e00640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b38003ba0 0x7f7b38003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b52800640 1 -- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b38003ba0 msgr2=0x7f7b38003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b52800640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b38003ba0 0x7f7b38003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b52800640 1 -- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b38004810 msgr2=0x7f7b38004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b52800640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b38004810 0x7f7b38004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b52800640 1 -- 172.21.15.87:0/1104344683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b38003530 con 0x7f7b38005220 2024-04-25T10:21:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b52800640 1 -- 172.21.15.87:0/1104344683 reap_dead start 2024-04-25T10:21:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b52800640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b3800b9d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c001fc0 tx=0x7f7b4c01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb62a5f585af583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c001c10 con 0x7f7b38005220 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c0a8c90 con 0x7f7b38005220 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b51400640 1 -- 172.21.15.87:0/1104344683 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c003b80 con 0x7f7b38005220 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b59200640 1 -- 172.21.15.87:0/1104344683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 msgr2=0x7f7b3800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b59200640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b3800b9d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c001fc0 tx=0x7f7b4c01f070 comp rx=0 tx=0).stop 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b59200640 1 -- 172.21.15.87:0/1104344683 shutdown_connections 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b59200640 1 --2- 172.21.15.87:0/1104344683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b3800b9d0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b59200640 1 -- 172.21.15.87:0/1104344683 >> 172.21.15.87:0/1104344683 conn(0x7f7b5406ac90 msgr2=0x7f7b5406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b59200640 1 -- 172.21.15.87:0/1104344683 shutdown_connections 2024-04-25T10:21:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.094+0000 7f7b59200640 1 -- 172.21.15.87:0/1104344683 wait complete. 2024-04-25T10:21:22.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 Processor -- start 2024-04-25T10:21:22.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 -- start start 2024-04-25T10:21:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b540b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541f3e50 0x7f7b540b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b541f42b0 0x7f7b54151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b541624a0 con 0x7f7b38005220 2024-04-25T10:21:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b54162320 con 0x7f7b541f3e50 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b54162620 con 0x7f7b541f42b0 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b540b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b540b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32888/0 (socket says 172.21.15.87:32888) 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 -- 172.21.15.87:0/1076697173 learned_addr learned my addr 172.21.15.87:0/1076697173 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b49400640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541f3e50 0x7f7b540b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b52800640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b541f42b0 0x7f7b54151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 -- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b541f42b0 msgr2=0x7f7b54151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b541f42b0 0x7f7b54151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 -- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541f3e50 msgr2=0x7f7b540b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541f3e50 0x7f7b540b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 -- 172.21.15.87:0/1076697173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540b0b30 con 0x7f7b38005220 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b51e00640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b540b0400 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7b44011930 tx=0x7f7b44011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b4aa00640 1 -- 172.21.15.87:0/1076697173 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400fbd0 con 0x7f7b38005220 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b4aa00640 1 -- 172.21.15.87:0/1076697173 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4400fd70 con 0x7f7b38005220 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b540b0e10 con 0x7f7b38005220 2024-04-25T10:21:22.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b4aa00640 1 -- 172.21.15.87:0/1076697173 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44015660 con 0x7f7b38005220 2024-04-25T10:21:22.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.098+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b540b13d0 con 0x7f7b38005220 2024-04-25T10:21:22.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.102+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5404ef30 con 0x7f7b38005220 2024-04-25T10:21:22.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.102+0000 7f7b4aa00640 1 -- 172.21.15.87:0/1076697173 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b44015800 con 0x7f7b38005220 2024-04-25T10:21:22.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.102+0000 7f7b4aa00640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.102+0000 7f7b4aa00640 1 -- 172.21.15.87:0/1076697173 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7b44085b90 con 0x7f7b38005220 2024-04-25T10:21:22.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.102+0000 7f7b49400640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.102+0000 7f7b49400640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0017f0 tx=0x7f7b3c007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.106+0000 7f7b4aa00640 1 -- 172.21.15.87:0/1076697173 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b44019030 con 0x7f7b38005220 2024-04-25T10:21:22.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.290+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b540719b0 con 0x7f7b38005220 2024-04-25T10:21:22.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.290+0000 7f7b4aa00640 1 -- 172.21.15.87:0/1076697173 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7b44056ac0 con 0x7f7b38005220 2024-04-25T10:21:22.296 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:22.296 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:22.296 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 msgr2=0x7f7b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0017f0 tx=0x7f7b3c007040 comp rx=0 tx=0).stop 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 msgr2=0x7f7b540b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b540b0400 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7b44011930 tx=0x7f7b44011df0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 shutdown_connections 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b541f42b0 0x7f7b54151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b541f3e50 0x7f7b540b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 --2- 172.21.15.87:0/1076697173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b38005220 0x7f7b540b0400 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 >> 172.21.15.87:0/1076697173 conn(0x7f7b5406ac90 msgr2=0x7f7b5406e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 shutdown_connections 2024-04-25T10:21:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.294+0000 7f7b59200640 1 -- 172.21.15.87:0/1076697173 wait complete. 2024-04-25T10:21:22.316 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:22.316 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.446+0000 7ff51c400640 1 Processor -- start 2024-04-25T10:21:22.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.446+0000 7ff51c400640 1 -- start start 2024-04-25T10:21:22.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff51c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5140d0040 0x7ff51415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff51c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff51415b740 0x7ff5140b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff51c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff514071050 con 0x7ff5141550b0 2024-04-25T10:21:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff51c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51415d040 con 0x7ff51415b740 2024-04-25T10:21:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff51c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5140712f0 con 0x7ff5140d0040 2024-04-25T10:21:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5140d0040 0x7ff51415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff51a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff51415b740 0x7ff5140b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5140d0040 0x7ff51415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:56294/0 (socket says 172.21.15.87:56294) 2024-04-25T10:21:22.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519000640 1 -- 172.21.15.87:0/1470248437 learned_addr learned my addr 172.21.15.87:0/1470248437 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:22.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519a00640 1 --1- 172.21.15.87:0/1470248437 >> v1:172.21.15.169:6789/0 conn(0x7ff5141550b0 0x7ff5140cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:52614/0 (socket says 172.21.15.87:52614) 2024-04-25T10:21:22.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519000640 1 -- 172.21.15.87:0/1470248437 >> v2:172.21.15.169:3300/0 conn(0x7ff51415b740 msgr2=0x7ff5140b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:22.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519000640 1 --2- 172.21.15.87:0/1470248437 >> v2:172.21.15.169:3300/0 conn(0x7ff51415b740 0x7ff5140b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519000640 1 -- 172.21.15.87:0/1470248437 >> v1:172.21.15.169:6789/0 conn(0x7ff5141550b0 legacy=0x7ff5140cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff519000640 1 -- 172.21.15.87:0/1470248437 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5140715d0 con 0x7ff5140d0040 2024-04-25T10:21:22.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.450+0000 7ff513e00640 1 -- 172.21.15.87:0/1470248437 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098750028 0 0) 0x7ff514071050 con 0x7ff5141550b0 2024-04-25T10:21:22.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff519000640 1 --2- 172.21.15.87:0/1470248437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5140d0040 0x7ff51415b200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff500009870 tx=0x7ff50001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e19ef8e595b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff513e00640 1 -- 172.21.15.87:0/1470248437 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000a95f0 con 0x7ff5140d0040 2024-04-25T10:21:22.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff513e00640 1 -- 172.21.15.87:0/1470248437 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5000a9790 con 0x7ff5140d0040 2024-04-25T10:21:22.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff513e00640 1 -- 172.21.15.87:0/1470248437 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000a9b60 con 0x7ff5140d0040 2024-04-25T10:21:22.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 -- 172.21.15.87:0/1470248437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5140d0040 msgr2=0x7ff51415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 --2- 172.21.15.87:0/1470248437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5140d0040 0x7ff51415b200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff500009870 tx=0x7ff50001c530 comp rx=0 tx=0).stop 2024-04-25T10:21:22.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 -- 172.21.15.87:0/1470248437 shutdown_connections 2024-04-25T10:21:22.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 --2- 172.21.15.87:0/1470248437 >> v2:172.21.15.169:3300/0 conn(0x7ff51415b740 0x7ff5140b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 --2- 172.21.15.87:0/1470248437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5140d0040 0x7ff51415b200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 -- 172.21.15.87:0/1470248437 >> 172.21.15.87:0/1470248437 conn(0x7ff514066b70 msgr2=0x7ff514066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 -- 172.21.15.87:0/1470248437 shutdown_connections 2024-04-25T10:21:22.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 -- 172.21.15.87:0/1470248437 wait complete. 2024-04-25T10:21:22.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 Processor -- start 2024-04-25T10:21:22.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.454+0000 7ff51c400640 1 -- start start 2024-04-25T10:21:22.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5140d0040 0x7ff51414af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5141550b0 0x7ff514158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51415b740 0x7ff5140b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415bcd0 con 0x7ff5140d0040 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415bb50 con 0x7ff51415b740 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415c030 con 0x7ff5141550b0 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5140d0040 0x7ff51414af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5140d0040 0x7ff51414af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32900/0 (socket says 172.21.15.87:32900) 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 -- 172.21.15.87:0/2937015686 learned_addr learned my addr 172.21.15.87:0/2937015686 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519000640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5141550b0 0x7ff514158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51a400640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51415b740 0x7ff5140b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 -- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5141550b0 msgr2=0x7ff514158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5141550b0 0x7ff514158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 -- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51415b740 msgr2=0x7ff5140b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51415b740 0x7ff5140b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 -- 172.21.15.87:0/2937015686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514155530 con 0x7ff5140d0040 2024-04-25T10:21:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff519a00640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5140d0040 0x7ff51414af10 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ff508001570 tx=0x7ff50800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff511600640 1 -- 172.21.15.87:0/2937015686 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508018070 con 0x7ff5140d0040 2024-04-25T10:21:22.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff511600640 1 -- 172.21.15.87:0/2937015686 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50800a640 con 0x7ff5140d0040 2024-04-25T10:21:22.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff514159370 con 0x7ff5140d0040 2024-04-25T10:21:22.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff511600640 1 -- 172.21.15.87:0/2937015686 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800aa30 con 0x7ff5140d0040 2024-04-25T10:21:22.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff514159930 con 0x7ff5140d0040 2024-04-25T10:21:22.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51404ef30 con 0x7ff5140d0040 2024-04-25T10:21:22.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.458+0000 7ff511600640 1 -- 172.21.15.87:0/2937015686 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff50801b070 con 0x7ff5140d0040 2024-04-25T10:21:22.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.462+0000 7ff511600640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f90 0x7ff4f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.462+0000 7ff511600640 1 -- 172.21.15.87:0/2937015686 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff5080846b0 con 0x7ff5140d0040 2024-04-25T10:21:22.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.462+0000 7ff519000640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f90 0x7ff4f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.462+0000 7ff519000640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f90 0x7ff4f4069450 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ae000 tx=0x7ff5000b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.466+0000 7ff511600640 1 -- 172.21.15.87:0/2937015686 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff508055710 con 0x7ff5140d0040 2024-04-25T10:21:22.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.674+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff5140671a0 con 0x7ff5140d0040 2024-04-25T10:21:22.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.674+0000 7ff511600640 1 -- 172.21.15.87:0/2937015686 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1771 (secure 0 0 0) 0x7ff508055010 con 0x7ff5140d0040 2024-04-25T10:21:22.678 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:22.678 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":564,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076636944,"bytes_used":2603368448,"bytes_avail":770490744832,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2358845,"recovering_keys_per_sec":0,"num_objects_recovered":27,"num_bytes_recovered":28311552,"num_keys_recovered":0,"write_bytes_sec":5967413,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:21:17.237973+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f90 msgr2=0x7ff4f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f90 0x7ff4f4069450 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ae000 tx=0x7ff5000b3040 comp rx=0 tx=0).stop 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5140d0040 msgr2=0x7ff51414af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5140d0040 0x7ff51414af10 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ff508001570 tx=0x7ff50800cd20 comp rx=0 tx=0).stop 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 shutdown_connections 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4f4066f90 0x7ff4f4069450 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff51415b740 0x7ff5140b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5141550b0 0x7ff514158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 --2- 172.21.15.87:0/2937015686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5140d0040 0x7ff51414af10 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 >> 172.21.15.87:0/2937015686 conn(0x7ff514066b70 msgr2=0x7ff51415c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 shutdown_connections 2024-04-25T10:21:22.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:22.678+0000 7ff51c400640 1 -- 172.21.15.87:0/2937015686 wait complete. 2024-04-25T10:21:22.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:21:23.606 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:23.610 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:23.702 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:23.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.842+0000 7fe34a000640 1 Processor -- start 2024-04-25T10:21:23.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.842+0000 7fe34a000640 1 -- start start 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe34a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe34415b750 0x7fe3440cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe34a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3440cd590 0x7fe3440cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe34a000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3440b2460 con 0x7fe3440b3f80 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe34a000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3440b2d70 con 0x7fe3440cd590 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe34a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe344070fb0 con 0x7fe34415b750 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe33be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3440cd590 0x7fe3440cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe349600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe3440b3f80 0x7fe3440b3200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42770/0 (socket says 172.21.15.87:42770) 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe349600640 1 -- 172.21.15.87:0/3711107539 learned_addr learned my addr 172.21.15.87:0/3711107539 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe348c00640 1 --2- 172.21.15.87:0/3711107539 >> v2:172.21.15.169:3300/0 conn(0x7fe34415b750 0x7fe3440cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe33be00640 1 -- 172.21.15.87:0/3711107539 >> v2:172.21.15.169:3300/0 conn(0x7fe34415b750 msgr2=0x7fe3440cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:23.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe33be00640 1 --2- 172.21.15.87:0/3711107539 >> v2:172.21.15.169:3300/0 conn(0x7fe34415b750 0x7fe3440cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:23.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe33be00640 1 -- 172.21.15.87:0/3711107539 >> v1:172.21.15.87:6789/0 conn(0x7fe3440b3f80 legacy=0x7fe3440b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:23.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe33be00640 1 -- 172.21.15.87:0/3711107539 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3440b2720 con 0x7fe3440cd590 2024-04-25T10:21:23.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.846+0000 7fe343e00640 1 -- 172.21.15.87:0/3711107539 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079769141 0 0) 0x7fe3440b2460 con 0x7fe3440b3f80 2024-04-25T10:21:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe33be00640 1 --2- 172.21.15.87:0/3711107539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3440cd590 0x7fe3440cd9e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe344067980 tx=0x7fe32c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c5944399fc121b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe343e00640 1 -- 172.21.15.87:0/3711107539 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0ab610 con 0x7fe3440cd590 2024-04-25T10:21:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe343e00640 1 -- 172.21.15.87:0/3711107539 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c0b0070 con 0x7fe3440cd590 2024-04-25T10:21:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe343e00640 1 -- 172.21.15.87:0/3711107539 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0ab9d0 con 0x7fe3440cd590 2024-04-25T10:21:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 -- 172.21.15.87:0/3711107539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3440cd590 msgr2=0x7fe3440cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 --2- 172.21.15.87:0/3711107539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3440cd590 0x7fe3440cd9e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe344067980 tx=0x7fe32c01c450 comp rx=0 tx=0).stop 2024-04-25T10:21:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 -- 172.21.15.87:0/3711107539 shutdown_connections 2024-04-25T10:21:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 --2- 172.21.15.87:0/3711107539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3440cd590 0x7fe3440cd9e0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 --2- 172.21.15.87:0/3711107539 >> v2:172.21.15.169:3300/0 conn(0x7fe34415b750 0x7fe3440cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 -- 172.21.15.87:0/3711107539 >> 172.21.15.87:0/3711107539 conn(0x7fe3440743f0 msgr2=0x7fe3440669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 -- 172.21.15.87:0/3711107539 shutdown_connections 2024-04-25T10:21:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 -- 172.21.15.87:0/3711107539 wait complete. 2024-04-25T10:21:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 Processor -- start 2024-04-25T10:21:23.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.850+0000 7fe34a000640 1 -- start start 2024-04-25T10:21:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3440b3f80 0x7fe3441537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3440cd590 0x7fe3440ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe34415b750 0x7fe3440add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34415bb30 con 0x7fe34415b750 2024-04-25T10:21:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe344070b60 con 0x7fe3440cd590 2024-04-25T10:21:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3441527a0 con 0x7fe3440b3f80 2024-04-25T10:21:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe34415b750 0x7fe3440add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe34415b750 0x7fe3440add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32920/0 (socket says 172.21.15.87:32920) 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 -- 172.21.15.87:0/3043574004 learned_addr learned my addr 172.21.15.87:0/3043574004 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe33be00640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3440cd590 0x7fe3440ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe348c00640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3440b3f80 0x7fe3441537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 -- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3440b3f80 msgr2=0x7fe3441537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3440b3f80 0x7fe3441537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 -- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3440cd590 msgr2=0x7fe3440ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3440cd590 0x7fe3440ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 -- 172.21.15.87:0/3043574004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34406e160 con 0x7fe34415b750 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe33be00640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3440cd590 0x7fe3440ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe348c00640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3440b3f80 0x7fe3441537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe349600640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe34415b750 0x7fe3440add80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c700 tx=0x7fe33c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe341600640 1 -- 172.21.15.87:0/3043574004 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c018070 con 0x7fe34415b750 2024-04-25T10:21:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe341600640 1 -- 172.21.15.87:0/3043574004 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c00ce10 con 0x7fe34415b750 2024-04-25T10:21:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe341600640 1 -- 172.21.15.87:0/3043574004 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c00a880 con 0x7fe34415b750 2024-04-25T10:21:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3441529f0 con 0x7fe34415b750 2024-04-25T10:21:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe344152fb0 con 0x7fe34415b750 2024-04-25T10:21:23.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe344075120 con 0x7fe34415b750 2024-04-25T10:21:23.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.854+0000 7fe341600640 1 -- 172.21.15.87:0/3043574004 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe33c00aa20 con 0x7fe34415b750 2024-04-25T10:21:23.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.858+0000 7fe341600640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe320066f60 0x7fe320069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:23.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.858+0000 7fe341600640 1 -- 172.21.15.87:0/3043574004 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe33c084610 con 0x7fe34415b750 2024-04-25T10:21:23.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.858+0000 7fe348c00640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe320066f60 0x7fe320069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:23.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.858+0000 7fe348c00640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe320066f60 0x7fe320069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe334007a70 tx=0x7fe334008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:23.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:23.862+0000 7fe341600640 1 -- 172.21.15.87:0/3043574004 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe33c0555d0 con 0x7fe34415b750 2024-04-25T10:21:24.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.046+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe34415f130 con 0x7fe34415b750 2024-04-25T10:21:24.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.046+0000 7fe341600640 1 -- 172.21.15.87:0/3043574004 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe33c054ed0 con 0x7fe34415b750 2024-04-25T10:21:24.051 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:24.051 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:24.052 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe320066f60 msgr2=0x7fe320069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe320066f60 0x7fe320069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe334007a70 tx=0x7fe334008040 comp rx=0 tx=0).stop 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe34415b750 msgr2=0x7fe3440add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe34415b750 0x7fe3440add80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c700 tx=0x7fe33c00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 shutdown_connections 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe320066f60 0x7fe320069420 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe34415b750 0x7fe3440add80 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3440cd590 0x7fe3440ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 --2- 172.21.15.87:0/3043574004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3440b3f80 0x7fe3441537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 >> 172.21.15.87:0/3043574004 conn(0x7fe3440743f0 msgr2=0x7fe344150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 shutdown_connections 2024-04-25T10:21:24.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.050+0000 7fe34a000640 1 -- 172.21.15.87:0/3043574004 wait complete. 2024-04-25T10:21:24.073 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:21:24.073 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:21:24.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22da400640 1 Processor -- start 2024-04-25T10:21:24.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22da400640 1 -- start start 2024-04-25T10:21:24.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22da400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22d415ddf0 0x7f22d40aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22da400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22d40b2fa0 0x7f22d40b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22da400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d40b39f0 con 0x7f22d406df80 2024-04-25T10:21:24.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22da400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d415e1d0 con 0x7f22d40b2fa0 2024-04-25T10:21:24.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22da400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d40ae250 con 0x7f22d415ddf0 2024-04-25T10:21:24.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22d40b2fa0 0x7f22d40b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22d40b2fa0 0x7f22d40b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32926/0 (socket says 172.21.15.87:32926) 2024-04-25T10:21:24.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2a00640 1 -- 172.21.15.87:0/4166679637 learned_addr learned my addr 172.21.15.87:0/4166679637 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:24.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d3e00640 1 --1- 172.21.15.87:0/4166679637 >> v1:172.21.15.87:6789/0 conn(0x7f22d406df80 0x7f22d4070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42778/0 (socket says 172.21.15.87:42778) 2024-04-25T10:21:24.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d3400640 1 --2- 172.21.15.87:0/4166679637 >> v2:172.21.15.169:3300/0 conn(0x7f22d415ddf0 0x7f22d40aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2a00640 1 -- 172.21.15.87:0/4166679637 >> v2:172.21.15.169:3300/0 conn(0x7f22d415ddf0 msgr2=0x7f22d40aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:24.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2a00640 1 --2- 172.21.15.87:0/4166679637 >> v2:172.21.15.169:3300/0 conn(0x7f22d415ddf0 0x7f22d40aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2a00640 1 -- 172.21.15.87:0/4166679637 >> v1:172.21.15.87:6789/0 conn(0x7f22d406df80 legacy=0x7f22d4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2000640 1 -- 172.21.15.87:0/4166679637 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403671915 0 0) 0x7f22d40b39f0 con 0x7f22d406df80 2024-04-25T10:21:24.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.206+0000 7f22d2a00640 1 -- 172.21.15.87:0/4166679637 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40ae4a0 con 0x7f22d40b2fa0 2024-04-25T10:21:24.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22d2a00640 1 --2- 172.21.15.87:0/4166679637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d40b2fa0 0x7f22d40b3400 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f22d4062910 tx=0x7f22bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d156be6de688bd04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22d2000640 1 -- 172.21.15.87:0/4166679637 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0ab660 con 0x7f22d40b2fa0 2024-04-25T10:21:24.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22d2000640 1 -- 172.21.15.87:0/4166679637 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc0b0070 con 0x7f22d40b2fa0 2024-04-25T10:21:24.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22d2000640 1 -- 172.21.15.87:0/4166679637 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0aba20 con 0x7f22d40b2fa0 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 -- 172.21.15.87:0/4166679637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d40b2fa0 msgr2=0x7f22d40b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 --2- 172.21.15.87:0/4166679637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d40b2fa0 0x7f22d40b3400 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f22d4062910 tx=0x7f22bc01c460 comp rx=0 tx=0).stop 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 -- 172.21.15.87:0/4166679637 shutdown_connections 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 --2- 172.21.15.87:0/4166679637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d40b2fa0 0x7f22d40b3400 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 --2- 172.21.15.87:0/4166679637 >> v2:172.21.15.169:3300/0 conn(0x7f22d415ddf0 0x7f22d40aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 -- 172.21.15.87:0/4166679637 >> 172.21.15.87:0/4166679637 conn(0x7f22d40b1ca0 msgr2=0x7f22d40b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 -- 172.21.15.87:0/4166679637 shutdown_connections 2024-04-25T10:21:24.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.210+0000 7f22da400640 1 -- 172.21.15.87:0/4166679637 wait complete. 2024-04-25T10:21:24.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 Processor -- start 2024-04-25T10:21:24.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 -- start start 2024-04-25T10:21:24.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d406df80 0x7f22d4147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d40b2fa0 0x7f22d4162a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d415ddf0 0x7f22d415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406f520 con 0x7f22d415ddf0 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406f3a0 con 0x7f22d406df80 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d40af4c0 con 0x7f22d40b2fa0 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d415ddf0 0x7f22d415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d415ddf0 0x7f22d415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32940/0 (socket says 172.21.15.87:32940) 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 -- 172.21.15.87:0/1691633774 learned_addr learned my addr 172.21.15.87:0/1691633774 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d406df80 0x7f22d4147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d2a00640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d40b2fa0 0x7f22d4162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 -- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d40b2fa0 msgr2=0x7f22d4162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d40b2fa0 0x7f22d4162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 -- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d406df80 msgr2=0x7f22d4147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d406df80 0x7f22d4147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 -- 172.21.15.87:0/1691633774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d415fef0 con 0x7f22d415ddf0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22d3e00640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d415ddf0 0x7f22d415f3c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c780 tx=0x7f22cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22cb400640 1 -- 172.21.15.87:0/1691633774 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc017070 con 0x7f22d415ddf0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22cb400640 1 -- 172.21.15.87:0/1691633774 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc00a5e0 con 0x7f22d415ddf0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22cb400640 1 -- 172.21.15.87:0/1691633774 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00a9d0 con 0x7f22d415ddf0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d41601d0 con 0x7f22d415ddf0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.214+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d4166800 con 0x7f22d415ddf0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.218+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d404ef30 con 0x7f22d415ddf0 2024-04-25T10:21:24.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.218+0000 7f22cb400640 1 -- 172.21.15.87:0/1691633774 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22cc01a070 con 0x7f22d415ddf0 2024-04-25T10:21:24.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.218+0000 7f22cb400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22b0066f60 0x7f22b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.218+0000 7f22cb400640 1 -- 172.21.15.87:0/1691633774 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f22cc084620 con 0x7f22d415ddf0 2024-04-25T10:21:24.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.218+0000 7f22d3400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22b0066f60 0x7f22b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.218+0000 7f22d3400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22b0066f60 0x7f22b0069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f22c400b9d0 tx=0x7f22c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.222+0000 7f22cb400640 1 -- 172.21.15.87:0/1691633774 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22cc055680 con 0x7f22d415ddf0 2024-04-25T10:21:24.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.430+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f22d40b1a10 con 0x7f22d415ddf0 2024-04-25T10:21:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.430+0000 7f22cb400640 1 -- 172.21.15.87:0/1691633774 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1748 (secure 0 0 0) 0x7f22cc054f80 con 0x7f22d415ddf0 2024-04-25T10:21:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:24.434 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":566,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076636944,"bytes_used":2603368448,"bytes_avail":770490744832,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2358826,"recovering_keys_per_sec":0,"num_objects_recovered":27,"num_bytes_recovered":28311552,"num_keys_recovered":0,"write_bytes_sec":3321621,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":80,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T10:21:23.239026+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.430+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22b0066f60 msgr2=0x7f22b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.430+0000 7f22da400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22b0066f60 0x7f22b0069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f22c400b9d0 tx=0x7f22c4008040 comp rx=0 tx=0).stop 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d415ddf0 msgr2=0x7f22d415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d415ddf0 0x7f22d415f3c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c780 tx=0x7f22cc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 shutdown_connections 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22b0066f60 0x7f22b0069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d415ddf0 0x7f22d415f3c0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d40b2fa0 0x7f22d4162a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 --2- 172.21.15.87:0/1691633774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d406df80 0x7f22d4147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 >> 172.21.15.87:0/1691633774 conn(0x7f22d40b1ca0 msgr2=0x7f22d4157a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 shutdown_connections 2024-04-25T10:21:24.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.434+0000 7f22da400640 1 -- 172.21.15.87:0/1691633774 wait complete. 2024-04-25T10:21:24.456 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:21:24.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:21:24.456 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:24.558 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:24.646 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:21:24.646 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:21:24.647 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:24.655 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:21:24.655 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:24.707 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:24.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb9d400640 1 Processor -- start 2024-04-25T10:21:24.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb9d400640 1 -- start start 2024-04-25T10:21:24.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb9d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb981550b0 0x7fcb980cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb9d400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb98071050 con 0x7fcb9815b910 2024-04-25T10:21:24.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb9d400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9815d040 con 0x7fcb980cfe70 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb9d400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9815bd80 con 0x7fcb981550b0 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb96800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb981550b0 0x7fcb980cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb97200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcb9815b910 0x7fcb980b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42792/0 (socket says 172.21.15.87:42792) 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb97200640 1 -- 172.21.15.87:0/1890849010 learned_addr learned my addr 172.21.15.87:0/1890849010 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb96800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb981550b0 0x7fcb980cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32946/0 (socket says 172.21.15.87:32946) 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb96800640 1 -- 172.21.15.87:0/1890849010 >> v1:172.21.15.169:6789/0 conn(0x7fcb980cfe70 legacy=0x7fcb9815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb96800640 1 -- 172.21.15.87:0/1890849010 >> v1:172.21.15.87:6789/0 conn(0x7fcb9815b910 legacy=0x7fcb980b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.890+0000 7fcb96800640 1 -- 172.21.15.87:0/1890849010 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb980712f0 con 0x7fcb981550b0 2024-04-25T10:21:24.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb96800640 1 --2- 172.21.15.87:0/1890849010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb981550b0 0x7fcb980cf930 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fcb880097f0 tx=0x7fcb8801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad383fa3dad61047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb95e00640 1 -- 172.21.15.87:0/1890849010 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880b0070 con 0x7fcb981550b0 2024-04-25T10:21:24.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb95e00640 1 -- 172.21.15.87:0/1890849010 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb880ab600 con 0x7fcb981550b0 2024-04-25T10:21:24.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb95e00640 1 -- 172.21.15.87:0/1890849010 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880ab9d0 con 0x7fcb981550b0 2024-04-25T10:21:24.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb9d400640 1 -- 172.21.15.87:0/1890849010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb981550b0 msgr2=0x7fcb980cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb9d400640 1 --2- 172.21.15.87:0/1890849010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb981550b0 0x7fcb980cf930 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fcb880097f0 tx=0x7fcb8801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb9d400640 1 -- 172.21.15.87:0/1890849010 shutdown_connections 2024-04-25T10:21:24.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb9d400640 1 --2- 172.21.15.87:0/1890849010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb981550b0 0x7fcb980cf930 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb9d400640 1 -- 172.21.15.87:0/1890849010 >> 172.21.15.87:0/1890849010 conn(0x7fcb98066b70 msgr2=0x7fcb98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb9d400640 1 -- 172.21.15.87:0/1890849010 shutdown_connections 2024-04-25T10:21:24.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.894+0000 7fcb9d400640 1 -- 172.21.15.87:0/1890849010 wait complete. 2024-04-25T10:21:24.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 Processor -- start 2024-04-25T10:21:24.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 -- start start 2024-04-25T10:21:24.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb980cfe70 0x7fcb981fab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb981550b0 0x7fcb98157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb9815b910 0x7fcb9815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb981684c0 con 0x7fcb9815b910 2024-04-25T10:21:24.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980720b0 con 0x7fcb980cfe70 2024-04-25T10:21:24.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98168640 con 0x7fcb981550b0 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb9815b910 0x7fcb9815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb9815b910 0x7fcb9815d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32952/0 (socket says 172.21.15.87:32952) 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 -- 172.21.15.87:0/2348408144 learned_addr learned my addr 172.21.15.87:0/2348408144 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 -- 172.21.15.87:0/2348408144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb981550b0 msgr2=0x7fcb98157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb981550b0 0x7fcb98157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 -- 172.21.15.87:0/2348408144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb980cfe70 msgr2=0x7fcb981fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb980cfe70 0x7fcb981fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 -- 172.21.15.87:0/2348408144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb981fa0b0 con 0x7fcb9815b910 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb97200640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb9815b910 0x7fcb9815d760 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000cbc0 tx=0x7fcb9000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb8f400640 1 -- 172.21.15.87:0/2348408144 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90017070 con 0x7fcb9815b910 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb8f400640 1 -- 172.21.15.87:0/2348408144 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9000ad40 con 0x7fcb9815b910 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb8f400640 1 -- 172.21.15.87:0/2348408144 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90012600 con 0x7fcb9815b910 2024-04-25T10:21:24.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb981fa390 con 0x7fcb9815b910 2024-04-25T10:21:24.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb98150ea0 con 0x7fcb9815b910 2024-04-25T10:21:24.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.898+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9804ef30 con 0x7fcb9815b910 2024-04-25T10:21:24.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.902+0000 7fcb8f400640 1 -- 172.21.15.87:0/2348408144 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb9001a070 con 0x7fcb9815b910 2024-04-25T10:21:24.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.902+0000 7fcb8f400640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb74066f60 0x7fcb74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.902+0000 7fcb8f400640 1 -- 172.21.15.87:0/2348408144 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcb90084d00 con 0x7fcb9815b910 2024-04-25T10:21:24.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.902+0000 7fcb96800640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb74066f60 0x7fcb74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.902+0000 7fcb96800640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb74066f60 0x7fcb74069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ad000 tx=0x7fcb88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:24.906+0000 7fcb8f400640 1 -- 172.21.15.87:0/2348408144 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb90055680 con 0x7fcb9815b910 2024-04-25T10:21:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.062+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb980ced30 con 0x7fcb9815b910 2024-04-25T10:21:25.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.062+0000 7fcb8f400640 1 -- 172.21.15.87:0/2348408144 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16709 (secure 0 0 0) 0x7fcb90059720 con 0x7fcb9815b910 2024-04-25T10:21:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:25.066 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":113,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:06.984221+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:37.371368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:38.004409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.062+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb74066f60 msgr2=0x7fcb74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.062+0000 7fcb9d400640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb74066f60 0x7fcb74069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ad000 tx=0x7fcb88008040 comp rx=0 tx=0).stop 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.062+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb9815b910 msgr2=0x7fcb9815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.062+0000 7fcb9d400640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb9815b910 0x7fcb9815d760 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000cbc0 tx=0x7fcb9000a430 comp rx=0 tx=0).stop 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 shutdown_connections 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb74066f60 0x7fcb74069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb9815b910 0x7fcb9815d760 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb981550b0 0x7fcb98157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 --2- 172.21.15.87:0/2348408144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb980cfe70 0x7fcb981fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 >> 172.21.15.87:0/2348408144 conn(0x7fcb98066b70 msgr2=0x7fcb980b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 shutdown_connections 2024-04-25T10:21:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.066+0000 7fcb9d400640 1 -- 172.21.15.87:0/2348408144 wait complete. 2024-04-25T10:21:25.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:20:37.371368+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:38.004409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:25.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:25.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.222+0000 7ff333600640 1 Processor -- start 2024-04-25T10:21:25.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.222+0000 7ff333600640 1 -- start start 2024-04-25T10:21:25.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff333600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff32c159540 0x7ff32c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff333600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c159920 con 0x7ff32c0aebd0 2024-04-25T10:21:25.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff333600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c15c2f0 con 0x7ff32c0721f0 2024-04-25T10:21:25.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff333600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c15c5d0 con 0x7ff32c159540 2024-04-25T10:21:25.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff32c0aebd0 0x7ff32c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42800/0 (socket says 172.21.15.87:42800) 2024-04-25T10:21:25.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 -- 172.21.15.87:0/2826009424 learned_addr learned my addr 172.21.15.87:0/2826009424 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:25.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595020480 0 0) 0x7ff32c159920 con 0x7ff32c0aebd0 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff310003530 con 0x7ff32c0aebd0 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff330c00640 1 --2- 172.21.15.87:0/2826009424 >> v2:172.21.15.169:3300/0 conn(0x7ff32c159540 0x7ff32c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 744413349 0 0) 0x7ff310003530 con 0x7ff32c0aebd0 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 >> v2:172.21.15.169:3300/0 conn(0x7ff32c159540 msgr2=0x7ff32c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 --2- 172.21.15.87:0/2826009424 >> v2:172.21.15.169:3300/0 conn(0x7ff32c159540 0x7ff32c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 >> v1:172.21.15.169:6789/0 conn(0x7ff32c0721f0 legacy=0x7ff32c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c15c8b0 con 0x7ff32c0aebd0 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ff318001680 con 0x7ff32c0aebd0 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 >> v1:172.21.15.87:6789/0 conn(0x7ff32c0aebd0 legacy=0x7ff32c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310003a10 0x7ff310003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff310004810 0x7ff310004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff310005220 0x7ff31000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15c5d0 con 0x7ff310004810 2024-04-25T10:21:25.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31000c170 con 0x7ff310005220 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31000c370 con 0x7ff310003a10 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff330c00640 1 --2- 172.21.15.87:0/2826009424 >> v2:172.21.15.169:3300/0 conn(0x7ff32c159540 0x7ff32c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff310004810 0x7ff310004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff331600640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff310005220 0x7ff31000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff330c00640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310003a10 0x7ff310003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 -- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310003a10 msgr2=0x7ff310003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310003a10 0x7ff310003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 -- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff310005220 msgr2=0x7ff31000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff310005220 0x7ff31000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff32be00640 1 -- 172.21.15.87:0/2826009424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff310003530 con 0x7ff310004810 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff331600640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff310005220 0x7ff31000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:25.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.226+0000 7ff331600640 1 -- 172.21.15.87:0/2826009424 reap_dead start 2024-04-25T10:21:25.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff32be00640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff310004810 0x7ff310004c90 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff31800c110 tx=0x7ff31801f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed6c1d6cf467507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318007c50 con 0x7ff310004810 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff318005b00 con 0x7ff310004810 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff32b400640 1 -- 172.21.15.87:0/2826009424 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318007070 con 0x7ff310004810 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 -- 172.21.15.87:0/2826009424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff310004810 msgr2=0x7ff310004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff310004810 0x7ff310004c90 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff31800c110 tx=0x7ff31801f310 comp rx=0 tx=0).stop 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 -- 172.21.15.87:0/2826009424 shutdown_connections 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 --2- 172.21.15.87:0/2826009424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff310004810 0x7ff310004c90 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 -- 172.21.15.87:0/2826009424 >> 172.21.15.87:0/2826009424 conn(0x7ff32c0669a0 msgr2=0x7ff32c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 -- 172.21.15.87:0/2826009424 shutdown_connections 2024-04-25T10:21:25.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 -- 172.21.15.87:0/2826009424 wait complete. 2024-04-25T10:21:25.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 Processor -- start 2024-04-25T10:21:25.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.230+0000 7ff333600640 1 -- start start 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310004810 0x7ff32c0af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff32c0b0f00 0x7ff32c1e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff32c14f430 0x7ff32c154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c06c0d0 con 0x7ff32c0b0f00 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15bfa0 con 0x7ff32c14f430 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0ae690 con 0x7ff310004810 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff32c0b0f00 0x7ff32c1e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff32c0b0f00 0x7ff32c1e67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32978/0 (socket says 172.21.15.87:32978) 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 -- 172.21.15.87:0/1796390734 learned_addr learned my addr 172.21.15.87:0/1796390734 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:25.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff330c00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310004810 0x7ff32c0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff331600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff32c14f430 0x7ff32c154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 -- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310004810 msgr2=0x7ff32c0af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310004810 0x7ff32c0af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 -- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff32c14f430 msgr2=0x7ff32c154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff32c14f430 0x7ff32c154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 -- 172.21.15.87:0/1796390734 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0afe60 con 0x7ff32c0b0f00 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff330c00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310004810 0x7ff32c0af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff331600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff32c14f430 0x7ff32c154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff32be00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff32c0b0f00 0x7ff32c1e67e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff31800c110 tx=0x7ff310005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff328c00640 1 -- 172.21.15.87:0/1796390734 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3180049f0 con 0x7ff32c0b0f00 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff328c00640 1 -- 172.21.15.87:0/1796390734 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3180b6070 con 0x7ff32c0b0f00 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff328c00640 1 -- 172.21.15.87:0/1796390734 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318004dd0 con 0x7ff32c0b0f00 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c0b0140 con 0x7ff32c0b0f00 2024-04-25T10:21:25.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c0b0700 con 0x7ff32c0b0f00 2024-04-25T10:21:25.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.234+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c04ef30 con 0x7ff32c0b0f00 2024-04-25T10:21:25.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.238+0000 7ff328c00640 1 -- 172.21.15.87:0/1796390734 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff318007070 con 0x7ff32c0b0f00 2024-04-25T10:21:25.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.238+0000 7ff328c00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff30c066f90 0x7ff30c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.238+0000 7ff328c00640 1 -- 172.21.15.87:0/1796390734 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff318123180 con 0x7ff32c0b0f00 2024-04-25T10:21:25.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.238+0000 7ff330c00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff30c066f90 0x7ff30c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.238+0000 7ff330c00640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff30c066f90 0x7ff30c069450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009ac0 tx=0x7ff31c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.242+0000 7ff328c00640 1 -- 172.21.15.87:0/1796390734 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3180ca020 con 0x7ff32c0b0f00 2024-04-25T10:21:25.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.394+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff32c071f80 con 0x7ff32c0b0f00 2024-04-25T10:21:25.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.394+0000 7ff328c00640 1 -- 172.21.15.87:0/1796390734 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff318019020 con 0x7ff32c0b0f00 2024-04-25T10:21:25.400 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:21:25.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff30c066f90 msgr2=0x7ff30c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff30c066f90 0x7ff30c069450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009ac0 tx=0x7ff31c00b040 comp rx=0 tx=0).stop 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff32c0b0f00 msgr2=0x7ff32c1e67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff32c0b0f00 0x7ff32c1e67e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff31800c110 tx=0x7ff310005840 comp rx=0 tx=0).stop 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 shutdown_connections 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff30c066f90 0x7ff30c069450 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff32c14f430 0x7ff32c154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff32c0b0f00 0x7ff32c1e67e0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 --2- 172.21.15.87:0/1796390734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff310004810 0x7ff32c0af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 >> 172.21.15.87:0/1796390734 conn(0x7ff32c0669a0 msgr2=0x7ff32c15e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 shutdown_connections 2024-04-25T10:21:25.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.398+0000 7ff333600640 1 -- 172.21.15.87:0/1796390734 wait complete. 2024-04-25T10:21:25.420 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:21:25.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.554+0000 7f601a600640 1 Processor -- start 2024-04-25T10:21:25.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.554+0000 7f601a600640 1 -- start start 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f601a600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6014159540 0x7f60140ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f601a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60140aebd0 0x7f6014071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f601a600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6014072020 0x7f601415f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f601a600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6014159920 con 0x7f6014072020 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f601a600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601415c2f0 con 0x7f6014159540 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f601a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601415c5d0 con 0x7f60140aebd0 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6014072020 0x7f601415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6014072020 0x7f601415f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32982/0 (socket says 172.21.15.87:32982) 2024-04-25T10:21:25.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 -- 172.21.15.87:0/3169246338 learned_addr learned my addr 172.21.15.87:0/3169246338 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6012a00640 1 --2- 172.21.15.87:0/3169246338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60140aebd0 0x7f6014071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013400640 1 --2- 172.21.15.87:0/3169246338 >> v2:172.21.15.169:3300/0 conn(0x7f6014159540 0x7f60140ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 -- 172.21.15.87:0/3169246338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60140aebd0 msgr2=0x7f6014071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 --2- 172.21.15.87:0/3169246338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60140aebd0 0x7f6014071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 -- 172.21.15.87:0/3169246338 >> v2:172.21.15.169:3300/0 conn(0x7f6014159540 msgr2=0x7f60140ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 --2- 172.21.15.87:0/3169246338 >> v2:172.21.15.169:3300/0 conn(0x7f6014159540 0x7f60140ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013e00640 1 -- 172.21.15.87:0/3169246338 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601415c8b0 con 0x7f6014072020 2024-04-25T10:21:25.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6012a00640 1 --2- 172.21.15.87:0/3169246338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60140aebd0 0x7f6014071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:25.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.558+0000 7f6013400640 1 --2- 172.21.15.87:0/3169246338 >> v2:172.21.15.169:3300/0 conn(0x7f6014159540 0x7f60140ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:25.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f6013e00640 1 --2- 172.21.15.87:0/3169246338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f601415f590 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f60140b4c20 tx=0x7f600c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad44284476517038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f6012000640 1 -- 172.21.15.87:0/3169246338 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0b0070 con 0x7f6014072020 2024-04-25T10:21:25.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f6012000640 1 -- 172.21.15.87:0/3169246338 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c0ab600 con 0x7f6014072020 2024-04-25T10:21:25.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f6012000640 1 -- 172.21.15.87:0/3169246338 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0ab9d0 con 0x7f6014072020 2024-04-25T10:21:25.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 -- 172.21.15.87:0/3169246338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 msgr2=0x7f601415f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 --2- 172.21.15.87:0/3169246338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f601415f590 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f60140b4c20 tx=0x7f600c01c260 comp rx=0 tx=0).stop 2024-04-25T10:21:25.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 -- 172.21.15.87:0/3169246338 shutdown_connections 2024-04-25T10:21:25.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 --2- 172.21.15.87:0/3169246338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f601415f590 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 --2- 172.21.15.87:0/3169246338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60140aebd0 0x7f6014071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 --2- 172.21.15.87:0/3169246338 >> v2:172.21.15.169:3300/0 conn(0x7f6014159540 0x7f60140ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 -- 172.21.15.87:0/3169246338 >> 172.21.15.87:0/3169246338 conn(0x7f60140669a0 msgr2=0x7f6014066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 -- 172.21.15.87:0/3169246338 shutdown_connections 2024-04-25T10:21:25.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 -- 172.21.15.87:0/3169246338 wait complete. 2024-04-25T10:21:25.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 Processor -- start 2024-04-25T10:21:25.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.562+0000 7f601a600640 1 -- start start 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f60141f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60140aebd0 0x7f601415d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6014159540 0x7f60140b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6014070350 con 0x7f6014072020 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601406c0d0 con 0x7f60140aebd0 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60140704d0 con 0x7f6014159540 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f60141f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f60141f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32988/0 (socket says 172.21.15.87:32988) 2024-04-25T10:21:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 -- 172.21.15.87:0/1238222298 learned_addr learned my addr 172.21.15.87:0/1238222298 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013e00640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6014159540 0x7f60140b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6012a00640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60140aebd0 0x7f601415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 -- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6014159540 msgr2=0x7f60140b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6014159540 0x7f60140b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 -- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60140aebd0 msgr2=0x7f601415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60140aebd0 0x7f601415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 -- 172.21.15.87:0/1238222298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6014152980 con 0x7f6014072020 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f6013400640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f60141f4dc0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f600400ce00 tx=0x7f600400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f600b400640 1 -- 172.21.15.87:0/1238222298 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400f890 con 0x7f6014072020 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f600b400640 1 -- 172.21.15.87:0/1238222298 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600400fa30 con 0x7f6014072020 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f600b400640 1 -- 172.21.15.87:0/1238222298 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400fe00 con 0x7f6014072020 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6014152c60 con 0x7f6014072020 2024-04-25T10:21:25.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6014153220 con 0x7f6014072020 2024-04-25T10:21:25.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601404ef30 con 0x7f6014072020 2024-04-25T10:21:25.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.566+0000 7f600b400640 1 -- 172.21.15.87:0/1238222298 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6004018400 con 0x7f6014072020 2024-04-25T10:21:25.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.570+0000 7f600b400640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ff4066f60 0x7f5ff4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.570+0000 7f600b400640 1 -- 172.21.15.87:0/1238222298 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6004085d40 con 0x7f6014072020 2024-04-25T10:21:25.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.570+0000 7f6012a00640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ff4066f60 0x7f5ff4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.570+0000 7f6012a00640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ff4066f60 0x7f5ff4069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f601415bdb0 tx=0x7f6000008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.574+0000 7f600b400640 1 -- 172.21.15.87:0/1238222298 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6004014030 con 0x7f6014072020 2024-04-25T10:21:25.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.726+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6014166700 con 0x7f6014072020 2024-04-25T10:21:25.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.726+0000 7f600b400640 1 -- 172.21.15.87:0/1238222298 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f600400ae20 con 0x7f6014072020 2024-04-25T10:21:25.730 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.726+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ff4066f60 msgr2=0x7f5ff4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.726+0000 7f601a600640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ff4066f60 0x7f5ff4069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f601415bdb0 tx=0x7f6000008040 comp rx=0 tx=0).stop 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 msgr2=0x7f60141f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f60141f4dc0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f600400ce00 tx=0x7f600400a650 comp rx=0 tx=0).stop 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 shutdown_connections 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ff4066f60 0x7f5ff4069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6014159540 0x7f60140b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60140aebd0 0x7f601415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 --2- 172.21.15.87:0/1238222298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6014072020 0x7f60141f4dc0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 >> 172.21.15.87:0/1238222298 conn(0x7f60140669a0 msgr2=0x7f6014066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 shutdown_connections 2024-04-25T10:21:25.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.730+0000 7f601a600640 1 -- 172.21.15.87:0/1238222298 wait complete. 2024-04-25T10:21:25.751 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:21:25.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.886+0000 7f5467000640 1 Processor -- start 2024-04-25T10:21:25.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.886+0000 7f5467000640 1 -- start start 2024-04-25T10:21:25.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.886+0000 7f5467000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54600aef80 0x7f546015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.886+0000 7f5467000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54600b31c0 0x7f54600b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.886+0000 7f5467000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546015b080 con 0x7f5460070350 2024-04-25T10:21:25.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.886+0000 7f5467000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f546015adb0 con 0x7f54600b31c0 2024-04-25T10:21:25.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5467000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546015eb60 con 0x7f54600aef80 2024-04-25T10:21:25.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54600b31c0 0x7f54600b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54600b31c0 0x7f54600b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32998/0 (socket says 172.21.15.87:32998) 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 -- 172.21.15.87:0/852191248 learned_addr learned my addr 172.21.15.87:0/852191248 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5457400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5460070350 0x7f546015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42810/0 (socket says 172.21.15.87:42810) 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f545fe00640 1 --2- 172.21.15.87:0/852191248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54600aef80 0x7f546015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 -- 172.21.15.87:0/852191248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54600aef80 msgr2=0x7f546015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 --2- 172.21.15.87:0/852191248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54600aef80 0x7f546015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 -- 172.21.15.87:0/852191248 >> v1:172.21.15.87:6789/0 conn(0x7f5460070350 legacy=0x7f546015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 -- 172.21.15.87:0/852191248 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546015ee40 con 0x7f54600b31c0 2024-04-25T10:21:25.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f545f400640 1 -- 172.21.15.87:0/852191248 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801977229 0 0) 0x7f546015b080 con 0x7f5460070350 2024-04-25T10:21:25.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f5465000640 1 --2- 172.21.15.87:0/852191248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600b31c0 0x7f54600b3610 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f545800d150 tx=0x7f545801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31af85d02e36d463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f545f400640 1 -- 172.21.15.87:0/852191248 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545800b2d0 con 0x7f54600b31c0 2024-04-25T10:21:25.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f545f400640 1 -- 172.21.15.87:0/852191248 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545800b470 con 0x7f54600b31c0 2024-04-25T10:21:25.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.890+0000 7f545f400640 1 -- 172.21.15.87:0/852191248 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580a7c30 con 0x7f54600b31c0 2024-04-25T10:21:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- 172.21.15.87:0/852191248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600b31c0 msgr2=0x7f54600b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 --2- 172.21.15.87:0/852191248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600b31c0 0x7f54600b3610 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f545800d150 tx=0x7f545801ec30 comp rx=0 tx=0).stop 2024-04-25T10:21:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- 172.21.15.87:0/852191248 shutdown_connections 2024-04-25T10:21:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 --2- 172.21.15.87:0/852191248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600b31c0 0x7f54600b3610 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 --2- 172.21.15.87:0/852191248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54600aef80 0x7f546015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- 172.21.15.87:0/852191248 >> 172.21.15.87:0/852191248 conn(0x7f54600669a0 msgr2=0x7f5460066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- 172.21.15.87:0/852191248 shutdown_connections 2024-04-25T10:21:25.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- 172.21.15.87:0/852191248 wait complete. 2024-04-25T10:21:25.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 Processor -- start 2024-04-25T10:21:25.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- start start 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5460070350 0x7f5460152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600aef80 0x7f54600ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54600b31c0 0x7f5460155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546006c250 con 0x7f54600aef80 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546006c0d0 con 0x7f54600b31c0 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5467000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546006c3d0 con 0x7f5460070350 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600aef80 0x7f54600ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600aef80 0x7f54600ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33008/0 (socket says 172.21.15.87:33008) 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 -- 172.21.15.87:0/2539931213 learned_addr learned my addr 172.21.15.87:0/2539931213 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f545fe00640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5460070350 0x7f5460152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5465000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54600b31c0 0x7f5460155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 -- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5460070350 msgr2=0x7f5460152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5460070350 0x7f5460152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 -- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54600b31c0 msgr2=0x7f5460155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54600b31c0 0x7f5460155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5457400640 1 -- 172.21.15.87:0/2539931213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54601fa130 con 0x7f54600aef80 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.894+0000 7f5465000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54600b31c0 0x7f5460155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f5457400640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600aef80 0x7f54600ce040 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f54480011f0 tx=0x7f544800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f545cc00640 1 -- 172.21.15.87:0/2539931213 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448018070 con 0x7f54600aef80 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f545cc00640 1 -- 172.21.15.87:0/2539931213 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54480074e0 con 0x7f54600aef80 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f545cc00640 1 -- 172.21.15.87:0/2539931213 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448004a10 con 0x7f54600aef80 2024-04-25T10:21:25.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54601fa410 con 0x7f54600aef80 2024-04-25T10:21:25.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54601fa9d0 con 0x7f54600aef80 2024-04-25T10:21:25.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54601fadb0 con 0x7f54600aef80 2024-04-25T10:21:25.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f545cc00640 1 -- 172.21.15.87:0/2539931213 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5448004bb0 con 0x7f54600aef80 2024-04-25T10:21:25.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f545cc00640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f543c066f60 0x7f543c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f545cc00640 1 -- 172.21.15.87:0/2539931213 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5448084430 con 0x7f54600aef80 2024-04-25T10:21:25.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.898+0000 7f545fe00640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f543c066f60 0x7f543c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.902+0000 7f545fe00640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f543c066f60 0x7f543c069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5450006aa0 tx=0x7f5450008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:25.902+0000 7f545cc00640 1 -- 172.21.15.87:0/2539931213 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54601fadb0 con 0x7f54600aef80 2024-04-25T10:21:26.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.066+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f546004ef30 con 0x7f54600aef80 2024-04-25T10:21:26.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.066+0000 7f545cc00640 1 -- 172.21.15.87:0/2539931213 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5448054d10 con 0x7f54600aef80 2024-04-25T10:21:26.070 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:21:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f543c066f60 msgr2=0x7f543c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f543c066f60 0x7f543c069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5450006aa0 tx=0x7f5450008040 comp rx=0 tx=0).stop 2024-04-25T10:21:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600aef80 msgr2=0x7f54600ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600aef80 0x7f54600ce040 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f54480011f0 tx=0x7f544800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 shutdown_connections 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f543c066f60 0x7f543c069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54600b31c0 0x7f5460155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54600aef80 0x7f54600ce040 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 --2- 172.21.15.87:0/2539931213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5460070350 0x7f5460152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 >> 172.21.15.87:0/2539931213 conn(0x7f54600669a0 msgr2=0x7f546015b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 shutdown_connections 2024-04-25T10:21:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.070+0000 7f5467000640 1 -- 172.21.15.87:0/2539931213 wait complete. 2024-04-25T10:21:26.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:21:26.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90ba200640 1 Processor -- start 2024-04-25T10:21:26.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90ba200640 1 -- start start 2024-04-25T10:21:26.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90ba200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f90b415f9b0 0x7f90b4071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90ba200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b4147760 con 0x7f90b4071690 2024-04-25T10:21:26.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90ba200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b40b57a0 con 0x7f90b4147380 2024-04-25T10:21:26.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90ba200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b4072030 con 0x7f90b415f9b0 2024-04-25T10:21:26.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f90b4071690 0x7f90b415f2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:42820/0 (socket says 172.21.15.87:42820) 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b3e00640 1 -- 172.21.15.87:0/4137400760 learned_addr learned my addr 172.21.15.87:0/4137400760 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b9800640 1 --2- 172.21.15.87:0/4137400760 >> v2:172.21.15.169:3300/0 conn(0x7f90b415f9b0 0x7f90b4071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b3400640 1 -- 172.21.15.87:0/4137400760 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96833821 0 0) 0x7f90b4147760 con 0x7f90b4071690 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b3400640 1 -- 172.21.15.87:0/4137400760 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9098003530 con 0x7f90b4071690 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b9800640 1 -- 172.21.15.87:0/4137400760 >> v1:172.21.15.169:6789/0 conn(0x7f90b4147380 legacy=0x7f90b40b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b9800640 1 -- 172.21.15.87:0/4137400760 >> v1:172.21.15.87:6789/0 conn(0x7f90b4071690 legacy=0x7f90b415f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b3400640 1 -- 172.21.15.87:0/4137400760 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3812310393 0 0) 0x7f9098003530 con 0x7f90b4071690 2024-04-25T10:21:26.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.226+0000 7f90b9800640 1 -- 172.21.15.87:0/4137400760 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b415ed60 con 0x7f90b415f9b0 2024-04-25T10:21:26.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90b9800640 1 --2- 172.21.15.87:0/4137400760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b415f9b0 0x7f90b4071af0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009d90 tx=0x7f90ac01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea5577ce79070fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90b3400640 1 -- 172.21.15.87:0/4137400760 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0b0070 con 0x7f90b415f9b0 2024-04-25T10:21:26.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90b3400640 1 -- 172.21.15.87:0/4137400760 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac0039a0 con 0x7f90b415f9b0 2024-04-25T10:21:26.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90b3400640 1 -- 172.21.15.87:0/4137400760 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0001680 con 0x7f90b415f9b0 2024-04-25T10:21:26.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90ba200640 1 -- 172.21.15.87:0/4137400760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b415f9b0 msgr2=0x7f90b4071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90ba200640 1 --2- 172.21.15.87:0/4137400760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b415f9b0 0x7f90b4071af0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009d90 tx=0x7f90ac01c890 comp rx=0 tx=0).stop 2024-04-25T10:21:26.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90ba200640 1 -- 172.21.15.87:0/4137400760 shutdown_connections 2024-04-25T10:21:26.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90ba200640 1 --2- 172.21.15.87:0/4137400760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b415f9b0 0x7f90b4071af0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90ba200640 1 -- 172.21.15.87:0/4137400760 >> 172.21.15.87:0/4137400760 conn(0x7f90b406b090 msgr2=0x7f90b406b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90ba200640 1 -- 172.21.15.87:0/4137400760 shutdown_connections 2024-04-25T10:21:26.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.230+0000 7f90ba200640 1 -- 172.21.15.87:0/4137400760 wait complete. 2024-04-25T10:21:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 Processor -- start 2024-04-25T10:21:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 -- start start 2024-04-25T10:21:26.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b4071690 0x7f90b413b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4147380 0x7f90b40cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90b415f9b0 0x7f90b4139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b416b8b0 con 0x7f90b4071690 2024-04-25T10:21:26.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b416b730 con 0x7f90b4147380 2024-04-25T10:21:26.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b416ba30 con 0x7f90b415f9b0 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b4071690 0x7f90b413b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b4071690 0x7f90b413b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33020/0 (socket says 172.21.15.87:33020) 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4147380 0x7f90b40cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 -- 172.21.15.87:0/802960584 learned_addr learned my addr 172.21.15.87:0/802960584 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b9800640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90b415f9b0 0x7f90b4139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 -- 172.21.15.87:0/802960584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90b415f9b0 msgr2=0x7f90b4139230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90b415f9b0 0x7f90b4139230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 -- 172.21.15.87:0/802960584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4147380 msgr2=0x7f90b40cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4147380 0x7f90b40cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 -- 172.21.15.87:0/802960584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40d0420 con 0x7f90b4071690 2024-04-25T10:21:26.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b8e00640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b4071690 0x7f90b413b9c0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c500 tx=0x7f90a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b0c00640 1 -- 172.21.15.87:0/802960584 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4016020 con 0x7f90b4071690 2024-04-25T10:21:26.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b0c00640 1 -- 172.21.15.87:0/802960584 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a4005220 con 0x7f90b4071690 2024-04-25T10:21:26.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b413e990 con 0x7f90b4071690 2024-04-25T10:21:26.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90b0c00640 1 -- 172.21.15.87:0/802960584 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4007030 con 0x7f90b4071690 2024-04-25T10:21:26.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.234+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b413ec30 con 0x7f90b4071690 2024-04-25T10:21:26.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.238+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b413ee30 con 0x7f90b4071690 2024-04-25T10:21:26.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.238+0000 7f90b0c00640 1 -- 172.21.15.87:0/802960584 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90a40071f0 con 0x7f90b4071690 2024-04-25T10:21:26.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.238+0000 7f90b0c00640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9094066f60 0x7f9094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.238+0000 7f90b0c00640 1 -- 172.21.15.87:0/802960584 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f90a4084970 con 0x7f90b4071690 2024-04-25T10:21:26.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.238+0000 7f90b3e00640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9094066f60 0x7f9094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.238+0000 7f90b3e00640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9094066f60 0x7f9094069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90b413e7f0 tx=0x7f90a0001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.242+0000 7f90b0c00640 1 -- 172.21.15.87:0/802960584 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90b413ee30 con 0x7f90b4071690 2024-04-25T10:21:26.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.406+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f90b415f600 con 0x7f90b4071690 2024-04-25T10:21:26.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.406+0000 7f90b0c00640 1 -- 172.21.15.87:0/802960584 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7f90a4020020 con 0x7f90b4071690 2024-04-25T10:21:26.411 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:21:26.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9094066f60 msgr2=0x7f9094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9094066f60 0x7f9094069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90b413e7f0 tx=0x7f90a0001700 comp rx=0 tx=0).stop 2024-04-25T10:21:26.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b4071690 msgr2=0x7f90b413b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b4071690 0x7f90b413b9c0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c500 tx=0x7f90a400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 shutdown_connections 2024-04-25T10:21:26.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9094066f60 0x7f9094069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90b415f9b0 0x7f90b4139230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4147380 0x7f90b40cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 --2- 172.21.15.87:0/802960584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b4071690 0x7f90b413b9c0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 >> 172.21.15.87:0/802960584 conn(0x7f90b406b090 msgr2=0x7f90b40b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 shutdown_connections 2024-04-25T10:21:26.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:26.410+0000 7f90ba200640 1 -- 172.21.15.87:0/802960584 wait complete. 2024-04-25T10:21:26.432 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:21:26.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:26.432 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:27.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:21:27.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:27.139 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:21:27.149 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:21:27.149 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:27.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:27.159 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:21:27.159 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:27.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:21:27.202 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:21:27.202 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:27.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:27.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:27.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.374+0000 7f8d1b200640 1 Processor -- start 2024-04-25T10:21:27.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d1b200640 1 -- start start 2024-04-25T10:21:27.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d1b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d1c151930 0x7f8d1c070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d1b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c0b2260 0x7f8d1c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d1b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c15bf50 con 0x7f8d1c0711f0 2024-04-25T10:21:27.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d1b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c151d10 con 0x7f8d1c151930 2024-04-25T10:21:27.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d1b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b3980 con 0x7f8d1c0b2260 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d19e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d1c151930 0x7f8d1c070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d19e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d1c151930 0x7f8d1c070cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43774/0 (socket says 172.21.15.87:43774) 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d19e00640 1 -- 172.21.15.87:0/2471960901 learned_addr learned my addr 172.21.15.87:0/2471960901 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d1a800640 1 --2- 172.21.15.87:0/2471960901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c0b2260 0x7f8d1c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d19e00640 1 -- 172.21.15.87:0/2471960901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c0b2260 msgr2=0x7f8d1c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d19e00640 1 --2- 172.21.15.87:0/2471960901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c0b2260 0x7f8d1c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d19e00640 1 -- 172.21.15.87:0/2471960901 >> v1:172.21.15.169:6789/0 conn(0x7f8d1c0711f0 legacy=0x7f8d1c0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:27.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.378+0000 7f8d19e00640 1 -- 172.21.15.87:0/2471960901 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c15b0a0 con 0x7f8d1c151930 2024-04-25T10:21:27.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d19e00640 1 --2- 172.21.15.87:0/2471960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c151930 0x7f8d1c070cb0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0b2760 tx=0x7f8d0401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e757677699394b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d18a00640 1 -- 172.21.15.87:0/2471960901 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040b0070 con 0x7f8d1c151930 2024-04-25T10:21:27.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d18a00640 1 -- 172.21.15.87:0/2471960901 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d040ab600 con 0x7f8d1c151930 2024-04-25T10:21:27.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d18a00640 1 -- 172.21.15.87:0/2471960901 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040ab9d0 con 0x7f8d1c151930 2024-04-25T10:21:27.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 -- 172.21.15.87:0/2471960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c151930 msgr2=0x7f8d1c070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2471960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c151930 0x7f8d1c070cb0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0b2760 tx=0x7f8d0401c450 comp rx=0 tx=0).stop 2024-04-25T10:21:27.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 -- 172.21.15.87:0/2471960901 shutdown_connections 2024-04-25T10:21:27.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2471960901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c0b2260 0x7f8d1c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2471960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c151930 0x7f8d1c070cb0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 -- 172.21.15.87:0/2471960901 >> 172.21.15.87:0/2471960901 conn(0x7f8d1c06ac90 msgr2=0x7f8d1c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 -- 172.21.15.87:0/2471960901 shutdown_connections 2024-04-25T10:21:27.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 -- 172.21.15.87:0/2471960901 wait complete. 2024-04-25T10:21:27.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 Processor -- start 2024-04-25T10:21:27.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.382+0000 7f8d1b200640 1 -- start start 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c0711f0 0x7f8d1c0ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d1c0b2260 0x7f8d1c167170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c151930 0x7f8d1c1676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b4fc0 con 0x7f8d1c0711f0 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b4e40 con 0x7f8d1c0b2260 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b5140 con 0x7f8d1c151930 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c0711f0 0x7f8d1c0ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c0711f0 0x7f8d1c0ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43778/0 (socket says 172.21.15.87:43778) 2024-04-25T10:21:27.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 -- 172.21.15.87:0/2912475584 learned_addr learned my addr 172.21.15.87:0/2912475584 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1a800640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c151930 0x7f8d1c1676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19400640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d1c0b2260 0x7f8d1c167170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 -- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c151930 msgr2=0x7f8d1c1676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c151930 0x7f8d1c1676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 -- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d1c0b2260 msgr2=0x7f8d1c167170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d1c0b2260 0x7f8d1c167170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 -- 172.21.15.87:0/2912475584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c166550 con 0x7f8d1c0711f0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d19e00640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c0711f0 0x7f8d1c0ccef0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8d040ae040 tx=0x7f8d040191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d12000640 1 -- 172.21.15.87:0/2912475584 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040b0040 con 0x7f8d1c0711f0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d12000640 1 -- 172.21.15.87:0/2912475584 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d040b4420 con 0x7f8d1c0711f0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d12000640 1 -- 172.21.15.87:0/2912475584 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040b4810 con 0x7f8d1c0711f0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c0cfe70 con 0x7f8d1c0711f0 2024-04-25T10:21:27.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c166970 con 0x7f8d1c0711f0 2024-04-25T10:21:27.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c166780 con 0x7f8d1c0711f0 2024-04-25T10:21:27.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.386+0000 7f8d12000640 1 -- 172.21.15.87:0/2912475584 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d040b49b0 con 0x7f8d1c0711f0 2024-04-25T10:21:27.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.390+0000 7f8d12000640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8cf8066f60 0x7f8cf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.390+0000 7f8d12000640 1 -- 172.21.15.87:0/2912475584 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8d0411ec40 con 0x7f8d1c0711f0 2024-04-25T10:21:27.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.390+0000 7f8d19400640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8cf8066f60 0x7f8cf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.390+0000 7f8d19400640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8cf8066f60 0x7f8cf8069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c006fd0 tx=0x7f8d0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.394+0000 7f8d12000640 1 -- 172.21.15.87:0/2912475584 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d040160d0 con 0x7f8d1c0711f0 2024-04-25T10:21:27.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.578+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d1c0b3530 con 0x7f8d1c0711f0 2024-04-25T10:21:27.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.578+0000 7f8d12000640 1 -- 172.21.15.87:0/2912475584 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8d040efc00 con 0x7f8d1c0711f0 2024-04-25T10:21:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:27.583 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:27.584 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:27.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8cf8066f60 msgr2=0x7f8cf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8cf8066f60 0x7f8cf8069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c006fd0 tx=0x7f8d0c008040 comp rx=0 tx=0).stop 2024-04-25T10:21:27.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c0711f0 msgr2=0x7f8d1c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c0711f0 0x7f8d1c0ccef0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8d040ae040 tx=0x7f8d040191f0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 shutdown_connections 2024-04-25T10:21:27.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8cf8066f60 0x7f8cf8069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d1c151930 0x7f8d1c1676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d1c0b2260 0x7f8d1c167170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 --2- 172.21.15.87:0/2912475584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d1c0711f0 0x7f8d1c0ccef0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 >> 172.21.15.87:0/2912475584 conn(0x7f8d1c06ac90 msgr2=0x7f8d1c148130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 shutdown_connections 2024-04-25T10:21:27.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.582+0000 7f8d1b200640 1 -- 172.21.15.87:0/2912475584 wait complete. 2024-04-25T10:21:27.605 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:27.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.738+0000 7f033e200640 1 Processor -- start 2024-04-25T10:21:27.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.738+0000 7f033e200640 1 -- start start 2024-04-25T10:21:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f033e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f033806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f033e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03380cda10 0x7f03380b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f033e200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380b2770 con 0x7f03380cf930 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f033e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033806c670 con 0x7f03380cda10 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f033e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033806c950 con 0x7f033815bc40 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f0337e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f03380cf930 0x7f033815f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45650/0 (socket says 172.21.15.87:45650) 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f0337e00640 1 -- 172.21.15.87:0/1692256031 learned_addr learned my addr 172.21.15.87:0/1692256031 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f032fe00640 1 --2- 172.21.15.87:0/1692256031 >> v2:172.21.15.87:3300/0 conn(0x7f03380cda10 0x7f03380b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f0337400640 1 --2- 172.21.15.87:0/1692256031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f033806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f032fe00640 1 -- 172.21.15.87:0/1692256031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 msgr2=0x7f033806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f032fe00640 1 --2- 172.21.15.87:0/1692256031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f033806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f032fe00640 1 -- 172.21.15.87:0/1692256031 >> v1:172.21.15.87:6789/0 conn(0x7f03380cf930 legacy=0x7f033815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.742+0000 7f032fe00640 1 -- 172.21.15.87:0/1692256031 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033806cc30 con 0x7f03380cda10 2024-04-25T10:21:27.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f032fe00640 1 --2- 172.21.15.87:0/1692256031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cda10 0x7f03380b4610 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f03200097c0 tx=0x7f032001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee29feb65581de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f0336a00640 1 -- 172.21.15.87:0/1692256031 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200b1070 con 0x7f03380cda10 2024-04-25T10:21:27.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f0336a00640 1 -- 172.21.15.87:0/1692256031 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03200ac410 con 0x7f03380cda10 2024-04-25T10:21:27.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f0336a00640 1 -- 172.21.15.87:0/1692256031 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200ac7e0 con 0x7f03380cda10 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- 172.21.15.87:0/1692256031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cda10 msgr2=0x7f03380b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 --2- 172.21.15.87:0/1692256031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cda10 0x7f03380b4610 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f03200097c0 tx=0x7f032001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- 172.21.15.87:0/1692256031 shutdown_connections 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 --2- 172.21.15.87:0/1692256031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cda10 0x7f03380b4610 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 --2- 172.21.15.87:0/1692256031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f033806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- 172.21.15.87:0/1692256031 >> 172.21.15.87:0/1692256031 conn(0x7f0338066b70 msgr2=0x7f0338066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- 172.21.15.87:0/1692256031 shutdown_connections 2024-04-25T10:21:27.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- 172.21.15.87:0/1692256031 wait complete. 2024-04-25T10:21:27.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 Processor -- start 2024-04-25T10:21:27.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- start start 2024-04-25T10:21:27.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03380cda10 0x7f0338147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cf930 0x7f0338154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f0338154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338168b50 con 0x7f03380cf930 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03381689d0 con 0x7f03380cda10 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f033e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0338168cd0 con 0x7f033815bc40 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f032fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cf930 0x7f0338154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.746+0000 7f032fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cf930 0x7f0338154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43802/0 (socket says 172.21.15.87:43802) 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032fe00640 1 -- 172.21.15.87:0/2065382399 learned_addr learned my addr 172.21.15.87:0/2065382399 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f0337e00640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f0338154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f0337400640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03380cda10 0x7f0338147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032fe00640 1 -- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 msgr2=0x7f0338154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032fe00640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f0338154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032fe00640 1 -- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03380cda10 msgr2=0x7f0338147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032fe00640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03380cda10 0x7f0338147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032fe00640 1 -- 172.21.15.87:0/2065382399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03381f5210 con 0x7f03380cf930 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032fe00640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cf930 0x7f0338154a70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f03200af040 tx=0x7f0320019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032f400640 1 -- 172.21.15.87:0/2065382399 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200b1040 con 0x7f03380cf930 2024-04-25T10:21:27.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032f400640 1 -- 172.21.15.87:0/2065382399 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03200bb070 con 0x7f03380cf930 2024-04-25T10:21:27.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032f400640 1 -- 172.21.15.87:0/2065382399 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200b5670 con 0x7f03380cf930 2024-04-25T10:21:27.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03381f54f0 con 0x7f03380cf930 2024-04-25T10:21:27.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03381f4ef0 con 0x7f03380cf930 2024-04-25T10:21:27.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03380cbcb0 con 0x7f03380cf930 2024-04-25T10:21:27.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032f400640 1 -- 172.21.15.87:0/2065382399 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03200b5810 con 0x7f03380cf930 2024-04-25T10:21:27.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032f400640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0314066f90 0x7f0314069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f032f400640 1 -- 172.21.15.87:0/2065382399 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f032011ead0 con 0x7f03380cf930 2024-04-25T10:21:27.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.750+0000 7f0337400640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0314066f90 0x7f0314069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.754+0000 7f0337400640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0314066f90 0x7f0314069450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f033814c600 tx=0x7f0328008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.754+0000 7f032f400640 1 -- 172.21.15.87:0/2065382399 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0320016090 con 0x7f03380cf930 2024-04-25T10:21:27.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.910+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f033804ef30 con 0x7f03380cf930 2024-04-25T10:21:27.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.910+0000 7f032f400640 1 -- 172.21.15.87:0/2065382399 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16709 (secure 0 0 0) 0x7f03200efa00 con 0x7f03380cf930 2024-04-25T10:21:27.914 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:27.914 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":113,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:06.984221+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:37.371368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:38.004409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:27.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0314066f90 msgr2=0x7f0314069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0314066f90 0x7f0314069450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f033814c600 tx=0x7f0328008040 comp rx=0 tx=0).stop 2024-04-25T10:21:27.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cf930 msgr2=0x7f0338154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cf930 0x7f0338154a70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f03200af040 tx=0x7f0320019240 comp rx=0 tx=0).stop 2024-04-25T10:21:27.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 shutdown_connections 2024-04-25T10:21:27.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0314066f90 0x7f0314069450 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f033815bc40 0x7f0338154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03380cf930 0x7f0338154a70 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 --2- 172.21.15.87:0/2065382399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03380cda10 0x7f0338147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 >> 172.21.15.87:0/2065382399 conn(0x7f0338066b70 msgr2=0x7f033815a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 shutdown_connections 2024-04-25T10:21:27.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:27.914+0000 7f033e200640 1 -- 172.21.15.87:0/2065382399 wait complete. 2024-04-25T10:21:27.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:20:37.371368+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:38.004409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:27.935 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:28.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d8a000640 1 Processor -- start 2024-04-25T10:21:28.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d8a000640 1 -- start start 2024-04-25T10:21:28.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d8a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d840cf4c0 0x7f8d84072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d8a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d84073d10 0x7f8d840cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d8a000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d84072b10 con 0x7f8d8415bcb0 2024-04-25T10:21:28.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d8a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840ce180 con 0x7f8d840cf4c0 2024-04-25T10:21:28.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d8a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840ce460 con 0x7f8d84073d10 2024-04-25T10:21:28.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d84073d10 0x7f8d840cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d7be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d840cf4c0 0x7f8d84072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d84073d10 0x7f8d840cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:50024/0 (socket says 172.21.15.87:50024) 2024-04-25T10:21:28.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d83e00640 1 -- 172.21.15.87:0/2459193750 learned_addr learned my addr 172.21.15.87:0/2459193750 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:28.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d83400640 1 --1- 172.21.15.87:0/2459193750 >> v1:172.21.15.169:6789/0 conn(0x7f8d8415bcb0 0x7f8d840cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:59782/0 (socket says 172.21.15.87:59782) 2024-04-25T10:21:28.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d7be00640 1 -- 172.21.15.87:0/2459193750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d84073d10 msgr2=0x7f8d840cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d7be00640 1 --2- 172.21.15.87:0/2459193750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d84073d10 0x7f8d840cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d7be00640 1 -- 172.21.15.87:0/2459193750 >> v1:172.21.15.169:6789/0 conn(0x7f8d8415bcb0 legacy=0x7f8d840cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.070+0000 7f8d7be00640 1 -- 172.21.15.87:0/2459193750 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840ce740 con 0x7f8d840cf4c0 2024-04-25T10:21:28.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d7be00640 1 --2- 172.21.15.87:0/2459193750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 0x7f8d84072400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0097c0 tx=0x7f8d6c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186e0e526124b60f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d82a00640 1 -- 172.21.15.87:0/2459193750 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0b0070 con 0x7f8d840cf4c0 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d82a00640 1 -- 172.21.15.87:0/2459193750 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0ab600 con 0x7f8d840cf4c0 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d82a00640 1 -- 172.21.15.87:0/2459193750 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0ab9d0 con 0x7f8d840cf4c0 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 -- 172.21.15.87:0/2459193750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 msgr2=0x7f8d84072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 --2- 172.21.15.87:0/2459193750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 0x7f8d84072400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0097c0 tx=0x7f8d6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 -- 172.21.15.87:0/2459193750 shutdown_connections 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 --2- 172.21.15.87:0/2459193750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d84073d10 0x7f8d840cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 --2- 172.21.15.87:0/2459193750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 0x7f8d84072400 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 -- 172.21.15.87:0/2459193750 >> 172.21.15.87:0/2459193750 conn(0x7f8d840b1ca0 msgr2=0x7f8d840b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 -- 172.21.15.87:0/2459193750 shutdown_connections 2024-04-25T10:21:28.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.074+0000 7f8d8a000640 1 -- 172.21.15.87:0/2459193750 wait complete. 2024-04-25T10:21:28.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 Processor -- start 2024-04-25T10:21:28.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 -- start start 2024-04-25T10:21:28.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d84073d10 0x7f8d84159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 0x7f8d841573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d8415bcb0 0x7f8d8414b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cd6a0 con 0x7f8d84073d10 2024-04-25T10:21:28.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cd520 con 0x7f8d840cf4c0 2024-04-25T10:21:28.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d8a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cd820 con 0x7f8d8415bcb0 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d83400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d84073d10 0x7f8d84159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d83400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d84073d10 0x7f8d84159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43806/0 (socket says 172.21.15.87:43806) 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d83400640 1 -- 172.21.15.87:0/3263154537 learned_addr learned my addr 172.21.15.87:0/3263154537 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d7be00640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 0x7f8d841573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.078+0000 7f8d83e00640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d8415bcb0 0x7f8d8414b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d83400640 1 -- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d8415bcb0 msgr2=0x7f8d8414b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d83400640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d8415bcb0 0x7f8d8414b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d83400640 1 -- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 msgr2=0x7f8d841573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d83400640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 0x7f8d841573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d83400640 1 -- 172.21.15.87:0/3263154537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d84157e40 con 0x7f8d84073d10 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d83400640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d84073d10 0x7f8d84159840 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c370 tx=0x7f8d7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d7b400640 1 -- 172.21.15.87:0/3263154537 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740074e0 con 0x7f8d84073d10 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d7b400640 1 -- 172.21.15.87:0/3263154537 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74007680 con 0x7f8d84073d10 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d7b400640 1 -- 172.21.15.87:0/3263154537 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74004a30 con 0x7f8d84073d10 2024-04-25T10:21:28.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d840b32f0 con 0x7f8d84073d10 2024-04-25T10:21:28.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d84150500 con 0x7f8d84073d10 2024-04-25T10:21:28.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8404ef30 con 0x7f8d84073d10 2024-04-25T10:21:28.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d7b400640 1 -- 172.21.15.87:0/3263154537 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d7401a020 con 0x7f8d84073d10 2024-04-25T10:21:28.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d7b400640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d60066f90 0x7f8d60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d7b400640 1 -- 172.21.15.87:0/3263154537 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8d740847c0 con 0x7f8d84073d10 2024-04-25T10:21:28.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.082+0000 7f8d7be00640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d60066f90 0x7f8d60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.086+0000 7f8d7be00640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d60066f90 0x7f8d60069450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0ad000 tx=0x7f8d6c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.090+0000 7f8d7b400640 1 -- 172.21.15.87:0/3263154537 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d74055010 con 0x7f8d84073d10 2024-04-25T10:21:28.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d840b0040 con 0x7f8d84073d10 2024-04-25T10:21:28.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d7b400640 1 -- 172.21.15.87:0/3263154537 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d740590b0 con 0x7f8d84073d10 2024-04-25T10:21:28.249 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:21:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d60066f90 msgr2=0x7f8d60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d60066f90 0x7f8d60069450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0ad000 tx=0x7f8d6c008040 comp rx=0 tx=0).stop 2024-04-25T10:21:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d84073d10 msgr2=0x7f8d84159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d84073d10 0x7f8d84159840 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c370 tx=0x7f8d7400c830 comp rx=0 tx=0).stop 2024-04-25T10:21:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 shutdown_connections 2024-04-25T10:21:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d60066f90 0x7f8d60069450 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d8415bcb0 0x7f8d8414b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d840cf4c0 0x7f8d841573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 --2- 172.21.15.87:0/3263154537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d84073d10 0x7f8d84159840 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 >> 172.21.15.87:0/3263154537 conn(0x7f8d840b1ca0 msgr2=0x7f8d8406dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 shutdown_connections 2024-04-25T10:21:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.246+0000 7f8d8a000640 1 -- 172.21.15.87:0/3263154537 wait complete. 2024-04-25T10:21:28.270 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:21:28.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff50200640 1 Processor -- start 2024-04-25T10:21:28.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff50200640 1 -- start start 2024-04-25T10:21:28.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff50200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff480aef80 0x7eff4815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff50200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff480b31c0 0x7eff480b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff50200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4815b080 con 0x7eff48070350 2024-04-25T10:21:28.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff50200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4815adb0 con 0x7eff480b31c0 2024-04-25T10:21:28.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff50200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4815eb60 con 0x7eff480aef80 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff480b31c0 0x7eff480b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff480aef80 0x7eff4815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff480b31c0 0x7eff480b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:45366/0 (socket says 172.21.15.87:45366) 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4e200640 1 -- 172.21.15.87:0/3360483424 learned_addr learned my addr 172.21.15.87:0/3360483424 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4ce00640 1 --1- 172.21.15.87:0/3360483424 >> v1:172.21.15.169:6789/0 conn(0x7eff48070350 0x7eff4815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:59788/0 (socket says 172.21.15.87:59788) 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4e200640 1 -- 172.21.15.87:0/3360483424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff480aef80 msgr2=0x7eff4815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4e200640 1 --2- 172.21.15.87:0/3360483424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff480aef80 0x7eff4815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4e200640 1 -- 172.21.15.87:0/3360483424 >> v1:172.21.15.169:6789/0 conn(0x7eff48070350 legacy=0x7eff4815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.406+0000 7eff4e200640 1 -- 172.21.15.87:0/3360483424 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4815ee40 con 0x7eff480b31c0 2024-04-25T10:21:28.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff4e200640 1 --2- 172.21.15.87:0/3360483424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 0x7eff480b3610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7eff4000d150 tx=0x7eff4001ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f18e82d640295f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff47e00640 1 -- 172.21.15.87:0/3360483424 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4000b2d0 con 0x7eff480b31c0 2024-04-25T10:21:28.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff47e00640 1 -- 172.21.15.87:0/3360483424 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4000b470 con 0x7eff480b31c0 2024-04-25T10:21:28.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff47e00640 1 -- 172.21.15.87:0/3360483424 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400a7c30 con 0x7eff480b31c0 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 -- 172.21.15.87:0/3360483424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 msgr2=0x7eff480b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 --2- 172.21.15.87:0/3360483424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 0x7eff480b3610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7eff4000d150 tx=0x7eff4001ec30 comp rx=0 tx=0).stop 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 -- 172.21.15.87:0/3360483424 shutdown_connections 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 --2- 172.21.15.87:0/3360483424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 0x7eff480b3610 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 --2- 172.21.15.87:0/3360483424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff480aef80 0x7eff4815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 -- 172.21.15.87:0/3360483424 >> 172.21.15.87:0/3360483424 conn(0x7eff480669a0 msgr2=0x7eff48066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 -- 172.21.15.87:0/3360483424 shutdown_connections 2024-04-25T10:21:28.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.410+0000 7eff50200640 1 -- 172.21.15.87:0/3360483424 wait complete. 2024-04-25T10:21:28.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 Processor -- start 2024-04-25T10:21:28.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 -- start start 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff48070350 0x7eff4813b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff480aef80 0x7eff4814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 0x7eff481580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4806c250 con 0x7eff480aef80 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4806c0d0 con 0x7eff480b31c0 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4806c3d0 con 0x7eff48070350 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff480aef80 0x7eff4814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff480aef80 0x7eff4814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43812/0 (socket says 172.21.15.87:43812) 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 -- 172.21.15.87:0/596454740 learned_addr learned my addr 172.21.15.87:0/596454740 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:28.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4d800640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff48070350 0x7eff4813b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4e200640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 0x7eff481580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 -- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff48070350 msgr2=0x7eff4813b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff48070350 0x7eff4813b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 -- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 msgr2=0x7eff481580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 0x7eff481580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 -- 172.21.15.87:0/596454740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4813be20 con 0x7eff480aef80 2024-04-25T10:21:28.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff4ce00640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff480aef80 0x7eff4814fc70 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c420 tx=0x7eff3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff45600640 1 -- 172.21.15.87:0/596454740 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff340074e0 con 0x7eff480aef80 2024-04-25T10:21:28.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff45600640 1 -- 172.21.15.87:0/596454740 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff34007680 con 0x7eff480aef80 2024-04-25T10:21:28.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff48139290 con 0x7eff480aef80 2024-04-25T10:21:28.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff45600640 1 -- 172.21.15.87:0/596454740 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34004a30 con 0x7eff480aef80 2024-04-25T10:21:28.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.414+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff48139850 con 0x7eff480aef80 2024-04-25T10:21:28.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.418+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4813a020 con 0x7eff480aef80 2024-04-25T10:21:28.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.418+0000 7eff45600640 1 -- 172.21.15.87:0/596454740 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff3401a020 con 0x7eff480aef80 2024-04-25T10:21:28.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.418+0000 7eff45600640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff28066f60 0x7eff28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.418+0000 7eff45600640 1 -- 172.21.15.87:0/596454740 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7eff34084780 con 0x7eff480aef80 2024-04-25T10:21:28.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.418+0000 7eff4d800640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff28066f60 0x7eff28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.418+0000 7eff4d800640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff28066f60 0x7eff28069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009b60 tx=0x7eff3c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.422+0000 7eff45600640 1 -- 172.21.15.87:0/596454740 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff4813be20 con 0x7eff480aef80 2024-04-25T10:21:28.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff480b2fa0 con 0x7eff480aef80 2024-04-25T10:21:28.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff45600640 1 -- 172.21.15.87:0/596454740 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff340550f0 con 0x7eff480aef80 2024-04-25T10:21:28.580 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:21:28.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff28066f60 msgr2=0x7eff28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff28066f60 0x7eff28069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009b60 tx=0x7eff3c009190 comp rx=0 tx=0).stop 2024-04-25T10:21:28.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff480aef80 msgr2=0x7eff4814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff480aef80 0x7eff4814fc70 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c420 tx=0x7eff3400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 shutdown_connections 2024-04-25T10:21:28.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff28066f60 0x7eff28069420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff480b31c0 0x7eff481580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff480aef80 0x7eff4814fc70 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 --2- 172.21.15.87:0/596454740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff48070350 0x7eff4813b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 >> 172.21.15.87:0/596454740 conn(0x7eff480669a0 msgr2=0x7eff48066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 shutdown_connections 2024-04-25T10:21:28.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.578+0000 7eff50200640 1 -- 172.21.15.87:0/596454740 wait complete. 2024-04-25T10:21:28.602 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:21:28.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc183400640 1 Processor -- start 2024-04-25T10:21:28.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc183400640 1 -- start start 2024-04-25T10:21:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc183400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc17c0aef80 0x7fc17c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc183400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc183400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c15b080 con 0x7fc17c15bac0 2024-04-25T10:21:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc183400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17c15adb0 con 0x7fc17c0aef80 2024-04-25T10:21:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc183400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c15eb60 con 0x7fc17c070350 2024-04-25T10:21:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc180a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc17c0aef80 0x7fc17c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc181400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc17c15bac0 0x7fc17c0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45666/0 (socket says 172.21.15.87:45666) 2024-04-25T10:21:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc181400640 1 -- 172.21.15.87:0/3450702979 learned_addr learned my addr 172.21.15.87:0/3450702979 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc17be00640 1 --2- 172.21.15.87:0/3450702979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc180a00640 1 -- 172.21.15.87:0/3450702979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 msgr2=0x7fc17c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc180a00640 1 --2- 172.21.15.87:0/3450702979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc180a00640 1 -- 172.21.15.87:0/3450702979 >> v1:172.21.15.87:6789/0 conn(0x7fc17c15bac0 legacy=0x7fc17c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc180a00640 1 -- 172.21.15.87:0/3450702979 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c15ee40 con 0x7fc17c0aef80 2024-04-25T10:21:28.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.738+0000 7fc17b400640 1 -- 172.21.15.87:0/3450702979 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436586501 0 0) 0x7fc17c15b080 con 0x7fc17c15bac0 2024-04-25T10:21:28.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc180a00640 1 --2- 172.21.15.87:0/3450702979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c0aef80 0x7fc17c15e480 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc1700097c0 tx=0x7fc17001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2dfc3fcd6226a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc17b400640 1 -- 172.21.15.87:0/3450702979 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1700ab680 con 0x7fc17c0aef80 2024-04-25T10:21:28.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc17b400640 1 -- 172.21.15.87:0/3450702979 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1700b0070 con 0x7fc17c0aef80 2024-04-25T10:21:28.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc17b400640 1 -- 172.21.15.87:0/3450702979 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1700aba40 con 0x7fc17c0aef80 2024-04-25T10:21:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 -- 172.21.15.87:0/3450702979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c0aef80 msgr2=0x7fc17c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 --2- 172.21.15.87:0/3450702979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c0aef80 0x7fc17c15e480 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc1700097c0 tx=0x7fc17001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 -- 172.21.15.87:0/3450702979 shutdown_connections 2024-04-25T10:21:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 --2- 172.21.15.87:0/3450702979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 --2- 172.21.15.87:0/3450702979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c0aef80 0x7fc17c15e480 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 -- 172.21.15.87:0/3450702979 >> 172.21.15.87:0/3450702979 conn(0x7fc17c0669a0 msgr2=0x7fc17c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 -- 172.21.15.87:0/3450702979 shutdown_connections 2024-04-25T10:21:28.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.742+0000 7fc183400640 1 -- 172.21.15.87:0/3450702979 wait complete. 2024-04-25T10:21:28.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 Processor -- start 2024-04-25T10:21:28.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 -- start start 2024-04-25T10:21:28.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c0aef80 0x7fc17c14c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c15bac0 0x7fc17c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c167820 con 0x7fc17c15bac0 2024-04-25T10:21:28.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0b3b20 con 0x7fc17c0aef80 2024-04-25T10:21:28.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c1679a0 con 0x7fc17c070350 2024-04-25T10:21:28.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c15bac0 0x7fc17c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c15bac0 0x7fc17c06c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43822/0 (socket says 172.21.15.87:43822) 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 -- 172.21.15.87:0/2007891701 learned_addr learned my addr 172.21.15.87:0/2007891701 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc180a00640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 -- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 msgr2=0x7fc17c0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc17be00640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c0aef80 0x7fc17c14c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 -- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c0aef80 msgr2=0x7fc17c14c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c0aef80 0x7fc17c14c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 -- 172.21.15.87:0/2007891701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c14c700 con 0x7fc17c15bac0 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc180a00640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc17be00640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c0aef80 0x7fc17c14c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc181400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c15bac0 0x7fc17c06c0d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c700 tx=0x7fc17400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc178c00640 1 -- 172.21.15.87:0/2007891701 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174018070 con 0x7fc17c15bac0 2024-04-25T10:21:28.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc178c00640 1 -- 172.21.15.87:0/2007891701 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400ce10 con 0x7fc17c15bac0 2024-04-25T10:21:28.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c0b23b0 con 0x7fc17c15bac0 2024-04-25T10:21:28.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc178c00640 1 -- 172.21.15.87:0/2007891701 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400a860 con 0x7fc17c15bac0 2024-04-25T10:21:28.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c1f6a50 con 0x7fc17c15bac0 2024-04-25T10:21:28.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.746+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c04ef30 con 0x7fc17c15bac0 2024-04-25T10:21:28.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.750+0000 7fc178c00640 1 -- 172.21.15.87:0/2007891701 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc17401b070 con 0x7fc17c15bac0 2024-04-25T10:21:28.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.750+0000 7fc178c00640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc15c066f60 0x7fc15c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.750+0000 7fc178c00640 1 -- 172.21.15.87:0/2007891701 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc1740845d0 con 0x7fc17c15bac0 2024-04-25T10:21:28.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.750+0000 7fc180a00640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc15c066f60 0x7fc15c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.750+0000 7fc180a00640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc15c066f60 0x7fc15c069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc1700af000 tx=0x7fc170008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.754+0000 7fc178c00640 1 -- 172.21.15.87:0/2007891701 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc174055590 con 0x7fc17c15bac0 2024-04-25T10:21:28.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc17c071720 con 0x7fc17c15bac0 2024-04-25T10:21:28.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc178c00640 1 -- 172.21.15.87:0/2007891701 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc174054e90 con 0x7fc17c15bac0 2024-04-25T10:21:28.913 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc15c066f60 msgr2=0x7fc15c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc15c066f60 0x7fc15c069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc1700af000 tx=0x7fc170008040 comp rx=0 tx=0).stop 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c15bac0 msgr2=0x7fc17c06c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c15bac0 0x7fc17c06c0d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c700 tx=0x7fc17400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 shutdown_connections 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc15c066f60 0x7fc15c069420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc17c15bac0 0x7fc17c06c0d0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc17c0aef80 0x7fc17c14c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 --2- 172.21.15.87:0/2007891701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc17c070350 0x7fc17c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 >> 172.21.15.87:0/2007891701 conn(0x7fc17c0669a0 msgr2=0x7fc17c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 shutdown_connections 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:28.910+0000 7fc183400640 1 -- 172.21.15.87:0/2007891701 wait complete. 2024-04-25T10:21:28.933 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:29.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6412800640 1 Processor -- start 2024-04-25T10:21:29.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6412800640 1 -- start start 2024-04-25T10:21:29.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6412800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f640c0d1820 0x7f640c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6412800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f640c15e050 0x7f640c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6412800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640c0b1af0 con 0x7f640c0b13e0 2024-04-25T10:21:29.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6412800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0b1dd0 con 0x7f640c0d1820 2024-04-25T10:21:29.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6412800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0b20b0 con 0x7f640c15e050 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f640be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f640c0b13e0 0x7f640c0ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45682/0 (socket says 172.21.15.87:45682) 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f640be00640 1 -- 172.21.15.87:0/1373641391 learned_addr learned my addr 172.21.15.87:0/1373641391 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f640b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f640c0d1820 0x7f640c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.066+0000 7f6402a00640 1 --2- 172.21.15.87:0/1373641391 >> v2:172.21.15.169:3300/0 conn(0x7f640c15e050 0x7f640c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640b400640 1 -- 172.21.15.87:0/1373641391 >> v2:172.21.15.169:3300/0 conn(0x7f640c15e050 msgr2=0x7f640c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640b400640 1 --2- 172.21.15.87:0/1373641391 >> v2:172.21.15.169:3300/0 conn(0x7f640c15e050 0x7f640c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640b400640 1 -- 172.21.15.87:0/1373641391 >> v1:172.21.15.87:6789/0 conn(0x7f640c0b13e0 legacy=0x7f640c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640b400640 1 -- 172.21.15.87:0/1373641391 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0b2390 con 0x7f640c0d1820 2024-04-25T10:21:29.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640aa00640 1 -- 172.21.15.87:0/1373641391 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199114152 0 0) 0x7f640c0b1af0 con 0x7f640c0b13e0 2024-04-25T10:21:29.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640b400640 1 --2- 172.21.15.87:0/1373641391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0d1820 0x7f640c0d1c00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f640c0c5400 tx=0x7f63fc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2223c6cead13c982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640aa00640 1 -- 172.21.15.87:0/1373641391 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0aa600 con 0x7f640c0d1820 2024-04-25T10:21:29.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640aa00640 1 -- 172.21.15.87:0/1373641391 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0af070 con 0x7f640c0d1820 2024-04-25T10:21:29.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f640aa00640 1 -- 172.21.15.87:0/1373641391 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0aa9c0 con 0x7f640c0d1820 2024-04-25T10:21:29.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 -- 172.21.15.87:0/1373641391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0d1820 msgr2=0x7f640c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 --2- 172.21.15.87:0/1373641391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0d1820 0x7f640c0d1c00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f640c0c5400 tx=0x7f63fc01c480 comp rx=0 tx=0).stop 2024-04-25T10:21:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 -- 172.21.15.87:0/1373641391 shutdown_connections 2024-04-25T10:21:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 --2- 172.21.15.87:0/1373641391 >> v2:172.21.15.169:3300/0 conn(0x7f640c15e050 0x7f640c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 --2- 172.21.15.87:0/1373641391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0d1820 0x7f640c0d1c00 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 -- 172.21.15.87:0/1373641391 >> 172.21.15.87:0/1373641391 conn(0x7f640c09a9d0 msgr2=0x7f640c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 -- 172.21.15.87:0/1373641391 shutdown_connections 2024-04-25T10:21:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.070+0000 7f6412800640 1 -- 172.21.15.87:0/1373641391 wait complete. 2024-04-25T10:21:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 Processor -- start 2024-04-25T10:21:29.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 -- start start 2024-04-25T10:21:29.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0b13e0 0x7f640c151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c0d1820 0x7f640c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f640c15e050 0x7f640c06e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c16ab20 con 0x7f640c0b13e0 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c16a9a0 con 0x7f640c0d1820 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c16aca0 con 0x7f640c15e050 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0b13e0 0x7f640c151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0b13e0 0x7f640c151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43848/0 (socket says 172.21.15.87:43848) 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 -- 172.21.15.87:0/3302295915 learned_addr learned my addr 172.21.15.87:0/3302295915 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640be00640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f640c15e050 0x7f640c06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6402a00640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c0d1820 0x7f640c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 -- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f640c15e050 msgr2=0x7f640c06e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f640c15e050 0x7f640c06e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 -- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c0d1820 msgr2=0x7f640c157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c0d1820 0x7f640c157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 -- 172.21.15.87:0/3302295915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c150e00 con 0x7f640c0b13e0 2024-04-25T10:21:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f640b400640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0b13e0 0x7f640c151ef0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0ac040 tx=0x7f63fc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6403e00640 1 -- 172.21.15.87:0/3302295915 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0be610 con 0x7f640c0b13e0 2024-04-25T10:21:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6403e00640 1 -- 172.21.15.87:0/3302295915 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0be7b0 con 0x7f640c0b13e0 2024-04-25T10:21:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c1510e0 con 0x7f640c0b13e0 2024-04-25T10:21:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6403e00640 1 -- 172.21.15.87:0/3302295915 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0beba0 con 0x7f640c0b13e0 2024-04-25T10:21:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.074+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c1516a0 con 0x7f640c0b13e0 2024-04-25T10:21:29.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.078+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c04ef30 con 0x7f640c0b13e0 2024-04-25T10:21:29.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.078+0000 7f6403e00640 1 -- 172.21.15.87:0/3302295915 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63fc0bc020 con 0x7f640c0b13e0 2024-04-25T10:21:29.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.078+0000 7f6403e00640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63e8066f90 0x7f63e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.078+0000 7f6403e00640 1 -- 172.21.15.87:0/3302295915 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f63fc11f7c0 con 0x7f640c0b13e0 2024-04-25T10:21:29.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.078+0000 7f6402a00640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63e8066f90 0x7f63e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.078+0000 7f6402a00640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63e8066f90 0x7f63e8069450 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f640c15f4d0 tx=0x7f63f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.082+0000 7f6403e00640 1 -- 172.21.15.87:0/3302295915 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63fc016070 con 0x7f640c0b13e0 2024-04-25T10:21:29.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f640c0741a0 con 0x7f640c0b13e0 2024-04-25T10:21:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6403e00640 1 -- 172.21.15.87:0/3302295915 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f63fc0f0660 con 0x7f640c0b13e0 2024-04-25T10:21:29.273 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:29.273 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:29.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63e8066f90 msgr2=0x7f63e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63e8066f90 0x7f63e8069450 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f640c15f4d0 tx=0x7f63f4008040 comp rx=0 tx=0).stop 2024-04-25T10:21:29.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0b13e0 msgr2=0x7f640c151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0b13e0 0x7f640c151ef0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0ac040 tx=0x7f63fc01acc0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 shutdown_connections 2024-04-25T10:21:29.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f63e8066f90 0x7f63e8069450 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f640c15e050 0x7f640c06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c0d1820 0x7f640c157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 --2- 172.21.15.87:0/3302295915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f640c0b13e0 0x7f640c151ef0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 >> 172.21.15.87:0/3302295915 conn(0x7f640c09a9d0 msgr2=0x7f640c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 shutdown_connections 2024-04-25T10:21:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.270+0000 7f6412800640 1 -- 172.21.15.87:0/3302295915 wait complete. 2024-04-25T10:21:29.294 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:29.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f3413a00640 1 Processor -- start 2024-04-25T10:21:29.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f3413a00640 1 -- start start 2024-04-25T10:21:29.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f3413a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 0x7f340c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f3413a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f340c15b910 0x7f340c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f3413a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c071050 con 0x7f340c0cfe70 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f3413a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c15bda0 con 0x7f340c15b910 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f3413a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c15d040 con 0x7f340c1550b0 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f340be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f340c0cfe70 0x7f340c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45692/0 (socket says 172.21.15.87:45692) 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.426+0000 7f340be00640 1 -- 172.21.15.87:0/2092939551 learned_addr learned my addr 172.21.15.87:0/2092939551 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411000640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 0x7f340c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 --2- 172.21.15.87:0/2092939551 >> v2:172.21.15.169:3300/0 conn(0x7f340c15b910 0x7f340c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227536901 0 0) 0x7f340c071050 con 0x7f340c0cfe70 2024-04-25T10:21:29.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f0003530 con 0x7f340c0cfe70 2024-04-25T10:21:29.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 384367731 0 0) 0x7f33f0003530 con 0x7f340c0cfe70 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 msgr2=0x7f340c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 0x7f340c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 >> v2:172.21.15.169:3300/0 conn(0x7f340c15b910 msgr2=0x7f340c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 --2- 172.21.15.87:0/2092939551 >> v2:172.21.15.169:3300/0 conn(0x7f340c15b910 0x7f340c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0712f0 con 0x7f340c0cfe70 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f33f8001680 con 0x7f340c0cfe70 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 >> v1:172.21.15.87:6789/0 conn(0x7f340c0cfe70 legacy=0x7f340c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 0x7f33f00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33f0003f40 0x7f33f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f33f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c15d040 con 0x7f33f0004d50 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f000b9d0 con 0x7f33f0003f40 2024-04-25T10:21:29.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f000bb50 con 0x7f340c1550b0 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 --2- 172.21.15.87:0/2092939551 >> v2:172.21.15.169:3300/0 conn(0x7f340c15b910 0x7f340c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411000640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 0x7f33f00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f33f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f340be00640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33f0003f40 0x7f33f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 -- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 msgr2=0x7f33f00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 0x7f33f00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 -- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33f0003f40 msgr2=0x7f33f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33f0003f40 0x7f33f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411a00640 1 -- 172.21.15.87:0/2092939551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f0003530 con 0x7f33f0004d50 2024-04-25T10:21:29.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.430+0000 7f3411000640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1550b0 0x7f33f00039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:29.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3411a00640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f33f00051d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3404009c90 tx=0x7f340401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a6beb3f5cc9743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404009860 con 0x7f33f0004d50 2024-04-25T10:21:29.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3404004020 con 0x7f33f0004d50 2024-04-25T10:21:29.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f340b400640 1 -- 172.21.15.87:0/2092939551 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404004450 con 0x7f33f0004d50 2024-04-25T10:21:29.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- 172.21.15.87:0/2092939551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 msgr2=0x7f33f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 --2- 172.21.15.87:0/2092939551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f33f00051d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3404009c90 tx=0x7f340401cc10 comp rx=0 tx=0).stop 2024-04-25T10:21:29.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3411a00640 1 -- 172.21.15.87:0/2092939551 reap_dead start 2024-04-25T10:21:29.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- 172.21.15.87:0/2092939551 shutdown_connections 2024-04-25T10:21:29.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- 172.21.15.87:0/2092939551 >> 172.21.15.87:0/2092939551 conn(0x7f340c066b70 msgr2=0x7f340c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- 172.21.15.87:0/2092939551 shutdown_connections 2024-04-25T10:21:29.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- 172.21.15.87:0/2092939551 wait complete. 2024-04-25T10:21:29.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 Processor -- start 2024-04-25T10:21:29.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- start start 2024-04-25T10:21:29.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f340c146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1471c0 0x7f340c147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f340c147b80 0x7f340c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c071b90 con 0x7f33f0004d50 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0718b0 con 0x7f340c147b80 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.434+0000 7f3413a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0720b0 con 0x7f340c1471c0 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f340c146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f340c146b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43868/0 (socket says 172.21.15.87:43868) 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 -- 172.21.15.87:0/707087936 learned_addr learned my addr 172.21.15.87:0/707087936 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f340be00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1471c0 0x7f340c147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411a00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f340c147b80 0x7f340c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 -- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1471c0 msgr2=0x7f340c147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1471c0 0x7f340c147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 -- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f340c147b80 msgr2=0x7f340c158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f340c147b80 0x7f340c158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 -- 172.21.15.87:0/707087936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0cd3e0 con 0x7f33f0004d50 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3411000640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f340c146b10 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00f460 tx=0x7f33fc00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3408c00640 1 -- 172.21.15.87:0/707087936 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc003ba0 con 0x7f33f0004d50 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3408c00640 1 -- 172.21.15.87:0/707087936 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc003d40 con 0x7f33f0004d50 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3408c00640 1 -- 172.21.15.87:0/707087936 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc015860 con 0x7f33f0004d50 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c0cd6c0 con 0x7f33f0004d50 2024-04-25T10:21:29.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c0cdc80 con 0x7f33f0004d50 2024-04-25T10:21:29.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c04ef30 con 0x7f33f0004d50 2024-04-25T10:21:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.438+0000 7f3408c00640 1 -- 172.21.15.87:0/707087936 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33fc015a00 con 0x7f33f0004d50 2024-04-25T10:21:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.442+0000 7f3408c00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.442+0000 7f3408c00640 1 -- 172.21.15.87:0/707087936 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f33fc0861c0 con 0x7f33f0004d50 2024-04-25T10:21:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.442+0000 7f340be00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.442+0000 7f340be00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f33f8001060 tx=0x7f33f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.446+0000 7f3408c00640 1 -- 172.21.15.87:0/707087936 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33fc019030 con 0x7f33f0004d50 2024-04-25T10:21:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.598+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f340c150460 con 0x7f33f0004d50 2024-04-25T10:21:29.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3408c00640 1 -- 172.21.15.87:0/707087936 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16709 (secure 0 0 0) 0x7f33fc010040 con 0x7f33f0004d50 2024-04-25T10:21:29.604 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:29.605 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":113,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:06.984221+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:37.371368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:38.004409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 msgr2=0x7f33ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f33f8001060 tx=0x7f33f8009040 comp rx=0 tx=0).stop 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 msgr2=0x7f340c146b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f340c146b10 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00f460 tx=0x7f33fc00f920 comp rx=0 tx=0).stop 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 shutdown_connections 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f340c147b80 0x7f340c158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f340c1471c0 0x7f340c147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 --2- 172.21.15.87:0/707087936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f0004d50 0x7f340c146b10 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 >> 172.21.15.87:0/707087936 conn(0x7f340c066b70 msgr2=0x7f340c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 shutdown_connections 2024-04-25T10:21:29.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.602+0000 7f3413a00640 1 -- 172.21.15.87:0/707087936 wait complete. 2024-04-25T10:21:29.626 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:21:29.626 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:29.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.758+0000 7f37b2400640 1 Processor -- start 2024-04-25T10:21:29.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.758+0000 7f37b2400640 1 -- start start 2024-04-25T10:21:29.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37b2400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f37ac15bc40 0x7f37ac06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37b2400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0b2770 con 0x7f37ac0cda10 2024-04-25T10:21:29.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37b2400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac06c670 con 0x7f37ac0cf930 2024-04-25T10:21:29.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37b2400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac06c950 con 0x7f37ac15bc40 2024-04-25T10:21:29.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f37ac0cda10 0x7f37ac0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45704/0 (socket says 172.21.15.87:45704) 2024-04-25T10:21:29.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 -- 172.21.15.87:0/4179116906 learned_addr learned my addr 172.21.15.87:0/4179116906 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37ab400640 1 --2- 172.21.15.87:0/4179116906 >> v2:172.21.15.169:3300/0 conn(0x7f37ac15bc40 0x7f37ac06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463548367 0 0) 0x7f37ac0b2770 con 0x7f37ac0cda10 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3790003530 con 0x7f37ac0cda10 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042536181 0 0) 0x7f3790003530 con 0x7f37ac0cda10 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 >> v2:172.21.15.169:3300/0 conn(0x7f37ac15bc40 msgr2=0x7f37ac06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 --2- 172.21.15.87:0/4179116906 >> v2:172.21.15.169:3300/0 conn(0x7f37ac15bc40 0x7f37ac06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 >> v1:172.21.15.169:6789/0 conn(0x7f37ac0cf930 legacy=0x7f37ac0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac06cc30 con 0x7f37ac0cda10 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f3794001680 con 0x7f37ac0cda10 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 >> v1:172.21.15.87:6789/0 conn(0x7f37ac0cda10 legacy=0x7f37ac0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3790003a10 0x7f3790003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f3790004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790005220 0x7f379000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac06c950 con 0x7f3790004810 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379000c170 con 0x7f3790003a10 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379000c370 con 0x7f3790005220 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f3790004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37ab400640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3790003a10 0x7f3790003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37abe00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790005220 0x7f379000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 -- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790005220 msgr2=0x7f379000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790005220 0x7f379000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 -- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3790003a10 msgr2=0x7f3790003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3790003a10 0x7f3790003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37a2a00640 1 -- 172.21.15.87:0/4179116906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3790003530 con 0x7f3790004810 2024-04-25T10:21:29.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.762+0000 7f37abe00640 1 -- 172.21.15.87:0/4179116906 reap_dead start 2024-04-25T10:21:29.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37a2a00640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f3790004c90 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f379400c0e0 tx=0x7f379401cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49761910aaf80238 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794007c50 con 0x7f3790004810 2024-04-25T10:21:29.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37940a9070 con 0x7f3790004810 2024-04-25T10:21:29.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37aaa00640 1 -- 172.21.15.87:0/4179116906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794005d80 con 0x7f3790004810 2024-04-25T10:21:29.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 -- 172.21.15.87:0/4179116906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 msgr2=0x7f3790004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f3790004c90 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f379400c0e0 tx=0x7f379401cb40 comp rx=0 tx=0).stop 2024-04-25T10:21:29.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 -- 172.21.15.87:0/4179116906 shutdown_connections 2024-04-25T10:21:29.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 --2- 172.21.15.87:0/4179116906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f3790004c90 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 -- 172.21.15.87:0/4179116906 >> 172.21.15.87:0/4179116906 conn(0x7f37ac066b70 msgr2=0x7f37ac066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 -- 172.21.15.87:0/4179116906 shutdown_connections 2024-04-25T10:21:29.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 -- 172.21.15.87:0/4179116906 wait complete. 2024-04-25T10:21:29.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 Processor -- start 2024-04-25T10:21:29.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.766+0000 7f37b2400640 1 -- start start 2024-04-25T10:21:29.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f37ac1e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37ac1e89f0 0x7f37ac153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac15e8e0 0x7f37ac15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0d00d0 con 0x7f3790004810 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0cff50 con 0x7f37ac15e8e0 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0d0250 con 0x7f37ac1e89f0 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f37ac1e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f37ac1e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43886/0 (socket says 172.21.15.87:43886) 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 -- 172.21.15.87:0/486231325 learned_addr learned my addr 172.21.15.87:0/486231325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37a2a00640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37ac1e89f0 0x7f37ac153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37abe00640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac15e8e0 0x7f37ac15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 -- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37ac1e89f0 msgr2=0x7f37ac153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37ac1e89f0 0x7f37ac153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 -- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac15e8e0 msgr2=0x7f37ac15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac15e8e0 0x7f37ac15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 -- 172.21.15.87:0/486231325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac15cc10 con 0x7f3790004810 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37ab400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f37ac1e81e0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f379c011960 tx=0x7f379c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37a3e00640 1 -- 172.21.15.87:0/486231325 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c019070 con 0x7f3790004810 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37a3e00640 1 -- 172.21.15.87:0/486231325 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c004240 con 0x7f3790004810 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37a3e00640 1 -- 172.21.15.87:0/486231325 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00f880 con 0x7f3790004810 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac1e9150 con 0x7f3790004810 2024-04-25T10:21:29.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac1e9710 con 0x7f3790004810 2024-04-25T10:21:29.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.770+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac04ef30 con 0x7f3790004810 2024-04-25T10:21:29.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.774+0000 7f37a3e00640 1 -- 172.21.15.87:0/486231325 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f379c01c070 con 0x7f3790004810 2024-04-25T10:21:29.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.774+0000 7f37a3e00640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3788066f60 0x7f3788069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.774+0000 7f37a3e00640 1 -- 172.21.15.87:0/486231325 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f379c0854c0 con 0x7f3790004810 2024-04-25T10:21:29.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.774+0000 7f37a2a00640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3788066f60 0x7f3788069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.774+0000 7f37a2a00640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3788066f60 0x7f3788069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f37940ab000 tx=0x7f37940b0040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.778+0000 7f37a3e00640 1 -- 172.21.15.87:0/486231325 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f379c0563f0 con 0x7f3790004810 2024-04-25T10:21:29.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.962+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37ac0af1c0 con 0x7f3790004810 2024-04-25T10:21:29.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.962+0000 7f37a3e00640 1 -- 172.21.15.87:0/486231325 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f379c055cf0 con 0x7f3790004810 2024-04-25T10:21:29.966 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:29.966 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:29.967 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:29.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3788066f60 msgr2=0x7f3788069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3788066f60 0x7f3788069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f37940ab000 tx=0x7f37940b0040 comp rx=0 tx=0).stop 2024-04-25T10:21:29.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 msgr2=0x7f37ac1e81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f37ac1e81e0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f379c011960 tx=0x7f379c011e20 comp rx=0 tx=0).stop 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 shutdown_connections 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3788066f60 0x7f3788069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37ac15e8e0 0x7f37ac15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37ac1e89f0 0x7f37ac153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 --2- 172.21.15.87:0/486231325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3790004810 0x7f37ac1e81e0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 >> 172.21.15.87:0/486231325 conn(0x7f37ac066b70 msgr2=0x7f37ac0b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 shutdown_connections 2024-04-25T10:21:29.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:29.966+0000 7f37b2400640 1 -- 172.21.15.87:0/486231325 wait complete. 2024-04-25T10:21:29.987 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:30.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa92e400640 1 Processor -- start 2024-04-25T10:21:30.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa92e400640 1 -- start start 2024-04-25T10:21:30.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa92e400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa928070c20 0x7fa92815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa92e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9280ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa92e400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280ad840 con 0x7fa9280cf930 2024-04-25T10:21:30.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa92e400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92815bb90 con 0x7fa928070c20 2024-04-25T10:21:30.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa92e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92815e9f0 con 0x7fa9280ae020 2024-04-25T10:21:30.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa91ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa928070c20 0x7fa92815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa91ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa928070c20 0x7fa92815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43888/0 (socket says 172.21.15.87:43888) 2024-04-25T10:21:30.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa91ea00640 1 -- 172.21.15.87:0/2815836694 learned_addr learned my addr 172.21.15.87:0/2815836694 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:30.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa927e00640 1 --2- 172.21.15.87:0/2815836694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9280ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa91ea00640 1 -- 172.21.15.87:0/2815836694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 msgr2=0x7fa9280ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:30.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa91ea00640 1 --2- 172.21.15.87:0/2815836694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9280ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa91ea00640 1 -- 172.21.15.87:0/2815836694 >> v1:172.21.15.169:6789/0 conn(0x7fa9280cf930 legacy=0x7fa9280cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:30.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa91ea00640 1 -- 172.21.15.87:0/2815836694 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92815ecd0 con 0x7fa928070c20 2024-04-25T10:21:30.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.122+0000 7fa927e00640 1 --2- 172.21.15.87:0/2815836694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9280ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:30.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa91ea00640 1 --2- 172.21.15.87:0/2815836694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa928070c20 0x7fa92815e170 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa910009870 tx=0x7fa91001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e2ef1291f0507f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa926a00640 1 -- 172.21.15.87:0/2815836694 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100aa610 con 0x7fa928070c20 2024-04-25T10:21:30.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa926a00640 1 -- 172.21.15.87:0/2815836694 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9100af070 con 0x7fa928070c20 2024-04-25T10:21:30.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa926a00640 1 -- 172.21.15.87:0/2815836694 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100aa9d0 con 0x7fa928070c20 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 -- 172.21.15.87:0/2815836694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa928070c20 msgr2=0x7fa92815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 --2- 172.21.15.87:0/2815836694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa928070c20 0x7fa92815e170 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa910009870 tx=0x7fa91001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 -- 172.21.15.87:0/2815836694 shutdown_connections 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 --2- 172.21.15.87:0/2815836694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9280ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 --2- 172.21.15.87:0/2815836694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa928070c20 0x7fa92815e170 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 -- 172.21.15.87:0/2815836694 >> 172.21.15.87:0/2815836694 conn(0x7fa928066b70 msgr2=0x7fa928066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 -- 172.21.15.87:0/2815836694 shutdown_connections 2024-04-25T10:21:30.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.126+0000 7fa92e400640 1 -- 172.21.15.87:0/2815836694 wait complete. 2024-04-25T10:21:30.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 Processor -- start 2024-04-25T10:21:30.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 -- start start 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa928070c20 0x7fa928155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9281554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9280cf930 0x7fa928153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92816a8e0 con 0x7fa9280cf930 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92815e6b0 con 0x7fa928070c20 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92816aa60 con 0x7fa9280ae020 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9280cf930 0x7fa928153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9280cf930 0x7fa928153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43890/0 (socket says 172.21.15.87:43890) 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 -- 172.21.15.87:0/3951354191 learned_addr learned my addr 172.21.15.87:0/3951354191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa91ea00640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9281554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa928070c20 0x7fa928155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 -- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 msgr2=0x7fa9281554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9281554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 -- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa928070c20 msgr2=0x7fa928155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa928070c20 0x7fa928155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 -- 172.21.15.87:0/3951354191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928156330 con 0x7fa9280cf930 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa927e00640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9280cf930 0x7fa928153e80 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa92000cef0 tx=0x7fa92000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa91fe00640 1 -- 172.21.15.87:0/3951354191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92001a070 con 0x7fa9280cf930 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa91fe00640 1 -- 172.21.15.87:0/3951354191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92000d860 con 0x7fa9280cf930 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9281565d0 con 0x7fa9280cf930 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa91fe00640 1 -- 172.21.15.87:0/3951354191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000dc50 con 0x7fa9280cf930 2024-04-25T10:21:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.130+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa928153680 con 0x7fa9280cf930 2024-04-25T10:21:30.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.134+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928153b60 con 0x7fa9280cf930 2024-04-25T10:21:30.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.134+0000 7fa91fe00640 1 -- 172.21.15.87:0/3951354191 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa92001d070 con 0x7fa9280cf930 2024-04-25T10:21:30.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.134+0000 7fa91fe00640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa904066f60 0x7fa904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.134+0000 7fa91fe00640 1 -- 172.21.15.87:0/3951354191 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa920085880 con 0x7fa9280cf930 2024-04-25T10:21:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.134+0000 7fa927400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa904066f60 0x7fa904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.134+0000 7fa927400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa904066f60 0x7fa904069420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098e0 tx=0x7fa918009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.138+0000 7fa91fe00640 1 -- 172.21.15.87:0/3951354191 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa928153b60 con 0x7fa9280cf930 2024-04-25T10:21:30.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa92804ef30 con 0x7fa9280cf930 2024-04-25T10:21:30.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa91fe00640 1 -- 172.21.15.87:0/3951354191 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fa9200562b0 con 0x7fa9280cf930 2024-04-25T10:21:30.329 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:30.329 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:39.010608+0000","modified":"2024-04-25T10:20:42.026817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8523":{"gid":8523,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/3078360268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3078360268},{"type":"v1","addr":"172.21.15.87:6837","nonce":3078360268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:21:30.330 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 80 2024-04-25T10:21:30.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa904066f60 msgr2=0x7fa904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa904066f60 0x7fa904069420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098e0 tx=0x7fa918009060 comp rx=0 tx=0).stop 2024-04-25T10:21:30.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9280cf930 msgr2=0x7fa928153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9280cf930 0x7fa928153e80 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa92000cef0 tx=0x7fa92000a710 comp rx=0 tx=0).stop 2024-04-25T10:21:30.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 shutdown_connections 2024-04-25T10:21:30.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa904066f60 0x7fa904069420 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9280cf930 0x7fa928153e80 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280ae020 0x7fa9281554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 --2- 172.21.15.87:0/3951354191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa928070c20 0x7fa928155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 >> 172.21.15.87:0/3951354191 conn(0x7fa928066b70 msgr2=0x7fa9280cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:30.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 shutdown_connections 2024-04-25T10:21:30.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.326+0000 7fa92e400640 1 -- 172.21.15.87:0/3951354191 wait complete. 2024-04-25T10:21:30.351 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:30.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.482+0000 7f55f0c00640 1 Processor -- start 2024-04-25T10:21:30.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.482+0000 7f55f0c00640 1 -- start start 2024-04-25T10:21:30.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55f0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55f0c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55ec066fe0 0x7f55ec0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55f0c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ec0b5c20 con 0x7f55ec06ed30 2024-04-25T10:21:30.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55f0c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ec0af6e0 con 0x7f55ec066fe0 2024-04-25T10:21:30.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55f0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0af9c0 con 0x7f55ec066c00 2024-04-25T10:21:30.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55ec066fe0 0x7f55ec0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55ec066fe0 0x7f55ec0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:45464/0 (socket says 172.21.15.87:45464) 2024-04-25T10:21:30.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:50118/0 (socket says 172.21.15.87:50118) 2024-04-25T10:21:30.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9400640 1 -- 172.21.15.87:0/351940625 learned_addr learned my addr 172.21.15.87:0/351940625 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:30.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9400640 1 -- 172.21.15.87:0/351940625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 msgr2=0x7f55ec15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:30.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9400640 1 --2- 172.21.15.87:0/351940625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9400640 1 -- 172.21.15.87:0/351940625 >> v1:172.21.15.169:6789/0 conn(0x7f55ec06ed30 legacy=0x7f55ec157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.486+0000 7f55e9400640 1 -- 172.21.15.87:0/351940625 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0afca0 con 0x7f55ec066fe0 2024-04-25T10:21:30.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55e9400640 1 --2- 172.21.15.87:0/351940625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec066fe0 0x7f55ec0b5560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009870 tx=0x7f55d801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7767ad9691cdf3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55e8a00640 1 -- 172.21.15.87:0/351940625 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80a9610 con 0x7f55ec066fe0 2024-04-25T10:21:30.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55e8a00640 1 -- 172.21.15.87:0/351940625 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d80a97b0 con 0x7f55ec066fe0 2024-04-25T10:21:30.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55e8a00640 1 -- 172.21.15.87:0/351940625 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80a9b80 con 0x7f55ec066fe0 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- 172.21.15.87:0/351940625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec066fe0 msgr2=0x7f55ec0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 --2- 172.21.15.87:0/351940625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec066fe0 0x7f55ec0b5560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009870 tx=0x7f55d801c530 comp rx=0 tx=0).stop 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- 172.21.15.87:0/351940625 shutdown_connections 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 --2- 172.21.15.87:0/351940625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec066fe0 0x7f55ec0b5560 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 --2- 172.21.15.87:0/351940625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- 172.21.15.87:0/351940625 >> 172.21.15.87:0/351940625 conn(0x7f55ec0b24f0 msgr2=0x7f55ec0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- 172.21.15.87:0/351940625 shutdown_connections 2024-04-25T10:21:30.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- 172.21.15.87:0/351940625 wait complete. 2024-04-25T10:21:30.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 Processor -- start 2024-04-25T10:21:30.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- start start 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55ec066fe0 0x7f55ec14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec06ed30 0x7f55ec148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec06ac90 con 0x7f55ec066fe0 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0af270 con 0x7f55ec06ed30 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.490+0000 7f55f0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec06ae10 con 0x7f55ec066c00 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55ec066fe0 0x7f55ec14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55ec066fe0 0x7f55ec14bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43900/0 (socket says 172.21.15.87:43900) 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 -- 172.21.15.87:0/2174470081 learned_addr learned my addr 172.21.15.87:0/2174470081 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9e00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55ea800640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec06ed30 0x7f55ec148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 -- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 msgr2=0x7f55ec15b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 -- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec06ed30 msgr2=0x7f55ec148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec06ed30 0x7f55ec148360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 -- 172.21.15.87:0/2174470081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec14c4b0 con 0x7f55ec066fe0 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e9400640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55ec066fe0 0x7f55ec14bc60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f55d801a160 tx=0x7f55d801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e2000640 1 -- 172.21.15.87:0/2174470081 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80a9610 con 0x7f55ec066fe0 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e2000640 1 -- 172.21.15.87:0/2174470081 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d80bc670 con 0x7f55ec066fe0 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ec14c790 con 0x7f55ec066fe0 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e2000640 1 -- 172.21.15.87:0/2174470081 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80bca60 con 0x7f55ec066fe0 2024-04-25T10:21:30.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec14cd50 con 0x7f55ec066fe0 2024-04-25T10:21:30.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ec04ef30 con 0x7f55ec066fe0 2024-04-25T10:21:30.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.494+0000 7f55e2000640 1 -- 172.21.15.87:0/2174470081 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55d80bcc00 con 0x7f55ec066fe0 2024-04-25T10:21:30.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.498+0000 7f55e2000640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f55cc066f60 0x7f55cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.498+0000 7f55e2000640 1 -- 172.21.15.87:0/2174470081 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f55d811f670 con 0x7f55ec066fe0 2024-04-25T10:21:30.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.498+0000 7f55e9e00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f55cc066f60 0x7f55cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.498+0000 7f55e9e00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f55cc066f60 0x7f55cc069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f55ec1fa490 tx=0x7f55dc00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.502+0000 7f55e2000640 1 -- 172.21.15.87:0/2174470081 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55d80ad020 con 0x7f55ec066fe0 2024-04-25T10:21:30.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55ec1698d0 con 0x7f55ec066fe0 2024-04-25T10:21:30.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55e2000640 1 -- 172.21.15.87:0/2174470081 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16709 (secure 0 0 0) 0x7f55d8016150 con 0x7f55ec066fe0 2024-04-25T10:21:30.661 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:30.661 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":113,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:06.984221+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:37.371368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:38.004409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f55cc066f60 msgr2=0x7f55cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f55cc066f60 0x7f55cc069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f55ec1fa490 tx=0x7f55dc00b040 comp rx=0 tx=0).stop 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55ec066fe0 msgr2=0x7f55ec14bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55ec066fe0 0x7f55ec14bc60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f55d801a160 tx=0x7f55d801ad70 comp rx=0 tx=0).stop 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 shutdown_connections 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f55cc066f60 0x7f55cc069420 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ec06ed30 0x7f55ec148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55ec066fe0 0x7f55ec14bc60 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 --2- 172.21.15.87:0/2174470081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55ec066c00 0x7f55ec15b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.658+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 >> 172.21.15.87:0/2174470081 conn(0x7f55ec0b24f0 msgr2=0x7f55ec06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.662+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 shutdown_connections 2024-04-25T10:21:30.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.662+0000 7f55f0c00640 1 -- 172.21.15.87:0/2174470081 wait complete. 2024-04-25T10:21:30.682 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:21:30.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42be00640 1 Processor -- start 2024-04-25T10:21:30.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42be00640 1 -- start start 2024-04-25T10:21:30.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc42c0b3360 0x7fc42c15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c0b5200 con 0x7fc42c0cf9b0 2024-04-25T10:21:30.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c0b38b0 con 0x7fc42c071120 2024-04-25T10:21:30.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42c15ef10 con 0x7fc42c0b3360 2024-04-25T10:21:30.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc42c0cf9b0 0x7fc42c070a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45708/0 (socket says 172.21.15.87:45708) 2024-04-25T10:21:30.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42aa00640 1 -- 172.21.15.87:0/3657176941 learned_addr learned my addr 172.21.15.87:0/3657176941 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42b400640 1 --2- 172.21.15.87:0/3657176941 >> v2:172.21.15.87:3300/0 conn(0x7fc42c0b3360 0x7fc42c15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42a000640 1 -- 172.21.15.87:0/3657176941 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382778841 0 0) 0x7fc42c0b5200 con 0x7fc42c0cf9b0 2024-04-25T10:21:30.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42a000640 1 -- 172.21.15.87:0/3657176941 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc410003530 con 0x7fc42c0cf9b0 2024-04-25T10:21:30.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42b400640 1 -- 172.21.15.87:0/3657176941 >> v1:172.21.15.169:6789/0 conn(0x7fc42c071120 legacy=0x7fc42c1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42b400640 1 -- 172.21.15.87:0/3657176941 >> v1:172.21.15.87:6789/0 conn(0x7fc42c0cf9b0 legacy=0x7fc42c070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.818+0000 7fc42b400640 1 -- 172.21.15.87:0/3657176941 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c0b3b90 con 0x7fc42c0b3360 2024-04-25T10:21:30.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42b400640 1 --2- 172.21.15.87:0/3657176941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c0b3360 0x7fc42c15e9d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc424009d90 tx=0x7fc42401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34da2ad46afbd870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42a000640 1 -- 172.21.15.87:0/3657176941 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4240b2070 con 0x7fc42c0b3360 2024-04-25T10:21:30.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42a000640 1 -- 172.21.15.87:0/3657176941 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4240a58b0 con 0x7fc42c0b3360 2024-04-25T10:21:30.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42a000640 1 -- 172.21.15.87:0/3657176941 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4240a5c80 con 0x7fc42c0b3360 2024-04-25T10:21:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 -- 172.21.15.87:0/3657176941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c0b3360 msgr2=0x7fc42c15e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 --2- 172.21.15.87:0/3657176941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c0b3360 0x7fc42c15e9d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc424009d90 tx=0x7fc42401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 -- 172.21.15.87:0/3657176941 shutdown_connections 2024-04-25T10:21:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 --2- 172.21.15.87:0/3657176941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c0b3360 0x7fc42c15e9d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 -- 172.21.15.87:0/3657176941 >> 172.21.15.87:0/3657176941 conn(0x7fc42c0cfe80 msgr2=0x7fc42c0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 -- 172.21.15.87:0/3657176941 shutdown_connections 2024-04-25T10:21:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 -- 172.21.15.87:0/3657176941 wait complete. 2024-04-25T10:21:30.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 Processor -- start 2024-04-25T10:21:30.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.822+0000 7fc42be00640 1 -- start start 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c071120 0x7fc42c1495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c0b3360 0x7fc42c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc42c0cf9b0 0x7fc42c146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c16bcd0 con 0x7fc42c071120 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c16bb50 con 0x7fc42c0b3360 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c16be50 con 0x7fc42c0cf9b0 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c071120 0x7fc42c1495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c071120 0x7fc42c1495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43920/0 (socket says 172.21.15.87:43920) 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 -- 172.21.15.87:0/3830484730 learned_addr learned my addr 172.21.15.87:0/3830484730 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc422000640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c0b3360 0x7fc42c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 -- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc42c0cf9b0 msgr2=0x7fc42c146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42b400640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc42c0cf9b0 0x7fc42c146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc42c0cf9b0 0x7fc42c146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 -- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c0b3360 msgr2=0x7fc42c0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c0b3360 0x7fc42c0afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 -- 172.21.15.87:0/3830484730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c155d20 con 0x7fc42c071120 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc422000640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c0b3360 0x7fc42c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42b400640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc42c0cf9b0 0x7fc42c146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42aa00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c071120 0x7fc42c1495b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c420 tx=0x7fc41c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc423400640 1 -- 172.21.15.87:0/3830484730 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c017070 con 0x7fc42c071120 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc423400640 1 -- 172.21.15.87:0/3830484730 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c005cf0 con 0x7fc42c071120 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc423400640 1 -- 172.21.15.87:0/3830484730 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0072d0 con 0x7fc42c071120 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c156000 con 0x7fc42c071120 2024-04-25T10:21:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c1565c0 con 0x7fc42c071120 2024-04-25T10:21:30.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.826+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c075120 con 0x7fc42c071120 2024-04-25T10:21:30.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.830+0000 7fc423400640 1 -- 172.21.15.87:0/3830484730 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc41c01a070 con 0x7fc42c071120 2024-04-25T10:21:30.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.830+0000 7fc423400640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc408066f90 0x7fc408069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.830+0000 7fc423400640 1 -- 172.21.15.87:0/3830484730 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc41c084420 con 0x7fc42c071120 2024-04-25T10:21:30.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.830+0000 7fc422000640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc408066f90 0x7fc408069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.830+0000 7fc422000640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc408066f90 0x7fc408069450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0af390 tx=0x7fc414009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:30.834+0000 7fc423400640 1 -- 172.21.15.87:0/3830484730 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc41c054d00 con 0x7fc42c071120 2024-04-25T10:21:31.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.018+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc42c0740e0 con 0x7fc42c071120 2024-04-25T10:21:31.022 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:21:31.018+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:21:31.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc423400640 1 -- 172.21.15.87:0/3830484730 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc41c058da0 con 0x7fc42c071120 2024-04-25T10:21:31.024 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:21:31.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc408066f90 msgr2=0x7fc408069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc408066f90 0x7fc408069450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0af390 tx=0x7fc414009040 comp rx=0 tx=0).stop 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c071120 msgr2=0x7fc42c1495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c071120 0x7fc42c1495b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c420 tx=0x7fc41c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 shutdown_connections 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc408066f90 0x7fc408069450 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc42c0cf9b0 0x7fc42c146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c0b3360 0x7fc42c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 --2- 172.21.15.87:0/3830484730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc42c071120 0x7fc42c1495b0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 >> 172.21.15.87:0/3830484730 conn(0x7fc42c0cfe80 msgr2=0x7fc42c14c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 shutdown_connections 2024-04-25T10:21:31.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.022+0000 7fc42be00640 1 -- 172.21.15.87:0/3830484730 wait complete. 2024-04-25T10:21:31.048 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:21:31.095 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:21:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1187c00640 1 Processor -- start 2024-04-25T10:21:31.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1187c00640 1 -- start start 2024-04-25T10:21:31.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1187c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f118015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1187c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f11800b31c0 0x7f11800b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1187c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118015b080 con 0x7f1180070350 2024-04-25T10:21:31.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1187c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118015adb0 con 0x7f11800b31c0 2024-04-25T10:21:31.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1187c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118015eb60 con 0x7f11800aef80 2024-04-25T10:21:31.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1185c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f11800b31c0 0x7f11800b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f117fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1180070350 0x7f118015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45734/0 (socket says 172.21.15.87:45734) 2024-04-25T10:21:31.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f117fe00640 1 -- 172.21.15.87:0/509543420 learned_addr learned my addr 172.21.15.87:0/509543420 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:31.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1185200640 1 --2- 172.21.15.87:0/509543420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f118015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1185c00640 1 -- 172.21.15.87:0/509543420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 msgr2=0x7f118015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:31.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1185c00640 1 --2- 172.21.15.87:0/509543420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f118015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1185c00640 1 -- 172.21.15.87:0/509543420 >> v1:172.21.15.87:6789/0 conn(0x7f1180070350 legacy=0x7f118015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f1185c00640 1 -- 172.21.15.87:0/509543420 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118015ee40 con 0x7f11800b31c0 2024-04-25T10:21:31.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.186+0000 7f117f400640 1 -- 172.21.15.87:0/509543420 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637316753 0 0) 0x7f118015b080 con 0x7f1180070350 2024-04-25T10:21:31.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1185c00640 1 --2- 172.21.15.87:0/509543420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11800b31c0 0x7f11800b3610 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f1178009d90 tx=0x7f117801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfc24e6b1c646c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f117f400640 1 -- 172.21.15.87:0/509543420 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11780b2070 con 0x7f11800b31c0 2024-04-25T10:21:31.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f117f400640 1 -- 172.21.15.87:0/509543420 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11780a5940 con 0x7f11800b31c0 2024-04-25T10:21:31.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f117f400640 1 -- 172.21.15.87:0/509543420 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11780a5d10 con 0x7f11800b31c0 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 -- 172.21.15.87:0/509543420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11800b31c0 msgr2=0x7f11800b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 --2- 172.21.15.87:0/509543420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11800b31c0 0x7f11800b3610 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f1178009d90 tx=0x7f117801c970 comp rx=0 tx=0).stop 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 -- 172.21.15.87:0/509543420 shutdown_connections 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 --2- 172.21.15.87:0/509543420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11800b31c0 0x7f11800b3610 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 --2- 172.21.15.87:0/509543420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f118015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 -- 172.21.15.87:0/509543420 >> 172.21.15.87:0/509543420 conn(0x7f11800669a0 msgr2=0x7f1180066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 -- 172.21.15.87:0/509543420 shutdown_connections 2024-04-25T10:21:31.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 -- 172.21.15.87:0/509543420 wait complete. 2024-04-25T10:21:31.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 Processor -- start 2024-04-25T10:21:31.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.190+0000 7f1187c00640 1 -- start start 2024-04-25T10:21:31.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1180070350 0x7f1180140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f1180151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11800b31c0 0x7f1180152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180165990 con 0x7f1180070350 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180165810 con 0x7f11800b31c0 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1180165b10 con 0x7f11800aef80 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1180070350 0x7f1180140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1180070350 0x7f1180140190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43948/0 (socket says 172.21.15.87:43948) 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 -- 172.21.15.87:0/2768051816 learned_addr learned my addr 172.21.15.87:0/2768051816 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f117fe00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f1180151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185c00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11800b31c0 0x7f1180152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 -- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 msgr2=0x7f1180151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f1180151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 -- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11800b31c0 msgr2=0x7f1180152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11800b31c0 0x7f1180152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 -- 172.21.15.87:0/2768051816 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11801521d0 con 0x7f1180070350 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1185200640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1180070350 0x7f1180140190 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f117400e960 tx=0x7f117400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f117cc00640 1 -- 172.21.15.87:0/2768051816 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174017070 con 0x7f1180070350 2024-04-25T10:21:31.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f117cc00640 1 -- 172.21.15.87:0/2768051816 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400ca00 con 0x7f1180070350 2024-04-25T10:21:31.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1180159960 con 0x7f1180070350 2024-04-25T10:21:31.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f117cc00640 1 -- 172.21.15.87:0/2768051816 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400cdb0 con 0x7f1180070350 2024-04-25T10:21:31.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1180159f20 con 0x7f1180070350 2024-04-25T10:21:31.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118004ef30 con 0x7f1180070350 2024-04-25T10:21:31.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.194+0000 7f117cc00640 1 -- 172.21.15.87:0/2768051816 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f117401a070 con 0x7f1180070350 2024-04-25T10:21:31.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.198+0000 7f117cc00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1160066f60 0x7f1160069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.198+0000 7f117cc00640 1 -- 172.21.15.87:0/2768051816 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f11740854e0 con 0x7f1180070350 2024-04-25T10:21:31.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.198+0000 7f117fe00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1160066f60 0x7f1160069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.198+0000 7f117fe00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1160066f60 0x7f1160069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1180068240 tx=0x7f116c007b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.202+0000 7f117cc00640 1 -- 172.21.15.87:0/2768051816 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1174056450 con 0x7f1180070350 2024-04-25T10:21:31.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:31.394+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f11801564e0 con 0x7f1180070350 2024-04-25T10:21:32.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f117cc00640 1 -- 172.21.15.87:0/2768051816 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1174055d50 con 0x7f1180070350 2024-04-25T10:21:32.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1160066f60 msgr2=0x7f1160069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1160066f60 0x7f1160069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1180068240 tx=0x7f116c007b60 comp rx=0 tx=0).stop 2024-04-25T10:21:32.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1180070350 msgr2=0x7f1180140190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1180070350 0x7f1180140190 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f117400e960 tx=0x7f117400ee20 comp rx=0 tx=0).stop 2024-04-25T10:21:32.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 shutdown_connections 2024-04-25T10:21:32.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1160066f60 0x7f1160069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11800b31c0 0x7f1180152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11800aef80 0x7f1180151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 --2- 172.21.15.87:0/2768051816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1180070350 0x7f1180140190 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 >> 172.21.15.87:0/2768051816 conn(0x7f11800669a0 msgr2=0x7f118006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:32.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 shutdown_connections 2024-04-25T10:21:32.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.102+0000 7f1187c00640 1 -- 172.21.15.87:0/2768051816 wait complete. 2024-04-25T10:21:32.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4921e00640 1 Processor -- start 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4921e00640 1 -- start start 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4921e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15bd10 0x7f491c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4921e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491c06c0e0 con 0x7f491c0cf9e0 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4921e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491c15c6a0 con 0x7f491c15c0f0 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4921e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0b2ef0 con 0x7f491c15bd10 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f491b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f491c0cf9e0 0x7f491c06c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45738/0 (socket says 172.21.15.87:45738) 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f491b400640 1 -- 172.21.15.87:0/4119289905 learned_addr learned my addr 172.21.15.87:0/4119289905 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f491aa00640 1 --2- 172.21.15.87:0/4119289905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15bd10 0x7f491c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4919600640 1 -- 172.21.15.87:0/4119289905 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885147754 0 0) 0x7f491c06c0e0 con 0x7f491c0cf9e0 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f4919600640 1 -- 172.21.15.87:0/4119289905 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4900003530 con 0x7f491c0cf9e0 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f491aa00640 1 -- 172.21.15.87:0/4119289905 >> v1:172.21.15.169:6789/0 conn(0x7f491c15c0f0 legacy=0x7f491c0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f491aa00640 1 -- 172.21.15.87:0/4119289905 >> v1:172.21.15.87:6789/0 conn(0x7f491c0cf9e0 legacy=0x7f491c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.310+0000 7f491aa00640 1 -- 172.21.15.87:0/4119289905 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c15c980 con 0x7f491c15bd10 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f491aa00640 1 --2- 172.21.15.87:0/4119289905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15bd10 0x7f491c0b4610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f490c009dc0 tx=0x7f490c01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f366fb330e53818c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4919600640 1 -- 172.21.15.87:0/4119289905 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0b2070 con 0x7f491c15bd10 2024-04-25T10:21:32.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4919600640 1 -- 172.21.15.87:0/4119289905 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c0a5a40 con 0x7f491c15bd10 2024-04-25T10:21:32.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4919600640 1 -- 172.21.15.87:0/4119289905 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0a5e10 con 0x7f491c15bd10 2024-04-25T10:21:32.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4921e00640 1 -- 172.21.15.87:0/4119289905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15bd10 msgr2=0x7f491c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4921e00640 1 --2- 172.21.15.87:0/4119289905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15bd10 0x7f491c0b4610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f490c009dc0 tx=0x7f490c01c960 comp rx=0 tx=0).stop 2024-04-25T10:21:32.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4921e00640 1 -- 172.21.15.87:0/4119289905 shutdown_connections 2024-04-25T10:21:32.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4921e00640 1 --2- 172.21.15.87:0/4119289905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15bd10 0x7f491c0b4610 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4921e00640 1 -- 172.21.15.87:0/4119289905 >> 172.21.15.87:0/4119289905 conn(0x7f491c066b70 msgr2=0x7f491c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:32.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4921e00640 1 -- 172.21.15.87:0/4119289905 shutdown_connections 2024-04-25T10:21:32.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.314+0000 7f4921e00640 1 -- 172.21.15.87:0/4119289905 wait complete. 2024-04-25T10:21:32.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 Processor -- start 2024-04-25T10:21:32.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 -- start start 2024-04-25T10:21:32.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f491c0cf9e0 0x7f491c14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491c15bd10 0x7f491c0afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15c0f0 0x7f491c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0d0200 con 0x7f491c15bd10 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c16a2e0 con 0x7f491c0cf9e0 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c14fb90 con 0x7f491c15c0f0 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491c15bd10 0x7f491c0afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491c15bd10 0x7f491c0afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43962/0 (socket says 172.21.15.87:43962) 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 -- 172.21.15.87:0/3372868476 learned_addr learned my addr 172.21.15.87:0/3372868476 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491aa00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f491c0cf9e0 0x7f491c14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491b400640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15c0f0 0x7f491c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 -- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15c0f0 msgr2=0x7f491c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15c0f0 0x7f491c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 -- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f491c0cf9e0 msgr2=0x7f491c14f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f491c0cf9e0 0x7f491c14f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 -- 172.21.15.87:0/3372868476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c154b70 con 0x7f491c15bd10 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f491a000640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491c15bd10 0x7f491c0afde0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f490800c370 tx=0x7f490800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4913400640 1 -- 172.21.15.87:0/3372868476 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080052d0 con 0x7f491c15bd10 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4913400640 1 -- 172.21.15.87:0/3372868476 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908005470 con 0x7f491c15bd10 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4913400640 1 -- 172.21.15.87:0/3372868476 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908007260 con 0x7f491c15bd10 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c154e50 con 0x7f491c15bd10 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.318+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c155410 con 0x7f491c15bd10 2024-04-25T10:21:32.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.322+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c04ef30 con 0x7f491c15bd10 2024-04-25T10:21:32.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.322+0000 7f4913400640 1 -- 172.21.15.87:0/3372868476 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4908007400 con 0x7f491c15bd10 2024-04-25T10:21:32.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.322+0000 7f4913400640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48fc066f60 0x7f48fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.322+0000 7f4913400640 1 -- 172.21.15.87:0/3372868476 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4908084540 con 0x7f491c15bd10 2024-04-25T10:21:32.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.322+0000 7f491aa00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48fc066f60 0x7f48fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.322+0000 7f491aa00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48fc066f60 0x7f48fc069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f490c0af000 tx=0x7f490c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.326+0000 7f4913400640 1 -- 172.21.15.87:0/3372868476 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4908054dd0 con 0x7f491c15bd10 2024-04-25T10:21:32.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:32.482+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f491c1555e0 con 0x7f491c15bd10 2024-04-25T10:21:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.102+0000 7f4913400640 1 -- 172.21.15.87:0/3372868476 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4908058e70 con 0x7f491c15bd10 2024-04-25T10:21:33.107 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:21:33.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48fc066f60 msgr2=0x7f48fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48fc066f60 0x7f48fc069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f490c0af000 tx=0x7f490c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:21:33.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491c15bd10 msgr2=0x7f491c0afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491c15bd10 0x7f491c0afde0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f490800c370 tx=0x7f490800c830 comp rx=0 tx=0).stop 2024-04-25T10:21:33.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 shutdown_connections 2024-04-25T10:21:33.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48fc066f60 0x7f48fc069420 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491c15c0f0 0x7f491c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491c15bd10 0x7f491c0afde0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 --2- 172.21.15.87:0/3372868476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f491c0cf9e0 0x7f491c14f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 >> 172.21.15.87:0/3372868476 conn(0x7f491c066b70 msgr2=0x7f491c0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:33.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 shutdown_connections 2024-04-25T10:21:33.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.106+0000 7f4921e00640 1 -- 172.21.15.87:0/3372868476 wait complete. 2024-04-25T10:21:33.133 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:21:33.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.270+0000 7f84cb800640 1 Processor -- start 2024-04-25T10:21:33.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.270+0000 7f84cb800640 1 -- start start 2024-04-25T10:21:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84cb800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 0x7f84c40cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84cb800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c4071f80 con 0x7f84c415bc00 2024-04-25T10:21:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84cb800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c415b120 con 0x7f84c4071ab0 2024-04-25T10:21:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84cb800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4072260 con 0x7f84c40cf040 2024-04-25T10:21:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84bbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f84c415bc00 0x7f84c40713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45740/0 (socket says 172.21.15.87:45740) 2024-04-25T10:21:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84bbe00640 1 -- 172.21.15.87:0/4003346239 learned_addr learned my addr 172.21.15.87:0/4003346239 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c8e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 0x7f84c40cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147089197 0 0) 0x7f84c4071f80 con 0x7f84c415bc00 2024-04-25T10:21:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84a8003530 con 0x7f84c415bc00 2024-04-25T10:21:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701929895 0 0) 0x7f84a8003530 con 0x7f84c415bc00 2024-04-25T10:21:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 msgr2=0x7f84c40cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 0x7f84c40cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 >> v1:172.21.15.169:6789/0 conn(0x7f84c4071ab0 legacy=0x7f84c40ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c415b400 con 0x7f84c415bc00 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f84ac001680 con 0x7f84c415bc00 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 >> v1:172.21.15.87:6789/0 conn(0x7f84c415bc00 legacy=0x7f84c40713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a80039f0 0x7f84a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 0x7f84a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a8004d50 0x7f84a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800ba80 con 0x7f84a80039f0 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800bc30 con 0x7f84a8004d50 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800be30 con 0x7f84c40cf040 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c8e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a80039f0 0x7f84a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84bbe00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 0x7f84a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c9800640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a8004d50 0x7f84a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c8e00640 1 -- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 msgr2=0x7f84a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c8e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40cf040 0x7f84a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c8e00640 1 -- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a8004d50 msgr2=0x7f84a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:33.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c8e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a8004d50 0x7f84a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.274+0000 7f84c8e00640 1 -- 172.21.15.87:0/4003346239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8003530 con 0x7f84a80039f0 2024-04-25T10:21:33.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84c8e00640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a80039f0 0x7f84a8003e10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009ab0 tx=0x7f84b401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f30bf37b5f2b8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:33.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b40096b0 con 0x7f84a80039f0 2024-04-25T10:21:33.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b40a6430 con 0x7f84a80039f0 2024-04-25T10:21:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84c3e00640 1 -- 172.21.15.87:0/4003346239 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b40a6820 con 0x7f84a80039f0 2024-04-25T10:21:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 -- 172.21.15.87:0/4003346239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a80039f0 msgr2=0x7f84a8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 --2- 172.21.15.87:0/4003346239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84a80039f0 0x7f84a8003e10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009ab0 tx=0x7f84b401ca50 comp rx=0 tx=0).stop 2024-04-25T10:21:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84c9800640 1 -- 172.21.15.87:0/4003346239 reap_dead start 2024-04-25T10:21:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 -- 172.21.15.87:0/4003346239 shutdown_connections 2024-04-25T10:21:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 -- 172.21.15.87:0/4003346239 >> 172.21.15.87:0/4003346239 conn(0x7f84c4066b70 msgr2=0x7f84c4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 -- 172.21.15.87:0/4003346239 shutdown_connections 2024-04-25T10:21:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 -- 172.21.15.87:0/4003346239 wait complete. 2024-04-25T10:21:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 Processor -- start 2024-04-25T10:21:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.278+0000 7f84cb800640 1 -- start start 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a80039f0 0x7f84c4158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84c41513e0 0x7f84c415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40ad840 0x7f84c40adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40cfbf0 con 0x7f84c41513e0 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40cf630 con 0x7f84a80039f0 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40cfd70 con 0x7f84c40ad840 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84c41513e0 0x7f84c415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84c41513e0 0x7f84c415a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43972/0 (socket says 172.21.15.87:43972) 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 -- 172.21.15.87:0/427956953 learned_addr learned my addr 172.21.15.87:0/427956953 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84c8e00640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a80039f0 0x7f84c4158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84c9800640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40ad840 0x7f84c40adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 -- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40ad840 msgr2=0x7f84c40adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40ad840 0x7f84c40adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 -- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a80039f0 msgr2=0x7f84c4158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a80039f0 0x7f84c4158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 -- 172.21.15.87:0/427956953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4159140 con 0x7f84c41513e0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84bbe00640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84c41513e0 0x7f84c415a620 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f84ac004f80 tx=0x7f84ac0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84c1600640 1 -- 172.21.15.87:0/427956953 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac007740 con 0x7f84c41513e0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84c1600640 1 -- 172.21.15.87:0/427956953 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac018070 con 0x7f84c41513e0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c4159420 con 0x7f84c41513e0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84c1600640 1 -- 172.21.15.87:0/427956953 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00fbe0 con 0x7f84c41513e0 2024-04-25T10:21:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c41599e0 con 0x7f84c41513e0 2024-04-25T10:21:33.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c404ef30 con 0x7f84c41513e0 2024-04-25T10:21:33.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.282+0000 7f84c1600640 1 -- 172.21.15.87:0/427956953 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84ac01d460 con 0x7f84c41513e0 2024-04-25T10:21:33.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.286+0000 7f84c1600640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84a0066f60 0x7f84a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.286+0000 7f84c1600640 1 -- 172.21.15.87:0/427956953 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f84ac084f10 con 0x7f84c41513e0 2024-04-25T10:21:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.286+0000 7f84c8e00640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84a0066f60 0x7f84a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.286+0000 7f84c8e00640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84a0066f60 0x7f84a0069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f84b40ad000 tx=0x7f84b40b2040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:33.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.290+0000 7f84c1600640 1 -- 172.21.15.87:0/427956953 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84ac056030 con 0x7f84c41513e0 2024-04-25T10:21:33.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:33.446+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84c40b2a10 con 0x7f84c41513e0 2024-04-25T10:21:34.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84c1600640 1 -- 172.21.15.87:0/427956953 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7f84ac055930 con 0x7f84c41513e0 2024-04-25T10:21:34.109 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:21:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84a0066f60 msgr2=0x7f84a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84a0066f60 0x7f84a0069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f84b40ad000 tx=0x7f84b40b2040 comp rx=0 tx=0).stop 2024-04-25T10:21:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84c41513e0 msgr2=0x7f84c415a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84c41513e0 0x7f84c415a620 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f84ac004f80 tx=0x7f84ac0074f0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 shutdown_connections 2024-04-25T10:21:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f84a0066f60 0x7f84a0069420 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c40ad840 0x7f84c40adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84c41513e0 0x7f84c415a620 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 --2- 172.21.15.87:0/427956953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a80039f0 0x7f84c4158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 >> 172.21.15.87:0/427956953 conn(0x7f84c4066b70 msgr2=0x7f84c415e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.106+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 shutdown_connections 2024-04-25T10:21:34.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.110+0000 7f84cb800640 1 -- 172.21.15.87:0/427956953 wait complete. 2024-04-25T10:21:34.146 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:21:34.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 Processor -- start 2024-04-25T10:21:34.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 -- start start 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7be40aef80 0x7f7be415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7be4070350 0x7f7be415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be415bac0 0x7f7be40b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be415b080 con 0x7f7be4070350 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be415adb0 con 0x7f7be40aef80 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be415eb60 con 0x7f7be415bac0 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7be4070350 0x7f7be415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7be4070350 0x7f7be415b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43978/0 (socket says 172.21.15.87:43978) 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 -- 172.21.15.87:0/3319209306 learned_addr learned my addr 172.21.15.87:0/3319209306 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:34.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be2a00640 1 --2- 172.21.15.87:0/3319209306 >> v2:172.21.15.169:3300/0 conn(0x7f7be40aef80 0x7f7be415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7be3400640 1 --2- 172.21.15.87:0/3319209306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be415bac0 0x7f7be40b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 -- 172.21.15.87:0/3319209306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be415bac0 msgr2=0x7f7be40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:34.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 --2- 172.21.15.87:0/3319209306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be415bac0 0x7f7be40b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 -- 172.21.15.87:0/3319209306 >> v2:172.21.15.169:3300/0 conn(0x7f7be40aef80 msgr2=0x7f7be415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:34.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 --2- 172.21.15.87:0/3319209306 >> v2:172.21.15.169:3300/0 conn(0x7f7be40aef80 0x7f7be415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.282+0000 7f7bdbe00640 1 -- 172.21.15.87:0/3319209306 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be415ee40 con 0x7f7be4070350 2024-04-25T10:21:34.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7bdbe00640 1 --2- 172.21.15.87:0/3319209306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be4070350 0x7f7be415b580 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7be406e220 tx=0x7f7bcc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1814827b788af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be2000640 1 -- 172.21.15.87:0/3319209306 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0ab5f0 con 0x7f7be4070350 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be2000640 1 -- 172.21.15.87:0/3319209306 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0b0070 con 0x7f7be4070350 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be2000640 1 -- 172.21.15.87:0/3319209306 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0ab9b0 con 0x7f7be4070350 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 -- 172.21.15.87:0/3319209306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be4070350 msgr2=0x7f7be415b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 --2- 172.21.15.87:0/3319209306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be4070350 0x7f7be415b580 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7be406e220 tx=0x7f7bcc01c520 comp rx=0 tx=0).stop 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 -- 172.21.15.87:0/3319209306 shutdown_connections 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 --2- 172.21.15.87:0/3319209306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be4070350 0x7f7be415b580 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 --2- 172.21.15.87:0/3319209306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be415bac0 0x7f7be40b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 --2- 172.21.15.87:0/3319209306 >> v2:172.21.15.169:3300/0 conn(0x7f7be40aef80 0x7f7be415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 -- 172.21.15.87:0/3319209306 >> 172.21.15.87:0/3319209306 conn(0x7f7be40669a0 msgr2=0x7f7be4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 -- 172.21.15.87:0/3319209306 shutdown_connections 2024-04-25T10:21:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.286+0000 7f7be9800640 1 -- 172.21.15.87:0/3319209306 wait complete. 2024-04-25T10:21:34.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 Processor -- start 2024-04-25T10:21:34.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 -- start start 2024-04-25T10:21:34.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be4070350 0x7f7be41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be40aef80 0x7f7be414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be415bac0 0x7f7be4151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be406d570 con 0x7f7be40aef80 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be406d3f0 con 0x7f7be415bac0 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be406d6f0 con 0x7f7be4070350 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be40aef80 0x7f7be414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be40aef80 0x7f7be414b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43986/0 (socket says 172.21.15.87:43986) 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be4070350 0x7f7be41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 -- 172.21.15.87:0/2298798237 learned_addr learned my addr 172.21.15.87:0/2298798237 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be3400640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be415bac0 0x7f7be4151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 -- 172.21.15.87:0/2298798237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be4070350 msgr2=0x7f7be41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be4070350 0x7f7be41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 -- 172.21.15.87:0/2298798237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be415bac0 msgr2=0x7f7be4151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be415bac0 0x7f7be4151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 -- 172.21.15.87:0/2298798237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be4156ef0 con 0x7f7be40aef80 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdbe00640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be40aef80 0x7f7be414b6b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc01abd0 tx=0x7f7bcc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdaa00640 1 -- 172.21.15.87:0/2298798237 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0be5f0 con 0x7f7be40aef80 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdaa00640 1 -- 172.21.15.87:0/2298798237 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0be790 con 0x7f7be40aef80 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7bdaa00640 1 -- 172.21.15.87:0/2298798237 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0beb80 con 0x7f7be40aef80 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be4157160 con 0x7f7be40aef80 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.290+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be41501c0 con 0x7f7be40aef80 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.294+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be404ef30 con 0x7f7be40aef80 2024-04-25T10:21:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.294+0000 7f7bdaa00640 1 -- 172.21.15.87:0/2298798237 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7bcc0b43f0 con 0x7f7be40aef80 2024-04-25T10:21:34.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.294+0000 7f7bdaa00640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7bc0066f60 0x7f7bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.294+0000 7f7bdaa00640 1 -- 172.21.15.87:0/2298798237 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7bcc11f3a0 con 0x7f7be40aef80 2024-04-25T10:21:34.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.294+0000 7f7be2a00640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7bc0066f60 0x7f7bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.294+0000 7f7be2a00640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7bc0066f60 0x7f7bc0069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7be4157e30 tx=0x7f7bd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.298+0000 7f7bdaa00640 1 -- 172.21.15.87:0/2298798237 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7bcc016110 con 0x7f7be40aef80 2024-04-25T10:21:34.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.450+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7be4158090 con 0x7f7be40aef80 2024-04-25T10:21:34.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.454+0000 7f7bdaa00640 1 -- 172.21.15.87:0/2298798237 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7bcc0f0170 con 0x7f7be40aef80 2024-04-25T10:21:34.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7bc0066f60 msgr2=0x7f7bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7bc0066f60 0x7f7bc0069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7be4157e30 tx=0x7f7bd4008040 comp rx=0 tx=0).stop 2024-04-25T10:21:34.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be40aef80 msgr2=0x7f7be414b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be40aef80 0x7f7be414b6b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc01abd0 tx=0x7f7bcc01ad60 comp rx=0 tx=0).stop 2024-04-25T10:21:34.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 shutdown_connections 2024-04-25T10:21:34.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7bc0066f60 0x7f7bc0069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be415bac0 0x7f7be4151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7be40aef80 0x7f7be414b6b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 --2- 172.21.15.87:0/2298798237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be4070350 0x7f7be41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 >> 172.21.15.87:0/2298798237 conn(0x7f7be40669a0 msgr2=0x7f7be406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 shutdown_connections 2024-04-25T10:21:34.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.458+0000 7f7be9800640 1 -- 172.21.15.87:0/2298798237 wait complete. 2024-04-25T10:21:34.480 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:21:34.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.610+0000 7f175f200640 1 Processor -- start 2024-04-25T10:21:34.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175f200640 1 -- start start 2024-04-25T10:21:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f175815bc40 0x7f175806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17580cf930 0x7f17580cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580b2770 con 0x7f17580cda10 2024-04-25T10:21:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175806c670 con 0x7f17580cf930 2024-04-25T10:21:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175806c950 con 0x7f175815bc40 2024-04-25T10:21:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17580cf930 0x7f17580cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17580cf930 0x7f17580cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43988/0 (socket says 172.21.15.87:43988) 2024-04-25T10:21:34.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175d200640 1 -- 172.21.15.87:0/3860253764 learned_addr learned my addr 172.21.15.87:0/3860253764 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:34.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f1757e00640 1 --2- 172.21.15.87:0/3860253764 >> v2:172.21.15.169:3300/0 conn(0x7f175815bc40 0x7f175806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175d200640 1 -- 172.21.15.87:0/3860253764 >> v2:172.21.15.169:3300/0 conn(0x7f175815bc40 msgr2=0x7f175806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:34.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175d200640 1 --2- 172.21.15.87:0/3860253764 >> v2:172.21.15.169:3300/0 conn(0x7f175815bc40 0x7f175806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175d200640 1 -- 172.21.15.87:0/3860253764 >> v1:172.21.15.169:6789/0 conn(0x7f17580cda10 legacy=0x7f17580b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:34.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.614+0000 7f175d200640 1 -- 172.21.15.87:0/3860253764 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175806cc30 con 0x7f17580cf930 2024-04-25T10:21:34.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175d200640 1 --2- 172.21.15.87:0/3860253764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cf930 0x7f17580cfdc0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f17580d0800 tx=0x7f175001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a64c7eef372dde0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f1757400640 1 -- 172.21.15.87:0/3860253764 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500a55e0 con 0x7f17580cf930 2024-04-25T10:21:34.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f1757400640 1 -- 172.21.15.87:0/3860253764 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17500a5780 con 0x7f17580cf930 2024-04-25T10:21:34.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f1757400640 1 -- 172.21.15.87:0/3860253764 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500a5b50 con 0x7f17580cf930 2024-04-25T10:21:34.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 -- 172.21.15.87:0/3860253764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cf930 msgr2=0x7f17580cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 --2- 172.21.15.87:0/3860253764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cf930 0x7f17580cfdc0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f17580d0800 tx=0x7f175001c920 comp rx=0 tx=0).stop 2024-04-25T10:21:34.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 -- 172.21.15.87:0/3860253764 shutdown_connections 2024-04-25T10:21:34.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 --2- 172.21.15.87:0/3860253764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cf930 0x7f17580cfdc0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 --2- 172.21.15.87:0/3860253764 >> v2:172.21.15.169:3300/0 conn(0x7f175815bc40 0x7f175806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 -- 172.21.15.87:0/3860253764 >> 172.21.15.87:0/3860253764 conn(0x7f1758066b70 msgr2=0x7f1758066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 -- 172.21.15.87:0/3860253764 shutdown_connections 2024-04-25T10:21:34.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 -- 172.21.15.87:0/3860253764 wait complete. 2024-04-25T10:21:34.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 Processor -- start 2024-04-25T10:21:34.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.618+0000 7f175f200640 1 -- start start 2024-04-25T10:21:34.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cda10 0x7f175814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580cf930 0x7f1758156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f175815bc40 0x7f1758157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580b3850 con 0x7f17580cda10 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580b36d0 con 0x7f17580cf930 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580b39d0 con 0x7f175815bc40 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cda10 0x7f175814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cda10 0x7f175814bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44004/0 (socket says 172.21.15.87:44004) 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 -- 172.21.15.87:0/3271337543 learned_addr learned my addr 172.21.15.87:0/3271337543 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f174f400640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580cf930 0x7f1758156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175d200640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f175815bc40 0x7f1758157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 -- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f175815bc40 msgr2=0x7f1758157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f175815bc40 0x7f1758157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 -- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580cf930 msgr2=0x7f1758156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580cf930 0x7f1758156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 -- 172.21.15.87:0/3271337543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175815ed40 con 0x7f17580cda10 2024-04-25T10:21:34.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1757e00640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cda10 0x7f175814bb60 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f174800d5f0 tx=0x7f174800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1754c00640 1 -- 172.21.15.87:0/3271337543 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480125f0 con 0x7f17580cda10 2024-04-25T10:21:34.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1754c00640 1 -- 172.21.15.87:0/3271337543 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1748019070 con 0x7f17580cda10 2024-04-25T10:21:34.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175815f020 con 0x7f17580cda10 2024-04-25T10:21:34.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1754c00640 1 -- 172.21.15.87:0/3271337543 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480129b0 con 0x7f17580cda10 2024-04-25T10:21:34.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17580b00d0 con 0x7f17580cda10 2024-04-25T10:21:34.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175804ef30 con 0x7f17580cda10 2024-04-25T10:21:34.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.622+0000 7f1754c00640 1 -- 172.21.15.87:0/3271337543 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1748012b50 con 0x7f17580cda10 2024-04-25T10:21:34.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.626+0000 7f1754c00640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1734066f90 0x7f1734069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.626+0000 7f1754c00640 1 -- 172.21.15.87:0/3271337543 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1748084f70 con 0x7f17580cda10 2024-04-25T10:21:34.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.626+0000 7f174f400640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1734066f90 0x7f1734069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.626+0000 7f174f400640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1734066f90 0x7f1734069450 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17581559a0 tx=0x7f1740009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.630+0000 7f1754c00640 1 -- 172.21.15.87:0/3271337543 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1748055c20 con 0x7f17580cda10 2024-04-25T10:21:34.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:34.838+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f1758159f90 con 0x7f17580cda10 2024-04-25T10:21:35.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f1754c00640 1 -- 172.21.15.87:0/3271337543 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v452) v1 ==== 166+0+0 (secure 0 0 0) 0x7f1748059cc0 con 0x7f17580cda10 2024-04-25T10:21:35.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1734066f90 msgr2=0x7f1734069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1734066f90 0x7f1734069450 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17581559a0 tx=0x7f1740009060 comp rx=0 tx=0).stop 2024-04-25T10:21:35.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cda10 msgr2=0x7f175814bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cda10 0x7f175814bb60 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f174800d5f0 tx=0x7f174800dab0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 shutdown_connections 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1734066f90 0x7f1734069450 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f175815bc40 0x7f1758157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580cf930 0x7f1758156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 --2- 172.21.15.87:0/3271337543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17580cda10 0x7f175814bb60 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 >> 172.21.15.87:0/3271337543 conn(0x7f1758066b70 msgr2=0x7f17580ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 shutdown_connections 2024-04-25T10:21:35.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.118+0000 7f175f200640 1 -- 172.21.15.87:0/3271337543 wait complete. 2024-04-25T10:21:35.141 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:21:35.142 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:21:35.142 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:35.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409ec00640 1 Processor -- start 2024-04-25T10:21:35.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409ec00640 1 -- start start 2024-04-25T10:21:35.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40980cda10 0x7f409806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409ec00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40980cf930 0x7f40980cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409ec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40980b2770 con 0x7f409815bc40 2024-04-25T10:21:35.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409ec00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409806c670 con 0x7f40980cf930 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409ec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409806c950 con 0x7f40980cda10 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40980cf930 0x7f40980cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40980cf930 0x7f40980cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44016/0 (socket says 172.21.15.87:44016) 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409cc00640 1 -- 172.21.15.87:0/3607996773 learned_addr learned my addr 172.21.15.87:0/3607996773 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f4097400640 1 --2- 172.21.15.87:0/3607996773 >> v2:172.21.15.169:3300/0 conn(0x7f40980cda10 0x7f409806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409cc00640 1 -- 172.21.15.87:0/3607996773 >> v2:172.21.15.169:3300/0 conn(0x7f40980cda10 msgr2=0x7f409806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409cc00640 1 --2- 172.21.15.87:0/3607996773 >> v2:172.21.15.169:3300/0 conn(0x7f40980cda10 0x7f409806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409cc00640 1 -- 172.21.15.87:0/3607996773 >> v1:172.21.15.169:6789/0 conn(0x7f409815bc40 legacy=0x7f40980b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:35.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.286+0000 7f409cc00640 1 -- 172.21.15.87:0/3607996773 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409806cc30 con 0x7f40980cf930 2024-04-25T10:21:35.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409cc00640 1 --2- 172.21.15.87:0/3607996773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cf930 0x7f40980cfdc0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f40980726f0 tx=0x7f409001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7903c767ea2672a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f4096a00640 1 -- 172.21.15.87:0/3607996773 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900a55e0 con 0x7f40980cf930 2024-04-25T10:21:35.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f4096a00640 1 -- 172.21.15.87:0/3607996773 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40900a5780 con 0x7f40980cf930 2024-04-25T10:21:35.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f4096a00640 1 -- 172.21.15.87:0/3607996773 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900a5b50 con 0x7f40980cf930 2024-04-25T10:21:35.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 -- 172.21.15.87:0/3607996773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cf930 msgr2=0x7f40980cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 --2- 172.21.15.87:0/3607996773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cf930 0x7f40980cfdc0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f40980726f0 tx=0x7f409001c920 comp rx=0 tx=0).stop 2024-04-25T10:21:35.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 -- 172.21.15.87:0/3607996773 shutdown_connections 2024-04-25T10:21:35.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 --2- 172.21.15.87:0/3607996773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cf930 0x7f40980cfdc0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 --2- 172.21.15.87:0/3607996773 >> v2:172.21.15.169:3300/0 conn(0x7f40980cda10 0x7f409806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 -- 172.21.15.87:0/3607996773 >> 172.21.15.87:0/3607996773 conn(0x7f4098066b70 msgr2=0x7f4098066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 -- 172.21.15.87:0/3607996773 shutdown_connections 2024-04-25T10:21:35.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.290+0000 7f409ec00640 1 -- 172.21.15.87:0/3607996773 wait complete. 2024-04-25T10:21:35.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 Processor -- start 2024-04-25T10:21:35.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 -- start start 2024-04-25T10:21:35.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cda10 0x7f409815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40980cf930 0x7f40980b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409815bc40 0x7f409813d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40980b3850 con 0x7f40980cda10 2024-04-25T10:21:35.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40980b36d0 con 0x7f40980cf930 2024-04-25T10:21:35.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40980b39d0 con 0x7f409815bc40 2024-04-25T10:21:35.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cda10 0x7f409815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cda10 0x7f409815b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44024/0 (socket says 172.21.15.87:44024) 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 -- 172.21.15.87:0/1304968325 learned_addr learned my addr 172.21.15.87:0/1304968325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409cc00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409815bc40 0x7f409813d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097400640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40980cf930 0x7f40980b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 -- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409815bc40 msgr2=0x7f409813d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409815bc40 0x7f409813d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 -- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40980cf930 msgr2=0x7f40980b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40980cf930 0x7f40980b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 -- 172.21.15.87:0/1304968325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980b1990 con 0x7f40980cda10 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f4097e00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cda10 0x7f409815b110 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f408800b410 tx=0x7f408800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f408fe00640 1 -- 172.21.15.87:0/1304968325 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088015030 con 0x7f40980cda10 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f408fe00640 1 -- 172.21.15.87:0/1304968325 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4088004020 con 0x7f40980cda10 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409814d950 con 0x7f40980cda10 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f408fe00640 1 -- 172.21.15.87:0/1304968325 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088004450 con 0x7f40980cda10 2024-04-25T10:21:35.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.294+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409814df10 con 0x7f40980cda10 2024-04-25T10:21:35.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.298+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409814e110 con 0x7f40980cda10 2024-04-25T10:21:35.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.298+0000 7f408fe00640 1 -- 172.21.15.87:0/1304968325 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4088018070 con 0x7f40980cda10 2024-04-25T10:21:35.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.298+0000 7f408fe00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4078066f90 0x7f4078069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.298+0000 7f408fe00640 1 -- 172.21.15.87:0/1304968325 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4088084800 con 0x7f40980cda10 2024-04-25T10:21:35.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.298+0000 7f4097400640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4078066f90 0x7f4078069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.298+0000 7f4097400640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4078066f90 0x7f4078069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4098152e10 tx=0x7f4084008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.302+0000 7f408fe00640 1 -- 172.21.15.87:0/1304968325 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4088055420 con 0x7f40980cda10 2024-04-25T10:21:35.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.478+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f409804ef30 con 0x7f40980cda10 2024-04-25T10:21:35.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.478+0000 7f408fe00640 1 -- 172.21.15.87:0/1304968325 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13018 (secure 0 0 0) 0x7f40880594c0 con 0x7f40980cda10 2024-04-25T10:21:35.484 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:35.484 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":116,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:34.108165+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/3078360268":"2024-04-26T10:21:31.023790+0000","172.21.15.87:6835/3078360268":"2024-04-26T10:21:31.023790+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:35.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4078066f90 msgr2=0x7f4078069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4078066f90 0x7f4078069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4098152e10 tx=0x7f4084008040 comp rx=0 tx=0).stop 2024-04-25T10:21:35.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cda10 msgr2=0x7f409815b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cda10 0x7f409815b110 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f408800b410 tx=0x7f408800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 shutdown_connections 2024-04-25T10:21:35.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4078066f90 0x7f4078069450 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f409815bc40 0x7f409813d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40980cf930 0x7f40980b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 --2- 172.21.15.87:0/1304968325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40980cda10 0x7f409815b110 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 >> 172.21.15.87:0/1304968325 conn(0x7f4098066b70 msgr2=0x7f4098156360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 shutdown_connections 2024-04-25T10:21:35.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.482+0000 7f409ec00640 1 -- 172.21.15.87:0/1304968325 wait complete. 2024-04-25T10:21:35.505 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:35.506 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:35.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 Processor -- start 2024-04-25T10:21:35.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 -- start start 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3cc071e40 0x7fb3cc15b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3cc06e7d0 0x7fb3cc06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc15f720 con 0x7fb3cc06e7d0 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc15f9c0 con 0x7fb3cc071e40 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15fca0 con 0x7fb3cc072220 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3cc06e7d0 0x7fb3cc06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3cc06e7d0 0x7fb3cc06ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44040/0 (socket says 172.21.15.87:44040) 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 -- 172.21.15.87:0/1186319357 learned_addr learned my addr 172.21.15.87:0/1186319357 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3cbe00640 1 --2- 172.21.15.87:0/1186319357 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc071e40 0x7fb3cc15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3cb400640 1 --2- 172.21.15.87:0/1186319357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 -- 172.21.15.87:0/1186319357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 msgr2=0x7fb3cc15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 --2- 172.21.15.87:0/1186319357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 -- 172.21.15.87:0/1186319357 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc071e40 msgr2=0x7fb3cc15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 --2- 172.21.15.87:0/1186319357 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc071e40 0x7fb3cc15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.642+0000 7fb3d1000640 1 -- 172.21.15.87:0/1186319357 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc15ff80 con 0x7fb3cc06e7d0 2024-04-25T10:21:35.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d1000640 1 --2- 172.21.15.87:0/1186319357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc06e7d0 0x7fb3cc06ec60 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0714c0 tx=0x7fb3c401d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbbf27d656b02fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3caa00640 1 -- 172.21.15.87:0/1186319357 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c40a5890 con 0x7fb3cc06e7d0 2024-04-25T10:21:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3caa00640 1 -- 172.21.15.87:0/1186319357 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c40a5a30 con 0x7fb3cc06e7d0 2024-04-25T10:21:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3caa00640 1 -- 172.21.15.87:0/1186319357 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c40a5e00 con 0x7fb3cc06e7d0 2024-04-25T10:21:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d3000640 1 -- 172.21.15.87:0/1186319357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc06e7d0 msgr2=0x7fb3cc06ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d3000640 1 --2- 172.21.15.87:0/1186319357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc06e7d0 0x7fb3cc06ec60 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0714c0 tx=0x7fb3c401d450 comp rx=0 tx=0).stop 2024-04-25T10:21:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d3000640 1 -- 172.21.15.87:0/1186319357 shutdown_connections 2024-04-25T10:21:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d3000640 1 --2- 172.21.15.87:0/1186319357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc06e7d0 0x7fb3cc06ec60 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d3000640 1 --2- 172.21.15.87:0/1186319357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d3000640 1 --2- 172.21.15.87:0/1186319357 >> v2:172.21.15.169:3300/0 conn(0x7fb3cc071e40 0x7fb3cc15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.646+0000 7fb3d3000640 1 -- 172.21.15.87:0/1186319357 >> 172.21.15.87:0/1186319357 conn(0x7fb3cc06ac90 msgr2=0x7fb3cc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 -- 172.21.15.87:0/1186319357 shutdown_connections 2024-04-25T10:21:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 -- 172.21.15.87:0/1186319357 wait complete. 2024-04-25T10:21:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 Processor -- start 2024-04-25T10:21:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 -- start start 2024-04-25T10:21:35.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc06e7d0 0x7fb3cc157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc071e40 0x7fb3cc151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc06df60 con 0x7fb3cc071e40 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc06dde0 con 0x7fb3cc06e7d0 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc06e0e0 con 0x7fb3cc072220 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc071e40 0x7fb3cc151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc06e7d0 0x7fb3cc157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc071e40 0x7fb3cc151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44048/0 (socket says 172.21.15.87:44048) 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 -- 172.21.15.87:0/3517814048 learned_addr learned my addr 172.21.15.87:0/3517814048 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3d1000640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 -- 172.21.15.87:0/3517814048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 msgr2=0x7fb3cc165dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc165dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 -- 172.21.15.87:0/3517814048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc06e7d0 msgr2=0x7fb3cc157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc06e7d0 0x7fb3cc157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 -- 172.21.15.87:0/3517814048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0cebb0 con 0x7fb3cc071e40 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.650+0000 7fb3cb400640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc071e40 0x7fb3cc151c60 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800d800 tx=0x7fb3b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3c3e00640 1 -- 172.21.15.87:0/3517814048 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800b6c0 con 0x7fb3cc071e40 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3c3e00640 1 -- 172.21.15.87:0/3517814048 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b8019070 con 0x7fb3cc071e40 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3c3e00640 1 -- 172.21.15.87:0/3517814048 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800ba60 con 0x7fb3cc071e40 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc158340 con 0x7fb3cc071e40 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc14d180 con 0x7fb3cc071e40 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc04ef30 con 0x7fb3cc071e40 2024-04-25T10:21:35.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3c3e00640 1 -- 172.21.15.87:0/3517814048 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3b800bc00 con 0x7fb3cc071e40 2024-04-25T10:21:35.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3c3e00640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f60 0x7fb3ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3c3e00640 1 -- 172.21.15.87:0/3517814048 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3b8085070 con 0x7fb3cc071e40 2024-04-25T10:21:35.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3cbe00640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f60 0x7fb3ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.654+0000 7fb3cbe00640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f60 0x7fb3ac069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc007a60 tx=0x7fb3bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.658+0000 7fb3c3e00640 1 -- 172.21.15.87:0/3517814048 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3b8055d30 con 0x7fb3cc071e40 2024-04-25T10:21:35.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3cc071790 con 0x7fb3cc071e40 2024-04-25T10:21:35.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3c3e00640 1 -- 172.21.15.87:0/3517814048 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3b8059dd0 con 0x7fb3cc071e40 2024-04-25T10:21:35.817 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f60 msgr2=0x7fb3ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f60 0x7fb3ac069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc007a60 tx=0x7fb3bc008040 comp rx=0 tx=0).stop 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc071e40 msgr2=0x7fb3cc151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc071e40 0x7fb3cc151c60 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800d800 tx=0x7fb3b800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 shutdown_connections 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb3ac066f60 0x7fb3ac069420 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3cc072220 0x7fb3cc165dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3cc071e40 0x7fb3cc151c60 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 --2- 172.21.15.87:0/3517814048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3cc06e7d0 0x7fb3cc157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 >> 172.21.15.87:0/3517814048 conn(0x7fb3cc06ac90 msgr2=0x7fb3cc14b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 shutdown_connections 2024-04-25T10:21:35.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.814+0000 7fb3d3000640 1 -- 172.21.15.87:0/3517814048 wait complete. 2024-04-25T10:21:35.838 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:21:35.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.974+0000 7f06a7200640 1 Processor -- start 2024-04-25T10:21:35.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.974+0000 7f06a7200640 1 -- start start 2024-04-25T10:21:35.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06a815dfa0 0x7f06a80d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a80d0930 0x7f06a80d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a7200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f06a80d2390 0x7f06a80b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a7200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a80d0d90 con 0x7f06a80d2390 2024-04-25T10:21:35.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a80b74b0 con 0x7f06a815dfa0 2024-04-25T10:21:35.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80d2bf0 con 0x7f06a80d0930 2024-04-25T10:21:35.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06a815dfa0 0x7f06a80d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f06a80d2390 0x7f06a80b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06a815dfa0 0x7f06a80d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:45606/0 (socket says 172.21.15.87:45606) 2024-04-25T10:21:35.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f06a5e00640 1 -- 172.21.15.87:0/1221363088 learned_addr learned my addr 172.21.15.87:0/1221363088 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:35.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.978+0000 7f069d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a80d0930 0x7f06a80d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.982+0000 7f06a6800640 1 -- 172.21.15.87:0/1221363088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a80d0930 msgr2=0x7f06a80d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.982+0000 7f06a6800640 1 --2- 172.21.15.87:0/1221363088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a80d0930 0x7f06a80d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.982+0000 7f06a6800640 1 -- 172.21.15.87:0/1221363088 >> v2:172.21.15.169:3300/0 conn(0x7f06a815dfa0 msgr2=0x7f06a80d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.982+0000 7f06a6800640 1 --2- 172.21.15.87:0/1221363088 >> v2:172.21.15.169:3300/0 conn(0x7f06a815dfa0 0x7f06a80d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.982+0000 7f06a6800640 1 -- 172.21.15.87:0/1221363088 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80d2ed0 con 0x7f06a80d2390 2024-04-25T10:21:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.982+0000 7f069d400640 1 --2- 172.21.15.87:0/1221363088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a80d0930 0x7f06a80d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:35.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.990+0000 7f06a6800640 1 --2- 172.21.15.87:0/1221363088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d2390 0x7f06a80b5460 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f06a80d11d0 tx=0x7f06a001c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1385e6192a1a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.990+0000 7f06a5400640 1 -- 172.21.15.87:0/1221363088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a00a66f0 con 0x7f06a80d2390 2024-04-25T10:21:35.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.990+0000 7f06a5400640 1 -- 172.21.15.87:0/1221363088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a00a6890 con 0x7f06a80d2390 2024-04-25T10:21:35.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.990+0000 7f06a5400640 1 -- 172.21.15.87:0/1221363088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a00a66f0 con 0x7f06a80d2390 2024-04-25T10:21:35.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 -- 172.21.15.87:0/1221363088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d2390 msgr2=0x7f06a80b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 --2- 172.21.15.87:0/1221363088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d2390 0x7f06a80b5460 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f06a80d11d0 tx=0x7f06a001c640 comp rx=0 tx=0).stop 2024-04-25T10:21:35.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 -- 172.21.15.87:0/1221363088 shutdown_connections 2024-04-25T10:21:35.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 --2- 172.21.15.87:0/1221363088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d2390 0x7f06a80b5460 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 --2- 172.21.15.87:0/1221363088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a80d0930 0x7f06a80d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 --2- 172.21.15.87:0/1221363088 >> v2:172.21.15.169:3300/0 conn(0x7f06a815dfa0 0x7f06a80d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 -- 172.21.15.87:0/1221363088 >> 172.21.15.87:0/1221363088 conn(0x7f06a80669a0 msgr2=0x7f06a8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 -- 172.21.15.87:0/1221363088 shutdown_connections 2024-04-25T10:21:35.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.994+0000 7f06a7200640 1 -- 172.21.15.87:0/1221363088 wait complete. 2024-04-25T10:21:36.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 Processor -- start 2024-04-25T10:21:36.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 -- start start 2024-04-25T10:21:36.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d0930 0x7f06a814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a80d2390 0x7f06a815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a815dfa0 0x7f06a815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80b7aa0 con 0x7f06a80d0930 2024-04-25T10:21:36.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80b7920 con 0x7f06a80d2390 2024-04-25T10:21:36.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80b7c20 con 0x7f06a815dfa0 2024-04-25T10:21:36.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d0930 0x7f06a814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d0930 0x7f06a814cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44070/0 (socket says 172.21.15.87:44070) 2024-04-25T10:21:36.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:35.998+0000 7f06a5e00640 1 -- 172.21.15.87:0/159671220 learned_addr learned my addr 172.21.15.87:0/159671220 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:36.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a5e00640 1 -- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a815dfa0 msgr2=0x7f06a815f2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:21:36.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f069d400640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a80d2390 0x7f06a815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a6800640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a815dfa0 0x7f06a815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a5e00640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a815dfa0 0x7f06a815f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a5e00640 1 -- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a80d2390 msgr2=0x7f06a815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a5e00640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a80d2390 0x7f06a815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a5e00640 1 -- 172.21.15.87:0/159671220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8154500 con 0x7f06a80d0930 2024-04-25T10:21:36.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f069d400640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a80d2390 0x7f06a815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:36.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a5e00640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d0930 0x7f06a814cfa0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f069000e7f0 tx=0x7f069000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:36.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f069ea00640 1 -- 172.21.15.87:0/159671220 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690017070 con 0x7f06a80d0930 2024-04-25T10:21:36.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a814d4e0 con 0x7f06a80d0930 2024-04-25T10:21:36.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a815fbe0 con 0x7f06a80d0930 2024-04-25T10:21:36.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f069ea00640 1 -- 172.21.15.87:0/159671220 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000c940 con 0x7f06a80d0930 2024-04-25T10:21:36.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.002+0000 7f069ea00640 1 -- 172.21.15.87:0/159671220 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690017070 con 0x7f06a80d0930 2024-04-25T10:21:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.006+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06800052a0 con 0x7f06a80d0930 2024-04-25T10:21:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.006+0000 7f069ea00640 1 -- 172.21.15.87:0/159671220 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f069001a070 con 0x7f06a80d0930 2024-04-25T10:21:36.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.006+0000 7f069ea00640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0684067160 0x7f0684069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.006+0000 7f069ea00640 1 -- 172.21.15.87:0/159671220 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f06900847d0 con 0x7f06a80d0930 2024-04-25T10:21:36.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.010+0000 7f069d400640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0684067160 0x7f0684069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.010+0000 7f069d400640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0684067160 0x7f0684069620 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f06a80ca580 tx=0x7f0698008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.014+0000 7f069ea00640 1 -- 172.21.15.87:0/159671220 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0690055ae0 con 0x7f06a80d0930 2024-04-25T10:21:36.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:36.230+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f0680005860 con 0x7f06a80d0930 2024-04-25T10:21:37.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f069ea00640 1 -- 172.21.15.87:0/159671220 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v454) v1 ==== 169+0+0 (secure 0 0 0) 0x7f06900553e0 con 0x7f06a80d0930 2024-04-25T10:21:37.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0684067160 msgr2=0x7f0684069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0684067160 0x7f0684069620 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f06a80ca580 tx=0x7f0698008040 comp rx=0 tx=0).stop 2024-04-25T10:21:37.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d0930 msgr2=0x7f06a814cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d0930 0x7f06a814cfa0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f069000e7f0 tx=0x7f069000ecb0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 shutdown_connections 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0684067160 0x7f0684069620 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a815dfa0 0x7f06a815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a80d2390 0x7f06a815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 --2- 172.21.15.87:0/159671220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a80d0930 0x7f06a814cfa0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 >> 172.21.15.87:0/159671220 conn(0x7f06a80669a0 msgr2=0x7f06a8157b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 shutdown_connections 2024-04-25T10:21:37.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.126+0000 7f06a7200640 1 -- 172.21.15.87:0/159671220 wait complete. 2024-04-25T10:21:37.154 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:21:37.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f9546600640 1 Processor -- start 2024-04-25T10:21:37.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f9546600640 1 -- start start 2024-04-25T10:21:37.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f9546600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9540159540 0x7f95400ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f9546600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 0x7f9540070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f9546600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9540159920 con 0x7f95400aebd0 2024-04-25T10:21:37.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f9546600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954015c2f0 con 0x7f9540159540 2024-04-25T10:21:37.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f9546600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954015c5d0 con 0x7f95400721f0 2024-04-25T10:21:37.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f953ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f95400aebd0 0x7f9540071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45744/0 (socket says 172.21.15.87:45744) 2024-04-25T10:21:37.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.294+0000 7f953ea00640 1 -- 172.21.15.87:0/3428788100 learned_addr learned my addr 172.21.15.87:0/3428788100 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:37.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953fe00640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 0x7f9540070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953f400640 1 --2- 172.21.15.87:0/3428788100 >> v2:172.21.15.169:3300/0 conn(0x7f9540159540 0x7f95400ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111794262 0 0) 0x7f9540159920 con 0x7f95400aebd0 2024-04-25T10:21:37.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9524003530 con 0x7f95400aebd0 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4116445971 0 0) 0x7f9524003530 con 0x7f95400aebd0 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 msgr2=0x7f9540070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 0x7f9540070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 >> v2:172.21.15.169:3300/0 conn(0x7f9540159540 msgr2=0x7f95400ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 --2- 172.21.15.87:0/3428788100 >> v2:172.21.15.169:3300/0 conn(0x7f9540159540 0x7f95400ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954015c8b0 con 0x7f95400aebd0 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f952c001680 con 0x7f95400aebd0 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 >> v1:172.21.15.87:6789/0 conn(0x7f95400aebd0 legacy=0x7f9540071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 0x7f95240039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9524003f40 0x7f9524004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954015c2f0 con 0x7f9524004d50 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954015c5d0 con 0x7f9524003f40 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952400b9d0 con 0x7f95400721f0 2024-04-25T10:21:37.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953fe00640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953ea00640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9524003f40 0x7f9524004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953f400640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 0x7f95240039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953fe00640 1 -- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 msgr2=0x7f95240039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953fe00640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 0x7f95240039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953fe00640 1 -- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9524003f40 msgr2=0x7f9524004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953fe00640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9524003f40 0x7f9524004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953fe00640 1 -- 172.21.15.87:0/3428788100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524003530 con 0x7f9524004d50 2024-04-25T10:21:37.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.298+0000 7f953f400640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400721f0 0x7f95240039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:37.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953fe00640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95240051d0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9538009fc0 tx=0x7f953801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef747ad9d28cd8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380b2070 con 0x7f9524004d50 2024-04-25T10:21:37.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95380a5c90 con 0x7f9524004d50 2024-04-25T10:21:37.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953e000640 1 -- 172.21.15.87:0/3428788100 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538009c10 con 0x7f9524004d50 2024-04-25T10:21:37.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- 172.21.15.87:0/3428788100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 msgr2=0x7f95240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 --2- 172.21.15.87:0/3428788100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95240051d0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9538009fc0 tx=0x7f953801cc20 comp rx=0 tx=0).stop 2024-04-25T10:21:37.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953fe00640 1 -- 172.21.15.87:0/3428788100 reap_dead start 2024-04-25T10:21:37.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- 172.21.15.87:0/3428788100 shutdown_connections 2024-04-25T10:21:37.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- 172.21.15.87:0/3428788100 >> 172.21.15.87:0/3428788100 conn(0x7f95400669a0 msgr2=0x7f9540066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- 172.21.15.87:0/3428788100 shutdown_connections 2024-04-25T10:21:37.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- 172.21.15.87:0/3428788100 wait complete. 2024-04-25T10:21:37.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 Processor -- start 2024-04-25T10:21:37.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- start start 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95400b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95400b1a90 0x7f954014ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400b1f10 0x7f954014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954006c0d0 con 0x7f9524004d50 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954015bfa0 con 0x7f95400b1a90 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f9546600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9540070350 con 0x7f95400b1f10 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95400b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95400b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44088/0 (socket says 172.21.15.87:44088) 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.302+0000 7f953f400640 1 -- 172.21.15.87:0/1037111314 learned_addr learned my addr 172.21.15.87:0/1037111314 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953ea00640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95400b1a90 0x7f954014ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953fe00640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400b1f10 0x7f954014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953f400640 1 -- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400b1f10 msgr2=0x7f954014b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953f400640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400b1f10 0x7f954014b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953f400640 1 -- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95400b1a90 msgr2=0x7f954014ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953f400640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95400b1a90 0x7f954014ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953f400640 1 -- 172.21.15.87:0/1037111314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540156fb0 con 0x7f9524004d50 2024-04-25T10:21:37.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953f400640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95400b2530 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f95300118d0 tx=0x7f9530011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9537400640 1 -- 172.21.15.87:0/1037111314 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530004220 con 0x7f9524004d50 2024-04-25T10:21:37.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9537400640 1 -- 172.21.15.87:0/1037111314 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95300043c0 con 0x7f9524004d50 2024-04-25T10:21:37.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9537400640 1 -- 172.21.15.87:0/1037111314 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000fbd0 con 0x7f9524004d50 2024-04-25T10:21:37.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9540153560 con 0x7f9524004d50 2024-04-25T10:21:37.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9540157500 con 0x7f9524004d50 2024-04-25T10:21:37.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954004ef30 con 0x7f9524004d50 2024-04-25T10:21:37.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9537400640 1 -- 172.21.15.87:0/1037111314 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9530003620 con 0x7f9524004d50 2024-04-25T10:21:37.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9537400640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9520066f60 0x7f9520069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f9537400640 1 -- 172.21.15.87:0/1037111314 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f95300853b0 con 0x7f9524004d50 2024-04-25T10:21:37.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.306+0000 7f953ea00640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9520066f60 0x7f9520069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.310+0000 7f953ea00640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9520066f60 0x7f9520069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f95400ae540 tx=0x7f952c007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.310+0000 7f9537400640 1 -- 172.21.15.87:0/1037111314 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9530019030 con 0x7f9524004d50 2024-04-25T10:21:37.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.466+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f954006d7f0 con 0x7f9524004d50 2024-04-25T10:21:37.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.470+0000 7f9537400640 1 -- 172.21.15.87:0/1037111314 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9530009e20 con 0x7f9524004d50 2024-04-25T10:21:37.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9520066f60 msgr2=0x7f9520069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9520066f60 0x7f9520069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f95400ae540 tx=0x7f952c007900 comp rx=0 tx=0).stop 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 msgr2=0x7f95400b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95400b2530 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f95300118d0 tx=0x7f9530011d90 comp rx=0 tx=0).stop 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 shutdown_connections 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9520066f60 0x7f9520069420 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95400b1f10 0x7f954014b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95400b1a90 0x7f954014ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 --2- 172.21.15.87:0/1037111314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9524004d50 0x7f95400b2530 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 >> 172.21.15.87:0/1037111314 conn(0x7f95400669a0 msgr2=0x7f954006c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 shutdown_connections 2024-04-25T10:21:37.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.474+0000 7f9546600640 1 -- 172.21.15.87:0/1037111314 wait complete. 2024-04-25T10:21:37.496 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:21:37.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:37.598 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:37.710 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:21:37.710 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:21:37.710 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:37.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:21:37.719 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:21:37.719 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:37.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:37.728 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:21:37.728 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:37.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:21:37.767 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:21:37.767 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:37.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:37.779 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:21:37.779 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:37.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c95200640 1 Processor -- start 2024-04-25T10:21:37.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c95200640 1 -- start start 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c95200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c9015bc40 0x7f7c9006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c95200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c900cf930 0x7f7c900cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c95200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c900b2770 con 0x7f7c900cda10 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c95200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9006c670 con 0x7f7c900cf930 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c95200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9006c950 con 0x7f7c9015bc40 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c900cf930 0x7f7c900cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c900cf930 0x7f7c900cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35274/0 (socket says 172.21.15.87:35274) 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8e800640 1 -- 172.21.15.87:0/1617658512 learned_addr learned my addr 172.21.15.87:0/1617658512 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:37.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8de00640 1 --2- 172.21.15.87:0/1617658512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c9015bc40 0x7f7c9006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8e800640 1 -- 172.21.15.87:0/1617658512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c9015bc40 msgr2=0x7f7c9006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8e800640 1 --2- 172.21.15.87:0/1617658512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c9015bc40 0x7f7c9006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8e800640 1 -- 172.21.15.87:0/1617658512 >> v1:172.21.15.169:6789/0 conn(0x7f7c900cda10 legacy=0x7f7c900b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.942+0000 7f7c8e800640 1 -- 172.21.15.87:0/1617658512 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9006cc30 con 0x7f7c900cf930 2024-04-25T10:21:37.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c8e800640 1 --2- 172.21.15.87:0/1617658512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c900cf930 0x7f7c900cfde0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009870 tx=0x7f7c8801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dcfc2d3eef9569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c8d400640 1 -- 172.21.15.87:0/1617658512 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c880b0070 con 0x7f7c900cf930 2024-04-25T10:21:37.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c8d400640 1 -- 172.21.15.87:0/1617658512 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c880ab600 con 0x7f7c900cf930 2024-04-25T10:21:37.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c8d400640 1 -- 172.21.15.87:0/1617658512 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c880ab9d0 con 0x7f7c900cf930 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 -- 172.21.15.87:0/1617658512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c900cf930 msgr2=0x7f7c900cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 --2- 172.21.15.87:0/1617658512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c900cf930 0x7f7c900cfde0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009870 tx=0x7f7c8801c490 comp rx=0 tx=0).stop 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 -- 172.21.15.87:0/1617658512 shutdown_connections 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 --2- 172.21.15.87:0/1617658512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c900cf930 0x7f7c900cfde0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 --2- 172.21.15.87:0/1617658512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c9015bc40 0x7f7c9006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 -- 172.21.15.87:0/1617658512 >> 172.21.15.87:0/1617658512 conn(0x7f7c90066b70 msgr2=0x7f7c90066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 -- 172.21.15.87:0/1617658512 shutdown_connections 2024-04-25T10:21:37.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.946+0000 7f7c95200640 1 -- 172.21.15.87:0/1617658512 wait complete. 2024-04-25T10:21:37.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 Processor -- start 2024-04-25T10:21:37.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 -- start start 2024-04-25T10:21:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c900cda10 0x7f7c901fb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c900cf930 0x7f7c901fc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c9015bc40 0x7f7c9015f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900b3850 con 0x7f7c9015bc40 2024-04-25T10:21:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900b36d0 con 0x7f7c900cda10 2024-04-25T10:21:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900b39d0 con 0x7f7c900cf930 2024-04-25T10:21:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c9015bc40 0x7f7c9015f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c9015bc40 0x7f7c9015f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35288/0 (socket says 172.21.15.87:35288) 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 -- 172.21.15.87:0/3018702093 learned_addr learned my addr 172.21.15.87:0/3018702093 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8de00640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c900cda10 0x7f7c901fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c85400640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c900cf930 0x7f7c901fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 -- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c900cf930 msgr2=0x7f7c901fc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c900cf930 0x7f7c901fc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 -- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c900cda10 msgr2=0x7f7c901fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c900cda10 0x7f7c901fb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 -- 172.21.15.87:0/3018702093 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90153520 con 0x7f7c9015bc40 2024-04-25T10:21:37.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c8e800640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c9015bc40 0x7f7c9015f880 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7c880ae040 tx=0x7f7c88019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c86a00640 1 -- 172.21.15.87:0/3018702093 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c880b8020 con 0x7f7c9015bc40 2024-04-25T10:21:37.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c86a00640 1 -- 172.21.15.87:0/3018702093 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c880b4610 con 0x7f7c9015bc40 2024-04-25T10:21:37.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c901fb5b0 con 0x7f7c9015bc40 2024-04-25T10:21:37.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c86a00640 1 -- 172.21.15.87:0/3018702093 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c880b0040 con 0x7f7c9015bc40 2024-04-25T10:21:37.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.950+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c901fb850 con 0x7f7c9015bc40 2024-04-25T10:21:37.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.954+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9004ef30 con 0x7f7c9015bc40 2024-04-25T10:21:37.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.954+0000 7f7c86a00640 1 -- 172.21.15.87:0/3018702093 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c880bb020 con 0x7f7c9015bc40 2024-04-25T10:21:37.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.954+0000 7f7c86a00640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c6c066f60 0x7f7c6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.954+0000 7f7c86a00640 1 -- 172.21.15.87:0/3018702093 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7c8811e630 con 0x7f7c9015bc40 2024-04-25T10:21:37.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.954+0000 7f7c8de00640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c6c066f60 0x7f7c6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.954+0000 7f7c8de00640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c6c066f60 0x7f7c6c069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7c90158ca0 tx=0x7f7c80009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:37.958+0000 7f7c86a00640 1 -- 172.21.15.87:0/3018702093 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c880c2070 con 0x7f7c9015bc40 2024-04-25T10:21:38.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c9006d960 con 0x7f7c9015bc40 2024-04-25T10:21:38.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c86a00640 1 -- 172.21.15.87:0/3018702093 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f7c90153520 con 0x7f7c9015bc40 2024-04-25T10:21:38.153 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:38.153 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1387424822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1387424822},{"type":"v1","addr":"172.21.15.169:6835","nonce":1387424822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4171115123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4171115123},{"type":"v1","addr":"172.21.15.169:6834","nonce":4171115123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1690993515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1690993515},{"type":"v1","addr":"172.21.15.87:6836","nonce":1690993515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9027,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/924288930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":924288930},{"type":"v1","addr":"172.21.15.87:6837","nonce":924288930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[]} 2024-04-25T10:21:38.154 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 82 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c6c066f60 msgr2=0x7f7c6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c6c066f60 0x7f7c6c069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7c90158ca0 tx=0x7f7c80009830 comp rx=0 tx=0).stop 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c9015bc40 msgr2=0x7f7c9015f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c9015bc40 0x7f7c9015f880 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7c880ae040 tx=0x7f7c88019230 comp rx=0 tx=0).stop 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 shutdown_connections 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c6c066f60 0x7f7c6c069420 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c9015bc40 0x7f7c9015f880 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c900cf930 0x7f7c901fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 --2- 172.21.15.87:0/3018702093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c900cda10 0x7f7c901fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 >> 172.21.15.87:0/3018702093 conn(0x7f7c90066b70 msgr2=0x7f7c901554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 shutdown_connections 2024-04-25T10:21:38.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.150+0000 7f7c95200640 1 -- 172.21.15.87:0/3018702093 wait complete. 2024-04-25T10:21:38.175 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:21:38.175 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:21:38.175 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:21:38.175 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:38.176 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:21:38.176 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:21:38.176 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:21:38.176 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:38.176 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:21:38.176 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:21:38.177 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:21:38.177 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:38.177 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:21:38.177 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:21:38.177 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:21:38.177 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:38.190 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:21:38.191 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:21:38.191 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:21:38.191 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:21:38.194 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:21:38.194 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:21:38.195 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:21:38.195 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:21:38.237 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:21:38.238 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:38.241 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:21:38.241 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:38.244 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:21:38.244 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:38.246 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:21:38.246 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:38.246 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:21:38.322 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:21:38.325 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:21:38.333 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:21:38.334 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:21:38.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa691200640 1 Processor -- start 2024-04-25T10:21:38.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa691200640 1 -- start start 2024-04-25T10:21:38.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa691200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cda10 0x7fa68c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa691200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c0b2770 con 0x7fa68c15bc40 2024-04-25T10:21:38.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa691200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c06c670 con 0x7fa68c0cf930 2024-04-25T10:21:38.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa691200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c06c950 con 0x7fa68c0cda10 2024-04-25T10:21:38.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa689e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa68c15bc40 0x7fa68c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48968/0 (socket says 172.21.15.87:48968) 2024-04-25T10:21:38.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa689e00640 1 -- 172.21.15.87:0/3560469809 learned_addr learned my addr 172.21.15.87:0/3560469809 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:38.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa689400640 1 --2- 172.21.15.87:0/3560469809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cda10 0x7fa68c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa688a00640 1 -- 172.21.15.87:0/3560469809 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425851104 0 0) 0x7fa68c0b2770 con 0x7fa68c15bc40 2024-04-25T10:21:38.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa688a00640 1 -- 172.21.15.87:0/3560469809 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa670003530 con 0x7fa68c15bc40 2024-04-25T10:21:38.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa688a00640 1 -- 172.21.15.87:0/3560469809 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356584462 0 0) 0x7fa68c06c670 con 0x7fa68c0cf930 2024-04-25T10:21:38.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa689400640 1 -- 172.21.15.87:0/3560469809 >> v1:172.21.15.169:6789/0 conn(0x7fa68c0cf930 legacy=0x7fa68c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa689400640 1 -- 172.21.15.87:0/3560469809 >> v1:172.21.15.87:6789/0 conn(0x7fa68c15bc40 legacy=0x7fa68c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.390+0000 7fa689400640 1 -- 172.21.15.87:0/3560469809 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c06cc30 con 0x7fa68c0cda10 2024-04-25T10:21:38.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa689400640 1 --2- 172.21.15.87:0/3560469809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cda10 0x7fa68c06c090 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa6780097f0 tx=0x7fa67801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3b2b7afa5c4b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:38.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa688a00640 1 -- 172.21.15.87:0/3560469809 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780b2070 con 0x7fa68c0cda10 2024-04-25T10:21:38.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa688a00640 1 -- 172.21.15.87:0/3560469809 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6780a9070 con 0x7fa68c0cda10 2024-04-25T10:21:38.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa688a00640 1 -- 172.21.15.87:0/3560469809 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780ad880 con 0x7fa68c0cda10 2024-04-25T10:21:38.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa691200640 1 -- 172.21.15.87:0/3560469809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cda10 msgr2=0x7fa68c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa691200640 1 --2- 172.21.15.87:0/3560469809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cda10 0x7fa68c06c090 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa6780097f0 tx=0x7fa67801c450 comp rx=0 tx=0).stop 2024-04-25T10:21:38.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa691200640 1 -- 172.21.15.87:0/3560469809 shutdown_connections 2024-04-25T10:21:38.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa691200640 1 --2- 172.21.15.87:0/3560469809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cda10 0x7fa68c06c090 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa691200640 1 -- 172.21.15.87:0/3560469809 >> 172.21.15.87:0/3560469809 conn(0x7fa68c066b70 msgr2=0x7fa68c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:38.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa691200640 1 -- 172.21.15.87:0/3560469809 shutdown_connections 2024-04-25T10:21:38.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.394+0000 7fa691200640 1 -- 172.21.15.87:0/3560469809 wait complete. 2024-04-25T10:21:38.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 Processor -- start 2024-04-25T10:21:38.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 -- start start 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c0cda10 0x7fa68c14eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cf930 0x7fa68c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa68c15bc40 0x7fa68c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c169d70 con 0x7fa68c15bc40 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c169bf0 con 0x7fa68c0cda10 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c169ef0 con 0x7fa68c0cf930 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa68c15bc40 0x7fa68c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa68c15bc40 0x7fa68c15da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35318/0 (socket says 172.21.15.87:35318) 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 -- 172.21.15.87:0/2422081426 learned_addr learned my addr 172.21.15.87:0/2422081426 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:38.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa689e00640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c0cda10 0x7fa68c14eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa689400640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cf930 0x7fa68c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 -- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cf930 msgr2=0x7fa68c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cf930 0x7fa68c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 -- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c0cda10 msgr2=0x7fa68c14eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c0cda10 0x7fa68c14eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 -- 172.21.15.87:0/2422081426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c15c6d0 con 0x7fa68c15bc40 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa689e00640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c0cda10 0x7fa68c14eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa689400640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cf930 0x7fa68c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa68a800640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa68c15bc40 0x7fa68c15da10 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c780 tx=0x7fa68400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa682000640 1 -- 172.21.15.87:0/2422081426 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684003db0 con 0x7fa68c15bc40 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa682000640 1 -- 172.21.15.87:0/2422081426 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa684005070 con 0x7fa68c15bc40 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c15c970 con 0x7fa68c15bc40 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa682000640 1 -- 172.21.15.87:0/2422081426 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684005460 con 0x7fa68c15bc40 2024-04-25T10:21:38.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c15cf30 con 0x7fa68c15bc40 2024-04-25T10:21:38.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.398+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c04ef30 con 0x7fa68c15bc40 2024-04-25T10:21:38.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.402+0000 7fa682000640 1 -- 172.21.15.87:0/2422081426 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa684005600 con 0x7fa68c15bc40 2024-04-25T10:21:38.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.402+0000 7fa682000640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa66c066f60 0x7fa66c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.402+0000 7fa682000640 1 -- 172.21.15.87:0/2422081426 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa684084dd0 con 0x7fa68c15bc40 2024-04-25T10:21:38.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.402+0000 7fa689e00640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa66c066f60 0x7fa66c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.402+0000 7fa689e00640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa66c066f60 0x7fa66c069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005890 tx=0x7fa67c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:38.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.406+0000 7fa682000640 1 -- 172.21.15.87:0/2422081426 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa684055fb0 con 0x7fa68c15bc40 2024-04-25T10:21:38.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:38.570+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa68c06d960 con 0x7fa68c15bc40 2024-04-25T10:21:39.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.142+0000 7fa682000640 1 -- 172.21.15.87:0/2422081426 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa6840558b0 con 0x7fa68c15bc40 2024-04-25T10:21:39.147 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:21:39.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa66c066f60 msgr2=0x7fa66c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa66c066f60 0x7fa66c069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005890 tx=0x7fa67c009040 comp rx=0 tx=0).stop 2024-04-25T10:21:39.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa68c15bc40 msgr2=0x7fa68c15da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa68c15bc40 0x7fa68c15da10 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c780 tx=0x7fa68400cc40 comp rx=0 tx=0).stop 2024-04-25T10:21:39.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 shutdown_connections 2024-04-25T10:21:39.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa66c066f60 0x7fa66c069420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa68c15bc40 0x7fa68c15da10 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68c0cf930 0x7fa68c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 --2- 172.21.15.87:0/2422081426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68c0cda10 0x7fa68c14eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 >> 172.21.15.87:0/2422081426 conn(0x7fa68c066b70 msgr2=0x7fa68c159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:39.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 shutdown_connections 2024-04-25T10:21:39.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.146+0000 7fa691200640 1 -- 172.21.15.87:0/2422081426 wait complete. 2024-04-25T10:21:39.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:21:39.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.306+0000 7f275ba00640 1 Processor -- start 2024-04-25T10:21:39.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.306+0000 7f275ba00640 1 -- start start 2024-04-25T10:21:39.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f275ba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f27541550b0 0x7f27540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f275ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 0x7f275415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f275ba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2754071050 con 0x7f275415b740 2024-04-25T10:21:39.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f275ba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275415d040 con 0x7f27541550b0 2024-04-25T10:21:39.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f275ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27540712f0 con 0x7f27540cfe70 2024-04-25T10:21:39.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f275415b740 0x7f275406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48972/0 (socket says 172.21.15.87:48972) 2024-04-25T10:21:39.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 -- 172.21.15.87:0/1937985504 learned_addr learned my addr 172.21.15.87:0/1937985504 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759000640 1 --2- 172.21.15.87:0/1937985504 >> v2:172.21.15.169:3300/0 conn(0x7f27541550b0 0x7f27540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753e00640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 0x7f275415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021221581 0 0) 0x7f2754071050 con 0x7f275415b740 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2738003530 con 0x7f275415b740 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015116842 0 0) 0x7f2738003530 con 0x7f275415b740 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 msgr2=0x7f275415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 0x7f275415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 >> v2:172.21.15.169:3300/0 conn(0x7f27541550b0 msgr2=0x7f27540cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 --2- 172.21.15.87:0/1937985504 >> v2:172.21.15.169:3300/0 conn(0x7f27541550b0 0x7f27540cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27540715d0 con 0x7f275415b740 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f274c001a00 con 0x7f275415b740 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 >> v1:172.21.15.87:6789/0 conn(0x7f275415b740 legacy=0x7f275406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 0x7f27380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2738003f90 0x7f2738004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2738004d50 0x7f27380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275415d040 con 0x7f2738004d50 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27540712f0 con 0x7f2738003f90 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273800b9d0 con 0x7f27540cfe70 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2738004d50 0x7f27380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2753e00640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2738003f90 0x7f2738004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759000640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 0x7f27380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 -- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 msgr2=0x7f27380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27540cfe70 0x7f27380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 -- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2738003f90 msgr2=0x7f2738004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2738003f90 0x7f2738004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.310+0000 7f2759a00640 1 -- 172.21.15.87:0/1937985504 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738003530 con 0x7f2738004d50 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f2759a00640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2738004d50 0x7f27380051d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f274c00a7c0 tx=0x7f274c01d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1f73b4054ee741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c00a410 con 0x7f2738004d50 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274c0ad630 con 0x7f2738004d50 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f2753400640 1 -- 172.21.15.87:0/1937985504 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c0ada20 con 0x7f2738004d50 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 -- 172.21.15.87:0/1937985504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2738004d50 msgr2=0x7f27380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 --2- 172.21.15.87:0/1937985504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2738004d50 0x7f27380051d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f274c00a7c0 tx=0x7f274c01d2c0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f2759a00640 1 -- 172.21.15.87:0/1937985504 reap_dead start 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 -- 172.21.15.87:0/1937985504 shutdown_connections 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 -- 172.21.15.87:0/1937985504 >> 172.21.15.87:0/1937985504 conn(0x7f2754066b70 msgr2=0x7f2754066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 -- 172.21.15.87:0/1937985504 shutdown_connections 2024-04-25T10:21:39.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 -- 172.21.15.87:0/1937985504 wait complete. 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 Processor -- start 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.314+0000 7f275ba00640 1 -- start start 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2738004d50 0x7f27540b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f275406d4f0 0x7f2754142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f275406d970 0x7f27541434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2754071b90 con 0x7f275406d4f0 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27540718b0 con 0x7f275406d970 2024-04-25T10:21:39.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27540720b0 con 0x7f2738004d50 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f275406d4f0 0x7f2754142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f275406d4f0 0x7f2754142f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35328/0 (socket says 172.21.15.87:35328) 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 -- 172.21.15.87:0/1295299081 learned_addr learned my addr 172.21.15.87:0/1295299081 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2759000640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2738004d50 0x7f27540b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2759a00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f275406d970 0x7f27541434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 -- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2738004d50 msgr2=0x7f27540b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2738004d50 0x7f27540b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 -- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f275406d970 msgr2=0x7f27541434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f275406d970 0x7f27541434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 -- 172.21.15.87:0/1295299081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275406deb0 con 0x7f275406d4f0 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2759a00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f275406d970 0x7f27541434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2753e00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f275406d4f0 0x7f2754142f60 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f273c011a00 tx=0x7f273c011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2750c00640 1 -- 172.21.15.87:0/1295299081 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00faf0 con 0x7f275406d4f0 2024-04-25T10:21:39.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2750c00640 1 -- 172.21.15.87:0/1295299081 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c00fc90 con 0x7f275406d4f0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27540cc9d0 con 0x7f275406d4f0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2750c00640 1 -- 172.21.15.87:0/1295299081 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c014440 con 0x7f275406d4f0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2754159d70 con 0x7f275406d4f0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275404ef30 con 0x7f275406d4f0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.318+0000 7f2750c00640 1 -- 172.21.15.87:0/1295299081 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f273c012040 con 0x7f275406d4f0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.322+0000 7f2750c00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2730066f90 0x7f2730069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.322+0000 7f2750c00640 1 -- 172.21.15.87:0/1295299081 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f273c085400 con 0x7f275406d4f0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.322+0000 7f2759000640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2730066f90 0x7f2730069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.322+0000 7f2759000640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2730066f90 0x7f2730069450 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2744009bc0 tx=0x7f2744009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.326+0000 7f2750c00640 1 -- 172.21.15.87:0/1295299081 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f273c019030 con 0x7f275406d4f0 2024-04-25T10:21:39.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.530+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2754151090 con 0x7f275406d4f0 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.530+0000 7f2750c00640 1 -- 172.21.15.87:0/1295299081 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f273c056130 con 0x7f275406d4f0 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:21:39.534 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:21:39.535 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:21:39.536 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:21:39.538 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.540 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:21:39.541 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:21:39.542 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.544 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:21:39.546 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:21:39.547 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:21:39.547 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:21:39.547 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:21:39.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:39.547 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:21:39.547 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:39.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2730066f90 msgr2=0x7f2730069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2730066f90 0x7f2730069450 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2744009bc0 tx=0x7f2744009190 comp rx=0 tx=0).stop 2024-04-25T10:21:39.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f275406d4f0 msgr2=0x7f2754142f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f275406d4f0 0x7f2754142f60 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f273c011a00 tx=0x7f273c011e00 comp rx=0 tx=0).stop 2024-04-25T10:21:39.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 shutdown_connections 2024-04-25T10:21:39.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2730066f90 0x7f2730069450 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f275406d970 0x7f27541434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f275406d4f0 0x7f2754142f60 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 --2- 172.21.15.87:0/1295299081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2738004d50 0x7f27540b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 >> 172.21.15.87:0/1295299081 conn(0x7f2754066b70 msgr2=0x7f27540ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:39.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 shutdown_connections 2024-04-25T10:21:39.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.534+0000 7f275ba00640 1 -- 172.21.15.87:0/1295299081 wait complete. 2024-04-25T10:21:39.555 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:39.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.686+0000 7f1027c00640 1 Processor -- start 2024-04-25T10:21:39.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.686+0000 7f1027c00640 1 -- start start 2024-04-25T10:21:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.686+0000 7f1027c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f102015b910 0x7f102015c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.686+0000 7f1027c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102015bd60 con 0x7f10200d0040 2024-04-25T10:21:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.686+0000 7f1027c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1020071050 con 0x7f10201550b0 2024-04-25T10:21:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.686+0000 7f1027c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10200712f0 con 0x7f102015b910 2024-04-25T10:21:39.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f101fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f10200d0040 0x7f102015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48988/0 (socket says 172.21.15.87:48988) 2024-04-25T10:21:39.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f101fe00640 1 -- 172.21.15.87:0/274073834 learned_addr learned my addr 172.21.15.87:0/274073834 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:39.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f1025c00640 1 --2- 172.21.15.87:0/274073834 >> v2:172.21.15.169:3300/0 conn(0x7f102015b910 0x7f102015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f101f400640 1 -- 172.21.15.87:0/274073834 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067718036 0 0) 0x7f102015bd60 con 0x7f10200d0040 2024-04-25T10:21:39.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f101f400640 1 -- 172.21.15.87:0/274073834 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1004003530 con 0x7f10200d0040 2024-04-25T10:21:39.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f101f400640 1 -- 172.21.15.87:0/274073834 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778665976 0 0) 0x7f1020071050 con 0x7f10201550b0 2024-04-25T10:21:39.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f1025c00640 1 -- 172.21.15.87:0/274073834 >> v1:172.21.15.169:6789/0 conn(0x7f10201550b0 legacy=0x7f10200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f1025c00640 1 -- 172.21.15.87:0/274073834 >> v1:172.21.15.87:6789/0 conn(0x7f10200d0040 legacy=0x7f102015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f1025c00640 1 -- 172.21.15.87:0/274073834 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200715d0 con 0x7f102015b910 2024-04-25T10:21:39.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.690+0000 7f1025c00640 1 --2- 172.21.15.87:0/274073834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f102015b910 0x7f102015c030 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1018009d90 tx=0x7f101801c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1271805c697ec4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f101f400640 1 -- 172.21.15.87:0/274073834 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180b2070 con 0x7f102015b910 2024-04-25T10:21:39.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f101f400640 1 -- 172.21.15.87:0/274073834 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10180a5600 con 0x7f102015b910 2024-04-25T10:21:39.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f101f400640 1 -- 172.21.15.87:0/274073834 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008001680 con 0x7f102015b910 2024-04-25T10:21:39.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- 172.21.15.87:0/274073834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f102015b910 msgr2=0x7f102015c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 --2- 172.21.15.87:0/274073834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f102015b910 0x7f102015c030 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1018009d90 tx=0x7f101801c780 comp rx=0 tx=0).stop 2024-04-25T10:21:39.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- 172.21.15.87:0/274073834 shutdown_connections 2024-04-25T10:21:39.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 --2- 172.21.15.87:0/274073834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f102015b910 0x7f102015c030 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- 172.21.15.87:0/274073834 >> 172.21.15.87:0/274073834 conn(0x7f1020066b70 msgr2=0x7f1020066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:39.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- 172.21.15.87:0/274073834 shutdown_connections 2024-04-25T10:21:39.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- 172.21.15.87:0/274073834 wait complete. 2024-04-25T10:21:39.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 Processor -- start 2024-04-25T10:21:39.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- start start 2024-04-25T10:21:39.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10200d0040 0x7f102014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201550b0 0x7f10201554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f102015b910 0x7f1020150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102015cf00 con 0x7f102015b910 2024-04-25T10:21:39.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102015cd80 con 0x7f10201550b0 2024-04-25T10:21:39.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1027c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102015d080 con 0x7f10200d0040 2024-04-25T10:21:39.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1025c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f102015b910 0x7f1020150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1025c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f102015b910 0x7f1020150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35330/0 (socket says 172.21.15.87:35330) 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1025c00640 1 -- 172.21.15.87:0/2386211077 learned_addr learned my addr 172.21.15.87:0/2386211077 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f101fe00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201550b0 0x7f10201554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1025c00640 1 -- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10200d0040 msgr2=0x7f102014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.694+0000 7f1025200640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10200d0040 0x7f102014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1025c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10200d0040 0x7f102014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1025c00640 1 -- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201550b0 msgr2=0x7f10201554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1025c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201550b0 0x7f10201554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1025c00640 1 -- 172.21.15.87:0/2386211077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200afac0 con 0x7f102015b910 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f101fe00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201550b0 0x7f10201554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:39.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1025200640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10200d0040 0x7f102014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:39.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1025c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f102015b910 0x7f1020150da0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f10180b0040 tx=0x7f1018019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f101cc00640 1 -- 172.21.15.87:0/2386211077 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180bb020 con 0x7f102015b910 2024-04-25T10:21:39.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f101cc00640 1 -- 172.21.15.87:0/2386211077 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10180a59b0 con 0x7f102015b910 2024-04-25T10:21:39.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f101cc00640 1 -- 172.21.15.87:0/2386211077 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180b2040 con 0x7f102015b910 2024-04-25T10:21:39.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10200afda0 con 0x7f102015b910 2024-04-25T10:21:39.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1020155a30 con 0x7f102015b910 2024-04-25T10:21:39.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102004ef30 con 0x7f102015b910 2024-04-25T10:21:39.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.698+0000 7f101cc00640 1 -- 172.21.15.87:0/2386211077 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10180be020 con 0x7f102015b910 2024-04-25T10:21:39.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.702+0000 7f101cc00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ffc066f60 0x7f0ffc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.702+0000 7f101cc00640 1 -- 172.21.15.87:0/2386211077 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f101811f3c0 con 0x7f102015b910 2024-04-25T10:21:39.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.702+0000 7f1025200640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ffc066f60 0x7f0ffc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.702+0000 7f1025200640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ffc066f60 0x7f0ffc069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f102015add0 tx=0x7f1010009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.706+0000 7f101cc00640 1 -- 172.21.15.87:0/2386211077 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10180c5030 con 0x7f102015b910 2024-04-25T10:21:39.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.858+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10200cb750 con 0x7f102015b910 2024-04-25T10:21:39.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f101cc00640 1 -- 172.21.15.87:0/2386211077 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f1018016460 con 0x7f102015b910 2024-04-25T10:21:39.864 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:39.865 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":120,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:39.157187+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:39.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ffc066f60 msgr2=0x7f0ffc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ffc066f60 0x7f0ffc069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f102015add0 tx=0x7f1010009040 comp rx=0 tx=0).stop 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f102015b910 msgr2=0x7f1020150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f102015b910 0x7f1020150da0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f10180b0040 tx=0x7f1018019520 comp rx=0 tx=0).stop 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 shutdown_connections 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ffc066f60 0x7f0ffc069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f102015b910 0x7f1020150da0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201550b0 0x7f10201554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 --2- 172.21.15.87:0/2386211077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10200d0040 0x7f102014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 >> 172.21.15.87:0/2386211077 conn(0x7f1020066b70 msgr2=0x7f1020154320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 shutdown_connections 2024-04-25T10:21:39.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:39.862+0000 7f1027c00640 1 -- 172.21.15.87:0/2386211077 wait complete. 2024-04-25T10:21:39.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:39.885 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:40.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ef1200640 1 Processor -- start 2024-04-25T10:21:40.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ef1200640 1 -- start start 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ef1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec15b150 0x7f5eec0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ef1200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5eec06c0d0 0x7f5eec06c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ef1200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec0b2fd0 con 0x7f5eec15a0e0 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ef1200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0aef90 con 0x7f5eec06c0d0 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ef1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0af1d0 con 0x7f5eec15b150 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5eea800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5eec06c0d0 0x7f5eec06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5eea800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5eec06c0d0 0x7f5eec06c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35340/0 (socket says 172.21.15.87:35340) 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5eea800640 1 -- 172.21.15.87:0/2132010671 learned_addr learned my addr 172.21.15.87:0/2132010671 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:40.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.022+0000 7f5ee1400640 1 --2- 172.21.15.87:0/2132010671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec15b150 0x7f5eec0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5eea800640 1 -- 172.21.15.87:0/2132010671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec15b150 msgr2=0x7f5eec0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:40.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5eea800640 1 --2- 172.21.15.87:0/2132010671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec15b150 0x7f5eec0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5eea800640 1 -- 172.21.15.87:0/2132010671 >> v1:172.21.15.169:6789/0 conn(0x7f5eec15a0e0 legacy=0x7f5eec0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5eea800640 1 -- 172.21.15.87:0/2132010671 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0b32b0 con 0x7f5eec06c0d0 2024-04-25T10:21:40.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5eea800640 1 --2- 172.21.15.87:0/2132010671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec06c0d0 0x7f5eec06c520 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009870 tx=0x7f5ee401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763d58d41231eacf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ee9400640 1 -- 172.21.15.87:0/2132010671 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40b0070 con 0x7f5eec06c0d0 2024-04-25T10:21:40.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ee9400640 1 -- 172.21.15.87:0/2132010671 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee40ab600 con 0x7f5eec06c0d0 2024-04-25T10:21:40.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ee9400640 1 -- 172.21.15.87:0/2132010671 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40ab9d0 con 0x7f5eec06c0d0 2024-04-25T10:21:40.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ef1200640 1 -- 172.21.15.87:0/2132010671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec06c0d0 msgr2=0x7f5eec06c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ef1200640 1 --2- 172.21.15.87:0/2132010671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec06c0d0 0x7f5eec06c520 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009870 tx=0x7f5ee401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ef1200640 1 -- 172.21.15.87:0/2132010671 shutdown_connections 2024-04-25T10:21:40.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ef1200640 1 --2- 172.21.15.87:0/2132010671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec06c0d0 0x7f5eec06c520 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ef1200640 1 --2- 172.21.15.87:0/2132010671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec15b150 0x7f5eec0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ef1200640 1 -- 172.21.15.87:0/2132010671 >> 172.21.15.87:0/2132010671 conn(0x7f5eec0669a0 msgr2=0x7f5eec066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:40.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.026+0000 7f5ef1200640 1 -- 172.21.15.87:0/2132010671 shutdown_connections 2024-04-25T10:21:40.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 -- 172.21.15.87:0/2132010671 wait complete. 2024-04-25T10:21:40.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 Processor -- start 2024-04-25T10:21:40.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 -- start start 2024-04-25T10:21:40.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec06c0d0 0x7f5eec144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec15a0e0 0x7f5eec14b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eec15b150 0x7f5eec157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b4620 con 0x7f5eec15a0e0 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b44a0 con 0x7f5eec15b150 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0705f0 con 0x7f5eec06c0d0 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec15a0e0 0x7f5eec14b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec15a0e0 0x7f5eec14b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35350/0 (socket says 172.21.15.87:35350) 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 -- 172.21.15.87:0/3639913338 learned_addr learned my addr 172.21.15.87:0/3639913338 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee9e00640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec06c0d0 0x7f5eec144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5eea800640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eec15b150 0x7f5eec157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 -- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec06c0d0 msgr2=0x7f5eec144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec06c0d0 0x7f5eec144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 -- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eec15b150 msgr2=0x7f5eec157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eec15b150 0x7f5eec157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 -- 172.21.15.87:0/3639913338 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec146ba0 con 0x7f5eec15a0e0 2024-04-25T10:21:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee1400640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec15a0e0 0x7f5eec14b420 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e870 tx=0x7f5ed400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee2a00640 1 -- 172.21.15.87:0/3639913338 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4017070 con 0x7f5eec15a0e0 2024-04-25T10:21:40.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee2a00640 1 -- 172.21.15.87:0/3639913338 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed400c940 con 0x7f5eec15a0e0 2024-04-25T10:21:40.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ee2a00640 1 -- 172.21.15.87:0/3639913338 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400ccf0 con 0x7f5eec15a0e0 2024-04-25T10:21:40.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.030+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec146300 con 0x7f5eec15a0e0 2024-04-25T10:21:40.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.034+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec1468c0 con 0x7f5eec15a0e0 2024-04-25T10:21:40.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.034+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec04ef30 con 0x7f5eec15a0e0 2024-04-25T10:21:40.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.034+0000 7f5ee2a00640 1 -- 172.21.15.87:0/3639913338 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ed401a070 con 0x7f5eec15a0e0 2024-04-25T10:21:40.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.034+0000 7f5ee2a00640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ec8066f60 0x7f5ec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.034+0000 7f5ee2a00640 1 -- 172.21.15.87:0/3639913338 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ed4084b00 con 0x7f5eec15a0e0 2024-04-25T10:21:40.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.034+0000 7f5ee9e00640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ec8066f60 0x7f5ec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.034+0000 7f5ee9e00640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ec8066f60 0x7f5ec8069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0012f0 tx=0x7f5edc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.038+0000 7f5ee2a00640 1 -- 172.21.15.87:0/3639913338 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ed4055800 con 0x7f5eec15a0e0 2024-04-25T10:21:40.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.198+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5eec06e1f0 con 0x7f5eec15a0e0 2024-04-25T10:21:40.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.198+0000 7f5ee2a00640 1 -- 172.21.15.87:0/3639913338 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ed40598a0 con 0x7f5eec15a0e0 2024-04-25T10:21:40.203 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:21:40.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ec8066f60 msgr2=0x7f5ec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ec8066f60 0x7f5ec8069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0012f0 tx=0x7f5edc009040 comp rx=0 tx=0).stop 2024-04-25T10:21:40.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec15a0e0 msgr2=0x7f5eec14b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec15a0e0 0x7f5eec14b420 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e870 tx=0x7f5ed400ec70 comp rx=0 tx=0).stop 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 shutdown_connections 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ec8066f60 0x7f5ec8069420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eec15b150 0x7f5eec157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec15a0e0 0x7f5eec14b420 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 --2- 172.21.15.87:0/3639913338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eec06c0d0 0x7f5eec144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 >> 172.21.15.87:0/3639913338 conn(0x7f5eec0669a0 msgr2=0x7f5eec06d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 shutdown_connections 2024-04-25T10:21:40.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.202+0000 7f5ef1200640 1 -- 172.21.15.87:0/3639913338 wait complete. 2024-04-25T10:21:40.224 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:21:40.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.354+0000 7f21a8c00640 1 Processor -- start 2024-04-25T10:21:40.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.354+0000 7f21a8c00640 1 -- start start 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a8c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21a415bc40 0x7f21a406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a40cf930 0x7f21a40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a8c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a40b2770 con 0x7f21a40cda10 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a8c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a406c670 con 0x7f21a415bc40 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a406c950 con 0x7f21a40cf930 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a40cf930 0x7f21a40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a40cf930 0x7f21a40cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35534/0 (socket says 172.21.15.87:35534) 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a2800640 1 -- 172.21.15.87:0/484993205 learned_addr learned my addr 172.21.15.87:0/484993205 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a1e00640 1 --2- 172.21.15.87:0/484993205 >> v2:172.21.15.169:3300/0 conn(0x7f21a415bc40 0x7f21a406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a2800640 1 -- 172.21.15.87:0/484993205 >> v2:172.21.15.169:3300/0 conn(0x7f21a415bc40 msgr2=0x7f21a406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a2800640 1 --2- 172.21.15.87:0/484993205 >> v2:172.21.15.169:3300/0 conn(0x7f21a415bc40 0x7f21a406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a2800640 1 -- 172.21.15.87:0/484993205 >> v1:172.21.15.169:6789/0 conn(0x7f21a40cda10 legacy=0x7f21a40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a2800640 1 -- 172.21.15.87:0/484993205 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a406cc30 con 0x7f21a40cf930 2024-04-25T10:21:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.358+0000 7f21a1e00640 1 --2- 172.21.15.87:0/484993205 >> v2:172.21.15.169:3300/0 conn(0x7f21a415bc40 0x7f21a406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a2800640 1 --2- 172.21.15.87:0/484993205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a40cf930 0x7f21a40cfde0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f219c009870 tx=0x7f219c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae00fbd4cd29070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a1400640 1 -- 172.21.15.87:0/484993205 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0b0070 con 0x7f21a40cf930 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a1400640 1 -- 172.21.15.87:0/484993205 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c0ab600 con 0x7f21a40cf930 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a1400640 1 -- 172.21.15.87:0/484993205 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0ab9d0 con 0x7f21a40cf930 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 -- 172.21.15.87:0/484993205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a40cf930 msgr2=0x7f21a40cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 --2- 172.21.15.87:0/484993205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a40cf930 0x7f21a40cfde0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f219c009870 tx=0x7f219c01c490 comp rx=0 tx=0).stop 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 -- 172.21.15.87:0/484993205 shutdown_connections 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 --2- 172.21.15.87:0/484993205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a40cf930 0x7f21a40cfde0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 --2- 172.21.15.87:0/484993205 >> v2:172.21.15.169:3300/0 conn(0x7f21a415bc40 0x7f21a406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 -- 172.21.15.87:0/484993205 >> 172.21.15.87:0/484993205 conn(0x7f21a4066b70 msgr2=0x7f21a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:40.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 -- 172.21.15.87:0/484993205 shutdown_connections 2024-04-25T10:21:40.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 -- 172.21.15.87:0/484993205 wait complete. 2024-04-25T10:21:40.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 Processor -- start 2024-04-25T10:21:40.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.362+0000 7f21a8c00640 1 -- start start 2024-04-25T10:21:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a40cda10 0x7f21a414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21a40cf930 0x7f21a4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a415bc40 0x7f21a4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40b3890 con 0x7f21a40cf930 2024-04-25T10:21:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40b3710 con 0x7f21a40cda10 2024-04-25T10:21:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40b3a10 con 0x7f21a415bc40 2024-04-25T10:21:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21a40cf930 0x7f21a4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21a40cf930 0x7f21a4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35352/0 (socket says 172.21.15.87:35352) 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 -- 172.21.15.87:0/366836344 learned_addr learned my addr 172.21.15.87:0/366836344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a1e00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a40cda10 0x7f21a414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a2800640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a415bc40 0x7f21a4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 -- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a415bc40 msgr2=0x7f21a4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a415bc40 0x7f21a4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 -- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a40cda10 msgr2=0x7f21a414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a40cda10 0x7f21a414b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 -- 172.21.15.87:0/366836344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40b0200 con 0x7f21a40cf930 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a2800640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a415bc40 0x7f21a4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f2199400640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21a40cf930 0x7f21a4070800 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b410 tx=0x7f218c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f219aa00640 1 -- 172.21.15.87:0/366836344 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c015030 con 0x7f21a40cf930 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f219aa00640 1 -- 172.21.15.87:0/366836344 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c004020 con 0x7f21a40cf930 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a414f700 con 0x7f21a40cf930 2024-04-25T10:21:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f219aa00640 1 -- 172.21.15.87:0/366836344 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c004450 con 0x7f21a40cf930 2024-04-25T10:21:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a414fcc0 con 0x7f21a40cf930 2024-04-25T10:21:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a404ef30 con 0x7f21a40cf930 2024-04-25T10:21:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.366+0000 7f219aa00640 1 -- 172.21.15.87:0/366836344 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f218c018070 con 0x7f21a40cf930 2024-04-25T10:21:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.370+0000 7f219aa00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2180066f90 0x7f2180069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.370+0000 7f219aa00640 1 -- 172.21.15.87:0/366836344 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f218c084800 con 0x7f21a40cf930 2024-04-25T10:21:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.370+0000 7f21a1e00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2180066f90 0x7f2180069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.370+0000 7f21a1e00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2180066f90 0x7f2180069450 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f219400f500 tx=0x7f2194007770 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.374+0000 7f219aa00640 1 -- 172.21.15.87:0/366836344 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f218c055b50 con 0x7f21a40cf930 2024-04-25T10:21:40.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.526+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f21a414fe90 con 0x7f21a40cf930 2024-04-25T10:21:40.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.526+0000 7f219aa00640 1 -- 172.21.15.87:0/366836344 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+65 (secure 0 0 0) 0x7f218c055450 con 0x7f21a40cf930 2024-04-25T10:21:40.531 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:40.531 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:21:40.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:21:40.532 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:21:40.532 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:21:40.532 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:21:40.532 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:21:40.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2180066f90 msgr2=0x7f2180069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2180066f90 0x7f2180069450 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f219400f500 tx=0x7f2194007770 comp rx=0 tx=0).stop 2024-04-25T10:21:40.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21a40cf930 msgr2=0x7f21a4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21a40cf930 0x7f21a4070800 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b410 tx=0x7f218c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 shutdown_connections 2024-04-25T10:21:40.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2180066f90 0x7f2180069450 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a415bc40 0x7f21a4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21a40cf930 0x7f21a4070800 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 --2- 172.21.15.87:0/366836344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a40cda10 0x7f21a414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 >> 172.21.15.87:0/366836344 conn(0x7f21a4066b70 msgr2=0x7f21a4066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:40.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 shutdown_connections 2024-04-25T10:21:40.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.530+0000 7f21a8c00640 1 -- 172.21.15.87:0/366836344 wait complete. 2024-04-25T10:21:40.553 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:21:40.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:21:40.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.686+0000 7fcd5e800640 1 Processor -- start 2024-04-25T10:21:40.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.686+0000 7fcd5e800640 1 -- start start 2024-04-25T10:21:40.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd5e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd5e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd5815bc00 0x7fcd580718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd5e800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd58071f80 con 0x7fcd580cf040 2024-04-25T10:21:40.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd5e800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd58072260 con 0x7fcd5815bc00 2024-04-25T10:21:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd5e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580b3700 con 0x7fcd580d00f0 2024-04-25T10:21:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd57e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd5815bc00 0x7fcd580718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd57400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcd580cf040 0x7fcd580cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48990/0 (socket says 172.21.15.87:48990) 2024-04-25T10:21:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd57400640 1 -- 172.21.15.87:0/2751457203 learned_addr learned my addr 172.21.15.87:0/2751457203 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd56a00640 1 --2- 172.21.15.87:0/2751457203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd57e00640 1 -- 172.21.15.87:0/2751457203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 msgr2=0x7fcd580713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd57e00640 1 --2- 172.21.15.87:0/2751457203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd57e00640 1 -- 172.21.15.87:0/2751457203 >> v1:172.21.15.87:6789/0 conn(0x7fcd580cf040 legacy=0x7fcd580cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd56000640 1 -- 172.21.15.87:0/2751457203 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394205042 0 0) 0x7fcd58071f80 con 0x7fcd580cf040 2024-04-25T10:21:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.690+0000 7fcd57e00640 1 -- 172.21.15.87:0/2751457203 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5815b120 con 0x7fcd5815bc00 2024-04-25T10:21:40.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd57e00640 1 --2- 172.21.15.87:0/2751457203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd580718e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd5001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171297b0aeffc57c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd56000640 1 -- 172.21.15.87:0/2751457203 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b2070 con 0x7fcd5815bc00 2024-04-25T10:21:40.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd56000640 1 -- 172.21.15.87:0/2751457203 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500ad640 con 0x7fcd5815bc00 2024-04-25T10:21:40.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd56000640 1 -- 172.21.15.87:0/2751457203 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ada10 con 0x7fcd5815bc00 2024-04-25T10:21:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 -- 172.21.15.87:0/2751457203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 msgr2=0x7fcd580718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2751457203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd580718e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd5001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 -- 172.21.15.87:0/2751457203 shutdown_connections 2024-04-25T10:21:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2751457203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd580718e0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2751457203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 -- 172.21.15.87:0/2751457203 >> 172.21.15.87:0/2751457203 conn(0x7fcd58066b70 msgr2=0x7fcd58066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:40.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 -- 172.21.15.87:0/2751457203 shutdown_connections 2024-04-25T10:21:40.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 -- 172.21.15.87:0/2751457203 wait complete. 2024-04-25T10:21:40.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 Processor -- start 2024-04-25T10:21:40.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.694+0000 7fcd5e800640 1 -- start start 2024-04-25T10:21:40.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd580cf040 0x7fcd58147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd5814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580aec90 con 0x7fcd5815bc00 2024-04-25T10:21:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580cf630 con 0x7fcd580cf040 2024-04-25T10:21:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580aee10 con 0x7fcd580d00f0 2024-04-25T10:21:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd5814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd5814f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35368/0 (socket says 172.21.15.87:35368) 2024-04-25T10:21:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 -- 172.21.15.87:0/2912500336 learned_addr learned my addr 172.21.15.87:0/2912500336 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd56a00640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57400640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd580cf040 0x7fcd58147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 -- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 msgr2=0x7fcd580ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 -- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd580cf040 msgr2=0x7fcd58147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd580cf040 0x7fcd58147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 -- 172.21.15.87:0/2912500336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5814b220 con 0x7fcd5815bc00 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57400640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd580cf040 0x7fcd58147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd57e00640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd5814f510 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd50019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd4f400640 1 -- 172.21.15.87:0/2912500336 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ba020 con 0x7fcd5815bc00 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd4f400640 1 -- 172.21.15.87:0/2912500336 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500b6410 con 0x7fcd5815bc00 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd4f400640 1 -- 172.21.15.87:0/2912500336 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b2040 con 0x7fcd5815bc00 2024-04-25T10:21:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5814b500 con 0x7fcd5815bc00 2024-04-25T10:21:40.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd58148c60 con 0x7fcd5815bc00 2024-04-25T10:21:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5804ef30 con 0x7fcd5815bc00 2024-04-25T10:21:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.698+0000 7fcd4f400640 1 -- 172.21.15.87:0/2912500336 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd500bd020 con 0x7fcd5815bc00 2024-04-25T10:21:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.702+0000 7fcd4f400640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd38066f60 0x7fcd38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.702+0000 7fcd4f400640 1 -- 172.21.15.87:0/2912500336 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd5011eb10 con 0x7fcd5815bc00 2024-04-25T10:21:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.702+0000 7fcd57400640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd38066f60 0x7fcd38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.702+0000 7fcd57400640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd38066f60 0x7fcd38069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcd48007fd0 tx=0x7fcd48005b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.706+0000 7fcd4f400640 1 -- 172.21.15.87:0/2912500336 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd500c4030 con 0x7fcd5815bc00 2024-04-25T10:21:40.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:40.862+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcd580cdf60 con 0x7fcd5815bc00 2024-04-25T10:21:41.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.158+0000 7fcd4f400640 1 -- 172.21.15.87:0/2912500336 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcd50016090 con 0x7fcd5815bc00 2024-04-25T10:21:41.162 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd38066f60 msgr2=0x7fcd38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd38066f60 0x7fcd38069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcd48007fd0 tx=0x7fcd48005b80 comp rx=0 tx=0).stop 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 msgr2=0x7fcd5814f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd5814f510 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd50019240 comp rx=0 tx=0).stop 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 shutdown_connections 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd38066f60 0x7fcd38069420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd5815bc00 0x7fcd5814f510 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd580d00f0 0x7fcd580ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 --2- 172.21.15.87:0/2912500336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd580cf040 0x7fcd58147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 >> 172.21.15.87:0/2912500336 conn(0x7fcd58066b70 msgr2=0x7fcd5815c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:41.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 shutdown_connections 2024-04-25T10:21:41.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.162+0000 7fcd5e800640 1 -- 172.21.15.87:0/2912500336 wait complete. 2024-04-25T10:21:41.188 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:21:41.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.326+0000 7fcc61a00640 1 Processor -- start 2024-04-25T10:21:41.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.326+0000 7fcc61a00640 1 -- start start 2024-04-25T10:21:41.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc61a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcc5c15bbb0 0x7fcc5c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc61a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c06ca90 con 0x7fcc5c0cf960 2024-04-25T10:21:41.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc61a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c06cd70 con 0x7fcc5c0b4660 2024-04-25T10:21:41.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc61a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c06d050 con 0x7fcc5c15bbb0 2024-04-25T10:21:41.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc5aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcc5c15bbb0 0x7fcc5c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc5b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcc5c0cf960 0x7fcc5c0b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48998/0 (socket says 172.21.15.87:48998) 2024-04-25T10:21:41.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc5b400640 1 -- 172.21.15.87:0/2335731435 learned_addr learned my addr 172.21.15.87:0/2335731435 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:41.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc5aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcc5c15bbb0 0x7fcc5c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35378/0 (socket says 172.21.15.87:35378) 2024-04-25T10:21:41.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2335731435 >> v1:172.21.15.169:6789/0 conn(0x7fcc5c0b4660 legacy=0x7fcc5c06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:41.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2335731435 >> v1:172.21.15.87:6789/0 conn(0x7fcc5c0cf960 legacy=0x7fcc5c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc5aa00640 1 -- 172.21.15.87:0/2335731435 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0b2880 con 0x7fcc5c15bbb0 2024-04-25T10:21:41.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.330+0000 7fcc59600640 1 -- 172.21.15.87:0/2335731435 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737913568 0 0) 0x7fcc5c06ca90 con 0x7fcc5c0cf960 2024-04-25T10:21:41.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc5aa00640 1 --2- 172.21.15.87:0/2335731435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c15bbb0 0x7fcc5c0cda80 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0097c0 tx=0x7fcc4c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e09d5599706d59c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc59600640 1 -- 172.21.15.87:0/2335731435 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0b0070 con 0x7fcc5c15bbb0 2024-04-25T10:21:41.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc59600640 1 -- 172.21.15.87:0/2335731435 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c0ab600 con 0x7fcc5c15bbb0 2024-04-25T10:21:41.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc59600640 1 -- 172.21.15.87:0/2335731435 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0ab9d0 con 0x7fcc5c15bbb0 2024-04-25T10:21:41.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 -- 172.21.15.87:0/2335731435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c15bbb0 msgr2=0x7fcc5c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2335731435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c15bbb0 0x7fcc5c0cda80 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0097c0 tx=0x7fcc4c01c460 comp rx=0 tx=0).stop 2024-04-25T10:21:41.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 -- 172.21.15.87:0/2335731435 shutdown_connections 2024-04-25T10:21:41.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2335731435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c15bbb0 0x7fcc5c0cda80 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 -- 172.21.15.87:0/2335731435 >> 172.21.15.87:0/2335731435 conn(0x7fcc5c066b70 msgr2=0x7fcc5c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:41.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 -- 172.21.15.87:0/2335731435 shutdown_connections 2024-04-25T10:21:41.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 -- 172.21.15.87:0/2335731435 wait complete. 2024-04-25T10:21:41.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 Processor -- start 2024-04-25T10:21:41.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.334+0000 7fcc61a00640 1 -- start start 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c0b4660 0x7fcc5c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c0cf960 0x7fcc5c153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c15bbb0 0x7fcc5c156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0ce140 con 0x7fcc5c0cf960 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0cdfc0 con 0x7fcc5c0b4660 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c16a180 con 0x7fcc5c15bbb0 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c0cf960 0x7fcc5c153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c0cf960 0x7fcc5c153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35388/0 (socket says 172.21.15.87:35388) 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 -- 172.21.15.87:0/2036220575 learned_addr learned my addr 172.21.15.87:0/2036220575 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5aa00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c0b4660 0x7fcc5c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5b400640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c15bbb0 0x7fcc5c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 -- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c15bbb0 msgr2=0x7fcc5c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c15bbb0 0x7fcc5c156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 -- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c0b4660 msgr2=0x7fcc5c151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c0b4660 0x7fcc5c151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 -- 172.21.15.87:0/2036220575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0cba70 con 0x7fcc5c0cf960 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5b400640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c15bbb0 0x7fcc5c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc5a000640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c0cf960 0x7fcc5c153600 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800b3e0 tx=0x7fcc4800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc52a00640 1 -- 172.21.15.87:0/2036220575 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48015020 con 0x7fcc5c0cf960 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc52a00640 1 -- 172.21.15.87:0/2036220575 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48011830 con 0x7fcc5c0cf960 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc52a00640 1 -- 172.21.15.87:0/2036220575 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48011c60 con 0x7fcc5c0cf960 2024-04-25T10:21:41.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c15d6b0 con 0x7fcc5c0cf960 2024-04-25T10:21:41.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c15dc70 con 0x7fcc5c0cf960 2024-04-25T10:21:41.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c04ef30 con 0x7fcc5c0cf960 2024-04-25T10:21:41.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.338+0000 7fcc52a00640 1 -- 172.21.15.87:0/2036220575 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc48018020 con 0x7fcc5c0cf960 2024-04-25T10:21:41.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.342+0000 7fcc52a00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc3c066f60 0x7fcc3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.342+0000 7fcc52a00640 1 -- 172.21.15.87:0/2036220575 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcc48084fe0 con 0x7fcc5c0cf960 2024-04-25T10:21:41.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.342+0000 7fcc5aa00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc3c066f60 0x7fcc3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.342+0000 7fcc5aa00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc3c066f60 0x7fcc3c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0ad000 tx=0x7fcc4c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.342+0000 7fcc52a00640 1 -- 172.21.15.87:0/2036220575 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc4801f030 con 0x7fcc5c0cf960 2024-04-25T10:21:41.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:41.498+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcc5c0b0a60 con 0x7fcc5c0cf960 2024-04-25T10:21:42.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.170+0000 7fcc52a00640 1 -- 172.21.15.87:0/2036220575 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcc480560a0 con 0x7fcc5c0cf960 2024-04-25T10:21:42.174 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:21:42.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc3c066f60 msgr2=0x7fcc3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc3c066f60 0x7fcc3c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0ad000 tx=0x7fcc4c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:21:42.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c0cf960 msgr2=0x7fcc5c153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c0cf960 0x7fcc5c153600 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800b3e0 tx=0x7fcc4800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 shutdown_connections 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcc3c066f60 0x7fcc3c069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c15bbb0 0x7fcc5c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc5c0cf960 0x7fcc5c153600 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 --2- 172.21.15.87:0/2036220575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c0b4660 0x7fcc5c151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 >> 172.21.15.87:0/2036220575 conn(0x7fcc5c066b70 msgr2=0x7fcc5c0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 shutdown_connections 2024-04-25T10:21:42.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.174+0000 7fcc61a00640 1 -- 172.21.15.87:0/2036220575 wait complete. 2024-04-25T10:21:42.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:21:42.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa69ce00640 1 Processor -- start 2024-04-25T10:21:42.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa69ce00640 1 -- start start 2024-04-25T10:21:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa69ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa698071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa69ce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa698159920 con 0x7fa698159540 2024-04-25T10:21:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa69ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69815c2f0 con 0x7fa698072020 2024-04-25T10:21:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa69ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69815c5d0 con 0x7fa6980aeda0 2024-04-25T10:21:42.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa695e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa698159540 0x7fa6980ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:49008/0 (socket says 172.21.15.87:49008) 2024-04-25T10:21:42.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa695e00640 1 -- 172.21.15.87:0/764710458 learned_addr learned my addr 172.21.15.87:0/764710458 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:42.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.354+0000 7fa68d400640 1 --2- 172.21.15.87:0/764710458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa698071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa695400640 1 -- 172.21.15.87:0/764710458 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015167605 0 0) 0x7fa698159920 con 0x7fa698159540 2024-04-25T10:21:42.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa695400640 1 -- 172.21.15.87:0/764710458 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa67c003530 con 0x7fa698159540 2024-04-25T10:21:42.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa68d400640 1 -- 172.21.15.87:0/764710458 >> v1:172.21.15.169:6789/0 conn(0x7fa698072020 legacy=0x7fa698070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa695400640 1 -- 172.21.15.87:0/764710458 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106041781 0 0) 0x7fa69815c2f0 con 0x7fa698072020 2024-04-25T10:21:42.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa68d400640 1 -- 172.21.15.87:0/764710458 >> v1:172.21.15.87:6789/0 conn(0x7fa698159540 legacy=0x7fa6980ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa68d400640 1 -- 172.21.15.87:0/764710458 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69815c8b0 con 0x7fa6980aeda0 2024-04-25T10:21:42.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa68d400640 1 --2- 172.21.15.87:0/764710458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa698071ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6800097f0 tx=0x7fa68001c2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488a72fdd0de46c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa695400640 1 -- 172.21.15.87:0/764710458 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6800b0070 con 0x7fa6980aeda0 2024-04-25T10:21:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa695400640 1 -- 172.21.15.87:0/764710458 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6800ab400 con 0x7fa6980aeda0 2024-04-25T10:21:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.358+0000 7fa695400640 1 -- 172.21.15.87:0/764710458 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688001a00 con 0x7fa6980aeda0 2024-04-25T10:21:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- 172.21.15.87:0/764710458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 msgr2=0x7fa698071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 --2- 172.21.15.87:0/764710458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa698071ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6800097f0 tx=0x7fa68001c2f0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- 172.21.15.87:0/764710458 shutdown_connections 2024-04-25T10:21:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 --2- 172.21.15.87:0/764710458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa698071ae0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- 172.21.15.87:0/764710458 >> 172.21.15.87:0/764710458 conn(0x7fa6980669a0 msgr2=0x7fa698066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- 172.21.15.87:0/764710458 shutdown_connections 2024-04-25T10:21:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- 172.21.15.87:0/764710458 wait complete. 2024-04-25T10:21:42.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 Processor -- start 2024-04-25T10:21:42.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- start start 2024-04-25T10:21:42.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa698072020 0x7fa698156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa6980af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa698159540 0x7fa69813ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698169940 con 0x7fa698159540 2024-04-25T10:21:42.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698072430 con 0x7fa698072020 2024-04-25T10:21:42.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa69ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa698169ac0 con 0x7fa6980aeda0 2024-04-25T10:21:42.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa698159540 0x7fa69813ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa698159540 0x7fa69813ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35394/0 (socket says 172.21.15.87:35394) 2024-04-25T10:21:42.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 -- 172.21.15.87:0/2534314684 learned_addr learned my addr 172.21.15.87:0/2534314684 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:42.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa68d400640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa6980af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa695e00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa698072020 0x7fa698156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 -- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 msgr2=0x7fa6980af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa6980af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 -- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa698072020 msgr2=0x7fa698156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa698072020 0x7fa698156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa696800640 1 -- 172.21.15.87:0/2534314684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6981573e0 con 0x7fa698159540 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.362+0000 7fa68d400640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa6980af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa696800640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa698159540 0x7fa69813ed10 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c410 tx=0x7fa69000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa68ea00640 1 -- 172.21.15.87:0/2534314684 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690017070 con 0x7fa698159540 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa68ea00640 1 -- 172.21.15.87:0/2534314684 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa690003600 con 0x7fa698159540 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6981576c0 con 0x7fa698159540 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa68ea00640 1 -- 172.21.15.87:0/2534314684 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6900039f0 con 0x7fa698159540 2024-04-25T10:21:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa698157c80 con 0x7fa698159540 2024-04-25T10:21:42.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69804ef30 con 0x7fa698159540 2024-04-25T10:21:42.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa68ea00640 1 -- 172.21.15.87:0/2534314684 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa69001a070 con 0x7fa698159540 2024-04-25T10:21:42.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa68ea00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.366+0000 7fa68ea00640 1 -- 172.21.15.87:0/2534314684 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa690084670 con 0x7fa698159540 2024-04-25T10:21:42.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.370+0000 7fa695e00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.370+0000 7fa695e00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa6880012f0 tx=0x7fa688005b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:42.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.370+0000 7fa68ea00640 1 -- 172.21.15.87:0/2534314684 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa690021020 con 0x7fa698159540 2024-04-25T10:21:42.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:42.558+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa69806d7f0 con 0x7fa698159540 2024-04-25T10:21:43.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.170+0000 7fa68ea00640 1 -- 172.21.15.87:0/2534314684 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v85) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa690055850 con 0x7fa698159540 2024-04-25T10:21:43.175 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 18 and data pool 19 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa674066f60 msgr2=0x7fa674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa6880012f0 tx=0x7fa688005b80 comp rx=0 tx=0).stop 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa698159540 msgr2=0x7fa69813ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa698159540 0x7fa69813ed10 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c410 tx=0x7fa69000c810 comp rx=0 tx=0).stop 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 shutdown_connections 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa698159540 0x7fa69813ed10 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6980aeda0 0x7fa6980af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 --2- 172.21.15.87:0/2534314684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa698072020 0x7fa698156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 >> 172.21.15.87:0/2534314684 conn(0x7fa6980669a0 msgr2=0x7fa6980ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 shutdown_connections 2024-04-25T10:21:43.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.174+0000 7fa69ce00640 1 -- 172.21.15.87:0/2534314684 wait complete. 2024-04-25T10:21:43.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:43.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feed2800640 1 Processor -- start 2024-04-25T10:21:43.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feed2800640 1 -- start start 2024-04-25T10:21:43.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feed2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feed2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feecc0cf930 0x7feecc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feed2800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feecc0b2770 con 0x7feecc0cda10 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feed2800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feecc06c670 con 0x7feecc0cf930 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feed2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc06c950 con 0x7feecc15bc40 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feec2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7feecc0cda10 0x7feecc0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:49018/0 (socket says 172.21.15.87:49018) 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feec2a00640 1 -- 172.21.15.87:0/1287087394 learned_addr learned my addr 172.21.15.87:0/1287087394 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.354+0000 7feecbe00640 1 --2- 172.21.15.87:0/1287087394 >> v2:172.21.15.87:3300/0 conn(0x7feecc0cf930 0x7feecc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecb400640 1 --2- 172.21.15.87:0/1287087394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecaa00640 1 -- 172.21.15.87:0/1287087394 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721272439 0 0) 0x7feecc0b2770 con 0x7feecc0cda10 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecaa00640 1 -- 172.21.15.87:0/1287087394 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb0003530 con 0x7feecc0cda10 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecbe00640 1 -- 172.21.15.87:0/1287087394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 msgr2=0x7feecc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecbe00640 1 --2- 172.21.15.87:0/1287087394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecbe00640 1 -- 172.21.15.87:0/1287087394 >> v1:172.21.15.87:6789/0 conn(0x7feecc0cda10 legacy=0x7feecc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecbe00640 1 -- 172.21.15.87:0/1287087394 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc06cc30 con 0x7feecc0cf930 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecb400640 1 --2- 172.21.15.87:0/1287087394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecbe00640 1 --2- 172.21.15.87:0/1287087394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0cfde0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7feec40097c0 tx=0x7feec401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718e0aa40282bc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecaa00640 1 -- 172.21.15.87:0/1287087394 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40b0070 con 0x7feecc0cf930 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecaa00640 1 -- 172.21.15.87:0/1287087394 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40ab600 con 0x7feecc0cf930 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feecaa00640 1 -- 172.21.15.87:0/1287087394 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40ab9d0 con 0x7feecc0cf930 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feed2800640 1 -- 172.21.15.87:0/1287087394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 msgr2=0x7feecc0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.358+0000 7feed2800640 1 --2- 172.21.15.87:0/1287087394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0cfde0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7feec40097c0 tx=0x7feec401c410 comp rx=0 tx=0).stop 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- 172.21.15.87:0/1287087394 shutdown_connections 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 --2- 172.21.15.87:0/1287087394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0cfde0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 --2- 172.21.15.87:0/1287087394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- 172.21.15.87:0/1287087394 >> 172.21.15.87:0/1287087394 conn(0x7feecc066b70 msgr2=0x7feecc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:43.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- 172.21.15.87:0/1287087394 shutdown_connections 2024-04-25T10:21:43.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- 172.21.15.87:0/1287087394 wait complete. 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 Processor -- start 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- start start 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feecc0cda10 0x7feecc1fad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc14b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc0b3850 con 0x7feecc0cf930 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc0b36d0 con 0x7feecc0cda10 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feed2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc0b39d0 con 0x7feecc15bc40 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35404/0 (socket says 172.21.15.87:35404) 2024-04-25T10:21:43.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feecb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feecc0cda10 0x7feecc1fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 -- 172.21.15.87:0/3275610569 learned_addr learned my addr 172.21.15.87:0/3275610569 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feecbe00640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc14b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 -- 172.21.15.87:0/3275610569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 msgr2=0x7feecc14b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc14b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 -- 172.21.15.87:0/3275610569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feecc0cda10 msgr2=0x7feecc1fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feecc0cda10 0x7feecc1fad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 -- 172.21.15.87:0/3275610569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0b0b40 con 0x7feecc0cf930 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.362+0000 7feec2a00640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0b0600 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3f0 tx=0x7feeb400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feec3e00640 1 -- 172.21.15.87:0/3275610569 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4017070 con 0x7feecc0cf930 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feec3e00640 1 -- 172.21.15.87:0/3275610569 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4005cf0 con 0x7feecc0cf930 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feec3e00640 1 -- 172.21.15.87:0/3275610569 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40072f0 con 0x7feecc0cf930 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feecc0b0de0 con 0x7feecc0cf930 2024-04-25T10:21:43.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feecc0b13a0 con 0x7feecc0cf930 2024-04-25T10:21:43.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feecc04ef30 con 0x7feecc0cf930 2024-04-25T10:21:43.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feec3e00640 1 -- 172.21.15.87:0/3275610569 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feeb401a070 con 0x7feecc0cf930 2024-04-25T10:21:43.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feec3e00640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feea8066f90 0x7feea8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.366+0000 7feec3e00640 1 -- 172.21.15.87:0/3275610569 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeb4084430 con 0x7feecc0cf930 2024-04-25T10:21:43.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.370+0000 7feecb400640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feea8066f90 0x7feea8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.370+0000 7feecb400640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feea8066f90 0x7feea8069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feebc00f500 tx=0x7feebc007770 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.374+0000 7feec3e00640 1 -- 172.21.15.87:0/3275610569 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feeb4054d50 con 0x7feecc0cf930 2024-04-25T10:21:43.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.526+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feecc151ca0 con 0x7feecc0cf930 2024-04-25T10:21:43.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.526+0000 7feec3e00640 1 -- 172.21.15.87:0/3275610569 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16680 (secure 0 0 0) 0x7feeb4058df0 con 0x7feecc0cf930 2024-04-25T10:21:43.530 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:43.530 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":123,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:42.562711+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:40.912468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:41.547768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:43.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feea8066f90 msgr2=0x7feea8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feea8066f90 0x7feea8069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feebc00f500 tx=0x7feebc007770 comp rx=0 tx=0).stop 2024-04-25T10:21:43.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 msgr2=0x7feecc0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0b0600 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3f0 tx=0x7feeb400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 shutdown_connections 2024-04-25T10:21:43.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feea8066f90 0x7feea8069450 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feecc15bc40 0x7feecc14b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feecc0cf930 0x7feecc0b0600 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 --2- 172.21.15.87:0/3275610569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feecc0cda10 0x7feecc1fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 >> 172.21.15.87:0/3275610569 conn(0x7feecc066b70 msgr2=0x7feecc159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:43.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 shutdown_connections 2024-04-25T10:21:43.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.530+0000 7feed2800640 1 -- 172.21.15.87:0/3275610569 wait complete. 2024-04-25T10:21:43.552 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:43.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.682+0000 7f3614800640 1 Processor -- start 2024-04-25T10:21:43.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3614800640 1 -- start start 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3614800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cda10 0x7f360c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3614800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f360c0cf930 0x7f360c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3614800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c0b2770 con 0x7f360c15bc40 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3614800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c06c670 con 0x7f360c0cf930 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3614800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c06c950 con 0x7f360c0cda10 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3611400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cda10 0x7f360c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3611400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cda10 0x7f360c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35608/0 (socket says 172.21.15.87:35608) 2024-04-25T10:21:43.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3611400640 1 -- 172.21.15.87:0/1848973606 learned_addr learned my addr 172.21.15.87:0/1848973606 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:43.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3612800640 1 --2- 172.21.15.87:0/1848973606 >> v2:172.21.15.169:3300/0 conn(0x7f360c0cf930 0x7f360c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3611400640 1 -- 172.21.15.87:0/1848973606 >> v2:172.21.15.169:3300/0 conn(0x7f360c0cf930 msgr2=0x7f360c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:43.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3611400640 1 --2- 172.21.15.87:0/1848973606 >> v2:172.21.15.169:3300/0 conn(0x7f360c0cf930 0x7f360c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3611400640 1 -- 172.21.15.87:0/1848973606 >> v1:172.21.15.169:6789/0 conn(0x7f360c15bc40 legacy=0x7f360c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3611400640 1 -- 172.21.15.87:0/1848973606 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c06cc30 con 0x7f360c0cda10 2024-04-25T10:21:43.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.686+0000 7f3612800640 1 --2- 172.21.15.87:0/1848973606 >> v2:172.21.15.169:3300/0 conn(0x7f360c0cf930 0x7f360c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:43.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3611400640 1 --2- 172.21.15.87:0/1848973606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cda10 0x7f360c06c090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009870 tx=0x7f35fc01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a2704ec870b609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3610a00640 1 -- 172.21.15.87:0/1848973606 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0b0070 con 0x7f360c0cda10 2024-04-25T10:21:43.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3610a00640 1 -- 172.21.15.87:0/1848973606 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35fc0ab600 con 0x7f360c0cda10 2024-04-25T10:21:43.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3610a00640 1 -- 172.21.15.87:0/1848973606 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0ab9d0 con 0x7f360c0cda10 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 -- 172.21.15.87:0/1848973606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cda10 msgr2=0x7f360c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 --2- 172.21.15.87:0/1848973606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cda10 0x7f360c06c090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009870 tx=0x7f35fc01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 -- 172.21.15.87:0/1848973606 shutdown_connections 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 --2- 172.21.15.87:0/1848973606 >> v2:172.21.15.169:3300/0 conn(0x7f360c0cf930 0x7f360c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 --2- 172.21.15.87:0/1848973606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cda10 0x7f360c06c090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 -- 172.21.15.87:0/1848973606 >> 172.21.15.87:0/1848973606 conn(0x7f360c066b70 msgr2=0x7f360c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 -- 172.21.15.87:0/1848973606 shutdown_connections 2024-04-25T10:21:43.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.690+0000 7f3614800640 1 -- 172.21.15.87:0/1848973606 wait complete. 2024-04-25T10:21:43.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 Processor -- start 2024-04-25T10:21:43.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 -- start start 2024-04-25T10:21:43.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c0cda10 0x7f360c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cf930 0x7f360c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f360c15bc40 0x7f360c14c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0b3850 con 0x7f360c0cda10 2024-04-25T10:21:43.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0b36d0 con 0x7f360c15bc40 2024-04-25T10:21:43.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0b39d0 con 0x7f360c0cf930 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c0cda10 0x7f360c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c0cda10 0x7f360c15c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35414/0 (socket says 172.21.15.87:35414) 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 -- 172.21.15.87:0/4123239931 learned_addr learned my addr 172.21.15.87:0/4123239931 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3612800640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f360c15bc40 0x7f360c14c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611400640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cf930 0x7f360c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 -- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cf930 msgr2=0x7f360c0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cf930 0x7f360c0b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 -- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f360c15bc40 msgr2=0x7f360c14c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f360c15bc40 0x7f360c14c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 -- 172.21.15.87:0/4123239931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c15c9e0 con 0x7f360c0cda10 2024-04-25T10:21:43.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3611e00640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c0cda10 0x7f360c15c190 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f360000c370 tx=0x7f360000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3606000640 1 -- 172.21.15.87:0/4123239931 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000052d0 con 0x7f360c0cda10 2024-04-25T10:21:43.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3606000640 1 -- 172.21.15.87:0/4123239931 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3600005470 con 0x7f360c0cda10 2024-04-25T10:21:43.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3606000640 1 -- 172.21.15.87:0/4123239931 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600007260 con 0x7f360c0cda10 2024-04-25T10:21:43.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c15ccc0 con 0x7f360c0cda10 2024-04-25T10:21:43.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c15d560 con 0x7f360c0cda10 2024-04-25T10:21:43.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.694+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c04ef30 con 0x7f360c0cda10 2024-04-25T10:21:43.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.698+0000 7f3606000640 1 -- 172.21.15.87:0/4123239931 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3600007400 con 0x7f360c0cda10 2024-04-25T10:21:43.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.698+0000 7f3606000640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35f0066f90 0x7f35f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.698+0000 7f3606000640 1 -- 172.21.15.87:0/4123239931 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3600084260 con 0x7f360c0cda10 2024-04-25T10:21:43.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.698+0000 7f3611400640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35f0066f90 0x7f35f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.698+0000 7f3611400640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35f0066f90 0x7f35f0069450 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ad000 tx=0x7f35fc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.702+0000 7f3606000640 1 -- 172.21.15.87:0/4123239931 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3600054cb0 con 0x7f360c0cda10 2024-04-25T10:21:43.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.858+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360c0af1c0 con 0x7f360c0cda10 2024-04-25T10:21:43.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.858+0000 7f3606000640 1 -- 172.21.15.87:0/4123239931 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f3600058d50 con 0x7f360c0cda10 2024-04-25T10:21:43.863 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:43.863 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":123,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:42.562711+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:40.912468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:41.547768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:43.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35f0066f90 msgr2=0x7f35f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35f0066f90 0x7f35f0069450 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ad000 tx=0x7f35fc008040 comp rx=0 tx=0).stop 2024-04-25T10:21:43.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c0cda10 msgr2=0x7f360c15c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c0cda10 0x7f360c15c190 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f360000c370 tx=0x7f360000c770 comp rx=0 tx=0).stop 2024-04-25T10:21:43.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 shutdown_connections 2024-04-25T10:21:43.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35f0066f90 0x7f35f0069450 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f360c15bc40 0x7f360c14c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f360c0cf930 0x7f360c0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 --2- 172.21.15.87:0/4123239931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c0cda10 0x7f360c15c190 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 >> 172.21.15.87:0/4123239931 conn(0x7f360c066b70 msgr2=0x7f360c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:43.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 shutdown_connections 2024-04-25T10:21:43.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:43.862+0000 7f3614800640 1 -- 172.21.15.87:0/4123239931 wait complete. 2024-04-25T10:21:43.884 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:21:44.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.018+0000 7ffa5f400640 1 Processor -- start 2024-04-25T10:21:44.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.018+0000 7ffa5f400640 1 -- start start 2024-04-25T10:21:44.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.018+0000 7ffa5f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffa580d0d00 0x7ffa580ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5f400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580cdab0 con 0x7ffa580cc200 2024-04-25T10:21:44.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580cd210 con 0x7ffa5806cf40 2024-04-25T10:21:44.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5f400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa580cd4f0 con 0x7ffa580d0d00 2024-04-25T10:21:44.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5ca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffa580d0d00 0x7ffa580ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa4fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ffa580cc200 0x7ffa58162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:49022/0 (socket says 172.21.15.87:49022) 2024-04-25T10:21:44.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa4fe00640 1 -- 172.21.15.87:0/1368034593 learned_addr learned my addr 172.21.15.87:0/1368034593 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:44.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5ca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffa580d0d00 0x7ffa580ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35420/0 (socket says 172.21.15.87:35420) 2024-04-25T10:21:44.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5ca00640 1 -- 172.21.15.87:0/1368034593 >> v1:172.21.15.169:6789/0 conn(0x7ffa5806cf40 legacy=0x7ffa5815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5ca00640 1 -- 172.21.15.87:0/1368034593 >> v1:172.21.15.87:6789/0 conn(0x7ffa580cc200 legacy=0x7ffa58162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa57e00640 1 -- 172.21.15.87:0/1368034593 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456449256 0 0) 0x7ffa580cdab0 con 0x7ffa580cc200 2024-04-25T10:21:44.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5ca00640 1 -- 172.21.15.87:0/1368034593 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580cd7d0 con 0x7ffa580d0d00 2024-04-25T10:21:44.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa5ca00640 1 --2- 172.21.15.87:0/1368034593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa580d0d00 0x7ffa580ccc30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ffa480097c0 tx=0x7ffa4801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acff7bc2be36f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa57e00640 1 -- 172.21.15.87:0/1368034593 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480b0070 con 0x7ffa580d0d00 2024-04-25T10:21:44.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa57e00640 1 -- 172.21.15.87:0/1368034593 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa480ab600 con 0x7ffa580d0d00 2024-04-25T10:21:44.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.022+0000 7ffa57e00640 1 -- 172.21.15.87:0/1368034593 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480ab9d0 con 0x7ffa580d0d00 2024-04-25T10:21:44.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- 172.21.15.87:0/1368034593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa580d0d00 msgr2=0x7ffa580ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1368034593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa580d0d00 0x7ffa580ccc30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ffa480097c0 tx=0x7ffa4801c410 comp rx=0 tx=0).stop 2024-04-25T10:21:44.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- 172.21.15.87:0/1368034593 shutdown_connections 2024-04-25T10:21:44.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1368034593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa580d0d00 0x7ffa580ccc30 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- 172.21.15.87:0/1368034593 >> 172.21.15.87:0/1368034593 conn(0x7ffa58072290 msgr2=0x7ffa580726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:44.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- 172.21.15.87:0/1368034593 shutdown_connections 2024-04-25T10:21:44.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- 172.21.15.87:0/1368034593 wait complete. 2024-04-25T10:21:44.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 Processor -- start 2024-04-25T10:21:44.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- start start 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa5806cf40 0x7ffa58156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa580cc200 0x7ffa580c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa580d0d00 0x7ffa58158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58162a10 con 0x7ffa5806cf40 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58162890 con 0x7ffa580cc200 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5816c690 con 0x7ffa580d0d00 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa5806cf40 0x7ffa58156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa5806cf40 0x7ffa58156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35428/0 (socket says 172.21.15.87:35428) 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 -- 172.21.15.87:0/1898556407 learned_addr learned my addr 172.21.15.87:0/1898556407 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:44.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa4fe00640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa580cc200 0x7ffa580c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 -- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa580d0d00 msgr2=0x7ffa58158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5d400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa580d0d00 0x7ffa58158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa580d0d00 0x7ffa58158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 -- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa580cc200 msgr2=0x7ffa580c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa580cc200 0x7ffa580c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5ca00640 1 -- 172.21.15.87:0/1898556407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580c3c80 con 0x7ffa5806cf40 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa4fe00640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa580cc200 0x7ffa580c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.026+0000 7ffa5d400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa580d0d00 0x7ffa58158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa5ca00640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa5806cf40 0x7ffa58156220 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ffa480ae040 tx=0x7ffa4801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa55600640 1 -- 172.21.15.87:0/1898556407 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480b0040 con 0x7ffa5806cf40 2024-04-25T10:21:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa55600640 1 -- 172.21.15.87:0/1898556407 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa48004030 con 0x7ffa5806cf40 2024-04-25T10:21:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa580c3f60 con 0x7ffa5806cf40 2024-04-25T10:21:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa55600640 1 -- 172.21.15.87:0/1898556407 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480043e0 con 0x7ffa5806cf40 2024-04-25T10:21:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa580c4520 con 0x7ffa5806cf40 2024-04-25T10:21:44.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5804ef30 con 0x7ffa5806cf40 2024-04-25T10:21:44.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa55600640 1 -- 172.21.15.87:0/1898556407 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa48003620 con 0x7ffa5806cf40 2024-04-25T10:21:44.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa55600640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa34066f60 0x7ffa34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa55600640 1 -- 172.21.15.87:0/1898556407 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa4811fcf0 con 0x7ffa5806cf40 2024-04-25T10:21:44.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.030+0000 7ffa4fe00640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa34066f60 0x7ffa34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.034+0000 7ffa4fe00640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa34066f60 0x7ffa34069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffa581fa120 tx=0x7ffa40009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.034+0000 7ffa55600640 1 -- 172.21.15.87:0/1898556407 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa48124070 con 0x7ffa5806cf40 2024-04-25T10:21:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:44.226+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffa58072ff0 con 0x7ffa5806cf40 2024-04-25T10:21:45.150 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T10:21:45.151+0000 7fea28c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:21:45.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.206+0000 7ffa55600640 1 -- 172.21.15.87:0/1898556407 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffa48016090 con 0x7ffa5806cf40 2024-04-25T10:21:45.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa34066f60 msgr2=0x7ffa34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa34066f60 0x7ffa34069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffa581fa120 tx=0x7ffa40009040 comp rx=0 tx=0).stop 2024-04-25T10:21:45.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa5806cf40 msgr2=0x7ffa58156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa5806cf40 0x7ffa58156220 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ffa480ae040 tx=0x7ffa4801ac50 comp rx=0 tx=0).stop 2024-04-25T10:21:45.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 shutdown_connections 2024-04-25T10:21:45.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffa34066f60 0x7ffa34069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa580d0d00 0x7ffa58158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa580cc200 0x7ffa580c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 --2- 172.21.15.87:0/1898556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa5806cf40 0x7ffa58156220 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 >> 172.21.15.87:0/1898556407 conn(0x7ffa58072290 msgr2=0x7ffa580ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:45.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 shutdown_connections 2024-04-25T10:21:45.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.210+0000 7ffa5f400640 1 -- 172.21.15.87:0/1898556407 wait complete. 2024-04-25T10:21:45.236 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:21:45.236 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:21:45.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7e1a00640 1 Processor -- start 2024-04-25T10:21:45.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7e1a00640 1 -- start start 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7e1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc0aef80 0x7ff7dc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7e1a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff7dc0b31c0 0x7ff7dc0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7e1a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc15b080 con 0x7ff7dc070350 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7e1a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc15adb0 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7e1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc15eb60 con 0x7ff7dc0aef80 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7db400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff7dc0b31c0 0x7ff7dc0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7db400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff7dc0b31c0 0x7ff7dc0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35434/0 (socket says 172.21.15.87:35434) 2024-04-25T10:21:45.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7db400640 1 -- 172.21.15.87:0/769732130 learned_addr learned my addr 172.21.15.87:0/769732130 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7da000640 1 --1- 172.21.15.87:0/769732130 >> v1:172.21.15.87:6789/0 conn(0x7ff7dc070350 0x7ff7dc15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:49034/0 (socket says 172.21.15.87:49034) 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7daa00640 1 --2- 172.21.15.87:0/769732130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc0aef80 0x7ff7dc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7db400640 1 -- 172.21.15.87:0/769732130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc0aef80 msgr2=0x7ff7dc15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7db400640 1 --2- 172.21.15.87:0/769732130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc0aef80 0x7ff7dc15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7db400640 1 -- 172.21.15.87:0/769732130 >> v1:172.21.15.87:6789/0 conn(0x7ff7dc070350 legacy=0x7ff7dc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7db400640 1 -- 172.21.15.87:0/769732130 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc15ee40 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7d9600640 1 -- 172.21.15.87:0/769732130 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308597360 0 0) 0x7ff7dc15b080 con 0x7ff7dc070350 2024-04-25T10:21:45.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.374+0000 7ff7daa00640 1 --2- 172.21.15.87:0/769732130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc0aef80 0x7ff7dc15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7db400640 1 --2- 172.21.15.87:0/769732130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0b3610 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009e50 tx=0x7ff7d401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f530568e9c5a351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7d9600640 1 -- 172.21.15.87:0/769732130 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40a5810 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7d9600640 1 -- 172.21.15.87:0/769732130 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d40a59b0 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7d9600640 1 -- 172.21.15.87:0/769732130 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40a5d80 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 -- 172.21.15.87:0/769732130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 msgr2=0x7ff7dc0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/769732130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0b3610 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009e50 tx=0x7ff7d401ca30 comp rx=0 tx=0).stop 2024-04-25T10:21:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 -- 172.21.15.87:0/769732130 shutdown_connections 2024-04-25T10:21:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/769732130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0b3610 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/769732130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc0aef80 0x7ff7dc15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 -- 172.21.15.87:0/769732130 >> 172.21.15.87:0/769732130 conn(0x7ff7dc0669a0 msgr2=0x7ff7dc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:45.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 -- 172.21.15.87:0/769732130 shutdown_connections 2024-04-25T10:21:45.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.378+0000 7ff7e1a00640 1 -- 172.21.15.87:0/769732130 wait complete. 2024-04-25T10:21:45.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 Processor -- start 2024-04-25T10:21:45.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 -- start start 2024-04-25T10:21:45.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc070350 0x7ff7dc1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc0aef80 0x7ff7dc0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc1679a0 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc167820 con 0x7ff7dc0aef80 2024-04-25T10:21:45.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc167b20 con 0x7ff7dc070350 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35440/0 (socket says 172.21.15.87:35440) 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 -- 172.21.15.87:0/1823606587 learned_addr learned my addr 172.21.15.87:0/1823606587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7daa00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc070350 0x7ff7dc1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7da000640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc0aef80 0x7ff7dc0b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 -- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc070350 msgr2=0x7ff7dc1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc070350 0x7ff7dc1544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 -- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc0aef80 msgr2=0x7ff7dc0b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc0aef80 0x7ff7dc0b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 -- 172.21.15.87:0/1823606587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc14f870 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7db400640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0cc930 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff7d401a660 tx=0x7ff7d40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7d2a00640 1 -- 172.21.15.87:0/1823606587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40bc070 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7d2a00640 1 -- 172.21.15.87:0/1823606587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d40b8660 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7d2a00640 1 -- 172.21.15.87:0/1823606587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40b8a50 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc14fb50 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc150110 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.382+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc04ef30 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.386+0000 7ff7d2a00640 1 -- 172.21.15.87:0/1823606587 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7d40bf070 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.386+0000 7ff7d2a00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7bc066f90 0x7ff7bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.386+0000 7ff7d2a00640 1 -- 172.21.15.87:0/1823606587 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7d411fa30 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.386+0000 7ff7daa00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7bc066f90 0x7ff7bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.386+0000 7ff7daa00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7bc066f90 0x7ff7bc069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c900 tx=0x7ff7cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.390+0000 7ff7d2a00640 1 -- 172.21.15.87:0/1823606587 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7d40c6070 con 0x7ff7dc0b31c0 2024-04-25T10:21:45.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:45.574+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff7dc0b2fa0 con 0x7ff7dc0b31c0 2024-04-25T10:21:46.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7d2a00640 1 -- 172.21.15.87:0/1823606587 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff7d4016620 con 0x7ff7dc0b31c0 2024-04-25T10:21:46.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7bc066f90 msgr2=0x7ff7bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7bc066f90 0x7ff7bc069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c900 tx=0x7ff7cc008040 comp rx=0 tx=0).stop 2024-04-25T10:21:46.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 msgr2=0x7ff7dc0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0cc930 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff7d401a660 tx=0x7ff7d40197d0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 shutdown_connections 2024-04-25T10:21:46.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7bc066f90 0x7ff7bc069450 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7dc0b31c0 0x7ff7dc0cc930 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc0aef80 0x7ff7dc0b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 --2- 172.21.15.87:0/1823606587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc070350 0x7ff7dc1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 >> 172.21.15.87:0/1823606587 conn(0x7ff7dc0669a0 msgr2=0x7ff7dc15bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 shutdown_connections 2024-04-25T10:21:46.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.218+0000 7ff7e1a00640 1 -- 172.21.15.87:0/1823606587 wait complete. 2024-04-25T10:21:46.245 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:21:46.245 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:46.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a95200640 1 Processor -- start 2024-04-25T10:21:46.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a95200640 1 -- start start 2024-04-25T10:21:46.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a95200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900d1820 0x7f6a900d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a95200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6a900b15b0 0x7f6a900cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a95200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a900b1af0 con 0x7f6a9015e050 2024-04-25T10:21:46.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a95200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a900b1dd0 con 0x7f6a900b15b0 2024-04-25T10:21:46.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a95200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a900b20b0 con 0x7f6a900d1820 2024-04-25T10:21:46.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900d1820 0x7f6a900d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900d1820 0x7f6a900d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35662/0 (socket says 172.21.15.87:35662) 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8de00640 1 -- 172.21.15.87:0/2088517439 learned_addr learned my addr 172.21.15.87:0/2088517439 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8e800640 1 --2- 172.21.15.87:0/2088517439 >> v2:172.21.15.169:3300/0 conn(0x7f6a900b15b0 0x7f6a900cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8de00640 1 -- 172.21.15.87:0/2088517439 >> v2:172.21.15.169:3300/0 conn(0x7f6a900b15b0 msgr2=0x7f6a900cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8de00640 1 --2- 172.21.15.87:0/2088517439 >> v2:172.21.15.169:3300/0 conn(0x7f6a900b15b0 0x7f6a900cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8de00640 1 -- 172.21.15.87:0/2088517439 >> v1:172.21.15.169:6789/0 conn(0x7f6a9015e050 legacy=0x7f6a900b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8de00640 1 -- 172.21.15.87:0/2088517439 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900b2390 con 0x7f6a900d1820 2024-04-25T10:21:46.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.382+0000 7f6a8d400640 1 -- 172.21.15.87:0/2088517439 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208925308 0 0) 0x7f6a900b1af0 con 0x7f6a9015e050 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a8de00640 1 --2- 172.21.15.87:0/2088517439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900d1820 0x7f6a900d1c00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009870 tx=0x7f6a8001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2905f23866ab9b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a8d400640 1 -- 172.21.15.87:0/2088517439 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800a9640 con 0x7f6a900d1820 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a8d400640 1 -- 172.21.15.87:0/2088517439 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a800a97e0 con 0x7f6a900d1820 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a8d400640 1 -- 172.21.15.87:0/2088517439 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800a9bb0 con 0x7f6a900d1820 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 -- 172.21.15.87:0/2088517439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900d1820 msgr2=0x7f6a900d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 --2- 172.21.15.87:0/2088517439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900d1820 0x7f6a900d1c00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009870 tx=0x7f6a8001c560 comp rx=0 tx=0).stop 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 -- 172.21.15.87:0/2088517439 shutdown_connections 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 --2- 172.21.15.87:0/2088517439 >> v2:172.21.15.169:3300/0 conn(0x7f6a900b15b0 0x7f6a900cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 --2- 172.21.15.87:0/2088517439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900d1820 0x7f6a900d1c00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 -- 172.21.15.87:0/2088517439 >> 172.21.15.87:0/2088517439 conn(0x7f6a9009a9d0 msgr2=0x7f6a9009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 -- 172.21.15.87:0/2088517439 shutdown_connections 2024-04-25T10:21:46.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.386+0000 7f6a95200640 1 -- 172.21.15.87:0/2088517439 wait complete. 2024-04-25T10:21:46.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 Processor -- start 2024-04-25T10:21:46.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 -- start start 2024-04-25T10:21:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900b15b0 0x7f6a900cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900d1820 0x7f6a90144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a9015e050 0x7f6a9006ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a900d0d90 con 0x7f6a900d1820 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a900d0c10 con 0x7f6a9015e050 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a900d0f10 con 0x7f6a900b15b0 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900d1820 0x7f6a90144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900d1820 0x7f6a90144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35448/0 (socket says 172.21.15.87:35448) 2024-04-25T10:21:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 -- 172.21.15.87:0/153460954 learned_addr learned my addr 172.21.15.87:0/153460954 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a8de00640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900b15b0 0x7f6a900cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a8e800640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a9015e050 0x7f6a9006ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 -- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900b15b0 msgr2=0x7f6a900cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900b15b0 0x7f6a900cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 -- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a9015e050 msgr2=0x7f6a9006ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a9015e050 0x7f6a9006ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 -- 172.21.15.87:0/153460954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900ccf40 con 0x7f6a900d1820 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a85400640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900d1820 0x7f6a90144aa0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800c520 tx=0x7f6a7800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a86a00640 1 -- 172.21.15.87:0/153460954 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78017070 con 0x7f6a900d1820 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a86a00640 1 -- 172.21.15.87:0/153460954 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a78003600 con 0x7f6a900d1820 2024-04-25T10:21:46.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a86a00640 1 -- 172.21.15.87:0/153460954 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780039f0 con 0x7f6a900d1820 2024-04-25T10:21:46.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a900cc6a0 con 0x7f6a900d1820 2024-04-25T10:21:46.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.390+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a900ccc60 con 0x7f6a900d1820 2024-04-25T10:21:46.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.394+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9004ef30 con 0x7f6a900d1820 2024-04-25T10:21:46.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.394+0000 7f6a86a00640 1 -- 172.21.15.87:0/153460954 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a7801a070 con 0x7f6a900d1820 2024-04-25T10:21:46.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.394+0000 7f6a86a00640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a6c066f60 0x7f6a6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.394+0000 7f6a86a00640 1 -- 172.21.15.87:0/153460954 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a78084400 con 0x7f6a900d1820 2024-04-25T10:21:46.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.394+0000 7f6a8de00640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a6c066f60 0x7f6a6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.394+0000 7f6a8de00640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a6c066f60 0x7f6a6c069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6a800ae000 tx=0x7f6a800b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.398+0000 7f6a86a00640 1 -- 172.21.15.87:0/153460954 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a780554a0 con 0x7f6a900d1820 2024-04-25T10:21:46.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.582+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a900d23a0 con 0x7f6a900d1820 2024-04-25T10:21:46.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.582+0000 7f6a86a00640 1 -- 172.21.15.87:0/153460954 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f6a78054da0 con 0x7f6a900d1820 2024-04-25T10:21:46.586 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:46.586 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:21:46.586 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:21:46.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a6c066f60 msgr2=0x7f6a6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a6c066f60 0x7f6a6c069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6a800ae000 tx=0x7f6a800b3040 comp rx=0 tx=0).stop 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900d1820 msgr2=0x7f6a90144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900d1820 0x7f6a90144aa0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800c520 tx=0x7f6a7800c9e0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 shutdown_connections 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6a6c066f60 0x7f6a6c069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a9015e050 0x7f6a9006ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900d1820 0x7f6a90144aa0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 --2- 172.21.15.87:0/153460954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a900b15b0 0x7f6a900cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 >> 172.21.15.87:0/153460954 conn(0x7f6a9009a9d0 msgr2=0x7f6a900af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 shutdown_connections 2024-04-25T10:21:46.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.586+0000 7f6a95200640 1 -- 172.21.15.87:0/153460954 wait complete. 2024-04-25T10:21:46.608 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:46.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.738+0000 7f5751e00640 1 Processor -- start 2024-04-25T10:21:46.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751e00640 1 -- start start 2024-04-25T10:21:46.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f574c15b850 0x7f574c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f574c0af6a0 0x7f574c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c15bc30 con 0x7f574c06ee60 2024-04-25T10:21:46.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c06f2b0 con 0x7f574c15b850 2024-04-25T10:21:46.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c0b2fd0 con 0x7f574c0af6a0 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5750a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f574c15b850 0x7f574c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f574c06ee60 0x7f574c06cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:49036/0 (socket says 172.21.15.87:49036) 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5750a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f574c15b850 0x7f574c0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35452/0 (socket says 172.21.15.87:35452) 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5751400640 1 -- 172.21.15.87:0/1182707912 learned_addr learned my addr 172.21.15.87:0/1182707912 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f574be00640 1 --2- 172.21.15.87:0/1182707912 >> v2:172.21.15.169:3300/0 conn(0x7f574c0af6a0 0x7f574c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5750a00640 1 -- 172.21.15.87:0/1182707912 >> v2:172.21.15.169:3300/0 conn(0x7f574c0af6a0 msgr2=0x7f574c06e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5750a00640 1 --2- 172.21.15.87:0/1182707912 >> v2:172.21.15.169:3300/0 conn(0x7f574c0af6a0 0x7f574c06e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5750a00640 1 -- 172.21.15.87:0/1182707912 >> v1:172.21.15.87:6789/0 conn(0x7f574c06ee60 legacy=0x7f574c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f574b400640 1 -- 172.21.15.87:0/1182707912 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299634058 0 0) 0x7f574c15bc30 con 0x7f574c06ee60 2024-04-25T10:21:46.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.742+0000 7f5750a00640 1 -- 172.21.15.87:0/1182707912 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0cf360 con 0x7f574c15b850 2024-04-25T10:21:46.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5750a00640 1 --2- 172.21.15.87:0/1182707912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c15b850 0x7f574c0cdfd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f574c0d0f40 tx=0x7f574001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9456842ea6a864a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f574b400640 1 -- 172.21.15.87:0/1182707912 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400ab650 con 0x7f574c15b850 2024-04-25T10:21:46.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f574b400640 1 -- 172.21.15.87:0/1182707912 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57400b0070 con 0x7f574c15b850 2024-04-25T10:21:46.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f574b400640 1 -- 172.21.15.87:0/1182707912 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400aba10 con 0x7f574c15b850 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 -- 172.21.15.87:0/1182707912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c15b850 msgr2=0x7f574c0cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 --2- 172.21.15.87:0/1182707912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c15b850 0x7f574c0cdfd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f574c0d0f40 tx=0x7f574001c450 comp rx=0 tx=0).stop 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 -- 172.21.15.87:0/1182707912 shutdown_connections 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 --2- 172.21.15.87:0/1182707912 >> v2:172.21.15.169:3300/0 conn(0x7f574c0af6a0 0x7f574c06e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 --2- 172.21.15.87:0/1182707912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c15b850 0x7f574c0cdfd0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 -- 172.21.15.87:0/1182707912 >> 172.21.15.87:0/1182707912 conn(0x7f574c0cfe80 msgr2=0x7f574c0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 -- 172.21.15.87:0/1182707912 shutdown_connections 2024-04-25T10:21:46.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 -- 172.21.15.87:0/1182707912 wait complete. 2024-04-25T10:21:46.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 Processor -- start 2024-04-25T10:21:46.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.746+0000 7f5751e00640 1 -- start start 2024-04-25T10:21:46.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c06ee60 0x7f574c0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c0af6a0 0x7f574c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c15b850 0x7f574c071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0cf140 con 0x7f574c0af6a0 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0cefc0 con 0x7f574c06ee60 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c168400 con 0x7f574c15b850 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c0af6a0 0x7f574c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c0af6a0 0x7f574c071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35460/0 (socket says 172.21.15.87:35460) 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 -- 172.21.15.87:0/2477701529 learned_addr learned my addr 172.21.15.87:0/2477701529 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5750a00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c06ee60 0x7f574c0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751400640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c15b850 0x7f574c071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 -- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c15b850 msgr2=0x7f574c071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c15b850 0x7f574c071fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 -- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c06ee60 msgr2=0x7f574c0afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c06ee60 0x7f574c0afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 -- 172.21.15.87:0/2477701529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0b0070 con 0x7f574c0af6a0 2024-04-25T10:21:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f574be00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c0af6a0 0x7f574c071a60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f573800d800 tx=0x7f573800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5748c00640 1 -- 172.21.15.87:0/2477701529 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800b6c0 con 0x7f574c0af6a0 2024-04-25T10:21:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5748c00640 1 -- 172.21.15.87:0/2477701529 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738019070 con 0x7f574c0af6a0 2024-04-25T10:21:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5748c00640 1 -- 172.21.15.87:0/2477701529 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800ba60 con 0x7f574c0af6a0 2024-04-25T10:21:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c0b0340 con 0x7f574c0af6a0 2024-04-25T10:21:46.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c0b0900 con 0x7f574c0af6a0 2024-04-25T10:21:46.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c075120 con 0x7f574c0af6a0 2024-04-25T10:21:46.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.750+0000 7f5748c00640 1 -- 172.21.15.87:0/2477701529 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f573800bc00 con 0x7f574c0af6a0 2024-04-25T10:21:46.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.754+0000 7f5748c00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f572c066f60 0x7f572c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.754+0000 7f5748c00640 1 -- 172.21.15.87:0/2477701529 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57380853e0 con 0x7f574c0af6a0 2024-04-25T10:21:46.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.754+0000 7f5750a00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f572c066f60 0x7f572c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.754+0000 7f5750a00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f572c066f60 0x7f572c069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f57400af000 tx=0x7f57400a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.758+0000 7f5748c00640 1 -- 172.21.15.87:0/2477701529 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5738056480 con 0x7f574c0af6a0 2024-04-25T10:21:46.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f574c068e00 con 0x7f574c0af6a0 2024-04-25T10:21:46.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5748c00640 1 -- 172.21.15.87:0/2477701529 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f5738055d80 con 0x7f574c0af6a0 2024-04-25T10:21:46.917 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:46.917 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":124,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:21:44.190853+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:40.912468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:41.547768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:46.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f572c066f60 msgr2=0x7f572c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f572c066f60 0x7f572c069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f57400af000 tx=0x7f57400a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c0af6a0 msgr2=0x7f574c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c0af6a0 0x7f574c071a60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f573800d800 tx=0x7f573800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 shutdown_connections 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f572c066f60 0x7f572c069420 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c15b850 0x7f574c071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.914+0000 7f5751e00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f574c0af6a0 0x7f574c071a60 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.918+0000 7f5751e00640 1 --2- 172.21.15.87:0/2477701529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c06ee60 0x7f574c0afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.918+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 >> 172.21.15.87:0/2477701529 conn(0x7f574c0cfe80 msgr2=0x7f574c15a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.918+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 shutdown_connections 2024-04-25T10:21:46.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:46.918+0000 7f5751e00640 1 -- 172.21.15.87:0/2477701529 wait complete. 2024-04-25T10:21:46.939 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:21:46.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:21:47.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.070+0000 7fb183a00640 1 Processor -- start 2024-04-25T10:21:47.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.070+0000 7fb183a00640 1 -- start start 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb183a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb17c0d0040 0x7fb17c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb183a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb17c15b740 0x7fb17c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb183a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c071050 con 0x7fb17c1550b0 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb183a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c15d040 con 0x7fb17c15b740 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb183a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c0712f0 con 0x7fb17c0d0040 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb181a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb17c15b740 0x7fb17c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb181a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb17c15b740 0x7fb17c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35466/0 (socket says 172.21.15.87:35466) 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb181a00640 1 -- 172.21.15.87:0/29966928 learned_addr learned my addr 172.21.15.87:0/29966928 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb173e00640 1 --2- 172.21.15.87:0/29966928 >> v2:172.21.15.169:3300/0 conn(0x7fb17c0d0040 0x7fb17c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb181a00640 1 -- 172.21.15.87:0/29966928 >> v2:172.21.15.169:3300/0 conn(0x7fb17c0d0040 msgr2=0x7fb17c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb181a00640 1 --2- 172.21.15.87:0/29966928 >> v2:172.21.15.169:3300/0 conn(0x7fb17c0d0040 0x7fb17c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb181a00640 1 -- 172.21.15.87:0/29966928 >> v1:172.21.15.169:6789/0 conn(0x7fb17c1550b0 legacy=0x7fb17c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.074+0000 7fb181a00640 1 -- 172.21.15.87:0/29966928 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0715d0 con 0x7fb17c15b740 2024-04-25T10:21:47.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb181a00640 1 --2- 172.21.15.87:0/29966928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c15b740 0x7fb17c0b3260 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb17c070160 tx=0x7fb17401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e141ea54d7fab31e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb17be00640 1 -- 172.21.15.87:0/29966928 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1740a5880 con 0x7fb17c15b740 2024-04-25T10:21:47.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb17be00640 1 -- 172.21.15.87:0/29966928 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1740a5a20 con 0x7fb17c15b740 2024-04-25T10:21:47.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb17be00640 1 -- 172.21.15.87:0/29966928 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1740a5df0 con 0x7fb17c15b740 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 -- 172.21.15.87:0/29966928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c15b740 msgr2=0x7fb17c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 --2- 172.21.15.87:0/29966928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c15b740 0x7fb17c0b3260 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb17c070160 tx=0x7fb17401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 -- 172.21.15.87:0/29966928 shutdown_connections 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 --2- 172.21.15.87:0/29966928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c15b740 0x7fb17c0b3260 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 --2- 172.21.15.87:0/29966928 >> v2:172.21.15.169:3300/0 conn(0x7fb17c0d0040 0x7fb17c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 -- 172.21.15.87:0/29966928 >> 172.21.15.87:0/29966928 conn(0x7fb17c066b70 msgr2=0x7fb17c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 -- 172.21.15.87:0/29966928 shutdown_connections 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 -- 172.21.15.87:0/29966928 wait complete. 2024-04-25T10:21:47.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 Processor -- start 2024-04-25T10:21:47.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.078+0000 7fb183a00640 1 -- start start 2024-04-25T10:21:47.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17c0d0040 0x7fb17c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c1550b0 0x7fb17c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb17c15b740 0x7fb17c157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c15bd10 con 0x7fb17c1550b0 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c15bb90 con 0x7fb17c15b740 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c15c030 con 0x7fb17c0d0040 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c1550b0 0x7fb17c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c1550b0 0x7fb17c153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35470/0 (socket says 172.21.15.87:35470) 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 -- 172.21.15.87:0/1140378333 learned_addr learned my addr 172.21.15.87:0/1140378333 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 -- 172.21.15.87:0/1140378333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17c0d0040 msgr2=0x7fb17c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17c0d0040 0x7fb17c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 -- 172.21.15.87:0/1140378333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb17c15b740 msgr2=0x7fb17c157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb17c15b740 0x7fb17c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 -- 172.21.15.87:0/1140378333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c14fda0 con 0x7fb17c1550b0 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb173e00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c1550b0 0x7fb17c153fa0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e960 tx=0x7fb16400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb179600640 1 -- 172.21.15.87:0/1140378333 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164017070 con 0x7fb17c1550b0 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb179600640 1 -- 172.21.15.87:0/1140378333 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16400ca00 con 0x7fb17c1550b0 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb179600640 1 -- 172.21.15.87:0/1140378333 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400cdb0 con 0x7fb17c1550b0 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c150080 con 0x7fb17c1550b0 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c150640 con 0x7fb17c1550b0 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c04ef30 con 0x7fb17c1550b0 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb179600640 1 -- 172.21.15.87:0/1140378333 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb16401a070 con 0x7fb17c1550b0 2024-04-25T10:21:47.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.082+0000 7fb179600640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb158066f60 0x7fb158069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.086+0000 7fb179600640 1 -- 172.21.15.87:0/1140378333 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb164085450 con 0x7fb17c1550b0 2024-04-25T10:21:47.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.086+0000 7fb181000640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb158066f60 0x7fb158069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.086+0000 7fb181000640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb158066f60 0x7fb158069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb16c000f70 tx=0x7fb16c0079e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.086+0000 7fb179600640 1 -- 172.21.15.87:0/1140378333 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb164056450 con 0x7fb17c1550b0 2024-04-25T10:21:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.242+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb17c0ceed0 con 0x7fb158066f60 2024-04-25T10:21:47.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.246+0000 7fb179600640 1 -- 172.21.15.87:0/1140378333 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7fb17c0ceed0 con 0x7fb158066f60 2024-04-25T10:21:47.338 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:47.340 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb158066f60 msgr2=0x7fb158069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb158066f60 0x7fb158069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb16c000f70 tx=0x7fb16c0079e0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c1550b0 msgr2=0x7fb17c153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c1550b0 0x7fb17c153fa0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e960 tx=0x7fb16400ee20 comp rx=0 tx=0).stop 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 shutdown_connections 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb158066f60 0x7fb158069420 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb17c15b740 0x7fb17c157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb17c1550b0 0x7fb17c153fa0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 --2- 172.21.15.87:0/1140378333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17c0d0040 0x7fb17c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 >> 172.21.15.87:0/1140378333 conn(0x7fb17c066b70 msgr2=0x7fb17c06c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 shutdown_connections 2024-04-25T10:21:47.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.250+0000 7fb183a00640 1 -- 172.21.15.87:0/1140378333 wait complete. 2024-04-25T10:21:47.343 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-04-25T10:21:47.243532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":450192,"kb_used_data":13320,"kb_used_omap":12,"kb_used_meta":436723,"kb_avail":754524528,"statfs":{"total":773094113280,"available":772633116672,"internally_reserved":0,"allocated":13639680,"data_stored":6580212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":447204913},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001155"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992665+0000","last_change":"2024-04-25T10:21:43.201253+0000","last_active":"2024-04-25T10:21:44.992665+0000","last_peered":"2024-04-25T10:21:44.992665+0000","last_clean":"2024-04-25T10:21:44.992665+0000","last_became_active":"2024-04-25T10:21:43.201002+0000","last_became_peered":"2024-04-25T10:21:43.201002+0000","last_unstale":"2024-04-25T10:21:44.992665+0000","last_undegraded":"2024-04-25T10:21:44.992665+0000","last_fullsized":"2024-04-25T10:21:44.992665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:04.595700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201055+0000","last_change":"2024-04-25T10:21:43.194524+0000","last_active":"2024-04-25T10:21:44.201055+0000","last_peered":"2024-04-25T10:21:44.201055+0000","last_clean":"2024-04-25T10:21:44.201055+0000","last_became_active":"2024-04-25T10:21:43.194058+0000","last_became_peered":"2024-04-25T10:21:43.194058+0000","last_unstale":"2024-04-25T10:21:44.201055+0000","last_undegraded":"2024-04-25T10:21:44.201055+0000","last_fullsized":"2024-04-25T10:21:44.201055+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:30.975965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209810+0000","last_change":"2024-04-25T10:21:43.202628+0000","last_active":"2024-04-25T10:21:44.209810+0000","last_peered":"2024-04-25T10:21:44.209810+0000","last_clean":"2024-04-25T10:21:44.209810+0000","last_became_active":"2024-04-25T10:21:43.202306+0000","last_became_peered":"2024-04-25T10:21:43.202306+0000","last_unstale":"2024-04-25T10:21:44.209810+0000","last_undegraded":"2024-04-25T10:21:44.209810+0000","last_fullsized":"2024-04-25T10:21:44.209810+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.656126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992015+0000","last_change":"2024-04-25T10:21:43.217095+0000","last_active":"2024-04-25T10:21:44.992015+0000","last_peered":"2024-04-25T10:21:44.992015+0000","last_clean":"2024-04-25T10:21:44.992015+0000","last_became_active":"2024-04-25T10:21:43.216779+0000","last_became_peered":"2024-04-25T10:21:43.216779+0000","last_unstale":"2024-04-25T10:21:44.992015+0000","last_undegraded":"2024-04-25T10:21:44.992015+0000","last_fullsized":"2024-04-25T10:21:44.992015+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:14.620441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585141+0000","last_change":"2024-04-25T10:21:43.218462+0000","last_active":"2024-04-25T10:21:44.585141+0000","last_peered":"2024-04-25T10:21:44.585141+0000","last_clean":"2024-04-25T10:21:44.585141+0000","last_became_active":"2024-04-25T10:21:43.217837+0000","last_became_peered":"2024-04-25T10:21:43.217837+0000","last_unstale":"2024-04-25T10:21:44.585141+0000","last_undegraded":"2024-04-25T10:21:44.585141+0000","last_fullsized":"2024-04-25T10:21:44.585141+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:34.691817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992738+0000","last_change":"2024-04-25T10:21:43.204095+0000","last_active":"2024-04-25T10:21:44.992738+0000","last_peered":"2024-04-25T10:21:44.992738+0000","last_clean":"2024-04-25T10:21:44.992738+0000","last_became_active":"2024-04-25T10:21:43.203806+0000","last_became_peered":"2024-04-25T10:21:43.203806+0000","last_unstale":"2024-04-25T10:21:44.992738+0000","last_undegraded":"2024-04-25T10:21:44.992738+0000","last_fullsized":"2024-04-25T10:21:44.992738+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:32.101037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992502+0000","last_change":"2024-04-25T10:21:43.203046+0000","last_active":"2024-04-25T10:21:44.992502+0000","last_peered":"2024-04-25T10:21:44.992502+0000","last_clean":"2024-04-25T10:21:44.992502+0000","last_became_active":"2024-04-25T10:21:43.201225+0000","last_became_peered":"2024-04-25T10:21:43.201225+0000","last_unstale":"2024-04-25T10:21:44.992502+0000","last_undegraded":"2024-04-25T10:21:44.992502+0000","last_fullsized":"2024-04-25T10:21:44.992502+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:25.435097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209032+0000","last_change":"2024-04-25T10:21:43.203113+0000","last_active":"2024-04-25T10:21:44.209032+0000","last_peered":"2024-04-25T10:21:44.209032+0000","last_clean":"2024-04-25T10:21:44.209032+0000","last_became_active":"2024-04-25T10:21:43.202741+0000","last_became_peered":"2024-04-25T10:21:43.202741+0000","last_unstale":"2024-04-25T10:21:44.209032+0000","last_undegraded":"2024-04-25T10:21:44.209032+0000","last_fullsized":"2024-04-25T10:21:44.209032+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:00.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210031+0000","last_change":"2024-04-25T10:21:43.216045+0000","last_active":"2024-04-25T10:21:44.210031+0000","last_peered":"2024-04-25T10:21:44.210031+0000","last_clean":"2024-04-25T10:21:44.210031+0000","last_became_active":"2024-04-25T10:21:43.215756+0000","last_became_peered":"2024-04-25T10:21:43.215756+0000","last_unstale":"2024-04-25T10:21:44.210031+0000","last_undegraded":"2024-04-25T10:21:44.210031+0000","last_fullsized":"2024-04-25T10:21:44.210031+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:25.435097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201041+0000","last_change":"2024-04-25T10:21:43.198121+0000","last_active":"2024-04-25T10:21:44.201041+0000","last_peered":"2024-04-25T10:21:44.201041+0000","last_clean":"2024-04-25T10:21:44.201041+0000","last_became_active":"2024-04-25T10:21:43.195957+0000","last_became_peered":"2024-04-25T10:21:43.195957+0000","last_unstale":"2024-04-25T10:21:44.201041+0000","last_undegraded":"2024-04-25T10:21:44.201041+0000","last_fullsized":"2024-04-25T10:21:44.201041+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:58.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992411+0000","last_change":"2024-04-25T10:21:43.207398+0000","last_active":"2024-04-25T10:21:44.992411+0000","last_peered":"2024-04-25T10:21:44.992411+0000","last_clean":"2024-04-25T10:21:44.992411+0000","last_became_active":"2024-04-25T10:21:43.207112+0000","last_became_peered":"2024-04-25T10:21:43.207112+0000","last_unstale":"2024-04-25T10:21:44.992411+0000","last_undegraded":"2024-04-25T10:21:44.992411+0000","last_fullsized":"2024-04-25T10:21:44.992411+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:58.483697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209506+0000","last_change":"2024-04-25T10:21:43.216581+0000","last_active":"2024-04-25T10:21:44.209506+0000","last_peered":"2024-04-25T10:21:44.209506+0000","last_clean":"2024-04-25T10:21:44.209506+0000","last_became_active":"2024-04-25T10:21:43.214485+0000","last_became_peered":"2024-04-25T10:21:43.214485+0000","last_unstale":"2024-04-25T10:21:44.209506+0000","last_undegraded":"2024-04-25T10:21:44.209506+0000","last_fullsized":"2024-04-25T10:21:44.209506+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:40.540103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992016+0000","last_change":"2024-04-25T10:21:43.203020+0000","last_active":"2024-04-25T10:21:44.992016+0000","last_peered":"2024-04-25T10:21:44.992016+0000","last_clean":"2024-04-25T10:21:44.992016+0000","last_became_active":"2024-04-25T10:21:43.199772+0000","last_became_peered":"2024-04-25T10:21:43.199772+0000","last_unstale":"2024-04-25T10:21:44.992016+0000","last_undegraded":"2024-04-25T10:21:44.992016+0000","last_fullsized":"2024-04-25T10:21:44.992016+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:39.321957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201070+0000","last_change":"2024-04-25T10:21:43.202143+0000","last_active":"2024-04-25T10:21:44.201070+0000","last_peered":"2024-04-25T10:21:44.201070+0000","last_clean":"2024-04-25T10:21:44.201070+0000","last_became_active":"2024-04-25T10:21:43.199517+0000","last_became_peered":"2024-04-25T10:21:43.199517+0000","last_unstale":"2024-04-25T10:21:44.201070+0000","last_undegraded":"2024-04-25T10:21:44.201070+0000","last_fullsized":"2024-04-25T10:21:44.201070+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:11.257163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991630+0000","last_change":"2024-04-25T10:21:43.199033+0000","last_active":"2024-04-25T10:21:44.991630+0000","last_peered":"2024-04-25T10:21:44.991630+0000","last_clean":"2024-04-25T10:21:44.991630+0000","last_became_active":"2024-04-25T10:21:43.198661+0000","last_became_peered":"2024-04-25T10:21:43.198661+0000","last_unstale":"2024-04-25T10:21:44.991630+0000","last_undegraded":"2024-04-25T10:21:44.991630+0000","last_fullsized":"2024-04-25T10:21:44.991630+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:19.932443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991087+0000","last_change":"2024-04-25T10:21:43.208744+0000","last_active":"2024-04-25T10:21:44.991087+0000","last_peered":"2024-04-25T10:21:44.991087+0000","last_clean":"2024-04-25T10:21:44.991087+0000","last_became_active":"2024-04-25T10:21:43.206607+0000","last_became_peered":"2024-04-25T10:21:43.206607+0000","last_unstale":"2024-04-25T10:21:44.991087+0000","last_undegraded":"2024-04-25T10:21:44.991087+0000","last_fullsized":"2024-04-25T10:21:44.991087+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:52.724378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992962+0000","last_change":"2024-04-25T10:21:43.211416+0000","last_active":"2024-04-25T10:21:44.992962+0000","last_peered":"2024-04-25T10:21:44.992962+0000","last_clean":"2024-04-25T10:21:44.992962+0000","last_became_active":"2024-04-25T10:21:43.211143+0000","last_became_peered":"2024-04-25T10:21:43.211143+0000","last_unstale":"2024-04-25T10:21:44.992962+0000","last_undegraded":"2024-04-25T10:21:44.992962+0000","last_fullsized":"2024-04-25T10:21:44.992962+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:09.370407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992354+0000","last_change":"2024-04-25T10:21:43.207544+0000","last_active":"2024-04-25T10:21:44.992354+0000","last_peered":"2024-04-25T10:21:44.992354+0000","last_clean":"2024-04-25T10:21:44.992354+0000","last_became_active":"2024-04-25T10:21:43.206225+0000","last_became_peered":"2024-04-25T10:21:43.206225+0000","last_unstale":"2024-04-25T10:21:44.992354+0000","last_undegraded":"2024-04-25T10:21:44.992354+0000","last_fullsized":"2024-04-25T10:21:44.992354+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.396791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992863+0000","last_change":"2024-04-25T10:21:43.203032+0000","last_active":"2024-04-25T10:21:44.992863+0000","last_peered":"2024-04-25T10:21:44.992863+0000","last_clean":"2024-04-25T10:21:44.992863+0000","last_became_active":"2024-04-25T10:21:43.202569+0000","last_became_peered":"2024-04-25T10:21:43.202569+0000","last_unstale":"2024-04-25T10:21:44.992863+0000","last_undegraded":"2024-04-25T10:21:44.992863+0000","last_fullsized":"2024-04-25T10:21:44.992863+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:44.429556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991771+0000","last_change":"2024-04-25T10:21:43.209869+0000","last_active":"2024-04-25T10:21:44.991771+0000","last_peered":"2024-04-25T10:21:44.991771+0000","last_clean":"2024-04-25T10:21:44.991771+0000","last_became_active":"2024-04-25T10:21:43.209621+0000","last_became_peered":"2024-04-25T10:21:43.209621+0000","last_unstale":"2024-04-25T10:21:44.991771+0000","last_undegraded":"2024-04-25T10:21:44.991771+0000","last_fullsized":"2024-04-25T10:21:44.991771+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.480924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584927+0000","last_change":"2024-04-25T10:21:43.215047+0000","last_active":"2024-04-25T10:21:44.584927+0000","last_peered":"2024-04-25T10:21:44.584927+0000","last_clean":"2024-04-25T10:21:44.584927+0000","last_became_active":"2024-04-25T10:21:43.214672+0000","last_became_peered":"2024-04-25T10:21:43.214672+0000","last_unstale":"2024-04-25T10:21:44.584927+0000","last_undegraded":"2024-04-25T10:21:44.584927+0000","last_fullsized":"2024-04-25T10:21:44.584927+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:20.776879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992010+0000","last_change":"2024-04-25T10:21:43.206666+0000","last_active":"2024-04-25T10:21:44.992010+0000","last_peered":"2024-04-25T10:21:44.992010+0000","last_clean":"2024-04-25T10:21:44.992010+0000","last_became_active":"2024-04-25T10:21:43.203595+0000","last_became_peered":"2024-04-25T10:21:43.203595+0000","last_unstale":"2024-04-25T10:21:44.992010+0000","last_undegraded":"2024-04-25T10:21:44.992010+0000","last_fullsized":"2024-04-25T10:21:44.992010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:56.377756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209295+0000","last_change":"2024-04-25T10:21:43.210607+0000","last_active":"2024-04-25T10:21:44.209295+0000","last_peered":"2024-04-25T10:21:44.209295+0000","last_clean":"2024-04-25T10:21:44.209295+0000","last_became_active":"2024-04-25T10:21:43.209928+0000","last_became_peered":"2024-04-25T10:21:43.209928+0000","last_unstale":"2024-04-25T10:21:44.209295+0000","last_undegraded":"2024-04-25T10:21:44.209295+0000","last_fullsized":"2024-04-25T10:21:44.209295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:23.217506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991563+0000","last_change":"2024-04-25T10:21:43.207662+0000","last_active":"2024-04-25T10:21:44.991563+0000","last_peered":"2024-04-25T10:21:44.991563+0000","last_clean":"2024-04-25T10:21:44.991563+0000","last_became_active":"2024-04-25T10:21:43.207398+0000","last_became_peered":"2024-04-25T10:21:43.207398+0000","last_unstale":"2024-04-25T10:21:44.991563+0000","last_undegraded":"2024-04-25T10:21:44.991563+0000","last_fullsized":"2024-04-25T10:21:44.991563+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:00.066361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201010+0000","last_change":"2024-04-25T10:21:43.209526+0000","last_active":"2024-04-25T10:21:44.201010+0000","last_peered":"2024-04-25T10:21:44.201010+0000","last_clean":"2024-04-25T10:21:44.201010+0000","last_became_active":"2024-04-25T10:21:43.209257+0000","last_became_peered":"2024-04-25T10:21:43.209257+0000","last_unstale":"2024-04-25T10:21:44.201010+0000","last_undegraded":"2024-04-25T10:21:44.201010+0000","last_fullsized":"2024-04-25T10:21:44.201010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:27.156541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992770+0000","last_change":"2024-04-25T10:21:43.209196+0000","last_active":"2024-04-25T10:21:44.992770+0000","last_peered":"2024-04-25T10:21:44.992770+0000","last_clean":"2024-04-25T10:21:44.992770+0000","last_became_active":"2024-04-25T10:21:43.208885+0000","last_became_peered":"2024-04-25T10:21:43.208885+0000","last_unstale":"2024-04-25T10:21:44.992770+0000","last_undegraded":"2024-04-25T10:21:44.992770+0000","last_fullsized":"2024-04-25T10:21:44.992770+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:34.691817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992313+0000","last_change":"2024-04-25T10:21:43.192882+0000","last_active":"2024-04-25T10:21:44.992313+0000","last_peered":"2024-04-25T10:21:44.992313+0000","last_clean":"2024-04-25T10:21:44.992313+0000","last_became_active":"2024-04-25T10:21:43.192607+0000","last_became_peered":"2024-04-25T10:21:43.192607+0000","last_unstale":"2024-04-25T10:21:44.992313+0000","last_undegraded":"2024-04-25T10:21:44.992313+0000","last_fullsized":"2024-04-25T10:21:44.992313+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:38.403042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209981+0000","last_change":"2024-04-25T10:21:43.201774+0000","last_active":"2024-04-25T10:21:44.209981+0000","last_peered":"2024-04-25T10:21:44.209981+0000","last_clean":"2024-04-25T10:21:44.209981+0000","last_became_active":"2024-04-25T10:21:43.201510+0000","last_became_peered":"2024-04-25T10:21:43.201510+0000","last_unstale":"2024-04-25T10:21:44.209981+0000","last_undegraded":"2024-04-25T10:21:44.209981+0000","last_fullsized":"2024-04-25T10:21:44.209981+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:50.548800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992099+0000","last_change":"2024-04-25T10:21:43.197596+0000","last_active":"2024-04-25T10:21:44.992099+0000","last_peered":"2024-04-25T10:21:44.992099+0000","last_clean":"2024-04-25T10:21:44.992099+0000","last_became_active":"2024-04-25T10:21:43.197155+0000","last_became_peered":"2024-04-25T10:21:43.197155+0000","last_unstale":"2024-04-25T10:21:44.992099+0000","last_undegraded":"2024-04-25T10:21:44.992099+0000","last_fullsized":"2024-04-25T10:21:44.992099+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:16.809818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991669+0000","last_change":"2024-04-25T10:21:43.197314+0000","last_active":"2024-04-25T10:21:44.991669+0000","last_peered":"2024-04-25T10:21:44.991669+0000","last_clean":"2024-04-25T10:21:44.991669+0000","last_became_active":"2024-04-25T10:21:43.197022+0000","last_became_peered":"2024-04-25T10:21:43.197022+0000","last_unstale":"2024-04-25T10:21:44.991669+0000","last_undegraded":"2024-04-25T10:21:44.991669+0000","last_fullsized":"2024-04-25T10:21:44.991669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:24.199287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584984+0000","last_change":"2024-04-25T10:21:43.218323+0000","last_active":"2024-04-25T10:21:44.584984+0000","last_peered":"2024-04-25T10:21:44.584984+0000","last_clean":"2024-04-25T10:21:44.584984+0000","last_became_active":"2024-04-25T10:21:43.217251+0000","last_became_peered":"2024-04-25T10:21:43.217251+0000","last_unstale":"2024-04-25T10:21:44.584984+0000","last_undegraded":"2024-04-25T10:21:44.584984+0000","last_fullsized":"2024-04-25T10:21:44.584984+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:06.693589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992500+0000","last_change":"2024-04-25T10:21:43.202091+0000","last_active":"2024-04-25T10:21:44.992500+0000","last_peered":"2024-04-25T10:21:44.992500+0000","last_clean":"2024-04-25T10:21:44.992500+0000","last_became_active":"2024-04-25T10:21:43.201459+0000","last_became_peered":"2024-04-25T10:21:43.201459+0000","last_unstale":"2024-04-25T10:21:44.992500+0000","last_undegraded":"2024-04-25T10:21:44.992500+0000","last_fullsized":"2024-04-25T10:21:44.992500+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.656126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992591+0000","last_change":"2024-04-25T10:21:43.211164+0000","last_active":"2024-04-25T10:21:44.992591+0000","last_peered":"2024-04-25T10:21:44.992591+0000","last_clean":"2024-04-25T10:21:44.992591+0000","last_became_active":"2024-04-25T10:21:43.210819+0000","last_became_peered":"2024-04-25T10:21:43.210819+0000","last_unstale":"2024-04-25T10:21:44.992591+0000","last_undegraded":"2024-04-25T10:21:44.992591+0000","last_fullsized":"2024-04-25T10:21:44.992591+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:06.693589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"123'4","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585169+0000","last_change":"2024-04-25T10:21:42.202833+0000","last_active":"2024-04-25T10:21:44.585169+0000","last_peered":"2024-04-25T10:21:44.585169+0000","last_clean":"2024-04-25T10:21:44.585169+0000","last_became_active":"2024-04-25T10:21:42.202514+0000","last_became_peered":"2024-04-25T10:21:42.202514+0000","last_unstale":"2024-04-25T10:21:44.585169+0000","last_undegraded":"2024-04-25T10:21:44.585169+0000","last_fullsized":"2024-04-25T10:21:44.585169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.653391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210194+0000","last_change":"2024-04-25T10:21:43.207078+0000","last_active":"2024-04-25T10:21:44.210194+0000","last_peered":"2024-04-25T10:21:44.210194+0000","last_clean":"2024-04-25T10:21:44.210194+0000","last_became_active":"2024-04-25T10:21:43.206798+0000","last_became_peered":"2024-04-25T10:21:43.206798+0000","last_unstale":"2024-04-25T10:21:44.210194+0000","last_undegraded":"2024-04-25T10:21:44.210194+0000","last_fullsized":"2024-04-25T10:21:44.210194+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:52.818109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210111+0000","last_change":"2024-04-25T10:21:42.196878+0000","last_active":"2024-04-25T10:21:44.210111+0000","last_peered":"2024-04-25T10:21:44.210111+0000","last_clean":"2024-04-25T10:21:44.210111+0000","last_became_active":"2024-04-25T10:21:42.196578+0000","last_became_peered":"2024-04-25T10:21:42.196578+0000","last_unstale":"2024-04-25T10:21:44.210111+0000","last_undegraded":"2024-04-25T10:21:44.210111+0000","last_fullsized":"2024-04-25T10:21:44.210111+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:21.077385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585367+0000","last_change":"2024-04-25T10:21:43.219063+0000","last_active":"2024-04-25T10:21:44.585367+0000","last_peered":"2024-04-25T10:21:44.585367+0000","last_clean":"2024-04-25T10:21:44.585367+0000","last_became_active":"2024-04-25T10:21:43.216590+0000","last_became_peered":"2024-04-25T10:21:43.216590+0000","last_unstale":"2024-04-25T10:21:44.585367+0000","last_undegraded":"2024-04-25T10:21:44.585367+0000","last_fullsized":"2024-04-25T10:21:44.585367+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:44.845476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992478+0000","last_change":"2024-04-25T10:21:42.197149+0000","last_active":"2024-04-25T10:21:44.992478+0000","last_peered":"2024-04-25T10:21:44.992478+0000","last_clean":"2024-04-25T10:21:44.992478+0000","last_became_active":"2024-04-25T10:21:42.196839+0000","last_became_peered":"2024-04-25T10:21:42.196839+0000","last_unstale":"2024-04-25T10:21:44.992478+0000","last_undegraded":"2024-04-25T10:21:44.992478+0000","last_fullsized":"2024-04-25T10:21:44.992478+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:19.774144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585194+0000","last_change":"2024-04-25T10:21:43.216038+0000","last_active":"2024-04-25T10:21:44.585194+0000","last_peered":"2024-04-25T10:21:44.585194+0000","last_clean":"2024-04-25T10:21:44.585194+0000","last_became_active":"2024-04-25T10:21:43.215755+0000","last_became_peered":"2024-04-25T10:21:43.215755+0000","last_unstale":"2024-04-25T10:21:44.585194+0000","last_undegraded":"2024-04-25T10:21:44.585194+0000","last_fullsized":"2024-04-25T10:21:44.585194+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:10.650657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200849+0000","last_change":"2024-04-25T10:21:42.196094+0000","last_active":"2024-04-25T10:21:44.200849+0000","last_peered":"2024-04-25T10:21:44.200849+0000","last_clean":"2024-04-25T10:21:44.200849+0000","last_became_active":"2024-04-25T10:21:42.195175+0000","last_became_peered":"2024-04-25T10:21:42.195175+0000","last_unstale":"2024-04-25T10:21:44.200849+0000","last_undegraded":"2024-04-25T10:21:44.200849+0000","last_fullsized":"2024-04-25T10:21:44.200849+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:13.617706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991302+0000","last_change":"2024-04-25T10:21:43.207403+0000","last_active":"2024-04-25T10:21:44.991302+0000","last_peered":"2024-04-25T10:21:44.991302+0000","last_clean":"2024-04-25T10:21:44.991302+0000","last_became_active":"2024-04-25T10:21:43.206240+0000","last_became_peered":"2024-04-25T10:21:43.206240+0000","last_unstale":"2024-04-25T10:21:44.991302+0000","last_undegraded":"2024-04-25T10:21:44.991302+0000","last_fullsized":"2024-04-25T10:21:44.991302+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:04.019536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200666+0000","last_change":"2024-04-25T10:21:42.193788+0000","last_active":"2024-04-25T10:21:44.200666+0000","last_peered":"2024-04-25T10:21:44.200666+0000","last_clean":"2024-04-25T10:21:44.200666+0000","last_became_active":"2024-04-25T10:21:42.192252+0000","last_became_peered":"2024-04-25T10:21:42.192252+0000","last_unstale":"2024-04-25T10:21:44.200666+0000","last_undegraded":"2024-04-25T10:21:44.200666+0000","last_fullsized":"2024-04-25T10:21:44.200666+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:38.319222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992688+0000","last_change":"2024-04-25T10:21:43.201685+0000","last_active":"2024-04-25T10:21:44.992688+0000","last_peered":"2024-04-25T10:21:44.992688+0000","last_clean":"2024-04-25T10:21:44.992688+0000","last_became_active":"2024-04-25T10:21:43.201337+0000","last_became_peered":"2024-04-25T10:21:43.201337+0000","last_unstale":"2024-04-25T10:21:44.992688+0000","last_undegraded":"2024-04-25T10:21:44.992688+0000","last_fullsized":"2024-04-25T10:21:44.992688+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:52.818109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991008+0000","last_change":"2024-04-25T10:21:42.194638+0000","last_active":"2024-04-25T10:21:44.991008+0000","last_peered":"2024-04-25T10:21:44.991008+0000","last_clean":"2024-04-25T10:21:44.991008+0000","last_became_active":"2024-04-25T10:21:42.194151+0000","last_became_peered":"2024-04-25T10:21:42.194151+0000","last_unstale":"2024-04-25T10:21:44.991008+0000","last_undegraded":"2024-04-25T10:21:44.991008+0000","last_fullsized":"2024-04-25T10:21:44.991008+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:25.524782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209375+0000","last_change":"2024-04-25T10:21:43.203179+0000","last_active":"2024-04-25T10:21:44.209375+0000","last_peered":"2024-04-25T10:21:44.209375+0000","last_clean":"2024-04-25T10:21:44.209375+0000","last_became_active":"2024-04-25T10:21:43.202761+0000","last_became_peered":"2024-04-25T10:21:43.202761+0000","last_unstale":"2024-04-25T10:21:44.209375+0000","last_undegraded":"2024-04-25T10:21:44.209375+0000","last_fullsized":"2024-04-25T10:21:44.209375+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:26.527517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992312+0000","last_change":"2024-04-25T10:21:42.196369+0000","last_active":"2024-04-25T10:21:44.992312+0000","last_peered":"2024-04-25T10:21:44.992312+0000","last_clean":"2024-04-25T10:21:44.992312+0000","last_became_active":"2024-04-25T10:21:42.196062+0000","last_became_peered":"2024-04-25T10:21:42.196062+0000","last_unstale":"2024-04-25T10:21:44.992312+0000","last_undegraded":"2024-04-25T10:21:44.992312+0000","last_fullsized":"2024-04-25T10:21:44.992312+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:20.415473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200544+0000","last_change":"2024-04-25T10:21:43.200621+0000","last_active":"2024-04-25T10:21:44.200544+0000","last_peered":"2024-04-25T10:21:44.200544+0000","last_clean":"2024-04-25T10:21:44.200544+0000","last_became_active":"2024-04-25T10:21:43.200333+0000","last_became_peered":"2024-04-25T10:21:43.200333+0000","last_unstale":"2024-04-25T10:21:44.200544+0000","last_undegraded":"2024-04-25T10:21:44.200544+0000","last_fullsized":"2024-04-25T10:21:44.200544+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:22.080120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200572+0000","last_change":"2024-04-25T10:21:42.192901+0000","last_active":"2024-04-25T10:21:44.200572+0000","last_peered":"2024-04-25T10:21:44.200572+0000","last_clean":"2024-04-25T10:21:44.200572+0000","last_became_active":"2024-04-25T10:21:42.192477+0000","last_became_peered":"2024-04-25T10:21:42.192477+0000","last_unstale":"2024-04-25T10:21:44.200572+0000","last_undegraded":"2024-04-25T10:21:44.200572+0000","last_fullsized":"2024-04-25T10:21:44.200572+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:15.807083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992497+0000","last_change":"2024-04-25T10:21:43.200054+0000","last_active":"2024-04-25T10:21:44.992497+0000","last_peered":"2024-04-25T10:21:44.992497+0000","last_clean":"2024-04-25T10:21:44.992497+0000","last_became_active":"2024-04-25T10:21:43.199382+0000","last_became_peered":"2024-04-25T10:21:43.199382+0000","last_unstale":"2024-04-25T10:21:44.992497+0000","last_undegraded":"2024-04-25T10:21:44.992497+0000","last_fullsized":"2024-04-25T10:21:44.992497+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:34.268092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210438+0000","last_change":"2024-04-25T10:21:42.195851+0000","last_active":"2024-04-25T10:21:44.210438+0000","last_peered":"2024-04-25T10:21:44.210438+0000","last_clean":"2024-04-25T10:21:44.210438+0000","last_became_active":"2024-04-25T10:21:42.194348+0000","last_became_peered":"2024-04-25T10:21:42.194348+0000","last_unstale":"2024-04-25T10:21:44.210438+0000","last_undegraded":"2024-04-25T10:21:44.210438+0000","last_fullsized":"2024-04-25T10:21:44.210438+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:32.323570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210389+0000","last_change":"2024-04-25T10:21:43.202665+0000","last_active":"2024-04-25T10:21:44.210389+0000","last_peered":"2024-04-25T10:21:44.210389+0000","last_clean":"2024-04-25T10:21:44.210389+0000","last_became_active":"2024-04-25T10:21:43.202399+0000","last_became_peered":"2024-04-25T10:21:43.202399+0000","last_unstale":"2024-04-25T10:21:44.210389+0000","last_undegraded":"2024-04-25T10:21:44.210389+0000","last_fullsized":"2024-04-25T10:21:44.210389+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.260659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992641+0000","last_change":"2024-04-25T10:21:42.199391+0000","last_active":"2024-04-25T10:21:44.992641+0000","last_peered":"2024-04-25T10:21:44.992641+0000","last_clean":"2024-04-25T10:21:44.992641+0000","last_became_active":"2024-04-25T10:21:42.197892+0000","last_became_peered":"2024-04-25T10:21:42.197892+0000","last_unstale":"2024-04-25T10:21:44.992641+0000","last_undegraded":"2024-04-25T10:21:44.992641+0000","last_fullsized":"2024-04-25T10:21:44.992641+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.453147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585455+0000","last_change":"2024-04-25T10:21:43.217477+0000","last_active":"2024-04-25T10:21:44.585455+0000","last_peered":"2024-04-25T10:21:44.585455+0000","last_clean":"2024-04-25T10:21:44.585455+0000","last_became_active":"2024-04-25T10:21:43.216872+0000","last_became_peered":"2024-04-25T10:21:43.216872+0000","last_unstale":"2024-04-25T10:21:44.585455+0000","last_undegraded":"2024-04-25T10:21:44.585455+0000","last_fullsized":"2024-04-25T10:21:44.585455+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:42.008315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"123'3","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209838+0000","last_change":"2024-04-25T10:21:42.191713+0000","last_active":"2024-04-25T10:21:44.209838+0000","last_peered":"2024-04-25T10:21:44.209838+0000","last_clean":"2024-04-25T10:21:44.209838+0000","last_became_active":"2024-04-25T10:21:42.191398+0000","last_became_peered":"2024-04-25T10:21:42.191398+0000","last_unstale":"2024-04-25T10:21:44.209838+0000","last_undegraded":"2024-04-25T10:21:44.209838+0000","last_fullsized":"2024-04-25T10:21:44.209838+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:07.927606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585244+0000","last_change":"2024-04-25T10:21:43.215513+0000","last_active":"2024-04-25T10:21:44.585244+0000","last_peered":"2024-04-25T10:21:44.585244+0000","last_clean":"2024-04-25T10:21:44.585244+0000","last_became_active":"2024-04-25T10:21:43.215212+0000","last_became_peered":"2024-04-25T10:21:43.215212+0000","last_unstale":"2024-04-25T10:21:44.585244+0000","last_undegraded":"2024-04-25T10:21:44.585244+0000","last_fullsized":"2024-04-25T10:21:44.585244+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.281285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"123'6","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209554+0000","last_change":"2024-04-25T10:21:42.200411+0000","last_active":"2024-04-25T10:21:44.209554+0000","last_peered":"2024-04-25T10:21:44.209554+0000","last_clean":"2024-04-25T10:21:44.209554+0000","last_became_active":"2024-04-25T10:21:42.200033+0000","last_became_peered":"2024-04-25T10:21:42.200033+0000","last_unstale":"2024-04-25T10:21:44.209554+0000","last_undegraded":"2024-04-25T10:21:44.209554+0000","last_fullsized":"2024-04-25T10:21:44.209554+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:01.918133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992172+0000","last_change":"2024-04-25T10:21:43.202751+0000","last_active":"2024-04-25T10:21:44.992172+0000","last_peered":"2024-04-25T10:21:44.992172+0000","last_clean":"2024-04-25T10:21:44.992172+0000","last_became_active":"2024-04-25T10:21:43.199393+0000","last_became_peered":"2024-04-25T10:21:43.199393+0000","last_unstale":"2024-04-25T10:21:44.992172+0000","last_undegraded":"2024-04-25T10:21:44.992172+0000","last_fullsized":"2024-04-25T10:21:44.992172+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.539372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991729+0000","last_change":"2024-04-25T10:21:42.194643+0000","last_active":"2024-04-25T10:21:44.991729+0000","last_peered":"2024-04-25T10:21:44.991729+0000","last_clean":"2024-04-25T10:21:44.991729+0000","last_became_active":"2024-04-25T10:21:42.194366+0000","last_became_peered":"2024-04-25T10:21:42.194366+0000","last_unstale":"2024-04-25T10:21:44.991729+0000","last_undegraded":"2024-04-25T10:21:44.991729+0000","last_fullsized":"2024-04-25T10:21:44.991729+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:54.311366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200508+0000","last_change":"2024-04-25T10:21:43.194160+0000","last_active":"2024-04-25T10:21:44.200508+0000","last_peered":"2024-04-25T10:21:44.200508+0000","last_clean":"2024-04-25T10:21:44.200508+0000","last_became_active":"2024-04-25T10:21:43.193866+0000","last_became_peered":"2024-04-25T10:21:43.193866+0000","last_unstale":"2024-04-25T10:21:44.200508+0000","last_undegraded":"2024-04-25T10:21:44.200508+0000","last_fullsized":"2024-04-25T10:21:44.200508+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:20.602942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200594+0000","last_change":"2024-04-25T10:21:42.197914+0000","last_active":"2024-04-25T10:21:44.200594+0000","last_peered":"2024-04-25T10:21:44.200594+0000","last_clean":"2024-04-25T10:21:44.200594+0000","last_became_active":"2024-04-25T10:21:42.197665+0000","last_became_peered":"2024-04-25T10:21:42.197665+0000","last_unstale":"2024-04-25T10:21:44.200594+0000","last_undegraded":"2024-04-25T10:21:44.200594+0000","last_fullsized":"2024-04-25T10:21:44.200594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:18.929708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209150+0000","last_change":"2024-04-25T10:21:43.210914+0000","last_active":"2024-04-25T10:21:44.209150+0000","last_peered":"2024-04-25T10:21:44.209150+0000","last_clean":"2024-04-25T10:21:44.209150+0000","last_became_active":"2024-04-25T10:21:43.210314+0000","last_became_peered":"2024-04-25T10:21:43.210314+0000","last_unstale":"2024-04-25T10:21:44.209150+0000","last_undegraded":"2024-04-25T10:21:44.209150+0000","last_fullsized":"2024-04-25T10:21:44.209150+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:13.774084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"123'8","reported_seq":22,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209168+0000","last_change":"2024-04-25T10:21:42.200339+0000","last_active":"2024-04-25T10:21:44.209168+0000","last_peered":"2024-04-25T10:21:44.209168+0000","last_clean":"2024-04-25T10:21:44.209168+0000","last_became_active":"2024-04-25T10:21:42.200009+0000","last_became_peered":"2024-04-25T10:21:42.200009+0000","last_unstale":"2024-04-25T10:21:44.209168+0000","last_undegraded":"2024-04-25T10:21:44.209168+0000","last_fullsized":"2024-04-25T10:21:44.209168+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.312727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210041+0000","last_change":"2024-04-25T10:21:43.208035+0000","last_active":"2024-04-25T10:21:44.210041+0000","last_peered":"2024-04-25T10:21:44.210041+0000","last_clean":"2024-04-25T10:21:44.210041+0000","last_became_active":"2024-04-25T10:21:43.207774+0000","last_became_peered":"2024-04-25T10:21:43.207774+0000","last_unstale":"2024-04-25T10:21:44.210041+0000","last_undegraded":"2024-04-25T10:21:44.210041+0000","last_fullsized":"2024-04-25T10:21:44.210041+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:20.455882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210495+0000","last_change":"2024-04-25T10:21:42.196789+0000","last_active":"2024-04-25T10:21:44.210495+0000","last_peered":"2024-04-25T10:21:44.210495+0000","last_clean":"2024-04-25T10:21:44.210495+0000","last_became_active":"2024-04-25T10:21:42.195840+0000","last_became_peered":"2024-04-25T10:21:42.195840+0000","last_unstale":"2024-04-25T10:21:44.210495+0000","last_undegraded":"2024-04-25T10:21:44.210495+0000","last_fullsized":"2024-04-25T10:21:44.210495+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:47.827547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210445+0000","last_change":"2024-04-25T10:21:43.201013+0000","last_active":"2024-04-25T10:21:44.210445+0000","last_peered":"2024-04-25T10:21:44.210445+0000","last_clean":"2024-04-25T10:21:44.210445+0000","last_became_active":"2024-04-25T10:21:43.200653+0000","last_became_peered":"2024-04-25T10:21:43.200653+0000","last_unstale":"2024-04-25T10:21:44.210445+0000","last_undegraded":"2024-04-25T10:21:44.210445+0000","last_fullsized":"2024-04-25T10:21:44.210445+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:25.152746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992753+0000","last_change":"2024-04-25T10:21:42.195596+0000","last_active":"2024-04-25T10:21:44.992753+0000","last_peered":"2024-04-25T10:21:44.992753+0000","last_clean":"2024-04-25T10:21:44.992753+0000","last_became_active":"2024-04-25T10:21:42.195342+0000","last_became_peered":"2024-04-25T10:21:42.195342+0000","last_unstale":"2024-04-25T10:21:44.992753+0000","last_undegraded":"2024-04-25T10:21:44.992753+0000","last_fullsized":"2024-04-25T10:21:44.992753+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:33.689082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991904+0000","last_change":"2024-04-25T10:21:43.205811+0000","last_active":"2024-04-25T10:21:44.991904+0000","last_peered":"2024-04-25T10:21:44.991904+0000","last_clean":"2024-04-25T10:21:44.991904+0000","last_became_active":"2024-04-25T10:21:43.204655+0000","last_became_peered":"2024-04-25T10:21:43.204655+0000","last_unstale":"2024-04-25T10:21:44.991904+0000","last_undegraded":"2024-04-25T10:21:44.991904+0000","last_fullsized":"2024-04-25T10:21:44.991904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:20.776879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992806+0000","last_change":"2024-04-25T10:21:42.194339+0000","last_active":"2024-04-25T10:21:44.992806+0000","last_peered":"2024-04-25T10:21:44.992806+0000","last_clean":"2024-04-25T10:21:44.992806+0000","last_became_active":"2024-04-25T10:21:42.193055+0000","last_became_peered":"2024-04-25T10:21:42.193055+0000","last_unstale":"2024-04-25T10:21:44.992806+0000","last_undegraded":"2024-04-25T10:21:44.992806+0000","last_fullsized":"2024-04-25T10:21:44.992806+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:21.749283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201184+0000","last_change":"2024-04-25T10:21:43.200940+0000","last_active":"2024-04-25T10:21:44.201184+0000","last_peered":"2024-04-25T10:21:44.201184+0000","last_clean":"2024-04-25T10:21:44.201184+0000","last_became_active":"2024-04-25T10:21:43.200205+0000","last_became_peered":"2024-04-25T10:21:43.200205+0000","last_unstale":"2024-04-25T10:21:44.201184+0000","last_undegraded":"2024-04-25T10:21:44.201184+0000","last_fullsized":"2024-04-25T10:21:44.201184+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.529868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209902+0000","last_change":"2024-04-25T10:21:42.197762+0000","last_active":"2024-04-25T10:21:44.209902+0000","last_peered":"2024-04-25T10:21:44.209902+0000","last_clean":"2024-04-25T10:21:44.209902+0000","last_became_active":"2024-04-25T10:21:42.196531+0000","last_became_peered":"2024-04-25T10:21:42.196531+0000","last_unstale":"2024-04-25T10:21:44.209902+0000","last_undegraded":"2024-04-25T10:21:44.209902+0000","last_fullsized":"2024-04-25T10:21:44.209902+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:40.069187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991719+0000","last_change":"2024-04-25T10:21:43.192082+0000","last_active":"2024-04-25T10:21:44.991719+0000","last_peered":"2024-04-25T10:21:44.991719+0000","last_clean":"2024-04-25T10:21:44.991719+0000","last_became_active":"2024-04-25T10:21:43.191796+0000","last_became_peered":"2024-04-25T10:21:43.191796+0000","last_unstale":"2024-04-25T10:21:44.991719+0000","last_undegraded":"2024-04-25T10:21:44.991719+0000","last_fullsized":"2024-04-25T10:21:44.991719+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:57.621592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209688+0000","last_change":"2024-04-25T10:21:42.197087+0000","last_active":"2024-04-25T10:21:44.209688+0000","last_peered":"2024-04-25T10:21:44.209688+0000","last_clean":"2024-04-25T10:21:44.209688+0000","last_became_active":"2024-04-25T10:21:42.196749+0000","last_became_peered":"2024-04-25T10:21:42.196749+0000","last_unstale":"2024-04-25T10:21:44.209688+0000","last_undegraded":"2024-04-25T10:21:44.209688+0000","last_fullsized":"2024-04-25T10:21:44.209688+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:19.600207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992621+0000","last_change":"2024-04-25T10:21:43.197995+0000","last_active":"2024-04-25T10:21:44.992621+0000","last_peered":"2024-04-25T10:21:44.992621+0000","last_clean":"2024-04-25T10:21:44.992621+0000","last_became_active":"2024-04-25T10:21:43.197712+0000","last_became_peered":"2024-04-25T10:21:43.197712+0000","last_unstale":"2024-04-25T10:21:44.992621+0000","last_undegraded":"2024-04-25T10:21:44.992621+0000","last_fullsized":"2024-04-25T10:21:44.992621+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:20.455882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991134+0000","last_change":"2024-04-25T10:21:42.198592+0000","last_active":"2024-04-25T10:21:44.991134+0000","last_peered":"2024-04-25T10:21:44.991134+0000","last_clean":"2024-04-25T10:21:44.991134+0000","last_became_active":"2024-04-25T10:21:42.198295+0000","last_became_peered":"2024-04-25T10:21:42.198295+0000","last_unstale":"2024-04-25T10:21:44.991134+0000","last_undegraded":"2024-04-25T10:21:44.991134+0000","last_fullsized":"2024-04-25T10:21:44.991134+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.653391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200513+0000","last_change":"2024-04-25T10:21:43.201320+0000","last_active":"2024-04-25T10:21:44.200513+0000","last_peered":"2024-04-25T10:21:44.200513+0000","last_clean":"2024-04-25T10:21:44.200513+0000","last_became_active":"2024-04-25T10:21:43.201073+0000","last_became_peered":"2024-04-25T10:21:43.201073+0000","last_unstale":"2024-04-25T10:21:44.200513+0000","last_undegraded":"2024-04-25T10:21:44.200513+0000","last_fullsized":"2024-04-25T10:21:44.200513+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:41.071922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.990953+0000","last_change":"2024-04-25T10:21:42.189561+0000","last_active":"2024-04-25T10:21:44.990953+0000","last_peered":"2024-04-25T10:21:44.990953+0000","last_clean":"2024-04-25T10:21:44.990953+0000","last_became_active":"2024-04-25T10:21:42.189232+0000","last_became_peered":"2024-04-25T10:21:42.189232+0000","last_unstale":"2024-04-25T10:21:44.990953+0000","last_undegraded":"2024-04-25T10:21:44.990953+0000","last_fullsized":"2024-04-25T10:21:44.990953+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:39.537368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584882+0000","last_change":"2024-04-25T10:21:43.218061+0000","last_active":"2024-04-25T10:21:44.584882+0000","last_peered":"2024-04-25T10:21:44.584882+0000","last_clean":"2024-04-25T10:21:44.584882+0000","last_became_active":"2024-04-25T10:21:43.217509+0000","last_became_peered":"2024-04-25T10:21:43.217509+0000","last_unstale":"2024-04-25T10:21:44.584882+0000","last_undegraded":"2024-04-25T10:21:44.584882+0000","last_fullsized":"2024-04-25T10:21:44.584882+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.480924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992594+0000","last_change":"2024-04-25T10:21:42.196842+0000","last_active":"2024-04-25T10:21:44.992594+0000","last_peered":"2024-04-25T10:21:44.992594+0000","last_clean":"2024-04-25T10:21:44.992594+0000","last_became_active":"2024-04-25T10:21:42.195580+0000","last_became_peered":"2024-04-25T10:21:42.195580+0000","last_unstale":"2024-04-25T10:21:44.992594+0000","last_undegraded":"2024-04-25T10:21:44.992594+0000","last_fullsized":"2024-04-25T10:21:44.992594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:30.394056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.993041+0000","last_change":"2024-04-25T10:21:43.196276+0000","last_active":"2024-04-25T10:21:44.993041+0000","last_peered":"2024-04-25T10:21:44.993041+0000","last_clean":"2024-04-25T10:21:44.993041+0000","last_became_active":"2024-04-25T10:21:43.195135+0000","last_became_peered":"2024-04-25T10:21:43.195135+0000","last_unstale":"2024-04-25T10:21:44.993041+0000","last_undegraded":"2024-04-25T10:21:44.993041+0000","last_fullsized":"2024-04-25T10:21:44.993041+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:20.912701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992633+0000","last_change":"2024-04-25T10:21:42.190602+0000","last_active":"2024-04-25T10:21:44.992633+0000","last_peered":"2024-04-25T10:21:44.992633+0000","last_clean":"2024-04-25T10:21:44.992633+0000","last_became_active":"2024-04-25T10:21:42.189881+0000","last_became_peered":"2024-04-25T10:21:42.189881+0000","last_unstale":"2024-04-25T10:21:44.992633+0000","last_undegraded":"2024-04-25T10:21:44.992633+0000","last_fullsized":"2024-04-25T10:21:44.992633+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:13.764058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992578+0000","last_change":"2024-04-25T10:21:43.200122+0000","last_active":"2024-04-25T10:21:44.992578+0000","last_peered":"2024-04-25T10:21:44.992578+0000","last_clean":"2024-04-25T10:21:44.992578+0000","last_became_active":"2024-04-25T10:21:43.199126+0000","last_became_peered":"2024-04-25T10:21:43.199126+0000","last_unstale":"2024-04-25T10:21:44.992578+0000","last_undegraded":"2024-04-25T10:21:44.992578+0000","last_fullsized":"2024-04-25T10:21:44.992578+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:39.063334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"124'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585078+0000","last_change":"2024-04-25T10:21:42.201935+0000","last_active":"2024-04-25T10:21:44.585078+0000","last_peered":"2024-04-25T10:21:44.585078+0000","last_clean":"2024-04-25T10:21:44.585078+0000","last_became_active":"2024-04-25T10:21:42.201513+0000","last_became_peered":"2024-04-25T10:21:42.201513+0000","last_unstale":"2024-04-25T10:21:44.585078+0000","last_undegraded":"2024-04-25T10:21:44.585078+0000","last_fullsized":"2024-04-25T10:21:44.585078+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:39.537368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200946+0000","last_change":"2024-04-25T10:21:43.215932+0000","last_active":"2024-04-25T10:21:44.200946+0000","last_peered":"2024-04-25T10:21:44.200946+0000","last_clean":"2024-04-25T10:21:44.200946+0000","last_became_active":"2024-04-25T10:21:43.215702+0000","last_became_peered":"2024-04-25T10:21:43.215702+0000","last_unstale":"2024-04-25T10:21:44.200946+0000","last_undegraded":"2024-04-25T10:21:44.200946+0000","last_fullsized":"2024-04-25T10:21:44.200946+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.830282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201096+0000","last_change":"2024-04-25T10:21:42.193210+0000","last_active":"2024-04-25T10:21:44.201096+0000","last_peered":"2024-04-25T10:21:44.201096+0000","last_clean":"2024-04-25T10:21:44.201096+0000","last_became_active":"2024-04-25T10:21:42.192904+0000","last_became_peered":"2024-04-25T10:21:42.192904+0000","last_unstale":"2024-04-25T10:21:44.201096+0000","last_undegraded":"2024-04-25T10:21:44.201096+0000","last_fullsized":"2024-04-25T10:21:44.201096+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:23.196552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992953+0000","last_change":"2024-04-25T10:21:43.194074+0000","last_active":"2024-04-25T10:21:44.992953+0000","last_peered":"2024-04-25T10:21:44.992953+0000","last_clean":"2024-04-25T10:21:44.992953+0000","last_became_active":"2024-04-25T10:21:43.193276+0000","last_became_peered":"2024-04-25T10:21:43.193276+0000","last_unstale":"2024-04-25T10:21:44.992953+0000","last_undegraded":"2024-04-25T10:21:44.992953+0000","last_fullsized":"2024-04-25T10:21:44.992953+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:05.448121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992035+0000","last_change":"2024-04-25T10:21:42.199465+0000","last_active":"2024-04-25T10:21:44.992035+0000","last_peered":"2024-04-25T10:21:44.992035+0000","last_clean":"2024-04-25T10:21:44.992035+0000","last_became_active":"2024-04-25T10:21:42.198919+0000","last_became_peered":"2024-04-25T10:21:42.198919+0000","last_unstale":"2024-04-25T10:21:44.992035+0000","last_undegraded":"2024-04-25T10:21:44.992035+0000","last_fullsized":"2024-04-25T10:21:44.992035+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:09.647923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584866+0000","last_change":"2024-04-25T10:21:43.219944+0000","last_active":"2024-04-25T10:21:44.584866+0000","last_peered":"2024-04-25T10:21:44.584866+0000","last_clean":"2024-04-25T10:21:44.584866+0000","last_became_active":"2024-04-25T10:21:43.219612+0000","last_became_peered":"2024-04-25T10:21:43.219612+0000","last_unstale":"2024-04-25T10:21:44.584866+0000","last_undegraded":"2024-04-25T10:21:44.584866+0000","last_fullsized":"2024-04-25T10:21:44.584866+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:26.788421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200873+0000","last_change":"2024-04-25T10:21:42.199189+0000","last_active":"2024-04-25T10:21:44.200873+0000","last_peered":"2024-04-25T10:21:44.200873+0000","last_clean":"2024-04-25T10:21:44.200873+0000","last_became_active":"2024-04-25T10:21:42.198850+0000","last_became_peered":"2024-04-25T10:21:42.198850+0000","last_unstale":"2024-04-25T10:21:44.200873+0000","last_undegraded":"2024-04-25T10:21:44.200873+0000","last_fullsized":"2024-04-25T10:21:44.200873+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:56.618857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209740+0000","last_change":"2024-04-25T10:21:43.194775+0000","last_active":"2024-04-25T10:21:44.209740+0000","last_peered":"2024-04-25T10:21:44.209740+0000","last_clean":"2024-04-25T10:21:44.209740+0000","last_became_active":"2024-04-25T10:21:43.194502+0000","last_became_peered":"2024-04-25T10:21:43.194502+0000","last_unstale":"2024-04-25T10:21:44.209740+0000","last_undegraded":"2024-04-25T10:21:44.209740+0000","last_fullsized":"2024-04-25T10:21:44.209740+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.529868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584972+0000","last_change":"2024-04-25T10:21:42.203621+0000","last_active":"2024-04-25T10:21:44.584972+0000","last_peered":"2024-04-25T10:21:44.584972+0000","last_clean":"2024-04-25T10:21:44.584972+0000","last_became_active":"2024-04-25T10:21:42.203335+0000","last_became_peered":"2024-04-25T10:21:42.203335+0000","last_unstale":"2024-04-25T10:21:44.584972+0000","last_undegraded":"2024-04-25T10:21:44.584972+0000","last_fullsized":"2024-04-25T10:21:44.584972+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:24.432362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991308+0000","last_change":"2024-04-25T10:21:43.209015+0000","last_active":"2024-04-25T10:21:44.991308+0000","last_peered":"2024-04-25T10:21:44.991308+0000","last_clean":"2024-04-25T10:21:44.991308+0000","last_became_active":"2024-04-25T10:21:43.204745+0000","last_became_peered":"2024-04-25T10:21:43.204745+0000","last_unstale":"2024-04-25T10:21:44.991308+0000","last_undegraded":"2024-04-25T10:21:44.991308+0000","last_fullsized":"2024-04-25T10:21:44.991308+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:26.788421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991068+0000","last_change":"2024-04-25T10:21:42.198952+0000","last_active":"2024-04-25T10:21:44.991068+0000","last_peered":"2024-04-25T10:21:44.991068+0000","last_clean":"2024-04-25T10:21:44.991068+0000","last_became_active":"2024-04-25T10:21:42.198699+0000","last_became_peered":"2024-04-25T10:21:42.198699+0000","last_unstale":"2024-04-25T10:21:44.991068+0000","last_undegraded":"2024-04-25T10:21:44.991068+0000","last_fullsized":"2024-04-25T10:21:44.991068+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:24.432362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200578+0000","last_change":"2024-04-25T10:21:43.208531+0000","last_active":"2024-04-25T10:21:44.200578+0000","last_peered":"2024-04-25T10:21:44.200578+0000","last_clean":"2024-04-25T10:21:44.200578+0000","last_became_active":"2024-04-25T10:21:43.208238+0000","last_became_peered":"2024-04-25T10:21:43.208238+0000","last_unstale":"2024-04-25T10:21:44.200578+0000","last_undegraded":"2024-04-25T10:21:44.200578+0000","last_fullsized":"2024-04-25T10:21:44.200578+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:33.326305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991652+0000","last_change":"2024-04-25T10:21:42.194731+0000","last_active":"2024-04-25T10:21:44.991652+0000","last_peered":"2024-04-25T10:21:44.991652+0000","last_clean":"2024-04-25T10:21:44.991652+0000","last_became_active":"2024-04-25T10:21:42.194445+0000","last_became_peered":"2024-04-25T10:21:42.194445+0000","last_unstale":"2024-04-25T10:21:44.991652+0000","last_undegraded":"2024-04-25T10:21:44.991652+0000","last_fullsized":"2024-04-25T10:21:44.991652+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:05.690854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584940+0000","last_change":"2024-04-25T10:21:43.215215+0000","last_active":"2024-04-25T10:21:44.584940+0000","last_peered":"2024-04-25T10:21:44.584940+0000","last_clean":"2024-04-25T10:21:44.584940+0000","last_became_active":"2024-04-25T10:21:43.214904+0000","last_became_peered":"2024-04-25T10:21:43.214904+0000","last_unstale":"2024-04-25T10:21:44.584940+0000","last_undegraded":"2024-04-25T10:21:44.584940+0000","last_fullsized":"2024-04-25T10:21:44.584940+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.396791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585221+0000","last_change":"2024-04-25T10:21:42.203881+0000","last_active":"2024-04-25T10:21:44.585221+0000","last_peered":"2024-04-25T10:21:44.585221+0000","last_clean":"2024-04-25T10:21:44.585221+0000","last_became_active":"2024-04-25T10:21:42.203521+0000","last_became_peered":"2024-04-25T10:21:42.203521+0000","last_unstale":"2024-04-25T10:21:44.585221+0000","last_undegraded":"2024-04-25T10:21:44.585221+0000","last_fullsized":"2024-04-25T10:21:44.585221+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.453147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":356,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584699+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:21:44.584699+0000","last_peered":"2024-04-25T10:21:44.584699+0000","last_clean":"2024-04-25T10:21:44.584699+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:21:44.584699+0000","last_undegraded":"2024-04-25T10:21:44.584699+0000","last_fullsized":"2024-04-25T10:21:44.584699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55508,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316332,"statfs":{"total":96636764160,"available":96579923968,"internally_reserved":0,"allocated":2166784,"data_stored":1266950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59080,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312760,"statfs":{"total":96636764160,"available":96576266240,"internally_reserved":0,"allocated":1560576,"data_stored":675944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46592,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325248,"statfs":{"total":96636764160,"available":96589053952,"internally_reserved":0,"allocated":1556480,"data_stored":674444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54908,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316932,"statfs":{"total":96636764160,"available":96580538368,"internally_reserved":0,"allocated":1552384,"data_stored":674402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1556480,"data_stored":674492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50752,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321088,"statfs":{"total":96636764160,"available":96584794112,"internally_reserved":0,"allocated":1556480,"data_stored":674948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77116,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294724,"statfs":{"total":96636764160,"available":96557797376,"internally_reserved":0,"allocated":1548288,"data_stored":674332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2142208,"data_stored":1264700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:21:47.347 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:21:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a16a00640 1 Processor -- start 2024-04-25T10:21:47.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a16a00640 1 -- start start 2024-04-25T10:21:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a16a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a1015e050 0x7f3a100b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a16a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a100cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a16a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a100b1850 con 0x7f3a100d1820 2024-04-25T10:21:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a16a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a100b1af0 con 0x7f3a1015e050 2024-04-25T10:21:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a16a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100b1dd0 con 0x7f3a100b13e0 2024-04-25T10:21:47.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a14a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a100cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a07e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a1015e050 0x7f3a100b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a0fe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f3a100d1820 0x7f3a100d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:50168/0 (socket says 172.21.15.87:50168) 2024-04-25T10:21:47.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a14a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a100cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:51410/0 (socket says 172.21.15.87:51410) 2024-04-25T10:21:47.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a0fe00640 1 -- 172.21.15.87:0/4030871343 learned_addr learned my addr 172.21.15.87:0/4030871343 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:47.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a07e00640 1 -- 172.21.15.87:0/4030871343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 msgr2=0x7f3a100cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a07e00640 1 --2- 172.21.15.87:0/4030871343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a100cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a07e00640 1 -- 172.21.15.87:0/4030871343 >> v1:172.21.15.169:6789/0 conn(0x7f3a100d1820 legacy=0x7f3a100d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.482+0000 7f3a07e00640 1 -- 172.21.15.87:0/4030871343 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100b20b0 con 0x7f3a1015e050 2024-04-25T10:21:47.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a07e00640 1 --2- 172.21.15.87:0/4030871343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a100b0ea0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3383a87ec34869da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a0f400640 1 -- 172.21.15.87:0/4030871343 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80ab610 con 0x7f3a1015e050 2024-04-25T10:21:47.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a0f400640 1 -- 172.21.15.87:0/4030871343 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f80b0070 con 0x7f3a1015e050 2024-04-25T10:21:47.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a0f400640 1 -- 172.21.15.87:0/4030871343 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80ab9d0 con 0x7f3a1015e050 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 -- 172.21.15.87:0/4030871343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 msgr2=0x7f3a100b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 --2- 172.21.15.87:0/4030871343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a100b0ea0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801c410 comp rx=0 tx=0).stop 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 -- 172.21.15.87:0/4030871343 shutdown_connections 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 --2- 172.21.15.87:0/4030871343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a100cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 --2- 172.21.15.87:0/4030871343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a100b0ea0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 -- 172.21.15.87:0/4030871343 >> 172.21.15.87:0/4030871343 conn(0x7f3a1009a9d0 msgr2=0x7f3a1009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 -- 172.21.15.87:0/4030871343 shutdown_connections 2024-04-25T10:21:47.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.486+0000 7f3a16a00640 1 -- 172.21.15.87:0/4030871343 wait complete. 2024-04-25T10:21:47.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 Processor -- start 2024-04-25T10:21:47.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 -- start start 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a101501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a100d1820 0x7f3a10155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a10156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100d0dd0 con 0x7f3a100d1820 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100d0c50 con 0x7f3a1015e050 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100d0f50 con 0x7f3a100b13e0 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a100d1820 0x7f3a10155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a100d1820 0x7f3a10155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59674/0 (socket says 172.21.15.87:59674) 2024-04-25T10:21:47.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 -- 172.21.15.87:0/2072425524 learned_addr learned my addr 172.21.15.87:0/2072425524 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a0fe00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a101501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a14a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a10156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 -- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 msgr2=0x7f3a101501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a101501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 -- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 msgr2=0x7f3a10156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a10156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 -- 172.21.15.87:0/2072425524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a101f9d50 con 0x7f3a100d1820 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a14a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a10156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a07e00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a100d1820 0x7f3a10155cd0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ad040 tx=0x7f39f801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a0cc00640 1 -- 172.21.15.87:0/2072425524 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80bf610 con 0x7f3a100d1820 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a0cc00640 1 -- 172.21.15.87:0/2072425524 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f80bf7d0 con 0x7f3a100d1820 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a101f9fc0 con 0x7f3a100d1820 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a0cc00640 1 -- 172.21.15.87:0/2072425524 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80bfbc0 con 0x7f3a100d1820 2024-04-25T10:21:47.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1013f6a0 con 0x7f3a100d1820 2024-04-25T10:21:47.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.490+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1004ef30 con 0x7f3a100d1820 2024-04-25T10:21:47.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.494+0000 7f3a0cc00640 1 -- 172.21.15.87:0/2072425524 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39f80bd020 con 0x7f3a100d1820 2024-04-25T10:21:47.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.494+0000 7f3a0cc00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39ec066f60 0x7f39ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.494+0000 7f3a0cc00640 1 -- 172.21.15.87:0/2072425524 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39f811f650 con 0x7f3a100d1820 2024-04-25T10:21:47.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.494+0000 7f3a0fe00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39ec066f60 0x7f39ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.494+0000 7f3a0fe00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39ec066f60 0x7f39ec069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000f70 tx=0x7f3a00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.498+0000 7f3a0cc00640 1 -- 172.21.15.87:0/2072425524 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39f80a8050 con 0x7f3a100d1820 2024-04-25T10:21:47.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.650+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a100741c0 con 0x7f39ec066f60 2024-04-25T10:21:47.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.654+0000 7f3a0cc00640 1 -- 172.21.15.87:0/2072425524 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7f3a100741c0 con 0x7f39ec066f60 2024-04-25T10:21:47.661 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:47.662 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39ec066f60 msgr2=0x7f39ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39ec066f60 0x7f39ec069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000f70 tx=0x7f3a00009040 comp rx=0 tx=0).stop 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a100d1820 msgr2=0x7f3a10155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a100d1820 0x7f3a10155cd0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ad040 tx=0x7f39f801ac50 comp rx=0 tx=0).stop 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 shutdown_connections 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39ec066f60 0x7f39ec069420 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1015e050 0x7f3a10156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a100d1820 0x7f3a10155cd0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 --2- 172.21.15.87:0/2072425524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a100b13e0 0x7f3a101501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 >> 172.21.15.87:0/2072425524 conn(0x7f3a1009a9d0 msgr2=0x7f3a1006e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 shutdown_connections 2024-04-25T10:21:47.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.658+0000 7f3a16a00640 1 -- 172.21.15.87:0/2072425524 wait complete. 2024-04-25T10:21:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-04-25T10:21:47.243532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":450192,"kb_used_data":13320,"kb_used_omap":12,"kb_used_meta":436723,"kb_avail":754524528,"statfs":{"total":773094113280,"available":772633116672,"internally_reserved":0,"allocated":13639680,"data_stored":6580212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":447204913},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001155"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992665+0000","last_change":"2024-04-25T10:21:43.201253+0000","last_active":"2024-04-25T10:21:44.992665+0000","last_peered":"2024-04-25T10:21:44.992665+0000","last_clean":"2024-04-25T10:21:44.992665+0000","last_became_active":"2024-04-25T10:21:43.201002+0000","last_became_peered":"2024-04-25T10:21:43.201002+0000","last_unstale":"2024-04-25T10:21:44.992665+0000","last_undegraded":"2024-04-25T10:21:44.992665+0000","last_fullsized":"2024-04-25T10:21:44.992665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:04.595700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201055+0000","last_change":"2024-04-25T10:21:43.194524+0000","last_active":"2024-04-25T10:21:44.201055+0000","last_peered":"2024-04-25T10:21:44.201055+0000","last_clean":"2024-04-25T10:21:44.201055+0000","last_became_active":"2024-04-25T10:21:43.194058+0000","last_became_peered":"2024-04-25T10:21:43.194058+0000","last_unstale":"2024-04-25T10:21:44.201055+0000","last_undegraded":"2024-04-25T10:21:44.201055+0000","last_fullsized":"2024-04-25T10:21:44.201055+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:30.975965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209810+0000","last_change":"2024-04-25T10:21:43.202628+0000","last_active":"2024-04-25T10:21:44.209810+0000","last_peered":"2024-04-25T10:21:44.209810+0000","last_clean":"2024-04-25T10:21:44.209810+0000","last_became_active":"2024-04-25T10:21:43.202306+0000","last_became_peered":"2024-04-25T10:21:43.202306+0000","last_unstale":"2024-04-25T10:21:44.209810+0000","last_undegraded":"2024-04-25T10:21:44.209810+0000","last_fullsized":"2024-04-25T10:21:44.209810+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.656126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992015+0000","last_change":"2024-04-25T10:21:43.217095+0000","last_active":"2024-04-25T10:21:44.992015+0000","last_peered":"2024-04-25T10:21:44.992015+0000","last_clean":"2024-04-25T10:21:44.992015+0000","last_became_active":"2024-04-25T10:21:43.216779+0000","last_became_peered":"2024-04-25T10:21:43.216779+0000","last_unstale":"2024-04-25T10:21:44.992015+0000","last_undegraded":"2024-04-25T10:21:44.992015+0000","last_fullsized":"2024-04-25T10:21:44.992015+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:14.620441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585141+0000","last_change":"2024-04-25T10:21:43.218462+0000","last_active":"2024-04-25T10:21:44.585141+0000","last_peered":"2024-04-25T10:21:44.585141+0000","last_clean":"2024-04-25T10:21:44.585141+0000","last_became_active":"2024-04-25T10:21:43.217837+0000","last_became_peered":"2024-04-25T10:21:43.217837+0000","last_unstale":"2024-04-25T10:21:44.585141+0000","last_undegraded":"2024-04-25T10:21:44.585141+0000","last_fullsized":"2024-04-25T10:21:44.585141+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:34.691817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992738+0000","last_change":"2024-04-25T10:21:43.204095+0000","last_active":"2024-04-25T10:21:44.992738+0000","last_peered":"2024-04-25T10:21:44.992738+0000","last_clean":"2024-04-25T10:21:44.992738+0000","last_became_active":"2024-04-25T10:21:43.203806+0000","last_became_peered":"2024-04-25T10:21:43.203806+0000","last_unstale":"2024-04-25T10:21:44.992738+0000","last_undegraded":"2024-04-25T10:21:44.992738+0000","last_fullsized":"2024-04-25T10:21:44.992738+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:32.101037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992502+0000","last_change":"2024-04-25T10:21:43.203046+0000","last_active":"2024-04-25T10:21:44.992502+0000","last_peered":"2024-04-25T10:21:44.992502+0000","last_clean":"2024-04-25T10:21:44.992502+0000","last_became_active":"2024-04-25T10:21:43.201225+0000","last_became_peered":"2024-04-25T10:21:43.201225+0000","last_unstale":"2024-04-25T10:21:44.992502+0000","last_undegraded":"2024-04-25T10:21:44.992502+0000","last_fullsized":"2024-04-25T10:21:44.992502+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:25.435097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209032+0000","last_change":"2024-04-25T10:21:43.203113+0000","last_active":"2024-04-25T10:21:44.209032+0000","last_peered":"2024-04-25T10:21:44.209032+0000","last_clean":"2024-04-25T10:21:44.209032+0000","last_became_active":"2024-04-25T10:21:43.202741+0000","last_became_peered":"2024-04-25T10:21:43.202741+0000","last_unstale":"2024-04-25T10:21:44.209032+0000","last_undegraded":"2024-04-25T10:21:44.209032+0000","last_fullsized":"2024-04-25T10:21:44.209032+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:00.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210031+0000","last_change":"2024-04-25T10:21:43.216045+0000","last_active":"2024-04-25T10:21:44.210031+0000","last_peered":"2024-04-25T10:21:44.210031+0000","last_clean":"2024-04-25T10:21:44.210031+0000","last_became_active":"2024-04-25T10:21:43.215756+0000","last_became_peered":"2024-04-25T10:21:43.215756+0000","last_unstale":"2024-04-25T10:21:44.210031+0000","last_undegraded":"2024-04-25T10:21:44.210031+0000","last_fullsized":"2024-04-25T10:21:44.210031+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:25.435097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201041+0000","last_change":"2024-04-25T10:21:43.198121+0000","last_active":"2024-04-25T10:21:44.201041+0000","last_peered":"2024-04-25T10:21:44.201041+0000","last_clean":"2024-04-25T10:21:44.201041+0000","last_became_active":"2024-04-25T10:21:43.195957+0000","last_became_peered":"2024-04-25T10:21:43.195957+0000","last_unstale":"2024-04-25T10:21:44.201041+0000","last_undegraded":"2024-04-25T10:21:44.201041+0000","last_fullsized":"2024-04-25T10:21:44.201041+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:58.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992411+0000","last_change":"2024-04-25T10:21:43.207398+0000","last_active":"2024-04-25T10:21:44.992411+0000","last_peered":"2024-04-25T10:21:44.992411+0000","last_clean":"2024-04-25T10:21:44.992411+0000","last_became_active":"2024-04-25T10:21:43.207112+0000","last_became_peered":"2024-04-25T10:21:43.207112+0000","last_unstale":"2024-04-25T10:21:44.992411+0000","last_undegraded":"2024-04-25T10:21:44.992411+0000","last_fullsized":"2024-04-25T10:21:44.992411+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:58.483697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209506+0000","last_change":"2024-04-25T10:21:43.216581+0000","last_active":"2024-04-25T10:21:44.209506+0000","last_peered":"2024-04-25T10:21:44.209506+0000","last_clean":"2024-04-25T10:21:44.209506+0000","last_became_active":"2024-04-25T10:21:43.214485+0000","last_became_peered":"2024-04-25T10:21:43.214485+0000","last_unstale":"2024-04-25T10:21:44.209506+0000","last_undegraded":"2024-04-25T10:21:44.209506+0000","last_fullsized":"2024-04-25T10:21:44.209506+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:40.540103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992016+0000","last_change":"2024-04-25T10:21:43.203020+0000","last_active":"2024-04-25T10:21:44.992016+0000","last_peered":"2024-04-25T10:21:44.992016+0000","last_clean":"2024-04-25T10:21:44.992016+0000","last_became_active":"2024-04-25T10:21:43.199772+0000","last_became_peered":"2024-04-25T10:21:43.199772+0000","last_unstale":"2024-04-25T10:21:44.992016+0000","last_undegraded":"2024-04-25T10:21:44.992016+0000","last_fullsized":"2024-04-25T10:21:44.992016+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:39.321957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201070+0000","last_change":"2024-04-25T10:21:43.202143+0000","last_active":"2024-04-25T10:21:44.201070+0000","last_peered":"2024-04-25T10:21:44.201070+0000","last_clean":"2024-04-25T10:21:44.201070+0000","last_became_active":"2024-04-25T10:21:43.199517+0000","last_became_peered":"2024-04-25T10:21:43.199517+0000","last_unstale":"2024-04-25T10:21:44.201070+0000","last_undegraded":"2024-04-25T10:21:44.201070+0000","last_fullsized":"2024-04-25T10:21:44.201070+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:11.257163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991630+0000","last_change":"2024-04-25T10:21:43.199033+0000","last_active":"2024-04-25T10:21:44.991630+0000","last_peered":"2024-04-25T10:21:44.991630+0000","last_clean":"2024-04-25T10:21:44.991630+0000","last_became_active":"2024-04-25T10:21:43.198661+0000","last_became_peered":"2024-04-25T10:21:43.198661+0000","last_unstale":"2024-04-25T10:21:44.991630+0000","last_undegraded":"2024-04-25T10:21:44.991630+0000","last_fullsized":"2024-04-25T10:21:44.991630+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:19.932443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991087+0000","last_change":"2024-04-25T10:21:43.208744+0000","last_active":"2024-04-25T10:21:44.991087+0000","last_peered":"2024-04-25T10:21:44.991087+0000","last_clean":"2024-04-25T10:21:44.991087+0000","last_became_active":"2024-04-25T10:21:43.206607+0000","last_became_peered":"2024-04-25T10:21:43.206607+0000","last_unstale":"2024-04-25T10:21:44.991087+0000","last_undegraded":"2024-04-25T10:21:44.991087+0000","last_fullsized":"2024-04-25T10:21:44.991087+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:52.724378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992962+0000","last_change":"2024-04-25T10:21:43.211416+0000","last_active":"2024-04-25T10:21:44.992962+0000","last_peered":"2024-04-25T10:21:44.992962+0000","last_clean":"2024-04-25T10:21:44.992962+0000","last_became_active":"2024-04-25T10:21:43.211143+0000","last_became_peered":"2024-04-25T10:21:43.211143+0000","last_unstale":"2024-04-25T10:21:44.992962+0000","last_undegraded":"2024-04-25T10:21:44.992962+0000","last_fullsized":"2024-04-25T10:21:44.992962+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:09.370407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992354+0000","last_change":"2024-04-25T10:21:43.207544+0000","last_active":"2024-04-25T10:21:44.992354+0000","last_peered":"2024-04-25T10:21:44.992354+0000","last_clean":"2024-04-25T10:21:44.992354+0000","last_became_active":"2024-04-25T10:21:43.206225+0000","last_became_peered":"2024-04-25T10:21:43.206225+0000","last_unstale":"2024-04-25T10:21:44.992354+0000","last_undegraded":"2024-04-25T10:21:44.992354+0000","last_fullsized":"2024-04-25T10:21:44.992354+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.396791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992863+0000","last_change":"2024-04-25T10:21:43.203032+0000","last_active":"2024-04-25T10:21:44.992863+0000","last_peered":"2024-04-25T10:21:44.992863+0000","last_clean":"2024-04-25T10:21:44.992863+0000","last_became_active":"2024-04-25T10:21:43.202569+0000","last_became_peered":"2024-04-25T10:21:43.202569+0000","last_unstale":"2024-04-25T10:21:44.992863+0000","last_undegraded":"2024-04-25T10:21:44.992863+0000","last_fullsized":"2024-04-25T10:21:44.992863+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:44.429556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991771+0000","last_change":"2024-04-25T10:21:43.209869+0000","last_active":"2024-04-25T10:21:44.991771+0000","last_peered":"2024-04-25T10:21:44.991771+0000","last_clean":"2024-04-25T10:21:44.991771+0000","last_became_active":"2024-04-25T10:21:43.209621+0000","last_became_peered":"2024-04-25T10:21:43.209621+0000","last_unstale":"2024-04-25T10:21:44.991771+0000","last_undegraded":"2024-04-25T10:21:44.991771+0000","last_fullsized":"2024-04-25T10:21:44.991771+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.480924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584927+0000","last_change":"2024-04-25T10:21:43.215047+0000","last_active":"2024-04-25T10:21:44.584927+0000","last_peered":"2024-04-25T10:21:44.584927+0000","last_clean":"2024-04-25T10:21:44.584927+0000","last_became_active":"2024-04-25T10:21:43.214672+0000","last_became_peered":"2024-04-25T10:21:43.214672+0000","last_unstale":"2024-04-25T10:21:44.584927+0000","last_undegraded":"2024-04-25T10:21:44.584927+0000","last_fullsized":"2024-04-25T10:21:44.584927+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:20.776879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992010+0000","last_change":"2024-04-25T10:21:43.206666+0000","last_active":"2024-04-25T10:21:44.992010+0000","last_peered":"2024-04-25T10:21:44.992010+0000","last_clean":"2024-04-25T10:21:44.992010+0000","last_became_active":"2024-04-25T10:21:43.203595+0000","last_became_peered":"2024-04-25T10:21:43.203595+0000","last_unstale":"2024-04-25T10:21:44.992010+0000","last_undegraded":"2024-04-25T10:21:44.992010+0000","last_fullsized":"2024-04-25T10:21:44.992010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:56.377756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209295+0000","last_change":"2024-04-25T10:21:43.210607+0000","last_active":"2024-04-25T10:21:44.209295+0000","last_peered":"2024-04-25T10:21:44.209295+0000","last_clean":"2024-04-25T10:21:44.209295+0000","last_became_active":"2024-04-25T10:21:43.209928+0000","last_became_peered":"2024-04-25T10:21:43.209928+0000","last_unstale":"2024-04-25T10:21:44.209295+0000","last_undegraded":"2024-04-25T10:21:44.209295+0000","last_fullsized":"2024-04-25T10:21:44.209295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:23.217506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991563+0000","last_change":"2024-04-25T10:21:43.207662+0000","last_active":"2024-04-25T10:21:44.991563+0000","last_peered":"2024-04-25T10:21:44.991563+0000","last_clean":"2024-04-25T10:21:44.991563+0000","last_became_active":"2024-04-25T10:21:43.207398+0000","last_became_peered":"2024-04-25T10:21:43.207398+0000","last_unstale":"2024-04-25T10:21:44.991563+0000","last_undegraded":"2024-04-25T10:21:44.991563+0000","last_fullsized":"2024-04-25T10:21:44.991563+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:00.066361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201010+0000","last_change":"2024-04-25T10:21:43.209526+0000","last_active":"2024-04-25T10:21:44.201010+0000","last_peered":"2024-04-25T10:21:44.201010+0000","last_clean":"2024-04-25T10:21:44.201010+0000","last_became_active":"2024-04-25T10:21:43.209257+0000","last_became_peered":"2024-04-25T10:21:43.209257+0000","last_unstale":"2024-04-25T10:21:44.201010+0000","last_undegraded":"2024-04-25T10:21:44.201010+0000","last_fullsized":"2024-04-25T10:21:44.201010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:27.156541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992770+0000","last_change":"2024-04-25T10:21:43.209196+0000","last_active":"2024-04-25T10:21:44.992770+0000","last_peered":"2024-04-25T10:21:44.992770+0000","last_clean":"2024-04-25T10:21:44.992770+0000","last_became_active":"2024-04-25T10:21:43.208885+0000","last_became_peered":"2024-04-25T10:21:43.208885+0000","last_unstale":"2024-04-25T10:21:44.992770+0000","last_undegraded":"2024-04-25T10:21:44.992770+0000","last_fullsized":"2024-04-25T10:21:44.992770+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:34.691817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992313+0000","last_change":"2024-04-25T10:21:43.192882+0000","last_active":"2024-04-25T10:21:44.992313+0000","last_peered":"2024-04-25T10:21:44.992313+0000","last_clean":"2024-04-25T10:21:44.992313+0000","last_became_active":"2024-04-25T10:21:43.192607+0000","last_became_peered":"2024-04-25T10:21:43.192607+0000","last_unstale":"2024-04-25T10:21:44.992313+0000","last_undegraded":"2024-04-25T10:21:44.992313+0000","last_fullsized":"2024-04-25T10:21:44.992313+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:38.403042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209981+0000","last_change":"2024-04-25T10:21:43.201774+0000","last_active":"2024-04-25T10:21:44.209981+0000","last_peered":"2024-04-25T10:21:44.209981+0000","last_clean":"2024-04-25T10:21:44.209981+0000","last_became_active":"2024-04-25T10:21:43.201510+0000","last_became_peered":"2024-04-25T10:21:43.201510+0000","last_unstale":"2024-04-25T10:21:44.209981+0000","last_undegraded":"2024-04-25T10:21:44.209981+0000","last_fullsized":"2024-04-25T10:21:44.209981+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:50.548800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992099+0000","last_change":"2024-04-25T10:21:43.197596+0000","last_active":"2024-04-25T10:21:44.992099+0000","last_peered":"2024-04-25T10:21:44.992099+0000","last_clean":"2024-04-25T10:21:44.992099+0000","last_became_active":"2024-04-25T10:21:43.197155+0000","last_became_peered":"2024-04-25T10:21:43.197155+0000","last_unstale":"2024-04-25T10:21:44.992099+0000","last_undegraded":"2024-04-25T10:21:44.992099+0000","last_fullsized":"2024-04-25T10:21:44.992099+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:16.809818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991669+0000","last_change":"2024-04-25T10:21:43.197314+0000","last_active":"2024-04-25T10:21:44.991669+0000","last_peered":"2024-04-25T10:21:44.991669+0000","last_clean":"2024-04-25T10:21:44.991669+0000","last_became_active":"2024-04-25T10:21:43.197022+0000","last_became_peered":"2024-04-25T10:21:43.197022+0000","last_unstale":"2024-04-25T10:21:44.991669+0000","last_undegraded":"2024-04-25T10:21:44.991669+0000","last_fullsized":"2024-04-25T10:21:44.991669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:24.199287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584984+0000","last_change":"2024-04-25T10:21:43.218323+0000","last_active":"2024-04-25T10:21:44.584984+0000","last_peered":"2024-04-25T10:21:44.584984+0000","last_clean":"2024-04-25T10:21:44.584984+0000","last_became_active":"2024-04-25T10:21:43.217251+0000","last_became_peered":"2024-04-25T10:21:43.217251+0000","last_unstale":"2024-04-25T10:21:44.584984+0000","last_undegraded":"2024-04-25T10:21:44.584984+0000","last_fullsized":"2024-04-25T10:21:44.584984+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:06.693589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992500+0000","last_change":"2024-04-25T10:21:43.202091+0000","last_active":"2024-04-25T10:21:44.992500+0000","last_peered":"2024-04-25T10:21:44.992500+0000","last_clean":"2024-04-25T10:21:44.992500+0000","last_became_active":"2024-04-25T10:21:43.201459+0000","last_became_peered":"2024-04-25T10:21:43.201459+0000","last_unstale":"2024-04-25T10:21:44.992500+0000","last_undegraded":"2024-04-25T10:21:44.992500+0000","last_fullsized":"2024-04-25T10:21:44.992500+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.656126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992591+0000","last_change":"2024-04-25T10:21:43.211164+0000","last_active":"2024-04-25T10:21:44.992591+0000","last_peered":"2024-04-25T10:21:44.992591+0000","last_clean":"2024-04-25T10:21:44.992591+0000","last_became_active":"2024-04-25T10:21:43.210819+0000","last_became_peered":"2024-04-25T10:21:43.210819+0000","last_unstale":"2024-04-25T10:21:44.992591+0000","last_undegraded":"2024-04-25T10:21:44.992591+0000","last_fullsized":"2024-04-25T10:21:44.992591+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:06.693589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"123'4","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585169+0000","last_change":"2024-04-25T10:21:42.202833+0000","last_active":"2024-04-25T10:21:44.585169+0000","last_peered":"2024-04-25T10:21:44.585169+0000","last_clean":"2024-04-25T10:21:44.585169+0000","last_became_active":"2024-04-25T10:21:42.202514+0000","last_became_peered":"2024-04-25T10:21:42.202514+0000","last_unstale":"2024-04-25T10:21:44.585169+0000","last_undegraded":"2024-04-25T10:21:44.585169+0000","last_fullsized":"2024-04-25T10:21:44.585169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.653391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210194+0000","last_change":"2024-04-25T10:21:43.207078+0000","last_active":"2024-04-25T10:21:44.210194+0000","last_peered":"2024-04-25T10:21:44.210194+0000","last_clean":"2024-04-25T10:21:44.210194+0000","last_became_active":"2024-04-25T10:21:43.206798+0000","last_became_peered":"2024-04-25T10:21:43.206798+0000","last_unstale":"2024-04-25T10:21:44.210194+0000","last_undegraded":"2024-04-25T10:21:44.210194+0000","last_fullsized":"2024-04-25T10:21:44.210194+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:52.818109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210111+0000","last_change":"2024-04-25T10:21:42.196878+0000","last_active":"2024-04-25T10:21:44.210111+0000","last_peered":"2024-04-25T10:21:44.210111+0000","last_clean":"2024-04-25T10:21:44.210111+0000","last_became_active":"2024-04-25T10:21:42.196578+0000","last_became_peered":"2024-04-25T10:21:42.196578+0000","last_unstale":"2024-04-25T10:21:44.210111+0000","last_undegraded":"2024-04-25T10:21:44.210111+0000","last_fullsized":"2024-04-25T10:21:44.210111+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:21.077385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585367+0000","last_change":"2024-04-25T10:21:43.219063+0000","last_active":"2024-04-25T10:21:44.585367+0000","last_peered":"2024-04-25T10:21:44.585367+0000","last_clean":"2024-04-25T10:21:44.585367+0000","last_became_active":"2024-04-25T10:21:43.216590+0000","last_became_peered":"2024-04-25T10:21:43.216590+0000","last_unstale":"2024-04-25T10:21:44.585367+0000","last_undegraded":"2024-04-25T10:21:44.585367+0000","last_fullsized":"2024-04-25T10:21:44.585367+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:44.845476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992478+0000","last_change":"2024-04-25T10:21:42.197149+0000","last_active":"2024-04-25T10:21:44.992478+0000","last_peered":"2024-04-25T10:21:44.992478+0000","last_clean":"2024-04-25T10:21:44.992478+0000","last_became_active":"2024-04-25T10:21:42.196839+0000","last_became_peered":"2024-04-25T10:21:42.196839+0000","last_unstale":"2024-04-25T10:21:44.992478+0000","last_undegraded":"2024-04-25T10:21:44.992478+0000","last_fullsized":"2024-04-25T10:21:44.992478+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:19.774144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585194+0000","last_change":"2024-04-25T10:21:43.216038+0000","last_active":"2024-04-25T10:21:44.585194+0000","last_peered":"2024-04-25T10:21:44.585194+0000","last_clean":"2024-04-25T10:21:44.585194+0000","last_became_active":"2024-04-25T10:21:43.215755+0000","last_became_peered":"2024-04-25T10:21:43.215755+0000","last_unstale":"2024-04-25T10:21:44.585194+0000","last_undegraded":"2024-04-25T10:21:44.585194+0000","last_fullsized":"2024-04-25T10:21:44.585194+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:10.650657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200849+0000","last_change":"2024-04-25T10:21:42.196094+0000","last_active":"2024-04-25T10:21:44.200849+0000","last_peered":"2024-04-25T10:21:44.200849+0000","last_clean":"2024-04-25T10:21:44.200849+0000","last_became_active":"2024-04-25T10:21:42.195175+0000","last_became_peered":"2024-04-25T10:21:42.195175+0000","last_unstale":"2024-04-25T10:21:44.200849+0000","last_undegraded":"2024-04-25T10:21:44.200849+0000","last_fullsized":"2024-04-25T10:21:44.200849+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:13.617706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991302+0000","last_change":"2024-04-25T10:21:43.207403+0000","last_active":"2024-04-25T10:21:44.991302+0000","last_peered":"2024-04-25T10:21:44.991302+0000","last_clean":"2024-04-25T10:21:44.991302+0000","last_became_active":"2024-04-25T10:21:43.206240+0000","last_became_peered":"2024-04-25T10:21:43.206240+0000","last_unstale":"2024-04-25T10:21:44.991302+0000","last_undegraded":"2024-04-25T10:21:44.991302+0000","last_fullsized":"2024-04-25T10:21:44.991302+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:04.019536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200666+0000","last_change":"2024-04-25T10:21:42.193788+0000","last_active":"2024-04-25T10:21:44.200666+0000","last_peered":"2024-04-25T10:21:44.200666+0000","last_clean":"2024-04-25T10:21:44.200666+0000","last_became_active":"2024-04-25T10:21:42.192252+0000","last_became_peered":"2024-04-25T10:21:42.192252+0000","last_unstale":"2024-04-25T10:21:44.200666+0000","last_undegraded":"2024-04-25T10:21:44.200666+0000","last_fullsized":"2024-04-25T10:21:44.200666+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:38.319222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992688+0000","last_change":"2024-04-25T10:21:43.201685+0000","last_active":"2024-04-25T10:21:44.992688+0000","last_peered":"2024-04-25T10:21:44.992688+0000","last_clean":"2024-04-25T10:21:44.992688+0000","last_became_active":"2024-04-25T10:21:43.201337+0000","last_became_peered":"2024-04-25T10:21:43.201337+0000","last_unstale":"2024-04-25T10:21:44.992688+0000","last_undegraded":"2024-04-25T10:21:44.992688+0000","last_fullsized":"2024-04-25T10:21:44.992688+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:52.818109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991008+0000","last_change":"2024-04-25T10:21:42.194638+0000","last_active":"2024-04-25T10:21:44.991008+0000","last_peered":"2024-04-25T10:21:44.991008+0000","last_clean":"2024-04-25T10:21:44.991008+0000","last_became_active":"2024-04-25T10:21:42.194151+0000","last_became_peered":"2024-04-25T10:21:42.194151+0000","last_unstale":"2024-04-25T10:21:44.991008+0000","last_undegraded":"2024-04-25T10:21:44.991008+0000","last_fullsized":"2024-04-25T10:21:44.991008+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:25.524782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209375+0000","last_change":"2024-04-25T10:21:43.203179+0000","last_active":"2024-04-25T10:21:44.209375+0000","last_peered":"2024-04-25T10:21:44.209375+0000","last_clean":"2024-04-25T10:21:44.209375+0000","last_became_active":"2024-04-25T10:21:43.202761+0000","last_became_peered":"2024-04-25T10:21:43.202761+0000","last_unstale":"2024-04-25T10:21:44.209375+0000","last_undegraded":"2024-04-25T10:21:44.209375+0000","last_fullsized":"2024-04-25T10:21:44.209375+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:26.527517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992312+0000","last_change":"2024-04-25T10:21:42.196369+0000","last_active":"2024-04-25T10:21:44.992312+0000","last_peered":"2024-04-25T10:21:44.992312+0000","last_clean":"2024-04-25T10:21:44.992312+0000","last_became_active":"2024-04-25T10:21:42.196062+0000","last_became_peered":"2024-04-25T10:21:42.196062+0000","last_unstale":"2024-04-25T10:21:44.992312+0000","last_undegraded":"2024-04-25T10:21:44.992312+0000","last_fullsized":"2024-04-25T10:21:44.992312+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:20.415473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200544+0000","last_change":"2024-04-25T10:21:43.200621+0000","last_active":"2024-04-25T10:21:44.200544+0000","last_peered":"2024-04-25T10:21:44.200544+0000","last_clean":"2024-04-25T10:21:44.200544+0000","last_became_active":"2024-04-25T10:21:43.200333+0000","last_became_peered":"2024-04-25T10:21:43.200333+0000","last_unstale":"2024-04-25T10:21:44.200544+0000","last_undegraded":"2024-04-25T10:21:44.200544+0000","last_fullsized":"2024-04-25T10:21:44.200544+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:22.080120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200572+0000","last_change":"2024-04-25T10:21:42.192901+0000","last_active":"2024-04-25T10:21:44.200572+0000","last_peered":"2024-04-25T10:21:44.200572+0000","last_clean":"2024-04-25T10:21:44.200572+0000","last_became_active":"2024-04-25T10:21:42.192477+0000","last_became_peered":"2024-04-25T10:21:42.192477+0000","last_unstale":"2024-04-25T10:21:44.200572+0000","last_undegraded":"2024-04-25T10:21:44.200572+0000","last_fullsized":"2024-04-25T10:21:44.200572+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:15.807083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992497+0000","last_change":"2024-04-25T10:21:43.200054+0000","last_active":"2024-04-25T10:21:44.992497+0000","last_peered":"2024-04-25T10:21:44.992497+0000","last_clean":"2024-04-25T10:21:44.992497+0000","last_became_active":"2024-04-25T10:21:43.199382+0000","last_became_peered":"2024-04-25T10:21:43.199382+0000","last_unstale":"2024-04-25T10:21:44.992497+0000","last_undegraded":"2024-04-25T10:21:44.992497+0000","last_fullsized":"2024-04-25T10:21:44.992497+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:34.268092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210438+0000","last_change":"2024-04-25T10:21:42.195851+0000","last_active":"2024-04-25T10:21:44.210438+0000","last_peered":"2024-04-25T10:21:44.210438+0000","last_clean":"2024-04-25T10:21:44.210438+0000","last_became_active":"2024-04-25T10:21:42.194348+0000","last_became_peered":"2024-04-25T10:21:42.194348+0000","last_unstale":"2024-04-25T10:21:44.210438+0000","last_undegraded":"2024-04-25T10:21:44.210438+0000","last_fullsized":"2024-04-25T10:21:44.210438+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:32.323570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210389+0000","last_change":"2024-04-25T10:21:43.202665+0000","last_active":"2024-04-25T10:21:44.210389+0000","last_peered":"2024-04-25T10:21:44.210389+0000","last_clean":"2024-04-25T10:21:44.210389+0000","last_became_active":"2024-04-25T10:21:43.202399+0000","last_became_peered":"2024-04-25T10:21:43.202399+0000","last_unstale":"2024-04-25T10:21:44.210389+0000","last_undegraded":"2024-04-25T10:21:44.210389+0000","last_fullsized":"2024-04-25T10:21:44.210389+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.260659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992641+0000","last_change":"2024-04-25T10:21:42.199391+0000","last_active":"2024-04-25T10:21:44.992641+0000","last_peered":"2024-04-25T10:21:44.992641+0000","last_clean":"2024-04-25T10:21:44.992641+0000","last_became_active":"2024-04-25T10:21:42.197892+0000","last_became_peered":"2024-04-25T10:21:42.197892+0000","last_unstale":"2024-04-25T10:21:44.992641+0000","last_undegraded":"2024-04-25T10:21:44.992641+0000","last_fullsized":"2024-04-25T10:21:44.992641+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.453147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585455+0000","last_change":"2024-04-25T10:21:43.217477+0000","last_active":"2024-04-25T10:21:44.585455+0000","last_peered":"2024-04-25T10:21:44.585455+0000","last_clean":"2024-04-25T10:21:44.585455+0000","last_became_active":"2024-04-25T10:21:43.216872+0000","last_became_peered":"2024-04-25T10:21:43.216872+0000","last_unstale":"2024-04-25T10:21:44.585455+0000","last_undegraded":"2024-04-25T10:21:44.585455+0000","last_fullsized":"2024-04-25T10:21:44.585455+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:42.008315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"123'3","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209838+0000","last_change":"2024-04-25T10:21:42.191713+0000","last_active":"2024-04-25T10:21:44.209838+0000","last_peered":"2024-04-25T10:21:44.209838+0000","last_clean":"2024-04-25T10:21:44.209838+0000","last_became_active":"2024-04-25T10:21:42.191398+0000","last_became_peered":"2024-04-25T10:21:42.191398+0000","last_unstale":"2024-04-25T10:21:44.209838+0000","last_undegraded":"2024-04-25T10:21:44.209838+0000","last_fullsized":"2024-04-25T10:21:44.209838+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:07.927606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585244+0000","last_change":"2024-04-25T10:21:43.215513+0000","last_active":"2024-04-25T10:21:44.585244+0000","last_peered":"2024-04-25T10:21:44.585244+0000","last_clean":"2024-04-25T10:21:44.585244+0000","last_became_active":"2024-04-25T10:21:43.215212+0000","last_became_peered":"2024-04-25T10:21:43.215212+0000","last_unstale":"2024-04-25T10:21:44.585244+0000","last_undegraded":"2024-04-25T10:21:44.585244+0000","last_fullsized":"2024-04-25T10:21:44.585244+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.281285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"123'6","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209554+0000","last_change":"2024-04-25T10:21:42.200411+0000","last_active":"2024-04-25T10:21:44.209554+0000","last_peered":"2024-04-25T10:21:44.209554+0000","last_clean":"2024-04-25T10:21:44.209554+0000","last_became_active":"2024-04-25T10:21:42.200033+0000","last_became_peered":"2024-04-25T10:21:42.200033+0000","last_unstale":"2024-04-25T10:21:44.209554+0000","last_undegraded":"2024-04-25T10:21:44.209554+0000","last_fullsized":"2024-04-25T10:21:44.209554+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:01.918133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992172+0000","last_change":"2024-04-25T10:21:43.202751+0000","last_active":"2024-04-25T10:21:44.992172+0000","last_peered":"2024-04-25T10:21:44.992172+0000","last_clean":"2024-04-25T10:21:44.992172+0000","last_became_active":"2024-04-25T10:21:43.199393+0000","last_became_peered":"2024-04-25T10:21:43.199393+0000","last_unstale":"2024-04-25T10:21:44.992172+0000","last_undegraded":"2024-04-25T10:21:44.992172+0000","last_fullsized":"2024-04-25T10:21:44.992172+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.539372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991729+0000","last_change":"2024-04-25T10:21:42.194643+0000","last_active":"2024-04-25T10:21:44.991729+0000","last_peered":"2024-04-25T10:21:44.991729+0000","last_clean":"2024-04-25T10:21:44.991729+0000","last_became_active":"2024-04-25T10:21:42.194366+0000","last_became_peered":"2024-04-25T10:21:42.194366+0000","last_unstale":"2024-04-25T10:21:44.991729+0000","last_undegraded":"2024-04-25T10:21:44.991729+0000","last_fullsized":"2024-04-25T10:21:44.991729+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:54.311366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200508+0000","last_change":"2024-04-25T10:21:43.194160+0000","last_active":"2024-04-25T10:21:44.200508+0000","last_peered":"2024-04-25T10:21:44.200508+0000","last_clean":"2024-04-25T10:21:44.200508+0000","last_became_active":"2024-04-25T10:21:43.193866+0000","last_became_peered":"2024-04-25T10:21:43.193866+0000","last_unstale":"2024-04-25T10:21:44.200508+0000","last_undegraded":"2024-04-25T10:21:44.200508+0000","last_fullsized":"2024-04-25T10:21:44.200508+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:20.602942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200594+0000","last_change":"2024-04-25T10:21:42.197914+0000","last_active":"2024-04-25T10:21:44.200594+0000","last_peered":"2024-04-25T10:21:44.200594+0000","last_clean":"2024-04-25T10:21:44.200594+0000","last_became_active":"2024-04-25T10:21:42.197665+0000","last_became_peered":"2024-04-25T10:21:42.197665+0000","last_unstale":"2024-04-25T10:21:44.200594+0000","last_undegraded":"2024-04-25T10:21:44.200594+0000","last_fullsized":"2024-04-25T10:21:44.200594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:18.929708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209150+0000","last_change":"2024-04-25T10:21:43.210914+0000","last_active":"2024-04-25T10:21:44.209150+0000","last_peered":"2024-04-25T10:21:44.209150+0000","last_clean":"2024-04-25T10:21:44.209150+0000","last_became_active":"2024-04-25T10:21:43.210314+0000","last_became_peered":"2024-04-25T10:21:43.210314+0000","last_unstale":"2024-04-25T10:21:44.209150+0000","last_undegraded":"2024-04-25T10:21:44.209150+0000","last_fullsized":"2024-04-25T10:21:44.209150+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:13.774084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"123'8","reported_seq":22,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209168+0000","last_change":"2024-04-25T10:21:42.200339+0000","last_active":"2024-04-25T10:21:44.209168+0000","last_peered":"2024-04-25T10:21:44.209168+0000","last_clean":"2024-04-25T10:21:44.209168+0000","last_became_active":"2024-04-25T10:21:42.200009+0000","last_became_peered":"2024-04-25T10:21:42.200009+0000","last_unstale":"2024-04-25T10:21:44.209168+0000","last_undegraded":"2024-04-25T10:21:44.209168+0000","last_fullsized":"2024-04-25T10:21:44.209168+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.312727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210041+0000","last_change":"2024-04-25T10:21:43.208035+0000","last_active":"2024-04-25T10:21:44.210041+0000","last_peered":"2024-04-25T10:21:44.210041+0000","last_clean":"2024-04-25T10:21:44.210041+0000","last_became_active":"2024-04-25T10:21:43.207774+0000","last_became_peered":"2024-04-25T10:21:43.207774+0000","last_unstale":"2024-04-25T10:21:44.210041+0000","last_undegraded":"2024-04-25T10:21:44.210041+0000","last_fullsized":"2024-04-25T10:21:44.210041+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:20.455882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210495+0000","last_change":"2024-04-25T10:21:42.196789+0000","last_active":"2024-04-25T10:21:44.210495+0000","last_peered":"2024-04-25T10:21:44.210495+0000","last_clean":"2024-04-25T10:21:44.210495+0000","last_became_active":"2024-04-25T10:21:42.195840+0000","last_became_peered":"2024-04-25T10:21:42.195840+0000","last_unstale":"2024-04-25T10:21:44.210495+0000","last_undegraded":"2024-04-25T10:21:44.210495+0000","last_fullsized":"2024-04-25T10:21:44.210495+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:47.827547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.210445+0000","last_change":"2024-04-25T10:21:43.201013+0000","last_active":"2024-04-25T10:21:44.210445+0000","last_peered":"2024-04-25T10:21:44.210445+0000","last_clean":"2024-04-25T10:21:44.210445+0000","last_became_active":"2024-04-25T10:21:43.200653+0000","last_became_peered":"2024-04-25T10:21:43.200653+0000","last_unstale":"2024-04-25T10:21:44.210445+0000","last_undegraded":"2024-04-25T10:21:44.210445+0000","last_fullsized":"2024-04-25T10:21:44.210445+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:25.152746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992753+0000","last_change":"2024-04-25T10:21:42.195596+0000","last_active":"2024-04-25T10:21:44.992753+0000","last_peered":"2024-04-25T10:21:44.992753+0000","last_clean":"2024-04-25T10:21:44.992753+0000","last_became_active":"2024-04-25T10:21:42.195342+0000","last_became_peered":"2024-04-25T10:21:42.195342+0000","last_unstale":"2024-04-25T10:21:44.992753+0000","last_undegraded":"2024-04-25T10:21:44.992753+0000","last_fullsized":"2024-04-25T10:21:44.992753+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:33.689082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991904+0000","last_change":"2024-04-25T10:21:43.205811+0000","last_active":"2024-04-25T10:21:44.991904+0000","last_peered":"2024-04-25T10:21:44.991904+0000","last_clean":"2024-04-25T10:21:44.991904+0000","last_became_active":"2024-04-25T10:21:43.204655+0000","last_became_peered":"2024-04-25T10:21:43.204655+0000","last_unstale":"2024-04-25T10:21:44.991904+0000","last_undegraded":"2024-04-25T10:21:44.991904+0000","last_fullsized":"2024-04-25T10:21:44.991904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:20.776879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992806+0000","last_change":"2024-04-25T10:21:42.194339+0000","last_active":"2024-04-25T10:21:44.992806+0000","last_peered":"2024-04-25T10:21:44.992806+0000","last_clean":"2024-04-25T10:21:44.992806+0000","last_became_active":"2024-04-25T10:21:42.193055+0000","last_became_peered":"2024-04-25T10:21:42.193055+0000","last_unstale":"2024-04-25T10:21:44.992806+0000","last_undegraded":"2024-04-25T10:21:44.992806+0000","last_fullsized":"2024-04-25T10:21:44.992806+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:21.749283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201184+0000","last_change":"2024-04-25T10:21:43.200940+0000","last_active":"2024-04-25T10:21:44.201184+0000","last_peered":"2024-04-25T10:21:44.201184+0000","last_clean":"2024-04-25T10:21:44.201184+0000","last_became_active":"2024-04-25T10:21:43.200205+0000","last_became_peered":"2024-04-25T10:21:43.200205+0000","last_unstale":"2024-04-25T10:21:44.201184+0000","last_undegraded":"2024-04-25T10:21:44.201184+0000","last_fullsized":"2024-04-25T10:21:44.201184+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.529868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209902+0000","last_change":"2024-04-25T10:21:42.197762+0000","last_active":"2024-04-25T10:21:44.209902+0000","last_peered":"2024-04-25T10:21:44.209902+0000","last_clean":"2024-04-25T10:21:44.209902+0000","last_became_active":"2024-04-25T10:21:42.196531+0000","last_became_peered":"2024-04-25T10:21:42.196531+0000","last_unstale":"2024-04-25T10:21:44.209902+0000","last_undegraded":"2024-04-25T10:21:44.209902+0000","last_fullsized":"2024-04-25T10:21:44.209902+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:40.069187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991719+0000","last_change":"2024-04-25T10:21:43.192082+0000","last_active":"2024-04-25T10:21:44.991719+0000","last_peered":"2024-04-25T10:21:44.991719+0000","last_clean":"2024-04-25T10:21:44.991719+0000","last_became_active":"2024-04-25T10:21:43.191796+0000","last_became_peered":"2024-04-25T10:21:43.191796+0000","last_unstale":"2024-04-25T10:21:44.991719+0000","last_undegraded":"2024-04-25T10:21:44.991719+0000","last_fullsized":"2024-04-25T10:21:44.991719+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:57.621592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209688+0000","last_change":"2024-04-25T10:21:42.197087+0000","last_active":"2024-04-25T10:21:44.209688+0000","last_peered":"2024-04-25T10:21:44.209688+0000","last_clean":"2024-04-25T10:21:44.209688+0000","last_became_active":"2024-04-25T10:21:42.196749+0000","last_became_peered":"2024-04-25T10:21:42.196749+0000","last_unstale":"2024-04-25T10:21:44.209688+0000","last_undegraded":"2024-04-25T10:21:44.209688+0000","last_fullsized":"2024-04-25T10:21:44.209688+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:19.600207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992621+0000","last_change":"2024-04-25T10:21:43.197995+0000","last_active":"2024-04-25T10:21:44.992621+0000","last_peered":"2024-04-25T10:21:44.992621+0000","last_clean":"2024-04-25T10:21:44.992621+0000","last_became_active":"2024-04-25T10:21:43.197712+0000","last_became_peered":"2024-04-25T10:21:43.197712+0000","last_unstale":"2024-04-25T10:21:44.992621+0000","last_undegraded":"2024-04-25T10:21:44.992621+0000","last_fullsized":"2024-04-25T10:21:44.992621+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:20.455882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991134+0000","last_change":"2024-04-25T10:21:42.198592+0000","last_active":"2024-04-25T10:21:44.991134+0000","last_peered":"2024-04-25T10:21:44.991134+0000","last_clean":"2024-04-25T10:21:44.991134+0000","last_became_active":"2024-04-25T10:21:42.198295+0000","last_became_peered":"2024-04-25T10:21:42.198295+0000","last_unstale":"2024-04-25T10:21:44.991134+0000","last_undegraded":"2024-04-25T10:21:44.991134+0000","last_fullsized":"2024-04-25T10:21:44.991134+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.653391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200513+0000","last_change":"2024-04-25T10:21:43.201320+0000","last_active":"2024-04-25T10:21:44.200513+0000","last_peered":"2024-04-25T10:21:44.200513+0000","last_clean":"2024-04-25T10:21:44.200513+0000","last_became_active":"2024-04-25T10:21:43.201073+0000","last_became_peered":"2024-04-25T10:21:43.201073+0000","last_unstale":"2024-04-25T10:21:44.200513+0000","last_undegraded":"2024-04-25T10:21:44.200513+0000","last_fullsized":"2024-04-25T10:21:44.200513+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:41.071922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.990953+0000","last_change":"2024-04-25T10:21:42.189561+0000","last_active":"2024-04-25T10:21:44.990953+0000","last_peered":"2024-04-25T10:21:44.990953+0000","last_clean":"2024-04-25T10:21:44.990953+0000","last_became_active":"2024-04-25T10:21:42.189232+0000","last_became_peered":"2024-04-25T10:21:42.189232+0000","last_unstale":"2024-04-25T10:21:44.990953+0000","last_undegraded":"2024-04-25T10:21:44.990953+0000","last_fullsized":"2024-04-25T10:21:44.990953+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:39.537368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584882+0000","last_change":"2024-04-25T10:21:43.218061+0000","last_active":"2024-04-25T10:21:44.584882+0000","last_peered":"2024-04-25T10:21:44.584882+0000","last_clean":"2024-04-25T10:21:44.584882+0000","last_became_active":"2024-04-25T10:21:43.217509+0000","last_became_peered":"2024-04-25T10:21:43.217509+0000","last_unstale":"2024-04-25T10:21:44.584882+0000","last_undegraded":"2024-04-25T10:21:44.584882+0000","last_fullsized":"2024-04-25T10:21:44.584882+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.480924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992594+0000","last_change":"2024-04-25T10:21:42.196842+0000","last_active":"2024-04-25T10:21:44.992594+0000","last_peered":"2024-04-25T10:21:44.992594+0000","last_clean":"2024-04-25T10:21:44.992594+0000","last_became_active":"2024-04-25T10:21:42.195580+0000","last_became_peered":"2024-04-25T10:21:42.195580+0000","last_unstale":"2024-04-25T10:21:44.992594+0000","last_undegraded":"2024-04-25T10:21:44.992594+0000","last_fullsized":"2024-04-25T10:21:44.992594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:30.394056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.993041+0000","last_change":"2024-04-25T10:21:43.196276+0000","last_active":"2024-04-25T10:21:44.993041+0000","last_peered":"2024-04-25T10:21:44.993041+0000","last_clean":"2024-04-25T10:21:44.993041+0000","last_became_active":"2024-04-25T10:21:43.195135+0000","last_became_peered":"2024-04-25T10:21:43.195135+0000","last_unstale":"2024-04-25T10:21:44.993041+0000","last_undegraded":"2024-04-25T10:21:44.993041+0000","last_fullsized":"2024-04-25T10:21:44.993041+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:20.912701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992633+0000","last_change":"2024-04-25T10:21:42.190602+0000","last_active":"2024-04-25T10:21:44.992633+0000","last_peered":"2024-04-25T10:21:44.992633+0000","last_clean":"2024-04-25T10:21:44.992633+0000","last_became_active":"2024-04-25T10:21:42.189881+0000","last_became_peered":"2024-04-25T10:21:42.189881+0000","last_unstale":"2024-04-25T10:21:44.992633+0000","last_undegraded":"2024-04-25T10:21:44.992633+0000","last_fullsized":"2024-04-25T10:21:44.992633+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:13.764058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992578+0000","last_change":"2024-04-25T10:21:43.200122+0000","last_active":"2024-04-25T10:21:44.992578+0000","last_peered":"2024-04-25T10:21:44.992578+0000","last_clean":"2024-04-25T10:21:44.992578+0000","last_became_active":"2024-04-25T10:21:43.199126+0000","last_became_peered":"2024-04-25T10:21:43.199126+0000","last_unstale":"2024-04-25T10:21:44.992578+0000","last_undegraded":"2024-04-25T10:21:44.992578+0000","last_fullsized":"2024-04-25T10:21:44.992578+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:39.063334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"124'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585078+0000","last_change":"2024-04-25T10:21:42.201935+0000","last_active":"2024-04-25T10:21:44.585078+0000","last_peered":"2024-04-25T10:21:44.585078+0000","last_clean":"2024-04-25T10:21:44.585078+0000","last_became_active":"2024-04-25T10:21:42.201513+0000","last_became_peered":"2024-04-25T10:21:42.201513+0000","last_unstale":"2024-04-25T10:21:44.585078+0000","last_undegraded":"2024-04-25T10:21:44.585078+0000","last_fullsized":"2024-04-25T10:21:44.585078+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:39.537368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200946+0000","last_change":"2024-04-25T10:21:43.215932+0000","last_active":"2024-04-25T10:21:44.200946+0000","last_peered":"2024-04-25T10:21:44.200946+0000","last_clean":"2024-04-25T10:21:44.200946+0000","last_became_active":"2024-04-25T10:21:43.215702+0000","last_became_peered":"2024-04-25T10:21:43.215702+0000","last_unstale":"2024-04-25T10:21:44.200946+0000","last_undegraded":"2024-04-25T10:21:44.200946+0000","last_fullsized":"2024-04-25T10:21:44.200946+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.830282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.201096+0000","last_change":"2024-04-25T10:21:42.193210+0000","last_active":"2024-04-25T10:21:44.201096+0000","last_peered":"2024-04-25T10:21:44.201096+0000","last_clean":"2024-04-25T10:21:44.201096+0000","last_became_active":"2024-04-25T10:21:42.192904+0000","last_became_peered":"2024-04-25T10:21:42.192904+0000","last_unstale":"2024-04-25T10:21:44.201096+0000","last_undegraded":"2024-04-25T10:21:44.201096+0000","last_fullsized":"2024-04-25T10:21:44.201096+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:23.196552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992953+0000","last_change":"2024-04-25T10:21:43.194074+0000","last_active":"2024-04-25T10:21:44.992953+0000","last_peered":"2024-04-25T10:21:44.992953+0000","last_clean":"2024-04-25T10:21:44.992953+0000","last_became_active":"2024-04-25T10:21:43.193276+0000","last_became_peered":"2024-04-25T10:21:43.193276+0000","last_unstale":"2024-04-25T10:21:44.992953+0000","last_undegraded":"2024-04-25T10:21:44.992953+0000","last_fullsized":"2024-04-25T10:21:44.992953+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:05.448121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.992035+0000","last_change":"2024-04-25T10:21:42.199465+0000","last_active":"2024-04-25T10:21:44.992035+0000","last_peered":"2024-04-25T10:21:44.992035+0000","last_clean":"2024-04-25T10:21:44.992035+0000","last_became_active":"2024-04-25T10:21:42.198919+0000","last_became_peered":"2024-04-25T10:21:42.198919+0000","last_unstale":"2024-04-25T10:21:44.992035+0000","last_undegraded":"2024-04-25T10:21:44.992035+0000","last_fullsized":"2024-04-25T10:21:44.992035+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:09.647923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584866+0000","last_change":"2024-04-25T10:21:43.219944+0000","last_active":"2024-04-25T10:21:44.584866+0000","last_peered":"2024-04-25T10:21:44.584866+0000","last_clean":"2024-04-25T10:21:44.584866+0000","last_became_active":"2024-04-25T10:21:43.219612+0000","last_became_peered":"2024-04-25T10:21:43.219612+0000","last_unstale":"2024-04-25T10:21:44.584866+0000","last_undegraded":"2024-04-25T10:21:44.584866+0000","last_fullsized":"2024-04-25T10:21:44.584866+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:26.788421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200873+0000","last_change":"2024-04-25T10:21:42.199189+0000","last_active":"2024-04-25T10:21:44.200873+0000","last_peered":"2024-04-25T10:21:44.200873+0000","last_clean":"2024-04-25T10:21:44.200873+0000","last_became_active":"2024-04-25T10:21:42.198850+0000","last_became_peered":"2024-04-25T10:21:42.198850+0000","last_unstale":"2024-04-25T10:21:44.200873+0000","last_undegraded":"2024-04-25T10:21:44.200873+0000","last_fullsized":"2024-04-25T10:21:44.200873+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:56.618857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.209740+0000","last_change":"2024-04-25T10:21:43.194775+0000","last_active":"2024-04-25T10:21:44.209740+0000","last_peered":"2024-04-25T10:21:44.209740+0000","last_clean":"2024-04-25T10:21:44.209740+0000","last_became_active":"2024-04-25T10:21:43.194502+0000","last_became_peered":"2024-04-25T10:21:43.194502+0000","last_unstale":"2024-04-25T10:21:44.209740+0000","last_undegraded":"2024-04-25T10:21:44.209740+0000","last_fullsized":"2024-04-25T10:21:44.209740+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.529868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584972+0000","last_change":"2024-04-25T10:21:42.203621+0000","last_active":"2024-04-25T10:21:44.584972+0000","last_peered":"2024-04-25T10:21:44.584972+0000","last_clean":"2024-04-25T10:21:44.584972+0000","last_became_active":"2024-04-25T10:21:42.203335+0000","last_became_peered":"2024-04-25T10:21:42.203335+0000","last_unstale":"2024-04-25T10:21:44.584972+0000","last_undegraded":"2024-04-25T10:21:44.584972+0000","last_fullsized":"2024-04-25T10:21:44.584972+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:24.432362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991308+0000","last_change":"2024-04-25T10:21:43.209015+0000","last_active":"2024-04-25T10:21:44.991308+0000","last_peered":"2024-04-25T10:21:44.991308+0000","last_clean":"2024-04-25T10:21:44.991308+0000","last_became_active":"2024-04-25T10:21:43.204745+0000","last_became_peered":"2024-04-25T10:21:43.204745+0000","last_unstale":"2024-04-25T10:21:44.991308+0000","last_undegraded":"2024-04-25T10:21:44.991308+0000","last_fullsized":"2024-04-25T10:21:44.991308+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:26.788421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991068+0000","last_change":"2024-04-25T10:21:42.198952+0000","last_active":"2024-04-25T10:21:44.991068+0000","last_peered":"2024-04-25T10:21:44.991068+0000","last_clean":"2024-04-25T10:21:44.991068+0000","last_became_active":"2024-04-25T10:21:42.198699+0000","last_became_peered":"2024-04-25T10:21:42.198699+0000","last_unstale":"2024-04-25T10:21:44.991068+0000","last_undegraded":"2024-04-25T10:21:44.991068+0000","last_fullsized":"2024-04-25T10:21:44.991068+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:24.432362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.200578+0000","last_change":"2024-04-25T10:21:43.208531+0000","last_active":"2024-04-25T10:21:44.200578+0000","last_peered":"2024-04-25T10:21:44.200578+0000","last_clean":"2024-04-25T10:21:44.200578+0000","last_became_active":"2024-04-25T10:21:43.208238+0000","last_became_peered":"2024-04-25T10:21:43.208238+0000","last_unstale":"2024-04-25T10:21:44.200578+0000","last_undegraded":"2024-04-25T10:21:44.200578+0000","last_fullsized":"2024-04-25T10:21:44.200578+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:33.326305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.991652+0000","last_change":"2024-04-25T10:21:42.194731+0000","last_active":"2024-04-25T10:21:44.991652+0000","last_peered":"2024-04-25T10:21:44.991652+0000","last_clean":"2024-04-25T10:21:44.991652+0000","last_became_active":"2024-04-25T10:21:42.194445+0000","last_became_peered":"2024-04-25T10:21:42.194445+0000","last_unstale":"2024-04-25T10:21:44.991652+0000","last_undegraded":"2024-04-25T10:21:44.991652+0000","last_fullsized":"2024-04-25T10:21:44.991652+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:05.690854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584940+0000","last_change":"2024-04-25T10:21:43.215215+0000","last_active":"2024-04-25T10:21:44.584940+0000","last_peered":"2024-04-25T10:21:44.584940+0000","last_clean":"2024-04-25T10:21:44.584940+0000","last_became_active":"2024-04-25T10:21:43.214904+0000","last_became_peered":"2024-04-25T10:21:43.214904+0000","last_unstale":"2024-04-25T10:21:44.584940+0000","last_undegraded":"2024-04-25T10:21:44.584940+0000","last_fullsized":"2024-04-25T10:21:44.584940+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:42.164322+0000","last_clean_scrub_stamp":"2024-04-25T10:21:42.164322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.396791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.585221+0000","last_change":"2024-04-25T10:21:42.203881+0000","last_active":"2024-04-25T10:21:44.585221+0000","last_peered":"2024-04-25T10:21:44.585221+0000","last_clean":"2024-04-25T10:21:44.585221+0000","last_became_active":"2024-04-25T10:21:42.203521+0000","last_became_peered":"2024-04-25T10:21:42.203521+0000","last_unstale":"2024-04-25T10:21:44.585221+0000","last_undegraded":"2024-04-25T10:21:44.585221+0000","last_fullsized":"2024-04-25T10:21:44.585221+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:41.161587+0000","last_clean_scrub_stamp":"2024-04-25T10:21:41.161587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.453147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":356,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:21:44.584699+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:21:44.584699+0000","last_peered":"2024-04-25T10:21:44.584699+0000","last_clean":"2024-04-25T10:21:44.584699+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:21:44.584699+0000","last_undegraded":"2024-04-25T10:21:44.584699+0000","last_fullsized":"2024-04-25T10:21:44.584699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55508,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316332,"statfs":{"total":96636764160,"available":96579923968,"internally_reserved":0,"allocated":2166784,"data_stored":1266950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59080,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312760,"statfs":{"total":96636764160,"available":96576266240,"internally_reserved":0,"allocated":1560576,"data_stored":675944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46592,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325248,"statfs":{"total":96636764160,"available":96589053952,"internally_reserved":0,"allocated":1556480,"data_stored":674444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54908,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316932,"statfs":{"total":96636764160,"available":96580538368,"internally_reserved":0,"allocated":1552384,"data_stored":674402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1556480,"data_stored":674492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50752,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321088,"statfs":{"total":96636764160,"available":96584794112,"internally_reserved":0,"allocated":1556480,"data_stored":674948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77116,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294724,"statfs":{"total":96636764160,"available":96557797376,"internally_reserved":0,"allocated":1548288,"data_stored":674332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2142208,"data_stored":1264700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:21:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:21:47.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:21:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04b0c00640 1 Processor -- start 2024-04-25T10:21:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04b0c00640 1 -- start start 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04b0c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f04ac163060 0x7f04ac15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04b0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac0b44a0 0x7f04ac0cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04b0c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac1634e0 con 0x7f04ac15c1c0 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04b0c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ac15cae0 con 0x7f04ac163060 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04b0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac06c120 con 0x7f04ac0b44a0 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a9e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f04ac15c1c0 0x7f04ac163740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53508/0 (socket says 172.21.15.87:53508) 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a9e00640 1 -- 172.21.15.87:0/1974262088 learned_addr learned my addr 172.21.15.87:0/1974262088 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a9400640 1 --2- 172.21.15.87:0/1974262088 >> v2:172.21.15.169:3300/0 conn(0x7f04ac163060 0x7f04ac15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04aa800640 1 --2- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac0b44a0 0x7f04ac0cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a8a00640 1 -- 172.21.15.87:0/1974262088 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815063534 0 0) 0x7f04ac1634e0 con 0x7f04ac15c1c0 2024-04-25T10:21:47.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a8a00640 1 -- 172.21.15.87:0/1974262088 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0490003530 con 0x7f04ac15c1c0 2024-04-25T10:21:47.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a9400640 1 -- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac0b44a0 msgr2=0x7f04ac0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a9400640 1 --2- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac0b44a0 0x7f04ac0cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a9400640 1 -- 172.21.15.87:0/1974262088 >> v1:172.21.15.87:6789/0 conn(0x7f04ac15c1c0 legacy=0x7f04ac163740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.866+0000 7f04a9400640 1 -- 172.21.15.87:0/1974262088 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0b4c10 con 0x7f04ac163060 2024-04-25T10:21:47.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04a9400640 1 --2- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 0x7f04ac15c5a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f04940097c0 tx=0x7f049401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ca965a6d807f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04a8a00640 1 -- 172.21.15.87:0/1974262088 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04940b0070 con 0x7f04ac163060 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04a8a00640 1 -- 172.21.15.87:0/1974262088 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04940ab600 con 0x7f04ac163060 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04a8a00640 1 -- 172.21.15.87:0/1974262088 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04940ab9d0 con 0x7f04ac163060 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 -- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 msgr2=0x7f04ac15c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 --2- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 0x7f04ac15c5a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f04940097c0 tx=0x7f049401c3e0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 -- 172.21.15.87:0/1974262088 shutdown_connections 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 --2- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac0b44a0 0x7f04ac0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 --2- 172.21.15.87:0/1974262088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 0x7f04ac15c5a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 -- 172.21.15.87:0/1974262088 >> 172.21.15.87:0/1974262088 conn(0x7f04ac066b70 msgr2=0x7f04ac066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 -- 172.21.15.87:0/1974262088 shutdown_connections 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.870+0000 7f04b0c00640 1 -- 172.21.15.87:0/1974262088 wait complete. 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 Processor -- start 2024-04-25T10:21:47.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 -- start start 2024-04-25T10:21:47.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac0b44a0 0x7f04ac071b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac15c1c0 0x7f04ac144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 0x7f04ac144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac165700 con 0x7f04ac0b44a0 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0b48b0 con 0x7f04ac163060 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac165880 con 0x7f04ac15c1c0 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac0b44a0 0x7f04ac071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac0b44a0 0x7f04ac071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59680/0 (socket says 172.21.15.87:59680) 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 -- 172.21.15.87:0/2888523825 learned_addr learned my addr 172.21.15.87:0/2888523825 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04aa800640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 0x7f04ac144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9400640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac15c1c0 0x7f04ac144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 -- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac15c1c0 msgr2=0x7f04ac144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac15c1c0 0x7f04ac144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 -- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 msgr2=0x7f04ac144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 0x7f04ac144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 -- 172.21.15.87:0/2888523825 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac1460f0 con 0x7f04ac0b44a0 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a9e00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac0b44a0 0x7f04ac071b30 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c450 tx=0x7f049c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a2000640 1 -- 172.21.15.87:0/2888523825 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c017070 con 0x7f04ac0b44a0 2024-04-25T10:21:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a2000640 1 -- 172.21.15.87:0/2888523825 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c005cf0 con 0x7f04ac0b44a0 2024-04-25T10:21:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac1463d0 con 0x7f04ac0b44a0 2024-04-25T10:21:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04a2000640 1 -- 172.21.15.87:0/2888523825 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c007290 con 0x7f04ac0b44a0 2024-04-25T10:21:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac146990 con 0x7f04ac0b44a0 2024-04-25T10:21:47.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.874+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac04ef30 con 0x7f04ac0b44a0 2024-04-25T10:21:47.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.878+0000 7f04a2000640 1 -- 172.21.15.87:0/2888523825 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f049c01a070 con 0x7f04ac0b44a0 2024-04-25T10:21:47.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.878+0000 7f04a2000640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0488066f90 0x7f0488069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.878+0000 7f04a2000640 1 -- 172.21.15.87:0/2888523825 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f049c084340 con 0x7f04ac0b44a0 2024-04-25T10:21:47.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.878+0000 7f04a9400640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0488066f90 0x7f0488069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.878+0000 7f04a9400640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0488066f90 0x7f0488069450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f04940ad000 tx=0x7f0494008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:47.882+0000 7f04a2000640 1 -- 172.21.15.87:0/2888523825 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f049c054c70 con 0x7f04ac0b44a0 2024-04-25T10:21:48.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.090+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f04ac068020 con 0x7f04ac0b44a0 2024-04-25T10:21:48.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.090+0000 7f04a2000640 1 -- 172.21.15.87:0/2888523825 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f049c058d10 con 0x7f04ac0b44a0 2024-04-25T10:21:48.095 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:21:48.095 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:21:48.095 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:21:48.095 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:21:48.095 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:21:48.095 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:21:48.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0488066f90 msgr2=0x7f0488069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0488066f90 0x7f0488069450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f04940ad000 tx=0x7f0494008040 comp rx=0 tx=0).stop 2024-04-25T10:21:48.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac0b44a0 msgr2=0x7f04ac071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac0b44a0 0x7f04ac071b30 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c450 tx=0x7f049c00c910 comp rx=0 tx=0).stop 2024-04-25T10:21:48.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 shutdown_connections 2024-04-25T10:21:48.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0488066f90 0x7f0488069450 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ac163060 0x7f04ac144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ac15c1c0 0x7f04ac144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 --2- 172.21.15.87:0/2888523825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac0b44a0 0x7f04ac071b30 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 >> 172.21.15.87:0/2888523825 conn(0x7f04ac066b70 msgr2=0x7f04ac0ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:48.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 shutdown_connections 2024-04-25T10:21:48.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.094+0000 7f04b0c00640 1 -- 172.21.15.87:0/2888523825 wait complete. 2024-04-25T10:21:48.115 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:48.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 Processor -- start 2024-04-25T10:21:48.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 -- start start 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f612815a0e0 0x7f612815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6128071f70 0x7f61280b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f61280b2fc0 0x7f612806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612815b6e0 con 0x7f612815a0e0 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6128070ed0 con 0x7f61280b2fc0 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.250+0000 7f612d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61280711b0 con 0x7f6128071f70 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f612815a0e0 0x7f612815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f612815a0e0 0x7f612815b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59688/0 (socket says 172.21.15.87:59688) 2024-04-25T10:21:48.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 -- 172.21.15.87:0/998448313 learned_addr learned my addr 172.21.15.87:0/998448313 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f611d400640 1 --2- 172.21.15.87:0/998448313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6128071f70 0x7f61280b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 -- 172.21.15.87:0/998448313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6128071f70 msgr2=0x7f61280b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6126800640 1 --2- 172.21.15.87:0/998448313 >> v2:172.21.15.169:3300/0 conn(0x7f61280b2fc0 0x7f612806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 --2- 172.21.15.87:0/998448313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6128071f70 0x7f61280b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 -- 172.21.15.87:0/998448313 >> v2:172.21.15.169:3300/0 conn(0x7f61280b2fc0 msgr2=0x7f612806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 --2- 172.21.15.87:0/998448313 >> v2:172.21.15.169:3300/0 conn(0x7f61280b2fc0 0x7f612806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 -- 172.21.15.87:0/998448313 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128071490 con 0x7f612815a0e0 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f611d400640 1 --2- 172.21.15.87:0/998448313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6128071f70 0x7f61280b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:48.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6126800640 1 --2- 172.21.15.87:0/998448313 >> v2:172.21.15.169:3300/0 conn(0x7f61280b2fc0 0x7f612806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125e00640 1 --2- 172.21.15.87:0/998448313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f612815b1a0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f61280b4ae0 tx=0x7f611801db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b52b3cfb7f0e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.254+0000 7f6125400640 1 -- 172.21.15.87:0/998448313 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180a67f0 con 0x7f612815a0e0 2024-04-25T10:21:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f6125400640 1 -- 172.21.15.87:0/998448313 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61180a6990 con 0x7f612815a0e0 2024-04-25T10:21:48.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f6125400640 1 -- 172.21.15.87:0/998448313 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180a6d60 con 0x7f612815a0e0 2024-04-25T10:21:48.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- 172.21.15.87:0/998448313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 msgr2=0x7f612815b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 --2- 172.21.15.87:0/998448313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f612815b1a0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f61280b4ae0 tx=0x7f611801db00 comp rx=0 tx=0).stop 2024-04-25T10:21:48.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- 172.21.15.87:0/998448313 shutdown_connections 2024-04-25T10:21:48.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 --2- 172.21.15.87:0/998448313 >> v2:172.21.15.169:3300/0 conn(0x7f61280b2fc0 0x7f612806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 --2- 172.21.15.87:0/998448313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6128071f70 0x7f61280b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 --2- 172.21.15.87:0/998448313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f612815b1a0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- 172.21.15.87:0/998448313 >> 172.21.15.87:0/998448313 conn(0x7f6128066b70 msgr2=0x7f6128066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- 172.21.15.87:0/998448313 shutdown_connections 2024-04-25T10:21:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- 172.21.15.87:0/998448313 wait complete. 2024-04-25T10:21:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 Processor -- start 2024-04-25T10:21:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- start start 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6128071f70 0x7f6128156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61280b2fc0 0x7f612814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f6128158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128070b50 con 0x7f612815a0e0 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612815a4c0 con 0x7f6128071f70 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f612d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6128072350 con 0x7f61280b2fc0 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f6126800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f6128158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f6126800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f6128158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59692/0 (socket says 172.21.15.87:59692) 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f6126800640 1 -- 172.21.15.87:0/4202860701 learned_addr learned my addr 172.21.15.87:0/4202860701 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f6125e00640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6128071f70 0x7f6128156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.258+0000 7f611d400640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61280b2fc0 0x7f612814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f6126800640 1 -- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61280b2fc0 msgr2=0x7f612814c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f6126800640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61280b2fc0 0x7f612814c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f6126800640 1 -- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6128071f70 msgr2=0x7f6128156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f6126800640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6128071f70 0x7f6128156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f6126800640 1 -- 172.21.15.87:0/4202860701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128157080 con 0x7f612815a0e0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f6126800640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f6128158fb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f612000c820 tx=0x7f612000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f611ea00640 1 -- 172.21.15.87:0/4202860701 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120015070 con 0x7f612815a0e0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f611ea00640 1 -- 172.21.15.87:0/4202860701 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6120007a10 con 0x7f612815a0e0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f611ea00640 1 -- 172.21.15.87:0/4202860701 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120007e00 con 0x7f612815a0e0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6128157360 con 0x7f612815a0e0 2024-04-25T10:21:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6128149a90 con 0x7f612815a0e0 2024-04-25T10:21:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612804ef30 con 0x7f612815a0e0 2024-04-25T10:21:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f611ea00640 1 -- 172.21.15.87:0/4202860701 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6120018070 con 0x7f612815a0e0 2024-04-25T10:21:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f611ea00640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6104066f60 0x7f6104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f611ea00640 1 -- 172.21.15.87:0/4202860701 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6120084ed0 con 0x7f612815a0e0 2024-04-25T10:21:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.262+0000 7f6125e00640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6104066f60 0x7f6104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.266+0000 7f6125e00640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6104066f60 0x7f6104069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f61180af000 tx=0x7f61180a8070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:48.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.266+0000 7f611ea00640 1 -- 172.21.15.87:0/4202860701 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6120055880 con 0x7f612815a0e0 2024-04-25T10:21:48.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.450+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6128068180 con 0x7f612815a0e0 2024-04-25T10:21:48.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f611ea00640 1 -- 172.21.15.87:0/4202860701 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f6120059920 con 0x7f612815a0e0 2024-04-25T10:21:48.456 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:48.456 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:21:48.456 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6104066f60 msgr2=0x7f6104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6104066f60 0x7f6104069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f61180af000 tx=0x7f61180a8070 comp rx=0 tx=0).stop 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 msgr2=0x7f6128158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f6128158fb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f612000c820 tx=0x7f612000cce0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 shutdown_connections 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6104066f60 0x7f6104069420 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f612815a0e0 0x7f6128158fb0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61280b2fc0 0x7f612814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 --2- 172.21.15.87:0/4202860701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6128071f70 0x7f6128156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 >> 172.21.15.87:0/4202860701 conn(0x7f6128066b70 msgr2=0x7f61281500b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:48.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 shutdown_connections 2024-04-25T10:21:48.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:48.454+0000 7f612d200640 1 -- 172.21.15.87:0/4202860701 wait complete. 2024-04-25T10:21:48.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:21:42.563054+0000', 'modified': '2024-04-25T10:21:45.580473+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9114}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9114': {'gid': 9114, 'name': 'd', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6834/85260299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 85260299}, {'type': 'v1', 'addr': '172.21.15.169:6834', 'nonce': 85260299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:21:48.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:21:48.880 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:21:49.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e9000640 1 Processor -- start 2024-04-25T10:21:49.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e9000640 1 -- start start 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e9000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19e415bc40 0x7f19e406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19e40cda10 0x7f19e40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e9000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e40b2770 con 0x7f19e40cf930 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e9000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e406c670 con 0x7f19e40cda10 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e9000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e406c950 con 0x7f19e415bc40 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19d9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19e40cda10 0x7f19e40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19d9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19e40cda10 0x7f19e40b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59696/0 (socket says 172.21.15.87:59696) 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19d9400640 1 -- 172.21.15.87:0/3692990659 learned_addr learned my addr 172.21.15.87:0/3692990659 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.022+0000 7f19e1e00640 1 --2- 172.21.15.87:0/3692990659 >> v2:172.21.15.169:3300/0 conn(0x7f19e415bc40 0x7f19e406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19d9400640 1 -- 172.21.15.87:0/3692990659 >> v2:172.21.15.169:3300/0 conn(0x7f19e415bc40 msgr2=0x7f19e406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19d9400640 1 --2- 172.21.15.87:0/3692990659 >> v2:172.21.15.169:3300/0 conn(0x7f19e415bc40 0x7f19e406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19d9400640 1 -- 172.21.15.87:0/3692990659 >> v1:172.21.15.169:6789/0 conn(0x7f19e40cf930 legacy=0x7f19e415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19d9400640 1 -- 172.21.15.87:0/3692990659 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e406cc30 con 0x7f19e40cda10 2024-04-25T10:21:49.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19d9400640 1 --2- 172.21.15.87:0/3692990659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e40b4610 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f19e40d0800 tx=0x7f19cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba37f478766bbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e1400640 1 -- 172.21.15.87:0/3692990659 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0ab430 con 0x7f19e40cda10 2024-04-25T10:21:49.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e1400640 1 -- 172.21.15.87:0/3692990659 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc0b0070 con 0x7f19e40cda10 2024-04-25T10:21:49.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e1400640 1 -- 172.21.15.87:0/3692990659 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0ab7f0 con 0x7f19e40cda10 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 -- 172.21.15.87:0/3692990659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 msgr2=0x7f19e40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 --2- 172.21.15.87:0/3692990659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e40b4610 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f19e40d0800 tx=0x7f19cc01c520 comp rx=0 tx=0).stop 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 -- 172.21.15.87:0/3692990659 shutdown_connections 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 --2- 172.21.15.87:0/3692990659 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e40b4610 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 --2- 172.21.15.87:0/3692990659 >> v2:172.21.15.169:3300/0 conn(0x7f19e415bc40 0x7f19e406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 -- 172.21.15.87:0/3692990659 >> 172.21.15.87:0/3692990659 conn(0x7f19e4066b70 msgr2=0x7f19e4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 -- 172.21.15.87:0/3692990659 shutdown_connections 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.026+0000 7f19e9000640 1 -- 172.21.15.87:0/3692990659 wait complete. 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 Processor -- start 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 -- start start 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e41f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e40cf930 0x7f19e41f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e415bc40 0x7f19e40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4168b50 con 0x7f19e40cda10 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e41689d0 con 0x7f19e415bc40 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4168cd0 con 0x7f19e40cf930 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e41f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e41f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59702/0 (socket says 172.21.15.87:59702) 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 -- 172.21.15.87:0/1417554966 learned_addr learned my addr 172.21.15.87:0/1417554966 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e2800640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e415bc40 0x7f19e40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 -- 172.21.15.87:0/1417554966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e40cf930 msgr2=0x7f19e41f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e40cf930 0x7f19e41f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 -- 172.21.15.87:0/1417554966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e415bc40 msgr2=0x7f19e40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e415bc40 0x7f19e40ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 -- 172.21.15.87:0/1417554966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e40704b0 con 0x7f19e40cda10 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e1e00640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e41f71d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f19d4007f20 tx=0x7f19d400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19daa00640 1 -- 172.21.15.87:0/1417554966 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4014620 con 0x7f19e40cda10 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19daa00640 1 -- 172.21.15.87:0/1417554966 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4019070 con 0x7f19e40cda10 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19daa00640 1 -- 172.21.15.87:0/1417554966 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4014a00 con 0x7f19e40cda10 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e4070750 con 0x7f19e40cda10 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.030+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e4070d10 con 0x7f19e40cda10 2024-04-25T10:21:49.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.034+0000 7f19daa00640 1 -- 172.21.15.87:0/1417554966 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19d4014ba0 con 0x7f19e40cda10 2024-04-25T10:21:49.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.034+0000 7f19daa00640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19c0066f60 0x7f19c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.034+0000 7f19daa00640 1 -- 172.21.15.87:0/1417554966 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19d4085a20 con 0x7f19e40cda10 2024-04-25T10:21:49.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.034+0000 7f19d9400640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19c0066f60 0x7f19c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.034+0000 7f19ec41c000 1 Processor -- start 2024-04-25T10:21:49.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.034+0000 7f19ec41c000 1 -- start start 2024-04-25T10:21:49.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.034+0000 7f19d9400640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19c0066f60 0x7f19c0069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af000 tx=0x7f19cc004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea75edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea75a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea748b5f0 con 0x55bea75ee240 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea73f2da0 con 0x55bea75ed920 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bea737b150 con 0x55bea75a3f30 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59718/0 (socket says 172.21.15.87:59718) 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 -- 172.21.15.87:0/2605838725 learned_addr learned my addr 172.21.15.87:0/2605838725 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:49.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 -- 172.21.15.87:0/2605838725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 msgr2=0x55bea75a4310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:21:49.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 --2- 172.21.15.87:0/2605838725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea75a4310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 -- 172.21.15.87:0/2605838725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 msgr2=0x55bea75edd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:21:49.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 --2- 172.21.15.87:0/2605838725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea75edd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 -- 172.21.15.87:0/2605838725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bea7394ca0 con 0x55bea75ee240 2024-04-25T10:21:49.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d9400640 1 --2- 172.21.15.87:0/2605838725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a39f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000c10 tx=0x7f19cc0a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0768ec20d553bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d2a00640 1 -- 172.21.15.87:0/2605838725 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0c5070 con 0x55bea75ee240 2024-04-25T10:21:49.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d2a00640 1 -- 172.21.15.87:0/2605838725 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc0bb050 con 0x55bea75ee240 2024-04-25T10:21:49.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19d2a00640 1 -- 172.21.15.87:0/2605838725 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc004430 con 0x55bea75ee240 2024-04-25T10:21:49.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- 172.21.15.87:0/2605838725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 msgr2=0x55bea75a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- 172.21.15.87:0/2605838725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a39f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000c10 tx=0x7f19cc0a9000 comp rx=0 tx=0).stop 2024-04-25T10:21:49.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- 172.21.15.87:0/2605838725 shutdown_connections 2024-04-25T10:21:49.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- 172.21.15.87:0/2605838725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea75a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- 172.21.15.87:0/2605838725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a39f0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- 172.21.15.87:0/2605838725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea75edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- 172.21.15.87:0/2605838725 >> 172.21.15.87:0/2605838725 conn(0x55bea75aa2a0 msgr2=0x55bea75a22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- 172.21.15.87:0/2605838725 shutdown_connections 2024-04-25T10:21:49.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- 172.21.15.87:0/2605838725 wait complete. 2024-04-25T10:21:49.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- start start 2024-04-25T10:21:49.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 Processor -- start 2024-04-25T10:21:49.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea75f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea75f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.038+0000 7f19ec41c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea73ad990 con 0x55bea75ee240 2024-04-25T10:21:49.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea73eee40 con 0x55bea75ed920 2024-04-25T10:21:49.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bea737e900 con 0x55bea75a3f30 2024-04-25T10:21:49.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 10 client.0 _ll_register_callbacks cb 0x55bea74a61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59730/0 (socket says 172.21.15.87:59730) 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 -- 172.21.15.87:0/1276388837 learned_addr learned my addr 172.21.15.87:0/1276388837 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e1e00640 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea75f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19d9400640 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea75f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 -- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 msgr2=0x55bea75f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea75f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 -- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 msgr2=0x55bea75f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea75f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 -- 172.21.15.87:0/1276388837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bea73ab630 con 0x55bea75ee240 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e2800640 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a7aa0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f19dc001da0 tx=0x7f19dc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6034079a6eefab70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19c7400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19c7400640 1 -- 172.21.15.87:0/1276388837 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc019060 con 0x55bea75ee240 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19c7400640 1 -- 172.21.15.87:0/1276388837 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19dc0050d0 con 0x55bea75ee240 2024-04-25T10:21:49.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19cc0bb050 con 0x55bea75ee240 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19c7400640 1 -- 172.21.15.87:0/1276388837 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19dc01f050 con 0x55bea75ee240 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 10 client.9243 fetch_fsmap learned FSMap version 89 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55bea737b3a0 con 0x55bea75ee240 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19c7400640 1 -- 172.21.15.87:0/1276388837 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc0bb050 con 0x55bea75ee240 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19c7400640 1 -- 172.21.15.87:0/1276388837 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f19dc022020 con 0x55bea75ee240 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 10 client.9243 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 10 client.9243 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 20 client.9243 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea75a9160 0x55bea75feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 4 client.9243 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bea738ea50 con 0x55bea75a9160 2024-04-25T10:21:49.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e1e00640 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea75a9160 0x55bea75feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19e1e00640 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea75a9160 0x55bea75feec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d09d3f62c5af68 server_cookie=69920d72c11b2f21 in_seq=0 out_seq=0 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.042+0000 7f19c7400640 10 client.9243 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19c7400640 1 -- 172.21.15.87:0/1276388837 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bea738ea50 con 0x55bea75a9160 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19c7400640 10 client.9243 handle_command_reply: tid=0 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 client.9243 shutdown 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 msgr2=0x55bea75a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a7aa0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f19dc001da0 tx=0x7f19dc00e530 comp rx=0 tx=0).stop 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 shutdown_connections 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea75a9160 0x55bea75feec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea75a7aa0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea75f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 --2- 172.21.15.87:0/1276388837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea75f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19c7400640 0 client.9243 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 >> 172.21.15.87:0/1276388837 conn(0x55bea75aa2a0 msgr2=0x55bea75a29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 shutdown_connections 2024-04-25T10:21:49.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 1 -- 172.21.15.87:0/1276388837 wait complete. 2024-04-25T10:21:49.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.046+0000 7f19ec41c000 20 client.9243 trim_cache size 0 max 16384 2024-04-25T10:21:49.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 Processor -- start 2024-04-25T10:21:49.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- start start 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea7648f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea7649490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea76499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4152dc0 con 0x55bea75ee240 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e41521c0 con 0x55bea75ed920 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e41527c0 con 0x55bea75a3f30 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea76499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea76499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59738/0 (socket says 172.21.15.87:59738) 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 -- 172.21.15.87:0/4067112446 learned_addr learned my addr 172.21.15.87:0/4067112446 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e1e00640 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea7648f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19d9400640 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea7649490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 -- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 msgr2=0x55bea7648f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea7648f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 -- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 msgr2=0x55bea7649490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea7649490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 -- 172.21.15.87:0/4067112446 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bea75ee070 con 0x55bea75ee240 2024-04-25T10:21:49.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19e2800640 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea76499d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f19dc000eb0 tx=0x7f19dc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dcbfccdbf5b263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19d2000640 1 -- 172.21.15.87:0/4067112446 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00ea30 con 0x55bea75ee240 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19d2000640 1 -- 172.21.15.87:0/4067112446 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0050d0 con 0x55bea75ee240 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19d2000640 1 -- 172.21.15.87:0/4067112446 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc007260 con 0x55bea75ee240 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- 172.21.15.87:0/4067112446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 msgr2=0x55bea76499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea76499d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f19dc000eb0 tx=0x7f19dc00a000 comp rx=0 tx=0).stop 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- 172.21.15.87:0/4067112446 shutdown_connections 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75ee240 0x55bea76499d0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ed920 0x55bea7649490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4067112446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75a3f30 0x55bea7648f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- 172.21.15.87:0/4067112446 >> 172.21.15.87:0/4067112446 conn(0x55bea75aa2a0 msgr2=0x55bea75a7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- 172.21.15.87:0/4067112446 shutdown_connections 2024-04-25T10:21:49.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- 172.21.15.87:0/4067112446 wait complete. 2024-04-25T10:21:49.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 -- start start 2024-04-25T10:21:49.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.062+0000 7f19ec41c000 1 Processor -- start 2024-04-25T10:21:49.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75a3f30 0x55bea7634f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75ed920 0x55bea764ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ee240 0x55bea764b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea73f73d0 con 0x55bea75a3f30 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea74aed50 con 0x55bea75ee240 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bea738d980 con 0x55bea75ed920 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 10 client.0 _ll_register_callbacks cb 0x7f19cc0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75a3f30 0x55bea7634f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75a3f30 0x55bea7634f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59750/0 (socket says 172.21.15.87:59750) 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 -- 172.21.15.87:0/4024432509 learned_addr learned my addr 172.21.15.87:0/4024432509 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19d9400640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75ed920 0x55bea764ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e2800640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ee240 0x55bea764b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 -- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75ed920 msgr2=0x55bea764ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75ed920 0x55bea764ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 -- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ee240 msgr2=0x55bea764b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ee240 0x55bea764b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 -- 172.21.15.87:0/4024432509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bea7474600 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19d9400640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75ed920 0x55bea764ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19e1e00640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75a3f30 0x55bea7634f60 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f19d4055f90 tx=0x7f19d4055bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9f66f5a3c4b69b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 1 -- 172.21.15.87:0/4024432509 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d405a0c0 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 1 -- 172.21.15.87:0/4024432509 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19d4052900 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19dc0050d0 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 1 -- 172.21.15.87:0/4024432509 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d40564f0 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 10 client.9249 fetch_fsmap learned FSMap version 89 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55bea73879b0 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 1 -- 172.21.15.87:0/4024432509 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0050d0 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 1 -- 172.21.15.87:0/4024432509 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f19d405e510 con 0x55bea75a3f30 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 10 client.9249 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:21:49.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 10 client.9249 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 20 client.9249 populate_metadata read hostname 'smithi087' 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea764b7a0 0x55bea764bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 4 client.9249 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bea74002e0 con 0x55bea764b7a0 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19d9400640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea764b7a0 0x55bea764bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19d9400640 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea764b7a0 0x55bea764bb80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5814afd68bf6f873 server_cookie=6c400ba09078352d in_seq=0 out_seq=0 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 10 client.9249 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 1 -- 172.21.15.87:0/4024432509 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55bea74002e0 con 0x55bea764b7a0 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 10 client.9249 handle_command_reply: tid=0 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 client.9249 shutdown 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75a3f30 msgr2=0x55bea7634f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75a3f30 0x55bea7634f60 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f19d4055f90 tx=0x7f19d4055bf0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 shutdown_connections 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55bea764b7a0 0x55bea764bb80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bea75ee240 0x55bea764b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bea75ed920 0x55bea764ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19c7400640 0 client.9249 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 --2- 172.21.15.87:0/4024432509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bea75a3f30 0x55bea7634f60 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 >> 172.21.15.87:0/4024432509 conn(0x55bea75aa2a0 msgr2=0x55bea75f5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 shutdown_connections 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 1 -- 172.21.15.87:0/4024432509 wait complete. 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.066+0000 7f19ec41c000 20 client.9249 trim_cache size 0 max 16384 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:21:49.075 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 9114, 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8944079089999999, 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 89, 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 124, 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 124, 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.733950882 2024-04-25T10:21:49.076 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:21:49.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19c0066f60 msgr2=0x7f19c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19c0066f60 0x7f19c0069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af000 tx=0x7f19cc004740 comp rx=0 tx=0).stop 2024-04-25T10:21:49.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 msgr2=0x7f19e41f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e41f71d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f19d4007f20 tx=0x7f19d400e570 comp rx=0 tx=0).stop 2024-04-25T10:21:49.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 shutdown_connections 2024-04-25T10:21:49.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f19c0066f60 0x7f19c0069420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e415bc40 0x7f19e40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e40cf930 0x7f19e41f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 --2- 172.21.15.87:0/1417554966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19e40cda10 0x7f19e41f71d0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 >> 172.21.15.87:0/1417554966 conn(0x7f19e4066b70 msgr2=0x7f19e40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 shutdown_connections 2024-04-25T10:21:49.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.070+0000 7f19e9000640 1 -- 172.21.15.87:0/1417554966 wait complete. 2024-04-25T10:21:49.090 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:21:49.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.222+0000 7fbd82e00640 1 Processor -- start 2024-04-25T10:21:49.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.222+0000 7fbd82e00640 1 -- start start 2024-04-25T10:21:49.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd82e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c06ca50 0x7fbd7c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd82e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbd7c070310 0x7fbd7c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd82e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c0b4470 con 0x7fbd7c0cf930 2024-04-25T10:21:49.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd82e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7c0cf4d0 con 0x7fbd7c070310 2024-04-25T10:21:49.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd82e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0b4cd0 con 0x7fbd7c06ca50 2024-04-25T10:21:49.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd7be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbd7c0cf930 0x7fbd7c0cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:50194/0 (socket says 172.21.15.87:50194) 2024-04-25T10:21:49.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd7be00640 1 -- 172.21.15.87:0/3735490984 learned_addr learned my addr 172.21.15.87:0/3735490984 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:49.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd80e00640 1 --2- 172.21.15.87:0/3735490984 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c070310 0x7fbd7c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd73400640 1 --2- 172.21.15.87:0/3735490984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c06ca50 0x7fbd7c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd7b400640 1 -- 172.21.15.87:0/3735490984 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807938031 0 0) 0x7fbd7c0b4470 con 0x7fbd7c0cf930 2024-04-25T10:21:49.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd7b400640 1 -- 172.21.15.87:0/3735490984 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd60003530 con 0x7fbd7c0cf930 2024-04-25T10:21:49.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd73400640 1 -- 172.21.15.87:0/3735490984 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c070310 msgr2=0x7fbd7c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd73400640 1 --2- 172.21.15.87:0/3735490984 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c070310 0x7fbd7c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd73400640 1 -- 172.21.15.87:0/3735490984 >> v1:172.21.15.169:6789/0 conn(0x7fbd7c0cf930 legacy=0x7fbd7c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd73400640 1 -- 172.21.15.87:0/3735490984 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0b4710 con 0x7fbd7c06ca50 2024-04-25T10:21:49.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.226+0000 7fbd73400640 1 --2- 172.21.15.87:0/3735490984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c06ca50 0x7fbd7c0cef90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbd640097f0 tx=0x7fbd6401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2affe2235ede1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd7b400640 1 -- 172.21.15.87:0/3735490984 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640b0070 con 0x7fbd7c06ca50 2024-04-25T10:21:49.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd7b400640 1 -- 172.21.15.87:0/3735490984 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd640ab600 con 0x7fbd7c06ca50 2024-04-25T10:21:49.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd7b400640 1 -- 172.21.15.87:0/3735490984 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640ab9d0 con 0x7fbd7c06ca50 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 -- 172.21.15.87:0/3735490984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c06ca50 msgr2=0x7fbd7c0cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 --2- 172.21.15.87:0/3735490984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c06ca50 0x7fbd7c0cef90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbd640097f0 tx=0x7fbd6401c410 comp rx=0 tx=0).stop 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 -- 172.21.15.87:0/3735490984 shutdown_connections 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 --2- 172.21.15.87:0/3735490984 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c070310 0x7fbd7c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 --2- 172.21.15.87:0/3735490984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c06ca50 0x7fbd7c0cef90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 -- 172.21.15.87:0/3735490984 >> 172.21.15.87:0/3735490984 conn(0x7fbd7c066b70 msgr2=0x7fbd7c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 -- 172.21.15.87:0/3735490984 shutdown_connections 2024-04-25T10:21:49.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 -- 172.21.15.87:0/3735490984 wait complete. 2024-04-25T10:21:49.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.230+0000 7fbd82e00640 1 Processor -- start 2024-04-25T10:21:49.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 -- start start 2024-04-25T10:21:49.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd7c06ca50 0x7fbd7c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c070310 0x7fbd7c1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c0cf930 0x7fbd7c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0b2e30 con 0x7fbd7c06ca50 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c072050 con 0x7fbd7c070310 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0b2fb0 con 0x7fbd7c0cf930 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd7c06ca50 0x7fbd7c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd7c06ca50 0x7fbd7c156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59760/0 (socket says 172.21.15.87:59760) 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 -- 172.21.15.87:0/2550956919 learned_addr learned my addr 172.21.15.87:0/2550956919 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd73400640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c070310 0x7fbd7c1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd80e00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c0cf930 0x7fbd7c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 -- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c0cf930 msgr2=0x7fbd7c1543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c0cf930 0x7fbd7c1543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 -- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c070310 msgr2=0x7fbd7c1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c070310 0x7fbd7c1555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 -- 172.21.15.87:0/2550956919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c155b20 con 0x7fbd7c06ca50 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd7be00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd7c06ca50 0x7fbd7c156e60 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cb40 tx=0x7fbd6c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd78c00640 1 -- 172.21.15.87:0/2550956919 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c017070 con 0x7fbd7c06ca50 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd78c00640 1 -- 172.21.15.87:0/2550956919 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00aca0 con 0x7fbd7c06ca50 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd78c00640 1 -- 172.21.15.87:0/2550956919 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c012620 con 0x7fbd7c06ca50 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c154930 con 0x7fbd7c06ca50 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c1560a0 con 0x7fbd7c06ca50 2024-04-25T10:21:49.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.234+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c04ef30 con 0x7fbd7c06ca50 2024-04-25T10:21:49.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.238+0000 7fbd78c00640 1 -- 172.21.15.87:0/2550956919 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd6c01a070 con 0x7fbd7c06ca50 2024-04-25T10:21:49.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.238+0000 7fbd78c00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd58066f60 0x7fbd58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.238+0000 7fbd78c00640 1 -- 172.21.15.87:0/2550956919 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd6c084e50 con 0x7fbd7c06ca50 2024-04-25T10:21:49.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.238+0000 7fbd73400640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd58066f60 0x7fbd58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.238+0000 7fbd73400640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd58066f60 0x7fbd58069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbd640ad000 tx=0x7fbd64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.242+0000 7fbd78c00640 1 -- 172.21.15.87:0/2550956919 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd6c055770 con 0x7fbd7c06ca50 2024-04-25T10:21:49.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbd7c0b1070 con 0x7fbd7c06ca50 2024-04-25T10:21:49.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd78c00640 1 -- 172.21.15.87:0/2550956919 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbd6c059810 con 0x7fbd7c06ca50 2024-04-25T10:21:49.433 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:21:49.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd58066f60 msgr2=0x7fbd58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd58066f60 0x7fbd58069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbd640ad000 tx=0x7fbd64008040 comp rx=0 tx=0).stop 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd7c06ca50 msgr2=0x7fbd7c156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd7c06ca50 0x7fbd7c156e60 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cb40 tx=0x7fbd6c00a060 comp rx=0 tx=0).stop 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 shutdown_connections 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbd58066f60 0x7fbd58069420 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c0cf930 0x7fbd7c1543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c070310 0x7fbd7c1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 --2- 172.21.15.87:0/2550956919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd7c06ca50 0x7fbd7c156e60 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 >> 172.21.15.87:0/2550956919 conn(0x7fbd7c066b70 msgr2=0x7fbd7c0cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 shutdown_connections 2024-04-25T10:21:49.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:49.430+0000 7fbd82e00640 1 -- 172.21.15.87:0/2550956919 wait complete. 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:21:49.454 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:49.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:49.455 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:21:49.464 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:21:49.464 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:21:49.464 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:21:49.464 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.464 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41904sec preferred_lft 41904sec 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:21:49.465 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:21:49.466 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.467 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:21:49.467 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:21:49.467 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:21:49.467 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:49.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:49.467 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:21:49.531 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:21:49.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:49.535 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:21:49.587 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:21:49.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:49.589 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:21:49.589 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:49.589 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:21:49.589 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:21:49.589 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:21:49.589 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:49.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:49.679 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:21:49.731 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:21:49.731 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:21:49.732 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:49.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:49.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:21:49.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:21:49.798 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:49.847 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:49.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:21:49.847 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:49.902 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:21:49.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:49.902 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:21:49.994 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:21:49.994 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:21:49.994 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:21:50.009 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:21:50.009 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:21:50.010 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:21:50.010 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:21:50.010 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:21:50.018 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:21:50.018 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:21:50.019 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:21:50.019 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:21:50.019 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:21:50.019 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:21:50.019 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:21:50.019 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:21:50.022 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:21:50.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:50.022 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:50.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:50.042 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:21:50.106 INFO:teuthology.orchestra.run.smithi087.stdout:client7487 2024-04-25T10:21:50.106 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:21:50.106 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:50.155 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:50.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb895200640 1 Processor -- start 2024-04-25T10:21:50.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb895200640 1 -- start start 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb895200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8900d0040 0x7fb89015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb895200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb89015b740 0x7fb8900b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb895200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb890071050 con 0x7fb8901550b0 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb895200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89015d040 con 0x7fb8900d0040 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb895200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8900712f0 con 0x7fb89015b740 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8900d0040 0x7fb89015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8900d0040 0x7fb89015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59776/0 (socket says 172.21.15.87:59776) 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88d400640 1 -- 172.21.15.87:0/3565195064 learned_addr learned my addr 172.21.15.87:0/3565195064 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88e800640 1 --2- 172.21.15.87:0/3565195064 >> v2:172.21.15.169:3300/0 conn(0x7fb89015b740 0x7fb8900b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88d400640 1 -- 172.21.15.87:0/3565195064 >> v2:172.21.15.169:3300/0 conn(0x7fb89015b740 msgr2=0x7fb8900b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88d400640 1 --2- 172.21.15.87:0/3565195064 >> v2:172.21.15.169:3300/0 conn(0x7fb89015b740 0x7fb8900b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88d400640 1 -- 172.21.15.87:0/3565195064 >> v1:172.21.15.169:6789/0 conn(0x7fb8901550b0 legacy=0x7fb8900cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88d400640 1 -- 172.21.15.87:0/3565195064 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900715d0 con 0x7fb8900d0040 2024-04-25T10:21:50.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.334+0000 7fb88e800640 1 --2- 172.21.15.87:0/3565195064 >> v2:172.21.15.169:3300/0 conn(0x7fb89015b740 0x7fb8900b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb88d400640 1 --2- 172.21.15.87:0/3565195064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb89015b200 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fb8900d04d0 tx=0x7fb87c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2bf19bff581350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb88ca00640 1 -- 172.21.15.87:0/3565195064 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0aa650 con 0x7fb8900d0040 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb88ca00640 1 -- 172.21.15.87:0/3565195064 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c0af070 con 0x7fb8900d0040 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb88ca00640 1 -- 172.21.15.87:0/3565195064 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0aaa10 con 0x7fb8900d0040 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 -- 172.21.15.87:0/3565195064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 msgr2=0x7fb89015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 --2- 172.21.15.87:0/3565195064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb89015b200 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fb8900d04d0 tx=0x7fb87c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 -- 172.21.15.87:0/3565195064 shutdown_connections 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 --2- 172.21.15.87:0/3565195064 >> v2:172.21.15.169:3300/0 conn(0x7fb89015b740 0x7fb8900b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 --2- 172.21.15.87:0/3565195064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb89015b200 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 -- 172.21.15.87:0/3565195064 >> 172.21.15.87:0/3565195064 conn(0x7fb890066b70 msgr2=0x7fb890066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 -- 172.21.15.87:0/3565195064 shutdown_connections 2024-04-25T10:21:50.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.338+0000 7fb895200640 1 -- 172.21.15.87:0/3565195064 wait complete. 2024-04-25T10:21:50.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 Processor -- start 2024-04-25T10:21:50.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 -- start start 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb8901542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8901550b0 0x7fb89014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb89015b740 0x7fb890152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015bcd0 con 0x7fb8900d0040 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015bb50 con 0x7fb8901550b0 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015c030 con 0x7fb89015b740 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb8901542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb8901542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59780/0 (socket says 172.21.15.87:59780) 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 -- 172.21.15.87:0/2049984121 learned_addr learned my addr 172.21.15.87:0/2049984121 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88d400640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8901550b0 0x7fb89014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88e800640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb89015b740 0x7fb890152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 -- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb89015b740 msgr2=0x7fb890152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb89015b740 0x7fb890152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 -- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8901550b0 msgr2=0x7fb89014c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8901550b0 0x7fb89014c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 -- 172.21.15.87:0/2049984121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890153210 con 0x7fb8900d0040 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88d400640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8901550b0 0x7fb89014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb88de00640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb8901542b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c450 tx=0x7fb88000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb886000640 1 -- 172.21.15.87:0/2049984121 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880016020 con 0x7fb8900d0040 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb886000640 1 -- 172.21.15.87:0/2049984121 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8800052d0 con 0x7fb8900d0040 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8901534f0 con 0x7fb8900d0040 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb886000640 1 -- 172.21.15.87:0/2049984121 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880007260 con 0x7fb8900d0040 2024-04-25T10:21:50.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb890153ab0 con 0x7fb8900d0040 2024-04-25T10:21:50.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.342+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89004ef30 con 0x7fb8900d0040 2024-04-25T10:21:50.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.346+0000 7fb886000640 1 -- 172.21.15.87:0/2049984121 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb880007400 con 0x7fb8900d0040 2024-04-25T10:21:50.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.346+0000 7fb886000640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb870066f90 0x7fb870069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.346+0000 7fb886000640 1 -- 172.21.15.87:0/2049984121 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb880084aa0 con 0x7fb8900d0040 2024-04-25T10:21:50.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.346+0000 7fb88d400640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb870066f90 0x7fb870069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.346+0000 7fb88d400640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb870066f90 0x7fb870069450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0ae000 tx=0x7fb87c0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.350+0000 7fb886000640 1 -- 172.21.15.87:0/2049984121 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb880020020 con 0x7fb8900d0040 2024-04-25T10:21:50.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb890152260 con 0x7fb8900d0040 2024-04-25T10:21:50.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb886000640 1 -- 172.21.15.87:0/2049984121 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fb880005480 con 0x7fb8900d0040 2024-04-25T10:21:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:50.537 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:21:50.537 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:21:50.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb870066f90 msgr2=0x7fb870069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb870066f90 0x7fb870069450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0ae000 tx=0x7fb87c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:50.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 msgr2=0x7fb8901542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb8901542b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c450 tx=0x7fb88000c910 comp rx=0 tx=0).stop 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 shutdown_connections 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb870066f90 0x7fb870069450 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb89015b740 0x7fb890152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8901550b0 0x7fb89014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 --2- 172.21.15.87:0/2049984121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8900d0040 0x7fb8901542b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 >> 172.21.15.87:0/2049984121 conn(0x7fb890066b70 msgr2=0x7fb890151500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 shutdown_connections 2024-04-25T10:21:50.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.534+0000 7fb895200640 1 -- 172.21.15.87:0/2049984121 wait complete. 2024-04-25T10:21:50.559 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:50.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5cba00640 1 Processor -- start 2024-04-25T10:21:50.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5cba00640 1 -- start start 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5cba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c4072c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5cba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5c40b14f0 0x7fc5c40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5cba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c4073150 con 0x7fc5c4074160 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5cba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c40afb10 con 0x7fc5c40b14f0 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5cba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4074690 con 0x7fc5c415bb20 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5c3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc5c4074160 0x7fc5c40b0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53516/0 (socket says 172.21.15.87:53516) 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5c3e00640 1 -- 172.21.15.87:0/39942992 learned_addr learned my addr 172.21.15.87:0/39942992 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5c9000640 1 --2- 172.21.15.87:0/39942992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c4072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.690+0000 7fc5c9a00640 1 --2- 172.21.15.87:0/39942992 >> v2:172.21.15.169:3300/0 conn(0x7fc5c40b14f0 0x7fc5c40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c3400640 1 -- 172.21.15.87:0/39942992 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770164969 0 0) 0x7fc5c4073150 con 0x7fc5c4074160 2024-04-25T10:21:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c3400640 1 -- 172.21.15.87:0/39942992 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5ac003530 con 0x7fc5c4074160 2024-04-25T10:21:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c9000640 1 -- 172.21.15.87:0/39942992 >> v2:172.21.15.169:3300/0 conn(0x7fc5c40b14f0 msgr2=0x7fc5c40cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c9000640 1 --2- 172.21.15.87:0/39942992 >> v2:172.21.15.169:3300/0 conn(0x7fc5c40b14f0 0x7fc5c40cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c9000640 1 -- 172.21.15.87:0/39942992 >> v1:172.21.15.87:6789/0 conn(0x7fc5c4074160 legacy=0x7fc5c40b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c9000640 1 -- 172.21.15.87:0/39942992 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40afd60 con 0x7fc5c415bb20 2024-04-25T10:21:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c9000640 1 --2- 172.21.15.87:0/39942992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c4072c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097f0 tx=0x7fc5b801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8726b2cfdec94a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c3400640 1 -- 172.21.15.87:0/39942992 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80b0070 con 0x7fc5c415bb20 2024-04-25T10:21:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c3400640 1 -- 172.21.15.87:0/39942992 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b80ab600 con 0x7fc5c415bb20 2024-04-25T10:21:50.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.694+0000 7fc5c3400640 1 -- 172.21.15.87:0/39942992 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80ab9d0 con 0x7fc5c415bb20 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- 172.21.15.87:0/39942992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 msgr2=0x7fc5c4072c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 --2- 172.21.15.87:0/39942992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c4072c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097f0 tx=0x7fc5b801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- 172.21.15.87:0/39942992 shutdown_connections 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 --2- 172.21.15.87:0/39942992 >> v2:172.21.15.169:3300/0 conn(0x7fc5c40b14f0 0x7fc5c40cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 --2- 172.21.15.87:0/39942992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c4072c10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- 172.21.15.87:0/39942992 >> 172.21.15.87:0/39942992 conn(0x7fc5c40b1d50 msgr2=0x7fc5c40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- 172.21.15.87:0/39942992 shutdown_connections 2024-04-25T10:21:50.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- 172.21.15.87:0/39942992 wait complete. 2024-04-25T10:21:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 Processor -- start 2024-04-25T10:21:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- start start 2024-04-25T10:21:50.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5c4074160 0x7fc5c4099640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c40b14f0 0x7fc5c414ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c40b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4168c50 con 0x7fc5c40b14f0 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4168ad0 con 0x7fc5c4074160 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4168dd0 con 0x7fc5c415bb20 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c40b14f0 0x7fc5c414ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c40b14f0 0x7fc5c414ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59782/0 (socket says 172.21.15.87:59782) 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 -- 172.21.15.87:0/3781974526 learned_addr learned my addr 172.21.15.87:0/3781974526 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c9000640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5c4074160 0x7fc5c4099640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c9a00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c40b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 -- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 msgr2=0x7fc5c40b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c40b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 -- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5c4074160 msgr2=0x7fc5c4099640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5c4074160 0x7fc5c4099640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 -- 172.21.15.87:0/3781974526 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4099b80 con 0x7fc5c40b14f0 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c3e00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c40b14f0 0x7fc5c414ef10 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c3f0 tx=0x7fc5b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c0c00640 1 -- 172.21.15.87:0/3781974526 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0017070 con 0x7fc5c40b14f0 2024-04-25T10:21:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c0c00640 1 -- 172.21.15.87:0/3781974526 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b0005cf0 con 0x7fc5c40b14f0 2024-04-25T10:21:50.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5c0c00640 1 -- 172.21.15.87:0/3781974526 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00072f0 con 0x7fc5c40b14f0 2024-04-25T10:21:50.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.698+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c415abc0 con 0x7fc5c40b14f0 2024-04-25T10:21:50.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.702+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c415b180 con 0x7fc5c40b14f0 2024-04-25T10:21:50.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.702+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c404ef30 con 0x7fc5c40b14f0 2024-04-25T10:21:50.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.702+0000 7fc5c0c00640 1 -- 172.21.15.87:0/3781974526 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5b001a070 con 0x7fc5c40b14f0 2024-04-25T10:21:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.702+0000 7fc5c0c00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5a4066f90 0x7fc5a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.702+0000 7fc5c0c00640 1 -- 172.21.15.87:0/3781974526 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5b0084350 con 0x7fc5c40b14f0 2024-04-25T10:21:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.702+0000 7fc5c9000640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5a4066f90 0x7fc5a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.702+0000 7fc5c9000640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5a4066f90 0x7fc5a4069450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ad000 tx=0x7fc5b80a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.706+0000 7fc5c0c00640 1 -- 172.21.15.87:0/3781974526 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5c4099b80 con 0x7fc5c40b14f0 2024-04-25T10:21:50.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5c4151670 con 0x7fc5c40b14f0 2024-04-25T10:21:50.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5c0c00640 1 -- 172.21.15.87:0/3781974526 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fc5b0054d00 con 0x7fc5c40b14f0 2024-04-25T10:21:50.896 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:21:50.896 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:21:50.896 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:21:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5a4066f90 msgr2=0x7fc5a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5a4066f90 0x7fc5a4069450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ad000 tx=0x7fc5b80a6000 comp rx=0 tx=0).stop 2024-04-25T10:21:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c40b14f0 msgr2=0x7fc5c414ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c40b14f0 0x7fc5c414ef10 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c3f0 tx=0x7fc5b000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 shutdown_connections 2024-04-25T10:21:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5a4066f90 0x7fc5a4069450 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5c415bb20 0x7fc5c40b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c40b14f0 0x7fc5c414ef10 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 --2- 172.21.15.87:0/3781974526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5c4074160 0x7fc5c4099640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 >> 172.21.15.87:0/3781974526 conn(0x7fc5c40b1d50 msgr2=0x7fc5c40ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 shutdown_connections 2024-04-25T10:21:50.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:21:50.894+0000 7fc5cba00640 1 -- 172.21.15.87:0/3781974526 wait complete. 2024-04-25T10:21:50.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:50.916 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:50.916 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:50.917 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:21:50.917 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:50.917 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:50.917 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:50.917 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:50.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:50.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:50.957 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151135 s, 69.4 MB/s 2024-04-25T10:21:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:50.979 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020253 s, 51.8 MB/s 2024-04-25T10:21:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186866 s, 56.1 MB/s 2024-04-25T10:21:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016747 s, 62.6 MB/s 2024-04-25T10:21:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244022 s, 43.0 MB/s 2024-04-25T10:21:51.067 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.067 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214537 s, 48.9 MB/s 2024-04-25T10:21:51.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.089 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196994 s, 53.2 MB/s 2024-04-25T10:21:51.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.106 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155897 s, 67.3 MB/s 2024-04-25T10:21:51.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.127 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196036 s, 53.5 MB/s 2024-04-25T10:21:51.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170872 s, 61.4 MB/s 2024-04-25T10:21:51.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195452 s, 53.6 MB/s 2024-04-25T10:21:51.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.195 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.195 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258076 s, 40.6 MB/s 2024-04-25T10:21:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177886 s, 58.9 MB/s 2024-04-25T10:21:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174911 s, 59.9 MB/s 2024-04-25T10:21:51.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.257 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208338 s, 50.3 MB/s 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01945 s, 53.9 MB/s 2024-04-25T10:21:51.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.300 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200476 s, 52.3 MB/s 2024-04-25T10:21:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015104 s, 69.4 MB/s 2024-04-25T10:21:51.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.338 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195571 s, 53.6 MB/s 2024-04-25T10:21:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205106 s, 51.1 MB/s 2024-04-25T10:21:51.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.383 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020762 s, 50.5 MB/s 2024-04-25T10:21:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187234 s, 56.0 MB/s 2024-04-25T10:21:51.425 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.425 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.425 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191929 s, 54.6 MB/s 2024-04-25T10:21:51.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.444 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.444 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176165 s, 59.5 MB/s 2024-04-25T10:21:51.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144016 s, 72.8 MB/s 2024-04-25T10:21:51.476 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.476 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.476 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144087 s, 72.8 MB/s 2024-04-25T10:21:51.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.493 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144497 s, 72.6 MB/s 2024-04-25T10:21:51.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.509 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149129 s, 70.3 MB/s 2024-04-25T10:21:51.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143373 s, 73.1 MB/s 2024-04-25T10:21:51.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.544 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169311 s, 61.9 MB/s 2024-04-25T10:21:51.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199357 s, 52.6 MB/s 2024-04-25T10:21:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016465 s, 63.7 MB/s 2024-04-25T10:21:51.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:51.586 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:51.586 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:51.586 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:21:51.586 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:51.586 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:51.586 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:51.586 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:51.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140205 s, 74.8 MB/s 2024-04-25T10:21:51.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.673 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166876 s, 62.8 MB/s 2024-04-25T10:21:51.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126725 s, 82.7 MB/s 2024-04-25T10:21:51.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153581 s, 68.3 MB/s 2024-04-25T10:21:51.728 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.729 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224671 s, 46.7 MB/s 2024-04-25T10:21:51.748 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.748 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.749 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178989 s, 58.6 MB/s 2024-04-25T10:21:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183188 s, 57.2 MB/s 2024-04-25T10:21:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017774 s, 59.0 MB/s 2024-04-25T10:21:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170805 s, 61.4 MB/s 2024-04-25T10:21:51.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.833 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246519 s, 42.5 MB/s 2024-04-25T10:21:51.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.855 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191722 s, 54.7 MB/s 2024-04-25T10:21:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149553 s, 70.1 MB/s 2024-04-25T10:21:51.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148644 s, 70.5 MB/s 2024-04-25T10:21:51.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175734 s, 59.7 MB/s 2024-04-25T10:21:51.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150764 s, 69.6 MB/s 2024-04-25T10:21:51.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.946 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201887 s, 51.9 MB/s 2024-04-25T10:21:51.965 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.965 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.965 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171897 s, 61.0 MB/s 2024-04-25T10:21:51.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150646 s, 69.6 MB/s 2024-04-25T10:21:52.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.002 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187935 s, 55.8 MB/s 2024-04-25T10:21:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.019 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151445 s, 69.2 MB/s 2024-04-25T10:21:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179566 s, 58.4 MB/s 2024-04-25T10:21:52.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.060 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194818 s, 53.8 MB/s 2024-04-25T10:21:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169456 s, 61.9 MB/s 2024-04-25T10:21:52.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149924 s, 69.9 MB/s 2024-04-25T10:21:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164908 s, 63.6 MB/s 2024-04-25T10:21:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.133 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175383 s, 59.8 MB/s 2024-04-25T10:21:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014922 s, 70.3 MB/s 2024-04-25T10:21:52.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.167 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149072 s, 70.3 MB/s 2024-04-25T10:21:52.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.185 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016916 s, 62.0 MB/s 2024-04-25T10:21:52.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.205 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182262 s, 57.5 MB/s 2024-04-25T10:21:52.221 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.221 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.221 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141767 s, 74.0 MB/s 2024-04-25T10:21:52.238 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.238 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.238 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153443 s, 68.3 MB/s 2024-04-25T10:21:52.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:52.240 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:52.240 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:52.240 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:21:52.240 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:52.240 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:52.240 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:52.240 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:52.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184547 s, 56.8 MB/s 2024-04-25T10:21:52.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.333 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150267 s, 69.8 MB/s 2024-04-25T10:21:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.350 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162954 s, 64.3 MB/s 2024-04-25T10:21:52.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.366 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143706 s, 73.0 MB/s 2024-04-25T10:21:52.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.383 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149812 s, 70.0 MB/s 2024-04-25T10:21:52.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183446 s, 57.2 MB/s 2024-04-25T10:21:52.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.419 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140569 s, 74.6 MB/s 2024-04-25T10:21:52.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.435 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144308 s, 72.7 MB/s 2024-04-25T10:21:52.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171628 s, 61.1 MB/s 2024-04-25T10:21:52.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151869 s, 69.0 MB/s 2024-04-25T10:21:52.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179651 s, 58.4 MB/s 2024-04-25T10:21:52.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151985 s, 69.0 MB/s 2024-04-25T10:21:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150348 s, 69.7 MB/s 2024-04-25T10:21:52.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.541 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145662 s, 72.0 MB/s 2024-04-25T10:21:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.561 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181455 s, 57.8 MB/s 2024-04-25T10:21:52.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.583 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206022 s, 50.9 MB/s 2024-04-25T10:21:52.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.599 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014401 s, 72.8 MB/s 2024-04-25T10:21:52.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.615 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013914 s, 75.4 MB/s 2024-04-25T10:21:52.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.638 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218076 s, 48.1 MB/s 2024-04-25T10:21:52.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.654 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014164 s, 74.0 MB/s 2024-04-25T10:21:52.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.676 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202115 s, 51.9 MB/s 2024-04-25T10:21:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142622 s, 73.5 MB/s 2024-04-25T10:21:52.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.713 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184796 s, 56.7 MB/s 2024-04-25T10:21:52.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.730 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153346 s, 68.4 MB/s 2024-04-25T10:21:52.751 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.751 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.752 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196984 s, 53.2 MB/s 2024-04-25T10:21:52.768 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.768 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.768 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148748 s, 70.5 MB/s 2024-04-25T10:21:52.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.784 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137852 s, 76.1 MB/s 2024-04-25T10:21:52.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.802 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162459 s, 64.5 MB/s 2024-04-25T10:21:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142436 s, 73.6 MB/s 2024-04-25T10:21:52.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193565 s, 54.2 MB/s 2024-04-25T10:21:52.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.858 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170978 s, 61.3 MB/s 2024-04-25T10:21:52.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.874 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142182 s, 73.7 MB/s 2024-04-25T10:21:52.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:52.875 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:52.875 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:52.875 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:21:52.875 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:52.875 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:52.876 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:52.876 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:52.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.952 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185918 s, 56.4 MB/s 2024-04-25T10:21:52.971 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.971 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.971 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181871 s, 57.7 MB/s 2024-04-25T10:21:52.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:52.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:52.991 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185979 s, 56.4 MB/s 2024-04-25T10:21:53.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.008 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143514 s, 73.1 MB/s 2024-04-25T10:21:53.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.023 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136952 s, 76.6 MB/s 2024-04-25T10:21:53.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.047 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021768 s, 48.2 MB/s 2024-04-25T10:21:53.066 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.066 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.066 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178102 s, 58.9 MB/s 2024-04-25T10:21:53.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.090 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223677 s, 46.9 MB/s 2024-04-25T10:21:53.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160539 s, 65.3 MB/s 2024-04-25T10:21:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.134 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234452 s, 44.7 MB/s 2024-04-25T10:21:53.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.151 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158845 s, 66.0 MB/s 2024-04-25T10:21:53.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142356 s, 73.7 MB/s 2024-04-25T10:21:53.185 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.186 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170983 s, 61.3 MB/s 2024-04-25T10:21:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195537 s, 53.6 MB/s 2024-04-25T10:21:53.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164122 s, 63.9 MB/s 2024-04-25T10:21:53.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.243 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162713 s, 64.4 MB/s 2024-04-25T10:21:53.262 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.262 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.262 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177843 s, 59.0 MB/s 2024-04-25T10:21:53.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.282 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174091 s, 60.2 MB/s 2024-04-25T10:21:53.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.299 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.299 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155791 s, 67.3 MB/s 2024-04-25T10:21:53.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.320 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.320 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187872 s, 55.8 MB/s 2024-04-25T10:21:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.342 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210508 s, 49.8 MB/s 2024-04-25T10:21:53.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161256 s, 65.0 MB/s 2024-04-25T10:21:53.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.379 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165783 s, 63.2 MB/s 2024-04-25T10:21:53.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.395 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140863 s, 74.4 MB/s 2024-04-25T10:21:53.417 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.417 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.418 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209122 s, 50.1 MB/s 2024-04-25T10:21:53.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.436 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171413 s, 61.2 MB/s 2024-04-25T10:21:53.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141173 s, 74.3 MB/s 2024-04-25T10:21:53.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.467 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128308 s, 81.7 MB/s 2024-04-25T10:21:53.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.483 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148534 s, 70.6 MB/s 2024-04-25T10:21:53.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.502 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016841 s, 62.3 MB/s 2024-04-25T10:21:53.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.520 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162965 s, 64.3 MB/s 2024-04-25T10:21:53.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211644 s, 49.5 MB/s 2024-04-25T10:21:53.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:53.545 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:53.545 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:53.545 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:21:53.545 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:53.545 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:53.545 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:53.545 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:53.620 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.621 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195968 s, 53.5 MB/s 2024-04-25T10:21:53.636 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.637 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.637 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144569 s, 72.5 MB/s 2024-04-25T10:21:53.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138191 s, 75.9 MB/s 2024-04-25T10:21:53.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.670 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161087 s, 65.1 MB/s 2024-04-25T10:21:53.672 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:53.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199684 s, 52.5 MB/s 2024-04-25T10:21:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184488 s, 56.8 MB/s 2024-04-25T10:21:53.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.730 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157081 s, 66.8 MB/s 2024-04-25T10:21:53.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.746 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146781 s, 71.4 MB/s 2024-04-25T10:21:53.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.763 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148188 s, 70.8 MB/s 2024-04-25T10:21:53.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.779 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147691 s, 71.0 MB/s 2024-04-25T10:21:53.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182501 s, 57.5 MB/s 2024-04-25T10:21:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233859 s, 44.8 MB/s 2024-04-25T10:21:53.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.841 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147607 s, 71.0 MB/s 2024-04-25T10:21:53.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.858 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146827 s, 71.4 MB/s 2024-04-25T10:21:53.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.875 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159415 s, 65.8 MB/s 2024-04-25T10:21:53.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.895 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181358 s, 57.8 MB/s 2024-04-25T10:21:53.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.912 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149119 s, 70.3 MB/s 2024-04-25T10:21:53.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.933 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189171 s, 55.4 MB/s 2024-04-25T10:21:53.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.949 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148136 s, 70.8 MB/s 2024-04-25T10:21:53.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.968 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174604 s, 60.1 MB/s 2024-04-25T10:21:53.984 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:53.984 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:53.984 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141594 s, 74.1 MB/s 2024-04-25T10:21:54.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.001 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.001 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145167 s, 72.2 MB/s 2024-04-25T10:21:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143027 s, 73.3 MB/s 2024-04-25T10:21:54.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.036 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178798 s, 58.6 MB/s 2024-04-25T10:21:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01481 s, 70.8 MB/s 2024-04-25T10:21:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.069 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146735 s, 71.5 MB/s 2024-04-25T10:21:54.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.086 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149504 s, 70.1 MB/s 2024-04-25T10:21:54.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.104 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163067 s, 64.3 MB/s 2024-04-25T10:21:54.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148132 s, 70.8 MB/s 2024-04-25T10:21:54.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.147 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237003 s, 44.2 MB/s 2024-04-25T10:21:54.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.164 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156555 s, 67.0 MB/s 2024-04-25T10:21:54.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014636 s, 71.6 MB/s 2024-04-25T10:21:54.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:54.182 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:54.182 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:54.182 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:21:54.182 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:54.182 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:54.182 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:54.182 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:54.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147655 s, 71.0 MB/s 2024-04-25T10:21:54.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207112 s, 50.6 MB/s 2024-04-25T10:21:54.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.291 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147973 s, 70.9 MB/s 2024-04-25T10:21:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017762 s, 59.0 MB/s 2024-04-25T10:21:54.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.327 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148239 s, 70.7 MB/s 2024-04-25T10:21:54.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020078 s, 52.2 MB/s 2024-04-25T10:21:54.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.363 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123671 s, 84.8 MB/s 2024-04-25T10:21:54.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.381 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165332 s, 63.4 MB/s 2024-04-25T10:21:54.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.400 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165656 s, 63.3 MB/s 2024-04-25T10:21:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.416 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144681 s, 72.5 MB/s 2024-04-25T10:21:54.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152789 s, 68.6 MB/s 2024-04-25T10:21:54.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.449 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146515 s, 71.6 MB/s 2024-04-25T10:21:54.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.470 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.470 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173019 s, 60.6 MB/s 2024-04-25T10:21:54.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.488 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.488 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175868 s, 59.6 MB/s 2024-04-25T10:21:54.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.504 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149195 s, 70.3 MB/s 2024-04-25T10:21:54.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.520 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142961 s, 73.3 MB/s 2024-04-25T10:21:54.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.537 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.537 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146275 s, 71.7 MB/s 2024-04-25T10:21:54.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.559 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203734 s, 51.5 MB/s 2024-04-25T10:21:54.578 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.578 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.578 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017533 s, 59.8 MB/s 2024-04-25T10:21:54.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141722 s, 74.0 MB/s 2024-04-25T10:21:54.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.611 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156138 s, 67.2 MB/s 2024-04-25T10:21:54.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013132 s, 79.8 MB/s 2024-04-25T10:21:54.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.646 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.646 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182741 s, 57.4 MB/s 2024-04-25T10:21:54.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.663 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146514 s, 71.6 MB/s 2024-04-25T10:21:54.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.682 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175419 s, 59.8 MB/s 2024-04-25T10:21:54.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.701 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169333 s, 61.9 MB/s 2024-04-25T10:21:54.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.717 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145554 s, 72.0 MB/s 2024-04-25T10:21:54.734 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.734 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.734 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152235 s, 68.9 MB/s 2024-04-25T10:21:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146905 s, 71.4 MB/s 2024-04-25T10:21:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147949 s, 70.9 MB/s 2024-04-25T10:21:54.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.784 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148033 s, 70.8 MB/s 2024-04-25T10:21:54.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195879 s, 53.5 MB/s 2024-04-25T10:21:54.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:54.806 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:54.807 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:54.807 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:21:54.807 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:54.807 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:54.807 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:54.807 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:54.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.875 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146321 s, 71.7 MB/s 2024-04-25T10:21:54.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.894 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169761 s, 61.8 MB/s 2024-04-25T10:21:54.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.912 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167036 s, 62.8 MB/s 2024-04-25T10:21:54.937 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.938 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.938 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234751 s, 44.7 MB/s 2024-04-25T10:21:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167726 s, 62.5 MB/s 2024-04-25T10:21:54.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.978 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.978 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205139 s, 51.1 MB/s 2024-04-25T10:21:54.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:54.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:54.995 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015092 s, 69.5 MB/s 2024-04-25T10:21:55.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.019 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220796 s, 47.5 MB/s 2024-04-25T10:21:55.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.036 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148998 s, 70.4 MB/s 2024-04-25T10:21:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.052 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014913 s, 70.3 MB/s 2024-04-25T10:21:55.066 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.066 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.066 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120064 s, 87.3 MB/s 2024-04-25T10:21:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145741 s, 71.9 MB/s 2024-04-25T10:21:55.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.102 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181368 s, 57.8 MB/s 2024-04-25T10:21:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.125 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205315 s, 51.1 MB/s 2024-04-25T10:21:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176484 s, 59.4 MB/s 2024-04-25T10:21:55.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.166 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198789 s, 52.7 MB/s 2024-04-25T10:21:55.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.183 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149896 s, 70.0 MB/s 2024-04-25T10:21:55.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144915 s, 72.4 MB/s 2024-04-25T10:21:55.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.218 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169654 s, 61.8 MB/s 2024-04-25T10:21:55.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150123 s, 69.8 MB/s 2024-04-25T10:21:55.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.254 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177878 s, 58.9 MB/s 2024-04-25T10:21:55.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182072 s, 57.6 MB/s 2024-04-25T10:21:55.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.293 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172086 s, 60.9 MB/s 2024-04-25T10:21:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169975 s, 61.7 MB/s 2024-04-25T10:21:55.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.330 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160524 s, 65.3 MB/s 2024-04-25T10:21:55.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163983 s, 63.9 MB/s 2024-04-25T10:21:55.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168898 s, 62.1 MB/s 2024-04-25T10:21:55.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211607 s, 49.6 MB/s 2024-04-25T10:21:55.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.407 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157119 s, 66.7 MB/s 2024-04-25T10:21:55.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154655 s, 67.8 MB/s 2024-04-25T10:21:55.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177201 s, 59.2 MB/s 2024-04-25T10:21:55.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.465 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196203 s, 53.4 MB/s 2024-04-25T10:21:55.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:55.466 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:55.467 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:55.467 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:21:55.467 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:55.467 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:55.467 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:55.467 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179243 s, 58.5 MB/s 2024-04-25T10:21:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146205 s, 71.7 MB/s 2024-04-25T10:21:55.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146713 s, 71.5 MB/s 2024-04-25T10:21:55.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.593 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157541 s, 66.6 MB/s 2024-04-25T10:21:55.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.613 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185371 s, 56.6 MB/s 2024-04-25T10:21:55.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197228 s, 53.2 MB/s 2024-04-25T10:21:55.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179521 s, 58.4 MB/s 2024-04-25T10:21:55.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.673 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016558 s, 63.3 MB/s 2024-04-25T10:21:55.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.693 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177373 s, 59.1 MB/s 2024-04-25T10:21:55.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198677 s, 52.8 MB/s 2024-04-25T10:21:55.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.736 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206271 s, 50.8 MB/s 2024-04-25T10:21:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162598 s, 64.5 MB/s 2024-04-25T10:21:55.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.771 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150741 s, 69.6 MB/s 2024-04-25T10:21:55.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.790 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.790 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170968 s, 61.3 MB/s 2024-04-25T10:21:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.807 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149478 s, 70.1 MB/s 2024-04-25T10:21:55.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.822 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129712 s, 80.8 MB/s 2024-04-25T10:21:55.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020836 s, 50.3 MB/s 2024-04-25T10:21:55.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.861 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148617 s, 70.6 MB/s 2024-04-25T10:21:55.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.880 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174421 s, 60.1 MB/s 2024-04-25T10:21:55.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.897 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150442 s, 69.7 MB/s 2024-04-25T10:21:55.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.913 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148174 s, 70.8 MB/s 2024-04-25T10:21:55.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.932 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164398 s, 63.8 MB/s 2024-04-25T10:21:55.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.948 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147584 s, 71.0 MB/s 2024-04-25T10:21:55.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.965 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147344 s, 71.2 MB/s 2024-04-25T10:21:55.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.981 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142241 s, 73.7 MB/s 2024-04-25T10:21:55.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:55.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:55.997 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144596 s, 72.5 MB/s 2024-04-25T10:21:56.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016225 s, 64.6 MB/s 2024-04-25T10:21:56.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.037 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205434 s, 51.0 MB/s 2024-04-25T10:21:56.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.056 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175951 s, 59.6 MB/s 2024-04-25T10:21:56.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206061 s, 50.9 MB/s 2024-04-25T10:21:56.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.104 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236706 s, 44.3 MB/s 2024-04-25T10:21:56.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148823 s, 70.5 MB/s 2024-04-25T10:21:56.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:56.122 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:56.122 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:56.122 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:21:56.122 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:56.123 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:56.123 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:56.123 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:56.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.196 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.196 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154738 s, 67.8 MB/s 2024-04-25T10:21:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162065 s, 64.7 MB/s 2024-04-25T10:21:56.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.232 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.232 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158065 s, 66.3 MB/s 2024-04-25T10:21:56.249 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.250 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157708 s, 66.5 MB/s 2024-04-25T10:21:56.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.268 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161041 s, 65.1 MB/s 2024-04-25T10:21:56.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.284 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149175 s, 70.3 MB/s 2024-04-25T10:21:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146867 s, 71.4 MB/s 2024-04-25T10:21:56.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.317 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146049 s, 71.8 MB/s 2024-04-25T10:21:56.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.333 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141832 s, 73.9 MB/s 2024-04-25T10:21:56.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.350 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151384 s, 69.3 MB/s 2024-04-25T10:21:56.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.366 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014552 s, 72.1 MB/s 2024-04-25T10:21:56.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.387 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185397 s, 56.6 MB/s 2024-04-25T10:21:56.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.407 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181395 s, 57.8 MB/s 2024-04-25T10:21:56.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185639 s, 56.5 MB/s 2024-04-25T10:21:56.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.442 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130048 s, 80.6 MB/s 2024-04-25T10:21:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162265 s, 64.6 MB/s 2024-04-25T10:21:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195072 s, 53.8 MB/s 2024-04-25T10:21:56.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.501 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.502 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185781 s, 56.4 MB/s 2024-04-25T10:21:56.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.521 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017506 s, 59.9 MB/s 2024-04-25T10:21:56.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.537 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.537 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141131 s, 74.3 MB/s 2024-04-25T10:21:56.553 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.553 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150734 s, 69.6 MB/s 2024-04-25T10:21:56.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140728 s, 74.5 MB/s 2024-04-25T10:21:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140544 s, 74.6 MB/s 2024-04-25T10:21:56.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.603 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147714 s, 71.0 MB/s 2024-04-25T10:21:56.622 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.622 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.622 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181742 s, 57.7 MB/s 2024-04-25T10:21:56.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.642 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177884 s, 58.9 MB/s 2024-04-25T10:21:56.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.658 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146535 s, 71.6 MB/s 2024-04-25T10:21:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174375 s, 60.1 MB/s 2024-04-25T10:21:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158151 s, 66.3 MB/s 2024-04-25T10:21:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146596 s, 71.5 MB/s 2024-04-25T10:21:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.729 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151573 s, 69.2 MB/s 2024-04-25T10:21:56.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.752 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213727 s, 49.1 MB/s 2024-04-25T10:21:56.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:56.753 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:56.753 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:56.753 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:21:56.753 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:56.754 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:56.754 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:56.754 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:56.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145556 s, 72.0 MB/s 2024-04-25T10:21:56.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145054 s, 72.3 MB/s 2024-04-25T10:21:56.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.861 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151832 s, 69.1 MB/s 2024-04-25T10:21:56.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.879 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166595 s, 62.9 MB/s 2024-04-25T10:21:56.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.899 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176672 s, 59.4 MB/s 2024-04-25T10:21:56.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.915 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015276 s, 68.6 MB/s 2024-04-25T10:21:56.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171737 s, 61.1 MB/s 2024-04-25T10:21:56.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.956 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.956 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019727 s, 53.2 MB/s 2024-04-25T10:21:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149244 s, 70.3 MB/s 2024-04-25T10:21:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150417 s, 69.7 MB/s 2024-04-25T10:21:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147889 s, 70.9 MB/s 2024-04-25T10:21:57.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.023 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014735 s, 71.2 MB/s 2024-04-25T10:21:57.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.044 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.044 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198512 s, 52.8 MB/s 2024-04-25T10:21:57.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.063 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172355 s, 60.8 MB/s 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149381 s, 70.2 MB/s 2024-04-25T10:21:57.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.101 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.101 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194037 s, 54.0 MB/s 2024-04-25T10:21:57.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.121 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018798 s, 55.8 MB/s 2024-04-25T10:21:57.140 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.140 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.140 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173838 s, 60.3 MB/s 2024-04-25T10:21:57.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205501 s, 51.0 MB/s 2024-04-25T10:21:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.183 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189622 s, 55.3 MB/s 2024-04-25T10:21:57.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.205 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019674 s, 53.3 MB/s 2024-04-25T10:21:57.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195907 s, 53.5 MB/s 2024-04-25T10:21:57.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.246 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018429 s, 56.9 MB/s 2024-04-25T10:21:57.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.269 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206554 s, 50.8 MB/s 2024-04-25T10:21:57.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.285 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147933 s, 70.9 MB/s 2024-04-25T10:21:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151483 s, 69.2 MB/s 2024-04-25T10:21:57.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.321 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169164 s, 62.0 MB/s 2024-04-25T10:21:57.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.342 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.342 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191154 s, 54.9 MB/s 2024-04-25T10:21:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.359 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149847 s, 70.0 MB/s 2024-04-25T10:21:57.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.381 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02038 s, 51.5 MB/s 2024-04-25T10:21:57.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.400 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175766 s, 59.7 MB/s 2024-04-25T10:21:57.425 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.425 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.425 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233931 s, 44.8 MB/s 2024-04-25T10:21:57.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:57.427 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:57.427 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:57.427 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:21:57.427 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:57.427 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:57.427 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:57.427 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:57.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169542 s, 61.8 MB/s 2024-04-25T10:21:57.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215763 s, 48.6 MB/s 2024-04-25T10:21:57.547 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.547 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.548 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169911 s, 61.7 MB/s 2024-04-25T10:21:57.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163609 s, 64.1 MB/s 2024-04-25T10:21:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.582 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145497 s, 72.1 MB/s 2024-04-25T10:21:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144417 s, 72.6 MB/s 2024-04-25T10:21:57.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.616 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156885 s, 66.8 MB/s 2024-04-25T10:21:57.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.634 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161396 s, 65.0 MB/s 2024-04-25T10:21:57.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.655 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.655 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199518 s, 52.6 MB/s 2024-04-25T10:21:57.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.671 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140179 s, 74.8 MB/s 2024-04-25T10:21:57.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.688 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152223 s, 68.9 MB/s 2024-04-25T10:21:57.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.704 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146502 s, 71.6 MB/s 2024-04-25T10:21:57.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.720 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140392 s, 74.7 MB/s 2024-04-25T10:21:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150313 s, 69.8 MB/s 2024-04-25T10:21:57.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.753 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146103 s, 71.8 MB/s 2024-04-25T10:21:57.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.770 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150086 s, 69.9 MB/s 2024-04-25T10:21:57.786 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.786 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.786 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142831 s, 73.4 MB/s 2024-04-25T10:21:57.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170326 s, 61.6 MB/s 2024-04-25T10:21:57.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.824 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176139 s, 59.5 MB/s 2024-04-25T10:21:57.840 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.841 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147049 s, 71.3 MB/s 2024-04-25T10:21:57.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.863 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205026 s, 51.1 MB/s 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140192 s, 74.8 MB/s 2024-04-25T10:21:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164647 s, 63.7 MB/s 2024-04-25T10:21:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162968 s, 64.3 MB/s 2024-04-25T10:21:57.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.932 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158583 s, 66.1 MB/s 2024-04-25T10:21:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152618 s, 68.7 MB/s 2024-04-25T10:21:57.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216668 s, 48.4 MB/s 2024-04-25T10:21:57.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:57.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:57.989 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146444 s, 71.6 MB/s 2024-04-25T10:21:58.012 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.012 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.012 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021446 s, 48.9 MB/s 2024-04-25T10:21:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150995 s, 69.4 MB/s 2024-04-25T10:21:58.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.047 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.047 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164691 s, 63.7 MB/s 2024-04-25T10:21:58.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.064 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149397 s, 70.2 MB/s 2024-04-25T10:21:58.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.065 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:58.065 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:58.066 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:21:58.066 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:58.066 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:58.066 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:58.066 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.136 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153928 s, 68.1 MB/s 2024-04-25T10:21:58.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.154 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160978 s, 65.1 MB/s 2024-04-25T10:21:58.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.171 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.171 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149664 s, 70.1 MB/s 2024-04-25T10:21:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160888 s, 65.2 MB/s 2024-04-25T10:21:58.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.207 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160369 s, 65.4 MB/s 2024-04-25T10:21:58.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018208 s, 57.6 MB/s 2024-04-25T10:21:58.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.243 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146314 s, 71.7 MB/s 2024-04-25T10:21:58.262 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.262 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.262 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177322 s, 59.1 MB/s 2024-04-25T10:21:58.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.281 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170494 s, 61.5 MB/s 2024-04-25T10:21:58.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.303 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020607 s, 50.9 MB/s 2024-04-25T10:21:58.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172644 s, 60.7 MB/s 2024-04-25T10:21:58.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162657 s, 64.5 MB/s 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221136 s, 47.4 MB/s 2024-04-25T10:21:58.382 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.382 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.382 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151536 s, 69.2 MB/s 2024-04-25T10:21:58.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.401 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016802 s, 62.4 MB/s 2024-04-25T10:21:58.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.423 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202842 s, 51.7 MB/s 2024-04-25T10:21:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160087 s, 65.5 MB/s 2024-04-25T10:21:58.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.459 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166312 s, 63.0 MB/s 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209558 s, 50.0 MB/s 2024-04-25T10:21:58.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.498 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146999 s, 71.3 MB/s 2024-04-25T10:21:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204279 s, 51.3 MB/s 2024-04-25T10:21:58.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.539 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166546 s, 63.0 MB/s 2024-04-25T10:21:58.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.558 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176952 s, 59.3 MB/s 2024-04-25T10:21:58.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.577 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176081 s, 59.6 MB/s 2024-04-25T10:21:58.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.595 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165084 s, 63.5 MB/s 2024-04-25T10:21:58.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.618 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211887 s, 49.5 MB/s 2024-04-25T10:21:58.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014809 s, 70.8 MB/s 2024-04-25T10:21:58.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.657 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205941 s, 50.9 MB/s 2024-04-25T10:21:58.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.676 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170217 s, 61.6 MB/s 2024-04-25T10:21:58.693 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.693 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.693 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152648 s, 68.7 MB/s 2024-04-25T10:21:58.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.709 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014419 s, 72.7 MB/s 2024-04-25T10:21:58.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.732 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208835 s, 50.2 MB/s 2024-04-25T10:21:58.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.733 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:58.734 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:58.734 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:21:58.734 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:58.734 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:58.734 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:58.734 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:58.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.803 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143965 s, 72.8 MB/s 2024-04-25T10:21:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011954 s, 87.7 MB/s 2024-04-25T10:21:58.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.835 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.835 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170826 s, 61.4 MB/s 2024-04-25T10:21:58.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.852 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151312 s, 69.3 MB/s 2024-04-25T10:21:58.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.870 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164179 s, 63.9 MB/s 2024-04-25T10:21:58.886 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.886 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.886 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144694 s, 72.5 MB/s 2024-04-25T10:21:58.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.905 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.905 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166874 s, 62.8 MB/s 2024-04-25T10:21:58.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.922 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155904 s, 67.3 MB/s 2024-04-25T10:21:58.942 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.943 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.943 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186235 s, 56.3 MB/s 2024-04-25T10:21:58.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.961 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168924 s, 62.1 MB/s 2024-04-25T10:21:58.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.980 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166084 s, 63.1 MB/s 2024-04-25T10:21:58.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:58.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:58.997 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155698 s, 67.3 MB/s 2024-04-25T10:21:59.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.014 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152634 s, 68.7 MB/s 2024-04-25T10:21:59.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.035 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198046 s, 52.9 MB/s 2024-04-25T10:21:59.051 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.052 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.052 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146078 s, 71.8 MB/s 2024-04-25T10:21:59.068 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.068 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.068 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147796 s, 70.9 MB/s 2024-04-25T10:21:59.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.092 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218081 s, 48.1 MB/s 2024-04-25T10:21:59.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.109 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152788 s, 68.6 MB/s 2024-04-25T10:21:59.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.125 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.125 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142872 s, 73.4 MB/s 2024-04-25T10:21:59.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.143 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168248 s, 62.3 MB/s 2024-04-25T10:21:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165747 s, 63.3 MB/s 2024-04-25T10:21:59.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164215 s, 63.9 MB/s 2024-04-25T10:21:59.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169323 s, 61.9 MB/s 2024-04-25T10:21:59.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.217 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158902 s, 66.0 MB/s 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141789 s, 74.0 MB/s 2024-04-25T10:21:59.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.250 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153209 s, 68.4 MB/s 2024-04-25T10:21:59.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.270 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182831 s, 57.4 MB/s 2024-04-25T10:21:59.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.286 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014138 s, 74.2 MB/s 2024-04-25T10:21:59.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.303 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150791 s, 69.5 MB/s 2024-04-25T10:21:59.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178238 s, 58.8 MB/s 2024-04-25T10:21:59.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.338 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140913 s, 74.4 MB/s 2024-04-25T10:21:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158829 s, 66.0 MB/s 2024-04-25T10:21:59.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:59.357 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:21:59.358 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:21:59.358 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:21:59.358 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:21:59.358 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:21:59.358 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:21:59.358 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:21:59.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.428 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153072 s, 68.5 MB/s 2024-04-25T10:21:59.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.448 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188747 s, 55.6 MB/s 2024-04-25T10:21:59.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.469 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187904 s, 55.8 MB/s 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141847 s, 73.9 MB/s 2024-04-25T10:21:59.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.502 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154797 s, 67.7 MB/s 2024-04-25T10:21:59.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207103 s, 50.6 MB/s 2024-04-25T10:21:59.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.541 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144912 s, 72.4 MB/s 2024-04-25T10:21:59.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.561 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188376 s, 55.7 MB/s 2024-04-25T10:21:59.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.583 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195831 s, 53.5 MB/s 2024-04-25T10:21:59.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.603 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018164 s, 57.7 MB/s 2024-04-25T10:21:59.620 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.620 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.620 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157227 s, 66.7 MB/s 2024-04-25T10:21:59.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.645 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233911 s, 44.8 MB/s 2024-04-25T10:21:59.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.667 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195916 s, 53.5 MB/s 2024-04-25T10:21:59.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.682 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.683 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143364 s, 73.1 MB/s 2024-04-25T10:21:59.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.704 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198481 s, 52.8 MB/s 2024-04-25T10:21:59.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.727 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206315 s, 50.8 MB/s 2024-04-25T10:21:59.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.744 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.744 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164043 s, 63.9 MB/s 2024-04-25T10:21:59.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149941 s, 69.9 MB/s 2024-04-25T10:21:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163841 s, 64.0 MB/s 2024-04-25T10:21:59.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.795 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142838 s, 73.4 MB/s 2024-04-25T10:21:59.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.815 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179976 s, 58.3 MB/s 2024-04-25T10:21:59.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.831 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148072 s, 70.8 MB/s 2024-04-25T10:21:59.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.849 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153995 s, 68.1 MB/s 2024-04-25T10:21:59.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.869 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186942 s, 56.1 MB/s 2024-04-25T10:21:59.885 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.885 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.885 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140511 s, 74.6 MB/s 2024-04-25T10:21:59.901 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.901 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.901 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143686 s, 73.0 MB/s 2024-04-25T10:21:59.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.917 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014503 s, 72.3 MB/s 2024-04-25T10:21:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214055 s, 49.0 MB/s 2024-04-25T10:21:59.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.963 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170518 s, 61.5 MB/s 2024-04-25T10:21:59.976 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.976 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150443 s, 69.7 MB/s 2024-04-25T10:21:59.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:21:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:21:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144057 s, 72.8 MB/s 2024-04-25T10:22:00.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.010 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151789 s, 69.1 MB/s 2024-04-25T10:22:00.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:00.011 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:00.011 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:00.011 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:22:00.011 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:00.011 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:00.011 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:00.011 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:00.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.095 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242628 s, 43.2 MB/s 2024-04-25T10:22:00.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.113 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172741 s, 60.7 MB/s 2024-04-25T10:22:00.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167375 s, 62.6 MB/s 2024-04-25T10:22:00.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.147 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141338 s, 74.2 MB/s 2024-04-25T10:22:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144653 s, 72.5 MB/s 2024-04-25T10:22:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.181 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156295 s, 67.1 MB/s 2024-04-25T10:22:00.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.198 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154893 s, 67.7 MB/s 2024-04-25T10:22:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015528 s, 67.5 MB/s 2024-04-25T10:22:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140315 s, 74.7 MB/s 2024-04-25T10:22:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145269 s, 72.2 MB/s 2024-04-25T10:22:00.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.275 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.275 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254052 s, 41.3 MB/s 2024-04-25T10:22:00.290 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.291 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141614 s, 74.0 MB/s 2024-04-25T10:22:00.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143285 s, 73.2 MB/s 2024-04-25T10:22:00.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.323 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147784 s, 71.0 MB/s 2024-04-25T10:22:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.339 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140638 s, 74.6 MB/s 2024-04-25T10:22:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.357 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155371 s, 67.5 MB/s 2024-04-25T10:22:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225493 s, 46.5 MB/s 2024-04-25T10:22:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146484 s, 71.6 MB/s 2024-04-25T10:22:00.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.417 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.417 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175778 s, 59.7 MB/s 2024-04-25T10:22:00.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.440 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207694 s, 50.5 MB/s 2024-04-25T10:22:00.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.457 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163171 s, 64.3 MB/s 2024-04-25T10:22:00.476 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176062 s, 59.6 MB/s 2024-04-25T10:22:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140769 s, 74.5 MB/s 2024-04-25T10:22:00.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.514 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.515 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173311 s, 60.5 MB/s 2024-04-25T10:22:00.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155636 s, 67.4 MB/s 2024-04-25T10:22:00.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.551 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.551 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197387 s, 53.1 MB/s 2024-04-25T10:22:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203295 s, 51.6 MB/s 2024-04-25T10:22:00.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.595 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201121 s, 52.1 MB/s 2024-04-25T10:22:00.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.611 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.611 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148557 s, 70.6 MB/s 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014667 s, 71.5 MB/s 2024-04-25T10:22:00.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014108 s, 74.3 MB/s 2024-04-25T10:22:00.660 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.660 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.660 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146788 s, 71.4 MB/s 2024-04-25T10:22:00.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:00.662 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:00.662 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:00.662 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:22:00.662 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:00.662 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:00.662 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:00.662 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:00.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.730 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140587 s, 74.6 MB/s 2024-04-25T10:22:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.753 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209351 s, 50.1 MB/s 2024-04-25T10:22:00.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.769 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149281 s, 70.2 MB/s 2024-04-25T10:22:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014136 s, 74.2 MB/s 2024-04-25T10:22:00.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.803 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153443 s, 68.3 MB/s 2024-04-25T10:22:00.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.825 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208544 s, 50.3 MB/s 2024-04-25T10:22:00.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174343 s, 60.1 MB/s 2024-04-25T10:22:00.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.863 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167837 s, 62.5 MB/s 2024-04-25T10:22:00.878 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.879 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014055 s, 74.6 MB/s 2024-04-25T10:22:00.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171027 s, 61.3 MB/s 2024-04-25T10:22:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191887 s, 54.6 MB/s 2024-04-25T10:22:00.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.935 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147848 s, 70.9 MB/s 2024-04-25T10:22:00.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.953 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164839 s, 63.6 MB/s 2024-04-25T10:22:00.971 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.972 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.972 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166639 s, 62.9 MB/s 2024-04-25T10:22:00.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:00.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:00.987 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141226 s, 74.2 MB/s 2024-04-25T10:22:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.004 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151359 s, 69.3 MB/s 2024-04-25T10:22:01.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.023 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174985 s, 59.9 MB/s 2024-04-25T10:22:01.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141173 s, 74.3 MB/s 2024-04-25T10:22:01.061 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.061 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.061 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192667 s, 54.4 MB/s 2024-04-25T10:22:01.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166577 s, 62.9 MB/s 2024-04-25T10:22:01.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.098 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170399 s, 61.5 MB/s 2024-04-25T10:22:01.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194532 s, 53.9 MB/s 2024-04-25T10:22:01.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185209 s, 56.6 MB/s 2024-04-25T10:22:01.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166814 s, 62.9 MB/s 2024-04-25T10:22:01.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.176 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161659 s, 64.9 MB/s 2024-04-25T10:22:01.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155139 s, 67.6 MB/s 2024-04-25T10:22:01.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.211 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165976 s, 63.2 MB/s 2024-04-25T10:22:01.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159166 s, 65.9 MB/s 2024-04-25T10:22:01.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.246 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015647 s, 67.0 MB/s 2024-04-25T10:22:01.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.263 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151296 s, 69.3 MB/s 2024-04-25T10:22:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014887 s, 70.4 MB/s 2024-04-25T10:22:01.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.302 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200276 s, 52.4 MB/s 2024-04-25T10:22:01.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:01.303 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:01.303 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:01.303 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:22:01.303 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:01.303 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:01.303 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:01.304 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:01.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201661 s, 52.0 MB/s 2024-04-25T10:22:01.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.397 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150445 s, 69.7 MB/s 2024-04-25T10:22:01.415 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.416 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.416 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168082 s, 62.4 MB/s 2024-04-25T10:22:01.432 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.432 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.432 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149331 s, 70.2 MB/s 2024-04-25T10:22:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.456 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221342 s, 47.4 MB/s 2024-04-25T10:22:01.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158609 s, 66.1 MB/s 2024-04-25T10:22:01.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148672 s, 70.5 MB/s 2024-04-25T10:22:01.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150584 s, 69.6 MB/s 2024-04-25T10:22:01.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144961 s, 72.3 MB/s 2024-04-25T10:22:01.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179388 s, 58.5 MB/s 2024-04-25T10:22:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152758 s, 68.6 MB/s 2024-04-25T10:22:01.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.579 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169176 s, 62.0 MB/s 2024-04-25T10:22:01.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.601 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200745 s, 52.2 MB/s 2024-04-25T10:22:01.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.619 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163168 s, 64.3 MB/s 2024-04-25T10:22:01.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147972 s, 70.9 MB/s 2024-04-25T10:22:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01702 s, 61.6 MB/s 2024-04-25T10:22:01.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.670 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144034 s, 72.8 MB/s 2024-04-25T10:22:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.687 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.687 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143523 s, 73.1 MB/s 2024-04-25T10:22:01.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.700 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124297 s, 84.4 MB/s 2024-04-25T10:22:01.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.720 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.720 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176569 s, 59.4 MB/s 2024-04-25T10:22:01.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.737 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150189 s, 69.8 MB/s 2024-04-25T10:22:01.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.755 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.755 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166952 s, 62.8 MB/s 2024-04-25T10:22:01.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.772 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148544 s, 70.6 MB/s 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207263 s, 50.6 MB/s 2024-04-25T10:22:01.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.811 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149426 s, 70.2 MB/s 2024-04-25T10:22:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148173 s, 70.8 MB/s 2024-04-25T10:22:01.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015806 s, 66.3 MB/s 2024-04-25T10:22:01.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01512 s, 69.4 MB/s 2024-04-25T10:22:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150131 s, 69.8 MB/s 2024-04-25T10:22:01.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.895 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147166 s, 71.3 MB/s 2024-04-25T10:22:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155597 s, 67.4 MB/s 2024-04-25T10:22:01.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:01.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:01.933 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182638 s, 57.4 MB/s 2024-04-25T10:22:01.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:01.934 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:01.934 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:01.934 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:22:01.934 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:01.934 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:01.934 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:01.935 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:02.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.004 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.004 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148856 s, 70.4 MB/s 2024-04-25T10:22:02.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.024 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172713 s, 60.7 MB/s 2024-04-25T10:22:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144523 s, 72.6 MB/s 2024-04-25T10:22:02.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144324 s, 72.7 MB/s 2024-04-25T10:22:02.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167545 s, 62.6 MB/s 2024-04-25T10:22:02.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.091 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.091 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149575 s, 70.1 MB/s 2024-04-25T10:22:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.107 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014949 s, 70.1 MB/s 2024-04-25T10:22:02.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.130 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202277 s, 51.8 MB/s 2024-04-25T10:22:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184259 s, 56.9 MB/s 2024-04-25T10:22:02.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.166 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147931 s, 70.9 MB/s 2024-04-25T10:22:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143682 s, 73.0 MB/s 2024-04-25T10:22:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159712 s, 65.7 MB/s 2024-04-25T10:22:02.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.223 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167894 s, 62.5 MB/s 2024-04-25T10:22:02.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.239 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186475 s, 56.2 MB/s 2024-04-25T10:22:02.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.259 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185091 s, 56.7 MB/s 2024-04-25T10:22:02.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.283 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021583 s, 48.6 MB/s 2024-04-25T10:22:02.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.297 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125921 s, 83.3 MB/s 2024-04-25T10:22:02.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.314 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.314 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149035 s, 70.4 MB/s 2024-04-25T10:22:02.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.331 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158481 s, 66.2 MB/s 2024-04-25T10:22:02.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.353 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199623 s, 52.5 MB/s 2024-04-25T10:22:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.374 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187553 s, 55.9 MB/s 2024-04-25T10:22:02.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.390 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.390 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144373 s, 72.6 MB/s 2024-04-25T10:22:02.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144865 s, 72.4 MB/s 2024-04-25T10:22:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160728 s, 65.2 MB/s 2024-04-25T10:22:02.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.440 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142426 s, 73.6 MB/s 2024-04-25T10:22:02.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.458 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158901 s, 66.0 MB/s 2024-04-25T10:22:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145187 s, 72.2 MB/s 2024-04-25T10:22:02.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.493 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173147 s, 60.6 MB/s 2024-04-25T10:22:02.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.510 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143675 s, 73.0 MB/s 2024-04-25T10:22:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149678 s, 70.1 MB/s 2024-04-25T10:22:02.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146992 s, 71.3 MB/s 2024-04-25T10:22:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143808 s, 72.9 MB/s 2024-04-25T10:22:02.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:02.561 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:02.561 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:02.561 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:22:02.561 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:02.561 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:02.561 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:02.561 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:02.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.641 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.641 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235678 s, 44.5 MB/s 2024-04-25T10:22:02.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.657 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014484 s, 72.4 MB/s 2024-04-25T10:22:02.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.674 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014772 s, 71.0 MB/s 2024-04-25T10:22:02.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147429 s, 71.1 MB/s 2024-04-25T10:22:02.706 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.707 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142502 s, 73.6 MB/s 2024-04-25T10:22:02.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.724 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150954 s, 69.5 MB/s 2024-04-25T10:22:02.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.739 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.739 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136306 s, 76.9 MB/s 2024-04-25T10:22:02.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.757 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164947 s, 63.6 MB/s 2024-04-25T10:22:02.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.773 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146386 s, 71.6 MB/s 2024-04-25T10:22:02.791 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.792 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016497 s, 63.6 MB/s 2024-04-25T10:22:02.810 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.810 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.810 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165915 s, 63.2 MB/s 2024-04-25T10:22:02.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.824 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122592 s, 85.5 MB/s 2024-04-25T10:22:02.840 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.840 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.840 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148526 s, 70.6 MB/s 2024-04-25T10:22:02.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.862 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020237 s, 51.8 MB/s 2024-04-25T10:22:02.883 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.883 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.883 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187651 s, 55.9 MB/s 2024-04-25T10:22:02.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151005 s, 69.4 MB/s 2024-04-25T10:22:02.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.916 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149138 s, 70.3 MB/s 2024-04-25T10:22:02.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.933 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150994 s, 69.4 MB/s 2024-04-25T10:22:02.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.949 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.949 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141552 s, 74.1 MB/s 2024-04-25T10:22:02.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.969 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177843 s, 59.0 MB/s 2024-04-25T10:22:02.983 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:02.983 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:02.983 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129285 s, 81.1 MB/s 2024-04-25T10:22:03.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.000 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.000 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148831 s, 70.5 MB/s 2024-04-25T10:22:03.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.021 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196318 s, 53.4 MB/s 2024-04-25T10:22:03.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.038 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014615 s, 71.7 MB/s 2024-04-25T10:22:03.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015497 s, 67.7 MB/s 2024-04-25T10:22:03.076 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.076 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.076 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190231 s, 55.1 MB/s 2024-04-25T10:22:03.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.093 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150755 s, 69.6 MB/s 2024-04-25T10:22:03.111 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.111 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.111 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166325 s, 63.0 MB/s 2024-04-25T10:22:03.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.130 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168983 s, 62.1 MB/s 2024-04-25T10:22:03.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.149 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173928 s, 60.3 MB/s 2024-04-25T10:22:03.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.169 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185267 s, 56.6 MB/s 2024-04-25T10:22:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150127 s, 69.8 MB/s 2024-04-25T10:22:03.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:03.188 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:03.188 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:03.188 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:22:03.188 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:03.188 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:03.188 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:03.188 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:03.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.259 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144846 s, 72.4 MB/s 2024-04-25T10:22:03.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.277 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160837 s, 65.2 MB/s 2024-04-25T10:22:03.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148124 s, 70.8 MB/s 2024-04-25T10:22:03.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.313 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.313 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174393 s, 60.1 MB/s 2024-04-25T10:22:03.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143527 s, 73.1 MB/s 2024-04-25T10:22:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190293 s, 55.1 MB/s 2024-04-25T10:22:03.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.367 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149318 s, 70.2 MB/s 2024-04-25T10:22:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179384 s, 58.5 MB/s 2024-04-25T10:22:03.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014511 s, 72.3 MB/s 2024-04-25T10:22:03.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.422 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.422 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158226 s, 66.3 MB/s 2024-04-25T10:22:03.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.438 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014925 s, 70.3 MB/s 2024-04-25T10:22:03.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.457 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176767 s, 59.3 MB/s 2024-04-25T10:22:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.473 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145723 s, 72.0 MB/s 2024-04-25T10:22:03.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.490 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143258 s, 73.2 MB/s 2024-04-25T10:22:03.505 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.506 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141788 s, 74.0 MB/s 2024-04-25T10:22:03.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.521 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141177 s, 74.3 MB/s 2024-04-25T10:22:03.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.538 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147334 s, 71.2 MB/s 2024-04-25T10:22:03.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145482 s, 72.1 MB/s 2024-04-25T10:22:03.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.570 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143976 s, 72.8 MB/s 2024-04-25T10:22:03.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147104 s, 71.3 MB/s 2024-04-25T10:22:03.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.603 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144093 s, 72.8 MB/s 2024-04-25T10:22:03.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.621 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161294 s, 65.0 MB/s 2024-04-25T10:22:03.637 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.638 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151345 s, 69.3 MB/s 2024-04-25T10:22:03.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.653 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.653 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139778 s, 75.0 MB/s 2024-04-25T10:22:03.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.673 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178378 s, 58.8 MB/s 2024-04-25T10:22:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.690 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151115 s, 69.4 MB/s 2024-04-25T10:22:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180292 s, 58.2 MB/s 2024-04-25T10:22:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205337 s, 51.1 MB/s 2024-04-25T10:22:03.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.746 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119908 s, 87.4 MB/s 2024-04-25T10:22:03.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.763 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148901 s, 70.4 MB/s 2024-04-25T10:22:03.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.779 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.779 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014295 s, 73.4 MB/s 2024-04-25T10:22:03.797 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.797 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.797 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167146 s, 62.7 MB/s 2024-04-25T10:22:03.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:03.800 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:03.800 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:03.800 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:22:03.800 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:03.800 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:03.800 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:03.801 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:03.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.873 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.873 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159464 s, 65.8 MB/s 2024-04-25T10:22:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149373 s, 70.2 MB/s 2024-04-25T10:22:03.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.910 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.910 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186438 s, 56.2 MB/s 2024-04-25T10:22:03.927 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.927 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.927 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155186 s, 67.6 MB/s 2024-04-25T10:22:03.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.952 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226154 s, 46.4 MB/s 2024-04-25T10:22:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208357 s, 50.3 MB/s 2024-04-25T10:22:03.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:03.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:03.993 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173314 s, 60.5 MB/s 2024-04-25T10:22:04.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.010 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149155 s, 70.3 MB/s 2024-04-25T10:22:04.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.027 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152676 s, 68.7 MB/s 2024-04-25T10:22:04.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.043 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147671 s, 71.0 MB/s 2024-04-25T10:22:04.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.065 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192432 s, 54.5 MB/s 2024-04-25T10:22:04.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.082 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015494 s, 67.7 MB/s 2024-04-25T10:22:04.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.102 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189025 s, 55.5 MB/s 2024-04-25T10:22:04.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.119 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.119 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151925 s, 69.0 MB/s 2024-04-25T10:22:04.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.141 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202598 s, 51.8 MB/s 2024-04-25T10:22:04.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153108 s, 68.5 MB/s 2024-04-25T10:22:04.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.175 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150072 s, 69.9 MB/s 2024-04-25T10:22:04.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157615 s, 66.5 MB/s 2024-04-25T10:22:04.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147732 s, 71.0 MB/s 2024-04-25T10:22:04.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148585 s, 70.6 MB/s 2024-04-25T10:22:04.242 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.242 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.243 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147672 s, 71.0 MB/s 2024-04-25T10:22:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.264 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196277 s, 53.4 MB/s 2024-04-25T10:22:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154827 s, 67.7 MB/s 2024-04-25T10:22:04.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.304 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215474 s, 48.7 MB/s 2024-04-25T10:22:04.323 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.324 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173678 s, 60.4 MB/s 2024-04-25T10:22:04.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147719 s, 71.0 MB/s 2024-04-25T10:22:04.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157755 s, 66.5 MB/s 2024-04-25T10:22:04.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.374 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145522 s, 72.1 MB/s 2024-04-25T10:22:04.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.392 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162429 s, 64.6 MB/s 2024-04-25T10:22:04.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.411 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016928 s, 61.9 MB/s 2024-04-25T10:22:04.431 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.431 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.431 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182637 s, 57.4 MB/s 2024-04-25T10:22:04.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.450 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.450 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168246 s, 62.3 MB/s 2024-04-25T10:22:04.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:04.451 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:04.451 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:04.451 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:22:04.451 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:04.451 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:04.451 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:04.451 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016943 s, 61.9 MB/s 2024-04-25T10:22:04.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.550 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.550 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187085 s, 56.0 MB/s 2024-04-25T10:22:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014672 s, 71.5 MB/s 2024-04-25T10:22:04.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.583 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146772 s, 71.4 MB/s 2024-04-25T10:22:04.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.602 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170381 s, 61.5 MB/s 2024-04-25T10:22:04.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.618 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144759 s, 72.4 MB/s 2024-04-25T10:22:04.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158135 s, 66.3 MB/s 2024-04-25T10:22:04.656 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.656 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.656 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185305 s, 56.6 MB/s 2024-04-25T10:22:04.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.680 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022114 s, 47.4 MB/s 2024-04-25T10:22:04.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237252 s, 44.2 MB/s 2024-04-25T10:22:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159716 s, 65.7 MB/s 2024-04-25T10:22:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189673 s, 55.3 MB/s 2024-04-25T10:22:04.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.765 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019605 s, 53.5 MB/s 2024-04-25T10:22:04.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.781 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142078 s, 73.8 MB/s 2024-04-25T10:22:04.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168478 s, 62.2 MB/s 2024-04-25T10:22:04.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.815 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144091 s, 72.8 MB/s 2024-04-25T10:22:04.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170324 s, 61.6 MB/s 2024-04-25T10:22:04.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017975 s, 58.3 MB/s 2024-04-25T10:22:04.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.871 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.871 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014779 s, 71.0 MB/s 2024-04-25T10:22:04.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.889 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.889 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166944 s, 62.8 MB/s 2024-04-25T10:22:04.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.911 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204562 s, 51.3 MB/s 2024-04-25T10:22:04.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.928 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.928 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150861 s, 69.5 MB/s 2024-04-25T10:22:04.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.946 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164649 s, 63.7 MB/s 2024-04-25T10:22:04.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.963 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151472 s, 69.2 MB/s 2024-04-25T10:22:04.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.981 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.981 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165212 s, 63.5 MB/s 2024-04-25T10:22:04.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:04.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:04.997 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014378 s, 72.9 MB/s 2024-04-25T10:22:05.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.016 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164357 s, 63.8 MB/s 2024-04-25T10:22:05.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.031 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138398 s, 75.8 MB/s 2024-04-25T10:22:05.051 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.051 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.051 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178938 s, 58.6 MB/s 2024-04-25T10:22:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166366 s, 63.0 MB/s 2024-04-25T10:22:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151657 s, 69.1 MB/s 2024-04-25T10:22:05.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.103 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143506 s, 73.1 MB/s 2024-04-25T10:22:05.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:05.104 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:05.104 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:05.104 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:22:05.104 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:05.104 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:05.104 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:05.104 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:05.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.176 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154874 s, 67.7 MB/s 2024-04-25T10:22:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015959 s, 65.7 MB/s 2024-04-25T10:22:05.216 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.216 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.216 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204842 s, 51.2 MB/s 2024-04-25T10:22:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.233 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154117 s, 68.0 MB/s 2024-04-25T10:22:05.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.250 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153752 s, 68.2 MB/s 2024-04-25T10:22:05.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.268 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.268 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161456 s, 64.9 MB/s 2024-04-25T10:22:05.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.289 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.289 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018734 s, 56.0 MB/s 2024-04-25T10:22:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.305 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146701 s, 71.5 MB/s 2024-04-25T10:22:05.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.325 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178809 s, 58.6 MB/s 2024-04-25T10:22:05.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.341 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.341 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145907 s, 71.9 MB/s 2024-04-25T10:22:05.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166032 s, 63.2 MB/s 2024-04-25T10:22:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144744 s, 72.4 MB/s 2024-04-25T10:22:05.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180632 s, 58.1 MB/s 2024-04-25T10:22:05.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.412 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145893 s, 71.9 MB/s 2024-04-25T10:22:05.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.428 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148046 s, 70.8 MB/s 2024-04-25T10:22:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148301 s, 70.7 MB/s 2024-04-25T10:22:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143247 s, 73.2 MB/s 2024-04-25T10:22:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150584 s, 69.6 MB/s 2024-04-25T10:22:05.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.494 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144263 s, 72.7 MB/s 2024-04-25T10:22:05.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.511 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.511 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015124 s, 69.3 MB/s 2024-04-25T10:22:05.534 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.534 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.534 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206848 s, 50.7 MB/s 2024-04-25T10:22:05.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.553 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017266 s, 60.7 MB/s 2024-04-25T10:22:05.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.573 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183646 s, 57.1 MB/s 2024-04-25T10:22:05.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.595 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208808 s, 50.2 MB/s 2024-04-25T10:22:05.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166425 s, 63.0 MB/s 2024-04-25T10:22:05.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.630 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148405 s, 70.7 MB/s 2024-04-25T10:22:05.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.648 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164729 s, 63.7 MB/s 2024-04-25T10:22:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170697 s, 61.4 MB/s 2024-04-25T10:22:05.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.685 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160464 s, 65.3 MB/s 2024-04-25T10:22:05.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.707 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210083 s, 49.9 MB/s 2024-04-25T10:22:05.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.724 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140773 s, 74.5 MB/s 2024-04-25T10:22:05.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.740 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146407 s, 71.6 MB/s 2024-04-25T10:22:05.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:05.742 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:05.742 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:05.742 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:22:05.742 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:05.742 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:05.743 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:05.743 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:05.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.812 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.812 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158471 s, 66.2 MB/s 2024-04-25T10:22:05.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.830 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161097 s, 65.1 MB/s 2024-04-25T10:22:05.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.849 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169349 s, 61.9 MB/s 2024-04-25T10:22:05.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.864 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143059 s, 73.3 MB/s 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167998 s, 62.4 MB/s 2024-04-25T10:22:05.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.899 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142913 s, 73.4 MB/s 2024-04-25T10:22:05.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.916 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014713 s, 71.3 MB/s 2024-04-25T10:22:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.936 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185064 s, 56.7 MB/s 2024-04-25T10:22:05.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.952 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146816 s, 71.4 MB/s 2024-04-25T10:22:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.971 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167181 s, 62.7 MB/s 2024-04-25T10:22:05.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:05.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:05.988 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160332 s, 65.4 MB/s 2024-04-25T10:22:06.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.005 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146999 s, 71.3 MB/s 2024-04-25T10:22:06.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.022 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149138 s, 70.3 MB/s 2024-04-25T10:22:06.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175255 s, 59.8 MB/s 2024-04-25T10:22:06.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.060 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170609 s, 61.5 MB/s 2024-04-25T10:22:06.078 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.079 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169697 s, 61.8 MB/s 2024-04-25T10:22:06.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153584 s, 68.3 MB/s 2024-04-25T10:22:06.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.113 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.113 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153824 s, 68.2 MB/s 2024-04-25T10:22:06.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.132 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173138 s, 60.6 MB/s 2024-04-25T10:22:06.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.149 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149509 s, 70.1 MB/s 2024-04-25T10:22:06.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.171 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.171 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199851 s, 52.5 MB/s 2024-04-25T10:22:06.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.192 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201186 s, 52.1 MB/s 2024-04-25T10:22:06.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.210 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159772 s, 65.6 MB/s 2024-04-25T10:22:06.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.227 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.227 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014529 s, 72.2 MB/s 2024-04-25T10:22:06.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157123 s, 66.7 MB/s 2024-04-25T10:22:06.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.263 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171275 s, 61.2 MB/s 2024-04-25T10:22:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185789 s, 56.4 MB/s 2024-04-25T10:22:06.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.303 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174221 s, 60.2 MB/s 2024-04-25T10:22:06.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.319 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150231 s, 69.8 MB/s 2024-04-25T10:22:06.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:06.336 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:06.336 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143218 s, 73.2 MB/s 2024-04-25T10:22:07.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.291 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.95387 s, 1.1 MB/s 2024-04-25T10:22:07.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.308 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147823 s, 70.9 MB/s 2024-04-25T10:22:07.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:07.310 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:07.310 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:07.310 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:22:07.310 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:07.310 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:07.310 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:07.310 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:07.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.388 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172954 s, 60.6 MB/s 2024-04-25T10:22:07.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.407 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.407 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173538 s, 60.4 MB/s 2024-04-25T10:22:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.423 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140956 s, 74.4 MB/s 2024-04-25T10:22:07.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.442 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167162 s, 62.7 MB/s 2024-04-25T10:22:07.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.459 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.459 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142962 s, 73.3 MB/s 2024-04-25T10:22:07.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.478 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.478 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174672 s, 60.0 MB/s 2024-04-25T10:22:07.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.494 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145396 s, 72.1 MB/s 2024-04-25T10:22:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.513 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164613 s, 63.7 MB/s 2024-04-25T10:22:07.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122836 s, 85.4 MB/s 2024-04-25T10:22:07.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.544 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.544 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151255 s, 69.3 MB/s 2024-04-25T10:22:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.563 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016706 s, 62.8 MB/s 2024-04-25T10:22:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173245 s, 60.5 MB/s 2024-04-25T10:22:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165545 s, 63.3 MB/s 2024-04-25T10:22:07.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.618 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017222 s, 60.9 MB/s 2024-04-25T10:22:07.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149987 s, 69.9 MB/s 2024-04-25T10:22:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148601 s, 70.6 MB/s 2024-04-25T10:22:07.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.674 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204378 s, 51.3 MB/s 2024-04-25T10:22:07.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169066 s, 62.0 MB/s 2024-04-25T10:22:07.711 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.711 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.711 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169257 s, 62.0 MB/s 2024-04-25T10:22:07.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.732 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195903 s, 53.5 MB/s 2024-04-25T10:22:07.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.753 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190273 s, 55.1 MB/s 2024-04-25T10:22:07.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.770 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145177 s, 72.2 MB/s 2024-04-25T10:22:07.786 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.786 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.787 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015178 s, 69.1 MB/s 2024-04-25T10:22:07.806 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.806 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.806 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173822 s, 60.3 MB/s 2024-04-25T10:22:07.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.822 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148782 s, 70.5 MB/s 2024-04-25T10:22:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170975 s, 61.3 MB/s 2024-04-25T10:22:07.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.865 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021752 s, 48.2 MB/s 2024-04-25T10:22:07.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.881 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014696 s, 71.4 MB/s 2024-04-25T10:22:07.899 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169281 s, 61.9 MB/s 2024-04-25T10:22:07.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.922 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.922 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201927 s, 51.9 MB/s 2024-04-25T10:22:07.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.939 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157255 s, 66.7 MB/s 2024-04-25T10:22:07.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:07.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:07.958 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017367 s, 60.4 MB/s 2024-04-25T10:22:07.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:07.959 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:07.960 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:07.960 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:22:07.960 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:07.960 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:07.960 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:07.960 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:08.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.037 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169742 s, 61.8 MB/s 2024-04-25T10:22:08.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.053 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.053 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146085 s, 71.8 MB/s 2024-04-25T10:22:08.071 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.071 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.071 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160845 s, 65.2 MB/s 2024-04-25T10:22:08.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.094 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211052 s, 49.7 MB/s 2024-04-25T10:22:08.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.113 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016715 s, 62.7 MB/s 2024-04-25T10:22:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150143 s, 69.8 MB/s 2024-04-25T10:22:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.147 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159069 s, 65.9 MB/s 2024-04-25T10:22:08.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.164 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.164 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015489 s, 67.7 MB/s 2024-04-25T10:22:08.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140865 s, 74.4 MB/s 2024-04-25T10:22:08.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.200 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176725 s, 59.3 MB/s 2024-04-25T10:22:08.216 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.216 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.217 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015391 s, 68.1 MB/s 2024-04-25T10:22:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.236 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017702 s, 59.2 MB/s 2024-04-25T10:22:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163259 s, 64.2 MB/s 2024-04-25T10:22:08.273 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.273 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176004 s, 59.6 MB/s 2024-04-25T10:22:08.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.298 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223949 s, 46.8 MB/s 2024-04-25T10:22:08.314 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.314 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.314 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014569 s, 72.0 MB/s 2024-04-25T10:22:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.331 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155488 s, 67.4 MB/s 2024-04-25T10:22:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146014 s, 71.8 MB/s 2024-04-25T10:22:08.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.365 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151393 s, 69.3 MB/s 2024-04-25T10:22:08.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.386 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.386 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194311 s, 54.0 MB/s 2024-04-25T10:22:08.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153378 s, 68.4 MB/s 2024-04-25T10:22:08.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.419 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140916 s, 74.4 MB/s 2024-04-25T10:22:08.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.436 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153284 s, 68.4 MB/s 2024-04-25T10:22:08.455 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.455 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.455 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169352 s, 61.9 MB/s 2024-04-25T10:22:08.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.472 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151276 s, 69.3 MB/s 2024-04-25T10:22:08.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.489 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158928 s, 66.0 MB/s 2024-04-25T10:22:08.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.515 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.515 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243248 s, 43.1 MB/s 2024-04-25T10:22:08.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.535 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174077 s, 60.2 MB/s 2024-04-25T10:22:08.551 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.551 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.551 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146517 s, 71.6 MB/s 2024-04-25T10:22:08.570 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.571 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.571 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177489 s, 59.1 MB/s 2024-04-25T10:22:08.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144491 s, 72.6 MB/s 2024-04-25T10:22:08.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.609 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.609 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202823 s, 51.7 MB/s 2024-04-25T10:22:08.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:08.610 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:08.610 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:08.610 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:22:08.611 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:08.611 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:08.611 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:08.611 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183257 s, 57.2 MB/s 2024-04-25T10:22:08.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.699 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140354 s, 74.7 MB/s 2024-04-25T10:22:08.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.715 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149899 s, 70.0 MB/s 2024-04-25T10:22:08.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.732 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.732 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014975 s, 70.0 MB/s 2024-04-25T10:22:08.751 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.751 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.751 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173896 s, 60.3 MB/s 2024-04-25T10:22:08.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.773 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195666 s, 53.6 MB/s 2024-04-25T10:22:08.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.789 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150118 s, 69.9 MB/s 2024-04-25T10:22:08.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.811 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198906 s, 52.7 MB/s 2024-04-25T10:22:08.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.827 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146426 s, 71.6 MB/s 2024-04-25T10:22:08.847 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.848 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188348 s, 55.7 MB/s 2024-04-25T10:22:08.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.865 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015208 s, 68.9 MB/s 2024-04-25T10:22:08.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.884 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175104 s, 59.9 MB/s 2024-04-25T10:22:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172071 s, 60.9 MB/s 2024-04-25T10:22:08.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019512 s, 53.7 MB/s 2024-04-25T10:22:08.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.940 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147718 s, 71.0 MB/s 2024-04-25T10:22:08.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.961 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018472 s, 56.8 MB/s 2024-04-25T10:22:08.976 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141662 s, 74.0 MB/s 2024-04-25T10:22:08.994 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:08.994 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:08.994 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153054 s, 68.5 MB/s 2024-04-25T10:22:09.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.010 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143681 s, 73.0 MB/s 2024-04-25T10:22:09.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.026 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014795 s, 70.9 MB/s 2024-04-25T10:22:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.043 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146854 s, 71.4 MB/s 2024-04-25T10:22:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204208 s, 51.3 MB/s 2024-04-25T10:22:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.081 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142824 s, 73.4 MB/s 2024-04-25T10:22:09.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.099 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158118 s, 66.3 MB/s 2024-04-25T10:22:09.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145995 s, 71.8 MB/s 2024-04-25T10:22:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215691 s, 48.6 MB/s 2024-04-25T10:22:09.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177575 s, 59.0 MB/s 2024-04-25T10:22:09.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.175 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.175 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153077 s, 68.5 MB/s 2024-04-25T10:22:09.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.191 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143534 s, 73.1 MB/s 2024-04-25T10:22:09.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143974 s, 72.8 MB/s 2024-04-25T10:22:09.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157286 s, 66.7 MB/s 2024-04-25T10:22:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.245 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184045 s, 57.0 MB/s 2024-04-25T10:22:09.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:09.247 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:09.247 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:09.247 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:22:09.247 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:09.247 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:09.247 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:09.247 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172001 s, 61.0 MB/s 2024-04-25T10:22:09.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.338 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.339 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152585 s, 68.7 MB/s 2024-04-25T10:22:09.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.364 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238625 s, 43.9 MB/s 2024-04-25T10:22:09.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141152 s, 74.3 MB/s 2024-04-25T10:22:09.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.395 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136849 s, 76.6 MB/s 2024-04-25T10:22:09.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.414 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167217 s, 62.7 MB/s 2024-04-25T10:22:09.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017298 s, 60.6 MB/s 2024-04-25T10:22:09.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.452 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176049 s, 59.6 MB/s 2024-04-25T10:22:09.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.469 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148699 s, 70.5 MB/s 2024-04-25T10:22:09.488 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.488 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.488 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174561 s, 60.1 MB/s 2024-04-25T10:22:09.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017743 s, 59.1 MB/s 2024-04-25T10:22:09.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.530 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.530 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210362 s, 49.8 MB/s 2024-04-25T10:22:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157881 s, 66.4 MB/s 2024-04-25T10:22:09.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.565 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.565 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156879 s, 66.8 MB/s 2024-04-25T10:22:09.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200806 s, 52.2 MB/s 2024-04-25T10:22:09.604 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.604 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.604 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015084 s, 69.5 MB/s 2024-04-25T10:22:09.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.621 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150471 s, 69.7 MB/s 2024-04-25T10:22:09.637 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.637 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.637 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149592 s, 70.1 MB/s 2024-04-25T10:22:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179809 s, 58.3 MB/s 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148789 s, 70.5 MB/s 2024-04-25T10:22:09.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.690 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.690 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147579 s, 71.1 MB/s 2024-04-25T10:22:09.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.709 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170499 s, 61.5 MB/s 2024-04-25T10:22:09.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.726 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147303 s, 71.2 MB/s 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171317 s, 61.2 MB/s 2024-04-25T10:22:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150131 s, 69.8 MB/s 2024-04-25T10:22:09.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143391 s, 73.1 MB/s 2024-04-25T10:22:09.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.792 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.792 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128226 s, 81.8 MB/s 2024-04-25T10:22:09.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.808 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140148 s, 74.8 MB/s 2024-04-25T10:22:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.830 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204581 s, 51.3 MB/s 2024-04-25T10:22:09.847 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.847 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.847 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147109 s, 71.3 MB/s 2024-04-25T10:22:09.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.863 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.863 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014198 s, 73.9 MB/s 2024-04-25T10:22:09.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.882 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168411 s, 62.3 MB/s 2024-04-25T10:22:09.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:09.883 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:09.883 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:09.883 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:22:09.883 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:09.883 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:09.883 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:09.883 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:09.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.957 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170213 s, 61.6 MB/s 2024-04-25T10:22:09.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175821 s, 59.6 MB/s 2024-04-25T10:22:09.998 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:09.998 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:09.999 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202623 s, 51.8 MB/s 2024-04-25T10:22:10.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.022 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.022 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201399 s, 52.1 MB/s 2024-04-25T10:22:10.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.037 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.037 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145949 s, 71.8 MB/s 2024-04-25T10:22:10.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163904 s, 64.0 MB/s 2024-04-25T10:22:10.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168455 s, 62.2 MB/s 2024-04-25T10:22:10.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.090 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145957 s, 71.8 MB/s 2024-04-25T10:22:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228758 s, 45.8 MB/s 2024-04-25T10:22:10.134 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.135 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178583 s, 58.7 MB/s 2024-04-25T10:22:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.151 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147522 s, 71.1 MB/s 2024-04-25T10:22:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155997 s, 67.2 MB/s 2024-04-25T10:22:10.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022559 s, 46.5 MB/s 2024-04-25T10:22:10.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.210 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.210 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154617 s, 67.8 MB/s 2024-04-25T10:22:10.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.226 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.226 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142129 s, 73.8 MB/s 2024-04-25T10:22:10.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.243 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155023 s, 67.6 MB/s 2024-04-25T10:22:10.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.260 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014822 s, 70.7 MB/s 2024-04-25T10:22:10.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.277 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.277 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156171 s, 67.1 MB/s 2024-04-25T10:22:10.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.295 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157465 s, 66.6 MB/s 2024-04-25T10:22:10.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.311 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139645 s, 75.1 MB/s 2024-04-25T10:22:10.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170698 s, 61.4 MB/s 2024-04-25T10:22:10.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.352 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205994 s, 50.9 MB/s 2024-04-25T10:22:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146117 s, 71.8 MB/s 2024-04-25T10:22:10.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.386 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158809 s, 66.0 MB/s 2024-04-25T10:22:10.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.412 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243342 s, 43.1 MB/s 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233237 s, 45.0 MB/s 2024-04-25T10:22:10.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157007 s, 66.8 MB/s 2024-04-25T10:22:10.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152288 s, 68.9 MB/s 2024-04-25T10:22:10.488 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.488 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.488 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148238 s, 70.7 MB/s 2024-04-25T10:22:10.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.510 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205621 s, 51.0 MB/s 2024-04-25T10:22:10.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.526 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145344 s, 72.1 MB/s 2024-04-25T10:22:10.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175958 s, 59.6 MB/s 2024-04-25T10:22:10.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:10.547 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:10.547 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:10.547 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:22:10.548 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:10.548 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:10.548 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:10.548 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.627 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182093 s, 57.6 MB/s 2024-04-25T10:22:10.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164167 s, 63.9 MB/s 2024-04-25T10:22:10.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168732 s, 62.1 MB/s 2024-04-25T10:22:10.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139399 s, 75.2 MB/s 2024-04-25T10:22:10.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.695 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147853 s, 70.9 MB/s 2024-04-25T10:22:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201845 s, 51.9 MB/s 2024-04-25T10:22:10.733 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.733 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.733 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014769 s, 71.0 MB/s 2024-04-25T10:22:10.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.757 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02194 s, 47.8 MB/s 2024-04-25T10:22:10.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.773 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146973 s, 71.3 MB/s 2024-04-25T10:22:10.791 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.791 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.791 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015756 s, 66.6 MB/s 2024-04-25T10:22:10.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.807 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144915 s, 72.4 MB/s 2024-04-25T10:22:10.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.823 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144243 s, 72.7 MB/s 2024-04-25T10:22:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016771 s, 62.5 MB/s 2024-04-25T10:22:10.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.860 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163084 s, 64.3 MB/s 2024-04-25T10:22:10.878 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.878 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.878 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164612 s, 63.7 MB/s 2024-04-25T10:22:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175764 s, 59.7 MB/s 2024-04-25T10:22:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148509 s, 70.6 MB/s 2024-04-25T10:22:10.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.930 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138496 s, 75.7 MB/s 2024-04-25T10:22:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145657 s, 72.0 MB/s 2024-04-25T10:22:10.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.963 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145831 s, 71.9 MB/s 2024-04-25T10:22:10.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.979 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.979 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145463 s, 72.1 MB/s 2024-04-25T10:22:10.995 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:10.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:10.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146825 s, 71.4 MB/s 2024-04-25T10:22:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.012 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144261 s, 72.7 MB/s 2024-04-25T10:22:11.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.031 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015151 s, 69.2 MB/s 2024-04-25T10:22:11.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.045 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146234 s, 71.7 MB/s 2024-04-25T10:22:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173363 s, 60.5 MB/s 2024-04-25T10:22:11.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.083 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.083 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168947 s, 62.1 MB/s 2024-04-25T10:22:11.099 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.099 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.099 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147919 s, 70.9 MB/s 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195643 s, 53.6 MB/s 2024-04-25T10:22:11.137 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.137 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.137 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151249 s, 69.3 MB/s 2024-04-25T10:22:11.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.157 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.157 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178557 s, 58.7 MB/s 2024-04-25T10:22:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.173 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.174 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145343 s, 72.1 MB/s 2024-04-25T10:22:11.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.175 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:11.175 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:11.175 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:22:11.175 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:11.175 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:11.175 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:11.175 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:11.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.245 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119801 s, 87.5 MB/s 2024-04-25T10:22:11.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.263 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164514 s, 63.7 MB/s 2024-04-25T10:22:11.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.286 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.286 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213197 s, 49.2 MB/s 2024-04-25T10:22:11.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.302 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141572 s, 74.1 MB/s 2024-04-25T10:22:11.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.316 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125605 s, 83.5 MB/s 2024-04-25T10:22:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.332 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140898 s, 74.4 MB/s 2024-04-25T10:22:11.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148851 s, 70.4 MB/s 2024-04-25T10:22:11.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179355 s, 58.5 MB/s 2024-04-25T10:22:11.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185342 s, 56.6 MB/s 2024-04-25T10:22:11.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.412 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208251 s, 50.4 MB/s 2024-04-25T10:22:11.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199767 s, 52.5 MB/s 2024-04-25T10:22:11.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.451 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162825 s, 64.4 MB/s 2024-04-25T10:22:11.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.469 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.470 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166117 s, 63.1 MB/s 2024-04-25T10:22:11.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.493 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217507 s, 48.2 MB/s 2024-04-25T10:22:11.512 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.512 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.512 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170997 s, 61.3 MB/s 2024-04-25T10:22:11.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151227 s, 69.3 MB/s 2024-04-25T10:22:11.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144496 s, 72.6 MB/s 2024-04-25T10:22:11.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.563 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160579 s, 65.3 MB/s 2024-04-25T10:22:11.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.583 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184699 s, 56.8 MB/s 2024-04-25T10:22:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.600 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147301 s, 71.2 MB/s 2024-04-25T10:22:11.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.618 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016352 s, 64.1 MB/s 2024-04-25T10:22:11.636 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.636 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.636 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166818 s, 62.9 MB/s 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199416 s, 52.6 MB/s 2024-04-25T10:22:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.676 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166315 s, 63.0 MB/s 2024-04-25T10:22:11.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.695 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.695 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176486 s, 59.4 MB/s 2024-04-25T10:22:11.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.713 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159786 s, 65.6 MB/s 2024-04-25T10:22:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145844 s, 71.9 MB/s 2024-04-25T10:22:11.747 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.747 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.747 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155703 s, 67.3 MB/s 2024-04-25T10:22:11.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.766 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167503 s, 62.6 MB/s 2024-04-25T10:22:11.782 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.783 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149101 s, 70.3 MB/s 2024-04-25T10:22:11.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.798 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141034 s, 74.3 MB/s 2024-04-25T10:22:11.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.820 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198588 s, 52.8 MB/s 2024-04-25T10:22:11.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.822 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:22:11.822 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:22:11.822 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:22:11.822 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:22:11.822 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:22:11.822 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:22:11.822 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:22:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172635 s, 60.7 MB/s 2024-04-25T10:22:11.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.914 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147981 s, 70.9 MB/s 2024-04-25T10:22:11.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.930 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145255 s, 72.2 MB/s 2024-04-25T10:22:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148826 s, 70.5 MB/s 2024-04-25T10:22:11.965 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.966 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.966 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171282 s, 61.2 MB/s 2024-04-25T10:22:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144915 s, 72.4 MB/s 2024-04-25T10:22:12.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.006 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222553 s, 47.1 MB/s 2024-04-25T10:22:12.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.024 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.024 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156593 s, 67.0 MB/s 2024-04-25T10:22:12.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.046 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.046 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204768 s, 51.2 MB/s 2024-04-25T10:22:12.065 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.065 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.065 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174431 s, 60.1 MB/s 2024-04-25T10:22:12.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.086 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.086 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018941 s, 55.4 MB/s 2024-04-25T10:22:12.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201516 s, 52.0 MB/s 2024-04-25T10:22:12.124 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.124 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.124 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014736 s, 71.2 MB/s 2024-04-25T10:22:12.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202117 s, 51.9 MB/s 2024-04-25T10:22:12.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149605 s, 70.1 MB/s 2024-04-25T10:22:12.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.180 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.180 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154948 s, 67.7 MB/s 2024-04-25T10:22:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.198 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155758 s, 67.3 MB/s 2024-04-25T10:22:12.216 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.216 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.216 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169661 s, 61.8 MB/s 2024-04-25T10:22:12.238 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.238 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.239 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202927 s, 51.7 MB/s 2024-04-25T10:22:12.261 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.261 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.261 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195981 s, 53.5 MB/s 2024-04-25T10:22:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165745 s, 63.3 MB/s 2024-04-25T10:22:12.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206352 s, 50.8 MB/s 2024-04-25T10:22:12.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.318 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161322 s, 65.0 MB/s 2024-04-25T10:22:12.337 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.337 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.337 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173585 s, 60.4 MB/s 2024-04-25T10:22:12.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190035 s, 55.2 MB/s 2024-04-25T10:22:12.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.380 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202659 s, 51.7 MB/s 2024-04-25T10:22:12.394 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.395 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124324 s, 84.3 MB/s 2024-04-25T10:22:12.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.412 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162546 s, 64.5 MB/s 2024-04-25T10:22:12.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.428 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.428 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140829 s, 74.5 MB/s 2024-04-25T10:22:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160045 s, 65.5 MB/s 2024-04-25T10:22:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137272 s, 76.4 MB/s 2024-04-25T10:22:12.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:22:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:22:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179911 s, 58.3 MB/s 2024-04-25T10:22:12.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.483 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:22:12.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7e6a00640 1 Processor -- start 2024-04-25T10:22:12.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7e6a00640 1 -- start start 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7e6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7e00cf040 0x7fd7e00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7e6a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e0071f80 con 0x7fd7e0071ab0 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7e6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e015b120 con 0x7fd7e015bc00 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7e6a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e0072260 con 0x7fd7e00cf040 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7e00cf040 0x7fd7e00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7e00cf040 0x7fd7e00cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34716/0 (socket says 172.21.15.87:34716) 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7e4a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd7e0071ab0 0x7fd7e00ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36656/0 (socket says 172.21.15.87:36656) 2024-04-25T10:22:12.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.662+0000 7fd7dfe00640 1 -- 172.21.15.87:0/3910901427 learned_addr learned my addr 172.21.15.87:0/3910901427 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7dfe00640 1 -- 172.21.15.87:0/3910901427 >> v1:172.21.15.169:6789/0 conn(0x7fd7e015bc00 legacy=0x7fd7e00713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7dfe00640 1 -- 172.21.15.87:0/3910901427 >> v1:172.21.15.87:6789/0 conn(0x7fd7e0071ab0 legacy=0x7fd7e00ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7dfe00640 1 -- 172.21.15.87:0/3910901427 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e015b400 con 0x7fd7e00cf040 2024-04-25T10:22:12.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/3910901427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e00cf040 0x7fd7e00cf9e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009e50 tx=0x7fd7d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c868917370b169e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7dea00640 1 -- 172.21.15.87:0/3910901427 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00a56c0 con 0x7fd7e00cf040 2024-04-25T10:22:12.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7dea00640 1 -- 172.21.15.87:0/3910901427 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00a5860 con 0x7fd7e00cf040 2024-04-25T10:22:12.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7dea00640 1 -- 172.21.15.87:0/3910901427 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00a5c30 con 0x7fd7e00cf040 2024-04-25T10:22:12.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7e6a00640 1 -- 172.21.15.87:0/3910901427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e00cf040 msgr2=0x7fd7e00cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/3910901427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e00cf040 0x7fd7e00cf9e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009e50 tx=0x7fd7d001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7e6a00640 1 -- 172.21.15.87:0/3910901427 shutdown_connections 2024-04-25T10:22:12.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/3910901427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e00cf040 0x7fd7e00cf9e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.666+0000 7fd7e6a00640 1 -- 172.21.15.87:0/3910901427 >> 172.21.15.87:0/3910901427 conn(0x7fd7e0066b70 msgr2=0x7fd7e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- 172.21.15.87:0/3910901427 shutdown_connections 2024-04-25T10:22:12.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- 172.21.15.87:0/3910901427 wait complete. 2024-04-25T10:22:12.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 Processor -- start 2024-04-25T10:22:12.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- start start 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e0071ab0 0x7fd7e0152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e00cf040 0x7fd7e01f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e015bc00 0x7fd7e00b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00d0330 con 0x7fd7e0071ab0 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00cf630 con 0x7fd7e015bc00 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0168870 con 0x7fd7e00cf040 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e0071ab0 0x7fd7e0152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e0071ab0 0x7fd7e0152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34728/0 (socket says 172.21.15.87:34728) 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 -- 172.21.15.87:0/2765353967 learned_addr learned my addr 172.21.15.87:0/2765353967 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e4a00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e015bc00 0x7fd7e00b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7df400640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e00cf040 0x7fd7e01f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 -- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e00cf040 msgr2=0x7fd7e01f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e00cf040 0x7fd7e01f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 -- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e015bc00 msgr2=0x7fd7e00b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e015bc00 0x7fd7e00b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 -- 172.21.15.87:0/2765353967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e0152570 con 0x7fd7e0071ab0 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e0071ab0 0x7fd7e0152030 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00af040 tx=0x7fd7d0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7d7e00640 1 -- 172.21.15.87:0/2765353967 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00b8670 con 0x7fd7e0071ab0 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7d7e00640 1 -- 172.21.15.87:0/2765353967 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00be070 con 0x7fd7e0071ab0 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7d7e00640 1 -- 172.21.15.87:0/2765353967 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00b8a50 con 0x7fd7e0071ab0 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e0152840 con 0x7fd7e0071ab0 2024-04-25T10:22:12.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.670+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e0152e00 con 0x7fd7e0071ab0 2024-04-25T10:22:12.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.674+0000 7fd7d7e00640 1 -- 172.21.15.87:0/2765353967 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7d00b8bf0 con 0x7fd7e0071ab0 2024-04-25T10:22:12.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.674+0000 7fd7d7e00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7c0066f60 0x7fd7c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.674+0000 7fd7d7e00640 1 -- 172.21.15.87:0/2765353967 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7d0120250 con 0x7fd7e0071ab0 2024-04-25T10:22:12.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.674+0000 7fd7df400640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7c0066f60 0x7fd7c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.674+0000 7fd7e9e1a000 1 Processor -- start 2024-04-25T10:22:12.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.674+0000 7fd7e9e1a000 1 -- start start 2024-04-25T10:22:12.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7df400640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7c0066f60 0x7fd7c0069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd7e015ae90 tx=0x7fd7cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a80bd50 con 0x561d6aa073f0 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a793e40 con 0x561d6aa07d10 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d6a807df0 con 0x561d6a9be520 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7df400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34736/0 (socket says 172.21.15.87:34736) 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 -- 172.21.15.87:0/4245326567 learned_addr learned my addr 172.21.15.87:0/4245326567 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e4a00640 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 -- 172.21.15.87:0/4245326567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 msgr2=0x561d6a9be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 -- 172.21.15.87:0/4245326567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 msgr2=0x561d6a9bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 -- 172.21.15.87:0/4245326567 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d6a7a8d90 con 0x561d6aa073f0 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa077d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00f0840 tx=0x7fd7d0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826b02eb756f6a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7d4c00640 1 -- 172.21.15.87:0/4245326567 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0120050 con 0x561d6aa073f0 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7d4c00640 1 -- 172.21.15.87:0/4245326567 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00f0b40 con 0x561d6aa073f0 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7d4c00640 1 -- 172.21.15.87:0/4245326567 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00f4c90 con 0x561d6aa073f0 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/4245326567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 msgr2=0x561d6aa077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa077d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00f0840 tx=0x7fd7d0009810 comp rx=0 tx=0).stop 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/4245326567 shutdown_connections 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/4245326567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa077d0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/4245326567 >> 172.21.15.87:0/4245326567 conn(0x561d6a9ba4d0 msgr2=0x561d6a9bc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/4245326567 shutdown_connections 2024-04-25T10:22:12.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/4245326567 wait complete. 2024-04-25T10:22:12.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 -- start start 2024-04-25T10:22:12.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.678+0000 7fd7e9e1a000 1 Processor -- start 2024-04-25T10:22:12.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a7b1610 con 0x561d6aa073f0 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a7c69a0 con 0x561d6aa07d10 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d6a810380 con 0x561d6a9be520 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 10 client.0 _ll_register_callbacks cb 0x561d6a8c00d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34748/0 (socket says 172.21.15.87:34748) 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 -- 172.21.15.87:0/1546971224 learned_addr learned my addr 172.21.15.87:0/1546971224 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e4a00640 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 -- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 msgr2=0x561d6a9c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 -- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 msgr2=0x561d6a9f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 -- 172.21.15.87:0/1546971224 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d6a806d40 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7df400640 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9f50c0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00ed10 tx=0x7fd7cc003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43a55445281274d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 1 -- 172.21.15.87:0/1546971224 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc025020 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 1 -- 172.21.15.87:0/1546971224 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7cc010040 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7d00f0b40 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 1 -- 172.21.15.87:0/1546971224 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7cc02f050 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 10 client.9279 fetch_fsmap learned FSMap version 89 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x561d6a88ef00 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 1 -- 172.21.15.87:0/1546971224 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00f0b40 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 1 -- 172.21.15.87:0/1546971224 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7cc032020 con 0x561d6aa073f0 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 10 client.9279 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 10 client.9279 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 20 client.9279 populate_metadata read hostname 'smithi087' 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa0e830 0x561d6aa0ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 4 client.9279 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:22:12.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d6a7a7a20 con 0x561d6aa0e830 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa0e830 0x561d6aa0ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa0e830 0x561d6aa0ec40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83ec7e00270c4a4 server_cookie=90793025c6adbcb9 in_seq=0 out_seq=0 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 10 client.9279 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 1 -- 172.21.15.87:0/1546971224 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d6a7a7a20 con 0x561d6aa0e830 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 10 client.9279 handle_command_reply: tid=0 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 client.9279 shutdown 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 msgr2=0x561d6a9f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9f50c0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00ed10 tx=0x7fd7cc003770 comp rx=0 tx=0).stop 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 shutdown_connections 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa0e830 0x561d6aa0ec40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6a9f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9f50c0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7c9600640 0 client.9279 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1546971224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6a9c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.682+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 >> 172.21.15.87:0/1546971224 conn(0x561d6a9ba4d0 msgr2=0x561d6a9bd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.686+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 shutdown_connections 2024-04-25T10:22:12.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.686+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1546971224 wait complete. 2024-04-25T10:22:12.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.686+0000 7fd7e9e1a000 20 client.9279 trim_cache size 0 max 16384 2024-04-25T10:22:12.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.698+0000 7fd7e9e1a000 1 Processor -- start 2024-04-25T10:22:12.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.698+0000 7fd7e9e1a000 1 -- start start 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.698+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6a9be520 0x561d6a9bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6aa07d10 0x561d6aa0a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0203430 con 0x561d6aa073f0 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0202620 con 0x561d6a9be520 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01f8240 con 0x561d6aa07d10 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9bde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34754/0 (socket says 172.21.15.87:34754) 2024-04-25T10:22:12.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 -- 172.21.15.87:0/1247498381 learned_addr learned my addr 172.21.15.87:0/1247498381 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6a9be520 0x561d6a9bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e4a00640 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6aa07d10 0x561d6aa0a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 -- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6aa07d10 msgr2=0x561d6aa0a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6aa07d10 0x561d6aa0a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 -- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6a9be520 msgr2=0x561d6a9bd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6a9be520 0x561d6a9bd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 -- 172.21.15.87:0/1247498381 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d6aa07b40 con 0x561d6aa073f0 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6a9be520 0x561d6a9bd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9bde10 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc015d10 tx=0x7fd7cc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd77e5225f18b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7cbe00640 1 -- 172.21.15.87:0/1247498381 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc038070 con 0x561d6aa073f0 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7cbe00640 1 -- 172.21.15.87:0/1247498381 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0053e0 con 0x561d6aa073f0 2024-04-25T10:22:12.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7cbe00640 1 -- 172.21.15.87:0/1247498381 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc025030 con 0x561d6aa073f0 2024-04-25T10:22:12.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1247498381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 msgr2=0x561d6a9bde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9bde10 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc015d10 tx=0x7fd7cc021000 comp rx=0 tx=0).stop 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1247498381 shutdown_connections 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6aa07d10 0x561d6aa0a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6a9bde10 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/1247498381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6a9be520 0x561d6a9bd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1247498381 >> 172.21.15.87:0/1247498381 conn(0x561d6a9ba4d0 msgr2=0x561d6aa05f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1247498381 shutdown_connections 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/1247498381 wait complete. 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- start start 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 Processor -- start 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6aa0cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa6d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6aa6d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a7c2dc0 con 0x561d6aa073f0 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00ce7a0 con 0x561d6aa07d10 2024-04-25T10:22:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d6a80b060 con 0x561d6a9be520 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e9e1a000 10 client.0 _ll_register_callbacks cb 0x7fd7d0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6aa0cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6aa0cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35810/0 (socket says 172.21.15.87:35810) 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7dfe00640 1 -- 172.21.15.87:0/623668195 learned_addr learned my addr 172.21.15.87:0/623668195 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7df400640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa6d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.702+0000 7fd7e4a00640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6aa6d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7dfe00640 1 -- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 msgr2=0x561d6aa6d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6aa6d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7dfe00640 1 -- 172.21.15.87:0/623668195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 msgr2=0x561d6aa6d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa6d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7dfe00640 1 -- 172.21.15.87:0/623668195 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d6a865330 con 0x561d6a9be520 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7df400640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa6d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7dfe00640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6aa0cc70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7d001d290 tx=0x7fd7d00f7040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7ba1d4f38e1dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7c9600640 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7c9600640 1 -- 172.21.15.87:0/623668195 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00f0b40 con 0x561d6a9be520 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7c9600640 1 -- 172.21.15.87:0/623668195 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7d00ed1b0 con 0x561d6a9be520 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7cc0053e0 con 0x561d6a9be520 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7c9600640 1 -- 172.21.15.87:0/623668195 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d00f9250 con 0x561d6a9be520 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 10 client.7511 fetch_fsmap learned FSMap version 89 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x561d6a7d85f0 con 0x561d6a9be520 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7c9600640 1 -- 172.21.15.87:0/623668195 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0053e0 con 0x561d6a9be520 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7c9600640 1 -- 172.21.15.87:0/623668195 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7d011e210 con 0x561d6a9be520 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 10 client.7511 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 10 client.7511 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 20 client.7511 populate_metadata read hostname 'smithi087' 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa6db70 0x561d6aa6df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 4 client.7511 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x561d6a819290 con 0x561d6aa6db70 2024-04-25T10:22:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7df400640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa6db70 0x561d6aa6df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7df400640 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa6db70 0x561d6aa6df50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2788aacbdce82d2c server_cookie=8dea69a379d390e0 in_seq=0 out_seq=0 2024-04-25T10:22:12.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.706+0000 7fd7c9600640 10 client.7511 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7c9600640 1 -- 172.21.15.87:0/623668195 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561d6a819290 con 0x561d6aa6db70 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7c9600640 10 client.7511 handle_command_reply: tid=0 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 client.7511 shutdown 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 msgr2=0x561d6aa0cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6aa0cc70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7d001d290 tx=0x7fd7d00f7040 comp rx=0 tx=0).stop 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 shutdown_connections 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x561d6aa6db70 0x561d6aa6df50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561d6aa07d10 0x561d6aa6d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d6aa073f0 0x561d6aa6d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7c9600640 0 client.7511 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 --2- 172.21.15.87:0/623668195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561d6a9be520 0x561d6aa0cc70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 >> 172.21.15.87:0/623668195 conn(0x561d6a9ba4d0 msgr2=0x561d6aa067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 shutdown_connections 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 1 -- 172.21.15.87:0/623668195 wait complete. 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e9e1a000 20 client.7511 trim_cache size 0 max 16384 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:22:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "6e648a7c-db42-45b7-b2ba-2e34d6cecb87", 2024-04-25T10:22:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:22:12.716 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:22:12.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7c0066f60 msgr2=0x7fd7c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7c0066f60 0x7fd7c0069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd7e015ae90 tx=0x7fd7cc009040 comp rx=0 tx=0).stop 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e0071ab0 msgr2=0x7fd7e0152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e0071ab0 0x7fd7e0152030 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00af040 tx=0x7fd7d0019790 comp rx=0 tx=0).stop 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 shutdown_connections 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd7c0066f60 0x7fd7c0069420 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e015bc00 0x7fd7e00b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e00cf040 0x7fd7e01f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 --2- 172.21.15.87:0/2765353967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7e0071ab0 0x7fd7e0152030 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 >> 172.21.15.87:0/2765353967 conn(0x7fd7e0066b70 msgr2=0x7fd7e0066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 shutdown_connections 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.710+0000 7fd7e6a00640 1 -- 172.21.15.87:0/2765353967 wait complete. 2024-04-25T10:22:12.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.733 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T10:22:12.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.882+0000 7ff638800640 1 Processor -- start 2024-04-25T10:22:12.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.882+0000 7ff638800640 1 -- start start 2024-04-25T10:22:12.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff638800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff6300ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff638800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff63015ee90 0x7ff6300d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff638800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6300d0f90 0x7ff6300b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff638800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63015dfc0 con 0x7ff6300d0f90 2024-04-25T10:22:12.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff638800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6300d1fc0 con 0x7ff63015ee90 2024-04-25T10:22:12.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff638800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300d22a0 con 0x7ff63015dbe0 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6300d0f90 0x7ff6300b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6300d0f90 0x7ff6300b8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34772/0 (socket says 172.21.15.87:34772) 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 -- 172.21.15.87:0/3911343163 learned_addr learned my addr 172.21.15.87:0/3911343163 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff635e00640 1 --2- 172.21.15.87:0/3911343163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff6300ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff635400640 1 --2- 172.21.15.87:0/3911343163 >> v2:172.21.15.169:3300/0 conn(0x7ff63015ee90 0x7ff6300d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 -- 172.21.15.87:0/3911343163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 msgr2=0x7ff6300ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 --2- 172.21.15.87:0/3911343163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff6300ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 -- 172.21.15.87:0/3911343163 >> v2:172.21.15.169:3300/0 conn(0x7ff63015ee90 msgr2=0x7ff6300d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 --2- 172.21.15.87:0/3911343163 >> v2:172.21.15.169:3300/0 conn(0x7ff63015ee90 0x7ff6300d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff636800640 1 -- 172.21.15.87:0/3911343163 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6300d2580 con 0x7ff6300d0f90 2024-04-25T10:22:12.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.886+0000 7ff635400640 1 --2- 172.21.15.87:0/3911343163 >> v2:172.21.15.169:3300/0 conn(0x7ff63015ee90 0x7ff6300d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:12.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff636800640 1 --2- 172.21.15.87:0/3911343163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff6300b8c90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff6300b66f0 tx=0x7ff62c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6d4333ccfbd573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff634a00640 1 -- 172.21.15.87:0/3911343163 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0a5670 con 0x7ff6300d0f90 2024-04-25T10:22:12.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff634a00640 1 -- 172.21.15.87:0/3911343163 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c0a5810 con 0x7ff6300d0f90 2024-04-25T10:22:12.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff634a00640 1 -- 172.21.15.87:0/3911343163 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0a5be0 con 0x7ff6300d0f90 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- 172.21.15.87:0/3911343163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 msgr2=0x7ff6300b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 --2- 172.21.15.87:0/3911343163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff6300b8c90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff6300b66f0 tx=0x7ff62c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- 172.21.15.87:0/3911343163 shutdown_connections 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 --2- 172.21.15.87:0/3911343163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff6300b8c90 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 --2- 172.21.15.87:0/3911343163 >> v2:172.21.15.169:3300/0 conn(0x7ff63015ee90 0x7ff6300d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 --2- 172.21.15.87:0/3911343163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff6300ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- 172.21.15.87:0/3911343163 >> 172.21.15.87:0/3911343163 conn(0x7ff63006b090 msgr2=0x7ff63006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- 172.21.15.87:0/3911343163 shutdown_connections 2024-04-25T10:22:12.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- 172.21.15.87:0/3911343163 wait complete. 2024-04-25T10:22:12.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 Processor -- start 2024-04-25T10:22:12.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- start start 2024-04-25T10:22:12.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff63014e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff63015fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff63015ee90 0x7ff630160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630161b00 con 0x7ff6300d0f90 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630161980 con 0x7ff63015ee90 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.890+0000 7ff638800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff630161c80 con 0x7ff63015dbe0 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff63014e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff63014e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34782/0 (socket says 172.21.15.87:34782) 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 -- 172.21.15.87:0/3397956556 learned_addr learned my addr 172.21.15.87:0/3397956556 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635400640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff63015fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff636800640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff63015ee90 0x7ff630160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 -- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 msgr2=0x7ff63015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff63015fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 -- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff63015ee90 msgr2=0x7ff630160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff63015ee90 0x7ff630160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 -- 172.21.15.87:0/3397956556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63014eba0 con 0x7ff6300d0f90 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff635e00640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff63014e660 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff62400d800 tx=0x7ff62400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff62a000640 1 -- 172.21.15.87:0/3397956556 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624018020 con 0x7ff6300d0f90 2024-04-25T10:22:12.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff62a000640 1 -- 172.21.15.87:0/3397956556 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240096a0 con 0x7ff6300d0f90 2024-04-25T10:22:12.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff62a000640 1 -- 172.21.15.87:0/3397956556 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624009a50 con 0x7ff6300d0f90 2024-04-25T10:22:12.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63014f330 con 0x7ff6300d0f90 2024-04-25T10:22:12.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff630155890 con 0x7ff6300d0f90 2024-04-25T10:22:12.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.894+0000 7ff62a000640 1 -- 172.21.15.87:0/3397956556 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff62401b020 con 0x7ff6300d0f90 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff62a000640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff614066f60 0x7ff614069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff62a000640 1 -- 172.21.15.87:0/3397956556 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff624085440 con 0x7ff6300d0f90 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff614066f60 0x7ff614069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 Processor -- start 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 -- start start 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff614066f60 0x7ff614069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff6301f9d40 tx=0x7ff620008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010988e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010943850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x561010944170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610108261f0 con 0x561010989360 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101078ce90 con 0x561010988a40 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561010714f50 con 0x561010943d90 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010943850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010943850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34796/0 (socket says 172.21.15.87:34796) 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 -- 172.21.15.87:0/2350136869 learned_addr learned my addr 172.21.15.87:0/2350136869 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635e00640 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010988e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff636800640 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x561010944170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 -- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 msgr2=0x561010944170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x561010944170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 -- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 msgr2=0x561010988e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010988e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 -- 172.21.15.87:0/2350136869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610107648b0 con 0x561010989360 2024-04-25T10:22:12.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635e00640 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010988e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:12.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff635400640 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010943850 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff62000b7e0 tx=0x7ff620013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5337f591b10115f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff61ea00640 1 -- 172.21.15.87:0/2350136869 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62001c050 con 0x561010989360 2024-04-25T10:22:12.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff61ea00640 1 -- 172.21.15.87:0/2350136869 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620026070 con 0x561010989360 2024-04-25T10:22:12.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.898+0000 7ff61ea00640 1 -- 172.21.15.87:0/2350136869 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620016c00 con 0x561010989360 2024-04-25T10:22:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2350136869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 msgr2=0x561010943850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010943850 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff62000b7e0 tx=0x7ff620013000 comp rx=0 tx=0).stop 2024-04-25T10:22:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2350136869 shutdown_connections 2024-04-25T10:22:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x561010944170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010943850 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2350136869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010988e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2350136869 >> 172.21.15.87:0/2350136869 conn(0x56101093fd40 msgr2=0x561010942130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2350136869 shutdown_connections 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2350136869 wait complete. 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- start start 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 Processor -- start 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x5610109901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010990700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010949db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561010747540 con 0x561010989360 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561010788f30 con 0x561010988a40 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610107597c0 con 0x561010943d90 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 10 client.0 _ll_register_callbacks cb 0x561010841180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010949db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010949db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34804/0 (socket says 172.21.15.87:34804) 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 -- 172.21.15.87:0/1747910701 learned_addr learned my addr 172.21.15.87:0/1747910701 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff635400640 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010990700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff635e00640 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x5610109901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 -- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 msgr2=0x5610109901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x5610109901c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 -- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 msgr2=0x561010990700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010990700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 -- 172.21.15.87:0/1747910701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610107151a0 con 0x561010989360 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff636800640 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010949db0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0072a0 tx=0x7ff62c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0835f80bfed95a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff613400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff613400640 1 -- 172.21.15.87:0/1747910701 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0a5860 con 0x561010989360 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff613400640 1 -- 172.21.15.87:0/1747910701 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff62c0c2b10 con 0x561010989360 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff620026070 con 0x561010989360 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff613400640 1 -- 172.21.15.87:0/1747910701 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff62c0c2d30 con 0x561010989360 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 10 client.9297 fetch_fsmap learned FSMap version 89 2024-04-25T10:22:12.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x56101080f840 con 0x561010989360 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.902+0000 7ff613400640 1 -- 172.21.15.87:0/1747910701 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620026070 con 0x561010989360 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff613400640 1 -- 172.21.15.87:0/1747910701 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff62c0c9050 con 0x561010989360 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 10 client.9297 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 10 client.9297 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 20 client.9297 populate_metadata read hostname 'smithi087' 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101094ab20 0x56101094af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 4 client.9297 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561010728be0 con 0x56101094ab20 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff635e00640 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101094ab20 0x56101094af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff635e00640 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101094ab20 0x56101094af00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11edcf6adfe209f9 server_cookie=1c4c3692adf55e2c in_seq=0 out_seq=0 2024-04-25T10:22:12.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff613400640 10 client.9297 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff613400640 1 -- 172.21.15.87:0/1747910701 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561010728be0 con 0x56101094ab20 2024-04-25T10:22:12.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff613400640 10 client.9297 handle_command_reply: tid=0 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 client.9297 shutdown 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 msgr2=0x561010949db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010949db0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0072a0 tx=0x7ff62c004a50 comp rx=0 tx=0).stop 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 shutdown_connections 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101094ab20 0x56101094af00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x561010949db0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010988a40 0x561010990700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff613400640 0 client.9297 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/1747910701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010943d90 0x5610109901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 >> 172.21.15.87:0/1747910701 conn(0x56101093fd40 msgr2=0x5610109429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 shutdown_connections 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 1 -- 172.21.15.87:0/1747910701 wait complete. 2024-04-25T10:22:12.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.906+0000 7ff63bbdd000 20 client.9297 trim_cache size 0 max 16384 2024-04-25T10:22:12.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 Processor -- start 2024-04-25T10:22:12.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- start start 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010943d90 0x5610109e5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010989360 0x5610109e5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630203030 con 0x561010943d90 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630202220 con 0x561010989360 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63015c860 con 0x561010988a40 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010943d90 0x5610109e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010943d90 0x5610109e5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34808/0 (socket says 172.21.15.87:34808) 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 -- 172.21.15.87:0/2283143532 learned_addr learned my addr 172.21.15.87:0/2283143532 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff636800640 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010989360 0x5610109e5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 -- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 msgr2=0x5610109e57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635400640 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 -- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010989360 msgr2=0x5610109e5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010989360 0x5610109e5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 -- 172.21.15.87:0/2283143532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561010949cf0 con 0x561010943d90 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff636800640 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010989360 0x5610109e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635400640 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff635e00640 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010943d90 0x5610109e5290 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff624055b50 tx=0x7ff624016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dd92211dd5d8e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff61e000640 1 -- 172.21.15.87:0/2283143532 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62405c0b0 con 0x561010943d90 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff61e000640 1 -- 172.21.15.87:0/2283143532 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240524c0 con 0x561010943d90 2024-04-25T10:22:12.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff61e000640 1 -- 172.21.15.87:0/2283143532 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62405e020 con 0x561010943d90 2024-04-25T10:22:12.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2283143532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010943d90 msgr2=0x5610109e5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010943d90 0x5610109e5290 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff624055b50 tx=0x7ff624016040 comp rx=0 tx=0).stop 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2283143532 shutdown_connections 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010989360 0x5610109e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/2283143532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010943d90 0x5610109e5290 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2283143532 >> 172.21.15.87:0/2283143532 conn(0x56101093fd40 msgr2=0x5610109870e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2283143532 shutdown_connections 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- 172.21.15.87:0/2283143532 wait complete. 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.922+0000 7ff63bbdd000 1 -- start start 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 Processor -- start 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010943d90 0x5610109e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x56101098ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101078c1a0 con 0x561010989360 2024-04-25T10:22:12.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630156fa0 con 0x561010943d90 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561010788f30 con 0x561010988a40 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 10 client.0 _ll_register_callbacks cb 0x56101074fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x56101098ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x56101098ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34822/0 (socket says 172.21.15.87:34822) 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 -- 172.21.15.87:0/3394600908 learned_addr learned my addr 172.21.15.87:0/3394600908 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff635400640 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff635e00640 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010943d90 0x5610109e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 -- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 msgr2=0x5610109e9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 -- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010943d90 msgr2=0x5610109e96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010943d90 0x5610109e96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 -- 172.21.15.87:0/3394600908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610107e6a10 con 0x561010989360 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff636800640 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x56101098ded0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01ced0 tx=0x7ff62c01b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98b168faec3b5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:22:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 1 -- 172.21.15.87:0/3394600908 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c009af0 con 0x561010989360 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 1 -- 172.21.15.87:0/3394600908 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff62c003bb0 con 0x561010989360 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6240524c0 con 0x561010989360 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 1 -- 172.21.15.87:0/3394600908 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff62c003db0 con 0x561010989360 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 10 client.9303 fetch_fsmap learned FSMap version 89 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x561010721b00 con 0x561010989360 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 1 -- 172.21.15.87:0/3394600908 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240524c0 con 0x561010989360 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 1 -- 172.21.15.87:0/3394600908 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff62c008070 con 0x561010989360 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 10 client.9303 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 10 client.9303 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 20 client.9303 populate_metadata read hostname 'smithi087' 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101098e8c0 0x56101098ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 4 client.9303 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x56101079a3d0 con 0x56101098e8c0 2024-04-25T10:22:12.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff635e00640 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101098e8c0 0x56101098ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff635e00640 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101098e8c0 0x56101098ecd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f51678415893ff server_cookie=1a17a121a249563f in_seq=0 out_seq=0 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 10 client.9303 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 1 -- 172.21.15.87:0/3394600908 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56101079a3d0 con 0x56101098e8c0 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff613400640 10 client.9303 handle_command_reply: tid=0 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.926+0000 7ff63bbdd000 1 client.9303 shutdown 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 msgr2=0x56101098ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x56101098ded0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01ced0 tx=0x7ff62c01b230 comp rx=0 tx=0).stop 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 shutdown_connections 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x56101098e8c0 0x56101098ecd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561010989360 0x56101098ded0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561010988a40 0x5610109e9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff613400640 0 client.9303 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 --2- 172.21.15.87:0/3394600908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561010943d90 0x5610109e96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 >> 172.21.15.87:0/3394600908 conn(0x56101093fd40 msgr2=0x561010987970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 shutdown_connections 2024-04-25T10:22:12.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 1 -- 172.21.15.87:0/3394600908 wait complete. 2024-04-25T10:22:12.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff63bbdd000 20 client.9303 trim_cache size 0 max 16384 2024-04-25T10:22:12.936 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:22:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0 2024-04-25T10:22:12.936 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff614066f60 msgr2=0x7ff614069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff614066f60 0x7ff614069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff6301f9d40 tx=0x7ff620008040 comp rx=0 tx=0).stop 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 msgr2=0x7ff63014e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff63014e660 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff62400d800 tx=0x7ff62400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 shutdown_connections 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff614066f60 0x7ff614069420 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff63015ee90 0x7ff630160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff63015dbe0 0x7ff63015fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 --2- 172.21.15.87:0/3397956556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6300d0f90 0x7ff63014e660 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 >> 172.21.15.87:0/3397956556 conn(0x7ff63006b090 msgr2=0x7ff63015f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 shutdown_connections 2024-04-25T10:22:12.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:12.930+0000 7ff638800640 1 -- 172.21.15.87:0/3397956556 wait complete. 2024-04-25T10:22:12.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.950 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T10:22:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1fc00640 1 Processor -- start 2024-04-25T10:22:13.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1fc00640 1 -- start start 2024-04-25T10:22:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b18159540 0x7f8b180ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1fc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b180aebd0 0x7f8b18071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1fc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b18071360 con 0x7f8b18072020 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b18159920 con 0x7f8b18159540 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1fc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1815c2f0 con 0x7f8b180aebd0 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b18159540 0x7f8b180ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b18159540 0x7f8b180ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34830/0 (socket says 172.21.15.87:34830) 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1d200640 1 -- 172.21.15.87:0/374965842 learned_addr learned my addr 172.21.15.87:0/374965842 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1dc00640 1 --1- 172.21.15.87:0/374965842 >> v1:172.21.15.87:6789/0 conn(0x7f8b18072020 0x7f8b1815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36668/0 (socket says 172.21.15.87:36668) 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b17e00640 1 --2- 172.21.15.87:0/374965842 >> v2:172.21.15.169:3300/0 conn(0x7f8b180aebd0 0x7f8b18071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1d200640 1 -- 172.21.15.87:0/374965842 >> v2:172.21.15.169:3300/0 conn(0x7f8b180aebd0 msgr2=0x7f8b18071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1d200640 1 --2- 172.21.15.87:0/374965842 >> v2:172.21.15.169:3300/0 conn(0x7f8b180aebd0 0x7f8b18071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1d200640 1 -- 172.21.15.87:0/374965842 >> v1:172.21.15.87:6789/0 conn(0x7f8b18072020 legacy=0x7f8b1815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b1d200640 1 -- 172.21.15.87:0/374965842 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1815c5d0 con 0x7f8b18159540 2024-04-25T10:22:13.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.086+0000 7f8b17e00640 1 --2- 172.21.15.87:0/374965842 >> v2:172.21.15.169:3300/0 conn(0x7f8b180aebd0 0x7f8b18071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:13.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1d200640 1 --2- 172.21.15.87:0/374965842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b180ae690 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f8b180cd4e0 tx=0x7f8b0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2382a2df9d1c1ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b17400640 1 -- 172.21.15.87:0/374965842 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0b0070 con 0x7f8b18159540 2024-04-25T10:22:13.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b17400640 1 -- 172.21.15.87:0/374965842 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c0ab600 con 0x7f8b18159540 2024-04-25T10:22:13.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b17400640 1 -- 172.21.15.87:0/374965842 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0ab9d0 con 0x7f8b18159540 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 -- 172.21.15.87:0/374965842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 msgr2=0x7f8b180ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/374965842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b180ae690 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f8b180cd4e0 tx=0x7f8b0c01c520 comp rx=0 tx=0).stop 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 -- 172.21.15.87:0/374965842 shutdown_connections 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/374965842 >> v2:172.21.15.169:3300/0 conn(0x7f8b180aebd0 0x7f8b18071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/374965842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b180ae690 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 -- 172.21.15.87:0/374965842 >> 172.21.15.87:0/374965842 conn(0x7f8b180669a0 msgr2=0x7f8b18066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 -- 172.21.15.87:0/374965842 shutdown_connections 2024-04-25T10:22:13.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.090+0000 7f8b1fc00640 1 -- 172.21.15.87:0/374965842 wait complete. 2024-04-25T10:22:13.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 Processor -- start 2024-04-25T10:22:13.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 -- start start 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b18072020 0x7f8b181523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b180aebd0 0x7f8b1815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b1813dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b18072430 con 0x7f8b18159540 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1806c0d0 con 0x7f8b180aebd0 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b18168930 con 0x7f8b18072020 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b1813dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b1813dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34844/0 (socket says 172.21.15.87:34844) 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1dc00640 1 -- 172.21.15.87:0/2267944649 learned_addr learned my addr 172.21.15.87:0/2267944649 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b18072020 0x7f8b181523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b17e00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b180aebd0 0x7f8b1815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1d200640 1 -- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b180aebd0 msgr2=0x7f8b1815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1d200640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b180aebd0 0x7f8b1815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1d200640 1 -- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 msgr2=0x7f8b1813dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1d200640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b1813dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1d200640 1 -- 172.21.15.87:0/2267944649 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b18152ce0 con 0x7f8b18072020 2024-04-25T10:22:13.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b1813dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:13.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1d200640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b18072020 0x7f8b181523e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0ab000 tx=0x7f8b0c0192c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b14c00640 1 -- 172.21.15.87:0/2267944649 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0bb070 con 0x7f8b18072020 2024-04-25T10:22:13.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b14c00640 1 -- 172.21.15.87:0/2267944649 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c0b6630 con 0x7f8b18072020 2024-04-25T10:22:13.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b14c00640 1 -- 172.21.15.87:0/2267944649 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0b0040 con 0x7f8b18072020 2024-04-25T10:22:13.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b18152fc0 con 0x7f8b18072020 2024-04-25T10:22:13.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.094+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b18153580 con 0x7f8b18072020 2024-04-25T10:22:13.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b14c00640 1 -- 172.21.15.87:0/2267944649 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b0c0be070 con 0x7f8b18072020 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b14c00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8af8066f60 0x7f8af8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 Processor -- start 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b14c00640 1 -- 172.21.15.87:0/2267944649 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8b0c11e7a0 con 0x7f8b18072020 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 -- start start 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b17e00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8af8066f60 0x7f8af8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b17e00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8af8066f60 0x7f8af8069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b18068260 tx=0x7f8b04008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c6e30 0x55c9e29c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e297cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e297d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e27c5ff0 con 0x55c9e29c6e30 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e274e150 con 0x55c9e29c7750 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9e27c2090 con 0x55c9e297d4b0 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c6e30 0x55c9e29c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c6e30 0x55c9e29c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34848/0 (socket says 172.21.15.87:34848) 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 -- 172.21.15.87:0/429080769 learned_addr learned my addr 172.21.15.87:0/429080769 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b17e00640 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e297cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e297d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 -- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 msgr2=0x55c9e297d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e297d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 -- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 msgr2=0x55c9e297cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e297cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 -- 172.21.15.87:0/429080769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9e295c0d0 con 0x55c9e29c6e30 2024-04-25T10:22:13.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b17e00640 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e297cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b1d200640 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c6e30 0x55c9e29c7210 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0f7240 tx=0x7f8b0c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baf486bd2bbe9ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.098+0000 7f8b09600640 1 -- 172.21.15.87:0/429080769 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0f31b0 con 0x55c9e29c6e30 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b09600640 1 -- 172.21.15.87:0/429080769 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c0f7770 con 0x55c9e29c6e30 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b09600640 1 -- 172.21.15.87:0/429080769 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0ab4c0 con 0x55c9e29c6e30 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- 172.21.15.87:0/429080769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c6e30 msgr2=0x55c9e29c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c6e30 0x55c9e29c7210 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0f7240 tx=0x7f8b0c0095f0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- 172.21.15.87:0/429080769 shutdown_connections 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e297d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e297cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 --2- 172.21.15.87:0/429080769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c6e30 0x55c9e29c7210 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- 172.21.15.87:0/429080769 >> 172.21.15.87:0/429080769 conn(0x55c9e2975020 msgr2=0x55c9e2977440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- 172.21.15.87:0/429080769 shutdown_connections 2024-04-25T10:22:13.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- 172.21.15.87:0/429080769 wait complete. 2024-04-25T10:22:13.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- start start 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 Processor -- start 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e297d4b0 0x55c9e29c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e2983360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e29838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e27ca620 con 0x55c9e297d4b0 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e2780690 con 0x55c9e29c7750 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9e274ea40 con 0x55c9e29c6e30 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 10 client.0 _ll_register_callbacks cb 0x55c9e2963e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e297d4b0 0x55c9e29c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e297d4b0 0x55c9e29c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34862/0 (socket says 172.21.15.87:34862) 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b17e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e2983360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 -- 172.21.15.87:0/4144452350 learned_addr learned my addr 172.21.15.87:0/4144452350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e29838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 -- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 msgr2=0x55c9e2983360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e2983360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 -- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 msgr2=0x55c9e29838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e29838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 -- 172.21.15.87:0/4144452350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9e28731c0 con 0x55c9e297d4b0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b1d200640 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e297d4b0 0x55c9e29c6810 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c000c10 tx=0x7f8b0c0b3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c9cb6c25706b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8afe000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8afe000640 1 -- 172.21.15.87:0/4144452350 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0eb780 con 0x55c9e297d4b0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8afe000640 1 -- 172.21.15.87:0/4144452350 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8b0c0f72e0 con 0x55c9e297d4b0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b0c0f7770 con 0x55c9e297d4b0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8afe000640 1 -- 172.21.15.87:0/4144452350 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b0c0eb920 con 0x55c9e297d4b0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 10 client.9318 fetch_fsmap learned FSMap version 89 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.102+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55c9e286f170 con 0x55c9e297d4b0 2024-04-25T10:22:13.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8afe000640 1 -- 172.21.15.87:0/4144452350 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c0f7770 con 0x55c9e297d4b0 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8afe000640 1 -- 172.21.15.87:0/4144452350 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b0c0ee560 con 0x55c9e297d4b0 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 10 client.9318 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 10 client.9318 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 20 client.9318 populate_metadata read hostname 'smithi087' 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e2984640 0x55c9e29d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 4 client.9318 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9e2761c60 con 0x55c9e2984640 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b17e00640 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e2984640 0x55c9e29d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b17e00640 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e2984640 0x55c9e29d90f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162700e45cbd11e server_cookie=31ab6e1b94e69cb2 in_seq=0 out_seq=0 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8afe000640 10 client.9318 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8afe000640 1 -- 172.21.15.87:0/4144452350 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9e2761c60 con 0x55c9e2984640 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8afe000640 10 client.9318 handle_command_reply: tid=0 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 client.9318 shutdown 2024-04-25T10:22:13.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e297d4b0 msgr2=0x55c9e29c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e297d4b0 0x55c9e29c6810 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c000c10 tx=0x7f8b0c0b3000 comp rx=0 tx=0).stop 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 shutdown_connections 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e2984640 0x55c9e29d90f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c7750 0x55c9e29838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e2983360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 --2- 172.21.15.87:0/4144452350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e297d4b0 0x55c9e29c6810 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 >> 172.21.15.87:0/4144452350 conn(0x55c9e2975020 msgr2=0x55c9e297e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8afe000640 0 client.9318 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 shutdown_connections 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 1 -- 172.21.15.87:0/4144452350 wait complete. 2024-04-25T10:22:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.106+0000 7f8b23030000 20 client.9318 trim_cache size 0 max 16384 2024-04-25T10:22:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 Processor -- start 2024-04-25T10:22:13.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- start start 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e29a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c6e30 0x55c9e29db110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29db650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180b4250 con 0x55c9e29c7750 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180b3470 con 0x55c9e29c6e30 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180b2870 con 0x55c9e297d4b0 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29db650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29db650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34874/0 (socket says 172.21.15.87:34874) 2024-04-25T10:22:13.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 -- 172.21.15.87:0/4197203893 learned_addr learned my addr 172.21.15.87:0/4197203893 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b17e00640 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c6e30 0x55c9e29db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1d200640 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e29a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 -- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 msgr2=0x55c9e29a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e29a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 -- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c6e30 msgr2=0x55c9e29db110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c6e30 0x55c9e29db110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 -- 172.21.15.87:0/4197203893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9e280cde0 con 0x55c9e29c7750 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29db650 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f8b100020c0 tx=0x7f8b10001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943f9791e4f3e9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b08c00640 1 -- 172.21.15.87:0/4197203893 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1001a070 con 0x55c9e29c7750 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b08c00640 1 -- 172.21.15.87:0/4197203893 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10016420 con 0x55c9e29c7750 2024-04-25T10:22:13.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b08c00640 1 -- 172.21.15.87:0/4197203893 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b100167d0 con 0x55c9e29c7750 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- 172.21.15.87:0/4197203893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 msgr2=0x55c9e29db650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29db650 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f8b100020c0 tx=0x7f8b10001740 comp rx=0 tx=0).stop 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- 172.21.15.87:0/4197203893 shutdown_connections 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29db650 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e29c6e30 0x55c9e29db110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 --2- 172.21.15.87:0/4197203893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e297d4b0 0x55c9e29a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- 172.21.15.87:0/4197203893 >> 172.21.15.87:0/4197203893 conn(0x55c9e2975020 msgr2=0x55c9e29d3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- 172.21.15.87:0/4197203893 shutdown_connections 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.122+0000 7f8b23030000 1 -- 172.21.15.87:0/4197203893 wait complete. 2024-04-25T10:22:13.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 -- start start 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 Processor -- start 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e297d4b0 0x55c9e29d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e29d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e2760e00 con 0x55c9e29c7750 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180b2e70 con 0x55c9e297d4b0 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9e27c5300 con 0x55c9e29c6e30 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 10 client.0 _ll_register_callbacks cb 0x55c9e2944040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29d9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34884/0 (socket says 172.21.15.87:34884) 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 -- 172.21.15.87:0/1715469242 learned_addr learned my addr 172.21.15.87:0/1715469242 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1d200640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e297d4b0 0x55c9e29d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b17e00640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e29d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 -- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 msgr2=0x55c9e29d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e29d9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 -- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e297d4b0 msgr2=0x55c9e29d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e297d4b0 0x55c9e29d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 -- 172.21.15.87:0/1715469242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9e27678e0 con 0x55c9e29c7750 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b17e00640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e29d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1dc00640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29d9b70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8b10007800 tx=0x7f8b10001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9553b0592709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8afe000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8afe000640 1 -- 172.21.15.87:0/1715469242 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10026020 con 0x55c9e29c7750 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8afe000640 1 -- 172.21.15.87:0/1715469242 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8b10025850 con 0x55c9e29c7750 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b10016420 con 0x55c9e29c7750 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8afe000640 1 -- 172.21.15.87:0/1715469242 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b1001a070 con 0x55c9e29c7750 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 10 client.9324 fetch_fsmap learned FSMap version 89 2024-04-25T10:22:13.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55c9e281f8f0 con 0x55c9e29c7750 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8afe000640 1 -- 172.21.15.87:0/1715469242 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10016420 con 0x55c9e29c7750 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8afe000640 1 -- 172.21.15.87:0/1715469242 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b10025a50 con 0x55c9e29c7750 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 10 client.9324 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 10 client.9324 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 20 client.9324 populate_metadata read hostname 'smithi087' 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e29da6f0 0x55c9e2a2efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 4 client.9324 mds_command: new command op to 9114 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 --> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c9e27d3530 con 0x55c9e29da6f0 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1d200640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e29da6f0 0x55c9e2a2efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8b1d200640 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e29da6f0 0x55c9e2a2efe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900b882d0a06c34e server_cookie=44af274b7bbe4b60 in_seq=0 out_seq=0 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.126+0000 7f8afe000640 10 client.9324 ms_handle_connect on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8afe000640 1 -- 172.21.15.87:0/1715469242 <== mds.0 v2:172.21.15.169:6832/85260299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55c9e27d3530 con 0x55c9e29da6f0 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8afe000640 10 client.9324 handle_command_reply: tid=0 2024-04-25T10:22:13.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 client.9324 shutdown 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 msgr2=0x55c9e29d9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29d9b70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8b10007800 tx=0x7f8b10001260 comp rx=0 tx=0).stop 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 shutdown_connections 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:6832/85260299,v1:172.21.15.169:6834/85260299] conn(0x55c9e29da6f0 0x55c9e2a2efe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9e29c7750 0x55c9e29d9b70 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8afe000640 0 client.9324 ms_handle_reset on v2:172.21.15.169:6832/85260299 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9e29c6e30 0x55c9e29d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 --2- 172.21.15.87:0/1715469242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9e297d4b0 0x55c9e29d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 >> 172.21.15.87:0/1715469242 conn(0x55c9e2975020 msgr2=0x55c9e29d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 shutdown_connections 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 1 -- 172.21.15.87:0/1715469242 wait complete. 2024-04-25T10:22:13.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b23030000 20 client.9324 trim_cache size 0 max 16384 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "6e648a7c-db42-45b7-b2ba-2e34d6cecb87": { 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "6e648a7c-db42-45b7-b2ba-2e34d6cecb87", 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive" 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:13.137 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:22:13.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8af8066f60 msgr2=0x7f8af8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8af8066f60 0x7f8af8069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b18068260 tx=0x7f8b04008040 comp rx=0 tx=0).stop 2024-04-25T10:22:13.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b18072020 msgr2=0x7f8b181523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b18072020 0x7f8b181523e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0ab000 tx=0x7f8b0c0192c0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 shutdown_connections 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8af8066f60 0x7f8af8069420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b18159540 0x7f8b1813dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b180aebd0 0x7f8b1815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 --2- 172.21.15.87:0/2267944649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b18072020 0x7f8b181523e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 >> 172.21.15.87:0/2267944649 conn(0x7f8b180669a0 msgr2=0x7f8b1806de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 shutdown_connections 2024-04-25T10:22:13.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.130+0000 7f8b1fc00640 1 -- 172.21.15.87:0/2267944649 wait complete. 2024-04-25T10:22:13.150 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:13.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.282+0000 7f3b17e00640 1 Processor -- start 2024-04-25T10:22:13.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17e00640 1 -- start start 2024-04-25T10:22:13.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3b1815dbe0 0x7f3b180ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180b27d0 0x7f3b180b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b180cf4e0 0x7f3b180cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b180b3170 con 0x7f3b1815dbe0 2024-04-25T10:22:13.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1806c690 con 0x7f3b180cf4e0 2024-04-25T10:22:13.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b180b3fb0 con 0x7f3b180b27d0 2024-04-25T10:22:13.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3b1815dbe0 0x7f3b180ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3b1815dbe0 0x7f3b180ae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34900/0 (socket says 172.21.15.87:34900) 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 -- 172.21.15.87:0/1386677832 learned_addr learned my addr 172.21.15.87:0/1386677832 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b17400640 1 --2- 172.21.15.87:0/1386677832 >> v2:172.21.15.169:3300/0 conn(0x7f3b180cf4e0 0x7f3b180cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b0fe00640 1 --2- 172.21.15.87:0/1386677832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180b27d0 0x7f3b180b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 -- 172.21.15.87:0/1386677832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180b27d0 msgr2=0x7f3b180b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 --2- 172.21.15.87:0/1386677832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180b27d0 0x7f3b180b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 -- 172.21.15.87:0/1386677832 >> v2:172.21.15.169:3300/0 conn(0x7f3b180cf4e0 msgr2=0x7f3b180cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 --2- 172.21.15.87:0/1386677832 >> v2:172.21.15.169:3300/0 conn(0x7f3b180cf4e0 0x7f3b180cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.286+0000 7f3b16a00640 1 -- 172.21.15.87:0/1386677832 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b180b4250 con 0x7f3b1815dbe0 2024-04-25T10:22:13.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b16a00640 1 --2- 172.21.15.87:0/1386677832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1815dbe0 0x7f3b180ae250 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f3b180aec20 tx=0x7f3b0801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4765c35fb7cae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b16000640 1 -- 172.21.15.87:0/1386677832 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080b0070 con 0x7f3b1815dbe0 2024-04-25T10:22:13.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b16000640 1 -- 172.21.15.87:0/1386677832 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b080ab600 con 0x7f3b1815dbe0 2024-04-25T10:22:13.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b16000640 1 -- 172.21.15.87:0/1386677832 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080ab9d0 con 0x7f3b1815dbe0 2024-04-25T10:22:13.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 -- 172.21.15.87:0/1386677832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1815dbe0 msgr2=0x7f3b180ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1386677832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1815dbe0 0x7f3b180ae250 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f3b180aec20 tx=0x7f3b0801c490 comp rx=0 tx=0).stop 2024-04-25T10:22:13.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 -- 172.21.15.87:0/1386677832 shutdown_connections 2024-04-25T10:22:13.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1386677832 >> v2:172.21.15.169:3300/0 conn(0x7f3b180cf4e0 0x7f3b180cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1386677832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180b27d0 0x7f3b180b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1386677832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1815dbe0 0x7f3b180ae250 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 -- 172.21.15.87:0/1386677832 >> 172.21.15.87:0/1386677832 conn(0x7f3b180743f0 msgr2=0x7f3b180669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 -- 172.21.15.87:0/1386677832 shutdown_connections 2024-04-25T10:22:13.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 -- 172.21.15.87:0/1386677832 wait complete. 2024-04-25T10:22:13.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 Processor -- start 2024-04-25T10:22:13.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.290+0000 7f3b17e00640 1 -- start start 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b180b27d0 0x7f3b1814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180cf4e0 0x7f3b180cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1815dbe0 0x7f3b18070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b181616a0 con 0x7f3b180b27d0 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b180720a0 con 0x7f3b1815dbe0 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18161820 con 0x7f3b180cf4e0 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b180b27d0 0x7f3b1814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b180b27d0 0x7f3b1814d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34906/0 (socket says 172.21.15.87:34906) 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 -- 172.21.15.87:0/1588779058 learned_addr learned my addr 172.21.15.87:0/1588779058 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b0fe00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180cf4e0 0x7f3b180cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17400640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1815dbe0 0x7f3b18070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 -- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180cf4e0 msgr2=0x7f3b180cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180cf4e0 0x7f3b180cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 -- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1815dbe0 msgr2=0x7f3b18070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1815dbe0 0x7f3b18070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 -- 172.21.15.87:0/1588779058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b180ccd00 con 0x7f3b180b27d0 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b16a00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b180b27d0 0x7f3b1814d090 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3b0801ab40 tx=0x7f3b0801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b0ea00640 1 -- 172.21.15.87:0/1588779058 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080ba020 con 0x7f3b180b27d0 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b0ea00640 1 -- 172.21.15.87:0/1588779058 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b080b6420 con 0x7f3b180b27d0 2024-04-25T10:22:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b180ccfe0 con 0x7f3b180b27d0 2024-04-25T10:22:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b0ea00640 1 -- 172.21.15.87:0/1588779058 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080b0040 con 0x7f3b180b27d0 2024-04-25T10:22:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b180cd5a0 con 0x7f3b180b27d0 2024-04-25T10:22:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b18075120 con 0x7f3b180b27d0 2024-04-25T10:22:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.294+0000 7f3b0ea00640 1 -- 172.21.15.87:0/1588779058 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b080bd020 con 0x7f3b180b27d0 2024-04-25T10:22:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.298+0000 7f3b0ea00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3af4066f60 0x7f3af4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.298+0000 7f3b0ea00640 1 -- 172.21.15.87:0/1588779058 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3b0811ef70 con 0x7f3b180b27d0 2024-04-25T10:22:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.298+0000 7f3b0fe00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3af4066f60 0x7f3af4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.298+0000 7f3b0fe00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3af4066f60 0x7f3af4069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3b181fa080 tx=0x7f3b00009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.302+0000 7f3b0ea00640 1 -- 172.21.15.87:0/1588779058 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b080c4030 con 0x7f3b180b27d0 2024-04-25T10:22:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.490+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b18070540 con 0x7f3b180b27d0 2024-04-25T10:22:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.490+0000 7f3b0ea00640 1 -- 172.21.15.87:0/1588779058 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f3b08016110 con 0x7f3b180b27d0 2024-04-25T10:22:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:13.495 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3af4066f60 msgr2=0x7f3af4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3af4066f60 0x7f3af4069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3b181fa080 tx=0x7f3b00009190 comp rx=0 tx=0).stop 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b180b27d0 msgr2=0x7f3b1814d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b180b27d0 0x7f3b1814d090 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3b0801ab40 tx=0x7f3b0801acd0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 shutdown_connections 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3af4066f60 0x7f3af4069420 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1815dbe0 0x7f3b18070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b180cf4e0 0x7f3b180cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 --2- 172.21.15.87:0/1588779058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b180b27d0 0x7f3b1814d090 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 >> 172.21.15.87:0/1588779058 conn(0x7f3b180743f0 msgr2=0x7f3b18160130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 shutdown_connections 2024-04-25T10:22:13.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.494+0000 7f3b17e00640 1 -- 172.21.15.87:0/1588779058 wait complete. 2024-04-25T10:22:13.515 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:22:13.515 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:22:13.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.646+0000 7f2ea2000640 1 Processor -- start 2024-04-25T10:22:13.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2ea2000640 1 -- start start 2024-04-25T10:22:13.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2ea2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e9c0d1820 0x7f2e9c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2ea2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c15e050 0x7f2e9c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2ea2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e9c0b13e0 0x7f2e9c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2ea2000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b1870 con 0x7f2e9c0d1820 2024-04-25T10:22:13.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2ea2000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b1af0 con 0x7f2e9c0b13e0 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2ea2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b1dd0 con 0x7f2e9c15e050 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e9c0d1820 0x7f2e9c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e9c0d1820 0x7f2e9c0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34910/0 (socket says 172.21.15.87:34910) 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 -- 172.21.15.87:0/52168 learned_addr learned my addr 172.21.15.87:0/52168 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9be00640 1 --2- 172.21.15.87:0/52168 >> v2:172.21.15.169:3300/0 conn(0x7f2e9c0b13e0 0x7f2e9c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e92a00640 1 --2- 172.21.15.87:0/52168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c15e050 0x7f2e9c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 -- 172.21.15.87:0/52168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c15e050 msgr2=0x7f2e9c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 --2- 172.21.15.87:0/52168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c15e050 0x7f2e9c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 -- 172.21.15.87:0/52168 >> v2:172.21.15.169:3300/0 conn(0x7f2e9c0b13e0 msgr2=0x7f2e9c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 --2- 172.21.15.87:0/52168 >> v2:172.21.15.169:3300/0 conn(0x7f2e9c0b13e0 0x7f2e9c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e9b400640 1 -- 172.21.15.87:0/52168 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0b20b0 con 0x7f2e9c0d1820 2024-04-25T10:22:13.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.650+0000 7f2e92a00640 1 --2- 172.21.15.87:0/52168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c15e050 0x7f2e9c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:13.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2e9b400640 1 --2- 172.21.15.87:0/52168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c0d1820 0x7f2e9c0d1c00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c06ce10 tx=0x7f2e8c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e28ddb4016ccba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2e9aa00640 1 -- 172.21.15.87:0/52168 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0a55b0 con 0x7f2e9c0d1820 2024-04-25T10:22:13.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2e9aa00640 1 -- 172.21.15.87:0/52168 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c0a5750 con 0x7f2e9c0d1820 2024-04-25T10:22:13.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2e9aa00640 1 -- 172.21.15.87:0/52168 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0a5b20 con 0x7f2e9c0d1820 2024-04-25T10:22:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 -- 172.21.15.87:0/52168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c0d1820 msgr2=0x7f2e9c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 --2- 172.21.15.87:0/52168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c0d1820 0x7f2e9c0d1c00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c06ce10 tx=0x7f2e8c01c730 comp rx=0 tx=0).stop 2024-04-25T10:22:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 -- 172.21.15.87:0/52168 shutdown_connections 2024-04-25T10:22:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 --2- 172.21.15.87:0/52168 >> v2:172.21.15.169:3300/0 conn(0x7f2e9c0b13e0 0x7f2e9c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 --2- 172.21.15.87:0/52168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c15e050 0x7f2e9c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 --2- 172.21.15.87:0/52168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c0d1820 0x7f2e9c0d1c00 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 -- 172.21.15.87:0/52168 >> 172.21.15.87:0/52168 conn(0x7f2e9c09a9d0 msgr2=0x7f2e9c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 -- 172.21.15.87:0/52168 shutdown_connections 2024-04-25T10:22:13.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 -- 172.21.15.87:0/52168 wait complete. 2024-04-25T10:22:13.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 Processor -- start 2024-04-25T10:22:13.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.654+0000 7f2ea2000640 1 -- start start 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e9c0b13e0 0x7f2e9c14f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c0d1820 0x7f2e9c14da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c15e050 0x7f2e9c155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cfdf0 con 0x7f2e9c15e050 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cfc70 con 0x7f2e9c0b13e0 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cff70 con 0x7f2e9c0d1820 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c15e050 0x7f2e9c155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c15e050 0x7f2e9c155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34922/0 (socket says 172.21.15.87:34922) 2024-04-25T10:22:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 -- 172.21.15.87:0/3474629352 learned_addr learned my addr 172.21.15.87:0/3474629352 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9b400640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e9c0b13e0 0x7f2e9c14f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e92a00640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c0d1820 0x7f2e9c14da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 -- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c0d1820 msgr2=0x7f2e9c14da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c0d1820 0x7f2e9c14da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 -- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e9c0b13e0 msgr2=0x7f2e9c14f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e9c0b13e0 0x7f2e9c14f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 -- 172.21.15.87:0/3474629352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c14df60 con 0x7f2e9c15e050 2024-04-25T10:22:13.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e9be00640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c15e050 0x7f2e9c155ab0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400d800 tx=0x7f2e9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e93e00640 1 -- 172.21.15.87:0/3474629352 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400b6c0 con 0x7f2e9c15e050 2024-04-25T10:22:13.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e93e00640 1 -- 172.21.15.87:0/3474629352 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e94019070 con 0x7f2e9c15e050 2024-04-25T10:22:13.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c14e340 con 0x7f2e9c15e050 2024-04-25T10:22:13.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e93e00640 1 -- 172.21.15.87:0/3474629352 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400baa0 con 0x7f2e9c15e050 2024-04-25T10:22:13.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c14e900 con 0x7f2e9c15e050 2024-04-25T10:22:13.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c04ef30 con 0x7f2e9c15e050 2024-04-25T10:22:13.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.658+0000 7f2e93e00640 1 -- 172.21.15.87:0/3474629352 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e9400bc40 con 0x7f2e9c15e050 2024-04-25T10:22:13.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.662+0000 7f2e93e00640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.662+0000 7f2e93e00640 1 -- 172.21.15.87:0/3474629352 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2e94085420 con 0x7f2e9c15e050 2024-04-25T10:22:13.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.662+0000 7f2e9b400640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.662+0000 7f2e9b400640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e78066f60 0x7f2e78069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0ae000 tx=0x7f2e8c0a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.666+0000 7f2e93e00640 1 -- 172.21.15.87:0/3474629352 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e94055d00 con 0x7f2e9c15e050 2024-04-25T10:22:13.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2e9c0d1600 con 0x7f2e9c15e050 2024-04-25T10:22:13.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2e93e00640 1 -- 172.21.15.87:0/3474629352 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1362 (secure 0 0 0) 0x7f2e94059da0 con 0x7f2e9c15e050 2024-04-25T10:22:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:13.877 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":615,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":126,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":984,"data_bytes":1009123379,"bytes_used":2455371776,"bytes_avail":770638741504,"bytes_total":773094113280,"recovering_objects_per_sec":3,"recovering_bytes_per_sec":3407241,"recovering_keys_per_sec":0,"num_objects_recovered":26,"num_bytes_recovered":27262976,"num_keys_recovered":0,"write_bytes_sec":59372825,"read_op_per_sec":0,"write_op_per_sec":59},"fsmap":{"epoch":89,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9114}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T10:21:37.241801+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e78066f60 msgr2=0x7f2e78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e78066f60 0x7f2e78069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0ae000 tx=0x7f2e8c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c15e050 msgr2=0x7f2e9c155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c15e050 0x7f2e9c155ab0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400d800 tx=0x7f2e9400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 shutdown_connections 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e9c15e050 0x7f2e9c155ab0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e9c0d1820 0x7f2e9c14da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 --2- 172.21.15.87:0/3474629352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e9c0b13e0 0x7f2e9c14f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 >> 172.21.15.87:0/3474629352 conn(0x7f2e9c09a9d0 msgr2=0x7f2e9c157580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 shutdown_connections 2024-04-25T10:22:13.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:13.874+0000 7f2ea2000640 1 -- 172.21.15.87:0/3474629352 wait complete. 2024-04-25T10:22:13.897 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:22:14.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:15.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9f400640 1 Processor -- start 2024-04-25T10:22:15.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9f400640 1 -- start start 2024-04-25T10:22:15.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9815bc40 0x7faf9806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faf980cf930 0x7faf980cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf980b2770 con 0x7faf980cda10 2024-04-25T10:22:15.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9f400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9806c670 con 0x7faf980cf930 2024-04-25T10:22:15.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9806c950 con 0x7faf9815bc40 2024-04-25T10:22:15.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faf980cf930 0x7faf980cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faf980cf930 0x7faf980cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34938/0 (socket says 172.21.15.87:34938) 2024-04-25T10:22:15.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9d400640 1 -- 172.21.15.87:0/2168740558 learned_addr learned my addr 172.21.15.87:0/2168740558 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:15.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9ca00640 1 --2- 172.21.15.87:0/2168740558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9815bc40 0x7faf9806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9d400640 1 -- 172.21.15.87:0/2168740558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9815bc40 msgr2=0x7faf9806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9d400640 1 --2- 172.21.15.87:0/2168740558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9815bc40 0x7faf9806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9d400640 1 -- 172.21.15.87:0/2168740558 >> v1:172.21.15.169:6789/0 conn(0x7faf980cda10 legacy=0x7faf980b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.034+0000 7faf9d400640 1 -- 172.21.15.87:0/2168740558 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9806cc30 con 0x7faf980cf930 2024-04-25T10:22:15.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9d400640 1 --2- 172.21.15.87:0/2168740558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf980cf930 0x7faf980cfde0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faf90009870 tx=0x7faf9001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e533f7e6d8e4595e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf97400640 1 -- 172.21.15.87:0/2168740558 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900b0070 con 0x7faf980cf930 2024-04-25T10:22:15.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf97400640 1 -- 172.21.15.87:0/2168740558 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf900ab600 con 0x7faf980cf930 2024-04-25T10:22:15.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf97400640 1 -- 172.21.15.87:0/2168740558 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900ab9d0 con 0x7faf980cf930 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 -- 172.21.15.87:0/2168740558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf980cf930 msgr2=0x7faf980cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 --2- 172.21.15.87:0/2168740558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf980cf930 0x7faf980cfde0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faf90009870 tx=0x7faf9001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 -- 172.21.15.87:0/2168740558 shutdown_connections 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 --2- 172.21.15.87:0/2168740558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf980cf930 0x7faf980cfde0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 --2- 172.21.15.87:0/2168740558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9815bc40 0x7faf9806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 -- 172.21.15.87:0/2168740558 >> 172.21.15.87:0/2168740558 conn(0x7faf98066b70 msgr2=0x7faf98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 -- 172.21.15.87:0/2168740558 shutdown_connections 2024-04-25T10:22:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.038+0000 7faf9f400640 1 -- 172.21.15.87:0/2168740558 wait complete. 2024-04-25T10:22:15.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 Processor -- start 2024-04-25T10:22:15.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 -- start start 2024-04-25T10:22:15.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf980cda10 0x7faf9815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf980cf930 0x7faf9814e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf9815bc40 0x7faf98154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b3850 con 0x7faf9815bc40 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b36d0 con 0x7faf980cda10 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b39d0 con 0x7faf980cf930 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf9815bc40 0x7faf98154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf9815bc40 0x7faf98154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34948/0 (socket says 172.21.15.87:34948) 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 -- 172.21.15.87:0/4056482245 learned_addr learned my addr 172.21.15.87:0/4056482245 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf97e00640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf980cf930 0x7faf9814e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9ca00640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf980cda10 0x7faf9815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 -- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf980cf930 msgr2=0x7faf9814e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf980cf930 0x7faf9814e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 -- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf980cda10 msgr2=0x7faf9815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf980cda10 0x7faf9815ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 -- 172.21.15.87:0/4056482245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9815b140 con 0x7faf9815bc40 2024-04-25T10:22:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9d400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf9815bc40 0x7faf98154e50 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7faf900ae040 tx=0x7faf90019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf94c00640 1 -- 172.21.15.87:0/4056482245 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900b0040 con 0x7faf9815bc40 2024-04-25T10:22:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf980aa000 con 0x7faf9815bc40 2024-04-25T10:22:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf94c00640 1 -- 172.21.15.87:0/4056482245 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf900ba070 con 0x7faf9815bc40 2024-04-25T10:22:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf94c00640 1 -- 172.21.15.87:0/4056482245 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900b4850 con 0x7faf9815bc40 2024-04-25T10:22:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.042+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf980aa5c0 con 0x7faf9815bc40 2024-04-25T10:22:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.046+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9804ef30 con 0x7faf9815bc40 2024-04-25T10:22:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.046+0000 7faf94c00640 1 -- 172.21.15.87:0/4056482245 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf900b49f0 con 0x7faf9815bc40 2024-04-25T10:22:15.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.046+0000 7faf94c00640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f60 0x7faf78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.046+0000 7faf94c00640 1 -- 172.21.15.87:0/4056482245 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faf9011e700 con 0x7faf9815bc40 2024-04-25T10:22:15.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.046+0000 7faf9ca00640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f60 0x7faf78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.046+0000 7faf9ca00640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f60 0x7faf78069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faf88003ab0 tx=0x7faf88009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.050+0000 7faf94c00640 1 -- 172.21.15.87:0/4056482245 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf900c3020 con 0x7faf9815bc40 2024-04-25T10:22:15.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf980af1c0 con 0x7faf9815bc40 2024-04-25T10:22:15.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf94c00640 1 -- 172.21.15.87:0/4056482245 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7faf90016150 con 0x7faf9815bc40 2024-04-25T10:22:15.245 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:15.245 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:15.245 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f60 msgr2=0x7faf78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f60 0x7faf78069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faf88003ab0 tx=0x7faf88009830 comp rx=0 tx=0).stop 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf9815bc40 msgr2=0x7faf98154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf9815bc40 0x7faf98154e50 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7faf900ae040 tx=0x7faf90019270 comp rx=0 tx=0).stop 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 shutdown_connections 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f60 0x7faf78069420 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf9815bc40 0x7faf98154e50 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf980cf930 0x7faf9814e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 --2- 172.21.15.87:0/4056482245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf980cda10 0x7faf9815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 >> 172.21.15.87:0/4056482245 conn(0x7faf98066b70 msgr2=0x7faf98155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 shutdown_connections 2024-04-25T10:22:15.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.242+0000 7faf9f400640 1 -- 172.21.15.87:0/4056482245 wait complete. 2024-04-25T10:22:15.265 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:22:15.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:22:15.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.398+0000 7fb4d3400640 1 Processor -- start 2024-04-25T10:22:15.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.398+0000 7fb4d3400640 1 -- start start 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cfd90 0x7fb4cc0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb4cc0cd110 0x7fb4cc0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d3400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc15f3b0 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d3400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc15ef20 con 0x7fb4cc0cd110 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d3400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0ca810 con 0x7fb4cc0cfd90 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d0a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb4cc0b0f70 0x7fb4cc0b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36682/0 (socket says 172.21.15.87:36682) 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb4cc0cd110 0x7fb4cc0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d0a00640 1 -- 172.21.15.87:0/895570641 learned_addr learned my addr 172.21.15.87:0/895570641 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:15.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4c3e00640 1 --2- 172.21.15.87:0/895570641 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cfd90 0x7fb4cc0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d1400640 1 -- 172.21.15.87:0/895570641 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cfd90 msgr2=0x7fb4cc0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d1400640 1 --2- 172.21.15.87:0/895570641 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cfd90 0x7fb4cc0ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d1400640 1 -- 172.21.15.87:0/895570641 >> v1:172.21.15.87:6789/0 conn(0x7fb4cc0b0f70 legacy=0x7fb4cc0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4d1400640 1 -- 172.21.15.87:0/895570641 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc15e5f0 con 0x7fb4cc0cd110 2024-04-25T10:22:15.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4cbe00640 1 -- 172.21.15.87:0/895570641 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935190766 0 0) 0x7fb4cc15f3b0 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.402+0000 7fb4c3e00640 1 --2- 172.21.15.87:0/895570641 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cfd90 0x7fb4cc0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:15.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d1400640 1 --2- 172.21.15.87:0/895570641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0cd110 0x7fb4cc0cba20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc071c10 tx=0x7fb4c401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a054a473a900fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4cbe00640 1 -- 172.21.15.87:0/895570641 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40b2070 con 0x7fb4cc0cd110 2024-04-25T10:22:15.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4cbe00640 1 -- 172.21.15.87:0/895570641 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c40a57f0 con 0x7fb4cc0cd110 2024-04-25T10:22:15.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4cbe00640 1 -- 172.21.15.87:0/895570641 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40a5bc0 con 0x7fb4cc0cd110 2024-04-25T10:22:15.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- 172.21.15.87:0/895570641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0cd110 msgr2=0x7fb4cc0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 --2- 172.21.15.87:0/895570641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0cd110 0x7fb4cc0cba20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc071c10 tx=0x7fb4c401c930 comp rx=0 tx=0).stop 2024-04-25T10:22:15.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- 172.21.15.87:0/895570641 shutdown_connections 2024-04-25T10:22:15.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 --2- 172.21.15.87:0/895570641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0cd110 0x7fb4cc0cba20 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 --2- 172.21.15.87:0/895570641 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cfd90 0x7fb4cc0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- 172.21.15.87:0/895570641 >> 172.21.15.87:0/895570641 conn(0x7fb4cc0b1d50 msgr2=0x7fb4cc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- 172.21.15.87:0/895570641 shutdown_connections 2024-04-25T10:22:15.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- 172.21.15.87:0/895570641 wait complete. 2024-04-25T10:22:15.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 Processor -- start 2024-04-25T10:22:15.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- start start 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0b0f70 0x7fb4cc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd110 0x7fb4cc0af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0cfd90 0x7fb4cc0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc161b50 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1619d0 con 0x7fb4cc0cfd90 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d3400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc161cd0 con 0x7fb4cc0cd110 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0b0f70 0x7fb4cc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0b0f70 0x7fb4cc155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34954/0 (socket says 172.21.15.87:34954) 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.406+0000 7fb4d0a00640 1 -- 172.21.15.87:0/3069954341 learned_addr learned my addr 172.21.15.87:0/3069954341 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d1400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0cfd90 0x7fb4cc0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c3e00640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd110 0x7fb4cc0af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d0a00640 1 -- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd110 msgr2=0x7fb4cc0af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d0a00640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd110 0x7fb4cc0af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d0a00640 1 -- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0cfd90 msgr2=0x7fb4cc0afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d0a00640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0cfd90 0x7fb4cc0afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d0a00640 1 -- 172.21.15.87:0/3069954341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0ad970 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d1400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0cfd90 0x7fb4cc0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d0a00640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0b0f70 0x7fb4cc155f70 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0011f0 tx=0x7fb4bc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c9600640 1 -- 172.21.15.87:0/3069954341 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc018070 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c9600640 1 -- 172.21.15.87:0/3069954341 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc0074e0 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c9600640 1 -- 172.21.15.87:0/3069954341 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc004a10 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc0adc50 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc158130 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc04ef30 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c9600640 1 -- 172.21.15.87:0/3069954341 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4bc004bb0 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c9600640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4a8066f60 0x7fb4a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c9600640 1 -- 172.21.15.87:0/3069954341 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb4bc084450 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.410+0000 7fb4c3e00640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4a8066f60 0x7fb4a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.414+0000 7fb4c3e00640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4a8066f60 0x7fb4a8069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006fd0 tx=0x7fb4b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.418+0000 7fb4c9600640 1 -- 172.21.15.87:0/3069954341 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4bc055410 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.622+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb4cc068180 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.622+0000 7fb4c9600640 1 -- 172.21.15.87:0/3069954341 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1805 (secure 0 0 0) 0x7fb4bc054d10 con 0x7fb4cc0b0f70 2024-04-25T10:22:15.626 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:15.626 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":617,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":126,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":984,"data_bytes":1009123379,"bytes_used":2455371776,"bytes_avail":770638741504,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":3038225,"recovering_keys_per_sec":0,"num_objects_recovered":26,"num_bytes_recovered":27262976,"num_keys_recovered":0,"write_bytes_sec":23428365,"read_op_per_sec":0,"write_op_per_sec":23},"fsmap":{"epoch":89,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9114}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T10:22:15.249031+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4a8066f60 msgr2=0x7fb4a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4a8066f60 0x7fb4a8069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006fd0 tx=0x7fb4b4008040 comp rx=0 tx=0).stop 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0b0f70 msgr2=0x7fb4cc155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0b0f70 0x7fb4cc155f70 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0011f0 tx=0x7fb4bc00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 shutdown_connections 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb4a8066f60 0x7fb4a8069420 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc0cfd90 0x7fb4cc0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd110 0x7fb4cc0af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 --2- 172.21.15.87:0/3069954341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4cc0b0f70 0x7fb4cc155f70 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 >> 172.21.15.87:0/3069954341 conn(0x7fb4cc0b1d50 msgr2=0x7fb4cc0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 shutdown_connections 2024-04-25T10:22:15.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.626+0000 7fb4d3400640 1 -- 172.21.15.87:0/3069954341 wait complete. 2024-04-25T10:22:15.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:22:15.647 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:22:15.647 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:15.666 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:22:15.666 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'c', 'a', 'b'} 2024-04-25T10:22:15.666 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T10:22:15.812 INFO:teuthology.orchestra.run.smithi087.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T10:22:15.812 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T10:22:15.813 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:15.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.942+0000 7efe39400640 1 Processor -- start 2024-04-25T10:22:15.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe39400640 1 -- start start 2024-04-25T10:22:15.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe39400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 0x7efe340cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe39400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe340ccfb0 con 0x7efe340d1c30 2024-04-25T10:22:15.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe39400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe340cd290 con 0x7efe3415bdc0 2024-04-25T10:22:15.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe39400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe340cd570 con 0x7efe3415b9e0 2024-04-25T10:22:15.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe33200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efe340d1c30 0x7efe340c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36684/0 (socket says 172.21.15.87:36684) 2024-04-25T10:22:15.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe33200640 1 -- 172.21.15.87:0/1709755755 learned_addr learned my addr 172.21.15.87:0/1709755755 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:15.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe32800640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 0x7efe340cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693429341 0 0) 0x7efe340ccfb0 con 0x7efe340d1c30 2024-04-25T10:22:15.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe18003530 con 0x7efe340d1c30 2024-04-25T10:22:15.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129449183 0 0) 0x7efe340cd290 con 0x7efe3415bdc0 2024-04-25T10:22:15.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe340ccfb0 con 0x7efe3415bdc0 2024-04-25T10:22:15.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1256885052 0 0) 0x7efe18003530 con 0x7efe340d1c30 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 msgr2=0x7efe340cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 0x7efe340cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 >> v1:172.21.15.169:6789/0 conn(0x7efe3415bdc0 legacy=0x7efe340d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340cd850 con 0x7efe340d1c30 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7efe2c001680 con 0x7efe340d1c30 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 >> v1:172.21.15.87:6789/0 conn(0x7efe340d1c30 legacy=0x7efe340c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe18003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe18004810 0x7efe18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 0x7efe180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340cd570 con 0x7efe18003b80 2024-04-25T10:22:15.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1800ba80 con 0x7efe18004810 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1800bc50 con 0x7efe3415b9e0 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe32800640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe18003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe31e00640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe18004810 0x7efe18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe33200640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 0x7efe180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe32800640 1 -- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 msgr2=0x7efe180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe32800640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415b9e0 0x7efe180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe32800640 1 -- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe18004810 msgr2=0x7efe18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe32800640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe18004810 0x7efe18004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.946+0000 7efe32800640 1 -- 172.21.15.87:0/1709755755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18003530 con 0x7efe18003b80 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe32800640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe18003f80 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7efe24009fc0 tx=0x7efe2401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c9cd8e4485c128 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240b2070 con 0x7efe18003b80 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe240a5a10 con 0x7efe18003b80 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe31400640 1 -- 172.21.15.87:0/1709755755 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24009c10 con 0x7efe18003b80 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe39400640 1 -- 172.21.15.87:0/1709755755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 msgr2=0x7efe18003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe39400640 1 --2- 172.21.15.87:0/1709755755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe18003f80 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7efe24009fc0 tx=0x7efe2401cc20 comp rx=0 tx=0).stop 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe33200640 1 -- 172.21.15.87:0/1709755755 reap_dead start 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe39400640 1 -- 172.21.15.87:0/1709755755 shutdown_connections 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe39400640 1 -- 172.21.15.87:0/1709755755 >> 172.21.15.87:0/1709755755 conn(0x7efe3409a9d0 msgr2=0x7efe3409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe39400640 1 -- 172.21.15.87:0/1709755755 shutdown_connections 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.950+0000 7efe39400640 1 -- 172.21.15.87:0/1709755755 wait complete. 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 Processor -- start 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 -- start start 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe341f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415c2e0 0x7efe340b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe340b6620 0x7efe340b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340c6570 con 0x7efe18003b80 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340c63f0 con 0x7efe340b6620 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe340c66f0 con 0x7efe3415c2e0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe341f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe341f6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34974/0 (socket says 172.21.15.87:34974) 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 -- 172.21.15.87:0/3479732364 learned_addr learned my addr 172.21.15.87:0/3479732364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe31e00640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415c2e0 0x7efe340b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe33200640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe340b6620 0x7efe340b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 -- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415c2e0 msgr2=0x7efe340b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415c2e0 0x7efe340b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 -- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe340b6620 msgr2=0x7efe340b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe340b6620 0x7efe340b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 -- 172.21.15.87:0/3479732364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340b5640 con 0x7efe18003b80 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe32800640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe341f6850 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7efe24098530 tx=0x7efe24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe2aa00640 1 -- 172.21.15.87:0/3479732364 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240b2040 con 0x7efe18003b80 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe2aa00640 1 -- 172.21.15.87:0/3479732364 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe240a5810 con 0x7efe18003b80 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe2aa00640 1 -- 172.21.15.87:0/3479732364 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24003be0 con 0x7efe18003b80 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe340cb870 con 0x7efe18003b80 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3415d060 con 0x7efe18003b80 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.954+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3406ce20 con 0x7efe18003b80 2024-04-25T10:22:15.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.958+0000 7efe2aa00640 1 -- 172.21.15.87:0/3479732364 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe240bf030 con 0x7efe18003b80 2024-04-25T10:22:15.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.958+0000 7efe2aa00640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe14066f90 0x7efe14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.958+0000 7efe2aa00640 1 -- 172.21.15.87:0/3479732364 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efe2411f0b0 con 0x7efe18003b80 2024-04-25T10:22:15.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.958+0000 7efe31e00640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe14066f90 0x7efe14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.958+0000 7efe31e00640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe14066f90 0x7efe14069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7efe3415ca50 tx=0x7efe20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:15.962+0000 7efe2aa00640 1 -- 172.21.15.87:0/3479732364 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe240c6020 con 0x7efe18003b80 2024-04-25T10:22:16.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.146+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe340cdd70 con 0x7efe18003b80 2024-04-25T10:22:16.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.146+0000 7efe2aa00640 1 -- 172.21.15.87:0/3479732364 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7efe24016970 con 0x7efe18003b80 2024-04-25T10:22:16.150 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:16.150 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:16.151 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe14066f90 msgr2=0x7efe14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe14066f90 0x7efe14069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7efe3415ca50 tx=0x7efe20009040 comp rx=0 tx=0).stop 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 msgr2=0x7efe341f6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe341f6850 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7efe24098530 tx=0x7efe24009750 comp rx=0 tx=0).stop 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 shutdown_connections 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe14066f90 0x7efe14069450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe340b6620 0x7efe340b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe3415c2e0 0x7efe340b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 --2- 172.21.15.87:0/3479732364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe18003b80 0x7efe341f6850 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 >> 172.21.15.87:0/3479732364 conn(0x7efe3409a9d0 msgr2=0x7efe3415f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 shutdown_connections 2024-04-25T10:22:16.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:16.150+0000 7efe39400640 1 -- 172.21.15.87:0/3479732364 wait complete. 2024-04-25T10:22:16.171 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:22:21.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.314+0000 7fe801a00640 1 Processor -- start 2024-04-25T10:22:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.314+0000 7fe801a00640 1 -- start start 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe801a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe7fc071260 0x7fe7fc0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe801a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7fc071be0 0x7fe7fc0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe801a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc06de60 con 0x7fe7fc15a780 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe801a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc06cdc0 con 0x7fe7fc071260 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe801a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc06d0a0 con 0x7fe7fc071be0 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe7f3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe7fc071260 0x7fe7fc0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe7f3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe7fc071260 0x7fe7fc0716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35190/0 (socket says 172.21.15.87:35190) 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe7f3400640 1 -- 172.21.15.87:0/3850987204 learned_addr learned my addr 172.21.15.87:0/3850987204 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe801000640 1 --2- 172.21.15.87:0/3850987204 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc071be0 0x7fe7fc0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe7f3400640 1 -- 172.21.15.87:0/3850987204 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc071be0 msgr2=0x7fe7fc0b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe7f3400640 1 --2- 172.21.15.87:0/3850987204 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc071be0 0x7fe7fc0b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe7f3400640 1 -- 172.21.15.87:0/3850987204 >> v1:172.21.15.169:6789/0 conn(0x7fe7fc15a780 legacy=0x7fe7fc15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.318+0000 7fe7f3400640 1 -- 172.21.15.87:0/3850987204 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc06d380 con 0x7fe7fc071260 2024-04-25T10:22:21.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe7f3400640 1 --2- 172.21.15.87:0/3850987204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc071260 0x7fe7fc0716a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc067940 tx=0x7fe7e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1726ed521cb1e91a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe7fb400640 1 -- 172.21.15.87:0/3850987204 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40b0070 con 0x7fe7fc071260 2024-04-25T10:22:21.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe7fb400640 1 -- 172.21.15.87:0/3850987204 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e40ab600 con 0x7fe7fc071260 2024-04-25T10:22:21.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe7fb400640 1 -- 172.21.15.87:0/3850987204 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40ab9d0 con 0x7fe7fc071260 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 -- 172.21.15.87:0/3850987204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc071260 msgr2=0x7fe7fc0716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 --2- 172.21.15.87:0/3850987204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc071260 0x7fe7fc0716a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc067940 tx=0x7fe7e401c520 comp rx=0 tx=0).stop 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 -- 172.21.15.87:0/3850987204 shutdown_connections 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 --2- 172.21.15.87:0/3850987204 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc071be0 0x7fe7fc0b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 --2- 172.21.15.87:0/3850987204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc071260 0x7fe7fc0716a0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 -- 172.21.15.87:0/3850987204 >> 172.21.15.87:0/3850987204 conn(0x7fe7fc0743f0 msgr2=0x7fe7fc0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 -- 172.21.15.87:0/3850987204 shutdown_connections 2024-04-25T10:22:21.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 -- 172.21.15.87:0/3850987204 wait complete. 2024-04-25T10:22:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 Processor -- start 2024-04-25T10:22:21.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.322+0000 7fe801a00640 1 -- start start 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc071260 0x7fe7fc13dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc071be0 0x7fe7fc14be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc15a780 0x7fe7fc0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b3330 con 0x7fe7fc15a780 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b31b0 con 0x7fe7fc071260 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b34b0 con 0x7fe7fc071be0 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc15a780 0x7fe7fc0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc15a780 0x7fe7fc0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35200/0 (socket says 172.21.15.87:35200) 2024-04-25T10:22:21.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 -- 172.21.15.87:0/3727412301 learned_addr learned my addr 172.21.15.87:0/3727412301 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe7f3400640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc071be0 0x7fe7fc14be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe7fbe00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc071260 0x7fe7fc13dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 -- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc071be0 msgr2=0x7fe7fc14be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc071be0 0x7fe7fc14be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 -- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc071260 msgr2=0x7fe7fc13dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc071260 0x7fe7fc13dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 -- 172.21.15.87:0/3727412301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc14c360 con 0x7fe7fc15a780 2024-04-25T10:22:21.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801000640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc15a780 0x7fe7fc0cd5d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4007e70 tx=0x7fe7f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe7f8c00640 1 -- 172.21.15.87:0/3727412301 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4014620 con 0x7fe7fc15a780 2024-04-25T10:22:21.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe7f8c00640 1 -- 172.21.15.87:0/3727412301 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f4019070 con 0x7fe7fc15a780 2024-04-25T10:22:21.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe7f8c00640 1 -- 172.21.15.87:0/3727412301 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4014a20 con 0x7fe7fc15a780 2024-04-25T10:22:21.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc14c600 con 0x7fe7fc15a780 2024-04-25T10:22:21.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc14cbc0 con 0x7fe7fc15a780 2024-04-25T10:22:21.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc075120 con 0x7fe7fc15a780 2024-04-25T10:22:21.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.326+0000 7fe7f8c00640 1 -- 172.21.15.87:0/3727412301 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7f4014bc0 con 0x7fe7fc15a780 2024-04-25T10:22:21.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.330+0000 7fe7f8c00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe7d8066f60 0x7fe7d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.330+0000 7fe7f8c00640 1 -- 172.21.15.87:0/3727412301 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe7f40857e0 con 0x7fe7fc15a780 2024-04-25T10:22:21.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.330+0000 7fe7fbe00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe7d8066f60 0x7fe7d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.330+0000 7fe7fbe00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe7d8066f60 0x7fe7d8069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec001810 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.334+0000 7fe7f8c00640 1 -- 172.21.15.87:0/3727412301 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7f4056150 con 0x7fe7fc15a780 2024-04-25T10:22:21.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.518+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7fc0cf6c0 con 0x7fe7fc15a780 2024-04-25T10:22:21.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe7f8c00640 1 -- 172.21.15.87:0/3727412301 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fe7f405a1f0 con 0x7fe7fc15a780 2024-04-25T10:22:21.524 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:21.524 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:21.524 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:22:21.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe7d8066f60 msgr2=0x7fe7d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe7d8066f60 0x7fe7d8069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec001810 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2024-04-25T10:22:21.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc15a780 msgr2=0x7fe7fc0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc15a780 0x7fe7fc0cd5d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4007e70 tx=0x7fe7f400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 shutdown_connections 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe7d8066f60 0x7fe7d8069420 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7fc15a780 0x7fe7fc0cd5d0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc071be0 0x7fe7fc14be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 --2- 172.21.15.87:0/3727412301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc071260 0x7fe7fc13dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 >> 172.21.15.87:0/3727412301 conn(0x7fe7fc0743f0 msgr2=0x7fe7fc15ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 shutdown_connections 2024-04-25T10:22:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:21.522+0000 7fe801a00640 1 -- 172.21.15.87:0/3727412301 wait complete. 2024-04-25T10:22:21.544 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:22:23.730 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:23.734 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:26.545 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:26.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff890a00640 1 Processor -- start 2024-04-25T10:22:26.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff890a00640 1 -- start start 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff890a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c15e050 0x7ff88c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff890a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c0b1870 con 0x7ff88c0d1820 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff890a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c0b1af0 con 0x7ff88c0b13e0 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff890a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b1dd0 con 0x7ff88c15e050 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff889e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff88c0d1820 0x7ff88c0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53964/0 (socket says 172.21.15.87:53964) 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff889e00640 1 -- 172.21.15.87:0/3014633807 learned_addr learned my addr 172.21.15.87:0/3014633807 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff889400640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c15e050 0x7ff88c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731370170 0 0) 0x7ff87c001120 con 0x7ff88c0d1820 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870003530 con 0x7ff88c0d1820 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254257233 0 0) 0x7ff88c0b1af0 con 0x7ff88c0b13e0 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff87c001120 con 0x7ff88c0b13e0 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257802038 0 0) 0x7ff870003530 con 0x7ff88c0d1820 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c15e050 msgr2=0x7ff88c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c15e050 0x7ff88c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 >> v1:172.21.15.169:6789/0 conn(0x7ff88c0b13e0 legacy=0x7ff88c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0b20b0 con 0x7ff88c0d1820 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ff87c001850 con 0x7ff88c0d1820 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 >> v1:172.21.15.87:6789/0 conn(0x7ff88c0d1820 legacy=0x7ff88c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870003b80 0x7ff870003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff870004810 0x7ff870004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8700051d0 0x7ff87000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b1dd0 con 0x7ff8700051d0 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000c0a0 con 0x7ff870003b80 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000c2a0 con 0x7ff870004810 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff889400640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c15e050 0x7ff88c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff88a800640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8700051d0 0x7ff87000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff889e00640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870003b80 0x7ff870003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.689+0000 7ff889400640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff870004810 0x7ff870004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff88a800640 1 -- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff870004810 msgr2=0x7ff870004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff88a800640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff870004810 0x7ff870004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff88a800640 1 -- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870003b80 msgr2=0x7ff870003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:26.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff88a800640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870003b80 0x7ff870003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff88a800640 1 -- 172.21.15.87:0/3014633807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870003530 con 0x7ff8700051d0 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff88a800640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8700051d0 0x7ff87000b9d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ff884001fc0 tx=0x7ff88401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31aacca9c21add63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884001c10 con 0x7ff8700051d0 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840a8c90 con 0x7ff8700051d0 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff888a00640 1 -- 172.21.15.87:0/3014633807 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884003ba0 con 0x7ff8700051d0 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff890a00640 1 -- 172.21.15.87:0/3014633807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8700051d0 msgr2=0x7ff87000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff890a00640 1 --2- 172.21.15.87:0/3014633807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8700051d0 0x7ff87000b9d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ff884001fc0 tx=0x7ff88401f070 comp rx=0 tx=0).stop 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff88a800640 1 -- 172.21.15.87:0/3014633807 reap_dead start 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff890a00640 1 -- 172.21.15.87:0/3014633807 shutdown_connections 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.693+0000 7ff890a00640 1 -- 172.21.15.87:0/3014633807 >> 172.21.15.87:0/3014633807 conn(0x7ff88c09a9d0 msgr2=0x7ff88c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 -- 172.21.15.87:0/3014633807 shutdown_connections 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 -- 172.21.15.87:0/3014633807 wait complete. 2024-04-25T10:22:26.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 Processor -- start 2024-04-25T10:22:26.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 -- start start 2024-04-25T10:22:26.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88c0c6020 0x7ff88c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff88c0c5430 0x7ff88c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c0b4500 0x7ff88c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b0ea0 con 0x7ff88c0c5430 2024-04-25T10:22:26.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b2670 con 0x7ff88c0c6020 2024-04-25T10:22:26.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b1020 con 0x7ff88c0b4500 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff88c0c5430 0x7ff88c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff88c0c5430 0x7ff88c15ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35222/0 (socket says 172.21.15.87:35222) 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 -- 172.21.15.87:0/1608545481 learned_addr learned my addr 172.21.15.87:0/1608545481 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889e00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88c0c6020 0x7ff88c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff88a800640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c0b4500 0x7ff88c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 -- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c0b4500 msgr2=0x7ff88c0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c0b4500 0x7ff88c0b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 -- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88c0c6020 msgr2=0x7ff88c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88c0c6020 0x7ff88c15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 -- 172.21.15.87:0/1608545481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c1f3ad0 con 0x7ff88c0c5430 2024-04-25T10:22:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff889400640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff88c0c5430 0x7ff88c15ada0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff874011a00 tx=0x7ff874011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:26.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff882000640 1 -- 172.21.15.87:0/1608545481 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87400fb50 con 0x7ff88c0c5430 2024-04-25T10:22:26.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff882000640 1 -- 172.21.15.87:0/1608545481 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87400fcf0 con 0x7ff88c0c5430 2024-04-25T10:22:26.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff882000640 1 -- 172.21.15.87:0/1608545481 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87400fea0 con 0x7ff88c0c5430 2024-04-25T10:22:26.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.697+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88c1f3db0 con 0x7ff88c0c5430 2024-04-25T10:22:26.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.701+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c1f4370 con 0x7ff88c0c5430 2024-04-25T10:22:26.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.701+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c04ef30 con 0x7ff88c0c5430 2024-04-25T10:22:26.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.701+0000 7ff882000640 1 -- 172.21.15.87:0/1608545481 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff874012040 con 0x7ff88c0c5430 2024-04-25T10:22:26.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.701+0000 7ff882000640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff868066f60 0x7ff868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.701+0000 7ff882000640 1 -- 172.21.15.87:0/1608545481 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff8740857a0 con 0x7ff88c0c5430 2024-04-25T10:22:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.701+0000 7ff889e00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff868066f60 0x7ff868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.701+0000 7ff889e00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff868066f60 0x7ff868069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004850 tx=0x7ff87c007980 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:26.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.705+0000 7ff882000640 1 -- 172.21.15.87:0/1608545481 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff874019030 con 0x7ff88c0c5430 2024-04-25T10:22:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff88c06b480 con 0x7ff88c0c5430 2024-04-25T10:22:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff882000640 1 -- 172.21.15.87:0/1608545481 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff874056640 con 0x7ff88c0c5430 2024-04-25T10:22:26.897 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:26.897 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:26.897 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:22:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff868066f60 msgr2=0x7ff868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff868066f60 0x7ff868069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004850 tx=0x7ff87c007980 comp rx=0 tx=0).stop 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff88c0c5430 msgr2=0x7ff88c15ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff88c0c5430 0x7ff88c15ada0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff874011a00 tx=0x7ff874011e00 comp rx=0 tx=0).stop 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 shutdown_connections 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff868066f60 0x7ff868069420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff88c0b4500 0x7ff88c0b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff88c0c5430 0x7ff88c15ada0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 --2- 172.21.15.87:0/1608545481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff88c0c6020 0x7ff88c15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 >> 172.21.15.87:0/1608545481 conn(0x7ff88c09a9d0 msgr2=0x7ff88c15c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 shutdown_connections 2024-04-25T10:22:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:26.893+0000 7ff890a00640 1 -- 172.21.15.87:0/1608545481 wait complete. 2024-04-25T10:22:26.917 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:22:31.918 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:32.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.061+0000 7f37bb600640 1 Processor -- start 2024-04-25T10:22:32.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.061+0000 7f37bb600640 1 -- start start 2024-04-25T10:22:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.061+0000 7f37bb600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f37b415bc40 0x7f37b406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.061+0000 7f37bb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b40cf930 0x7f37b40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.061+0000 7f37bb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b40b2770 con 0x7f37b40cda10 2024-04-25T10:22:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.061+0000 7f37bb600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b406c670 con 0x7f37b415bc40 2024-04-25T10:22:32.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.061+0000 7f37bb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b406c950 con 0x7f37b40cf930 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b40cf930 0x7f37b40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b8c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f37b415bc40 0x7f37b406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37abe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f37b40cda10 0x7f37b40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:33398/0 (socket says 172.21.15.87:33398) 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37abe00640 1 -- 172.21.15.87:0/2493617592 learned_addr learned my addr 172.21.15.87:0/2493617592 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b9600640 1 -- 172.21.15.87:0/2493617592 >> v2:172.21.15.169:3300/0 conn(0x7f37b415bc40 msgr2=0x7f37b406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b9600640 1 --2- 172.21.15.87:0/2493617592 >> v2:172.21.15.169:3300/0 conn(0x7f37b415bc40 0x7f37b406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b9600640 1 -- 172.21.15.87:0/2493617592 >> v1:172.21.15.169:6789/0 conn(0x7f37b40cda10 legacy=0x7f37b40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b9600640 1 -- 172.21.15.87:0/2493617592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b406cc30 con 0x7f37b40cf930 2024-04-25T10:22:32.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b3e00640 1 -- 172.21.15.87:0/2493617592 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828382870 0 0) 0x7f37b40b2770 con 0x7f37b40cda10 2024-04-25T10:22:32.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.065+0000 7f37b9600640 1 --2- 172.21.15.87:0/2493617592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b40cf930 0x7f37b40cfde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009d90 tx=0x7f37ac01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9434077776deb29b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37b3e00640 1 -- 172.21.15.87:0/2493617592 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0b2070 con 0x7f37b40cf930 2024-04-25T10:22:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37b3e00640 1 -- 172.21.15.87:0/2493617592 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac0a57a0 con 0x7f37b40cf930 2024-04-25T10:22:32.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37b3e00640 1 -- 172.21.15.87:0/2493617592 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0a5b70 con 0x7f37b40cf930 2024-04-25T10:22:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- 172.21.15.87:0/2493617592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b40cf930 msgr2=0x7f37b40cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 --2- 172.21.15.87:0/2493617592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b40cf930 0x7f37b40cfde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009d90 tx=0x7f37ac01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- 172.21.15.87:0/2493617592 shutdown_connections 2024-04-25T10:22:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 --2- 172.21.15.87:0/2493617592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b40cf930 0x7f37b40cfde0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 --2- 172.21.15.87:0/2493617592 >> v2:172.21.15.169:3300/0 conn(0x7f37b415bc40 0x7f37b406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- 172.21.15.87:0/2493617592 >> 172.21.15.87:0/2493617592 conn(0x7f37b4066b70 msgr2=0x7f37b4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- 172.21.15.87:0/2493617592 shutdown_connections 2024-04-25T10:22:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- 172.21.15.87:0/2493617592 wait complete. 2024-04-25T10:22:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 Processor -- start 2024-04-25T10:22:32.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- start start 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b40cda10 0x7f37b413b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b40cf930 0x7f37b4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b415bc40 0x7f37b4154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b3890 con 0x7f37b40cf930 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b3710 con 0x7f37b40cda10 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37bb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b3a10 con 0x7f37b415bc40 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37abe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b40cf930 0x7f37b4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.069+0000 7f37abe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b40cf930 0x7f37b4154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42086/0 (socket says 172.21.15.87:42086) 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37abe00640 1 -- 172.21.15.87:0/1551995814 learned_addr learned my addr 172.21.15.87:0/1551995814 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b8c00640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b40cda10 0x7f37b413b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b9600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b415bc40 0x7f37b4154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37abe00640 1 -- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b415bc40 msgr2=0x7f37b4154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37abe00640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b415bc40 0x7f37b4154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37abe00640 1 -- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b40cda10 msgr2=0x7f37b413b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37abe00640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b40cda10 0x7f37b413b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37abe00640 1 -- 172.21.15.87:0/1551995814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b413b610 con 0x7f37b40cf930 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37abe00640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b40cf930 0x7f37b4154840 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c480 tx=0x7f379c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b1600640 1 -- 172.21.15.87:0/1551995814 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c018070 con 0x7f37b40cf930 2024-04-25T10:22:32.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b1600640 1 -- 172.21.15.87:0/1551995814 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c007500 con 0x7f37b40cf930 2024-04-25T10:22:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b41f4d20 con 0x7f37b40cf930 2024-04-25T10:22:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b1600640 1 -- 172.21.15.87:0/1551995814 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c004a30 con 0x7f37b40cf930 2024-04-25T10:22:32.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b41f52e0 con 0x7f37b40cf930 2024-04-25T10:22:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b404ef30 con 0x7f37b40cf930 2024-04-25T10:22:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b1600640 1 -- 172.21.15.87:0/1551995814 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f379c01b070 con 0x7f37b40cf930 2024-04-25T10:22:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b1600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b1600640 1 -- 172.21.15.87:0/1551995814 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f379c084310 con 0x7f37b40cf930 2024-04-25T10:22:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.073+0000 7f37b8c00640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.077+0000 7f37b8c00640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009a50 tx=0x7f37a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.077+0000 7f37b1600640 1 -- 172.21.15.87:0/1551995814 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f379c055370 con 0x7f37b40cf930 2024-04-25T10:22:32.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.265+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37b40af1c0 con 0x7f37b40cf930 2024-04-25T10:22:32.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.265+0000 7f37b1600640 1 -- 172.21.15.87:0/1551995814 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f379c054c70 con 0x7f37b40cf930 2024-04-25T10:22:32.271 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:32.271 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:21:45.580473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9114":{"gid":9114,"name":"d","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/85260299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":85260299},{"type":"v1","addr":"172.21.15.169:6834","nonce":85260299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:32.271 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 89 2024-04-25T10:22:32.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 msgr2=0x7f3790069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009a50 tx=0x7f37a4008040 comp rx=0 tx=0).stop 2024-04-25T10:22:32.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b40cf930 msgr2=0x7f37b4154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b40cf930 0x7f37b4154840 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c480 tx=0x7f379c00c940 comp rx=0 tx=0).stop 2024-04-25T10:22:32.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 shutdown_connections 2024-04-25T10:22:32.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b415bc40 0x7f37b4154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b40cf930 0x7f37b4154840 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 --2- 172.21.15.87:0/1551995814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b40cda10 0x7f37b413b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 >> 172.21.15.87:0/1551995814 conn(0x7f37b4066b70 msgr2=0x7f37b414bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:32.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 shutdown_connections 2024-04-25T10:22:32.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:32.269+0000 7f37bb600640 1 -- 172.21.15.87:0/1551995814 wait complete. 2024-04-25T10:22:32.292 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:22:37.293 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:37.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2369000640 1 Processor -- start 2024-04-25T10:22:37.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2369000640 1 -- start start 2024-04-25T10:22:37.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2369000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23640d0040 0x7f236406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2369000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23640b2770 con 0x7f236415bc40 2024-04-25T10:22:37.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2369000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236406c670 con 0x7f23640cda10 2024-04-25T10:22:37.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2369000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236406c950 con 0x7f23640d0040 2024-04-25T10:22:37.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2361e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f236415bc40 0x7f23640b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56956/0 (socket says 172.21.15.87:56956) 2024-04-25T10:22:37.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2361e00640 1 -- 172.21.15.87:0/1021184353 learned_addr learned my addr 172.21.15.87:0/1021184353 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:37.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2362800640 1 --2- 172.21.15.87:0/1021184353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23640d0040 0x7f236406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2361400640 1 -- 172.21.15.87:0/1021184353 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356587400 0 0) 0x7f23640b2770 con 0x7f236415bc40 2024-04-25T10:22:37.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2361400640 1 -- 172.21.15.87:0/1021184353 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2348003530 con 0x7f236415bc40 2024-04-25T10:22:37.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2361400640 1 -- 172.21.15.87:0/1021184353 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171030447 0 0) 0x7f236406c670 con 0x7f23640cda10 2024-04-25T10:22:37.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2362800640 1 -- 172.21.15.87:0/1021184353 >> v1:172.21.15.169:6789/0 conn(0x7f23640cda10 legacy=0x7f23640cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2362800640 1 -- 172.21.15.87:0/1021184353 >> v1:172.21.15.87:6789/0 conn(0x7f236415bc40 legacy=0x7f23640b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.437+0000 7f2362800640 1 -- 172.21.15.87:0/1021184353 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236406cc30 con 0x7f23640d0040 2024-04-25T10:22:37.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2362800640 1 --2- 172.21.15.87:0/1021184353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23640d0040 0x7f236406c090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f235c0097f0 tx=0x7f235c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1c815ad66fa5da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:37.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2361400640 1 -- 172.21.15.87:0/1021184353 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0b2070 con 0x7f23640d0040 2024-04-25T10:22:37.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2361400640 1 -- 172.21.15.87:0/1021184353 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c0a9070 con 0x7f23640d0040 2024-04-25T10:22:37.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2361400640 1 -- 172.21.15.87:0/1021184353 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0ad880 con 0x7f23640d0040 2024-04-25T10:22:37.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2369000640 1 -- 172.21.15.87:0/1021184353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23640d0040 msgr2=0x7f236406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2369000640 1 --2- 172.21.15.87:0/1021184353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23640d0040 0x7f236406c090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f235c0097f0 tx=0x7f235c01c450 comp rx=0 tx=0).stop 2024-04-25T10:22:37.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2369000640 1 -- 172.21.15.87:0/1021184353 shutdown_connections 2024-04-25T10:22:37.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2369000640 1 --2- 172.21.15.87:0/1021184353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23640d0040 0x7f236406c090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2369000640 1 -- 172.21.15.87:0/1021184353 >> 172.21.15.87:0/1021184353 conn(0x7f2364066b70 msgr2=0x7f2364066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:37.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2369000640 1 -- 172.21.15.87:0/1021184353 shutdown_connections 2024-04-25T10:22:37.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.441+0000 7f2369000640 1 -- 172.21.15.87:0/1021184353 wait complete. 2024-04-25T10:22:37.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 Processor -- start 2024-04-25T10:22:37.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 -- start start 2024-04-25T10:22:37.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23640cda10 0x7f236414c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23640d0040 0x7f2364155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236415bc40 0x7f23641513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2364169980 con 0x7f23640d0040 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2364169800 con 0x7f23640cda10 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2364169b00 con 0x7f236415bc40 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23640d0040 0x7f2364155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23640d0040 0x7f2364155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54234/0 (socket says 172.21.15.87:54234) 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 -- 172.21.15.87:0/576907850 learned_addr learned my addr 172.21.15.87:0/576907850 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2361e00640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23640cda10 0x7f236414c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2362800640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236415bc40 0x7f23641513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 -- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236415bc40 msgr2=0x7f23641513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236415bc40 0x7f23641513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 -- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23640cda10 msgr2=0x7f236414c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23640cda10 0x7f236414c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 -- 172.21.15.87:0/576907850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364150300 con 0x7f23640d0040 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2359400640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23640d0040 0x7f2364155cc0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c500 tx=0x7f234c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f235aa00640 1 -- 172.21.15.87:0/576907850 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c017070 con 0x7f23640d0040 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f235aa00640 1 -- 172.21.15.87:0/576907850 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c003600 con 0x7f23640d0040 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f235aa00640 1 -- 172.21.15.87:0/576907850 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c0039f0 con 0x7f23640d0040 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23641505e0 con 0x7f23640d0040 2024-04-25T10:22:37.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2364150ba0 con 0x7f23640d0040 2024-04-25T10:22:37.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.445+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236404ef30 con 0x7f23640d0040 2024-04-25T10:22:37.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.449+0000 7f235aa00640 1 -- 172.21.15.87:0/576907850 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f234c01a070 con 0x7f23640d0040 2024-04-25T10:22:37.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.449+0000 7f235aa00640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2340066f60 0x7f2340069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.449+0000 7f235aa00640 1 -- 172.21.15.87:0/576907850 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f234c0843c0 con 0x7f23640d0040 2024-04-25T10:22:37.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.449+0000 7f2361e00640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2340066f60 0x7f2340069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.449+0000 7f2361e00640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2340066f60 0x7f2340069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2354005c10 tx=0x7f2354009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:37.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.453+0000 7f235aa00640 1 -- 172.21.15.87:0/576907850 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f234c054e20 con 0x7f23640d0040 2024-04-25T10:22:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.637+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23640714e0 con 0x7f23640d0040 2024-04-25T10:22:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f235aa00640 1 -- 172.21.15.87:0/576907850 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f234c058ec0 con 0x7f23640d0040 2024-04-25T10:22:37.644 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:37.644 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:22:34.985474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9111},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9111":{"gid":9111,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":17,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:37.644 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 93 2024-04-25T10:22:37.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2340066f60 msgr2=0x7f2340069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2340066f60 0x7f2340069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2354005c10 tx=0x7f2354009040 comp rx=0 tx=0).stop 2024-04-25T10:22:37.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23640d0040 msgr2=0x7f2364155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23640d0040 0x7f2364155cc0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c500 tx=0x7f234c00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 shutdown_connections 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2340066f60 0x7f2340069420 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236415bc40 0x7f23641513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23640d0040 0x7f2364155cc0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 --2- 172.21.15.87:0/576907850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23640cda10 0x7f236414c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 >> 172.21.15.87:0/576907850 conn(0x7f2364066b70 msgr2=0x7f23640ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 shutdown_connections 2024-04-25T10:22:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.641+0000 7f2369000640 1 -- 172.21.15.87:0/576907850 wait complete. 2024-04-25T10:22:37.665 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T10:22:37.665 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52a00640 1 Processor -- start 2024-04-25T10:22:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52a00640 1 -- start start 2024-04-25T10:22:37.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e4c15dff0 0x7f2e4c06b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06bbe0 0x7f2e4c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c0cbff0 con 0x7f2e4c15dc10 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c06c050 con 0x7f2e4c15dff0 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c0b45a0 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e50c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e4c15dff0 0x7f2e4c06b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e50c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e4c15dff0 0x7f2e4c06b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54236/0 (socket says 172.21.15.87:54236) 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e50c00640 1 -- 172.21.15.87:0/600904248 learned_addr learned my addr 172.21.15.87:0/600904248 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e52000640 1 --2- 172.21.15.87:0/600904248 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06bbe0 0x7f2e4c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e50c00640 1 -- 172.21.15.87:0/600904248 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06bbe0 msgr2=0x7f2e4c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e50c00640 1 --2- 172.21.15.87:0/600904248 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06bbe0 0x7f2e4c0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e50c00640 1 -- 172.21.15.87:0/600904248 >> v1:172.21.15.169:6789/0 conn(0x7f2e4c15dc10 legacy=0x7f2e4c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.797+0000 7f2e50c00640 1 -- 172.21.15.87:0/600904248 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0cc2d0 con 0x7f2e4c15dff0 2024-04-25T10:22:37.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e50c00640 1 --2- 172.21.15.87:0/600904248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c15dff0 0x7f2e4c06b6a0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c074960 tx=0x7f2e3801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e12f24ae3b0bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:37.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e4be00640 1 -- 172.21.15.87:0/600904248 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380ab610 con 0x7f2e4c15dff0 2024-04-25T10:22:37.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e4be00640 1 -- 172.21.15.87:0/600904248 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e380b0070 con 0x7f2e4c15dff0 2024-04-25T10:22:37.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e4be00640 1 -- 172.21.15.87:0/600904248 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380ab9d0 con 0x7f2e4c15dff0 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 -- 172.21.15.87:0/600904248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c15dff0 msgr2=0x7f2e4c06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 --2- 172.21.15.87:0/600904248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c15dff0 0x7f2e4c06b6a0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c074960 tx=0x7f2e3801c520 comp rx=0 tx=0).stop 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 -- 172.21.15.87:0/600904248 shutdown_connections 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 --2- 172.21.15.87:0/600904248 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06bbe0 0x7f2e4c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 --2- 172.21.15.87:0/600904248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c15dff0 0x7f2e4c06b6a0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 -- 172.21.15.87:0/600904248 >> 172.21.15.87:0/600904248 conn(0x7f2e4c0d1750 msgr2=0x7f2e4c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 -- 172.21.15.87:0/600904248 shutdown_connections 2024-04-25T10:22:37.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 -- 172.21.15.87:0/600904248 wait complete. 2024-04-25T10:22:37.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 Processor -- start 2024-04-25T10:22:37.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.801+0000 7f2e52a00640 1 -- start start 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c06bbe0 0x7f2e4c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c15dc10 0x7f2e4c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c15dff0 0x7f2e4c151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c06b1e0 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c06b060 con 0x7f2e4c15dff0 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0cdb00 con 0x7f2e4c15dc10 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c06bbe0 0x7f2e4c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c06bbe0 0x7f2e4c15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54238/0 (socket says 172.21.15.87:54238) 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 -- 172.21.15.87:0/2973775703 learned_addr learned my addr 172.21.15.87:0/2973775703 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52000640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c15dff0 0x7f2e4c151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e50c00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c15dc10 0x7f2e4c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 -- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c15dc10 msgr2=0x7f2e4c151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c15dc10 0x7f2e4c151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 -- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c15dff0 msgr2=0x7f2e4c151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c15dff0 0x7f2e4c151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 -- 172.21.15.87:0/2973775703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c159360 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e51600640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c06bbe0 0x7f2e4c15c370 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000c450 tx=0x7f2e4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e49600640 1 -- 172.21.15.87:0/2973775703 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40016020 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e49600640 1 -- 172.21.15.87:0/2973775703 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e400052d0 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e49600640 1 -- 172.21.15.87:0/2973775703 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40007260 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c15c8b0 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c1598e0 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.805+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c075120 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.809+0000 7f2e49600640 1 -- 172.21.15.87:0/2973775703 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e40007400 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.809+0000 7f2e49600640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e2c066f60 0x7f2e2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.809+0000 7f2e49600640 1 -- 172.21.15.87:0/2973775703 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2e40084910 con 0x7f2e4c06bbe0 2024-04-25T10:22:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.809+0000 7f2e50c00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e2c066f60 0x7f2e2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.809+0000 7f2e50c00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e2c066f60 0x7f2e2c069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2e380af000 tx=0x7f2e380a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:37.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:37.813+0000 7f2e49600640 1 -- 172.21.15.87:0/2973775703 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e40020020 con 0x7f2e4c06bbe0 2024-04-25T10:22:38.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.001+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e4c06b700 con 0x7f2e4c06bbe0 2024-04-25T10:22:38.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.001+0000 7f2e49600640 1 -- 172.21.15.87:0/2973775703 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f2e40005480 con 0x7f2e4c06bbe0 2024-04-25T10:22:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:38.005 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:22:34.985474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9111},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9111":{"gid":9111,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":17,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:38.006 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 93 2024-04-25T10:22:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e2c066f60 msgr2=0x7f2e2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e2c066f60 0x7f2e2c069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2e380af000 tx=0x7f2e380a7000 comp rx=0 tx=0).stop 2024-04-25T10:22:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c06bbe0 msgr2=0x7f2e4c15c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c06bbe0 0x7f2e4c15c370 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000c450 tx=0x7f2e4000c910 comp rx=0 tx=0).stop 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 shutdown_connections 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e2c066f60 0x7f2e2c069420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c15dff0 0x7f2e4c151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c15dc10 0x7f2e4c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 --2- 172.21.15.87:0/2973775703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e4c06bbe0 0x7f2e4c15c370 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 >> 172.21.15.87:0/2973775703 conn(0x7f2e4c0d1750 msgr2=0x7f2e4c0cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 shutdown_connections 2024-04-25T10:22:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.005+0000 7f2e52a00640 1 -- 172.21.15.87:0/2973775703 wait complete. 2024-04-25T10:22:38.025 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:22:38.026 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:22:38.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c42e00640 1 Processor -- start 2024-04-25T10:22:38.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c42e00640 1 -- start start 2024-04-25T10:22:38.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c42e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 0x7f8c3c166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c42e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c0ccfb0 con 0x7f8c3c0ccb20 2024-04-25T10:22:38.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c42e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c0cd290 con 0x7f8c3c15b9e0 2024-04-25T10:22:38.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c42e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0cd570 con 0x7f8c3c0c6a20 2024-04-25T10:22:38.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c33400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8c3c0ccb20 0x7f8c3c0d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56962/0 (socket says 172.21.15.87:56962) 2024-04-25T10:22:38.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c33400640 1 -- 172.21.15.87:0/2739658671 learned_addr learned my addr 172.21.15.87:0/2739658671 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:38.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035848318 0 0) 0x7f8c3c0ccfb0 con 0x7f8c3c0ccb20 2024-04-25T10:22:38.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c20003530 con 0x7f8c3c0ccb20 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034409122 0 0) 0x7f8c20003530 con 0x7f8c3c0ccb20 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c40e00640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 0x7f8c3c166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 msgr2=0x7f8c3c166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 0x7f8c3c166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 >> v1:172.21.15.169:6789/0 conn(0x7f8c3c15b9e0 legacy=0x7f8c3c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0cd850 con 0x7f8c3c0ccb20 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f8c240016a0 con 0x7f8c3c0ccb20 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c40e00640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 0x7f8c3c166820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 >> v1:172.21.15.87:6789/0 conn(0x7f8c3c0ccb20 legacy=0x7f8c3c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c200039f0 0x7f8c20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c20004810 0x7f8c20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 0x7f8c20005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0cd570 con 0x7f8c20004810 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2000bb60 con 0x7f8c200039f0 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.157+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2000bd60 con 0x7f8c3c0c6a20 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c33400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c20004810 0x7f8c20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c33400640 1 -- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 msgr2=0x7f8c20005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c33400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c6a20 0x7f8c20005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c33400640 1 -- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c200039f0 msgr2=0x7f8c20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c33400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c200039f0 0x7f8c20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c33400640 1 -- 172.21.15.87:0/2739658671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20003530 con 0x7f8c20004810 2024-04-25T10:22:38.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c33400640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c20004810 0x7f8c20004c90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0c8450 tx=0x7f8c2401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336b8bb96597082a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24007cf0 con 0x7f8c20004810 2024-04-25T10:22:38.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c240a9070 con 0x7f8c20004810 2024-04-25T10:22:38.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.161+0000 7f8c3b400640 1 -- 172.21.15.87:0/2739658671 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24005d20 con 0x7f8c20004810 2024-04-25T10:22:38.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- 172.21.15.87:0/2739658671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c20004810 msgr2=0x7f8c20004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 --2- 172.21.15.87:0/2739658671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c20004810 0x7f8c20004c90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0c8450 tx=0x7f8c2401cba0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 -- 172.21.15.87:0/2739658671 reap_dead start 2024-04-25T10:22:38.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- 172.21.15.87:0/2739658671 shutdown_connections 2024-04-25T10:22:38.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- 172.21.15.87:0/2739658671 >> 172.21.15.87:0/2739658671 conn(0x7f8c3c09a9d0 msgr2=0x7f8c3c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:38.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- 172.21.15.87:0/2739658671 shutdown_connections 2024-04-25T10:22:38.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- 172.21.15.87:0/2739658671 wait complete. 2024-04-25T10:22:38.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 Processor -- start 2024-04-25T10:22:38.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- start start 2024-04-25T10:22:38.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c20004810 0x7f8c3c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c89d0 0x7f8c3c1f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c3c06e9d0 0x7f8c3c0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0d1c30 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0cf740 con 0x7f8c20004810 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c42e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0d1db0 con 0x7f8c3c0c89d0 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c3c06e9d0 0x7f8c3c0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c33400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c89d0 0x7f8c3c1f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c3c06e9d0 0x7f8c3c0c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54264/0 (socket says 172.21.15.87:54264) 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 -- 172.21.15.87:0/583556358 learned_addr learned my addr 172.21.15.87:0/583556358 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c3be00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c20004810 0x7f8c3c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 -- 172.21.15.87:0/583556358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c89d0 msgr2=0x7f8c3c1f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c89d0 0x7f8c3c1f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 -- 172.21.15.87:0/583556358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c20004810 msgr2=0x7f8c3c159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c20004810 0x7f8c3c159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.165+0000 7f8c40e00640 1 -- 172.21.15.87:0/583556358 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0cb910 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c40e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c3c06e9d0 0x7f8c3c0c3f30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0732f0 tx=0x7f8c3400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c38c00640 1 -- 172.21.15.87:0/583556358 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34011800 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c38c00640 1 -- 172.21.15.87:0/583556358 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c34018070 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c38c00640 1 -- 172.21.15.87:0/583556358 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34011bc0 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c0cb390 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c1f8230 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c04ef30 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c38c00640 1 -- 172.21.15.87:0/583556358 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c3401d460 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c38c00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c18066f60 0x7f8c18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c38c00640 1 -- 172.21.15.87:0/583556358 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8c34084a60 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.169+0000 7f8c3be00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c18066f60 0x7f8c18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.173+0000 7f8c3be00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c18066f60 0x7f8c18069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0049a0 tx=0x7f8c2c00c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.177+0000 7f8c38c00640 1 -- 172.21.15.87:0/583556358 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c340552f0 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8c3c0b5d50 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c38c00640 1 -- 172.21.15.87:0/583556358 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1468 (secure 0 0 0) 0x7f8c34059390 con 0x7f8c3c06e9d0 2024-04-25T10:22:38.420 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:38.420 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":640,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":127,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039930,"num_in_osds":8,"osd_in_since":1714039920,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077122811,"bytes_used":2630754304,"bytes_avail":770463358976,"bytes_total":773094113280,"read_bytes_sec":245489,"write_bytes_sec":75243,"read_op_per_sec":11,"write_op_per_sec":36},"fsmap":{"epoch":93,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9111}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T10:22:33.253685+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c18066f60 msgr2=0x7f8c18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c18066f60 0x7f8c18069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0049a0 tx=0x7f8c2c00c040 comp rx=0 tx=0).stop 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c3c06e9d0 msgr2=0x7f8c3c0c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c3c06e9d0 0x7f8c3c0c3f30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0732f0 tx=0x7f8c3400c890 comp rx=0 tx=0).stop 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 shutdown_connections 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c18066f60 0x7f8c18069420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c3c06e9d0 0x7f8c3c0c3f30 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c3c0c89d0 0x7f8c3c1f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 --2- 172.21.15.87:0/583556358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c20004810 0x7f8c3c159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.417+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 >> 172.21.15.87:0/583556358 conn(0x7f8c3c09a9d0 msgr2=0x7f8c3c15ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:38.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.421+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 shutdown_connections 2024-04-25T10:22:38.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.421+0000 7f8c42e00640 1 -- 172.21.15.87:0/583556358 wait complete. 2024-04-25T10:22:38.442 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:22:38.442 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:22:38.442 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:38.498 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:38.566 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:22:38.566 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:22:38.567 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:38.575 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:22:38.575 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:38.631 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:38.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff247e00640 1 Processor -- start 2024-04-25T10:22:38.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff247e00640 1 -- start start 2024-04-25T10:22:38.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff247e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2401550b0 0x7ff2400cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff247e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff24015b910 0x7ff2400b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff247e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff240071050 con 0x7ff2400cfe70 2024-04-25T10:22:38.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff247e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24015bda0 con 0x7ff24015b910 2024-04-25T10:22:38.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff247e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24015d040 con 0x7ff2401550b0 2024-04-25T10:22:38.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff245400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2401550b0 0x7ff2400cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff245400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2401550b0 0x7ff2400cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:33814/0 (socket says 172.21.15.87:33814) 2024-04-25T10:22:38.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff245400640 1 -- 172.21.15.87:0/2651474325 learned_addr learned my addr 172.21.15.87:0/2651474325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:38.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.809+0000 7ff245e00640 1 --2- 172.21.15.87:0/2651474325 >> v2:172.21.15.169:3300/0 conn(0x7ff24015b910 0x7ff2400b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff245400640 1 -- 172.21.15.87:0/2651474325 >> v2:172.21.15.169:3300/0 conn(0x7ff24015b910 msgr2=0x7ff2400b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff245400640 1 --2- 172.21.15.87:0/2651474325 >> v2:172.21.15.169:3300/0 conn(0x7ff24015b910 0x7ff2400b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff245400640 1 -- 172.21.15.87:0/2651474325 >> v1:172.21.15.169:6789/0 conn(0x7ff2400cfe70 legacy=0x7ff24015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff245400640 1 -- 172.21.15.87:0/2651474325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2400712f0 con 0x7ff2401550b0 2024-04-25T10:22:38.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff245400640 1 --2- 172.21.15.87:0/2651474325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2401550b0 0x7ff2400cf930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff234009870 tx=0x7ff23401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2c31bbf63d5ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff23fe00640 1 -- 172.21.15.87:0/2651474325 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2340ab650 con 0x7ff2401550b0 2024-04-25T10:22:38.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff23fe00640 1 -- 172.21.15.87:0/2651474325 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2340b0070 con 0x7ff2401550b0 2024-04-25T10:22:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.813+0000 7ff23fe00640 1 -- 172.21.15.87:0/2651474325 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2340aba10 con 0x7ff2401550b0 2024-04-25T10:22:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- 172.21.15.87:0/2651474325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2401550b0 msgr2=0x7ff2400cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 --2- 172.21.15.87:0/2651474325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2401550b0 0x7ff2400cf930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff234009870 tx=0x7ff23401c560 comp rx=0 tx=0).stop 2024-04-25T10:22:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- 172.21.15.87:0/2651474325 shutdown_connections 2024-04-25T10:22:38.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 --2- 172.21.15.87:0/2651474325 >> v2:172.21.15.169:3300/0 conn(0x7ff24015b910 0x7ff2400b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 --2- 172.21.15.87:0/2651474325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2401550b0 0x7ff2400cf930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- 172.21.15.87:0/2651474325 >> 172.21.15.87:0/2651474325 conn(0x7ff240066b70 msgr2=0x7ff240066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:38.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- 172.21.15.87:0/2651474325 shutdown_connections 2024-04-25T10:22:38.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- 172.21.15.87:0/2651474325 wait complete. 2024-04-25T10:22:38.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 Processor -- start 2024-04-25T10:22:38.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- start start 2024-04-25T10:22:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2400cfe70 0x7ff2401f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2401550b0 0x7ff2400cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24015b910 0x7ff240155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24015c030 con 0x7ff2401550b0 2024-04-25T10:22:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400720b0 con 0x7ff2400cfe70 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff247e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24015c1b0 con 0x7ff24015b910 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2401550b0 0x7ff2400cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2401550b0 0x7ff2400cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54274/0 (socket says 172.21.15.87:54274) 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 -- 172.21.15.87:0/642963943 learned_addr learned my addr 172.21.15.87:0/642963943 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff245400640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2400cfe70 0x7ff2401f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff245e00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24015b910 0x7ff240155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 -- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24015b910 msgr2=0x7ff240155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24015b910 0x7ff240155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 -- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2400cfe70 msgr2=0x7ff2401f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2400cfe70 0x7ff2401f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.817+0000 7ff244a00640 1 -- 172.21.15.87:0/642963943 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff240155e70 con 0x7ff2401550b0 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff244a00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2401550b0 0x7ff2400cadd0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c420 tx=0x7ff22c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff23d600640 1 -- 172.21.15.87:0/642963943 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0052d0 con 0x7ff2401550b0 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff23d600640 1 -- 172.21.15.87:0/642963943 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c005470 con 0x7ff2401550b0 2024-04-25T10:22:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff23d600640 1 -- 172.21.15.87:0/642963943 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c007260 con 0x7ff2401550b0 2024-04-25T10:22:38.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff240158c50 con 0x7ff2401550b0 2024-04-25T10:22:38.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff240159210 con 0x7ff2401550b0 2024-04-25T10:22:38.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2401f6ee0 con 0x7ff2401550b0 2024-04-25T10:22:38.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff23d600640 1 -- 172.21.15.87:0/642963943 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff22c007400 con 0x7ff2401550b0 2024-04-25T10:22:38.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff23d600640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff220066f60 0x7ff220069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:38.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff23d600640 1 -- 172.21.15.87:0/642963943 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff22c084580 con 0x7ff2401550b0 2024-04-25T10:22:38.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.821+0000 7ff245400640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff220066f60 0x7ff220069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:38.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.825+0000 7ff245400640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff220066f60 0x7ff220069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff2340af000 tx=0x7ff2340a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:38.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.825+0000 7ff23d600640 1 -- 172.21.15.87:0/642963943 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2401f6ee0 con 0x7ff2401550b0 2024-04-25T10:22:38.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.981+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff24004ef30 con 0x7ff2401550b0 2024-04-25T10:22:38.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.985+0000 7ff23d600640 1 -- 172.21.15.87:0/642963943 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16836 (secure 0 0 0) 0x7ff22c054ea0 con 0x7ff2401550b0 2024-04-25T10:22:38.989 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:38.990 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":127,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:32.952911+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:40.912468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:41.547768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/85260299":"2024-04-26T10:22:32.952837+0000","172.21.15.169:6832/85260299":"2024-04-26T10:22:32.952837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:38.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff220066f60 msgr2=0x7ff220069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff220066f60 0x7ff220069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff2340af000 tx=0x7ff2340a7000 comp rx=0 tx=0).stop 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2401550b0 msgr2=0x7ff2400cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2401550b0 0x7ff2400cadd0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c420 tx=0x7ff22c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 shutdown_connections 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff220066f60 0x7ff220069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24015b910 0x7ff240155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2401550b0 0x7ff2400cadd0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 --2- 172.21.15.87:0/642963943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2400cfe70 0x7ff2401f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 >> 172.21.15.87:0/642963943 conn(0x7ff240066b70 msgr2=0x7ff24015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 shutdown_connections 2024-04-25T10:22:38.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:38.989+0000 7ff247e00640 1 -- 172.21.15.87:0/642963943 wait complete. 2024-04-25T10:22:39.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:21:40.912468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:21:41.547768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:39.012 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:39.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.145+0000 7fddb0400640 1 Processor -- start 2024-04-25T10:22:39.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.145+0000 7fddb0400640 1 -- start start 2024-04-25T10:22:39.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddb0400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdda8070350 0x7fdda815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddb0400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdda80b31c0 0x7fdda80b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddb0400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda815b080 con 0x7fdda80aef80 2024-04-25T10:22:39.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddb0400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda815adb0 con 0x7fdda8070350 2024-04-25T10:22:39.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddb0400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda815eb60 con 0x7fdda80b31c0 2024-04-25T10:22:39.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddad000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdda8070350 0x7fdda815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddad000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdda8070350 0x7fdda815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54276/0 (socket says 172.21.15.87:54276) 2024-04-25T10:22:39.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddad000640 1 -- 172.21.15.87:0/2761390234 learned_addr learned my addr 172.21.15.87:0/2761390234 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:39.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddad000640 1 -- 172.21.15.87:0/2761390234 >> v2:172.21.15.169:3300/0 conn(0x7fdda80b31c0 msgr2=0x7fdda80b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:39.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddad000640 1 --2- 172.21.15.87:0/2761390234 >> v2:172.21.15.169:3300/0 conn(0x7fdda80b31c0 0x7fdda80b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddad000640 1 -- 172.21.15.87:0/2761390234 >> v1:172.21.15.169:6789/0 conn(0x7fdda80aef80 legacy=0x7fdda815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:39.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.149+0000 7fddad000640 1 -- 172.21.15.87:0/2761390234 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda815ee40 con 0x7fdda8070350 2024-04-25T10:22:39.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddad000640 1 --2- 172.21.15.87:0/2761390234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda8070350 0x7fdda815e480 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fdda80d0750 tx=0x7fdd9401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e718e4748b8157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fdda7e00640 1 -- 172.21.15.87:0/2761390234 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd940ab5f0 con 0x7fdda8070350 2024-04-25T10:22:39.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fdda7e00640 1 -- 172.21.15.87:0/2761390234 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd940b0070 con 0x7fdda8070350 2024-04-25T10:22:39.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fdda7e00640 1 -- 172.21.15.87:0/2761390234 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd940ab9b0 con 0x7fdda8070350 2024-04-25T10:22:39.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- 172.21.15.87:0/2761390234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda8070350 msgr2=0x7fdda815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 --2- 172.21.15.87:0/2761390234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda8070350 0x7fdda815e480 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fdda80d0750 tx=0x7fdd9401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- 172.21.15.87:0/2761390234 shutdown_connections 2024-04-25T10:22:39.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 --2- 172.21.15.87:0/2761390234 >> v2:172.21.15.169:3300/0 conn(0x7fdda80b31c0 0x7fdda80b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 --2- 172.21.15.87:0/2761390234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda8070350 0x7fdda815e480 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- 172.21.15.87:0/2761390234 >> 172.21.15.87:0/2761390234 conn(0x7fdda80669a0 msgr2=0x7fdda8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- 172.21.15.87:0/2761390234 shutdown_connections 2024-04-25T10:22:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- 172.21.15.87:0/2761390234 wait complete. 2024-04-25T10:22:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 Processor -- start 2024-04-25T10:22:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- start start 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdda8070350 0x7fdda8152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda80aef80 0x7fdda80afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdda80b31c0 0x7fdda80b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda81679a0 con 0x7fdda80aef80 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda8167820 con 0x7fdda80b31c0 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddb0400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda8167b20 con 0x7fdda8070350 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.153+0000 7fddad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda80aef80 0x7fdda80afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda80aef80 0x7fdda80afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54282/0 (socket says 172.21.15.87:54282) 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 -- 172.21.15.87:0/4057270257 learned_addr learned my addr 172.21.15.87:0/4057270257 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 -- 172.21.15.87:0/4057270257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdda8070350 msgr2=0x7fdda8152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdda8070350 0x7fdda8152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 -- 172.21.15.87:0/4057270257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdda80b31c0 msgr2=0x7fdda80b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdda80b31c0 0x7fdda80b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 -- 172.21.15.87:0/4057270257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda8153880 con 0x7fdda80aef80 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddad000640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda80aef80 0x7fdda80afe00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fdd940ad040 tx=0x7fdd9401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fdda5600640 1 -- 172.21.15.87:0/4057270257 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd940be5f0 con 0x7fdda80aef80 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fdda5600640 1 -- 172.21.15.87:0/4057270257 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd940be7b0 con 0x7fdda80aef80 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fdda5600640 1 -- 172.21.15.87:0/4057270257 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd940beba0 con 0x7fdda80aef80 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda8153cb0 con 0x7fdda80aef80 2024-04-25T10:22:39.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda8154270 con 0x7fdda80aef80 2024-04-25T10:22:39.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda804ef30 con 0x7fdda80aef80 2024-04-25T10:22:39.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fdda5600640 1 -- 172.21.15.87:0/4057270257 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd940b43f0 con 0x7fdda80aef80 2024-04-25T10:22:39.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fdda5600640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd88066f60 0x7fdd88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fdda5600640 1 -- 172.21.15.87:0/4057270257 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdd9411f0c0 con 0x7fdda80aef80 2024-04-25T10:22:39.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.157+0000 7fddada00640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd88066f60 0x7fdd88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.161+0000 7fddada00640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd88066f60 0x7fdd88069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c007fd0 tx=0x7fdd9c0079a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.161+0000 7fdda5600640 1 -- 172.21.15.87:0/4057270257 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd940160d0 con 0x7fdda80aef80 2024-04-25T10:22:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.321+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdda80b2fa0 con 0x7fdda80aef80 2024-04-25T10:22:39.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.321+0000 7fdda5600640 1 -- 172.21.15.87:0/4057270257 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd940f00c0 con 0x7fdda80aef80 2024-04-25T10:22:39.327 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd88066f60 msgr2=0x7fdd88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd88066f60 0x7fdd88069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c007fd0 tx=0x7fdd9c0079a0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda80aef80 msgr2=0x7fdda80afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda80aef80 0x7fdda80afe00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fdd940ad040 tx=0x7fdd9401ad20 comp rx=0 tx=0).stop 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 shutdown_connections 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd88066f60 0x7fdd88069420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdda80b31c0 0x7fdda80b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda80aef80 0x7fdda80afe00 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 --2- 172.21.15.87:0/4057270257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdda8070350 0x7fdda8152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 >> 172.21.15.87:0/4057270257 conn(0x7fdda80669a0 msgr2=0x7fdda80b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 shutdown_connections 2024-04-25T10:22:39.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.325+0000 7fddb0400640 1 -- 172.21.15.87:0/4057270257 wait complete. 2024-04-25T10:22:39.348 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:22:39.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61c1000640 1 Processor -- start 2024-04-25T10:22:39.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61c1000640 1 -- start start 2024-04-25T10:22:39.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61c1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 0x7f61bc15f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61c1000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc0b1770 con 0x7f61bc0d1830 2024-04-25T10:22:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61c1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc15fec0 con 0x7f61bc0cf980 2024-04-25T10:22:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61c1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1601a0 con 0x7f61bc15bc10 2024-04-25T10:22:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f61bc0d1830 0x7f61bc0d1c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56978/0 (socket says 172.21.15.87:56978) 2024-04-25T10:22:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 -- 172.21.15.87:0/6520112 learned_addr learned my addr 172.21.15.87:0/6520112 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665059018 0 0) 0x7f61bc0b1770 con 0x7f61bc0d1830 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a0003530 con 0x7f61bc0d1830 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b9e00640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 0x7f61bc15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1732107206 0 0) 0x7f61a0003530 con 0x7f61bc0d1830 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 msgr2=0x7f61bc15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 0x7f61bc15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 >> v1:172.21.15.169:6789/0 conn(0x7f61bc0cf980 legacy=0x7f61bc0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc0cc5b0 con 0x7f61bc0d1830 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f61b4001a00 con 0x7f61bc0d1830 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b9e00640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 0x7f61bc15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 >> v1:172.21.15.87:6789/0 conn(0x7f61bc0d1830 legacy=0x7f61bc0d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 0x7f61a00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61a0003fe0 0x7f61a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61a0004d50 0x7f61a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1601a0 con 0x7f61a0004d50 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a000bb30 con 0x7f61a0003fe0 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a000bd30 con 0x7f61bc15bc10 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61a0004d50 0x7f61a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b9e00640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 0x7f61a00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b9400640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61a0003fe0 0x7f61a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 -- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 msgr2=0x7f61a00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc15bc10 0x7f61a00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 -- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61a0003fe0 msgr2=0x7f61a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:39.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61a0003fe0 0x7f61a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61ba800640 1 -- 172.21.15.87:0/6520112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0003530 con 0x7f61a0004d50 2024-04-25T10:22:39.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.485+0000 7f61b9400640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61a0003fe0 0x7f61a0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61ba800640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61a0004d50 0x7f61a00051d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f61b400a860 tx=0x7f61b401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86986048e187946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b400a480 con 0x7f61a0004d50 2024-04-25T10:22:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b40ad630 con 0x7f61a0004d50 2024-04-25T10:22:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61b8a00640 1 -- 172.21.15.87:0/6520112 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40adaa0 con 0x7f61a0004d50 2024-04-25T10:22:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61c1000640 1 -- 172.21.15.87:0/6520112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61a0004d50 msgr2=0x7f61a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61c1000640 1 --2- 172.21.15.87:0/6520112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61a0004d50 0x7f61a00051d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f61b400a860 tx=0x7f61b401d380 comp rx=0 tx=0).stop 2024-04-25T10:22:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61ba800640 1 -- 172.21.15.87:0/6520112 reap_dead start 2024-04-25T10:22:39.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61c1000640 1 -- 172.21.15.87:0/6520112 shutdown_connections 2024-04-25T10:22:39.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61c1000640 1 -- 172.21.15.87:0/6520112 >> 172.21.15.87:0/6520112 conn(0x7f61bc09a9d0 msgr2=0x7f61bc09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61c1000640 1 -- 172.21.15.87:0/6520112 shutdown_connections 2024-04-25T10:22:39.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.489+0000 7f61c1000640 1 -- 172.21.15.87:0/6520112 wait complete. 2024-04-25T10:22:39.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 Processor -- start 2024-04-25T10:22:39.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 -- start start 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0b7270 0x7f61bc15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61bc1f3f00 0x7f61bc1f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0ce270 0x7f61bc0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc15f600 con 0x7f61bc1f3f00 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc160480 con 0x7f61bc0b7270 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc15f780 con 0x7f61bc0ce270 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61bc1f3f00 0x7f61bc1f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61bc1f3f00 0x7f61bc1f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54296/0 (socket says 172.21.15.87:54296) 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 -- 172.21.15.87:0/3281483445 learned_addr learned my addr 172.21.15.87:0/3281483445 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9e00640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0b7270 0x7f61bc15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61ba800640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0ce270 0x7f61bc0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 -- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0ce270 msgr2=0x7f61bc0ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0ce270 0x7f61bc0ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 -- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0b7270 msgr2=0x7f61bc15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0b7270 0x7f61bc15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 -- 172.21.15.87:0/3281483445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc1f4990 con 0x7f61bc1f3f00 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9e00640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0b7270 0x7f61bc15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b9400640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61bc1f3f00 0x7f61bc1f4380 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e4e0 tx=0x7f61a800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b2000640 1 -- 172.21.15.87:0/3281483445 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80053a0 con 0x7f61bc1f3f00 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b2000640 1 -- 172.21.15.87:0/3281483445 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a8018070 con 0x7f61bc1f3f00 2024-04-25T10:22:39.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc1f4c70 con 0x7f61bc1f3f00 2024-04-25T10:22:39.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61b2000640 1 -- 172.21.15.87:0/3281483445 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8003820 con 0x7f61bc1f3f00 2024-04-25T10:22:39.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.493+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc15df20 con 0x7f61bc1f3f00 2024-04-25T10:22:39.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.497+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc0cec30 con 0x7f61bc1f3f00 2024-04-25T10:22:39.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.497+0000 7f61b2000640 1 -- 172.21.15.87:0/3281483445 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61a80039c0 con 0x7f61bc1f3f00 2024-04-25T10:22:39.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.497+0000 7f61b2000640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f619c066f60 0x7f619c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.497+0000 7f61b2000640 1 -- 172.21.15.87:0/3281483445 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f61a8084d00 con 0x7f61bc1f3f00 2024-04-25T10:22:39.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.497+0000 7f61b9e00640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f619c066f60 0x7f619c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.497+0000 7f61b9e00640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f619c066f60 0x7f619c069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009990 tx=0x7f61ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.501+0000 7f61b2000640 1 -- 172.21.15.87:0/3281483445 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61a8055c70 con 0x7f61bc1f3f00 2024-04-25T10:22:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.653+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61bc04ef30 con 0x7f61bc1f3f00 2024-04-25T10:22:39.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.653+0000 7f61b2000640 1 -- 172.21.15.87:0/3281483445 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61a8055570 con 0x7f61bc1f3f00 2024-04-25T10:22:39.658 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:22:39.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f619c066f60 msgr2=0x7f619c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f619c066f60 0x7f619c069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009990 tx=0x7f61ac008040 comp rx=0 tx=0).stop 2024-04-25T10:22:39.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61bc1f3f00 msgr2=0x7f61bc1f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61bc1f3f00 0x7f61bc1f4380 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e4e0 tx=0x7f61a800e9a0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 shutdown_connections 2024-04-25T10:22:39.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f619c066f60 0x7f619c069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0ce270 0x7f61bc0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61bc1f3f00 0x7f61bc1f4380 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 --2- 172.21.15.87:0/3281483445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0b7270 0x7f61bc15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 >> 172.21.15.87:0/3281483445 conn(0x7f61bc09a9d0 msgr2=0x7f61bc0b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 shutdown_connections 2024-04-25T10:22:39.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.657+0000 7f61c1000640 1 -- 172.21.15.87:0/3281483445 wait complete. 2024-04-25T10:22:39.678 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:22:39.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d90200640 1 Processor -- start 2024-04-25T10:22:39.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d90200640 1 -- start start 2024-04-25T10:22:39.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d90200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3d88159540 0x7f3d880ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d90200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d880721f0 0x7f3d88070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d90200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d88159920 con 0x7f3d880aebd0 2024-04-25T10:22:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d90200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8815c2f0 con 0x7f3d880721f0 2024-04-25T10:22:39.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d90200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8815c5d0 con 0x7f3d88159540 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d880721f0 0x7f3d88070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d880721f0 0x7f3d88070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54298/0 (socket says 172.21.15.87:54298) 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8e200640 1 -- 172.21.15.87:0/365980124 learned_addr learned my addr 172.21.15.87:0/365980124 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8d800640 1 --2- 172.21.15.87:0/365980124 >> v2:172.21.15.169:3300/0 conn(0x7f3d88159540 0x7f3d880ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8e200640 1 -- 172.21.15.87:0/365980124 >> v2:172.21.15.169:3300/0 conn(0x7f3d88159540 msgr2=0x7f3d880ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8e200640 1 --2- 172.21.15.87:0/365980124 >> v2:172.21.15.169:3300/0 conn(0x7f3d88159540 0x7f3d880ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8e200640 1 -- 172.21.15.87:0/365980124 >> v1:172.21.15.169:6789/0 conn(0x7f3d880aebd0 legacy=0x7f3d88071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d8e200640 1 -- 172.21.15.87:0/365980124 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8815c8b0 con 0x7f3d880721f0 2024-04-25T10:22:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.813+0000 7f3d87e00640 1 -- 172.21.15.87:0/365980124 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013252732 0 0) 0x7f3d88159920 con 0x7f3d880aebd0 2024-04-25T10:22:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d8e200640 1 --2- 172.21.15.87:0/365980124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880721f0 0x7f3d88070350 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3d880b4c20 tx=0x7f3d8001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7c844609af6243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d87e00640 1 -- 172.21.15.87:0/365980124 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800a57d0 con 0x7f3d880721f0 2024-04-25T10:22:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d87e00640 1 -- 172.21.15.87:0/365980124 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d800a5970 con 0x7f3d880721f0 2024-04-25T10:22:39.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d87e00640 1 -- 172.21.15.87:0/365980124 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800a5d40 con 0x7f3d880721f0 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 -- 172.21.15.87:0/365980124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880721f0 msgr2=0x7f3d88070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 --2- 172.21.15.87:0/365980124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880721f0 0x7f3d88070350 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3d880b4c20 tx=0x7f3d8001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 -- 172.21.15.87:0/365980124 shutdown_connections 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 --2- 172.21.15.87:0/365980124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880721f0 0x7f3d88070350 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 --2- 172.21.15.87:0/365980124 >> v2:172.21.15.169:3300/0 conn(0x7f3d88159540 0x7f3d880ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 -- 172.21.15.87:0/365980124 >> 172.21.15.87:0/365980124 conn(0x7f3d880669a0 msgr2=0x7f3d88066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 -- 172.21.15.87:0/365980124 shutdown_connections 2024-04-25T10:22:39.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.817+0000 7f3d90200640 1 -- 172.21.15.87:0/365980124 wait complete. 2024-04-25T10:22:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 Processor -- start 2024-04-25T10:22:39.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 -- start start 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d880721f0 0x7f3d8813c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880aebd0 0x7f3d88152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d88159540 0x7f3d88159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88071220 con 0x7f3d880aebd0 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880710a0 con 0x7f3d88159540 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880713a0 con 0x7f3d880721f0 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880aebd0 0x7f3d88152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d880721f0 0x7f3d8813c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880aebd0 0x7f3d88152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54300/0 (socket says 172.21.15.87:54300) 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8ce00640 1 -- 172.21.15.87:0/1624341377 learned_addr learned my addr 172.21.15.87:0/1624341377 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8e200640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d88159540 0x7f3d88159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8d800640 1 -- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d88159540 msgr2=0x7f3d88159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8d800640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d88159540 0x7f3d88159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8d800640 1 -- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880aebd0 msgr2=0x7f3d88152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8d800640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880aebd0 0x7f3d88152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8d800640 1 -- 172.21.15.87:0/1624341377 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8813f670 con 0x7f3d880721f0 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8ce00640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880aebd0 0x7f3d88152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:39.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d8d800640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d880721f0 0x7f3d8813c810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00e8a0 tx=0x7f3d7c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d85600640 1 -- 172.21.15.87:0/1624341377 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c017070 con 0x7f3d880721f0 2024-04-25T10:22:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d85600640 1 -- 172.21.15.87:0/1624341377 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c00c940 con 0x7f3d880721f0 2024-04-25T10:22:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d85600640 1 -- 172.21.15.87:0/1624341377 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00cd10 con 0x7f3d880721f0 2024-04-25T10:22:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8813f950 con 0x7f3d880721f0 2024-04-25T10:22:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8813ff10 con 0x7f3d880721f0 2024-04-25T10:22:39.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.821+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8813d170 con 0x7f3d880721f0 2024-04-25T10:22:39.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.825+0000 7f3d85600640 1 -- 172.21.15.87:0/1624341377 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d7c01a070 con 0x7f3d880721f0 2024-04-25T10:22:39.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.825+0000 7f3d85600640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d68066f60 0x7f3d68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:39.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.825+0000 7f3d85600640 1 -- 172.21.15.87:0/1624341377 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3d7c0852c0 con 0x7f3d880721f0 2024-04-25T10:22:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.825+0000 7f3d8ce00640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d68066f60 0x7f3d68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.825+0000 7f3d8ce00640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d68066f60 0x7f3d68069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3d88154240 tx=0x7f3d74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.829+0000 7f3d85600640 1 -- 172.21.15.87:0/1624341377 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d7c055b50 con 0x7f3d880721f0 2024-04-25T10:22:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.981+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d8815eed0 con 0x7f3d880721f0 2024-04-25T10:22:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.981+0000 7f3d85600640 1 -- 172.21.15.87:0/1624341377 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d7c059bf0 con 0x7f3d880721f0 2024-04-25T10:22:39.986 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:22:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d68066f60 msgr2=0x7f3d68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d68066f60 0x7f3d68069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3d88154240 tx=0x7f3d74009040 comp rx=0 tx=0).stop 2024-04-25T10:22:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d880721f0 msgr2=0x7f3d8813c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d880721f0 0x7f3d8813c810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00e8a0 tx=0x7f3d7c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 shutdown_connections 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3d68066f60 0x7f3d68069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d88159540 0x7f3d88159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d880aebd0 0x7f3d88152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 --2- 172.21.15.87:0/1624341377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d880721f0 0x7f3d8813c810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 >> 172.21.15.87:0/1624341377 conn(0x7f3d880669a0 msgr2=0x7f3d8815fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 shutdown_connections 2024-04-25T10:22:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:39.985+0000 7f3d90200640 1 -- 172.21.15.87:0/1624341377 wait complete. 2024-04-25T10:22:40.008 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:22:40.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7f1800640 1 Processor -- start 2024-04-25T10:22:40.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7f1800640 1 -- start start 2024-04-25T10:22:40.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7f1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec0d0040 0x7ff7ec15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7f1800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff7ec15b740 0x7ff7ec0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7f1800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ec071050 con 0x7ff7ec1550b0 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7f1800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec15d040 con 0x7ff7ec15b740 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7f1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0712f0 con 0x7ff7ec0d0040 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7ea000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec0d0040 0x7ff7ec15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7eaa00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff7ec1550b0 0x7ff7ec0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:44862/0 (socket says 172.21.15.87:44862) 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7eb400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff7ec15b740 0x7ff7ec0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7eaa00640 1 -- 172.21.15.87:0/3399790998 learned_addr learned my addr 172.21.15.87:0/3399790998 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:40.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7ea000640 1 -- 172.21.15.87:0/3399790998 >> v2:172.21.15.169:3300/0 conn(0x7ff7ec15b740 msgr2=0x7ff7ec0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:40.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7ea000640 1 --2- 172.21.15.87:0/3399790998 >> v2:172.21.15.169:3300/0 conn(0x7ff7ec15b740 0x7ff7ec0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7ea000640 1 -- 172.21.15.87:0/3399790998 >> v1:172.21.15.169:6789/0 conn(0x7ff7ec1550b0 legacy=0x7ff7ec0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7ea000640 1 -- 172.21.15.87:0/3399790998 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0715d0 con 0x7ff7ec0d0040 2024-04-25T10:22:40.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.141+0000 7ff7eb400640 1 --2- 172.21.15.87:0/3399790998 >> v2:172.21.15.169:3300/0 conn(0x7ff7ec15b740 0x7ff7ec0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:40.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7ea000640 1 --2- 172.21.15.87:0/3399790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec0d0040 0x7ff7ec15b200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80097c0 tx=0x7ff7d801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a18305e4bbf67c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7e9600640 1 -- 172.21.15.87:0/3399790998 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80a7070 con 0x7ff7ec0d0040 2024-04-25T10:22:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7e9600640 1 -- 172.21.15.87:0/3399790998 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d80b0070 con 0x7ff7ec0d0040 2024-04-25T10:22:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7e9600640 1 -- 172.21.15.87:0/3399790998 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80ab890 con 0x7ff7ec0d0040 2024-04-25T10:22:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 -- 172.21.15.87:0/3399790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec0d0040 msgr2=0x7ff7ec15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 --2- 172.21.15.87:0/3399790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec0d0040 0x7ff7ec15b200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80097c0 tx=0x7ff7d801c470 comp rx=0 tx=0).stop 2024-04-25T10:22:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 -- 172.21.15.87:0/3399790998 shutdown_connections 2024-04-25T10:22:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 --2- 172.21.15.87:0/3399790998 >> v2:172.21.15.169:3300/0 conn(0x7ff7ec15b740 0x7ff7ec0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 --2- 172.21.15.87:0/3399790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec0d0040 0x7ff7ec15b200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 -- 172.21.15.87:0/3399790998 >> 172.21.15.87:0/3399790998 conn(0x7ff7ec066b70 msgr2=0x7ff7ec066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:40.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 -- 172.21.15.87:0/3399790998 shutdown_connections 2024-04-25T10:22:40.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.145+0000 7ff7f1800640 1 -- 172.21.15.87:0/3399790998 wait complete. 2024-04-25T10:22:40.151 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:22:40.149+0000 7fde0d000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:22:40.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 Processor -- start 2024-04-25T10:22:40.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 -- start start 2024-04-25T10:22:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0d0040 0x7ff7ec0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec1550b0 0x7ff7ec13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7ec15b740 0x7ff7ec14bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec15bcd0 con 0x7ff7ec15b740 2024-04-25T10:22:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec15bb50 con 0x7ff7ec0d0040 2024-04-25T10:22:40.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec15c030 con 0x7ff7ec1550b0 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7ec15b740 0x7ff7ec14bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7ec15b740 0x7ff7ec14bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54302/0 (socket says 172.21.15.87:54302) 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 -- 172.21.15.87:0/818325391 learned_addr learned my addr 172.21.15.87:0/818325391 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eaa00640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0d0040 0x7ff7ec0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7ea000640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec1550b0 0x7ff7ec13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 -- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec1550b0 msgr2=0x7ff7ec13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec1550b0 0x7ff7ec13bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 -- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0d0040 msgr2=0x7ff7ec0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0d0040 0x7ff7ec0cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 -- 172.21.15.87:0/818325391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec14b640 con 0x7ff7ec15b740 2024-04-25T10:22:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7eb400640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7ec15b740 0x7ff7ec14bb20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009e60 tx=0x7ff7e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7e2a00640 1 -- 172.21.15.87:0/818325391 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4018030 con 0x7ff7ec15b740 2024-04-25T10:22:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7e2a00640 1 -- 172.21.15.87:0/818325391 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400cdf0 con 0x7ff7ec15b740 2024-04-25T10:22:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7e2a00640 1 -- 172.21.15.87:0/818325391 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4003bc0 con 0x7ff7ec15b740 2024-04-25T10:22:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec14f0a0 con 0x7ff7ec15b740 2024-04-25T10:22:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.149+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec14f660 con 0x7ff7ec15b740 2024-04-25T10:22:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.153+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec04ef30 con 0x7ff7ec15b740 2024-04-25T10:22:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.153+0000 7ff7e2a00640 1 -- 172.21.15.87:0/818325391 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7e401b070 con 0x7ff7ec15b740 2024-04-25T10:22:40.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.153+0000 7ff7e2a00640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7cc066f60 0x7ff7cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.153+0000 7ff7e2a00640 1 -- 172.21.15.87:0/818325391 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff7e4085b20 con 0x7ff7ec15b740 2024-04-25T10:22:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.153+0000 7ff7eaa00640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7cc066f60 0x7ff7cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.153+0000 7ff7eaa00640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7cc066f60 0x7ff7cc069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc000f70 tx=0x7ff7dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:40.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.157+0000 7ff7e2a00640 1 -- 172.21.15.87:0/818325391 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7e4010040 con 0x7ff7ec15b740 2024-04-25T10:22:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.321+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff7ec0b4800 con 0x7ff7ec15b740 2024-04-25T10:22:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.321+0000 7ff7e2a00640 1 -- 172.21.15.87:0/818325391 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v127) v1 ==== 80+0+120 (secure 0 0 0) 0x7ff7e4056a00 con 0x7ff7ec15b740 2024-04-25T10:22:40.327 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.169:6834/85260299 2024-04-26T10:22:32.952837+0000 2024-04-25T10:22:40.327 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.169:6832/85260299 2024-04-26T10:22:32.952837+0000 2024-04-25T10:22:40.327 INFO:teuthology.orchestra.run.smithi087.stderr:listed 2 entries 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7cc066f60 msgr2=0x7ff7cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7cc066f60 0x7ff7cc069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc000f70 tx=0x7ff7dc009040 comp rx=0 tx=0).stop 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7ec15b740 msgr2=0x7ff7ec14bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7ec15b740 0x7ff7ec14bb20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009e60 tx=0x7ff7e400c490 comp rx=0 tx=0).stop 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 shutdown_connections 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff7cc066f60 0x7ff7cc069420 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7ec15b740 0x7ff7ec14bb20 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7ec1550b0 0x7ff7ec13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 --2- 172.21.15.87:0/818325391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0d0040 0x7ff7ec0cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 >> 172.21.15.87:0/818325391 conn(0x7ff7ec066b70 msgr2=0x7ff7ec06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 shutdown_connections 2024-04-25T10:22:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.325+0000 7ff7f1800640 1 -- 172.21.15.87:0/818325391 wait complete. 2024-04-25T10:22:40.347 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:22:40.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:40.348 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:22:40.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:22:40.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:40.373 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:22:40.427 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:22:40.427 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:40.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:40.436 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:22:40.436 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:40.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:40.483 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:22:40.483 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:40.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:40.492 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:40.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455fc00640 1 Processor -- start 2024-04-25T10:22:40.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455fc00640 1 -- start start 2024-04-25T10:22:40.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f45580d0920 0x7f45580d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455fc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580d2490 con 0x7f45580d1ce0 2024-04-25T10:22:40.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455fc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580d2770 con 0x7f455815df00 2024-04-25T10:22:40.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45580d2a50 con 0x7f45580d0920 2024-04-25T10:22:40.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f4557e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f45580d0920 0x7f45580d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455dc00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f45580d1ce0 0x7f45580b71e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56986/0 (socket says 172.21.15.87:56986) 2024-04-25T10:22:40.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f455dc00640 1 -- 172.21.15.87:0/2404980363 learned_addr learned my addr 172.21.15.87:0/2404980363 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:40.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f4557e00640 1 -- 172.21.15.87:0/2404980363 >> v1:172.21.15.169:6789/0 conn(0x7f455815df00 legacy=0x7f45580d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f4557e00640 1 -- 172.21.15.87:0/2404980363 >> v1:172.21.15.87:6789/0 conn(0x7f45580d1ce0 legacy=0x7f45580b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f4557e00640 1 -- 172.21.15.87:0/2404980363 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45580d2d30 con 0x7f45580d0920 2024-04-25T10:22:40.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.661+0000 7f4557400640 1 -- 172.21.15.87:0/2404980363 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141363617 0 0) 0x7f45580d2490 con 0x7f45580d1ce0 2024-04-25T10:22:40.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f4557e00640 1 --2- 172.21.15.87:0/2404980363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45580d0920 0x7f45580d17a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f45440097c0 tx=0x7f454401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2639b55e94a5af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:40.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f4557400640 1 -- 172.21.15.87:0/2404980363 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440b1070 con 0x7f45580d0920 2024-04-25T10:22:40.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f4557400640 1 -- 172.21.15.87:0/2404980363 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45440ac600 con 0x7f45580d0920 2024-04-25T10:22:40.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f4557400640 1 -- 172.21.15.87:0/2404980363 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440ac9d0 con 0x7f45580d0920 2024-04-25T10:22:40.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f455fc00640 1 -- 172.21.15.87:0/2404980363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45580d0920 msgr2=0x7f45580d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f455fc00640 1 --2- 172.21.15.87:0/2404980363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45580d0920 0x7f45580d17a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f45440097c0 tx=0x7f454401c470 comp rx=0 tx=0).stop 2024-04-25T10:22:40.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f455fc00640 1 -- 172.21.15.87:0/2404980363 shutdown_connections 2024-04-25T10:22:40.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f455fc00640 1 --2- 172.21.15.87:0/2404980363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45580d0920 0x7f45580d17a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f455fc00640 1 -- 172.21.15.87:0/2404980363 >> 172.21.15.87:0/2404980363 conn(0x7f455806b290 msgr2=0x7f45580669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:40.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f455fc00640 1 -- 172.21.15.87:0/2404980363 shutdown_connections 2024-04-25T10:22:40.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.665+0000 7f455fc00640 1 -- 172.21.15.87:0/2404980363 wait complete. 2024-04-25T10:22:40.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 Processor -- start 2024-04-25T10:22:40.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 -- start start 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45580d0920 0x7f455814b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45580d1ce0 0x7f4558160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f455815df00 0x7f45580cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45580b8280 con 0x7f455815df00 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45580b8100 con 0x7f45580d0920 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45580b8400 con 0x7f45580d1ce0 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f455815df00 0x7f45580cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f455815df00 0x7f45580cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54318/0 (socket says 172.21.15.87:54318) 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 -- 172.21.15.87:0/186017819 learned_addr learned my addr 172.21.15.87:0/186017819 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f4557e00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45580d1ce0 0x7f4558160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455d200640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45580d0920 0x7f455814b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 -- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45580d1ce0 msgr2=0x7f4558160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45580d1ce0 0x7f4558160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 -- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45580d0920 msgr2=0x7f455814b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45580d0920 0x7f455814b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 -- 172.21.15.87:0/186017819 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455814b820 con 0x7f455815df00 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455dc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f455815df00 0x7f45580cce90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f455000c7a0 tx=0x7f455000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f4554c00640 1 -- 172.21.15.87:0/186017819 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550018070 con 0x7f455815df00 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f4554c00640 1 -- 172.21.15.87:0/186017819 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455000a640 con 0x7f455815df00 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45581611f0 con 0x7f455815df00 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f4554c00640 1 -- 172.21.15.87:0/186017819 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455000aab0 con 0x7f455815df00 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.669+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45580cb070 con 0x7f455815df00 2024-04-25T10:22:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.673+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455815c300 con 0x7f455815df00 2024-04-25T10:22:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.673+0000 7f4554c00640 1 -- 172.21.15.87:0/186017819 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f455001b070 con 0x7f455815df00 2024-04-25T10:22:40.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.673+0000 7f4554c00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4538066f60 0x7f4538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:40.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.673+0000 7f4554c00640 1 -- 172.21.15.87:0/186017819 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4550084570 con 0x7f455815df00 2024-04-25T10:22:40.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.673+0000 7f455d200640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4538066f60 0x7f4538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:40.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.673+0000 7f455d200640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4538066f60 0x7f4538069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4548007fd0 tx=0x7f4548009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:40.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.677+0000 7f4554c00640 1 -- 172.21.15.87:0/186017819 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4550054f30 con 0x7f455815df00 2024-04-25T10:22:40.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f455804ef30 con 0x7f455815df00 2024-04-25T10:22:40.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f4554c00640 1 -- 172.21.15.87:0/186017819 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f4550058050 con 0x7f455815df00 2024-04-25T10:22:40.868 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:40.869 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:22:34.985474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9111},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9111":{"gid":9111,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":17,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:40.869 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 93 2024-04-25T10:22:40.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4538066f60 msgr2=0x7f4538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4538066f60 0x7f4538069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4548007fd0 tx=0x7f4548009040 comp rx=0 tx=0).stop 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f455815df00 msgr2=0x7f45580cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f455815df00 0x7f45580cce90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f455000c7a0 tx=0x7f455000cc60 comp rx=0 tx=0).stop 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 shutdown_connections 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4538066f60 0x7f4538069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f455815df00 0x7f45580cce90 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45580d1ce0 0x7f4558160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 --2- 172.21.15.87:0/186017819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45580d0920 0x7f455814b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 >> 172.21.15.87:0/186017819 conn(0x7f455806b290 msgr2=0x7f4558151490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 shutdown_connections 2024-04-25T10:22:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:40.865+0000 7f455fc00640 1 -- 172.21.15.87:0/186017819 wait complete. 2024-04-25T10:22:40.890 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:41.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae2a00640 1 Processor -- start 2024-04-25T10:22:41.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae2a00640 1 -- start start 2024-04-25T10:22:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 0x7f4adc0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae2a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc0b1870 con 0x7f4adc0b13e0 2024-04-25T10:22:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc0b1af0 con 0x7f4adc0d1820 2024-04-25T10:22:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0b1dd0 con 0x7f4adc15e050 2024-04-25T10:22:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae0a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4adc0b13e0 0x7f4adc0cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56998/0 (socket says 172.21.15.87:56998) 2024-04-25T10:22:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae0a00640 1 -- 172.21.15.87:0/730101944 learned_addr learned my addr 172.21.15.87:0/730101944 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adb400640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 0x7f4adc0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365926341 0 0) 0x7f4ad40014a0 con 0x7f4adc0b13e0 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac0003530 con 0x7f4adc0b13e0 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550411443 0 0) 0x7f4adc0b1af0 con 0x7f4adc0d1820 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ad40014a0 con 0x7f4adc0d1820 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 851066674 0 0) 0x7f4ac0003530 con 0x7f4adc0b13e0 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 msgr2=0x7f4adc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 0x7f4adc0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 >> v1:172.21.15.169:6789/0 conn(0x7f4adc0d1820 legacy=0x7f4adc0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc0b20b0 con 0x7f4adc0b13e0 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f4ad4001bd0 con 0x7f4adc0b13e0 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 >> v1:172.21.15.87:6789/0 conn(0x7f4adc0b13e0 legacy=0x7f4adc0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac0003b80 0x7f4ac0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 0x7f4ac0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0b1dd0 con 0x7f4ac0004810 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac000bb60 con 0x7f4ac0003b80 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac000bd30 con 0x7f4adc15e050 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adb400640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adbe00640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac0003b80 0x7f4ac0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4ae0a00640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 0x7f4ac0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adb400640 1 -- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 msgr2=0x7f4ac0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adb400640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc15e050 0x7f4ac0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adb400640 1 -- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac0003b80 msgr2=0x7f4ac0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adb400640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac0003b80 0x7f4ac0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.025+0000 7f4adb400640 1 -- 172.21.15.87:0/730101944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac0003530 con 0x7f4ac0004810 2024-04-25T10:22:41.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4adb400640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009ae0 tx=0x7f4ac801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325e9ed87aa36b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80b2070 con 0x7f4ac0004810 2024-04-25T10:22:41.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac80a5510 con 0x7f4ac0004810 2024-04-25T10:22:41.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4adaa00640 1 -- 172.21.15.87:0/730101944 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80096b0 con 0x7f4ac0004810 2024-04-25T10:22:41.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4ae2a00640 1 -- 172.21.15.87:0/730101944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 msgr2=0x7f4ac0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4ae2a00640 1 --2- 172.21.15.87:0/730101944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4ac0004c90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009ae0 tx=0x7f4ac801ca30 comp rx=0 tx=0).stop 2024-04-25T10:22:41.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4ae0a00640 1 -- 172.21.15.87:0/730101944 reap_dead start 2024-04-25T10:22:41.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4ae2a00640 1 -- 172.21.15.87:0/730101944 shutdown_connections 2024-04-25T10:22:41.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4ae2a00640 1 -- 172.21.15.87:0/730101944 >> 172.21.15.87:0/730101944 conn(0x7f4adc09a9d0 msgr2=0x7f4adc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4ae2a00640 1 -- 172.21.15.87:0/730101944 shutdown_connections 2024-04-25T10:22:41.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.029+0000 7f4ae2a00640 1 -- 172.21.15.87:0/730101944 wait complete. 2024-04-25T10:22:41.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 Processor -- start 2024-04-25T10:22:41.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 -- start start 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4adc1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc1510d0 0x7f4adc1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4adc156620 0x7f4adc15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0d0fa0 con 0x7f4ac0004810 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0d0e20 con 0x7f4adc156620 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1699a0 con 0x7f4adc1510d0 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4adc1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4adc1506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54344/0 (socket says 172.21.15.87:54344) 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 -- 172.21.15.87:0/1832476791 learned_addr learned my addr 172.21.15.87:0/1832476791 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae0a00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4adc156620 0x7f4adc15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adb400640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc1510d0 0x7f4adc1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 -- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc1510d0 msgr2=0x7f4adc1560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc1510d0 0x7f4adc1560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 -- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4adc156620 msgr2=0x7f4adc15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4adc156620 0x7f4adc15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 -- 172.21.15.87:0/1832476791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc166c60 con 0x7f4ac0004810 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4adbe00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4adc1506d0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e550 tx=0x7f4acc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ad3e00640 1 -- 172.21.15.87:0/1832476791 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0052e0 con 0x7f4ac0004810 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ad3e00640 1 -- 172.21.15.87:0/1832476791 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc018070 con 0x7f4ac0004810 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc166f40 con 0x7f4ac0004810 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ad3e00640 1 -- 172.21.15.87:0/1832476791 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc003840 con 0x7f4ac0004810 2024-04-25T10:22:41.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc167500 con 0x7f4ac0004810 2024-04-25T10:22:41.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.033+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc04ef30 con 0x7f4ac0004810 2024-04-25T10:22:41.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.037+0000 7f4ad3e00640 1 -- 172.21.15.87:0/1832476791 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4acc0039e0 con 0x7f4ac0004810 2024-04-25T10:22:41.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.037+0000 7f4ad3e00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4abc066f60 0x7f4abc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.037+0000 7f4ad3e00640 1 -- 172.21.15.87:0/1832476791 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4acc084f30 con 0x7f4ac0004810 2024-04-25T10:22:41.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.037+0000 7f4adb400640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4abc066f60 0x7f4abc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.037+0000 7f4adb400640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4abc066f60 0x7f4abc069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80af000 tx=0x7f4ac80046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.041+0000 7f4ad3e00640 1 -- 172.21.15.87:0/1832476791 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4acc055e10 con 0x7f4ac0004810 2024-04-25T10:22:41.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4adc06b480 con 0x7f4ac0004810 2024-04-25T10:22:41.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ad3e00640 1 -- 172.21.15.87:0/1832476791 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16836 (secure 0 0 0) 0x7f4acc055710 con 0x7f4ac0004810 2024-04-25T10:22:41.200 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:41.201 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":127,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:32.952911+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:40.912468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:41.547768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/85260299":"2024-04-26T10:22:32.952837+0000","172.21.15.169:6832/85260299":"2024-04-26T10:22:32.952837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4abc066f60 msgr2=0x7f4abc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4abc066f60 0x7f4abc069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80af000 tx=0x7f4ac80046d0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 msgr2=0x7f4adc1506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4adc1506d0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e550 tx=0x7f4acc00ea10 comp rx=0 tx=0).stop 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 shutdown_connections 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4abc066f60 0x7f4abc069420 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4adc156620 0x7f4adc15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4adc1510d0 0x7f4adc1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 --2- 172.21.15.87:0/1832476791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ac0004810 0x7f4adc1506d0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 >> 172.21.15.87:0/1832476791 conn(0x7f4adc09a9d0 msgr2=0x7f4adc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 shutdown_connections 2024-04-25T10:22:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.197+0000 7f4ae2a00640 1 -- 172.21.15.87:0/1832476791 wait complete. 2024-04-25T10:22:41.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:21:40.912468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:21:41.547768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:41.222 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:41.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f45a7600640 1 Processor -- start 2024-04-25T10:22:41.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f45a7600640 1 -- start start 2024-04-25T10:22:41.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f45a7600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f45a00b38d0 0x7f45a015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f45a7600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a015b080 con 0x7f45a0070350 2024-04-25T10:22:41.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f45a7600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a015adb0 con 0x7f45a00aef80 2024-04-25T10:22:41.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f45a7600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a015eb60 con 0x7f45a00b38d0 2024-04-25T10:22:41.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f45a0070350 0x7f45a00b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57008/0 (socket says 172.21.15.87:57008) 2024-04-25T10:22:41.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459fe00640 1 -- 172.21.15.87:0/316424970 learned_addr learned my addr 172.21.15.87:0/316424970 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f45a5600640 1 --2- 172.21.15.87:0/316424970 >> v2:172.21.15.169:3300/0 conn(0x7f45a00b38d0 0x7f45a015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652934716 0 0) 0x7f45a015b080 con 0x7f45a0070350 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4584003530 con 0x7f45a0070350 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2282355807 0 0) 0x7f4584003530 con 0x7f45a0070350 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 >> v2:172.21.15.169:3300/0 conn(0x7f45a00b38d0 msgr2=0x7f45a015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 --2- 172.21.15.87:0/316424970 >> v2:172.21.15.169:3300/0 conn(0x7f45a00b38d0 0x7f45a015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 >> v1:172.21.15.169:6789/0 conn(0x7f45a00aef80 legacy=0x7f45a015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a015ee40 con 0x7f45a0070350 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f458c001680 con 0x7f45a0070350 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 >> v1:172.21.15.87:6789/0 conn(0x7f45a0070350 legacy=0x7f45a00b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45840039c0 0x7f4584003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45840043c0 0x7f4584004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4584005090 0x7f458400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.357+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a015eb60 con 0x7f45840039c0 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458400bff0 con 0x7f45840043c0 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458400c1f0 con 0x7f4584005090 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a4c00640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45840039c0 0x7f4584003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a5600640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4584005090 0x7f458400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f459fe00640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45840043c0 0x7f4584004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a4c00640 1 -- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4584005090 msgr2=0x7f458400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a4c00640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4584005090 0x7f458400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a4c00640 1 -- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45840043c0 msgr2=0x7f4584004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a4c00640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45840043c0 0x7f4584004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a4c00640 1 -- 172.21.15.87:0/316424970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4584003530 con 0x7f45840039c0 2024-04-25T10:22:41.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a5600640 1 -- 172.21.15.87:0/316424970 reap_dead start 2024-04-25T10:22:41.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f45a4c00640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45840039c0 0x7f4584003da0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4590002010 tx=0x7f459001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc9639d4a342a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45900b2070 con 0x7f45840039c0 2024-04-25T10:22:41.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45900a5770 con 0x7f45840039c0 2024-04-25T10:22:41.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.361+0000 7f459f400640 1 -- 172.21.15.87:0/316424970 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590001c10 con 0x7f45840039c0 2024-04-25T10:22:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- 172.21.15.87:0/316424970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45840039c0 msgr2=0x7f4584003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45840039c0 0x7f4584003da0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4590002010 tx=0x7f459001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- 172.21.15.87:0/316424970 shutdown_connections 2024-04-25T10:22:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 --2- 172.21.15.87:0/316424970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45840039c0 0x7f4584003da0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- 172.21.15.87:0/316424970 >> 172.21.15.87:0/316424970 conn(0x7f45a00669a0 msgr2=0x7f45a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- 172.21.15.87:0/316424970 shutdown_connections 2024-04-25T10:22:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- 172.21.15.87:0/316424970 wait complete. 2024-04-25T10:22:41.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 Processor -- start 2024-04-25T10:22:41.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- start start 2024-04-25T10:22:41.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45840039c0 0x7f45a0155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a0151ad0 0x7f45a0151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a0156e60 0x7f45a01572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b35d0 con 0x7f45a0151ad0 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a015bd90 con 0x7f45a0156e60 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a7600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b3750 con 0x7f45840039c0 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a0151ad0 0x7f45a0151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a0151ad0 0x7f45a0151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54370/0 (socket says 172.21.15.87:54370) 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 -- 172.21.15.87:0/1828261286 learned_addr learned my addr 172.21.15.87:0/1828261286 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a5600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a0156e60 0x7f45a01572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a4c00640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45840039c0 0x7f45a0155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 -- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45840039c0 msgr2=0x7f45a0155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45840039c0 0x7f45a0155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 -- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a0156e60 msgr2=0x7f45a01572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a0156e60 0x7f45a01572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 -- 172.21.15.87:0/1828261286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a00cc160 con 0x7f45a0151ad0 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f45a5600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a0156e60 0x7f45a01572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.365+0000 7f459fe00640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a0151ad0 0x7f45a0151210 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f458c004ed0 tx=0x7f458c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f459cc00640 1 -- 172.21.15.87:0/1828261286 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c00f7b0 con 0x7f45a0151ad0 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f459cc00640 1 -- 172.21.15.87:0/1828261286 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c018070 con 0x7f45a0151ad0 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f459cc00640 1 -- 172.21.15.87:0/1828261286 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c00fb70 con 0x7f45a0151ad0 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a0152460 con 0x7f45a0151ad0 2024-04-25T10:22:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01529c0 con 0x7f45a0151ad0 2024-04-25T10:22:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a004ef30 con 0x7f45a0151ad0 2024-04-25T10:22:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f459cc00640 1 -- 172.21.15.87:0/1828261286 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f458c00fd10 con 0x7f45a0151ad0 2024-04-25T10:22:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f459cc00640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f90 0x7f4580069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f459cc00640 1 -- 172.21.15.87:0/1828261286 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f458c0850c0 con 0x7f45a0151ad0 2024-04-25T10:22:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.369+0000 7f45a4c00640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f90 0x7f4580069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.373+0000 7f45a4c00640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f90 0x7f4580069450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f45900af000 tx=0x7f4590009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.373+0000 7f459cc00640 1 -- 172.21.15.87:0/1828261286 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45a0152460 con 0x7f45a0151ad0 2024-04-25T10:22:41.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45a015b080 con 0x7f45a0151ad0 2024-04-25T10:22:41.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f459cc00640 1 -- 172.21.15.87:0/1828261286 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f458c055fa0 con 0x7f45a0151ad0 2024-04-25T10:22:41.533 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f90 msgr2=0x7f4580069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f90 0x7f4580069450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f45900af000 tx=0x7f4590009040 comp rx=0 tx=0).stop 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a0151ad0 msgr2=0x7f45a0151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a0151ad0 0x7f45a0151210 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f458c004ed0 tx=0x7f458c007430 comp rx=0 tx=0).stop 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 shutdown_connections 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f90 0x7f4580069450 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a0156e60 0x7f45a01572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a0151ad0 0x7f45a0151210 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 --2- 172.21.15.87:0/1828261286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45840039c0 0x7f45a0155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 >> 172.21.15.87:0/1828261286 conn(0x7f45a00669a0 msgr2=0x7f45a015ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 shutdown_connections 2024-04-25T10:22:41.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.529+0000 7f45a7600640 1 -- 172.21.15.87:0/1828261286 wait complete. 2024-04-25T10:22:41.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:22:41.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af3e00640 1 Processor -- start 2024-04-25T10:22:41.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af3e00640 1 -- start start 2024-04-25T10:22:41.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7aec15bc10 0x7f7aec15f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7aec0cf980 0x7f7aec0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af3e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec15fec0 con 0x7f7aec0d1830 2024-04-25T10:22:41.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af3e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec1601a0 con 0x7f7aec0cf980 2024-04-25T10:22:41.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec0cc5b0 con 0x7f7aec15bc10 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af1e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7aec0d1830 0x7f7aec0d1c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57018/0 (socket says 172.21.15.87:57018) 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af1e00640 1 -- 172.21.15.87:0/301689336 learned_addr learned my addr 172.21.15.87:0/301689336 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af0a00640 1 --2- 172.21.15.87:0/301689336 >> v2:172.21.15.87:3300/0 conn(0x7f7aec0cf980 0x7f7aec0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af1400640 1 --2- 172.21.15.87:0/301689336 >> v2:172.21.15.169:3300/0 conn(0x7f7aec15bc10 0x7f7aec15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7aebe00640 1 -- 172.21.15.87:0/301689336 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93257430 0 0) 0x7f7aec15fec0 con 0x7f7aec0d1830 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7aebe00640 1 -- 172.21.15.87:0/301689336 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad4003530 con 0x7f7aec0d1830 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af0a00640 1 -- 172.21.15.87:0/301689336 >> v2:172.21.15.169:3300/0 conn(0x7f7aec15bc10 msgr2=0x7f7aec15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af0a00640 1 --2- 172.21.15.87:0/301689336 >> v2:172.21.15.169:3300/0 conn(0x7f7aec15bc10 0x7f7aec15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af0a00640 1 -- 172.21.15.87:0/301689336 >> v1:172.21.15.87:6789/0 conn(0x7f7aec0d1830 legacy=0x7f7aec0d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.689+0000 7f7af0a00640 1 -- 172.21.15.87:0/301689336 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0cc890 con 0x7f7aec0cf980 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af0a00640 1 --2- 172.21.15.87:0/301689336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0cf980 0x7f7aec0b1060 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7aec067940 tx=0x7f7ad801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34d488fc07d6b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7aebe00640 1 -- 172.21.15.87:0/301689336 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80b1070 con 0x7f7aec0cf980 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7aebe00640 1 -- 172.21.15.87:0/301689336 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad80ac5f0 con 0x7f7aec0cf980 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7aebe00640 1 -- 172.21.15.87:0/301689336 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80ac9c0 con 0x7f7aec0cf980 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 -- 172.21.15.87:0/301689336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0cf980 msgr2=0x7f7aec0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 --2- 172.21.15.87:0/301689336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0cf980 0x7f7aec0b1060 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7aec067940 tx=0x7f7ad801c480 comp rx=0 tx=0).stop 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 -- 172.21.15.87:0/301689336 shutdown_connections 2024-04-25T10:22:41.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 --2- 172.21.15.87:0/301689336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0cf980 0x7f7aec0b1060 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 --2- 172.21.15.87:0/301689336 >> v2:172.21.15.169:3300/0 conn(0x7f7aec15bc10 0x7f7aec15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 -- 172.21.15.87:0/301689336 >> 172.21.15.87:0/301689336 conn(0x7f7aec09a9d0 msgr2=0x7f7aec09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 -- 172.21.15.87:0/301689336 shutdown_connections 2024-04-25T10:22:41.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.693+0000 7f7af3e00640 1 -- 172.21.15.87:0/301689336 wait complete. 2024-04-25T10:22:41.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 Processor -- start 2024-04-25T10:22:41.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 -- start start 2024-04-25T10:22:41.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aec0cf980 0x7f7aec1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0d1830 0x7f7aec0b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aec15bc10 0x7f7aec0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0cd5c0 con 0x7f7aec0d1830 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b17b0 con 0x7f7aec0cf980 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0cd740 con 0x7f7aec15bc10 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0d1830 0x7f7aec0b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0d1830 0x7f7aec0b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54390/0 (socket says 172.21.15.87:54390) 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 -- 172.21.15.87:0/280542414 learned_addr learned my addr 172.21.15.87:0/280542414 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af1400640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aec0cf980 0x7f7aec1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af1e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aec15bc10 0x7f7aec0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 -- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aec15bc10 msgr2=0x7f7aec0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aec15bc10 0x7f7aec0b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 -- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aec0cf980 msgr2=0x7f7aec1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aec0cf980 0x7f7aec1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 -- 172.21.15.87:0/280542414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec06afa0 con 0x7f7aec0d1830 2024-04-25T10:22:41.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af1e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aec15bc10 0x7f7aec0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:41.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af0a00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0d1830 0x7f7aec0b1d80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f7ad801ab30 tx=0x7f7ad8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7ae9600640 1 -- 172.21.15.87:0/280542414 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80b1040 con 0x7f7aec0d1830 2024-04-25T10:22:41.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7ae9600640 1 -- 172.21.15.87:0/280542414 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad80bb070 con 0x7f7aec0d1830 2024-04-25T10:22:41.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7ae9600640 1 -- 172.21.15.87:0/280542414 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8006e20 con 0x7f7aec0d1830 2024-04-25T10:22:41.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec06b280 con 0x7f7aec0d1830 2024-04-25T10:22:41.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.697+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec06b840 con 0x7f7aec0d1830 2024-04-25T10:22:41.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.701+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec04ef30 con 0x7f7aec0d1830 2024-04-25T10:22:41.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.701+0000 7f7ae9600640 1 -- 172.21.15.87:0/280542414 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ad80b5650 con 0x7f7aec0d1830 2024-04-25T10:22:41.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.701+0000 7f7ae9600640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7acc066f60 0x7f7acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.701+0000 7f7ae9600640 1 -- 172.21.15.87:0/280542414 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7ad811ebc0 con 0x7f7aec0d1830 2024-04-25T10:22:41.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.701+0000 7f7af1400640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7acc066f60 0x7f7acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.701+0000 7f7af1400640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7acc066f60 0x7f7acc069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0b31e0 tx=0x7f7ae000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.705+0000 7f7ae9600640 1 -- 172.21.15.87:0/280542414 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ad80a9070 con 0x7f7aec0d1830 2024-04-25T10:22:41.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.857+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7aec06e800 con 0x7f7aec0d1830 2024-04-25T10:22:41.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.857+0000 7f7ae9600640 1 -- 172.21.15.87:0/280542414 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ad8016070 con 0x7f7aec0d1830 2024-04-25T10:22:41.863 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:22:41.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7acc066f60 msgr2=0x7f7acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7acc066f60 0x7f7acc069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0b31e0 tx=0x7f7ae000b040 comp rx=0 tx=0).stop 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0d1830 msgr2=0x7f7aec0b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0d1830 0x7f7aec0b1d80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f7ad801ab30 tx=0x7f7ad8019220 comp rx=0 tx=0).stop 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 shutdown_connections 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7acc066f60 0x7f7acc069420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aec15bc10 0x7f7aec0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec0d1830 0x7f7aec0b1d80 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 --2- 172.21.15.87:0/280542414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aec0cf980 0x7f7aec1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 >> 172.21.15.87:0/280542414 conn(0x7f7aec09a9d0 msgr2=0x7f7aec0d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 shutdown_connections 2024-04-25T10:22:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:41.861+0000 7f7af3e00640 1 -- 172.21.15.87:0/280542414 wait complete. 2024-04-25T10:22:41.884 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7580400640 1 Processor -- start 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7580400640 1 -- start start 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7580400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75780d1830 0x7f75780d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7580400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75780b1770 con 0x7f757815bc10 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7580400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757815fec0 con 0x7f75780cf980 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7580400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75781601a0 con 0x7f75780d1830 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f757815bc10 0x7f757815f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57024/0 (socket says 172.21.15.87:57024) 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 -- 172.21.15.87:0/3550272852 learned_addr learned my addr 172.21.15.87:0/3550272852 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757e400640 1 --2- 172.21.15.87:0/3550272852 >> v2:172.21.15.169:3300/0 conn(0x7f75780d1830 0x7f75780d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821898371 0 0) 0x7f75780b1770 con 0x7f757815bc10 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7560003530 con 0x7f757815bc10 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566613109 0 0) 0x7f757815fec0 con 0x7f75780cf980 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75780b1770 con 0x7f75780cf980 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893215313 0 0) 0x7f7560003530 con 0x7f757815bc10 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 >> v2:172.21.15.169:3300/0 conn(0x7f75780d1830 msgr2=0x7f75780d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 --2- 172.21.15.87:0/3550272852 >> v2:172.21.15.169:3300/0 conn(0x7f75780d1830 0x7f75780d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 >> v1:172.21.15.169:6789/0 conn(0x7f75780cf980 legacy=0x7f75780b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780cc5b0 con 0x7f757815bc10 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f756c001680 con 0x7f757815bc10 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 >> v1:172.21.15.87:6789/0 conn(0x7f757815bc10 legacy=0x7f757815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7560003ba0 0x7f7560003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7560004810 0x7f7560004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600051d0 0x7f756000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75781601a0 con 0x7f7560003ba0 2024-04-25T10:22:42.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756000c0a0 con 0x7f75600051d0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756000c2a0 con 0x7f7560004810 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7560003ba0 0x7f7560003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757e400640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600051d0 0x7f756000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757d000640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7560004810 0x7f7560004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 -- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7560004810 msgr2=0x7f7560004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7560004810 0x7f7560004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 -- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600051d0 msgr2=0x7f756000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600051d0 0x7f756000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757da00640 1 -- 172.21.15.87:0/3550272852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560003530 con 0x7f7560003ba0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757e400640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600051d0 0x7f756000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.021+0000 7f757e400640 1 -- 172.21.15.87:0/3550272852 reap_dead start 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f757da00640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7560003ba0 0x7f7560003fa0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f756c007fb0 tx=0x7f756c01cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7656ef7cb46fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c007bd0 con 0x7f7560003ba0 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c0a9020 con 0x7f7560003ba0 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7577e00640 1 -- 172.21.15.87:0/3550272852 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c005b00 con 0x7f7560003ba0 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7580400640 1 -- 172.21.15.87:0/3550272852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7560003ba0 msgr2=0x7f7560003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7580400640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7560003ba0 0x7f7560003fa0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f756c007fb0 tx=0x7f756c01cb40 comp rx=0 tx=0).stop 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7580400640 1 -- 172.21.15.87:0/3550272852 shutdown_connections 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7580400640 1 --2- 172.21.15.87:0/3550272852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7560003ba0 0x7f7560003fa0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7580400640 1 -- 172.21.15.87:0/3550272852 >> 172.21.15.87:0/3550272852 conn(0x7f757809a9d0 msgr2=0x7f757809adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7580400640 1 -- 172.21.15.87:0/3550272852 shutdown_connections 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.025+0000 7f7580400640 1 -- 172.21.15.87:0/3550272852 wait complete. 2024-04-25T10:22:42.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 Processor -- start 2024-04-25T10:22:42.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 -- start start 2024-04-25T10:22:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7560003ba0 0x7f757815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75780c54d0 0x7f757815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757815c640 0x7f757815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780d21a0 con 0x7f75780c54d0 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780d2020 con 0x7f7560003ba0 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75780ccdb0 con 0x7f757815c640 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75780c54d0 0x7f757815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75780c54d0 0x7f757815c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54408/0 (socket says 172.21.15.87:54408) 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 -- 172.21.15.87:0/1368516182 learned_addr learned my addr 172.21.15.87:0/1368516182 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757e400640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757815c640 0x7f757815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757da00640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7560003ba0 0x7f757815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 -- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757815c640 msgr2=0x7f757815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757815c640 0x7f757815acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 -- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7560003ba0 msgr2=0x7f757815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7560003ba0 0x7f757815a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 -- 172.21.15.87:0/1368516182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780c60c0 con 0x7f75780c54d0 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757da00640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7560003ba0 0x7f757815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f757d000640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75780c54d0 0x7f757815c100 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f756400f770 tx=0x7f756400fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7575600640 1 -- 172.21.15.87:0/1368516182 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756401a070 con 0x7f75780c54d0 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7575600640 1 -- 172.21.15.87:0/1368516182 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564005110 con 0x7f75780c54d0 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75780c9bb0 con 0x7f75780c54d0 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7575600640 1 -- 172.21.15.87:0/1368516182 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564007030 con 0x7f75780c54d0 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.029+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75780ca170 con 0x7f75780c54d0 2024-04-25T10:22:42.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.033+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757804ef30 con 0x7f75780c54d0 2024-04-25T10:22:42.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.033+0000 7f7575600640 1 -- 172.21.15.87:0/1368516182 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f756401d070 con 0x7f75780c54d0 2024-04-25T10:22:42.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.033+0000 7f7575600640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7558066f60 0x7f7558069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.033+0000 7f7575600640 1 -- 172.21.15.87:0/1368516182 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7564084b30 con 0x7f75780c54d0 2024-04-25T10:22:42.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.033+0000 7f757da00640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7558066f60 0x7f7558069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.033+0000 7f757da00640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7558066f60 0x7f7558069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f756c0aa000 tx=0x7f7578160090 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.037+0000 7f7575600640 1 -- 172.21.15.87:0/1368516182 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75640553c0 con 0x7f75780c54d0 2024-04-25T10:22:42.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f757806b4f0 con 0x7f75780c54d0 2024-04-25T10:22:42.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7575600640 1 -- 172.21.15.87:0/1368516182 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7564059460 con 0x7f75780c54d0 2024-04-25T10:22:42.196 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:22:42.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7558066f60 msgr2=0x7f7558069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7558066f60 0x7f7558069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f756c0aa000 tx=0x7f7578160090 comp rx=0 tx=0).stop 2024-04-25T10:22:42.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75780c54d0 msgr2=0x7f757815c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75780c54d0 0x7f757815c100 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f756400f770 tx=0x7f756400fc30 comp rx=0 tx=0).stop 2024-04-25T10:22:42.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 shutdown_connections 2024-04-25T10:22:42.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7558066f60 0x7f7558069420 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f757815c640 0x7f757815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75780c54d0 0x7f757815c100 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 --2- 172.21.15.87:0/1368516182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7560003ba0 0x7f757815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 >> 172.21.15.87:0/1368516182 conn(0x7f757809a9d0 msgr2=0x7f75780b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 shutdown_connections 2024-04-25T10:22:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.193+0000 7f7580400640 1 -- 172.21.15.87:0/1368516182 wait complete. 2024-04-25T10:22:42.217 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:42.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5f400640 1 Processor -- start 2024-04-25T10:22:42.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5f400640 1 -- start start 2024-04-25T10:22:42.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b580d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b580b15b0 0x7f7b580cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b580b1af0 con 0x7f7b5815e050 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5f400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b580b1dd0 con 0x7f7b580b15b0 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580b20b0 con 0x7f7b580d1820 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b580b15b0 0x7f7b580cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b580b15b0 0x7f7b580cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54412/0 (socket says 172.21.15.87:54412) 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5d400640 1 -- 172.21.15.87:0/2316234921 learned_addr learned my addr 172.21.15.87:0/2316234921 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5ca00640 1 --2- 172.21.15.87:0/2316234921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b580d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5d400640 1 -- 172.21.15.87:0/2316234921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 msgr2=0x7f7b580d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5d400640 1 --2- 172.21.15.87:0/2316234921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b580d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5d400640 1 -- 172.21.15.87:0/2316234921 >> v1:172.21.15.169:6789/0 conn(0x7f7b5815e050 legacy=0x7f7b580b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5d400640 1 -- 172.21.15.87:0/2316234921 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580b2390 con 0x7f7b580b15b0 2024-04-25T10:22:42.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.349+0000 7f7b5ca00640 1 --2- 172.21.15.87:0/2316234921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b580d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:42.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5d400640 1 --2- 172.21.15.87:0/2316234921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b580cfc70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009e50 tx=0x7f7b5001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf5a9c114e48993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b57400640 1 -- 172.21.15.87:0/2316234921 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500a5920 con 0x7f7b580b15b0 2024-04-25T10:22:42.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b57400640 1 -- 172.21.15.87:0/2316234921 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b500a5ac0 con 0x7f7b580b15b0 2024-04-25T10:22:42.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b57400640 1 -- 172.21.15.87:0/2316234921 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500ad3f0 con 0x7f7b580b15b0 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 -- 172.21.15.87:0/2316234921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 msgr2=0x7f7b580cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 --2- 172.21.15.87:0/2316234921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b580cfc70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009e50 tx=0x7f7b5001ca30 comp rx=0 tx=0).stop 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 -- 172.21.15.87:0/2316234921 shutdown_connections 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 --2- 172.21.15.87:0/2316234921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b580cfc70 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 --2- 172.21.15.87:0/2316234921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b580d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 -- 172.21.15.87:0/2316234921 >> 172.21.15.87:0/2316234921 conn(0x7f7b5809a9d0 msgr2=0x7f7b5809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 -- 172.21.15.87:0/2316234921 shutdown_connections 2024-04-25T10:22:42.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.353+0000 7f7b5f400640 1 -- 172.21.15.87:0/2316234921 wait complete. 2024-04-25T10:22:42.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 Processor -- start 2024-04-25T10:22:42.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 -- start start 2024-04-25T10:22:42.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b58153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b58154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b5815e050 0x7f7b58149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580d0d50 con 0x7f7b580b15b0 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580d0bd0 con 0x7f7b5815e050 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580d0ed0 con 0x7f7b580d1820 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b58153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b58153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54422/0 (socket says 172.21.15.87:54422) 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 -- 172.21.15.87:0/3161858456 learned_addr learned my addr 172.21.15.87:0/3161858456 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b57e00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b58154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5d400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b5815e050 0x7f7b58149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 -- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 msgr2=0x7f7b58154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b58154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 -- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b5815e050 msgr2=0x7f7b58149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b5815e050 0x7f7b58149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 -- 172.21.15.87:0/3161858456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b58153e90 con 0x7f7b580b15b0 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b57e00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b58154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5d400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b5815e050 0x7f7b58149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5ca00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b58153950 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00e810 tx=0x7f7b4c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b54c00640 1 -- 172.21.15.87:0/3161858456 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c017070 con 0x7f7b580b15b0 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b54c00640 1 -- 172.21.15.87:0/3161858456 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00c8d0 con 0x7f7b580b15b0 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b54c00640 1 -- 172.21.15.87:0/3161858456 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00f040 con 0x7f7b580b15b0 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b58148b30 con 0x7f7b580b15b0 2024-04-25T10:22:42.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.357+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b581490f0 con 0x7f7b580b15b0 2024-04-25T10:22:42.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.361+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5804ef30 con 0x7f7b580b15b0 2024-04-25T10:22:42.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.361+0000 7f7b54c00640 1 -- 172.21.15.87:0/3161858456 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b4c01a070 con 0x7f7b580b15b0 2024-04-25T10:22:42.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.361+0000 7f7b54c00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b38066f60 0x7f7b38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.361+0000 7f7b54c00640 1 -- 172.21.15.87:0/3161858456 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7b4c084d30 con 0x7f7b580b15b0 2024-04-25T10:22:42.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.361+0000 7f7b57e00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b38066f60 0x7f7b38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.361+0000 7f7b57e00640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b38066f60 0x7f7b38069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7b580742a0 tx=0x7f7b44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.365+0000 7f7b54c00640 1 -- 172.21.15.87:0/3161858456 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b58153e90 con 0x7f7b580b15b0 2024-04-25T10:22:42.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.557+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b580d1f20 con 0x7f7b580b15b0 2024-04-25T10:22:42.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.557+0000 7f7b54c00640 1 -- 172.21.15.87:0/3161858456 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f7b4c055c10 con 0x7f7b580b15b0 2024-04-25T10:22:42.561 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:42.561 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:22:34.985474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9111},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9111":{"gid":9111,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":17,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:42.562 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 93 2024-04-25T10:22:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.557+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b38066f60 msgr2=0x7f7b38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.557+0000 7f7b5f400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b38066f60 0x7f7b38069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7b580742a0 tx=0x7f7b44009040 comp rx=0 tx=0).stop 2024-04-25T10:22:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 msgr2=0x7f7b58153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b58153950 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00e810 tx=0x7f7b4c00ecd0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 shutdown_connections 2024-04-25T10:22:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7b38066f60 0x7f7b38069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b5815e050 0x7f7b58149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b580d1820 0x7f7b58154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 --2- 172.21.15.87:0/3161858456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b580b15b0 0x7f7b58153950 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 >> 172.21.15.87:0/3161858456 conn(0x7f7b5809a9d0 msgr2=0x7f7b5815d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 shutdown_connections 2024-04-25T10:22:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.561+0000 7f7b5f400640 1 -- 172.21.15.87:0/3161858456 wait complete. 2024-04-25T10:22:42.583 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:42.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.713+0000 7fb68ce00640 1 Processor -- start 2024-04-25T10:22:42.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.713+0000 7fb68ce00640 1 -- start start 2024-04-25T10:22:42.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb68ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6880cda10 0x7fb68806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb68ce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6880b2770 con 0x7fb68815bc40 2024-04-25T10:22:42.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb68ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68806c670 con 0x7fb6880cf930 2024-04-25T10:22:42.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb68ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68806c950 con 0x7fb6880cda10 2024-04-25T10:22:42.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb68815bc40 0x7fb6880b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57026/0 (socket says 172.21.15.87:57026) 2024-04-25T10:22:42.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685e00640 1 -- 172.21.15.87:0/1182999079 learned_addr learned my addr 172.21.15.87:0/1182999079 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:42.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb67d400640 1 --2- 172.21.15.87:0/1182999079 >> v2:172.21.15.169:3300/0 conn(0x7fb6880cda10 0x7fb68806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873878382 0 0) 0x7fb6880b2770 con 0x7fb68815bc40 2024-04-25T10:22:42.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb66c003530 con 0x7fb68815bc40 2024-04-25T10:22:42.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2052411686 0 0) 0x7fb66c003530 con 0x7fb68815bc40 2024-04-25T10:22:42.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 >> v2:172.21.15.169:3300/0 conn(0x7fb6880cda10 msgr2=0x7fb68806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 --2- 172.21.15.87:0/1182999079 >> v2:172.21.15.169:3300/0 conn(0x7fb6880cda10 0x7fb68806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 >> v1:172.21.15.169:6789/0 conn(0x7fb6880cf930 legacy=0x7fb6880b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68806cc30 con 0x7fb68815bc40 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb678001a00 con 0x7fb68815bc40 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 >> v1:172.21.15.87:6789/0 conn(0x7fb68815bc40 legacy=0x7fb6880b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c0039c0 0x7fb66c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb66c0043c0 0x7fb66c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb66c005130 0x7fb66c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68806c950 con 0x7fb66c005130 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c00c070 con 0x7fb66c0043c0 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c00c270 con 0x7fb66c0039c0 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb686800640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb66c005130 0x7fb66c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb685e00640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c0039c0 0x7fb66c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb67d400640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb66c0043c0 0x7fb66c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb686800640 1 -- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c0039c0 msgr2=0x7fb66c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb686800640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c0039c0 0x7fb66c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb686800640 1 -- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb66c0043c0 msgr2=0x7fb66c004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb686800640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb66c0043c0 0x7fb66c004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb686800640 1 -- 172.21.15.87:0/1182999079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c003530 con 0x7fb66c005130 2024-04-25T10:22:42.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.717+0000 7fb67d400640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb66c0043c0 0x7fb66c004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:42.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb686800640 1 -- 172.21.15.87:0/1182999079 reap_dead start 2024-04-25T10:22:42.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb686800640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb66c005130 0x7fb66c00b9d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb680001860 tx=0x7fb68001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8fb5ab09825ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680001460 con 0x7fb66c005130 2024-04-25T10:22:42.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6800052d0 con 0x7fb66c005130 2024-04-25T10:22:42.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb685400640 1 -- 172.21.15.87:0/1182999079 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680003820 con 0x7fb66c005130 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 -- 172.21.15.87:0/1182999079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb66c005130 msgr2=0x7fb66c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb66c005130 0x7fb66c00b9d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb680001860 tx=0x7fb68001c500 comp rx=0 tx=0).stop 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 -- 172.21.15.87:0/1182999079 shutdown_connections 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 --2- 172.21.15.87:0/1182999079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb66c005130 0x7fb66c00b9d0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 -- 172.21.15.87:0/1182999079 >> 172.21.15.87:0/1182999079 conn(0x7fb688066b70 msgr2=0x7fb688066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 -- 172.21.15.87:0/1182999079 shutdown_connections 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 -- 172.21.15.87:0/1182999079 wait complete. 2024-04-25T10:22:42.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.721+0000 7fb68ce00640 1 Processor -- start 2024-04-25T10:22:42.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 -- start start 2024-04-25T10:22:42.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c005130 0x7fb6880adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6880cc8a0 0x7fb6880ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb688070310 0x7fb688070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68806c420 con 0x7fb688070310 2024-04-25T10:22:42.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68806c2a0 con 0x7fb6880cc8a0 2024-04-25T10:22:42.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880b2c70 con 0x7fb66c005130 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb688070310 0x7fb688070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb688070310 0x7fb688070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54440/0 (socket says 172.21.15.87:54440) 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 -- 172.21.15.87:0/3148631843 learned_addr learned my addr 172.21.15.87:0/3148631843 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb67d400640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6880cc8a0 0x7fb6880ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb685e00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c005130 0x7fb6880adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 -- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c005130 msgr2=0x7fb6880adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c005130 0x7fb6880adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 -- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6880cc8a0 msgr2=0x7fb6880ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6880cc8a0 0x7fb6880ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 -- 172.21.15.87:0/3148631843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68815df60 con 0x7fb688070310 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb67d400640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6880cc8a0 0x7fb6880ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb686800640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb688070310 0x7fb688070780 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb68001cda0 tx=0x7fb66c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb67ea00640 1 -- 172.21.15.87:0/3148631843 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680001440 con 0x7fb688070310 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb67ea00640 1 -- 172.21.15.87:0/3148631843 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680005480 con 0x7fb688070310 2024-04-25T10:22:42.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb67ea00640 1 -- 172.21.15.87:0/3148631843 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680003c40 con 0x7fb688070310 2024-04-25T10:22:42.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68815e240 con 0x7fb688070310 2024-04-25T10:22:42.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68815e800 con 0x7fb688070310 2024-04-25T10:22:42.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.725+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68804ef30 con 0x7fb688070310 2024-04-25T10:22:42.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.729+0000 7fb67ea00640 1 -- 172.21.15.87:0/3148631843 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6800aa070 con 0x7fb688070310 2024-04-25T10:22:42.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.729+0000 7fb67ea00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb664066f60 0x7fb664069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.729+0000 7fb67ea00640 1 -- 172.21.15.87:0/3148631843 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb68011dd60 con 0x7fb688070310 2024-04-25T10:22:42.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.729+0000 7fb685e00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb664066f60 0x7fb664069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.729+0000 7fb685e00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb664066f60 0x7fb664069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb678001470 tx=0x7fb678006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.733+0000 7fb67ea00640 1 -- 172.21.15.87:0/3148631843 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb680016210 con 0x7fb688070310 2024-04-25T10:22:42.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.885+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6880b2770 con 0x7fb688070310 2024-04-25T10:22:42.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb67ea00640 1 -- 172.21.15.87:0/3148631843 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16836 (secure 0 0 0) 0x7fb6800eecd0 con 0x7fb688070310 2024-04-25T10:22:42.891 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:42.891 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":127,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:32.952911+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:40.912468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:41.547768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/85260299":"2024-04-26T10:22:32.952837+0000","172.21.15.169:6832/85260299":"2024-04-26T10:22:32.952837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:42.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb664066f60 msgr2=0x7fb664069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb664066f60 0x7fb664069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb678001470 tx=0x7fb678006040 comp rx=0 tx=0).stop 2024-04-25T10:22:42.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb688070310 msgr2=0x7fb688070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb688070310 0x7fb688070780 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb68001cda0 tx=0x7fb66c005840 comp rx=0 tx=0).stop 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 shutdown_connections 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb664066f60 0x7fb664069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb688070310 0x7fb688070780 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6880cc8a0 0x7fb6880ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 --2- 172.21.15.87:0/3148631843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb66c005130 0x7fb6880adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 >> 172.21.15.87:0/3148631843 conn(0x7fb688066b70 msgr2=0x7fb688150710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 shutdown_connections 2024-04-25T10:22:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:42.889+0000 7fb68ce00640 1 -- 172.21.15.87:0/3148631843 wait complete. 2024-04-25T10:22:42.913 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:22:42.913 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.045+0000 7f1972600640 1 Processor -- start 2024-04-25T10:22:43.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.045+0000 7f1972600640 1 -- start start 2024-04-25T10:22:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f1972600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f196c1550b0 0x7f196c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f1972600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f1972600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c071050 con 0x7f196c0cfe70 2024-04-25T10:22:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f1972600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c15bda0 con 0x7f196c1550b0 2024-04-25T10:22:43.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f1972600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c15d040 con 0x7f196c15b910 2024-04-25T10:22:43.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f1962a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f196c0cfe70 0x7f196c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:44914/0 (socket says 172.21.15.87:44914) 2024-04-25T10:22:43.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f1962a00640 1 -- 172.21.15.87:0/4181046083 learned_addr learned my addr 172.21.15.87:0/4181046083 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:43.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196be00640 1 --2- 172.21.15.87:0/4181046083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:34024/0 (socket says 172.21.15.87:34024) 2024-04-25T10:22:43.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196b400640 1 --2- 172.21.15.87:0/4181046083 >> v2:172.21.15.169:3300/0 conn(0x7f196c1550b0 0x7f196c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196aa00640 1 -- 172.21.15.87:0/4181046083 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585611648 0 0) 0x7f196c071050 con 0x7f196c0cfe70 2024-04-25T10:22:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196be00640 1 -- 172.21.15.87:0/4181046083 >> v2:172.21.15.169:3300/0 conn(0x7f196c1550b0 msgr2=0x7f196c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196be00640 1 --2- 172.21.15.87:0/4181046083 >> v2:172.21.15.169:3300/0 conn(0x7f196c1550b0 0x7f196c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196be00640 1 -- 172.21.15.87:0/4181046083 >> v1:172.21.15.169:6789/0 conn(0x7f196c0cfe70 legacy=0x7f196c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.049+0000 7f196be00640 1 -- 172.21.15.87:0/4181046083 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0712f0 con 0x7f196c15b910 2024-04-25T10:22:43.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f196be00640 1 --2- 172.21.15.87:0/4181046083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c0b3260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19640097f0 tx=0x7f196401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4315c62d88e23a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f196aa00640 1 -- 172.21.15.87:0/4181046083 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640b2070 con 0x7f196c15b910 2024-04-25T10:22:43.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f196aa00640 1 -- 172.21.15.87:0/4181046083 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19640ad640 con 0x7f196c15b910 2024-04-25T10:22:43.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f196aa00640 1 -- 172.21.15.87:0/4181046083 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640ada10 con 0x7f196c15b910 2024-04-25T10:22:43.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 -- 172.21.15.87:0/4181046083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 msgr2=0x7f196c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 --2- 172.21.15.87:0/4181046083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c0b3260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19640097f0 tx=0x7f196401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 -- 172.21.15.87:0/4181046083 shutdown_connections 2024-04-25T10:22:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 --2- 172.21.15.87:0/4181046083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c0b3260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 --2- 172.21.15.87:0/4181046083 >> v2:172.21.15.169:3300/0 conn(0x7f196c1550b0 0x7f196c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 -- 172.21.15.87:0/4181046083 >> 172.21.15.87:0/4181046083 conn(0x7f196c066b70 msgr2=0x7f196c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 -- 172.21.15.87:0/4181046083 shutdown_connections 2024-04-25T10:22:43.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 -- 172.21.15.87:0/4181046083 wait complete. 2024-04-25T10:22:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 Processor -- start 2024-04-25T10:22:43.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.053+0000 7f1972600640 1 -- start start 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f196c0cfe70 0x7f196c1f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c1550b0 0x7f196c13b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c15c030 con 0x7f196c0cfe70 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0720b0 con 0x7f196c1550b0 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c15c1b0 con 0x7f196c15b910 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f196c0cfe70 0x7f196c1f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f196c0cfe70 0x7f196c1f9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54442/0 (socket says 172.21.15.87:54442) 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 -- 172.21.15.87:0/1219365872 learned_addr learned my addr 172.21.15.87:0/1219365872 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 -- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 msgr2=0x7f196c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196be00640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1962a00640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c1550b0 0x7f196c13b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 -- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c1550b0 msgr2=0x7f196c13b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c1550b0 0x7f196c13b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 -- 172.21.15.87:0/1219365872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c158ca0 con 0x7f196c0cfe70 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196be00640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1962a00640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c1550b0 0x7f196c13b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f196b400640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f196c0cfe70 0x7f196c1f9de0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f195c00efc0 tx=0x7f195c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1963e00640 1 -- 172.21.15.87:0/1219365872 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c018020 con 0x7f196c0cfe70 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1963e00640 1 -- 172.21.15.87:0/1219365872 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c014480 con 0x7f196c0cfe70 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1963e00640 1 -- 172.21.15.87:0/1219365872 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c014870 con 0x7f196c0cfe70 2024-04-25T10:22:43.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c1fa4e0 con 0x7f196c0cfe70 2024-04-25T10:22:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c1faaa0 con 0x7f196c0cfe70 2024-04-25T10:22:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c1face0 con 0x7f196c0cfe70 2024-04-25T10:22:43.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.057+0000 7f1963e00640 1 -- 172.21.15.87:0/1219365872 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f195c01b020 con 0x7f196c0cfe70 2024-04-25T10:22:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.061+0000 7f1963e00640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1948066f60 0x7f1948069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.061+0000 7f1963e00640 1 -- 172.21.15.87:0/1219365872 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f195c085cc0 con 0x7f196c0cfe70 2024-04-25T10:22:43.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.061+0000 7f1962a00640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1948066f60 0x7f1948069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.061+0000 7f1962a00640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1948066f60 0x7f1948069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f196c0cc440 tx=0x7f1954009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.065+0000 7f1963e00640 1 -- 172.21.15.87:0/1219365872 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f196c1face0 con 0x7f196c0cfe70 2024-04-25T10:22:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f196c04ef30 con 0x7f196c0cfe70 2024-04-25T10:22:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1963e00640 1 -- 172.21.15.87:0/1219365872 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f195c08c080 con 0x7f196c0cfe70 2024-04-25T10:22:43.257 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:43.257 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:22:34.985474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9111},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9111":{"gid":9111,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":17,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:43.257 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 93 2024-04-25T10:22:43.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1948066f60 msgr2=0x7f1948069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1948066f60 0x7f1948069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f196c0cc440 tx=0x7f1954009040 comp rx=0 tx=0).stop 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f196c0cfe70 msgr2=0x7f196c1f9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f196c0cfe70 0x7f196c1f9de0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f195c00efc0 tx=0x7f195c00c490 comp rx=0 tx=0).stop 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 shutdown_connections 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1948066f60 0x7f1948069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f196c15b910 0x7f196c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c1550b0 0x7f196c13b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 --2- 172.21.15.87:0/1219365872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f196c0cfe70 0x7f196c1f9de0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 >> 172.21.15.87:0/1219365872 conn(0x7f196c066b70 msgr2=0x7f196c0adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 shutdown_connections 2024-04-25T10:22:43.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.253+0000 7f1972600640 1 -- 172.21.15.87:0/1219365872 wait complete. 2024-04-25T10:22:43.278 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:43.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.441+0000 7efe6ce00640 1 Processor -- start 2024-04-25T10:22:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.441+0000 7efe6ce00640 1 -- start start 2024-04-25T10:22:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.441+0000 7efe6ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efe681550b0 0x7efe680cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.441+0000 7efe6ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe680cfe70 0x7efe6815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe6ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe6806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe6ce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe68071050 con 0x7efe680cfe70 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe6ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe680712f0 con 0x7efe681550b0 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe6ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe680715d0 con 0x7efe6815b740 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe680cfe70 0x7efe6815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe680cfe70 0x7efe6815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54454/0 (socket says 172.21.15.87:54454) 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 -- 172.21.15.87:0/439814042 learned_addr learned my addr 172.21.15.87:0/439814042 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe66800640 1 --2- 172.21.15.87:0/439814042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe6806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe65e00640 1 --2- 172.21.15.87:0/439814042 >> v2:172.21.15.169:3300/0 conn(0x7efe681550b0 0x7efe680cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 -- 172.21.15.87:0/439814042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 msgr2=0x7efe6806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 --2- 172.21.15.87:0/439814042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe6806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 -- 172.21.15.87:0/439814042 >> v2:172.21.15.169:3300/0 conn(0x7efe681550b0 msgr2=0x7efe680cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 --2- 172.21.15.87:0/439814042 >> v2:172.21.15.169:3300/0 conn(0x7efe681550b0 0x7efe680cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 -- 172.21.15.87:0/439814042 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe680718b0 con 0x7efe680cfe70 2024-04-25T10:22:43.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe5d400640 1 --2- 172.21.15.87:0/439814042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe680cfe70 0x7efe6815b200 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7efe50009870 tx=0x7efe5001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa19c28c5a33d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe65400640 1 -- 172.21.15.87:0/439814042 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500b0070 con 0x7efe680cfe70 2024-04-25T10:22:43.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe65400640 1 -- 172.21.15.87:0/439814042 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe500ab600 con 0x7efe680cfe70 2024-04-25T10:22:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.445+0000 7efe65400640 1 -- 172.21.15.87:0/439814042 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500ab9d0 con 0x7efe680cfe70 2024-04-25T10:22:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- 172.21.15.87:0/439814042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe680cfe70 msgr2=0x7efe6815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 --2- 172.21.15.87:0/439814042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe680cfe70 0x7efe6815b200 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7efe50009870 tx=0x7efe5001c260 comp rx=0 tx=0).stop 2024-04-25T10:22:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- 172.21.15.87:0/439814042 shutdown_connections 2024-04-25T10:22:43.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 --2- 172.21.15.87:0/439814042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe680cfe70 0x7efe6815b200 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 --2- 172.21.15.87:0/439814042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe6806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 --2- 172.21.15.87:0/439814042 >> v2:172.21.15.169:3300/0 conn(0x7efe681550b0 0x7efe680cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- 172.21.15.87:0/439814042 >> 172.21.15.87:0/439814042 conn(0x7efe68066b70 msgr2=0x7efe68066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- 172.21.15.87:0/439814042 shutdown_connections 2024-04-25T10:22:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- 172.21.15.87:0/439814042 wait complete. 2024-04-25T10:22:43.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 Processor -- start 2024-04-25T10:22:43.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- start start 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe680cfe70 0x7efe6814a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe681550b0 0x7efe68157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe681594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe680b3260 con 0x7efe681550b0 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6815bd60 con 0x7efe680cfe70 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe6ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe680b33e0 con 0x7efe6815b740 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe681550b0 0x7efe68157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe681550b0 0x7efe68157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54460/0 (socket says 172.21.15.87:54460) 2024-04-25T10:22:43.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 -- 172.21.15.87:0/2481856637 learned_addr learned my addr 172.21.15.87:0/2481856637 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe66800640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe681594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe65e00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe680cfe70 0x7efe6814a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 -- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 msgr2=0x7efe681594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe681594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 -- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe680cfe70 msgr2=0x7efe6814a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe680cfe70 0x7efe6814a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe5d400640 1 -- 172.21.15.87:0/2481856637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68157920 con 0x7efe681550b0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.449+0000 7efe66800640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe681594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe5d400640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe681550b0 0x7efe68157020 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7efe5001cb00 tx=0x7efe50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe5ea00640 1 -- 172.21.15.87:0/2481856637 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500ba070 con 0x7efe681550b0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe5ea00640 1 -- 172.21.15.87:0/2481856637 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe500b4420 con 0x7efe681550b0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe5ea00640 1 -- 172.21.15.87:0/2481856637 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500b0040 con 0x7efe681550b0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe68157c00 con 0x7efe681550b0 2024-04-25T10:22:43.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe681581c0 con 0x7efe681550b0 2024-04-25T10:22:43.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6804ef30 con 0x7efe681550b0 2024-04-25T10:22:43.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe5ea00640 1 -- 172.21.15.87:0/2481856637 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe500b4810 con 0x7efe681550b0 2024-04-25T10:22:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe5ea00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe44066f60 0x7efe44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe5ea00640 1 -- 172.21.15.87:0/2481856637 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7efe5011e260 con 0x7efe681550b0 2024-04-25T10:22:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.453+0000 7efe65e00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe44066f60 0x7efe44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.457+0000 7efe65e00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe44066f60 0x7efe44069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efe58007a60 tx=0x7efe58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.457+0000 7efe5ea00640 1 -- 172.21.15.87:0/2481856637 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe50016110 con 0x7efe681550b0 2024-04-25T10:22:43.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.641+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe680cead0 con 0x7efe681550b0 2024-04-25T10:22:43.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.641+0000 7efe5ea00640 1 -- 172.21.15.87:0/2481856637 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4155 (secure 0 0 0) 0x7efe500eec00 con 0x7efe681550b0 2024-04-25T10:22:43.649 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:43.650 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:42.563054+0000","modified":"2024-04-25T10:22:34.985474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9111},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9111":{"gid":9111,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":17,"addr":"172.21.15.87:6837/2894154211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2894154211},{"type":"v1","addr":"172.21.15.87:6837","nonce":2894154211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:22:43.650 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 93 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe44066f60 msgr2=0x7efe44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe44066f60 0x7efe44069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efe58007a60 tx=0x7efe58008040 comp rx=0 tx=0).stop 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe681550b0 msgr2=0x7efe68157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe681550b0 0x7efe68157020 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7efe5001cb00 tx=0x7efe50019000 comp rx=0 tx=0).stop 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 shutdown_connections 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efe44066f60 0x7efe44069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe6815b740 0x7efe681594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe681550b0 0x7efe68157020 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 --2- 172.21.15.87:0/2481856637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe680cfe70 0x7efe6814a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 >> 172.21.15.87:0/2481856637 conn(0x7efe68066b70 msgr2=0x7efe680b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 shutdown_connections 2024-04-25T10:22:43.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.645+0000 7efe6ce00640 1 -- 172.21.15.87:0/2481856637 wait complete. 2024-04-25T10:22:43.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 Processor -- start 2024-04-25T10:22:43.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 -- start start 2024-04-25T10:22:43.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c7415bc40 0x7f4c7406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4c740cda10 0x7f4c740b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c740cf930 0x7f4c740cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740b2770 con 0x7f4c740cda10 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7406c670 con 0x7f4c740cf930 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c79400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7406c950 con 0x7f4c7415bc40 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4c740cda10 0x7f4c740b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4c740cda10 0x7f4c740b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54466/0 (socket says 172.21.15.87:54466) 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 -- 172.21.15.87:0/2864119392 learned_addr learned my addr 172.21.15.87:0/2864119392 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c73200640 1 --2- 172.21.15.87:0/2864119392 >> v2:172.21.15.169:3300/0 conn(0x7f4c740cf930 0x7f4c740cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c72800640 1 --2- 172.21.15.87:0/2864119392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c7415bc40 0x7f4c7406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 -- 172.21.15.87:0/2864119392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c7415bc40 msgr2=0x7f4c7406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 --2- 172.21.15.87:0/2864119392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c7415bc40 0x7f4c7406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 -- 172.21.15.87:0/2864119392 >> v2:172.21.15.169:3300/0 conn(0x7f4c740cf930 msgr2=0x7f4c740cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:43.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 --2- 172.21.15.87:0/2864119392 >> v2:172.21.15.169:3300/0 conn(0x7f4c740cf930 0x7f4c740cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.801+0000 7f4c71e00640 1 -- 172.21.15.87:0/2864119392 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7406cc30 con 0x7f4c740cda10 2024-04-25T10:22:43.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c71e00640 1 --2- 172.21.15.87:0/2864119392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c740cda10 0x7f4c740b4610 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4c740b3f80 tx=0x7f4c6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acda87894a890718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c71400640 1 -- 172.21.15.87:0/2864119392 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c600b0070 con 0x7f4c740cda10 2024-04-25T10:22:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c71400640 1 -- 172.21.15.87:0/2864119392 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c600ab600 con 0x7f4c740cda10 2024-04-25T10:22:43.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c71400640 1 -- 172.21.15.87:0/2864119392 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c600ab9d0 con 0x7f4c740cda10 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 -- 172.21.15.87:0/2864119392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c740cda10 msgr2=0x7f4c740b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 --2- 172.21.15.87:0/2864119392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c740cda10 0x7f4c740b4610 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4c740b3f80 tx=0x7f4c6001c520 comp rx=0 tx=0).stop 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 -- 172.21.15.87:0/2864119392 shutdown_connections 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 --2- 172.21.15.87:0/2864119392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c740cda10 0x7f4c740b4610 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 --2- 172.21.15.87:0/2864119392 >> v2:172.21.15.169:3300/0 conn(0x7f4c740cf930 0x7f4c740cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 --2- 172.21.15.87:0/2864119392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c7415bc40 0x7f4c7406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 -- 172.21.15.87:0/2864119392 >> 172.21.15.87:0/2864119392 conn(0x7f4c74066b70 msgr2=0x7f4c74066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 -- 172.21.15.87:0/2864119392 shutdown_connections 2024-04-25T10:22:43.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.805+0000 7f4c79400640 1 -- 172.21.15.87:0/2864119392 wait complete. 2024-04-25T10:22:43.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 Processor -- start 2024-04-25T10:22:43.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 -- start start 2024-04-25T10:22:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c740cda10 0x7f4c74146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c740cf930 0x7f4c7414ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c7415bc40 0x7f4c7414b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74160230 con 0x7f4c7415bc40 2024-04-25T10:22:43.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c741600b0 con 0x7f4c740cf930 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c741603b0 con 0x7f4c740cda10 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c7415bc40 0x7f4c7414b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c7415bc40 0x7f4c7414b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54468/0 (socket says 172.21.15.87:54468) 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 -- 172.21.15.87:0/3191143595 learned_addr learned my addr 172.21.15.87:0/3191143595 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c72800640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c740cda10 0x7f4c74146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c71e00640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c740cf930 0x7f4c7414ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 -- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c740cda10 msgr2=0x7f4c74146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c740cda10 0x7f4c74146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 -- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c740cf930 msgr2=0x7f4c7414ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c740cf930 0x7f4c7414ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 -- 172.21.15.87:0/3191143595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7414b6d0 con 0x7f4c7415bc40 2024-04-25T10:22:43.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c73200640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c7415bc40 0x7f4c7414b080 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e820 tx=0x7f4c6c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c6aa00640 1 -- 172.21.15.87:0/3191143595 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c017070 con 0x7f4c7415bc40 2024-04-25T10:22:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c6aa00640 1 -- 172.21.15.87:0/3191143595 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00c730 con 0x7f4c7415bc40 2024-04-25T10:22:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c6aa00640 1 -- 172.21.15.87:0/3191143595 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00cae0 con 0x7f4c7415bc40 2024-04-25T10:22:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c741f7630 con 0x7f4c7415bc40 2024-04-25T10:22:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7415ef50 con 0x7f4c7415bc40 2024-04-25T10:22:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.809+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7404ef30 con 0x7f4c7415bc40 2024-04-25T10:22:43.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.813+0000 7f4c6aa00640 1 -- 172.21.15.87:0/3191143595 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c6c01a070 con 0x7f4c7415bc40 2024-04-25T10:22:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.813+0000 7f4c6aa00640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c54066f90 0x7f4c54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.813+0000 7f4c6aa00640 1 -- 172.21.15.87:0/3191143595 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4c6c0851b0 con 0x7f4c7415bc40 2024-04-25T10:22:43.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.813+0000 7f4c72800640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c54066f90 0x7f4c54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:43.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.813+0000 7f4c72800640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c54066f90 0x7f4c54069450 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4c741f84e0 tx=0x7f4c6400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:43.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.817+0000 7f4c6aa00640 1 -- 172.21.15.87:0/3191143595 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c6c055ad0 con 0x7f4c7415bc40 2024-04-25T10:22:43.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c74151320 con 0x7f4c7415bc40 2024-04-25T10:22:43.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c6aa00640 1 -- 172.21.15.87:0/3191143595 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16836 (secure 0 0 0) 0x7f4c6c059b70 con 0x7f4c7415bc40 2024-04-25T10:22:43.976 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:43.977 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":127,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:32.952911+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:40.912468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:41.547768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/85260299":"2024-04-26T10:22:32.952837+0000","172.21.15.169:6832/85260299":"2024-04-26T10:22:32.952837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c54066f90 msgr2=0x7f4c54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c54066f90 0x7f4c54069450 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4c741f84e0 tx=0x7f4c6400b040 comp rx=0 tx=0).stop 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c7415bc40 msgr2=0x7f4c7414b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c7415bc40 0x7f4c7414b080 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e820 tx=0x7f4c6c00ece0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 shutdown_connections 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4c54066f90 0x7f4c54069450 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c7415bc40 0x7f4c7414b080 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c740cf930 0x7f4c7414ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 --2- 172.21.15.87:0/3191143595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c740cda10 0x7f4c74146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:43.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 >> 172.21.15.87:0/3191143595 conn(0x7f4c74066b70 msgr2=0x7f4c740cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:43.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 shutdown_connections 2024-04-25T10:22:43.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:43.973+0000 7f4c79400640 1 -- 172.21.15.87:0/3191143595 wait complete. 2024-04-25T10:22:43.998 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:22:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.129+0000 7f4088a00640 1 Processor -- start 2024-04-25T10:22:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.129+0000 7f4088a00640 1 -- start start 2024-04-25T10:22:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.129+0000 7f4088a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40840cda10 0x7f408406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.129+0000 7f4088a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840b2770 con 0x7f40840cf930 2024-04-25T10:22:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.129+0000 7f4088a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408406c670 con 0x7f408415bc40 2024-04-25T10:22:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.129+0000 7f4088a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408406c950 con 0x7f40840cda10 2024-04-25T10:22:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4082800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f40840cf930 0x7f40840b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57038/0 (socket says 172.21.15.87:57038) 2024-04-25T10:22:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4082800640 1 -- 172.21.15.87:0/543234914 learned_addr learned my addr 172.21.15.87:0/543234914 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4079400640 1 --2- 172.21.15.87:0/543234914 >> v2:172.21.15.87:3300/0 conn(0x7f40840cda10 0x7f408406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4079400640 1 -- 172.21.15.87:0/543234914 >> v1:172.21.15.169:6789/0 conn(0x7f408415bc40 legacy=0x7f40840b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:44.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4079400640 1 -- 172.21.15.87:0/543234914 >> v1:172.21.15.87:6789/0 conn(0x7f40840cf930 legacy=0x7f40840b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4081400640 1 -- 172.21.15.87:0/543234914 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876933057 0 0) 0x7f40840b2770 con 0x7f40840cf930 2024-04-25T10:22:44.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4079400640 1 -- 172.21.15.87:0/543234914 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408406cc30 con 0x7f40840cda10 2024-04-25T10:22:44.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4079400640 1 --2- 172.21.15.87:0/543234914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f408406c090 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f406c0097c0 tx=0x7f406c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857ee66c869a7557 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:44.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4081400640 1 -- 172.21.15.87:0/543234914 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c0b0070 con 0x7f40840cda10 2024-04-25T10:22:44.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4081400640 1 -- 172.21.15.87:0/543234914 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c0ab600 con 0x7f40840cda10 2024-04-25T10:22:44.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.133+0000 7f4081400640 1 -- 172.21.15.87:0/543234914 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c0ab9d0 con 0x7f40840cda10 2024-04-25T10:22:44.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- 172.21.15.87:0/543234914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 msgr2=0x7f408406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 --2- 172.21.15.87:0/543234914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f408406c090 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f406c0097c0 tx=0x7f406c01c410 comp rx=0 tx=0).stop 2024-04-25T10:22:44.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- 172.21.15.87:0/543234914 shutdown_connections 2024-04-25T10:22:44.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 --2- 172.21.15.87:0/543234914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f408406c090 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- 172.21.15.87:0/543234914 >> 172.21.15.87:0/543234914 conn(0x7f4084066b70 msgr2=0x7f4084066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- 172.21.15.87:0/543234914 shutdown_connections 2024-04-25T10:22:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- 172.21.15.87:0/543234914 wait complete. 2024-04-25T10:22:44.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 Processor -- start 2024-04-25T10:22:44.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- start start 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f40840af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40840cf930 0x7f4084134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408415bc40 0x7f4084134640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4084169d70 con 0x7f40840cda10 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4084169bf0 con 0x7f40840cf930 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4088a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4084169ef0 con 0x7f408415bc40 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f40840af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f40840af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54484/0 (socket says 172.21.15.87:54484) 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 -- 172.21.15.87:0/3365690778 learned_addr learned my addr 172.21.15.87:0/3365690778 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4079400640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40840cf930 0x7f4084134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4082800640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408415bc40 0x7f4084134640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 -- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408415bc40 msgr2=0x7f4084134640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408415bc40 0x7f4084134640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 -- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40840cf930 msgr2=0x7f4084134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40840cf930 0x7f4084134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.137+0000 7f4081e00640 1 -- 172.21.15.87:0/3365690778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084132070 con 0x7f40840cda10 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f4081e00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f40840af960 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f407400c500 tx=0x7f407400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f407aa00640 1 -- 172.21.15.87:0/3365690778 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074016020 con 0x7f40840cda10 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f407aa00640 1 -- 172.21.15.87:0/3365690778 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074005220 con 0x7f40840cda10 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f407aa00640 1 -- 172.21.15.87:0/3365690778 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074007030 con 0x7f40840cda10 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4084149b80 con 0x7f40840cda10 2024-04-25T10:22:44.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408414a140 con 0x7f40840cda10 2024-04-25T10:22:44.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4084132240 con 0x7f40840cda10 2024-04-25T10:22:44.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f407aa00640 1 -- 172.21.15.87:0/3365690778 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40740071f0 con 0x7f40840cda10 2024-04-25T10:22:44.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f407aa00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4060066f60 0x7f4060069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f407aa00640 1 -- 172.21.15.87:0/3365690778 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f40740849b0 con 0x7f40840cda10 2024-04-25T10:22:44.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.141+0000 7f4079400640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4060066f60 0x7f4060069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.145+0000 7f4079400640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4060066f60 0x7f4060069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f406c0ad000 tx=0x7f406c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:44.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.145+0000 7f407aa00640 1 -- 172.21.15.87:0/3365690778 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4074020020 con 0x7f40840cda10 2024-04-25T10:22:44.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.333+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f40840af1c0 con 0x7f40840cda10 2024-04-25T10:22:44.337 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:22:44.333+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:22:44.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.337+0000 7f407aa00640 1 -- 172.21.15.87:0/3365690778 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4074055250 con 0x7f40840cda10 2024-04-25T10:22:44.340 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.337+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4060066f60 msgr2=0x7f4060069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.337+0000 7f4088a00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4060066f60 0x7f4060069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f406c0ad000 tx=0x7f406c008040 comp rx=0 tx=0).stop 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.337+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 msgr2=0x7f40840af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.337+0000 7f4088a00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f40840af960 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f407400c500 tx=0x7f407400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 shutdown_connections 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4060066f60 0x7f4060069420 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f408415bc40 0x7f4084134640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40840cf930 0x7f4084134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 --2- 172.21.15.87:0/3365690778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40840cda10 0x7f40840af960 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 >> 172.21.15.87:0/3365690778 conn(0x7f4084066b70 msgr2=0x7f40841574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:44.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 shutdown_connections 2024-04-25T10:22:44.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.341+0000 7f4088a00640 1 -- 172.21.15.87:0/3365690778 wait complete. 2024-04-25T10:22:44.364 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:22:44.410 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:22:44.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.501+0000 7f4900400640 1 Processor -- start 2024-04-25T10:22:44.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.501+0000 7f4900400640 1 -- start start 2024-04-25T10:22:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f4900400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48f80b3480 0x7f48f815e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f4900400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f80ad840 con 0x7f48f8071bc0 2024-04-25T10:22:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f4900400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f815e710 con 0x7f48f80cfb50 2024-04-25T10:22:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f4900400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f815bb90 con 0x7f48f80b3480 2024-04-25T10:22:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f48fda00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48f80b3480 0x7f48f815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f48fe400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f48f8071bc0 0x7f48f80add20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57054/0 (socket says 172.21.15.87:57054) 2024-04-25T10:22:44.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f48fe400640 1 -- 172.21.15.87:0/17655979 learned_addr learned my addr 172.21.15.87:0/17655979 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:44.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f48fda00640 1 -- 172.21.15.87:0/17655979 >> v1:172.21.15.169:6789/0 conn(0x7f48f80cfb50 legacy=0x7f48f8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f48fda00640 1 -- 172.21.15.87:0/17655979 >> v1:172.21.15.87:6789/0 conn(0x7f48f8071bc0 legacy=0x7f48f80add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f48f7e00640 1 -- 172.21.15.87:0/17655979 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133877423 0 0) 0x7f48f80ad840 con 0x7f48f8071bc0 2024-04-25T10:22:44.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.505+0000 7f48fda00640 1 -- 172.21.15.87:0/17655979 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f815e9f0 con 0x7f48f80b3480 2024-04-25T10:22:44.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f48fda00640 1 --2- 172.21.15.87:0/17655979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f80b3480 0x7f48f815e130 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0097c0 tx=0x7f48ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a281073300048c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:44.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f48f7e00640 1 -- 172.21.15.87:0/17655979 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0b0070 con 0x7f48f80b3480 2024-04-25T10:22:44.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f48f7e00640 1 -- 172.21.15.87:0/17655979 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec0ab600 con 0x7f48f80b3480 2024-04-25T10:22:44.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f48f7e00640 1 -- 172.21.15.87:0/17655979 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0ab9d0 con 0x7f48f80b3480 2024-04-25T10:22:44.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 -- 172.21.15.87:0/17655979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f80b3480 msgr2=0x7f48f815e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 --2- 172.21.15.87:0/17655979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f80b3480 0x7f48f815e130 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0097c0 tx=0x7f48ec01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 -- 172.21.15.87:0/17655979 shutdown_connections 2024-04-25T10:22:44.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 --2- 172.21.15.87:0/17655979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f80b3480 0x7f48f815e130 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 -- 172.21.15.87:0/17655979 >> 172.21.15.87:0/17655979 conn(0x7f48f80669a0 msgr2=0x7f48f8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:44.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 -- 172.21.15.87:0/17655979 shutdown_connections 2024-04-25T10:22:44.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 -- 172.21.15.87:0/17655979 wait complete. 2024-04-25T10:22:44.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 Processor -- start 2024-04-25T10:22:44.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 -- start start 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f8071bc0 0x7f48f8164570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.509+0000 7f4900400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f80b3480 0x7f48f81615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f4900400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48f80cfb50 0x7f48f8156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f4900400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8072400 con 0x7f48f8071bc0 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f4900400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80ae840 con 0x7f48f80b3480 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f4900400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f816c0b0 con 0x7f48f80cfb50 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f8071bc0 0x7f48f8164570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f8071bc0 0x7f48f8164570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54528/0 (socket says 172.21.15.87:54528) 2024-04-25T10:22:44.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 -- 172.21.15.87:0/2410527268 learned_addr learned my addr 172.21.15.87:0/2410527268 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 -- 172.21.15.87:0/2410527268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48f80cfb50 msgr2=0x7f48f8156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48f80cfb50 0x7f48f8156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 -- 172.21.15.87:0/2410527268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f80b3480 msgr2=0x7f48f81615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f80b3480 0x7f48f81615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 -- 172.21.15.87:0/2410527268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f81634d0 con 0x7f48f8071bc0 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48fda00640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f8071bc0 0x7f48f8164570 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ae040 tx=0x7f48ec01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48f5600640 1 -- 172.21.15.87:0/2410527268 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0b0050 con 0x7f48f8071bc0 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48f5600640 1 -- 172.21.15.87:0/2410527268 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec0bc070 con 0x7f48f8071bc0 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48f5600640 1 -- 172.21.15.87:0/2410527268 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0b5a70 con 0x7f48f8071bc0 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f81637b0 con 0x7f48f8071bc0 2024-04-25T10:22:44.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f8163d70 con 0x7f48f8071bc0 2024-04-25T10:22:44.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f804ef30 con 0x7f48f8071bc0 2024-04-25T10:22:44.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.513+0000 7f48f5600640 1 -- 172.21.15.87:0/2410527268 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48ec0ab4a0 con 0x7f48f8071bc0 2024-04-25T10:22:44.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.517+0000 7f48f5600640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48d8066f60 0x7f48d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:44.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.517+0000 7f48f5600640 1 -- 172.21.15.87:0/2410527268 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f48ec120520 con 0x7f48f8071bc0 2024-04-25T10:22:44.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.517+0000 7f48fd000640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48d8066f60 0x7f48d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:44.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.517+0000 7f48fd000640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48d8066f60 0x7f48d8069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000f70 tx=0x7f48e40079c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:44.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.517+0000 7f48f5600640 1 -- 172.21.15.87:0/2410527268 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48ec016090 con 0x7f48f8071bc0 2024-04-25T10:22:44.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:44.705+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f48f80ae210 con 0x7f48f8071bc0 2024-04-25T10:22:45.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.337+0000 7f48f5600640 1 -- 172.21.15.87:0/2410527268 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 104+0+0 (secure 0 0 0) 0x7f48ec0f15d0 con 0x7f48f8071bc0 2024-04-25T10:22:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48d8066f60 msgr2=0x7f48d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48d8066f60 0x7f48d8069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000f70 tx=0x7f48e40079c0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f8071bc0 msgr2=0x7f48f8164570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f8071bc0 0x7f48f8164570 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ae040 tx=0x7f48ec01ace0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 shutdown_connections 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48d8066f60 0x7f48d8069420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48f80cfb50 0x7f48f8156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48f80b3480 0x7f48f81615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 --2- 172.21.15.87:0/2410527268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f8071bc0 0x7f48f8164570 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 >> 172.21.15.87:0/2410527268 conn(0x7f48f80669a0 msgr2=0x7f48f806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 shutdown_connections 2024-04-25T10:22:45.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.341+0000 7f4900400640 1 -- 172.21.15.87:0/2410527268 wait complete. 2024-04-25T10:22:45.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:22:45.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.545+0000 7f1fd9800640 1 Processor -- start 2024-04-25T10:22:45.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.545+0000 7f1fd9800640 1 -- start start 2024-04-25T10:22:45.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd9800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1fd4157cb0 0x7f1fd40720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:45.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4070ba0 0x7f1fd415b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:45.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd9800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1fd40b2af0 0x7f1fd40b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:45.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd9800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40af150 con 0x7f1fd40b2af0 2024-04-25T10:22:45.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd9800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40b3450 con 0x7f1fd4157cb0 2024-04-25T10:22:45.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd415b7a0 con 0x7f1fd4070ba0 2024-04-25T10:22:45.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1fd40b2af0 0x7f1fd40b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1fd40b2af0 0x7f1fd40b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54532/0 (socket says 172.21.15.87:54532) 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 -- 172.21.15.87:0/504946054 learned_addr learned my addr 172.21.15.87:0/504946054 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd2a00640 1 --2- 172.21.15.87:0/504946054 >> v2:172.21.15.169:3300/0 conn(0x7f1fd4157cb0 0x7f1fd40720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fcbe00640 1 --2- 172.21.15.87:0/504946054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4070ba0 0x7f1fd415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 -- 172.21.15.87:0/504946054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4070ba0 msgr2=0x7f1fd415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 --2- 172.21.15.87:0/504946054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4070ba0 0x7f1fd415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 -- 172.21.15.87:0/504946054 >> v2:172.21.15.169:3300/0 conn(0x7f1fd4157cb0 msgr2=0x7f1fd40720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:45.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 --2- 172.21.15.87:0/504946054 >> v2:172.21.15.169:3300/0 conn(0x7f1fd4157cb0 0x7f1fd40720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.549+0000 7f1fd3400640 1 -- 172.21.15.87:0/504946054 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd4071040 con 0x7f1fd40b2af0 2024-04-25T10:22:45.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd3400640 1 --2- 172.21.15.87:0/504946054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd40b2af0 0x7f1fd40b2f10 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f1fd406e830 tx=0x7f1fcc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523747678d85f559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:45.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd2000640 1 -- 172.21.15.87:0/504946054 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0a56e0 con 0x7f1fd40b2af0 2024-04-25T10:22:45.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd2000640 1 -- 172.21.15.87:0/504946054 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc0a5880 con 0x7f1fd40b2af0 2024-04-25T10:22:45.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd2000640 1 -- 172.21.15.87:0/504946054 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0a5c50 con 0x7f1fd40b2af0 2024-04-25T10:22:45.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 -- 172.21.15.87:0/504946054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd40b2af0 msgr2=0x7f1fd40b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:45.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 --2- 172.21.15.87:0/504946054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd40b2af0 0x7f1fd40b2f10 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f1fd406e830 tx=0x7f1fcc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 -- 172.21.15.87:0/504946054 shutdown_connections 2024-04-25T10:22:45.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 --2- 172.21.15.87:0/504946054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd40b2af0 0x7f1fd40b2f10 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 --2- 172.21.15.87:0/504946054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4070ba0 0x7f1fd415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 --2- 172.21.15.87:0/504946054 >> v2:172.21.15.169:3300/0 conn(0x7f1fd4157cb0 0x7f1fd40720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 -- 172.21.15.87:0/504946054 >> 172.21.15.87:0/504946054 conn(0x7f1fd4066b70 msgr2=0x7f1fd4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:45.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 -- 172.21.15.87:0/504946054 shutdown_connections 2024-04-25T10:22:45.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 -- 172.21.15.87:0/504946054 wait complete. 2024-04-25T10:22:45.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 Processor -- start 2024-04-25T10:22:45.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.553+0000 7f1fd9800640 1 -- start start 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd4070ba0 0x7f1fd4147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fd40b2af0 0x7f1fd4156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4157cb0 0x7f1fd40b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4158090 con 0x7f1fd4070ba0 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4071bc0 con 0x7f1fd40b2af0 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4158210 con 0x7f1fd4157cb0 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd4070ba0 0x7f1fd4147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fcbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fd40b2af0 0x7f1fd4156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4157cb0 0x7f1fd40b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fcbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fd40b2af0 0x7f1fd4156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:56988/0 (socket says 172.21.15.87:56988) 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fcbe00640 1 -- 172.21.15.87:0/3343556191 learned_addr learned my addr 172.21.15.87:0/3343556191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd2a00640 1 -- 172.21.15.87:0/3343556191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4157cb0 msgr2=0x7f1fd40b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd2a00640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4157cb0 0x7f1fd40b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd2a00640 1 -- 172.21.15.87:0/3343556191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fd40b2af0 msgr2=0x7f1fd4156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:45.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd2a00640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fd40b2af0 0x7f1fd4156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:45.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd2a00640 1 -- 172.21.15.87:0/3343556191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd415ba20 con 0x7f1fd4070ba0 2024-04-25T10:22:45.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd2a00640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd4070ba0 0x7f1fd4147010 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400e8a0 tx=0x7f1fc400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:45.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fcaa00640 1 -- 172.21.15.87:0/3343556191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4017070 con 0x7f1fd4070ba0 2024-04-25T10:22:45.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fcaa00640 1 -- 172.21.15.87:0/3343556191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc400c940 con 0x7f1fd4070ba0 2024-04-25T10:22:45.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd415c1b0 con 0x7f1fd4070ba0 2024-04-25T10:22:45.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fcaa00640 1 -- 172.21.15.87:0/3343556191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400cd10 con 0x7f1fd4070ba0 2024-04-25T10:22:45.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd4156240 con 0x7f1fd4070ba0 2024-04-25T10:22:45.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd404ef30 con 0x7f1fd4070ba0 2024-04-25T10:22:45.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.557+0000 7f1fcaa00640 1 -- 172.21.15.87:0/3343556191 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fc401a070 con 0x7f1fd4070ba0 2024-04-25T10:22:45.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.561+0000 7f1fcaa00640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fb0066f60 0x7f1fb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:45.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.561+0000 7f1fcaa00640 1 -- 172.21.15.87:0/3343556191 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f1fc4084ee0 con 0x7f1fd4070ba0 2024-04-25T10:22:45.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.561+0000 7f1fcbe00640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fb0066f60 0x7f1fb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:45.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.561+0000 7f1fcbe00640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fb0066f60 0x7f1fb0069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fd415ecd0 tx=0x7f1fbc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:45.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.565+0000 7f1fcaa00640 1 -- 172.21.15.87:0/3343556191 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fc4055710 con 0x7f1fd4070ba0 2024-04-25T10:22:45.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:45.717+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1fd4070430 con 0x7f1fd4070ba0 2024-04-25T10:22:46.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fcaa00640 1 -- 172.21.15.87:0/3343556191 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1fc40597b0 con 0x7f1fd4070ba0 2024-04-25T10:22:46.344 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fb0066f60 msgr2=0x7f1fb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fb0066f60 0x7f1fb0069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fd415ecd0 tx=0x7f1fbc009190 comp rx=0 tx=0).stop 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd4070ba0 msgr2=0x7f1fd4147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd4070ba0 0x7f1fd4147010 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400e8a0 tx=0x7f1fc400ed60 comp rx=0 tx=0).stop 2024-04-25T10:22:46.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 shutdown_connections 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fb0066f60 0x7f1fb0069420 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fd4157cb0 0x7f1fd40b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fd40b2af0 0x7f1fd4156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 --2- 172.21.15.87:0/3343556191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fd4070ba0 0x7f1fd4147010 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 >> 172.21.15.87:0/3343556191 conn(0x7f1fd4066b70 msgr2=0x7f1fd406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 shutdown_connections 2024-04-25T10:22:46.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.341+0000 7f1fd9800640 1 -- 172.21.15.87:0/3343556191 wait complete. 2024-04-25T10:22:46.371 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:22:46.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf04800640 1 Processor -- start 2024-04-25T10:22:46.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf04800640 1 -- start start 2024-04-25T10:22:46.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf04800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdefc06b0d0 0x7fdefc0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf04800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0cf290 con 0x7fdefc06b800 2024-04-25T10:22:46.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf04800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0cfdb0 con 0x7fdefc06bbe0 2024-04-25T10:22:46.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf04800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc0cd030 con 0x7fdefc06b0d0 2024-04-25T10:22:46.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf01e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fdefc06b800 0x7fdefc0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57068/0 (socket says 172.21.15.87:57068) 2024-04-25T10:22:46.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf01e00640 1 -- 172.21.15.87:0/4171163843 learned_addr learned my addr 172.21.15.87:0/4171163843 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf01400640 1 --2- 172.21.15.87:0/4171163843 >> v2:172.21.15.169:3300/0 conn(0x7fdefc06b0d0 0x7fdefc0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622215056 0 0) 0x7fdefc0cf290 con 0x7fdefc06b800 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee4003530 con 0x7fdefc06b800 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1429879386 0 0) 0x7fdee4003530 con 0x7fdefc06b800 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 >> v2:172.21.15.169:3300/0 conn(0x7fdefc06b0d0 msgr2=0x7fdefc0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 --2- 172.21.15.87:0/4171163843 >> v2:172.21.15.169:3300/0 conn(0x7fdefc06b0d0 0x7fdefc0cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 >> v1:172.21.15.169:6789/0 conn(0x7fdefc06bbe0 legacy=0x7fdefc15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0d0090 con 0x7fdefc06b800 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fdef0001a00 con 0x7fdefc06b800 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 >> v1:172.21.15.87:6789/0 conn(0x7fdefc06b800 legacy=0x7fdefc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdee40039c0 0x7fdee4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee4004810 0x7fdee4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40051d0 0x7fdee400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0cd030 con 0x7fdee40039c0 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee400c080 con 0x7fdee40051d0 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.509+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee400c280 con 0x7fdee4004810 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01e00640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdee40039c0 0x7fdee4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf02800640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40051d0 0x7fdee400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01400640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee4004810 0x7fdee4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01e00640 1 -- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee4004810 msgr2=0x7fdee4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01e00640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee4004810 0x7fdee4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01e00640 1 -- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40051d0 msgr2=0x7fdee400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01e00640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40051d0 0x7fdee400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01e00640 1 -- 172.21.15.87:0/4171163843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4003530 con 0x7fdee40039c0 2024-04-25T10:22:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf02800640 1 -- 172.21.15.87:0/4171163843 reap_dead start 2024-04-25T10:22:46.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf01e00640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdee40039c0 0x7fdee4003de0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fdef000a840 tx=0x7fdef001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5db3cf921a8bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:46.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000a460 con 0x7fdee40039c0 2024-04-25T10:22:46.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef00aa710 con 0x7fdee40039c0 2024-04-25T10:22:46.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.513+0000 7fdf00a00640 1 -- 172.21.15.87:0/4171163843 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00aab40 con 0x7fdee40039c0 2024-04-25T10:22:46.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- 172.21.15.87:0/4171163843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdee40039c0 msgr2=0x7fdee4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdee40039c0 0x7fdee4003de0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fdef000a840 tx=0x7fdef001fac0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- 172.21.15.87:0/4171163843 shutdown_connections 2024-04-25T10:22:46.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 --2- 172.21.15.87:0/4171163843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdee40039c0 0x7fdee4003de0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- 172.21.15.87:0/4171163843 >> 172.21.15.87:0/4171163843 conn(0x7fdefc0d19d0 msgr2=0x7fdefc0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:46.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- 172.21.15.87:0/4171163843 shutdown_connections 2024-04-25T10:22:46.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- 172.21.15.87:0/4171163843 wait complete. 2024-04-25T10:22:46.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 Processor -- start 2024-04-25T10:22:46.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- start start 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40039c0 0x7fdefc1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc0c5500 0x7fdefc06f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdefc0c5980 0x7fdefc06fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0d0930 con 0x7fdefc0c5500 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0d0650 con 0x7fdee40039c0 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf04800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0d0e50 con 0x7fdefc0c5980 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc0c5500 0x7fdefc06f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc0c5500 0x7fdefc06f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54552/0 (socket says 172.21.15.87:54552) 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 -- 172.21.15.87:0/1016716290 learned_addr learned my addr 172.21.15.87:0/1016716290 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01e00640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40039c0 0x7fdefc1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf02800640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdefc0c5980 0x7fdefc06fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 -- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdefc0c5980 msgr2=0x7fdefc06fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdefc0c5980 0x7fdefc06fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 -- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40039c0 msgr2=0x7fdefc1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40039c0 0x7fdefc1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 -- 172.21.15.87:0/1016716290 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0b5f70 con 0x7fdefc0c5500 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.517+0000 7fdf01400640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc0c5500 0x7fdefc06f800 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fdeec011530 tx=0x7fdeec0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdef6000640 1 -- 172.21.15.87:0/1016716290 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec015870 con 0x7fdefc0c5500 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdef6000640 1 -- 172.21.15.87:0/1016716290 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec015a10 con 0x7fdefc0c5500 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdef6000640 1 -- 172.21.15.87:0/1016716290 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec015dc0 con 0x7fdefc0c5500 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc0b6250 con 0x7fdefc0c5500 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc0b2ae0 con 0x7fdefc0c5500 2024-04-25T10:22:46.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc04ef30 con 0x7fdefc0c5500 2024-04-25T10:22:46.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdef6000640 1 -- 172.21.15.87:0/1016716290 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdeec013400 con 0x7fdefc0c5500 2024-04-25T10:22:46.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdef6000640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdee0066fe0 0x7fdee00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:46.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdef6000640 1 -- 172.21.15.87:0/1016716290 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fdeec085560 con 0x7fdefc0c5500 2024-04-25T10:22:46.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.521+0000 7fdf01e00640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdee0066fe0 0x7fdee00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:46.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.525+0000 7fdf01e00640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdee0066fe0 0x7fdee00694a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdef00b1000 tx=0x7fdef0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:46.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.525+0000 7fdef6000640 1 -- 172.21.15.87:0/1016716290 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdeec019030 con 0x7fdefc0c5500 2024-04-25T10:22:46.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:46.681+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdefc0d12e0 con 0x7fdefc0c5500 2024-04-25T10:22:47.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.341+0000 7fdef6000640 1 -- 172.21.15.87:0/1016716290 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdeec056060 con 0x7fdefc0c5500 2024-04-25T10:22:47.346 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdee0066fe0 msgr2=0x7fdee00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdee0066fe0 0x7fdee00694a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdef00b1000 tx=0x7fdef0009040 comp rx=0 tx=0).stop 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc0c5500 msgr2=0x7fdefc06f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc0c5500 0x7fdefc06f800 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fdeec011530 tx=0x7fdeec0119f0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 shutdown_connections 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdee0066fe0 0x7fdee00694a0 secure :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdef00b1000 tx=0x7fdef0009040 comp rx=0 tx=0).stop 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdefc0c5980 0x7fdefc06fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc0c5500 0x7fdefc06f800 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 --2- 172.21.15.87:0/1016716290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdee40039c0 0x7fdefc1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 >> 172.21.15.87:0/1016716290 conn(0x7fdefc0d19d0 msgr2=0x7fdefc0cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:47.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.345+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 shutdown_connections 2024-04-25T10:22:47.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.349+0000 7fdf04800640 1 -- 172.21.15.87:0/1016716290 wait complete. 2024-04-25T10:22:47.384 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:22:47.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.517+0000 7f37b6200640 1 Processor -- start 2024-04-25T10:22:47.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.517+0000 7f37b6200640 1 -- start start 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b6200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f37b00d0380 0x7f37b00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b6200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00ce280 con 0x7f37b015ed10 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b6200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b00ce030 con 0x7f37b00d0380 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00ce560 con 0x7f37b00cffa0 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b5800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f37b015ed10 0x7f37b015f8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:42518/0 (socket says 172.21.15.87:42518) 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b5800640 1 -- 172.21.15.87:0/103379868 learned_addr learned my addr 172.21.15.87:0/103379868 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:47.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37afe00640 1 --2- 172.21.15.87:0/103379868 >> v2:172.21.15.169:3300/0 conn(0x7f37b00d0380 0x7f37b00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37af400640 1 -- 172.21.15.87:0/103379868 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106426500 0 0) 0x7f37b00ce280 con 0x7f37b015ed10 2024-04-25T10:22:47.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37af400640 1 -- 172.21.15.87:0/103379868 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3794003530 con 0x7f37b015ed10 2024-04-25T10:22:47.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37afe00640 1 -- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 msgr2=0x7f37b00b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:47.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37afe00640 1 --2- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37afe00640 1 -- 172.21.15.87:0/103379868 >> v1:172.21.15.169:6789/0 conn(0x7f37b015ed10 legacy=0x7f37b015f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37afe00640 1 -- 172.21.15.87:0/103379868 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00ce840 con 0x7f37b00d0380 2024-04-25T10:22:47.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.521+0000 7f37b4e00640 1 --2- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:47.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37afe00640 1 --2- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 0x7f37b00b3dd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f379c0097c0 tx=0x7f379c01c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fd5f23d6efb653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:47.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37af400640 1 -- 172.21.15.87:0/103379868 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0b0070 con 0x7f37b00d0380 2024-04-25T10:22:47.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37af400640 1 -- 172.21.15.87:0/103379868 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c004230 con 0x7f37b00d0380 2024-04-25T10:22:47.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37af400640 1 -- 172.21.15.87:0/103379868 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c003620 con 0x7f37b00d0380 2024-04-25T10:22:47.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 -- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 msgr2=0x7f37b00b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 --2- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 0x7f37b00b3dd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f379c0097c0 tx=0x7f379c01c350 comp rx=0 tx=0).stop 2024-04-25T10:22:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 -- 172.21.15.87:0/103379868 shutdown_connections 2024-04-25T10:22:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 --2- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 0x7f37b00b3dd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 --2- 172.21.15.87:0/103379868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 -- 172.21.15.87:0/103379868 >> 172.21.15.87:0/103379868 conn(0x7f37b00b32b0 msgr2=0x7f37b00b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 -- 172.21.15.87:0/103379868 shutdown_connections 2024-04-25T10:22:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 -- 172.21.15.87:0/103379868 wait complete. 2024-04-25T10:22:47.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 Processor -- start 2024-04-25T10:22:47.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.525+0000 7f37b6200640 1 -- start start 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 0x7f37b0156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b015ed10 0x7f37b006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00cd690 con 0x7f37b015ed10 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00cd510 con 0x7f37b00d0380 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0168c00 con 0x7f37b00cffa0 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b015ed10 0x7f37b006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b015ed10 0x7f37b006f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37612/0 (socket says 172.21.15.87:37612) 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 -- 172.21.15.87:0/4107205899 learned_addr learned my addr 172.21.15.87:0/4107205899 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b4e00640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37afe00640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 0x7f37b0156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 -- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 msgr2=0x7f37b00b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 -- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 msgr2=0x7f37b0156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 0x7f37b0156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 -- 172.21.15.87:0/4107205899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00cfb70 con 0x7f37b015ed10 2024-04-25T10:22:47.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b5800640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b015ed10 0x7f37b006f5f0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c450 tx=0x7f37a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:47.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37acc00640 1 -- 172.21.15.87:0/4107205899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8017070 con 0x7f37b015ed10 2024-04-25T10:22:47.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37acc00640 1 -- 172.21.15.87:0/4107205899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8005cf0 con 0x7f37b015ed10 2024-04-25T10:22:47.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37acc00640 1 -- 172.21.15.87:0/4107205899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80072b0 con 0x7f37b015ed10 2024-04-25T10:22:47.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b00c8bd0 con 0x7f37b015ed10 2024-04-25T10:22:47.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b00c9190 con 0x7f37b015ed10 2024-04-25T10:22:47.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.529+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b0075120 con 0x7f37b015ed10 2024-04-25T10:22:47.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.533+0000 7f37acc00640 1 -- 172.21.15.87:0/4107205899 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37a8007450 con 0x7f37b015ed10 2024-04-25T10:22:47.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.533+0000 7f37acc00640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.533+0000 7f37acc00640 1 -- 172.21.15.87:0/4107205899 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f37a8083f60 con 0x7f37b015ed10 2024-04-25T10:22:47.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.533+0000 7f37b4e00640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.533+0000 7f37b4e00640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f37b00b16b0 tx=0x7f37a000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:47.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.537+0000 7f37acc00640 1 -- 172.21.15.87:0/4107205899 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37a8007740 con 0x7f37b015ed10 2024-04-25T10:22:47.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.693+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f37b0159cb0 con 0x7f37b015ed10 2024-04-25T10:22:47.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37acc00640 1 -- 172.21.15.87:0/4107205899 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f37a8054c50 con 0x7f37b015ed10 2024-04-25T10:22:47.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 msgr2=0x7f3790069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f37b00b16b0 tx=0x7f37a000b040 comp rx=0 tx=0).stop 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b015ed10 msgr2=0x7f37b006f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b015ed10 0x7f37b006f5f0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c450 tx=0x7f37a800c910 comp rx=0 tx=0).stop 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 shutdown_connections 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3790066f60 0x7f3790069420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37b015ed10 0x7f37b006f5f0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b00d0380 0x7f37b0156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 --2- 172.21.15.87:0/4107205899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37b00cffa0 0x7f37b00b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 >> 172.21.15.87:0/4107205899 conn(0x7f37b00b32b0 msgr2=0x7f37b015a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 shutdown_connections 2024-04-25T10:22:47.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.697+0000 7f37b6200640 1 -- 172.21.15.87:0/4107205899 wait complete. 2024-04-25T10:22:47.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:22:47.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.849+0000 7fe75c800640 1 Processor -- start 2024-04-25T10:22:47.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.849+0000 7fe75c800640 1 -- start start 2024-04-25T10:22:47.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe75c800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe7541550b0 0x7fe7540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe75c800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7540cfe70 0x7fe75415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe75c800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe754071050 con 0x7fe75415b740 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe75c800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75415d040 con 0x7fe7541550b0 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe75c800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7540712f0 con 0x7fe7540cfe70 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe7541550b0 0x7fe7540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe7541550b0 0x7fe7540cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37616/0 (socket says 172.21.15.87:37616) 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759e00640 1 -- 172.21.15.87:0/3901480430 learned_addr learned my addr 172.21.15.87:0/3901480430 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759400640 1 --2- 172.21.15.87:0/3901480430 >> v2:172.21.15.169:3300/0 conn(0x7fe7540cfe70 0x7fe75415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759e00640 1 -- 172.21.15.87:0/3901480430 >> v2:172.21.15.169:3300/0 conn(0x7fe7540cfe70 msgr2=0x7fe75415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759e00640 1 --2- 172.21.15.87:0/3901480430 >> v2:172.21.15.169:3300/0 conn(0x7fe7540cfe70 0x7fe75415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759e00640 1 -- 172.21.15.87:0/3901480430 >> v1:172.21.15.169:6789/0 conn(0x7fe75415b740 legacy=0x7fe75406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.853+0000 7fe759e00640 1 -- 172.21.15.87:0/3901480430 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7540715d0 con 0x7fe7541550b0 2024-04-25T10:22:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe759e00640 1 --2- 172.21.15.87:0/3901480430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7541550b0 0x7fe7540cf930 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe748009870 tx=0x7fe74801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925372d17fe2c6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe758a00640 1 -- 172.21.15.87:0/3901480430 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7480b0070 con 0x7fe7541550b0 2024-04-25T10:22:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe758a00640 1 -- 172.21.15.87:0/3901480430 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7480ab600 con 0x7fe7541550b0 2024-04-25T10:22:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe758a00640 1 -- 172.21.15.87:0/3901480430 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7480ab9d0 con 0x7fe7541550b0 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 -- 172.21.15.87:0/3901480430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7541550b0 msgr2=0x7fe7540cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 --2- 172.21.15.87:0/3901480430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7541550b0 0x7fe7540cf930 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe748009870 tx=0x7fe74801c520 comp rx=0 tx=0).stop 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 -- 172.21.15.87:0/3901480430 shutdown_connections 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 --2- 172.21.15.87:0/3901480430 >> v2:172.21.15.169:3300/0 conn(0x7fe7540cfe70 0x7fe75415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 --2- 172.21.15.87:0/3901480430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7541550b0 0x7fe7540cf930 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 -- 172.21.15.87:0/3901480430 >> 172.21.15.87:0/3901480430 conn(0x7fe754066b70 msgr2=0x7fe754066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 -- 172.21.15.87:0/3901480430 shutdown_connections 2024-04-25T10:22:47.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 -- 172.21.15.87:0/3901480430 wait complete. 2024-04-25T10:22:47.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 Processor -- start 2024-04-25T10:22:47.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.857+0000 7fe75c800640 1 -- start start 2024-04-25T10:22:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7540cfe70 0x7fe75413a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7541550b0 0x7fe7541541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75415b740 0x7fe75414cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415bcd0 con 0x7fe75415b740 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415bb50 con 0x7fe7540cfe70 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415c030 con 0x7fe7541550b0 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75415b740 0x7fe75414cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe759e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7540cfe70 0x7fe75413a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75415b740 0x7fe75414cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37626/0 (socket says 172.21.15.87:37626) 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 -- 172.21.15.87:0/125691204 learned_addr learned my addr 172.21.15.87:0/125691204 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe759400640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7541550b0 0x7fe7541541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 -- 172.21.15.87:0/125691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7541550b0 msgr2=0x7fe7541541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7541550b0 0x7fe7541541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 -- 172.21.15.87:0/125691204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7540cfe70 msgr2=0x7fe75413a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7540cfe70 0x7fe75413a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 -- 172.21.15.87:0/125691204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754149790 con 0x7fe75415b740 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75a800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75415b740 0x7fe75414cf60 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe75000c730 tx=0x7fe75000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe74e000640 1 -- 172.21.15.87:0/125691204 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750016020 con 0x7fe75415b740 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe74e000640 1 -- 172.21.15.87:0/125691204 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe750006d60 con 0x7fe75415b740 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe754152710 con 0x7fe75415b740 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe74e000640 1 -- 172.21.15.87:0/125691204 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750003bc0 con 0x7fe75415b740 2024-04-25T10:22:47.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe754152cd0 con 0x7fe75415b740 2024-04-25T10:22:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75404ef30 con 0x7fe75415b740 2024-04-25T10:22:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.861+0000 7fe74e000640 1 -- 172.21.15.87:0/125691204 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7500050d0 con 0x7fe75415b740 2024-04-25T10:22:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.865+0000 7fe74e000640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe738066f60 0x7fe738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.865+0000 7fe74e000640 1 -- 172.21.15.87:0/125691204 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe7500848b0 con 0x7fe75415b740 2024-04-25T10:22:47.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.865+0000 7fe759e00640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe738066f60 0x7fe738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:47.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.865+0000 7fe759e00640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe738066f60 0x7fe738069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ad000 tx=0x7fe7480a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:47.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:47.869+0000 7fe74e000640 1 -- 172.21.15.87:0/125691204 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe750020020 con 0x7fe75415b740 2024-04-25T10:22:48.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.121+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7fe7540b3380 con 0x7fe75415b740 2024-04-25T10:22:48.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.345+0000 7fe74e000640 1 -- 172.21.15.87:0/125691204 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v507) v1 ==== 166+0+0 (secure 0 0 0) 0x7fe750055500 con 0x7fe75415b740 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe738066f60 msgr2=0x7fe738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe738066f60 0x7fe738069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ad000 tx=0x7fe7480a6000 comp rx=0 tx=0).stop 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75415b740 msgr2=0x7fe75414cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75415b740 0x7fe75414cf60 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe75000c730 tx=0x7fe75000cb30 comp rx=0 tx=0).stop 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 shutdown_connections 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe738066f60 0x7fe738069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe75415b740 0x7fe75414cf60 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7541550b0 0x7fe7541541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 --2- 172.21.15.87:0/125691204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7540cfe70 0x7fe75413a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 >> 172.21.15.87:0/125691204 conn(0x7fe754066b70 msgr2=0x7fe75406c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:48.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 shutdown_connections 2024-04-25T10:22:48.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.349+0000 7fe75c800640 1 -- 172.21.15.87:0/125691204 wait complete. 2024-04-25T10:22:48.375 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:22:48.376 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:22:48.376 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:48.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 Processor -- start 2024-04-25T10:22:48.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 -- start start 2024-04-25T10:22:48.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca8c1550b0 0x7fca8c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca8c0cfe70 0x7fca8c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c15b740 0x7fca8c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c071050 con 0x7fca8c1550b0 2024-04-25T10:22:48.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0712f0 con 0x7fca8c0cfe70 2024-04-25T10:22:48.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca90a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0715d0 con 0x7fca8c15b740 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca8c1550b0 0x7fca8c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca8c1550b0 0x7fca8c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37636/0 (socket says 172.21.15.87:37636) 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 -- 172.21.15.87:0/2122336436 learned_addr learned my addr 172.21.15.87:0/2122336436 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89400640 1 --2- 172.21.15.87:0/2122336436 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0cfe70 0x7fca8c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca8a800640 1 --2- 172.21.15.87:0/2122336436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c15b740 0x7fca8c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 -- 172.21.15.87:0/2122336436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c15b740 msgr2=0x7fca8c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 --2- 172.21.15.87:0/2122336436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c15b740 0x7fca8c06d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 -- 172.21.15.87:0/2122336436 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0cfe70 msgr2=0x7fca8c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 --2- 172.21.15.87:0/2122336436 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0cfe70 0x7fca8c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89e00640 1 -- 172.21.15.87:0/2122336436 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0718b0 con 0x7fca8c1550b0 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca89400640 1 --2- 172.21.15.87:0/2122336436 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0cfe70 0x7fca8c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.509+0000 7fca8a800640 1 --2- 172.21.15.87:0/2122336436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c15b740 0x7fca8c06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:48.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca89e00640 1 --2- 172.21.15.87:0/2122336436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c1550b0 0x7fca8c0cf930 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00bb10 tx=0x7fca7c01e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fa1f91e47749f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:48.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca88a00640 1 -- 172.21.15.87:0/2122336436 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b4070 con 0x7fca8c1550b0 2024-04-25T10:22:48.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca88a00640 1 -- 172.21.15.87:0/2122336436 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c0a7840 con 0x7fca8c1550b0 2024-04-25T10:22:48.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca88a00640 1 -- 172.21.15.87:0/2122336436 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0a7c10 con 0x7fca8c1550b0 2024-04-25T10:22:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 -- 172.21.15.87:0/2122336436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c1550b0 msgr2=0x7fca8c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 --2- 172.21.15.87:0/2122336436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c1550b0 0x7fca8c0cf930 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00bb10 tx=0x7fca7c01e930 comp rx=0 tx=0).stop 2024-04-25T10:22:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 -- 172.21.15.87:0/2122336436 shutdown_connections 2024-04-25T10:22:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 --2- 172.21.15.87:0/2122336436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c15b740 0x7fca8c06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 --2- 172.21.15.87:0/2122336436 >> v2:172.21.15.169:3300/0 conn(0x7fca8c0cfe70 0x7fca8c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 --2- 172.21.15.87:0/2122336436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c1550b0 0x7fca8c0cf930 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 -- 172.21.15.87:0/2122336436 >> 172.21.15.87:0/2122336436 conn(0x7fca8c066b70 msgr2=0x7fca8c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 -- 172.21.15.87:0/2122336436 shutdown_connections 2024-04-25T10:22:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 -- 172.21.15.87:0/2122336436 wait complete. 2024-04-25T10:22:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.513+0000 7fca90a00640 1 Processor -- start 2024-04-25T10:22:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 -- start start 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c0cfe70 0x7fca8c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c1550b0 0x7fca8c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c15b740 0x7fca8c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b3260 con 0x7fca8c0cfe70 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c15bd60 con 0x7fca8c15b740 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b33e0 con 0x7fca8c1550b0 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c0cfe70 0x7fca8c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c0cfe70 0x7fca8c0b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37638/0 (socket says 172.21.15.87:37638) 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 -- 172.21.15.87:0/2091617502 learned_addr learned my addr 172.21.15.87:0/2091617502 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:48.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca8a800640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c15b740 0x7fca8c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89400640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c1550b0 0x7fca8c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 -- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c1550b0 msgr2=0x7fca8c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c1550b0 0x7fca8c0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 -- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c15b740 msgr2=0x7fca8c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c15b740 0x7fca8c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 -- 172.21.15.87:0/2091617502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0cd330 con 0x7fca8c0cfe70 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca89e00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c0cfe70 0x7fca8c0b1680 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fca7c01cfe0 tx=0x7fca7c01d170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca82000640 1 -- 172.21.15.87:0/2091617502 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0bf070 con 0x7fca8c0cfe70 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca82000640 1 -- 172.21.15.87:0/2091617502 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c0af8f0 con 0x7fca8c0cfe70 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca82000640 1 -- 172.21.15.87:0/2091617502 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b4070 con 0x7fca8c0cfe70 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c0cd610 con 0x7fca8c0cfe70 2024-04-25T10:22:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c149aa0 con 0x7fca8c0cfe70 2024-04-25T10:22:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.517+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c04ef30 con 0x7fca8c0cfe70 2024-04-25T10:22:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.521+0000 7fca82000640 1 -- 172.21.15.87:0/2091617502 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca7c0afce0 con 0x7fca8c0cfe70 2024-04-25T10:22:48.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.521+0000 7fca82000640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca6c066f60 0x7fca6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.521+0000 7fca82000640 1 -- 172.21.15.87:0/2091617502 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fca7c11ef70 con 0x7fca8c0cfe70 2024-04-25T10:22:48.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.521+0000 7fca89400640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca6c066f60 0x7fca6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.521+0000 7fca89400640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca6c066f60 0x7fca6c069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fca8c14c3b0 tx=0x7fca78008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:48.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.525+0000 7fca82000640 1 -- 172.21.15.87:0/2091617502 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca7c0187e0 con 0x7fca8c0cfe70 2024-04-25T10:22:48.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca8c0af710 con 0x7fca8c0cfe70 2024-04-25T10:22:48.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca82000640 1 -- 172.21.15.87:0/2091617502 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+13146 (secure 0 0 0) 0x7fca7c0f0190 con 0x7fca8c0cfe70 2024-04-25T10:22:48.684 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:48.684 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":130,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:47.346270+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/2894154211":"2024-04-26T10:22:44.340111+0000","172.21.15.87:6836/2894154211":"2024-04-26T10:22:44.340111+0000","172.21.15.169:6834/85260299":"2024-04-26T10:22:32.952837+0000","172.21.15.169:6832/85260299":"2024-04-26T10:22:32.952837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca6c066f60 msgr2=0x7fca6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca6c066f60 0x7fca6c069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fca8c14c3b0 tx=0x7fca78008040 comp rx=0 tx=0).stop 2024-04-25T10:22:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c0cfe70 msgr2=0x7fca8c0b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c0cfe70 0x7fca8c0b1680 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fca7c01cfe0 tx=0x7fca7c01d170 comp rx=0 tx=0).stop 2024-04-25T10:22:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 shutdown_connections 2024-04-25T10:22:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca6c066f60 0x7fca6c069420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca8c15b740 0x7fca8c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8c1550b0 0x7fca8c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 --2- 172.21.15.87:0/2091617502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca8c0cfe70 0x7fca8c0b1680 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 >> 172.21.15.87:0/2091617502 conn(0x7fca8c066b70 msgr2=0x7fca8c14c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 shutdown_connections 2024-04-25T10:22:48.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.681+0000 7fca90a00640 1 -- 172.21.15.87:0/2091617502 wait complete. 2024-04-25T10:22:48.705 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:48.705 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:48.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.837+0000 7ff409e00640 1 Processor -- start 2024-04-25T10:22:48.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.837+0000 7ff409e00640 1 -- start start 2024-04-25T10:22:48.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff409e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff404066c00 0x7ff404161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff409e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4040b5560 0x7ff4040ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff409e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040b40c0 con 0x7ff404066fe0 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff409e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff404161d60 con 0x7ff4040b5560 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff409e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b5b00 con 0x7ff404066c00 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff3fa000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff404066fe0 0x7ff404072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37188/0 (socket says 172.21.15.87:37188) 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff3fa000640 1 -- 172.21.15.87:0/3132173703 learned_addr learned my addr 172.21.15.87:0/3132173703 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff403400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4040b5560 0x7ff4040ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff402a00640 1 --2- 172.21.15.87:0/3132173703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff404066c00 0x7ff404161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff402000640 1 -- 172.21.15.87:0/3132173703 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383047884 0 0) 0x7ff4040b40c0 con 0x7ff404066fe0 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff402000640 1 -- 172.21.15.87:0/3132173703 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e8003530 con 0x7ff404066fe0 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff402a00640 1 -- 172.21.15.87:0/3132173703 >> v2:172.21.15.87:3300/0 conn(0x7ff4040b5560 msgr2=0x7ff4040ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff402a00640 1 --2- 172.21.15.87:0/3132173703 >> v2:172.21.15.87:3300/0 conn(0x7ff4040b5560 0x7ff4040ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff402a00640 1 -- 172.21.15.87:0/3132173703 >> v1:172.21.15.87:6789/0 conn(0x7ff404066fe0 legacy=0x7ff404072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff402a00640 1 -- 172.21.15.87:0/3132173703 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040b3a00 con 0x7ff404066c00 2024-04-25T10:22:48.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.841+0000 7ff403400640 1 --2- 172.21.15.87:0/3132173703 >> v2:172.21.15.87:3300/0 conn(0x7ff4040b5560 0x7ff4040ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:48.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff402a00640 1 --2- 172.21.15.87:0/3132173703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff404066c00 0x7ff404161820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009d90 tx=0x7ff3f401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d67d23c60c101c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:48.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff402000640 1 -- 172.21.15.87:0/3132173703 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40b2070 con 0x7ff404066c00 2024-04-25T10:22:48.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff402000640 1 -- 172.21.15.87:0/3132173703 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f40a5820 con 0x7ff404066c00 2024-04-25T10:22:48.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff402000640 1 -- 172.21.15.87:0/3132173703 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40a5bd0 con 0x7ff404066c00 2024-04-25T10:22:48.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- 172.21.15.87:0/3132173703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff404066c00 msgr2=0x7ff404161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 --2- 172.21.15.87:0/3132173703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff404066c00 0x7ff404161820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009d90 tx=0x7ff3f401c850 comp rx=0 tx=0).stop 2024-04-25T10:22:48.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- 172.21.15.87:0/3132173703 shutdown_connections 2024-04-25T10:22:48.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 --2- 172.21.15.87:0/3132173703 >> v2:172.21.15.87:3300/0 conn(0x7ff4040b5560 0x7ff4040ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 --2- 172.21.15.87:0/3132173703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff404066c00 0x7ff404161820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- 172.21.15.87:0/3132173703 >> 172.21.15.87:0/3132173703 conn(0x7ff4040b2560 msgr2=0x7ff4040b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:48.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- 172.21.15.87:0/3132173703 shutdown_connections 2024-04-25T10:22:48.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- 172.21.15.87:0/3132173703 wait complete. 2024-04-25T10:22:48.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 Processor -- start 2024-04-25T10:22:48.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- start start 2024-04-25T10:22:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff404066c00 0x7ff40413c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff404066fe0 0x7ff404151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4040b5560 0x7ff40414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40415e540 con 0x7ff404066fe0 2024-04-25T10:22:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4041585a0 con 0x7ff404066c00 2024-04-25T10:22:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.845+0000 7ff409e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40415e6c0 con 0x7ff4040b5560 2024-04-25T10:22:48.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff404066fe0 0x7ff404151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff404066fe0 0x7ff404151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37646/0 (socket says 172.21.15.87:37646) 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 -- 172.21.15.87:0/1057621490 learned_addr learned my addr 172.21.15.87:0/1057621490 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff402a00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff404066c00 0x7ff40413c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff403400640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4040b5560 0x7ff40414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 -- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4040b5560 msgr2=0x7ff40414f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4040b5560 0x7ff40414f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 -- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff404066c00 msgr2=0x7ff40413c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff404066c00 0x7ff40413c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 -- 172.21.15.87:0/1057621490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff404152300 con 0x7ff404066fe0 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fa000640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff404066fe0 0x7ff404151800 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00c450 tx=0x7ff3ec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fbe00640 1 -- 172.21.15.87:0/1057621490 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec017070 con 0x7ff404066fe0 2024-04-25T10:22:48.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fbe00640 1 -- 172.21.15.87:0/1057621490 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec005cf0 con 0x7ff404066fe0 2024-04-25T10:22:48.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fbe00640 1 -- 172.21.15.87:0/1057621490 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0072f0 con 0x7ff404066fe0 2024-04-25T10:22:48.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4041525e0 con 0x7ff404066fe0 2024-04-25T10:22:48.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff404152e80 con 0x7ff404066fe0 2024-04-25T10:22:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4041527b0 con 0x7ff404066fe0 2024-04-25T10:22:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fbe00640 1 -- 172.21.15.87:0/1057621490 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3ec01a070 con 0x7ff404066fe0 2024-04-25T10:22:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.849+0000 7ff3fbe00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3e0066f90 0x7ff3e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.853+0000 7ff3fbe00640 1 -- 172.21.15.87:0/1057621490 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff3ec084140 con 0x7ff404066fe0 2024-04-25T10:22:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.853+0000 7ff402a00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3e0066f90 0x7ff3e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:48.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.853+0000 7ff402a00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3e0066f90 0x7ff3e0069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40af000 tx=0x7ff3f40a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:48.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:48.853+0000 7ff3fbe00640 1 -- 172.21.15.87:0/1057621490 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3ec054d00 con 0x7ff404066fe0 2024-04-25T10:22:49.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.009+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff40404ef30 con 0x7ff404066fe0 2024-04-25T10:22:49.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.009+0000 7ff3fbe00640 1 -- 172.21.15.87:0/1057621490 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3ec058da0 con 0x7ff404066fe0 2024-04-25T10:22:49.013 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:22:49.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3e0066f90 msgr2=0x7ff3e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3e0066f90 0x7ff3e0069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40af000 tx=0x7ff3f40a7070 comp rx=0 tx=0).stop 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff404066fe0 msgr2=0x7ff404151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff404066fe0 0x7ff404151800 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00c450 tx=0x7ff3ec00c850 comp rx=0 tx=0).stop 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 shutdown_connections 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff3e0066f90 0x7ff3e0069450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4040b5560 0x7ff40414f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff404066fe0 0x7ff404151800 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 --2- 172.21.15.87:0/1057621490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff404066c00 0x7ff40413c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 >> 172.21.15.87:0/1057621490 conn(0x7ff4040b2560 msgr2=0x7ff40406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 shutdown_connections 2024-04-25T10:22:49.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.013+0000 7ff409e00640 1 -- 172.21.15.87:0/1057621490 wait complete. 2024-04-25T10:22:49.035 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:22:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b88a00640 1 Processor -- start 2024-04-25T10:22:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b88a00640 1 -- start start 2024-04-25T10:22:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b88a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d0920 0x7f6b840d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b88a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b840d1ce0 0x7f6b840b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b88a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b840d2490 con 0x7f6b8415df00 2024-04-25T10:22:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b88a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840d2770 con 0x7f6b840d1ce0 2024-04-25T10:22:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b88a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840d2a50 con 0x7f6b840d0920 2024-04-25T10:22:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b81e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6b8415df00 0x7f6b840d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37204/0 (socket says 172.21.15.87:37204) 2024-04-25T10:22:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b81e00640 1 -- 172.21.15.87:0/2819967622 learned_addr learned my addr 172.21.15.87:0/2819967622 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b82800640 1 --2- 172.21.15.87:0/2819967622 >> v2:172.21.15.169:3300/0 conn(0x7f6b840d1ce0 0x7f6b840b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b79400640 1 --2- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d0920 0x7f6b840d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b81400640 1 -- 172.21.15.87:0/2819967622 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336889945 0 0) 0x7f6b840d2490 con 0x7f6b8415df00 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b81400640 1 -- 172.21.15.87:0/2819967622 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b68003530 con 0x7f6b8415df00 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b82800640 1 -- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d0920 msgr2=0x7f6b840d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b82800640 1 --2- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d0920 0x7f6b840d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b82800640 1 -- 172.21.15.87:0/2819967622 >> v1:172.21.15.87:6789/0 conn(0x7f6b8415df00 legacy=0x7f6b840d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.169+0000 7f6b82800640 1 -- 172.21.15.87:0/2819967622 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b840d2d30 con 0x7f6b840d1ce0 2024-04-25T10:22:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b82800640 1 --2- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b840d1ce0 0x7f6b840b71e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009d90 tx=0x7f6b7c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56466c36f9c0632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b81400640 1 -- 172.21.15.87:0/2819967622 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0b2070 con 0x7f6b840d1ce0 2024-04-25T10:22:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b81400640 1 -- 172.21.15.87:0/2819967622 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c0a5a20 con 0x7f6b840d1ce0 2024-04-25T10:22:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b81400640 1 -- 172.21.15.87:0/2819967622 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0a5dd0 con 0x7f6b840d1ce0 2024-04-25T10:22:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 -- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b840d1ce0 msgr2=0x7f6b840b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 --2- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b840d1ce0 0x7f6b840b71e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009d90 tx=0x7f6b7c01c940 comp rx=0 tx=0).stop 2024-04-25T10:22:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 -- 172.21.15.87:0/2819967622 shutdown_connections 2024-04-25T10:22:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 --2- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b840d1ce0 0x7f6b840b71e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 --2- 172.21.15.87:0/2819967622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d0920 0x7f6b840d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 -- 172.21.15.87:0/2819967622 >> 172.21.15.87:0/2819967622 conn(0x7f6b8406b290 msgr2=0x7f6b840669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 -- 172.21.15.87:0/2819967622 shutdown_connections 2024-04-25T10:22:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.173+0000 7f6b88a00640 1 -- 172.21.15.87:0/2819967622 wait complete. 2024-04-25T10:22:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 Processor -- start 2024-04-25T10:22:49.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 -- start start 2024-04-25T10:22:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b840d0920 0x7f6b8415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d1ce0 0x7f6b8414d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b8415df00 0x7f6b8414d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840b80b0 con 0x7f6b840d0920 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840b7f30 con 0x7f6b8415df00 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840b8230 con 0x7f6b840d1ce0 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b840d0920 0x7f6b8415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b840d0920 0x7f6b8415b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37650/0 (socket says 172.21.15.87:37650) 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 -- 172.21.15.87:0/621080459 learned_addr learned my addr 172.21.15.87:0/621080459 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b79400640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d1ce0 0x7f6b8414d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b82800640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b8415df00 0x7f6b8414d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 -- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d1ce0 msgr2=0x7f6b8414d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d1ce0 0x7f6b8414d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 -- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b8415df00 msgr2=0x7f6b8414d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b8415df00 0x7f6b8414d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 -- 172.21.15.87:0/621080459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84153b50 con 0x7f6b840d0920 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b81e00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b840d0920 0x7f6b8415b9f0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00c3f0 tx=0x7f6b6c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b7aa00640 1 -- 172.21.15.87:0/621080459 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c017070 con 0x7f6b840d0920 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b7aa00640 1 -- 172.21.15.87:0/621080459 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c005cf0 con 0x7f6b840d0920 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b7aa00640 1 -- 172.21.15.87:0/621080459 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0072f0 con 0x7f6b840d0920 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b84153e30 con 0x7f6b840d0920 2024-04-25T10:22:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.177+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b841543f0 con 0x7f6b840d0920 2024-04-25T10:22:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.181+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8404ef30 con 0x7f6b840d0920 2024-04-25T10:22:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.181+0000 7f6b7aa00640 1 -- 172.21.15.87:0/621080459 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b6c01a070 con 0x7f6b840d0920 2024-04-25T10:22:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.181+0000 7f6b7aa00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b60066f60 0x7f6b60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.181+0000 7f6b7aa00640 1 -- 172.21.15.87:0/621080459 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6b6c084190 con 0x7f6b840d0920 2024-04-25T10:22:49.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.181+0000 7f6b79400640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b60066f60 0x7f6b60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:49.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.181+0000 7f6b79400640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b60066f60 0x7f6b60069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009b60 tx=0x7f6b74009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:49.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.185+0000 7f6b7aa00640 1 -- 172.21.15.87:0/621080459 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b6c054d50 con 0x7f6b840d0920 2024-04-25T10:22:49.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:49.397+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f6b8414ec70 con 0x7f6b840d0920 2024-04-25T10:22:50.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.353+0000 7f6b7aa00640 1 -- 172.21.15.87:0/621080459 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v509) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6b6c058df0 con 0x7f6b840d0920 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b60066f60 msgr2=0x7f6b60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b60066f60 0x7f6b60069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009b60 tx=0x7f6b74009190 comp rx=0 tx=0).stop 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b840d0920 msgr2=0x7f6b8415b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b840d0920 0x7f6b8415b9f0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00c3f0 tx=0x7f6b6c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 shutdown_connections 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6b60066f60 0x7f6b60069420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b8415df00 0x7f6b8414d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b840d1ce0 0x7f6b8414d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 --2- 172.21.15.87:0/621080459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b840d0920 0x7f6b8415b9f0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 >> 172.21.15.87:0/621080459 conn(0x7f6b8406b290 msgr2=0x7f6b840b8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 shutdown_connections 2024-04-25T10:22:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.357+0000 7f6b88a00640 1 -- 172.21.15.87:0/621080459 wait complete. 2024-04-25T10:22:50.385 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:22:50.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7473800640 1 Processor -- start 2024-04-25T10:22:50.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7473800640 1 -- start start 2024-04-25T10:22:50.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7473800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f746c070350 0x7f746c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:50.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7473800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c15b080 con 0x7f746c0aef80 2024-04-25T10:22:50.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7473800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c15adb0 con 0x7f746c0b31c0 2024-04-25T10:22:50.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7473800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c15eb60 con 0x7f746c070350 2024-04-25T10:22:50.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7470e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f746c0aef80 0x7f746c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37214/0 (socket says 172.21.15.87:37214) 2024-04-25T10:22:50.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f7470e00640 1 -- 172.21.15.87:0/3392031638 learned_addr learned my addr 172.21.15.87:0/3392031638 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:50.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f746be00640 1 --2- 172.21.15.87:0/3392031638 >> v2:172.21.15.87:3300/0 conn(0x7f746c070350 0x7f746c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:50.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f746be00640 1 -- 172.21.15.87:0/3392031638 >> v1:172.21.15.169:6789/0 conn(0x7f746c0b31c0 legacy=0x7f746c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:50.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f746be00640 1 -- 172.21.15.87:0/3392031638 >> v1:172.21.15.87:6789/0 conn(0x7f746c0aef80 legacy=0x7f746c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f746b400640 1 -- 172.21.15.87:0/3392031638 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388621638 0 0) 0x7f746c15b080 con 0x7f746c0aef80 2024-04-25T10:22:50.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.525+0000 7f746be00640 1 -- 172.21.15.87:0/3392031638 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c15ee40 con 0x7f746c070350 2024-04-25T10:22:50.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f746be00640 1 --2- 172.21.15.87:0/3392031638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c070350 0x7f746c15e480 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f74580097c0 tx=0x7f745801e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79e2a0e834163d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:50.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f746b400640 1 -- 172.21.15.87:0/3392031638 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74580ab620 con 0x7f746c070350 2024-04-25T10:22:50.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f746b400640 1 -- 172.21.15.87:0/3392031638 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74580ab7c0 con 0x7f746c070350 2024-04-25T10:22:50.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f746b400640 1 -- 172.21.15.87:0/3392031638 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74580abb90 con 0x7f746c070350 2024-04-25T10:22:50.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 -- 172.21.15.87:0/3392031638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c070350 msgr2=0x7f746c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 --2- 172.21.15.87:0/3392031638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c070350 0x7f746c15e480 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f74580097c0 tx=0x7f745801e670 comp rx=0 tx=0).stop 2024-04-25T10:22:50.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 -- 172.21.15.87:0/3392031638 shutdown_connections 2024-04-25T10:22:50.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 --2- 172.21.15.87:0/3392031638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c070350 0x7f746c15e480 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 -- 172.21.15.87:0/3392031638 >> 172.21.15.87:0/3392031638 conn(0x7f746c0669a0 msgr2=0x7f746c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:50.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 -- 172.21.15.87:0/3392031638 shutdown_connections 2024-04-25T10:22:50.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 -- 172.21.15.87:0/3392031638 wait complete. 2024-04-25T10:22:50.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 Processor -- start 2024-04-25T10:22:50.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.529+0000 7f7473800640 1 -- start start 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c070350 0x7f746c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c0aef80 0x7f746c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0b31c0 0x7f746c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0cfef0 con 0x7f746c0aef80 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0cfd70 con 0x7f746c0b31c0 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0b3610 con 0x7f746c070350 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c0aef80 0x7f746c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c0aef80 0x7f746c1f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37664/0 (socket says 172.21.15.87:37664) 2024-04-25T10:22:50.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 -- 172.21.15.87:0/1426493657 learned_addr learned my addr 172.21.15.87:0/1426493657 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7471800640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0b31c0 0x7f746c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7470e00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c070350 0x7f746c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 -- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c070350 msgr2=0x7f746c0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c070350 0x7f746c0b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 -- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0b31c0 msgr2=0x7f746c150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0b31c0 0x7f746c150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 -- 172.21.15.87:0/1426493657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c148460 con 0x7f746c0aef80 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7470e00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c070350 0x7f746c0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f746be00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c0aef80 0x7f746c1f3610 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f745801b280 tx=0x7f745801ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7468c00640 1 -- 172.21.15.87:0/1426493657 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74580c1070 con 0x7f746c0aef80 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7468c00640 1 -- 172.21.15.87:0/1426493657 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74580abd80 con 0x7f746c0aef80 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7468c00640 1 -- 172.21.15.87:0/1426493657 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74580bc650 con 0x7f746c0aef80 2024-04-25T10:22:50.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c0b26e0 con 0x7f746c0aef80 2024-04-25T10:22:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c148b90 con 0x7f746c0aef80 2024-04-25T10:22:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.533+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c04ef30 con 0x7f746c0aef80 2024-04-25T10:22:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.537+0000 7f7468c00640 1 -- 172.21.15.87:0/1426493657 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74580b33f0 con 0x7f746c0aef80 2024-04-25T10:22:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.537+0000 7f7468c00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f744c066f60 0x7f744c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.537+0000 7f7468c00640 1 -- 172.21.15.87:0/1426493657 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7458120d30 con 0x7f746c0aef80 2024-04-25T10:22:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.537+0000 7f7470e00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f744c066f60 0x7f744c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.537+0000 7f7470e00640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f744c066f60 0x7f744c069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7460000f70 tx=0x7f7460009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:50.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.541+0000 7f7468c00640 1 -- 172.21.15.87:0/1426493657 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74580af020 con 0x7f746c0aef80 2024-04-25T10:22:50.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.693+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f746c0b2fa0 con 0x7f746c0aef80 2024-04-25T10:22:50.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.697+0000 7f7468c00640 1 -- 172.21.15.87:0/1426493657 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7458018260 con 0x7f746c0aef80 2024-04-25T10:22:50.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f744c066f60 msgr2=0x7f744c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f744c066f60 0x7f744c069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7460000f70 tx=0x7f7460009040 comp rx=0 tx=0).stop 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c0aef80 msgr2=0x7f746c1f3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c0aef80 0x7f746c1f3610 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f745801b280 tx=0x7f745801ceb0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 shutdown_connections 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f744c066f60 0x7f744c069420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0b31c0 0x7f746c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f746c0aef80 0x7f746c1f3610 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 --2- 172.21.15.87:0/1426493657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c070350 0x7f746c0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 >> 172.21.15.87:0/1426493657 conn(0x7f746c0669a0 msgr2=0x7f746c068bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 shutdown_connections 2024-04-25T10:22:50.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:50.701+0000 7f7473800640 1 -- 172.21.15.87:0/1426493657 wait complete. 2024-04-25T10:22:50.722 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:22:50.722 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:50.781 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:50.878 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:22:50.878 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:22:50.878 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:50.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:50.887 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:22:50.887 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:50.896 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:50.896 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:22:50.896 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:50.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:50.938 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:22:50.939 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:50.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:50.947 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:22:50.948 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:51.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d72e00640 1 Processor -- start 2024-04-25T10:22:51.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d72e00640 1 -- start start 2024-04-25T10:22:51.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d72e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7d6c0ae020 0x7f7d6c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d72e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c0ad840 con 0x7f7d6c0cf930 2024-04-25T10:22:51.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d72e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c15e9f0 con 0x7f7d6c070c20 2024-04-25T10:22:51.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d72e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6c15bb90 con 0x7f7d6c0ae020 2024-04-25T10:22:51.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d6be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7d6c0cf930 0x7f7d6c0cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37230/0 (socket says 172.21.15.87:37230) 2024-04-25T10:22:51.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d6be00640 1 -- 172.21.15.87:0/2039047730 learned_addr learned my addr 172.21.15.87:0/2039047730 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d70e00640 1 --2- 172.21.15.87:0/2039047730 >> v2:172.21.15.169:3300/0 conn(0x7f7d6c0ae020 0x7f7d6c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.109+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430590068 0 0) 0x7f7d6c0ad840 con 0x7f7d6c0cf930 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d50003530 con 0x7f7d6c0cf930 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4010128150 0 0) 0x7f7d50003530 con 0x7f7d6c0cf930 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 >> v2:172.21.15.169:3300/0 conn(0x7f7d6c0ae020 msgr2=0x7f7d6c0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 --2- 172.21.15.87:0/2039047730 >> v2:172.21.15.169:3300/0 conn(0x7f7d6c0ae020 0x7f7d6c0ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 >> v1:172.21.15.169:6789/0 conn(0x7f7d6c070c20 legacy=0x7f7d6c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c15ecd0 con 0x7f7d6c0cf930 2024-04-25T10:22:51.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f7d5c001680 con 0x7f7d6c0cf930 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 >> v1:172.21.15.87:6789/0 conn(0x7f7d6c0cf930 legacy=0x7f7d6c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d500039c0 0x7f7d50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d50004810 0x7f7d50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500051d0 0x7f7d5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15bb90 con 0x7f7d500039c0 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5000c120 con 0x7f7d500051d0 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5000c320 con 0x7f7d50004810 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6be00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d500039c0 0x7f7d50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d70e00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500051d0 0x7f7d5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6b400640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d50004810 0x7f7d50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6be00640 1 -- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d50004810 msgr2=0x7f7d50004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6be00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d50004810 0x7f7d50004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6be00640 1 -- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500051d0 msgr2=0x7f7d5000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6be00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500051d0 0x7f7d5000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d6be00640 1 -- 172.21.15.87:0/2039047730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50003530 con 0x7f7d500039c0 2024-04-25T10:22:51.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.113+0000 7f7d70e00640 1 -- 172.21.15.87:0/2039047730 reap_dead start 2024-04-25T10:22:51.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6be00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d500039c0 0x7f7d50003de0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c1c0 tx=0x7f7d5c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b585c32b6b8c7476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:51.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c007d00 con 0x7f7d500039c0 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c0a9020 con 0x7f7d500039c0 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6aa00640 1 -- 172.21.15.87:0/2039047730 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0a5b80 con 0x7f7d500039c0 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- 172.21.15.87:0/2039047730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d500039c0 msgr2=0x7f7d50003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d500039c0 0x7f7d50003de0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c1c0 tx=0x7f7d5c01cbf0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- 172.21.15.87:0/2039047730 shutdown_connections 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 --2- 172.21.15.87:0/2039047730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d500039c0 0x7f7d50003de0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- 172.21.15.87:0/2039047730 >> 172.21.15.87:0/2039047730 conn(0x7f7d6c066b70 msgr2=0x7f7d6c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- 172.21.15.87:0/2039047730 shutdown_connections 2024-04-25T10:22:51.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- 172.21.15.87:0/2039047730 wait complete. 2024-04-25T10:22:51.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 Processor -- start 2024-04-25T10:22:51.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- start start 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d500039c0 0x7f7d6c0c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6c1562a0 0x7f7d6c1468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c161500 0x7f7d6c161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c06e130 con 0x7f7d6c1562a0 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c0ae840 con 0x7f7d6c161500 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d72e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c0b30f0 con 0x7f7d500039c0 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6c1562a0 0x7f7d6c1468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6c1562a0 0x7f7d6c1468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37688/0 (socket says 172.21.15.87:37688) 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6b400640 1 -- 172.21.15.87:0/3572599510 learned_addr learned my addr 172.21.15.87:0/3572599510 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.117+0000 7f7d6be00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d500039c0 0x7f7d6c0c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d70e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c161500 0x7f7d6c161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d6b400640 1 -- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d500039c0 msgr2=0x7f7d6c0c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d6b400640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d500039c0 0x7f7d6c0c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d6b400640 1 -- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c161500 msgr2=0x7f7d6c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d6b400640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c161500 0x7f7d6c161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d6b400640 1 -- 172.21.15.87:0/3572599510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c0c4c20 con 0x7f7d6c1562a0 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d70e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c161500 0x7f7d6c161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d6b400640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6c1562a0 0x7f7d6c1468b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800f820 tx=0x7f7d5800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d63e00640 1 -- 172.21.15.87:0/3572599510 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58001cf0 con 0x7f7d6c1562a0 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d63e00640 1 -- 172.21.15.87:0/3572599510 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58018070 con 0x7f7d6c1562a0 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c148b30 con 0x7f7d6c1562a0 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d63e00640 1 -- 172.21.15.87:0/3572599510 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800d730 con 0x7f7d6c1562a0 2024-04-25T10:22:51.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c0c5180 con 0x7f7d6c1562a0 2024-04-25T10:22:51.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c04ef30 con 0x7f7d6c1562a0 2024-04-25T10:22:51.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d63e00640 1 -- 172.21.15.87:0/3572599510 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d5800d8d0 con 0x7f7d6c1562a0 2024-04-25T10:22:51.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d63e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d4c066f60 0x7f7d4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d63e00640 1 -- 172.21.15.87:0/3572599510 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7d58084c90 con 0x7f7d6c1562a0 2024-04-25T10:22:51.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.121+0000 7f7d6be00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d4c066f60 0x7f7d4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.125+0000 7f7d6be00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d4c066f60 0x7f7d4c069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0aa000 tx=0x7f7d6c15ebc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:51.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.125+0000 7f7d63e00640 1 -- 172.21.15.87:0/3572599510 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d6c0c4c20 con 0x7f7d6c1562a0 2024-04-25T10:22:51.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d6c1471e0 con 0x7f7d6c1562a0 2024-04-25T10:22:51.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d63e00640 1 -- 172.21.15.87:0/3572599510 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+2905 (secure 0 0 0) 0x7f7d58055e10 con 0x7f7d6c1562a0 2024-04-25T10:22:51.316 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:51.317 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":95,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2713911036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2713911036},{"type":"v1","addr":"172.21.15.169:6835","nonce":2713911036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1769937639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1769937639},{"type":"v1","addr":"172.21.15.87:6835","nonce":1769937639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9495,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/88703150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":88703150},{"type":"v1","addr":"172.21.15.87:6837","nonce":88703150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[]} 2024-04-25T10:22:51.317 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 95 2024-04-25T10:22:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d4c066f60 msgr2=0x7f7d4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d4c066f60 0x7f7d4c069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0aa000 tx=0x7f7d6c15ebc0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6c1562a0 msgr2=0x7f7d6c1468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6c1562a0 0x7f7d6c1468b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800f820 tx=0x7f7d5800fce0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 shutdown_connections 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7d4c066f60 0x7f7d4c069420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c161500 0x7f7d6c161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6c1562a0 0x7f7d6c1468b0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 --2- 172.21.15.87:0/3572599510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d500039c0 0x7f7d6c0c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 >> 172.21.15.87:0/3572599510 conn(0x7f7d6c066b70 msgr2=0x7f7d6c068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 shutdown_connections 2024-04-25T10:22:51.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.313+0000 7f7d72e00640 1 -- 172.21.15.87:0/3572599510 wait complete. 2024-04-25T10:22:51.339 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:22:51.339 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:22:51.339 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:22:51.339 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:51.339 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:22:51.339 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:22:51.340 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:22:51.340 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:51.340 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:22:51.340 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:22:51.340 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:22:51.340 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:51.340 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:22:51.340 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:22:51.344 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:22:51.344 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:51.351 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:22:51.352 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:22:51.352 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:22:51.352 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:22:51.354 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:22:51.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:22:51.402 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:22:51.402 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:22:51.402 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:51.402 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:51.403 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:22:51.403 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:51.403 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:22:51.437 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:22:51.489 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:22:51.489 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:22:51.490 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:22:51.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e7d800640 1 Processor -- start 2024-04-25T10:22:51.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e7d800640 1 -- start start 2024-04-25T10:22:51.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e7d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e7d800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e780ccfb0 con 0x7f8e7815b9e0 2024-04-25T10:22:51.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e7d800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e780cd290 con 0x7f8e780ccb20 2024-04-25T10:22:51.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e7d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e780cd570 con 0x7f8e780c6a20 2024-04-25T10:22:51.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e76a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8e7815b9e0 0x7f8e780cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37256/0 (socket says 172.21.15.87:37256) 2024-04-25T10:22:51.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e76a00640 1 -- 172.21.15.87:0/2316352972 learned_addr learned my addr 172.21.15.87:0/2316352972 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:51.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e77400640 1 --2- 172.21.15.87:0/2316352972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e76000640 1 -- 172.21.15.87:0/2316352972 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637055003 0 0) 0x7f8e780ccfb0 con 0x7f8e7815b9e0 2024-04-25T10:22:51.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e76000640 1 -- 172.21.15.87:0/2316352972 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e5c003530 con 0x7f8e7815b9e0 2024-04-25T10:22:51.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e76000640 1 -- 172.21.15.87:0/2316352972 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405082310 0 0) 0x7f8e780cd290 con 0x7f8e780ccb20 2024-04-25T10:22:51.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e76000640 1 -- 172.21.15.87:0/2316352972 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e780ccfb0 con 0x7f8e780ccb20 2024-04-25T10:22:51.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e77400640 1 -- 172.21.15.87:0/2316352972 >> v1:172.21.15.169:6789/0 conn(0x7f8e780ccb20 legacy=0x7f8e780d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e77400640 1 -- 172.21.15.87:0/2316352972 >> v1:172.21.15.87:6789/0 conn(0x7f8e7815b9e0 legacy=0x7f8e780cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.545+0000 7f8e77400640 1 -- 172.21.15.87:0/2316352972 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e780cd850 con 0x7f8e780c6a20 2024-04-25T10:22:51.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e77400640 1 --2- 172.21.15.87:0/2316352972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78166820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009dc0 tx=0x7f8e7001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b714c93690dfbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:51.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e76000640 1 -- 172.21.15.87:0/2316352972 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700b2070 con 0x7f8e780c6a20 2024-04-25T10:22:51.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e76000640 1 -- 172.21.15.87:0/2316352972 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e700a5900 con 0x7f8e780c6a20 2024-04-25T10:22:51.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e76000640 1 -- 172.21.15.87:0/2316352972 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700a5cd0 con 0x7f8e780c6a20 2024-04-25T10:22:51.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 -- 172.21.15.87:0/2316352972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 msgr2=0x7f8e78166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 --2- 172.21.15.87:0/2316352972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78166820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009dc0 tx=0x7f8e7001c990 comp rx=0 tx=0).stop 2024-04-25T10:22:51.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 -- 172.21.15.87:0/2316352972 shutdown_connections 2024-04-25T10:22:51.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 --2- 172.21.15.87:0/2316352972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78166820 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 -- 172.21.15.87:0/2316352972 >> 172.21.15.87:0/2316352972 conn(0x7f8e7809a9d0 msgr2=0x7f8e7809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:51.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 -- 172.21.15.87:0/2316352972 shutdown_connections 2024-04-25T10:22:51.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 -- 172.21.15.87:0/2316352972 wait complete. 2024-04-25T10:22:51.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 Processor -- start 2024-04-25T10:22:51.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.549+0000 7f8e7d800640 1 -- start start 2024-04-25T10:22:51.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e7815b9e0 0x7f8e7806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7806c4b0 0x7f8e7815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e78169910 con 0x7f8e7806c4b0 2024-04-25T10:22:51.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e78169790 con 0x7f8e7815b9e0 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e78169a90 con 0x7f8e780c6a20 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7806c4b0 0x7f8e7815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7806c4b0 0x7f8e7815ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37726/0 (socket says 172.21.15.87:37726) 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 -- 172.21.15.87:0/3084313783 learned_addr learned my addr 172.21.15.87:0/3084313783 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e76a00640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e6fe00640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e7815b9e0 0x7f8e7806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 -- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 msgr2=0x7f8e78152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 -- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e7815b9e0 msgr2=0x7f8e7806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e7815b9e0 0x7f8e7806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 -- 172.21.15.87:0/3084313783 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7806c950 con 0x7f8e7806c4b0 2024-04-25T10:22:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e77400640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7806c4b0 0x7f8e7815ab50 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f8e700b0040 tx=0x7f8e70019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:51.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e6ea00640 1 -- 172.21.15.87:0/3084313783 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700b2050 con 0x7f8e7806c4b0 2024-04-25T10:22:51.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e6ea00640 1 -- 172.21.15.87:0/3084313783 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e700be070 con 0x7f8e7806c4b0 2024-04-25T10:22:51.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7815d1d0 con 0x7f8e7806c4b0 2024-04-25T10:22:51.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e6ea00640 1 -- 172.21.15.87:0/3084313783 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700ad890 con 0x7f8e7806c4b0 2024-04-25T10:22:51.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7815d750 con 0x7f8e7806c4b0 2024-04-25T10:22:51.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.553+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7804ef30 con 0x7f8e7806c4b0 2024-04-25T10:22:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.557+0000 7f8e6ea00640 1 -- 172.21.15.87:0/3084313783 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e700ada30 con 0x7f8e7806c4b0 2024-04-25T10:22:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.557+0000 7f8e6ea00640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8e54066f60 0x7f8e54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.557+0000 7f8e6ea00640 1 -- 172.21.15.87:0/3084313783 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8e7011fde0 con 0x7f8e7806c4b0 2024-04-25T10:22:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.557+0000 7f8e76a00640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8e54066f60 0x7f8e54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.557+0000 7f8e76a00640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8e54066f60 0x7f8e54069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8e68007fd0 tx=0x7f8e68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.561+0000 7f8e6ea00640 1 -- 172.21.15.87:0/3084313783 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e70016580 con 0x7f8e7806c4b0 2024-04-25T10:22:51.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:51.729+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8e780c5900 con 0x7f8e7806c4b0 2024-04-25T10:22:52.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.709+0000 7f8e6ea00640 1 -- 172.21.15.87:0/3084313783 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v134) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8e700f0a60 con 0x7f8e7806c4b0 2024-04-25T10:22:52.713 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:22:52.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8e54066f60 msgr2=0x7f8e54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8e54066f60 0x7f8e54069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8e68007fd0 tx=0x7f8e68009040 comp rx=0 tx=0).stop 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7806c4b0 msgr2=0x7f8e7815ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7806c4b0 0x7f8e7815ab50 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f8e700b0040 tx=0x7f8e70019730 comp rx=0 tx=0).stop 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 shutdown_connections 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8e54066f60 0x7f8e54069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7806c4b0 0x7f8e7815ab50 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e7815b9e0 0x7f8e7806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 --2- 172.21.15.87:0/3084313783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780c6a20 0x7f8e78152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 >> 172.21.15.87:0/3084313783 conn(0x7f8e7809a9d0 msgr2=0x7f8e7806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 shutdown_connections 2024-04-25T10:22:52.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.713+0000 7f8e7d800640 1 -- 172.21.15.87:0/3084313783 wait complete. 2024-04-25T10:22:52.739 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:22:52.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.873+0000 7fb7e3e00640 1 Processor -- start 2024-04-25T10:22:52.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.873+0000 7fb7e3e00640 1 -- start start 2024-04-25T10:22:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7dc15a0e0 0x7fb7dc15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7dc071f70 0x7fb7dc0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc15b6e0 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e3e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7dc0b3420 con 0x7fb7dc15a0e0 2024-04-25T10:22:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7dc070ed0 con 0x7fb7dc071f70 2024-04-25T10:22:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7dc15a0e0 0x7fb7dc15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7dc15a0e0 0x7fb7dc15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37738/0 (socket says 172.21.15.87:37738) 2024-04-25T10:22:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e1400640 1 -- 172.21.15.87:0/904219114 learned_addr learned my addr 172.21.15.87:0/904219114 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e0a00640 1 --2- 172.21.15.87:0/904219114 >> v2:172.21.15.169:3300/0 conn(0x7fb7dc071f70 0x7fb7dc0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e1400640 1 -- 172.21.15.87:0/904219114 >> v2:172.21.15.169:3300/0 conn(0x7fb7dc071f70 msgr2=0x7fb7dc0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e1400640 1 --2- 172.21.15.87:0/904219114 >> v2:172.21.15.169:3300/0 conn(0x7fb7dc071f70 0x7fb7dc0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e1400640 1 -- 172.21.15.87:0/904219114 >> v1:172.21.15.169:6789/0 conn(0x7fb7dc0b2fc0 legacy=0x7fb7dc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.877+0000 7fb7e1400640 1 -- 172.21.15.87:0/904219114 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc0711b0 con 0x7fb7dc15a0e0 2024-04-25T10:22:52.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e1400640 1 --2- 172.21.15.87:0/904219114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc15b1a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc06e5d0 tx=0x7fb7d001e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371a5b7cea7024f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:52.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7dbe00640 1 -- 172.21.15.87:0/904219114 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00ad070 con 0x7fb7dc15a0e0 2024-04-25T10:22:52.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7dbe00640 1 -- 172.21.15.87:0/904219114 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d0004230 con 0x7fb7dc15a0e0 2024-04-25T10:22:52.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7dbe00640 1 -- 172.21.15.87:0/904219114 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00a7810 con 0x7fb7dc15a0e0 2024-04-25T10:22:52.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 -- 172.21.15.87:0/904219114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc15a0e0 msgr2=0x7fb7dc15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/904219114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc15b1a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc06e5d0 tx=0x7fb7d001e9a0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 -- 172.21.15.87:0/904219114 shutdown_connections 2024-04-25T10:22:52.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/904219114 >> v2:172.21.15.169:3300/0 conn(0x7fb7dc071f70 0x7fb7dc0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/904219114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc15b1a0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 -- 172.21.15.87:0/904219114 >> 172.21.15.87:0/904219114 conn(0x7fb7dc066b70 msgr2=0x7fb7dc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:52.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 -- 172.21.15.87:0/904219114 shutdown_connections 2024-04-25T10:22:52.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 -- 172.21.15.87:0/904219114 wait complete. 2024-04-25T10:22:52.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 Processor -- start 2024-04-25T10:22:52.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 -- start start 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.881+0000 7fb7e3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7dc071f70 0x7fb7dc15e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc0b2fc0 0x7fb7dc0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc0af030 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc071a50 con 0x7fb7dc15a0e0 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc15a4c0 con 0x7fb7dc071f70 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc0b2fc0 0x7fb7dc0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc0b2fc0 0x7fb7dc0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37754/0 (socket says 172.21.15.87:37754) 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 -- 172.21.15.87:0/1118022293 learned_addr learned my addr 172.21.15.87:0/1118022293 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:52.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e1400640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7dc071f70 0x7fb7dc15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e1e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 -- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7dc071f70 msgr2=0x7fb7dc15e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7dc071f70 0x7fb7dc15e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 -- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7dc15a0e0 msgr2=0x7fb7dc14c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc14c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 -- 172.21.15.87:0/1118022293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc0b0320 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e1e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc14c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e0a00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc0b2fc0 0x7fb7dc0ad8e0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800d830 tx=0x7fb7c800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7d9600640 1 -- 172.21.15.87:0/1118022293 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800b6c0 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7d9600640 1 -- 172.21.15.87:0/1118022293 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8019070 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7d9600640 1 -- 172.21.15.87:0/1118022293 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800ba60 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc0b0600 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc0ad0e0 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7dc04ef30 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.885+0000 7fb7d9600640 1 -- 172.21.15.87:0/1118022293 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7c801e460 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.889+0000 7fb7d9600640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb7bc066f60 0x7fb7bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:52.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.889+0000 7fb7d9600640 1 -- 172.21.15.87:0/1118022293 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb7c8085340 con 0x7fb7dc0b2fc0 2024-04-25T10:22:52.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.889+0000 7fb7e1400640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb7bc066f60 0x7fb7bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:52.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.889+0000 7fb7e1400640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb7bc066f60 0x7fb7bc069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00b1000 tx=0x7fb7d00b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:52.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:52.893+0000 7fb7d9600640 1 -- 172.21.15.87:0/1118022293 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7c8056690 con 0x7fb7dc0b2fc0 2024-04-25T10:22:53.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb7dc0703e0 con 0x7fb7dc0b2fc0 2024-04-25T10:22:53.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7d9600640 1 -- 172.21.15.87:0/1118022293 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb7c8055f90 con 0x7fb7dc0b2fc0 2024-04-25T10:22:53.104 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:53.104 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:22:53.104 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:22:53.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.105 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.106 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.107 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:22:53.108 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.110 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:22:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.113 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:22:53.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:22:53.117 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:22:53.117 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:22:53.117 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.117 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:53.117 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:22:53.117 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb7bc066f60 msgr2=0x7fb7bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb7bc066f60 0x7fb7bc069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00b1000 tx=0x7fb7d00b3040 comp rx=0 tx=0).stop 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc0b2fc0 msgr2=0x7fb7dc0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc0b2fc0 0x7fb7dc0ad8e0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800d830 tx=0x7fb7c800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 shutdown_connections 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb7bc066f60 0x7fb7bc069420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7dc15a0e0 0x7fb7dc14c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7dc0b2fc0 0x7fb7dc0ad8e0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 --2- 172.21.15.87:0/1118022293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7dc071f70 0x7fb7dc15e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 >> 172.21.15.87:0/1118022293 conn(0x7fb7dc066b70 msgr2=0x7fb7dc15d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:53.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 shutdown_connections 2024-04-25T10:22:53.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.101+0000 7fb7e3e00640 1 -- 172.21.15.87:0/1118022293 wait complete. 2024-04-25T10:22:53.126 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:53.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.257+0000 7f5dd2200640 1 Processor -- start 2024-04-25T10:22:53.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.257+0000 7f5dd2200640 1 -- start start 2024-04-25T10:22:53.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dd2200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5dcc0cda10 0x7f5dcc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dd2200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5dcc0cf930 0x7f5dcc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dd2200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc0b2770 con 0x7f5dcc15bc40 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dd2200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc06c670 con 0x7f5dcc0cf930 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dd2200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc06c950 con 0x7f5dcc0cda10 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5dcc0cf930 0x7f5dcc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5dcc0cf930 0x7f5dcc0cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37758/0 (socket says 172.21.15.87:37758) 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcbe00640 1 -- 172.21.15.87:0/1539340999 learned_addr learned my addr 172.21.15.87:0/1539340999 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcb400640 1 --1- 172.21.15.87:0/1539340999 >> v1:172.21.15.87:6789/0 conn(0x7f5dcc15bc40 0x7f5dcc0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37264/0 (socket says 172.21.15.87:37264) 2024-04-25T10:22:53.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dc3e00640 1 --2- 172.21.15.87:0/1539340999 >> v2:172.21.15.169:3300/0 conn(0x7f5dcc0cda10 0x7f5dcc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcbe00640 1 -- 172.21.15.87:0/1539340999 >> v2:172.21.15.169:3300/0 conn(0x7f5dcc0cda10 msgr2=0x7f5dcc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:53.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcbe00640 1 --2- 172.21.15.87:0/1539340999 >> v2:172.21.15.169:3300/0 conn(0x7f5dcc0cda10 0x7f5dcc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcbe00640 1 -- 172.21.15.87:0/1539340999 >> v1:172.21.15.87:6789/0 conn(0x7f5dcc15bc40 legacy=0x7f5dcc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.261+0000 7f5dcbe00640 1 -- 172.21.15.87:0/1539340999 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc06cc30 con 0x7f5dcc0cf930 2024-04-25T10:22:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dcbe00640 1 --2- 172.21.15.87:0/1539340999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0cfdc0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0726f0 tx=0x7f5dc401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1652427a8bced00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dcaa00640 1 -- 172.21.15.87:0/1539340999 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40a5620 con 0x7f5dcc0cf930 2024-04-25T10:22:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dcaa00640 1 -- 172.21.15.87:0/1539340999 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc40a57c0 con 0x7f5dcc0cf930 2024-04-25T10:22:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dcaa00640 1 -- 172.21.15.87:0/1539340999 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40a5b90 con 0x7f5dcc0cf930 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- 172.21.15.87:0/1539340999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 msgr2=0x7f5dcc0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1539340999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0cfdc0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0726f0 tx=0x7f5dc401c960 comp rx=0 tx=0).stop 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- 172.21.15.87:0/1539340999 shutdown_connections 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1539340999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0cfdc0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1539340999 >> v2:172.21.15.169:3300/0 conn(0x7f5dcc0cda10 0x7f5dcc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- 172.21.15.87:0/1539340999 >> 172.21.15.87:0/1539340999 conn(0x7f5dcc066b70 msgr2=0x7f5dcc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- 172.21.15.87:0/1539340999 shutdown_connections 2024-04-25T10:22:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- 172.21.15.87:0/1539340999 wait complete. 2024-04-25T10:22:53.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 Processor -- start 2024-04-25T10:22:53.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- start start 2024-04-25T10:22:53.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dcc0cda10 0x7f5dcc144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc15bc40 0x7f5dcc155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0b3850 con 0x7f5dcc0cf930 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0b36d0 con 0x7f5dcc15bc40 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.265+0000 7f5dd2200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0b39d0 con 0x7f5dcc0cda10 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37760/0 (socket says 172.21.15.87:37760) 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 -- 172.21.15.87:0/1491535053 learned_addr learned my addr 172.21.15.87:0/1491535053 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dcbe00640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc15bc40 0x7f5dcc155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 -- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dcc0cda10 msgr2=0x7f5dcc144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dcc0cda10 0x7f5dcc144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 -- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc15bc40 msgr2=0x7f5dcc155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc15bc40 0x7f5dcc155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 -- 172.21.15.87:0/1491535053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc1526d0 con 0x7f5dcc0cf930 2024-04-25T10:22:53.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dcbe00640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc15bc40 0x7f5dcc155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3e00640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0aae50 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f5db400d620 tx=0x7f5db400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3400640 1 -- 172.21.15.87:0/1491535053 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400b620 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3400640 1 -- 172.21.15.87:0/1491535053 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4019070 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3400640 1 -- 172.21.15.87:0/1491535053 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400b9e0 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc1529b0 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc152f70 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc0aa5b0 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3400640 1 -- 172.21.15.87:0/1491535053 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5db400bb80 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.269+0000 7f5dc3400640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5da8066f60 0x7f5da8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.273+0000 7f5dc3400640 1 -- 172.21.15.87:0/1491535053 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5db4085190 con 0x7f5dcc0cf930 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.273+0000 7f5dcb400640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5da8066f60 0x7f5da8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.273+0000 7f5dcb400640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5da8066f60 0x7f5da8069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc000f70 tx=0x7f5dbc007a10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.277+0000 7f5dc3400640 1 -- 172.21.15.87:0/1491535053 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5db4056450 con 0x7f5dcc0cf930 2024-04-25T10:22:53.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dcc04ef30 con 0x7f5dcc0cf930 2024-04-25T10:22:53.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dc3400640 1 -- 172.21.15.87:0/1491535053 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f5db4055d50 con 0x7f5dcc0cf930 2024-04-25T10:22:53.436 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:53.436 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":134,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:52.712672+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:53.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5da8066f60 msgr2=0x7f5da8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5da8066f60 0x7f5da8069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc000f70 tx=0x7f5dbc007a10 comp rx=0 tx=0).stop 2024-04-25T10:22:53.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 msgr2=0x7f5dcc0aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0aae50 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f5db400d620 tx=0x7f5db400dae0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 shutdown_connections 2024-04-25T10:22:53.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5da8066f60 0x7f5da8069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc15bc40 0x7f5dcc155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dcc0cf930 0x7f5dcc0aae50 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 --2- 172.21.15.87:0/1491535053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dcc0cda10 0x7f5dcc144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 >> 172.21.15.87:0/1491535053 conn(0x7f5dcc066b70 msgr2=0x7f5dcc157dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:53.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 shutdown_connections 2024-04-25T10:22:53.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.433+0000 7f5dd2200640 1 -- 172.21.15.87:0/1491535053 wait complete. 2024-04-25T10:22:53.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:53.458 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:53.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 Processor -- start 2024-04-25T10:22:53.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 -- start start 2024-04-25T10:22:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f326c0cfb50 0x7f326c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f326c071bc0 0x7f326c072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c15bb90 con 0x7f326c071bc0 2024-04-25T10:22:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c0ad840 con 0x7f326c0cfb50 2024-04-25T10:22:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3270c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c15e710 con 0x7f326c0b3480 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f326c071bc0 0x7f326c072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f326c071bc0 0x7f326c072050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37762/0 (socket says 172.21.15.87:37762) 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 -- 172.21.15.87:0/2172416973 learned_addr learned my addr 172.21.15.87:0/2172416973 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3269400640 1 --2- 172.21.15.87:0/2172416973 >> v2:172.21.15.169:3300/0 conn(0x7f326c0cfb50 0x7f326c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 -- 172.21.15.87:0/2172416973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 msgr2=0x7f326c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3269e00640 1 --2- 172.21.15.87:0/2172416973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 --2- 172.21.15.87:0/2172416973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c15e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 -- 172.21.15.87:0/2172416973 >> v2:172.21.15.169:3300/0 conn(0x7f326c0cfb50 msgr2=0x7f326c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 --2- 172.21.15.87:0/2172416973 >> v2:172.21.15.169:3300/0 conn(0x7f326c0cfb50 0x7f326c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f326a800640 1 -- 172.21.15.87:0/2172416973 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c15e9f0 con 0x7f326c071bc0 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3269400640 1 --2- 172.21.15.87:0/2172416973 >> v2:172.21.15.169:3300/0 conn(0x7f326c0cfb50 0x7f326c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.593+0000 7f3269e00640 1 --2- 172.21.15.87:0/2172416973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f326a800640 1 --2- 172.21.15.87:0/2172416973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c071bc0 0x7f326c072050 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f326c06e4f0 tx=0x7f326401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0cad259846ea99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3268a00640 1 -- 172.21.15.87:0/2172416973 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640a5670 con 0x7f326c071bc0 2024-04-25T10:22:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3268a00640 1 -- 172.21.15.87:0/2172416973 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32640a5810 con 0x7f326c071bc0 2024-04-25T10:22:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3268a00640 1 -- 172.21.15.87:0/2172416973 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640a5be0 con 0x7f326c071bc0 2024-04-25T10:22:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 -- 172.21.15.87:0/2172416973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c071bc0 msgr2=0x7f326c072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 --2- 172.21.15.87:0/2172416973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c071bc0 0x7f326c072050 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f326c06e4f0 tx=0x7f326401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 -- 172.21.15.87:0/2172416973 shutdown_connections 2024-04-25T10:22:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 --2- 172.21.15.87:0/2172416973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c071bc0 0x7f326c072050 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 --2- 172.21.15.87:0/2172416973 >> v2:172.21.15.169:3300/0 conn(0x7f326c0cfb50 0x7f326c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 --2- 172.21.15.87:0/2172416973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 -- 172.21.15.87:0/2172416973 >> 172.21.15.87:0/2172416973 conn(0x7f326c0669a0 msgr2=0x7f326c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:53.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 -- 172.21.15.87:0/2172416973 shutdown_connections 2024-04-25T10:22:53.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 -- 172.21.15.87:0/2172416973 wait complete. 2024-04-25T10:22:53.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.597+0000 7f3270c00640 1 Processor -- start 2024-04-25T10:22:53.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 -- start start 2024-04-25T10:22:53.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f326c071bc0 0x7f326c14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c0cfb50 0x7f326c1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c070a10 con 0x7f326c0cfb50 2024-04-25T10:22:53.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c070890 con 0x7f326c071bc0 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0ae530 con 0x7f326c0b3480 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c0cfb50 0x7f326c1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c0cfb50 0x7f326c1590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37770/0 (socket says 172.21.15.87:37770) 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 -- 172.21.15.87:0/2678232716 learned_addr learned my addr 172.21.15.87:0/2678232716 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3269400640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3269e00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f326c071bc0 0x7f326c14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 -- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 msgr2=0x7f326c14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 -- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f326c071bc0 msgr2=0x7f326c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f326c071bc0 0x7f326c14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 -- 172.21.15.87:0/2678232716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c14bec0 con 0x7f326c0cfb50 2024-04-25T10:22:53.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f326a800640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c0cfb50 0x7f326c1590b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f32640af040 tx=0x7f3264019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3262000640 1 -- 172.21.15.87:0/2678232716 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640bc020 con 0x7f326c0cfb50 2024-04-25T10:22:53.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3262000640 1 -- 172.21.15.87:0/2678232716 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32640b86b0 con 0x7f326c0cfb50 2024-04-25T10:22:53.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3262000640 1 -- 172.21.15.87:0/2678232716 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640b8aa0 con 0x7f326c0cfb50 2024-04-25T10:22:53.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c14c1a0 con 0x7f326c0cfb50 2024-04-25T10:22:53.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c156190 con 0x7f326c0cfb50 2024-04-25T10:22:53.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.601+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c04ef30 con 0x7f326c0cfb50 2024-04-25T10:22:53.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.605+0000 7f3262000640 1 -- 172.21.15.87:0/2678232716 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32640bf020 con 0x7f326c0cfb50 2024-04-25T10:22:53.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.605+0000 7f3262000640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f324c066f60 0x7f324c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.605+0000 7f3262000640 1 -- 172.21.15.87:0/2678232716 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f326411fb40 con 0x7f326c0cfb50 2024-04-25T10:22:53.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.605+0000 7f3269e00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f324c066f60 0x7f324c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.605+0000 7f3269e00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f324c066f60 0x7f324c069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f325c006b20 tx=0x7f325c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.609+0000 7f3262000640 1 -- 172.21.15.87:0/2678232716 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32640165e0 con 0x7f326c0cfb50 2024-04-25T10:22:53.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.761+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f326c0aee90 con 0x7f326c0cfb50 2024-04-25T10:22:53.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3262000640 1 -- 172.21.15.87:0/2678232716 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32640a57e0 con 0x7f326c0cfb50 2024-04-25T10:22:53.767 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:22:53.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f324c066f60 msgr2=0x7f324c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f324c066f60 0x7f324c069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f325c006b20 tx=0x7f325c008040 comp rx=0 tx=0).stop 2024-04-25T10:22:53.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c0cfb50 msgr2=0x7f326c1590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c0cfb50 0x7f326c1590b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f32640af040 tx=0x7f3264019790 comp rx=0 tx=0).stop 2024-04-25T10:22:53.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 shutdown_connections 2024-04-25T10:22:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f324c066f60 0x7f324c069420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f326c0cfb50 0x7f326c1590b0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f326c0b3480 0x7f326c14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 --2- 172.21.15.87:0/2678232716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f326c071bc0 0x7f326c14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 >> 172.21.15.87:0/2678232716 conn(0x7f326c0669a0 msgr2=0x7f326c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 shutdown_connections 2024-04-25T10:22:53.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.765+0000 7f3270c00640 1 -- 172.21.15.87:0/2678232716 wait complete. 2024-04-25T10:22:53.788 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:22:53.794 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:53.842 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:53.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.921+0000 7f6f3bc00640 1 Processor -- start 2024-04-25T10:22:53.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.921+0000 7f6f3bc00640 1 -- start start 2024-04-25T10:22:53.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f3bc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f3415bc40 0x7f6f3406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f3bc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6f340cf930 0x7f6f340cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f3bc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f340b2770 con 0x7f6f340cda10 2024-04-25T10:22:53.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f3bc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3406c670 con 0x7f6f340cf930 2024-04-25T10:22:53.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f3bc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3406c950 con 0x7f6f3415bc40 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6f340cf930 0x7f6f340cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6f340cf930 0x7f6f340cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37780/0 (socket says 172.21.15.87:37780) 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 -- 172.21.15.87:0/760446907 learned_addr learned my addr 172.21.15.87:0/760446907 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39200640 1 --2- 172.21.15.87:0/760446907 >> v2:172.21.15.169:3300/0 conn(0x7f6f3415bc40 0x7f6f3406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 -- 172.21.15.87:0/760446907 >> v2:172.21.15.169:3300/0 conn(0x7f6f3415bc40 msgr2=0x7f6f3406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 --2- 172.21.15.87:0/760446907 >> v2:172.21.15.169:3300/0 conn(0x7f6f3415bc40 0x7f6f3406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 -- 172.21.15.87:0/760446907 >> v1:172.21.15.169:6789/0 conn(0x7f6f340cda10 legacy=0x7f6f340b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 -- 172.21.15.87:0/760446907 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3406cc30 con 0x7f6f340cf930 2024-04-25T10:22:53.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.925+0000 7f6f39c00640 1 --2- 172.21.15.87:0/760446907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cf930 0x7f6f340cfdc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6f340d0800 tx=0x7f6f2c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd18095295d21ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f33400640 1 -- 172.21.15.87:0/760446907 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0b0070 con 0x7f6f340cf930 2024-04-25T10:22:53.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f33400640 1 -- 172.21.15.87:0/760446907 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c0ab600 con 0x7f6f340cf930 2024-04-25T10:22:53.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f33400640 1 -- 172.21.15.87:0/760446907 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0ab9d0 con 0x7f6f340cf930 2024-04-25T10:22:53.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- 172.21.15.87:0/760446907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cf930 msgr2=0x7f6f340cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/760446907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cf930 0x7f6f340cfdc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6f340d0800 tx=0x7f6f2c01c490 comp rx=0 tx=0).stop 2024-04-25T10:22:53.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- 172.21.15.87:0/760446907 shutdown_connections 2024-04-25T10:22:53.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/760446907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cf930 0x7f6f340cfdc0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/760446907 >> v2:172.21.15.169:3300/0 conn(0x7f6f3415bc40 0x7f6f3406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- 172.21.15.87:0/760446907 >> 172.21.15.87:0/760446907 conn(0x7f6f34066b70 msgr2=0x7f6f34066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:53.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- 172.21.15.87:0/760446907 shutdown_connections 2024-04-25T10:22:53.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- 172.21.15.87:0/760446907 wait complete. 2024-04-25T10:22:53.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 Processor -- start 2024-04-25T10:22:53.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- start start 2024-04-25T10:22:53.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cda10 0x7f6f34151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f340cf930 0x7f6f3415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f3415bc40 0x7f6f34151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340b3850 con 0x7f6f340cda10 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340b36d0 con 0x7f6f340cf930 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f3bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340b39d0 con 0x7f6f3415bc40 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f39200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cda10 0x7f6f34151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f39200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cda10 0x7f6f34151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37796/0 (socket says 172.21.15.87:37796) 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f39200640 1 -- 172.21.15.87:0/3354360254 learned_addr learned my addr 172.21.15.87:0/3354360254 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f33e00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f340cf930 0x7f6f3415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.929+0000 7f6f39c00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f3415bc40 0x7f6f34151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f39200640 1 -- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f3415bc40 msgr2=0x7f6f34151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f39200640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f3415bc40 0x7f6f34151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f39200640 1 -- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f340cf930 msgr2=0x7f6f3415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f39200640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f340cf930 0x7f6f3415e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f39200640 1 -- 172.21.15.87:0/3354360254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f341f4c70 con 0x7f6f340cda10 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f39200640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cda10 0x7f6f34151100 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6f24007e60 tx=0x7f6f2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f30c00640 1 -- 172.21.15.87:0/3354360254 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24014620 con 0x7f6f340cda10 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f30c00640 1 -- 172.21.15.87:0/3354360254 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f24019070 con 0x7f6f340cda10 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f341f4f50 con 0x7f6f340cda10 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f30c00640 1 -- 172.21.15.87:0/3354360254 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24014a20 con 0x7f6f340cda10 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f341f5510 con 0x7f6f340cda10 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3404ef30 con 0x7f6f340cda10 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f30c00640 1 -- 172.21.15.87:0/3354360254 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f24014bc0 con 0x7f6f340cda10 2024-04-25T10:22:53.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f30c00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6f14066f60 0x7f6f14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:53.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f30c00640 1 -- 172.21.15.87:0/3354360254 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f24085580 con 0x7f6f340cda10 2024-04-25T10:22:53.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.933+0000 7f6f33e00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6f14066f60 0x7f6f14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:53.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.937+0000 7f6f33e00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6f14066f60 0x7f6f14069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6f3415af30 tx=0x7f6f20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:53.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:53.937+0000 7f6f30c00640 1 -- 172.21.15.87:0/3354360254 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f24056840 con 0x7f6f340cda10 2024-04-25T10:22:54.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.093+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6f340ae2d0 con 0x7f6f340cda10 2024-04-25T10:22:54.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.093+0000 7f6f30c00640 1 -- 172.21.15.87:0/3354360254 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6f2405c020 con 0x7f6f340cda10 2024-04-25T10:22:54.098 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:54.098 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:22:54.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:22:54.099 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:22:54.099 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:22:54.099 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:22:54.099 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:22:54.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6f14066f60 msgr2=0x7f6f14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:54.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6f14066f60 0x7f6f14069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6f3415af30 tx=0x7f6f20009040 comp rx=0 tx=0).stop 2024-04-25T10:22:54.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cda10 msgr2=0x7f6f34151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cda10 0x7f6f34151100 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6f24007e60 tx=0x7f6f2400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 shutdown_connections 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6f14066f60 0x7f6f14069420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f3415bc40 0x7f6f34151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f340cf930 0x7f6f3415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 --2- 172.21.15.87:0/3354360254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f340cda10 0x7f6f34151100 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 >> 172.21.15.87:0/3354360254 conn(0x7f6f34066b70 msgr2=0x7f6f340b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 shutdown_connections 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.097+0000 7f6f3bc00640 1 -- 172.21.15.87:0/3354360254 wait complete. 2024-04-25T10:22:54.118 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:22:54.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:22:54.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.301+0000 7f0afb000640 1 Processor -- start 2024-04-25T10:22:54.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.301+0000 7f0afb000640 1 -- start start 2024-04-25T10:22:54.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0afb000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0af406bf80 0x7f0af40d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:54.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0afb000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0af40d0c70 0x7f0af415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:54.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0afb000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af406aee0 con 0x7f0af41619a0 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0afb000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af406ac90 con 0x7f0af40d0c70 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0afb000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af406b1c0 con 0x7f0af406bf80 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0af40d0c70 0x7f0af415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0af40d0c70 0x7f0af415e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37802/0 (socket says 172.21.15.87:37802) 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0af41619a0 0x7f0af4161d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37272/0 (socket says 172.21.15.87:37272) 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af9000640 1 -- 172.21.15.87:0/3383932541 learned_addr learned my addr 172.21.15.87:0/3383932541 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0aeb400640 1 --2- 172.21.15.87:0/3383932541 >> v2:172.21.15.169:3300/0 conn(0x7f0af406bf80 0x7f0af40d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af9000640 1 -- 172.21.15.87:0/3383932541 >> v2:172.21.15.169:3300/0 conn(0x7f0af406bf80 msgr2=0x7f0af40d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af9000640 1 --2- 172.21.15.87:0/3383932541 >> v2:172.21.15.169:3300/0 conn(0x7f0af406bf80 0x7f0af40d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af9000640 1 -- 172.21.15.87:0/3383932541 >> v1:172.21.15.87:6789/0 conn(0x7f0af41619a0 legacy=0x7f0af4161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af9000640 1 -- 172.21.15.87:0/3383932541 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af406b4a0 con 0x7f0af40d0c70 2024-04-25T10:22:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0aeb400640 1 --2- 172.21.15.87:0/3383932541 >> v2:172.21.15.169:3300/0 conn(0x7f0af406bf80 0x7f0af40d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.305+0000 7f0af3400640 1 -- 172.21.15.87:0/3383932541 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663532536 0 0) 0x7f0af406aee0 con 0x7f0af41619a0 2024-04-25T10:22:54.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0af9000640 1 --2- 172.21.15.87:0/3383932541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af40d0c70 0x7f0af415e1c0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f0af4071a10 tx=0x7f0aec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e874b2d669433615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:54.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0af3400640 1 -- 172.21.15.87:0/3383932541 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0a5880 con 0x7f0af40d0c70 2024-04-25T10:22:54.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0af3400640 1 -- 172.21.15.87:0/3383932541 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec0a5a20 con 0x7f0af40d0c70 2024-04-25T10:22:54.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0af3400640 1 -- 172.21.15.87:0/3383932541 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0a5df0 con 0x7f0af40d0c70 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- 172.21.15.87:0/3383932541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af40d0c70 msgr2=0x7f0af415e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 --2- 172.21.15.87:0/3383932541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af40d0c70 0x7f0af415e1c0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f0af4071a10 tx=0x7f0aec01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- 172.21.15.87:0/3383932541 shutdown_connections 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 --2- 172.21.15.87:0/3383932541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af40d0c70 0x7f0af415e1c0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 --2- 172.21.15.87:0/3383932541 >> v2:172.21.15.169:3300/0 conn(0x7f0af406bf80 0x7f0af40d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- 172.21.15.87:0/3383932541 >> 172.21.15.87:0/3383932541 conn(0x7f0af40b34a0 msgr2=0x7f0af40b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- 172.21.15.87:0/3383932541 shutdown_connections 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- 172.21.15.87:0/3383932541 wait complete. 2024-04-25T10:22:54.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 Processor -- start 2024-04-25T10:22:54.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- start start 2024-04-25T10:22:54.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af406bf80 0x7f0af415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af40d0c70 0x7f0af4168370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af41619a0 0x7f0af4152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415f2a0 con 0x7f0af41619a0 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415f120 con 0x7f0af406bf80 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0afb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af416ace0 con 0x7f0af40d0c70 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0af9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af41619a0 0x7f0af4152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0af9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af41619a0 0x7f0af4152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37812/0 (socket says 172.21.15.87:37812) 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.309+0000 7f0af9000640 1 -- 172.21.15.87:0/1801475364 learned_addr learned my addr 172.21.15.87:0/1801475364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0aeb400640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af40d0c70 0x7f0af4168370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af3e00640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af406bf80 0x7f0af415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af9000640 1 -- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af40d0c70 msgr2=0x7f0af4168370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af9000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af40d0c70 0x7f0af4168370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af9000640 1 -- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af406bf80 msgr2=0x7f0af415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af9000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af406bf80 0x7f0af415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af9000640 1 -- 172.21.15.87:0/1801475364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af415d9f0 con 0x7f0af41619a0 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af3e00640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af406bf80 0x7f0af415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af9000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af41619a0 0x7f0af4152ef0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0195d0 tx=0x7f0aec019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af0c00640 1 -- 172.21.15.87:0/1801475364 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0b8670 con 0x7f0af41619a0 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af0c00640 1 -- 172.21.15.87:0/1801475364 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec0be070 con 0x7f0af41619a0 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af41688b0 con 0x7f0af41619a0 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af0c00640 1 -- 172.21.15.87:0/1801475364 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0b8a70 con 0x7f0af41619a0 2024-04-25T10:22:54.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af415b860 con 0x7f0af41619a0 2024-04-25T10:22:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af404ef30 con 0x7f0af41619a0 2024-04-25T10:22:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af0c00640 1 -- 172.21.15.87:0/1801475364 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0aec0b8c10 con 0x7f0af41619a0 2024-04-25T10:22:54.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af0c00640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ad0066f60 0x7f0ad0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:54.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af0c00640 1 -- 172.21.15.87:0/1801475364 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0aec11ffe0 con 0x7f0af41619a0 2024-04-25T10:22:54.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.313+0000 7f0af3e00640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ad0066f60 0x7f0ad0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:54.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.317+0000 7f0af3e00640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ad0066f60 0x7f0ad0069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4000f70 tx=0x7f0ae4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:54.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.317+0000 7f0af0c00640 1 -- 172.21.15.87:0/1801475364 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0aec0165e0 con 0x7f0af41619a0 2024-04-25T10:22:54.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:54.473+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0af406ddd0 con 0x7f0af41619a0 2024-04-25T10:22:55.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.373+0000 7f0af0c00640 1 -- 172.21.15.87:0/1801475364 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0aec0f12e0 con 0x7f0af41619a0 2024-04-25T10:22:55.379 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ad0066f60 msgr2=0x7f0ad0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ad0066f60 0x7f0ad0069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4000f70 tx=0x7f0ae4009040 comp rx=0 tx=0).stop 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af41619a0 msgr2=0x7f0af4152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af41619a0 0x7f0af4152ef0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0195d0 tx=0x7f0aec019760 comp rx=0 tx=0).stop 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 shutdown_connections 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0ad0066f60 0x7f0ad0069420 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0af41619a0 0x7f0af4152ef0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af40d0c70 0x7f0af4168370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 --2- 172.21.15.87:0/1801475364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af406bf80 0x7f0af415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 >> 172.21.15.87:0/1801475364 conn(0x7f0af40b34a0 msgr2=0x7f0af406e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 shutdown_connections 2024-04-25T10:22:55.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.377+0000 7f0afb000640 1 -- 172.21.15.87:0/1801475364 wait complete. 2024-04-25T10:22:55.410 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:22:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 Processor -- start 2024-04-25T10:22:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 -- start start 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcf18161370 0x7fcf1806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcf1806ac90 0x7fcf1806b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf180b0640 0x7fcf180cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf180ce020 con 0x7fcf1806ac90 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf180ceba0 con 0x7fcf18161370 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180ce270 con 0x7fcf180b0640 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.553+0000 7fcf1ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcf1806ac90 0x7fcf1806b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcf1806ac90 0x7fcf1806b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37820/0 (socket says 172.21.15.87:37820) 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 -- 172.21.15.87:0/974473299 learned_addr learned my addr 172.21.15.87:0/974473299 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1e200640 1 --2- 172.21.15.87:0/974473299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf180b0640 0x7fcf180cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1d800640 1 --2- 172.21.15.87:0/974473299 >> v2:172.21.15.169:3300/0 conn(0x7fcf18161370 0x7fcf1806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 -- 172.21.15.87:0/974473299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf180b0640 msgr2=0x7fcf180cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 --2- 172.21.15.87:0/974473299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf180b0640 0x7fcf180cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 -- 172.21.15.87:0/974473299 >> v2:172.21.15.169:3300/0 conn(0x7fcf18161370 msgr2=0x7fcf1806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 --2- 172.21.15.87:0/974473299 >> v2:172.21.15.169:3300/0 conn(0x7fcf18161370 0x7fcf1806b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 -- 172.21.15.87:0/974473299 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf180ce550 con 0x7fcf1806ac90 2024-04-25T10:22:55.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf1ce00640 1 --2- 172.21.15.87:0/974473299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf1806ac90 0x7fcf1806b790 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fcf18071d30 tx=0x7fcf0401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaa8dd4afe37b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:55.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf17e00640 1 -- 172.21.15.87:0/974473299 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf040b2070 con 0x7fcf1806ac90 2024-04-25T10:22:55.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf17e00640 1 -- 172.21.15.87:0/974473299 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf040a9070 con 0x7fcf1806ac90 2024-04-25T10:22:55.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.557+0000 7fcf17e00640 1 -- 172.21.15.87:0/974473299 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf040ad880 con 0x7fcf1806ac90 2024-04-25T10:22:55.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- 172.21.15.87:0/974473299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf1806ac90 msgr2=0x7fcf1806b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/974473299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf1806ac90 0x7fcf1806b790 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fcf18071d30 tx=0x7fcf0401c490 comp rx=0 tx=0).stop 2024-04-25T10:22:55.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- 172.21.15.87:0/974473299 shutdown_connections 2024-04-25T10:22:55.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/974473299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf1806ac90 0x7fcf1806b790 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/974473299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf180b0640 0x7fcf180cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/974473299 >> v2:172.21.15.169:3300/0 conn(0x7fcf18161370 0x7fcf1806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- 172.21.15.87:0/974473299 >> 172.21.15.87:0/974473299 conn(0x7fcf1809a9d0 msgr2=0x7fcf180d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:55.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- 172.21.15.87:0/974473299 shutdown_connections 2024-04-25T10:22:55.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- 172.21.15.87:0/974473299 wait complete. 2024-04-25T10:22:55.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 Processor -- start 2024-04-25T10:22:55.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- start start 2024-04-25T10:22:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf1806ac90 0x7fcf1814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf180b0640 0x7fcf180caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf18161370 0x7fcf180c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180cc2a0 con 0x7fcf180b0640 2024-04-25T10:22:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180cc120 con 0x7fcf18161370 2024-04-25T10:22:55.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180cc420 con 0x7fcf1806ac90 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf180b0640 0x7fcf180caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf180b0640 0x7fcf180caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37822/0 (socket says 172.21.15.87:37822) 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 -- 172.21.15.87:0/3184455058 learned_addr learned my addr 172.21.15.87:0/3184455058 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1d800640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf1806ac90 0x7fcf1814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1e200640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf18161370 0x7fcf180c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 -- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf1806ac90 msgr2=0x7fcf1814d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf1806ac90 0x7fcf1814d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 -- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf18161370 msgr2=0x7fcf180c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf18161370 0x7fcf180c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.561+0000 7fcf1ce00640 1 -- 172.21.15.87:0/3184455058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1814dd40 con 0x7fcf180b0640 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf1ce00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf180b0640 0x7fcf180caca0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fcf0401ab40 tx=0x7fcf04019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf15600640 1 -- 172.21.15.87:0/3184455058 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf040bb070 con 0x7fcf180b0640 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf15600640 1 -- 172.21.15.87:0/3184455058 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf040adc70 con 0x7fcf180b0640 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf15600640 1 -- 172.21.15.87:0/3184455058 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf040b2040 con 0x7fcf180b0640 2024-04-25T10:22:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1814e020 con 0x7fcf180b0640 2024-04-25T10:22:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1814e5e0 con 0x7fcf180b0640 2024-04-25T10:22:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1814e910 con 0x7fcf180b0640 2024-04-25T10:22:55.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf15600640 1 -- 172.21.15.87:0/3184455058 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf040ab070 con 0x7fcf180b0640 2024-04-25T10:22:55.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf15600640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcef8066f60 0x7fcef8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:55.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf15600640 1 -- 172.21.15.87:0/3184455058 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcf0411dc10 con 0x7fcf180b0640 2024-04-25T10:22:55.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.565+0000 7fcf1d800640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcef8066f60 0x7fcef8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:55.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.569+0000 7fcf1d800640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcef8066f60 0x7fcef8069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009a50 tx=0x7fcf0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:55.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.573+0000 7fcf15600640 1 -- 172.21.15.87:0/3184455058 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf04016110 con 0x7fcf180b0640 2024-04-25T10:22:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:55.725+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcf180c8480 con 0x7fcf180b0640 2024-04-25T10:22:56.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.377+0000 7fcf15600640 1 -- 172.21.15.87:0/3184455058 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcf040ade20 con 0x7fcf180b0640 2024-04-25T10:22:56.382 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:22:56.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcef8066f60 msgr2=0x7fcef8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:56.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcef8066f60 0x7fcef8069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009a50 tx=0x7fcf0c008040 comp rx=0 tx=0).stop 2024-04-25T10:22:56.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf180b0640 msgr2=0x7fcf180caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:56.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf180b0640 0x7fcf180caca0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fcf0401ab40 tx=0x7fcf04019230 comp rx=0 tx=0).stop 2024-04-25T10:22:56.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 shutdown_connections 2024-04-25T10:22:56.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcef8066f60 0x7fcef8069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf18161370 0x7fcf180c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf180b0640 0x7fcf180caca0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 --2- 172.21.15.87:0/3184455058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf1806ac90 0x7fcf1814d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 >> 172.21.15.87:0/3184455058 conn(0x7fcf1809a9d0 msgr2=0x7fcf180b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:56.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 shutdown_connections 2024-04-25T10:22:56.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.381+0000 7fcf1ec00640 1 -- 172.21.15.87:0/3184455058 wait complete. 2024-04-25T10:22:56.421 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:22:56.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9c4e00640 1 Processor -- start 2024-04-25T10:22:56.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9c4e00640 1 -- start start 2024-04-25T10:22:56.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9c4e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff9c00aef80 0x7ff9c015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:56.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9c4e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c015b080 con 0x7ff9c0070350 2024-04-25T10:22:56.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9c4e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c015adb0 con 0x7ff9c00b31c0 2024-04-25T10:22:56.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9c4e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c015eb60 con 0x7ff9c00aef80 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bd400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff9c0070350 0x7ff9c015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:37288/0 (socket says 172.21.15.87:37288) 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bd400640 1 -- 172.21.15.87:0/2370533651 learned_addr learned my addr 172.21.15.87:0/2370533651 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bde00640 1 --2- 172.21.15.87:0/2370533651 >> v2:172.21.15.169:3300/0 conn(0x7ff9c00aef80 0x7ff9c015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bca00640 1 -- 172.21.15.87:0/2370533651 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713529288 0 0) 0x7ff9c015b080 con 0x7ff9c0070350 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bca00640 1 -- 172.21.15.87:0/2370533651 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a4003530 con 0x7ff9c0070350 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bca00640 1 -- 172.21.15.87:0/2370533651 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241891692 0 0) 0x7ff9c015adb0 con 0x7ff9c00b31c0 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bde00640 1 -- 172.21.15.87:0/2370533651 >> v1:172.21.15.169:6789/0 conn(0x7ff9c00b31c0 legacy=0x7ff9c00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bde00640 1 -- 172.21.15.87:0/2370533651 >> v1:172.21.15.87:6789/0 conn(0x7ff9c0070350 legacy=0x7ff9c015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:56.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.557+0000 7ff9bde00640 1 -- 172.21.15.87:0/2370533651 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c015ee40 con 0x7ff9c00aef80 2024-04-25T10:22:56.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9bde00640 1 --2- 172.21.15.87:0/2370533651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00aef80 0x7ff9c015e480 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00097c0 tx=0x7ff9b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=480f0ad406e4edcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:56.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9bca00640 1 -- 172.21.15.87:0/2370533651 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00ab680 con 0x7ff9c00aef80 2024-04-25T10:22:56.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9bca00640 1 -- 172.21.15.87:0/2370533651 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b00b0070 con 0x7ff9c00aef80 2024-04-25T10:22:56.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9bca00640 1 -- 172.21.15.87:0/2370533651 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00aba40 con 0x7ff9c00aef80 2024-04-25T10:22:56.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9c4e00640 1 -- 172.21.15.87:0/2370533651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00aef80 msgr2=0x7ff9c015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:56.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/2370533651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00aef80 0x7ff9c015e480 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00097c0 tx=0x7ff9b001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9c4e00640 1 -- 172.21.15.87:0/2370533651 shutdown_connections 2024-04-25T10:22:56.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/2370533651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00aef80 0x7ff9c015e480 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9c4e00640 1 -- 172.21.15.87:0/2370533651 >> 172.21.15.87:0/2370533651 conn(0x7ff9c00669a0 msgr2=0x7ff9c0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:56.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9c4e00640 1 -- 172.21.15.87:0/2370533651 shutdown_connections 2024-04-25T10:22:56.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.561+0000 7ff9c4e00640 1 -- 172.21.15.87:0/2370533651 wait complete. 2024-04-25T10:22:56.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 Processor -- start 2024-04-25T10:22:56.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 -- start start 2024-04-25T10:22:56.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9c0070350 0x7ff9c014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c00aef80 0x7ff9c014ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00b31c0 0x7ff9c014f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00cfef0 con 0x7ff9c00aef80 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00cfd70 con 0x7ff9c00b31c0 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00b35d0 con 0x7ff9c0070350 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c00aef80 0x7ff9c014ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c00aef80 0x7ff9c014ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37830/0 (socket says 172.21.15.87:37830) 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 -- 172.21.15.87:0/3971976915 learned_addr learned my addr 172.21.15.87:0/3971976915 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bde00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9c0070350 0x7ff9c014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9be800640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00b31c0 0x7ff9c014f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 -- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9c0070350 msgr2=0x7ff9c014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9c0070350 0x7ff9c014e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 -- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00b31c0 msgr2=0x7ff9c014f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00b31c0 0x7ff9c014f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 -- 172.21.15.87:0/3971976915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c014e940 con 0x7ff9c00aef80 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bde00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9c0070350 0x7ff9c014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9bd400640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c00aef80 0x7ff9c014ebc0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c450 tx=0x7ff9ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9b6000640 1 -- 172.21.15.87:0/3971976915 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac017070 con 0x7ff9c00aef80 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9b6000640 1 -- 172.21.15.87:0/3971976915 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac005cf0 con 0x7ff9c00aef80 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9b6000640 1 -- 172.21.15.87:0/3971976915 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac007290 con 0x7ff9c00aef80 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c0157a50 con 0x7ff9c00aef80 2024-04-25T10:22:56.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c0158010 con 0x7ff9c00aef80 2024-04-25T10:22:56.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.565+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c004ef30 con 0x7ff9c00aef80 2024-04-25T10:22:56.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.569+0000 7ff9b6000640 1 -- 172.21.15.87:0/3971976915 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9ac01a070 con 0x7ff9c00aef80 2024-04-25T10:22:56.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.569+0000 7ff9b6000640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9a0066f60 0x7ff9a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:56.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.569+0000 7ff9b6000640 1 -- 172.21.15.87:0/3971976915 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff9ac0843f0 con 0x7ff9c00aef80 2024-04-25T10:22:56.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.569+0000 7ff9bde00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9a0066f60 0x7ff9a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:56.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.569+0000 7ff9bde00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9a0066f60 0x7ff9a0069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00af000 tx=0x7ff9b00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:56.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.573+0000 7ff9b6000640 1 -- 172.21.15.87:0/3971976915 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9ac055450 con 0x7ff9c00aef80 2024-04-25T10:22:56.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:56.757+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff9c00cc0f0 con 0x7ff9c00aef80 2024-04-25T10:22:57.411 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 20 and data pool 21 2024-04-25T10:22:57.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.401+0000 7ff9b6000640 1 -- 172.21.15.87:0/3971976915 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v97) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff9ac054d50 con 0x7ff9c00aef80 2024-04-25T10:22:57.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9a0066f60 msgr2=0x7ff9a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9a0066f60 0x7ff9a0069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00af000 tx=0x7ff9b00a7000 comp rx=0 tx=0).stop 2024-04-25T10:22:57.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c00aef80 msgr2=0x7ff9c014ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c00aef80 0x7ff9c014ebc0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c450 tx=0x7ff9ac00c910 comp rx=0 tx=0).stop 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 shutdown_connections 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9a0066f60 0x7ff9a0069420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9c00b31c0 0x7ff9c014f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c00aef80 0x7ff9c014ebc0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 --2- 172.21.15.87:0/3971976915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9c0070350 0x7ff9c014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 >> 172.21.15.87:0/3971976915 conn(0x7ff9c00669a0 msgr2=0x7ff9c00cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 shutdown_connections 2024-04-25T10:22:57.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.421+0000 7ff9c4e00640 1 -- 172.21.15.87:0/3971976915 wait complete. 2024-04-25T10:22:57.458 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:57.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c9600640 1 Processor -- start 2024-04-25T10:22:57.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c9600640 1 -- start start 2024-04-25T10:22:57.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f76c40b0f70 0x7f76c40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c9600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76c40cfd90 0x7f76c40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c9600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c415f3b0 con 0x7f76c40cd110 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c9600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c415ef20 con 0x7f76c40cfd90 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c9600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c40ca810 con 0x7f76c40b0f70 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76c40cfd90 0x7f76c40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76c40cfd90 0x7f76c40ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52034/0 (socket says 172.21.15.87:52034) 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f76c40cd110 0x7f76c40cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59734/0 (socket says 172.21.15.87:59734) 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c2000640 1 -- 172.21.15.87:0/2393926938 learned_addr learned my addr 172.21.15.87:0/2393926938 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.589+0000 7f76c2a00640 1 --2- 172.21.15.87:0/2393926938 >> v2:172.21.15.169:3300/0 conn(0x7f76c40b0f70 0x7f76c40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c2000640 1 -- 172.21.15.87:0/2393926938 >> v2:172.21.15.169:3300/0 conn(0x7f76c40b0f70 msgr2=0x7f76c40b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c2000640 1 --2- 172.21.15.87:0/2393926938 >> v2:172.21.15.169:3300/0 conn(0x7f76c40b0f70 0x7f76c40b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c2000640 1 -- 172.21.15.87:0/2393926938 >> v1:172.21.15.87:6789/0 conn(0x7f76c40cd110 legacy=0x7f76c40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c2000640 1 -- 172.21.15.87:0/2393926938 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40b1890 con 0x7f76c40cfd90 2024-04-25T10:22:57.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c2a00640 1 --2- 172.21.15.87:0/2393926938 >> v2:172.21.15.169:3300/0 conn(0x7f76c40b0f70 0x7f76c40b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:57.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c2000640 1 --2- 172.21.15.87:0/2393926938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40cfd90 0x7f76c40ccbd0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f76c4071230 tx=0x7f76b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994b94bdfcf8a1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c1600640 1 -- 172.21.15.87:0/2393926938 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00b0070 con 0x7f76c40cfd90 2024-04-25T10:22:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c1600640 1 -- 172.21.15.87:0/2393926938 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00ab600 con 0x7f76c40cfd90 2024-04-25T10:22:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c1600640 1 -- 172.21.15.87:0/2393926938 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00ab9d0 con 0x7f76c40cfd90 2024-04-25T10:22:57.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c9600640 1 -- 172.21.15.87:0/2393926938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40cfd90 msgr2=0x7f76c40ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c9600640 1 --2- 172.21.15.87:0/2393926938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40cfd90 0x7f76c40ccbd0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f76c4071230 tx=0x7f76b001c520 comp rx=0 tx=0).stop 2024-04-25T10:22:57.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c9600640 1 -- 172.21.15.87:0/2393926938 shutdown_connections 2024-04-25T10:22:57.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c9600640 1 --2- 172.21.15.87:0/2393926938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40cfd90 0x7f76c40ccbd0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c9600640 1 --2- 172.21.15.87:0/2393926938 >> v2:172.21.15.169:3300/0 conn(0x7f76c40b0f70 0x7f76c40b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c9600640 1 -- 172.21.15.87:0/2393926938 >> 172.21.15.87:0/2393926938 conn(0x7f76c40b1d50 msgr2=0x7f76c40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:57.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.593+0000 7f76c9600640 1 -- 172.21.15.87:0/2393926938 shutdown_connections 2024-04-25T10:22:57.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 -- 172.21.15.87:0/2393926938 wait complete. 2024-04-25T10:22:57.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 Processor -- start 2024-04-25T10:22:57.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 -- start start 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40b0f70 0x7f76c4155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40cd110 0x7f76c4157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76c40cfd90 0x7f76c4099360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c41619d0 con 0x7f76c40b0f70 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40cd520 con 0x7f76c40cd110 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4161b50 con 0x7f76c40cfd90 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40b0f70 0x7f76c4155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40b0f70 0x7f76c4155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52050/0 (socket says 172.21.15.87:52050) 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 -- 172.21.15.87:0/1547014635 learned_addr learned my addr 172.21.15.87:0/1547014635 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2000640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40cd110 0x7f76c4157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c3400640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76c40cfd90 0x7f76c4099360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 -- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76c40cfd90 msgr2=0x7f76c4099360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76c40cfd90 0x7f76c4099360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 -- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40cd110 msgr2=0x7f76c4157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40cd110 0x7f76c4157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 -- 172.21.15.87:0/1547014635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40af3d0 con 0x7f76c40b0f70 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c2a00640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40b0f70 0x7f76c4155020 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f76b400cf90 tx=0x7f76b400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76baa00640 1 -- 172.21.15.87:0/1547014635 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b401a070 con 0x7f76c40b0f70 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76baa00640 1 -- 172.21.15.87:0/1547014635 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b400d840 con 0x7f76c40b0f70 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76baa00640 1 -- 172.21.15.87:0/1547014635 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400dc10 con 0x7f76c40b0f70 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.597+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c40af6b0 con 0x7f76c40b0f70 2024-04-25T10:22:57.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.601+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c40afc70 con 0x7f76c40b0f70 2024-04-25T10:22:57.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.601+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c404ef30 con 0x7f76c40b0f70 2024-04-25T10:22:57.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.601+0000 7f76baa00640 1 -- 172.21.15.87:0/1547014635 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76b401d070 con 0x7f76c40b0f70 2024-04-25T10:22:57.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.601+0000 7f76baa00640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76a4066f60 0x7f76a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.601+0000 7f76baa00640 1 -- 172.21.15.87:0/1547014635 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76b4085820 con 0x7f76c40b0f70 2024-04-25T10:22:57.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.601+0000 7f76c2000640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76a4066f60 0x7f76a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.601+0000 7f76c2000640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76a4066f60 0x7f76a4069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f76b00ad000 tx=0x7f76b0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.605+0000 7f76baa00640 1 -- 172.21.15.87:0/1547014635 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76b40561d0 con 0x7f76c40b0f70 2024-04-25T10:22:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.761+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76c40cda20 con 0x7f76c40b0f70 2024-04-25T10:22:57.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.761+0000 7f76baa00640 1 -- 172.21.15.87:0/1547014635 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f76b405a270 con 0x7f76c40b0f70 2024-04-25T10:22:57.765 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:57.765 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":137,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:56.765745+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:54.523910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:55.777113+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:57.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76a4066f60 msgr2=0x7f76a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76a4066f60 0x7f76a4069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f76b00ad000 tx=0x7f76b0008040 comp rx=0 tx=0).stop 2024-04-25T10:22:57.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40b0f70 msgr2=0x7f76c4155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40b0f70 0x7f76c4155020 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f76b400cf90 tx=0x7f76b400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 shutdown_connections 2024-04-25T10:22:57.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76a4066f60 0x7f76a4069420 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76c40cfd90 0x7f76c4099360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40cd110 0x7f76c4157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 --2- 172.21.15.87:0/1547014635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c40b0f70 0x7f76c4155020 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 >> 172.21.15.87:0/1547014635 conn(0x7f76c40b1d50 msgr2=0x7f76c4160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:57.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 shutdown_connections 2024-04-25T10:22:57.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.765+0000 7f76c9600640 1 -- 172.21.15.87:0/1547014635 wait complete. 2024-04-25T10:22:57.787 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:57.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 Processor -- start 2024-04-25T10:22:57.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 -- start start 2024-04-25T10:22:57.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16a815bc40 0x7f16a806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16a80cda10 0x7f16a80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a80b2770 con 0x7f16a80cda10 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a806c670 con 0x7f16a815bc40 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16ae800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a806c950 con 0x7f16a80cf930 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f169ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16a80cda10 0x7f16a80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f169ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16a80cda10 0x7f16a80b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52064/0 (socket says 172.21.15.87:52064) 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f169ea00640 1 -- 172.21.15.87:0/2520404191 learned_addr learned my addr 172.21.15.87:0/2520404191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16a7e00640 1 --2- 172.21.15.87:0/2520404191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.921+0000 7f16a7400640 1 --2- 172.21.15.87:0/2520404191 >> v2:172.21.15.169:3300/0 conn(0x7f16a815bc40 0x7f16a806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f169ea00640 1 -- 172.21.15.87:0/2520404191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 msgr2=0x7f16a80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f169ea00640 1 --2- 172.21.15.87:0/2520404191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f169ea00640 1 -- 172.21.15.87:0/2520404191 >> v2:172.21.15.169:3300/0 conn(0x7f16a815bc40 msgr2=0x7f16a806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:57.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f169ea00640 1 --2- 172.21.15.87:0/2520404191 >> v2:172.21.15.169:3300/0 conn(0x7f16a815bc40 0x7f16a806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f169ea00640 1 -- 172.21.15.87:0/2520404191 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a806cc30 con 0x7f16a80cda10 2024-04-25T10:22:57.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f169ea00640 1 --2- 172.21.15.87:0/2520404191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a80b4610 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f16a80d0800 tx=0x7f169001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8b59926321b315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16a6a00640 1 -- 172.21.15.87:0/2520404191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900b0070 con 0x7f16a80cda10 2024-04-25T10:22:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16a6a00640 1 -- 172.21.15.87:0/2520404191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16900ab600 con 0x7f16a80cda10 2024-04-25T10:22:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16a6a00640 1 -- 172.21.15.87:0/2520404191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900ab9d0 con 0x7f16a80cda10 2024-04-25T10:22:57.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 -- 172.21.15.87:0/2520404191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 msgr2=0x7f16a80b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 --2- 172.21.15.87:0/2520404191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a80b4610 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f16a80d0800 tx=0x7f169001c520 comp rx=0 tx=0).stop 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 -- 172.21.15.87:0/2520404191 shutdown_connections 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 --2- 172.21.15.87:0/2520404191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a80b4610 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 --2- 172.21.15.87:0/2520404191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 --2- 172.21.15.87:0/2520404191 >> v2:172.21.15.169:3300/0 conn(0x7f16a815bc40 0x7f16a806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 -- 172.21.15.87:0/2520404191 >> 172.21.15.87:0/2520404191 conn(0x7f16a8066b70 msgr2=0x7f16a8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 -- 172.21.15.87:0/2520404191 shutdown_connections 2024-04-25T10:22:57.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.925+0000 7f16ae800640 1 -- 172.21.15.87:0/2520404191 wait complete. 2024-04-25T10:22:57.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 Processor -- start 2024-04-25T10:22:57.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 -- start start 2024-04-25T10:22:57.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a815bc40 0x7f16a815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8160270 con 0x7f16a80cda10 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a81600f0 con 0x7f16a815bc40 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a81603f0 con 0x7f16a80cf930 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a815cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52068/0 (socket says 172.21.15.87:52068) 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 -- 172.21.15.87:0/3249206790 learned_addr learned my addr 172.21.15.87:0/3249206790 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f169ea00640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7e00640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a815bc40 0x7f16a815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 -- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 msgr2=0x7f16a815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 -- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a815bc40 msgr2=0x7f16a815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a815bc40 0x7f16a815dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:57.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 -- 172.21.15.87:0/3249206790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a815d460 con 0x7f16a80cda10 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f169ea00640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16a7400640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a815cba0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1698007e40 tx=0x7f169800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f169fe00640 1 -- 172.21.15.87:0/3249206790 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698014620 con 0x7f16a80cda10 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f169fe00640 1 -- 172.21.15.87:0/3249206790 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1698019070 con 0x7f16a80cda10 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f169fe00640 1 -- 172.21.15.87:0/3249206790 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698014a00 con 0x7f16a80cda10 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a815d180 con 0x7f16a80cda10 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.929+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a8156040 con 0x7f16a80cda10 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.933+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a804ef30 con 0x7f16a80cda10 2024-04-25T10:22:57.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.933+0000 7f169fe00640 1 -- 172.21.15.87:0/3249206790 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1698014ba0 con 0x7f16a80cda10 2024-04-25T10:22:57.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.933+0000 7f169fe00640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1684066f90 0x7f1684069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:57.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.933+0000 7f169fe00640 1 -- 172.21.15.87:0/3249206790 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16980858a0 con 0x7f16a80cda10 2024-04-25T10:22:57.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.933+0000 7f169ea00640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1684066f90 0x7f1684069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:57.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.933+0000 7f169ea00640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1684066f90 0x7f1684069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f16900ad000 tx=0x7f1690008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:57.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:57.937+0000 7f169fe00640 1 -- 172.21.15.87:0/3249206790 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16980568a0 con 0x7f16a80cda10 2024-04-25T10:22:58.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.093+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16a815a960 con 0x7f16a80cda10 2024-04-25T10:22:58.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.093+0000 7f169fe00640 1 -- 172.21.15.87:0/3249206790 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f16980561a0 con 0x7f16a80cda10 2024-04-25T10:22:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:22:58.098 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":137,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:56.765745+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:54.523910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:55.777113+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1684066f90 msgr2=0x7f1684069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1684066f90 0x7f1684069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f16900ad000 tx=0x7f1690008040 comp rx=0 tx=0).stop 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 msgr2=0x7f16a815cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a815cba0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1698007e40 tx=0x7f169800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 shutdown_connections 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1684066f90 0x7f1684069450 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a815bc40 0x7f16a815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a80cf930 0x7f16a815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 --2- 172.21.15.87:0/3249206790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80cda10 0x7f16a815cba0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 >> 172.21.15.87:0/3249206790 conn(0x7f16a8066b70 msgr2=0x7f16a80cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 shutdown_connections 2024-04-25T10:22:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.097+0000 7f16ae800640 1 -- 172.21.15.87:0/3249206790 wait complete. 2024-04-25T10:22:58.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:22:58.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01c600640 1 Processor -- start 2024-04-25T10:22:58.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01c600640 1 -- start start 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01c600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0140aef80 0x7fe01415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0140b31c0 0x7fe0140b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01c600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01415b080 con 0x7fe014070350 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01c600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01415adb0 con 0x7fe0140aef80 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01415eb60 con 0x7fe0140b31c0 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe019c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0140aef80 0x7fe01415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe019200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe014070350 0x7fe01415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59744/0 (socket says 172.21.15.87:59744) 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe019200640 1 -- 172.21.15.87:0/1597189612 learned_addr learned my addr 172.21.15.87:0/1597189612 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.253+0000 7fe01a600640 1 --2- 172.21.15.87:0/1597189612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0140b31c0 0x7fe0140b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe013e00640 1 -- 172.21.15.87:0/1597189612 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000866198 0 0) 0x7fe01415b080 con 0x7fe014070350 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe013e00640 1 -- 172.21.15.87:0/1597189612 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdffc003530 con 0x7fe014070350 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe019c00640 1 -- 172.21.15.87:0/1597189612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0140b31c0 msgr2=0x7fe0140b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe019c00640 1 --2- 172.21.15.87:0/1597189612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0140b31c0 0x7fe0140b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe019c00640 1 -- 172.21.15.87:0/1597189612 >> v1:172.21.15.87:6789/0 conn(0x7fe014070350 legacy=0x7fe01415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe019c00640 1 -- 172.21.15.87:0/1597189612 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01415ee40 con 0x7fe0140aef80 2024-04-25T10:22:58.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe019c00640 1 --2- 172.21.15.87:0/1597189612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140aef80 0x7fe01415e480 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820bf1dcb43a102d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:58.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe013e00640 1 -- 172.21.15.87:0/1597189612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080b2070 con 0x7fe0140aef80 2024-04-25T10:22:58.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe013e00640 1 -- 172.21.15.87:0/1597189612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0080a5940 con 0x7fe0140aef80 2024-04-25T10:22:58.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe013e00640 1 -- 172.21.15.87:0/1597189612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080a5cf0 con 0x7fe0140aef80 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe01c600640 1 -- 172.21.15.87:0/1597189612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140aef80 msgr2=0x7fe01415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe01c600640 1 --2- 172.21.15.87:0/1597189612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140aef80 0x7fe01415e480 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00801c970 comp rx=0 tx=0).stop 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe01c600640 1 -- 172.21.15.87:0/1597189612 shutdown_connections 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe01c600640 1 --2- 172.21.15.87:0/1597189612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0140b31c0 0x7fe0140b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.257+0000 7fe01c600640 1 --2- 172.21.15.87:0/1597189612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140aef80 0x7fe01415e480 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 -- 172.21.15.87:0/1597189612 >> 172.21.15.87:0/1597189612 conn(0x7fe0140669a0 msgr2=0x7fe014066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 -- 172.21.15.87:0/1597189612 shutdown_connections 2024-04-25T10:22:58.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 -- 172.21.15.87:0/1597189612 wait complete. 2024-04-25T10:22:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 Processor -- start 2024-04-25T10:22:58.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 -- start start 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe014070350 0x7fe0141562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0140aef80 0x7fe0140ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140b31c0 0x7fe0140cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0141679a0 con 0x7fe0140b31c0 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe014167820 con 0x7fe0140aef80 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe014167b20 con 0x7fe014070350 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140b31c0 0x7fe0140cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140b31c0 0x7fe0140cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52092/0 (socket says 172.21.15.87:52092) 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 -- 172.21.15.87:0/2346303817 learned_addr learned my addr 172.21.15.87:0/2346303817 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe019c00640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe014070350 0x7fe0141562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe019200640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0140aef80 0x7fe0140ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 -- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe014070350 msgr2=0x7fe0141562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe014070350 0x7fe0141562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 -- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0140aef80 msgr2=0x7fe0140ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0140aef80 0x7fe0140ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 -- 172.21.15.87:0/2346303817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014156830 con 0x7fe0140b31c0 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe019200640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0140aef80 0x7fe0140ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe01a600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140b31c0 0x7fe0140cd190 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00e7f0 tx=0x7fe00c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe011600640 1 -- 172.21.15.87:0/2346303817 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c017070 con 0x7fe0140b31c0 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe011600640 1 -- 172.21.15.87:0/2346303817 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c00c940 con 0x7fe0140b31c0 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.261+0000 7fe011600640 1 -- 172.21.15.87:0/2346303817 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00eea0 con 0x7fe0140b31c0 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0141f6a40 con 0x7fe0140b31c0 2024-04-25T10:22:58.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0141f7000 con 0x7fe0140b31c0 2024-04-25T10:22:58.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01404ef30 con 0x7fe0140b31c0 2024-04-25T10:22:58.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe011600640 1 -- 172.21.15.87:0/2346303817 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe00c01a070 con 0x7fe0140b31c0 2024-04-25T10:22:58.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe011600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff4066f60 0x7fdff4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:58.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe011600640 1 -- 172.21.15.87:0/2346303817 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe00c085670 con 0x7fe0140b31c0 2024-04-25T10:22:58.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe019c00640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff4066f60 0x7fdff4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:58.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.265+0000 7fe019c00640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff4066f60 0x7fdff4069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe0080af000 tx=0x7fe0080a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:58.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.269+0000 7fe011600640 1 -- 172.21.15.87:0/2346303817 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe00c0566f0 con 0x7fe0140b31c0 2024-04-25T10:22:58.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:58.457+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe0140711b0 con 0x7fe0140b31c0 2024-04-25T10:22:59.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.409+0000 7fe011600640 1 -- 172.21.15.87:0/2346303817 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe00c055ff0 con 0x7fe0140b31c0 2024-04-25T10:22:59.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff4066f60 msgr2=0x7fdff4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff4066f60 0x7fdff4069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe0080af000 tx=0x7fe0080a7070 comp rx=0 tx=0).stop 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140b31c0 msgr2=0x7fe0140cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140b31c0 0x7fe0140cd190 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00e7f0 tx=0x7fe00c00ecb0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 shutdown_connections 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff4066f60 0x7fdff4069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0140b31c0 0x7fe0140cd190 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0140aef80 0x7fe0140ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 --2- 172.21.15.87:0/2346303817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe014070350 0x7fe0141562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 >> 172.21.15.87:0/2346303817 conn(0x7fe0140669a0 msgr2=0x7fe01415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 shutdown_connections 2024-04-25T10:22:59.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.413+0000 7fe01c600640 1 -- 172.21.15.87:0/2346303817 wait complete. 2024-04-25T10:22:59.442 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:22:59.442 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:22:59.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd92e00640 1 Processor -- start 2024-04-25T10:22:59.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd92e00640 1 -- start start 2024-04-25T10:22:59.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd92e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd92e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd8c06cf40 0x7efd8c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd92e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c0cdab0 con 0x7efd8c0d0d00 2024-04-25T10:22:59.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd92e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c0cd210 con 0x7efd8c06cf40 2024-04-25T10:22:59.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd92e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0cd4f0 con 0x7efd8c0cc200 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd8b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd8b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:36196/0 (socket says 172.21.15.87:36196) 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd8b400640 1 -- 172.21.15.87:0/1054345892 learned_addr learned my addr 172.21.15.87:0/1054345892 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd90e00640 1 --2- 172.21.15.87:0/1054345892 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06cf40 0x7efd8c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.581+0000 7efd8be00640 1 --1- 172.21.15.87:0/1054345892 >> v1:172.21.15.169:6789/0 conn(0x7efd8c0d0d00 0x7efd8c162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:36736/0 (socket says 172.21.15.87:36736) 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8b400640 1 -- 172.21.15.87:0/1054345892 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06cf40 msgr2=0x7efd8c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8b400640 1 --2- 172.21.15.87:0/1054345892 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06cf40 0x7efd8c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8b400640 1 -- 172.21.15.87:0/1054345892 >> v1:172.21.15.169:6789/0 conn(0x7efd8c0d0d00 legacy=0x7efd8c162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8b400640 1 -- 172.21.15.87:0/1054345892 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0cd7d0 con 0x7efd8c0cc200 2024-04-25T10:22:59.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8b400640 1 --2- 172.21.15.87:0/1054345892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0ccc30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efd74009870 tx=0x7efd7401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3d4e20d2b6ebf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:59.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8aa00640 1 -- 172.21.15.87:0/1054345892 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd740b0070 con 0x7efd8c0cc200 2024-04-25T10:22:59.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8aa00640 1 -- 172.21.15.87:0/1054345892 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd740ab600 con 0x7efd8c0cc200 2024-04-25T10:22:59.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.585+0000 7efd8aa00640 1 -- 172.21.15.87:0/1054345892 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd740ab9d0 con 0x7efd8c0cc200 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- 172.21.15.87:0/1054345892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 msgr2=0x7efd8c0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 --2- 172.21.15.87:0/1054345892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0ccc30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efd74009870 tx=0x7efd7401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- 172.21.15.87:0/1054345892 shutdown_connections 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 --2- 172.21.15.87:0/1054345892 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06cf40 0x7efd8c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 --2- 172.21.15.87:0/1054345892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0ccc30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- 172.21.15.87:0/1054345892 >> 172.21.15.87:0/1054345892 conn(0x7efd8c072290 msgr2=0x7efd8c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- 172.21.15.87:0/1054345892 shutdown_connections 2024-04-25T10:22:59.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- 172.21.15.87:0/1054345892 wait complete. 2024-04-25T10:22:59.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 Processor -- start 2024-04-25T10:22:59.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- start start 2024-04-25T10:22:59.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd8c06cf40 0x7efd8c15fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0d0d00 0x7efd8c099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c15e040 con 0x7efd8c06cf40 2024-04-25T10:22:59.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c162aa0 con 0x7efd8c0d0d00 2024-04-25T10:22:59.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd92e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c15e1c0 con 0x7efd8c0cc200 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd8c06cf40 0x7efd8c15fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd8c06cf40 0x7efd8c15fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52094/0 (socket says 172.21.15.87:52094) 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 -- 172.21.15.87:0/1924226467 learned_addr learned my addr 172.21.15.87:0/1924226467 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd90e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0d0d00 0x7efd8c099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8b400640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 -- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 msgr2=0x7efd8c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 -- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0d0d00 msgr2=0x7efd8c099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0d0d00 0x7efd8c099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 -- 172.21.15.87:0/1924226467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c157150 con 0x7efd8c06cf40 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.589+0000 7efd8be00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd8c06cf40 0x7efd8c15fa60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c370 tx=0x7efd7c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd83e00640 1 -- 172.21.15.87:0/1924226467 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0074e0 con 0x7efd8c06cf40 2024-04-25T10:22:59.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd83e00640 1 -- 172.21.15.87:0/1924226467 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c007680 con 0x7efd8c06cf40 2024-04-25T10:22:59.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c157430 con 0x7efd8c06cf40 2024-04-25T10:22:59.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd83e00640 1 -- 172.21.15.87:0/1924226467 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c004a30 con 0x7efd8c06cf40 2024-04-25T10:22:59.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c1579f0 con 0x7efd8c06cf40 2024-04-25T10:22:59.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c04ef30 con 0x7efd8c06cf40 2024-04-25T10:22:59.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd83e00640 1 -- 172.21.15.87:0/1924226467 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd7c01a020 con 0x7efd8c06cf40 2024-04-25T10:22:59.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd83e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd68066f60 0x7efd68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:59.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd83e00640 1 -- 172.21.15.87:0/1924226467 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd7c084750 con 0x7efd8c06cf40 2024-04-25T10:22:59.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.593+0000 7efd8b400640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd68066f60 0x7efd68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:59.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.597+0000 7efd8b400640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd68066f60 0x7efd68069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efd740ad000 tx=0x7efd74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:59.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.597+0000 7efd83e00640 1 -- 172.21.15.87:0/1924226467 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd7c055070 con 0x7efd8c06cf40 2024-04-25T10:22:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:22:59.785+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efd8c0b61a0 con 0x7efd8c06cf40 2024-04-25T10:23:00.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd83e00640 1 -- 172.21.15.87:0/1924226467 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) v1 ==== 122+0+0 (secure 0 0 0) 0x7efd7c059110 con 0x7efd8c06cf40 2024-04-25T10:23:00.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd68066f60 msgr2=0x7efd68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd68066f60 0x7efd68069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efd740ad000 tx=0x7efd74008040 comp rx=0 tx=0).stop 2024-04-25T10:23:00.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd8c06cf40 msgr2=0x7efd8c15fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd8c06cf40 0x7efd8c15fa60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c370 tx=0x7efd7c00c830 comp rx=0 tx=0).stop 2024-04-25T10:23:00.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 shutdown_connections 2024-04-25T10:23:00.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd68066f60 0x7efd68069420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0d0d00 0x7efd8c099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c0cc200 0x7efd8c0afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 --2- 172.21.15.87:0/1924226467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd8c06cf40 0x7efd8c15fa60 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 >> 172.21.15.87:0/1924226467 conn(0x7efd8c072290 msgr2=0x7efd8c0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:00.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 shutdown_connections 2024-04-25T10:23:00.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.417+0000 7efd92e00640 1 -- 172.21.15.87:0/1924226467 wait complete. 2024-04-25T10:23:00.440 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:23:00.441 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:00.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35fba00640 1 Processor -- start 2024-04-25T10:23:00.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35fba00640 1 -- start start 2024-04-25T10:23:00.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35fba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35f40ae020 0x7f35f40ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35fba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f40ad840 con 0x7f35f40cf930 2024-04-25T10:23:00.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35fba00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f415e9f0 con 0x7f35f4070c20 2024-04-25T10:23:00.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35fba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f415bb90 con 0x7f35f40ae020 2024-04-25T10:23:00.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f9000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f35f40cf930 0x7f35f40cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59760/0 (socket says 172.21.15.87:59760) 2024-04-25T10:23:00.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f9000640 1 -- 172.21.15.87:0/2932751058 learned_addr learned my addr 172.21.15.87:0/2932751058 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f9a00640 1 --2- 172.21.15.87:0/2932751058 >> v2:172.21.15.169:3300/0 conn(0x7f35f40ae020 0x7f35f40ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179497657 0 0) 0x7f35f40ad840 con 0x7f35f40cf930 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d8003530 con 0x7f35f40cf930 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746952409 0 0) 0x7f35f415e9f0 con 0x7f35f4070c20 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f40ad840 con 0x7f35f4070c20 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572488000 0 0) 0x7f35d8003530 con 0x7f35f40cf930 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 >> v2:172.21.15.169:3300/0 conn(0x7f35f40ae020 msgr2=0x7f35f40ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 --2- 172.21.15.87:0/2932751058 >> v2:172.21.15.169:3300/0 conn(0x7f35f40ae020 0x7f35f40ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 >> v1:172.21.15.169:6789/0 conn(0x7f35f4070c20 legacy=0x7f35f415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f415ecd0 con 0x7f35f40cf930 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f35e4001680 con 0x7f35f40cf930 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 >> v1:172.21.15.87:6789/0 conn(0x7f35f40cf930 legacy=0x7f35f40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8003b50 0x7f35d8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35d8004810 0x7f35d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35d80051d0 0x7f35d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f415bb90 con 0x7f35d8004810 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d800c050 con 0x7f35d8003b50 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.573+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d800c250 con 0x7f35d80051d0 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3e00640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35d8004810 0x7f35d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f9000640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8003b50 0x7f35d8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f9a00640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35d80051d0 0x7f35d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3e00640 1 -- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35d80051d0 msgr2=0x7f35d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3e00640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35d80051d0 0x7f35d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3e00640 1 -- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8003b50 msgr2=0x7f35d8003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3e00640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8003b50 0x7f35d8003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3e00640 1 -- 172.21.15.87:0/2932751058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8003530 con 0x7f35d8004810 2024-04-25T10:23:00.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f9a00640 1 -- 172.21.15.87:0/2932751058 reap_dead start 2024-04-25T10:23:00.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3e00640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35d8004810 0x7f35d8004c90 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f35e00018d0 tx=0x7f35e001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d99698a6f74560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:00.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0001520 con 0x7f35d8004810 2024-04-25T10:23:00.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e00052f0 con 0x7f35d8004810 2024-04-25T10:23:00.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.577+0000 7f35f3400640 1 -- 172.21.15.87:0/2932751058 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00a88f0 con 0x7f35d8004810 2024-04-25T10:23:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- 172.21.15.87:0/2932751058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35d8004810 msgr2=0x7f35d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35d8004810 0x7f35d8004c90 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f35e00018d0 tx=0x7f35e001ec90 comp rx=0 tx=0).stop 2024-04-25T10:23:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- 172.21.15.87:0/2932751058 shutdown_connections 2024-04-25T10:23:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 --2- 172.21.15.87:0/2932751058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35d8004810 0x7f35d8004c90 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- 172.21.15.87:0/2932751058 >> 172.21.15.87:0/2932751058 conn(0x7f35f4066b70 msgr2=0x7f35f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- 172.21.15.87:0/2932751058 shutdown_connections 2024-04-25T10:23:00.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- 172.21.15.87:0/2932751058 wait complete. 2024-04-25T10:23:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 Processor -- start 2024-04-25T10:23:00.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- start start 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8004810 0x7f35f4159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f4161500 0x7f35f4161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35f41f3990 0x7f35f41f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f406e130 con 0x7f35f41f3990 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40adb00 con 0x7f35d8004810 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35fba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40ae4b0 con 0x7f35f4161500 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35f41f3990 0x7f35f41f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35f41f3990 0x7f35f41f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52102/0 (socket says 172.21.15.87:52102) 2024-04-25T10:23:00.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 -- 172.21.15.87:0/1266961133 learned_addr learned my addr 172.21.15.87:0/1266961133 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9000640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8004810 0x7f35f4159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f3e00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f4161500 0x7f35f4161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 -- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f4161500 msgr2=0x7f35f4161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f4161500 0x7f35f4161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 -- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8004810 msgr2=0x7f35f4159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8004810 0x7f35f4159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 -- 172.21.15.87:0/1266961133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f41f4750 con 0x7f35f41f3990 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9000640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8004810 0x7f35f4159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.581+0000 7f35f9a00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35f41f3990 0x7f35f41f3e10 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f35f40af190 tx=0x7f35ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35f0c00640 1 -- 172.21.15.87:0/1266961133 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec011e10 con 0x7f35f41f3990 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35f0c00640 1 -- 172.21.15.87:0/1266961133 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec01b070 con 0x7f35f41f3990 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35f0c00640 1 -- 172.21.15.87:0/1266961133 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0039a0 con 0x7f35f41f3990 2024-04-25T10:23:00.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f41f4a80 con 0x7f35f41f3990 2024-04-25T10:23:00.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f41f5040 con 0x7f35f41f3990 2024-04-25T10:23:00.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f404ef30 con 0x7f35f41f3990 2024-04-25T10:23:00.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35f0c00640 1 -- 172.21.15.87:0/1266961133 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35ec003b60 con 0x7f35f41f3990 2024-04-25T10:23:00.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35f0c00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35d4066f60 0x7f35d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35f0c00640 1 -- 172.21.15.87:0/1266961133 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35ec085890 con 0x7f35f41f3990 2024-04-25T10:23:00.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.585+0000 7f35f9000640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35d4066f60 0x7f35d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.589+0000 7f35f9000640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35d4066f60 0x7f35d4069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f35e4000c10 tx=0x7f35e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:00.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.589+0000 7f35f0c00640 1 -- 172.21.15.87:0/1266961133 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35ec025020 con 0x7f35f41f3990 2024-04-25T10:23:00.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35f406da10 con 0x7f35f41f3990 2024-04-25T10:23:00.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35f0c00640 1 -- 172.21.15.87:0/1266961133 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4944 (secure 0 0 0) 0x7f35ec056930 con 0x7f35f41f3990 2024-04-25T10:23:00.780 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:00.781 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8755,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4075242775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4075242775},{"type":"v1","addr":"172.21.15.87:6836","nonce":4075242775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:22:56.766091+0000","modified":"2024-04-25T10:22:59.792927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8758":{"gid":8758,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1025107360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1025107360},{"type":"v1","addr":"172.21.15.87:6837","nonce":1025107360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:23:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 100 2024-04-25T10:23:00.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35d4066f60 msgr2=0x7f35d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35d4066f60 0x7f35d4069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f35e4000c10 tx=0x7f35e4009040 comp rx=0 tx=0).stop 2024-04-25T10:23:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35f41f3990 msgr2=0x7f35f41f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35f41f3990 0x7f35f41f3e10 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f35f40af190 tx=0x7f35ec0115a0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 shutdown_connections 2024-04-25T10:23:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f35d4066f60 0x7f35d4069420 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35f41f3990 0x7f35f41f3e10 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f4161500 0x7f35f4161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 --2- 172.21.15.87:0/1266961133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35d8004810 0x7f35f4159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 >> 172.21.15.87:0/1266961133 conn(0x7f35f4066b70 msgr2=0x7f35f4068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 shutdown_connections 2024-04-25T10:23:00.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.777+0000 7f35fba00640 1 -- 172.21.15.87:0/1266961133 wait complete. 2024-04-25T10:23:00.801 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:00.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd85b000640 1 Processor -- start 2024-04-25T10:23:00.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd85b000640 1 -- start start 2024-04-25T10:23:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd85b000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd8541666f0 0x7fd8540c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd85b000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540cf2e0 con 0x7fd8540c4e10 2024-04-25T10:23:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd85b000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540b2dd0 con 0x7fd854162020 2024-04-25T10:23:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd85b000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd854166260 con 0x7fd8541666f0 2024-04-25T10:23:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd859000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd8540c4e10 0x7fd8540b1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59762/0 (socket says 172.21.15.87:59762) 2024-04-25T10:23:00.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd859000640 1 -- 172.21.15.87:0/4087028675 learned_addr learned my addr 172.21.15.87:0/4087028675 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918118271 0 0) 0x7fd8540cf2e0 con 0x7fd8540c4e10 2024-04-25T10:23:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd838003530 con 0x7fd8540c4e10 2024-04-25T10:23:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd853400640 1 --2- 172.21.15.87:0/4087028675 >> v2:172.21.15.169:3300/0 conn(0x7fd8541666f0 0x7fd8540c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101431296 0 0) 0x7fd838003530 con 0x7fd8540c4e10 2024-04-25T10:23:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 >> v2:172.21.15.169:3300/0 conn(0x7fd8541666f0 msgr2=0x7fd8540c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 --2- 172.21.15.87:0/4087028675 >> v2:172.21.15.169:3300/0 conn(0x7fd8541666f0 0x7fd8540c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 >> v1:172.21.15.169:6789/0 conn(0x7fd854162020 legacy=0x7fd8540cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540b30b0 con 0x7fd8540c4e10 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd84c001680 con 0x7fd8540c4e10 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 >> v1:172.21.15.87:6789/0 conn(0x7fd8540c4e10 legacy=0x7fd8540b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380039c0 0x7fd838003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838004810 0x7fd838004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380051d0 0x7fd83800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd854166260 con 0x7fd8380039c0 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83800c120 con 0x7fd8380051d0 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.933+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83800c320 con 0x7fd838004810 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853e00640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380039c0 0x7fd838003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853400640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838004810 0x7fd838004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd859000640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380051d0 0x7fd83800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853e00640 1 -- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838004810 msgr2=0x7fd838004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853e00640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd838004810 0x7fd838004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853e00640 1 -- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380051d0 msgr2=0x7fd83800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853e00640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8380051d0 0x7fd83800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853e00640 1 -- 172.21.15.87:0/4087028675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd838003530 con 0x7fd8380039c0 2024-04-25T10:23:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd859000640 1 -- 172.21.15.87:0/4087028675 reap_dead start 2024-04-25T10:23:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd853e00640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380039c0 0x7fd838003de0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fd844001e70 tx=0x7fd84401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecda32651ee00548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8440b0070 con 0x7fd8380039c0 2024-04-25T10:23:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844003b80 con 0x7fd8380039c0 2024-04-25T10:23:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.937+0000 7fd852a00640 1 -- 172.21.15.87:0/4087028675 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844001a90 con 0x7fd8380039c0 2024-04-25T10:23:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- 172.21.15.87:0/4087028675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380039c0 msgr2=0x7fd838003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380039c0 0x7fd838003de0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fd844001e70 tx=0x7fd84401c760 comp rx=0 tx=0).stop 2024-04-25T10:23:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- 172.21.15.87:0/4087028675 shutdown_connections 2024-04-25T10:23:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 --2- 172.21.15.87:0/4087028675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8380039c0 0x7fd838003de0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- 172.21.15.87:0/4087028675 >> 172.21.15.87:0/4087028675 conn(0x7fd85409a9d0 msgr2=0x7fd85409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- 172.21.15.87:0/4087028675 shutdown_connections 2024-04-25T10:23:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- 172.21.15.87:0/4087028675 wait complete. 2024-04-25T10:23:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 Processor -- start 2024-04-25T10:23:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- start start 2024-04-25T10:23:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8380039c0 0x7fd8540d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85415b070 0x7fd85415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8540cf2e0 0x7fd8541604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540cf160 con 0x7fd85415b070 2024-04-25T10:23:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540cefe0 con 0x7fd8540cf2e0 2024-04-25T10:23:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd85b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd854162400 con 0x7fd8380039c0 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85415b070 0x7fd85415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85415b070 0x7fd85415aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52110/0 (socket says 172.21.15.87:52110) 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 -- 172.21.15.87:0/1131506517 learned_addr learned my addr 172.21.15.87:0/1131506517 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853e00640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8380039c0 0x7fd8540d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd859000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8540cf2e0 0x7fd8541604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 -- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8380039c0 msgr2=0x7fd8540d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8380039c0 0x7fd8540d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 -- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8540cf2e0 msgr2=0x7fd8541604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8540cf2e0 0x7fd8541604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 -- 172.21.15.87:0/1131506517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540cfac0 con 0x7fd85415b070 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853e00640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8380039c0 0x7fd8540d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd859000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8540cf2e0 0x7fd8541604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:00.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.941+0000 7fd853400640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85415b070 0x7fd85415aa60 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b6f0 tx=0x7fd84000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:00.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd84be00640 1 -- 172.21.15.87:0/1131506517 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840018070 con 0x7fd85415b070 2024-04-25T10:23:00.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd84be00640 1 -- 172.21.15.87:0/1131506517 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840010830 con 0x7fd85415b070 2024-04-25T10:23:00.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8540cfda0 con 0x7fd85415b070 2024-04-25T10:23:00.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd84be00640 1 -- 172.21.15.87:0/1131506517 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840010c00 con 0x7fd85415b070 2024-04-25T10:23:00.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8540d0360 con 0x7fd85415b070 2024-04-25T10:23:00.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85404ef30 con 0x7fd85415b070 2024-04-25T10:23:00.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd84be00640 1 -- 172.21.15.87:0/1131506517 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd84001b070 con 0x7fd85415b070 2024-04-25T10:23:00.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd84be00640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd834066f90 0x7fd834069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:00.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd84be00640 1 -- 172.21.15.87:0/1131506517 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd840085320 con 0x7fd85415b070 2024-04-25T10:23:00.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.945+0000 7fd853e00640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd834066f90 0x7fd834069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:00.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.949+0000 7fd853e00640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd834066f90 0x7fd834069450 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd8440ad000 tx=0x7fd844001520 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:00.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:00.949+0000 7fd84be00640 1 -- 172.21.15.87:0/1131506517 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd840056290 con 0x7fd85415b070 2024-04-25T10:23:01.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.105+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd854068020 con 0x7fd85415b070 2024-04-25T10:23:01.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.105+0000 7fd84be00640 1 -- 172.21.15.87:0/1131506517 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16667 (secure 0 0 0) 0x7fd840055b90 con 0x7fd85415b070 2024-04-25T10:23:01.109 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:01.109 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":138,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:22:58.400079+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:54.523910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:55.777113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.105+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd834066f90 msgr2=0x7fd834069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.105+0000 7fd85b000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd834066f90 0x7fd834069450 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd8440ad000 tx=0x7fd844001520 comp rx=0 tx=0).stop 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.105+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85415b070 msgr2=0x7fd85415aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.105+0000 7fd85b000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85415b070 0x7fd85415aa60 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b6f0 tx=0x7fd84000bbb0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 shutdown_connections 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd834066f90 0x7fd834069450 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8540cf2e0 0x7fd8541604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85415b070 0x7fd85415aa60 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 --2- 172.21.15.87:0/1131506517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8380039c0 0x7fd8540d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 >> 172.21.15.87:0/1131506517 conn(0x7fd85409a9d0 msgr2=0x7fd8540ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:01.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 shutdown_connections 2024-04-25T10:23:01.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.109+0000 7fd85b000640 1 -- 172.21.15.87:0/1131506517 wait complete. 2024-04-25T10:23:01.131 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:23:01.131 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:23:01.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 Processor -- start 2024-04-25T10:23:01.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 -- start start 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066c00 0x7f3eb815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3eb8066fe0 0x7f3eb80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3eb806ed30 0x7f3eb8157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb80b5c20 con 0x7f3eb806ed30 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb80af6e0 con 0x7f3eb8066fe0 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3ebe000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80af9c0 con 0x7f3eb8066c00 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3eb806ed30 0x7f3eb8157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3eb806ed30 0x7f3eb8157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52124/0 (socket says 172.21.15.87:52124) 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 -- 172.21.15.87:0/2742907403 learned_addr learned my addr 172.21.15.87:0/2742907403 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb6a00640 1 --2- 172.21.15.87:0/2742907403 >> v2:172.21.15.169:3300/0 conn(0x7f3eb8066fe0 0x7f3eb80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7400640 1 --2- 172.21.15.87:0/2742907403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066c00 0x7f3eb815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 -- 172.21.15.87:0/2742907403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066c00 msgr2=0x7f3eb815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 --2- 172.21.15.87:0/2742907403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066c00 0x7f3eb815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 -- 172.21.15.87:0/2742907403 >> v2:172.21.15.169:3300/0 conn(0x7f3eb8066fe0 msgr2=0x7f3eb80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 --2- 172.21.15.87:0/2742907403 >> v2:172.21.15.169:3300/0 conn(0x7f3eb8066fe0 0x7f3eb80b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7e00640 1 -- 172.21.15.87:0/2742907403 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80afca0 con 0x7f3eb806ed30 2024-04-25T10:23:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb6a00640 1 --2- 172.21.15.87:0/2742907403 >> v2:172.21.15.169:3300/0 conn(0x7f3eb8066fe0 0x7f3eb80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:01.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.265+0000 7f3eb7400640 1 --2- 172.21.15.87:0/2742907403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066c00 0x7f3eb815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3eb7e00640 1 --2- 172.21.15.87:0/2742907403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb806ed30 0x7f3eb8157b50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f3eb806bd20 tx=0x7f3eb001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e8f9d3e33227fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3eb6000640 1 -- 172.21.15.87:0/2742907403 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00a5860 con 0x7f3eb806ed30 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3eb6000640 1 -- 172.21.15.87:0/2742907403 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb00a5a00 con 0x7f3eb806ed30 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3eb6000640 1 -- 172.21.15.87:0/2742907403 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00a5dd0 con 0x7f3eb806ed30 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 -- 172.21.15.87:0/2742907403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb806ed30 msgr2=0x7f3eb8157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2742907403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb806ed30 0x7f3eb8157b50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f3eb806bd20 tx=0x7f3eb001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 -- 172.21.15.87:0/2742907403 shutdown_connections 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2742907403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb806ed30 0x7f3eb8157b50 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2742907403 >> v2:172.21.15.169:3300/0 conn(0x7f3eb8066fe0 0x7f3eb80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2742907403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066c00 0x7f3eb815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 -- 172.21.15.87:0/2742907403 >> 172.21.15.87:0/2742907403 conn(0x7f3eb80b24f0 msgr2=0x7f3eb80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:01.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 -- 172.21.15.87:0/2742907403 shutdown_connections 2024-04-25T10:23:01.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.269+0000 7f3ebe000640 1 -- 172.21.15.87:0/2742907403 wait complete. 2024-04-25T10:23:01.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 Processor -- start 2024-04-25T10:23:01.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 -- start start 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb8066c00 0x7f3eb8145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066fe0 0x7f3eb814d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3eb806ed30 0x7f3eb80d0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80af1e0 con 0x7f3eb8066c00 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80af060 con 0x7f3eb806ed30 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb81617e0 con 0x7f3eb8066fe0 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb8066c00 0x7f3eb8145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb8066c00 0x7f3eb8145370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52130/0 (socket says 172.21.15.87:52130) 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 -- 172.21.15.87:0/2088504214 learned_addr learned my addr 172.21.15.87:0/2088504214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:01.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 -- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066fe0 msgr2=0x7f3eb814d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066fe0 0x7f3eb814d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 -- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3eb806ed30 msgr2=0x7f3eb80d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7e00640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3eb806ed30 0x7f3eb80d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3eb806ed30 0x7f3eb80d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 -- 172.21.15.87:0/2088504214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb814dad0 con 0x7f3eb8066c00 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7e00640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3eb806ed30 0x7f3eb80d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eb7400640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb8066c00 0x7f3eb8145370 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800c7f0 tx=0x7f3ea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:01.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eaf400640 1 -- 172.21.15.87:0/2088504214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8007a30 con 0x7f3eb8066c00 2024-04-25T10:23:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eaf400640 1 -- 172.21.15.87:0/2088504214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea8007bd0 con 0x7f3eb8066c00 2024-04-25T10:23:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3eaf400640 1 -- 172.21.15.87:0/2088504214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80113f0 con 0x7f3eb8066c00 2024-04-25T10:23:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb814ddb0 con 0x7f3eb8066c00 2024-04-25T10:23:01.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb81458b0 con 0x7f3eb8066c00 2024-04-25T10:23:01.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.273+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb804ef30 con 0x7f3eb8066c00 2024-04-25T10:23:01.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.277+0000 7f3eaf400640 1 -- 172.21.15.87:0/2088504214 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ea8011590 con 0x7f3eb8066c00 2024-04-25T10:23:01.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.277+0000 7f3eaf400640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e98066f90 0x7f3e98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.277+0000 7f3eaf400640 1 -- 172.21.15.87:0/2088504214 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ea80851a0 con 0x7f3eb8066c00 2024-04-25T10:23:01.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.277+0000 7f3eb6a00640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e98066f90 0x7f3e98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.277+0000 7f3eb6a00640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e98066f90 0x7f3e98069450 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3eb814c490 tx=0x7f3ea4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:01.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.281+0000 7f3eaf400640 1 -- 172.21.15.87:0/2088504214 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ea8015020 con 0x7f3eb8066c00 2024-04-25T10:23:01.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.433+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3eb8071c70 con 0x7f3e98066f90 2024-04-25T10:23:01.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.437+0000 7f3eaf400640 1 -- 172.21.15.87:0/2088504214 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7f3eb8071c70 con 0x7f3e98066f90 2024-04-25T10:23:01.445 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:01.447 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:23:01.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e98066f90 msgr2=0x7f3e98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e98066f90 0x7f3e98069450 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3eb814c490 tx=0x7f3ea4008040 comp rx=0 tx=0).stop 2024-04-25T10:23:01.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb8066c00 msgr2=0x7f3eb8145370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb8066c00 0x7f3eb8145370 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800c7f0 tx=0x7f3ea800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 shutdown_connections 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e98066f90 0x7f3e98069450 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3eb806ed30 0x7f3eb80d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3eb8066fe0 0x7f3eb814d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 --2- 172.21.15.87:0/2088504214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3eb8066c00 0x7f3eb8145370 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 >> 172.21.15.87:0/2088504214 conn(0x7f3eb80b24f0 msgr2=0x7f3eb806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 shutdown_connections 2024-04-25T10:23:01.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.441+0000 7f3ebe000640 1 -- 172.21.15.87:0/2088504214 wait complete. 2024-04-25T10:23:01.466 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-04-25T10:23:01.259779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":477880,"kb_used_data":14728,"kb_used_omap":12,"kb_used_meta":463027,"kb_avail":754496840,"statfs":{"total":773094113280,"available":772604764160,"internally_reserved":0,"allocated":15081472,"data_stored":7238252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":474140209},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.877800"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405405+0000","last_change":"2024-04-25T10:22:57.428256+0000","last_active":"2024-04-25T10:22:58.405405+0000","last_peered":"2024-04-25T10:22:58.405405+0000","last_clean":"2024-04-25T10:22:58.405405+0000","last_became_active":"2024-04-25T10:22:57.428007+0000","last_became_peered":"2024-04-25T10:22:57.428007+0000","last_unstale":"2024-04-25T10:22:58.405405+0000","last_undegraded":"2024-04-25T10:22:58.405405+0000","last_fullsized":"2024-04-25T10:22:58.405405+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:31.853880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883926+0000","last_change":"2024-04-25T10:22:57.424019+0000","last_active":"2024-04-25T10:22:58.883926+0000","last_peered":"2024-04-25T10:22:58.883926+0000","last_clean":"2024-04-25T10:22:58.883926+0000","last_became_active":"2024-04-25T10:22:57.423735+0000","last_became_peered":"2024-04-25T10:22:57.423735+0000","last_unstale":"2024-04-25T10:22:58.883926+0000","last_undegraded":"2024-04-25T10:22:58.883926+0000","last_fullsized":"2024-04-25T10:22:58.883926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.139463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459563+0000","last_change":"2024-04-25T10:22:57.425360+0000","last_active":"2024-04-25T10:22:59.459563+0000","last_peered":"2024-04-25T10:22:59.459563+0000","last_clean":"2024-04-25T10:22:59.459563+0000","last_became_active":"2024-04-25T10:22:57.424969+0000","last_became_peered":"2024-04-25T10:22:57.424969+0000","last_unstale":"2024-04-25T10:22:59.459563+0000","last_undegraded":"2024-04-25T10:22:59.459563+0000","last_fullsized":"2024-04-25T10:22:59.459563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:43.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408119+0000","last_change":"2024-04-25T10:22:57.417290+0000","last_active":"2024-04-25T10:22:58.408119+0000","last_peered":"2024-04-25T10:22:58.408119+0000","last_clean":"2024-04-25T10:22:58.408119+0000","last_became_active":"2024-04-25T10:22:57.415224+0000","last_became_peered":"2024-04-25T10:22:57.415224+0000","last_unstale":"2024-04-25T10:22:58.408119+0000","last_undegraded":"2024-04-25T10:22:58.408119+0000","last_fullsized":"2024-04-25T10:22:58.408119+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:53.411478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407422+0000","last_change":"2024-04-25T10:22:57.422694+0000","last_active":"2024-04-25T10:22:58.407422+0000","last_peered":"2024-04-25T10:22:58.407422+0000","last_clean":"2024-04-25T10:22:58.407422+0000","last_became_active":"2024-04-25T10:22:57.422381+0000","last_became_peered":"2024-04-25T10:22:57.422381+0000","last_unstale":"2024-04-25T10:22:58.407422+0000","last_undegraded":"2024-04-25T10:22:58.407422+0000","last_fullsized":"2024-04-25T10:22:58.407422+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:01.498118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404783+0000","last_change":"2024-04-25T10:22:57.425599+0000","last_active":"2024-04-25T10:22:58.404783+0000","last_peered":"2024-04-25T10:22:58.404783+0000","last_clean":"2024-04-25T10:22:58.404783+0000","last_became_active":"2024-04-25T10:22:57.425204+0000","last_became_peered":"2024-04-25T10:22:57.425204+0000","last_unstale":"2024-04-25T10:22:58.404783+0000","last_undegraded":"2024-04-25T10:22:58.404783+0000","last_fullsized":"2024-04-25T10:22:58.404783+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:10.594590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458017+0000","last_change":"2024-04-25T10:22:57.425112+0000","last_active":"2024-04-25T10:22:59.458017+0000","last_peered":"2024-04-25T10:22:59.458017+0000","last_clean":"2024-04-25T10:22:59.458017+0000","last_became_active":"2024-04-25T10:22:57.424860+0000","last_became_peered":"2024-04-25T10:22:57.424860+0000","last_unstale":"2024-04-25T10:22:59.458017+0000","last_undegraded":"2024-04-25T10:22:59.458017+0000","last_fullsized":"2024-04-25T10:22:59.458017+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.285372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404623+0000","last_change":"2024-04-25T10:22:57.419844+0000","last_active":"2024-04-25T10:22:58.404623+0000","last_peered":"2024-04-25T10:22:58.404623+0000","last_clean":"2024-04-25T10:22:58.404623+0000","last_became_active":"2024-04-25T10:22:57.419551+0000","last_became_peered":"2024-04-25T10:22:57.419551+0000","last_unstale":"2024-04-25T10:22:58.404623+0000","last_undegraded":"2024-04-25T10:22:58.404623+0000","last_fullsized":"2024-04-25T10:22:58.404623+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:14.283194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883753+0000","last_change":"2024-04-25T10:22:57.425142+0000","last_active":"2024-04-25T10:22:58.883753+0000","last_peered":"2024-04-25T10:22:58.883753+0000","last_clean":"2024-04-25T10:22:58.883753+0000","last_became_active":"2024-04-25T10:22:57.424896+0000","last_became_peered":"2024-04-25T10:22:57.424896+0000","last_unstale":"2024-04-25T10:22:58.883753+0000","last_undegraded":"2024-04-25T10:22:58.883753+0000","last_fullsized":"2024-04-25T10:22:58.883753+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:03.598286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431773+0000","last_change":"2024-04-25T10:22:57.435334+0000","last_active":"2024-04-25T10:22:58.431773+0000","last_peered":"2024-04-25T10:22:58.431773+0000","last_clean":"2024-04-25T10:22:58.431773+0000","last_became_active":"2024-04-25T10:22:57.433441+0000","last_became_peered":"2024-04-25T10:22:57.433441+0000","last_unstale":"2024-04-25T10:22:58.431773+0000","last_undegraded":"2024-04-25T10:22:58.431773+0000","last_fullsized":"2024-04-25T10:22:58.431773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:17.108874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405017+0000","last_change":"2024-04-25T10:22:57.419920+0000","last_active":"2024-04-25T10:22:58.405017+0000","last_peered":"2024-04-25T10:22:58.405017+0000","last_clean":"2024-04-25T10:22:58.405017+0000","last_became_active":"2024-04-25T10:22:57.419606+0000","last_became_peered":"2024-04-25T10:22:57.419606+0000","last_unstale":"2024-04-25T10:22:58.405017+0000","last_undegraded":"2024-04-25T10:22:58.405017+0000","last_fullsized":"2024-04-25T10:22:58.405017+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.700530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431436+0000","last_change":"2024-04-25T10:22:57.428429+0000","last_active":"2024-04-25T10:22:58.431436+0000","last_peered":"2024-04-25T10:22:58.431436+0000","last_clean":"2024-04-25T10:22:58.431436+0000","last_became_active":"2024-04-25T10:22:57.428123+0000","last_became_peered":"2024-04-25T10:22:57.428123+0000","last_unstale":"2024-04-25T10:22:58.431436+0000","last_undegraded":"2024-04-25T10:22:58.431436+0000","last_fullsized":"2024-04-25T10:22:58.431436+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:53.989570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407951+0000","last_change":"2024-04-25T10:22:57.422202+0000","last_active":"2024-04-25T10:22:58.407951+0000","last_peered":"2024-04-25T10:22:58.407951+0000","last_clean":"2024-04-25T10:22:58.407951+0000","last_became_active":"2024-04-25T10:22:57.421834+0000","last_became_peered":"2024-04-25T10:22:57.421834+0000","last_unstale":"2024-04-25T10:22:58.407951+0000","last_undegraded":"2024-04-25T10:22:58.407951+0000","last_fullsized":"2024-04-25T10:22:58.407951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:31.647346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407078+0000","last_change":"2024-04-25T10:22:57.426203+0000","last_active":"2024-04-25T10:22:58.407078+0000","last_peered":"2024-04-25T10:22:58.407078+0000","last_clean":"2024-04-25T10:22:58.407078+0000","last_became_active":"2024-04-25T10:22:57.425553+0000","last_became_peered":"2024-04-25T10:22:57.425553+0000","last_unstale":"2024-04-25T10:22:58.407078+0000","last_undegraded":"2024-04-25T10:22:58.407078+0000","last_fullsized":"2024-04-25T10:22:58.407078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:58.202829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431438+0000","last_change":"2024-04-25T10:22:57.420873+0000","last_active":"2024-04-25T10:22:58.431438+0000","last_peered":"2024-04-25T10:22:58.431438+0000","last_clean":"2024-04-25T10:22:58.431438+0000","last_became_active":"2024-04-25T10:22:57.420405+0000","last_became_peered":"2024-04-25T10:22:57.420405+0000","last_unstale":"2024-04-25T10:22:58.431438+0000","last_undegraded":"2024-04-25T10:22:58.431438+0000","last_fullsized":"2024-04-25T10:22:58.431438+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:12.153600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457893+0000","last_change":"2024-04-25T10:22:57.423395+0000","last_active":"2024-04-25T10:22:59.457893+0000","last_peered":"2024-04-25T10:22:59.457893+0000","last_clean":"2024-04-25T10:22:59.457893+0000","last_became_active":"2024-04-25T10:22:57.423144+0000","last_became_peered":"2024-04-25T10:22:57.423144+0000","last_unstale":"2024-04-25T10:22:59.457893+0000","last_undegraded":"2024-04-25T10:22:59.457893+0000","last_fullsized":"2024-04-25T10:22:59.457893+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:23.952786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431994+0000","last_change":"2024-04-25T10:22:57.434763+0000","last_active":"2024-04-25T10:22:58.431994+0000","last_peered":"2024-04-25T10:22:58.431994+0000","last_clean":"2024-04-25T10:22:58.431994+0000","last_became_active":"2024-04-25T10:22:57.434509+0000","last_became_peered":"2024-04-25T10:22:57.434509+0000","last_unstale":"2024-04-25T10:22:58.431994+0000","last_undegraded":"2024-04-25T10:22:58.431994+0000","last_fullsized":"2024-04-25T10:22:58.431994+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:16.713083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459658+0000","last_change":"2024-04-25T10:22:57.426289+0000","last_active":"2024-04-25T10:22:59.459658+0000","last_peered":"2024-04-25T10:22:59.459658+0000","last_clean":"2024-04-25T10:22:59.459658+0000","last_became_active":"2024-04-25T10:22:57.426011+0000","last_became_peered":"2024-04-25T10:22:57.426011+0000","last_unstale":"2024-04-25T10:22:59.459658+0000","last_undegraded":"2024-04-25T10:22:59.459658+0000","last_fullsized":"2024-04-25T10:22:59.459658+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:19.500478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405320+0000","last_change":"2024-04-25T10:22:57.426466+0000","last_active":"2024-04-25T10:22:58.405320+0000","last_peered":"2024-04-25T10:22:58.405320+0000","last_clean":"2024-04-25T10:22:58.405320+0000","last_became_active":"2024-04-25T10:22:57.425941+0000","last_became_peered":"2024-04-25T10:22:57.425941+0000","last_unstale":"2024-04-25T10:22:58.405320+0000","last_undegraded":"2024-04-25T10:22:58.405320+0000","last_fullsized":"2024-04-25T10:22:58.405320+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:42.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457729+0000","last_change":"2024-04-25T10:22:57.420833+0000","last_active":"2024-04-25T10:22:59.457729+0000","last_peered":"2024-04-25T10:22:59.457729+0000","last_clean":"2024-04-25T10:22:59.457729+0000","last_became_active":"2024-04-25T10:22:57.420367+0000","last_became_peered":"2024-04-25T10:22:57.420367+0000","last_unstale":"2024-04-25T10:22:59.457729+0000","last_undegraded":"2024-04-25T10:22:59.457729+0000","last_fullsized":"2024-04-25T10:22:59.457729+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:35.129534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431649+0000","last_change":"2024-04-25T10:22:57.435015+0000","last_active":"2024-04-25T10:22:58.431649+0000","last_peered":"2024-04-25T10:22:58.431649+0000","last_clean":"2024-04-25T10:22:58.431649+0000","last_became_active":"2024-04-25T10:22:57.434768+0000","last_became_peered":"2024-04-25T10:22:57.434768+0000","last_unstale":"2024-04-25T10:22:58.431649+0000","last_undegraded":"2024-04-25T10:22:58.431649+0000","last_fullsized":"2024-04-25T10:22:58.431649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:20.542240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407274+0000","last_change":"2024-04-25T10:22:57.419029+0000","last_active":"2024-04-25T10:22:58.407274+0000","last_peered":"2024-04-25T10:22:58.407274+0000","last_clean":"2024-04-25T10:22:58.407274+0000","last_became_active":"2024-04-25T10:22:57.413844+0000","last_became_peered":"2024-04-25T10:22:57.413844+0000","last_unstale":"2024-04-25T10:22:58.407274+0000","last_undegraded":"2024-04-25T10:22:58.407274+0000","last_fullsized":"2024-04-25T10:22:58.407274+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:44.029725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407113+0000","last_change":"2024-04-25T10:22:57.427202+0000","last_active":"2024-04-25T10:22:58.407113+0000","last_peered":"2024-04-25T10:22:58.407113+0000","last_clean":"2024-04-25T10:22:58.407113+0000","last_became_active":"2024-04-25T10:22:57.425700+0000","last_became_peered":"2024-04-25T10:22:57.425700+0000","last_unstale":"2024-04-25T10:22:58.407113+0000","last_undegraded":"2024-04-25T10:22:58.407113+0000","last_fullsized":"2024-04-25T10:22:58.407113+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:32.251839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431312+0000","last_change":"2024-04-25T10:22:57.423072+0000","last_active":"2024-04-25T10:22:58.431312+0000","last_peered":"2024-04-25T10:22:58.431312+0000","last_clean":"2024-04-25T10:22:58.431312+0000","last_became_active":"2024-04-25T10:22:57.422626+0000","last_became_peered":"2024-04-25T10:22:57.422626+0000","last_unstale":"2024-04-25T10:22:58.431312+0000","last_undegraded":"2024-04-25T10:22:58.431312+0000","last_fullsized":"2024-04-25T10:22:58.431312+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:02.186849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457467+0000","last_change":"2024-04-25T10:22:57.422702+0000","last_active":"2024-04-25T10:22:59.457467+0000","last_peered":"2024-04-25T10:22:59.457467+0000","last_clean":"2024-04-25T10:22:59.457467+0000","last_became_active":"2024-04-25T10:22:57.422464+0000","last_became_peered":"2024-04-25T10:22:57.422464+0000","last_unstale":"2024-04-25T10:22:59.457467+0000","last_undegraded":"2024-04-25T10:22:59.457467+0000","last_fullsized":"2024-04-25T10:22:59.457467+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:46.317870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405074+0000","last_change":"2024-04-25T10:22:57.419978+0000","last_active":"2024-04-25T10:22:58.405074+0000","last_peered":"2024-04-25T10:22:58.405074+0000","last_clean":"2024-04-25T10:22:58.405074+0000","last_became_active":"2024-04-25T10:22:57.419693+0000","last_became_peered":"2024-04-25T10:22:57.419693+0000","last_unstale":"2024-04-25T10:22:58.405074+0000","last_undegraded":"2024-04-25T10:22:58.405074+0000","last_fullsized":"2024-04-25T10:22:58.405074+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:08.970371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457891+0000","last_change":"2024-04-25T10:22:57.421289+0000","last_active":"2024-04-25T10:22:59.457891+0000","last_peered":"2024-04-25T10:22:59.457891+0000","last_clean":"2024-04-25T10:22:59.457891+0000","last_became_active":"2024-04-25T10:22:57.419037+0000","last_became_peered":"2024-04-25T10:22:57.419037+0000","last_unstale":"2024-04-25T10:22:59.457891+0000","last_undegraded":"2024-04-25T10:22:59.457891+0000","last_fullsized":"2024-04-25T10:22:59.457891+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:48.757776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458356+0000","last_change":"2024-04-25T10:22:57.423839+0000","last_active":"2024-04-25T10:22:59.458356+0000","last_peered":"2024-04-25T10:22:59.458356+0000","last_clean":"2024-04-25T10:22:59.458356+0000","last_became_active":"2024-04-25T10:22:57.423553+0000","last_became_peered":"2024-04-25T10:22:57.423553+0000","last_unstale":"2024-04-25T10:22:59.458356+0000","last_undegraded":"2024-04-25T10:22:59.458356+0000","last_fullsized":"2024-04-25T10:22:59.458356+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:07.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405054+0000","last_change":"2024-04-25T10:22:57.420415+0000","last_active":"2024-04-25T10:22:58.405054+0000","last_peered":"2024-04-25T10:22:58.405054+0000","last_clean":"2024-04-25T10:22:58.405054+0000","last_became_active":"2024-04-25T10:22:57.419249+0000","last_became_peered":"2024-04-25T10:22:57.419249+0000","last_unstale":"2024-04-25T10:22:58.405054+0000","last_undegraded":"2024-04-25T10:22:58.405054+0000","last_fullsized":"2024-04-25T10:22:58.405054+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:58.646389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407321+0000","last_change":"2024-04-25T10:22:57.425954+0000","last_active":"2024-04-25T10:22:58.407321+0000","last_peered":"2024-04-25T10:22:58.407321+0000","last_clean":"2024-04-25T10:22:58.407321+0000","last_became_active":"2024-04-25T10:22:57.425177+0000","last_became_peered":"2024-04-25T10:22:57.425177+0000","last_unstale":"2024-04-25T10:22:58.407321+0000","last_undegraded":"2024-04-25T10:22:58.407321+0000","last_fullsized":"2024-04-25T10:22:58.407321+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:50.730324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883588+0000","last_change":"2024-04-25T10:22:57.412963+0000","last_active":"2024-04-25T10:22:58.883588+0000","last_peered":"2024-04-25T10:22:58.883588+0000","last_clean":"2024-04-25T10:22:58.883588+0000","last_became_active":"2024-04-25T10:22:57.412648+0000","last_became_peered":"2024-04-25T10:22:57.412648+0000","last_unstale":"2024-04-25T10:22:58.883588+0000","last_undegraded":"2024-04-25T10:22:58.883588+0000","last_fullsized":"2024-04-25T10:22:58.883588+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:07.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458061+0000","last_change":"2024-04-25T10:22:57.418894+0000","last_active":"2024-04-25T10:22:59.458061+0000","last_peered":"2024-04-25T10:22:59.458061+0000","last_clean":"2024-04-25T10:22:59.458061+0000","last_became_active":"2024-04-25T10:22:57.418457+0000","last_became_peered":"2024-04-25T10:22:57.418457+0000","last_unstale":"2024-04-25T10:22:59.458061+0000","last_undegraded":"2024-04-25T10:22:59.458061+0000","last_fullsized":"2024-04-25T10:22:59.458061+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:43.514012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431852+0000","last_change":"2024-04-25T10:22:57.431160+0000","last_active":"2024-04-25T10:22:58.431852+0000","last_peered":"2024-04-25T10:22:58.431852+0000","last_clean":"2024-04-25T10:22:58.431852+0000","last_became_active":"2024-04-25T10:22:57.430903+0000","last_became_peered":"2024-04-25T10:22:57.430903+0000","last_unstale":"2024-04-25T10:22:58.431852+0000","last_undegraded":"2024-04-25T10:22:58.431852+0000","last_fullsized":"2024-04-25T10:22:58.431852+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:22.363914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"137'4","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431839+0000","last_change":"2024-04-25T10:22:56.406672+0000","last_active":"2024-04-25T10:22:58.431839+0000","last_peered":"2024-04-25T10:22:58.431839+0000","last_clean":"2024-04-25T10:22:58.431839+0000","last_became_active":"2024-04-25T10:22:56.406415+0000","last_became_peered":"2024-04-25T10:22:56.406415+0000","last_unstale":"2024-04-25T10:22:58.431839+0000","last_undegraded":"2024-04-25T10:22:58.431839+0000","last_fullsized":"2024-04-25T10:22:58.431839+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:17.476656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883782+0000","last_change":"2024-04-25T10:22:57.427432+0000","last_active":"2024-04-25T10:22:58.883782+0000","last_peered":"2024-04-25T10:22:58.883782+0000","last_clean":"2024-04-25T10:22:58.883782+0000","last_became_active":"2024-04-25T10:22:57.427153+0000","last_became_peered":"2024-04-25T10:22:57.427153+0000","last_unstale":"2024-04-25T10:22:58.883782+0000","last_undegraded":"2024-04-25T10:22:58.883782+0000","last_fullsized":"2024-04-25T10:22:58.883782+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:55.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883847+0000","last_change":"2024-04-25T10:22:56.404525+0000","last_active":"2024-04-25T10:22:58.883847+0000","last_peered":"2024-04-25T10:22:58.883847+0000","last_clean":"2024-04-25T10:22:58.883847+0000","last_became_active":"2024-04-25T10:22:56.404252+0000","last_became_peered":"2024-04-25T10:22:56.404252+0000","last_unstale":"2024-04-25T10:22:58.883847+0000","last_undegraded":"2024-04-25T10:22:58.883847+0000","last_fullsized":"2024-04-25T10:22:58.883847+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:59.730643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405124+0000","last_change":"2024-04-25T10:22:57.423863+0000","last_active":"2024-04-25T10:22:58.405124+0000","last_peered":"2024-04-25T10:22:58.405124+0000","last_clean":"2024-04-25T10:22:58.405124+0000","last_became_active":"2024-04-25T10:22:57.423623+0000","last_became_peered":"2024-04-25T10:22:57.423623+0000","last_unstale":"2024-04-25T10:22:58.405124+0000","last_undegraded":"2024-04-25T10:22:58.405124+0000","last_fullsized":"2024-04-25T10:22:58.405124+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:32.192334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407768+0000","last_change":"2024-04-25T10:22:56.401905+0000","last_active":"2024-04-25T10:22:58.407768+0000","last_peered":"2024-04-25T10:22:58.407768+0000","last_clean":"2024-04-25T10:22:58.407768+0000","last_became_active":"2024-04-25T10:22:56.401638+0000","last_became_peered":"2024-04-25T10:22:56.401638+0000","last_unstale":"2024-04-25T10:22:58.407768+0000","last_undegraded":"2024-04-25T10:22:58.407768+0000","last_fullsized":"2024-04-25T10:22:58.407768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.538199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431657+0000","last_change":"2024-04-25T10:22:57.428702+0000","last_active":"2024-04-25T10:22:58.431657+0000","last_peered":"2024-04-25T10:22:58.431657+0000","last_clean":"2024-04-25T10:22:58.431657+0000","last_became_active":"2024-04-25T10:22:57.427858+0000","last_became_peered":"2024-04-25T10:22:57.427858+0000","last_unstale":"2024-04-25T10:22:58.431657+0000","last_undegraded":"2024-04-25T10:22:58.431657+0000","last_fullsized":"2024-04-25T10:22:58.431657+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:58.791815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408053+0000","last_change":"2024-04-25T10:22:56.402508+0000","last_active":"2024-04-25T10:22:58.408053+0000","last_peered":"2024-04-25T10:22:58.408053+0000","last_clean":"2024-04-25T10:22:58.408053+0000","last_became_active":"2024-04-25T10:22:56.402249+0000","last_became_peered":"2024-04-25T10:22:56.402249+0000","last_unstale":"2024-04-25T10:22:58.408053+0000","last_undegraded":"2024-04-25T10:22:58.408053+0000","last_fullsized":"2024-04-25T10:22:58.408053+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:09.044675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883550+0000","last_change":"2024-04-25T10:22:57.416996+0000","last_active":"2024-04-25T10:22:58.883550+0000","last_peered":"2024-04-25T10:22:58.883550+0000","last_clean":"2024-04-25T10:22:58.883550+0000","last_became_active":"2024-04-25T10:22:57.416585+0000","last_became_peered":"2024-04-25T10:22:57.416585+0000","last_unstale":"2024-04-25T10:22:58.883550+0000","last_undegraded":"2024-04-25T10:22:58.883550+0000","last_fullsized":"2024-04-25T10:22:58.883550+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:49.433729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457227+0000","last_change":"2024-04-25T10:22:56.410126+0000","last_active":"2024-04-25T10:22:59.457227+0000","last_peered":"2024-04-25T10:22:59.457227+0000","last_clean":"2024-04-25T10:22:59.457227+0000","last_became_active":"2024-04-25T10:22:56.409634+0000","last_became_peered":"2024-04-25T10:22:56.409634+0000","last_unstale":"2024-04-25T10:22:59.457227+0000","last_undegraded":"2024-04-25T10:22:59.457227+0000","last_fullsized":"2024-04-25T10:22:59.457227+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:32.860372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407636+0000","last_change":"2024-04-25T10:22:57.426676+0000","last_active":"2024-04-25T10:22:58.407636+0000","last_peered":"2024-04-25T10:22:58.407636+0000","last_clean":"2024-04-25T10:22:58.407636+0000","last_became_active":"2024-04-25T10:22:57.425820+0000","last_became_peered":"2024-04-25T10:22:57.425820+0000","last_unstale":"2024-04-25T10:22:58.407636+0000","last_undegraded":"2024-04-25T10:22:58.407636+0000","last_fullsized":"2024-04-25T10:22:58.407636+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:49.146689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407718+0000","last_change":"2024-04-25T10:22:56.398019+0000","last_active":"2024-04-25T10:22:58.407718+0000","last_peered":"2024-04-25T10:22:58.407718+0000","last_clean":"2024-04-25T10:22:58.407718+0000","last_became_active":"2024-04-25T10:22:56.397728+0000","last_became_peered":"2024-04-25T10:22:56.397728+0000","last_unstale":"2024-04-25T10:22:58.407718+0000","last_undegraded":"2024-04-25T10:22:58.407718+0000","last_fullsized":"2024-04-25T10:22:58.407718+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:50.846601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883527+0000","last_change":"2024-04-25T10:22:57.408521+0000","last_active":"2024-04-25T10:22:58.883527+0000","last_peered":"2024-04-25T10:22:58.883527+0000","last_clean":"2024-04-25T10:22:58.883527+0000","last_became_active":"2024-04-25T10:22:57.408071+0000","last_became_peered":"2024-04-25T10:22:57.408071+0000","last_unstale":"2024-04-25T10:22:58.883527+0000","last_undegraded":"2024-04-25T10:22:58.883527+0000","last_fullsized":"2024-04-25T10:22:58.883527+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.285372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457691+0000","last_change":"2024-04-25T10:22:56.405829+0000","last_active":"2024-04-25T10:22:59.457691+0000","last_peered":"2024-04-25T10:22:59.457691+0000","last_clean":"2024-04-25T10:22:59.457691+0000","last_became_active":"2024-04-25T10:22:56.405574+0000","last_became_peered":"2024-04-25T10:22:56.405574+0000","last_unstale":"2024-04-25T10:22:59.457691+0000","last_undegraded":"2024-04-25T10:22:59.457691+0000","last_fullsized":"2024-04-25T10:22:59.457691+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:26.731280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407623+0000","last_change":"2024-04-25T10:22:57.423215+0000","last_active":"2024-04-25T10:22:58.407623+0000","last_peered":"2024-04-25T10:22:58.407623+0000","last_clean":"2024-04-25T10:22:58.407623+0000","last_became_active":"2024-04-25T10:22:57.419438+0000","last_became_peered":"2024-04-25T10:22:57.419438+0000","last_unstale":"2024-04-25T10:22:58.407623+0000","last_undegraded":"2024-04-25T10:22:58.407623+0000","last_fullsized":"2024-04-25T10:22:58.407623+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:56.225148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.406699+0000","last_change":"2024-04-25T10:22:56.400182+0000","last_active":"2024-04-25T10:22:58.406699+0000","last_peered":"2024-04-25T10:22:58.406699+0000","last_clean":"2024-04-25T10:22:58.406699+0000","last_became_active":"2024-04-25T10:22:56.399914+0000","last_became_peered":"2024-04-25T10:22:56.399914+0000","last_unstale":"2024-04-25T10:22:58.406699+0000","last_undegraded":"2024-04-25T10:22:58.406699+0000","last_fullsized":"2024-04-25T10:22:58.406699+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:30.846368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408335+0000","last_change":"2024-04-25T10:22:57.419455+0000","last_active":"2024-04-25T10:22:58.408335+0000","last_peered":"2024-04-25T10:22:58.408335+0000","last_clean":"2024-04-25T10:22:58.408335+0000","last_became_active":"2024-04-25T10:22:57.417451+0000","last_became_peered":"2024-04-25T10:22:57.417451+0000","last_unstale":"2024-04-25T10:22:58.408335+0000","last_undegraded":"2024-04-25T10:22:58.408335+0000","last_fullsized":"2024-04-25T10:22:58.408335+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:14.283194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883811+0000","last_change":"2024-04-25T10:22:56.404968+0000","last_active":"2024-04-25T10:22:58.883811+0000","last_peered":"2024-04-25T10:22:58.883811+0000","last_clean":"2024-04-25T10:22:58.883811+0000","last_became_active":"2024-04-25T10:22:56.404275+0000","last_became_peered":"2024-04-25T10:22:56.404275+0000","last_unstale":"2024-04-25T10:22:58.883811+0000","last_undegraded":"2024-04-25T10:22:58.883811+0000","last_fullsized":"2024-04-25T10:22:58.883811+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:52.403966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431961+0000","last_change":"2024-04-25T10:22:57.434286+0000","last_active":"2024-04-25T10:22:58.431961+0000","last_peered":"2024-04-25T10:22:58.431961+0000","last_clean":"2024-04-25T10:22:58.431961+0000","last_became_active":"2024-04-25T10:22:57.433877+0000","last_became_peered":"2024-04-25T10:22:57.433877+0000","last_unstale":"2024-04-25T10:22:58.431961+0000","last_undegraded":"2024-04-25T10:22:58.431961+0000","last_fullsized":"2024-04-25T10:22:58.431961+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.757444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.432009+0000","last_change":"2024-04-25T10:22:56.409876+0000","last_active":"2024-04-25T10:22:58.432009+0000","last_peered":"2024-04-25T10:22:58.432009+0000","last_clean":"2024-04-25T10:22:58.432009+0000","last_became_active":"2024-04-25T10:22:56.409594+0000","last_became_peered":"2024-04-25T10:22:56.409594+0000","last_unstale":"2024-04-25T10:22:58.432009+0000","last_undegraded":"2024-04-25T10:22:58.432009+0000","last_fullsized":"2024-04-25T10:22:58.432009+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:19.741870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457591+0000","last_change":"2024-04-25T10:22:57.428074+0000","last_active":"2024-04-25T10:22:59.457591+0000","last_peered":"2024-04-25T10:22:59.457591+0000","last_clean":"2024-04-25T10:22:59.457591+0000","last_became_active":"2024-04-25T10:22:57.427820+0000","last_became_peered":"2024-04-25T10:22:57.427820+0000","last_unstale":"2024-04-25T10:22:59.457591+0000","last_undegraded":"2024-04-25T10:22:59.457591+0000","last_fullsized":"2024-04-25T10:22:59.457591+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:19.664954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"137'6","reported_seq":20,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458360+0000","last_change":"2024-04-25T10:22:56.406465+0000","last_active":"2024-04-25T10:22:59.458360+0000","last_peered":"2024-04-25T10:22:59.458360+0000","last_clean":"2024-04-25T10:22:59.458360+0000","last_became_active":"2024-04-25T10:22:56.406172+0000","last_became_peered":"2024-04-25T10:22:56.406172+0000","last_unstale":"2024-04-25T10:22:59.458360+0000","last_undegraded":"2024-04-25T10:22:59.458360+0000","last_fullsized":"2024-04-25T10:22:59.458360+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:59.574617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459340+0000","last_change":"2024-04-25T10:22:57.429703+0000","last_active":"2024-04-25T10:22:59.459340+0000","last_peered":"2024-04-25T10:22:59.459340+0000","last_clean":"2024-04-25T10:22:59.459340+0000","last_became_active":"2024-04-25T10:22:57.429433+0000","last_became_peered":"2024-04-25T10:22:57.429433+0000","last_unstale":"2024-04-25T10:22:59.459340+0000","last_undegraded":"2024-04-25T10:22:59.459340+0000","last_fullsized":"2024-04-25T10:22:59.459340+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:48.183876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459294+0000","last_change":"2024-04-25T10:22:56.405202+0000","last_active":"2024-04-25T10:22:59.459294+0000","last_peered":"2024-04-25T10:22:59.459294+0000","last_clean":"2024-04-25T10:22:59.459294+0000","last_became_active":"2024-04-25T10:22:56.404825+0000","last_became_peered":"2024-04-25T10:22:56.404825+0000","last_unstale":"2024-04-25T10:22:59.459294+0000","last_undegraded":"2024-04-25T10:22:59.459294+0000","last_fullsized":"2024-04-25T10:22:59.459294+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:41.838590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883769+0000","last_change":"2024-04-25T10:22:57.427686+0000","last_active":"2024-04-25T10:22:58.883769+0000","last_peered":"2024-04-25T10:22:58.883769+0000","last_clean":"2024-04-25T10:22:58.883769+0000","last_became_active":"2024-04-25T10:22:57.427241+0000","last_became_peered":"2024-04-25T10:22:57.427241+0000","last_unstale":"2024-04-25T10:22:58.883769+0000","last_undegraded":"2024-04-25T10:22:58.883769+0000","last_fullsized":"2024-04-25T10:22:58.883769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:20.944185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459129+0000","last_change":"2024-04-25T10:22:56.408162+0000","last_active":"2024-04-25T10:22:59.459129+0000","last_peered":"2024-04-25T10:22:59.459129+0000","last_clean":"2024-04-25T10:22:59.459129+0000","last_became_active":"2024-04-25T10:22:56.407814+0000","last_became_peered":"2024-04-25T10:22:56.407814+0000","last_unstale":"2024-04-25T10:22:59.459129+0000","last_undegraded":"2024-04-25T10:22:59.459129+0000","last_fullsized":"2024-04-25T10:22:59.459129+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:30.846368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407047+0000","last_change":"2024-04-25T10:22:57.422397+0000","last_active":"2024-04-25T10:22:58.407047+0000","last_peered":"2024-04-25T10:22:58.407047+0000","last_clean":"2024-04-25T10:22:58.407047+0000","last_became_active":"2024-04-25T10:22:57.421862+0000","last_became_peered":"2024-04-25T10:22:57.421862+0000","last_unstale":"2024-04-25T10:22:58.407047+0000","last_undegraded":"2024-04-25T10:22:58.407047+0000","last_fullsized":"2024-04-25T10:22:58.407047+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:24.867491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404659+0000","last_change":"2024-04-25T10:22:56.401669+0000","last_active":"2024-04-25T10:22:58.404659+0000","last_peered":"2024-04-25T10:22:58.404659+0000","last_clean":"2024-04-25T10:22:58.404659+0000","last_became_active":"2024-04-25T10:22:56.401377+0000","last_became_peered":"2024-04-25T10:22:56.401377+0000","last_unstale":"2024-04-25T10:22:58.404659+0000","last_undegraded":"2024-04-25T10:22:58.404659+0000","last_fullsized":"2024-04-25T10:22:58.404659+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:50.846601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459123+0000","last_change":"2024-04-25T10:22:57.415605+0000","last_active":"2024-04-25T10:22:59.459123+0000","last_peered":"2024-04-25T10:22:59.459123+0000","last_clean":"2024-04-25T10:22:59.459123+0000","last_became_active":"2024-04-25T10:22:57.415044+0000","last_became_peered":"2024-04-25T10:22:57.415044+0000","last_unstale":"2024-04-25T10:22:59.459123+0000","last_undegraded":"2024-04-25T10:22:59.459123+0000","last_fullsized":"2024-04-25T10:22:59.459123+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:58.395291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"137'8","reported_seq":22,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458009+0000","last_change":"2024-04-25T10:22:56.409221+0000","last_active":"2024-04-25T10:22:59.458009+0000","last_peered":"2024-04-25T10:22:59.458009+0000","last_clean":"2024-04-25T10:22:59.458009+0000","last_became_active":"2024-04-25T10:22:56.408831+0000","last_became_peered":"2024-04-25T10:22:56.408831+0000","last_unstale":"2024-04-25T10:22:59.458009+0000","last_undegraded":"2024-04-25T10:22:59.458009+0000","last_fullsized":"2024-04-25T10:22:59.458009+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:05.527321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431950+0000","last_change":"2024-04-25T10:22:57.437169+0000","last_active":"2024-04-25T10:22:58.431950+0000","last_peered":"2024-04-25T10:22:58.431950+0000","last_clean":"2024-04-25T10:22:58.431950+0000","last_became_active":"2024-04-25T10:22:57.436865+0000","last_became_peered":"2024-04-25T10:22:57.436865+0000","last_unstale":"2024-04-25T10:22:58.431950+0000","last_undegraded":"2024-04-25T10:22:58.431950+0000","last_fullsized":"2024-04-25T10:22:58.431950+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:34.662479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408258+0000","last_change":"2024-04-25T10:22:56.406651+0000","last_active":"2024-04-25T10:22:58.408258+0000","last_peered":"2024-04-25T10:22:58.408258+0000","last_clean":"2024-04-25T10:22:58.408258+0000","last_became_active":"2024-04-25T10:22:56.406261+0000","last_became_peered":"2024-04-25T10:22:56.406261+0000","last_unstale":"2024-04-25T10:22:58.408258+0000","last_undegraded":"2024-04-25T10:22:58.408258+0000","last_fullsized":"2024-04-25T10:22:58.408258+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:42.506500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457645+0000","last_change":"2024-04-25T10:22:57.422166+0000","last_active":"2024-04-25T10:22:59.457645+0000","last_peered":"2024-04-25T10:22:59.457645+0000","last_clean":"2024-04-25T10:22:59.457645+0000","last_became_active":"2024-04-25T10:22:57.421066+0000","last_became_peered":"2024-04-25T10:22:57.421066+0000","last_unstale":"2024-04-25T10:22:59.457645+0000","last_undegraded":"2024-04-25T10:22:59.457645+0000","last_fullsized":"2024-04-25T10:22:59.457645+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:23.587240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883953+0000","last_change":"2024-04-25T10:22:56.409097+0000","last_active":"2024-04-25T10:22:58.883953+0000","last_peered":"2024-04-25T10:22:58.883953+0000","last_clean":"2024-04-25T10:22:58.883953+0000","last_became_active":"2024-04-25T10:22:56.408810+0000","last_became_peered":"2024-04-25T10:22:56.408810+0000","last_unstale":"2024-04-25T10:22:58.883953+0000","last_undegraded":"2024-04-25T10:22:58.883953+0000","last_fullsized":"2024-04-25T10:22:58.883953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:48.139177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459379+0000","last_change":"2024-04-25T10:22:57.427658+0000","last_active":"2024-04-25T10:22:59.459379+0000","last_peered":"2024-04-25T10:22:59.459379+0000","last_clean":"2024-04-25T10:22:59.459379+0000","last_became_active":"2024-04-25T10:22:57.426492+0000","last_became_peered":"2024-04-25T10:22:57.426492+0000","last_unstale":"2024-04-25T10:22:59.459379+0000","last_undegraded":"2024-04-25T10:22:59.459379+0000","last_fullsized":"2024-04-25T10:22:59.459379+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:19.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457760+0000","last_change":"2024-04-25T10:22:56.405058+0000","last_active":"2024-04-25T10:22:59.457760+0000","last_peered":"2024-04-25T10:22:59.457760+0000","last_clean":"2024-04-25T10:22:59.457760+0000","last_became_active":"2024-04-25T10:22:56.404793+0000","last_became_peered":"2024-04-25T10:22:56.404793+0000","last_unstale":"2024-04-25T10:22:59.457760+0000","last_undegraded":"2024-04-25T10:22:59.457760+0000","last_fullsized":"2024-04-25T10:22:59.457760+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.958679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405171+0000","last_change":"2024-04-25T10:22:57.429135+0000","last_active":"2024-04-25T10:22:58.405171+0000","last_peered":"2024-04-25T10:22:58.405171+0000","last_clean":"2024-04-25T10:22:58.405171+0000","last_became_active":"2024-04-25T10:22:57.428826+0000","last_became_peered":"2024-04-25T10:22:57.428826+0000","last_unstale":"2024-04-25T10:22:58.405171+0000","last_undegraded":"2024-04-25T10:22:58.405171+0000","last_fullsized":"2024-04-25T10:22:58.405171+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:41.917144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883587+0000","last_change":"2024-04-25T10:22:56.405467+0000","last_active":"2024-04-25T10:22:58.883587+0000","last_peered":"2024-04-25T10:22:58.883587+0000","last_clean":"2024-04-25T10:22:58.883587+0000","last_became_active":"2024-04-25T10:22:56.405200+0000","last_became_peered":"2024-04-25T10:22:56.405200+0000","last_unstale":"2024-04-25T10:22:58.883587+0000","last_undegraded":"2024-04-25T10:22:58.883587+0000","last_fullsized":"2024-04-25T10:22:58.883587+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.022213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407453+0000","last_change":"2024-04-25T10:22:57.424819+0000","last_active":"2024-04-25T10:22:58.407453+0000","last_peered":"2024-04-25T10:22:58.407453+0000","last_clean":"2024-04-25T10:22:58.407453+0000","last_became_active":"2024-04-25T10:22:57.423557+0000","last_became_peered":"2024-04-25T10:22:57.423557+0000","last_unstale":"2024-04-25T10:22:58.407453+0000","last_undegraded":"2024-04-25T10:22:58.407453+0000","last_fullsized":"2024-04-25T10:22:58.407453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:00.738155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458318+0000","last_change":"2024-04-25T10:22:56.403915+0000","last_active":"2024-04-25T10:22:59.458318+0000","last_peered":"2024-04-25T10:22:59.458318+0000","last_clean":"2024-04-25T10:22:59.458318+0000","last_became_active":"2024-04-25T10:22:56.403656+0000","last_became_peered":"2024-04-25T10:22:56.403656+0000","last_unstale":"2024-04-25T10:22:59.458318+0000","last_undegraded":"2024-04-25T10:22:59.458318+0000","last_fullsized":"2024-04-25T10:22:59.458318+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.742849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458362+0000","last_change":"2024-04-25T10:22:57.426609+0000","last_active":"2024-04-25T10:22:59.458362+0000","last_peered":"2024-04-25T10:22:59.458362+0000","last_clean":"2024-04-25T10:22:59.458362+0000","last_became_active":"2024-04-25T10:22:57.426326+0000","last_became_peered":"2024-04-25T10:22:57.426326+0000","last_unstale":"2024-04-25T10:22:59.458362+0000","last_undegraded":"2024-04-25T10:22:59.458362+0000","last_fullsized":"2024-04-25T10:22:59.458362+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:41.758097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431286+0000","last_change":"2024-04-25T10:22:56.406936+0000","last_active":"2024-04-25T10:22:58.431286+0000","last_peered":"2024-04-25T10:22:58.431286+0000","last_clean":"2024-04-25T10:22:58.431286+0000","last_became_active":"2024-04-25T10:22:56.406321+0000","last_became_peered":"2024-04-25T10:22:56.406321+0000","last_unstale":"2024-04-25T10:22:58.431286+0000","last_undegraded":"2024-04-25T10:22:58.431286+0000","last_fullsized":"2024-04-25T10:22:58.431286+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.406774+0000","last_change":"2024-04-25T10:22:57.426142+0000","last_active":"2024-04-25T10:22:58.406774+0000","last_peered":"2024-04-25T10:22:58.406774+0000","last_clean":"2024-04-25T10:22:58.406774+0000","last_became_active":"2024-04-25T10:22:57.425790+0000","last_became_peered":"2024-04-25T10:22:57.425790+0000","last_unstale":"2024-04-25T10:22:58.406774+0000","last_undegraded":"2024-04-25T10:22:58.406774+0000","last_fullsized":"2024-04-25T10:22:58.406774+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:19.500478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431185+0000","last_change":"2024-04-25T10:22:56.411953+0000","last_active":"2024-04-25T10:22:58.431185+0000","last_peered":"2024-04-25T10:22:58.431185+0000","last_clean":"2024-04-25T10:22:58.431185+0000","last_became_active":"2024-04-25T10:22:56.411594+0000","last_became_peered":"2024-04-25T10:22:56.411594+0000","last_unstale":"2024-04-25T10:22:58.431185+0000","last_undegraded":"2024-04-25T10:22:58.431185+0000","last_fullsized":"2024-04-25T10:22:58.431185+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:59.574617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457923+0000","last_change":"2024-04-25T10:22:57.425323+0000","last_active":"2024-04-25T10:22:59.457923+0000","last_peered":"2024-04-25T10:22:59.457923+0000","last_clean":"2024-04-25T10:22:59.457923+0000","last_became_active":"2024-04-25T10:22:57.425030+0000","last_became_peered":"2024-04-25T10:22:57.425030+0000","last_unstale":"2024-04-25T10:22:59.457923+0000","last_undegraded":"2024-04-25T10:22:59.457923+0000","last_fullsized":"2024-04-25T10:22:59.457923+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:20.808204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458912+0000","last_change":"2024-04-25T10:22:56.412818+0000","last_active":"2024-04-25T10:22:59.458912+0000","last_peered":"2024-04-25T10:22:59.458912+0000","last_clean":"2024-04-25T10:22:59.458912+0000","last_became_active":"2024-04-25T10:22:56.412466+0000","last_became_peered":"2024-04-25T10:22:56.412466+0000","last_unstale":"2024-04-25T10:22:59.458912+0000","last_undegraded":"2024-04-25T10:22:59.458912+0000","last_fullsized":"2024-04-25T10:22:59.458912+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:31.184822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408388+0000","last_change":"2024-04-25T10:22:57.429051+0000","last_active":"2024-04-25T10:22:58.408388+0000","last_peered":"2024-04-25T10:22:58.408388+0000","last_clean":"2024-04-25T10:22:58.408388+0000","last_became_active":"2024-04-25T10:22:57.428603+0000","last_became_peered":"2024-04-25T10:22:57.428603+0000","last_unstale":"2024-04-25T10:22:58.408388+0000","last_undegraded":"2024-04-25T10:22:58.408388+0000","last_fullsized":"2024-04-25T10:22:58.408388+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:10.594590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.432015+0000","last_change":"2024-04-25T10:22:56.410115+0000","last_active":"2024-04-25T10:22:58.432015+0000","last_peered":"2024-04-25T10:22:58.432015+0000","last_clean":"2024-04-25T10:22:58.432015+0000","last_became_active":"2024-04-25T10:22:56.409783+0000","last_became_peered":"2024-04-25T10:22:56.409783+0000","last_unstale":"2024-04-25T10:22:58.432015+0000","last_undegraded":"2024-04-25T10:22:58.432015+0000","last_fullsized":"2024-04-25T10:22:58.432015+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:43.989532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459666+0000","last_change":"2024-04-25T10:22:57.429612+0000","last_active":"2024-04-25T10:22:59.459666+0000","last_peered":"2024-04-25T10:22:59.459666+0000","last_clean":"2024-04-25T10:22:59.459666+0000","last_became_active":"2024-04-25T10:22:57.429307+0000","last_became_peered":"2024-04-25T10:22:57.429307+0000","last_unstale":"2024-04-25T10:22:59.459666+0000","last_undegraded":"2024-04-25T10:22:59.459666+0000","last_fullsized":"2024-04-25T10:22:59.459666+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:58.202829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"138'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405218+0000","last_change":"2024-04-25T10:22:56.401059+0000","last_active":"2024-04-25T10:22:58.405218+0000","last_peered":"2024-04-25T10:22:58.405218+0000","last_clean":"2024-04-25T10:22:58.405218+0000","last_became_active":"2024-04-25T10:22:56.400831+0000","last_became_peered":"2024-04-25T10:22:56.400831+0000","last_unstale":"2024-04-25T10:22:58.405218+0000","last_undegraded":"2024-04-25T10:22:58.405218+0000","last_fullsized":"2024-04-25T10:22:58.405218+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:05.933699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883635+0000","last_change":"2024-04-25T10:22:57.408929+0000","last_active":"2024-04-25T10:22:58.883635+0000","last_peered":"2024-04-25T10:22:58.883635+0000","last_clean":"2024-04-25T10:22:58.883635+0000","last_became_active":"2024-04-25T10:22:57.408636+0000","last_became_peered":"2024-04-25T10:22:57.408636+0000","last_unstale":"2024-04-25T10:22:58.883635+0000","last_undegraded":"2024-04-25T10:22:58.883635+0000","last_fullsized":"2024-04-25T10:22:58.883635+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:41.758097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405295+0000","last_change":"2024-04-25T10:22:56.406043+0000","last_active":"2024-04-25T10:22:58.405295+0000","last_peered":"2024-04-25T10:22:58.405295+0000","last_clean":"2024-04-25T10:22:58.405295+0000","last_became_active":"2024-04-25T10:22:56.405764+0000","last_became_peered":"2024-04-25T10:22:56.405764+0000","last_unstale":"2024-04-25T10:22:58.405295+0000","last_undegraded":"2024-04-25T10:22:58.405295+0000","last_fullsized":"2024-04-25T10:22:58.405295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:17.228857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405240+0000","last_change":"2024-04-25T10:22:57.426198+0000","last_active":"2024-04-25T10:22:58.405240+0000","last_peered":"2024-04-25T10:22:58.405240+0000","last_clean":"2024-04-25T10:22:58.405240+0000","last_became_active":"2024-04-25T10:22:57.425603+0000","last_became_peered":"2024-04-25T10:22:57.425603+0000","last_unstale":"2024-04-25T10:22:58.405240+0000","last_undegraded":"2024-04-25T10:22:58.405240+0000","last_fullsized":"2024-04-25T10:22:58.405240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:47.261911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404991+0000","last_change":"2024-04-25T10:22:56.406395+0000","last_active":"2024-04-25T10:22:58.404991+0000","last_peered":"2024-04-25T10:22:58.404991+0000","last_clean":"2024-04-25T10:22:58.404991+0000","last_became_active":"2024-04-25T10:22:56.405805+0000","last_became_peered":"2024-04-25T10:22:56.405805+0000","last_unstale":"2024-04-25T10:22:58.404991+0000","last_undegraded":"2024-04-25T10:22:58.404991+0000","last_fullsized":"2024-04-25T10:22:58.404991+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:42.506500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457447+0000","last_change":"2024-04-25T10:22:57.427188+0000","last_active":"2024-04-25T10:22:59.457447+0000","last_peered":"2024-04-25T10:22:59.457447+0000","last_clean":"2024-04-25T10:22:59.457447+0000","last_became_active":"2024-04-25T10:22:57.426833+0000","last_became_peered":"2024-04-25T10:22:57.426833+0000","last_unstale":"2024-04-25T10:22:59.457447+0000","last_undegraded":"2024-04-25T10:22:59.457447+0000","last_fullsized":"2024-04-25T10:22:59.457447+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:07.034942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458411+0000","last_change":"2024-04-25T10:22:56.399245+0000","last_active":"2024-04-25T10:22:59.458411+0000","last_peered":"2024-04-25T10:22:59.458411+0000","last_clean":"2024-04-25T10:22:59.458411+0000","last_became_active":"2024-04-25T10:22:56.398919+0000","last_became_peered":"2024-04-25T10:22:56.398919+0000","last_unstale":"2024-04-25T10:22:59.458411+0000","last_undegraded":"2024-04-25T10:22:59.458411+0000","last_fullsized":"2024-04-25T10:22:59.458411+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407680+0000","last_change":"2024-04-25T10:22:57.419140+0000","last_active":"2024-04-25T10:22:58.407680+0000","last_peered":"2024-04-25T10:22:58.407680+0000","last_clean":"2024-04-25T10:22:58.407680+0000","last_became_active":"2024-04-25T10:22:57.418497+0000","last_became_peered":"2024-04-25T10:22:57.418497+0000","last_unstale":"2024-04-25T10:22:58.407680+0000","last_undegraded":"2024-04-25T10:22:58.407680+0000","last_fullsized":"2024-04-25T10:22:58.407680+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:38.984827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431185+0000","last_change":"2024-04-25T10:22:56.407652+0000","last_active":"2024-04-25T10:22:58.431185+0000","last_peered":"2024-04-25T10:22:58.431185+0000","last_clean":"2024-04-25T10:22:58.431185+0000","last_became_active":"2024-04-25T10:22:56.407372+0000","last_became_peered":"2024-04-25T10:22:56.407372+0000","last_unstale":"2024-04-25T10:22:58.431185+0000","last_undegraded":"2024-04-25T10:22:58.431185+0000","last_fullsized":"2024-04-25T10:22:58.431185+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.742849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431325+0000","last_change":"2024-04-25T10:22:57.422870+0000","last_active":"2024-04-25T10:22:58.431325+0000","last_peered":"2024-04-25T10:22:58.431325+0000","last_clean":"2024-04-25T10:22:58.431325+0000","last_became_active":"2024-04-25T10:22:57.422302+0000","last_became_peered":"2024-04-25T10:22:57.422302+0000","last_unstale":"2024-04-25T10:22:58.431325+0000","last_undegraded":"2024-04-25T10:22:58.431325+0000","last_fullsized":"2024-04-25T10:22:58.431325+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.333103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459031+0000","last_change":"2024-04-25T10:22:56.404329+0000","last_active":"2024-04-25T10:22:59.459031+0000","last_peered":"2024-04-25T10:22:59.459031+0000","last_clean":"2024-04-25T10:22:59.459031+0000","last_became_active":"2024-04-25T10:22:56.404004+0000","last_became_peered":"2024-04-25T10:22:56.404004+0000","last_unstale":"2024-04-25T10:22:59.459031+0000","last_undegraded":"2024-04-25T10:22:59.459031+0000","last_fullsized":"2024-04-25T10:22:59.459031+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:40.909632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407435+0000","last_change":"2024-04-25T10:22:57.416491+0000","last_active":"2024-04-25T10:22:58.407435+0000","last_peered":"2024-04-25T10:22:58.407435+0000","last_clean":"2024-04-25T10:22:58.407435+0000","last_became_active":"2024-04-25T10:22:57.416144+0000","last_became_peered":"2024-04-25T10:22:57.416144+0000","last_unstale":"2024-04-25T10:22:58.407435+0000","last_undegraded":"2024-04-25T10:22:58.407435+0000","last_fullsized":"2024-04-25T10:22:58.407435+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:59.062309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883663+0000","last_change":"2024-04-25T10:22:56.405780+0000","last_active":"2024-04-25T10:22:58.883663+0000","last_peered":"2024-04-25T10:22:58.883663+0000","last_clean":"2024-04-25T10:22:58.883663+0000","last_became_active":"2024-04-25T10:22:56.405542+0000","last_became_peered":"2024-04-25T10:22:56.405542+0000","last_unstale":"2024-04-25T10:22:58.883663+0000","last_undegraded":"2024-04-25T10:22:58.883663+0000","last_fullsized":"2024-04-25T10:22:58.883663+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:49.722812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458284+0000","last_change":"2024-04-25T10:22:57.425724+0000","last_active":"2024-04-25T10:22:59.458284+0000","last_peered":"2024-04-25T10:22:59.458284+0000","last_clean":"2024-04-25T10:22:59.458284+0000","last_became_active":"2024-04-25T10:22:57.425440+0000","last_became_peered":"2024-04-25T10:22:57.425440+0000","last_unstale":"2024-04-25T10:22:59.458284+0000","last_undegraded":"2024-04-25T10:22:59.458284+0000","last_fullsized":"2024-04-25T10:22:59.458284+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:49.433729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"137'3","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459497+0000","last_change":"2024-04-25T10:22:56.409604+0000","last_active":"2024-04-25T10:22:59.459497+0000","last_peered":"2024-04-25T10:22:59.459497+0000","last_clean":"2024-04-25T10:22:59.459497+0000","last_became_active":"2024-04-25T10:22:56.409342+0000","last_became_peered":"2024-04-25T10:22:56.409342+0000","last_unstale":"2024-04-25T10:22:59.459497+0000","last_undegraded":"2024-04-25T10:22:59.459497+0000","last_fullsized":"2024-04-25T10:22:59.459497+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:46.254399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":383,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431393+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:22:58.431393+0000","last_peered":"2024-04-25T10:22:58.431393+0000","last_clean":"2024-04-25T10:22:58.431393+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:22:58.431393+0000","last_undegraded":"2024-04-25T10:22:58.431393+0000","last_fullsized":"2024-04-25T10:22:58.431393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2330624,"data_stored":1347607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81392,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290448,"statfs":{"total":96636764160,"available":96553418752,"internally_reserved":0,"allocated":1740800,"data_stored":756733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81590726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50920,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320920,"statfs":{"total":96636764160,"available":96584622080,"internally_reserved":0,"allocated":1732608,"data_stored":756609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":1728512,"data_stored":756587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55088,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316752,"statfs":{"total":96636764160,"available":96580354048,"internally_reserved":0,"allocated":1732608,"data_stored":756677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50932,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320908,"statfs":{"total":96636764160,"available":96584609792,"internally_reserved":0,"allocated":1744896,"data_stored":758269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64816,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307024,"statfs":{"total":96636764160,"available":96570392576,"internally_reserved":0,"allocated":1732608,"data_stored":757169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55672,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316168,"statfs":{"total":96636764160,"available":96579756032,"internally_reserved":0,"allocated":2338816,"data_stored":1348601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:23:01.471 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:23:01.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 Processor -- start 2024-04-25T10:23:01.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 -- start start 2024-04-25T10:23:01.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f050c0b5010 0x7f050c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f050c06b2c0 0x7f050c06b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050c06bc70 con 0x7f050c0b5010 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050c15e460 con 0x7f050c06b2c0 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0513200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0b5780 con 0x7f050c15db40 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f050c0b5010 0x7f050c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f050c0b5010 0x7f050c15df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52138/0 (socket says 172.21.15.87:52138) 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 -- 172.21.15.87:0/220387078 learned_addr learned my addr 172.21.15.87:0/220387078 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0512800640 1 --2- 172.21.15.87:0/220387078 >> v2:172.21.15.169:3300/0 conn(0x7f050c06b2c0 0x7f050c06b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511e00640 1 --2- 172.21.15.87:0/220387078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 -- 172.21.15.87:0/220387078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 msgr2=0x7f050c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 --2- 172.21.15.87:0/220387078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 -- 172.21.15.87:0/220387078 >> v2:172.21.15.169:3300/0 conn(0x7f050c06b2c0 msgr2=0x7f050c06b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 --2- 172.21.15.87:0/220387078 >> v2:172.21.15.169:3300/0 conn(0x7f050c06b2c0 0x7f050c06b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511400640 1 -- 172.21.15.87:0/220387078 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c0b5a60 con 0x7f050c0b5010 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0512800640 1 --2- 172.21.15.87:0/220387078 >> v2:172.21.15.169:3300/0 conn(0x7f050c06b2c0 0x7f050c06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:01.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.641+0000 7f0511e00640 1 --2- 172.21.15.87:0/220387078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:01.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0511400640 1 --2- 172.21.15.87:0/220387078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15df20 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f050c06dbd0 tx=0x7f04fc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570963ae76ba7bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:01.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0510a00640 1 -- 172.21.15.87:0/220387078 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0b5070 con 0x7f050c0b5010 2024-04-25T10:23:01.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0510a00640 1 -- 172.21.15.87:0/220387078 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc0a89d0 con 0x7f050c0b5010 2024-04-25T10:23:01.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0510a00640 1 -- 172.21.15.87:0/220387078 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0a8da0 con 0x7f050c0b5010 2024-04-25T10:23:01.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 -- 172.21.15.87:0/220387078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 msgr2=0x7f050c15df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 --2- 172.21.15.87:0/220387078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15df20 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f050c06dbd0 tx=0x7f04fc01ed10 comp rx=0 tx=0).stop 2024-04-25T10:23:01.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 -- 172.21.15.87:0/220387078 shutdown_connections 2024-04-25T10:23:01.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 --2- 172.21.15.87:0/220387078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15df20 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 --2- 172.21.15.87:0/220387078 >> v2:172.21.15.169:3300/0 conn(0x7f050c06b2c0 0x7f050c06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 --2- 172.21.15.87:0/220387078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 -- 172.21.15.87:0/220387078 >> 172.21.15.87:0/220387078 conn(0x7f050c0d17b0 msgr2=0x7f050c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:01.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 -- 172.21.15.87:0/220387078 shutdown_connections 2024-04-25T10:23:01.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.645+0000 7f0513200640 1 -- 172.21.15.87:0/220387078 wait complete. 2024-04-25T10:23:01.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 Processor -- start 2024-04-25T10:23:01.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 -- start start 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06b2c0 0x7f050c1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c09b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0b6300 con 0x7f050c0b5010 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0b6020 con 0x7f050c06b2c0 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0b5450 con 0x7f050c15db40 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52154/0 (socket says 172.21.15.87:52154) 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 -- 172.21.15.87:0/3961070946 learned_addr learned my addr 172.21.15.87:0/3961070946 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511e00640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06b2c0 0x7f050c1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0512800640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c09b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 -- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 msgr2=0x7f050c09b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c09b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 -- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06b2c0 msgr2=0x7f050c1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06b2c0 0x7f050c1f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 -- 172.21.15.87:0/3961070946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c15bba0 con 0x7f050c0b5010 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0511400640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15b660 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f04fc01d3c0 tx=0x7f04fc01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0506000640 1 -- 172.21.15.87:0/3961070946 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0b5070 con 0x7f050c0b5010 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0506000640 1 -- 172.21.15.87:0/3961070946 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc0b0ba0 con 0x7f050c0b5010 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0506000640 1 -- 172.21.15.87:0/3961070946 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0bb410 con 0x7f050c0b5010 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c0cc940 con 0x7f050c0b5010 2024-04-25T10:23:01.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.649+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c0ccf00 con 0x7f050c0b5010 2024-04-25T10:23:01.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.653+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c075120 con 0x7f050c0b5010 2024-04-25T10:23:01.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.653+0000 7f0506000640 1 -- 172.21.15.87:0/3961070946 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04fc0c2060 con 0x7f050c0b5010 2024-04-25T10:23:01.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.653+0000 7f0506000640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f0066f90 0x7f04f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:01.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.653+0000 7f0506000640 1 -- 172.21.15.87:0/3961070946 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04fc1221d0 con 0x7f050c0b5010 2024-04-25T10:23:01.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.653+0000 7f0511e00640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f0066f90 0x7f04f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:01.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.653+0000 7f0511e00640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f0066f90 0x7f04f0069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f050c1f4850 tx=0x7f0500008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:01.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.657+0000 7f0506000640 1 -- 172.21.15.87:0/3961070946 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f050c15bba0 con 0x7f050c0b5010 2024-04-25T10:23:01.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.805+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f050c0cc6f0 con 0x7f04f0066f90 2024-04-25T10:23:01.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.813+0000 7f0506000640 1 -- 172.21.15.87:0/3961070946 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7f050c0cc6f0 con 0x7f04f0066f90 2024-04-25T10:23:01.819 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:01.821 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:23:01.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f0066f90 msgr2=0x7f04f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f0066f90 0x7f04f0069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f050c1f4850 tx=0x7f0500008040 comp rx=0 tx=0).stop 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 msgr2=0x7f050c15b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15b660 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f04fc01d3c0 tx=0x7f04fc01bab0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 shutdown_connections 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f04f0066f90 0x7f04f0069450 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f050c15db40 0x7f050c09b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f050c0b5010 0x7f050c15b660 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 --2- 172.21.15.87:0/3961070946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06b2c0 0x7f050c1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 >> 172.21.15.87:0/3961070946 conn(0x7f050c0d17b0 msgr2=0x7f050c0c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 shutdown_connections 2024-04-25T10:23:01.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:01.817+0000 7f0513200640 1 -- 172.21.15.87:0/3961070946 wait complete. 2024-04-25T10:23:01.838 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-04-25T10:23:01.259779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":477880,"kb_used_data":14728,"kb_used_omap":12,"kb_used_meta":463027,"kb_avail":754496840,"statfs":{"total":773094113280,"available":772604764160,"internally_reserved":0,"allocated":15081472,"data_stored":7238252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":474140209},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.877800"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405405+0000","last_change":"2024-04-25T10:22:57.428256+0000","last_active":"2024-04-25T10:22:58.405405+0000","last_peered":"2024-04-25T10:22:58.405405+0000","last_clean":"2024-04-25T10:22:58.405405+0000","last_became_active":"2024-04-25T10:22:57.428007+0000","last_became_peered":"2024-04-25T10:22:57.428007+0000","last_unstale":"2024-04-25T10:22:58.405405+0000","last_undegraded":"2024-04-25T10:22:58.405405+0000","last_fullsized":"2024-04-25T10:22:58.405405+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:31.853880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883926+0000","last_change":"2024-04-25T10:22:57.424019+0000","last_active":"2024-04-25T10:22:58.883926+0000","last_peered":"2024-04-25T10:22:58.883926+0000","last_clean":"2024-04-25T10:22:58.883926+0000","last_became_active":"2024-04-25T10:22:57.423735+0000","last_became_peered":"2024-04-25T10:22:57.423735+0000","last_unstale":"2024-04-25T10:22:58.883926+0000","last_undegraded":"2024-04-25T10:22:58.883926+0000","last_fullsized":"2024-04-25T10:22:58.883926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.139463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459563+0000","last_change":"2024-04-25T10:22:57.425360+0000","last_active":"2024-04-25T10:22:59.459563+0000","last_peered":"2024-04-25T10:22:59.459563+0000","last_clean":"2024-04-25T10:22:59.459563+0000","last_became_active":"2024-04-25T10:22:57.424969+0000","last_became_peered":"2024-04-25T10:22:57.424969+0000","last_unstale":"2024-04-25T10:22:59.459563+0000","last_undegraded":"2024-04-25T10:22:59.459563+0000","last_fullsized":"2024-04-25T10:22:59.459563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:43.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408119+0000","last_change":"2024-04-25T10:22:57.417290+0000","last_active":"2024-04-25T10:22:58.408119+0000","last_peered":"2024-04-25T10:22:58.408119+0000","last_clean":"2024-04-25T10:22:58.408119+0000","last_became_active":"2024-04-25T10:22:57.415224+0000","last_became_peered":"2024-04-25T10:22:57.415224+0000","last_unstale":"2024-04-25T10:22:58.408119+0000","last_undegraded":"2024-04-25T10:22:58.408119+0000","last_fullsized":"2024-04-25T10:22:58.408119+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:53.411478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407422+0000","last_change":"2024-04-25T10:22:57.422694+0000","last_active":"2024-04-25T10:22:58.407422+0000","last_peered":"2024-04-25T10:22:58.407422+0000","last_clean":"2024-04-25T10:22:58.407422+0000","last_became_active":"2024-04-25T10:22:57.422381+0000","last_became_peered":"2024-04-25T10:22:57.422381+0000","last_unstale":"2024-04-25T10:22:58.407422+0000","last_undegraded":"2024-04-25T10:22:58.407422+0000","last_fullsized":"2024-04-25T10:22:58.407422+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:01.498118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404783+0000","last_change":"2024-04-25T10:22:57.425599+0000","last_active":"2024-04-25T10:22:58.404783+0000","last_peered":"2024-04-25T10:22:58.404783+0000","last_clean":"2024-04-25T10:22:58.404783+0000","last_became_active":"2024-04-25T10:22:57.425204+0000","last_became_peered":"2024-04-25T10:22:57.425204+0000","last_unstale":"2024-04-25T10:22:58.404783+0000","last_undegraded":"2024-04-25T10:22:58.404783+0000","last_fullsized":"2024-04-25T10:22:58.404783+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:10.594590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458017+0000","last_change":"2024-04-25T10:22:57.425112+0000","last_active":"2024-04-25T10:22:59.458017+0000","last_peered":"2024-04-25T10:22:59.458017+0000","last_clean":"2024-04-25T10:22:59.458017+0000","last_became_active":"2024-04-25T10:22:57.424860+0000","last_became_peered":"2024-04-25T10:22:57.424860+0000","last_unstale":"2024-04-25T10:22:59.458017+0000","last_undegraded":"2024-04-25T10:22:59.458017+0000","last_fullsized":"2024-04-25T10:22:59.458017+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.285372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404623+0000","last_change":"2024-04-25T10:22:57.419844+0000","last_active":"2024-04-25T10:22:58.404623+0000","last_peered":"2024-04-25T10:22:58.404623+0000","last_clean":"2024-04-25T10:22:58.404623+0000","last_became_active":"2024-04-25T10:22:57.419551+0000","last_became_peered":"2024-04-25T10:22:57.419551+0000","last_unstale":"2024-04-25T10:22:58.404623+0000","last_undegraded":"2024-04-25T10:22:58.404623+0000","last_fullsized":"2024-04-25T10:22:58.404623+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:14.283194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883753+0000","last_change":"2024-04-25T10:22:57.425142+0000","last_active":"2024-04-25T10:22:58.883753+0000","last_peered":"2024-04-25T10:22:58.883753+0000","last_clean":"2024-04-25T10:22:58.883753+0000","last_became_active":"2024-04-25T10:22:57.424896+0000","last_became_peered":"2024-04-25T10:22:57.424896+0000","last_unstale":"2024-04-25T10:22:58.883753+0000","last_undegraded":"2024-04-25T10:22:58.883753+0000","last_fullsized":"2024-04-25T10:22:58.883753+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:03.598286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431773+0000","last_change":"2024-04-25T10:22:57.435334+0000","last_active":"2024-04-25T10:22:58.431773+0000","last_peered":"2024-04-25T10:22:58.431773+0000","last_clean":"2024-04-25T10:22:58.431773+0000","last_became_active":"2024-04-25T10:22:57.433441+0000","last_became_peered":"2024-04-25T10:22:57.433441+0000","last_unstale":"2024-04-25T10:22:58.431773+0000","last_undegraded":"2024-04-25T10:22:58.431773+0000","last_fullsized":"2024-04-25T10:22:58.431773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:17.108874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405017+0000","last_change":"2024-04-25T10:22:57.419920+0000","last_active":"2024-04-25T10:22:58.405017+0000","last_peered":"2024-04-25T10:22:58.405017+0000","last_clean":"2024-04-25T10:22:58.405017+0000","last_became_active":"2024-04-25T10:22:57.419606+0000","last_became_peered":"2024-04-25T10:22:57.419606+0000","last_unstale":"2024-04-25T10:22:58.405017+0000","last_undegraded":"2024-04-25T10:22:58.405017+0000","last_fullsized":"2024-04-25T10:22:58.405017+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.700530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431436+0000","last_change":"2024-04-25T10:22:57.428429+0000","last_active":"2024-04-25T10:22:58.431436+0000","last_peered":"2024-04-25T10:22:58.431436+0000","last_clean":"2024-04-25T10:22:58.431436+0000","last_became_active":"2024-04-25T10:22:57.428123+0000","last_became_peered":"2024-04-25T10:22:57.428123+0000","last_unstale":"2024-04-25T10:22:58.431436+0000","last_undegraded":"2024-04-25T10:22:58.431436+0000","last_fullsized":"2024-04-25T10:22:58.431436+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:53.989570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407951+0000","last_change":"2024-04-25T10:22:57.422202+0000","last_active":"2024-04-25T10:22:58.407951+0000","last_peered":"2024-04-25T10:22:58.407951+0000","last_clean":"2024-04-25T10:22:58.407951+0000","last_became_active":"2024-04-25T10:22:57.421834+0000","last_became_peered":"2024-04-25T10:22:57.421834+0000","last_unstale":"2024-04-25T10:22:58.407951+0000","last_undegraded":"2024-04-25T10:22:58.407951+0000","last_fullsized":"2024-04-25T10:22:58.407951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:31.647346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407078+0000","last_change":"2024-04-25T10:22:57.426203+0000","last_active":"2024-04-25T10:22:58.407078+0000","last_peered":"2024-04-25T10:22:58.407078+0000","last_clean":"2024-04-25T10:22:58.407078+0000","last_became_active":"2024-04-25T10:22:57.425553+0000","last_became_peered":"2024-04-25T10:22:57.425553+0000","last_unstale":"2024-04-25T10:22:58.407078+0000","last_undegraded":"2024-04-25T10:22:58.407078+0000","last_fullsized":"2024-04-25T10:22:58.407078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:58.202829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431438+0000","last_change":"2024-04-25T10:22:57.420873+0000","last_active":"2024-04-25T10:22:58.431438+0000","last_peered":"2024-04-25T10:22:58.431438+0000","last_clean":"2024-04-25T10:22:58.431438+0000","last_became_active":"2024-04-25T10:22:57.420405+0000","last_became_peered":"2024-04-25T10:22:57.420405+0000","last_unstale":"2024-04-25T10:22:58.431438+0000","last_undegraded":"2024-04-25T10:22:58.431438+0000","last_fullsized":"2024-04-25T10:22:58.431438+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:12.153600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457893+0000","last_change":"2024-04-25T10:22:57.423395+0000","last_active":"2024-04-25T10:22:59.457893+0000","last_peered":"2024-04-25T10:22:59.457893+0000","last_clean":"2024-04-25T10:22:59.457893+0000","last_became_active":"2024-04-25T10:22:57.423144+0000","last_became_peered":"2024-04-25T10:22:57.423144+0000","last_unstale":"2024-04-25T10:22:59.457893+0000","last_undegraded":"2024-04-25T10:22:59.457893+0000","last_fullsized":"2024-04-25T10:22:59.457893+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:23.952786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431994+0000","last_change":"2024-04-25T10:22:57.434763+0000","last_active":"2024-04-25T10:22:58.431994+0000","last_peered":"2024-04-25T10:22:58.431994+0000","last_clean":"2024-04-25T10:22:58.431994+0000","last_became_active":"2024-04-25T10:22:57.434509+0000","last_became_peered":"2024-04-25T10:22:57.434509+0000","last_unstale":"2024-04-25T10:22:58.431994+0000","last_undegraded":"2024-04-25T10:22:58.431994+0000","last_fullsized":"2024-04-25T10:22:58.431994+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:16.713083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459658+0000","last_change":"2024-04-25T10:22:57.426289+0000","last_active":"2024-04-25T10:22:59.459658+0000","last_peered":"2024-04-25T10:22:59.459658+0000","last_clean":"2024-04-25T10:22:59.459658+0000","last_became_active":"2024-04-25T10:22:57.426011+0000","last_became_peered":"2024-04-25T10:22:57.426011+0000","last_unstale":"2024-04-25T10:22:59.459658+0000","last_undegraded":"2024-04-25T10:22:59.459658+0000","last_fullsized":"2024-04-25T10:22:59.459658+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:19.500478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405320+0000","last_change":"2024-04-25T10:22:57.426466+0000","last_active":"2024-04-25T10:22:58.405320+0000","last_peered":"2024-04-25T10:22:58.405320+0000","last_clean":"2024-04-25T10:22:58.405320+0000","last_became_active":"2024-04-25T10:22:57.425941+0000","last_became_peered":"2024-04-25T10:22:57.425941+0000","last_unstale":"2024-04-25T10:22:58.405320+0000","last_undegraded":"2024-04-25T10:22:58.405320+0000","last_fullsized":"2024-04-25T10:22:58.405320+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:42.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457729+0000","last_change":"2024-04-25T10:22:57.420833+0000","last_active":"2024-04-25T10:22:59.457729+0000","last_peered":"2024-04-25T10:22:59.457729+0000","last_clean":"2024-04-25T10:22:59.457729+0000","last_became_active":"2024-04-25T10:22:57.420367+0000","last_became_peered":"2024-04-25T10:22:57.420367+0000","last_unstale":"2024-04-25T10:22:59.457729+0000","last_undegraded":"2024-04-25T10:22:59.457729+0000","last_fullsized":"2024-04-25T10:22:59.457729+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:35.129534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431649+0000","last_change":"2024-04-25T10:22:57.435015+0000","last_active":"2024-04-25T10:22:58.431649+0000","last_peered":"2024-04-25T10:22:58.431649+0000","last_clean":"2024-04-25T10:22:58.431649+0000","last_became_active":"2024-04-25T10:22:57.434768+0000","last_became_peered":"2024-04-25T10:22:57.434768+0000","last_unstale":"2024-04-25T10:22:58.431649+0000","last_undegraded":"2024-04-25T10:22:58.431649+0000","last_fullsized":"2024-04-25T10:22:58.431649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:20.542240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407274+0000","last_change":"2024-04-25T10:22:57.419029+0000","last_active":"2024-04-25T10:22:58.407274+0000","last_peered":"2024-04-25T10:22:58.407274+0000","last_clean":"2024-04-25T10:22:58.407274+0000","last_became_active":"2024-04-25T10:22:57.413844+0000","last_became_peered":"2024-04-25T10:22:57.413844+0000","last_unstale":"2024-04-25T10:22:58.407274+0000","last_undegraded":"2024-04-25T10:22:58.407274+0000","last_fullsized":"2024-04-25T10:22:58.407274+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:44.029725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407113+0000","last_change":"2024-04-25T10:22:57.427202+0000","last_active":"2024-04-25T10:22:58.407113+0000","last_peered":"2024-04-25T10:22:58.407113+0000","last_clean":"2024-04-25T10:22:58.407113+0000","last_became_active":"2024-04-25T10:22:57.425700+0000","last_became_peered":"2024-04-25T10:22:57.425700+0000","last_unstale":"2024-04-25T10:22:58.407113+0000","last_undegraded":"2024-04-25T10:22:58.407113+0000","last_fullsized":"2024-04-25T10:22:58.407113+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:32.251839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431312+0000","last_change":"2024-04-25T10:22:57.423072+0000","last_active":"2024-04-25T10:22:58.431312+0000","last_peered":"2024-04-25T10:22:58.431312+0000","last_clean":"2024-04-25T10:22:58.431312+0000","last_became_active":"2024-04-25T10:22:57.422626+0000","last_became_peered":"2024-04-25T10:22:57.422626+0000","last_unstale":"2024-04-25T10:22:58.431312+0000","last_undegraded":"2024-04-25T10:22:58.431312+0000","last_fullsized":"2024-04-25T10:22:58.431312+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:02.186849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457467+0000","last_change":"2024-04-25T10:22:57.422702+0000","last_active":"2024-04-25T10:22:59.457467+0000","last_peered":"2024-04-25T10:22:59.457467+0000","last_clean":"2024-04-25T10:22:59.457467+0000","last_became_active":"2024-04-25T10:22:57.422464+0000","last_became_peered":"2024-04-25T10:22:57.422464+0000","last_unstale":"2024-04-25T10:22:59.457467+0000","last_undegraded":"2024-04-25T10:22:59.457467+0000","last_fullsized":"2024-04-25T10:22:59.457467+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:46.317870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405074+0000","last_change":"2024-04-25T10:22:57.419978+0000","last_active":"2024-04-25T10:22:58.405074+0000","last_peered":"2024-04-25T10:22:58.405074+0000","last_clean":"2024-04-25T10:22:58.405074+0000","last_became_active":"2024-04-25T10:22:57.419693+0000","last_became_peered":"2024-04-25T10:22:57.419693+0000","last_unstale":"2024-04-25T10:22:58.405074+0000","last_undegraded":"2024-04-25T10:22:58.405074+0000","last_fullsized":"2024-04-25T10:22:58.405074+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:08.970371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457891+0000","last_change":"2024-04-25T10:22:57.421289+0000","last_active":"2024-04-25T10:22:59.457891+0000","last_peered":"2024-04-25T10:22:59.457891+0000","last_clean":"2024-04-25T10:22:59.457891+0000","last_became_active":"2024-04-25T10:22:57.419037+0000","last_became_peered":"2024-04-25T10:22:57.419037+0000","last_unstale":"2024-04-25T10:22:59.457891+0000","last_undegraded":"2024-04-25T10:22:59.457891+0000","last_fullsized":"2024-04-25T10:22:59.457891+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:48.757776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458356+0000","last_change":"2024-04-25T10:22:57.423839+0000","last_active":"2024-04-25T10:22:59.458356+0000","last_peered":"2024-04-25T10:22:59.458356+0000","last_clean":"2024-04-25T10:22:59.458356+0000","last_became_active":"2024-04-25T10:22:57.423553+0000","last_became_peered":"2024-04-25T10:22:57.423553+0000","last_unstale":"2024-04-25T10:22:59.458356+0000","last_undegraded":"2024-04-25T10:22:59.458356+0000","last_fullsized":"2024-04-25T10:22:59.458356+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:07.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405054+0000","last_change":"2024-04-25T10:22:57.420415+0000","last_active":"2024-04-25T10:22:58.405054+0000","last_peered":"2024-04-25T10:22:58.405054+0000","last_clean":"2024-04-25T10:22:58.405054+0000","last_became_active":"2024-04-25T10:22:57.419249+0000","last_became_peered":"2024-04-25T10:22:57.419249+0000","last_unstale":"2024-04-25T10:22:58.405054+0000","last_undegraded":"2024-04-25T10:22:58.405054+0000","last_fullsized":"2024-04-25T10:22:58.405054+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:58.646389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407321+0000","last_change":"2024-04-25T10:22:57.425954+0000","last_active":"2024-04-25T10:22:58.407321+0000","last_peered":"2024-04-25T10:22:58.407321+0000","last_clean":"2024-04-25T10:22:58.407321+0000","last_became_active":"2024-04-25T10:22:57.425177+0000","last_became_peered":"2024-04-25T10:22:57.425177+0000","last_unstale":"2024-04-25T10:22:58.407321+0000","last_undegraded":"2024-04-25T10:22:58.407321+0000","last_fullsized":"2024-04-25T10:22:58.407321+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:50.730324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883588+0000","last_change":"2024-04-25T10:22:57.412963+0000","last_active":"2024-04-25T10:22:58.883588+0000","last_peered":"2024-04-25T10:22:58.883588+0000","last_clean":"2024-04-25T10:22:58.883588+0000","last_became_active":"2024-04-25T10:22:57.412648+0000","last_became_peered":"2024-04-25T10:22:57.412648+0000","last_unstale":"2024-04-25T10:22:58.883588+0000","last_undegraded":"2024-04-25T10:22:58.883588+0000","last_fullsized":"2024-04-25T10:22:58.883588+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:07.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458061+0000","last_change":"2024-04-25T10:22:57.418894+0000","last_active":"2024-04-25T10:22:59.458061+0000","last_peered":"2024-04-25T10:22:59.458061+0000","last_clean":"2024-04-25T10:22:59.458061+0000","last_became_active":"2024-04-25T10:22:57.418457+0000","last_became_peered":"2024-04-25T10:22:57.418457+0000","last_unstale":"2024-04-25T10:22:59.458061+0000","last_undegraded":"2024-04-25T10:22:59.458061+0000","last_fullsized":"2024-04-25T10:22:59.458061+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:43.514012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431852+0000","last_change":"2024-04-25T10:22:57.431160+0000","last_active":"2024-04-25T10:22:58.431852+0000","last_peered":"2024-04-25T10:22:58.431852+0000","last_clean":"2024-04-25T10:22:58.431852+0000","last_became_active":"2024-04-25T10:22:57.430903+0000","last_became_peered":"2024-04-25T10:22:57.430903+0000","last_unstale":"2024-04-25T10:22:58.431852+0000","last_undegraded":"2024-04-25T10:22:58.431852+0000","last_fullsized":"2024-04-25T10:22:58.431852+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:22.363914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"137'4","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431839+0000","last_change":"2024-04-25T10:22:56.406672+0000","last_active":"2024-04-25T10:22:58.431839+0000","last_peered":"2024-04-25T10:22:58.431839+0000","last_clean":"2024-04-25T10:22:58.431839+0000","last_became_active":"2024-04-25T10:22:56.406415+0000","last_became_peered":"2024-04-25T10:22:56.406415+0000","last_unstale":"2024-04-25T10:22:58.431839+0000","last_undegraded":"2024-04-25T10:22:58.431839+0000","last_fullsized":"2024-04-25T10:22:58.431839+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:17.476656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883782+0000","last_change":"2024-04-25T10:22:57.427432+0000","last_active":"2024-04-25T10:22:58.883782+0000","last_peered":"2024-04-25T10:22:58.883782+0000","last_clean":"2024-04-25T10:22:58.883782+0000","last_became_active":"2024-04-25T10:22:57.427153+0000","last_became_peered":"2024-04-25T10:22:57.427153+0000","last_unstale":"2024-04-25T10:22:58.883782+0000","last_undegraded":"2024-04-25T10:22:58.883782+0000","last_fullsized":"2024-04-25T10:22:58.883782+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:55.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883847+0000","last_change":"2024-04-25T10:22:56.404525+0000","last_active":"2024-04-25T10:22:58.883847+0000","last_peered":"2024-04-25T10:22:58.883847+0000","last_clean":"2024-04-25T10:22:58.883847+0000","last_became_active":"2024-04-25T10:22:56.404252+0000","last_became_peered":"2024-04-25T10:22:56.404252+0000","last_unstale":"2024-04-25T10:22:58.883847+0000","last_undegraded":"2024-04-25T10:22:58.883847+0000","last_fullsized":"2024-04-25T10:22:58.883847+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:59.730643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405124+0000","last_change":"2024-04-25T10:22:57.423863+0000","last_active":"2024-04-25T10:22:58.405124+0000","last_peered":"2024-04-25T10:22:58.405124+0000","last_clean":"2024-04-25T10:22:58.405124+0000","last_became_active":"2024-04-25T10:22:57.423623+0000","last_became_peered":"2024-04-25T10:22:57.423623+0000","last_unstale":"2024-04-25T10:22:58.405124+0000","last_undegraded":"2024-04-25T10:22:58.405124+0000","last_fullsized":"2024-04-25T10:22:58.405124+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:32.192334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407768+0000","last_change":"2024-04-25T10:22:56.401905+0000","last_active":"2024-04-25T10:22:58.407768+0000","last_peered":"2024-04-25T10:22:58.407768+0000","last_clean":"2024-04-25T10:22:58.407768+0000","last_became_active":"2024-04-25T10:22:56.401638+0000","last_became_peered":"2024-04-25T10:22:56.401638+0000","last_unstale":"2024-04-25T10:22:58.407768+0000","last_undegraded":"2024-04-25T10:22:58.407768+0000","last_fullsized":"2024-04-25T10:22:58.407768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.538199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431657+0000","last_change":"2024-04-25T10:22:57.428702+0000","last_active":"2024-04-25T10:22:58.431657+0000","last_peered":"2024-04-25T10:22:58.431657+0000","last_clean":"2024-04-25T10:22:58.431657+0000","last_became_active":"2024-04-25T10:22:57.427858+0000","last_became_peered":"2024-04-25T10:22:57.427858+0000","last_unstale":"2024-04-25T10:22:58.431657+0000","last_undegraded":"2024-04-25T10:22:58.431657+0000","last_fullsized":"2024-04-25T10:22:58.431657+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:58.791815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408053+0000","last_change":"2024-04-25T10:22:56.402508+0000","last_active":"2024-04-25T10:22:58.408053+0000","last_peered":"2024-04-25T10:22:58.408053+0000","last_clean":"2024-04-25T10:22:58.408053+0000","last_became_active":"2024-04-25T10:22:56.402249+0000","last_became_peered":"2024-04-25T10:22:56.402249+0000","last_unstale":"2024-04-25T10:22:58.408053+0000","last_undegraded":"2024-04-25T10:22:58.408053+0000","last_fullsized":"2024-04-25T10:22:58.408053+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:09.044675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883550+0000","last_change":"2024-04-25T10:22:57.416996+0000","last_active":"2024-04-25T10:22:58.883550+0000","last_peered":"2024-04-25T10:22:58.883550+0000","last_clean":"2024-04-25T10:22:58.883550+0000","last_became_active":"2024-04-25T10:22:57.416585+0000","last_became_peered":"2024-04-25T10:22:57.416585+0000","last_unstale":"2024-04-25T10:22:58.883550+0000","last_undegraded":"2024-04-25T10:22:58.883550+0000","last_fullsized":"2024-04-25T10:22:58.883550+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:49.433729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457227+0000","last_change":"2024-04-25T10:22:56.410126+0000","last_active":"2024-04-25T10:22:59.457227+0000","last_peered":"2024-04-25T10:22:59.457227+0000","last_clean":"2024-04-25T10:22:59.457227+0000","last_became_active":"2024-04-25T10:22:56.409634+0000","last_became_peered":"2024-04-25T10:22:56.409634+0000","last_unstale":"2024-04-25T10:22:59.457227+0000","last_undegraded":"2024-04-25T10:22:59.457227+0000","last_fullsized":"2024-04-25T10:22:59.457227+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:32.860372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407636+0000","last_change":"2024-04-25T10:22:57.426676+0000","last_active":"2024-04-25T10:22:58.407636+0000","last_peered":"2024-04-25T10:22:58.407636+0000","last_clean":"2024-04-25T10:22:58.407636+0000","last_became_active":"2024-04-25T10:22:57.425820+0000","last_became_peered":"2024-04-25T10:22:57.425820+0000","last_unstale":"2024-04-25T10:22:58.407636+0000","last_undegraded":"2024-04-25T10:22:58.407636+0000","last_fullsized":"2024-04-25T10:22:58.407636+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:49.146689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407718+0000","last_change":"2024-04-25T10:22:56.398019+0000","last_active":"2024-04-25T10:22:58.407718+0000","last_peered":"2024-04-25T10:22:58.407718+0000","last_clean":"2024-04-25T10:22:58.407718+0000","last_became_active":"2024-04-25T10:22:56.397728+0000","last_became_peered":"2024-04-25T10:22:56.397728+0000","last_unstale":"2024-04-25T10:22:58.407718+0000","last_undegraded":"2024-04-25T10:22:58.407718+0000","last_fullsized":"2024-04-25T10:22:58.407718+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:50.846601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883527+0000","last_change":"2024-04-25T10:22:57.408521+0000","last_active":"2024-04-25T10:22:58.883527+0000","last_peered":"2024-04-25T10:22:58.883527+0000","last_clean":"2024-04-25T10:22:58.883527+0000","last_became_active":"2024-04-25T10:22:57.408071+0000","last_became_peered":"2024-04-25T10:22:57.408071+0000","last_unstale":"2024-04-25T10:22:58.883527+0000","last_undegraded":"2024-04-25T10:22:58.883527+0000","last_fullsized":"2024-04-25T10:22:58.883527+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.285372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457691+0000","last_change":"2024-04-25T10:22:56.405829+0000","last_active":"2024-04-25T10:22:59.457691+0000","last_peered":"2024-04-25T10:22:59.457691+0000","last_clean":"2024-04-25T10:22:59.457691+0000","last_became_active":"2024-04-25T10:22:56.405574+0000","last_became_peered":"2024-04-25T10:22:56.405574+0000","last_unstale":"2024-04-25T10:22:59.457691+0000","last_undegraded":"2024-04-25T10:22:59.457691+0000","last_fullsized":"2024-04-25T10:22:59.457691+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:26.731280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407623+0000","last_change":"2024-04-25T10:22:57.423215+0000","last_active":"2024-04-25T10:22:58.407623+0000","last_peered":"2024-04-25T10:22:58.407623+0000","last_clean":"2024-04-25T10:22:58.407623+0000","last_became_active":"2024-04-25T10:22:57.419438+0000","last_became_peered":"2024-04-25T10:22:57.419438+0000","last_unstale":"2024-04-25T10:22:58.407623+0000","last_undegraded":"2024-04-25T10:22:58.407623+0000","last_fullsized":"2024-04-25T10:22:58.407623+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:56.225148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.406699+0000","last_change":"2024-04-25T10:22:56.400182+0000","last_active":"2024-04-25T10:22:58.406699+0000","last_peered":"2024-04-25T10:22:58.406699+0000","last_clean":"2024-04-25T10:22:58.406699+0000","last_became_active":"2024-04-25T10:22:56.399914+0000","last_became_peered":"2024-04-25T10:22:56.399914+0000","last_unstale":"2024-04-25T10:22:58.406699+0000","last_undegraded":"2024-04-25T10:22:58.406699+0000","last_fullsized":"2024-04-25T10:22:58.406699+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:30.846368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408335+0000","last_change":"2024-04-25T10:22:57.419455+0000","last_active":"2024-04-25T10:22:58.408335+0000","last_peered":"2024-04-25T10:22:58.408335+0000","last_clean":"2024-04-25T10:22:58.408335+0000","last_became_active":"2024-04-25T10:22:57.417451+0000","last_became_peered":"2024-04-25T10:22:57.417451+0000","last_unstale":"2024-04-25T10:22:58.408335+0000","last_undegraded":"2024-04-25T10:22:58.408335+0000","last_fullsized":"2024-04-25T10:22:58.408335+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:14.283194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883811+0000","last_change":"2024-04-25T10:22:56.404968+0000","last_active":"2024-04-25T10:22:58.883811+0000","last_peered":"2024-04-25T10:22:58.883811+0000","last_clean":"2024-04-25T10:22:58.883811+0000","last_became_active":"2024-04-25T10:22:56.404275+0000","last_became_peered":"2024-04-25T10:22:56.404275+0000","last_unstale":"2024-04-25T10:22:58.883811+0000","last_undegraded":"2024-04-25T10:22:58.883811+0000","last_fullsized":"2024-04-25T10:22:58.883811+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:52.403966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431961+0000","last_change":"2024-04-25T10:22:57.434286+0000","last_active":"2024-04-25T10:22:58.431961+0000","last_peered":"2024-04-25T10:22:58.431961+0000","last_clean":"2024-04-25T10:22:58.431961+0000","last_became_active":"2024-04-25T10:22:57.433877+0000","last_became_peered":"2024-04-25T10:22:57.433877+0000","last_unstale":"2024-04-25T10:22:58.431961+0000","last_undegraded":"2024-04-25T10:22:58.431961+0000","last_fullsized":"2024-04-25T10:22:58.431961+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.757444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.432009+0000","last_change":"2024-04-25T10:22:56.409876+0000","last_active":"2024-04-25T10:22:58.432009+0000","last_peered":"2024-04-25T10:22:58.432009+0000","last_clean":"2024-04-25T10:22:58.432009+0000","last_became_active":"2024-04-25T10:22:56.409594+0000","last_became_peered":"2024-04-25T10:22:56.409594+0000","last_unstale":"2024-04-25T10:22:58.432009+0000","last_undegraded":"2024-04-25T10:22:58.432009+0000","last_fullsized":"2024-04-25T10:22:58.432009+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:19.741870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457591+0000","last_change":"2024-04-25T10:22:57.428074+0000","last_active":"2024-04-25T10:22:59.457591+0000","last_peered":"2024-04-25T10:22:59.457591+0000","last_clean":"2024-04-25T10:22:59.457591+0000","last_became_active":"2024-04-25T10:22:57.427820+0000","last_became_peered":"2024-04-25T10:22:57.427820+0000","last_unstale":"2024-04-25T10:22:59.457591+0000","last_undegraded":"2024-04-25T10:22:59.457591+0000","last_fullsized":"2024-04-25T10:22:59.457591+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:19.664954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"137'6","reported_seq":20,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458360+0000","last_change":"2024-04-25T10:22:56.406465+0000","last_active":"2024-04-25T10:22:59.458360+0000","last_peered":"2024-04-25T10:22:59.458360+0000","last_clean":"2024-04-25T10:22:59.458360+0000","last_became_active":"2024-04-25T10:22:56.406172+0000","last_became_peered":"2024-04-25T10:22:56.406172+0000","last_unstale":"2024-04-25T10:22:59.458360+0000","last_undegraded":"2024-04-25T10:22:59.458360+0000","last_fullsized":"2024-04-25T10:22:59.458360+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:59.574617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459340+0000","last_change":"2024-04-25T10:22:57.429703+0000","last_active":"2024-04-25T10:22:59.459340+0000","last_peered":"2024-04-25T10:22:59.459340+0000","last_clean":"2024-04-25T10:22:59.459340+0000","last_became_active":"2024-04-25T10:22:57.429433+0000","last_became_peered":"2024-04-25T10:22:57.429433+0000","last_unstale":"2024-04-25T10:22:59.459340+0000","last_undegraded":"2024-04-25T10:22:59.459340+0000","last_fullsized":"2024-04-25T10:22:59.459340+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:48.183876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459294+0000","last_change":"2024-04-25T10:22:56.405202+0000","last_active":"2024-04-25T10:22:59.459294+0000","last_peered":"2024-04-25T10:22:59.459294+0000","last_clean":"2024-04-25T10:22:59.459294+0000","last_became_active":"2024-04-25T10:22:56.404825+0000","last_became_peered":"2024-04-25T10:22:56.404825+0000","last_unstale":"2024-04-25T10:22:59.459294+0000","last_undegraded":"2024-04-25T10:22:59.459294+0000","last_fullsized":"2024-04-25T10:22:59.459294+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:41.838590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883769+0000","last_change":"2024-04-25T10:22:57.427686+0000","last_active":"2024-04-25T10:22:58.883769+0000","last_peered":"2024-04-25T10:22:58.883769+0000","last_clean":"2024-04-25T10:22:58.883769+0000","last_became_active":"2024-04-25T10:22:57.427241+0000","last_became_peered":"2024-04-25T10:22:57.427241+0000","last_unstale":"2024-04-25T10:22:58.883769+0000","last_undegraded":"2024-04-25T10:22:58.883769+0000","last_fullsized":"2024-04-25T10:22:58.883769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:20.944185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459129+0000","last_change":"2024-04-25T10:22:56.408162+0000","last_active":"2024-04-25T10:22:59.459129+0000","last_peered":"2024-04-25T10:22:59.459129+0000","last_clean":"2024-04-25T10:22:59.459129+0000","last_became_active":"2024-04-25T10:22:56.407814+0000","last_became_peered":"2024-04-25T10:22:56.407814+0000","last_unstale":"2024-04-25T10:22:59.459129+0000","last_undegraded":"2024-04-25T10:22:59.459129+0000","last_fullsized":"2024-04-25T10:22:59.459129+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:30.846368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407047+0000","last_change":"2024-04-25T10:22:57.422397+0000","last_active":"2024-04-25T10:22:58.407047+0000","last_peered":"2024-04-25T10:22:58.407047+0000","last_clean":"2024-04-25T10:22:58.407047+0000","last_became_active":"2024-04-25T10:22:57.421862+0000","last_became_peered":"2024-04-25T10:22:57.421862+0000","last_unstale":"2024-04-25T10:22:58.407047+0000","last_undegraded":"2024-04-25T10:22:58.407047+0000","last_fullsized":"2024-04-25T10:22:58.407047+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:24.867491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404659+0000","last_change":"2024-04-25T10:22:56.401669+0000","last_active":"2024-04-25T10:22:58.404659+0000","last_peered":"2024-04-25T10:22:58.404659+0000","last_clean":"2024-04-25T10:22:58.404659+0000","last_became_active":"2024-04-25T10:22:56.401377+0000","last_became_peered":"2024-04-25T10:22:56.401377+0000","last_unstale":"2024-04-25T10:22:58.404659+0000","last_undegraded":"2024-04-25T10:22:58.404659+0000","last_fullsized":"2024-04-25T10:22:58.404659+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:50.846601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459123+0000","last_change":"2024-04-25T10:22:57.415605+0000","last_active":"2024-04-25T10:22:59.459123+0000","last_peered":"2024-04-25T10:22:59.459123+0000","last_clean":"2024-04-25T10:22:59.459123+0000","last_became_active":"2024-04-25T10:22:57.415044+0000","last_became_peered":"2024-04-25T10:22:57.415044+0000","last_unstale":"2024-04-25T10:22:59.459123+0000","last_undegraded":"2024-04-25T10:22:59.459123+0000","last_fullsized":"2024-04-25T10:22:59.459123+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:58.395291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"137'8","reported_seq":22,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458009+0000","last_change":"2024-04-25T10:22:56.409221+0000","last_active":"2024-04-25T10:22:59.458009+0000","last_peered":"2024-04-25T10:22:59.458009+0000","last_clean":"2024-04-25T10:22:59.458009+0000","last_became_active":"2024-04-25T10:22:56.408831+0000","last_became_peered":"2024-04-25T10:22:56.408831+0000","last_unstale":"2024-04-25T10:22:59.458009+0000","last_undegraded":"2024-04-25T10:22:59.458009+0000","last_fullsized":"2024-04-25T10:22:59.458009+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:05.527321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431950+0000","last_change":"2024-04-25T10:22:57.437169+0000","last_active":"2024-04-25T10:22:58.431950+0000","last_peered":"2024-04-25T10:22:58.431950+0000","last_clean":"2024-04-25T10:22:58.431950+0000","last_became_active":"2024-04-25T10:22:57.436865+0000","last_became_peered":"2024-04-25T10:22:57.436865+0000","last_unstale":"2024-04-25T10:22:58.431950+0000","last_undegraded":"2024-04-25T10:22:58.431950+0000","last_fullsized":"2024-04-25T10:22:58.431950+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:34.662479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408258+0000","last_change":"2024-04-25T10:22:56.406651+0000","last_active":"2024-04-25T10:22:58.408258+0000","last_peered":"2024-04-25T10:22:58.408258+0000","last_clean":"2024-04-25T10:22:58.408258+0000","last_became_active":"2024-04-25T10:22:56.406261+0000","last_became_peered":"2024-04-25T10:22:56.406261+0000","last_unstale":"2024-04-25T10:22:58.408258+0000","last_undegraded":"2024-04-25T10:22:58.408258+0000","last_fullsized":"2024-04-25T10:22:58.408258+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:42.506500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457645+0000","last_change":"2024-04-25T10:22:57.422166+0000","last_active":"2024-04-25T10:22:59.457645+0000","last_peered":"2024-04-25T10:22:59.457645+0000","last_clean":"2024-04-25T10:22:59.457645+0000","last_became_active":"2024-04-25T10:22:57.421066+0000","last_became_peered":"2024-04-25T10:22:57.421066+0000","last_unstale":"2024-04-25T10:22:59.457645+0000","last_undegraded":"2024-04-25T10:22:59.457645+0000","last_fullsized":"2024-04-25T10:22:59.457645+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:23.587240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883953+0000","last_change":"2024-04-25T10:22:56.409097+0000","last_active":"2024-04-25T10:22:58.883953+0000","last_peered":"2024-04-25T10:22:58.883953+0000","last_clean":"2024-04-25T10:22:58.883953+0000","last_became_active":"2024-04-25T10:22:56.408810+0000","last_became_peered":"2024-04-25T10:22:56.408810+0000","last_unstale":"2024-04-25T10:22:58.883953+0000","last_undegraded":"2024-04-25T10:22:58.883953+0000","last_fullsized":"2024-04-25T10:22:58.883953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:48.139177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459379+0000","last_change":"2024-04-25T10:22:57.427658+0000","last_active":"2024-04-25T10:22:59.459379+0000","last_peered":"2024-04-25T10:22:59.459379+0000","last_clean":"2024-04-25T10:22:59.459379+0000","last_became_active":"2024-04-25T10:22:57.426492+0000","last_became_peered":"2024-04-25T10:22:57.426492+0000","last_unstale":"2024-04-25T10:22:59.459379+0000","last_undegraded":"2024-04-25T10:22:59.459379+0000","last_fullsized":"2024-04-25T10:22:59.459379+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:19.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457760+0000","last_change":"2024-04-25T10:22:56.405058+0000","last_active":"2024-04-25T10:22:59.457760+0000","last_peered":"2024-04-25T10:22:59.457760+0000","last_clean":"2024-04-25T10:22:59.457760+0000","last_became_active":"2024-04-25T10:22:56.404793+0000","last_became_peered":"2024-04-25T10:22:56.404793+0000","last_unstale":"2024-04-25T10:22:59.457760+0000","last_undegraded":"2024-04-25T10:22:59.457760+0000","last_fullsized":"2024-04-25T10:22:59.457760+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.958679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405171+0000","last_change":"2024-04-25T10:22:57.429135+0000","last_active":"2024-04-25T10:22:58.405171+0000","last_peered":"2024-04-25T10:22:58.405171+0000","last_clean":"2024-04-25T10:22:58.405171+0000","last_became_active":"2024-04-25T10:22:57.428826+0000","last_became_peered":"2024-04-25T10:22:57.428826+0000","last_unstale":"2024-04-25T10:22:58.405171+0000","last_undegraded":"2024-04-25T10:22:58.405171+0000","last_fullsized":"2024-04-25T10:22:58.405171+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:41.917144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883587+0000","last_change":"2024-04-25T10:22:56.405467+0000","last_active":"2024-04-25T10:22:58.883587+0000","last_peered":"2024-04-25T10:22:58.883587+0000","last_clean":"2024-04-25T10:22:58.883587+0000","last_became_active":"2024-04-25T10:22:56.405200+0000","last_became_peered":"2024-04-25T10:22:56.405200+0000","last_unstale":"2024-04-25T10:22:58.883587+0000","last_undegraded":"2024-04-25T10:22:58.883587+0000","last_fullsized":"2024-04-25T10:22:58.883587+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.022213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407453+0000","last_change":"2024-04-25T10:22:57.424819+0000","last_active":"2024-04-25T10:22:58.407453+0000","last_peered":"2024-04-25T10:22:58.407453+0000","last_clean":"2024-04-25T10:22:58.407453+0000","last_became_active":"2024-04-25T10:22:57.423557+0000","last_became_peered":"2024-04-25T10:22:57.423557+0000","last_unstale":"2024-04-25T10:22:58.407453+0000","last_undegraded":"2024-04-25T10:22:58.407453+0000","last_fullsized":"2024-04-25T10:22:58.407453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:00.738155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458318+0000","last_change":"2024-04-25T10:22:56.403915+0000","last_active":"2024-04-25T10:22:59.458318+0000","last_peered":"2024-04-25T10:22:59.458318+0000","last_clean":"2024-04-25T10:22:59.458318+0000","last_became_active":"2024-04-25T10:22:56.403656+0000","last_became_peered":"2024-04-25T10:22:56.403656+0000","last_unstale":"2024-04-25T10:22:59.458318+0000","last_undegraded":"2024-04-25T10:22:59.458318+0000","last_fullsized":"2024-04-25T10:22:59.458318+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.742849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458362+0000","last_change":"2024-04-25T10:22:57.426609+0000","last_active":"2024-04-25T10:22:59.458362+0000","last_peered":"2024-04-25T10:22:59.458362+0000","last_clean":"2024-04-25T10:22:59.458362+0000","last_became_active":"2024-04-25T10:22:57.426326+0000","last_became_peered":"2024-04-25T10:22:57.426326+0000","last_unstale":"2024-04-25T10:22:59.458362+0000","last_undegraded":"2024-04-25T10:22:59.458362+0000","last_fullsized":"2024-04-25T10:22:59.458362+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:41.758097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431286+0000","last_change":"2024-04-25T10:22:56.406936+0000","last_active":"2024-04-25T10:22:58.431286+0000","last_peered":"2024-04-25T10:22:58.431286+0000","last_clean":"2024-04-25T10:22:58.431286+0000","last_became_active":"2024-04-25T10:22:56.406321+0000","last_became_peered":"2024-04-25T10:22:56.406321+0000","last_unstale":"2024-04-25T10:22:58.431286+0000","last_undegraded":"2024-04-25T10:22:58.431286+0000","last_fullsized":"2024-04-25T10:22:58.431286+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.406774+0000","last_change":"2024-04-25T10:22:57.426142+0000","last_active":"2024-04-25T10:22:58.406774+0000","last_peered":"2024-04-25T10:22:58.406774+0000","last_clean":"2024-04-25T10:22:58.406774+0000","last_became_active":"2024-04-25T10:22:57.425790+0000","last_became_peered":"2024-04-25T10:22:57.425790+0000","last_unstale":"2024-04-25T10:22:58.406774+0000","last_undegraded":"2024-04-25T10:22:58.406774+0000","last_fullsized":"2024-04-25T10:22:58.406774+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:19.500478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431185+0000","last_change":"2024-04-25T10:22:56.411953+0000","last_active":"2024-04-25T10:22:58.431185+0000","last_peered":"2024-04-25T10:22:58.431185+0000","last_clean":"2024-04-25T10:22:58.431185+0000","last_became_active":"2024-04-25T10:22:56.411594+0000","last_became_peered":"2024-04-25T10:22:56.411594+0000","last_unstale":"2024-04-25T10:22:58.431185+0000","last_undegraded":"2024-04-25T10:22:58.431185+0000","last_fullsized":"2024-04-25T10:22:58.431185+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:59.574617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457923+0000","last_change":"2024-04-25T10:22:57.425323+0000","last_active":"2024-04-25T10:22:59.457923+0000","last_peered":"2024-04-25T10:22:59.457923+0000","last_clean":"2024-04-25T10:22:59.457923+0000","last_became_active":"2024-04-25T10:22:57.425030+0000","last_became_peered":"2024-04-25T10:22:57.425030+0000","last_unstale":"2024-04-25T10:22:59.457923+0000","last_undegraded":"2024-04-25T10:22:59.457923+0000","last_fullsized":"2024-04-25T10:22:59.457923+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:20.808204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458912+0000","last_change":"2024-04-25T10:22:56.412818+0000","last_active":"2024-04-25T10:22:59.458912+0000","last_peered":"2024-04-25T10:22:59.458912+0000","last_clean":"2024-04-25T10:22:59.458912+0000","last_became_active":"2024-04-25T10:22:56.412466+0000","last_became_peered":"2024-04-25T10:22:56.412466+0000","last_unstale":"2024-04-25T10:22:59.458912+0000","last_undegraded":"2024-04-25T10:22:59.458912+0000","last_fullsized":"2024-04-25T10:22:59.458912+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:31.184822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.408388+0000","last_change":"2024-04-25T10:22:57.429051+0000","last_active":"2024-04-25T10:22:58.408388+0000","last_peered":"2024-04-25T10:22:58.408388+0000","last_clean":"2024-04-25T10:22:58.408388+0000","last_became_active":"2024-04-25T10:22:57.428603+0000","last_became_peered":"2024-04-25T10:22:57.428603+0000","last_unstale":"2024-04-25T10:22:58.408388+0000","last_undegraded":"2024-04-25T10:22:58.408388+0000","last_fullsized":"2024-04-25T10:22:58.408388+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:10.594590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.432015+0000","last_change":"2024-04-25T10:22:56.410115+0000","last_active":"2024-04-25T10:22:58.432015+0000","last_peered":"2024-04-25T10:22:58.432015+0000","last_clean":"2024-04-25T10:22:58.432015+0000","last_became_active":"2024-04-25T10:22:56.409783+0000","last_became_peered":"2024-04-25T10:22:56.409783+0000","last_unstale":"2024-04-25T10:22:58.432015+0000","last_undegraded":"2024-04-25T10:22:58.432015+0000","last_fullsized":"2024-04-25T10:22:58.432015+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:43.989532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459666+0000","last_change":"2024-04-25T10:22:57.429612+0000","last_active":"2024-04-25T10:22:59.459666+0000","last_peered":"2024-04-25T10:22:59.459666+0000","last_clean":"2024-04-25T10:22:59.459666+0000","last_became_active":"2024-04-25T10:22:57.429307+0000","last_became_peered":"2024-04-25T10:22:57.429307+0000","last_unstale":"2024-04-25T10:22:59.459666+0000","last_undegraded":"2024-04-25T10:22:59.459666+0000","last_fullsized":"2024-04-25T10:22:59.459666+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:58.202829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"138'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405218+0000","last_change":"2024-04-25T10:22:56.401059+0000","last_active":"2024-04-25T10:22:58.405218+0000","last_peered":"2024-04-25T10:22:58.405218+0000","last_clean":"2024-04-25T10:22:58.405218+0000","last_became_active":"2024-04-25T10:22:56.400831+0000","last_became_peered":"2024-04-25T10:22:56.400831+0000","last_unstale":"2024-04-25T10:22:58.405218+0000","last_undegraded":"2024-04-25T10:22:58.405218+0000","last_fullsized":"2024-04-25T10:22:58.405218+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:05.933699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883635+0000","last_change":"2024-04-25T10:22:57.408929+0000","last_active":"2024-04-25T10:22:58.883635+0000","last_peered":"2024-04-25T10:22:58.883635+0000","last_clean":"2024-04-25T10:22:58.883635+0000","last_became_active":"2024-04-25T10:22:57.408636+0000","last_became_peered":"2024-04-25T10:22:57.408636+0000","last_unstale":"2024-04-25T10:22:58.883635+0000","last_undegraded":"2024-04-25T10:22:58.883635+0000","last_fullsized":"2024-04-25T10:22:58.883635+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:41.758097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405295+0000","last_change":"2024-04-25T10:22:56.406043+0000","last_active":"2024-04-25T10:22:58.405295+0000","last_peered":"2024-04-25T10:22:58.405295+0000","last_clean":"2024-04-25T10:22:58.405295+0000","last_became_active":"2024-04-25T10:22:56.405764+0000","last_became_peered":"2024-04-25T10:22:56.405764+0000","last_unstale":"2024-04-25T10:22:58.405295+0000","last_undegraded":"2024-04-25T10:22:58.405295+0000","last_fullsized":"2024-04-25T10:22:58.405295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:17.228857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.405240+0000","last_change":"2024-04-25T10:22:57.426198+0000","last_active":"2024-04-25T10:22:58.405240+0000","last_peered":"2024-04-25T10:22:58.405240+0000","last_clean":"2024-04-25T10:22:58.405240+0000","last_became_active":"2024-04-25T10:22:57.425603+0000","last_became_peered":"2024-04-25T10:22:57.425603+0000","last_unstale":"2024-04-25T10:22:58.405240+0000","last_undegraded":"2024-04-25T10:22:58.405240+0000","last_fullsized":"2024-04-25T10:22:58.405240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:47.261911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.404991+0000","last_change":"2024-04-25T10:22:56.406395+0000","last_active":"2024-04-25T10:22:58.404991+0000","last_peered":"2024-04-25T10:22:58.404991+0000","last_clean":"2024-04-25T10:22:58.404991+0000","last_became_active":"2024-04-25T10:22:56.405805+0000","last_became_peered":"2024-04-25T10:22:56.405805+0000","last_unstale":"2024-04-25T10:22:58.404991+0000","last_undegraded":"2024-04-25T10:22:58.404991+0000","last_fullsized":"2024-04-25T10:22:58.404991+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:42.506500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.457447+0000","last_change":"2024-04-25T10:22:57.427188+0000","last_active":"2024-04-25T10:22:59.457447+0000","last_peered":"2024-04-25T10:22:59.457447+0000","last_clean":"2024-04-25T10:22:59.457447+0000","last_became_active":"2024-04-25T10:22:57.426833+0000","last_became_peered":"2024-04-25T10:22:57.426833+0000","last_unstale":"2024-04-25T10:22:59.457447+0000","last_undegraded":"2024-04-25T10:22:59.457447+0000","last_fullsized":"2024-04-25T10:22:59.457447+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:07.034942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458411+0000","last_change":"2024-04-25T10:22:56.399245+0000","last_active":"2024-04-25T10:22:59.458411+0000","last_peered":"2024-04-25T10:22:59.458411+0000","last_clean":"2024-04-25T10:22:59.458411+0000","last_became_active":"2024-04-25T10:22:56.398919+0000","last_became_peered":"2024-04-25T10:22:56.398919+0000","last_unstale":"2024-04-25T10:22:59.458411+0000","last_undegraded":"2024-04-25T10:22:59.458411+0000","last_fullsized":"2024-04-25T10:22:59.458411+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407680+0000","last_change":"2024-04-25T10:22:57.419140+0000","last_active":"2024-04-25T10:22:58.407680+0000","last_peered":"2024-04-25T10:22:58.407680+0000","last_clean":"2024-04-25T10:22:58.407680+0000","last_became_active":"2024-04-25T10:22:57.418497+0000","last_became_peered":"2024-04-25T10:22:57.418497+0000","last_unstale":"2024-04-25T10:22:58.407680+0000","last_undegraded":"2024-04-25T10:22:58.407680+0000","last_fullsized":"2024-04-25T10:22:58.407680+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:38.984827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431185+0000","last_change":"2024-04-25T10:22:56.407652+0000","last_active":"2024-04-25T10:22:58.431185+0000","last_peered":"2024-04-25T10:22:58.431185+0000","last_clean":"2024-04-25T10:22:58.431185+0000","last_became_active":"2024-04-25T10:22:56.407372+0000","last_became_peered":"2024-04-25T10:22:56.407372+0000","last_unstale":"2024-04-25T10:22:58.431185+0000","last_undegraded":"2024-04-25T10:22:58.431185+0000","last_fullsized":"2024-04-25T10:22:58.431185+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.742849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431325+0000","last_change":"2024-04-25T10:22:57.422870+0000","last_active":"2024-04-25T10:22:58.431325+0000","last_peered":"2024-04-25T10:22:58.431325+0000","last_clean":"2024-04-25T10:22:58.431325+0000","last_became_active":"2024-04-25T10:22:57.422302+0000","last_became_peered":"2024-04-25T10:22:57.422302+0000","last_unstale":"2024-04-25T10:22:58.431325+0000","last_undegraded":"2024-04-25T10:22:58.431325+0000","last_fullsized":"2024-04-25T10:22:58.431325+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.333103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459031+0000","last_change":"2024-04-25T10:22:56.404329+0000","last_active":"2024-04-25T10:22:59.459031+0000","last_peered":"2024-04-25T10:22:59.459031+0000","last_clean":"2024-04-25T10:22:59.459031+0000","last_became_active":"2024-04-25T10:22:56.404004+0000","last_became_peered":"2024-04-25T10:22:56.404004+0000","last_unstale":"2024-04-25T10:22:59.459031+0000","last_undegraded":"2024-04-25T10:22:59.459031+0000","last_fullsized":"2024-04-25T10:22:59.459031+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:40.909632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.407435+0000","last_change":"2024-04-25T10:22:57.416491+0000","last_active":"2024-04-25T10:22:58.407435+0000","last_peered":"2024-04-25T10:22:58.407435+0000","last_clean":"2024-04-25T10:22:58.407435+0000","last_became_active":"2024-04-25T10:22:57.416144+0000","last_became_peered":"2024-04-25T10:22:57.416144+0000","last_unstale":"2024-04-25T10:22:58.407435+0000","last_undegraded":"2024-04-25T10:22:58.407435+0000","last_fullsized":"2024-04-25T10:22:58.407435+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:59.062309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.883663+0000","last_change":"2024-04-25T10:22:56.405780+0000","last_active":"2024-04-25T10:22:58.883663+0000","last_peered":"2024-04-25T10:22:58.883663+0000","last_clean":"2024-04-25T10:22:58.883663+0000","last_became_active":"2024-04-25T10:22:56.405542+0000","last_became_peered":"2024-04-25T10:22:56.405542+0000","last_unstale":"2024-04-25T10:22:58.883663+0000","last_undegraded":"2024-04-25T10:22:58.883663+0000","last_fullsized":"2024-04-25T10:22:58.883663+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:49.722812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.458284+0000","last_change":"2024-04-25T10:22:57.425724+0000","last_active":"2024-04-25T10:22:59.458284+0000","last_peered":"2024-04-25T10:22:59.458284+0000","last_clean":"2024-04-25T10:22:59.458284+0000","last_became_active":"2024-04-25T10:22:57.425440+0000","last_became_peered":"2024-04-25T10:22:57.425440+0000","last_unstale":"2024-04-25T10:22:59.458284+0000","last_undegraded":"2024-04-25T10:22:59.458284+0000","last_fullsized":"2024-04-25T10:22:59.458284+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:56.381155+0000","last_clean_scrub_stamp":"2024-04-25T10:22:56.381155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:49.433729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"137'3","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:59.459497+0000","last_change":"2024-04-25T10:22:56.409604+0000","last_active":"2024-04-25T10:22:59.459497+0000","last_peered":"2024-04-25T10:22:59.459497+0000","last_clean":"2024-04-25T10:22:59.459497+0000","last_became_active":"2024-04-25T10:22:56.409342+0000","last_became_peered":"2024-04-25T10:22:56.409342+0000","last_unstale":"2024-04-25T10:22:59.459497+0000","last_undegraded":"2024-04-25T10:22:59.459497+0000","last_fullsized":"2024-04-25T10:22:59.459497+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:55.373643+0000","last_clean_scrub_stamp":"2024-04-25T10:22:55.373643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:46.254399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":383,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T10:22:58.431393+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:22:58.431393+0000","last_peered":"2024-04-25T10:22:58.431393+0000","last_clean":"2024-04-25T10:22:58.431393+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:22:58.431393+0000","last_undegraded":"2024-04-25T10:22:58.431393+0000","last_fullsized":"2024-04-25T10:22:58.431393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2330624,"data_stored":1347607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81392,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290448,"statfs":{"total":96636764160,"available":96553418752,"internally_reserved":0,"allocated":1740800,"data_stored":756733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81590726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50920,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320920,"statfs":{"total":96636764160,"available":96584622080,"internally_reserved":0,"allocated":1732608,"data_stored":756609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":1728512,"data_stored":756587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55088,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316752,"statfs":{"total":96636764160,"available":96580354048,"internally_reserved":0,"allocated":1732608,"data_stored":756677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50932,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320908,"statfs":{"total":96636764160,"available":96584609792,"internally_reserved":0,"allocated":1744896,"data_stored":758269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64816,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307024,"statfs":{"total":96636764160,"available":96570392576,"internally_reserved":0,"allocated":1732608,"data_stored":757169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55672,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316168,"statfs":{"total":96636764160,"available":96579756032,"internally_reserved":0,"allocated":2338816,"data_stored":1348601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:23:01.842 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:23:01.843 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:23:02.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bf1600640 1 Processor -- start 2024-04-25T10:23:02.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bf1600640 1 -- start start 2024-04-25T10:23:02.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bf1600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4bec15dfb0 0x7f4bec0af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b08f0 0x7f4bec0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bf1600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec0af850 con 0x7f4bec0b1430 2024-04-25T10:23:02.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bf1600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0b0460 con 0x7f4bec15dfb0 2024-04-25T10:23:02.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bf1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0afb30 con 0x7f4bec0b08f0 2024-04-25T10:23:02.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4be2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b08f0 0x7f4bec0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4beb400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4bec0b1430 0x7f4bec06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:36768/0 (socket says 172.21.15.87:36768) 2024-04-25T10:23:02.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4beb400640 1 -- 172.21.15.87:0/1425302585 learned_addr learned my addr 172.21.15.87:0/1425302585 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:02.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4beaa00640 1 --2- 172.21.15.87:0/1425302585 >> v2:172.21.15.169:3300/0 conn(0x7f4bec15dfb0 0x7f4bec0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4bea000640 1 -- 172.21.15.87:0/1425302585 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599775622 0 0) 0x7f4bec0af850 con 0x7f4bec0b1430 2024-04-25T10:23:02.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4be2000640 1 -- 172.21.15.87:0/1425302585 >> v2:172.21.15.169:3300/0 conn(0x7f4bec15dfb0 msgr2=0x7f4bec0af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:02.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4be2000640 1 --2- 172.21.15.87:0/1425302585 >> v2:172.21.15.169:3300/0 conn(0x7f4bec15dfb0 0x7f4bec0af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4be2000640 1 -- 172.21.15.87:0/1425302585 >> v1:172.21.15.169:6789/0 conn(0x7f4bec0b1430 legacy=0x7f4bec06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.021+0000 7f4be2000640 1 -- 172.21.15.87:0/1425302585 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0afe10 con 0x7f4bec0b08f0 2024-04-25T10:23:02.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4be2000640 1 --2- 172.21.15.87:0/1425302585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b08f0 0x7f4bec0b0ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40097c0 tx=0x7f4bd401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45029925471f8114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:02.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bea000640 1 -- 172.21.15.87:0/1425302585 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40b5070 con 0x7f4bec0b08f0 2024-04-25T10:23:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bea000640 1 -- 172.21.15.87:0/1425302585 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd40a8990 con 0x7f4bec0b08f0 2024-04-25T10:23:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bea000640 1 -- 172.21.15.87:0/1425302585 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40a8d60 con 0x7f4bec0b08f0 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 -- 172.21.15.87:0/1425302585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b08f0 msgr2=0x7f4bec0b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 --2- 172.21.15.87:0/1425302585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b08f0 0x7f4bec0b0ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40097c0 tx=0x7f4bd401ec90 comp rx=0 tx=0).stop 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 -- 172.21.15.87:0/1425302585 shutdown_connections 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 --2- 172.21.15.87:0/1425302585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b08f0 0x7f4bec0b0ef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 --2- 172.21.15.87:0/1425302585 >> v2:172.21.15.169:3300/0 conn(0x7f4bec15dfb0 0x7f4bec0af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 -- 172.21.15.87:0/1425302585 >> 172.21.15.87:0/1425302585 conn(0x7f4bec0b3580 msgr2=0x7f4bec0b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 -- 172.21.15.87:0/1425302585 shutdown_connections 2024-04-25T10:23:02.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.025+0000 7f4bf1600640 1 -- 172.21.15.87:0/1425302585 wait complete. 2024-04-25T10:23:02.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 Processor -- start 2024-04-25T10:23:02.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 -- start start 2024-04-25T10:23:02.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bec0b08f0 0x7f4bec152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b1430 0x7f4bec09b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec15dfb0 0x7f4bec09d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec06bfd0 con 0x7f4bec0b08f0 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec06be50 con 0x7f4bec15dfb0 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0b18c0 con 0x7f4bec0b1430 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bec0b08f0 0x7f4bec152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bec0b08f0 0x7f4bec152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52160/0 (socket says 172.21.15.87:52160) 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 -- 172.21.15.87:0/3072419458 learned_addr learned my addr 172.21.15.87:0/3072419458 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beb400640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec15dfb0 0x7f4bec09d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4be2000640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b1430 0x7f4bec09b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 -- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b1430 msgr2=0x7f4bec09b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b1430 0x7f4bec09b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 -- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec15dfb0 msgr2=0x7f4bec09d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec15dfb0 0x7f4bec09d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 -- 172.21.15.87:0/3072419458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec1533d0 con 0x7f4bec0b08f0 2024-04-25T10:23:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4beaa00640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bec0b08f0 0x7f4bec152e90 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00e8a0 tx=0x7f4bdc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4be3400640 1 -- 172.21.15.87:0/3072419458 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc017070 con 0x7f4bec0b08f0 2024-04-25T10:23:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4be3400640 1 -- 172.21.15.87:0/3072419458 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00c940 con 0x7f4bec0b08f0 2024-04-25T10:23:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec09e240 con 0x7f4bec0b08f0 2024-04-25T10:23:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4be3400640 1 -- 172.21.15.87:0/3072419458 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00ccf0 con 0x7f4bec0b08f0 2024-04-25T10:23:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.029+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec09e800 con 0x7f4bec0b08f0 2024-04-25T10:23:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.033+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec04ef30 con 0x7f4bec0b08f0 2024-04-25T10:23:02.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.033+0000 7f4be3400640 1 -- 172.21.15.87:0/3072419458 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bdc01a070 con 0x7f4bec0b08f0 2024-04-25T10:23:02.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.033+0000 7f4be3400640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4bc8066f60 0x7f4bc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.033+0000 7f4be3400640 1 -- 172.21.15.87:0/3072419458 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bdc085290 con 0x7f4bec0b08f0 2024-04-25T10:23:02.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.033+0000 7f4be2000640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4bc8066f60 0x7f4bc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.033+0000 7f4be2000640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4bc8066f60 0x7f4bc8069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40b2000 tx=0x7f4bd40aa070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:02.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.037+0000 7f4be3400640 1 -- 172.21.15.87:0/3072419458 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4bdc0562a0 con 0x7f4bec0b08f0 2024-04-25T10:23:02.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.245+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4bec0b1f00 con 0x7f4bec0b08f0 2024-04-25T10:23:02.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.249+0000 7f4be3400640 1 -- 172.21.15.87:0/3072419458 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4bdc055ba0 con 0x7f4bec0b08f0 2024-04-25T10:23:02.254 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:23:02.254 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:23:02.254 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:23:02.254 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:23:02.254 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:23:02.254 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:23:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4bc8066f60 msgr2=0x7f4bc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4bc8066f60 0x7f4bc8069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40b2000 tx=0x7f4bd40aa070 comp rx=0 tx=0).stop 2024-04-25T10:23:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bec0b08f0 msgr2=0x7f4bec152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bec0b08f0 0x7f4bec152e90 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00e8a0 tx=0x7f4bdc00eca0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 shutdown_connections 2024-04-25T10:23:02.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4bc8066f60 0x7f4bc8069420 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec15dfb0 0x7f4bec09d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0b1430 0x7f4bec09b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 --2- 172.21.15.87:0/3072419458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bec0b08f0 0x7f4bec152e90 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 >> 172.21.15.87:0/3072419458 conn(0x7f4bec0b3580 msgr2=0x7f4bec1603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:02.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 shutdown_connections 2024-04-25T10:23:02.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.253+0000 7f4bf1600640 1 -- 172.21.15.87:0/3072419458 wait complete. 2024-04-25T10:23:02.276 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:02.375 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:23:02.373+0000 7fae21c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:23:02.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f1290200640 1 Processor -- start 2024-04-25T10:23:02.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f1290200640 1 -- start start 2024-04-25T10:23:02.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f1290200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1288156030 0x7f1288071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f1290200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f12880669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f1290200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128815c640 con 0x7f1288066ee0 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f1290200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1288067350 con 0x7f1288156030 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f1290200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128815d260 con 0x7f128815c0a0 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128d800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1288156030 0x7f1288071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128d800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1288156030 0x7f1288071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52164/0 (socket says 172.21.15.87:52164) 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128d800640 1 -- 172.21.15.87:0/2891872408 learned_addr learned my addr 172.21.15.87:0/2891872408 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128ce00640 1 --2- 172.21.15.87:0/2891872408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f12880669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128d800640 1 -- 172.21.15.87:0/2891872408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 msgr2=0x7f12880669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128d800640 1 --2- 172.21.15.87:0/2891872408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f12880669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128d800640 1 -- 172.21.15.87:0/2891872408 >> v1:172.21.15.169:6789/0 conn(0x7f1288066ee0 legacy=0x7f12880af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.409+0000 7f128d800640 1 -- 172.21.15.87:0/2891872408 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128815c920 con 0x7f1288156030 2024-04-25T10:23:02.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f128d800640 1 --2- 172.21.15.87:0/2891872408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f1288071f10 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f127c009870 tx=0x7f127c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d44616ff3a1e0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:02.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1287e00640 1 -- 172.21.15.87:0/2891872408 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0a95f0 con 0x7f1288156030 2024-04-25T10:23:02.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1287e00640 1 -- 172.21.15.87:0/2891872408 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c0a9790 con 0x7f1288156030 2024-04-25T10:23:02.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1287e00640 1 -- 172.21.15.87:0/2891872408 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0a9b60 con 0x7f1288156030 2024-04-25T10:23:02.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 -- 172.21.15.87:0/2891872408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 msgr2=0x7f1288071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 --2- 172.21.15.87:0/2891872408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f1288071f10 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f127c009870 tx=0x7f127c01c530 comp rx=0 tx=0).stop 2024-04-25T10:23:02.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 -- 172.21.15.87:0/2891872408 shutdown_connections 2024-04-25T10:23:02.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 --2- 172.21.15.87:0/2891872408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f12880669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 --2- 172.21.15.87:0/2891872408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f1288071f10 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 -- 172.21.15.87:0/2891872408 >> 172.21.15.87:0/2891872408 conn(0x7f128806ac90 msgr2=0x7f128806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:02.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 -- 172.21.15.87:0/2891872408 shutdown_connections 2024-04-25T10:23:02.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.413+0000 7f1290200640 1 -- 172.21.15.87:0/2891872408 wait complete. 2024-04-25T10:23:02.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 Processor -- start 2024-04-25T10:23:02.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 -- start start 2024-04-25T10:23:02.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1288066ee0 0x7f12881441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f12880b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f128815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1288156590 con 0x7f1288156030 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1288156410 con 0x7f1288066ee0 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128815d6f0 con 0x7f128815c0a0 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f12880b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f12880b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52176/0 (socket says 172.21.15.87:52176) 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 -- 172.21.15.87:0/557673053 learned_addr learned my addr 172.21.15.87:0/557673053 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128e200640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f128815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128d800640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1288066ee0 0x7f12881441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 -- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 msgr2=0x7f128815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f128815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 -- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1288066ee0 msgr2=0x7f12881441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1288066ee0 0x7f12881441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 -- 172.21.15.87:0/557673053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12880b4370 con 0x7f1288156030 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128d800640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1288066ee0 0x7f12881441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f128ce00640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f12880b38f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f127400d620 tx=0x7f127400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1285600640 1 -- 172.21.15.87:0/557673053 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127400b620 con 0x7f1288156030 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1285600640 1 -- 172.21.15.87:0/557673053 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1274019070 con 0x7f1288156030 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12880b4650 con 0x7f1288156030 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1285600640 1 -- 172.21.15.87:0/557673053 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127400b9e0 con 0x7f1288156030 2024-04-25T10:23:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12880b4c10 con 0x7f1288156030 2024-04-25T10:23:02.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.417+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12881545d0 con 0x7f1288156030 2024-04-25T10:23:02.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.421+0000 7f1285600640 1 -- 172.21.15.87:0/557673053 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f127400bb80 con 0x7f1288156030 2024-04-25T10:23:02.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.421+0000 7f1285600640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1268066f60 0x7f1268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:02.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.421+0000 7f1285600640 1 -- 172.21.15.87:0/557673053 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12740854a0 con 0x7f1288156030 2024-04-25T10:23:02.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.421+0000 7f128d800640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1268066f60 0x7f1268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:02.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.421+0000 7f128d800640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1268066f60 0x7f1268069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f127c0ae000 tx=0x7f127c0b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:02.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.425+0000 7f1285600640 1 -- 172.21.15.87:0/557673053 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12740564a0 con 0x7f1288156030 2024-04-25T10:23:02.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.609+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f128804ef30 con 0x7f1288156030 2024-04-25T10:23:02.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.609+0000 7f1285600640 1 -- 172.21.15.87:0/557673053 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4944 (secure 0 0 0) 0x7f1274055da0 con 0x7f1288156030 2024-04-25T10:23:02.614 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:02.615 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8755,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4075242775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4075242775},{"type":"v1","addr":"172.21.15.87:6836","nonce":4075242775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:22:56.766091+0000","modified":"2024-04-25T10:22:59.792927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8758":{"gid":8758,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1025107360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1025107360},{"type":"v1","addr":"172.21.15.87:6837","nonce":1025107360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:23:02.615 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 100 2024-04-25T10:23:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1268066f60 msgr2=0x7f1268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1268066f60 0x7f1268069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f127c0ae000 tx=0x7f127c0b3040 comp rx=0 tx=0).stop 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 msgr2=0x7f12880b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f12880b38f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f127400d620 tx=0x7f127400dae0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 shutdown_connections 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1268066f60 0x7f1268069420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128815c0a0 0x7f128815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1288156030 0x7f12880b38f0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 --2- 172.21.15.87:0/557673053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1288066ee0 0x7f12881441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 >> 172.21.15.87:0/557673053 conn(0x7f128806ac90 msgr2=0x7f1288166f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 shutdown_connections 2024-04-25T10:23:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:02.613+0000 7f1290200640 1 -- 172.21.15.87:0/557673053 wait complete. 2024-04-25T10:23:03.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:22:56.766091+0000', 'modified': '2024-04-25T10:22:59.792927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8758}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8758': {'gid': 8758, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/1025107360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 1025107360}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 1025107360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:23:03.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:23:03.039 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:23:03.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfea00640 1 Processor -- start 2024-04-25T10:23:03.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfea00640 1 -- start start 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7facf80cda10 0x7facf806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf80cf930 0x7facf80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf80b2770 con 0x7facf815bc40 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf806c670 con 0x7facf80cda10 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf806c950 con 0x7facf80cf930 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7facf815bc40 0x7facf80b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59764/0 (socket says 172.21.15.87:59764) 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf7e00640 1 -- 172.21.15.87:0/1083291098 learned_addr learned my addr 172.21.15.87:0/1083291098 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:03.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf7400640 1 --2- 172.21.15.87:0/1083291098 >> v2:172.21.15.87:3300/0 conn(0x7facf80cda10 0x7facf806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facfca00640 1 --2- 172.21.15.87:0/1083291098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf80cf930 0x7facf80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf7400640 1 -- 172.21.15.87:0/1083291098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf80cf930 msgr2=0x7facf80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf7400640 1 --2- 172.21.15.87:0/1083291098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf80cf930 0x7facf80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf7400640 1 -- 172.21.15.87:0/1083291098 >> v1:172.21.15.87:6789/0 conn(0x7facf815bc40 legacy=0x7facf80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf7400640 1 -- 172.21.15.87:0/1083291098 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf806cc30 con 0x7facf80cda10 2024-04-25T10:23:03.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.173+0000 7facf6a00640 1 -- 172.21.15.87:0/1083291098 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605430506 0 0) 0x7facf80b2770 con 0x7facf815bc40 2024-04-25T10:23:03.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facf7400640 1 --2- 172.21.15.87:0/1083291098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cda10 0x7facf806c090 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7face40097c0 tx=0x7face401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebfe7c577167427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facf6a00640 1 -- 172.21.15.87:0/1083291098 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40b0070 con 0x7facf80cda10 2024-04-25T10:23:03.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facf6a00640 1 -- 172.21.15.87:0/1083291098 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40ab600 con 0x7facf80cda10 2024-04-25T10:23:03.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facf6a00640 1 -- 172.21.15.87:0/1083291098 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40ab9d0 con 0x7facf80cda10 2024-04-25T10:23:03.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 -- 172.21.15.87:0/1083291098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cda10 msgr2=0x7facf806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 --2- 172.21.15.87:0/1083291098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cda10 0x7facf806c090 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7face40097c0 tx=0x7face401c410 comp rx=0 tx=0).stop 2024-04-25T10:23:03.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 -- 172.21.15.87:0/1083291098 shutdown_connections 2024-04-25T10:23:03.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 --2- 172.21.15.87:0/1083291098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf80cf930 0x7facf80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 --2- 172.21.15.87:0/1083291098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cda10 0x7facf806c090 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 -- 172.21.15.87:0/1083291098 >> 172.21.15.87:0/1083291098 conn(0x7facf8066b70 msgr2=0x7facf8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 -- 172.21.15.87:0/1083291098 shutdown_connections 2024-04-25T10:23:03.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.177+0000 7facfea00640 1 -- 172.21.15.87:0/1083291098 wait complete. 2024-04-25T10:23:03.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 Processor -- start 2024-04-25T10:23:03.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 -- start start 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf80cda10 0x7facf8152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cf930 0x7facf80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf815bc40 0x7facf80add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b3890 con 0x7facf80cf930 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b3710 con 0x7facf80cda10 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b3a10 con 0x7facf815bc40 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cf930 0x7facf80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cf930 0x7facf80ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52204/0 (socket says 172.21.15.87:52204) 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 -- 172.21.15.87:0/1195776470 learned_addr learned my addr 172.21.15.87:0/1195776470 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfca00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf815bc40 0x7facf80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7e00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf80cda10 0x7facf8152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 -- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf815bc40 msgr2=0x7facf80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf815bc40 0x7facf80add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 -- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf80cda10 msgr2=0x7facf8152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf80cda10 0x7facf8152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 -- 172.21.15.87:0/1195776470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8070310 con 0x7facf80cf930 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7e00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf80cda10 0x7facf8152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facf7400640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cf930 0x7facf80ad840 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7face401a040 tx=0x7face401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facefe00640 1 -- 172.21.15.87:0/1195776470 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40b0050 con 0x7facf80cf930 2024-04-25T10:23:03.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facefe00640 1 -- 172.21.15.87:0/1195776470 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40bc070 con 0x7facf80cf930 2024-04-25T10:23:03.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf8072200 con 0x7facf80cf930 2024-04-25T10:23:03.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facefe00640 1 -- 172.21.15.87:0/1195776470 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40aac00 con 0x7facf80cf930 2024-04-25T10:23:03.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.181+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf8070890 con 0x7facf80cf930 2024-04-25T10:23:03.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facefe00640 1 -- 172.21.15.87:0/1195776470 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7face40c1460 con 0x7facf80cf930 2024-04-25T10:23:03.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facefe00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facd8066f60 0x7facd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facefe00640 1 -- 172.21.15.87:0/1195776470 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7face411fa40 con 0x7facf80cf930 2024-04-25T10:23:03.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facf7e00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facd8066f60 0x7facd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 Processor -- start 2024-04-25T10:23:03.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 -- start start 2024-04-25T10:23:03.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facf7e00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facd8066f60 0x7facd8069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7face8000f70 tx=0x7face8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f3920 0x55b6e19f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f4240 0x55b6e19a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e18915f0 con 0x55b6e19f4240 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e17f8da0 con 0x55b6e19a9f30 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6e1781150 con 0x55b6e19f3920 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f4240 0x55b6e19a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f4240 0x55b6e19a99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52208/0 (socket says 172.21.15.87:52208) 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facf7400640 1 -- 172.21.15.87:0/2695498101 learned_addr learned my addr 172.21.15.87:0/2695498101 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.185+0000 7facf7e00640 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f3920 0x55b6e19f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facfca00640 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 -- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f3920 msgr2=0x55b6e19f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f3920 0x55b6e19f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 -- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 msgr2=0x55b6e19aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 -- 172.21.15.87:0/2695498101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e179aca0 con 0x55b6e19f4240 2024-04-25T10:23:03.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f4240 0x55b6e19a99f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7face40ec710 tx=0x7face4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707113e1ddde20e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facecc00640 1 -- 172.21.15.87:0/2695498101 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40f02b0 con 0x55b6e19f4240 2024-04-25T10:23:03.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facecc00640 1 -- 172.21.15.87:0/2695498101 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40b1040 con 0x55b6e19f4240 2024-04-25T10:23:03.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facecc00640 1 -- 172.21.15.87:0/2695498101 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40f4400 con 0x55b6e19f4240 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- 172.21.15.87:0/2695498101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f4240 msgr2=0x55b6e19a99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f4240 0x55b6e19a99f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7face40ec710 tx=0x7face4009190 comp rx=0 tx=0).stop 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- 172.21.15.87:0/2695498101 shutdown_connections 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f4240 0x55b6e19a99f0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 --2- 172.21.15.87:0/2695498101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f3920 0x55b6e19f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- 172.21.15.87:0/2695498101 >> 172.21.15.87:0/2695498101 conn(0x55b6e19b02a0 msgr2=0x55b6e19a82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- 172.21.15.87:0/2695498101 shutdown_connections 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- 172.21.15.87:0/2695498101 wait complete. 2024-04-25T10:23:03.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- start start 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 Processor -- start 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e17b3990 con 0x55b6e19f3920 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e17f4e40 con 0x55b6e19a9f30 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6e1784900 con 0x55b6e19f4240 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7fad01fad000 10 client.0 _ll_register_callbacks cb 0x55b6e18ac1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19fb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52216/0 (socket says 172.21.15.87:52216) 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7400640 1 -- 172.21.15.87:0/2407794277 learned_addr learned my addr 172.21.15.87:0/2407794277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:03.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.189+0000 7facf7e00640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facfca00640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7400640 1 -- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 msgr2=0x55b6e19adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7400640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7400640 1 -- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 msgr2=0x55b6e19fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7400640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7400640 1 -- 172.21.15.87:0/2407794277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6e17b1630 con 0x55b6e19f3920 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facfca00640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7400640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19fb1c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7face4097ce0 tx=0x7face40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f141e153f35ce7e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 1 -- 172.21.15.87:0/2407794277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face411b070 con 0x55b6e19f3920 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 1 -- 172.21.15.87:0/2407794277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7face410fda0 con 0x55b6e19f3920 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7face40b1040 con 0x55b6e19f3920 2024-04-25T10:23:03.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 1 -- 172.21.15.87:0/2407794277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7face40abe10 con 0x55b6e19f3920 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 10 client.9714 fetch_fsmap learned FSMap version 100 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55b6e17813a0 con 0x55b6e19f3920 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 1 -- 172.21.15.87:0/2407794277 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40b1040 con 0x55b6e19f3920 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 1 -- 172.21.15.87:0/2407794277 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 100) v1 ==== 3294+0+0 (secure 0 0 0) 0x7face40f8530 con 0x55b6e19f3920 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 10 client.9714 fetch_fsmap finished waiting for FSMap version 100 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 10 client.9714 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 20 client.9714 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e19ae4b0 0x55b6e19ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 4 client.9714 mds_command: new command op to 8758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 --> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6e1794a50 con 0x55b6e19ae4b0 2024-04-25T10:23:03.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7e00640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e19ae4b0 0x55b6e19ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7facf7e00640 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e19ae4b0 0x55b6e19ae890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c835292f1387cfc server_cookie=5f59a2e66e5281 in_seq=0 out_seq=0 2024-04-25T10:23:03.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 10 client.9714 ms_handle_connect on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:03.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 1 -- 172.21.15.87:0/2407794277 <== mds.0 v2:172.21.15.87:6835/1025107360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b6e1794a50 con 0x55b6e19ae4b0 2024-04-25T10:23:03.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 10 client.9714 handle_command_reply: tid=0 2024-04-25T10:23:03.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 client.9714 shutdown 2024-04-25T10:23:03.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 msgr2=0x55b6e19fb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19fb1c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7face4097ce0 tx=0x7face40191b0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 shutdown_connections 2024-04-25T10:23:03.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e19ae4b0 0x55b6e19ae890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19fb1c0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 --2- 172.21.15.87:0/2407794277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7face1600640 0 client.9714 ms_handle_reset on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 >> 172.21.15.87:0/2407794277 conn(0x55b6e19b02a0 msgr2=0x55b6e19a89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 shutdown_connections 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 1 -- 172.21.15.87:0/2407794277 wait complete. 2024-04-25T10:23:03.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.193+0000 7fad01fad000 20 client.9714 trim_cache size 0 max 16384 2024-04-25T10:23:03.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 Processor -- start 2024-04-25T10:23:03.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- start start 2024-04-25T10:23:03.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19e0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19fd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf81f7880 con 0x55b6e19f3920 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf815cbb0 con 0x55b6e19a9f30 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf81f7280 con 0x55b6e19f4240 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19e0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19e0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52220/0 (socket says 172.21.15.87:52220) 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/253107267 learned_addr learned my addr 172.21.15.87:0/253107267 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7e00640 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facfca00640 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 msgr2=0x55b6e19fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 msgr2=0x55b6e19dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19dfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/253107267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e19f4070 con 0x55b6e19f3920 2024-04-25T10:23:03.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7e00640 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:03.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19e0190 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7face4000c10 tx=0x7face4109050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4a99aada334b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7face3e00640 1 -- 172.21.15.87:0/253107267 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40efc10 con 0x55b6e19f3920 2024-04-25T10:23:03.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7face3e00640 1 -- 172.21.15.87:0/253107267 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40f4400 con 0x55b6e19f3920 2024-04-25T10:23:03.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7face3e00640 1 -- 172.21.15.87:0/253107267 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40ee5b0 con 0x55b6e19f3920 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- 172.21.15.87:0/253107267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 msgr2=0x55b6e19e0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19e0190 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7face4000c10 tx=0x7face4109050 comp rx=0 tx=0).stop 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- 172.21.15.87:0/253107267 shutdown_connections 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e19fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e19e0190 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- 172.21.15.87:0/253107267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e19dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- 172.21.15.87:0/253107267 >> 172.21.15.87:0/253107267 conn(0x55b6e19b02a0 msgr2=0x55b6e19ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- 172.21.15.87:0/253107267 shutdown_connections 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- 172.21.15.87:0/253107267 wait complete. 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- start start 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 Processor -- start 2024-04-25T10:23:03.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e1a00db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e1a4c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e1a4ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e17fd3d0 con 0x55b6e19f3920 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e18b4d50 con 0x55b6e19a9f30 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6e1793980 con 0x55b6e19f4240 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7fad01fad000 10 client.0 _ll_register_callbacks cb 0x7face4006c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e1a4c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e1a4c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52236/0 (socket says 172.21.15.87:52236) 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facfca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e1a4ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/3120218134 learned_addr learned my addr 172.21.15.87:0/3120218134 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7e00640 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e1a00db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/3120218134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 msgr2=0x55b6e1a4ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e1a4ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/3120218134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 msgr2=0x55b6e1a00db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e1a00db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.213+0000 7facf7400640 1 -- 172.21.15.87:0/3120218134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6e187a600 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facf7400640 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e1a4c910 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7face4006cf0 tx=0x7face40b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1809be93ef572d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 1 -- 172.21.15.87:0/3120218134 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40f8530 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 1 -- 172.21.15.87:0/3120218134 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7face4117380 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7face40f4400 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 1 -- 172.21.15.87:0/3120218134 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7face4117580 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 10 client.9720 fetch_fsmap learned FSMap version 100 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55b6e178d9b0 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 1 -- 172.21.15.87:0/3120218134 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40f4400 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 1 -- 172.21.15.87:0/3120218134 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 100) v1 ==== 3294+0+0 (secure 0 0 0) 0x7face40f4020 con 0x55b6e19f3920 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 10 client.9720 fetch_fsmap finished waiting for FSMap version 100 2024-04-25T10:23:03.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 10 client.9720 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 20 client.9720 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e1a4d390 0x55b6e1a53e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 4 client.9720 mds_command: new command op to 8758 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 --> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b6e18062e0 con 0x55b6e1a4d390 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facf7e00640 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e1a4d390 0x55b6e1a53e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facf7e00640 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e1a4d390 0x55b6e1a53e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1449f27c347623f4 server_cookie=54d98b35591ffb99 in_seq=0 out_seq=0 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 10 client.9720 ms_handle_connect on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 1 -- 172.21.15.87:0/3120218134 <== mds.0 v2:172.21.15.87:6835/1025107360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55b6e18062e0 con 0x55b6e1a4d390 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 10 client.9720 handle_command_reply: tid=0 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 client.9720 shutdown 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 msgr2=0x55b6e1a4c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e1a4c910 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7face4006cf0 tx=0x7face40b5000 comp rx=0 tx=0).stop 2024-04-25T10:23:03.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 shutdown_connections 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55b6e1a4d390 0x55b6e1a53e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b6e19f4240 0x55b6e1a4ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b6e19f3920 0x55b6e1a4c910 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 --2- 172.21.15.87:0/3120218134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6e19a9f30 0x55b6e1a00db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7face1600640 0 client.9720 ms_handle_reset on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 >> 172.21.15.87:0/3120218134 conn(0x55b6e19b02a0 msgr2=0x55b6e19dec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 shutdown_connections 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 1 -- 172.21.15.87:0/3120218134 wait complete. 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7fad01fad000 20 client.9720 trim_cache size 0 max 16384 2024-04-25T10:23:03.227 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 8758, 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8075481050000004, 2024-04-25T10:23:03.228 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 100, 2024-04-25T10:23:03.229 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 138, 2024-04-25T10:23:03.229 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 138, 2024-04-25T10:23:03.229 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 11.728224624999999 2024-04-25T10:23:03.229 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:23:03.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facd8066f60 msgr2=0x7facd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facd8066f60 0x7facd8069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7face8000f70 tx=0x7face8009040 comp rx=0 tx=0).stop 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cf930 msgr2=0x7facf80ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cf930 0x7facf80ad840 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7face401a040 tx=0x7face401ac50 comp rx=0 tx=0).stop 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 shutdown_connections 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7facd8066f60 0x7facd8069420 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facf815bc40 0x7facf80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf80cf930 0x7facf80ad840 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 --2- 172.21.15.87:0/1195776470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf80cda10 0x7facf8152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 >> 172.21.15.87:0/1195776470 conn(0x7facf8066b70 msgr2=0x7facf8155cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 shutdown_connections 2024-04-25T10:23:03.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.217+0000 7facfea00640 1 -- 172.21.15.87:0/1195776470 wait complete. 2024-04-25T10:23:03.241 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:23:03.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.373+0000 7f61d8a00640 1 Processor -- start 2024-04-25T10:23:03.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.373+0000 7f61d8a00640 1 -- start start 2024-04-25T10:23:03.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f61d40ca730 0x7f61d406c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d8a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d406d060 con 0x7f61d406c8b0 2024-04-25T10:23:03.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d406d340 con 0x7f61d415b390 2024-04-25T10:23:03.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d406d620 con 0x7f61d40ca730 2024-04-25T10:23:03.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f61d406c8b0 0x7f61d40cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:59772/0 (socket says 172.21.15.87:59772) 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d3e00640 1 -- 172.21.15.87:0/3445613907 learned_addr learned my addr 172.21.15.87:0/3445613907 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2a00640 1 --2- 172.21.15.87:0/3445613907 >> v2:172.21.15.169:3300/0 conn(0x7f61d40ca730 0x7f61d406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105716501 0 0) 0x7f61d406d060 con 0x7f61d406c8b0 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b8003450 con 0x7f61d406c8b0 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739918102 0 0) 0x7f61b8003450 con 0x7f61d406c8b0 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 >> v2:172.21.15.169:3300/0 conn(0x7f61d40ca730 msgr2=0x7f61d406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 --2- 172.21.15.87:0/3445613907 >> v2:172.21.15.169:3300/0 conn(0x7f61d40ca730 0x7f61d406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 >> v1:172.21.15.169:6789/0 conn(0x7f61d415b390 legacy=0x7f61d40ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d406d900 con 0x7f61d406c8b0 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f61cc001680 con 0x7f61d406c8b0 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 >> v1:172.21.15.87:6789/0 conn(0x7f61d406c8b0 legacy=0x7f61d40cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b8003ab0 0x7f61b8003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61b800ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b800b4f0 0x7f61b800d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406d620 con 0x7f61b8008860 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b800e290 con 0x7f61b8003ab0 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b800e490 con 0x7f61b800b4f0 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2a00640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61b800ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d3400640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b8003ab0 0x7f61b8003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d3e00640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b800b4f0 0x7f61b800d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2a00640 1 -- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b800b4f0 msgr2=0x7f61b800d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2a00640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b800b4f0 0x7f61b800d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2a00640 1 -- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b8003ab0 msgr2=0x7f61b8003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2a00640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b8003ab0 0x7f61b8003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d2a00640 1 -- 172.21.15.87:0/3445613907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8003450 con 0x7f61b8008860 2024-04-25T10:23:03.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.377+0000 7f61d3e00640 1 -- 172.21.15.87:0/3445613907 reap_dead start 2024-04-25T10:23:03.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d2a00640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61b800ac50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009ae0 tx=0x7f61c001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b74a44dffe82cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b2070 con 0x7f61b8008860 2024-04-25T10:23:03.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00a5660 con 0x7f61b8008860 2024-04-25T10:23:03.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d2000640 1 -- 172.21.15.87:0/3445613907 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00096b0 con 0x7f61b8008860 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d8a00640 1 -- 172.21.15.87:0/3445613907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 msgr2=0x7f61b800ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d8a00640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61b800ac50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009ae0 tx=0x7f61c001ca70 comp rx=0 tx=0).stop 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d8a00640 1 -- 172.21.15.87:0/3445613907 shutdown_connections 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d8a00640 1 --2- 172.21.15.87:0/3445613907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61b800ac50 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d8a00640 1 -- 172.21.15.87:0/3445613907 >> 172.21.15.87:0/3445613907 conn(0x7f61d40b16a0 msgr2=0x7f61d40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d8a00640 1 -- 172.21.15.87:0/3445613907 shutdown_connections 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.381+0000 7f61d8a00640 1 -- 172.21.15.87:0/3445613907 wait complete. 2024-04-25T10:23:03.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 Processor -- start 2024-04-25T10:23:03.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 -- start start 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61d41f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d4151060 0x7f61d41514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d4144230 0x7f61d41446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406c370 con 0x7f61b8008860 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406ccd0 con 0x7f61d4151060 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406c4f0 con 0x7f61d4144230 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61d41f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61d41f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52252/0 (socket says 172.21.15.87:52252) 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 -- 172.21.15.87:0/2199758613 learned_addr learned my addr 172.21.15.87:0/2199758613 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d2a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d4151060 0x7f61d41514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3e00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d4144230 0x7f61d41446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 -- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d4144230 msgr2=0x7f61d41446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d4144230 0x7f61d41446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 -- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d4151060 msgr2=0x7f61d41514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d4151060 0x7f61d41514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 -- 172.21.15.87:0/2199758613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4144f90 con 0x7f61b8008860 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d2a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d4151060 0x7f61d41514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d3400640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61d41f78d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f61c4011d60 tx=0x7f61c400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61cb400640 1 -- 172.21.15.87:0/2199758613 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4019020 con 0x7f61b8008860 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61cb400640 1 -- 172.21.15.87:0/2199758613 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c400fe40 con 0x7f61b8008860 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d41f3af0 con 0x7f61b8008860 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61cb400640 1 -- 172.21.15.87:0/2199758613 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4003bc0 con 0x7f61b8008860 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d41f40b0 con 0x7f61b8008860 2024-04-25T10:23:03.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.385+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d4075120 con 0x7f61b8008860 2024-04-25T10:23:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.389+0000 7f61cb400640 1 -- 172.21.15.87:0/2199758613 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61c401c020 con 0x7f61b8008860 2024-04-25T10:23:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.389+0000 7f61cb400640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61b4066f90 0x7f61b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.389+0000 7f61cb400640 1 -- 172.21.15.87:0/2199758613 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61c40889e0 con 0x7f61b8008860 2024-04-25T10:23:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.389+0000 7f61d2a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61b4066f90 0x7f61b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:03.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.389+0000 7f61d2a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61b4066f90 0x7f61b4069450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f61c00af000 tx=0x7f61c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:03.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.393+0000 7f61cb400640 1 -- 172.21.15.87:0/2199758613 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61c4023020 con 0x7f61b8008860 2024-04-25T10:23:03.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61d40728b0 con 0x7f61b8008860 2024-04-25T10:23:03.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61cb400640 1 -- 172.21.15.87:0/2199758613 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7f61c4059950 con 0x7f61b8008860 2024-04-25T10:23:03.591 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:23:03.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61b4066f90 msgr2=0x7f61b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61b4066f90 0x7f61b4069450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f61c00af000 tx=0x7f61c0008040 comp rx=0 tx=0).stop 2024-04-25T10:23:03.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 msgr2=0x7f61d41f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:03.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61d41f78d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f61c4011d60 tx=0x7f61c400f5a0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 shutdown_connections 2024-04-25T10:23:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61b4066f90 0x7f61b4069450 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d4144230 0x7f61d41446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d4151060 0x7f61d41514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 --2- 172.21.15.87:0/2199758613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61b8008860 0x7f61d41f78d0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 >> 172.21.15.87:0/2199758613 conn(0x7f61d40b16a0 msgr2=0x7f61d415b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 shutdown_connections 2024-04-25T10:23:03.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:03.589+0000 7f61d8a00640 1 -- 172.21.15.87:0/2199758613 wait complete. 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:23:03.613 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:23:03.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:03.614 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:23:03.623 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:23:03.623 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:23:03.623 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:23:03.623 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.623 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:23:03.623 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41830sec preferred_lft 41830sec 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.624 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.625 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:23:03.626 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:23:03.626 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:23:03.626 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:03.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:03.626 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:23:03.684 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:23:03.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:03.686 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:23:03.739 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:23:03.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:03.740 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:23:03.740 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:03.740 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:23:03.740 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:23:03.740 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:23:03.740 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:03.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:03.830 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:23:03.838 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:23:03.838 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:23:03.839 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:23:03.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:03.839 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:23:03.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:23:03.901 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:03.950 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:23:03.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:23:03.951 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:04.005 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:23:04.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:04.006 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:04.093 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:04.094 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:23:04.094 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:23:04.108 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:04.109 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:23:04.109 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:23:04.109 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:23:04.109 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:23:04.116 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:04.116 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:23:04.116 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:23:04.116 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:23:04.116 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:23:04.116 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:23:04.117 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:23:04.117 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:23:04.120 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:23:04.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:04.120 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:04.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:04.140 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:23:04.198 INFO:teuthology.orchestra.run.smithi087.stdout:client9729 2024-04-25T10:23:04.199 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:23:04.199 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:04.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:04.251 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:04.251 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:04.251 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:23:04.251 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:04.251 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:04.251 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:04.251 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:04.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.331 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.331 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162233 s, 64.6 MB/s 2024-04-25T10:23:04.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.355 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.355 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227785 s, 46.0 MB/s 2024-04-25T10:23:04.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.383 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.383 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261367 s, 40.1 MB/s 2024-04-25T10:23:04.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211551 s, 49.6 MB/s 2024-04-25T10:23:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196051 s, 53.5 MB/s 2024-04-25T10:23:04.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247104 s, 42.4 MB/s 2024-04-25T10:23:04.475 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.475 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.475 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196802 s, 53.3 MB/s 2024-04-25T10:23:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166669 s, 62.9 MB/s 2024-04-25T10:23:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.513 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175825 s, 59.6 MB/s 2024-04-25T10:23:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208456 s, 50.3 MB/s 2024-04-25T10:23:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.555 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.555 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169333 s, 61.9 MB/s 2024-04-25T10:23:04.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.578 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.578 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212129 s, 49.4 MB/s 2024-04-25T10:23:04.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.599 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.599 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198288 s, 52.9 MB/s 2024-04-25T10:23:04.625 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.625 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.625 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240025 s, 43.7 MB/s 2024-04-25T10:23:04.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.645 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.645 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178998 s, 58.6 MB/s 2024-04-25T10:23:04.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.664 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179631 s, 58.4 MB/s 2024-04-25T10:23:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.686 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196811 s, 53.3 MB/s 2024-04-25T10:23:04.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173798 s, 60.3 MB/s 2024-04-25T10:23:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.723 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167287 s, 62.7 MB/s 2024-04-25T10:23:04.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.742 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.742 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168371 s, 62.3 MB/s 2024-04-25T10:23:04.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.763 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196742 s, 53.3 MB/s 2024-04-25T10:23:04.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.781 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162177 s, 64.7 MB/s 2024-04-25T10:23:04.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.802 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.802 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189701 s, 55.3 MB/s 2024-04-25T10:23:04.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.818 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144611 s, 72.5 MB/s 2024-04-25T10:23:04.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.837 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.837 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172463 s, 60.8 MB/s 2024-04-25T10:23:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219138 s, 47.9 MB/s 2024-04-25T10:23:04.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.875 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.875 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123051 s, 85.2 MB/s 2024-04-25T10:23:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196712 s, 53.3 MB/s 2024-04-25T10:23:04.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.915 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169004 s, 62.0 MB/s 2024-04-25T10:23:04.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.936 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192608 s, 54.4 MB/s 2024-04-25T10:23:04.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.958 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.958 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202961 s, 51.7 MB/s 2024-04-25T10:23:04.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:04.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:04.988 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252617 s, 41.5 MB/s 2024-04-25T10:23:04.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:04.989 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:04.989 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:04.989 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:23:04.989 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:04.990 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:04.990 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:04.990 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.069 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210889 s, 49.7 MB/s 2024-04-25T10:23:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018941 s, 55.4 MB/s 2024-04-25T10:23:05.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182001 s, 57.6 MB/s 2024-04-25T10:23:05.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.128 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169745 s, 61.8 MB/s 2024-04-25T10:23:05.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.149 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188639 s, 55.6 MB/s 2024-04-25T10:23:05.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.169 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183566 s, 57.1 MB/s 2024-04-25T10:23:05.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.189 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018225 s, 57.5 MB/s 2024-04-25T10:23:05.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163055 s, 64.3 MB/s 2024-04-25T10:23:05.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155438 s, 67.5 MB/s 2024-04-25T10:23:05.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176753 s, 59.3 MB/s 2024-04-25T10:23:05.261 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.261 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.261 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154236 s, 68.0 MB/s 2024-04-25T10:23:05.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.282 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183863 s, 57.0 MB/s 2024-04-25T10:23:05.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.298 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146327 s, 71.7 MB/s 2024-04-25T10:23:05.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.317 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171233 s, 61.2 MB/s 2024-04-25T10:23:05.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.333 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148769 s, 70.5 MB/s 2024-04-25T10:23:05.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.352 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168111 s, 62.4 MB/s 2024-04-25T10:23:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203357 s, 51.6 MB/s 2024-04-25T10:23:05.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173751 s, 60.3 MB/s 2024-04-25T10:23:05.415 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.415 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.415 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019814 s, 52.9 MB/s 2024-04-25T10:23:05.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.436 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190609 s, 55.0 MB/s 2024-04-25T10:23:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172163 s, 60.9 MB/s 2024-04-25T10:23:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169819 s, 61.7 MB/s 2024-04-25T10:23:05.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.495 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.495 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199624 s, 52.5 MB/s 2024-04-25T10:23:05.512 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.512 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.512 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152038 s, 69.0 MB/s 2024-04-25T10:23:05.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.535 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209993 s, 49.9 MB/s 2024-04-25T10:23:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.558 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211394 s, 49.6 MB/s 2024-04-25T10:23:05.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.579 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192839 s, 54.4 MB/s 2024-04-25T10:23:05.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.602 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210031 s, 49.9 MB/s 2024-04-25T10:23:05.623 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.623 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.623 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197423 s, 53.1 MB/s 2024-04-25T10:23:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188276 s, 55.7 MB/s 2024-04-25T10:23:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205558 s, 51.0 MB/s 2024-04-25T10:23:05.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.683 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.683 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151846 s, 69.1 MB/s 2024-04-25T10:23:05.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:05.684 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:05.684 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:05.685 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:23:05.685 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:05.685 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:05.685 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:05.685 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:05.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.754 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014669 s, 71.5 MB/s 2024-04-25T10:23:05.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.774 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.774 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177418 s, 59.1 MB/s 2024-04-25T10:23:05.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.796 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.796 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200065 s, 52.4 MB/s 2024-04-25T10:23:05.814 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.814 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.814 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017159 s, 61.1 MB/s 2024-04-25T10:23:05.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225417 s, 46.5 MB/s 2024-04-25T10:23:05.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.861 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.861 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197399 s, 53.1 MB/s 2024-04-25T10:23:05.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.880 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184895 s, 56.7 MB/s 2024-04-25T10:23:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178708 s, 58.7 MB/s 2024-04-25T10:23:05.916 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.917 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147266 s, 71.2 MB/s 2024-04-25T10:23:05.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159599 s, 65.7 MB/s 2024-04-25T10:23:05.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.954 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176601 s, 59.4 MB/s 2024-04-25T10:23:05.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174207 s, 60.2 MB/s 2024-04-25T10:23:05.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:05.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:05.990 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148196 s, 70.8 MB/s 2024-04-25T10:23:06.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.011 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198267 s, 52.9 MB/s 2024-04-25T10:23:06.033 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.034 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.034 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209326 s, 50.1 MB/s 2024-04-25T10:23:06.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187675 s, 55.9 MB/s 2024-04-25T10:23:06.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177226 s, 59.2 MB/s 2024-04-25T10:23:06.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.092 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167932 s, 62.4 MB/s 2024-04-25T10:23:06.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160007 s, 65.5 MB/s 2024-04-25T10:23:06.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.132 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198661 s, 52.8 MB/s 2024-04-25T10:23:06.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.152 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018301 s, 57.3 MB/s 2024-04-25T10:23:06.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.170 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016284 s, 64.4 MB/s 2024-04-25T10:23:06.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.191 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192061 s, 54.6 MB/s 2024-04-25T10:23:06.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162631 s, 64.5 MB/s 2024-04-25T10:23:06.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01855 s, 56.5 MB/s 2024-04-25T10:23:06.249 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.250 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189119 s, 55.4 MB/s 2024-04-25T10:23:06.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.267 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.267 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152302 s, 68.8 MB/s 2024-04-25T10:23:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.284 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147422 s, 71.1 MB/s 2024-04-25T10:23:06.300 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.301 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156659 s, 66.9 MB/s 2024-04-25T10:23:06.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.318 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.318 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151826 s, 69.1 MB/s 2024-04-25T10:23:06.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.335 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.335 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150034 s, 69.9 MB/s 2024-04-25T10:23:06.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.351 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.351 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143883 s, 72.9 MB/s 2024-04-25T10:23:06.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:06.352 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:06.352 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:06.352 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:23:06.352 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:06.352 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:06.353 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:06.353 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:06.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.429 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176261 s, 59.5 MB/s 2024-04-25T10:23:06.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.447 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157096 s, 66.7 MB/s 2024-04-25T10:23:06.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.468 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019351 s, 54.2 MB/s 2024-04-25T10:23:06.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.485 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147574 s, 71.1 MB/s 2024-04-25T10:23:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.502 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153667 s, 68.2 MB/s 2024-04-25T10:23:06.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.525 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.525 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204777 s, 51.2 MB/s 2024-04-25T10:23:06.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.541 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147237 s, 71.2 MB/s 2024-04-25T10:23:06.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.558 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151405 s, 69.3 MB/s 2024-04-25T10:23:06.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.574 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142674 s, 73.5 MB/s 2024-04-25T10:23:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150206 s, 69.8 MB/s 2024-04-25T10:23:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205439 s, 51.0 MB/s 2024-04-25T10:23:06.634 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194824 s, 53.8 MB/s 2024-04-25T10:23:06.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157974 s, 66.4 MB/s 2024-04-25T10:23:06.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.670 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163789 s, 64.0 MB/s 2024-04-25T10:23:06.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.686 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140561 s, 74.6 MB/s 2024-04-25T10:23:06.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203577 s, 51.5 MB/s 2024-04-25T10:23:06.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150996 s, 69.4 MB/s 2024-04-25T10:23:06.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.743 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.743 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164279 s, 63.8 MB/s 2024-04-25T10:23:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.761 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165663 s, 63.3 MB/s 2024-04-25T10:23:06.780 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.780 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.780 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172825 s, 60.7 MB/s 2024-04-25T10:23:06.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.795 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131321 s, 79.8 MB/s 2024-04-25T10:23:06.809 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.810 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.810 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128129 s, 81.8 MB/s 2024-04-25T10:23:06.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.826 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149045 s, 70.4 MB/s 2024-04-25T10:23:06.842 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.842 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.842 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139681 s, 75.1 MB/s 2024-04-25T10:23:06.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.858 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145562 s, 72.0 MB/s 2024-04-25T10:23:06.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.876 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158254 s, 66.3 MB/s 2024-04-25T10:23:06.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.895 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.895 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170611 s, 61.5 MB/s 2024-04-25T10:23:06.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.913 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164797 s, 63.6 MB/s 2024-04-25T10:23:06.931 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.931 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.932 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166977 s, 62.8 MB/s 2024-04-25T10:23:06.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.948 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143791 s, 72.9 MB/s 2024-04-25T10:23:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123 s, 85.3 MB/s 2024-04-25T10:23:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:06.984 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209006 s, 50.2 MB/s 2024-04-25T10:23:06.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:06.986 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:06.986 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:06.986 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:23:06.986 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:06.986 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:06.986 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:06.986 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:07.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.056 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.056 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167198 s, 62.7 MB/s 2024-04-25T10:23:07.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.073 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147565 s, 71.1 MB/s 2024-04-25T10:23:07.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.089 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.089 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148464 s, 70.6 MB/s 2024-04-25T10:23:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166877 s, 62.8 MB/s 2024-04-25T10:23:07.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.132 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223305 s, 47.0 MB/s 2024-04-25T10:23:07.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.149 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153146 s, 68.5 MB/s 2024-04-25T10:23:07.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.166 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.166 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151354 s, 69.3 MB/s 2024-04-25T10:23:07.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164601 s, 63.7 MB/s 2024-04-25T10:23:07.203 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.203 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.203 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175007 s, 59.9 MB/s 2024-04-25T10:23:07.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.228 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229476 s, 45.7 MB/s 2024-04-25T10:23:07.245 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.246 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159443 s, 65.8 MB/s 2024-04-25T10:23:07.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.263 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.263 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015942 s, 65.8 MB/s 2024-04-25T10:23:07.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.281 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.281 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153949 s, 68.1 MB/s 2024-04-25T10:23:07.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.302 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.302 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193249 s, 54.3 MB/s 2024-04-25T10:23:07.319 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.320 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.320 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162584 s, 64.5 MB/s 2024-04-25T10:23:07.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181396 s, 57.8 MB/s 2024-04-25T10:23:07.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.770 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.428003 s, 2.4 MB/s 2024-04-25T10:23:07.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.787 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159132 s, 65.9 MB/s 2024-04-25T10:23:07.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.806 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016278 s, 64.4 MB/s 2024-04-25T10:23:07.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.826 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.826 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191042 s, 54.9 MB/s 2024-04-25T10:23:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.841 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130657 s, 80.3 MB/s 2024-04-25T10:23:07.864 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.865 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216468 s, 48.4 MB/s 2024-04-25T10:23:07.882 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.882 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.882 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154956 s, 67.7 MB/s 2024-04-25T10:23:07.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.904 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194358 s, 54.0 MB/s 2024-04-25T10:23:07.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.930 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245612 s, 42.7 MB/s 2024-04-25T10:23:07.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.947 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015638 s, 67.1 MB/s 2024-04-25T10:23:07.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.970 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.970 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208439 s, 50.3 MB/s 2024-04-25T10:23:07.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:07.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:07.989 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179722 s, 58.3 MB/s 2024-04-25T10:23:08.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.009 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178657 s, 58.7 MB/s 2024-04-25T10:23:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.026 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149943 s, 69.9 MB/s 2024-04-25T10:23:08.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.048 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020803 s, 50.4 MB/s 2024-04-25T10:23:08.068 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.069 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185121 s, 56.6 MB/s 2024-04-25T10:23:08.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:08.070 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:08.070 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:08.070 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:23:08.070 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:08.070 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:08.070 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:08.071 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:08.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.140 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120633 s, 86.9 MB/s 2024-04-25T10:23:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209435 s, 50.1 MB/s 2024-04-25T10:23:08.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.181 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169131 s, 62.0 MB/s 2024-04-25T10:23:08.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160069 s, 65.5 MB/s 2024-04-25T10:23:08.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.215 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.215 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151817 s, 69.1 MB/s 2024-04-25T10:23:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122124 s, 85.9 MB/s 2024-04-25T10:23:08.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.247 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015892 s, 66.0 MB/s 2024-04-25T10:23:08.272 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.272 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.272 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229855 s, 45.6 MB/s 2024-04-25T10:23:08.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.294 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.294 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207216 s, 50.6 MB/s 2024-04-25T10:23:08.314 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.314 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.314 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182439 s, 57.5 MB/s 2024-04-25T10:23:08.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.330 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.330 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137915 s, 76.0 MB/s 2024-04-25T10:23:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164612 s, 63.7 MB/s 2024-04-25T10:23:08.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.366 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169738 s, 61.8 MB/s 2024-04-25T10:23:08.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.388 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019463 s, 53.9 MB/s 2024-04-25T10:23:08.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.410 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207412 s, 50.6 MB/s 2024-04-25T10:23:08.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.426 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141979 s, 73.9 MB/s 2024-04-25T10:23:08.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.448 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201634 s, 52.0 MB/s 2024-04-25T10:23:08.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021179 s, 49.5 MB/s 2024-04-25T10:23:08.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182181 s, 57.6 MB/s 2024-04-25T10:23:08.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.513 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.513 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205219 s, 51.1 MB/s 2024-04-25T10:23:08.531 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.531 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.531 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158719 s, 66.1 MB/s 2024-04-25T10:23:08.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.549 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.549 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158275 s, 66.3 MB/s 2024-04-25T10:23:08.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.569 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.569 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184962 s, 56.7 MB/s 2024-04-25T10:23:08.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.586 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148671 s, 70.5 MB/s 2024-04-25T10:23:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.608 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202002 s, 51.9 MB/s 2024-04-25T10:23:08.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164967 s, 63.6 MB/s 2024-04-25T10:23:08.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.642 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.642 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143968 s, 72.8 MB/s 2024-04-25T10:23:08.659 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.659 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.659 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148464 s, 70.6 MB/s 2024-04-25T10:23:08.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.678 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.678 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172296 s, 60.9 MB/s 2024-04-25T10:23:08.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.697 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169015 s, 62.0 MB/s 2024-04-25T10:23:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188551 s, 55.6 MB/s 2024-04-25T10:23:08.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.736 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01681 s, 62.4 MB/s 2024-04-25T10:23:08.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:08.737 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:08.737 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:08.737 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:23:08.737 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:08.737 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:08.737 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:08.737 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:08.810 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.811 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.811 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191941 s, 54.6 MB/s 2024-04-25T10:23:08.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.829 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163567 s, 64.1 MB/s 2024-04-25T10:23:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.845 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147897 s, 70.9 MB/s 2024-04-25T10:23:08.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.865 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180055 s, 58.2 MB/s 2024-04-25T10:23:08.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.881 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014661 s, 71.5 MB/s 2024-04-25T10:23:08.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149013 s, 70.4 MB/s 2024-04-25T10:23:08.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.919 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189705 s, 55.3 MB/s 2024-04-25T10:23:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149836 s, 70.0 MB/s 2024-04-25T10:23:08.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.954 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.954 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162162 s, 64.7 MB/s 2024-04-25T10:23:08.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178847 s, 58.6 MB/s 2024-04-25T10:23:08.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:08.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:08.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208585 s, 50.3 MB/s 2024-04-25T10:23:09.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172714 s, 60.7 MB/s 2024-04-25T10:23:09.032 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.032 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.032 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149026 s, 70.4 MB/s 2024-04-25T10:23:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.058 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245425 s, 42.7 MB/s 2024-04-25T10:23:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145383 s, 72.1 MB/s 2024-04-25T10:23:09.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.094 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018043 s, 58.1 MB/s 2024-04-25T10:23:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.111 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149777 s, 70.0 MB/s 2024-04-25T10:23:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203717 s, 51.5 MB/s 2024-04-25T10:23:09.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.154 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193403 s, 54.2 MB/s 2024-04-25T10:23:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161429 s, 65.0 MB/s 2024-04-25T10:23:09.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.191 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.191 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017195 s, 61.0 MB/s 2024-04-25T10:23:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151554 s, 69.2 MB/s 2024-04-25T10:23:09.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197806 s, 53.0 MB/s 2024-04-25T10:23:09.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.251 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.252 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201082 s, 52.1 MB/s 2024-04-25T10:23:09.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.269 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159065 s, 65.9 MB/s 2024-04-25T10:23:09.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.291 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.291 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202715 s, 51.7 MB/s 2024-04-25T10:23:09.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.308 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.308 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153017 s, 68.5 MB/s 2024-04-25T10:23:09.324 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.325 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.325 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014607 s, 71.8 MB/s 2024-04-25T10:23:09.343 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.343 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.343 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164997 s, 63.6 MB/s 2024-04-25T10:23:09.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.364 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191504 s, 54.8 MB/s 2024-04-25T10:23:09.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.384 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185746 s, 56.5 MB/s 2024-04-25T10:23:09.400 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.401 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.401 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146613 s, 71.5 MB/s 2024-04-25T10:23:09.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:09.402 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:09.402 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:09.402 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:23:09.402 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:09.402 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:09.402 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:09.402 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:09.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.471 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.471 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152848 s, 68.6 MB/s 2024-04-25T10:23:09.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.487 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143698 s, 73.0 MB/s 2024-04-25T10:23:09.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.504 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.504 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014869 s, 70.5 MB/s 2024-04-25T10:23:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.521 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156727 s, 66.9 MB/s 2024-04-25T10:23:09.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.538 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153408 s, 68.4 MB/s 2024-04-25T10:23:09.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197779 s, 53.0 MB/s 2024-04-25T10:23:09.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.582 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.582 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202798 s, 51.7 MB/s 2024-04-25T10:23:09.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.600 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.600 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159695 s, 65.7 MB/s 2024-04-25T10:23:09.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.616 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.616 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146531 s, 71.6 MB/s 2024-04-25T10:23:09.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.633 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.633 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155892 s, 67.3 MB/s 2024-04-25T10:23:09.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017293 s, 60.6 MB/s 2024-04-25T10:23:09.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.669 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148817 s, 70.5 MB/s 2024-04-25T10:23:09.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.689 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182674 s, 57.4 MB/s 2024-04-25T10:23:09.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.708 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.708 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171614 s, 61.1 MB/s 2024-04-25T10:23:09.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.725 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149092 s, 70.3 MB/s 2024-04-25T10:23:09.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.745 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183971 s, 57.0 MB/s 2024-04-25T10:23:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144846 s, 72.4 MB/s 2024-04-25T10:23:09.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144886 s, 72.4 MB/s 2024-04-25T10:23:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196416 s, 53.4 MB/s 2024-04-25T10:23:09.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154427 s, 67.9 MB/s 2024-04-25T10:23:09.832 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.832 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.832 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143286 s, 73.2 MB/s 2024-04-25T10:23:09.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.849 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147651 s, 71.0 MB/s 2024-04-25T10:23:09.865 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.866 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.866 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148264 s, 70.7 MB/s 2024-04-25T10:23:09.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.891 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.891 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230459 s, 45.5 MB/s 2024-04-25T10:23:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014995 s, 69.9 MB/s 2024-04-25T10:23:09.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144429 s, 72.6 MB/s 2024-04-25T10:23:09.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.945 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196967 s, 53.2 MB/s 2024-04-25T10:23:09.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.962 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150307 s, 69.8 MB/s 2024-04-25T10:23:09.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:09.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:09.987 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231719 s, 45.3 MB/s 2024-04-25T10:23:10.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.003 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.003 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146171 s, 71.7 MB/s 2024-04-25T10:23:10.020 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.020 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.020 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151448 s, 69.2 MB/s 2024-04-25T10:23:10.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.039 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.040 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174153 s, 60.2 MB/s 2024-04-25T10:23:10.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:10.041 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:10.041 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:10.041 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:23:10.041 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:10.041 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:10.041 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:10.041 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.117 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020431 s, 51.3 MB/s 2024-04-25T10:23:10.137 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.138 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195675 s, 53.6 MB/s 2024-04-25T10:23:10.162 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.163 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.163 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230639 s, 45.5 MB/s 2024-04-25T10:23:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.179 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148543 s, 70.6 MB/s 2024-04-25T10:23:10.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.201 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199842 s, 52.5 MB/s 2024-04-25T10:23:10.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.217 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141135 s, 74.3 MB/s 2024-04-25T10:23:10.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.237 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017949 s, 58.4 MB/s 2024-04-25T10:23:10.253 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.253 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.253 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146576 s, 71.5 MB/s 2024-04-25T10:23:10.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.270 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152005 s, 69.0 MB/s 2024-04-25T10:23:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.306 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328636 s, 31.9 MB/s 2024-04-25T10:23:10.328 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.328 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.328 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211047 s, 49.7 MB/s 2024-04-25T10:23:10.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.350 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199082 s, 52.7 MB/s 2024-04-25T10:23:10.367 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160394 s, 65.4 MB/s 2024-04-25T10:23:10.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.385 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.386 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015891 s, 66.0 MB/s 2024-04-25T10:23:10.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159834 s, 65.6 MB/s 2024-04-25T10:23:10.420 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.420 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.420 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148698 s, 70.5 MB/s 2024-04-25T10:23:10.441 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.441 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.441 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192314 s, 54.5 MB/s 2024-04-25T10:23:10.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.458 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.458 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155806 s, 67.3 MB/s 2024-04-25T10:23:10.476 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.476 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.476 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156051 s, 67.2 MB/s 2024-04-25T10:23:10.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.492 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145425 s, 72.1 MB/s 2024-04-25T10:23:10.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131938 s, 79.5 MB/s 2024-04-25T10:23:10.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.523 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142906 s, 73.4 MB/s 2024-04-25T10:23:10.539 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.539 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.539 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143535 s, 73.1 MB/s 2024-04-25T10:23:10.553 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126493 s, 82.9 MB/s 2024-04-25T10:23:10.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.577 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212221 s, 49.4 MB/s 2024-04-25T10:23:10.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151845 s, 69.1 MB/s 2024-04-25T10:23:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148046 s, 70.8 MB/s 2024-04-25T10:23:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171731 s, 61.1 MB/s 2024-04-25T10:23:10.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.652 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208621 s, 50.3 MB/s 2024-04-25T10:23:10.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.670 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.670 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159518 s, 65.7 MB/s 2024-04-25T10:23:10.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.686 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149554 s, 70.1 MB/s 2024-04-25T10:23:10.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.706 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172664 s, 60.7 MB/s 2024-04-25T10:23:10.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:10.707 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:10.707 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:10.707 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:23:10.707 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:10.707 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:10.707 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:10.707 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:10.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.777 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143593 s, 73.0 MB/s 2024-04-25T10:23:10.797 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.797 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.797 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177029 s, 59.2 MB/s 2024-04-25T10:23:10.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174561 s, 60.1 MB/s 2024-04-25T10:23:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154306 s, 68.0 MB/s 2024-04-25T10:23:10.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.854 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.854 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018571 s, 56.5 MB/s 2024-04-25T10:23:10.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.872 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166363 s, 63.0 MB/s 2024-04-25T10:23:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.894 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195814 s, 53.5 MB/s 2024-04-25T10:23:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142746 s, 73.5 MB/s 2024-04-25T10:23:10.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.930 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.930 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186825 s, 56.1 MB/s 2024-04-25T10:23:10.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.948 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.948 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164082 s, 63.9 MB/s 2024-04-25T10:23:10.972 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.972 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.972 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218957 s, 47.9 MB/s 2024-04-25T10:23:10.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:10.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:10.988 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014507 s, 72.3 MB/s 2024-04-25T10:23:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.005 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151086 s, 69.4 MB/s 2024-04-25T10:23:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164714 s, 63.7 MB/s 2024-04-25T10:23:11.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160739 s, 65.2 MB/s 2024-04-25T10:23:11.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.058 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150358 s, 69.7 MB/s 2024-04-25T10:23:11.077 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.077 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.077 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176777 s, 59.3 MB/s 2024-04-25T10:23:11.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.098 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.098 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186899 s, 56.1 MB/s 2024-04-25T10:23:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.114 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148424 s, 70.6 MB/s 2024-04-25T10:23:11.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145967 s, 71.8 MB/s 2024-04-25T10:23:11.147 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.148 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150001 s, 69.9 MB/s 2024-04-25T10:23:11.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.167 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179946 s, 58.3 MB/s 2024-04-25T10:23:11.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150389 s, 69.7 MB/s 2024-04-25T10:23:11.200 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.201 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146994 s, 71.3 MB/s 2024-04-25T10:23:11.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.217 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.217 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146569 s, 71.5 MB/s 2024-04-25T10:23:11.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.239 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.239 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202317 s, 51.8 MB/s 2024-04-25T10:23:11.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.258 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.258 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168909 s, 62.1 MB/s 2024-04-25T10:23:11.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146523 s, 71.6 MB/s 2024-04-25T10:23:11.292 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.293 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164635 s, 63.7 MB/s 2024-04-25T10:23:11.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155609 s, 67.4 MB/s 2024-04-25T10:23:11.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.333 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213899 s, 49.0 MB/s 2024-04-25T10:23:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230389 s, 45.5 MB/s 2024-04-25T10:23:11.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:11.360 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:11.360 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:11.360 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:23:11.360 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:11.360 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:11.360 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:11.360 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:11.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.436 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.436 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165518 s, 63.4 MB/s 2024-04-25T10:23:11.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227414 s, 46.1 MB/s 2024-04-25T10:23:11.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.477 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.477 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152819 s, 68.6 MB/s 2024-04-25T10:23:11.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193129 s, 54.3 MB/s 2024-04-25T10:23:11.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.523 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225416 s, 46.5 MB/s 2024-04-25T10:23:11.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.545 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.545 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195925 s, 53.5 MB/s 2024-04-25T10:23:11.563 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.564 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.564 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164022 s, 63.9 MB/s 2024-04-25T10:23:11.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.587 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.587 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209807 s, 50.0 MB/s 2024-04-25T10:23:11.605 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.605 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.605 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165902 s, 63.2 MB/s 2024-04-25T10:23:11.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.622 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151924 s, 69.0 MB/s 2024-04-25T10:23:11.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206467 s, 50.8 MB/s 2024-04-25T10:23:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.669 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232579 s, 45.1 MB/s 2024-04-25T10:23:11.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.688 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177569 s, 59.1 MB/s 2024-04-25T10:23:11.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145547 s, 72.0 MB/s 2024-04-25T10:23:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147229 s, 71.2 MB/s 2024-04-25T10:23:11.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.745 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210385 s, 49.8 MB/s 2024-04-25T10:23:11.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.765 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.765 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192856 s, 54.4 MB/s 2024-04-25T10:23:11.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.784 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172182 s, 60.9 MB/s 2024-04-25T10:23:11.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.803 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173174 s, 60.6 MB/s 2024-04-25T10:23:11.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.820 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.820 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149199 s, 70.3 MB/s 2024-04-25T10:23:11.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.844 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.844 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220268 s, 47.6 MB/s 2024-04-25T10:23:11.866 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.866 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.866 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203193 s, 51.6 MB/s 2024-04-25T10:23:11.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.881 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.881 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139643 s, 75.1 MB/s 2024-04-25T10:23:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143611 s, 73.0 MB/s 2024-04-25T10:23:11.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.919 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.919 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196828 s, 53.3 MB/s 2024-04-25T10:23:11.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.941 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202221 s, 51.9 MB/s 2024-04-25T10:23:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199484 s, 52.6 MB/s 2024-04-25T10:23:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.982 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176858 s, 59.3 MB/s 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:11.999 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148031 s, 70.8 MB/s 2024-04-25T10:23:12.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.016 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.016 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01521 s, 68.9 MB/s 2024-04-25T10:23:12.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.036 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176838 s, 59.3 MB/s 2024-04-25T10:23:12.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165026 s, 63.5 MB/s 2024-04-25T10:23:12.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:12.055 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:12.055 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:12.055 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:23:12.055 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:12.055 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:12.055 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:12.055 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:12.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149425 s, 70.2 MB/s 2024-04-25T10:23:12.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.152 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184677 s, 56.8 MB/s 2024-04-25T10:23:12.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.169 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157574 s, 66.5 MB/s 2024-04-25T10:23:12.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.186 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153388 s, 68.4 MB/s 2024-04-25T10:23:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143342 s, 73.2 MB/s 2024-04-25T10:23:12.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203335 s, 51.6 MB/s 2024-04-25T10:23:12.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173332 s, 60.5 MB/s 2024-04-25T10:23:12.261 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.262 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.262 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160521 s, 65.3 MB/s 2024-04-25T10:23:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.278 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014941 s, 70.2 MB/s 2024-04-25T10:23:12.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.298 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.298 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177607 s, 59.0 MB/s 2024-04-25T10:23:12.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.317 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.317 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174839 s, 60.0 MB/s 2024-04-25T10:23:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150406 s, 69.7 MB/s 2024-04-25T10:23:12.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.353 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175556 s, 59.7 MB/s 2024-04-25T10:23:12.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140802 s, 74.5 MB/s 2024-04-25T10:23:12.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.384 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126959 s, 82.6 MB/s 2024-04-25T10:23:12.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.405 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194689 s, 53.9 MB/s 2024-04-25T10:23:12.425 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.425 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.425 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177686 s, 59.0 MB/s 2024-04-25T10:23:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.447 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199663 s, 52.5 MB/s 2024-04-25T10:23:12.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121643 s, 86.2 MB/s 2024-04-25T10:23:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176956 s, 59.3 MB/s 2024-04-25T10:23:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.502 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203805 s, 51.4 MB/s 2024-04-25T10:23:12.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.523 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194087 s, 54.0 MB/s 2024-04-25T10:23:12.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183863 s, 57.0 MB/s 2024-04-25T10:23:12.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147176 s, 71.2 MB/s 2024-04-25T10:23:12.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.586 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240298 s, 43.6 MB/s 2024-04-25T10:23:12.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.603 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154105 s, 68.0 MB/s 2024-04-25T10:23:12.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.619 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.619 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145849 s, 71.9 MB/s 2024-04-25T10:23:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.636 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154152 s, 68.0 MB/s 2024-04-25T10:23:12.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.658 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198181 s, 52.9 MB/s 2024-04-25T10:23:12.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.679 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197899 s, 53.0 MB/s 2024-04-25T10:23:12.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.697 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158147 s, 66.3 MB/s 2024-04-25T10:23:12.713 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147758 s, 71.0 MB/s 2024-04-25T10:23:12.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:12.715 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:12.715 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:12.715 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:23:12.715 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:12.715 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:12.715 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:12.715 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:12.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.789 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169056 s, 62.0 MB/s 2024-04-25T10:23:12.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148047 s, 70.8 MB/s 2024-04-25T10:23:12.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.824 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166539 s, 63.0 MB/s 2024-04-25T10:23:12.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.843 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183253 s, 57.2 MB/s 2024-04-25T10:23:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.860 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148546 s, 70.6 MB/s 2024-04-25T10:23:12.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.876 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143187 s, 73.2 MB/s 2024-04-25T10:23:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.900 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219368 s, 47.8 MB/s 2024-04-25T10:23:12.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.917 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.917 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157102 s, 66.7 MB/s 2024-04-25T10:23:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.933 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143487 s, 73.1 MB/s 2024-04-25T10:23:12.950 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.951 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.951 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015229 s, 68.9 MB/s 2024-04-25T10:23:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.964 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120652 s, 86.9 MB/s 2024-04-25T10:23:12.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:12.985 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:12.985 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191081 s, 54.9 MB/s 2024-04-25T10:23:13.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.007 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198491 s, 52.8 MB/s 2024-04-25T10:23:13.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.025 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.025 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161789 s, 64.8 MB/s 2024-04-25T10:23:13.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.041 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141994 s, 73.8 MB/s 2024-04-25T10:23:13.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.060 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.060 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173596 s, 60.4 MB/s 2024-04-25T10:23:13.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.081 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.081 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018949 s, 55.3 MB/s 2024-04-25T10:23:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.097 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145661 s, 72.0 MB/s 2024-04-25T10:23:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.114 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150753 s, 69.6 MB/s 2024-04-25T10:23:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150758 s, 69.6 MB/s 2024-04-25T10:23:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.148 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158514 s, 66.2 MB/s 2024-04-25T10:23:13.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.165 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147739 s, 71.0 MB/s 2024-04-25T10:23:13.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.181 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.181 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142876 s, 73.4 MB/s 2024-04-25T10:23:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150036 s, 69.9 MB/s 2024-04-25T10:23:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.214 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143302 s, 73.2 MB/s 2024-04-25T10:23:13.230 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.230 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.230 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144974 s, 72.3 MB/s 2024-04-25T10:23:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159982 s, 65.5 MB/s 2024-04-25T10:23:13.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.271 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.271 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211063 s, 49.7 MB/s 2024-04-25T10:23:13.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.285 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.285 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125938 s, 83.3 MB/s 2024-04-25T10:23:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216001 s, 48.5 MB/s 2024-04-25T10:23:13.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181064 s, 57.9 MB/s 2024-04-25T10:23:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.345 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149196 s, 70.3 MB/s 2024-04-25T10:23:13.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:13.347 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:13.347 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:13.347 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:23:13.347 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:13.347 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:13.347 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:13.347 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:13.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.427 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.427 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186014 s, 56.4 MB/s 2024-04-25T10:23:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202549 s, 51.8 MB/s 2024-04-25T10:23:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229015 s, 45.8 MB/s 2024-04-25T10:23:13.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.490 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149055 s, 70.3 MB/s 2024-04-25T10:23:13.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160742 s, 65.2 MB/s 2024-04-25T10:23:13.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144258 s, 72.7 MB/s 2024-04-25T10:23:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.542 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155007 s, 67.6 MB/s 2024-04-25T10:23:13.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169077 s, 62.0 MB/s 2024-04-25T10:23:13.578 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.578 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.578 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161658 s, 64.9 MB/s 2024-04-25T10:23:13.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.594 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143165 s, 73.2 MB/s 2024-04-25T10:23:13.612 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.612 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.612 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166271 s, 63.1 MB/s 2024-04-25T10:23:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176172 s, 59.5 MB/s 2024-04-25T10:23:13.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.651 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.652 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181277 s, 57.8 MB/s 2024-04-25T10:23:13.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.675 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214022 s, 49.0 MB/s 2024-04-25T10:23:13.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.691 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.691 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144958 s, 72.3 MB/s 2024-04-25T10:23:13.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174327 s, 60.1 MB/s 2024-04-25T10:23:13.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.729 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.729 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169301 s, 61.9 MB/s 2024-04-25T10:23:13.748 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.749 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175801 s, 59.6 MB/s 2024-04-25T10:23:13.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.767 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169477 s, 61.9 MB/s 2024-04-25T10:23:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015833 s, 66.2 MB/s 2024-04-25T10:23:13.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.801 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.801 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146017 s, 71.8 MB/s 2024-04-25T10:23:13.819 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.819 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.819 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143473 s, 73.1 MB/s 2024-04-25T10:23:13.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201428 s, 52.1 MB/s 2024-04-25T10:23:13.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.857 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162945 s, 64.4 MB/s 2024-04-25T10:23:13.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.877 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017515 s, 59.9 MB/s 2024-04-25T10:23:13.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.893 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148961 s, 70.4 MB/s 2024-04-25T10:23:13.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.912 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.912 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174656 s, 60.0 MB/s 2024-04-25T10:23:13.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196888 s, 53.3 MB/s 2024-04-25T10:23:13.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.952 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.952 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166341 s, 63.0 MB/s 2024-04-25T10:23:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170025 s, 61.7 MB/s 2024-04-25T10:23:13.992 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198926 s, 52.7 MB/s 2024-04-25T10:23:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01971 s, 53.2 MB/s 2024-04-25T10:23:14.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:14.015 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:14.015 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:14.016 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:23:14.016 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:14.016 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:14.016 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:14.016 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204664 s, 51.2 MB/s 2024-04-25T10:23:14.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.109 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155238 s, 67.5 MB/s 2024-04-25T10:23:14.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.131 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.131 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198375 s, 52.9 MB/s 2024-04-25T10:23:14.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.150 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.151 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177463 s, 59.1 MB/s 2024-04-25T10:23:14.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.167 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.167 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149306 s, 70.2 MB/s 2024-04-25T10:23:14.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151109 s, 69.4 MB/s 2024-04-25T10:23:14.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214961 s, 48.8 MB/s 2024-04-25T10:23:14.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.224 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152505 s, 68.8 MB/s 2024-04-25T10:23:14.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.246 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202709 s, 51.7 MB/s 2024-04-25T10:23:14.266 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.266 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.266 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181637 s, 57.7 MB/s 2024-04-25T10:23:14.283 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.284 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.284 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154891 s, 67.7 MB/s 2024-04-25T10:23:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021544 s, 48.7 MB/s 2024-04-25T10:23:14.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203335 s, 51.6 MB/s 2024-04-25T10:23:14.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170445 s, 61.5 MB/s 2024-04-25T10:23:14.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.372 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219684 s, 47.7 MB/s 2024-04-25T10:23:14.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.390 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154228 s, 68.0 MB/s 2024-04-25T10:23:14.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.414 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.414 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234191 s, 44.8 MB/s 2024-04-25T10:23:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171538 s, 61.1 MB/s 2024-04-25T10:23:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143627 s, 73.0 MB/s 2024-04-25T10:23:14.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.467 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161798 s, 64.8 MB/s 2024-04-25T10:23:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151629 s, 69.2 MB/s 2024-04-25T10:23:14.503 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.503 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.503 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171228 s, 61.2 MB/s 2024-04-25T10:23:14.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.520 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.520 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155835 s, 67.3 MB/s 2024-04-25T10:23:14.537 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.538 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154171 s, 68.0 MB/s 2024-04-25T10:23:14.553 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.554 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.554 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142051 s, 73.8 MB/s 2024-04-25T10:23:14.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.574 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.574 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187066 s, 56.1 MB/s 2024-04-25T10:23:14.598 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.598 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.598 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020718 s, 50.6 MB/s 2024-04-25T10:23:14.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.621 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.621 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227412 s, 46.1 MB/s 2024-04-25T10:23:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208047 s, 50.4 MB/s 2024-04-25T10:23:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123456 s, 84.9 MB/s 2024-04-25T10:23:14.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.676 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.676 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166613 s, 62.9 MB/s 2024-04-25T10:23:14.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.696 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180089 s, 58.2 MB/s 2024-04-25T10:23:14.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:14.697 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:14.697 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:14.698 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:23:14.698 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:14.698 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:14.698 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:14.698 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:14.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.767 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.767 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156608 s, 67.0 MB/s 2024-04-25T10:23:14.784 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160376 s, 65.4 MB/s 2024-04-25T10:23:14.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.803 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.803 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169767 s, 61.8 MB/s 2024-04-25T10:23:14.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.822 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174721 s, 60.0 MB/s 2024-04-25T10:23:14.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149982 s, 69.9 MB/s 2024-04-25T10:23:14.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.857 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163712 s, 64.1 MB/s 2024-04-25T10:23:14.878 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.878 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.878 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193373 s, 54.2 MB/s 2024-04-25T10:23:14.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.898 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.898 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181784 s, 57.7 MB/s 2024-04-25T10:23:14.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.915 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.915 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01539 s, 68.1 MB/s 2024-04-25T10:23:14.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.932 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.932 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149833 s, 70.0 MB/s 2024-04-25T10:23:14.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.953 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190693 s, 55.0 MB/s 2024-04-25T10:23:14.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225175 s, 46.6 MB/s 2024-04-25T10:23:14.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:14.996 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:14.996 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166538 s, 63.0 MB/s 2024-04-25T10:23:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177542 s, 59.1 MB/s 2024-04-25T10:23:15.032 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.032 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.032 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150371 s, 69.7 MB/s 2024-04-25T10:23:15.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.055 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.055 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212963 s, 49.2 MB/s 2024-04-25T10:23:15.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177544 s, 59.1 MB/s 2024-04-25T10:23:15.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.090 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.090 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141193 s, 74.3 MB/s 2024-04-25T10:23:15.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.106 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.106 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144127 s, 72.8 MB/s 2024-04-25T10:23:15.123 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.123 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.123 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148133 s, 70.8 MB/s 2024-04-25T10:23:15.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144182 s, 72.7 MB/s 2024-04-25T10:23:15.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.156 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.156 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150088 s, 69.9 MB/s 2024-04-25T10:23:15.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.172 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146668 s, 71.5 MB/s 2024-04-25T10:23:15.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.189 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.189 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147684 s, 71.0 MB/s 2024-04-25T10:23:15.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.206 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153454 s, 68.3 MB/s 2024-04-25T10:23:15.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.225 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.225 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173772 s, 60.3 MB/s 2024-04-25T10:23:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.242 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.242 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014708 s, 71.3 MB/s 2024-04-25T10:23:15.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.260 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160612 s, 65.3 MB/s 2024-04-25T10:23:15.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.279 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172575 s, 60.8 MB/s 2024-04-25T10:23:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.296 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150195 s, 69.8 MB/s 2024-04-25T10:23:15.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.315 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175397 s, 59.8 MB/s 2024-04-25T10:23:15.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.333 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.333 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016684 s, 62.8 MB/s 2024-04-25T10:23:15.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:15.335 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:15.335 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:15.335 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:23:15.335 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:15.335 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:15.335 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:15.335 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:15.415 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.415 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.415 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233551 s, 44.9 MB/s 2024-04-25T10:23:15.434 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.434 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.434 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173997 s, 60.3 MB/s 2024-04-25T10:23:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.451 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148742 s, 70.5 MB/s 2024-04-25T10:23:15.467 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151988 s, 69.0 MB/s 2024-04-25T10:23:15.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.487 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.487 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178773 s, 58.7 MB/s 2024-04-25T10:23:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169113 s, 62.0 MB/s 2024-04-25T10:23:15.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.522 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149299 s, 70.2 MB/s 2024-04-25T10:23:15.539 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.539 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.539 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150626 s, 69.6 MB/s 2024-04-25T10:23:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.559 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171731 s, 61.1 MB/s 2024-04-25T10:23:15.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152728 s, 68.7 MB/s 2024-04-25T10:23:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174849 s, 60.0 MB/s 2024-04-25T10:23:15.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.614 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.614 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176813 s, 59.3 MB/s 2024-04-25T10:23:15.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145537 s, 72.0 MB/s 2024-04-25T10:23:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147332 s, 71.2 MB/s 2024-04-25T10:23:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142196 s, 73.7 MB/s 2024-04-25T10:23:15.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.680 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147661 s, 71.0 MB/s 2024-04-25T10:23:15.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.696 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015096 s, 69.5 MB/s 2024-04-25T10:23:15.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.716 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173328 s, 60.5 MB/s 2024-04-25T10:23:15.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.731 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141659 s, 74.0 MB/s 2024-04-25T10:23:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162772 s, 64.4 MB/s 2024-04-25T10:23:15.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.772 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.772 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204007 s, 51.4 MB/s 2024-04-25T10:23:15.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.789 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015195 s, 69.0 MB/s 2024-04-25T10:23:15.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147744 s, 71.0 MB/s 2024-04-25T10:23:15.822 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.823 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153919 s, 68.1 MB/s 2024-04-25T10:23:15.842 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.843 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183918 s, 57.0 MB/s 2024-04-25T10:23:15.859 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.859 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.859 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145293 s, 72.2 MB/s 2024-04-25T10:23:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.877 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158055 s, 66.3 MB/s 2024-04-25T10:23:15.892 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.893 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142266 s, 73.7 MB/s 2024-04-25T10:23:15.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.914 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198291 s, 52.9 MB/s 2024-04-25T10:23:15.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.933 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172701 s, 60.7 MB/s 2024-04-25T10:23:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018143 s, 57.8 MB/s 2024-04-25T10:23:15.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:15.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:15.969 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014392 s, 72.9 MB/s 2024-04-25T10:23:15.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:15.971 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:15.971 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:15.971 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:23:15.971 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:15.971 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:15.971 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:15.971 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:16.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.041 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.042 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014193 s, 73.9 MB/s 2024-04-25T10:23:16.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.058 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.058 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145546 s, 72.0 MB/s 2024-04-25T10:23:16.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143139 s, 73.3 MB/s 2024-04-25T10:23:16.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.088 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.089 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125702 s, 83.4 MB/s 2024-04-25T10:23:16.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.105 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.105 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147632 s, 71.0 MB/s 2024-04-25T10:23:16.122 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.123 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.123 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156226 s, 67.1 MB/s 2024-04-25T10:23:16.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150728 s, 69.6 MB/s 2024-04-25T10:23:16.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.158 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.158 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167523 s, 62.6 MB/s 2024-04-25T10:23:16.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.174 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.174 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147858 s, 70.9 MB/s 2024-04-25T10:23:16.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167274 s, 62.7 MB/s 2024-04-25T10:23:16.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147404 s, 71.1 MB/s 2024-04-25T10:23:16.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.230 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182845 s, 57.3 MB/s 2024-04-25T10:23:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171018 s, 61.3 MB/s 2024-04-25T10:23:16.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.266 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.266 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154136 s, 68.0 MB/s 2024-04-25T10:23:16.288 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.289 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.289 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209605 s, 50.0 MB/s 2024-04-25T10:23:16.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.305 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.305 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150624 s, 69.6 MB/s 2024-04-25T10:23:16.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.322 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.322 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01518 s, 69.1 MB/s 2024-04-25T10:23:16.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.339 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155181 s, 67.6 MB/s 2024-04-25T10:23:16.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.358 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.358 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016566 s, 63.3 MB/s 2024-04-25T10:23:16.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.375 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156154 s, 67.1 MB/s 2024-04-25T10:23:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240416 s, 43.6 MB/s 2024-04-25T10:23:16.417 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.418 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.418 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014816 s, 70.8 MB/s 2024-04-25T10:23:16.434 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.434 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.434 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015232 s, 68.8 MB/s 2024-04-25T10:23:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.451 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.452 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151542 s, 69.2 MB/s 2024-04-25T10:23:16.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.468 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.468 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014347 s, 73.1 MB/s 2024-04-25T10:23:16.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.485 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.485 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015214 s, 68.9 MB/s 2024-04-25T10:23:16.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.499 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.499 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124417 s, 84.3 MB/s 2024-04-25T10:23:16.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.528 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.528 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257325 s, 40.7 MB/s 2024-04-25T10:23:16.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.549 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189169 s, 55.4 MB/s 2024-04-25T10:23:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024204 s, 43.3 MB/s 2024-04-25T10:23:16.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.593 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016506 s, 63.5 MB/s 2024-04-25T10:23:16.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.610 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.610 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151363 s, 69.3 MB/s 2024-04-25T10:23:16.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:16.612 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:16.612 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:16.612 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:23:16.612 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:16.612 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:16.612 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:16.612 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:16.688 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.689 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.689 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173326 s, 60.5 MB/s 2024-04-25T10:23:16.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144949 s, 72.3 MB/s 2024-04-25T10:23:16.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.724 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.725 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178029 s, 58.9 MB/s 2024-04-25T10:23:16.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.741 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.741 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150972 s, 69.5 MB/s 2024-04-25T10:23:16.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.758 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.758 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146927 s, 71.4 MB/s 2024-04-25T10:23:16.780 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.780 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.780 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206986 s, 50.7 MB/s 2024-04-25T10:23:16.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223529 s, 46.9 MB/s 2024-04-25T10:23:16.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.823 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.824 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172383 s, 60.8 MB/s 2024-04-25T10:23:16.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.842 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166363 s, 63.0 MB/s 2024-04-25T10:23:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.858 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144769 s, 72.4 MB/s 2024-04-25T10:23:16.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.880 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.880 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207798 s, 50.5 MB/s 2024-04-25T10:23:16.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.903 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.903 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207962 s, 50.4 MB/s 2024-04-25T10:23:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169231 s, 62.0 MB/s 2024-04-25T10:23:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.939 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157182 s, 66.7 MB/s 2024-04-25T10:23:16.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.957 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.957 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165649 s, 63.3 MB/s 2024-04-25T10:23:16.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.973 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.973 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139183 s, 75.3 MB/s 2024-04-25T10:23:16.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:16.988 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:16.988 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139453 s, 75.2 MB/s 2024-04-25T10:23:17.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.007 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170048 s, 61.7 MB/s 2024-04-25T10:23:17.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.026 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.026 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168718 s, 62.1 MB/s 2024-04-25T10:23:17.050 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.050 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.051 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229792 s, 45.6 MB/s 2024-04-25T10:23:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158266 s, 66.3 MB/s 2024-04-25T10:23:17.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.085 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156541 s, 67.0 MB/s 2024-04-25T10:23:17.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.104 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.104 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170825 s, 61.4 MB/s 2024-04-25T10:23:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172668 s, 60.7 MB/s 2024-04-25T10:23:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142557 s, 73.6 MB/s 2024-04-25T10:23:17.156 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.156 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.156 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148907 s, 70.4 MB/s 2024-04-25T10:23:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.179 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209942 s, 49.9 MB/s 2024-04-25T10:23:17.198 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178655 s, 58.7 MB/s 2024-04-25T10:23:17.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.218 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.218 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177346 s, 59.1 MB/s 2024-04-25T10:23:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.234 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014657 s, 71.5 MB/s 2024-04-25T10:23:17.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.250 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144592 s, 72.5 MB/s 2024-04-25T10:23:17.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.269 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.269 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164094 s, 63.9 MB/s 2024-04-25T10:23:17.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:17.271 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:17.271 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:17.271 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:23:17.271 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:17.271 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:17.271 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:17.271 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:17.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.348 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.348 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160885 s, 65.2 MB/s 2024-04-25T10:23:17.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.364 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.364 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150452 s, 69.7 MB/s 2024-04-25T10:23:17.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.384 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180338 s, 58.1 MB/s 2024-04-25T10:23:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179564 s, 58.4 MB/s 2024-04-25T10:23:17.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.426 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.426 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206316 s, 50.8 MB/s 2024-04-25T10:23:17.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.447 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190222 s, 55.1 MB/s 2024-04-25T10:23:17.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.466 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147308 s, 71.2 MB/s 2024-04-25T10:23:17.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.486 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.486 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211968 s, 49.5 MB/s 2024-04-25T10:23:17.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.507 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183269 s, 57.2 MB/s 2024-04-25T10:23:17.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015231 s, 68.8 MB/s 2024-04-25T10:23:17.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.547 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208155 s, 50.4 MB/s 2024-04-25T10:23:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.566 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180907 s, 58.0 MB/s 2024-04-25T10:23:17.585 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.586 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171197 s, 61.2 MB/s 2024-04-25T10:23:17.604 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.604 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.604 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168869 s, 62.1 MB/s 2024-04-25T10:23:17.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.626 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.626 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204348 s, 51.3 MB/s 2024-04-25T10:23:17.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.643 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.643 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154005 s, 68.1 MB/s 2024-04-25T10:23:17.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.662 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.662 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169732 s, 61.8 MB/s 2024-04-25T10:23:17.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.679 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.679 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154243 s, 68.0 MB/s 2024-04-25T10:23:17.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.699 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.700 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186898 s, 56.1 MB/s 2024-04-25T10:23:17.717 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.718 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.718 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162439 s, 64.6 MB/s 2024-04-25T10:23:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.737 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178352 s, 58.8 MB/s 2024-04-25T10:23:17.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.753 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142048 s, 73.8 MB/s 2024-04-25T10:23:17.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.773 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.773 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185728 s, 56.5 MB/s 2024-04-25T10:23:17.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.795 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.795 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201359 s, 52.1 MB/s 2024-04-25T10:23:17.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.813 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.814 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163373 s, 64.2 MB/s 2024-04-25T10:23:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174989 s, 59.9 MB/s 2024-04-25T10:23:17.851 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.852 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171609 s, 61.1 MB/s 2024-04-25T10:23:17.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.869 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.869 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015678 s, 66.9 MB/s 2024-04-25T10:23:17.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.890 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193508 s, 54.2 MB/s 2024-04-25T10:23:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.907 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150768 s, 69.5 MB/s 2024-04-25T10:23:17.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141843 s, 73.9 MB/s 2024-04-25T10:23:17.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:17.940 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:17.941 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158949 s, 66.0 MB/s 2024-04-25T10:23:17.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:17.942 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:17.942 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:17.942 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:23:17.942 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:17.942 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:17.942 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:17.943 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:18.014 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.015 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.015 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156584 s, 67.0 MB/s 2024-04-25T10:23:18.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.036 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.036 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193479 s, 54.2 MB/s 2024-04-25T10:23:18.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.054 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.054 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016601 s, 63.2 MB/s 2024-04-25T10:23:18.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.072 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168292 s, 62.3 MB/s 2024-04-25T10:23:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.094 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019508 s, 53.8 MB/s 2024-04-25T10:23:18.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143044 s, 73.3 MB/s 2024-04-25T10:23:18.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.127 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.127 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151611 s, 69.2 MB/s 2024-04-25T10:23:18.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.143 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.143 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145756 s, 71.9 MB/s 2024-04-25T10:23:18.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.169 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.169 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240964 s, 43.5 MB/s 2024-04-25T10:23:18.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.186 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.186 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152689 s, 68.7 MB/s 2024-04-25T10:23:18.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.206 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174211 s, 60.2 MB/s 2024-04-25T10:23:18.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.222 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.222 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149619 s, 70.1 MB/s 2024-04-25T10:23:18.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.245 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202656 s, 51.7 MB/s 2024-04-25T10:23:18.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.260 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.260 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143593 s, 73.0 MB/s 2024-04-25T10:23:18.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.282 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.282 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189301 s, 55.4 MB/s 2024-04-25T10:23:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122348 s, 85.7 MB/s 2024-04-25T10:23:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178846 s, 58.6 MB/s 2024-04-25T10:23:18.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.334 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.334 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178278 s, 58.8 MB/s 2024-04-25T10:23:18.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.352 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.352 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01616 s, 64.9 MB/s 2024-04-25T10:23:18.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155495 s, 67.4 MB/s 2024-04-25T10:23:18.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173519 s, 60.4 MB/s 2024-04-25T10:23:18.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.407 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158871 s, 66.0 MB/s 2024-04-25T10:23:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.424 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157932 s, 66.4 MB/s 2024-04-25T10:23:18.440 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.441 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.441 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015168 s, 69.1 MB/s 2024-04-25T10:23:18.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.457 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.457 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143124 s, 73.3 MB/s 2024-04-25T10:23:18.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.474 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.474 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148977 s, 70.4 MB/s 2024-04-25T10:23:18.492 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.493 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.493 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170797 s, 61.4 MB/s 2024-04-25T10:23:18.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.510 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152042 s, 69.0 MB/s 2024-04-25T10:23:18.526 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150314 s, 69.8 MB/s 2024-04-25T10:23:18.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.548 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.548 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193025 s, 54.3 MB/s 2024-04-25T10:23:18.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.572 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.572 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223956 s, 46.8 MB/s 2024-04-25T10:23:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.589 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.590 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016292 s, 64.4 MB/s 2024-04-25T10:23:18.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:18.591 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:18.591 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:18.591 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:23:18.591 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:18.591 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:18.591 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:18.591 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:18.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.666 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.666 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173533 s, 60.4 MB/s 2024-04-25T10:23:18.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.689 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178178 s, 58.8 MB/s 2024-04-25T10:23:18.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180882 s, 58.0 MB/s 2024-04-25T10:23:18.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.721 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.721 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141333 s, 74.2 MB/s 2024-04-25T10:23:18.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.740 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.740 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173313 s, 60.5 MB/s 2024-04-25T10:23:18.757 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.758 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.758 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152676 s, 68.7 MB/s 2024-04-25T10:23:18.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.778 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.778 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190848 s, 54.9 MB/s 2024-04-25T10:23:18.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019436 s, 54.0 MB/s 2024-04-25T10:23:18.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.818 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.818 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167598 s, 62.6 MB/s 2024-04-25T10:23:18.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.834 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.834 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150462 s, 69.7 MB/s 2024-04-25T10:23:18.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164492 s, 63.7 MB/s 2024-04-25T10:23:18.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.872 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.872 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175287 s, 59.8 MB/s 2024-04-25T10:23:18.890 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.894 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.894 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168319 s, 62.3 MB/s 2024-04-25T10:23:18.906 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.906 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.906 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141597 s, 74.1 MB/s 2024-04-25T10:23:18.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.923 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149801 s, 70.0 MB/s 2024-04-25T10:23:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.945 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203518 s, 51.5 MB/s 2024-04-25T10:23:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.962 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.962 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144148 s, 72.7 MB/s 2024-04-25T10:23:18.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:18.980 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:18.981 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172403 s, 60.8 MB/s 2024-04-25T10:23:19.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.002 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.002 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201603 s, 52.0 MB/s 2024-04-25T10:23:19.018 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.019 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146283 s, 71.7 MB/s 2024-04-25T10:23:19.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.035 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149049 s, 70.4 MB/s 2024-04-25T10:23:19.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.057 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.057 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203864 s, 51.4 MB/s 2024-04-25T10:23:19.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.074 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.074 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145847 s, 71.9 MB/s 2024-04-25T10:23:19.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.097 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.097 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206003 s, 50.9 MB/s 2024-04-25T10:23:19.114 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165181 s, 63.5 MB/s 2024-04-25T10:23:19.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.138 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.138 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219422 s, 47.8 MB/s 2024-04-25T10:23:19.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.155 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.155 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149421 s, 70.2 MB/s 2024-04-25T10:23:19.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.176 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.176 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192012 s, 54.6 MB/s 2024-04-25T10:23:19.194 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.194 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.194 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169303 s, 61.9 MB/s 2024-04-25T10:23:19.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.213 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165111 s, 63.5 MB/s 2024-04-25T10:23:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.235 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204431 s, 51.3 MB/s 2024-04-25T10:23:19.255 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.256 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.256 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018811 s, 55.7 MB/s 2024-04-25T10:23:19.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:19.257 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:19.257 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:19.257 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:23:19.257 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:19.258 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:19.258 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:19.258 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:19.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.327 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.328 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016461 s, 63.7 MB/s 2024-04-25T10:23:19.350 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.404 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.404 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212662 s, 49.3 MB/s 2024-04-25T10:23:19.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.405 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167057 s, 62.8 MB/s 2024-04-25T10:23:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184216 s, 56.9 MB/s 2024-04-25T10:23:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015046 s, 69.7 MB/s 2024-04-25T10:23:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.424 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.425 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173535 s, 60.4 MB/s 2024-04-25T10:23:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016912 s, 62.0 MB/s 2024-04-25T10:23:19.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.460 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.460 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151307 s, 69.3 MB/s 2024-04-25T10:23:19.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.479 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171031 s, 61.3 MB/s 2024-04-25T10:23:19.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.498 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.498 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169659 s, 61.8 MB/s 2024-04-25T10:23:19.517 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.518 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.518 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181005 s, 57.9 MB/s 2024-04-25T10:23:19.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232641 s, 45.1 MB/s 2024-04-25T10:23:19.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.559 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.559 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143804 s, 72.9 MB/s 2024-04-25T10:23:19.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.580 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.580 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187711 s, 55.9 MB/s 2024-04-25T10:23:19.596 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.596 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.596 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148051 s, 70.8 MB/s 2024-04-25T10:23:19.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.615 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.615 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168188 s, 62.3 MB/s 2024-04-25T10:23:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234923 s, 44.6 MB/s 2024-04-25T10:23:19.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.657 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.657 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154737 s, 67.8 MB/s 2024-04-25T10:23:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159586 s, 65.7 MB/s 2024-04-25T10:23:19.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155818 s, 67.3 MB/s 2024-04-25T10:23:19.709 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159116 s, 65.9 MB/s 2024-04-25T10:23:19.726 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.727 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.727 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152295 s, 68.9 MB/s 2024-04-25T10:23:19.745 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.746 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.746 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172057 s, 60.9 MB/s 2024-04-25T10:23:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146808 s, 71.4 MB/s 2024-04-25T10:23:19.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.783 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.783 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192159 s, 54.6 MB/s 2024-04-25T10:23:19.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.800 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.800 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151035 s, 69.4 MB/s 2024-04-25T10:23:19.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.817 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.817 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153908 s, 68.1 MB/s 2024-04-25T10:23:19.838 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.839 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.839 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199004 s, 52.7 MB/s 2024-04-25T10:23:19.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.860 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.860 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201011 s, 52.2 MB/s 2024-04-25T10:23:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.877 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151765 s, 69.1 MB/s 2024-04-25T10:23:19.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.896 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169912 s, 61.7 MB/s 2024-04-25T10:23:19.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.914 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.914 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145814 s, 71.9 MB/s 2024-04-25T10:23:19.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:19.914 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:19.914 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:19.915 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:23:19.915 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:19.915 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:19.915 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:19.915 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:19.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:19.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:19.990 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142215 s, 73.7 MB/s 2024-04-25T10:23:20.006 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.007 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.007 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149411 s, 70.2 MB/s 2024-04-25T10:23:20.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.027 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185464 s, 56.5 MB/s 2024-04-25T10:23:20.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.043 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.043 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145871 s, 71.9 MB/s 2024-04-25T10:23:20.065 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.065 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.065 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194388 s, 53.9 MB/s 2024-04-25T10:23:20.087 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.087 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.087 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203968 s, 51.4 MB/s 2024-04-25T10:23:20.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.110 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.110 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218811 s, 47.9 MB/s 2024-04-25T10:23:20.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.130 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.130 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177688 s, 59.0 MB/s 2024-04-25T10:23:20.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.152 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.152 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200121 s, 52.4 MB/s 2024-04-25T10:23:20.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.172 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.173 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188917 s, 55.5 MB/s 2024-04-25T10:23:20.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.192 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.192 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178071 s, 58.9 MB/s 2024-04-25T10:23:20.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.209 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149968 s, 69.9 MB/s 2024-04-25T10:23:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018223 s, 57.5 MB/s 2024-04-25T10:23:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.250 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195721 s, 53.6 MB/s 2024-04-25T10:23:20.273 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.273 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.273 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212823 s, 49.3 MB/s 2024-04-25T10:23:20.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.293 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.293 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180805 s, 58.0 MB/s 2024-04-25T10:23:20.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.310 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.310 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154715 s, 67.8 MB/s 2024-04-25T10:23:20.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.329 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016856 s, 62.2 MB/s 2024-04-25T10:23:20.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202702 s, 51.7 MB/s 2024-04-25T10:23:20.371 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.372 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.372 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187369 s, 56.0 MB/s 2024-04-25T10:23:20.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.389 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.389 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156894 s, 66.8 MB/s 2024-04-25T10:23:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.409 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.410 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183887 s, 57.0 MB/s 2024-04-25T10:23:20.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.430 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017615 s, 59.5 MB/s 2024-04-25T10:23:20.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.448 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.448 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171374 s, 61.2 MB/s 2024-04-25T10:23:20.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.465 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.465 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015151 s, 69.2 MB/s 2024-04-25T10:23:20.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.483 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.483 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165218 s, 63.5 MB/s 2024-04-25T10:23:20.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.506 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210472 s, 49.8 MB/s 2024-04-25T10:23:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.523 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.523 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146002 s, 71.8 MB/s 2024-04-25T10:23:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183981 s, 57.0 MB/s 2024-04-25T10:23:20.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015443 s, 67.9 MB/s 2024-04-25T10:23:20.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.579 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.579 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173797 s, 60.3 MB/s 2024-04-25T10:23:20.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.601 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198729 s, 52.8 MB/s 2024-04-25T10:23:20.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:20.603 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:20.603 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:20.603 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:23:20.603 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:20.603 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:20.604 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:20.604 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:20.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.680 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201739 s, 52.0 MB/s 2024-04-25T10:23:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159189 s, 65.9 MB/s 2024-04-25T10:23:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143706 s, 73.0 MB/s 2024-04-25T10:23:20.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.736 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.736 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199738 s, 52.5 MB/s 2024-04-25T10:23:20.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.752 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146952 s, 71.4 MB/s 2024-04-25T10:23:20.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.770 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154645 s, 67.8 MB/s 2024-04-25T10:23:20.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.789 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.789 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175265 s, 59.8 MB/s 2024-04-25T10:23:20.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.808 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.808 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169846 s, 61.7 MB/s 2024-04-25T10:23:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.829 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194073 s, 54.0 MB/s 2024-04-25T10:23:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229088 s, 45.8 MB/s 2024-04-25T10:23:20.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.879 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.879 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237288 s, 44.2 MB/s 2024-04-25T10:23:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.897 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016657 s, 63.0 MB/s 2024-04-25T10:23:20.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.918 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.918 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196146 s, 53.5 MB/s 2024-04-25T10:23:20.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.935 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.935 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148335 s, 70.7 MB/s 2024-04-25T10:23:20.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.953 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.953 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166222 s, 63.1 MB/s 2024-04-25T10:23:20.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.969 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.969 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144244 s, 72.7 MB/s 2024-04-25T10:23:20.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:20.991 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:20.992 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202612 s, 51.8 MB/s 2024-04-25T10:23:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154168 s, 68.0 MB/s 2024-04-25T10:23:21.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.027 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.027 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016536 s, 63.4 MB/s 2024-04-25T10:23:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.045 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168262 s, 62.3 MB/s 2024-04-25T10:23:21.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.062 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156727 s, 66.9 MB/s 2024-04-25T10:23:21.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.082 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.082 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165968 s, 63.2 MB/s 2024-04-25T10:23:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174682 s, 60.0 MB/s 2024-04-25T10:23:21.118 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.118 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.118 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158319 s, 66.2 MB/s 2024-04-25T10:23:21.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.141 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.141 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02128 s, 49.3 MB/s 2024-04-25T10:23:21.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.159 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01643 s, 63.8 MB/s 2024-04-25T10:23:21.182 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.183 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.183 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219129 s, 47.9 MB/s 2024-04-25T10:23:21.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.199 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.199 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149039 s, 70.4 MB/s 2024-04-25T10:23:21.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.223 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221351 s, 47.4 MB/s 2024-04-25T10:23:21.240 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.240 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.240 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148974 s, 70.4 MB/s 2024-04-25T10:23:21.256 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.256 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.256 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145215 s, 72.2 MB/s 2024-04-25T10:23:21.273 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.274 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.274 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155769 s, 67.3 MB/s 2024-04-25T10:23:21.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:21.275 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:21.275 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:21.275 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:23:21.275 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:21.275 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:21.275 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:21.275 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:21.353 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.354 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.354 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186001 s, 56.4 MB/s 2024-04-25T10:23:21.373 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.374 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.374 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181633 s, 57.7 MB/s 2024-04-25T10:23:21.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.393 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177108 s, 59.2 MB/s 2024-04-25T10:23:21.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.410 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.410 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153996 s, 68.1 MB/s 2024-04-25T10:23:21.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.429 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.429 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172591 s, 60.8 MB/s 2024-04-25T10:23:21.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.447 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.447 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155052 s, 67.6 MB/s 2024-04-25T10:23:21.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.464 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.464 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154344 s, 67.9 MB/s 2024-04-25T10:23:21.481 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.482 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.482 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157749 s, 66.5 MB/s 2024-04-25T10:23:21.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.506 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.506 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222785 s, 47.1 MB/s 2024-04-25T10:23:21.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.529 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.529 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211694 s, 49.5 MB/s 2024-04-25T10:23:21.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.546 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.546 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015197 s, 69.0 MB/s 2024-04-25T10:23:21.568 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.568 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.568 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198517 s, 52.8 MB/s 2024-04-25T10:23:21.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.586 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.586 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161406 s, 65.0 MB/s 2024-04-25T10:23:21.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.603 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.603 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150872 s, 69.5 MB/s 2024-04-25T10:23:21.623 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.624 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190268 s, 55.1 MB/s 2024-04-25T10:23:21.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.648 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227407 s, 46.1 MB/s 2024-04-25T10:23:21.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.668 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.668 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184821 s, 56.7 MB/s 2024-04-25T10:23:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.686 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156697 s, 66.9 MB/s 2024-04-25T10:23:21.702 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.702 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.702 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141363 s, 74.2 MB/s 2024-04-25T10:23:21.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.715 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.715 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117368 s, 89.3 MB/s 2024-04-25T10:23:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01631 s, 64.3 MB/s 2024-04-25T10:23:21.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.754 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.754 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165831 s, 63.2 MB/s 2024-04-25T10:23:21.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.771 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.771 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174953 s, 59.9 MB/s 2024-04-25T10:23:21.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.787 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145387 s, 72.1 MB/s 2024-04-25T10:23:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164091 s, 63.9 MB/s 2024-04-25T10:23:21.827 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.828 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.828 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201817 s, 52.0 MB/s 2024-04-25T10:23:21.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.849 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.849 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199979 s, 52.4 MB/s 2024-04-25T10:23:21.868 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.868 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.868 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175302 s, 59.8 MB/s 2024-04-25T10:23:21.884 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.885 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.885 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146759 s, 71.4 MB/s 2024-04-25T10:23:21.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.904 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.904 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017367 s, 60.4 MB/s 2024-04-25T10:23:21.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.923 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170956 s, 61.3 MB/s 2024-04-25T10:23:21.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:21.945 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:21.945 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020504 s, 51.1 MB/s 2024-04-25T10:23:21.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:21.947 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:21.947 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:21.947 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:23:21.947 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:21.947 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:21.947 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:21.947 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:22.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.019 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.019 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015019 s, 69.8 MB/s 2024-04-25T10:23:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147071 s, 71.3 MB/s 2024-04-25T10:23:22.051 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.052 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.052 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144905 s, 72.4 MB/s 2024-04-25T10:23:22.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.069 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.069 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156973 s, 66.8 MB/s 2024-04-25T10:23:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222827 s, 47.1 MB/s 2024-04-25T10:23:22.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.115 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202704 s, 51.7 MB/s 2024-04-25T10:23:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.133 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015749 s, 66.6 MB/s 2024-04-25T10:23:22.153 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.154 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.154 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191563 s, 54.7 MB/s 2024-04-25T10:23:22.170 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.171 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.171 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153815 s, 68.2 MB/s 2024-04-25T10:23:22.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.193 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.193 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206764 s, 50.7 MB/s 2024-04-25T10:23:22.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.211 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.211 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163408 s, 64.2 MB/s 2024-04-25T10:23:22.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.228 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.228 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152931 s, 68.6 MB/s 2024-04-25T10:23:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.247 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.247 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168737 s, 62.1 MB/s 2024-04-25T10:23:22.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.265 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.265 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171068 s, 61.3 MB/s 2024-04-25T10:23:22.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.287 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.287 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198136 s, 52.9 MB/s 2024-04-25T10:23:22.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.304 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.304 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149592 s, 70.1 MB/s 2024-04-25T10:23:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157446 s, 66.6 MB/s 2024-04-25T10:23:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168974 s, 62.1 MB/s 2024-04-25T10:23:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017818 s, 58.8 MB/s 2024-04-25T10:23:22.378 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.379 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173918 s, 60.3 MB/s 2024-04-25T10:23:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148675 s, 70.5 MB/s 2024-04-25T10:23:22.411 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.412 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.412 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143957 s, 72.8 MB/s 2024-04-25T10:23:22.430 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.430 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.430 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169873 s, 61.7 MB/s 2024-04-25T10:23:22.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.454 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219573 s, 47.8 MB/s 2024-04-25T10:23:22.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.473 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172619 s, 60.7 MB/s 2024-04-25T10:23:22.490 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155984 s, 67.2 MB/s 2024-04-25T10:23:22.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.509 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.509 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164484 s, 63.7 MB/s 2024-04-25T10:23:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164922 s, 63.6 MB/s 2024-04-25T10:23:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108194 s, 96.9 MB/s 2024-04-25T10:23:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196624 s, 53.3 MB/s 2024-04-25T10:23:22.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.583 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.583 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204078 s, 51.4 MB/s 2024-04-25T10:23:22.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.602 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.602 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169851 s, 61.7 MB/s 2024-04-25T10:23:22.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:22.604 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:22.604 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:22.604 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:23:22.604 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:22.604 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:22.604 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:22.604 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:22.673 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.674 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144669 s, 72.5 MB/s 2024-04-25T10:23:22.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.692 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162845 s, 64.4 MB/s 2024-04-25T10:23:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016863 s, 62.2 MB/s 2024-04-25T10:23:22.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.737 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.737 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246545 s, 42.5 MB/s 2024-04-25T10:23:22.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.753 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.753 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147157 s, 71.3 MB/s 2024-04-25T10:23:22.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.770 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.770 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150024 s, 69.9 MB/s 2024-04-25T10:23:22.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.788 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.788 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015756 s, 66.6 MB/s 2024-04-25T10:23:22.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.804 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.804 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147901 s, 70.9 MB/s 2024-04-25T10:23:22.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.821 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.821 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152968 s, 68.5 MB/s 2024-04-25T10:23:22.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.843 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.843 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199725 s, 52.5 MB/s 2024-04-25T10:23:22.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.862 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.863 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178788 s, 58.6 MB/s 2024-04-25T10:23:22.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.887 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226064 s, 46.4 MB/s 2024-04-25T10:23:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.908 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194224 s, 54.0 MB/s 2024-04-25T10:23:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144442 s, 72.6 MB/s 2024-04-25T10:23:22.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.946 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.946 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197417 s, 53.1 MB/s 2024-04-25T10:23:22.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.968 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.968 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202187 s, 51.9 MB/s 2024-04-25T10:23:22.989 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:22.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:22.990 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197918 s, 53.0 MB/s 2024-04-25T10:23:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173056 s, 60.6 MB/s 2024-04-25T10:23:23.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.031 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.031 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201517 s, 52.0 MB/s 2024-04-25T10:23:23.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.048 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.048 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155989 s, 67.2 MB/s 2024-04-25T10:23:23.067 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.067 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.067 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173767 s, 60.3 MB/s 2024-04-25T10:23:23.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.092 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.092 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234431 s, 44.7 MB/s 2024-04-25T10:23:23.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.109 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.109 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151714 s, 69.1 MB/s 2024-04-25T10:23:23.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.128 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.128 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165584 s, 63.3 MB/s 2024-04-25T10:23:23.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.149 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.149 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192904 s, 54.4 MB/s 2024-04-25T10:23:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146186 s, 71.7 MB/s 2024-04-25T10:23:23.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170973 s, 61.3 MB/s 2024-04-25T10:23:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196182 s, 53.4 MB/s 2024-04-25T10:23:23.223 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.224 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164983 s, 63.6 MB/s 2024-04-25T10:23:23.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.241 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.241 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161831 s, 64.8 MB/s 2024-04-25T10:23:23.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.257 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139486 s, 75.2 MB/s 2024-04-25T10:23:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203436 s, 51.5 MB/s 2024-04-25T10:23:23.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:23.281 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:23.281 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:23.281 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:23:23.281 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:23.281 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:23.281 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:23.281 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:23.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.357 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.357 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181127 s, 57.9 MB/s 2024-04-25T10:23:23.375 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.376 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.376 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163846 s, 64.0 MB/s 2024-04-25T10:23:23.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.393 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01546 s, 67.8 MB/s 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225024 s, 46.6 MB/s 2024-04-25T10:23:23.434 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.435 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159158 s, 65.9 MB/s 2024-04-25T10:23:23.454 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.455 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.455 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181684 s, 57.7 MB/s 2024-04-25T10:23:23.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.472 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.472 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159642 s, 65.7 MB/s 2024-04-25T10:23:23.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.489 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.489 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149049 s, 70.4 MB/s 2024-04-25T10:23:23.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168582 s, 62.2 MB/s 2024-04-25T10:23:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148218 s, 70.7 MB/s 2024-04-25T10:23:23.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.543 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.543 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168811 s, 62.1 MB/s 2024-04-25T10:23:23.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156133 s, 67.2 MB/s 2024-04-25T10:23:23.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.577 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.577 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152532 s, 68.7 MB/s 2024-04-25T10:23:23.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.597 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.597 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179659 s, 58.4 MB/s 2024-04-25T10:23:23.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.618 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.619 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191077 s, 54.9 MB/s 2024-04-25T10:23:23.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.638 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.638 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177973 s, 58.9 MB/s 2024-04-25T10:23:23.656 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.656 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.656 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162097 s, 64.7 MB/s 2024-04-25T10:23:23.674 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.675 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.675 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172293 s, 60.9 MB/s 2024-04-25T10:23:23.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.692 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.693 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159906 s, 65.6 MB/s 2024-04-25T10:23:23.711 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.711 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.711 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169 s, 62.0 MB/s 2024-04-25T10:23:23.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174221 s, 60.2 MB/s 2024-04-25T10:23:23.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.750 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.750 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178361 s, 58.8 MB/s 2024-04-25T10:23:23.768 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.769 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171003 s, 61.3 MB/s 2024-04-25T10:23:23.786 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.787 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164499 s, 63.7 MB/s 2024-04-25T10:23:23.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.808 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019039 s, 55.1 MB/s 2024-04-25T10:23:23.830 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.831 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.831 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212178 s, 49.4 MB/s 2024-04-25T10:23:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199114 s, 52.7 MB/s 2024-04-25T10:23:23.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.870 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.870 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163267 s, 64.2 MB/s 2024-04-25T10:23:23.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.893 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.893 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211636 s, 49.5 MB/s 2024-04-25T10:23:23.900 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:23.903 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:23.910 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.911 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.911 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156703 s, 66.9 MB/s 2024-04-25T10:23:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.929 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164499 s, 63.7 MB/s 2024-04-25T10:23:23.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:23.947 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:23.947 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159848 s, 65.6 MB/s 2024-04-25T10:23:23.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:23.948 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:23.948 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:23.948 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:23:23.948 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:23.948 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:23.948 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:23.948 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:24.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.038 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.038 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196555 s, 53.3 MB/s 2024-04-25T10:23:24.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.063 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.063 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230891 s, 45.4 MB/s 2024-04-25T10:23:24.079 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.080 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.080 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146331 s, 71.7 MB/s 2024-04-25T10:23:24.095 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.096 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.096 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145268 s, 72.2 MB/s 2024-04-25T10:23:24.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.112 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.113 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152983 s, 68.5 MB/s 2024-04-25T10:23:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148266 s, 70.7 MB/s 2024-04-25T10:23:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141963 s, 73.9 MB/s 2024-04-25T10:23:24.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.168 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.168 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209727 s, 50.0 MB/s 2024-04-25T10:23:24.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.190 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.190 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201629 s, 52.0 MB/s 2024-04-25T10:23:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163526 s, 64.1 MB/s 2024-04-25T10:23:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146628 s, 71.5 MB/s 2024-04-25T10:23:24.243 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.244 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.244 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172974 s, 60.6 MB/s 2024-04-25T10:23:24.262 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.262 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.262 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171848 s, 61.0 MB/s 2024-04-25T10:23:24.279 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.280 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.280 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015427 s, 68.0 MB/s 2024-04-25T10:23:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.301 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202835 s, 51.7 MB/s 2024-04-25T10:23:24.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.316 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.316 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128456 s, 81.6 MB/s 2024-04-25T10:23:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.333 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146138 s, 71.8 MB/s 2024-04-25T10:23:24.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.349 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.349 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145851 s, 71.9 MB/s 2024-04-25T10:23:24.368 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180058 s, 58.2 MB/s 2024-04-25T10:23:24.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.384 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.385 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142606 s, 73.5 MB/s 2024-04-25T10:23:24.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.403 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.403 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167647 s, 62.5 MB/s 2024-04-25T10:23:24.420 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.421 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160268 s, 65.4 MB/s 2024-04-25T10:23:24.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.443 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.443 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207723 s, 50.5 MB/s 2024-04-25T10:23:24.462 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.462 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.462 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168183 s, 62.3 MB/s 2024-04-25T10:23:24.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.479 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.479 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154193 s, 68.0 MB/s 2024-04-25T10:23:24.496 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.497 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.497 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014778 s, 71.0 MB/s 2024-04-25T10:23:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.518 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.519 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206096 s, 50.9 MB/s 2024-04-25T10:23:24.535 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.536 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.536 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153493 s, 68.3 MB/s 2024-04-25T10:23:24.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.552 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.552 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015007 s, 69.9 MB/s 2024-04-25T10:23:24.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.575 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.575 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204028 s, 51.4 MB/s 2024-04-25T10:23:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.593 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170117 s, 61.6 MB/s 2024-04-25T10:23:24.612 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.612 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.612 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017206 s, 60.9 MB/s 2024-04-25T10:23:24.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:24.614 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:24.614 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:24.614 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:23:24.614 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:24.614 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:24.614 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:24.614 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:24.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.686 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.686 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142729 s, 73.5 MB/s 2024-04-25T10:23:24.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.705 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176343 s, 59.5 MB/s 2024-04-25T10:23:24.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.731 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239535 s, 43.8 MB/s 2024-04-25T10:23:24.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.749 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.749 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016482 s, 63.6 MB/s 2024-04-25T10:23:24.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.766 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.767 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157298 s, 66.7 MB/s 2024-04-25T10:23:24.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.787 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.788 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192423 s, 54.5 MB/s 2024-04-25T10:23:24.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.807 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.807 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178636 s, 58.7 MB/s 2024-04-25T10:23:24.824 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.825 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.825 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156634 s, 66.9 MB/s 2024-04-25T10:23:24.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.841 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.841 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146358 s, 71.6 MB/s 2024-04-25T10:23:24.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.857 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.857 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145252 s, 72.2 MB/s 2024-04-25T10:23:24.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.874 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.874 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151883 s, 69.0 MB/s 2024-04-25T10:23:24.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.896 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.896 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199024 s, 52.7 MB/s 2024-04-25T10:23:24.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.913 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.913 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149769 s, 70.0 MB/s 2024-04-25T10:23:24.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.934 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.934 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189561 s, 55.3 MB/s 2024-04-25T10:23:24.955 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.955 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.955 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198333 s, 52.9 MB/s 2024-04-25T10:23:24.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.977 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.977 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203099 s, 51.6 MB/s 2024-04-25T10:23:24.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:24.997 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:24.998 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187625 s, 55.9 MB/s 2024-04-25T10:23:25.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.021 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.021 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221678 s, 47.3 MB/s 2024-04-25T10:23:25.042 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.042 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.043 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192748 s, 54.4 MB/s 2024-04-25T10:23:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196806 s, 53.3 MB/s 2024-04-25T10:23:25.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.085 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.085 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194608 s, 53.9 MB/s 2024-04-25T10:23:25.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.102 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.102 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153751 s, 68.2 MB/s 2024-04-25T10:23:25.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.120 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.120 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165088 s, 63.5 MB/s 2024-04-25T10:23:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.142 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200823 s, 52.2 MB/s 2024-04-25T10:23:25.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.159 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.159 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147881 s, 70.9 MB/s 2024-04-25T10:23:25.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.184 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.184 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232253 s, 45.1 MB/s 2024-04-25T10:23:25.203 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.203 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.203 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172306 s, 60.9 MB/s 2024-04-25T10:23:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.219 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143565 s, 73.0 MB/s 2024-04-25T10:23:25.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.236 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.236 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151677 s, 69.1 MB/s 2024-04-25T10:23:25.252 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.253 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.253 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015047 s, 69.7 MB/s 2024-04-25T10:23:25.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.270 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.270 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158964 s, 66.0 MB/s 2024-04-25T10:23:25.292 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.292 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.292 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198402 s, 52.9 MB/s 2024-04-25T10:23:25.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:25.293 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:23:25.293 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:25.293 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:23:25.294 DEBUG:teuthology.orchestra.run.smithi087:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:23:25.294 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:23:25.294 DEBUG:teuthology.orchestra.run.smithi087:> done 2024-04-25T10:23:25.294 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:25.365 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.366 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.366 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145214 s, 72.2 MB/s 2024-04-25T10:23:25.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.379 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.380 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122544 s, 85.6 MB/s 2024-04-25T10:23:25.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.397 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.397 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159558 s, 65.7 MB/s 2024-04-25T10:23:25.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.419 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.420 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203334 s, 51.6 MB/s 2024-04-25T10:23:25.438 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.439 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.439 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173758 s, 60.3 MB/s 2024-04-25T10:23:25.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.456 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.456 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155502 s, 67.4 MB/s 2024-04-25T10:23:25.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.473 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.473 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150097 s, 69.9 MB/s 2024-04-25T10:23:25.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.491 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.491 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163571 s, 64.1 MB/s 2024-04-25T10:23:25.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.508 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.508 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150143 s, 69.8 MB/s 2024-04-25T10:23:25.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.524 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.524 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014571 s, 72.0 MB/s 2024-04-25T10:23:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157557 s, 66.6 MB/s 2024-04-25T10:23:25.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.560 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.560 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016319 s, 64.3 MB/s 2024-04-25T10:23:25.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.576 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.576 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143475 s, 73.1 MB/s 2024-04-25T10:23:25.594 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.595 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.595 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171126 s, 61.3 MB/s 2024-04-25T10:23:25.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.613 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.613 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167665 s, 62.5 MB/s 2024-04-25T10:23:25.630 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.631 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.631 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156659 s, 66.9 MB/s 2024-04-25T10:23:25.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.648 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.648 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156408 s, 67.0 MB/s 2024-04-25T10:23:25.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.664 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.664 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140612 s, 74.6 MB/s 2024-04-25T10:23:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.680 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.681 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148019 s, 70.8 MB/s 2024-04-25T10:23:25.696 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.697 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.697 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144953 s, 72.3 MB/s 2024-04-25T10:23:25.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.714 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.714 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151138 s, 69.4 MB/s 2024-04-25T10:23:25.730 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.731 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.731 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151846 s, 69.1 MB/s 2024-04-25T10:23:25.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.752 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.752 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195264 s, 53.7 MB/s 2024-04-25T10:23:25.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.769 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.769 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149846 s, 70.0 MB/s 2024-04-25T10:23:25.781 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.782 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.782 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112774 s, 93.0 MB/s 2024-04-25T10:23:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.799 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.799 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015321 s, 68.4 MB/s 2024-04-25T10:23:25.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.816 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.816 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155645 s, 67.4 MB/s 2024-04-25T10:23:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179039 s, 58.6 MB/s 2024-04-25T10:23:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146618 s, 71.5 MB/s 2024-04-25T10:23:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014067 s, 74.5 MB/s 2024-04-25T10:23:25.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.888 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176334 s, 59.5 MB/s 2024-04-25T10:23:25.909 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T10:23:25.910 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T10:23:25.910 INFO:teuthology.orchestra.run.smithi087.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200644 s, 52.3 MB/s 2024-04-25T10:23:25.911 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T10:23:26.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe509e00640 1 Processor -- start 2024-04-25T10:23:26.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe509e00640 1 -- start start 2024-04-25T10:23:26.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe509e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5040aef80 0x7fe50415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe509e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe504070350 0x7fe50415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe509e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50415b080 con 0x7fe50415bac0 2024-04-25T10:23:26.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe509e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50415adb0 con 0x7fe504070350 2024-04-25T10:23:26.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe509e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50415eb60 con 0x7fe5040aef80 2024-04-25T10:23:26.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe502a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5040aef80 0x7fe50415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe503400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe50415bac0 0x7fe5040b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:49772/0 (socket says 172.21.15.87:49772) 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe502000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe504070350 0x7fe50415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe503400640 1 -- 172.21.15.87:0/524131767 learned_addr learned my addr 172.21.15.87:0/524131767 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe502a00640 1 -- 172.21.15.87:0/524131767 >> v2:172.21.15.169:3300/0 conn(0x7fe504070350 msgr2=0x7fe50415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe502a00640 1 --2- 172.21.15.87:0/524131767 >> v2:172.21.15.169:3300/0 conn(0x7fe504070350 0x7fe50415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe502a00640 1 -- 172.21.15.87:0/524131767 >> v1:172.21.15.169:6789/0 conn(0x7fe50415bac0 legacy=0x7fe5040b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe502a00640 1 -- 172.21.15.87:0/524131767 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50415ee40 con 0x7fe5040aef80 2024-04-25T10:23:26.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.093+0000 7fe502000640 1 --2- 172.21.15.87:0/524131767 >> v2:172.21.15.169:3300/0 conn(0x7fe504070350 0x7fe50415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:26.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe502a00640 1 --2- 172.21.15.87:0/524131767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5040aef80 0x7fe50415e480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009d90 tx=0x7fe4f401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49dc19707aa53e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe501600640 1 -- 172.21.15.87:0/524131767 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40b2070 con 0x7fe5040aef80 2024-04-25T10:23:26.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe501600640 1 -- 172.21.15.87:0/524131767 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f40a5a10 con 0x7fe5040aef80 2024-04-25T10:23:26.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe501600640 1 -- 172.21.15.87:0/524131767 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40a5de0 con 0x7fe5040aef80 2024-04-25T10:23:26.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 -- 172.21.15.87:0/524131767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5040aef80 msgr2=0x7fe50415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 --2- 172.21.15.87:0/524131767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5040aef80 0x7fe50415e480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009d90 tx=0x7fe4f401c970 comp rx=0 tx=0).stop 2024-04-25T10:23:26.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 -- 172.21.15.87:0/524131767 shutdown_connections 2024-04-25T10:23:26.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 --2- 172.21.15.87:0/524131767 >> v2:172.21.15.169:3300/0 conn(0x7fe504070350 0x7fe50415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 --2- 172.21.15.87:0/524131767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5040aef80 0x7fe50415e480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 -- 172.21.15.87:0/524131767 >> 172.21.15.87:0/524131767 conn(0x7fe5040669a0 msgr2=0x7fe504066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:26.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 -- 172.21.15.87:0/524131767 shutdown_connections 2024-04-25T10:23:26.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.097+0000 7fe509e00640 1 -- 172.21.15.87:0/524131767 wait complete. 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 Processor -- start 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 -- start start 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe504070350 0x7fe50413f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5040aef80 0x7fe50415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe50415bac0 0x7fe50415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50406c250 con 0x7fe50415bac0 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50406c0d0 con 0x7fe5040aef80 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50406c3d0 con 0x7fe504070350 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe50415bac0 0x7fe50415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe50415bac0 0x7fe50415cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44538/0 (socket says 172.21.15.87:44538) 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 -- 172.21.15.87:0/2979525829 learned_addr learned my addr 172.21.15.87:0/2979525829 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe502a00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe504070350 0x7fe50413f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe502000640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5040aef80 0x7fe50415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 -- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe504070350 msgr2=0x7fe50413f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe504070350 0x7fe50413f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 -- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5040aef80 msgr2=0x7fe50415c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5040aef80 0x7fe50415c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 -- 172.21.15.87:0/2979525829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50413f650 con 0x7fe50415bac0 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe502000640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5040aef80 0x7fe50415c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe503400640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe50415bac0 0x7fe50415cce0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c3f0 tx=0x7fe4fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe4fb400640 1 -- 172.21.15.87:0/2979525829 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc017070 con 0x7fe50415bac0 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe4fb400640 1 -- 172.21.15.87:0/2979525829 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc005cf0 con 0x7fe50415bac0 2024-04-25T10:23:26.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe4fb400640 1 -- 172.21.15.87:0/2979525829 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0072f0 con 0x7fe50415bac0 2024-04-25T10:23:26.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe504159e30 con 0x7fe50415bac0 2024-04-25T10:23:26.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.101+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50414d690 con 0x7fe50415bac0 2024-04-25T10:23:26.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.105+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50404ef30 con 0x7fe50415bac0 2024-04-25T10:23:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.105+0000 7fe4fb400640 1 -- 172.21.15.87:0/2979525829 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4fc01a070 con 0x7fe50415bac0 2024-04-25T10:23:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.105+0000 7fe4fb400640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe4e4066f60 0x7fe4e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.105+0000 7fe4fb400640 1 -- 172.21.15.87:0/2979525829 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe4fc0842d0 con 0x7fe50415bac0 2024-04-25T10:23:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.105+0000 7fe502a00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe4e4066f60 0x7fe4e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.105+0000 7fe502a00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe4e4066f60 0x7fe4e4069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40af000 tx=0x7fe4f40a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.109+0000 7fe4fb400640 1 -- 172.21.15.87:0/2979525829 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4fc054c60 con 0x7fe50415bac0 2024-04-25T10:23:26.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.297+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fe5040b2fa0 con 0x7fe50415bac0 2024-04-25T10:23:26.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.485+0000 7fe4fb400640 1 -- 172.21.15.87:0/2979525829 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v101) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe4fc058d00 con 0x7fe50415bac0 2024-04-25T10:23:26.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.485+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe4e4066f60 msgr2=0x7fe4e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.485+0000 7fe509e00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe4e4066f60 0x7fe4e4069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40af000 tx=0x7fe4f40a7070 comp rx=0 tx=0).stop 2024-04-25T10:23:26.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.485+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe50415bac0 msgr2=0x7fe50415cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.485+0000 7fe509e00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe50415bac0 0x7fe50415cce0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c3f0 tx=0x7fe4fc00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 shutdown_connections 2024-04-25T10:23:26.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe4e4066f60 0x7fe4e4069420 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe50415bac0 0x7fe50415cce0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5040aef80 0x7fe50415c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 --2- 172.21.15.87:0/2979525829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe504070350 0x7fe50413f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 >> 172.21.15.87:0/2979525829 conn(0x7fe5040669a0 msgr2=0x7fe50415d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:26.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 shutdown_connections 2024-04-25T10:23:26.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.489+0000 7fe509e00640 1 -- 172.21.15.87:0/2979525829 wait complete. 2024-04-25T10:23:26.508 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:23:26.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.649+0000 7fbf9c400640 1 Processor -- start 2024-04-25T10:23:26.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf9c400640 1 -- start start 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf9c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbf94159540 0x7fbf940ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf9c400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf94159920 con 0x7fbf940aebd0 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf9c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9415c2f0 con 0x7fbf940721f0 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf9c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9415c5d0 con 0x7fbf94159540 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf99a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbf94159540 0x7fbf940ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf99000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbf940aebd0 0x7fbf94071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:40886/0 (socket says 172.21.15.87:40886) 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf99000640 1 -- 172.21.15.87:0/1770877446 learned_addr learned my addr 172.21.15.87:0/1770877446 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:26.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf99a00640 1 -- 172.21.15.87:0/1770877446 >> v1:172.21.15.169:6789/0 conn(0x7fbf940721f0 legacy=0x7fbf94070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf99a00640 1 -- 172.21.15.87:0/1770877446 >> v1:172.21.15.87:6789/0 conn(0x7fbf940aebd0 legacy=0x7fbf94071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.653+0000 7fbf99a00640 1 -- 172.21.15.87:0/1770877446 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9415c8b0 con 0x7fbf94159540 2024-04-25T10:23:26.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf99a00640 1 --2- 172.21.15.87:0/1770877446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf94159540 0x7fbf940ae690 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fbf880097c0 tx=0x7fbf8801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366830210abf87a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf93e00640 1 -- 172.21.15.87:0/1770877446 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880b1070 con 0x7fbf94159540 2024-04-25T10:23:26.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf93e00640 1 -- 172.21.15.87:0/1770877446 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf880ac410 con 0x7fbf94159540 2024-04-25T10:23:26.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf93e00640 1 -- 172.21.15.87:0/1770877446 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880ac7e0 con 0x7fbf94159540 2024-04-25T10:23:26.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 -- 172.21.15.87:0/1770877446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf94159540 msgr2=0x7fbf940ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1770877446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf94159540 0x7fbf940ae690 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fbf880097c0 tx=0x7fbf8801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 -- 172.21.15.87:0/1770877446 shutdown_connections 2024-04-25T10:23:26.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1770877446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf94159540 0x7fbf940ae690 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 -- 172.21.15.87:0/1770877446 >> 172.21.15.87:0/1770877446 conn(0x7fbf940669a0 msgr2=0x7fbf94066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:26.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 -- 172.21.15.87:0/1770877446 shutdown_connections 2024-04-25T10:23:26.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 -- 172.21.15.87:0/1770877446 wait complete. 2024-04-25T10:23:26.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 Processor -- start 2024-04-25T10:23:26.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.657+0000 7fbf9c400640 1 -- start start 2024-04-25T10:23:26.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf940721f0 0x7fbf94153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf940aebd0 0x7fbf940b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf94159540 0x7fbf940adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf94169ac0 con 0x7fbf940aebd0 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf94169940 con 0x7fbf94159540 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf94169c40 con 0x7fbf940721f0 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf940aebd0 0x7fbf940b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf940aebd0 0x7fbf940b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44562/0 (socket says 172.21.15.87:44562) 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 -- 172.21.15.87:0/1329540687 learned_addr learned my addr 172.21.15.87:0/1329540687 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99a00640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf940721f0 0x7fbf94153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9a400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf94159540 0x7fbf940adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 -- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf940721f0 msgr2=0x7fbf94153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf940721f0 0x7fbf94153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 -- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf94159540 msgr2=0x7fbf940adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:26.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf94159540 0x7fbf940adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:26.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 -- 172.21.15.87:0/1329540687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9413d730 con 0x7fbf940aebd0 2024-04-25T10:23:26.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9a400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf94159540 0x7fbf940adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:26.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf99000640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf940aebd0 0x7fbf940b4200 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000c500 tx=0x7fbf8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf91600640 1 -- 172.21.15.87:0/1329540687 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80017070 con 0x7fbf940aebd0 2024-04-25T10:23:26.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf91600640 1 -- 172.21.15.87:0/1329540687 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80005cf0 con 0x7fbf940aebd0 2024-04-25T10:23:26.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9413d9d0 con 0x7fbf940aebd0 2024-04-25T10:23:26.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf91600640 1 -- 172.21.15.87:0/1329540687 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf800072f0 con 0x7fbf940aebd0 2024-04-25T10:23:26.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9413df90 con 0x7fbf940aebd0 2024-04-25T10:23:26.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9404ef30 con 0x7fbf940aebd0 2024-04-25T10:23:26.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.661+0000 7fbf91600640 1 -- 172.21.15.87:0/1329540687 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf8001a070 con 0x7fbf940aebd0 2024-04-25T10:23:26.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.665+0000 7fbf91600640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f60 0x7fbf74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:26.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.665+0000 7fbf91600640 1 -- 172.21.15.87:0/1329540687 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbf80084280 con 0x7fbf940aebd0 2024-04-25T10:23:26.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.665+0000 7fbf99a00640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f60 0x7fbf74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:26.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.665+0000 7fbf99a00640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f60 0x7fbf74069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbf880ae000 tx=0x7fbf88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:26.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.669+0000 7fbf91600640 1 -- 172.21.15.87:0/1329540687 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf80054cb0 con 0x7fbf940aebd0 2024-04-25T10:23:26.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:26.853+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fbf9415e8e0 con 0x7fbf940aebd0 2024-04-25T10:23:27.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.497+0000 7fbf91600640 1 -- 172.21.15.87:0/1329540687 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v102) v1 ==== 121+0+0 (secure 0 0 0) 0x7fbf80058d50 con 0x7fbf940aebd0 2024-04-25T10:23:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f60 msgr2=0x7fbf74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f60 0x7fbf74069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbf880ae000 tx=0x7fbf88008040 comp rx=0 tx=0).stop 2024-04-25T10:23:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf940aebd0 msgr2=0x7fbf940b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf940aebd0 0x7fbf940b4200 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000c500 tx=0x7fbf8000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 shutdown_connections 2024-04-25T10:23:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbf74066f60 0x7fbf74069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf94159540 0x7fbf940adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf940aebd0 0x7fbf940b4200 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 --2- 172.21.15.87:0/1329540687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf940721f0 0x7fbf94153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 >> 172.21.15.87:0/1329540687 conn(0x7fbf940669a0 msgr2=0x7fbf9415e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 shutdown_connections 2024-04-25T10:23:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.501+0000 7fbf9c400640 1 -- 172.21.15.87:0/1329540687 wait complete. 2024-04-25T10:23:27.522 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:27.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.661+0000 7f1b0e200640 1 Processor -- start 2024-04-25T10:23:27.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.661+0000 7f1b0e200640 1 -- start start 2024-04-25T10:23:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b0e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1b0815b910 0x7f1b0815c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b0e200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0815bd60 con 0x7f1b081550b0 2024-04-25T10:23:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b0e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b08071050 con 0x7f1b080d0040 2024-04-25T10:23:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b0e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b080712f0 con 0x7f1b0815b910 2024-04-25T10:23:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b07e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1b0815b910 0x7f1b0815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:27.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b07400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1b081550b0 0x7f1b080cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53462/0 (socket says 172.21.15.87:53462) 2024-04-25T10:23:27.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b07400640 1 -- 172.21.15.87:0/2457274411 learned_addr learned my addr 172.21.15.87:0/2457274411 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:27.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b07e00640 1 -- 172.21.15.87:0/2457274411 >> v1:172.21.15.169:6789/0 conn(0x7f1b080d0040 legacy=0x7f1b0815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:27.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b07e00640 1 -- 172.21.15.87:0/2457274411 >> v1:172.21.15.87:6789/0 conn(0x7f1b081550b0 legacy=0x7f1b080cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.665+0000 7f1b07e00640 1 -- 172.21.15.87:0/2457274411 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080715d0 con 0x7f1b0815b910 2024-04-25T10:23:27.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b07e00640 1 --2- 172.21.15.87:0/2457274411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0815b910 0x7f1b0815c030 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1b000097c0 tx=0x7f1b0001e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912a7a1a32c4f9f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:27.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b06000640 1 -- 172.21.15.87:0/2457274411 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b000a8610 con 0x7f1b0815b910 2024-04-25T10:23:27.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b06000640 1 -- 172.21.15.87:0/2457274411 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b000a87b0 con 0x7f1b0815b910 2024-04-25T10:23:27.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b06000640 1 -- 172.21.15.87:0/2457274411 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b000a8b80 con 0x7f1b0815b910 2024-04-25T10:23:27.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- 172.21.15.87:0/2457274411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0815b910 msgr2=0x7f1b0815c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 --2- 172.21.15.87:0/2457274411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0815b910 0x7f1b0815c030 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1b000097c0 tx=0x7f1b0001e640 comp rx=0 tx=0).stop 2024-04-25T10:23:27.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- 172.21.15.87:0/2457274411 shutdown_connections 2024-04-25T10:23:27.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 --2- 172.21.15.87:0/2457274411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0815b910 0x7f1b0815c030 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- 172.21.15.87:0/2457274411 >> 172.21.15.87:0/2457274411 conn(0x7f1b08066b70 msgr2=0x7f1b08066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:27.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- 172.21.15.87:0/2457274411 shutdown_connections 2024-04-25T10:23:27.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- 172.21.15.87:0/2457274411 wait complete. 2024-04-25T10:23:27.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 Processor -- start 2024-04-25T10:23:27.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- start start 2024-04-25T10:23:27.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b080d0040 0x7f1b0806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:27.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b081550b0 0x7f1b08147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:27.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0815b910 0x7f1b080ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:27.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815cf00 con 0x7f1b080d0040 2024-04-25T10:23:27.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815cd80 con 0x7f1b0815b910 2024-04-25T10:23:27.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.669+0000 7f1b0e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815d080 con 0x7f1b081550b0 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b080d0040 0x7f1b0806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b080d0040 0x7f1b0806d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39544/0 (socket says 172.21.15.87:39544) 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 -- 172.21.15.87:0/1575408949 learned_addr learned my addr 172.21.15.87:0/1575408949 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 -- 172.21.15.87:0/1575408949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b081550b0 msgr2=0x7f1b08147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b081550b0 0x7f1b08147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 -- 172.21.15.87:0/1575408949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0815b910 msgr2=0x7f1b080ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0815b910 0x7f1b080ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 -- 172.21.15.87:0/1575408949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b081477d0 con 0x7f1b080d0040 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b07400640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b080d0040 0x7f1b0806d2b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f1af800cb40 tx=0x7f1af800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:27.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1aff400640 1 -- 172.21.15.87:0/1575408949 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8017070 con 0x7f1b080d0040 2024-04-25T10:23:27.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1aff400640 1 -- 172.21.15.87:0/1575408949 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af800aca0 con 0x7f1b080d0040 2024-04-25T10:23:27.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0814e680 con 0x7f1b080d0040 2024-04-25T10:23:27.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1aff400640 1 -- 172.21.15.87:0/1575408949 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8012620 con 0x7f1b080d0040 2024-04-25T10:23:27.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0814ec40 con 0x7f1b080d0040 2024-04-25T10:23:27.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0804ef30 con 0x7f1b080d0040 2024-04-25T10:23:27.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1aff400640 1 -- 172.21.15.87:0/1575408949 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1af801a070 con 0x7f1b080d0040 2024-04-25T10:23:27.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1aff400640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1ae8066f60 0x7f1ae8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:27.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1aff400640 1 -- 172.21.15.87:0/1575408949 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1af8084e70 con 0x7f1b080d0040 2024-04-25T10:23:27.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.673+0000 7f1b06a00640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1ae8066f60 0x7f1ae8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:27.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.677+0000 7f1b06a00640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1ae8066f60 0x7f1ae8069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1b0806d8f0 tx=0x7f1af40079c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:27.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.677+0000 7f1aff400640 1 -- 172.21.15.87:0/1575408949 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1af8055dc0 con 0x7f1b080d0040 2024-04-25T10:23:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b080cf160 con 0x7f1b080d0040 2024-04-25T10:23:27.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1aff400640 1 -- 172.21.15.87:0/1575408949 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f1af80556c0 con 0x7f1b080d0040 2024-04-25T10:23:27.864 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:27.864 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":103,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:22:56.766091+0000","modified":"2024-04-25T10:23:27.501228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8755":{"gid":8755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6836/4075242775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4075242775},{"type":"v1","addr":"172.21.15.87:6836","nonce":4075242775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8758":{"gid":8758,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1025107360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1025107360},{"type":"v1","addr":"172.21.15.87:6837","nonce":1025107360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:23:27.865 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 103 2024-04-25T10:23:27.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1ae8066f60 msgr2=0x7f1ae8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1ae8066f60 0x7f1ae8069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1b0806d8f0 tx=0x7f1af40079c0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b080d0040 msgr2=0x7f1b0806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b080d0040 0x7f1b0806d2b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f1af800cb40 tx=0x7f1af800a060 comp rx=0 tx=0).stop 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 shutdown_connections 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1ae8066f60 0x7f1ae8069420 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0815b910 0x7f1b080ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b081550b0 0x7f1b08147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 --2- 172.21.15.87:0/1575408949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b080d0040 0x7f1b0806d2b0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.865+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 >> 172.21.15.87:0/1575408949 conn(0x7f1b08066b70 msgr2=0x7f1b080b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.869+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 shutdown_connections 2024-04-25T10:23:27.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:27.869+0000 7f1b0e200640 1 -- 172.21.15.87:0/1575408949 wait complete. 2024-04-25T10:23:28.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:22:56.766091+0000', 'modified': '2024-04-25T10:23:27.501228+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8758}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8755': {'gid': 8755, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.87:6836/4075242775', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 4075242775}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 4075242775}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8758': {'gid': 8758, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/1025107360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 1025107360}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 1025107360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:23:28.288 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:23:28.288 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:23:28.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.429+0000 7f2307400640 1 Processor -- start 2024-04-25T10:23:28.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.429+0000 7f2307400640 1 -- start start 2024-04-25T10:23:28.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.429+0000 7f2307400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000d0040 0x7f230006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.429+0000 7f2307400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23000b2770 con 0x7f230015bc40 2024-04-25T10:23:28.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.429+0000 7f2307400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230006c670 con 0x7f23000cda10 2024-04-25T10:23:28.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.429+0000 7f2307400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230006c950 con 0x7f23000d0040 2024-04-25T10:23:28.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f2304a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f230015bc40 0x7f23000b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53464/0 (socket says 172.21.15.87:53464) 2024-04-25T10:23:28.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f2304a00640 1 -- 172.21.15.87:0/3358103307 learned_addr learned my addr 172.21.15.87:0/3358103307 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:28.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f2305400640 1 --2- 172.21.15.87:0/3358103307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000d0040 0x7f230006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f22ff400640 1 -- 172.21.15.87:0/3358103307 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434412464 0 0) 0x7f23000b2770 con 0x7f230015bc40 2024-04-25T10:23:28.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f22ff400640 1 -- 172.21.15.87:0/3358103307 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e8003530 con 0x7f230015bc40 2024-04-25T10:23:28.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f22ff400640 1 -- 172.21.15.87:0/3358103307 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589675855 0 0) 0x7f230006c670 con 0x7f23000cda10 2024-04-25T10:23:28.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f22ff400640 1 -- 172.21.15.87:0/3358103307 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23000b2770 con 0x7f23000cda10 2024-04-25T10:23:28.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f2305400640 1 -- 172.21.15.87:0/3358103307 >> v1:172.21.15.169:6789/0 conn(0x7f23000cda10 legacy=0x7f23000cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f2305400640 1 -- 172.21.15.87:0/3358103307 >> v1:172.21.15.87:6789/0 conn(0x7f230015bc40 legacy=0x7f23000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f2305400640 1 -- 172.21.15.87:0/3358103307 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230006cc30 con 0x7f23000d0040 2024-04-25T10:23:28.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.433+0000 7f2305400640 1 --2- 172.21.15.87:0/3358103307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000d0040 0x7f230006c090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009dc0 tx=0x7f22f801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b086f8121ba553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f22ff400640 1 -- 172.21.15.87:0/3358103307 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80b2070 con 0x7f23000d0040 2024-04-25T10:23:28.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f22ff400640 1 -- 172.21.15.87:0/3358103307 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f80a58d0 con 0x7f23000d0040 2024-04-25T10:23:28.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f22ff400640 1 -- 172.21.15.87:0/3358103307 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80a5ca0 con 0x7f23000d0040 2024-04-25T10:23:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- 172.21.15.87:0/3358103307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000d0040 msgr2=0x7f230006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 --2- 172.21.15.87:0/3358103307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000d0040 0x7f230006c090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009dc0 tx=0x7f22f801c900 comp rx=0 tx=0).stop 2024-04-25T10:23:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- 172.21.15.87:0/3358103307 shutdown_connections 2024-04-25T10:23:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 --2- 172.21.15.87:0/3358103307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000d0040 0x7f230006c090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- 172.21.15.87:0/3358103307 >> 172.21.15.87:0/3358103307 conn(0x7f2300066b70 msgr2=0x7f2300066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- 172.21.15.87:0/3358103307 shutdown_connections 2024-04-25T10:23:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- 172.21.15.87:0/3358103307 wait complete. 2024-04-25T10:23:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 Processor -- start 2024-04-25T10:23:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- start start 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000cda10 0x7f230014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23000d0040 0x7f23001563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f230015bc40 0x7f2300152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300169980 con 0x7f23000d0040 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300169800 con 0x7f230015bc40 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2307400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300169b00 con 0x7f23000cda10 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23000d0040 0x7f23001563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23000d0040 0x7f23001563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39552/0 (socket says 172.21.15.87:39552) 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f22ffe00640 1 -- 172.21.15.87:0/4210580480 learned_addr learned my addr 172.21.15.87:0/4210580480 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.437+0000 7f2304a00640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000cda10 0x7f230014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f2305400640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f230015bc40 0x7f2300152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22ffe00640 1 -- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000cda10 msgr2=0x7f230014b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22ffe00640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000cda10 0x7f230014b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22ffe00640 1 -- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f230015bc40 msgr2=0x7f2300152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22ffe00640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f230015bc40 0x7f2300152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22ffe00640 1 -- 172.21.15.87:0/4210580480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2300156d20 con 0x7f23000d0040 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22ffe00640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23000d0040 0x7f23001563b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00c430 tx=0x7f22ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22fcc00640 1 -- 172.21.15.87:0/4210580480 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec017070 con 0x7f23000d0040 2024-04-25T10:23:28.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22fcc00640 1 -- 172.21.15.87:0/4210580480 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec0052f0 con 0x7f23000d0040 2024-04-25T10:23:28.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22fcc00640 1 -- 172.21.15.87:0/4210580480 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec003840 con 0x7f23000d0040 2024-04-25T10:23:28.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2300157000 con 0x7f23000d0040 2024-04-25T10:23:28.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23001575c0 con 0x7f23000d0040 2024-04-25T10:23:28.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22fcc00640 1 -- 172.21.15.87:0/4210580480 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22ec01a070 con 0x7f23000d0040 2024-04-25T10:23:28.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22fcc00640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22e0066f60 0x7f22e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f22fcc00640 1 -- 172.21.15.87:0/4210580480 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f22ec084450 con 0x7f23000d0040 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.441+0000 7f2304a00640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22e0066f60 0x7f22e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 Processor -- start 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- start start 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f2304a00640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22e0066f60 0x7f22e0069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f22f40011f0 tx=0x7f22f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a420dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a40ab5f0 con 0x55f2a420e240 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a4012da0 con 0x55f2a41c3f30 2024-04-25T10:23:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2a3f9b150 con 0x55f2a420d920 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39568/0 (socket says 172.21.15.87:39568) 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 -- 172.21.15.87:0/305579635 learned_addr learned my addr 172.21.15.87:0/305579635 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f2304a00640 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a420dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f2305400640 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 -- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 msgr2=0x55f2a420dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a420dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 -- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 msgr2=0x55f2a41c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 -- 172.21.15.87:0/305579635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2a3fb4ca0 con 0x55f2a420e240 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22ffe00640 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c39f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00a040 tx=0x7f22ec054d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3babfa08425a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22f1600640 1 -- 172.21.15.87:0/305579635 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec05b0c0 con 0x55f2a420e240 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22f1600640 1 -- 172.21.15.87:0/305579635 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec053030 con 0x55f2a420e240 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f22f1600640 1 -- 172.21.15.87:0/305579635 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0513a0 con 0x55f2a420e240 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- 172.21.15.87:0/305579635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 msgr2=0x55f2a41c39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c39f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00a040 tx=0x7f22ec054d30 comp rx=0 tx=0).stop 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- 172.21.15.87:0/305579635 shutdown_connections 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c39f0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 --2- 172.21.15.87:0/305579635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a420dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- 172.21.15.87:0/305579635 >> 172.21.15.87:0/305579635 conn(0x55f2a41ca2a0 msgr2=0x55f2a41c22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- 172.21.15.87:0/305579635 shutdown_connections 2024-04-25T10:23:28.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- 172.21.15.87:0/305579635 wait complete. 2024-04-25T10:23:28.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.445+0000 7f230a89f000 1 -- start start 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 Processor -- start 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a41c3f30 0x55f2a4214cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a420d920 0x55f2a4215230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a3fcd990 con 0x55f2a420e240 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a400ee40 con 0x55f2a420d920 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2a3f9e900 con 0x55f2a41c3f30 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 10 client.0 _ll_register_callbacks cb 0x55f2a40c61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39570/0 (socket says 172.21.15.87:39570) 2024-04-25T10:23:28.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 -- 172.21.15.87:0/1680167687 learned_addr learned my addr 172.21.15.87:0/1680167687 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2304a00640 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a41c3f30 0x55f2a4214cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22ffe00640 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a420d920 0x55f2a4215230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 -- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a41c3f30 msgr2=0x55f2a4214cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a41c3f30 0x55f2a4214cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 -- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a420d920 msgr2=0x55f2a4215230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a420d920 0x55f2a4215230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 -- 172.21.15.87:0/1680167687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2a3fcb630 con 0x55f2a420e240 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2305400640 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c7aa0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f22f8019510 tx=0x7f22f801b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa39089403e40995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22e6000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22e6000640 1 -- 172.21.15.87:0/1680167687 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80b2040 con 0x55f2a420e240 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22e6000640 1 -- 172.21.15.87:0/1680167687 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f22f80a5d70 con 0x55f2a420e240 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22ec053030 con 0x55f2a420e240 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22e6000640 1 -- 172.21.15.87:0/1680167687 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22f80ca050 con 0x55f2a420e240 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 10 client.9762 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:28.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55f2a3f9b3a0 con 0x55f2a420e240 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22e6000640 1 -- 172.21.15.87:0/1680167687 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec053030 con 0x55f2a420e240 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22e6000640 1 -- 172.21.15.87:0/1680167687 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f22f80ad440 con 0x55f2a420e240 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 10 client.9762 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 10 client.9762 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 20 client.9762 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9160 0x55f2a421ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 4 client.9762 mds_command: new command op to 8758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 --> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2a3faea50 con 0x55f2a41c9160 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2304a00640 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9160 0x55f2a421ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f2304a00640 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9160 0x55f2a421ed50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb4a53e1aca3586 server_cookie=3461af1c20d8ab1f in_seq=0 out_seq=0 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.449+0000 7f22e6000640 10 client.9762 ms_handle_connect on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f22e6000640 1 -- 172.21.15.87:0/1680167687 <== mds.0 v2:172.21.15.87:6835/1025107360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2a3faea50 con 0x55f2a41c9160 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f22e6000640 10 client.9762 handle_command_reply: tid=0 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 client.9762 shutdown 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 msgr2=0x55f2a41c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c7aa0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f22f8019510 tx=0x7f22f801b140 comp rx=0 tx=0).stop 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 shutdown_connections 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9160 0x55f2a421ed50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a41c7aa0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a420d920 0x55f2a4215230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 --2- 172.21.15.87:0/1680167687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a41c3f30 0x55f2a4214cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f22e6000640 0 client.9762 ms_handle_reset on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 >> 172.21.15.87:0/1680167687 conn(0x55f2a41ca2a0 msgr2=0x55f2a41c29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 shutdown_connections 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 1 -- 172.21.15.87:0/1680167687 wait complete. 2024-04-25T10:23:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.453+0000 7f230a89f000 20 client.9762 trim_cache size 0 max 16384 2024-04-25T10:23:28.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 Processor -- start 2024-04-25T10:23:28.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- start start 2024-04-25T10:23:28.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a41c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a4217850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300202e40 con 0x55f2a420e240 2024-04-25T10:23:28.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300201600 con 0x55f2a41c3f30 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300201ab0 con 0x55f2a420d920 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a4217850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a4217850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39584/0 (socket says 172.21.15.87:39584) 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 -- 172.21.15.87:0/307603212 learned_addr learned my addr 172.21.15.87:0/307603212 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2304a00640 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f22ffe00640 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a41c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 -- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 msgr2=0x55f2a41c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a41c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 -- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 msgr2=0x55f2a41c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 -- 172.21.15.87:0/307603212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2a420e070 con 0x55f2a420e240 2024-04-25T10:23:28.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f2305400640 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a4217850 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f22f801afb0 tx=0x7f22f80b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2444f2a1c41a41e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f22f0c00640 1 -- 172.21.15.87:0/307603212 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80b2070 con 0x55f2a420e240 2024-04-25T10:23:28.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f22f0c00640 1 -- 172.21.15.87:0/307603212 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f80ad8a0 con 0x55f2a420e240 2024-04-25T10:23:28.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f22f0c00640 1 -- 172.21.15.87:0/307603212 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80a5c70 con 0x55f2a420e240 2024-04-25T10:23:28.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- 172.21.15.87:0/307603212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 msgr2=0x55f2a4217850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a4217850 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f22f801afb0 tx=0x7f22f80b9000 comp rx=0 tx=0).stop 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- 172.21.15.87:0/307603212 shutdown_connections 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420e240 0x55f2a4217850 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420d920 0x55f2a41c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 --2- 172.21.15.87:0/307603212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a41c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- 172.21.15.87:0/307603212 >> 172.21.15.87:0/307603212 conn(0x55f2a41ca2a0 msgr2=0x55f2a41fa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- 172.21.15.87:0/307603212 shutdown_connections 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- 172.21.15.87:0/307603212 wait complete. 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.469+0000 7f230a89f000 1 -- start start 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 Processor -- start 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a421a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420d920 0x55f2a426e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420e240 0x55f2a426e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a40173d0 con 0x55f2a420d920 2024-04-25T10:23:28.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a40ced50 con 0x55f2a41c3f30 2024-04-25T10:23:28.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2a3fad980 con 0x55f2a420e240 2024-04-25T10:23:28.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 10 client.0 _ll_register_callbacks cb 0x7f22ec054b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420d920 0x55f2a426e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420d920 0x55f2a426e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39594/0 (socket says 172.21.15.87:39594) 2024-04-25T10:23:28.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 -- 172.21.15.87:0/4173811829 learned_addr learned my addr 172.21.15.87:0/4173811829 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f2305400640 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420e240 0x55f2a426e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f2304a00640 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a421a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 -- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420e240 msgr2=0x55f2a426e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420e240 0x55f2a426e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 -- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 msgr2=0x55f2a421a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a421a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 -- 172.21.15.87:0/4173811829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2a4094600 con 0x55f2a420d920 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22ffe00640 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420d920 0x55f2a426e400 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00cdf0 tx=0x7f22ec000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62021d9969edd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 1 -- 172.21.15.87:0/4173811829 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0840e0 con 0x55f2a420d920 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 1 -- 172.21.15.87:0/4173811829 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f22ec0513a0 con 0x55f2a420d920 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22f80ad8a0 con 0x55f2a420d920 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 1 -- 172.21.15.87:0/4173811829 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22ec054180 con 0x55f2a420d920 2024-04-25T10:23:28.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 10 client.9768 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55f2a3fa79b0 con 0x55f2a420d920 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 1 -- 172.21.15.87:0/4173811829 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f80ad8a0 con 0x55f2a420d920 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 1 -- 172.21.15.87:0/4173811829 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f22ec054320 con 0x55f2a420d920 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 10 client.9768 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 10 client.9768 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 20 client.9768 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9e80 0x55f2a426ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 4 client.9768 mds_command: new command op to 8758 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 --> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f2a40202e0 con 0x55f2a41c9e80 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f2304a00640 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9e80 0x55f2a426ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 10 client.9768 ms_handle_connect on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:28.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f2304a00640 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9e80 0x55f2a426ee80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa787b96ee498e5 server_cookie=fdc45947d4757d40 in_seq=0 out_seq=0 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 1 -- 172.21.15.87:0/4173811829 <== mds.0 v2:172.21.15.87:6835/1025107360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55f2a40202e0 con 0x55f2a41c9e80 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 10 client.9768 handle_command_reply: tid=0 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 client.9768 shutdown 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420d920 msgr2=0x55f2a426e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420d920 0x55f2a426e400 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00cdf0 tx=0x7f22ec000bd0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 shutdown_connections 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:6835/1025107360,v1:172.21.15.87:6837/1025107360] conn(0x55f2a41c9e80 0x55f2a426ee80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2a420e240 0x55f2a426e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2a420d920 0x55f2a426e400 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 --2- 172.21.15.87:0/4173811829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2a41c3f30 0x55f2a421a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f22e6000640 0 client.9768 ms_handle_reset on v2:172.21.15.87:6835/1025107360 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.473+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 >> 172.21.15.87:0/4173811829 conn(0x55f2a41ca2a0 msgr2=0x55f2a41c6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 shutdown_connections 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f230a89f000 1 -- 172.21.15.87:0/4173811829 wait complete. 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f230a89f000 20 client.9768 trim_cache size 0 max 16384 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 8758, 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 31.061199206000001, 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 103, 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 140, 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 140, 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 36.981876720000002 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22e0066f60 msgr2=0x7f22e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22e0066f60 0x7f22e0069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f22f40011f0 tx=0x7f22f4009040 comp rx=0 tx=0).stop 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23000d0040 msgr2=0x7f23001563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23000d0040 0x7f23001563b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00c430 tx=0x7f22ec00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 shutdown_connections 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f22e0066f60 0x7f22e0069420 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f230015bc40 0x7f2300152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23000d0040 0x7f23001563b0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 --2- 172.21.15.87:0/4210580480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23000cda10 0x7f230014b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 >> 172.21.15.87:0/4210580480 conn(0x7f2300066b70 msgr2=0x7f2300158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 shutdown_connections 2024-04-25T10:23:28.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.477+0000 7f2307400640 1 -- 172.21.15.87:0/4210580480 wait complete. 2024-04-25T10:23:28.494 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub start abs_test_path recursive 2024-04-25T10:23:28.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd29cc00640 1 Processor -- start 2024-04-25T10:23:28.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd29cc00640 1 -- start start 2024-04-25T10:23:28.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd29cc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd29815c0f0 0x7fd2980b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd29cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29806c0e0 con 0x7fd2980cf9e0 2024-04-25T10:23:28.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd29cc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29815c6a0 con 0x7fd29815bd10 2024-04-25T10:23:28.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd29cc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29815c980 con 0x7fd29815c0f0 2024-04-25T10:23:28.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd296800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd2980cf9e0 0x7fd29806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53474/0 (socket says 172.21.15.87:53474) 2024-04-25T10:23:28.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd296800640 1 -- 172.21.15.87:0/4221029753 learned_addr learned my addr 172.21.15.87:0/4221029753 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:28.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295400640 1 --2- 172.21.15.87:0/4221029753 >> v2:172.21.15.169:3300/0 conn(0x7fd29815c0f0 0x7fd2980b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003436046 0 0) 0x7fd29806c0e0 con 0x7fd2980cf9e0 2024-04-25T10:23:28.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd27c003530 con 0x7fd2980cf9e0 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970415379 0 0) 0x7fd27c003530 con 0x7fd2980cf9e0 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 >> v2:172.21.15.169:3300/0 conn(0x7fd29815c0f0 msgr2=0x7fd2980b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 --2- 172.21.15.87:0/4221029753 >> v2:172.21.15.169:3300/0 conn(0x7fd29815c0f0 0x7fd2980b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 >> v1:172.21.15.169:6789/0 conn(0x7fd29815bd10 legacy=0x7fd2980b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29815cc60 con 0x7fd2980cf9e0 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd290001a00 con 0x7fd2980cf9e0 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 >> v1:172.21.15.87:6789/0 conn(0x7fd2980cf9e0 legacy=0x7fd29806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c0039c0 0x7fd27c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27c004810 0x7fd27c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0051d0 0x7fd27c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29815c980 con 0x7fd27c0039c0 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00c070 con 0x7fd27c004810 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00c270 con 0x7fd27c0051d0 2024-04-25T10:23:28.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295e00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c0039c0 0x7fd27c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295400640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27c004810 0x7fd27c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd296800640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0051d0 0x7fd27c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295e00640 1 -- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0051d0 msgr2=0x7fd27c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295e00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0051d0 0x7fd27c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295e00640 1 -- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27c004810 msgr2=0x7fd27c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295e00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27c004810 0x7fd27c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd295e00640 1 -- 172.21.15.87:0/4221029753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c003530 con 0x7fd27c0039c0 2024-04-25T10:23:28.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.637+0000 7fd296800640 1 -- 172.21.15.87:0/4221029753 reap_dead start 2024-04-25T10:23:28.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd295e00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c0039c0 0x7fd27c003de0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd288001780 tx=0x7fd28801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53d0e7d2213f4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880013a0 con 0x7fd27c0039c0 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2880052f0 con 0x7fd27c0039c0 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd294a00640 1 -- 172.21.15.87:0/4221029753 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288003840 con 0x7fd27c0039c0 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd29cc00640 1 -- 172.21.15.87:0/4221029753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c0039c0 msgr2=0x7fd27c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd29cc00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c0039c0 0x7fd27c003de0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd288001780 tx=0x7fd28801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd29cc00640 1 -- 172.21.15.87:0/4221029753 shutdown_connections 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd29cc00640 1 --2- 172.21.15.87:0/4221029753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c0039c0 0x7fd27c003de0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd29cc00640 1 -- 172.21.15.87:0/4221029753 >> 172.21.15.87:0/4221029753 conn(0x7fd298066b70 msgr2=0x7fd298066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd29cc00640 1 -- 172.21.15.87:0/4221029753 shutdown_connections 2024-04-25T10:23:28.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.641+0000 7fd29cc00640 1 -- 172.21.15.87:0/4221029753 wait complete. 2024-04-25T10:23:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 Processor -- start 2024-04-25T10:23:28.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 -- start start 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0039c0 0x7fd29814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd298153330 0x7fd2980713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2980718f0 0x7fd298153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980b2b70 con 0x7fd2980718f0 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980b29f0 con 0x7fd298153330 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980b2cf0 con 0x7fd27c0039c0 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2980718f0 0x7fd298153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2980718f0 0x7fd298153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39604/0 (socket says 172.21.15.87:39604) 2024-04-25T10:23:28.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 -- 172.21.15.87:0/2979049463 learned_addr learned my addr 172.21.15.87:0/2979049463 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd295400640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd298153330 0x7fd2980713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd295e00640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0039c0 0x7fd29814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 -- 172.21.15.87:0/2979049463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0039c0 msgr2=0x7fd29814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27c0039c0 0x7fd29814a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 -- 172.21.15.87:0/2979049463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd298153330 msgr2=0x7fd2980713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd298153330 0x7fd2980713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 -- 172.21.15.87:0/2979049463 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29814a7b0 con 0x7fd2980718f0 2024-04-25T10:23:28.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd296800640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2980718f0 0x7fd298153f40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fd290000f90 tx=0x7fd290007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd28e000640 1 -- 172.21.15.87:0/2979049463 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000ea40 con 0x7fd2980718f0 2024-04-25T10:23:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd28e000640 1 -- 172.21.15.87:0/2979049463 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290018070 con 0x7fd2980718f0 2024-04-25T10:23:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd28e000640 1 -- 172.21.15.87:0/2979049463 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000ee60 con 0x7fd2980718f0 2024-04-25T10:23:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 -- 172.21.15.87:0/2979049463 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd298152290 con 0x7fd2980718f0 2024-04-25T10:23:28.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.645+0000 7fd29cc00640 1 -- 172.21.15.87:0/2979049463 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd298152850 con 0x7fd2980718f0 2024-04-25T10:23:28.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.649+0000 7fd28e000640 1 -- 172.21.15.87:0/2979049463 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd29001d460 con 0x7fd2980718f0 2024-04-25T10:23:28.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.649+0000 7fd28e000640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd278066f90 0x7fd278069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.649+0000 7fd28e000640 1 -- 172.21.15.87:0/2979049463 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd290085230 con 0x7fd2980718f0 2024-04-25T10:23:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.649+0000 7fd295e00640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd278066f90 0x7fd278069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.649+0000 7fd2a004c000 1 Processor -- start 2024-04-25T10:23:28.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.649+0000 7fd2a004c000 1 -- start start 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.649+0000 7fd295e00640 1 --2- 172.21.15.87:0/2979049463 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd278066f90 0x7fd278069450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd288001750 tx=0x7fd2880011a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a7d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a7d9d80 0x56115a78ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a78d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115a5ddc30 con 0x56115a78d210 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115a5668c0 con 0x56115a7d9460 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56115a5d9cd0 con 0x56115a7d9d80 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a78d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a78d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39614/0 (socket says 172.21.15.87:39614) 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 -- 172.21.15.87:0/3023963137 learned_addr learned my addr 172.21.15.87:0/3023963137 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd295e00640 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a7d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd295400640 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a7d9d80 0x56115a78ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 -- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a7d9d80 msgr2=0x56115a78ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a7d9d80 0x56115a78ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 -- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 msgr2=0x56115a7d9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a7d9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 -- 172.21.15.87:0/3023963137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115a666130 con 0x56115a78d210 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd295400640 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a7d9d80 0x56115a78ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd296800640 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a78d5f0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fd290001290 tx=0x7fd290055350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fe57150e1699e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd282a00640 1 -- 172.21.15.87:0/3023963137 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290059b40 con 0x56115a78d210 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd282a00640 1 -- 172.21.15.87:0/3023963137 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29005e190 con 0x56115a78d210 2024-04-25T10:23:28.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd282a00640 1 -- 172.21.15.87:0/3023963137 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290052060 con 0x56115a78d210 2024-04-25T10:23:28.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- 172.21.15.87:0/3023963137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 msgr2=0x56115a78d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a78d5f0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fd290001290 tx=0x7fd290055350 comp rx=0 tx=0).stop 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- 172.21.15.87:0/3023963137 shutdown_connections 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a78d5f0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a7d9d80 0x56115a78ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 --2- 172.21.15.87:0/3023963137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a7d9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- 172.21.15.87:0/3023963137 >> 172.21.15.87:0/3023963137 conn(0x56115a677c80 msgr2=0x56115a7bc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- 172.21.15.87:0/3023963137 shutdown_connections 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- 172.21.15.87:0/3023963137 wait complete. 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 -- start start 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.653+0000 7fd2a004c000 1 Processor -- start 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a79a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a7d9d80 0x56115a79a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115a586ad0 con 0x56115a7d9d80 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115a598fa0 con 0x56115a7d9460 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56115a5dcc00 con 0x56115a78d210 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 10 client.0 _ll_register_callbacks cb 0x56115a77ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a7d9d80 0x56115a79a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a7d9d80 0x56115a79a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39626/0 (socket says 172.21.15.87:39626) 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 -- 172.21.15.87:0/4091292372 learned_addr learned my addr 172.21.15.87:0/4091292372 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd295e00640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd295400640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a79a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 -- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 msgr2=0x56115a7d8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7d8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 -- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 msgr2=0x56115a79a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a79a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 -- 172.21.15.87:0/4091292372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56115a623f90 con 0x56115a7d9d80 2024-04-25T10:23:28.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd295400640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a79a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd296800640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a7d9d80 0x56115a79a540 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd2900554c0 tx=0x7fd2900021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e1263e81aac366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 1 -- 172.21.15.87:0/4091292372 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290072060 con 0x56115a7d9d80 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 1 -- 172.21.15.87:0/4091292372 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd290083170 con 0x56115a7d9d80 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd29005e190 con 0x56115a7d9d80 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 1 -- 172.21.15.87:0/4091292372 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd290078050 con 0x56115a7d9d80 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 10 client.9780 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x56115a5a2630 con 0x56115a7d9d80 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 1 -- 172.21.15.87:0/4091292372 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29005e190 con 0x56115a7d9d80 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 1 -- 172.21.15.87:0/4091292372 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fd29007b020 con 0x56115a7d9d80 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 10 client.9780 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:28.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 10 client.9780 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 20 client.9780 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a79b2b0 0x56115a7eb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 4 client.9780 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56115a579f80 con 0x56115a79b2b0 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd295e00640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a79b2b0 0x56115a7eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd295e00640 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a79b2b0 0x56115a7eb720 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=2cd8ebf377728030 server_cookie=afd9625f12325e9 in_seq=0 out_seq=0 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 10 client.9780 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 1 -- 172.21.15.87:0/4091292372 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56115a579f80 con 0x56115a79b2b0 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 10 client.9780 handle_command_reply: tid=0 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 client.9780 shutdown 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a7d9d80 msgr2=0x56115a79a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a7d9d80 0x56115a79a540 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd2900554c0 tx=0x7fd2900021e0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 shutdown_connections 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a79b2b0 0x56115a7eb720 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a7d9d80 0x56115a79a540 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a7d9460 0x56115a79a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 --2- 172.21.15.87:0/4091292372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7d8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd277400640 0 client.9780 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 >> 172.21.15.87:0/4091292372 conn(0x56115a677c80 msgr2=0x56115a78c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 shutdown_connections 2024-04-25T10:23:28.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 1 -- 172.21.15.87:0/4091292372 wait complete. 2024-04-25T10:23:28.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.657+0000 7fd2a004c000 20 client.9780 trim_cache size 0 max 16384 2024-04-25T10:23:28.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 Processor -- start 2024-04-25T10:23:28.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 -- start start 2024-04-25T10:23:28.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a791980 0x56115a7d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a677c80 0x56115a7d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a7bb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2982030e0 con 0x56115a78d210 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298203740 con 0x56115a791980 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980aff00 con 0x56115a677c80 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.673+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a7bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a7bb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39636/0 (socket says 172.21.15.87:39636) 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 -- 172.21.15.87:0/1395659257 learned_addr learned my addr 172.21.15.87:0/1395659257 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd295e00640 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a791980 0x56115a7d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd295400640 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a677c80 0x56115a7d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 -- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a677c80 msgr2=0x56115a7d8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a677c80 0x56115a7d8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 -- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a791980 msgr2=0x56115a7d7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a791980 0x56115a7d7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 -- 172.21.15.87:0/1395659257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115a566b10 con 0x56115a78d210 2024-04-25T10:23:28.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a7bb790 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd29000a040 tx=0x7fd290080070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3989261fd0282391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd282000640 1 -- 172.21.15.87:0/1395659257 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2900833c0 con 0x56115a78d210 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd282000640 1 -- 172.21.15.87:0/1395659257 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290082030 con 0x56115a78d210 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd282000640 1 -- 172.21.15.87:0/1395659257 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2900560f0 con 0x56115a78d210 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- 172.21.15.87:0/1395659257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 msgr2=0x56115a7bb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a7bb790 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd29000a040 tx=0x7fd290080070 comp rx=0 tx=0).stop 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- 172.21.15.87:0/1395659257 shutdown_connections 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a78d210 0x56115a7bb790 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a677c80 0x56115a7d8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 --2- 172.21.15.87:0/1395659257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a791980 0x56115a7d7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- 172.21.15.87:0/1395659257 >> 172.21.15.87:0/1395659257 conn(0x56115a794950 msgr2=0x56115a7d7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- 172.21.15.87:0/1395659257 shutdown_connections 2024-04-25T10:23:28.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- 172.21.15.87:0/1395659257 wait complete. 2024-04-25T10:23:28.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- start start 2024-04-25T10:23:28.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 Processor -- start 2024-04-25T10:23:28.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a677c80 0x56115a7ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a791980 0x56115a7fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115a78cd80 con 0x56115a791980 2024-04-25T10:23:28.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298156af0 con 0x56115a677c80 2024-04-25T10:23:28.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56115a586ad0 con 0x56115a78d210 2024-04-25T10:23:28.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd2a004c000 10 client.0 _ll_register_callbacks cb 0x7fd290001e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a791980 0x56115a7fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a791980 0x56115a7fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39640/0 (socket says 172.21.15.87:39640) 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 -- 172.21.15.87:0/2728868532 learned_addr learned my addr 172.21.15.87:0/2728868532 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd295e00640 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a677c80 0x56115a7ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd295400640 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.677+0000 7fd296800640 1 -- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 msgr2=0x56115a7ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd296800640 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd296800640 1 -- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a677c80 msgr2=0x56115a7ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd296800640 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a677c80 0x56115a7ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd296800640 1 -- 172.21.15.87:0/2728868532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56115a636cf0 con 0x56115a791980 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd296800640 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a791980 0x56115a7fff00 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fd290055980 tx=0x7fd290085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ff1734529ea0e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 1 -- 172.21.15.87:0/2728868532 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2900833c0 con 0x56115a791980 2024-04-25T10:23:28.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 1 -- 172.21.15.87:0/2728868532 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd290055050 con 0x56115a791980 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd290082030 con 0x56115a791980 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 1 -- 172.21.15.87:0/2728868532 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2900560f0 con 0x56115a791980 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 10 client.9786 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x56115a3c3a30 con 0x56115a791980 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 1 -- 172.21.15.87:0/2728868532 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290082030 con 0x56115a791980 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 1 -- 172.21.15.87:0/2728868532 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fd29007b040 con 0x56115a791980 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 10 client.9786 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 10 client.9786 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 20 client.9786 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a7d9460 0x56115a842df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 4 client.9786 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}) v1 -- 0x56115a5eb170 con 0x56115a7d9460 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd295e00640 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a7d9460 0x56115a842df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 10 client.9786 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd295e00640 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a7d9460 0x56115a842df0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=11f366733dc90bf4 server_cookie=585541a3a93d469d in_seq=0 out_seq=0 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 1 -- 172.21.15.87:0/2728868532 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x56115a5eb170 con 0x56115a7d9460 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 10 client.9786 handle_command_reply: tid=0 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 client.9786 shutdown 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a791980 msgr2=0x56115a7fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a791980 0x56115a7fff00 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fd290055980 tx=0x7fd290085040 comp rx=0 tx=0).stop 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 shutdown_connections 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56115a7d9460 0x56115a842df0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56115a791980 0x56115a7fff00 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56115a78d210 0x56115a7ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 --2- 172.21.15.87:0/2728868532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56115a677c80 0x56115a7ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd277400640 0 client.9786 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 >> 172.21.15.87:0/2728868532 conn(0x56115a794950 msgr2=0x56115a7d7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 shutdown_connections 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 1 -- 172.21.15.87:0/2728868532 wait complete. 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.681+0000 7fd2a004c000 20 client.9786 trim_cache size 0 max 16384 2024-04-25T10:23:28.684 INFO:teuthology.orchestra.run.smithi087.stderr:Error EINVAL: MDS is not active 2024-04-25T10:23:28.687 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:23:28.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub pause 2024-04-25T10:23:28.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.825+0000 7f54aa800640 1 Processor -- start 2024-04-25T10:23:28.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.825+0000 7f54aa800640 1 -- start start 2024-04-25T10:23:28.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54aa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54aa800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54a40b31c0 0x7f54a40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54aa800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a415b080 con 0x7f54a4070350 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54aa800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a415adb0 con 0x7f54a40b31c0 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54aa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a415eb60 con 0x7f54a40aef80 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54a40b31c0 0x7f54a40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f549aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f54a4070350 0x7f54a415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53486/0 (socket says 172.21.15.87:53486) 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54a40b31c0 0x7f54a40b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39642/0 (socket says 172.21.15.87:39642) 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f549aa00640 1 -- 172.21.15.87:0/3942308558 learned_addr learned my addr 172.21.15.87:0/3942308558 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3400640 1 --2- 172.21.15.87:0/3942308558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3e00640 1 -- 172.21.15.87:0/3942308558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 msgr2=0x7f54a415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3942308558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3e00640 1 -- 172.21.15.87:0/3942308558 >> v1:172.21.15.87:6789/0 conn(0x7f54a4070350 legacy=0x7f54a415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a2a00640 1 -- 172.21.15.87:0/3942308558 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423958263 0 0) 0x7f54a415b080 con 0x7f54a4070350 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3e00640 1 -- 172.21.15.87:0/3942308558 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a415ee40 con 0x7f54a40b31c0 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3400640 1 --2- 172.21.15.87:0/3942308558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3942308558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a40b31c0 0x7f54a40b3610 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f549c0097c0 tx=0x7f549c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbd30523e6db2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.829+0000 7f54a2a00640 1 -- 172.21.15.87:0/3942308558 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0b1070 con 0x7f54a40b31c0 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a2a00640 1 -- 172.21.15.87:0/3942308558 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c0ac410 con 0x7f54a40b31c0 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a2a00640 1 -- 172.21.15.87:0/3942308558 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0ac7e0 con 0x7f54a40b31c0 2024-04-25T10:23:28.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- 172.21.15.87:0/3942308558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a40b31c0 msgr2=0x7f54a40b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 --2- 172.21.15.87:0/3942308558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a40b31c0 0x7f54a40b3610 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f549c0097c0 tx=0x7f549c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- 172.21.15.87:0/3942308558 shutdown_connections 2024-04-25T10:23:28.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 --2- 172.21.15.87:0/3942308558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a40b31c0 0x7f54a40b3610 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 --2- 172.21.15.87:0/3942308558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- 172.21.15.87:0/3942308558 >> 172.21.15.87:0/3942308558 conn(0x7f54a40669a0 msgr2=0x7f54a4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- 172.21.15.87:0/3942308558 shutdown_connections 2024-04-25T10:23:28.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- 172.21.15.87:0/3942308558 wait complete. 2024-04-25T10:23:28.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 Processor -- start 2024-04-25T10:23:28.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- start start 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a4070350 0x7f54a4154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a40ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a40b31c0 0x7f54a4153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41679a0 con 0x7f54a4070350 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4167820 con 0x7f54a40b31c0 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54aa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4167b20 con 0x7f54a40aef80 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a4070350 0x7f54a4154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a4070350 0x7f54a4154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39646/0 (socket says 172.21.15.87:39646) 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 -- 172.21.15.87:0/2394410477 learned_addr learned my addr 172.21.15.87:0/2394410477 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f549aa00640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a40ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3e00640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a40b31c0 0x7f54a4153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 -- 172.21.15.87:0/2394410477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 msgr2=0x7f54a40ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a40aef80 0x7f54a40ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 -- 172.21.15.87:0/2394410477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a40b31c0 msgr2=0x7f54a4153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a40b31c0 0x7f54a4153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.833+0000 7f54a3400640 1 -- 172.21.15.87:0/2394410477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40cdf60 con 0x7f54a4070350 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f54a3400640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a4070350 0x7f54a4154b00 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f549400cf60 tx=0x7f549400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f549be00640 1 -- 172.21.15.87:0/2394410477 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549401a070 con 0x7f54a4070350 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f549be00640 1 -- 172.21.15.87:0/2394410477 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549400d840 con 0x7f54a4070350 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f549be00640 1 -- 172.21.15.87:0/2394410477 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400dc10 con 0x7f54a4070350 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f54aa800640 1 -- 172.21.15.87:0/2394410477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a40cdc80 con 0x7f54a4070350 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f54aa800640 1 -- 172.21.15.87:0/2394410477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a40aa830 con 0x7f54a4070350 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f549be00640 1 -- 172.21.15.87:0/2394410477 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f549401d070 con 0x7f54a4070350 2024-04-25T10:23:28.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f549be00640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5480066f90 0x7f5480069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f549be00640 1 -- 172.21.15.87:0/2394410477 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5494085800 con 0x7f54a4070350 2024-04-25T10:23:28.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f549aa00640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5480066f90 0x7f5480069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.837+0000 7f54adcd0000 1 Processor -- start 2024-04-25T10:23:28.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- start start 2024-04-25T10:23:28.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f549aa00640 1 --2- 172.21.15.87:0/2394410477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5480066f90 0x7f5480069450 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f54a40cbd30 tx=0x7f548c007b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x56535050bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56535050c320 0x5653504c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x5653504c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653503a4ad0 con 0x56535050ba00 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56535030bf80 con 0x56535050c320 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x565350293ee0 con 0x5653504c1df0 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x56535050bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x56535050bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39656/0 (socket says 172.21.15.87:39656) 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 -- 172.21.15.87:0/2642433135 learned_addr learned my addr 172.21.15.87:0/2642433135 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3e00640 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x5653504c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 -- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 msgr2=0x5653504c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f549aa00640 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56535050c320 0x5653504c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x5653504c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 -- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56535050c320 msgr2=0x5653504c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56535050c320 0x5653504c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 -- 172.21.15.87:0/2642433135 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653502c4930 con 0x56535050ba00 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3e00640 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x5653504c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f549aa00640 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56535050c320 0x5653504c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54a3400640 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x56535050bde0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f54940525f0 tx=0x7f5494055c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dba68761b8947d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f5493e00640 1 -- 172.21.15.87:0/2642433135 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400ddb0 con 0x56535050ba00 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f5493e00640 1 -- 172.21.15.87:0/2642433135 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494054490 con 0x56535050ba00 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f5493e00640 1 -- 172.21.15.87:0/2642433135 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549405e360 con 0x56535050ba00 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- 172.21.15.87:0/2642433135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 msgr2=0x56535050bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x56535050bde0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f54940525f0 tx=0x7f5494055c20 comp rx=0 tx=0).stop 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- 172.21.15.87:0/2642433135 shutdown_connections 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x5653504c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56535050c320 0x5653504c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2642433135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x56535050bde0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- 172.21.15.87:0/2642433135 >> 172.21.15.87:0/2642433135 conn(0x5653504c3f60 msgr2=0x5653504bbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- 172.21.15.87:0/2642433135 shutdown_connections 2024-04-25T10:23:28.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- 172.21.15.87:0/2642433135 wait complete. 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 -- start start 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.841+0000 7f54adcd0000 1 Processor -- start 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350512b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350513090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050c320 0x5653504f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653502c6390 con 0x56535050c320 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x565350308020 con 0x5653504c1df0 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653502d9620 con 0x56535050ba00 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 10 client.0 _ll_register_callbacks cb 0x5653503bf6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050c320 0x5653504f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050c320 0x5653504f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39672/0 (socket says 172.21.15.87:39672) 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3e00640 1 -- 172.21.15.87:0/35276507 learned_addr learned my addr 172.21.15.87:0/35276507 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3400640 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350512b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f549aa00640 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350513090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3400640 1 -- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 msgr2=0x565350513090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3400640 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350513090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3400640 1 -- 172.21.15.87:0/35276507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050c320 msgr2=0x5653504f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3400640 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050c320 0x5653504f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3400640 1 -- 172.21.15.87:0/35276507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565350294130 con 0x5653504c1df0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54a3400640 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350512b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f549400ac60 tx=0x7f5494000f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d105934645fd0cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 1 -- 172.21.15.87:0/35276507 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494073070 con 0x5653504c1df0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 1 -- 172.21.15.87:0/35276507 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f549405e560 con 0x5653504c1df0 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5494054490 con 0x5653504c1df0 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 1 -- 172.21.15.87:0/35276507 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5494052430 con 0x5653504c1df0 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 10 client.8932 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x5653502c6aa0 con 0x5653504c1df0 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 1 -- 172.21.15.87:0/35276507 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494054490 con 0x5653504c1df0 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 1 -- 172.21.15.87:0/35276507 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f54940555e0 con 0x5653504c1df0 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 10 client.8932 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 10 client.8932 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 20 client.8932 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x5653504f9c60 0x5653504fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 4 client.8932 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653502a7a30 con 0x5653504f9c60 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f549aa00640 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x5653504f9c60 0x5653504fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f549aa00640 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x5653504f9c60 0x5653504fa040 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=3e8e2c68d5d354d1 server_cookie=18c8e01e74c0bf64 in_seq=0 out_seq=0 2024-04-25T10:23:28.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 10 client.8932 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 1 -- 172.21.15.87:0/35276507 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653502a7a30 con 0x5653504f9c60 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f5490c00640 10 client.8932 handle_command_reply: tid=0 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.845+0000 7f54adcd0000 1 client.8932 shutdown 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 msgr2=0x565350512b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350512b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f549400ac60 tx=0x7f5494000f50 comp rx=0 tx=0).stop 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 shutdown_connections 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x5653504f9c60 0x5653504fa040 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050c320 0x5653504f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350513090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 --2- 172.21.15.87:0/35276507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350512b50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 >> 172.21.15.87:0/35276507 conn(0x5653504c3f60 msgr2=0x5653504c3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f5490c00640 0 client.8932 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 shutdown_connections 2024-04-25T10:23:28.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 1 -- 172.21.15.87:0/35276507 wait complete. 2024-04-25T10:23:28.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.849+0000 7f54adcd0000 20 client.8932 trim_cache size 0 max 16384 2024-04-25T10:23:28.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 Processor -- start 2024-04-25T10:23:28.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 -- start start 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c3f60 0x565350567a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x565350567fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x565350568500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653503105b0 con 0x56535050ba00 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x565350308020 con 0x5653504c3f60 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653502c6390 con 0x5653504c1df0 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x565350568500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x565350568500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39686/0 (socket says 172.21.15.87:39686) 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.861+0000 7f54a3e00640 1 -- 172.21.15.87:0/1893215415 learned_addr learned my addr 172.21.15.87:0/1893215415 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3400640 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c3f60 0x565350567a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x565350567fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3e00640 1 -- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 msgr2=0x565350567fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3e00640 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x565350567fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3e00640 1 -- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c3f60 msgr2=0x565350567a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3e00640 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c3f60 0x565350567a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3e00640 1 -- 172.21.15.87:0/1893215415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56535038daf0 con 0x56535050ba00 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3e00640 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x565350568500 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f549c01c980 tx=0x7f549c0a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45ee69ff375f494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f5493400640 1 -- 172.21.15.87:0/1893215415 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0b1040 con 0x56535050ba00 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f5493400640 1 -- 172.21.15.87:0/1893215415 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c0ac7e0 con 0x56535050ba00 2024-04-25T10:23:28.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f5493400640 1 -- 172.21.15.87:0/1893215415 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0acb90 con 0x56535050ba00 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- 172.21.15.87:0/1893215415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 msgr2=0x565350568500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x565350568500 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f549c01c980 tx=0x7f549c0a6040 comp rx=0 tx=0).stop 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- 172.21.15.87:0/1893215415 shutdown_connections 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56535050ba00 0x565350568500 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653504c1df0 0x565350567fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 --2- 172.21.15.87:0/1893215415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c3f60 0x565350567a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- 172.21.15.87:0/1893215415 >> 172.21.15.87:0/1893215415 conn(0x565350512f30 msgr2=0x5653504c30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- 172.21.15.87:0/1893215415 shutdown_connections 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- 172.21.15.87:0/1893215415 wait complete. 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- start start 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 Processor -- start 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350568bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653504c3f60 0x565350510670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350510bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653503c8230 con 0x5653504c3f60 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653502d9620 con 0x5653504c1df0 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653502b2a50 con 0x56535050ba00 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54adcd0000 10 client.0 _ll_register_callbacks cb 0x565350312450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653504c3f60 0x565350510670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653504c3f60 0x565350510670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39688/0 (socket says 172.21.15.87:39688) 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 -- 172.21.15.87:0/2318477732 learned_addr learned my addr 172.21.15.87:0/2318477732 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3e00640 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350510bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f54a3400640 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350568bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 -- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 msgr2=0x565350510bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350510bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 -- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 msgr2=0x565350568bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.865+0000 7f549aa00640 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350568bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f549aa00640 1 -- 172.21.15.87:0/2318477732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565350363d50 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f549aa00640 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653504c3f60 0x565350510670 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f548c0011f0 tx=0x7f548c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1136a15fee9a55e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 1 -- 172.21.15.87:0/2318477732 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0258c0 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 1 -- 172.21.15.87:0/2318477732 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f548c02b920 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f549c0ac7e0 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 1 -- 172.21.15.87:0/2318477732 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f548c02bb20 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 10 client.9807 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x5653502b2e90 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 1 -- 172.21.15.87:0/2318477732 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c0ac7e0 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 1 -- 172.21.15.87:0/2318477732 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f548c038070 con 0x5653504c3f60 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 10 client.9807 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 10 client.9807 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:28.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 20 client.9807 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56535050c320 0x5653505110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 4 client.9807 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:23:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5653503194c0 con 0x56535050c320 2024-04-25T10:23:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54a3400640 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56535050c320 0x5653505110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 10 client.9807 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54a3400640 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56535050c320 0x5653505110f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=83250fa75fbb4d56 server_cookie=a1be07017e867e08 in_seq=0 out_seq=0 2024-04-25T10:23:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 1 -- 172.21.15.87:0/2318477732 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x5653503194c0 con 0x56535050c320 2024-04-25T10:23:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 10 client.9807 handle_command_reply: tid=0 2024-04-25T10:23:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 client.9807 shutdown 2024-04-25T10:23:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653504c3f60 msgr2=0x565350510670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653504c3f60 0x565350510670 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f548c0011f0 tx=0x7f548c011000 comp rx=0 tx=0).stop 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 shutdown_connections 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x56535050c320 0x5653505110f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56535050ba00 0x565350510bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653504c3f60 0x565350510670 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f5490c00640 0 client.9807 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 --2- 172.21.15.87:0/2318477732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653504c1df0 0x565350568bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 >> 172.21.15.87:0/2318477732 conn(0x565350512f30 msgr2=0x56535050a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 shutdown_connections 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 1 -- 172.21.15.87:0/2318477732 wait complete. 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:28.869+0000 7f54adcd0000 20 client.9807 trim_cache size 0 max 16384 2024-04-25T10:23:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:Error EINVAL: MDS is not active 2024-04-25T10:23:28.875 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:23:28.876 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub abort 2024-04-25T10:23:29.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.013+0000 7f7685e00640 1 Processor -- start 2024-04-25T10:23:29.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.013+0000 7f7685e00640 1 -- start start 2024-04-25T10:23:29.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f7685e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 0x7f76800ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f7685e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76800721f0 0x7f7680070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f7685e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7680159920 con 0x7f76800aebd0 2024-04-25T10:23:29.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f7685e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76800af030 con 0x7f76800721f0 2024-04-25T10:23:29.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f7685e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768015c2f0 con 0x7f7680159540 2024-04-25T10:23:29.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76800721f0 0x7f7680070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76800721f0 0x7f7680070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39692/0 (socket says 172.21.15.87:39692) 2024-04-25T10:23:29.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767f400640 1 -- 172.21.15.87:0/164518288 learned_addr learned my addr 172.21.15.87:0/164518288 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767e000640 1 --1- 172.21.15.87:0/164518288 >> v1:172.21.15.87:6789/0 conn(0x7f76800aebd0 0x7f7680071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53490/0 (socket says 172.21.15.87:53490) 2024-04-25T10:23:29.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767ea00640 1 --2- 172.21.15.87:0/164518288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 0x7f76800ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767f400640 1 -- 172.21.15.87:0/164518288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 msgr2=0x7f76800ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767f400640 1 --2- 172.21.15.87:0/164518288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 0x7f76800ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767f400640 1 -- 172.21.15.87:0/164518288 >> v1:172.21.15.87:6789/0 conn(0x7f76800aebd0 legacy=0x7f7680071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767f400640 1 -- 172.21.15.87:0/164518288 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768015c5d0 con 0x7f76800721f0 2024-04-25T10:23:29.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.017+0000 7f767d600640 1 -- 172.21.15.87:0/164518288 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307046546 0 0) 0x7f7680159920 con 0x7f76800aebd0 2024-04-25T10:23:29.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767f400640 1 --2- 172.21.15.87:0/164518288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 0x7f7680070350 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f7678009870 tx=0x7f767801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1d1034a9b4ef8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767d600640 1 -- 172.21.15.87:0/164518288 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780a95f0 con 0x7f76800721f0 2024-04-25T10:23:29.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767d600640 1 -- 172.21.15.87:0/164518288 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76780a9790 con 0x7f76800721f0 2024-04-25T10:23:29.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767d600640 1 -- 172.21.15.87:0/164518288 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780a9b60 con 0x7f76800721f0 2024-04-25T10:23:29.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- 172.21.15.87:0/164518288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 msgr2=0x7f7680070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 --2- 172.21.15.87:0/164518288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 0x7f7680070350 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f7678009870 tx=0x7f767801c530 comp rx=0 tx=0).stop 2024-04-25T10:23:29.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- 172.21.15.87:0/164518288 shutdown_connections 2024-04-25T10:23:29.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 --2- 172.21.15.87:0/164518288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 0x7f7680070350 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 --2- 172.21.15.87:0/164518288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 0x7f76800ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- 172.21.15.87:0/164518288 >> 172.21.15.87:0/164518288 conn(0x7f76800669a0 msgr2=0x7f7680066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- 172.21.15.87:0/164518288 shutdown_connections 2024-04-25T10:23:29.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- 172.21.15.87:0/164518288 wait complete. 2024-04-25T10:23:29.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 Processor -- start 2024-04-25T10:23:29.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- start start 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 0x7f7680140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76800aebd0 0x7f7680151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 0x7f76800af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76800710a0 con 0x7f76800721f0 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768006c0d0 con 0x7f76800aebd0 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f7685e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7680071220 con 0x7f7680159540 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 0x7f7680140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 0x7f7680140190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39696/0 (socket says 172.21.15.87:39696) 2024-04-25T10:23:29.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767ea00640 1 -- 172.21.15.87:0/2551953785 learned_addr learned my addr 172.21.15.87:0/2551953785 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767f400640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 0x7f76800af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.021+0000 7f767e000640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76800aebd0 0x7f7680151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f767ea00640 1 -- 172.21.15.87:0/2551953785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 msgr2=0x7f76800af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f767ea00640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7680159540 0x7f76800af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f767ea00640 1 -- 172.21.15.87:0/2551953785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76800aebd0 msgr2=0x7f7680151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f767ea00640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76800aebd0 0x7f7680151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f767ea00640 1 -- 172.21.15.87:0/2551953785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7680150e30 con 0x7f76800721f0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f767ea00640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76800721f0 0x7f7680140190 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7670009db0 tx=0x7f767000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7677400640 1 -- 172.21.15.87:0/2551953785 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670018020 con 0x7f76800721f0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7677400640 1 -- 172.21.15.87:0/2551953785 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670014480 con 0x7f76800721f0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7677400640 1 -- 172.21.15.87:0/2551953785 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670014850 con 0x7f76800721f0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7685e00640 1 -- 172.21.15.87:0/2551953785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7680151110 con 0x7f76800721f0 2024-04-25T10:23:29.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7685e00640 1 -- 172.21.15.87:0/2551953785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76801516d0 con 0x7f76800721f0 2024-04-25T10:23:29.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7677400640 1 -- 172.21.15.87:0/2551953785 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f767001b020 con 0x7f76800721f0 2024-04-25T10:23:29.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7677400640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7660066f60 0x7f7660069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.025+0000 7f7677400640 1 -- 172.21.15.87:0/2551953785 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7670085c30 con 0x7f76800721f0 2024-04-25T10:23:29.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767e000640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7660066f60 0x7f7660069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 Processor -- start 2024-04-25T10:23:29.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- start start 2024-04-25T10:23:29.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767e000640 1 --2- 172.21.15.87:0/2551953785 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7660066f60 0x7f7660069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7680067330 tx=0x7f766c007b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96e10d0 0x557ef96964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96969e0 0x557ef9696dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef957dad0 con 0x557ef96e07b0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef94e4f80 con 0x557ef96969e0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ef946cee0 con 0x557ef96e10d0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96e0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39706/0 (socket says 172.21.15.87:39706) 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 -- 172.21.15.87:0/3429949027 learned_addr learned my addr 172.21.15.87:0/3429949027 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 -- 172.21.15.87:0/3429949027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96e10d0 msgr2=0x557ef96964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 --2- 172.21.15.87:0/3429949027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96e10d0 0x557ef96964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 -- 172.21.15.87:0/3429949027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96969e0 msgr2=0x557ef9696dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 --2- 172.21.15.87:0/3429949027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96969e0 0x557ef9696dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 -- 172.21.15.87:0/3429949027 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef949d930 con 0x557ef96e07b0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f767ea00640 1 --2- 172.21.15.87:0/3429949027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96e0b90 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f767000d010 tx=0x7f7670056550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f71c796f006aa9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f766be00640 1 -- 172.21.15.87:0/3429949027 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670054790 con 0x557ef96e07b0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f766be00640 1 -- 172.21.15.87:0/3429949027 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670052b00 con 0x557ef96e07b0 2024-04-25T10:23:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f766be00640 1 -- 172.21.15.87:0/3429949027 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767005a370 con 0x557ef96e07b0 2024-04-25T10:23:29.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- 172.21.15.87:0/3429949027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 msgr2=0x557ef96e0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 --2- 172.21.15.87:0/3429949027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96e0b90 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f767000d010 tx=0x7f7670056550 comp rx=0 tx=0).stop 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- 172.21.15.87:0/3429949027 shutdown_connections 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 --2- 172.21.15.87:0/3429949027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96969e0 0x557ef9696dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 --2- 172.21.15.87:0/3429949027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96e10d0 0x557ef96964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 --2- 172.21.15.87:0/3429949027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96e0b90 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- 172.21.15.87:0/3429949027 >> 172.21.15.87:0/3429949027 conn(0x557ef969cf60 msgr2=0x557ef9694d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- 172.21.15.87:0/3429949027 shutdown_connections 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- 172.21.15.87:0/3429949027 wait complete. 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 -- start start 2024-04-25T10:23:29.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.029+0000 7f7689221000 1 Processor -- start 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef969be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e10d0 0x557ef96a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef949f390 con 0x557ef96e07b0 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef94e1020 con 0x557ef96e10d0 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ef94e1ff0 con 0x557ef96969e0 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 10 client.0 _ll_register_callbacks cb 0x557ef95986a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39708/0 (socket says 172.21.15.87:39708) 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 -- 172.21.15.87:0/2844977511 learned_addr learned my addr 172.21.15.87:0/2844977511 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767ea00640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef969be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767f400640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e10d0 0x557ef96a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 -- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 msgr2=0x557ef969be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef969be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 -- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e10d0 msgr2=0x557ef96a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e10d0 0x557ef96a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 -- 172.21.15.87:0/2844977511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ef946d130 con 0x557ef96e07b0 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767ea00640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef969be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767e000640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96a27c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7680143970 tx=0x7f766c001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20513aa4687d83da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7668c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7668c00640 1 -- 172.21.15.87:0/2844977511 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c015110 con 0x557ef96e07b0 2024-04-25T10:23:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7668c00640 1 -- 172.21.15.87:0/2844977511 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f766c001b90 con 0x557ef96e07b0 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7670052b00 con 0x557ef96e07b0 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7668c00640 1 -- 172.21.15.87:0/2844977511 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f766c013420 con 0x557ef96e07b0 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 10 client.9822 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x557ef949faa0 con 0x557ef96e07b0 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7668c00640 1 -- 172.21.15.87:0/2844977511 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670052b00 con 0x557ef96e07b0 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7668c00640 1 -- 172.21.15.87:0/2844977511 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f766c02a050 con 0x557ef96e07b0 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 10 client.9822 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 10 client.9822 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 20 client.9822 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e7e00 0x557ef96e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 4 client.9822 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ef9480a30 con 0x557ef96e7e00 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767ea00640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e7e00 0x557ef96e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f767ea00640 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e7e00 0x557ef96e81e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=7b0dba43184a287b server_cookie=95f0c3018b648847 in_seq=0 out_seq=0 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.033+0000 7f7668c00640 10 client.9822 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7668c00640 1 -- 172.21.15.87:0/2844977511 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ef9480a30 con 0x557ef96e7e00 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7668c00640 10 client.9822 handle_command_reply: tid=0 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 client.9822 shutdown 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 msgr2=0x557ef96a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96a27c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7680143970 tx=0x7f766c001090 comp rx=0 tx=0).stop 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 shutdown_connections 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e7e00 0x557ef96e81e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e10d0 0x557ef96a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef96a27c0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 --2- 172.21.15.87:0/2844977511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef969be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7668c00640 0 client.9822 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 >> 172.21.15.87:0/2844977511 conn(0x557ef969cf60 msgr2=0x557ef9695490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 shutdown_connections 2024-04-25T10:23:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 1 -- 172.21.15.87:0/2844977511 wait complete. 2024-04-25T10:23:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.037+0000 7f7689221000 20 client.9822 trim_cache size 0 max 16384 2024-04-25T10:23:29.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 Processor -- start 2024-04-25T10:23:29.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 -- start start 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef969cf60 0x557ef973c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef973cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e07b0 0x557ef973d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7680202570 con 0x557ef969cf60 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7680202b70 con 0x557ef96e07b0 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7680141330 con 0x557ef96969e0 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.049+0000 7f767ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef969cf60 0x557ef973c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef969cf60 0x557ef973c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39724/0 (socket says 172.21.15.87:39724) 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 -- 172.21.15.87:0/1941648038 learned_addr learned my addr 172.21.15.87:0/1941648038 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 -- 172.21.15.87:0/1941648038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 msgr2=0x557ef973cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 --2- 172.21.15.87:0/1941648038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef973cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 -- 172.21.15.87:0/1941648038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e07b0 msgr2=0x557ef973d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 --2- 172.21.15.87:0/1941648038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e07b0 0x557ef973d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 -- 172.21.15.87:0/1941648038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef9566af0 con 0x557ef969cf60 2024-04-25T10:23:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 --2- 172.21.15.87:0/1941648038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef969cf60 0x557ef973c9e0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7670009d80 tx=0x7f767005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73eeba64ac650db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f766b400640 1 -- 172.21.15.87:0/1941648038 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670081040 con 0x557ef969cf60 2024-04-25T10:23:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f766b400640 1 -- 172.21.15.87:0/1941648038 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670052b00 con 0x557ef969cf60 2024-04-25T10:23:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f766b400640 1 -- 172.21.15.87:0/1941648038 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767005a150 con 0x557ef969cf60 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- 172.21.15.87:0/1941648038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef969cf60 msgr2=0x557ef973c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 --2- 172.21.15.87:0/1941648038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef969cf60 0x557ef973c9e0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7670009d80 tx=0x7f767005a000 comp rx=0 tx=0).stop 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- 172.21.15.87:0/1941648038 shutdown_connections 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 --2- 172.21.15.87:0/1941648038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef96e07b0 0x557ef973d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 --2- 172.21.15.87:0/1941648038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef973cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 --2- 172.21.15.87:0/1941648038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef969cf60 0x557ef973c9e0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- 172.21.15.87:0/1941648038 >> 172.21.15.87:0/1941648038 conn(0x557ef969c760 msgr2=0x557ef96e0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- 172.21.15.87:0/1941648038 shutdown_connections 2024-04-25T10:23:29.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- 172.21.15.87:0/1941648038 wait complete. 2024-04-25T10:23:29.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- start start 2024-04-25T10:23:29.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 Processor -- start 2024-04-25T10:23:29.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef96e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef969cf60 0x557ef9727b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef9728050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef94e4290 con 0x557ef96e07b0 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7680154370 con 0x557ef969cf60 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ef948ba50 con 0x557ef96969e0 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f7689221000 10 client.0 _ll_register_callbacks cb 0x557ef94eb450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef9728050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef9728050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39738/0 (socket says 172.21.15.87:39738) 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767f400640 1 -- 172.21.15.87:0/3836030493 learned_addr learned my addr 172.21.15.87:0/3836030493 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767e000640 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef969cf60 0x557ef9727b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767ea00640 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef96e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.053+0000 7f767f400640 1 -- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 msgr2=0x557ef96e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f767f400640 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef96e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f767f400640 1 -- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef969cf60 msgr2=0x557ef9727b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f767f400640 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef969cf60 0x557ef9727b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f767f400640 1 -- 172.21.15.87:0/3836030493 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ef953cd50 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f767f400640 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef9728050 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f7678009870 tx=0x7f767801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8162b716ea65cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 1 -- 172.21.15.87:0/3836030493 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780c1060 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 1 -- 172.21.15.87:0/3836030493 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f76780b1af0 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7670052b00 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 1 -- 172.21.15.87:0/3836030493 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76780b1cf0 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 10 client.9828 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x557ef948be90 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 1 -- 172.21.15.87:0/3836030493 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670052b00 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 1 -- 172.21.15.87:0/3836030493 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f76780bc400 con 0x557ef96e07b0 2024-04-25T10:23:29.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 10 client.9828 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 10 client.9828 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 20 client.9828 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e10d0 0x557ef973df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 4 client.9828 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x557ef94f24c0 con 0x557ef96e10d0 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f767ea00640 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e10d0 0x557ef973df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f767ea00640 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e10d0 0x557ef973df10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=5909c4943c6c325b server_cookie=95fb662d7ff7370a in_seq=0 out_seq=0 2024-04-25T10:23:29.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 10 client.9828 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 1 -- 172.21.15.87:0/3836030493 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x557ef94f24c0 con 0x557ef96e10d0 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 10 client.9828 handle_command_reply: tid=0 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 client.9828 shutdown 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 msgr2=0x557ef9728050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef9728050 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f7678009870 tx=0x7f767801ad70 comp rx=0 tx=0).stop 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 shutdown_connections 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x557ef96e10d0 0x557ef973df10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557ef96e07b0 0x557ef9728050 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ef969cf60 0x557ef9727b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7668c00640 0 client.9828 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 --2- 172.21.15.87:0/3836030493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557ef96969e0 0x557ef96e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 >> 172.21.15.87:0/3836030493 conn(0x557ef969c760 msgr2=0x557ef96a1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 shutdown_connections 2024-04-25T10:23:29.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 1 -- 172.21.15.87:0/3836030493 wait complete. 2024-04-25T10:23:29.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.057+0000 7f7689221000 20 client.9828 trim_cache size 0 max 16384 2024-04-25T10:23:29.063 INFO:teuthology.orchestra.run.smithi087.stderr:Error EINVAL: MDS is not active 2024-04-25T10:23:29.063 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:23:29.064 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub resume 2024-04-25T10:23:29.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f266ac00640 1 Processor -- start 2024-04-25T10:23:29.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f266ac00640 1 -- start start 2024-04-25T10:23:29.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f266ac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2664159540 0x7f26640ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f266ac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2664159920 con 0x7f26640721f0 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f266ac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266415c2f0 con 0x7f26640aebd0 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f266ac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266415c5d0 con 0x7f2664159540 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2663e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2664159540 0x7f26640ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2663e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2664159540 0x7f26640ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39754/0 (socket says 172.21.15.87:39754) 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2668c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f26640721f0 0x7f2664070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53494/0 (socket says 172.21.15.87:53494) 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2663e00640 1 -- 172.21.15.87:0/504233838 learned_addr learned my addr 172.21.15.87:0/504233838 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2663e00640 1 -- 172.21.15.87:0/504233838 >> v1:172.21.15.169:6789/0 conn(0x7f26640aebd0 legacy=0x7f2664071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2663e00640 1 -- 172.21.15.87:0/504233838 >> v1:172.21.15.87:6789/0 conn(0x7f26640721f0 legacy=0x7f2664070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2663e00640 1 -- 172.21.15.87:0/504233838 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266415c8b0 con 0x7f2664159540 2024-04-25T10:23:29.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.205+0000 7f2662a00640 1 -- 172.21.15.87:0/504233838 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702468389 0 0) 0x7f2664159920 con 0x7f26640721f0 2024-04-25T10:23:29.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f2663e00640 1 --2- 172.21.15.87:0/504233838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2664159540 0x7f26640ae690 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2654009870 tx=0x7f265401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a229e0a84e522f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f2662a00640 1 -- 172.21.15.87:0/504233838 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540a7070 con 0x7f2664159540 2024-04-25T10:23:29.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f2662a00640 1 -- 172.21.15.87:0/504233838 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26540b0070 con 0x7f2664159540 2024-04-25T10:23:29.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f2662a00640 1 -- 172.21.15.87:0/504233838 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540ab890 con 0x7f2664159540 2024-04-25T10:23:29.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 -- 172.21.15.87:0/504233838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2664159540 msgr2=0x7f26640ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 --2- 172.21.15.87:0/504233838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2664159540 0x7f26640ae690 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2654009870 tx=0x7f265401c560 comp rx=0 tx=0).stop 2024-04-25T10:23:29.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 -- 172.21.15.87:0/504233838 shutdown_connections 2024-04-25T10:23:29.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 --2- 172.21.15.87:0/504233838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2664159540 0x7f26640ae690 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 -- 172.21.15.87:0/504233838 >> 172.21.15.87:0/504233838 conn(0x7f26640669a0 msgr2=0x7f2664066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 -- 172.21.15.87:0/504233838 shutdown_connections 2024-04-25T10:23:29.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 -- 172.21.15.87:0/504233838 wait complete. 2024-04-25T10:23:29.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 Processor -- start 2024-04-25T10:23:29.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.209+0000 7f266ac00640 1 -- start start 2024-04-25T10:23:29.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26640721f0 0x7f26640adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26640aebd0 0x7f2664153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2664159540 0x7f266413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2664169ac0 con 0x7f26640721f0 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2664169940 con 0x7f2664159540 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2664169c40 con 0x7f26640aebd0 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26640721f0 0x7f26640adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26640721f0 0x7f26640adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39768/0 (socket says 172.21.15.87:39768) 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 -- 172.21.15.87:0/4225530188 learned_addr learned my addr 172.21.15.87:0/4225530188 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 -- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26640aebd0 msgr2=0x7f2664153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2668c00640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2664159540 0x7f266413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663400640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26640aebd0 0x7f2664153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26640aebd0 0x7f2664153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 -- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2664159540 msgr2=0x7f266413ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2664159540 0x7f266413ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 -- 172.21.15.87:0/4225530188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640af420 con 0x7f26640721f0 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663400640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26640aebd0 0x7f2664153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2668c00640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2664159540 0x7f266413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:29.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f2663e00640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26640721f0 0x7f26640adaa0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f26540ad040 tx=0x7f265401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f265be00640 1 -- 172.21.15.87:0/4225530188 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540b4800 con 0x7f26640721f0 2024-04-25T10:23:29.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f265be00640 1 -- 172.21.15.87:0/4225530188 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26540b49c0 con 0x7f26640721f0 2024-04-25T10:23:29.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f265be00640 1 -- 172.21.15.87:0/4225530188 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540b4db0 con 0x7f26640721f0 2024-04-25T10:23:29.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 -- 172.21.15.87:0/4225530188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640af700 con 0x7f26640721f0 2024-04-25T10:23:29.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.213+0000 7f266ac00640 1 -- 172.21.15.87:0/4225530188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640afcc0 con 0x7f26640721f0 2024-04-25T10:23:29.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f265be00640 1 -- 172.21.15.87:0/4225530188 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2664071270 con 0x7f26640721f0 2024-04-25T10:23:29.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f265be00640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2644066f60 0x7f2644069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f265be00640 1 -- 172.21.15.87:0/4225530188 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f265411f170 con 0x7f26640721f0 2024-04-25T10:23:29.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663400640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2644066f60 0x7f2644069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 Processor -- start 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 -- start start 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663400640 1 --2- 172.21.15.87:0/4225530188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2644066f60 0x7f2644069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26640b3fc0 tx=0x7f2650009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507fac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x558507fad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558507e8f1f0 con 0x558507ff1a40 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558507df5e90 con 0x558507facd90 2024-04-25T10:23:29.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558507d7df50 con 0x558507ff2360 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39784/0 (socket says 172.21.15.87:39784) 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 -- 172.21.15.87:0/2523505484 learned_addr learned my addr 172.21.15.87:0/2523505484 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663400640 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507fac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2668c00640 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x558507fad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 -- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 msgr2=0x558507fac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507fac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 -- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 msgr2=0x558507fad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x558507fad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 -- 172.21.15.87:0/2523505484 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558507dcd8b0 con 0x558507ff1a40 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.217+0000 7f2663e00640 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff1e20 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2654019170 tx=0x7f26540a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30400e02b9e9bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2658c00640 1 -- 172.21.15.87:0/2523505484 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540c4070 con 0x558507ff1a40 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2658c00640 1 -- 172.21.15.87:0/2523505484 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26540b0050 con 0x558507ff1a40 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2658c00640 1 -- 172.21.15.87:0/2523505484 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540f8200 con 0x558507ff1a40 2024-04-25T10:23:29.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- 172.21.15.87:0/2523505484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 msgr2=0x558507ff1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff1e20 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2654019170 tx=0x7f26540a9000 comp rx=0 tx=0).stop 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- 172.21.15.87:0/2523505484 shutdown_connections 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x558507fad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507fac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 --2- 172.21.15.87:0/2523505484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff1e20 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- 172.21.15.87:0/2523505484 >> 172.21.15.87:0/2523505484 conn(0x558507fa8d40 msgr2=0x558507fab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- 172.21.15.87:0/2523505484 shutdown_connections 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- 172.21.15.87:0/2523505484 wait complete. 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- start start 2024-04-25T10:23:29.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 Processor -- start 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507ff91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff2360 0x558507fb2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558507db0540 con 0x558507ff1a40 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558507df1f30 con 0x558507ff2360 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558507dc27c0 con 0x558507facd90 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f266e07d000 10 client.0 _ll_register_callbacks cb 0x558507eaa180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39790/0 (socket says 172.21.15.87:39790) 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 -- 172.21.15.87:0/2216068491 learned_addr learned my addr 172.21.15.87:0/2216068491 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663e00640 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507ff91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2668c00640 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff2360 0x558507fb2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 -- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 msgr2=0x558507ff91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507ff91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 -- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff2360 msgr2=0x558507fb2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff2360 0x558507fb2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 -- 172.21.15.87:0/2216068491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558507d7e1a0 con 0x558507ff1a40 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f2663400640 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff9700 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f2650007ec0 tx=0x7f2650000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e7c35b95f8f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f264d600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f264d600640 1 -- 172.21.15.87:0/2216068491 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650020420 con 0x558507ff1a40 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.221+0000 7f264d600640 1 -- 172.21.15.87:0/2216068491 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f26500205c0 con 0x558507ff1a40 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26540b0050 con 0x558507ff1a40 2024-04-25T10:23:29.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f264d600640 1 -- 172.21.15.87:0/2216068491 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f265002d050 con 0x558507ff1a40 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 10 client.9843 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x558507e78840 con 0x558507ff1a40 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f264d600640 1 -- 172.21.15.87:0/2216068491 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26540b0050 con 0x558507ff1a40 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f264d600640 1 -- 172.21.15.87:0/2216068491 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f2650030020 con 0x558507ff1a40 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 10 client.9843 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 10 client.9843 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 20 client.9843 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558507fb3ba0 0x558507fb3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 4 client.9843 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558507d91be0 con 0x558507fb3ba0 2024-04-25T10:23:29.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f2663e00640 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558507fb3ba0 0x558507fb3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f2663e00640 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558507fb3ba0 0x558507fb3f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=209e7d8b080f0ff0 server_cookie=2ffbb6f3ad63fd0 in_seq=0 out_seq=0 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f264d600640 10 client.9843 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f264d600640 1 -- 172.21.15.87:0/2216068491 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558507d91be0 con 0x558507fb3ba0 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f264d600640 10 client.9843 handle_command_reply: tid=0 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 client.9843 shutdown 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 msgr2=0x558507ff9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff9700 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f2650007ec0 tx=0x7f2650000ee0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 shutdown_connections 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558507fb3ba0 0x558507fb3f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff2360 0x558507fb2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558507ff9700 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f264d600640 0 client.9843 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 --2- 172.21.15.87:0/2216068491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507ff91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 >> 172.21.15.87:0/2216068491 conn(0x558507fa8d40 msgr2=0x558507fab9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 shutdown_connections 2024-04-25T10:23:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 1 -- 172.21.15.87:0/2216068491 wait complete. 2024-04-25T10:23:29.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.225+0000 7f266e07d000 20 client.9843 trim_cache size 0 max 16384 2024-04-25T10:23:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 Processor -- start 2024-04-25T10:23:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 -- start start 2024-04-25T10:23:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507fab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff1a40 0x558507fab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff2360 0x558507ff4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2664202850 con 0x558507ff2360 2024-04-25T10:23:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2664202e50 con 0x558507ff1a40 2024-04-25T10:23:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b2870 con 0x558507facd90 2024-04-25T10:23:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff2360 0x558507ff4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff2360 0x558507ff4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39794/0 (socket says 172.21.15.87:39794) 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 -- 172.21.15.87:0/3896042709 learned_addr learned my addr 172.21.15.87:0/3896042709 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2663400640 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff1a40 0x558507fab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2663e00640 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507fab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 -- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 msgr2=0x558507fab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507fab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 -- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff1a40 msgr2=0x558507fab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff1a40 0x558507fab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 -- 172.21.15.87:0/3896042709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558507ff2190 con 0x558507ff2360 2024-04-25T10:23:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2663400640 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff1a40 0x558507fab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f2668c00640 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff2360 0x558507ff4750 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f265c007420 tx=0x7f265c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51aca1bef2acd896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f264fe00640 1 -- 172.21.15.87:0/3896042709 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c004d80 con 0x558507ff2360 2024-04-25T10:23:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f264fe00640 1 -- 172.21.15.87:0/3896042709 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c001c90 con 0x558507ff2360 2024-04-25T10:23:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f264fe00640 1 -- 172.21.15.87:0/3896042709 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c001e80 con 0x558507ff2360 2024-04-25T10:23:29.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 -- 172.21.15.87:0/3896042709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff2360 msgr2=0x558507ff4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff2360 0x558507ff4750 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f265c007420 tx=0x7f265c0078e0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 -- 172.21.15.87:0/3896042709 shutdown_connections 2024-04-25T10:23:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff2360 0x558507ff4750 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507ff1a40 0x558507fab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 --2- 172.21.15.87:0/3896042709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507facd90 0x558507fab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.241+0000 7f266e07d000 1 -- 172.21.15.87:0/3896042709 >> 172.21.15.87:0/3896042709 conn(0x558507fa8d40 msgr2=0x558507ff0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- 172.21.15.87:0/3896042709 shutdown_connections 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- 172.21.15.87:0/3896042709 wait complete. 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- start start 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 Processor -- start 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x5585080526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558508052c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507ff6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558507db7e50 con 0x558507ff1a40 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b2e70 con 0x558507facd90 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558507df1f30 con 0x558507ff2360 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 10 client.0 _ll_register_callbacks cb 0x558507db8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558508052c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558508052c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39800/0 (socket says 172.21.15.87:39800) 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 -- 172.21.15.87:0/3265349979 learned_addr learned my addr 172.21.15.87:0/3265349979 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2668c00640 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507ff6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663e00640 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x5585080526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 -- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 msgr2=0x558507ff6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507ff6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 -- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 msgr2=0x5585080526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x5585080526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 -- 172.21.15.87:0/3265349979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558507e4fa10 con 0x558507ff1a40 2024-04-25T10:23:29.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663400640 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558508052c00 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f2650007ec0 tx=0x7f2650012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c15fe62ff88e223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f264d600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f264d600640 1 -- 172.21.15.87:0/3265349979 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650014c80 con 0x558507ff1a40 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f264d600640 1 -- 172.21.15.87:0/3265349979 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2650020630 con 0x558507ff1a40 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f265c001c90 con 0x558507ff1a40 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f264d600640 1 -- 172.21.15.87:0/3265349979 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2650041050 con 0x558507ff1a40 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 10 client.9849 fetch_fsmap learned FSMap version 103 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x558507d8ab00 con 0x558507ff1a40 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f264d600640 1 -- 172.21.15.87:0/3265349979 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c001c90 con 0x558507ff1a40 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f264d600640 1 -- 172.21.15.87:0/3265349979 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 103) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f2650044020 con 0x558507ff1a40 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 10 client.9849 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T10:23:29.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 10 client.9849 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 20 client.9849 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558508059520 0x558507ff7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 4 client.9849 mds_command: new command op to 8755 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 --> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x558507e033d0 con 0x558508059520 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.245+0000 7f2663e00640 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558508059520 0x558507ff7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f264d600640 10 client.9849 ms_handle_connect on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f2663e00640 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558508059520 0x558507ff7950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8755 client_cookie=bb27b7ffeca5fd2a server_cookie=ebad65ba5b319b4b in_seq=0 out_seq=0 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f264d600640 1 -- 172.21.15.87:0/3265349979 <== mds.8755 v2:172.21.15.87:6834/4075242775 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x558507e033d0 con 0x558508059520 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f264d600640 10 client.9849 handle_command_reply: tid=0 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 client.9849 shutdown 2024-04-25T10:23:29.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 msgr2=0x558508052c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558508052c00 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f2650007ec0 tx=0x7f2650012000 comp rx=0 tx=0).stop 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 shutdown_connections 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:6834/4075242775,v1:172.21.15.87:6836/4075242775] conn(0x558508059520 0x558507ff7950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558507ff2360 0x558507ff6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558507ff1a40 0x558508052c00 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f264d600640 0 client.9849 ms_handle_reset on v2:172.21.15.87:6834/4075242775 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 --2- 172.21.15.87:0/3265349979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558507facd90 0x5585080526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 >> 172.21.15.87:0/3265349979 conn(0x558507fa8d40 msgr2=0x558507ff09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 shutdown_connections 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 1 -- 172.21.15.87:0/3265349979 wait complete. 2024-04-25T10:23:29.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.249+0000 7f266e07d000 20 client.9849 trim_cache size 0 max 16384 2024-04-25T10:23:29.256 INFO:teuthology.orchestra.run.smithi087.stderr:Error EINVAL: MDS is not active 2024-04-25T10:23:29.256 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:23:29.257 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:23:29.257 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:23:29.337 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:23:29.402 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:23:29.402 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:23:29.402 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:29.411 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:23:29.412 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:29.468 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:29.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.645+0000 7f5c8ba00640 1 Processor -- start 2024-04-25T10:23:29.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.645+0000 7f5c8ba00640 1 -- start start 2024-04-25T10:23:29.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c8ba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5c8415bc40 0x7f5c8406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c8ba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c840cf930 0x7f5c840cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c8ba00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c840b2770 con 0x7f5c840cda10 2024-04-25T10:23:29.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c8ba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8406c670 con 0x7f5c840cf930 2024-04-25T10:23:29.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c8ba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8406c950 con 0x7f5c8415bc40 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c840cf930 0x7f5c840cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c840cf930 0x7f5c840cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39806/0 (socket says 172.21.15.87:39806) 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 -- 172.21.15.87:0/990647572 learned_addr learned my addr 172.21.15.87:0/990647572 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89000640 1 --2- 172.21.15.87:0/990647572 >> v2:172.21.15.169:3300/0 conn(0x7f5c8415bc40 0x7f5c8406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 -- 172.21.15.87:0/990647572 >> v2:172.21.15.169:3300/0 conn(0x7f5c8415bc40 msgr2=0x7f5c8406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 --2- 172.21.15.87:0/990647572 >> v2:172.21.15.169:3300/0 conn(0x7f5c8415bc40 0x7f5c8406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 -- 172.21.15.87:0/990647572 >> v1:172.21.15.169:6789/0 conn(0x7f5c840cda10 legacy=0x7f5c840b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 -- 172.21.15.87:0/990647572 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8406cc30 con 0x7f5c840cf930 2024-04-25T10:23:29.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.649+0000 7f5c89a00640 1 --2- 172.21.15.87:0/990647572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c840cfdc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5c840d0800 tx=0x7f5c7c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9c53db621c61af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c83e00640 1 -- 172.21.15.87:0/990647572 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0a5620 con 0x7f5c840cf930 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c83e00640 1 -- 172.21.15.87:0/990647572 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0a57c0 con 0x7f5c840cf930 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c83e00640 1 -- 172.21.15.87:0/990647572 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0a5b90 con 0x7f5c840cf930 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- 172.21.15.87:0/990647572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 msgr2=0x7f5c840cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/990647572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c840cfdc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5c840d0800 tx=0x7f5c7c01c960 comp rx=0 tx=0).stop 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- 172.21.15.87:0/990647572 shutdown_connections 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/990647572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c840cfdc0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/990647572 >> v2:172.21.15.169:3300/0 conn(0x7f5c8415bc40 0x7f5c8406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- 172.21.15.87:0/990647572 >> 172.21.15.87:0/990647572 conn(0x7f5c84066b70 msgr2=0x7f5c84066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- 172.21.15.87:0/990647572 shutdown_connections 2024-04-25T10:23:29.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- 172.21.15.87:0/990647572 wait complete. 2024-04-25T10:23:29.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 Processor -- start 2024-04-25T10:23:29.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- start start 2024-04-25T10:23:29.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c840cda10 0x7f5c841579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c841fb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c8415bc40 0x7f5c84165f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840b3850 con 0x7f5c840cf930 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840b36d0 con 0x7f5c8415bc40 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c8ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840b39d0 con 0x7f5c840cda10 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c841fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c841fb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39820/0 (socket says 172.21.15.87:39820) 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c7be00640 1 -- 172.21.15.87:0/669105944 learned_addr learned my addr 172.21.15.87:0/669105944 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.653+0000 7f5c89000640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c840cda10 0x7f5c841579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c89a00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c8415bc40 0x7f5c84165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c7be00640 1 -- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c840cda10 msgr2=0x7f5c841579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c7be00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c840cda10 0x7f5c841579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c7be00640 1 -- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c8415bc40 msgr2=0x7f5c84165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c7be00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c8415bc40 0x7f5c84165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c7be00640 1 -- 172.21.15.87:0/669105944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c841f7290 con 0x7f5c840cf930 2024-04-25T10:23:29.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c7be00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c841fb230 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c3a0 tx=0x7f5c6c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c81600640 1 -- 172.21.15.87:0/669105944 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0052d0 con 0x7f5c840cf930 2024-04-25T10:23:29.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c81600640 1 -- 172.21.15.87:0/669105944 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c005470 con 0x7f5c840cf930 2024-04-25T10:23:29.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c81600640 1 -- 172.21.15.87:0/669105944 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c007260 con 0x7f5c840cf930 2024-04-25T10:23:29.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c841f7570 con 0x7f5c840cf930 2024-04-25T10:23:29.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c841f8050 con 0x7f5c840cf930 2024-04-25T10:23:29.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8404ef30 con 0x7f5c840cf930 2024-04-25T10:23:29.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.657+0000 7f5c81600640 1 -- 172.21.15.87:0/669105944 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c6c007400 con 0x7f5c840cf930 2024-04-25T10:23:29.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.661+0000 7f5c81600640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c60066f90 0x7f5c60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.661+0000 7f5c81600640 1 -- 172.21.15.87:0/669105944 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5c6c0844d0 con 0x7f5c840cf930 2024-04-25T10:23:29.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.661+0000 7f5c89000640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c60066f90 0x7f5c60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.661+0000 7f5c89000640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c60066f90 0x7f5c60069450 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5c8415ef00 tx=0x7f5c74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.665+0000 7f5c81600640 1 -- 172.21.15.87:0/669105944 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c6c054dd0 con 0x7f5c840cf930 2024-04-25T10:23:29.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c8406d960 con 0x7f5c840cf930 2024-04-25T10:23:29.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c81600640 1 -- 172.21.15.87:0/669105944 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16717 (secure 0 0 0) 0x7f5c6c058e70 con 0x7f5c840cf930 2024-04-25T10:23:29.820 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:29.820 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":140,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:07.453597+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:54.523910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:55.777113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.36","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c60066f90 msgr2=0x7f5c60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c60066f90 0x7f5c60069450 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5c8415ef00 tx=0x7f5c74008040 comp rx=0 tx=0).stop 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 msgr2=0x7f5c841fb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c841fb230 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c3a0 tx=0x7f5c6c00c860 comp rx=0 tx=0).stop 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 shutdown_connections 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5c60066f90 0x7f5c60069450 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c8415bc40 0x7f5c84165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c840cf930 0x7f5c841fb230 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 --2- 172.21.15.87:0/669105944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c840cda10 0x7f5c841579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 >> 172.21.15.87:0/669105944 conn(0x7f5c84066b70 msgr2=0x7f5c840b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 shutdown_connections 2024-04-25T10:23:29.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.821+0000 7f5c8ba00640 1 -- 172.21.15.87:0/669105944 wait complete. 2024-04-25T10:23:29.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:22:54.523910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:22:55.777113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:23:29.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:23:29.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1317a00640 1 Processor -- start 2024-04-25T10:23:29.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1317a00640 1 -- start start 2024-04-25T10:23:29.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1317a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13100aeda0 0x7f1310071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1317a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1310072020 0x7f1310070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1317a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1310159920 con 0x7f1310159540 2024-04-25T10:23:29.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1317a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131015c2f0 con 0x7f1310072020 2024-04-25T10:23:29.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1317a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131015c5d0 con 0x7f13100aeda0 2024-04-25T10:23:29.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1315a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1310072020 0x7f1310070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1315a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1310072020 0x7f1310070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39836/0 (socket says 172.21.15.87:39836) 2024-04-25T10:23:29.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1315a00640 1 -- 172.21.15.87:0/3401891325 learned_addr learned my addr 172.21.15.87:0/3401891325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1307e00640 1 --2- 172.21.15.87:0/3401891325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13100aeda0 0x7f1310071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1315a00640 1 -- 172.21.15.87:0/3401891325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13100aeda0 msgr2=0x7f1310071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1315a00640 1 --2- 172.21.15.87:0/3401891325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13100aeda0 0x7f1310071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1315a00640 1 -- 172.21.15.87:0/3401891325 >> v1:172.21.15.169:6789/0 conn(0x7f1310159540 legacy=0x7f13100ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:29.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.981+0000 7f1315a00640 1 -- 172.21.15.87:0/3401891325 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131015c8b0 con 0x7f1310072020 2024-04-25T10:23:29.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1315a00640 1 --2- 172.21.15.87:0/3401891325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1310072020 0x7f1310070350 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f1308009e50 tx=0x7f130801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba5a92fc376d5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f130fe00640 1 -- 172.21.15.87:0/3401891325 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13080a5860 con 0x7f1310072020 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f130fe00640 1 -- 172.21.15.87:0/3401891325 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13080a5a00 con 0x7f1310072020 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f130fe00640 1 -- 172.21.15.87:0/3401891325 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13080a5dd0 con 0x7f1310072020 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 -- 172.21.15.87:0/3401891325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1310072020 msgr2=0x7f1310070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 --2- 172.21.15.87:0/3401891325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1310072020 0x7f1310070350 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f1308009e50 tx=0x7f130801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 -- 172.21.15.87:0/3401891325 shutdown_connections 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 --2- 172.21.15.87:0/3401891325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1310072020 0x7f1310070350 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 --2- 172.21.15.87:0/3401891325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13100aeda0 0x7f1310071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 -- 172.21.15.87:0/3401891325 >> 172.21.15.87:0/3401891325 conn(0x7f13100669a0 msgr2=0x7f1310066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 -- 172.21.15.87:0/3401891325 shutdown_connections 2024-04-25T10:23:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.985+0000 7f1317a00640 1 -- 172.21.15.87:0/3401891325 wait complete. 2024-04-25T10:23:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 Processor -- start 2024-04-25T10:23:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 -- start start 2024-04-25T10:23:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1310072020 0x7f1310153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13100aeda0 0x7f13100af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1310159540 0x7f13100ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13100710a0 con 0x7f13100aeda0 2024-04-25T10:23:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131006c0d0 con 0x7f1310072020 2024-04-25T10:23:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1310071220 con 0x7f1310159540 2024-04-25T10:23:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13100aeda0 0x7f13100af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13100aeda0 0x7f13100af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39850/0 (socket says 172.21.15.87:39850) 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 -- 172.21.15.87:0/3027686182 learned_addr learned my addr 172.21.15.87:0/3027686182 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1315000640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1310072020 0x7f1310153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1315a00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1310159540 0x7f13100ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 -- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1310159540 msgr2=0x7f13100ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1310159540 0x7f13100ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 -- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1310072020 msgr2=0x7f1310153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1310072020 0x7f1310153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 -- 172.21.15.87:0/3027686182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310154350 con 0x7f13100aeda0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1307e00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13100aeda0 0x7f13100af670 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d800 tx=0x7f12f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f130d600640 1 -- 172.21.15.87:0/3027686182 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800b6c0 con 0x7f13100aeda0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f130d600640 1 -- 172.21.15.87:0/3027686182 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8019070 con 0x7f13100aeda0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f130d600640 1 -- 172.21.15.87:0/3027686182 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800ba60 con 0x7f13100aeda0 2024-04-25T10:23:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13100aff00 con 0x7f13100aeda0 2024-04-25T10:23:29.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13100b04c0 con 0x7f13100aeda0 2024-04-25T10:23:29.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.989+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131004ef30 con 0x7f13100aeda0 2024-04-25T10:23:29.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.993+0000 7f130d600640 1 -- 172.21.15.87:0/3027686182 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12f800bc00 con 0x7f13100aeda0 2024-04-25T10:23:29.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.993+0000 7f130d600640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12ec066f60 0x7f12ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:29.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.993+0000 7f130d600640 1 -- 172.21.15.87:0/3027686182 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f12f8085530 con 0x7f13100aeda0 2024-04-25T10:23:29.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.993+0000 7f1315000640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12ec066f60 0x7f12ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:29.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.993+0000 7f1315000640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12ec066f60 0x7f12ec069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f13100b3c30 tx=0x7f1300009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:29.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:29.997+0000 7f130d600640 1 -- 172.21.15.87:0/3027686182 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12f8056480 con 0x7f13100aeda0 2024-04-25T10:23:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.149+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f131006d7f0 con 0x7f13100aeda0 2024-04-25T10:23:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.149+0000 7f130d600640 1 -- 172.21.15.87:0/3027686182 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12f8055d80 con 0x7f13100aeda0 2024-04-25T10:23:30.149 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12ec066f60 msgr2=0x7f12ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12ec066f60 0x7f12ec069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f13100b3c30 tx=0x7f1300009060 comp rx=0 tx=0).stop 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13100aeda0 msgr2=0x7f13100af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13100aeda0 0x7f13100af670 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d800 tx=0x7f12f800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 shutdown_connections 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f12ec066f60 0x7f12ec069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1310159540 0x7f13100ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13100aeda0 0x7f13100af670 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 --2- 172.21.15.87:0/3027686182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1310072020 0x7f1310153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 >> 172.21.15.87:0/3027686182 conn(0x7f13100669a0 msgr2=0x7f1310066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 shutdown_connections 2024-04-25T10:23:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.153+0000 7f1317a00640 1 -- 172.21.15.87:0/3027686182 wait complete. 2024-04-25T10:23:30.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:23:30.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30e6c00640 1 Processor -- start 2024-04-25T10:23:30.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30e6c00640 1 -- start start 2024-04-25T10:23:30.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30e6c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30e00aef80 0x7f30e015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30e6c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e015b080 con 0x7f30e00b31c0 2024-04-25T10:23:30.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30e6c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e015adb0 con 0x7f30e0070350 2024-04-25T10:23:30.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30e6c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e015eb60 con 0x7f30e00aef80 2024-04-25T10:23:30.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30e00aef80 0x7f30e015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30e00aef80 0x7f30e015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39856/0 (socket says 172.21.15.87:39856) 2024-04-25T10:23:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30dfe00640 1 -- 172.21.15.87:0/1836166663 learned_addr learned my addr 172.21.15.87:0/1836166663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30dfe00640 1 -- 172.21.15.87:0/1836166663 >> v1:172.21.15.169:6789/0 conn(0x7f30e0070350 legacy=0x7f30e015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30dfe00640 1 -- 172.21.15.87:0/1836166663 >> v1:172.21.15.87:6789/0 conn(0x7f30e00b31c0 legacy=0x7f30e00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.309+0000 7f30dfe00640 1 -- 172.21.15.87:0/1836166663 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e015ee40 con 0x7f30e00aef80 2024-04-25T10:23:30.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30dfe00640 1 --2- 172.21.15.87:0/1836166663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e00aef80 0x7f30e015e480 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009870 tx=0x7f30d001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624b27ba5f0f56d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30dea00640 1 -- 172.21.15.87:0/1836166663 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00a9400 con 0x7f30e00aef80 2024-04-25T10:23:30.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30dea00640 1 -- 172.21.15.87:0/1836166663 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d00a95a0 con 0x7f30e00aef80 2024-04-25T10:23:30.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30dea00640 1 -- 172.21.15.87:0/1836166663 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00a9970 con 0x7f30e00aef80 2024-04-25T10:23:30.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30e6c00640 1 -- 172.21.15.87:0/1836166663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e00aef80 msgr2=0x7f30e015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30e6c00640 1 --2- 172.21.15.87:0/1836166663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e00aef80 0x7f30e015e480 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009870 tx=0x7f30d001c560 comp rx=0 tx=0).stop 2024-04-25T10:23:30.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30e6c00640 1 -- 172.21.15.87:0/1836166663 shutdown_connections 2024-04-25T10:23:30.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30e6c00640 1 --2- 172.21.15.87:0/1836166663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e00aef80 0x7f30e015e480 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30e6c00640 1 -- 172.21.15.87:0/1836166663 >> 172.21.15.87:0/1836166663 conn(0x7f30e00669a0 msgr2=0x7f30e0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:30.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30e6c00640 1 -- 172.21.15.87:0/1836166663 shutdown_connections 2024-04-25T10:23:30.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.313+0000 7f30e6c00640 1 -- 172.21.15.87:0/1836166663 wait complete. 2024-04-25T10:23:30.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 Processor -- start 2024-04-25T10:23:30.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 -- start start 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0070350 0x7f30e00c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30e00aef80 0x7f30e00cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e00b31c0 0x7f30e01590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00cfef0 con 0x7f30e0070350 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00cfd70 con 0x7f30e00aef80 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b35d0 con 0x7f30e00b31c0 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0070350 0x7f30e00c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0070350 0x7f30e00c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39860/0 (socket says 172.21.15.87:39860) 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 -- 172.21.15.87:0/3625458036 learned_addr learned my addr 172.21.15.87:0/3625458036 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30df400640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30e00aef80 0x7f30e00cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e4c00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e00b31c0 0x7f30e01590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 -- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e00b31c0 msgr2=0x7f30e01590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e00b31c0 0x7f30e01590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 -- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30e00aef80 msgr2=0x7f30e00cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30e00aef80 0x7f30e00cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 -- 172.21.15.87:0/3625458036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00cb5a0 con 0x7f30e0070350 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30df400640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30e00aef80 0x7f30e00cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30dfe00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0070350 0x7f30e00c93d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f30d001a190 tx=0x7f30d001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30d7e00640 1 -- 172.21.15.87:0/3625458036 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00b1800 con 0x7f30e0070350 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30d7e00640 1 -- 172.21.15.87:0/3625458036 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d00b19c0 con 0x7f30e0070350 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30d7e00640 1 -- 172.21.15.87:0/3625458036 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00b1db0 con 0x7f30e0070350 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e0144f30 con 0x7f30e0070350 2024-04-25T10:23:30.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e01454f0 con 0x7f30e0070350 2024-04-25T10:23:30.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.317+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e004ef30 con 0x7f30e0070350 2024-04-25T10:23:30.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.321+0000 7f30d7e00640 1 -- 172.21.15.87:0/3625458036 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30d00a8360 con 0x7f30e0070350 2024-04-25T10:23:30.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.321+0000 7f30d7e00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f30c0066f90 0x7f30c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.321+0000 7f30d7e00640 1 -- 172.21.15.87:0/3625458036 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f30d011f600 con 0x7f30e0070350 2024-04-25T10:23:30.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.321+0000 7f30df400640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f30c0066f90 0x7f30c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.321+0000 7f30df400640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f30c0066f90 0x7f30c0069450 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f30e01fa4c0 tx=0x7f30cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.325+0000 7f30d7e00640 1 -- 172.21.15.87:0/3625458036 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30d00c3030 con 0x7f30e0070350 2024-04-25T10:23:30.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.477+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30e00b27f0 con 0x7f30e0070350 2024-04-25T10:23:30.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.477+0000 7f30d7e00640 1 -- 172.21.15.87:0/3625458036 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30d00ad020 con 0x7f30e0070350 2024-04-25T10:23:30.476 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.477+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f30c0066f90 msgr2=0x7f30c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.477+0000 7f30e6c00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f30c0066f90 0x7f30c0069450 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f30e01fa4c0 tx=0x7f30cc009040 comp rx=0 tx=0).stop 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.477+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0070350 msgr2=0x7f30e00c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.477+0000 7f30e6c00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0070350 0x7f30e00c93d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f30d001a190 tx=0x7f30d001ada0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 shutdown_connections 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f30c0066f90 0x7f30c0069450 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e00b31c0 0x7f30e01590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30e00aef80 0x7f30e00cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 --2- 172.21.15.87:0/3625458036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0070350 0x7f30e00c93d0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 >> 172.21.15.87:0/3625458036 conn(0x7f30e00669a0 msgr2=0x7f30e006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:30.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 shutdown_connections 2024-04-25T10:23:30.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.481+0000 7f30e6c00640 1 -- 172.21.15.87:0/3625458036 wait complete. 2024-04-25T10:23:30.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:23:30.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 Processor -- start 2024-04-25T10:23:30.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 -- start start 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9e2c0aef80 0x7f9e2c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c070350 0x7f9e2c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e2c15bac0 0x7f9e2c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2c15b080 con 0x7f9e2c0aef80 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2c15adb0 con 0x7f9e2c15bac0 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e32600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c15eb60 con 0x7f9e2c070350 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9e2c0aef80 0x7f9e2c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9e2c0aef80 0x7f9e2c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39870/0 (socket says 172.21.15.87:39870) 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 -- 172.21.15.87:0/2088266995 learned_addr learned my addr 172.21.15.87:0/2088266995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 -- 172.21.15.87:0/2088266995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c070350 msgr2=0x7f9e2c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 --2- 172.21.15.87:0/2088266995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c070350 0x7f9e2c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 -- 172.21.15.87:0/2088266995 >> v2:172.21.15.169:3300/0 conn(0x7f9e2c15bac0 msgr2=0x7f9e2c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:30.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 --2- 172.21.15.87:0/2088266995 >> v2:172.21.15.169:3300/0 conn(0x7f9e2c15bac0 0x7f9e2c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.637+0000 7f9e2b400640 1 -- 172.21.15.87:0/2088266995 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c15ee40 con 0x7f9e2c0aef80 2024-04-25T10:23:30.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e2b400640 1 --2- 172.21.15.87:0/2088266995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c15e480 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0cc330 tx=0x7f9e1c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc66ca144f38262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e2aa00640 1 -- 172.21.15.87:0/2088266995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0ab5f0 con 0x7f9e2c0aef80 2024-04-25T10:23:30.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e2aa00640 1 -- 172.21.15.87:0/2088266995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c0b0070 con 0x7f9e2c0aef80 2024-04-25T10:23:30.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e2aa00640 1 -- 172.21.15.87:0/2088266995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0ab9b0 con 0x7f9e2c0aef80 2024-04-25T10:23:30.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 -- 172.21.15.87:0/2088266995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 msgr2=0x7f9e2c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 --2- 172.21.15.87:0/2088266995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c15e480 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0cc330 tx=0x7f9e1c01c520 comp rx=0 tx=0).stop 2024-04-25T10:23:30.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 -- 172.21.15.87:0/2088266995 shutdown_connections 2024-04-25T10:23:30.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 --2- 172.21.15.87:0/2088266995 >> v2:172.21.15.169:3300/0 conn(0x7f9e2c15bac0 0x7f9e2c0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 --2- 172.21.15.87:0/2088266995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c070350 0x7f9e2c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 --2- 172.21.15.87:0/2088266995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c15e480 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 -- 172.21.15.87:0/2088266995 >> 172.21.15.87:0/2088266995 conn(0x7f9e2c0669a0 msgr2=0x7f9e2c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:30.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 -- 172.21.15.87:0/2088266995 shutdown_connections 2024-04-25T10:23:30.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.641+0000 7f9e32600640 1 -- 172.21.15.87:0/2088266995 wait complete. 2024-04-25T10:23:30.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 Processor -- start 2024-04-25T10:23:30.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 -- start start 2024-04-25T10:23:30.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e2c070350 0x7f9e2c14cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c15bac0 0x7f9e2c155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c06d5b0 con 0x7f9e2c0aef80 2024-04-25T10:23:30.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c06d430 con 0x7f9e2c070350 2024-04-25T10:23:30.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c06d730 con 0x7f9e2c15bac0 2024-04-25T10:23:30.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e2c070350 0x7f9e2c14cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39882/0 (socket says 172.21.15.87:39882) 2024-04-25T10:23:30.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 -- 172.21.15.87:0/2688969525 learned_addr learned my addr 172.21.15.87:0/2688969525 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:30.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e2be00640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c15bac0 0x7f9e2c155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 -- 172.21.15.87:0/2688969525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c15bac0 msgr2=0x7f9e2c155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c15bac0 0x7f9e2c155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 -- 172.21.15.87:0/2688969525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e2c070350 msgr2=0x7f9e2c14cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e2c070350 0x7f9e2c14cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 -- 172.21.15.87:0/2688969525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c14e720 con 0x7f9e2c0aef80 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23e00640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c155310 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b580 tx=0x7f9e1400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23400640 1 -- 172.21.15.87:0/2688969525 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14014070 con 0x7f9e2c0aef80 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23400640 1 -- 172.21.15.87:0/2688969525 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14010890 con 0x7f9e2c0aef80 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c14d680 con 0x7f9e2c0aef80 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e23400640 1 -- 172.21.15.87:0/2688969525 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14010c80 con 0x7f9e2c0aef80 2024-04-25T10:23:30.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.645+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c14dc40 con 0x7f9e2c0aef80 2024-04-25T10:23:30.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.649+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2c04ef30 con 0x7f9e2c0aef80 2024-04-25T10:23:30.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.649+0000 7f9e23400640 1 -- 172.21.15.87:0/2688969525 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e14017070 con 0x7f9e2c0aef80 2024-04-25T10:23:30.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.649+0000 7f9e23400640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9e08066f60 0x7f9e08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.649+0000 7f9e23400640 1 -- 172.21.15.87:0/2688969525 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9e14084e50 con 0x7f9e2c0aef80 2024-04-25T10:23:30.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.649+0000 7f9e2b400640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9e08066f60 0x7f9e08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.649+0000 7f9e2b400640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9e08066f60 0x7f9e08069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0af000 tx=0x7f9e1c0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.653+0000 7f9e23400640 1 -- 172.21.15.87:0/2688969525 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e14055760 con 0x7f9e2c0aef80 2024-04-25T10:23:30.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e2c070fb0 con 0x7f9e2c0aef80 2024-04-25T10:23:30.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e23400640 1 -- 172.21.15.87:0/2688969525 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e14010e20 con 0x7f9e2c0aef80 2024-04-25T10:23:30.807 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:23:30.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9e08066f60 msgr2=0x7f9e08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9e08066f60 0x7f9e08069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0af000 tx=0x7f9e1c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:23:30.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 msgr2=0x7f9e2c155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c155310 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b580 tx=0x7f9e1400ba40 comp rx=0 tx=0).stop 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 shutdown_connections 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9e08066f60 0x7f9e08069420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2c15bac0 0x7f9e2c155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e2c0aef80 0x7f9e2c155310 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 --2- 172.21.15.87:0/2688969525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e2c070350 0x7f9e2c14cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 >> 172.21.15.87:0/2688969525 conn(0x7f9e2c0669a0 msgr2=0x7f9e2c152720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 shutdown_connections 2024-04-25T10:23:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.809+0000 7f9e32600640 1 -- 172.21.15.87:0/2688969525 wait complete. 2024-04-25T10:23:30.828 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:23:30.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bfa200640 1 Processor -- start 2024-04-25T10:23:30.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bfa200640 1 -- start start 2024-04-25T10:23:30.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bfa200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf415bc40 0x7f5bf406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bfa200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5bf40cf930 0x7f5bf40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bfa200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf40b2770 con 0x7f5bf40cda10 2024-04-25T10:23:30.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bfa200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf406c670 con 0x7f5bf40cf930 2024-04-25T10:23:30.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bfa200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf406c950 con 0x7f5bf415bc40 2024-04-25T10:23:30.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bf3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf415bc40 0x7f5bf406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bf3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5bf40cf930 0x7f5bf40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5beaa00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5bf40cda10 0x7f5bf40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:42204/0 (socket says 172.21.15.87:42204) 2024-04-25T10:23:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bf3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf415bc40 0x7f5bf406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:37646/0 (socket says 172.21.15.87:37646) 2024-04-25T10:23:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5beaa00640 1 -- 172.21.15.87:0/2048429865 learned_addr learned my addr 172.21.15.87:0/2048429865 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bf3e00640 1 -- 172.21.15.87:0/2048429865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf415bc40 msgr2=0x7f5bf406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bf3e00640 1 --2- 172.21.15.87:0/2048429865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf415bc40 0x7f5bf406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bf3e00640 1 -- 172.21.15.87:0/2048429865 >> v1:172.21.15.169:6789/0 conn(0x7f5bf40cda10 legacy=0x7f5bf40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.965+0000 7f5bf3e00640 1 -- 172.21.15.87:0/2048429865 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf406cc30 con 0x7f5bf40cf930 2024-04-25T10:23:30.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bf3e00640 1 --2- 172.21.15.87:0/2048429865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 0x7f5bf40cfde0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d90 tx=0x7f5bec01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61492eda9250ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bf2a00640 1 -- 172.21.15.87:0/2048429865 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0b2070 con 0x7f5bf40cf930 2024-04-25T10:23:30.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bf2a00640 1 -- 172.21.15.87:0/2048429865 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec0a57a0 con 0x7f5bf40cf930 2024-04-25T10:23:30.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bf2a00640 1 -- 172.21.15.87:0/2048429865 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0a5b70 con 0x7f5bf40cf930 2024-04-25T10:23:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 -- 172.21.15.87:0/2048429865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 msgr2=0x7f5bf40cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 --2- 172.21.15.87:0/2048429865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 0x7f5bf40cfde0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d90 tx=0x7f5bec01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 -- 172.21.15.87:0/2048429865 shutdown_connections 2024-04-25T10:23:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 --2- 172.21.15.87:0/2048429865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 0x7f5bf40cfde0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 --2- 172.21.15.87:0/2048429865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf415bc40 0x7f5bf406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 -- 172.21.15.87:0/2048429865 >> 172.21.15.87:0/2048429865 conn(0x7f5bf4066b70 msgr2=0x7f5bf4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:30.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 -- 172.21.15.87:0/2048429865 shutdown_connections 2024-04-25T10:23:30.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.969+0000 7f5bfa200640 1 -- 172.21.15.87:0/2048429865 wait complete. 2024-04-25T10:23:30.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 Processor -- start 2024-04-25T10:23:30.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 -- start start 2024-04-25T10:23:30.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cda10 0x7f5bf4150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 0x7f5bf40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bf415bc40 0x7f5bf4155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40b3850 con 0x7f5bf415bc40 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40b36d0 con 0x7f5bf40cf930 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bfa200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40b39d0 con 0x7f5bf40cda10 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bf415bc40 0x7f5bf4155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bf415bc40 0x7f5bf4155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39892/0 (socket says 172.21.15.87:39892) 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 -- 172.21.15.87:0/3347452225 learned_addr learned my addr 172.21.15.87:0/3347452225 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5beaa00640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 0x7f5bf40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3400640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cda10 0x7f5bf4150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 -- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cda10 msgr2=0x7f5bf4150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cda10 0x7f5bf4150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 -- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 msgr2=0x7f5bf40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 0x7f5bf40ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 -- 172.21.15.87:0/3347452225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf4151170 con 0x7f5bf415bc40 2024-04-25T10:23:30.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bf3e00640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bf415bc40 0x7f5bf4155030 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0b0040 tx=0x7f5bec019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.973+0000 7f5bebe00640 1 -- 172.21.15.87:0/3347452225 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0b2050 con 0x7f5bf415bc40 2024-04-25T10:23:30.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bebe00640 1 -- 172.21.15.87:0/3347452225 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec0be070 con 0x7f5bf415bc40 2024-04-25T10:23:30.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bebe00640 1 -- 172.21.15.87:0/3347452225 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec009450 con 0x7f5bf415bc40 2024-04-25T10:23:30.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf415b330 con 0x7f5bf415bc40 2024-04-25T10:23:30.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf415a850 con 0x7f5bf415bc40 2024-04-25T10:23:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf404ef30 con 0x7f5bf415bc40 2024-04-25T10:23:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bebe00640 1 -- 172.21.15.87:0/3347452225 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bec0adc40 con 0x7f5bf415bc40 2024-04-25T10:23:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bebe00640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bd0066f60 0x7f5bd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bebe00640 1 -- 172.21.15.87:0/3347452225 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5bec11fff0 con 0x7f5bf415bc40 2024-04-25T10:23:30.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bf3400640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bd0066f60 0x7f5bd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:30.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.977+0000 7f5bf3400640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bd0066f60 0x7f5bd0069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5bf415ffb0 tx=0x7f5be4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:30.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:30.981+0000 7f5bebe00640 1 -- 172.21.15.87:0/3347452225 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bec016560 con 0x7f5bf415bc40 2024-04-25T10:23:31.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5bf40af1c0 con 0x7f5bf415bc40 2024-04-25T10:23:31.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bebe00640 1 -- 172.21.15.87:0/3347452225 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5bec0f0950 con 0x7f5bf415bc40 2024-04-25T10:23:31.148 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:23:31.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bd0066f60 msgr2=0x7f5bd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:31.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bd0066f60 0x7f5bd0069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5bf415ffb0 tx=0x7f5be4009190 comp rx=0 tx=0).stop 2024-04-25T10:23:31.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bf415bc40 msgr2=0x7f5bf4155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bf415bc40 0x7f5bf4155030 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0b0040 tx=0x7f5bec019680 comp rx=0 tx=0).stop 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 shutdown_connections 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bd0066f60 0x7f5bd0069420 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bf415bc40 0x7f5bf4155030 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40cf930 0x7f5bf40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 --2- 172.21.15.87:0/3347452225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cda10 0x7f5bf4150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 >> 172.21.15.87:0/3347452225 conn(0x7f5bf4066b70 msgr2=0x7f5bf4155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.149+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 shutdown_connections 2024-04-25T10:23:31.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:31.153+0000 7f5bfa200640 1 -- 172.21.15.87:0/3347452225 wait complete. 2024-04-25T10:23:31.169 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:23:31.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:31.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:31.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:23:31.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:31.909 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:23:31.918 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:23:31.918 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:31.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:23:31.927 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:23:31.927 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:31.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:23:31.970 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:23:31.971 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:31.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:23:31.980 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:32.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4f800640 1 Processor -- start 2024-04-25T10:23:32.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4f800640 1 -- start start 2024-04-25T10:23:32.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4815b9e0 0x7f1d480cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4f800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d4815bdc0 0x7f1d480cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4f800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d480ccfb0 con 0x7f1d480d1520 2024-04-25T10:23:32.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4f800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d480cd290 con 0x7f1d4815bdc0 2024-04-25T10:23:32.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480cd570 con 0x7f1d4815b9e0 2024-04-25T10:23:32.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d3fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d4815bdc0 0x7f1d480cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d4d800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1d480d1520 0x7f1d480d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53510/0 (socket says 172.21.15.87:53510) 2024-04-25T10:23:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.149+0000 7f1d3fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d4815bdc0 0x7f1d480cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39896/0 (socket says 172.21.15.87:39896) 2024-04-25T10:23:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d4d800640 1 -- 172.21.15.87:0/2175810302 learned_addr learned my addr 172.21.15.87:0/2175810302 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d4ce00640 1 --2- 172.21.15.87:0/2175810302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4815b9e0 0x7f1d480cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d3fe00640 1 -- 172.21.15.87:0/2175810302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4815b9e0 msgr2=0x7f1d480cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d3fe00640 1 --2- 172.21.15.87:0/2175810302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4815b9e0 0x7f1d480cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d3fe00640 1 -- 172.21.15.87:0/2175810302 >> v1:172.21.15.87:6789/0 conn(0x7f1d480d1520 legacy=0x7f1d480d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d3fe00640 1 -- 172.21.15.87:0/2175810302 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480cd850 con 0x7f1d4815bdc0 2024-04-25T10:23:32.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d47e00640 1 -- 172.21.15.87:0/2175810302 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652947232 0 0) 0x7f1d480ccfb0 con 0x7f1d480d1520 2024-04-25T10:23:32.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d3fe00640 1 --2- 172.21.15.87:0/2175810302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d480cc950 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1d300097c0 tx=0x7f1d3001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a4ecc9b4d9eea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d47e00640 1 -- 172.21.15.87:0/2175810302 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300b0070 con 0x7f1d4815bdc0 2024-04-25T10:23:32.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d47e00640 1 -- 172.21.15.87:0/2175810302 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30004230 con 0x7f1d4815bdc0 2024-04-25T10:23:32.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.153+0000 7f1d47e00640 1 -- 172.21.15.87:0/2175810302 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30003620 con 0x7f1d4815bdc0 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- 172.21.15.87:0/2175810302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 msgr2=0x7f1d480cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2175810302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d480cc950 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1d300097c0 tx=0x7f1d3001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- 172.21.15.87:0/2175810302 shutdown_connections 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2175810302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d480cc950 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2175810302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4815b9e0 0x7f1d480cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- 172.21.15.87:0/2175810302 >> 172.21.15.87:0/2175810302 conn(0x7f1d4809a9d0 msgr2=0x7f1d4809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- 172.21.15.87:0/2175810302 shutdown_connections 2024-04-25T10:23:32.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- 172.21.15.87:0/2175810302 wait complete. 2024-04-25T10:23:32.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 Processor -- start 2024-04-25T10:23:32.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- start start 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d480d1520 0x7f1d480c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815b9e0 0x7f1d480c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d48157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480c6630 con 0x7f1d4815bdc0 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480c64b0 con 0x7f1d4815b9e0 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480c67b0 con 0x7f1d480d1520 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d48157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d48157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39908/0 (socket says 172.21.15.87:39908) 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 -- 172.21.15.87:0/2171463714 learned_addr learned my addr 172.21.15.87:0/2171463714 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:32.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d480d1520 0x7f1d480c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d3fe00640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815b9e0 0x7f1d480c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 -- 172.21.15.87:0/2171463714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d480d1520 msgr2=0x7f1d480c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d480d1520 0x7f1d480c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 -- 172.21.15.87:0/2171463714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815b9e0 msgr2=0x7f1d480c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815b9e0 0x7f1d480c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 -- 172.21.15.87:0/2171463714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d481446b0 con 0x7f1d4815bdc0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.157+0000 7f1d4d800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d48157bc0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c7d0 tx=0x7f1d4000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d45600640 1 -- 172.21.15.87:0/2171463714 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40018070 con 0x7f1d4815bdc0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d45600640 1 -- 172.21.15.87:0/2171463714 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4000a640 con 0x7f1d4815bdc0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d481664c0 con 0x7f1d4815bdc0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d45600640 1 -- 172.21.15.87:0/2171463714 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000aa70 con 0x7f1d4815bdc0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d480ca4f0 con 0x7f1d4815bdc0 2024-04-25T10:23:32.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d480ca320 con 0x7f1d4815bdc0 2024-04-25T10:23:32.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d45600640 1 -- 172.21.15.87:0/2171463714 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d4001b070 con 0x7f1d4815bdc0 2024-04-25T10:23:32.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d45600640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d24066f90 0x7f1d24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d45600640 1 -- 172.21.15.87:0/2171463714 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1d40084630 con 0x7f1d4815bdc0 2024-04-25T10:23:32.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d4ce00640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d24066f90 0x7f1d24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.161+0000 7f1d4ce00640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d24066f90 0x7f1d24069450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007a60 tx=0x7f1d38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.165+0000 7f1d45600640 1 -- 172.21.15.87:0/2171463714 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d40054ea0 con 0x7f1d4815bdc0 2024-04-25T10:23:32.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d4804ef30 con 0x7f1d4815bdc0 2024-04-25T10:23:32.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d45600640 1 -- 172.21.15.87:0/2171463714 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f1d40058f40 con 0x7f1d4815bdc0 2024-04-25T10:23:32.353 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:32.353 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":103,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:22:56.766091+0000","modified":"2024-04-25T10:23:27.501228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8755":{"gid":8755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6836/4075242775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4075242775},{"type":"v1","addr":"172.21.15.87:6836","nonce":4075242775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8758":{"gid":8758,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1025107360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1025107360},{"type":"v1","addr":"172.21.15.87:6837","nonce":1025107360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:23:32.354 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 103 2024-04-25T10:23:32.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d24066f90 msgr2=0x7f1d24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d24066f90 0x7f1d24069450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007a60 tx=0x7f1d38008040 comp rx=0 tx=0).stop 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 msgr2=0x7f1d48157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d48157bc0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c7d0 tx=0x7f1d4000cc90 comp rx=0 tx=0).stop 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 shutdown_connections 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1d24066f90 0x7f1d24069450 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d4815bdc0 0x7f1d48157bc0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815b9e0 0x7f1d480c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 --2- 172.21.15.87:0/2171463714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d480d1520 0x7f1d480c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 >> 172.21.15.87:0/2171463714 conn(0x7f1d4809a9d0 msgr2=0x7f1d4815aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:32.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 shutdown_connections 2024-04-25T10:23:32.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.353+0000 7f1d4f800640 1 -- 172.21.15.87:0/2171463714 wait complete. 2024-04-25T10:23:32.374 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:32.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.509+0000 7fc5ddc00640 1 Processor -- start 2024-04-25T10:23:32.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.509+0000 7fc5ddc00640 1 -- start start 2024-04-25T10:23:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5ddc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5d80c6a20 0x7fc5d8166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5ddc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80ccfb0 con 0x7fc5d815b9e0 2024-04-25T10:23:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5ddc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80cd290 con 0x7fc5d80ccb20 2024-04-25T10:23:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5ddc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d80cd570 con 0x7fc5d80c6a20 2024-04-25T10:23:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5d6a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc5d815b9e0 0x7fc5d80cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53512/0 (socket says 172.21.15.87:53512) 2024-04-25T10:23:32.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5d6a00640 1 -- 172.21.15.87:0/2410728989 learned_addr learned my addr 172.21.15.87:0/2410728989 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:32.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5d7400640 1 --2- 172.21.15.87:0/2410728989 >> v2:172.21.15.169:3300/0 conn(0x7fc5d80c6a20 0x7fc5d8166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5d7400640 1 -- 172.21.15.87:0/2410728989 >> v1:172.21.15.169:6789/0 conn(0x7fc5d80ccb20 legacy=0x7fc5d80d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5d7400640 1 -- 172.21.15.87:0/2410728989 >> v1:172.21.15.87:6789/0 conn(0x7fc5d815b9e0 legacy=0x7fc5d80cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.513+0000 7fc5d7400640 1 -- 172.21.15.87:0/2410728989 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80cd850 con 0x7fc5d80c6a20 2024-04-25T10:23:32.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5d7400640 1 --2- 172.21.15.87:0/2410728989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d80c6a20 0x7fc5d8166820 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009d90 tx=0x7fc5d001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a98327d920c91d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5d6000640 1 -- 172.21.15.87:0/2410728989 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00b2070 con 0x7fc5d80c6a20 2024-04-25T10:23:32.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5d6000640 1 -- 172.21.15.87:0/2410728989 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d00a58e0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5d6000640 1 -- 172.21.15.87:0/2410728989 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00a5cb0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2410728989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d80c6a20 msgr2=0x7fc5d8166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2410728989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d80c6a20 0x7fc5d8166820 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009d90 tx=0x7fc5d001c970 comp rx=0 tx=0).stop 2024-04-25T10:23:32.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2410728989 shutdown_connections 2024-04-25T10:23:32.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2410728989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d80c6a20 0x7fc5d8166820 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2410728989 >> 172.21.15.87:0/2410728989 conn(0x7fc5d809a9d0 msgr2=0x7fc5d809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:32.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2410728989 shutdown_connections 2024-04-25T10:23:32.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2410728989 wait complete. 2024-04-25T10:23:32.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 Processor -- start 2024-04-25T10:23:32.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.517+0000 7fc5ddc00640 1 -- start start 2024-04-25T10:23:32.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d80c6a20 0x7fc5d80b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5d815b9e0 0x7fc5d815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d8159f10 0x7fc5d806f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8169910 con 0x7fc5d80c6a20 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8169790 con 0x7fc5d8159f10 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8169a90 con 0x7fc5d815b9e0 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d80c6a20 0x7fc5d80b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d80c6a20 0x7fc5d80b6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39922/0 (socket says 172.21.15.87:39922) 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 -- 172.21.15.87:0/2462290095 learned_addr learned my addr 172.21.15.87:0/2462290095 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d7400640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d8159f10 0x7fc5d806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ce000640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5d815b9e0 0x7fc5d815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 -- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5d815b9e0 msgr2=0x7fc5d815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5d815b9e0 0x7fc5d815cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 -- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d8159f10 msgr2=0x7fc5d806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d8159f10 0x7fc5d806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 -- 172.21.15.87:0/2462290095 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d815d1b0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5d6a00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d80c6a20 0x7fc5d80b6f50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c400 tx=0x7fc5c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5cf400640 1 -- 172.21.15.87:0/2462290095 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8017070 con 0x7fc5d80c6a20 2024-04-25T10:23:32.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5cf400640 1 -- 172.21.15.87:0/2462290095 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8005cf0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d815d430 con 0x7fc5d80c6a20 2024-04-25T10:23:32.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5cf400640 1 -- 172.21.15.87:0/2462290095 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80072d0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d815d9f0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d804ef30 con 0x7fc5d80c6a20 2024-04-25T10:23:32.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.521+0000 7fc5cf400640 1 -- 172.21.15.87:0/2462290095 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5c801a070 con 0x7fc5d80c6a20 2024-04-25T10:23:32.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.525+0000 7fc5cf400640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5b4066f90 0x7fc5b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.525+0000 7fc5cf400640 1 -- 172.21.15.87:0/2462290095 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc5c80843b0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.525+0000 7fc5ce000640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5b4066f90 0x7fc5b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.525+0000 7fc5ce000640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5b4066f90 0x7fc5b4069450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80b7490 tx=0x7fc5c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.529+0000 7fc5cf400640 1 -- 172.21.15.87:0/2462290095 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5d815d1b0 con 0x7fc5d80c6a20 2024-04-25T10:23:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.685+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5d806bf40 con 0x7fc5d80c6a20 2024-04-25T10:23:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.685+0000 7fc5cf400640 1 -- 172.21.15.87:0/2462290095 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16717 (secure 0 0 0) 0x7fc5c8054c30 con 0x7fc5d80c6a20 2024-04-25T10:23:32.684 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:32.684 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":140,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:07.453597+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:54.523910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:55.777113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.36","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:32.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.685+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5b4066f90 msgr2=0x7fc5b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.685+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5b4066f90 0x7fc5b4069450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80b7490 tx=0x7fc5c0009040 comp rx=0 tx=0).stop 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.685+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d80c6a20 msgr2=0x7fc5d80b6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.685+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d80c6a20 0x7fc5d80b6f50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c400 tx=0x7fc5c800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 shutdown_connections 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc5b4066f90 0x7fc5b4069450 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d8159f10 0x7fc5d806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5d815b9e0 0x7fc5d815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 --2- 172.21.15.87:0/2462290095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d80c6a20 0x7fc5d80b6f50 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 >> 172.21.15.87:0/2462290095 conn(0x7fc5d809a9d0 msgr2=0x7fc5d806e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 shutdown_connections 2024-04-25T10:23:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.689+0000 7fc5ddc00640 1 -- 172.21.15.87:0/2462290095 wait complete. 2024-04-25T10:23:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:22:54.523910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:22:55.777113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:23:32.707 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:23:32.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a03800640 1 Processor -- start 2024-04-25T10:23:32.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a03800640 1 -- start start 2024-04-25T10:23:32.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a03800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f39fc15a0e0 0x7f39fc15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a03800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39fc0aea50 0x7f39fc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a03800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc0b2fd0 con 0x7f39fc0b42d0 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a03800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc0af1d0 con 0x7f39fc0aea50 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a03800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc0b32b0 con 0x7f39fc15a0e0 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f39f3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39fc0aea50 0x7f39fc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a01800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f39fc0b42d0 0x7f39fc06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53526/0 (socket says 172.21.15.87:53526) 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.845+0000 7f3a01800640 1 -- 172.21.15.87:0/1455171412 learned_addr learned my addr 172.21.15.87:0/1455171412 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f3a00e00640 1 --2- 172.21.15.87:0/1455171412 >> v2:172.21.15.169:3300/0 conn(0x7f39fc15a0e0 0x7f39fc15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39f3e00640 1 -- 172.21.15.87:0/1455171412 >> v2:172.21.15.169:3300/0 conn(0x7f39fc15a0e0 msgr2=0x7f39fc15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:32.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39f3e00640 1 --2- 172.21.15.87:0/1455171412 >> v2:172.21.15.169:3300/0 conn(0x7f39fc15a0e0 0x7f39fc15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39f3e00640 1 -- 172.21.15.87:0/1455171412 >> v1:172.21.15.87:6789/0 conn(0x7f39fc0b42d0 legacy=0x7f39fc06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39f3e00640 1 -- 172.21.15.87:0/1455171412 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0b3590 con 0x7f39fc0aea50 2024-04-25T10:23:32.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39fbe00640 1 -- 172.21.15.87:0/1455171412 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967462488 0 0) 0x7f39fc0b2fd0 con 0x7f39fc0b42d0 2024-04-25T10:23:32.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f3a00e00640 1 --2- 172.21.15.87:0/1455171412 >> v2:172.21.15.169:3300/0 conn(0x7f39fc15a0e0 0x7f39fc15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:32.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39f3e00640 1 --2- 172.21.15.87:0/1455171412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0b3d90 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f39fc06e4c0 tx=0x7f39e401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93941e06f430a5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39fbe00640 1 -- 172.21.15.87:0/1455171412 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40b0070 con 0x7f39fc0aea50 2024-04-25T10:23:32.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39fbe00640 1 -- 172.21.15.87:0/1455171412 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e40ab600 con 0x7f39fc0aea50 2024-04-25T10:23:32.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f39fbe00640 1 -- 172.21.15.87:0/1455171412 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40ab9d0 con 0x7f39fc0aea50 2024-04-25T10:23:32.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f3a03800640 1 -- 172.21.15.87:0/1455171412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 msgr2=0x7f39fc0b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.849+0000 7f3a03800640 1 --2- 172.21.15.87:0/1455171412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0b3d90 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f39fc06e4c0 tx=0x7f39e401c360 comp rx=0 tx=0).stop 2024-04-25T10:23:32.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- 172.21.15.87:0/1455171412 shutdown_connections 2024-04-25T10:23:32.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 --2- 172.21.15.87:0/1455171412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0b3d90 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 --2- 172.21.15.87:0/1455171412 >> v2:172.21.15.169:3300/0 conn(0x7f39fc15a0e0 0x7f39fc15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- 172.21.15.87:0/1455171412 >> 172.21.15.87:0/1455171412 conn(0x7f39fc0669a0 msgr2=0x7f39fc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:32.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- 172.21.15.87:0/1455171412 shutdown_connections 2024-04-25T10:23:32.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- 172.21.15.87:0/1455171412 wait complete. 2024-04-25T10:23:32.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 Processor -- start 2024-04-25T10:23:32.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- start start 2024-04-25T10:23:32.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39fc0b42d0 0x7f39fc155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc15a0e0 0x7f39fc1444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc070770 con 0x7f39fc0aea50 2024-04-25T10:23:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0705f0 con 0x7f39fc15a0e0 2024-04-25T10:23:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0708f0 con 0x7f39fc0b42d0 2024-04-25T10:23:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39948/0 (socket says 172.21.15.87:39948) 2024-04-25T10:23:32.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 -- 172.21.15.87:0/122853724 learned_addr learned my addr 172.21.15.87:0/122853724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 -- 172.21.15.87:0/122853724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39fc0b42d0 msgr2=0x7f39fc155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39fc0b42d0 0x7f39fc155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 -- 172.21.15.87:0/122853724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc15a0e0 msgr2=0x7f39fc1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc15a0e0 0x7f39fc1444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 -- 172.21.15.87:0/122853724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc1594a0 con 0x7f39fc0aea50 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a00e00640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0cc150 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00d830 tx=0x7f39ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f39f9600640 1 -- 172.21.15.87:0/122853724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00b6c0 con 0x7f39fc0aea50 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f39f9600640 1 -- 172.21.15.87:0/122853724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec019070 con 0x7f39fc0aea50 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f39f9600640 1 -- 172.21.15.87:0/122853724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00baa0 con 0x7f39fc0aea50 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.853+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc159780 con 0x7f39fc0aea50 2024-04-25T10:23:32.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.857+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc1556a0 con 0x7f39fc0aea50 2024-04-25T10:23:32.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.857+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc1551b0 con 0x7f39fc0aea50 2024-04-25T10:23:32.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.857+0000 7f39f9600640 1 -- 172.21.15.87:0/122853724 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39ec01e460 con 0x7f39fc0aea50 2024-04-25T10:23:32.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.857+0000 7f39f9600640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39d8066f90 0x7f39d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:32.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.857+0000 7f39f9600640 1 -- 172.21.15.87:0/122853724 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f39ec085720 con 0x7f39fc0aea50 2024-04-25T10:23:32.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.857+0000 7f39f3e00640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39d8066f90 0x7f39d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:32.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.857+0000 7f39f3e00640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39d8066f90 0x7f39d8069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f39e40ad000 tx=0x7f39e4004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:32.861+0000 7f39f9600640 1 -- 172.21.15.87:0/122853724 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39ec056670 con 0x7f39fc0aea50 2024-04-25T10:23:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39fc04ef30 con 0x7f39fc0aea50 2024-04-25T10:23:33.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f39f9600640 1 -- 172.21.15.87:0/122853724 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39ec055f70 con 0x7f39fc0aea50 2024-04-25T10:23:33.016 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39d8066f90 msgr2=0x7f39d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39d8066f90 0x7f39d8069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f39e40ad000 tx=0x7f39e4004740 comp rx=0 tx=0).stop 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 msgr2=0x7f39fc0cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0cc150 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00d830 tx=0x7f39ec00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 shutdown_connections 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f39d8066f90 0x7f39d8069450 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc15a0e0 0x7f39fc1444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39fc0b42d0 0x7f39fc155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 --2- 172.21.15.87:0/122853724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39fc0aea50 0x7f39fc0cc150 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 >> 172.21.15.87:0/122853724 conn(0x7f39fc0669a0 msgr2=0x7f39fc071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:33.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 shutdown_connections 2024-04-25T10:23:33.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.017+0000 7f3a03800640 1 -- 172.21.15.87:0/122853724 wait complete. 2024-04-25T10:23:33.038 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:23:33.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.177+0000 7ff905800640 1 Processor -- start 2024-04-25T10:23:33.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.177+0000 7ff905800640 1 -- start start 2024-04-25T10:23:33.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.177+0000 7ff905800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff90015bc10 0x7ff90015f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.177+0000 7ff905800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff9000cf980 0x7ff9000b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.177+0000 7ff905800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90015fec0 con 0x7ff9000d1830 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.177+0000 7ff905800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9001601a0 con 0x7ff9000cf980 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.177+0000 7ff905800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000cc5b0 con 0x7ff90015bc10 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8f6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff9000cf980 0x7ff9000b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8fea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff90015bc10 0x7ff90015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8ff400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff9000d1830 0x7ff9000d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:42220/0 (socket says 172.21.15.87:42220) 2024-04-25T10:23:33.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8ff400640 1 -- 172.21.15.87:0/1211873924 learned_addr learned my addr 172.21.15.87:0/1211873924 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8f6000640 1 -- 172.21.15.87:0/1211873924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff90015bc10 msgr2=0x7ff90015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8f6000640 1 --2- 172.21.15.87:0/1211873924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff90015bc10 0x7ff90015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8f6000640 1 -- 172.21.15.87:0/1211873924 >> v1:172.21.15.169:6789/0 conn(0x7ff9000d1830 legacy=0x7ff9000d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8f6000640 1 -- 172.21.15.87:0/1211873924 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000cc890 con 0x7ff9000cf980 2024-04-25T10:23:33.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8fe000640 1 -- 172.21.15.87:0/1211873924 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814541932 0 0) 0x7ff90015fec0 con 0x7ff9000d1830 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8f6000640 1 --2- 172.21.15.87:0/1211873924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9000cf980 0x7ff9000b1060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80097c0 tx=0x7ff8e801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e957dc956a95db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8fe000640 1 -- 172.21.15.87:0/1211873924 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80b0070 con 0x7ff9000cf980 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8fe000640 1 -- 172.21.15.87:0/1211873924 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e80ab600 con 0x7ff9000cf980 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.181+0000 7ff8fe000640 1 -- 172.21.15.87:0/1211873924 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80ab9d0 con 0x7ff9000cf980 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- 172.21.15.87:0/1211873924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9000cf980 msgr2=0x7ff9000b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 --2- 172.21.15.87:0/1211873924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9000cf980 0x7ff9000b1060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80097c0 tx=0x7ff8e801c410 comp rx=0 tx=0).stop 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- 172.21.15.87:0/1211873924 shutdown_connections 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 --2- 172.21.15.87:0/1211873924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9000cf980 0x7ff9000b1060 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 --2- 172.21.15.87:0/1211873924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff90015bc10 0x7ff90015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- 172.21.15.87:0/1211873924 >> 172.21.15.87:0/1211873924 conn(0x7ff90009a9d0 msgr2=0x7ff90009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- 172.21.15.87:0/1211873924 shutdown_connections 2024-04-25T10:23:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- 172.21.15.87:0/1211873924 wait complete. 2024-04-25T10:23:33.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 Processor -- start 2024-04-25T10:23:33.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- start start 2024-04-25T10:23:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9000cf980 0x7ff9000c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9000d1830 0x7ff90015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff90015bc10 0x7ff9000b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000cd740 con 0x7ff9000cf980 2024-04-25T10:23:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000cd5c0 con 0x7ff90015bc10 2024-04-25T10:23:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff905800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff900168960 con 0x7ff9000d1830 2024-04-25T10:23:33.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9000cf980 0x7ff9000c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9000cf980 0x7ff9000c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39956/0 (socket says 172.21.15.87:39956) 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8fea00640 1 -- 172.21.15.87:0/483838161 learned_addr learned my addr 172.21.15.87:0/483838161 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8f6000640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9000d1830 0x7ff90015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8ff400640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff90015bc10 0x7ff9000b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8fea00640 1 -- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9000d1830 msgr2=0x7ff90015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8fea00640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9000d1830 0x7ff90015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8fea00640 1 -- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff90015bc10 msgr2=0x7ff9000b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.185+0000 7ff8fea00640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff90015bc10 0x7ff9000b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8fea00640 1 -- 172.21.15.87:0/483838161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000c7740 con 0x7ff9000cf980 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8fea00640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9000cf980 0x7ff9000c7200 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000e960 tx=0x7ff8f000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8f7400640 1 -- 172.21.15.87:0/483838161 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0017070 con 0x7ff9000cf980 2024-04-25T10:23:33.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8f7400640 1 -- 172.21.15.87:0/483838161 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f000ca00 con 0x7ff9000cf980 2024-04-25T10:23:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9000b6260 con 0x7ff9000cf980 2024-04-25T10:23:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8f7400640 1 -- 172.21.15.87:0/483838161 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000cdb0 con 0x7ff9000cf980 2024-04-25T10:23:33.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9000b6820 con 0x7ff9000cf980 2024-04-25T10:23:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90004ef30 con 0x7ff9000cf980 2024-04-25T10:23:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8f7400640 1 -- 172.21.15.87:0/483838161 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8f001a070 con 0x7ff9000cf980 2024-04-25T10:23:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8f7400640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8dc066f90 0x7ff8dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8f7400640 1 -- 172.21.15.87:0/483838161 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff8f0085060 con 0x7ff9000cf980 2024-04-25T10:23:33.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.189+0000 7ff8f6000640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8dc066f90 0x7ff8dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.193+0000 7ff8f6000640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8dc066f90 0x7ff8dc069450 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80ad000 tx=0x7ff8e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.193+0000 7ff8f7400640 1 -- 172.21.15.87:0/483838161 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8f0055960 con 0x7ff9000cf980 2024-04-25T10:23:33.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9000b0da0 con 0x7ff9000cf980 2024-04-25T10:23:33.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff8f7400640 1 -- 172.21.15.87:0/483838161 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8f0059a00 con 0x7ff9000cf980 2024-04-25T10:23:33.349 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:23:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8dc066f90 msgr2=0x7ff8dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8dc066f90 0x7ff8dc069450 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80ad000 tx=0x7ff8e8008040 comp rx=0 tx=0).stop 2024-04-25T10:23:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9000cf980 msgr2=0x7ff9000c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9000cf980 0x7ff9000c7200 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000e960 tx=0x7ff8f000ee20 comp rx=0 tx=0).stop 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 shutdown_connections 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8dc066f90 0x7ff8dc069450 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff90015bc10 0x7ff9000b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9000d1830 0x7ff90015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 --2- 172.21.15.87:0/483838161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9000cf980 0x7ff9000c7200 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.349+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 >> 172.21.15.87:0/483838161 conn(0x7ff90009a9d0 msgr2=0x7ff90006fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.353+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 shutdown_connections 2024-04-25T10:23:33.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.353+0000 7ff905800640 1 -- 172.21.15.87:0/483838161 wait complete. 2024-04-25T10:23:33.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:23:33.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f85600640 1 Processor -- start 2024-04-25T10:23:33.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f85600640 1 -- start start 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f85600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f80070350 0x7f7f8015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f85600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f800b31c0 0x7f7f800b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f85600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8015b080 con 0x7f7f800aef80 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f85600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8015adb0 con 0x7f7f800b31c0 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f85600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8015eb60 con 0x7f7f80070350 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f800b31c0 0x7f7f800b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7f800aef80 0x7f7f8015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53534/0 (socket says 172.21.15.87:53534) 2024-04-25T10:23:33.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7ea00640 1 -- 172.21.15.87:0/4164463841 learned_addr learned my addr 172.21.15.87:0/4164463841 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7e000640 1 --2- 172.21.15.87:0/4164463841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f80070350 0x7f7f8015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7f400640 1 -- 172.21.15.87:0/4164463841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f80070350 msgr2=0x7f7f8015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7f400640 1 --2- 172.21.15.87:0/4164463841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f80070350 0x7f7f8015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7f400640 1 -- 172.21.15.87:0/4164463841 >> v1:172.21.15.87:6789/0 conn(0x7f7f800aef80 legacy=0x7f7f8015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7f400640 1 -- 172.21.15.87:0/4164463841 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8015ee40 con 0x7f7f800b31c0 2024-04-25T10:23:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7e000640 1 --2- 172.21.15.87:0/4164463841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f80070350 0x7f7f8015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.509+0000 7f7f7d600640 1 -- 172.21.15.87:0/4164463841 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983682283 0 0) 0x7f7f8015b080 con 0x7f7f800aef80 2024-04-25T10:23:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f7f400640 1 --2- 172.21.15.87:0/4164463841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800b31c0 0x7f7f800b3610 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009d90 tx=0x7f7f7801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f539f0fd8a7589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f7d600640 1 -- 172.21.15.87:0/4164463841 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780b2070 con 0x7f7f800b31c0 2024-04-25T10:23:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f7d600640 1 -- 172.21.15.87:0/4164463841 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f780a5940 con 0x7f7f800b31c0 2024-04-25T10:23:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f7d600640 1 -- 172.21.15.87:0/4164463841 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780a5d10 con 0x7f7f800b31c0 2024-04-25T10:23:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 -- 172.21.15.87:0/4164463841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800b31c0 msgr2=0x7f7f800b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 --2- 172.21.15.87:0/4164463841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800b31c0 0x7f7f800b3610 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009d90 tx=0x7f7f7801c970 comp rx=0 tx=0).stop 2024-04-25T10:23:33.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 -- 172.21.15.87:0/4164463841 shutdown_connections 2024-04-25T10:23:33.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 --2- 172.21.15.87:0/4164463841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800b31c0 0x7f7f800b3610 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 --2- 172.21.15.87:0/4164463841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f80070350 0x7f7f8015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 -- 172.21.15.87:0/4164463841 >> 172.21.15.87:0/4164463841 conn(0x7f7f800669a0 msgr2=0x7f7f80066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:33.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 -- 172.21.15.87:0/4164463841 shutdown_connections 2024-04-25T10:23:33.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 -- 172.21.15.87:0/4164463841 wait complete. 2024-04-25T10:23:33.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.513+0000 7f7f85600640 1 Processor -- start 2024-04-25T10:23:33.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 -- start start 2024-04-25T10:23:33.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f80070350 0x7f7f8015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800aef80 0x7f7f8015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f800b31c0 0x7f7f80142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f801679a0 con 0x7f7f800aef80 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f80167820 con 0x7f7f80070350 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f80167b20 con 0x7f7f800b31c0 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800aef80 0x7f7f8015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800aef80 0x7f7f8015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39974/0 (socket says 172.21.15.87:39974) 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 -- 172.21.15.87:0/699970869 learned_addr learned my addr 172.21.15.87:0/699970869 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7f400640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f800b31c0 0x7f7f80142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7ea00640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f80070350 0x7f7f8015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 -- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f800b31c0 msgr2=0x7f7f80142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f800b31c0 0x7f7f80142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 -- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f80070350 msgr2=0x7f7f8015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f80070350 0x7f7f8015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 -- 172.21.15.87:0/699970869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f800af560 con 0x7f7f800aef80 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7ea00640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f80070350 0x7f7f8015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f7e000640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800aef80 0x7f7f8015d8b0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c7f0 tx=0x7f7f6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f76a00640 1 -- 172.21.15.87:0/699970869 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c007a30 con 0x7f7f800aef80 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f76a00640 1 -- 172.21.15.87:0/699970869 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c007bd0 con 0x7f7f800aef80 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f76a00640 1 -- 172.21.15.87:0/699970869 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0113f0 con 0x7f7f800aef80 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f80142830 con 0x7f7f800aef80 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f800afb00 con 0x7f7f800aef80 2024-04-25T10:23:33.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.517+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8004ef30 con 0x7f7f800aef80 2024-04-25T10:23:33.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.521+0000 7f7f76a00640 1 -- 172.21.15.87:0/699970869 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f6c011590 con 0x7f7f800aef80 2024-04-25T10:23:33.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.521+0000 7f7f76a00640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f60066f60 0x7f7f60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.521+0000 7f7f76a00640 1 -- 172.21.15.87:0/699970869 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7f6c085660 con 0x7f7f800aef80 2024-04-25T10:23:33.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.521+0000 7f7f7ea00640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f60066f60 0x7f7f60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.521+0000 7f7f7ea00640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f60066f60 0x7f7f60069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007fd0 tx=0x7f7f70007c20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.525+0000 7f7f76a00640 1 -- 172.21.15.87:0/699970869 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f6c015020 con 0x7f7f800aef80 2024-04-25T10:23:33.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f800b2fa0 con 0x7f7f800aef80 2024-04-25T10:23:33.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f76a00640 1 -- 172.21.15.87:0/699970869 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f6c0566a0 con 0x7f7f800aef80 2024-04-25T10:23:33.680 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:23:33.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f60066f60 msgr2=0x7f7f60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f60066f60 0x7f7f60069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007fd0 tx=0x7f7f70007c20 comp rx=0 tx=0).stop 2024-04-25T10:23:33.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800aef80 msgr2=0x7f7f8015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800aef80 0x7f7f8015d8b0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c7f0 tx=0x7f7f6c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 shutdown_connections 2024-04-25T10:23:33.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f60066f60 0x7f7f60069420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f800b31c0 0x7f7f80142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f800aef80 0x7f7f8015d8b0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 --2- 172.21.15.87:0/699970869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f80070350 0x7f7f8015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 >> 172.21.15.87:0/699970869 conn(0x7f7f800669a0 msgr2=0x7f7f800ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:33.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 shutdown_connections 2024-04-25T10:23:33.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.681+0000 7f7f85600640 1 -- 172.21.15.87:0/699970869 wait complete. 2024-04-25T10:23:33.702 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:33.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.837+0000 7f0290c00640 1 Processor -- start 2024-04-25T10:23:33.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0290c00640 1 -- start start 2024-04-25T10:23:33.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0290c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f028c15bc40 0x7f028c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0290c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f028c0cf930 0x7f028c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0290c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c0b2770 con 0x7f028c0cda10 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0290c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c06c670 con 0x7f028c15bc40 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0290c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c06c950 con 0x7f028c0cf930 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0289e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f028c15bc40 0x7f028c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0289e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f028c15bc40 0x7f028c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39988/0 (socket says 172.21.15.87:39988) 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0289e00640 1 -- 172.21.15.87:0/4199088430 learned_addr learned my addr 172.21.15.87:0/4199088430 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f028a800640 1 --2- 172.21.15.87:0/4199088430 >> v2:172.21.15.169:3300/0 conn(0x7f028c0cf930 0x7f028c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0289e00640 1 -- 172.21.15.87:0/4199088430 >> v2:172.21.15.169:3300/0 conn(0x7f028c0cf930 msgr2=0x7f028c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0289e00640 1 --2- 172.21.15.87:0/4199088430 >> v2:172.21.15.169:3300/0 conn(0x7f028c0cf930 0x7f028c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0289e00640 1 -- 172.21.15.87:0/4199088430 >> v1:172.21.15.169:6789/0 conn(0x7f028c0cda10 legacy=0x7f028c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:33.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.841+0000 7f0289e00640 1 -- 172.21.15.87:0/4199088430 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c06cc30 con 0x7f028c15bc40 2024-04-25T10:23:33.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0289e00640 1 --2- 172.21.15.87:0/4199088430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c06c090 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f028c0ae4c0 tx=0x7f027c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcadaafbe8d1234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0289400640 1 -- 172.21.15.87:0/4199088430 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0b0070 con 0x7f028c15bc40 2024-04-25T10:23:33.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0289400640 1 -- 172.21.15.87:0/4199088430 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c0ab600 con 0x7f028c15bc40 2024-04-25T10:23:33.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0289400640 1 -- 172.21.15.87:0/4199088430 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0ab9d0 con 0x7f028c15bc40 2024-04-25T10:23:33.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 -- 172.21.15.87:0/4199088430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 msgr2=0x7f028c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 --2- 172.21.15.87:0/4199088430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c06c090 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f028c0ae4c0 tx=0x7f027c01c450 comp rx=0 tx=0).stop 2024-04-25T10:23:33.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 -- 172.21.15.87:0/4199088430 shutdown_connections 2024-04-25T10:23:33.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 --2- 172.21.15.87:0/4199088430 >> v2:172.21.15.169:3300/0 conn(0x7f028c0cf930 0x7f028c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 --2- 172.21.15.87:0/4199088430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c06c090 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 -- 172.21.15.87:0/4199088430 >> 172.21.15.87:0/4199088430 conn(0x7f028c066b70 msgr2=0x7f028c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:33.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 -- 172.21.15.87:0/4199088430 shutdown_connections 2024-04-25T10:23:33.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.845+0000 7f0290c00640 1 -- 172.21.15.87:0/4199088430 wait complete. 2024-04-25T10:23:33.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 Processor -- start 2024-04-25T10:23:33.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 -- start start 2024-04-25T10:23:33.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f028c0cda10 0x7f028c13f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028c0cf930 0x7f028c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b3850 con 0x7f028c15bc40 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b36d0 con 0x7f028c0cda10 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b39d0 con 0x7f028c0cf930 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39994/0 (socket says 172.21.15.87:39994) 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 -- 172.21.15.87:0/66021500 learned_addr learned my addr 172.21.15.87:0/66021500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0289e00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f028c0cda10 0x7f028c13f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0281400640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028c0cf930 0x7f028c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 -- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028c0cf930 msgr2=0x7f028c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028c0cf930 0x7f028c159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 -- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f028c0cda10 msgr2=0x7f028c13f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f028c0cda10 0x7f028c13f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 -- 172.21.15.87:0/66021500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c0b2280 con 0x7f028c15bc40 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0281400640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028c0cf930 0x7f028c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f028a800640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c15a270 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f0284007e60 tx=0x7f028400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0282a00640 1 -- 172.21.15.87:0/66021500 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284014420 con 0x7f028c15bc40 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0282a00640 1 -- 172.21.15.87:0/66021500 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0284019070 con 0x7f028c15bc40 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0282a00640 1 -- 172.21.15.87:0/66021500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284014820 con 0x7f028c15bc40 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c0b1cc0 con 0x7f028c15bc40 2024-04-25T10:23:33.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c15c2c0 con 0x7f028c15bc40 2024-04-25T10:23:33.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.849+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c04ef30 con 0x7f028c15bc40 2024-04-25T10:23:33.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.853+0000 7f0282a00640 1 -- 172.21.15.87:0/66021500 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02840149c0 con 0x7f028c15bc40 2024-04-25T10:23:33.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.853+0000 7f0282a00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0268066f60 0x7f0268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:33.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.853+0000 7f0282a00640 1 -- 172.21.15.87:0/66021500 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0284085820 con 0x7f028c15bc40 2024-04-25T10:23:33.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.853+0000 7f0289e00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0268066f60 0x7f0268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:33.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.853+0000 7f0289e00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0268066f60 0x7f0268069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f027c0ad000 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:33.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:33.857+0000 7f0282a00640 1 -- 172.21.15.87:0/66021500 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0284056800 con 0x7f028c15bc40 2024-04-25T10:23:34.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.041+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f028c0cf4c0 con 0x7f028c15bc40 2024-04-25T10:23:34.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.041+0000 7f0282a00640 1 -- 172.21.15.87:0/66021500 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f0284056100 con 0x7f028c15bc40 2024-04-25T10:23:34.042 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:34.042 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":103,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:22:56.766091+0000","modified":"2024-04-25T10:23:27.501228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8755":{"gid":8755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6836/4075242775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4075242775},{"type":"v1","addr":"172.21.15.87:6836","nonce":4075242775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8758":{"gid":8758,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1025107360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1025107360},{"type":"v1","addr":"172.21.15.87:6837","nonce":1025107360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:23:34.042 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 103 2024-04-25T10:23:34.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0268066f60 msgr2=0x7f0268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0268066f60 0x7f0268069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f027c0ad000 tx=0x7f027c008040 comp rx=0 tx=0).stop 2024-04-25T10:23:34.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 msgr2=0x7f028c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c15a270 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f0284007e60 tx=0x7f028400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 shutdown_connections 2024-04-25T10:23:34.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0268066f60 0x7f0268069420 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f028c15bc40 0x7f028c15a270 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028c0cf930 0x7f028c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 --2- 172.21.15.87:0/66021500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f028c0cda10 0x7f028c13f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 >> 172.21.15.87:0/66021500 conn(0x7f028c066b70 msgr2=0x7f028c0ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 shutdown_connections 2024-04-25T10:23:34.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.045+0000 7f0290c00640 1 -- 172.21.15.87:0/66021500 wait complete. 2024-04-25T10:23:34.063 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:34.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.197+0000 7fc160c00640 1 Processor -- start 2024-04-25T10:23:34.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.197+0000 7fc160c00640 1 -- start start 2024-04-25T10:23:34.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc160c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc15c15bc40 0x7fc15c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc160c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c0cf930 0x7fc15c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc160c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0b2770 con 0x7fc15c0cda10 2024-04-25T10:23:34.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc160c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c06c670 con 0x7fc15c15bc40 2024-04-25T10:23:34.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc160c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c06c950 con 0x7fc15c0cf930 2024-04-25T10:23:34.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc159e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc15c15bc40 0x7fc15c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc159400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc15c0cda10 0x7fc15c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53548/0 (socket says 172.21.15.87:53548) 2024-04-25T10:23:34.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc159400640 1 -- 172.21.15.87:0/2490586031 learned_addr learned my addr 172.21.15.87:0/2490586031 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:34.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc15a800640 1 --2- 172.21.15.87:0/2490586031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c0cf930 0x7fc15c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc159e00640 1 -- 172.21.15.87:0/2490586031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c0cf930 msgr2=0x7fc15c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:34.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc159e00640 1 --2- 172.21.15.87:0/2490586031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c0cf930 0x7fc15c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc159e00640 1 -- 172.21.15.87:0/2490586031 >> v1:172.21.15.87:6789/0 conn(0x7fc15c0cda10 legacy=0x7fc15c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc159e00640 1 -- 172.21.15.87:0/2490586031 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c06cc30 con 0x7fc15c15bc40 2024-04-25T10:23:34.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.201+0000 7fc158a00640 1 -- 172.21.15.87:0/2490586031 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191264301 0 0) 0x7fc15c0b2770 con 0x7fc15c0cda10 2024-04-25T10:23:34.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc159e00640 1 --2- 172.21.15.87:0/2490586031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c15bc40 0x7fc15c06c090 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0097c0 tx=0x7fc14c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3439b41d1fd9a040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc158a00640 1 -- 172.21.15.87:0/2490586031 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0b0070 con 0x7fc15c15bc40 2024-04-25T10:23:34.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc158a00640 1 -- 172.21.15.87:0/2490586031 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c0ab600 con 0x7fc15c15bc40 2024-04-25T10:23:34.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc158a00640 1 -- 172.21.15.87:0/2490586031 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0ab9d0 con 0x7fc15c15bc40 2024-04-25T10:23:34.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 -- 172.21.15.87:0/2490586031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c15bc40 msgr2=0x7fc15c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 --2- 172.21.15.87:0/2490586031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c15bc40 0x7fc15c06c090 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0097c0 tx=0x7fc14c01c410 comp rx=0 tx=0).stop 2024-04-25T10:23:34.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 -- 172.21.15.87:0/2490586031 shutdown_connections 2024-04-25T10:23:34.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 --2- 172.21.15.87:0/2490586031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c0cf930 0x7fc15c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 --2- 172.21.15.87:0/2490586031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c15bc40 0x7fc15c06c090 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 -- 172.21.15.87:0/2490586031 >> 172.21.15.87:0/2490586031 conn(0x7fc15c066b70 msgr2=0x7fc15c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 -- 172.21.15.87:0/2490586031 shutdown_connections 2024-04-25T10:23:34.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 -- 172.21.15.87:0/2490586031 wait complete. 2024-04-25T10:23:34.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 Processor -- start 2024-04-25T10:23:34.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.205+0000 7fc160c00640 1 -- start start 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc15c0cda10 0x7fc15c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c0cf930 0x7fc15c14b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c15bc40 0x7fc15c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0b3890 con 0x7fc15c0cf930 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0b3710 con 0x7fc15c0cda10 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0b3a10 con 0x7fc15c15bc40 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c0cf930 0x7fc15c14b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c0cf930 0x7fc15c14b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40004/0 (socket says 172.21.15.87:40004) 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 -- 172.21.15.87:0/4073762245 learned_addr learned my addr 172.21.15.87:0/4073762245 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159e00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc15c0cda10 0x7fc15c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc15a800640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c15bc40 0x7fc15c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 -- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c15bc40 msgr2=0x7fc15c0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c15bc40 0x7fc15c0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 -- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc15c0cda10 msgr2=0x7fc15c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc15c0cda10 0x7fc15c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 -- 172.21.15.87:0/4073762245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c14bf20 con 0x7fc15c0cf930 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc159400640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c0cf930 0x7fc15c14b540 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c4b0 tx=0x7fc14800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc152000640 1 -- 172.21.15.87:0/4073762245 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148018070 con 0x7fc15c0cf930 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc152000640 1 -- 172.21.15.87:0/4073762245 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148007500 con 0x7fc15c0cf930 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c14c200 con 0x7fc15c0cf930 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc152000640 1 -- 172.21.15.87:0/4073762245 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148004a30 con 0x7fc15c0cf930 2024-04-25T10:23:34.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c14c7c0 con 0x7fc15c0cf930 2024-04-25T10:23:34.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c04ef30 con 0x7fc15c0cf930 2024-04-25T10:23:34.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.209+0000 7fc152000640 1 -- 172.21.15.87:0/4073762245 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc14801b070 con 0x7fc15c0cf930 2024-04-25T10:23:34.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.213+0000 7fc152000640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc13c066f90 0x7fc13c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.213+0000 7fc152000640 1 -- 172.21.15.87:0/4073762245 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc1480842e0 con 0x7fc15c0cf930 2024-04-25T10:23:34.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.213+0000 7fc159e00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc13c066f90 0x7fc13c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.213+0000 7fc159e00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc13c066f90 0x7fc13c069450 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0ad000 tx=0x7fc14c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.217+0000 7fc152000640 1 -- 172.21.15.87:0/4073762245 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc148054be0 con 0x7fc15c0cf930 2024-04-25T10:23:34.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.369+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc15c15f430 con 0x7fc15c0cf930 2024-04-25T10:23:34.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc152000640 1 -- 172.21.15.87:0/4073762245 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16717 (secure 0 0 0) 0x7fc148058c80 con 0x7fc15c0cf930 2024-04-25T10:23:34.371 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:34.371 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":140,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:07.453597+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:54.523910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:55.777113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.36","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:34.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc13c066f90 msgr2=0x7fc13c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc13c066f90 0x7fc13c069450 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0ad000 tx=0x7fc14c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:23:34.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c0cf930 msgr2=0x7fc15c14b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c0cf930 0x7fc15c14b540 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c4b0 tx=0x7fc14800c970 comp rx=0 tx=0).stop 2024-04-25T10:23:34.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 shutdown_connections 2024-04-25T10:23:34.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc13c066f90 0x7fc13c069450 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15c15bc40 0x7fc15c0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc15c0cf930 0x7fc15c14b540 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 --2- 172.21.15.87:0/4073762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc15c0cda10 0x7fc15c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 >> 172.21.15.87:0/4073762245 conn(0x7fc15c066b70 msgr2=0x7fc15c155320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 shutdown_connections 2024-04-25T10:23:34.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.373+0000 7fc160c00640 1 -- 172.21.15.87:0/4073762245 wait complete. 2024-04-25T10:23:34.393 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:23:34.393 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:34.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5be00640 1 Processor -- start 2024-04-25T10:23:34.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5be00640 1 -- start start 2024-04-25T10:23:34.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 0x7f1b5c0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c15f5e0 con 0x7f1b5c06d2a0 2024-04-25T10:23:34.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c15f8c0 con 0x7f1b5c1451d0 2024-04-25T10:23:34.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0b3040 con 0x7f1b5c0b26c0 2024-04-25T10:23:34.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1b5c06d2a0 0x7f1b5c06d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53556/0 (socket says 172.21.15.87:53556) 2024-04-25T10:23:34.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 -- 172.21.15.87:0/2255227387 learned_addr learned my addr 172.21.15.87:0/2255227387 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5a000640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 0x7f1b5c0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085273374 0 0) 0x7f1b5c15f5e0 con 0x7f1b5c06d2a0 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b40003530 con 0x7f1b5c06d2a0 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4168917842 0 0) 0x7f1b40003530 con 0x7f1b5c06d2a0 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 msgr2=0x7f1b5c0b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 0x7f1b5c0b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 >> v1:172.21.15.169:6789/0 conn(0x7f1b5c1451d0 legacy=0x7f1b5c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c15fba0 con 0x7f1b5c06d2a0 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f1b54001a00 con 0x7f1b5c06d2a0 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 >> v1:172.21.15.87:6789/0 conn(0x7f1b5c06d2a0 legacy=0x7f1b5c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 0x7f1b400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b40003f30 0x7f1b40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0b3040 con 0x7f1b40004d50 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4000fef0 con 0x7f1b40003f30 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400100f0 con 0x7f1b5c0b26c0 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5a000640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b40003f30 0x7f1b40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5aa00640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 0x7f1b400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 -- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 msgr2=0x7f1b400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c0b26c0 0x7f1b400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 -- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b40003f30 msgr2=0x7f1b40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b40003f30 0x7f1b40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.533+0000 7f1b5b400640 1 -- 172.21.15.87:0/2255227387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b40003530 con 0x7f1b40004d50 2024-04-25T10:23:34.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5b400640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b400051d0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400a800 tx=0x7f1b5401d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3670ef538750abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5400a450 con 0x7f1b40004d50 2024-04-25T10:23:34.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b540ad650 con 0x7f1b40004d50 2024-04-25T10:23:34.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b59600640 1 -- 172.21.15.87:0/2255227387 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540ada80 con 0x7f1b40004d50 2024-04-25T10:23:34.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5be00640 1 -- 172.21.15.87:0/2255227387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 msgr2=0x7f1b400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5be00640 1 --2- 172.21.15.87:0/2255227387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b400051d0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400a800 tx=0x7f1b5401d300 comp rx=0 tx=0).stop 2024-04-25T10:23:34.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5b400640 1 -- 172.21.15.87:0/2255227387 reap_dead start 2024-04-25T10:23:34.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5be00640 1 -- 172.21.15.87:0/2255227387 shutdown_connections 2024-04-25T10:23:34.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5be00640 1 -- 172.21.15.87:0/2255227387 >> 172.21.15.87:0/2255227387 conn(0x7f1b5c074310 msgr2=0x7f1b5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5be00640 1 -- 172.21.15.87:0/2255227387 shutdown_connections 2024-04-25T10:23:34.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.537+0000 7f1b5be00640 1 -- 172.21.15.87:0/2255227387 wait complete. 2024-04-25T10:23:34.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 Processor -- start 2024-04-25T10:23:34.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 -- start start 2024-04-25T10:23:34.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b5c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c15d4f0 0x7f1b5c13bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b5c15e4c0 0x7f1b5c15e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ce240 con 0x7f1b40004d50 2024-04-25T10:23:34.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c06ece0 con 0x7f1b5c15e4c0 2024-04-25T10:23:34.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c06ee60 con 0x7f1b5c15d4f0 2024-04-25T10:23:34.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b5c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b5c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40022/0 (socket says 172.21.15.87:40022) 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 -- 172.21.15.87:0/1380231892 learned_addr learned my addr 172.21.15.87:0/1380231892 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5b400640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b5c15e4c0 0x7f1b5c15e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5a000640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c15d4f0 0x7f1b5c13bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 -- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c15d4f0 msgr2=0x7f1b5c13bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c15d4f0 0x7f1b5c13bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 -- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b5c15e4c0 msgr2=0x7f1b5c15e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b5c15e4c0 0x7f1b5c15e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 -- 172.21.15.87:0/1380231892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c15c160 con 0x7f1b40004d50 2024-04-25T10:23:34.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5aa00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b5c15abd0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00f840 tx=0x7f1b4c00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b52a00640 1 -- 172.21.15.87:0/1380231892 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c01a070 con 0x7f1b40004d50 2024-04-25T10:23:34.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b52a00640 1 -- 172.21.15.87:0/1380231892 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c005110 con 0x7f1b40004d50 2024-04-25T10:23:34.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b52a00640 1 -- 172.21.15.87:0/1380231892 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c007030 con 0x7f1b40004d50 2024-04-25T10:23:34.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c15c440 con 0x7f1b40004d50 2024-04-25T10:23:34.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.541+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c15dcc0 con 0x7f1b40004d50 2024-04-25T10:23:34.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.545+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c075120 con 0x7f1b40004d50 2024-04-25T10:23:34.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.545+0000 7f1b52a00640 1 -- 172.21.15.87:0/1380231892 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b4c0071d0 con 0x7f1b40004d50 2024-04-25T10:23:34.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.545+0000 7f1b52a00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b3c066f60 0x7f1b3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.545+0000 7f1b52a00640 1 -- 172.21.15.87:0/1380231892 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1b4c084740 con 0x7f1b40004d50 2024-04-25T10:23:34.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.545+0000 7f1b5a000640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b3c066f60 0x7f1b3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.545+0000 7f1b5a000640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b3c066f60 0x7f1b3c069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c1f5f50 tx=0x7f1b48009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.549+0000 7f1b52a00640 1 -- 172.21.15.87:0/1380231892 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b4c0550d0 con 0x7f1b40004d50 2024-04-25T10:23:34.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b5c0cff20 con 0x7f1b40004d50 2024-04-25T10:23:34.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b52a00640 1 -- 172.21.15.87:0/1380231892 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f1b4c059170 con 0x7f1b40004d50 2024-04-25T10:23:34.735 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:34.735 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":103,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:22:56.766091+0000","modified":"2024-04-25T10:23:27.501228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8755":{"gid":8755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6836/4075242775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4075242775},{"type":"v1","addr":"172.21.15.87:6836","nonce":4075242775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8758":{"gid":8758,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1025107360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1025107360},{"type":"v1","addr":"172.21.15.87:6837","nonce":1025107360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:23:34.736 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 103 2024-04-25T10:23:34.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b3c066f60 msgr2=0x7f1b3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b3c066f60 0x7f1b3c069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c1f5f50 tx=0x7f1b48009190 comp rx=0 tx=0).stop 2024-04-25T10:23:34.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 msgr2=0x7f1b5c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b5c15abd0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00f840 tx=0x7f1b4c00fc40 comp rx=0 tx=0).stop 2024-04-25T10:23:34.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 shutdown_connections 2024-04-25T10:23:34.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1b3c066f60 0x7f1b3c069420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b5c15e4c0 0x7f1b5c15e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b5c15d4f0 0x7f1b5c13bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 --2- 172.21.15.87:0/1380231892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b40004d50 0x7f1b5c15abd0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 >> 172.21.15.87:0/1380231892 conn(0x7f1b5c074310 msgr2=0x7f1b5c1447f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 shutdown_connections 2024-04-25T10:23:34.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.737+0000 7f1b5be00640 1 -- 172.21.15.87:0/1380231892 wait complete. 2024-04-25T10:23:34.757 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11fdc00640 1 Processor -- start 2024-04-25T10:23:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11fdc00640 1 -- start start 2024-04-25T10:23:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11fdc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f11f815bc40 0x7f11f806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11fdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cf930 0x7f11f80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11fdc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80b2770 con 0x7f11f80cda10 2024-04-25T10:23:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11fdc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f806c670 con 0x7f11f815bc40 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11fdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f806c950 con 0x7f11f80cf930 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11f6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f11f815bc40 0x7f11f806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11f6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f11f815bc40 0x7f11f806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40026/0 (socket says 172.21.15.87:40026) 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11f6a00640 1 -- 172.21.15.87:0/4013679309 learned_addr learned my addr 172.21.15.87:0/4013679309 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.893+0000 7f11f7400640 1 --2- 172.21.15.87:0/4013679309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cf930 0x7f11f80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f6a00640 1 -- 172.21.15.87:0/4013679309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cf930 msgr2=0x7f11f80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f6a00640 1 --2- 172.21.15.87:0/4013679309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cf930 0x7f11f80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f6a00640 1 -- 172.21.15.87:0/4013679309 >> v1:172.21.15.169:6789/0 conn(0x7f11f80cda10 legacy=0x7f11f80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f6a00640 1 -- 172.21.15.87:0/4013679309 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f806cc30 con 0x7f11f815bc40 2024-04-25T10:23:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f7400640 1 --2- 172.21.15.87:0/4013679309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cf930 0x7f11f80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f6a00640 1 --2- 172.21.15.87:0/4013679309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f806c090 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009870 tx=0x7f11e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b14382e4c997d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f5600640 1 -- 172.21.15.87:0/4013679309 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00b0070 con 0x7f11f815bc40 2024-04-25T10:23:34.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f5600640 1 -- 172.21.15.87:0/4013679309 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e00ab600 con 0x7f11f815bc40 2024-04-25T10:23:34.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11f5600640 1 -- 172.21.15.87:0/4013679309 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00ab9d0 con 0x7f11f815bc40 2024-04-25T10:23:34.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11fdc00640 1 -- 172.21.15.87:0/4013679309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 msgr2=0x7f11f806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.897+0000 7f11fdc00640 1 --2- 172.21.15.87:0/4013679309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f806c090 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009870 tx=0x7f11e001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- 172.21.15.87:0/4013679309 shutdown_connections 2024-04-25T10:23:34.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 --2- 172.21.15.87:0/4013679309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cf930 0x7f11f80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 --2- 172.21.15.87:0/4013679309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f806c090 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- 172.21.15.87:0/4013679309 >> 172.21.15.87:0/4013679309 conn(0x7f11f8066b70 msgr2=0x7f11f8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:34.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- 172.21.15.87:0/4013679309 shutdown_connections 2024-04-25T10:23:34.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- 172.21.15.87:0/4013679309 wait complete. 2024-04-25T10:23:34.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 Processor -- start 2024-04-25T10:23:34.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- start start 2024-04-25T10:23:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cda10 0x7f11f8152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f80cf930 0x7f11f815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b3890 con 0x7f11f815bc40 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b3710 con 0x7f11f80cf930 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11fdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b3a10 con 0x7f11f80cda10 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f815deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40036/0 (socket says 172.21.15.87:40036) 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 -- 172.21.15.87:0/2571971072 learned_addr learned my addr 172.21.15.87:0/2571971072 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f6a00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cda10 0x7f11f8152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f6000640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f80cf930 0x7f11f815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 -- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cda10 msgr2=0x7f11f8152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cda10 0x7f11f8152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 -- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f80cf930 msgr2=0x7f11f815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f80cf930 0x7f11f815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 -- 172.21.15.87:0/2571971072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f815ed80 con 0x7f11f815bc40 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11f7400640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f815deb0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f11f0007df0 tx=0x7f11f000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.901+0000 7f11eea00640 1 -- 172.21.15.87:0/2571971072 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f001a070 con 0x7f11f815bc40 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11eea00640 1 -- 172.21.15.87:0/2571971072 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00039a0 con 0x7f11f815bc40 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f815f060 con 0x7f11f815bc40 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11eea00640 1 -- 172.21.15.87:0/2571971072 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0003d90 con 0x7f11f815bc40 2024-04-25T10:23:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80a9960 con 0x7f11f815bc40 2024-04-25T10:23:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f804ef30 con 0x7f11f815bc40 2024-04-25T10:23:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11eea00640 1 -- 172.21.15.87:0/2571971072 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11f001d070 con 0x7f11f815bc40 2024-04-25T10:23:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11eea00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11d4066f60 0x7f11d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11eea00640 1 -- 172.21.15.87:0/2571971072 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f11f00852d0 con 0x7f11f815bc40 2024-04-25T10:23:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11f6a00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11d4066f60 0x7f11d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.905+0000 7f11f6a00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11d4066f60 0x7f11d4069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f11e00ad000 tx=0x7f11e00a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:34.909+0000 7f11eea00640 1 -- 172.21.15.87:0/2571971072 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11f0010070 con 0x7f11f815bc40 2024-04-25T10:23:35.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11f815f400 con 0x7f11f815bc40 2024-04-25T10:23:35.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11eea00640 1 -- 172.21.15.87:0/2571971072 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f11f00562b0 con 0x7f11f815bc40 2024-04-25T10:23:35.096 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:35.096 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":103,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:22:56.766091+0000","modified":"2024-04-25T10:23:27.501228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8755":{"gid":8755,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6836/4075242775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4075242775},{"type":"v1","addr":"172.21.15.87:6836","nonce":4075242775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8758":{"gid":8758,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1025107360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1025107360},{"type":"v1","addr":"172.21.15.87:6837","nonce":1025107360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:23:35.096 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 103 2024-04-25T10:23:35.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11d4066f60 msgr2=0x7f11d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11d4066f60 0x7f11d4069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f11e00ad000 tx=0x7f11e00a6000 comp rx=0 tx=0).stop 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 msgr2=0x7f11f815deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f815deb0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f11f0007df0 tx=0x7f11f000d4a0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 shutdown_connections 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11d4066f60 0x7f11d4069420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11f815bc40 0x7f11f815deb0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f80cf930 0x7f11f815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 --2- 172.21.15.87:0/2571971072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11f80cda10 0x7f11f8152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 >> 172.21.15.87:0/2571971072 conn(0x7f11f8066b70 msgr2=0x7f11f8155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 shutdown_connections 2024-04-25T10:23:35.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.097+0000 7f11fdc00640 1 -- 172.21.15.87:0/2571971072 wait complete. 2024-04-25T10:23:35.116 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:35.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff623800640 1 Processor -- start 2024-04-25T10:23:35.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff623800640 1 -- start start 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff623800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff61c1550b0 0x7ff61c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff623800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c071050 con 0x7ff61c15b910 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff623800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c15d040 con 0x7ff61c0cfe70 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff623800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c15bd80 con 0x7ff61c1550b0 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff620e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff61c1550b0 0x7ff61c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff621800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff61c15b910 0x7ff61c0b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53570/0 (socket says 172.21.15.87:53570) 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff620e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff61c1550b0 0x7ff61c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40052/0 (socket says 172.21.15.87:40052) 2024-04-25T10:23:35.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff621800640 1 -- 172.21.15.87:0/3250559724 learned_addr learned my addr 172.21.15.87:0/3250559724 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:35.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff620e00640 1 -- 172.21.15.87:0/3250559724 >> v1:172.21.15.169:6789/0 conn(0x7ff61c0cfe70 legacy=0x7ff61c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:35.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff620e00640 1 -- 172.21.15.87:0/3250559724 >> v1:172.21.15.87:6789/0 conn(0x7ff61c15b910 legacy=0x7ff61c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff620e00640 1 -- 172.21.15.87:0/3250559724 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0712f0 con 0x7ff61c1550b0 2024-04-25T10:23:35.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.253+0000 7ff61be00640 1 -- 172.21.15.87:0/3250559724 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982596400 0 0) 0x7ff61c071050 con 0x7ff61c15b910 2024-04-25T10:23:35.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff620e00640 1 --2- 172.21.15.87:0/3250559724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c0cf930 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00ba90 tx=0x7ff60c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdfae4a2edba207 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff61be00640 1 -- 172.21.15.87:0/3250559724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0b4070 con 0x7ff61c1550b0 2024-04-25T10:23:35.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff61be00640 1 -- 172.21.15.87:0/3250559724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c0a79f0 con 0x7ff61c1550b0 2024-04-25T10:23:35.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff61be00640 1 -- 172.21.15.87:0/3250559724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0a7dc0 con 0x7ff61c1550b0 2024-04-25T10:23:35.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff623800640 1 -- 172.21.15.87:0/3250559724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 msgr2=0x7ff61c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff623800640 1 --2- 172.21.15.87:0/3250559724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c0cf930 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00ba90 tx=0x7ff60c01eb30 comp rx=0 tx=0).stop 2024-04-25T10:23:35.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff623800640 1 -- 172.21.15.87:0/3250559724 shutdown_connections 2024-04-25T10:23:35.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff623800640 1 --2- 172.21.15.87:0/3250559724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c0cf930 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff623800640 1 -- 172.21.15.87:0/3250559724 >> 172.21.15.87:0/3250559724 conn(0x7ff61c066b70 msgr2=0x7ff61c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff623800640 1 -- 172.21.15.87:0/3250559724 shutdown_connections 2024-04-25T10:23:35.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.257+0000 7ff623800640 1 -- 172.21.15.87:0/3250559724 wait complete. 2024-04-25T10:23:35.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 Processor -- start 2024-04-25T10:23:35.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 -- start start 2024-04-25T10:23:35.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff61c0cfe70 0x7ff61c13b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff61c15b910 0x7ff61c1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c06c090 con 0x7ff61c1550b0 2024-04-25T10:23:35.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0720b0 con 0x7ff61c15b910 2024-04-25T10:23:35.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c06c210 con 0x7ff61c0cfe70 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40068/0 (socket says 172.21.15.87:40068) 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 -- 172.21.15.87:0/3343013779 learned_addr learned my addr 172.21.15.87:0/3343013779 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff621800640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff61c15b910 0x7ff61c1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff620e00640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff61c0cfe70 0x7ff61c13b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 -- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff61c0cfe70 msgr2=0x7ff61c13b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff61c0cfe70 0x7ff61c13b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 -- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff61c15b910 msgr2=0x7ff61c1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff61c15b910 0x7ff61c1532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 -- 172.21.15.87:0/3343013779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c13b610 con 0x7ff61c1550b0 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff613e00640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c157cd0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff60400c450 tx=0x7ff60400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff619600640 1 -- 172.21.15.87:0/3343013779 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604012830 con 0x7ff61c1550b0 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff619600640 1 -- 172.21.15.87:0/3343013779 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604017070 con 0x7ff61c1550b0 2024-04-25T10:23:35.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c0ce4f0 con 0x7ff61c1550b0 2024-04-25T10:23:35.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff619600640 1 -- 172.21.15.87:0/3343013779 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604012c30 con 0x7ff61c1550b0 2024-04-25T10:23:35.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c0cdf30 con 0x7ff61c1550b0 2024-04-25T10:23:35.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.261+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c04ef30 con 0x7ff61c1550b0 2024-04-25T10:23:35.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.265+0000 7ff619600640 1 -- 172.21.15.87:0/3343013779 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff60401c460 con 0x7ff61c1550b0 2024-04-25T10:23:35.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.265+0000 7ff619600640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5f8066f90 0x7ff5f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.265+0000 7ff619600640 1 -- 172.21.15.87:0/3343013779 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff604084d30 con 0x7ff61c1550b0 2024-04-25T10:23:35.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.265+0000 7ff620e00640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5f8066f90 0x7ff5f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.265+0000 7ff620e00640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5f8066f90 0x7ff5f8069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b1000 tx=0x7ff60c0a9070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.269+0000 7ff619600640 1 -- 172.21.15.87:0/3343013779 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff604055db0 con 0x7ff61c1550b0 2024-04-25T10:23:35.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff61c158210 con 0x7ff61c1550b0 2024-04-25T10:23:35.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff619600640 1 -- 172.21.15.87:0/3343013779 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16717 (secure 0 0 0) 0x7ff6040556b0 con 0x7ff61c1550b0 2024-04-25T10:23:35.424 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:35.424 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":140,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:07.453597+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:54.523910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:55.777113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.36","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5f8066f90 msgr2=0x7ff5f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5f8066f90 0x7ff5f8069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b1000 tx=0x7ff60c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 msgr2=0x7ff61c157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c157cd0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff60400c450 tx=0x7ff60400c910 comp rx=0 tx=0).stop 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 shutdown_connections 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff5f8066f90 0x7ff5f8069450 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff61c15b910 0x7ff61c1532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c1550b0 0x7ff61c157cd0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 --2- 172.21.15.87:0/3343013779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff61c0cfe70 0x7ff61c13b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 >> 172.21.15.87:0/3343013779 conn(0x7ff61c066b70 msgr2=0x7ff61c06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 shutdown_connections 2024-04-25T10:23:35.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.425+0000 7ff623800640 1 -- 172.21.15.87:0/3343013779 wait complete. 2024-04-25T10:23:35.445 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:23:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 Processor -- start 2024-04-25T10:23:35.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 -- start start 2024-04-25T10:23:35.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f283015bc40 0x7f283006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f28300cda10 0x7f28300b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cf930 0x7f28300cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28300b2770 con 0x7f283015bc40 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283006c670 con 0x7f28300cda10 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2836200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283006c950 con 0x7f28300cf930 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f283015bc40 0x7f283006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f283015bc40 0x7f283006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40084/0 (socket says 172.21.15.87:40084) 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 -- 172.21.15.87:0/3598131484 learned_addr learned my addr 172.21.15.87:0/3598131484 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f2826a00640 1 --2- 172.21.15.87:0/3598131484 >> v2:172.21.15.169:3300/0 conn(0x7f28300cda10 0x7f28300b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282fe00640 1 --2- 172.21.15.87:0/3598131484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cf930 0x7f28300cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 -- 172.21.15.87:0/3598131484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cf930 msgr2=0x7f28300cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:35.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 --2- 172.21.15.87:0/3598131484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cf930 0x7f28300cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 -- 172.21.15.87:0/3598131484 >> v2:172.21.15.169:3300/0 conn(0x7f28300cda10 msgr2=0x7f28300b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:35.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 --2- 172.21.15.87:0/3598131484 >> v2:172.21.15.169:3300/0 conn(0x7f28300cda10 0x7f28300b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.585+0000 7f282f400640 1 -- 172.21.15.87:0/3598131484 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283006cc30 con 0x7f283015bc40 2024-04-25T10:23:35.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f282f400640 1 --2- 172.21.15.87:0/3598131484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f283015bc40 0x7f283006c090 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f28300726f0 tx=0x7f282001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f44758fb218d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f282ea00640 1 -- 172.21.15.87:0/3598131484 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28200b0070 con 0x7f283015bc40 2024-04-25T10:23:35.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f282ea00640 1 -- 172.21.15.87:0/3598131484 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28200ab600 con 0x7f283015bc40 2024-04-25T10:23:35.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f282ea00640 1 -- 172.21.15.87:0/3598131484 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28200ab9d0 con 0x7f283015bc40 2024-04-25T10:23:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 -- 172.21.15.87:0/3598131484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f283015bc40 msgr2=0x7f283006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 --2- 172.21.15.87:0/3598131484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f283015bc40 0x7f283006c090 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f28300726f0 tx=0x7f282001c520 comp rx=0 tx=0).stop 2024-04-25T10:23:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 -- 172.21.15.87:0/3598131484 shutdown_connections 2024-04-25T10:23:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 --2- 172.21.15.87:0/3598131484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cf930 0x7f28300cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 --2- 172.21.15.87:0/3598131484 >> v2:172.21.15.169:3300/0 conn(0x7f28300cda10 0x7f28300b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 --2- 172.21.15.87:0/3598131484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f283015bc40 0x7f283006c090 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 -- 172.21.15.87:0/3598131484 >> 172.21.15.87:0/3598131484 conn(0x7f2830066b70 msgr2=0x7f2830066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 -- 172.21.15.87:0/3598131484 shutdown_connections 2024-04-25T10:23:35.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.589+0000 7f2836200640 1 -- 172.21.15.87:0/3598131484 wait complete. 2024-04-25T10:23:35.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 Processor -- start 2024-04-25T10:23:35.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 -- start start 2024-04-25T10:23:35.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cda10 0x7f28301fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28300cf930 0x7f28301462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f283015bc40 0x7f283015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2830160270 con 0x7f28300cf930 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28301600f0 con 0x7f283015bc40 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28301603f0 con 0x7f28300cda10 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28300cf930 0x7f28301462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28300cf930 0x7f28301462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40100/0 (socket says 172.21.15.87:40100) 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 -- 172.21.15.87:0/430890840 learned_addr learned my addr 172.21.15.87:0/430890840 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f282f400640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cda10 0x7f28301fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f282fe00640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f283015bc40 0x7f283015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 -- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cda10 msgr2=0x7f28301fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cda10 0x7f28301fb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 -- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f283015bc40 msgr2=0x7f283015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f283015bc40 0x7f283015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 -- 172.21.15.87:0/430890840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2830146ca0 con 0x7f28300cf930 2024-04-25T10:23:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f282fe00640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f283015bc40 0x7f283015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2826a00640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28300cf930 0x7f28301462f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f281800e870 tx=0x7f281800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2827e00640 1 -- 172.21.15.87:0/430890840 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818017070 con 0x7f28300cf930 2024-04-25T10:23:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2827e00640 1 -- 172.21.15.87:0/430890840 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281800ee60 con 0x7f28300cf930 2024-04-25T10:23:35.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2827e00640 1 -- 172.21.15.87:0/430890840 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281800cb80 con 0x7f28300cf930 2024-04-25T10:23:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2830155c90 con 0x7f28300cf930 2024-04-25T10:23:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2830142ea0 con 0x7f28300cf930 2024-04-25T10:23:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.593+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28301429e0 con 0x7f28300cf930 2024-04-25T10:23:35.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.597+0000 7f2827e00640 1 -- 172.21.15.87:0/430890840 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f281801a070 con 0x7f28300cf930 2024-04-25T10:23:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.597+0000 7f2827e00640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f280c066f60 0x7f280c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.597+0000 7f2827e00640 1 -- 172.21.15.87:0/430890840 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2818084ed0 con 0x7f28300cf930 2024-04-25T10:23:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.597+0000 7f282f400640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f280c066f60 0x7f280c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.597+0000 7f282f400640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f280c066f60 0x7f280c069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f28200ad000 tx=0x7f28200a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.601+0000 7f2827e00640 1 -- 172.21.15.87:0/430890840 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28301429e0 con 0x7f28300cf930 2024-04-25T10:23:35.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.785+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f283004ef30 con 0x7f28300cf930 2024-04-25T10:23:35.786 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:23:35.789+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:23:35.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.789+0000 7f2827e00640 1 -- 172.21.15.87:0/430890840 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7f28180557d0 con 0x7f28300cf930 2024-04-25T10:23:35.789 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:23:35.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f280c066f60 msgr2=0x7f280c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f280c066f60 0x7f280c069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f28200ad000 tx=0x7f28200a6000 comp rx=0 tx=0).stop 2024-04-25T10:23:35.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28300cf930 msgr2=0x7f28301462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28300cf930 0x7f28301462f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f281800e870 tx=0x7f281800ed30 comp rx=0 tx=0).stop 2024-04-25T10:23:35.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 shutdown_connections 2024-04-25T10:23:35.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f280c066f60 0x7f280c069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f283015bc40 0x7f283015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28300cf930 0x7f28301462f0 secure :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f281800e870 tx=0x7f281800ed30 comp rx=0 tx=0).stop 2024-04-25T10:23:35.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 --2- 172.21.15.87:0/430890840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28300cda10 0x7f28301fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 >> 172.21.15.87:0/430890840 conn(0x7f2830066b70 msgr2=0x7f28300cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 shutdown_connections 2024-04-25T10:23:35.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.793+0000 7f2836200640 1 -- 172.21.15.87:0/430890840 wait complete. 2024-04-25T10:23:35.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:23:35.859 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:23:35.861 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:23:35.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.957+0000 7fbb94800640 1 Processor -- start 2024-04-25T10:23:35.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.957+0000 7fbb94800640 1 -- start start 2024-04-25T10:23:35.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb94800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c0b38d0 0x7fbb8c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb94800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c15b080 con 0x7fbb8c070350 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb94800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c15adb0 con 0x7fbb8c0aef80 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb94800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c15eb60 con 0x7fbb8c0b38d0 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbb8c070350 0x7fbb8c0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53586/0 (socket says 172.21.15.87:53586) 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 -- 172.21.15.87:0/3417136817 learned_addr learned my addr 172.21.15.87:0/3417136817 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940117676 0 0) 0x7fbb8c15b080 con 0x7fbb8c070350 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb74003530 con 0x7fbb8c070350 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb92800640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c0b38d0 0x7fbb8c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458758712 0 0) 0x7fbb74003530 con 0x7fbb8c070350 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c0b38d0 msgr2=0x7fbb8c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c0b38d0 0x7fbb8c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 >> v1:172.21.15.169:6789/0 conn(0x7fbb8c0aef80 legacy=0x7fbb8c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c15ee40 con 0x7fbb8c070350 2024-04-25T10:23:35.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fbb7c001680 con 0x7fbb8c070350 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 >> v1:172.21.15.87:6789/0 conn(0x7fbb8c070350 legacy=0x7fbb8c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb740039f0 0x7fbb74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb740043a0 0x7fbb74004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb74005070 0x7fbb7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c15eb60 con 0x7fbb740043a0 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7400bf10 con 0x7fbb74005070 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7400c0e0 con 0x7fbb740039f0 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb740043a0 0x7fbb74004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91e00640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb740039f0 0x7fbb74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb92800640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb74005070 0x7fbb7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 -- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb740039f0 msgr2=0x7fbb74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:35.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb740039f0 0x7fbb74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 -- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb74005070 msgr2=0x7fbb7400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:35.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb74005070 0x7fbb7400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb91400640 1 -- 172.21.15.87:0/3417136817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74003530 con 0x7fbb740043a0 2024-04-25T10:23:35.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.961+0000 7fbb92800640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb74005070 0x7fbb7400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:35.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb91400640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb740043a0 0x7fbb74004870 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c190 tx=0x7fbb7c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ab854460c6e714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c007c70 con 0x7fbb740043a0 2024-04-25T10:23:35.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c005ae0 con 0x7fbb740043a0 2024-04-25T10:23:35.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb90a00640 1 -- 172.21.15.87:0/3417136817 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c007070 con 0x7fbb740043a0 2024-04-25T10:23:35.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 -- 172.21.15.87:0/3417136817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb740043a0 msgr2=0x7fbb74004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 --2- 172.21.15.87:0/3417136817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb740043a0 0x7fbb74004870 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c190 tx=0x7fbb7c01f330 comp rx=0 tx=0).stop 2024-04-25T10:23:35.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb92800640 1 -- 172.21.15.87:0/3417136817 reap_dead start 2024-04-25T10:23:35.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 -- 172.21.15.87:0/3417136817 shutdown_connections 2024-04-25T10:23:35.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 -- 172.21.15.87:0/3417136817 >> 172.21.15.87:0/3417136817 conn(0x7fbb8c0669a0 msgr2=0x7fbb8c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:35.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 -- 172.21.15.87:0/3417136817 shutdown_connections 2024-04-25T10:23:35.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 -- 172.21.15.87:0/3417136817 wait complete. 2024-04-25T10:23:35.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 Processor -- start 2024-04-25T10:23:35.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.965+0000 7fbb94800640 1 -- start start 2024-04-25T10:23:35.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb740043a0 0x7fbb8c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0b0f80 0x7fbb8c0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c14a6b0 0x7fbb8c14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c15bc90 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c071a00 con 0x7fbb740043a0 2024-04-25T10:23:35.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c15e480 con 0x7fbb8c14a6b0 2024-04-25T10:23:35.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0b0f80 0x7fbb8c0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0b0f80 0x7fbb8c0b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40142/0 (socket says 172.21.15.87:40142) 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 -- 172.21.15.87:0/3326350638 learned_addr learned my addr 172.21.15.87:0/3326350638 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91e00640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb740043a0 0x7fbb8c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb92800640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c14a6b0 0x7fbb8c14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 -- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c14a6b0 msgr2=0x7fbb8c14ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c14a6b0 0x7fbb8c14ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 -- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb740043a0 msgr2=0x7fbb8c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb740043a0 0x7fbb8c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 -- 172.21.15.87:0/3326350638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c14b540 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb91400640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0b0f80 0x7fbb8c0b1400 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c190 tx=0x7fbb74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb86000640 1 -- 172.21.15.87:0/3326350638 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0047f0 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb86000640 1 -- 172.21.15.87:0/3326350638 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c004990 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c0b20b0 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb86000640 1 -- 172.21.15.87:0/3326350638 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c004d80 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c0b1dd0 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c04ef30 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.969+0000 7fbb86000640 1 -- 172.21.15.87:0/3326350638 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb7c007070 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.973+0000 7fbb86000640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb70066f90 0x7fbb70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:35.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.973+0000 7fbb86000640 1 -- 172.21.15.87:0/3326350638 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fbb7c122710 con 0x7fbb8c0b0f80 2024-04-25T10:23:35.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.973+0000 7fbb91e00640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb70066f90 0x7fbb70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:35.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.973+0000 7fbb91e00640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb70066f90 0x7fbb70069450 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbb80007fd0 tx=0x7fbb80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:35.977+0000 7fbb86000640 1 -- 172.21.15.87:0/3326350638 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb7c0b7070 con 0x7fbb8c0b0f80 2024-04-25T10:23:36.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.161+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb8c0b4120 con 0x7fbb8c0b0f80 2024-04-25T10:23:36.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.793+0000 7fbb86000640 1 -- 172.21.15.87:0/3326350638 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb7c019040 con 0x7fbb8c0b0f80 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.793+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb70066f90 msgr2=0x7fbb70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.793+0000 7fbb94800640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb70066f90 0x7fbb70069450 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbb80007fd0 tx=0x7fbb80009040 comp rx=0 tx=0).stop 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.793+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0b0f80 msgr2=0x7fbb8c0b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.793+0000 7fbb94800640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0b0f80 0x7fbb8c0b1400 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c190 tx=0x7fbb74005840 comp rx=0 tx=0).stop 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 shutdown_connections 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb70066f90 0x7fbb70069450 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c14a6b0 0x7fbb8c14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0b0f80 0x7fbb8c0b1400 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 --2- 172.21.15.87:0/3326350638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb740043a0 0x7fbb8c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:36.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 >> 172.21.15.87:0/3326350638 conn(0x7fbb8c0669a0 msgr2=0x7fbb8c15d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:36.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 shutdown_connections 2024-04-25T10:23:36.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:36.797+0000 7fbb94800640 1 -- 172.21.15.87:0/3326350638 wait complete. 2024-04-25T10:23:36.818 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:23:37.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeb4a00640 1 Processor -- start 2024-04-25T10:23:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeb4a00640 1 -- start start 2024-04-25T10:23:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb015e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeb4a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feeb00669a0 0x7feeb0066df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeb4a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb01478a0 con 0x7feeb01474c0 2024-04-25T10:23:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeb4a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb0067330 con 0x7feeb00669a0 2024-04-25T10:23:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeb4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0161960 con 0x7feeb015e1f0 2024-04-25T10:23:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeae800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feeb00669a0 0x7feeb0066df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feea7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeae800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feeb00669a0 0x7feeb0066df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:55880/0 (socket says 172.21.15.87:55880) 2024-04-25T10:23:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeade00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7feeb01474c0 0x7feeb0162140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:42302/0 (socket says 172.21.15.87:42302) 2024-04-25T10:23:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feeae800640 1 -- 172.21.15.87:0/1278291230 learned_addr learned my addr 172.21.15.87:0/1278291230 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feead400640 1 -- 172.21.15.87:0/1278291230 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329498356 0 0) 0x7feea0001120 con 0x7feeb01474c0 2024-04-25T10:23:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feea7e00640 1 -- 172.21.15.87:0/1278291230 >> v2:172.21.15.169:3300/0 conn(0x7feeb00669a0 msgr2=0x7feeb0066df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feea7e00640 1 --2- 172.21.15.87:0/1278291230 >> v2:172.21.15.169:3300/0 conn(0x7feeb00669a0 0x7feeb0066df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feea7e00640 1 -- 172.21.15.87:0/1278291230 >> v1:172.21.15.169:6789/0 conn(0x7feeb01474c0 legacy=0x7feeb0162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.005+0000 7feea7e00640 1 -- 172.21.15.87:0/1278291230 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb015ecb0 con 0x7feeb015e1f0 2024-04-25T10:23:37.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feea7e00640 1 --2- 172.21.15.87:0/1278291230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb015e670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee980097c0 tx=0x7fee9801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d063d5c153471aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feead400640 1 -- 172.21.15.87:0/1278291230 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980ab610 con 0x7feeb015e1f0 2024-04-25T10:23:37.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feead400640 1 -- 172.21.15.87:0/1278291230 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee980b0070 con 0x7feeb015e1f0 2024-04-25T10:23:37.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feead400640 1 -- 172.21.15.87:0/1278291230 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980ab9d0 con 0x7feeb015e1f0 2024-04-25T10:23:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 -- 172.21.15.87:0/1278291230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 msgr2=0x7feeb015e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 --2- 172.21.15.87:0/1278291230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb015e670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee980097c0 tx=0x7fee9801c410 comp rx=0 tx=0).stop 2024-04-25T10:23:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 -- 172.21.15.87:0/1278291230 shutdown_connections 2024-04-25T10:23:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 --2- 172.21.15.87:0/1278291230 >> v2:172.21.15.169:3300/0 conn(0x7feeb00669a0 0x7feeb0066df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 --2- 172.21.15.87:0/1278291230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb015e670 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 -- 172.21.15.87:0/1278291230 >> 172.21.15.87:0/1278291230 conn(0x7feeb00b2560 msgr2=0x7feeb00b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 -- 172.21.15.87:0/1278291230 shutdown_connections 2024-04-25T10:23:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.009+0000 7feeb4a00640 1 -- 172.21.15.87:0/1278291230 wait complete. 2024-04-25T10:23:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 Processor -- start 2024-04-25T10:23:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 -- start start 2024-04-25T10:23:37.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feeb00669a0 0x7feeb0155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feeb01474c0 0x7feeb0144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb0145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb006de80 con 0x7feeb01474c0 2024-04-25T10:23:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb006dd00 con 0x7feeb00669a0 2024-04-25T10:23:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb006e000 con 0x7feeb015e1f0 2024-04-25T10:23:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feeb01474c0 0x7feeb0144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feeb01474c0 0x7feeb0144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40152/0 (socket says 172.21.15.87:40152) 2024-04-25T10:23:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 -- 172.21.15.87:0/762065072 learned_addr learned my addr 172.21.15.87:0/762065072 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeade00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feeb00669a0 0x7feeb0155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeae800640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb0145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 -- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 msgr2=0x7feeb0145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb0145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 -- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feeb00669a0 msgr2=0x7feeb0155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feeb00669a0 0x7feeb0155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 -- 172.21.15.87:0/762065072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0141f60 con 0x7feeb01474c0 2024-04-25T10:23:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea7e00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feeb01474c0 0x7feeb0144ac0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fee980ad040 tx=0x7fee980191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea6000640 1 -- 172.21.15.87:0/762065072 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98004220 con 0x7feeb01474c0 2024-04-25T10:23:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea6000640 1 -- 172.21.15.87:0/762065072 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee980043c0 con 0x7feeb01474c0 2024-04-25T10:23:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feea6000640 1 -- 172.21.15.87:0/762065072 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980ab9e0 con 0x7feeb01474c0 2024-04-25T10:23:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb0142240 con 0x7feeb01474c0 2024-04-25T10:23:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.013+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb0145ee0 con 0x7feeb01474c0 2024-04-25T10:23:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.017+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb004ef30 con 0x7feeb01474c0 2024-04-25T10:23:37.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.017+0000 7feea6000640 1 -- 172.21.15.87:0/762065072 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee980abb80 con 0x7feeb01474c0 2024-04-25T10:23:37.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.017+0000 7feea6000640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fee8c066f60 0x7fee8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.017+0000 7feea6000640 1 -- 172.21.15.87:0/762065072 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fee9811ef80 con 0x7feeb01474c0 2024-04-25T10:23:37.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.017+0000 7feeade00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fee8c066f60 0x7fee8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.017+0000 7feeade00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fee8c066f60 0x7fee8c069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7feea0007fd0 tx=0x7feea0007980 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.021+0000 7feea6000640 1 -- 172.21.15.87:0/762065072 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee980b9070 con 0x7feeb01474c0 2024-04-25T10:23:37.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.177+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feeb0071e00 con 0x7feeb01474c0 2024-04-25T10:23:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feea6000640 1 -- 172.21.15.87:0/762065072 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) v1 ==== 181+0+0 (secure 0 0 0) 0x7fee980a8050 con 0x7feeb01474c0 2024-04-25T10:23:37.799 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:23:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fee8c066f60 msgr2=0x7fee8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fee8c066f60 0x7fee8c069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7feea0007fd0 tx=0x7feea0007980 comp rx=0 tx=0).stop 2024-04-25T10:23:37.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feeb01474c0 msgr2=0x7feeb0144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feeb01474c0 0x7feeb0144ac0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fee980ad040 tx=0x7fee980191b0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 shutdown_connections 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fee8c066f60 0x7fee8c069420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feeb015e1f0 0x7feeb0145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feeb01474c0 0x7feeb0144ac0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 --2- 172.21.15.87:0/762065072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feeb00669a0 0x7feeb0155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 >> 172.21.15.87:0/762065072 conn(0x7feeb00b2560 msgr2=0x7feeb00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 shutdown_connections 2024-04-25T10:23:37.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.801+0000 7feeb4a00640 1 -- 172.21.15.87:0/762065072 wait complete. 2024-04-25T10:23:37.824 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:23:37.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e42800640 1 Processor -- start 2024-04-25T10:23:37.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e42800640 1 -- start start 2024-04-25T10:23:37.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e42800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3e3c06b6a0 0x7f3e3c06ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e42800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e3c0b3fc0 0x7f3e3c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e42800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3c0b2f50 con 0x7f3e3c0b26b0 2024-04-25T10:23:37.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e42800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c06bfc0 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e42800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c06b040 con 0x7f3e3c0b3fc0 2024-04-25T10:23:37.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3e3c06b6a0 0x7f3e3c06ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3e3c06b6a0 0x7f3e3c06ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53118/0 (socket says 172.21.15.87:53118) 2024-04-25T10:23:37.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3b400640 1 -- 172.21.15.87:0/637716625 learned_addr learned my addr 172.21.15.87:0/637716625 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:37.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3b400640 1 -- 172.21.15.87:0/637716625 >> v2:172.21.15.169:3300/0 conn(0x7f3e3c0b3fc0 msgr2=0x7f3e3c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:37.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3be00640 1 --2- 172.21.15.87:0/637716625 >> v2:172.21.15.169:3300/0 conn(0x7f3e3c0b3fc0 0x7f3e3c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3b400640 1 --2- 172.21.15.87:0/637716625 >> v2:172.21.15.169:3300/0 conn(0x7f3e3c0b3fc0 0x7f3e3c0b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3b400640 1 -- 172.21.15.87:0/637716625 >> v1:172.21.15.169:6789/0 conn(0x7f3e3c0b26b0 legacy=0x7f3e3c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:37.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3b400640 1 -- 172.21.15.87:0/637716625 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c15f360 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.965+0000 7f3e3be00640 1 --2- 172.21.15.87:0/637716625 >> v2:172.21.15.169:3300/0 conn(0x7f3e3c0b3fc0 0x7f3e3c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e3b400640 1 --2- 172.21.15.87:0/637716625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c06ba80 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009870 tx=0x7f3e2c01c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282c551b4c5e174f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e3aa00640 1 -- 172.21.15.87:0/637716625 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0b0070 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e3aa00640 1 -- 172.21.15.87:0/637716625 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c0ab600 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e3aa00640 1 -- 172.21.15.87:0/637716625 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0ab9d0 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 -- 172.21.15.87:0/637716625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 msgr2=0x7f3e3c06ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 --2- 172.21.15.87:0/637716625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c06ba80 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009870 tx=0x7f3e2c01c270 comp rx=0 tx=0).stop 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 -- 172.21.15.87:0/637716625 shutdown_connections 2024-04-25T10:23:37.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 --2- 172.21.15.87:0/637716625 >> v2:172.21.15.169:3300/0 conn(0x7f3e3c0b3fc0 0x7f3e3c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 --2- 172.21.15.87:0/637716625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c06ba80 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 -- 172.21.15.87:0/637716625 >> 172.21.15.87:0/637716625 conn(0x7f3e3c0b34a0 msgr2=0x7f3e3c0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 -- 172.21.15.87:0/637716625 shutdown_connections 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.969+0000 7f3e42800640 1 -- 172.21.15.87:0/637716625 wait complete. 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 Processor -- start 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 -- start start 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e3c0b26b0 0x7f3e3c161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3c0b3fc0 0x7f3e3c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c16b380 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c16b200 con 0x7f3e3c0b3fc0 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c16b500 con 0x7f3e3c0b26b0 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c1562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53130/0 (socket says 172.21.15.87:53130) 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 -- 172.21.15.87:0/2890944558 learned_addr learned my addr 172.21.15.87:0/2890944558 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:37.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3be00640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3c0b3fc0 0x7f3e3c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e33e00640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e3c0b26b0 0x7f3e3c161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 -- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e3c0b26b0 msgr2=0x7f3e3c161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e3c0b26b0 0x7f3e3c161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 -- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3c0b3fc0 msgr2=0x7f3e3c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3c0b3fc0 0x7f3e3c154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 -- 172.21.15.87:0/2890944558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c0b48a0 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e3b400640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c1562b0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0ae040 tx=0x7f3e2c019010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e33400640 1 -- 172.21.15.87:0/2890944558 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0b0040 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e33400640 1 -- 172.21.15.87:0/2890944558 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c0b9070 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c0b0580 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e33400640 1 -- 172.21.15.87:0/2890944558 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0b4680 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c0b4590 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.973+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c04ef30 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.977+0000 7f3e33400640 1 -- 172.21.15.87:0/2890944558 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e2c0b4820 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.977+0000 7f3e33400640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e18066f90 0x7f3e18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:37.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.977+0000 7f3e33400640 1 -- 172.21.15.87:0/2890944558 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f3e2c11ebe0 con 0x7f3e3c06b6a0 2024-04-25T10:23:37.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.977+0000 7f3e33e00640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e18066f90 0x7f3e18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:37.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.977+0000 7f3e33e00640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e18066f90 0x7f3e18069450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b1da0 tx=0x7f3e24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:37.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:37.981+0000 7f3e33400640 1 -- 172.21.15.87:0/2890944558 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e3c0b2f50 con 0x7f3e3c06b6a0 2024-04-25T10:23:38.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.137+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3e3c06b480 con 0x7f3e3c06b6a0 2024-04-25T10:23:38.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.805+0000 7f3e33400640 1 -- 172.21.15.87:0/2890944558 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3e2c0efc90 con 0x7f3e3c06b6a0 2024-04-25T10:23:38.806 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:23:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e18066f90 msgr2=0x7f3e18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e18066f90 0x7f3e18069450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b1da0 tx=0x7f3e24008040 comp rx=0 tx=0).stop 2024-04-25T10:23:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 msgr2=0x7f3e3c1562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c1562b0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0ae040 tx=0x7f3e2c019010 comp rx=0 tx=0).stop 2024-04-25T10:23:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 shutdown_connections 2024-04-25T10:23:38.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3e18066f90 0x7f3e18069450 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3c0b3fc0 0x7f3e3c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e3c0b26b0 0x7f3e3c161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 --2- 172.21.15.87:0/2890944558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e3c06b6a0 0x7f3e3c1562b0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 >> 172.21.15.87:0/2890944558 conn(0x7f3e3c0b34a0 msgr2=0x7f3e3c0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 shutdown_connections 2024-04-25T10:23:38.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.809+0000 7f3e42800640 1 -- 172.21.15.87:0/2890944558 wait complete. 2024-04-25T10:23:38.840 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:23:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6031000640 1 Processor -- start 2024-04-25T10:23:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6031000640 1 -- start start 2024-04-25T10:23:38.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6031000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f602c0722d0 0x7f602c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6031000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c0ad840 con 0x7f602c0b3480 2024-04-25T10:23:38.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6031000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c15e710 con 0x7f602c15e130 2024-04-25T10:23:38.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6031000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602c0d0090 con 0x7f602c0722d0 2024-04-25T10:23:38.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6029e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f602c0b3480 0x7f602c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39716/0 (socket says 172.21.15.87:39716) 2024-04-25T10:23:38.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6029e00640 1 -- 172.21.15.87:0/2226782768 learned_addr learned my addr 172.21.15.87:0/2226782768 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:38.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f602a800640 1 --2- 172.21.15.87:0/2226782768 >> v2:172.21.15.87:3300/0 conn(0x7f602c0722d0 0x7f602c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6028a00640 1 -- 172.21.15.87:0/2226782768 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771323644 0 0) 0x7f602c0ad840 con 0x7f602c0b3480 2024-04-25T10:23:38.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f6028a00640 1 -- 172.21.15.87:0/2226782768 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6010003530 con 0x7f602c0b3480 2024-04-25T10:23:38.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f602a800640 1 -- 172.21.15.87:0/2226782768 >> v1:172.21.15.169:6789/0 conn(0x7f602c15e130 legacy=0x7f602c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f602a800640 1 -- 172.21.15.87:0/2226782768 >> v1:172.21.15.87:6789/0 conn(0x7f602c0b3480 legacy=0x7f602c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.981+0000 7f602a800640 1 -- 172.21.15.87:0/2226782768 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c15e9f0 con 0x7f602c0722d0 2024-04-25T10:23:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f602a800640 1 --2- 172.21.15.87:0/2226782768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c0cfb50 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f6024009d90 tx=0x7f602401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadec93985626a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6028a00640 1 -- 172.21.15.87:0/2226782768 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240b2070 con 0x7f602c0722d0 2024-04-25T10:23:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6028a00640 1 -- 172.21.15.87:0/2226782768 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60240a57a0 con 0x7f602c0722d0 2024-04-25T10:23:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6028a00640 1 -- 172.21.15.87:0/2226782768 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240a5b70 con 0x7f602c0722d0 2024-04-25T10:23:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6031000640 1 -- 172.21.15.87:0/2226782768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 msgr2=0x7f602c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6031000640 1 --2- 172.21.15.87:0/2226782768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c0cfb50 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f6024009d90 tx=0x7f602401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6031000640 1 -- 172.21.15.87:0/2226782768 shutdown_connections 2024-04-25T10:23:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6031000640 1 --2- 172.21.15.87:0/2226782768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c0cfb50 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6031000640 1 -- 172.21.15.87:0/2226782768 >> 172.21.15.87:0/2226782768 conn(0x7f602c0669a0 msgr2=0x7f602c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6031000640 1 -- 172.21.15.87:0/2226782768 shutdown_connections 2024-04-25T10:23:38.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.985+0000 7f6031000640 1 -- 172.21.15.87:0/2226782768 wait complete. 2024-04-25T10:23:38.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 Processor -- start 2024-04-25T10:23:38.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 -- start start 2024-04-25T10:23:38.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602c0b3480 0x7f602c1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f602c15e130 0x7f602c135b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c15bb90 con 0x7f602c0722d0 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0ae7c0 con 0x7f602c0b3480 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c16b8a0 con 0x7f602c15e130 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c1566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53144/0 (socket says 172.21.15.87:53144) 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 -- 172.21.15.87:0/680612098 learned_addr learned my addr 172.21.15.87:0/680612098 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f602a800640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f602c15e130 0x7f602c135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029400640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602c0b3480 0x7f602c1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 -- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f602c15e130 msgr2=0x7f602c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f602c15e130 0x7f602c135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 -- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602c0b3480 msgr2=0x7f602c1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602c0b3480 0x7f602c1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 -- 172.21.15.87:0/680612098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c14c470 con 0x7f602c0722d0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f602a800640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f602c15e130 0x7f602c135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6029e00640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c1566a0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c4d0 tx=0x7f601c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6022000640 1 -- 172.21.15.87:0/680612098 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c017070 con 0x7f602c0722d0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6022000640 1 -- 172.21.15.87:0/680612098 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c005cf0 con 0x7f602c0722d0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c1380f0 con 0x7f602c0722d0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6022000640 1 -- 172.21.15.87:0/680612098 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c007270 con 0x7f602c0722d0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c1386b0 con 0x7f602c0722d0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.989+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602c04ef30 con 0x7f602c0722d0 2024-04-25T10:23:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.993+0000 7f6022000640 1 -- 172.21.15.87:0/680612098 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f601c01a070 con 0x7f602c0722d0 2024-04-25T10:23:38.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.993+0000 7f6022000640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f600c066f90 0x7f600c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:38.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.993+0000 7f6022000640 1 -- 172.21.15.87:0/680612098 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f601c084060 con 0x7f602c0722d0 2024-04-25T10:23:38.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.993+0000 7f6029400640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f600c066f90 0x7f600c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:38.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.993+0000 7f6029400640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f600c066f90 0x7f600c069450 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f602c155f80 tx=0x7f6018009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:38.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:38.997+0000 7f6022000640 1 -- 172.21.15.87:0/680612098 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f601c054c80 con 0x7f602c0722d0 2024-04-25T10:23:39.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.181+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f602c153d60 con 0x7f602c0722d0 2024-04-25T10:23:39.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.189+0000 7f6022000640 1 -- 172.21.15.87:0/680612098 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f601c058d20 con 0x7f602c0722d0 2024-04-25T10:23:39.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f600c066f90 msgr2=0x7f600c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f600c066f90 0x7f600c069450 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f602c155f80 tx=0x7f6018009040 comp rx=0 tx=0).stop 2024-04-25T10:23:39.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 msgr2=0x7f602c1566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c1566a0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c4d0 tx=0x7f601c00c990 comp rx=0 tx=0).stop 2024-04-25T10:23:39.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 shutdown_connections 2024-04-25T10:23:39.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f600c066f90 0x7f600c069450 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f602c15e130 0x7f602c135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602c0b3480 0x7f602c1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 --2- 172.21.15.87:0/680612098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f602c0722d0 0x7f602c1566a0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 >> 172.21.15.87:0/680612098 conn(0x7f602c0669a0 msgr2=0x7f602c06cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:39.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 shutdown_connections 2024-04-25T10:23:39.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.193+0000 7f6031000640 1 -- 172.21.15.87:0/680612098 wait complete. 2024-04-25T10:23:39.215 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:23:39.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.353+0000 7eff64000640 1 Processor -- start 2024-04-25T10:23:39.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.353+0000 7eff64000640 1 -- start start 2024-04-25T10:23:39.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.353+0000 7eff64000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.353+0000 7eff64000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff5c0cda10 0x7eff5c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.353+0000 7eff64000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5c0b2770 con 0x7eff5c0cf930 2024-04-25T10:23:39.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.353+0000 7eff64000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c06c670 con 0x7eff5c0cda10 2024-04-25T10:23:39.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.353+0000 7eff64000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c06c950 con 0x7eff5c15bc40 2024-04-25T10:23:39.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff60c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff5c0cda10 0x7eff5c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff61600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff62000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7eff5c0cf930 0x7eff5c15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:51822/0 (socket says 172.21.15.87:51822) 2024-04-25T10:23:39.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff62000640 1 -- 172.21.15.87:0/780974154 learned_addr learned my addr 172.21.15.87:0/780974154 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:39.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff60c00640 1 -- 172.21.15.87:0/780974154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c15bc40 msgr2=0x7eff5c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:39.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff60c00640 1 --2- 172.21.15.87:0/780974154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff60c00640 1 -- 172.21.15.87:0/780974154 >> v1:172.21.15.169:6789/0 conn(0x7eff5c0cf930 legacy=0x7eff5c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff60c00640 1 -- 172.21.15.87:0/780974154 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c06cc30 con 0x7eff5c0cda10 2024-04-25T10:23:39.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff5be00640 1 -- 172.21.15.87:0/780974154 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482953111 0 0) 0x7eff5c0b2770 con 0x7eff5c0cf930 2024-04-25T10:23:39.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff60c00640 1 --2- 172.21.15.87:0/780974154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 0x7eff5c0b4610 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7eff480097c0 tx=0x7eff4801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb0734189fd2cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:39.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff5be00640 1 -- 172.21.15.87:0/780974154 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480b1070 con 0x7eff5c0cda10 2024-04-25T10:23:39.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff5be00640 1 -- 172.21.15.87:0/780974154 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff480ac410 con 0x7eff5c0cda10 2024-04-25T10:23:39.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.357+0000 7eff5be00640 1 -- 172.21.15.87:0/780974154 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480ac7e0 con 0x7eff5c0cda10 2024-04-25T10:23:39.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- 172.21.15.87:0/780974154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 msgr2=0x7eff5c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 --2- 172.21.15.87:0/780974154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 0x7eff5c0b4610 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7eff480097c0 tx=0x7eff4801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- 172.21.15.87:0/780974154 shutdown_connections 2024-04-25T10:23:39.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 --2- 172.21.15.87:0/780974154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 0x7eff5c0b4610 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 --2- 172.21.15.87:0/780974154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- 172.21.15.87:0/780974154 >> 172.21.15.87:0/780974154 conn(0x7eff5c066b70 msgr2=0x7eff5c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:39.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- 172.21.15.87:0/780974154 shutdown_connections 2024-04-25T10:23:39.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- 172.21.15.87:0/780974154 wait complete. 2024-04-25T10:23:39.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 Processor -- start 2024-04-25T10:23:39.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- start start 2024-04-25T10:23:39.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 0x7eff5c148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0cf930 0x7eff5c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff5c15bc40 0x7eff5c15d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c168b50 con 0x7eff5c15bc40 2024-04-25T10:23:39.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c1689d0 con 0x7eff5c0cda10 2024-04-25T10:23:39.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff64000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c168cd0 con 0x7eff5c0cf930 2024-04-25T10:23:39.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff5c15bc40 0x7eff5c15d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff5c15bc40 0x7eff5c15d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53152/0 (socket says 172.21.15.87:53152) 2024-04-25T10:23:39.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 -- 172.21.15.87:0/3657401469 learned_addr learned my addr 172.21.15.87:0/3657401469 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:39.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff61600640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 0x7eff5c148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff60c00640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0cf930 0x7eff5c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 -- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0cf930 msgr2=0x7eff5c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0cf930 0x7eff5c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 -- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 msgr2=0x7eff5c148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 0x7eff5c148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.361+0000 7eff62000640 1 -- 172.21.15.87:0/3657401469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c067fa0 con 0x7eff5c15bc40 2024-04-25T10:23:39.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff60c00640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0cf930 0x7eff5c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:39.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff62000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff5c15bc40 0x7eff5c15d800 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c480 tx=0x7eff5400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:39.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff59600640 1 -- 172.21.15.87:0/3657401469 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54018070 con 0x7eff5c15bc40 2024-04-25T10:23:39.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff59600640 1 -- 172.21.15.87:0/3657401469 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff540074e0 con 0x7eff5c15bc40 2024-04-25T10:23:39.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff59600640 1 -- 172.21.15.87:0/3657401469 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54004a30 con 0x7eff5c15bc40 2024-04-25T10:23:39.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c14c840 con 0x7eff5c15bc40 2024-04-25T10:23:39.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c149730 con 0x7eff5c15bc40 2024-04-25T10:23:39.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c14cd40 con 0x7eff5c15bc40 2024-04-25T10:23:39.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff59600640 1 -- 172.21.15.87:0/3657401469 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff54004bd0 con 0x7eff5c15bc40 2024-04-25T10:23:39.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff59600640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff3c066f60 0x7eff3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:39.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff59600640 1 -- 172.21.15.87:0/3657401469 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7eff54083ff0 con 0x7eff5c15bc40 2024-04-25T10:23:39.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.365+0000 7eff61600640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff3c066f60 0x7eff3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:39.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.369+0000 7eff61600640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff3c066f60 0x7eff3c069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7eff5c1590f0 tx=0x7eff5000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:39.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.369+0000 7eff59600640 1 -- 172.21.15.87:0/3657401469 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff54054ce0 con 0x7eff5c15bc40 2024-04-25T10:23:39.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.581+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7eff5c04ef30 con 0x7eff5c15bc40 2024-04-25T10:23:39.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.809+0000 7eff59600640 1 -- 172.21.15.87:0/3657401469 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v547) v1 ==== 159+0+0 (secure 0 0 0) 0x7eff54058d80 con 0x7eff5c15bc40 2024-04-25T10:23:39.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff3c066f60 msgr2=0x7eff3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff3c066f60 0x7eff3c069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7eff5c1590f0 tx=0x7eff5000b040 comp rx=0 tx=0).stop 2024-04-25T10:23:39.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff5c15bc40 msgr2=0x7eff5c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:39.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff5c15bc40 0x7eff5c15d800 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c480 tx=0x7eff5400c940 comp rx=0 tx=0).stop 2024-04-25T10:23:39.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 shutdown_connections 2024-04-25T10:23:39.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7eff3c066f60 0x7eff3c069420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff5c15bc40 0x7eff5c15d800 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0cf930 0x7eff5c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 --2- 172.21.15.87:0/3657401469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cda10 0x7eff5c148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:39.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 >> 172.21.15.87:0/3657401469 conn(0x7eff5c066b70 msgr2=0x7eff5c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:39.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 shutdown_connections 2024-04-25T10:23:39.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:39.813+0000 7eff64000640 1 -- 172.21.15.87:0/3657401469 wait complete. 2024-04-25T10:23:39.897 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:23:39.897 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:23:39.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:40.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.033+0000 7fa124800640 1 Processor -- start 2024-04-25T10:23:40.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.033+0000 7fa124800640 1 -- start start 2024-04-25T10:23:40.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa124800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa124800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa11c15b910 0x7fa11c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa124800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c071050 con 0x7fa11c0cfe70 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa124800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c15bda0 con 0x7fa11c15b910 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa124800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15d040 con 0x7fa11c1550b0 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa122800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa11c15b910 0x7fa11c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa122800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa11c15b910 0x7fa11c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43430/0 (socket says 172.21.15.87:43430) 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa122800640 1 -- 172.21.15.87:0/3174464670 learned_addr learned my addr 172.21.15.87:0/3174464670 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa121e00640 1 --2- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa122800640 1 -- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 msgr2=0x7fa11c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa122800640 1 --2- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa122800640 1 -- 172.21.15.87:0/3174464670 >> v1:172.21.15.169:6789/0 conn(0x7fa11c0cfe70 legacy=0x7fa11c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.037+0000 7fa122800640 1 -- 172.21.15.87:0/3174464670 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0712f0 con 0x7fa11c15b910 2024-04-25T10:23:40.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa122800640 1 --2- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c15b910 0x7fa11c0b3260 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa118009e50 tx=0x7fa11801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77968515979e3ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa120a00640 1 -- 172.21.15.87:0/3174464670 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180a57f0 con 0x7fa11c15b910 2024-04-25T10:23:40.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa120a00640 1 -- 172.21.15.87:0/3174464670 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1180a5990 con 0x7fa11c15b910 2024-04-25T10:23:40.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa120a00640 1 -- 172.21.15.87:0/3174464670 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180a5d60 con 0x7fa11c15b910 2024-04-25T10:23:40.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c15b910 msgr2=0x7fa11c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 --2- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c15b910 0x7fa11c0b3260 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa118009e50 tx=0x7fa11801ca30 comp rx=0 tx=0).stop 2024-04-25T10:23:40.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- 172.21.15.87:0/3174464670 shutdown_connections 2024-04-25T10:23:40.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 --2- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c15b910 0x7fa11c0b3260 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 --2- 172.21.15.87:0/3174464670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- 172.21.15.87:0/3174464670 >> 172.21.15.87:0/3174464670 conn(0x7fa11c066b70 msgr2=0x7fa11c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:40.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- 172.21.15.87:0/3174464670 shutdown_connections 2024-04-25T10:23:40.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- 172.21.15.87:0/3174464670 wait complete. 2024-04-25T10:23:40.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 Processor -- start 2024-04-25T10:23:40.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- start start 2024-04-25T10:23:40.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0cfe70 0x7fa11c150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa11c15b910 0x7fa11c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15c030 con 0x7fa11c15b910 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0720b0 con 0x7fa11c0cfe70 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa124800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15c1b0 con 0x7fa11c1550b0 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa122800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa11c15b910 0x7fa11c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa122800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa11c15b910 0x7fa11c156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53158/0 (socket says 172.21.15.87:53158) 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.041+0000 7fa122800640 1 -- 172.21.15.87:0/560255269 learned_addr learned my addr 172.21.15.87:0/560255269 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:40.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa121e00640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0cfe70 0x7fa11c150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa121400640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa122800640 1 -- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 msgr2=0x7fa11c14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa122800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c14c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa122800640 1 -- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0cfe70 msgr2=0x7fa11c150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa122800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0cfe70 0x7fa11c150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa122800640 1 -- 172.21.15.87:0/560255269 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c150dc0 con 0x7fa11c15b910 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa121400640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa122800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa11c15b910 0x7fa11c156720 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fa11801a660 tx=0x7fa1180197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa116000640 1 -- 172.21.15.87:0/560255269 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180bc070 con 0x7fa11c15b910 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa116000640 1 -- 172.21.15.87:0/560255269 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1180b8640 con 0x7fa11c15b910 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa116000640 1 -- 172.21.15.87:0/560255269 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b8a30 con 0x7fa11c15b910 2024-04-25T10:23:40.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c14f500 con 0x7fa11c15b910 2024-04-25T10:23:40.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c14fac0 con 0x7fa11c15b910 2024-04-25T10:23:40.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c04ef30 con 0x7fa11c15b910 2024-04-25T10:23:40.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa116000640 1 -- 172.21.15.87:0/560255269 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1180bf070 con 0x7fa11c15b910 2024-04-25T10:23:40.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa116000640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa100066f60 0x7fa100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa116000640 1 -- 172.21.15.87:0/560255269 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa11811fdf0 con 0x7fa11c15b910 2024-04-25T10:23:40.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.045+0000 7fa121e00640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa100066f60 0x7fa100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.049+0000 7fa121e00640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa100066f60 0x7fa100069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cfd50 tx=0x7fa110009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.049+0000 7fa116000640 1 -- 172.21.15.87:0/560255269 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa11c150dc0 con 0x7fa11c15b910 2024-04-25T10:23:40.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.205+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa11c067270 con 0x7fa11c15b910 2024-04-25T10:23:40.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.205+0000 7fa116000640 1 -- 172.21.15.87:0/560255269 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+13018 (secure 0 0 0) 0x7fa118016620 con 0x7fa11c15b910 2024-04-25T10:23:40.206 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:40.206 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:38.802606+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/1025107360":"2024-04-26T10:23:35.789597+0000","172.21.15.87:6835/1025107360":"2024-04-26T10:23:35.789597+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa100066f60 msgr2=0x7fa100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa100066f60 0x7fa100069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cfd50 tx=0x7fa110009190 comp rx=0 tx=0).stop 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa11c15b910 msgr2=0x7fa11c156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa11c15b910 0x7fa11c156720 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fa11801a660 tx=0x7fa1180197d0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 shutdown_connections 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa100066f60 0x7fa100069420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa11c15b910 0x7fa11c156720 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c1550b0 0x7fa11c14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 --2- 172.21.15.87:0/560255269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0cfe70 0x7fa11c150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 >> 172.21.15.87:0/560255269 conn(0x7fa11c066b70 msgr2=0x7fa11c0cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 shutdown_connections 2024-04-25T10:23:40.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.209+0000 7fa124800640 1 -- 172.21.15.87:0/560255269 wait complete. 2024-04-25T10:23:40.227 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:23:40.228 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:23:40.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.365+0000 7fcfabe00640 1 Processor -- start 2024-04-25T10:23:40.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.365+0000 7fcfabe00640 1 -- start start 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfabe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfa40b3480 0x7fcfa415e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40cfb50 0x7fcfa4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfabe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcfa4071bc0 0x7fcfa4072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfabe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa415bb90 con 0x7fcfa40b3480 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfabe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa40ad840 con 0x7fcfa4071bc0 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfabe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa415e710 con 0x7fcfa40cfb50 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfa40b3480 0x7fcfa415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfa40b3480 0x7fcfa415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53162/0 (socket says 172.21.15.87:53162) 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 -- 172.21.15.87:0/2643910720 learned_addr learned my addr 172.21.15.87:0/2643910720 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:40.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9e00640 1 --2- 172.21.15.87:0/2643910720 >> v2:172.21.15.169:3300/0 conn(0x7fcfa4071bc0 0x7fcfa4072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa8a00640 1 --2- 172.21.15.87:0/2643910720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40cfb50 0x7fcfa4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 -- 172.21.15.87:0/2643910720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40cfb50 msgr2=0x7fcfa4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 --2- 172.21.15.87:0/2643910720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40cfb50 0x7fcfa4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 -- 172.21.15.87:0/2643910720 >> v2:172.21.15.169:3300/0 conn(0x7fcfa4071bc0 msgr2=0x7fcfa4072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 --2- 172.21.15.87:0/2643910720 >> v2:172.21.15.169:3300/0 conn(0x7fcfa4071bc0 0x7fcfa4072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9400640 1 -- 172.21.15.87:0/2643910720 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa415e9f0 con 0x7fcfa40b3480 2024-04-25T10:23:40.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.369+0000 7fcfa9e00640 1 --2- 172.21.15.87:0/2643910720 >> v2:172.21.15.169:3300/0 conn(0x7fcfa4071bc0 0x7fcfa4072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:40.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfa9400640 1 --2- 172.21.15.87:0/2643910720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa40b3480 0x7fcfa415e130 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40b41f0 tx=0x7fcf9401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2545ac066d2bb6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfa3e00640 1 -- 172.21.15.87:0/2643910720 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940b0070 con 0x7fcfa40b3480 2024-04-25T10:23:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfa3e00640 1 -- 172.21.15.87:0/2643910720 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf940ab600 con 0x7fcfa40b3480 2024-04-25T10:23:40.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfa3e00640 1 -- 172.21.15.87:0/2643910720 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940ab9d0 con 0x7fcfa40b3480 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 -- 172.21.15.87:0/2643910720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa40b3480 msgr2=0x7fcfa415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 --2- 172.21.15.87:0/2643910720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa40b3480 0x7fcfa415e130 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40b41f0 tx=0x7fcf9401c520 comp rx=0 tx=0).stop 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 -- 172.21.15.87:0/2643910720 shutdown_connections 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 --2- 172.21.15.87:0/2643910720 >> v2:172.21.15.169:3300/0 conn(0x7fcfa4071bc0 0x7fcfa4072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 --2- 172.21.15.87:0/2643910720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40cfb50 0x7fcfa4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 --2- 172.21.15.87:0/2643910720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa40b3480 0x7fcfa415e130 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 -- 172.21.15.87:0/2643910720 >> 172.21.15.87:0/2643910720 conn(0x7fcfa40669a0 msgr2=0x7fcfa4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 -- 172.21.15.87:0/2643910720 shutdown_connections 2024-04-25T10:23:40.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 -- 172.21.15.87:0/2643910720 wait complete. 2024-04-25T10:23:40.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 Processor -- start 2024-04-25T10:23:40.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 -- start start 2024-04-25T10:23:40.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa4071bc0 0x7fcfa4153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.373+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40b3480 0x7fcfa415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa40cfb50 0x7fcfa4153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfabe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4070a10 con 0x7fcfa4071bc0 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfabe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4070890 con 0x7fcfa40cfb50 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfabe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40ae530 con 0x7fcfa40b3480 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa4071bc0 0x7fcfa4153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa4071bc0 0x7fcfa4153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53166/0 (socket says 172.21.15.87:53166) 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 -- 172.21.15.87:0/3002997435 learned_addr learned my addr 172.21.15.87:0/3002997435 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9e00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa40cfb50 0x7fcfa4153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa8a00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40b3480 0x7fcfa415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 -- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40b3480 msgr2=0x7fcfa415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40b3480 0x7fcfa415f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 -- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa40cfb50 msgr2=0x7fcfa4153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa40cfb50 0x7fcfa4153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 -- 172.21.15.87:0/3002997435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4153da0 con 0x7fcfa4071bc0 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa9400640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa4071bc0 0x7fcfa4153750 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fcf9401cdc0 tx=0x7fcf940192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa1600640 1 -- 172.21.15.87:0/3002997435 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940b0040 con 0x7fcfa4071bc0 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa1600640 1 -- 172.21.15.87:0/3002997435 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf940b4420 con 0x7fcfa4071bc0 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa415fc90 con 0x7fcfa4071bc0 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa1600640 1 -- 172.21.15.87:0/3002997435 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940b4810 con 0x7fcfa4071bc0 2024-04-25T10:23:40.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa414b910 con 0x7fcfa4071bc0 2024-04-25T10:23:40.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa414d040 con 0x7fcfa4071bc0 2024-04-25T10:23:40.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa1600640 1 -- 172.21.15.87:0/3002997435 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf940b49b0 con 0x7fcfa4071bc0 2024-04-25T10:23:40.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.377+0000 7fcfa1600640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf88066f60 0x7fcf88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.381+0000 7fcfa1600640 1 -- 172.21.15.87:0/3002997435 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcf9411e940 con 0x7fcfa4071bc0 2024-04-25T10:23:40.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.381+0000 7fcfa8a00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf88066f60 0x7fcf88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.381+0000 7fcfa8a00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf88066f60 0x7fcf88069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcfa414d240 tx=0x7fcf98008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.381+0000 7fcfa1600640 1 -- 172.21.15.87:0/3002997435 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfa414d040 con 0x7fcfa4071bc0 2024-04-25T10:23:40.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfa404ef30 con 0x7fcfa4071bc0 2024-04-25T10:23:40.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfa1600640 1 -- 172.21.15.87:0/3002997435 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf94016110 con 0x7fcfa4071bc0 2024-04-25T10:23:40.536 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf88066f60 msgr2=0x7fcf88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf88066f60 0x7fcf88069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcfa414d240 tx=0x7fcf98008040 comp rx=0 tx=0).stop 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa4071bc0 msgr2=0x7fcfa4153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa4071bc0 0x7fcfa4153750 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fcf9401cdc0 tx=0x7fcf940192c0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 shutdown_connections 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf88066f60 0x7fcf88069420 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa40cfb50 0x7fcfa4153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfa40b3480 0x7fcfa415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 --2- 172.21.15.87:0/3002997435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfa4071bc0 0x7fcfa4153750 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 >> 172.21.15.87:0/3002997435 conn(0x7fcfa40669a0 msgr2=0x7fcfa415bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:40.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 shutdown_connections 2024-04-25T10:23:40.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.537+0000 7fcfabe00640 1 -- 172.21.15.87:0/3002997435 wait complete. 2024-04-25T10:23:40.558 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:23:40.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f177e600640 1 Processor -- start 2024-04-25T10:23:40.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f177e600640 1 -- start start 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f177e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f17781550b0 0x7f17780cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f177e600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f177815b910 0x7f17780b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f177e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1778071050 con 0x7f17780cfe70 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f177e600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f177815bd80 con 0x7f177815b910 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f177e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f177815d040 con 0x7f17781550b0 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f1777e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f177815b910 0x7f17780b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f1777e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f177815b910 0x7f17780b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53178/0 (socket says 172.21.15.87:53178) 2024-04-25T10:23:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f1777e00640 1 -- 172.21.15.87:0/3989084052 learned_addr learned my addr 172.21.15.87:0/3989084052 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.693+0000 7f1777400640 1 --2- 172.21.15.87:0/3989084052 >> v2:172.21.15.169:3300/0 conn(0x7f17781550b0 0x7f17780cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1777e00640 1 -- 172.21.15.87:0/3989084052 >> v2:172.21.15.169:3300/0 conn(0x7f17781550b0 msgr2=0x7f17780cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1777e00640 1 --2- 172.21.15.87:0/3989084052 >> v2:172.21.15.169:3300/0 conn(0x7f17781550b0 0x7f17780cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1777e00640 1 -- 172.21.15.87:0/3989084052 >> v1:172.21.15.169:6789/0 conn(0x7f17780cfe70 legacy=0x7f177815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1777e00640 1 -- 172.21.15.87:0/3989084052 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17780712f0 con 0x7f177815b910 2024-04-25T10:23:40.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1777e00640 1 --2- 172.21.15.87:0/3989084052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f177815b910 0x7f17780b3260 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f1770009e50 tx=0x7f177001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b572a18a1bcf1733 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1776a00640 1 -- 172.21.15.87:0/3989084052 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17700a5760 con 0x7f177815b910 2024-04-25T10:23:40.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1776a00640 1 -- 172.21.15.87:0/3989084052 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17700a5900 con 0x7f177815b910 2024-04-25T10:23:40.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f1776a00640 1 -- 172.21.15.87:0/3989084052 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17700a5cd0 con 0x7f177815b910 2024-04-25T10:23:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 -- 172.21.15.87:0/3989084052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f177815b910 msgr2=0x7f17780b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 --2- 172.21.15.87:0/3989084052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f177815b910 0x7f17780b3260 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f1770009e50 tx=0x7f177001c9b0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 -- 172.21.15.87:0/3989084052 shutdown_connections 2024-04-25T10:23:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 --2- 172.21.15.87:0/3989084052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f177815b910 0x7f17780b3260 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 --2- 172.21.15.87:0/3989084052 >> v2:172.21.15.169:3300/0 conn(0x7f17781550b0 0x7f17780cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 -- 172.21.15.87:0/3989084052 >> 172.21.15.87:0/3989084052 conn(0x7f1778066b70 msgr2=0x7f1778066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:40.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 -- 172.21.15.87:0/3989084052 shutdown_connections 2024-04-25T10:23:40.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.697+0000 7f177e600640 1 -- 172.21.15.87:0/3989084052 wait complete. 2024-04-25T10:23:40.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 Processor -- start 2024-04-25T10:23:40.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 -- start start 2024-04-25T10:23:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17780cfe70 0x7f17780cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781550b0 0x7f177815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f177815b910 0x7f177815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177815c030 con 0x7f17780cfe70 2024-04-25T10:23:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17780720b0 con 0x7f17781550b0 2024-04-25T10:23:40.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177815c1b0 con 0x7f177815b910 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17780cfe70 0x7f17780cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17780cfe70 0x7f17780cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53192/0 (socket says 172.21.15.87:53192) 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 -- 172.21.15.87:0/2807329820 learned_addr learned my addr 172.21.15.87:0/2807329820 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777e00640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f177815b910 0x7f177815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f176ea00640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781550b0 0x7f177815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 -- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f177815b910 msgr2=0x7f177815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f177815b910 0x7f177815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 -- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781550b0 msgr2=0x7f177815d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781550b0 0x7f177815d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 -- 172.21.15.87:0/2807329820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17780cb910 con 0x7f17780cfe70 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f176ea00640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781550b0 0x7f177815d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f1777400640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17780cfe70 0x7f17780cb3d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f176800d800 tx=0x7f176800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f176fe00640 1 -- 172.21.15.87:0/2807329820 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800b6c0 con 0x7f17780cfe70 2024-04-25T10:23:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f176fe00640 1 -- 172.21.15.87:0/2807329820 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1768019070 con 0x7f17780cfe70 2024-04-25T10:23:40.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1778156bd0 con 0x7f17780cfe70 2024-04-25T10:23:40.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.701+0000 7f176fe00640 1 -- 172.21.15.87:0/2807329820 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800ba60 con 0x7f17780cfe70 2024-04-25T10:23:40.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.705+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1778157190 con 0x7f17780cfe70 2024-04-25T10:23:40.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.705+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177804ef30 con 0x7f17780cfe70 2024-04-25T10:23:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.705+0000 7f176fe00640 1 -- 172.21.15.87:0/2807329820 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f176800bc00 con 0x7f17780cfe70 2024-04-25T10:23:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.705+0000 7f176fe00640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1754066f60 0x7f1754069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.705+0000 7f176fe00640 1 -- 172.21.15.87:0/2807329820 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1768085180 con 0x7f17780cfe70 2024-04-25T10:23:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.705+0000 7f176ea00640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1754066f60 0x7f1754069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:40.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.705+0000 7f176ea00640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1754066f60 0x7f1754069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae240 tx=0x7f1760009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:40.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.709+0000 7f176fe00640 1 -- 172.21.15.87:0/2807329820 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17780cb910 con 0x7f17780cfe70 2024-04-25T10:23:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:40.953+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f1778142970 con 0x7f17780cfe70 2024-04-25T10:23:41.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f176fe00640 1 -- 172.21.15.87:0/2807329820 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v549) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1768056480 con 0x7f17780cfe70 2024-04-25T10:23:41.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1754066f60 msgr2=0x7f1754069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1754066f60 0x7f1754069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae240 tx=0x7f1760009040 comp rx=0 tx=0).stop 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17780cfe70 msgr2=0x7f17780cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17780cfe70 0x7f17780cb3d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f176800d800 tx=0x7f176800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 shutdown_connections 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1754066f60 0x7f1754069420 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f177815b910 0x7f177815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781550b0 0x7f177815d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 --2- 172.21.15.87:0/2807329820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17780cfe70 0x7f17780cb3d0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 >> 172.21.15.87:0/2807329820 conn(0x7f1778066b70 msgr2=0x7f17780ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 shutdown_connections 2024-04-25T10:23:41.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.817+0000 7f177e600640 1 -- 172.21.15.87:0/2807329820 wait complete. 2024-04-25T10:23:41.839 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:23:41.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.977+0000 7f41cb200640 1 Processor -- start 2024-04-25T10:23:41.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.977+0000 7f41cb200640 1 -- start start 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41cb200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f41cc15b950 0x7f41cc161750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41cb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f41cc0b06b0 0x7f41cc0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41cb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41cc06b3a0 con 0x7f41cc0b46e0 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41cb200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc161d30 con 0x7f41cc15b950 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41cb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc162010 con 0x7f41cc0b06b0 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41c9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f41cc15b950 0x7f41cc161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41c9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f41cc15b950 0x7f41cc161750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53208/0 (socket says 172.21.15.87:53208) 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41c9e00640 1 -- 172.21.15.87:0/2202355699 learned_addr learned my addr 172.21.15.87:0/2202355699 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:41.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41ca800640 1 --2- 172.21.15.87:0/2202355699 >> v2:172.21.15.169:3300/0 conn(0x7f41cc0b06b0 0x7f41cc0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41c9e00640 1 -- 172.21.15.87:0/2202355699 >> v2:172.21.15.169:3300/0 conn(0x7f41cc0b06b0 msgr2=0x7f41cc0b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:41.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41c9e00640 1 --2- 172.21.15.87:0/2202355699 >> v2:172.21.15.169:3300/0 conn(0x7f41cc0b06b0 0x7f41cc0b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41c9e00640 1 -- 172.21.15.87:0/2202355699 >> v1:172.21.15.169:6789/0 conn(0x7f41cc0b46e0 legacy=0x7f41cc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.981+0000 7f41c9e00640 1 -- 172.21.15.87:0/2202355699 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc1622f0 con 0x7f41cc15b950 2024-04-25T10:23:41.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41c9e00640 1 --2- 172.21.15.87:0/2202355699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc15b950 0x7f41cc161750 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f41cc06c860 tx=0x7f41bc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc429aa30065f102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:41.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41c8a00640 1 -- 172.21.15.87:0/2202355699 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc0090d0 con 0x7f41cc15b950 2024-04-25T10:23:41.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41c8a00640 1 -- 172.21.15.87:0/2202355699 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc0ae070 con 0x7f41cc15b950 2024-04-25T10:23:41.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41c8a00640 1 -- 172.21.15.87:0/2202355699 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc009270 con 0x7f41cc15b950 2024-04-25T10:23:41.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- 172.21.15.87:0/2202355699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc15b950 msgr2=0x7f41cc161750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 --2- 172.21.15.87:0/2202355699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc15b950 0x7f41cc161750 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f41cc06c860 tx=0x7f41bc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- 172.21.15.87:0/2202355699 shutdown_connections 2024-04-25T10:23:41.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 --2- 172.21.15.87:0/2202355699 >> v2:172.21.15.169:3300/0 conn(0x7f41cc0b06b0 0x7f41cc0b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 --2- 172.21.15.87:0/2202355699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc15b950 0x7f41cc161750 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- 172.21.15.87:0/2202355699 >> 172.21.15.87:0/2202355699 conn(0x7f41cc0b33e0 msgr2=0x7f41cc0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:41.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- 172.21.15.87:0/2202355699 shutdown_connections 2024-04-25T10:23:41.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- 172.21.15.87:0/2202355699 wait complete. 2024-04-25T10:23:41.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 Processor -- start 2024-04-25T10:23:41.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- start start 2024-04-25T10:23:41.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41cc0b06b0 0x7f41cc150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc0b46e0 0x7f41cc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41cc15b950 0x7f41cc15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc16ae80 con 0x7f41cc0b46e0 2024-04-25T10:23:41.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc16ad00 con 0x7f41cc0b06b0 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc16b000 con 0x7f41cc15b950 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.985+0000 7f41c9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc0b46e0 0x7f41cc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc0b46e0 0x7f41cc15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53224/0 (socket says 172.21.15.87:53224) 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 -- 172.21.15.87:0/770030678 learned_addr learned my addr 172.21.15.87:0/770030678 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41ca800640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41cc15b950 0x7f41cc15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9e00640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41cc0b06b0 0x7f41cc150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 -- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41cc15b950 msgr2=0x7f41cc15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41cc15b950 0x7f41cc15ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 -- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41cc0b06b0 msgr2=0x7f41cc150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41cc0b06b0 0x7f41cc150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 -- 172.21.15.87:0/770030678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc151d00 con 0x7f41cc0b46e0 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9e00640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41cc0b06b0 0x7f41cc150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41ca800640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41cc15b950 0x7f41cc15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:41.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9400640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc0b46e0 0x7f41cc15c8e0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f41b800c370 tx=0x7f41b800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:41.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c2000640 1 -- 172.21.15.87:0/770030678 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b80052d0 con 0x7f41cc0b46e0 2024-04-25T10:23:41.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c2000640 1 -- 172.21.15.87:0/770030678 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b8005470 con 0x7f41cc0b46e0 2024-04-25T10:23:41.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c2000640 1 -- 172.21.15.87:0/770030678 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8007260 con 0x7f41cc0b46e0 2024-04-25T10:23:41.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc151fe0 con 0x7f41cc0b46e0 2024-04-25T10:23:41.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc1525a0 con 0x7f41cc0b46e0 2024-04-25T10:23:41.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc075120 con 0x7f41cc0b46e0 2024-04-25T10:23:41.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c2000640 1 -- 172.21.15.87:0/770030678 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41b8007400 con 0x7f41cc0b46e0 2024-04-25T10:23:41.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c2000640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f41ac066f90 0x7f41ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:41.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c2000640 1 -- 172.21.15.87:0/770030678 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f41b8084210 con 0x7f41cc0b46e0 2024-04-25T10:23:41.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.989+0000 7f41c9e00640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f41ac066f90 0x7f41ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:41.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.993+0000 7f41c9e00640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f41ac066f90 0x7f41ac069450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0ad000 tx=0x7f41bc0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:41.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:41.993+0000 7f41c2000640 1 -- 172.21.15.87:0/770030678 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41b8055480 con 0x7f41cc0b46e0 2024-04-25T10:23:42.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.145+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f41cc153790 con 0x7f41cc0b46e0 2024-04-25T10:23:42.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.149+0000 7f41c2000640 1 -- 172.21.15.87:0/770030678 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f41b8054d80 con 0x7f41cc0b46e0 2024-04-25T10:23:42.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f41ac066f90 msgr2=0x7f41ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f41ac066f90 0x7f41ac069450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0ad000 tx=0x7f41bc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc0b46e0 msgr2=0x7f41cc15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc0b46e0 0x7f41cc15c8e0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f41b800c370 tx=0x7f41b800c830 comp rx=0 tx=0).stop 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 shutdown_connections 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f41ac066f90 0x7f41ac069450 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41cc15b950 0x7f41cc15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41cc0b46e0 0x7f41cc15c8e0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 --2- 172.21.15.87:0/770030678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41cc0b06b0 0x7f41cc150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 >> 172.21.15.87:0/770030678 conn(0x7f41cc0b33e0 msgr2=0x7f41cc157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 shutdown_connections 2024-04-25T10:23:42.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.153+0000 7f41cb200640 1 -- 172.21.15.87:0/770030678 wait complete. 2024-04-25T10:23:42.171 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:23:42.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:23:42.222 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:23:42.325 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:23:42.326 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:23:42.326 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:42.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:23:42.335 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:23:42.335 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:42.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:23:42.344 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:23:42.344 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:42.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:23:42.386 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:23:42.386 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:42.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:23:42.395 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:23:42.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f9a600640 1 Processor -- start 2024-04-25T10:23:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f9a600640 1 -- start start 2024-04-25T10:23:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f9a600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f940d1820 0x7f5f940d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f9a600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f940b1af0 con 0x7f5f940b15b0 2024-04-25T10:23:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f9a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f940b1dd0 con 0x7f5f9415e050 2024-04-25T10:23:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f9a600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f940b20b0 con 0x7f5f940d1820 2024-04-25T10:23:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f93e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5f940b15b0 0x7f5f940cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39732/0 (socket says 172.21.15.87:39732) 2024-04-25T10:23:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f93e00640 1 -- 172.21.15.87:0/828156893 learned_addr learned my addr 172.21.15.87:0/828156893 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946194755 0 0) 0x7f5f940b1af0 con 0x7f5f940b15b0 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f93400640 1 --2- 172.21.15.87:0/828156893 >> v2:172.21.15.169:3300/0 conn(0x7f5f940d1820 0x7f5f940d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f78003530 con 0x7f5f940b15b0 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983386520 0 0) 0x7f5f78003530 con 0x7f5f940b15b0 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 >> v2:172.21.15.169:3300/0 conn(0x7f5f940d1820 msgr2=0x7f5f940d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 --2- 172.21.15.87:0/828156893 >> v2:172.21.15.169:3300/0 conn(0x7f5f940d1820 0x7f5f940d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 >> v1:172.21.15.169:6789/0 conn(0x7f5f9415e050 legacy=0x7f5f940b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940b2390 con 0x7f5f940b15b0 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5f8c001680 con 0x7f5f940b15b0 2024-04-25T10:23:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 >> v1:172.21.15.87:6789/0 conn(0x7f5f940b15b0 legacy=0x7f5f940cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f780039c0 0x7f5f78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f78004810 0x7f5f78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f780051d0 0x7f5f7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940b20b0 con 0x7f5f78004810 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7800c070 con 0x7f5f780051d0 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7800c270 con 0x7f5f780039c0 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.565+0000 7f5f8aa00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f78004810 0x7f5f78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f93400640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f780039c0 0x7f5f78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f93e00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f780051d0 0x7f5f7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f8aa00640 1 -- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f780039c0 msgr2=0x7f5f78003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f8aa00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f780039c0 0x7f5f78003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f8aa00640 1 -- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f780051d0 msgr2=0x7f5f7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f8aa00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f780051d0 0x7f5f7800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f8aa00640 1 -- 172.21.15.87:0/828156893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f78003530 con 0x7f5f78004810 2024-04-25T10:23:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f93e00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f780051d0 0x7f5f7800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f93e00640 1 -- 172.21.15.87:0/828156893 reap_dead start 2024-04-25T10:23:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f8aa00640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f78004810 0x7f5f78004c90 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001750 tx=0x7f5f7c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f644c9f2633d4658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0013a0 con 0x7f5f78004810 2024-04-25T10:23:42.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c0052f0 con 0x7f5f78004810 2024-04-25T10:23:42.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f92a00640 1 -- 172.21.15.87:0/828156893 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c003840 con 0x7f5f78004810 2024-04-25T10:23:42.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f9a600640 1 -- 172.21.15.87:0/828156893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f78004810 msgr2=0x7f5f78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f9a600640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f78004810 0x7f5f78004c90 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001750 tx=0x7f5f7c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f9a600640 1 -- 172.21.15.87:0/828156893 shutdown_connections 2024-04-25T10:23:42.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f9a600640 1 --2- 172.21.15.87:0/828156893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f78004810 0x7f5f78004c90 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f9a600640 1 -- 172.21.15.87:0/828156893 >> 172.21.15.87:0/828156893 conn(0x7f5f9409a9d0 msgr2=0x7f5f9409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:42.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.569+0000 7f5f9a600640 1 -- 172.21.15.87:0/828156893 shutdown_connections 2024-04-25T10:23:42.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 -- 172.21.15.87:0/828156893 wait complete. 2024-04-25T10:23:42.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 Processor -- start 2024-04-25T10:23:42.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 -- start start 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f78004810 0x7f5f9409ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f941f4160 0x7f5f940b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f940b4ce0 0x7f5f940c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940b1020 con 0x7f5f940b4ce0 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940b0ea0 con 0x7f5f941f4160 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940b11a0 con 0x7f5f78004810 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f940b4ce0 0x7f5f940c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f940b4ce0 0x7f5f940c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53242/0 (socket says 172.21.15.87:53242) 2024-04-25T10:23:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 -- 172.21.15.87:0/301578587 learned_addr learned my addr 172.21.15.87:0/301578587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93400640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f78004810 0x7f5f9409ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f8aa00640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f941f4160 0x7f5f940b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 -- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f78004810 msgr2=0x7f5f9409ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f78004810 0x7f5f9409ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 -- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f941f4160 msgr2=0x7f5f940b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f941f4160 0x7f5f940b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 -- 172.21.15.87:0/301578587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940cac50 con 0x7f5f940b4ce0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f8aa00640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f941f4160 0x7f5f940b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f93e00640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f940b4ce0 0x7f5f940c9ed0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f5f940cf340 tx=0x7f5f8c010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f8be00640 1 -- 172.21.15.87:0/301578587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c01a070 con 0x7f5f940b4ce0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f8be00640 1 -- 172.21.15.87:0/301578587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c011040 con 0x7f5f940b4ce0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f940ccf80 con 0x7f5f940b4ce0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f8be00640 1 -- 172.21.15.87:0/301578587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00ecc0 con 0x7f5f940b4ce0 2024-04-25T10:23:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.573+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f940cd540 con 0x7f5f940b4ce0 2024-04-25T10:23:42.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.577+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9404ef30 con 0x7f5f940b4ce0 2024-04-25T10:23:42.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.577+0000 7f5f8be00640 1 -- 172.21.15.87:0/301578587 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f8c01d070 con 0x7f5f940b4ce0 2024-04-25T10:23:42.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.577+0000 7f5f8be00640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f70066f60 0x7f5f70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.577+0000 7f5f8be00640 1 -- 172.21.15.87:0/301578587 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5f8c0873a0 con 0x7f5f940b4ce0 2024-04-25T10:23:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.577+0000 7f5f93400640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f70066f60 0x7f5f70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:42.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.577+0000 7f5f93400640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f70066f60 0x7f5f70069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5f9406a000 tx=0x7f5f8400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:42.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.581+0000 7f5f8be00640 1 -- 172.21.15.87:0/301578587 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f8c00ee60 con 0x7f5f940b4ce0 2024-04-25T10:23:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.765+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f9406be50 con 0x7f5f940b4ce0 2024-04-25T10:23:42.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f8be00640 1 -- 172.21.15.87:0/301578587 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3702 (secure 0 0 0) 0x7f5f8c058050 con 0x7f5f940b4ce0 2024-04-25T10:23:42.767 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:42.767 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/278619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":278619734},{"type":"v1","addr":"172.21.15.169:6835","nonce":278619734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/130741627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":130741627},{"type":"v1","addr":"172.21.15.169:6833","nonce":130741627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":9966,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2468213277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2468213277},{"type":"v1","addr":"172.21.15.87:6835","nonce":2468213277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105},{"gid":9969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3559717443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3559717443},{"type":"v1","addr":"172.21.15.87:6837","nonce":3559717443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[]} 2024-04-25T10:23:42.767 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 105 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f70066f60 msgr2=0x7f5f70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f70066f60 0x7f5f70069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5f9406a000 tx=0x7f5f8400b040 comp rx=0 tx=0).stop 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f940b4ce0 msgr2=0x7f5f940c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f940b4ce0 0x7f5f940c9ed0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f5f940cf340 tx=0x7f5f8c010ca0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 shutdown_connections 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5f70066f60 0x7f5f70069420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f940b4ce0 0x7f5f940c9ed0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f941f4160 0x7f5f940b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 --2- 172.21.15.87:0/301578587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f78004810 0x7f5f9409ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 >> 172.21.15.87:0/301578587 conn(0x7f5f9409a9d0 msgr2=0x7f5f940ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 shutdown_connections 2024-04-25T10:23:42.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:42.769+0000 7f5f9a600640 1 -- 172.21.15.87:0/301578587 wait complete. 2024-04-25T10:23:42.788 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:23:42.788 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:23:42.788 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:23:42.788 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:23:42.788 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:23:42.788 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:23:42.789 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:23:42.789 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:23:42.789 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:23:42.789 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:23:42.789 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:23:42.789 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:23:42.789 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:23:42.789 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:23:42.790 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:23:42.790 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:23:42.800 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:23:42.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:23:42.801 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:23:42.801 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:23:42.804 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:23:42.804 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:23:42.805 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:23:42.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:23:42.845 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:23:42.845 DEBUG:teuthology.parallel:result is None 2024-04-25T10:23:42.846 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:23:42.846 DEBUG:teuthology.parallel:result is None 2024-04-25T10:23:42.853 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:23:42.853 DEBUG:teuthology.parallel:result is None 2024-04-25T10:23:42.855 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:23:42.855 DEBUG:teuthology.parallel:result is None 2024-04-25T10:23:42.855 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:23:42.931 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:23:42.931 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:23:42.941 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:23:42.942 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:23:43.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.000+0000 7f4975800640 1 Processor -- start 2024-04-25T10:23:43.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.000+0000 7f4975800640 1 -- start start 2024-04-25T10:23:43.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f4975800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f49701550b0 0x7f49700cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f4975800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4970071050 con 0x7f49700cfe70 2024-04-25T10:23:43.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f4975800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497015d040 con 0x7f497015b910 2024-04-25T10:23:43.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f4975800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497015bd80 con 0x7f49701550b0 2024-04-25T10:23:43.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496e000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f49700cfe70 0x7f497015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39756/0 (socket says 172.21.15.87:39756) 2024-04-25T10:23:43.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496e000640 1 -- 172.21.15.87:0/1348041528 learned_addr learned my addr 172.21.15.87:0/1348041528 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164739497 0 0) 0x7f4970071050 con 0x7f49700cfe70 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496ea00640 1 --2- 172.21.15.87:0/1348041528 >> v2:172.21.15.169:3300/0 conn(0x7f49701550b0 0x7f49700cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4954003530 con 0x7f49700cfe70 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1733061184 0 0) 0x7f4954003530 con 0x7f49700cfe70 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 >> v2:172.21.15.169:3300/0 conn(0x7f49701550b0 msgr2=0x7f49700cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 --2- 172.21.15.87:0/1348041528 >> v2:172.21.15.169:3300/0 conn(0x7f49701550b0 0x7f49700cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 >> v1:172.21.15.169:6789/0 conn(0x7f497015b910 legacy=0x7f49700b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700712f0 con 0x7f49700cfe70 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f495c001680 con 0x7f49700cfe70 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 >> v1:172.21.15.87:6789/0 conn(0x7f49700cfe70 legacy=0x7f497015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49540039c0 0x7f4954003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4954004810 0x7f4954004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540051d0 0x7f495400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497015bd80 con 0x7f4954004810 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495400c020 con 0x7f49540039c0 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495400c220 con 0x7f49540051d0 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496ea00640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49540039c0 0x7f4954003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496e000640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4954004810 0x7f4954004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496f400640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540051d0 0x7f495400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496ea00640 1 -- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540051d0 msgr2=0x7f495400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496ea00640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540051d0 0x7f495400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496ea00640 1 -- 172.21.15.87:0/1348041528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4954004810 msgr2=0x7f4954004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496ea00640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4954004810 0x7f4954004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496ea00640 1 -- 172.21.15.87:0/1348041528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700712f0 con 0x7f49540039c0 2024-04-25T10:23:43.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496e000640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4954004810 0x7f4954004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:43.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.004+0000 7f496f400640 1 -- 172.21.15.87:0/1348041528 reap_dead start 2024-04-25T10:23:43.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f496ea00640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49540039c0 0x7f4954003de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4960009ae0 tx=0x7f496001ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=706a968409d6c9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:43.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49600b2070 con 0x7f49540039c0 2024-04-25T10:23:43.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49600a5550 con 0x7f49540039c0 2024-04-25T10:23:43.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f496d600640 1 -- 172.21.15.87:0/1348041528 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49600096b0 con 0x7f49540039c0 2024-04-25T10:23:43.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 -- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49540039c0 msgr2=0x7f4954003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:43.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49540039c0 0x7f4954003de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4960009ae0 tx=0x7f496001ca70 comp rx=0 tx=0).stop 2024-04-25T10:23:43.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 -- 172.21.15.87:0/1348041528 shutdown_connections 2024-04-25T10:23:43.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 --2- 172.21.15.87:0/1348041528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49540039c0 0x7f4954003de0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 -- 172.21.15.87:0/1348041528 >> 172.21.15.87:0/1348041528 conn(0x7f4970066b70 msgr2=0x7f4970066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:43.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 -- 172.21.15.87:0/1348041528 shutdown_connections 2024-04-25T10:23:43.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 -- 172.21.15.87:0/1348041528 wait complete. 2024-04-25T10:23:43.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 Processor -- start 2024-04-25T10:23:43.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.008+0000 7f4975800640 1 -- start start 2024-04-25T10:23:43.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540039c0 0x7f4970157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49701f7380 0x7f49701f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49700cea60 0x7f49700ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970071b90 con 0x7f49700cea60 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700718b0 con 0x7f49701f7380 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49700720b0 con 0x7f49540039c0 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49700cea60 0x7f49700ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49700cea60 0x7f49700ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53296/0 (socket says 172.21.15.87:53296) 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 -- 172.21.15.87:0/1685917178 learned_addr learned my addr 172.21.15.87:0/1685917178 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496ea00640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540039c0 0x7f4970157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496e000640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49701f7380 0x7f49701f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 -- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540039c0 msgr2=0x7f4970157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540039c0 0x7f4970157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 -- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49701f7380 msgr2=0x7f49701f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49701f7380 0x7f49701f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 -- 172.21.15.87:0/1685917178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700c4880 con 0x7f49700cea60 2024-04-25T10:23:43.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f496f400640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49700cea60 0x7f49700ceee0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f4970070930 tx=0x7f496800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:43.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4966a00640 1 -- 172.21.15.87:0/1685917178 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968003b90 con 0x7f49700cea60 2024-04-25T10:23:43.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4966a00640 1 -- 172.21.15.87:0/1685917178 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968018070 con 0x7f49700cea60 2024-04-25T10:23:43.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4966a00640 1 -- 172.21.15.87:0/1685917178 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680050d0 con 0x7f49700cea60 2024-04-25T10:23:43.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49700c4b60 con 0x7f49700cea60 2024-04-25T10:23:43.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49701559a0 con 0x7f49700cea60 2024-04-25T10:23:43.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4970155ba0 con 0x7f49700cea60 2024-04-25T10:23:43.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.012+0000 7f4966a00640 1 -- 172.21.15.87:0/1685917178 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4968005270 con 0x7f49700cea60 2024-04-25T10:23:43.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.016+0000 7f4966a00640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4950066f60 0x7f4950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:43.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.016+0000 7f4966a00640 1 -- 172.21.15.87:0/1685917178 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f49680849d0 con 0x7f49700cea60 2024-04-25T10:23:43.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.016+0000 7f496ea00640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4950066f60 0x7f4950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:43.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.016+0000 7f496ea00640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4950066f60 0x7f4950069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f49600af000 tx=0x7f49600046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:43.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.020+0000 7f4966a00640 1 -- 172.21.15.87:0/1685917178 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4970155ba0 con 0x7f49700cea60 2024-04-25T10:23:43.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.184+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f497004ef30 con 0x7f49700cea60 2024-04-25T10:23:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.832+0000 7f4966a00640 1 -- 172.21.15.87:0/1685917178 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4968055ce0 con 0x7f49700cea60 2024-04-25T10:23:43.834 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:23:43.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4950066f60 msgr2=0x7f4950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:43.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4950066f60 0x7f4950069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f49600af000 tx=0x7f49600046d0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49700cea60 msgr2=0x7f49700ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:43.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49700cea60 0x7f49700ceee0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f4970070930 tx=0x7f496800fc80 comp rx=0 tx=0).stop 2024-04-25T10:23:43.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 shutdown_connections 2024-04-25T10:23:43.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4950066f60 0x7f4950069420 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49700cea60 0x7f49700ceee0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49701f7380 0x7f49701f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 --2- 172.21.15.87:0/1685917178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540039c0 0x7f4970157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:43.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 >> 172.21.15.87:0/1685917178 conn(0x7f4970066b70 msgr2=0x7f497006d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:43.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 shutdown_connections 2024-04-25T10:23:43.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:43.836+0000 7f4975800640 1 -- 172.21.15.87:0/1685917178 wait complete. 2024-04-25T10:23:43.863 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:23:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.000+0000 7f5bee400640 1 Processor -- start 2024-04-25T10:23:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.000+0000 7f5bee400640 1 -- start start 2024-04-25T10:23:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bee400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 0x7f5be80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bee400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be8071050 con 0x7f5be80cfe70 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bee400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be815d040 con 0x7f5be815b910 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bee400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be815bda0 con 0x7f5be81550b0 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5be80cfe70 0x7f5be815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39768/0 (socket says 172.21.15.87:39768) 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 -- 172.21.15.87:0/388345600 learned_addr learned my addr 172.21.15.87:0/388345600 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be7400640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 0x7f5be80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613204198 0 0) 0x7f5be8071050 con 0x7f5be80cfe70 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bcc003530 con 0x7f5be80cfe70 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259065598 0 0) 0x7f5be815d040 con 0x7f5be815b910 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be8071050 con 0x7f5be815b910 2024-04-25T10:23:44.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2768073407 0 0) 0x7f5bcc003530 con 0x7f5be80cfe70 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 msgr2=0x7f5be80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 0x7f5be80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 >> v1:172.21.15.169:6789/0 conn(0x7f5be815b910 legacy=0x7f5be80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be80712f0 con 0x7f5be80cfe70 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5bd0001680 con 0x7f5be80cfe70 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 >> v1:172.21.15.87:6789/0 conn(0x7f5be80cfe70 legacy=0x7f5be815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bcc003b80 0x7f5bcc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5bcc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 0x7f5bcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc00ba80 con 0x7f5bcc004810 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc00de10 con 0x7f5bcc003b80 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc00e010 con 0x7f5be81550b0 2024-04-25T10:23:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5bcc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be7400640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bcc003b80 0x7f5bcc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5be7e00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 0x7f5bcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 -- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 msgr2=0x7f5bcc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be81550b0 0x7f5bcc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 -- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bcc003b80 msgr2=0x7f5bcc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bcc003b80 0x7f5bcc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.004+0000 7f5bdea00640 1 -- 172.21.15.87:0/388345600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc003530 con 0x7f5bcc004810 2024-04-25T10:23:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bdea00640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5bcc004c90 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c0e0 tx=0x7f5bd001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4177844b95402a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0007c50 con 0x7f5bcc004810 2024-04-25T10:23:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd00a9070 con 0x7f5bcc004810 2024-04-25T10:23:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5be6a00640 1 -- 172.21.15.87:0/388345600 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0005d60 con 0x7f5bcc004810 2024-04-25T10:23:44.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 -- 172.21.15.87:0/388345600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 msgr2=0x7f5bcc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 --2- 172.21.15.87:0/388345600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5bcc004c90 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c0e0 tx=0x7f5bd001cb40 comp rx=0 tx=0).stop 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5be7e00640 1 -- 172.21.15.87:0/388345600 reap_dead start 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 -- 172.21.15.87:0/388345600 shutdown_connections 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 -- 172.21.15.87:0/388345600 >> 172.21.15.87:0/388345600 conn(0x7f5be8066b70 msgr2=0x7f5be8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 -- 172.21.15.87:0/388345600 shutdown_connections 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 -- 172.21.15.87:0/388345600 wait complete. 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 Processor -- start 2024-04-25T10:23:44.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.008+0000 7f5bee400640 1 -- start start 2024-04-25T10:23:44.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5be8151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be8157790 0x7f5be8157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be80b3510 0x7f5be80b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be8071b90 con 0x7f5bcc004810 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80718b0 con 0x7f5be80b3510 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80720b0 con 0x7f5be8157790 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5be8151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5be8151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53324/0 (socket says 172.21.15.87:53324) 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 -- 172.21.15.87:0/833555032 learned_addr learned my addr 172.21.15.87:0/833555032 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 -- 172.21.15.87:0/833555032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be8157790 msgr2=0x7f5be8157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be8157790 0x7f5be8157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 -- 172.21.15.87:0/833555032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be80b3510 msgr2=0x7f5be80b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be80b3510 0x7f5be80b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 -- 172.21.15.87:0/833555032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be81582c0 con 0x7f5bcc004810 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5be7400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5be8151630 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80119d0 tx=0x7f5bd8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bdfe00640 1 -- 172.21.15.87:0/833555032 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800fb50 con 0x7f5bcc004810 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bdfe00640 1 -- 172.21.15.87:0/833555032 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd800fcf0 con 0x7f5bcc004810 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bdfe00640 1 -- 172.21.15.87:0/833555032 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800fea0 con 0x7f5bcc004810 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be81f2ae0 con 0x7f5bcc004810 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be81f30a0 con 0x7f5bcc004810 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be8152180 con 0x7f5bcc004810 2024-04-25T10:23:44.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bdfe00640 1 -- 172.21.15.87:0/833555032 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bd8012040 con 0x7f5bcc004810 2024-04-25T10:23:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.012+0000 7f5bdfe00640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bc4066f60 0x7f5bc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.016+0000 7f5bdfe00640 1 -- 172.21.15.87:0/833555032 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5bd80852e0 con 0x7f5bcc004810 2024-04-25T10:23:44.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.016+0000 7f5bdea00640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bc4066f60 0x7f5bc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.016+0000 7f5bdea00640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bc4066f60 0x7f5bc4069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00ab000 tx=0x7f5bd0001980 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.016+0000 7f5bdfe00640 1 -- 172.21.15.87:0/833555032 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bd8019030 con 0x7f5bcc004810 2024-04-25T10:23:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.228+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5be804ef30 con 0x7f5bcc004810 2024-04-25T10:23:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.228+0000 7f5bdfe00640 1 -- 172.21.15.87:0/833555032 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5bd80566f0 con 0x7f5bcc004810 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.229 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.231 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:23:44.232 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.233 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:23:44.234 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:23:44.236 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.238 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.240 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:23:44.241 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:23:44.242 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.243 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:44.243 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:23:44.243 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bc4066f60 msgr2=0x7f5bc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bc4066f60 0x7f5bc4069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00ab000 tx=0x7f5bd0001980 comp rx=0 tx=0).stop 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 msgr2=0x7f5be8151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5be8151630 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80119d0 tx=0x7f5bd8011e90 comp rx=0 tx=0).stop 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 shutdown_connections 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5bc4066f60 0x7f5bc4069420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be80b3510 0x7f5be80b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be8157790 0x7f5be8157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 --2- 172.21.15.87:0/833555032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bcc004810 0x7f5be8151630 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 >> 172.21.15.87:0/833555032 conn(0x7f5be8066b70 msgr2=0x7f5be806ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 shutdown_connections 2024-04-25T10:23:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.232+0000 7f5bee400640 1 -- 172.21.15.87:0/833555032 wait complete. 2024-04-25T10:23:44.251 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:44.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84fc00640 1 Processor -- start 2024-04-25T10:23:44.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84fc00640 1 -- start start 2024-04-25T10:23:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84fc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8481550b0 0x7fa8480cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa8480cfe70 0x7fa84815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84fc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa848071050 con 0x7fa84815b740 2024-04-25T10:23:44.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84815d040 con 0x7fa8480cfe70 2024-04-25T10:23:44.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84fc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8480712f0 con 0x7fa8481550b0 2024-04-25T10:23:44.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa83fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa8480cfe70 0x7fa84815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa83fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa8480cfe70 0x7fa84815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53334/0 (socket says 172.21.15.87:53334) 2024-04-25T10:23:44.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa83fe00640 1 -- 172.21.15.87:0/3669932013 learned_addr learned my addr 172.21.15.87:0/3669932013 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:44.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa84d200640 1 --2- 172.21.15.87:0/3669932013 >> v2:172.21.15.169:3300/0 conn(0x7fa8481550b0 0x7fa8480cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa83fe00640 1 -- 172.21.15.87:0/3669932013 >> v2:172.21.15.169:3300/0 conn(0x7fa8481550b0 msgr2=0x7fa8480cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa83fe00640 1 --2- 172.21.15.87:0/3669932013 >> v2:172.21.15.169:3300/0 conn(0x7fa8481550b0 0x7fa8480cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa83fe00640 1 -- 172.21.15.87:0/3669932013 >> v1:172.21.15.169:6789/0 conn(0x7fa84815b740 legacy=0x7fa84806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.388+0000 7fa83fe00640 1 -- 172.21.15.87:0/3669932013 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8480715d0 con 0x7fa8480cfe70 2024-04-25T10:23:44.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa83fe00640 1 --2- 172.21.15.87:0/3669932013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84815b200 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fa830009870 tx=0x7fa83001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d20bd8d6c8e4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa847e00640 1 -- 172.21.15.87:0/3669932013 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8300b0070 con 0x7fa8480cfe70 2024-04-25T10:23:44.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa847e00640 1 -- 172.21.15.87:0/3669932013 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8300ab600 con 0x7fa8480cfe70 2024-04-25T10:23:44.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa847e00640 1 -- 172.21.15.87:0/3669932013 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8300ab9d0 con 0x7fa8480cfe70 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 -- 172.21.15.87:0/3669932013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 msgr2=0x7fa84815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 --2- 172.21.15.87:0/3669932013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84815b200 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fa830009870 tx=0x7fa83001c520 comp rx=0 tx=0).stop 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 -- 172.21.15.87:0/3669932013 shutdown_connections 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 --2- 172.21.15.87:0/3669932013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84815b200 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 --2- 172.21.15.87:0/3669932013 >> v2:172.21.15.169:3300/0 conn(0x7fa8481550b0 0x7fa8480cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 -- 172.21.15.87:0/3669932013 >> 172.21.15.87:0/3669932013 conn(0x7fa848066b70 msgr2=0x7fa848066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 -- 172.21.15.87:0/3669932013 shutdown_connections 2024-04-25T10:23:44.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.392+0000 7fa84fc00640 1 -- 172.21.15.87:0/3669932013 wait complete. 2024-04-25T10:23:44.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 Processor -- start 2024-04-25T10:23:44.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 -- start start 2024-04-25T10:23:44.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84814b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8481550b0 0x7fa8480af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa84815b740 0x7fa8480af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815bcd0 con 0x7fa8480cfe70 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815bb50 con 0x7fa8481550b0 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815c030 con 0x7fa84815b740 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84814b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84814b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53338/0 (socket says 172.21.15.87:53338) 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 -- 172.21.15.87:0/154381087 learned_addr learned my addr 172.21.15.87:0/154381087 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa83fe00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8481550b0 0x7fa8480af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84dc00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa84815b740 0x7fa8480af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 -- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa84815b740 msgr2=0x7fa8480af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa84815b740 0x7fa8480af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 -- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8481550b0 msgr2=0x7fa8480af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8481550b0 0x7fa8480af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 -- 172.21.15.87:0/154381087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848155490 con 0x7fa8480cfe70 2024-04-25T10:23:44.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84d200640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84814b5b0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa83800e930 tx=0x7fa83800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa845600640 1 -- 172.21.15.87:0/154381087 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838017070 con 0x7fa8480cfe70 2024-04-25T10:23:44.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa845600640 1 -- 172.21.15.87:0/154381087 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83800ca00 con 0x7fa8480cfe70 2024-04-25T10:23:44.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa845600640 1 -- 172.21.15.87:0/154381087 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83800cdb0 con 0x7fa8480cfe70 2024-04-25T10:23:44.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8480cd170 con 0x7fa8480cfe70 2024-04-25T10:23:44.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84814f2e0 con 0x7fa8480cfe70 2024-04-25T10:23:44.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.396+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84804ef30 con 0x7fa8480cfe70 2024-04-25T10:23:44.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.400+0000 7fa845600640 1 -- 172.21.15.87:0/154381087 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa83801a070 con 0x7fa8480cfe70 2024-04-25T10:23:44.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.400+0000 7fa845600640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa824066f60 0x7fa824069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.400+0000 7fa845600640 1 -- 172.21.15.87:0/154381087 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8380850e0 con 0x7fa8480cfe70 2024-04-25T10:23:44.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.400+0000 7fa83fe00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa824066f60 0x7fa824069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.400+0000 7fa83fe00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa824066f60 0x7fa824069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa8300ad000 tx=0x7fa830008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.404+0000 7fa845600640 1 -- 172.21.15.87:0/154381087 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa838056500 con 0x7fa8480cfe70 2024-04-25T10:23:44.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.560+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8480b1030 con 0x7fa8480cfe70 2024-04-25T10:23:44.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.560+0000 7fa845600640 1 -- 172.21.15.87:0/154381087 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+12889 (secure 0 0 0) 0x7fa838055e00 con 0x7fa8480cfe70 2024-04-25T10:23:44.561 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:44.562 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":147,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:43.848308+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa824066f60 msgr2=0x7fa824069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa824066f60 0x7fa824069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa8300ad000 tx=0x7fa830008040 comp rx=0 tx=0).stop 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 msgr2=0x7fa84814b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84814b5b0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa83800e930 tx=0x7fa83800edf0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 shutdown_connections 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa824066f60 0x7fa824069420 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa84815b740 0x7fa8480af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8481550b0 0x7fa8480af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 --2- 172.21.15.87:0/154381087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8480cfe70 0x7fa84814b5b0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 >> 172.21.15.87:0/154381087 conn(0x7fa848066b70 msgr2=0x7fa84806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 shutdown_connections 2024-04-25T10:23:44.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.564+0000 7fa84fc00640 1 -- 172.21.15.87:0/154381087 wait complete. 2024-04-25T10:23:44.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:23:44.584 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:23:44.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.720+0000 7f1c7e200640 1 Processor -- start 2024-04-25T10:23:44.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.720+0000 7f1c7e200640 1 -- start start 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c7e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c7815bca0 0x7f1c780cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c7e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c780d07b0 0x7f1c780ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c7e200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c780cc9d0 con 0x7f1c7815f760 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c7e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7815c080 con 0x7f1c780d07b0 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c7e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780cddf0 con 0x7f1c7815bca0 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c77e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c780d07b0 0x7f1c780ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c76a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1c7815f760 0x7f1c780d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39780/0 (socket says 172.21.15.87:39780) 2024-04-25T10:23:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c76a00640 1 -- 172.21.15.87:0/4030757903 learned_addr learned my addr 172.21.15.87:0/4030757903 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:44.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c76000640 1 -- 172.21.15.87:0/4030757903 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030904462 0 0) 0x7f1c780cc9d0 con 0x7f1c7815f760 2024-04-25T10:23:44.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c77400640 1 --2- 172.21.15.87:0/4030757903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c7815bca0 0x7f1c780cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c77e00640 1 -- 172.21.15.87:0/4030757903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c7815bca0 msgr2=0x7f1c780cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:44.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c77e00640 1 --2- 172.21.15.87:0/4030757903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c7815bca0 0x7f1c780cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c77e00640 1 -- 172.21.15.87:0/4030757903 >> v1:172.21.15.87:6789/0 conn(0x7f1c7815f760 legacy=0x7f1c780d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c77e00640 1 -- 172.21.15.87:0/4030757903 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7815fc40 con 0x7f1c780d07b0 2024-04-25T10:23:44.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.724+0000 7f1c77400640 1 --2- 172.21.15.87:0/4030757903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c7815bca0 0x7f1c780cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:44.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c77e00640 1 --2- 172.21.15.87:0/4030757903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c780d07b0 0x7f1c780ce280 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1c700097c0 tx=0x7f1c7001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15f93070ddfc7be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c76000640 1 -- 172.21.15.87:0/4030757903 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a7070 con 0x7f1c780d07b0 2024-04-25T10:23:44.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c76000640 1 -- 172.21.15.87:0/4030757903 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c700b0070 con 0x7f1c780d07b0 2024-04-25T10:23:44.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c76000640 1 -- 172.21.15.87:0/4030757903 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700ab890 con 0x7f1c780d07b0 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 -- 172.21.15.87:0/4030757903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c780d07b0 msgr2=0x7f1c780ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 --2- 172.21.15.87:0/4030757903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c780d07b0 0x7f1c780ce280 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1c700097c0 tx=0x7f1c7001c470 comp rx=0 tx=0).stop 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 -- 172.21.15.87:0/4030757903 shutdown_connections 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 --2- 172.21.15.87:0/4030757903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c780d07b0 0x7f1c780ce280 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 --2- 172.21.15.87:0/4030757903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c7815bca0 0x7f1c780cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 -- 172.21.15.87:0/4030757903 >> 172.21.15.87:0/4030757903 conn(0x7f1c78072290 msgr2=0x7f1c780726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 -- 172.21.15.87:0/4030757903 shutdown_connections 2024-04-25T10:23:44.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 -- 172.21.15.87:0/4030757903 wait complete. 2024-04-25T10:23:44.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 Processor -- start 2024-04-25T10:23:44.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 -- start start 2024-04-25T10:23:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c780d07b0 0x7f1c78154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7815bca0 0x7f1c780b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.728+0000 7f1c7e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c7815f760 0x7f1c780b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c7e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780cefd0 con 0x7f1c7815f760 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c7e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780d0f00 con 0x7f1c7815bca0 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c7e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780cf150 con 0x7f1c780d07b0 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c7815f760 0x7f1c780b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c7815f760 0x7f1c780b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53346/0 (socket says 172.21.15.87:53346) 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 -- 172.21.15.87:0/1401043367 learned_addr learned my addr 172.21.15.87:0/1401043367 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 -- 172.21.15.87:0/1401043367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c780d07b0 msgr2=0x7f1c78154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77400640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c780d07b0 0x7f1c78154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c780d07b0 0x7f1c78154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 -- 172.21.15.87:0/1401043367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7815bca0 msgr2=0x7f1c780b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7815bca0 0x7f1c780b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 -- 172.21.15.87:0/1401043367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c78155150 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c77e00640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c7815f760 0x7f1c780b2280 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1c7001a0a0 tx=0x7f1c7001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c6f400640 1 -- 172.21.15.87:0/1401043367 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700bf610 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c6f400640 1 -- 172.21.15.87:0/1401043367 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c700bf7b0 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c78157290 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c6f400640 1 -- 172.21.15.87:0/1401043367 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700bfba0 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7815b300 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7804ef30 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c6f400640 1 -- 172.21.15.87:0/1401043367 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c700bd020 con 0x7f1c7815f760 2024-04-25T10:23:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.732+0000 7f1c6f400640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:44.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.736+0000 7f1c6f400640 1 -- 172.21.15.87:0/1401043367 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1c7011f5a0 con 0x7f1c7815f760 2024-04-25T10:23:44.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.736+0000 7f1c77400640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:44.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.736+0000 7f1c77400640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c58066f60 0x7f1c58069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1c68008f80 tx=0x7f1c680084d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:44.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.736+0000 7f1c6f400640 1 -- 172.21.15.87:0/1401043367 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c70016090 con 0x7f1c7815f760 2024-04-25T10:23:44.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c78072e80 con 0x7f1c7815f760 2024-04-25T10:23:44.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c6f400640 1 -- 172.21.15.87:0/1401043367 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c700f07d0 con 0x7f1c7815f760 2024-04-25T10:23:44.900 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:23:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c58066f60 msgr2=0x7f1c58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c58066f60 0x7f1c58069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1c68008f80 tx=0x7f1c680084d0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c7815f760 msgr2=0x7f1c780b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c7815f760 0x7f1c780b2280 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1c7001a0a0 tx=0x7f1c7001acb0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 shutdown_connections 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c7815f760 0x7f1c780b2280 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c7815bca0 0x7f1c780b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 --2- 172.21.15.87:0/1401043367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c780d07b0 0x7f1c78154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 >> 172.21.15.87:0/1401043367 conn(0x7f1c78072290 msgr2=0x7f1c780adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 shutdown_connections 2024-04-25T10:23:44.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:44.900+0000 7f1c7e200640 1 -- 172.21.15.87:0/1401043367 wait complete. 2024-04-25T10:23:44.921 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:23:45.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.056+0000 7fc0c6800640 1 Processor -- start 2024-04-25T10:23:45.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.056+0000 7fc0c6800640 1 -- start start 2024-04-25T10:23:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0c6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0c00cf930 0x7fc0c00cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0c6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0c6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0c00ae020 0x7fc0c00ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0c6800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c015bb90 con 0x7fc0c00ae020 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0c6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c00d0250 con 0x7fc0c00cf930 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0c6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00ad840 con 0x7fc0c0070c20 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0c00ae020 0x7fc0c00ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0c00ae020 0x7fc0c00ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53356/0 (socket says 172.21.15.87:53356) 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 -- 172.21.15.87:0/3389277025 learned_addr learned my addr 172.21.15.87:0/3389277025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bf400640 1 --2- 172.21.15.87:0/3389277025 >> v2:172.21.15.169:3300/0 conn(0x7fc0c00cf930 0x7fc0c00cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bea00640 1 --2- 172.21.15.87:0/3389277025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 -- 172.21.15.87:0/3389277025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 msgr2=0x7fc0c015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:45.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 --2- 172.21.15.87:0/3389277025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 -- 172.21.15.87:0/3389277025 >> v2:172.21.15.169:3300/0 conn(0x7fc0c00cf930 msgr2=0x7fc0c00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:45.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 --2- 172.21.15.87:0/3389277025 >> v2:172.21.15.169:3300/0 conn(0x7fc0c00cf930 0x7fc0c00cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bfe00640 1 -- 172.21.15.87:0/3389277025 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c015e9f0 con 0x7fc0c00ae020 2024-04-25T10:23:45.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bea00640 1 --2- 172.21.15.87:0/3389277025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:45.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.060+0000 7fc0bf400640 1 --2- 172.21.15.87:0/3389277025 >> v2:172.21.15.169:3300/0 conn(0x7fc0c00cf930 0x7fc0c00cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0bfe00640 1 --2- 172.21.15.87:0/3389277025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c00ae4b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00aec50 tx=0x7fc0b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0e80d5f42022d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0be000640 1 -- 172.21.15.87:0/3389277025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80aa610 con 0x7fc0c00ae020 2024-04-25T10:23:45.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0be000640 1 -- 172.21.15.87:0/3389277025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b80af070 con 0x7fc0c00ae020 2024-04-25T10:23:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0be000640 1 -- 172.21.15.87:0/3389277025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80aa9d0 con 0x7fc0c00ae020 2024-04-25T10:23:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- 172.21.15.87:0/3389277025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 msgr2=0x7fc0c00ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 --2- 172.21.15.87:0/3389277025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c00ae4b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00aec50 tx=0x7fc0b801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- 172.21.15.87:0/3389277025 shutdown_connections 2024-04-25T10:23:45.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 --2- 172.21.15.87:0/3389277025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c00ae4b0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 --2- 172.21.15.87:0/3389277025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 --2- 172.21.15.87:0/3389277025 >> v2:172.21.15.169:3300/0 conn(0x7fc0c00cf930 0x7fc0c00cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- 172.21.15.87:0/3389277025 >> 172.21.15.87:0/3389277025 conn(0x7fc0c0066b70 msgr2=0x7fc0c0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:45.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- 172.21.15.87:0/3389277025 shutdown_connections 2024-04-25T10:23:45.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- 172.21.15.87:0/3389277025 wait complete. 2024-04-25T10:23:45.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 Processor -- start 2024-04-25T10:23:45.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- start start 2024-04-25T10:23:45.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c00cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c01460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c00cf930 0x7fc0c0158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c015e6b0 con 0x7fc0c00ae020 2024-04-25T10:23:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006e130 con 0x7fc0c00cf930 2024-04-25T10:23:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0c6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0161790 con 0x7fc0c0070c20 2024-04-25T10:23:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0bea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c01460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0bea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c01460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53366/0 (socket says 172.21.15.87:53366) 2024-04-25T10:23:45.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.064+0000 7fc0bea00640 1 -- 172.21.15.87:0/465000479 learned_addr learned my addr 172.21.15.87:0/465000479 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:45.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bf400640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c00cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bfe00640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c00cf930 0x7fc0c0158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bea00640 1 -- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 msgr2=0x7fc0c00cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bea00640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c00cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bea00640 1 -- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c00cf930 msgr2=0x7fc0c0158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bea00640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c00cf930 0x7fc0c0158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bea00640 1 -- 172.21.15.87:0/465000479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0158e30 con 0x7fc0c00ae020 2024-04-25T10:23:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bea00640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c01460f0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00c7c0 tx=0x7fc0ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0b7400640 1 -- 172.21.15.87:0/465000479 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac015070 con 0x7fc0c00ae020 2024-04-25T10:23:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0b7400640 1 -- 172.21.15.87:0/465000479 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac007a30 con 0x7fc0c00ae020 2024-04-25T10:23:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c0159110 con 0x7fc0c00ae020 2024-04-25T10:23:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0b7400640 1 -- 172.21.15.87:0/465000479 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac007e00 con 0x7fc0c00ae020 2024-04-25T10:23:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c01543e0 con 0x7fc0c00ae020 2024-04-25T10:23:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c004ef30 con 0x7fc0c00ae020 2024-04-25T10:23:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0b7400640 1 -- 172.21.15.87:0/465000479 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0ac018070 con 0x7fc0c00ae020 2024-04-25T10:23:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0b7400640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0a0066f60 0x7fc0a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0b7400640 1 -- 172.21.15.87:0/465000479 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc0ac084c20 con 0x7fc0c00ae020 2024-04-25T10:23:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.068+0000 7fc0bf400640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0a0066f60 0x7fc0a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.072+0000 7fc0bf400640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0a0066f60 0x7fc0a0069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007510 tx=0x7fc0b00074a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:45.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.072+0000 7fc0b7400640 1 -- 172.21.15.87:0/465000479 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0ac055f80 con 0x7fc0c00ae020 2024-04-25T10:23:45.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.228+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc0c015d9c0 con 0x7fc0c00ae020 2024-04-25T10:23:45.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.228+0000 7fc0b7400640 1 -- 172.21.15.87:0/465000479 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc0ac055880 con 0x7fc0c00ae020 2024-04-25T10:23:45.230 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:45.230 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:23:45.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:23:45.230 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:23:45.230 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:23:45.231 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:45.231 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0a0066f60 msgr2=0x7fc0a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0a0066f60 0x7fc0a0069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007510 tx=0x7fc0b00074a0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 msgr2=0x7fc0c01460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c01460f0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00c7c0 tx=0x7fc0ac00cc80 comp rx=0 tx=0).stop 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 shutdown_connections 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc0a0066f60 0x7fc0a0069420 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0c00cf930 0x7fc0c0158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0c00ae020 0x7fc0c01460f0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 --2- 172.21.15.87:0/465000479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0c0070c20 0x7fc0c00cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 >> 172.21.15.87:0/465000479 conn(0x7fc0c0066b70 msgr2=0x7fc0c0066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:45.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 shutdown_connections 2024-04-25T10:23:45.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.232+0000 7fc0c6800640 1 -- 172.21.15.87:0/465000479 wait complete. 2024-04-25T10:23:45.251 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:23:45.251 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:23:45.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03f0600640 1 Processor -- start 2024-04-25T10:23:45.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03f0600640 1 -- start start 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03f0600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03e815c0f0 0x7f03e80b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03f0600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e806c0e0 con 0x7f03e80cf9e0 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03f0600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e815c6a0 con 0x7f03e815bd10 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03f0600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e815c980 con 0x7f03e815c0f0 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03ed200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03e815c0f0 0x7f03e80b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03ee600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f03e80cf9e0 0x7f03e806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39782/0 (socket says 172.21.15.87:39782) 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03ee600640 1 -- 172.21.15.87:0/3543110959 learned_addr learned my addr 172.21.15.87:0/3543110959 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03ed200640 1 -- 172.21.15.87:0/3543110959 >> v1:172.21.15.169:6789/0 conn(0x7f03e815bd10 legacy=0x7f03e80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03ed200640 1 -- 172.21.15.87:0/3543110959 >> v1:172.21.15.87:6789/0 conn(0x7f03e80cf9e0 legacy=0x7f03e806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.392+0000 7f03ed200640 1 -- 172.21.15.87:0/3543110959 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e815cc60 con 0x7f03e815c0f0 2024-04-25T10:23:45.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03ed200640 1 --2- 172.21.15.87:0/3543110959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e815c0f0 0x7f03e80b27e0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f03d40097c0 tx=0x7f03d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff8d6aad3cb6721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:45.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03e7e00640 1 -- 172.21.15.87:0/3543110959 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d40b1070 con 0x7f03e815c0f0 2024-04-25T10:23:45.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03e7e00640 1 -- 172.21.15.87:0/3543110959 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d40ac410 con 0x7f03e815c0f0 2024-04-25T10:23:45.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03e7e00640 1 -- 172.21.15.87:0/3543110959 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d40ac7e0 con 0x7f03e815c0f0 2024-04-25T10:23:45.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03f0600640 1 -- 172.21.15.87:0/3543110959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e815c0f0 msgr2=0x7f03e80b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03f0600640 1 --2- 172.21.15.87:0/3543110959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e815c0f0 0x7f03e80b27e0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f03d40097c0 tx=0x7f03d401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03f0600640 1 -- 172.21.15.87:0/3543110959 shutdown_connections 2024-04-25T10:23:45.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03f0600640 1 --2- 172.21.15.87:0/3543110959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e815c0f0 0x7f03e80b27e0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03f0600640 1 -- 172.21.15.87:0/3543110959 >> 172.21.15.87:0/3543110959 conn(0x7f03e8066b70 msgr2=0x7f03e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:45.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03f0600640 1 -- 172.21.15.87:0/3543110959 shutdown_connections 2024-04-25T10:23:45.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.396+0000 7f03f0600640 1 -- 172.21.15.87:0/3543110959 wait complete. 2024-04-25T10:23:45.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 Processor -- start 2024-04-25T10:23:45.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 -- start start 2024-04-25T10:23:45.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e80cf9e0 0x7f03e814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e815bd10 0x7f03e8154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e815c0f0 0x7f03e8155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e816a460 con 0x7f03e80cf9e0 2024-04-25T10:23:45.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e816a2e0 con 0x7f03e815bd10 2024-04-25T10:23:45.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e816a5e0 con 0x7f03e815c0f0 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e80cf9e0 0x7f03e814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e80cf9e0 0x7f03e814dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53372/0 (socket says 172.21.15.87:53372) 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 -- 172.21.15.87:0/3561092697 learned_addr learned my addr 172.21.15.87:0/3561092697 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03ed200640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e815bd10 0x7f03e8154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03ee600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e815c0f0 0x7f03e8155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 -- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e815c0f0 msgr2=0x7f03e8155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e815c0f0 0x7f03e8155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 -- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e815bd10 msgr2=0x7f03e8154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e815bd10 0x7f03e8154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 -- 172.21.15.87:0/3561092697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e814e5d0 con 0x7f03e80cf9e0 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03edc00640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e80cf9e0 0x7f03e814dc10 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c420 tx=0x7f03dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03e5600640 1 -- 172.21.15.87:0/3561092697 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0052d0 con 0x7f03e80cf9e0 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03e5600640 1 -- 172.21.15.87:0/3561092697 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc005470 con 0x7f03e80cf9e0 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03e5600640 1 -- 172.21.15.87:0/3561092697 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc007260 con 0x7f03e80cf9e0 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e81509b0 con 0x7f03e80cf9e0 2024-04-25T10:23:45.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e8150110 con 0x7f03e80cf9e0 2024-04-25T10:23:45.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.400+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e804ef30 con 0x7f03e80cf9e0 2024-04-25T10:23:45.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.404+0000 7f03e5600640 1 -- 172.21.15.87:0/3561092697 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03dc007400 con 0x7f03e80cf9e0 2024-04-25T10:23:45.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.404+0000 7f03e5600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03c8066f60 0x7f03c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:45.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.404+0000 7f03e5600640 1 -- 172.21.15.87:0/3561092697 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f03dc0841d0 con 0x7f03e80cf9e0 2024-04-25T10:23:45.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.404+0000 7f03ed200640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03c8066f60 0x7f03c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:45.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.404+0000 7f03ed200640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03c8066f60 0x7f03c8069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f03d40ae000 tx=0x7f03d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:45.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.408+0000 7f03e5600640 1 -- 172.21.15.87:0/3561092697 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03dc054e40 con 0x7f03e80cf9e0 2024-04-25T10:23:45.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.564+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f03e81678a0 con 0x7f03e80cf9e0 2024-04-25T10:23:45.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.836+0000 7f03e5600640 1 -- 172.21.15.87:0/3561092697 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v148) v1 ==== 138+0+0 (secure 0 0 0) 0x7f03dc058ee0 con 0x7f03e80cf9e0 2024-04-25T10:23:45.837 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:23:45.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03c8066f60 msgr2=0x7f03c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03c8066f60 0x7f03c8069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f03d40ae000 tx=0x7f03d4008040 comp rx=0 tx=0).stop 2024-04-25T10:23:45.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e80cf9e0 msgr2=0x7f03e814dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:45.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e80cf9e0 0x7f03e814dc10 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c420 tx=0x7f03dc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 shutdown_connections 2024-04-25T10:23:45.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03c8066f60 0x7f03c8069420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e815c0f0 0x7f03e8155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e815bd10 0x7f03e8154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 --2- 172.21.15.87:0/3561092697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03e80cf9e0 0x7f03e814dc10 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:45.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 >> 172.21.15.87:0/3561092697 conn(0x7f03e8066b70 msgr2=0x7f03e8068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:45.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 shutdown_connections 2024-04-25T10:23:45.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:45.840+0000 7f03f0600640 1 -- 172.21.15.87:0/3561092697 wait complete. 2024-04-25T10:23:45.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:23:46.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d76000640 1 Processor -- start 2024-04-25T10:23:46.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d76000640 1 -- start start 2024-04-25T10:23:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d76000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 0x7f2d7006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d76000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7006ca90 con 0x7f2d700cf960 2024-04-25T10:23:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d76000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7006cd70 con 0x7f2d7015bbb0 2024-04-25T10:23:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d76000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7006d050 con 0x7f2d700b4660 2024-04-25T10:23:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2d700cf960 0x7f2d700b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:39798/0 (socket says 172.21.15.87:39798) 2024-04-25T10:23:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6fe00640 1 -- 172.21.15.87:0/2557188577 learned_addr learned my addr 172.21.15.87:0/2557188577 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d66a00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 0x7f2d7006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322493813 0 0) 0x7f2d7006ca90 con 0x7f2d700cf960 2024-04-25T10:23:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d54003530 con 0x7f2d700cf960 2024-04-25T10:23:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3672015754 0 0) 0x7f2d54003530 con 0x7f2d700cf960 2024-04-25T10:23:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 msgr2=0x7f2d7006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6ea00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 0x7f2d7006c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 >> v1:172.21.15.169:6789/0 conn(0x7f2d7015bbb0 legacy=0x7f2d700cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.004+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700b2880 con 0x7f2d700cf960 2024-04-25T10:23:46.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f2d68001a00 con 0x7f2d700cf960 2024-04-25T10:23:46.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 >> v1:172.21.15.87:6789/0 conn(0x7f2d700cf960 legacy=0x7f2d700b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d540039f0 0x7f2d54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 0x7f2d54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d54004d50 0x7f2d540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7006d050 con 0x7f2d54004d50 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5400ba80 con 0x7f2d540039f0 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5400bc80 con 0x7f2d700b4660 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6fe00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d54004d50 0x7f2d540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d66a00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 0x7f2d54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6f400640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d540039f0 0x7f2d54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6fe00640 1 -- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 msgr2=0x7f2d54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6fe00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d700b4660 0x7f2d54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6fe00640 1 -- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d540039f0 msgr2=0x7f2d54003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6fe00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d540039f0 0x7f2d54003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6fe00640 1 -- 172.21.15.87:0/2557188577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54003530 con 0x7f2d54004d50 2024-04-25T10:23:46.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6fe00640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d54004d50 0x7f2d540051d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800a830 tx=0x7f2d6801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a99b29716d2ec4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:46.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.008+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800a480 con 0x7f2d54004d50 2024-04-25T10:23:46.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d680ad630 con 0x7f2d54004d50 2024-04-25T10:23:46.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d6ea00640 1 -- 172.21.15.87:0/2557188577 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680ada60 con 0x7f2d54004d50 2024-04-25T10:23:46.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- 172.21.15.87:0/2557188577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d54004d50 msgr2=0x7f2d540051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 --2- 172.21.15.87:0/2557188577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d54004d50 0x7f2d540051d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800a830 tx=0x7f2d6801d360 comp rx=0 tx=0).stop 2024-04-25T10:23:46.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d6fe00640 1 -- 172.21.15.87:0/2557188577 reap_dead start 2024-04-25T10:23:46.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- 172.21.15.87:0/2557188577 shutdown_connections 2024-04-25T10:23:46.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- 172.21.15.87:0/2557188577 >> 172.21.15.87:0/2557188577 conn(0x7f2d70066b70 msgr2=0x7f2d70066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:46.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- 172.21.15.87:0/2557188577 shutdown_connections 2024-04-25T10:23:46.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- 172.21.15.87:0/2557188577 wait complete. 2024-04-25T10:23:46.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 Processor -- start 2024-04-25T10:23:46.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- start start 2024-04-25T10:23:46.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d54004d50 0x7f2d70146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7015e2e0 0x7f2d7015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7015b030 0x7f2d70157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700b3910 con 0x7f2d7015e2e0 2024-04-25T10:23:46.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700b3790 con 0x7f2d7015b030 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d76000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700b3a90 con 0x7f2d54004d50 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7015e2e0 0x7f2d7015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7015e2e0 0x7f2d7015aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53394/0 (socket says 172.21.15.87:53394) 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d66a00640 1 -- 172.21.15.87:0/540102561 learned_addr learned my addr 172.21.15.87:0/540102561 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d6fe00640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7015b030 0x7f2d70157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.012+0000 7f2d6f400640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d54004d50 0x7f2d70146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d66a00640 1 -- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d54004d50 msgr2=0x7f2d70146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d66a00640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d54004d50 0x7f2d70146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d66a00640 1 -- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7015b030 msgr2=0x7f2d70157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d66a00640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7015b030 0x7f2d70157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d66a00640 1 -- 172.21.15.87:0/540102561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7015d240 con 0x7f2d7015e2e0 2024-04-25T10:23:46.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d66a00640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7015e2e0 0x7f2d7015aaf0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f2d58011940 tx=0x7f2d58011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:46.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d67e00640 1 -- 172.21.15.87:0/540102561 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58019070 con 0x7f2d7015e2e0 2024-04-25T10:23:46.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d67e00640 1 -- 172.21.15.87:0/540102561 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58004240 con 0x7f2d7015e2e0 2024-04-25T10:23:46.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d67e00640 1 -- 172.21.15.87:0/540102561 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800f8e0 con 0x7f2d7015e2e0 2024-04-25T10:23:46.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7015d520 con 0x7f2d7015e2e0 2024-04-25T10:23:46.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7015dae0 con 0x7f2d7015e2e0 2024-04-25T10:23:46.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7004ef30 con 0x7f2d7015e2e0 2024-04-25T10:23:46.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d67e00640 1 -- 172.21.15.87:0/540102561 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d5801c070 con 0x7f2d7015e2e0 2024-04-25T10:23:46.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d67e00640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d4c066f60 0x7f2d4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:46.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.016+0000 7f2d67e00640 1 -- 172.21.15.87:0/540102561 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2d58085800 con 0x7f2d7015e2e0 2024-04-25T10:23:46.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.020+0000 7f2d6f400640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d4c066f60 0x7f2d4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:46.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.020+0000 7f2d6f400640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d4c066f60 0x7f2d4c069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000c10 tx=0x7f2d60009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:46.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.020+0000 7f2d67e00640 1 -- 172.21.15.87:0/540102561 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d58056380 con 0x7f2d7015e2e0 2024-04-25T10:23:46.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.176+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2d7006d980 con 0x7f2d7015e2e0 2024-04-25T10:23:46.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.840+0000 7f2d67e00640 1 -- 172.21.15.87:0/540102561 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v149) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2d5805a420 con 0x7f2d7015e2e0 2024-04-25T10:23:46.840 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:23:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.840+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d4c066f60 msgr2=0x7f2d4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.840+0000 7f2d76000640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d4c066f60 0x7f2d4c069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000c10 tx=0x7f2d60009040 comp rx=0 tx=0).stop 2024-04-25T10:23:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.840+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7015e2e0 msgr2=0x7f2d7015aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.840+0000 7f2d76000640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7015e2e0 0x7f2d7015aaf0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f2d58011940 tx=0x7f2d58011e00 comp rx=0 tx=0).stop 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 shutdown_connections 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d4c066f60 0x7f2d4c069420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7015b030 0x7f2d70157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7015e2e0 0x7f2d7015aaf0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 --2- 172.21.15.87:0/540102561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d54004d50 0x7f2d70146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 >> 172.21.15.87:0/540102561 conn(0x7f2d70066b70 msgr2=0x7f2d700ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 shutdown_connections 2024-04-25T10:23:46.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:46.844+0000 7f2d76000640 1 -- 172.21.15.87:0/540102561 wait complete. 2024-04-25T10:23:46.881 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:23:47.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 Processor -- start 2024-04-25T10:23:47.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 -- start start 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead415dd30 0x7fead406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fead40cfb30 0x7fead40d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fead40ce220 0x7fead40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead415e110 con 0x7fead40ce220 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead406b1d0 con 0x7fead40cfb30 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feadc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead406b470 con 0x7fead415dd30 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fead40ce220 0x7fead40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fead40ce220 0x7fead40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53410/0 (socket says 172.21.15.87:53410) 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 -- 172.21.15.87:0/2516268547 learned_addr learned my addr 172.21.15.87:0/2516268547 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:47.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7fead9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead415dd30 0x7fead406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:47.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7fead8c00640 1 --2- 172.21.15.87:0/2516268547 >> v2:172.21.15.169:3300/0 conn(0x7fead40cfb30 0x7fead40d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:47.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 -- 172.21.15.87:0/2516268547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead415dd30 msgr2=0x7fead406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:47.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 --2- 172.21.15.87:0/2516268547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead415dd30 0x7fead406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 -- 172.21.15.87:0/2516268547 >> v2:172.21.15.169:3300/0 conn(0x7fead40cfb30 msgr2=0x7fead40d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:47.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 --2- 172.21.15.87:0/2516268547 >> v2:172.21.15.169:3300/0 conn(0x7fead40cfb30 0x7fead40d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.020+0000 7feada000640 1 -- 172.21.15.87:0/2516268547 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead406b750 con 0x7fead40ce220 2024-04-25T10:23:47.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feada000640 1 --2- 172.21.15.87:0/2516268547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead40d1de0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fead40cf4e0 tx=0x7feacc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718d7fc13b48af28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:47.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7fead3e00640 1 -- 172.21.15.87:0/2516268547 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a5760 con 0x7fead40ce220 2024-04-25T10:23:47.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7fead3e00640 1 -- 172.21.15.87:0/2516268547 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc0a5900 con 0x7fead40ce220 2024-04-25T10:23:47.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7fead3e00640 1 -- 172.21.15.87:0/2516268547 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a5cd0 con 0x7fead40ce220 2024-04-25T10:23:47.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 -- 172.21.15.87:0/2516268547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 msgr2=0x7fead40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:47.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 --2- 172.21.15.87:0/2516268547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead40d1de0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fead40cf4e0 tx=0x7feacc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 -- 172.21.15.87:0/2516268547 shutdown_connections 2024-04-25T10:23:47.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 --2- 172.21.15.87:0/2516268547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead40d1de0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 --2- 172.21.15.87:0/2516268547 >> v2:172.21.15.169:3300/0 conn(0x7fead40cfb30 0x7fead40d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 --2- 172.21.15.87:0/2516268547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead415dd30 0x7fead406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 -- 172.21.15.87:0/2516268547 >> 172.21.15.87:0/2516268547 conn(0x7fead409a9d0 msgr2=0x7fead409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:47.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 -- 172.21.15.87:0/2516268547 shutdown_connections 2024-04-25T10:23:47.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.024+0000 7feadc000640 1 -- 172.21.15.87:0/2516268547 wait complete. 2024-04-25T10:23:47.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 Processor -- start 2024-04-25T10:23:47.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 -- start start 2024-04-25T10:23:47.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead4157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:47.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead40cfb30 0x7fead415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:47.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead415dd30 0x7fead40c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:47.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead4162360 con 0x7fead40ce220 2024-04-25T10:23:47.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead41621e0 con 0x7fead415dd30 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead41624e0 con 0x7fead40cfb30 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead4157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead4157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53414/0 (socket says 172.21.15.87:53414) 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 -- 172.21.15.87:0/3975171081 learned_addr learned my addr 172.21.15.87:0/3975171081 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 -- 172.21.15.87:0/3975171081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead40cfb30 msgr2=0x7fead415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead40cfb30 0x7fead415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 -- 172.21.15.87:0/3975171081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead415dd30 msgr2=0x7fead40c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead415dd30 0x7fead40c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 -- 172.21.15.87:0/3975171081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead415c0f0 con 0x7fead40ce220 2024-04-25T10:23:47.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead9600640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead4157440 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7feac800e960 tx=0x7feac800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:47.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead1600640 1 -- 172.21.15.87:0/3975171081 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac8017070 con 0x7fead40ce220 2024-04-25T10:23:47.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead1600640 1 -- 172.21.15.87:0/3975171081 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac800ca00 con 0x7fead40ce220 2024-04-25T10:23:47.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead415c3d0 con 0x7fead40ce220 2024-04-25T10:23:47.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7fead1600640 1 -- 172.21.15.87:0/3975171081 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac800cdb0 con 0x7fead40ce220 2024-04-25T10:23:47.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead415c990 con 0x7fead40ce220 2024-04-25T10:23:47.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.028+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead404ef30 con 0x7fead40ce220 2024-04-25T10:23:47.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.032+0000 7fead1600640 1 -- 172.21.15.87:0/3975171081 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feac801a070 con 0x7fead40ce220 2024-04-25T10:23:47.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.032+0000 7fead1600640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feab4066f90 0x7feab4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:47.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.032+0000 7fead1600640 1 -- 172.21.15.87:0/3975171081 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5609+0+0 (secure 0 0 0) 0x7feac8085740 con 0x7fead40ce220 2024-04-25T10:23:47.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.032+0000 7fead8c00640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feab4066f90 0x7feab4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:47.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.032+0000 7fead8c00640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feab4066f90 0x7feab4069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7feac0006aa0 tx=0x7feac0006a30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:47.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.036+0000 7fead1600640 1 -- 172.21.15.87:0/3975171081 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feac8056820 con 0x7fead40ce220 2024-04-25T10:23:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.220+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fead40b7920 con 0x7fead40ce220 2024-04-25T10:23:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.860+0000 7fead1600640 1 -- 172.21.15.87:0/3975171081 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v108) v1 ==== 174+0+0 (secure 0 0 0) 0x7feac8056120 con 0x7fead40ce220 2024-04-25T10:23:47.860 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 22 and data pool 23 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feab4066f90 msgr2=0x7feab4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feab4066f90 0x7feab4069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7feac0006aa0 tx=0x7feac0006a30 comp rx=0 tx=0).stop 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 msgr2=0x7fead4157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead4157440 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7feac800e960 tx=0x7feac800ee20 comp rx=0 tx=0).stop 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 shutdown_connections 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feab4066f90 0x7feab4069450 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead415dd30 0x7fead40c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead40cfb30 0x7fead415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 --2- 172.21.15.87:0/3975171081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fead40ce220 0x7fead4157440 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:47.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 >> 172.21.15.87:0/3975171081 conn(0x7fead409a9d0 msgr2=0x7fead40ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 shutdown_connections 2024-04-25T10:23:47.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:47.864+0000 7feadc000640 1 -- 172.21.15.87:0/3975171081 wait complete. 2024-04-25T10:23:47.901 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.036+0000 7f747f400640 1 Processor -- start 2024-04-25T10:23:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.036+0000 7f747f400640 1 -- start start 2024-04-25T10:23:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f747815bc40 0x7f747806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747f400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74780b2770 con 0x7f74780cda10 2024-04-25T10:23:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747806c670 con 0x7f74780cf930 2024-04-25T10:23:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747806c950 con 0x7f747815bc40 2024-04-25T10:23:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f746fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f74780cda10 0x7f74780b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54108/0 (socket says 172.21.15.87:54108) 2024-04-25T10:23:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f746fe00640 1 -- 172.21.15.87:0/3158421481 learned_addr learned my addr 172.21.15.87:0/3158421481 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:48.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747ca00640 1 --2- 172.21.15.87:0/3158421481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f747815bc40 0x7f747806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167530984 0 0) 0x7f74780b2770 con 0x7f74780cda10 2024-04-25T10:23:48.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f745c003530 con 0x7f74780cda10 2024-04-25T10:23:48.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026674706 0 0) 0x7f747806c670 con 0x7f74780cf930 2024-04-25T10:23:48.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74780b2770 con 0x7f74780cf930 2024-04-25T10:23:48.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747ca00640 1 -- 172.21.15.87:0/3158421481 >> v1:172.21.15.169:6789/0 conn(0x7f74780cf930 legacy=0x7f74780b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383515086 0 0) 0x7f745c003530 con 0x7f74780cda10 2024-04-25T10:23:48.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747ca00640 1 -- 172.21.15.87:0/3158421481 >> v1:172.21.15.87:6789/0 conn(0x7f74780cda10 legacy=0x7f74780b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.040+0000 7f747ca00640 1 -- 172.21.15.87:0/3158421481 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747806cc30 con 0x7f747815bc40 2024-04-25T10:23:48.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747ca00640 1 --2- 172.21.15.87:0/3158421481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f747815bc40 0x7f747806c090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f74680097f0 tx=0x7f746801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e5e46f785939808 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74680b0070 con 0x7f747815bc40 2024-04-25T10:23:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74680ab400 con 0x7f747815bc40 2024-04-25T10:23:48.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f7477e00640 1 -- 172.21.15.87:0/3158421481 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460001680 con 0x7f747815bc40 2024-04-25T10:23:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 -- 172.21.15.87:0/3158421481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f747815bc40 msgr2=0x7f747806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 --2- 172.21.15.87:0/3158421481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f747815bc40 0x7f747806c090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f74680097f0 tx=0x7f746801c370 comp rx=0 tx=0).stop 2024-04-25T10:23:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 -- 172.21.15.87:0/3158421481 shutdown_connections 2024-04-25T10:23:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 --2- 172.21.15.87:0/3158421481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f747815bc40 0x7f747806c090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 -- 172.21.15.87:0/3158421481 >> 172.21.15.87:0/3158421481 conn(0x7f7478066b70 msgr2=0x7f7478066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 -- 172.21.15.87:0/3158421481 shutdown_connections 2024-04-25T10:23:48.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 -- 172.21.15.87:0/3158421481 wait complete. 2024-04-25T10:23:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 Processor -- start 2024-04-25T10:23:48.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.044+0000 7f747f400640 1 -- start start 2024-04-25T10:23:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74780cda10 0x7f747815e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74780cf930 0x7f747815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f747815bc40 0x7f747814c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7478169d70 con 0x7f74780cda10 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7478169bf0 con 0x7f747815bc40 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7478169ef0 con 0x7f74780cf930 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74780cda10 0x7f747815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74780cda10 0x7f747815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50028/0 (socket says 172.21.15.87:50028) 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 -- 172.21.15.87:0/430021802 learned_addr learned my addr 172.21.15.87:0/430021802 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747d400640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f747815bc40 0x7f747814c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f746fe00640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74780cf930 0x7f747815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 -- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74780cf930 msgr2=0x7f747815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74780cf930 0x7f747815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 -- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f747815bc40 msgr2=0x7f747814c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f747815bc40 0x7f747814c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 -- 172.21.15.87:0/430021802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478070810 con 0x7f74780cda10 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747ca00640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74780cda10 0x7f747815e480 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f74680ae040 tx=0x7f746801abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f7475600640 1 -- 172.21.15.87:0/430021802 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74680bb020 con 0x7f74780cda10 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f7475600640 1 -- 172.21.15.87:0/430021802 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74680b1040 con 0x7f74780cda10 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7478070af0 con 0x7f74780cda10 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f7475600640 1 -- 172.21.15.87:0/430021802 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74680b0050 con 0x7f74780cda10 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74780710b0 con 0x7f74780cda10 2024-04-25T10:23:48.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747804ef30 con 0x7f74780cda10 2024-04-25T10:23:48.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.048+0000 7f7475600640 1 -- 172.21.15.87:0/430021802 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74680be070 con 0x7f74780cda10 2024-04-25T10:23:48.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.052+0000 7f7475600640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7454066f60 0x7f7454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.052+0000 7f7475600640 1 -- 172.21.15.87:0/430021802 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f746811e370 con 0x7f74780cda10 2024-04-25T10:23:48.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.052+0000 7f746fe00640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7454066f60 0x7f7454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.052+0000 7f746fe00640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7454066f60 0x7f7454069420 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74780b0a20 tx=0x7f7460001160 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.056+0000 7f7475600640 1 -- 172.21.15.87:0/430021802 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7468015ff0 con 0x7f74780cda10 2024-04-25T10:23:48.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.212+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f747806d960 con 0x7f74780cda10 2024-04-25T10:23:48.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.212+0000 7f7475600640 1 -- 172.21.15.87:0/430021802 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16672 (secure 0 0 0) 0x7f74680eec90 con 0x7f74780cda10 2024-04-25T10:23:48.213 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:48.213 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":150,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:47.224389+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:23:45.609544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:23:46.223978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7454066f60 msgr2=0x7f7454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7454066f60 0x7f7454069420 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74780b0a20 tx=0x7f7460001160 comp rx=0 tx=0).stop 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74780cda10 msgr2=0x7f747815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74780cda10 0x7f747815e480 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f74680ae040 tx=0x7f746801abb0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 shutdown_connections 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7454066f60 0x7f7454069420 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f747815bc40 0x7f747814c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74780cf930 0x7f747815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 --2- 172.21.15.87:0/430021802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74780cda10 0x7f747815e480 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 >> 172.21.15.87:0/430021802 conn(0x7f7478066b70 msgr2=0x7f747815a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:48.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 shutdown_connections 2024-04-25T10:23:48.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.216+0000 7f747f400640 1 -- 172.21.15.87:0/430021802 wait complete. 2024-04-25T10:23:48.235 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:48.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.368+0000 7f128b000640 1 Processor -- start 2024-04-25T10:23:48.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.368+0000 7f128b000640 1 -- start start 2024-04-25T10:23:48.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.368+0000 7f128b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12840cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.368+0000 7f128b000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f128415bdc0 0x7f12840cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.368+0000 7f128b000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12840ccfb0 con 0x7f12840d1520 2024-04-25T10:23:48.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.368+0000 7f128b000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12840cd290 con 0x7f128415bdc0 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.368+0000 7f128b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12840cd570 con 0x7f128415b9e0 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f128415bdc0 0x7f12840cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f128415bdc0 0x7f12840cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50042/0 (socket says 172.21.15.87:50042) 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 -- 172.21.15.87:0/3298918493 learned_addr learned my addr 172.21.15.87:0/3298918493 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283e00640 1 --2- 172.21.15.87:0/3298918493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12840cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 -- 172.21.15.87:0/3298918493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 msgr2=0x7f12840cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 --2- 172.21.15.87:0/3298918493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12840cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 -- 172.21.15.87:0/3298918493 >> v1:172.21.15.169:6789/0 conn(0x7f12840d1520 legacy=0x7f12840d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 -- 172.21.15.87:0/3298918493 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12840cd850 con 0x7f128415bdc0 2024-04-25T10:23:48.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1283400640 1 --2- 172.21.15.87:0/3298918493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128415bdc0 0x7f12840cc950 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f126c009870 tx=0x7f126c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecb0165a866af42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1282a00640 1 -- 172.21.15.87:0/3298918493 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c0ab610 con 0x7f128415bdc0 2024-04-25T10:23:48.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.372+0000 7f1282a00640 1 -- 172.21.15.87:0/3298918493 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c0b0070 con 0x7f128415bdc0 2024-04-25T10:23:48.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1282a00640 1 -- 172.21.15.87:0/3298918493 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c0ab9d0 con 0x7f128415bdc0 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- 172.21.15.87:0/3298918493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128415bdc0 msgr2=0x7f12840cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 --2- 172.21.15.87:0/3298918493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128415bdc0 0x7f12840cc950 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f126c009870 tx=0x7f126c01c560 comp rx=0 tx=0).stop 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- 172.21.15.87:0/3298918493 shutdown_connections 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 --2- 172.21.15.87:0/3298918493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128415bdc0 0x7f12840cc950 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 --2- 172.21.15.87:0/3298918493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12840cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- 172.21.15.87:0/3298918493 >> 172.21.15.87:0/3298918493 conn(0x7f128409a9d0 msgr2=0x7f128409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- 172.21.15.87:0/3298918493 shutdown_connections 2024-04-25T10:23:48.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- 172.21.15.87:0/3298918493 wait complete. 2024-04-25T10:23:48.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 Processor -- start 2024-04-25T10:23:48.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- start start 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12840d1520 0x7f1284155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12841568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128415bdc0 0x7f12840c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840c6630 con 0x7f12840d1520 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840c64b0 con 0x7f128415bdc0 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f128b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12840c67b0 con 0x7f128415b9e0 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12840d1520 0x7f1284155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12840d1520 0x7f1284155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50050/0 (socket says 172.21.15.87:50050) 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 -- 172.21.15.87:0/568624758 learned_addr learned my addr 172.21.15.87:0/568624758 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283400640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12841568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1289000640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128415bdc0 0x7f12840c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 -- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 msgr2=0x7f12841568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12841568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 -- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128415bdc0 msgr2=0x7f12840c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128415bdc0 0x7f12840c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283e00640 1 -- 172.21.15.87:0/568624758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12840b4670 con 0x7f12840d1520 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.376+0000 7f1283400640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12841568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:48.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f1283e00640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12840d1520 0x7f1284155e20 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f1274009db0 tx=0x7f127400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f127be00640 1 -- 172.21.15.87:0/568624758 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274018020 con 0x7f12840d1520 2024-04-25T10:23:48.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f127be00640 1 -- 172.21.15.87:0/568624758 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1274014480 con 0x7f12840d1520 2024-04-25T10:23:48.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f127be00640 1 -- 172.21.15.87:0/568624758 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274014850 con 0x7f12840d1520 2024-04-25T10:23:48.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12840b4950 con 0x7f12840d1520 2024-04-25T10:23:48.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12840b4f10 con 0x7f12840d1520 2024-04-25T10:23:48.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128404ef30 con 0x7f12840d1520 2024-04-25T10:23:48.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f127be00640 1 -- 172.21.15.87:0/568624758 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f127401b020 con 0x7f12840d1520 2024-04-25T10:23:48.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f127be00640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1260066f60 0x7f1260069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f127be00640 1 -- 172.21.15.87:0/568624758 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1274085cb0 con 0x7f12840d1520 2024-04-25T10:23:48.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.380+0000 7f1283400640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1260066f60 0x7f1260069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.384+0000 7f1283400640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1260066f60 0x7f1260069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f126c0af000 tx=0x7f126c0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.384+0000 7f127be00640 1 -- 172.21.15.87:0/568624758 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1274022070 con 0x7f12840d1520 2024-04-25T10:23:48.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.540+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f128415b360 con 0x7f12840d1520 2024-04-25T10:23:48.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.540+0000 7f127be00640 1 -- 172.21.15.87:0/568624758 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16672 (secure 0 0 0) 0x7f1274010040 con 0x7f12840d1520 2024-04-25T10:23:48.541 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:48.542 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":150,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:47.224389+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:23:45.609544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:23:46.223978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:48.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1260066f60 msgr2=0x7f1260069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1260066f60 0x7f1260069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f126c0af000 tx=0x7f126c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12840d1520 msgr2=0x7f1284155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12840d1520 0x7f1284155e20 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f1274009db0 tx=0x7f127400c490 comp rx=0 tx=0).stop 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 shutdown_connections 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1260066f60 0x7f1260069420 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128415bdc0 0x7f12840c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f128415b9e0 0x7f12841568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 --2- 172.21.15.87:0/568624758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12840d1520 0x7f1284155e20 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 >> 172.21.15.87:0/568624758 conn(0x7f128409a9d0 msgr2=0x7f12840cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 shutdown_connections 2024-04-25T10:23:48.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.544+0000 7f128b000640 1 -- 172.21.15.87:0/568624758 wait complete. 2024-04-25T10:23:48.563 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:23:48.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.700+0000 7f433fe00640 1 Processor -- start 2024-04-25T10:23:48.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.700+0000 7f433fe00640 1 -- start start 2024-04-25T10:23:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f43400cde90 0x7f43400b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434006be90 con 0x7f43400b1f50 2024-04-25T10:23:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434006b8d0 con 0x7f43400b23c0 2024-04-25T10:23:48.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43400d0b50 con 0x7f43400cde90 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433ea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f43400b1f50 0x7f434006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54122/0 (socket says 172.21.15.87:54122) 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433ea00640 1 -- 172.21.15.87:0/622292794 learned_addr learned my addr 172.21.15.87:0/622292794 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350704037 0 0) 0x7f4330001120 con 0x7f43400b1f50 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4324003530 con 0x7f43400b1f50 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433f400640 1 --2- 172.21.15.87:0/622292794 >> v2:172.21.15.169:3300/0 conn(0x7f43400cde90 0x7f43400b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3883859748 0 0) 0x7f4324003530 con 0x7f43400b1f50 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 >> v2:172.21.15.169:3300/0 conn(0x7f43400cde90 msgr2=0x7f43400b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 --2- 172.21.15.87:0/622292794 >> v2:172.21.15.169:3300/0 conn(0x7f43400cde90 0x7f43400b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 >> v1:172.21.15.169:6789/0 conn(0x7f43400b23c0 legacy=0x7f43400cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434006bbb0 con 0x7f43400b1f50 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f4330001850 con 0x7f43400b1f50 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433f400640 1 --2- 172.21.15.87:0/622292794 >> v2:172.21.15.169:3300/0 conn(0x7f43400cde90 0x7f43400b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 >> v1:172.21.15.87:6789/0 conn(0x7f43400b1f50 legacy=0x7f434006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4324003a30 0x7f4324003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324004810 0x7f4324004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324005380 0x7f432400b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432400c2a0 con 0x7f4324004810 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432400c450 con 0x7f4324005380 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432400c650 con 0x7f4324003a30 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433e000640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324004810 0x7f4324004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433ea00640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4324003a30 0x7f4324003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433f400640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324005380 0x7f432400b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433e000640 1 -- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4324003a30 msgr2=0x7f4324003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433e000640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4324003a30 0x7f4324003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433e000640 1 -- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324005380 msgr2=0x7f432400b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433e000640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324005380 0x7f432400b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433e000640 1 -- 172.21.15.87:0/622292794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324003530 con 0x7f4324004810 2024-04-25T10:23:48.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.704+0000 7f433f400640 1 -- 172.21.15.87:0/622292794 reap_dead start 2024-04-25T10:23:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433e000640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324004810 0x7f4324004c90 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f432c001780 tx=0x7f432c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507b6583e89f3d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c0013a0 con 0x7f4324004810 2024-04-25T10:23:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c0052f0 con 0x7f4324004810 2024-04-25T10:23:48.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433d600640 1 -- 172.21.15.87:0/622292794 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c003840 con 0x7f4324004810 2024-04-25T10:23:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 -- 172.21.15.87:0/622292794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324004810 msgr2=0x7f4324004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324004810 0x7f4324004c90 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f432c001780 tx=0x7f432c01c400 comp rx=0 tx=0).stop 2024-04-25T10:23:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 -- 172.21.15.87:0/622292794 shutdown_connections 2024-04-25T10:23:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 --2- 172.21.15.87:0/622292794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4324004810 0x7f4324004c90 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 -- 172.21.15.87:0/622292794 >> 172.21.15.87:0/622292794 conn(0x7f43400b32b0 msgr2=0x7f43400b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 -- 172.21.15.87:0/622292794 shutdown_connections 2024-04-25T10:23:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 -- 172.21.15.87:0/622292794 wait complete. 2024-04-25T10:23:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 Processor -- start 2024-04-25T10:23:48.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.708+0000 7f433fe00640 1 -- start start 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324004810 0x7f434015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4340152d00 0x7f4340160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43400b6370 0x7f43400b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400d0500 con 0x7f4340152d00 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400d0380 con 0x7f4324004810 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400b41e0 con 0x7f43400b6370 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4340152d00 0x7f4340160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4340152d00 0x7f4340160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50078/0 (socket says 172.21.15.87:50078) 2024-04-25T10:23:48.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 -- 172.21.15.87:0/3995265565 learned_addr learned my addr 172.21.15.87:0/3995265565 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:48.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433ea00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324004810 0x7f434015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433f400640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43400b6370 0x7f43400b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 -- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43400b6370 msgr2=0x7f43400b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43400b6370 0x7f43400b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 -- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324004810 msgr2=0x7f434015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:48.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324004810 0x7f434015c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:48.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 -- 172.21.15.87:0/3995265565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340161270 con 0x7f4340152d00 2024-04-25T10:23:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433e000640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4340152d00 0x7f4340160d30 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f432c01c230 tx=0x7f4324005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f4337400640 1 -- 172.21.15.87:0/3995265565 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c007a40 con 0x7f4340152d00 2024-04-25T10:23:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f4337400640 1 -- 172.21.15.87:0/3995265565 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c0ba070 con 0x7f4340152d00 2024-04-25T10:23:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f4337400640 1 -- 172.21.15.87:0/3995265565 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c007250 con 0x7f4340152d00 2024-04-25T10:23:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43400c75f0 con 0x7f4340152d00 2024-04-25T10:23:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43400c7bb0 con 0x7f4340152d00 2024-04-25T10:23:48.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4340075120 con 0x7f4340152d00 2024-04-25T10:23:48.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.712+0000 7f4337400640 1 -- 172.21.15.87:0/3995265565 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f432c0052f0 con 0x7f4340152d00 2024-04-25T10:23:48.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.716+0000 7f4337400640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4320066f60 0x7f4320069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:48.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.716+0000 7f4337400640 1 -- 172.21.15.87:0/3995265565 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f432c11f1a0 con 0x7f4340152d00 2024-04-25T10:23:48.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.716+0000 7f433ea00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4320066f60 0x7f4320069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:48.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.716+0000 7f433ea00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4320066f60 0x7f4320069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c10 tx=0x7f4330007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:48.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.720+0000 7f4337400640 1 -- 172.21.15.87:0/3995265565 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f432c016110 con 0x7f4340152d00 2024-04-25T10:23:48.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:48.908+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f43400b7390 con 0x7f4340152d00 2024-04-25T10:23:49.836 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:23:49.836+0000 7fa05f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:23:49.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f4337400640 1 -- 172.21.15.87:0/3995265565 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v110) v1 ==== 118+0+0 (secure 0 0 0) 0x7f432c0efbf0 con 0x7f4340152d00 2024-04-25T10:23:49.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4320066f60 msgr2=0x7f4320069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:49.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4320066f60 0x7f4320069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c10 tx=0x7f4330007a80 comp rx=0 tx=0).stop 2024-04-25T10:23:49.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4340152d00 msgr2=0x7f4340160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:49.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4340152d00 0x7f4340160d30 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f432c01c230 tx=0x7f4324005840 comp rx=0 tx=0).stop 2024-04-25T10:23:49.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 shutdown_connections 2024-04-25T10:23:49.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4320066f60 0x7f4320069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:49.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43400b6370 0x7f43400b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:49.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4340152d00 0x7f4340160d30 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:49.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 --2- 172.21.15.87:0/3995265565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4324004810 0x7f434015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:49.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 >> 172.21.15.87:0/3995265565 conn(0x7f43400b32b0 msgr2=0x7f434006e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:49.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 shutdown_connections 2024-04-25T10:23:49.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:49.892+0000 7f433fe00640 1 -- 172.21.15.87:0/3995265565 wait complete. 2024-04-25T10:23:49.914 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:23:49.914 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:23:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.052+0000 7f3295600640 1 Processor -- start 2024-04-25T10:23:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.052+0000 7f3295600640 1 -- start start 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f3295600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3290159540 0x7f32900ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f3295600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32900aebd0 0x7f3290071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f3295600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3290071360 con 0x7f3290072020 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f3295600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3290159920 con 0x7f3290159540 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f3295600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329015c2f0 con 0x7f32900aebd0 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3290159540 0x7f32900ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3290159540 0x7f32900ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50084/0 (socket says 172.21.15.87:50084) 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 -- 172.21.15.87:0/1078046935 learned_addr learned my addr 172.21.15.87:0/1078046935 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328e000640 1 --2- 172.21.15.87:0/1078046935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32900aebd0 0x7f3290071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 -- 172.21.15.87:0/1078046935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32900aebd0 msgr2=0x7f3290071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:50.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 --2- 172.21.15.87:0/1078046935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32900aebd0 0x7f3290071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 -- 172.21.15.87:0/1078046935 >> v1:172.21.15.169:6789/0 conn(0x7f3290072020 legacy=0x7f329015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 -- 172.21.15.87:0/1078046935 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329015c5d0 con 0x7f3290159540 2024-04-25T10:23:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328e000640 1 --2- 172.21.15.87:0/1078046935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32900aebd0 0x7f3290071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.056+0000 7f328ea00640 1 --2- 172.21.15.87:0/1078046935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3290159540 0x7f32900ae690 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f3280009870 tx=0x7f328001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907518b4ad7fe165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328d600640 1 -- 172.21.15.87:0/1078046935 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32800b0070 con 0x7f3290159540 2024-04-25T10:23:50.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328d600640 1 -- 172.21.15.87:0/1078046935 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32800ab600 con 0x7f3290159540 2024-04-25T10:23:50.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328d600640 1 -- 172.21.15.87:0/1078046935 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32800ab9d0 con 0x7f3290159540 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- 172.21.15.87:0/1078046935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3290159540 msgr2=0x7f32900ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 --2- 172.21.15.87:0/1078046935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3290159540 0x7f32900ae690 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f3280009870 tx=0x7f328001c560 comp rx=0 tx=0).stop 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- 172.21.15.87:0/1078046935 shutdown_connections 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 --2- 172.21.15.87:0/1078046935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32900aebd0 0x7f3290071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 --2- 172.21.15.87:0/1078046935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3290159540 0x7f32900ae690 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- 172.21.15.87:0/1078046935 >> 172.21.15.87:0/1078046935 conn(0x7f32900669a0 msgr2=0x7f3290066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- 172.21.15.87:0/1078046935 shutdown_connections 2024-04-25T10:23:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- 172.21.15.87:0/1078046935 wait complete. 2024-04-25T10:23:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 Processor -- start 2024-04-25T10:23:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- start start 2024-04-25T10:23:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3290072020 0x7f3290140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32900aebd0 0x7f3290153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3290159540 0x7f3290151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3290072430 con 0x7f32900aebd0 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329006c0d0 con 0x7f3290072020 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f3295600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3290168930 con 0x7f3290159540 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32900aebd0 0x7f3290153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32900aebd0 0x7f3290153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50090/0 (socket says 172.21.15.87:50090) 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328e000640 1 -- 172.21.15.87:0/2781145697 learned_addr learned my addr 172.21.15.87:0/2781145697 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328ea00640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3290072020 0x7f3290140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.060+0000 7f328f400640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3290159540 0x7f3290151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f328e000640 1 -- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3290159540 msgr2=0x7f3290151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f328e000640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3290159540 0x7f3290151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f328e000640 1 -- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3290072020 msgr2=0x7f3290140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f328e000640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3290072020 0x7f3290140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f328e000640 1 -- 172.21.15.87:0/2781145697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3290140760 con 0x7f32900aebd0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f328e000640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32900aebd0 0x7f3290153090 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f327c00e810 tx=0x7f327c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3286a00640 1 -- 172.21.15.87:0/2781145697 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c017070 con 0x7f32900aebd0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3286a00640 1 -- 172.21.15.87:0/2781145697 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c00c8d0 con 0x7f32900aebd0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3286a00640 1 -- 172.21.15.87:0/2781145697 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c00f040 con 0x7f32900aebd0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3290153b30 con 0x7f32900aebd0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32901540f0 con 0x7f32900aebd0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329004ef30 con 0x7f32900aebd0 2024-04-25T10:23:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3286a00640 1 -- 172.21.15.87:0/2781145697 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f327c01a070 con 0x7f32900aebd0 2024-04-25T10:23:50.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3286a00640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3270066f60 0x7f3270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:50.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f3286a00640 1 -- 172.21.15.87:0/2781145697 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f327c084ad0 con 0x7f32900aebd0 2024-04-25T10:23:50.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.064+0000 7f328ea00640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3270066f60 0x7f3270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:50.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.068+0000 7f328ea00640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3270066f60 0x7f3270069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f32800ad000 tx=0x7f3280008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:50.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.068+0000 7f3286a00640 1 -- 172.21.15.87:0/2781145697 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f327c055480 con 0x7f32900aebd0 2024-04-25T10:23:50.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.256+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3290166700 con 0x7f32900aebd0 2024-04-25T10:23:50.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.900+0000 7f3286a00640 1 -- 172.21.15.87:0/2781145697 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v111) v1 ==== 122+0+0 (secure 0 0 0) 0x7f327c059520 con 0x7f32900aebd0 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3270066f60 msgr2=0x7f3270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3270066f60 0x7f3270069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f32800ad000 tx=0x7f3280008040 comp rx=0 tx=0).stop 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32900aebd0 msgr2=0x7f3290153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32900aebd0 0x7f3290153090 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f327c00e810 tx=0x7f327c00ecd0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 shutdown_connections 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3270066f60 0x7f3270069420 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3290159540 0x7f3290151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32900aebd0 0x7f3290153090 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 --2- 172.21.15.87:0/2781145697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3290072020 0x7f3290140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:50.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 >> 172.21.15.87:0/2781145697 conn(0x7f32900669a0 msgr2=0x7f32900ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:50.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 shutdown_connections 2024-04-25T10:23:50.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:50.904+0000 7f3295600640 1 -- 172.21.15.87:0/2781145697 wait complete. 2024-04-25T10:23:50.927 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:23:50.927 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:51.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.064+0000 7f42a7c00640 1 Processor -- start 2024-04-25T10:23:51.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.064+0000 7f42a7c00640 1 -- start start 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f42a7c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42a01550b0 0x7f42a00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f42a7c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f42a00cfe70 0x7f42a015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f42a7c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a0071050 con 0x7f42a015b740 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f42a7c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a015d040 con 0x7f42a00cfe70 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f42a7c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a00712f0 con 0x7f42a01550b0 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f4297e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f42a00cfe70 0x7f42a015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f4297e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f42a00cfe70 0x7f42a015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50106/0 (socket says 172.21.15.87:50106) 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f4297e00640 1 -- 172.21.15.87:0/3859598592 learned_addr learned my addr 172.21.15.87:0/3859598592 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f42a5200640 1 --2- 172.21.15.87:0/3859598592 >> v2:172.21.15.169:3300/0 conn(0x7f42a01550b0 0x7f42a00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f4297e00640 1 -- 172.21.15.87:0/3859598592 >> v2:172.21.15.169:3300/0 conn(0x7f42a01550b0 msgr2=0x7f42a00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f4297e00640 1 --2- 172.21.15.87:0/3859598592 >> v2:172.21.15.169:3300/0 conn(0x7f42a01550b0 0x7f42a00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f4297e00640 1 -- 172.21.15.87:0/3859598592 >> v1:172.21.15.169:6789/0 conn(0x7f42a015b740 legacy=0x7f42a006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.068+0000 7f4297e00640 1 -- 172.21.15.87:0/3859598592 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00715d0 con 0x7f42a00cfe70 2024-04-25T10:23:51.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f4297e00640 1 --2- 172.21.15.87:0/3859598592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a015b200 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f4288009870 tx=0x7f428801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5f5bb0dd7d7c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f429fe00640 1 -- 172.21.15.87:0/3859598592 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880b0070 con 0x7f42a00cfe70 2024-04-25T10:23:51.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f429fe00640 1 -- 172.21.15.87:0/3859598592 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42880ab600 con 0x7f42a00cfe70 2024-04-25T10:23:51.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f429fe00640 1 -- 172.21.15.87:0/3859598592 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880ab9d0 con 0x7f42a00cfe70 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 -- 172.21.15.87:0/3859598592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 msgr2=0x7f42a015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 --2- 172.21.15.87:0/3859598592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a015b200 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f4288009870 tx=0x7f428801c520 comp rx=0 tx=0).stop 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 -- 172.21.15.87:0/3859598592 shutdown_connections 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 --2- 172.21.15.87:0/3859598592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a015b200 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 --2- 172.21.15.87:0/3859598592 >> v2:172.21.15.169:3300/0 conn(0x7f42a01550b0 0x7f42a00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 -- 172.21.15.87:0/3859598592 >> 172.21.15.87:0/3859598592 conn(0x7f42a0066b70 msgr2=0x7f42a0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 -- 172.21.15.87:0/3859598592 shutdown_connections 2024-04-25T10:23:51.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 -- 172.21.15.87:0/3859598592 wait complete. 2024-04-25T10:23:51.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 Processor -- start 2024-04-25T10:23:51.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.072+0000 7f42a7c00640 1 -- start start 2024-04-25T10:23:51.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a013faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a01550b0 0x7f42a00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b740 0x7f42a00b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a015bcd0 con 0x7f42a00cfe70 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a015bb50 con 0x7f42a015b740 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a015c030 con 0x7f42a01550b0 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a013faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a013faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50116/0 (socket says 172.21.15.87:50116) 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 -- 172.21.15.87:0/1206824479 learned_addr learned my addr 172.21.15.87:0/1206824479 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f4297e00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a01550b0 0x7f42a00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5c00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b740 0x7f42a00b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 -- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a01550b0 msgr2=0x7f42a00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a01550b0 0x7f42a00cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 -- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b740 msgr2=0x7f42a00b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b740 0x7f42a00b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 -- 172.21.15.87:0/1206824479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00ce2e0 con 0x7f42a00cfe70 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a5200640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a013faa0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f429000d830 tx=0x7f429000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f429d600640 1 -- 172.21.15.87:0/1206824479 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000b6c0 con 0x7f42a00cfe70 2024-04-25T10:23:51.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f429d600640 1 -- 172.21.15.87:0/1206824479 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4290019070 con 0x7f42a00cfe70 2024-04-25T10:23:51.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f429d600640 1 -- 172.21.15.87:0/1206824479 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000ba60 con 0x7f42a00cfe70 2024-04-25T10:23:51.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a00ce5c0 con 0x7f42a00cfe70 2024-04-25T10:23:51.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a01536d0 con 0x7f42a00cfe70 2024-04-25T10:23:51.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.076+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a004ef30 con 0x7f42a00cfe70 2024-04-25T10:23:51.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.080+0000 7f429d600640 1 -- 172.21.15.87:0/1206824479 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f429001e460 con 0x7f42a00cfe70 2024-04-25T10:23:51.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.080+0000 7f429d600640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f427c066f60 0x7f427c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.080+0000 7f429d600640 1 -- 172.21.15.87:0/1206824479 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4290085680 con 0x7f42a00cfe70 2024-04-25T10:23:51.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.080+0000 7f4297e00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f427c066f60 0x7f427c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.080+0000 7f4297e00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f427c066f60 0x7f427c069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f42880ad000 tx=0x7f4288008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.084+0000 7f429d600640 1 -- 172.21.15.87:0/1206824479 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4290056720 con 0x7f42a00cfe70 2024-04-25T10:23:51.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.268+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42a006b390 con 0x7f42a00cfe70 2024-04-25T10:23:51.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f429d600640 1 -- 172.21.15.87:0/1206824479 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4955 (secure 0 0 0) 0x7f4290056020 con 0x7f42a00cfe70 2024-04-25T10:23:51.271 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:51.271 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":111,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4139990719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4139990719},{"type":"v1","addr":"172.21.15.169:6834","nonce":4139990719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3962905873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3962905873},{"type":"v1","addr":"172.21.15.169:6835","nonce":3962905873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3473546915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3473546915},{"type":"v1","addr":"172.21.15.87:6836","nonce":3473546915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:23:47.224714+0000","modified":"2024-04-25T10:23:50.259650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10044":{"gid":10044,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/564498676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":564498676},{"type":"v1","addr":"172.21.15.87:6837","nonce":564498676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:23:51.271 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 111 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f427c066f60 msgr2=0x7f427c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f427c066f60 0x7f427c069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f42880ad000 tx=0x7f4288008040 comp rx=0 tx=0).stop 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 msgr2=0x7f42a013faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a013faa0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f429000d830 tx=0x7f429000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 shutdown_connections 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f427c066f60 0x7f427c069420 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b740 0x7f42a00b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a01550b0 0x7f42a00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 --2- 172.21.15.87:0/1206824479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a00cfe70 0x7f42a013faa0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 >> 172.21.15.87:0/1206824479 conn(0x7f42a0066b70 msgr2=0x7f42a006c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 shutdown_connections 2024-04-25T10:23:51.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.272+0000 7f42a7c00640 1 -- 172.21.15.87:0/1206824479 wait complete. 2024-04-25T10:23:51.293 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc905c00640 1 Processor -- start 2024-04-25T10:23:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc905c00640 1 -- start start 2024-04-25T10:23:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc905c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc9000cda10 0x7fc90006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc905c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc9000cf930 0x7fc9000cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc905c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9000b2770 con 0x7fc90015bc40 2024-04-25T10:23:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc905c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90006c670 con 0x7fc9000cf930 2024-04-25T10:23:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc905c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90006c950 con 0x7fc9000cda10 2024-04-25T10:23:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8ff400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc9000cf930 0x7fc9000cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8fea00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc90015bc40 0x7fc9000b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54138/0 (socket says 172.21.15.87:54138) 2024-04-25T10:23:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8fea00640 1 -- 172.21.15.87:0/3612985991 learned_addr learned my addr 172.21.15.87:0/3612985991 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8fe000640 1 --2- 172.21.15.87:0/3612985991 >> v2:172.21.15.169:3300/0 conn(0x7fc9000cda10 0x7fc90006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8ff400640 1 -- 172.21.15.87:0/3612985991 >> v2:172.21.15.169:3300/0 conn(0x7fc9000cda10 msgr2=0x7fc90006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:51.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8ff400640 1 --2- 172.21.15.87:0/3612985991 >> v2:172.21.15.169:3300/0 conn(0x7fc9000cda10 0x7fc90006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8ff400640 1 -- 172.21.15.87:0/3612985991 >> v1:172.21.15.87:6789/0 conn(0x7fc90015bc40 legacy=0x7fc9000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8ff400640 1 -- 172.21.15.87:0/3612985991 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90006cc30 con 0x7fc9000cf930 2024-04-25T10:23:51.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8fe000640 1 --2- 172.21.15.87:0/3612985991 >> v2:172.21.15.169:3300/0 conn(0x7fc9000cda10 0x7fc90006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:51.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.428+0000 7fc8fd600640 1 -- 172.21.15.87:0/3612985991 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673245798 0 0) 0x7fc9000b2770 con 0x7fc90015bc40 2024-04-25T10:23:51.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc8ff400640 1 --2- 172.21.15.87:0/3612985991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cf930 0x7fc9000cfdc0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fc9000726f0 tx=0x7fc8f801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75531ae0f0c8a8a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc8fd600640 1 -- 172.21.15.87:0/3612985991 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80b2070 con 0x7fc9000cf930 2024-04-25T10:23:51.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc8fd600640 1 -- 172.21.15.87:0/3612985991 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f80a5760 con 0x7fc9000cf930 2024-04-25T10:23:51.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc8fd600640 1 -- 172.21.15.87:0/3612985991 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80a5b30 con 0x7fc9000cf930 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 -- 172.21.15.87:0/3612985991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cf930 msgr2=0x7fc9000cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 --2- 172.21.15.87:0/3612985991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cf930 0x7fc9000cfdc0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fc9000726f0 tx=0x7fc8f801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 -- 172.21.15.87:0/3612985991 shutdown_connections 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 --2- 172.21.15.87:0/3612985991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cf930 0x7fc9000cfdc0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 --2- 172.21.15.87:0/3612985991 >> v2:172.21.15.169:3300/0 conn(0x7fc9000cda10 0x7fc90006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 -- 172.21.15.87:0/3612985991 >> 172.21.15.87:0/3612985991 conn(0x7fc900066b70 msgr2=0x7fc900066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 -- 172.21.15.87:0/3612985991 shutdown_connections 2024-04-25T10:23:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.432+0000 7fc905c00640 1 -- 172.21.15.87:0/3612985991 wait complete. 2024-04-25T10:23:51.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 Processor -- start 2024-04-25T10:23:51.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 -- start start 2024-04-25T10:23:51.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cda10 0x7fc900071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9000cf930 0x7fc900154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc90015bc40 0x7fc900159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000b3850 con 0x7fc9000cda10 2024-04-25T10:23:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000b36d0 con 0x7fc90015bc40 2024-04-25T10:23:51.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000b39d0 con 0x7fc9000cf930 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cda10 0x7fc900071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cda10 0x7fc900071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50138/0 (socket says 172.21.15.87:50138) 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 -- 172.21.15.87:0/3512503241 learned_addr learned my addr 172.21.15.87:0/3512503241 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fe000640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9000cf930 0x7fc900154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8ff400640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc90015bc40 0x7fc900159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 -- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9000cf930 msgr2=0x7fc900154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9000cf930 0x7fc900154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 -- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc90015bc40 msgr2=0x7fc900159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc90015bc40 0x7fc900159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 -- 172.21.15.87:0/3512503241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9000722a0 con 0x7fc9000cda10 2024-04-25T10:23:51.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8fea00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cda10 0x7fc900071d60 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c480 tx=0x7fc8f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8f6a00640 1 -- 172.21.15.87:0/3512503241 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0018070 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8f6a00640 1 -- 172.21.15.87:0/3512503241 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0007500 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc8f6a00640 1 -- 172.21.15.87:0/3512503241 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0004a30 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9000709e0 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.436+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc900070fa0 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.440+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90004ef30 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.440+0000 7fc8f6a00640 1 -- 172.21.15.87:0/3512503241 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8f0004bd0 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.440+0000 7fc8f6a00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8e0066f60 0x7fc8e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.440+0000 7fc8f6a00640 1 -- 172.21.15.87:0/3512503241 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8f0084230 con 0x7fc9000cda10 2024-04-25T10:23:51.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.440+0000 7fc8fe000640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8e0066f60 0x7fc8e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.440+0000 7fc8fe000640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8e0066f60 0x7fc8e0069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc9000b0f00 tx=0x7fc8ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.444+0000 7fc8f6a00640 1 -- 172.21.15.87:0/3512503241 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8f0054be0 con 0x7fc9000cda10 2024-04-25T10:23:51.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.600+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc900155360 con 0x7fc9000cda10 2024-04-25T10:23:51.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.600+0000 7fc8f6a00640 1 -- 172.21.15.87:0/3512503241 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16659 (secure 0 0 0) 0x7fc8f0058c80 con 0x7fc9000cda10 2024-04-25T10:23:51.602 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:51.602 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":151,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:48.878767+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:23:45.609544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:23:46.223978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8e0066f60 msgr2=0x7fc8e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8e0066f60 0x7fc8e0069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc9000b0f00 tx=0x7fc8ec008040 comp rx=0 tx=0).stop 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cda10 msgr2=0x7fc900071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cda10 0x7fc900071d60 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c480 tx=0x7fc8f000c940 comp rx=0 tx=0).stop 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 shutdown_connections 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc8e0066f60 0x7fc8e0069420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc90015bc40 0x7fc900159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9000cf930 0x7fc900154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 --2- 172.21.15.87:0/3512503241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9000cda10 0x7fc900071d60 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 >> 172.21.15.87:0/3512503241 conn(0x7fc900066b70 msgr2=0x7fc900158460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:51.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 shutdown_connections 2024-04-25T10:23:51.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.604+0000 7fc905c00640 1 -- 172.21.15.87:0/3512503241 wait complete. 2024-04-25T10:23:51.623 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:23:51.624 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:23:51.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.756+0000 7f340f000640 1 Processor -- start 2024-04-25T10:23:51.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340f000640 1 -- start start 2024-04-25T10:23:51.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3408151930 0x7f3408070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340f000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f34080b2260 0x7f34080b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340f000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340815bf50 con 0x7f34080711f0 2024-04-25T10:23:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340f000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3408151d10 con 0x7f34080b2260 2024-04-25T10:23:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34080b3980 con 0x7f3408151930 2024-04-25T10:23:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340e600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f34080b2260 0x7f34080b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340d200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f34080711f0 0x7f34080b1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54140/0 (socket says 172.21.15.87:54140) 2024-04-25T10:23:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340d200640 1 -- 172.21.15.87:0/1797362633 learned_addr learned my addr 172.21.15.87:0/1797362633 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340dc00640 1 --2- 172.21.15.87:0/1797362633 >> v2:172.21.15.169:3300/0 conn(0x7f3408151930 0x7f3408070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340e600640 1 -- 172.21.15.87:0/1797362633 >> v2:172.21.15.169:3300/0 conn(0x7f3408151930 msgr2=0x7f3408070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340e600640 1 --2- 172.21.15.87:0/1797362633 >> v2:172.21.15.169:3300/0 conn(0x7f3408151930 0x7f3408070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340e600640 1 -- 172.21.15.87:0/1797362633 >> v1:172.21.15.87:6789/0 conn(0x7f34080711f0 legacy=0x7f34080b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.760+0000 7f340e600640 1 -- 172.21.15.87:0/1797362633 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340815b0a0 con 0x7f34080b2260 2024-04-25T10:23:51.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340e600640 1 --2- 172.21.15.87:0/1797362633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34080b2260 0x7f34080b40f0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f34080b2760 tx=0x7f340001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c229f1cd96e33e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f3407e00640 1 -- 172.21.15.87:0/1797362633 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000b2070 con 0x7f34080b2260 2024-04-25T10:23:51.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f3407e00640 1 -- 172.21.15.87:0/1797362633 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34000a5760 con 0x7f34080b2260 2024-04-25T10:23:51.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f3407e00640 1 -- 172.21.15.87:0/1797362633 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000a5b30 con 0x7f34080b2260 2024-04-25T10:23:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 -- 172.21.15.87:0/1797362633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34080b2260 msgr2=0x7f34080b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 --2- 172.21.15.87:0/1797362633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34080b2260 0x7f34080b40f0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f34080b2760 tx=0x7f340001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 -- 172.21.15.87:0/1797362633 shutdown_connections 2024-04-25T10:23:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 --2- 172.21.15.87:0/1797362633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34080b2260 0x7f34080b40f0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 --2- 172.21.15.87:0/1797362633 >> v2:172.21.15.169:3300/0 conn(0x7f3408151930 0x7f3408070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 -- 172.21.15.87:0/1797362633 >> 172.21.15.87:0/1797362633 conn(0x7f340806ac90 msgr2=0x7f340806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 -- 172.21.15.87:0/1797362633 shutdown_connections 2024-04-25T10:23:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 -- 172.21.15.87:0/1797362633 wait complete. 2024-04-25T10:23:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 Processor -- start 2024-04-25T10:23:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.764+0000 7f340f000640 1 -- start start 2024-04-25T10:23:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34080711f0 0x7f3408152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34080b2260 0x7f340813e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3408151930 0x7f3408159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34080b4fc0 con 0x7f3408151930 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34080b4e40 con 0x7f34080711f0 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34080b5140 con 0x7f34080b2260 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3408151930 0x7f3408159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3408151930 0x7f3408159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50158/0 (socket says 172.21.15.87:50158) 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 -- 172.21.15.87:0/4027919832 learned_addr learned my addr 172.21.15.87:0/4027919832 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340d200640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34080b2260 0x7f340813e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340dc00640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34080711f0 0x7f3408152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 -- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34080b2260 msgr2=0x7f340813e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34080b2260 0x7f340813e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 -- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34080711f0 msgr2=0x7f3408152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34080711f0 0x7f3408152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 -- 172.21.15.87:0/4027919832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3408154710 con 0x7f3408151930 2024-04-25T10:23:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340d200640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34080b2260 0x7f340813e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340e600640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3408151930 0x7f3408159860 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f34000194b0 tx=0x7f3400019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f3405600640 1 -- 172.21.15.87:0/4027919832 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000b2050 con 0x7f3408151930 2024-04-25T10:23:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f3405600640 1 -- 172.21.15.87:0/4027919832 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34000be070 con 0x7f3408151930 2024-04-25T10:23:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f3405600640 1 -- 172.21.15.87:0/4027919832 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000ad890 con 0x7f3408151930 2024-04-25T10:23:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3408152190 con 0x7f3408151930 2024-04-25T10:23:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3408152710 con 0x7f3408151930 2024-04-25T10:23:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3408074520 con 0x7f3408151930 2024-04-25T10:23:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.768+0000 7f3405600640 1 -- 172.21.15.87:0/4027919832 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34000ada30 con 0x7f3408151930 2024-04-25T10:23:51.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.772+0000 7f3405600640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:51.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.772+0000 7f3405600640 1 -- 172.21.15.87:0/4027919832 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f340011f920 con 0x7f3408151930 2024-04-25T10:23:51.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.772+0000 7f340dc00640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:51.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.772+0000 7f340dc00640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007a60 tx=0x7f33fc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:51.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.776+0000 7f3405600640 1 -- 172.21.15.87:0/4027919832 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34000c7020 con 0x7f3408151930 2024-04-25T10:23:51.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.924+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34080b3530 con 0x7f33ec066f90 2024-04-25T10:23:51.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.932+0000 7f3405600640 1 -- 172.21.15.87:0/4027919832 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7f34080b3530 con 0x7f33ec066f90 2024-04-25T10:23:51.935 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:51.937 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:23:51.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 msgr2=0x7f33ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007a60 tx=0x7f33fc008040 comp rx=0 tx=0).stop 2024-04-25T10:23:51.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3408151930 msgr2=0x7f3408159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3408151930 0x7f3408159860 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f34000194b0 tx=0x7f3400019640 comp rx=0 tx=0).stop 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 shutdown_connections 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f33ec066f90 0x7f33ec069450 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3408151930 0x7f3408159860 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34080b2260 0x7f340813e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 --2- 172.21.15.87:0/4027919832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34080711f0 0x7f3408152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 >> 172.21.15.87:0/4027919832 conn(0x7f340806ac90 msgr2=0x7f340814ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 shutdown_connections 2024-04-25T10:23:51.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:51.936+0000 7f340f000640 1 -- 172.21.15.87:0/4027919832 wait complete. 2024-04-25T10:23:51.956 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-04-25T10:23:51.269530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466736,"kb_used_data":16040,"kb_used_omap":12,"kb_used_meta":450547,"kb_avail":754507984,"statfs":{"total":773094113280,"available":772616175616,"internally_reserved":0,"allocated":16424960,"data_stored":7839756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":461360689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.428849"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965950+0000","last_change":"2024-04-25T10:23:47.899568+0000","last_active":"2024-04-25T10:23:48.965950+0000","last_peered":"2024-04-25T10:23:48.965950+0000","last_clean":"2024-04-25T10:23:48.965950+0000","last_became_active":"2024-04-25T10:23:47.899053+0000","last_became_peered":"2024-04-25T10:23:47.899053+0000","last_unstale":"2024-04-25T10:23:48.965950+0000","last_undegraded":"2024-04-25T10:23:48.965950+0000","last_fullsized":"2024-04-25T10:23:48.965950+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:43.819391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966156+0000","last_change":"2024-04-25T10:23:47.889897+0000","last_active":"2024-04-25T10:23:48.966156+0000","last_peered":"2024-04-25T10:23:48.966156+0000","last_clean":"2024-04-25T10:23:48.966156+0000","last_became_active":"2024-04-25T10:23:47.889640+0000","last_became_peered":"2024-04-25T10:23:47.889640+0000","last_unstale":"2024-04-25T10:23:48.966156+0000","last_undegraded":"2024-04-25T10:23:48.966156+0000","last_fullsized":"2024-04-25T10:23:48.966156+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:55.216062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893667+0000","last_change":"2024-04-25T10:23:47.875267+0000","last_active":"2024-04-25T10:23:48.893667+0000","last_peered":"2024-04-25T10:23:48.893667+0000","last_clean":"2024-04-25T10:23:48.893667+0000","last_became_active":"2024-04-25T10:23:47.874939+0000","last_became_peered":"2024-04-25T10:23:47.874939+0000","last_unstale":"2024-04-25T10:23:48.893667+0000","last_undegraded":"2024-04-25T10:23:48.893667+0000","last_fullsized":"2024-04-25T10:23:48.893667+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:11.402803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888591+0000","last_change":"2024-04-25T10:23:47.894328+0000","last_active":"2024-04-25T10:23:48.888591+0000","last_peered":"2024-04-25T10:23:48.888591+0000","last_clean":"2024-04-25T10:23:48.888591+0000","last_became_active":"2024-04-25T10:23:47.894003+0000","last_became_peered":"2024-04-25T10:23:47.894003+0000","last_unstale":"2024-04-25T10:23:48.888591+0000","last_undegraded":"2024-04-25T10:23:48.888591+0000","last_fullsized":"2024-04-25T10:23:48.888591+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:49.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966336+0000","last_change":"2024-04-25T10:23:47.894073+0000","last_active":"2024-04-25T10:23:48.966336+0000","last_peered":"2024-04-25T10:23:48.966336+0000","last_clean":"2024-04-25T10:23:48.966336+0000","last_became_active":"2024-04-25T10:23:47.893780+0000","last_became_peered":"2024-04-25T10:23:47.893780+0000","last_unstale":"2024-04-25T10:23:48.966336+0000","last_undegraded":"2024-04-25T10:23:48.966336+0000","last_fullsized":"2024-04-25T10:23:48.966336+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:04.202025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953119+0000","last_change":"2024-04-25T10:23:47.883884+0000","last_active":"2024-04-25T10:23:48.953119+0000","last_peered":"2024-04-25T10:23:48.953119+0000","last_clean":"2024-04-25T10:23:48.953119+0000","last_became_active":"2024-04-25T10:23:47.883539+0000","last_became_peered":"2024-04-25T10:23:47.883539+0000","last_unstale":"2024-04-25T10:23:48.953119+0000","last_undegraded":"2024-04-25T10:23:48.953119+0000","last_fullsized":"2024-04-25T10:23:48.953119+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:48.661447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964326+0000","last_change":"2024-04-25T10:23:47.888762+0000","last_active":"2024-04-25T10:23:48.964326+0000","last_peered":"2024-04-25T10:23:48.964326+0000","last_clean":"2024-04-25T10:23:48.964326+0000","last_became_active":"2024-04-25T10:23:47.888514+0000","last_became_peered":"2024-04-25T10:23:47.888514+0000","last_unstale":"2024-04-25T10:23:48.964326+0000","last_undegraded":"2024-04-25T10:23:48.964326+0000","last_fullsized":"2024-04-25T10:23:48.964326+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:08.157797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965443+0000","last_change":"2024-04-25T10:23:47.895127+0000","last_active":"2024-04-25T10:23:48.965443+0000","last_peered":"2024-04-25T10:23:48.965443+0000","last_clean":"2024-04-25T10:23:48.965443+0000","last_became_active":"2024-04-25T10:23:47.894863+0000","last_became_peered":"2024-04-25T10:23:47.894863+0000","last_unstale":"2024-04-25T10:23:48.965443+0000","last_undegraded":"2024-04-25T10:23:48.965443+0000","last_fullsized":"2024-04-25T10:23:48.965443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:49.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966483+0000","last_change":"2024-04-25T10:23:47.892750+0000","last_active":"2024-04-25T10:23:48.966483+0000","last_peered":"2024-04-25T10:23:48.966483+0000","last_clean":"2024-04-25T10:23:48.966483+0000","last_became_active":"2024-04-25T10:23:47.892474+0000","last_became_peered":"2024-04-25T10:23:47.892474+0000","last_unstale":"2024-04-25T10:23:48.966483+0000","last_undegraded":"2024-04-25T10:23:48.966483+0000","last_fullsized":"2024-04-25T10:23:48.966483+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:29.478098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966289+0000","last_change":"2024-04-25T10:23:47.898056+0000","last_active":"2024-04-25T10:23:48.966289+0000","last_peered":"2024-04-25T10:23:48.966289+0000","last_clean":"2024-04-25T10:23:48.966289+0000","last_became_active":"2024-04-25T10:23:47.897203+0000","last_became_peered":"2024-04-25T10:23:47.897203+0000","last_unstale":"2024-04-25T10:23:48.966289+0000","last_undegraded":"2024-04-25T10:23:48.966289+0000","last_fullsized":"2024-04-25T10:23:48.966289+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:47.981597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966159+0000","last_change":"2024-04-25T10:23:47.886143+0000","last_active":"2024-04-25T10:23:48.966159+0000","last_peered":"2024-04-25T10:23:48.966159+0000","last_clean":"2024-04-25T10:23:48.966159+0000","last_became_active":"2024-04-25T10:23:47.885886+0000","last_became_peered":"2024-04-25T10:23:47.885886+0000","last_unstale":"2024-04-25T10:23:48.966159+0000","last_undegraded":"2024-04-25T10:23:48.966159+0000","last_fullsized":"2024-04-25T10:23:48.966159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:25.121097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966139+0000","last_change":"2024-04-25T10:23:47.894171+0000","last_active":"2024-04-25T10:23:48.966139+0000","last_peered":"2024-04-25T10:23:48.966139+0000","last_clean":"2024-04-25T10:23:48.966139+0000","last_became_active":"2024-04-25T10:23:47.893895+0000","last_became_peered":"2024-04-25T10:23:47.893895+0000","last_unstale":"2024-04-25T10:23:48.966139+0000","last_undegraded":"2024-04-25T10:23:48.966139+0000","last_fullsized":"2024-04-25T10:23:48.966139+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:12.822532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893946+0000","last_change":"2024-04-25T10:23:47.871608+0000","last_active":"2024-04-25T10:23:48.893946+0000","last_peered":"2024-04-25T10:23:48.893946+0000","last_clean":"2024-04-25T10:23:48.893946+0000","last_became_active":"2024-04-25T10:23:47.871289+0000","last_became_peered":"2024-04-25T10:23:47.871289+0000","last_unstale":"2024-04-25T10:23:48.893946+0000","last_undegraded":"2024-04-25T10:23:48.893946+0000","last_fullsized":"2024-04-25T10:23:48.893946+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.645468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893644+0000","last_change":"2024-04-25T10:23:47.888245+0000","last_active":"2024-04-25T10:23:48.893644+0000","last_peered":"2024-04-25T10:23:48.893644+0000","last_clean":"2024-04-25T10:23:48.893644+0000","last_became_active":"2024-04-25T10:23:47.887948+0000","last_became_peered":"2024-04-25T10:23:47.887948+0000","last_unstale":"2024-04-25T10:23:48.893644+0000","last_undegraded":"2024-04-25T10:23:48.893644+0000","last_fullsized":"2024-04-25T10:23:48.893644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:54.056904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965253+0000","last_change":"2024-04-25T10:23:47.899024+0000","last_active":"2024-04-25T10:23:48.965253+0000","last_peered":"2024-04-25T10:23:48.965253+0000","last_clean":"2024-04-25T10:23:48.965253+0000","last_became_active":"2024-04-25T10:23:47.898751+0000","last_became_peered":"2024-04-25T10:23:47.898751+0000","last_unstale":"2024-04-25T10:23:48.965253+0000","last_undegraded":"2024-04-25T10:23:48.965253+0000","last_fullsized":"2024-04-25T10:23:48.965253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:08.235972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963856+0000","last_change":"2024-04-25T10:23:47.890526+0000","last_active":"2024-04-25T10:23:48.963856+0000","last_peered":"2024-04-25T10:23:48.963856+0000","last_clean":"2024-04-25T10:23:48.963856+0000","last_became_active":"2024-04-25T10:23:47.890228+0000","last_became_peered":"2024-04-25T10:23:47.890228+0000","last_unstale":"2024-04-25T10:23:48.963856+0000","last_undegraded":"2024-04-25T10:23:48.963856+0000","last_fullsized":"2024-04-25T10:23:48.963856+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:30.054530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953167+0000","last_change":"2024-04-25T10:23:47.884410+0000","last_active":"2024-04-25T10:23:48.953167+0000","last_peered":"2024-04-25T10:23:48.953167+0000","last_clean":"2024-04-25T10:23:48.953167+0000","last_became_active":"2024-04-25T10:23:47.884145+0000","last_became_peered":"2024-04-25T10:23:47.884145+0000","last_unstale":"2024-04-25T10:23:48.953167+0000","last_undegraded":"2024-04-25T10:23:48.953167+0000","last_fullsized":"2024-04-25T10:23:48.953167+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:22.710458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963861+0000","last_change":"2024-04-25T10:23:47.881653+0000","last_active":"2024-04-25T10:23:48.963861+0000","last_peered":"2024-04-25T10:23:48.963861+0000","last_clean":"2024-04-25T10:23:48.963861+0000","last_became_active":"2024-04-25T10:23:47.881234+0000","last_became_peered":"2024-04-25T10:23:47.881234+0000","last_unstale":"2024-04-25T10:23:48.963861+0000","last_undegraded":"2024-04-25T10:23:48.963861+0000","last_fullsized":"2024-04-25T10:23:48.963861+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:15.981407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953037+0000","last_change":"2024-04-25T10:23:47.890463+0000","last_active":"2024-04-25T10:23:48.953037+0000","last_peered":"2024-04-25T10:23:48.953037+0000","last_clean":"2024-04-25T10:23:48.953037+0000","last_became_active":"2024-04-25T10:23:47.890022+0000","last_became_peered":"2024-04-25T10:23:47.890022+0000","last_unstale":"2024-04-25T10:23:48.953037+0000","last_undegraded":"2024-04-25T10:23:48.953037+0000","last_fullsized":"2024-04-25T10:23:48.953037+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.959096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893616+0000","last_change":"2024-04-25T10:23:47.887796+0000","last_active":"2024-04-25T10:23:48.893616+0000","last_peered":"2024-04-25T10:23:48.893616+0000","last_clean":"2024-04-25T10:23:48.893616+0000","last_became_active":"2024-04-25T10:23:47.887477+0000","last_became_peered":"2024-04-25T10:23:47.887477+0000","last_unstale":"2024-04-25T10:23:48.893616+0000","last_undegraded":"2024-04-25T10:23:48.893616+0000","last_fullsized":"2024-04-25T10:23:48.893616+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:32.216715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953041+0000","last_change":"2024-04-25T10:23:47.881506+0000","last_active":"2024-04-25T10:23:48.953041+0000","last_peered":"2024-04-25T10:23:48.953041+0000","last_clean":"2024-04-25T10:23:48.953041+0000","last_became_active":"2024-04-25T10:23:47.881227+0000","last_became_peered":"2024-04-25T10:23:47.881227+0000","last_unstale":"2024-04-25T10:23:48.953041+0000","last_undegraded":"2024-04-25T10:23:48.953041+0000","last_fullsized":"2024-04-25T10:23:48.953041+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.477988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966243+0000","last_change":"2024-04-25T10:23:47.885757+0000","last_active":"2024-04-25T10:23:48.966243+0000","last_peered":"2024-04-25T10:23:48.966243+0000","last_clean":"2024-04-25T10:23:48.966243+0000","last_became_active":"2024-04-25T10:23:47.885430+0000","last_became_peered":"2024-04-25T10:23:47.885430+0000","last_unstale":"2024-04-25T10:23:48.966243+0000","last_undegraded":"2024-04-25T10:23:48.966243+0000","last_fullsized":"2024-04-25T10:23:48.966243+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.171701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952891+0000","last_change":"2024-04-25T10:23:47.895615+0000","last_active":"2024-04-25T10:23:48.952891+0000","last_peered":"2024-04-25T10:23:48.952891+0000","last_clean":"2024-04-25T10:23:48.952891+0000","last_became_active":"2024-04-25T10:23:47.893516+0000","last_became_peered":"2024-04-25T10:23:47.893516+0000","last_unstale":"2024-04-25T10:23:48.952891+0000","last_undegraded":"2024-04-25T10:23:48.952891+0000","last_fullsized":"2024-04-25T10:23:48.952891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:48.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965513+0000","last_change":"2024-04-25T10:23:47.882805+0000","last_active":"2024-04-25T10:23:48.965513+0000","last_peered":"2024-04-25T10:23:48.965513+0000","last_clean":"2024-04-25T10:23:48.965513+0000","last_became_active":"2024-04-25T10:23:47.882495+0000","last_became_peered":"2024-04-25T10:23:47.882495+0000","last_unstale":"2024-04-25T10:23:48.965513+0000","last_undegraded":"2024-04-25T10:23:48.965513+0000","last_fullsized":"2024-04-25T10:23:48.965513+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:07.567492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894052+0000","last_change":"2024-04-25T10:23:47.875169+0000","last_active":"2024-04-25T10:23:48.894052+0000","last_peered":"2024-04-25T10:23:48.894052+0000","last_clean":"2024-04-25T10:23:48.894052+0000","last_became_active":"2024-04-25T10:23:47.874868+0000","last_became_peered":"2024-04-25T10:23:47.874868+0000","last_unstale":"2024-04-25T10:23:48.894052+0000","last_undegraded":"2024-04-25T10:23:48.894052+0000","last_fullsized":"2024-04-25T10:23:48.894052+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:47.791721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964359+0000","last_change":"2024-04-25T10:23:47.896864+0000","last_active":"2024-04-25T10:23:48.964359+0000","last_peered":"2024-04-25T10:23:48.964359+0000","last_clean":"2024-04-25T10:23:48.964359+0000","last_became_active":"2024-04-25T10:23:47.896412+0000","last_became_peered":"2024-04-25T10:23:47.896412+0000","last_unstale":"2024-04-25T10:23:48.964359+0000","last_undegraded":"2024-04-25T10:23:48.964359+0000","last_fullsized":"2024-04-25T10:23:48.964359+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:00.433416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893936+0000","last_change":"2024-04-25T10:23:47.896472+0000","last_active":"2024-04-25T10:23:48.893936+0000","last_peered":"2024-04-25T10:23:48.893936+0000","last_clean":"2024-04-25T10:23:48.893936+0000","last_became_active":"2024-04-25T10:23:47.896192+0000","last_became_peered":"2024-04-25T10:23:47.896192+0000","last_unstale":"2024-04-25T10:23:48.893936+0000","last_undegraded":"2024-04-25T10:23:48.893936+0000","last_fullsized":"2024-04-25T10:23:48.893936+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:00.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888624+0000","last_change":"2024-04-25T10:23:47.895183+0000","last_active":"2024-04-25T10:23:48.888624+0000","last_peered":"2024-04-25T10:23:48.888624+0000","last_clean":"2024-04-25T10:23:48.888624+0000","last_became_active":"2024-04-25T10:23:47.894925+0000","last_became_peered":"2024-04-25T10:23:47.894925+0000","last_unstale":"2024-04-25T10:23:48.888624+0000","last_undegraded":"2024-04-25T10:23:48.888624+0000","last_fullsized":"2024-04-25T10:23:48.888624+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:07.567492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894169+0000","last_change":"2024-04-25T10:23:47.895937+0000","last_active":"2024-04-25T10:23:48.894169+0000","last_peered":"2024-04-25T10:23:48.894169+0000","last_clean":"2024-04-25T10:23:48.894169+0000","last_became_active":"2024-04-25T10:23:47.895527+0000","last_became_peered":"2024-04-25T10:23:47.895527+0000","last_unstale":"2024-04-25T10:23:48.894169+0000","last_undegraded":"2024-04-25T10:23:48.894169+0000","last_fullsized":"2024-04-25T10:23:48.894169+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:02.612218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893127+0000","last_change":"2024-04-25T10:23:47.891641+0000","last_active":"2024-04-25T10:23:48.893127+0000","last_peered":"2024-04-25T10:23:48.893127+0000","last_clean":"2024-04-25T10:23:48.893127+0000","last_became_active":"2024-04-25T10:23:47.891287+0000","last_became_peered":"2024-04-25T10:23:47.891287+0000","last_unstale":"2024-04-25T10:23:48.893127+0000","last_undegraded":"2024-04-25T10:23:48.893127+0000","last_fullsized":"2024-04-25T10:23:48.893127+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:36.498519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893089+0000","last_change":"2024-04-25T10:23:47.886157+0000","last_active":"2024-04-25T10:23:48.893089+0000","last_peered":"2024-04-25T10:23:48.893089+0000","last_clean":"2024-04-25T10:23:48.893089+0000","last_became_active":"2024-04-25T10:23:47.885802+0000","last_became_peered":"2024-04-25T10:23:47.885802+0000","last_unstale":"2024-04-25T10:23:48.893089+0000","last_undegraded":"2024-04-25T10:23:48.893089+0000","last_fullsized":"2024-04-25T10:23:48.893089+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:23.350803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963496+0000","last_change":"2024-04-25T10:23:47.882454+0000","last_active":"2024-04-25T10:23:48.963496+0000","last_peered":"2024-04-25T10:23:48.963496+0000","last_clean":"2024-04-25T10:23:48.963496+0000","last_became_active":"2024-04-25T10:23:47.882217+0000","last_became_peered":"2024-04-25T10:23:47.882217+0000","last_unstale":"2024-04-25T10:23:48.963496+0000","last_undegraded":"2024-04-25T10:23:48.963496+0000","last_fullsized":"2024-04-25T10:23:48.963496+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.996466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964531+0000","last_change":"2024-04-25T10:23:47.896657+0000","last_active":"2024-04-25T10:23:48.964531+0000","last_peered":"2024-04-25T10:23:48.964531+0000","last_clean":"2024-04-25T10:23:48.964531+0000","last_became_active":"2024-04-25T10:23:47.896047+0000","last_became_peered":"2024-04-25T10:23:47.896047+0000","last_unstale":"2024-04-25T10:23:48.964531+0000","last_undegraded":"2024-04-25T10:23:48.964531+0000","last_fullsized":"2024-04-25T10:23:48.964531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:15.029459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"150'4","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965900+0000","last_change":"2024-04-25T10:23:46.866817+0000","last_active":"2024-04-25T10:23:48.965900+0000","last_peered":"2024-04-25T10:23:48.965900+0000","last_clean":"2024-04-25T10:23:48.965900+0000","last_became_active":"2024-04-25T10:23:46.865717+0000","last_became_peered":"2024-04-25T10:23:46.865717+0000","last_unstale":"2024-04-25T10:23:48.965900+0000","last_undegraded":"2024-04-25T10:23:48.965900+0000","last_fullsized":"2024-04-25T10:23:48.965900+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:39.136065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894630+0000","last_change":"2024-04-25T10:23:47.873537+0000","last_active":"2024-04-25T10:23:48.894630+0000","last_peered":"2024-04-25T10:23:48.894630+0000","last_clean":"2024-04-25T10:23:48.894630+0000","last_became_active":"2024-04-25T10:23:47.873252+0000","last_became_peered":"2024-04-25T10:23:47.873252+0000","last_unstale":"2024-04-25T10:23:48.894630+0000","last_undegraded":"2024-04-25T10:23:48.894630+0000","last_fullsized":"2024-04-25T10:23:48.894630+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:25.114389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894419+0000","last_change":"2024-04-25T10:23:46.869057+0000","last_active":"2024-04-25T10:23:48.894419+0000","last_peered":"2024-04-25T10:23:48.894419+0000","last_clean":"2024-04-25T10:23:48.894419+0000","last_became_active":"2024-04-25T10:23:46.868748+0000","last_became_peered":"2024-04-25T10:23:46.868748+0000","last_unstale":"2024-04-25T10:23:48.894419+0000","last_undegraded":"2024-04-25T10:23:48.894419+0000","last_fullsized":"2024-04-25T10:23:48.894419+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:06.561084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952967+0000","last_change":"2024-04-25T10:23:47.889649+0000","last_active":"2024-04-25T10:23:48.952967+0000","last_peered":"2024-04-25T10:23:48.952967+0000","last_clean":"2024-04-25T10:23:48.952967+0000","last_became_active":"2024-04-25T10:23:47.888825+0000","last_became_peered":"2024-04-25T10:23:47.888825+0000","last_unstale":"2024-04-25T10:23:48.952967+0000","last_undegraded":"2024-04-25T10:23:48.952967+0000","last_fullsized":"2024-04-25T10:23:48.952967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:38.642494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888709+0000","last_change":"2024-04-25T10:23:46.868298+0000","last_active":"2024-04-25T10:23:48.888709+0000","last_peered":"2024-04-25T10:23:48.888709+0000","last_clean":"2024-04-25T10:23:48.888709+0000","last_became_active":"2024-04-25T10:23:46.868037+0000","last_became_peered":"2024-04-25T10:23:46.868037+0000","last_unstale":"2024-04-25T10:23:48.888709+0000","last_undegraded":"2024-04-25T10:23:48.888709+0000","last_fullsized":"2024-04-25T10:23:48.888709+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.210307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893903+0000","last_change":"2024-04-25T10:23:47.887973+0000","last_active":"2024-04-25T10:23:48.893903+0000","last_peered":"2024-04-25T10:23:48.893903+0000","last_clean":"2024-04-25T10:23:48.893903+0000","last_became_active":"2024-04-25T10:23:47.887673+0000","last_became_peered":"2024-04-25T10:23:47.887673+0000","last_unstale":"2024-04-25T10:23:48.893903+0000","last_undegraded":"2024-04-25T10:23:48.893903+0000","last_fullsized":"2024-04-25T10:23:48.893903+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.437799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888676+0000","last_change":"2024-04-25T10:23:46.864177+0000","last_active":"2024-04-25T10:23:48.888676+0000","last_peered":"2024-04-25T10:23:48.888676+0000","last_clean":"2024-04-25T10:23:48.888676+0000","last_became_active":"2024-04-25T10:23:46.863865+0000","last_became_peered":"2024-04-25T10:23:46.863865+0000","last_unstale":"2024-04-25T10:23:48.888676+0000","last_undegraded":"2024-04-25T10:23:48.888676+0000","last_fullsized":"2024-04-25T10:23:48.888676+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:48.005183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964370+0000","last_change":"2024-04-25T10:23:47.892558+0000","last_active":"2024-04-25T10:23:48.964370+0000","last_peered":"2024-04-25T10:23:48.964370+0000","last_clean":"2024-04-25T10:23:48.964370+0000","last_became_active":"2024-04-25T10:23:47.892292+0000","last_became_peered":"2024-04-25T10:23:47.892292+0000","last_unstale":"2024-04-25T10:23:48.964370+0000","last_undegraded":"2024-04-25T10:23:48.964370+0000","last_fullsized":"2024-04-25T10:23:48.964370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:15.431452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894203+0000","last_change":"2024-04-25T10:23:46.859300+0000","last_active":"2024-04-25T10:23:48.894203+0000","last_peered":"2024-04-25T10:23:48.894203+0000","last_clean":"2024-04-25T10:23:48.894203+0000","last_became_active":"2024-04-25T10:23:46.858976+0000","last_became_peered":"2024-04-25T10:23:46.858976+0000","last_unstale":"2024-04-25T10:23:48.894203+0000","last_undegraded":"2024-04-25T10:23:48.894203+0000","last_fullsized":"2024-04-25T10:23:48.894203+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:21.704050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888511+0000","last_change":"2024-04-25T10:23:47.888684+0000","last_active":"2024-04-25T10:23:48.888511+0000","last_peered":"2024-04-25T10:23:48.888511+0000","last_clean":"2024-04-25T10:23:48.888511+0000","last_became_active":"2024-04-25T10:23:47.888435+0000","last_became_peered":"2024-04-25T10:23:47.888435+0000","last_unstale":"2024-04-25T10:23:48.888511+0000","last_undegraded":"2024-04-25T10:23:48.888511+0000","last_fullsized":"2024-04-25T10:23:48.888511+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.000858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888357+0000","last_change":"2024-04-25T10:23:46.864168+0000","last_active":"2024-04-25T10:23:48.888357+0000","last_peered":"2024-04-25T10:23:48.888357+0000","last_clean":"2024-04-25T10:23:48.888357+0000","last_became_active":"2024-04-25T10:23:46.863926+0000","last_became_peered":"2024-04-25T10:23:46.863926+0000","last_unstale":"2024-04-25T10:23:48.888357+0000","last_undegraded":"2024-04-25T10:23:48.888357+0000","last_fullsized":"2024-04-25T10:23:48.888357+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:34.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888332+0000","last_change":"2024-04-25T10:23:47.892605+0000","last_active":"2024-04-25T10:23:48.888332+0000","last_peered":"2024-04-25T10:23:48.888332+0000","last_clean":"2024-04-25T10:23:48.888332+0000","last_became_active":"2024-04-25T10:23:47.887431+0000","last_became_peered":"2024-04-25T10:23:47.887431+0000","last_unstale":"2024-04-25T10:23:48.888332+0000","last_undegraded":"2024-04-25T10:23:48.888332+0000","last_fullsized":"2024-04-25T10:23:48.888332+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:02.612218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965514+0000","last_change":"2024-04-25T10:23:46.871892+0000","last_active":"2024-04-25T10:23:48.965514+0000","last_peered":"2024-04-25T10:23:48.965514+0000","last_clean":"2024-04-25T10:23:48.965514+0000","last_became_active":"2024-04-25T10:23:46.871516+0000","last_became_peered":"2024-04-25T10:23:46.871516+0000","last_unstale":"2024-04-25T10:23:48.965514+0000","last_undegraded":"2024-04-25T10:23:48.965514+0000","last_fullsized":"2024-04-25T10:23:48.965514+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:38.885940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952853+0000","last_change":"2024-04-25T10:23:47.883796+0000","last_active":"2024-04-25T10:23:48.952853+0000","last_peered":"2024-04-25T10:23:48.952853+0000","last_clean":"2024-04-25T10:23:48.952853+0000","last_became_active":"2024-04-25T10:23:47.883526+0000","last_became_peered":"2024-04-25T10:23:47.883526+0000","last_unstale":"2024-04-25T10:23:48.952853+0000","last_undegraded":"2024-04-25T10:23:48.952853+0000","last_fullsized":"2024-04-25T10:23:48.952853+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:28.082437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963985+0000","last_change":"2024-04-25T10:23:46.859215+0000","last_active":"2024-04-25T10:23:48.963985+0000","last_peered":"2024-04-25T10:23:48.963985+0000","last_clean":"2024-04-25T10:23:48.963985+0000","last_became_active":"2024-04-25T10:23:46.858249+0000","last_became_peered":"2024-04-25T10:23:46.858249+0000","last_unstale":"2024-04-25T10:23:48.963985+0000","last_undegraded":"2024-04-25T10:23:48.963985+0000","last_fullsized":"2024-04-25T10:23:48.963985+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:28.471690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964159+0000","last_change":"2024-04-25T10:23:47.894132+0000","last_active":"2024-04-25T10:23:48.964159+0000","last_peered":"2024-04-25T10:23:48.964159+0000","last_clean":"2024-04-25T10:23:48.964159+0000","last_became_active":"2024-04-25T10:23:47.893888+0000","last_became_peered":"2024-04-25T10:23:47.893888+0000","last_unstale":"2024-04-25T10:23:48.964159+0000","last_undegraded":"2024-04-25T10:23:48.964159+0000","last_fullsized":"2024-04-25T10:23:48.964159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.316322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953133+0000","last_change":"2024-04-25T10:23:46.866252+0000","last_active":"2024-04-25T10:23:48.953133+0000","last_peered":"2024-04-25T10:23:48.953133+0000","last_clean":"2024-04-25T10:23:48.953133+0000","last_became_active":"2024-04-25T10:23:46.865507+0000","last_became_peered":"2024-04-25T10:23:46.865507+0000","last_unstale":"2024-04-25T10:23:48.953133+0000","last_undegraded":"2024-04-25T10:23:48.953133+0000","last_fullsized":"2024-04-25T10:23:48.953133+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:36.714122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963829+0000","last_change":"2024-04-25T10:23:47.893355+0000","last_active":"2024-04-25T10:23:48.963829+0000","last_peered":"2024-04-25T10:23:48.963829+0000","last_clean":"2024-04-25T10:23:48.963829+0000","last_became_active":"2024-04-25T10:23:47.893089+0000","last_became_peered":"2024-04-25T10:23:47.893089+0000","last_unstale":"2024-04-25T10:23:48.963829+0000","last_undegraded":"2024-04-25T10:23:48.963829+0000","last_fullsized":"2024-04-25T10:23:48.963829+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:30.802161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"150'3","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953045+0000","last_change":"2024-04-25T10:23:46.861040+0000","last_active":"2024-04-25T10:23:48.953045+0000","last_peered":"2024-04-25T10:23:48.953045+0000","last_clean":"2024-04-25T10:23:48.953045+0000","last_became_active":"2024-04-25T10:23:46.860641+0000","last_became_peered":"2024-04-25T10:23:46.860641+0000","last_unstale":"2024-04-25T10:23:48.953045+0000","last_undegraded":"2024-04-25T10:23:48.953045+0000","last_fullsized":"2024-04-25T10:23:48.953045+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:48.098600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894630+0000","last_change":"2024-04-25T10:23:47.874004+0000","last_active":"2024-04-25T10:23:48.894630+0000","last_peered":"2024-04-25T10:23:48.894630+0000","last_clean":"2024-04-25T10:23:48.894630+0000","last_became_active":"2024-04-25T10:23:47.873585+0000","last_became_peered":"2024-04-25T10:23:47.873585+0000","last_unstale":"2024-04-25T10:23:48.894630+0000","last_undegraded":"2024-04-25T10:23:48.894630+0000","last_fullsized":"2024-04-25T10:23:48.894630+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:12.933405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"150'6","reported_seq":20,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894112+0000","last_change":"2024-04-25T10:23:46.858960+0000","last_active":"2024-04-25T10:23:48.894112+0000","last_peered":"2024-04-25T10:23:48.894112+0000","last_clean":"2024-04-25T10:23:48.894112+0000","last_became_active":"2024-04-25T10:23:46.858706+0000","last_became_peered":"2024-04-25T10:23:46.858706+0000","last_unstale":"2024-04-25T10:23:48.894112+0000","last_undegraded":"2024-04-25T10:23:48.894112+0000","last_fullsized":"2024-04-25T10:23:48.894112+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:09.994450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963578+0000","last_change":"2024-04-25T10:23:47.880639+0000","last_active":"2024-04-25T10:23:48.963578+0000","last_peered":"2024-04-25T10:23:48.963578+0000","last_clean":"2024-04-25T10:23:48.963578+0000","last_became_active":"2024-04-25T10:23:47.880379+0000","last_became_peered":"2024-04-25T10:23:47.880379+0000","last_unstale":"2024-04-25T10:23:48.963578+0000","last_undegraded":"2024-04-25T10:23:48.963578+0000","last_fullsized":"2024-04-25T10:23:48.963578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:15.029459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888564+0000","last_change":"2024-04-25T10:23:46.862900+0000","last_active":"2024-04-25T10:23:48.888564+0000","last_peered":"2024-04-25T10:23:48.888564+0000","last_clean":"2024-04-25T10:23:48.888564+0000","last_became_active":"2024-04-25T10:23:46.862644+0000","last_became_peered":"2024-04-25T10:23:46.862644+0000","last_unstale":"2024-04-25T10:23:48.888564+0000","last_undegraded":"2024-04-25T10:23:48.888564+0000","last_fullsized":"2024-04-25T10:23:48.888564+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:35.415987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964588+0000","last_change":"2024-04-25T10:23:47.886608+0000","last_active":"2024-04-25T10:23:48.964588+0000","last_peered":"2024-04-25T10:23:48.964588+0000","last_clean":"2024-04-25T10:23:48.964588+0000","last_became_active":"2024-04-25T10:23:47.885880+0000","last_became_peered":"2024-04-25T10:23:47.885880+0000","last_unstale":"2024-04-25T10:23:48.964588+0000","last_undegraded":"2024-04-25T10:23:48.964588+0000","last_fullsized":"2024-04-25T10:23:48.964588+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:29.133237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963629+0000","last_change":"2024-04-25T10:23:46.862780+0000","last_active":"2024-04-25T10:23:48.963629+0000","last_peered":"2024-04-25T10:23:48.963629+0000","last_clean":"2024-04-25T10:23:48.963629+0000","last_became_active":"2024-04-25T10:23:46.861367+0000","last_became_peered":"2024-04-25T10:23:46.861367+0000","last_unstale":"2024-04-25T10:23:48.963629+0000","last_undegraded":"2024-04-25T10:23:48.963629+0000","last_fullsized":"2024-04-25T10:23:48.963629+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:22.344396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888477+0000","last_change":"2024-04-25T10:23:47.883952+0000","last_active":"2024-04-25T10:23:48.888477+0000","last_peered":"2024-04-25T10:23:48.888477+0000","last_clean":"2024-04-25T10:23:48.888477+0000","last_became_active":"2024-04-25T10:23:47.879298+0000","last_became_peered":"2024-04-25T10:23:47.879298+0000","last_unstale":"2024-04-25T10:23:48.888477+0000","last_undegraded":"2024-04-25T10:23:48.888477+0000","last_fullsized":"2024-04-25T10:23:48.888477+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:44.448189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888310+0000","last_change":"2024-04-25T10:23:46.867853+0000","last_active":"2024-04-25T10:23:48.888310+0000","last_peered":"2024-04-25T10:23:48.888310+0000","last_clean":"2024-04-25T10:23:48.888310+0000","last_became_active":"2024-04-25T10:23:46.867459+0000","last_became_peered":"2024-04-25T10:23:46.867459+0000","last_unstale":"2024-04-25T10:23:48.888310+0000","last_undegraded":"2024-04-25T10:23:48.888310+0000","last_fullsized":"2024-04-25T10:23:48.888310+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:55.737582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893012+0000","last_change":"2024-04-25T10:23:47.883973+0000","last_active":"2024-04-25T10:23:48.893012+0000","last_peered":"2024-04-25T10:23:48.893012+0000","last_clean":"2024-04-25T10:23:48.893012+0000","last_became_active":"2024-04-25T10:23:47.883704+0000","last_became_peered":"2024-04-25T10:23:47.883704+0000","last_unstale":"2024-04-25T10:23:48.893012+0000","last_undegraded":"2024-04-25T10:23:48.893012+0000","last_fullsized":"2024-04-25T10:23:48.893012+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:56.743990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"150'8","reported_seq":22,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893190+0000","last_change":"2024-04-25T10:23:46.858934+0000","last_active":"2024-04-25T10:23:48.893190+0000","last_peered":"2024-04-25T10:23:48.893190+0000","last_clean":"2024-04-25T10:23:48.893190+0000","last_became_active":"2024-04-25T10:23:46.858661+0000","last_became_peered":"2024-04-25T10:23:46.858661+0000","last_unstale":"2024-04-25T10:23:48.893190+0000","last_undegraded":"2024-04-25T10:23:48.893190+0000","last_fullsized":"2024-04-25T10:23:48.893190+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:48.244025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888957+0000","last_change":"2024-04-25T10:23:47.888992+0000","last_active":"2024-04-25T10:23:48.888957+0000","last_peered":"2024-04-25T10:23:48.888957+0000","last_clean":"2024-04-25T10:23:48.888957+0000","last_became_active":"2024-04-25T10:23:47.885530+0000","last_became_peered":"2024-04-25T10:23:47.885530+0000","last_unstale":"2024-04-25T10:23:48.888957+0000","last_undegraded":"2024-04-25T10:23:48.888957+0000","last_fullsized":"2024-04-25T10:23:48.888957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:25.121097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894005+0000","last_change":"2024-04-25T10:23:46.870270+0000","last_active":"2024-04-25T10:23:48.894005+0000","last_peered":"2024-04-25T10:23:48.894005+0000","last_clean":"2024-04-25T10:23:48.894005+0000","last_became_active":"2024-04-25T10:23:46.870037+0000","last_became_peered":"2024-04-25T10:23:46.870037+0000","last_unstale":"2024-04-25T10:23:48.894005+0000","last_undegraded":"2024-04-25T10:23:48.894005+0000","last_fullsized":"2024-04-25T10:23:48.894005+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:33.481936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964137+0000","last_change":"2024-04-25T10:23:47.876542+0000","last_active":"2024-04-25T10:23:48.964137+0000","last_peered":"2024-04-25T10:23:48.964137+0000","last_clean":"2024-04-25T10:23:48.964137+0000","last_became_active":"2024-04-25T10:23:47.876239+0000","last_became_peered":"2024-04-25T10:23:47.876239+0000","last_unstale":"2024-04-25T10:23:48.964137+0000","last_undegraded":"2024-04-25T10:23:48.964137+0000","last_fullsized":"2024-04-25T10:23:48.964137+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:40.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893819+0000","last_change":"2024-04-25T10:23:46.857710+0000","last_active":"2024-04-25T10:23:48.893819+0000","last_peered":"2024-04-25T10:23:48.893819+0000","last_clean":"2024-04-25T10:23:48.893819+0000","last_became_active":"2024-04-25T10:23:46.857424+0000","last_became_peered":"2024-04-25T10:23:46.857424+0000","last_unstale":"2024-04-25T10:23:48.893819+0000","last_undegraded":"2024-04-25T10:23:48.893819+0000","last_fullsized":"2024-04-25T10:23:48.893819+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:50.190366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888741+0000","last_change":"2024-04-25T10:23:47.886113+0000","last_active":"2024-04-25T10:23:48.888741+0000","last_peered":"2024-04-25T10:23:48.888741+0000","last_clean":"2024-04-25T10:23:48.888741+0000","last_became_active":"2024-04-25T10:23:47.885854+0000","last_became_peered":"2024-04-25T10:23:47.885854+0000","last_unstale":"2024-04-25T10:23:48.888741+0000","last_undegraded":"2024-04-25T10:23:48.888741+0000","last_fullsized":"2024-04-25T10:23:48.888741+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:12.822532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965866+0000","last_change":"2024-04-25T10:23:46.864350+0000","last_active":"2024-04-25T10:23:48.965866+0000","last_peered":"2024-04-25T10:23:48.965866+0000","last_clean":"2024-04-25T10:23:48.965866+0000","last_became_active":"2024-04-25T10:23:46.863870+0000","last_became_peered":"2024-04-25T10:23:46.863870+0000","last_unstale":"2024-04-25T10:23:48.965866+0000","last_undegraded":"2024-04-25T10:23:48.965866+0000","last_fullsized":"2024-04-25T10:23:48.965866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:27.404149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964436+0000","last_change":"2024-04-25T10:23:47.891062+0000","last_active":"2024-04-25T10:23:48.964436+0000","last_peered":"2024-04-25T10:23:48.964436+0000","last_clean":"2024-04-25T10:23:48.964436+0000","last_became_active":"2024-04-25T10:23:47.890772+0000","last_became_peered":"2024-04-25T10:23:47.890772+0000","last_unstale":"2024-04-25T10:23:48.964436+0000","last_undegraded":"2024-04-25T10:23:48.964436+0000","last_fullsized":"2024-04-25T10:23:48.964436+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:15.374928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965724+0000","last_change":"2024-04-25T10:23:46.869636+0000","last_active":"2024-04-25T10:23:48.965724+0000","last_peered":"2024-04-25T10:23:48.965724+0000","last_clean":"2024-04-25T10:23:48.965724+0000","last_became_active":"2024-04-25T10:23:46.869018+0000","last_became_peered":"2024-04-25T10:23:46.869018+0000","last_unstale":"2024-04-25T10:23:48.965724+0000","last_undegraded":"2024-04-25T10:23:48.965724+0000","last_fullsized":"2024-04-25T10:23:48.965724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:48.131868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893466+0000","last_change":"2024-04-25T10:23:47.873959+0000","last_active":"2024-04-25T10:23:48.893466+0000","last_peered":"2024-04-25T10:23:48.893466+0000","last_clean":"2024-04-25T10:23:48.893466+0000","last_became_active":"2024-04-25T10:23:47.873043+0000","last_became_peered":"2024-04-25T10:23:47.873043+0000","last_unstale":"2024-04-25T10:23:48.893466+0000","last_undegraded":"2024-04-25T10:23:48.893466+0000","last_fullsized":"2024-04-25T10:23:48.893466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:57.731770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888753+0000","last_change":"2024-04-25T10:23:46.867256+0000","last_active":"2024-04-25T10:23:48.888753+0000","last_peered":"2024-04-25T10:23:48.888753+0000","last_clean":"2024-04-25T10:23:48.888753+0000","last_became_active":"2024-04-25T10:23:46.866787+0000","last_became_peered":"2024-04-25T10:23:46.866787+0000","last_unstale":"2024-04-25T10:23:48.888753+0000","last_undegraded":"2024-04-25T10:23:48.888753+0000","last_fullsized":"2024-04-25T10:23:48.888753+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:10.260415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964653+0000","last_change":"2024-04-25T10:23:47.886189+0000","last_active":"2024-04-25T10:23:48.964653+0000","last_peered":"2024-04-25T10:23:48.964653+0000","last_clean":"2024-04-25T10:23:48.964653+0000","last_became_active":"2024-04-25T10:23:47.882053+0000","last_became_peered":"2024-04-25T10:23:47.882053+0000","last_unstale":"2024-04-25T10:23:48.964653+0000","last_undegraded":"2024-04-25T10:23:48.964653+0000","last_fullsized":"2024-04-25T10:23:48.964653+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:30.802161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965515+0000","last_change":"2024-04-25T10:23:46.869218+0000","last_active":"2024-04-25T10:23:48.965515+0000","last_peered":"2024-04-25T10:23:48.965515+0000","last_clean":"2024-04-25T10:23:48.965515+0000","last_became_active":"2024-04-25T10:23:46.868941+0000","last_became_peered":"2024-04-25T10:23:46.868941+0000","last_unstale":"2024-04-25T10:23:48.965515+0000","last_undegraded":"2024-04-25T10:23:48.965515+0000","last_fullsized":"2024-04-25T10:23:48.965515+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:43.899274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965287+0000","last_change":"2024-04-25T10:23:47.899199+0000","last_active":"2024-04-25T10:23:48.965287+0000","last_peered":"2024-04-25T10:23:48.965287+0000","last_clean":"2024-04-25T10:23:48.965287+0000","last_became_active":"2024-04-25T10:23:47.898887+0000","last_became_peered":"2024-04-25T10:23:47.898887+0000","last_unstale":"2024-04-25T10:23:48.965287+0000","last_undegraded":"2024-04-25T10:23:48.965287+0000","last_fullsized":"2024-04-25T10:23:48.965287+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:58.656369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888335+0000","last_change":"2024-04-25T10:23:46.858444+0000","last_active":"2024-04-25T10:23:48.888335+0000","last_peered":"2024-04-25T10:23:48.888335+0000","last_clean":"2024-04-25T10:23:48.888335+0000","last_became_active":"2024-04-25T10:23:46.858164+0000","last_became_peered":"2024-04-25T10:23:46.858164+0000","last_unstale":"2024-04-25T10:23:48.888335+0000","last_undegraded":"2024-04-25T10:23:48.888335+0000","last_fullsized":"2024-04-25T10:23:48.888335+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:38.885940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952997+0000","last_change":"2024-04-25T10:23:47.893576+0000","last_active":"2024-04-25T10:23:48.952997+0000","last_peered":"2024-04-25T10:23:48.952997+0000","last_clean":"2024-04-25T10:23:48.952997+0000","last_became_active":"2024-04-25T10:23:47.893296+0000","last_became_peered":"2024-04-25T10:23:47.893296+0000","last_unstale":"2024-04-25T10:23:48.952997+0000","last_undegraded":"2024-04-25T10:23:48.952997+0000","last_fullsized":"2024-04-25T10:23:48.952997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:10.048909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965339+0000","last_change":"2024-04-25T10:23:46.862330+0000","last_active":"2024-04-25T10:23:48.965339+0000","last_peered":"2024-04-25T10:23:48.965339+0000","last_clean":"2024-04-25T10:23:48.965339+0000","last_became_active":"2024-04-25T10:23:46.862027+0000","last_became_peered":"2024-04-25T10:23:46.862027+0000","last_unstale":"2024-04-25T10:23:48.965339+0000","last_undegraded":"2024-04-25T10:23:48.965339+0000","last_fullsized":"2024-04-25T10:23:48.965339+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:46.309914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964292+0000","last_change":"2024-04-25T10:23:47.895761+0000","last_active":"2024-04-25T10:23:48.964292+0000","last_peered":"2024-04-25T10:23:48.964292+0000","last_clean":"2024-04-25T10:23:48.964292+0000","last_became_active":"2024-04-25T10:23:47.895475+0000","last_became_peered":"2024-04-25T10:23:47.895475+0000","last_unstale":"2024-04-25T10:23:48.964292+0000","last_undegraded":"2024-04-25T10:23:48.964292+0000","last_fullsized":"2024-04-25T10:23:48.964292+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:49.138276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964188+0000","last_change":"2024-04-25T10:23:46.867975+0000","last_active":"2024-04-25T10:23:48.964188+0000","last_peered":"2024-04-25T10:23:48.964188+0000","last_clean":"2024-04-25T10:23:48.964188+0000","last_became_active":"2024-04-25T10:23:46.867653+0000","last_became_peered":"2024-04-25T10:23:46.867653+0000","last_unstale":"2024-04-25T10:23:48.964188+0000","last_undegraded":"2024-04-25T10:23:48.964188+0000","last_fullsized":"2024-04-25T10:23:48.964188+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:54.775658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964505+0000","last_change":"2024-04-25T10:23:47.888040+0000","last_active":"2024-04-25T10:23:48.964505+0000","last_peered":"2024-04-25T10:23:48.964505+0000","last_clean":"2024-04-25T10:23:48.964505+0000","last_became_active":"2024-04-25T10:23:47.887784+0000","last_became_peered":"2024-04-25T10:23:47.887784+0000","last_unstale":"2024-04-25T10:23:48.964505+0000","last_undegraded":"2024-04-25T10:23:48.964505+0000","last_fullsized":"2024-04-25T10:23:48.964505+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.996466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893878+0000","last_change":"2024-04-25T10:23:46.857071+0000","last_active":"2024-04-25T10:23:48.893878+0000","last_peered":"2024-04-25T10:23:48.893878+0000","last_clean":"2024-04-25T10:23:48.893878+0000","last_became_active":"2024-04-25T10:23:46.856792+0000","last_became_peered":"2024-04-25T10:23:46.856792+0000","last_unstale":"2024-04-25T10:23:48.893878+0000","last_undegraded":"2024-04-25T10:23:48.893878+0000","last_fullsized":"2024-04-25T10:23:48.893878+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:40.182535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952963+0000","last_change":"2024-04-25T10:23:47.891766+0000","last_active":"2024-04-25T10:23:48.952963+0000","last_peered":"2024-04-25T10:23:48.952963+0000","last_clean":"2024-04-25T10:23:48.952963+0000","last_became_active":"2024-04-25T10:23:47.891508+0000","last_became_peered":"2024-04-25T10:23:47.891508+0000","last_unstale":"2024-04-25T10:23:48.952963+0000","last_undegraded":"2024-04-25T10:23:48.952963+0000","last_fullsized":"2024-04-25T10:23:48.952963+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.853909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966219+0000","last_change":"2024-04-25T10:23:46.870852+0000","last_active":"2024-04-25T10:23:48.966219+0000","last_peered":"2024-04-25T10:23:48.966219+0000","last_clean":"2024-04-25T10:23:48.966219+0000","last_became_active":"2024-04-25T10:23:46.870574+0000","last_became_peered":"2024-04-25T10:23:46.870574+0000","last_unstale":"2024-04-25T10:23:48.966219+0000","last_undegraded":"2024-04-25T10:23:48.966219+0000","last_fullsized":"2024-04-25T10:23:48.966219+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.210307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963887+0000","last_change":"2024-04-25T10:23:47.887091+0000","last_active":"2024-04-25T10:23:48.963887+0000","last_peered":"2024-04-25T10:23:48.963887+0000","last_clean":"2024-04-25T10:23:48.963887+0000","last_became_active":"2024-04-25T10:23:47.886663+0000","last_became_peered":"2024-04-25T10:23:47.886663+0000","last_unstale":"2024-04-25T10:23:48.963887+0000","last_undegraded":"2024-04-25T10:23:48.963887+0000","last_fullsized":"2024-04-25T10:23:48.963887+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:44.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963761+0000","last_change":"2024-04-25T10:23:46.853251+0000","last_active":"2024-04-25T10:23:48.963761+0000","last_peered":"2024-04-25T10:23:48.963761+0000","last_clean":"2024-04-25T10:23:48.963761+0000","last_became_active":"2024-04-25T10:23:46.852966+0000","last_became_peered":"2024-04-25T10:23:46.852966+0000","last_unstale":"2024-04-25T10:23:48.963761+0000","last_undegraded":"2024-04-25T10:23:48.963761+0000","last_fullsized":"2024-04-25T10:23:48.963761+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.492112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963617+0000","last_change":"2024-04-25T10:23:47.879029+0000","last_active":"2024-04-25T10:23:48.963617+0000","last_peered":"2024-04-25T10:23:48.963617+0000","last_clean":"2024-04-25T10:23:48.963617+0000","last_became_active":"2024-04-25T10:23:47.878769+0000","last_became_peered":"2024-04-25T10:23:47.878769+0000","last_unstale":"2024-04-25T10:23:48.963617+0000","last_undegraded":"2024-04-25T10:23:48.963617+0000","last_fullsized":"2024-04-25T10:23:48.963617+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:29.133237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893545+0000","last_change":"2024-04-25T10:23:46.856632+0000","last_active":"2024-04-25T10:23:48.893545+0000","last_peered":"2024-04-25T10:23:48.893545+0000","last_clean":"2024-04-25T10:23:48.893545+0000","last_became_active":"2024-04-25T10:23:46.856153+0000","last_became_peered":"2024-04-25T10:23:46.856153+0000","last_unstale":"2024-04-25T10:23:48.893545+0000","last_undegraded":"2024-04-25T10:23:48.893545+0000","last_fullsized":"2024-04-25T10:23:48.893545+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:24.114690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893488+0000","last_change":"2024-04-25T10:23:47.891149+0000","last_active":"2024-04-25T10:23:48.893488+0000","last_peered":"2024-04-25T10:23:48.893488+0000","last_clean":"2024-04-25T10:23:48.893488+0000","last_became_active":"2024-04-25T10:23:47.890835+0000","last_became_peered":"2024-04-25T10:23:47.890835+0000","last_unstale":"2024-04-25T10:23:48.893488+0000","last_undegraded":"2024-04-25T10:23:48.893488+0000","last_fullsized":"2024-04-25T10:23:48.893488+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:55.782065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888340+0000","last_change":"2024-04-25T10:23:46.860202+0000","last_active":"2024-04-25T10:23:48.888340+0000","last_peered":"2024-04-25T10:23:48.888340+0000","last_clean":"2024-04-25T10:23:48.888340+0000","last_became_active":"2024-04-25T10:23:46.859950+0000","last_became_peered":"2024-04-25T10:23:46.859950+0000","last_unstale":"2024-04-25T10:23:48.888340+0000","last_undegraded":"2024-04-25T10:23:48.888340+0000","last_fullsized":"2024-04-25T10:23:48.888340+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:10.201593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963851+0000","last_change":"2024-04-25T10:23:47.890965+0000","last_active":"2024-04-25T10:23:48.963851+0000","last_peered":"2024-04-25T10:23:48.963851+0000","last_clean":"2024-04-25T10:23:48.963851+0000","last_became_active":"2024-04-25T10:23:47.889986+0000","last_became_peered":"2024-04-25T10:23:47.889986+0000","last_unstale":"2024-04-25T10:23:48.963851+0000","last_undegraded":"2024-04-25T10:23:48.963851+0000","last_fullsized":"2024-04-25T10:23:48.963851+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:49.541419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965784+0000","last_change":"2024-04-25T10:23:46.869313+0000","last_active":"2024-04-25T10:23:48.965784+0000","last_peered":"2024-04-25T10:23:48.965784+0000","last_clean":"2024-04-25T10:23:48.965784+0000","last_became_active":"2024-04-25T10:23:46.868519+0000","last_became_peered":"2024-04-25T10:23:46.868519+0000","last_unstale":"2024-04-25T10:23:48.965784+0000","last_undegraded":"2024-04-25T10:23:48.965784+0000","last_fullsized":"2024-04-25T10:23:48.965784+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:56.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893241+0000","last_change":"2024-04-25T10:23:47.882533+0000","last_active":"2024-04-25T10:23:48.893241+0000","last_peered":"2024-04-25T10:23:48.893241+0000","last_clean":"2024-04-25T10:23:48.893241+0000","last_became_active":"2024-04-25T10:23:47.881581+0000","last_became_peered":"2024-04-25T10:23:47.881581+0000","last_unstale":"2024-04-25T10:23:48.893241+0000","last_undegraded":"2024-04-25T10:23:48.893241+0000","last_fullsized":"2024-04-25T10:23:48.893241+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:39.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893053+0000","last_change":"2024-04-25T10:23:46.862519+0000","last_active":"2024-04-25T10:23:48.893053+0000","last_peered":"2024-04-25T10:23:48.893053+0000","last_clean":"2024-04-25T10:23:48.893053+0000","last_became_active":"2024-04-25T10:23:46.862239+0000","last_became_peered":"2024-04-25T10:23:46.862239+0000","last_unstale":"2024-04-25T10:23:48.893053+0000","last_undegraded":"2024-04-25T10:23:48.893053+0000","last_fullsized":"2024-04-25T10:23:48.893053+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:11.816124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963984+0000","last_change":"2024-04-25T10:23:47.896019+0000","last_active":"2024-04-25T10:23:48.963984+0000","last_peered":"2024-04-25T10:23:48.963984+0000","last_clean":"2024-04-25T10:23:48.963984+0000","last_became_active":"2024-04-25T10:23:47.895781+0000","last_became_peered":"2024-04-25T10:23:47.895781+0000","last_unstale":"2024-04-25T10:23:48.963984+0000","last_undegraded":"2024-04-25T10:23:48.963984+0000","last_fullsized":"2024-04-25T10:23:48.963984+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:28.410557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894564+0000","last_change":"2024-04-25T10:23:46.865259+0000","last_active":"2024-04-25T10:23:48.894564+0000","last_peered":"2024-04-25T10:23:48.894564+0000","last_clean":"2024-04-25T10:23:48.894564+0000","last_became_active":"2024-04-25T10:23:46.864959+0000","last_became_peered":"2024-04-25T10:23:46.864959+0000","last_unstale":"2024-04-25T10:23:48.894564+0000","last_undegraded":"2024-04-25T10:23:48.894564+0000","last_fullsized":"2024-04-25T10:23:48.894564+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:28.471690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":406,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965455+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:23:48.965455+0000","last_peered":"2024-04-25T10:23:48.965455+0000","last_clean":"2024-04-25T10:23:48.965455+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:23:48.965455+0000","last_undegraded":"2024-04-25T10:23:48.965455+0000","last_fullsized":"2024-04-25T10:23:48.965455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60012,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311828,"statfs":{"total":96636764160,"available":96575311872,"internally_reserved":0,"allocated":2514944,"data_stored":1424497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64908,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306932,"statfs":{"total":96636764160,"available":96570298368,"internally_reserved":0,"allocated":1900544,"data_stored":831797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51088,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320752,"statfs":{"total":96636764160,"available":96584450048,"internally_reserved":0,"allocated":1904640,"data_stored":832379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":1912832,"data_stored":833457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574992,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55248,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316592,"statfs":{"total":96636764160,"available":96580190208,"internally_reserved":0,"allocated":1900544,"data_stored":831809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55252,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316588,"statfs":{"total":96636764160,"available":96580186112,"internally_reserved":0,"allocated":1900544,"data_stored":831865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306864,"statfs":{"total":96636764160,"available":96570228736,"internally_reserved":0,"allocated":1900544,"data_stored":831809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55828,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316012,"statfs":{"total":96636764160,"available":96579596288,"internally_reserved":0,"allocated":2490368,"data_stored":1422143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:23:51.961 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:23:52.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0008000640 1 Processor -- start 2024-04-25T10:23:52.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0008000640 1 -- start start 2024-04-25T10:23:52.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0008000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f000015bc40 0x7f000006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0008000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00000b2770 con 0x7f00000cda10 2024-04-25T10:23:52.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0008000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000006c670 con 0x7f00000cf930 2024-04-25T10:23:52.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0008000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000006c950 con 0x7f000015bc40 2024-04-25T10:23:52.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0004c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f00000cda10 0x7f00000b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54154/0 (socket says 172.21.15.87:54154) 2024-04-25T10:23:52.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0004c00640 1 -- 172.21.15.87:0/1797192604 learned_addr learned my addr 172.21.15.87:0/1797192604 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:52.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7f0005600640 1 --2- 172.21.15.87:0/1797192604 >> v2:172.21.15.169:3300/0 conn(0x7f000015bc40 0x7f000006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623810607 0 0) 0x7f00000b2770 con 0x7f00000cda10 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.136+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe8003530 con 0x7f00000cda10 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1583356339 0 0) 0x7effe8003530 con 0x7f00000cda10 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 >> v2:172.21.15.169:3300/0 conn(0x7f000015bc40 msgr2=0x7f000006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 --2- 172.21.15.87:0/1797192604 >> v2:172.21.15.169:3300/0 conn(0x7f000015bc40 0x7f000006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 >> v1:172.21.15.169:6789/0 conn(0x7f00000cf930 legacy=0x7f00000b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000006cc30 con 0x7f00000cda10 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7effec001680 con 0x7f00000cda10 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 >> v1:172.21.15.87:6789/0 conn(0x7f00000cda10 legacy=0x7f00000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effe80039c0 0x7effe8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe8004810 0x7effe8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80051d0 0x7effe800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000006c950 con 0x7effe80051d0 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe800c170 con 0x7effe80039c0 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe800c370 con 0x7effe8004810 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0005600640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effe80039c0 0x7effe8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0006000640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80051d0 0x7effe800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0004c00640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe8004810 0x7effe8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0005600640 1 -- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe8004810 msgr2=0x7effe8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0005600640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe8004810 0x7effe8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0005600640 1 -- 172.21.15.87:0/1797192604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80051d0 msgr2=0x7effe800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0005600640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80051d0 0x7effe800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0005600640 1 -- 172.21.15.87:0/1797192604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe8003530 con 0x7effe80039c0 2024-04-25T10:23:52.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.140+0000 7f0006000640 1 -- 172.21.15.87:0/1797192604 reap_dead start 2024-04-25T10:23:52.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0005600640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effe80039c0 0x7effe8003de0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efff4009fc0 tx=0x7efff401cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59482a29bca45a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40b2070 con 0x7effe80039c0 2024-04-25T10:23:52.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff40a5c90 con 0x7effe80039c0 2024-04-25T10:23:52.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7effffe00640 1 -- 172.21.15.87:0/1797192604 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4009c10 con 0x7effe80039c0 2024-04-25T10:23:52.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effe80039c0 msgr2=0x7effe8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effe80039c0 0x7effe8003de0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efff4009fc0 tx=0x7efff401cc80 comp rx=0 tx=0).stop 2024-04-25T10:23:52.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- 172.21.15.87:0/1797192604 shutdown_connections 2024-04-25T10:23:52.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 --2- 172.21.15.87:0/1797192604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effe80039c0 0x7effe8003de0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- 172.21.15.87:0/1797192604 >> 172.21.15.87:0/1797192604 conn(0x7f0000066b70 msgr2=0x7f0000066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:52.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- 172.21.15.87:0/1797192604 shutdown_connections 2024-04-25T10:23:52.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- 172.21.15.87:0/1797192604 wait complete. 2024-04-25T10:23:52.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 Processor -- start 2024-04-25T10:23:52.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- start start 2024-04-25T10:23:52.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe80039c0 0x7f000015c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00000afbd0 0x7f0000071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f000015d750 0x7f00000ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000006c210 con 0x7f00000afbd0 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000006c090 con 0x7f000015d750 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0008000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000006c390 con 0x7effe80039c0 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0004c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00000afbd0 0x7f0000071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0004c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00000afbd0 0x7f0000071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50174/0 (socket says 172.21.15.87:50174) 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0004c00640 1 -- 172.21.15.87:0/601922963 learned_addr learned my addr 172.21.15.87:0/601922963 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.144+0000 7f0006000640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f000015d750 0x7f00000ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0005600640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe80039c0 0x7f000015c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0004c00640 1 -- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe80039c0 msgr2=0x7f000015c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0004c00640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe80039c0 0x7f000015c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0004c00640 1 -- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f000015d750 msgr2=0x7f00000ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0004c00640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f000015d750 0x7f00000ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0004c00640 1 -- 172.21.15.87:0/601922963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000015c6b0 con 0x7f00000afbd0 2024-04-25T10:23:52.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0004c00640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00000afbd0 0x7f0000071e00 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7effec0124f0 tx=0x7effec0129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7efffd600640 1 -- 172.21.15.87:0/601922963 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec01c070 con 0x7f00000afbd0 2024-04-25T10:23:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7efffd600640 1 -- 172.21.15.87:0/601922963 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec014830 con 0x7f00000afbd0 2024-04-25T10:23:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000015c990 con 0x7f00000afbd0 2024-04-25T10:23:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7efffd600640 1 -- 172.21.15.87:0/601922963 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec014c20 con 0x7f00000afbd0 2024-04-25T10:23:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000015cf50 con 0x7f00000afbd0 2024-04-25T10:23:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000004ef30 con 0x7f00000afbd0 2024-04-25T10:23:52.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7efffd600640 1 -- 172.21.15.87:0/601922963 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effec01f070 con 0x7f00000afbd0 2024-04-25T10:23:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7efffd600640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effe0066f60 0x7effe0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7efffd600640 1 -- 172.21.15.87:0/601922963 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effec087a70 con 0x7f00000afbd0 2024-04-25T10:23:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.148+0000 7f0005600640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effe0066f60 0x7effe0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.152+0000 7f0005600640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effe0066f60 0x7effe0069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efff40af000 tx=0x7efff4004ac0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.152+0000 7efffd600640 1 -- 172.21.15.87:0/601922963 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effec058300 con 0x7f00000afbd0 2024-04-25T10:23:52.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.304+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0000072340 con 0x7effe0066f60 2024-04-25T10:23:52.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.312+0000 7efffd600640 1 -- 172.21.15.87:0/601922963 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7efff40a5b10 con 0x7effe0066f60 2024-04-25T10:23:52.313 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:52.314 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effe0066f60 msgr2=0x7effe0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effe0066f60 0x7effe0069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efff40af000 tx=0x7efff4004ac0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00000afbd0 msgr2=0x7f0000071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00000afbd0 0x7f0000071e00 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7effec0124f0 tx=0x7effec0129b0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 shutdown_connections 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7effe0066f60 0x7effe0069420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f000015d750 0x7f00000ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00000afbd0 0x7f0000071e00 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 --2- 172.21.15.87:0/601922963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effe80039c0 0x7f000015c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 >> 172.21.15.87:0/601922963 conn(0x7f0000066b70 msgr2=0x7f000015f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 shutdown_connections 2024-04-25T10:23:52.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.316+0000 7f0008000640 1 -- 172.21.15.87:0/601922963 wait complete. 2024-04-25T10:23:52.334 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-04-25T10:23:51.269530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466736,"kb_used_data":16040,"kb_used_omap":12,"kb_used_meta":450547,"kb_avail":754507984,"statfs":{"total":773094113280,"available":772616175616,"internally_reserved":0,"allocated":16424960,"data_stored":7839756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":461360689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.428849"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965950+0000","last_change":"2024-04-25T10:23:47.899568+0000","last_active":"2024-04-25T10:23:48.965950+0000","last_peered":"2024-04-25T10:23:48.965950+0000","last_clean":"2024-04-25T10:23:48.965950+0000","last_became_active":"2024-04-25T10:23:47.899053+0000","last_became_peered":"2024-04-25T10:23:47.899053+0000","last_unstale":"2024-04-25T10:23:48.965950+0000","last_undegraded":"2024-04-25T10:23:48.965950+0000","last_fullsized":"2024-04-25T10:23:48.965950+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:43.819391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966156+0000","last_change":"2024-04-25T10:23:47.889897+0000","last_active":"2024-04-25T10:23:48.966156+0000","last_peered":"2024-04-25T10:23:48.966156+0000","last_clean":"2024-04-25T10:23:48.966156+0000","last_became_active":"2024-04-25T10:23:47.889640+0000","last_became_peered":"2024-04-25T10:23:47.889640+0000","last_unstale":"2024-04-25T10:23:48.966156+0000","last_undegraded":"2024-04-25T10:23:48.966156+0000","last_fullsized":"2024-04-25T10:23:48.966156+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:55.216062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893667+0000","last_change":"2024-04-25T10:23:47.875267+0000","last_active":"2024-04-25T10:23:48.893667+0000","last_peered":"2024-04-25T10:23:48.893667+0000","last_clean":"2024-04-25T10:23:48.893667+0000","last_became_active":"2024-04-25T10:23:47.874939+0000","last_became_peered":"2024-04-25T10:23:47.874939+0000","last_unstale":"2024-04-25T10:23:48.893667+0000","last_undegraded":"2024-04-25T10:23:48.893667+0000","last_fullsized":"2024-04-25T10:23:48.893667+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:11.402803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888591+0000","last_change":"2024-04-25T10:23:47.894328+0000","last_active":"2024-04-25T10:23:48.888591+0000","last_peered":"2024-04-25T10:23:48.888591+0000","last_clean":"2024-04-25T10:23:48.888591+0000","last_became_active":"2024-04-25T10:23:47.894003+0000","last_became_peered":"2024-04-25T10:23:47.894003+0000","last_unstale":"2024-04-25T10:23:48.888591+0000","last_undegraded":"2024-04-25T10:23:48.888591+0000","last_fullsized":"2024-04-25T10:23:48.888591+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:49.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966336+0000","last_change":"2024-04-25T10:23:47.894073+0000","last_active":"2024-04-25T10:23:48.966336+0000","last_peered":"2024-04-25T10:23:48.966336+0000","last_clean":"2024-04-25T10:23:48.966336+0000","last_became_active":"2024-04-25T10:23:47.893780+0000","last_became_peered":"2024-04-25T10:23:47.893780+0000","last_unstale":"2024-04-25T10:23:48.966336+0000","last_undegraded":"2024-04-25T10:23:48.966336+0000","last_fullsized":"2024-04-25T10:23:48.966336+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:04.202025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953119+0000","last_change":"2024-04-25T10:23:47.883884+0000","last_active":"2024-04-25T10:23:48.953119+0000","last_peered":"2024-04-25T10:23:48.953119+0000","last_clean":"2024-04-25T10:23:48.953119+0000","last_became_active":"2024-04-25T10:23:47.883539+0000","last_became_peered":"2024-04-25T10:23:47.883539+0000","last_unstale":"2024-04-25T10:23:48.953119+0000","last_undegraded":"2024-04-25T10:23:48.953119+0000","last_fullsized":"2024-04-25T10:23:48.953119+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:48.661447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964326+0000","last_change":"2024-04-25T10:23:47.888762+0000","last_active":"2024-04-25T10:23:48.964326+0000","last_peered":"2024-04-25T10:23:48.964326+0000","last_clean":"2024-04-25T10:23:48.964326+0000","last_became_active":"2024-04-25T10:23:47.888514+0000","last_became_peered":"2024-04-25T10:23:47.888514+0000","last_unstale":"2024-04-25T10:23:48.964326+0000","last_undegraded":"2024-04-25T10:23:48.964326+0000","last_fullsized":"2024-04-25T10:23:48.964326+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:08.157797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965443+0000","last_change":"2024-04-25T10:23:47.895127+0000","last_active":"2024-04-25T10:23:48.965443+0000","last_peered":"2024-04-25T10:23:48.965443+0000","last_clean":"2024-04-25T10:23:48.965443+0000","last_became_active":"2024-04-25T10:23:47.894863+0000","last_became_peered":"2024-04-25T10:23:47.894863+0000","last_unstale":"2024-04-25T10:23:48.965443+0000","last_undegraded":"2024-04-25T10:23:48.965443+0000","last_fullsized":"2024-04-25T10:23:48.965443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:49.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966483+0000","last_change":"2024-04-25T10:23:47.892750+0000","last_active":"2024-04-25T10:23:48.966483+0000","last_peered":"2024-04-25T10:23:48.966483+0000","last_clean":"2024-04-25T10:23:48.966483+0000","last_became_active":"2024-04-25T10:23:47.892474+0000","last_became_peered":"2024-04-25T10:23:47.892474+0000","last_unstale":"2024-04-25T10:23:48.966483+0000","last_undegraded":"2024-04-25T10:23:48.966483+0000","last_fullsized":"2024-04-25T10:23:48.966483+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:29.478098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966289+0000","last_change":"2024-04-25T10:23:47.898056+0000","last_active":"2024-04-25T10:23:48.966289+0000","last_peered":"2024-04-25T10:23:48.966289+0000","last_clean":"2024-04-25T10:23:48.966289+0000","last_became_active":"2024-04-25T10:23:47.897203+0000","last_became_peered":"2024-04-25T10:23:47.897203+0000","last_unstale":"2024-04-25T10:23:48.966289+0000","last_undegraded":"2024-04-25T10:23:48.966289+0000","last_fullsized":"2024-04-25T10:23:48.966289+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:47.981597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966159+0000","last_change":"2024-04-25T10:23:47.886143+0000","last_active":"2024-04-25T10:23:48.966159+0000","last_peered":"2024-04-25T10:23:48.966159+0000","last_clean":"2024-04-25T10:23:48.966159+0000","last_became_active":"2024-04-25T10:23:47.885886+0000","last_became_peered":"2024-04-25T10:23:47.885886+0000","last_unstale":"2024-04-25T10:23:48.966159+0000","last_undegraded":"2024-04-25T10:23:48.966159+0000","last_fullsized":"2024-04-25T10:23:48.966159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:25.121097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966139+0000","last_change":"2024-04-25T10:23:47.894171+0000","last_active":"2024-04-25T10:23:48.966139+0000","last_peered":"2024-04-25T10:23:48.966139+0000","last_clean":"2024-04-25T10:23:48.966139+0000","last_became_active":"2024-04-25T10:23:47.893895+0000","last_became_peered":"2024-04-25T10:23:47.893895+0000","last_unstale":"2024-04-25T10:23:48.966139+0000","last_undegraded":"2024-04-25T10:23:48.966139+0000","last_fullsized":"2024-04-25T10:23:48.966139+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:12.822532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893946+0000","last_change":"2024-04-25T10:23:47.871608+0000","last_active":"2024-04-25T10:23:48.893946+0000","last_peered":"2024-04-25T10:23:48.893946+0000","last_clean":"2024-04-25T10:23:48.893946+0000","last_became_active":"2024-04-25T10:23:47.871289+0000","last_became_peered":"2024-04-25T10:23:47.871289+0000","last_unstale":"2024-04-25T10:23:48.893946+0000","last_undegraded":"2024-04-25T10:23:48.893946+0000","last_fullsized":"2024-04-25T10:23:48.893946+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.645468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893644+0000","last_change":"2024-04-25T10:23:47.888245+0000","last_active":"2024-04-25T10:23:48.893644+0000","last_peered":"2024-04-25T10:23:48.893644+0000","last_clean":"2024-04-25T10:23:48.893644+0000","last_became_active":"2024-04-25T10:23:47.887948+0000","last_became_peered":"2024-04-25T10:23:47.887948+0000","last_unstale":"2024-04-25T10:23:48.893644+0000","last_undegraded":"2024-04-25T10:23:48.893644+0000","last_fullsized":"2024-04-25T10:23:48.893644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:54.056904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965253+0000","last_change":"2024-04-25T10:23:47.899024+0000","last_active":"2024-04-25T10:23:48.965253+0000","last_peered":"2024-04-25T10:23:48.965253+0000","last_clean":"2024-04-25T10:23:48.965253+0000","last_became_active":"2024-04-25T10:23:47.898751+0000","last_became_peered":"2024-04-25T10:23:47.898751+0000","last_unstale":"2024-04-25T10:23:48.965253+0000","last_undegraded":"2024-04-25T10:23:48.965253+0000","last_fullsized":"2024-04-25T10:23:48.965253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:08.235972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963856+0000","last_change":"2024-04-25T10:23:47.890526+0000","last_active":"2024-04-25T10:23:48.963856+0000","last_peered":"2024-04-25T10:23:48.963856+0000","last_clean":"2024-04-25T10:23:48.963856+0000","last_became_active":"2024-04-25T10:23:47.890228+0000","last_became_peered":"2024-04-25T10:23:47.890228+0000","last_unstale":"2024-04-25T10:23:48.963856+0000","last_undegraded":"2024-04-25T10:23:48.963856+0000","last_fullsized":"2024-04-25T10:23:48.963856+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:30.054530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953167+0000","last_change":"2024-04-25T10:23:47.884410+0000","last_active":"2024-04-25T10:23:48.953167+0000","last_peered":"2024-04-25T10:23:48.953167+0000","last_clean":"2024-04-25T10:23:48.953167+0000","last_became_active":"2024-04-25T10:23:47.884145+0000","last_became_peered":"2024-04-25T10:23:47.884145+0000","last_unstale":"2024-04-25T10:23:48.953167+0000","last_undegraded":"2024-04-25T10:23:48.953167+0000","last_fullsized":"2024-04-25T10:23:48.953167+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:22.710458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963861+0000","last_change":"2024-04-25T10:23:47.881653+0000","last_active":"2024-04-25T10:23:48.963861+0000","last_peered":"2024-04-25T10:23:48.963861+0000","last_clean":"2024-04-25T10:23:48.963861+0000","last_became_active":"2024-04-25T10:23:47.881234+0000","last_became_peered":"2024-04-25T10:23:47.881234+0000","last_unstale":"2024-04-25T10:23:48.963861+0000","last_undegraded":"2024-04-25T10:23:48.963861+0000","last_fullsized":"2024-04-25T10:23:48.963861+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:15.981407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953037+0000","last_change":"2024-04-25T10:23:47.890463+0000","last_active":"2024-04-25T10:23:48.953037+0000","last_peered":"2024-04-25T10:23:48.953037+0000","last_clean":"2024-04-25T10:23:48.953037+0000","last_became_active":"2024-04-25T10:23:47.890022+0000","last_became_peered":"2024-04-25T10:23:47.890022+0000","last_unstale":"2024-04-25T10:23:48.953037+0000","last_undegraded":"2024-04-25T10:23:48.953037+0000","last_fullsized":"2024-04-25T10:23:48.953037+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.959096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893616+0000","last_change":"2024-04-25T10:23:47.887796+0000","last_active":"2024-04-25T10:23:48.893616+0000","last_peered":"2024-04-25T10:23:48.893616+0000","last_clean":"2024-04-25T10:23:48.893616+0000","last_became_active":"2024-04-25T10:23:47.887477+0000","last_became_peered":"2024-04-25T10:23:47.887477+0000","last_unstale":"2024-04-25T10:23:48.893616+0000","last_undegraded":"2024-04-25T10:23:48.893616+0000","last_fullsized":"2024-04-25T10:23:48.893616+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:32.216715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953041+0000","last_change":"2024-04-25T10:23:47.881506+0000","last_active":"2024-04-25T10:23:48.953041+0000","last_peered":"2024-04-25T10:23:48.953041+0000","last_clean":"2024-04-25T10:23:48.953041+0000","last_became_active":"2024-04-25T10:23:47.881227+0000","last_became_peered":"2024-04-25T10:23:47.881227+0000","last_unstale":"2024-04-25T10:23:48.953041+0000","last_undegraded":"2024-04-25T10:23:48.953041+0000","last_fullsized":"2024-04-25T10:23:48.953041+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.477988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966243+0000","last_change":"2024-04-25T10:23:47.885757+0000","last_active":"2024-04-25T10:23:48.966243+0000","last_peered":"2024-04-25T10:23:48.966243+0000","last_clean":"2024-04-25T10:23:48.966243+0000","last_became_active":"2024-04-25T10:23:47.885430+0000","last_became_peered":"2024-04-25T10:23:47.885430+0000","last_unstale":"2024-04-25T10:23:48.966243+0000","last_undegraded":"2024-04-25T10:23:48.966243+0000","last_fullsized":"2024-04-25T10:23:48.966243+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.171701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952891+0000","last_change":"2024-04-25T10:23:47.895615+0000","last_active":"2024-04-25T10:23:48.952891+0000","last_peered":"2024-04-25T10:23:48.952891+0000","last_clean":"2024-04-25T10:23:48.952891+0000","last_became_active":"2024-04-25T10:23:47.893516+0000","last_became_peered":"2024-04-25T10:23:47.893516+0000","last_unstale":"2024-04-25T10:23:48.952891+0000","last_undegraded":"2024-04-25T10:23:48.952891+0000","last_fullsized":"2024-04-25T10:23:48.952891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:48.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965513+0000","last_change":"2024-04-25T10:23:47.882805+0000","last_active":"2024-04-25T10:23:48.965513+0000","last_peered":"2024-04-25T10:23:48.965513+0000","last_clean":"2024-04-25T10:23:48.965513+0000","last_became_active":"2024-04-25T10:23:47.882495+0000","last_became_peered":"2024-04-25T10:23:47.882495+0000","last_unstale":"2024-04-25T10:23:48.965513+0000","last_undegraded":"2024-04-25T10:23:48.965513+0000","last_fullsized":"2024-04-25T10:23:48.965513+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:07.567492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894052+0000","last_change":"2024-04-25T10:23:47.875169+0000","last_active":"2024-04-25T10:23:48.894052+0000","last_peered":"2024-04-25T10:23:48.894052+0000","last_clean":"2024-04-25T10:23:48.894052+0000","last_became_active":"2024-04-25T10:23:47.874868+0000","last_became_peered":"2024-04-25T10:23:47.874868+0000","last_unstale":"2024-04-25T10:23:48.894052+0000","last_undegraded":"2024-04-25T10:23:48.894052+0000","last_fullsized":"2024-04-25T10:23:48.894052+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:47.791721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964359+0000","last_change":"2024-04-25T10:23:47.896864+0000","last_active":"2024-04-25T10:23:48.964359+0000","last_peered":"2024-04-25T10:23:48.964359+0000","last_clean":"2024-04-25T10:23:48.964359+0000","last_became_active":"2024-04-25T10:23:47.896412+0000","last_became_peered":"2024-04-25T10:23:47.896412+0000","last_unstale":"2024-04-25T10:23:48.964359+0000","last_undegraded":"2024-04-25T10:23:48.964359+0000","last_fullsized":"2024-04-25T10:23:48.964359+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:00.433416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893936+0000","last_change":"2024-04-25T10:23:47.896472+0000","last_active":"2024-04-25T10:23:48.893936+0000","last_peered":"2024-04-25T10:23:48.893936+0000","last_clean":"2024-04-25T10:23:48.893936+0000","last_became_active":"2024-04-25T10:23:47.896192+0000","last_became_peered":"2024-04-25T10:23:47.896192+0000","last_unstale":"2024-04-25T10:23:48.893936+0000","last_undegraded":"2024-04-25T10:23:48.893936+0000","last_fullsized":"2024-04-25T10:23:48.893936+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:00.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888624+0000","last_change":"2024-04-25T10:23:47.895183+0000","last_active":"2024-04-25T10:23:48.888624+0000","last_peered":"2024-04-25T10:23:48.888624+0000","last_clean":"2024-04-25T10:23:48.888624+0000","last_became_active":"2024-04-25T10:23:47.894925+0000","last_became_peered":"2024-04-25T10:23:47.894925+0000","last_unstale":"2024-04-25T10:23:48.888624+0000","last_undegraded":"2024-04-25T10:23:48.888624+0000","last_fullsized":"2024-04-25T10:23:48.888624+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:07.567492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894169+0000","last_change":"2024-04-25T10:23:47.895937+0000","last_active":"2024-04-25T10:23:48.894169+0000","last_peered":"2024-04-25T10:23:48.894169+0000","last_clean":"2024-04-25T10:23:48.894169+0000","last_became_active":"2024-04-25T10:23:47.895527+0000","last_became_peered":"2024-04-25T10:23:47.895527+0000","last_unstale":"2024-04-25T10:23:48.894169+0000","last_undegraded":"2024-04-25T10:23:48.894169+0000","last_fullsized":"2024-04-25T10:23:48.894169+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:02.612218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893127+0000","last_change":"2024-04-25T10:23:47.891641+0000","last_active":"2024-04-25T10:23:48.893127+0000","last_peered":"2024-04-25T10:23:48.893127+0000","last_clean":"2024-04-25T10:23:48.893127+0000","last_became_active":"2024-04-25T10:23:47.891287+0000","last_became_peered":"2024-04-25T10:23:47.891287+0000","last_unstale":"2024-04-25T10:23:48.893127+0000","last_undegraded":"2024-04-25T10:23:48.893127+0000","last_fullsized":"2024-04-25T10:23:48.893127+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:36.498519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893089+0000","last_change":"2024-04-25T10:23:47.886157+0000","last_active":"2024-04-25T10:23:48.893089+0000","last_peered":"2024-04-25T10:23:48.893089+0000","last_clean":"2024-04-25T10:23:48.893089+0000","last_became_active":"2024-04-25T10:23:47.885802+0000","last_became_peered":"2024-04-25T10:23:47.885802+0000","last_unstale":"2024-04-25T10:23:48.893089+0000","last_undegraded":"2024-04-25T10:23:48.893089+0000","last_fullsized":"2024-04-25T10:23:48.893089+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:23.350803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963496+0000","last_change":"2024-04-25T10:23:47.882454+0000","last_active":"2024-04-25T10:23:48.963496+0000","last_peered":"2024-04-25T10:23:48.963496+0000","last_clean":"2024-04-25T10:23:48.963496+0000","last_became_active":"2024-04-25T10:23:47.882217+0000","last_became_peered":"2024-04-25T10:23:47.882217+0000","last_unstale":"2024-04-25T10:23:48.963496+0000","last_undegraded":"2024-04-25T10:23:48.963496+0000","last_fullsized":"2024-04-25T10:23:48.963496+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.996466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964531+0000","last_change":"2024-04-25T10:23:47.896657+0000","last_active":"2024-04-25T10:23:48.964531+0000","last_peered":"2024-04-25T10:23:48.964531+0000","last_clean":"2024-04-25T10:23:48.964531+0000","last_became_active":"2024-04-25T10:23:47.896047+0000","last_became_peered":"2024-04-25T10:23:47.896047+0000","last_unstale":"2024-04-25T10:23:48.964531+0000","last_undegraded":"2024-04-25T10:23:48.964531+0000","last_fullsized":"2024-04-25T10:23:48.964531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:15.029459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"150'4","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965900+0000","last_change":"2024-04-25T10:23:46.866817+0000","last_active":"2024-04-25T10:23:48.965900+0000","last_peered":"2024-04-25T10:23:48.965900+0000","last_clean":"2024-04-25T10:23:48.965900+0000","last_became_active":"2024-04-25T10:23:46.865717+0000","last_became_peered":"2024-04-25T10:23:46.865717+0000","last_unstale":"2024-04-25T10:23:48.965900+0000","last_undegraded":"2024-04-25T10:23:48.965900+0000","last_fullsized":"2024-04-25T10:23:48.965900+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:39.136065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894630+0000","last_change":"2024-04-25T10:23:47.873537+0000","last_active":"2024-04-25T10:23:48.894630+0000","last_peered":"2024-04-25T10:23:48.894630+0000","last_clean":"2024-04-25T10:23:48.894630+0000","last_became_active":"2024-04-25T10:23:47.873252+0000","last_became_peered":"2024-04-25T10:23:47.873252+0000","last_unstale":"2024-04-25T10:23:48.894630+0000","last_undegraded":"2024-04-25T10:23:48.894630+0000","last_fullsized":"2024-04-25T10:23:48.894630+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:25.114389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894419+0000","last_change":"2024-04-25T10:23:46.869057+0000","last_active":"2024-04-25T10:23:48.894419+0000","last_peered":"2024-04-25T10:23:48.894419+0000","last_clean":"2024-04-25T10:23:48.894419+0000","last_became_active":"2024-04-25T10:23:46.868748+0000","last_became_peered":"2024-04-25T10:23:46.868748+0000","last_unstale":"2024-04-25T10:23:48.894419+0000","last_undegraded":"2024-04-25T10:23:48.894419+0000","last_fullsized":"2024-04-25T10:23:48.894419+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:06.561084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952967+0000","last_change":"2024-04-25T10:23:47.889649+0000","last_active":"2024-04-25T10:23:48.952967+0000","last_peered":"2024-04-25T10:23:48.952967+0000","last_clean":"2024-04-25T10:23:48.952967+0000","last_became_active":"2024-04-25T10:23:47.888825+0000","last_became_peered":"2024-04-25T10:23:47.888825+0000","last_unstale":"2024-04-25T10:23:48.952967+0000","last_undegraded":"2024-04-25T10:23:48.952967+0000","last_fullsized":"2024-04-25T10:23:48.952967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:38.642494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888709+0000","last_change":"2024-04-25T10:23:46.868298+0000","last_active":"2024-04-25T10:23:48.888709+0000","last_peered":"2024-04-25T10:23:48.888709+0000","last_clean":"2024-04-25T10:23:48.888709+0000","last_became_active":"2024-04-25T10:23:46.868037+0000","last_became_peered":"2024-04-25T10:23:46.868037+0000","last_unstale":"2024-04-25T10:23:48.888709+0000","last_undegraded":"2024-04-25T10:23:48.888709+0000","last_fullsized":"2024-04-25T10:23:48.888709+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.210307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893903+0000","last_change":"2024-04-25T10:23:47.887973+0000","last_active":"2024-04-25T10:23:48.893903+0000","last_peered":"2024-04-25T10:23:48.893903+0000","last_clean":"2024-04-25T10:23:48.893903+0000","last_became_active":"2024-04-25T10:23:47.887673+0000","last_became_peered":"2024-04-25T10:23:47.887673+0000","last_unstale":"2024-04-25T10:23:48.893903+0000","last_undegraded":"2024-04-25T10:23:48.893903+0000","last_fullsized":"2024-04-25T10:23:48.893903+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.437799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888676+0000","last_change":"2024-04-25T10:23:46.864177+0000","last_active":"2024-04-25T10:23:48.888676+0000","last_peered":"2024-04-25T10:23:48.888676+0000","last_clean":"2024-04-25T10:23:48.888676+0000","last_became_active":"2024-04-25T10:23:46.863865+0000","last_became_peered":"2024-04-25T10:23:46.863865+0000","last_unstale":"2024-04-25T10:23:48.888676+0000","last_undegraded":"2024-04-25T10:23:48.888676+0000","last_fullsized":"2024-04-25T10:23:48.888676+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:48.005183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964370+0000","last_change":"2024-04-25T10:23:47.892558+0000","last_active":"2024-04-25T10:23:48.964370+0000","last_peered":"2024-04-25T10:23:48.964370+0000","last_clean":"2024-04-25T10:23:48.964370+0000","last_became_active":"2024-04-25T10:23:47.892292+0000","last_became_peered":"2024-04-25T10:23:47.892292+0000","last_unstale":"2024-04-25T10:23:48.964370+0000","last_undegraded":"2024-04-25T10:23:48.964370+0000","last_fullsized":"2024-04-25T10:23:48.964370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:15.431452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894203+0000","last_change":"2024-04-25T10:23:46.859300+0000","last_active":"2024-04-25T10:23:48.894203+0000","last_peered":"2024-04-25T10:23:48.894203+0000","last_clean":"2024-04-25T10:23:48.894203+0000","last_became_active":"2024-04-25T10:23:46.858976+0000","last_became_peered":"2024-04-25T10:23:46.858976+0000","last_unstale":"2024-04-25T10:23:48.894203+0000","last_undegraded":"2024-04-25T10:23:48.894203+0000","last_fullsized":"2024-04-25T10:23:48.894203+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:21.704050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888511+0000","last_change":"2024-04-25T10:23:47.888684+0000","last_active":"2024-04-25T10:23:48.888511+0000","last_peered":"2024-04-25T10:23:48.888511+0000","last_clean":"2024-04-25T10:23:48.888511+0000","last_became_active":"2024-04-25T10:23:47.888435+0000","last_became_peered":"2024-04-25T10:23:47.888435+0000","last_unstale":"2024-04-25T10:23:48.888511+0000","last_undegraded":"2024-04-25T10:23:48.888511+0000","last_fullsized":"2024-04-25T10:23:48.888511+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.000858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888357+0000","last_change":"2024-04-25T10:23:46.864168+0000","last_active":"2024-04-25T10:23:48.888357+0000","last_peered":"2024-04-25T10:23:48.888357+0000","last_clean":"2024-04-25T10:23:48.888357+0000","last_became_active":"2024-04-25T10:23:46.863926+0000","last_became_peered":"2024-04-25T10:23:46.863926+0000","last_unstale":"2024-04-25T10:23:48.888357+0000","last_undegraded":"2024-04-25T10:23:48.888357+0000","last_fullsized":"2024-04-25T10:23:48.888357+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:34.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888332+0000","last_change":"2024-04-25T10:23:47.892605+0000","last_active":"2024-04-25T10:23:48.888332+0000","last_peered":"2024-04-25T10:23:48.888332+0000","last_clean":"2024-04-25T10:23:48.888332+0000","last_became_active":"2024-04-25T10:23:47.887431+0000","last_became_peered":"2024-04-25T10:23:47.887431+0000","last_unstale":"2024-04-25T10:23:48.888332+0000","last_undegraded":"2024-04-25T10:23:48.888332+0000","last_fullsized":"2024-04-25T10:23:48.888332+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:02.612218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965514+0000","last_change":"2024-04-25T10:23:46.871892+0000","last_active":"2024-04-25T10:23:48.965514+0000","last_peered":"2024-04-25T10:23:48.965514+0000","last_clean":"2024-04-25T10:23:48.965514+0000","last_became_active":"2024-04-25T10:23:46.871516+0000","last_became_peered":"2024-04-25T10:23:46.871516+0000","last_unstale":"2024-04-25T10:23:48.965514+0000","last_undegraded":"2024-04-25T10:23:48.965514+0000","last_fullsized":"2024-04-25T10:23:48.965514+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:38.885940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952853+0000","last_change":"2024-04-25T10:23:47.883796+0000","last_active":"2024-04-25T10:23:48.952853+0000","last_peered":"2024-04-25T10:23:48.952853+0000","last_clean":"2024-04-25T10:23:48.952853+0000","last_became_active":"2024-04-25T10:23:47.883526+0000","last_became_peered":"2024-04-25T10:23:47.883526+0000","last_unstale":"2024-04-25T10:23:48.952853+0000","last_undegraded":"2024-04-25T10:23:48.952853+0000","last_fullsized":"2024-04-25T10:23:48.952853+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:28.082437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963985+0000","last_change":"2024-04-25T10:23:46.859215+0000","last_active":"2024-04-25T10:23:48.963985+0000","last_peered":"2024-04-25T10:23:48.963985+0000","last_clean":"2024-04-25T10:23:48.963985+0000","last_became_active":"2024-04-25T10:23:46.858249+0000","last_became_peered":"2024-04-25T10:23:46.858249+0000","last_unstale":"2024-04-25T10:23:48.963985+0000","last_undegraded":"2024-04-25T10:23:48.963985+0000","last_fullsized":"2024-04-25T10:23:48.963985+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:28.471690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964159+0000","last_change":"2024-04-25T10:23:47.894132+0000","last_active":"2024-04-25T10:23:48.964159+0000","last_peered":"2024-04-25T10:23:48.964159+0000","last_clean":"2024-04-25T10:23:48.964159+0000","last_became_active":"2024-04-25T10:23:47.893888+0000","last_became_peered":"2024-04-25T10:23:47.893888+0000","last_unstale":"2024-04-25T10:23:48.964159+0000","last_undegraded":"2024-04-25T10:23:48.964159+0000","last_fullsized":"2024-04-25T10:23:48.964159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.316322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953133+0000","last_change":"2024-04-25T10:23:46.866252+0000","last_active":"2024-04-25T10:23:48.953133+0000","last_peered":"2024-04-25T10:23:48.953133+0000","last_clean":"2024-04-25T10:23:48.953133+0000","last_became_active":"2024-04-25T10:23:46.865507+0000","last_became_peered":"2024-04-25T10:23:46.865507+0000","last_unstale":"2024-04-25T10:23:48.953133+0000","last_undegraded":"2024-04-25T10:23:48.953133+0000","last_fullsized":"2024-04-25T10:23:48.953133+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:36.714122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963829+0000","last_change":"2024-04-25T10:23:47.893355+0000","last_active":"2024-04-25T10:23:48.963829+0000","last_peered":"2024-04-25T10:23:48.963829+0000","last_clean":"2024-04-25T10:23:48.963829+0000","last_became_active":"2024-04-25T10:23:47.893089+0000","last_became_peered":"2024-04-25T10:23:47.893089+0000","last_unstale":"2024-04-25T10:23:48.963829+0000","last_undegraded":"2024-04-25T10:23:48.963829+0000","last_fullsized":"2024-04-25T10:23:48.963829+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:30.802161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"150'3","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.953045+0000","last_change":"2024-04-25T10:23:46.861040+0000","last_active":"2024-04-25T10:23:48.953045+0000","last_peered":"2024-04-25T10:23:48.953045+0000","last_clean":"2024-04-25T10:23:48.953045+0000","last_became_active":"2024-04-25T10:23:46.860641+0000","last_became_peered":"2024-04-25T10:23:46.860641+0000","last_unstale":"2024-04-25T10:23:48.953045+0000","last_undegraded":"2024-04-25T10:23:48.953045+0000","last_fullsized":"2024-04-25T10:23:48.953045+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:48.098600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894630+0000","last_change":"2024-04-25T10:23:47.874004+0000","last_active":"2024-04-25T10:23:48.894630+0000","last_peered":"2024-04-25T10:23:48.894630+0000","last_clean":"2024-04-25T10:23:48.894630+0000","last_became_active":"2024-04-25T10:23:47.873585+0000","last_became_peered":"2024-04-25T10:23:47.873585+0000","last_unstale":"2024-04-25T10:23:48.894630+0000","last_undegraded":"2024-04-25T10:23:48.894630+0000","last_fullsized":"2024-04-25T10:23:48.894630+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:12.933405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"150'6","reported_seq":20,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894112+0000","last_change":"2024-04-25T10:23:46.858960+0000","last_active":"2024-04-25T10:23:48.894112+0000","last_peered":"2024-04-25T10:23:48.894112+0000","last_clean":"2024-04-25T10:23:48.894112+0000","last_became_active":"2024-04-25T10:23:46.858706+0000","last_became_peered":"2024-04-25T10:23:46.858706+0000","last_unstale":"2024-04-25T10:23:48.894112+0000","last_undegraded":"2024-04-25T10:23:48.894112+0000","last_fullsized":"2024-04-25T10:23:48.894112+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:09.994450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963578+0000","last_change":"2024-04-25T10:23:47.880639+0000","last_active":"2024-04-25T10:23:48.963578+0000","last_peered":"2024-04-25T10:23:48.963578+0000","last_clean":"2024-04-25T10:23:48.963578+0000","last_became_active":"2024-04-25T10:23:47.880379+0000","last_became_peered":"2024-04-25T10:23:47.880379+0000","last_unstale":"2024-04-25T10:23:48.963578+0000","last_undegraded":"2024-04-25T10:23:48.963578+0000","last_fullsized":"2024-04-25T10:23:48.963578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:15.029459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888564+0000","last_change":"2024-04-25T10:23:46.862900+0000","last_active":"2024-04-25T10:23:48.888564+0000","last_peered":"2024-04-25T10:23:48.888564+0000","last_clean":"2024-04-25T10:23:48.888564+0000","last_became_active":"2024-04-25T10:23:46.862644+0000","last_became_peered":"2024-04-25T10:23:46.862644+0000","last_unstale":"2024-04-25T10:23:48.888564+0000","last_undegraded":"2024-04-25T10:23:48.888564+0000","last_fullsized":"2024-04-25T10:23:48.888564+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:35.415987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964588+0000","last_change":"2024-04-25T10:23:47.886608+0000","last_active":"2024-04-25T10:23:48.964588+0000","last_peered":"2024-04-25T10:23:48.964588+0000","last_clean":"2024-04-25T10:23:48.964588+0000","last_became_active":"2024-04-25T10:23:47.885880+0000","last_became_peered":"2024-04-25T10:23:47.885880+0000","last_unstale":"2024-04-25T10:23:48.964588+0000","last_undegraded":"2024-04-25T10:23:48.964588+0000","last_fullsized":"2024-04-25T10:23:48.964588+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:29.133237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963629+0000","last_change":"2024-04-25T10:23:46.862780+0000","last_active":"2024-04-25T10:23:48.963629+0000","last_peered":"2024-04-25T10:23:48.963629+0000","last_clean":"2024-04-25T10:23:48.963629+0000","last_became_active":"2024-04-25T10:23:46.861367+0000","last_became_peered":"2024-04-25T10:23:46.861367+0000","last_unstale":"2024-04-25T10:23:48.963629+0000","last_undegraded":"2024-04-25T10:23:48.963629+0000","last_fullsized":"2024-04-25T10:23:48.963629+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:22.344396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888477+0000","last_change":"2024-04-25T10:23:47.883952+0000","last_active":"2024-04-25T10:23:48.888477+0000","last_peered":"2024-04-25T10:23:48.888477+0000","last_clean":"2024-04-25T10:23:48.888477+0000","last_became_active":"2024-04-25T10:23:47.879298+0000","last_became_peered":"2024-04-25T10:23:47.879298+0000","last_unstale":"2024-04-25T10:23:48.888477+0000","last_undegraded":"2024-04-25T10:23:48.888477+0000","last_fullsized":"2024-04-25T10:23:48.888477+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:44.448189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888310+0000","last_change":"2024-04-25T10:23:46.867853+0000","last_active":"2024-04-25T10:23:48.888310+0000","last_peered":"2024-04-25T10:23:48.888310+0000","last_clean":"2024-04-25T10:23:48.888310+0000","last_became_active":"2024-04-25T10:23:46.867459+0000","last_became_peered":"2024-04-25T10:23:46.867459+0000","last_unstale":"2024-04-25T10:23:48.888310+0000","last_undegraded":"2024-04-25T10:23:48.888310+0000","last_fullsized":"2024-04-25T10:23:48.888310+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:55.737582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893012+0000","last_change":"2024-04-25T10:23:47.883973+0000","last_active":"2024-04-25T10:23:48.893012+0000","last_peered":"2024-04-25T10:23:48.893012+0000","last_clean":"2024-04-25T10:23:48.893012+0000","last_became_active":"2024-04-25T10:23:47.883704+0000","last_became_peered":"2024-04-25T10:23:47.883704+0000","last_unstale":"2024-04-25T10:23:48.893012+0000","last_undegraded":"2024-04-25T10:23:48.893012+0000","last_fullsized":"2024-04-25T10:23:48.893012+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:56.743990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"150'8","reported_seq":22,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893190+0000","last_change":"2024-04-25T10:23:46.858934+0000","last_active":"2024-04-25T10:23:48.893190+0000","last_peered":"2024-04-25T10:23:48.893190+0000","last_clean":"2024-04-25T10:23:48.893190+0000","last_became_active":"2024-04-25T10:23:46.858661+0000","last_became_peered":"2024-04-25T10:23:46.858661+0000","last_unstale":"2024-04-25T10:23:48.893190+0000","last_undegraded":"2024-04-25T10:23:48.893190+0000","last_fullsized":"2024-04-25T10:23:48.893190+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:48.244025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888957+0000","last_change":"2024-04-25T10:23:47.888992+0000","last_active":"2024-04-25T10:23:48.888957+0000","last_peered":"2024-04-25T10:23:48.888957+0000","last_clean":"2024-04-25T10:23:48.888957+0000","last_became_active":"2024-04-25T10:23:47.885530+0000","last_became_peered":"2024-04-25T10:23:47.885530+0000","last_unstale":"2024-04-25T10:23:48.888957+0000","last_undegraded":"2024-04-25T10:23:48.888957+0000","last_fullsized":"2024-04-25T10:23:48.888957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:25.121097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894005+0000","last_change":"2024-04-25T10:23:46.870270+0000","last_active":"2024-04-25T10:23:48.894005+0000","last_peered":"2024-04-25T10:23:48.894005+0000","last_clean":"2024-04-25T10:23:48.894005+0000","last_became_active":"2024-04-25T10:23:46.870037+0000","last_became_peered":"2024-04-25T10:23:46.870037+0000","last_unstale":"2024-04-25T10:23:48.894005+0000","last_undegraded":"2024-04-25T10:23:48.894005+0000","last_fullsized":"2024-04-25T10:23:48.894005+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:33.481936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964137+0000","last_change":"2024-04-25T10:23:47.876542+0000","last_active":"2024-04-25T10:23:48.964137+0000","last_peered":"2024-04-25T10:23:48.964137+0000","last_clean":"2024-04-25T10:23:48.964137+0000","last_became_active":"2024-04-25T10:23:47.876239+0000","last_became_peered":"2024-04-25T10:23:47.876239+0000","last_unstale":"2024-04-25T10:23:48.964137+0000","last_undegraded":"2024-04-25T10:23:48.964137+0000","last_fullsized":"2024-04-25T10:23:48.964137+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:40.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893819+0000","last_change":"2024-04-25T10:23:46.857710+0000","last_active":"2024-04-25T10:23:48.893819+0000","last_peered":"2024-04-25T10:23:48.893819+0000","last_clean":"2024-04-25T10:23:48.893819+0000","last_became_active":"2024-04-25T10:23:46.857424+0000","last_became_peered":"2024-04-25T10:23:46.857424+0000","last_unstale":"2024-04-25T10:23:48.893819+0000","last_undegraded":"2024-04-25T10:23:48.893819+0000","last_fullsized":"2024-04-25T10:23:48.893819+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:50.190366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888741+0000","last_change":"2024-04-25T10:23:47.886113+0000","last_active":"2024-04-25T10:23:48.888741+0000","last_peered":"2024-04-25T10:23:48.888741+0000","last_clean":"2024-04-25T10:23:48.888741+0000","last_became_active":"2024-04-25T10:23:47.885854+0000","last_became_peered":"2024-04-25T10:23:47.885854+0000","last_unstale":"2024-04-25T10:23:48.888741+0000","last_undegraded":"2024-04-25T10:23:48.888741+0000","last_fullsized":"2024-04-25T10:23:48.888741+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:12.822532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965866+0000","last_change":"2024-04-25T10:23:46.864350+0000","last_active":"2024-04-25T10:23:48.965866+0000","last_peered":"2024-04-25T10:23:48.965866+0000","last_clean":"2024-04-25T10:23:48.965866+0000","last_became_active":"2024-04-25T10:23:46.863870+0000","last_became_peered":"2024-04-25T10:23:46.863870+0000","last_unstale":"2024-04-25T10:23:48.965866+0000","last_undegraded":"2024-04-25T10:23:48.965866+0000","last_fullsized":"2024-04-25T10:23:48.965866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:27.404149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964436+0000","last_change":"2024-04-25T10:23:47.891062+0000","last_active":"2024-04-25T10:23:48.964436+0000","last_peered":"2024-04-25T10:23:48.964436+0000","last_clean":"2024-04-25T10:23:48.964436+0000","last_became_active":"2024-04-25T10:23:47.890772+0000","last_became_peered":"2024-04-25T10:23:47.890772+0000","last_unstale":"2024-04-25T10:23:48.964436+0000","last_undegraded":"2024-04-25T10:23:48.964436+0000","last_fullsized":"2024-04-25T10:23:48.964436+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:15.374928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965724+0000","last_change":"2024-04-25T10:23:46.869636+0000","last_active":"2024-04-25T10:23:48.965724+0000","last_peered":"2024-04-25T10:23:48.965724+0000","last_clean":"2024-04-25T10:23:48.965724+0000","last_became_active":"2024-04-25T10:23:46.869018+0000","last_became_peered":"2024-04-25T10:23:46.869018+0000","last_unstale":"2024-04-25T10:23:48.965724+0000","last_undegraded":"2024-04-25T10:23:48.965724+0000","last_fullsized":"2024-04-25T10:23:48.965724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:48.131868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893466+0000","last_change":"2024-04-25T10:23:47.873959+0000","last_active":"2024-04-25T10:23:48.893466+0000","last_peered":"2024-04-25T10:23:48.893466+0000","last_clean":"2024-04-25T10:23:48.893466+0000","last_became_active":"2024-04-25T10:23:47.873043+0000","last_became_peered":"2024-04-25T10:23:47.873043+0000","last_unstale":"2024-04-25T10:23:48.893466+0000","last_undegraded":"2024-04-25T10:23:48.893466+0000","last_fullsized":"2024-04-25T10:23:48.893466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:57.731770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888753+0000","last_change":"2024-04-25T10:23:46.867256+0000","last_active":"2024-04-25T10:23:48.888753+0000","last_peered":"2024-04-25T10:23:48.888753+0000","last_clean":"2024-04-25T10:23:48.888753+0000","last_became_active":"2024-04-25T10:23:46.866787+0000","last_became_peered":"2024-04-25T10:23:46.866787+0000","last_unstale":"2024-04-25T10:23:48.888753+0000","last_undegraded":"2024-04-25T10:23:48.888753+0000","last_fullsized":"2024-04-25T10:23:48.888753+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:10.260415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964653+0000","last_change":"2024-04-25T10:23:47.886189+0000","last_active":"2024-04-25T10:23:48.964653+0000","last_peered":"2024-04-25T10:23:48.964653+0000","last_clean":"2024-04-25T10:23:48.964653+0000","last_became_active":"2024-04-25T10:23:47.882053+0000","last_became_peered":"2024-04-25T10:23:47.882053+0000","last_unstale":"2024-04-25T10:23:48.964653+0000","last_undegraded":"2024-04-25T10:23:48.964653+0000","last_fullsized":"2024-04-25T10:23:48.964653+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:30.802161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965515+0000","last_change":"2024-04-25T10:23:46.869218+0000","last_active":"2024-04-25T10:23:48.965515+0000","last_peered":"2024-04-25T10:23:48.965515+0000","last_clean":"2024-04-25T10:23:48.965515+0000","last_became_active":"2024-04-25T10:23:46.868941+0000","last_became_peered":"2024-04-25T10:23:46.868941+0000","last_unstale":"2024-04-25T10:23:48.965515+0000","last_undegraded":"2024-04-25T10:23:48.965515+0000","last_fullsized":"2024-04-25T10:23:48.965515+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:43.899274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965287+0000","last_change":"2024-04-25T10:23:47.899199+0000","last_active":"2024-04-25T10:23:48.965287+0000","last_peered":"2024-04-25T10:23:48.965287+0000","last_clean":"2024-04-25T10:23:48.965287+0000","last_became_active":"2024-04-25T10:23:47.898887+0000","last_became_peered":"2024-04-25T10:23:47.898887+0000","last_unstale":"2024-04-25T10:23:48.965287+0000","last_undegraded":"2024-04-25T10:23:48.965287+0000","last_fullsized":"2024-04-25T10:23:48.965287+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:58.656369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888335+0000","last_change":"2024-04-25T10:23:46.858444+0000","last_active":"2024-04-25T10:23:48.888335+0000","last_peered":"2024-04-25T10:23:48.888335+0000","last_clean":"2024-04-25T10:23:48.888335+0000","last_became_active":"2024-04-25T10:23:46.858164+0000","last_became_peered":"2024-04-25T10:23:46.858164+0000","last_unstale":"2024-04-25T10:23:48.888335+0000","last_undegraded":"2024-04-25T10:23:48.888335+0000","last_fullsized":"2024-04-25T10:23:48.888335+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:38.885940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952997+0000","last_change":"2024-04-25T10:23:47.893576+0000","last_active":"2024-04-25T10:23:48.952997+0000","last_peered":"2024-04-25T10:23:48.952997+0000","last_clean":"2024-04-25T10:23:48.952997+0000","last_became_active":"2024-04-25T10:23:47.893296+0000","last_became_peered":"2024-04-25T10:23:47.893296+0000","last_unstale":"2024-04-25T10:23:48.952997+0000","last_undegraded":"2024-04-25T10:23:48.952997+0000","last_fullsized":"2024-04-25T10:23:48.952997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:10.048909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965339+0000","last_change":"2024-04-25T10:23:46.862330+0000","last_active":"2024-04-25T10:23:48.965339+0000","last_peered":"2024-04-25T10:23:48.965339+0000","last_clean":"2024-04-25T10:23:48.965339+0000","last_became_active":"2024-04-25T10:23:46.862027+0000","last_became_peered":"2024-04-25T10:23:46.862027+0000","last_unstale":"2024-04-25T10:23:48.965339+0000","last_undegraded":"2024-04-25T10:23:48.965339+0000","last_fullsized":"2024-04-25T10:23:48.965339+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:46.309914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964292+0000","last_change":"2024-04-25T10:23:47.895761+0000","last_active":"2024-04-25T10:23:48.964292+0000","last_peered":"2024-04-25T10:23:48.964292+0000","last_clean":"2024-04-25T10:23:48.964292+0000","last_became_active":"2024-04-25T10:23:47.895475+0000","last_became_peered":"2024-04-25T10:23:47.895475+0000","last_unstale":"2024-04-25T10:23:48.964292+0000","last_undegraded":"2024-04-25T10:23:48.964292+0000","last_fullsized":"2024-04-25T10:23:48.964292+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:49.138276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"150'2","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964188+0000","last_change":"2024-04-25T10:23:46.867975+0000","last_active":"2024-04-25T10:23:48.964188+0000","last_peered":"2024-04-25T10:23:48.964188+0000","last_clean":"2024-04-25T10:23:48.964188+0000","last_became_active":"2024-04-25T10:23:46.867653+0000","last_became_peered":"2024-04-25T10:23:46.867653+0000","last_unstale":"2024-04-25T10:23:48.964188+0000","last_undegraded":"2024-04-25T10:23:48.964188+0000","last_fullsized":"2024-04-25T10:23:48.964188+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:54.775658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.964505+0000","last_change":"2024-04-25T10:23:47.888040+0000","last_active":"2024-04-25T10:23:48.964505+0000","last_peered":"2024-04-25T10:23:48.964505+0000","last_clean":"2024-04-25T10:23:48.964505+0000","last_became_active":"2024-04-25T10:23:47.887784+0000","last_became_peered":"2024-04-25T10:23:47.887784+0000","last_unstale":"2024-04-25T10:23:48.964505+0000","last_undegraded":"2024-04-25T10:23:48.964505+0000","last_fullsized":"2024-04-25T10:23:48.964505+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:24.996466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"150'1","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893878+0000","last_change":"2024-04-25T10:23:46.857071+0000","last_active":"2024-04-25T10:23:48.893878+0000","last_peered":"2024-04-25T10:23:48.893878+0000","last_clean":"2024-04-25T10:23:48.893878+0000","last_became_active":"2024-04-25T10:23:46.856792+0000","last_became_peered":"2024-04-25T10:23:46.856792+0000","last_unstale":"2024-04-25T10:23:48.893878+0000","last_undegraded":"2024-04-25T10:23:48.893878+0000","last_fullsized":"2024-04-25T10:23:48.893878+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:40.182535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.952963+0000","last_change":"2024-04-25T10:23:47.891766+0000","last_active":"2024-04-25T10:23:48.952963+0000","last_peered":"2024-04-25T10:23:48.952963+0000","last_clean":"2024-04-25T10:23:48.952963+0000","last_became_active":"2024-04-25T10:23:47.891508+0000","last_became_peered":"2024-04-25T10:23:47.891508+0000","last_unstale":"2024-04-25T10:23:48.952963+0000","last_undegraded":"2024-04-25T10:23:48.952963+0000","last_fullsized":"2024-04-25T10:23:48.952963+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.853909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.966219+0000","last_change":"2024-04-25T10:23:46.870852+0000","last_active":"2024-04-25T10:23:48.966219+0000","last_peered":"2024-04-25T10:23:48.966219+0000","last_clean":"2024-04-25T10:23:48.966219+0000","last_became_active":"2024-04-25T10:23:46.870574+0000","last_became_peered":"2024-04-25T10:23:46.870574+0000","last_unstale":"2024-04-25T10:23:48.966219+0000","last_undegraded":"2024-04-25T10:23:48.966219+0000","last_fullsized":"2024-04-25T10:23:48.966219+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.210307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963887+0000","last_change":"2024-04-25T10:23:47.887091+0000","last_active":"2024-04-25T10:23:48.963887+0000","last_peered":"2024-04-25T10:23:48.963887+0000","last_clean":"2024-04-25T10:23:48.963887+0000","last_became_active":"2024-04-25T10:23:47.886663+0000","last_became_peered":"2024-04-25T10:23:47.886663+0000","last_unstale":"2024-04-25T10:23:48.963887+0000","last_undegraded":"2024-04-25T10:23:48.963887+0000","last_fullsized":"2024-04-25T10:23:48.963887+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:44.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963761+0000","last_change":"2024-04-25T10:23:46.853251+0000","last_active":"2024-04-25T10:23:48.963761+0000","last_peered":"2024-04-25T10:23:48.963761+0000","last_clean":"2024-04-25T10:23:48.963761+0000","last_became_active":"2024-04-25T10:23:46.852966+0000","last_became_peered":"2024-04-25T10:23:46.852966+0000","last_unstale":"2024-04-25T10:23:48.963761+0000","last_undegraded":"2024-04-25T10:23:48.963761+0000","last_fullsized":"2024-04-25T10:23:48.963761+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.492112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963617+0000","last_change":"2024-04-25T10:23:47.879029+0000","last_active":"2024-04-25T10:23:48.963617+0000","last_peered":"2024-04-25T10:23:48.963617+0000","last_clean":"2024-04-25T10:23:48.963617+0000","last_became_active":"2024-04-25T10:23:47.878769+0000","last_became_peered":"2024-04-25T10:23:47.878769+0000","last_unstale":"2024-04-25T10:23:48.963617+0000","last_undegraded":"2024-04-25T10:23:48.963617+0000","last_fullsized":"2024-04-25T10:23:48.963617+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:29.133237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893545+0000","last_change":"2024-04-25T10:23:46.856632+0000","last_active":"2024-04-25T10:23:48.893545+0000","last_peered":"2024-04-25T10:23:48.893545+0000","last_clean":"2024-04-25T10:23:48.893545+0000","last_became_active":"2024-04-25T10:23:46.856153+0000","last_became_peered":"2024-04-25T10:23:46.856153+0000","last_unstale":"2024-04-25T10:23:48.893545+0000","last_undegraded":"2024-04-25T10:23:48.893545+0000","last_fullsized":"2024-04-25T10:23:48.893545+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:24.114690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893488+0000","last_change":"2024-04-25T10:23:47.891149+0000","last_active":"2024-04-25T10:23:48.893488+0000","last_peered":"2024-04-25T10:23:48.893488+0000","last_clean":"2024-04-25T10:23:48.893488+0000","last_became_active":"2024-04-25T10:23:47.890835+0000","last_became_peered":"2024-04-25T10:23:47.890835+0000","last_unstale":"2024-04-25T10:23:48.893488+0000","last_undegraded":"2024-04-25T10:23:48.893488+0000","last_fullsized":"2024-04-25T10:23:48.893488+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:55.782065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.888340+0000","last_change":"2024-04-25T10:23:46.860202+0000","last_active":"2024-04-25T10:23:48.888340+0000","last_peered":"2024-04-25T10:23:48.888340+0000","last_clean":"2024-04-25T10:23:48.888340+0000","last_became_active":"2024-04-25T10:23:46.859950+0000","last_became_peered":"2024-04-25T10:23:46.859950+0000","last_unstale":"2024-04-25T10:23:48.888340+0000","last_undegraded":"2024-04-25T10:23:48.888340+0000","last_fullsized":"2024-04-25T10:23:48.888340+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:10.201593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963851+0000","last_change":"2024-04-25T10:23:47.890965+0000","last_active":"2024-04-25T10:23:48.963851+0000","last_peered":"2024-04-25T10:23:48.963851+0000","last_clean":"2024-04-25T10:23:48.963851+0000","last_became_active":"2024-04-25T10:23:47.889986+0000","last_became_peered":"2024-04-25T10:23:47.889986+0000","last_unstale":"2024-04-25T10:23:48.963851+0000","last_undegraded":"2024-04-25T10:23:48.963851+0000","last_fullsized":"2024-04-25T10:23:48.963851+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:49.541419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965784+0000","last_change":"2024-04-25T10:23:46.869313+0000","last_active":"2024-04-25T10:23:48.965784+0000","last_peered":"2024-04-25T10:23:48.965784+0000","last_clean":"2024-04-25T10:23:48.965784+0000","last_became_active":"2024-04-25T10:23:46.868519+0000","last_became_peered":"2024-04-25T10:23:46.868519+0000","last_unstale":"2024-04-25T10:23:48.965784+0000","last_undegraded":"2024-04-25T10:23:48.965784+0000","last_fullsized":"2024-04-25T10:23:48.965784+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:56.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893241+0000","last_change":"2024-04-25T10:23:47.882533+0000","last_active":"2024-04-25T10:23:48.893241+0000","last_peered":"2024-04-25T10:23:48.893241+0000","last_clean":"2024-04-25T10:23:48.893241+0000","last_became_active":"2024-04-25T10:23:47.881581+0000","last_became_peered":"2024-04-25T10:23:47.881581+0000","last_unstale":"2024-04-25T10:23:48.893241+0000","last_undegraded":"2024-04-25T10:23:48.893241+0000","last_fullsized":"2024-04-25T10:23:48.893241+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:39.892347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.893053+0000","last_change":"2024-04-25T10:23:46.862519+0000","last_active":"2024-04-25T10:23:48.893053+0000","last_peered":"2024-04-25T10:23:48.893053+0000","last_clean":"2024-04-25T10:23:48.893053+0000","last_became_active":"2024-04-25T10:23:46.862239+0000","last_became_peered":"2024-04-25T10:23:46.862239+0000","last_unstale":"2024-04-25T10:23:48.893053+0000","last_undegraded":"2024-04-25T10:23:48.893053+0000","last_fullsized":"2024-04-25T10:23:48.893053+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:11.816124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.963984+0000","last_change":"2024-04-25T10:23:47.896019+0000","last_active":"2024-04-25T10:23:48.963984+0000","last_peered":"2024-04-25T10:23:48.963984+0000","last_clean":"2024-04-25T10:23:48.963984+0000","last_became_active":"2024-04-25T10:23:47.895781+0000","last_became_peered":"2024-04-25T10:23:47.895781+0000","last_unstale":"2024-04-25T10:23:48.963984+0000","last_undegraded":"2024-04-25T10:23:48.963984+0000","last_fullsized":"2024-04-25T10:23:48.963984+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:46.839773+0000","last_clean_scrub_stamp":"2024-04-25T10:23:46.839773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:28.410557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.894564+0000","last_change":"2024-04-25T10:23:46.865259+0000","last_active":"2024-04-25T10:23:48.894564+0000","last_peered":"2024-04-25T10:23:48.894564+0000","last_clean":"2024-04-25T10:23:48.894564+0000","last_became_active":"2024-04-25T10:23:46.864959+0000","last_became_peered":"2024-04-25T10:23:46.864959+0000","last_unstale":"2024-04-25T10:23:48.894564+0000","last_undegraded":"2024-04-25T10:23:48.894564+0000","last_fullsized":"2024-04-25T10:23:48.894564+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:23:45.833365+0000","last_clean_scrub_stamp":"2024-04-25T10:23:45.833365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:28.471690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":406,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-04-25T10:23:48.965455+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:23:48.965455+0000","last_peered":"2024-04-25T10:23:48.965455+0000","last_clean":"2024-04-25T10:23:48.965455+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:23:48.965455+0000","last_undegraded":"2024-04-25T10:23:48.965455+0000","last_fullsized":"2024-04-25T10:23:48.965455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60012,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311828,"statfs":{"total":96636764160,"available":96575311872,"internally_reserved":0,"allocated":2514944,"data_stored":1424497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64908,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306932,"statfs":{"total":96636764160,"available":96570298368,"internally_reserved":0,"allocated":1900544,"data_stored":831797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51088,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320752,"statfs":{"total":96636764160,"available":96584450048,"internally_reserved":0,"allocated":1904640,"data_stored":832379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":1912832,"data_stored":833457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574992,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55248,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316592,"statfs":{"total":96636764160,"available":96580190208,"internally_reserved":0,"allocated":1900544,"data_stored":831809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55252,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316588,"statfs":{"total":96636764160,"available":96580186112,"internally_reserved":0,"allocated":1900544,"data_stored":831865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306864,"statfs":{"total":96636764160,"available":96570228736,"internally_reserved":0,"allocated":1900544,"data_stored":831809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55828,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316012,"statfs":{"total":96636764160,"available":96579596288,"internally_reserved":0,"allocated":2490368,"data_stored":1422143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:23:52.338 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:23:52.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:23:52.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99f200640 1 Processor -- start 2024-04-25T10:23:52.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99f200640 1 -- start start 2024-04-25T10:23:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 0x7fe9a00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a00cebc0 con 0x7fe9a00cfda0 2024-04-25T10:23:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a00ceea0 con 0x7fe9a015be10 2024-04-25T10:23:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cca70 con 0x7fe9a00b1680 2024-04-25T10:23:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99de00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe9a00cfda0 0x7fe9a00d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54168/0 (socket says 172.21.15.87:54168) 2024-04-25T10:23:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99de00640 1 -- 172.21.15.87:0/563696830 learned_addr learned my addr 172.21.15.87:0/563696830 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99e800640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 0x7fe9a00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489308217 0 0) 0x7fe9a00cebc0 con 0x7fe9a00cfda0 2024-04-25T10:23:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe984003530 con 0x7fe9a00cfda0 2024-04-25T10:23:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540728626 0 0) 0x7fe9a00ceea0 con 0x7fe9a015be10 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a00cebc0 con 0x7fe9a015be10 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 281418513 0 0) 0x7fe984003530 con 0x7fe9a00cfda0 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 msgr2=0x7fe9a00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 0x7fe9a00b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 >> v1:172.21.15.169:6789/0 conn(0x7fe9a015be10 legacy=0x7fe9a00ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00cc700 con 0x7fe9a00cfda0 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fe990001680 con 0x7fe9a00cfda0 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 >> v1:172.21.15.87:6789/0 conn(0x7fe9a00cfda0 legacy=0x7fe9a00d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 0x7fe984003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe984004130 0x7fe984004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004d50 0x7fe9840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cca70 con 0x7fe984004130 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98400bb10 con 0x7fe984004d50 2024-04-25T10:23:52.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98400bd10 con 0x7fe9a00b1680 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe995400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe984004130 0x7fe984004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99de00640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 0x7fe984003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.524+0000 7fe99e800640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004d50 0x7fe9840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe995400640 1 -- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 msgr2=0x7fe984003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe995400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00b1680 0x7fe984003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe995400640 1 -- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004d50 msgr2=0x7fe9840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe995400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004d50 0x7fe9840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe995400640 1 -- 172.21.15.87:0/563696830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984003530 con 0x7fe984004130 2024-04-25T10:23:52.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe995400640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe984004130 0x7fe984004810 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fe9880018d0 tx=0x7fe98801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dccc5546664962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988001520 con 0x7fe984004130 2024-04-25T10:23:52.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880052f0 con 0x7fe984004130 2024-04-25T10:23:52.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99d400640 1 -- 172.21.15.87:0/563696830 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880a8890 con 0x7fe984004130 2024-04-25T10:23:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99f200640 1 -- 172.21.15.87:0/563696830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe984004130 msgr2=0x7fe984004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99f200640 1 --2- 172.21.15.87:0/563696830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe984004130 0x7fe984004810 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fe9880018d0 tx=0x7fe98801ec30 comp rx=0 tx=0).stop 2024-04-25T10:23:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99e800640 1 -- 172.21.15.87:0/563696830 reap_dead start 2024-04-25T10:23:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99f200640 1 -- 172.21.15.87:0/563696830 shutdown_connections 2024-04-25T10:23:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.528+0000 7fe99f200640 1 -- 172.21.15.87:0/563696830 >> 172.21.15.87:0/563696830 conn(0x7fe9a0072820 msgr2=0x7fe9a009a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- 172.21.15.87:0/563696830 shutdown_connections 2024-04-25T10:23:52.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- 172.21.15.87:0/563696830 wait complete. 2024-04-25T10:23:52.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 Processor -- start 2024-04-25T10:23:52.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- start start 2024-04-25T10:23:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004130 0x7fe9a00b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0630 0x7fe9a00c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00c9cc0 0x7fe9a0154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cf870 con 0x7fe9a00b0630 2024-04-25T10:23:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cc3b0 con 0x7fe984004130 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cf9f0 con 0x7fe9a00c9cc0 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0630 0x7fe9a00c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0630 0x7fe9a00c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50206/0 (socket says 172.21.15.87:50206) 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 -- 172.21.15.87:0/732846469 learned_addr learned my addr 172.21.15.87:0/732846469 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99e800640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00c9cc0 0x7fe9a0154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99de00640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004130 0x7fe9a00b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 -- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00c9cc0 msgr2=0x7fe9a0154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00c9cc0 0x7fe9a0154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 -- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004130 msgr2=0x7fe9a00b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004130 0x7fe9a00b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 -- 172.21.15.87:0/732846469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00af050 con 0x7fe9a00b0630 2024-04-25T10:23:52.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe995400640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0630 0x7fe9a00c9780 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fe98801ea60 tx=0x7fe984005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe996a00640 1 -- 172.21.15.87:0/732846469 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880a8a00 con 0x7fe9a00b0630 2024-04-25T10:23:52.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe996a00640 1 -- 172.21.15.87:0/732846469 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880be070 con 0x7fe9a00b0630 2024-04-25T10:23:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe996a00640 1 -- 172.21.15.87:0/732846469 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880a8dc0 con 0x7fe9a00b0630 2024-04-25T10:23:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a00af330 con 0x7fe9a00b0630 2024-04-25T10:23:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.532+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a00af8f0 con 0x7fe9a00b0630 2024-04-25T10:23:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.536+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a00d2690 con 0x7fe9a00b0630 2024-04-25T10:23:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.536+0000 7fe996a00640 1 -- 172.21.15.87:0/732846469 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe988005190 con 0x7fe9a00b0630 2024-04-25T10:23:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.536+0000 7fe996a00640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe97c066f60 0x7fe97c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.536+0000 7fe996a00640 1 -- 172.21.15.87:0/732846469 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe988126360 con 0x7fe9a00b0630 2024-04-25T10:23:52.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.536+0000 7fe99de00640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe97c066f60 0x7fe97c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.536+0000 7fe99de00640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe97c066f60 0x7fe97c069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c10 tx=0x7fe990009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.540+0000 7fe996a00640 1 -- 172.21.15.87:0/732846469 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9880ca030 con 0x7fe9a00b0630 2024-04-25T10:23:52.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.744+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe9a00ca0d0 con 0x7fe9a00b0630 2024-04-25T10:23:52.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.748+0000 7fe996a00640 1 -- 172.21.15.87:0/732846469 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe988018940 con 0x7fe9a00b0630 2024-04-25T10:23:52.749 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:23:52.749 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:23:52.750 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:23:52.750 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:23:52.750 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:23:52.752 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:23:52.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe97c066f60 msgr2=0x7fe97c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe97c066f60 0x7fe97c069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c10 tx=0x7fe990009040 comp rx=0 tx=0).stop 2024-04-25T10:23:52.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0630 msgr2=0x7fe9a00c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0630 0x7fe9a00c9780 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fe98801ea60 tx=0x7fe984005840 comp rx=0 tx=0).stop 2024-04-25T10:23:52.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 shutdown_connections 2024-04-25T10:23:52.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe97c066f60 0x7fe97c069420 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a00c9cc0 0x7fe9a0154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0630 0x7fe9a00c9780 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 --2- 172.21.15.87:0/732846469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984004130 0x7fe9a00b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 >> 172.21.15.87:0/732846469 conn(0x7fe9a0072820 msgr2=0x7fe9a015b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:52.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 shutdown_connections 2024-04-25T10:23:52.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.752+0000 7fe99f200640 1 -- 172.21.15.87:0/732846469 wait complete. 2024-04-25T10:23:52.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:23:52.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.904+0000 7fdc68400640 1 Processor -- start 2024-04-25T10:23:52.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.904+0000 7fdc68400640 1 -- start start 2024-04-25T10:23:52.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc68400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdc6015bc40 0x7fdc6006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc68400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdc600cf930 0x7fdc600cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc68400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc600b2770 con 0x7fdc600cda10 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc68400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6006c670 con 0x7fdc600cf930 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc68400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6006c950 con 0x7fdc6015bc40 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc66400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdc600cf930 0x7fdc600cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc65000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fdc600cda10 0x7fdc600b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54184/0 (socket says 172.21.15.87:54184) 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc65000640 1 -- 172.21.15.87:0/1952149275 learned_addr learned my addr 172.21.15.87:0/1952149275 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc65a00640 1 --2- 172.21.15.87:0/1952149275 >> v2:172.21.15.169:3300/0 conn(0x7fdc6015bc40 0x7fdc6006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc5fe00640 1 -- 172.21.15.87:0/1952149275 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240659571 0 0) 0x7fdc600b2770 con 0x7fdc600cda10 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc5fe00640 1 -- 172.21.15.87:0/1952149275 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc48003530 con 0x7fdc600cda10 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc66400640 1 -- 172.21.15.87:0/1952149275 >> v2:172.21.15.169:3300/0 conn(0x7fdc6015bc40 msgr2=0x7fdc6006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc66400640 1 --2- 172.21.15.87:0/1952149275 >> v2:172.21.15.169:3300/0 conn(0x7fdc6015bc40 0x7fdc6006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc66400640 1 -- 172.21.15.87:0/1952149275 >> v1:172.21.15.87:6789/0 conn(0x7fdc600cda10 legacy=0x7fdc600b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.908+0000 7fdc66400640 1 -- 172.21.15.87:0/1952149275 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6006cc30 con 0x7fdc600cf930 2024-04-25T10:23:52.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.912+0000 7fdc66400640 1 --2- 172.21.15.87:0/1952149275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc600cfdc0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdc600d0800 tx=0x7fdc5801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8997cd4e742e5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc5fe00640 1 -- 172.21.15.87:0/1952149275 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc580b0070 con 0x7fdc600cf930 2024-04-25T10:23:52.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc5fe00640 1 -- 172.21.15.87:0/1952149275 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc580ab600 con 0x7fdc600cf930 2024-04-25T10:23:52.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc5fe00640 1 -- 172.21.15.87:0/1952149275 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc580ab9d0 con 0x7fdc600cf930 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 -- 172.21.15.87:0/1952149275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 msgr2=0x7fdc600cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 --2- 172.21.15.87:0/1952149275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc600cfdc0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdc600d0800 tx=0x7fdc5801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 -- 172.21.15.87:0/1952149275 shutdown_connections 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 --2- 172.21.15.87:0/1952149275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc600cfdc0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 --2- 172.21.15.87:0/1952149275 >> v2:172.21.15.169:3300/0 conn(0x7fdc6015bc40 0x7fdc6006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 -- 172.21.15.87:0/1952149275 >> 172.21.15.87:0/1952149275 conn(0x7fdc60066b70 msgr2=0x7fdc60066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 -- 172.21.15.87:0/1952149275 shutdown_connections 2024-04-25T10:23:52.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 -- 172.21.15.87:0/1952149275 wait complete. 2024-04-25T10:23:52.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 Processor -- start 2024-04-25T10:23:52.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.916+0000 7fdc68400640 1 -- start start 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc600cda10 0x7fdc6015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc60152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6015bc40 0x7fdc600af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600b3850 con 0x7fdc600cf930 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600b36d0 con 0x7fdc600cda10 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600b39d0 con 0x7fdc6015bc40 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc60152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc60152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50222/0 (socket says 172.21.15.87:50222) 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 -- 172.21.15.87:0/2477273145 learned_addr learned my addr 172.21.15.87:0/2477273145 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65a00640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc600cda10 0x7fdc6015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc66400640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6015bc40 0x7fdc600af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 -- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6015bc40 msgr2=0x7fdc600af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6015bc40 0x7fdc600af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 -- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc600cda10 msgr2=0x7fdc6015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc600cda10 0x7fdc6015d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 -- 172.21.15.87:0/2477273145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc600afe10 con 0x7fdc600cf930 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc65000640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc60152f20 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00c3f0 tx=0x7fdc4c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc5d600640 1 -- 172.21.15.87:0/2477273145 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c017070 con 0x7fdc600cf930 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc5d600640 1 -- 172.21.15.87:0/2477273145 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c005cf0 con 0x7fdc600cf930 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc600b00f0 con 0x7fdc600cf930 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc5d600640 1 -- 172.21.15.87:0/2477273145 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0072f0 con 0x7fdc600cf930 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc60071380 con 0x7fdc600cf930 2024-04-25T10:23:52.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc60071190 con 0x7fdc600cf930 2024-04-25T10:23:52.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.920+0000 7fdc5d600640 1 -- 172.21.15.87:0/2477273145 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc4c01a070 con 0x7fdc600cf930 2024-04-25T10:23:52.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.924+0000 7fdc5d600640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc40066f60 0x7fdc40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:52.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.924+0000 7fdc5d600640 1 -- 172.21.15.87:0/2477273145 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc4c0843d0 con 0x7fdc600cf930 2024-04-25T10:23:52.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.924+0000 7fdc65a00640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc40066f60 0x7fdc40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:52.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.924+0000 7fdc65a00640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc40066f60 0x7fdc40069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ae930 tx=0x7fdc54009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:52.928+0000 7fdc5d600640 1 -- 172.21.15.87:0/2477273145 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc4c054d80 con 0x7fdc600cf930 2024-04-25T10:23:53.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc6004ef30 con 0x7fdc600cf930 2024-04-25T10:23:53.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc5d600640 1 -- 172.21.15.87:0/2477273145 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4955 (secure 0 0 0) 0x7fdc4c05b0d0 con 0x7fdc600cf930 2024-04-25T10:23:53.112 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:53.112 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":111,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4139990719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4139990719},{"type":"v1","addr":"172.21.15.169:6834","nonce":4139990719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3962905873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3962905873},{"type":"v1","addr":"172.21.15.169:6835","nonce":3962905873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3473546915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3473546915},{"type":"v1","addr":"172.21.15.87:6836","nonce":3473546915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:23:47.224714+0000","modified":"2024-04-25T10:23:50.259650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10044":{"gid":10044,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/564498676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":564498676},{"type":"v1","addr":"172.21.15.87:6837","nonce":564498676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:23:53.112 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 111 2024-04-25T10:23:53.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc40066f60 msgr2=0x7fdc40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc40066f60 0x7fdc40069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ae930 tx=0x7fdc54009830 comp rx=0 tx=0).stop 2024-04-25T10:23:53.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 msgr2=0x7fdc60152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc60152f20 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00c3f0 tx=0x7fdc4c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 shutdown_connections 2024-04-25T10:23:53.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdc40066f60 0x7fdc40069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6015bc40 0x7fdc600af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc600cf930 0x7fdc60152f20 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 --2- 172.21.15.87:0/2477273145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc600cda10 0x7fdc6015d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.112+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 >> 172.21.15.87:0/2477273145 conn(0x7fdc60066b70 msgr2=0x7fdc6015af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.116+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 shutdown_connections 2024-04-25T10:23:53.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.116+0000 7fdc68400640 1 -- 172.21.15.87:0/2477273145 wait complete. 2024-04-25T10:23:53.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:23:47.224714+0000', 'modified': '2024-04-25T10:23:50.259650+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10044}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10044': {'gid': 10044, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/564498676', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 564498676}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 564498676}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:23:53.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:23:53.537 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:23:53.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e2a00640 1 Processor -- start 2024-04-25T10:23:53.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e2a00640 1 -- start start 2024-04-25T10:23:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9dc06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e2a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc0cec70 con 0x7fc9dc06af30 2024-04-25T10:23:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc0b1da0 con 0x7fc9dc0b0b80 2024-04-25T10:23:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0b3270 con 0x7fc9dc0aed20 2024-04-25T10:23:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc9dc06af30 0x7fc9dc0b2b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54190/0 (socket says 172.21.15.87:54190) 2024-04-25T10:23:53.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1327040180 learned_addr learned my addr 172.21.15.87:0/1327040180 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9dc06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127791830 0 0) 0x7fc9dc0cec70 con 0x7fc9dc06af30 2024-04-25T10:23:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9c0003530 con 0x7fc9dc06af30 2024-04-25T10:23:53.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185044426 0 0) 0x7fc9dc0b1da0 con 0x7fc9dc0b0b80 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9dc0cec70 con 0x7fc9dc0b0b80 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 373121208 0 0) 0x7fc9c0003530 con 0x7fc9dc06af30 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 msgr2=0x7fc9dc06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9dc06b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 >> v1:172.21.15.169:6789/0 conn(0x7fc9dc0b0b80 legacy=0x7fc9dc0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0b2080 con 0x7fc9dc06af30 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc9cc001680 con 0x7fc9dc06af30 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9dc06b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 >> v1:172.21.15.87:6789/0 conn(0x7fc9dc06af30 legacy=0x7fc9dc0b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9c0003b80 0x7fc9c0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0004d50 0x7fc9c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0b3270 con 0x7fc9c0003b80 2024-04-25T10:23:53.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c000bae0 con 0x7fc9c0004d50 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c000bce0 con 0x7fc9dc0aed20 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9c0003b80 0x7fc9c0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0004d50 0x7fc9c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9db400640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 msgr2=0x7fc9c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0004d50 msgr2=0x7fc9c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0004d50 0x7fc9c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1327040180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0003530 con 0x7fc9c0003b80 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9db400640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0aed20 0x7fc9c0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:53.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.676+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0004d50 0x7fc9c00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9c0003b80 0x7fc9c0003f80 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c1c0 tx=0x7fc9cc01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e89dda0a04e78f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc007cf0 con 0x7fc9c0003b80 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc0a9020 con 0x7fc9c0003b80 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9daa00640 1 -- 172.21.15.87:0/1327040180 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc005b00 con 0x7fc9c0003b80 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9e2a00640 1 -- 172.21.15.87:0/1327040180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9c0003b80 msgr2=0x7fc9c0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9e2a00640 1 --2- 172.21.15.87:0/1327040180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9c0003b80 0x7fc9c0003f80 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c1c0 tx=0x7fc9cc01cc20 comp rx=0 tx=0).stop 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9e0a00640 1 -- 172.21.15.87:0/1327040180 reap_dead start 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9e2a00640 1 -- 172.21.15.87:0/1327040180 shutdown_connections 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9e2a00640 1 -- 172.21.15.87:0/1327040180 >> 172.21.15.87:0/1327040180 conn(0x7fc9dc0d1900 msgr2=0x7fc9dc0d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9e2a00640 1 -- 172.21.15.87:0/1327040180 shutdown_connections 2024-04-25T10:23:53.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.680+0000 7fc9e2a00640 1 -- 172.21.15.87:0/1327040180 wait complete. 2024-04-25T10:23:53.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 Processor -- start 2024-04-25T10:23:53.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 -- start start 2024-04-25T10:23:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0003b80 0x7fc9dc15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9dc0b4520 0x7fc9dc160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0b49a0 0x7fc9dc160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc06beb0 con 0x7fc9dc0b4520 2024-04-25T10:23:53.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc06bd30 con 0x7fc9c0003b80 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc169b10 con 0x7fc9dc0b49a0 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9dc0b4520 0x7fc9dc160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9dc0b4520 0x7fc9dc160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50236/0 (socket says 172.21.15.87:50236) 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 -- 172.21.15.87:0/3141760732 learned_addr learned my addr 172.21.15.87:0/3141760732 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0003b80 0x7fc9dc15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0b49a0 0x7fc9dc160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 -- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0b49a0 msgr2=0x7fc9dc160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0b49a0 0x7fc9dc160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 -- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0003b80 msgr2=0x7fc9dc15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0003b80 0x7fc9dc15d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 -- 172.21.15.87:0/3141760732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc15fc40 con 0x7fc9dc0b4520 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0b49a0 0x7fc9dc160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:53.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9db400640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9dc0b4520 0x7fc9dc160a00 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800e7e0 tx=0x7fc9c800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9d3e00640 1 -- 172.21.15.87:0/3141760732 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80050d0 con 0x7fc9dc0b4520 2024-04-25T10:23:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9d3e00640 1 -- 172.21.15.87:0/3141760732 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8005270 con 0x7fc9dc0b4520 2024-04-25T10:23:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9d3e00640 1 -- 172.21.15.87:0/3141760732 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8007020 con 0x7fc9dc0b4520 2024-04-25T10:23:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc15ff20 con 0x7fc9dc0b4520 2024-04-25T10:23:53.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.684+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc0c6df0 con 0x7fc9dc0b4520 2024-04-25T10:23:53.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.688+0000 7fc9d3e00640 1 -- 172.21.15.87:0/3141760732 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9c801a020 con 0x7fc9dc0b4520 2024-04-25T10:23:53.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.688+0000 7fc9d3e00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc9bc066f60 0x7fc9bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.688+0000 7fc9e5dbd000 1 Processor -- start 2024-04-25T10:23:53.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.688+0000 7fc9d3e00640 1 -- 172.21.15.87:0/3141760732 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c80846c0 con 0x7fc9dc0b4520 2024-04-25T10:23:53.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.688+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc9bc066f60 0x7fc9bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.688+0000 7fc9e5dbd000 1 -- start start 2024-04-25T10:23:53.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc9bc066f60 0x7fc9bc069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0aa000 tx=0x7fc9cc0b1040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb8540d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8541240 0x564cb84f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb84f6f30 0x564cb84f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb83de5f0 con 0x564cb8540920 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8345da0 con 0x564cb8541240 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cb82ce150 con 0x564cb84f6f30 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb8540d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb8540d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50240/0 (socket says 172.21.15.87:50240) 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1759640472 learned_addr learned my addr 172.21.15.87:0/1759640472 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb84f6f30 0x564cb84f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9db400640 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8541240 0x564cb84f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb84f6f30 msgr2=0x564cb84f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb84f6f30 0x564cb84f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8541240 msgr2=0x564cb84f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8541240 0x564cb84f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 -- 172.21.15.87:0/1759640472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb82e7ca0 con 0x564cb8540920 2024-04-25T10:23:53.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9db400640 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8541240 0x564cb84f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:53.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb8540d00 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc01ca50 tx=0x7fc9cc0019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30011b047125d157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9d0c00640 1 -- 172.21.15.87:0/1759640472 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc007110 con 0x564cb8540920 2024-04-25T10:23:53.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9d0c00640 1 -- 172.21.15.87:0/1759640472 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc0072d0 con 0x564cb8540920 2024-04-25T10:23:53.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9d0c00640 1 -- 172.21.15.87:0/1759640472 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0076c0 con 0x564cb8540920 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1759640472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 msgr2=0x564cb8540d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb8540d00 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc01ca50 tx=0x7fc9cc0019d0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1759640472 shutdown_connections 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb84f6f30 0x564cb84f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8541240 0x564cb84f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1759640472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb8540d00 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1759640472 >> 172.21.15.87:0/1759640472 conn(0x564cb84fd2a0 msgr2=0x564cb84f52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1759640472 shutdown_connections 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1759640472 wait complete. 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 -- start start 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.692+0000 7fc9e5dbd000 1 Processor -- start 2024-04-25T10:23:53.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb84f6f30 0x564cb8547cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8540920 0x564cb8548230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb84faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8300990 con 0x564cb84f6f30 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8341e40 con 0x564cb8540920 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cb82d1900 con 0x564cb8541240 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 10 client.0 _ll_register_callbacks cb 0x564cb83f91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb84f6f30 0x564cb8547cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb84f6f30 0x564cb8547cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50250/0 (socket says 172.21.15.87:50250) 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 -- 172.21.15.87:0/2227745906 learned_addr learned my addr 172.21.15.87:0/2227745906 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb84faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9db400640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8540920 0x564cb8548230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 -- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 msgr2=0x564cb84faaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb84faaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 -- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8540920 msgr2=0x564cb8548230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8540920 0x564cb8548230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 -- 172.21.15.87:0/2227745906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cb82fe630 con 0x564cb84f6f30 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9db400640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8540920 0x564cb8548230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb84faaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb84f6f30 0x564cb8547cf0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c190 tx=0x7fc9cc0199c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8839b47ae848bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 1 -- 172.21.15.87:0/2227745906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0a5900 con 0x564cb84f6f30 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9cc0072d0 con 0x564cb84f6f30 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 1 -- 172.21.15.87:0/2227745906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9cc0a5aa0 con 0x564cb84f6f30 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 1 -- 172.21.15.87:0/2227745906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9cc0d6050 con 0x564cb84f6f30 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 10 client.10185 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x564cb82ce3a0 con 0x564cb84f6f30 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 1 -- 172.21.15.87:0/2227745906 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc0072d0 con 0x564cb84f6f30 2024-04-25T10:23:53.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 1 -- 172.21.15.87:0/2227745906 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9cc0a5ca0 con 0x564cb84f6f30 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 10 client.10185 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 10 client.10185 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 20 client.10185 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb84fc160 0x564cb8551d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 4 client.10185 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cb82e1a50 con 0x564cb84fc160 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9db400640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb84fc160 0x564cb8551d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9db400640 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb84fc160 0x564cb8551d50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817112e4c75cb3de server_cookie=13c78511d0c50325 in_seq=0 out_seq=0 2024-04-25T10:23:53.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 10 client.10185 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 1 -- 172.21.15.87:0/2227745906 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cb82e1a50 con 0x564cb84fc160 2024-04-25T10:23:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 10 client.10185 handle_command_reply: tid=0 2024-04-25T10:23:53.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 client.10185 shutdown 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb84f6f30 msgr2=0x564cb8547cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb84f6f30 0x564cb8547cf0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c190 tx=0x7fc9cc0199c0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 shutdown_connections 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb84faaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb8540920 0x564cb8548230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb84fc160 0x564cb8551d50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/2227745906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb84f6f30 0x564cb8547cf0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9c5600640 0 client.10185 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 >> 172.21.15.87:0/2227745906 conn(0x564cb84fd2a0 msgr2=0x564cb84f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 shutdown_connections 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.696+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/2227745906 wait complete. 2024-04-25T10:23:53.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.700+0000 7fc9e5dbd000 20 client.10185 trim_cache size 0 max 16384 2024-04-25T10:23:53.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.712+0000 7fc9e5dbd000 1 Processor -- start 2024-04-25T10:23:53.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.712+0000 7fc9e5dbd000 1 -- start start 2024-04-25T10:23:53.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb84fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8540920 0x564cb84fa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8541240 0x564cb8549920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc202640 con 0x564cb8541240 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc15f2f0 con 0x564cb84f6f30 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc1fab80 con 0x564cb8540920 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8541240 0x564cb8549920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8541240 0x564cb8549920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50260/0 (socket says 172.21.15.87:50260) 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 -- 172.21.15.87:0/1632383212 learned_addr learned my addr 172.21.15.87:0/1632383212 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb84fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9db400640 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8540920 0x564cb84fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 -- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8540920 msgr2=0x564cb84fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8540920 0x564cb84fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 -- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 msgr2=0x564cb84fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb84fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 -- 172.21.15.87:0/1632383212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb8541070 con 0x564cb8541240 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9db400640 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8540920 0x564cb84fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8541240 0x564cb8549920 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0b0f60 tx=0x7fc9d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91204f0d7bfddc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9c7e00640 1 -- 172.21.15.87:0/1632383212 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40039a0 con 0x564cb8541240 2024-04-25T10:23:53.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9c7e00640 1 -- 172.21.15.87:0/1632383212 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4003b40 con 0x564cb8541240 2024-04-25T10:23:53.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9c7e00640 1 -- 172.21.15.87:0/1632383212 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4018630 con 0x564cb8541240 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1632383212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8541240 msgr2=0x564cb8549920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8541240 0x564cb8549920 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0b0f60 tx=0x7fc9d4009750 comp rx=0 tx=0).stop 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1632383212 shutdown_connections 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8541240 0x564cb8549920 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8540920 0x564cb84fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1632383212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb84fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1632383212 >> 172.21.15.87:0/1632383212 conn(0x564cb84fd2a0 msgr2=0x564cb84f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1632383212 shutdown_connections 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1632383212 wait complete. 2024-04-25T10:23:53.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- start start 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 Processor -- start 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb854cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb858cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb858d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb834a3d0 con 0x564cb8540920 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb8401d50 con 0x564cb84f6f30 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cb82e0980 con 0x564cb8541240 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.716+0000 7fc9e5dbd000 10 client.0 _ll_register_callbacks cb 0x7fc9c000b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb858cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb858cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50268/0 (socket says 172.21.15.87:50268) 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 -- 172.21.15.87:0/1105832530 learned_addr learned my addr 172.21.15.87:0/1105832530 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb854cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e0a00640 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb858d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 -- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 msgr2=0x564cb858d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb858d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 -- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 msgr2=0x564cb854cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb854cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 -- 172.21.15.87:0/1105832530 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cb83c7600 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9db400640 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb858cb50 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8007fd0 tx=0x7fc9c8054c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5837ffeba8271f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 1 -- 172.21.15.87:0/1105832530 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80516d0 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 1 -- 172.21.15.87:0/1105832530 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c805d2e0 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9d4003b40 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 1 -- 172.21.15.87:0/1105832530 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9c8051870 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 10 client.10191 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x564cb82da9b0 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 1 -- 172.21.15.87:0/1105832530 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4003b40 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 1 -- 172.21.15.87:0/1105832530 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9c80544b0 con 0x564cb8540920 2024-04-25T10:23:53.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 10 client.10191 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 10 client.10191 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 20 client.10191 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb858d8a0 0x564cb858dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 4 client.10191 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564cb83532e0 con 0x564cb858d8a0 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb858d8a0 0x564cb858dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 10 client.10191 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9dbe00640 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb858d8a0 0x564cb858dc80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4c6c7073974680 server_cookie=d885fa4c25d2ee34 in_seq=0 out_seq=0 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 1 -- 172.21.15.87:0/1105832530 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x564cb83532e0 con 0x564cb858d8a0 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 10 client.10191 handle_command_reply: tid=0 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 client.10191 shutdown 2024-04-25T10:23:53.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 msgr2=0x564cb858cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb858cb50 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8007fd0 tx=0x7fc9c8054c90 comp rx=0 tx=0).stop 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 shutdown_connections 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564cb8541240 0x564cb858d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x564cb858d8a0 0x564cb858dc80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb8540920 0x564cb858cb50 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 --2- 172.21.15.87:0/1105832530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564cb84f6f30 0x564cb854cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9c5600640 0 client.10191 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 >> 172.21.15.87:0/1105832530 conn(0x564cb84fd2a0 msgr2=0x564cb84fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 shutdown_connections 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 1 -- 172.21.15.87:0/1105832530 wait complete. 2024-04-25T10:23:53.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e5dbd000 20 client.10191 trim_cache size 0 max 16384 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 10044, 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8603787479999996, 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 111, 2024-04-25T10:23:53.727 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 151, 2024-04-25T10:23:53.728 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 151, 2024-04-25T10:23:53.728 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.784007602999999 2024-04-25T10:23:53.728 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc9bc066f60 msgr2=0x7fc9bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc9bc066f60 0x7fc9bc069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0aa000 tx=0x7fc9cc0b1040 comp rx=0 tx=0).stop 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9dc0b4520 msgr2=0x7fc9dc160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9dc0b4520 0x7fc9dc160a00 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800e7e0 tx=0x7fc9c800eca0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 shutdown_connections 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc9bc066f60 0x7fc9bc069420 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9dc0b49a0 0x7fc9dc160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9dc0b4520 0x7fc9dc160a00 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 --2- 172.21.15.87:0/3141760732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9c0003b80 0x7fc9dc15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 >> 172.21.15.87:0/3141760732 conn(0x7fc9dc0d1900 msgr2=0x7fc9dc0cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 shutdown_connections 2024-04-25T10:23:53.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.720+0000 7fc9e2a00640 1 -- 172.21.15.87:0/3141760732 wait complete. 2024-04-25T10:23:53.742 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:23:53.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.876+0000 7fe416400640 1 Processor -- start 2024-04-25T10:23:53.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe416400640 1 -- start start 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe416400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe41015a0e0 0x7fe41015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe416400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe410071f70 0x7fe4100b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe416400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41015b6e0 con 0x7fe4100b2fc0 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe416400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe410070ed0 con 0x7fe41015a0e0 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe416400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100711b0 con 0x7fe410071f70 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe40f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe41015a0e0 0x7fe41015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe40f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe41015a0e0 0x7fe41015b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50282/0 (socket says 172.21.15.87:50282) 2024-04-25T10:23:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe40f400640 1 -- 172.21.15.87:0/3476930683 learned_addr learned my addr 172.21.15.87:0/3476930683 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe406a00640 1 --2- 172.21.15.87:0/3476930683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe410071f70 0x7fe4100b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe40f400640 1 -- 172.21.15.87:0/3476930683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe410071f70 msgr2=0x7fe4100b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe40f400640 1 --2- 172.21.15.87:0/3476930683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe410071f70 0x7fe4100b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe40f400640 1 -- 172.21.15.87:0/3476930683 >> v1:172.21.15.169:6789/0 conn(0x7fe4100b2fc0 legacy=0x7fe41006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe40f400640 1 -- 172.21.15.87:0/3476930683 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410071490 con 0x7fe41015a0e0 2024-04-25T10:23:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.880+0000 7fe406a00640 1 --2- 172.21.15.87:0/3476930683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe410071f70 0x7fe4100b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe40f400640 1 --2- 172.21.15.87:0/3476930683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe41015a0e0 0x7fe41015b1a0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe400009e50 tx=0x7fe40001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34a7bfa30fc2d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe40ea00640 1 -- 172.21.15.87:0/3476930683 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4000ac020 con 0x7fe41015a0e0 2024-04-25T10:23:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe40ea00640 1 -- 172.21.15.87:0/3476930683 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4000a6420 con 0x7fe41015a0e0 2024-04-25T10:23:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe40ea00640 1 -- 172.21.15.87:0/3476930683 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4000a67f0 con 0x7fe41015a0e0 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 -- 172.21.15.87:0/3476930683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe41015a0e0 msgr2=0x7fe41015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 --2- 172.21.15.87:0/3476930683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe41015a0e0 0x7fe41015b1a0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe400009e50 tx=0x7fe40001c810 comp rx=0 tx=0).stop 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 -- 172.21.15.87:0/3476930683 shutdown_connections 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 --2- 172.21.15.87:0/3476930683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe410071f70 0x7fe4100b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 --2- 172.21.15.87:0/3476930683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe41015a0e0 0x7fe41015b1a0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 -- 172.21.15.87:0/3476930683 >> 172.21.15.87:0/3476930683 conn(0x7fe410066b70 msgr2=0x7fe410066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 -- 172.21.15.87:0/3476930683 shutdown_connections 2024-04-25T10:23:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 -- 172.21.15.87:0/3476930683 wait complete. 2024-04-25T10:23:53.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 Processor -- start 2024-04-25T10:23:53.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.884+0000 7fe416400640 1 -- start start 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe410071f70 0x7fe4101560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4100b2fc0 0x7fe41014c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe41015a0e0 0x7fe4101582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41015a4c0 con 0x7fe4100b2fc0 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100af030 con 0x7fe410071f70 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe410070b50 con 0x7fe41015a0e0 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4100b2fc0 0x7fe41014c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4100b2fc0 0x7fe41014c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50294/0 (socket says 172.21.15.87:50294) 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 -- 172.21.15.87:0/4103300240 learned_addr learned my addr 172.21.15.87:0/4103300240 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe40f400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe410071f70 0x7fe4101560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe40fe00640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe41015a0e0 0x7fe4101582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 -- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe41015a0e0 msgr2=0x7fe4101582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe41015a0e0 0x7fe4101582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 -- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe410071f70 msgr2=0x7fe4101560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe410071f70 0x7fe4101560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 -- 172.21.15.87:0/4103300240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410158810 con 0x7fe4100b2fc0 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe406a00640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4100b2fc0 0x7fe41014c570 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c820 tx=0x7fe3f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe407e00640 1 -- 172.21.15.87:0/4103300240 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8015070 con 0x7fe4100b2fc0 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe407e00640 1 -- 172.21.15.87:0/4103300240 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8007a30 con 0x7fe4100b2fc0 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe407e00640 1 -- 172.21.15.87:0/4103300240 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8007e40 con 0x7fe4100b2fc0 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe410156760 con 0x7fe4100b2fc0 2024-04-25T10:23:53.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe410156d20 con 0x7fe4100b2fc0 2024-04-25T10:23:53.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41004ef30 con 0x7fe4100b2fc0 2024-04-25T10:23:53.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.888+0000 7fe407e00640 1 -- 172.21.15.87:0/4103300240 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3f8018070 con 0x7fe4100b2fc0 2024-04-25T10:23:53.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.892+0000 7fe407e00640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3ec066f60 0x7fe3ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:53.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.892+0000 7fe407e00640 1 -- 172.21.15.87:0/4103300240 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3f8084e60 con 0x7fe4100b2fc0 2024-04-25T10:23:53.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.892+0000 7fe40f400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3ec066f60 0x7fe3ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:53.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.892+0000 7fe40f400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3ec066f60 0x7fe3ec069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe4000ad000 tx=0x7fe4000b4040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:53.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:53.896+0000 7fe407e00640 1 -- 172.21.15.87:0/4103300240 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3f8055820 con 0x7fe4100b2fc0 2024-04-25T10:23:53.981 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:53.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.088+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe41015dca0 con 0x7fe4100b2fc0 2024-04-25T10:23:54.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.088+0000 7fe407e00640 1 -- 172.21.15.87:0/4103300240 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe3f80598c0 con 0x7fe4100b2fc0 2024-04-25T10:23:54.089 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:23:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3ec066f60 msgr2=0x7fe3ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3ec066f60 0x7fe3ec069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe4000ad000 tx=0x7fe4000b4040 comp rx=0 tx=0).stop 2024-04-25T10:23:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4100b2fc0 msgr2=0x7fe41014c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4100b2fc0 0x7fe41014c570 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c820 tx=0x7fe3f800cce0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 shutdown_connections 2024-04-25T10:23:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe3ec066f60 0x7fe3ec069420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe41015a0e0 0x7fe4101582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4100b2fc0 0x7fe41014c570 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 --2- 172.21.15.87:0/4103300240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe410071f70 0x7fe4101560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 >> 172.21.15.87:0/4103300240 conn(0x7fe410066b70 msgr2=0x7fe410150250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 shutdown_connections 2024-04-25T10:23:54.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.092+0000 7fe416400640 1 -- 172.21.15.87:0/4103300240 wait complete. 2024-04-25T10:23:54.110 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:23:54.110 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:23:54.110 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:23:54.110 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:23:54.110 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:54.111 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:23:54.111 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:23:54.111 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:23:54.111 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:23:54.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.111 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:54.120 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41780sec preferred_lft 41780sec 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:23:54.121 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:23:54.122 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:23:54.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.124 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:23:54.183 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:23:54.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.185 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:23:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:23:54.239 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:23:54.239 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:23:54.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.239 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:23:54.239 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:23:54.239 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:23:54.239 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:23:54.239 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:23:54.239 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:23:54.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.337 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:23:54.346 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:23:54.346 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:23:54.347 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:23:54.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.347 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:23:54.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:23:54.409 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:54.458 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:23:54.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:23:54.458 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:54.509 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:23:54.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.509 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:54.597 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:54.597 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:23:54.597 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:23:54.612 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:54.612 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:23:54.612 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:23:54.612 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:23:54.612 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:23:54.623 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:23:54.627 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:23:54.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.627 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:23:54.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:54.647 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:23:54.705 INFO:teuthology.orchestra.run.smithi087.stdout:client9277 2024-04-25T10:23:54.705 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:23:54.706 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:54.758 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:23:54.758 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:23:54.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.940+0000 7f36b6a00640 1 Processor -- start 2024-04-25T10:23:54.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.940+0000 7f36b6a00640 1 -- start start 2024-04-25T10:23:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f36b00721f0 0x7f36b0166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b6a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b0071360 con 0x7f36b00aeda0 2024-04-25T10:23:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b0159920 con 0x7f36b0159540 2024-04-25T10:23:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b6a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b015c2f0 con 0x7f36b00721f0 2024-04-25T10:23:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36af400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f36b00aeda0 0x7f36b0071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54206/0 (socket says 172.21.15.87:54206) 2024-04-25T10:23:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36af400640 1 -- 172.21.15.87:0/951848709 learned_addr learned my addr 172.21.15.87:0/951848709 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b4a00640 1 --2- 172.21.15.87:0/951848709 >> v2:172.21.15.169:3300/0 conn(0x7f36b00721f0 0x7f36b0166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36aea00640 1 -- 172.21.15.87:0/951848709 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929046826 0 0) 0x7f36b0071360 con 0x7f36b00aeda0 2024-04-25T10:23:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36aea00640 1 -- 172.21.15.87:0/951848709 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3694003530 con 0x7f36b00aeda0 2024-04-25T10:23:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b4a00640 1 -- 172.21.15.87:0/951848709 >> v1:172.21.15.169:6789/0 conn(0x7f36b0159540 legacy=0x7f36b00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b4a00640 1 -- 172.21.15.87:0/951848709 >> v1:172.21.15.87:6789/0 conn(0x7f36b00aeda0 legacy=0x7f36b0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.944+0000 7f36b4a00640 1 -- 172.21.15.87:0/951848709 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b015c5d0 con 0x7f36b00721f0 2024-04-25T10:23:54.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b4a00640 1 --2- 172.21.15.87:0/951848709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00721f0 0x7f36b0166920 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f36a80097c0 tx=0x7f36a801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc18c6998cff897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:54.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36aea00640 1 -- 172.21.15.87:0/951848709 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80a7070 con 0x7f36b00721f0 2024-04-25T10:23:54.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36aea00640 1 -- 172.21.15.87:0/951848709 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a80b0070 con 0x7f36b00721f0 2024-04-25T10:23:54.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36aea00640 1 -- 172.21.15.87:0/951848709 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80ab890 con 0x7f36b00721f0 2024-04-25T10:23:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 -- 172.21.15.87:0/951848709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00721f0 msgr2=0x7f36b0166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 --2- 172.21.15.87:0/951848709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00721f0 0x7f36b0166920 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f36a80097c0 tx=0x7f36a801c470 comp rx=0 tx=0).stop 2024-04-25T10:23:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 -- 172.21.15.87:0/951848709 shutdown_connections 2024-04-25T10:23:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 --2- 172.21.15.87:0/951848709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00721f0 0x7f36b0166920 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 -- 172.21.15.87:0/951848709 >> 172.21.15.87:0/951848709 conn(0x7f36b00669a0 msgr2=0x7f36b0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 -- 172.21.15.87:0/951848709 shutdown_connections 2024-04-25T10:23:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 -- 172.21.15.87:0/951848709 wait complete. 2024-04-25T10:23:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 Processor -- start 2024-04-25T10:23:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.948+0000 7f36b6a00640 1 -- start start 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b00721f0 0x7f36b0159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00aeda0 0x7f36b00b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36b0159540 0x7f36b0153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0169680 con 0x7f36b0159540 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b006c0d0 con 0x7f36b00aeda0 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0169800 con 0x7f36b00721f0 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36b0159540 0x7f36b0153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36b0159540 0x7f36b0153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50308/0 (socket says 172.21.15.87:50308) 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 -- 172.21.15.87:0/1956871906 learned_addr learned my addr 172.21.15.87:0/1956871906 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36af400640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00aeda0 0x7f36b00b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36afe00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b00721f0 0x7f36b0159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 -- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b00721f0 msgr2=0x7f36b0159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b00721f0 0x7f36b0159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 -- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00aeda0 msgr2=0x7f36b00b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00aeda0 0x7f36b00b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 -- 172.21.15.87:0/1956871906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b015b4b0 con 0x7f36b0159540 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36afe00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b00721f0 0x7f36b0159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36b0159540 0x7f36b0153ee0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f36a80ad040 tx=0x7f36a801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36a7e00640 1 -- 172.21.15.87:0/1956871906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80abe60 con 0x7f36b0159540 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36a7e00640 1 -- 172.21.15.87:0/1956871906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a80bf610 con 0x7f36b0159540 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b015a410 con 0x7f36b0159540 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36a7e00640 1 -- 172.21.15.87:0/1956871906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80bf9e0 con 0x7f36b0159540 2024-04-25T10:23:54.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b015a9d0 con 0x7f36b0159540 2024-04-25T10:23:54.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.952+0000 7f36a7e00640 1 -- 172.21.15.87:0/1956871906 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36a80bd020 con 0x7f36b0159540 2024-04-25T10:23:54.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36a7e00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3690066f60 0x7f3690069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36a7e00640 1 -- 172.21.15.87:0/1956871906 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36a811f6e0 con 0x7f36b0159540 2024-04-25T10:23:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3690066f60 0x7f3690069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 Processor -- start 2024-04-25T10:23:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- start start 2024-04-25T10:23:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3690066f60 0x7f3690069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f36a0001d70 tx=0x7f36a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5180 0x55d5463e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d54639b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d54639b600 0x55d54639b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5461e8f20 con 0x55d5463e5180 2024-04-25T10:23:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d546171000 con 0x55d5463e5aa0 2024-04-25T10:23:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5461e4fc0 con 0x55d54639b600 2024-04-25T10:23:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5180 0x55d5463e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5180 0x55d5463e5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50324/0 (socket says 172.21.15.87:50324) 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 -- 172.21.15.87:0/1829342517 learned_addr learned my addr 172.21.15.87:0/1829342517 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36af400640 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d54639b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d54639b600 0x55d54639b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 -- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d54639b600 msgr2=0x55d54639b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d54639b600 0x55d54639b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 -- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 msgr2=0x55d54639b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d54639b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 -- 172.21.15.87:0/1829342517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5461c4ba0 con 0x55d5463e5180 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36afe00640 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5180 0x55d5463e5560 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f36a0003b60 tx=0x7f36a0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffd814eb874995e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36a4c00640 1 -- 172.21.15.87:0/1829342517 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00052e0 con 0x55d5463e5180 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36a4c00640 1 -- 172.21.15.87:0/1829342517 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0005480 con 0x55d5463e5180 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36a4c00640 1 -- 172.21.15.87:0/1829342517 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0021450 con 0x55d5463e5180 2024-04-25T10:23:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1829342517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5180 msgr2=0x55d5463e5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5180 0x55d5463e5560 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f36a0003b60 tx=0x7f36a0011000 comp rx=0 tx=0).stop 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1829342517 shutdown_connections 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d54639b600 0x55d54639b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d54639b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1829342517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5180 0x55d5463e5560 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1829342517 >> 172.21.15.87:0/1829342517 conn(0x55d5463a1760 msgr2=0x55d5463999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1829342517 shutdown_connections 2024-04-25T10:23:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.956+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1829342517 wait complete. 2024-04-25T10:23:54.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 -- start start 2024-04-25T10:23:54.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 Processor -- start 2024-04-25T10:23:54.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d5463a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5180 0x55d5463a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5aa0 0x55d5463d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5462a69c0 con 0x55d54639b600 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5461a35d0 con 0x55d5463e5180 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d54619dd40 con 0x55d5463e5aa0 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 10 client.0 _ll_register_callbacks cb 0x55d546386a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d5463a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d5463a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50338/0 (socket says 172.21.15.87:50338) 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 -- 172.21.15.87:0/3069239722 learned_addr learned my addr 172.21.15.87:0/3069239722 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36af400640 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5180 0x55d5463a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b4a00640 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5aa0 0x55d5463d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 -- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5aa0 msgr2=0x55d5463d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5aa0 0x55d5463d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 -- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5180 msgr2=0x55d5463a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5180 0x55d5463a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 -- 172.21.15.87:0/3069239722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d54622f5a0 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36afe00640 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d5463a06a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x55d5461eef00 tx=0x7f36a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1bbd96720b8ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f3699600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f3699600640 1 -- 172.21.15.87:0/3069239722 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0021660 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f3699600640 1 -- 172.21.15.87:0/3069239722 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36a0010050 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36a0005480 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f3699600640 1 -- 172.21.15.87:0/3069239722 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a0033050 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 10 client.10209 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55d5461c02b0 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f3699600640 1 -- 172.21.15.87:0/3069239722 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0005480 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f3699600640 1 -- 172.21.15.87:0/3069239722 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f36a0036020 con 0x55d54639b600 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 10 client.10209 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 10 client.10209 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 20 client.10209 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d5463d2e60 0x55d5463d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 4 client.10209 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d546184c70 con 0x55d5463d2e60 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.960+0000 7f36af400640 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d5463d2e60 0x55d5463d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36af400640 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d5463d2e60 0x55d5463d3e60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa45161bb250e70 server_cookie=56501195ec26367e in_seq=0 out_seq=0 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f3699600640 10 client.10209 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f3699600640 1 -- 172.21.15.87:0/3069239722 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d546184c70 con 0x55d5463d2e60 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f3699600640 10 client.10209 handle_command_reply: tid=0 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 client.10209 shutdown 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 msgr2=0x55d5463a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d5463a06a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x55d5461eef00 tx=0x7f36a0013000 comp rx=0 tx=0).stop 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 shutdown_connections 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5aa0 0x55d5463d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5180 0x55d5463a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d5463d2e60 0x55d5463d3e60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/3069239722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d5463a06a0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f3699600640 0 client.10209 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:54.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 >> 172.21.15.87:0/3069239722 conn(0x55d5463a1760 msgr2=0x55d54639a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 shutdown_connections 2024-04-25T10:23:54.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 1 -- 172.21.15.87:0/3069239722 wait complete. 2024-04-25T10:23:54.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.964+0000 7f36b9f4d000 20 client.10209 trim_cache size 0 max 16384 2024-04-25T10:23:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 Processor -- start 2024-04-25T10:23:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 -- start start 2024-04-25T10:23:54.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d54639a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5463e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d5463e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0200840 con 0x55d54639b600 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b01ffa00 con 0x55d5463e5aa0 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b01fef20 con 0x55d5463e5180 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d54639a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d54639a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50342/0 (socket says 172.21.15.87:50342) 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 -- 172.21.15.87:0/1554661489 learned_addr learned my addr 172.21.15.87:0/1554661489 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36af400640 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5463e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d5463e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 -- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 msgr2=0x55d5463e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5463e45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 -- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 msgr2=0x55d5463e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d5463e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 -- 172.21.15.87:0/1554661489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d546171250 con 0x55d54639b600 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d5463e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f36afe00640 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d54639a330 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005040 tx=0x7f36a001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26ffaa0a1a1fc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f369be00640 1 -- 172.21.15.87:0/1554661489 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0021660 con 0x55d54639b600 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f369be00640 1 -- 172.21.15.87:0/1554661489 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a00053a0 con 0x55d54639b600 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.976+0000 7f369be00640 1 -- 172.21.15.87:0/1554661489 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00218d0 con 0x55d54639b600 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1554661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 msgr2=0x55d54639a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d54639a330 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005040 tx=0x7f36a001d000 comp rx=0 tx=0).stop 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1554661489 shutdown_connections 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d5463e5aa0 0x55d5463e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5463e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1554661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d54639b600 0x55d54639a330 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1554661489 >> 172.21.15.87:0/1554661489 conn(0x55d5463a1760 msgr2=0x55d5463e3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1554661489 shutdown_connections 2024-04-25T10:23:54.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1554661489 wait complete. 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- start start 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 Processor -- start 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d54639b600 0x55d546449460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5464499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5aa0 0x55d546404cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54618fc90 con 0x55d5463e5aa0 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b01ff400 con 0x55d54639b600 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d54619fdc0 con 0x55d5463e5180 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 10 client.0 _ll_register_callbacks cb 0x55d546367600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:54.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5aa0 0x55d546404cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5aa0 0x55d546404cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50344/0 (socket says 172.21.15.87:50344) 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 -- 172.21.15.87:0/1645881232 learned_addr learned my addr 172.21.15.87:0/1645881232 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36afe00640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d54639b600 0x55d546449460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36af400640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5464499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 -- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 msgr2=0x55d5464499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5464499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 -- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d54639b600 msgr2=0x55d546449460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d54639b600 0x55d546449460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 -- 172.21.15.87:0/1645881232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d54617db90 con 0x55d5463e5aa0 2024-04-25T10:23:54.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36afe00640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d54639b600 0x55d546449460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b4a00640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5aa0 0x55d546404cb0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f36a80eff90 tx=0x7f36a80efef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcd3d9b4fde35df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f3699600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f3699600640 1 -- 172.21.15.87:0/1645881232 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80bfd80 con 0x55d5463e5aa0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f3699600640 1 -- 172.21.15.87:0/1645881232 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36a8118b80 con 0x55d5463e5aa0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36a00053a0 con 0x55d5463e5aa0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f3699600640 1 -- 172.21.15.87:0/1645881232 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a80f40a0 con 0x55d5463e5aa0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 10 client.10215 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55d546190010 con 0x55d5463e5aa0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f3699600640 1 -- 172.21.15.87:0/1645881232 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a00053a0 con 0x55d5463e5aa0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f3699600640 1 -- 172.21.15.87:0/1645881232 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f36a8107ce0 con 0x55d5463e5aa0 2024-04-25T10:23:54.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 10 client.10215 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 10 client.10215 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 20 client.10215 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d546405b50 0x55d546405f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 4 client.10215 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5463d37e0 con 0x55d546405b50 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.980+0000 7f36afe00640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d546405b50 0x55d546405f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36afe00640 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d546405b50 0x55d546405f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626791cf718ec004 server_cookie=8a90f20d39c4f86f in_seq=0 out_seq=0 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f3699600640 10 client.10215 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f3699600640 1 -- 172.21.15.87:0/1645881232 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5463d37e0 con 0x55d546405b50 2024-04-25T10:23:54.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f3699600640 10 client.10215 handle_command_reply: tid=0 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 client.10215 shutdown 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5aa0 msgr2=0x55d546404cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5aa0 0x55d546404cb0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f36a80eff90 tx=0x7f36a80efef0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 shutdown_connections 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55d546405b50 0x55d546405f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5463e5aa0 0x55d546404cb0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d5463e5180 0x55d5464499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 --2- 172.21.15.87:0/1645881232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d54639b600 0x55d546449460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f3699600640 0 client.10215 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 >> 172.21.15.87:0/1645881232 conn(0x55d5463a1760 msgr2=0x55d5463e4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 shutdown_connections 2024-04-25T10:23:54.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 1 -- 172.21.15.87:0/1645881232 wait complete. 2024-04-25T10:23:54.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b9f4d000 20 client.10215 trim_cache size 0 max 16384 2024-04-25T10:23:54.995 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3690066f60 msgr2=0x7f3690069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3690066f60 0x7f3690069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f36a0001d70 tx=0x7f36a0009040 comp rx=0 tx=0).stop 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36b0159540 msgr2=0x7f36b0153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36b0159540 0x7f36b0153ee0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f36a80ad040 tx=0x7f36a801acb0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 shutdown_connections 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3690066f60 0x7f3690069420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36b0159540 0x7f36b0153ee0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b00aeda0 0x7f36b00b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 --2- 172.21.15.87:0/1956871906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b00721f0 0x7f36b0159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 >> 172.21.15.87:0/1956871906 conn(0x7f36b00669a0 msgr2=0x7f36b0066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 shutdown_connections 2024-04-25T10:23:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:54.984+0000 7f36b6a00640 1 -- 172.21.15.87:0/1956871906 wait complete. 2024-04-25T10:23:55.003 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:23:55.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.136+0000 7fb27ac00640 1 Processor -- start 2024-04-25T10:23:55.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.136+0000 7fb27ac00640 1 -- start start 2024-04-25T10:23:55.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.136+0000 7fb27ac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb27415b910 0x7fb27415c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.136+0000 7fb27ac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27415bd60 con 0x7fb2740d0040 2024-04-25T10:23:55.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.136+0000 7fb27ac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb274071050 con 0x7fb2741550b0 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.136+0000 7fb27ac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2740712f0 con 0x7fb27415b910 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb26b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb2740d0040 0x7fb27415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54222/0 (socket says 172.21.15.87:54222) 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb26b400640 1 -- 172.21.15.87:0/3176600373 learned_addr learned my addr 172.21.15.87:0/3176600373 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 --2- 172.21.15.87:0/3176600373 >> v2:172.21.15.169:3300/0 conn(0x7fb27415b910 0x7fb27415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877224300 0 0) 0x7fb27415bd60 con 0x7fb2740d0040 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb258003530 con 0x7fb2740d0040 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 437391262 0 0) 0x7fb258003530 con 0x7fb2740d0040 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 >> v2:172.21.15.169:3300/0 conn(0x7fb27415b910 msgr2=0x7fb27415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 --2- 172.21.15.87:0/3176600373 >> v2:172.21.15.169:3300/0 conn(0x7fb27415b910 0x7fb27415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 >> v1:172.21.15.169:6789/0 conn(0x7fb2741550b0 legacy=0x7fb2740cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740715d0 con 0x7fb2740d0040 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb25c001680 con 0x7fb2740d0040 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 >> v1:172.21.15.87:6789/0 conn(0x7fb2740d0040 legacy=0x7fb27415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2580039c0 0x7fb258003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb258004810 0x7fb258004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb25800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740712f0 con 0x7fb2580051d0 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25800c070 con 0x7fb2580039c0 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25800c270 con 0x7fb258004810 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 --2- 172.21.15.87:0/3176600373 >> v2:172.21.15.169:3300/0 conn(0x7fb27415b910 0x7fb27415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb273e00640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2580039c0 0x7fb258003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb25800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb26b400640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb258004810 0x7fb258004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 -- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb258004810 msgr2=0x7fb258004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb258004810 0x7fb258004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 -- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2580039c0 msgr2=0x7fb258003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2580039c0 0x7fb258003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.140+0000 7fb278c00640 1 -- 172.21.15.87:0/3176600373 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258003530 con 0x7fb2580051d0 2024-04-25T10:23:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb278c00640 1 -- 172.21.15.87:0/3176600373 reap_dead start 2024-04-25T10:23:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb278c00640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb25800b9d0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fb26c010700 tx=0x7fb26c01f590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44ec17cabca0dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0af020 con 0x7fb2580051d0 2024-04-25T10:23:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c003ba0 con 0x7fb2580051d0 2024-04-25T10:23:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb273400640 1 -- 172.21.15.87:0/3176600373 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c010350 con 0x7fb2580051d0 2024-04-25T10:23:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 -- 172.21.15.87:0/3176600373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 msgr2=0x7fb25800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb25800b9d0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fb26c010700 tx=0x7fb26c01f590 comp rx=0 tx=0).stop 2024-04-25T10:23:55.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 -- 172.21.15.87:0/3176600373 shutdown_connections 2024-04-25T10:23:55.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 --2- 172.21.15.87:0/3176600373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb25800b9d0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 -- 172.21.15.87:0/3176600373 >> 172.21.15.87:0/3176600373 conn(0x7fb274066b70 msgr2=0x7fb274066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 -- 172.21.15.87:0/3176600373 shutdown_connections 2024-04-25T10:23:55.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 -- 172.21.15.87:0/3176600373 wait complete. 2024-04-25T10:23:55.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 Processor -- start 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.144+0000 7fb27ac00640 1 -- start start 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb2740cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb27415a1c0 0x7fb27414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb274157610 0x7fb274157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740720b0 con 0x7fb2580051d0 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb274071b90 con 0x7fb274157610 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27415b200 con 0x7fb27415a1c0 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb2740cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb2740cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50362/0 (socket says 172.21.15.87:50362) 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 -- 172.21.15.87:0/506468082 learned_addr learned my addr 172.21.15.87:0/506468082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb278c00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb274157610 0x7fb274157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb26b400640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb27415a1c0 0x7fb27414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 -- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb27415a1c0 msgr2=0x7fb27414e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb27415a1c0 0x7fb27414e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 -- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb274157610 msgr2=0x7fb274157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb274157610 0x7fb274157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 -- 172.21.15.87:0/506468082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274155530 con 0x7fb2580051d0 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb278c00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb274157610 0x7fb274157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb273e00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb2740cd200 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fb26400f620 tx=0x7fb26400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb270c00640 1 -- 172.21.15.87:0/506468082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2640052e0 con 0x7fb2580051d0 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb270c00640 1 -- 172.21.15.87:0/506468082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb264018070 con 0x7fb2580051d0 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb270c00640 1 -- 172.21.15.87:0/506468082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264003820 con 0x7fb2580051d0 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27415a640 con 0x7fb2580051d0 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2740cdac0 con 0x7fb2580051d0 2024-04-25T10:23:55.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27404ef30 con 0x7fb2580051d0 2024-04-25T10:23:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.148+0000 7fb270c00640 1 -- 172.21.15.87:0/506468082 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2640039c0 con 0x7fb2580051d0 2024-04-25T10:23:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.152+0000 7fb270c00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.152+0000 7fb270c00640 1 -- 172.21.15.87:0/506468082 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb264084e20 con 0x7fb2580051d0 2024-04-25T10:23:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.152+0000 7fb26b400640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.152+0000 7fb26b400640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb250066f60 0x7fb250069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb2740ce7b0 tx=0x7fb25c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.156+0000 7fb270c00640 1 -- 172.21.15.87:0/506468082 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb26401a310 con 0x7fb2580051d0 2024-04-25T10:23:55.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.340+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb2740b07e0 con 0x7fb2580051d0 2024-04-25T10:23:55.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.340+0000 7fb270c00640 1 -- 172.21.15.87:0/506468082 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb264005480 con 0x7fb2580051d0 2024-04-25T10:23:55.341 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:23:55.341 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:23:55.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb250066f60 msgr2=0x7fb250069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb250066f60 0x7fb250069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb2740ce7b0 tx=0x7fb25c009040 comp rx=0 tx=0).stop 2024-04-25T10:23:55.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 msgr2=0x7fb2740cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb2740cd200 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fb26400f620 tx=0x7fb26400fae0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 shutdown_connections 2024-04-25T10:23:55.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb274157610 0x7fb274157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb27415a1c0 0x7fb27414e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 --2- 172.21.15.87:0/506468082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580051d0 0x7fb2740cd200 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 >> 172.21.15.87:0/506468082 conn(0x7fb274066b70 msgr2=0x7fb2740b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 shutdown_connections 2024-04-25T10:23:55.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.344+0000 7fb27ac00640 1 -- 172.21.15.87:0/506468082 wait complete. 2024-04-25T10:23:55.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:55.362 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start '~mdsdir' recursive 2024-04-25T10:23:55.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.500+0000 7f4606200640 1 Processor -- start 2024-04-25T10:23:55.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.500+0000 7f4606200640 1 -- start start 2024-04-25T10:23:55.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f4606200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4600070ba0 0x7f46000720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f4606200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4600071040 con 0x7f4600157cb0 2024-04-25T10:23:55.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f4606200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4600071320 con 0x7f46000b2af0 2024-04-25T10:23:55.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f4606200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46000af150 con 0x7f4600070ba0 2024-04-25T10:23:55.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45ff400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4600157cb0 0x7f460015b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54238/0 (socket says 172.21.15.87:54238) 2024-04-25T10:23:55.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45ff400640 1 -- 172.21.15.87:0/3485116456 learned_addr learned my addr 172.21.15.87:0/3485116456 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fea00640 1 --2- 172.21.15.87:0/3485116456 >> v2:172.21.15.169:3300/0 conn(0x7f4600070ba0 0x7f46000720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968884050 0 0) 0x7f4600071040 con 0x7f4600157cb0 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e4003530 con 0x7f4600157cb0 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2741364438 0 0) 0x7f45e4003530 con 0x7f4600157cb0 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 >> v2:172.21.15.169:3300/0 conn(0x7f4600070ba0 msgr2=0x7f46000720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 --2- 172.21.15.87:0/3485116456 >> v2:172.21.15.169:3300/0 conn(0x7f4600070ba0 0x7f46000720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 >> v1:172.21.15.169:6789/0 conn(0x7f46000b2af0 legacy=0x7f460006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4600071600 con 0x7f4600157cb0 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f45f0001680 con 0x7f4600157cb0 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 >> v1:172.21.15.87:6789/0 conn(0x7f4600157cb0 legacy=0x7f460015b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039c0 0x7f45e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f45e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45e40051d0 0x7f45e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e400c140 con 0x7f45e4004810 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e400c2f0 con 0x7f45e40039c0 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e400c4f0 con 0x7f45e40051d0 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fea00640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f45e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45ffe00640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45e40051d0 0x7f45e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45ff400640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039c0 0x7f45e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fea00640 1 -- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45e40051d0 msgr2=0x7f45e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fea00640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45e40051d0 0x7f45e400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fea00640 1 -- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039c0 msgr2=0x7f45e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fea00640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039c0 0x7f45e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45fea00640 1 -- 172.21.15.87:0/3485116456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e4003530 con 0x7f45e4004810 2024-04-25T10:23:55.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.504+0000 7f45ffe00640 1 -- 172.21.15.87:0/3485116456 reap_dead start 2024-04-25T10:23:55.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f45fea00640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f45e4004c90 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009ae0 tx=0x7f45ec01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0043bf28be92d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0b2070 con 0x7f45e4004810 2024-04-25T10:23:55.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0a5520 con 0x7f45e4004810 2024-04-25T10:23:55.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f45fe000640 1 -- 172.21.15.87:0/3485116456 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0096b0 con 0x7f45e4004810 2024-04-25T10:23:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 -- 172.21.15.87:0/3485116456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 msgr2=0x7f45e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f45e4004c90 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009ae0 tx=0x7f45ec01ca40 comp rx=0 tx=0).stop 2024-04-25T10:23:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 -- 172.21.15.87:0/3485116456 shutdown_connections 2024-04-25T10:23:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 --2- 172.21.15.87:0/3485116456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f45e4004c90 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 -- 172.21.15.87:0/3485116456 >> 172.21.15.87:0/3485116456 conn(0x7f4600066b70 msgr2=0x7f4600066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 -- 172.21.15.87:0/3485116456 shutdown_connections 2024-04-25T10:23:55.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 -- 172.21.15.87:0/3485116456 wait complete. 2024-04-25T10:23:55.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 Processor -- start 2024-04-25T10:23:55.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.508+0000 7f4606200640 1 -- start start 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f460014b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4600157180 0x7f4600152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600152690 0x7f460014cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460015b670 con 0x7f45e4004810 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600158090 con 0x7f4600152690 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4600156a50 con 0x7f4600157180 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f460014b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f460014b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50380/0 (socket says 172.21.15.87:50380) 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 -- 172.21.15.87:0/2816282640 learned_addr learned my addr 172.21.15.87:0/2816282640 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45fea00640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4600157180 0x7f4600152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ffe00640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600152690 0x7f460014cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 -- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4600157180 msgr2=0x7f4600152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4600157180 0x7f4600152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 -- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600152690 msgr2=0x7f460014cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600152690 0x7f460014cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 -- 172.21.15.87:0/2816282640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4600156490 con 0x7f45e4004810 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45ff400640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f460014b5a0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e780 tx=0x7f45f000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45f7400640 1 -- 172.21.15.87:0/2816282640 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000ee30 con 0x7f45e4004810 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45f7400640 1 -- 172.21.15.87:0/2816282640 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f0018070 con 0x7f45e4004810 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4600156770 con 0x7f45e4004810 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45f7400640 1 -- 172.21.15.87:0/2816282640 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000cad0 con 0x7f45e4004810 2024-04-25T10:23:55.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460014c250 con 0x7f45e4004810 2024-04-25T10:23:55.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.512+0000 7f45f7400640 1 -- 172.21.15.87:0/2816282640 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45f000cc70 con 0x7f45e4004810 2024-04-25T10:23:55.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45f7400640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f45e0066f60 0x7f45e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45f7400640 1 -- 172.21.15.87:0/2816282640 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f0084bc0 con 0x7f45e4004810 2024-04-25T10:23:55.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f45e0066f60 0x7f45e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 Processor -- start 2024-04-25T10:23:55.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 -- start start 2024-04-25T10:23:55.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f45e0066f60 0x7f45e0069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0af000 tx=0x7f45ec0046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bb3fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb401e0 0x55a75baf5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a75b943dd0 con 0x55a75bb401e0 2024-04-25T10:23:55.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a75b8cc010 con 0x55a75baf5650 2024-04-25T10:23:55.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a75b93fe70 con 0x55a75bb3f8c0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb401e0 0x55a75baf5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb401e0 0x55a75baf5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50394/0 (socket says 172.21.15.87:50394) 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 -- 172.21.15.87:0/1451387165 learned_addr learned my addr 172.21.15.87:0/1451387165 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45ff400640 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bb3fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45ffe00640 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 -- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 msgr2=0x55a75bb3fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bb3fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 -- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 msgr2=0x55a75baf5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 -- 172.21.15.87:0/1451387165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a75b8fc9c0 con 0x55a75bb401e0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45fea00640 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb401e0 0x55a75baf5110 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f45ec000c10 tx=0x7f45ec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4da7dc495aca30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45ebe00640 1 -- 172.21.15.87:0/1451387165 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0c8070 con 0x55a75bb401e0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45ebe00640 1 -- 172.21.15.87:0/1451387165 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0a5c00 con 0x55a75bb401e0 2024-04-25T10:23:55.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f45ebe00640 1 -- 172.21.15.87:0/1451387165 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0b2040 con 0x55a75bb401e0 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 -- 172.21.15.87:0/1451387165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb401e0 msgr2=0x55a75baf5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.516+0000 7f46095c2000 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb401e0 0x55a75baf5110 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f45ec000c10 tx=0x7f45ec004740 comp rx=0 tx=0).stop 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- 172.21.15.87:0/1451387165 shutdown_connections 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb401e0 0x55a75baf5110 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 --2- 172.21.15.87:0/1451387165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bb3fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- 172.21.15.87:0/1451387165 >> 172.21.15.87:0/1451387165 conn(0x55a75b9dd870 msgr2=0x55a75baf39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- 172.21.15.87:0/1451387165 shutdown_connections 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- 172.21.15.87:0/1451387165 wait complete. 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- start start 2024-04-25T10:23:55.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 Processor -- start 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75baf9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bafa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb401e0 0x55a75bafa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a75b8fe420 con 0x55a75baf5650 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a75b9c6470 con 0x55a75bb401e0 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a75bafec20 con 0x55a75bb3f8c0 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 10 client.0 _ll_register_callbacks cb 0x55a75bae0f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75baf9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75baf9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50402/0 (socket says 172.21.15.87:50402) 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 -- 172.21.15.87:0/274668122 learned_addr learned my addr 172.21.15.87:0/274668122 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45fea00640 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bafa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ffe00640 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb401e0 0x55a75bafa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 -- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 msgr2=0x55a75bafa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bafa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 -- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb401e0 msgr2=0x55a75bafa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb401e0 0x55a75bafa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 -- 172.21.15.87:0/274668122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a75b8cc260 con 0x55a75baf5650 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45ff400640 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75baf9d50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f45f00830e0 tx=0x7f45f00556b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b89ed86aa909d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45e8c00640 1 -- 172.21.15.87:0/274668122 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f006d020 con 0x55a75baf5650 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45e8c00640 1 -- 172.21.15.87:0/274668122 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f0059350 con 0x55a75baf5650 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45ec0a5c00 con 0x55a75baf5650 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45e8c00640 1 -- 172.21.15.87:0/274668122 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f0055340 con 0x55a75baf5650 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 10 client.10233 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55a75b8feb30 con 0x55a75baf5650 2024-04-25T10:23:55.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45e8c00640 1 -- 172.21.15.87:0/274668122 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0a5c00 con 0x55a75baf5650 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45e8c00640 1 -- 172.21.15.87:0/274668122 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f45f006f050 con 0x55a75baf5650 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 10 client.10233 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 10 client.10233 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 20 client.10233 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bafb570 0x55a75bb50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 4 client.10233 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a75b8dfc80 con 0x55a75bafb570 2024-04-25T10:23:55.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.520+0000 7f45fea00640 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bafb570 0x55a75bb50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f45fea00640 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bafb570 0x55a75bb50960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2f810bd2c4d31c server_cookie=fba00a9721b26a89 in_seq=0 out_seq=0 2024-04-25T10:23:55.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f45e8c00640 10 client.10233 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f45e8c00640 1 -- 172.21.15.87:0/274668122 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a75b8dfc80 con 0x55a75bafb570 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f45e8c00640 10 client.10233 handle_command_reply: tid=0 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 client.10233 shutdown 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 msgr2=0x55a75baf9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75baf9d50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f45f00830e0 tx=0x7f45f00556b0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 shutdown_connections 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb401e0 0x55a75bafa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb3f8c0 0x55a75bafa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bafb570 0x55a75bb50960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 --2- 172.21.15.87:0/274668122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75baf9d50 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 >> 172.21.15.87:0/274668122 conn(0x55a75b9dd870 msgr2=0x55a75baf39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f45e8c00640 0 client.10233 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 shutdown_connections 2024-04-25T10:23:55.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 1 -- 172.21.15.87:0/274668122 wait complete. 2024-04-25T10:23:55.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.524+0000 7f46095c2000 20 client.10233 trim_cache size 0 max 16384 2024-04-25T10:23:55.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 Processor -- start 2024-04-25T10:23:55.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- start start 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb3f8c0 0x55a75baf6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600203950 con 0x55a75bb3f8c0 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600202b90 con 0x55a75baf5650 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460015d2e0 con 0x55a75bb401e0 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb3f8c0 0x55a75baf6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb3f8c0 0x55a75baf6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50408/0 (socket says 172.21.15.87:50408) 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 -- 172.21.15.87:0/2763559700 learned_addr learned my addr 172.21.15.87:0/2763559700 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45ffe00640 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45ff400640 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 -- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 msgr2=0x55a75bb50960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb50960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 -- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 msgr2=0x55a75baf69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 -- 172.21.15.87:0/2763559700 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a75b9c96d0 con 0x55a75bb3f8c0 2024-04-25T10:23:55.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45fea00640 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb3f8c0 0x55a75baf6f10 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f45ec01cf20 tx=0x7f45ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc208681063db273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45eb400640 1 -- 172.21.15.87:0/2763559700 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0c8040 con 0x55a75bb3f8c0 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45eb400640 1 -- 172.21.15.87:0/2763559700 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0ca040 con 0x55a75bb3f8c0 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f45eb400640 1 -- 172.21.15.87:0/2763559700 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0b2040 con 0x55a75bb3f8c0 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- 172.21.15.87:0/2763559700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb3f8c0 msgr2=0x55a75baf6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb3f8c0 0x55a75baf6f10 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f45ec01cf20 tx=0x7f45ec008040 comp rx=0 tx=0).stop 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- 172.21.15.87:0/2763559700 shutdown_connections 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75bb3f8c0 0x55a75baf6f10 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 --2- 172.21.15.87:0/2763559700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75baf5650 0x55a75baf69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- 172.21.15.87:0/2763559700 >> 172.21.15.87:0/2763559700 conn(0x55a75b9dd870 msgr2=0x55a75baf62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- 172.21.15.87:0/2763559700 shutdown_connections 2024-04-25T10:23:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- 172.21.15.87:0/2763559700 wait complete. 2024-04-25T10:23:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 -- start start 2024-04-25T10:23:55.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.540+0000 7f46095c2000 1 Processor -- start 2024-04-25T10:23:55.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75bba0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb3f8c0 0x55a75bba09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a75b9430e0 con 0x55a75baf5650 2024-04-25T10:23:55.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600202790 con 0x55a75bb3f8c0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a75b8eaae0 con 0x55a75bb401e0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 10 client.0 _ll_register_callbacks cb 0x55a75b8e0200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75bba0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75bba0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50424/0 (socket says 172.21.15.87:50424) 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 -- 172.21.15.87:0/407226337 learned_addr learned my addr 172.21.15.87:0/407226337 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45fea00640 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb3f8c0 0x55a75bba09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ffe00640 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 -- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 msgr2=0x55a75bb843b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb843b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 -- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb3f8c0 msgr2=0x55a75bba09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb3f8c0 0x55a75bba09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 -- 172.21.15.87:0/407226337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a75b9c6900 con 0x55a75baf5650 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45ff400640 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75bba0490 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f45f0053820 tx=0x7f45f0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b0d8bc060ef11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:55.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 1 -- 172.21.15.87:0/407226337 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0055570 con 0x55a75baf5650 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 1 -- 172.21.15.87:0/407226337 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f0054430 con 0x55a75baf5650 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45ec0ca040 con 0x55a75baf5650 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 1 -- 172.21.15.87:0/407226337 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f005dbd0 con 0x55a75baf5650 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 10 client.10239 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55a75b8eaf20 con 0x55a75baf5650 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 1 -- 172.21.15.87:0/407226337 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0ca040 con 0x55a75baf5650 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 1 -- 172.21.15.87:0/407226337 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f45f006f070 con 0x55a75baf5650 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 10 client.10239 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 10 client.10239 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 20 client.10239 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bb848f0 0x55a75bb84d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 4 client.10239 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55a75b951310 con 0x55a75bb848f0 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45fea00640 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bb848f0 0x55a75bb84d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45fea00640 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bb848f0 0x55a75bb84d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8458a6879edc5d8 server_cookie=70f7605d3dd36fa6 in_seq=0 out_seq=0 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 10 client.10239 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 1 -- 172.21.15.87:0/407226337 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a75b951310 con 0x55a75bb848f0 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 10 client.10239 handle_command_reply: tid=0 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 client.10239 shutdown 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 msgr2=0x55a75bba0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75bba0490 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f45f0053820 tx=0x7f45f0012000 comp rx=0 tx=0).stop 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 shutdown_connections 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a75bb401e0 0x55a75bb843b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a75bb3f8c0 0x55a75bba09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55a75bb848f0 0x55a75bb84d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 --2- 172.21.15.87:0/407226337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a75baf5650 0x55a75bba0490 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f45e8c00640 0 client.10239 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.544+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 >> 172.21.15.87:0/407226337 conn(0x55a75b9dd870 msgr2=0x55a75baf69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 shutdown_connections 2024-04-25T10:23:55.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f46095c2000 1 -- 172.21.15.87:0/407226337 wait complete. 2024-04-25T10:23:55.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f46095c2000 20 client.10239 trim_cache size 0 max 16384 2024-04-25T10:23:55.550 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:23:55.550 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:23:55.550 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "c8b3516a-3cdb-4f64-80a1-9fa93722c98e", 2024-04-25T10:23:55.550 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:23:55.550 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:23:55.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f45e0066f60 msgr2=0x7f45e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f45e0066f60 0x7f45e0069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0af000 tx=0x7f45ec0046d0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 msgr2=0x7f460014b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f460014b5a0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e780 tx=0x7f45f000ec40 comp rx=0 tx=0).stop 2024-04-25T10:23:55.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 shutdown_connections 2024-04-25T10:23:55.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f45e0066f60 0x7f45e0069420 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600152690 0x7f460014cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4600157180 0x7f4600152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 --2- 172.21.15.87:0/2816282640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45e4004810 0x7f460014b5a0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 >> 172.21.15.87:0/2816282640 conn(0x7f4600066b70 msgr2=0x7f4600159bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 shutdown_connections 2024-04-25T10:23:55.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.548+0000 7f4606200640 1 -- 172.21.15.87:0/2816282640 wait complete. 2024-04-25T10:23:55.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:55.566 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:23:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9032c00640 1 Processor -- start 2024-04-25T10:23:55.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9032c00640 1 -- start start 2024-04-25T10:23:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9032c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 0x7f902c1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9032c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f902c0b45b0 0x7f902c0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9032c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902c0b3dd0 con 0x7f902c0b23c0 2024-04-25T10:23:55.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9032c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902c0d0b90 con 0x7f902c0b45b0 2024-04-25T10:23:55.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9032c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0b40b0 con 0x7f902c0b1c70 2024-04-25T10:23:55.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9030e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f902c0b23c0 0x7f902c161f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54250/0 (socket says 172.21.15.87:54250) 2024-04-25T10:23:55.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.704+0000 7f9030e00640 1 -- 172.21.15.87:0/1684838299 learned_addr learned my addr 172.21.15.87:0/1684838299 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:23:55.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9031800640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 0x7f902c1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9032200640 1 --2- 172.21.15.87:0/1684838299 >> v2:172.21.15.169:3300/0 conn(0x7f902c0b45b0 0x7f902c0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763849274 0 0) 0x7f902c0b3dd0 con 0x7f902c0b23c0 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9014003530 con 0x7f902c0b23c0 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 426710731 0 0) 0x7f9014003530 con 0x7f902c0b23c0 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 msgr2=0x7f902c1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 0x7f902c1619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 >> v2:172.21.15.169:3300/0 conn(0x7f902c0b45b0 msgr2=0x7f902c0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 --2- 172.21.15.87:0/1684838299 >> v2:172.21.15.169:3300/0 conn(0x7f902c0b45b0 0x7f902c0b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0d0820 con 0x7f902c0b23c0 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f9018001680 con 0x7f902c0b23c0 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 >> v1:172.21.15.87:6789/0 conn(0x7f902c0b23c0 legacy=0x7f902c161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90140039c0 0x7f9014003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f9014004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 0x7f90140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0d0b90 con 0x7f9014004810 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0b40b0 con 0x7f90140039c0 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901400ba00 con 0x7f902c0b1c70 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9030e00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f9014004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9031800640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90140039c0 0x7f9014003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9032200640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 0x7f90140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9030e00640 1 -- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 msgr2=0x7f90140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9030e00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0b1c70 0x7f90140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9030e00640 1 -- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90140039c0 msgr2=0x7f9014003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9030e00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90140039c0 0x7f9014003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9030e00640 1 -- 172.21.15.87:0/1684838299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014003530 con 0x7f9014004810 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.708+0000 7f9030e00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f9014004c90 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f901800c190 tx=0x7f901801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55365bef72edcfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018007c90 con 0x7f9014004810 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90180a9020 con 0x7f9014004810 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f902be00640 1 -- 172.21.15.87:0/1684838299 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018005ae0 con 0x7f9014004810 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- 172.21.15.87:0/1684838299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 msgr2=0x7f9014004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 --2- 172.21.15.87:0/1684838299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f9014004c90 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f901800c190 tx=0x7f901801cc00 comp rx=0 tx=0).stop 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032200640 1 -- 172.21.15.87:0/1684838299 reap_dead start 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- 172.21.15.87:0/1684838299 shutdown_connections 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- 172.21.15.87:0/1684838299 >> 172.21.15.87:0/1684838299 conn(0x7f902c0b32b0 msgr2=0x7f902c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- 172.21.15.87:0/1684838299 shutdown_connections 2024-04-25T10:23:55.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- 172.21.15.87:0/1684838299 wait complete. 2024-04-25T10:23:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 Processor -- start 2024-04-25T10:23:55.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- start start 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f902c0c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902c0c6950 0x7f902c152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0c5b10 0x7f902c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0b21d0 con 0x7f9014004810 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0b2050 con 0x7f902c0c6950 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9032c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0b2ad0 con 0x7f902c0c5b10 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9031800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f902c0c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9031800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f902c0c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50452/0 (socket says 172.21.15.87:50452) 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.712+0000 7f9031800640 1 -- 172.21.15.87:0/3211799661 learned_addr learned my addr 172.21.15.87:0/3211799661 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9032200640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0c5b10 0x7f902c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9030e00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902c0c6950 0x7f902c152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9031800640 1 -- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0c5b10 msgr2=0x7f902c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9031800640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0c5b10 0x7f902c0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9031800640 1 -- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902c0c6950 msgr2=0x7f902c152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9031800640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902c0c6950 0x7f902c152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9031800640 1 -- 172.21.15.87:0/3211799661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c160150 con 0x7f9014004810 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9030e00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902c0c6950 0x7f902c152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9031800640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f902c0c5290 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f9020011990 tx=0x7f9020011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9029600640 1 -- 172.21.15.87:0/3211799661 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902000fad0 con 0x7f9014004810 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9029600640 1 -- 172.21.15.87:0/3211799661 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902000fc70 con 0x7f9014004810 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9029600640 1 -- 172.21.15.87:0/3211799661 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020014440 con 0x7f9014004810 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902c160430 con 0x7f9014004810 2024-04-25T10:23:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902c1609f0 con 0x7f9014004810 2024-04-25T10:23:55.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9029600640 1 -- 172.21.15.87:0/3211799661 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9020012040 con 0x7f9014004810 2024-04-25T10:23:55.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9029600640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f900c066f90 0x7f900c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9029600640 1 -- 172.21.15.87:0/3211799661 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9020085450 con 0x7f9014004810 2024-04-25T10:23:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.716+0000 7f9030e00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f900c066f90 0x7f900c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 Processor -- start 2024-04-25T10:23:55.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9030e00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f900c066f90 0x7f900c069450 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f90180aa000 tx=0x7f90180b0040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- start start 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc86cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86d2f0 0x55eadc822b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc8230a0 0x55eadc823480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadc66fff0 con 0x55eadc8230a0 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadc5f8150 con 0x55eadc86c9d0 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eadc66c090 con 0x55eadc86d2f0 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc8230a0 0x55eadc823480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc8230a0 0x55eadc823480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50458/0 (socket says 172.21.15.87:50458) 2024-04-25T10:23:55.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 -- 172.21.15.87:0/326277082 learned_addr learned my addr 172.21.15.87:0/326277082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9031800640 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc86cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9030e00640 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86d2f0 0x55eadc822b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 -- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86d2f0 msgr2=0x55eadc822b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86d2f0 0x55eadc822b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 -- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 msgr2=0x55eadc86cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc86cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 -- 172.21.15.87:0/326277082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eadc8060d0 con 0x55eadc8230a0 2024-04-25T10:23:55.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9032200640 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc8230a0 0x55eadc823480 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f902c0b4990 tx=0x7f9024009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885c0c8e850df7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f901e000640 1 -- 172.21.15.87:0/326277082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024016810 con 0x55eadc8230a0 2024-04-25T10:23:55.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f901e000640 1 -- 172.21.15.87:0/326277082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90240169b0 con 0x55eadc8230a0 2024-04-25T10:23:55.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f901e000640 1 -- 172.21.15.87:0/326277082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024016da0 con 0x55eadc8230a0 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- 172.21.15.87:0/326277082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc8230a0 msgr2=0x55eadc823480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc8230a0 0x55eadc823480 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f902c0b4990 tx=0x7f9024009750 comp rx=0 tx=0).stop 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- 172.21.15.87:0/326277082 shutdown_connections 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc8230a0 0x55eadc823480 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86d2f0 0x55eadc822b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 --2- 172.21.15.87:0/326277082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc86cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- 172.21.15.87:0/326277082 >> 172.21.15.87:0/326277082 conn(0x55eadc81f020 msgr2=0x55eadc821440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- 172.21.15.87:0/326277082 shutdown_connections 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- 172.21.15.87:0/326277082 wait complete. 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.720+0000 7f9037672000 1 -- start start 2024-04-25T10:23:55.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 Processor -- start 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc851d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc852270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadc634250 con 0x55eadc86d2f0 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadc62a690 con 0x55eadc8230a0 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eadc5f8a40 con 0x55eadc86c9d0 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 10 client.0 _ll_register_callbacks cb 0x55eadc80de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc852270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc852270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50472/0 (socket says 172.21.15.87:50472) 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 -- 172.21.15.87:0/3965534908 learned_addr learned my addr 172.21.15.87:0/3965534908 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9030e00640 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc851d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9031800640 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 -- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 msgr2=0x55eadc851d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc851d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 -- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 msgr2=0x55eadc8517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 -- 172.21.15.87:0/3965534908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eadc71d1c0 con 0x55eadc86d2f0 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9032200640 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc852270 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f902400b010 tx=0x7f9024004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f40eb69413d77e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9012a00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9012a00640 1 -- 172.21.15.87:0/3965534908 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024029880 con 0x55eadc86d2f0 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9012a00640 1 -- 172.21.15.87:0/3965534908 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9024029a20 con 0x55eadc86d2f0 2024-04-25T10:23:55.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90240169b0 con 0x55eadc86d2f0 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9012a00640 1 -- 172.21.15.87:0/3965534908 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9024029c20 con 0x55eadc86d2f0 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 10 client.10251 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55eadc719170 con 0x55eadc86d2f0 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9012a00640 1 -- 172.21.15.87:0/3965534908 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90240169b0 con 0x55eadc86d2f0 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9012a00640 1 -- 172.21.15.87:0/3965534908 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f902401e420 con 0x55eadc86d2f0 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 10 client.10251 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 10 client.10251 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 20 client.10251 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87cc80 0x55eadc87f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 4 client.10251 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eadc60bc60 con 0x55eadc87cc80 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9031800640 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87cc80 0x55eadc87f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9031800640 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87cc80 0x55eadc87f070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9f52f277837501 server_cookie=306cd3df2fe1106e in_seq=0 out_seq=0 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.724+0000 7f9012a00640 10 client.10251 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9012a00640 1 -- 172.21.15.87:0/3965534908 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eadc60bc60 con 0x55eadc87cc80 2024-04-25T10:23:55.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9012a00640 10 client.10251 handle_command_reply: tid=0 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 client.10251 shutdown 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 msgr2=0x55eadc852270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc852270 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f902400b010 tx=0x7f9024004a50 comp rx=0 tx=0).stop 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 shutdown_connections 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87cc80 0x55eadc87f070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc852270 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc851d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9012a00640 0 client.10251 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 --2- 172.21.15.87:0/3965534908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 >> 172.21.15.87:0/3965534908 conn(0x55eadc81f020 msgr2=0x55eadc821ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 shutdown_connections 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 1 -- 172.21.15.87:0/3965534908 wait complete. 2024-04-25T10:23:55.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.728+0000 7f9037672000 20 client.10251 trim_cache size 0 max 16384 2024-04-25T10:23:55.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.740+0000 7f9037672000 1 Processor -- start 2024-04-25T10:23:55.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.740+0000 7f9037672000 1 -- start start 2024-04-25T10:23:55.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc8c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc8c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c2003c0 con 0x55eadc86d2f0 2024-04-25T10:23:55.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c1531c0 con 0x55eadc8230a0 2024-04-25T10:23:55.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0b0690 con 0x55eadc86c9d0 2024-04-25T10:23:55.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc8c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc8c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50474/0 (socket says 172.21.15.87:50474) 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 -- 172.21.15.87:0/3319481855 learned_addr learned my addr 172.21.15.87:0/3319481855 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9030e00640 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc8c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9031800640 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 -- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 msgr2=0x55eadc8c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc8c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 -- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 msgr2=0x55eadc8c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 -- 172.21.15.87:0/3319481855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eadc6b6de0 con 0x55eadc86d2f0 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9032200640 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc8c9960 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f9024009790 tx=0x7f9024025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869f79e2057a36f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f901d600640 1 -- 172.21.15.87:0/3319481855 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902401e420 con 0x55eadc86d2f0 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f901d600640 1 -- 172.21.15.87:0/3319481855 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902402f070 con 0x55eadc86d2f0 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f901d600640 1 -- 172.21.15.87:0/3319481855 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90240298a0 con 0x55eadc86d2f0 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- 172.21.15.87:0/3319481855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 msgr2=0x55eadc8c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc8c9960 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f9024009790 tx=0x7f9024025000 comp rx=0 tx=0).stop 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- 172.21.15.87:0/3319481855 shutdown_connections 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc8c9960 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc86c9d0 0x55eadc8c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- 172.21.15.87:0/3319481855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc8230a0 0x55eadc8c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- 172.21.15.87:0/3319481855 >> 172.21.15.87:0/3319481855 conn(0x55eadc81f020 msgr2=0x55eadc879fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- 172.21.15.87:0/3319481855 shutdown_connections 2024-04-25T10:23:55.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- 172.21.15.87:0/3319481855 wait complete. 2024-04-25T10:23:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- start start 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 Processor -- start 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc8230a0 0x55eadc8ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc8cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc87efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadc66f300 con 0x55eadc86d2f0 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadc60ae00 con 0x55eadc86c9d0 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eadc674620 con 0x55eadc8230a0 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.744+0000 7f9037672000 10 client.0 _ll_register_callbacks cb 0x55eadc7ee040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc87efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc87efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50488/0 (socket says 172.21.15.87:50488) 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 -- 172.21.15.87:0/598298213 learned_addr learned my addr 172.21.15.87:0/598298213 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9031800640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc8230a0 0x55eadc8ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9030e00640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc8cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 -- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc8230a0 msgr2=0x55eadc8ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc8230a0 0x55eadc8ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 -- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 msgr2=0x55eadc8cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc8cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 -- 172.21.15.87:0/598298213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eadc6118e0 con 0x55eadc86d2f0 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9031800640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc8230a0 0x55eadc8ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032200640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc87efc0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f9024018010 tx=0x7f902401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3fc28374fae789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 1 -- 172.21.15.87:0/598298213 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024029890 con 0x55eadc86d2f0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f902402f070 con 0x55eadc86d2f0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 1 -- 172.21.15.87:0/598298213 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f902403b440 con 0x55eadc86d2f0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 1 -- 172.21.15.87:0/598298213 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9024051050 con 0x55eadc86d2f0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 10 client.10257 fetch_fsmap learned FSMap version 111 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55eadc6c98f0 con 0x55eadc86d2f0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 1 -- 172.21.15.87:0/598298213 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902402f070 con 0x55eadc86d2f0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 1 -- 172.21.15.87:0/598298213 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9024054020 con 0x55eadc86d2f0 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 10 client.10257 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 10 client.10257 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 20 client.10257 populate_metadata read hostname 'smithi087' 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87f920 0x55eadc87fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 4 client.10257 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:23:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eadc67d530 con 0x55eadc87f920 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9031800640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87f920 0x55eadc87fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 10 client.10257 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9031800640 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87f920 0x55eadc87fd00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9216ab74d44cd037 server_cookie=845134bf4d794f7a in_seq=0 out_seq=0 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 1 -- 172.21.15.87:0/598298213 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x55eadc67d530 con 0x55eadc87f920 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 10 client.10257 handle_command_reply: tid=0 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 client.10257 shutdown 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 msgr2=0x55eadc87efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc87efc0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f9024018010 tx=0x7f902401a000 comp rx=0 tx=0).stop 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 shutdown_connections 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55eadc87f920 0x55eadc87fd00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eadc86d2f0 0x55eadc87efc0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eadc86c9d0 0x55eadc8cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 --2- 172.21.15.87:0/598298213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eadc8230a0 0x55eadc8ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9012a00640 0 client.10257 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:23:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 >> 172.21.15.87:0/598298213 conn(0x55eadc81f020 msgr2=0x55eadc86b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 shutdown_connections 2024-04-25T10:23:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 1 -- 172.21.15.87:0/598298213 wait complete. 2024-04-25T10:23:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9037672000 20 client.10257 trim_cache size 0 max 16384 2024-04-25T10:23:55.755 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: "c8b3516a-3cdb-4f64-80a1-9fa93722c98e": { 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "~mds0", 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "c8b3516a-3cdb-4f64-80a1-9fa93722c98e", 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive" 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:23:55.756 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:23:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f900c066f90 msgr2=0x7f900c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f900c066f90 0x7f900c069450 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f90180aa000 tx=0x7f90180b0040 comp rx=0 tx=0).stop 2024-04-25T10:23:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 msgr2=0x7f902c0c5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f902c0c5290 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f9020011990 tx=0x7f9020011e50 comp rx=0 tx=0).stop 2024-04-25T10:23:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 shutdown_connections 2024-04-25T10:23:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f900c066f90 0x7f900c069450 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f902c0c5b10 0x7f902c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902c0c6950 0x7f902c152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 --2- 172.21.15.87:0/3211799661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014004810 0x7f902c0c5290 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.748+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 >> 172.21.15.87:0/3211799661 conn(0x7f902c0b32b0 msgr2=0x7f902c15cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.752+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 shutdown_connections 2024-04-25T10:23:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:23:55.752+0000 7f9032c00640 1 -- 172.21.15.87:0/3211799661 wait complete. 2024-04-25T10:23:55.768 INFO:tasks.cephfs.filesystem:scrub status for tag:c8b3516a-3cdb-4f64-80a1-9fa93722c98e - {'path': '~mds0', 'tag': 'c8b3516a-3cdb-4f64-80a1-9fa93722c98e', 'options': 'recursive'} 2024-04-25T10:24:24.245 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:24:24.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:24:25.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:25.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:24:25.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab9e00640 1 Processor -- start 2024-04-25T10:24:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab9e00640 1 -- start start 2024-04-25T10:24:26.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab9e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ab40aef80 0x7f8ab415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab4070350 0x7f8ab415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab9e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab415b080 con 0x7f8ab415bac0 2024-04-25T10:24:26.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab9e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab415adb0 con 0x7f8ab40aef80 2024-04-25T10:24:26.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab415eb60 con 0x7f8ab4070350 2024-04-25T10:24:26.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ab40aef80 0x7f8ab415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab4070350 0x7f8ab415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab4070350 0x7f8ab415b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:40158/0 (socket says 172.21.15.87:40158) 2024-04-25T10:24:26.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2000640 1 -- 172.21.15.87:0/796823495 learned_addr learned my addr 172.21.15.87:0/796823495 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2a00640 1 -- 172.21.15.87:0/796823495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab4070350 msgr2=0x7f8ab415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/796823495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab4070350 0x7f8ab415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2a00640 1 -- 172.21.15.87:0/796823495 >> v1:172.21.15.169:6789/0 conn(0x7f8ab415bac0 legacy=0x7f8ab40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.920+0000 7f8ab2a00640 1 -- 172.21.15.87:0/796823495 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab415ee40 con 0x7f8ab40aef80 2024-04-25T10:24:26.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/796823495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 0x7f8ab415e480 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009d90 tx=0x7f8aa401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca63a6537f1f29a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab1600640 1 -- 172.21.15.87:0/796823495 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40b2070 con 0x7f8ab40aef80 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab1600640 1 -- 172.21.15.87:0/796823495 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa40a5a10 con 0x7f8ab40aef80 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab1600640 1 -- 172.21.15.87:0/796823495 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40a5de0 con 0x7f8ab40aef80 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 -- 172.21.15.87:0/796823495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 msgr2=0x7f8ab415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/796823495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 0x7f8ab415e480 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009d90 tx=0x7f8aa401c970 comp rx=0 tx=0).stop 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 -- 172.21.15.87:0/796823495 shutdown_connections 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/796823495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab4070350 0x7f8ab415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/796823495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 0x7f8ab415e480 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 -- 172.21.15.87:0/796823495 >> 172.21.15.87:0/796823495 conn(0x7f8ab40669a0 msgr2=0x7f8ab4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 -- 172.21.15.87:0/796823495 shutdown_connections 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.924+0000 7f8ab9e00640 1 -- 172.21.15.87:0/796823495 wait complete. 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 Processor -- start 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 -- start start 2024-04-25T10:24:26.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ab4070350 0x7f8ab41536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 0x7f8ab40a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab415bac0 0x7f8ab40a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab41679a0 con 0x7f8ab4070350 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4167820 con 0x7f8ab40aef80 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4167b20 con 0x7f8ab415bac0 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ab4070350 0x7f8ab41536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ab4070350 0x7f8ab41536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55084/0 (socket says 172.21.15.87:55084) 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 -- 172.21.15.87:0/4165379568 learned_addr learned my addr 172.21.15.87:0/4165379568 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab3400640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab415bac0 0x7f8ab40a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2000640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 0x7f8ab40a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 -- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab415bac0 msgr2=0x7f8ab40a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab415bac0 0x7f8ab40a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 -- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 msgr2=0x7f8ab40a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 0x7f8ab40a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 -- 172.21.15.87:0/4165379568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab41468e0 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ab4070350 0x7f8ab41536e0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f8aa401b020 tx=0x7f8aa4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8aab400640 1 -- 172.21.15.87:0/4165379568 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40b2040 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8aab400640 1 -- 172.21.15.87:0/4165379568 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa40ad440 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab4146bc0 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8aab400640 1 -- 172.21.15.87:0/4165379568 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40ad7f0 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.928+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab4147180 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8aab400640 1 -- 172.21.15.87:0/4165379568 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8aa40bf030 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8aab400640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a94066f60 0x7f8a94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8aab400640 1 -- 172.21.15.87:0/4165379568 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aa411f4f0 con 0x7f8ab4070350 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8ab2000640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a94066f60 0x7f8a94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 Processor -- start 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 -- start start 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8ab2000640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a94066f60 0x7f8a94069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40672c0 tx=0x7f8aa0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e023db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0dfda480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0de26ff0 con 0x560d0e0242f0 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0ddaf150 con 0x560d0e0239d0 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0de23090 con 0x560d0dfda0a0 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.932+0000 7f8ab2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55088/0 (socket says 172.21.15.87:55088) 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 -- 172.21.15.87:0/3878765818 learned_addr learned my addr 172.21.15.87:0/3878765818 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab3400640 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0dfda480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e023db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 -- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 msgr2=0x560d0dfda480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0dfda480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 -- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 msgr2=0x560d0e023db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e023db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 -- 172.21.15.87:0/3878765818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0dfbd0d0 con 0x560d0e0242f0 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9b60 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0011cf0 tx=0x7f8aa0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd92d01b6d22c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8a9fe00640 1 -- 172.21.15.87:0/3878765818 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0011d20 con 0x560d0e0242f0 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8a9fe00640 1 -- 172.21.15.87:0/3878765818 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0026070 con 0x560d0e0242f0 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8a9fe00640 1 -- 172.21.15.87:0/3878765818 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0021640 con 0x560d0e0242f0 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- 172.21.15.87:0/3878765818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 msgr2=0x560d0dfd9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9b60 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0011cf0 tx=0x7f8aa0014000 comp rx=0 tx=0).stop 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- 172.21.15.87:0/3878765818 shutdown_connections 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0dfda480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9b60 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 --2- 172.21.15.87:0/3878765818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e023db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- 172.21.15.87:0/3878765818 >> 172.21.15.87:0/3878765818 conn(0x560d0dfd6020 msgr2=0x560d0dfd8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- 172.21.15.87:0/3878765818 shutdown_connections 2024-04-25T10:24:26.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- 172.21.15.87:0/3878765818 wait complete. 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- start start 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 Processor -- start 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0e0087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e009270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0ddeb250 con 0x560d0e0242f0 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0dde1690 con 0x560d0e0239d0 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0ddafa40 con 0x560d0dfda0a0 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8abd264000 10 client.0 _ll_register_callbacks cb 0x560d0dfc4e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:26.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e009270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e009270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55092/0 (socket says 172.21.15.87:55092) 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab3400640 1 -- 172.21.15.87:0/3353741846 learned_addr learned my addr 172.21.15.87:0/3353741846 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0e0087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.936+0000 7f8ab2000640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab3400640 1 -- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 msgr2=0x560d0e0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab3400640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0e0087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab3400640 1 -- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 msgr2=0x560d0e008d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab3400640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e008d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab3400640 1 -- 172.21.15.87:0/3353741846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0ded41c0 con 0x560d0e0242f0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab2000640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e008d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab3400640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e009270 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001d50 tx=0x7f8aac011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c11540f5b1c96d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 1 -- 172.21.15.87:0/3353741846 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0050d0 con 0x560d0e0242f0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 1 -- 172.21.15.87:0/3353741846 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aac005270 con 0x560d0e0242f0 2024-04-25T10:24:26.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8aa0026070 con 0x560d0e0242f0 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 1 -- 172.21.15.87:0/3353741846 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aac022050 con 0x560d0e0242f0 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 10 client.10266 fetch_fsmap learned FSMap version 111 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x560d0ded0170 con 0x560d0e0242f0 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 1 -- 172.21.15.87:0/3353741846 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0026070 con 0x560d0e0242f0 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 1 -- 172.21.15.87:0/3353741846 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8aac025020 con 0x560d0e0242f0 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 10 client.10266 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 10 client.10266 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 20 client.10266 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e033c80 0x560d0e036070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 4 client.10266 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d0ddc2c60 con 0x560d0e033c80 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e033c80 0x560d0e036070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 10 client.10266 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e033c80 0x560d0e036070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=728694cc790d6655 server_cookie=34cad37b0cdbd94 in_seq=0 out_seq=0 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 1 -- 172.21.15.87:0/3353741846 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d0ddc2c60 con 0x560d0e033c80 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 10 client.10266 handle_command_reply: tid=0 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 client.10266 shutdown 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 msgr2=0x560d0e009270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e009270 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001d50 tx=0x7f8aac011620 comp rx=0 tx=0).stop 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 shutdown_connections 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e033c80 0x560d0e036070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e009270 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0e0239d0 0x560d0e008d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8a9cc00640 0 client.10266 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 --2- 172.21.15.87:0/3353741846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0dfda0a0 0x560d0e0087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 >> 172.21.15.87:0/3353741846 conn(0x560d0dfd6020 msgr2=0x560d0dfd8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 shutdown_connections 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 1 -- 172.21.15.87:0/3353741846 wait complete. 2024-04-25T10:24:26.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.940+0000 7f8abd264000 20 client.10266 trim_cache size 0 max 16384 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 Processor -- start 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 -- start start 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e022e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0dfd9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab42072d0 con 0x560d0e0242f0 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4148810 con 0x560d0dfda0a0 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4147c10 con 0x560d0e0239d0 2024-04-25T10:24:26.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55102/0 (socket says 172.21.15.87:55102) 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 -- 172.21.15.87:0/1801694858 learned_addr learned my addr 172.21.15.87:0/1801694858 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab2000640 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0dfd9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e022e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 -- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 msgr2=0x560d0dfd9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0dfd9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 -- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 msgr2=0x560d0e022e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e022e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 -- 172.21.15.87:0/1801694858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0de6dde0 con 0x560d0e0242f0 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8ab3400640 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9960 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001710 tx=0x7f8aac00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f4a7ba528f6549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8a9f400640 1 -- 172.21.15.87:0/1801694858 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac011b20 con 0x560d0e0242f0 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8a9f400640 1 -- 172.21.15.87:0/1801694858 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac01a070 con 0x560d0e0242f0 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.956+0000 7f8a9f400640 1 -- 172.21.15.87:0/1801694858 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0050d0 con 0x560d0e0242f0 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/1801694858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 msgr2=0x560d0dfd9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9960 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001710 tx=0x7f8aac00c000 comp rx=0 tx=0).stop 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/1801694858 shutdown_connections 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0dfd9960 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0dfd9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- 172.21.15.87:0/1801694858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e022e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/1801694858 >> 172.21.15.87:0/1801694858 conn(0x560d0dfd6020 msgr2=0x560d0e0220d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/1801694858 shutdown_connections 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/1801694858 wait complete. 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- start start 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 Processor -- start 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e084310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0e084850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e036380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0ddc1e00 con 0x560d0e0242f0 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4148210 con 0x560d0dfda0a0 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0de26300 con 0x560d0e0239d0 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 10 client.0 _ll_register_callbacks cb 0x560d0dfa5040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e036380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e036380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55112/0 (socket says 172.21.15.87:55112) 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 -- 172.21.15.87:0/2357400462 learned_addr learned my addr 172.21.15.87:0/2357400462 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e084310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab2000640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0e084850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 -- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 msgr2=0x560d0e084850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0e084850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 -- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 msgr2=0x560d0e084310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e084310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 -- 172.21.15.87:0/2357400462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0ddc88e0 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e084310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8ab3400640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e036380 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c10 tx=0x7f8aac001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a5771255c5ae77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8a9cc00640 1 -- 172.21.15.87:0/2357400462 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac034030 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8a9cc00640 1 -- 172.21.15.87:0/2357400462 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aac0069f0 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8aac01a070 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8a9cc00640 1 -- 172.21.15.87:0/2357400462 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aac011b20 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 10 client.10272 fetch_fsmap learned FSMap version 111 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x560d0de808f0 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8a9cc00640 1 -- 172.21.15.87:0/2357400462 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac01a070 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8a9cc00640 1 -- 172.21.15.87:0/2357400462 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8aac025070 con 0x560d0e0242f0 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 10 client.10272 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 10 client.10272 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 20 client.10272 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e036c80 0x560d0e037090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 4 client.10272 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.960+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d0de34530 con 0x560d0e036c80 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e036c80 0x560d0e037090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab2a00640 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e036c80 0x560d0e037090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dca1a51e7e4e818 server_cookie=25e6b27502021eff in_seq=0 out_seq=0 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8a9cc00640 10 client.10272 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8a9cc00640 1 -- 172.21.15.87:0/2357400462 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560d0de34530 con 0x560d0e036c80 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8a9cc00640 10 client.10272 handle_command_reply: tid=0 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 client.10272 shutdown 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 msgr2=0x560d0e036380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e036380 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c10 tx=0x7f8aac001090 comp rx=0 tx=0).stop 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 shutdown_connections 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x560d0e036c80 0x560d0e037090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d0e0242f0 0x560d0e036380 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560d0e0239d0 0x560d0e084850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8a9cc00640 0 client.10272 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 --2- 172.21.15.87:0/2357400462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d0dfda0a0 0x560d0e084310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 >> 172.21.15.87:0/2357400462 conn(0x560d0dfd6020 msgr2=0x560d0e022960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 shutdown_connections 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 1 -- 172.21.15.87:0/2357400462 wait complete. 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8abd264000 20 client.10272 trim_cache size 0 max 16384 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:24:26.018 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a94066f60 msgr2=0x7f8a94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a94066f60 0x7f8a94069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40672c0 tx=0x7f8aa0008040 comp rx=0 tx=0).stop 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ab4070350 msgr2=0x7f8ab41536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ab4070350 0x7f8ab41536e0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f8aa401b020 tx=0x7f8aa4019710 comp rx=0 tx=0).stop 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 shutdown_connections 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a94066f60 0x7f8a94069420 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab415bac0 0x7f8ab40a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab40aef80 0x7f8ab40a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 --2- 172.21.15.87:0/4165379568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ab4070350 0x7f8ab41536e0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 >> 172.21.15.87:0/4165379568 conn(0x7f8ab40669a0 msgr2=0x7f8ab41582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 shutdown_connections 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:25.964+0000 7f8ab9e00640 1 -- 172.21.15.87:0/4165379568 wait complete. 2024-04-25T10:24:26.021 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:24:26.021 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:24:26.021 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:24:26.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f2a000640 1 Processor -- start 2024-04-25T10:24:26.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f2a000640 1 -- start start 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f2a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9f2415b640 0x7f9f240b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f2a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f2a000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f240b2d00 con 0x7f9f2415f400 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f2a000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f240b3910 con 0x7f9f2415b640 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f2a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240b2fe0 con 0x7f9f2415ba20 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f28c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9f2415b640 0x7f9f240b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f29600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9f2415f400 0x7f9f24070190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46568/0 (socket says 172.21.15.87:46568) 2024-04-25T10:24:26.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f29600640 1 -- 172.21.15.87:0/2259923636 learned_addr learned my addr 172.21.15.87:0/2259923636 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f23e00640 1 --2- 172.21.15.87:0/2259923636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f28c00640 1 -- 172.21.15.87:0/2259923636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 msgr2=0x7f9f240b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f23400640 1 -- 172.21.15.87:0/2259923636 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779491468 0 0) 0x7f9f240b2d00 con 0x7f9f2415f400 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2259923636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f28c00640 1 -- 172.21.15.87:0/2259923636 >> v1:172.21.15.87:6789/0 conn(0x7f9f2415f400 legacy=0x7f9f24070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f28c00640 1 -- 172.21.15.87:0/2259923636 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240b32c0 con 0x7f9f2415b640 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.160+0000 7f9f23e00640 1 --2- 172.21.15.87:0/2259923636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:26.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2259923636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f240b24d0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f9f180097c0 tx=0x7f9f1801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8eada4ba1d74de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f23400640 1 -- 172.21.15.87:0/2259923636 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180ab650 con 0x7f9f2415b640 2024-04-25T10:24:26.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f23400640 1 -- 172.21.15.87:0/2259923636 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180b0070 con 0x7f9f2415b640 2024-04-25T10:24:26.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f23400640 1 -- 172.21.15.87:0/2259923636 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180aba10 con 0x7f9f2415b640 2024-04-25T10:24:26.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 -- 172.21.15.87:0/2259923636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 msgr2=0x7f9f240b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 --2- 172.21.15.87:0/2259923636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f240b24d0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f9f180097c0 tx=0x7f9f1801c470 comp rx=0 tx=0).stop 2024-04-25T10:24:26.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 -- 172.21.15.87:0/2259923636 shutdown_connections 2024-04-25T10:24:26.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 --2- 172.21.15.87:0/2259923636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 --2- 172.21.15.87:0/2259923636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f240b24d0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 -- 172.21.15.87:0/2259923636 >> 172.21.15.87:0/2259923636 conn(0x7f9f240669a0 msgr2=0x7f9f24066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 -- 172.21.15.87:0/2259923636 shutdown_connections 2024-04-25T10:24:26.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.164+0000 7f9f2a000640 1 -- 172.21.15.87:0/2259923636 wait complete. 2024-04-25T10:24:26.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 Processor -- start 2024-04-25T10:24:26.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 -- start start 2024-04-25T10:24:26.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f24164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2415f400 0x7f9f2406c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f241689b0 con 0x7f9f2415b640 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240b4980 con 0x7f9f2415f400 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24168b30 con 0x7f9f2415ba20 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f24164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f24164960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55118/0 (socket says 172.21.15.87:55118) 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 -- 172.21.15.87:0/974410149 learned_addr learned my addr 172.21.15.87:0/974410149 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f23e00640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f29600640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2415f400 0x7f9f2406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 -- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 msgr2=0x7f9f240b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 -- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2415f400 msgr2=0x7f9f2406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2415f400 0x7f9f2406c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 -- 172.21.15.87:0/974410149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f241651b0 con 0x7f9f2415b640 2024-04-25T10:24:26.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f29600640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2415f400 0x7f9f2406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f28c00640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f24164960 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f9f1801a0a0 tx=0x7f9f1801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f20c00640 1 -- 172.21.15.87:0/974410149 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180bf610 con 0x7f9f2415b640 2024-04-25T10:24:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f20c00640 1 -- 172.21.15.87:0/974410149 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180bf7d0 con 0x7f9f2415b640 2024-04-25T10:24:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f20c00640 1 -- 172.21.15.87:0/974410149 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180bfbc0 con 0x7f9f2415b640 2024-04-25T10:24:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f241654f0 con 0x7f9f2415b640 2024-04-25T10:24:26.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.168+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24164ea0 con 0x7f9f2415b640 2024-04-25T10:24:26.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f20c00640 1 -- 172.21.15.87:0/974410149 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f180bd020 con 0x7f9f2415b640 2024-04-25T10:24:26.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f20c00640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f04066f60 0x7f9f04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f20c00640 1 -- 172.21.15.87:0/974410149 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f1811f630 con 0x7f9f2415b640 2024-04-25T10:24:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f23e00640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f04066f60 0x7f9f04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 Processor -- start 2024-04-25T10:24:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 -- start start 2024-04-25T10:24:26.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f23e00640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f04066f60 0x7f9f04069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f240cf330 tx=0x7f9f10008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee4e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee4eaa0 0x55bd1ee040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1ee049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ec51f20 con 0x55bd1ee4e180 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ebda000 con 0x55bd1ee04600 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ec4dfc0 con 0x55bd1ee4eaa0 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee4e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee4e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55130/0 (socket says 172.21.15.87:55130) 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 -- 172.21.15.87:0/2110271426 learned_addr learned my addr 172.21.15.87:0/2110271426 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f29600640 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1ee049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f23e00640 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee4eaa0 0x55bd1ee040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 -- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee4eaa0 msgr2=0x55bd1ee040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee4eaa0 0x55bd1ee040c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 -- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 msgr2=0x55bd1ee049e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1ee049e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 -- 172.21.15.87:0/2110271426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd1ec2dba0 con 0x55bd1ee4e180 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f23e00640 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee4eaa0 0x55bd1ee040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee4e560 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f9f180ec300 tx=0x7f9f180f05a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f30b88592b97d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.172+0000 7f9f15600640 1 -- 172.21.15.87:0/2110271426 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18103e60 con 0x55bd1ee4e180 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f15600640 1 -- 172.21.15.87:0/2110271426 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180b0030 con 0x55bd1ee4e180 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f15600640 1 -- 172.21.15.87:0/2110271426 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180f3ff0 con 0x55bd1ee4e180 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2110271426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 msgr2=0x55bd1ee4e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee4e560 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f9f180ec300 tx=0x7f9f180f05a0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2110271426 shutdown_connections 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1ee049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee4eaa0 0x55bd1ee040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2110271426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee4e560 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2110271426 >> 172.21.15.87:0/2110271426 conn(0x55bd1ee0a760 msgr2=0x55bd1ee029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2110271426 shutdown_connections 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2110271426 wait complete. 2024-04-25T10:24:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- start start 2024-04-25T10:24:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 Processor -- start 2024-04-25T10:24:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee04600 0x55bd1ee096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee09be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4eaa0 0x55bd1ee3c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ed0f9c0 con 0x55bd1ee4e180 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ec0c5d0 con 0x55bd1ee4eaa0 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ec51f20 con 0x55bd1ee04600 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 10 client.0 _ll_register_callbacks cb 0x55bd1edefa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee09be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee09be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55146/0 (socket says 172.21.15.87:55146) 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 -- 172.21.15.87:0/2060337224 learned_addr learned my addr 172.21.15.87:0/2060337224 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee04600 0x55bd1ee096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f29600640 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4eaa0 0x55bd1ee3c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 -- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee04600 msgr2=0x55bd1ee096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee04600 0x55bd1ee096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 -- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4eaa0 msgr2=0x55bd1ee3c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4eaa0 0x55bd1ee3c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 -- 172.21.15.87:0/2060337224 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd1ec985a0 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f23e00640 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee09be0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f9f10014460 tx=0x7f9f10004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d151e9c46c508780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f0a000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f0a000640 1 -- 172.21.15.87:0/2060337224 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10025020 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f0a000640 1 -- 172.21.15.87:0/2060337224 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f10021830 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.176+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f180b0030 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f0a000640 1 -- 172.21.15.87:0/2060337224 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f1002f050 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 10 client.10287 fetch_fsmap learned FSMap version 111 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55bd1ec292b0 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f0a000640 1 -- 172.21.15.87:0/2060337224 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180b0030 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f0a000640 1 -- 172.21.15.87:0/2060337224 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9f10032020 con 0x55bd1ee4e180 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 10 client.10287 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 10 client.10287 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:24:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 20 client.10287 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee3be60 0x55bd1ee3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 4 client.10287 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd1ebedc70 con 0x55bd1ee3be60 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee3be60 0x55bd1ee3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f28c00640 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee3be60 0x55bd1ee3ce60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7338fc03f770a397 server_cookie=f71f6df48460c9a in_seq=0 out_seq=0 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f0a000640 10 client.10287 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f0a000640 1 -- 172.21.15.87:0/2060337224 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd1ebedc70 con 0x55bd1ee3be60 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f0a000640 10 client.10287 handle_command_reply: tid=0 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 client.10287 shutdown 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 msgr2=0x55bd1ee09be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee09be0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f9f10014460 tx=0x7f9f10004740 comp rx=0 tx=0).stop 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 shutdown_connections 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4eaa0 0x55bd1ee3c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee3be60 0x55bd1ee3ce60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee09be0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/2060337224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee04600 0x55bd1ee096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f0a000640 0 client.10287 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 >> 172.21.15.87:0/2060337224 conn(0x55bd1ee0a760 msgr2=0x55bd1ee030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 shutdown_connections 2024-04-25T10:24:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 1 -- 172.21.15.87:0/2060337224 wait complete. 2024-04-25T10:24:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.180+0000 7f9f2eae0000 20 client.10287 trim_cache size 0 max 16384 2024-04-25T10:24:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 Processor -- start 2024-04-25T10:24:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 -- start start 2024-04-25T10:24:26.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1ee64690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee04600 0x55bd1ee03120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4e180 0x55bd1ee03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24202ea0 con 0x55bd1ee04600 2024-04-25T10:24:26.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24202080 con 0x55bd1ee4e180 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f242014b0 con 0x55bd1ee0a760 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee04600 0x55bd1ee03120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee04600 0x55bd1ee03120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55152/0 (socket says 172.21.15.87:55152) 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 -- 172.21.15.87:0/1415796899 learned_addr learned my addr 172.21.15.87:0/1415796899 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f29600640 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4e180 0x55bd1ee03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f28c00640 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1ee64690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 -- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 msgr2=0x55bd1ee64690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1ee64690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 -- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4e180 msgr2=0x55bd1ee03660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4e180 0x55bd1ee03660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.192+0000 7f9f23e00640 1 -- 172.21.15.87:0/1415796899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd1ee3d1d0 con 0x55bd1ee04600 2024-04-25T10:24:26.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f23e00640 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee04600 0x55bd1ee03120 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000a6f0 tx=0x7f9f10020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d7215fb44646b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f14c00640 1 -- 172.21.15.87:0/1415796899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10038070 con 0x55bd1ee04600 2024-04-25T10:24:26.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f14c00640 1 -- 172.21.15.87:0/1415796899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1001c040 con 0x55bd1ee04600 2024-04-25T10:24:26.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f14c00640 1 -- 172.21.15.87:0/1415796899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10025070 con 0x55bd1ee04600 2024-04-25T10:24:26.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- 172.21.15.87:0/1415796899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee04600 msgr2=0x55bd1ee03120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee04600 0x55bd1ee03120 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000a6f0 tx=0x7f9f10020000 comp rx=0 tx=0).stop 2024-04-25T10:24:26.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- 172.21.15.87:0/1415796899 shutdown_connections 2024-04-25T10:24:26.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee4e180 0x55bd1ee03660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee04600 0x55bd1ee03120 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/1415796899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1ee64690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- 172.21.15.87:0/1415796899 >> 172.21.15.87:0/1415796899 conn(0x55bd1ee3c5b0 msgr2=0x55bd1ee5b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- 172.21.15.87:0/1415796899 shutdown_connections 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- 172.21.15.87:0/1415796899 wait complete. 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- start start 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 Processor -- start 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1eeb6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1eeb74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee62af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24201ab0 con 0x55bd1ee4e180 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ebf8c90 con 0x55bd1ee04600 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd1ec51230 con 0x55bd1ee0a760 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f2eae0000 10 client.0 _ll_register_callbacks cb 0x55bd1edd0600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee62af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee62af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55156/0 (socket says 172.21.15.87:55156) 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 -- 172.21.15.87:0/3693597513 learned_addr learned my addr 172.21.15.87:0/3693597513 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f28c00640 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1eeb6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f23e00640 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1eeb74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 -- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 msgr2=0x55bd1eeb74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1eeb74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 -- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 msgr2=0x55bd1eeb6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1eeb6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 -- 172.21.15.87:0/3693597513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd1ebda250 con 0x55bd1ee4e180 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f29600640 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee62af0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c015e60 tx=0x7f9f1c0073d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da888db30c1fd27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.196+0000 7f9f0a000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 1 -- 172.21.15.87:0/3693597513 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c01f670 con 0x55bd1ee4e180 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 1 -- 172.21.15.87:0/3693597513 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f1c016420 con 0x55bd1ee4e180 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f1001c040 con 0x55bd1ee4e180 2024-04-25T10:24:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 1 -- 172.21.15.87:0/3693597513 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f1c016660 con 0x55bd1ee4e180 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 10 client.10293 fetch_fsmap learned FSMap version 111 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55bd1ebe6b90 con 0x55bd1ee4e180 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 1 -- 172.21.15.87:0/3693597513 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1001c040 con 0x55bd1ee4e180 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 1 -- 172.21.15.87:0/3693597513 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9f1c01f810 con 0x55bd1ee4e180 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 10 client.10293 fetch_fsmap finished waiting for FSMap version 111 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 10 client.10293 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 20 client.10293 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee4eaa0 0x55bd1ee63ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 4 client.10293 mds_command: new command op to 10044 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 --> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd1ec5f460 con 0x55bd1ee4eaa0 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f28c00640 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee4eaa0 0x55bd1ee63ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 10 client.10293 ms_handle_connect on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f28c00640 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee4eaa0 0x55bd1ee63ad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b8a777e7163953 server_cookie=d232903c226fcd71 in_seq=0 out_seq=0 2024-04-25T10:24:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 1 -- 172.21.15.87:0/3693597513 <== mds.0 v2:172.21.15.87:6835/564498676 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd1ec5f460 con 0x55bd1ee4eaa0 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 10 client.10293 handle_command_reply: tid=0 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 client.10293 shutdown 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 msgr2=0x55bd1ee62af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee62af0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c015e60 tx=0x7f9f1c0073d0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 shutdown_connections 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:6835/564498676,v1:172.21.15.87:6837/564498676] conn(0x55bd1ee4eaa0 0x55bd1ee63ad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd1ee4e180 0x55bd1ee62af0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bd1ee0a760 0x55bd1eeb74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f0a000640 0 client.10293 ms_handle_reset on v2:172.21.15.87:6835/564498676 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 --2- 172.21.15.87:0/3693597513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd1ee04600 0x55bd1eeb6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 >> 172.21.15.87:0/3693597513 conn(0x55bd1ee3c5b0 msgr2=0x55bd1ee02110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 shutdown_connections 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 1 -- 172.21.15.87:0/3693597513 wait complete. 2024-04-25T10:24:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2eae0000 20 client.10293 trim_cache size 0 max 16384 2024-04-25T10:24:26.206 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f04066f60 msgr2=0x7f9f04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f04066f60 0x7f9f04069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f240cf330 tx=0x7f9f10008040 comp rx=0 tx=0).stop 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 msgr2=0x7f9f24164960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f24164960 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f9f1801a0a0 tx=0x7f9f1801acb0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 shutdown_connections 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f04066f60 0x7f9f04069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2415f400 0x7f9f2406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2415ba20 0x7f9f240b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 --2- 172.21.15.87:0/974410149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2415b640 0x7f9f24164960 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 >> 172.21.15.87:0/974410149 conn(0x7f9f240669a0 msgr2=0x7f9f240b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 shutdown_connections 2024-04-25T10:24:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.200+0000 7f9f2a000640 1 -- 172.21.15.87:0/974410149 wait complete. 2024-04-25T10:24:26.220 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:24:26.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.356+0000 7f2ec3200640 1 Processor -- start 2024-04-25T10:24:26.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.356+0000 7f2ec3200640 1 -- start start 2024-04-25T10:24:26.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ebc0670b0 0x7f2ebc06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec3200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc06f200 con 0x7f2ebc0afda0 2024-04-25T10:24:26.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc0ccc10 con 0x7f2ebc0af9c0 2024-04-25T10:24:26.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc06d380 con 0x7f2ebc0670b0 2024-04-25T10:24:26.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ebb400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2ebc0afda0 0x7f2ebc0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46574/0 (socket says 172.21.15.87:46574) 2024-04-25T10:24:26.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ebb400640 1 -- 172.21.15.87:0/3385624096 learned_addr learned my addr 172.21.15.87:0/3385624096 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:26.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec1200640 1 --2- 172.21.15.87:0/3385624096 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc0670b0 0x7f2ebc06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ebaa00640 1 -- 172.21.15.87:0/3385624096 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634383326 0 0) 0x7f2ea8001120 con 0x7f2ebc0afda0 2024-04-25T10:24:26.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ebaa00640 1 -- 172.21.15.87:0/3385624096 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea0003530 con 0x7f2ebc0afda0 2024-04-25T10:24:26.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec1200640 1 -- 172.21.15.87:0/3385624096 >> v1:172.21.15.169:6789/0 conn(0x7f2ebc0af9c0 legacy=0x7f2ebc0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec1200640 1 -- 172.21.15.87:0/3385624096 >> v1:172.21.15.87:6789/0 conn(0x7f2ebc0afda0 legacy=0x7f2ebc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.360+0000 7f2ec1200640 1 -- 172.21.15.87:0/3385624096 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0ccef0 con 0x7f2ebc0670b0 2024-04-25T10:24:26.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec1200640 1 --2- 172.21.15.87:0/3385624096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 0x7f2ebc06ddd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d90 tx=0x7f2eb401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc3e44a447381c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ebaa00640 1 -- 172.21.15.87:0/3385624096 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40b2070 con 0x7f2ebc0670b0 2024-04-25T10:24:26.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ebaa00640 1 -- 172.21.15.87:0/3385624096 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb40a5970 con 0x7f2ebc0670b0 2024-04-25T10:24:26.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ebaa00640 1 -- 172.21.15.87:0/3385624096 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40a5d20 con 0x7f2ebc0670b0 2024-04-25T10:24:26.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 -- 172.21.15.87:0/3385624096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 msgr2=0x7f2ebc06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3385624096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 0x7f2ebc06ddd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d90 tx=0x7f2eb401c8b0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 -- 172.21.15.87:0/3385624096 shutdown_connections 2024-04-25T10:24:26.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3385624096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 0x7f2ebc06ddd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 -- 172.21.15.87:0/3385624096 >> 172.21.15.87:0/3385624096 conn(0x7f2ebc0b2410 msgr2=0x7f2ebc0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 -- 172.21.15.87:0/3385624096 shutdown_connections 2024-04-25T10:24:26.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 -- 172.21.15.87:0/3385624096 wait complete. 2024-04-25T10:24:26.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 Processor -- start 2024-04-25T10:24:26.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.364+0000 7f2ec3200640 1 -- start start 2024-04-25T10:24:26.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 0x7f2ebc0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0af9c0 0x7f2ebc155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ebc0afda0 0x7f2ebc15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc160460 con 0x7f2ebc0afda0 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc1602e0 con 0x7f2ebc0670b0 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc0b4870 con 0x7f2ebc0af9c0 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ebc0afda0 0x7f2ebc15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ebc0afda0 0x7f2ebc15bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55170/0 (socket says 172.21.15.87:55170) 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 -- 172.21.15.87:0/3436211887 learned_addr learned my addr 172.21.15.87:0/3436211887 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ebb400640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0af9c0 0x7f2ebc155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ebbe00640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 0x7f2ebc0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 -- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0af9c0 msgr2=0x7f2ebc155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0af9c0 0x7f2ebc155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 -- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 msgr2=0x7f2ebc0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 0x7f2ebc0b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 -- 172.21.15.87:0/3436211887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0b4b50 con 0x7f2ebc0afda0 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec1200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ebc0afda0 0x7f2ebc15bf80 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d90 tx=0x7f2eb4019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2eb3e00640 1 -- 172.21.15.87:0/3436211887 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40b2050 con 0x7f2ebc0afda0 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2eb3e00640 1 -- 172.21.15.87:0/3436211887 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb40b3040 con 0x7f2ebc0afda0 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2eb3e00640 1 -- 172.21.15.87:0/3436211887 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40ad850 con 0x7f2ebc0afda0 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc159a80 con 0x7f2ebc0afda0 2024-04-25T10:24:26.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc0b4560 con 0x7f2ebc0afda0 2024-04-25T10:24:26.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc04ef30 con 0x7f2ebc0afda0 2024-04-25T10:24:26.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.368+0000 7f2eb3e00640 1 -- 172.21.15.87:0/3436211887 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eb40c0020 con 0x7f2ebc0afda0 2024-04-25T10:24:26.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.372+0000 7f2eb3e00640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e9c066f60 0x7f2e9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:26.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.372+0000 7f2eb3e00640 1 -- 172.21.15.87:0/3436211887 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2eb411faf0 con 0x7f2ebc0afda0 2024-04-25T10:24:26.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.372+0000 7f2ebbe00640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e9c066f60 0x7f2e9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:26.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.372+0000 7f2ebbe00640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e9c066f60 0x7f2e9c069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2eac001690 tx=0x7f2eac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:26.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.376+0000 7f2eb3e00640 1 -- 172.21.15.87:0/3436211887 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2eb4016560 con 0x7f2ebc0afda0 2024-04-25T10:24:26.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2ebc071420 con 0x7f2ebc0afda0 2024-04-25T10:24:26.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2eb3e00640 1 -- 172.21.15.87:0/3436211887 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2eb40f0c10 con 0x7f2ebc0afda0 2024-04-25T10:24:26.563 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:26.563 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:24:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e9c066f60 msgr2=0x7f2e9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e9c066f60 0x7f2e9c069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2eac001690 tx=0x7f2eac009040 comp rx=0 tx=0).stop 2024-04-25T10:24:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ebc0afda0 msgr2=0x7f2ebc15bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ebc0afda0 0x7f2ebc15bf80 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d90 tx=0x7f2eb4019650 comp rx=0 tx=0).stop 2024-04-25T10:24:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 shutdown_connections 2024-04-25T10:24:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e9c066f60 0x7f2e9c069420 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ebc0afda0 0x7f2ebc15bf80 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0af9c0 0x7f2ebc155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 --2- 172.21.15.87:0/3436211887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0670b0 0x7f2ebc0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 >> 172.21.15.87:0/3436211887 conn(0x7f2ebc0b2410 msgr2=0x7f2ebc0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 shutdown_connections 2024-04-25T10:24:26.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:26.564+0000 7f2ec3200640 1 -- 172.21.15.87:0/3436211887 wait complete. 2024-04-25T10:24:26.584 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:24:26.585 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:26.688 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:26.785 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:24:26.785 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:24:26.785 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:26.794 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:24:26.794 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:26.845 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:27.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c10600640 1 Processor -- start 2024-04-25T10:24:27.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c10600640 1 -- start start 2024-04-25T10:24:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c10600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c080ccb20 0x7f6c080d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c10600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c0815bdc0 0x7f6c080d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c10600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c080ccfb0 con 0x7f6c0815b9e0 2024-04-25T10:24:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c10600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c080cd290 con 0x7f6c0815bdc0 2024-04-25T10:24:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c10600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080cd570 con 0x7f6c080ccb20 2024-04-25T10:24:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6c0815b9e0 0x7f6c080cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46580/0 (socket says 172.21.15.87:46580) 2024-04-25T10:24:27.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 -- 172.21.15.87:0/2900363416 learned_addr learned my addr 172.21.15.87:0/2900363416 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384564976 0 0) 0x7f6c080ccfb0 con 0x7f6c0815b9e0 2024-04-25T10:24:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf0003530 con 0x7f6c0815b9e0 2024-04-25T10:24:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0d200640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c080ccb20 0x7f6c080d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0e600640 1 --2- 172.21.15.87:0/2900363416 >> v2:172.21.15.169:3300/0 conn(0x7f6c0815bdc0 0x7f6c080d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3274073890 0 0) 0x7f6bf0003530 con 0x7f6c0815b9e0 2024-04-25T10:24:27.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c080ccb20 msgr2=0x7f6c080d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c080ccb20 0x7f6c080d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 >> v2:172.21.15.169:3300/0 conn(0x7f6c0815bdc0 msgr2=0x7f6c080d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 --2- 172.21.15.87:0/2900363416 >> v2:172.21.15.169:3300/0 conn(0x7f6c0815bdc0 0x7f6c080d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c080cd850 con 0x7f6c0815b9e0 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f6bfc001a00 con 0x7f6c0815b9e0 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 >> v1:172.21.15.87:6789/0 conn(0x7f6c0815b9e0 legacy=0x7f6c080cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf00039c0 0x7f6bf0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf0004810 0x7f6bf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00051d0 0x7f6bf000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080cd290 con 0x7f6bf00039c0 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080cd570 con 0x7f6bf0004810 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf000bf10 con 0x7f6bf00051d0 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf00039c0 0x7f6bf0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0d200640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf0004810 0x7f6bf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0e600640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00051d0 0x7f6bf000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 -- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00051d0 msgr2=0x7f6bf000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00051d0 0x7f6bf000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 -- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf0004810 msgr2=0x7f6bf0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf0004810 0x7f6bf0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.032+0000 7f6c0dc00640 1 -- 172.21.15.87:0/2900363416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf0003530 con 0x7f6bf00039c0 2024-04-25T10:24:27.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c0dc00640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf00039c0 0x7f6bf0003de0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00a8f0 tx=0x7f6bfc01d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7216d358a92522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00a4c0 con 0x7f6bf00039c0 2024-04-25T10:24:27.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc0ad650 con 0x7f6bf00039c0 2024-04-25T10:24:27.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c07e00640 1 -- 172.21.15.87:0/2900363416 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0ada40 con 0x7f6bf00039c0 2024-04-25T10:24:27.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c10600640 1 -- 172.21.15.87:0/2900363416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf00039c0 msgr2=0x7f6bf0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c10600640 1 --2- 172.21.15.87:0/2900363416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf00039c0 0x7f6bf0003de0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00a8f0 tx=0x7f6bfc01d400 comp rx=0 tx=0).stop 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c0e600640 1 -- 172.21.15.87:0/2900363416 reap_dead start 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c10600640 1 -- 172.21.15.87:0/2900363416 shutdown_connections 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c10600640 1 -- 172.21.15.87:0/2900363416 >> 172.21.15.87:0/2900363416 conn(0x7f6c0809a9d0 msgr2=0x7f6c0809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c10600640 1 -- 172.21.15.87:0/2900363416 shutdown_connections 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.036+0000 7f6c10600640 1 -- 172.21.15.87:0/2900363416 wait complete. 2024-04-25T10:24:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 Processor -- start 2024-04-25T10:24:27.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 -- start start 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00039c0 0x7f6c0806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0806c9c0 0x7f6c0806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c0806b380 0x7f6c080c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080c65b0 con 0x7f6c0806c9c0 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080c6430 con 0x7f6bf00039c0 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080c6730 con 0x7f6c0806b380 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0806c9c0 0x7f6c0806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0806c9c0 0x7f6c0806f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55182/0 (socket says 172.21.15.87:55182) 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 -- 172.21.15.87:0/657065028 learned_addr learned my addr 172.21.15.87:0/657065028 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0dc00640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00039c0 0x7f6c0806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0e600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c0806b380 0x7f6c080c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 -- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c0806b380 msgr2=0x7f6c080c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c0806b380 0x7f6c080c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 -- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00039c0 msgr2=0x7f6c0806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00039c0 0x7f6c0806f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 -- 172.21.15.87:0/657065028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0806b920 con 0x7f6c0806c9c0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c0d200640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0806c9c0 0x7f6c0806f9b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4011990 tx=0x7f6bf4011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c05600640 1 -- 172.21.15.87:0/657065028 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400fad0 con 0x7f6c0806c9c0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c05600640 1 -- 172.21.15.87:0/657065028 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf400fc70 con 0x7f6c0806c9c0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0806bc00 con 0x7f6c0806c9c0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c05600640 1 -- 172.21.15.87:0/657065028 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4014440 con 0x7f6c0806c9c0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.040+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0806c1c0 con 0x7f6c0806c9c0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.044+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0804ef30 con 0x7f6c0806c9c0 2024-04-25T10:24:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.044+0000 7f6c05600640 1 -- 172.21.15.87:0/657065028 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bf4012040 con 0x7f6c0806c9c0 2024-04-25T10:24:27.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.044+0000 7f6c05600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6be8066f90 0x7f6be8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.044+0000 7f6c05600640 1 -- 172.21.15.87:0/657065028 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bf4085530 con 0x7f6c0806c9c0 2024-04-25T10:24:27.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.044+0000 7f6c0dc00640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6be8066f90 0x7f6be8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.044+0000 7f6c0dc00640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6be8066f90 0x7f6be8069450 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0ad000 tx=0x7f6bfc001dd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.048+0000 7f6c05600640 1 -- 172.21.15.87:0/657065028 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6bf4019030 con 0x7f6c0806c9c0 2024-04-25T10:24:27.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c0806d4d0 con 0x7f6c0806c9c0 2024-04-25T10:24:27.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c05600640 1 -- 172.21.15.87:0/657065028 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16659 (secure 0 0 0) 0x7f6bf4055ef0 con 0x7f6c0806c9c0 2024-04-25T10:24:27.204 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:27.204 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":151,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:48.878767+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:23:45.609544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:23:46.223978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:27.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6be8066f90 msgr2=0x7f6be8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6be8066f90 0x7f6be8069450 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0ad000 tx=0x7f6bfc001dd0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0806c9c0 msgr2=0x7f6c0806f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0806c9c0 0x7f6c0806f9b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4011990 tx=0x7f6bf4011e50 comp rx=0 tx=0).stop 2024-04-25T10:24:27.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 shutdown_connections 2024-04-25T10:24:27.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6be8066f90 0x7f6be8069450 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c0806b380 0x7f6c080c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.204+0000 7f6c10600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0806c9c0 0x7f6c0806f9b0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.208+0000 7f6c10600640 1 --2- 172.21.15.87:0/657065028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00039c0 0x7f6c0806f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.208+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 >> 172.21.15.87:0/657065028 conn(0x7f6c0809a9d0 msgr2=0x7f6c0815b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:27.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.208+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 shutdown_connections 2024-04-25T10:24:27.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.208+0000 7f6c10600640 1 -- 172.21.15.87:0/657065028 wait complete. 2024-04-25T10:24:27.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:23:45.609544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:23:46.223978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:27.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.364+0000 7fc22be00640 1 Processor -- start 2024-04-25T10:24:27.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.364+0000 7fc22be00640 1 -- start start 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc22c15a780 0x7fc22c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc22c0cff00 0x7fc22c0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c06de60 con 0x7fc22c0cf3b0 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c15f2f0 con 0x7fc22c15a780 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c06cdc0 con 0x7fc22c0cff00 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc22c15a780 0x7fc22c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc22c15a780 0x7fc22c15edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41004/0 (socket says 172.21.15.87:41004) 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22aa00640 1 -- 172.21.15.87:0/1089431025 learned_addr learned my addr 172.21.15.87:0/1089431025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22b400640 1 --2- 172.21.15.87:0/1089431025 >> v2:172.21.15.169:3300/0 conn(0x7fc22c0cff00 0x7fc22c0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22aa00640 1 -- 172.21.15.87:0/1089431025 >> v2:172.21.15.169:3300/0 conn(0x7fc22c0cff00 msgr2=0x7fc22c0b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22aa00640 1 --2- 172.21.15.87:0/1089431025 >> v2:172.21.15.169:3300/0 conn(0x7fc22c0cff00 0x7fc22c0b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22aa00640 1 -- 172.21.15.87:0/1089431025 >> v1:172.21.15.169:6789/0 conn(0x7fc22c0cf3b0 legacy=0x7fc22c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.368+0000 7fc22aa00640 1 -- 172.21.15.87:0/1089431025 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c06d0a0 con 0x7fc22c15a780 2024-04-25T10:24:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22aa00640 1 --2- 172.21.15.87:0/1089431025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c15a780 0x7fc22c15edb0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0cc220 tx=0x7fc21c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ca7877b02eb465 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22a000640 1 -- 172.21.15.87:0/1089431025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0b0070 con 0x7fc22c15a780 2024-04-25T10:24:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22a000640 1 -- 172.21.15.87:0/1089431025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c0ab600 con 0x7fc22c15a780 2024-04-25T10:24:27.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22a000640 1 -- 172.21.15.87:0/1089431025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0ab9d0 con 0x7fc22c15a780 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- 172.21.15.87:0/1089431025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c15a780 msgr2=0x7fc22c15edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 --2- 172.21.15.87:0/1089431025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c15a780 0x7fc22c15edb0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0cc220 tx=0x7fc21c01c520 comp rx=0 tx=0).stop 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- 172.21.15.87:0/1089431025 shutdown_connections 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 --2- 172.21.15.87:0/1089431025 >> v2:172.21.15.169:3300/0 conn(0x7fc22c0cff00 0x7fc22c0b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 --2- 172.21.15.87:0/1089431025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c15a780 0x7fc22c15edb0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- 172.21.15.87:0/1089431025 >> 172.21.15.87:0/1089431025 conn(0x7fc22c0743f0 msgr2=0x7fc22c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- 172.21.15.87:0/1089431025 shutdown_connections 2024-04-25T10:24:27.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- 172.21.15.87:0/1089431025 wait complete. 2024-04-25T10:24:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 Processor -- start 2024-04-25T10:24:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- start start 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c0cf3b0 0x7fc22c1393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c0cff00 0x7fc22c14a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc22c15a780 0x7fc22c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b3330 con 0x7fc22c0cff00 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b31b0 con 0x7fc22c0cf3b0 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc22be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b34b0 con 0x7fc22c15a780 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc222000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c0cff00 0x7fc22c14a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc222000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c0cff00 0x7fc22c14a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41012/0 (socket says 172.21.15.87:41012) 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.372+0000 7fc222000640 1 -- 172.21.15.87:0/687861497 learned_addr learned my addr 172.21.15.87:0/687861497 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc22aa00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c0cf3b0 0x7fc22c1393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc22b400640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc22c15a780 0x7fc22c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc222000640 1 -- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc22c15a780 msgr2=0x7fc22c0aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc222000640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc22c15a780 0x7fc22c0aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc222000640 1 -- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c0cf3b0 msgr2=0x7fc22c1393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc222000640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c0cf3b0 0x7fc22c1393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc222000640 1 -- 172.21.15.87:0/687861497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c139910 con 0x7fc22c0cff00 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc222000640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c0cff00 0x7fc22c14a7e0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fc21400c3a0 tx=0x7fc21400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc223400640 1 -- 172.21.15.87:0/687861497 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140052d0 con 0x7fc22c0cff00 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc223400640 1 -- 172.21.15.87:0/687861497 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214005470 con 0x7fc22c0cff00 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c14d660 con 0x7fc22c0cff00 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc223400640 1 -- 172.21.15.87:0/687861497 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214007260 con 0x7fc22c0cff00 2024-04-25T10:24:27.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c14dc20 con 0x7fc22c0cff00 2024-04-25T10:24:27.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c075120 con 0x7fc22c0cff00 2024-04-25T10:24:27.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc223400640 1 -- 172.21.15.87:0/687861497 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc214007400 con 0x7fc22c0cff00 2024-04-25T10:24:27.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc223400640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc208066f60 0x7fc208069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.376+0000 7fc223400640 1 -- 172.21.15.87:0/687861497 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2140844b0 con 0x7fc22c0cff00 2024-04-25T10:24:27.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.380+0000 7fc22aa00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc208066f60 0x7fc208069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.380+0000 7fc22aa00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc208066f60 0x7fc208069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0ad000 tx=0x7fc21c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.384+0000 7fc223400640 1 -- 172.21.15.87:0/687861497 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc22c139910 con 0x7fc22c0cff00 2024-04-25T10:24:27.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc22c070c40 con 0x7fc22c0cff00 2024-04-25T10:24:27.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc223400640 1 -- 172.21.15.87:0/687861497 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc214054dd0 con 0x7fc22c0cff00 2024-04-25T10:24:27.538 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:24:27.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc208066f60 msgr2=0x7fc208069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc208066f60 0x7fc208069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0ad000 tx=0x7fc21c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:27.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c0cff00 msgr2=0x7fc22c14a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c0cff00 0x7fc22c14a7e0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fc21400c3a0 tx=0x7fc21400c860 comp rx=0 tx=0).stop 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 shutdown_connections 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc208066f60 0x7fc208069420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc22c15a780 0x7fc22c0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc22c0cff00 0x7fc22c14a7e0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 --2- 172.21.15.87:0/687861497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c0cf3b0 0x7fc22c1393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 >> 172.21.15.87:0/687861497 conn(0x7fc22c0743f0 msgr2=0x7fc22c15eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 shutdown_connections 2024-04-25T10:24:27.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.540+0000 7fc22be00640 1 -- 172.21.15.87:0/687861497 wait complete. 2024-04-25T10:24:27.562 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:24:27.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.700+0000 7fc31bc00640 1 Processor -- start 2024-04-25T10:24:27.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.700+0000 7fc31bc00640 1 -- start start 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc31bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 0x7fc31415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc31bc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31415b080 con 0x7fc3140aef80 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc31bc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31415adb0 con 0x7fc3140b31c0 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc31bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31415eb60 con 0x7fc314070350 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc3140aef80 0x7fc31415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41266/0 (socket says 172.21.15.87:41266) 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319200640 1 -- 172.21.15.87:0/507621203 learned_addr learned my addr 172.21.15.87:0/507621203 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc30be00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 0x7fc31415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897622996 0 0) 0x7fc31415b080 con 0x7fc3140aef80 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f8003530 con 0x7fc3140aef80 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377378705 0 0) 0x7fc2f8003530 con 0x7fc3140aef80 2024-04-25T10:24:27.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 msgr2=0x7fc31415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 0x7fc31415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 >> v1:172.21.15.169:6789/0 conn(0x7fc3140b31c0 legacy=0x7fc3140cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31415ee40 con 0x7fc3140aef80 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc304001680 con 0x7fc3140aef80 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 >> v1:172.21.15.87:6789/0 conn(0x7fc3140aef80 legacy=0x7fc31415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f80039f0 0x7fc2f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 0x7fc2f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2f8004d50 0x7fc2f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31415eb60 con 0x7fc2f8004d50 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f800ba80 con 0x7fc2f80039f0 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f800bc80 con 0x7fc314070350 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319c00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2f8004d50 0x7fc2f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319200640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f80039f0 0x7fc2f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc30be00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 0x7fc2f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319c00640 1 -- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 msgr2=0x7fc2f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319c00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc314070350 0x7fc2f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319c00640 1 -- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f80039f0 msgr2=0x7fc2f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319c00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2f80039f0 0x7fc2f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.704+0000 7fc319c00640 1 -- 172.21.15.87:0/507621203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8003530 con 0x7fc2f8004d50 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc319c00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2f8004d50 0x7fc2f80051d0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fc30c001e40 tx=0x7fc30c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8aee404c163de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c001a90 con 0x7fc2f8004d50 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c003b80 con 0x7fc2f8004d50 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc313e00640 1 -- 172.21.15.87:0/507621203 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c005080 con 0x7fc2f8004d50 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 -- 172.21.15.87:0/507621203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2f8004d50 msgr2=0x7fc2f80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 --2- 172.21.15.87:0/507621203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2f8004d50 0x7fc2f80051d0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fc30c001e40 tx=0x7fc30c01c760 comp rx=0 tx=0).stop 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc319c00640 1 -- 172.21.15.87:0/507621203 reap_dead start 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 -- 172.21.15.87:0/507621203 shutdown_connections 2024-04-25T10:24:27.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 -- 172.21.15.87:0/507621203 >> 172.21.15.87:0/507621203 conn(0x7fc3140669a0 msgr2=0x7fc314066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:27.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 -- 172.21.15.87:0/507621203 shutdown_connections 2024-04-25T10:24:27.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 -- 172.21.15.87:0/507621203 wait complete. 2024-04-25T10:24:27.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 Processor -- start 2024-04-25T10:24:27.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.708+0000 7fc31bc00640 1 -- start start 2024-04-25T10:24:27.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3141f1000 0x7fc314158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3141f13e0 0x7fc314159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3141f4e40 0x7fc3141f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31415bc90 con 0x7fc3141f13e0 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc314071a00 con 0x7fc3141f1000 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140cfd70 con 0x7fc3141f4e40 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3141f13e0 0x7fc314159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3141f13e0 0x7fc314159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41038/0 (socket says 172.21.15.87:41038) 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 -- 172.21.15.87:0/1911775954 learned_addr learned my addr 172.21.15.87:0/1911775954 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc319200640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3141f1000 0x7fc314158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc319c00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3141f4e40 0x7fc3141f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 -- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3141f4e40 msgr2=0x7fc3141f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3141f4e40 0x7fc3141f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 -- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3141f1000 msgr2=0x7fc314158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3141f1000 0x7fc314158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 -- 172.21.15.87:0/1911775954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3141f5b70 con 0x7fc3141f13e0 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc30be00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3141f13e0 0x7fc314159200 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0119d0 tx=0x7fc2fc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc311600640 1 -- 172.21.15.87:0/1911775954 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00fb50 con 0x7fc3141f13e0 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc311600640 1 -- 172.21.15.87:0/1911775954 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc00fcf0 con 0x7fc3141f13e0 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc311600640 1 -- 172.21.15.87:0/1911775954 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00fea0 con 0x7fc3141f13e0 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc314147fb0 con 0x7fc3141f13e0 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc314148570 con 0x7fc3141f13e0 2024-04-25T10:24:27.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31404ef30 con 0x7fc3141f13e0 2024-04-25T10:24:27.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.712+0000 7fc311600640 1 -- 172.21.15.87:0/1911775954 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2fc012040 con 0x7fc3141f13e0 2024-04-25T10:24:27.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.716+0000 7fc311600640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2f0066f60 0x7fc2f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:27.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.716+0000 7fc311600640 1 -- 172.21.15.87:0/1911775954 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2fc0856f0 con 0x7fc3141f13e0 2024-04-25T10:24:27.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.716+0000 7fc319200640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2f0066f60 0x7fc2f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:27.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.716+0000 7fc319200640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2f0066f60 0x7fc2f0069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc304004850 tx=0x7fc304009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:27.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.720+0000 7fc311600640 1 -- 172.21.15.87:0/1911775954 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2fc019030 con 0x7fc3141f13e0 2024-04-25T10:24:27.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.872+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3140b2fa0 con 0x7fc3141f13e0 2024-04-25T10:24:27.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.872+0000 7fc311600640 1 -- 172.21.15.87:0/1911775954 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2fc0567b0 con 0x7fc3141f13e0 2024-04-25T10:24:27.872 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:24:27.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2f0066f60 msgr2=0x7fc2f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2f0066f60 0x7fc2f0069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc304004850 tx=0x7fc304009040 comp rx=0 tx=0).stop 2024-04-25T10:24:27.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3141f13e0 msgr2=0x7fc314159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:27.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3141f13e0 0x7fc314159200 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0119d0 tx=0x7fc2fc011e90 comp rx=0 tx=0).stop 2024-04-25T10:24:27.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 shutdown_connections 2024-04-25T10:24:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc2f0066f60 0x7fc2f0069420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3141f4e40 0x7fc3141f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3141f13e0 0x7fc314159200 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 --2- 172.21.15.87:0/1911775954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3141f1000 0x7fc314158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 >> 172.21.15.87:0/1911775954 conn(0x7fc3140669a0 msgr2=0x7fc314066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 shutdown_connections 2024-04-25T10:24:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:27.876+0000 7fc31bc00640 1 -- 172.21.15.87:0/1911775954 wait complete. 2024-04-25T10:24:27.895 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:24:28.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814d800640 1 Processor -- start 2024-04-25T10:24:28.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814d800640 1 -- start start 2024-04-25T10:24:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814d800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f81480b1f50 0x7f81480b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814d800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814806be90 con 0x7f81480cd780 2024-04-25T10:24:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814d800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814806b8d0 con 0x7f81480b3dd0 2024-04-25T10:24:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814d800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81480d0b50 con 0x7f81480b1f50 2024-04-25T10:24:28.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814ce00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f81480cd780 0x7f81480cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41274/0 (socket says 172.21.15.87:41274) 2024-04-25T10:24:28.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f814ce00640 1 -- 172.21.15.87:0/416193308 learned_addr learned my addr 172.21.15.87:0/416193308 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:28.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f8147e00640 1 --2- 172.21.15.87:0/416193308 >> v2:172.21.15.169:3300/0 conn(0x7f81480b1f50 0x7f81480b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f8147400640 1 -- 172.21.15.87:0/416193308 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024044547 0 0) 0x7f8140001120 con 0x7f81480cd780 2024-04-25T10:24:28.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f8147400640 1 -- 172.21.15.87:0/416193308 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f812c003530 con 0x7f81480cd780 2024-04-25T10:24:28.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f8147e00640 1 -- 172.21.15.87:0/416193308 >> v1:172.21.15.169:6789/0 conn(0x7f81480b3dd0 legacy=0x7f814806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f8147e00640 1 -- 172.21.15.87:0/416193308 >> v1:172.21.15.87:6789/0 conn(0x7f81480cd780 legacy=0x7f81480cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.036+0000 7f8147e00640 1 -- 172.21.15.87:0/416193308 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814806bbb0 con 0x7f81480b1f50 2024-04-25T10:24:28.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f8147e00640 1 --2- 172.21.15.87:0/416193308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b1f50 0x7f81480b23c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f81380097c0 tx=0x7f813801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff820363e354ff21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f8147400640 1 -- 172.21.15.87:0/416193308 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380b0070 con 0x7f81480b1f50 2024-04-25T10:24:28.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f8147400640 1 -- 172.21.15.87:0/416193308 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81380ab600 con 0x7f81480b1f50 2024-04-25T10:24:28.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f8147400640 1 -- 172.21.15.87:0/416193308 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380ab9d0 con 0x7f81480b1f50 2024-04-25T10:24:28.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f814d800640 1 -- 172.21.15.87:0/416193308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b1f50 msgr2=0x7f81480b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f814d800640 1 --2- 172.21.15.87:0/416193308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b1f50 0x7f81480b23c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f81380097c0 tx=0x7f813801c380 comp rx=0 tx=0).stop 2024-04-25T10:24:28.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f814d800640 1 -- 172.21.15.87:0/416193308 shutdown_connections 2024-04-25T10:24:28.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f814d800640 1 --2- 172.21.15.87:0/416193308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b1f50 0x7f81480b23c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f814d800640 1 -- 172.21.15.87:0/416193308 >> 172.21.15.87:0/416193308 conn(0x7f81480b32b0 msgr2=0x7f81480b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:28.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f814d800640 1 -- 172.21.15.87:0/416193308 shutdown_connections 2024-04-25T10:24:28.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.040+0000 7f814d800640 1 -- 172.21.15.87:0/416193308 wait complete. 2024-04-25T10:24:28.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 Processor -- start 2024-04-25T10:24:28.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 -- start start 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81480b1f50 0x7f81480b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b3dd0 0x7f81480c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81480cd780 0x7f8148158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814816c230 con 0x7f81480cd780 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81480d0550 con 0x7f81480b3dd0 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814816c3b0 con 0x7f81480b1f50 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81480cd780 0x7f8148158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81480cd780 0x7f8148158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41042/0 (socket says 172.21.15.87:41042) 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 -- 172.21.15.87:0/17785876 learned_addr learned my addr 172.21.15.87:0/17785876 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:28.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f813f400640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b3dd0 0x7f81480c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f8147e00640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81480b1f50 0x7f81480b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 -- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81480b1f50 msgr2=0x7f81480b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81480b1f50 0x7f81480b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 -- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b3dd0 msgr2=0x7f81480c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b3dd0 0x7f81480c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 -- 172.21.15.87:0/17785876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81480c9fb0 con 0x7f81480cd780 2024-04-25T10:24:28.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814ce00640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81480cd780 0x7f8148158550 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f814000c3c0 tx=0x7f814000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f8144c00640 1 -- 172.21.15.87:0/17785876 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140017070 con 0x7f81480cd780 2024-04-25T10:24:28.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f8144c00640 1 -- 172.21.15.87:0/17785876 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8140005cf0 con 0x7f81480cd780 2024-04-25T10:24:28.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f8144c00640 1 -- 172.21.15.87:0/17785876 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81400072f0 con 0x7f81480cd780 2024-04-25T10:24:28.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81480ca280 con 0x7f81480cd780 2024-04-25T10:24:28.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.044+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81480ca840 con 0x7f81480cd780 2024-04-25T10:24:28.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.048+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8148075120 con 0x7f81480cd780 2024-04-25T10:24:28.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.048+0000 7f8144c00640 1 -- 172.21.15.87:0/17785876 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f814001a070 con 0x7f81480cd780 2024-04-25T10:24:28.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.048+0000 7f8144c00640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8124066f60 0x7f8124069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.048+0000 7f8144c00640 1 -- 172.21.15.87:0/17785876 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8140084350 con 0x7f81480cd780 2024-04-25T10:24:28.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.048+0000 7f8147e00640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8124066f60 0x7f8124069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.048+0000 7f8147e00640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8124066f60 0x7f8124069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f81380ad000 tx=0x7f81380a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.052+0000 7f8144c00640 1 -- 172.21.15.87:0/17785876 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f814806be90 con 0x7f81480cd780 2024-04-25T10:24:28.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.204+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81480cd410 con 0x7f81480cd780 2024-04-25T10:24:28.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.204+0000 7f8144c00640 1 -- 172.21.15.87:0/17785876 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8140054d00 con 0x7f81480cd780 2024-04-25T10:24:28.205 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8124066f60 msgr2=0x7f8124069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8124066f60 0x7f8124069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f81380ad000 tx=0x7f81380a6000 comp rx=0 tx=0).stop 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81480cd780 msgr2=0x7f8148158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81480cd780 0x7f8148158550 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f814000c3c0 tx=0x7f814000c880 comp rx=0 tx=0).stop 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 shutdown_connections 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8124066f60 0x7f8124069420 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81480cd780 0x7f8148158550 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480b3dd0 0x7f81480c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 --2- 172.21.15.87:0/17785876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81480b1f50 0x7f81480b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 >> 172.21.15.87:0/17785876 conn(0x7f81480b32b0 msgr2=0x7f81480cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:28.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 shutdown_connections 2024-04-25T10:24:28.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.208+0000 7f814d800640 1 -- 172.21.15.87:0/17785876 wait complete. 2024-04-25T10:24:28.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:24:28.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd166e00640 1 Processor -- start 2024-04-25T10:24:28.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd166e00640 1 -- start start 2024-04-25T10:24:28.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd166e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd160071e50 0x7fd1601472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd166e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16006c600 con 0x7fd160146ed0 2024-04-25T10:24:28.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd166e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd160161bd0 con 0x7fd1600726b0 2024-04-25T10:24:28.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd166e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd160161980 con 0x7fd160071e50 2024-04-25T10:24:28.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd15f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd160071e50 0x7fd1601472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd15fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd160146ed0 0x7fd160071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41280/0 (socket says 172.21.15.87:41280) 2024-04-25T10:24:28.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd15fe00640 1 -- 172.21.15.87:0/1988193377 learned_addr learned my addr 172.21.15.87:0/1988193377 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:28.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd15f400640 1 -- 172.21.15.87:0/1988193377 >> v1:172.21.15.169:6789/0 conn(0x7fd1600726b0 legacy=0x7fd1600ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:28.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd15f400640 1 -- 172.21.15.87:0/1988193377 >> v1:172.21.15.87:6789/0 conn(0x7fd160146ed0 legacy=0x7fd160071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd15f400640 1 -- 172.21.15.87:0/1988193377 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160161eb0 con 0x7fd160071e50 2024-04-25T10:24:28.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.364+0000 7fd15ea00640 1 -- 172.21.15.87:0/1988193377 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508844740 0 0) 0x7fd16006c600 con 0x7fd160146ed0 2024-04-25T10:24:28.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd15f400640 1 --2- 172.21.15.87:0/1988193377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160071e50 0x7fd1601472b0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0097c0 tx=0x7fd14c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5402dc7f3faa5821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd15ea00640 1 -- 172.21.15.87:0/1988193377 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0a7070 con 0x7fd160071e50 2024-04-25T10:24:28.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd15ea00640 1 -- 172.21.15.87:0/1988193377 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c0b0070 con 0x7fd160071e50 2024-04-25T10:24:28.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd15ea00640 1 -- 172.21.15.87:0/1988193377 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0ab890 con 0x7fd160071e50 2024-04-25T10:24:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 -- 172.21.15.87:0/1988193377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160071e50 msgr2=0x7fd1601472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 --2- 172.21.15.87:0/1988193377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160071e50 0x7fd1601472b0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0097c0 tx=0x7fd14c01c470 comp rx=0 tx=0).stop 2024-04-25T10:24:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 -- 172.21.15.87:0/1988193377 shutdown_connections 2024-04-25T10:24:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 --2- 172.21.15.87:0/1988193377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160071e50 0x7fd1601472b0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 -- 172.21.15.87:0/1988193377 >> 172.21.15.87:0/1988193377 conn(0x7fd1600669a0 msgr2=0x7fd160066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 -- 172.21.15.87:0/1988193377 shutdown_connections 2024-04-25T10:24:28.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 -- 172.21.15.87:0/1988193377 wait complete. 2024-04-25T10:24:28.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 Processor -- start 2024-04-25T10:24:28.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.368+0000 7fd166e00640 1 -- start start 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd160071e50 0x7fd16015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1600726b0 0x7fd160145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160146ed0 0x7fd16015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600ae8e0 con 0x7fd160146ed0 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600ae760 con 0x7fd160071e50 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600aea60 con 0x7fd1600726b0 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160146ed0 0x7fd16015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160146ed0 0x7fd16015fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41066/0 (socket says 172.21.15.87:41066) 2024-04-25T10:24:28.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 -- 172.21.15.87:0/3656961765 learned_addr learned my addr 172.21.15.87:0/3656961765 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd15fe00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd160071e50 0x7fd16015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd15f400640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1600726b0 0x7fd160145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 -- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1600726b0 msgr2=0x7fd160145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1600726b0 0x7fd160145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 -- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd160071e50 msgr2=0x7fd16015ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd160071e50 0x7fd16015ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 -- 172.21.15.87:0/3656961765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160144670 con 0x7fd160146ed0 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd15f400640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1600726b0 0x7fd160145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd164e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160146ed0 0x7fd16015fd70 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c7d0 tx=0x7fd15800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd157e00640 1 -- 172.21.15.87:0/3656961765 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158003b80 con 0x7fd160146ed0 2024-04-25T10:24:28.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd157e00640 1 -- 172.21.15.87:0/3656961765 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd158003d20 con 0x7fd160146ed0 2024-04-25T10:24:28.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd160144950 con 0x7fd160146ed0 2024-04-25T10:24:28.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd157e00640 1 -- 172.21.15.87:0/3656961765 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800aa30 con 0x7fd160146ed0 2024-04-25T10:24:28.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd160144f10 con 0x7fd160146ed0 2024-04-25T10:24:28.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16004ef30 con 0x7fd160146ed0 2024-04-25T10:24:28.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.372+0000 7fd157e00640 1 -- 172.21.15.87:0/3656961765 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd15800abd0 con 0x7fd160146ed0 2024-04-25T10:24:28.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.376+0000 7fd157e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd140066f60 0x7fd140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.376+0000 7fd157e00640 1 -- 172.21.15.87:0/3656961765 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd158084eb0 con 0x7fd160146ed0 2024-04-25T10:24:28.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.376+0000 7fd15fe00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd140066f60 0x7fd140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.376+0000 7fd15fe00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd140066f60 0x7fd140069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd150000f70 tx=0x7fd150007240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.380+0000 7fd157e00640 1 -- 172.21.15.87:0/3656961765 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd158055eb0 con 0x7fd160146ed0 2024-04-25T10:24:28.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd160072490 con 0x7fd160146ed0 2024-04-25T10:24:28.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd157e00640 1 -- 172.21.15.87:0/3656961765 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v151) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd1580557b0 con 0x7fd160146ed0 2024-04-25T10:24:28.544 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd140066f60 msgr2=0x7fd140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd140066f60 0x7fd140069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd150000f70 tx=0x7fd150007240 comp rx=0 tx=0).stop 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160146ed0 msgr2=0x7fd16015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160146ed0 0x7fd16015fd70 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c7d0 tx=0x7fd15800cc90 comp rx=0 tx=0).stop 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 shutdown_connections 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd140066f60 0x7fd140069420 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd160146ed0 0x7fd16015fd70 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1600726b0 0x7fd160145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 --2- 172.21.15.87:0/3656961765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd160071e50 0x7fd16015ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 >> 172.21.15.87:0/3656961765 conn(0x7fd1600669a0 msgr2=0x7fd16006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 shutdown_connections 2024-04-25T10:24:28.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.544+0000 7fd166e00640 1 -- 172.21.15.87:0/3656961765 wait complete. 2024-04-25T10:24:28.565 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:24:28.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:28.565 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:28.591 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:24:28.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:28.592 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:24:28.643 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:24:28.643 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:28.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:28.653 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:24:28.653 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:28.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:24:28.693 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:24:28.693 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:28.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:28.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d59e00640 1 Processor -- start 2024-04-25T10:24:28.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d59e00640 1 -- start start 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d59e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d540ccb20 0x7f8d540d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d59e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d5415bdc0 0x7f8d540d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d59e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540ccfb0 con 0x7f8d5415b9e0 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d59e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540cd290 con 0x7f8d5415bdc0 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d59e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540cd570 con 0x7f8d540ccb20 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d53400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d5415bdc0 0x7f8d540d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d53400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d5415bdc0 0x7f8d540d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41074/0 (socket says 172.21.15.87:41074) 2024-04-25T10:24:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d53400640 1 -- 172.21.15.87:0/135587343 learned_addr learned my addr 172.21.15.87:0/135587343 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d53400640 1 -- 172.21.15.87:0/135587343 >> v2:172.21.15.169:3300/0 conn(0x7f8d540ccb20 msgr2=0x7f8d540d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d53400640 1 --2- 172.21.15.87:0/135587343 >> v2:172.21.15.169:3300/0 conn(0x7f8d540ccb20 0x7f8d540d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d53400640 1 -- 172.21.15.87:0/135587343 >> v1:172.21.15.169:6789/0 conn(0x7f8d5415b9e0 legacy=0x7f8d540cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:28.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.872+0000 7f8d53400640 1 -- 172.21.15.87:0/135587343 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540cd850 con 0x7f8d5415bdc0 2024-04-25T10:24:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d53400640 1 --2- 172.21.15.87:0/135587343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415bdc0 0x7f8d540d1a60 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8d54067980 tx=0x7f8d4c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94a146c37e0dc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d51600640 1 -- 172.21.15.87:0/135587343 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0b0070 con 0x7f8d5415bdc0 2024-04-25T10:24:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d51600640 1 -- 172.21.15.87:0/135587343 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c0ab600 con 0x7f8d5415bdc0 2024-04-25T10:24:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d51600640 1 -- 172.21.15.87:0/135587343 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0ab9d0 con 0x7f8d5415bdc0 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 -- 172.21.15.87:0/135587343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415bdc0 msgr2=0x7f8d540d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 --2- 172.21.15.87:0/135587343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415bdc0 0x7f8d540d1a60 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8d54067980 tx=0x7f8d4c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 -- 172.21.15.87:0/135587343 shutdown_connections 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 --2- 172.21.15.87:0/135587343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415bdc0 0x7f8d540d1a60 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 --2- 172.21.15.87:0/135587343 >> v2:172.21.15.169:3300/0 conn(0x7f8d540ccb20 0x7f8d540d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 -- 172.21.15.87:0/135587343 >> 172.21.15.87:0/135587343 conn(0x7f8d5409a9d0 msgr2=0x7f8d5409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 -- 172.21.15.87:0/135587343 shutdown_connections 2024-04-25T10:24:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.876+0000 7f8d59e00640 1 -- 172.21.15.87:0/135587343 wait complete. 2024-04-25T10:24:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 Processor -- start 2024-04-25T10:24:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 -- start start 2024-04-25T10:24:28.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415b9e0 0x7f8d5414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d5415bdc0 0x7f8d5414b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d5414bcb0 0x7f8d5414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540c65b0 con 0x7f8d5415b9e0 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540c6430 con 0x7f8d5415bdc0 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540c6730 con 0x7f8d5414bcb0 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415b9e0 0x7f8d5414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415b9e0 0x7f8d5414c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41076/0 (socket says 172.21.15.87:41076) 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 -- 172.21.15.87:0/2609314774 learned_addr learned my addr 172.21.15.87:0/2609314774 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d53400640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d5414bcb0 0x7f8d5414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52000640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d5415bdc0 0x7f8d5414b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 -- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d5414bcb0 msgr2=0x7f8d5414a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d5414bcb0 0x7f8d5414a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 -- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d5415bdc0 msgr2=0x7f8d5414b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d5415bdc0 0x7f8d5414b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 -- 172.21.15.87:0/2609314774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5415ed40 con 0x7f8d5415b9e0 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52000640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d5415bdc0 0x7f8d5414b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d52a00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415b9e0 0x7f8d5414c800 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400c7d0 tx=0x7f8d4400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d4b400640 1 -- 172.21.15.87:0/2609314774 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44017070 con 0x7f8d5415b9e0 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d4b400640 1 -- 172.21.15.87:0/2609314774 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d44003db0 con 0x7f8d5415b9e0 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d4b400640 1 -- 172.21.15.87:0/2609314774 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d440052b0 con 0x7f8d5415b9e0 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5415f020 con 0x7f8d5415b9e0 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5414a910 con 0x7f8d5415b9e0 2024-04-25T10:24:28.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.880+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5404ef30 con 0x7f8d5415b9e0 2024-04-25T10:24:28.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.884+0000 7f8d4b400640 1 -- 172.21.15.87:0/2609314774 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d44005450 con 0x7f8d5415b9e0 2024-04-25T10:24:28.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.884+0000 7f8d4b400640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d30066f60 0x7f8d30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:28.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.884+0000 7f8d4b400640 1 -- 172.21.15.87:0/2609314774 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d44084600 con 0x7f8d5415b9e0 2024-04-25T10:24:28.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.884+0000 7f8d52000640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d30066f60 0x7f8d30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:28.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.884+0000 7f8d52000640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d30066f60 0x7f8d30069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8d540c7e40 tx=0x7f8d3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:28.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:28.888+0000 7f8d4b400640 1 -- 172.21.15.87:0/2609314774 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d44055600 con 0x7f8d5415b9e0 2024-04-25T10:24:29.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d540c7e70 con 0x7f8d5415b9e0 2024-04-25T10:24:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d4b400640 1 -- 172.21.15.87:0/2609314774 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4955 (secure 0 0 0) 0x7f8d44054f00 con 0x7f8d5415b9e0 2024-04-25T10:24:29.076 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:29.076 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":111,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4139990719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4139990719},{"type":"v1","addr":"172.21.15.169:6834","nonce":4139990719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3962905873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3962905873},{"type":"v1","addr":"172.21.15.169:6835","nonce":3962905873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3473546915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3473546915},{"type":"v1","addr":"172.21.15.87:6836","nonce":3473546915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:23:47.224714+0000","modified":"2024-04-25T10:23:50.259650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10044":{"gid":10044,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/564498676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":564498676},{"type":"v1","addr":"172.21.15.87:6837","nonce":564498676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:24:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 111 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d30066f60 msgr2=0x7f8d30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d30066f60 0x7f8d30069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8d540c7e40 tx=0x7f8d3c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415b9e0 msgr2=0x7f8d5414c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415b9e0 0x7f8d5414c800 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400c7d0 tx=0x7f8d4400cc90 comp rx=0 tx=0).stop 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 shutdown_connections 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d30066f60 0x7f8d30069420 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d5414bcb0 0x7f8d5414a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d5415bdc0 0x7f8d5414b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 --2- 172.21.15.87:0/2609314774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d5415b9e0 0x7f8d5414c800 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 >> 172.21.15.87:0/2609314774 conn(0x7f8d5409a9d0 msgr2=0x7f8d540c6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.076+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 shutdown_connections 2024-04-25T10:24:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.080+0000 7f8d59e00640 1 -- 172.21.15.87:0/2609314774 wait complete. 2024-04-25T10:24:29.096 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.232+0000 7f86b8400640 1 Processor -- start 2024-04-25T10:24:29.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.232+0000 7f86b8400640 1 -- start start 2024-04-25T10:24:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.232+0000 7f86b8400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86b00d0040 0x7f86b006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.232+0000 7f86b8400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b00b2770 con 0x7f86b015bc40 2024-04-25T10:24:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.232+0000 7f86b8400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b006c670 con 0x7f86b00cda10 2024-04-25T10:24:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.232+0000 7f86b8400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b006c950 con 0x7f86b00d0040 2024-04-25T10:24:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86b6400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f86b00d0040 0x7f86b006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86b5a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f86b015bc40 0x7f86b00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41292/0 (socket says 172.21.15.87:41292) 2024-04-25T10:24:29.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86b5a00640 1 -- 172.21.15.87:0/2834912128 learned_addr learned my addr 172.21.15.87:0/2834912128 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:29.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86afe00640 1 -- 172.21.15.87:0/2834912128 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332620319 0 0) 0x7f86b00b2770 con 0x7f86b015bc40 2024-04-25T10:24:29.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86b6400640 1 -- 172.21.15.87:0/2834912128 >> v1:172.21.15.169:6789/0 conn(0x7f86b00cda10 legacy=0x7f86b00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:29.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86b6400640 1 -- 172.21.15.87:0/2834912128 >> v1:172.21.15.87:6789/0 conn(0x7f86b015bc40 legacy=0x7f86b00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86b6400640 1 -- 172.21.15.87:0/2834912128 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b006cc30 con 0x7f86b00d0040 2024-04-25T10:24:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86b6400640 1 --2- 172.21.15.87:0/2834912128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b00d0040 0x7f86b006c090 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009d90 tx=0x7f86a801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a910f55092a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86afe00640 1 -- 172.21.15.87:0/2834912128 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80b2070 con 0x7f86b00d0040 2024-04-25T10:24:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86afe00640 1 -- 172.21.15.87:0/2834912128 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a80a5870 con 0x7f86b00d0040 2024-04-25T10:24:29.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.236+0000 7f86afe00640 1 -- 172.21.15.87:0/2834912128 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80a5c40 con 0x7f86b00d0040 2024-04-25T10:24:29.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- 172.21.15.87:0/2834912128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b00d0040 msgr2=0x7f86b006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 --2- 172.21.15.87:0/2834912128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b00d0040 0x7f86b006c090 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009d90 tx=0x7f86a801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- 172.21.15.87:0/2834912128 shutdown_connections 2024-04-25T10:24:29.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 --2- 172.21.15.87:0/2834912128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b00d0040 0x7f86b006c090 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- 172.21.15.87:0/2834912128 >> 172.21.15.87:0/2834912128 conn(0x7f86b0066b70 msgr2=0x7f86b0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:29.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- 172.21.15.87:0/2834912128 shutdown_connections 2024-04-25T10:24:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- 172.21.15.87:0/2834912128 wait complete. 2024-04-25T10:24:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 Processor -- start 2024-04-25T10:24:29.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- start start 2024-04-25T10:24:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00cda10 0x7f86b0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00d0040 0x7f86b01555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b015bc40 0x7f86b00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0169980 con 0x7f86b015bc40 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0169800 con 0x7f86b00d0040 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b8400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0169b00 con 0x7f86b00cda10 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b015bc40 0x7f86b00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b015bc40 0x7f86b00adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41086/0 (socket says 172.21.15.87:41086) 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 -- 172.21.15.87:0/3350909298 learned_addr learned my addr 172.21.15.87:0/3350909298 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b5a00640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00cda10 0x7f86b0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b5000640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00d0040 0x7f86b01555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 -- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00cda10 msgr2=0x7f86b0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00cda10 0x7f86b0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 -- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00d0040 msgr2=0x7f86b01555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00d0040 0x7f86b01555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.240+0000 7f86b6400640 1 -- 172.21.15.87:0/3350909298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b01540b0 con 0x7f86b015bc40 2024-04-25T10:24:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86b6400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b015bc40 0x7f86b00adaa0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f86a80b0040 tx=0x7f86a8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86ad600640 1 -- 172.21.15.87:0/3350909298 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80bd030 con 0x7f86b015bc40 2024-04-25T10:24:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86ad600640 1 -- 172.21.15.87:0/3350909298 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a80a5670 con 0x7f86b015bc40 2024-04-25T10:24:29.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86ad600640 1 -- 172.21.15.87:0/3350909298 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80b2040 con 0x7f86b015bc40 2024-04-25T10:24:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b0154390 con 0x7f86b015bc40 2024-04-25T10:24:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b0154950 con 0x7f86b015bc40 2024-04-25T10:24:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b004ef30 con 0x7f86b015bc40 2024-04-25T10:24:29.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86ad600640 1 -- 172.21.15.87:0/3350909298 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86a80c0070 con 0x7f86b015bc40 2024-04-25T10:24:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86ad600640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8690066f60 0x7f8690069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86ad600640 1 -- 172.21.15.87:0/3350909298 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a811fce0 con 0x7f86b015bc40 2024-04-25T10:24:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.244+0000 7f86b5a00640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8690066f60 0x7f8690069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.248+0000 7f86b5a00640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8690066f60 0x7f8690069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000f70 tx=0x7f86a4007df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.248+0000 7f86ad600640 1 -- 172.21.15.87:0/3350909298 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86a80c7070 con 0x7f86b015bc40 2024-04-25T10:24:29.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.404+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86b00b2010 con 0x7f86b015bc40 2024-04-25T10:24:29.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86ad600640 1 -- 172.21.15.87:0/3350909298 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16659 (secure 0 0 0) 0x7f86a8016520 con 0x7f86b015bc40 2024-04-25T10:24:29.406 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:29.409 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":151,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:48.878767+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:23:45.609544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:23:46.223978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:29.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8690066f60 msgr2=0x7f8690069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8690066f60 0x7f8690069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000f70 tx=0x7f86a4007df0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b015bc40 msgr2=0x7f86b00adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b015bc40 0x7f86b00adaa0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f86a80b0040 tx=0x7f86a8019640 comp rx=0 tx=0).stop 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 shutdown_connections 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8690066f60 0x7f8690069420 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86b015bc40 0x7f86b00adaa0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00d0040 0x7f86b01555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 --2- 172.21.15.87:0/3350909298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00cda10 0x7f86b0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 >> 172.21.15.87:0/3350909298 conn(0x7f86b0066b70 msgr2=0x7f86b00b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 shutdown_connections 2024-04-25T10:24:29.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.408+0000 7f86b8400640 1 -- 172.21.15.87:0/3350909298 wait complete. 2024-04-25T10:24:29.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:23:45.609544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:23:46.223978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:29.428 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:29.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2498800640 1 Processor -- start 2024-04-25T10:24:29.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2498800640 1 -- start start 2024-04-25T10:24:29.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2498800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24900722d0 0x7f24900cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2498800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24900ad840 con 0x7f249015e130 2024-04-25T10:24:29.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2498800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249015e710 con 0x7f24900b3480 2024-04-25T10:24:29.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2498800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24900d0090 con 0x7f24900722d0 2024-04-25T10:24:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2496800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24900722d0 0x7f24900cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2495400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f249015e130 0x7f2490071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41298/0 (socket says 172.21.15.87:41298) 2024-04-25T10:24:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2495400640 1 -- 172.21.15.87:0/1327085327 learned_addr learned my addr 172.21.15.87:0/1327085327 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2496800640 1 -- 172.21.15.87:0/1327085327 >> v1:172.21.15.169:6789/0 conn(0x7f24900b3480 legacy=0x7f2490070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2496800640 1 -- 172.21.15.87:0/1327085327 >> v1:172.21.15.87:6789/0 conn(0x7f249015e130 legacy=0x7f2490071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2494a00640 1 -- 172.21.15.87:0/1327085327 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624861270 0 0) 0x7f24900ad840 con 0x7f249015e130 2024-04-25T10:24:29.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.568+0000 7f2496800640 1 -- 172.21.15.87:0/1327085327 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249015e9f0 con 0x7f24900722d0 2024-04-25T10:24:29.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2496800640 1 --2- 172.21.15.87:0/1327085327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24900722d0 0x7f24900cfb50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f248c009d90 tx=0x7f248c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34edbb66739e58b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2494a00640 1 -- 172.21.15.87:0/1327085327 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c0b2070 con 0x7f24900722d0 2024-04-25T10:24:29.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2494a00640 1 -- 172.21.15.87:0/1327085327 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c0a57a0 con 0x7f24900722d0 2024-04-25T10:24:29.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2494a00640 1 -- 172.21.15.87:0/1327085327 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c0a5b70 con 0x7f24900722d0 2024-04-25T10:24:29.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2498800640 1 -- 172.21.15.87:0/1327085327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24900722d0 msgr2=0x7f24900cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2498800640 1 --2- 172.21.15.87:0/1327085327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24900722d0 0x7f24900cfb50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f248c009d90 tx=0x7f248c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2498800640 1 -- 172.21.15.87:0/1327085327 shutdown_connections 2024-04-25T10:24:29.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2498800640 1 --2- 172.21.15.87:0/1327085327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24900722d0 0x7f24900cfb50 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2498800640 1 -- 172.21.15.87:0/1327085327 >> 172.21.15.87:0/1327085327 conn(0x7f24900669a0 msgr2=0x7f2490066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2498800640 1 -- 172.21.15.87:0/1327085327 shutdown_connections 2024-04-25T10:24:29.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.572+0000 7f2498800640 1 -- 172.21.15.87:0/1327085327 wait complete. 2024-04-25T10:24:29.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 Processor -- start 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 -- start start 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24900722d0 0x7f249015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24900b3480 0x7f2490153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f249015e130 0x7f24901542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249015bb90 con 0x7f249015e130 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24900ae780 con 0x7f24900b3480 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249016b8a0 con 0x7f24900722d0 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f249015e130 0x7f24901542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2495e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24900722d0 0x7f249015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2495400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24900b3480 0x7f2490153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f249015e130 0x7f24901542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41102/0 (socket says 172.21.15.87:41102) 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 -- 172.21.15.87:0/3386421477 learned_addr learned my addr 172.21.15.87:0/3386421477 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:29.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 -- 172.21.15.87:0/3386421477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24900722d0 msgr2=0x7f249015f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24900722d0 0x7f249015f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 -- 172.21.15.87:0/3386421477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24900b3480 msgr2=0x7f2490153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24900b3480 0x7f2490153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 -- 172.21.15.87:0/3386421477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2490160060 con 0x7f249015e130 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2496800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f249015e130 0x7f24901542b0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f248c01a510 tx=0x7f248c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f248a000640 1 -- 172.21.15.87:0/3386421477 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c0b2050 con 0x7f249015e130 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f248a000640 1 -- 172.21.15.87:0/3386421477 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c0be070 con 0x7f249015e130 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f248a000640 1 -- 172.21.15.87:0/3386421477 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c0ada50 con 0x7f249015e130 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2490160340 con 0x7f249015e130 2024-04-25T10:24:29.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2490160900 con 0x7f249015e130 2024-04-25T10:24:29.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.576+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249004ef30 con 0x7f249015e130 2024-04-25T10:24:29.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.580+0000 7f248a000640 1 -- 172.21.15.87:0/3386421477 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f248c0adbf0 con 0x7f249015e130 2024-04-25T10:24:29.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.580+0000 7f248a000640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2474066f90 0x7f2474069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.580+0000 7f248a000640 1 -- 172.21.15.87:0/3386421477 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f248c11fed0 con 0x7f249015e130 2024-04-25T10:24:29.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.580+0000 7f2495e00640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2474066f90 0x7f2474069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.580+0000 7f2495e00640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2474066f90 0x7f2474069450 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f24901f9c60 tx=0x7f2484009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.584+0000 7f248a000640 1 -- 172.21.15.87:0/3386421477 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f248c016560 con 0x7f249015e130 2024-04-25T10:24:29.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.736+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24901571d0 con 0x7f249015e130 2024-04-25T10:24:29.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f248a000640 1 -- 172.21.15.87:0/3386421477 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7f248c0f09a0 con 0x7f249015e130 2024-04-25T10:24:29.738 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:24:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2474066f90 msgr2=0x7f2474069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2474066f90 0x7f2474069450 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f24901f9c60 tx=0x7f2484009040 comp rx=0 tx=0).stop 2024-04-25T10:24:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f249015e130 msgr2=0x7f24901542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f249015e130 0x7f24901542b0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f248c01a510 tx=0x7f248c019680 comp rx=0 tx=0).stop 2024-04-25T10:24:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 shutdown_connections 2024-04-25T10:24:29.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2474066f90 0x7f2474069450 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f249015e130 0x7f24901542b0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24900b3480 0x7f2490153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 --2- 172.21.15.87:0/3386421477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24900722d0 0x7f249015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 >> 172.21.15.87:0/3386421477 conn(0x7f24900669a0 msgr2=0x7f24900b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 shutdown_connections 2024-04-25T10:24:29.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.740+0000 7f2498800640 1 -- 172.21.15.87:0/3386421477 wait complete. 2024-04-25T10:24:29.759 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:24:29.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 Processor -- start 2024-04-25T10:24:29.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 -- start start 2024-04-25T10:24:29.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4edc0aef80 0x7f4edc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4edc15bac0 0x7f4edc0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc15b080 con 0x7f4edc0aef80 2024-04-25T10:24:29.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc15adb0 con 0x7f4edc15bac0 2024-04-25T10:24:29.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ee2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc15eb60 con 0x7f4edc070350 2024-04-25T10:24:29.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4edc0aef80 0x7f4edc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4edc0aef80 0x7f4edc15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41104/0 (socket says 172.21.15.87:41104) 2024-04-25T10:24:29.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 -- 172.21.15.87:0/694366104 learned_addr learned my addr 172.21.15.87:0/694366104 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edbe00640 1 --2- 172.21.15.87:0/694366104 >> v2:172.21.15.169:3300/0 conn(0x7f4edc15bac0 0x7f4edc0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ed3e00640 1 --2- 172.21.15.87:0/694366104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 -- 172.21.15.87:0/694366104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 msgr2=0x7f4edc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 --2- 172.21.15.87:0/694366104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 -- 172.21.15.87:0/694366104 >> v2:172.21.15.169:3300/0 conn(0x7f4edc15bac0 msgr2=0x7f4edc0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 --2- 172.21.15.87:0/694366104 >> v2:172.21.15.169:3300/0 conn(0x7f4edc15bac0 0x7f4edc0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edb400640 1 -- 172.21.15.87:0/694366104 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc15ee40 con 0x7f4edc0aef80 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4edbe00640 1 --2- 172.21.15.87:0/694366104 >> v2:172.21.15.169:3300/0 conn(0x7f4edc15bac0 0x7f4edc0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:29.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.900+0000 7f4ed3e00640 1 --2- 172.21.15.87:0/694366104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:29.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4edb400640 1 --2- 172.21.15.87:0/694366104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc15e480 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0cc330 tx=0x7f4ecc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8ce697c4bdf418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4edaa00640 1 -- 172.21.15.87:0/694366104 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0ab5f0 con 0x7f4edc0aef80 2024-04-25T10:24:29.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4edaa00640 1 -- 172.21.15.87:0/694366104 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc0b0070 con 0x7f4edc0aef80 2024-04-25T10:24:29.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4edaa00640 1 -- 172.21.15.87:0/694366104 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0ab9b0 con 0x7f4edc0aef80 2024-04-25T10:24:29.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 -- 172.21.15.87:0/694366104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 msgr2=0x7f4edc15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 --2- 172.21.15.87:0/694366104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc15e480 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0cc330 tx=0x7f4ecc01c520 comp rx=0 tx=0).stop 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 -- 172.21.15.87:0/694366104 shutdown_connections 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 --2- 172.21.15.87:0/694366104 >> v2:172.21.15.169:3300/0 conn(0x7f4edc15bac0 0x7f4edc0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 --2- 172.21.15.87:0/694366104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 --2- 172.21.15.87:0/694366104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc15e480 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 -- 172.21.15.87:0/694366104 >> 172.21.15.87:0/694366104 conn(0x7f4edc0669a0 msgr2=0x7f4edc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 -- 172.21.15.87:0/694366104 shutdown_connections 2024-04-25T10:24:29.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 -- 172.21.15.87:0/694366104 wait complete. 2024-04-25T10:24:29.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 Processor -- start 2024-04-25T10:24:29.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.904+0000 7f4ee2000640 1 -- start start 2024-04-25T10:24:29.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4edc15bac0 0x7f4edc1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06d570 con 0x7f4edc0aef80 2024-04-25T10:24:29.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06d3f0 con 0x7f4edc15bac0 2024-04-25T10:24:29.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06d6f0 con 0x7f4edc070350 2024-04-25T10:24:29.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc0ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41120/0 (socket says 172.21.15.87:41120) 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 -- 172.21.15.87:0/4104615982 learned_addr learned my addr 172.21.15.87:0/4104615982 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4edb400640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4edbe00640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4edc15bac0 0x7f4edc1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 -- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 msgr2=0x7f4edc1460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc1460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 -- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4edc15bac0 msgr2=0x7f4edc1f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4edc15bac0 0x7f4edc1f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 -- 172.21.15.87:0/4104615982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0a9cc0 con 0x7f4edc0aef80 2024-04-25T10:24:29.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3e00640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc0ae450 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c7c0 tx=0x7f4ec400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3400640 1 -- 172.21.15.87:0/4104615982 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4015070 con 0x7f4edc0aef80 2024-04-25T10:24:29.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3400640 1 -- 172.21.15.87:0/4104615982 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec4007a30 con 0x7f4edc0aef80 2024-04-25T10:24:29.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc0a9fa0 con 0x7f4edc0aef80 2024-04-25T10:24:29.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ed3400640 1 -- 172.21.15.87:0/4104615982 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4007e00 con 0x7f4edc0aef80 2024-04-25T10:24:29.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc0aa560 con 0x7f4edc0aef80 2024-04-25T10:24:29.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.908+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc04ef30 con 0x7f4edc0aef80 2024-04-25T10:24:29.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.912+0000 7f4ed3400640 1 -- 172.21.15.87:0/4104615982 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ec4018070 con 0x7f4edc0aef80 2024-04-25T10:24:29.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.912+0000 7f4ed3400640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4eb8066f60 0x7f4eb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:29.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.912+0000 7f4ed3400640 1 -- 172.21.15.87:0/4104615982 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ec4084f80 con 0x7f4edc0aef80 2024-04-25T10:24:29.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.912+0000 7f4edb400640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4eb8066f60 0x7f4eb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:29.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.912+0000 7f4edb400640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4eb8066f60 0x7f4eb8069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0af000 tx=0x7f4ecc0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:29.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:29.916+0000 7f4ed3400640 1 -- 172.21.15.87:0/4104615982 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ec40558a0 con 0x7f4edc0aef80 2024-04-25T10:24:30.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.068+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4edc0b2fa0 con 0x7f4edc0aef80 2024-04-25T10:24:30.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ed3400640 1 -- 172.21.15.87:0/4104615982 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ec4059940 con 0x7f4edc0aef80 2024-04-25T10:24:30.070 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:24:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4eb8066f60 msgr2=0x7f4eb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4eb8066f60 0x7f4eb8069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0af000 tx=0x7f4ecc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:24:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 msgr2=0x7f4edc0ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc0ae450 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c7c0 tx=0x7f4ec400cc80 comp rx=0 tx=0).stop 2024-04-25T10:24:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 shutdown_connections 2024-04-25T10:24:30.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4eb8066f60 0x7f4eb8069420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4edc15bac0 0x7f4edc1f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4edc0aef80 0x7f4edc0ae450 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 --2- 172.21.15.87:0/4104615982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4edc070350 0x7f4edc1460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 >> 172.21.15.87:0/4104615982 conn(0x7f4edc0669a0 msgr2=0x7f4edc0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 shutdown_connections 2024-04-25T10:24:30.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.072+0000 7f4ee2000640 1 -- 172.21.15.87:0/4104615982 wait complete. 2024-04-25T10:24:30.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:24:30.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 Processor -- start 2024-04-25T10:24:30.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 -- start start 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f4066c00 0x7ff8f4161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff8f4066fe0 0x7ff8f4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8f40b5560 0x7ff8f40ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f4161d60 con 0x7ff8f40b5560 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f40b40c0 con 0x7ff8f4066fe0 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8fb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40b5b00 con 0x7ff8f4066c00 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8f40b5560 0x7ff8f40ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8f40b5560 0x7ff8f40ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41128/0 (socket says 172.21.15.87:41128) 2024-04-25T10:24:30.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 -- 172.21.15.87:0/2418959592 learned_addr learned my addr 172.21.15.87:0/2418959592 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f3400640 1 --2- 172.21.15.87:0/2418959592 >> v2:172.21.15.169:3300/0 conn(0x7ff8f4066fe0 0x7ff8f4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f3e00640 1 --2- 172.21.15.87:0/2418959592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f4066c00 0x7ff8f4161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 -- 172.21.15.87:0/2418959592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f4066c00 msgr2=0x7ff8f4161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 --2- 172.21.15.87:0/2418959592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f4066c00 0x7ff8f4161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 -- 172.21.15.87:0/2418959592 >> v2:172.21.15.169:3300/0 conn(0x7ff8f4066fe0 msgr2=0x7ff8f4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 --2- 172.21.15.87:0/2418959592 >> v2:172.21.15.169:3300/0 conn(0x7ff8f4066fe0 0x7ff8f4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f9000640 1 -- 172.21.15.87:0/2418959592 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f40b3a00 con 0x7ff8f40b5560 2024-04-25T10:24:30.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.232+0000 7ff8f3e00640 1 --2- 172.21.15.87:0/2418959592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f4066c00 0x7ff8f4161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:30.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8f9000640 1 --2- 172.21.15.87:0/2418959592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40b5560 0x7ff8f40ae850 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40d1990 tx=0x7ff8ec01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3624296c27a8820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8f2a00640 1 -- 172.21.15.87:0/2418959592 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0a5550 con 0x7ff8f40b5560 2024-04-25T10:24:30.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8f2a00640 1 -- 172.21.15.87:0/2418959592 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec0a56f0 con 0x7ff8f40b5560 2024-04-25T10:24:30.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8f2a00640 1 -- 172.21.15.87:0/2418959592 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0a5ac0 con 0x7ff8f40b5560 2024-04-25T10:24:30.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 -- 172.21.15.87:0/2418959592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40b5560 msgr2=0x7ff8f40ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 --2- 172.21.15.87:0/2418959592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40b5560 0x7ff8f40ae850 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40d1990 tx=0x7ff8ec01c8d0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 -- 172.21.15.87:0/2418959592 shutdown_connections 2024-04-25T10:24:30.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 --2- 172.21.15.87:0/2418959592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40b5560 0x7ff8f40ae850 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 --2- 172.21.15.87:0/2418959592 >> v2:172.21.15.169:3300/0 conn(0x7ff8f4066fe0 0x7ff8f4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 --2- 172.21.15.87:0/2418959592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f4066c00 0x7ff8f4161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 -- 172.21.15.87:0/2418959592 >> 172.21.15.87:0/2418959592 conn(0x7ff8f40b2560 msgr2=0x7ff8f40b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:30.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 -- 172.21.15.87:0/2418959592 shutdown_connections 2024-04-25T10:24:30.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 -- 172.21.15.87:0/2418959592 wait complete. 2024-04-25T10:24:30.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.236+0000 7ff8fb000640 1 Processor -- start 2024-04-25T10:24:30.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 -- start start 2024-04-25T10:24:30.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f4066c00 0x7ff8f414b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4066fe0 0x7ff8f41542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b5560 0x7ff8f4155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41584d0 con 0x7ff8f4066fe0 2024-04-25T10:24:30.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4158350 con 0x7ff8f4066c00 2024-04-25T10:24:30.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4158650 con 0x7ff8f40b5560 2024-04-25T10:24:30.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4066fe0 0x7ff8f41542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4066fe0 0x7ff8f41542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41140/0 (socket says 172.21.15.87:41140) 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 -- 172.21.15.87:0/1766653750 learned_addr learned my addr 172.21.15.87:0/1766653750 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3e00640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f4066c00 0x7ff8f414b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f9000640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b5560 0x7ff8f4155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 -- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b5560 msgr2=0x7ff8f4155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b5560 0x7ff8f4155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 -- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f4066c00 msgr2=0x7ff8f414b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f4066c00 0x7ff8f414b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 -- 172.21.15.87:0/1766653750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f41509b0 con 0x7ff8f4066fe0 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3e00640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f4066c00 0x7ff8f414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8f3400640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4066fe0 0x7ff8f41542d0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e870 tx=0x7ff8e000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8ebe00640 1 -- 172.21.15.87:0/1766653750 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0017070 con 0x7ff8f4066fe0 2024-04-25T10:24:30.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8ebe00640 1 -- 172.21.15.87:0/1766653750 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e000c940 con 0x7ff8f4066fe0 2024-04-25T10:24:30.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f414f910 con 0x7ff8f4066fe0 2024-04-25T10:24:30.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8ebe00640 1 -- 172.21.15.87:0/1766653750 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000ccf0 con 0x7ff8f4066fe0 2024-04-25T10:24:30.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f414fed0 con 0x7ff8f4066fe0 2024-04-25T10:24:30.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.240+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f404ef30 con 0x7ff8f4066fe0 2024-04-25T10:24:30.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.244+0000 7ff8ebe00640 1 -- 172.21.15.87:0/1766653750 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8e001a070 con 0x7ff8f4066fe0 2024-04-25T10:24:30.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.244+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d4066f60 0x7ff8d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.244+0000 7ff8ebe00640 1 -- 172.21.15.87:0/1766653750 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8e0084f00 con 0x7ff8f4066fe0 2024-04-25T10:24:30.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.244+0000 7ff8f3e00640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d4066f60 0x7ff8d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.244+0000 7ff8f3e00640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d4066f60 0x7ff8d4069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004630 tx=0x7ff8e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.248+0000 7ff8ebe00640 1 -- 172.21.15.87:0/1766653750 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8e0055820 con 0x7ff8f4066fe0 2024-04-25T10:24:30.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.400+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8f41699a0 con 0x7ff8f4066fe0 2024-04-25T10:24:30.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.400+0000 7ff8ebe00640 1 -- 172.21.15.87:0/1766653750 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff8e00598c0 con 0x7ff8f4066fe0 2024-04-25T10:24:30.400 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:24:30.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d4066f60 msgr2=0x7ff8d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d4066f60 0x7ff8d4069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004630 tx=0x7ff8e4008040 comp rx=0 tx=0).stop 2024-04-25T10:24:30.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4066fe0 msgr2=0x7ff8f41542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4066fe0 0x7ff8f41542d0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e870 tx=0x7ff8e000ec70 comp rx=0 tx=0).stop 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 shutdown_connections 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d4066f60 0x7ff8d4069420 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b5560 0x7ff8f4155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4066fe0 0x7ff8f41542d0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 --2- 172.21.15.87:0/1766653750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f4066c00 0x7ff8f414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 >> 172.21.15.87:0/1766653750 conn(0x7ff8f40b2560 msgr2=0x7ff8f40b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 shutdown_connections 2024-04-25T10:24:30.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.404+0000 7ff8fb000640 1 -- 172.21.15.87:0/1766653750 wait complete. 2024-04-25T10:24:30.422 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:30.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.556+0000 7fc251c00640 1 Processor -- start 2024-04-25T10:24:30.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.556+0000 7fc251c00640 1 -- start start 2024-04-25T10:24:30.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc251c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc24c0d0040 0x7fc24c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc251c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c071050 con 0x7fc24c1550b0 2024-04-25T10:24:30.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc251c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c15d040 con 0x7fc24c15b740 2024-04-25T10:24:30.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc251c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0712f0 con 0x7fc24c0d0040 2024-04-25T10:24:30.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc24c1550b0 0x7fc24c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41300/0 (socket says 172.21.15.87:41300) 2024-04-25T10:24:30.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24aa00640 1 -- 172.21.15.87:0/199439188 learned_addr learned my addr 172.21.15.87:0/199439188 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 --2- 172.21.15.87:0/199439188 >> v2:172.21.15.169:3300/0 conn(0x7fc24c0d0040 0x7fc24c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796838221 0 0) 0x7fc24c071050 con 0x7fc24c1550b0 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc230003530 con 0x7fc24c1550b0 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863455727 0 0) 0x7fc230003530 con 0x7fc24c1550b0 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 >> v2:172.21.15.169:3300/0 conn(0x7fc24c0d0040 msgr2=0x7fc24c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 --2- 172.21.15.87:0/199439188 >> v2:172.21.15.169:3300/0 conn(0x7fc24c0d0040 0x7fc24c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 >> v1:172.21.15.169:6789/0 conn(0x7fc24c15b740 legacy=0x7fc24c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0715d0 con 0x7fc24c1550b0 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fc23c001a00 con 0x7fc24c1550b0 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 --2- 172.21.15.87:0/199439188 >> v2:172.21.15.169:3300/0 conn(0x7fc24c0d0040 0x7fc24c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 >> v1:172.21.15.87:6789/0 conn(0x7fc24c1550b0 legacy=0x7fc24c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230003a10 0x7fc230003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc230004810 0x7fc230004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2300051d0 0x7fc23000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0712f0 con 0x7fc230004810 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23000c0d0 con 0x7fc2300051d0 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23000e4e0 con 0x7fc230003a10 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc230004810 0x7fc230004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24aa00640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230003a10 0x7fc230003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24b400640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2300051d0 0x7fc23000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 -- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230003a10 msgr2=0x7fc230003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230003a10 0x7fc230003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 -- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2300051d0 msgr2=0x7fc23000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2300051d0 0x7fc23000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24a000640 1 -- 172.21.15.87:0/199439188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230003530 con 0x7fc230004810 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24b400640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2300051d0 0x7fc23000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:30.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.560+0000 7fc24b400640 1 -- 172.21.15.87:0/199439188 reap_dead start 2024-04-25T10:24:30.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc24a000640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc230004810 0x7fc230004c90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fc238009cb0 tx=0x7fc23801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9461d490f76743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2380b2070 con 0x7fc230004810 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2380a57c0 con 0x7fc230004810 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc249600640 1 -- 172.21.15.87:0/199439188 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238009880 con 0x7fc230004810 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc251c00640 1 -- 172.21.15.87:0/199439188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc230004810 msgr2=0x7fc230004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc251c00640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc230004810 0x7fc230004c90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fc238009cb0 tx=0x7fc23801cc10 comp rx=0 tx=0).stop 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc251c00640 1 -- 172.21.15.87:0/199439188 shutdown_connections 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc251c00640 1 --2- 172.21.15.87:0/199439188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc230004810 0x7fc230004c90 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc251c00640 1 -- 172.21.15.87:0/199439188 >> 172.21.15.87:0/199439188 conn(0x7fc24c066b70 msgr2=0x7fc24c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc251c00640 1 -- 172.21.15.87:0/199439188 shutdown_connections 2024-04-25T10:24:30.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.564+0000 7fc251c00640 1 -- 172.21.15.87:0/199439188 wait complete. 2024-04-25T10:24:30.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 Processor -- start 2024-04-25T10:24:30.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 -- start start 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230004810 0x7fc24c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc24c14d580 0x7fc24c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc24c0add80 0x7fc24c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0720b0 con 0x7fc24c0add80 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c071b90 con 0x7fc24c14d580 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c15bb90 con 0x7fc230004810 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc24c0add80 0x7fc24c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc24c0add80 0x7fc24c0cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41150/0 (socket says 172.21.15.87:41150) 2024-04-25T10:24:30.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 -- 172.21.15.87:0/228938742 learned_addr learned my addr 172.21.15.87:0/228938742 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24aa00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230004810 0x7fc24c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24a000640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc24c14d580 0x7fc24c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 -- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230004810 msgr2=0x7fc24c0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230004810 0x7fc24c0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 -- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc24c14d580 msgr2=0x7fc24c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc24c14d580 0x7fc24c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 -- 172.21.15.87:0/228938742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c072230 con 0x7fc24c0add80 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc24b400640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc24c0add80 0x7fc24c0cc7f0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fc24c070930 tx=0x7fc24400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc242a00640 1 -- 172.21.15.87:0/228938742 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244019070 con 0x7fc24c0add80 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc242a00640 1 -- 172.21.15.87:0/228938742 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244007e30 con 0x7fc24c0add80 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c156810 con 0x7fc24c0add80 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc242a00640 1 -- 172.21.15.87:0/228938742 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244003600 con 0x7fc24c0add80 2024-04-25T10:24:30.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c155490 con 0x7fc24c0add80 2024-04-25T10:24:30.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.568+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c04ef30 con 0x7fc24c0add80 2024-04-25T10:24:30.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.572+0000 7fc242a00640 1 -- 172.21.15.87:0/228938742 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc24401c070 con 0x7fc24c0add80 2024-04-25T10:24:30.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.572+0000 7fc242a00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc22c066f60 0x7fc22c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.572+0000 7fc242a00640 1 -- 172.21.15.87:0/228938742 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2440842b0 con 0x7fc24c0add80 2024-04-25T10:24:30.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.572+0000 7fc24aa00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc22c066f60 0x7fc22c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.572+0000 7fc24aa00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc22c066f60 0x7fc22c069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc23c001f90 tx=0x7fc23c001800 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.576+0000 7fc242a00640 1 -- 172.21.15.87:0/228938742 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc244054b40 con 0x7fc24c0add80 2024-04-25T10:24:30.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.760+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc24c155770 con 0x7fc24c0add80 2024-04-25T10:24:30.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.760+0000 7fc242a00640 1 -- 172.21.15.87:0/228938742 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4955 (secure 0 0 0) 0x7fc244058be0 con 0x7fc24c0add80 2024-04-25T10:24:30.761 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:30.761 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":111,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4139990719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4139990719},{"type":"v1","addr":"172.21.15.169:6834","nonce":4139990719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3962905873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3962905873},{"type":"v1","addr":"172.21.15.169:6835","nonce":3962905873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3473546915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3473546915},{"type":"v1","addr":"172.21.15.87:6836","nonce":3473546915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:23:47.224714+0000","modified":"2024-04-25T10:23:50.259650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10044":{"gid":10044,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/564498676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":564498676},{"type":"v1","addr":"172.21.15.87:6837","nonce":564498676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:24:30.761 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 111 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc22c066f60 msgr2=0x7fc22c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc22c066f60 0x7fc22c069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc23c001f90 tx=0x7fc23c001800 comp rx=0 tx=0).stop 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc24c0add80 msgr2=0x7fc24c0cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc24c0add80 0x7fc24c0cc7f0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fc24c070930 tx=0x7fc24400e820 comp rx=0 tx=0).stop 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 shutdown_connections 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc22c066f60 0x7fc22c069420 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc24c0add80 0x7fc24c0cc7f0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc24c14d580 0x7fc24c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 --2- 172.21.15.87:0/228938742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc230004810 0x7fc24c0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 >> 172.21.15.87:0/228938742 conn(0x7fc24c066b70 msgr2=0x7fc24c06d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:30.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 shutdown_connections 2024-04-25T10:24:30.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.764+0000 7fc251c00640 1 -- 172.21.15.87:0/228938742 wait complete. 2024-04-25T10:24:30.782 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:30.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.916+0000 7f8d95000640 1 Processor -- start 2024-04-25T10:24:30.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d95000640 1 -- start start 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d95000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d90071e50 0x7f8d901472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d95000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9006c600 con 0x7f8d900726b0 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d95000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d90161bd0 con 0x7f8d90146ed0 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d95000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d90161980 con 0x7f8d90071e50 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d8e800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8d900726b0 0x7f8d900ad840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41312/0 (socket says 172.21.15.87:41312) 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d8e800640 1 -- 172.21.15.87:0/819530321 learned_addr learned my addr 172.21.15.87:0/819530321 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d85400640 1 --2- 172.21.15.87:0/819530321 >> v2:172.21.15.169:3300/0 conn(0x7f8d90071e50 0x7f8d901472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610577246 0 0) 0x7f8d9006c600 con 0x7f8d900726b0 2024-04-25T10:24:30.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d74003530 con 0x7f8d900726b0 2024-04-25T10:24:30.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478898646 0 0) 0x7f8d90161bd0 con 0x7f8d90146ed0 2024-04-25T10:24:30.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d9006c600 con 0x7f8d90146ed0 2024-04-25T10:24:30.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4142592310 0 0) 0x7f8d74003530 con 0x7f8d900726b0 2024-04-25T10:24:30.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d85400640 1 -- 172.21.15.87:0/819530321 >> v1:172.21.15.169:6789/0 conn(0x7f8d90146ed0 legacy=0x7f8d90071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d85400640 1 -- 172.21.15.87:0/819530321 >> v1:172.21.15.87:6789/0 conn(0x7f8d900726b0 legacy=0x7f8d900ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.920+0000 7f8d85400640 1 -- 172.21.15.87:0/819530321 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90161eb0 con 0x7f8d90071e50 2024-04-25T10:24:30.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d85400640 1 --2- 172.21.15.87:0/819530321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d901472b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8d780097c0 tx=0x7f8d7801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb02dc23657d8ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780b1070 con 0x7f8d90071e50 2024-04-25T10:24:30.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d780ac440 con 0x7f8d90071e50 2024-04-25T10:24:30.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d8d400640 1 -- 172.21.15.87:0/819530321 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88001680 con 0x7f8d90071e50 2024-04-25T10:24:30.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 -- 172.21.15.87:0/819530321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 msgr2=0x7f8d901472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 --2- 172.21.15.87:0/819530321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d901472b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8d780097c0 tx=0x7f8d7801c390 comp rx=0 tx=0).stop 2024-04-25T10:24:30.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 -- 172.21.15.87:0/819530321 shutdown_connections 2024-04-25T10:24:30.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 --2- 172.21.15.87:0/819530321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d901472b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 -- 172.21.15.87:0/819530321 >> 172.21.15.87:0/819530321 conn(0x7f8d900669a0 msgr2=0x7f8d90066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:30.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 -- 172.21.15.87:0/819530321 shutdown_connections 2024-04-25T10:24:30.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 -- 172.21.15.87:0/819530321 wait complete. 2024-04-25T10:24:30.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 Processor -- start 2024-04-25T10:24:30.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.924+0000 7f8d95000640 1 -- start start 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d90160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d900726b0 0x7f8d900b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d90146ed0 0x7f8d9015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900ae8e0 con 0x7f8d900726b0 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900ae760 con 0x7f8d90071e50 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900aea60 con 0x7f8d90146ed0 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d900726b0 0x7f8d900b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d900726b0 0x7f8d900b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41164/0 (socket says 172.21.15.87:41164) 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 -- 172.21.15.87:0/1533672852 learned_addr learned my addr 172.21.15.87:0/1533672852 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 -- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d90146ed0 msgr2=0x7f8d9015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d90146ed0 0x7f8d9015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 -- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 msgr2=0x7f8d90160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d8de00640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d90160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d90160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 -- 172.21.15.87:0/1533672852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90140bf0 con 0x7f8d900726b0 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d8de00640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d90160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d85400640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d900726b0 0x7f8d900b4880 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f8d78018fa0 tx=0x7f8d78019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d86a00640 1 -- 172.21.15.87:0/1533672852 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780b1040 con 0x7f8d900726b0 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d86a00640 1 -- 172.21.15.87:0/1533672852 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d780036a0 con 0x7f8d900726b0 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d86a00640 1 -- 172.21.15.87:0/1533672852 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78003a50 con 0x7f8d900726b0 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d90163cf0 con 0x7f8d900726b0 2024-04-25T10:24:30.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d90162c50 con 0x7f8d900726b0 2024-04-25T10:24:30.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9006ed20 con 0x7f8d900726b0 2024-04-25T10:24:30.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.928+0000 7f8d86a00640 1 -- 172.21.15.87:0/1533672852 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d780bc030 con 0x7f8d900726b0 2024-04-25T10:24:30.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.932+0000 7f8d86a00640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d6c066f60 0x7f8d6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:30.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.932+0000 7f8d86a00640 1 -- 172.21.15.87:0/1533672852 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d7811e5d0 con 0x7f8d900726b0 2024-04-25T10:24:30.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.932+0000 7f8d8de00640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d6c066f60 0x7f8d6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:30.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.932+0000 7f8d8de00640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d6c066f60 0x7f8d6c069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8d80001570 tx=0x7f8d80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:30.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:30.936+0000 7f8d86a00640 1 -- 172.21.15.87:0/1533672852 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d780c3020 con 0x7f8d900726b0 2024-04-25T10:24:31.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.092+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d900670e0 con 0x7f8d900726b0 2024-04-25T10:24:31.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.092+0000 7f8d86a00640 1 -- 172.21.15.87:0/1533672852 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16659 (secure 0 0 0) 0x7f8d78015fb0 con 0x7f8d900726b0 2024-04-25T10:24:31.093 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:31.093 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":151,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:48.878767+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:23:45.609544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:23:46.223978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:31.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d6c066f60 msgr2=0x7f8d6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d6c066f60 0x7f8d6c069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8d80001570 tx=0x7f8d80009040 comp rx=0 tx=0).stop 2024-04-25T10:24:31.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d900726b0 msgr2=0x7f8d900b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d900726b0 0x7f8d900b4880 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f8d78018fa0 tx=0x7f8d78019130 comp rx=0 tx=0).stop 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 shutdown_connections 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d6c066f60 0x7f8d6c069420 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d90146ed0 0x7f8d9015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d900726b0 0x7f8d900b4880 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 --2- 172.21.15.87:0/1533672852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d90071e50 0x7f8d90160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 >> 172.21.15.87:0/1533672852 conn(0x7f8d900669a0 msgr2=0x7f8d9006daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 shutdown_connections 2024-04-25T10:24:31.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.096+0000 7f8d95000640 1 -- 172.21.15.87:0/1533672852 wait complete. 2024-04-25T10:24:31.114 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:24:31.115 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:31.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8c800640 1 Processor -- start 2024-04-25T10:24:31.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8c800640 1 -- start start 2024-04-25T10:24:31.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8c800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c8415dce0 0x7f7c840669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8c800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c840ad850 0x7f7c840adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8c800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8406e6a0 con 0x7f7c8406eec0 2024-04-25T10:24:31.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8c800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c840674d0 con 0x7f7c8415dce0 2024-04-25T10:24:31.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8c800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8415e0c0 con 0x7f7c840ad850 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c89e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c8415dce0 0x7f7c840669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c89e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c8415dce0 0x7f7c840669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41166/0 (socket says 172.21.15.87:41166) 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c89e00640 1 -- 172.21.15.87:0/2098478884 learned_addr learned my addr 172.21.15.87:0/2098478884 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c8a800640 1 --2- 172.21.15.87:0/2098478884 >> v2:172.21.15.169:3300/0 conn(0x7f7c840ad850 0x7f7c840adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c89e00640 1 -- 172.21.15.87:0/2098478884 >> v2:172.21.15.169:3300/0 conn(0x7f7c840ad850 msgr2=0x7f7c840adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c89e00640 1 --2- 172.21.15.87:0/2098478884 >> v2:172.21.15.169:3300/0 conn(0x7f7c840ad850 0x7f7c840adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c89e00640 1 -- 172.21.15.87:0/2098478884 >> v1:172.21.15.169:6789/0 conn(0x7f7c8406eec0 legacy=0x7f7c8406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:31.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.252+0000 7f7c89e00640 1 -- 172.21.15.87:0/2098478884 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840671f0 con 0x7f7c8415dce0 2024-04-25T10:24:31.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c89e00640 1 --2- 172.21.15.87:0/2098478884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8415dce0 0x7f7c840669a0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f7c840715d0 tx=0x7f7c7801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d562a07f2a64dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c88a00640 1 -- 172.21.15.87:0/2098478884 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780b0070 con 0x7f7c8415dce0 2024-04-25T10:24:31.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c88a00640 1 -- 172.21.15.87:0/2098478884 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c780ab600 con 0x7f7c8415dce0 2024-04-25T10:24:31.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c88a00640 1 -- 172.21.15.87:0/2098478884 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780ab9d0 con 0x7f7c8415dce0 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 -- 172.21.15.87:0/2098478884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8415dce0 msgr2=0x7f7c840669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 --2- 172.21.15.87:0/2098478884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8415dce0 0x7f7c840669a0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f7c840715d0 tx=0x7f7c7801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 -- 172.21.15.87:0/2098478884 shutdown_connections 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 --2- 172.21.15.87:0/2098478884 >> v2:172.21.15.169:3300/0 conn(0x7f7c840ad850 0x7f7c840adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 --2- 172.21.15.87:0/2098478884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8415dce0 0x7f7c840669a0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 -- 172.21.15.87:0/2098478884 >> 172.21.15.87:0/2098478884 conn(0x7f7c840b2410 msgr2=0x7f7c840b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 -- 172.21.15.87:0/2098478884 shutdown_connections 2024-04-25T10:24:31.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.256+0000 7f7c8c800640 1 -- 172.21.15.87:0/2098478884 wait complete. 2024-04-25T10:24:31.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 Processor -- start 2024-04-25T10:24:31.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 -- start start 2024-04-25T10:24:31.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8406eec0 0x7f7c8415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c840ad850 0x7f7c8415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c8415dce0 0x7f7c840c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840d0320 con 0x7f7c8406eec0 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84066ee0 con 0x7f7c8415dce0 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840d04a0 con 0x7f7c840ad850 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8406eec0 0x7f7c8415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8406eec0 0x7f7c8415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41182/0 (socket says 172.21.15.87:41182) 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 -- 172.21.15.87:0/4145102849 learned_addr learned my addr 172.21.15.87:0/4145102849 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89400640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c840ad850 0x7f7c8415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8a800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c8415dce0 0x7f7c840c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 -- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c840ad850 msgr2=0x7f7c8415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c840ad850 0x7f7c8415abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 -- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c8415dce0 msgr2=0x7f7c840c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c8415dce0 0x7f7c840c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 -- 172.21.15.87:0/4145102849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840cb6b0 con 0x7f7c8406eec0 2024-04-25T10:24:31.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8a800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c8415dce0 0x7f7c840c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:31.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c89e00640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8406eec0 0x7f7c8415a680 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ae040 tx=0x7f7c7801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c7e000640 1 -- 172.21.15.87:0/4145102849 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780ba020 con 0x7f7c8406eec0 2024-04-25T10:24:31.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c7e000640 1 -- 172.21.15.87:0/4145102849 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c780b6640 con 0x7f7c8406eec0 2024-04-25T10:24:31.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c840cb990 con 0x7f7c8406eec0 2024-04-25T10:24:31.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c7e000640 1 -- 172.21.15.87:0/4145102849 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780b0040 con 0x7f7c8406eec0 2024-04-25T10:24:31.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.260+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c840cbf50 con 0x7f7c8406eec0 2024-04-25T10:24:31.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.264+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8404ef30 con 0x7f7c8406eec0 2024-04-25T10:24:31.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.264+0000 7f7c7e000640 1 -- 172.21.15.87:0/4145102849 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c780bd020 con 0x7f7c8406eec0 2024-04-25T10:24:31.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.264+0000 7f7c7e000640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c68066f90 0x7f7c68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.264+0000 7f7c7e000640 1 -- 172.21.15.87:0/4145102849 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c7811f280 con 0x7f7c8406eec0 2024-04-25T10:24:31.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.264+0000 7f7c89400640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c68066f90 0x7f7c68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.264+0000 7f7c89400640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c68066f90 0x7f7c68069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7c840b0370 tx=0x7f7c74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.268+0000 7f7c7e000640 1 -- 172.21.15.87:0/4145102849 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c780c4030 con 0x7f7c8406eec0 2024-04-25T10:24:31.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.452+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c840af770 con 0x7f7c8406eec0 2024-04-25T10:24:31.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.452+0000 7f7c7e000640 1 -- 172.21.15.87:0/4145102849 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4955 (secure 0 0 0) 0x7f7c780160d0 con 0x7f7c8406eec0 2024-04-25T10:24:31.454 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:31.455 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":111,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4139990719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4139990719},{"type":"v1","addr":"172.21.15.169:6834","nonce":4139990719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3962905873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3962905873},{"type":"v1","addr":"172.21.15.169:6835","nonce":3962905873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3473546915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3473546915},{"type":"v1","addr":"172.21.15.87:6836","nonce":3473546915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:23:47.224714+0000","modified":"2024-04-25T10:23:50.259650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10044":{"gid":10044,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/564498676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":564498676},{"type":"v1","addr":"172.21.15.87:6837","nonce":564498676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:24:31.455 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 111 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c68066f90 msgr2=0x7f7c68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c68066f90 0x7f7c68069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7c840b0370 tx=0x7f7c74008040 comp rx=0 tx=0).stop 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8406eec0 msgr2=0x7f7c8415a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8406eec0 0x7f7c8415a680 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ae040 tx=0x7f7c7801ac00 comp rx=0 tx=0).stop 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 shutdown_connections 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c68066f90 0x7f7c68069450 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c8415dce0 0x7f7c840c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c840ad850 0x7f7c8415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 --2- 172.21.15.87:0/4145102849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c8406eec0 0x7f7c8415a680 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 >> 172.21.15.87:0/4145102849 conn(0x7f7c840b2410 msgr2=0x7f7c840b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 shutdown_connections 2024-04-25T10:24:31.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.456+0000 7f7c8c800640 1 -- 172.21.15.87:0/4145102849 wait complete. 2024-04-25T10:24:31.475 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:31.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2c200640 1 Processor -- start 2024-04-25T10:24:31.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2c200640 1 -- start start 2024-04-25T10:24:31.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2c200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8a240cda10 0x7f8a2406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2c200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a240cf930 0x7f8a240cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2c200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a240b2770 con 0x7f8a2415bc40 2024-04-25T10:24:31.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2c200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2406c670 con 0x7f8a240cda10 2024-04-25T10:24:31.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2c200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2406c950 con 0x7f8a240cf930 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a28e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8a240cda10 0x7f8a2406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a240cf930 0x7f8a240cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a29800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8a2415bc40 0x7f8a240b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:59744/0 (socket says 172.21.15.87:59744) 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a29800640 1 -- 172.21.15.87:0/3953531018 learned_addr learned my addr 172.21.15.87:0/3953531018 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a23e00640 1 -- 172.21.15.87:0/3953531018 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981991493 0 0) 0x7f8a240b2770 con 0x7f8a2415bc40 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a23e00640 1 -- 172.21.15.87:0/3953531018 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a0c003530 con 0x7f8a2415bc40 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a28e00640 1 -- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a240cf930 msgr2=0x7f8a240cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a28e00640 1 --2- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a240cf930 0x7f8a240cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a28e00640 1 -- 172.21.15.87:0/3953531018 >> v1:172.21.15.169:6789/0 conn(0x7f8a2415bc40 legacy=0x7f8a240b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a28e00640 1 -- 172.21.15.87:0/3953531018 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2406cc30 con 0x7f8a240cda10 2024-04-25T10:24:31.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.612+0000 7f8a2a200640 1 --2- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a240cf930 0x7f8a240cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:31.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a28e00640 1 --2- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cda10 0x7f8a2406c090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a100097c0 tx=0x7f8a1001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d1641d4cc1f660f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a23e00640 1 -- 172.21.15.87:0/3953531018 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100b0070 con 0x7f8a240cda10 2024-04-25T10:24:31.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a23e00640 1 -- 172.21.15.87:0/3953531018 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a100ab600 con 0x7f8a240cda10 2024-04-25T10:24:31.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a23e00640 1 -- 172.21.15.87:0/3953531018 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100ab9d0 con 0x7f8a240cda10 2024-04-25T10:24:31.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 -- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cda10 msgr2=0x7f8a2406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cda10 0x7f8a2406c090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a100097c0 tx=0x7f8a1001c410 comp rx=0 tx=0).stop 2024-04-25T10:24:31.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 -- 172.21.15.87:0/3953531018 shutdown_connections 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a240cf930 0x7f8a240cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3953531018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cda10 0x7f8a2406c090 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 -- 172.21.15.87:0/3953531018 >> 172.21.15.87:0/3953531018 conn(0x7f8a24066b70 msgr2=0x7f8a24066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 -- 172.21.15.87:0/3953531018 shutdown_connections 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.616+0000 7f8a2c200640 1 -- 172.21.15.87:0/3953531018 wait complete. 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 Processor -- start 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 -- start start 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a240cda10 0x7f8a24152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cf930 0x7f8a240af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2415bc40 0x7f8a24071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b3890 con 0x7f8a240cda10 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b3710 con 0x7f8a240cf930 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b3a10 con 0x7f8a2415bc40 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a240cda10 0x7f8a24152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a240cda10 0x7f8a24152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41192/0 (socket says 172.21.15.87:41192) 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 -- 172.21.15.87:0/3477001808 learned_addr learned my addr 172.21.15.87:0/3477001808 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a28e00640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cf930 0x7f8a240af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2a200640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2415bc40 0x7f8a24071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 -- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2415bc40 msgr2=0x7f8a24071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2415bc40 0x7f8a24071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 -- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cf930 msgr2=0x7f8a240af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cf930 0x7f8a240af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 -- 172.21.15.87:0/3477001808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240708d0 con 0x7f8a240cda10 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a29800640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a240cda10 0x7f8a24152e90 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800c3c0 tx=0x7f8a1800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a21600640 1 -- 172.21.15.87:0/3477001808 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18017070 con 0x7f8a240cda10 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a21600640 1 -- 172.21.15.87:0/3477001808 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a18005db0 con 0x7f8a240cda10 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a21600640 1 -- 172.21.15.87:0/3477001808 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a180072f0 con 0x7f8a240cda10 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a24070bb0 con 0x7f8a240cda10 2024-04-25T10:24:31.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a24071170 con 0x7f8a240cda10 2024-04-25T10:24:31.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.620+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2404ef30 con 0x7f8a240cda10 2024-04-25T10:24:31.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.624+0000 7f8a21600640 1 -- 172.21.15.87:0/3477001808 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a1801a070 con 0x7f8a240cda10 2024-04-25T10:24:31.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.624+0000 7f8a21600640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a04066f60 0x7f8a04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.624+0000 7f8a21600640 1 -- 172.21.15.87:0/3477001808 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a18084480 con 0x7f8a240cda10 2024-04-25T10:24:31.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.624+0000 7f8a28e00640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a04066f60 0x7f8a04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.624+0000 7f8a28e00640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a04066f60 0x7f8a04069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a100ad000 tx=0x7f8a10008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.628+0000 7f8a21600640 1 -- 172.21.15.87:0/3477001808 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a18054da0 con 0x7f8a240cda10 2024-04-25T10:24:31.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.812+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a2406d960 con 0x7f8a240cda10 2024-04-25T10:24:31.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.812+0000 7f8a21600640 1 -- 172.21.15.87:0/3477001808 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4955 (secure 0 0 0) 0x7f8a18058e40 con 0x7f8a240cda10 2024-04-25T10:24:31.812 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:31.812 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":111,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4139990719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4139990719},{"type":"v1","addr":"172.21.15.169:6834","nonce":4139990719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3962905873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3962905873},{"type":"v1","addr":"172.21.15.169:6835","nonce":3962905873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3473546915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3473546915},{"type":"v1","addr":"172.21.15.87:6836","nonce":3473546915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:23:47.224714+0000","modified":"2024-04-25T10:23:50.259650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10044":{"gid":10044,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/564498676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":564498676},{"type":"v1","addr":"172.21.15.87:6837","nonce":564498676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:24:31.812 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 111 2024-04-25T10:24:31.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a04066f60 msgr2=0x7f8a04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a04066f60 0x7f8a04069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a100ad000 tx=0x7f8a10008040 comp rx=0 tx=0).stop 2024-04-25T10:24:31.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a240cda10 msgr2=0x7f8a24152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a240cda10 0x7f8a24152e90 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800c3c0 tx=0x7f8a1800c7c0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 shutdown_connections 2024-04-25T10:24:31.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8a04066f60 0x7f8a04069420 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a2415bc40 0x7f8a24071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a240cf930 0x7f8a240af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 --2- 172.21.15.87:0/3477001808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a240cda10 0x7f8a24152e90 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 >> 172.21.15.87:0/3477001808 conn(0x7f8a24066b70 msgr2=0x7f8a24158660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:31.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 shutdown_connections 2024-04-25T10:24:31.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.816+0000 7f8a2c200640 1 -- 172.21.15.87:0/3477001808 wait complete. 2024-04-25T10:24:31.834 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:31.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.968+0000 7f5bbf200640 1 Processor -- start 2024-04-25T10:24:31.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.968+0000 7f5bbf200640 1 -- start start 2024-04-25T10:24:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bbf200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5bb8071f70 0x7f5bb80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bbf200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5bb80b2fc0 0x7f5bb806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bbf200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb8070ed0 con 0x7f5bb815a0e0 2024-04-25T10:24:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bbf200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80b3420 con 0x7f5bb8071f70 2024-04-25T10:24:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bbf200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80711b0 con 0x7f5bb80b2fc0 2024-04-25T10:24:31.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5bb8071f70 0x7f5bb80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5bb8071f70 0x7f5bb80b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41194/0 (socket says 172.21.15.87:41194) 2024-04-25T10:24:31.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 -- 172.21.15.87:0/3638051478 learned_addr learned my addr 172.21.15.87:0/3638051478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:31.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 -- 172.21.15.87:0/3638051478 >> v2:172.21.15.169:3300/0 conn(0x7f5bb80b2fc0 msgr2=0x7f5bb806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:31.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 --2- 172.21.15.87:0/3638051478 >> v2:172.21.15.169:3300/0 conn(0x7f5bb80b2fc0 0x7f5bb806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 -- 172.21.15.87:0/3638051478 >> v1:172.21.15.169:6789/0 conn(0x7f5bb815a0e0 legacy=0x7f5bb815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:31.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 -- 172.21.15.87:0/3638051478 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8071490 con 0x7f5bb8071f70 2024-04-25T10:24:31.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb7400640 1 --2- 172.21.15.87:0/3638051478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb8071f70 0x7f5bb80b2a80 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80d0670 tx=0x7f5ba401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3420d9c560efbc5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb6a00640 1 -- 172.21.15.87:0/3638051478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40ab430 con 0x7f5bb8071f70 2024-04-25T10:24:31.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb6a00640 1 -- 172.21.15.87:0/3638051478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba40b0070 con 0x7f5bb8071f70 2024-04-25T10:24:31.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.972+0000 7f5bb6a00640 1 -- 172.21.15.87:0/3638051478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40ab7f0 con 0x7f5bb8071f70 2024-04-25T10:24:31.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- 172.21.15.87:0/3638051478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb8071f70 msgr2=0x7f5bb80b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 --2- 172.21.15.87:0/3638051478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb8071f70 0x7f5bb80b2a80 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80d0670 tx=0x7f5ba401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- 172.21.15.87:0/3638051478 shutdown_connections 2024-04-25T10:24:31.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 --2- 172.21.15.87:0/3638051478 >> v2:172.21.15.169:3300/0 conn(0x7f5bb80b2fc0 0x7f5bb806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 --2- 172.21.15.87:0/3638051478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb8071f70 0x7f5bb80b2a80 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- 172.21.15.87:0/3638051478 >> 172.21.15.87:0/3638051478 conn(0x7f5bb8066b70 msgr2=0x7f5bb8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- 172.21.15.87:0/3638051478 shutdown_connections 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- 172.21.15.87:0/3638051478 wait complete. 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 Processor -- start 2024-04-25T10:24:31.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- start start 2024-04-25T10:24:31.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb8071f70 0x7f5bb815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bb80b2fc0 0x7f5bb8148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb815a0e0 0x7f5bb815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8070b50 con 0x7f5bb815a0e0 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb815a4c0 con 0x7f5bb8071f70 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80723f0 con 0x7f5bb80b2fc0 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbd200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb815a0e0 0x7f5bb815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bb7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb8071f70 0x7f5bb815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bb7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bb80b2fc0 0x7f5bb8148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbd200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb815a0e0 0x7f5bb815c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41206/0 (socket says 172.21.15.87:41206) 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbd200640 1 -- 172.21.15.87:0/1996066450 learned_addr learned my addr 172.21.15.87:0/1996066450 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbd200640 1 -- 172.21.15.87:0/1996066450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bb80b2fc0 msgr2=0x7f5bb8148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbd200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bb80b2fc0 0x7f5bb8148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbd200640 1 -- 172.21.15.87:0/1996066450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb8071f70 msgr2=0x7f5bb815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:31.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.976+0000 7f5bbd200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb8071f70 0x7f5bb815a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:31.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bbd200640 1 -- 172.21.15.87:0/1996066450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb815c930 con 0x7f5bb815a0e0 2024-04-25T10:24:31.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bbd200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb815a0e0 0x7f5bb815c0c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b460 tx=0x7f5bb000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bafe00640 1 -- 172.21.15.87:0/1996066450 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0015020 con 0x7f5bb815a0e0 2024-04-25T10:24:31.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bafe00640 1 -- 172.21.15.87:0/1996066450 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0011890 con 0x7f5bb815a0e0 2024-04-25T10:24:31.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bafe00640 1 -- 172.21.15.87:0/1996066450 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0011c80 con 0x7f5bb815a0e0 2024-04-25T10:24:31.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb815cc10 con 0x7f5bb815a0e0 2024-04-25T10:24:31.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb815d1d0 con 0x7f5bb815a0e0 2024-04-25T10:24:31.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb804ef30 con 0x7f5bb815a0e0 2024-04-25T10:24:31.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bafe00640 1 -- 172.21.15.87:0/1996066450 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bb0018020 con 0x7f5bb815a0e0 2024-04-25T10:24:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bafe00640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b98066f90 0x7f5b98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bafe00640 1 -- 172.21.15.87:0/1996066450 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb0084ff0 con 0x7f5bb815a0e0 2024-04-25T10:24:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.980+0000 7f5bb7e00640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b98066f90 0x7f5b98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:31.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.984+0000 7f5bb7e00640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b98066f90 0x7f5b98069450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009fd0 tx=0x7f5ba800d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:31.984+0000 7f5bafe00640 1 -- 172.21.15.87:0/1996066450 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bb001f020 con 0x7f5bb815a0e0 2024-04-25T10:24:32.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.140+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bb80703e0 con 0x7f5bb815a0e0 2024-04-25T10:24:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.140+0000 7f5bafe00640 1 -- 172.21.15.87:0/1996066450 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16659 (secure 0 0 0) 0x7f5bb000d040 con 0x7f5bb815a0e0 2024-04-25T10:24:32.141 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:32.142 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":151,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:23:48.878767+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:23:45.609544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:23:46.223978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b98066f90 msgr2=0x7f5b98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b98066f90 0x7f5b98069450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009fd0 tx=0x7f5ba800d040 comp rx=0 tx=0).stop 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb815a0e0 msgr2=0x7f5bb815c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb815a0e0 0x7f5bb815c0c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b460 tx=0x7f5bb000b920 comp rx=0 tx=0).stop 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 shutdown_connections 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5b98066f90 0x7f5b98069450 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bb815a0e0 0x7f5bb815c0c0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bb80b2fc0 0x7f5bb8148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 --2- 172.21.15.87:0/1996066450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb8071f70 0x7f5bb815a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 >> 172.21.15.87:0/1996066450 conn(0x7f5bb8066b70 msgr2=0x7f5bb814d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 shutdown_connections 2024-04-25T10:24:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.144+0000 7f5bbf200640 1 -- 172.21.15.87:0/1996066450 wait complete. 2024-04-25T10:24:32.163 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:24:32.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdafa600640 1 Processor -- start 2024-04-25T10:24:32.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdafa600640 1 -- start start 2024-04-25T10:24:32.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdafa600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdaf41550b0 0x7fdaf40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdafa600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf4071050 con 0x7fdaf40cfe70 2024-04-25T10:24:32.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdafa600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf415d040 con 0x7fdaf415b910 2024-04-25T10:24:32.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdafa600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf415bd80 con 0x7fdaf41550b0 2024-04-25T10:24:32.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdaf2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fdaf40cfe70 0x7fdaf415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41324/0 (socket says 172.21.15.87:41324) 2024-04-25T10:24:32.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdaf2a00640 1 -- 172.21.15.87:0/1376470874 learned_addr learned my addr 172.21.15.87:0/1376470874 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:32.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1376470874 >> v2:172.21.15.169:3300/0 conn(0x7fdaf41550b0 0x7fdaf40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404474025 0 0) 0x7fdaf4071050 con 0x7fdaf40cfe70 2024-04-25T10:24:32.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.300+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad8003530 con 0x7fdaf40cfe70 2024-04-25T10:24:32.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 638731391 0 0) 0x7fdad8003530 con 0x7fdaf40cfe70 2024-04-25T10:24:32.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 >> v2:172.21.15.169:3300/0 conn(0x7fdaf41550b0 msgr2=0x7fdaf40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:32.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 --2- 172.21.15.87:0/1376470874 >> v2:172.21.15.169:3300/0 conn(0x7fdaf41550b0 0x7fdaf40cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 >> v1:172.21.15.169:6789/0 conn(0x7fdaf415b910 legacy=0x7fdaf40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf40712f0 con 0x7fdaf40cfe70 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fdae0001680 con 0x7fdaf40cfe70 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1376470874 >> v2:172.21.15.169:3300/0 conn(0x7fdaf41550b0 0x7fdaf40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 >> v1:172.21.15.87:6789/0 conn(0x7fdaf40cfe70 legacy=0x7fdaf415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdad8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad8004810 0x7fdad8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad80051d0 0x7fdad800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf415bd80 con 0x7fdad80039c0 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad800c0a0 con 0x7fdad80051d0 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad800c2a0 con 0x7fdad8004810 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdad8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3e00640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad80051d0 0x7fdad800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf2a00640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad8004810 0x7fdad8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 -- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad8004810 msgr2=0x7fdad8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:32.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad8004810 0x7fdad8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 -- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad80051d0 msgr2=0x7fdad800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:32.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad80051d0 0x7fdad800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 -- 172.21.15.87:0/1376470874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8003530 con 0x7fdad80039c0 2024-04-25T10:24:32.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3e00640 1 -- 172.21.15.87:0/1376470874 reap_dead start 2024-04-25T10:24:32.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.304+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdad8003da0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fdae40098d0 tx=0x7fdae401c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6ef7ef20f5b626 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:32.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae400f040 con 0x7fdad80039c0 2024-04-25T10:24:32.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae40af070 con 0x7fdad80039c0 2024-04-25T10:24:32.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf2000640 1 -- 172.21.15.87:0/1376470874 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40aa880 con 0x7fdad80039c0 2024-04-25T10:24:32.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- 172.21.15.87:0/1376470874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 msgr2=0x7fdad8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdad8003da0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fdae40098d0 tx=0x7fdae401c590 comp rx=0 tx=0).stop 2024-04-25T10:24:32.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- 172.21.15.87:0/1376470874 shutdown_connections 2024-04-25T10:24:32.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 --2- 172.21.15.87:0/1376470874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdad8003da0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- 172.21.15.87:0/1376470874 >> 172.21.15.87:0/1376470874 conn(0x7fdaf4066b70 msgr2=0x7fdaf4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:32.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- 172.21.15.87:0/1376470874 shutdown_connections 2024-04-25T10:24:32.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- 172.21.15.87:0/1376470874 wait complete. 2024-04-25T10:24:32.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 Processor -- start 2024-04-25T10:24:32.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- start start 2024-04-25T10:24:32.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdaf4148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaf41f3e60 0x7fdaf41f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf406d0a0 0x7fdaf406d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4071b90 con 0x7fdad80039c0 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40718b0 con 0x7fdaf406d0a0 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdafa600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40720b0 con 0x7fdaf41f3e60 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdaf4148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdaf4148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41220/0 (socket says 172.21.15.87:41220) 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf3400640 1 -- 172.21.15.87:0/1518737063 learned_addr learned my addr 172.21.15.87:0/1518737063 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf2a00640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaf41f3e60 0x7fdaf41f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.308+0000 7fdaf3e00640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf406d0a0 0x7fdaf406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaf3400640 1 -- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaf41f3e60 msgr2=0x7fdaf41f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaf41f3e60 0x7fdaf41f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaf3400640 1 -- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf406d0a0 msgr2=0x7fdaf406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf406d0a0 0x7fdaf406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaf3400640 1 -- 172.21.15.87:0/1518737063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf40b0150 con 0x7fdad80039c0 2024-04-25T10:24:32.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaf3400640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdaf4148180 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fdae401c3c0 tx=0x7fdad8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:32.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaeb400640 1 -- 172.21.15.87:0/1518737063 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40c1070 con 0x7fdad80039c0 2024-04-25T10:24:32.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaeb400640 1 -- 172.21.15.87:0/1518737063 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae40aab80 con 0x7fdad80039c0 2024-04-25T10:24:32.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf40b0430 con 0x7fdad80039c0 2024-04-25T10:24:32.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaeb400640 1 -- 172.21.15.87:0/1518737063 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40bc410 con 0x7fdad80039c0 2024-04-25T10:24:32.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf40b09f0 con 0x7fdad80039c0 2024-04-25T10:24:32.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf406da60 con 0x7fdad80039c0 2024-04-25T10:24:32.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaeb400640 1 -- 172.21.15.87:0/1518737063 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdae40c4070 con 0x7fdad80039c0 2024-04-25T10:24:32.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaeb400640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdad4066f90 0x7fdad4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaeb400640 1 -- 172.21.15.87:0/1518737063 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdae411f0b0 con 0x7fdad80039c0 2024-04-25T10:24:32.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.312+0000 7fdaf2a00640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdad4066f90 0x7fdad4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.316+0000 7fdaf2a00640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdad4066f90 0x7fdad4069450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4072670 tx=0x7fdae0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:32.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.316+0000 7fdaeb400640 1 -- 172.21.15.87:0/1518737063 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdae40a6040 con 0x7fdad80039c0 2024-04-25T10:24:32.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.504+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdaf41593d0 con 0x7fdad80039c0 2024-04-25T10:24:32.504 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:24:32.504+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:24:32.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.508+0000 7fdaeb400640 1 -- 172.21.15.87:0/1518737063 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdae40162e0 con 0x7fdad80039c0 2024-04-25T10:24:32.507 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:24:32.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.508+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdad4066f90 msgr2=0x7fdad4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.508+0000 7fdafa600640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdad4066f90 0x7fdad4069450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4072670 tx=0x7fdae0009040 comp rx=0 tx=0).stop 2024-04-25T10:24:32.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 msgr2=0x7fdaf4148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdaf4148180 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fdae401c3c0 tx=0x7fdad8005840 comp rx=0 tx=0).stop 2024-04-25T10:24:32.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 shutdown_connections 2024-04-25T10:24:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdad4066f90 0x7fdad4069450 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf406d0a0 0x7fdaf406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaf41f3e60 0x7fdaf41f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 --2- 172.21.15.87:0/1518737063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad80039c0 0x7fdaf4148180 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 >> 172.21.15.87:0/1518737063 conn(0x7fdaf4066b70 msgr2=0x7fdaf4159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 shutdown_connections 2024-04-25T10:24:32.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.512+0000 7fdafa600640 1 -- 172.21.15.87:0/1518737063 wait complete. 2024-04-25T10:24:32.517 INFO:tasks.ceph.mds.a.smithi087.stderr: -3444> 2024-04-25T10:23:49.836+0000 7fa05f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:24:32.531 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:24:32.564 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:24:32.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9ba000640 1 Processor -- start 2024-04-25T10:24:32.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9ba000640 1 -- start start 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9ba000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9b40cc200 0x7fe9b40ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9ba000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b406cf40 0x7fe9b406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9ba000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b40cdab0 con 0x7fe9b40d0d00 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9ba000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b406d8f0 con 0x7fe9b40cc200 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9ba000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40cd210 con 0x7fe9b406cf40 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe9b40d0d00 0x7fe9b4162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41348/0 (socket says 172.21.15.87:41348) 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b3400640 1 -- 172.21.15.87:0/99891487 learned_addr learned my addr 172.21.15.87:0/99891487 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b3e00640 1 --2- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b406cf40 0x7fe9b406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b2a00640 1 --2- 172.21.15.87:0/99891487 >> v2:172.21.15.169:3300/0 conn(0x7fe9b40cc200 0x7fe9b40ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b2000640 1 -- 172.21.15.87:0/99891487 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171295594 0 0) 0x7fe9b40cdab0 con 0x7fe9b40d0d00 2024-04-25T10:24:32.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b2000640 1 -- 172.21.15.87:0/99891487 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe998003530 con 0x7fe9b40d0d00 2024-04-25T10:24:32.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b2a00640 1 -- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b406cf40 msgr2=0x7fe9b406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:32.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b2a00640 1 --2- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b406cf40 0x7fe9b406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b2a00640 1 -- 172.21.15.87:0/99891487 >> v1:172.21.15.87:6789/0 conn(0x7fe9b40d0d00 legacy=0x7fe9b4162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.672+0000 7fe9b2a00640 1 -- 172.21.15.87:0/99891487 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40cd4f0 con 0x7fe9b40cc200 2024-04-25T10:24:32.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9b2a00640 1 --2- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40ccc30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00097c0 tx=0x7fe9a001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55dbd95951c6ff0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:32.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9b2000640 1 -- 172.21.15.87:0/99891487 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00b0070 con 0x7fe9b40cc200 2024-04-25T10:24:32.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9b2000640 1 -- 172.21.15.87:0/99891487 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a00ab600 con 0x7fe9b40cc200 2024-04-25T10:24:32.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9b2000640 1 -- 172.21.15.87:0/99891487 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00ab9d0 con 0x7fe9b40cc200 2024-04-25T10:24:32.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 -- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 msgr2=0x7fe9b40ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 --2- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40ccc30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00097c0 tx=0x7fe9a001c410 comp rx=0 tx=0).stop 2024-04-25T10:24:32.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 -- 172.21.15.87:0/99891487 shutdown_connections 2024-04-25T10:24:32.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 --2- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b406cf40 0x7fe9b406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 --2- 172.21.15.87:0/99891487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40ccc30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 -- 172.21.15.87:0/99891487 >> 172.21.15.87:0/99891487 conn(0x7fe9b4072290 msgr2=0x7fe9b40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:32.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 -- 172.21.15.87:0/99891487 shutdown_connections 2024-04-25T10:24:32.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.676+0000 7fe9ba000640 1 -- 172.21.15.87:0/99891487 wait complete. 2024-04-25T10:24:32.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 Processor -- start 2024-04-25T10:24:32.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 -- start start 2024-04-25T10:24:32.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9b406cf40 0x7fe9b40998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b40d0d00 0x7fe9b40ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4162a10 con 0x7fe9b406cf40 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4162890 con 0x7fe9b40cc200 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b415e040 con 0x7fe9b40d0d00 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9b406cf40 0x7fe9b40998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9b406cf40 0x7fe9b40998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41236/0 (socket says 172.21.15.87:41236) 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 -- 172.21.15.87:0/3667827621 learned_addr learned my addr 172.21.15.87:0/3667827621 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b2a00640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3e00640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b40d0d00 0x7fe9b40ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 -- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b40d0d00 msgr2=0x7fe9b40ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b40d0d00 0x7fe9b40ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 -- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 msgr2=0x7fe9b40c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 -- 172.21.15.87:0/3667827621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40742b0 con 0x7fe9b406cf40 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3e00640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b40d0d00 0x7fe9b40ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b2a00640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9b3400640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9b406cf40 0x7fe9b40998a0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c3f0 tx=0x7fe9a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ab400640 1 -- 172.21.15.87:0/3667827621 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4017070 con 0x7fe9b406cf40 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ab400640 1 -- 172.21.15.87:0/3667827621 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4005cf0 con 0x7fe9b406cf40 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b40cfc70 con 0x7fe9b406cf40 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ab400640 1 -- 172.21.15.87:0/3667827621 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40072f0 con 0x7fe9b406cf40 2024-04-25T10:24:32.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b41596b0 con 0x7fe9b406cf40 2024-04-25T10:24:32.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.680+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b404ef30 con 0x7fe9b406cf40 2024-04-25T10:24:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.684+0000 7fe9ab400640 1 -- 172.21.15.87:0/3667827621 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9a401a070 con 0x7fe9b406cf40 2024-04-25T10:24:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.684+0000 7fe9ab400640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe994066f90 0x7fe994069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.684+0000 7fe9ab400640 1 -- 172.21.15.87:0/3667827621 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe9a4084540 con 0x7fe9b406cf40 2024-04-25T10:24:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.684+0000 7fe9b2a00640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe994066f90 0x7fe994069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:32.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.684+0000 7fe9b2a00640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe994066f90 0x7fe994069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00ad000 tx=0x7fe9a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:32.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.688+0000 7fe9ab400640 1 -- 172.21.15.87:0/3667827621 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9a4055450 con 0x7fe9b406cf40 2024-04-25T10:24:32.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:32.876+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9b4072ff0 con 0x7fe9b406cf40 2024-04-25T10:24:33.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.508+0000 7fe9ab400640 1 -- 172.21.15.87:0/3667827621 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9a4054d50 con 0x7fe9b406cf40 2024-04-25T10:24:33.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe994066f90 msgr2=0x7fe994069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:33.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe994066f90 0x7fe994069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00ad000 tx=0x7fe9a0008040 comp rx=0 tx=0).stop 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9b406cf40 msgr2=0x7fe9b40998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9b406cf40 0x7fe9b40998a0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c3f0 tx=0x7fe9a400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 shutdown_connections 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe994066f90 0x7fe994069450 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9b40d0d00 0x7fe9b40ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40cc200 0x7fe9b40c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 --2- 172.21.15.87:0/3667827621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9b406cf40 0x7fe9b40998a0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 >> 172.21.15.87:0/3667827621 conn(0x7fe9b4072290 msgr2=0x7fe9b40cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 shutdown_connections 2024-04-25T10:24:33.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.512+0000 7fe9ba000640 1 -- 172.21.15.87:0/3667827621 wait complete. 2024-04-25T10:24:33.534 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:24:33.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.720+0000 7f38daa00640 1 Processor -- start 2024-04-25T10:24:33.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.720+0000 7f38daa00640 1 -- start start 2024-04-25T10:24:33.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38daa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f38d415bcd0 0x7f38d40d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:33.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38daa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40d0370 0x7f38d40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:33.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38daa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40cf5a0 con 0x7f38d40b0930 2024-04-25T10:24:33.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38daa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d40cedb0 con 0x7f38d415bcd0 2024-04-25T10:24:33.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38daa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cf880 con 0x7f38d40d0370 2024-04-25T10:24:33.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38d8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40d0370 0x7f38d40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:33.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38cbe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f38d40b0930 0x7f38d40cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:59778/0 (socket says 172.21.15.87:59778) 2024-04-25T10:24:33.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38cbe00640 1 -- 172.21.15.87:0/740596408 learned_addr learned my addr 172.21.15.87:0/740596408 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:33.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38d3e00640 1 --2- 172.21.15.87:0/740596408 >> v2:172.21.15.169:3300/0 conn(0x7f38d415bcd0 0x7f38d40d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:33.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38d8a00640 1 -- 172.21.15.87:0/740596408 >> v2:172.21.15.169:3300/0 conn(0x7f38d415bcd0 msgr2=0x7f38d40d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:33.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38d8a00640 1 --2- 172.21.15.87:0/740596408 >> v2:172.21.15.169:3300/0 conn(0x7f38d415bcd0 0x7f38d40d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38d3400640 1 -- 172.21.15.87:0/740596408 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892498562 0 0) 0x7f38d40cf5a0 con 0x7f38d40b0930 2024-04-25T10:24:33.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38d8a00640 1 -- 172.21.15.87:0/740596408 >> v1:172.21.15.169:6789/0 conn(0x7f38d40b0930 legacy=0x7f38d40cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:33.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.724+0000 7f38d8a00640 1 -- 172.21.15.87:0/740596408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40cf000 con 0x7f38d40d0370 2024-04-25T10:24:33.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38d8a00640 1 --2- 172.21.15.87:0/740596408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40d0370 0x7f38d40cc930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009d90 tx=0x7f38cc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88eec0ab419d8a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:33.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38d3400640 1 -- 172.21.15.87:0/740596408 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0b2070 con 0x7f38d40d0370 2024-04-25T10:24:33.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38d3400640 1 -- 172.21.15.87:0/740596408 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc0a5830 con 0x7f38d40d0370 2024-04-25T10:24:33.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38d3400640 1 -- 172.21.15.87:0/740596408 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0a5c00 con 0x7f38d40d0370 2024-04-25T10:24:33.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 -- 172.21.15.87:0/740596408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40d0370 msgr2=0x7f38d40cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:33.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 --2- 172.21.15.87:0/740596408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40d0370 0x7f38d40cc930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009d90 tx=0x7f38cc01c970 comp rx=0 tx=0).stop 2024-04-25T10:24:33.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 -- 172.21.15.87:0/740596408 shutdown_connections 2024-04-25T10:24:33.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 --2- 172.21.15.87:0/740596408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40d0370 0x7f38d40cc930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 --2- 172.21.15.87:0/740596408 >> v2:172.21.15.169:3300/0 conn(0x7f38d415bcd0 0x7f38d40d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 -- 172.21.15.87:0/740596408 >> 172.21.15.87:0/740596408 conn(0x7f38d40d19d0 msgr2=0x7f38d40d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:33.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 -- 172.21.15.87:0/740596408 shutdown_connections 2024-04-25T10:24:33.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 -- 172.21.15.87:0/740596408 wait complete. 2024-04-25T10:24:33.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 Processor -- start 2024-04-25T10:24:33.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.728+0000 7f38daa00640 1 -- start start 2024-04-25T10:24:33.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40b0930 0x7f38d406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d40d0370 0x7f38d4148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d415bcd0 0x7f38d40c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4168c60 con 0x7f38d40d0370 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4168de0 con 0x7f38d415bcd0 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4168f60 con 0x7f38d40b0930 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d40d0370 0x7f38d4148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d40d0370 0x7f38d4148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41246/0 (socket says 172.21.15.87:41246) 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 -- 172.21.15.87:0/862347094 learned_addr learned my addr 172.21.15.87:0/862347094 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38d3e00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40b0930 0x7f38d406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38d8a00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d415bcd0 0x7f38d40c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 -- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40b0930 msgr2=0x7f38d406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40b0930 0x7f38d406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 -- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d415bcd0 msgr2=0x7f38d40c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d415bcd0 0x7f38d40c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 -- 172.21.15.87:0/862347094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40b4a70 con 0x7f38d40d0370 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38d8a00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d415bcd0 0x7f38d40c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38cbe00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d40d0370 0x7f38d4148c30 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00c520 tx=0x7f38bc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38d0c00640 1 -- 172.21.15.87:0/862347094 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc018070 con 0x7f38d40d0370 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38d0c00640 1 -- 172.21.15.87:0/862347094 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc0074e0 con 0x7f38d40d0370 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38d0c00640 1 -- 172.21.15.87:0/862347094 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0049f0 con 0x7f38d40d0370 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d40b4d50 con 0x7f38d40d0370 2024-04-25T10:24:33.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d4070260 con 0x7f38d40d0370 2024-04-25T10:24:33.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d404ef30 con 0x7f38d40d0370 2024-04-25T10:24:33.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.732+0000 7f38d0c00640 1 -- 172.21.15.87:0/862347094 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38bc004b90 con 0x7f38d40d0370 2024-04-25T10:24:33.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.736+0000 7f38d0c00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38b0066f60 0x7f38b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:33.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.736+0000 7f38d0c00640 1 -- 172.21.15.87:0/862347094 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f38bc084360 con 0x7f38d40d0370 2024-04-25T10:24:33.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.736+0000 7f38d3e00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38b0066f60 0x7f38b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:33.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.736+0000 7f38d3e00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38b0066f60 0x7f38b0069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009ac0 tx=0x7f38c4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:33.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.740+0000 7f38d0c00640 1 -- 172.21.15.87:0/862347094 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38bc054cb0 con 0x7f38d40d0370 2024-04-25T10:24:33.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:33.896+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38d40b32a0 con 0x7f38d40d0370 2024-04-25T10:24:34.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.512+0000 7f38d0c00640 1 -- 172.21.15.87:0/862347094 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 181+0+0 (secure 0 0 0) 0x7f38bc058d50 con 0x7f38d40d0370 2024-04-25T10:24:34.511 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38b0066f60 msgr2=0x7f38b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38b0066f60 0x7f38b0069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009ac0 tx=0x7f38c4009190 comp rx=0 tx=0).stop 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d40d0370 msgr2=0x7f38d4148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d40d0370 0x7f38d4148c30 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00c520 tx=0x7f38bc00c9e0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 shutdown_connections 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f38b0066f60 0x7f38b0069420 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d415bcd0 0x7f38d40c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38d40d0370 0x7f38d4148c30 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 --2- 172.21.15.87:0/862347094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d40b0930 0x7f38d406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 >> 172.21.15.87:0/862347094 conn(0x7f38d40d19d0 msgr2=0x7f38d415af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:34.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 shutdown_connections 2024-04-25T10:24:34.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.516+0000 7f38daa00640 1 -- 172.21.15.87:0/862347094 wait complete. 2024-04-25T10:24:34.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:24:34.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.676+0000 7f0de6a00640 1 Processor -- start 2024-04-25T10:24:34.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.676+0000 7f0de6a00640 1 -- start start 2024-04-25T10:24:34.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0de6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0de0157cb0 0x7f0de00720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0de6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0de0070ba0 0x7f0de015b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0de6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de015b7a0 con 0x7f0de00b2af0 2024-04-25T10:24:34.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0de6a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de00af150 con 0x7f0de0070ba0 2024-04-25T10:24:34.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0de6a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de0071040 con 0x7f0de0157cb0 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0de0070ba0 0x7f0de015b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddf400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0de0070ba0 0x7f0de015b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41258/0 (socket says 172.21.15.87:41258) 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddf400640 1 -- 172.21.15.87:0/1339217567 learned_addr learned my addr 172.21.15.87:0/1339217567 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddfe00640 1 --2- 172.21.15.87:0/1339217567 >> v2:172.21.15.169:3300/0 conn(0x7f0de0157cb0 0x7f0de00720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddf400640 1 -- 172.21.15.87:0/1339217567 >> v2:172.21.15.169:3300/0 conn(0x7f0de0157cb0 msgr2=0x7f0de00720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddf400640 1 --2- 172.21.15.87:0/1339217567 >> v2:172.21.15.169:3300/0 conn(0x7f0de0157cb0 0x7f0de00720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddf400640 1 -- 172.21.15.87:0/1339217567 >> v1:172.21.15.169:6789/0 conn(0x7f0de00b2af0 legacy=0x7f0de006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddf400640 1 -- 172.21.15.87:0/1339217567 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de0071320 con 0x7f0de0070ba0 2024-04-25T10:24:34.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.680+0000 7f0ddfe00640 1 --2- 172.21.15.87:0/1339217567 >> v2:172.21.15.169:3300/0 conn(0x7f0de0157cb0 0x7f0de00720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:34.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0ddf400640 1 --2- 172.21.15.87:0/1339217567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0070ba0 0x7f0de015b260 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f0de006e830 tx=0x7f0dcc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2922f42cc746b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:34.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0ddea00640 1 -- 172.21.15.87:0/1339217567 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0b2070 con 0x7f0de0070ba0 2024-04-25T10:24:34.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0ddea00640 1 -- 172.21.15.87:0/1339217567 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0ad680 con 0x7f0de0070ba0 2024-04-25T10:24:34.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0ddea00640 1 -- 172.21.15.87:0/1339217567 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0ada50 con 0x7f0de0070ba0 2024-04-25T10:24:34.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 -- 172.21.15.87:0/1339217567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0070ba0 msgr2=0x7f0de015b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 --2- 172.21.15.87:0/1339217567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0070ba0 0x7f0de015b260 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f0de006e830 tx=0x7f0dcc01c490 comp rx=0 tx=0).stop 2024-04-25T10:24:34.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 -- 172.21.15.87:0/1339217567 shutdown_connections 2024-04-25T10:24:34.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 --2- 172.21.15.87:0/1339217567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0070ba0 0x7f0de015b260 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 --2- 172.21.15.87:0/1339217567 >> v2:172.21.15.169:3300/0 conn(0x7f0de0157cb0 0x7f0de00720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 -- 172.21.15.87:0/1339217567 >> 172.21.15.87:0/1339217567 conn(0x7f0de0066b70 msgr2=0x7f0de0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:34.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 -- 172.21.15.87:0/1339217567 shutdown_connections 2024-04-25T10:24:34.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 -- 172.21.15.87:0/1339217567 wait complete. 2024-04-25T10:24:34.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 Processor -- start 2024-04-25T10:24:34.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.684+0000 7f0de6a00640 1 -- start start 2024-04-25T10:24:34.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0de0070ba0 0x7f0de01504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0de00b2af0 0x7f0de0159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0157cb0 0x7f0de0144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de00b3310 con 0x7f0de0157cb0 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de006cc00 con 0x7f0de00b2af0 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de0148950 con 0x7f0de0070ba0 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0157cb0 0x7f0de0144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0157cb0 0x7f0de0144f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41264/0 (socket says 172.21.15.87:41264) 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 -- 172.21.15.87:0/2613607755 learned_addr learned my addr 172.21.15.87:0/2613607755 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0ddfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0de0070ba0 0x7f0de01504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0ddf400640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0de00b2af0 0x7f0de0159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 -- 172.21.15.87:0/2613607755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0de0070ba0 msgr2=0x7f0de01504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0de0070ba0 0x7f0de01504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 -- 172.21.15.87:0/2613607755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0de00b2af0 msgr2=0x7f0de0159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0de00b2af0 0x7f0de0159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 -- 172.21.15.87:0/2613607755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de0150a10 con 0x7f0de0157cb0 2024-04-25T10:24:34.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de4a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0157cb0 0x7f0de0144f30 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800c460 tx=0x7f0dd800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:34.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0dd7e00640 1 -- 172.21.15.87:0/2613607755 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8016020 con 0x7f0de0157cb0 2024-04-25T10:24:34.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0dd7e00640 1 -- 172.21.15.87:0/2613607755 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd8005220 con 0x7f0de0157cb0 2024-04-25T10:24:34.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0dd7e00640 1 -- 172.21.15.87:0/2613607755 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8007030 con 0x7f0de0157cb0 2024-04-25T10:24:34.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de0150cb0 con 0x7f0de0157cb0 2024-04-25T10:24:34.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de0151270 con 0x7f0de0157cb0 2024-04-25T10:24:34.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de004ef30 con 0x7f0de0157cb0 2024-04-25T10:24:34.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.688+0000 7f0dd7e00640 1 -- 172.21.15.87:0/2613607755 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0dd80071d0 con 0x7f0de0157cb0 2024-04-25T10:24:34.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.692+0000 7f0dd7e00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0dc0066f60 0x7f0dc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:34.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.692+0000 7f0dd7e00640 1 -- 172.21.15.87:0/2613607755 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f0dd8084690 con 0x7f0de0157cb0 2024-04-25T10:24:34.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.692+0000 7f0ddfe00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0dc0066f60 0x7f0dc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:34.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.692+0000 7f0ddfe00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0dc0066f60 0x7f0dc0069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009fd0 tx=0x7f0dd00097f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:34.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.696+0000 7f0dd7e00640 1 -- 172.21.15.87:0/2613607755 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0dd8020020 con 0x7f0de0157cb0 2024-04-25T10:24:34.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:34.848+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0de0153d60 con 0x7f0de0157cb0 2024-04-25T10:24:35.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.512+0000 7f0dd7e00640 1 -- 172.21.15.87:0/2613607755 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0de015b7a0 con 0x7f0de0157cb0 2024-04-25T10:24:35.513 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0dc0066f60 msgr2=0x7f0dc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0dc0066f60 0x7f0dc0069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009fd0 tx=0x7f0dd00097f0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0157cb0 msgr2=0x7f0de0144f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0157cb0 0x7f0de0144f30 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800c460 tx=0x7f0dd800c920 comp rx=0 tx=0).stop 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 shutdown_connections 2024-04-25T10:24:35.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0dc0066f60 0x7f0dc0069420 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de0157cb0 0x7f0de0144f30 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0de00b2af0 0x7f0de0159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 --2- 172.21.15.87:0/2613607755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0de0070ba0 0x7f0de01504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 >> 172.21.15.87:0/2613607755 conn(0x7f0de0066b70 msgr2=0x7f0de006c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:35.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 shutdown_connections 2024-04-25T10:24:35.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.516+0000 7f0de6a00640 1 -- 172.21.15.87:0/2613607755 wait complete. 2024-04-25T10:24:35.538 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:24:35.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f034b200640 1 Processor -- start 2024-04-25T10:24:35.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f034b200640 1 -- start start 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f034b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f034c06dea0 0x7f034c161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f034b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c0711d0 con 0x7f034c06da30 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f034b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c0b2140 con 0x7f034c15da50 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f034b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c071490 con 0x7f034c06dea0 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f0341400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f034c06da30 0x7f034c1616f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41352/0 (socket says 172.21.15.87:41352) 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f0341400640 1 -- 172.21.15.87:0/876013810 learned_addr learned my addr 172.21.15.87:0/876013810 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.676+0000 7f034a800640 1 --2- 172.21.15.87:0/876013810 >> v2:172.21.15.87:3300/0 conn(0x7f034c06dea0 0x7f034c161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f0349400640 1 -- 172.21.15.87:0/876013810 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173556226 0 0) 0x7f034c0711d0 con 0x7f034c06da30 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f0349400640 1 -- 172.21.15.87:0/876013810 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0330003530 con 0x7f034c06da30 2024-04-25T10:24:35.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f034a800640 1 -- 172.21.15.87:0/876013810 >> v1:172.21.15.169:6789/0 conn(0x7f034c15da50 legacy=0x7f034c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:35.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f034a800640 1 -- 172.21.15.87:0/876013810 >> v1:172.21.15.87:6789/0 conn(0x7f034c06da30 legacy=0x7f034c1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f034a800640 1 -- 172.21.15.87:0/876013810 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c071770 con 0x7f034c06dea0 2024-04-25T10:24:35.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f034a800640 1 --2- 172.21.15.87:0/876013810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c06dea0 0x7f034c161e00 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f0344009d90 tx=0x7f034401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb90af1cc17142f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:35.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f0349400640 1 -- 172.21.15.87:0/876013810 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440b2070 con 0x7f034c06dea0 2024-04-25T10:24:35.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f0349400640 1 -- 172.21.15.87:0/876013810 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03440a5900 con 0x7f034c06dea0 2024-04-25T10:24:35.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f0349400640 1 -- 172.21.15.87:0/876013810 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440a5cd0 con 0x7f034c06dea0 2024-04-25T10:24:35.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f034b200640 1 -- 172.21.15.87:0/876013810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c06dea0 msgr2=0x7f034c161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.680+0000 7f034b200640 1 --2- 172.21.15.87:0/876013810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c06dea0 0x7f034c161e00 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f0344009d90 tx=0x7f034401c970 comp rx=0 tx=0).stop 2024-04-25T10:24:35.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- 172.21.15.87:0/876013810 shutdown_connections 2024-04-25T10:24:35.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 --2- 172.21.15.87:0/876013810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c06dea0 0x7f034c161e00 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- 172.21.15.87:0/876013810 >> 172.21.15.87:0/876013810 conn(0x7f034c074110 msgr2=0x7f034c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:35.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- 172.21.15.87:0/876013810 shutdown_connections 2024-04-25T10:24:35.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- 172.21.15.87:0/876013810 wait complete. 2024-04-25T10:24:35.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 Processor -- start 2024-04-25T10:24:35.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- start start 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f034c06da30 0x7f034c14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f034c06dea0 0x7f034c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c15da50 0x7f034c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c15de30 con 0x7f034c15da50 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c072380 con 0x7f034c06dea0 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c15dfb0 con 0x7f034c06da30 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c15da50 0x7f034c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c15da50 0x7f034c15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41282/0 (socket says 172.21.15.87:41282) 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 -- 172.21.15.87:0/2951334044 learned_addr learned my addr 172.21.15.87:0/2951334044 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:35.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f0341400640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f034c06dea0 0x7f034c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:35.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f0349e00640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f034c06da30 0x7f034c14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:35.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 -- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f034c06da30 msgr2=0x7f034c14b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f034c06da30 0x7f034c14b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 -- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f034c06dea0 msgr2=0x7f034c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f034c06dea0 0x7f034c159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 -- 172.21.15.87:0/2951334044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c15aaf0 con 0x7f034c15da50 2024-04-25T10:24:35.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034a800640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c15da50 0x7f034c15ba00 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f034401a5a0 tx=0x7f0344019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:35.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f0342a00640 1 -- 172.21.15.87:0/2951334044 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440b2050 con 0x7f034c15da50 2024-04-25T10:24:35.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f0342a00640 1 -- 172.21.15.87:0/2951334044 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03440be070 con 0x7f034c15da50 2024-04-25T10:24:35.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c15a810 con 0x7f034c15da50 2024-04-25T10:24:35.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.684+0000 7f0342a00640 1 -- 172.21.15.87:0/2951334044 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440ada50 con 0x7f034c15da50 2024-04-25T10:24:35.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.688+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c0ce9a0 con 0x7f034c15da50 2024-04-25T10:24:35.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.688+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c075120 con 0x7f034c15da50 2024-04-25T10:24:35.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.688+0000 7f0342a00640 1 -- 172.21.15.87:0/2951334044 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03440adbf0 con 0x7f034c15da50 2024-04-25T10:24:35.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.688+0000 7f0342a00640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0328066f60 0x7f0328069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:35.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.688+0000 7f0342a00640 1 -- 172.21.15.87:0/2951334044 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f034411ff50 con 0x7f034c15da50 2024-04-25T10:24:35.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.688+0000 7f0349e00640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0328066f60 0x7f0328069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:35.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.688+0000 7f0349e00640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0328066f60 0x7f0328069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f033c0015b0 tx=0x7f033c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:35.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.692+0000 7f0342a00640 1 -- 172.21.15.87:0/2951334044 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0344016560 con 0x7f034c15da50 2024-04-25T10:24:35.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.848+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f034c0cd710 con 0x7f034c15da50 2024-04-25T10:24:35.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f0342a00640 1 -- 172.21.15.87:0/2951334044 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f03440ab340 con 0x7f034c15da50 2024-04-25T10:24:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0328066f60 msgr2=0x7f0328069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0328066f60 0x7f0328069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f033c0015b0 tx=0x7f033c009040 comp rx=0 tx=0).stop 2024-04-25T10:24:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c15da50 msgr2=0x7f034c15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c15da50 0x7f034c15ba00 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f034401a5a0 tx=0x7f0344019710 comp rx=0 tx=0).stop 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 shutdown_connections 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0328066f60 0x7f0328069420 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c15da50 0x7f034c15ba00 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f034c06dea0 0x7f034c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 --2- 172.21.15.87:0/2951334044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f034c06da30 0x7f034c14b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 >> 172.21.15.87:0/2951334044 conn(0x7f034c074110 msgr2=0x7f034c0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 shutdown_connections 2024-04-25T10:24:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:35.852+0000 7f034b200640 1 -- 172.21.15.87:0/2951334044 wait complete. 2024-04-25T10:24:35.873 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:24:36.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9279400640 1 Processor -- start 2024-04-25T10:24:36.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9279400640 1 -- start start 2024-04-25T10:24:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9279400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f927415bc40 0x7f927406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9279400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92740cf930 0x7f92740cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9279400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92740b2770 con 0x7f92740cda10 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9279400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927406c670 con 0x7f927415bc40 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9279400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927406c950 con 0x7f92740cf930 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9272800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f927415bc40 0x7f927406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9272800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f927415bc40 0x7f927406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41290/0 (socket says 172.21.15.87:41290) 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9272800640 1 -- 172.21.15.87:0/856541727 learned_addr learned my addr 172.21.15.87:0/856541727 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9273200640 1 --2- 172.21.15.87:0/856541727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92740cf930 0x7f92740cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9272800640 1 -- 172.21.15.87:0/856541727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92740cf930 msgr2=0x7f92740cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9272800640 1 --2- 172.21.15.87:0/856541727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92740cf930 0x7f92740cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9272800640 1 -- 172.21.15.87:0/856541727 >> v1:172.21.15.169:6789/0 conn(0x7f92740cda10 legacy=0x7f92740b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.012+0000 7f9272800640 1 -- 172.21.15.87:0/856541727 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927406cc30 con 0x7f927415bc40 2024-04-25T10:24:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9272800640 1 --2- 172.21.15.87:0/856541727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927415bc40 0x7f927406c090 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f9264009870 tx=0x7f926401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0a89b71d86801a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9271400640 1 -- 172.21.15.87:0/856541727 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640b0070 con 0x7f927415bc40 2024-04-25T10:24:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9271400640 1 -- 172.21.15.87:0/856541727 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640ab600 con 0x7f927415bc40 2024-04-25T10:24:36.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9271400640 1 -- 172.21.15.87:0/856541727 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640ab9d0 con 0x7f927415bc40 2024-04-25T10:24:36.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 -- 172.21.15.87:0/856541727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927415bc40 msgr2=0x7f927406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 --2- 172.21.15.87:0/856541727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927415bc40 0x7f927406c090 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f9264009870 tx=0x7f926401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 -- 172.21.15.87:0/856541727 shutdown_connections 2024-04-25T10:24:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 --2- 172.21.15.87:0/856541727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92740cf930 0x7f92740cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 --2- 172.21.15.87:0/856541727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927415bc40 0x7f927406c090 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 -- 172.21.15.87:0/856541727 >> 172.21.15.87:0/856541727 conn(0x7f9274066b70 msgr2=0x7f9274066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 -- 172.21.15.87:0/856541727 shutdown_connections 2024-04-25T10:24:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.016+0000 7f9279400640 1 -- 172.21.15.87:0/856541727 wait complete. 2024-04-25T10:24:36.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 Processor -- start 2024-04-25T10:24:36.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 -- start start 2024-04-25T10:24:36.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92740cda10 0x7f9274151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92740cf930 0x7f92740ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f927415bc40 0x7f9274150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740b3890 con 0x7f92740cf930 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740b3710 con 0x7f92740cda10 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740b3a10 con 0x7f927415bc40 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92740cf930 0x7f92740ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92740cf930 0x7f92740ac620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41306/0 (socket says 172.21.15.87:41306) 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 -- 172.21.15.87:0/3823500161 learned_addr learned my addr 172.21.15.87:0/3823500161 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9272800640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92740cda10 0x7f9274151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9273200640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f927415bc40 0x7f9274150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 -- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f927415bc40 msgr2=0x7f9274150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f927415bc40 0x7f9274150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 -- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92740cda10 msgr2=0x7f9274151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92740cda10 0x7f9274151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 -- 172.21.15.87:0/3823500161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740ace70 con 0x7f92740cf930 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9273200640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f927415bc40 0x7f9274150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9271e00640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92740cf930 0x7f92740ac620 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f926000c370 tx=0x7f926000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f926aa00640 1 -- 172.21.15.87:0/3823500161 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600052d0 con 0x7f92740cf930 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f926aa00640 1 -- 172.21.15.87:0/3823500161 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260005470 con 0x7f92740cf930 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f926aa00640 1 -- 172.21.15.87:0/3823500161 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260007260 con 0x7f92740cf930 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92740ad150 con 0x7f92740cf930 2024-04-25T10:24:36.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.020+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92740ad710 con 0x7f92740cf930 2024-04-25T10:24:36.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.024+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927404ef30 con 0x7f92740cf930 2024-04-25T10:24:36.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.024+0000 7f926aa00640 1 -- 172.21.15.87:0/3823500161 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9260007400 con 0x7f92740cf930 2024-04-25T10:24:36.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.024+0000 7f926aa00640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9254066f90 0x7f9254069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.024+0000 7f926aa00640 1 -- 172.21.15.87:0/3823500161 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f92600840d0 con 0x7f92740cf930 2024-04-25T10:24:36.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.024+0000 7f9272800640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9254066f90 0x7f9254069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.024+0000 7f9272800640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9254066f90 0x7f9254069450 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f92640ad000 tx=0x7f92640a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:36.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.028+0000 7f926aa00640 1 -- 172.21.15.87:0/3823500161 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9260054d80 con 0x7f92740cf930 2024-04-25T10:24:36.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.248+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f927415f400 con 0x7f92740cf930 2024-04-25T10:24:36.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f926aa00640 1 -- 172.21.15.87:0/3823500161 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v589) v1 ==== 140+0+0 (secure 0 0 0) 0x7f9260058e20 con 0x7f92740cf930 2024-04-25T10:24:36.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9254066f90 msgr2=0x7f9254069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9254066f90 0x7f9254069450 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f92640ad000 tx=0x7f92640a6000 comp rx=0 tx=0).stop 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92740cf930 msgr2=0x7f92740ac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92740cf930 0x7f92740ac620 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f926000c370 tx=0x7f926000c830 comp rx=0 tx=0).stop 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 shutdown_connections 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9254066f90 0x7f9254069450 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f927415bc40 0x7f9274150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92740cf930 0x7f92740ac620 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 --2- 172.21.15.87:0/3823500161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92740cda10 0x7f9274151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 >> 172.21.15.87:0/3823500161 conn(0x7f9274066b70 msgr2=0x7f9274155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 shutdown_connections 2024-04-25T10:24:36.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.516+0000 7f9279400640 1 -- 172.21.15.87:0/3823500161 wait complete. 2024-04-25T10:24:36.536 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:24:36.537 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:24:36.537 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:36.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.672+0000 7f812e200640 1 Processor -- start 2024-04-25T10:24:36.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.672+0000 7f812e200640 1 -- start start 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.672+0000 7f812e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81281550b0 0x7f81280cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.672+0000 7f812e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81280cfe70 0x7f812815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.672+0000 7f812e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8128071050 con 0x7f812815b740 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.672+0000 7f812e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812815d040 con 0x7f81280cfe70 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.672+0000 7f812e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81280712f0 con 0x7f81281550b0 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81280cfe70 0x7f812815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81280cfe70 0x7f812815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41316/0 (socket says 172.21.15.87:41316) 2024-04-25T10:24:36.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 -- 172.21.15.87:0/1242950772 learned_addr learned my addr 172.21.15.87:0/1242950772 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:36.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8127400640 1 --2- 172.21.15.87:0/1242950772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81281550b0 0x7f81280cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 -- 172.21.15.87:0/1242950772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81281550b0 msgr2=0x7f81280cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:36.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 --2- 172.21.15.87:0/1242950772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81281550b0 0x7f81280cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 -- 172.21.15.87:0/1242950772 >> v1:172.21.15.169:6789/0 conn(0x7f812815b740 legacy=0x7f812806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 -- 172.21.15.87:0/1242950772 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280715d0 con 0x7f81280cfe70 2024-04-25T10:24:36.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126a00640 1 --2- 172.21.15.87:0/1242950772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f812815b200 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f811400bb10 tx=0x7f811401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b93113f3a93e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:36.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126000640 1 -- 172.21.15.87:0/1242950772 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81140b4070 con 0x7f81280cfe70 2024-04-25T10:24:36.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126000640 1 -- 172.21.15.87:0/1242950772 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81140a7b10 con 0x7f81280cfe70 2024-04-25T10:24:36.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.676+0000 7f8126000640 1 -- 172.21.15.87:0/1242950772 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81140af440 con 0x7f81280cfe70 2024-04-25T10:24:36.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- 172.21.15.87:0/1242950772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 msgr2=0x7f812815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 --2- 172.21.15.87:0/1242950772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f812815b200 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f811400bb10 tx=0x7f811401ec30 comp rx=0 tx=0).stop 2024-04-25T10:24:36.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- 172.21.15.87:0/1242950772 shutdown_connections 2024-04-25T10:24:36.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 --2- 172.21.15.87:0/1242950772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f812815b200 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 --2- 172.21.15.87:0/1242950772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81281550b0 0x7f81280cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- 172.21.15.87:0/1242950772 >> 172.21.15.87:0/1242950772 conn(0x7f8128066b70 msgr2=0x7f8128066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:36.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- 172.21.15.87:0/1242950772 shutdown_connections 2024-04-25T10:24:36.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- 172.21.15.87:0/1242950772 wait complete. 2024-04-25T10:24:36.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 Processor -- start 2024-04-25T10:24:36.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- start start 2024-04-25T10:24:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f8128136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81281550b0 0x7f8128137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f812815b740 0x7f81280ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812815bcd0 con 0x7f81280cfe70 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812815bb50 con 0x7f81281550b0 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f812e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812815c030 con 0x7f812815b740 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f8128136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f8128136c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41324/0 (socket says 172.21.15.87:41324) 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 -- 172.21.15.87:0/82494812 learned_addr learned my addr 172.21.15.87:0/82494812 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8126a00640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81281550b0 0x7f8128137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127e00640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f812815b740 0x7f81280ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 -- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f812815b740 msgr2=0x7f81280ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f812815b740 0x7f81280ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 -- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81281550b0 msgr2=0x7f8128137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81281550b0 0x7f8128137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8127400640 1 -- 172.21.15.87:0/82494812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812815ce70 con 0x7f81280cfe70 2024-04-25T10:24:36.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.680+0000 7f8126a00640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81281550b0 0x7f8128137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:36.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f8127400640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f8128136c40 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f8118009db0 tx=0x7f811800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:36.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f811f400640 1 -- 172.21.15.87:0/82494812 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118018020 con 0x7f81280cfe70 2024-04-25T10:24:36.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f811f400640 1 -- 172.21.15.87:0/82494812 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118014480 con 0x7f81280cfe70 2024-04-25T10:24:36.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81280ce3e0 con 0x7f81280cfe70 2024-04-25T10:24:36.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f811f400640 1 -- 172.21.15.87:0/82494812 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118014850 con 0x7f81280cfe70 2024-04-25T10:24:36.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812815a100 con 0x7f81280cfe70 2024-04-25T10:24:36.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812804ef30 con 0x7f81280cfe70 2024-04-25T10:24:36.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f811f400640 1 -- 172.21.15.87:0/82494812 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f811801b020 con 0x7f81280cfe70 2024-04-25T10:24:36.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f811f400640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:36.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f811f400640 1 -- 172.21.15.87:0/82494812 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8118085820 con 0x7f81280cfe70 2024-04-25T10:24:36.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.684+0000 7f8126a00640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:36.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.688+0000 7f8126a00640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f81140b1000 tx=0x7f8114008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:36.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.688+0000 7f811f400640 1 -- 172.21.15.87:0/82494812 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8118022070 con 0x7f81280cfe70 2024-04-25T10:24:36.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.844+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81280ce5c0 con 0x7f81280cfe70 2024-04-25T10:24:36.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.844+0000 7f811f400640 1 -- 172.21.15.87:0/82494812 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13016 (secure 0 0 0) 0x7f81280ce3e0 con 0x7f81280cfe70 2024-04-25T10:24:36.845 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:36.845 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":154,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:24:35.513947+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/564498676":"2024-04-26T10:24:32.508095+0000","172.21.15.87:6835/564498676":"2024-04-26T10:24:32.508095+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 msgr2=0x7f8108069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f81140b1000 tx=0x7f8114008040 comp rx=0 tx=0).stop 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 msgr2=0x7f8128136c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f8128136c40 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f8118009db0 tx=0x7f811800c490 comp rx=0 tx=0).stop 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 shutdown_connections 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8108066f60 0x7f8108069420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f812815b740 0x7f81280ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81281550b0 0x7f8128137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 --2- 172.21.15.87:0/82494812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81280cfe70 0x7f8128136c40 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:36.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 >> 172.21.15.87:0/82494812 conn(0x7f8128066b70 msgr2=0x7f81280af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:36.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 shutdown_connections 2024-04-25T10:24:36.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:36.848+0000 7f812e200640 1 -- 172.21.15.87:0/82494812 wait complete. 2024-04-25T10:24:36.866 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:36.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 Processor -- start 2024-04-25T10:24:37.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 -- start start 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffb800cc200 0x7ffb80162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffb801626c0 0x7ffb8006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb800cdab0 con 0x7ffb800cc200 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb800cd210 con 0x7ffb801626c0 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb88000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800cd4f0 con 0x7ffb800d0d00 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.004+0000 7ffb84c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffb800cc200 0x7ffb80162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffb800cc200 0x7ffb80162180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41338/0 (socket says 172.21.15.87:41338) 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 -- 172.21.15.87:0/1748850992 learned_addr learned my addr 172.21.15.87:0/1748850992 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:37.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb86000640 1 --2- 172.21.15.87:0/1748850992 >> v2:172.21.15.169:3300/0 conn(0x7ffb801626c0 0x7ffb8006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb85600640 1 --2- 172.21.15.87:0/1748850992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 -- 172.21.15.87:0/1748850992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 msgr2=0x7ffb800ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 --2- 172.21.15.87:0/1748850992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 -- 172.21.15.87:0/1748850992 >> v2:172.21.15.169:3300/0 conn(0x7ffb801626c0 msgr2=0x7ffb8006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 --2- 172.21.15.87:0/1748850992 >> v2:172.21.15.169:3300/0 conn(0x7ffb801626c0 0x7ffb8006cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 -- 172.21.15.87:0/1748850992 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb800cd7d0 con 0x7ffb800cc200 2024-04-25T10:24:37.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb86000640 1 --2- 172.21.15.87:0/1748850992 >> v2:172.21.15.169:3300/0 conn(0x7ffb801626c0 0x7ffb8006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:37.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb84c00640 1 --2- 172.21.15.87:0/1748850992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb80162180 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7ffb80070510 tx=0x7ffb6c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faef786893da1b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb7fe00640 1 -- 172.21.15.87:0/1748850992 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0b0070 con 0x7ffb800cc200 2024-04-25T10:24:37.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb7fe00640 1 -- 172.21.15.87:0/1748850992 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c0ab600 con 0x7ffb800cc200 2024-04-25T10:24:37.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.008+0000 7ffb7fe00640 1 -- 172.21.15.87:0/1748850992 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0ab9d0 con 0x7ffb800cc200 2024-04-25T10:24:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- 172.21.15.87:0/1748850992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 msgr2=0x7ffb80162180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 --2- 172.21.15.87:0/1748850992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb80162180 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7ffb80070510 tx=0x7ffb6c01c490 comp rx=0 tx=0).stop 2024-04-25T10:24:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- 172.21.15.87:0/1748850992 shutdown_connections 2024-04-25T10:24:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 --2- 172.21.15.87:0/1748850992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb80162180 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 --2- 172.21.15.87:0/1748850992 >> v2:172.21.15.169:3300/0 conn(0x7ffb801626c0 0x7ffb8006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 --2- 172.21.15.87:0/1748850992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- 172.21.15.87:0/1748850992 >> 172.21.15.87:0/1748850992 conn(0x7ffb80072290 msgr2=0x7ffb800726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- 172.21.15.87:0/1748850992 shutdown_connections 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- 172.21.15.87:0/1748850992 wait complete. 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 Processor -- start 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- start start 2024-04-25T10:24:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb800b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb801626c0 0x7ffb800afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8006ed00 con 0x7ffb800cc200 2024-04-25T10:24:37.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8006eb80 con 0x7ffb801626c0 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb88000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8006ee80 con 0x7ffb800d0d00 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb800b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb800b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41352/0 (socket says 172.21.15.87:41352) 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 -- 172.21.15.87:0/3753010172 learned_addr learned my addr 172.21.15.87:0/3753010172 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb86000640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb801626c0 0x7ffb800afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb84c00640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 -- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 msgr2=0x7ffb800993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 -- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb801626c0 msgr2=0x7ffb800afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb801626c0 0x7ffb800afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.012+0000 7ffb85600640 1 -- 172.21.15.87:0/3753010172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80157290 con 0x7ffb800cc200 2024-04-25T10:24:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb85600640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb800b0de0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400d800 tx=0x7ffb7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb7d600640 1 -- 172.21.15.87:0/3753010172 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74018030 con 0x7ffb800cc200 2024-04-25T10:24:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb7d600640 1 -- 172.21.15.87:0/3753010172 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb740096a0 con 0x7ffb800cc200 2024-04-25T10:24:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb800b0300 con 0x7ffb800cc200 2024-04-25T10:24:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb7d600640 1 -- 172.21.15.87:0/3753010172 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74009870 con 0x7ffb800cc200 2024-04-25T10:24:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb800976c0 con 0x7ffb800cc200 2024-04-25T10:24:37.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8004ef30 con 0x7ffb800cc200 2024-04-25T10:24:37.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb7d600640 1 -- 172.21.15.87:0/3753010172 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb7401b070 con 0x7ffb800cc200 2024-04-25T10:24:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb7d600640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb60066f60 0x7ffb60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb7d600640 1 -- 172.21.15.87:0/3753010172 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffb74084ef0 con 0x7ffb800cc200 2024-04-25T10:24:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.016+0000 7ffb84c00640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb60066f60 0x7ffb60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.020+0000 7ffb84c00640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb60066f60 0x7ffb60069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ad000 tx=0x7ffb6c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.020+0000 7ffb7d600640 1 -- 172.21.15.87:0/3753010172 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb74055ba0 con 0x7ffb800cc200 2024-04-25T10:24:37.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.176+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb80072ff0 con 0x7ffb800cc200 2024-04-25T10:24:37.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.176+0000 7ffb7d600640 1 -- 172.21.15.87:0/3753010172 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb74059c40 con 0x7ffb800cc200 2024-04-25T10:24:37.177 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:24:37.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb60066f60 msgr2=0x7ffb60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb60066f60 0x7ffb60069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ad000 tx=0x7ffb6c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 msgr2=0x7ffb800b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb800b0de0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400d800 tx=0x7ffb7400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 shutdown_connections 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb60066f60 0x7ffb60069420 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb801626c0 0x7ffb800afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb800d0d00 0x7ffb800993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 --2- 172.21.15.87:0/3753010172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb800cc200 0x7ffb800b0de0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 >> 172.21.15.87:0/3753010172 conn(0x7ffb80072290 msgr2=0x7ffb800ad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 shutdown_connections 2024-04-25T10:24:37.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.180+0000 7ffb88000640 1 -- 172.21.15.87:0/3753010172 wait complete. 2024-04-25T10:24:37.199 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:24:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.332+0000 7f9862600640 1 Processor -- start 2024-04-25T10:24:37.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f9862600640 1 -- start start 2024-04-25T10:24:37.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f9862600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f985c0d0040 0x7f985c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f9862600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c0b2770 con 0x7f985c0cda10 2024-04-25T10:24:37.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f9862600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c06c670 con 0x7f985c15bc40 2024-04-25T10:24:37.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f9862600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c06c950 con 0x7f985c0d0040 2024-04-25T10:24:37.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f985c0cda10 0x7f985c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41368/0 (socket says 172.21.15.87:41368) 2024-04-25T10:24:37.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985aa00640 1 -- 172.21.15.87:0/617373768 learned_addr learned my addr 172.21.15.87:0/617373768 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:37.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985be00640 1 --2- 172.21.15.87:0/617373768 >> v2:172.21.15.87:3300/0 conn(0x7f985c0d0040 0x7f985c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985a000640 1 -- 172.21.15.87:0/617373768 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055911125 0 0) 0x7f985c0b2770 con 0x7f985c0cda10 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985a000640 1 -- 172.21.15.87:0/617373768 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9840003530 con 0x7f985c0cda10 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985be00640 1 -- 172.21.15.87:0/617373768 >> v1:172.21.15.169:6789/0 conn(0x7f985c15bc40 legacy=0x7f985c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985be00640 1 -- 172.21.15.87:0/617373768 >> v1:172.21.15.87:6789/0 conn(0x7f985c0cda10 legacy=0x7f985c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.336+0000 7f985be00640 1 -- 172.21.15.87:0/617373768 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c06cc30 con 0x7f985c0d0040 2024-04-25T10:24:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f985be00640 1 --2- 172.21.15.87:0/617373768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0d0040 0x7f985c06c090 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f98540097c0 tx=0x7f985401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73bf9c7e82af914 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f985a000640 1 -- 172.21.15.87:0/617373768 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540b0070 con 0x7f985c0d0040 2024-04-25T10:24:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f985a000640 1 -- 172.21.15.87:0/617373768 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540ab600 con 0x7f985c0d0040 2024-04-25T10:24:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f985a000640 1 -- 172.21.15.87:0/617373768 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540ab9d0 con 0x7f985c0d0040 2024-04-25T10:24:37.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 -- 172.21.15.87:0/617373768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0d0040 msgr2=0x7f985c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 --2- 172.21.15.87:0/617373768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0d0040 0x7f985c06c090 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f98540097c0 tx=0x7f985401c410 comp rx=0 tx=0).stop 2024-04-25T10:24:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 -- 172.21.15.87:0/617373768 shutdown_connections 2024-04-25T10:24:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 --2- 172.21.15.87:0/617373768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0d0040 0x7f985c06c090 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 -- 172.21.15.87:0/617373768 >> 172.21.15.87:0/617373768 conn(0x7f985c066b70 msgr2=0x7f985c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 -- 172.21.15.87:0/617373768 shutdown_connections 2024-04-25T10:24:37.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 -- 172.21.15.87:0/617373768 wait complete. 2024-04-25T10:24:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 Processor -- start 2024-04-25T10:24:37.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.340+0000 7f9862600640 1 -- start start 2024-04-25T10:24:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0cda10 0x7f985c14dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f985c0d0040 0x7f985c0715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f985c15bc40 0x7f985c13bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c169980 con 0x7f985c0cda10 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c169800 con 0x7f985c0d0040 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c169b00 con 0x7f985c15bc40 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0cda10 0x7f985c14dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0cda10 0x7f985c14dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49380/0 (socket says 172.21.15.87:49380) 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 -- 172.21.15.87:0/2424071764 learned_addr learned my addr 172.21.15.87:0/2424071764 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985aa00640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f985c0d0040 0x7f985c0715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985be00640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f985c15bc40 0x7f985c13bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 -- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f985c15bc40 msgr2=0x7f985c13bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f985c15bc40 0x7f985c13bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 -- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f985c0d0040 msgr2=0x7f985c0715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f985c0d0040 0x7f985c0715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 -- 172.21.15.87:0/2424071764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c15c3e0 con 0x7f985c0cda10 2024-04-25T10:24:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f985b400640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0cda10 0x7f985c14dd00 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c7c0 tx=0x7f984c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9853400640 1 -- 172.21.15.87:0/2424071764 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c016020 con 0x7f985c0cda10 2024-04-25T10:24:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9853400640 1 -- 172.21.15.87:0/2424071764 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c006d60 con 0x7f985c0cda10 2024-04-25T10:24:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c15c6c0 con 0x7f985c0cda10 2024-04-25T10:24:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9853400640 1 -- 172.21.15.87:0/2424071764 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c003bc0 con 0x7f985c0cda10 2024-04-25T10:24:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c15cc80 con 0x7f985c0cda10 2024-04-25T10:24:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c04ef30 con 0x7f985c0cda10 2024-04-25T10:24:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.344+0000 7f9853400640 1 -- 172.21.15.87:0/2424071764 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f984c00a7f0 con 0x7f985c0cda10 2024-04-25T10:24:37.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.348+0000 7f9853400640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:37.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.348+0000 7f9853400640 1 -- 172.21.15.87:0/2424071764 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f984c0848c0 con 0x7f985c0cda10 2024-04-25T10:24:37.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.348+0000 7f985aa00640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.348+0000 7f985aa00640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f985c1fbd70 tx=0x7f9848007d40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.352+0000 7f9853400640 1 -- 172.21.15.87:0/2424071764 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f984c020020 con 0x7f985c0cda10 2024-04-25T10:24:37.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:37.560+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f985c0af1c0 con 0x7f985c0cda10 2024-04-25T10:24:38.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9853400640 1 -- 172.21.15.87:0/2424071764 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v591) v1 ==== 136+0+0 (secure 0 0 0) 0x7f984c055570 con 0x7f985c0cda10 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f983c066f60 msgr2=0x7f983c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f985c1fbd70 tx=0x7f9848007d40 comp rx=0 tx=0).stop 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0cda10 msgr2=0x7f985c14dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0cda10 0x7f985c14dd00 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c7c0 tx=0x7f984c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 shutdown_connections 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f985c15bc40 0x7f985c13bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f985c0d0040 0x7f985c0715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 --2- 172.21.15.87:0/2424071764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f985c0cda10 0x7f985c14dd00 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 >> 172.21.15.87:0/2424071764 conn(0x7f985c066b70 msgr2=0x7f985c0ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 shutdown_connections 2024-04-25T10:24:38.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.524+0000 7f9862600640 1 -- 172.21.15.87:0/2424071764 wait complete. 2024-04-25T10:24:38.548 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:24:38.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 Processor -- start 2024-04-25T10:24:38.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 -- start start 2024-04-25T10:24:38.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:38.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb240aebd0 0x7fbb24071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:38.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb24072020 0x7fbb2415f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:38.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240af030 con 0x7fbb24072020 2024-04-25T10:24:38.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb24159920 con 0x7fbb240aebd0 2024-04-25T10:24:38.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb2aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2415c2f0 con 0x7fbb24159540 2024-04-25T10:24:38.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb28a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb24072020 0x7fbb2415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:38.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb28a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb24072020 0x7fbb2415f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49392/0 (socket says 172.21.15.87:49392) 2024-04-25T10:24:38.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.688+0000 7fbb28a00640 1 -- 172.21.15.87:0/3601545881 learned_addr learned my addr 172.21.15.87:0/3601545881 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:38.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb23e00640 1 --2- 172.21.15.87:0/3601545881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:38.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb1b400640 1 --2- 172.21.15.87:0/3601545881 >> v2:172.21.15.169:3300/0 conn(0x7fbb240aebd0 0x7fbb24071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:38.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb28a00640 1 -- 172.21.15.87:0/3601545881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 msgr2=0x7fbb240ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:38.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb28a00640 1 --2- 172.21.15.87:0/3601545881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb28a00640 1 -- 172.21.15.87:0/3601545881 >> v2:172.21.15.169:3300/0 conn(0x7fbb240aebd0 msgr2=0x7fbb24071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:38.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb28a00640 1 --2- 172.21.15.87:0/3601545881 >> v2:172.21.15.169:3300/0 conn(0x7fbb240aebd0 0x7fbb24071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb28a00640 1 -- 172.21.15.87:0/3601545881 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2415c5d0 con 0x7fbb24072020 2024-04-25T10:24:38.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb28a00640 1 --2- 172.21.15.87:0/3601545881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb2415f590 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b4b30 tx=0x7fbb1c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ab0a2a13a6d43b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:38.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb23400640 1 -- 172.21.15.87:0/3601545881 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0b0070 con 0x7fbb24072020 2024-04-25T10:24:38.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb23400640 1 -- 172.21.15.87:0/3601545881 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0ab600 con 0x7fbb24072020 2024-04-25T10:24:38.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb23400640 1 -- 172.21.15.87:0/3601545881 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0ab9d0 con 0x7fbb24072020 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb2aa00640 1 -- 172.21.15.87:0/3601545881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 msgr2=0x7fbb2415f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/3601545881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb2415f590 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b4b30 tx=0x7fbb1c01c230 comp rx=0 tx=0).stop 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb2aa00640 1 -- 172.21.15.87:0/3601545881 shutdown_connections 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/3601545881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb2415f590 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/3601545881 >> v2:172.21.15.169:3300/0 conn(0x7fbb240aebd0 0x7fbb24071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.692+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/3601545881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- 172.21.15.87:0/3601545881 >> 172.21.15.87:0/3601545881 conn(0x7fbb240669a0 msgr2=0x7fbb24066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- 172.21.15.87:0/3601545881 shutdown_connections 2024-04-25T10:24:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- 172.21.15.87:0/3601545881 wait complete. 2024-04-25T10:24:38.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 Processor -- start 2024-04-25T10:24:38.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- start start 2024-04-25T10:24:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb24157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb240aebd0 0x7fbb240af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2406c0d0 con 0x7fbb24072020 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24072450 con 0x7fbb240aebd0 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24070350 con 0x7fbb24159540 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb24157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb24157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49396/0 (socket says 172.21.15.87:49396) 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 -- 172.21.15.87:0/1512207510 learned_addr learned my addr 172.21.15.87:0/1512207510 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb1b400640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb240aebd0 0x7fbb240af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb28a00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 -- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 msgr2=0x7fbb240afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 -- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb240aebd0 msgr2=0x7fbb240af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb240aebd0 0x7fbb240af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 -- 172.21.15.87:0/1512207510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240b0490 con 0x7fbb24072020 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb28a00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb23e00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb24157790 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007e40 tx=0x7fbb1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb20c00640 1 -- 172.21.15.87:0/1512207510 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14014620 con 0x7fbb24072020 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb20c00640 1 -- 172.21.15.87:0/1512207510 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb14019070 con 0x7fbb24072020 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb20c00640 1 -- 172.21.15.87:0/1512207510 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14014a00 con 0x7fbb24072020 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.696+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb240b0770 con 0x7fbb24072020 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.700+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb240b0d30 con 0x7fbb24072020 2024-04-25T10:24:38.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.700+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2404ef30 con 0x7fbb24072020 2024-04-25T10:24:38.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.700+0000 7fbb20c00640 1 -- 172.21.15.87:0/1512207510 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb14014ba0 con 0x7fbb24072020 2024-04-25T10:24:38.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.700+0000 7fbb20c00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb00066f60 0x7fbb00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:38.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.700+0000 7fbb20c00640 1 -- 172.21.15.87:0/1512207510 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbb140855e0 con 0x7fbb24072020 2024-04-25T10:24:38.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.700+0000 7fbb1b400640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb00066f60 0x7fbb00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:38.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.700+0000 7fbb1b400640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb00066f60 0x7fbb00069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbb241f7f70 tx=0x7fbb0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:38.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.704+0000 7fbb20c00640 1 -- 172.21.15.87:0/1512207510 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb14056850 con 0x7fbb24072020 2024-04-25T10:24:38.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.860+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbb2406d7f0 con 0x7fbb24072020 2024-04-25T10:24:38.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.860+0000 7fbb20c00640 1 -- 172.21.15.87:0/1512207510 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb14056150 con 0x7fbb24072020 2024-04-25T10:24:38.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb00066f60 msgr2=0x7fbb00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:38.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb00066f60 0x7fbb00069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbb241f7f70 tx=0x7fbb0c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:38.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 msgr2=0x7fbb24157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:38.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb24157790 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007e40 tx=0x7fbb1400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 shutdown_connections 2024-04-25T10:24:38.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbb00066f60 0x7fbb00069420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb24159540 0x7fbb240afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb240aebd0 0x7fbb240af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 --2- 172.21.15.87:0/1512207510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb24072020 0x7fbb24157790 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:38.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 >> 172.21.15.87:0/1512207510 conn(0x7fbb240669a0 msgr2=0x7fbb24066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:38.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 shutdown_connections 2024-04-25T10:24:38.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:38.864+0000 7fbb2aa00640 1 -- 172.21.15.87:0/1512207510 wait complete. 2024-04-25T10:24:38.883 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:24:38.883 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:38.937 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:39.037 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:24:39.037 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:24:39.037 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:39.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:24:39.046 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:24:39.046 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:39.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:39.056 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:24:39.057 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:39.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:24:39.097 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:24:39.097 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:39.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:39.112 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:24:39.112 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:39.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.272+0000 7fca48800640 1 Processor -- start 2024-04-25T10:24:39.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.272+0000 7fca48800640 1 -- start start 2024-04-25T10:24:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca48800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca400cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca48800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca400cfe70 0x7fca4015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca48800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca4015b740 0x7fca4006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca48800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca40071050 con 0x7fca400cfe70 2024-04-25T10:24:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca48800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca400712f0 con 0x7fca4015b740 2024-04-25T10:24:39.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca48800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca400715d0 con 0x7fca401550b0 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca400cfe70 0x7fca4015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca400cfe70 0x7fca4015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49404/0 (socket says 172.21.15.87:49404) 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 -- 172.21.15.87:0/1861762905 learned_addr learned my addr 172.21.15.87:0/1861762905 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca46800640 1 --2- 172.21.15.87:0/1861762905 >> v2:172.21.15.169:3300/0 conn(0x7fca4015b740 0x7fca4006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 -- 172.21.15.87:0/1861762905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 msgr2=0x7fca400cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45e00640 1 --2- 172.21.15.87:0/1861762905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca400cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 --2- 172.21.15.87:0/1861762905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca400cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 -- 172.21.15.87:0/1861762905 >> v2:172.21.15.169:3300/0 conn(0x7fca4015b740 msgr2=0x7fca4006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 --2- 172.21.15.87:0/1861762905 >> v2:172.21.15.169:3300/0 conn(0x7fca4015b740 0x7fca4006d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45400640 1 -- 172.21.15.87:0/1861762905 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400718b0 con 0x7fca400cfe70 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca46800640 1 --2- 172.21.15.87:0/1861762905 >> v2:172.21.15.169:3300/0 conn(0x7fca4015b740 0x7fca4006d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:39.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.276+0000 7fca45e00640 1 --2- 172.21.15.87:0/1861762905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca400cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca45400640 1 --2- 172.21.15.87:0/1861762905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca400cfe70 0x7fca4015b200 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fca400d04d0 tx=0x7fca3001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94de1248abc2107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca44a00640 1 -- 172.21.15.87:0/1861762905 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300b0070 con 0x7fca400cfe70 2024-04-25T10:24:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca44a00640 1 -- 172.21.15.87:0/1861762905 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca300ab600 con 0x7fca400cfe70 2024-04-25T10:24:39.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca44a00640 1 -- 172.21.15.87:0/1861762905 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300ab9d0 con 0x7fca400cfe70 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 -- 172.21.15.87:0/1861762905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca400cfe70 msgr2=0x7fca4015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 --2- 172.21.15.87:0/1861762905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca400cfe70 0x7fca4015b200 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fca400d04d0 tx=0x7fca3001c260 comp rx=0 tx=0).stop 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 -- 172.21.15.87:0/1861762905 shutdown_connections 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 --2- 172.21.15.87:0/1861762905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca400cfe70 0x7fca4015b200 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 --2- 172.21.15.87:0/1861762905 >> v2:172.21.15.169:3300/0 conn(0x7fca4015b740 0x7fca4006d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 --2- 172.21.15.87:0/1861762905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca400cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 -- 172.21.15.87:0/1861762905 >> 172.21.15.87:0/1861762905 conn(0x7fca40066b70 msgr2=0x7fca40066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:39.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 -- 172.21.15.87:0/1861762905 shutdown_connections 2024-04-25T10:24:39.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 -- 172.21.15.87:0/1861762905 wait complete. 2024-04-25T10:24:39.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 Processor -- start 2024-04-25T10:24:39.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.280+0000 7fca48800640 1 -- start start 2024-04-25T10:24:39.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca400cfe70 0x7fca4015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca40140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca4015b740 0x7fca400ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca400b3260 con 0x7fca4015b740 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015bd60 con 0x7fca400cfe70 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca400b33e0 con 0x7fca401550b0 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca4015b740 0x7fca400ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca4015b740 0x7fca400ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49410/0 (socket says 172.21.15.87:49410) 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 -- 172.21.15.87:0/749914616 learned_addr learned my addr 172.21.15.87:0/749914616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca45400640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca40140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca45e00640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca400cfe70 0x7fca4015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 -- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 msgr2=0x7fca40140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca40140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 -- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca400cfe70 msgr2=0x7fca4015db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca400cfe70 0x7fca4015db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 -- 172.21.15.87:0/749914616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400add90 con 0x7fca4015b740 2024-04-25T10:24:39.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca45e00640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca400cfe70 0x7fca4015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:39.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca46800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca4015b740 0x7fca400ceaa0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00cd50 tx=0x7fca3c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:39.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca3a000640 1 -- 172.21.15.87:0/749914616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00f680 con 0x7fca4015b740 2024-04-25T10:24:39.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca3a000640 1 -- 172.21.15.87:0/749914616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00f840 con 0x7fca4015b740 2024-04-25T10:24:39.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca40158b30 con 0x7fca4015b740 2024-04-25T10:24:39.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca3a000640 1 -- 172.21.15.87:0/749914616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00fc30 con 0x7fca4015b740 2024-04-25T10:24:39.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4015e090 con 0x7fca4015b740 2024-04-25T10:24:39.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4004ef30 con 0x7fca4015b740 2024-04-25T10:24:39.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.284+0000 7fca3a000640 1 -- 172.21.15.87:0/749914616 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca3c0213f0 con 0x7fca4015b740 2024-04-25T10:24:39.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.288+0000 7fca3a000640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca24066f60 0x7fca24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.288+0000 7fca3a000640 1 -- 172.21.15.87:0/749914616 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fca3c085a00 con 0x7fca4015b740 2024-04-25T10:24:39.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.288+0000 7fca45e00640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca24066f60 0x7fca24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.288+0000 7fca45e00640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca24066f60 0x7fca24069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fca34006fd0 tx=0x7fca34008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:39.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.292+0000 7fca3a000640 1 -- 172.21.15.87:0/749914616 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca3c014030 con 0x7fca4015b740 2024-04-25T10:24:39.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.476+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca4006d4c0 con 0x7fca4015b740 2024-04-25T10:24:39.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.476+0000 7fca3a000640 1 -- 172.21.15.87:0/749914616 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+3712 (secure 0 0 0) 0x7fca3c056d00 con 0x7fca4015b740 2024-04-25T10:24:39.478 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:39.478 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":113,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4139990719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4139990719},{"type":"v1","addr":"172.21.15.169:6834","nonce":4139990719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3962905873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3962905873},{"type":"v1","addr":"172.21.15.169:6835","nonce":3962905873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":10035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3473546915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3473546915},{"type":"v1","addr":"172.21.15.87:6836","nonce":3473546915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10407,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2964495303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":2964495303},{"type":"v1","addr":"172.21.15.87:6837","nonce":2964495303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[]} 2024-04-25T10:24:39.478 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 113 2024-04-25T10:24:39.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca24066f60 msgr2=0x7fca24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca24066f60 0x7fca24069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fca34006fd0 tx=0x7fca34008040 comp rx=0 tx=0).stop 2024-04-25T10:24:39.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca4015b740 msgr2=0x7fca400ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca4015b740 0x7fca400ceaa0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00cd50 tx=0x7fca3c00a650 comp rx=0 tx=0).stop 2024-04-25T10:24:39.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 shutdown_connections 2024-04-25T10:24:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca24066f60 0x7fca24069420 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca4015b740 0x7fca400ceaa0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca401550b0 0x7fca40140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 --2- 172.21.15.87:0/749914616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca400cfe70 0x7fca4015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 >> 172.21.15.87:0/749914616 conn(0x7fca40066b70 msgr2=0x7fca401509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 shutdown_connections 2024-04-25T10:24:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.480+0000 7fca48800640 1 -- 172.21.15.87:0/749914616 wait complete. 2024-04-25T10:24:39.499 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:24:39.499 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:24:39.499 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:24:39.499 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:39.500 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:24:39.500 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:24:39.500 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:24:39.500 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:39.500 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:24:39.500 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:24:39.501 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:24:39.501 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:39.501 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:24:39.501 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:24:39.501 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:24:39.501 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:39.516 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:24:39.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:24:39.518 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:24:39.518 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:24:39.519 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:24:39.519 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:24:39.520 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:24:39.520 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:24:39.561 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:24:39.561 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:39.562 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:24:39.563 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:39.568 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:24:39.569 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:39.570 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:24:39.570 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:39.570 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:24:39.649 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:24:39.649 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:24:39.657 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:24:39.657 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:24:39.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af3200640 1 Processor -- start 2024-04-25T10:24:39.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af3200640 1 -- start start 2024-04-25T10:24:39.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af3200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5af415b750 0x7f5af40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40cd590 0x7f5af40cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af3200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40b2460 con 0x7f5af40b3f80 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af3200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af40b2d70 con 0x7f5af415b750 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4070fb0 con 0x7f5af40cd590 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5af415b750 0x7f5af40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5af415b750 0x7f5af40cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49446/0 (socket says 172.21.15.87:49446) 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1e00640 1 -- 172.21.15.87:0/2908195701 learned_addr learned my addr 172.21.15.87:0/2908195701 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:39.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1400640 1 --2- 172.21.15.87:0/2908195701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40cd590 0x7f5af40cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1e00640 1 -- 172.21.15.87:0/2908195701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40cd590 msgr2=0x7f5af40cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:39.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1e00640 1 --2- 172.21.15.87:0/2908195701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40cd590 0x7f5af40cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1e00640 1 -- 172.21.15.87:0/2908195701 >> v1:172.21.15.87:6789/0 conn(0x7f5af40b3f80 legacy=0x7f5af40b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.716+0000 7f5af1e00640 1 -- 172.21.15.87:0/2908195701 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40b2720 con 0x7f5af415b750 2024-04-25T10:24:39.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af1e00640 1 --2- 172.21.15.87:0/2908195701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af415b750 0x7f5af40cbc30 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f5af40b2ef0 tx=0x7f5ae401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cb037c317ace24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:39.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af0a00640 1 -- 172.21.15.87:0/2908195701 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40aa430 con 0x7f5af415b750 2024-04-25T10:24:39.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af0a00640 1 -- 172.21.15.87:0/2908195701 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae40af070 con 0x7f5af415b750 2024-04-25T10:24:39.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af0a00640 1 -- 172.21.15.87:0/2908195701 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40aa7f0 con 0x7f5af415b750 2024-04-25T10:24:39.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 -- 172.21.15.87:0/2908195701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af415b750 msgr2=0x7f5af40cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 --2- 172.21.15.87:0/2908195701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af415b750 0x7f5af40cbc30 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f5af40b2ef0 tx=0x7f5ae401c510 comp rx=0 tx=0).stop 2024-04-25T10:24:39.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 -- 172.21.15.87:0/2908195701 shutdown_connections 2024-04-25T10:24:39.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 --2- 172.21.15.87:0/2908195701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40cd590 0x7f5af40cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 --2- 172.21.15.87:0/2908195701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af415b750 0x7f5af40cbc30 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 -- 172.21.15.87:0/2908195701 >> 172.21.15.87:0/2908195701 conn(0x7f5af40743f0 msgr2=0x7f5af40669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:39.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 -- 172.21.15.87:0/2908195701 shutdown_connections 2024-04-25T10:24:39.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 -- 172.21.15.87:0/2908195701 wait complete. 2024-04-25T10:24:39.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.720+0000 7f5af3200640 1 Processor -- start 2024-04-25T10:24:39.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 -- start start 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40b3f80 0x7f5af414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af40cd590 0x7f5af41520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b750 0x7f5af406c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40b47a0 con 0x7f5af40cd590 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4070b60 con 0x7f5af415b750 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4152940 con 0x7f5af40b3f80 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af40cd590 0x7f5af41520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af40cd590 0x7f5af41520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49450/0 (socket says 172.21.15.87:49450) 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 -- 172.21.15.87:0/748735158 learned_addr learned my addr 172.21.15.87:0/748735158 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:39.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1e00640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40b3f80 0x7f5af414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af2800640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b750 0x7f5af406c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 -- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40b3f80 msgr2=0x7f5af414f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40b3f80 0x7f5af414f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 -- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b750 msgr2=0x7f5af406c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b750 0x7f5af406c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 -- 172.21.15.87:0/748735158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af4158fd0 con 0x7f5af40cd590 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af1400640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af40cd590 0x7f5af41520f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000e930 tx=0x7f5ae000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5aea000640 1 -- 172.21.15.87:0/748735158 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0017070 con 0x7f5af40cd590 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5aea000640 1 -- 172.21.15.87:0/748735158 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae000ca00 con 0x7f5af40cd590 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af41592b0 con 0x7f5af40cd590 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5aea000640 1 -- 172.21.15.87:0/748735158 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000cdb0 con 0x7f5af40cd590 2024-04-25T10:24:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af414fc80 con 0x7f5af40cd590 2024-04-25T10:24:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.724+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4075120 con 0x7f5af40cd590 2024-04-25T10:24:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.728+0000 7f5aea000640 1 -- 172.21.15.87:0/748735158 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ae001a070 con 0x7f5af40cd590 2024-04-25T10:24:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.728+0000 7f5aea000640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ad4066f60 0x7f5ad4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.728+0000 7f5aea000640 1 -- 172.21.15.87:0/748735158 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5ae00851d0 con 0x7f5af40cd590 2024-04-25T10:24:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.728+0000 7f5af1e00640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ad4066f60 0x7f5ad4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.728+0000 7f5af1e00640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ad4066f60 0x7f5ad4069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40ae000 tx=0x7f5ae40a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:39.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.732+0000 7f5aea000640 1 -- 172.21.15.87:0/748735158 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ae0056500 con 0x7f5af40cd590 2024-04-25T10:24:39.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:39.900+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5af415f130 con 0x7f5af40cd590 2024-04-25T10:24:40.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.888+0000 7f5aea000640 1 -- 172.21.15.87:0/748735158 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v159) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5ae0055e00 con 0x7f5af40cd590 2024-04-25T10:24:40.888 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:24:40.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ad4066f60 msgr2=0x7f5ad4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:40.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ad4066f60 0x7f5ad4069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40ae000 tx=0x7f5ae40a7000 comp rx=0 tx=0).stop 2024-04-25T10:24:40.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af40cd590 msgr2=0x7f5af41520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:40.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af40cd590 0x7f5af41520f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000e930 tx=0x7f5ae000edf0 comp rx=0 tx=0).stop 2024-04-25T10:24:40.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 shutdown_connections 2024-04-25T10:24:40.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ad4066f60 0x7f5ad4069420 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:40.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b750 0x7f5af406c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:40.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5af40cd590 0x7f5af41520f0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:40.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 --2- 172.21.15.87:0/748735158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af40b3f80 0x7f5af414f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:40.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 >> 172.21.15.87:0/748735158 conn(0x7f5af40743f0 msgr2=0x7f5af415c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:40.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 shutdown_connections 2024-04-25T10:24:40.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:40.892+0000 7f5af3200640 1 -- 172.21.15.87:0/748735158 wait complete. 2024-04-25T10:24:40.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:24:41.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.048+0000 7f79be400640 1 Processor -- start 2024-04-25T10:24:41.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.048+0000 7f79be400640 1 -- start start 2024-04-25T10:24:41.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.048+0000 7f79be400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 0x7f79b80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.048+0000 7f79be400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b8071050 con 0x7f79b80cfe70 2024-04-25T10:24:41.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.048+0000 7f79be400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b815d040 con 0x7f79b815b910 2024-04-25T10:24:41.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.048+0000 7f79be400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b815bda0 con 0x7f79b81550b0 2024-04-25T10:24:41.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f79b80cfe70 0x7f79b815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44350/0 (socket says 172.21.15.87:44350) 2024-04-25T10:24:41.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6a00640 1 -- 172.21.15.87:0/2670035858 learned_addr learned my addr 172.21.15.87:0/2670035858 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:41.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7400640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 0x7f79b80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347067142 0 0) 0x7f79b8071050 con 0x7f79b80cfe70 2024-04-25T10:24:41.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c003530 con 0x7f79b80cfe70 2024-04-25T10:24:41.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220184911 0 0) 0x7f799c003530 con 0x7f79b80cfe70 2024-04-25T10:24:41.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 msgr2=0x7f79b80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:41.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 0x7f79b80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 >> v1:172.21.15.169:6789/0 conn(0x7f79b815b910 legacy=0x7f79b80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80712f0 con 0x7f79b80cfe70 2024-04-25T10:24:41.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f79a4001680 con 0x7f79b80cfe70 2024-04-25T10:24:41.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 >> v1:172.21.15.87:6789/0 conn(0x7f79b80cfe70 legacy=0x7f79b815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 0x7f799c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f799c003f70 0x7f799c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f799c004d50 0x7f799c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00dc90 con 0x7f799c004d50 2024-04-25T10:24:41.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00de40 con 0x7f799c003f70 2024-04-25T10:24:41.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00e040 con 0x7f79b81550b0 2024-04-25T10:24:41.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7e00640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f799c004d50 0x7f799c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7400640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 0x7f799c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6a00640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f799c003f70 0x7f799c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7e00640 1 -- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 msgr2=0x7f799c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:41.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7e00640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b81550b0 0x7f799c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7e00640 1 -- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f799c003f70 msgr2=0x7f799c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:41.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7e00640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f799c003f70 0x7f799c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b7e00640 1 -- 172.21.15.87:0/2670035858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c003530 con 0x7f799c004d50 2024-04-25T10:24:41.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.052+0000 7f79b6a00640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f799c003f70 0x7f799c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:41.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79b7e00640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f799c004d50 0x7f799c0051d0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f79b0001780 tx=0x7f79b001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a32f4e90666cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b00013a0 con 0x7f799c004d50 2024-04-25T10:24:41.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b00a9070 con 0x7f799c004d50 2024-04-25T10:24:41.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79b6000640 1 -- 172.21.15.87:0/2670035858 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0003600 con 0x7f799c004d50 2024-04-25T10:24:41.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 -- 172.21.15.87:0/2670035858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f799c004d50 msgr2=0x7f799c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 --2- 172.21.15.87:0/2670035858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f799c004d50 0x7f799c0051d0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f79b0001780 tx=0x7f79b001c420 comp rx=0 tx=0).stop 2024-04-25T10:24:41.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79b7e00640 1 -- 172.21.15.87:0/2670035858 reap_dead start 2024-04-25T10:24:41.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 -- 172.21.15.87:0/2670035858 shutdown_connections 2024-04-25T10:24:41.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 -- 172.21.15.87:0/2670035858 >> 172.21.15.87:0/2670035858 conn(0x7f79b8066b70 msgr2=0x7f79b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:41.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 -- 172.21.15.87:0/2670035858 shutdown_connections 2024-04-25T10:24:41.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 -- 172.21.15.87:0/2670035858 wait complete. 2024-04-25T10:24:41.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 Processor -- start 2024-04-25T10:24:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.056+0000 7f79be400640 1 -- start start 2024-04-25T10:24:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8142f90 0x7f79b80cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b80ccde0 0x7f79b8145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b8145a60 0x7f79b8145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b8071b90 con 0x7f79b80ccde0 2024-04-25T10:24:41.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80718b0 con 0x7f79b8145a60 2024-04-25T10:24:41.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80720b0 con 0x7f79b8142f90 2024-04-25T10:24:41.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b80ccde0 0x7f79b8145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b80ccde0 0x7f79b8145520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49460/0 (socket says 172.21.15.87:49460) 2024-04-25T10:24:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 -- 172.21.15.87:0/2529537048 learned_addr learned my addr 172.21.15.87:0/2529537048 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b7400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8142f90 0x7f79b80cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b7e00640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b8145a60 0x7f79b8145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 -- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8142f90 msgr2=0x7f79b80cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8142f90 0x7f79b80cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 -- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b8145a60 msgr2=0x7f79b8145ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b8145a60 0x7f79b8145ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 -- 172.21.15.87:0/2529537048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80cd610 con 0x7f79b80ccde0 2024-04-25T10:24:41.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b7e00640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b8145a60 0x7f79b8145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:41.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79b6a00640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b80ccde0 0x7f79b8145520 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f79a400e8c0 tx=0x7f79a400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79af400640 1 -- 172.21.15.87:0/2529537048 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400c8d0 con 0x7f79b80ccde0 2024-04-25T10:24:41.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79af400640 1 -- 172.21.15.87:0/2529537048 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4018070 con 0x7f79b80ccde0 2024-04-25T10:24:41.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79af400640 1 -- 172.21.15.87:0/2529537048 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400ccb0 con 0x7f79b80ccde0 2024-04-25T10:24:41.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b80cd8f0 con 0x7f79b80ccde0 2024-04-25T10:24:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b80cdeb0 con 0x7f79b80ccde0 2024-04-25T10:24:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.060+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b804ef30 con 0x7f79b80ccde0 2024-04-25T10:24:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.064+0000 7f79af400640 1 -- 172.21.15.87:0/2529537048 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79a401d460 con 0x7f79b80ccde0 2024-04-25T10:24:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.064+0000 7f79af400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.064+0000 7f79af400640 1 -- 172.21.15.87:0/2529537048 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f79a4084d00 con 0x7f79b80ccde0 2024-04-25T10:24:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.064+0000 7f79b7400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.064+0000 7f79b7400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f79a8014510 tx=0x7f79a800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.068+0000 7f79af400640 1 -- 172.21.15.87:0/2529537048 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79a4056050 con 0x7f79b80ccde0 2024-04-25T10:24:41.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.272+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f79b80671d0 con 0x7f79b80ccde0 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.272+0000 7f79af400640 1 -- 172.21.15.87:0/2529537048 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f79a4055950 con 0x7f79b80ccde0 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:24:41.273 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.274 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.275 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:24:41.276 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.278 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.279 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.281 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:24:41.285 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:24:41.286 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:24:41.287 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.287 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:41.287 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:24:41.287 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 msgr2=0x7f7998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f79a8014510 tx=0x7f79a800b040 comp rx=0 tx=0).stop 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b80ccde0 msgr2=0x7f79b8145520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b80ccde0 0x7f79b8145520 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f79a400e8c0 tx=0x7f79a400ed80 comp rx=0 tx=0).stop 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 shutdown_connections 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b8145a60 0x7f79b8145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79b80ccde0 0x7f79b8145520 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 --2- 172.21.15.87:0/2529537048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79b8142f90 0x7f79b80cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 >> 172.21.15.87:0/2529537048 conn(0x7f79b8066b70 msgr2=0x7f79b80b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:41.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 shutdown_connections 2024-04-25T10:24:41.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.276+0000 7f79be400640 1 -- 172.21.15.87:0/2529537048 wait complete. 2024-04-25T10:24:41.295 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:41.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5327a00640 1 Processor -- start 2024-04-25T10:24:41.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5327a00640 1 -- start start 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5327a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f532006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5327a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53200b2770 con 0x7f532015bc40 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5327a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532006c670 con 0x7f53200cda10 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5327a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532006c950 con 0x7f53200d0040 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5325000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f532015bc40 0x7f53200b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44360/0 (socket says 172.21.15.87:44360) 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5325000640 1 -- 172.21.15.87:0/681764012 learned_addr learned my addr 172.21.15.87:0/681764012 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5325a00640 1 --2- 172.21.15.87:0/681764012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f532006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211890798 0 0) 0x7f53200b2770 con 0x7f532015bc40 2024-04-25T10:24:41.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5304003530 con 0x7f532015bc40 2024-04-25T10:24:41.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154484386 0 0) 0x7f532006c670 con 0x7f53200cda10 2024-04-25T10:24:41.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53200b2770 con 0x7f53200cda10 2024-04-25T10:24:41.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1226042290 0 0) 0x7f5304003530 con 0x7f532015bc40 2024-04-25T10:24:41.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5325a00640 1 -- 172.21.15.87:0/681764012 >> v1:172.21.15.169:6789/0 conn(0x7f53200cda10 legacy=0x7f53200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5325a00640 1 -- 172.21.15.87:0/681764012 >> v1:172.21.15.87:6789/0 conn(0x7f532015bc40 legacy=0x7f53200b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.432+0000 7f5325a00640 1 -- 172.21.15.87:0/681764012 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532006cc30 con 0x7f53200d0040 2024-04-25T10:24:41.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5325a00640 1 --2- 172.21.15.87:0/681764012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f532006c090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5318009dc0 tx=0x7f531801c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd80bb8fa756977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180b2070 con 0x7f53200d0040 2024-04-25T10:24:41.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53180a5570 con 0x7f53200d0040 2024-04-25T10:24:41.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f531fe00640 1 -- 172.21.15.87:0/681764012 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310001600 con 0x7f53200d0040 2024-04-25T10:24:41.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5327a00640 1 -- 172.21.15.87:0/681764012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 msgr2=0x7f532006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5327a00640 1 --2- 172.21.15.87:0/681764012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f532006c090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5318009dc0 tx=0x7f531801c820 comp rx=0 tx=0).stop 2024-04-25T10:24:41.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5327a00640 1 -- 172.21.15.87:0/681764012 shutdown_connections 2024-04-25T10:24:41.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5327a00640 1 --2- 172.21.15.87:0/681764012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f532006c090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5327a00640 1 -- 172.21.15.87:0/681764012 >> 172.21.15.87:0/681764012 conn(0x7f5320066b70 msgr2=0x7f5320066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:41.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5327a00640 1 -- 172.21.15.87:0/681764012 shutdown_connections 2024-04-25T10:24:41.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.436+0000 7f5327a00640 1 -- 172.21.15.87:0/681764012 wait complete. 2024-04-25T10:24:41.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 Processor -- start 2024-04-25T10:24:41.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 -- start start 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53200cda10 0x7f53200b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f5320151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532015bc40 0x7f532014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5320169980 con 0x7f532015bc40 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5320169800 con 0x7f53200cda10 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5320169b00 con 0x7f53200d0040 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532015bc40 0x7f532014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532015bc40 0x7f532014e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49474/0 (socket says 172.21.15.87:49474) 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 -- 172.21.15.87:0/674960345 learned_addr learned my addr 172.21.15.87:0/674960345 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5317e00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f5320151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325000640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53200cda10 0x7f53200b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 -- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 msgr2=0x7f5320151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f5320151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 -- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53200cda10 msgr2=0x7f53200b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53200cda10 0x7f53200b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 -- 172.21.15.87:0/674960345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532015ee80 con 0x7f532015bc40 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5325a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532015bc40 0x7f532014e5d0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f531801cd00 tx=0x7f53180195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f531d600640 1 -- 172.21.15.87:0/674960345 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180b2040 con 0x7f532015bc40 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f531d600640 1 -- 172.21.15.87:0/674960345 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318003bc0 con 0x7f532015bc40 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f531d600640 1 -- 172.21.15.87:0/674960345 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318009450 con 0x7f532015bc40 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.440+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532014f460 con 0x7f532015bc40 2024-04-25T10:24:41.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.444+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532014fa20 con 0x7f532015bc40 2024-04-25T10:24:41.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.444+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532004ef30 con 0x7f532015bc40 2024-04-25T10:24:41.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.444+0000 7f531d600640 1 -- 172.21.15.87:0/674960345 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5318004030 con 0x7f532015bc40 2024-04-25T10:24:41.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.444+0000 7f531d600640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52fc066f90 0x7f52fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.444+0000 7f531d600640 1 -- 172.21.15.87:0/674960345 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f531811fd80 con 0x7f532015bc40 2024-04-25T10:24:41.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.444+0000 7f5325000640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52fc066f90 0x7f52fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.444+0000 7f5325000640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52fc066f90 0x7f52fc069450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5310005890 tx=0x7f5310009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.448+0000 7f531d600640 1 -- 172.21.15.87:0/674960345 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53180c7070 con 0x7f532015bc40 2024-04-25T10:24:41.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f532006d960 con 0x7f532015bc40 2024-04-25T10:24:41.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f531d600640 1 -- 172.21.15.87:0/674960345 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f53180164a0 con 0x7f532015bc40 2024-04-25T10:24:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:41.608 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":159,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:24:40.887853+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:41.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52fc066f90 msgr2=0x7f52fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52fc066f90 0x7f52fc069450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5310005890 tx=0x7f5310009040 comp rx=0 tx=0).stop 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532015bc40 msgr2=0x7f532014e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532015bc40 0x7f532014e5d0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f531801cd00 tx=0x7f53180195c0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 shutdown_connections 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f52fc066f90 0x7f52fc069450 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532015bc40 0x7f532014e5d0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53200d0040 0x7f5320151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 --2- 172.21.15.87:0/674960345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53200cda10 0x7f53200b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 >> 172.21.15.87:0/674960345 conn(0x7f5320066b70 msgr2=0x7f5320159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:41.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 shutdown_connections 2024-04-25T10:24:41.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.608+0000 7f5327a00640 1 -- 172.21.15.87:0/674960345 wait complete. 2024-04-25T10:24:41.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:41.630 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:41.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3094e00640 1 Processor -- start 2024-04-25T10:24:41.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3094e00640 1 -- start start 2024-04-25T10:24:41.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3094e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30900721f0 0x7f3090166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3094e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3090071360 con 0x7f30900aeda0 2024-04-25T10:24:41.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3094e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3090159920 con 0x7f3090159540 2024-04-25T10:24:41.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3094e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309015c2f0 con 0x7f30900721f0 2024-04-25T10:24:41.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3085400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f30900aeda0 0x7f3090071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44376/0 (socket says 172.21.15.87:44376) 2024-04-25T10:24:41.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f3085400640 1 -- 172.21.15.87:0/2141666985 learned_addr learned my addr 172.21.15.87:0/2141666985 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:41.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.768+0000 7f308e800640 1 --2- 172.21.15.87:0/2141666985 >> v2:172.21.15.169:3300/0 conn(0x7f30900721f0 0x7f3090166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308d400640 1 -- 172.21.15.87:0/2141666985 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509332338 0 0) 0x7f3090071360 con 0x7f30900aeda0 2024-04-25T10:24:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308d400640 1 -- 172.21.15.87:0/2141666985 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3074003530 con 0x7f30900aeda0 2024-04-25T10:24:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308d400640 1 -- 172.21.15.87:0/2141666985 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431527612 0 0) 0x7f3090159920 con 0x7f3090159540 2024-04-25T10:24:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308d400640 1 -- 172.21.15.87:0/2141666985 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3090071360 con 0x7f3090159540 2024-04-25T10:24:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308e800640 1 -- 172.21.15.87:0/2141666985 >> v1:172.21.15.169:6789/0 conn(0x7f3090159540 legacy=0x7f30900ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308e800640 1 -- 172.21.15.87:0/2141666985 >> v1:172.21.15.87:6789/0 conn(0x7f30900aeda0 legacy=0x7f3090071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308e800640 1 -- 172.21.15.87:0/2141666985 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309015c5d0 con 0x7f30900721f0 2024-04-25T10:24:41.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308e800640 1 --2- 172.21.15.87:0/2141666985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f3090166920 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f30880097c0 tx=0x7f308801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80c9b7e073a2c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308d400640 1 -- 172.21.15.87:0/2141666985 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880a7070 con 0x7f30900721f0 2024-04-25T10:24:41.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308d400640 1 -- 172.21.15.87:0/2141666985 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30880b0070 con 0x7f30900721f0 2024-04-25T10:24:41.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.772+0000 7f308d400640 1 -- 172.21.15.87:0/2141666985 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880ab890 con 0x7f30900721f0 2024-04-25T10:24:41.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- 172.21.15.87:0/2141666985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 msgr2=0x7f3090166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 --2- 172.21.15.87:0/2141666985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f3090166920 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f30880097c0 tx=0x7f308801c470 comp rx=0 tx=0).stop 2024-04-25T10:24:41.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- 172.21.15.87:0/2141666985 shutdown_connections 2024-04-25T10:24:41.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 --2- 172.21.15.87:0/2141666985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f3090166920 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- 172.21.15.87:0/2141666985 >> 172.21.15.87:0/2141666985 conn(0x7f30900669a0 msgr2=0x7f3090066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:41.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- 172.21.15.87:0/2141666985 shutdown_connections 2024-04-25T10:24:41.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- 172.21.15.87:0/2141666985 wait complete. 2024-04-25T10:24:41.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 Processor -- start 2024-04-25T10:24:41.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- start start 2024-04-25T10:24:41.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f309014f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30900aeda0 0x7f309015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3090159540 0x7f309015b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3090169680 con 0x7f3090159540 2024-04-25T10:24:41.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309006c0d0 con 0x7f30900721f0 2024-04-25T10:24:41.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3094e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3090169800 con 0x7f30900aeda0 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3090159540 0x7f309015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3090159540 0x7f309015b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49490/0 (socket says 172.21.15.87:49490) 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 -- 172.21.15.87:0/1423548387 learned_addr learned my addr 172.21.15.87:0/1423548387 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3085400640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30900aeda0 0x7f309015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308de00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f309014f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 -- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30900aeda0 msgr2=0x7f309015b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30900aeda0 0x7f309015b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 -- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 msgr2=0x7f309014f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f309014f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 -- 172.21.15.87:0/1423548387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309015bc40 con 0x7f3090159540 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f3085400640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30900aeda0 0x7f309015b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308de00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f309014f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.776+0000 7f308e800640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3090159540 0x7f309015b610 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f30880ad040 tx=0x7f308801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3086a00640 1 -- 172.21.15.87:0/1423548387 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880bf410 con 0x7f3090159540 2024-04-25T10:24:41.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3086a00640 1 -- 172.21.15.87:0/1423548387 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30880bf5b0 con 0x7f3090159540 2024-04-25T10:24:41.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309014ffe0 con 0x7f3090159540 2024-04-25T10:24:41.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3086a00640 1 -- 172.21.15.87:0/1423548387 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880bf960 con 0x7f3090159540 2024-04-25T10:24:41.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3090153690 con 0x7f3090159540 2024-04-25T10:24:41.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309004ef30 con 0x7f3090159540 2024-04-25T10:24:41.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3086a00640 1 -- 172.21.15.87:0/1423548387 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30880bd020 con 0x7f3090159540 2024-04-25T10:24:41.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3086a00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f306c066f60 0x7f306c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:41.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f3086a00640 1 -- 172.21.15.87:0/1423548387 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f308811f670 con 0x7f3090159540 2024-04-25T10:24:41.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f308de00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f306c066f60 0x7f306c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:41.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.780+0000 7f308de00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f306c066f60 0x7f306c069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3080001d30 tx=0x7f3080009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:41.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.784+0000 7f3086a00640 1 -- 172.21.15.87:0/1423548387 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3088016090 con 0x7f3090159540 2024-04-25T10:24:41.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.940+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f309006d140 con 0x7f3090159540 2024-04-25T10:24:41.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.940+0000 7f3086a00640 1 -- 172.21.15.87:0/1423548387 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30880f0ae0 con 0x7f3090159540 2024-04-25T10:24:41.941 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:24:41.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f306c066f60 msgr2=0x7f306c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f306c066f60 0x7f306c069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3080001d30 tx=0x7f3080009040 comp rx=0 tx=0).stop 2024-04-25T10:24:41.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3090159540 msgr2=0x7f309015b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:41.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3090159540 0x7f309015b610 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f30880ad040 tx=0x7f308801acb0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 shutdown_connections 2024-04-25T10:24:41.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f306c066f60 0x7f306c069420 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3090159540 0x7f309015b610 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30900aeda0 0x7f309015b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 --2- 172.21.15.87:0/1423548387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30900721f0 0x7f309014f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:41.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 >> 172.21.15.87:0/1423548387 conn(0x7f30900669a0 msgr2=0x7f3090071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:41.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 shutdown_connections 2024-04-25T10:24:41.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:41.944+0000 7f3094e00640 1 -- 172.21.15.87:0/1423548387 wait complete. 2024-04-25T10:24:41.962 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:24:42.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.096+0000 7fd28dc00640 1 Processor -- start 2024-04-25T10:24:42.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd28dc00640 1 -- start start 2024-04-25T10:24:42.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd28dc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2881476f0 0x7fd28806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd288161760 0x7fd288161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd28dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd288162220 con 0x7fd2880cc3e0 2024-04-25T10:24:42.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd28dc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28806c740 con 0x7fd2881476f0 2024-04-25T10:24:42.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd28dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd288162500 con 0x7fd288161760 2024-04-25T10:24:42.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd287e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2881476f0 0x7fd28806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd287e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2881476f0 0x7fd28806d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49500/0 (socket says 172.21.15.87:49500) 2024-04-25T10:24:42.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd287e00640 1 -- 172.21.15.87:0/2884723798 learned_addr learned my addr 172.21.15.87:0/2884723798 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:42.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd27fe00640 1 --2- 172.21.15.87:0/2884723798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd288161760 0x7fd288161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd287e00640 1 -- 172.21.15.87:0/2884723798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd288161760 msgr2=0x7fd288161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:42.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd287e00640 1 --2- 172.21.15.87:0/2884723798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd288161760 0x7fd288161bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd287e00640 1 -- 172.21.15.87:0/2884723798 >> v1:172.21.15.169:6789/0 conn(0x7fd2880cc3e0 legacy=0x7fd2880ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.100+0000 7fd287e00640 1 -- 172.21.15.87:0/2884723798 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28806c990 con 0x7fd2881476f0 2024-04-25T10:24:42.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd287e00640 1 --2- 172.21.15.87:0/2884723798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2881476f0 0x7fd28806d190 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fd278009870 tx=0x7fd27801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc5009e5646bb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:42.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd287400640 1 -- 172.21.15.87:0/2884723798 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780b1070 con 0x7fd2881476f0 2024-04-25T10:24:42.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd287400640 1 -- 172.21.15.87:0/2884723798 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2780ac620 con 0x7fd2881476f0 2024-04-25T10:24:42.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd287400640 1 -- 172.21.15.87:0/2884723798 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780ac9f0 con 0x7fd2881476f0 2024-04-25T10:24:42.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 -- 172.21.15.87:0/2884723798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2881476f0 msgr2=0x7fd28806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 --2- 172.21.15.87:0/2884723798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2881476f0 0x7fd28806d190 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fd278009870 tx=0x7fd27801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 -- 172.21.15.87:0/2884723798 shutdown_connections 2024-04-25T10:24:42.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 --2- 172.21.15.87:0/2884723798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd288161760 0x7fd288161bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 --2- 172.21.15.87:0/2884723798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2881476f0 0x7fd28806d190 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 -- 172.21.15.87:0/2884723798 >> 172.21.15.87:0/2884723798 conn(0x7fd2880b16a0 msgr2=0x7fd2880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:42.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 -- 172.21.15.87:0/2884723798 shutdown_connections 2024-04-25T10:24:42.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 -- 172.21.15.87:0/2884723798 wait complete. 2024-04-25T10:24:42.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 Processor -- start 2024-04-25T10:24:42.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.104+0000 7fd28dc00640 1 -- start start 2024-04-25T10:24:42.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2880cc3e0 0x7fd2880b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2881476f0 0x7fd28813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288161760 0x7fd2881457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806e2a0 con 0x7fd2880cc3e0 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806e120 con 0x7fd288161760 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28816b2d0 con 0x7fd2881476f0 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2880cc3e0 0x7fd2880b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2880cc3e0 0x7fd2880b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49504/0 (socket says 172.21.15.87:49504) 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 -- 172.21.15.87:0/3884902418 learned_addr learned my addr 172.21.15.87:0/3884902418 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd27fe00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2881476f0 0x7fd28813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28d200640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288161760 0x7fd2881457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 -- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2881476f0 msgr2=0x7fd28813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2881476f0 0x7fd28813d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 -- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288161760 msgr2=0x7fd2881457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288161760 0x7fd2881457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 -- 172.21.15.87:0/3884902418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2880b3690 con 0x7fd2880cc3e0 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28d200640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288161760 0x7fd2881457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd287e00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2880cc3e0 0x7fd2880b3150 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fd2780af040 tx=0x7fd278019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:42.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd284c00640 1 -- 172.21.15.87:0/3884902418 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780b1040 con 0x7fd2880cc3e0 2024-04-25T10:24:42.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd284c00640 1 -- 172.21.15.87:0/3884902418 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2780ba070 con 0x7fd2880cc3e0 2024-04-25T10:24:42.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28815ca10 con 0x7fd2880cc3e0 2024-04-25T10:24:42.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd284c00640 1 -- 172.21.15.87:0/3884902418 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780b5660 con 0x7fd2880cc3e0 2024-04-25T10:24:42.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28815cfd0 con 0x7fd2880cc3e0 2024-04-25T10:24:42.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd288075120 con 0x7fd2880cc3e0 2024-04-25T10:24:42.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.108+0000 7fd284c00640 1 -- 172.21.15.87:0/3884902418 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2780b5800 con 0x7fd2880cc3e0 2024-04-25T10:24:42.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.112+0000 7fd284c00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd264066f60 0x7fd264069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.112+0000 7fd284c00640 1 -- 172.21.15.87:0/3884902418 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd27811ea40 con 0x7fd2880cc3e0 2024-04-25T10:24:42.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.112+0000 7fd27fe00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd264066f60 0x7fd264069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.112+0000 7fd27fe00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd264066f60 0x7fd264069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd288146ab0 tx=0x7fd270008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:42.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.116+0000 7fd284c00640 1 -- 172.21.15.87:0/3884902418 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd278016150 con 0x7fd2880cc3e0 2024-04-25T10:24:42.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.268+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd288068c10 con 0x7fd2880cc3e0 2024-04-25T10:24:42.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.268+0000 7fd284c00640 1 -- 172.21.15.87:0/3884902418 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v159) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd2780efda0 con 0x7fd2880cc3e0 2024-04-25T10:24:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:24:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:24:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:24:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:24:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:24:42.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd264066f60 msgr2=0x7fd264069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd264066f60 0x7fd264069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd288146ab0 tx=0x7fd270008040 comp rx=0 tx=0).stop 2024-04-25T10:24:42.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2880cc3e0 msgr2=0x7fd2880b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2880cc3e0 0x7fd2880b3150 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fd2780af040 tx=0x7fd278019270 comp rx=0 tx=0).stop 2024-04-25T10:24:42.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 shutdown_connections 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd264066f60 0x7fd264069420 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288161760 0x7fd2881457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2881476f0 0x7fd28813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 --2- 172.21.15.87:0/3884902418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2880cc3e0 0x7fd2880b3150 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 >> 172.21.15.87:0/3884902418 conn(0x7fd2880b16a0 msgr2=0x7fd2880706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 shutdown_connections 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.272+0000 7fd28dc00640 1 -- 172.21.15.87:0/3884902418 wait complete. 2024-04-25T10:24:42.290 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:24:42.290 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:24:42.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.428+0000 7f7262800640 1 Processor -- start 2024-04-25T10:24:42.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.428+0000 7f7262800640 1 -- start start 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f7262800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f725c1474c0 0x7f725c162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f7262800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f725c15e1f0 0x7f725c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f7262800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f7262800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c067350 con 0x7f725c15e1f0 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f7262800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c1478a0 con 0x7f725c1474c0 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f7262800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c161960 con 0x7f725c0669a0 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f725c15e1f0 0x7f725c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f725c15e1f0 0x7f725c15e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49514/0 (socket says 172.21.15.87:49514) 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 -- 172.21.15.87:0/751925369 learned_addr learned my addr 172.21.15.87:0/751925369 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:42.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725be00640 1 --2- 172.21.15.87:0/751925369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725b400640 1 --2- 172.21.15.87:0/751925369 >> v2:172.21.15.169:3300/0 conn(0x7f725c1474c0 0x7f725c162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 -- 172.21.15.87:0/751925369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 msgr2=0x7f725c066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:42.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 --2- 172.21.15.87:0/751925369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c066e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 -- 172.21.15.87:0/751925369 >> v2:172.21.15.169:3300/0 conn(0x7f725c1474c0 msgr2=0x7f725c162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:42.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 --2- 172.21.15.87:0/751925369 >> v2:172.21.15.169:3300/0 conn(0x7f725c1474c0 0x7f725c162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.432+0000 7f725aa00640 1 -- 172.21.15.87:0/751925369 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c15ecb0 con 0x7f725c15e1f0 2024-04-25T10:24:42.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f725aa00640 1 --2- 172.21.15.87:0/751925369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c15e1f0 0x7f725c15e6a0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f725c06bcd0 tx=0x7f724801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b211578958d16e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:42.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f725a000640 1 -- 172.21.15.87:0/751925369 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72480ab650 con 0x7f725c15e1f0 2024-04-25T10:24:42.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f725a000640 1 -- 172.21.15.87:0/751925369 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72480b0070 con 0x7f725c15e1f0 2024-04-25T10:24:42.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f725a000640 1 -- 172.21.15.87:0/751925369 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72480aba10 con 0x7f725c15e1f0 2024-04-25T10:24:42.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 -- 172.21.15.87:0/751925369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c15e1f0 msgr2=0x7f725c15e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 --2- 172.21.15.87:0/751925369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c15e1f0 0x7f725c15e6a0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f725c06bcd0 tx=0x7f724801c490 comp rx=0 tx=0).stop 2024-04-25T10:24:42.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 -- 172.21.15.87:0/751925369 shutdown_connections 2024-04-25T10:24:42.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 --2- 172.21.15.87:0/751925369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c15e1f0 0x7f725c15e6a0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 --2- 172.21.15.87:0/751925369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c066e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 --2- 172.21.15.87:0/751925369 >> v2:172.21.15.169:3300/0 conn(0x7f725c1474c0 0x7f725c162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 -- 172.21.15.87:0/751925369 >> 172.21.15.87:0/751925369 conn(0x7f725c0b2560 msgr2=0x7f725c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:42.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 -- 172.21.15.87:0/751925369 shutdown_connections 2024-04-25T10:24:42.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 -- 172.21.15.87:0/751925369 wait complete. 2024-04-25T10:24:42.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 Processor -- start 2024-04-25T10:24:42.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 -- start start 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1474c0 0x7f725c132be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.436+0000 7f7262800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15e1f0 0x7f725c141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7262800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0724d0 con 0x7f725c1474c0 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7262800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c072350 con 0x7f725c15e1f0 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7262800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c06dd00 con 0x7f725c0669a0 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1474c0 0x7f725c132be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1474c0 0x7f725c132be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49530/0 (socket says 172.21.15.87:49530) 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 -- 172.21.15.87:0/4196739100 learned_addr learned my addr 172.21.15.87:0/4196739100 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725b400640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725be00640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15e1f0 0x7f725c141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 -- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 msgr2=0x7f725c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 -- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15e1f0 msgr2=0x7f725c141540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15e1f0 0x7f725c141540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:42.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 -- 172.21.15.87:0/4196739100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c1550e0 con 0x7f725c1474c0 2024-04-25T10:24:42.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f725aa00640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1474c0 0x7f725c132be0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f72480ad040 tx=0x7f7248019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:42.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7253400640 1 -- 172.21.15.87:0/4196739100 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72480bf610 con 0x7f725c1474c0 2024-04-25T10:24:42.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7253400640 1 -- 172.21.15.87:0/4196739100 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72480bf7b0 con 0x7f725c1474c0 2024-04-25T10:24:42.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7253400640 1 -- 172.21.15.87:0/4196739100 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72480bfba0 con 0x7f725c1474c0 2024-04-25T10:24:42.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c1349d0 con 0x7f725c1474c0 2024-04-25T10:24:42.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c155370 con 0x7f725c1474c0 2024-04-25T10:24:42.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c04ef30 con 0x7f725c1474c0 2024-04-25T10:24:42.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.440+0000 7f7253400640 1 -- 172.21.15.87:0/4196739100 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72480bd020 con 0x7f725c1474c0 2024-04-25T10:24:42.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.444+0000 7f7253400640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:42.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.444+0000 7f7253400640 1 -- 172.21.15.87:0/4196739100 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f724811f6b0 con 0x7f725c1474c0 2024-04-25T10:24:42.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.444+0000 7f725b400640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:42.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.444+0000 7f725b400640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f723c066f60 0x7f723c069420 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f724c0079e0 tx=0x7f724c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:42.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.448+0000 7f7253400640 1 -- 172.21.15.87:0/4196739100 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7248016110 con 0x7f725c1474c0 2024-04-25T10:24:42.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:42.604+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f725c071e00 con 0x7f725c1474c0 2024-04-25T10:24:43.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7253400640 1 -- 172.21.15.87:0/4196739100 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v160) v1 ==== 138+0+0 (secure 0 0 0) 0x7f72480f0440 con 0x7f725c1474c0 2024-04-25T10:24:43.543 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:24:43.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f723c066f60 msgr2=0x7f723c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f723c066f60 0x7f723c069420 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f724c0079e0 tx=0x7f724c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:43.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1474c0 msgr2=0x7f725c132be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1474c0 0x7f725c132be0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f72480ad040 tx=0x7f7248019230 comp rx=0 tx=0).stop 2024-04-25T10:24:43.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 shutdown_connections 2024-04-25T10:24:43.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15e1f0 0x7f725c141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1474c0 0x7f725c132be0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 --2- 172.21.15.87:0/4196739100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0669a0 0x7f725c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 >> 172.21.15.87:0/4196739100 conn(0x7f725c0b2560 msgr2=0x7f725c143920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 shutdown_connections 2024-04-25T10:24:43.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.544+0000 7f7262800640 1 -- 172.21.15.87:0/4196739100 wait complete. 2024-04-25T10:24:43.574 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:24:43.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d8b000640 1 Processor -- start 2024-04-25T10:24:43.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d8b000640 1 -- start start 2024-04-25T10:24:43.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d8b000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d840b4660 0x7f4d8406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d8b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 0x7f4d840cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d8b000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8406ca90 con 0x7f4d8415bbb0 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d8b000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8406d050 con 0x7f4d840b4660 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d8b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8406cd70 con 0x7f4d840cf960 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d7b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d840b4660 0x7f4d8406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d7b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d840b4660 0x7f4d8406c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49546/0 (socket says 172.21.15.87:49546) 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d7b400640 1 -- 172.21.15.87:0/4203290779 learned_addr learned my addr 172.21.15.87:0/4203290779 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d83e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4d8415bbb0 0x7f4d840cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44392/0 (socket says 172.21.15.87:44392) 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d89000640 1 --2- 172.21.15.87:0/4203290779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 0x7f4d840cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d7b400640 1 -- 172.21.15.87:0/4203290779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 msgr2=0x7f4d840cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d7b400640 1 --2- 172.21.15.87:0/4203290779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 0x7f4d840cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d7b400640 1 -- 172.21.15.87:0/4203290779 >> v1:172.21.15.87:6789/0 conn(0x7f4d8415bbb0 legacy=0x7f4d840cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.720+0000 7f4d7b400640 1 -- 172.21.15.87:0/4203290779 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840b2880 con 0x7f4d840b4660 2024-04-25T10:24:43.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d7b400640 1 --2- 172.21.15.87:0/4203290779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8406c120 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009870 tx=0x7f4d6c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3a13150a8f9f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d83400640 1 -- 172.21.15.87:0/4203290779 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0b2070 con 0x7f4d840b4660 2024-04-25T10:24:43.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d83400640 1 -- 172.21.15.87:0/4203290779 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c0a9070 con 0x7f4d840b4660 2024-04-25T10:24:43.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d83400640 1 -- 172.21.15.87:0/4203290779 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0ad880 con 0x7f4d840b4660 2024-04-25T10:24:43.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 -- 172.21.15.87:0/4203290779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 msgr2=0x7f4d8406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 --2- 172.21.15.87:0/4203290779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8406c120 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009870 tx=0x7f4d6c01c560 comp rx=0 tx=0).stop 2024-04-25T10:24:43.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 -- 172.21.15.87:0/4203290779 shutdown_connections 2024-04-25T10:24:43.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 --2- 172.21.15.87:0/4203290779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 0x7f4d840cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 --2- 172.21.15.87:0/4203290779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8406c120 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 -- 172.21.15.87:0/4203290779 >> 172.21.15.87:0/4203290779 conn(0x7f4d84066b70 msgr2=0x7f4d84066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 -- 172.21.15.87:0/4203290779 shutdown_connections 2024-04-25T10:24:43.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.724+0000 7f4d8b000640 1 -- 172.21.15.87:0/4203290779 wait complete. 2024-04-25T10:24:43.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 Processor -- start 2024-04-25T10:24:43.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 -- start start 2024-04-25T10:24:43.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 0x7f4d84153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d8415bbb0 0x7f4d8414a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b3ae0 con 0x7f4d840b4660 2024-04-25T10:24:43.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8406c870 con 0x7f4d8415bbb0 2024-04-25T10:24:43.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84168960 con 0x7f4d840cf960 2024-04-25T10:24:43.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8415eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49562/0 (socket says 172.21.15.87:49562) 2024-04-25T10:24:43.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 -- 172.21.15.87:0/39626754 learned_addr learned my addr 172.21.15.87:0/39626754 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:43.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 -- 172.21.15.87:0/39626754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 msgr2=0x7f4d84153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 0x7f4d84153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 -- 172.21.15.87:0/39626754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d8415bbb0 msgr2=0x7f4d8414a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d8415bbb0 0x7f4d8414a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 -- 172.21.15.87:0/39626754 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d841544c0 con 0x7f4d840b4660 2024-04-25T10:24:43.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d83e00640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8415eaa0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c7c0 tx=0x7f4d7400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d80c00640 1 -- 172.21.15.87:0/39626754 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74017070 con 0x7f4d840b4660 2024-04-25T10:24:43.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d80c00640 1 -- 172.21.15.87:0/39626754 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7400a640 con 0x7f4d840b4660 2024-04-25T10:24:43.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d84153f00 con 0x7f4d840b4660 2024-04-25T10:24:43.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d80c00640 1 -- 172.21.15.87:0/39626754 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400a9f0 con 0x7f4d840b4660 2024-04-25T10:24:43.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.728+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8414c1a0 con 0x7f4d840b4660 2024-04-25T10:24:43.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.732+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8414b370 con 0x7f4d840b4660 2024-04-25T10:24:43.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.732+0000 7f4d80c00640 1 -- 172.21.15.87:0/39626754 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d7401a070 con 0x7f4d840b4660 2024-04-25T10:24:43.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.732+0000 7f4d80c00640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d60066f60 0x7f4d60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.732+0000 7f4d80c00640 1 -- 172.21.15.87:0/39626754 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4d74084290 con 0x7f4d840b4660 2024-04-25T10:24:43.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.732+0000 7f4d7b400640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d60066f60 0x7f4d60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.732+0000 7f4d7b400640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d60066f60 0x7f4d60069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0af000 tx=0x7f4d6c004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.736+0000 7f4d80c00640 1 -- 172.21.15.87:0/39626754 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d74054ec0 con 0x7f4d840b4660 2024-04-25T10:24:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:43.892+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4d8404ef30 con 0x7f4d840b4660 2024-04-25T10:24:44.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d80c00640 1 -- 172.21.15.87:0/39626754 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v161) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4d74058f60 con 0x7f4d840b4660 2024-04-25T10:24:44.550 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:24:44.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d60066f60 msgr2=0x7f4d60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d60066f60 0x7f4d60069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0af000 tx=0x7f4d6c004740 comp rx=0 tx=0).stop 2024-04-25T10:24:44.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 msgr2=0x7f4d8415eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8415eaa0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c7c0 tx=0x7f4d7400cc80 comp rx=0 tx=0).stop 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 shutdown_connections 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d60066f60 0x7f4d60069420 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d8415bbb0 0x7f4d8414a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d840cf960 0x7f4d84153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 --2- 172.21.15.87:0/39626754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d840b4660 0x7f4d8415eaa0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 >> 172.21.15.87:0/39626754 conn(0x7f4d84066b70 msgr2=0x7f4d8415fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 shutdown_connections 2024-04-25T10:24:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.552+0000 7f4d8b000640 1 -- 172.21.15.87:0/39626754 wait complete. 2024-04-25T10:24:44.590 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:24:44.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.728+0000 7fc35a000640 1 Processor -- start 2024-04-25T10:24:44.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc35a000640 1 -- start start 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc35a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc354159540 0x7fc3540ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc35a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3540aebd0 0x7fc354071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc35a000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc354071360 con 0x7fc354072020 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc35a000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc354159920 con 0x7fc354159540 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc35a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35415c2f0 con 0x7fc3540aebd0 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc353400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc354159540 0x7fc3540ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc353e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc354072020 0x7fc35415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44396/0 (socket says 172.21.15.87:44396) 2024-04-25T10:24:44.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc353e00640 1 -- 172.21.15.87:0/1676725023 learned_addr learned my addr 172.21.15.87:0/1676725023 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:44.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc34aa00640 1 --2- 172.21.15.87:0/1676725023 >> v2:172.21.15.169:3300/0 conn(0x7fc3540aebd0 0x7fc354071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc352a00640 1 -- 172.21.15.87:0/1676725023 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229766824 0 0) 0x7fc354071360 con 0x7fc354072020 2024-04-25T10:24:44.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc352a00640 1 -- 172.21.15.87:0/1676725023 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc338003530 con 0x7fc354072020 2024-04-25T10:24:44.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc353400640 1 -- 172.21.15.87:0/1676725023 >> v2:172.21.15.169:3300/0 conn(0x7fc3540aebd0 msgr2=0x7fc354071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc353400640 1 --2- 172.21.15.87:0/1676725023 >> v2:172.21.15.169:3300/0 conn(0x7fc3540aebd0 0x7fc354071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc353400640 1 -- 172.21.15.87:0/1676725023 >> v1:172.21.15.87:6789/0 conn(0x7fc354072020 legacy=0x7fc35415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc353400640 1 -- 172.21.15.87:0/1676725023 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35415c5d0 con 0x7fc354159540 2024-04-25T10:24:44.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.732+0000 7fc34aa00640 1 --2- 172.21.15.87:0/1676725023 >> v2:172.21.15.169:3300/0 conn(0x7fc3540aebd0 0x7fc354071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc353400640 1 --2- 172.21.15.87:0/1676725023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc354159540 0x7fc3540ae690 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc3540cd4e0 tx=0x7fc34401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6097d4649d947de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc352a00640 1 -- 172.21.15.87:0/1676725023 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3440b0070 con 0x7fc354159540 2024-04-25T10:24:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc352a00640 1 -- 172.21.15.87:0/1676725023 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3440ab600 con 0x7fc354159540 2024-04-25T10:24:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc352a00640 1 -- 172.21.15.87:0/1676725023 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3440ab9d0 con 0x7fc354159540 2024-04-25T10:24:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 -- 172.21.15.87:0/1676725023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc354159540 msgr2=0x7fc3540ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 --2- 172.21.15.87:0/1676725023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc354159540 0x7fc3540ae690 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc3540cd4e0 tx=0x7fc34401c480 comp rx=0 tx=0).stop 2024-04-25T10:24:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 -- 172.21.15.87:0/1676725023 shutdown_connections 2024-04-25T10:24:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 --2- 172.21.15.87:0/1676725023 >> v2:172.21.15.169:3300/0 conn(0x7fc3540aebd0 0x7fc354071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 --2- 172.21.15.87:0/1676725023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc354159540 0x7fc3540ae690 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 -- 172.21.15.87:0/1676725023 >> 172.21.15.87:0/1676725023 conn(0x7fc3540669a0 msgr2=0x7fc354066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 -- 172.21.15.87:0/1676725023 shutdown_connections 2024-04-25T10:24:44.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 -- 172.21.15.87:0/1676725023 wait complete. 2024-04-25T10:24:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 Processor -- start 2024-04-25T10:24:44.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.736+0000 7fc35a000640 1 -- start start 2024-04-25T10:24:44.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc354072020 0x7fc3541508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3540aebd0 0x7fc35414f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc354159540 0x7fc3541531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc354072430 con 0x7fc3540aebd0 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35406c0d0 con 0x7fc354159540 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc354168930 con 0x7fc354072020 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3540aebd0 0x7fc35414f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3540aebd0 0x7fc35414f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49590/0 (socket says 172.21.15.87:49590) 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 -- 172.21.15.87:0/4133328265 learned_addr learned my addr 172.21.15.87:0/4133328265 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc353e00640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc354159540 0x7fc3541531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc353400640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc354072020 0x7fc3541508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 -- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc354072020 msgr2=0x7fc3541508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc354072020 0x7fc3541508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 -- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc354159540 msgr2=0x7fc3541531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc354159540 0x7fc3541531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 -- 172.21.15.87:0/4133328265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354153980 con 0x7fc3540aebd0 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc353e00640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc354159540 0x7fc3541531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34aa00640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3540aebd0 0x7fc35414f580 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00e810 tx=0x7fc33c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34be00640 1 -- 172.21.15.87:0/4133328265 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c017070 con 0x7fc3540aebd0 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34be00640 1 -- 172.21.15.87:0/4133328265 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c00c8d0 con 0x7fc3540aebd0 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc34be00640 1 -- 172.21.15.87:0/4133328265 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00f040 con 0x7fc3540aebd0 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc354153c60 con 0x7fc3540aebd0 2024-04-25T10:24:44.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc354154220 con 0x7fc3540aebd0 2024-04-25T10:24:44.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.740+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35404ef30 con 0x7fc3540aebd0 2024-04-25T10:24:44.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.744+0000 7fc34be00640 1 -- 172.21.15.87:0/4133328265 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc33c01a070 con 0x7fc3540aebd0 2024-04-25T10:24:44.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.744+0000 7fc34be00640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc330066f90 0x7fc330069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.744+0000 7fc34be00640 1 -- 172.21.15.87:0/4133328265 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc33c084fb0 con 0x7fc3540aebd0 2024-04-25T10:24:44.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.744+0000 7fc353400640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc330066f90 0x7fc330069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.744+0000 7fc353400640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc330066f90 0x7fc330069450 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc3440ad000 tx=0x7fc344008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.748+0000 7fc34be00640 1 -- 172.21.15.87:0/4133328265 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc33c055a40 con 0x7fc3540aebd0 2024-04-25T10:24:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:44.932+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc35406d7f0 con 0x7fc3540aebd0 2024-04-25T10:24:45.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.560+0000 7fc34be00640 1 -- 172.21.15.87:0/4133328265 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v115) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc33c059ae0 con 0x7fc3540aebd0 2024-04-25T10:24:45.565 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 24 and data pool 25 2024-04-25T10:24:45.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc330066f90 msgr2=0x7fc330069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc330066f90 0x7fc330069450 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc3440ad000 tx=0x7fc344008040 comp rx=0 tx=0).stop 2024-04-25T10:24:45.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3540aebd0 msgr2=0x7fc35414f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3540aebd0 0x7fc35414f580 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00e810 tx=0x7fc33c00ecd0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 shutdown_connections 2024-04-25T10:24:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc330066f90 0x7fc330069450 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc354159540 0x7fc3541531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3540aebd0 0x7fc35414f580 secure :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00e810 tx=0x7fc33c00ecd0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 --2- 172.21.15.87:0/4133328265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc354072020 0x7fc3541508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 >> 172.21.15.87:0/4133328265 conn(0x7fc3540669a0 msgr2=0x7fc35415df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:45.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.568+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 shutdown_connections 2024-04-25T10:24:45.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.572+0000 7fc35a000640 1 -- 172.21.15.87:0/4133328265 wait complete. 2024-04-25T10:24:45.608 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:45.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.744+0000 7fe365c00640 1 Processor -- start 2024-04-25T10:24:45.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.744+0000 7fe365c00640 1 -- start start 2024-04-25T10:24:45.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe365c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe360071f70 0x7fe3600b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe365c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe3600b2fc0 0x7fe36006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe365c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe360070ed0 con 0x7fe36015a0e0 2024-04-25T10:24:45.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe365c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3600b3420 con 0x7fe360071f70 2024-04-25T10:24:45.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe365c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3600711b0 con 0x7fe3600b2fc0 2024-04-25T10:24:45.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe356000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe360071f70 0x7fe3600b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe356000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe360071f70 0x7fe3600b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49598/0 (socket says 172.21.15.87:49598) 2024-04-25T10:24:45.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe356000640 1 -- 172.21.15.87:0/2928745732 learned_addr learned my addr 172.21.15.87:0/2928745732 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:45.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe35f400640 1 --2- 172.21.15.87:0/2928745732 >> v2:172.21.15.169:3300/0 conn(0x7fe3600b2fc0 0x7fe36006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe356000640 1 -- 172.21.15.87:0/2928745732 >> v2:172.21.15.169:3300/0 conn(0x7fe3600b2fc0 msgr2=0x7fe36006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:45.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe356000640 1 --2- 172.21.15.87:0/2928745732 >> v2:172.21.15.169:3300/0 conn(0x7fe3600b2fc0 0x7fe36006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe356000640 1 -- 172.21.15.87:0/2928745732 >> v1:172.21.15.169:6789/0 conn(0x7fe36015a0e0 legacy=0x7fe36015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:45.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe356000640 1 -- 172.21.15.87:0/2928745732 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360071490 con 0x7fe360071f70 2024-04-25T10:24:45.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.748+0000 7fe35f400640 1 --2- 172.21.15.87:0/2928745732 >> v2:172.21.15.169:3300/0 conn(0x7fe3600b2fc0 0x7fe36006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:45.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe356000640 1 --2- 172.21.15.87:0/2928745732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3600b2a80 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d0670 tx=0x7fe34801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55be43fd25cb7cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:45.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe35e000640 1 -- 172.21.15.87:0/2928745732 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3480ab430 con 0x7fe360071f70 2024-04-25T10:24:45.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe35e000640 1 -- 172.21.15.87:0/2928745732 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3480b0070 con 0x7fe360071f70 2024-04-25T10:24:45.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe35e000640 1 -- 172.21.15.87:0/2928745732 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3480ab7f0 con 0x7fe360071f70 2024-04-25T10:24:45.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 -- 172.21.15.87:0/2928745732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 msgr2=0x7fe3600b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 --2- 172.21.15.87:0/2928745732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3600b2a80 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d0670 tx=0x7fe34801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 -- 172.21.15.87:0/2928745732 shutdown_connections 2024-04-25T10:24:45.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 --2- 172.21.15.87:0/2928745732 >> v2:172.21.15.169:3300/0 conn(0x7fe3600b2fc0 0x7fe36006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 --2- 172.21.15.87:0/2928745732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3600b2a80 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 -- 172.21.15.87:0/2928745732 >> 172.21.15.87:0/2928745732 conn(0x7fe360066b70 msgr2=0x7fe360066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:45.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 -- 172.21.15.87:0/2928745732 shutdown_connections 2024-04-25T10:24:45.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 -- 172.21.15.87:0/2928745732 wait complete. 2024-04-25T10:24:45.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 Processor -- start 2024-04-25T10:24:45.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.752+0000 7fe365c00640 1 -- start start 2024-04-25T10:24:45.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3601538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3600b2fc0 0x7fe36014a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36015a0e0 0x7fe3601557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360070b50 con 0x7fe360071f70 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36015a4c0 con 0x7fe36015a0e0 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600723f0 con 0x7fe3600b2fc0 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3601538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3601538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49610/0 (socket says 172.21.15.87:49610) 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 -- 172.21.15.87:0/1406144073 learned_addr learned my addr 172.21.15.87:0/1406144073 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe356000640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3600b2fc0 0x7fe36014a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35f400640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36015a0e0 0x7fe3601557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 -- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3600b2fc0 msgr2=0x7fe36014a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3600b2fc0 0x7fe36014a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 -- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36015a0e0 msgr2=0x7fe3601557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36015a0e0 0x7fe3601557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 -- 172.21.15.87:0/1406144073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360151f10 con 0x7fe360071f70 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe35ea00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3601538c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fe35000b410 tx=0x7fe35000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:45.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe357400640 1 -- 172.21.15.87:0/1406144073 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350015030 con 0x7fe360071f70 2024-04-25T10:24:45.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe357400640 1 -- 172.21.15.87:0/1406144073 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe350004020 con 0x7fe360071f70 2024-04-25T10:24:45.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe357400640 1 -- 172.21.15.87:0/1406144073 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350004450 con 0x7fe360071f70 2024-04-25T10:24:45.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3601521f0 con 0x7fe360071f70 2024-04-25T10:24:45.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36014b190 con 0x7fe360071f70 2024-04-25T10:24:45.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36015b9d0 con 0x7fe360071f70 2024-04-25T10:24:45.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.756+0000 7fe357400640 1 -- 172.21.15.87:0/1406144073 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe350018070 con 0x7fe360071f70 2024-04-25T10:24:45.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.760+0000 7fe357400640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe33c066f60 0x7fe33c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:45.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.760+0000 7fe357400640 1 -- 172.21.15.87:0/1406144073 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe350084b50 con 0x7fe360071f70 2024-04-25T10:24:45.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.760+0000 7fe356000640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe33c066f60 0x7fe33c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:45.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.760+0000 7fe356000640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe33c066f60 0x7fe33c069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe3480af000 tx=0x7fe348008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:45.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.764+0000 7fe357400640 1 -- 172.21.15.87:0/1406144073 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe350055bf0 con 0x7fe360071f70 2024-04-25T10:24:45.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.916+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe36004ef30 con 0x7fe360071f70 2024-04-25T10:24:45.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.916+0000 7fe357400640 1 -- 172.21.15.87:0/1406144073 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16680 (secure 0 0 0) 0x7fe3500554f0 con 0x7fe360071f70 2024-04-25T10:24:45.917 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:45.918 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":162,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:24:44.936058+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:42.650296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:43.940319+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:45.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe33c066f60 msgr2=0x7fe33c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe33c066f60 0x7fe33c069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe3480af000 tx=0x7fe348008040 comp rx=0 tx=0).stop 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 msgr2=0x7fe3601538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3601538c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fe35000b410 tx=0x7fe35000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 shutdown_connections 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe33c066f60 0x7fe33c069420 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36015a0e0 0x7fe3601557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3600b2fc0 0x7fe36014a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 --2- 172.21.15.87:0/1406144073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe360071f70 0x7fe3601538c0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 >> 172.21.15.87:0/1406144073 conn(0x7fe360066b70 msgr2=0x7fe360154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 shutdown_connections 2024-04-25T10:24:45.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:45.920+0000 7fe365c00640 1 -- 172.21.15.87:0/1406144073 wait complete. 2024-04-25T10:24:45.938 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:46.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.072+0000 7fccf2800640 1 Processor -- start 2024-04-25T10:24:46.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.072+0000 7fccf2800640 1 -- start start 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fccec0b27d0 0x7fccec0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec0cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec0b3fb0 con 0x7fccec15dbe0 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf2800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0b3170 con 0x7fccec0b27d0 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec06c690 con 0x7fccec0cf4e0 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fccec0b27d0 0x7fccec0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fccec0b27d0 0x7fccec0b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49612/0 (socket says 172.21.15.87:49612) 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf0a00640 1 -- 172.21.15.87:0/1702818555 learned_addr learned my addr 172.21.15.87:0/1702818555 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf1e00640 1 --2- 172.21.15.87:0/1702818555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec0cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf0a00640 1 -- 172.21.15.87:0/1702818555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 msgr2=0x7fccec0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:46.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf0a00640 1 --2- 172.21.15.87:0/1702818555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec0cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf0a00640 1 -- 172.21.15.87:0/1702818555 >> v1:172.21.15.169:6789/0 conn(0x7fccec15dbe0 legacy=0x7fccec0ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.076+0000 7fccf0a00640 1 -- 172.21.15.87:0/1702818555 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0b4250 con 0x7fccec0b27d0 2024-04-25T10:24:46.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf0a00640 1 --2- 172.21.15.87:0/1702818555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec0b2c30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009870 tx=0x7fccd801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e757528c8ea3686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:46.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccebe00640 1 -- 172.21.15.87:0/1702818555 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b0070 con 0x7fccec0b27d0 2024-04-25T10:24:46.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccebe00640 1 -- 172.21.15.87:0/1702818555 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd80ab600 con 0x7fccec0b27d0 2024-04-25T10:24:46.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccebe00640 1 -- 172.21.15.87:0/1702818555 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80ab9d0 con 0x7fccec0b27d0 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 -- 172.21.15.87:0/1702818555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 msgr2=0x7fccec0b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 --2- 172.21.15.87:0/1702818555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec0b2c30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009870 tx=0x7fccd801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 -- 172.21.15.87:0/1702818555 shutdown_connections 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 --2- 172.21.15.87:0/1702818555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 --2- 172.21.15.87:0/1702818555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec0b2c30 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 -- 172.21.15.87:0/1702818555 >> 172.21.15.87:0/1702818555 conn(0x7fccec0743f0 msgr2=0x7fccec0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 -- 172.21.15.87:0/1702818555 shutdown_connections 2024-04-25T10:24:46.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 -- 172.21.15.87:0/1702818555 wait complete. 2024-04-25T10:24:46.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 Processor -- start 2024-04-25T10:24:46.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.080+0000 7fccf2800640 1 -- start start 2024-04-25T10:24:46.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec13bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccec15dbe0 0x7fccec13a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec16abc0 con 0x7fccec0b27d0 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec16aa40 con 0x7fccec15dbe0 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec16ad40 con 0x7fccec0cf4e0 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec13bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec13bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49616/0 (socket says 172.21.15.87:49616) 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1e00640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccec15dbe0 0x7fccec13a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf0a00640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 -- 172.21.15.87:0/3390441528 learned_addr learned my addr 172.21.15.87:0/3390441528 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 -- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 msgr2=0x7fccec151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 -- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccec15dbe0 msgr2=0x7fccec13a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccec15dbe0 0x7fccec13a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 -- 172.21.15.87:0/3390441528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec13c820 con 0x7fccec0b27d0 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf1400640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec13bfd0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c3a0 tx=0x7fcce000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fcce9600640 1 -- 172.21.15.87:0/3390441528 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00052d0 con 0x7fccec0b27d0 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fcce9600640 1 -- 172.21.15.87:0/3390441528 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce0005470 con 0x7fccec0b27d0 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec13cb00 con 0x7fccec0b27d0 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fcce9600640 1 -- 172.21.15.87:0/3390441528 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0007260 con 0x7fccec0b27d0 2024-04-25T10:24:46.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec13d0c0 con 0x7fccec0b27d0 2024-04-25T10:24:46.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec075120 con 0x7fccec0b27d0 2024-04-25T10:24:46.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.084+0000 7fcce9600640 1 -- 172.21.15.87:0/3390441528 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcce0007400 con 0x7fccec0b27d0 2024-04-25T10:24:46.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.088+0000 7fcce9600640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcccc066f60 0x7fcccc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.088+0000 7fcce9600640 1 -- 172.21.15.87:0/3390441528 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcce0084410 con 0x7fccec0b27d0 2024-04-25T10:24:46.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.088+0000 7fccf0a00640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcccc066f60 0x7fcccc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.088+0000 7fccf0a00640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcccc066f60 0x7fcccc069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ad000 tx=0x7fccd80a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.092+0000 7fcce9600640 1 -- 172.21.15.87:0/3390441528 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcce0054dd0 con 0x7fccec0b27d0 2024-04-25T10:24:46.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.244+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccec160f30 con 0x7fccec0b27d0 2024-04-25T10:24:46.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.244+0000 7fcce9600640 1 -- 172.21.15.87:0/3390441528 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16680 (secure 0 0 0) 0x7fcce0058e70 con 0x7fccec0b27d0 2024-04-25T10:24:46.245 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:46.246 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":162,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:24:44.936058+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:42.650296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:43.940319+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:46.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcccc066f60 msgr2=0x7fcccc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcccc066f60 0x7fcccc069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ad000 tx=0x7fccd80a6000 comp rx=0 tx=0).stop 2024-04-25T10:24:46.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 msgr2=0x7fccec13bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec13bfd0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c3a0 tx=0x7fcce000c860 comp rx=0 tx=0).stop 2024-04-25T10:24:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 shutdown_connections 2024-04-25T10:24:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcccc066f60 0x7fcccc069420 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccec15dbe0 0x7fccec13a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccec0cf4e0 0x7fccec151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 --2- 172.21.15.87:0/3390441528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccec0b27d0 0x7fccec13bfd0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 >> 172.21.15.87:0/3390441528 conn(0x7fccec0743f0 msgr2=0x7fccec0b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:46.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 shutdown_connections 2024-04-25T10:24:46.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.248+0000 7fccf2800640 1 -- 172.21.15.87:0/3390441528 wait complete. 2024-04-25T10:24:46.267 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:24:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.404+0000 7fb599000640 1 Processor -- start 2024-04-25T10:24:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.404+0000 7fb599000640 1 -- start start 2024-04-25T10:24:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.404+0000 7fb599000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5940aef80 0x7fb59415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb599000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940b31c0 0x7fb5940b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb599000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59415b080 con 0x7fb594070350 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb599000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59415adb0 con 0x7fb5940aef80 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb599000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59415eb60 con 0x7fb5940b31c0 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5940aef80 0x7fb59415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb589400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb594070350 0x7fb59415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:44406/0 (socket says 172.21.15.87:44406) 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5940aef80 0x7fb59415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49622/0 (socket says 172.21.15.87:49622) 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb589400640 1 -- 172.21.15.87:0/506656753 learned_addr learned my addr 172.21.15.87:0/506656753 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb592800640 1 --2- 172.21.15.87:0/506656753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940b31c0 0x7fb5940b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591e00640 1 -- 172.21.15.87:0/506656753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940b31c0 msgr2=0x7fb5940b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591e00640 1 --2- 172.21.15.87:0/506656753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940b31c0 0x7fb5940b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591e00640 1 -- 172.21.15.87:0/506656753 >> v1:172.21.15.87:6789/0 conn(0x7fb594070350 legacy=0x7fb59415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591400640 1 -- 172.21.15.87:0/506656753 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470556157 0 0) 0x7fb59415b080 con 0x7fb594070350 2024-04-25T10:24:46.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591e00640 1 -- 172.21.15.87:0/506656753 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59415ee40 con 0x7fb5940aef80 2024-04-25T10:24:46.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591e00640 1 --2- 172.21.15.87:0/506656753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940aef80 0x7fb59415e480 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fb5840097c0 tx=0x7fb58401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df961d7a72c7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:46.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591400640 1 -- 172.21.15.87:0/506656753 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840b1070 con 0x7fb5940aef80 2024-04-25T10:24:46.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591400640 1 -- 172.21.15.87:0/506656753 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5840ac410 con 0x7fb5940aef80 2024-04-25T10:24:46.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.408+0000 7fb591400640 1 -- 172.21.15.87:0/506656753 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840ac7e0 con 0x7fb5940aef80 2024-04-25T10:24:46.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- 172.21.15.87:0/506656753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940aef80 msgr2=0x7fb59415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 --2- 172.21.15.87:0/506656753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940aef80 0x7fb59415e480 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fb5840097c0 tx=0x7fb58401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- 172.21.15.87:0/506656753 shutdown_connections 2024-04-25T10:24:46.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 --2- 172.21.15.87:0/506656753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940b31c0 0x7fb5940b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 --2- 172.21.15.87:0/506656753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940aef80 0x7fb59415e480 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- 172.21.15.87:0/506656753 >> 172.21.15.87:0/506656753 conn(0x7fb5940669a0 msgr2=0x7fb594066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:46.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- 172.21.15.87:0/506656753 shutdown_connections 2024-04-25T10:24:46.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- 172.21.15.87:0/506656753 wait complete. 2024-04-25T10:24:46.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 Processor -- start 2024-04-25T10:24:46.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- start start 2024-04-25T10:24:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070350 0x7fb594153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940aef80 0x7fb594157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940b31c0 0x7fb594141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941679a0 con 0x7fb5940b31c0 2024-04-25T10:24:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594167820 con 0x7fb5940aef80 2024-04-25T10:24:46.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb599000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb594167b20 con 0x7fb594070350 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb592800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940b31c0 0x7fb594141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb592800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940b31c0 0x7fb594141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49632/0 (socket says 172.21.15.87:49632) 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb592800640 1 -- 172.21.15.87:0/3348404684 learned_addr learned my addr 172.21.15.87:0/3348404684 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb589400640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940aef80 0x7fb594157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.412+0000 7fb591e00640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070350 0x7fb594153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb592800640 1 -- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070350 msgr2=0x7fb594153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb592800640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070350 0x7fb594153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb592800640 1 -- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940aef80 msgr2=0x7fb594157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb592800640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940aef80 0x7fb594157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb592800640 1 -- 172.21.15.87:0/3348404684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594151fe0 con 0x7fb5940b31c0 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb592800640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940b31c0 0x7fb594141d20 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb58c007e60 tx=0x7fb58c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb58aa00640 1 -- 172.21.15.87:0/3348404684 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c014420 con 0x7fb5940b31c0 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb58aa00640 1 -- 172.21.15.87:0/3348404684 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c019070 con 0x7fb5940b31c0 2024-04-25T10:24:46.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb58aa00640 1 -- 172.21.15.87:0/3348404684 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c014820 con 0x7fb5940b31c0 2024-04-25T10:24:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5941522c0 con 0x7fb5940b31c0 2024-04-25T10:24:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb594152880 con 0x7fb5940b31c0 2024-04-25T10:24:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb594152f80 con 0x7fb5940b31c0 2024-04-25T10:24:46.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb58aa00640 1 -- 172.21.15.87:0/3348404684 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb58c0149c0 con 0x7fb5940b31c0 2024-04-25T10:24:46.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb58aa00640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb570066f60 0x7fb570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:46.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb58aa00640 1 -- 172.21.15.87:0/3348404684 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb58c0857e0 con 0x7fb5940b31c0 2024-04-25T10:24:46.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.416+0000 7fb591e00640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb570066f60 0x7fb570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:46.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.420+0000 7fb591e00640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb570066f60 0x7fb570069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb5840ae000 tx=0x7fb584008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:46.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.420+0000 7fb58aa00640 1 -- 172.21.15.87:0/3348404684 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb58c056100 con 0x7fb5940b31c0 2024-04-25T10:24:46.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:46.612+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5940cc110 con 0x7fb5940b31c0 2024-04-25T10:24:47.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.588+0000 7fb58aa00640 1 -- 172.21.15.87:0/3348404684 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v118) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb58c05a1a0 con 0x7fb5940b31c0 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb570066f60 msgr2=0x7fb570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb570066f60 0x7fb570069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb5840ae000 tx=0x7fb584008040 comp rx=0 tx=0).stop 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940b31c0 msgr2=0x7fb594141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940b31c0 0x7fb594141d20 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb58c007e60 tx=0x7fb58c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 shutdown_connections 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb570066f60 0x7fb570069420 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5940b31c0 0x7fb594141d20 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940aef80 0x7fb594157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 --2- 172.21.15.87:0/3348404684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070350 0x7fb594153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 >> 172.21.15.87:0/3348404684 conn(0x7fb5940669a0 msgr2=0x7fb59415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 shutdown_connections 2024-04-25T10:24:47.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.592+0000 7fb599000640 1 -- 172.21.15.87:0/3348404684 wait complete. 2024-04-25T10:24:47.614 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:24:47.615 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:24:47.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.756+0000 7f728ae00640 1 Processor -- start 2024-04-25T10:24:47.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.756+0000 7f728ae00640 1 -- start start 2024-04-25T10:24:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728415dc30 0x7f728406f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728ae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7284071330 0x7f728406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728ae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72840b4250 con 0x7f72840b3d50 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728ae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72840b4530 con 0x7f7284071330 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72840ad970 con 0x7f728415dc30 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f7289000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f72840b3d50 0x7f7284070c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60166/0 (socket says 172.21.15.87:60166) 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f7289000640 1 -- 172.21.15.87:0/202334870 learned_addr learned my addr 172.21.15.87:0/202334870 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728a400640 1 --2- 172.21.15.87:0/202334870 >> v2:172.21.15.87:3300/0 conn(0x7f7284071330 0x7f728406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f7289a00640 1 --2- 172.21.15.87:0/202334870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728415dc30 0x7f728406f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f7283e00640 1 -- 172.21.15.87:0/202334870 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047175186 0 0) 0x7f72840b4250 con 0x7f72840b3d50 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f7283e00640 1 -- 172.21.15.87:0/202334870 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f726c003530 con 0x7f72840b3d50 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728a400640 1 -- 172.21.15.87:0/202334870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728415dc30 msgr2=0x7f728406f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728a400640 1 --2- 172.21.15.87:0/202334870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728415dc30 0x7f728406f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728a400640 1 -- 172.21.15.87:0/202334870 >> v1:172.21.15.87:6789/0 conn(0x7f72840b3d50 legacy=0x7f7284070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728a400640 1 -- 172.21.15.87:0/202334870 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840adc50 con 0x7f7284071330 2024-04-25T10:24:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.760+0000 7f728a400640 1 --2- 172.21.15.87:0/202334870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7284071330 0x7f728406cbf0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f727c009d90 tx=0x7f727c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9975d015258b8fac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f7283e00640 1 -- 172.21.15.87:0/202334870 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0b2070 con 0x7f7284071330 2024-04-25T10:24:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f7283e00640 1 -- 172.21.15.87:0/202334870 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c0a5710 con 0x7f7284071330 2024-04-25T10:24:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f7283e00640 1 -- 172.21.15.87:0/202334870 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0a5ae0 con 0x7f7284071330 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- 172.21.15.87:0/202334870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7284071330 msgr2=0x7f728406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 --2- 172.21.15.87:0/202334870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7284071330 0x7f728406cbf0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f727c009d90 tx=0x7f727c01c850 comp rx=0 tx=0).stop 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- 172.21.15.87:0/202334870 shutdown_connections 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 --2- 172.21.15.87:0/202334870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7284071330 0x7f728406cbf0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 --2- 172.21.15.87:0/202334870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728415dc30 0x7f728406f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- 172.21.15.87:0/202334870 >> 172.21.15.87:0/202334870 conn(0x7f728406b7d0 msgr2=0x7f72840b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- 172.21.15.87:0/202334870 shutdown_connections 2024-04-25T10:24:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- 172.21.15.87:0/202334870 wait complete. 2024-04-25T10:24:47.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 Processor -- start 2024-04-25T10:24:47.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- start start 2024-04-25T10:24:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7284071330 0x7f7284149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72840b3d50 0x7f7284147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728415dc30 0x7f7284147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72840af030 con 0x7f72840b3d50 2024-04-25T10:24:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72840aeeb0 con 0x7f728415dc30 2024-04-25T10:24:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72840af1b0 con 0x7f7284071330 2024-04-25T10:24:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f7289000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72840b3d50 0x7f7284147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:47.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f7289000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72840b3d50 0x7f7284147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43782/0 (socket says 172.21.15.87:43782) 2024-04-25T10:24:47.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f7289000640 1 -- 172.21.15.87:0/4041639054 learned_addr learned my addr 172.21.15.87:0/4041639054 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:47.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f7289a00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7284071330 0x7f7284149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:47.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.764+0000 7f728a400640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728415dc30 0x7f7284147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:47.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7289000640 1 -- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7284071330 msgr2=0x7f7284149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:47.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7289000640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7284071330 0x7f7284149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7289000640 1 -- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728415dc30 msgr2=0x7f7284147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:47.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7289000640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728415dc30 0x7f7284147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:47.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7289000640 1 -- 172.21.15.87:0/4041639054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7284154d90 con 0x7f72840b3d50 2024-04-25T10:24:47.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7289000640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72840b3d50 0x7f7284147840 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f727000c3f0 tx=0x7f727000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:47.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7281600640 1 -- 172.21.15.87:0/4041639054 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270017070 con 0x7f72840b3d50 2024-04-25T10:24:47.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7281600640 1 -- 172.21.15.87:0/4041639054 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7270005cf0 con 0x7f72840b3d50 2024-04-25T10:24:47.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72841587d0 con 0x7f72840b3d50 2024-04-25T10:24:47.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7281600640 1 -- 172.21.15.87:0/4041639054 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72700072f0 con 0x7f72840b3d50 2024-04-25T10:24:47.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7284149840 con 0x7f72840b3d50 2024-04-25T10:24:47.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7284075120 con 0x7f72840b3d50 2024-04-25T10:24:47.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7281600640 1 -- 172.21.15.87:0/4041639054 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f727001a070 con 0x7f72840b3d50 2024-04-25T10:24:47.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7281600640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7264066f60 0x7f7264069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:47.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7281600640 1 -- 172.21.15.87:0/4041639054 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72700843a0 con 0x7f72840b3d50 2024-04-25T10:24:47.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.768+0000 7f7289a00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7264066f60 0x7f7264069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:47.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.772+0000 7f7289a00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7264066f60 0x7f7264069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f728415e390 tx=0x7f7278008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:47.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.772+0000 7f7281600640 1 -- 172.21.15.87:0/4041639054 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7270054d50 con 0x7f72840b3d50 2024-04-25T10:24:47.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:47.960+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72840008e0 con 0x7f72840b3d50 2024-04-25T10:24:48.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.588+0000 7f7281600640 1 -- 172.21.15.87:0/4041639054 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v119) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72840008e0 con 0x7f72840b3d50 2024-04-25T10:24:48.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7264066f60 msgr2=0x7f7264069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7264066f60 0x7f7264069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f728415e390 tx=0x7f7278008040 comp rx=0 tx=0).stop 2024-04-25T10:24:48.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72840b3d50 msgr2=0x7f7284147840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72840b3d50 0x7f7284147840 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f727000c3f0 tx=0x7f727000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 shutdown_connections 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7264066f60 0x7f7264069420 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728415dc30 0x7f7284147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72840b3d50 0x7f7284147840 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 --2- 172.21.15.87:0/4041639054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7284071330 0x7f7284149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 >> 172.21.15.87:0/4041639054 conn(0x7f728406b7d0 msgr2=0x7f72840b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 shutdown_connections 2024-04-25T10:24:48.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.592+0000 7f728ae00640 1 -- 172.21.15.87:0/4041639054 wait complete. 2024-04-25T10:24:48.614 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:24:48.614 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:48.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.752+0000 7f9c41400640 1 Processor -- start 2024-04-25T10:24:48.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.752+0000 7f9c41400640 1 -- start start 2024-04-25T10:24:48.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c41400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c3c0d0040 0x7f9c3c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:48.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c41400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c15b740 0x7f9c3c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:48.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c41400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c071050 con 0x7f9c3c1550b0 2024-04-25T10:24:48.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c41400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c15d040 con 0x7f9c3c0d0040 2024-04-25T10:24:48.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c41400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0712f0 con 0x7f9c3c15b740 2024-04-25T10:24:48.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c15b740 0x7f9c3c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:48.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c15b740 0x7f9c3c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:54314/0 (socket says 172.21.15.87:54314) 2024-04-25T10:24:48.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3b200640 1 -- 172.21.15.87:0/1660039857 learned_addr learned my addr 172.21.15.87:0/1660039857 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:48.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3a800640 1 --1- 172.21.15.87:0/1660039857 >> v1:172.21.15.169:6789/0 conn(0x7f9c3c1550b0 0x7f9c3c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:55074/0 (socket says 172.21.15.87:55074) 2024-04-25T10:24:48.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c39e00640 1 --2- 172.21.15.87:0/1660039857 >> v2:172.21.15.169:3300/0 conn(0x7f9c3c0d0040 0x7f9c3c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:48.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3b200640 1 -- 172.21.15.87:0/1660039857 >> v2:172.21.15.169:3300/0 conn(0x7f9c3c0d0040 msgr2=0x7f9c3c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:48.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3b200640 1 --2- 172.21.15.87:0/1660039857 >> v2:172.21.15.169:3300/0 conn(0x7f9c3c0d0040 0x7f9c3c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3b200640 1 -- 172.21.15.87:0/1660039857 >> v1:172.21.15.169:6789/0 conn(0x7f9c3c1550b0 legacy=0x7f9c3c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.756+0000 7f9c3b200640 1 -- 172.21.15.87:0/1660039857 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0715d0 con 0x7f9c3c15b740 2024-04-25T10:24:48.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c3b200640 1 --2- 172.21.15.87:0/1660039857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c15b740 0x7f9c3c0b3260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009e50 tx=0x7f9c3401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f62e0e9cbd2215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:48.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c39400640 1 -- 172.21.15.87:0/1660039857 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340a58d0 con 0x7f9c3c15b740 2024-04-25T10:24:48.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c39400640 1 -- 172.21.15.87:0/1660039857 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c340a5a70 con 0x7f9c3c15b740 2024-04-25T10:24:48.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c39400640 1 -- 172.21.15.87:0/1660039857 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340a5e40 con 0x7f9c3c15b740 2024-04-25T10:24:48.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 -- 172.21.15.87:0/1660039857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c15b740 msgr2=0x7f9c3c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 --2- 172.21.15.87:0/1660039857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c15b740 0x7f9c3c0b3260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009e50 tx=0x7f9c3401ca00 comp rx=0 tx=0).stop 2024-04-25T10:24:48.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 -- 172.21.15.87:0/1660039857 shutdown_connections 2024-04-25T10:24:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 --2- 172.21.15.87:0/1660039857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c15b740 0x7f9c3c0b3260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 --2- 172.21.15.87:0/1660039857 >> v2:172.21.15.169:3300/0 conn(0x7f9c3c0d0040 0x7f9c3c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 -- 172.21.15.87:0/1660039857 >> 172.21.15.87:0/1660039857 conn(0x7f9c3c066b70 msgr2=0x7f9c3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 -- 172.21.15.87:0/1660039857 shutdown_connections 2024-04-25T10:24:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 -- 172.21.15.87:0/1660039857 wait complete. 2024-04-25T10:24:48.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 Processor -- start 2024-04-25T10:24:48.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.760+0000 7f9c41400640 1 -- start start 2024-04-25T10:24:48.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0d0040 0x7f9c3c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:48.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c1550b0 0x7f9c3c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c3c15b740 0x7f9c3c143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c15bd50 con 0x7f9c3c15b740 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c15bbd0 con 0x7f9c3c1550b0 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c15c030 con 0x7f9c3c0d0040 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c3c15b740 0x7f9c3c143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0d0040 0x7f9c3c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c3c15b740 0x7f9c3c143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43796/0 (socket says 172.21.15.87:43796) 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 -- 172.21.15.87:0/1464601803 learned_addr learned my addr 172.21.15.87:0/1464601803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c39e00640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c1550b0 0x7f9c3c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 -- 172.21.15.87:0/1464601803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0d0040 msgr2=0x7f9c3c1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0d0040 0x7f9c3c1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 -- 172.21.15.87:0/1464601803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c1550b0 msgr2=0x7f9c3c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c1550b0 0x7f9c3c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 -- 172.21.15.87:0/1464601803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0ce9e0 con 0x7f9c3c15b740 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c3b200640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c3c15b740 0x7f9c3c143080 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f9c3401a630 tx=0x7f9c340197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c32a00640 1 -- 172.21.15.87:0/1464601803 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340bc070 con 0x7f9c3c15b740 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c32a00640 1 -- 172.21.15.87:0/1464601803 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c340ade10 con 0x7f9c3c15b740 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c158aa0 con 0x7f9c3c15b740 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c32a00640 1 -- 172.21.15.87:0/1464601803 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340a58d0 con 0x7f9c3c15b740 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c1579f0 con 0x7f9c3c15b740 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c158c70 con 0x7f9c3c15b740 2024-04-25T10:24:48.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.764+0000 7f9c32a00640 1 -- 172.21.15.87:0/1464601803 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c340bf070 con 0x7f9c3c15b740 2024-04-25T10:24:48.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.768+0000 7f9c32a00640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9c1c066f60 0x7f9c1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:48.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.768+0000 7f9c32a00640 1 -- 172.21.15.87:0/1464601803 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c34120210 con 0x7f9c3c15b740 2024-04-25T10:24:48.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.768+0000 7f9c3a800640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9c1c066f60 0x7f9c1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:48.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.768+0000 7f9c3a800640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9c1c066f60 0x7f9c1c069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c1f9d70 tx=0x7f9c2c007e90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:48.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.772+0000 7f9c32a00640 1 -- 172.21.15.87:0/1464601803 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c3c0ce9e0 con 0x7f9c3c15b740 2024-04-25T10:24:48.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.956+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c3c04ef30 con 0x7f9c3c15b740 2024-04-25T10:24:48.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.956+0000 7f9c32a00640 1 -- 172.21.15.87:0/1464601803 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f9c34016620 con 0x7f9c3c15b740 2024-04-25T10:24:48.956 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:48.956 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:24:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9c1c066f60 msgr2=0x7f9c1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9c1c066f60 0x7f9c1c069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c1f9d70 tx=0x7f9c2c007e90 comp rx=0 tx=0).stop 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c3c15b740 msgr2=0x7f9c3c143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c3c15b740 0x7f9c3c143080 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f9c3401a630 tx=0x7f9c340197a0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 shutdown_connections 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9c1c066f60 0x7f9c1c069420 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c3c15b740 0x7f9c3c143080 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c1550b0 0x7f9c3c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 --2- 172.21.15.87:0/1464601803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0d0040 0x7f9c3c1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:48.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 >> 172.21.15.87:0/1464601803 conn(0x7f9c3c066b70 msgr2=0x7f9c3c0b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:48.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 shutdown_connections 2024-04-25T10:24:48.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:48.960+0000 7f9c41400640 1 -- 172.21.15.87:0/1464601803 wait complete. 2024-04-25T10:24:48.978 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:49.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.112+0000 7f8647000640 1 Processor -- start 2024-04-25T10:24:49.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.112+0000 7f8647000640 1 -- start start 2024-04-25T10:24:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f8647000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86401550b0 0x7f86400cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f8647000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f864015b910 0x7f86400b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f8647000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8640071050 con 0x7f86400cfe70 2024-04-25T10:24:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f8647000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864015bda0 con 0x7f864015b910 2024-04-25T10:24:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f8647000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864015d040 con 0x7f86401550b0 2024-04-25T10:24:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86401550b0 0x7f86400cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86401550b0 0x7f86400cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:54330/0 (socket says 172.21.15.87:54330) 2024-04-25T10:24:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863fe00640 1 -- 172.21.15.87:0/983437228 learned_addr learned my addr 172.21.15.87:0/983437228 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f8645000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f864015b910 0x7f86400b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863f400640 1 --1- 172.21.15.87:0/983437228 >> v1:172.21.15.169:6789/0 conn(0x7f86400cfe70 0x7f864015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:55084/0 (socket says 172.21.15.87:55084) 2024-04-25T10:24:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863fe00640 1 -- 172.21.15.87:0/983437228 >> v2:172.21.15.169:3300/0 conn(0x7f864015b910 msgr2=0x7f86400b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863fe00640 1 --2- 172.21.15.87:0/983437228 >> v2:172.21.15.169:3300/0 conn(0x7f864015b910 0x7f86400b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863fe00640 1 -- 172.21.15.87:0/983437228 >> v1:172.21.15.169:6789/0 conn(0x7f86400cfe70 legacy=0x7f864015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.116+0000 7f863fe00640 1 -- 172.21.15.87:0/983437228 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400712f0 con 0x7f86401550b0 2024-04-25T10:24:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f863fe00640 1 --2- 172.21.15.87:0/983437228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86401550b0 0x7f86400cf930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8630009870 tx=0x7f863001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c55dcba98c57f66f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f863ea00640 1 -- 172.21.15.87:0/983437228 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300ab650 con 0x7f86401550b0 2024-04-25T10:24:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f863ea00640 1 -- 172.21.15.87:0/983437228 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86300b0070 con 0x7f86401550b0 2024-04-25T10:24:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f863ea00640 1 -- 172.21.15.87:0/983437228 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300aba10 con 0x7f86401550b0 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- 172.21.15.87:0/983437228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86401550b0 msgr2=0x7f86400cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 --2- 172.21.15.87:0/983437228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86401550b0 0x7f86400cf930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8630009870 tx=0x7f863001c560 comp rx=0 tx=0).stop 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- 172.21.15.87:0/983437228 shutdown_connections 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 --2- 172.21.15.87:0/983437228 >> v2:172.21.15.169:3300/0 conn(0x7f864015b910 0x7f86400b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 --2- 172.21.15.87:0/983437228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86401550b0 0x7f86400cf930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- 172.21.15.87:0/983437228 >> 172.21.15.87:0/983437228 conn(0x7f8640066b70 msgr2=0x7f8640066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- 172.21.15.87:0/983437228 shutdown_connections 2024-04-25T10:24:49.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- 172.21.15.87:0/983437228 wait complete. 2024-04-25T10:24:49.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 Processor -- start 2024-04-25T10:24:49.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- start start 2024-04-25T10:24:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86400cfe70 0x7f86401fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86401550b0 0x7f864014ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f864015b910 0x7f8640153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864015c030 con 0x7f864015b910 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400720b0 con 0x7f86401550b0 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.120+0000 7f8647000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864015c1b0 con 0x7f86400cfe70 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f863fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86400cfe70 0x7f86401fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f863fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86400cfe70 0x7f86401fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:54332/0 (socket says 172.21.15.87:54332) 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f863fe00640 1 -- 172.21.15.87:0/4220830004 learned_addr learned my addr 172.21.15.87:0/4220830004 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8645000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f864015b910 0x7f8640153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f863f400640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86401550b0 0x7f864014ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8645000640 1 -- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86400cfe70 msgr2=0x7f86401fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8645000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86400cfe70 0x7f86401fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8645000640 1 -- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86401550b0 msgr2=0x7f864014ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8645000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86401550b0 0x7f864014ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8645000640 1 -- 172.21.15.87:0/4220830004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86401fabc0 con 0x7f864015b910 2024-04-25T10:24:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f863fe00640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86400cfe70 0x7f86401fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8645000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f864015b910 0x7f8640153d40 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f863800eba0 tx=0x7f863800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8637e00640 1 -- 172.21.15.87:0/4220830004 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638018020 con 0x7f864015b910 2024-04-25T10:24:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8637e00640 1 -- 172.21.15.87:0/4220830004 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863800cd20 con 0x7f864015b910 2024-04-25T10:24:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8637e00640 1 -- 172.21.15.87:0/4220830004 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380146c0 con 0x7f864015b910 2024-04-25T10:24:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86400ccc40 con 0x7f864015b910 2024-04-25T10:24:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86400cd200 con 0x7f864015b910 2024-04-25T10:24:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864004ef30 con 0x7f864015b910 2024-04-25T10:24:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.124+0000 7f8637e00640 1 -- 172.21.15.87:0/4220830004 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f863801b020 con 0x7f864015b910 2024-04-25T10:24:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.128+0000 7f8637e00640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8620066f60 0x7f8620069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.128+0000 7f8637e00640 1 -- 172.21.15.87:0/4220830004 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8638085bb0 con 0x7f864015b910 2024-04-25T10:24:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.128+0000 7f863fe00640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8620066f60 0x7f8620069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.128+0000 7f863fe00640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8620066f60 0x7f8620069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f863001c5a0 tx=0x7f86300a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.132+0000 7f8637e00640 1 -- 172.21.15.87:0/4220830004 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8638022030 con 0x7f864015b910 2024-04-25T10:24:49.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.284+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86400b10b0 con 0x7f864015b910 2024-04-25T10:24:49.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8637e00640 1 -- 172.21.15.87:0/4220830004 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f8638010040 con 0x7f864015b910 2024-04-25T10:24:49.290 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:49.290 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":163,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:24:46.563973+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:42.650296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:43.940319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8620066f60 msgr2=0x7f8620069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8620066f60 0x7f8620069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f863001c5a0 tx=0x7f86300a7000 comp rx=0 tx=0).stop 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f864015b910 msgr2=0x7f8640153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f864015b910 0x7f8640153d40 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f863800eba0 tx=0x7f863800c490 comp rx=0 tx=0).stop 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 shutdown_connections 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8620066f60 0x7f8620069420 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f864015b910 0x7f8640153d40 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86401550b0 0x7f864014ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 --2- 172.21.15.87:0/4220830004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86400cfe70 0x7f86401fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 >> 172.21.15.87:0/4220830004 conn(0x7f8640066b70 msgr2=0x7f8640066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 shutdown_connections 2024-04-25T10:24:49.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.288+0000 7f8647000640 1 -- 172.21.15.87:0/4220830004 wait complete. 2024-04-25T10:24:49.308 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:24:49.308 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:24:49.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f4266000640 1 Processor -- start 2024-04-25T10:24:49.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f4266000640 1 -- start start 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f4266000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f426015ddc0 0x7f426006b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f4266000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f426015e1a0 0x7f426006bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f4266000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42600cc060 con 0x7f42600d2460 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f4266000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42600cf2f0 con 0x7f426015ddc0 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f4266000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426006af30 con 0x7f426015e1a0 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f426015ddc0 0x7f426006b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f42600d2460 0x7f42600cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60168/0 (socket says 172.21.15.87:60168) 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425fe00640 1 -- 172.21.15.87:0/43085347 learned_addr learned my addr 172.21.15.87:0/43085347 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425ea00640 1 --2- 172.21.15.87:0/43085347 >> v2:172.21.15.169:3300/0 conn(0x7f426015e1a0 0x7f426006bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425f400640 1 -- 172.21.15.87:0/43085347 >> v2:172.21.15.169:3300/0 conn(0x7f426015e1a0 msgr2=0x7f426006bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425f400640 1 --2- 172.21.15.87:0/43085347 >> v2:172.21.15.169:3300/0 conn(0x7f426015e1a0 0x7f426006bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425f400640 1 -- 172.21.15.87:0/43085347 >> v1:172.21.15.87:6789/0 conn(0x7f42600d2460 legacy=0x7f42600cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425e000640 1 -- 172.21.15.87:0/43085347 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472219725 0 0) 0x7f42600cc060 con 0x7f42600d2460 2024-04-25T10:24:49.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.444+0000 7f425f400640 1 -- 172.21.15.87:0/43085347 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600cc340 con 0x7f426015ddc0 2024-04-25T10:24:49.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f425f400640 1 --2- 172.21.15.87:0/43085347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f426015ddc0 0x7f426006b610 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f42600cbf50 tx=0x7f425001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b6a2fbde3472e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f425e000640 1 -- 172.21.15.87:0/43085347 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500b2070 con 0x7f426015ddc0 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f425e000640 1 -- 172.21.15.87:0/43085347 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42500a57f0 con 0x7f426015ddc0 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f425e000640 1 -- 172.21.15.87:0/43085347 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500a5bc0 con 0x7f426015ddc0 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 -- 172.21.15.87:0/43085347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f426015ddc0 msgr2=0x7f426006b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 --2- 172.21.15.87:0/43085347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f426015ddc0 0x7f426006b610 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f42600cbf50 tx=0x7f425001c930 comp rx=0 tx=0).stop 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 -- 172.21.15.87:0/43085347 shutdown_connections 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 --2- 172.21.15.87:0/43085347 >> v2:172.21.15.169:3300/0 conn(0x7f426015e1a0 0x7f426006bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 --2- 172.21.15.87:0/43085347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f426015ddc0 0x7f426006b610 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 -- 172.21.15.87:0/43085347 >> 172.21.15.87:0/43085347 conn(0x7f42600d1970 msgr2=0x7f42600d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 -- 172.21.15.87:0/43085347 shutdown_connections 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.448+0000 7f4266000640 1 -- 172.21.15.87:0/43085347 wait complete. 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 Processor -- start 2024-04-25T10:24:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 -- start start 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42600d2460 0x7f426014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f426015ddc0 0x7f4260156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426015e1a0 0x7f4260155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426016ac90 con 0x7f42600d2460 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426016ab10 con 0x7f426015ddc0 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426016ae10 con 0x7f426015e1a0 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42600d2460 0x7f426014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42600d2460 0x7f426014b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43830/0 (socket says 172.21.15.87:43830) 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 -- 172.21.15.87:0/3605519006 learned_addr learned my addr 172.21.15.87:0/3605519006 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425ea00640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f426015ddc0 0x7f4260156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425fe00640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426015e1a0 0x7f4260155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 -- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426015e1a0 msgr2=0x7f4260155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426015e1a0 0x7f4260155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 -- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f426015ddc0 msgr2=0x7f4260156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f426015ddc0 0x7f4260156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 -- 172.21.15.87:0/3605519006 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260155760 con 0x7f42600d2460 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425fe00640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426015e1a0 0x7f4260155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f425f400640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42600d2460 0x7f426014b470 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f42500b0040 tx=0x7f42500196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4257400640 1 -- 172.21.15.87:0/3605519006 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500b2040 con 0x7f42600d2460 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4257400640 1 -- 172.21.15.87:0/3605519006 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42500bf070 con 0x7f42600d2460 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4260155a40 con 0x7f42600d2460 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4257400640 1 -- 172.21.15.87:0/3605519006 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500b8670 con 0x7f42600d2460 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.452+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4260156000 con 0x7f42600d2460 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.456+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426006ce20 con 0x7f42600d2460 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.456+0000 7f4257400640 1 -- 172.21.15.87:0/3605519006 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42500b8810 con 0x7f42600d2460 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.456+0000 7f4257400640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4240066f60 0x7f4240069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.456+0000 7f4257400640 1 -- 172.21.15.87:0/3605519006 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4250120270 con 0x7f42600d2460 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.456+0000 7f425ea00640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4240066f60 0x7f4240069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.456+0000 7f425ea00640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4240066f60 0x7f4240069420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f42600b4710 tx=0x7f424c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.460+0000 7f4257400640 1 -- 172.21.15.87:0/3605519006 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4250016520 con 0x7f42600d2460 2024-04-25T10:24:49.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.612+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f426004ef30 con 0x7f4240066f60 2024-04-25T10:24:49.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.620+0000 7f4257400640 1 -- 172.21.15.87:0/3605519006 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252411 (secure 0 0 0) 0x7f426004ef30 con 0x7f4240066f60 2024-04-25T10:24:49.622 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:49.623 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4240066f60 msgr2=0x7f4240069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4240066f60 0x7f4240069420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f42600b4710 tx=0x7f424c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42600d2460 msgr2=0x7f426014b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42600d2460 0x7f426014b470 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f42500b0040 tx=0x7f42500196d0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 shutdown_connections 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4240066f60 0x7f4240069420 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426015e1a0 0x7f4260155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f426015ddc0 0x7f4260156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 --2- 172.21.15.87:0/3605519006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42600d2460 0x7f426014b470 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 >> 172.21.15.87:0/3605519006 conn(0x7f42600d1970 msgr2=0x7f42600aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 shutdown_connections 2024-04-25T10:24:49.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.624+0000 7f4266000640 1 -- 172.21.15.87:0/3605519006 wait complete. 2024-04-25T10:24:49.643 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-04-25T10:24:49.282952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":471996,"kb_used_data":17136,"kb_used_omap":12,"kb_used_meta":454707,"kb_avail":754502724,"statfs":{"total":773094113280,"available":772610789376,"internally_reserved":0,"allocated":17547264,"data_stored":8330007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":465620529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.736346"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595886+0000","last_change":"2024-04-25T10:24:45.595191+0000","last_active":"2024-04-25T10:24:45.595886+0000","last_peered":"2024-04-25T10:24:45.595886+0000","last_clean":"2024-04-25T10:24:45.595886+0000","last_became_active":"2024-04-25T10:24:45.594880+0000","last_became_peered":"2024-04-25T10:24:45.594880+0000","last_unstale":"2024-04-25T10:24:45.595886+0000","last_undegraded":"2024-04-25T10:24:45.595886+0000","last_fullsized":"2024-04-25T10:24:45.595886+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:56.983176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587961+0000","last_change":"2024-04-25T10:24:45.585203+0000","last_active":"2024-04-25T10:24:45.587961+0000","last_peered":"2024-04-25T10:24:45.587961+0000","last_clean":"2024-04-25T10:24:45.587961+0000","last_became_active":"2024-04-25T10:24:45.584034+0000","last_became_peered":"2024-04-25T10:24:45.584034+0000","last_unstale":"2024-04-25T10:24:45.587961+0000","last_undegraded":"2024-04-25T10:24:45.587961+0000","last_fullsized":"2024-04-25T10:24:45.587961+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:56.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.567025+0000","last_change":"2024-04-25T10:24:45.563254+0000","last_active":"2024-04-25T10:24:45.567025+0000","last_peered":"2024-04-25T10:24:45.567025+0000","last_clean":"2024-04-25T10:24:45.567025+0000","last_became_active":"2024-04-25T10:24:45.562985+0000","last_became_peered":"2024-04-25T10:24:45.562985+0000","last_unstale":"2024-04-25T10:24:45.567025+0000","last_undegraded":"2024-04-25T10:24:45.567025+0000","last_fullsized":"2024-04-25T10:24:45.567025+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:03.831803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592028+0000","last_change":"2024-04-25T10:24:45.584879+0000","last_active":"2024-04-25T10:24:45.592028+0000","last_peered":"2024-04-25T10:24:45.592028+0000","last_clean":"2024-04-25T10:24:45.592028+0000","last_became_active":"2024-04-25T10:24:45.584609+0000","last_became_peered":"2024-04-25T10:24:45.584609+0000","last_unstale":"2024-04-25T10:24:45.592028+0000","last_undegraded":"2024-04-25T10:24:45.592028+0000","last_fullsized":"2024-04-25T10:24:45.592028+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.163732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.569338+0000","last_change":"2024-04-25T10:24:45.566024+0000","last_active":"2024-04-25T10:24:45.569338+0000","last_peered":"2024-04-25T10:24:45.569338+0000","last_clean":"2024-04-25T10:24:45.569338+0000","last_became_active":"2024-04-25T10:24:45.565755+0000","last_became_peered":"2024-04-25T10:24:45.565755+0000","last_unstale":"2024-04-25T10:24:45.569338+0000","last_undegraded":"2024-04-25T10:24:45.569338+0000","last_fullsized":"2024-04-25T10:24:45.569338+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:08.707430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.582549+0000","last_change":"2024-04-25T10:24:45.581329+0000","last_active":"2024-04-25T10:24:45.582549+0000","last_peered":"2024-04-25T10:24:45.582549+0000","last_clean":"2024-04-25T10:24:45.582549+0000","last_became_active":"2024-04-25T10:24:45.581084+0000","last_became_peered":"2024-04-25T10:24:45.581084+0000","last_unstale":"2024-04-25T10:24:45.582549+0000","last_undegraded":"2024-04-25T10:24:45.582549+0000","last_fullsized":"2024-04-25T10:24:45.582549+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.163732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.581397+0000","last_change":"2024-04-25T10:24:45.579956+0000","last_active":"2024-04-25T10:24:45.581397+0000","last_peered":"2024-04-25T10:24:45.581397+0000","last_clean":"2024-04-25T10:24:45.581397+0000","last_became_active":"2024-04-25T10:24:45.579671+0000","last_became_peered":"2024-04-25T10:24:45.579671+0000","last_unstale":"2024-04-25T10:24:45.581397+0000","last_undegraded":"2024-04-25T10:24:45.581397+0000","last_fullsized":"2024-04-25T10:24:45.581397+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.781823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.572906+0000","last_change":"2024-04-25T10:24:45.567486+0000","last_active":"2024-04-25T10:24:45.572906+0000","last_peered":"2024-04-25T10:24:45.572906+0000","last_clean":"2024-04-25T10:24:45.572906+0000","last_became_active":"2024-04-25T10:24:45.567000+0000","last_became_peered":"2024-04-25T10:24:45.567000+0000","last_unstale":"2024-04-25T10:24:45.572906+0000","last_undegraded":"2024-04-25T10:24:45.572906+0000","last_fullsized":"2024-04-25T10:24:45.572906+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:31.742923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.582458+0000","last_change":"2024-04-25T10:24:45.581610+0000","last_active":"2024-04-25T10:24:45.582458+0000","last_peered":"2024-04-25T10:24:45.582458+0000","last_clean":"2024-04-25T10:24:45.582458+0000","last_became_active":"2024-04-25T10:24:45.581330+0000","last_became_peered":"2024-04-25T10:24:45.581330+0000","last_unstale":"2024-04-25T10:24:45.582458+0000","last_undegraded":"2024-04-25T10:24:45.582458+0000","last_fullsized":"2024-04-25T10:24:45.582458+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:58.139988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595137+0000","last_change":"2024-04-25T10:24:45.588483+0000","last_active":"2024-04-25T10:24:45.595137+0000","last_peered":"2024-04-25T10:24:45.595137+0000","last_clean":"2024-04-25T10:24:45.595137+0000","last_became_active":"2024-04-25T10:24:45.586922+0000","last_became_peered":"2024-04-25T10:24:45.586922+0000","last_unstale":"2024-04-25T10:24:45.595137+0000","last_undegraded":"2024-04-25T10:24:45.595137+0000","last_fullsized":"2024-04-25T10:24:45.595137+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:15.030738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.580650+0000","last_change":"2024-04-25T10:24:45.579477+0000","last_active":"2024-04-25T10:24:45.580650+0000","last_peered":"2024-04-25T10:24:45.580650+0000","last_clean":"2024-04-25T10:24:45.580650+0000","last_became_active":"2024-04-25T10:24:45.578694+0000","last_became_peered":"2024-04-25T10:24:45.578694+0000","last_unstale":"2024-04-25T10:24:45.580650+0000","last_undegraded":"2024-04-25T10:24:45.580650+0000","last_fullsized":"2024-04-25T10:24:45.580650+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.525963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591998+0000","last_change":"2024-04-25T10:24:45.585138+0000","last_active":"2024-04-25T10:24:45.591998+0000","last_peered":"2024-04-25T10:24:45.591998+0000","last_clean":"2024-04-25T10:24:45.591998+0000","last_became_active":"2024-04-25T10:24:45.583029+0000","last_became_peered":"2024-04-25T10:24:45.583029+0000","last_unstale":"2024-04-25T10:24:45.591998+0000","last_undegraded":"2024-04-25T10:24:45.591998+0000","last_fullsized":"2024-04-25T10:24:45.591998+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.942544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587151+0000","last_change":"2024-04-25T10:24:45.582245+0000","last_active":"2024-04-25T10:24:45.587151+0000","last_peered":"2024-04-25T10:24:45.587151+0000","last_clean":"2024-04-25T10:24:45.587151+0000","last_became_active":"2024-04-25T10:24:45.581888+0000","last_became_peered":"2024-04-25T10:24:45.581888+0000","last_unstale":"2024-04-25T10:24:45.587151+0000","last_undegraded":"2024-04-25T10:24:45.587151+0000","last_fullsized":"2024-04-25T10:24:45.587151+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:23.304199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.590820+0000","last_change":"2024-04-25T10:24:45.589448+0000","last_active":"2024-04-25T10:24:45.590820+0000","last_peered":"2024-04-25T10:24:45.590820+0000","last_clean":"2024-04-25T10:24:45.590820+0000","last_became_active":"2024-04-25T10:24:45.589173+0000","last_became_peered":"2024-04-25T10:24:45.589173+0000","last_unstale":"2024-04-25T10:24:45.590820+0000","last_undegraded":"2024-04-25T10:24:45.590820+0000","last_fullsized":"2024-04-25T10:24:45.590820+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:22.703038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587096+0000","last_change":"2024-04-25T10:24:45.581893+0000","last_active":"2024-04-25T10:24:45.587096+0000","last_peered":"2024-04-25T10:24:45.587096+0000","last_clean":"2024-04-25T10:24:45.587096+0000","last_became_active":"2024-04-25T10:24:45.581556+0000","last_became_peered":"2024-04-25T10:24:45.581556+0000","last_unstale":"2024-04-25T10:24:45.587096+0000","last_undegraded":"2024-04-25T10:24:45.587096+0000","last_fullsized":"2024-04-25T10:24:45.587096+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:03.831803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.572887+0000","last_change":"2024-04-25T10:24:45.567278+0000","last_active":"2024-04-25T10:24:45.572887+0000","last_peered":"2024-04-25T10:24:45.572887+0000","last_clean":"2024-04-25T10:24:45.572887+0000","last_became_active":"2024-04-25T10:24:45.566600+0000","last_became_peered":"2024-04-25T10:24:45.566600+0000","last_unstale":"2024-04-25T10:24:45.572887+0000","last_undegraded":"2024-04-25T10:24:45.572887+0000","last_fullsized":"2024-04-25T10:24:45.572887+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:57.922135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.588963+0000","last_change":"2024-04-25T10:24:45.585721+0000","last_active":"2024-04-25T10:24:45.588963+0000","last_peered":"2024-04-25T10:24:45.588963+0000","last_clean":"2024-04-25T10:24:45.588963+0000","last_became_active":"2024-04-25T10:24:45.584409+0000","last_became_peered":"2024-04-25T10:24:45.584409+0000","last_unstale":"2024-04-25T10:24:45.588963+0000","last_undegraded":"2024-04-25T10:24:45.588963+0000","last_fullsized":"2024-04-25T10:24:45.588963+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:27.184670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.590230+0000","last_change":"2024-04-25T10:24:45.586705+0000","last_active":"2024-04-25T10:24:45.590230+0000","last_peered":"2024-04-25T10:24:45.590230+0000","last_clean":"2024-04-25T10:24:45.590230+0000","last_became_active":"2024-04-25T10:24:45.586412+0000","last_became_peered":"2024-04-25T10:24:45.586412+0000","last_unstale":"2024-04-25T10:24:45.590230+0000","last_undegraded":"2024-04-25T10:24:45.590230+0000","last_fullsized":"2024-04-25T10:24:45.590230+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:23.489417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595366+0000","last_change":"2024-04-25T10:24:45.589116+0000","last_active":"2024-04-25T10:24:45.595366+0000","last_peered":"2024-04-25T10:24:45.595366+0000","last_clean":"2024-04-25T10:24:45.595366+0000","last_became_active":"2024-04-25T10:24:45.588859+0000","last_became_peered":"2024-04-25T10:24:45.588859+0000","last_unstale":"2024-04-25T10:24:45.595366+0000","last_undegraded":"2024-04-25T10:24:45.595366+0000","last_fullsized":"2024-04-25T10:24:45.595366+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:59.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593225+0000","last_change":"2024-04-25T10:24:45.587679+0000","last_active":"2024-04-25T10:24:45.593225+0000","last_peered":"2024-04-25T10:24:45.593225+0000","last_clean":"2024-04-25T10:24:45.593225+0000","last_became_active":"2024-04-25T10:24:45.587398+0000","last_became_peered":"2024-04-25T10:24:45.587398+0000","last_unstale":"2024-04-25T10:24:45.593225+0000","last_undegraded":"2024-04-25T10:24:45.593225+0000","last_fullsized":"2024-04-25T10:24:45.593225+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:45.688169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.574474+0000","last_change":"2024-04-25T10:24:45.569441+0000","last_active":"2024-04-25T10:24:45.574474+0000","last_peered":"2024-04-25T10:24:45.574474+0000","last_clean":"2024-04-25T10:24:45.574474+0000","last_became_active":"2024-04-25T10:24:45.569177+0000","last_became_peered":"2024-04-25T10:24:45.569177+0000","last_unstale":"2024-04-25T10:24:45.574474+0000","last_undegraded":"2024-04-25T10:24:45.574474+0000","last_fullsized":"2024-04-25T10:24:45.574474+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:10.529104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592202+0000","last_change":"2024-04-25T10:24:45.584119+0000","last_active":"2024-04-25T10:24:45.592202+0000","last_peered":"2024-04-25T10:24:45.592202+0000","last_clean":"2024-04-25T10:24:45.592202+0000","last_became_active":"2024-04-25T10:24:45.583261+0000","last_became_peered":"2024-04-25T10:24:45.583261+0000","last_unstale":"2024-04-25T10:24:45.592202+0000","last_undegraded":"2024-04-25T10:24:45.592202+0000","last_fullsized":"2024-04-25T10:24:45.592202+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.754615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.579589+0000","last_change":"2024-04-25T10:24:45.578681+0000","last_active":"2024-04-25T10:24:45.579589+0000","last_peered":"2024-04-25T10:24:45.579589+0000","last_clean":"2024-04-25T10:24:45.579589+0000","last_became_active":"2024-04-25T10:24:45.578429+0000","last_became_peered":"2024-04-25T10:24:45.578429+0000","last_unstale":"2024-04-25T10:24:45.579589+0000","last_undegraded":"2024-04-25T10:24:45.579589+0000","last_fullsized":"2024-04-25T10:24:45.579589+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.221137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.572866+0000","last_change":"2024-04-25T10:24:45.565377+0000","last_active":"2024-04-25T10:24:45.572866+0000","last_peered":"2024-04-25T10:24:45.572866+0000","last_clean":"2024-04-25T10:24:45.572866+0000","last_became_active":"2024-04-25T10:24:45.565133+0000","last_became_peered":"2024-04-25T10:24:45.565133+0000","last_unstale":"2024-04-25T10:24:45.572866+0000","last_undegraded":"2024-04-25T10:24:45.572866+0000","last_fullsized":"2024-04-25T10:24:45.572866+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:29.444440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593665+0000","last_change":"2024-04-25T10:24:45.591120+0000","last_active":"2024-04-25T10:24:45.593665+0000","last_peered":"2024-04-25T10:24:45.593665+0000","last_clean":"2024-04-25T10:24:45.593665+0000","last_became_active":"2024-04-25T10:24:45.590785+0000","last_became_peered":"2024-04-25T10:24:45.590785+0000","last_unstale":"2024-04-25T10:24:45.593665+0000","last_undegraded":"2024-04-25T10:24:45.593665+0000","last_fullsized":"2024-04-25T10:24:45.593665+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.781823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595552+0000","last_change":"2024-04-25T10:24:45.594686+0000","last_active":"2024-04-25T10:24:45.595552+0000","last_peered":"2024-04-25T10:24:45.595552+0000","last_clean":"2024-04-25T10:24:45.595552+0000","last_became_active":"2024-04-25T10:24:45.594273+0000","last_became_peered":"2024-04-25T10:24:45.594273+0000","last_unstale":"2024-04-25T10:24:45.595552+0000","last_undegraded":"2024-04-25T10:24:45.595552+0000","last_fullsized":"2024-04-25T10:24:45.595552+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:31.184710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.584473+0000","last_change":"2024-04-25T10:24:45.581845+0000","last_active":"2024-04-25T10:24:45.584473+0000","last_peered":"2024-04-25T10:24:45.584473+0000","last_clean":"2024-04-25T10:24:45.584473+0000","last_became_active":"2024-04-25T10:24:45.581593+0000","last_became_peered":"2024-04-25T10:24:45.581593+0000","last_unstale":"2024-04-25T10:24:45.584473+0000","last_undegraded":"2024-04-25T10:24:45.584473+0000","last_fullsized":"2024-04-25T10:24:45.584473+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.942544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587800+0000","last_change":"2024-04-25T10:24:45.583629+0000","last_active":"2024-04-25T10:24:45.587800+0000","last_peered":"2024-04-25T10:24:45.587800+0000","last_clean":"2024-04-25T10:24:45.587800+0000","last_became_active":"2024-04-25T10:24:45.583315+0000","last_became_peered":"2024-04-25T10:24:45.583315+0000","last_unstale":"2024-04-25T10:24:45.587800+0000","last_undegraded":"2024-04-25T10:24:45.587800+0000","last_fullsized":"2024-04-25T10:24:45.587800+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:04.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587049+0000","last_change":"2024-04-25T10:24:45.581610+0000","last_active":"2024-04-25T10:24:45.587049+0000","last_peered":"2024-04-25T10:24:45.587049+0000","last_clean":"2024-04-25T10:24:45.587049+0000","last_became_active":"2024-04-25T10:24:45.581309+0000","last_became_peered":"2024-04-25T10:24:45.581309+0000","last_unstale":"2024-04-25T10:24:45.587049+0000","last_undegraded":"2024-04-25T10:24:45.587049+0000","last_fullsized":"2024-04-25T10:24:45.587049+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:46.844848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.585940+0000","last_change":"2024-04-25T10:24:45.583765+0000","last_active":"2024-04-25T10:24:45.585940+0000","last_peered":"2024-04-25T10:24:45.585940+0000","last_clean":"2024-04-25T10:24:45.585940+0000","last_became_active":"2024-04-25T10:24:45.583210+0000","last_became_peered":"2024-04-25T10:24:45.583210+0000","last_unstale":"2024-04-25T10:24:45.585940+0000","last_undegraded":"2024-04-25T10:24:45.585940+0000","last_fullsized":"2024-04-25T10:24:45.585940+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.081500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592258+0000","last_change":"2024-04-25T10:24:45.586141+0000","last_active":"2024-04-25T10:24:45.592258+0000","last_peered":"2024-04-25T10:24:45.592258+0000","last_clean":"2024-04-25T10:24:45.592258+0000","last_became_active":"2024-04-25T10:24:45.584975+0000","last_became_peered":"2024-04-25T10:24:45.584975+0000","last_unstale":"2024-04-25T10:24:45.592258+0000","last_undegraded":"2024-04-25T10:24:45.592258+0000","last_fullsized":"2024-04-25T10:24:45.592258+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:56.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.573248+0000","last_change":"2024-04-25T10:24:45.568815+0000","last_active":"2024-04-25T10:24:45.573248+0000","last_peered":"2024-04-25T10:24:45.573248+0000","last_clean":"2024-04-25T10:24:45.573248+0000","last_became_active":"2024-04-25T10:24:45.568574+0000","last_became_peered":"2024-04-25T10:24:45.568574+0000","last_unstale":"2024-04-25T10:24:45.573248+0000","last_undegraded":"2024-04-25T10:24:45.573248+0000","last_fullsized":"2024-04-25T10:24:45.573248+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:05.274064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578381+0000","last_change":"2024-04-25T10:24:45.569683+0000","last_active":"2024-04-25T10:24:45.578381+0000","last_peered":"2024-04-25T10:24:45.578381+0000","last_clean":"2024-04-25T10:24:45.578381+0000","last_became_active":"2024-04-25T10:24:45.569449+0000","last_became_peered":"2024-04-25T10:24:45.569449+0000","last_unstale":"2024-04-25T10:24:45.578381+0000","last_undegraded":"2024-04-25T10:24:45.578381+0000","last_fullsized":"2024-04-25T10:24:45.578381+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:36.679989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"162'4","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587440+0000","last_change":"2024-04-25T10:24:44.578368+0000","last_active":"2024-04-25T10:24:45.587440+0000","last_peered":"2024-04-25T10:24:45.587440+0000","last_clean":"2024-04-25T10:24:45.587440+0000","last_became_active":"2024-04-25T10:24:44.578082+0000","last_became_peered":"2024-04-25T10:24:44.578082+0000","last_unstale":"2024-04-25T10:24:45.587440+0000","last_undegraded":"2024-04-25T10:24:45.587440+0000","last_fullsized":"2024-04-25T10:24:45.587440+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:49.345469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595962+0000","last_change":"2024-04-25T10:24:45.594760+0000","last_active":"2024-04-25T10:24:45.595962+0000","last_peered":"2024-04-25T10:24:45.595962+0000","last_clean":"2024-04-25T10:24:45.595962+0000","last_became_active":"2024-04-25T10:24:45.592775+0000","last_became_peered":"2024-04-25T10:24:45.592775+0000","last_unstale":"2024-04-25T10:24:45.595962+0000","last_undegraded":"2024-04-25T10:24:45.595962+0000","last_fullsized":"2024-04-25T10:24:45.595962+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:57.445586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591323+0000","last_change":"2024-04-25T10:24:44.570037+0000","last_active":"2024-04-25T10:24:45.591323+0000","last_peered":"2024-04-25T10:24:45.591323+0000","last_clean":"2024-04-25T10:24:45.591323+0000","last_became_active":"2024-04-25T10:24:44.568812+0000","last_became_peered":"2024-04-25T10:24:44.568812+0000","last_unstale":"2024-04-25T10:24:45.591323+0000","last_undegraded":"2024-04-25T10:24:45.591323+0000","last_fullsized":"2024-04-25T10:24:45.591323+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:11.805224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.574927+0000","last_change":"2024-04-25T10:24:45.573319+0000","last_active":"2024-04-25T10:24:45.574927+0000","last_peered":"2024-04-25T10:24:45.574927+0000","last_clean":"2024-04-25T10:24:45.574927+0000","last_became_active":"2024-04-25T10:24:45.572780+0000","last_became_peered":"2024-04-25T10:24:45.572780+0000","last_unstale":"2024-04-25T10:24:45.574927+0000","last_undegraded":"2024-04-25T10:24:45.574927+0000","last_fullsized":"2024-04-25T10:24:45.574927+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.163732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591052+0000","last_change":"2024-04-25T10:24:44.581954+0000","last_active":"2024-04-25T10:24:45.591052+0000","last_peered":"2024-04-25T10:24:45.591052+0000","last_clean":"2024-04-25T10:24:45.591052+0000","last_became_active":"2024-04-25T10:24:44.580891+0000","last_became_peered":"2024-04-25T10:24:44.580891+0000","last_unstale":"2024-04-25T10:24:45.591052+0000","last_undegraded":"2024-04-25T10:24:45.591052+0000","last_fullsized":"2024-04-25T10:24:45.591052+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:44.016323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.570497+0000","last_change":"2024-04-25T10:24:45.568432+0000","last_active":"2024-04-25T10:24:45.570497+0000","last_peered":"2024-04-25T10:24:45.570497+0000","last_clean":"2024-04-25T10:24:45.570497+0000","last_became_active":"2024-04-25T10:24:45.568091+0000","last_became_peered":"2024-04-25T10:24:45.568091+0000","last_unstale":"2024-04-25T10:24:45.570497+0000","last_undegraded":"2024-04-25T10:24:45.570497+0000","last_fullsized":"2024-04-25T10:24:45.570497+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:05.864369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593737+0000","last_change":"2024-04-25T10:24:44.574923+0000","last_active":"2024-04-25T10:24:45.593737+0000","last_peered":"2024-04-25T10:24:45.593737+0000","last_clean":"2024-04-25T10:24:45.593737+0000","last_became_active":"2024-04-25T10:24:44.573640+0000","last_became_peered":"2024-04-25T10:24:44.573640+0000","last_unstale":"2024-04-25T10:24:45.593737+0000","last_undegraded":"2024-04-25T10:24:45.593737+0000","last_fullsized":"2024-04-25T10:24:45.593737+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:36.842473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592181+0000","last_change":"2024-04-25T10:24:45.583940+0000","last_active":"2024-04-25T10:24:45.592181+0000","last_peered":"2024-04-25T10:24:45.592181+0000","last_clean":"2024-04-25T10:24:45.592181+0000","last_became_active":"2024-04-25T10:24:45.582839+0000","last_became_peered":"2024-04-25T10:24:45.582839+0000","last_unstale":"2024-04-25T10:24:45.592181+0000","last_undegraded":"2024-04-25T10:24:45.592181+0000","last_fullsized":"2024-04-25T10:24:45.592181+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:48.551078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592978+0000","last_change":"2024-04-25T10:24:44.578208+0000","last_active":"2024-04-25T10:24:45.592978+0000","last_peered":"2024-04-25T10:24:45.592978+0000","last_clean":"2024-04-25T10:24:45.592978+0000","last_became_active":"2024-04-25T10:24:44.577944+0000","last_became_peered":"2024-04-25T10:24:44.577944+0000","last_unstale":"2024-04-25T10:24:45.592978+0000","last_undegraded":"2024-04-25T10:24:45.592978+0000","last_fullsized":"2024-04-25T10:24:45.592978+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:41.148190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.580020+0000","last_change":"2024-04-25T10:24:45.572122+0000","last_active":"2024-04-25T10:24:45.580020+0000","last_peered":"2024-04-25T10:24:45.580020+0000","last_clean":"2024-04-25T10:24:45.580020+0000","last_became_active":"2024-04-25T10:24:45.571770+0000","last_became_peered":"2024-04-25T10:24:45.571770+0000","last_unstale":"2024-04-25T10:24:45.580020+0000","last_undegraded":"2024-04-25T10:24:45.580020+0000","last_fullsized":"2024-04-25T10:24:45.580020+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:36.377896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592304+0000","last_change":"2024-04-25T10:24:44.563909+0000","last_active":"2024-04-25T10:24:45.592304+0000","last_peered":"2024-04-25T10:24:45.592304+0000","last_clean":"2024-04-25T10:24:45.592304+0000","last_became_active":"2024-04-25T10:24:44.563612+0000","last_became_peered":"2024-04-25T10:24:44.563612+0000","last_unstale":"2024-04-25T10:24:45.592304+0000","last_undegraded":"2024-04-25T10:24:45.592304+0000","last_fullsized":"2024-04-25T10:24:45.592304+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:46.867085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578652+0000","last_change":"2024-04-25T10:24:45.575992+0000","last_active":"2024-04-25T10:24:45.578652+0000","last_peered":"2024-04-25T10:24:45.578652+0000","last_clean":"2024-04-25T10:24:45.578652+0000","last_became_active":"2024-04-25T10:24:45.575723+0000","last_became_peered":"2024-04-25T10:24:45.575723+0000","last_unstale":"2024-04-25T10:24:45.578652+0000","last_undegraded":"2024-04-25T10:24:45.578652+0000","last_fullsized":"2024-04-25T10:24:45.578652+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.942544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.569868+0000","last_change":"2024-04-25T10:24:44.578499+0000","last_active":"2024-04-25T10:24:45.569868+0000","last_peered":"2024-04-25T10:24:45.569868+0000","last_clean":"2024-04-25T10:24:45.569868+0000","last_became_active":"2024-04-25T10:24:44.578193+0000","last_became_peered":"2024-04-25T10:24:44.578193+0000","last_unstale":"2024-04-25T10:24:45.569868+0000","last_undegraded":"2024-04-25T10:24:45.569868+0000","last_fullsized":"2024-04-25T10:24:45.569868+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:41.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.568854+0000","last_change":"2024-04-25T10:24:45.567801+0000","last_active":"2024-04-25T10:24:45.568854+0000","last_peered":"2024-04-25T10:24:45.568854+0000","last_clean":"2024-04-25T10:24:45.568854+0000","last_became_active":"2024-04-25T10:24:45.567536+0000","last_became_peered":"2024-04-25T10:24:45.567536+0000","last_unstale":"2024-04-25T10:24:45.568854+0000","last_undegraded":"2024-04-25T10:24:45.568854+0000","last_fullsized":"2024-04-25T10:24:45.568854+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:23.304199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.563531+0000","last_change":"2024-04-25T10:24:44.571653+0000","last_active":"2024-04-25T10:24:45.563531+0000","last_peered":"2024-04-25T10:24:45.563531+0000","last_clean":"2024-04-25T10:24:45.563531+0000","last_became_active":"2024-04-25T10:24:44.571299+0000","last_became_peered":"2024-04-25T10:24:44.571299+0000","last_unstale":"2024-04-25T10:24:45.563531+0000","last_undegraded":"2024-04-25T10:24:45.563531+0000","last_fullsized":"2024-04-25T10:24:45.563531+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:02.825232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592017+0000","last_change":"2024-04-25T10:24:45.589729+0000","last_active":"2024-04-25T10:24:45.592017+0000","last_peered":"2024-04-25T10:24:45.592017+0000","last_clean":"2024-04-25T10:24:45.592017+0000","last_became_active":"2024-04-25T10:24:45.589058+0000","last_became_peered":"2024-04-25T10:24:45.589058+0000","last_unstale":"2024-04-25T10:24:45.592017+0000","last_undegraded":"2024-04-25T10:24:45.592017+0000","last_fullsized":"2024-04-25T10:24:45.592017+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.076168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.564815+0000","last_change":"2024-04-25T10:24:44.574709+0000","last_active":"2024-04-25T10:24:45.564815+0000","last_peered":"2024-04-25T10:24:45.564815+0000","last_clean":"2024-04-25T10:24:45.564815+0000","last_became_active":"2024-04-25T10:24:44.574448+0000","last_became_peered":"2024-04-25T10:24:44.574448+0000","last_unstale":"2024-04-25T10:24:45.564815+0000","last_undegraded":"2024-04-25T10:24:45.564815+0000","last_fullsized":"2024-04-25T10:24:45.564815+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.312219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.570831+0000","last_change":"2024-04-25T10:24:45.569706+0000","last_active":"2024-04-25T10:24:45.570831+0000","last_peered":"2024-04-25T10:24:45.570831+0000","last_clean":"2024-04-25T10:24:45.570831+0000","last_became_active":"2024-04-25T10:24:45.569374+0000","last_became_peered":"2024-04-25T10:24:45.569374+0000","last_unstale":"2024-04-25T10:24:45.570831+0000","last_undegraded":"2024-04-25T10:24:45.570831+0000","last_fullsized":"2024-04-25T10:24:45.570831+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.221137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"162'3","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.589235+0000","last_change":"2024-04-25T10:24:44.574732+0000","last_active":"2024-04-25T10:24:45.589235+0000","last_peered":"2024-04-25T10:24:45.589235+0000","last_clean":"2024-04-25T10:24:45.589235+0000","last_became_active":"2024-04-25T10:24:44.574465+0000","last_became_peered":"2024-04-25T10:24:44.574465+0000","last_unstale":"2024-04-25T10:24:45.589235+0000","last_undegraded":"2024-04-25T10:24:45.589235+0000","last_fullsized":"2024-04-25T10:24:45.589235+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:57.298082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595387+0000","last_change":"2024-04-25T10:24:45.589220+0000","last_active":"2024-04-25T10:24:45.595387+0000","last_peered":"2024-04-25T10:24:45.595387+0000","last_clean":"2024-04-25T10:24:45.595387+0000","last_became_active":"2024-04-25T10:24:45.588939+0000","last_became_peered":"2024-04-25T10:24:45.588939+0000","last_unstale":"2024-04-25T10:24:45.595387+0000","last_undegraded":"2024-04-25T10:24:45.595387+0000","last_fullsized":"2024-04-25T10:24:45.595387+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.873656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"162'6","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595470+0000","last_change":"2024-04-25T10:24:44.574592+0000","last_active":"2024-04-25T10:24:45.595470+0000","last_peered":"2024-04-25T10:24:45.595470+0000","last_clean":"2024-04-25T10:24:45.595470+0000","last_became_active":"2024-04-25T10:24:44.574095+0000","last_became_peered":"2024-04-25T10:24:44.574095+0000","last_unstale":"2024-04-25T10:24:45.595470+0000","last_undegraded":"2024-04-25T10:24:45.595470+0000","last_fullsized":"2024-04-25T10:24:45.595470+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:25.110558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592365+0000","last_change":"2024-04-25T10:24:45.591073+0000","last_active":"2024-04-25T10:24:45.592365+0000","last_peered":"2024-04-25T10:24:45.592365+0000","last_clean":"2024-04-25T10:24:45.592365+0000","last_became_active":"2024-04-25T10:24:45.590785+0000","last_became_peered":"2024-04-25T10:24:45.590785+0000","last_unstale":"2024-04-25T10:24:45.592365+0000","last_undegraded":"2024-04-25T10:24:45.592365+0000","last_fullsized":"2024-04-25T10:24:45.592365+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.525963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.596768+0000","last_change":"2024-04-25T10:24:44.578556+0000","last_active":"2024-04-25T10:24:45.596768+0000","last_peered":"2024-04-25T10:24:45.596768+0000","last_clean":"2024-04-25T10:24:45.596768+0000","last_became_active":"2024-04-25T10:24:44.578252+0000","last_became_peered":"2024-04-25T10:24:44.578252+0000","last_unstale":"2024-04-25T10:24:45.596768+0000","last_undegraded":"2024-04-25T10:24:45.596768+0000","last_fullsized":"2024-04-25T10:24:45.596768+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:32.350924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592774+0000","last_change":"2024-04-25T10:24:45.590443+0000","last_active":"2024-04-25T10:24:45.592774+0000","last_peered":"2024-04-25T10:24:45.592774+0000","last_clean":"2024-04-25T10:24:45.592774+0000","last_became_active":"2024-04-25T10:24:45.590192+0000","last_became_peered":"2024-04-25T10:24:45.590192+0000","last_unstale":"2024-04-25T10:24:45.592774+0000","last_undegraded":"2024-04-25T10:24:45.592774+0000","last_fullsized":"2024-04-25T10:24:45.592774+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:32.873972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.575588+0000","last_change":"2024-04-25T10:24:44.575626+0000","last_active":"2024-04-25T10:24:45.575588+0000","last_peered":"2024-04-25T10:24:45.575588+0000","last_clean":"2024-04-25T10:24:45.575588+0000","last_became_active":"2024-04-25T10:24:44.575337+0000","last_became_peered":"2024-04-25T10:24:44.575337+0000","last_unstale":"2024-04-25T10:24:45.575588+0000","last_undegraded":"2024-04-25T10:24:45.575588+0000","last_fullsized":"2024-04-25T10:24:45.575588+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:34.028064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593028+0000","last_change":"2024-04-25T10:24:45.591758+0000","last_active":"2024-04-25T10:24:45.593028+0000","last_peered":"2024-04-25T10:24:45.593028+0000","last_clean":"2024-04-25T10:24:45.593028+0000","last_became_active":"2024-04-25T10:24:45.591292+0000","last_became_peered":"2024-04-25T10:24:45.591292+0000","last_unstale":"2024-04-25T10:24:45.593028+0000","last_undegraded":"2024-04-25T10:24:45.593028+0000","last_fullsized":"2024-04-25T10:24:45.593028+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:12.736031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.561704+0000","last_change":"2024-04-25T10:24:44.573726+0000","last_active":"2024-04-25T10:24:45.561704+0000","last_peered":"2024-04-25T10:24:45.561704+0000","last_clean":"2024-04-25T10:24:45.561704+0000","last_became_active":"2024-04-25T10:24:44.573458+0000","last_became_peered":"2024-04-25T10:24:44.573458+0000","last_unstale":"2024-04-25T10:24:45.561704+0000","last_undegraded":"2024-04-25T10:24:45.561704+0000","last_fullsized":"2024-04-25T10:24:45.561704+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:43.165781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.589342+0000","last_change":"2024-04-25T10:24:45.578450+0000","last_active":"2024-04-25T10:24:45.589342+0000","last_peered":"2024-04-25T10:24:45.589342+0000","last_clean":"2024-04-25T10:24:45.589342+0000","last_became_active":"2024-04-25T10:24:45.578164+0000","last_became_peered":"2024-04-25T10:24:45.578164+0000","last_unstale":"2024-04-25T10:24:45.589342+0000","last_undegraded":"2024-04-25T10:24:45.589342+0000","last_fullsized":"2024-04-25T10:24:45.589342+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:27.233033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"162'8","reported_seq":20,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.596790+0000","last_change":"2024-04-25T10:24:44.578694+0000","last_active":"2024-04-25T10:24:45.596790+0000","last_peered":"2024-04-25T10:24:45.596790+0000","last_clean":"2024-04-25T10:24:45.596790+0000","last_became_active":"2024-04-25T10:24:44.578226+0000","last_became_peered":"2024-04-25T10:24:44.578226+0000","last_unstale":"2024-04-25T10:24:45.596790+0000","last_undegraded":"2024-04-25T10:24:45.596790+0000","last_fullsized":"2024-04-25T10:24:45.596790+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:35.371325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592651+0000","last_change":"2024-04-25T10:24:45.587097+0000","last_active":"2024-04-25T10:24:45.592651+0000","last_peered":"2024-04-25T10:24:45.592651+0000","last_clean":"2024-04-25T10:24:45.592651+0000","last_became_active":"2024-04-25T10:24:45.586780+0000","last_became_peered":"2024-04-25T10:24:45.586780+0000","last_unstale":"2024-04-25T10:24:45.592651+0000","last_undegraded":"2024-04-25T10:24:45.592651+0000","last_fullsized":"2024-04-25T10:24:45.592651+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:17.086213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.594005+0000","last_change":"2024-04-25T10:24:44.579142+0000","last_active":"2024-04-25T10:24:45.594005+0000","last_peered":"2024-04-25T10:24:45.594005+0000","last_clean":"2024-04-25T10:24:45.594005+0000","last_became_active":"2024-04-25T10:24:44.578800+0000","last_became_peered":"2024-04-25T10:24:44.578800+0000","last_unstale":"2024-04-25T10:24:45.594005+0000","last_undegraded":"2024-04-25T10:24:45.594005+0000","last_fullsized":"2024-04-25T10:24:45.594005+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:44.491722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593384+0000","last_change":"2024-04-25T10:24:45.592225+0000","last_active":"2024-04-25T10:24:45.593384+0000","last_peered":"2024-04-25T10:24:45.593384+0000","last_clean":"2024-04-25T10:24:45.593384+0000","last_became_active":"2024-04-25T10:24:45.591917+0000","last_became_peered":"2024-04-25T10:24:45.591917+0000","last_unstale":"2024-04-25T10:24:45.593384+0000","last_undegraded":"2024-04-25T10:24:45.593384+0000","last_fullsized":"2024-04-25T10:24:45.593384+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:26.839809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.588618+0000","last_change":"2024-04-25T10:24:44.571704+0000","last_active":"2024-04-25T10:24:45.588618+0000","last_peered":"2024-04-25T10:24:45.588618+0000","last_clean":"2024-04-25T10:24:45.588618+0000","last_became_active":"2024-04-25T10:24:44.571438+0000","last_became_peered":"2024-04-25T10:24:44.571438+0000","last_unstale":"2024-04-25T10:24:45.588618+0000","last_undegraded":"2024-04-25T10:24:45.588618+0000","last_fullsized":"2024-04-25T10:24:45.588618+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:36.468500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.585962+0000","last_change":"2024-04-25T10:24:45.583633+0000","last_active":"2024-04-25T10:24:45.585962+0000","last_peered":"2024-04-25T10:24:45.585962+0000","last_clean":"2024-04-25T10:24:45.585962+0000","last_became_active":"2024-04-25T10:24:45.583339+0000","last_became_peered":"2024-04-25T10:24:45.583339+0000","last_unstale":"2024-04-25T10:24:45.585962+0000","last_undegraded":"2024-04-25T10:24:45.585962+0000","last_fullsized":"2024-04-25T10:24:45.585962+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:13.138024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.596104+0000","last_change":"2024-04-25T10:24:44.579724+0000","last_active":"2024-04-25T10:24:45.596104+0000","last_peered":"2024-04-25T10:24:45.596104+0000","last_clean":"2024-04-25T10:24:45.596104+0000","last_became_active":"2024-04-25T10:24:44.579374+0000","last_became_peered":"2024-04-25T10:24:44.579374+0000","last_unstale":"2024-04-25T10:24:45.596104+0000","last_undegraded":"2024-04-25T10:24:45.596104+0000","last_fullsized":"2024-04-25T10:24:45.596104+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:53.227740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591398+0000","last_change":"2024-04-25T10:24:45.586876+0000","last_active":"2024-04-25T10:24:45.591398+0000","last_peered":"2024-04-25T10:24:45.591398+0000","last_clean":"2024-04-25T10:24:45.591398+0000","last_became_active":"2024-04-25T10:24:45.586236+0000","last_became_peered":"2024-04-25T10:24:45.586236+0000","last_unstale":"2024-04-25T10:24:45.591398+0000","last_undegraded":"2024-04-25T10:24:45.591398+0000","last_fullsized":"2024-04-25T10:24:45.591398+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.525963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.563752+0000","last_change":"2024-04-25T10:24:44.578917+0000","last_active":"2024-04-25T10:24:45.563752+0000","last_peered":"2024-04-25T10:24:45.563752+0000","last_clean":"2024-04-25T10:24:45.563752+0000","last_became_active":"2024-04-25T10:24:44.578565+0000","last_became_peered":"2024-04-25T10:24:44.578565+0000","last_unstale":"2024-04-25T10:24:45.563752+0000","last_undegraded":"2024-04-25T10:24:45.563752+0000","last_fullsized":"2024-04-25T10:24:45.563752+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:22.297627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592221+0000","last_change":"2024-04-25T10:24:45.584304+0000","last_active":"2024-04-25T10:24:45.592221+0000","last_peered":"2024-04-25T10:24:45.592221+0000","last_clean":"2024-04-25T10:24:45.592221+0000","last_became_active":"2024-04-25T10:24:45.583638+0000","last_became_peered":"2024-04-25T10:24:45.583638+0000","last_unstale":"2024-04-25T10:24:45.592221+0000","last_undegraded":"2024-04-25T10:24:45.592221+0000","last_fullsized":"2024-04-25T10:24:45.592221+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:46.841052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591880+0000","last_change":"2024-04-25T10:24:44.579944+0000","last_active":"2024-04-25T10:24:45.591880+0000","last_peered":"2024-04-25T10:24:45.591880+0000","last_clean":"2024-04-25T10:24:45.591880+0000","last_became_active":"2024-04-25T10:24:44.579691+0000","last_became_peered":"2024-04-25T10:24:44.579691+0000","last_unstale":"2024-04-25T10:24:45.591880+0000","last_undegraded":"2024-04-25T10:24:45.591880+0000","last_fullsized":"2024-04-25T10:24:45.591880+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:45.838277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578826+0000","last_change":"2024-04-25T10:24:45.577183+0000","last_active":"2024-04-25T10:24:45.578826+0000","last_peered":"2024-04-25T10:24:45.578826+0000","last_clean":"2024-04-25T10:24:45.578826+0000","last_became_active":"2024-04-25T10:24:45.575091+0000","last_became_peered":"2024-04-25T10:24:45.575091+0000","last_unstale":"2024-04-25T10:24:45.578826+0000","last_undegraded":"2024-04-25T10:24:45.578826+0000","last_fullsized":"2024-04-25T10:24:45.578826+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:56.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.562862+0000","last_change":"2024-04-25T10:24:44.573889+0000","last_active":"2024-04-25T10:24:45.562862+0000","last_peered":"2024-04-25T10:24:45.562862+0000","last_clean":"2024-04-25T10:24:45.562862+0000","last_became_active":"2024-04-25T10:24:44.572072+0000","last_became_peered":"2024-04-25T10:24:44.572072+0000","last_unstale":"2024-04-25T10:24:45.562862+0000","last_undegraded":"2024-04-25T10:24:45.562862+0000","last_fullsized":"2024-04-25T10:24:45.562862+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:43.137800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.583532+0000","last_change":"2024-04-25T10:24:45.579966+0000","last_active":"2024-04-25T10:24:45.583532+0000","last_peered":"2024-04-25T10:24:45.583532+0000","last_clean":"2024-04-25T10:24:45.583532+0000","last_became_active":"2024-04-25T10:24:45.579697+0000","last_became_peered":"2024-04-25T10:24:45.579697+0000","last_unstale":"2024-04-25T10:24:45.583532+0000","last_undegraded":"2024-04-25T10:24:45.583532+0000","last_fullsized":"2024-04-25T10:24:45.583532+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:45.022894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.558317+0000","last_change":"2024-04-25T10:24:44.579396+0000","last_active":"2024-04-25T10:24:45.558317+0000","last_peered":"2024-04-25T10:24:45.558317+0000","last_clean":"2024-04-25T10:24:45.558317+0000","last_became_active":"2024-04-25T10:24:44.579139+0000","last_became_peered":"2024-04-25T10:24:44.579139+0000","last_unstale":"2024-04-25T10:24:45.558317+0000","last_undegraded":"2024-04-25T10:24:45.558317+0000","last_fullsized":"2024-04-25T10:24:45.558317+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:36.842473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.573416+0000","last_change":"2024-04-25T10:24:45.568429+0000","last_active":"2024-04-25T10:24:45.573416+0000","last_peered":"2024-04-25T10:24:45.573416+0000","last_clean":"2024-04-25T10:24:45.573416+0000","last_became_active":"2024-04-25T10:24:45.568166+0000","last_became_peered":"2024-04-25T10:24:45.568166+0000","last_unstale":"2024-04-25T10:24:45.573416+0000","last_undegraded":"2024-04-25T10:24:45.573416+0000","last_fullsized":"2024-04-25T10:24:45.573416+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:46.957005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.562107+0000","last_change":"2024-04-25T10:24:44.572059+0000","last_active":"2024-04-25T10:24:45.562107+0000","last_peered":"2024-04-25T10:24:45.562107+0000","last_clean":"2024-04-25T10:24:45.562107+0000","last_became_active":"2024-04-25T10:24:44.571754+0000","last_became_peered":"2024-04-25T10:24:44.571754+0000","last_unstale":"2024-04-25T10:24:45.562107+0000","last_undegraded":"2024-04-25T10:24:45.562107+0000","last_fullsized":"2024-04-25T10:24:45.562107+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:46.241419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.584415+0000","last_change":"2024-04-25T10:24:45.570143+0000","last_active":"2024-04-25T10:24:45.584415+0000","last_peered":"2024-04-25T10:24:45.584415+0000","last_clean":"2024-04-25T10:24:45.584415+0000","last_became_active":"2024-04-25T10:24:45.569862+0000","last_became_peered":"2024-04-25T10:24:45.569862+0000","last_unstale":"2024-04-25T10:24:45.584415+0000","last_undegraded":"2024-04-25T10:24:45.584415+0000","last_fullsized":"2024-04-25T10:24:45.584415+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:22.827670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595166+0000","last_change":"2024-04-25T10:24:44.580541+0000","last_active":"2024-04-25T10:24:45.595166+0000","last_peered":"2024-04-25T10:24:45.595166+0000","last_clean":"2024-04-25T10:24:45.595166+0000","last_became_active":"2024-04-25T10:24:44.580192+0000","last_became_peered":"2024-04-25T10:24:44.580192+0000","last_unstale":"2024-04-25T10:24:45.595166+0000","last_undegraded":"2024-04-25T10:24:45.595166+0000","last_fullsized":"2024-04-25T10:24:45.595166+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:05.305075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595054+0000","last_change":"2024-04-25T10:24:45.588764+0000","last_active":"2024-04-25T10:24:45.595054+0000","last_peered":"2024-04-25T10:24:45.595054+0000","last_clean":"2024-04-25T10:24:45.595054+0000","last_became_active":"2024-04-25T10:24:45.588299+0000","last_became_peered":"2024-04-25T10:24:45.588299+0000","last_unstale":"2024-04-25T10:24:45.595054+0000","last_undegraded":"2024-04-25T10:24:45.595054+0000","last_fullsized":"2024-04-25T10:24:45.595054+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:44.172352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.581395+0000","last_change":"2024-04-25T10:24:44.581302+0000","last_active":"2024-04-25T10:24:45.581395+0000","last_peered":"2024-04-25T10:24:45.581395+0000","last_clean":"2024-04-25T10:24:45.581395+0000","last_became_active":"2024-04-25T10:24:44.581055+0000","last_became_peered":"2024-04-25T10:24:44.581055+0000","last_unstale":"2024-04-25T10:24:45.581395+0000","last_undegraded":"2024-04-25T10:24:45.581395+0000","last_fullsized":"2024-04-25T10:24:45.581395+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:17.242801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.579667+0000","last_change":"2024-04-25T10:24:45.570736+0000","last_active":"2024-04-25T10:24:45.579667+0000","last_peered":"2024-04-25T10:24:45.579667+0000","last_clean":"2024-04-25T10:24:45.579667+0000","last_became_active":"2024-04-25T10:24:45.570479+0000","last_became_peered":"2024-04-25T10:24:45.570479+0000","last_unstale":"2024-04-25T10:24:45.579667+0000","last_undegraded":"2024-04-25T10:24:45.579667+0000","last_fullsized":"2024-04-25T10:24:45.579667+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:42.417735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.579745+0000","last_change":"2024-04-25T10:24:44.579996+0000","last_active":"2024-04-25T10:24:45.579745+0000","last_peered":"2024-04-25T10:24:45.579745+0000","last_clean":"2024-04-25T10:24:45.579745+0000","last_became_active":"2024-04-25T10:24:44.579721+0000","last_became_peered":"2024-04-25T10:24:44.579721+0000","last_unstale":"2024-04-25T10:24:45.579745+0000","last_undegraded":"2024-04-25T10:24:45.579745+0000","last_fullsized":"2024-04-25T10:24:45.579745+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:11.154380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578532+0000","last_change":"2024-04-25T10:24:45.568731+0000","last_active":"2024-04-25T10:24:45.578532+0000","last_peered":"2024-04-25T10:24:45.578532+0000","last_clean":"2024-04-25T10:24:45.578532+0000","last_became_active":"2024-04-25T10:24:45.568484+0000","last_became_peered":"2024-04-25T10:24:45.568484+0000","last_unstale":"2024-04-25T10:24:45.578532+0000","last_undegraded":"2024-04-25T10:24:45.578532+0000","last_fullsized":"2024-04-25T10:24:45.578532+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:56.760510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.563225+0000","last_change":"2024-04-25T10:24:44.572962+0000","last_active":"2024-04-25T10:24:45.563225+0000","last_peered":"2024-04-25T10:24:45.563225+0000","last_clean":"2024-04-25T10:24:45.563225+0000","last_became_active":"2024-04-25T10:24:44.572615+0000","last_became_peered":"2024-04-25T10:24:44.572615+0000","last_unstale":"2024-04-25T10:24:45.563225+0000","last_undegraded":"2024-04-25T10:24:45.563225+0000","last_fullsized":"2024-04-25T10:24:45.563225+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:56.439015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.586964+0000","last_change":"2024-04-25T10:24:45.585182+0000","last_active":"2024-04-25T10:24:45.586964+0000","last_peered":"2024-04-25T10:24:45.586964+0000","last_clean":"2024-04-25T10:24:45.586964+0000","last_became_active":"2024-04-25T10:24:45.584861+0000","last_became_peered":"2024-04-25T10:24:45.584861+0000","last_unstale":"2024-04-25T10:24:45.586964+0000","last_undegraded":"2024-04-25T10:24:45.586964+0000","last_fullsized":"2024-04-25T10:24:45.586964+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:19.995860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.561736+0000","last_change":"2024-04-25T10:24:44.576166+0000","last_active":"2024-04-25T10:24:45.561736+0000","last_peered":"2024-04-25T10:24:45.561736+0000","last_clean":"2024-04-25T10:24:45.561736+0000","last_became_active":"2024-04-25T10:24:44.575777+0000","last_became_peered":"2024-04-25T10:24:44.575777+0000","last_unstale":"2024-04-25T10:24:45.561736+0000","last_undegraded":"2024-04-25T10:24:45.561736+0000","last_fullsized":"2024-04-25T10:24:45.561736+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:16.850314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.582319+0000","last_change":"2024-04-25T10:24:45.579532+0000","last_active":"2024-04-25T10:24:45.582319+0000","last_peered":"2024-04-25T10:24:45.582319+0000","last_clean":"2024-04-25T10:24:45.582319+0000","last_became_active":"2024-04-25T10:24:45.578966+0000","last_became_peered":"2024-04-25T10:24:45.578966+0000","last_unstale":"2024-04-25T10:24:45.582319+0000","last_undegraded":"2024-04-25T10:24:45.582319+0000","last_fullsized":"2024-04-25T10:24:45.582319+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.561938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.562110+0000","last_change":"2024-04-25T10:24:44.573498+0000","last_active":"2024-04-25T10:24:45.562110+0000","last_peered":"2024-04-25T10:24:45.562110+0000","last_clean":"2024-04-25T10:24:45.562110+0000","last_became_active":"2024-04-25T10:24:44.572908+0000","last_became_peered":"2024-04-25T10:24:44.572908+0000","last_unstale":"2024-04-25T10:24:45.562110+0000","last_undegraded":"2024-04-25T10:24:45.562110+0000","last_fullsized":"2024-04-25T10:24:45.562110+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:26.754531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587224+0000","last_change":"2024-04-25T10:24:45.583396+0000","last_active":"2024-04-25T10:24:45.587224+0000","last_peered":"2024-04-25T10:24:45.587224+0000","last_clean":"2024-04-25T10:24:45.587224+0000","last_became_active":"2024-04-25T10:24:45.583082+0000","last_became_peered":"2024-04-25T10:24:45.583082+0000","last_unstale":"2024-04-25T10:24:45.587224+0000","last_undegraded":"2024-04-25T10:24:45.587224+0000","last_fullsized":"2024-04-25T10:24:45.587224+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.221137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.576743+0000","last_change":"2024-04-25T10:24:44.567927+0000","last_active":"2024-04-25T10:24:45.576743+0000","last_peered":"2024-04-25T10:24:45.576743+0000","last_clean":"2024-04-25T10:24:45.576743+0000","last_became_active":"2024-04-25T10:24:44.567656+0000","last_became_peered":"2024-04-25T10:24:44.567656+0000","last_unstale":"2024-04-25T10:24:45.576743+0000","last_undegraded":"2024-04-25T10:24:45.576743+0000","last_fullsized":"2024-04-25T10:24:45.576743+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:25.110558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.588850+0000","last_change":"2024-04-25T10:24:45.582612+0000","last_active":"2024-04-25T10:24:45.588850+0000","last_peered":"2024-04-25T10:24:45.588850+0000","last_clean":"2024-04-25T10:24:45.588850+0000","last_became_active":"2024-04-25T10:24:45.582373+0000","last_became_peered":"2024-04-25T10:24:45.582373+0000","last_unstale":"2024-04-25T10:24:45.588850+0000","last_undegraded":"2024-04-25T10:24:45.588850+0000","last_fullsized":"2024-04-25T10:24:45.588850+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:52.922635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.557904+0000","last_change":"2024-04-25T10:24:44.572730+0000","last_active":"2024-04-25T10:24:45.557904+0000","last_peered":"2024-04-25T10:24:45.557904+0000","last_clean":"2024-04-25T10:24:45.557904+0000","last_became_active":"2024-04-25T10:24:44.572457+0000","last_became_peered":"2024-04-25T10:24:44.572457+0000","last_unstale":"2024-04-25T10:24:45.557904+0000","last_undegraded":"2024-04-25T10:24:45.557904+0000","last_fullsized":"2024-04-25T10:24:45.557904+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:41.605683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593097+0000","last_change":"2024-04-25T10:24:45.587826+0000","last_active":"2024-04-25T10:24:45.593097+0000","last_peered":"2024-04-25T10:24:45.593097+0000","last_clean":"2024-04-25T10:24:45.593097+0000","last_became_active":"2024-04-25T10:24:45.587554+0000","last_became_peered":"2024-04-25T10:24:45.587554+0000","last_unstale":"2024-04-25T10:24:45.593097+0000","last_undegraded":"2024-04-25T10:24:45.593097+0000","last_fullsized":"2024-04-25T10:24:45.593097+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:03.911288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.571205+0000","last_change":"2024-04-25T10:24:44.575358+0000","last_active":"2024-04-25T10:24:45.571205+0000","last_peered":"2024-04-25T10:24:45.571205+0000","last_clean":"2024-04-25T10:24:45.571205+0000","last_became_active":"2024-04-25T10:24:44.574965+0000","last_became_peered":"2024-04-25T10:24:44.574965+0000","last_unstale":"2024-04-25T10:24:45.571205+0000","last_undegraded":"2024-04-25T10:24:45.571205+0000","last_fullsized":"2024-04-25T10:24:45.571205+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:57.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":428,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.570117+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:24:45.570117+0000","last_peered":"2024-04-25T10:24:45.570117+0000","last_clean":"2024-04-25T10:24:45.570117+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:24:45.570117+0000","last_undegraded":"2024-04-25T10:24:45.570117+0000","last_fullsized":"2024-04-25T10:24:45.570117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60132,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311708,"statfs":{"total":96636764160,"available":96575188992,"internally_reserved":0,"allocated":2641920,"data_stored":1484462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65052,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306788,"statfs":{"total":96636764160,"available":96570150912,"internally_reserved":0,"allocated":2039808,"data_stored":893990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":2039808,"data_stored":893990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59560,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312280,"statfs":{"total":96636764160,"available":96575774720,"internally_reserved":0,"allocated":2056192,"data_stored":896184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2052096,"data_stored":894184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55372,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316468,"statfs":{"total":96636764160,"available":96580063232,"internally_reserved":0,"allocated":2027520,"data_stored":887215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65124,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306716,"statfs":{"total":96636764160,"available":96570077184,"internally_reserved":0,"allocated":2052096,"data_stored":895602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55972,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315868,"statfs":{"total":96636764160,"available":96579448832,"internally_reserved":0,"allocated":2637824,"data_stored":1484380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:24:49.650 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:24:49.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f40800640 1 Processor -- start 2024-04-25T10:24:49.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f40800640 1 -- start start 2024-04-25T10:24:49.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f40800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3806ca50 0x7f7f380cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f40800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f38070310 0x7f7f380707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f40800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f380b4470 con 0x7f7f380cf930 2024-04-25T10:24:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f40800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f380cf4d0 con 0x7f7f38070310 2024-04-25T10:24:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f40800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b4cd0 con 0x7f7f3806ca50 2024-04-25T10:24:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f38070310 0x7f7f380707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3de00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7f380cf930 0x7f7f380cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60170/0 (socket says 172.21.15.87:60170) 2024-04-25T10:24:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3de00640 1 -- 172.21.15.87:0/1450912725 learned_addr learned my addr 172.21.15.87:0/1450912725 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3d400640 1 --2- 172.21.15.87:0/1450912725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3806ca50 0x7f7f380cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3e800640 1 -- 172.21.15.87:0/1450912725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3806ca50 msgr2=0x7f7f380cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3e800640 1 --2- 172.21.15.87:0/1450912725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3806ca50 0x7f7f380cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3e800640 1 -- 172.21.15.87:0/1450912725 >> v1:172.21.15.87:6789/0 conn(0x7f7f380cf930 legacy=0x7f7f380cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.820+0000 7f7f3e800640 1 -- 172.21.15.87:0/1450912725 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380b4710 con 0x7f7f38070310 2024-04-25T10:24:49.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f3e800640 1 --2- 172.21.15.87:0/1450912725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f38070310 0x7f7f380707a0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009dc0 tx=0x7f7f3401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86e428abbef278d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f3ca00640 1 -- 172.21.15.87:0/1450912725 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f340b2070 con 0x7f7f38070310 2024-04-25T10:24:49.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f3ca00640 1 -- 172.21.15.87:0/1450912725 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f340a5870 con 0x7f7f38070310 2024-04-25T10:24:49.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f3ca00640 1 -- 172.21.15.87:0/1450912725 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f340a5c40 con 0x7f7f38070310 2024-04-25T10:24:49.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 -- 172.21.15.87:0/1450912725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f38070310 msgr2=0x7f7f380707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 --2- 172.21.15.87:0/1450912725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f38070310 0x7f7f380707a0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009dc0 tx=0x7f7f3401c8c0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 -- 172.21.15.87:0/1450912725 shutdown_connections 2024-04-25T10:24:49.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 --2- 172.21.15.87:0/1450912725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f38070310 0x7f7f380707a0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 --2- 172.21.15.87:0/1450912725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f3806ca50 0x7f7f380cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 -- 172.21.15.87:0/1450912725 >> 172.21.15.87:0/1450912725 conn(0x7f7f38066b70 msgr2=0x7f7f38066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 -- 172.21.15.87:0/1450912725 shutdown_connections 2024-04-25T10:24:49.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.824+0000 7f7f40800640 1 -- 172.21.15.87:0/1450912725 wait complete. 2024-04-25T10:24:49.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 Processor -- start 2024-04-25T10:24:49.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 -- start start 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3806ca50 0x7f7f38157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f38070310 0x7f7f3815c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f380cf930 0x7f7f3815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38071fc0 con 0x7f7f3806ca50 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38071e40 con 0x7f7f380cf930 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b2e30 con 0x7f7f38070310 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3806ca50 0x7f7f38157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3806ca50 0x7f7f38157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43846/0 (socket says 172.21.15.87:43846) 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 -- 172.21.15.87:0/4073218182 learned_addr learned my addr 172.21.15.87:0/4073218182 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:49.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3d400640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f38070310 0x7f7f3815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3e800640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f380cf930 0x7f7f3815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 -- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f38070310 msgr2=0x7f7f3815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f38070310 0x7f7f3815c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 -- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f380cf930 msgr2=0x7f7f3815ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f380cf930 0x7f7f3815ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 -- 172.21.15.87:0/4073218182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38162380 con 0x7f7f3806ca50 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3d400640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f38070310 0x7f7f3815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f3de00640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3806ca50 0x7f7f38157c30 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00c3a0 tx=0x7f7f2c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f32000640 1 -- 172.21.15.87:0/4073218182 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c017070 con 0x7f7f3806ca50 2024-04-25T10:24:49.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f32000640 1 -- 172.21.15.87:0/4073218182 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c005cf0 con 0x7f7f3806ca50 2024-04-25T10:24:49.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f32000640 1 -- 172.21.15.87:0/4073218182 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0072f0 con 0x7f7f3806ca50 2024-04-25T10:24:49.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f38158170 con 0x7f7f3806ca50 2024-04-25T10:24:49.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.828+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f381601c0 con 0x7f7f3806ca50 2024-04-25T10:24:49.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.832+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3804ef30 con 0x7f7f3806ca50 2024-04-25T10:24:49.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.832+0000 7f7f32000640 1 -- 172.21.15.87:0/4073218182 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f2c01a070 con 0x7f7f3806ca50 2024-04-25T10:24:49.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.832+0000 7f7f32000640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f1c066f60 0x7f7f1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.832+0000 7f7f32000640 1 -- 172.21.15.87:0/4073218182 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f2c084320 con 0x7f7f3806ca50 2024-04-25T10:24:49.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.832+0000 7f7f3d400640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f1c066f60 0x7f7f1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.832+0000 7f7f3d400640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f1c066f60 0x7f7f1c069420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7f3815e2d0 tx=0x7f7f28008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.836+0000 7f7f32000640 1 -- 172.21.15.87:0/4073218182 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f2c054ce0 con 0x7f7f3806ca50 2024-04-25T10:24:49.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.988+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f380cda90 con 0x7f7f1c066f60 2024-04-25T10:24:49.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.992+0000 7f7f32000640 1 -- 172.21.15.87:0/4073218182 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252411 (secure 0 0 0) 0x7f7f380cda90 con 0x7f7f1c066f60 2024-04-25T10:24:49.995 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:24:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f1c066f60 msgr2=0x7f7f1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f1c066f60 0x7f7f1c069420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7f3815e2d0 tx=0x7f7f28008040 comp rx=0 tx=0).stop 2024-04-25T10:24:49.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3806ca50 msgr2=0x7f7f38157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3806ca50 0x7f7f38157c30 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00c3a0 tx=0x7f7f2c00c860 comp rx=0 tx=0).stop 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 shutdown_connections 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f1c066f60 0x7f7f1c069420 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f380cf930 0x7f7f3815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f38070310 0x7f7f3815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 --2- 172.21.15.87:0/4073218182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f3806ca50 0x7f7f38157c30 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 >> 172.21.15.87:0/4073218182 conn(0x7f7f38066b70 msgr2=0x7f7f3815ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 shutdown_connections 2024-04-25T10:24:49.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:49.996+0000 7f7f40800640 1 -- 172.21.15.87:0/4073218182 wait complete. 2024-04-25T10:24:50.016 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-04-25T10:24:49.282952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":471996,"kb_used_data":17136,"kb_used_omap":12,"kb_used_meta":454707,"kb_avail":754502724,"statfs":{"total":773094113280,"available":772610789376,"internally_reserved":0,"allocated":17547264,"data_stored":8330007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":465620529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.736346"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595886+0000","last_change":"2024-04-25T10:24:45.595191+0000","last_active":"2024-04-25T10:24:45.595886+0000","last_peered":"2024-04-25T10:24:45.595886+0000","last_clean":"2024-04-25T10:24:45.595886+0000","last_became_active":"2024-04-25T10:24:45.594880+0000","last_became_peered":"2024-04-25T10:24:45.594880+0000","last_unstale":"2024-04-25T10:24:45.595886+0000","last_undegraded":"2024-04-25T10:24:45.595886+0000","last_fullsized":"2024-04-25T10:24:45.595886+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:56.983176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587961+0000","last_change":"2024-04-25T10:24:45.585203+0000","last_active":"2024-04-25T10:24:45.587961+0000","last_peered":"2024-04-25T10:24:45.587961+0000","last_clean":"2024-04-25T10:24:45.587961+0000","last_became_active":"2024-04-25T10:24:45.584034+0000","last_became_peered":"2024-04-25T10:24:45.584034+0000","last_unstale":"2024-04-25T10:24:45.587961+0000","last_undegraded":"2024-04-25T10:24:45.587961+0000","last_fullsized":"2024-04-25T10:24:45.587961+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:56.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.567025+0000","last_change":"2024-04-25T10:24:45.563254+0000","last_active":"2024-04-25T10:24:45.567025+0000","last_peered":"2024-04-25T10:24:45.567025+0000","last_clean":"2024-04-25T10:24:45.567025+0000","last_became_active":"2024-04-25T10:24:45.562985+0000","last_became_peered":"2024-04-25T10:24:45.562985+0000","last_unstale":"2024-04-25T10:24:45.567025+0000","last_undegraded":"2024-04-25T10:24:45.567025+0000","last_fullsized":"2024-04-25T10:24:45.567025+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:03.831803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592028+0000","last_change":"2024-04-25T10:24:45.584879+0000","last_active":"2024-04-25T10:24:45.592028+0000","last_peered":"2024-04-25T10:24:45.592028+0000","last_clean":"2024-04-25T10:24:45.592028+0000","last_became_active":"2024-04-25T10:24:45.584609+0000","last_became_peered":"2024-04-25T10:24:45.584609+0000","last_unstale":"2024-04-25T10:24:45.592028+0000","last_undegraded":"2024-04-25T10:24:45.592028+0000","last_fullsized":"2024-04-25T10:24:45.592028+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.163732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.569338+0000","last_change":"2024-04-25T10:24:45.566024+0000","last_active":"2024-04-25T10:24:45.569338+0000","last_peered":"2024-04-25T10:24:45.569338+0000","last_clean":"2024-04-25T10:24:45.569338+0000","last_became_active":"2024-04-25T10:24:45.565755+0000","last_became_peered":"2024-04-25T10:24:45.565755+0000","last_unstale":"2024-04-25T10:24:45.569338+0000","last_undegraded":"2024-04-25T10:24:45.569338+0000","last_fullsized":"2024-04-25T10:24:45.569338+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:08.707430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.582549+0000","last_change":"2024-04-25T10:24:45.581329+0000","last_active":"2024-04-25T10:24:45.582549+0000","last_peered":"2024-04-25T10:24:45.582549+0000","last_clean":"2024-04-25T10:24:45.582549+0000","last_became_active":"2024-04-25T10:24:45.581084+0000","last_became_peered":"2024-04-25T10:24:45.581084+0000","last_unstale":"2024-04-25T10:24:45.582549+0000","last_undegraded":"2024-04-25T10:24:45.582549+0000","last_fullsized":"2024-04-25T10:24:45.582549+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.163732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.581397+0000","last_change":"2024-04-25T10:24:45.579956+0000","last_active":"2024-04-25T10:24:45.581397+0000","last_peered":"2024-04-25T10:24:45.581397+0000","last_clean":"2024-04-25T10:24:45.581397+0000","last_became_active":"2024-04-25T10:24:45.579671+0000","last_became_peered":"2024-04-25T10:24:45.579671+0000","last_unstale":"2024-04-25T10:24:45.581397+0000","last_undegraded":"2024-04-25T10:24:45.581397+0000","last_fullsized":"2024-04-25T10:24:45.581397+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.781823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.572906+0000","last_change":"2024-04-25T10:24:45.567486+0000","last_active":"2024-04-25T10:24:45.572906+0000","last_peered":"2024-04-25T10:24:45.572906+0000","last_clean":"2024-04-25T10:24:45.572906+0000","last_became_active":"2024-04-25T10:24:45.567000+0000","last_became_peered":"2024-04-25T10:24:45.567000+0000","last_unstale":"2024-04-25T10:24:45.572906+0000","last_undegraded":"2024-04-25T10:24:45.572906+0000","last_fullsized":"2024-04-25T10:24:45.572906+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:31.742923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.582458+0000","last_change":"2024-04-25T10:24:45.581610+0000","last_active":"2024-04-25T10:24:45.582458+0000","last_peered":"2024-04-25T10:24:45.582458+0000","last_clean":"2024-04-25T10:24:45.582458+0000","last_became_active":"2024-04-25T10:24:45.581330+0000","last_became_peered":"2024-04-25T10:24:45.581330+0000","last_unstale":"2024-04-25T10:24:45.582458+0000","last_undegraded":"2024-04-25T10:24:45.582458+0000","last_fullsized":"2024-04-25T10:24:45.582458+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:58.139988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595137+0000","last_change":"2024-04-25T10:24:45.588483+0000","last_active":"2024-04-25T10:24:45.595137+0000","last_peered":"2024-04-25T10:24:45.595137+0000","last_clean":"2024-04-25T10:24:45.595137+0000","last_became_active":"2024-04-25T10:24:45.586922+0000","last_became_peered":"2024-04-25T10:24:45.586922+0000","last_unstale":"2024-04-25T10:24:45.595137+0000","last_undegraded":"2024-04-25T10:24:45.595137+0000","last_fullsized":"2024-04-25T10:24:45.595137+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:15.030738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.580650+0000","last_change":"2024-04-25T10:24:45.579477+0000","last_active":"2024-04-25T10:24:45.580650+0000","last_peered":"2024-04-25T10:24:45.580650+0000","last_clean":"2024-04-25T10:24:45.580650+0000","last_became_active":"2024-04-25T10:24:45.578694+0000","last_became_peered":"2024-04-25T10:24:45.578694+0000","last_unstale":"2024-04-25T10:24:45.580650+0000","last_undegraded":"2024-04-25T10:24:45.580650+0000","last_fullsized":"2024-04-25T10:24:45.580650+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.525963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591998+0000","last_change":"2024-04-25T10:24:45.585138+0000","last_active":"2024-04-25T10:24:45.591998+0000","last_peered":"2024-04-25T10:24:45.591998+0000","last_clean":"2024-04-25T10:24:45.591998+0000","last_became_active":"2024-04-25T10:24:45.583029+0000","last_became_peered":"2024-04-25T10:24:45.583029+0000","last_unstale":"2024-04-25T10:24:45.591998+0000","last_undegraded":"2024-04-25T10:24:45.591998+0000","last_fullsized":"2024-04-25T10:24:45.591998+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.942544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587151+0000","last_change":"2024-04-25T10:24:45.582245+0000","last_active":"2024-04-25T10:24:45.587151+0000","last_peered":"2024-04-25T10:24:45.587151+0000","last_clean":"2024-04-25T10:24:45.587151+0000","last_became_active":"2024-04-25T10:24:45.581888+0000","last_became_peered":"2024-04-25T10:24:45.581888+0000","last_unstale":"2024-04-25T10:24:45.587151+0000","last_undegraded":"2024-04-25T10:24:45.587151+0000","last_fullsized":"2024-04-25T10:24:45.587151+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:23.304199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.590820+0000","last_change":"2024-04-25T10:24:45.589448+0000","last_active":"2024-04-25T10:24:45.590820+0000","last_peered":"2024-04-25T10:24:45.590820+0000","last_clean":"2024-04-25T10:24:45.590820+0000","last_became_active":"2024-04-25T10:24:45.589173+0000","last_became_peered":"2024-04-25T10:24:45.589173+0000","last_unstale":"2024-04-25T10:24:45.590820+0000","last_undegraded":"2024-04-25T10:24:45.590820+0000","last_fullsized":"2024-04-25T10:24:45.590820+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:22.703038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587096+0000","last_change":"2024-04-25T10:24:45.581893+0000","last_active":"2024-04-25T10:24:45.587096+0000","last_peered":"2024-04-25T10:24:45.587096+0000","last_clean":"2024-04-25T10:24:45.587096+0000","last_became_active":"2024-04-25T10:24:45.581556+0000","last_became_peered":"2024-04-25T10:24:45.581556+0000","last_unstale":"2024-04-25T10:24:45.587096+0000","last_undegraded":"2024-04-25T10:24:45.587096+0000","last_fullsized":"2024-04-25T10:24:45.587096+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:03.831803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.572887+0000","last_change":"2024-04-25T10:24:45.567278+0000","last_active":"2024-04-25T10:24:45.572887+0000","last_peered":"2024-04-25T10:24:45.572887+0000","last_clean":"2024-04-25T10:24:45.572887+0000","last_became_active":"2024-04-25T10:24:45.566600+0000","last_became_peered":"2024-04-25T10:24:45.566600+0000","last_unstale":"2024-04-25T10:24:45.572887+0000","last_undegraded":"2024-04-25T10:24:45.572887+0000","last_fullsized":"2024-04-25T10:24:45.572887+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:57.922135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.588963+0000","last_change":"2024-04-25T10:24:45.585721+0000","last_active":"2024-04-25T10:24:45.588963+0000","last_peered":"2024-04-25T10:24:45.588963+0000","last_clean":"2024-04-25T10:24:45.588963+0000","last_became_active":"2024-04-25T10:24:45.584409+0000","last_became_peered":"2024-04-25T10:24:45.584409+0000","last_unstale":"2024-04-25T10:24:45.588963+0000","last_undegraded":"2024-04-25T10:24:45.588963+0000","last_fullsized":"2024-04-25T10:24:45.588963+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:27.184670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.590230+0000","last_change":"2024-04-25T10:24:45.586705+0000","last_active":"2024-04-25T10:24:45.590230+0000","last_peered":"2024-04-25T10:24:45.590230+0000","last_clean":"2024-04-25T10:24:45.590230+0000","last_became_active":"2024-04-25T10:24:45.586412+0000","last_became_peered":"2024-04-25T10:24:45.586412+0000","last_unstale":"2024-04-25T10:24:45.590230+0000","last_undegraded":"2024-04-25T10:24:45.590230+0000","last_fullsized":"2024-04-25T10:24:45.590230+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:23.489417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595366+0000","last_change":"2024-04-25T10:24:45.589116+0000","last_active":"2024-04-25T10:24:45.595366+0000","last_peered":"2024-04-25T10:24:45.595366+0000","last_clean":"2024-04-25T10:24:45.595366+0000","last_became_active":"2024-04-25T10:24:45.588859+0000","last_became_peered":"2024-04-25T10:24:45.588859+0000","last_unstale":"2024-04-25T10:24:45.595366+0000","last_undegraded":"2024-04-25T10:24:45.595366+0000","last_fullsized":"2024-04-25T10:24:45.595366+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:59.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593225+0000","last_change":"2024-04-25T10:24:45.587679+0000","last_active":"2024-04-25T10:24:45.593225+0000","last_peered":"2024-04-25T10:24:45.593225+0000","last_clean":"2024-04-25T10:24:45.593225+0000","last_became_active":"2024-04-25T10:24:45.587398+0000","last_became_peered":"2024-04-25T10:24:45.587398+0000","last_unstale":"2024-04-25T10:24:45.593225+0000","last_undegraded":"2024-04-25T10:24:45.593225+0000","last_fullsized":"2024-04-25T10:24:45.593225+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:45.688169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.574474+0000","last_change":"2024-04-25T10:24:45.569441+0000","last_active":"2024-04-25T10:24:45.574474+0000","last_peered":"2024-04-25T10:24:45.574474+0000","last_clean":"2024-04-25T10:24:45.574474+0000","last_became_active":"2024-04-25T10:24:45.569177+0000","last_became_peered":"2024-04-25T10:24:45.569177+0000","last_unstale":"2024-04-25T10:24:45.574474+0000","last_undegraded":"2024-04-25T10:24:45.574474+0000","last_fullsized":"2024-04-25T10:24:45.574474+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:10.529104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592202+0000","last_change":"2024-04-25T10:24:45.584119+0000","last_active":"2024-04-25T10:24:45.592202+0000","last_peered":"2024-04-25T10:24:45.592202+0000","last_clean":"2024-04-25T10:24:45.592202+0000","last_became_active":"2024-04-25T10:24:45.583261+0000","last_became_peered":"2024-04-25T10:24:45.583261+0000","last_unstale":"2024-04-25T10:24:45.592202+0000","last_undegraded":"2024-04-25T10:24:45.592202+0000","last_fullsized":"2024-04-25T10:24:45.592202+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.754615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.579589+0000","last_change":"2024-04-25T10:24:45.578681+0000","last_active":"2024-04-25T10:24:45.579589+0000","last_peered":"2024-04-25T10:24:45.579589+0000","last_clean":"2024-04-25T10:24:45.579589+0000","last_became_active":"2024-04-25T10:24:45.578429+0000","last_became_peered":"2024-04-25T10:24:45.578429+0000","last_unstale":"2024-04-25T10:24:45.579589+0000","last_undegraded":"2024-04-25T10:24:45.579589+0000","last_fullsized":"2024-04-25T10:24:45.579589+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.221137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.572866+0000","last_change":"2024-04-25T10:24:45.565377+0000","last_active":"2024-04-25T10:24:45.572866+0000","last_peered":"2024-04-25T10:24:45.572866+0000","last_clean":"2024-04-25T10:24:45.572866+0000","last_became_active":"2024-04-25T10:24:45.565133+0000","last_became_peered":"2024-04-25T10:24:45.565133+0000","last_unstale":"2024-04-25T10:24:45.572866+0000","last_undegraded":"2024-04-25T10:24:45.572866+0000","last_fullsized":"2024-04-25T10:24:45.572866+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:29.444440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593665+0000","last_change":"2024-04-25T10:24:45.591120+0000","last_active":"2024-04-25T10:24:45.593665+0000","last_peered":"2024-04-25T10:24:45.593665+0000","last_clean":"2024-04-25T10:24:45.593665+0000","last_became_active":"2024-04-25T10:24:45.590785+0000","last_became_peered":"2024-04-25T10:24:45.590785+0000","last_unstale":"2024-04-25T10:24:45.593665+0000","last_undegraded":"2024-04-25T10:24:45.593665+0000","last_fullsized":"2024-04-25T10:24:45.593665+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:52.781823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595552+0000","last_change":"2024-04-25T10:24:45.594686+0000","last_active":"2024-04-25T10:24:45.595552+0000","last_peered":"2024-04-25T10:24:45.595552+0000","last_clean":"2024-04-25T10:24:45.595552+0000","last_became_active":"2024-04-25T10:24:45.594273+0000","last_became_peered":"2024-04-25T10:24:45.594273+0000","last_unstale":"2024-04-25T10:24:45.595552+0000","last_undegraded":"2024-04-25T10:24:45.595552+0000","last_fullsized":"2024-04-25T10:24:45.595552+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:31.184710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.584473+0000","last_change":"2024-04-25T10:24:45.581845+0000","last_active":"2024-04-25T10:24:45.584473+0000","last_peered":"2024-04-25T10:24:45.584473+0000","last_clean":"2024-04-25T10:24:45.584473+0000","last_became_active":"2024-04-25T10:24:45.581593+0000","last_became_peered":"2024-04-25T10:24:45.581593+0000","last_unstale":"2024-04-25T10:24:45.584473+0000","last_undegraded":"2024-04-25T10:24:45.584473+0000","last_fullsized":"2024-04-25T10:24:45.584473+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.942544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587800+0000","last_change":"2024-04-25T10:24:45.583629+0000","last_active":"2024-04-25T10:24:45.587800+0000","last_peered":"2024-04-25T10:24:45.587800+0000","last_clean":"2024-04-25T10:24:45.587800+0000","last_became_active":"2024-04-25T10:24:45.583315+0000","last_became_peered":"2024-04-25T10:24:45.583315+0000","last_unstale":"2024-04-25T10:24:45.587800+0000","last_undegraded":"2024-04-25T10:24:45.587800+0000","last_fullsized":"2024-04-25T10:24:45.587800+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:04.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587049+0000","last_change":"2024-04-25T10:24:45.581610+0000","last_active":"2024-04-25T10:24:45.587049+0000","last_peered":"2024-04-25T10:24:45.587049+0000","last_clean":"2024-04-25T10:24:45.587049+0000","last_became_active":"2024-04-25T10:24:45.581309+0000","last_became_peered":"2024-04-25T10:24:45.581309+0000","last_unstale":"2024-04-25T10:24:45.587049+0000","last_undegraded":"2024-04-25T10:24:45.587049+0000","last_fullsized":"2024-04-25T10:24:45.587049+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:46.844848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.585940+0000","last_change":"2024-04-25T10:24:45.583765+0000","last_active":"2024-04-25T10:24:45.585940+0000","last_peered":"2024-04-25T10:24:45.585940+0000","last_clean":"2024-04-25T10:24:45.585940+0000","last_became_active":"2024-04-25T10:24:45.583210+0000","last_became_peered":"2024-04-25T10:24:45.583210+0000","last_unstale":"2024-04-25T10:24:45.585940+0000","last_undegraded":"2024-04-25T10:24:45.585940+0000","last_fullsized":"2024-04-25T10:24:45.585940+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.081500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592258+0000","last_change":"2024-04-25T10:24:45.586141+0000","last_active":"2024-04-25T10:24:45.592258+0000","last_peered":"2024-04-25T10:24:45.592258+0000","last_clean":"2024-04-25T10:24:45.592258+0000","last_became_active":"2024-04-25T10:24:45.584975+0000","last_became_peered":"2024-04-25T10:24:45.584975+0000","last_unstale":"2024-04-25T10:24:45.592258+0000","last_undegraded":"2024-04-25T10:24:45.592258+0000","last_fullsized":"2024-04-25T10:24:45.592258+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:56.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.573248+0000","last_change":"2024-04-25T10:24:45.568815+0000","last_active":"2024-04-25T10:24:45.573248+0000","last_peered":"2024-04-25T10:24:45.573248+0000","last_clean":"2024-04-25T10:24:45.573248+0000","last_became_active":"2024-04-25T10:24:45.568574+0000","last_became_peered":"2024-04-25T10:24:45.568574+0000","last_unstale":"2024-04-25T10:24:45.573248+0000","last_undegraded":"2024-04-25T10:24:45.573248+0000","last_fullsized":"2024-04-25T10:24:45.573248+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:05.274064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578381+0000","last_change":"2024-04-25T10:24:45.569683+0000","last_active":"2024-04-25T10:24:45.578381+0000","last_peered":"2024-04-25T10:24:45.578381+0000","last_clean":"2024-04-25T10:24:45.578381+0000","last_became_active":"2024-04-25T10:24:45.569449+0000","last_became_peered":"2024-04-25T10:24:45.569449+0000","last_unstale":"2024-04-25T10:24:45.578381+0000","last_undegraded":"2024-04-25T10:24:45.578381+0000","last_fullsized":"2024-04-25T10:24:45.578381+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:36.679989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"162'4","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587440+0000","last_change":"2024-04-25T10:24:44.578368+0000","last_active":"2024-04-25T10:24:45.587440+0000","last_peered":"2024-04-25T10:24:45.587440+0000","last_clean":"2024-04-25T10:24:45.587440+0000","last_became_active":"2024-04-25T10:24:44.578082+0000","last_became_peered":"2024-04-25T10:24:44.578082+0000","last_unstale":"2024-04-25T10:24:45.587440+0000","last_undegraded":"2024-04-25T10:24:45.587440+0000","last_fullsized":"2024-04-25T10:24:45.587440+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:49.345469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595962+0000","last_change":"2024-04-25T10:24:45.594760+0000","last_active":"2024-04-25T10:24:45.595962+0000","last_peered":"2024-04-25T10:24:45.595962+0000","last_clean":"2024-04-25T10:24:45.595962+0000","last_became_active":"2024-04-25T10:24:45.592775+0000","last_became_peered":"2024-04-25T10:24:45.592775+0000","last_unstale":"2024-04-25T10:24:45.595962+0000","last_undegraded":"2024-04-25T10:24:45.595962+0000","last_fullsized":"2024-04-25T10:24:45.595962+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:57.445586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591323+0000","last_change":"2024-04-25T10:24:44.570037+0000","last_active":"2024-04-25T10:24:45.591323+0000","last_peered":"2024-04-25T10:24:45.591323+0000","last_clean":"2024-04-25T10:24:45.591323+0000","last_became_active":"2024-04-25T10:24:44.568812+0000","last_became_peered":"2024-04-25T10:24:44.568812+0000","last_unstale":"2024-04-25T10:24:45.591323+0000","last_undegraded":"2024-04-25T10:24:45.591323+0000","last_fullsized":"2024-04-25T10:24:45.591323+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:11.805224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.574927+0000","last_change":"2024-04-25T10:24:45.573319+0000","last_active":"2024-04-25T10:24:45.574927+0000","last_peered":"2024-04-25T10:24:45.574927+0000","last_clean":"2024-04-25T10:24:45.574927+0000","last_became_active":"2024-04-25T10:24:45.572780+0000","last_became_peered":"2024-04-25T10:24:45.572780+0000","last_unstale":"2024-04-25T10:24:45.574927+0000","last_undegraded":"2024-04-25T10:24:45.574927+0000","last_fullsized":"2024-04-25T10:24:45.574927+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.163732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591052+0000","last_change":"2024-04-25T10:24:44.581954+0000","last_active":"2024-04-25T10:24:45.591052+0000","last_peered":"2024-04-25T10:24:45.591052+0000","last_clean":"2024-04-25T10:24:45.591052+0000","last_became_active":"2024-04-25T10:24:44.580891+0000","last_became_peered":"2024-04-25T10:24:44.580891+0000","last_unstale":"2024-04-25T10:24:45.591052+0000","last_undegraded":"2024-04-25T10:24:45.591052+0000","last_fullsized":"2024-04-25T10:24:45.591052+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:44.016323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.570497+0000","last_change":"2024-04-25T10:24:45.568432+0000","last_active":"2024-04-25T10:24:45.570497+0000","last_peered":"2024-04-25T10:24:45.570497+0000","last_clean":"2024-04-25T10:24:45.570497+0000","last_became_active":"2024-04-25T10:24:45.568091+0000","last_became_peered":"2024-04-25T10:24:45.568091+0000","last_unstale":"2024-04-25T10:24:45.570497+0000","last_undegraded":"2024-04-25T10:24:45.570497+0000","last_fullsized":"2024-04-25T10:24:45.570497+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:05.864369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593737+0000","last_change":"2024-04-25T10:24:44.574923+0000","last_active":"2024-04-25T10:24:45.593737+0000","last_peered":"2024-04-25T10:24:45.593737+0000","last_clean":"2024-04-25T10:24:45.593737+0000","last_became_active":"2024-04-25T10:24:44.573640+0000","last_became_peered":"2024-04-25T10:24:44.573640+0000","last_unstale":"2024-04-25T10:24:45.593737+0000","last_undegraded":"2024-04-25T10:24:45.593737+0000","last_fullsized":"2024-04-25T10:24:45.593737+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:36.842473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592181+0000","last_change":"2024-04-25T10:24:45.583940+0000","last_active":"2024-04-25T10:24:45.592181+0000","last_peered":"2024-04-25T10:24:45.592181+0000","last_clean":"2024-04-25T10:24:45.592181+0000","last_became_active":"2024-04-25T10:24:45.582839+0000","last_became_peered":"2024-04-25T10:24:45.582839+0000","last_unstale":"2024-04-25T10:24:45.592181+0000","last_undegraded":"2024-04-25T10:24:45.592181+0000","last_fullsized":"2024-04-25T10:24:45.592181+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:48.551078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592978+0000","last_change":"2024-04-25T10:24:44.578208+0000","last_active":"2024-04-25T10:24:45.592978+0000","last_peered":"2024-04-25T10:24:45.592978+0000","last_clean":"2024-04-25T10:24:45.592978+0000","last_became_active":"2024-04-25T10:24:44.577944+0000","last_became_peered":"2024-04-25T10:24:44.577944+0000","last_unstale":"2024-04-25T10:24:45.592978+0000","last_undegraded":"2024-04-25T10:24:45.592978+0000","last_fullsized":"2024-04-25T10:24:45.592978+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:41.148190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.580020+0000","last_change":"2024-04-25T10:24:45.572122+0000","last_active":"2024-04-25T10:24:45.580020+0000","last_peered":"2024-04-25T10:24:45.580020+0000","last_clean":"2024-04-25T10:24:45.580020+0000","last_became_active":"2024-04-25T10:24:45.571770+0000","last_became_peered":"2024-04-25T10:24:45.571770+0000","last_unstale":"2024-04-25T10:24:45.580020+0000","last_undegraded":"2024-04-25T10:24:45.580020+0000","last_fullsized":"2024-04-25T10:24:45.580020+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:36.377896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592304+0000","last_change":"2024-04-25T10:24:44.563909+0000","last_active":"2024-04-25T10:24:45.592304+0000","last_peered":"2024-04-25T10:24:45.592304+0000","last_clean":"2024-04-25T10:24:45.592304+0000","last_became_active":"2024-04-25T10:24:44.563612+0000","last_became_peered":"2024-04-25T10:24:44.563612+0000","last_unstale":"2024-04-25T10:24:45.592304+0000","last_undegraded":"2024-04-25T10:24:45.592304+0000","last_fullsized":"2024-04-25T10:24:45.592304+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:46.867085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578652+0000","last_change":"2024-04-25T10:24:45.575992+0000","last_active":"2024-04-25T10:24:45.578652+0000","last_peered":"2024-04-25T10:24:45.578652+0000","last_clean":"2024-04-25T10:24:45.578652+0000","last_became_active":"2024-04-25T10:24:45.575723+0000","last_became_peered":"2024-04-25T10:24:45.575723+0000","last_unstale":"2024-04-25T10:24:45.578652+0000","last_undegraded":"2024-04-25T10:24:45.578652+0000","last_fullsized":"2024-04-25T10:24:45.578652+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.942544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.569868+0000","last_change":"2024-04-25T10:24:44.578499+0000","last_active":"2024-04-25T10:24:45.569868+0000","last_peered":"2024-04-25T10:24:45.569868+0000","last_clean":"2024-04-25T10:24:45.569868+0000","last_became_active":"2024-04-25T10:24:44.578193+0000","last_became_peered":"2024-04-25T10:24:44.578193+0000","last_unstale":"2024-04-25T10:24:45.569868+0000","last_undegraded":"2024-04-25T10:24:45.569868+0000","last_fullsized":"2024-04-25T10:24:45.569868+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:41.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.568854+0000","last_change":"2024-04-25T10:24:45.567801+0000","last_active":"2024-04-25T10:24:45.568854+0000","last_peered":"2024-04-25T10:24:45.568854+0000","last_clean":"2024-04-25T10:24:45.568854+0000","last_became_active":"2024-04-25T10:24:45.567536+0000","last_became_peered":"2024-04-25T10:24:45.567536+0000","last_unstale":"2024-04-25T10:24:45.568854+0000","last_undegraded":"2024-04-25T10:24:45.568854+0000","last_fullsized":"2024-04-25T10:24:45.568854+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:23.304199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.563531+0000","last_change":"2024-04-25T10:24:44.571653+0000","last_active":"2024-04-25T10:24:45.563531+0000","last_peered":"2024-04-25T10:24:45.563531+0000","last_clean":"2024-04-25T10:24:45.563531+0000","last_became_active":"2024-04-25T10:24:44.571299+0000","last_became_peered":"2024-04-25T10:24:44.571299+0000","last_unstale":"2024-04-25T10:24:45.563531+0000","last_undegraded":"2024-04-25T10:24:45.563531+0000","last_fullsized":"2024-04-25T10:24:45.563531+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:02.825232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592017+0000","last_change":"2024-04-25T10:24:45.589729+0000","last_active":"2024-04-25T10:24:45.592017+0000","last_peered":"2024-04-25T10:24:45.592017+0000","last_clean":"2024-04-25T10:24:45.592017+0000","last_became_active":"2024-04-25T10:24:45.589058+0000","last_became_peered":"2024-04-25T10:24:45.589058+0000","last_unstale":"2024-04-25T10:24:45.592017+0000","last_undegraded":"2024-04-25T10:24:45.592017+0000","last_fullsized":"2024-04-25T10:24:45.592017+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.076168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.564815+0000","last_change":"2024-04-25T10:24:44.574709+0000","last_active":"2024-04-25T10:24:45.564815+0000","last_peered":"2024-04-25T10:24:45.564815+0000","last_clean":"2024-04-25T10:24:45.564815+0000","last_became_active":"2024-04-25T10:24:44.574448+0000","last_became_peered":"2024-04-25T10:24:44.574448+0000","last_unstale":"2024-04-25T10:24:45.564815+0000","last_undegraded":"2024-04-25T10:24:45.564815+0000","last_fullsized":"2024-04-25T10:24:45.564815+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:59.312219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.570831+0000","last_change":"2024-04-25T10:24:45.569706+0000","last_active":"2024-04-25T10:24:45.570831+0000","last_peered":"2024-04-25T10:24:45.570831+0000","last_clean":"2024-04-25T10:24:45.570831+0000","last_became_active":"2024-04-25T10:24:45.569374+0000","last_became_peered":"2024-04-25T10:24:45.569374+0000","last_unstale":"2024-04-25T10:24:45.570831+0000","last_undegraded":"2024-04-25T10:24:45.570831+0000","last_fullsized":"2024-04-25T10:24:45.570831+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.221137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"162'3","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.589235+0000","last_change":"2024-04-25T10:24:44.574732+0000","last_active":"2024-04-25T10:24:45.589235+0000","last_peered":"2024-04-25T10:24:45.589235+0000","last_clean":"2024-04-25T10:24:45.589235+0000","last_became_active":"2024-04-25T10:24:44.574465+0000","last_became_peered":"2024-04-25T10:24:44.574465+0000","last_unstale":"2024-04-25T10:24:45.589235+0000","last_undegraded":"2024-04-25T10:24:45.589235+0000","last_fullsized":"2024-04-25T10:24:45.589235+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:57.298082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595387+0000","last_change":"2024-04-25T10:24:45.589220+0000","last_active":"2024-04-25T10:24:45.595387+0000","last_peered":"2024-04-25T10:24:45.595387+0000","last_clean":"2024-04-25T10:24:45.595387+0000","last_became_active":"2024-04-25T10:24:45.588939+0000","last_became_peered":"2024-04-25T10:24:45.588939+0000","last_unstale":"2024-04-25T10:24:45.595387+0000","last_undegraded":"2024-04-25T10:24:45.595387+0000","last_fullsized":"2024-04-25T10:24:45.595387+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.873656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"162'6","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595470+0000","last_change":"2024-04-25T10:24:44.574592+0000","last_active":"2024-04-25T10:24:45.595470+0000","last_peered":"2024-04-25T10:24:45.595470+0000","last_clean":"2024-04-25T10:24:45.595470+0000","last_became_active":"2024-04-25T10:24:44.574095+0000","last_became_peered":"2024-04-25T10:24:44.574095+0000","last_unstale":"2024-04-25T10:24:45.595470+0000","last_undegraded":"2024-04-25T10:24:45.595470+0000","last_fullsized":"2024-04-25T10:24:45.595470+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:25.110558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592365+0000","last_change":"2024-04-25T10:24:45.591073+0000","last_active":"2024-04-25T10:24:45.592365+0000","last_peered":"2024-04-25T10:24:45.592365+0000","last_clean":"2024-04-25T10:24:45.592365+0000","last_became_active":"2024-04-25T10:24:45.590785+0000","last_became_peered":"2024-04-25T10:24:45.590785+0000","last_unstale":"2024-04-25T10:24:45.592365+0000","last_undegraded":"2024-04-25T10:24:45.592365+0000","last_fullsized":"2024-04-25T10:24:45.592365+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.525963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.596768+0000","last_change":"2024-04-25T10:24:44.578556+0000","last_active":"2024-04-25T10:24:45.596768+0000","last_peered":"2024-04-25T10:24:45.596768+0000","last_clean":"2024-04-25T10:24:45.596768+0000","last_became_active":"2024-04-25T10:24:44.578252+0000","last_became_peered":"2024-04-25T10:24:44.578252+0000","last_unstale":"2024-04-25T10:24:45.596768+0000","last_undegraded":"2024-04-25T10:24:45.596768+0000","last_fullsized":"2024-04-25T10:24:45.596768+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:32.350924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592774+0000","last_change":"2024-04-25T10:24:45.590443+0000","last_active":"2024-04-25T10:24:45.592774+0000","last_peered":"2024-04-25T10:24:45.592774+0000","last_clean":"2024-04-25T10:24:45.592774+0000","last_became_active":"2024-04-25T10:24:45.590192+0000","last_became_peered":"2024-04-25T10:24:45.590192+0000","last_unstale":"2024-04-25T10:24:45.592774+0000","last_undegraded":"2024-04-25T10:24:45.592774+0000","last_fullsized":"2024-04-25T10:24:45.592774+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:32.873972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.575588+0000","last_change":"2024-04-25T10:24:44.575626+0000","last_active":"2024-04-25T10:24:45.575588+0000","last_peered":"2024-04-25T10:24:45.575588+0000","last_clean":"2024-04-25T10:24:45.575588+0000","last_became_active":"2024-04-25T10:24:44.575337+0000","last_became_peered":"2024-04-25T10:24:44.575337+0000","last_unstale":"2024-04-25T10:24:45.575588+0000","last_undegraded":"2024-04-25T10:24:45.575588+0000","last_fullsized":"2024-04-25T10:24:45.575588+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:34.028064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593028+0000","last_change":"2024-04-25T10:24:45.591758+0000","last_active":"2024-04-25T10:24:45.593028+0000","last_peered":"2024-04-25T10:24:45.593028+0000","last_clean":"2024-04-25T10:24:45.593028+0000","last_became_active":"2024-04-25T10:24:45.591292+0000","last_became_peered":"2024-04-25T10:24:45.591292+0000","last_unstale":"2024-04-25T10:24:45.593028+0000","last_undegraded":"2024-04-25T10:24:45.593028+0000","last_fullsized":"2024-04-25T10:24:45.593028+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:12.736031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.561704+0000","last_change":"2024-04-25T10:24:44.573726+0000","last_active":"2024-04-25T10:24:45.561704+0000","last_peered":"2024-04-25T10:24:45.561704+0000","last_clean":"2024-04-25T10:24:45.561704+0000","last_became_active":"2024-04-25T10:24:44.573458+0000","last_became_peered":"2024-04-25T10:24:44.573458+0000","last_unstale":"2024-04-25T10:24:45.561704+0000","last_undegraded":"2024-04-25T10:24:45.561704+0000","last_fullsized":"2024-04-25T10:24:45.561704+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:43.165781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.589342+0000","last_change":"2024-04-25T10:24:45.578450+0000","last_active":"2024-04-25T10:24:45.589342+0000","last_peered":"2024-04-25T10:24:45.589342+0000","last_clean":"2024-04-25T10:24:45.589342+0000","last_became_active":"2024-04-25T10:24:45.578164+0000","last_became_peered":"2024-04-25T10:24:45.578164+0000","last_unstale":"2024-04-25T10:24:45.589342+0000","last_undegraded":"2024-04-25T10:24:45.589342+0000","last_fullsized":"2024-04-25T10:24:45.589342+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:27.233033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"162'8","reported_seq":20,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.596790+0000","last_change":"2024-04-25T10:24:44.578694+0000","last_active":"2024-04-25T10:24:45.596790+0000","last_peered":"2024-04-25T10:24:45.596790+0000","last_clean":"2024-04-25T10:24:45.596790+0000","last_became_active":"2024-04-25T10:24:44.578226+0000","last_became_peered":"2024-04-25T10:24:44.578226+0000","last_unstale":"2024-04-25T10:24:45.596790+0000","last_undegraded":"2024-04-25T10:24:45.596790+0000","last_fullsized":"2024-04-25T10:24:45.596790+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:35.371325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592651+0000","last_change":"2024-04-25T10:24:45.587097+0000","last_active":"2024-04-25T10:24:45.592651+0000","last_peered":"2024-04-25T10:24:45.592651+0000","last_clean":"2024-04-25T10:24:45.592651+0000","last_became_active":"2024-04-25T10:24:45.586780+0000","last_became_peered":"2024-04-25T10:24:45.586780+0000","last_unstale":"2024-04-25T10:24:45.592651+0000","last_undegraded":"2024-04-25T10:24:45.592651+0000","last_fullsized":"2024-04-25T10:24:45.592651+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:17.086213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.594005+0000","last_change":"2024-04-25T10:24:44.579142+0000","last_active":"2024-04-25T10:24:45.594005+0000","last_peered":"2024-04-25T10:24:45.594005+0000","last_clean":"2024-04-25T10:24:45.594005+0000","last_became_active":"2024-04-25T10:24:44.578800+0000","last_became_peered":"2024-04-25T10:24:44.578800+0000","last_unstale":"2024-04-25T10:24:45.594005+0000","last_undegraded":"2024-04-25T10:24:45.594005+0000","last_fullsized":"2024-04-25T10:24:45.594005+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:44.491722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593384+0000","last_change":"2024-04-25T10:24:45.592225+0000","last_active":"2024-04-25T10:24:45.593384+0000","last_peered":"2024-04-25T10:24:45.593384+0000","last_clean":"2024-04-25T10:24:45.593384+0000","last_became_active":"2024-04-25T10:24:45.591917+0000","last_became_peered":"2024-04-25T10:24:45.591917+0000","last_unstale":"2024-04-25T10:24:45.593384+0000","last_undegraded":"2024-04-25T10:24:45.593384+0000","last_fullsized":"2024-04-25T10:24:45.593384+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:26.839809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.588618+0000","last_change":"2024-04-25T10:24:44.571704+0000","last_active":"2024-04-25T10:24:45.588618+0000","last_peered":"2024-04-25T10:24:45.588618+0000","last_clean":"2024-04-25T10:24:45.588618+0000","last_became_active":"2024-04-25T10:24:44.571438+0000","last_became_peered":"2024-04-25T10:24:44.571438+0000","last_unstale":"2024-04-25T10:24:45.588618+0000","last_undegraded":"2024-04-25T10:24:45.588618+0000","last_fullsized":"2024-04-25T10:24:45.588618+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:36.468500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.585962+0000","last_change":"2024-04-25T10:24:45.583633+0000","last_active":"2024-04-25T10:24:45.585962+0000","last_peered":"2024-04-25T10:24:45.585962+0000","last_clean":"2024-04-25T10:24:45.585962+0000","last_became_active":"2024-04-25T10:24:45.583339+0000","last_became_peered":"2024-04-25T10:24:45.583339+0000","last_unstale":"2024-04-25T10:24:45.585962+0000","last_undegraded":"2024-04-25T10:24:45.585962+0000","last_fullsized":"2024-04-25T10:24:45.585962+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:13.138024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.596104+0000","last_change":"2024-04-25T10:24:44.579724+0000","last_active":"2024-04-25T10:24:45.596104+0000","last_peered":"2024-04-25T10:24:45.596104+0000","last_clean":"2024-04-25T10:24:45.596104+0000","last_became_active":"2024-04-25T10:24:44.579374+0000","last_became_peered":"2024-04-25T10:24:44.579374+0000","last_unstale":"2024-04-25T10:24:45.596104+0000","last_undegraded":"2024-04-25T10:24:45.596104+0000","last_fullsized":"2024-04-25T10:24:45.596104+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:53.227740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591398+0000","last_change":"2024-04-25T10:24:45.586876+0000","last_active":"2024-04-25T10:24:45.591398+0000","last_peered":"2024-04-25T10:24:45.591398+0000","last_clean":"2024-04-25T10:24:45.591398+0000","last_became_active":"2024-04-25T10:24:45.586236+0000","last_became_peered":"2024-04-25T10:24:45.586236+0000","last_unstale":"2024-04-25T10:24:45.591398+0000","last_undegraded":"2024-04-25T10:24:45.591398+0000","last_fullsized":"2024-04-25T10:24:45.591398+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.525963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.563752+0000","last_change":"2024-04-25T10:24:44.578917+0000","last_active":"2024-04-25T10:24:45.563752+0000","last_peered":"2024-04-25T10:24:45.563752+0000","last_clean":"2024-04-25T10:24:45.563752+0000","last_became_active":"2024-04-25T10:24:44.578565+0000","last_became_peered":"2024-04-25T10:24:44.578565+0000","last_unstale":"2024-04-25T10:24:45.563752+0000","last_undegraded":"2024-04-25T10:24:45.563752+0000","last_fullsized":"2024-04-25T10:24:45.563752+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:22.297627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.592221+0000","last_change":"2024-04-25T10:24:45.584304+0000","last_active":"2024-04-25T10:24:45.592221+0000","last_peered":"2024-04-25T10:24:45.592221+0000","last_clean":"2024-04-25T10:24:45.592221+0000","last_became_active":"2024-04-25T10:24:45.583638+0000","last_became_peered":"2024-04-25T10:24:45.583638+0000","last_unstale":"2024-04-25T10:24:45.592221+0000","last_undegraded":"2024-04-25T10:24:45.592221+0000","last_fullsized":"2024-04-25T10:24:45.592221+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:46.841052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.591880+0000","last_change":"2024-04-25T10:24:44.579944+0000","last_active":"2024-04-25T10:24:45.591880+0000","last_peered":"2024-04-25T10:24:45.591880+0000","last_clean":"2024-04-25T10:24:45.591880+0000","last_became_active":"2024-04-25T10:24:44.579691+0000","last_became_peered":"2024-04-25T10:24:44.579691+0000","last_unstale":"2024-04-25T10:24:45.591880+0000","last_undegraded":"2024-04-25T10:24:45.591880+0000","last_fullsized":"2024-04-25T10:24:45.591880+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:45.838277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578826+0000","last_change":"2024-04-25T10:24:45.577183+0000","last_active":"2024-04-25T10:24:45.578826+0000","last_peered":"2024-04-25T10:24:45.578826+0000","last_clean":"2024-04-25T10:24:45.578826+0000","last_became_active":"2024-04-25T10:24:45.575091+0000","last_became_peered":"2024-04-25T10:24:45.575091+0000","last_unstale":"2024-04-25T10:24:45.578826+0000","last_undegraded":"2024-04-25T10:24:45.578826+0000","last_fullsized":"2024-04-25T10:24:45.578826+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:56.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.562862+0000","last_change":"2024-04-25T10:24:44.573889+0000","last_active":"2024-04-25T10:24:45.562862+0000","last_peered":"2024-04-25T10:24:45.562862+0000","last_clean":"2024-04-25T10:24:45.562862+0000","last_became_active":"2024-04-25T10:24:44.572072+0000","last_became_peered":"2024-04-25T10:24:44.572072+0000","last_unstale":"2024-04-25T10:24:45.562862+0000","last_undegraded":"2024-04-25T10:24:45.562862+0000","last_fullsized":"2024-04-25T10:24:45.562862+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:43.137800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.583532+0000","last_change":"2024-04-25T10:24:45.579966+0000","last_active":"2024-04-25T10:24:45.583532+0000","last_peered":"2024-04-25T10:24:45.583532+0000","last_clean":"2024-04-25T10:24:45.583532+0000","last_became_active":"2024-04-25T10:24:45.579697+0000","last_became_peered":"2024-04-25T10:24:45.579697+0000","last_unstale":"2024-04-25T10:24:45.583532+0000","last_undegraded":"2024-04-25T10:24:45.583532+0000","last_fullsized":"2024-04-25T10:24:45.583532+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:45.022894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.558317+0000","last_change":"2024-04-25T10:24:44.579396+0000","last_active":"2024-04-25T10:24:45.558317+0000","last_peered":"2024-04-25T10:24:45.558317+0000","last_clean":"2024-04-25T10:24:45.558317+0000","last_became_active":"2024-04-25T10:24:44.579139+0000","last_became_peered":"2024-04-25T10:24:44.579139+0000","last_unstale":"2024-04-25T10:24:45.558317+0000","last_undegraded":"2024-04-25T10:24:45.558317+0000","last_fullsized":"2024-04-25T10:24:45.558317+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:36.842473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.573416+0000","last_change":"2024-04-25T10:24:45.568429+0000","last_active":"2024-04-25T10:24:45.573416+0000","last_peered":"2024-04-25T10:24:45.573416+0000","last_clean":"2024-04-25T10:24:45.573416+0000","last_became_active":"2024-04-25T10:24:45.568166+0000","last_became_peered":"2024-04-25T10:24:45.568166+0000","last_unstale":"2024-04-25T10:24:45.573416+0000","last_undegraded":"2024-04-25T10:24:45.573416+0000","last_fullsized":"2024-04-25T10:24:45.573416+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:46.957005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.562107+0000","last_change":"2024-04-25T10:24:44.572059+0000","last_active":"2024-04-25T10:24:45.562107+0000","last_peered":"2024-04-25T10:24:45.562107+0000","last_clean":"2024-04-25T10:24:45.562107+0000","last_became_active":"2024-04-25T10:24:44.571754+0000","last_became_peered":"2024-04-25T10:24:44.571754+0000","last_unstale":"2024-04-25T10:24:45.562107+0000","last_undegraded":"2024-04-25T10:24:45.562107+0000","last_fullsized":"2024-04-25T10:24:45.562107+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:46.241419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.584415+0000","last_change":"2024-04-25T10:24:45.570143+0000","last_active":"2024-04-25T10:24:45.584415+0000","last_peered":"2024-04-25T10:24:45.584415+0000","last_clean":"2024-04-25T10:24:45.584415+0000","last_became_active":"2024-04-25T10:24:45.569862+0000","last_became_peered":"2024-04-25T10:24:45.569862+0000","last_unstale":"2024-04-25T10:24:45.584415+0000","last_undegraded":"2024-04-25T10:24:45.584415+0000","last_fullsized":"2024-04-25T10:24:45.584415+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:22.827670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595166+0000","last_change":"2024-04-25T10:24:44.580541+0000","last_active":"2024-04-25T10:24:45.595166+0000","last_peered":"2024-04-25T10:24:45.595166+0000","last_clean":"2024-04-25T10:24:45.595166+0000","last_became_active":"2024-04-25T10:24:44.580192+0000","last_became_peered":"2024-04-25T10:24:44.580192+0000","last_unstale":"2024-04-25T10:24:45.595166+0000","last_undegraded":"2024-04-25T10:24:45.595166+0000","last_fullsized":"2024-04-25T10:24:45.595166+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:05.305075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.595054+0000","last_change":"2024-04-25T10:24:45.588764+0000","last_active":"2024-04-25T10:24:45.595054+0000","last_peered":"2024-04-25T10:24:45.595054+0000","last_clean":"2024-04-25T10:24:45.595054+0000","last_became_active":"2024-04-25T10:24:45.588299+0000","last_became_peered":"2024-04-25T10:24:45.588299+0000","last_unstale":"2024-04-25T10:24:45.595054+0000","last_undegraded":"2024-04-25T10:24:45.595054+0000","last_fullsized":"2024-04-25T10:24:45.595054+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:44.172352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.581395+0000","last_change":"2024-04-25T10:24:44.581302+0000","last_active":"2024-04-25T10:24:45.581395+0000","last_peered":"2024-04-25T10:24:45.581395+0000","last_clean":"2024-04-25T10:24:45.581395+0000","last_became_active":"2024-04-25T10:24:44.581055+0000","last_became_peered":"2024-04-25T10:24:44.581055+0000","last_unstale":"2024-04-25T10:24:45.581395+0000","last_undegraded":"2024-04-25T10:24:45.581395+0000","last_fullsized":"2024-04-25T10:24:45.581395+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:17.242801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.579667+0000","last_change":"2024-04-25T10:24:45.570736+0000","last_active":"2024-04-25T10:24:45.579667+0000","last_peered":"2024-04-25T10:24:45.579667+0000","last_clean":"2024-04-25T10:24:45.579667+0000","last_became_active":"2024-04-25T10:24:45.570479+0000","last_became_peered":"2024-04-25T10:24:45.570479+0000","last_unstale":"2024-04-25T10:24:45.579667+0000","last_undegraded":"2024-04-25T10:24:45.579667+0000","last_fullsized":"2024-04-25T10:24:45.579667+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:42.417735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.579745+0000","last_change":"2024-04-25T10:24:44.579996+0000","last_active":"2024-04-25T10:24:45.579745+0000","last_peered":"2024-04-25T10:24:45.579745+0000","last_clean":"2024-04-25T10:24:45.579745+0000","last_became_active":"2024-04-25T10:24:44.579721+0000","last_became_peered":"2024-04-25T10:24:44.579721+0000","last_unstale":"2024-04-25T10:24:45.579745+0000","last_undegraded":"2024-04-25T10:24:45.579745+0000","last_fullsized":"2024-04-25T10:24:45.579745+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:11.154380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.578532+0000","last_change":"2024-04-25T10:24:45.568731+0000","last_active":"2024-04-25T10:24:45.578532+0000","last_peered":"2024-04-25T10:24:45.578532+0000","last_clean":"2024-04-25T10:24:45.578532+0000","last_became_active":"2024-04-25T10:24:45.568484+0000","last_became_peered":"2024-04-25T10:24:45.568484+0000","last_unstale":"2024-04-25T10:24:45.578532+0000","last_undegraded":"2024-04-25T10:24:45.578532+0000","last_fullsized":"2024-04-25T10:24:45.578532+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:56.760510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.563225+0000","last_change":"2024-04-25T10:24:44.572962+0000","last_active":"2024-04-25T10:24:45.563225+0000","last_peered":"2024-04-25T10:24:45.563225+0000","last_clean":"2024-04-25T10:24:45.563225+0000","last_became_active":"2024-04-25T10:24:44.572615+0000","last_became_peered":"2024-04-25T10:24:44.572615+0000","last_unstale":"2024-04-25T10:24:45.563225+0000","last_undegraded":"2024-04-25T10:24:45.563225+0000","last_fullsized":"2024-04-25T10:24:45.563225+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:56.439015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.586964+0000","last_change":"2024-04-25T10:24:45.585182+0000","last_active":"2024-04-25T10:24:45.586964+0000","last_peered":"2024-04-25T10:24:45.586964+0000","last_clean":"2024-04-25T10:24:45.586964+0000","last_became_active":"2024-04-25T10:24:45.584861+0000","last_became_peered":"2024-04-25T10:24:45.584861+0000","last_unstale":"2024-04-25T10:24:45.586964+0000","last_undegraded":"2024-04-25T10:24:45.586964+0000","last_fullsized":"2024-04-25T10:24:45.586964+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:19.995860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.561736+0000","last_change":"2024-04-25T10:24:44.576166+0000","last_active":"2024-04-25T10:24:45.561736+0000","last_peered":"2024-04-25T10:24:45.561736+0000","last_clean":"2024-04-25T10:24:45.561736+0000","last_became_active":"2024-04-25T10:24:44.575777+0000","last_became_peered":"2024-04-25T10:24:44.575777+0000","last_unstale":"2024-04-25T10:24:45.561736+0000","last_undegraded":"2024-04-25T10:24:45.561736+0000","last_fullsized":"2024-04-25T10:24:45.561736+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:16.850314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.582319+0000","last_change":"2024-04-25T10:24:45.579532+0000","last_active":"2024-04-25T10:24:45.582319+0000","last_peered":"2024-04-25T10:24:45.582319+0000","last_clean":"2024-04-25T10:24:45.582319+0000","last_became_active":"2024-04-25T10:24:45.578966+0000","last_became_peered":"2024-04-25T10:24:45.578966+0000","last_unstale":"2024-04-25T10:24:45.582319+0000","last_undegraded":"2024-04-25T10:24:45.582319+0000","last_fullsized":"2024-04-25T10:24:45.582319+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:42.561938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.562110+0000","last_change":"2024-04-25T10:24:44.573498+0000","last_active":"2024-04-25T10:24:45.562110+0000","last_peered":"2024-04-25T10:24:45.562110+0000","last_clean":"2024-04-25T10:24:45.562110+0000","last_became_active":"2024-04-25T10:24:44.572908+0000","last_became_peered":"2024-04-25T10:24:44.572908+0000","last_unstale":"2024-04-25T10:24:45.562110+0000","last_undegraded":"2024-04-25T10:24:45.562110+0000","last_fullsized":"2024-04-25T10:24:45.562110+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:26.754531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.587224+0000","last_change":"2024-04-25T10:24:45.583396+0000","last_active":"2024-04-25T10:24:45.587224+0000","last_peered":"2024-04-25T10:24:45.587224+0000","last_clean":"2024-04-25T10:24:45.587224+0000","last_became_active":"2024-04-25T10:24:45.583082+0000","last_became_peered":"2024-04-25T10:24:45.583082+0000","last_unstale":"2024-04-25T10:24:45.587224+0000","last_undegraded":"2024-04-25T10:24:45.587224+0000","last_fullsized":"2024-04-25T10:24:45.587224+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.221137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.576743+0000","last_change":"2024-04-25T10:24:44.567927+0000","last_active":"2024-04-25T10:24:45.576743+0000","last_peered":"2024-04-25T10:24:45.576743+0000","last_clean":"2024-04-25T10:24:45.576743+0000","last_became_active":"2024-04-25T10:24:44.567656+0000","last_became_peered":"2024-04-25T10:24:44.567656+0000","last_unstale":"2024-04-25T10:24:45.576743+0000","last_undegraded":"2024-04-25T10:24:45.576743+0000","last_fullsized":"2024-04-25T10:24:45.576743+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:25.110558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.588850+0000","last_change":"2024-04-25T10:24:45.582612+0000","last_active":"2024-04-25T10:24:45.588850+0000","last_peered":"2024-04-25T10:24:45.588850+0000","last_clean":"2024-04-25T10:24:45.588850+0000","last_became_active":"2024-04-25T10:24:45.582373+0000","last_became_peered":"2024-04-25T10:24:45.582373+0000","last_unstale":"2024-04-25T10:24:45.588850+0000","last_undegraded":"2024-04-25T10:24:45.588850+0000","last_fullsized":"2024-04-25T10:24:45.588850+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:52.922635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.557904+0000","last_change":"2024-04-25T10:24:44.572730+0000","last_active":"2024-04-25T10:24:45.557904+0000","last_peered":"2024-04-25T10:24:45.557904+0000","last_clean":"2024-04-25T10:24:45.557904+0000","last_became_active":"2024-04-25T10:24:44.572457+0000","last_became_peered":"2024-04-25T10:24:44.572457+0000","last_unstale":"2024-04-25T10:24:45.557904+0000","last_undegraded":"2024-04-25T10:24:45.557904+0000","last_fullsized":"2024-04-25T10:24:45.557904+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:41.605683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.593097+0000","last_change":"2024-04-25T10:24:45.587826+0000","last_active":"2024-04-25T10:24:45.593097+0000","last_peered":"2024-04-25T10:24:45.593097+0000","last_clean":"2024-04-25T10:24:45.593097+0000","last_became_active":"2024-04-25T10:24:45.587554+0000","last_became_peered":"2024-04-25T10:24:45.587554+0000","last_unstale":"2024-04-25T10:24:45.593097+0000","last_undegraded":"2024-04-25T10:24:45.593097+0000","last_fullsized":"2024-04-25T10:24:45.593097+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:44.546345+0000","last_clean_scrub_stamp":"2024-04-25T10:24:44.546345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:03.911288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.571205+0000","last_change":"2024-04-25T10:24:44.575358+0000","last_active":"2024-04-25T10:24:45.571205+0000","last_peered":"2024-04-25T10:24:45.571205+0000","last_clean":"2024-04-25T10:24:45.571205+0000","last_became_active":"2024-04-25T10:24:44.574965+0000","last_became_peered":"2024-04-25T10:24:44.574965+0000","last_unstale":"2024-04-25T10:24:45.571205+0000","last_undegraded":"2024-04-25T10:24:45.571205+0000","last_fullsized":"2024-04-25T10:24:45.571205+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:43.539774+0000","last_clean_scrub_stamp":"2024-04-25T10:24:43.539774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:57.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":428,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-04-25T10:24:45.570117+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:24:45.570117+0000","last_peered":"2024-04-25T10:24:45.570117+0000","last_clean":"2024-04-25T10:24:45.570117+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:24:45.570117+0000","last_undegraded":"2024-04-25T10:24:45.570117+0000","last_fullsized":"2024-04-25T10:24:45.570117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60132,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311708,"statfs":{"total":96636764160,"available":96575188992,"internally_reserved":0,"allocated":2641920,"data_stored":1484462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65052,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306788,"statfs":{"total":96636764160,"available":96570150912,"internally_reserved":0,"allocated":2039808,"data_stored":893990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":2039808,"data_stored":893990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59560,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312280,"statfs":{"total":96636764160,"available":96575774720,"internally_reserved":0,"allocated":2056192,"data_stored":896184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59556,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312284,"statfs":{"total":96636764160,"available":96575778816,"internally_reserved":0,"allocated":2052096,"data_stored":894184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55372,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316468,"statfs":{"total":96636764160,"available":96580063232,"internally_reserved":0,"allocated":2027520,"data_stored":887215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65124,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306716,"statfs":{"total":96636764160,"available":96570077184,"internally_reserved":0,"allocated":2052096,"data_stored":895602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55972,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315868,"statfs":{"total":96636764160,"available":96579448832,"internally_reserved":0,"allocated":2637824,"data_stored":1484380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:24:50.020 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:24:50.020 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:24:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff194a00640 1 Processor -- start 2024-04-25T10:24:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff194a00640 1 -- start start 2024-04-25T10:24:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff194a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1901433b0 0x7ff1900b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff194a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff190143790 0x7ff19006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff194a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19015f350 con 0x7ff19006d900 2024-04-25T10:24:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff194a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900b2e70 con 0x7ff1901433b0 2024-04-25T10:24:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff194a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900b2860 con 0x7ff190143790 2024-04-25T10:24:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1901433b0 0x7ff1900b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1901433b0 0x7ff1900b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43850/0 (socket says 172.21.15.87:43850) 2024-04-25T10:24:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18e800640 1 -- 172.21.15.87:0/913448020 learned_addr learned my addr 172.21.15.87:0/913448020 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:50.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18de00640 1 --2- 172.21.15.87:0/913448020 >> v2:172.21.15.169:3300/0 conn(0x7ff190143790 0x7ff19006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18e800640 1 -- 172.21.15.87:0/913448020 >> v2:172.21.15.169:3300/0 conn(0x7ff190143790 msgr2=0x7ff19006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:50.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18e800640 1 --2- 172.21.15.87:0/913448020 >> v2:172.21.15.169:3300/0 conn(0x7ff190143790 0x7ff19006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18e800640 1 -- 172.21.15.87:0/913448020 >> v1:172.21.15.87:6789/0 conn(0x7ff19006d900 legacy=0x7ff19015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18e800640 1 -- 172.21.15.87:0/913448020 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19015f070 con 0x7ff1901433b0 2024-04-25T10:24:50.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18de00640 1 --2- 172.21.15.87:0/913448020 >> v2:172.21.15.169:3300/0 conn(0x7ff190143790 0x7ff19006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:50.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.204+0000 7ff18d400640 1 -- 172.21.15.87:0/913448020 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516284402 0 0) 0x7ff19015f350 con 0x7ff19006d900 2024-04-25T10:24:50.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff18e800640 1 --2- 172.21.15.87:0/913448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff1900b3300 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7ff19015e350 tx=0x7ff18001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849b1d0134b038da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff18d400640 1 -- 172.21.15.87:0/913448020 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800a5730 con 0x7ff1901433b0 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff18d400640 1 -- 172.21.15.87:0/913448020 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1800a58d0 con 0x7ff1901433b0 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff18d400640 1 -- 172.21.15.87:0/913448020 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800a5ca0 con 0x7ff1901433b0 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 -- 172.21.15.87:0/913448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 msgr2=0x7ff1900b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 --2- 172.21.15.87:0/913448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff1900b3300 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7ff19015e350 tx=0x7ff18001c960 comp rx=0 tx=0).stop 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 -- 172.21.15.87:0/913448020 shutdown_connections 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 --2- 172.21.15.87:0/913448020 >> v2:172.21.15.169:3300/0 conn(0x7ff190143790 0x7ff19006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 --2- 172.21.15.87:0/913448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff1900b3300 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 -- 172.21.15.87:0/913448020 >> 172.21.15.87:0/913448020 conn(0x7ff1900743a0 msgr2=0x7ff1900669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:50.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 -- 172.21.15.87:0/913448020 shutdown_connections 2024-04-25T10:24:50.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.208+0000 7ff194a00640 1 -- 172.21.15.87:0/913448020 wait complete. 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 Processor -- start 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 -- start start 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006d900 0x7ff1901599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff19013bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff190143790 0x7ff190163160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900cdf40 con 0x7ff1901433b0 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900cddc0 con 0x7ff190143790 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900ce0c0 con 0x7ff19006d900 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff19013bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff19013bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43864/0 (socket says 172.21.15.87:43864) 2024-04-25T10:24:50.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 -- 172.21.15.87:0/2467150787 learned_addr learned my addr 172.21.15.87:0/2467150787 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18f200640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff190143790 0x7ff190163160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18e800640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006d900 0x7ff1901599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 -- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006d900 msgr2=0x7ff1901599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006d900 0x7ff1901599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 -- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff190143790 msgr2=0x7ff190163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff190143790 0x7ff190163160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 -- 172.21.15.87:0/2467150787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190161350 con 0x7ff1901433b0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff18de00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff19013bd40 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7ff17800e840 tx=0x7ff17800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff186a00640 1 -- 172.21.15.87:0/2467150787 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178017070 con 0x7ff1901433b0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff186a00640 1 -- 172.21.15.87:0/2467150787 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17800c8d0 con 0x7ff1901433b0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff186a00640 1 -- 172.21.15.87:0/2467150787 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17800f040 con 0x7ff1901433b0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff190161630 con 0x7ff1901433b0 2024-04-25T10:24:50.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.212+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff190161bf0 con 0x7ff1901433b0 2024-04-25T10:24:50.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.216+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff190075120 con 0x7ff1901433b0 2024-04-25T10:24:50.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.216+0000 7ff186a00640 1 -- 172.21.15.87:0/2467150787 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff17801a070 con 0x7ff1901433b0 2024-04-25T10:24:50.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.216+0000 7ff186a00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff16c066f60 0x7ff16c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.216+0000 7ff186a00640 1 -- 172.21.15.87:0/2467150787 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff178084c70 con 0x7ff1901433b0 2024-04-25T10:24:50.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.216+0000 7ff18e800640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff16c066f60 0x7ff16c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.216+0000 7ff18e800640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff16c066f60 0x7ff16c069420 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff1800ae000 tx=0x7ff1800a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.220+0000 7ff186a00640 1 -- 172.21.15.87:0/2467150787 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff178055be0 con 0x7ff1901433b0 2024-04-25T10:24:50.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.428+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff1900b3d40 con 0x7ff1901433b0 2024-04-25T10:24:50.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.428+0000 7ff186a00640 1 -- 172.21.15.87:0/2467150787 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff1780554e0 con 0x7ff1901433b0 2024-04-25T10:24:50.428 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:24:50.429 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:24:50.429 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:24:50.429 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:24:50.429 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:24:50.429 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:24:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff16c066f60 msgr2=0x7ff16c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff16c066f60 0x7ff16c069420 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff1800ae000 tx=0x7ff1800a7070 comp rx=0 tx=0).stop 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 msgr2=0x7ff19013bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff19013bd40 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7ff17800e840 tx=0x7ff17800ed00 comp rx=0 tx=0).stop 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 shutdown_connections 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff16c066f60 0x7ff16c069420 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff190143790 0x7ff190163160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1901433b0 0x7ff19013bd40 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 --2- 172.21.15.87:0/2467150787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006d900 0x7ff1901599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 >> 172.21.15.87:0/2467150787 conn(0x7ff1900743a0 msgr2=0x7ff19013b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 shutdown_connections 2024-04-25T10:24:50.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.432+0000 7ff194a00640 1 -- 172.21.15.87:0/2467150787 wait complete. 2024-04-25T10:24:50.450 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:50.538 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-04-25T10:24:50.540+0000 7f7829200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:24:50.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c26e00640 1 Processor -- start 2024-04-25T10:24:50.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c26e00640 1 -- start start 2024-04-25T10:24:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c26e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 0x7f1c200d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c26e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c200d0750 0x7f1c200d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c26e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c200d2490 con 0x7f1c200d1ce0 2024-04-25T10:24:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c26e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200d2770 con 0x7f1c200d0750 2024-04-25T10:24:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c26e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200d2a50 con 0x7f1c2015df00 2024-04-25T10:24:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c24e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1c200d1ce0 0x7f1c200b5610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60186/0 (socket says 172.21.15.87:60186) 2024-04-25T10:24:50.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c24e00640 1 -- 172.21.15.87:0/3519530443 learned_addr learned my addr 172.21.15.87:0/3519530443 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1f400640 1 --2- 172.21.15.87:0/3519530443 >> v2:172.21.15.169:3300/0 conn(0x7f1c200d0750 0x7f1c200d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114291640 0 0) 0x7f1c200d2490 con 0x7f1c200d1ce0 2024-04-25T10:24:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c04003530 con 0x7f1c200d1ce0 2024-04-25T10:24:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1fe00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 0x7f1c200d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2847602155 0 0) 0x7f1c04003530 con 0x7f1c200d1ce0 2024-04-25T10:24:50.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 msgr2=0x7f1c200d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 0x7f1c200d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 >> v2:172.21.15.169:3300/0 conn(0x7f1c200d0750 msgr2=0x7f1c200d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 --2- 172.21.15.87:0/3519530443 >> v2:172.21.15.169:3300/0 conn(0x7f1c200d0750 0x7f1c200d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200d2d30 con 0x7f1c200d1ce0 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f1c18001680 con 0x7f1c200d1ce0 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 >> v1:172.21.15.87:6789/0 conn(0x7f1c200d1ce0 legacy=0x7f1c200b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 0x7f1c040039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c04003f90 0x7f1c04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c04004d50 0x7f1c040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200d2770 con 0x7f1c04004d50 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200d2a50 con 0x7f1c04003f90 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.588+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0400b9d0 con 0x7f1c2015df00 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c24e00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c04004d50 0x7f1c040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c1f400640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c04003f90 0x7f1c04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c1fe00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 0x7f1c040039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c24e00640 1 -- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 msgr2=0x7f1c040039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c24e00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c2015df00 0x7f1c040039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c24e00640 1 -- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c04003f90 msgr2=0x7f1c04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c24e00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c04003f90 0x7f1c04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c24e00640 1 -- 172.21.15.87:0/3519530443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c04003530 con 0x7f1c04004d50 2024-04-25T10:24:50.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c24e00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c04004d50 0x7f1c040051d0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c190 tx=0x7f1c1801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb9d239624df928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18007c90 con 0x7f1c04004d50 2024-04-25T10:24:50.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c180a9020 con 0x7f1c04004d50 2024-04-25T10:24:50.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c1ea00640 1 -- 172.21.15.87:0/3519530443 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18005ae0 con 0x7f1c04004d50 2024-04-25T10:24:50.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.592+0000 7f1c26e00640 1 -- 172.21.15.87:0/3519530443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c04004d50 msgr2=0x7f1c040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 --2- 172.21.15.87:0/3519530443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c04004d50 0x7f1c040051d0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c190 tx=0x7f1c1801cbe0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c24e00640 1 -- 172.21.15.87:0/3519530443 reap_dead start 2024-04-25T10:24:50.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- 172.21.15.87:0/3519530443 shutdown_connections 2024-04-25T10:24:50.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- 172.21.15.87:0/3519530443 >> 172.21.15.87:0/3519530443 conn(0x7f1c2006b290 msgr2=0x7f1c200669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:50.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- 172.21.15.87:0/3519530443 shutdown_connections 2024-04-25T10:24:50.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- 172.21.15.87:0/3519530443 wait complete. 2024-04-25T10:24:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 Processor -- start 2024-04-25T10:24:50.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- start start 2024-04-25T10:24:50.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c20150050 0x7f1c200c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c200ca200 0x7f1c2015e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c2015eae0 0x7f1c200cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200d2170 con 0x7f1c20150050 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200d0bd0 con 0x7f1c2015eae0 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c26e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2015e2e0 con 0x7f1c200ca200 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c20150050 0x7f1c200c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c20150050 0x7f1c200c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43888/0 (socket says 172.21.15.87:43888) 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 -- 172.21.15.87:0/673022349 learned_addr learned my addr 172.21.15.87:0/673022349 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1f400640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c200ca200 0x7f1c2015e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c24e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c2015eae0 0x7f1c200cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 -- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c200ca200 msgr2=0x7f1c2015e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c200ca200 0x7f1c2015e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 -- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c2015eae0 msgr2=0x7f1c200cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c2015eae0 0x7f1c200cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.596+0000 7f1c1fe00640 1 -- 172.21.15.87:0/673022349 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200c8c20 con 0x7f1c20150050 2024-04-25T10:24:50.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c1fe00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c20150050 0x7f1c200c9cc0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1c10011e30 tx=0x7f1c1000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c17e00640 1 -- 172.21.15.87:0/673022349 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10015430 con 0x7f1c20150050 2024-04-25T10:24:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c17e00640 1 -- 172.21.15.87:0/673022349 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c100155d0 con 0x7f1c20150050 2024-04-25T10:24:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c17e00640 1 -- 172.21.15.87:0/673022349 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c100159a0 con 0x7f1c20150050 2024-04-25T10:24:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c200c8f00 con 0x7f1c20150050 2024-04-25T10:24:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c200c94c0 con 0x7f1c20150050 2024-04-25T10:24:50.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2015eec0 con 0x7f1c20150050 2024-04-25T10:24:50.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.600+0000 7f1c17e00640 1 -- 172.21.15.87:0/673022349 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c10015b40 con 0x7f1c20150050 2024-04-25T10:24:50.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.604+0000 7f1c17e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c00066f90 0x7f1c00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.604+0000 7f1c17e00640 1 -- 172.21.15.87:0/673022349 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c10086060 con 0x7f1c20150050 2024-04-25T10:24:50.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.604+0000 7f1c1f400640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c00066f90 0x7f1c00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.604+0000 7f1c1f400640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c00066f90 0x7f1c00069450 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1c2015cd60 tx=0x7f1c0c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.608+0000 7f1c17e00640 1 -- 172.21.15.87:0/673022349 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c2015eec0 con 0x7f1c20150050 2024-04-25T10:24:50.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c2004ef30 con 0x7f1c20150050 2024-04-25T10:24:50.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c17e00640 1 -- 172.21.15.87:0/673022349 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f1c10019030 con 0x7f1c20150050 2024-04-25T10:24:50.791 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:50.791 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:24:50.792 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:24:50.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c00066f90 msgr2=0x7f1c00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c00066f90 0x7f1c00069450 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1c2015cd60 tx=0x7f1c0c009190 comp rx=0 tx=0).stop 2024-04-25T10:24:50.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c20150050 msgr2=0x7f1c200c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c20150050 0x7f1c200c9cc0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1c10011e30 tx=0x7f1c1000f580 comp rx=0 tx=0).stop 2024-04-25T10:24:50.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 shutdown_connections 2024-04-25T10:24:50.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c00066f90 0x7f1c00069450 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c2015eae0 0x7f1c200cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c200ca200 0x7f1c2015e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 --2- 172.21.15.87:0/673022349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c20150050 0x7f1c200c9cc0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 >> 172.21.15.87:0/673022349 conn(0x7f1c2006b290 msgr2=0x7f1c200b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:50.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 shutdown_connections 2024-04-25T10:24:50.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:50.792+0000 7f1c26e00640 1 -- 172.21.15.87:0/673022349 wait complete. 2024-04-25T10:24:51.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:24:44.936393+0000', 'modified': '2024-04-25T10:24:47.962825+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10494': {'gid': 10494, 'name': 'c', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/1488445455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 1488445455}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 1488445455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:24:51.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:24:51.214 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:24:51.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.352+0000 7fcd5a200640 1 Processor -- start 2024-04-25T10:24:51.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.352+0000 7fcd5a200640 1 -- start start 2024-04-25T10:24:51.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd5a200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd540d0040 0x7fcd5415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd5a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd5415b740 0x7fcd540b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd5a200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd54071050 con 0x7fcd541550b0 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd5a200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5415d040 con 0x7fcd540d0040 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd5a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540712f0 con 0x7fcd5415b740 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd52a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd540d0040 0x7fcd5415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd52a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcd540d0040 0x7fcd5415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43892/0 (socket says 172.21.15.87:43892) 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd52a00640 1 -- 172.21.15.87:0/3237408721 learned_addr learned my addr 172.21.15.87:0/3237408721 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd53e00640 1 --2- 172.21.15.87:0/3237408721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd5415b740 0x7fcd540b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd52a00640 1 -- 172.21.15.87:0/3237408721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd5415b740 msgr2=0x7fcd540b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd52a00640 1 --2- 172.21.15.87:0/3237408721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd5415b740 0x7fcd540b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd52a00640 1 -- 172.21.15.87:0/3237408721 >> v1:172.21.15.169:6789/0 conn(0x7fcd541550b0 legacy=0x7fcd540cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.356+0000 7fcd52a00640 1 -- 172.21.15.87:0/3237408721 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540715d0 con 0x7fcd540d0040 2024-04-25T10:24:51.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd52a00640 1 --2- 172.21.15.87:0/3237408721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd5415b200 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009870 tx=0x7fcd4001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5694adff1c08bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd52000640 1 -- 172.21.15.87:0/3237408721 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400a95f0 con 0x7fcd540d0040 2024-04-25T10:24:51.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd52000640 1 -- 172.21.15.87:0/3237408721 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd400a9790 con 0x7fcd540d0040 2024-04-25T10:24:51.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd52000640 1 -- 172.21.15.87:0/3237408721 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400a9b60 con 0x7fcd540d0040 2024-04-25T10:24:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- 172.21.15.87:0/3237408721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 msgr2=0x7fcd5415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 --2- 172.21.15.87:0/3237408721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd5415b200 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009870 tx=0x7fcd4001c530 comp rx=0 tx=0).stop 2024-04-25T10:24:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- 172.21.15.87:0/3237408721 shutdown_connections 2024-04-25T10:24:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 --2- 172.21.15.87:0/3237408721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd5415b740 0x7fcd540b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 --2- 172.21.15.87:0/3237408721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd5415b200 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- 172.21.15.87:0/3237408721 >> 172.21.15.87:0/3237408721 conn(0x7fcd54066b70 msgr2=0x7fcd54066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- 172.21.15.87:0/3237408721 shutdown_connections 2024-04-25T10:24:51.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- 172.21.15.87:0/3237408721 wait complete. 2024-04-25T10:24:51.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 Processor -- start 2024-04-25T10:24:51.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- start start 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd54144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd541550b0 0x7fcd54157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5415b740 0x7fcd54157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415bcd0 con 0x7fcd540d0040 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415bb50 con 0x7fcd5415b740 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd5a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415c030 con 0x7fcd541550b0 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.360+0000 7fcd53400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd54144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd54144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43894/0 (socket says 172.21.15.87:43894) 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 -- 172.21.15.87:0/2950933733 learned_addr learned my addr 172.21.15.87:0/2950933733 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd52a00640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd541550b0 0x7fcd54157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53e00640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5415b740 0x7fcd54157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 -- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd541550b0 msgr2=0x7fcd54157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd541550b0 0x7fcd54157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 -- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5415b740 msgr2=0x7fcd54157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5415b740 0x7fcd54157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 -- 172.21.15.87:0/2950933733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd541453d0 con 0x7fcd540d0040 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd52a00640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd541550b0 0x7fcd54157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd53400640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd54144d20 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400c420 tx=0x7fcd4400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd4b400640 1 -- 172.21.15.87:0/2950933733 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440052d0 con 0x7fcd540d0040 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd4b400640 1 -- 172.21.15.87:0/2950933733 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44005470 con 0x7fcd540d0040 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd541456b0 con 0x7fcd540d0040 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd4b400640 1 -- 172.21.15.87:0/2950933733 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44007260 con 0x7fcd540d0040 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd54145c70 con 0x7fcd540d0040 2024-04-25T10:24:51.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd4b400640 1 -- 172.21.15.87:0/2950933733 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd44007400 con 0x7fcd540d0040 2024-04-25T10:24:51.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd4b400640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd34066f60 0x7fcd34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.364+0000 7fcd4b400640 1 -- 172.21.15.87:0/2950933733 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd44084590 con 0x7fcd540d0040 2024-04-25T10:24:51.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd52a00640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd34066f60 0x7fcd34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 Processor -- start 2024-04-25T10:24:51.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- start start 2024-04-25T10:24:51.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd52a00640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd34066f60 0x7fcd34069420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcd400ae000 tx=0x7fcd400b3040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bff1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8bfa79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfa8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8be8f5f0 con 0x55aa8bfa7f30 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8bdf6da0 con 0x55aa8bff1920 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa8bd7f150 con 0x55aa8bff2240 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfa8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bff1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8bfa79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bff1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:52268/0 (socket says 172.21.15.87:52268) 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53400640 1 -- 172.21.15.87:0/3229940259 learned_addr learned my addr 172.21.15.87:0/3229940259 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53e00640 1 -- 172.21.15.87:0/3229940259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 msgr2=0x55aa8bfa79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53e00640 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8bfa79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53e00640 1 -- 172.21.15.87:0/3229940259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 msgr2=0x55aa8bff1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53e00640 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bff1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53e00640 1 -- 172.21.15.87:0/3229940259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa8bd98ca0 con 0x55aa8bfa7f30 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53400640 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bff1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd53e00640 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfa8310 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009f20 tx=0x7fcd4c0077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242ea903d5fa4dec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd3fe00640 1 -- 172.21.15.87:0/3229940259 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0147e0 con 0x55aa8bfa7f30 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd3fe00640 1 -- 172.21.15.87:0/3229940259 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c01a070 con 0x55aa8bfa7f30 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd3fe00640 1 -- 172.21.15.87:0/3229940259 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c014ba0 con 0x55aa8bfa7f30 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- 172.21.15.87:0/3229940259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 msgr2=0x55aa8bfa8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfa8310 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009f20 tx=0x7fcd4c0077c0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- 172.21.15.87:0/3229940259 shutdown_connections 2024-04-25T10:24:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfa8310 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8bfa79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 --2- 172.21.15.87:0/3229940259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bff1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- 172.21.15.87:0/3229940259 >> 172.21.15.87:0/3229940259 conn(0x55aa8bfae2a0 msgr2=0x55aa8bfa62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- 172.21.15.87:0/3229940259 shutdown_connections 2024-04-25T10:24:51.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.368+0000 7fcd5d740000 1 -- 172.21.15.87:0/3229940259 wait complete. 2024-04-25T10:24:51.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 -- start start 2024-04-25T10:24:51.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 Processor -- start 2024-04-25T10:24:51.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bff8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff1920 0x55aa8bff91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff2240 0x55aa8bfabaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8bdb1990 con 0x55aa8bfa7f30 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8bdf2e40 con 0x55aa8bff2240 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa8bd82900 con 0x55aa8bff1920 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 10 client.0 _ll_register_callbacks cb 0x55aa8beaa1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bff8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bff8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43924/0 (socket says 172.21.15.87:43924) 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 -- 172.21.15.87:0/11038509 learned_addr learned my addr 172.21.15.87:0/11038509 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd52a00640 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff1920 0x55aa8bff91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53e00640 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff2240 0x55aa8bfabaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 -- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff1920 msgr2=0x55aa8bff91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff1920 0x55aa8bff91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 -- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff2240 msgr2=0x55aa8bfabaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff2240 0x55aa8bfabaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 -- 172.21.15.87:0/11038509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa8bdaf630 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd53400640 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bff8c80 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fcd4405d080 tx=0x7fcd4400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efee77d66248439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd3cc00640 1 -- 172.21.15.87:0/11038509 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44081060 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd3cc00640 1 -- 172.21.15.87:0/11038509 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd44051510 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd4c01a070 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd3cc00640 1 -- 172.21.15.87:0/11038509 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd44071050 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 10 client.10635 fetch_fsmap learned FSMap version 119 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55aa8bd7f3a0 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd3cc00640 1 -- 172.21.15.87:0/11038509 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c01a070 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd3cc00640 1 -- 172.21.15.87:0/11038509 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd4405d120 con 0x55aa8bfa7f30 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 10 client.10635 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 10 client.10635 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:24:51.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 20 client.10635 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8bfad160 0x55aa8c002c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 4 client.10635 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa8bd92a50 con 0x55aa8bfad160 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd52a00640 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8bfad160 0x55aa8c002c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd52a00640 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8bfad160 0x55aa8c002c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d8047d30bff969 server_cookie=4ce7d919df1e6500 in_seq=0 out_seq=0 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.372+0000 7fcd3cc00640 10 client.10635 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd3cc00640 1 -- 172.21.15.87:0/11038509 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa8bd92a50 con 0x55aa8bfad160 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd3cc00640 10 client.10635 handle_command_reply: tid=0 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 client.10635 shutdown 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 msgr2=0x55aa8bff8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bff8c80 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fcd4405d080 tx=0x7fcd4400cf90 comp rx=0 tx=0).stop 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 shutdown_connections 2024-04-25T10:24:51.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff2240 0x55aa8bfabaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff1920 0x55aa8bff91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8bfad160 0x55aa8c002c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 --2- 172.21.15.87:0/11038509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bff8c80 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 >> 172.21.15.87:0/11038509 conn(0x55aa8bfae2a0 msgr2=0x55aa8bfa69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd3cc00640 0 client.10635 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 shutdown_connections 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 1 -- 172.21.15.87:0/11038509 wait complete. 2024-04-25T10:24:51.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.376+0000 7fcd5d740000 20 client.10635 trim_cache size 0 max 16384 2024-04-25T10:24:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 Processor -- start 2024-04-25T10:24:51.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- start start 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bfab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c031b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd542021a0 con 0x55aa8bfa7f30 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414ed10 con 0x55aa8bff1920 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414abb0 con 0x55aa8bff2240 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43940/0 (socket says 172.21.15.87:43940) 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 -- 172.21.15.87:0/797728263 learned_addr learned my addr 172.21.15.87:0/797728263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd52a00640 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bfab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53e00640 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c031b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 -- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 msgr2=0x55aa8c031b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c031b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 -- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 msgr2=0x55aa8bfab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bfab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 -- 172.21.15.87:0/797728263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa8be78600 con 0x55aa8bfa7f30 2024-04-25T10:24:51.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd53400640 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfab330 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fcd4405d520 tx=0x7fcd44069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40392007cbebb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd3f400640 1 -- 172.21.15.87:0/797728263 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4406c070 con 0x55aa8bfa7f30 2024-04-25T10:24:51.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd3f400640 1 -- 172.21.15.87:0/797728263 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4407c0d0 con 0x55aa8bfa7f30 2024-04-25T10:24:51.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd3f400640 1 -- 172.21.15.87:0/797728263 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44068e60 con 0x55aa8bfa7f30 2024-04-25T10:24:51.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- 172.21.15.87:0/797728263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 msgr2=0x55aa8bfab330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfab330 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fcd4405d520 tx=0x7fcd44069000 comp rx=0 tx=0).stop 2024-04-25T10:24:51.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- 172.21.15.87:0/797728263 shutdown_connections 2024-04-25T10:24:51.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c031b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bff1920 0x55aa8bfab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 --2- 172.21.15.87:0/797728263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bfa7f30 0x55aa8bfab330 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- 172.21.15.87:0/797728263 >> 172.21.15.87:0/797728263 conn(0x55aa8bfae2a0 msgr2=0x55aa8bfde380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- 172.21.15.87:0/797728263 shutdown_connections 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- 172.21.15.87:0/797728263 wait complete. 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 -- start start 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.392+0000 7fcd5d740000 1 Processor -- start 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bfa7f30 0x55aa8bffea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bff1920 0x55aa8bffefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c0361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8bdfb3d0 con 0x55aa8bff1920 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8beb2d50 con 0x55aa8bfa7f30 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa8bd91980 con 0x55aa8bff2240 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 10 client.0 _ll_register_callbacks cb 0x7fcd4c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bff1920 0x55aa8bffefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd53400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bfa7f30 0x55aa8bffea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bff1920 0x55aa8bffefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43952/0 (socket says 172.21.15.87:43952) 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 -- 172.21.15.87:0/1916864964 learned_addr learned my addr 172.21.15.87:0/1916864964 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd53e00640 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c0361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 -- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 msgr2=0x55aa8c0361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c0361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 -- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bfa7f30 msgr2=0x55aa8bffea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bfa7f30 0x55aa8bffea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 -- 172.21.15.87:0/1916864964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa8bd8b9b0 con 0x55aa8bff1920 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd53400640 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bfa7f30 0x55aa8bffea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd52a00640 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bff1920 0x55aa8bffefd0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fcd4001abe0 tx=0x7fcd400046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b773a33b357856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 1 -- 172.21.15.87:0/1916864964 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400d1070 con 0x55aa8bff1920 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 1 -- 172.21.15.87:0/1916864964 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd400bbd10 con 0x55aa8bff1920 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd4407c0d0 con 0x55aa8bff1920 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 1 -- 172.21.15.87:0/1916864964 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd400dc050 con 0x55aa8bff1920 2024-04-25T10:24:51.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 10 client.10641 fetch_fsmap learned FSMap version 119 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55aa8bd9e0b0 con 0x55aa8bff1920 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 1 -- 172.21.15.87:0/1916864964 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4407c0d0 con 0x55aa8bff1920 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 1 -- 172.21.15.87:0/1916864964 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd400df020 con 0x55aa8bff1920 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 10 client.10641 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 10 client.10641 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 20 client.10641 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8c035bc0 0x55aa8c036700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 4 client.10641 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa8be042e0 con 0x55aa8c035bc0 2024-04-25T10:24:51.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd53400640 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8c035bc0 0x55aa8c036700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd53400640 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8c035bc0 0x55aa8c036700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bf64bb63e124d0 server_cookie=235bd8bdb87646d8 in_seq=0 out_seq=0 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 10 client.10641 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 1 -- 172.21.15.87:0/1916864964 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55aa8be042e0 con 0x55aa8c035bc0 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 10 client.10641 handle_command_reply: tid=0 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 client.10641 shutdown 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bff1920 msgr2=0x55aa8bffefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bff1920 0x55aa8bffefd0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fcd4001abe0 tx=0x7fcd400046d0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 shutdown_connections 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55aa8bff2240 0x55aa8c0361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55aa8c035bc0 0x55aa8c036700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa8bff1920 0x55aa8bffefd0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 --2- 172.21.15.87:0/1916864964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa8bfa7f30 0x55aa8bffea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd3cc00640 0 client.10641 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.396+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 >> 172.21.15.87:0/1916864964 conn(0x55aa8bfae2a0 msgr2=0x55aa8bfaa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 shutdown_connections 2024-04-25T10:24:51.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5d740000 1 -- 172.21.15.87:0/1916864964 wait complete. 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5d740000 20 client.10641 trim_cache size 0 max 16384 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 10494, 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:24:51.402 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8345541519999999, 2024-04-25T10:24:51.403 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 119, 2024-04-25T10:24:51.403 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 163, 2024-04-25T10:24:51.403 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 163, 2024-04-25T10:24:51.403 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 11.736986345 2024-04-25T10:24:51.403 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:24:51.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd34066f60 msgr2=0x7fcd34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd34066f60 0x7fcd34069420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcd400ae000 tx=0x7fcd400b3040 comp rx=0 tx=0).stop 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 msgr2=0x7fcd54144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd54144d20 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400c420 tx=0x7fcd4400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 shutdown_connections 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd34066f60 0x7fcd34069420 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5415b740 0x7fcd54157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd541550b0 0x7fcd54157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 --2- 172.21.15.87:0/2950933733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd540d0040 0x7fcd54144d20 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 >> 172.21.15.87:0/2950933733 conn(0x7fcd54066b70 msgr2=0x7fcd540b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 shutdown_connections 2024-04-25T10:24:51.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.400+0000 7fcd5a200640 1 -- 172.21.15.87:0/2950933733 wait complete. 2024-04-25T10:24:51.417 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:24:51.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.552+0000 7f1e23200640 1 Processor -- start 2024-04-25T10:24:51.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.552+0000 7f1e23200640 1 -- start start 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e23200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e1c0cfd90 0x7f1e1c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e23200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e23200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c15f3b0 con 0x7f1e1c0b0f70 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e23200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c15ef20 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e23200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0ca810 con 0x7f1e1c0cd110 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e1c0cfd90 0x7f1e1c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e1c0cfd90 0x7f1e1c0ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43968/0 (socket says 172.21.15.87:43968) 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 -- 172.21.15.87:0/1056476749 learned_addr learned my addr 172.21.15.87:0/1056476749 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e21200640 1 --2- 172.21.15.87:0/1056476749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 -- 172.21.15.87:0/1056476749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 msgr2=0x7f1e1c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 --2- 172.21.15.87:0/1056476749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 -- 172.21.15.87:0/1056476749 >> v1:172.21.15.169:6789/0 conn(0x7f1e1c0b0f70 legacy=0x7f1e1c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 -- 172.21.15.87:0/1056476749 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c15e5f0 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e13400640 1 --2- 172.21.15.87:0/1056476749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c0ccbd0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009870 tx=0x7f1e0401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d128f3126a58159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.556+0000 7f1e1b400640 1 -- 172.21.15.87:0/1056476749 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040b2070 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e1b400640 1 -- 172.21.15.87:0/1056476749 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e040a9070 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e1b400640 1 -- 172.21.15.87:0/1056476749 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040ad880 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- 172.21.15.87:0/1056476749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 msgr2=0x7f1e1c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 --2- 172.21.15.87:0/1056476749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c0ccbd0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009870 tx=0x7f1e0401c560 comp rx=0 tx=0).stop 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- 172.21.15.87:0/1056476749 shutdown_connections 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 --2- 172.21.15.87:0/1056476749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 --2- 172.21.15.87:0/1056476749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c0ccbd0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- 172.21.15.87:0/1056476749 >> 172.21.15.87:0/1056476749 conn(0x7f1e1c0b1d50 msgr2=0x7f1e1c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- 172.21.15.87:0/1056476749 shutdown_connections 2024-04-25T10:24:51.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- 172.21.15.87:0/1056476749 wait complete. 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 Processor -- start 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- start start 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e1c0b0f70 0x7f1e1c0b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c161b50 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1619d0 con 0x7f1e1c0b0f70 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e23200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c161cd0 con 0x7f1e1c0cd110 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43978/0 (socket says 172.21.15.87:43978) 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 -- 172.21.15.87:0/3775955488 learned_addr learned my addr 172.21.15.87:0/3775955488 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:51.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e1be00640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e1c0b0f70 0x7f1e1c0b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e13400640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 -- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 msgr2=0x7f1e1c0c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 -- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e1c0b0f70 msgr2=0x7f1e1c0b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e1c0b0f70 0x7f1e1c0b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e21200640 1 -- 172.21.15.87:0/3775955488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c0ca3f0 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e13400640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.560+0000 7f1e1be00640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e1c0b0f70 0x7f1e1c0b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e21200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c155cb0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400d800 tx=0x7f1e1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e18c00640 1 -- 172.21.15.87:0/3775955488 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400b6c0 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e18c00640 1 -- 172.21.15.87:0/3775955488 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e14019070 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e18c00640 1 -- 172.21.15.87:0/3775955488 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400baa0 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c0b48a0 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c0ad6f0 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c04ef30 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e18c00640 1 -- 172.21.15.87:0/3775955488 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e1400bc40 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e18c00640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1df8066f90 0x7f1df8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e18c00640 1 -- 172.21.15.87:0/3775955488 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e14085480 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.564+0000 7f1e1be00640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1df8066f90 0x7f1df8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.568+0000 7f1e1be00640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1df8066f90 0x7f1df8069450 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c001e10 tx=0x7f1e0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.568+0000 7f1e18c00640 1 -- 172.21.15.87:0/3775955488 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e14056480 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1e1c0cda20 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e18c00640 1 -- 172.21.15.87:0/3775955488 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v119) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1e14055d80 con 0x7f1e1c0cfd90 2024-04-25T10:24:51.758 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:24:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1df8066f90 msgr2=0x7f1df8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1df8066f90 0x7f1df8069450 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c001e10 tx=0x7f1e0c009040 comp rx=0 tx=0).stop 2024-04-25T10:24:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 msgr2=0x7f1e1c155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c155cb0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400d800 tx=0x7f1e1400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 shutdown_connections 2024-04-25T10:24:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1df8066f90 0x7f1df8069450 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c0cfd90 0x7f1e1c155cb0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e1c0cd110 0x7f1e1c0c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 --2- 172.21.15.87:0/3775955488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e1c0b0f70 0x7f1e1c0b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 >> 172.21.15.87:0/3775955488 conn(0x7f1e1c0b1d50 msgr2=0x7f1e1c0c6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 shutdown_connections 2024-04-25T10:24:51.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:51.760+0000 7f1e23200640 1 -- 172.21.15.87:0/3775955488 wait complete. 2024-04-25T10:24:51.778 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:24:51.778 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:24:51.779 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:24:51.779 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:24:51.779 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:51.779 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:24:51.779 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:24:51.779 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:24:51.779 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:24:51.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:51.779 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:24:51.788 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41722sec preferred_lft 41722sec 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:24:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:24:51.791 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:51.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:51.792 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:24:51.855 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:24:51.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:51.857 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:24:51.917 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:24:51.917 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:24:51.917 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:24:51.917 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:24:51.918 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:24:51.918 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:24:51.918 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:24:51.918 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:24:51.918 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:24:51.919 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:24:51.919 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:24:51.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:51.919 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:24:51.919 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:24:51.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:24:51.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:24:51.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:24:51.920 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:24:52.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:52.010 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:24:52.061 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:24:52.061 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:24:52.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:24:52.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:52.062 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:24:52.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:24:52.124 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:52.173 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:24:52.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:24:52.173 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:52.220 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:24:52.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:52.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:24:52.308 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:24:52.309 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:24:52.309 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:24:52.323 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:24:52.323 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:24:52.323 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:24:52.323 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:24:52.324 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:24:52.332 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:24:52.333 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:24:52.333 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:24:52.333 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:24:52.333 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:24:52.333 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:24:52.333 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:24:52.333 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:24:52.336 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:24:52.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:52.337 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:52.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:52.356 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:24:52.417 INFO:teuthology.orchestra.run.smithi087.stdout:client8435 2024-04-25T10:24:52.417 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:24:52.417 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:52.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:52.474 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T10:24:52.535 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:24:52.535 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:52.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:52.586 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:24:52.723 INFO:teuthology.orchestra.run.smithi087.stderr:6+0 records in 2024-04-25T10:24:52.723 INFO:teuthology.orchestra.run.smithi087.stderr:6+0 records out 2024-04-25T10:24:52.723 INFO:teuthology.orchestra.run.smithi087.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0842659 s, 74.7 MB/s 2024-04-25T10:24:52.725 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:24:52.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.769 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-04-25T10:24:52.770 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-04-25T10:24:52.770 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.770 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-04-25T10:24:52.770 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-04-25T10:24:52.770 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.770 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:24:52.770 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:24:52.771 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-04-25T10:24:52.772 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on skipped message = False 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.773 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-04-25T10:24:52.774 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: bdev async discard = True 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: bdev enable discard = True 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore allocator = bitmap 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-04-25T10:24:52.775 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 1/20 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 1/20 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 4/10 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-04-25T10:24:52.776 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-04-25T10:24:52.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:24:52.778 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-04-25T10:24:52.779 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: mds cache max size = 1000 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = True 2024-04-25T10:24:52.780 INFO:teuthology.orchestra.run.smithi087.stdout: mds log max segments = 1 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = False 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-25T10:24:52.781 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:24:52.782 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:24:52.783 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:24:52.784 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.785 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.786 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-04-25T10:24:52.787 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:24:52.788 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.789 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:24:52.790 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-04-25T10:24:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: mds cache max size = 1000 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2024-04-25T10:24:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: mds log max segments = 1 2024-04-25T10:24:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-25T10:24:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = False 2024-04-25T10:24:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:24:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.795 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:24:52.844 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-04-25T10:24:52.883 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:24:52.884 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-04-25T10:24:52.885 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on skipped message = False 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:24:52.886 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-04-25T10:24:52.887 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: bdev async discard = True 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: bdev enable discard = True 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore allocator = bitmap 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 1/20 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 1/20 2024-04-25T10:24:52.888 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 4/10 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:24:52.889 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-04-25T10:24:52.890 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:24:52.891 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-04-25T10:24:52.892 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: mds cache max size = 1000 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = False 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: mds log max segments = 1 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = False 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:24:52.893 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.894 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.895 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.896 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-25T10:24:52.897 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.898 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-25T10:24:52.899 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.900 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:24:52.901 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:24:52.902 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: mds cache max size = 1000 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = False 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: mds log max segments = 1 2024-04-25T10:24:52.904 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-25T10:24:52.905 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = False 2024-04-25T10:24:52.905 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:24:52.905 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:24:52.905 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:24:52.906 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:24:52.906 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> import stat 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> import json 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> import sys 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> except OSError as e: 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> sys.exit(e.errno) 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> print(json.dumps( 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> indent=2)) 2024-04-25T10:24:52.907 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:24:53.016 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:24:53.016 INFO:teuthology.orchestra.run.smithi087.stdout: "st_mode": 33204, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_ino": 1099511627777, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_dev": 44, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_nlink": 1, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_uid": 1000, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_gid": 1271, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_size": 6291456, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_atime": 1714040692.640149, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_mtime": 1714040692.6721487, 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout: "st_ctime": 1714040692.6721487 2024-04-25T10:24:53.017 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:24:53.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:24:53.026 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:53.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9c200640 1 Processor -- start 2024-04-25T10:24:53.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9c200640 1 -- start start 2024-04-25T10:24:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9c200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca941550b0 0x7fca940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9c200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca940cfe70 0x7fca9415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9c200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca94071050 con 0x7fca9415b740 2024-04-25T10:24:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9c200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9415d040 con 0x7fca940cfe70 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9c200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca940712f0 con 0x7fca941550b0 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca98e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca940cfe70 0x7fca9415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9a200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fca9415b740 0x7fca9406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60188/0 (socket says 172.21.15.87:60188) 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca9a200640 1 -- 172.21.15.87:0/658450775 learned_addr learned my addr 172.21.15.87:0/658450775 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.204+0000 7fca99800640 1 --2- 172.21.15.87:0/658450775 >> v2:172.21.15.169:3300/0 conn(0x7fca941550b0 0x7fca940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca98e00640 1 -- 172.21.15.87:0/658450775 >> v2:172.21.15.169:3300/0 conn(0x7fca941550b0 msgr2=0x7fca940cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca98e00640 1 --2- 172.21.15.87:0/658450775 >> v2:172.21.15.169:3300/0 conn(0x7fca941550b0 0x7fca940cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca98e00640 1 -- 172.21.15.87:0/658450775 >> v1:172.21.15.87:6789/0 conn(0x7fca9415b740 legacy=0x7fca9406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca98e00640 1 -- 172.21.15.87:0/658450775 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940715d0 con 0x7fca940cfe70 2024-04-25T10:24:53.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca93e00640 1 -- 172.21.15.87:0/658450775 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285817667 0 0) 0x7fca94071050 con 0x7fca9415b740 2024-04-25T10:24:53.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca98e00640 1 --2- 172.21.15.87:0/658450775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca940cfe70 0x7fca9415b200 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fca800097c0 tx=0x7fca8001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef85f32ec860288 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca93e00640 1 -- 172.21.15.87:0/658450775 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800b0070 con 0x7fca940cfe70 2024-04-25T10:24:53.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca93e00640 1 -- 172.21.15.87:0/658450775 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca800ab600 con 0x7fca940cfe70 2024-04-25T10:24:53.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca93e00640 1 -- 172.21.15.87:0/658450775 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800ab9d0 con 0x7fca940cfe70 2024-04-25T10:24:53.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 -- 172.21.15.87:0/658450775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca940cfe70 msgr2=0x7fca9415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 --2- 172.21.15.87:0/658450775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca940cfe70 0x7fca9415b200 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fca800097c0 tx=0x7fca8001c460 comp rx=0 tx=0).stop 2024-04-25T10:24:53.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 -- 172.21.15.87:0/658450775 shutdown_connections 2024-04-25T10:24:53.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 --2- 172.21.15.87:0/658450775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca940cfe70 0x7fca9415b200 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 --2- 172.21.15.87:0/658450775 >> v2:172.21.15.169:3300/0 conn(0x7fca941550b0 0x7fca940cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 -- 172.21.15.87:0/658450775 >> 172.21.15.87:0/658450775 conn(0x7fca94066b70 msgr2=0x7fca94066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 -- 172.21.15.87:0/658450775 shutdown_connections 2024-04-25T10:24:53.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.208+0000 7fca9c200640 1 -- 172.21.15.87:0/658450775 wait complete. 2024-04-25T10:24:53.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 Processor -- start 2024-04-25T10:24:53.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 -- start start 2024-04-25T10:24:53.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca940cfe70 0x7fca9414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca941550b0 0x7fca94145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9415b740 0x7fca9415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9415bcf0 con 0x7fca9415b740 2024-04-25T10:24:53.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9415bb70 con 0x7fca941550b0 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9415c030 con 0x7fca940cfe70 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9415b740 0x7fca9415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9415b740 0x7fca9415a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44006/0 (socket says 172.21.15.87:44006) 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 -- 172.21.15.87:0/3134923260 learned_addr learned my addr 172.21.15.87:0/3134923260 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca98e00640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca941550b0 0x7fca94145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca99800640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca940cfe70 0x7fca9414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 -- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca940cfe70 msgr2=0x7fca9414db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca940cfe70 0x7fca9414db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 -- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca941550b0 msgr2=0x7fca94145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca941550b0 0x7fca94145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 -- 172.21.15.87:0/3134923260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca94145ff0 con 0x7fca9415b740 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca99800640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca940cfe70 0x7fca9414db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:53.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9a200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9415b740 0x7fca9415a2b0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c730 tx=0x7fca8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca91600640 1 -- 172.21.15.87:0/3134923260 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c018070 con 0x7fca9415b740 2024-04-25T10:24:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca91600640 1 -- 172.21.15.87:0/3134923260 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c00ce40 con 0x7fca9415b740 2024-04-25T10:24:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca91600640 1 -- 172.21.15.87:0/3134923260 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00a860 con 0x7fca9415b740 2024-04-25T10:24:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.212+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca94152e80 con 0x7fca9415b740 2024-04-25T10:24:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.216+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca94153440 con 0x7fca9415b740 2024-04-25T10:24:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.216+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9404ef30 con 0x7fca9415b740 2024-04-25T10:24:53.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.216+0000 7fca91600640 1 -- 172.21.15.87:0/3134923260 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca8c00aa00 con 0x7fca9415b740 2024-04-25T10:24:53.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.216+0000 7fca91600640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca74066f60 0x7fca74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.216+0000 7fca91600640 1 -- 172.21.15.87:0/3134923260 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca8c0845b0 con 0x7fca9415b740 2024-04-25T10:24:53.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.216+0000 7fca99800640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca74066f60 0x7fca74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.216+0000 7fca99800640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca74066f60 0x7fca74069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fca88009840 tx=0x7fca88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.220+0000 7fca91600640 1 -- 172.21.15.87:0/3134923260 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca8c0555c0 con 0x7fca9415b740 2024-04-25T10:24:53.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.412+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca9414d730 con 0x7fca9415b740 2024-04-25T10:24:53.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.412+0000 7fca91600640 1 -- 172.21.15.87:0/3134923260 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7fca8c054ec0 con 0x7fca9415b740 2024-04-25T10:24:53.412 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:24:53.412 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:24:53.412 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:24:53.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca74066f60 msgr2=0x7fca74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca74066f60 0x7fca74069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fca88009840 tx=0x7fca88008040 comp rx=0 tx=0).stop 2024-04-25T10:24:53.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9415b740 msgr2=0x7fca9415a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9415b740 0x7fca9415a2b0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c730 tx=0x7fca8c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 shutdown_connections 2024-04-25T10:24:53.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fca74066f60 0x7fca74069420 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca9415b740 0x7fca9415a2b0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca941550b0 0x7fca94145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 --2- 172.21.15.87:0/3134923260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca940cfe70 0x7fca9414db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 >> 172.21.15.87:0/3134923260 conn(0x7fca94066b70 msgr2=0x7fca940b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 shutdown_connections 2024-04-25T10:24:53.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.416+0000 7fca9c200640 1 -- 172.21.15.87:0/3134923260 wait complete. 2024-04-25T10:24:53.434 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:24:53.599 INFO:teuthology.orchestra.run.smithi087.stdout:{"message":"","return_code":0} 2024-04-25T10:24:53.599 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:24:53.599 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-25T10:24:53.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 Processor -- start 2024-04-25T10:24:53.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- start start 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f48cdf70 0x5556f4954490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f495b7a0 0x5556f495c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 0x5556f495ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f482a9f0 con 0x5556f48cdf70 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f482e950 con 0x5556f495b7a0 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556f48b2510 con 0x5556f495bb80 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f48cdf70 0x5556f4954490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f48cdf70 0x5556f4954490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44016/0 (socket says 172.21.15.87:44016) 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 -- 172.21.15.87:0/939134196 learned_addr learned my addr 172.21.15.87:0/939134196 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 -- 172.21.15.87:0/939134196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 msgr2=0x5556f495ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 --2- 172.21.15.87:0/939134196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 0x5556f495ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 -- 172.21.15.87:0/939134196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f495b7a0 msgr2=0x5556f495c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 --2- 172.21.15.87:0/939134196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f495b7a0 0x5556f495c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 -- 172.21.15.87:0/939134196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556f48c8460 con 0x5556f48cdf70 2024-04-25T10:24:53.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449c000640 1 --2- 172.21.15.87:0/939134196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f48cdf70 0x5556f4954490 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f448c00b320 tx=0x7f448c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4f1ba49e068f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ac00640 1 -- 172.21.15.87:0/939134196 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00e5d0 con 0x5556f48cdf70 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ac00640 1 -- 172.21.15.87:0/939134196 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c00e770 con 0x5556f48cdf70 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ac00640 1 -- 172.21.15.87:0/939134196 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00eb20 con 0x5556f48cdf70 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- 172.21.15.87:0/939134196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f48cdf70 msgr2=0x5556f4954490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 --2- 172.21.15.87:0/939134196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f48cdf70 0x5556f4954490 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f448c00b320 tx=0x7f448c00b7e0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- 172.21.15.87:0/939134196 shutdown_connections 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 --2- 172.21.15.87:0/939134196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 0x5556f495ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 --2- 172.21.15.87:0/939134196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f495b7a0 0x5556f495c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 --2- 172.21.15.87:0/939134196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f48cdf70 0x5556f4954490 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- 172.21.15.87:0/939134196 >> 172.21.15.87:0/939134196 conn(0x5556f48ae760 msgr2=0x5556f4882940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- 172.21.15.87:0/939134196 shutdown_connections 2024-04-25T10:24:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.648+0000 7f449ee55080 1 -- 172.21.15.87:0/939134196 wait complete. 2024-04-25T10:24:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 Processor -- start 2024-04-25T10:24:53.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 -- start start 2024-04-25T10:24:53.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f48cdf70 0x5556f4955d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f495b7a0 0x5556f4953060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 0x5556f495cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f48b39c0 con 0x5556f495b7a0 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f48b3c60 con 0x5556f48cdf70 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556f48b3f00 con 0x5556f495bb80 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f495b7a0 0x5556f4953060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f495b7a0 0x5556f4953060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44018/0 (socket says 172.21.15.87:44018) 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 -- 172.21.15.87:0/422840105 learned_addr learned my addr 172.21.15.87:0/422840105 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ca00640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 0x5556f495cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449c000640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f48cdf70 0x5556f4955d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 -- 172.21.15.87:0/422840105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 msgr2=0x5556f495cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556f495bb80 0x5556f495cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 -- 172.21.15.87:0/422840105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f48cdf70 msgr2=0x5556f4955d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556f48cdf70 0x5556f4955d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 -- 172.21.15.87:0/422840105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556f49568c0 con 0x5556f495b7a0 2024-04-25T10:24:53.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4493e00640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f495b7a0 0x5556f4953060 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f448400bd00 tx=0x7f4484009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4499800640 1 -- 172.21.15.87:0/422840105 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484014030 con 0x5556f495b7a0 2024-04-25T10:24:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4499800640 1 -- 172.21.15.87:0/422840105 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448400f410 con 0x5556f495b7a0 2024-04-25T10:24:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5556f4956b60 con 0x5556f495b7a0 2024-04-25T10:24:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f4499800640 1 -- 172.21.15.87:0/422840105 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400a040 con 0x5556f495b7a0 2024-04-25T10:24:53.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.652+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556f49570a0 con 0x5556f495b7a0 2024-04-25T10:24:53.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f4499800640 1 -- 172.21.15.87:0/422840105 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4484017070 con 0x5556f495b7a0 2024-04-25T10:24:53.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f4499800640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4478065940 0x7f4478067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f4499800640 1 -- 172.21.15.87:0/422840105 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44840833b0 con 0x5556f495b7a0 2024-04-25T10:24:53.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449c000640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4478065940 0x7f4478067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449ee55080 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x5556f4958550 0x5556f4a000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 --> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] -- osd_op(unknown.0.0:1 25.7 25:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163) v9 -- 0x5556f49535a0 con 0x5556f4958550 2024-04-25T10:24:53.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449ca00640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x5556f4958550 0x5556f4a000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449c000640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4478065940 0x7f4478067e00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f448c000fc0 tx=0x7f448c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449ca00640 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x5556f4958550 0x5556f4a000f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449ca00640 1 -- 172.21.15.87:0/422840105 <== osd.0 v2:172.21.15.87:6800/2221934810 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v163'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f44940aa960 con 0x5556f4958550 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x5556f4958550 msgr2=0x5556f4a000f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.656+0000 7f449ee55080 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x5556f4958550 0x5556f4a000f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4478065940 msgr2=0x7f4478067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4478065940 0x7f4478067e00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f448c000fc0 tx=0x7f448c009000 comp rx=0 tx=0).stop 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f495b7a0 msgr2=0x5556f4953060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 --2- 172.21.15.87:0/422840105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5556f495b7a0 0x5556f4953060 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f448400bd00 tx=0x7f4484009540 comp rx=0 tx=0).stop 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ca00640 1 -- 172.21.15.87:0/422840105 reap_dead start 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 shutdown_connections 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 >> 172.21.15.87:0/422840105 conn(0x5556f48ae760 msgr2=0x5556f495c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 shutdown_connections 2024-04-25T10:24:53.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.660+0000 7f449ee55080 1 -- 172.21.15.87:0/422840105 wait complete. 2024-04-25T10:24:53.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:53.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub start / recursive,repair 2024-04-25T10:24:53.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe244e00640 1 Processor -- start 2024-04-25T10:24:53.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe244e00640 1 -- start start 2024-04-25T10:24:53.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe244e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe24015bd10 0x7fe2400b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe244e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24006c0e0 con 0x7fe2400cf9e0 2024-04-25T10:24:53.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe244e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24015c6a0 con 0x7fe24015c0f0 2024-04-25T10:24:53.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe244e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400b2ef0 con 0x7fe24015bd10 2024-04-25T10:24:53.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23e800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe2400cf9e0 0x7fe24006c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:60204/0 (socket says 172.21.15.87:60204) 2024-04-25T10:24:53.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23e800640 1 -- 172.21.15.87:0/1425134118 learned_addr learned my addr 172.21.15.87:0/1425134118 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:24:53.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23de00640 1 --2- 172.21.15.87:0/1425134118 >> v2:172.21.15.169:3300/0 conn(0x7fe24015bd10 0x7fe2400b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23ca00640 1 -- 172.21.15.87:0/1425134118 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020495542 0 0) 0x7fe24006c0e0 con 0x7fe2400cf9e0 2024-04-25T10:24:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23ca00640 1 -- 172.21.15.87:0/1425134118 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe224003530 con 0x7fe2400cf9e0 2024-04-25T10:24:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23ca00640 1 -- 172.21.15.87:0/1425134118 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048519011 0 0) 0x7fe24015c6a0 con 0x7fe24015c0f0 2024-04-25T10:24:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23ca00640 1 -- 172.21.15.87:0/1425134118 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe24006c0e0 con 0x7fe24015c0f0 2024-04-25T10:24:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23de00640 1 -- 172.21.15.87:0/1425134118 >> v1:172.21.15.169:6789/0 conn(0x7fe24015c0f0 legacy=0x7fe2400b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23de00640 1 -- 172.21.15.87:0/1425134118 >> v1:172.21.15.87:6789/0 conn(0x7fe2400cf9e0 legacy=0x7fe24006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.804+0000 7fe23de00640 1 -- 172.21.15.87:0/1425134118 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24015c980 con 0x7fe24015bd10 2024-04-25T10:24:53.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe23de00640 1 --2- 172.21.15.87:0/1425134118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2400b4610 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe2300097c0 tx=0x7fe23001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aafcda50d0f3a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe23ca00640 1 -- 172.21.15.87:0/1425134118 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300aa5f0 con 0x7fe24015bd10 2024-04-25T10:24:53.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe23ca00640 1 -- 172.21.15.87:0/1425134118 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2300af070 con 0x7fe24015bd10 2024-04-25T10:24:53.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe23ca00640 1 -- 172.21.15.87:0/1425134118 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300aa9b0 con 0x7fe24015bd10 2024-04-25T10:24:53.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe244e00640 1 -- 172.21.15.87:0/1425134118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 msgr2=0x7fe2400b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe244e00640 1 --2- 172.21.15.87:0/1425134118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2400b4610 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe2300097c0 tx=0x7fe23001c470 comp rx=0 tx=0).stop 2024-04-25T10:24:53.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe244e00640 1 -- 172.21.15.87:0/1425134118 shutdown_connections 2024-04-25T10:24:53.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe244e00640 1 --2- 172.21.15.87:0/1425134118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2400b4610 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe244e00640 1 -- 172.21.15.87:0/1425134118 >> 172.21.15.87:0/1425134118 conn(0x7fe240066b70 msgr2=0x7fe240066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe244e00640 1 -- 172.21.15.87:0/1425134118 shutdown_connections 2024-04-25T10:24:53.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.808+0000 7fe244e00640 1 -- 172.21.15.87:0/1425134118 wait complete. 2024-04-25T10:24:53.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 Processor -- start 2024-04-25T10:24:53.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 -- start start 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2400cf9e0 0x7fe24014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2401487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe24015c0f0 0x7fe240153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400d0200 con 0x7fe24015c0f0 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24016a2e0 con 0x7fe24015bd10 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24014b610 con 0x7fe2400cf9e0 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe24015c0f0 0x7fe240153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe24015c0f0 0x7fe240153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44022/0 (socket says 172.21.15.87:44022) 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 -- 172.21.15.87:0/1914081937 learned_addr learned my addr 172.21.15.87:0/1914081937 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23d400640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2401487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23de00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2400cf9e0 0x7fe24014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 -- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2400cf9e0 msgr2=0x7fe24014d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2400cf9e0 0x7fe24014d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 -- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 msgr2=0x7fe2401487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2401487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 -- 172.21.15.87:0/1914081937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400af130 con 0x7fe24015c0f0 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23d400640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2401487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe23e800640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe24015c0f0 0x7fe240153d80 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fe23800cbd0 tx=0x7fe23800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe236000640 1 -- 172.21.15.87:0/1914081937 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238017070 con 0x7fe24015c0f0 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe236000640 1 -- 172.21.15.87:0/1914081937 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23800ace0 con 0x7fe24015c0f0 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe236000640 1 -- 172.21.15.87:0/1914081937 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012620 con 0x7fe24015c0f0 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2400af410 con 0x7fe24015c0f0 2024-04-25T10:24:53.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.812+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2400af9d0 con 0x7fe24015c0f0 2024-04-25T10:24:53.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.816+0000 7fe236000640 1 -- 172.21.15.87:0/1914081937 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe23801a070 con 0x7fe24015c0f0 2024-04-25T10:24:53.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.816+0000 7fe236000640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe220066f90 0x7fe220069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.816+0000 7fe236000640 1 -- 172.21.15.87:0/1914081937 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe238084f80 con 0x7fe24015c0f0 2024-04-25T10:24:53.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.816+0000 7fe23de00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe220066f90 0x7fe220069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.816+0000 7fe248202000 1 Processor -- start 2024-04-25T10:24:53.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.816+0000 7fe248202000 1 -- start start 2024-04-25T10:24:53.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.816+0000 7fe23de00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe220066f90 0x7fe220069450 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ae000 tx=0x7fe2300a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e799160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e799a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783e67bf40 con 0x55783e7996a0 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783e5e3070 con 0x55783e7e30f0 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783e56b1a0 con 0x55783e7e3a10 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e799a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e799a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44034/0 (socket says 172.21.15.87:44034) 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 -- 172.21.15.87:0/1750094801 learned_addr learned my addr 172.21.15.87:0/1750094801 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23d400640 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e799160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23de00640 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 -- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 msgr2=0x55783e799160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e799160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 -- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 msgr2=0x55783e7e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7e34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 -- 172.21.15.87:0/1750094801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55783e569510 con 0x55783e7996a0 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe23e800640 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e799a80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fe23805da60 tx=0x7fe238055a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3877f0c5b2e22bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe22aa00640 1 -- 172.21.15.87:0/1750094801 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2380539d0 con 0x55783e7996a0 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe22aa00640 1 -- 172.21.15.87:0/1750094801 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe238053c30 con 0x55783e7996a0 2024-04-25T10:24:53.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe22aa00640 1 -- 172.21.15.87:0/1750094801 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23805db00 con 0x55783e7996a0 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- 172.21.15.87:0/1750094801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 msgr2=0x55783e799a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e799a80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fe23805da60 tx=0x7fe238055a70 comp rx=0 tx=0).stop 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- 172.21.15.87:0/1750094801 shutdown_connections 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e799a80 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e799160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 --2- 172.21.15.87:0/1750094801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- 172.21.15.87:0/1750094801 >> 172.21.15.87:0/1750094801 conn(0x55783e791240 msgr2=0x55783e793630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- 172.21.15.87:0/1750094801 shutdown_connections 2024-04-25T10:24:53.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- 172.21.15.87:0/1750094801 wait complete. 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 -- start start 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.820+0000 7fe248202000 1 Processor -- start 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e7ea450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7ea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783e59d6c0 con 0x55783e7996a0 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783e5df110 con 0x55783e7e30f0 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783e699f20 con 0x55783e7e3a10 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 10 client.0 _ll_register_callbacks cb 0x55783e780570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e7ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e7ea450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44036/0 (socket says 172.21.15.87:44036) 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 -- 172.21.15.87:0/2584348295 learned_addr learned my addr 172.21.15.87:0/2584348295 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23d400640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23e800640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 -- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 msgr2=0x55783e7d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 -- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 msgr2=0x55783e7ea990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7ea990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 -- 172.21.15.87:0/2584348295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55783e56b3b0 con 0x55783e7996a0 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23d400640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23e800640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23de00640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e7ea450 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ac040 tx=0x7fe23001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be04253b8c434772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 1 -- 172.21.15.87:0/2584348295 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300ce060 con 0x55783e7996a0 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 1 -- 172.21.15.87:0/2584348295 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2300aac30 con 0x55783e7996a0 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe238053c30 con 0x55783e7996a0 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 1 -- 172.21.15.87:0/2584348295 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2300d4050 con 0x55783e7996a0 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 10 client.10680 fetch_fsmap learned FSMap version 119 2024-04-25T10:24:53.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55783e664e00 con 0x55783e7996a0 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 1 -- 172.21.15.87:0/2584348295 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe238053c30 con 0x55783e7996a0 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 1 -- 172.21.15.87:0/2584348295 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe2300c8070 con 0x55783e7996a0 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 10 client.10680 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 10 client.10680 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 20 client.10680 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7d1a60 0x55783e7d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 4 client.10680 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55783e57ed60 con 0x55783e7d1a60 2024-04-25T10:24:53.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23d400640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7d1a60 0x55783e7d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 10 client.10680 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe23d400640 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7d1a60 0x55783e7d1e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a27838e07bf7697 server_cookie=fe483f58f950b1b1 in_seq=0 out_seq=0 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 1 -- 172.21.15.87:0/2584348295 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55783e57ed60 con 0x55783e7d1a60 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe21f400640 10 client.10680 handle_command_reply: tid=0 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 client.10680 shutdown 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 msgr2=0x55783e7ea450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e7ea450 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ac040 tx=0x7fe23001acb0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 shutdown_connections 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7e30f0 0x55783e7ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.824+0000 7fe248202000 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7d1a60 0x55783e7d1e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.828+0000 7fe248202000 1 --2- 172.21.15.87:0/2584348295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7996a0 0x55783e7ea450 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.828+0000 7fe21f400640 0 client.10680 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:53.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.828+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 >> 172.21.15.87:0/2584348295 conn(0x55783e791240 msgr2=0x55783e79a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.828+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 shutdown_connections 2024-04-25T10:24:53.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.828+0000 7fe248202000 1 -- 172.21.15.87:0/2584348295 wait complete. 2024-04-25T10:24:53.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.828+0000 7fe248202000 20 client.10680 trim_cache size 0 max 16384 2024-04-25T10:24:53.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.840+0000 7fe248202000 1 Processor -- start 2024-04-25T10:24:53.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.840+0000 7fe248202000 1 -- start start 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.840+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e79a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.840+0000 7fe248202000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e30f0 0x55783e79a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.840+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240071b10 con 0x55783e7e30f0 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240070d10 con 0x55783e7996a0 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2401f9720 con 0x55783e7e3a10 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e30f0 0x55783e79a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e30f0 0x55783e79a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44050/0 (socket says 172.21.15.87:44050) 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 -- 172.21.15.87:0/3283900367 learned_addr learned my addr 172.21.15.87:0/3283900367 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23e800640 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23de00640 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e79a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 -- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 msgr2=0x55783e7e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 -- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 msgr2=0x55783e79a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e79a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 -- 172.21.15.87:0/3283900367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55783e63ac50 con 0x55783e7e30f0 2024-04-25T10:24:53.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e30f0 0x55783e79a6d0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0054a0 tx=0x7fe22c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530a054da19d8cad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe22a000640 1 -- 172.21.15.87:0/3283900367 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c01a230 con 0x55783e7e30f0 2024-04-25T10:24:53.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe22a000640 1 -- 172.21.15.87:0/3283900367 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c01a3d0 con 0x55783e7e30f0 2024-04-25T10:24:53.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe22a000640 1 -- 172.21.15.87:0/3283900367 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c01a7a0 con 0x55783e7e30f0 2024-04-25T10:24:53.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- 172.21.15.87:0/3283900367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e30f0 msgr2=0x55783e79a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e30f0 0x55783e79a6d0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0054a0 tx=0x7fe22c000ee0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- 172.21.15.87:0/3283900367 shutdown_connections 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e3a10 0x55783e7e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e30f0 0x55783e79a6d0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 --2- 172.21.15.87:0/3283900367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e79a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- 172.21.15.87:0/3283900367 >> 172.21.15.87:0/3283900367 conn(0x55783e791240 msgr2=0x55783e799a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- 172.21.15.87:0/3283900367 shutdown_connections 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- 172.21.15.87:0/3283900367 wait complete. 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- start start 2024-04-25T10:24:53.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 Processor -- start 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e843d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e30f0 0x55783e8442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e3a10 0x55783e7e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783e5e2380 con 0x55783e7e3a10 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240070710 con 0x55783e7996a0 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783e79e200 con 0x55783e7e30f0 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe248202000 10 client.0 _ll_register_callbacks cb 0x7fe23800ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e3a10 0x55783e7e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e3a10 0x55783e7e8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44058/0 (socket says 172.21.15.87:44058) 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23e800640 1 -- 172.21.15.87:0/1709738429 learned_addr learned my addr 172.21.15.87:0/1709738429 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23de00640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e843d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.844+0000 7fe23d400640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e30f0 0x55783e8442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23e800640 1 -- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e30f0 msgr2=0x55783e8442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23e800640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e30f0 0x55783e8442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23e800640 1 -- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 msgr2=0x55783e843d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23e800640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e843d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23e800640 1 -- 172.21.15.87:0/1709738429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55783e5ab120 con 0x55783e7e3a10 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23d400640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e30f0 0x55783e8442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23e800640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e3a10 0x55783e7e8120 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fe238055870 tx=0x7fe2380014e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1cbaef5df759b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 1 -- 172.21.15.87:0/1709738429 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23805dd10 con 0x55783e7e3a10 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 1 -- 172.21.15.87:0/1709738429 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe23807ce00 con 0x55783e7e3a10 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe22c01a3d0 con 0x55783e7e3a10 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 1 -- 172.21.15.87:0/1709738429 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe23806d170 con 0x55783e7e3a10 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 10 client.10686 fetch_fsmap learned FSMap version 119 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55783e58a170 con 0x55783e7e3a10 2024-04-25T10:24:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 1 -- 172.21.15.87:0/1709738429 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c01a3d0 con 0x55783e7e3a10 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 1 -- 172.21.15.87:0/1709738429 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe238056bb0 con 0x55783e7e3a10 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 10 client.10686 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 10 client.10686 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 20 client.10686 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7e8b10 0x55783e7e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 4 client.10686 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55783e5f05b0 con 0x55783e7e8b10 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23de00640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7e8b10 0x55783e7e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe23de00640 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7e8b10 0x55783e7e8f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ab00c3b2d49289 server_cookie=1281c74cc28057e5 in_seq=0 out_seq=0 2024-04-25T10:24:53.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 10 client.10686 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 1 -- 172.21.15.87:0/1709738429 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55783e5f05b0 con 0x55783e7e8b10 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 10 client.10686 handle_command_reply: tid=0 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 client.10686 shutdown 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e3a10 msgr2=0x55783e7e8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e3a10 0x55783e7e8120 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fe238055870 tx=0x7fe2380014e0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 shutdown_connections 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55783e7e8b10 0x55783e7e8f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55783e7e3a10 0x55783e7e8120 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55783e7e30f0 0x55783e8442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 --2- 172.21.15.87:0/1709738429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55783e7996a0 0x55783e843d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.852 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-04-25T10:24:53.848+0000 7f7824200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:24:52.668148+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={8435=0-12582912@1} caps={8435=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=8435 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x56384b0e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(25)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:24:53.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 >> 172.21.15.87:0/1709738429 conn(0x55783e791240 msgr2=0x55783e79a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe21f400640 0 client.10686 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 shutdown_connections 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 1 -- 172.21.15.87:0/1709738429 wait complete. 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.848+0000 7fe248202000 20 client.10686 trim_cache size 0 max 16384 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "e85be63c-6ef5-46dc-a055-86453e16e092", 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:24:53.853 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:24:53.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe220066f90 msgr2=0x7fe220069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe220066f90 0x7fe220069450 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ae000 tx=0x7fe2300a7000 comp rx=0 tx=0).stop 2024-04-25T10:24:53.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe24015c0f0 msgr2=0x7fe240153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe24015c0f0 0x7fe240153d80 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fe23800cbd0 tx=0x7fe23800a430 comp rx=0 tx=0).stop 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 shutdown_connections 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe220066f90 0x7fe220069450 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe24015c0f0 0x7fe240153d80 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24015bd10 0x7fe2401487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 --2- 172.21.15.87:0/1914081937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2400cf9e0 0x7fe24014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 >> 172.21.15.87:0/1914081937 conn(0x7fe240066b70 msgr2=0x7fe2400b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 shutdown_connections 2024-04-25T10:24:53.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:53.852+0000 7fe244e00640 1 -- 172.21.15.87:0/1914081937 wait complete. 2024-04-25T10:24:53.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:53.868 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:24:54.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f949ca00640 1 Processor -- start 2024-04-25T10:24:54.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f949ca00640 1 -- start start 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f949ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94980aef80 0x7f949815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f949ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f94980b31c0 0x7f94980b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f949ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949815b080 con 0x7f9498070350 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f949ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949815adb0 con 0x7f94980b31c0 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f949ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949815eb60 con 0x7f94980aef80 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94980aef80 0x7f949815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94980aef80 0x7f949815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:54578/0 (socket says 172.21.15.87:54578) 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495e00640 1 -- 172.21.15.87:0/3988610541 learned_addr learned my addr 172.21.15.87:0/3988610541 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495400640 1 --1- 172.21.15.87:0/3988610541 >> v1:172.21.15.169:6789/0 conn(0x7f9498070350 0x7f949815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:55146/0 (socket says 172.21.15.87:55146) 2024-04-25T10:24:54.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9496800640 1 --2- 172.21.15.87:0/3988610541 >> v2:172.21.15.169:3300/0 conn(0x7f94980b31c0 0x7f94980b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495e00640 1 -- 172.21.15.87:0/3988610541 >> v2:172.21.15.169:3300/0 conn(0x7f94980b31c0 msgr2=0x7f94980b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495e00640 1 --2- 172.21.15.87:0/3988610541 >> v2:172.21.15.169:3300/0 conn(0x7f94980b31c0 0x7f94980b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495e00640 1 -- 172.21.15.87:0/3988610541 >> v1:172.21.15.169:6789/0 conn(0x7f9498070350 legacy=0x7f949815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.008+0000 7f9495e00640 1 -- 172.21.15.87:0/3988610541 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949815ee40 con 0x7f94980aef80 2024-04-25T10:24:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f9495e00640 1 --2- 172.21.15.87:0/3988610541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94980aef80 0x7f949815e480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9488009e50 tx=0x7f948801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0af9d71b057fddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f9494a00640 1 -- 172.21.15.87:0/3988610541 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880a5810 con 0x7f94980aef80 2024-04-25T10:24:54.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f9494a00640 1 -- 172.21.15.87:0/3988610541 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94880a59b0 con 0x7f94980aef80 2024-04-25T10:24:54.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f9494a00640 1 -- 172.21.15.87:0/3988610541 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880a5d80 con 0x7f94980aef80 2024-04-25T10:24:54.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 -- 172.21.15.87:0/3988610541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94980aef80 msgr2=0x7f949815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 --2- 172.21.15.87:0/3988610541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94980aef80 0x7f949815e480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9488009e50 tx=0x7f948801ca30 comp rx=0 tx=0).stop 2024-04-25T10:24:54.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 -- 172.21.15.87:0/3988610541 shutdown_connections 2024-04-25T10:24:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 --2- 172.21.15.87:0/3988610541 >> v2:172.21.15.169:3300/0 conn(0x7f94980b31c0 0x7f94980b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 --2- 172.21.15.87:0/3988610541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94980aef80 0x7f949815e480 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 -- 172.21.15.87:0/3988610541 >> 172.21.15.87:0/3988610541 conn(0x7f94980669a0 msgr2=0x7f9498066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 -- 172.21.15.87:0/3988610541 shutdown_connections 2024-04-25T10:24:54.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.012+0000 7f949ca00640 1 -- 172.21.15.87:0/3988610541 wait complete. 2024-04-25T10:24:54.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 Processor -- start 2024-04-25T10:24:54.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 -- start start 2024-04-25T10:24:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9498070350 0x7f94981596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94980aef80 0x7f94980adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94980b31c0 0x7f94981578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949806c250 con 0x7f94980aef80 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949806c0d0 con 0x7f94980b31c0 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949806c3d0 con 0x7f9498070350 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94980aef80 0x7f94980adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94980aef80 0x7f94980adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44066/0 (socket says 172.21.15.87:44066) 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 -- 172.21.15.87:0/3599171483 learned_addr learned my addr 172.21.15.87:0/3599171483 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9496800640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94980b31c0 0x7f94981578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495e00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9498070350 0x7f94981596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 -- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9498070350 msgr2=0x7f94981596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9498070350 0x7f94981596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 -- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94980b31c0 msgr2=0x7f94981578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94980b31c0 0x7f94981578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 -- 172.21.15.87:0/3599171483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498159fe0 con 0x7f94980aef80 2024-04-25T10:24:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f9495400640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94980aef80 0x7f94980adb10 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f948400c450 tx=0x7f948400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f948e000640 1 -- 172.21.15.87:0/3599171483 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484017070 con 0x7f94980aef80 2024-04-25T10:24:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f948e000640 1 -- 172.21.15.87:0/3599171483 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9484006480 con 0x7f94980aef80 2024-04-25T10:24:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f948e000640 1 -- 172.21.15.87:0/3599171483 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94840049f0 con 0x7f94980aef80 2024-04-25T10:24:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949815a2c0 con 0x7f94980aef80 2024-04-25T10:24:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.016+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9498154be0 con 0x7f94980aef80 2024-04-25T10:24:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f948e000640 1 -- 172.21.15.87:0/3599171483 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f948401a070 con 0x7f94980aef80 2024-04-25T10:24:54.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f948e000640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9478066f90 0x7f9478069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f948e000640 1 -- 172.21.15.87:0/3599171483 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94840842d0 con 0x7f94980aef80 2024-04-25T10:24:54.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9495e00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9478066f90 0x7f9478069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 Processor -- start 2024-04-25T10:24:54.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 -- start start 2024-04-25T10:24:54.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9495e00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9478066f90 0x7f9478069450 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f94880ae000 tx=0x7f94880a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718335db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b7182ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b7182ec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b718138ff0 con 0x55b7182ec0a0 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7180c1150 con 0x55b7183362f0 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b718135090 con 0x55b7183359d0 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b7182ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b7182ec480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44082/0 (socket says 172.21.15.87:44082) 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 -- 172.21.15.87:0/3044911027 learned_addr learned my addr 172.21.15.87:0/3044911027 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9495400640 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b7182ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9495e00640 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718335db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 -- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 msgr2=0x55b718335db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718335db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 -- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 msgr2=0x55b7182ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b7182ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9496800640 1 -- 172.21.15.87:0/3044911027 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7182cf0d0 con 0x55b7182ec0a0 2024-04-25T10:24:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.020+0000 7f9495e00640 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718335db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9496800640 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b7182ec480 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f9490009b80 tx=0x7f9490011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc1b4822f5db6d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9482a00640 1 -- 172.21.15.87:0/3044911027 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490014440 con 0x55b7182ec0a0 2024-04-25T10:24:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9482a00640 1 -- 172.21.15.87:0/3044911027 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900145e0 con 0x55b7182ec0a0 2024-04-25T10:24:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9482a00640 1 -- 172.21.15.87:0/3044911027 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490014990 con 0x55b7182ec0a0 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- 172.21.15.87:0/3044911027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 msgr2=0x55b7182ec480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b7182ec480 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f9490009b80 tx=0x7f9490011690 comp rx=0 tx=0).stop 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- 172.21.15.87:0/3044911027 shutdown_connections 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b7182ec480 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b7182ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 --2- 172.21.15.87:0/3044911027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718335db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- 172.21.15.87:0/3044911027 >> 172.21.15.87:0/3044911027 conn(0x55b7182e8020 msgr2=0x55b7182ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- 172.21.15.87:0/3044911027 shutdown_connections 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- 172.21.15.87:0/3044911027 wait complete. 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- start start 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 Processor -- start 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b71831a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183359d0 0x55b71831ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183362f0 0x55b71831b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7180fd250 con 0x55b7183359d0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7180f3690 con 0x55b7182ec0a0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7180c1a40 con 0x55b7183362f0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f949fe02000 10 client.0 _ll_register_callbacks cb 0x55b7182d6e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183359d0 0x55b71831ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183359d0 0x55b71831ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44096/0 (socket says 172.21.15.87:44096) 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 -- 172.21.15.87:0/2743334537 learned_addr learned my addr 172.21.15.87:0/2743334537 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495e00640 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b71831a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9496800640 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183362f0 0x55b71831b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 -- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183362f0 msgr2=0x55b71831b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183362f0 0x55b71831b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 -- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 msgr2=0x55b71831a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b71831a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 -- 172.21.15.87:0/2743334537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7181e61c0 con 0x55b7183359d0 2024-04-25T10:24:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9495400640 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183359d0 0x55b71831ad30 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f9484082760 tx=0x7f9484054d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb55f76ea47b2708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.024+0000 7f9477400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 1 -- 172.21.15.87:0/2743334537 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948405cfc0 con 0x55b7183359d0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 1 -- 172.21.15.87:0/2743334537 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9484059bb0 con 0x55b7183359d0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94900145e0 con 0x55b7183359d0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 1 -- 172.21.15.87:0/2743334537 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9484059db0 con 0x55b7183359d0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 10 client.10695 fetch_fsmap learned FSMap version 119 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55b7181e2170 con 0x55b7183359d0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 1 -- 172.21.15.87:0/2743334537 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900145e0 con 0x55b7183359d0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 1 -- 172.21.15.87:0/2743334537 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9484004b90 con 0x55b7183359d0 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 10 client.10695 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 10 client.10695 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:24:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 20 client.10695 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b718345c80 0x55b718348070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 4 client.10695 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7180d4c60 con 0x55b718345c80 2024-04-25T10:24:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9495e00640 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b718345c80 0x55b718348070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9495e00640 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b718345c80 0x55b718348070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c284ef9c42b075 server_cookie=384c66b2dea405ca in_seq=0 out_seq=0 2024-04-25T10:24:54.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 10 client.10695 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:54.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 1 -- 172.21.15.87:0/2743334537 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7180d4c60 con 0x55b718345c80 2024-04-25T10:24:54.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 10 client.10695 handle_command_reply: tid=0 2024-04-25T10:24:54.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 client.10695 shutdown 2024-04-25T10:24:54.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183359d0 msgr2=0x55b71831ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183359d0 0x55b71831ad30 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f9484082760 tx=0x7f9484054d30 comp rx=0 tx=0).stop 2024-04-25T10:24:54.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 shutdown_connections 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183362f0 0x55b71831b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b718345c80 0x55b718348070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183359d0 0x55b71831ad30 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 --2- 172.21.15.87:0/2743334537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b71831a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f9477400640 0 client.10695 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 >> 172.21.15.87:0/2743334537 conn(0x55b7182e8020 msgr2=0x55b7182eaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 shutdown_connections 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 1 -- 172.21.15.87:0/2743334537 wait complete. 2024-04-25T10:24:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.028+0000 7f949fe02000 20 client.10695 trim_cache size 0 max 16384 2024-04-25T10:24:54.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 Processor -- start 2024-04-25T10:24:54.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 -- start start 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b7183920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718392630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183362f0 0x55b718392b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498203220 con 0x55b7183362f0 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498156370 con 0x55b7182ec0a0 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9498152990 con 0x55b7183359d0 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183362f0 0x55b718392b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183362f0 0x55b718392b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44106/0 (socket says 172.21.15.87:44106) 2024-04-25T10:24:54.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 -- 172.21.15.87:0/3498169587 learned_addr learned my addr 172.21.15.87:0/3498169587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9495e00640 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b7183920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9495400640 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718392630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 -- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 msgr2=0x55b718392630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718392630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 -- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 msgr2=0x55b7183920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b7183920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 -- 172.21.15.87:0/3498169587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71817fde0 con 0x55b7183362f0 2024-04-25T10:24:54.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9496800640 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183362f0 0x55b718392b70 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f9490019000 tx=0x7f9490016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676a1689ae349291 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:54.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9482000640 1 -- 172.21.15.87:0/3498169587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490004230 con 0x55b7183362f0 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9482000640 1 -- 172.21.15.87:0/3498169587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490006a30 con 0x55b7183362f0 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f9482000640 1 -- 172.21.15.87:0/3498169587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490003620 con 0x55b7183362f0 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 -- 172.21.15.87:0/3498169587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183362f0 msgr2=0x55b718392b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183362f0 0x55b718392b70 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f9490019000 tx=0x7f9490016000 comp rx=0 tx=0).stop 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 -- 172.21.15.87:0/3498169587 shutdown_connections 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7183362f0 0x55b718392b70 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b718392630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.044+0000 7f949fe02000 1 --2- 172.21.15.87:0/3498169587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7182ec0a0 0x55b7183920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- 172.21.15.87:0/3498169587 >> 172.21.15.87:0/3498169587 conn(0x55b7182e8020 msgr2=0x55b718342fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- 172.21.15.87:0/3498169587 shutdown_connections 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- 172.21.15.87:0/3498169587 wait complete. 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- start start 2024-04-25T10:24:54.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 Processor -- start 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b718348380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b7183488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b718348e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7180d3e00 con 0x55b7182ec0a0 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498155d70 con 0x55b7183362f0 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b718138300 con 0x55b7183359d0 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 10 client.0 _ll_register_callbacks cb 0x55b7182b7040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b718348380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b718348380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44120/0 (socket says 172.21.15.87:44120) 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 -- 172.21.15.87:0/3315546156 learned_addr learned my addr 172.21.15.87:0/3315546156 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495400640 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b7183488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9496800640 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b718348e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 -- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 msgr2=0x55b7183488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b7183488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 -- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 msgr2=0x55b718348e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b718348e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 -- 172.21.15.87:0/3315546156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7180da8e0 con 0x55b7182ec0a0 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495e00640 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b718348380 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f94880b9040 tx=0x7f948801b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f259ac3bf44b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:54.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9477400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9477400640 1 -- 172.21.15.87:0/3315546156 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880a58f0 con 0x55b7182ec0a0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9477400640 1 -- 172.21.15.87:0/3315546156 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94880a5a90 con 0x55b7182ec0a0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9490006a30 con 0x55b7182ec0a0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9477400640 1 -- 172.21.15.87:0/3315546156 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94880d7050 con 0x55b7182ec0a0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 10 client.10701 fetch_fsmap learned FSMap version 119 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55b7181928f0 con 0x55b7182ec0a0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9477400640 1 -- 172.21.15.87:0/3315546156 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490006a30 con 0x55b7182ec0a0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9477400640 1 -- 172.21.15.87:0/3315546156 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f94880da020 con 0x55b7182ec0a0 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 10 client.10701 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 10 client.10701 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 20 client.10701 populate_metadata read hostname 'smithi087' 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b71839bf50 0x55b71839e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 4 client.10701 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b718146530 con 0x55b71839bf50 2024-04-25T10:24:54.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.048+0000 7f9495400640 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b71839bf50 0x55b71839e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f9495400640 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b71839bf50 0x55b71839e340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a051a9a1a16b5ed2 server_cookie=e2fa14d514f6bf87 in_seq=0 out_seq=0 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f9477400640 10 client.10701 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f9477400640 1 -- 172.21.15.87:0/3315546156 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55b718146530 con 0x55b71839bf50 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f9477400640 10 client.10701 handle_command_reply: tid=0 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 client.10701 shutdown 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 msgr2=0x55b718348380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b718348380 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f94880b9040 tx=0x7f948801b270 comp rx=0 tx=0).stop 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 shutdown_connections 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b7183362f0 0x55b718348e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b7183359d0 0x55b7183488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x55b71839bf50 0x55b71839e340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 --2- 172.21.15.87:0/3315546156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7182ec0a0 0x55b718348380 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 >> 172.21.15.87:0/3315546156 conn(0x55b7182e8020 msgr2=0x55b7183340a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:54.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f9477400640 0 client.10701 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 shutdown_connections 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 1 -- 172.21.15.87:0/3315546156 wait complete. 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949fe02000 20 client.10701 trim_cache size 0 max 16384 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stdout: "e85be63c-6ef5-46dc-a055-86453e16e092": { 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/", 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "e85be63c-6ef5-46dc-a055-86453e16e092", 2024-04-25T10:24:54.055 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive,repair" 2024-04-25T10:24:54.056 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:54.056 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:24:54.056 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:24:54.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9478066f90 msgr2=0x7f9478069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9478066f90 0x7f9478069450 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f94880ae000 tx=0x7f94880a7070 comp rx=0 tx=0).stop 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94980aef80 msgr2=0x7f94980adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94980aef80 0x7f94980adb10 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f948400c450 tx=0x7f948400c910 comp rx=0 tx=0).stop 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 shutdown_connections 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9478066f90 0x7f9478069450 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94980b31c0 0x7f94981578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94980aef80 0x7f94980adb10 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 --2- 172.21.15.87:0/3599171483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9498070350 0x7f94981596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 >> 172.21.15.87:0/3599171483 conn(0x7f94980669a0 msgr2=0x7f9498066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 shutdown_connections 2024-04-25T10:24:54.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:24:54.052+0000 7f949ca00640 1 -- 172.21.15.87:0/3599171483 wait complete. 2024-04-25T10:24:54.069 INFO:tasks.cephfs.filesystem:scrub status for tag:e85be63c-6ef5-46dc-a055-86453e16e092 - {'path': '/', 'tag': 'e85be63c-6ef5-46dc-a055-86453e16e092', 'options': 'recursive,repair'} 2024-04-25T10:24:54.307 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:24:54.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:24.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:24.072 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:25:24.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3067000640 1 Processor -- start 2024-04-25T10:25:24.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3067000640 1 -- start start 2024-04-25T10:25:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3067000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 0x7f306015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3067000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30600b31c0 0x7f30600b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3067000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306015b080 con 0x7f30600aef80 2024-04-25T10:25:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3067000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306015adb0 con 0x7f30600b31c0 2024-04-25T10:25:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3067000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306015eb60 con 0x7f3060070350 2024-04-25T10:25:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f305fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f30600aef80 0x7f306015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:51546/0 (socket says 172.21.15.87:51546) 2024-04-25T10:25:24.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f305fe00640 1 -- 172.21.15.87:0/1764013844 learned_addr learned my addr 172.21.15.87:0/1764013844 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3065000640 1 --2- 172.21.15.87:0/1764013844 >> v2:172.21.15.169:3300/0 conn(0x7f30600b31c0 0x7f30600b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.223+0000 7f3057400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 0x7f306015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381759371 0 0) 0x7f306015b080 con 0x7f30600aef80 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3044003530 con 0x7f30600aef80 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824808475 0 0) 0x7f3044003530 con 0x7f30600aef80 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 msgr2=0x7f306015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 0x7f306015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 >> v2:172.21.15.169:3300/0 conn(0x7f30600b31c0 msgr2=0x7f30600b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 --2- 172.21.15.87:0/1764013844 >> v2:172.21.15.169:3300/0 conn(0x7f30600b31c0 0x7f30600b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306015ee40 con 0x7f30600aef80 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f3050001a00 con 0x7f30600aef80 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3065000640 1 --2- 172.21.15.87:0/1764013844 >> v2:172.21.15.169:3300/0 conn(0x7f30600b31c0 0x7f30600b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3057400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 0x7f306015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 >> v1:172.21.15.87:6789/0 conn(0x7f30600aef80 legacy=0x7f306015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30440039c0 0x7f3044003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3044004810 0x7f3044004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 0x7f30440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306015adb0 con 0x7f3044004810 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306015eb60 con 0x7f30440039c0 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304400b9d0 con 0x7f3060070350 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3057400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3044004810 0x7f3044004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f305fe00640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30440039c0 0x7f3044003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3065000640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 0x7f30440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3057400640 1 -- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 msgr2=0x7f30440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3057400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070350 0x7f30440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3057400640 1 -- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30440039c0 msgr2=0x7f3044003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3057400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30440039c0 0x7f3044003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.227+0000 7f3057400640 1 -- 172.21.15.87:0/1764013844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044003530 con 0x7f3044004810 2024-04-25T10:25:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3057400640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3044004810 0x7f3044004c90 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f3048009cb0 tx=0x7f304801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f0a4a6bbd1c7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480b2070 con 0x7f3044004810 2024-04-25T10:25:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30480a56e0 con 0x7f3044004810 2024-04-25T10:25:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f305f400640 1 -- 172.21.15.87:0/1764013844 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048009880 con 0x7f3044004810 2024-04-25T10:25:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- 172.21.15.87:0/1764013844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3044004810 msgr2=0x7f3044004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 --2- 172.21.15.87:0/1764013844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3044004810 0x7f3044004c90 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f3048009cb0 tx=0x7f304801cc10 comp rx=0 tx=0).stop 2024-04-25T10:25:24.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3065000640 1 -- 172.21.15.87:0/1764013844 reap_dead start 2024-04-25T10:25:24.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- 172.21.15.87:0/1764013844 shutdown_connections 2024-04-25T10:25:24.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- 172.21.15.87:0/1764013844 >> 172.21.15.87:0/1764013844 conn(0x7f30600669a0 msgr2=0x7f3060066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- 172.21.15.87:0/1764013844 shutdown_connections 2024-04-25T10:25:24.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- 172.21.15.87:0/1764013844 wait complete. 2024-04-25T10:25:24.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 Processor -- start 2024-04-25T10:25:24.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- start start 2024-04-25T10:25:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3044004810 0x7f3060148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f306006cc60 0x7f30600af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060155300 0x7f3060155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306015bc90 con 0x7f3060155300 2024-04-25T10:25:24.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060071a00 con 0x7f306006cc60 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.231+0000 7f3067000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306015e480 con 0x7f3044004810 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060155300 0x7f3060155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060155300 0x7f3060155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48340/0 (socket says 172.21.15.87:48340) 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 -- 172.21.15.87:0/3046177964 learned_addr learned my addr 172.21.15.87:0/3046177964 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3057400640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f306006cc60 0x7f30600af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f305fe00640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3044004810 0x7f3060148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 -- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3044004810 msgr2=0x7f3060148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3044004810 0x7f3060148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 -- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f306006cc60 msgr2=0x7f30600af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f306006cc60 0x7f30600af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 -- 172.21.15.87:0/3046177964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600afce0 con 0x7f3060155300 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3065000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060155300 0x7f3060155780 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f30600b2f70 tx=0x7f3058011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f305cc00640 1 -- 172.21.15.87:0/3046177964 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058019070 con 0x7f3060155300 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f305cc00640 1 -- 172.21.15.87:0/3046177964 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058004230 con 0x7f3060155300 2024-04-25T10:25:24.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f305cc00640 1 -- 172.21.15.87:0/3046177964 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305800f6b0 con 0x7f3060155300 2024-04-25T10:25:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30600affc0 con 0x7f3060155300 2024-04-25T10:25:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30600b0580 con 0x7f3060155300 2024-04-25T10:25:24.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.235+0000 7f305cc00640 1 -- 172.21.15.87:0/3046177964 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f305801c070 con 0x7f3060155300 2024-04-25T10:25:24.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f305cc00640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f303c066f90 0x7f303c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f305cc00640 1 -- 172.21.15.87:0/3046177964 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f30580852c0 con 0x7f3060155300 2024-04-25T10:25:24.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f305fe00640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f303c066f90 0x7f303c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 Processor -- start 2024-04-25T10:25:24.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- start start 2024-04-25T10:25:24.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f305fe00640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f303c066f90 0x7f303c069450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3050001bf0 tx=0x7f3050001740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04929d0 0x557fe0492db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe0448b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe0449480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe0295ff0 con 0x557fe04490a0 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe021e150 con 0x557fe04932f0 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fe0292090 con 0x557fe04929d0 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe0449480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f305fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04929d0 0x557fe0492db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe0449480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48352/0 (socket says 172.21.15.87:48352) 2024-04-25T10:25:24.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 -- 172.21.15.87:0/2774043544 learned_addr learned my addr 172.21.15.87:0/2774043544 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3057400640 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe0448b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 -- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04929d0 msgr2=0x557fe0492db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04929d0 0x557fe0492db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 -- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 msgr2=0x557fe0448b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe0448b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 -- 172.21.15.87:0/2774043544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fe042c0d0 con 0x557fe04490a0 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3057400640 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe0448b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3065000640 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe0449480 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f3058001ec0 tx=0x7f3058055c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cc9590d5acc2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3054c00640 1 -- 172.21.15.87:0/2774043544 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058023040 con 0x557fe04490a0 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3054c00640 1 -- 172.21.15.87:0/2774043544 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305805e290 con 0x557fe04490a0 2024-04-25T10:25:24.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f3054c00640 1 -- 172.21.15.87:0/2774043544 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580850c0 con 0x557fe04490a0 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- 172.21.15.87:0/2774043544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 msgr2=0x557fe0449480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe0449480 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f3058001ec0 tx=0x7f3058055c30 comp rx=0 tx=0).stop 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- 172.21.15.87:0/2774043544 shutdown_connections 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe0449480 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe0448b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2774043544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04929d0 0x557fe0492db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- 172.21.15.87:0/2774043544 >> 172.21.15.87:0/2774043544 conn(0x557fe0445020 msgr2=0x557fe0447440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- 172.21.15.87:0/2774043544 shutdown_connections 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.239+0000 7f306a4b9000 1 -- 172.21.15.87:0/2774043544 wait complete. 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 -- start start 2024-04-25T10:25:24.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 Processor -- start 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe0477d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe0478270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe025a250 con 0x557fe04490a0 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe0250690 con 0x557fe04929d0 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fe021ea40 con 0x557fe04932f0 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 10 client.0 _ll_register_callbacks cb 0x557fe0433e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04777f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48358/0 (socket says 172.21.15.87:48358) 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 -- 172.21.15.87:0/1435748795 learned_addr learned my addr 172.21.15.87:0/1435748795 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f3057400640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe0477d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f3065000640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe0478270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 -- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 msgr2=0x557fe0478270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe0478270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 -- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 msgr2=0x557fe0477d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe0477d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 -- 172.21.15.87:0/1435748795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fe03431c0 con 0x557fe04490a0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f3065000640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe0478270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f3057400640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe0477d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f305fe00640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04777f0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f3050017490 tx=0x7f3050001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a29d6de190c64b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f304d600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f304d600640 1 -- 172.21.15.87:0/1435748795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050010040 con 0x557fe04490a0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f304d600640 1 -- 172.21.15.87:0/1435748795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f305001d890 con 0x557fe04490a0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f305805e290 con 0x557fe04490a0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f304d600640 1 -- 172.21.15.87:0/1435748795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f305001da90 con 0x557fe04490a0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 10 client.10713 fetch_fsmap learned FSMap version 119 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x557fe033f170 con 0x557fe04490a0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f304d600640 1 -- 172.21.15.87:0/1435748795 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050035070 con 0x557fe04490a0 2024-04-25T10:25:24.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f304d600640 1 -- 172.21.15.87:0/1435748795 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f305001dc30 con 0x557fe04490a0 2024-04-25T10:25:24.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 10 client.10713 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:25:24.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 10 client.10713 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:25:24.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 20 client.10713 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:24.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a2c80 0x557fe04a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 4 client.10713 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:24.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fe0231c60 con 0x557fe04a2c80 2024-04-25T10:25:24.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.243+0000 7f3057400640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a2c80 0x557fe04a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f3057400640 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a2c80 0x557fe04a5070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fefdb90f8e59f5 server_cookie=2fb13ee4fc6c116a in_seq=0 out_seq=0 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f304d600640 10 client.10713 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f304d600640 1 -- 172.21.15.87:0/1435748795 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fe0231c60 con 0x557fe04a2c80 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f304d600640 10 client.10713 handle_command_reply: tid=0 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 client.10713 shutdown 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 msgr2=0x557fe04777f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04777f0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f3050017490 tx=0x7f3050001dd0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 shutdown_connections 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe0478270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe0477d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a2c80 0x557fe04a5070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 --2- 172.21.15.87:0/1435748795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04777f0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 >> 172.21.15.87:0/1435748795 conn(0x557fe0445020 msgr2=0x557fe0447ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f304d600640 0 client.10713 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:25:24.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 shutdown_connections 2024-04-25T10:25:24.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 1 -- 172.21.15.87:0/1435748795 wait complete. 2024-04-25T10:25:24.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.247+0000 7f306a4b9000 20 client.10713 trim_cache size 0 max 16384 2024-04-25T10:25:24.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 Processor -- start 2024-04-25T10:25:24.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- start start 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe04ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe04ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060200890 con 0x557fe04490a0 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30601ffa70 con 0x557fe04929d0 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600b1d00 con 0x557fe04932f0 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48362/0 (socket says 172.21.15.87:48362) 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 -- 172.21.15.87:0/2033874648 learned_addr learned my addr 172.21.15.87:0/2033874648 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f3057400640 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe04ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f3065000640 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe04ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 -- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 msgr2=0x557fe04ef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe04ef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 -- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 msgr2=0x557fe04ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe04ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 -- 172.21.15.87:0/2033874648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fe02dcde0 con 0x557fe04490a0 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f305fe00640 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04eeee0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f3050001360 tx=0x7f3050026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f2d3fcd2adcd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f304fe00640 1 -- 172.21.15.87:0/2033874648 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050029020 con 0x557fe04490a0 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f304fe00640 1 -- 172.21.15.87:0/2033874648 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050035040 con 0x557fe04490a0 2024-04-25T10:25:24.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f304fe00640 1 -- 172.21.15.87:0/2033874648 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305001d220 con 0x557fe04490a0 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- 172.21.15.87:0/2033874648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 msgr2=0x557fe04eeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04eeee0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f3050001360 tx=0x7f3050026000 comp rx=0 tx=0).stop 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- 172.21.15.87:0/2033874648 shutdown_connections 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04932f0 0x557fe04ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04929d0 0x557fe04ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 --2- 172.21.15.87:0/2033874648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04490a0 0x557fe04eeee0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- 172.21.15.87:0/2033874648 >> 172.21.15.87:0/2033874648 conn(0x557fe0445020 msgr2=0x557fe04910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- 172.21.15.87:0/2033874648 shutdown_connections 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- 172.21.15.87:0/2033874648 wait complete. 2024-04-25T10:25:24.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 -- start start 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.263+0000 7f306a4b9000 1 Processor -- start 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04490a0 0x557fe04a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04929d0 0x557fe04a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe04a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe0230e00 con 0x557fe04929d0 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30601ff470 con 0x557fe04932f0 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fe0295300 con 0x557fe04490a0 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 10 client.0 _ll_register_callbacks cb 0x557fe0414040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04929d0 0x557fe04a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04929d0 0x557fe04a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48376/0 (socket says 172.21.15.87:48376) 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 -- 172.21.15.87:0/370908791 learned_addr learned my addr 172.21.15.87:0/370908791 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f305fe00640 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04490a0 0x557fe04a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3065000640 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe04a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 -- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04490a0 msgr2=0x557fe04a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04490a0 0x557fe04a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 -- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 msgr2=0x557fe04a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe04a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 -- 172.21.15.87:0/370908791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fe02378e0 con 0x557fe04929d0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f3057400640 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04929d0 0x557fe04a58c0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f304801d4b0 tx=0x7f30480199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4493e4550098cc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 1 -- 172.21.15.87:0/370908791 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480c4060 con 0x557fe04929d0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 1 -- 172.21.15.87:0/370908791 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f30480bfb70 con 0x557fe04929d0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3050035040 con 0x557fe04929d0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 1 -- 172.21.15.87:0/370908791 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30480ca050 con 0x557fe04929d0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 10 client.10719 fetch_fsmap learned FSMap version 119 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x557fe02ef8f0 con 0x557fe04929d0 2024-04-25T10:25:24.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 1 -- 172.21.15.87:0/370908791 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050035040 con 0x557fe04929d0 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 1 -- 172.21.15.87:0/370908791 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f30480cd020 con 0x557fe04929d0 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 10 client.10719 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 10 client.10719 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 20 client.10719 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a67c0 0x557fe04faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 4 client.10719 mds_command: new command op to 10494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 --> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557fe02a3530 con 0x557fe04a67c0 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f305fe00640 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a67c0 0x557fe04faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f305fe00640 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a67c0 0x557fe04faf60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762b1b3a37668fde server_cookie=1df437f0a138df0b in_seq=0 out_seq=0 2024-04-25T10:25:24.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 10 client.10719 ms_handle_connect on v2:172.21.15.87:6834/1488445455 2024-04-25T10:25:24.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 1 -- 172.21.15.87:0/370908791 <== mds.0 v2:172.21.15.87:6834/1488445455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557fe02a3530 con 0x557fe04a67c0 2024-04-25T10:25:24.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 10 client.10719 handle_command_reply: tid=0 2024-04-25T10:25:24.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 client.10719 shutdown 2024-04-25T10:25:24.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04929d0 msgr2=0x557fe04a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04929d0 0x557fe04a58c0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f304801d4b0 tx=0x7f30480199b0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 shutdown_connections 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557fe04932f0 0x557fe04a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:6834/1488445455,v1:172.21.15.87:6837/1488445455] conn(0x557fe04a67c0 0x557fe04faf60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557fe04929d0 0x557fe04a58c0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 --2- 172.21.15.87:0/370908791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557fe04490a0 0x557fe04a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f304d600640 0 client.10719 ms_handle_reset on v2:172.21.15.87:6834/1488445455 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 >> 172.21.15.87:0/370908791 conn(0x557fe0445020 msgr2=0x557fe0491960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 shutdown_connections 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 1 -- 172.21.15.87:0/370908791 wait complete. 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.267+0000 7f306a4b9000 20 client.10719 trim_cache size 0 max 16384 2024-04-25T10:25:24.273 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:25:24.274 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:25:24.274 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:25:24.274 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:25:24.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f303c066f90 msgr2=0x7f303c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f303c066f90 0x7f303c069450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3050001bf0 tx=0x7f3050001740 comp rx=0 tx=0).stop 2024-04-25T10:25:24.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060155300 msgr2=0x7f3060155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060155300 0x7f3060155780 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f30600b2f70 tx=0x7f3058011c90 comp rx=0 tx=0).stop 2024-04-25T10:25:24.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 shutdown_connections 2024-04-25T10:25:24.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f303c066f90 0x7f303c069450 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060155300 0x7f3060155780 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f306006cc60 0x7f30600af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 --2- 172.21.15.87:0/3046177964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3044004810 0x7f3060148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 >> 172.21.15.87:0/3046177964 conn(0x7f30600669a0 msgr2=0x7f30600b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 shutdown_connections 2024-04-25T10:25:24.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.271+0000 7f3067000640 1 -- 172.21.15.87:0/3046177964 wait complete. 2024-04-25T10:25:24.288 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:25:24.289 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:25:24.289 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:25:24.289 DEBUG:teuthology.orchestra.run.smithi087:> import stat 2024-04-25T10:25:24.289 DEBUG:teuthology.orchestra.run.smithi087:> import json 2024-04-25T10:25:24.289 DEBUG:teuthology.orchestra.run.smithi087:> import sys 2024-04-25T10:25:24.289 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:25:24.289 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> except OSError as e: 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> sys.exit(e.errno) 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> print(json.dumps( 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> indent=2)) 2024-04-25T10:25:24.290 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_mode": 33204, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_ino": 1099511627777, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_dev": 44, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_nlink": 1, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_uid": 1000, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_gid": 1271, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_size": 6291456, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_atime": 1714040692.640149, 2024-04-25T10:25:24.359 INFO:teuthology.orchestra.run.smithi087.stdout: "st_mtime": 1714040692.6721487, 2024-04-25T10:25:24.360 INFO:teuthology.orchestra.run.smithi087.stdout: "st_ctime": 1714040692.6721487 2024-04-25T10:25:24.360 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:25:24.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:25:24.363 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:24.366 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:24.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.547+0000 7fcb2d200640 1 Processor -- start 2024-04-25T10:25:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.547+0000 7fcb2d200640 1 -- start start 2024-04-25T10:25:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.547+0000 7fcb2d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb280cf9b0 0x7fcb2815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.547+0000 7fcb2d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb280b3360 0x7fcb28070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.547+0000 7fcb2d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb28071360 con 0x7fcb28070f50 2024-04-25T10:25:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.547+0000 7fcb2d200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2815ef10 con 0x7fcb280cf9b0 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.547+0000 7fcb2d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb280b5200 con 0x7fcb280b3360 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb280cf9b0 0x7fcb2815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb280cf9b0 0x7fcb2815e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48388/0 (socket says 172.21.15.87:48388) 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 -- 172.21.15.87:0/1450395868 learned_addr learned my addr 172.21.15.87:0/1450395868 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb1ea00640 1 --2- 172.21.15.87:0/1450395868 >> v2:172.21.15.169:3300/0 conn(0x7fcb280b3360 0x7fcb28070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 -- 172.21.15.87:0/1450395868 >> v2:172.21.15.169:3300/0 conn(0x7fcb280b3360 msgr2=0x7fcb28070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 --2- 172.21.15.87:0/1450395868 >> v2:172.21.15.169:3300/0 conn(0x7fcb280b3360 0x7fcb28070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 -- 172.21.15.87:0/1450395868 >> v1:172.21.15.169:6789/0 conn(0x7fcb28070f50 legacy=0x7fcb281540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 -- 172.21.15.87:0/1450395868 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280b38b0 con 0x7fcb280cf9b0 2024-04-25T10:25:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.551+0000 7fcb27400640 1 --2- 172.21.15.87:0/1450395868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280cf9b0 0x7fcb2815e9d0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fcb2806c3f0 tx=0x7fcb1801c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935f0de03d96471b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb26a00640 1 -- 172.21.15.87:0/1450395868 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb180a5560 con 0x7fcb280cf9b0 2024-04-25T10:25:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb26a00640 1 -- 172.21.15.87:0/1450395868 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb180a5700 con 0x7fcb280cf9b0 2024-04-25T10:25:24.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb26a00640 1 -- 172.21.15.87:0/1450395868 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb180a5ad0 con 0x7fcb280cf9b0 2024-04-25T10:25:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- 172.21.15.87:0/1450395868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280cf9b0 msgr2=0x7fcb2815e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 --2- 172.21.15.87:0/1450395868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280cf9b0 0x7fcb2815e9d0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fcb2806c3f0 tx=0x7fcb1801c7d0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- 172.21.15.87:0/1450395868 shutdown_connections 2024-04-25T10:25:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 --2- 172.21.15.87:0/1450395868 >> v2:172.21.15.169:3300/0 conn(0x7fcb280b3360 0x7fcb28070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 --2- 172.21.15.87:0/1450395868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280cf9b0 0x7fcb2815e9d0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- 172.21.15.87:0/1450395868 >> 172.21.15.87:0/1450395868 conn(0x7fcb280cfe80 msgr2=0x7fcb280d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- 172.21.15.87:0/1450395868 shutdown_connections 2024-04-25T10:25:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- 172.21.15.87:0/1450395868 wait complete. 2024-04-25T10:25:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 Processor -- start 2024-04-25T10:25:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- start start 2024-04-25T10:25:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28070f50 0x7fcb28140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280b3360 0x7fcb2815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb280cf9b0 0x7fcb28160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280add60 con 0x7fcb280b3360 2024-04-25T10:25:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280adbe0 con 0x7fcb28070f50 2024-04-25T10:25:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb2d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280adee0 con 0x7fcb280cf9b0 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb1ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280b3360 0x7fcb2815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb1ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280b3360 0x7fcb2815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48402/0 (socket says 172.21.15.87:48402) 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb1ea00640 1 -- 172.21.15.87:0/3424015921 learned_addr learned my addr 172.21.15.87:0/3424015921 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb27400640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28070f50 0x7fcb28140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.555+0000 7fcb27e00640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb280cf9b0 0x7fcb28160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1ea00640 1 -- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb280cf9b0 msgr2=0x7fcb28160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1ea00640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb280cf9b0 0x7fcb28160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1ea00640 1 -- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28070f50 msgr2=0x7fcb28140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1ea00640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28070f50 0x7fcb28140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1ea00640 1 -- 172.21.15.87:0/3424015921 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28140d80 con 0x7fcb280b3360 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1ea00640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280b3360 0x7fcb2815fcf0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d800 tx=0x7fcb1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1fe00640 1 -- 172.21.15.87:0/3424015921 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000b6c0 con 0x7fcb280b3360 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1fe00640 1 -- 172.21.15.87:0/3424015921 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10019070 con 0x7fcb280b3360 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1fe00640 1 -- 172.21.15.87:0/3424015921 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000ba60 con 0x7fcb280b3360 2024-04-25T10:25:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb281555f0 con 0x7fcb280b3360 2024-04-25T10:25:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb28155bb0 con 0x7fcb280b3360 2024-04-25T10:25:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb28075120 con 0x7fcb280b3360 2024-04-25T10:25:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1fe00640 1 -- 172.21.15.87:0/3424015921 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb1000bc00 con 0x7fcb280b3360 2024-04-25T10:25:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1fe00640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb04066f60 0x7fcb04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb1fe00640 1 -- 172.21.15.87:0/3424015921 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcb10085520 con 0x7fcb280b3360 2024-04-25T10:25:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.559+0000 7fcb27400640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb04066f60 0x7fcb04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.563+0000 7fcb27400640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb04066f60 0x7fcb04069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcb180ae000 tx=0x7fcb180a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.563+0000 7fcb1fe00640 1 -- 172.21.15.87:0/3424015921 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb10056450 con 0x7fcb280b3360 2024-04-25T10:25:24.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb280d04b0 con 0x7fcb280b3360 2024-04-25T10:25:24.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb1fe00640 1 -- 172.21.15.87:0/3424015921 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7fcb10055d50 con 0x7fcb280b3360 2024-04-25T10:25:24.755 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:24.755 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:25:24.755 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb04066f60 msgr2=0x7fcb04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb04066f60 0x7fcb04069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcb180ae000 tx=0x7fcb180a7070 comp rx=0 tx=0).stop 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280b3360 msgr2=0x7fcb2815fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280b3360 0x7fcb2815fcf0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d800 tx=0x7fcb1000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 shutdown_connections 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb04066f60 0x7fcb04069420 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb280cf9b0 0x7fcb28160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb280b3360 0x7fcb2815fcf0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 --2- 172.21.15.87:0/3424015921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb28070f50 0x7fcb28140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 >> 172.21.15.87:0/3424015921 conn(0x7fcb280cfe80 msgr2=0x7fcb280afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 shutdown_connections 2024-04-25T10:25:24.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.755+0000 7fcb2d200640 1 -- 172.21.15.87:0/3424015921 wait complete. 2024-04-25T10:25:24.776 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:25:24.936 INFO:teuthology.orchestra.run.smithi087.stdout:{"message":"","return_code":0} 2024-04-25T10:25:24.937 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:25:24.937 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:25:24.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 Processor -- start 2024-04-25T10:25:24.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 -- start start 2024-04-25T10:25:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 0x560c4448cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c445380b0 0x560c44539250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c4445d940 0x560c4445dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c444059f0 con 0x560c445380b0 2024-04-25T10:25:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c44409950 con 0x560c4445d940 2024-04-25T10:25:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb98449080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c44536f20 con 0x7ffb7c0022e0 2024-04-25T10:25:24.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb8fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c445380b0 0x560c44539250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb8fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c445380b0 0x560c44539250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48416/0 (socket says 172.21.15.87:48416) 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb8fe00640 1 -- 172.21.15.87:0/442526538 learned_addr learned my addr 172.21.15.87:0/442526538 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb95600640 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c4445d940 0x560c4445dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb94c00640 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 0x560c4448cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb8fe00640 1 -- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 msgr2=0x560c4448cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.983+0000 7ffb8fe00640 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 0x560c4448cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8fe00640 1 -- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c4445d940 msgr2=0x560c4445dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8fe00640 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c4445d940 0x560c4445dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8fe00640 1 -- 172.21.15.87:0/442526538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c44539790 con 0x560c445380b0 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8fe00640 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c445380b0 0x560c44539250 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009820 tx=0x7ffb80005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd65c40992bf3c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8ea00640 1 -- 172.21.15.87:0/442526538 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80009e60 con 0x560c445380b0 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8ea00640 1 -- 172.21.15.87:0/442526538 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8001a070 con 0x560c445380b0 2024-04-25T10:25:24.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8ea00640 1 -- 172.21.15.87:0/442526538 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80015460 con 0x560c445380b0 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- 172.21.15.87:0/442526538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c445380b0 msgr2=0x560c44539250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c445380b0 0x560c44539250 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009820 tx=0x7ffb80005a10 comp rx=0 tx=0).stop 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- 172.21.15.87:0/442526538 shutdown_connections 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c4445d940 0x560c4445dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c445380b0 0x560c44539250 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 --2- 172.21.15.87:0/442526538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 0x560c4448cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- 172.21.15.87:0/442526538 >> 172.21.15.87:0/442526538 conn(0x560c44489760 msgr2=0x560c444a7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- 172.21.15.87:0/442526538 shutdown_connections 2024-04-25T10:25:24.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- 172.21.15.87:0/442526538 wait complete. 2024-04-25T10:25:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 Processor -- start 2024-04-25T10:25:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- start start 2024-04-25T10:25:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c4445d940 0x560c4453dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c445380b0 0x560c44533250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 0x560c445403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c44518ed0 con 0x560c4445d940 2024-04-25T10:25:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4448e500 con 0x560c445380b0 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb98449080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c44539c30 con 0x7ffb7c0022e0 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c4445d940 0x560c4453dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c4445d940 0x560c4453dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48430/0 (socket says 172.21.15.87:48430) 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 -- 172.21.15.87:0/2128720625 learned_addr learned my addr 172.21.15.87:0/2128720625 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8fe00640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c445380b0 0x560c44533250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb95600640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 0x560c445403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 -- 172.21.15.87:0/2128720625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 msgr2=0x560c445403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0022e0 0x560c445403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 -- 172.21.15.87:0/2128720625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c445380b0 msgr2=0x560c44533250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c445380b0 0x560c44533250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 -- 172.21.15.87:0/2128720625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4453e560 con 0x560c4445d940 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb94c00640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c4445d940 0x560c4453dcc0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7ffb8801ad00 tx=0x7ffb7c004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.987+0000 7ffb8d600640 1 -- 172.21.15.87:0/2128720625 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880b0040 con 0x560c4445d940 2024-04-25T10:25:24.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb8d600640 1 -- 172.21.15.87:0/2128720625 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb880ba070 con 0x560c4445d940 2024-04-25T10:25:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c4453e800 con 0x560c4445d940 2024-04-25T10:25:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb8d600640 1 -- 172.21.15.87:0/2128720625 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880abc60 con 0x560c4445d940 2024-04-25T10:25:24.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c4453ed40 con 0x560c4445d940 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb8d600640 1 -- 172.21.15.87:0/2128720625 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb880bf460 con 0x560c4445d940 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb8d600640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb74065940 0x7ffb74067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb8d600640 1 -- 172.21.15.87:0/2128720625 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ffb8811de70 con 0x560c4445d940 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb8fe00640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb74065940 0x7ffb74067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb98449080 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x560c44516f70 0x560c444819e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 --> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] -- osd_op(unknown.0.0:1 25.7 25:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165) v9 -- 0x560c44481f20 con 0x560c44516f70 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb95600640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x560c44516f70 0x560c444819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb8fe00640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb74065940 0x7ffb74067e00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ffb80000c10 tx=0x7ffb80014000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.991+0000 7ffb95600640 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x560c44516f70 0x560c444819e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb95600640 1 -- 172.21.15.87:0/2128720625 <== osd.0 v2:172.21.15.87:6800/2221934810 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7ffb90012040 con 0x560c44516f70 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x560c44516f70 msgr2=0x560c444819e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x560c44516f70 0x560c444819e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:24.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb74065940 msgr2=0x7ffb74067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffb74065940 0x7ffb74067e00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ffb80000c10 tx=0x7ffb80014000 comp rx=0 tx=0).stop 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c4445d940 msgr2=0x560c4453dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 --2- 172.21.15.87:0/2128720625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c4445d940 0x560c4453dcc0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7ffb8801ad00 tx=0x7ffb7c004130 comp rx=0 tx=0).stop 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb95600640 1 -- 172.21.15.87:0/2128720625 reap_dead start 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 shutdown_connections 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 >> 172.21.15.87:0/2128720625 conn(0x560c44489760 msgr2=0x560c44534880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 shutdown_connections 2024-04-25T10:25:24.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:24.995+0000 7ffb98449080 1 -- 172.21.15.87:0/2128720625 wait complete. 2024-04-25T10:25:24.998 INFO:teuthology.orchestra.run.smithi087.stdout:Ysixmegssubdir 2024-04-25T10:25:24.998 DEBUG:teuthology.orchestra.run.smithi087:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:25:25.096 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511627777, 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: "ancestors": [ 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: "dirino": 1099511627776, 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: "dname": "sixmegs", 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 6 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: "dirino": 1, 2024-04-25T10:25:25.097 INFO:teuthology.orchestra.run.smithi087.stdout: "dname": "subdir", 2024-04-25T10:25:25.098 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 8 2024-04-25T10:25:25.098 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:25.098 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-04-25T10:25:25.098 INFO:teuthology.orchestra.run.smithi087.stdout: "pool": 25, 2024-04-25T10:25:25.098 INFO:teuthology.orchestra.run.smithi087.stdout: "old_pools": [] 2024-04-25T10:25:25.098 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:25:25.098 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:25.101 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:25:25.101 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:25.196 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:25.297 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:25:25.297 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:25.297 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:25.306 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:25.306 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:25.357 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:25.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0d1a00640 1 Processor -- start 2024-04-25T10:25:25.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0d1a00640 1 -- start start 2024-04-25T10:25:25.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0d1a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0cc0d0040 0x7fb0cc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0d1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc15b740 0x7fb0cc0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0d1a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0cc071050 con 0x7fb0cc1550b0 2024-04-25T10:25:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0d1a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0cc15d040 con 0x7fb0cc0d0040 2024-04-25T10:25:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0d1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc0712f0 con 0x7fb0cc15b740 2024-04-25T10:25:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0caa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb0cc1550b0 0x7fb0cc0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:51566/0 (socket says 172.21.15.87:51566) 2024-04-25T10:25:25.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.531+0000 7fb0caa00640 1 -- 172.21.15.87:0/1477359135 learned_addr learned my addr 172.21.15.87:0/1477359135 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0cb400640 1 --2- 172.21.15.87:0/1477359135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc15b740 0x7fb0cc0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0c2000640 1 --2- 172.21.15.87:0/1477359135 >> v2:172.21.15.169:3300/0 conn(0x7fb0cc0d0040 0x7fb0cc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0ca000640 1 -- 172.21.15.87:0/1477359135 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973016778 0 0) 0x7fb0cc071050 con 0x7fb0cc1550b0 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0ca000640 1 -- 172.21.15.87:0/1477359135 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0b0003530 con 0x7fb0cc1550b0 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0ca000640 1 -- 172.21.15.87:0/1477359135 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 381148885 0 0) 0x7fb0b0003530 con 0x7fb0cc1550b0 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0cb400640 1 -- 172.21.15.87:0/1477359135 >> v2:172.21.15.169:3300/0 conn(0x7fb0cc0d0040 msgr2=0x7fb0cc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0cb400640 1 --2- 172.21.15.87:0/1477359135 >> v2:172.21.15.169:3300/0 conn(0x7fb0cc0d0040 0x7fb0cc15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0cb400640 1 -- 172.21.15.87:0/1477359135 >> v1:172.21.15.87:6789/0 conn(0x7fb0cc1550b0 legacy=0x7fb0cc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0cb400640 1 -- 172.21.15.87:0/1477359135 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc0715d0 con 0x7fb0cc15b740 2024-04-25T10:25:25.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0cb400640 1 --2- 172.21.15.87:0/1477359135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc15b740 0x7fb0cc0b3260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009d90 tx=0x7fb0c401c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57db2c630d12de55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:25.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0ca000640 1 -- 172.21.15.87:0/1477359135 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40b2070 con 0x7fb0cc15b740 2024-04-25T10:25:25.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0ca000640 1 -- 172.21.15.87:0/1477359135 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c40a5660 con 0x7fb0cc15b740 2024-04-25T10:25:25.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.535+0000 7fb0ca000640 1 -- 172.21.15.87:0/1477359135 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc001680 con 0x7fb0cc15b740 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- 172.21.15.87:0/1477359135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc15b740 msgr2=0x7fb0cc0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/1477359135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc15b740 0x7fb0cc0b3260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009d90 tx=0x7fb0c401c860 comp rx=0 tx=0).stop 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- 172.21.15.87:0/1477359135 shutdown_connections 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/1477359135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc15b740 0x7fb0cc0b3260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/1477359135 >> v2:172.21.15.169:3300/0 conn(0x7fb0cc0d0040 0x7fb0cc15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- 172.21.15.87:0/1477359135 >> 172.21.15.87:0/1477359135 conn(0x7fb0cc066b70 msgr2=0x7fb0cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- 172.21.15.87:0/1477359135 shutdown_connections 2024-04-25T10:25:25.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- 172.21.15.87:0/1477359135 wait complete. 2024-04-25T10:25:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 Processor -- start 2024-04-25T10:25:25.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- start start 2024-04-25T10:25:25.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc0d0040 0x7fb0cc06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0cc1550b0 0x7fb0cc14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0cc15b740 0x7fb0cc138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc15bd50 con 0x7fb0cc1550b0 2024-04-25T10:25:25.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc15bbd0 con 0x7fb0cc15b740 2024-04-25T10:25:25.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0d1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc15c030 con 0x7fb0cc0d0040 2024-04-25T10:25:25.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0c2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0cc1550b0 0x7fb0cc14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0c2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0cc1550b0 0x7fb0cc14fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48432/0 (socket says 172.21.15.87:48432) 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0c2000640 1 -- 172.21.15.87:0/3529168984 learned_addr learned my addr 172.21.15.87:0/3529168984 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0caa00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc0d0040 0x7fb0cc06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.539+0000 7fb0cb400640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0cc15b740 0x7fb0cc138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c2000640 1 -- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc0d0040 msgr2=0x7fb0cc06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c2000640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc0d0040 0x7fb0cc06ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c2000640 1 -- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0cc15b740 msgr2=0x7fb0cc138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c2000640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0cc15b740 0x7fb0cc138110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c2000640 1 -- 172.21.15.87:0/3529168984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc15a2f0 con 0x7fb0cc1550b0 2024-04-25T10:25:25.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c2000640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0cc1550b0 0x7fb0cc14fb30 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e930 tx=0x7fb0b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c3400640 1 -- 172.21.15.87:0/3529168984 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4017070 con 0x7fb0cc1550b0 2024-04-25T10:25:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c3400640 1 -- 172.21.15.87:0/3529168984 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400ca00 con 0x7fb0cc1550b0 2024-04-25T10:25:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc14ed70 con 0x7fb0cc1550b0 2024-04-25T10:25:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c3400640 1 -- 172.21.15.87:0/3529168984 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400cdb0 con 0x7fb0cc1550b0 2024-04-25T10:25:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc14f330 con 0x7fb0cc1550b0 2024-04-25T10:25:25.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0cc04ef30 con 0x7fb0cc1550b0 2024-04-25T10:25:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c3400640 1 -- 172.21.15.87:0/3529168984 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0b401a070 con 0x7fb0cc1550b0 2024-04-25T10:25:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c3400640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0a8066f60 0x7fb0a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0c3400640 1 -- 172.21.15.87:0/3529168984 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb0b4084ff0 con 0x7fb0cc1550b0 2024-04-25T10:25:25.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.543+0000 7fb0caa00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0a8066f60 0x7fb0a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.547+0000 7fb0caa00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0a8066f60 0x7fb0a8069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc001ec0 tx=0x7fb0bc0016e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:25.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.551+0000 7fb0c3400640 1 -- 172.21.15.87:0/3529168984 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0b4055a20 con 0x7fb0cc1550b0 2024-04-25T10:25:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.707+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0cc15cd00 con 0x7fb0cc1550b0 2024-04-25T10:25:25.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0c3400640 1 -- 172.21.15.87:0/3529168984 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16765 (secure 0 0 0) 0x7fb0b4059ac0 con 0x7fb0cc1550b0 2024-04-25T10:25:25.710 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:25.710 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":165,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:07.649563+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:42.650296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:43.940319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.29","mappings":[{"from":0,"to":2}]},{"pgid":"25.39","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0a8066f60 msgr2=0x7fb0a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0a8066f60 0x7fb0a8069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc001ec0 tx=0x7fb0bc0016e0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0cc1550b0 msgr2=0x7fb0cc14fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0cc1550b0 0x7fb0cc14fb30 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e930 tx=0x7fb0b400edf0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 shutdown_connections 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0a8066f60 0x7fb0a8069420 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0cc15b740 0x7fb0cc138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0cc1550b0 0x7fb0cc14fb30 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 --2- 172.21.15.87:0/3529168984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0cc0d0040 0x7fb0cc06ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 >> 172.21.15.87:0/3529168984 conn(0x7fb0cc066b70 msgr2=0x7fb0cc0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:25.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 shutdown_connections 2024-04-25T10:25:25.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.711+0000 7fb0d1a00640 1 -- 172.21.15.87:0/3529168984 wait complete. 2024-04-25T10:25:25.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:24:42.650296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:24:43.940319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:25.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5edc00640 1 Processor -- start 2024-04-25T10:25:25.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5edc00640 1 -- start start 2024-04-25T10:25:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5edc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5e8159540 0x7fa5e80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5edc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e80721f0 0x7fa5e8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5edc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e8159920 con 0x7fa5e80aebd0 2024-04-25T10:25:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5edc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e815c2f0 con 0x7fa5e8159540 2024-04-25T10:25:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5edc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e815c5d0 con 0x7fa5e80721f0 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e80721f0 0x7fa5e8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e80721f0 0x7fa5e8070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:34818/0 (socket says 172.21.15.87:34818) 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e7400640 1 -- 172.21.15.87:0/220741757 learned_addr learned my addr 172.21.15.87:0/220741757 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e6000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa5e80aebd0 0x7fa5e8071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:34490/0 (socket says 172.21.15.87:34490) 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e6a00640 1 --2- 172.21.15.87:0/220741757 >> v2:172.21.15.169:3300/0 conn(0x7fa5e8159540 0x7fa5e80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e5600640 1 -- 172.21.15.87:0/220741757 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986016018 0 0) 0x7fa5e8159920 con 0x7fa5e80aebd0 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e7400640 1 -- 172.21.15.87:0/220741757 >> v2:172.21.15.169:3300/0 conn(0x7fa5e8159540 msgr2=0x7fa5e80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e7400640 1 --2- 172.21.15.87:0/220741757 >> v2:172.21.15.169:3300/0 conn(0x7fa5e8159540 0x7fa5e80ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e7400640 1 -- 172.21.15.87:0/220741757 >> v1:172.21.15.169:6789/0 conn(0x7fa5e80aebd0 legacy=0x7fa5e8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.871+0000 7fa5e7400640 1 -- 172.21.15.87:0/220741757 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e815c8b0 con 0x7fa5e80721f0 2024-04-25T10:25:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5e7400640 1 --2- 172.21.15.87:0/220741757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e80721f0 0x7fa5e8070350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009e50 tx=0x7fa5e001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e1133cabe83d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5e5600640 1 -- 172.21.15.87:0/220741757 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00a5810 con 0x7fa5e80721f0 2024-04-25T10:25:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5e5600640 1 -- 172.21.15.87:0/220741757 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e00a59b0 con 0x7fa5e80721f0 2024-04-25T10:25:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5e5600640 1 -- 172.21.15.87:0/220741757 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00a5d80 con 0x7fa5e80721f0 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 -- 172.21.15.87:0/220741757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e80721f0 msgr2=0x7fa5e8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 --2- 172.21.15.87:0/220741757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e80721f0 0x7fa5e8070350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009e50 tx=0x7fa5e001ca30 comp rx=0 tx=0).stop 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 -- 172.21.15.87:0/220741757 shutdown_connections 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 --2- 172.21.15.87:0/220741757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e80721f0 0x7fa5e8070350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 --2- 172.21.15.87:0/220741757 >> v2:172.21.15.169:3300/0 conn(0x7fa5e8159540 0x7fa5e80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 -- 172.21.15.87:0/220741757 >> 172.21.15.87:0/220741757 conn(0x7fa5e80669a0 msgr2=0x7fa5e8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 -- 172.21.15.87:0/220741757 shutdown_connections 2024-04-25T10:25:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.875+0000 7fa5edc00640 1 -- 172.21.15.87:0/220741757 wait complete. 2024-04-25T10:25:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 Processor -- start 2024-04-25T10:25:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 -- start start 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5e80721f0 0x7fa5e814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5e80aebd0 0x7fa5e8148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e8159540 0x7fa5e8154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8071220 con 0x7fa5e80721f0 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e80710a0 con 0x7fa5e80aebd0 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e80713a0 con 0x7fa5e8159540 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5e80721f0 0x7fa5e814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5e80721f0 0x7fa5e814f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48436/0 (socket says 172.21.15.87:48436) 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 -- 172.21.15.87:0/1806910400 learned_addr learned my addr 172.21.15.87:0/1806910400 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6000640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5e80aebd0 0x7fa5e8148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e7400640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e8159540 0x7fa5e8154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 -- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e8159540 msgr2=0x7fa5e8154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e8159540 0x7fa5e8154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 -- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5e80aebd0 msgr2=0x7fa5e8148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5e80aebd0 0x7fa5e8148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 -- 172.21.15.87:0/1806910400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e814eb40 con 0x7fa5e80721f0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6000640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5e80aebd0 0x7fa5e8148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5e6a00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5e80721f0 0x7fa5e814f8c0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e870 tx=0x7fa5d800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5dea00640 1 -- 172.21.15.87:0/1806910400 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8017070 con 0x7fa5e80721f0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5dea00640 1 -- 172.21.15.87:0/1806910400 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d800c8e0 con 0x7fa5e80721f0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5dea00640 1 -- 172.21.15.87:0/1806910400 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800cc90 con 0x7fa5e80721f0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e8152b00 con 0x7fa5e80721f0 2024-04-25T10:25:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e814f0c0 con 0x7fa5e80721f0 2024-04-25T10:25:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.879+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e804ef30 con 0x7fa5e80721f0 2024-04-25T10:25:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.883+0000 7fa5dea00640 1 -- 172.21.15.87:0/1806910400 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5d801a070 con 0x7fa5e80721f0 2024-04-25T10:25:25.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.883+0000 7fa5dea00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5c8066f90 0x7fa5c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:25.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.883+0000 7fa5dea00640 1 -- 172.21.15.87:0/1806910400 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa5d8084e50 con 0x7fa5e80721f0 2024-04-25T10:25:25.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.883+0000 7fa5e6000640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5c8066f90 0x7fa5c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.883+0000 7fa5e6000640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5c8066f90 0x7fa5c8069450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4007fd0 tx=0x7fa5d4007c20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:25.887+0000 7fa5dea00640 1 -- 172.21.15.87:0/1806910400 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5d80557c0 con 0x7fa5e80721f0 2024-04-25T10:25:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.039+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5e80cf470 con 0x7fa5e80721f0 2024-04-25T10:25:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.039+0000 7fa5dea00640 1 -- 172.21.15.87:0/1806910400 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5d8059860 con 0x7fa5e80721f0 2024-04-25T10:25:26.040 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:25:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5c8066f90 msgr2=0x7fa5c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5c8066f90 0x7fa5c8069450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4007fd0 tx=0x7fa5d4007c20 comp rx=0 tx=0).stop 2024-04-25T10:25:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5e80721f0 msgr2=0x7fa5e814f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5e80721f0 0x7fa5e814f8c0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e870 tx=0x7fa5d800ec70 comp rx=0 tx=0).stop 2024-04-25T10:25:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 shutdown_connections 2024-04-25T10:25:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa5c8066f90 0x7fa5c8069450 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5e8159540 0x7fa5e8154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5e80aebd0 0x7fa5e8148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 --2- 172.21.15.87:0/1806910400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5e80721f0 0x7fa5e814f8c0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 >> 172.21.15.87:0/1806910400 conn(0x7fa5e80669a0 msgr2=0x7fa5e8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 shutdown_connections 2024-04-25T10:25:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.043+0000 7fa5edc00640 1 -- 172.21.15.87:0/1806910400 wait complete. 2024-04-25T10:25:26.061 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:25:26.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfb1a00640 1 Processor -- start 2024-04-25T10:25:26.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfb1a00640 1 -- start start 2024-04-25T10:25:26.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 0x7fcfac166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfb1a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac071360 con 0x7fcfac159540 2024-04-25T10:25:26.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfb1a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac159920 con 0x7fcfac0aeda0 2024-04-25T10:25:26.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfb1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac15c2f0 con 0x7fcfac0721f0 2024-04-25T10:25:26.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfaaa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcfac159540 0x7fcfac0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:51582/0 (socket says 172.21.15.87:51582) 2024-04-25T10:25:26.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfaaa00640 1 -- 172.21.15.87:0/4199153932 learned_addr learned my addr 172.21.15.87:0/4199153932 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfab400640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 0x7fcfac166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163780951 0 0) 0x7fcfac071360 con 0x7fcfac159540 2024-04-25T10:25:26.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.199+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf90003530 con 0x7fcfac159540 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210036678 0 0) 0x7fcf90003530 con 0x7fcfac159540 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 msgr2=0x7fcfac166920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 0x7fcfac166920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 >> v1:172.21.15.169:6789/0 conn(0x7fcfac0aeda0 legacy=0x7fcfac071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac15c5d0 con 0x7fcfac159540 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fcf9c0016a0 con 0x7fcfac159540 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 >> v1:172.21.15.87:6789/0 conn(0x7fcfac159540 legacy=0x7fcfac0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf900039f0 0x7fcf90003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 0x7fcf90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf90004d50 0x7fcf900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac15c2f0 con 0x7fcf90004d50 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9000bab0 con 0x7fcf900039f0 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9000bcb0 con 0x7fcfac0721f0 2024-04-25T10:25:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfab400640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf90004d50 0x7fcf900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfaaa00640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf900039f0 0x7fcf90003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfaa000640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 0x7fcf90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfab400640 1 -- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 msgr2=0x7fcf90004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfab400640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac0721f0 0x7fcf90004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfab400640 1 -- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf900039f0 msgr2=0x7fcf90003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfab400640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf900039f0 0x7fcf90003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfab400640 1 -- 172.21.15.87:0/4199153932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf90003530 con 0x7fcf90004d50 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfaaa00640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf900039f0 0x7fcf90003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:26.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.203+0000 7fcfab400640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf90004d50 0x7fcf900051d0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009f90 tx=0x7fcfa401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad20f4bee8c73afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40b2070 con 0x7fcf90004d50 2024-04-25T10:25:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa40a5c50 con 0x7fcf90004d50 2024-04-25T10:25:26.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfa9600640 1 -- 172.21.15.87:0/4199153932 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4009c10 con 0x7fcf90004d50 2024-04-25T10:25:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- 172.21.15.87:0/4199153932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf90004d50 msgr2=0x7fcf900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 --2- 172.21.15.87:0/4199153932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf90004d50 0x7fcf900051d0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009f90 tx=0x7fcfa401cc40 comp rx=0 tx=0).stop 2024-04-25T10:25:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfab400640 1 -- 172.21.15.87:0/4199153932 reap_dead start 2024-04-25T10:25:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- 172.21.15.87:0/4199153932 shutdown_connections 2024-04-25T10:25:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- 172.21.15.87:0/4199153932 >> 172.21.15.87:0/4199153932 conn(0x7fcfac0669a0 msgr2=0x7fcfac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- 172.21.15.87:0/4199153932 shutdown_connections 2024-04-25T10:25:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- 172.21.15.87:0/4199153932 wait complete. 2024-04-25T10:25:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 Processor -- start 2024-04-25T10:25:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- start start 2024-04-25T10:25:26.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0adaa0 0x7fcfac1e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac1e58f0 0x7fcfac14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfac0ade80 0x7fcfac14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac15bfa0 con 0x7fcfac0ade80 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac15cb90 con 0x7fcfac0adaa0 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfb1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac06c0d0 con 0x7fcfac1e58f0 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfab400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfac0ade80 0x7fcfac14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfab400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfac0ade80 0x7fcfac14fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48468/0 (socket says 172.21.15.87:48468) 2024-04-25T10:25:26.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfab400640 1 -- 172.21.15.87:0/1482962168 learned_addr learned my addr 172.21.15.87:0/1482962168 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfaa000640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac1e58f0 0x7fcfac14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.207+0000 7fcfaaa00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0adaa0 0x7fcfac1e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfab400640 1 -- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac1e58f0 msgr2=0x7fcfac14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfab400640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac1e58f0 0x7fcfac14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfab400640 1 -- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0adaa0 msgr2=0x7fcfac1e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfab400640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0adaa0 0x7fcfac1e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfab400640 1 -- 172.21.15.87:0/1482962168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac1e5e70 con 0x7fcfac0ade80 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfab400640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfac0ade80 0x7fcfac14fe80 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009f90 tx=0x7fcfa4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfa2a00640 1 -- 172.21.15.87:0/1482962168 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40bd070 con 0x7fcfac0ade80 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfa2a00640 1 -- 172.21.15.87:0/1482962168 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa40a5a50 con 0x7fcfac0ade80 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfa2a00640 1 -- 172.21.15.87:0/1482962168 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40b2040 con 0x7fcfac0ade80 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac1f4270 con 0x7fcfac0ade80 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac1f4830 con 0x7fcfac0ade80 2024-04-25T10:25:26.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac04ef30 con 0x7fcfac0ade80 2024-04-25T10:25:26.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfa2a00640 1 -- 172.21.15.87:0/1482962168 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfa40c0070 con 0x7fcfac0ade80 2024-04-25T10:25:26.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfa2a00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfa2a00640 1 -- 172.21.15.87:0/1482962168 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcfa411ecc0 con 0x7fcfac0ade80 2024-04-25T10:25:26.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.211+0000 7fcfaaa00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.215+0000 7fcfaaa00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c001020 tx=0x7fcf9c007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.215+0000 7fcfa2a00640 1 -- 172.21.15.87:0/1482962168 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfa4016950 con 0x7fcfac0ade80 2024-04-25T10:25:26.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfac15f320 con 0x7fcfac0ade80 2024-04-25T10:25:26.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfa2a00640 1 -- 172.21.15.87:0/1482962168 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v165) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfa40a5c20 con 0x7fcfac0ade80 2024-04-25T10:25:26.370 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:25:26.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 msgr2=0x7fcf8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c001020 tx=0x7fcf9c007040 comp rx=0 tx=0).stop 2024-04-25T10:25:26.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfac0ade80 msgr2=0x7fcfac14fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfac0ade80 0x7fcfac14fe80 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009f90 tx=0x7fcfa4009750 comp rx=0 tx=0).stop 2024-04-25T10:25:26.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 shutdown_connections 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcf8c066f60 0x7fcf8c069420 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfac0ade80 0x7fcfac14fe80 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac1e58f0 0x7fcfac14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 --2- 172.21.15.87:0/1482962168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0adaa0 0x7fcfac1e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 >> 172.21.15.87:0/1482962168 conn(0x7fcfac0669a0 msgr2=0x7fcfac0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 shutdown_connections 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.371+0000 7fcfb1a00640 1 -- 172.21.15.87:0/1482962168 wait complete. 2024-04-25T10:25:26.391 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:25:26.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d4a800640 1 Processor -- start 2024-04-25T10:25:26.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d4a800640 1 -- start start 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d4a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d44159540 0x7f4d440ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d4a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d44071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d4a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d44071360 con 0x7f4d44072020 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d4a800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d44159920 con 0x7f4d44159540 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d4a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4415c2f0 con 0x7f4d440aebd0 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d43400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d44159540 0x7f4d440ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d43400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d44159540 0x7f4d440ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48476/0 (socket says 172.21.15.87:48476) 2024-04-25T10:25:26.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d43400640 1 -- 172.21.15.87:0/742852424 learned_addr learned my addr 172.21.15.87:0/742852424 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:26.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d43400640 1 -- 172.21.15.87:0/742852424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 msgr2=0x7f4d44071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d43400640 1 --2- 172.21.15.87:0/742852424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d44071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d43400640 1 -- 172.21.15.87:0/742852424 >> v1:172.21.15.169:6789/0 conn(0x7f4d44072020 legacy=0x7f4d4415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.531+0000 7f4d43400640 1 -- 172.21.15.87:0/742852424 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4415c5d0 con 0x7f4d44159540 2024-04-25T10:25:26.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d43400640 1 --2- 172.21.15.87:0/742852424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44159540 0x7f4d440ae690 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f4d440b4c20 tx=0x7f4d3401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d512d53b3a9280c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d42000640 1 -- 172.21.15.87:0/742852424 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340a5640 con 0x7f4d44159540 2024-04-25T10:25:26.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d42000640 1 -- 172.21.15.87:0/742852424 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d340a57e0 con 0x7f4d44159540 2024-04-25T10:25:26.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d42000640 1 -- 172.21.15.87:0/742852424 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340a5bb0 con 0x7f4d44159540 2024-04-25T10:25:26.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 -- 172.21.15.87:0/742852424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44159540 msgr2=0x7f4d440ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 --2- 172.21.15.87:0/742852424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44159540 0x7f4d440ae690 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f4d440b4c20 tx=0x7f4d3401c9b0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 -- 172.21.15.87:0/742852424 shutdown_connections 2024-04-25T10:25:26.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 --2- 172.21.15.87:0/742852424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d44071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 --2- 172.21.15.87:0/742852424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44159540 0x7f4d440ae690 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 -- 172.21.15.87:0/742852424 >> 172.21.15.87:0/742852424 conn(0x7f4d440669a0 msgr2=0x7f4d44066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:26.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 -- 172.21.15.87:0/742852424 shutdown_connections 2024-04-25T10:25:26.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 -- 172.21.15.87:0/742852424 wait complete. 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.535+0000 7f4d4a800640 1 Processor -- start 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 -- start start 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44072020 0x7f4d440afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d440b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d44159540 0x7f4d4415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4406c0d0 con 0x7f4d44072020 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4415bfa0 con 0x7f4d44159540 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44168930 con 0x7f4d440aebd0 2024-04-25T10:25:26.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44072020 0x7f4d440afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44072020 0x7f4d440afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48492/0 (socket says 172.21.15.87:48492) 2024-04-25T10:25:26.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 -- 172.21.15.87:0/2962310967 learned_addr learned my addr 172.21.15.87:0/2962310967 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43e00640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d44159540 0x7f4d4415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d42a00640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d440b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 -- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 msgr2=0x7f4d440b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d440b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 -- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d44159540 msgr2=0x7f4d4415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d44159540 0x7f4d4415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 -- 172.21.15.87:0/2962310967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4414c4c0 con 0x7f4d44072020 2024-04-25T10:25:26.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43e00640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d44159540 0x7f4d4415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d42a00640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d440b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d43400640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44072020 0x7f4d440afd10 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f4d340af040 tx=0x7f4d34019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d3b400640 1 -- 172.21.15.87:0/2962310967 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340bc020 con 0x7f4d44072020 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d3b400640 1 -- 172.21.15.87:0/2962310967 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d340b8690 con 0x7f4d44072020 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d3b400640 1 -- 172.21.15.87:0/2962310967 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340b8a80 con 0x7f4d44072020 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4414d450 con 0x7f4d44072020 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d44151870 con 0x7f4d44072020 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4404ef30 con 0x7f4d44072020 2024-04-25T10:25:26.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.539+0000 7f4d3b400640 1 -- 172.21.15.87:0/2962310967 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d340bf020 con 0x7f4d44072020 2024-04-25T10:25:26.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.543+0000 7f4d3b400640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d24066f60 0x7f4d24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.543+0000 7f4d3b400640 1 -- 172.21.15.87:0/2962310967 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4d34120150 con 0x7f4d44072020 2024-04-25T10:25:26.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.543+0000 7f4d42a00640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d24066f60 0x7f4d24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.543+0000 7f4d42a00640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d24066f60 0x7f4d24069420 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4d441f81d0 tx=0x7f4d30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.547+0000 7f4d3b400640 1 -- 172.21.15.87:0/2962310967 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d340c6030 con 0x7f4d44072020 2024-04-25T10:25:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.703+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d44066f20 con 0x7f4d44072020 2024-04-25T10:25:26.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.703+0000 7f4d3b400640 1 -- 172.21.15.87:0/2962310967 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v165) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d340165a0 con 0x7f4d44072020 2024-04-25T10:25:26.704 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:25:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d24066f60 msgr2=0x7f4d24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d24066f60 0x7f4d24069420 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4d441f81d0 tx=0x7f4d30008040 comp rx=0 tx=0).stop 2024-04-25T10:25:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44072020 msgr2=0x7f4d440afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44072020 0x7f4d440afd10 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f4d340af040 tx=0x7f4d34019750 comp rx=0 tx=0).stop 2024-04-25T10:25:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 shutdown_connections 2024-04-25T10:25:26.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4d24066f60 0x7f4d24069420 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d44159540 0x7f4d4415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d440aebd0 0x7f4d440b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 --2- 172.21.15.87:0/2962310967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44072020 0x7f4d440afd10 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 >> 172.21.15.87:0/2962310967 conn(0x7f4d440669a0 msgr2=0x7f4d4415d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 shutdown_connections 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.707+0000 7f4d4a800640 1 -- 172.21.15.87:0/2962310967 wait complete. 2024-04-25T10:25:26.726 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:25:26.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7fafa1000640 1 Processor -- start 2024-04-25T10:25:26.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7fafa1000640 1 -- start start 2024-04-25T10:25:26.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7fafa1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf9c0d0040 0x7faf9c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7fafa1000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c071050 con 0x7faf9c1550b0 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7fafa1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c15d040 con 0x7faf9c15b740 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7fafa1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c0712f0 con 0x7faf9c0d0040 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7faf9c1550b0 0x7faf9c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:51584/0 (socket says 172.21.15.87:51584) 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99e00640 1 -- 172.21.15.87:0/1694455897 learned_addr learned my addr 172.21.15.87:0/1694455897 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf91400640 1 --2- 172.21.15.87:0/1694455897 >> v2:172.21.15.169:3300/0 conn(0x7faf9c0d0040 0x7faf9c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503335344 0 0) 0x7faf9c071050 con 0x7faf9c1550b0 2024-04-25T10:25:26.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf80005740 con 0x7faf9c1550b0 2024-04-25T10:25:26.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329154178 0 0) 0x7faf9c15d040 con 0x7faf9c15b740 2024-04-25T10:25:26.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf9c071050 con 0x7faf9c15b740 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339362817 0 0) 0x7faf80005740 con 0x7faf9c1550b0 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 >> v2:172.21.15.169:3300/0 conn(0x7faf9c0d0040 msgr2=0x7faf9c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 --2- 172.21.15.87:0/1694455897 >> v2:172.21.15.169:3300/0 conn(0x7faf9c0d0040 0x7faf9c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 >> v1:172.21.15.169:6789/0 conn(0x7faf9c15b740 legacy=0x7faf9c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0715d0 con 0x7faf9c1550b0 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7faf8c001680 con 0x7faf9c1550b0 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 >> v1:172.21.15.87:6789/0 conn(0x7faf9c1550b0 legacy=0x7faf9c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf80005d60 0x7faf80006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf80006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf800073e0 0x7faf8000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0712f0 con 0x7faf80006a20 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8000e260 con 0x7faf800073e0 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8000e460 con 0x7faf80005d60 2024-04-25T10:25:26.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf91400640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf80006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf9a800640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf800073e0 0x7faf8000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf99e00640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf80005d60 0x7faf80006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf91400640 1 -- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf80005d60 msgr2=0x7faf80006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf91400640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf80005d60 0x7faf80006160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf91400640 1 -- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf800073e0 msgr2=0x7faf8000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf91400640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf800073e0 0x7faf8000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf91400640 1 -- 172.21.15.87:0/1694455897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80005740 con 0x7faf80006a20 2024-04-25T10:25:26.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.863+0000 7faf9a800640 1 -- 172.21.15.87:0/1694455897 reap_dead start 2024-04-25T10:25:26.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7faf91400640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf80006ea0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7faf84009ae0 tx=0x7faf8401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f08e79dc74008f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840b2070 con 0x7faf80006a20 2024-04-25T10:25:26.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf840a5550 con 0x7faf80006a20 2024-04-25T10:25:26.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7faf99400640 1 -- 172.21.15.87:0/1694455897 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840096b0 con 0x7faf80006a20 2024-04-25T10:25:26.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7fafa1000640 1 -- 172.21.15.87:0/1694455897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 msgr2=0x7faf80006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7fafa1000640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf80006ea0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7faf84009ae0 tx=0x7faf8401ca70 comp rx=0 tx=0).stop 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7fafa1000640 1 -- 172.21.15.87:0/1694455897 shutdown_connections 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7fafa1000640 1 --2- 172.21.15.87:0/1694455897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf80006ea0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7fafa1000640 1 -- 172.21.15.87:0/1694455897 >> 172.21.15.87:0/1694455897 conn(0x7faf9c066b70 msgr2=0x7faf9c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7fafa1000640 1 -- 172.21.15.87:0/1694455897 shutdown_connections 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.867+0000 7fafa1000640 1 -- 172.21.15.87:0/1694455897 wait complete. 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 Processor -- start 2024-04-25T10:25:26.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 -- start start 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf9c1f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf9c148330 0x7faf9c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9c0ad840 0x7faf9c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0720b0 con 0x7faf80006a20 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c071b90 con 0x7faf9c148330 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15bb90 con 0x7faf9c0ad840 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf9c1f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf9c1f7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48510/0 (socket says 172.21.15.87:48510) 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 -- 172.21.15.87:0/948985524 learned_addr learned my addr 172.21.15.87:0/948985524 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf9a800640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9c0ad840 0x7faf9c0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf91400640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf9c148330 0x7faf9c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 -- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9c0ad840 msgr2=0x7faf9c0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9c0ad840 0x7faf9c0adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 -- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf9c148330 msgr2=0x7faf9c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf9c148330 0x7faf9c156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 -- 172.21.15.87:0/948985524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c1f7cd0 con 0x7faf80006a20 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf99e00640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf9c1f7790 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00e7b0 tx=0x7faf8c00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf92a00640 1 -- 172.21.15.87:0/948985524 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00c6c0 con 0x7faf80006a20 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf92a00640 1 -- 172.21.15.87:0/948985524 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c018070 con 0x7faf80006a20 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c153b00 con 0x7faf80006a20 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7faf92a00640 1 -- 172.21.15.87:0/948985524 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00cac0 con 0x7faf80006a20 2024-04-25T10:25:26.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.871+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c152050 con 0x7faf80006a20 2024-04-25T10:25:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.875+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c04ef30 con 0x7faf80006a20 2024-04-25T10:25:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.875+0000 7faf92a00640 1 -- 172.21.15.87:0/948985524 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf8c00cc60 con 0x7faf80006a20 2024-04-25T10:25:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.875+0000 7faf92a00640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f90 0x7faf78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.875+0000 7faf92a00640 1 -- 172.21.15.87:0/948985524 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faf8c084db0 con 0x7faf80006a20 2024-04-25T10:25:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.875+0000 7faf91400640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f90 0x7faf78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.875+0000 7faf91400640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f90 0x7faf78069450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faf840af000 tx=0x7faf840046d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:26.879+0000 7faf92a00640 1 -- 172.21.15.87:0/948985524 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf8c089070 con 0x7faf80006a20 2024-04-25T10:25:27.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.043+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faf9c15e200 con 0x7faf80006a20 2024-04-25T10:25:27.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.043+0000 7faf92a00640 1 -- 172.21.15.87:0/948985524 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v165) v1 ==== 80+0+0 (secure 0 0 0) 0x7faf9c1f7cd0 con 0x7faf80006a20 2024-04-25T10:25:27.044 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f90 msgr2=0x7faf78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f90 0x7faf78069450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faf840af000 tx=0x7faf840046d0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 msgr2=0x7faf9c1f7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf9c1f7790 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00e7b0 tx=0x7faf8c00ebb0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 shutdown_connections 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7faf78066f90 0x7faf78069450 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf9c0ad840 0x7faf9c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf9c148330 0x7faf9c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 --2- 172.21.15.87:0/948985524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf80006a20 0x7faf9c1f7790 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 >> 172.21.15.87:0/948985524 conn(0x7faf9c066b70 msgr2=0x7faf9c06d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:27.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 shutdown_connections 2024-04-25T10:25:27.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.047+0000 7fafa1000640 1 -- 172.21.15.87:0/948985524 wait complete. 2024-04-25T10:25:27.065 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:25:27.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:27.065 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:27.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:25:27.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:27.110 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:25:27.166 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:25:27.166 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:27.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:27.175 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:27.175 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:27.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:27.221 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:25:27.222 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:27.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:27.230 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:27.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f8152800640 1 Processor -- start 2024-04-25T10:25:27.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f8152800640 1 -- start start 2024-04-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f8152800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f814c0d0040 0x7f814c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f8152800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 0x7f814c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f8152800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c071050 con 0x7f814c1550b0 2024-04-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f8152800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c15d040 con 0x7f814c0d0040 2024-04-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f8152800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0712f0 con 0x7f814c15b740 2024-04-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.399+0000 7f814b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f814c1550b0 0x7f814c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41326/0 (socket says 172.21.15.87:41326) 2024-04-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814b400640 1 -- 172.21.15.87:0/816943071 learned_addr learned my addr 172.21.15.87:0/816943071 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814aa00640 1 --2- 172.21.15.87:0/816943071 >> v2:172.21.15.169:3300/0 conn(0x7f814c0d0040 0x7f814c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814be00640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 0x7f814c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865219366 0 0) 0x7f814c071050 con 0x7f814c1550b0 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8130005740 con 0x7f814c1550b0 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2752415657 0 0) 0x7f8130005740 con 0x7f814c1550b0 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 msgr2=0x7f814c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 0x7f814c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 >> v2:172.21.15.169:3300/0 conn(0x7f814c0d0040 msgr2=0x7f814c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 --2- 172.21.15.87:0/816943071 >> v2:172.21.15.169:3300/0 conn(0x7f814c0d0040 0x7f814c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0715d0 con 0x7f814c1550b0 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f813c001680 con 0x7f814c1550b0 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814aa00640 1 --2- 172.21.15.87:0/816943071 >> v2:172.21.15.169:3300/0 conn(0x7f814c0d0040 0x7f814c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 >> v1:172.21.15.87:6789/0 conn(0x7f814c1550b0 legacy=0x7f814c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8130005bd0 0x7f8130005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8130006a20 0x7f8130006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 0x7f81300073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15d040 con 0x7f8130005bd0 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0712f0 con 0x7f8130006a20 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813000dbe0 con 0x7f814c15b740 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814b400640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8130005bd0 0x7f8130005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814aa00640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8130006a20 0x7f8130006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814be00640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 0x7f81300073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814b400640 1 -- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 msgr2=0x7f81300073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814b400640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814c15b740 0x7f81300073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814b400640 1 -- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8130006a20 msgr2=0x7f8130006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814b400640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8130006a20 0x7f8130006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.403+0000 7f814b400640 1 -- 172.21.15.87:0/816943071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130005740 con 0x7f8130005bd0 2024-04-25T10:25:27.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f814b400640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8130005bd0 0x7f8130005ff0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c240 tx=0x7f813c01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ac6437731ec954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:27.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c007d60 con 0x7f8130005bd0 2024-04-25T10:25:27.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c0a9020 con 0x7f8130005bd0 2024-04-25T10:25:27.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f814a000640 1 -- 172.21.15.87:0/816943071 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c005ae0 con 0x7f8130005bd0 2024-04-25T10:25:27.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- 172.21.15.87:0/816943071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8130005bd0 msgr2=0x7f8130005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 --2- 172.21.15.87:0/816943071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8130005bd0 0x7f8130005ff0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c240 tx=0x7f813c01cc90 comp rx=0 tx=0).stop 2024-04-25T10:25:27.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f814be00640 1 -- 172.21.15.87:0/816943071 reap_dead start 2024-04-25T10:25:27.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- 172.21.15.87:0/816943071 shutdown_connections 2024-04-25T10:25:27.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- 172.21.15.87:0/816943071 >> 172.21.15.87:0/816943071 conn(0x7f814c066b70 msgr2=0x7f814c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:27.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- 172.21.15.87:0/816943071 shutdown_connections 2024-04-25T10:25:27.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- 172.21.15.87:0/816943071 wait complete. 2024-04-25T10:25:27.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 Processor -- start 2024-04-25T10:25:27.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- start start 2024-04-25T10:25:27.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8130005bd0 0x7f814c13f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0b1230 0x7f814c150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c150aa0 0x7f814c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15bcd0 con 0x7f814c150aa0 2024-04-25T10:25:27.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15bb50 con 0x7f814c0b1230 2024-04-25T10:25:27.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.407+0000 7f8152800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15c840 con 0x7f8130005bd0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c150aa0 0x7f814c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c150aa0 0x7f814c156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60242/0 (socket says 172.21.15.87:60242) 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 -- 172.21.15.87:0/3960653437 learned_addr learned my addr 172.21.15.87:0/3960653437 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814aa00640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0b1230 0x7f814c150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814b400640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8130005bd0 0x7f814c13f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 -- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8130005bd0 msgr2=0x7f814c13f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8130005bd0 0x7f814c13f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 -- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0b1230 msgr2=0x7f814c150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0b1230 0x7f814c150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 -- 172.21.15.87:0/3960653437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0b0bd0 con 0x7f814c150aa0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f814be00640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c150aa0 0x7f814c156110 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f814c070930 tx=0x7f814400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8143400640 1 -- 172.21.15.87:0/3960653437 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144015430 con 0x7f814c150aa0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8143400640 1 -- 172.21.15.87:0/3960653437 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814401d070 con 0x7f814c150aa0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c155630 con 0x7f814c150aa0 2024-04-25T10:25:27.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8143400640 1 -- 172.21.15.87:0/3960653437 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144015810 con 0x7f814c150aa0 2024-04-25T10:25:27.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c156650 con 0x7f814c150aa0 2024-04-25T10:25:27.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c04ef30 con 0x7f814c150aa0 2024-04-25T10:25:27.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8143400640 1 -- 172.21.15.87:0/3960653437 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81440159b0 con 0x7f814c150aa0 2024-04-25T10:25:27.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8143400640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f812c066f60 0x7f812c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.411+0000 7f8143400640 1 -- 172.21.15.87:0/3960653437 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8144085bb0 con 0x7f814c150aa0 2024-04-25T10:25:27.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.415+0000 7f814b400640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f812c066f60 0x7f812c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.415+0000 7f814b400640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f812c066f60 0x7f812c069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f813c0aa000 tx=0x7f813c001ac0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:27.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.415+0000 7f8143400640 1 -- 172.21.15.87:0/3960653437 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8144010040 con 0x7f814c150aa0 2024-04-25T10:25:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.603+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f814c15cc10 con 0x7f814c150aa0 2024-04-25T10:25:27.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.603+0000 7f8143400640 1 -- 172.21.15.87:0/3960653437 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f8144056a50 con 0x7f814c150aa0 2024-04-25T10:25:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:27.605 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:25:27.605 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f812c066f60 msgr2=0x7f812c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f812c066f60 0x7f812c069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f813c0aa000 tx=0x7f813c001ac0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c150aa0 msgr2=0x7f814c156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c150aa0 0x7f814c156110 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f814c070930 tx=0x7f814400f580 comp rx=0 tx=0).stop 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 shutdown_connections 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f812c066f60 0x7f812c069420 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c150aa0 0x7f814c156110 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814c0b1230 0x7f814c150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 --2- 172.21.15.87:0/3960653437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8130005bd0 0x7f814c13f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 >> 172.21.15.87:0/3960653437 conn(0x7f814c066b70 msgr2=0x7f814c159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 shutdown_connections 2024-04-25T10:25:27.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.607+0000 7f8152800640 1 -- 172.21.15.87:0/3960653437 wait complete. 2024-04-25T10:25:27.626 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:27.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.763+0000 7f14cd400640 1 Processor -- start 2024-04-25T10:25:27.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.763+0000 7f14cd400640 1 -- start start 2024-04-25T10:25:27.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.763+0000 7f14cd400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14c80d0040 0x7f14c815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.763+0000 7f14cd400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14cd400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c8071050 con 0x7f14c81550b0 2024-04-25T10:25:27.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14cd400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c815d040 con 0x7f14c80d0040 2024-04-25T10:25:27.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14cd400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c80712f0 con 0x7f14c815b740 2024-04-25T10:25:27.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14c80d0040 0x7f14c815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c6800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f14c81550b0 0x7f14c80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41330/0 (socket says 172.21.15.87:41330) 2024-04-25T10:25:27.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c6800640 1 -- 172.21.15.87:0/3217819353 learned_addr learned my addr 172.21.15.87:0/3217819353 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:27.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c7200640 1 --2- 172.21.15.87:0/3217819353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5e00640 1 -- 172.21.15.87:0/3217819353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 msgr2=0x7f14c80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5e00640 1 --2- 172.21.15.87:0/3217819353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5e00640 1 -- 172.21.15.87:0/3217819353 >> v1:172.21.15.87:6789/0 conn(0x7f14c81550b0 legacy=0x7f14c80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5e00640 1 -- 172.21.15.87:0/3217819353 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c80715d0 con 0x7f14c80d0040 2024-04-25T10:25:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5400640 1 -- 172.21.15.87:0/3217819353 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518896705 0 0) 0x7f14c8071050 con 0x7f14c81550b0 2024-04-25T10:25:27.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c7200640 1 --2- 172.21.15.87:0/3217819353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:27.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5e00640 1 --2- 172.21.15.87:0/3217819353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c80d0040 0x7f14c815b200 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f14b40097c0 tx=0x7f14b401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a73508fe790bf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:27.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5400640 1 -- 172.21.15.87:0/3217819353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40a7070 con 0x7f14c80d0040 2024-04-25T10:25:27.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5400640 1 -- 172.21.15.87:0/3217819353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b40b0070 con 0x7f14c80d0040 2024-04-25T10:25:27.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.767+0000 7f14c5400640 1 -- 172.21.15.87:0/3217819353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40ab890 con 0x7f14c80d0040 2024-04-25T10:25:27.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- 172.21.15.87:0/3217819353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c80d0040 msgr2=0x7f14c815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 --2- 172.21.15.87:0/3217819353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c80d0040 0x7f14c815b200 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f14b40097c0 tx=0x7f14b401c470 comp rx=0 tx=0).stop 2024-04-25T10:25:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- 172.21.15.87:0/3217819353 shutdown_connections 2024-04-25T10:25:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 --2- 172.21.15.87:0/3217819353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 --2- 172.21.15.87:0/3217819353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c80d0040 0x7f14c815b200 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- 172.21.15.87:0/3217819353 >> 172.21.15.87:0/3217819353 conn(0x7f14c8066b70 msgr2=0x7f14c8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- 172.21.15.87:0/3217819353 shutdown_connections 2024-04-25T10:25:27.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- 172.21.15.87:0/3217819353 wait complete. 2024-04-25T10:25:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 Processor -- start 2024-04-25T10:25:27.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- start start 2024-04-25T10:25:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c80d0040 0x7f14c8147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c81550b0 0x7f14c81368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c815bcd0 con 0x7f14c81550b0 2024-04-25T10:25:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c815bb50 con 0x7f14c80d0040 2024-04-25T10:25:27.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14cd400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c815c030 con 0x7f14c815b740 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c81550b0 0x7f14c81368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c81550b0 0x7f14c81368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60264/0 (socket says 172.21.15.87:60264) 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 -- 172.21.15.87:0/929699658 learned_addr learned my addr 172.21.15.87:0/929699658 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c6800640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c80d0040 0x7f14c8147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c7200640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 -- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 msgr2=0x7f14c815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 -- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c80d0040 msgr2=0x7f14c8147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c80d0040 0x7f14c8147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 -- 172.21.15.87:0/929699658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c81352c0 con 0x7f14c81550b0 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.771+0000 7f14c5e00640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c81550b0 0x7f14c81368b0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f14b40097c0 tx=0x7f14b401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14bea00640 1 -- 172.21.15.87:0/929699658 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4004220 con 0x7f14c81550b0 2024-04-25T10:25:27.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14bea00640 1 -- 172.21.15.87:0/929699658 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b40043c0 con 0x7f14c81550b0 2024-04-25T10:25:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c8136df0 con 0x7f14c81550b0 2024-04-25T10:25:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14bea00640 1 -- 172.21.15.87:0/929699658 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4003820 con 0x7f14c81550b0 2024-04-25T10:25:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c8157930 con 0x7f14c81550b0 2024-04-25T10:25:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c81582d0 con 0x7f14c81550b0 2024-04-25T10:25:27.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14bea00640 1 -- 172.21.15.87:0/929699658 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14b40039c0 con 0x7f14c81550b0 2024-04-25T10:25:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14bea00640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14a8066f60 0x7f14a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14bea00640 1 -- 172.21.15.87:0/929699658 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f14b411faa0 con 0x7f14c81550b0 2024-04-25T10:25:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14c6800640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14a8066f60 0x7f14a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:27.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.775+0000 7f14c6800640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14a8066f60 0x7f14a8069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f14b8007fd0 tx=0x7f14b8007ee0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:27.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.779+0000 7f14bea00640 1 -- 172.21.15.87:0/929699658 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14c81582d0 con 0x7f14c81550b0 2024-04-25T10:25:27.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.935+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14c804ef30 con 0x7f14c81550b0 2024-04-25T10:25:27.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14bea00640 1 -- 172.21.15.87:0/929699658 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16765 (secure 0 0 0) 0x7f14b4016090 con 0x7f14c81550b0 2024-04-25T10:25:27.937 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:27.938 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":165,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:07.649563+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:42.650296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:43.940319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.29","mappings":[{"from":0,"to":2}]},{"pgid":"25.39","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:27.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14a8066f60 msgr2=0x7f14a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14a8066f60 0x7f14a8069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f14b8007fd0 tx=0x7f14b8007ee0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c81550b0 msgr2=0x7f14c81368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c81550b0 0x7f14c81368b0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f14b40097c0 tx=0x7f14b401acb0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 shutdown_connections 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f14a8066f60 0x7f14a8069420 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c815b740 0x7f14c815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c81550b0 0x7f14c81368b0 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 --2- 172.21.15.87:0/929699658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c80d0040 0x7f14c8147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 >> 172.21.15.87:0/929699658 conn(0x7f14c8066b70 msgr2=0x7f14c80cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 shutdown_connections 2024-04-25T10:25:27.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:27.939+0000 7f14cd400640 1 -- 172.21.15.87:0/929699658 wait complete. 2024-04-25T10:25:27.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:24:42.650296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:24:43.940319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:27.959 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:28.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb04600640 1 Processor -- start 2024-04-25T10:25:28.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb04600640 1 -- start start 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb04600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb04600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feafc070350 0x7feafc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb04600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feafc15b080 con 0x7feafc15bac0 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb04600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feafc15adb0 con 0x7feafc070350 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb04600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc15eb60 con 0x7feafc0aef80 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feafc070350 0x7feafc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feafc070350 0x7feafc15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60270/0 (socket says 172.21.15.87:60270) 2024-04-25T10:25:28.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01200640 1 -- 172.21.15.87:0/2454542395 learned_addr learned my addr 172.21.15.87:0/2454542395 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:28.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01c00640 1 --2- 172.21.15.87:0/2454542395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01200640 1 -- 172.21.15.87:0/2454542395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 msgr2=0x7feafc15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:28.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01200640 1 --2- 172.21.15.87:0/2454542395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01200640 1 -- 172.21.15.87:0/2454542395 >> v1:172.21.15.169:6789/0 conn(0x7feafc15bac0 legacy=0x7feafc0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.099+0000 7feb01200640 1 -- 172.21.15.87:0/2454542395 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc15ee40 con 0x7feafc070350 2024-04-25T10:25:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb01200640 1 --2- 172.21.15.87:0/2454542395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc070350 0x7feafc15b580 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7feae800bb10 tx=0x7feae801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2fa22083bc3298 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feafbe00640 1 -- 172.21.15.87:0/2454542395 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80b4070 con 0x7feafc070350 2024-04-25T10:25:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feafbe00640 1 -- 172.21.15.87:0/2454542395 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae80a7ce0 con 0x7feafc070350 2024-04-25T10:25:28.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feafbe00640 1 -- 172.21.15.87:0/2454542395 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80af660 con 0x7feafc070350 2024-04-25T10:25:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 -- 172.21.15.87:0/2454542395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc070350 msgr2=0x7feafc15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 --2- 172.21.15.87:0/2454542395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc070350 0x7feafc15b580 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7feae800bb10 tx=0x7feae801ec30 comp rx=0 tx=0).stop 2024-04-25T10:25:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 -- 172.21.15.87:0/2454542395 shutdown_connections 2024-04-25T10:25:28.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 --2- 172.21.15.87:0/2454542395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc070350 0x7feafc15b580 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 --2- 172.21.15.87:0/2454542395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 -- 172.21.15.87:0/2454542395 >> 172.21.15.87:0/2454542395 conn(0x7feafc0669a0 msgr2=0x7feafc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 -- 172.21.15.87:0/2454542395 shutdown_connections 2024-04-25T10:25:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 -- 172.21.15.87:0/2454542395 wait complete. 2024-04-25T10:25:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 Processor -- start 2024-04-25T10:25:28.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.103+0000 7feb04600640 1 -- start start 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feafc070350 0x7feafc150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc15bac0 0x7feafc152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc06c0d0 con 0x7feafc15bac0 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0b3ae0 con 0x7feafc070350 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc06c250 con 0x7feafc0aef80 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc15bac0 0x7feafc152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc15bac0 0x7feafc152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60274/0 (socket says 172.21.15.87:60274) 2024-04-25T10:25:28.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 -- 172.21.15.87:0/2036947405 learned_addr learned my addr 172.21.15.87:0/2036947405 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb01c00640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feafc070350 0x7feafc150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb01200640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 -- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 msgr2=0x7feafc151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 -- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feafc070350 msgr2=0x7feafc150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feafc070350 0x7feafc150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 -- 172.21.15.87:0/2036947405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc1507d0 con 0x7feafc15bac0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb02600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc15bac0 0x7feafc152100 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c760 tx=0x7feaf400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feaf9600640 1 -- 172.21.15.87:0/2036947405 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4003b80 con 0x7feafc15bac0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feaf9600640 1 -- 172.21.15.87:0/2036947405 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4003d20 con 0x7feafc15bac0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feaf9600640 1 -- 172.21.15.87:0/2036947405 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400aa30 con 0x7feafc15bac0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc150b10 con 0x7feafc15bac0 2024-04-25T10:25:28.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc1510d0 con 0x7feafc15bac0 2024-04-25T10:25:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feafc04ef30 con 0x7feafc15bac0 2024-04-25T10:25:28.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.107+0000 7feaf9600640 1 -- 172.21.15.87:0/2036947405 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaf400abd0 con 0x7feafc15bac0 2024-04-25T10:25:28.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.111+0000 7feaf9600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feadc066f60 0x7feadc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.111+0000 7feaf9600640 1 -- 172.21.15.87:0/2036947405 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feaf4084d40 con 0x7feafc15bac0 2024-04-25T10:25:28.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.111+0000 7feb01c00640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feadc066f60 0x7feadc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.111+0000 7feb01c00640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feadc066f60 0x7feadc069420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7feaf0007a60 tx=0x7feaf0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.115+0000 7feaf9600640 1 -- 172.21.15.87:0/2036947405 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaf4055750 con 0x7feafc15bac0 2024-04-25T10:25:28.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.271+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feafc0724c0 con 0x7feafc15bac0 2024-04-25T10:25:28.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.271+0000 7feaf9600640 1 -- 172.21.15.87:0/2036947405 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7feaf40597f0 con 0x7feafc15bac0 2024-04-25T10:25:28.272 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feadc066f60 msgr2=0x7feadc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feadc066f60 0x7feadc069420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7feaf0007a60 tx=0x7feaf0008040 comp rx=0 tx=0).stop 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc15bac0 msgr2=0x7feafc152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc15bac0 0x7feafc152100 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c760 tx=0x7feaf400cc20 comp rx=0 tx=0).stop 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 shutdown_connections 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feadc066f60 0x7feadc069420 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feafc15bac0 0x7feafc152100 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feafc0aef80 0x7feafc151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 --2- 172.21.15.87:0/2036947405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feafc070350 0x7feafc150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 >> 172.21.15.87:0/2036947405 conn(0x7feafc0669a0 msgr2=0x7feafc066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 shutdown_connections 2024-04-25T10:25:28.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.275+0000 7feb04600640 1 -- 172.21.15.87:0/2036947405 wait complete. 2024-04-25T10:25:28.293 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:25:28.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.431+0000 7fa307400640 1 Processor -- start 2024-04-25T10:25:28.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.431+0000 7fa307400640 1 -- start start 2024-04-25T10:25:28.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.431+0000 7fa307400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa300070350 0x7fa30015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.431+0000 7fa307400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 0x7fa3000b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.431+0000 7fa307400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30015b080 con 0x7fa3000aef80 2024-04-25T10:25:28.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.431+0000 7fa307400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30015adb0 con 0x7fa300070350 2024-04-25T10:25:28.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.431+0000 7fa307400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30015eb60 con 0x7fa3000b31c0 2024-04-25T10:25:28.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa3000aef80 0x7fa30015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41332/0 (socket says 172.21.15.87:41332) 2024-04-25T10:25:28.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 -- 172.21.15.87:0/1075660116 learned_addr learned my addr 172.21.15.87:0/1075660116 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:28.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa305400640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 0x7fa3000b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093345218 0 0) 0x7fa30015b080 con 0x7fa3000aef80 2024-04-25T10:25:28.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ffe00640 1 --2- 172.21.15.87:0/1075660116 >> v2:172.21.15.169:3300/0 conn(0x7fa300070350 0x7fa30015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e4003530 con 0x7fa3000aef80 2024-04-25T10:25:28.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2563120226 0 0) 0x7fa2e4003530 con 0x7fa3000aef80 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 msgr2=0x7fa3000b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 0x7fa3000b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 >> v2:172.21.15.169:3300/0 conn(0x7fa300070350 msgr2=0x7fa30015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 --2- 172.21.15.87:0/1075660116 >> v2:172.21.15.169:3300/0 conn(0x7fa300070350 0x7fa30015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30015ee40 con 0x7fa3000aef80 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fa2f0001a20 con 0x7fa3000aef80 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 >> v1:172.21.15.87:6789/0 conn(0x7fa3000aef80 legacy=0x7fa30015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e40039c0 0x7fa2e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 0x7fa2e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e4004d50 0x7fa2e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30015adb0 con 0x7fa2e40039c0 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30015eb60 con 0x7fa2e4004d50 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e400b9d0 con 0x7fa3000b31c0 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e40039c0 0x7fa2e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa2ffe00640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 0x7fa2e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa305400640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e4004d50 0x7fa2e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 -- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 msgr2=0x7fa2e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3000b31c0 0x7fa2e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 -- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e4004d50 msgr2=0x7fa2e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e4004d50 0x7fa2e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.435+0000 7fa304a00640 1 -- 172.21.15.87:0/1075660116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4003530 con 0x7fa2e40039c0 2024-04-25T10:25:28.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa304a00640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e40039c0 0x7fa2e4003de0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000a8b0 tx=0x7fa2f001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a985ff719275a663 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000a4d0 con 0x7fa2e40039c0 2024-04-25T10:25:28.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f00ad630 con 0x7fa2e40039c0 2024-04-25T10:25:28.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa2ff400640 1 -- 172.21.15.87:0/1075660116 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00ada20 con 0x7fa2e40039c0 2024-04-25T10:25:28.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 -- 172.21.15.87:0/1075660116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e40039c0 msgr2=0x7fa2e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 --2- 172.21.15.87:0/1075660116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e40039c0 0x7fa2e4003de0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000a8b0 tx=0x7fa2f001d380 comp rx=0 tx=0).stop 2024-04-25T10:25:28.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa305400640 1 -- 172.21.15.87:0/1075660116 reap_dead start 2024-04-25T10:25:28.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 -- 172.21.15.87:0/1075660116 shutdown_connections 2024-04-25T10:25:28.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 -- 172.21.15.87:0/1075660116 >> 172.21.15.87:0/1075660116 conn(0x7fa3000669a0 msgr2=0x7fa300066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:28.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 -- 172.21.15.87:0/1075660116 shutdown_connections 2024-04-25T10:25:28.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 -- 172.21.15.87:0/1075660116 wait complete. 2024-04-25T10:25:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 Processor -- start 2024-04-25T10:25:28.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.439+0000 7fa307400640 1 -- start start 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e40039c0 0x7fa300151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3000afff0 0x7fa3001f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3001596f0 0x7fa3000ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300165810 con 0x7fa3000afff0 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000b39e0 con 0x7fa3001596f0 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa300165990 con 0x7fa2e40039c0 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3000afff0 0x7fa3001f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3000afff0 0x7fa3001f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60288/0 (socket says 172.21.15.87:60288) 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 -- 172.21.15.87:0/2075486245 learned_addr learned my addr 172.21.15.87:0/2075486245 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa304a00640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e40039c0 0x7fa300151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa305400640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3001596f0 0x7fa3000ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 -- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e40039c0 msgr2=0x7fa300151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e40039c0 0x7fa300151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 -- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3001596f0 msgr2=0x7fa3000ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3001596f0 0x7fa3000ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 -- 172.21.15.87:0/2075486245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000af550 con 0x7fa3000afff0 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2ffe00640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3000afff0 0x7fa3001f6e90 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec011b00 tx=0x7fa2ec011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2fcc00640 1 -- 172.21.15.87:0/2075486245 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00fb50 con 0x7fa3000afff0 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2fcc00640 1 -- 172.21.15.87:0/2075486245 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec00fcf0 con 0x7fa3000afff0 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2fcc00640 1 -- 172.21.15.87:0/2075486245 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00fea0 con 0x7fa3000afff0 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3001f73d0 con 0x7fa3000afff0 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3000af7f0 con 0x7fa3000afff0 2024-04-25T10:25:28.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30004ef30 con 0x7fa3000afff0 2024-04-25T10:25:28.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.443+0000 7fa2fcc00640 1 -- 172.21.15.87:0/2075486245 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2ec012040 con 0x7fa3000afff0 2024-04-25T10:25:28.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.447+0000 7fa2fcc00640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa2e0066f60 0x7fa2e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.447+0000 7fa2fcc00640 1 -- 172.21.15.87:0/2075486245 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa2ec0854d0 con 0x7fa3000afff0 2024-04-25T10:25:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.447+0000 7fa304a00640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa2e0066f60 0x7fa2e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.447+0000 7fa304a00640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa2e0066f60 0x7fa2e0069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00ad000 tx=0x7fa2f0007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.451+0000 7fa2fcc00640 1 -- 172.21.15.87:0/2075486245 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2ec019030 con 0x7fa3000afff0 2024-04-25T10:25:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa30006d9d0 con 0x7fa3000afff0 2024-04-25T10:25:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa2fcc00640 1 -- 172.21.15.87:0/2075486245 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v165) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2ec0564a0 con 0x7fa3000afff0 2024-04-25T10:25:28.602 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa2e0066f60 msgr2=0x7fa2e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa2e0066f60 0x7fa2e0069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00ad000 tx=0x7fa2f0007040 comp rx=0 tx=0).stop 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3000afff0 msgr2=0x7fa3001f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3000afff0 0x7fa3001f6e90 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec011b00 tx=0x7fa2ec011f00 comp rx=0 tx=0).stop 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 shutdown_connections 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa2e0066f60 0x7fa2e0069420 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3001596f0 0x7fa3000ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3000afff0 0x7fa3001f6e90 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 --2- 172.21.15.87:0/2075486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e40039c0 0x7fa300151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 >> 172.21.15.87:0/2075486245 conn(0x7fa3000669a0 msgr2=0x7fa30015b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:28.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 shutdown_connections 2024-04-25T10:25:28.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.603+0000 7fa307400640 1 -- 172.21.15.87:0/2075486245 wait complete. 2024-04-25T10:25:28.623 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:25:28.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd83800640 1 Processor -- start 2024-04-25T10:25:28.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd83800640 1 -- start start 2024-04-25T10:25:28.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd83800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0721f0 0x7fcd7c166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd83800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c071360 con 0x7fcd7c159540 2024-04-25T10:25:28.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd83800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c159920 con 0x7fcd7c0aeda0 2024-04-25T10:25:28.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd83800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c15c2f0 con 0x7fcd7c0721f0 2024-04-25T10:25:28.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd80e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcd7c159540 0x7fcd7c0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41334/0 (socket says 172.21.15.87:41334) 2024-04-25T10:25:28.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd80e00640 1 -- 172.21.15.87:0/3673459341 learned_addr learned my addr 172.21.15.87:0/3673459341 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:28.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd81800640 1 --2- 172.21.15.87:0/3673459341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0721f0 0x7fcd7c166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd7b400640 1 -- 172.21.15.87:0/3673459341 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733208428 0 0) 0x7fcd7c071360 con 0x7fcd7c159540 2024-04-25T10:25:28.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd7b400640 1 -- 172.21.15.87:0/3673459341 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd64003530 con 0x7fcd7c159540 2024-04-25T10:25:28.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd7b400640 1 -- 172.21.15.87:0/3673459341 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730955910 0 0) 0x7fcd7c159920 con 0x7fcd7c0aeda0 2024-04-25T10:25:28.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd7b400640 1 -- 172.21.15.87:0/3673459341 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd7c071360 con 0x7fcd7c0aeda0 2024-04-25T10:25:28.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd81800640 1 -- 172.21.15.87:0/3673459341 >> v1:172.21.15.169:6789/0 conn(0x7fcd7c0aeda0 legacy=0x7fcd7c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd81800640 1 -- 172.21.15.87:0/3673459341 >> v1:172.21.15.87:6789/0 conn(0x7fcd7c159540 legacy=0x7fcd7c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.763+0000 7fcd81800640 1 -- 172.21.15.87:0/3673459341 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c15c5d0 con 0x7fcd7c0721f0 2024-04-25T10:25:28.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd81800640 1 --2- 172.21.15.87:0/3673459341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0721f0 0x7fcd7c166920 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009dc0 tx=0x7fcd7401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4506493c2b4c2cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd7b400640 1 -- 172.21.15.87:0/3673459341 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd740b2070 con 0x7fcd7c0721f0 2024-04-25T10:25:28.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd7b400640 1 -- 172.21.15.87:0/3673459341 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd740a5a20 con 0x7fcd7c0721f0 2024-04-25T10:25:28.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd7b400640 1 -- 172.21.15.87:0/3673459341 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd740a5dd0 con 0x7fcd7c0721f0 2024-04-25T10:25:28.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd83800640 1 -- 172.21.15.87:0/3673459341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0721f0 msgr2=0x7fcd7c166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd83800640 1 --2- 172.21.15.87:0/3673459341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0721f0 0x7fcd7c166920 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009dc0 tx=0x7fcd7401c960 comp rx=0 tx=0).stop 2024-04-25T10:25:28.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd83800640 1 -- 172.21.15.87:0/3673459341 shutdown_connections 2024-04-25T10:25:28.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd83800640 1 --2- 172.21.15.87:0/3673459341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0721f0 0x7fcd7c166920 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd83800640 1 -- 172.21.15.87:0/3673459341 >> 172.21.15.87:0/3673459341 conn(0x7fcd7c0669a0 msgr2=0x7fcd7c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:28.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd83800640 1 -- 172.21.15.87:0/3673459341 shutdown_connections 2024-04-25T10:25:28.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.767+0000 7fcd83800640 1 -- 172.21.15.87:0/3673459341 wait complete. 2024-04-25T10:25:28.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 Processor -- start 2024-04-25T10:25:28.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 -- start start 2024-04-25T10:25:28.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd7c0721f0 0x7fcd7c151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0aeda0 0x7fcd7c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd7c159540 0x7fcd7c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c169680 con 0x7fcd7c159540 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c06c0d0 con 0x7fcd7c0721f0 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c169800 con 0x7fcd7c0aeda0 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd7c159540 0x7fcd7c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd7c159540 0x7fcd7c150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60296/0 (socket says 172.21.15.87:60296) 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 -- 172.21.15.87:0/1368853704 learned_addr learned my addr 172.21.15.87:0/1368853704 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd80e00640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd7c0721f0 0x7fcd7c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd7be00640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0aeda0 0x7fcd7c153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 -- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0aeda0 msgr2=0x7fcd7c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0aeda0 0x7fcd7c153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 -- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd7c0721f0 msgr2=0x7fcd7c151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd7c0721f0 0x7fcd7c151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 -- 172.21.15.87:0/1368853704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c1550b0 con 0x7fcd7c159540 2024-04-25T10:25:28.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd7be00640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0aeda0 0x7fcd7c153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:28.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd80e00640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd7c0721f0 0x7fcd7c151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:28.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd81800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd7c159540 0x7fcd7c150900 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fcd7401a590 tx=0x7fcd74019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd78c00640 1 -- 172.21.15.87:0/1368853704 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd740b2050 con 0x7fcd7c159540 2024-04-25T10:25:28.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd78c00640 1 -- 172.21.15.87:0/1368853704 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd740be070 con 0x7fcd7c159540 2024-04-25T10:25:28.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c154af0 con 0x7fcd7c159540 2024-04-25T10:25:28.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd78c00640 1 -- 172.21.15.87:0/1368853704 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74004040 con 0x7fcd7c159540 2024-04-25T10:25:28.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.771+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c15b0f0 con 0x7fcd7c159540 2024-04-25T10:25:28.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.775+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c04ef30 con 0x7fcd7c159540 2024-04-25T10:25:28.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.775+0000 7fcd78c00640 1 -- 172.21.15.87:0/1368853704 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd740041e0 con 0x7fcd7c159540 2024-04-25T10:25:28.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.775+0000 7fcd78c00640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd5c066f60 0x7fcd5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:28.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.775+0000 7fcd78c00640 1 -- 172.21.15.87:0/1368853704 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcd7411fff0 con 0x7fcd7c159540 2024-04-25T10:25:28.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.775+0000 7fcd80e00640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd5c066f60 0x7fcd5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:28.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.775+0000 7fcd80e00640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd5c066f60 0x7fcd5c069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcd70007fd0 tx=0x7fcd70009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:28.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.779+0000 7fcd78c00640 1 -- 172.21.15.87:0/1368853704 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd740c8020 con 0x7fcd7c159540 2024-04-25T10:25:28.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.931+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd7c0cef10 con 0x7fcd7c159540 2024-04-25T10:25:28.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.931+0000 7fcd78c00640 1 -- 172.21.15.87:0/1368853704 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v165) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd74016580 con 0x7fcd7c159540 2024-04-25T10:25:28.933 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd5c066f60 msgr2=0x7fcd5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd5c066f60 0x7fcd5c069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcd70007fd0 tx=0x7fcd70009040 comp rx=0 tx=0).stop 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd7c159540 msgr2=0x7fcd7c150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd7c159540 0x7fcd7c150900 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fcd7401a590 tx=0x7fcd74019700 comp rx=0 tx=0).stop 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 shutdown_connections 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcd5c066f60 0x7fcd5c069420 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd7c159540 0x7fcd7c150900 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd7c0aeda0 0x7fcd7c153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 --2- 172.21.15.87:0/1368853704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd7c0721f0 0x7fcd7c151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 >> 172.21.15.87:0/1368853704 conn(0x7fcd7c0669a0 msgr2=0x7fcd7c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:28.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 shutdown_connections 2024-04-25T10:25:28.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:28.935+0000 7fcd83800640 1 -- 172.21.15.87:0/1368853704 wait complete. 2024-04-25T10:25:28.954 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:29.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156e00640 1 Processor -- start 2024-04-25T10:25:29.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156e00640 1 -- start start 2024-04-25T10:25:29.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41500ce8d0 0x7f41500b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f41500ced30 0x7f41500b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41500cd120 con 0x7f415015b9b0 2024-04-25T10:25:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41500d2600 con 0x7f41500ced30 2024-04-25T10:25:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41500cd400 con 0x7f41500ce8d0 2024-04-25T10:25:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f41500ced30 0x7f41500b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4155000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41500ce8d0 0x7f41500b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4155a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f415015b9b0 0x7f41500ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:44642/0 (socket says 172.21.15.87:44642) 2024-04-25T10:25:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4155a00640 1 -- 172.21.15.87:0/2285874748 learned_addr learned my addr 172.21.15.87:0/2285874748 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:29.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156400640 1 -- 172.21.15.87:0/2285874748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41500ce8d0 msgr2=0x7f41500b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:29.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156400640 1 --2- 172.21.15.87:0/2285874748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41500ce8d0 0x7f41500b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156400640 1 -- 172.21.15.87:0/2285874748 >> v1:172.21.15.169:6789/0 conn(0x7f415015b9b0 legacy=0x7f41500ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.091+0000 7f4156400640 1 -- 172.21.15.87:0/2285874748 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500b47a0 con 0x7f41500ced30 2024-04-25T10:25:29.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156400640 1 --2- 172.21.15.87:0/2285874748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500b54e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4148009d90 tx=0x7f414801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84858d76a603f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f414fe00640 1 -- 172.21.15.87:0/2285874748 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41480b4070 con 0x7f41500ced30 2024-04-25T10:25:29.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f414fe00640 1 -- 172.21.15.87:0/2285874748 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41480af600 con 0x7f41500ced30 2024-04-25T10:25:29.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f414fe00640 1 -- 172.21.15.87:0/2285874748 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41480af9d0 con 0x7f41500ced30 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 -- 172.21.15.87:0/2285874748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 msgr2=0x7f41500b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 --2- 172.21.15.87:0/2285874748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500b54e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4148009d90 tx=0x7f414801f150 comp rx=0 tx=0).stop 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 -- 172.21.15.87:0/2285874748 shutdown_connections 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 --2- 172.21.15.87:0/2285874748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500b54e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 --2- 172.21.15.87:0/2285874748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41500ce8d0 0x7f41500b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 -- 172.21.15.87:0/2285874748 >> 172.21.15.87:0/2285874748 conn(0x7f41500d17b0 msgr2=0x7f41500d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 -- 172.21.15.87:0/2285874748 shutdown_connections 2024-04-25T10:25:29.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.095+0000 7f4156e00640 1 -- 172.21.15.87:0/2285874748 wait complete. 2024-04-25T10:25:29.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 Processor -- start 2024-04-25T10:25:29.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 -- start start 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41500ce8d0 0x7f41500ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f415015b9b0 0x7f415014aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415015bf10 con 0x7f41500ce8d0 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415015bd90 con 0x7f41500ced30 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41500d0390 con 0x7f415015b9b0 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41500ce8d0 0x7f41500ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41500ce8d0 0x7f41500ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60298/0 (socket says 172.21.15.87:60298) 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 -- 172.21.15.87:0/4185058536 learned_addr learned my addr 172.21.15.87:0/4185058536 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:29.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156400640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f415015b9b0 0x7f415014aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155000640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 -- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f415015b9b0 msgr2=0x7f415014aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f415015b9b0 0x7f415014aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 -- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 msgr2=0x7f41500c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 -- 172.21.15.87:0/4185058536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500c6d90 con 0x7f41500ce8d0 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155000640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4155a00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41500ce8d0 0x7f41500ae760 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f414400c420 tx=0x7f414400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f414d600640 1 -- 172.21.15.87:0/4185058536 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144017070 con 0x7f41500ce8d0 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f414d600640 1 -- 172.21.15.87:0/4185058536 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144005cf0 con 0x7f41500ce8d0 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41500c7070 con 0x7f41500ce8d0 2024-04-25T10:25:29.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f414d600640 1 -- 172.21.15.87:0/4185058536 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440072f0 con 0x7f41500ce8d0 2024-04-25T10:25:29.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41500c7910 con 0x7f41500ce8d0 2024-04-25T10:25:29.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.099+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41500c7240 con 0x7f41500ce8d0 2024-04-25T10:25:29.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.103+0000 7f414d600640 1 -- 172.21.15.87:0/4185058536 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f414401a070 con 0x7f41500ce8d0 2024-04-25T10:25:29.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.103+0000 7f414d600640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4130066f60 0x7f4130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.103+0000 7f414d600640 1 -- 172.21.15.87:0/4185058536 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f41440843a0 con 0x7f41500ce8d0 2024-04-25T10:25:29.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.103+0000 7f4155000640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4130066f60 0x7f4130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.103+0000 7f4155000640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4130066f60 0x7f4130069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f41500b02c0 tx=0x7f413c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.107+0000 7f414d600640 1 -- 172.21.15.87:0/4185058536 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4144054d10 con 0x7f41500ce8d0 2024-04-25T10:25:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.295+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f415006d380 con 0x7f41500ce8d0 2024-04-25T10:25:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f414d600640 1 -- 172.21.15.87:0/4185058536 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f4144058db0 con 0x7f41500ce8d0 2024-04-25T10:25:29.298 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:29.298 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:25:29.298 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:25:29.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4130066f60 msgr2=0x7f4130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4130066f60 0x7f4130069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f41500b02c0 tx=0x7f413c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:29.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41500ce8d0 msgr2=0x7f41500ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41500ce8d0 0x7f41500ae760 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f414400c420 tx=0x7f414400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 shutdown_connections 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4130066f60 0x7f4130069420 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f415015b9b0 0x7f415014aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41500ced30 0x7f41500c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 --2- 172.21.15.87:0/4185058536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41500ce8d0 0x7f41500ae760 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 >> 172.21.15.87:0/4185058536 conn(0x7f41500d17b0 msgr2=0x7f41500b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 shutdown_connections 2024-04-25T10:25:29.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.299+0000 7f4156e00640 1 -- 172.21.15.87:0/4185058536 wait complete. 2024-04-25T10:25:29.318 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:29.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedaf600640 1 Processor -- start 2024-04-25T10:25:29.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedaf600640 1 -- start start 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedaf600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda8161150 0x7feda80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedaf600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feda80cf930 0x7feda80cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedaf600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda8070530 con 0x7feda8070cd0 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedaf600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda80b29b0 con 0x7feda80cf930 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedaf600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda80b2f50 con 0x7feda8161150 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7feda7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7feda8070cd0 0x7feda806c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41350/0 (socket says 172.21.15.87:41350) 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7feda7e00640 1 -- 172.21.15.87:0/1967636807 learned_addr learned my addr 172.21.15.87:0/1967636807 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedacc00640 1 --2- 172.21.15.87:0/1967636807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda8161150 0x7feda80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedad600640 1 --2- 172.21.15.87:0/1967636807 >> v2:172.21.15.169:3300/0 conn(0x7feda80cf930 0x7feda80cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7feda7400640 1 -- 172.21.15.87:0/1967636807 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802477812 0 0) 0x7fed94001120 con 0x7feda8070cd0 2024-04-25T10:25:29.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7feda7400640 1 -- 172.21.15.87:0/1967636807 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed90003530 con 0x7feda8070cd0 2024-04-25T10:25:29.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedacc00640 1 -- 172.21.15.87:0/1967636807 >> v2:172.21.15.169:3300/0 conn(0x7feda80cf930 msgr2=0x7feda80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:29.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedacc00640 1 --2- 172.21.15.87:0/1967636807 >> v2:172.21.15.169:3300/0 conn(0x7feda80cf930 0x7feda80cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedacc00640 1 -- 172.21.15.87:0/1967636807 >> v1:172.21.15.87:6789/0 conn(0x7feda8070cd0 legacy=0x7feda806c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedacc00640 1 -- 172.21.15.87:0/1967636807 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda80ae2f0 con 0x7feda8161150 2024-04-25T10:25:29.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.455+0000 7fedad600640 1 --2- 172.21.15.87:0/1967636807 >> v2:172.21.15.169:3300/0 conn(0x7feda80cf930 0x7feda80cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:29.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedacc00640 1 --2- 172.21.15.87:0/1967636807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda8161150 0x7feda80b43b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0097c0 tx=0x7fed9c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef20a61ed475a962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7feda7400640 1 -- 172.21.15.87:0/1967636807 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0b2070 con 0x7feda8161150 2024-04-25T10:25:29.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7feda7400640 1 -- 172.21.15.87:0/1967636807 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c0ad640 con 0x7feda8161150 2024-04-25T10:25:29.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7feda7400640 1 -- 172.21.15.87:0/1967636807 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0ada10 con 0x7feda8161150 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 -- 172.21.15.87:0/1967636807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda8161150 msgr2=0x7feda80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 --2- 172.21.15.87:0/1967636807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda8161150 0x7feda80b43b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0097c0 tx=0x7fed9c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 -- 172.21.15.87:0/1967636807 shutdown_connections 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 --2- 172.21.15.87:0/1967636807 >> v2:172.21.15.169:3300/0 conn(0x7feda80cf930 0x7feda80cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 --2- 172.21.15.87:0/1967636807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda8161150 0x7feda80b43b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 -- 172.21.15.87:0/1967636807 >> 172.21.15.87:0/1967636807 conn(0x7feda80669a0 msgr2=0x7feda8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 -- 172.21.15.87:0/1967636807 shutdown_connections 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.459+0000 7fedaf600640 1 -- 172.21.15.87:0/1967636807 wait complete. 2024-04-25T10:25:29.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 Processor -- start 2024-04-25T10:25:29.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 -- start start 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8070cd0 0x7feda81f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda80cf930 0x7feda81fae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feda8161150 0x7feda814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80b3d70 con 0x7feda8070cd0 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80b3bf0 con 0x7feda8161150 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedaf600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda80b3ef0 con 0x7feda80cf930 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8070cd0 0x7feda81f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8070cd0 0x7feda81f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60306/0 (socket says 172.21.15.87:60306) 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 -- 172.21.15.87:0/4257191950 learned_addr learned my addr 172.21.15.87:0/4257191950 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedad600640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feda8161150 0x7feda814e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7feda7e00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda80cf930 0x7feda81fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 -- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda80cf930 msgr2=0x7feda81fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda80cf930 0x7feda81fae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 -- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feda8161150 msgr2=0x7feda814e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feda8161150 0x7feda814e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 -- 172.21.15.87:0/4257191950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8150ed0 con 0x7feda8070cd0 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7feda7e00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda80cf930 0x7feda81fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7fedacc00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8070cd0 0x7feda81f6e20 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fed9c01a0d0 tx=0x7fed9c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7feda4c00640 1 -- 172.21.15.87:0/4257191950 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0ba070 con 0x7feda8070cd0 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7feda4c00640 1 -- 172.21.15.87:0/4257191950 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c0b6410 con 0x7feda8070cd0 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.463+0000 7feda4c00640 1 -- 172.21.15.87:0/4257191950 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0b2040 con 0x7feda8070cd0 2024-04-25T10:25:29.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda81fb360 con 0x7feda8070cd0 2024-04-25T10:25:29.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda81f7750 con 0x7feda8070cd0 2024-04-25T10:25:29.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda804ef30 con 0x7feda8070cd0 2024-04-25T10:25:29.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7feda4c00640 1 -- 172.21.15.87:0/4257191950 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed9c0bd070 con 0x7feda8070cd0 2024-04-25T10:25:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7feda4c00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fed88066f60 0x7fed88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7feda4c00640 1 -- 172.21.15.87:0/4257191950 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fed9c11eef0 con 0x7feda8070cd0 2024-04-25T10:25:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7feda7e00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fed88066f60 0x7fed88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.467+0000 7feda7e00640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fed88066f60 0x7fed88069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fed940012c0 tx=0x7fed94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.471+0000 7feda4c00640 1 -- 172.21.15.87:0/4257191950 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feda81fb360 con 0x7feda8070cd0 2024-04-25T10:25:29.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.627+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feda814e9e0 con 0x7feda8070cd0 2024-04-25T10:25:29.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.627+0000 7feda4c00640 1 -- 172.21.15.87:0/4257191950 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16765 (secure 0 0 0) 0x7fed9c016090 con 0x7feda8070cd0 2024-04-25T10:25:29.628 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:29.628 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":165,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:07.649563+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:42.650296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:43.940319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.29","mappings":[{"from":0,"to":2}]},{"pgid":"25.39","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fed88066f60 msgr2=0x7fed88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fed88066f60 0x7fed88069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fed940012c0 tx=0x7fed94009040 comp rx=0 tx=0).stop 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8070cd0 msgr2=0x7feda81f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8070cd0 0x7feda81f6e20 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fed9c01a0d0 tx=0x7fed9c019240 comp rx=0 tx=0).stop 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 shutdown_connections 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fed88066f60 0x7fed88069420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feda8161150 0x7feda814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feda80cf930 0x7feda81fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 --2- 172.21.15.87:0/4257191950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8070cd0 0x7feda81f6e20 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 >> 172.21.15.87:0/4257191950 conn(0x7feda80669a0 msgr2=0x7feda80afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 shutdown_connections 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.631+0000 7fedaf600640 1 -- 172.21.15.87:0/4257191950 wait complete. 2024-04-25T10:25:29.649 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:25:29.649 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:29.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3e00640 1 Processor -- start 2024-04-25T10:25:29.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3e00640 1 -- start start 2024-04-25T10:25:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8af415dff0 0x7f8af406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af40cbff0 con 0x7f8af415dc10 2024-04-25T10:25:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af40b45a0 con 0x7f8af415dff0 2024-04-25T10:25:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40cbda0 con 0x7f8af406bbe0 2024-04-25T10:25:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8af415dc10 0x7f8af40d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41360/0 (socket says 172.21.15.87:41360) 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af2a00640 1 -- 172.21.15.87:0/2349009097 learned_addr learned my addr 172.21.15.87:0/2349009097 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3400640 1 --2- 172.21.15.87:0/2349009097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8aebe00640 1 --2- 172.21.15.87:0/2349009097 >> v2:172.21.15.169:3300/0 conn(0x7f8af415dff0 0x7f8af406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af2000640 1 -- 172.21.15.87:0/2349009097 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770576687 0 0) 0x7f8af40cbff0 con 0x7f8af415dc10 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af2000640 1 -- 172.21.15.87:0/2349009097 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad8003530 con 0x7f8af415dc10 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af2000640 1 -- 172.21.15.87:0/2349009097 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1446773855 0 0) 0x7f8ad8003530 con 0x7f8af415dc10 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3400640 1 -- 172.21.15.87:0/2349009097 >> v2:172.21.15.169:3300/0 conn(0x7f8af415dff0 msgr2=0x7f8af406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3400640 1 --2- 172.21.15.87:0/2349009097 >> v2:172.21.15.169:3300/0 conn(0x7f8af415dff0 0x7f8af406b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3400640 1 -- 172.21.15.87:0/2349009097 >> v1:172.21.15.87:6789/0 conn(0x7f8af415dc10 legacy=0x7f8af40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8af3400640 1 -- 172.21.15.87:0/2349009097 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40cc2d0 con 0x7f8af406bbe0 2024-04-25T10:25:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.787+0000 7f8aebe00640 1 --2- 172.21.15.87:0/2349009097 >> v2:172.21.15.169:3300/0 conn(0x7f8af415dff0 0x7f8af406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:29.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3400640 1 --2- 172.21.15.87:0/2349009097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af40ccdb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009d90 tx=0x7f8aec01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90e49f84dff5cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af2000640 1 -- 172.21.15.87:0/2349009097 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0b2070 con 0x7f8af406bbe0 2024-04-25T10:25:29.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af2000640 1 -- 172.21.15.87:0/2349009097 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec0a5580 con 0x7f8af406bbe0 2024-04-25T10:25:29.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af2000640 1 -- 172.21.15.87:0/2349009097 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4001680 con 0x7f8af406bbe0 2024-04-25T10:25:29.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 -- 172.21.15.87:0/2349009097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 msgr2=0x7f8af40ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2349009097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af40ccdb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009d90 tx=0x7f8aec01c890 comp rx=0 tx=0).stop 2024-04-25T10:25:29.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 -- 172.21.15.87:0/2349009097 shutdown_connections 2024-04-25T10:25:29.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2349009097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af40ccdb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2349009097 >> v2:172.21.15.169:3300/0 conn(0x7f8af415dff0 0x7f8af406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 -- 172.21.15.87:0/2349009097 >> 172.21.15.87:0/2349009097 conn(0x7f8af40d1750 msgr2=0x7f8af40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:29.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 -- 172.21.15.87:0/2349009097 shutdown_connections 2024-04-25T10:25:29.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.791+0000 7f8af3e00640 1 -- 172.21.15.87:0/2349009097 wait complete. 2024-04-25T10:25:29.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 Processor -- start 2024-04-25T10:25:29.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 -- start start 2024-04-25T10:25:29.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af415dc10 0x7f8af41527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af415dff0 0x7f8af4152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af406b1e0 con 0x7f8af415dff0 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af406b060 con 0x7f8af415dc10 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40cdb00 con 0x7f8af406bbe0 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af415dff0 0x7f8af4152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af415dff0 0x7f8af4152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60312/0 (socket says 172.21.15.87:60312) 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 -- 172.21.15.87:0/2145925753 learned_addr learned my addr 172.21.15.87:0/2145925753 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8aebe00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af415dc10 0x7f8af41527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af2a00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 -- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 msgr2=0x7f8af414f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af414f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 -- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af415dc10 msgr2=0x7f8af41527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af415dc10 0x7f8af41527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 -- 172.21.15.87:0/2145925753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af414f930 con 0x7f8af415dff0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3400640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af415dff0 0x7f8af4152d20 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f8aec01d130 tx=0x7f8aec019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8aeb400640 1 -- 172.21.15.87:0/2145925753 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0bd020 con 0x7f8af415dff0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8aeb400640 1 -- 172.21.15.87:0/2145925753 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec0b8470 con 0x7f8af415dff0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af415bd80 con 0x7f8af415dff0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8aeb400640 1 -- 172.21.15.87:0/2145925753 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0b2040 con 0x7f8af415dff0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af415c340 con 0x7f8af415dff0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.795+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af4075120 con 0x7f8af415dff0 2024-04-25T10:25:29.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.799+0000 7f8aeb400640 1 -- 172.21.15.87:0/2145925753 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8aec0c0020 con 0x7f8af415dff0 2024-04-25T10:25:29.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.799+0000 7f8aeb400640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ad0066f60 0x7f8ad0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:29.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.799+0000 7f8aeb400640 1 -- 172.21.15.87:0/2145925753 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8aec11fd40 con 0x7f8af415dff0 2024-04-25T10:25:29.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.799+0000 7f8af2a00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ad0066f60 0x7f8ad0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:29.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.799+0000 7f8af2a00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ad0066f60 0x7f8ad0069420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40017f0 tx=0x7f8ae4001620 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:29.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.803+0000 7f8aeb400640 1 -- 172.21.15.87:0/2145925753 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8aec0c7030 con 0x7f8af415dff0 2024-04-25T10:25:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8af406b9c0 con 0x7f8af415dff0 2024-04-25T10:25:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8aeb400640 1 -- 172.21.15.87:0/2145925753 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f8aec016480 con 0x7f8af415dff0 2024-04-25T10:25:29.987 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:29.987 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:25:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ad0066f60 msgr2=0x7f8ad0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ad0066f60 0x7f8ad0069420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40017f0 tx=0x7f8ae4001620 comp rx=0 tx=0).stop 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af415dff0 msgr2=0x7f8af4152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af415dff0 0x7f8af4152d20 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f8aec01d130 tx=0x7f8aec019630 comp rx=0 tx=0).stop 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 shutdown_connections 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ad0066f60 0x7f8ad0069420 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af415dff0 0x7f8af4152d20 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af415dc10 0x7f8af41527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 --2- 172.21.15.87:0/2145925753 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af406bbe0 0x7f8af414f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 >> 172.21.15.87:0/2145925753 conn(0x7f8af40d1750 msgr2=0x7f8af406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 shutdown_connections 2024-04-25T10:25:29.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:29.987+0000 7f8af3e00640 1 -- 172.21.15.87:0/2145925753 wait complete. 2024-04-25T10:25:30.008 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:30.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f45a6400640 1 Processor -- start 2024-04-25T10:25:30.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f45a6400640 1 -- start start 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f45a6400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f45a00cda10 0x7f45a006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f45a6400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f45a00cf930 0x7f45a00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f45a6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a00b2770 con 0x7f45a015bc40 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f45a6400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a006c670 con 0x7f45a00cda10 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f45a6400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a006c950 con 0x7f45a00cf930 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f45a00cda10 0x7f45a006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f45a015bc40 0x7f45a00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41364/0 (socket says 172.21.15.87:41364) 2024-04-25T10:25:30.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459f400640 1 -- 172.21.15.87:0/4035195759 learned_addr learned my addr 172.21.15.87:0/4035195759 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:30.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459fe00640 1 --2- 172.21.15.87:0/4035195759 >> v2:172.21.15.169:3300/0 conn(0x7f45a00cf930 0x7f45a00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459ea00640 1 -- 172.21.15.87:0/4035195759 >> v2:172.21.15.169:3300/0 conn(0x7f45a00cf930 msgr2=0x7f45a00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:30.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459ea00640 1 --2- 172.21.15.87:0/4035195759 >> v2:172.21.15.169:3300/0 conn(0x7f45a00cf930 0x7f45a00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459ea00640 1 -- 172.21.15.87:0/4035195759 >> v1:172.21.15.87:6789/0 conn(0x7f45a015bc40 legacy=0x7f45a00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459ea00640 1 -- 172.21.15.87:0/4035195759 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a006cc30 con 0x7f45a00cda10 2024-04-25T10:25:30.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459e000640 1 -- 172.21.15.87:0/4035195759 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42577920 0 0) 0x7f45a00b2770 con 0x7f45a015bc40 2024-04-25T10:25:30.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.143+0000 7f459fe00640 1 --2- 172.21.15.87:0/4035195759 >> v2:172.21.15.169:3300/0 conn(0x7f45a00cf930 0x7f45a00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:30.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f459ea00640 1 --2- 172.21.15.87:0/4035195759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a006c090 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f45a00ae4c0 tx=0x7f458c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519cba204a8dc935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f459e000640 1 -- 172.21.15.87:0/4035195759 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0b0070 con 0x7f45a00cda10 2024-04-25T10:25:30.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f459e000640 1 -- 172.21.15.87:0/4035195759 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c0ab600 con 0x7f45a00cda10 2024-04-25T10:25:30.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f459e000640 1 -- 172.21.15.87:0/4035195759 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0ab9d0 con 0x7f45a00cda10 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 -- 172.21.15.87:0/4035195759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 msgr2=0x7f45a006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 --2- 172.21.15.87:0/4035195759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a006c090 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f45a00ae4c0 tx=0x7f458c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 -- 172.21.15.87:0/4035195759 shutdown_connections 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 --2- 172.21.15.87:0/4035195759 >> v2:172.21.15.169:3300/0 conn(0x7f45a00cf930 0x7f45a00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 --2- 172.21.15.87:0/4035195759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a006c090 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 -- 172.21.15.87:0/4035195759 >> 172.21.15.87:0/4035195759 conn(0x7f45a0066b70 msgr2=0x7f45a0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 -- 172.21.15.87:0/4035195759 shutdown_connections 2024-04-25T10:25:30.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.147+0000 7f45a6400640 1 -- 172.21.15.87:0/4035195759 wait complete. 2024-04-25T10:25:30.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 Processor -- start 2024-04-25T10:25:30.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 -- start start 2024-04-25T10:25:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a014a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45a00cf930 0x7f45a015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a015bc40 0x7f45a015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b3850 con 0x7f45a00cda10 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b36d0 con 0x7f45a015bc40 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b39d0 con 0x7f45a00cf930 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a014a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a014a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60324/0 (socket says 172.21.15.87:60324) 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 -- 172.21.15.87:0/713846520 learned_addr learned my addr 172.21.15.87:0/713846520 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459ea00640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45a00cf930 0x7f45a015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459fe00640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a015bc40 0x7f45a015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 -- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45a00cf930 msgr2=0x7f45a015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45a00cf930 0x7f45a015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 -- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a015bc40 msgr2=0x7f45a015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a015bc40 0x7f45a015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 -- 172.21.15.87:0/713846520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a015da60 con 0x7f45a00cda10 2024-04-25T10:25:30.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f459f400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a014a130 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f459000c7c0 tx=0x7f459000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f4597400640 1 -- 172.21.15.87:0/713846520 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590018070 con 0x7f45a00cda10 2024-04-25T10:25:30.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f4597400640 1 -- 172.21.15.87:0/713846520 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459000a640 con 0x7f45a00cda10 2024-04-25T10:25:30.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f4597400640 1 -- 172.21.15.87:0/713846520 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459000aa30 con 0x7f45a00cda10 2024-04-25T10:25:30.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a015dd40 con 0x7f45a00cda10 2024-04-25T10:25:30.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a015e300 con 0x7f45a00cda10 2024-04-25T10:25:30.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.151+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a004ef30 con 0x7f45a00cda10 2024-04-25T10:25:30.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.155+0000 7f4597400640 1 -- 172.21.15.87:0/713846520 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f459000abd0 con 0x7f45a00cda10 2024-04-25T10:25:30.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.155+0000 7f4597400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f60 0x7f4580069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.155+0000 7f4597400640 1 -- 172.21.15.87:0/713846520 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f45900847c0 con 0x7f45a00cda10 2024-04-25T10:25:30.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.155+0000 7f459ea00640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f60 0x7f4580069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.155+0000 7f459ea00640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f60 0x7f4580069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f458c0ad000 tx=0x7f458c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.159+0000 7f4597400640 1 -- 172.21.15.87:0/713846520 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45900556f0 con 0x7f45a00cda10 2024-04-25T10:25:30.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.343+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45a006d960 con 0x7f45a00cda10 2024-04-25T10:25:30.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.343+0000 7f4597400640 1 -- 172.21.15.87:0/713846520 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f4590054ff0 con 0x7f45a00cda10 2024-04-25T10:25:30.343 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:30.344 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":119,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:44.936393+0000","modified":"2024-04-25T10:24:47.962825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10494":{"gid":10494,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/1488445455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1488445455},{"type":"v1","addr":"172.21.15.87:6837","nonce":1488445455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:25:30.344 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 119 2024-04-25T10:25:30.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f60 msgr2=0x7f4580069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f60 0x7f4580069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f458c0ad000 tx=0x7f458c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:30.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 msgr2=0x7f45a014a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a014a130 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f459000c7c0 tx=0x7f459000cc80 comp rx=0 tx=0).stop 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 shutdown_connections 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4580066f60 0x7f4580069420 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45a015bc40 0x7f45a015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45a00cf930 0x7f45a015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 --2- 172.21.15.87:0/713846520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a00cda10 0x7f45a014a130 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 >> 172.21.15.87:0/713846520 conn(0x7f45a0066b70 msgr2=0x7f45a00b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 shutdown_connections 2024-04-25T10:25:30.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.347+0000 7f45a6400640 1 -- 172.21.15.87:0/713846520 wait complete. 2024-04-25T10:25:30.365 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:30.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.499+0000 7f7c6d600640 1 Processor -- start 2024-04-25T10:25:30.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.499+0000 7f7c6d600640 1 -- start start 2024-04-25T10:25:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071260 0x7f7c680716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6d600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c68071be0 0x7f7c680b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6806de60 con 0x7f7c6815a780 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6d600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6806cdc0 con 0x7f7c68071be0 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6806d0a0 con 0x7f7c68071260 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c68071be0 0x7f7c680b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c68071be0 0x7f7c680b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60336/0 (socket says 172.21.15.87:60336) 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6cc00640 1 -- 172.21.15.87:0/350960921 learned_addr learned my addr 172.21.15.87:0/350960921 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c67400640 1 --2- 172.21.15.87:0/350960921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071260 0x7f7c680716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6cc00640 1 -- 172.21.15.87:0/350960921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071260 msgr2=0x7f7c680716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6cc00640 1 --2- 172.21.15.87:0/350960921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071260 0x7f7c680716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6cc00640 1 -- 172.21.15.87:0/350960921 >> v1:172.21.15.169:6789/0 conn(0x7f7c6815a780 legacy=0x7f7c6815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.503+0000 7f7c6cc00640 1 -- 172.21.15.87:0/350960921 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6806d380 con 0x7f7c68071be0 2024-04-25T10:25:30.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6cc00640 1 --2- 172.21.15.87:0/350960921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071be0 0x7f7c680b2460 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009870 tx=0x7f7c6001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bf2a0c31c9a5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c66a00640 1 -- 172.21.15.87:0/350960921 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600b2070 con 0x7f7c68071be0 2024-04-25T10:25:30.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c66a00640 1 -- 172.21.15.87:0/350960921 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c600a9070 con 0x7f7c68071be0 2024-04-25T10:25:30.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c66a00640 1 -- 172.21.15.87:0/350960921 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600ad880 con 0x7f7c68071be0 2024-04-25T10:25:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 -- 172.21.15.87:0/350960921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071be0 msgr2=0x7f7c680b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 --2- 172.21.15.87:0/350960921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071be0 0x7f7c680b2460 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009870 tx=0x7f7c6001c560 comp rx=0 tx=0).stop 2024-04-25T10:25:30.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 -- 172.21.15.87:0/350960921 shutdown_connections 2024-04-25T10:25:30.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 --2- 172.21.15.87:0/350960921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071be0 0x7f7c680b2460 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 --2- 172.21.15.87:0/350960921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071260 0x7f7c680716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 -- 172.21.15.87:0/350960921 >> 172.21.15.87:0/350960921 conn(0x7f7c680743f0 msgr2=0x7f7c680669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:30.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 -- 172.21.15.87:0/350960921 shutdown_connections 2024-04-25T10:25:30.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 -- 172.21.15.87:0/350960921 wait complete. 2024-04-25T10:25:30.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 Processor -- start 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.507+0000 7f7c6d600640 1 -- start start 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071260 0x7f7c68168350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071be0 0x7f7c6814acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c6815a780 0x7f7c68145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b3330 con 0x7f7c68071260 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b31b0 con 0x7f7c6815a780 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b34b0 con 0x7f7c68071be0 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071260 0x7f7c68168350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071260 0x7f7c68168350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60352/0 (socket says 172.21.15.87:60352) 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 -- 172.21.15.87:0/3271164426 learned_addr learned my addr 172.21.15.87:0/3271164426 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67400640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071be0 0x7f7c6814acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6cc00640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c6815a780 0x7f7c68145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 -- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071be0 msgr2=0x7f7c6814acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071be0 0x7f7c6814acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 -- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c6815a780 msgr2=0x7f7c68145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c6815a780 0x7f7c68145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 -- 172.21.15.87:0/3271164426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6814b3f0 con 0x7f7c68071260 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6cc00640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c6815a780 0x7f7c68145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c67e00640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071260 0x7f7c68168350 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c790 tx=0x7f7c5800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c5fe00640 1 -- 172.21.15.87:0/3271164426 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58016020 con 0x7f7c68071260 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c5fe00640 1 -- 172.21.15.87:0/3271164426 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c58006d60 con 0x7f7c68071260 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c5fe00640 1 -- 172.21.15.87:0/3271164426 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58003bc0 con 0x7f7c68071260 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6815aea0 con 0x7f7c68071260 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6815b460 con 0x7f7c68071260 2024-04-25T10:25:30.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c68075120 con 0x7f7c68071260 2024-04-25T10:25:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.511+0000 7f7c5fe00640 1 -- 172.21.15.87:0/3271164426 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c5800a850 con 0x7f7c68071260 2024-04-25T10:25:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.515+0000 7f7c5fe00640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c48066f60 0x7f7c48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.515+0000 7f7c5fe00640 1 -- 172.21.15.87:0/3271164426 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7c58084ce0 con 0x7f7c68071260 2024-04-25T10:25:30.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.515+0000 7f7c67400640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c48066f60 0x7f7c48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.515+0000 7f7c67400640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c48066f60 0x7f7c48069420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7c68159bc0 tx=0x7f7c54008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.519+0000 7f7c5fe00640 1 -- 172.21.15.87:0/3271164426 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c58020020 con 0x7f7c68071260 2024-04-25T10:25:30.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.675+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c681510a0 con 0x7f7c68071260 2024-04-25T10:25:30.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.675+0000 7f7c5fe00640 1 -- 172.21.15.87:0/3271164426 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16765 (secure 0 0 0) 0x7f7c58055d40 con 0x7f7c68071260 2024-04-25T10:25:30.676 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:30.676 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":165,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:07.649563+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:42.650296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:43.940319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.29","mappings":[{"from":0,"to":2}]},{"pgid":"25.39","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c48066f60 msgr2=0x7f7c48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c48066f60 0x7f7c48069420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7c68159bc0 tx=0x7f7c54008040 comp rx=0 tx=0).stop 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071260 msgr2=0x7f7c68168350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071260 0x7f7c68168350 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c790 tx=0x7f7c5800cc50 comp rx=0 tx=0).stop 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 shutdown_connections 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c48066f60 0x7f7c48069420 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c6815a780 0x7f7c68145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c68071be0 0x7f7c6814acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 --2- 172.21.15.87:0/3271164426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c68071260 0x7f7c68168350 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 >> 172.21.15.87:0/3271164426 conn(0x7f7c680743f0 msgr2=0x7f7c680cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:30.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 shutdown_connections 2024-04-25T10:25:30.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.679+0000 7f7c6d600640 1 -- 172.21.15.87:0/3271164426 wait complete. 2024-04-25T10:25:30.697 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:25:30.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9c00640 1 Processor -- start 2024-04-25T10:25:30.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9c00640 1 -- start start 2024-04-25T10:25:30.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de415b750 0x7f6de40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6de40cd590 0x7f6de40cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de40b2460 con 0x7f6de40b3f80 2024-04-25T10:25:30.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de40b2d70 con 0x7f6de40cd590 2024-04-25T10:25:30.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4070fb0 con 0x7f6de415b750 2024-04-25T10:25:30.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6de40cd590 0x7f6de40cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de415b750 0x7f6de40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6de40b3f80 0x7f6de40b3200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:44662/0 (socket says 172.21.15.87:44662) 2024-04-25T10:25:30.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de9200640 1 -- 172.21.15.87:0/1544647510 learned_addr learned my addr 172.21.15.87:0/1544647510 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:30.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.835+0000 7f6de2a00640 1 -- 172.21.15.87:0/1544647510 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425270878 0 0) 0x7f6de40b2460 con 0x7f6de40b3f80 2024-04-25T10:25:30.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de2a00640 1 -- 172.21.15.87:0/1544647510 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dc8003530 con 0x7f6de40b3f80 2024-04-25T10:25:30.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de3400640 1 -- 172.21.15.87:0/1544647510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de415b750 msgr2=0x7f6de40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:30.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de3400640 1 --2- 172.21.15.87:0/1544647510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de415b750 0x7f6de40cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de3400640 1 -- 172.21.15.87:0/1544647510 >> v1:172.21.15.169:6789/0 conn(0x7f6de40b3f80 legacy=0x7f6de40b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de3400640 1 -- 172.21.15.87:0/1544647510 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40b2720 con 0x7f6de40cd590 2024-04-25T10:25:30.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de3400640 1 --2- 172.21.15.87:0/1544647510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40cd590 0x7f6de40cd9b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6de40b2ef0 tx=0x7f6dd001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc4df12d90f5b3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de2a00640 1 -- 172.21.15.87:0/1544647510 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00ab430 con 0x7f6de40cd590 2024-04-25T10:25:30.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de2a00640 1 -- 172.21.15.87:0/1544647510 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd00b0070 con 0x7f6de40cd590 2024-04-25T10:25:30.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de2a00640 1 -- 172.21.15.87:0/1544647510 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00ab7f0 con 0x7f6de40cd590 2024-04-25T10:25:30.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de9c00640 1 -- 172.21.15.87:0/1544647510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40cd590 msgr2=0x7f6de40cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.839+0000 7f6de9c00640 1 --2- 172.21.15.87:0/1544647510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40cd590 0x7f6de40cd9b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6de40b2ef0 tx=0x7f6dd001c470 comp rx=0 tx=0).stop 2024-04-25T10:25:30.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- 172.21.15.87:0/1544647510 shutdown_connections 2024-04-25T10:25:30.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 --2- 172.21.15.87:0/1544647510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40cd590 0x7f6de40cd9b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 --2- 172.21.15.87:0/1544647510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de415b750 0x7f6de40cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- 172.21.15.87:0/1544647510 >> 172.21.15.87:0/1544647510 conn(0x7f6de40743f0 msgr2=0x7f6de40669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:30.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- 172.21.15.87:0/1544647510 shutdown_connections 2024-04-25T10:25:30.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- 172.21.15.87:0/1544647510 wait complete. 2024-04-25T10:25:30.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 Processor -- start 2024-04-25T10:25:30.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- start start 2024-04-25T10:25:30.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de40b3f80 0x7f6de414f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de40cd590 0x7f6de415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de415b750 0x7f6de4151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40b47a0 con 0x7f6de40cd590 2024-04-25T10:25:30.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4070b60 con 0x7f6de415b750 2024-04-25T10:25:30.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de415da60 con 0x7f6de40b3f80 2024-04-25T10:25:30.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de40cd590 0x7f6de415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de40cd590 0x7f6de415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60364/0 (socket says 172.21.15.87:60364) 2024-04-25T10:25:30.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 -- 172.21.15.87:0/752937103 learned_addr learned my addr 172.21.15.87:0/752937103 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:30.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 -- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de40b3f80 msgr2=0x7f6de414f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de40b3f80 0x7f6de414f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 -- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de415b750 msgr2=0x7f6de4151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:30.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9200640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de415b750 0x7f6de4151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de415b750 0x7f6de4151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:30.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de3400640 1 -- 172.21.15.87:0/752937103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de415c9c0 con 0x7f6de40cd590 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.843+0000 7f6de9200640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de415b750 0x7f6de4151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6de3400640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de40cd590 0x7f6de415e530 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ad040 tx=0x7f6dd0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6ddbe00640 1 -- 172.21.15.87:0/752937103 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00abd50 con 0x7f6de40cd590 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6ddbe00640 1 -- 172.21.15.87:0/752937103 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd00bc070 con 0x7f6de40cd590 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de415cca0 con 0x7f6de40cd590 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6ddbe00640 1 -- 172.21.15.87:0/752937103 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00b4630 con 0x7f6de40cd590 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de415d260 con 0x7f6de40cd590 2024-04-25T10:25:30.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4075120 con 0x7f6de40cd590 2024-04-25T10:25:30.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6ddbe00640 1 -- 172.21.15.87:0/752937103 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6dd00b47d0 con 0x7f6de40cd590 2024-04-25T10:25:30.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6ddbe00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6dc4066f90 0x7f6dc4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:30.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6ddbe00640 1 -- 172.21.15.87:0/752937103 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6dd011eee0 con 0x7f6de40cd590 2024-04-25T10:25:30.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6de3e00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6dc4066f90 0x7f6dc4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:30.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.847+0000 7f6de3e00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6dc4066f90 0x7f6dc4069450 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006aa0 tx=0x7f6dd4006a30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:30.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:30.851+0000 7f6ddbe00640 1 -- 172.21.15.87:0/752937103 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6dd00c6020 con 0x7f6de40cd590 2024-04-25T10:25:31.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.039+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6de415f130 con 0x7f6de40cd590 2024-04-25T10:25:31.038 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:25:31.039+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:25:31.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.039+0000 7f6ddbe00640 1 -- 172.21.15.87:0/752937103 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v120) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6dd0016090 con 0x7f6de40cd590 2024-04-25T10:25:31.040 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6dc4066f90 msgr2=0x7f6dc4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6dc4066f90 0x7f6dc4069450 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006aa0 tx=0x7f6dd4006a30 comp rx=0 tx=0).stop 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de40cd590 msgr2=0x7f6de415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de40cd590 0x7f6de415e530 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ad040 tx=0x7f6dd0019210 comp rx=0 tx=0).stop 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 shutdown_connections 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6dc4066f90 0x7f6dc4069450 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de415b750 0x7f6de4151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de40cd590 0x7f6de415e530 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 --2- 172.21.15.87:0/752937103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6de40b3f80 0x7f6de414f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 >> 172.21.15.87:0/752937103 conn(0x7f6de40743f0 msgr2=0x7f6de4152e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:31.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 shutdown_connections 2024-04-25T10:25:31.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.043+0000 7f6de9c00640 1 -- 172.21.15.87:0/752937103 wait complete. 2024-04-25T10:25:31.054 INFO:tasks.ceph.mds.c.smithi087.stderr: -6145> 2024-04-25T10:24:50.540+0000 7f7829200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:25:31.058 INFO:tasks.ceph.mds.c.smithi087.stderr: -3268> 2024-04-25T10:24:53.848+0000 7f7824200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:24:52.668148+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={8435=0-12582912@1} caps={8435=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=8435 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x56384b0e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(25)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:25:31.063 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:25:31.106 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:25:31.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.203+0000 7f7699a00640 1 Processor -- start 2024-04-25T10:25:31.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.203+0000 7f7699a00640 1 -- start start 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7699a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7694159540 0x7f76940ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7699a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76940aebd0 0x7f7694071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7699a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7694072020 0x7f769415f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7699a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76940af030 con 0x7f76940aebd0 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7699a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7694159920 con 0x7f7694072020 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7699a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769415c2f0 con 0x7f7694159540 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76940aebd0 0x7f7694071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76940aebd0 0x7f7694071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60378/0 (socket says 172.21.15.87:60378) 2024-04-25T10:25:31.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 -- 172.21.15.87:0/1567104381 learned_addr learned my addr 172.21.15.87:0/1567104381 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:31.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 -- 172.21.15.87:0/1567104381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7694159540 msgr2=0x7f76940ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:31.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 --2- 172.21.15.87:0/1567104381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7694159540 0x7f76940ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 -- 172.21.15.87:0/1567104381 >> v2:172.21.15.169:3300/0 conn(0x7f7694072020 msgr2=0x7f769415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:31.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 --2- 172.21.15.87:0/1567104381 >> v2:172.21.15.169:3300/0 conn(0x7f7694072020 0x7f769415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 -- 172.21.15.87:0/1567104381 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769415c5d0 con 0x7f76940aebd0 2024-04-25T10:25:31.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7692000640 1 --2- 172.21.15.87:0/1567104381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76940aebd0 0x7f7694071ae0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f76940af250 tx=0x7f768001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2e48fc28bd1ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:31.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.207+0000 7f7691600640 1 -- 172.21.15.87:0/1567104381 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800b0070 con 0x7f76940aebd0 2024-04-25T10:25:31.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7691600640 1 -- 172.21.15.87:0/1567104381 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76800ab600 con 0x7f76940aebd0 2024-04-25T10:25:31.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7691600640 1 -- 172.21.15.87:0/1567104381 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800ab9d0 con 0x7f76940aebd0 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- 172.21.15.87:0/1567104381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76940aebd0 msgr2=0x7f7694071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 --2- 172.21.15.87:0/1567104381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76940aebd0 0x7f7694071ae0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f76940af250 tx=0x7f768001c230 comp rx=0 tx=0).stop 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- 172.21.15.87:0/1567104381 shutdown_connections 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 --2- 172.21.15.87:0/1567104381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76940aebd0 0x7f7694071ae0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 --2- 172.21.15.87:0/1567104381 >> v2:172.21.15.169:3300/0 conn(0x7f7694072020 0x7f769415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 --2- 172.21.15.87:0/1567104381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7694159540 0x7f76940ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- 172.21.15.87:0/1567104381 >> 172.21.15.87:0/1567104381 conn(0x7f76940669a0 msgr2=0x7f7694066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- 172.21.15.87:0/1567104381 shutdown_connections 2024-04-25T10:25:31.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- 172.21.15.87:0/1567104381 wait complete. 2024-04-25T10:25:31.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 Processor -- start 2024-04-25T10:25:31.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- start start 2024-04-25T10:25:31.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7694072020 0x7f7694150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76940aebd0 0x7f7694148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7694159540 0x7f7694152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7694072430 con 0x7f7694159540 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769406c0d0 con 0x7f7694072020 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7699a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7694070350 con 0x7f76940aebd0 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7693400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7694159540 0x7f7694152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7693400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7694159540 0x7f7694152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60384/0 (socket says 172.21.15.87:60384) 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7693400640 1 -- 172.21.15.87:0/2496878995 learned_addr learned my addr 172.21.15.87:0/2496878995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7692a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7694072020 0x7f7694150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.211+0000 7f7692000640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76940aebd0 0x7f7694148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7693400640 1 -- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76940aebd0 msgr2=0x7f7694148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:31.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7693400640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76940aebd0 0x7f7694148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7693400640 1 -- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7694072020 msgr2=0x7f7694150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7693400640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7694072020 0x7f7694150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7693400640 1 -- 172.21.15.87:0/2496878995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7694153280 con 0x7f7694159540 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7693400640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7694159540 0x7f7694152b10 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b430 tx=0x7f768c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f768aa00640 1 -- 172.21.15.87:0/2496878995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c015020 con 0x7f7694159540 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f768aa00640 1 -- 172.21.15.87:0/2496878995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c011830 con 0x7f7694159540 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f768aa00640 1 -- 172.21.15.87:0/2496878995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c011c20 con 0x7f7694159540 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7694153560 con 0x7f7694159540 2024-04-25T10:25:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7694153b20 con 0x7f7694159540 2024-04-25T10:25:31.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769404ef30 con 0x7f7694159540 2024-04-25T10:25:31.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f768aa00640 1 -- 172.21.15.87:0/2496878995 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f768c018020 con 0x7f7694159540 2024-04-25T10:25:31.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f768aa00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7674066f60 0x7f7674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:31.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.215+0000 7f768aa00640 1 -- 172.21.15.87:0/2496878995 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f768c084fd0 con 0x7f7694159540 2024-04-25T10:25:31.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.219+0000 7f7692a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7674066f60 0x7f7674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:31.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.219+0000 7f7692a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7674066f60 0x7f7674069420 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f768400a400 tx=0x7f768400d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:31.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.219+0000 7f768aa00640 1 -- 172.21.15.87:0/2496878995 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f768c01f020 con 0x7f7694159540 2024-04-25T10:25:31.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:31.411+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f76940b2d10 con 0x7f7694159540 2024-04-25T10:25:32.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.043+0000 7f768aa00640 1 -- 172.21.15.87:0/2496878995 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v121) v1 ==== 104+0+0 (secure 0 0 0) 0x7f768c055860 con 0x7f7694159540 2024-04-25T10:25:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.043+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7674066f60 msgr2=0x7f7674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.043+0000 7f7699a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7674066f60 0x7f7674069420 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f768400a400 tx=0x7f768400d040 comp rx=0 tx=0).stop 2024-04-25T10:25:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.043+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7694159540 msgr2=0x7f7694152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.043+0000 7f7699a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7694159540 0x7f7694152b10 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b430 tx=0x7f768c00b8f0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 shutdown_connections 2024-04-25T10:25:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7674066f60 0x7f7674069420 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7694159540 0x7f7694152b10 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76940aebd0 0x7f7694148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 --2- 172.21.15.87:0/2496878995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7694072020 0x7f7694150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 >> 172.21.15.87:0/2496878995 conn(0x7f76940669a0 msgr2=0x7f769415e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 shutdown_connections 2024-04-25T10:25:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.047+0000 7f7699a00640 1 -- 172.21.15.87:0/2496878995 wait complete. 2024-04-25T10:25:32.070 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:25:32.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f316be00640 1 Processor -- start 2024-04-25T10:25:32.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f316be00640 1 -- start start 2024-04-25T10:25:32.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f316be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f31641618a0 0x7f316415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:32.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f316be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31640b4f50 0x7f31640b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:32.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f316be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316415e9b0 con 0x7f3164162310 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f316be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316415f250 con 0x7f31640b4f50 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f316be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31640afce0 con 0x7f31641618a0 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f3169e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31640b4f50 0x7f31640b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f3169e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31640b4f50 0x7f31640b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60386/0 (socket says 172.21.15.87:60386) 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f3169e00640 1 -- 172.21.15.87:0/603549146 learned_addr learned my addr 172.21.15.87:0/603549146 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f3169e00640 1 -- 172.21.15.87:0/603549146 >> v2:172.21.15.169:3300/0 conn(0x7f31641618a0 msgr2=0x7f316415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f3169e00640 1 --2- 172.21.15.87:0/603549146 >> v2:172.21.15.169:3300/0 conn(0x7f31641618a0 0x7f316415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f3169e00640 1 -- 172.21.15.87:0/603549146 >> v1:172.21.15.169:6789/0 conn(0x7f3164162310 legacy=0x7f31640b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:32.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.255+0000 7f3169e00640 1 -- 172.21.15.87:0/603549146 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316415ec90 con 0x7f31640b4f50 2024-04-25T10:25:32.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f3169e00640 1 --2- 172.21.15.87:0/603549146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31640b4f50 0x7f31640b0170 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f3164071240 tx=0x7f315c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a13812644bac780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:32.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f3163e00640 1 -- 172.21.15.87:0/603549146 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0a5730 con 0x7f31640b4f50 2024-04-25T10:25:32.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f3163e00640 1 -- 172.21.15.87:0/603549146 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c0a58d0 con 0x7f31640b4f50 2024-04-25T10:25:32.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f3163e00640 1 -- 172.21.15.87:0/603549146 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0a5ca0 con 0x7f31640b4f50 2024-04-25T10:25:32.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 -- 172.21.15.87:0/603549146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31640b4f50 msgr2=0x7f31640b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:32.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 --2- 172.21.15.87:0/603549146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31640b4f50 0x7f31640b0170 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f3164071240 tx=0x7f315c01c890 comp rx=0 tx=0).stop 2024-04-25T10:25:32.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 -- 172.21.15.87:0/603549146 shutdown_connections 2024-04-25T10:25:32.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 --2- 172.21.15.87:0/603549146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31640b4f50 0x7f31640b0170 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 --2- 172.21.15.87:0/603549146 >> v2:172.21.15.169:3300/0 conn(0x7f31641618a0 0x7f316415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 -- 172.21.15.87:0/603549146 >> 172.21.15.87:0/603549146 conn(0x7f31640b5610 msgr2=0x7f31640b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:32.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 -- 172.21.15.87:0/603549146 shutdown_connections 2024-04-25T10:25:32.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.259+0000 7f316be00640 1 -- 172.21.15.87:0/603549146 wait complete. 2024-04-25T10:25:32.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 Processor -- start 2024-04-25T10:25:32.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 -- start start 2024-04-25T10:25:32.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31640b4f50 0x7f3164159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:32.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31641618a0 0x7f3164154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164162310 0x7f3164154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31640b1040 con 0x7f3164162310 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31640b0ec0 con 0x7f31641618a0 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31640b11c0 con 0x7f31640b4f50 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164162310 0x7f3164154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164162310 0x7f3164154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60394/0 (socket says 172.21.15.87:60394) 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 -- 172.21.15.87:0/789874400 learned_addr learned my addr 172.21.15.87:0/789874400 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3168a00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31641618a0 0x7f3164154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169400640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31640b4f50 0x7f3164159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 -- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31640b4f50 msgr2=0x7f3164159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31640b4f50 0x7f3164159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 -- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31641618a0 msgr2=0x7f3164154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31641618a0 0x7f3164154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 -- 172.21.15.87:0/789874400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164159bb0 con 0x7f3164162310 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3169e00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164162310 0x7f3164154e70 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f315c01a4c0 tx=0x7f315c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3161600640 1 -- 172.21.15.87:0/789874400 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0bc020 con 0x7f3164162310 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3161600640 1 -- 172.21.15.87:0/789874400 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c0b8670 con 0x7f3164162310 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3164159e90 con 0x7f3164162310 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f3161600640 1 -- 172.21.15.87:0/789874400 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0b8a60 con 0x7f3164162310 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.263+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316415a450 con 0x7f3164162310 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.267+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316404ef30 con 0x7f3164162310 2024-04-25T10:25:32.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.267+0000 7f3161600640 1 -- 172.21.15.87:0/789874400 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f315c0bf020 con 0x7f3164162310 2024-04-25T10:25:32.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.267+0000 7f3161600640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3144066f60 0x7f3144069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:32.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.267+0000 7f3161600640 1 -- 172.21.15.87:0/789874400 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f315c1201a0 con 0x7f3164162310 2024-04-25T10:25:32.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.267+0000 7f3169400640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3144066f60 0x7f3144069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:32.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.267+0000 7f3169400640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3144066f60 0x7f3144069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f315800b810 tx=0x7f3158008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:32.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.271+0000 7f3161600640 1 -- 172.21.15.87:0/789874400 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f315c0c6030 con 0x7f3164162310 2024-04-25T10:25:32.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:32.427+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f316406dab0 con 0x7f3164162310 2024-04-25T10:25:33.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.047+0000 7f3161600640 1 -- 172.21.15.87:0/789874400 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v167) v1 ==== 181+0+0 (secure 0 0 0) 0x7f315c0165a0 con 0x7f3164162310 2024-04-25T10:25:33.049 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:25:33.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3144066f60 msgr2=0x7f3144069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:33.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3144066f60 0x7f3144069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f315800b810 tx=0x7f3158008040 comp rx=0 tx=0).stop 2024-04-25T10:25:33.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164162310 msgr2=0x7f3164154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:33.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164162310 0x7f3164154e70 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f315c01a4c0 tx=0x7f315c019630 comp rx=0 tx=0).stop 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 shutdown_connections 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3144066f60 0x7f3144069420 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164162310 0x7f3164154e70 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31641618a0 0x7f3164154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 --2- 172.21.15.87:0/789874400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31640b4f50 0x7f3164159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 >> 172.21.15.87:0/789874400 conn(0x7f31640b5610 msgr2=0x7f3164160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 shutdown_connections 2024-04-25T10:25:33.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.051+0000 7f316be00640 1 -- 172.21.15.87:0/789874400 wait complete. 2024-04-25T10:25:33.073 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:25:33.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.211+0000 7f0e0e200640 1 Processor -- start 2024-04-25T10:25:33.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.211+0000 7f0e0e200640 1 -- start start 2024-04-25T10:25:33.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e0e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0e081618a0 0x7f0e0815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e0e200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0815e9b0 con 0x7f0e08162310 2024-04-25T10:25:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e0e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0815ec90 con 0x7f0e080b4f50 2024-04-25T10:25:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e0e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0815f250 con 0x7f0e081618a0 2024-04-25T10:25:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e06a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0e081618a0 0x7f0e0815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e06a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0e081618a0 0x7f0e0815e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60402/0 (socket says 172.21.15.87:60402) 2024-04-25T10:25:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e06a00640 1 -- 172.21.15.87:0/4238405360 learned_addr learned my addr 172.21.15.87:0/4238405360 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:33.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e07400640 1 --1- 172.21.15.87:0/4238405360 >> v1:172.21.15.87:6789/0 conn(0x7f0e08162310 0x7f0e080b4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41372/0 (socket says 172.21.15.87:41372) 2024-04-25T10:25:33.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e06a00640 1 -- 172.21.15.87:0/4238405360 >> v1:172.21.15.169:6789/0 conn(0x7f0e080b4f50 legacy=0x7f0e080b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:33.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e06a00640 1 -- 172.21.15.87:0/4238405360 >> v1:172.21.15.87:6789/0 conn(0x7f0e08162310 legacy=0x7f0e080b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:33.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.215+0000 7f0e06a00640 1 -- 172.21.15.87:0/4238405360 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0815ef70 con 0x7f0e081618a0 2024-04-25T10:25:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e06a00640 1 --2- 172.21.15.87:0/4238405360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e081618a0 0x7f0e0815e2b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f0e0815f3d0 tx=0x7f0df401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55bdc708e5ea4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e06000640 1 -- 172.21.15.87:0/4238405360 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df40b0070 con 0x7f0e081618a0 2024-04-25T10:25:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e06000640 1 -- 172.21.15.87:0/4238405360 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df40ab600 con 0x7f0e081618a0 2024-04-25T10:25:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e06000640 1 -- 172.21.15.87:0/4238405360 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df40ab9d0 con 0x7f0e081618a0 2024-04-25T10:25:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- 172.21.15.87:0/4238405360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e081618a0 msgr2=0x7f0e0815e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 --2- 172.21.15.87:0/4238405360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e081618a0 0x7f0e0815e2b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f0e0815f3d0 tx=0x7f0df401c3e0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- 172.21.15.87:0/4238405360 shutdown_connections 2024-04-25T10:25:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 --2- 172.21.15.87:0/4238405360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e081618a0 0x7f0e0815e2b0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- 172.21.15.87:0/4238405360 >> 172.21.15.87:0/4238405360 conn(0x7f0e080b5610 msgr2=0x7f0e080b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- 172.21.15.87:0/4238405360 shutdown_connections 2024-04-25T10:25:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- 172.21.15.87:0/4238405360 wait complete. 2024-04-25T10:25:33.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 Processor -- start 2024-04-25T10:25:33.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- start start 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e080b4f50 0x7f0e08150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e081618a0 0x7f0e0815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e08162310 0x7f0e0815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080b14a0 con 0x7f0e08162310 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080b1320 con 0x7f0e080b4f50 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e0e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0816ba40 con 0x7f0e081618a0 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e08162310 0x7f0e0815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e08162310 0x7f0e0815d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60412/0 (socket says 172.21.15.87:60412) 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.219+0000 7f0e07e00640 1 -- 172.21.15.87:0/3368740279 learned_addr learned my addr 172.21.15.87:0/3368740279 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07400640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e080b4f50 0x7f0e08150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e06a00640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e081618a0 0x7f0e0815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07e00640 1 -- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e081618a0 msgr2=0x7f0e0815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07e00640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e081618a0 0x7f0e0815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07e00640 1 -- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e080b4f50 msgr2=0x7f0e08150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07e00640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e080b4f50 0x7f0e08150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07e00640 1 -- 172.21.15.87:0/3368740279 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08151f10 con 0x7f0e08162310 2024-04-25T10:25:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07e00640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e08162310 0x7f0e0815d760 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000b410 tx=0x7f0e0000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0dff400640 1 -- 172.21.15.87:0/3368740279 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00015020 con 0x7f0e08162310 2024-04-25T10:25:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0dff400640 1 -- 172.21.15.87:0/3368740279 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e00011870 con 0x7f0e08162310 2024-04-25T10:25:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0dff400640 1 -- 172.21.15.87:0/3368740279 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00011c60 con 0x7f0e08162310 2024-04-25T10:25:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e081521d0 con 0x7f0e08162310 2024-04-25T10:25:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e08152790 con 0x7f0e08162310 2024-04-25T10:25:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0804ef30 con 0x7f0e08162310 2024-04-25T10:25:33.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0dff400640 1 -- 172.21.15.87:0/3368740279 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e00018020 con 0x7f0e08162310 2024-04-25T10:25:33.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0dff400640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0de8066f60 0x7f0de8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0dff400640 1 -- 172.21.15.87:0/3368740279 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f0e00084b80 con 0x7f0e08162310 2024-04-25T10:25:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.223+0000 7f0e07400640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0de8066f60 0x7f0de8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.227+0000 7f0e07400640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0de8066f60 0x7f0de8069420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007fd0 tx=0x7f0df8005b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.227+0000 7f0dff400640 1 -- 172.21.15.87:0/3368740279 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e0001f030 con 0x7f0e08162310 2024-04-25T10:25:33.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:33.383+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e08152b90 con 0x7f0e08162310 2024-04-25T10:25:34.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.059+0000 7f0dff400640 1 -- 172.21.15.87:0/3368740279 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v168) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0e00055b80 con 0x7f0e08162310 2024-04-25T10:25:34.058 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:25:34.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.059+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0de8066f60 msgr2=0x7f0de8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.059+0000 7f0e0e200640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0de8066f60 0x7f0de8069420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007fd0 tx=0x7f0df8005b80 comp rx=0 tx=0).stop 2024-04-25T10:25:34.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.059+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e08162310 msgr2=0x7f0e0815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.059+0000 7f0e0e200640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e08162310 0x7f0e0815d760 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000b410 tx=0x7f0e0000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 shutdown_connections 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0de8066f60 0x7f0de8069420 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e08162310 0x7f0e0815d760 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e081618a0 0x7f0e0815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 --2- 172.21.15.87:0/3368740279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e080b4f50 0x7f0e08150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 >> 172.21.15.87:0/3368740279 conn(0x7f0e080b5610 msgr2=0x7f0e0815f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 shutdown_connections 2024-04-25T10:25:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.063+0000 7f0e0e200640 1 -- 172.21.15.87:0/3368740279 wait complete. 2024-04-25T10:25:34.082 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:25:34.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.219+0000 7f0c3b200640 1 Processor -- start 2024-04-25T10:25:34.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.219+0000 7f0c3b200640 1 -- start start 2024-04-25T10:25:34.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c3b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c340d0d00 0x7f0c340ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c3b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c340cdab0 con 0x7f0c340cc200 2024-04-25T10:25:34.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c3b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c340cd210 con 0x7f0c3406cf40 2024-04-25T10:25:34.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c3b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c340cd4f0 con 0x7f0c340d0d00 2024-04-25T10:25:34.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c33e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c340d0d00 0x7f0c340ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c33e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c340d0d00 0x7f0c340ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60428/0 (socket says 172.21.15.87:60428) 2024-04-25T10:25:34.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c33e00640 1 -- 172.21.15.87:0/3149416004 learned_addr learned my addr 172.21.15.87:0/3149416004 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:34.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c33e00640 1 -- 172.21.15.87:0/3149416004 >> v1:172.21.15.169:6789/0 conn(0x7f0c3406cf40 legacy=0x7f0c3415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c33e00640 1 -- 172.21.15.87:0/3149416004 >> v1:172.21.15.87:6789/0 conn(0x7f0c340cc200 legacy=0x7f0c34162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.223+0000 7f0c33e00640 1 -- 172.21.15.87:0/3149416004 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c340cd7d0 con 0x7f0c340d0d00 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c33e00640 1 --2- 172.21.15.87:0/3149416004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340ccc30 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009e50 tx=0x7f0c2401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f4a8ac570c1f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c32a00640 1 -- 172.21.15.87:0/3149416004 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240a5660 con 0x7f0c340d0d00 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c32a00640 1 -- 172.21.15.87:0/3149416004 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c240a5800 con 0x7f0c340d0d00 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c32a00640 1 -- 172.21.15.87:0/3149416004 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240a5bd0 con 0x7f0c340d0d00 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 -- 172.21.15.87:0/3149416004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 msgr2=0x7f0c340ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 --2- 172.21.15.87:0/3149416004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340ccc30 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009e50 tx=0x7f0c2401c9a0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 -- 172.21.15.87:0/3149416004 shutdown_connections 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 --2- 172.21.15.87:0/3149416004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340ccc30 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 -- 172.21.15.87:0/3149416004 >> 172.21.15.87:0/3149416004 conn(0x7f0c34072290 msgr2=0x7f0c340726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:34.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 -- 172.21.15.87:0/3149416004 shutdown_connections 2024-04-25T10:25:34.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 -- 172.21.15.87:0/3149416004 wait complete. 2024-04-25T10:25:34.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 Processor -- start 2024-04-25T10:25:34.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.227+0000 7f0c3b200640 1 -- start start 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c3406cf40 0x7f0c3415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c340cc200 0x7f0c340c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34162a10 con 0x7f0c340d0d00 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34162890 con 0x7f0c3406cf40 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3416c690 con 0x7f0c340cc200 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60442/0 (socket says 172.21.15.87:60442) 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 -- 172.21.15.87:0/775993631 learned_addr learned my addr 172.21.15.87:0/775993631 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:34.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c33e00640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c3406cf40 0x7f0c3415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c33400640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c340cc200 0x7f0c340c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 -- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c340cc200 msgr2=0x7f0c340c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c340cc200 0x7f0c340c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 -- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c3406cf40 msgr2=0x7f0c3415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c3406cf40 0x7f0c3415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 -- 172.21.15.87:0/775993631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c340c9b20 con 0x7f0c340d0d00 2024-04-25T10:25:34.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c39200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340c7830 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c370 tx=0x7f0c2c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c2be00640 1 -- 172.21.15.87:0/775993631 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c005220 con 0x7f0c340d0d00 2024-04-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c2be00640 1 -- 172.21.15.87:0/775993631 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0053c0 con 0x7f0c340d0d00 2024-04-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c2be00640 1 -- 172.21.15.87:0/775993631 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c007270 con 0x7f0c340d0d00 2024-04-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c340c9dc0 con 0x7f0c340d0d00 2024-04-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c340ca380 con 0x7f0c340d0d00 2024-04-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3404ef30 con 0x7f0c340d0d00 2024-04-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.231+0000 7f0c2be00640 1 -- 172.21.15.87:0/775993631 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c2c019020 con 0x7f0c340d0d00 2024-04-25T10:25:34.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.235+0000 7f0c2be00640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c14066f90 0x7f0c14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.235+0000 7f0c2be00640 1 -- 172.21.15.87:0/775993631 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0c2c083f90 con 0x7f0c340d0d00 2024-04-25T10:25:34.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.235+0000 7f0c33e00640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c14066f90 0x7f0c14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.235+0000 7f0c33e00640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c14066f90 0x7f0c14069450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0c240ae000 tx=0x7f0c240a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:34.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.239+0000 7f0c2be00640 1 -- 172.21.15.87:0/775993631 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c2c054bb0 con 0x7f0c340d0d00 2024-04-25T10:25:34.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.403+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0c340c4cd0 con 0x7f0c340d0d00 2024-04-25T10:25:34.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.403+0000 7f0c2be00640 1 -- 172.21.15.87:0/775993631 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c2c058c50 con 0x7f0c340d0d00 2024-04-25T10:25:34.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c14066f90 msgr2=0x7f0c14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c14066f90 0x7f0c14069450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0c240ae000 tx=0x7f0c240a7070 comp rx=0 tx=0).stop 2024-04-25T10:25:34.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 msgr2=0x7f0c340c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340c7830 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c370 tx=0x7f0c2c00c830 comp rx=0 tx=0).stop 2024-04-25T10:25:34.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 shutdown_connections 2024-04-25T10:25:34.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0c14066f90 0x7f0c14069450 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c340d0d00 0x7f0c340c7830 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c340cc200 0x7f0c340c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 --2- 172.21.15.87:0/775993631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c3406cf40 0x7f0c3415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 >> 172.21.15.87:0/775993631 conn(0x7f0c34072290 msgr2=0x7f0c3415e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:34.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 shutdown_connections 2024-04-25T10:25:34.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.407+0000 7f0c3b200640 1 -- 172.21.15.87:0/775993631 wait complete. 2024-04-25T10:25:34.427 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:25:34.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f2290c00640 1 Processor -- start 2024-04-25T10:25:34.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f2290c00640 1 -- start start 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f2290c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c15a780 0x7f228c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f2290c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f228c0cff00 0x7f228c0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f2290c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c06de60 con 0x7f228c0cf3b0 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f2290c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228c15f2f0 con 0x7f228c0cff00 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f2290c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c06cdc0 con 0x7f228c15a780 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f228c0cff00 0x7f228c0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f228c0cf3b0 0x7f228c0cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41392/0 (socket says 172.21.15.87:41392) 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228aa00640 1 -- 172.21.15.87:0/3007856122 learned_addr learned my addr 172.21.15.87:0/3007856122 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:34.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228b400640 1 --2- 172.21.15.87:0/3007856122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c15a780 0x7f228c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228be00640 1 -- 172.21.15.87:0/3007856122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c15a780 msgr2=0x7f228c15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:34.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228be00640 1 --2- 172.21.15.87:0/3007856122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c15a780 0x7f228c15edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228be00640 1 -- 172.21.15.87:0/3007856122 >> v1:172.21.15.87:6789/0 conn(0x7f228c0cf3b0 legacy=0x7f228c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228be00640 1 -- 172.21.15.87:0/3007856122 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c06d0a0 con 0x7f228c0cff00 2024-04-25T10:25:34.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.563+0000 7f228a000640 1 -- 172.21.15.87:0/3007856122 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195265224 0 0) 0x7f228c06de60 con 0x7f228c0cf3b0 2024-04-25T10:25:34.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f228be00640 1 --2- 172.21.15.87:0/3007856122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c0cff00 0x7f228c0b2460 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f2284009d90 tx=0x7f228401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39926eba92fe2fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:34.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f228a000640 1 -- 172.21.15.87:0/3007856122 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840b2070 con 0x7f228c0cff00 2024-04-25T10:25:34.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f228a000640 1 -- 172.21.15.87:0/3007856122 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22840a5830 con 0x7f228c0cff00 2024-04-25T10:25:34.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f228a000640 1 -- 172.21.15.87:0/3007856122 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840a5c00 con 0x7f228c0cff00 2024-04-25T10:25:34.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 -- 172.21.15.87:0/3007856122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c0cff00 msgr2=0x7f228c0b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 --2- 172.21.15.87:0/3007856122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c0cff00 0x7f228c0b2460 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f2284009d90 tx=0x7f228401c970 comp rx=0 tx=0).stop 2024-04-25T10:25:34.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 -- 172.21.15.87:0/3007856122 shutdown_connections 2024-04-25T10:25:34.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 --2- 172.21.15.87:0/3007856122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c0cff00 0x7f228c0b2460 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 --2- 172.21.15.87:0/3007856122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c15a780 0x7f228c15edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 -- 172.21.15.87:0/3007856122 >> 172.21.15.87:0/3007856122 conn(0x7f228c0743f0 msgr2=0x7f228c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:34.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 -- 172.21.15.87:0/3007856122 shutdown_connections 2024-04-25T10:25:34.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 -- 172.21.15.87:0/3007856122 wait complete. 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.567+0000 7f2290c00640 1 Processor -- start 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 -- start start 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c0cf3b0 0x7f228c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f228c0cff00 0x7f228c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c15a780 0x7f228c14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0b3330 con 0x7f228c15a780 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0b31b0 con 0x7f228c0cff00 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0b34b0 con 0x7f228c0cf3b0 2024-04-25T10:25:34.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c15a780 0x7f228c14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c15a780 0x7f228c14dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60454/0 (socket says 172.21.15.87:60454) 2024-04-25T10:25:34.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 -- 172.21.15.87:0/1292009186 learned_addr learned my addr 172.21.15.87:0/1292009186 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:34.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228b400640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c0cf3b0 0x7f228c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228aa00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f228c0cff00 0x7f228c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 -- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c0cf3b0 msgr2=0x7f228c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c0cf3b0 0x7f228c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 -- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f228c0cff00 msgr2=0x7f228c14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f228c0cff00 0x7f228c14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 -- 172.21.15.87:0/1292009186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c167640 con 0x7f228c15a780 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228aa00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f228c0cff00 0x7f228c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f228be00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c15a780 0x7f228c14dd50 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f2284009d90 tx=0x7f2284019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2283400640 1 -- 172.21.15.87:0/1292009186 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840b2040 con 0x7f228c15a780 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2283400640 1 -- 172.21.15.87:0/1292009186 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22840bf070 con 0x7f228c15a780 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c167920 con 0x7f228c15a780 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2283400640 1 -- 172.21.15.87:0/1292009186 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840b88b0 con 0x7f228c15a780 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c167ee0 con 0x7f228c15a780 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.571+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c075120 con 0x7f228c15a780 2024-04-25T10:25:34.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.575+0000 7f2283400640 1 -- 172.21.15.87:0/1292009186 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22840a56b0 con 0x7f228c15a780 2024-04-25T10:25:34.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.575+0000 7f2283400640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f226c066f60 0x7f226c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:34.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.575+0000 7f2283400640 1 -- 172.21.15.87:0/1292009186 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f228411ff50 con 0x7f228c15a780 2024-04-25T10:25:34.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.575+0000 7f228b400640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f226c066f60 0x7f226c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:34.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.575+0000 7f228b400640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f226c066f60 0x7f226c069420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f227c007a60 tx=0x7f227c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:34.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.579+0000 7f2283400640 1 -- 172.21.15.87:0/1292009186 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2284016560 con 0x7f228c15a780 2024-04-25T10:25:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:34.783+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f228c06c8d0 con 0x7f228c15a780 2024-04-25T10:25:35.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.059+0000 7f2283400640 1 -- 172.21.15.87:0/1292009186 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v634) v1 ==== 133+0+0 (secure 0 0 0) 0x7f22840f1300 con 0x7f228c15a780 2024-04-25T10:25:35.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f226c066f60 msgr2=0x7f226c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f226c066f60 0x7f226c069420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f227c007a60 tx=0x7f227c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c15a780 msgr2=0x7f228c14dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c15a780 0x7f228c14dd50 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f2284009d90 tx=0x7f2284019710 comp rx=0 tx=0).stop 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 shutdown_connections 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f226c066f60 0x7f226c069420 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f228c15a780 0x7f228c14dd50 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f228c0cff00 0x7f228c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 --2- 172.21.15.87:0/1292009186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228c0cf3b0 0x7f228c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 >> 172.21.15.87:0/1292009186 conn(0x7f228c0743f0 msgr2=0x7f228c15eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 shutdown_connections 2024-04-25T10:25:35.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.063+0000 7f2290c00640 1 -- 172.21.15.87:0/1292009186 wait complete. 2024-04-25T10:25:35.082 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:25:35.082 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:25:35.083 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f75a00640 1 Processor -- start 2024-04-25T10:25:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f75a00640 1 -- start start 2024-04-25T10:25:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f75a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f7015b9e0 0x7f8f700cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f75a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8f700d1c30 0x7f8f700cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f75a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f700ccfb0 con 0x7f8f7015bdc0 2024-04-25T10:25:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f75a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f700cd290 con 0x7f8f700d1c30 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f75a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f700cd570 con 0x7f8f7015b9e0 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8f700d1c30 0x7f8f700cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6e000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8f7015bdc0 0x7f8f700d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:41400/0 (socket says 172.21.15.87:41400) 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6e000640 1 -- 172.21.15.87:0/2304650275 learned_addr learned my addr 172.21.15.87:0/2304650275 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6ea00640 1 --2- 172.21.15.87:0/2304650275 >> v2:172.21.15.169:3300/0 conn(0x7f8f7015b9e0 0x7f8f700cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6f400640 1 -- 172.21.15.87:0/2304650275 >> v2:172.21.15.169:3300/0 conn(0x7f8f7015b9e0 msgr2=0x7f8f700cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6f400640 1 --2- 172.21.15.87:0/2304650275 >> v2:172.21.15.169:3300/0 conn(0x7f8f7015b9e0 0x7f8f700cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6f400640 1 -- 172.21.15.87:0/2304650275 >> v1:172.21.15.87:6789/0 conn(0x7f8f7015bdc0 legacy=0x7f8f700d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6f400640 1 -- 172.21.15.87:0/2304650275 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f700cd850 con 0x7f8f700d1c30 2024-04-25T10:25:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.219+0000 7f8f6d600640 1 -- 172.21.15.87:0/2304650275 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892572029 0 0) 0x7f8f700ccfb0 con 0x7f8f7015bdc0 2024-04-25T10:25:35.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f6f400640 1 --2- 172.21.15.87:0/2304650275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f700cc950 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f8f70070da0 tx=0x7f8f6801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5287c2c3d3e9f3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f6d600640 1 -- 172.21.15.87:0/2304650275 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f680b0070 con 0x7f8f700d1c30 2024-04-25T10:25:35.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f6d600640 1 -- 172.21.15.87:0/2304650275 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f680ab600 con 0x7f8f700d1c30 2024-04-25T10:25:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f6d600640 1 -- 172.21.15.87:0/2304650275 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f680ab9d0 con 0x7f8f700d1c30 2024-04-25T10:25:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 -- 172.21.15.87:0/2304650275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 msgr2=0x7f8f700cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2304650275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f700cc950 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f8f70070da0 tx=0x7f8f6801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 -- 172.21.15.87:0/2304650275 shutdown_connections 2024-04-25T10:25:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2304650275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f700cc950 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2304650275 >> v2:172.21.15.169:3300/0 conn(0x7f8f7015b9e0 0x7f8f700cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 -- 172.21.15.87:0/2304650275 >> 172.21.15.87:0/2304650275 conn(0x7f8f7009a9d0 msgr2=0x7f8f7009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 -- 172.21.15.87:0/2304650275 shutdown_connections 2024-04-25T10:25:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.223+0000 7f8f75a00640 1 -- 172.21.15.87:0/2304650275 wait complete. 2024-04-25T10:25:35.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 Processor -- start 2024-04-25T10:25:35.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 -- start start 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f7014c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7015b9e0 0x7f8f700cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7015bdc0 0x7f8f7006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700c6630 con 0x7f8f700d1c30 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700c64b0 con 0x7f8f7015b9e0 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700c67b0 con 0x7f8f7015bdc0 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f7014c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f7014c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60484/0 (socket says 172.21.15.87:60484) 2024-04-25T10:25:35.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 -- 172.21.15.87:0/2693303336 learned_addr learned my addr 172.21.15.87:0/2693303336 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6e000640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7015b9e0 0x7f8f700cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6f400640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7015bdc0 0x7f8f7006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 -- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7015bdc0 msgr2=0x7f8f7006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7015bdc0 0x7f8f7006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 -- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7015b9e0 msgr2=0x7f8f700cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7015b9e0 0x7f8f700cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 -- 172.21.15.87:0/2693303336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f700c8bb0 con 0x7f8f700d1c30 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f6ea00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f7014c260 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f8f60007ef0 tx=0x7f8f6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f66a00640 1 -- 172.21.15.87:0/2693303336 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60014620 con 0x7f8f700d1c30 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f66a00640 1 -- 172.21.15.87:0/2693303336 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f60019070 con 0x7f8f700d1c30 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f70155b00 con 0x7f8f700d1c30 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f66a00640 1 -- 172.21.15.87:0/2693303336 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60014a00 con 0x7f8f700d1c30 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.227+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7015c380 con 0x7f8f700d1c30 2024-04-25T10:25:35.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.231+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7014d2c0 con 0x7f8f700d1c30 2024-04-25T10:25:35.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.231+0000 7f8f66a00640 1 -- 172.21.15.87:0/2693303336 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f60014ba0 con 0x7f8f700d1c30 2024-04-25T10:25:35.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.231+0000 7f8f66a00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8f50066f60 0x7f8f50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.231+0000 7f8f66a00640 1 -- 172.21.15.87:0/2693303336 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8f600854b0 con 0x7f8f700d1c30 2024-04-25T10:25:35.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.231+0000 7f8f6e000640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8f50066f60 0x7f8f50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.231+0000 7f8f6e000640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8f50066f60 0x7f8f50069420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8f70159650 tx=0x7f8f5c007df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.235+0000 7f8f66a00640 1 -- 172.21.15.87:0/2693303336 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f700c8bb0 con 0x7f8f700d1c30 2024-04-25T10:25:35.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.391+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f7015a470 con 0x7f8f700d1c30 2024-04-25T10:25:35.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.391+0000 7f8f66a00640 1 -- 172.21.15.87:0/2693303336 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+13018 (secure 0 0 0) 0x7f8f60056160 con 0x7f8f700d1c30 2024-04-25T10:25:35.392 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:35.392 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":168,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:34.053644+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/1488445455":"2024-04-26T10:25:31.042347+0000","172.21.15.87:6834/1488445455":"2024-04-26T10:25:31.042347+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8f50066f60 msgr2=0x7f8f50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8f50066f60 0x7f8f50069420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8f70159650 tx=0x7f8f5c007df0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 msgr2=0x7f8f7014c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f7014c260 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f8f60007ef0 tx=0x7f8f6000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 shutdown_connections 2024-04-25T10:25:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8f50066f60 0x7f8f50069420 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7015bdc0 0x7f8f7006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7015b9e0 0x7f8f700cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 --2- 172.21.15.87:0/2693303336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f700d1c30 0x7f8f7014c260 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 >> 172.21.15.87:0/2693303336 conn(0x7f8f7009a9d0 msgr2=0x7f8f70159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 shutdown_connections 2024-04-25T10:25:35.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.395+0000 7f8f75a00640 1 -- 172.21.15.87:0/2693303336 wait complete. 2024-04-25T10:25:35.413 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:35.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:35.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.551+0000 7f2c19a00640 1 Processor -- start 2024-04-25T10:25:35.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.551+0000 7f2c19a00640 1 -- start start 2024-04-25T10:25:35.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c19a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2c14159540 0x7f2c140ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c19a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140721f0 0x7f2c14070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c19a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c14159920 con 0x7f2c140aebd0 2024-04-25T10:25:35.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c19a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1415c2f0 con 0x7f2c14159540 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c19a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1415c5d0 con 0x7f2c140721f0 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c12a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2c14159540 0x7f2c140ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c12a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2c14159540 0x7f2c140ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60496/0 (socket says 172.21.15.87:60496) 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c12a00640 1 -- 172.21.15.87:0/1182761224 learned_addr learned my addr 172.21.15.87:0/1182761224 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c13400640 1 --2- 172.21.15.87:0/1182761224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140721f0 0x7f2c14070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c12a00640 1 -- 172.21.15.87:0/1182761224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140721f0 msgr2=0x7f2c14070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c12a00640 1 --2- 172.21.15.87:0/1182761224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140721f0 0x7f2c14070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c12a00640 1 -- 172.21.15.87:0/1182761224 >> v1:172.21.15.169:6789/0 conn(0x7f2c140aebd0 legacy=0x7f2c14071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.555+0000 7f2c12a00640 1 -- 172.21.15.87:0/1182761224 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1415c8b0 con 0x7f2c14159540 2024-04-25T10:25:35.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c12a00640 1 --2- 172.21.15.87:0/1182761224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c140ae690 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009870 tx=0x7f2bfc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d408d91c51abc1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c11600640 1 -- 172.21.15.87:0/1182761224 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0a7070 con 0x7f2c14159540 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c11600640 1 -- 172.21.15.87:0/1182761224 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0b0070 con 0x7f2c14159540 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c11600640 1 -- 172.21.15.87:0/1182761224 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0ab890 con 0x7f2c14159540 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- 172.21.15.87:0/1182761224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 msgr2=0x7f2c140ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 --2- 172.21.15.87:0/1182761224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c140ae690 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009870 tx=0x7f2bfc01c560 comp rx=0 tx=0).stop 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- 172.21.15.87:0/1182761224 shutdown_connections 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 --2- 172.21.15.87:0/1182761224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140721f0 0x7f2c14070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 --2- 172.21.15.87:0/1182761224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c140ae690 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- 172.21.15.87:0/1182761224 >> 172.21.15.87:0/1182761224 conn(0x7f2c140669a0 msgr2=0x7f2c14066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- 172.21.15.87:0/1182761224 shutdown_connections 2024-04-25T10:25:35.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- 172.21.15.87:0/1182761224 wait complete. 2024-04-25T10:25:35.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 Processor -- start 2024-04-25T10:25:35.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- start start 2024-04-25T10:25:35.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c140721f0 0x7f2c140b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140aebd0 0x7f2c14157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c1414a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14071220 con 0x7f2c14159540 2024-04-25T10:25:35.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c140710a0 con 0x7f2c140721f0 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.559+0000 7f2c19a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c140713a0 con 0x7f2c140aebd0 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c1414a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c1414a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60504/0 (socket says 172.21.15.87:60504) 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 -- 172.21.15.87:0/2543176088 learned_addr learned my addr 172.21.15.87:0/2543176088 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c12a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c140721f0 0x7f2c140b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c12000640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140aebd0 0x7f2c14157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 -- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140aebd0 msgr2=0x7f2c14157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140aebd0 0x7f2c14157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 -- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c140721f0 msgr2=0x7f2c140b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c140721f0 0x7f2c140b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 -- 172.21.15.87:0/2543176088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14157d60 con 0x7f2c14159540 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c13400640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c1414a0c0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007e70 tx=0x7f2c0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c0aa00640 1 -- 172.21.15.87:0/2543176088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c014620 con 0x7f2c14159540 2024-04-25T10:25:35.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c0aa00640 1 -- 172.21.15.87:0/2543176088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0c019070 con 0x7f2c14159540 2024-04-25T10:25:35.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c0aa00640 1 -- 172.21.15.87:0/2543176088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c014a20 con 0x7f2c14159540 2024-04-25T10:25:35.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c140b2f20 con 0x7f2c14159540 2024-04-25T10:25:35.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c140b34e0 con 0x7f2c14159540 2024-04-25T10:25:35.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1404ef30 con 0x7f2c14159540 2024-04-25T10:25:35.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.563+0000 7f2c0aa00640 1 -- 172.21.15.87:0/2543176088 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c0c014bc0 con 0x7f2c14159540 2024-04-25T10:25:35.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.567+0000 7f2c0aa00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bf0066f90 0x7f2bf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.567+0000 7f2c0aa00640 1 -- 172.21.15.87:0/2543176088 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2c0c085480 con 0x7f2c14159540 2024-04-25T10:25:35.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.567+0000 7f2c12a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bf0066f90 0x7f2bf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.567+0000 7f2c12a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bf0066f90 0x7f2bf0069450 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0af000 tx=0x7f2bfc0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.571+0000 7f2c0aa00640 1 -- 172.21.15.87:0/2543176088 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c0c056130 con 0x7f2c14159540 2024-04-25T10:25:35.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c1406d7f0 con 0x7f2c14159540 2024-04-25T10:25:35.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c0aa00640 1 -- 172.21.15.87:0/2543176088 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c0c05a1d0 con 0x7f2c14159540 2024-04-25T10:25:35.722 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:25:35.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bf0066f90 msgr2=0x7f2bf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bf0066f90 0x7f2bf0069450 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0af000 tx=0x7f2bfc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:25:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 msgr2=0x7f2c1414a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c1414a0c0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007e70 tx=0x7f2c0c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 shutdown_connections 2024-04-25T10:25:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bf0066f90 0x7f2bf0069450 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14159540 0x7f2c1414a0c0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c140aebd0 0x7f2c14157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 --2- 172.21.15.87:0/2543176088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c140721f0 0x7f2c140b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 >> 172.21.15.87:0/2543176088 conn(0x7f2c140669a0 msgr2=0x7f2c140cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 shutdown_connections 2024-04-25T10:25:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.723+0000 7f2c19a00640 1 -- 172.21.15.87:0/2543176088 wait complete. 2024-04-25T10:25:35.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:25:35.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75abe00640 1 Processor -- start 2024-04-25T10:25:35.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75abe00640 1 -- start start 2024-04-25T10:25:35.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75abe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a415e050 0x7f75a40b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75abe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75a40b13e0 0x7f75a40cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75abe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a40b1870 con 0x7f75a40d1820 2024-04-25T10:25:35.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75abe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a40b1af0 con 0x7f75a40b13e0 2024-04-25T10:25:35.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75abe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40b1dd0 con 0x7f75a415e050 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a415e050 0x7f75a40b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a415e050 0x7f75a40b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:57544/0 (socket says 172.21.15.87:57544) 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a8a00640 1 -- 172.21.15.87:0/672569318 learned_addr learned my addr 172.21.15.87:0/672569318 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a9e00640 1 --2- 172.21.15.87:0/672569318 >> v2:172.21.15.169:3300/0 conn(0x7f75a40b13e0 0x7f75a40cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a8a00640 1 -- 172.21.15.87:0/672569318 >> v2:172.21.15.169:3300/0 conn(0x7f75a40b13e0 msgr2=0x7f75a40cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a8a00640 1 --2- 172.21.15.87:0/672569318 >> v2:172.21.15.169:3300/0 conn(0x7f75a40b13e0 0x7f75a40cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a8a00640 1 -- 172.21.15.87:0/672569318 >> v1:172.21.15.169:6789/0 conn(0x7f75a40d1820 legacy=0x7f75a40d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a8a00640 1 -- 172.21.15.87:0/672569318 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a40b20b0 con 0x7f75a415e050 2024-04-25T10:25:35.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.879+0000 7f75a3e00640 1 -- 172.21.15.87:0/672569318 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362442824 0 0) 0x7f7598001120 con 0x7f75a40d1820 2024-04-25T10:25:35.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75a8a00640 1 --2- 172.21.15.87:0/672569318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a415e050 0x7f75a40b0ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7590009870 tx=0x7f759001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef2ad8f60f86471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75a3e00640 1 -- 172.21.15.87:0/672569318 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900aa430 con 0x7f75a415e050 2024-04-25T10:25:35.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75a3e00640 1 -- 172.21.15.87:0/672569318 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75900af070 con 0x7f75a415e050 2024-04-25T10:25:35.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75a3e00640 1 -- 172.21.15.87:0/672569318 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900aa7f0 con 0x7f75a415e050 2024-04-25T10:25:35.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 -- 172.21.15.87:0/672569318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a415e050 msgr2=0x7f75a40b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 --2- 172.21.15.87:0/672569318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a415e050 0x7f75a40b0ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7590009870 tx=0x7f759001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 -- 172.21.15.87:0/672569318 shutdown_connections 2024-04-25T10:25:35.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 --2- 172.21.15.87:0/672569318 >> v2:172.21.15.169:3300/0 conn(0x7f75a40b13e0 0x7f75a40cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 --2- 172.21.15.87:0/672569318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a415e050 0x7f75a40b0ea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 -- 172.21.15.87:0/672569318 >> 172.21.15.87:0/672569318 conn(0x7f75a409a9d0 msgr2=0x7f75a409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:35.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 -- 172.21.15.87:0/672569318 shutdown_connections 2024-04-25T10:25:35.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.883+0000 7f75abe00640 1 -- 172.21.15.87:0/672569318 wait complete. 2024-04-25T10:25:35.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 Processor -- start 2024-04-25T10:25:35.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 -- start start 2024-04-25T10:25:35.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40b13e0 0x7f75a40cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40d1820 0x7f75a40c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a415e050 0x7f75a40c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40d0d90 con 0x7f75a415e050 2024-04-25T10:25:35.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40d0c10 con 0x7f75a40d1820 2024-04-25T10:25:35.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40d0f10 con 0x7f75a40b13e0 2024-04-25T10:25:35.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a415e050 0x7f75a40c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a415e050 0x7f75a40c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60510/0 (socket says 172.21.15.87:60510) 2024-04-25T10:25:35.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 -- 172.21.15.87:0/1963283704 learned_addr learned my addr 172.21.15.87:0/1963283704 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:35.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9400640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40b13e0 0x7f75a40cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a8a00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40d1820 0x7f75a40c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 -- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40b13e0 msgr2=0x7f75a40cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40b13e0 0x7f75a40cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 -- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40d1820 msgr2=0x7f75a40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:35.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40d1820 0x7f75a40c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:35.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 -- 172.21.15.87:0/1963283704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a40cb830 con 0x7f75a415e050 2024-04-25T10:25:35.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a8a00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40d1820 0x7f75a40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:35.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9400640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40b13e0 0x7f75a40cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:35.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a9e00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a415e050 0x7f75a40c9410 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f759c009d30 tx=0x7f759c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a1600640 1 -- 172.21.15.87:0/1963283704 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c018020 con 0x7f75a415e050 2024-04-25T10:25:35.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a1600640 1 -- 172.21.15.87:0/1963283704 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c014480 con 0x7f75a415e050 2024-04-25T10:25:35.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75a1600640 1 -- 172.21.15.87:0/1963283704 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c014870 con 0x7f75a415e050 2024-04-25T10:25:35.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a415b1b0 con 0x7f75a415e050 2024-04-25T10:25:35.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.887+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a415b430 con 0x7f75a415e050 2024-04-25T10:25:35.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.891+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a404ef30 con 0x7f75a415e050 2024-04-25T10:25:35.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.891+0000 7f75a1600640 1 -- 172.21.15.87:0/1963283704 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f759c01b020 con 0x7f75a415e050 2024-04-25T10:25:35.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.891+0000 7f75a1600640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7584066f60 0x7f7584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:35.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.891+0000 7f75a1600640 1 -- 172.21.15.87:0/1963283704 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f759c0857f0 con 0x7f75a415e050 2024-04-25T10:25:35.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.891+0000 7f75a9400640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7584066f60 0x7f7584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:35.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.891+0000 7f75a9400640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7584066f60 0x7f7584069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7598000f70 tx=0x7f7598009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:35.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:35.895+0000 7f75a1600640 1 -- 172.21.15.87:0/1963283704 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f759c022070 con 0x7f75a415e050 2024-04-25T10:25:36.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:36.103+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f75a406b8f0 con 0x7f75a415e050 2024-04-25T10:25:37.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.067+0000 7f75a1600640 1 -- 172.21.15.87:0/1963283704 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v636) v1 ==== 150+0+0 (secure 0 0 0) 0x7f75a415b1b0 con 0x7f75a415e050 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7584066f60 msgr2=0x7f7584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7584066f60 0x7f7584069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7598000f70 tx=0x7f7598009040 comp rx=0 tx=0).stop 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a415e050 msgr2=0x7f75a40c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a415e050 0x7f75a40c9410 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f759c009d30 tx=0x7f759c00c490 comp rx=0 tx=0).stop 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 shutdown_connections 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7584066f60 0x7f7584069420 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a415e050 0x7f75a40c9410 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40d1820 0x7f75a40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 --2- 172.21.15.87:0/1963283704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40b13e0 0x7f75a40cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 >> 172.21.15.87:0/1963283704 conn(0x7f75a409a9d0 msgr2=0x7f75a40d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:37.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 shutdown_connections 2024-04-25T10:25:37.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.071+0000 7f75abe00640 1 -- 172.21.15.87:0/1963283704 wait complete. 2024-04-25T10:25:37.093 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:25:37.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f206cc00640 1 Processor -- start 2024-04-25T10:25:37.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f206cc00640 1 -- start start 2024-04-25T10:25:37.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f206cc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f20680aef80 0x7f206815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f206cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20680b31c0 0x7f20680b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f206cc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206815b080 con 0x7f2068070350 2024-04-25T10:25:37.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f206cc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206815adb0 con 0x7f20680aef80 2024-04-25T10:25:37.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f206cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206815eb60 con 0x7f20680b31c0 2024-04-25T10:25:37.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f2066800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20680b31c0 0x7f20680b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f2065400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2068070350 0x7f206815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:44726/0 (socket says 172.21.15.87:44726) 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f2066800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20680b31c0 0x7f20680b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:57564/0 (socket says 172.21.15.87:57564) 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.235+0000 7f2065400640 1 -- 172.21.15.87:0/3494045539 learned_addr learned my addr 172.21.15.87:0/3494045539 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2065e00640 1 --2- 172.21.15.87:0/3494045539 >> v2:172.21.15.169:3300/0 conn(0x7f20680aef80 0x7f206815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2066800640 1 -- 172.21.15.87:0/3494045539 >> v2:172.21.15.169:3300/0 conn(0x7f20680aef80 msgr2=0x7f206815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2064a00640 1 -- 172.21.15.87:0/3494045539 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914811054 0 0) 0x7f206815b080 con 0x7f2068070350 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2066800640 1 --2- 172.21.15.87:0/3494045539 >> v2:172.21.15.169:3300/0 conn(0x7f20680aef80 0x7f206815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2066800640 1 -- 172.21.15.87:0/3494045539 >> v1:172.21.15.169:6789/0 conn(0x7f2068070350 legacy=0x7f206815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2066800640 1 -- 172.21.15.87:0/3494045539 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206815ee40 con 0x7f20680b31c0 2024-04-25T10:25:37.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2066800640 1 --2- 172.21.15.87:0/3494045539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20680b31c0 0x7f20680b3610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f20600097c0 tx=0x7f206001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91cf52eabd5e9a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:37.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2064a00640 1 -- 172.21.15.87:0/3494045539 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20600b1070 con 0x7f20680b31c0 2024-04-25T10:25:37.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2064a00640 1 -- 172.21.15.87:0/3494045539 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20600ac410 con 0x7f20680b31c0 2024-04-25T10:25:37.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.239+0000 7f2064a00640 1 -- 172.21.15.87:0/3494045539 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20600ac7e0 con 0x7f20680b31c0 2024-04-25T10:25:37.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- 172.21.15.87:0/3494045539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20680b31c0 msgr2=0x7f20680b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 --2- 172.21.15.87:0/3494045539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20680b31c0 0x7f20680b3610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f20600097c0 tx=0x7f206001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- 172.21.15.87:0/3494045539 shutdown_connections 2024-04-25T10:25:37.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 --2- 172.21.15.87:0/3494045539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20680b31c0 0x7f20680b3610 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 --2- 172.21.15.87:0/3494045539 >> v2:172.21.15.169:3300/0 conn(0x7f20680aef80 0x7f206815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- 172.21.15.87:0/3494045539 >> 172.21.15.87:0/3494045539 conn(0x7f20680669a0 msgr2=0x7f2068066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:37.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- 172.21.15.87:0/3494045539 shutdown_connections 2024-04-25T10:25:37.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- 172.21.15.87:0/3494045539 wait complete. 2024-04-25T10:25:37.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 Processor -- start 2024-04-25T10:25:37.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- start start 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2068070350 0x7f2068151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20680aef80 0x7f20680b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20680b31c0 0x7f20680b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206806c250 con 0x7f20680aef80 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206806c0d0 con 0x7f20680b31c0 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f206cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206806c3d0 con 0x7f2068070350 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20680aef80 0x7f20680b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20680aef80 0x7f20680b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60518/0 (socket says 172.21.15.87:60518) 2024-04-25T10:25:37.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 -- 172.21.15.87:0/2037006343 learned_addr learned my addr 172.21.15.87:0/2037006343 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065e00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2068070350 0x7f2068151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2066800640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20680b31c0 0x7f20680b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 -- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2068070350 msgr2=0x7f2068151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2068070350 0x7f2068151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 -- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20680b31c0 msgr2=0x7f20680b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20680b31c0 0x7f20680b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 -- 172.21.15.87:0/2037006343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680b13c0 con 0x7f20680aef80 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.243+0000 7f2065400640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20680aef80 0x7f20680b0970 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f205400c550 tx=0x7f205400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f205e000640 1 -- 172.21.15.87:0/2037006343 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054017070 con 0x7f20680aef80 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f205e000640 1 -- 172.21.15.87:0/2037006343 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2054005d10 con 0x7f20680aef80 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2068152310 con 0x7f20680aef80 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f205e000640 1 -- 172.21.15.87:0/2037006343 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054007330 con 0x7f20680aef80 2024-04-25T10:25:37.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20680b1660 con 0x7f20680aef80 2024-04-25T10:25:37.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206804ef30 con 0x7f20680aef80 2024-04-25T10:25:37.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f205e000640 1 -- 172.21.15.87:0/2037006343 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f205401a070 con 0x7f20680aef80 2024-04-25T10:25:37.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f205e000640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2048066f90 0x7f2048069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f205e000640 1 -- 172.21.15.87:0/2037006343 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2054084120 con 0x7f20680aef80 2024-04-25T10:25:37.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f2065e00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2048066f90 0x7f2048069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.247+0000 7f2065e00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2048066f90 0x7f2048069450 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2058009430 tx=0x7f205800d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:37.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.251+0000 7f205e000640 1 -- 172.21.15.87:0/2037006343 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2054055420 con 0x7f20680aef80 2024-04-25T10:25:37.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.407+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f20680b2fa0 con 0x7f20680aef80 2024-04-25T10:25:37.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.411+0000 7f205e000640 1 -- 172.21.15.87:0/2037006343 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2054054d20 con 0x7f20680aef80 2024-04-25T10:25:37.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2048066f90 msgr2=0x7f2048069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2048066f90 0x7f2048069450 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2058009430 tx=0x7f205800d040 comp rx=0 tx=0).stop 2024-04-25T10:25:37.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20680aef80 msgr2=0x7f20680b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20680aef80 0x7f20680b0970 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f205400c550 tx=0x7f205400ca10 comp rx=0 tx=0).stop 2024-04-25T10:25:37.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 shutdown_connections 2024-04-25T10:25:37.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2048066f90 0x7f2048069450 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20680b31c0 0x7f20680b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20680aef80 0x7f20680b0970 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 --2- 172.21.15.87:0/2037006343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2068070350 0x7f2068151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 >> 172.21.15.87:0/2037006343 conn(0x7f20680669a0 msgr2=0x7f20681567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:37.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 shutdown_connections 2024-04-25T10:25:37.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.415+0000 7f206cc00640 1 -- 172.21.15.87:0/2037006343 wait complete. 2024-04-25T10:25:37.433 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:25:37.434 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:37.515 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:37.581 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:25:37.653 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:37.654 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:37.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:37.664 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:25:37.664 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:37.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:37.673 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:37.673 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:37.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:37.713 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:25:37.713 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:37.730 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:37.730 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:25:37.730 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:37.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.891+0000 7f0f64600640 1 Processor -- start 2024-04-25T10:25:37.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.891+0000 7f0f64600640 1 -- start start 2024-04-25T10:25:37.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f64600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0f5c0cda10 0x7f0f5c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f64600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c0b2770 con 0x7f0f5c15bc40 2024-04-25T10:25:37.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f64600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c06c670 con 0x7f0f5c0cf930 2024-04-25T10:25:37.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f64600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c06c950 con 0x7f0f5c0cda10 2024-04-25T10:25:37.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0f5c15bc40 0x7f0f5c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55820/0 (socket says 172.21.15.87:55820) 2024-04-25T10:25:37.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 -- 172.21.15.87:0/3114681530 learned_addr learned my addr 172.21.15.87:0/3114681530 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:37.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859356903 0 0) 0x7f0f5c0b2770 con 0x7f0f5c15bc40 2024-04-25T10:25:37.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f44003530 con 0x7f0f5c15bc40 2024-04-25T10:25:37.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61200640 1 --2- 172.21.15.87:0/3114681530 >> v2:172.21.15.169:3300/0 conn(0x7f0f5c0cda10 0x7f0f5c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430533997 0 0) 0x7f0f44003530 con 0x7f0f5c15bc40 2024-04-25T10:25:37.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 >> v2:172.21.15.169:3300/0 conn(0x7f0f5c0cda10 msgr2=0x7f0f5c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:37.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 --2- 172.21.15.87:0/3114681530 >> v2:172.21.15.169:3300/0 conn(0x7f0f5c0cda10 0x7f0f5c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 >> v1:172.21.15.169:6789/0 conn(0x7f0f5c0cf930 legacy=0x7f0f5c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c06cc30 con 0x7f0f5c15bc40 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0f50001680 con 0x7f0f5c15bc40 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 >> v1:172.21.15.87:6789/0 conn(0x7f0f5c15bc40 legacy=0x7f0f5c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f44003a10 0x7f0f44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f44004810 0x7f0f44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f440052a0 0x7f0f4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c06c950 con 0x7f0f44003a10 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4400c1f0 con 0x7f0f440052a0 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4400c3f0 con 0x7f0f44004810 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f44003a10 0x7f0f44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f62600640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f440052a0 0x7f0f4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61200640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f44004810 0x7f0f44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 -- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f44004810 msgr2=0x7f0f44004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:37.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f44004810 0x7f0f44004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 -- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f440052a0 msgr2=0x7f0f4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:37.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f440052a0 0x7f0f4400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61c00640 1 -- 172.21.15.87:0/3114681530 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44003530 con 0x7f0f44003a10 2024-04-25T10:25:37.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f62600640 1 -- 172.21.15.87:0/3114681530 reap_dead start 2024-04-25T10:25:37.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.895+0000 7f0f61200640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f44004810 0x7f0f44004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:37.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f61c00640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f44003a10 0x7f0f44003e10 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000c190 tx=0x7f0f5001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fad0f1236bf422e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:37.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50007c70 con 0x7f0f44003a10 2024-04-25T10:25:37.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f500a9070 con 0x7f0f44003a10 2024-04-25T10:25:37.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f5be00640 1 -- 172.21.15.87:0/3114681530 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50005d60 con 0x7f0f44003a10 2024-04-25T10:25:37.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 -- 172.21.15.87:0/3114681530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f44003a10 msgr2=0x7f0f44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f44003a10 0x7f0f44003e10 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000c190 tx=0x7f0f5001cbe0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 -- 172.21.15.87:0/3114681530 shutdown_connections 2024-04-25T10:25:37.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 --2- 172.21.15.87:0/3114681530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f44003a10 0x7f0f44003e10 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 -- 172.21.15.87:0/3114681530 >> 172.21.15.87:0/3114681530 conn(0x7f0f5c066b70 msgr2=0x7f0f5c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:37.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 -- 172.21.15.87:0/3114681530 shutdown_connections 2024-04-25T10:25:37.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 -- 172.21.15.87:0/3114681530 wait complete. 2024-04-25T10:25:37.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 Processor -- start 2024-04-25T10:25:37.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.899+0000 7f0f64600640 1 -- start start 2024-04-25T10:25:37.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f44003a10 0x7f0f5c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f5c1576f0 0x7f0f5c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c1e96c0 0x7f0f5c1e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b49a0 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b4820 con 0x7f0f44003a10 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b4b20 con 0x7f0f5c1576f0 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c1e96c0 0x7f0f5c1e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c1e96c0 0x7f0f5c1e9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52822/0 (socket says 172.21.15.87:52822) 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 -- 172.21.15.87:0/927915968 learned_addr learned my addr 172.21.15.87:0/927915968 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f61200640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f5c1576f0 0x7f0f5c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f61c00640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f44003a10 0x7f0f5c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 -- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f5c1576f0 msgr2=0x7f0f5c157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f5c1576f0 0x7f0f5c157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 -- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f44003a10 msgr2=0x7f0f5c0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f44003a10 0x7f0f5c0ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 -- 172.21.15.87:0/927915968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c152490 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f62600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c1e96c0 0x7f0f5c1e9b40 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f0f440057f0 tx=0x7f0f5400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:37.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f59600640 1 -- 172.21.15.87:0/927915968 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54006d70 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f59600640 1 -- 172.21.15.87:0/927915968 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54019070 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f59600640 1 -- 172.21.15.87:0/927915968 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400d860 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c152770 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c152d30 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c04ef30 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.903+0000 7f0f59600640 1 -- 172.21.15.87:0/927915968 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f5400da00 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.907+0000 7f0f59600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f3c066f90 0x7f0f3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:37.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.907+0000 7f0f59600640 1 -- 172.21.15.87:0/927915968 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0f54084980 con 0x7f0f5c1e96c0 2024-04-25T10:25:37.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.907+0000 7f0f61c00640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f3c066f90 0x7f0f3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:37.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.907+0000 7f0f61c00640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f3c066f90 0x7f0f3c069450 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0f500ab000 tx=0x7f0f500b1040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:37.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:37.911+0000 7f0f59600640 1 -- 172.21.15.87:0/927915968 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f54055640 con 0x7f0f5c1e96c0 2024-04-25T10:25:38.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f5c0af1c0 con 0x7f0f5c1e96c0 2024-04-25T10:25:38.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f59600640 1 -- 172.21.15.87:0/927915968 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f0f540596e0 con 0x7f0f5c1e96c0 2024-04-25T10:25:38.095 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:38.095 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":121,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4197742987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4197742987},{"type":"v1","addr":"172.21.15.169:6834","nonce":4197742987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3984828245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3984828245},{"type":"v1","addr":"172.21.15.169:6835","nonce":3984828245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":9571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/3256626659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3256626659},{"type":"v1","addr":"172.21.15.87:6836","nonce":3256626659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":10830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/268474972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":268474972},{"type":"v1","addr":"172.21.15.87:6837","nonce":268474972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":121}],"filesystems":[]} 2024-04-25T10:25:38.095 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 121 2024-04-25T10:25:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f3c066f90 msgr2=0x7f0f3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f3c066f90 0x7f0f3c069450 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0f500ab000 tx=0x7f0f500b1040 comp rx=0 tx=0).stop 2024-04-25T10:25:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c1e96c0 msgr2=0x7f0f5c1e9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:38.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c1e96c0 0x7f0f5c1e9b40 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f0f440057f0 tx=0x7f0f5400fc90 comp rx=0 tx=0).stop 2024-04-25T10:25:38.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 shutdown_connections 2024-04-25T10:25:38.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0f3c066f90 0x7f0f3c069450 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c1e96c0 0x7f0f5c1e9b40 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f5c1576f0 0x7f0f5c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 --2- 172.21.15.87:0/927915968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f44003a10 0x7f0f5c0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.095+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 >> 172.21.15.87:0/927915968 conn(0x7f0f5c066b70 msgr2=0x7f0f5c1552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.099+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 shutdown_connections 2024-04-25T10:25:38.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.099+0000 7f0f64600640 1 -- 172.21.15.87:0/927915968 wait complete. 2024-04-25T10:25:38.116 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:25:38.116 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:25:38.117 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:25:38.117 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:38.117 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:25:38.117 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:25:38.117 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:25:38.117 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:38.117 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:25:38.117 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:25:38.118 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:25:38.118 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:38.118 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:25:38.118 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:25:38.118 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:25:38.118 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:38.130 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:25:38.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:25:38.131 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:25:38.131 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:25:38.135 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:25:38.136 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:25:38.137 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:25:38.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:25:38.175 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:25:38.176 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:38.177 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:25:38.177 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:38.180 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:25:38.180 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:38.182 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:25:38.182 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:38.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:25:38.261 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:25:38.262 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:25:38.270 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:25:38.270 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:25:38.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 Processor -- start 2024-04-25T10:25:38.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 -- start start 2024-04-25T10:25:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d981550b0 0x7f6d980cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6d980cfe70 0x7f6d9815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d9815b740 0x7f6d9806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d98071050 con 0x7f6d980cfe70 2024-04-25T10:25:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d980712f0 con 0x7f6d981550b0 2024-04-25T10:25:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d9ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980715d0 con 0x7f6d9815b740 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d8f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6d980cfe70 0x7f6d9815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d8f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6d980cfe70 0x7f6d9815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52862/0 (socket says 172.21.15.87:52862) 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.327+0000 7f6d8f400640 1 -- 172.21.15.87:0/2054338539 learned_addr learned my addr 172.21.15.87:0/2054338539 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ca00640 1 --2- 172.21.15.87:0/2054338539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d9815b740 0x7f6d9806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d97e00640 1 --2- 172.21.15.87:0/2054338539 >> v2:172.21.15.169:3300/0 conn(0x7f6d981550b0 0x7f6d980cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d8f400640 1 -- 172.21.15.87:0/2054338539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d9815b740 msgr2=0x7f6d9806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d8f400640 1 --2- 172.21.15.87:0/2054338539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d9815b740 0x7f6d9806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d8f400640 1 -- 172.21.15.87:0/2054338539 >> v2:172.21.15.169:3300/0 conn(0x7f6d981550b0 msgr2=0x7f6d980cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d8f400640 1 --2- 172.21.15.87:0/2054338539 >> v2:172.21.15.169:3300/0 conn(0x7f6d981550b0 0x7f6d980cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d8f400640 1 -- 172.21.15.87:0/2054338539 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d980718b0 con 0x7f6d980cfe70 2024-04-25T10:25:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d8f400640 1 --2- 172.21.15.87:0/2054338539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d980cfe70 0x7f6d9815b200 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009870 tx=0x7f6d8001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695a7b954166ac61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d97400640 1 -- 172.21.15.87:0/2054338539 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800b0070 con 0x7f6d980cfe70 2024-04-25T10:25:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d97400640 1 -- 172.21.15.87:0/2054338539 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d800ab600 con 0x7f6d980cfe70 2024-04-25T10:25:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d97400640 1 -- 172.21.15.87:0/2054338539 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800ab9d0 con 0x7f6d980cfe70 2024-04-25T10:25:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ea00640 1 -- 172.21.15.87:0/2054338539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d980cfe70 msgr2=0x7f6d9815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/2054338539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d980cfe70 0x7f6d9815b200 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009870 tx=0x7f6d8001c260 comp rx=0 tx=0).stop 2024-04-25T10:25:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ea00640 1 -- 172.21.15.87:0/2054338539 shutdown_connections 2024-04-25T10:25:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/2054338539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d980cfe70 0x7f6d9815b200 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/2054338539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d9815b740 0x7f6d9806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/2054338539 >> v2:172.21.15.169:3300/0 conn(0x7f6d981550b0 0x7f6d980cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.331+0000 7f6d9ea00640 1 -- 172.21.15.87:0/2054338539 >> 172.21.15.87:0/2054338539 conn(0x7f6d98066b70 msgr2=0x7f6d98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- 172.21.15.87:0/2054338539 shutdown_connections 2024-04-25T10:25:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- 172.21.15.87:0/2054338539 wait complete. 2024-04-25T10:25:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 Processor -- start 2024-04-25T10:25:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- start start 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d980cfe70 0x7f6d9814fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d981550b0 0x7f6d98147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9815b740 0x7f6d980cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980b3260 con 0x7f6d981550b0 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9815bd60 con 0x7f6d9815b740 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980b33e0 con 0x7f6d980cfe70 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d981550b0 0x7f6d98147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d981550b0 0x7f6d98147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52874/0 (socket says 172.21.15.87:52874) 2024-04-25T10:25:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 -- 172.21.15.87:0/793563025 learned_addr learned my addr 172.21.15.87:0/793563025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d97e00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d980cfe70 0x7f6d9814fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ca00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9815b740 0x7f6d980cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 -- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d980cfe70 msgr2=0x7f6d9814fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d980cfe70 0x7f6d9814fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 -- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9815b740 msgr2=0x7f6d980cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9815b740 0x7f6d980cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 -- 172.21.15.87:0/793563025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98148290 con 0x7f6d981550b0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d8f400640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d981550b0 0x7f6d98147d50 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f6d800ae040 tx=0x7f6d80019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d94c00640 1 -- 172.21.15.87:0/793563025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800ba070 con 0x7f6d981550b0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d94c00640 1 -- 172.21.15.87:0/793563025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d800b4420 con 0x7f6d981550b0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d94c00640 1 -- 172.21.15.87:0/793563025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800b0040 con 0x7f6d981550b0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d980cc890 con 0x7f6d981550b0 2024-04-25T10:25:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.335+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d980ccb30 con 0x7f6d981550b0 2024-04-25T10:25:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.339+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9804ef30 con 0x7f6d981550b0 2024-04-25T10:25:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.339+0000 7f6d94c00640 1 -- 172.21.15.87:0/793563025 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d800bd070 con 0x7f6d981550b0 2024-04-25T10:25:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.339+0000 7f6d94c00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d74066f60 0x7f6d74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.339+0000 7f6d94c00640 1 -- 172.21.15.87:0/793563025 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d8011de60 con 0x7f6d981550b0 2024-04-25T10:25:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.339+0000 7f6d97e00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d74066f60 0x7f6d74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.339+0000 7f6d97e00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d74066f60 0x7f6d74069420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6d981585b0 tx=0x7f6d88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:38.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.343+0000 7f6d94c00640 1 -- 172.21.15.87:0/793563025 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d80016110 con 0x7f6d981550b0 2024-04-25T10:25:38.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:38.511+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6d980b2c60 con 0x7f6d981550b0 2024-04-25T10:25:39.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.083+0000 7f6d94c00640 1 -- 172.21.15.87:0/793563025 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v170) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6d800ef160 con 0x7f6d981550b0 2024-04-25T10:25:39.084 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:25:39.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d74066f60 msgr2=0x7f6d74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d74066f60 0x7f6d74069420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6d981585b0 tx=0x7f6d88008040 comp rx=0 tx=0).stop 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d981550b0 msgr2=0x7f6d98147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d981550b0 0x7f6d98147d50 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f6d800ae040 tx=0x7f6d80019000 comp rx=0 tx=0).stop 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 shutdown_connections 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d74066f60 0x7f6d74069420 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9815b740 0x7f6d980cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d981550b0 0x7f6d98147d50 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 --2- 172.21.15.87:0/793563025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d980cfe70 0x7f6d9814fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 >> 172.21.15.87:0/793563025 conn(0x7f6d98066b70 msgr2=0x7f6d980b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 shutdown_connections 2024-04-25T10:25:39.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.087+0000 7f6d9ea00640 1 -- 172.21.15.87:0/793563025 wait complete. 2024-04-25T10:25:39.114 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:25:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 Processor -- start 2024-04-25T10:25:39.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 -- start start 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc1fc15bc40 0x7fc1fc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cda10 0x7fc1fc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc1fc0cf930 0x7fc1fc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b2770 con 0x7fc1fc0cf930 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc06c670 con 0x7fc1fc15bc40 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc203c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc06c950 con 0x7fc1fc0cda10 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc1fc0cf930 0x7fc1fc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc1fc0cf930 0x7fc1fc0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52886/0 (socket says 172.21.15.87:52886) 2024-04-25T10:25:39.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 -- 172.21.15.87:0/4195366385 learned_addr learned my addr 172.21.15.87:0/4195366385 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc1f3e00640 1 --2- 172.21.15.87:0/4195366385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cda10 0x7fc1fc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201200640 1 --2- 172.21.15.87:0/4195366385 >> v2:172.21.15.169:3300/0 conn(0x7fc1fc15bc40 0x7fc1fc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 -- 172.21.15.87:0/4195366385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cda10 msgr2=0x7fc1fc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:39.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 --2- 172.21.15.87:0/4195366385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cda10 0x7fc1fc0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 -- 172.21.15.87:0/4195366385 >> v2:172.21.15.169:3300/0 conn(0x7fc1fc15bc40 msgr2=0x7fc1fc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 --2- 172.21.15.87:0/4195366385 >> v2:172.21.15.169:3300/0 conn(0x7fc1fc15bc40 0x7fc1fc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201c00640 1 -- 172.21.15.87:0/4195366385 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc06cc30 con 0x7fc1fc0cf930 2024-04-25T10:25:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc1f3e00640 1 --2- 172.21.15.87:0/4195366385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cda10 0x7fc1fc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:39.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.255+0000 7fc201200640 1 --2- 172.21.15.87:0/4195366385 >> v2:172.21.15.169:3300/0 conn(0x7fc1fc15bc40 0x7fc1fc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:39.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc201c00640 1 --2- 172.21.15.87:0/4195366385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc0cf930 0x7fc1fc0cfde0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0d0800 tx=0x7fc1f401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567d08e27bb1d849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc1fbe00640 1 -- 172.21.15.87:0/4195366385 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40a5670 con 0x7fc1fc0cf930 2024-04-25T10:25:39.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc1fbe00640 1 -- 172.21.15.87:0/4195366385 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40a5810 con 0x7fc1fc0cf930 2024-04-25T10:25:39.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc1fbe00640 1 -- 172.21.15.87:0/4195366385 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40a5be0 con 0x7fc1fc0cf930 2024-04-25T10:25:39.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 -- 172.21.15.87:0/4195366385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc0cf930 msgr2=0x7fc1fc0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 --2- 172.21.15.87:0/4195366385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc0cf930 0x7fc1fc0cfde0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0d0800 tx=0x7fc1f401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 -- 172.21.15.87:0/4195366385 shutdown_connections 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 --2- 172.21.15.87:0/4195366385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc0cf930 0x7fc1fc0cfde0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 --2- 172.21.15.87:0/4195366385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cda10 0x7fc1fc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 --2- 172.21.15.87:0/4195366385 >> v2:172.21.15.169:3300/0 conn(0x7fc1fc15bc40 0x7fc1fc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 -- 172.21.15.87:0/4195366385 >> 172.21.15.87:0/4195366385 conn(0x7fc1fc066b70 msgr2=0x7fc1fc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 -- 172.21.15.87:0/4195366385 shutdown_connections 2024-04-25T10:25:39.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.259+0000 7fc203c00640 1 -- 172.21.15.87:0/4195366385 wait complete. 2024-04-25T10:25:39.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 Processor -- start 2024-04-25T10:25:39.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 -- start start 2024-04-25T10:25:39.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1fc0cda10 0x7fc1fc14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cf930 0x7fc1fc14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc15bc40 0x7fc1fc14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc160230 con 0x7fc1fc15bc40 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1600b0 con 0x7fc1fc0cda10 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1603b0 con 0x7fc1fc0cf930 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc15bc40 0x7fc1fc14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc15bc40 0x7fc1fc14b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52892/0 (socket says 172.21.15.87:52892) 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 -- 172.21.15.87:0/4030441500 learned_addr learned my addr 172.21.15.87:0/4030441500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201200640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1fc0cda10 0x7fc1fc14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc1f3e00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cf930 0x7fc1fc14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 -- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cf930 msgr2=0x7fc1fc14b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cf930 0x7fc1fc14b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 -- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1fc0cda10 msgr2=0x7fc1fc14e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1fc0cda10 0x7fc1fc14e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 -- 172.21.15.87:0/4030441500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc151e10 con 0x7fc1fc15bc40 2024-04-25T10:25:39.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc1f3e00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cf930 0x7fc1fc14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:39.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201200640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1fc0cda10 0x7fc1fc14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:39.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc201c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc15bc40 0x7fc1fc14b670 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fc1f401b0a0 tx=0x7fc1f4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc1f9600640 1 -- 172.21.15.87:0/4030441500 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40b8470 con 0x7fc1fc15bc40 2024-04-25T10:25:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc1f9600640 1 -- 172.21.15.87:0/4030441500 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40b8610 con 0x7fc1fc15bc40 2024-04-25T10:25:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc1f9600640 1 -- 172.21.15.87:0/4030441500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40b8a00 con 0x7fc1fc15bc40 2024-04-25T10:25:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc1520f0 con 0x7fc1fc15bc40 2024-04-25T10:25:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc1526b0 con 0x7fc1fc15bc40 2024-04-25T10:25:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.263+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc04ef30 con 0x7fc1fc15bc40 2024-04-25T10:25:39.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.267+0000 7fc1f9600640 1 -- 172.21.15.87:0/4030441500 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1f40c0020 con 0x7fc1fc15bc40 2024-04-25T10:25:39.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.267+0000 7fc1f9600640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc1d8066f60 0x7fc1d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.267+0000 7fc1f9600640 1 -- 172.21.15.87:0/4030441500 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1f411f6d0 con 0x7fc1fc15bc40 2024-04-25T10:25:39.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.267+0000 7fc201200640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc1d8066f60 0x7fc1d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.267+0000 7fc201200640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc1d8066f60 0x7fc1d8069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc15f4e0 tx=0x7fc1ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.271+0000 7fc1f9600640 1 -- 172.21.15.87:0/4030441500 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1f40c7020 con 0x7fc1fc15bc40 2024-04-25T10:25:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.479+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc1fc0af1c0 con 0x7fc1fc15bc40 2024-04-25T10:25:39.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc1f9600640 1 -- 172.21.15.87:0/4030441500 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc1f40165e0 con 0x7fc1fc15bc40 2024-04-25T10:25:39.482 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:39.482 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:25:39.482 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:25:39.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.482 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:25:39.482 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:25:39.483 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.483 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.483 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:25:39.483 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:25:39.483 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.483 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:25:39.484 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.485 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:25:39.486 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:25:39.486 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.486 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.487 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:25:39.487 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:25:39.487 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.487 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.487 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:25:39.487 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.488 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.490 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.490 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.490 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:25:39.491 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:25:39.491 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.491 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.491 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.492 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.494 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.495 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.495 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:25:39.495 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:25:39.495 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.495 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.496 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.496 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.496 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:25:39.497 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:25:39.497 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.497 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:39.497 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:25:39.497 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.497 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.498 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:25:39.498 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:25:39.498 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.498 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.498 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:25:39.499 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:25:39.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.499 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:25:39.499 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:25:39.500 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:25:39.503 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:25:39.504 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:25:39.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:39.504 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:25:39.504 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc1d8066f60 msgr2=0x7fc1d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc1d8066f60 0x7fc1d8069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc15f4e0 tx=0x7fc1ec008040 comp rx=0 tx=0).stop 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc15bc40 msgr2=0x7fc1fc14b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc15bc40 0x7fc1fc14b670 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fc1f401b0a0 tx=0x7fc1f4019790 comp rx=0 tx=0).stop 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 shutdown_connections 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc1d8066f60 0x7fc1d8069420 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1fc15bc40 0x7fc1fc14b670 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1fc0cf930 0x7fc1fc14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 --2- 172.21.15.87:0/4030441500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1fc0cda10 0x7fc1fc14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 >> 172.21.15.87:0/4030441500 conn(0x7fc1fc066b70 msgr2=0x7fc1fc0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:39.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 shutdown_connections 2024-04-25T10:25:39.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.483+0000 7fc203c00640 1 -- 172.21.15.87:0/4030441500 wait complete. 2024-04-25T10:25:39.507 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:39.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.643+0000 7fc76b200640 1 Processor -- start 2024-04-25T10:25:39.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.643+0000 7fc76b200640 1 -- start start 2024-04-25T10:25:39.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.643+0000 7fc76b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc76415bc40 0x7fc76406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.643+0000 7fc76b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7640cda10 0x7fc7640b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.643+0000 7fc76b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7640b2770 con 0x7fc7640cf930 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.643+0000 7fc76b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76406c670 con 0x7fc7640cda10 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.643+0000 7fc76b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76406c950 con 0x7fc76415bc40 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7640cda10 0x7fc7640b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7640cda10 0x7fc7640b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52904/0 (socket says 172.21.15.87:52904) 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 -- 172.21.15.87:0/208346441 learned_addr learned my addr 172.21.15.87:0/208346441 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc763e00640 1 --2- 172.21.15.87:0/208346441 >> v2:172.21.15.169:3300/0 conn(0x7fc76415bc40 0x7fc76406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 -- 172.21.15.87:0/208346441 >> v2:172.21.15.169:3300/0 conn(0x7fc76415bc40 msgr2=0x7fc76406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 --2- 172.21.15.87:0/208346441 >> v2:172.21.15.169:3300/0 conn(0x7fc76415bc40 0x7fc76406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 -- 172.21.15.87:0/208346441 >> v1:172.21.15.169:6789/0 conn(0x7fc7640cf930 legacy=0x7fc76415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 -- 172.21.15.87:0/208346441 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76406cc30 con 0x7fc7640cda10 2024-04-25T10:25:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc75b400640 1 --2- 172.21.15.87:0/208346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cda10 0x7fc7640b4610 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fc7640d0800 tx=0x7fc74c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2de03168fc4d0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc763400640 1 -- 172.21.15.87:0/208346441 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0ab430 con 0x7fc7640cda10 2024-04-25T10:25:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc763400640 1 -- 172.21.15.87:0/208346441 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c0b0070 con 0x7fc7640cda10 2024-04-25T10:25:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.647+0000 7fc763400640 1 -- 172.21.15.87:0/208346441 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0ab7f0 con 0x7fc7640cda10 2024-04-25T10:25:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- 172.21.15.87:0/208346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cda10 msgr2=0x7fc7640b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 --2- 172.21.15.87:0/208346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cda10 0x7fc7640b4610 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fc7640d0800 tx=0x7fc74c01c520 comp rx=0 tx=0).stop 2024-04-25T10:25:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- 172.21.15.87:0/208346441 shutdown_connections 2024-04-25T10:25:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 --2- 172.21.15.87:0/208346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cda10 0x7fc7640b4610 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 --2- 172.21.15.87:0/208346441 >> v2:172.21.15.169:3300/0 conn(0x7fc76415bc40 0x7fc76406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- 172.21.15.87:0/208346441 >> 172.21.15.87:0/208346441 conn(0x7fc764066b70 msgr2=0x7fc764066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:39.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- 172.21.15.87:0/208346441 shutdown_connections 2024-04-25T10:25:39.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- 172.21.15.87:0/208346441 wait complete. 2024-04-25T10:25:39.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 Processor -- start 2024-04-25T10:25:39.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- start start 2024-04-25T10:25:39.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7640cda10 0x7fc76414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cf930 0x7fc76415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76415bc40 0x7fc76413cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764168b50 con 0x7fc7640cf930 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7641689d0 con 0x7fc7640cda10 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc76b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc764168cd0 con 0x7fc76415bc40 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cf930 0x7fc76415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cf930 0x7fc76415e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52914/0 (socket says 172.21.15.87:52914) 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 -- 172.21.15.87:0/3988144056 learned_addr learned my addr 172.21.15.87:0/3988144056 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc763e00640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7640cda10 0x7fc76414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc769200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76415bc40 0x7fc76413cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 -- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76415bc40 msgr2=0x7fc76413cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76415bc40 0x7fc76413cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 -- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7640cda10 msgr2=0x7fc76414faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7640cda10 0x7fc76414faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 -- 172.21.15.87:0/3988144056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76414cf00 con 0x7fc7640cf930 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc769200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76415bc40 0x7fc76413cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.651+0000 7fc75b400640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cf930 0x7fc76415e5c0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fc74c01a150 tx=0x7fc74c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc760c00640 1 -- 172.21.15.87:0/3988144056 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0be5f0 con 0x7fc7640cf930 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc760c00640 1 -- 172.21.15.87:0/3988144056 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c0be7b0 con 0x7fc7640cf930 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76414d1e0 con 0x7fc7640cf930 2024-04-25T10:25:39.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc760c00640 1 -- 172.21.15.87:0/3988144056 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0beba0 con 0x7fc7640cf930 2024-04-25T10:25:39.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc764150030 con 0x7fc7640cf930 2024-04-25T10:25:39.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76404ef30 con 0x7fc7640cf930 2024-04-25T10:25:39.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc760c00640 1 -- 172.21.15.87:0/3988144056 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc74c0c7430 con 0x7fc7640cf930 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc760c00640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc740066f90 0x7fc740069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc760c00640 1 -- 172.21.15.87:0/3988144056 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc74c0a8030 con 0x7fc7640cf930 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc763e00640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc740066f90 0x7fc740069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.655+0000 7fc763e00640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc740066f90 0x7fc740069450 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc7640b0370 tx=0x7fc754008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.659+0000 7fc760c00640 1 -- 172.21.15.87:0/3988144056 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc74c016110 con 0x7fc7640cf930 2024-04-25T10:25:39.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.815+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc76406d960 con 0x7fc7640cf930 2024-04-25T10:25:39.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.815+0000 7fc760c00640 1 -- 172.21.15.87:0/3988144056 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+12889 (secure 0 0 0) 0x7fc74c0f0500 con 0x7fc7640cf930 2024-04-25T10:25:39.816 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:39.816 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":172,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:39.096860+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc740066f90 msgr2=0x7fc740069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc740066f90 0x7fc740069450 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc7640b0370 tx=0x7fc754008040 comp rx=0 tx=0).stop 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cf930 msgr2=0x7fc76415e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cf930 0x7fc76415e5c0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fc74c01a150 tx=0x7fc74c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 shutdown_connections 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc740066f90 0x7fc740069450 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76415bc40 0x7fc76413cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7640cf930 0x7fc76415e5c0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 --2- 172.21.15.87:0/3988144056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7640cda10 0x7fc76414faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 >> 172.21.15.87:0/3988144056 conn(0x7fc764066b70 msgr2=0x7fc7640b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 shutdown_connections 2024-04-25T10:25:39.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.819+0000 7fc76b200640 1 -- 172.21.15.87:0/3988144056 wait complete. 2024-04-25T10:25:39.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:39.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:39.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6567200640 1 Processor -- start 2024-04-25T10:25:39.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6567200640 1 -- start start 2024-04-25T10:25:39.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6567200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6560159540 0x7f65600ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6567200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f6560070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6567200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6560159920 con 0x7f65600aebd0 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6567200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656015c2f0 con 0x7f6560159540 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6567200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656015c5d0 con 0x7f65600721f0 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6560159540 0x7f65600ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6560159540 0x7f65600ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52922/0 (socket says 172.21.15.87:52922) 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6557400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f65600aebd0 0x7f6560071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55836/0 (socket says 172.21.15.87:55836) 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655fe00640 1 -- 172.21.15.87:0/2850090531 learned_addr learned my addr 172.21.15.87:0/2850090531 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6565200640 1 --2- 172.21.15.87:0/2850090531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f6560070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655fe00640 1 -- 172.21.15.87:0/2850090531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 msgr2=0x7f6560070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655fe00640 1 --2- 172.21.15.87:0/2850090531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f6560070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655fe00640 1 -- 172.21.15.87:0/2850090531 >> v1:172.21.15.87:6789/0 conn(0x7f65600aebd0 legacy=0x7f6560071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655fe00640 1 -- 172.21.15.87:0/2850090531 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656015c8b0 con 0x7f6560159540 2024-04-25T10:25:39.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f6565200640 1 --2- 172.21.15.87:0/2850090531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f6560070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:39.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.975+0000 7f655f400640 1 -- 172.21.15.87:0/2850090531 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059026538 0 0) 0x7f6560159920 con 0x7f65600aebd0 2024-04-25T10:25:39.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f655fe00640 1 --2- 172.21.15.87:0/2850090531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6560159540 0x7f65600ae690 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f6550009870 tx=0x7f655001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a7e94998ebbd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f655f400640 1 -- 172.21.15.87:0/2850090531 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65500a7070 con 0x7f6560159540 2024-04-25T10:25:39.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f655f400640 1 -- 172.21.15.87:0/2850090531 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65500b0070 con 0x7f6560159540 2024-04-25T10:25:39.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f655f400640 1 -- 172.21.15.87:0/2850090531 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65500ab890 con 0x7f6560159540 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 -- 172.21.15.87:0/2850090531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6560159540 msgr2=0x7f65600ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 --2- 172.21.15.87:0/2850090531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6560159540 0x7f65600ae690 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f6550009870 tx=0x7f655001c560 comp rx=0 tx=0).stop 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 -- 172.21.15.87:0/2850090531 shutdown_connections 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 --2- 172.21.15.87:0/2850090531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f6560070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 --2- 172.21.15.87:0/2850090531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6560159540 0x7f65600ae690 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 -- 172.21.15.87:0/2850090531 >> 172.21.15.87:0/2850090531 conn(0x7f65600669a0 msgr2=0x7f6560066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 -- 172.21.15.87:0/2850090531 shutdown_connections 2024-04-25T10:25:39.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.979+0000 7f6567200640 1 -- 172.21.15.87:0/2850090531 wait complete. 2024-04-25T10:25:39.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 Processor -- start 2024-04-25T10:25:39.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 -- start start 2024-04-25T10:25:39.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f65601510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65600aebd0 0x7f65600adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6560159540 0x7f6560150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6560071220 con 0x7f65600aebd0 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600710a0 con 0x7f6560159540 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65600713a0 con 0x7f65600721f0 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65600aebd0 0x7f65600adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65600aebd0 0x7f65600adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52928/0 (socket says 172.21.15.87:52928) 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 -- 172.21.15.87:0/3638808769 learned_addr learned my addr 172.21.15.87:0/3638808769 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 -- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 msgr2=0x7f65601510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f65601510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 -- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6560159540 msgr2=0x7f6560150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6565200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6560159540 0x7f6560150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6560159540 0x7f6560150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 -- 172.21.15.87:0/3638808769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560150e90 con 0x7f65600aebd0 2024-04-25T10:25:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6565200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6560159540 0x7f6560150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6557400640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65600aebd0 0x7f65600adec0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f65480011f0 tx=0x7f654800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f655cc00640 1 -- 172.21.15.87:0/3638808769 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548018070 con 0x7f65600aebd0 2024-04-25T10:25:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f655cc00640 1 -- 172.21.15.87:0/3638808769 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65480074e0 con 0x7f65600aebd0 2024-04-25T10:25:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65600ab9d0 con 0x7f65600aebd0 2024-04-25T10:25:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f655cc00640 1 -- 172.21.15.87:0/3638808769 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548004a10 con 0x7f65600aebd0 2024-04-25T10:25:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.983+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65600ad6c0 con 0x7f65600aebd0 2024-04-25T10:25:39.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.987+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656004ef30 con 0x7f65600aebd0 2024-04-25T10:25:39.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.987+0000 7f655cc00640 1 -- 172.21.15.87:0/3638808769 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f654801b070 con 0x7f65600aebd0 2024-04-25T10:25:39.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.987+0000 7f655cc00640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f653c066f60 0x7f653c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:39.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.987+0000 7f655cc00640 1 -- 172.21.15.87:0/3638808769 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f65480840a0 con 0x7f65600aebd0 2024-04-25T10:25:39.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.987+0000 7f655fe00640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f653c066f60 0x7f653c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.987+0000 7f655fe00640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f653c066f60 0x7f653c069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f65500af000 tx=0x7f6550008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:39.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:39.991+0000 7f655cc00640 1 -- 172.21.15.87:0/3638808769 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6548055360 con 0x7f65600aebd0 2024-04-25T10:25:40.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.151+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f656006d7f0 con 0x7f65600aebd0 2024-04-25T10:25:40.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.151+0000 7f655cc00640 1 -- 172.21.15.87:0/3638808769 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6548054c60 con 0x7f65600aebd0 2024-04-25T10:25:40.152 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:25:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f653c066f60 msgr2=0x7f653c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f653c066f60 0x7f653c069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f65500af000 tx=0x7f6550008040 comp rx=0 tx=0).stop 2024-04-25T10:25:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65600aebd0 msgr2=0x7f65600adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65600aebd0 0x7f65600adec0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f65480011f0 tx=0x7f654800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 shutdown_connections 2024-04-25T10:25:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f653c066f60 0x7f653c069420 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6560159540 0x7f6560150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65600aebd0 0x7f65600adec0 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 --2- 172.21.15.87:0/3638808769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65600721f0 0x7f65601510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 >> 172.21.15.87:0/3638808769 conn(0x7f65600669a0 msgr2=0x7f656015e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 shutdown_connections 2024-04-25T10:25:40.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.155+0000 7f6567200640 1 -- 172.21.15.87:0/3638808769 wait complete. 2024-04-25T10:25:40.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:25:40.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bfc00640 1 Processor -- start 2024-04-25T10:25:40.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bfc00640 1 -- start start 2024-04-25T10:25:40.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bfc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 0x7f58b81472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bfc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b806c600 con 0x7f58b8146ed0 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bfc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b8161bd0 con 0x7f58b8071e50 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bfc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8161980 con 0x7f58b80726b0 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bd200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f58b8146ed0 0x7f58b8071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55844/0 (socket says 172.21.15.87:55844) 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bd200640 1 -- 172.21.15.87:0/2007191147 learned_addr learned my addr 172.21.15.87:0/2007191147 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58bdc00640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 0x7f58b81472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239514493 0 0) 0x7f58b806c600 con 0x7f58b8146ed0 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c003530 con 0x7f58b8146ed0 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 984195212 0 0) 0x7f589c003530 con 0x7f58b8146ed0 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 msgr2=0x7f58b81472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:40.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 0x7f58b81472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 >> v1:172.21.15.169:6789/0 conn(0x7f58b8071e50 legacy=0x7f58b80ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b8161eb0 con 0x7f58b8146ed0 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f58a8001680 con 0x7f58b8146ed0 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 >> v1:172.21.15.87:6789/0 conn(0x7f58b8146ed0 legacy=0x7f58b8071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.311+0000 7f58b7400640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f589c0039f0 0x7f589c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c004810 0x7f589c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 0x7f589c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8161980 con 0x7f589c0039f0 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c00bab0 con 0x7f589c004810 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c00bcb0 con 0x7f58b80726b0 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bd200640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f589c0039f0 0x7f589c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bdc00640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 0x7f589c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7e00640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c004810 0x7f589c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bd200640 1 -- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 msgr2=0x7f589c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:40.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bd200640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b80726b0 0x7f589c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bd200640 1 -- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c004810 msgr2=0x7f589c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:40.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bd200640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c004810 0x7f589c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bd200640 1 -- 172.21.15.87:0/2007191147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c003530 con 0x7f589c0039f0 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58bd200640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f589c0039f0 0x7f589c003e10 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c210 tx=0x7f58a801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b87b50ce708df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8007d20 con 0x7f589c0039f0 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a80a9070 con 0x7f589c0039f0 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.315+0000 7f58b7400640 1 -- 172.21.15.87:0/2007191147 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8005d00 con 0x7f589c0039f0 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- 172.21.15.87:0/2007191147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f589c0039f0 msgr2=0x7f589c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 --2- 172.21.15.87:0/2007191147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f589c0039f0 0x7f589c003e10 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c210 tx=0x7f58a801cc10 comp rx=0 tx=0).stop 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bdc00640 1 -- 172.21.15.87:0/2007191147 reap_dead start 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- 172.21.15.87:0/2007191147 shutdown_connections 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- 172.21.15.87:0/2007191147 >> 172.21.15.87:0/2007191147 conn(0x7f58b80669a0 msgr2=0x7f58b8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- 172.21.15.87:0/2007191147 shutdown_connections 2024-04-25T10:25:40.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- 172.21.15.87:0/2007191147 wait complete. 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 Processor -- start 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- start start 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c0039f0 0x7f58b815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58b815cb80 0x7f58b815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b815fa50 0x7f58b8162990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80ae8e0 con 0x7f58b815cb80 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80ae760 con 0x7f589c0039f0 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bfc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80aea60 con 0x7f58b815fa50 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58b815cb80 0x7f58b815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58b815cb80 0x7f58b815f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52946/0 (socket says 172.21.15.87:52946) 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 -- 172.21.15.87:0/3774156969 learned_addr learned my addr 172.21.15.87:0/3774156969 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bd200640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c0039f0 0x7f58b815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bdc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b815fa50 0x7f58b8162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 -- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b815fa50 msgr2=0x7f58b8162990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b815fa50 0x7f58b8162990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 -- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c0039f0 msgr2=0x7f58b815d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c0039f0 0x7f58b815d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 -- 172.21.15.87:0/3774156969 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b81611d0 con 0x7f58b815cb80 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bd200640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c0039f0 0x7f58b815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58bdc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b815fa50 0x7f58b8162990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.319+0000 7f58b7e00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58b815cb80 0x7f58b815f510 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f58a400f5f0 tx=0x7f58a400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58b4c00640 1 -- 172.21.15.87:0/3774156969 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40052e0 con 0x7f58b815cb80 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58b4c00640 1 -- 172.21.15.87:0/3774156969 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4018070 con 0x7f58b815cb80 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b81614b0 con 0x7f58b815cb80 2024-04-25T10:25:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58b4c00640 1 -- 172.21.15.87:0/3774156969 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4003820 con 0x7f58b815cb80 2024-04-25T10:25:40.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b815d000 con 0x7f58b815cb80 2024-04-25T10:25:40.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b815ee20 con 0x7f58b815cb80 2024-04-25T10:25:40.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58b4c00640 1 -- 172.21.15.87:0/3774156969 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58a40039c0 con 0x7f58b815cb80 2024-04-25T10:25:40.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58b4c00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5898066f90 0x7f5898069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58b4c00640 1 -- 172.21.15.87:0/3774156969 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f58a4084ab0 con 0x7f58b815cb80 2024-04-25T10:25:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.323+0000 7f58bd200640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5898066f90 0x7f5898069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.327+0000 7f58bd200640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5898066f90 0x7f5898069450 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f58a80ab000 tx=0x7f58a80a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:40.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.327+0000 7f58b4c00640 1 -- 172.21.15.87:0/3774156969 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58a4055e00 con 0x7f58b815cb80 2024-04-25T10:25:40.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.483+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f58b804ef30 con 0x7f58b815cb80 2024-04-25T10:25:40.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58b4c00640 1 -- 172.21.15.87:0/3774156969 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v172) v1 ==== 84+0+65 (secure 0 0 0) 0x7f58a4005480 con 0x7f58b815cb80 2024-04-25T10:25:40.485 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:40.486 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:25:40.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:25:40.486 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:25:40.486 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:25:40.486 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:25:40.486 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:25:40.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5898066f90 msgr2=0x7f5898069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5898066f90 0x7f5898069450 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f58a80ab000 tx=0x7f58a80a7070 comp rx=0 tx=0).stop 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58b815cb80 msgr2=0x7f58b815f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58b815cb80 0x7f58b815f510 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f58a400f5f0 tx=0x7f58a400fab0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 shutdown_connections 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5898066f90 0x7f5898069450 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58b815fa50 0x7f58b8162990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58b815cb80 0x7f58b815f510 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 --2- 172.21.15.87:0/3774156969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f589c0039f0 0x7f58b815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 >> 172.21.15.87:0/3774156969 conn(0x7f58b80669a0 msgr2=0x7f58b80b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 shutdown_connections 2024-04-25T10:25:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.487+0000 7f58bfc00640 1 -- 172.21.15.87:0/3774156969 wait complete. 2024-04-25T10:25:40.506 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:25:40.506 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:25:40.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f9a00640 1 Processor -- start 2024-04-25T10:25:40.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f9a00640 1 -- start start 2024-04-25T10:25:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f406eef0 0x7fa4f40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f9a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa4f40afc20 0x7fa4f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f9a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f40b3f00 con 0x7fa4f415dca0 2024-04-25T10:25:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f9a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f40b5b40 con 0x7fa4f40afc20 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40b5350 con 0x7fa4f406eef0 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa4f415dca0 0x7fa4f40b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55846/0 (socket says 172.21.15.87:55846) 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f2a00640 1 -- 172.21.15.87:0/2494581472 learned_addr learned my addr 172.21.15.87:0/2494581472 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f2000640 1 --2- 172.21.15.87:0/2494581472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f406eef0 0x7fa4f40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f3400640 1 --2- 172.21.15.87:0/2494581472 >> v2:172.21.15.169:3300/0 conn(0x7fa4f40afc20 0x7fa4f406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f1600640 1 -- 172.21.15.87:0/2494581472 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719731794 0 0) 0x7fa4f40b3f00 con 0x7fa4f415dca0 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f1600640 1 -- 172.21.15.87:0/2494581472 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d8003530 con 0x7fa4f415dca0 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f2000640 1 -- 172.21.15.87:0/2494581472 >> v2:172.21.15.169:3300/0 conn(0x7fa4f40afc20 msgr2=0x7fa4f406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f2000640 1 --2- 172.21.15.87:0/2494581472 >> v2:172.21.15.169:3300/0 conn(0x7fa4f40afc20 0x7fa4f406d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f2000640 1 -- 172.21.15.87:0/2494581472 >> v1:172.21.15.87:6789/0 conn(0x7fa4f415dca0 legacy=0x7fa4f40b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.647+0000 7fa4f2000640 1 -- 172.21.15.87:0/2494581472 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40720f0 con 0x7fa4f406eef0 2024-04-25T10:25:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.651+0000 7fa4f2000640 1 --2- 172.21.15.87:0/2494581472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f406eef0 0x7fa4f40af6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00097c0 tx=0x7fa4e001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ed9b11eb09b67c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:40.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.651+0000 7fa4f1600640 1 -- 172.21.15.87:0/2494581472 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b0070 con 0x7fa4f406eef0 2024-04-25T10:25:40.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.651+0000 7fa4f1600640 1 -- 172.21.15.87:0/2494581472 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e00ab600 con 0x7fa4f406eef0 2024-04-25T10:25:40.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.651+0000 7fa4f1600640 1 -- 172.21.15.87:0/2494581472 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00ab9d0 con 0x7fa4f406eef0 2024-04-25T10:25:40.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- 172.21.15.87:0/2494581472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f406eef0 msgr2=0x7fa4f40af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/2494581472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f406eef0 0x7fa4f40af6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00097c0 tx=0x7fa4e001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- 172.21.15.87:0/2494581472 shutdown_connections 2024-04-25T10:25:40.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/2494581472 >> v2:172.21.15.169:3300/0 conn(0x7fa4f40afc20 0x7fa4f406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/2494581472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f406eef0 0x7fa4f40af6e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- 172.21.15.87:0/2494581472 >> 172.21.15.87:0/2494581472 conn(0x7fa4f40b24f0 msgr2=0x7fa4f40b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:40.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- 172.21.15.87:0/2494581472 shutdown_connections 2024-04-25T10:25:40.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- 172.21.15.87:0/2494581472 wait complete. 2024-04-25T10:25:40.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 Processor -- start 2024-04-25T10:25:40.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- start start 2024-04-25T10:25:40.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4f406eef0 0x7fa4f415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f40afc20 0x7fa4f414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4f415dca0 0x7fa4f414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f406e350 con 0x7fa4f406eef0 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f406e1d0 con 0x7fa4f415dca0 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f406e4d0 con 0x7fa4f40afc20 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4f406eef0 0x7fa4f415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4f406eef0 0x7fa4f415cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52950/0 (socket says 172.21.15.87:52950) 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 -- 172.21.15.87:0/3394445198 learned_addr learned my addr 172.21.15.87:0/3394445198 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2000640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f40afc20 0x7fa4f414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 -- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f40afc20 msgr2=0x7fa4f414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f3400640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4f415dca0 0x7fa4f414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f40afc20 0x7fa4f414dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 -- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4f415dca0 msgr2=0x7fa4f414e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4f415dca0 0x7fa4f414e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 -- 172.21.15.87:0/3394445198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40cfb20 con 0x7fa4f406eef0 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2000640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f40afc20 0x7fa4f414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f3400640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4f415dca0 0x7fa4f414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.655+0000 7fa4f2a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4f406eef0 0x7fa4f415cd80 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400caa0 tx=0x7fa4e400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:40.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4eaa00640 1 -- 172.21.15.87:0/3394445198 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4017070 con 0x7fa4f406eef0 2024-04-25T10:25:40.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4eaa00640 1 -- 172.21.15.87:0/3394445198 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e400ac00 con 0x7fa4f406eef0 2024-04-25T10:25:40.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4eaa00640 1 -- 172.21.15.87:0/3394445198 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4012400 con 0x7fa4f406eef0 2024-04-25T10:25:40.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f40cecc0 con 0x7fa4f406eef0 2024-04-25T10:25:40.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f40cf560 con 0x7fa4f406eef0 2024-04-25T10:25:40.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f404ef30 con 0x7fa4f406eef0 2024-04-25T10:25:40.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4eaa00640 1 -- 172.21.15.87:0/3394445198 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4e401a070 con 0x7fa4f406eef0 2024-04-25T10:25:40.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4eaa00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4d4066f60 0x7fa4d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:40.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.659+0000 7fa4eaa00640 1 -- 172.21.15.87:0/3394445198 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa4e4084aa0 con 0x7fa4f406eef0 2024-04-25T10:25:40.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.663+0000 7fa4f2000640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4d4066f60 0x7fa4d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:40.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.663+0000 7fa4f2000640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4d4066f60 0x7fa4d4069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ad000 tx=0x7fa4e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:40.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.667+0000 7fa4eaa00640 1 -- 172.21.15.87:0/3394445198 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4e4055df0 con 0x7fa4f406eef0 2024-04-25T10:25:40.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:40.819+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa4f415ee00 con 0x7fa4f406eef0 2024-04-25T10:25:41.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4eaa00640 1 -- 172.21.15.87:0/3394445198 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v173) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa4e40556f0 con 0x7fa4f406eef0 2024-04-25T10:25:41.098 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:25:41.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4d4066f60 msgr2=0x7fa4d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:41.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4d4066f60 0x7fa4d4069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ad000 tx=0x7fa4e0008040 comp rx=0 tx=0).stop 2024-04-25T10:25:41.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4f406eef0 msgr2=0x7fa4f415cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:41.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4f406eef0 0x7fa4f415cd80 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400caa0 tx=0x7fa4e400cf60 comp rx=0 tx=0).stop 2024-04-25T10:25:41.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 shutdown_connections 2024-04-25T10:25:41.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa4d4066f60 0x7fa4d4069420 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4f415dca0 0x7fa4f414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4f40afc20 0x7fa4f414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 --2- 172.21.15.87:0/3394445198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4f406eef0 0x7fa4f415cd80 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 >> 172.21.15.87:0/3394445198 conn(0x7fa4f40b24f0 msgr2=0x7fa4f40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:41.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 shutdown_connections 2024-04-25T10:25:41.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.099+0000 7fa4f9a00640 1 -- 172.21.15.87:0/3394445198 wait complete. 2024-04-25T10:25:41.124 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:25:41.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200fc00640 1 Processor -- start 2024-04-25T10:25:41.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200fc00640 1 -- start start 2024-04-25T10:25:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20080b1600 0x7f20080cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200fc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080af850 con 0x7f20080b08f0 2024-04-25T10:25:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20080b0460 con 0x7f20080b1600 2024-04-25T10:25:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080afb30 con 0x7f200815dfb0 2024-04-25T10:25:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200dc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20080b1600 0x7f20080cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:41.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200dc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20080b1600 0x7f20080cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52966/0 (socket says 172.21.15.87:52966) 2024-04-25T10:25:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200dc00640 1 -- 172.21.15.87:0/1302850796 learned_addr learned my addr 172.21.15.87:0/1302850796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f2007e00640 1 --1- 172.21.15.87:0/1302850796 >> v1:172.21.15.87:6789/0 conn(0x7f20080b08f0 0x7f20080b0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55848/0 (socket says 172.21.15.87:55848) 2024-04-25T10:25:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200d200640 1 --2- 172.21.15.87:0/1302850796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200dc00640 1 -- 172.21.15.87:0/1302850796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 msgr2=0x7f20080af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200dc00640 1 --2- 172.21.15.87:0/1302850796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200dc00640 1 -- 172.21.15.87:0/1302850796 >> v1:172.21.15.87:6789/0 conn(0x7f20080b08f0 legacy=0x7f20080b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:41.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f200dc00640 1 -- 172.21.15.87:0/1302850796 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080afe10 con 0x7f20080b1600 2024-04-25T10:25:41.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.267+0000 7f2007400640 1 -- 172.21.15.87:0/1302850796 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467027032 0 0) 0x7f20080af850 con 0x7f20080b08f0 2024-04-25T10:25:41.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200dc00640 1 --2- 172.21.15.87:0/1302850796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080cfb80 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f2000009e50 tx=0x7f200001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab906b2cfe99bf2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:41.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f2007400640 1 -- 172.21.15.87:0/1302850796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20000b4070 con 0x7f20080b1600 2024-04-25T10:25:41.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f2007400640 1 -- 172.21.15.87:0/1302850796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20000af400 con 0x7f20080b1600 2024-04-25T10:25:41.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f2007400640 1 -- 172.21.15.87:0/1302850796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20000af7d0 con 0x7f20080b1600 2024-04-25T10:25:41.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 -- 172.21.15.87:0/1302850796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 msgr2=0x7f20080cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 --2- 172.21.15.87:0/1302850796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080cfb80 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f2000009e50 tx=0x7f200001f210 comp rx=0 tx=0).stop 2024-04-25T10:25:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 -- 172.21.15.87:0/1302850796 shutdown_connections 2024-04-25T10:25:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 --2- 172.21.15.87:0/1302850796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080cfb80 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 --2- 172.21.15.87:0/1302850796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 -- 172.21.15.87:0/1302850796 >> 172.21.15.87:0/1302850796 conn(0x7f20080b3580 msgr2=0x7f20080b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 -- 172.21.15.87:0/1302850796 shutdown_connections 2024-04-25T10:25:41.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.271+0000 7f200fc00640 1 -- 172.21.15.87:0/1302850796 wait complete. 2024-04-25T10:25:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 Processor -- start 2024-04-25T10:25:41.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 -- start start 2024-04-25T10:25:41.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20080b08f0 0x7f20081607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:41.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:41.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200806bfd0 con 0x7f20080b1600 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200806be50 con 0x7f20080b08f0 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080b1a90 con 0x7f200815dfb0 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52982/0 (socket says 172.21.15.87:52982) 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 -- 172.21.15.87:0/2468652026 learned_addr learned my addr 172.21.15.87:0/2468652026 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200d200640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20080b08f0 0x7f20081607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200dc00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 -- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 msgr2=0x7f20080c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 -- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20080b08f0 msgr2=0x7f20081607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20080b08f0 0x7f20081607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 -- 172.21.15.87:0/2468652026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080d01b0 con 0x7f20080b1600 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2007e00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080c5520 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40011f0 tx=0x7f1ff400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:41.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2004c00640 1 -- 172.21.15.87:0/2468652026 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4018070 con 0x7f20080b1600 2024-04-25T10:25:41.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2004c00640 1 -- 172.21.15.87:0/2468652026 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff40074e0 con 0x7f20080b1600 2024-04-25T10:25:41.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20080d2660 con 0x7f20080b1600 2024-04-25T10:25:41.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f2004c00640 1 -- 172.21.15.87:0/2468652026 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4004a10 con 0x7f20080b1600 2024-04-25T10:25:41.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.275+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20080c6370 con 0x7f20080b1600 2024-04-25T10:25:41.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.279+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200804ef30 con 0x7f20080b1600 2024-04-25T10:25:41.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.279+0000 7f2004c00640 1 -- 172.21.15.87:0/2468652026 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ff401b070 con 0x7f20080b1600 2024-04-25T10:25:41.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.279+0000 7f2004c00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fe8066f60 0x7f1fe8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:41.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.279+0000 7f2004c00640 1 -- 172.21.15.87:0/2468652026 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1ff4084180 con 0x7f20080b1600 2024-04-25T10:25:41.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.279+0000 7f200d200640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fe8066f60 0x7f1fe8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:41.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.279+0000 7f200d200640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fe8066f60 0x7f1fe8069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009ac0 tx=0x7f1ffc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:41.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.283+0000 7f2004c00640 1 -- 172.21.15.87:0/2468652026 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ff4054c80 con 0x7f20080b1600 2024-04-25T10:25:41.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:41.439+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f200806fcc0 con 0x7f20080b1600 2024-04-25T10:25:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.107+0000 7f2004c00640 1 -- 172.21.15.87:0/2468652026 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v174) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1ff4058d20 con 0x7f20080b1600 2024-04-25T10:25:42.108 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fe8066f60 msgr2=0x7f1fe8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fe8066f60 0x7f1fe8069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009ac0 tx=0x7f1ffc009190 comp rx=0 tx=0).stop 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 msgr2=0x7f20080c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080c5520 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40011f0 tx=0x7f1ff400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 shutdown_connections 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1fe8066f60 0x7f1fe8069420 secure :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009ac0 tx=0x7f1ffc009190 comp rx=0 tx=0).stop 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f200815dfb0 0x7f20080c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20080b1600 0x7f20080c5520 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 --2- 172.21.15.87:0/2468652026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20080b08f0 0x7f20081607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 >> 172.21.15.87:0/2468652026 conn(0x7f20080b3580 msgr2=0x7f20080b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:42.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 shutdown_connections 2024-04-25T10:25:42.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.111+0000 7f200fc00640 1 -- 172.21.15.87:0/2468652026 wait complete. 2024-04-25T10:25:42.150 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:25:42.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 Processor -- start 2024-04-25T10:25:42.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 -- start start 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e70147290 0x7f2e700d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e700adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e700ae020 0x7f2e700669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e700d1000 con 0x7f2e700ae020 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e701620f0 con 0x7f2e70147290 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e76000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700ae490 con 0x7f2e700d0ba0 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e700ae020 0x7f2e700669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e700ae020 0x7f2e700669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52996/0 (socket says 172.21.15.87:52996) 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 -- 172.21.15.87:0/3495155457 learned_addr learned my addr 172.21.15.87:0/3495155457 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e74c00640 1 --2- 172.21.15.87:0/3495155457 >> v2:172.21.15.169:3300/0 conn(0x7f2e70147290 0x7f2e700d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:42.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/3495155457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e700adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 -- 172.21.15.87:0/3495155457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 msgr2=0x7f2e700adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 --2- 172.21.15.87:0/3495155457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e700adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 -- 172.21.15.87:0/3495155457 >> v2:172.21.15.169:3300/0 conn(0x7f2e70147290 msgr2=0x7f2e700d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 --2- 172.21.15.87:0/3495155457 >> v2:172.21.15.169:3300/0 conn(0x7f2e70147290 0x7f2e700d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.291+0000 7f2e75600640 1 -- 172.21.15.87:0/3495155457 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e701617c0 con 0x7f2e700ae020 2024-04-25T10:25:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e75600640 1 --2- 172.21.15.87:0/3495155457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e700669a0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f2e7006bc70 tx=0x7f2e6801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fb10f25e42083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e6f400640 1 -- 172.21.15.87:0/3495155457 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680a64b0 con 0x7f2e700ae020 2024-04-25T10:25:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e6f400640 1 -- 172.21.15.87:0/3495155457 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680a6650 con 0x7f2e700ae020 2024-04-25T10:25:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e6f400640 1 -- 172.21.15.87:0/3495155457 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680a6a20 con 0x7f2e700ae020 2024-04-25T10:25:42.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 -- 172.21.15.87:0/3495155457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 msgr2=0x7f2e700669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:42.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 --2- 172.21.15.87:0/3495155457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e700669a0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f2e7006bc70 tx=0x7f2e6801c730 comp rx=0 tx=0).stop 2024-04-25T10:25:42.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 -- 172.21.15.87:0/3495155457 shutdown_connections 2024-04-25T10:25:42.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 --2- 172.21.15.87:0/3495155457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e700669a0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 --2- 172.21.15.87:0/3495155457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e700adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 --2- 172.21.15.87:0/3495155457 >> v2:172.21.15.169:3300/0 conn(0x7f2e70147290 0x7f2e700d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 -- 172.21.15.87:0/3495155457 >> 172.21.15.87:0/3495155457 conn(0x7f2e7006ac90 msgr2=0x7f2e7006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:42.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 -- 172.21.15.87:0/3495155457 shutdown_connections 2024-04-25T10:25:42.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 -- 172.21.15.87:0/3495155457 wait complete. 2024-04-25T10:25:42.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.295+0000 7f2e76000640 1 Processor -- start 2024-04-25T10:25:42.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 -- start start 2024-04-25T10:25:42.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e701665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:42.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e701605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:42.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e70147290 0x7f2e70164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:42.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70147670 con 0x7f2e700ae020 2024-04-25T10:25:42.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700ae6f0 con 0x7f2e70147290 2024-04-25T10:25:42.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701477f0 con 0x7f2e700d0ba0 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e701665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e701665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53000/0 (socket says 172.21.15.87:53000) 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 -- 172.21.15.87:0/4072820984 learned_addr learned my addr 172.21.15.87:0/4072820984 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e701605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e75600640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e70147290 0x7f2e70164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 -- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 msgr2=0x7f2e701605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e701605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 -- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e70147290 msgr2=0x7f2e70164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e70147290 0x7f2e70164830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 -- 172.21.15.87:0/4072820984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e70165500 con 0x7f2e700ae020 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e74c00640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e701665a0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400d800 tx=0x7f2e6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e6cc00640 1 -- 172.21.15.87:0/4072820984 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6400b660 con 0x7f2e700ae020 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e6cc00640 1 -- 172.21.15.87:0/4072820984 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e64019070 con 0x7f2e700ae020 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e701657e0 con 0x7f2e700ae020 2024-04-25T10:25:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e6cc00640 1 -- 172.21.15.87:0/4072820984 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6400ba40 con 0x7f2e700ae020 2024-04-25T10:25:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e70165da0 con 0x7f2e700ae020 2024-04-25T10:25:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.299+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e70075120 con 0x7f2e700ae020 2024-04-25T10:25:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.303+0000 7f2e6cc00640 1 -- 172.21.15.87:0/4072820984 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e6400bbe0 con 0x7f2e700ae020 2024-04-25T10:25:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.303+0000 7f2e6cc00640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e50066f60 0x7f2e50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.303+0000 7f2e6cc00640 1 -- 172.21.15.87:0/4072820984 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2e64085200 con 0x7f2e700ae020 2024-04-25T10:25:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.303+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e50066f60 0x7f2e50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:42.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.303+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e50066f60 0x7f2e50069420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2e70146b30 tx=0x7f2e5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:42.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.307+0000 7f2e6cc00640 1 -- 172.21.15.87:0/4072820984 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e64055b80 con 0x7f2e700ae020 2024-04-25T10:25:42.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:42.491+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2e7015c5b0 con 0x7f2e700ae020 2024-04-25T10:25:43.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.111+0000 7f2e6cc00640 1 -- 172.21.15.87:0/4072820984 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v124) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2e64059c20 con 0x7f2e700ae020 2024-04-25T10:25:43.111 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 26 and data pool 27 2024-04-25T10:25:43.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e50066f60 msgr2=0x7f2e50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e50066f60 0x7f2e50069420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2e70146b30 tx=0x7f2e5c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:43.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 msgr2=0x7f2e701665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e701665a0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400d800 tx=0x7f2e6400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 shutdown_connections 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e50066f60 0x7f2e50069420 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e70147290 0x7f2e70164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e700d0ba0 0x7f2e701605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 --2- 172.21.15.87:0/4072820984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e700ae020 0x7f2e701665a0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 >> 172.21.15.87:0/4072820984 conn(0x7f2e7006ac90 msgr2=0x7f2e700cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 shutdown_connections 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.115+0000 7f2e76000640 1 -- 172.21.15.87:0/4072820984 wait complete. 2024-04-25T10:25:43.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:43.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f121e200640 1 Processor -- start 2024-04-25T10:25:43.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f121e200640 1 -- start start 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f121e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f121815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f121e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f12180ae020 0x7f12180ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f121e200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12180ad840 con 0x7f12180cf930 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f121e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121815bb90 con 0x7f12180ae020 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f121e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121815e9f0 con 0x7f1218070c20 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1217400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f12180cf930 0x7f12180cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55854/0 (socket says 172.21.15.87:55854) 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1217400640 1 -- 172.21.15.87:0/3728301310 learned_addr learned my addr 172.21.15.87:0/3728301310 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1217e00640 1 --2- 172.21.15.87:0/3728301310 >> v2:172.21.15.87:3300/0 conn(0x7f12180ae020 0x7f12180ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f120ea00640 1 --2- 172.21.15.87:0/3728301310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f121815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1216a00640 1 -- 172.21.15.87:0/3728301310 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047211246 0 0) 0x7f12180ad840 con 0x7f12180cf930 2024-04-25T10:25:43.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1216a00640 1 -- 172.21.15.87:0/3728301310 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11fc003530 con 0x7f12180cf930 2024-04-25T10:25:43.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1217e00640 1 -- 172.21.15.87:0/3728301310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 msgr2=0x7f121815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1217e00640 1 --2- 172.21.15.87:0/3728301310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f121815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1217e00640 1 -- 172.21.15.87:0/3728301310 >> v1:172.21.15.87:6789/0 conn(0x7f12180cf930 legacy=0x7f12180cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f1217e00640 1 -- 172.21.15.87:0/3728301310 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121815ecd0 con 0x7f12180ae020 2024-04-25T10:25:43.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.299+0000 7f120ea00640 1 --2- 172.21.15.87:0/3728301310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f121815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:43.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f1217e00640 1 --2- 172.21.15.87:0/3728301310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180ae4b0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f1210009dc0 tx=0x7f121001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975ba90427fe8f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f1216a00640 1 -- 172.21.15.87:0/3728301310 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100b2070 con 0x7f12180ae020 2024-04-25T10:25:43.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f1216a00640 1 -- 172.21.15.87:0/3728301310 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12100a59e0 con 0x7f12180ae020 2024-04-25T10:25:43.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f1216a00640 1 -- 172.21.15.87:0/3728301310 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100a5db0 con 0x7f12180ae020 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 -- 172.21.15.87:0/3728301310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 msgr2=0x7f12180ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 --2- 172.21.15.87:0/3728301310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180ae4b0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f1210009dc0 tx=0x7f121001c920 comp rx=0 tx=0).stop 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 -- 172.21.15.87:0/3728301310 shutdown_connections 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 --2- 172.21.15.87:0/3728301310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180ae4b0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 --2- 172.21.15.87:0/3728301310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f121815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 -- 172.21.15.87:0/3728301310 >> 172.21.15.87:0/3728301310 conn(0x7f1218066b70 msgr2=0x7f1218066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 -- 172.21.15.87:0/3728301310 shutdown_connections 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.303+0000 7f121e200640 1 -- 172.21.15.87:0/3728301310 wait complete. 2024-04-25T10:25:43.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 Processor -- start 2024-04-25T10:25:43.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 -- start start 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f1218158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12180cf930 0x7f12180af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121816a8e0 con 0x7f12180ae020 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121815e6b0 con 0x7f12180cf930 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121816aa60 con 0x7f1218070c20 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53012/0 (socket says 172.21.15.87:53012) 2024-04-25T10:25:43.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 -- 172.21.15.87:0/2681422442 learned_addr learned my addr 172.21.15.87:0/2681422442 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f1217400640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f1218158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 -- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 msgr2=0x7f1218158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f1218158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 -- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12180cf930 msgr2=0x7f12180af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f1217e00640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12180cf930 0x7f12180af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12180cf930 0x7f12180af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 -- 172.21.15.87:0/2681422442 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218158650 con 0x7f12180ae020 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f1217400640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f1218158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f1217e00640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12180cf930 0x7f12180af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120ea00640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180b04f0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f120000e810 tx=0x7f120000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120fe00640 1 -- 172.21.15.87:0/2681422442 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200017070 con 0x7f12180ae020 2024-04-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120fe00640 1 -- 172.21.15.87:0/2681422442 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120000c8d0 con 0x7f12180ae020 2024-04-25T10:25:43.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12181551a0 con 0x7f12180ae020 2024-04-25T10:25:43.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f120fe00640 1 -- 172.21.15.87:0/2681422442 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120000f040 con 0x7f12180ae020 2024-04-25T10:25:43.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1218156240 con 0x7f12180ae020 2024-04-25T10:25:43.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.307+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121804ef30 con 0x7f12180ae020 2024-04-25T10:25:43.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.311+0000 7f120fe00640 1 -- 172.21.15.87:0/2681422442 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f120001a070 con 0x7f12180ae020 2024-04-25T10:25:43.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.311+0000 7f120fe00640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11f4066f60 0x7f11f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.311+0000 7f120fe00640 1 -- 172.21.15.87:0/2681422442 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1200084c60 con 0x7f12180ae020 2024-04-25T10:25:43.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.311+0000 7f1217400640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11f4066f60 0x7f11f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.311+0000 7f1217400640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11f4066f60 0x7f11f4069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1208005890 tx=0x7f1208009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.315+0000 7f120fe00640 1 -- 172.21.15.87:0/2681422442 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1200055c60 con 0x7f12180ae020 2024-04-25T10:25:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.471+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f121815ce00 con 0x7f12180ae020 2024-04-25T10:25:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.471+0000 7f120fe00640 1 -- 172.21.15.87:0/2681422442 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+16684 (secure 0 0 0) 0x7f1200055560 con 0x7f12180ae020 2024-04-25T10:25:43.472 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:43.473 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":175,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:42.497532+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:40.866786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:41.487045+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11f4066f60 msgr2=0x7f11f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11f4066f60 0x7f11f4069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1208005890 tx=0x7f1208009040 comp rx=0 tx=0).stop 2024-04-25T10:25:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 msgr2=0x7f12180b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180b04f0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f120000e810 tx=0x7f120000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 shutdown_connections 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f11f4066f60 0x7f11f4069420 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12180cf930 0x7f12180af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12180ae020 0x7f12180b04f0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 --2- 172.21.15.87:0/2681422442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1218070c20 0x7f1218158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 >> 172.21.15.87:0/2681422442 conn(0x7f1218066b70 msgr2=0x7f121815bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 shutdown_connections 2024-04-25T10:25:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.475+0000 7f121e200640 1 -- 172.21.15.87:0/2681422442 wait complete. 2024-04-25T10:25:43.494 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:43.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2be1c00640 1 Processor -- start 2024-04-25T10:25:43.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2be1c00640 1 -- start start 2024-04-25T10:25:43.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2be1c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2bdc1550b0 0x7f2bdc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2be1c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2bdc0cfe70 0x7f2bdc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2be1c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bdc071050 con 0x7f2bdc15b740 2024-04-25T10:25:43.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2be1c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bdc15d040 con 0x7f2bdc1550b0 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2be1c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bdc0712f0 con 0x7f2bdc0cfe70 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdaa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2bdc1550b0 0x7f2bdc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdaa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2bdc1550b0 0x7f2bdc0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53028/0 (socket says 172.21.15.87:53028) 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdaa00640 1 -- 172.21.15.87:0/2158217976 learned_addr learned my addr 172.21.15.87:0/2158217976 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdb400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2bdc15b740 0x7f2bdc06d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55866/0 (socket says 172.21.15.87:55866) 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bd2000640 1 --2- 172.21.15.87:0/2158217976 >> v2:172.21.15.169:3300/0 conn(0x7f2bdc0cfe70 0x7f2bdc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdaa00640 1 -- 172.21.15.87:0/2158217976 >> v2:172.21.15.169:3300/0 conn(0x7f2bdc0cfe70 msgr2=0x7f2bdc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdaa00640 1 --2- 172.21.15.87:0/2158217976 >> v2:172.21.15.169:3300/0 conn(0x7f2bdc0cfe70 0x7f2bdc15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdaa00640 1 -- 172.21.15.87:0/2158217976 >> v1:172.21.15.87:6789/0 conn(0x7f2bdc15b740 legacy=0x7f2bdc06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.631+0000 7f2bdaa00640 1 -- 172.21.15.87:0/2158217976 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0715d0 con 0x7f2bdc1550b0 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2bdaa00640 1 --2- 172.21.15.87:0/2158217976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc0cf930 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00d150 tx=0x7f2bcc01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f4ddcbb248ee19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2bda000640 1 -- 172.21.15.87:0/2158217976 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00b2d0 con 0x7f2bdc1550b0 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2bda000640 1 -- 172.21.15.87:0/2158217976 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc00b470 con 0x7f2bdc1550b0 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2bda000640 1 -- 172.21.15.87:0/2158217976 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0a7af0 con 0x7f2bdc1550b0 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 -- 172.21.15.87:0/2158217976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 msgr2=0x7f2bdc0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 --2- 172.21.15.87:0/2158217976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc0cf930 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00d150 tx=0x7f2bcc01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 -- 172.21.15.87:0/2158217976 shutdown_connections 2024-04-25T10:25:43.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 --2- 172.21.15.87:0/2158217976 >> v2:172.21.15.169:3300/0 conn(0x7f2bdc0cfe70 0x7f2bdc15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 --2- 172.21.15.87:0/2158217976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc0cf930 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 -- 172.21.15.87:0/2158217976 >> 172.21.15.87:0/2158217976 conn(0x7f2bdc066b70 msgr2=0x7f2bdc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 -- 172.21.15.87:0/2158217976 shutdown_connections 2024-04-25T10:25:43.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.635+0000 7f2be1c00640 1 -- 172.21.15.87:0/2158217976 wait complete. 2024-04-25T10:25:43.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 Processor -- start 2024-04-25T10:25:43.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 -- start start 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bdc0cfe70 0x7f2bdc14b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc1f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bdc15b740 0x7f2bdc1f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc15bcd0 con 0x7f2bdc1550b0 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc15bb50 con 0x7f2bdc15b740 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc15c030 con 0x7f2bdc0cfe70 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc1f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc1f9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53034/0 (socket says 172.21.15.87:53034) 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 -- 172.21.15.87:0/4017156478 learned_addr learned my addr 172.21.15.87:0/4017156478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bdb400640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bdc15b740 0x7f2bdc1f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 -- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bdc0cfe70 msgr2=0x7f2bdc14b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bdaa00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bdc0cfe70 0x7f2bdc14b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bdc0cfe70 0x7f2bdc14b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 -- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bdc15b740 msgr2=0x7f2bdc1f9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bdc15b740 0x7f2bdc1f9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 -- 172.21.15.87:0/4017156478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0cc890 con 0x7f2bdc1550b0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bdaa00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bdc0cfe70 0x7f2bdc14b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bdb400640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bdc15b740 0x7f2bdc1f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd2000640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc1f9970 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e960 tx=0x7f2bc400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd3400640 1 -- 172.21.15.87:0/4017156478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4017070 con 0x7f2bdc1550b0 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd3400640 1 -- 172.21.15.87:0/4017156478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc400ca00 con 0x7f2bdc1550b0 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2bd3400640 1 -- 172.21.15.87:0/4017156478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400cdb0 con 0x7f2bdc1550b0 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc0ccb70 con 0x7f2bdc1550b0 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.639+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc14c5c0 con 0x7f2bdc1550b0 2024-04-25T10:25:43.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.643+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bdc04ef30 con 0x7f2bdc1550b0 2024-04-25T10:25:43.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.643+0000 7f2bd3400640 1 -- 172.21.15.87:0/4017156478 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bc401a070 con 0x7f2bdc1550b0 2024-04-25T10:25:43.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.643+0000 7f2bd3400640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bb8066f60 0x7f2bb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.643+0000 7f2bd3400640 1 -- 172.21.15.87:0/4017156478 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bc4085540 con 0x7f2bdc1550b0 2024-04-25T10:25:43.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.643+0000 7f2bdaa00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bb8066f60 0x7f2bb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.643+0000 7f2bdaa00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bb8066f60 0x7f2bb8069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0b0000 tx=0x7f2bcc0a9070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.647+0000 7f2bd3400640 1 -- 172.21.15.87:0/4017156478 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bc4056420 con 0x7f2bdc1550b0 2024-04-25T10:25:43.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.831+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bdc0cda00 con 0x7f2bdc1550b0 2024-04-25T10:25:43.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.831+0000 7f2bd3400640 1 -- 172.21.15.87:0/4017156478 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+16684 (secure 0 0 0) 0x7f2bc4055d20 con 0x7f2bdc1550b0 2024-04-25T10:25:43.831 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:43.831 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":175,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:42.497532+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:40.866786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:41.487045+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:43.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bb8066f60 msgr2=0x7f2bb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bb8066f60 0x7f2bb8069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0b0000 tx=0x7f2bcc0a9070 comp rx=0 tx=0).stop 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 msgr2=0x7f2bdc1f9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc1f9970 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e960 tx=0x7f2bc400ee20 comp rx=0 tx=0).stop 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 shutdown_connections 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2bb8066f60 0x7f2bb8069420 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bdc15b740 0x7f2bdc1f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bdc1550b0 0x7f2bdc1f9970 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 --2- 172.21.15.87:0/4017156478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bdc0cfe70 0x7f2bdc14b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 >> 172.21.15.87:0/4017156478 conn(0x7f2bdc066b70 msgr2=0x7f2bdc0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 shutdown_connections 2024-04-25T10:25:43.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.835+0000 7f2be1c00640 1 -- 172.21.15.87:0/4017156478 wait complete. 2024-04-25T10:25:43.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:25:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f709d200640 1 Processor -- start 2024-04-25T10:25:43.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f709d200640 1 -- start start 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f709d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f70980670b0 0x7f709806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f709d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 0x7f70980ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f709d200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709815bef0 con 0x7f709815bb10 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f709d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709806e7e0 con 0x7f70980670b0 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f709d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709806ec00 con 0x7f709815f6b0 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f709815bb10 0x7f70980669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55878/0 (socket says 172.21.15.87:55878) 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 -- 172.21.15.87:0/628562062 learned_addr learned my addr 172.21.15.87:0/628562062 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7096800640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 0x7f70980ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095400640 1 --2- 172.21.15.87:0/628562062 >> v2:172.21.15.169:3300/0 conn(0x7f70980670b0 0x7f709806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971842267 0 0) 0x7f709815bef0 con 0x7f709815bb10 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f707c003530 con 0x7f709815bb10 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 632572779 0 0) 0x7f707c003530 con 0x7f709815bb10 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 msgr2=0x7f70980ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 0x7f70980ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 >> v2:172.21.15.169:3300/0 conn(0x7f70980670b0 msgr2=0x7f709806e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 --2- 172.21.15.87:0/628562062 >> v2:172.21.15.169:3300/0 conn(0x7f70980670b0 0x7f709806e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709815fc50 con 0x7f709815bb10 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f7088001680 con 0x7f709815bb10 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7096800640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 0x7f70980ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 >> v1:172.21.15.87:6789/0 conn(0x7f709815bb10 legacy=0x7f70980669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f707c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707c004810 0x7f707c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 0x7f707c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709806e7e0 con 0x7f707c0039c0 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709806ec00 con 0x7f707c004810 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c00b9d0 con 0x7f709815f6b0 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f707c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095400640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707c004810 0x7f707c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7096800640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 0x7f707c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 -- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 msgr2=0x7f707c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709815f6b0 0x7f707c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 -- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707c004810 msgr2=0x7f707c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707c004810 0x7f707c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.991+0000 7f7095e00640 1 -- 172.21.15.87:0/628562062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c003530 con 0x7f707c0039c0 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f7095e00640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f707c003de0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f708800c240 tx=0x7f708801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b73340962ee450e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088007d40 con 0x7f707c0039c0 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70880a9020 con 0x7f707c0039c0 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f7094a00640 1 -- 172.21.15.87:0/628562062 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088005ae0 con 0x7f707c0039c0 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f709d200640 1 -- 172.21.15.87:0/628562062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 msgr2=0x7f707c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f709d200640 1 --2- 172.21.15.87:0/628562062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f707c003de0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f708800c240 tx=0x7f708801cca0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f7096800640 1 -- 172.21.15.87:0/628562062 reap_dead start 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f709d200640 1 -- 172.21.15.87:0/628562062 shutdown_connections 2024-04-25T10:25:43.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f709d200640 1 -- 172.21.15.87:0/628562062 >> 172.21.15.87:0/628562062 conn(0x7f70980d0800 msgr2=0x7f70980d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f709d200640 1 -- 172.21.15.87:0/628562062 shutdown_connections 2024-04-25T10:25:43.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.995+0000 7f709d200640 1 -- 172.21.15.87:0/628562062 wait complete. 2024-04-25T10:25:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 Processor -- start 2024-04-25T10:25:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 -- start start 2024-04-25T10:25:44.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f709814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70981ec7a0 0x7f70981ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70980b1230 0x7f70980b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709806f880 con 0x7f707c0039c0 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980cef80 con 0x7f70980b1230 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709806fa00 con 0x7f70981ec7a0 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f709814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f709814c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53054/0 (socket says 172.21.15.87:53054) 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 -- 172.21.15.87:0/2336134074 learned_addr learned my addr 172.21.15.87:0/2336134074 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7096800640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70980b1230 0x7f70980b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095400640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70981ec7a0 0x7f70981ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 -- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70981ec7a0 msgr2=0x7f70981ecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70981ec7a0 0x7f70981ecc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 -- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70980b1230 msgr2=0x7f70980b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70980b1230 0x7f70980b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 -- 172.21.15.87:0/2336134074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709815c1d0 con 0x7f707c0039c0 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f7095e00640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f709814c890 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f7088007040 tx=0x7f707c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f708e000640 1 -- 172.21.15.87:0/2336134074 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70880049f0 con 0x7f707c0039c0 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f708e000640 1 -- 172.21.15.87:0/2336134074 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7088004b90 con 0x7f707c0039c0 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f708e000640 1 -- 172.21.15.87:0/2336134074 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70880070d0 con 0x7f707c0039c0 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709815c440 con 0x7f707c0039c0 2024-04-25T10:25:44.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:43.999+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709815ca00 con 0x7f707c0039c0 2024-04-25T10:25:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.003+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709804ef30 con 0x7f707c0039c0 2024-04-25T10:25:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.003+0000 7f708e000640 1 -- 172.21.15.87:0/2336134074 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7088007270 con 0x7f707c0039c0 2024-04-25T10:25:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.003+0000 7f708e000640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7078066f60 0x7f7078069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.003+0000 7f708e000640 1 -- 172.21.15.87:0/2336134074 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f708811e940 con 0x7f707c0039c0 2024-04-25T10:25:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.003+0000 7f7095400640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7078066f60 0x7f7078069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.003+0000 7f7095400640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7078066f60 0x7f7078069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f709815f100 tx=0x7f7084008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.007+0000 7f708e000640 1 -- 172.21.15.87:0/2336134074 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70880169b0 con 0x7f707c0039c0 2024-04-25T10:25:44.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:44.203+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f70980d1490 con 0x7f707c0039c0 2024-04-25T10:25:45.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.139+0000 7f708e000640 1 -- 172.21.15.87:0/2336134074 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v127) v1 ==== 118+0+0 (secure 0 0 0) 0x7f70880ef9c0 con 0x7f707c0039c0 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7078066f60 msgr2=0x7f7078069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7078066f60 0x7f7078069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f709815f100 tx=0x7f7084008040 comp rx=0 tx=0).stop 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 msgr2=0x7f709814c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f709814c890 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f7088007040 tx=0x7f707c005840 comp rx=0 tx=0).stop 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 shutdown_connections 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7078066f60 0x7f7078069420 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70980b1230 0x7f70980b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70981ec7a0 0x7f70981ecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 --2- 172.21.15.87:0/2336134074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f707c0039c0 0x7f709814c890 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 >> 172.21.15.87:0/2336134074 conn(0x7f70980d0800 msgr2=0x7f70981570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:45.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 shutdown_connections 2024-04-25T10:25:45.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.143+0000 7f709d200640 1 -- 172.21.15.87:0/2336134074 wait complete. 2024-04-25T10:25:45.167 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:25:45.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:25:45.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8edb200640 1 Processor -- start 2024-04-25T10:25:45.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8edb200640 1 -- start start 2024-04-25T10:25:45.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8edb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed4159540 0x7f8ed40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:45.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8edb200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8ed40721f0 0x7f8ed4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:45.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8edb200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed4159920 con 0x7f8ed40aebd0 2024-04-25T10:25:45.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8edb200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed40af030 con 0x7f8ed40721f0 2024-04-25T10:25:45.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8edb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed415c2f0 con 0x7f8ed4159540 2024-04-25T10:25:45.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed9200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8ed40721f0 0x7f8ed4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:45.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8ed40aebd0 0x7f8ed4071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:55892/0 (socket says 172.21.15.87:55892) 2024-04-25T10:25:45.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed3400640 1 -- 172.21.15.87:0/1300146960 learned_addr learned my addr 172.21.15.87:0/1300146960 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:45.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed3e00640 1 --2- 172.21.15.87:0/1300146960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed4159540 0x7f8ed40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:45.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed9200640 1 -- 172.21.15.87:0/1300146960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed4159540 msgr2=0x7f8ed40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:45.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed9200640 1 --2- 172.21.15.87:0/1300146960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed4159540 0x7f8ed40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed9200640 1 -- 172.21.15.87:0/1300146960 >> v1:172.21.15.87:6789/0 conn(0x7f8ed40aebd0 legacy=0x7f8ed4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:45.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed9200640 1 -- 172.21.15.87:0/1300146960 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed415c5d0 con 0x7f8ed40721f0 2024-04-25T10:25:45.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed2a00640 1 -- 172.21.15.87:0/1300146960 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945348838 0 0) 0x7f8ed4159920 con 0x7f8ed40aebd0 2024-04-25T10:25:45.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.311+0000 7f8ed3e00640 1 --2- 172.21.15.87:0/1300146960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed4159540 0x7f8ed40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:45.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8ed9200640 1 --2- 172.21.15.87:0/1300146960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed40721f0 0x7f8ed4070350 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96be911f175e085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:45.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8ed2a00640 1 -- 172.21.15.87:0/1300146960 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0b2070 con 0x7f8ed40721f0 2024-04-25T10:25:45.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8ed2a00640 1 -- 172.21.15.87:0/1300146960 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc0a5a00 con 0x7f8ed40721f0 2024-04-25T10:25:45.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8ed2a00640 1 -- 172.21.15.87:0/1300146960 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0a5dd0 con 0x7f8ed40721f0 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 -- 172.21.15.87:0/1300146960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed40721f0 msgr2=0x7f8ed4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 --2- 172.21.15.87:0/1300146960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed40721f0 0x7f8ed4070350 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc01c940 comp rx=0 tx=0).stop 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 -- 172.21.15.87:0/1300146960 shutdown_connections 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 --2- 172.21.15.87:0/1300146960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed40721f0 0x7f8ed4070350 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 --2- 172.21.15.87:0/1300146960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed4159540 0x7f8ed40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 -- 172.21.15.87:0/1300146960 >> 172.21.15.87:0/1300146960 conn(0x7f8ed40669a0 msgr2=0x7f8ed4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 -- 172.21.15.87:0/1300146960 shutdown_connections 2024-04-25T10:25:45.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.315+0000 7f8edb200640 1 -- 172.21.15.87:0/1300146960 wait complete. 2024-04-25T10:25:45.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 Processor -- start 2024-04-25T10:25:45.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 -- start start 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed40721f0 0x7f8ed41f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ed40aebd0 0x7f8ed40afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed4159540 0x7f8ed40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40710a0 con 0x7f8ed4159540 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed406c0d0 con 0x7f8ed40aebd0 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4071220 con 0x7f8ed40721f0 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed4159540 0x7f8ed40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed4159540 0x7f8ed40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53068/0 (socket says 172.21.15.87:53068) 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 -- 172.21.15.87:0/215784168 learned_addr learned my addr 172.21.15.87:0/215784168 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:45.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed3e00640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed40721f0 0x7f8ed41f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed3400640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ed40aebd0 0x7f8ed40afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 -- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed40721f0 msgr2=0x7f8ed41f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed40721f0 0x7f8ed41f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 -- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ed40aebd0 msgr2=0x7f8ed40afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ed40aebd0 0x7f8ed40afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 -- 172.21.15.87:0/215784168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed40b03b0 con 0x7f8ed4159540 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ed9200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed4159540 0x7f8ed40ad840 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc01aff0 tx=0x7f8ecc0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ecbe00640 1 -- 172.21.15.87:0/215784168 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0b2050 con 0x7f8ed4159540 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ecbe00640 1 -- 172.21.15.87:0/215784168 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc0be070 con 0x7f8ed4159540 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8ecbe00640 1 -- 172.21.15.87:0/215784168 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0adce0 con 0x7f8ed4159540 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed40b0690 con 0x7f8ed4159540 2024-04-25T10:25:45.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed40b0c50 con 0x7f8ed4159540 2024-04-25T10:25:45.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.319+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed41419d0 con 0x7f8ed4159540 2024-04-25T10:25:45.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.323+0000 7f8ecbe00640 1 -- 172.21.15.87:0/215784168 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ecc0b9450 con 0x7f8ed4159540 2024-04-25T10:25:45.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.323+0000 7f8ecbe00640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8eb4066f90 0x7f8eb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:45.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.323+0000 7f8ecbe00640 1 -- 172.21.15.87:0/215784168 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ecc120220 con 0x7f8ed4159540 2024-04-25T10:25:45.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.323+0000 7f8ed3e00640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8eb4066f90 0x7f8eb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:45.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.323+0000 7f8ed3e00640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8eb4066f90 0x7f8eb4069450 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006fd0 tx=0x7f8ec4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:45.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.327+0000 7f8ecbe00640 1 -- 172.21.15.87:0/215784168 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ed41419d0 con 0x7f8ed4159540 2024-04-25T10:25:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:45.511+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8ed404ef30 con 0x7f8ed4159540 2024-04-25T10:25:46.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.147+0000 7f8ecbe00640 1 -- 172.21.15.87:0/215784168 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v128) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ecc016560 con 0x7f8ed4159540 2024-04-25T10:25:46.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8eb4066f90 msgr2=0x7f8eb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8eb4066f90 0x7f8eb4069450 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006fd0 tx=0x7f8ec4008040 comp rx=0 tx=0).stop 2024-04-25T10:25:46.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed4159540 msgr2=0x7f8ed40ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed4159540 0x7f8ed40ad840 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc01aff0 tx=0x7f8ecc0196e0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 shutdown_connections 2024-04-25T10:25:46.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8eb4066f90 0x7f8eb4069450 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ed4159540 0x7f8ed40ad840 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ed40aebd0 0x7f8ed40afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 --2- 172.21.15.87:0/215784168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ed40721f0 0x7f8ed41f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 >> 172.21.15.87:0/215784168 conn(0x7f8ed40669a0 msgr2=0x7f8ed415d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 shutdown_connections 2024-04-25T10:25:46.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.151+0000 7f8edb200640 1 -- 172.21.15.87:0/215784168 wait complete. 2024-04-25T10:25:46.173 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:25:46.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:46.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 Processor -- start 2024-04-25T10:25:46.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 -- start start 2024-04-25T10:25:46.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01100b0f70 0x7f01100b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f01100cfd90 0x7f01100ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f01100cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011015ef20 con 0x7f01100cfd90 2024-04-25T10:25:46.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011015f3b0 con 0x7f01100b0f70 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0117a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01100ca810 con 0x7f01100cd110 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0107e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f01100cfd90 0x7f01100ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0107e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f01100cfd90 0x7f01100ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53076/0 (socket says 172.21.15.87:53076) 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.311+0000 7f0107e00640 1 -- 172.21.15.87:0/482154008 learned_addr learned my addr 172.21.15.87:0/482154008 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0115a00640 1 --2- 172.21.15.87:0/482154008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f01100cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0115000640 1 --2- 172.21.15.87:0/482154008 >> v2:172.21.15.169:3300/0 conn(0x7f01100b0f70 0x7f01100b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0107e00640 1 -- 172.21.15.87:0/482154008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 msgr2=0x7f01100cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0107e00640 1 --2- 172.21.15.87:0/482154008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f01100cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0107e00640 1 -- 172.21.15.87:0/482154008 >> v2:172.21.15.169:3300/0 conn(0x7f01100b0f70 msgr2=0x7f01100b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:46.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0107e00640 1 --2- 172.21.15.87:0/482154008 >> v2:172.21.15.169:3300/0 conn(0x7f01100b0f70 0x7f01100b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0107e00640 1 -- 172.21.15.87:0/482154008 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100b1890 con 0x7f01100cfd90 2024-04-25T10:25:46.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0115000640 1 --2- 172.21.15.87:0/482154008 >> v2:172.21.15.169:3300/0 conn(0x7f01100b0f70 0x7f01100b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:46.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0115a00640 1 --2- 172.21.15.87:0/482154008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f01100cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:46.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0107e00640 1 --2- 172.21.15.87:0/482154008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f01100ccbd0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f0110071230 tx=0x7f00f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f731d11ff307c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f010fe00640 1 -- 172.21.15.87:0/482154008 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80b0070 con 0x7f01100cfd90 2024-04-25T10:25:46.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f010fe00640 1 -- 172.21.15.87:0/482154008 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f80ab600 con 0x7f01100cfd90 2024-04-25T10:25:46.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f010fe00640 1 -- 172.21.15.87:0/482154008 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80ab9d0 con 0x7f01100cfd90 2024-04-25T10:25:46.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0117a00640 1 -- 172.21.15.87:0/482154008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 msgr2=0x7f01100ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0117a00640 1 --2- 172.21.15.87:0/482154008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f01100ccbd0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f0110071230 tx=0x7f00f801c520 comp rx=0 tx=0).stop 2024-04-25T10:25:46.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0117a00640 1 -- 172.21.15.87:0/482154008 shutdown_connections 2024-04-25T10:25:46.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0117a00640 1 --2- 172.21.15.87:0/482154008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f01100ccbd0 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0117a00640 1 --2- 172.21.15.87:0/482154008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f01100cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0117a00640 1 --2- 172.21.15.87:0/482154008 >> v2:172.21.15.169:3300/0 conn(0x7f01100b0f70 0x7f01100b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.315+0000 7f0117a00640 1 -- 172.21.15.87:0/482154008 >> 172.21.15.87:0/482154008 conn(0x7f01100b1d50 msgr2=0x7f01100b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 -- 172.21.15.87:0/482154008 shutdown_connections 2024-04-25T10:25:46.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 -- 172.21.15.87:0/482154008 wait complete. 2024-04-25T10:25:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 Processor -- start 2024-04-25T10:25:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 -- start start 2024-04-25T10:25:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01100b0f70 0x7f01100c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f0110099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f011006df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0110072200 con 0x7f01100cfd90 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100cd520 con 0x7f01100b0f70 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0110072380 con 0x7f01100cd110 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f011006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f011006df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53080/0 (socket says 172.21.15.87:53080) 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 -- 172.21.15.87:0/2113106984 learned_addr learned my addr 172.21.15.87:0/2113106984 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115000640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01100b0f70 0x7f01100c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0107e00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f0110099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 -- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 msgr2=0x7f0110099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f0110099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 -- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01100b0f70 msgr2=0x7f01100c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01100b0f70 0x7f01100c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 -- 172.21.15.87:0/2113106984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100c9cd0 con 0x7f01100cfd90 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0115a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f011006df10 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f010800cf20 tx=0x7f010800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f010d600640 1 -- 172.21.15.87:0/2113106984 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010801a070 con 0x7f01100cfd90 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f010d600640 1 -- 172.21.15.87:0/2113106984 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010800d840 con 0x7f01100cfd90 2024-04-25T10:25:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f010d600640 1 -- 172.21.15.87:0/2113106984 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800dc30 con 0x7f01100cfd90 2024-04-25T10:25:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.319+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01100c9fb0 con 0x7f01100cfd90 2024-04-25T10:25:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.323+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01100c9980 con 0x7f01100cfd90 2024-04-25T10:25:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.323+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011004ef30 con 0x7f01100cfd90 2024-04-25T10:25:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.323+0000 7f010d600640 1 -- 172.21.15.87:0/2113106984 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f010801d070 con 0x7f01100cfd90 2024-04-25T10:25:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.323+0000 7f010d600640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00ec066f60 0x7f00ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.323+0000 7f010d600640 1 -- 172.21.15.87:0/2113106984 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0108085820 con 0x7f01100cfd90 2024-04-25T10:25:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.323+0000 7f0115000640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00ec066f60 0x7f00ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.323+0000 7f0115000640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00ec066f60 0x7f00ec069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0100006b20 tx=0x7f0100008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.327+0000 7f010d600640 1 -- 172.21.15.87:0/2113106984 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01080561e0 con 0x7f01100cfd90 2024-04-25T10:25:46.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.511+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01100ca180 con 0x7f01100cfd90 2024-04-25T10:25:46.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.511+0000 7f010d600640 1 -- 172.21.15.87:0/2113106984 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f010805a280 con 0x7f01100cfd90 2024-04-25T10:25:46.514 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:46.514 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":128,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1787812144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1787812144},{"type":"v1","addr":"172.21.15.169:6834","nonce":1787812144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/554048725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":554048725},{"type":"v1","addr":"172.21.15.169:6835","nonce":554048725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1720179922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1720179922},{"type":"v1","addr":"172.21.15.87:6837","nonce":1720179922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:42.498010+0000","modified":"2024-04-25T10:25:45.516622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10920":{"gid":10920,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/3053712751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3053712751},{"type":"v1","addr":"172.21.15.87:6836","nonce":3053712751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:25:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 128 2024-04-25T10:25:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00ec066f60 msgr2=0x7f00ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00ec066f60 0x7f00ec069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0100006b20 tx=0x7f0100008040 comp rx=0 tx=0).stop 2024-04-25T10:25:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 msgr2=0x7f011006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f011006df10 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f010800cf20 tx=0x7f010800a7d0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 shutdown_connections 2024-04-25T10:25:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f00ec066f60 0x7f00ec069420 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01100cfd90 0x7f011006df10 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01100cd110 0x7f0110099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 --2- 172.21.15.87:0/2113106984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01100b0f70 0x7f01100c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 >> 172.21.15.87:0/2113106984 conn(0x7f01100b1d50 msgr2=0x7f011015f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 shutdown_connections 2024-04-25T10:25:46.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.515+0000 7f0117a00640 1 -- 172.21.15.87:0/2113106984 wait complete. 2024-04-25T10:25:46.534 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:46.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.667+0000 7f5994e00640 1 Processor -- start 2024-04-25T10:25:46.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f5994e00640 1 -- start start 2024-04-25T10:25:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f5994e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f599015bc40 0x7f599006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f5994e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59900cda10 0x7f59900b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f5994e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59900b2770 con 0x7f59900cf930 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f5994e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599006c670 con 0x7f599015bc40 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f5994e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599006c950 con 0x7f59900cda10 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f598de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f599015bc40 0x7f599006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f598de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f599015bc40 0x7f599006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53090/0 (socket says 172.21.15.87:53090) 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f598de00640 1 -- 172.21.15.87:0/2101770791 learned_addr learned my addr 172.21.15.87:0/2101770791 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f5985400640 1 --2- 172.21.15.87:0/2101770791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59900cda10 0x7f59900b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f598de00640 1 -- 172.21.15.87:0/2101770791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59900cda10 msgr2=0x7f59900b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f598de00640 1 --2- 172.21.15.87:0/2101770791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59900cda10 0x7f59900b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f598de00640 1 -- 172.21.15.87:0/2101770791 >> v1:172.21.15.169:6789/0 conn(0x7f59900cf930 legacy=0x7f599015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.671+0000 7f598de00640 1 -- 172.21.15.87:0/2101770791 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599006cc30 con 0x7f599015bc40 2024-04-25T10:25:46.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f598de00640 1 --2- 172.21.15.87:0/2101770791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f599015bc40 0x7f599006c090 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f5980009e50 tx=0x7f598001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354c024bcd2647f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f598d400640 1 -- 172.21.15.87:0/2101770791 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800a5810 con 0x7f599015bc40 2024-04-25T10:25:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f598d400640 1 -- 172.21.15.87:0/2101770791 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59800a59b0 con 0x7f599015bc40 2024-04-25T10:25:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f598d400640 1 -- 172.21.15.87:0/2101770791 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800a5d80 con 0x7f599015bc40 2024-04-25T10:25:46.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 -- 172.21.15.87:0/2101770791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f599015bc40 msgr2=0x7f599006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 --2- 172.21.15.87:0/2101770791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f599015bc40 0x7f599006c090 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f5980009e50 tx=0x7f598001ca30 comp rx=0 tx=0).stop 2024-04-25T10:25:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 -- 172.21.15.87:0/2101770791 shutdown_connections 2024-04-25T10:25:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 --2- 172.21.15.87:0/2101770791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59900cda10 0x7f59900b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 --2- 172.21.15.87:0/2101770791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f599015bc40 0x7f599006c090 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 -- 172.21.15.87:0/2101770791 >> 172.21.15.87:0/2101770791 conn(0x7f5990066b70 msgr2=0x7f5990066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 -- 172.21.15.87:0/2101770791 shutdown_connections 2024-04-25T10:25:46.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 -- 172.21.15.87:0/2101770791 wait complete. 2024-04-25T10:25:46.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 Processor -- start 2024-04-25T10:25:46.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.675+0000 7f5994e00640 1 -- start start 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59900cda10 0x7f599014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59900cf930 0x7f5990158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f599015bc40 0x7f599015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990168b50 con 0x7f59900cda10 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59901689d0 con 0x7f59900cf930 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5990168cd0 con 0x7f599015bc40 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59900cda10 0x7f599014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59900cda10 0x7f599014ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53102/0 (socket says 172.21.15.87:53102) 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 -- 172.21.15.87:0/3107084311 learned_addr learned my addr 172.21.15.87:0/3107084311 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5985400640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59900cf930 0x7f5990158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598e800640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f599015bc40 0x7f599015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 -- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f599015bc40 msgr2=0x7f599015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f599015bc40 0x7f599015d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 -- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59900cf930 msgr2=0x7f5990158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59900cf930 0x7f5990158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 -- 172.21.15.87:0/3107084311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599015c230 con 0x7f59900cda10 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598e800640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f599015bc40 0x7f599015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f598de00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59900cda10 0x7f599014ed20 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f598001a660 tx=0x7f59800197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5986a00640 1 -- 172.21.15.87:0/3107084311 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800b8470 con 0x7f59900cda10 2024-04-25T10:25:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5986a00640 1 -- 172.21.15.87:0/3107084311 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59800b8610 con 0x7f59900cda10 2024-04-25T10:25:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599015c510 con 0x7f59900cda10 2024-04-25T10:25:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5986a00640 1 -- 172.21.15.87:0/3107084311 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800b8a00 con 0x7f59900cda10 2024-04-25T10:25:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599015cad0 con 0x7f59900cda10 2024-04-25T10:25:46.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599004ef30 con 0x7f59900cda10 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.679+0000 7f5986a00640 1 -- 172.21.15.87:0/3107084311 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59800c0020 con 0x7f59900cda10 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.683+0000 7f5986a00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f596c066f60 0x7f596c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.683+0000 7f5986a00640 1 -- 172.21.15.87:0/3107084311 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f598011f820 con 0x7f59900cda10 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.683+0000 7f5985400640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f596c066f60 0x7f596c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.683+0000 7f5985400640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f596c066f60 0x7f596c069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5978009ac0 tx=0x7f5978009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.687+0000 7f5986a00640 1 -- 172.21.15.87:0/3107084311 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59800c7020 con 0x7f59900cda10 2024-04-25T10:25:46.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.839+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f599006d960 con 0x7f59900cda10 2024-04-25T10:25:46.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.839+0000 7f5986a00640 1 -- 172.21.15.87:0/3107084311 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f5980016620 con 0x7f59900cda10 2024-04-25T10:25:46.840 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:46.840 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":176,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:44.125291+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:40.866786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:41.487045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:46.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.839+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f596c066f60 msgr2=0x7f596c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.839+0000 7f5994e00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f596c066f60 0x7f596c069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5978009ac0 tx=0x7f5978009190 comp rx=0 tx=0).stop 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.839+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59900cda10 msgr2=0x7f599014ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.839+0000 7f5994e00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59900cda10 0x7f599014ed20 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f598001a660 tx=0x7f59800197d0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 shutdown_connections 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f596c066f60 0x7f596c069420 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f599015bc40 0x7f599015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59900cf930 0x7f5990158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 --2- 172.21.15.87:0/3107084311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59900cda10 0x7f599014ed20 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 >> 172.21.15.87:0/3107084311 conn(0x7f5990066b70 msgr2=0x7f5990159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 shutdown_connections 2024-04-25T10:25:46.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.843+0000 7f5994e00640 1 -- 172.21.15.87:0/3107084311 wait complete. 2024-04-25T10:25:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:25:46.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:25:46.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 Processor -- start 2024-04-25T10:25:46.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 -- start start 2024-04-25T10:25:46.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b541550b0 0x7f8b540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b5415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b5415b740 0x7f8b5406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b54071050 con 0x7f8b5415b740 2024-04-25T10:25:47.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b540712f0 con 0x7f8b541550b0 2024-04-25T10:25:47.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b59200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540715d0 con 0x7f8b540cfe70 2024-04-25T10:25:47.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b52800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b5415b740 0x7f8b5406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b51e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b541550b0 0x7f8b540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b51400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b5415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b52800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b5415b740 0x7f8b5406d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53110/0 (socket says 172.21.15.87:53110) 2024-04-25T10:25:47.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.995+0000 7f8b52800640 1 -- 172.21.15.87:0/937567250 learned_addr learned my addr 172.21.15.87:0/937567250 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:47.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b52800640 1 -- 172.21.15.87:0/937567250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 msgr2=0x7f8b5415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:47.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b52800640 1 --2- 172.21.15.87:0/937567250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b5415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b52800640 1 -- 172.21.15.87:0/937567250 >> v2:172.21.15.169:3300/0 conn(0x7f8b541550b0 msgr2=0x7f8b540cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:47.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b52800640 1 --2- 172.21.15.87:0/937567250 >> v2:172.21.15.169:3300/0 conn(0x7f8b541550b0 0x7f8b540cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b52800640 1 -- 172.21.15.87:0/937567250 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540718b0 con 0x7f8b5415b740 2024-04-25T10:25:47.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b51e00640 1 --2- 172.21.15.87:0/937567250 >> v2:172.21.15.169:3300/0 conn(0x7f8b541550b0 0x7f8b540cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b52800640 1 --2- 172.21.15.87:0/937567250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b5406d1a0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f8b54067980 tx=0x7f8b4c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6b64364b545110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b50a00640 1 -- 172.21.15.87:0/937567250 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0ac020 con 0x7f8b5415b740 2024-04-25T10:25:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b50a00640 1 -- 172.21.15.87:0/937567250 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c0a6440 con 0x7f8b5415b740 2024-04-25T10:25:47.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b50a00640 1 -- 172.21.15.87:0/937567250 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0a6810 con 0x7f8b5415b740 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b59200640 1 -- 172.21.15.87:0/937567250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 msgr2=0x7f8b5406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b59200640 1 --2- 172.21.15.87:0/937567250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b5406d1a0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f8b54067980 tx=0x7f8b4c01c730 comp rx=0 tx=0).stop 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b59200640 1 -- 172.21.15.87:0/937567250 shutdown_connections 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b59200640 1 --2- 172.21.15.87:0/937567250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b5406d1a0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:46.999+0000 7f8b59200640 1 --2- 172.21.15.87:0/937567250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b5415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 --2- 172.21.15.87:0/937567250 >> v2:172.21.15.169:3300/0 conn(0x7f8b541550b0 0x7f8b540cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- 172.21.15.87:0/937567250 >> 172.21.15.87:0/937567250 conn(0x7f8b54066b70 msgr2=0x7f8b54066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- 172.21.15.87:0/937567250 shutdown_connections 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- 172.21.15.87:0/937567250 wait complete. 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 Processor -- start 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- start start 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b541573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b541550b0 0x7f8b54149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b54153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540b33e0 con 0x7f8b5415b740 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540b3260 con 0x7f8b541550b0 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540b3560 con 0x7f8b540cfe70 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b54153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b54153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53116/0 (socket says 172.21.15.87:53116) 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 -- 172.21.15.87:0/1560198448 learned_addr learned my addr 172.21.15.87:0/1560198448 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b51e00640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b541573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b51400640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b541550b0 0x7f8b54149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 -- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 msgr2=0x7f8b541573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b541573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 -- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b541550b0 msgr2=0x7f8b54149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b541550b0 0x7f8b54149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 -- 172.21.15.87:0/1560198448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54154090 con 0x7f8b5415b740 2024-04-25T10:25:47.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b52800640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b54153920 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c01ade0 tx=0x7f8b4c0194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b4a000640 1 -- 172.21.15.87:0/1560198448 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0b7070 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b4a000640 1 -- 172.21.15.87:0/1560198448 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c0a6a00 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b4a000640 1 -- 172.21.15.87:0/1560198448 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0a6dd0 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.003+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b54154370 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.007+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5414a1f0 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.007+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5404ef30 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.007+0000 7f8b4a000640 1 -- 172.21.15.87:0/1560198448 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b4c0c2060 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.007+0000 7f8b4a000640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b34066f60 0x7f8b34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.007+0000 7f8b4a000640 1 -- 172.21.15.87:0/1560198448 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b4c11ed10 con 0x7f8b5415b740 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.007+0000 7f8b51e00640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b34066f60 0x7f8b34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.007+0000 7f8b51e00640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b34066f60 0x7f8b34069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ce620 tx=0x7f8b44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.011+0000 7f8b4a000640 1 -- 172.21.15.87:0/1560198448 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b4c124020 con 0x7f8b5415b740 2024-04-25T10:25:47.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.163+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b5414b750 con 0x7f8b34066f60 2024-04-25T10:25:47.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.167+0000 7f8b4a000640 1 -- 172.21.15.87:0/1560198448 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240233 (secure 0 0 0) 0x7f8b5414b750 con 0x7f8b34066f60 2024-04-25T10:25:47.171 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:47.172 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b34066f60 msgr2=0x7f8b34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b34066f60 0x7f8b34069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ce620 tx=0x7f8b44008040 comp rx=0 tx=0).stop 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 msgr2=0x7f8b54153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b54153920 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c01ade0 tx=0x7f8b4c0194d0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 shutdown_connections 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8b34066f60 0x7f8b34069420 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b5415b740 0x7f8b54153920 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b541550b0 0x7f8b54149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 --2- 172.21.15.87:0/1560198448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b540cfe70 0x7f8b541573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 >> 172.21.15.87:0/1560198448 conn(0x7f8b54066b70 msgr2=0x7f8b54158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:47.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 shutdown_connections 2024-04-25T10:25:47.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.171+0000 7f8b59200640 1 -- 172.21.15.87:0/1560198448 wait complete. 2024-04-25T10:25:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":576,"stamp":"2024-04-25T10:25:45.296471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":477172,"kb_used_data":18168,"kb_used_omap":12,"kb_used_meta":458867,"kb_avail":754497548,"statfs":{"total":773094113280,"available":772605489152,"internally_reserved":0,"allocated":18604032,"data_stored":8807852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":469880369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.193995"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.112930+0000","last_change":"2024-04-25T10:25:41.112930+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.112930+0000","last_undegraded":"2024-04-25T10:25:41.112930+0000","last_fullsized":"2024-04-25T10:25:41.112930+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.106892+0000","last_change":"2024-04-25T10:25:41.106892+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.106892+0000","last_undegraded":"2024-04-25T10:25:41.106892+0000","last_fullsized":"2024-04-25T10:25:41.106892+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.109214+0000","last_change":"2024-04-25T10:25:41.109214+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.109214+0000","last_undegraded":"2024-04-25T10:25:41.109214+0000","last_fullsized":"2024-04-25T10:25:41.109214+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.110204+0000","last_change":"2024-04-25T10:25:41.110204+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.110204+0000","last_undegraded":"2024-04-25T10:25:41.110204+0000","last_fullsized":"2024-04-25T10:25:41.110204+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.105925+0000","last_change":"2024-04-25T10:25:41.105925+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.105925+0000","last_undegraded":"2024-04-25T10:25:41.105925+0000","last_fullsized":"2024-04-25T10:25:41.105925+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.105474+0000","last_change":"2024-04-25T10:25:41.105474+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.105474+0000","last_undegraded":"2024-04-25T10:25:41.105474+0000","last_fullsized":"2024-04-25T10:25:41.105474+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107534+0000","last_change":"2024-04-25T10:25:41.107534+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107534+0000","last_undegraded":"2024-04-25T10:25:41.107534+0000","last_fullsized":"2024-04-25T10:25:41.107534+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.105673+0000","last_change":"2024-04-25T10:25:41.105673+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.105673+0000","last_undegraded":"2024-04-25T10:25:41.105673+0000","last_fullsized":"2024-04-25T10:25:41.105673+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107738+0000","last_change":"2024-04-25T10:25:41.107738+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107738+0000","last_undegraded":"2024-04-25T10:25:41.107738+0000","last_fullsized":"2024-04-25T10:25:41.107738+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107907+0000","last_change":"2024-04-25T10:25:41.107907+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107907+0000","last_undegraded":"2024-04-25T10:25:41.107907+0000","last_fullsized":"2024-04-25T10:25:41.107907+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.109627+0000","last_change":"2024-04-25T10:25:41.109627+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.109627+0000","last_undegraded":"2024-04-25T10:25:41.109627+0000","last_fullsized":"2024-04-25T10:25:41.109627+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107650+0000","last_change":"2024-04-25T10:25:41.107650+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107650+0000","last_undegraded":"2024-04-25T10:25:41.107650+0000","last_fullsized":"2024-04-25T10:25:41.107650+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107678+0000","last_change":"2024-04-25T10:25:41.107678+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107678+0000","last_undegraded":"2024-04-25T10:25:41.107678+0000","last_fullsized":"2024-04-25T10:25:41.107678+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.108321+0000","last_change":"2024-04-25T10:25:41.108321+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.108321+0000","last_undegraded":"2024-04-25T10:25:41.108321+0000","last_fullsized":"2024-04-25T10:25:41.108321+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107947+0000","last_change":"2024-04-25T10:25:41.107947+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107947+0000","last_undegraded":"2024-04-25T10:25:41.107947+0000","last_fullsized":"2024-04-25T10:25:41.107947+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.105147+0000","last_change":"2024-04-25T10:25:41.105147+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.105147+0000","last_undegraded":"2024-04-25T10:25:41.105147+0000","last_fullsized":"2024-04-25T10:25:41.105147+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.106255+0000","last_change":"2024-04-25T10:25:41.106255+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.106255+0000","last_undegraded":"2024-04-25T10:25:41.106255+0000","last_fullsized":"2024-04-25T10:25:41.106255+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107518+0000","last_change":"2024-04-25T10:25:41.107518+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107518+0000","last_undegraded":"2024-04-25T10:25:41.107518+0000","last_fullsized":"2024-04-25T10:25:41.107518+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.109469+0000","last_change":"2024-04-25T10:25:41.109469+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.109469+0000","last_undegraded":"2024-04-25T10:25:41.109469+0000","last_fullsized":"2024-04-25T10:25:41.109469+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.107334+0000","last_change":"2024-04-25T10:25:41.107334+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.107334+0000","last_undegraded":"2024-04-25T10:25:41.107334+0000","last_fullsized":"2024-04-25T10:25:41.107334+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.111571+0000","last_change":"2024-04-25T10:25:41.111571+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.111571+0000","last_undegraded":"2024-04-25T10:25:41.111571+0000","last_fullsized":"2024-04-25T10:25:41.111571+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.111571+0000","last_change":"2024-04-25T10:25:41.111571+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.111571+0000","last_undegraded":"2024-04-25T10:25:41.111571+0000","last_fullsized":"2024-04-25T10:25:41.111571+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.105120+0000","last_change":"2024-04-25T10:25:41.105120+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.105120+0000","last_undegraded":"2024-04-25T10:25:41.105120+0000","last_fullsized":"2024-04-25T10:25:41.105120+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.109394+0000","last_change":"2024-04-25T10:25:41.109394+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.109394+0000","last_undegraded":"2024-04-25T10:25:41.109394+0000","last_fullsized":"2024-04-25T10:25:41.109394+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.109640+0000","last_change":"2024-04-25T10:25:41.109640+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.109640+0000","last_undegraded":"2024-04-25T10:25:41.109640+0000","last_fullsized":"2024-04-25T10:25:41.109640+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.105131+0000","last_change":"2024-04-25T10:25:41.105131+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.105131+0000","last_undegraded":"2024-04-25T10:25:41.105131+0000","last_fullsized":"2024-04-25T10:25:41.105131+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.108930+0000","last_change":"2024-04-25T10:25:41.108930+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.108930+0000","last_undegraded":"2024-04-25T10:25:41.108930+0000","last_fullsized":"2024-04-25T10:25:41.108930+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.109955+0000","last_change":"2024-04-25T10:25:41.109955+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.109955+0000","last_undegraded":"2024-04-25T10:25:41.109955+0000","last_fullsized":"2024-04-25T10:25:41.109955+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.105072+0000","last_change":"2024-04-25T10:25:41.105072+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.105072+0000","last_undegraded":"2024-04-25T10:25:41.105072+0000","last_fullsized":"2024-04-25T10:25:41.105072+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.108845+0000","last_change":"2024-04-25T10:25:41.108845+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.108845+0000","last_undegraded":"2024-04-25T10:25:41.108845+0000","last_fullsized":"2024-04-25T10:25:41.108845+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.111876+0000","last_change":"2024-04-25T10:25:41.111876+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.111876+0000","last_undegraded":"2024-04-25T10:25:41.111876+0000","last_fullsized":"2024-04-25T10:25:41.111876+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:25:42.102395+0000","last_change":"2024-04-25T10:25:42.102395+0000","last_active":"2024-04-25T10:25:42.102395+0000","last_peered":"2024-04-25T10:25:42.102395+0000","last_clean":"2024-04-25T10:25:42.102395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:42.102395+0000","last_undegraded":"2024-04-25T10:25:42.102395+0000","last_fullsized":"2024-04-25T10:25:42.102395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":173,"state":"creating+peering","last_fresh":"2024-04-25T10:25:41.113658+0000","last_change":"2024-04-25T10:25:41.113658+0000","last_active":"2024-04-25T10:25:41.099887+0000","last_peered":"2024-04-25T10:25:41.099887+0000","last_clean":"2024-04-25T10:25:41.099887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:25:41.113658+0000","last_undegraded":"2024-04-25T10:25:41.113658+0000","last_fullsized":"2024-04-25T10:25:41.113658+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":447,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-04-25T10:25:41.109520+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:25:41.109520+0000","last_peered":"2024-04-25T10:25:41.109520+0000","last_clean":"2024-04-25T10:25:41.109520+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:25:41.109520+0000","last_undegraded":"2024-04-25T10:25:41.109520+0000","last_fullsized":"2024-04-25T10:25:41.109520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60260,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311580,"statfs":{"total":96636764160,"available":96575057920,"internally_reserved":0,"allocated":2777088,"data_stored":1546666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65172,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306668,"statfs":{"total":96636764160,"available":96570028032,"internally_reserved":0,"allocated":2170880,"data_stored":950481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51360,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320480,"statfs":{"total":96636764160,"available":96584171520,"internally_reserved":0,"allocated":2170880,"data_stored":950481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59680,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312160,"statfs":{"total":96636764160,"available":96575651840,"internally_reserved":0,"allocated":2183168,"data_stored":956298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59692,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312148,"statfs":{"total":96636764160,"available":96575639552,"internally_reserved":0,"allocated":2183168,"data_stored":956298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55508,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316332,"statfs":{"total":96636764160,"available":96579923968,"internally_reserved":0,"allocated":2170880,"data_stored":950481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69400,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302440,"statfs":{"total":96636764160,"available":96565698560,"internally_reserved":0,"allocated":2170880,"data_stored":950481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56100,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315740,"statfs":{"total":96636764160,"available":96579317760,"internally_reserved":0,"allocated":2777088,"data_stored":1546666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:25:47.195 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:25:47.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.367+0000 7fc33c400640 1 Processor -- start 2024-04-25T10:25:47.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.367+0000 7fc33c400640 1 -- start start 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc33c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc33c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3340cda10 0x7fc3340b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc33c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3340cf930 0x7fc3340cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc33c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3340b2770 con 0x7fc3340cda10 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc33c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33406c670 con 0x7fc3340cf930 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc33c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33406c950 con 0x7fc33415bc40 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3340cda10 0x7fc3340b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3340cda10 0x7fc3340b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37202/0 (socket says 172.21.15.87:37202) 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 -- 172.21.15.87:0/3732910462 learned_addr learned my addr 172.21.15.87:0/3732910462 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339a00640 1 --2- 172.21.15.87:0/3732910462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc33a400640 1 --2- 172.21.15.87:0/3732910462 >> v2:172.21.15.169:3300/0 conn(0x7fc3340cf930 0x7fc3340cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 -- 172.21.15.87:0/3732910462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 msgr2=0x7fc33406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 --2- 172.21.15.87:0/3732910462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 -- 172.21.15.87:0/3732910462 >> v2:172.21.15.169:3300/0 conn(0x7fc3340cf930 msgr2=0x7fc3340cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 --2- 172.21.15.87:0/3732910462 >> v2:172.21.15.169:3300/0 conn(0x7fc3340cf930 0x7fc3340cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 -- 172.21.15.87:0/3732910462 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33406cc30 con 0x7fc3340cda10 2024-04-25T10:25:47.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc339000640 1 --2- 172.21.15.87:0/3732910462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc3340b4610 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fc3340b3f80 tx=0x7fc32001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e864d02ac43f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc333e00640 1 -- 172.21.15.87:0/3732910462 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200b0070 con 0x7fc3340cda10 2024-04-25T10:25:47.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc333e00640 1 -- 172.21.15.87:0/3732910462 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3200ab600 con 0x7fc3340cda10 2024-04-25T10:25:47.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.371+0000 7fc333e00640 1 -- 172.21.15.87:0/3732910462 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200ab9d0 con 0x7fc3340cda10 2024-04-25T10:25:47.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- 172.21.15.87:0/3732910462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 msgr2=0x7fc3340b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 --2- 172.21.15.87:0/3732910462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc3340b4610 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fc3340b3f80 tx=0x7fc32001c520 comp rx=0 tx=0).stop 2024-04-25T10:25:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- 172.21.15.87:0/3732910462 shutdown_connections 2024-04-25T10:25:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 --2- 172.21.15.87:0/3732910462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc3340b4610 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 --2- 172.21.15.87:0/3732910462 >> v2:172.21.15.169:3300/0 conn(0x7fc3340cf930 0x7fc3340cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 --2- 172.21.15.87:0/3732910462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- 172.21.15.87:0/3732910462 >> 172.21.15.87:0/3732910462 conn(0x7fc334066b70 msgr2=0x7fc334066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- 172.21.15.87:0/3732910462 shutdown_connections 2024-04-25T10:25:47.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- 172.21.15.87:0/3732910462 wait complete. 2024-04-25T10:25:47.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 Processor -- start 2024-04-25T10:25:47.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- start start 2024-04-25T10:25:47.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc334157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340cf930 0x7fc334146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334160230 con 0x7fc3340cda10 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341600b0 con 0x7fc3340cf930 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341603b0 con 0x7fc33415bc40 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc339a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc334157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc339a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc334157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37212/0 (socket says 172.21.15.87:37212) 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc339a00640 1 -- 172.21.15.87:0/2910303622 learned_addr learned my addr 172.21.15.87:0/2910303622 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc339000640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340cf930 0x7fc334146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.375+0000 7fc33a400640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc339a00640 1 -- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 msgr2=0x7fc33415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc339a00640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc339a00640 1 -- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340cf930 msgr2=0x7fc334146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc339a00640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340cf930 0x7fc334146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc339a00640 1 -- 172.21.15.87:0/2910303622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33415ebf0 con 0x7fc3340cda10 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc339a00640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc334157570 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fc32800d800 tx=0x7fc32800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc331600640 1 -- 172.21.15.87:0/2910303622 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32800b6c0 con 0x7fc3340cda10 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc331600640 1 -- 172.21.15.87:0/2910303622 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc328019070 con 0x7fc3340cda10 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc331600640 1 -- 172.21.15.87:0/2910303622 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32800ba60 con 0x7fc3340cda10 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33415eed0 con 0x7fc3340cda10 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc334157050 con 0x7fc3340cda10 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc334156b80 con 0x7fc3340cda10 2024-04-25T10:25:47.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc331600640 1 -- 172.21.15.87:0/2910303622 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc32800bc00 con 0x7fc3340cda10 2024-04-25T10:25:47.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc331600640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc314066f60 0x7fc314069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc331600640 1 -- 172.21.15.87:0/2910303622 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc328085400 con 0x7fc3340cda10 2024-04-25T10:25:47.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.379+0000 7fc339000640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc314066f60 0x7fc314069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.383+0000 7fc339000640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc314066f60 0x7fc314069420 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc3200ad000 tx=0x7fc3200a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.383+0000 7fc331600640 1 -- 172.21.15.87:0/2910303622 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc328056400 con 0x7fc3340cda10 2024-04-25T10:25:47.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.535+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3340722d0 con 0x7fc314066f60 2024-04-25T10:25:47.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc331600640 1 -- 172.21.15.87:0/2910303622 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7fc3340722d0 con 0x7fc314066f60 2024-04-25T10:25:47.544 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:47.545 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:25:47.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc314066f60 msgr2=0x7fc314069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc33c400640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc314066f60 0x7fc314069420 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc3200ad000 tx=0x7fc3200a6000 comp rx=0 tx=0).stop 2024-04-25T10:25:47.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 msgr2=0x7fc334157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc33c400640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc334157570 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fc32800d800 tx=0x7fc32800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 shutdown_connections 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc33c400640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc314066f60 0x7fc314069420 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.543+0000 7fc33c400640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc33415bc40 0x7fc33415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.547+0000 7fc33c400640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340cf930 0x7fc334146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.547+0000 7fc33c400640 1 --2- 172.21.15.87:0/2910303622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3340cda10 0x7fc334157570 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.547+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 >> 172.21.15.87:0/2910303622 conn(0x7fc334066b70 msgr2=0x7fc33415a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.547+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 shutdown_connections 2024-04-25T10:25:47.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.547+0000 7fc33c400640 1 -- 172.21.15.87:0/2910303622 wait complete. 2024-04-25T10:25:47.564 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":577,"stamp":"2024-04-25T10:25:47.296821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481720,"kb_used_data":18568,"kb_used_omap":12,"kb_used_meta":463027,"kb_avail":754493000,"statfs":{"total":773094113280,"available":772600832000,"internally_reserved":0,"allocated":19013632,"data_stored":8990740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":474140209},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001087"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189226+0000","last_change":"2024-04-25T10:25:43.145116+0000","last_active":"2024-04-25T10:25:44.189226+0000","last_peered":"2024-04-25T10:25:44.189226+0000","last_clean":"2024-04-25T10:25:44.189226+0000","last_became_active":"2024-04-25T10:25:43.144853+0000","last_became_peered":"2024-04-25T10:25:43.144853+0000","last_unstale":"2024-04-25T10:25:44.189226+0000","last_undegraded":"2024-04-25T10:25:44.189226+0000","last_fullsized":"2024-04-25T10:25:44.189226+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:39.245568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141014+0000","last_change":"2024-04-25T10:25:43.125609+0000","last_active":"2024-04-25T10:25:44.141014+0000","last_peered":"2024-04-25T10:25:44.141014+0000","last_clean":"2024-04-25T10:25:44.141014+0000","last_became_active":"2024-04-25T10:25:43.125283+0000","last_became_peered":"2024-04-25T10:25:43.125283+0000","last_unstale":"2024-04-25T10:25:44.141014+0000","last_undegraded":"2024-04-25T10:25:44.141014+0000","last_fullsized":"2024-04-25T10:25:44.141014+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:40.117988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188884+0000","last_change":"2024-04-25T10:25:43.132272+0000","last_active":"2024-04-25T10:25:44.188884+0000","last_peered":"2024-04-25T10:25:44.188884+0000","last_clean":"2024-04-25T10:25:44.188884+0000","last_became_active":"2024-04-25T10:25:43.131788+0000","last_became_peered":"2024-04-25T10:25:43.131788+0000","last_unstale":"2024-04-25T10:25:44.188884+0000","last_undegraded":"2024-04-25T10:25:44.188884+0000","last_fullsized":"2024-04-25T10:25:44.188884+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:12.586788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189368+0000","last_change":"2024-04-25T10:25:43.144233+0000","last_active":"2024-04-25T10:25:44.189368+0000","last_peered":"2024-04-25T10:25:44.189368+0000","last_clean":"2024-04-25T10:25:44.189368+0000","last_became_active":"2024-04-25T10:25:43.143238+0000","last_became_peered":"2024-04-25T10:25:43.143238+0000","last_unstale":"2024-04-25T10:25:44.189368+0000","last_undegraded":"2024-04-25T10:25:44.189368+0000","last_fullsized":"2024-04-25T10:25:44.189368+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:33.816471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135613+0000","last_change":"2024-04-25T10:25:43.141017+0000","last_active":"2024-04-25T10:25:44.135613+0000","last_peered":"2024-04-25T10:25:44.135613+0000","last_clean":"2024-04-25T10:25:44.135613+0000","last_became_active":"2024-04-25T10:25:43.140757+0000","last_became_peered":"2024-04-25T10:25:43.140757+0000","last_unstale":"2024-04-25T10:25:44.135613+0000","last_undegraded":"2024-04-25T10:25:44.135613+0000","last_fullsized":"2024-04-25T10:25:44.135613+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.503089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135708+0000","last_change":"2024-04-25T10:25:43.126445+0000","last_active":"2024-04-25T10:25:44.135708+0000","last_peered":"2024-04-25T10:25:44.135708+0000","last_clean":"2024-04-25T10:25:44.135708+0000","last_became_active":"2024-04-25T10:25:43.124653+0000","last_became_peered":"2024-04-25T10:25:43.124653+0000","last_unstale":"2024-04-25T10:25:44.135708+0000","last_undegraded":"2024-04-25T10:25:44.135708+0000","last_fullsized":"2024-04-25T10:25:44.135708+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:58.878077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.187758+0000","last_change":"2024-04-25T10:25:43.154646+0000","last_active":"2024-04-25T10:25:44.187758+0000","last_peered":"2024-04-25T10:25:44.187758+0000","last_clean":"2024-04-25T10:25:44.187758+0000","last_became_active":"2024-04-25T10:25:43.154219+0000","last_became_peered":"2024-04-25T10:25:43.154219+0000","last_unstale":"2024-04-25T10:25:44.187758+0000","last_undegraded":"2024-04-25T10:25:44.187758+0000","last_fullsized":"2024-04-25T10:25:44.187758+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.239476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135413+0000","last_change":"2024-04-25T10:25:43.145591+0000","last_active":"2024-04-25T10:25:44.135413+0000","last_peered":"2024-04-25T10:25:44.135413+0000","last_clean":"2024-04-25T10:25:44.135413+0000","last_became_active":"2024-04-25T10:25:43.144695+0000","last_became_peered":"2024-04-25T10:25:43.144695+0000","last_unstale":"2024-04-25T10:25:44.135413+0000","last_undegraded":"2024-04-25T10:25:44.135413+0000","last_fullsized":"2024-04-25T10:25:44.135413+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:13.979982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141391+0000","last_change":"2024-04-25T10:25:43.132964+0000","last_active":"2024-04-25T10:25:44.141391+0000","last_peered":"2024-04-25T10:25:44.141391+0000","last_clean":"2024-04-25T10:25:44.141391+0000","last_became_active":"2024-04-25T10:25:43.132706+0000","last_became_peered":"2024-04-25T10:25:43.132706+0000","last_unstale":"2024-04-25T10:25:44.141391+0000","last_undegraded":"2024-04-25T10:25:44.141391+0000","last_fullsized":"2024-04-25T10:25:44.141391+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:42.464851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188666+0000","last_change":"2024-04-25T10:25:43.144730+0000","last_active":"2024-04-25T10:25:44.188666+0000","last_peered":"2024-04-25T10:25:44.188666+0000","last_clean":"2024-04-25T10:25:44.188666+0000","last_became_active":"2024-04-25T10:25:43.144225+0000","last_became_peered":"2024-04-25T10:25:43.144225+0000","last_unstale":"2024-04-25T10:25:44.188666+0000","last_undegraded":"2024-04-25T10:25:44.188666+0000","last_fullsized":"2024-04-25T10:25:44.188666+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:16.401891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189073+0000","last_change":"2024-04-25T10:25:43.144207+0000","last_active":"2024-04-25T10:25:44.189073+0000","last_peered":"2024-04-25T10:25:44.189073+0000","last_clean":"2024-04-25T10:25:44.189073+0000","last_became_active":"2024-04-25T10:25:43.143941+0000","last_became_peered":"2024-04-25T10:25:43.143941+0000","last_unstale":"2024-04-25T10:25:44.189073+0000","last_undegraded":"2024-04-25T10:25:44.189073+0000","last_fullsized":"2024-04-25T10:25:44.189073+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:56.935728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189162+0000","last_change":"2024-04-25T10:25:43.137232+0000","last_active":"2024-04-25T10:25:44.189162+0000","last_peered":"2024-04-25T10:25:44.189162+0000","last_clean":"2024-04-25T10:25:44.189162+0000","last_became_active":"2024-04-25T10:25:43.136930+0000","last_became_peered":"2024-04-25T10:25:43.136930+0000","last_unstale":"2024-04-25T10:25:44.189162+0000","last_undegraded":"2024-04-25T10:25:44.189162+0000","last_fullsized":"2024-04-25T10:25:44.189162+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:13.770516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189356+0000","last_change":"2024-04-25T10:25:43.142802+0000","last_active":"2024-04-25T10:25:44.189356+0000","last_peered":"2024-04-25T10:25:44.189356+0000","last_clean":"2024-04-25T10:25:44.189356+0000","last_became_active":"2024-04-25T10:25:43.142129+0000","last_became_peered":"2024-04-25T10:25:43.142129+0000","last_unstale":"2024-04-25T10:25:44.189356+0000","last_undegraded":"2024-04-25T10:25:44.189356+0000","last_fullsized":"2024-04-25T10:25:44.189356+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:13.979982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188372+0000","last_change":"2024-04-25T10:25:43.152925+0000","last_active":"2024-04-25T10:25:44.188372+0000","last_peered":"2024-04-25T10:25:44.188372+0000","last_clean":"2024-04-25T10:25:44.188372+0000","last_became_active":"2024-04-25T10:25:43.151490+0000","last_became_peered":"2024-04-25T10:25:43.151490+0000","last_unstale":"2024-04-25T10:25:44.188372+0000","last_undegraded":"2024-04-25T10:25:44.188372+0000","last_fullsized":"2024-04-25T10:25:44.188372+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:59.209975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.187562+0000","last_change":"2024-04-25T10:25:43.146700+0000","last_active":"2024-04-25T10:25:44.187562+0000","last_peered":"2024-04-25T10:25:44.187562+0000","last_clean":"2024-04-25T10:25:44.187562+0000","last_became_active":"2024-04-25T10:25:43.146439+0000","last_became_peered":"2024-04-25T10:25:43.146439+0000","last_unstale":"2024-04-25T10:25:44.187562+0000","last_undegraded":"2024-04-25T10:25:44.187562+0000","last_fullsized":"2024-04-25T10:25:44.187562+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:38.877000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.187473+0000","last_change":"2024-04-25T10:25:43.138158+0000","last_active":"2024-04-25T10:25:44.187473+0000","last_peered":"2024-04-25T10:25:44.187473+0000","last_clean":"2024-04-25T10:25:44.187473+0000","last_became_active":"2024-04-25T10:25:43.137910+0000","last_became_peered":"2024-04-25T10:25:43.137910+0000","last_unstale":"2024-04-25T10:25:44.187473+0000","last_undegraded":"2024-04-25T10:25:44.187473+0000","last_fullsized":"2024-04-25T10:25:44.187473+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:59.209975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189379+0000","last_change":"2024-04-25T10:25:43.144664+0000","last_active":"2024-04-25T10:25:44.189379+0000","last_peered":"2024-04-25T10:25:44.189379+0000","last_clean":"2024-04-25T10:25:44.189379+0000","last_became_active":"2024-04-25T10:25:43.144059+0000","last_became_peered":"2024-04-25T10:25:43.144059+0000","last_unstale":"2024-04-25T10:25:44.189379+0000","last_undegraded":"2024-04-25T10:25:44.189379+0000","last_fullsized":"2024-04-25T10:25:44.189379+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:30.006327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188837+0000","last_change":"2024-04-25T10:25:43.137368+0000","last_active":"2024-04-25T10:25:44.188837+0000","last_peered":"2024-04-25T10:25:44.188837+0000","last_clean":"2024-04-25T10:25:44.188837+0000","last_became_active":"2024-04-25T10:25:43.137034+0000","last_became_peered":"2024-04-25T10:25:43.137034+0000","last_unstale":"2024-04-25T10:25:44.188837+0000","last_undegraded":"2024-04-25T10:25:44.188837+0000","last_fullsized":"2024-04-25T10:25:44.188837+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:39.245568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189260+0000","last_change":"2024-04-25T10:25:43.148129+0000","last_active":"2024-04-25T10:25:44.189260+0000","last_peered":"2024-04-25T10:25:44.189260+0000","last_clean":"2024-04-25T10:25:44.189260+0000","last_became_active":"2024-04-25T10:25:43.147005+0000","last_became_peered":"2024-04-25T10:25:43.147005+0000","last_unstale":"2024-04-25T10:25:44.189260+0000","last_undegraded":"2024-04-25T10:25:44.189260+0000","last_fullsized":"2024-04-25T10:25:44.189260+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:15.159079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135828+0000","last_change":"2024-04-25T10:25:43.142143+0000","last_active":"2024-04-25T10:25:44.135828+0000","last_peered":"2024-04-25T10:25:44.135828+0000","last_clean":"2024-04-25T10:25:44.135828+0000","last_became_active":"2024-04-25T10:25:43.141885+0000","last_became_peered":"2024-04-25T10:25:43.141885+0000","last_unstale":"2024-04-25T10:25:44.135828+0000","last_undegraded":"2024-04-25T10:25:44.135828+0000","last_fullsized":"2024-04-25T10:25:44.135828+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:10.570619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189393+0000","last_change":"2024-04-25T10:25:43.138698+0000","last_active":"2024-04-25T10:25:44.189393+0000","last_peered":"2024-04-25T10:25:44.189393+0000","last_clean":"2024-04-25T10:25:44.189393+0000","last_became_active":"2024-04-25T10:25:43.138424+0000","last_became_peered":"2024-04-25T10:25:43.138424+0000","last_unstale":"2024-04-25T10:25:44.189393+0000","last_undegraded":"2024-04-25T10:25:44.189393+0000","last_fullsized":"2024-04-25T10:25:44.189393+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:58.338227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140273+0000","last_change":"2024-04-25T10:25:43.146271+0000","last_active":"2024-04-25T10:25:44.140273+0000","last_peered":"2024-04-25T10:25:44.140273+0000","last_clean":"2024-04-25T10:25:44.140273+0000","last_became_active":"2024-04-25T10:25:43.146001+0000","last_became_peered":"2024-04-25T10:25:43.146001+0000","last_unstale":"2024-04-25T10:25:44.140273+0000","last_undegraded":"2024-04-25T10:25:44.140273+0000","last_fullsized":"2024-04-25T10:25:44.140273+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:17.641913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.139931+0000","last_change":"2024-04-25T10:25:43.146835+0000","last_active":"2024-04-25T10:25:44.139931+0000","last_peered":"2024-04-25T10:25:44.139931+0000","last_clean":"2024-04-25T10:25:44.139931+0000","last_became_active":"2024-04-25T10:25:43.146565+0000","last_became_peered":"2024-04-25T10:25:43.146565+0000","last_unstale":"2024-04-25T10:25:44.139931+0000","last_undegraded":"2024-04-25T10:25:44.139931+0000","last_fullsized":"2024-04-25T10:25:44.139931+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:18.015906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189674+0000","last_change":"2024-04-25T10:25:43.138159+0000","last_active":"2024-04-25T10:25:44.189674+0000","last_peered":"2024-04-25T10:25:44.189674+0000","last_clean":"2024-04-25T10:25:44.189674+0000","last_became_active":"2024-04-25T10:25:43.137871+0000","last_became_peered":"2024-04-25T10:25:43.137871+0000","last_unstale":"2024-04-25T10:25:44.189674+0000","last_undegraded":"2024-04-25T10:25:44.189674+0000","last_fullsized":"2024-04-25T10:25:44.189674+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:30.006327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140936+0000","last_change":"2024-04-25T10:25:43.137386+0000","last_active":"2024-04-25T10:25:44.140936+0000","last_peered":"2024-04-25T10:25:44.140936+0000","last_clean":"2024-04-25T10:25:44.140936+0000","last_became_active":"2024-04-25T10:25:43.137056+0000","last_became_peered":"2024-04-25T10:25:43.137056+0000","last_unstale":"2024-04-25T10:25:44.140936+0000","last_undegraded":"2024-04-25T10:25:44.140936+0000","last_fullsized":"2024-04-25T10:25:44.140936+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:30.913545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189028+0000","last_change":"2024-04-25T10:25:43.139515+0000","last_active":"2024-04-25T10:25:44.189028+0000","last_peered":"2024-04-25T10:25:44.189028+0000","last_clean":"2024-04-25T10:25:44.189028+0000","last_became_active":"2024-04-25T10:25:43.139208+0000","last_became_peered":"2024-04-25T10:25:43.139208+0000","last_unstale":"2024-04-25T10:25:44.189028+0000","last_undegraded":"2024-04-25T10:25:44.189028+0000","last_fullsized":"2024-04-25T10:25:44.189028+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:16.401891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189318+0000","last_change":"2024-04-25T10:25:43.137377+0000","last_active":"2024-04-25T10:25:44.189318+0000","last_peered":"2024-04-25T10:25:44.189318+0000","last_clean":"2024-04-25T10:25:44.189318+0000","last_became_active":"2024-04-25T10:25:43.137097+0000","last_became_peered":"2024-04-25T10:25:43.137097+0000","last_unstale":"2024-04-25T10:25:44.189318+0000","last_undegraded":"2024-04-25T10:25:44.189318+0000","last_fullsized":"2024-04-25T10:25:44.189318+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:28.603848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188555+0000","last_change":"2024-04-25T10:25:43.131490+0000","last_active":"2024-04-25T10:25:44.188555+0000","last_peered":"2024-04-25T10:25:44.188555+0000","last_clean":"2024-04-25T10:25:44.188555+0000","last_became_active":"2024-04-25T10:25:43.131219+0000","last_became_peered":"2024-04-25T10:25:43.131219+0000","last_unstale":"2024-04-25T10:25:44.188555+0000","last_undegraded":"2024-04-25T10:25:44.188555+0000","last_fullsized":"2024-04-25T10:25:44.188555+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:14.642264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189200+0000","last_change":"2024-04-25T10:25:43.135757+0000","last_active":"2024-04-25T10:25:44.189200+0000","last_peered":"2024-04-25T10:25:44.189200+0000","last_clean":"2024-04-25T10:25:44.189200+0000","last_became_active":"2024-04-25T10:25:43.135452+0000","last_became_peered":"2024-04-25T10:25:43.135452+0000","last_unstale":"2024-04-25T10:25:44.189200+0000","last_undegraded":"2024-04-25T10:25:44.189200+0000","last_fullsized":"2024-04-25T10:25:44.189200+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:10.652671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140690+0000","last_change":"2024-04-25T10:25:43.129833+0000","last_active":"2024-04-25T10:25:44.140690+0000","last_peered":"2024-04-25T10:25:44.140690+0000","last_clean":"2024-04-25T10:25:44.140690+0000","last_became_active":"2024-04-25T10:25:43.129544+0000","last_became_peered":"2024-04-25T10:25:43.129544+0000","last_unstale":"2024-04-25T10:25:44.140690+0000","last_undegraded":"2024-04-25T10:25:44.140690+0000","last_fullsized":"2024-04-25T10:25:44.140690+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:51.790362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.187578+0000","last_change":"2024-04-25T10:25:43.139656+0000","last_active":"2024-04-25T10:25:44.187578+0000","last_peered":"2024-04-25T10:25:44.187578+0000","last_clean":"2024-04-25T10:25:44.187578+0000","last_became_active":"2024-04-25T10:25:43.139370+0000","last_became_peered":"2024-04-25T10:25:43.139370+0000","last_unstale":"2024-04-25T10:25:44.187578+0000","last_undegraded":"2024-04-25T10:25:44.187578+0000","last_fullsized":"2024-04-25T10:25:44.187578+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:07.216559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189125+0000","last_change":"2024-04-25T10:25:43.141558+0000","last_active":"2024-04-25T10:25:44.189125+0000","last_peered":"2024-04-25T10:25:44.189125+0000","last_clean":"2024-04-25T10:25:44.189125+0000","last_became_active":"2024-04-25T10:25:43.139632+0000","last_became_peered":"2024-04-25T10:25:43.139632+0000","last_unstale":"2024-04-25T10:25:44.189125+0000","last_undegraded":"2024-04-25T10:25:44.189125+0000","last_fullsized":"2024-04-25T10:25:44.189125+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:04.086435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141212+0000","last_change":"2024-04-25T10:25:43.133546+0000","last_active":"2024-04-25T10:25:44.141212+0000","last_peered":"2024-04-25T10:25:44.141212+0000","last_clean":"2024-04-25T10:25:44.141212+0000","last_became_active":"2024-04-25T10:25:43.131822+0000","last_became_peered":"2024-04-25T10:25:43.131822+0000","last_unstale":"2024-04-25T10:25:44.141212+0000","last_undegraded":"2024-04-25T10:25:44.141212+0000","last_fullsized":"2024-04-25T10:25:44.141212+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:17.551910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"175'4","reported_seq":18,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189339+0000","last_change":"2024-04-25T10:25:42.131905+0000","last_active":"2024-04-25T10:25:44.189339+0000","last_peered":"2024-04-25T10:25:44.189339+0000","last_clean":"2024-04-25T10:25:44.189339+0000","last_became_active":"2024-04-25T10:25:42.131584+0000","last_became_peered":"2024-04-25T10:25:42.131584+0000","last_unstale":"2024-04-25T10:25:44.189339+0000","last_undegraded":"2024-04-25T10:25:44.189339+0000","last_fullsized":"2024-04-25T10:25:44.189339+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:16.549402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140296+0000","last_change":"2024-04-25T10:25:43.134845+0000","last_active":"2024-04-25T10:25:44.140296+0000","last_peered":"2024-04-25T10:25:44.140296+0000","last_clean":"2024-04-25T10:25:44.140296+0000","last_became_active":"2024-04-25T10:25:43.134548+0000","last_became_peered":"2024-04-25T10:25:43.134548+0000","last_unstale":"2024-04-25T10:25:44.140296+0000","last_undegraded":"2024-04-25T10:25:44.140296+0000","last_fullsized":"2024-04-25T10:25:44.140296+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.503089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"175'1","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141429+0000","last_change":"2024-04-25T10:25:42.134604+0000","last_active":"2024-04-25T10:25:44.141429+0000","last_peered":"2024-04-25T10:25:44.141429+0000","last_clean":"2024-04-25T10:25:44.141429+0000","last_became_active":"2024-04-25T10:25:42.133983+0000","last_became_peered":"2024-04-25T10:25:42.133983+0000","last_unstale":"2024-04-25T10:25:44.141429+0000","last_undegraded":"2024-04-25T10:25:44.141429+0000","last_fullsized":"2024-04-25T10:25:44.141429+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:11.584280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188836+0000","last_change":"2024-04-25T10:25:43.140236+0000","last_active":"2024-04-25T10:25:44.188836+0000","last_peered":"2024-04-25T10:25:44.188836+0000","last_clean":"2024-04-25T10:25:44.188836+0000","last_became_active":"2024-04-25T10:25:43.139821+0000","last_became_peered":"2024-04-25T10:25:43.139821+0000","last_unstale":"2024-04-25T10:25:44.188836+0000","last_undegraded":"2024-04-25T10:25:44.188836+0000","last_fullsized":"2024-04-25T10:25:44.188836+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:58.338227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"175'2","reported_seq":16,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189123+0000","last_change":"2024-04-25T10:25:42.134766+0000","last_active":"2024-04-25T10:25:44.189123+0000","last_peered":"2024-04-25T10:25:44.189123+0000","last_clean":"2024-04-25T10:25:44.189123+0000","last_became_active":"2024-04-25T10:25:42.134467+0000","last_became_peered":"2024-04-25T10:25:42.134467+0000","last_unstale":"2024-04-25T10:25:44.189123+0000","last_undegraded":"2024-04-25T10:25:44.189123+0000","last_fullsized":"2024-04-25T10:25:44.189123+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:38.079505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189072+0000","last_change":"2024-04-25T10:25:43.144690+0000","last_active":"2024-04-25T10:25:44.189072+0000","last_peered":"2024-04-25T10:25:44.189072+0000","last_clean":"2024-04-25T10:25:44.189072+0000","last_became_active":"2024-04-25T10:25:43.144407+0000","last_became_peered":"2024-04-25T10:25:43.144407+0000","last_unstale":"2024-04-25T10:25:44.189072+0000","last_undegraded":"2024-04-25T10:25:44.189072+0000","last_fullsized":"2024-04-25T10:25:44.189072+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:30.006327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"175'2","reported_seq":16,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189653+0000","last_change":"2024-04-25T10:25:42.140691+0000","last_active":"2024-04-25T10:25:44.189653+0000","last_peered":"2024-04-25T10:25:44.189653+0000","last_clean":"2024-04-25T10:25:44.189653+0000","last_became_active":"2024-04-25T10:25:42.140379+0000","last_became_peered":"2024-04-25T10:25:42.140379+0000","last_unstale":"2024-04-25T10:25:44.189653+0000","last_undegraded":"2024-04-25T10:25:44.189653+0000","last_fullsized":"2024-04-25T10:25:44.189653+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:52.631482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189604+0000","last_change":"2024-04-25T10:25:43.138860+0000","last_active":"2024-04-25T10:25:44.189604+0000","last_peered":"2024-04-25T10:25:44.189604+0000","last_clean":"2024-04-25T10:25:44.189604+0000","last_became_active":"2024-04-25T10:25:43.138600+0000","last_became_peered":"2024-04-25T10:25:43.138600+0000","last_unstale":"2024-04-25T10:25:44.189604+0000","last_undegraded":"2024-04-25T10:25:44.189604+0000","last_fullsized":"2024-04-25T10:25:44.189604+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:39.245568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"175'1","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135757+0000","last_change":"2024-04-25T10:25:42.123302+0000","last_active":"2024-04-25T10:25:44.135757+0000","last_peered":"2024-04-25T10:25:44.135757+0000","last_clean":"2024-04-25T10:25:44.135757+0000","last_became_active":"2024-04-25T10:25:42.122998+0000","last_became_peered":"2024-04-25T10:25:42.122998+0000","last_unstale":"2024-04-25T10:25:44.135757+0000","last_undegraded":"2024-04-25T10:25:44.135757+0000","last_fullsized":"2024-04-25T10:25:44.135757+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:10.531249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135464+0000","last_change":"2024-04-25T10:25:43.128559+0000","last_active":"2024-04-25T10:25:44.135464+0000","last_peered":"2024-04-25T10:25:44.135464+0000","last_clean":"2024-04-25T10:25:44.135464+0000","last_became_active":"2024-04-25T10:25:43.127910+0000","last_became_peered":"2024-04-25T10:25:43.127910+0000","last_unstale":"2024-04-25T10:25:44.135464+0000","last_undegraded":"2024-04-25T10:25:44.135464+0000","last_fullsized":"2024-04-25T10:25:44.135464+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:17.641913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135299+0000","last_change":"2024-04-25T10:25:42.132849+0000","last_active":"2024-04-25T10:25:44.135299+0000","last_peered":"2024-04-25T10:25:44.135299+0000","last_clean":"2024-04-25T10:25:44.135299+0000","last_became_active":"2024-04-25T10:25:42.132562+0000","last_became_peered":"2024-04-25T10:25:42.132562+0000","last_unstale":"2024-04-25T10:25:44.135299+0000","last_undegraded":"2024-04-25T10:25:44.135299+0000","last_fullsized":"2024-04-25T10:25:44.135299+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:53.536718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189221+0000","last_change":"2024-04-25T10:25:43.143561+0000","last_active":"2024-04-25T10:25:44.189221+0000","last_peered":"2024-04-25T10:25:44.189221+0000","last_clean":"2024-04-25T10:25:44.189221+0000","last_became_active":"2024-04-25T10:25:43.143230+0000","last_became_peered":"2024-04-25T10:25:43.143230+0000","last_unstale":"2024-04-25T10:25:44.189221+0000","last_undegraded":"2024-04-25T10:25:44.189221+0000","last_fullsized":"2024-04-25T10:25:44.189221+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:07.216559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.139556+0000","last_change":"2024-04-25T10:25:42.126070+0000","last_active":"2024-04-25T10:25:44.139556+0000","last_peered":"2024-04-25T10:25:44.139556+0000","last_clean":"2024-04-25T10:25:44.139556+0000","last_became_active":"2024-04-25T10:25:42.125798+0000","last_became_peered":"2024-04-25T10:25:42.125798+0000","last_unstale":"2024-04-25T10:25:44.139556+0000","last_undegraded":"2024-04-25T10:25:44.139556+0000","last_fullsized":"2024-04-25T10:25:44.139556+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:53.536718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189812+0000","last_change":"2024-04-25T10:25:43.135378+0000","last_active":"2024-04-25T10:25:44.189812+0000","last_peered":"2024-04-25T10:25:44.189812+0000","last_clean":"2024-04-25T10:25:44.189812+0000","last_became_active":"2024-04-25T10:25:43.135094+0000","last_became_peered":"2024-04-25T10:25:43.135094+0000","last_unstale":"2024-04-25T10:25:44.189812+0000","last_undegraded":"2024-04-25T10:25:44.189812+0000","last_fullsized":"2024-04-25T10:25:44.189812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.952166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135780+0000","last_change":"2024-04-25T10:25:42.129116+0000","last_active":"2024-04-25T10:25:44.135780+0000","last_peered":"2024-04-25T10:25:44.135780+0000","last_clean":"2024-04-25T10:25:44.135780+0000","last_became_active":"2024-04-25T10:25:42.125231+0000","last_became_peered":"2024-04-25T10:25:42.125231+0000","last_unstale":"2024-04-25T10:25:44.135780+0000","last_undegraded":"2024-04-25T10:25:44.135780+0000","last_fullsized":"2024-04-25T10:25:44.135780+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:44.234297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189433+0000","last_change":"2024-04-25T10:25:43.143803+0000","last_active":"2024-04-25T10:25:44.189433+0000","last_peered":"2024-04-25T10:25:44.189433+0000","last_clean":"2024-04-25T10:25:44.189433+0000","last_became_active":"2024-04-25T10:25:43.138921+0000","last_became_peered":"2024-04-25T10:25:43.138921+0000","last_unstale":"2024-04-25T10:25:44.189433+0000","last_undegraded":"2024-04-25T10:25:44.189433+0000","last_fullsized":"2024-04-25T10:25:44.189433+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:45.429707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140300+0000","last_change":"2024-04-25T10:25:42.125732+0000","last_active":"2024-04-25T10:25:44.140300+0000","last_peered":"2024-04-25T10:25:44.140300+0000","last_clean":"2024-04-25T10:25:44.140300+0000","last_became_active":"2024-04-25T10:25:42.125416+0000","last_became_peered":"2024-04-25T10:25:42.125416+0000","last_unstale":"2024-04-25T10:25:44.140300+0000","last_undegraded":"2024-04-25T10:25:44.140300+0000","last_fullsized":"2024-04-25T10:25:44.140300+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.076969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189551+0000","last_change":"2024-04-25T10:25:43.142616+0000","last_active":"2024-04-25T10:25:44.189551+0000","last_peered":"2024-04-25T10:25:44.189551+0000","last_clean":"2024-04-25T10:25:44.189551+0000","last_became_active":"2024-04-25T10:25:43.142312+0000","last_became_peered":"2024-04-25T10:25:43.142312+0000","last_unstale":"2024-04-25T10:25:44.189551+0000","last_undegraded":"2024-04-25T10:25:44.189551+0000","last_fullsized":"2024-04-25T10:25:44.189551+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:16.401891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"175'3","reported_seq":17,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189342+0000","last_change":"2024-04-25T10:25:42.138616+0000","last_active":"2024-04-25T10:25:44.189342+0000","last_peered":"2024-04-25T10:25:44.189342+0000","last_clean":"2024-04-25T10:25:44.189342+0000","last_became_active":"2024-04-25T10:25:42.138034+0000","last_became_peered":"2024-04-25T10:25:42.138034+0000","last_unstale":"2024-04-25T10:25:44.189342+0000","last_undegraded":"2024-04-25T10:25:44.189342+0000","last_fullsized":"2024-04-25T10:25:44.189342+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:21.782296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188199+0000","last_change":"2024-04-25T10:25:43.142519+0000","last_active":"2024-04-25T10:25:44.188199+0000","last_peered":"2024-04-25T10:25:44.188199+0000","last_clean":"2024-04-25T10:25:44.188199+0000","last_became_active":"2024-04-25T10:25:43.142136+0000","last_became_peered":"2024-04-25T10:25:43.142136+0000","last_unstale":"2024-04-25T10:25:44.188199+0000","last_undegraded":"2024-04-25T10:25:44.188199+0000","last_fullsized":"2024-04-25T10:25:44.188199+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:30.682081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"175'6","reported_seq":20,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189646+0000","last_change":"2024-04-25T10:25:42.138551+0000","last_active":"2024-04-25T10:25:44.189646+0000","last_peered":"2024-04-25T10:25:44.189646+0000","last_clean":"2024-04-25T10:25:44.189646+0000","last_became_active":"2024-04-25T10:25:42.136953+0000","last_became_peered":"2024-04-25T10:25:42.136953+0000","last_unstale":"2024-04-25T10:25:44.189646+0000","last_undegraded":"2024-04-25T10:25:44.189646+0000","last_fullsized":"2024-04-25T10:25:44.189646+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:02.424997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135955+0000","last_change":"2024-04-25T10:25:43.145608+0000","last_active":"2024-04-25T10:25:44.135955+0000","last_peered":"2024-04-25T10:25:44.135955+0000","last_clean":"2024-04-25T10:25:44.135955+0000","last_became_active":"2024-04-25T10:25:43.145252+0000","last_became_peered":"2024-04-25T10:25:43.145252+0000","last_unstale":"2024-04-25T10:25:44.135955+0000","last_undegraded":"2024-04-25T10:25:44.135955+0000","last_fullsized":"2024-04-25T10:25:44.135955+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:54.878880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"175'2","reported_seq":16,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140444+0000","last_change":"2024-04-25T10:25:42.131031+0000","last_active":"2024-04-25T10:25:44.140444+0000","last_peered":"2024-04-25T10:25:44.140444+0000","last_clean":"2024-04-25T10:25:44.140444+0000","last_became_active":"2024-04-25T10:25:42.130761+0000","last_became_peered":"2024-04-25T10:25:42.130761+0000","last_unstale":"2024-04-25T10:25:44.140444+0000","last_undegraded":"2024-04-25T10:25:44.140444+0000","last_fullsized":"2024-04-25T10:25:44.140444+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:10.531249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188591+0000","last_change":"2024-04-25T10:25:43.138919+0000","last_active":"2024-04-25T10:25:44.188591+0000","last_peered":"2024-04-25T10:25:44.188591+0000","last_clean":"2024-04-25T10:25:44.188591+0000","last_became_active":"2024-04-25T10:25:43.138622+0000","last_became_peered":"2024-04-25T10:25:43.138622+0000","last_unstale":"2024-04-25T10:25:44.188591+0000","last_undegraded":"2024-04-25T10:25:44.188591+0000","last_fullsized":"2024-04-25T10:25:44.188591+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:58.338227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.139867+0000","last_change":"2024-04-25T10:25:42.137736+0000","last_active":"2024-04-25T10:25:44.139867+0000","last_peered":"2024-04-25T10:25:44.139867+0000","last_clean":"2024-04-25T10:25:44.139867+0000","last_became_active":"2024-04-25T10:25:42.137395+0000","last_became_peered":"2024-04-25T10:25:42.137395+0000","last_unstale":"2024-04-25T10:25:44.139867+0000","last_undegraded":"2024-04-25T10:25:44.139867+0000","last_fullsized":"2024-04-25T10:25:44.139867+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:36.223426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188071+0000","last_change":"2024-04-25T10:25:43.139785+0000","last_active":"2024-04-25T10:25:44.188071+0000","last_peered":"2024-04-25T10:25:44.188071+0000","last_clean":"2024-04-25T10:25:44.188071+0000","last_became_active":"2024-04-25T10:25:43.139536+0000","last_became_peered":"2024-04-25T10:25:43.139536+0000","last_unstale":"2024-04-25T10:25:44.188071+0000","last_undegraded":"2024-04-25T10:25:44.188071+0000","last_fullsized":"2024-04-25T10:25:44.188071+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:46.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.187673+0000","last_change":"2024-04-25T10:25:42.131051+0000","last_active":"2024-04-25T10:25:44.187673+0000","last_peered":"2024-04-25T10:25:44.187673+0000","last_clean":"2024-04-25T10:25:44.187673+0000","last_became_active":"2024-04-25T10:25:42.130790+0000","last_became_peered":"2024-04-25T10:25:42.130790+0000","last_unstale":"2024-04-25T10:25:44.187673+0000","last_undegraded":"2024-04-25T10:25:44.187673+0000","last_fullsized":"2024-04-25T10:25:44.187673+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:13.613827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.187624+0000","last_change":"2024-04-25T10:25:43.141662+0000","last_active":"2024-04-25T10:25:44.187624+0000","last_peered":"2024-04-25T10:25:44.187624+0000","last_clean":"2024-04-25T10:25:44.187624+0000","last_became_active":"2024-04-25T10:25:43.141385+0000","last_became_peered":"2024-04-25T10:25:43.141385+0000","last_unstale":"2024-04-25T10:25:44.187624+0000","last_undegraded":"2024-04-25T10:25:44.187624+0000","last_fullsized":"2024-04-25T10:25:44.187624+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:01.250699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"175'8","reported_seq":22,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188258+0000","last_change":"2024-04-25T10:25:42.134855+0000","last_active":"2024-04-25T10:25:44.188258+0000","last_peered":"2024-04-25T10:25:44.188258+0000","last_clean":"2024-04-25T10:25:44.188258+0000","last_became_active":"2024-04-25T10:25:42.134577+0000","last_became_peered":"2024-04-25T10:25:42.134577+0000","last_unstale":"2024-04-25T10:25:44.188258+0000","last_undegraded":"2024-04-25T10:25:44.188258+0000","last_fullsized":"2024-04-25T10:25:44.188258+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:02.496086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141227+0000","last_change":"2024-04-25T10:25:43.132179+0000","last_active":"2024-04-25T10:25:44.141227+0000","last_peered":"2024-04-25T10:25:44.141227+0000","last_clean":"2024-04-25T10:25:44.141227+0000","last_became_active":"2024-04-25T10:25:43.131460+0000","last_became_peered":"2024-04-25T10:25:43.131460+0000","last_unstale":"2024-04-25T10:25:44.141227+0000","last_undegraded":"2024-04-25T10:25:44.141227+0000","last_fullsized":"2024-04-25T10:25:44.141227+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:21.045467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"175'1","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135810+0000","last_change":"2024-04-25T10:25:42.133808+0000","last_active":"2024-04-25T10:25:44.135810+0000","last_peered":"2024-04-25T10:25:44.135810+0000","last_clean":"2024-04-25T10:25:44.135810+0000","last_became_active":"2024-04-25T10:25:42.133534+0000","last_became_peered":"2024-04-25T10:25:42.133534+0000","last_unstale":"2024-04-25T10:25:44.135810+0000","last_undegraded":"2024-04-25T10:25:44.135810+0000","last_fullsized":"2024-04-25T10:25:44.135810+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:40.228139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141250+0000","last_change":"2024-04-25T10:25:43.137911+0000","last_active":"2024-04-25T10:25:44.141250+0000","last_peered":"2024-04-25T10:25:44.141250+0000","last_clean":"2024-04-25T10:25:44.141250+0000","last_became_active":"2024-04-25T10:25:43.137609+0000","last_became_peered":"2024-04-25T10:25:43.137609+0000","last_unstale":"2024-04-25T10:25:44.141250+0000","last_undegraded":"2024-04-25T10:25:44.141250+0000","last_fullsized":"2024-04-25T10:25:44.141250+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:54.539227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"175'2","reported_seq":16,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141316+0000","last_change":"2024-04-25T10:25:42.134838+0000","last_active":"2024-04-25T10:25:44.141316+0000","last_peered":"2024-04-25T10:25:44.141316+0000","last_clean":"2024-04-25T10:25:44.141316+0000","last_became_active":"2024-04-25T10:25:42.134334+0000","last_became_peered":"2024-04-25T10:25:42.134334+0000","last_unstale":"2024-04-25T10:25:44.141316+0000","last_undegraded":"2024-04-25T10:25:44.141316+0000","last_fullsized":"2024-04-25T10:25:44.141316+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:55.933219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188554+0000","last_change":"2024-04-25T10:25:43.133011+0000","last_active":"2024-04-25T10:25:44.188554+0000","last_peered":"2024-04-25T10:25:44.188554+0000","last_clean":"2024-04-25T10:25:44.188554+0000","last_became_active":"2024-04-25T10:25:43.132696+0000","last_became_peered":"2024-04-25T10:25:43.132696+0000","last_unstale":"2024-04-25T10:25:44.188554+0000","last_undegraded":"2024-04-25T10:25:44.188554+0000","last_fullsized":"2024-04-25T10:25:44.188554+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:01.467339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"175'1","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189338+0000","last_change":"2024-04-25T10:25:42.134480+0000","last_active":"2024-04-25T10:25:44.189338+0000","last_peered":"2024-04-25T10:25:44.189338+0000","last_clean":"2024-04-25T10:25:44.189338+0000","last_became_active":"2024-04-25T10:25:42.132569+0000","last_became_peered":"2024-04-25T10:25:42.132569+0000","last_unstale":"2024-04-25T10:25:44.189338+0000","last_undegraded":"2024-04-25T10:25:44.189338+0000","last_fullsized":"2024-04-25T10:25:44.189338+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:46.009995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189272+0000","last_change":"2024-04-25T10:25:43.138690+0000","last_active":"2024-04-25T10:25:44.189272+0000","last_peered":"2024-04-25T10:25:44.189272+0000","last_clean":"2024-04-25T10:25:44.189272+0000","last_became_active":"2024-04-25T10:25:43.138406+0000","last_became_peered":"2024-04-25T10:25:43.138406+0000","last_unstale":"2024-04-25T10:25:44.189272+0000","last_undegraded":"2024-04-25T10:25:44.189272+0000","last_fullsized":"2024-04-25T10:25:44.189272+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.952166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188611+0000","last_change":"2024-04-25T10:25:42.130026+0000","last_active":"2024-04-25T10:25:44.188611+0000","last_peered":"2024-04-25T10:25:44.188611+0000","last_clean":"2024-04-25T10:25:44.188611+0000","last_became_active":"2024-04-25T10:25:42.129672+0000","last_became_peered":"2024-04-25T10:25:42.129672+0000","last_unstale":"2024-04-25T10:25:44.188611+0000","last_undegraded":"2024-04-25T10:25:44.188611+0000","last_fullsized":"2024-04-25T10:25:44.188611+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:20.717274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188310+0000","last_change":"2024-04-25T10:25:43.144646+0000","last_active":"2024-04-25T10:25:44.188310+0000","last_peered":"2024-04-25T10:25:44.188310+0000","last_clean":"2024-04-25T10:25:44.188310+0000","last_became_active":"2024-04-25T10:25:43.140528+0000","last_became_peered":"2024-04-25T10:25:43.140528+0000","last_unstale":"2024-04-25T10:25:44.188310+0000","last_undegraded":"2024-04-25T10:25:44.188310+0000","last_fullsized":"2024-04-25T10:25:44.188310+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:33.816471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"175'2","reported_seq":16,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188637+0000","last_change":"2024-04-25T10:25:42.133873+0000","last_active":"2024-04-25T10:25:44.188637+0000","last_peered":"2024-04-25T10:25:44.188637+0000","last_clean":"2024-04-25T10:25:44.188637+0000","last_became_active":"2024-04-25T10:25:42.133614+0000","last_became_peered":"2024-04-25T10:25:42.133614+0000","last_unstale":"2024-04-25T10:25:44.188637+0000","last_undegraded":"2024-04-25T10:25:44.188637+0000","last_fullsized":"2024-04-25T10:25:44.188637+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:42.943516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188361+0000","last_change":"2024-04-25T10:25:43.134946+0000","last_active":"2024-04-25T10:25:44.188361+0000","last_peered":"2024-04-25T10:25:44.188361+0000","last_clean":"2024-04-25T10:25:44.188361+0000","last_became_active":"2024-04-25T10:25:43.134657+0000","last_became_peered":"2024-04-25T10:25:43.134657+0000","last_unstale":"2024-04-25T10:25:44.188361+0000","last_undegraded":"2024-04-25T10:25:44.188361+0000","last_fullsized":"2024-04-25T10:25:44.188361+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:44.397103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188726+0000","last_change":"2024-04-25T10:25:42.135624+0000","last_active":"2024-04-25T10:25:44.188726+0000","last_peered":"2024-04-25T10:25:44.188726+0000","last_clean":"2024-04-25T10:25:44.188726+0000","last_became_active":"2024-04-25T10:25:42.135288+0000","last_became_peered":"2024-04-25T10:25:42.135288+0000","last_unstale":"2024-04-25T10:25:44.188726+0000","last_undegraded":"2024-04-25T10:25:44.188726+0000","last_fullsized":"2024-04-25T10:25:44.188726+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:09.568111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140513+0000","last_change":"2024-04-25T10:25:43.124131+0000","last_active":"2024-04-25T10:25:44.140513+0000","last_peered":"2024-04-25T10:25:44.140513+0000","last_clean":"2024-04-25T10:25:44.140513+0000","last_became_active":"2024-04-25T10:25:43.123785+0000","last_became_peered":"2024-04-25T10:25:43.123785+0000","last_unstale":"2024-04-25T10:25:44.140513+0000","last_undegraded":"2024-04-25T10:25:44.140513+0000","last_fullsized":"2024-04-25T10:25:44.140513+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:33.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.187806+0000","last_change":"2024-04-25T10:25:42.136672+0000","last_active":"2024-04-25T10:25:44.187806+0000","last_peered":"2024-04-25T10:25:44.187806+0000","last_clean":"2024-04-25T10:25:44.187806+0000","last_became_active":"2024-04-25T10:25:42.136098+0000","last_became_peered":"2024-04-25T10:25:42.136098+0000","last_unstale":"2024-04-25T10:25:44.187806+0000","last_undegraded":"2024-04-25T10:25:44.187806+0000","last_fullsized":"2024-04-25T10:25:44.187806+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.639404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.139969+0000","last_change":"2024-04-25T10:25:43.133649+0000","last_active":"2024-04-25T10:25:44.139969+0000","last_peered":"2024-04-25T10:25:44.139969+0000","last_clean":"2024-04-25T10:25:44.139969+0000","last_became_active":"2024-04-25T10:25:43.133377+0000","last_became_peered":"2024-04-25T10:25:43.133377+0000","last_unstale":"2024-04-25T10:25:44.139969+0000","last_undegraded":"2024-04-25T10:25:44.139969+0000","last_fullsized":"2024-04-25T10:25:44.139969+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:28.603848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135468+0000","last_change":"2024-04-25T10:25:42.124417+0000","last_active":"2024-04-25T10:25:44.135468+0000","last_peered":"2024-04-25T10:25:44.135468+0000","last_clean":"2024-04-25T10:25:44.135468+0000","last_became_active":"2024-04-25T10:25:42.124140+0000","last_became_peered":"2024-04-25T10:25:42.124140+0000","last_unstale":"2024-04-25T10:25:44.135468+0000","last_undegraded":"2024-04-25T10:25:44.135468+0000","last_fullsized":"2024-04-25T10:25:44.135468+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:41.462342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189133+0000","last_change":"2024-04-25T10:25:43.135745+0000","last_active":"2024-04-25T10:25:44.189133+0000","last_peered":"2024-04-25T10:25:44.189133+0000","last_clean":"2024-04-25T10:25:44.189133+0000","last_became_active":"2024-04-25T10:25:43.135440+0000","last_became_peered":"2024-04-25T10:25:43.135440+0000","last_unstale":"2024-04-25T10:25:44.189133+0000","last_undegraded":"2024-04-25T10:25:44.189133+0000","last_fullsized":"2024-04-25T10:25:44.189133+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.531513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"175'2","reported_seq":16,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189465+0000","last_change":"2024-04-25T10:25:42.129404+0000","last_active":"2024-04-25T10:25:44.189465+0000","last_peered":"2024-04-25T10:25:44.189465+0000","last_clean":"2024-04-25T10:25:44.189465+0000","last_became_active":"2024-04-25T10:25:42.129112+0000","last_became_peered":"2024-04-25T10:25:42.129112+0000","last_unstale":"2024-04-25T10:25:44.189465+0000","last_undegraded":"2024-04-25T10:25:44.189465+0000","last_fullsized":"2024-04-25T10:25:44.189465+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:49.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189414+0000","last_change":"2024-04-25T10:25:43.141770+0000","last_active":"2024-04-25T10:25:44.189414+0000","last_peered":"2024-04-25T10:25:44.189414+0000","last_clean":"2024-04-25T10:25:44.189414+0000","last_became_active":"2024-04-25T10:25:43.141139+0000","last_became_peered":"2024-04-25T10:25:43.141139+0000","last_unstale":"2024-04-25T10:25:44.189414+0000","last_undegraded":"2024-04-25T10:25:44.189414+0000","last_fullsized":"2024-04-25T10:25:44.189414+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.952166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"176'1","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189974+0000","last_change":"2024-04-25T10:25:42.137741+0000","last_active":"2024-04-25T10:25:44.189974+0000","last_peered":"2024-04-25T10:25:44.189974+0000","last_clean":"2024-04-25T10:25:44.189974+0000","last_became_active":"2024-04-25T10:25:42.137454+0000","last_became_peered":"2024-04-25T10:25:42.137454+0000","last_unstale":"2024-04-25T10:25:44.189974+0000","last_undegraded":"2024-04-25T10:25:44.189974+0000","last_fullsized":"2024-04-25T10:25:44.189974+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:21.305119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.141002+0000","last_change":"2024-04-25T10:25:43.146382+0000","last_active":"2024-04-25T10:25:44.141002+0000","last_peered":"2024-04-25T10:25:44.141002+0000","last_clean":"2024-04-25T10:25:44.141002+0000","last_became_active":"2024-04-25T10:25:43.146066+0000","last_became_peered":"2024-04-25T10:25:43.146066+0000","last_unstale":"2024-04-25T10:25:44.141002+0000","last_undegraded":"2024-04-25T10:25:44.141002+0000","last_fullsized":"2024-04-25T10:25:44.141002+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:03.867697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135641+0000","last_change":"2024-04-25T10:25:42.133683+0000","last_active":"2024-04-25T10:25:44.135641+0000","last_peered":"2024-04-25T10:25:44.135641+0000","last_clean":"2024-04-25T10:25:44.135641+0000","last_became_active":"2024-04-25T10:25:42.132547+0000","last_became_peered":"2024-04-25T10:25:42.132547+0000","last_unstale":"2024-04-25T10:25:44.135641+0000","last_undegraded":"2024-04-25T10:25:44.135641+0000","last_fullsized":"2024-04-25T10:25:44.135641+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.076969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140060+0000","last_change":"2024-04-25T10:25:43.147668+0000","last_active":"2024-04-25T10:25:44.140060+0000","last_peered":"2024-04-25T10:25:44.140060+0000","last_clean":"2024-04-25T10:25:44.140060+0000","last_became_active":"2024-04-25T10:25:43.147378+0000","last_became_peered":"2024-04-25T10:25:43.147378+0000","last_unstale":"2024-04-25T10:25:44.140060+0000","last_undegraded":"2024-04-25T10:25:44.140060+0000","last_fullsized":"2024-04-25T10:25:44.140060+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:13.979982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188692+0000","last_change":"2024-04-25T10:25:42.129418+0000","last_active":"2024-04-25T10:25:44.188692+0000","last_peered":"2024-04-25T10:25:44.188692+0000","last_clean":"2024-04-25T10:25:44.188692+0000","last_became_active":"2024-04-25T10:25:42.129122+0000","last_became_peered":"2024-04-25T10:25:42.129122+0000","last_unstale":"2024-04-25T10:25:44.188692+0000","last_undegraded":"2024-04-25T10:25:44.188692+0000","last_fullsized":"2024-04-25T10:25:44.188692+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:52.916483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188630+0000","last_change":"2024-04-25T10:25:43.153061+0000","last_active":"2024-04-25T10:25:44.188630+0000","last_peered":"2024-04-25T10:25:44.188630+0000","last_clean":"2024-04-25T10:25:44.188630+0000","last_became_active":"2024-04-25T10:25:43.152800+0000","last_became_peered":"2024-04-25T10:25:43.152800+0000","last_unstale":"2024-04-25T10:25:44.188630+0000","last_undegraded":"2024-04-25T10:25:44.188630+0000","last_fullsized":"2024-04-25T10:25:44.188630+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:33.816471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189512+0000","last_change":"2024-04-25T10:25:42.139424+0000","last_active":"2024-04-25T10:25:44.189512+0000","last_peered":"2024-04-25T10:25:44.189512+0000","last_clean":"2024-04-25T10:25:44.189512+0000","last_became_active":"2024-04-25T10:25:42.139056+0000","last_became_peered":"2024-04-25T10:25:42.139056+0000","last_unstale":"2024-04-25T10:25:44.189512+0000","last_undegraded":"2024-04-25T10:25:44.189512+0000","last_fullsized":"2024-04-25T10:25:44.189512+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:59.539838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189215+0000","last_change":"2024-04-25T10:25:43.142991+0000","last_active":"2024-04-25T10:25:44.189215+0000","last_peered":"2024-04-25T10:25:44.189215+0000","last_clean":"2024-04-25T10:25:44.189215+0000","last_became_active":"2024-04-25T10:25:43.142513+0000","last_became_peered":"2024-04-25T10:25:43.142513+0000","last_unstale":"2024-04-25T10:25:44.189215+0000","last_undegraded":"2024-04-25T10:25:44.189215+0000","last_fullsized":"2024-04-25T10:25:44.189215+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:18.015906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.135859+0000","last_change":"2024-04-25T10:25:42.125349+0000","last_active":"2024-04-25T10:25:44.135859+0000","last_peered":"2024-04-25T10:25:44.135859+0000","last_clean":"2024-04-25T10:25:44.135859+0000","last_became_active":"2024-04-25T10:25:42.124906+0000","last_became_peered":"2024-04-25T10:25:42.124906+0000","last_unstale":"2024-04-25T10:25:44.135859+0000","last_undegraded":"2024-04-25T10:25:44.135859+0000","last_fullsized":"2024-04-25T10:25:44.135859+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:36.223426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188208+0000","last_change":"2024-04-25T10:25:43.140944+0000","last_active":"2024-04-25T10:25:44.188208+0000","last_peered":"2024-04-25T10:25:44.188208+0000","last_clean":"2024-04-25T10:25:44.188208+0000","last_became_active":"2024-04-25T10:25:43.140672+0000","last_became_peered":"2024-04-25T10:25:43.140672+0000","last_unstale":"2024-04-25T10:25:44.188208+0000","last_undegraded":"2024-04-25T10:25:44.188208+0000","last_fullsized":"2024-04-25T10:25:44.188208+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:23.310665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.139992+0000","last_change":"2024-04-25T10:25:42.129649+0000","last_active":"2024-04-25T10:25:44.139992+0000","last_peered":"2024-04-25T10:25:44.139992+0000","last_clean":"2024-04-25T10:25:44.139992+0000","last_became_active":"2024-04-25T10:25:42.129385+0000","last_became_peered":"2024-04-25T10:25:42.129385+0000","last_unstale":"2024-04-25T10:25:44.139992+0000","last_undegraded":"2024-04-25T10:25:44.139992+0000","last_fullsized":"2024-04-25T10:25:44.139992+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:41.462342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.140588+0000","last_change":"2024-04-25T10:25:43.135141+0000","last_active":"2024-04-25T10:25:44.140588+0000","last_peered":"2024-04-25T10:25:44.140588+0000","last_clean":"2024-04-25T10:25:44.140588+0000","last_became_active":"2024-04-25T10:25:43.134824+0000","last_became_peered":"2024-04-25T10:25:43.134824+0000","last_unstale":"2024-04-25T10:25:44.140588+0000","last_undegraded":"2024-04-25T10:25:44.140588+0000","last_fullsized":"2024-04-25T10:25:44.140588+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:55.632218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188944+0000","last_change":"2024-04-25T10:25:42.138558+0000","last_active":"2024-04-25T10:25:44.188944+0000","last_peered":"2024-04-25T10:25:44.188944+0000","last_clean":"2024-04-25T10:25:44.188944+0000","last_became_active":"2024-04-25T10:25:42.138125+0000","last_became_peered":"2024-04-25T10:25:42.138125+0000","last_unstale":"2024-04-25T10:25:44.188944+0000","last_undegraded":"2024-04-25T10:25:44.188944+0000","last_fullsized":"2024-04-25T10:25:44.188944+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:53.536718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188505+0000","last_change":"2024-04-25T10:25:43.147703+0000","last_active":"2024-04-25T10:25:44.188505+0000","last_peered":"2024-04-25T10:25:44.188505+0000","last_clean":"2024-04-25T10:25:44.188505+0000","last_became_active":"2024-04-25T10:25:43.147424+0000","last_became_peered":"2024-04-25T10:25:43.147424+0000","last_unstale":"2024-04-25T10:25:44.188505+0000","last_undegraded":"2024-04-25T10:25:44.188505+0000","last_fullsized":"2024-04-25T10:25:44.188505+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:42.102395+0000","last_clean_scrub_stamp":"2024-04-25T10:25:42.102395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:07.216559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"175'1","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.189292+0000","last_change":"2024-04-25T10:25:42.135634+0000","last_active":"2024-04-25T10:25:44.189292+0000","last_peered":"2024-04-25T10:25:44.189292+0000","last_clean":"2024-04-25T10:25:44.189292+0000","last_became_active":"2024-04-25T10:25:42.135346+0000","last_became_peered":"2024-04-25T10:25:42.135346+0000","last_unstale":"2024-04-25T10:25:44.189292+0000","last_undegraded":"2024-04-25T10:25:44.189292+0000","last_fullsized":"2024-04-25T10:25:44.189292+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:41.099887+0000","last_clean_scrub_stamp":"2024-04-25T10:25:41.099887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:39.115480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":453,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-04-25T10:25:44.188883+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:25:44.188883+0000","last_peered":"2024-04-25T10:25:44.188883+0000","last_clean":"2024-04-25T10:25:44.188883+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:25:44.188883+0000","last_undegraded":"2024-04-25T10:25:44.188883+0000","last_fullsized":"2024-04-25T10:25:44.188883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575013,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64468,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307372,"statfs":{"total":96636764160,"available":96570748928,"internally_reserved":0,"allocated":2826240,"data_stored":1567628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65224,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306616,"statfs":{"total":96636764160,"available":96569974784,"internally_reserved":0,"allocated":2224128,"data_stored":976230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51400,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320440,"statfs":{"total":96636764160,"available":96584130560,"internally_reserved":0,"allocated":2224128,"data_stored":976230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59736,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312104,"statfs":{"total":96636764160,"available":96575594496,"internally_reserved":0,"allocated":2240512,"data_stored":977364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575015,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59736,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312104,"statfs":{"total":96636764160,"available":96575594496,"internally_reserved":0,"allocated":2232320,"data_stored":975830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575014,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2220032,"data_stored":975648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575014,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69460,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302380,"statfs":{"total":96636764160,"available":96565637120,"internally_reserved":0,"allocated":2228224,"data_stored":975704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56140,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315700,"statfs":{"total":96636764160,"available":96579276800,"internally_reserved":0,"allocated":2818048,"data_stored":1566106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:25:47.569 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:25:47.569 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:25:47.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.751+0000 7f7726000640 1 Processor -- start 2024-04-25T10:25:47.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.751+0000 7f7726000640 1 -- start start 2024-04-25T10:25:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.751+0000 7f7726000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f772015bbb0 0x7f77200cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.751+0000 7f7726000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200b4660 0x7f772006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.751+0000 7f7726000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772006ca90 con 0x7f77200cf960 2024-04-25T10:25:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.751+0000 7f7726000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772006d050 con 0x7f772015bbb0 2024-04-25T10:25:47.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.751+0000 7f7726000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772006cd70 con 0x7f77200b4660 2024-04-25T10:25:47.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f7716a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200b4660 0x7f772006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f771f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f772015bbb0 0x7f77200cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f771fe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f77200cf960 0x7f77200b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:48028/0 (socket says 172.21.15.87:48028) 2024-04-25T10:25:47.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f771fe00640 1 -- 172.21.15.87:0/2390451549 learned_addr learned my addr 172.21.15.87:0/2390451549 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:47.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f7716a00640 1 -- 172.21.15.87:0/2390451549 >> v2:172.21.15.169:3300/0 conn(0x7f772015bbb0 msgr2=0x7f77200cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:47.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f7716a00640 1 --2- 172.21.15.87:0/2390451549 >> v2:172.21.15.169:3300/0 conn(0x7f772015bbb0 0x7f77200cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f7716a00640 1 -- 172.21.15.87:0/2390451549 >> v1:172.21.15.169:6789/0 conn(0x7f77200cf960 legacy=0x7f77200b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f7716a00640 1 -- 172.21.15.87:0/2390451549 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200b2880 con 0x7f77200b4660 2024-04-25T10:25:47.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f771f400640 1 --2- 172.21.15.87:0/2390451549 >> v2:172.21.15.169:3300/0 conn(0x7f772015bbb0 0x7f77200cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:47.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f7716a00640 1 --2- 172.21.15.87:0/2390451549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200b4660 0x7f772006c120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f77080097c0 tx=0x7f770801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9332a6f8f8181182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f771ea00640 1 -- 172.21.15.87:0/2390451549 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080b2070 con 0x7f77200b4660 2024-04-25T10:25:47.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f771ea00640 1 -- 172.21.15.87:0/2390451549 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080ad640 con 0x7f77200b4660 2024-04-25T10:25:47.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.755+0000 7f771ea00640 1 -- 172.21.15.87:0/2390451549 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080ada10 con 0x7f77200b4660 2024-04-25T10:25:47.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- 172.21.15.87:0/2390451549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200b4660 msgr2=0x7f772006c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 --2- 172.21.15.87:0/2390451549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200b4660 0x7f772006c120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f77080097c0 tx=0x7f770801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- 172.21.15.87:0/2390451549 shutdown_connections 2024-04-25T10:25:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 --2- 172.21.15.87:0/2390451549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200b4660 0x7f772006c120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 --2- 172.21.15.87:0/2390451549 >> v2:172.21.15.169:3300/0 conn(0x7f772015bbb0 0x7f77200cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- 172.21.15.87:0/2390451549 >> 172.21.15.87:0/2390451549 conn(0x7f7720066b70 msgr2=0x7f7720066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- 172.21.15.87:0/2390451549 shutdown_connections 2024-04-25T10:25:47.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- 172.21.15.87:0/2390451549 wait complete. 2024-04-25T10:25:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 Processor -- start 2024-04-25T10:25:47.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- start start 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77200b4660 0x7f772014be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200cf960 0x7f7720070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f772015bbb0 0x7f772014e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77200ce140 con 0x7f772015bbb0 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77200cdfc0 con 0x7f77200b4660 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7726000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772006c660 con 0x7f77200cf960 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f771fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f772015bbb0 0x7f772014e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f771fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f772015bbb0 0x7f772014e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37218/0 (socket says 172.21.15.87:37218) 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f771fe00640 1 -- 172.21.15.87:0/3156004979 learned_addr learned my addr 172.21.15.87:0/3156004979 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f7716a00640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200cf960 0x7f7720070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.759+0000 7f771f400640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77200b4660 0x7f772014be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f771fe00640 1 -- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200cf960 msgr2=0x7f7720070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f771fe00640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200cf960 0x7f7720070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f771fe00640 1 -- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77200b4660 msgr2=0x7f772014be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f771fe00640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77200b4660 0x7f772014be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f771fe00640 1 -- 172.21.15.87:0/3156004979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772015d600 con 0x7f772015bbb0 2024-04-25T10:25:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f771fe00640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f772015bbb0 0x7f772014e0a0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f771800ca80 tx=0x7f771800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7717e00640 1 -- 172.21.15.87:0/3156004979 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718017070 con 0x7f772015bbb0 2024-04-25T10:25:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7717e00640 1 -- 172.21.15.87:0/3156004979 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771800abd0 con 0x7f772015bbb0 2024-04-25T10:25:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7717e00640 1 -- 172.21.15.87:0/3156004979 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718012400 con 0x7f772015bbb0 2024-04-25T10:25:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772015d8e0 con 0x7f772015bbb0 2024-04-25T10:25:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772015dea0 con 0x7f772015bbb0 2024-04-25T10:25:47.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772004ef30 con 0x7f772015bbb0 2024-04-25T10:25:47.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7717e00640 1 -- 172.21.15.87:0/3156004979 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f771801a070 con 0x7f772015bbb0 2024-04-25T10:25:47.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7717e00640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76fc066f90 0x7f76fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f7717e00640 1 -- 172.21.15.87:0/3156004979 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7718084c00 con 0x7f772015bbb0 2024-04-25T10:25:47.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.763+0000 7f771f400640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76fc066f90 0x7f76fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.767+0000 7f771f400640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76fc066f90 0x7f76fc069450 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7710009580 tx=0x7f7710009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.767+0000 7f7717e00640 1 -- 172.21.15.87:0/3156004979 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77180555b0 con 0x7f772015bbb0 2024-04-25T10:25:47.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.975+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f772006d980 con 0x7f772015bbb0 2024-04-25T10:25:47.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.975+0000 7f7717e00640 1 -- 172.21.15.87:0/3156004979 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7718059650 con 0x7f772015bbb0 2024-04-25T10:25:47.977 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:25:47.977 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:25:47.977 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:25:47.977 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:25:47.977 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:25:47.978 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76fc066f90 msgr2=0x7f76fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76fc066f90 0x7f76fc069450 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7710009580 tx=0x7f7710009190 comp rx=0 tx=0).stop 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f772015bbb0 msgr2=0x7f772014e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f772015bbb0 0x7f772014e0a0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f771800ca80 tx=0x7f771800cf40 comp rx=0 tx=0).stop 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 shutdown_connections 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f76fc066f90 0x7f76fc069450 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f772015bbb0 0x7f772014e0a0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77200cf960 0x7f7720070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 --2- 172.21.15.87:0/3156004979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77200b4660 0x7f772014be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 >> 172.21.15.87:0/3156004979 conn(0x7f7720066b70 msgr2=0x7f772015abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 shutdown_connections 2024-04-25T10:25:47.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:47.979+0000 7f7726000640 1 -- 172.21.15.87:0/3156004979 wait complete. 2024-04-25T10:25:47.999 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:48.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fdaa0000640 1 Processor -- start 2024-04-25T10:25:48.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fdaa0000640 1 -- start start 2024-04-25T10:25:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fdaa0000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fda9815b910 0x7fda9815c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fdaa0000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9815bd60 con 0x7fda980d0040 2024-04-25T10:25:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fdaa0000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda98071050 con 0x7fda981550b0 2024-04-25T10:25:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fdaa0000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda980712f0 con 0x7fda9815b910 2024-04-25T10:25:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fda9cc00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fda980d0040 0x7fda9815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38350/0 (socket says 172.21.15.87:38350) 2024-04-25T10:25:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fda9e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fda9815b910 0x7fda9815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fda9cc00640 1 -- 172.21.15.87:0/3412497154 learned_addr learned my addr 172.21.15.87:0/3412497154 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:48.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fda9e000640 1 -- 172.21.15.87:0/3412497154 >> v1:172.21.15.169:6789/0 conn(0x7fda981550b0 legacy=0x7fda980cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fda9e000640 1 -- 172.21.15.87:0/3412497154 >> v1:172.21.15.87:6789/0 conn(0x7fda980d0040 legacy=0x7fda9815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fda97e00640 1 -- 172.21.15.87:0/3412497154 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750643399 0 0) 0x7fda9815bd60 con 0x7fda980d0040 2024-04-25T10:25:48.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.135+0000 7fda9e000640 1 -- 172.21.15.87:0/3412497154 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda980715d0 con 0x7fda9815b910 2024-04-25T10:25:48.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fda9e000640 1 --2- 172.21.15.87:0/3412497154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda9815b910 0x7fda9815c030 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fda900097c0 tx=0x7fda9001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca39df8c123dd12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fda97e00640 1 -- 172.21.15.87:0/3412497154 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900a7070 con 0x7fda9815b910 2024-04-25T10:25:48.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fda97e00640 1 -- 172.21.15.87:0/3412497154 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda900b0070 con 0x7fda9815b910 2024-04-25T10:25:48.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fda97e00640 1 -- 172.21.15.87:0/3412497154 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900ab890 con 0x7fda9815b910 2024-04-25T10:25:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fdaa0000640 1 -- 172.21.15.87:0/3412497154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda9815b910 msgr2=0x7fda9815c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fdaa0000640 1 --2- 172.21.15.87:0/3412497154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda9815b910 0x7fda9815c030 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fda900097c0 tx=0x7fda9001c470 comp rx=0 tx=0).stop 2024-04-25T10:25:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fdaa0000640 1 -- 172.21.15.87:0/3412497154 shutdown_connections 2024-04-25T10:25:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fdaa0000640 1 --2- 172.21.15.87:0/3412497154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda9815b910 0x7fda9815c030 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fdaa0000640 1 -- 172.21.15.87:0/3412497154 >> 172.21.15.87:0/3412497154 conn(0x7fda98066b70 msgr2=0x7fda98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fdaa0000640 1 -- 172.21.15.87:0/3412497154 shutdown_connections 2024-04-25T10:25:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.139+0000 7fdaa0000640 1 -- 172.21.15.87:0/3412497154 wait complete. 2024-04-25T10:25:48.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 Processor -- start 2024-04-25T10:25:48.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 -- start start 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda980d0040 0x7fda9815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981550b0 0x7fda981554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda9815b910 0x7fda980cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9815cf00 con 0x7fda980d0040 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9815cd80 con 0x7fda9815b910 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9815d080 con 0x7fda981550b0 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda980d0040 0x7fda9815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda980d0040 0x7fda9815a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37242/0 (socket says 172.21.15.87:37242) 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 -- 172.21.15.87:0/533792862 learned_addr learned my addr 172.21.15.87:0/533792862 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9e000640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda9815b910 0x7fda980cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9cc00640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981550b0 0x7fda981554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 -- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981550b0 msgr2=0x7fda981554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981550b0 0x7fda981554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 -- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda9815b910 msgr2=0x7fda980cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda9815b910 0x7fda980cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 -- 172.21.15.87:0/533792862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda98155a30 con 0x7fda980d0040 2024-04-25T10:25:48.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda9d600640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda980d0040 0x7fda9815a5f0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c7f0 tx=0x7fda8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda95600640 1 -- 172.21.15.87:0/533792862 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c003b80 con 0x7fda980d0040 2024-04-25T10:25:48.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda95600640 1 -- 172.21.15.87:0/533792862 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c003d20 con 0x7fda980d0040 2024-04-25T10:25:48.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda980cc8c0 con 0x7fda980d0040 2024-04-25T10:25:48.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fda95600640 1 -- 172.21.15.87:0/533792862 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00aa90 con 0x7fda980d0040 2024-04-25T10:25:48.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.143+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda980cce80 con 0x7fda980d0040 2024-04-25T10:25:48.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.147+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9804ef30 con 0x7fda980d0040 2024-04-25T10:25:48.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.147+0000 7fda95600640 1 -- 172.21.15.87:0/533792862 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda8c00ac30 con 0x7fda980d0040 2024-04-25T10:25:48.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.147+0000 7fda95600640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda78066f60 0x7fda78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.147+0000 7fda95600640 1 -- 172.21.15.87:0/533792862 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda8c084ef0 con 0x7fda980d0040 2024-04-25T10:25:48.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.147+0000 7fda9cc00640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda78066f60 0x7fda78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.147+0000 7fda9cc00640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda78066f60 0x7fda78069420 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fda981f9ab0 tx=0x7fda84007240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.151+0000 7fda95600640 1 -- 172.21.15.87:0/533792862 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda8c055810 con 0x7fda980d0040 2024-04-25T10:25:48.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.339+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda980ce3f0 con 0x7fda980d0040 2024-04-25T10:25:48.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.339+0000 7fda95600640 1 -- 172.21.15.87:0/533792862 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fda8c0598b0 con 0x7fda980d0040 2024-04-25T10:25:48.341 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:48.341 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":128,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1787812144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1787812144},{"type":"v1","addr":"172.21.15.169:6834","nonce":1787812144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/554048725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":554048725},{"type":"v1","addr":"172.21.15.169:6835","nonce":554048725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1720179922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1720179922},{"type":"v1","addr":"172.21.15.87:6837","nonce":1720179922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:42.498010+0000","modified":"2024-04-25T10:25:45.516622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10920":{"gid":10920,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/3053712751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3053712751},{"type":"v1","addr":"172.21.15.87:6836","nonce":3053712751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:25:48.341 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 128 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda78066f60 msgr2=0x7fda78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda78066f60 0x7fda78069420 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fda981f9ab0 tx=0x7fda84007240 comp rx=0 tx=0).stop 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda980d0040 msgr2=0x7fda9815a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda980d0040 0x7fda9815a5f0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c7f0 tx=0x7fda8c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 shutdown_connections 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fda78066f60 0x7fda78069420 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda9815b910 0x7fda980cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981550b0 0x7fda981554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 --2- 172.21.15.87:0/533792862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda980d0040 0x7fda9815a5f0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 >> 172.21.15.87:0/533792862 conn(0x7fda98066b70 msgr2=0x7fda98066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 shutdown_connections 2024-04-25T10:25:48.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.343+0000 7fdaa0000640 1 -- 172.21.15.87:0/533792862 wait complete. 2024-04-25T10:25:48.765 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:25:42.498010+0000', 'modified': '2024-04-25T10:25:45.516622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10920}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10920': {'gid': 10920, 'name': 'a', 'rank': 0, 'incarnation': 124, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6836/3053712751', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 3053712751}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 3053712751}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:25:48.765 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:25:48.765 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:25:48.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.903+0000 7fcdf6800640 1 Processor -- start 2024-04-25T10:25:48.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.903+0000 7fcdf6800640 1 -- start start 2024-04-25T10:25:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdf6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcdf00b0f70 0x7fcdf00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdf6800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcdf00cd2e0 0x7fcdf00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdf6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf015f3b0 con 0x7fcdf00cfd90 2024-04-25T10:25:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdf6800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf015ef20 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdf6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00ca810 con 0x7fcdf00b0f70 2024-04-25T10:25:48.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcdf00cd2e0 0x7fcdf00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcdf00cd2e0 0x7fcdf00cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37246/0 (socket says 172.21.15.87:37246) 2024-04-25T10:25:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 -- 172.21.15.87:0/2918071699 learned_addr learned my addr 172.21.15.87:0/2918071699 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 -- 172.21.15.87:0/2918071699 >> v2:172.21.15.169:3300/0 conn(0x7fcdf00b0f70 msgr2=0x7fcdf00b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 --2- 172.21.15.87:0/2918071699 >> v2:172.21.15.169:3300/0 conn(0x7fcdf00b0f70 0x7fcdf00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 -- 172.21.15.87:0/2918071699 >> v1:172.21.15.169:6789/0 conn(0x7fcdf00cfd90 legacy=0x7fcdf00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 -- 172.21.15.87:0/2918071699 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00b1890 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.907+0000 7fcdefe00640 1 --2- 172.21.15.87:0/2918071699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00cba20 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0071230 tx=0x7fcde801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4343fd18bc2bfe2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdeea00640 1 -- 172.21.15.87:0/2918071699 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80b0070 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdeea00640 1 -- 172.21.15.87:0/2918071699 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80ab600 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdeea00640 1 -- 172.21.15.87:0/2918071699 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80ab9d0 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- 172.21.15.87:0/2918071699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 msgr2=0x7fcdf00cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 --2- 172.21.15.87:0/2918071699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00cba20 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0071230 tx=0x7fcde801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- 172.21.15.87:0/2918071699 shutdown_connections 2024-04-25T10:25:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 --2- 172.21.15.87:0/2918071699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00cba20 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 --2- 172.21.15.87:0/2918071699 >> v2:172.21.15.169:3300/0 conn(0x7fcdf00b0f70 0x7fcdf00b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- 172.21.15.87:0/2918071699 >> 172.21.15.87:0/2918071699 conn(0x7fcdf00b1d50 msgr2=0x7fcdf00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- 172.21.15.87:0/2918071699 shutdown_connections 2024-04-25T10:25:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- 172.21.15.87:0/2918071699 wait complete. 2024-04-25T10:25:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 Processor -- start 2024-04-25T10:25:48.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- start start 2024-04-25T10:25:48.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00b0f70 0x7fcdf00c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cfd90 0x7fcdf00996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0161b50 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf01619d0 con 0x7fcdf00cfd90 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdf6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0161cd0 con 0x7fcdf00b0f70 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37250/0 (socket says 172.21.15.87:37250) 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcde6a00640 1 -- 172.21.15.87:0/95105725 learned_addr learned my addr 172.21.15.87:0/95105725 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdef400640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00b0f70 0x7fcdf00c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.911+0000 7fcdefe00640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cfd90 0x7fcdf00996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde6a00640 1 -- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00b0f70 msgr2=0x7fcdf00c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde6a00640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00b0f70 0x7fcdf00c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde6a00640 1 -- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cfd90 msgr2=0x7fcdf00996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde6a00640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cfd90 0x7fcdf00996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde6a00640 1 -- 172.21.15.87:0/95105725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00c4720 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde6a00640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00c41e0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b9c0 tx=0x7fcdd800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde7e00640 1 -- 172.21.15.87:0/95105725 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800cba0 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde7e00640 1 -- 172.21.15.87:0/95105725 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd800cd40 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde7e00640 1 -- 172.21.15.87:0/95105725 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8003bc0 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf00c6360 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf00c6920 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde7e00640 1 -- 172.21.15.87:0/95105725 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdd8006d10 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde7e00640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdcc066f90 0x7fcdcc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.915+0000 7fcde7e00640 1 -- 172.21.15.87:0/95105725 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdd8084ba0 con 0x7fcdf00cd2e0 2024-04-25T10:25:48.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 Processor -- start 2024-04-25T10:25:48.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdef400640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdcc066f90 0x7fcdcc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- start start 2024-04-25T10:25:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdef400640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdcc066f90 0x7fcdcc069450 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009990 tx=0x7fcde0009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d986299d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d98624f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d986250310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9861375f0 con 0x55d98624ff30 2024-04-25T10:25:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98609eda0 con 0x55d98629a240 2024-04-25T10:25:48.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d986027150 con 0x55d986299920 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d986250310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdef400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d986299d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d986250310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37252/0 (socket says 172.21.15.87:37252) 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 -- 172.21.15.87:0/3753291166 learned_addr learned my addr 172.21.15.87:0/3753291166 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcde6a00640 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d98624f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 -- 172.21.15.87:0/3753291166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 msgr2=0x55d986299d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d986299d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 -- 172.21.15.87:0/3753291166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 msgr2=0x55d98624f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d98624f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 -- 172.21.15.87:0/3753291166 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d986040ca0 con 0x55d98624ff30 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdefe00640 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d986250310 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fcde801a110 tx=0x7fcde8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b38ea7ebe7385c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcddfe00640 1 -- 172.21.15.87:0/3753291166 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80b0040 con 0x55d98624ff30 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcddfe00640 1 -- 172.21.15.87:0/3753291166 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80abaa0 con 0x55d98624ff30 2024-04-25T10:25:48.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcddfe00640 1 -- 172.21.15.87:0/3753291166 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80b5440 con 0x55d98624ff30 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3753291166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 msgr2=0x55d986250310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d986250310 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fcde801a110 tx=0x7fcde8004740 comp rx=0 tx=0).stop 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3753291166 shutdown_connections 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d986250310 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d98624f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3753291166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d986299d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3753291166 >> 172.21.15.87:0/3753291166 conn(0x55d9862562a0 msgr2=0x55d98624e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3753291166 shutdown_connections 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3753291166 wait complete. 2024-04-25T10:25:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.919+0000 7fcdf9d1f000 1 -- start start 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 Processor -- start 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98624ff30 0x55d9862a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98629a240 0x55d986253aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d986059990 con 0x55d986299920 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98609ae40 con 0x55d98624ff30 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d98602a900 con 0x55d98629a240 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 10 client.0 _ll_register_callbacks cb 0x55d9861521c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37256/0 (socket says 172.21.15.87:37256) 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 -- 172.21.15.87:0/3116388634 learned_addr learned my addr 172.21.15.87:0/3116388634 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdef400640 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98624ff30 0x55d9862a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdefe00640 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98629a240 0x55d986253aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 -- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98629a240 msgr2=0x55d986253aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98629a240 0x55d986253aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 -- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98624ff30 msgr2=0x55d9862a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98624ff30 0x55d9862a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 -- 172.21.15.87:0/3116388634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d986057630 con 0x55d986299920 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcde6a00640 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862a11c0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8082fd0 tx=0x7fcdd8059620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e0a3a35eadb4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcddcc00640 1 -- 172.21.15.87:0/3116388634 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8059360 con 0x55d986299920 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcddcc00640 1 -- 172.21.15.87:0/3116388634 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdd805d890 con 0x55d986299920 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcde80abaa0 con 0x55d986299920 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcddcc00640 1 -- 172.21.15.87:0/3116388634 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdd8053880 con 0x55d986299920 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 10 client.11064 fetch_fsmap learned FSMap version 128 2024-04-25T10:25:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x55d9860273a0 con 0x55d986299920 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcddcc00640 1 -- 172.21.15.87:0/3116388634 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80abaa0 con 0x55d986299920 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcddcc00640 1 -- 172.21.15.87:0/3116388634 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcdd8054910 con 0x55d986299920 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 10 client.11064 fetch_fsmap finished waiting for FSMap version 128 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 10 client.11064 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 20 client.11064 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d986255160 0x55d9862aac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 4 client.11064 mds_command: new command op to 10920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 --> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d98603aa50 con 0x55d986255160 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdef400640 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d986255160 0x55d9862aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcdef400640 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d986255160 0x55d9862aac70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b77357970d4060a server_cookie=351806b42460c5ca in_seq=0 out_seq=0 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.923+0000 7fcddcc00640 10 client.11064 ms_handle_connect on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcddcc00640 1 -- 172.21.15.87:0/3116388634 <== mds.0 v2:172.21.15.87:6834/3053712751 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d98603aa50 con 0x55d986255160 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcddcc00640 10 client.11064 handle_command_reply: tid=0 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 client.11064 shutdown 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 msgr2=0x55d9862a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862a11c0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8082fd0 tx=0x7fcdd8059620 comp rx=0 tx=0).stop 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 shutdown_connections 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98629a240 0x55d986253aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862a11c0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d986255160 0x55d9862aac70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/3116388634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98624ff30 0x55d9862a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 >> 172.21.15.87:0/3116388634 conn(0x55d9862562a0 msgr2=0x55d98624e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcddcc00640 0 client.11064 ms_handle_reset on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 shutdown_connections 2024-04-25T10:25:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/3116388634 wait complete. 2024-04-25T10:25:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.927+0000 7fcdf9d1f000 20 client.11064 trim_cache size 0 max 16384 2024-04-25T10:25:48.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.939+0000 7fcdf9d1f000 1 Processor -- start 2024-04-25T10:25:48.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.939+0000 7fcdf9d1f000 1 -- start start 2024-04-25T10:25:48.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98624ff30 0x55d9862f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0202200 con 0x55d986299920 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0156b80 con 0x55d98629a240 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0201c00 con 0x55d98624ff30 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37272/0 (socket says 172.21.15.87:37272) 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 -- 172.21.15.87:0/2791803846 learned_addr learned my addr 172.21.15.87:0/2791803846 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdef400640 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98624ff30 0x55d9862f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdefe00640 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 -- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98624ff30 msgr2=0x55d9862f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98624ff30 0x55d9862f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 -- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 msgr2=0x55d9862f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 -- 172.21.15.87:0/2791803846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98629a070 con 0x55d986299920 2024-04-25T10:25:48.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcde6a00640 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862f50a0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8051e40 tx=0x7fcdd8059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ae72a5b1197893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcddf400640 1 -- 172.21.15.87:0/2791803846 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8055000 con 0x55d986299920 2024-04-25T10:25:48.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcddf400640 1 -- 172.21.15.87:0/2791803846 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd805db20 con 0x55d986299920 2024-04-25T10:25:48.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcddf400640 1 -- 172.21.15.87:0/2791803846 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80554c0 con 0x55d986299920 2024-04-25T10:25:48.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/2791803846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 msgr2=0x55d9862f50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862f50a0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8051e40 tx=0x7fcdd8059000 comp rx=0 tx=0).stop 2024-04-25T10:25:48.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/2791803846 shutdown_connections 2024-04-25T10:25:48.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d986299920 0x55d9862f50a0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/2791803846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d98624ff30 0x55d9862f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/2791803846 >> 172.21.15.87:0/2791803846 conn(0x55d9862562a0 msgr2=0x55d986285110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/2791803846 shutdown_connections 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/2791803846 wait complete. 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 -- start start 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.943+0000 7fcdf9d1f000 1 Processor -- start 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d9862a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d9862a6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9860a33d0 con 0x55d98624ff30 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98615ad50 con 0x55d98629a240 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d986039980 con 0x55d986299920 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 10 client.0 _ll_register_callbacks cb 0x7fcde80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d9862a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d9862a5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37282/0 (socket says 172.21.15.87:37282) 2024-04-25T10:25:48.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 -- 172.21.15.87:0/1110937001 learned_addr learned my addr 172.21.15.87:0/1110937001 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcde6a00640 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d9862a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdefe00640 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 -- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 msgr2=0x55d9862a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d9862a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 -- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 msgr2=0x55d9862dd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862dd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 -- 172.21.15.87:0/1110937001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d986120600 con 0x55d98624ff30 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdef400640 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d9862a5d50 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x55d9860a2990 tx=0x7fcde0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126109d67815f002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 1 -- 172.21.15.87:0/1110937001 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde001b040 con 0x55d98624ff30 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 1 -- 172.21.15.87:0/1110937001 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde0015950 con 0x55d98624ff30 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdd805db20 con 0x55d98624ff30 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 1 -- 172.21.15.87:0/1110937001 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde0039050 con 0x55d98624ff30 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 10 client.11070 fetch_fsmap learned FSMap version 128 2024-04-25T10:25:48.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x55d9860339b0 con 0x55d98624ff30 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 1 -- 172.21.15.87:0/1110937001 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd805db20 con 0x55d98624ff30 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 1 -- 172.21.15.87:0/1110937001 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcde003c020 con 0x55d98624ff30 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 10 client.11070 fetch_fsmap finished waiting for FSMap version 128 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 10 client.11070 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 20 client.11070 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d9862dd8f0 0x55d9862de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 4 client.11070 mds_command: new command op to 10920 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 --> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d9860ac2e0 con 0x55d9862dd8f0 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcde6a00640 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d9862dd8f0 0x55d9862de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcde6a00640 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d9862dd8f0 0x55d9862de1c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f99cb3a9598de server_cookie=b960807b77a7218a in_seq=0 out_seq=0 2024-04-25T10:25:48.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 10 client.11070 ms_handle_connect on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 1 -- 172.21.15.87:0/1110937001 <== mds.0 v2:172.21.15.87:6834/3053712751 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55d9860ac2e0 con 0x55d9862dd8f0 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 10 client.11070 handle_command_reply: tid=0 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 client.11070 shutdown 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 msgr2=0x55d9862a5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d9862a5d50 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x55d9860a2990 tx=0x7fcde0011000 comp rx=0 tx=0).stop 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 shutdown_connections 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55d9862dd8f0 0x55d9862de1c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d98629a240 0x55d9862dd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d986299920 0x55d9862a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcddcc00640 0 client.11070 ms_handle_reset on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:48.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 --2- 172.21.15.87:0/1110937001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d98624ff30 0x55d9862a5d50 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 >> 172.21.15.87:0/1110937001 conn(0x55d9862562a0 msgr2=0x55d9862860b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 shutdown_connections 2024-04-25T10:25:48.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 1 -- 172.21.15.87:0/1110937001 wait complete. 2024-04-25T10:25:48.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf9d1f000 20 client.11070 trim_cache size 0 max 16384 2024-04-25T10:25:48.953 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 10920, 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8365850239999997, 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 128, 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 176, 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 176, 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.682122828000001 2024-04-25T10:25:48.954 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:25:48.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdcc066f90 msgr2=0x7fcdcc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf6800640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdcc066f90 0x7fcdcc069450 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009990 tx=0x7fcde0009110 comp rx=0 tx=0).stop 2024-04-25T10:25:48.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 msgr2=0x7fcdf00c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.947+0000 7fcdf6800640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00c41e0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b9c0 tx=0x7fcdd800be80 comp rx=0 tx=0).stop 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 shutdown_connections 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcdcc066f90 0x7fcdcc069450 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf00cfd90 0x7fcdf00996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdf00cd2e0 0x7fcdf00c41e0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 --2- 172.21.15.87:0/95105725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf00b0f70 0x7fcdf00c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 >> 172.21.15.87:0/95105725 conn(0x7fcdf00b1d50 msgr2=0x7fcdf01608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 shutdown_connections 2024-04-25T10:25:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:48.951+0000 7fcdf6800640 1 -- 172.21.15.87:0/95105725 wait complete. 2024-04-25T10:25:48.968 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:25:49.089 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:25:49.091+0000 7f39f1a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:25:49.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1158800640 1 Processor -- start 2024-04-25T10:25:49.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1158800640 1 -- start start 2024-04-25T10:25:49.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1158800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f11500cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1158800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f11500cfe70 0x7f115015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1158800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1150071050 con 0x7f115015b740 2024-04-25T10:25:49.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1158800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115015d040 con 0x7f11500cfe70 2024-04-25T10:25:49.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1158800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11500712f0 con 0x7f11501550b0 2024-04-25T10:25:49.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1156800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f115015b740 0x7f115006d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38358/0 (socket says 172.21.15.87:38358) 2024-04-25T10:25:49.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1156800640 1 -- 172.21.15.87:0/3451593196 learned_addr learned my addr 172.21.15.87:0/3451593196 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:49.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1155e00640 1 --2- 172.21.15.87:0/3451593196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f11500cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.103+0000 7f1155400640 1 --2- 172.21.15.87:0/3451593196 >> v2:172.21.15.169:3300/0 conn(0x7f11500cfe70 0x7f115015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1154a00640 1 -- 172.21.15.87:0/3451593196 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069004455 0 0) 0x7f1150071050 con 0x7f115015b740 2024-04-25T10:25:49.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1154a00640 1 -- 172.21.15.87:0/3451593196 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1138003530 con 0x7f115015b740 2024-04-25T10:25:49.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1155e00640 1 -- 172.21.15.87:0/3451593196 >> v2:172.21.15.169:3300/0 conn(0x7f11500cfe70 msgr2=0x7f115015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:49.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1155e00640 1 --2- 172.21.15.87:0/3451593196 >> v2:172.21.15.169:3300/0 conn(0x7f11500cfe70 0x7f115015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1155e00640 1 -- 172.21.15.87:0/3451593196 >> v1:172.21.15.87:6789/0 conn(0x7f115015b740 legacy=0x7f115006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1155e00640 1 -- 172.21.15.87:0/3451593196 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500715d0 con 0x7f11501550b0 2024-04-25T10:25:49.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1155400640 1 --2- 172.21.15.87:0/3451593196 >> v2:172.21.15.169:3300/0 conn(0x7f11500cfe70 0x7f115015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:49.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.107+0000 7f1155e00640 1 --2- 172.21.15.87:0/3451593196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f11500cf930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f11440097c0 tx=0x7f114401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aafdfc5f65ac753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:49.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1154a00640 1 -- 172.21.15.87:0/3451593196 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440b0070 con 0x7f11501550b0 2024-04-25T10:25:49.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1154a00640 1 -- 172.21.15.87:0/3451593196 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11440ab600 con 0x7f11501550b0 2024-04-25T10:25:49.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1154a00640 1 -- 172.21.15.87:0/3451593196 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440ab9b0 con 0x7f11501550b0 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- 172.21.15.87:0/3451593196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 msgr2=0x7f11500cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 --2- 172.21.15.87:0/3451593196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f11500cf930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f11440097c0 tx=0x7f114401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- 172.21.15.87:0/3451593196 shutdown_connections 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 --2- 172.21.15.87:0/3451593196 >> v2:172.21.15.169:3300/0 conn(0x7f11500cfe70 0x7f115015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 --2- 172.21.15.87:0/3451593196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f11500cf930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- 172.21.15.87:0/3451593196 >> 172.21.15.87:0/3451593196 conn(0x7f1150066b70 msgr2=0x7f1150066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- 172.21.15.87:0/3451593196 shutdown_connections 2024-04-25T10:25:49.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- 172.21.15.87:0/3451593196 wait complete. 2024-04-25T10:25:49.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 Processor -- start 2024-04-25T10:25:49.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- start start 2024-04-25T10:25:49.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11500cfe70 0x7f1150169e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f115015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f115015b740 0x7f1150168d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115015bcd0 con 0x7f11500cfe70 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115015bb50 con 0x7f115015b740 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1158800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115015c030 con 0x7f11501550b0 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1155e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11500cfe70 0x7f1150169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1155e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11500cfe70 0x7f1150169e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37294/0 (socket says 172.21.15.87:37294) 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1155e00640 1 -- 172.21.15.87:0/1810587108 learned_addr learned my addr 172.21.15.87:0/1810587108 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1156800640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f115015b740 0x7f1150168d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.111+0000 7f1155400640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f115015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1155e00640 1 -- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 msgr2=0x7f115015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1155e00640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f115015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1155e00640 1 -- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f115015b740 msgr2=0x7f1150168d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1155e00640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f115015b740 0x7f1150168d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1155e00640 1 -- 172.21.15.87:0/1810587108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150071dd0 con 0x7f11500cfe70 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1155400640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f115015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1155e00640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11500cfe70 0x7f1150169e90 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f11440ae040 tx=0x7f1144019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f114a000640 1 -- 172.21.15.87:0/1810587108 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440b0040 con 0x7f11500cfe70 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f114a000640 1 -- 172.21.15.87:0/1810587108 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11440bb070 con 0x7f11500cfe70 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f114a000640 1 -- 172.21.15.87:0/1810587108 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440abc10 con 0x7f11500cfe70 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11500cd950 con 0x7f11500cfe70 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1150168560 con 0x7f11500cfe70 2024-04-25T10:25:49.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115004ef30 con 0x7f11500cfe70 2024-04-25T10:25:49.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.115+0000 7f114a000640 1 -- 172.21.15.87:0/1810587108 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11440b3070 con 0x7f11500cfe70 2024-04-25T10:25:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.119+0000 7f114a000640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1134066f60 0x7f1134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.119+0000 7f114a000640 1 -- 172.21.15.87:0/1810587108 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f114411f700 con 0x7f11500cfe70 2024-04-25T10:25:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.119+0000 7f1155400640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1134066f60 0x7f1134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.119+0000 7f1155400640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1134066f60 0x7f1134069420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f115014b9d0 tx=0x7f1140008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:49.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.123+0000 7f114a000640 1 -- 172.21.15.87:0/1810587108 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1144016090 con 0x7f11500cfe70 2024-04-25T10:25:49.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f115014fc30 con 0x7f11500cfe70 2024-04-25T10:25:49.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f114a000640 1 -- 172.21.15.87:0/1810587108 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v128) v1 ==== 53+0+73 (secure 0 0 0) 0x7f11440f06f0 con 0x7f11500cfe70 2024-04-25T10:25:49.310 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:25:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1134066f60 msgr2=0x7f1134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1134066f60 0x7f1134069420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f115014b9d0 tx=0x7f1140008040 comp rx=0 tx=0).stop 2024-04-25T10:25:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11500cfe70 msgr2=0x7f1150169e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11500cfe70 0x7f1150169e90 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f11440ae040 tx=0x7f1144019240 comp rx=0 tx=0).stop 2024-04-25T10:25:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 shutdown_connections 2024-04-25T10:25:49.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1134066f60 0x7f1134069420 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f115015b740 0x7f1150168d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11501550b0 0x7f115015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 --2- 172.21.15.87:0/1810587108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11500cfe70 0x7f1150169e90 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 >> 172.21.15.87:0/1810587108 conn(0x7f1150066b70 msgr2=0x7f11500b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:49.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 shutdown_connections 2024-04-25T10:25:49.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:49.311+0000 7f1158800640 1 -- 172.21.15.87:0/1810587108 wait complete. 2024-04-25T10:25:49.332 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:25:49.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:25:49.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:25:49.333 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:25:49.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:49.333 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:25:49.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:25:49.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:25:49.333 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:25:49.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.333 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:25:49.343 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41664sec preferred_lft 41664sec 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:25:49.344 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:49.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.346 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:25:49.402 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:25:49.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.404 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:25:49.456 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:25:49.457 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:25:49.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.457 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:25:49.457 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:25:49.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:25:49.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:25:49.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:25:49.457 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:25:49.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.544 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:25:49.553 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:25:49.553 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:25:49.553 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:25:49.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:25:49.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:25:49.615 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:49.664 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:25:49.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:25:49.665 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:49.717 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:25:49.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.717 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:25:49.808 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:25:49.809 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:25:49.809 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:25:49.824 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:25:49.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:25:49.824 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:25:49.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:25:49.824 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:25:49.834 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:25:49.834 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:25:49.834 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:25:49.834 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:25:49.834 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:25:49.835 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:25:49.835 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:25:49.835 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:25:49.839 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:25:49.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.839 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:49.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:49.858 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:25:49.915 INFO:teuthology.orchestra.run.smithi087.stdout:client10048 2024-04-25T10:25:49.915 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:49.915 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:49.965 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:49.965 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:50.016 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> import uuid 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> n = 1000 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> create_hard_links = False 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> if 0 > 0: 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> create_hard_links = True 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> dpath = os.path.dirname(path) 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> fnameprefix = os.path.basename(path) 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:25:50.017 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> for i in range(n): 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> f.write(f"{i}") 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> f.flush() 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(f.fileno()) 2024-04-25T10:25:50.018 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> os.unlink(fpath) 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(dirfd) 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> if create_hard_links: 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> for j in range(0): 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> os.fsync(dirfd) 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> finally: 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> os.close(dirfd) 2024-04-25T10:25:50.019 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T10:25:50.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:25:50.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:50.811 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T10:25:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54b1000640 1 Processor -- start 2024-04-25T10:25:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54b1000640 1 -- start start 2024-04-25T10:25:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54b1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac15e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54b1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f54ac0cfb50 0x7f54ac070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54b1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0ad840 con 0x7f54ac071bc0 2024-04-25T10:25:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54b1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac15bb90 con 0x7f54ac0cfb50 2024-04-25T10:25:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54b1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac15e710 con 0x7f54ac0b3480 2024-04-25T10:25:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f54ac0cfb50 0x7f54ac070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:47338/0 (socket says 172.21.15.87:47338) 2024-04-25T10:25:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a9e00640 1 -- 172.21.15.87:0/3347489100 learned_addr learned my addr 172.21.15.87:0/3347489100 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54aa800640 1 --1- 172.21.15.87:0/3347489100 >> v1:172.21.15.169:6789/0 conn(0x7f54ac071bc0 0x7f54ac0add20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:48042/0 (socket says 172.21.15.87:48042) 2024-04-25T10:25:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a9e00640 1 -- 172.21.15.87:0/3347489100 >> v2:172.21.15.169:3300/0 conn(0x7f54ac0cfb50 msgr2=0x7f54ac070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a9e00640 1 --2- 172.21.15.87:0/3347489100 >> v2:172.21.15.169:3300/0 conn(0x7f54ac0cfb50 0x7f54ac070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a9e00640 1 -- 172.21.15.87:0/3347489100 >> v1:172.21.15.169:6789/0 conn(0x7f54ac071bc0 legacy=0x7f54ac0add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.995+0000 7f54a9e00640 1 -- 172.21.15.87:0/3347489100 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac15e9f0 con 0x7f54ac0b3480 2024-04-25T10:25:50.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54a9e00640 1 --2- 172.21.15.87:0/3347489100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac15e130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1329aea9dfafa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54a9400640 1 -- 172.21.15.87:0/3347489100 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0b2070 con 0x7f54ac0b3480 2024-04-25T10:25:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54a9400640 1 -- 172.21.15.87:0/3347489100 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c0a9070 con 0x7f54ac0b3480 2024-04-25T10:25:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54a9400640 1 -- 172.21.15.87:0/3347489100 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0ad880 con 0x7f54ac0b3480 2024-04-25T10:25:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 -- 172.21.15.87:0/3347489100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 msgr2=0x7f54ac15e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 --2- 172.21.15.87:0/3347489100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac15e130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c01c560 comp rx=0 tx=0).stop 2024-04-25T10:25:50.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 -- 172.21.15.87:0/3347489100 shutdown_connections 2024-04-25T10:25:51.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 --2- 172.21.15.87:0/3347489100 >> v2:172.21.15.169:3300/0 conn(0x7f54ac0cfb50 0x7f54ac070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 --2- 172.21.15.87:0/3347489100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac15e130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 -- 172.21.15.87:0/3347489100 >> 172.21.15.87:0/3347489100 conn(0x7f54ac0669a0 msgr2=0x7f54ac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 -- 172.21.15.87:0/3347489100 shutdown_connections 2024-04-25T10:25:51.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:50.999+0000 7f54b1000640 1 -- 172.21.15.87:0/3347489100 wait complete. 2024-04-25T10:25:51.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 Processor -- start 2024-04-25T10:25:51.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 -- start start 2024-04-25T10:25:51.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ac071bc0 0x7f54ac0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54ac0cfb50 0x7f54ac1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0723e0 con 0x7f54ac071bc0 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0ae840 con 0x7f54ac0cfb50 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac151980 con 0x7f54ac0b3480 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ac071bc0 0x7f54ac0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ac071bc0 0x7f54ac0aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37304/0 (socket says 172.21.15.87:37304) 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 -- 172.21.15.87:0/3041330490 learned_addr learned my addr 172.21.15.87:0/3041330490 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54aa800640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54ac0cfb50 0x7f54ac1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 -- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 msgr2=0x7f54ac150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 -- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54ac0cfb50 msgr2=0x7f54ac1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54ac0cfb50 0x7f54ac1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 -- 172.21.15.87:0/3041330490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac150ad0 con 0x7f54ac071bc0 2024-04-25T10:25:51.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a9e00640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ac071bc0 0x7f54ac0aff20 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f549c0b0040 tx=0x7f549c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a2000640 1 -- 172.21.15.87:0/3041330490 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0b2040 con 0x7f54ac071bc0 2024-04-25T10:25:51.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a2000640 1 -- 172.21.15.87:0/3041330490 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c0be070 con 0x7f54ac071bc0 2024-04-25T10:25:51.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54a2000640 1 -- 172.21.15.87:0/3041330490 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0b8420 con 0x7f54ac071bc0 2024-04-25T10:25:51.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac150d70 con 0x7f54ac071bc0 2024-04-25T10:25:51.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.003+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac151330 con 0x7f54ac071bc0 2024-04-25T10:25:51.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a2000640 1 -- 172.21.15.87:0/3041330490 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f549c0ab070 con 0x7f54ac071bc0 2024-04-25T10:25:51.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a2000640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5488066f60 0x7f5488069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a2000640 1 -- 172.21.15.87:0/3041330490 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f549c11efc0 con 0x7f54ac071bc0 2024-04-25T10:25:51.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5488066f60 0x7f5488069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 Processor -- start 2024-04-25T10:25:51.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 -- start start 2024-04-25T10:25:51.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5488066f60 0x7f5488069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f54ac1fa570 tx=0x7f5494009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85a98db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85a4eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a4f0a0 0x55da85a4f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8589bff0 con 0x55da85a4f0a0 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da85824150 con 0x55da85a992f0 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da85898090 con 0x55da85a989d0 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a4f0a0 0x55da85a4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a4f0a0 0x55da85a4f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37308/0 (socket says 172.21.15.87:37308) 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 -- 172.21.15.87:0/2471711748 learned_addr learned my addr 172.21.15.87:0/2471711748 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a3e00640 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85a4eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a9e00640 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85a98db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 -- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 msgr2=0x55da85a98db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85a98db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 -- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 msgr2=0x55da85a4eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85a4eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 -- 172.21.15.87:0/2471711748 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da85a320d0 con 0x55da85a4f0a0 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a9e00640 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85a98db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54a3e00640 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85a4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.007+0000 7f54aa800640 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a4f0a0 0x55da85a4f480 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f54a4001b70 tx=0x7f54a4002030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f90e04522a681c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f549aa00640 1 -- 172.21.15.87:0/2471711748 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4007850 con 0x55da85a4f0a0 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f549aa00640 1 -- 172.21.15.87:0/2471711748 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4018070 con 0x55da85a4f0a0 2024-04-25T10:25:51.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f549aa00640 1 -- 172.21.15.87:0/2471711748 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a40133f0 con 0x55da85a4f0a0 2024-04-25T10:25:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- 172.21.15.87:0/2471711748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a4f0a0 msgr2=0x55da85a4f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a4f0a0 0x55da85a4f480 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f54a4001b70 tx=0x7f54a4002030 comp rx=0 tx=0).stop 2024-04-25T10:25:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- 172.21.15.87:0/2471711748 shutdown_connections 2024-04-25T10:25:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a4f0a0 0x55da85a4f480 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85a4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 --2- 172.21.15.87:0/2471711748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85a98db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- 172.21.15.87:0/2471711748 >> 172.21.15.87:0/2471711748 conn(0x55da85a4b020 msgr2=0x55da85a4d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- 172.21.15.87:0/2471711748 shutdown_connections 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- 172.21.15.87:0/2471711748 wait complete. 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- start start 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 Processor -- start 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 0x55da85a7d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85a7dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a992f0 0x55da85a7e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da85860250 con 0x55da85a989d0 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da85856690 con 0x55da85a4f0a0 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da85824a40 con 0x55da85a992f0 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 10 client.0 _ll_register_callbacks cb 0x55da85a39e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85a7dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85a7dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37318/0 (socket says 172.21.15.87:37318) 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 -- 172.21.15.87:0/878600490 learned_addr learned my addr 172.21.15.87:0/878600490 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 -- 172.21.15.87:0/878600490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a992f0 msgr2=0x55da85a7e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a992f0 0x55da85a7e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 -- 172.21.15.87:0/878600490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 msgr2=0x55da85a7d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 0x55da85a7d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 -- 172.21.15.87:0/878600490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da859491c0 con 0x55da85a989d0 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54a3e00640 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85a7dd30 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f54940149d0 tx=0x7f54940046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc959ce592b62b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f548f400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:51.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f548f400640 1 -- 172.21.15.87:0/878600490 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494027030 con 0x55da85a989d0 2024-04-25T10:25:51.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f548f400640 1 -- 172.21.15.87:0/878600490 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f549400f040 con 0x55da85a989d0 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.011+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54a4018070 con 0x55da85a989d0 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f548f400640 1 -- 172.21.15.87:0/878600490 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54940136f0 con 0x55da85a989d0 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 10 client.11085 fetch_fsmap learned FSMap version 128 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x55da85945170 con 0x55da85a989d0 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f548f400640 1 -- 172.21.15.87:0/878600490 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4018070 con 0x55da85a989d0 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f548f400640 1 -- 172.21.15.87:0/878600490 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5494020050 con 0x55da85a989d0 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 10 client.11085 fetch_fsmap finished waiting for FSMap version 128 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 10 client.11085 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 20 client.11085 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aa8c80 0x55da85aab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 4 client.11085 mds_command: new command op to 10920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 --> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da85837c60 con 0x55da85aa8c80 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54a9e00640 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aa8c80 0x55da85aab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54a9e00640 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aa8c80 0x55da85aab070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933d13695098fb92 server_cookie=18be7fe12281b6b8 in_seq=0 out_seq=0 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f548f400640 10 client.11085 ms_handle_connect on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f548f400640 1 -- 172.21.15.87:0/878600490 <== mds.0 v2:172.21.15.87:6834/3053712751 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da85837c60 con 0x55da85aa8c80 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f548f400640 10 client.11085 handle_command_reply: tid=0 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 client.11085 shutdown 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 msgr2=0x55da85a7dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85a7dd30 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f54940149d0 tx=0x7f54940046d0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 shutdown_connections 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a992f0 0x55da85a7e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85a7dd30 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aa8c80 0x55da85aab070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 --2- 172.21.15.87:0/878600490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 0x55da85a7d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f548f400640 0 client.11085 ms_handle_reset on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 >> 172.21.15.87:0/878600490 conn(0x55da85a4b020 msgr2=0x55da85a4dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 shutdown_connections 2024-04-25T10:25:51.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 1 -- 172.21.15.87:0/878600490 wait complete. 2024-04-25T10:25:51.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.015+0000 7f54b459e000 20 client.11085 trim_cache size 0 max 16384 2024-04-25T10:25:51.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 Processor -- start 2024-04-25T10:25:51.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- start start 2024-04-25T10:25:51.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a4f0a0 0x55da85afdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85afe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85afe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac203570 con 0x55da85a989d0 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac202860 con 0x55da85a992f0 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac152b80 con 0x55da85a4f0a0 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85afe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85afe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37332/0 (socket says 172.21.15.87:37332) 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 -- 172.21.15.87:0/3803431593 learned_addr learned my addr 172.21.15.87:0/3803431593 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a9e00640 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a4f0a0 0x55da85afdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 -- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a4f0a0 msgr2=0x55da85afdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54aa800640 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85afe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a4f0a0 0x55da85afdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 -- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 msgr2=0x55da85afe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85afe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 -- 172.21.15.87:0/3803431593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da858e2de0 con 0x55da85a989d0 2024-04-25T10:25:51.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a9e00640 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a4f0a0 0x55da85afdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54aa800640 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85afe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54a3e00640 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85afe050 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f5494009160 tx=0x7f549401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d337e2f9a750b1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f549a000640 1 -- 172.21.15.87:0/3803431593 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494027040 con 0x55da85a989d0 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f549a000640 1 -- 172.21.15.87:0/3803431593 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549401b070 con 0x55da85a989d0 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f549a000640 1 -- 172.21.15.87:0/3803431593 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494013a90 con 0x55da85a989d0 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- 172.21.15.87:0/3803431593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 msgr2=0x55da85afe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85afe050 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f5494009160 tx=0x7f549401b000 comp rx=0 tx=0).stop 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- 172.21.15.87:0/3803431593 shutdown_connections 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a992f0 0x55da85afe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a989d0 0x55da85afe050 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 --2- 172.21.15.87:0/3803431593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a4f0a0 0x55da85afdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- 172.21.15.87:0/3803431593 >> 172.21.15.87:0/3803431593 conn(0x55da85a4b020 msgr2=0x55da85aa5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- 172.21.15.87:0/3803431593 shutdown_connections 2024-04-25T10:25:51.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.031+0000 7f54b459e000 1 -- 172.21.15.87:0/3803431593 wait complete. 2024-04-25T10:25:51.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 -- start start 2024-04-25T10:25:51.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 Processor -- start 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 0x55da85aab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85aab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a992f0 0x55da85aabe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8589b300 con 0x55da85a992f0 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da85836e00 con 0x55da85a4f0a0 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da858a0620 con 0x55da85a989d0 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 10 client.0 _ll_register_callbacks cb 0x55da85a1a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a992f0 0x55da85aabe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a992f0 0x55da85aabe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37340/0 (socket says 172.21.15.87:37340) 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 -- 172.21.15.87:0/447255600 learned_addr learned my addr 172.21.15.87:0/447255600 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54a3e00640 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85aab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54a9e00640 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 0x55da85aab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 -- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 msgr2=0x55da85aab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85aab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 -- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 msgr2=0x55da85aab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 0x55da85aab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 -- 172.21.15.87:0/447255600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da8583d8e0 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54aa800640 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a992f0 0x55da85aabe00 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f54a4015040 tx=0x7f54a4001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f1cf817f17f85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f548f400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f548f400640 1 -- 172.21.15.87:0/447255600 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4004d00 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f548f400640 1 -- 172.21.15.87:0/447255600 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54a4013b10 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f549401b070 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f548f400640 1 -- 172.21.15.87:0/447255600 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a402f050 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 10 client.11091 fetch_fsmap learned FSMap version 128 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x55da858f58f0 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f548f400640 1 -- 172.21.15.87:0/447255600 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549401b070 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f548f400640 1 -- 172.21.15.87:0/447255600 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54a4032020 con 0x55da85a992f0 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 10 client.11091 fetch_fsmap finished waiting for FSMap version 128 2024-04-25T10:25:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 10 client.11091 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 20 client.11091 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aac7c0 0x55da85b09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 4 client.11091 mds_command: new command op to 10920 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 --> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da858a9530 con 0x55da85aac7c0 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.035+0000 7f54a9e00640 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aac7c0 0x55da85b09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54a9e00640 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aac7c0 0x55da85b09770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0d3de878853f7a server_cookie=cd984f13de817b29 in_seq=0 out_seq=0 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f548f400640 10 client.11091 ms_handle_connect on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f548f400640 1 -- 172.21.15.87:0/447255600 <== mds.0 v2:172.21.15.87:6834/3053712751 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da858a9530 con 0x55da85aac7c0 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f548f400640 10 client.11091 handle_command_reply: tid=0 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 client.11091 shutdown 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a992f0 msgr2=0x55da85aabe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a992f0 0x55da85aabe00 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f54a4015040 tx=0x7f54a4001260 comp rx=0 tx=0).stop 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 shutdown_connections 2024-04-25T10:25:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da85a992f0 0x55da85aabe00 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55da85a989d0 0x55da85aab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55da85aac7c0 0x55da85b09770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 --2- 172.21.15.87:0/447255600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da85a4f0a0 0x55da85aab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f548f400640 0 client.11091 ms_handle_reset on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 >> 172.21.15.87:0/447255600 conn(0x55da85a4b020 msgr2=0x55da85a970d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 shutdown_connections 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 1 -- 172.21.15.87:0/447255600 wait complete. 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b459e000 20 client.11091 trim_cache size 0 max 16384 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:25:51.042 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:25:51.043 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:25:51.043 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:25:51.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5488066f60 msgr2=0x7f5488069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5488066f60 0x7f5488069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f54ac1fa570 tx=0x7f5494009190 comp rx=0 tx=0).stop 2024-04-25T10:25:51.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ac071bc0 msgr2=0x7f54ac0aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ac071bc0 0x7f54ac0aff20 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f549c0b0040 tx=0x7f549c01ada0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 shutdown_connections 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5488066f60 0x7f5488069420 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54ac0cfb50 0x7f54ac1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54ac0b3480 0x7f54ac150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 --2- 172.21.15.87:0/3041330490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ac071bc0 0x7f54ac0aff20 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 >> 172.21.15.87:0/3041330490 conn(0x7f54ac0669a0 msgr2=0x7f54ac168110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 shutdown_connections 2024-04-25T10:25:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.039+0000 7f54b1000640 1 -- 172.21.15.87:0/3041330490 wait complete. 2024-04-25T10:25:51.056 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:25:51.056 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:25:51.056 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 damage ls 2024-04-25T10:25:51.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f8c00640 1 Processor -- start 2024-04-25T10:25:51.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f8c00640 1 -- start start 2024-04-25T10:25:51.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 0x7f80f415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f8c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f415b080 con 0x7f80f40b31c0 2024-04-25T10:25:51.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f8c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f415adb0 con 0x7f80f40aef80 2024-04-25T10:25:51.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f415eb60 con 0x7f80f4070350 2024-04-25T10:25:51.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f2800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f80f40b31c0 0x7f80f40cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38362/0 (socket says 172.21.15.87:38362) 2024-04-25T10:25:51.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f2800640 1 -- 172.21.15.87:0/2728266629 learned_addr learned my addr 172.21.15.87:0/2728266629 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80e9400640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 0x7f80f415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511384803 0 0) 0x7f80f415b080 con 0x7f80f40b31c0 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.199+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80d8005740 con 0x7f80f40b31c0 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1752075030 0 0) 0x7f80d8005740 con 0x7f80f40b31c0 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 msgr2=0x7f80f415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 0x7f80f415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 >> v1:172.21.15.169:6789/0 conn(0x7f80f40aef80 legacy=0x7f80f415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f415ee40 con 0x7f80f40b31c0 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f80ec001a00 con 0x7f80f40b31c0 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 >> v1:172.21.15.87:6789/0 conn(0x7f80f40b31c0 legacy=0x7f80f40cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80d8005c00 0x7f80d8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d80065b0 0x7f80d8006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 0x7f80d80072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f415eb60 con 0x7f80d8005c00 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8003530 con 0x7f80d80065b0 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8003730 con 0x7f80f4070350 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80d8005c00 0x7f80d8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80e9400640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d80065b0 0x7f80d8006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f2800640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 0x7f80d80072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1e00640 1 -- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 msgr2=0x7f80d80072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80f4070350 0x7f80d80072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1e00640 1 -- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d80065b0 msgr2=0x7f80d8006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d80065b0 0x7f80d8006a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1e00640 1 -- 172.21.15.87:0/2728266629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8005740 con 0x7f80d8005c00 2024-04-25T10:25:51.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80d8005c00 0x7f80d8005fe0 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001860 tx=0x7f80e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c064f077c43dfee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.203+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40b0070 con 0x7f80d8005c00 2024-04-25T10:25:51.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e40ab600 con 0x7f80d8005c00 2024-04-25T10:25:51.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f1400640 1 -- 172.21.15.87:0/2728266629 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4001460 con 0x7f80d8005c00 2024-04-25T10:25:51.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- 172.21.15.87:0/2728266629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80d8005c00 msgr2=0x7f80d8005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 --2- 172.21.15.87:0/2728266629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80d8005c00 0x7f80d8005fe0 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001860 tx=0x7f80e401c520 comp rx=0 tx=0).stop 2024-04-25T10:25:51.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f2800640 1 -- 172.21.15.87:0/2728266629 reap_dead start 2024-04-25T10:25:51.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- 172.21.15.87:0/2728266629 shutdown_connections 2024-04-25T10:25:51.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- 172.21.15.87:0/2728266629 >> 172.21.15.87:0/2728266629 conn(0x7f80f40669a0 msgr2=0x7f80f4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- 172.21.15.87:0/2728266629 shutdown_connections 2024-04-25T10:25:51.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- 172.21.15.87:0/2728266629 wait complete. 2024-04-25T10:25:51.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 Processor -- start 2024-04-25T10:25:51.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- start start 2024-04-25T10:25:51.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80d8005c00 0x7f80f40ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80f41f6250 0x7f80f41581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f4158730 0x7f80f40a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40cfef0 con 0x7f80f4158730 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40cfd70 con 0x7f80f41f6250 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f415e480 con 0x7f80d8005c00 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80d8005c00 0x7f80f40ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f4158730 0x7f80f40a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80d8005c00 0x7f80f40ac060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:47420/0 (socket says 172.21.15.87:47420) 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80f1e00640 1 -- 172.21.15.87:0/4091863346 learned_addr learned my addr 172.21.15.87:0/4091863346 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.207+0000 7f80e9400640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80f41f6250 0x7f80f41581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f1e00640 1 -- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80f41f6250 msgr2=0x7f80f41581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f1e00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80f41f6250 0x7f80f41581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f1e00640 1 -- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f4158730 msgr2=0x7f80f40a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f1e00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f4158730 0x7f80f40a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f1e00640 1 -- 172.21.15.87:0/4091863346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40b07a0 con 0x7f80d8005c00 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f2800640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f4158730 0x7f80f40a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f1e00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80d8005c00 0x7f80f40ac060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80e401ca00 tx=0x7f80d8007a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80eaa00640 1 -- 172.21.15.87:0/4091863346 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40b9020 con 0x7f80d8005c00 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80eaa00640 1 -- 172.21.15.87:0/4091863346 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e4001440 con 0x7f80d8005c00 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80eaa00640 1 -- 172.21.15.87:0/4091863346 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40b0040 con 0x7f80d8005c00 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f40b0a80 con 0x7f80d8005c00 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f40b1040 con 0x7f80d8005c00 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80eaa00640 1 -- 172.21.15.87:0/4091863346 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80e40bc020 con 0x7f80d8005c00 2024-04-25T10:25:51.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80eaa00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80d0066f90 0x7f80d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80eaa00640 1 -- 172.21.15.87:0/4091863346 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80e411eac0 con 0x7f80d8005c00 2024-04-25T10:25:51.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.211+0000 7f80e9400640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80d0066f90 0x7f80d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 Processor -- start 2024-04-25T10:25:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- start start 2024-04-25T10:25:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80d0066f90 0x7f80d0069450 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f80f415ea40 tx=0x7f80dc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581b180 0x55ffe581b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581baa0 0x55ffe57d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe561ef20 con 0x55ffe581baa0 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe55a7000 con 0x55ffe581b180 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffe561afc0 con 0x55ffe57d1600 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581baa0 0x55ffe57d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581baa0 0x55ffe57d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37362/0 (socket says 172.21.15.87:37362) 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 -- 172.21.15.87:0/3348120831 learned_addr learned my addr 172.21.15.87:0/3348120831 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80f1e00640 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581b180 0x55ffe581b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80f2800640 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 -- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 msgr2=0x55ffe57d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 -- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581b180 msgr2=0x55ffe581b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581b180 0x55ffe581b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 -- 172.21.15.87:0/3348120831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffe55faba0 con 0x55ffe581baa0 2024-04-25T10:25:51.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e9400640 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581baa0 0x55ffe57d10c0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009fd0 tx=0x7f80dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39a7035d21284f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e2a00640 1 -- 172.21.15.87:0/3348120831 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc01b070 con 0x55ffe581baa0 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e2a00640 1 -- 172.21.15.87:0/3348120831 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc015e30 con 0x55ffe581baa0 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80e2a00640 1 -- 172.21.15.87:0/3348120831 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc02a630 con 0x55ffe581baa0 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- 172.21.15.87:0/3348120831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581baa0 msgr2=0x55ffe57d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581baa0 0x55ffe57d10c0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009fd0 tx=0x7f80dc017000 comp rx=0 tx=0).stop 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- 172.21.15.87:0/3348120831 shutdown_connections 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581baa0 0x55ffe57d10c0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3348120831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581b180 0x55ffe581b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- 172.21.15.87:0/3348120831 >> 172.21.15.87:0/3348120831 conn(0x55ffe57d7760 msgr2=0x55ffe57cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- 172.21.15.87:0/3348120831 shutdown_connections 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- 172.21.15.87:0/3348120831 wait complete. 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.215+0000 7f80fc06d000 1 -- start start 2024-04-25T10:25:51.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 Processor -- start 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581b180 0x55ffe57d6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581baa0 0x55ffe5809450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe56dc9c0 con 0x55ffe581b180 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe55d95d0 con 0x55ffe581baa0 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffe55e94b0 con 0x55ffe57d1600 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 10 client.0 _ll_register_callbacks cb 0x55ffe57bca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581b180 0x55ffe57d6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80f1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581b180 0x55ffe57d6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37372/0 (socket says 172.21.15.87:37372) 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 -- 172.21.15.87:0/3601722827 learned_addr learned my addr 172.21.15.87:0/3601722827 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80f2800640 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581baa0 0x55ffe5809450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 -- 172.21.15.87:0/3601722827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 msgr2=0x55ffe57d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 -- 172.21.15.87:0/3601722827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581baa0 msgr2=0x55ffe5809450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581baa0 0x55ffe5809450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 -- 172.21.15.87:0/3601722827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffe56655a0 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80e9400640 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581b180 0x55ffe57d6be0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x55ffe5624f00 tx=0x7f80dc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d775defa99301168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80d7400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80d7400640 1 -- 172.21.15.87:0/3601722827 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc037060 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80d7400640 1 -- 172.21.15.87:0/3601722827 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80dc02ac50 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80dc015e30 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80d7400640 1 -- 172.21.15.87:0/3601722827 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80dc02ae50 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 10 client.11103 fetch_fsmap learned FSMap version 128 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x55ffe55f62b0 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80d7400640 1 -- 172.21.15.87:0/3601722827 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc015e30 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80d7400640 1 -- 172.21.15.87:0/3601722827 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f80dc032440 con 0x55ffe581b180 2024-04-25T10:25:51.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 10 client.11103 fetch_fsmap finished waiting for FSMap version 128 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 10 client.11103 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 20 client.11103 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe5808e60 0x55ffe5809e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 4 client.11103 mds_command: new command op to 10920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 --> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffe55bac70 con 0x55ffe5808e60 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80f1e00640 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe5808e60 0x55ffe5809e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80f1e00640 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe5808e60 0x55ffe5809e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0039693171970d1 server_cookie=617818e421758a21 in_seq=0 out_seq=0 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.219+0000 7f80d7400640 10 client.11103 ms_handle_connect on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80d7400640 1 -- 172.21.15.87:0/3601722827 <== mds.0 v2:172.21.15.87:6834/3053712751 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffe55bac70 con 0x55ffe5808e60 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80d7400640 10 client.11103 handle_command_reply: tid=0 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 client.11103 shutdown 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581b180 msgr2=0x55ffe57d6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581b180 0x55ffe57d6be0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x55ffe5624f00 tx=0x7f80dc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 shutdown_connections 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe5808e60 0x55ffe5809e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe581baa0 0x55ffe5809450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe581b180 0x55ffe57d6be0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3601722827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe57d1600 0x55ffe57d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80d7400640 0 client.11103 ms_handle_reset on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 >> 172.21.15.87:0/3601722827 conn(0x55ffe57d7760 msgr2=0x55ffe57d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 shutdown_connections 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 1 -- 172.21.15.87:0/3601722827 wait complete. 2024-04-25T10:25:51.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.223+0000 7f80fc06d000 20 client.11103 trim_cache size 0 max 16384 2024-04-25T10:25:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 Processor -- start 2024-04-25T10:25:51.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- start start 2024-04-25T10:25:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d7760 0x55ffe5831690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d1600 0x55ffe57d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 0x55ffe57d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41fef90 con 0x55ffe57d1600 2024-04-25T10:25:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41fe190 con 0x55ffe57d7760 2024-04-25T10:25:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40afd70 con 0x55ffe581b180 2024-04-25T10:25:51.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d1600 0x55ffe57d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d1600 0x55ffe57d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37380/0 (socket says 172.21.15.87:37380) 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 -- 172.21.15.87:0/3905115463 learned_addr learned my addr 172.21.15.87:0/3905115463 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 -- 172.21.15.87:0/3905115463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 msgr2=0x55ffe57d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 --2- 172.21.15.87:0/3905115463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 0x55ffe57d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 -- 172.21.15.87:0/3905115463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d7760 msgr2=0x55ffe5831690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 --2- 172.21.15.87:0/3905115463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d7760 0x55ffe5831690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 -- 172.21.15.87:0/3905115463 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffe580a1d0 con 0x55ffe57d1600 2024-04-25T10:25:51.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e9400640 1 --2- 172.21.15.87:0/3905115463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d1600 0x55ffe57d0120 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f80dc004710 tx=0x7f80dc02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf261d2961f40f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e2000640 1 -- 172.21.15.87:0/3905115463 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc041080 con 0x55ffe57d1600 2024-04-25T10:25:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e2000640 1 -- 172.21.15.87:0/3905115463 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc015c80 con 0x55ffe57d1600 2024-04-25T10:25:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80e2000640 1 -- 172.21.15.87:0/3905115463 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc0096f0 con 0x55ffe57d1600 2024-04-25T10:25:51.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- 172.21.15.87:0/3905115463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d1600 msgr2=0x55ffe57d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3905115463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d1600 0x55ffe57d0120 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f80dc004710 tx=0x7f80dc02c040 comp rx=0 tx=0).stop 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- 172.21.15.87:0/3905115463 shutdown_connections 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3905115463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 0x55ffe57d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3905115463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d1600 0x55ffe57d0120 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 --2- 172.21.15.87:0/3905115463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d7760 0x55ffe5831690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- 172.21.15.87:0/3905115463 >> 172.21.15.87:0/3905115463 conn(0x55ffe58095b0 msgr2=0x55ffe5828770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- 172.21.15.87:0/3905115463 shutdown_connections 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.235+0000 7f80fc06d000 1 -- 172.21.15.87:0/3905115463 wait complete. 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 -- start start 2024-04-25T10:25:51.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 Processor -- start 2024-04-25T10:25:51.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d1600 0x55ffe587b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d7760 0x55ffe587b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 0x55ffe582fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe55d5dc0 con 0x55ffe57d7760 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41fdbe0 con 0x55ffe57d1600 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffe5623550 con 0x55ffe581b180 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 10 client.0 _ll_register_callbacks cb 0x55ffe579d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d7760 0x55ffe587b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d7760 0x55ffe587b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37382/0 (socket says 172.21.15.87:37382) 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 -- 172.21.15.87:0/2780651014 learned_addr learned my addr 172.21.15.87:0/2780651014 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d1600 0x55ffe587b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80f2800640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 0x55ffe582fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 -- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 msgr2=0x55ffe582fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 0x55ffe582fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 -- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d1600 msgr2=0x55ffe587b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d1600 0x55ffe587b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 -- 172.21.15.87:0/2780651014 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffe55a7250 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d1600 0x55ffe587b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80e9400640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d7760 0x55ffe587b960 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00b010 tx=0x7f80dc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d12d38cc9b4164c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80d7400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80d7400640 1 -- 172.21.15.87:0/2780651014 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc041040 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80d7400640 1 -- 172.21.15.87:0/2780651014 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80dc04eaa0 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80dc015c80 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80d7400640 1 -- 172.21.15.87:0/2780651014 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80dc02ae60 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 10 client.11109 fetch_fsmap learned FSMap version 128 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x55ffe55b3b90 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80d7400640 1 -- 172.21.15.87:0/2780651014 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc015c80 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80d7400640 1 -- 172.21.15.87:0/2780651014 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f80dc04eca0 con 0x55ffe57d7760 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 10 client.11109 fetch_fsmap finished waiting for FSMap version 128 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 10 client.11109 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:25:51.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 20 client.11109 populate_metadata read hostname 'smithi087' 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe581baa0 0x55ffe5830690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 4 client.11109 mds_command: new command op to 10920 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 --> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffe562c460 con 0x55ffe581baa0 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe581baa0 0x55ffe5830690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80f1e00640 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe581baa0 0x55ffe5830690 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1168221b53da05 server_cookie=8ad593796c362793 in_seq=0 out_seq=0 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.239+0000 7f80d7400640 10 client.11109 ms_handle_connect on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80d7400640 1 -- 172.21.15.87:0/2780651014 <== mds.0 v2:172.21.15.87:6834/3053712751 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ffe562c460 con 0x55ffe581baa0 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80d7400640 10 client.11109 handle_command_reply: tid=0 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 client.11109 shutdown 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d7760 msgr2=0x55ffe587b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d7760 0x55ffe587b960 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00b010 tx=0x7f80dc01b000 comp rx=0 tx=0).stop 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 shutdown_connections 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffe581b180 0x55ffe582fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffe57d7760 0x55ffe587b960 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.87:6834/3053712751,v1:172.21.15.87:6836/3053712751] conn(0x55ffe581baa0 0x55ffe5830690 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 --2- 172.21.15.87:0/2780651014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffe57d1600 0x55ffe587b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80d7400640 0 client.11109 ms_handle_reset on v2:172.21.15.87:6834/3053712751 2024-04-25T10:25:51.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 >> 172.21.15.87:0/2780651014 conn(0x55ffe58095b0 msgr2=0x55ffe57cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 shutdown_connections 2024-04-25T10:25:51.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 1 -- 172.21.15.87:0/2780651014 wait complete. 2024-04-25T10:25:51.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80fc06d000 20 client.11109 trim_cache size 0 max 16384 2024-04-25T10:25:51.245 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-04-25T10:25:51.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80d0066f90 msgr2=0x7f80d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80d0066f90 0x7f80d0069450 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f80f415ea40 tx=0x7f80dc009190 comp rx=0 tx=0).stop 2024-04-25T10:25:51.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80d8005c00 msgr2=0x7f80f40ac060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80d8005c00 0x7f80f40ac060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80e401ca00 tx=0x7f80d8007a50 comp rx=0 tx=0).stop 2024-04-25T10:25:51.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 shutdown_connections 2024-04-25T10:25:51.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80d0066f90 0x7f80d0069450 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f4158730 0x7f80f40a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80f41f6250 0x7f80f41581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 --2- 172.21.15.87:0/4091863346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80d8005c00 0x7f80f40ac060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 >> 172.21.15.87:0/4091863346 conn(0x7f80f40669a0 msgr2=0x7f80f415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 shutdown_connections 2024-04-25T10:25:51.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.243+0000 7f80f8c00640 1 -- 172.21.15.87:0/4091863346 wait complete. 2024-04-25T10:25:51.261 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:25:51.261 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:51.359 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:51.456 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:25:51.457 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:51.457 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:51.465 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:51.466 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:51.517 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:51.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.695+0000 7f2602600640 1 Processor -- start 2024-04-25T10:25:51.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.695+0000 7f2602600640 1 -- start start 2024-04-25T10:25:51.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f2602600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 0x7f25fc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f2602600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25fc0b2770 con 0x7f25fc0cda10 2024-04-25T10:25:51.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f2602600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25fc06c670 con 0x7f25fc15bc40 2024-04-25T10:25:51.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f2602600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc06c950 con 0x7f25fc0d0040 2024-04-25T10:25:51.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25faa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f25fc0cda10 0x7f25fc0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38378/0 (socket says 172.21.15.87:38378) 2024-04-25T10:25:51.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25faa00640 1 -- 172.21.15.87:0/3526232492 learned_addr learned my addr 172.21.15.87:0/3526232492 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:51.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fbe00640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 0x7f25fc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739223594 0 0) 0x7f25fc0b2770 con 0x7f25fc0cda10 2024-04-25T10:25:51.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25e0003530 con 0x7f25fc0cda10 2024-04-25T10:25:51.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1460802251 0 0) 0x7f25e0003530 con 0x7f25fc0cda10 2024-04-25T10:25:51.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 msgr2=0x7f25fc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 0x7f25fc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 >> v1:172.21.15.169:6789/0 conn(0x7f25fc15bc40 legacy=0x7f25fc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc06cc30 con 0x7f25fc0cda10 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f25e8001680 con 0x7f25fc0cda10 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 >> v1:172.21.15.87:6789/0 conn(0x7f25fc0cda10 legacy=0x7f25fc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e00039f0 0x7f25e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 0x7f25e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e0004d50 0x7f25e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc06c950 con 0x7f25e0004d50 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e000bab0 con 0x7f25e00039f0 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e000bcb0 con 0x7f25fc0d0040 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fbe00640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e0004d50 0x7f25e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fb400640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e00039f0 0x7f25e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25faa00640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 0x7f25e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fbe00640 1 -- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 msgr2=0x7f25e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fbe00640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc0d0040 0x7f25e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fbe00640 1 -- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e00039f0 msgr2=0x7f25e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:51.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fbe00640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e00039f0 0x7f25e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.699+0000 7f25fbe00640 1 -- 172.21.15.87:0/3526232492 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0003530 con 0x7f25e0004d50 2024-04-25T10:25:51.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f25fbe00640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e0004d50 0x7f25e00051d0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0b4510 tx=0x7f25f401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37097a16c237cb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40b2070 con 0x7f25e0004d50 2024-04-25T10:25:51.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f40a5c30 con 0x7f25e0004d50 2024-04-25T10:25:51.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f25fa000640 1 -- 172.21.15.87:0/3526232492 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4009c10 con 0x7f25e0004d50 2024-04-25T10:25:51.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 -- 172.21.15.87:0/3526232492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e0004d50 msgr2=0x7f25e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 --2- 172.21.15.87:0/3526232492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e0004d50 0x7f25e00051d0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0b4510 tx=0x7f25f401cc20 comp rx=0 tx=0).stop 2024-04-25T10:25:51.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f25fbe00640 1 -- 172.21.15.87:0/3526232492 reap_dead start 2024-04-25T10:25:51.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 -- 172.21.15.87:0/3526232492 shutdown_connections 2024-04-25T10:25:51.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 -- 172.21.15.87:0/3526232492 >> 172.21.15.87:0/3526232492 conn(0x7f25fc066b70 msgr2=0x7f25fc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 -- 172.21.15.87:0/3526232492 shutdown_connections 2024-04-25T10:25:51.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 -- 172.21.15.87:0/3526232492 wait complete. 2024-04-25T10:25:51.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 Processor -- start 2024-04-25T10:25:51.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.703+0000 7f2602600640 1 -- start start 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc1411a0 0x7f25fc1e4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25fc1e6140 0x7f25fc15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc141580 0x7f25fc15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc06c210 con 0x7f25fc1e6140 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc06c090 con 0x7f25fc141580 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc06c390 con 0x7f25fc1411a0 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25fc1e6140 0x7f25fc15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25fc1e6140 0x7f25fc15cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37402/0 (socket says 172.21.15.87:37402) 2024-04-25T10:25:51.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 -- 172.21.15.87:0/3761747627 learned_addr learned my addr 172.21.15.87:0/3761747627 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25fbe00640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc141580 0x7f25fc15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25fb400640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc1411a0 0x7f25fc1e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 -- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc1411a0 msgr2=0x7f25fc1e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc1411a0 0x7f25fc1e4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 -- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc141580 msgr2=0x7f25fc15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc141580 0x7f25fc15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 -- 172.21.15.87:0/3761747627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc1e50a0 con 0x7f25fc1e6140 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25faa00640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25fc1e6140 0x7f25fc15cbc0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f25e80010f0 tx=0x7f25e800ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25f3400640 1 -- 172.21.15.87:0/3761747627 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e800c8d0 con 0x7f25fc1e6140 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25f3400640 1 -- 172.21.15.87:0/3761747627 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e8018070 con 0x7f25fc1e6140 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25f3400640 1 -- 172.21.15.87:0/3761747627 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e800ccb0 con 0x7f25fc1e6140 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc1e5380 con 0x7f25fc1e6140 2024-04-25T10:25:51.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc1e5940 con 0x7f25fc1e6140 2024-04-25T10:25:51.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc04ef30 con 0x7f25fc1e6140 2024-04-25T10:25:51.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.707+0000 7f25f3400640 1 -- 172.21.15.87:0/3761747627 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25e8013420 con 0x7f25fc1e6140 2024-04-25T10:25:51.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.711+0000 7f25f3400640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f25dc066f90 0x7f25dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.711+0000 7f25f3400640 1 -- 172.21.15.87:0/3761747627 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25e8084c00 con 0x7f25fc1e6140 2024-04-25T10:25:51.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.711+0000 7f25fb400640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f25dc066f90 0x7f25dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.711+0000 7f25fb400640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f25dc066f90 0x7f25dc069450 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f25ec007990 tx=0x7f25ec007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.715+0000 7f25f3400640 1 -- 172.21.15.87:0/3761747627 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25e8055b70 con 0x7f25fc1e6140 2024-04-25T10:25:51.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.867+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25fc0b2770 con 0x7f25fc1e6140 2024-04-25T10:25:51.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.867+0000 7f25f3400640 1 -- 172.21.15.87:0/3761747627 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f25e8055470 con 0x7f25fc1e6140 2024-04-25T10:25:51.869 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:51.869 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":176,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:44.125291+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:40.866786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:41.487045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f25dc066f90 msgr2=0x7f25dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f25dc066f90 0x7f25dc069450 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f25ec007990 tx=0x7f25ec007900 comp rx=0 tx=0).stop 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25fc1e6140 msgr2=0x7f25fc15cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25fc1e6140 0x7f25fc15cbc0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f25e80010f0 tx=0x7f25e800ef80 comp rx=0 tx=0).stop 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 shutdown_connections 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f25dc066f90 0x7f25dc069450 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc141580 0x7f25fc15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25fc1e6140 0x7f25fc15cbc0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 --2- 172.21.15.87:0/3761747627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc1411a0 0x7f25fc1e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 >> 172.21.15.87:0/3761747627 conn(0x7f25fc066b70 msgr2=0x7f25fc0af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 shutdown_connections 2024-04-25T10:25:51.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:51.871+0000 7f2602600640 1 -- 172.21.15.87:0/3761747627 wait complete. 2024-04-25T10:25:51.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:25:40.866786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:25:41.487045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:51.890 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:52.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.027+0000 7efd7d200640 1 Processor -- start 2024-04-25T10:25:52.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.027+0000 7efd7d200640 1 -- start start 2024-04-25T10:25:52.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd7d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd7815f760 0x7efd780d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd7d200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7815fc40 con 0x7efd7815bca0 2024-04-25T10:25:52.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd7d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7815ff20 con 0x7efd780d05e0 2024-04-25T10:25:52.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd7d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7815c080 con 0x7efd7815f760 2024-04-25T10:25:52.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd75e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efd7815bca0 0x7efd780cc490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38394/0 (socket says 172.21.15.87:38394) 2024-04-25T10:25:52.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd75e00640 1 -- 172.21.15.87:0/2046615313 learned_addr learned my addr 172.21.15.87:0/2046615313 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:52.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd6d400640 1 --2- 172.21.15.87:0/2046615313 >> v2:172.21.15.169:3300/0 conn(0x7efd7815f760 0x7efd780d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd75400640 1 -- 172.21.15.87:0/2046615313 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722216807 0 0) 0x7efd7815fc40 con 0x7efd7815bca0 2024-04-25T10:25:52.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd75400640 1 -- 172.21.15.87:0/2046615313 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd5c003530 con 0x7efd7815bca0 2024-04-25T10:25:52.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd75400640 1 -- 172.21.15.87:0/2046615313 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077384807 0 0) 0x7efd7815ff20 con 0x7efd780d05e0 2024-04-25T10:25:52.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd75400640 1 -- 172.21.15.87:0/2046615313 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd7815fc40 con 0x7efd780d05e0 2024-04-25T10:25:52.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd6d400640 1 -- 172.21.15.87:0/2046615313 >> v1:172.21.15.169:6789/0 conn(0x7efd780d05e0 legacy=0x7efd780ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd6d400640 1 -- 172.21.15.87:0/2046615313 >> v1:172.21.15.87:6789/0 conn(0x7efd7815bca0 legacy=0x7efd780cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.031+0000 7efd6d400640 1 -- 172.21.15.87:0/2046615313 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78160200 con 0x7efd7815f760 2024-04-25T10:25:52.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd6d400640 1 --2- 172.21.15.87:0/2046615313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815f760 0x7efd780d00a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7efd680097c0 tx=0x7efd6801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457b659abaaabc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd75400640 1 -- 172.21.15.87:0/2046615313 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680b0070 con 0x7efd7815f760 2024-04-25T10:25:52.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd75400640 1 -- 172.21.15.87:0/2046615313 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd680ab600 con 0x7efd7815f760 2024-04-25T10:25:52.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd75400640 1 -- 172.21.15.87:0/2046615313 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680ab9d0 con 0x7efd7815f760 2024-04-25T10:25:52.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 -- 172.21.15.87:0/2046615313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815f760 msgr2=0x7efd780d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 --2- 172.21.15.87:0/2046615313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815f760 0x7efd780d00a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7efd680097c0 tx=0x7efd6801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 -- 172.21.15.87:0/2046615313 shutdown_connections 2024-04-25T10:25:52.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 --2- 172.21.15.87:0/2046615313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815f760 0x7efd780d00a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 -- 172.21.15.87:0/2046615313 >> 172.21.15.87:0/2046615313 conn(0x7efd78072290 msgr2=0x7efd780726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 -- 172.21.15.87:0/2046615313 shutdown_connections 2024-04-25T10:25:52.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 -- 172.21.15.87:0/2046615313 wait complete. 2024-04-25T10:25:52.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 Processor -- start 2024-04-25T10:25:52.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.035+0000 7efd7d200640 1 -- start start 2024-04-25T10:25:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd780d05e0 0x7efd78156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815bca0 0x7efd780b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815f760 0x7efd7806df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd780cf320 con 0x7efd780d05e0 2024-04-25T10:25:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd780cf1a0 con 0x7efd7815bca0 2024-04-25T10:25:52.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd780cf4a0 con 0x7efd7815f760 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd780d05e0 0x7efd78156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd780d05e0 0x7efd78156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37418/0 (socket says 172.21.15.87:37418) 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 -- 172.21.15.87:0/3052185494 learned_addr learned my addr 172.21.15.87:0/3052185494 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd6d400640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815bca0 0x7efd780b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd76800640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815f760 0x7efd7806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 -- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815f760 msgr2=0x7efd7806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815f760 0x7efd7806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 -- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815bca0 msgr2=0x7efd780b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815bca0 0x7efd780b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 -- 172.21.15.87:0/3052185494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780c9d00 con 0x7efd780d05e0 2024-04-25T10:25:52.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd6d400640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815bca0 0x7efd780b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:52.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd75e00640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd780d05e0 0x7efd78156090 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7efd6000c450 tx=0x7efd6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd6ea00640 1 -- 172.21.15.87:0/3052185494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60017070 con 0x7efd780d05e0 2024-04-25T10:25:52.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd6ea00640 1 -- 172.21.15.87:0/3052185494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd60005cf0 con 0x7efd780d05e0 2024-04-25T10:25:52.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd6ea00640 1 -- 172.21.15.87:0/3052185494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60007310 con 0x7efd780d05e0 2024-04-25T10:25:52.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd781fa4e0 con 0x7efd780d05e0 2024-04-25T10:25:52.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd780c9fa0 con 0x7efd780d05e0 2024-04-25T10:25:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7804ef30 con 0x7efd780d05e0 2024-04-25T10:25:52.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.039+0000 7efd6ea00640 1 -- 172.21.15.87:0/3052185494 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd6001a070 con 0x7efd780d05e0 2024-04-25T10:25:52.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.043+0000 7efd6ea00640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd54066f60 0x7efd54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.043+0000 7efd6ea00640 1 -- 172.21.15.87:0/3052185494 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd600844a0 con 0x7efd780d05e0 2024-04-25T10:25:52.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.043+0000 7efd6d400640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd54066f60 0x7efd54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.043+0000 7efd6d400640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd54066f60 0x7efd54069420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efd680ad000 tx=0x7efd68008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.047+0000 7efd6ea00640 1 -- 172.21.15.87:0/3052185494 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd60055410 con 0x7efd780d05e0 2024-04-25T10:25:52.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.199+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd78072e80 con 0x7efd780d05e0 2024-04-25T10:25:52.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.199+0000 7efd6ea00640 1 -- 172.21.15.87:0/3052185494 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd60054d10 con 0x7efd780d05e0 2024-04-25T10:25:52.201 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:25:52.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd54066f60 msgr2=0x7efd54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd54066f60 0x7efd54069420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efd680ad000 tx=0x7efd68008040 comp rx=0 tx=0).stop 2024-04-25T10:25:52.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd780d05e0 msgr2=0x7efd78156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd780d05e0 0x7efd78156090 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7efd6000c450 tx=0x7efd6000c910 comp rx=0 tx=0).stop 2024-04-25T10:25:52.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 shutdown_connections 2024-04-25T10:25:52.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7efd54066f60 0x7efd54069420 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815f760 0x7efd7806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd7815bca0 0x7efd780b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 --2- 172.21.15.87:0/3052185494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd780d05e0 0x7efd78156090 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 >> 172.21.15.87:0/3052185494 conn(0x7efd78072290 msgr2=0x7efd7815a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 shutdown_connections 2024-04-25T10:25:52.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.203+0000 7efd7d200640 1 -- 172.21.15.87:0/3052185494 wait complete. 2024-04-25T10:25:52.222 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:25:52.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e60c00640 1 Processor -- start 2024-04-25T10:25:52.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e60c00640 1 -- start start 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e60c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5e5c159540 0x7f5e5c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e60c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5e5c0aebd0 0x7f5e5c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e60c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c071360 con 0x7f5e5c072020 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e60c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c159920 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e60c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c15c2f0 con 0x7f5e5c159540 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5e5c0aebd0 0x7f5e5c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5e5c0aebd0 0x7f5e5c071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37424/0 (socket says 172.21.15.87:37424) 2024-04-25T10:25:52.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59400640 1 -- 172.21.15.87:0/2207197229 learned_addr learned my addr 172.21.15.87:0/2207197229 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:52.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59e00640 1 --2- 172.21.15.87:0/2207197229 >> v2:172.21.15.169:3300/0 conn(0x7f5e5c159540 0x7f5e5c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59400640 1 -- 172.21.15.87:0/2207197229 >> v2:172.21.15.169:3300/0 conn(0x7f5e5c159540 msgr2=0x7f5e5c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:52.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59400640 1 --2- 172.21.15.87:0/2207197229 >> v2:172.21.15.169:3300/0 conn(0x7f5e5c159540 0x7f5e5c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59400640 1 -- 172.21.15.87:0/2207197229 >> v1:172.21.15.169:6789/0 conn(0x7f5e5c072020 legacy=0x7f5e5c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:52.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.363+0000 7f5e59400640 1 -- 172.21.15.87:0/2207197229 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c15c5d0 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e59400640 1 --2- 172.21.15.87:0/2207197229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c071ae0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0b4c20 tx=0x7f5e4801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf60274025ed35c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e58a00640 1 -- 172.21.15.87:0/2207197229 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e480b0070 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e58a00640 1 -- 172.21.15.87:0/2207197229 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e480ab600 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e58a00640 1 -- 172.21.15.87:0/2207197229 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e480ab9d0 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 -- 172.21.15.87:0/2207197229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 msgr2=0x7f5e5c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 --2- 172.21.15.87:0/2207197229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c071ae0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0b4c20 tx=0x7f5e4801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 -- 172.21.15.87:0/2207197229 shutdown_connections 2024-04-25T10:25:52.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 --2- 172.21.15.87:0/2207197229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c071ae0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 --2- 172.21.15.87:0/2207197229 >> v2:172.21.15.169:3300/0 conn(0x7f5e5c159540 0x7f5e5c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 -- 172.21.15.87:0/2207197229 >> 172.21.15.87:0/2207197229 conn(0x7f5e5c0669a0 msgr2=0x7f5e5c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 -- 172.21.15.87:0/2207197229 shutdown_connections 2024-04-25T10:25:52.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 -- 172.21.15.87:0/2207197229 wait complete. 2024-04-25T10:25:52.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 Processor -- start 2024-04-25T10:25:52.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.367+0000 7f5e60c00640 1 -- start start 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e5c072020 0x7f5e5c14b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e5c159540 0x7f5e5c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c06c0d0 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c072450 con 0x7f5e5c159540 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c168930 con 0x7f5e5c072020 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c149800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37438/0 (socket says 172.21.15.87:37438) 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 -- 172.21.15.87:0/117834572 learned_addr learned my addr 172.21.15.87:0/117834572 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59e00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e5c072020 0x7f5e5c14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e5a800640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e5c159540 0x7f5e5c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 -- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e5c072020 msgr2=0x7f5e5c14b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e5c072020 0x7f5e5c14b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 -- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e5c159540 msgr2=0x7f5e5c0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e5c159540 0x7f5e5c0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 -- 172.21.15.87:0/117834572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0680c0 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e59400640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c149800 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f5e4801a110 tx=0x7f5e48019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e52000640 1 -- 172.21.15.87:0/117834572 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e480b9070 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e52000640 1 -- 172.21.15.87:0/117834572 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e480b4400 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c0b1030 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e52000640 1 -- 172.21.15.87:0/117834572 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e480b0040 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c156b40 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c04ef30 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.371+0000 7f5e52000640 1 -- 172.21.15.87:0/117834572 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e480bc070 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.375+0000 7f5e52000640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5e3c066f90 0x7f5e3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.375+0000 7f5e52000640 1 -- 172.21.15.87:0/117834572 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e4811e0e0 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.375+0000 7f5e59e00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5e3c066f90 0x7f5e3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.375+0000 7f5e59e00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5e3c066f90 0x7f5e3c069450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0079c0 tx=0x7f5e4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.379+0000 7f5e52000640 1 -- 172.21.15.87:0/117834572 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e5c0b1030 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e5c0b12b0 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e52000640 1 -- 172.21.15.87:0/117834572 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v176) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e480160d0 con 0x7f5e5c0aebd0 2024-04-25T10:25:52.530 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:25:52.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5e3c066f90 msgr2=0x7f5e3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5e3c066f90 0x7f5e3c069450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0079c0 tx=0x7f5e4c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:52.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 msgr2=0x7f5e5c149800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c149800 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f5e4801a110 tx=0x7f5e48019280 comp rx=0 tx=0).stop 2024-04-25T10:25:52.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 shutdown_connections 2024-04-25T10:25:52.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5e3c066f90 0x7f5e3c069450 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e5c159540 0x7f5e5c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c0aebd0 0x7f5e5c149800 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 --2- 172.21.15.87:0/117834572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e5c072020 0x7f5e5c14b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 >> 172.21.15.87:0/117834572 conn(0x7f5e5c0669a0 msgr2=0x7f5e5c15e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 shutdown_connections 2024-04-25T10:25:52.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.531+0000 7f5e60c00640 1 -- 172.21.15.87:0/117834572 wait complete. 2024-04-25T10:25:52.551 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:25:52.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3efbc00640 1 Processor -- start 2024-04-25T10:25:52.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3efbc00640 1 -- start start 2024-04-25T10:25:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3efbc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ef4159540 0x7f3ef40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3efbc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef4159920 con 0x7f3ef40721f0 2024-04-25T10:25:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3efbc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef415c2f0 con 0x7f3ef40aebd0 2024-04-25T10:25:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3efbc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef415c5d0 con 0x7f3ef4159540 2024-04-25T10:25:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3ef9c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3ef40721f0 0x7f3ef4070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38406/0 (socket says 172.21.15.87:38406) 2024-04-25T10:25:52.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3ef9c00640 1 -- 172.21.15.87:0/1305259431 learned_addr learned my addr 172.21.15.87:0/1305259431 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:52.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3ef9200640 1 --2- 172.21.15.87:0/1305259431 >> v2:172.21.15.169:3300/0 conn(0x7f3ef4159540 0x7f3ef40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998519459 0 0) 0x7f3ef4159920 con 0x7f3ef40721f0 2024-04-25T10:25:52.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.691+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed8003530 con 0x7f3ef40721f0 2024-04-25T10:25:52.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492650655 0 0) 0x7f3ef415c2f0 con 0x7f3ef40aebd0 2024-04-25T10:25:52.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef4159920 con 0x7f3ef40aebd0 2024-04-25T10:25:52.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428698266 0 0) 0x7f3ed8003530 con 0x7f3ef40721f0 2024-04-25T10:25:52.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef9200640 1 -- 172.21.15.87:0/1305259431 >> v1:172.21.15.169:6789/0 conn(0x7f3ef40aebd0 legacy=0x7f3ef4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef9200640 1 -- 172.21.15.87:0/1305259431 >> v1:172.21.15.87:6789/0 conn(0x7f3ef40721f0 legacy=0x7f3ef4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef9200640 1 -- 172.21.15.87:0/1305259431 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef415c8b0 con 0x7f3ef4159540 2024-04-25T10:25:52.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef9200640 1 --2- 172.21.15.87:0/1305259431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 0x7f3ef40ae690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40097c0 tx=0x7f3ee401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c70a3ece471cee33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b0070 con 0x7f3ef4159540 2024-04-25T10:25:52.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40ab400 con 0x7f3ef4159540 2024-04-25T10:25:52.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.695+0000 7f3ef3e00640 1 -- 172.21.15.87:0/1305259431 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec001980 con 0x7f3ef4159540 2024-04-25T10:25:52.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- 172.21.15.87:0/1305259431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 msgr2=0x7f3ef40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 --2- 172.21.15.87:0/1305259431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 0x7f3ef40ae690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40097c0 tx=0x7f3ee401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- 172.21.15.87:0/1305259431 shutdown_connections 2024-04-25T10:25:52.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 --2- 172.21.15.87:0/1305259431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 0x7f3ef40ae690 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- 172.21.15.87:0/1305259431 >> 172.21.15.87:0/1305259431 conn(0x7f3ef40669a0 msgr2=0x7f3ef4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- 172.21.15.87:0/1305259431 shutdown_connections 2024-04-25T10:25:52.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- 172.21.15.87:0/1305259431 wait complete. 2024-04-25T10:25:52.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 Processor -- start 2024-04-25T10:25:52.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- start start 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ef40721f0 0x7f3ef4155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef40aebd0 0x7f3ef414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 0x7f3ef41571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4169ac0 con 0x7f3ef40721f0 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4169940 con 0x7f3ef4159540 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3efbc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4169c40 con 0x7f3ef40aebd0 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ef40721f0 0x7f3ef4155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ef40721f0 0x7f3ef4155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37440/0 (socket says 172.21.15.87:37440) 2024-04-25T10:25:52.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 -- 172.21.15.87:0/3429965609 learned_addr learned my addr 172.21.15.87:0/3429965609 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9c00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 0x7f3ef41571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3eebe00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef40aebd0 0x7f3ef414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 -- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef40aebd0 msgr2=0x7f3ef414e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef40aebd0 0x7f3ef414e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 -- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 msgr2=0x7f3ef41571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 0x7f3ef41571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 -- 172.21.15.87:0/3429965609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4154860 con 0x7f3ef40721f0 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef9200640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ef40721f0 0x7f3ef4155900 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40ae040 tx=0x7f3ee401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.699+0000 7f3ef1600640 1 -- 172.21.15.87:0/3429965609 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b0050 con 0x7f3ef40721f0 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3ef1600640 1 -- 172.21.15.87:0/3429965609 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40b1040 con 0x7f3ef40721f0 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4154b40 con 0x7f3ef40721f0 2024-04-25T10:25:52.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3ef1600640 1 -- 172.21.15.87:0/3429965609 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b7650 con 0x7f3ef40721f0 2024-04-25T10:25:52.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef4155100 con 0x7f3ef40721f0 2024-04-25T10:25:52.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef404ef30 con 0x7f3ef40721f0 2024-04-25T10:25:52.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3ef1600640 1 -- 172.21.15.87:0/3429965609 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ee40be030 con 0x7f3ef40721f0 2024-04-25T10:25:52.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3ef1600640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ed0066f60 0x7f3ed0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3ef1600640 1 -- 172.21.15.87:0/3429965609 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ee411ee50 con 0x7f3ef40721f0 2024-04-25T10:25:52.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3eebe00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ed0066f60 0x7f3ed0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.703+0000 7f3eebe00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ed0066f60 0x7f3ed0069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3ef415b8a0 tx=0x7f3edc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.707+0000 7f3ef1600640 1 -- 172.21.15.87:0/3429965609 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ee40c5020 con 0x7f3ef40721f0 2024-04-25T10:25:52.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.863+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ef4153be0 con 0x7f3ef40721f0 2024-04-25T10:25:52.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.863+0000 7f3ef1600640 1 -- 172.21.15.87:0/3429965609 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v176) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ee4015fb0 con 0x7f3ef40721f0 2024-04-25T10:25:52.864 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:25:52.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ed0066f60 msgr2=0x7f3ed0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ed0066f60 0x7f3ed0069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3ef415b8a0 tx=0x7f3edc009040 comp rx=0 tx=0).stop 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ef40721f0 msgr2=0x7f3ef4155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ef40721f0 0x7f3ef4155900 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40ae040 tx=0x7f3ee401ac00 comp rx=0 tx=0).stop 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 shutdown_connections 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3ed0066f60 0x7f3ed0069420 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef4159540 0x7f3ef41571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef40aebd0 0x7f3ef414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 --2- 172.21.15.87:0/3429965609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ef40721f0 0x7f3ef4155900 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 >> 172.21.15.87:0/3429965609 conn(0x7f3ef40669a0 msgr2=0x7f3ef415d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 shutdown_connections 2024-04-25T10:25:52.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:52.867+0000 7f3efbc00640 1 -- 172.21.15.87:0/3429965609 wait complete. 2024-04-25T10:25:52.884 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:25:53.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.019+0000 7f24f0e00640 1 Processor -- start 2024-04-25T10:25:53.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24f0e00640 1 -- start start 2024-04-25T10:25:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24f0e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f24ec15df60 0x7f24ec0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24f0e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ec0ccaa0 con 0x7f24ec0d0180 2024-04-25T10:25:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24f0e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ec0ccd80 con 0x7f24ec06aea0 2024-04-25T10:25:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24f0e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec15e340 con 0x7f24ec15df60 2024-04-25T10:25:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f24ec0d0180 0x7f24ec0d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38420/0 (socket says 172.21.15.87:38420) 2024-04-25T10:25:53.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 -- 172.21.15.87:0/2999461330 learned_addr learned my addr 172.21.15.87:0/2999461330 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:53.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9e00640 1 --2- 172.21.15.87:0/2999461330 >> v2:172.21.15.169:3300/0 conn(0x7f24ec15df60 0x7f24ec0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534389563 0 0) 0x7f24ec0ccaa0 con 0x7f24ec0d0180 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d0003530 con 0x7f24ec0d0180 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358352748 0 0) 0x7f24d0003530 con 0x7f24ec0d0180 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 >> v2:172.21.15.169:3300/0 conn(0x7f24ec15df60 msgr2=0x7f24ec0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 --2- 172.21.15.87:0/2999461330 >> v2:172.21.15.169:3300/0 conn(0x7f24ec15df60 0x7f24ec0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 >> v1:172.21.15.169:6789/0 conn(0x7f24ec06aea0 legacy=0x7f24ec06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0cd060 con 0x7f24ec0d0180 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f24d8001680 con 0x7f24ec0d0180 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 >> v1:172.21.15.87:6789/0 conn(0x7f24ec0d0180 legacy=0x7f24ec0d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d00039c0 0x7f24d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d00043c0 0x7f24d0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d0005130 0x7f24d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d000c040 con 0x7f24d00043c0 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d000c1f0 con 0x7f24d00039c0 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d000c3f0 con 0x7f24d0005130 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d00043c0 0x7f24d0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9e00640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d00039c0 0x7f24d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24ea800640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d0005130 0x7f24d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 -- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d0005130 msgr2=0x7f24d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d0005130 0x7f24d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 -- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d00039c0 msgr2=0x7f24d0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d00039c0 0x7f24d0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24e9400640 1 -- 172.21.15.87:0/2999461330 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d0003530 con 0x7f24d00043c0 2024-04-25T10:25:53.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.023+0000 7f24ea800640 1 -- 172.21.15.87:0/2999461330 reap_dead start 2024-04-25T10:25:53.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24e9400640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d00043c0 0x7f24d0004890 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f24d800c2c0 tx=0x7f24d801cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6880a74e0bab42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8007d60 con 0x7f24d00043c0 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d80a9070 con 0x7f24d00043c0 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24e8a00640 1 -- 172.21.15.87:0/2999461330 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d80a5e30 con 0x7f24d00043c0 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24f0e00640 1 -- 172.21.15.87:0/2999461330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d00043c0 msgr2=0x7f24d0004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24f0e00640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d00043c0 0x7f24d0004890 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f24d800c2c0 tx=0x7f24d801cd20 comp rx=0 tx=0).stop 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24f0e00640 1 -- 172.21.15.87:0/2999461330 shutdown_connections 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24f0e00640 1 --2- 172.21.15.87:0/2999461330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d00043c0 0x7f24d0004890 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24f0e00640 1 -- 172.21.15.87:0/2999461330 >> 172.21.15.87:0/2999461330 conn(0x7f24ec0d1910 msgr2=0x7f24ec0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24f0e00640 1 -- 172.21.15.87:0/2999461330 shutdown_connections 2024-04-25T10:25:53.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.027+0000 7f24f0e00640 1 -- 172.21.15.87:0/2999461330 wait complete. 2024-04-25T10:25:53.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 Processor -- start 2024-04-25T10:25:53.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 -- start start 2024-04-25T10:25:53.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d00043c0 0x7f24ec0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24ec14baa0 0x7f24ec0c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec15f930 0x7f24ec15fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0d0e70 con 0x7f24ec14baa0 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0cd620 con 0x7f24ec15f930 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0d0ff0 con 0x7f24d00043c0 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24ec14baa0 0x7f24ec0c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24ec14baa0 0x7f24ec0c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37458/0 (socket says 172.21.15.87:37458) 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 -- 172.21.15.87:0/1110726417 learned_addr learned my addr 172.21.15.87:0/1110726417 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24ea800640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec15f930 0x7f24ec15fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d00043c0 0x7f24ec0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 -- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d00043c0 msgr2=0x7f24ec0c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d00043c0 0x7f24ec0c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 -- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec15f930 msgr2=0x7f24ec15fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec15f930 0x7f24ec15fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 -- 172.21.15.87:0/1110726417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec14bf20 con 0x7f24ec14baa0 2024-04-25T10:25:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e9400640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24ec14baa0 0x7f24ec0c7e20 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f24d800c2c0 tx=0x7f24d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e2000640 1 -- 172.21.15.87:0/1110726417 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8007d60 con 0x7f24ec14baa0 2024-04-25T10:25:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e2000640 1 -- 172.21.15.87:0/1110726417 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d8001070 con 0x7f24ec14baa0 2024-04-25T10:25:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24e2000640 1 -- 172.21.15.87:0/1110726417 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d80a6e60 con 0x7f24ec14baa0 2024-04-25T10:25:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec0c8590 con 0x7f24ec14baa0 2024-04-25T10:25:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec14c6b0 con 0x7f24ec14baa0 2024-04-25T10:25:53.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.031+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ec04ef30 con 0x7f24ec14baa0 2024-04-25T10:25:53.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.035+0000 7f24e2000640 1 -- 172.21.15.87:0/1110726417 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24d80b6030 con 0x7f24ec14baa0 2024-04-25T10:25:53.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.035+0000 7f24e2000640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24cc066f60 0x7f24cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.035+0000 7f24e2000640 1 -- 172.21.15.87:0/1110726417 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24d811e3c0 con 0x7f24ec14baa0 2024-04-25T10:25:53.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.035+0000 7f24e9e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24cc066f60 0x7f24cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.035+0000 7f24e9e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24cc066f60 0x7f24cc069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009c60 tx=0x7f24dc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:53.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.039+0000 7f24e2000640 1 -- 172.21.15.87:0/1110726417 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24d8016a30 con 0x7f24ec14baa0 2024-04-25T10:25:53.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.203+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f24ec06d500 con 0x7f24ec14baa0 2024-04-25T10:25:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.203+0000 7f24e2000640 1 -- 172.21.15.87:0/1110726417 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v176) v1 ==== 80+0+0 (secure 0 0 0) 0x7f24d80eece0 con 0x7f24ec14baa0 2024-04-25T10:25:53.205 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24cc066f60 msgr2=0x7f24cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24cc066f60 0x7f24cc069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009c60 tx=0x7f24dc009190 comp rx=0 tx=0).stop 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24ec14baa0 msgr2=0x7f24ec0c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24ec14baa0 0x7f24ec0c7e20 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f24d800c2c0 tx=0x7f24d0005840 comp rx=0 tx=0).stop 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 shutdown_connections 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24cc066f60 0x7f24cc069420 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ec15f930 0x7f24ec15fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24ec14baa0 0x7f24ec0c7e20 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 --2- 172.21.15.87:0/1110726417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24d00043c0 0x7f24ec0c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 >> 172.21.15.87:0/1110726417 conn(0x7f24ec0d1910 msgr2=0x7f24ec06f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:53.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 shutdown_connections 2024-04-25T10:25:53.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:53.207+0000 7f24f0e00640 1 -- 172.21.15.87:0/1110726417 wait complete. 2024-04-25T10:25:53.226 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:25:53.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:53.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:53.774 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:25:53.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:53.775 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:25:53.785 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:25:53.785 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:53.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:53.793 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:25:53.793 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:53.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:53.837 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:25:53.837 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:53.846 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:53.846 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:54.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.015+0000 7fb917e00640 1 Processor -- start 2024-04-25T10:25:54.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.015+0000 7fb917e00640 1 -- start start 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.015+0000 7fb917e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9100d0040 0x7fb91015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.015+0000 7fb917e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb91015b740 0x7fb9100b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.015+0000 7fb917e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb910071050 con 0x7fb9101550b0 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.015+0000 7fb917e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91015d040 con 0x7fb91015b740 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.015+0000 7fb917e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9100712f0 con 0x7fb9100d0040 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb91015b740 0x7fb9100b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb91015b740 0x7fb9100b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37472/0 (socket says 172.21.15.87:37472) 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 -- 172.21.15.87:0/2483562491 learned_addr learned my addr 172.21.15.87:0/2483562491 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:54.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb914a00640 1 --2- 172.21.15.87:0/2483562491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9100d0040 0x7fb91015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 -- 172.21.15.87:0/2483562491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9100d0040 msgr2=0x7fb91015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 --2- 172.21.15.87:0/2483562491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9100d0040 0x7fb91015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 -- 172.21.15.87:0/2483562491 >> v1:172.21.15.169:6789/0 conn(0x7fb9101550b0 legacy=0x7fb9100cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 -- 172.21.15.87:0/2483562491 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100715d0 con 0x7fb91015b740 2024-04-25T10:25:54.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb915e00640 1 --2- 172.21.15.87:0/2483562491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb91015b740 0x7fb9100b3260 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fb908009e50 tx=0x7fb90801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecd777e2b8ab7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb90fe00640 1 -- 172.21.15.87:0/2483562491 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080a58d0 con 0x7fb91015b740 2024-04-25T10:25:54.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb90fe00640 1 -- 172.21.15.87:0/2483562491 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9080a5a70 con 0x7fb91015b740 2024-04-25T10:25:54.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.019+0000 7fb90fe00640 1 -- 172.21.15.87:0/2483562491 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080a5e40 con 0x7fb91015b740 2024-04-25T10:25:54.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- 172.21.15.87:0/2483562491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb91015b740 msgr2=0x7fb9100b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 --2- 172.21.15.87:0/2483562491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb91015b740 0x7fb9100b3260 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fb908009e50 tx=0x7fb90801ca00 comp rx=0 tx=0).stop 2024-04-25T10:25:54.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- 172.21.15.87:0/2483562491 shutdown_connections 2024-04-25T10:25:54.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 --2- 172.21.15.87:0/2483562491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb91015b740 0x7fb9100b3260 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 --2- 172.21.15.87:0/2483562491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9100d0040 0x7fb91015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- 172.21.15.87:0/2483562491 >> 172.21.15.87:0/2483562491 conn(0x7fb910066b70 msgr2=0x7fb910066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- 172.21.15.87:0/2483562491 shutdown_connections 2024-04-25T10:25:54.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- 172.21.15.87:0/2483562491 wait complete. 2024-04-25T10:25:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 Processor -- start 2024-04-25T10:25:54.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- start start 2024-04-25T10:25:54.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9100d0040 0x7fb910150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9101550b0 0x7fb910157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb91015b740 0x7fb910153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91015bd70 con 0x7fb9101550b0 2024-04-25T10:25:54.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91015bbf0 con 0x7fb9100d0040 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb917e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91015c030 con 0x7fb91015b740 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9101550b0 0x7fb910157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb915400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9100d0040 0x7fb910150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9101550b0 0x7fb910157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37488/0 (socket says 172.21.15.87:37488) 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 -- 172.21.15.87:0/2746732137 learned_addr learned my addr 172.21.15.87:0/2746732137 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb915400640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9100d0040 0x7fb910150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:43286/0 (socket says 172.21.15.87:43286) 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb915e00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb91015b740 0x7fb910153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 -- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb91015b740 msgr2=0x7fb910153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb91015b740 0x7fb910153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 -- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9100d0040 msgr2=0x7fb910150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9100d0040 0x7fb910150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.023+0000 7fb914a00640 1 -- 172.21.15.87:0/2746732137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100b0f30 con 0x7fb9101550b0 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb914a00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9101550b0 0x7fb910157da0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00e960 tx=0x7fb8fc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb90d600640 1 -- 172.21.15.87:0/2746732137 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc017070 con 0x7fb9101550b0 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb90d600640 1 -- 172.21.15.87:0/2746732137 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc00c9a0 con 0x7fb9101550b0 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb90d600640 1 -- 172.21.15.87:0/2746732137 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00cd50 con 0x7fb9101550b0 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb910159e30 con 0x7fb9101550b0 2024-04-25T10:25:54.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91015a3f0 con 0x7fb9101550b0 2024-04-25T10:25:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91004ef30 con 0x7fb9101550b0 2024-04-25T10:25:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb90d600640 1 -- 172.21.15.87:0/2746732137 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8fc01a070 con 0x7fb9101550b0 2024-04-25T10:25:54.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb90d600640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb8f0066f60 0x7fb8f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb90d600640 1 -- 172.21.15.87:0/2746732137 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8fc084ff0 con 0x7fb9101550b0 2024-04-25T10:25:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.027+0000 7fb915400640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb8f0066f60 0x7fb8f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.031+0000 7fb915400640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb8f0066f60 0x7fb8f0069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb910153ac0 tx=0x7fb904009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.031+0000 7fb90d600640 1 -- 172.21.15.87:0/2746732137 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8fc055910 con 0x7fb9101550b0 2024-04-25T10:25:54.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.219+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb910072350 con 0x7fb9101550b0 2024-04-25T10:25:54.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.219+0000 7fb90d600640 1 -- 172.21.15.87:0/2746732137 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fb8fc0599b0 con 0x7fb9101550b0 2024-04-25T10:25:54.223 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:54.223 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":128,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1787812144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1787812144},{"type":"v1","addr":"172.21.15.169:6834","nonce":1787812144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/554048725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":554048725},{"type":"v1","addr":"172.21.15.169:6835","nonce":554048725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1720179922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1720179922},{"type":"v1","addr":"172.21.15.87:6837","nonce":1720179922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:42.498010+0000","modified":"2024-04-25T10:25:45.516622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10920":{"gid":10920,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/3053712751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3053712751},{"type":"v1","addr":"172.21.15.87:6836","nonce":3053712751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 128 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb8f0066f60 msgr2=0x7fb8f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb8f0066f60 0x7fb8f0069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb910153ac0 tx=0x7fb904009040 comp rx=0 tx=0).stop 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9101550b0 msgr2=0x7fb910157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9101550b0 0x7fb910157da0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00e960 tx=0x7fb8fc00ee20 comp rx=0 tx=0).stop 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 shutdown_connections 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb8f0066f60 0x7fb8f0069420 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb91015b740 0x7fb910153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9101550b0 0x7fb910157da0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 --2- 172.21.15.87:0/2746732137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9100d0040 0x7fb910150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 >> 172.21.15.87:0/2746732137 conn(0x7fb910066b70 msgr2=0x7fb9100b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 shutdown_connections 2024-04-25T10:25:54.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.223+0000 7fb917e00640 1 -- 172.21.15.87:0/2746732137 wait complete. 2024-04-25T10:25:54.241 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:54.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.375+0000 7f290d200640 1 Processor -- start 2024-04-25T10:25:54.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.375+0000 7f290d200640 1 -- start start 2024-04-25T10:25:54.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f290d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2908071db0 0x7f2908165730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f290d200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29080b3470 con 0x7f290815a780 2024-04-25T10:25:54.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f290d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290806de60 con 0x7f2908071260 2024-04-25T10:25:54.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f290d200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290806cdc0 con 0x7f2908071db0 2024-04-25T10:25:54.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2907400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f290815a780 0x7f290815edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38432/0 (socket says 172.21.15.87:38432) 2024-04-25T10:25:54.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2907400640 1 -- 172.21.15.87:0/1712712494 learned_addr learned my addr 172.21.15.87:0/1712712494 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:54.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2907e00640 1 --2- 172.21.15.87:0/1712712494 >> v2:172.21.15.87:3300/0 conn(0x7f2908071db0 0x7f2908165730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2906a00640 1 -- 172.21.15.87:0/1712712494 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525522065 0 0) 0x7f28f8001120 con 0x7f290815a780 2024-04-25T10:25:54.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2906a00640 1 -- 172.21.15.87:0/1712712494 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ec003530 con 0x7f290815a780 2024-04-25T10:25:54.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2907e00640 1 -- 172.21.15.87:0/1712712494 >> v1:172.21.15.169:6789/0 conn(0x7f2908071260 legacy=0x7f29080716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:54.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2907e00640 1 -- 172.21.15.87:0/1712712494 >> v1:172.21.15.87:6789/0 conn(0x7f290815a780 legacy=0x7f290815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.379+0000 7f2907e00640 1 -- 172.21.15.87:0/1712712494 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290806d0a0 con 0x7f2908071db0 2024-04-25T10:25:54.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f2907e00640 1 --2- 172.21.15.87:0/1712712494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2908071db0 0x7f2908165730 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f2900009d90 tx=0x7f290001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4187fc6d877c49f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f2906a00640 1 -- 172.21.15.87:0/1712712494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000b2070 con 0x7f2908071db0 2024-04-25T10:25:54.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f2906a00640 1 -- 172.21.15.87:0/1712712494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29000a5830 con 0x7f2908071db0 2024-04-25T10:25:54.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f2906a00640 1 -- 172.21.15.87:0/1712712494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000a5c00 con 0x7f2908071db0 2024-04-25T10:25:54.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 -- 172.21.15.87:0/1712712494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2908071db0 msgr2=0x7f2908165730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 --2- 172.21.15.87:0/1712712494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2908071db0 0x7f2908165730 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f2900009d90 tx=0x7f290001c970 comp rx=0 tx=0).stop 2024-04-25T10:25:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 -- 172.21.15.87:0/1712712494 shutdown_connections 2024-04-25T10:25:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 --2- 172.21.15.87:0/1712712494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2908071db0 0x7f2908165730 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 -- 172.21.15.87:0/1712712494 >> 172.21.15.87:0/1712712494 conn(0x7f29080743f0 msgr2=0x7f29080669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 -- 172.21.15.87:0/1712712494 shutdown_connections 2024-04-25T10:25:54.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 -- 172.21.15.87:0/1712712494 wait complete. 2024-04-25T10:25:54.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 Processor -- start 2024-04-25T10:25:54.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.383+0000 7f290d200640 1 -- start start 2024-04-25T10:25:54.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908071260 0x7f2908149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2908071db0 0x7f29081519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f290815a780 0x7f29080ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908168610 con 0x7f290815a780 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908168490 con 0x7f2908071db0 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2908168790 con 0x7f2908071260 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f290815a780 0x7f29080ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f290815a780 0x7f29080ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37504/0 (socket says 172.21.15.87:37504) 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 -- 172.21.15.87:0/2816177285 learned_addr learned my addr 172.21.15.87:0/2816177285 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907400640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908071260 0x7f2908149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f28fea00640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2908071db0 0x7f29081519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 -- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908071260 msgr2=0x7f2908149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908071260 0x7f2908149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 -- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2908071db0 msgr2=0x7f29081519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2908071db0 0x7f29081519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 -- 172.21.15.87:0/2816177285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2908149cb0 con 0x7f290815a780 2024-04-25T10:25:54.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f2907e00640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f290815a780 0x7f29080ccc40 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f29000b0040 tx=0x7f2900019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f28ffe00640 1 -- 172.21.15.87:0/2816177285 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000b2050 con 0x7f290815a780 2024-04-25T10:25:54.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f28ffe00640 1 -- 172.21.15.87:0/2816177285 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29000be070 con 0x7f290815a780 2024-04-25T10:25:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f28ffe00640 1 -- 172.21.15.87:0/2816177285 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000ad870 con 0x7f290815a780 2024-04-25T10:25:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29081529a0 con 0x7f290815a780 2024-04-25T10:25:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29081526c0 con 0x7f290815a780 2024-04-25T10:25:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2908075120 con 0x7f290815a780 2024-04-25T10:25:54.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.387+0000 7f28ffe00640 1 -- 172.21.15.87:0/2816177285 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29000ada10 con 0x7f290815a780 2024-04-25T10:25:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.391+0000 7f28ffe00640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f28e4066f60 0x7f28e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.391+0000 7f28ffe00640 1 -- 172.21.15.87:0/2816177285 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f290011fe20 con 0x7f290815a780 2024-04-25T10:25:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.391+0000 7f2907400640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f28e4066f60 0x7f28e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.391+0000 7f2907400640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f28e4066f60 0x7f28e4069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f28f80012c0 tx=0x7f28f8007b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.395+0000 7f28ffe00640 1 -- 172.21.15.87:0/2816177285 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29081529a0 con 0x7f290815a780 2024-04-25T10:25:54.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:54.489 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:54.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.551+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f290815e620 con 0x7f290815a780 2024-04-25T10:25:54.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.551+0000 7f28ffe00640 1 -- 172.21.15.87:0/2816177285 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f2900016560 con 0x7f290815a780 2024-04-25T10:25:54.551 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:54.551 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":176,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:44.125291+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:40.866786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:41.487045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.551+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f28e4066f60 msgr2=0x7f28e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.551+0000 7f290d200640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f28e4066f60 0x7f28e4069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f28f80012c0 tx=0x7f28f8007b80 comp rx=0 tx=0).stop 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.551+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f290815a780 msgr2=0x7f29080ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.551+0000 7f290d200640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f290815a780 0x7f29080ccc40 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f29000b0040 tx=0x7f2900019710 comp rx=0 tx=0).stop 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 shutdown_connections 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f28e4066f60 0x7f28e4069420 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f290815a780 0x7f29080ccc40 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2908071db0 0x7f29081519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 --2- 172.21.15.87:0/2816177285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908071260 0x7f2908149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 >> 172.21.15.87:0/2816177285 conn(0x7f29080743f0 msgr2=0x7f29080cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 shutdown_connections 2024-04-25T10:25:54.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.555+0000 7f290d200640 1 -- 172.21.15.87:0/2816177285 wait complete. 2024-04-25T10:25:54.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:25:40.866786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:25:41.487045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:54.573 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6051800640 1 Processor -- start 2024-04-25T10:25:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6051800640 1 -- start start 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6051800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f604c071e40 0x7f604c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6051800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f604c06eee0 0x7f604c15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6051800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c15f720 con 0x7f604c072220 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6051800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c15f9c0 con 0x7f604c071e40 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6051800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c15fca0 con 0x7f604c06eee0 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f604c071e40 0x7f604c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f604c071e40 0x7f604c15b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37516/0 (socket says 172.21.15.87:37516) 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604aa00640 1 -- 172.21.15.87:0/3573634686 learned_addr learned my addr 172.21.15.87:0/3573634686 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f6042000640 1 --1- 172.21.15.87:0/3573634686 >> v1:172.21.15.87:6789/0 conn(0x7f604c072220 0x7f604c06e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38442/0 (socket says 172.21.15.87:38442) 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604b400640 1 --2- 172.21.15.87:0/3573634686 >> v2:172.21.15.169:3300/0 conn(0x7f604c06eee0 0x7f604c15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604aa00640 1 -- 172.21.15.87:0/3573634686 >> v2:172.21.15.169:3300/0 conn(0x7f604c06eee0 msgr2=0x7f604c15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:54.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604aa00640 1 --2- 172.21.15.87:0/3573634686 >> v2:172.21.15.169:3300/0 conn(0x7f604c06eee0 0x7f604c15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604aa00640 1 -- 172.21.15.87:0/3573634686 >> v1:172.21.15.87:6789/0 conn(0x7f604c072220 legacy=0x7f604c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604aa00640 1 -- 172.21.15.87:0/3573634686 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c15ff80 con 0x7f604c071e40 2024-04-25T10:25:54.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.755+0000 7f604a000640 1 -- 172.21.15.87:0/3573634686 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926014445 0 0) 0x7f604c15f720 con 0x7f604c072220 2024-04-25T10:25:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f604aa00640 1 --2- 172.21.15.87:0/3573634686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c071e40 0x7f604c15b460 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f604c0cdda0 tx=0x7f603c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aee6c500a5a0577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f604a000640 1 -- 172.21.15.87:0/3573634686 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5840 con 0x7f604c071e40 2024-04-25T10:25:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f604a000640 1 -- 172.21.15.87:0/3573634686 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0a59e0 con 0x7f604c071e40 2024-04-25T10:25:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f604a000640 1 -- 172.21.15.87:0/3573634686 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5db0 con 0x7f604c071e40 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 -- 172.21.15.87:0/3573634686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c071e40 msgr2=0x7f604c15b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 --2- 172.21.15.87:0/3573634686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c071e40 0x7f604c15b460 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f604c0cdda0 tx=0x7f603c01c960 comp rx=0 tx=0).stop 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 -- 172.21.15.87:0/3573634686 shutdown_connections 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 --2- 172.21.15.87:0/3573634686 >> v2:172.21.15.169:3300/0 conn(0x7f604c06eee0 0x7f604c15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 --2- 172.21.15.87:0/3573634686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c071e40 0x7f604c15b460 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 -- 172.21.15.87:0/3573634686 >> 172.21.15.87:0/3573634686 conn(0x7f604c06ac90 msgr2=0x7f604c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 -- 172.21.15.87:0/3573634686 shutdown_connections 2024-04-25T10:25:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.759+0000 7f6051800640 1 -- 172.21.15.87:0/3573634686 wait complete. 2024-04-25T10:25:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 Processor -- start 2024-04-25T10:25:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 -- start start 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c06eee0 0x7f604c1f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f604c071e40 0x7f604c14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f604c072220 0x7f604c151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0cf870 con 0x7f604c06eee0 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0cf6f0 con 0x7f604c072220 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0cf9f0 con 0x7f604c071e40 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c06eee0 0x7f604c1f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c06eee0 0x7f604c1f6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37522/0 (socket says 172.21.15.87:37522) 2024-04-25T10:25:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 -- 172.21.15.87:0/2122350718 learned_addr learned my addr 172.21.15.87:0/2122350718 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604b400640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f604c072220 0x7f604c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6042000640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f604c071e40 0x7f604c14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 -- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f604c071e40 msgr2=0x7f604c14bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f604c071e40 0x7f604c14bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 -- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f604c072220 msgr2=0x7f604c151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f604c072220 0x7f604c151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 -- 172.21.15.87:0/2122350718 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c1f6ec0 con 0x7f604c06eee0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6042000640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f604c071e40 0x7f604c14bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f604aa00640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c06eee0 0x7f604c1f6980 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f603c0af040 tx=0x7f603c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6043400640 1 -- 172.21.15.87:0/2122350718 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0bc020 con 0x7f604c06eee0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6043400640 1 -- 172.21.15.87:0/2122350718 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0b88e0 con 0x7f604c06eee0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c15d2b0 con 0x7f604c06eee0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6043400640 1 -- 172.21.15.87:0/2122350718 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0b8cd0 con 0x7f604c06eee0 2024-04-25T10:25:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.763+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c15d870 con 0x7f604c06eee0 2024-04-25T10:25:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.767+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c04ef30 con 0x7f604c06eee0 2024-04-25T10:25:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.767+0000 7f6043400640 1 -- 172.21.15.87:0/2122350718 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f603c0bf020 con 0x7f604c06eee0 2024-04-25T10:25:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.767+0000 7f6043400640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6028066f60 0x7f6028069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.767+0000 7f6043400640 1 -- 172.21.15.87:0/2122350718 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f603c120610 con 0x7f604c06eee0 2024-04-25T10:25:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.767+0000 7f6042000640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6028066f60 0x7f6028069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.767+0000 7f6042000640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6028066f60 0x7f6028069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f604c15c320 tx=0x7f6034007cb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.771+0000 7f6043400640 1 -- 172.21.15.87:0/2122350718 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f603c0165e0 con 0x7f604c06eee0 2024-04-25T10:25:54.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.927+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f604c0b1e50 con 0x7f604c06eee0 2024-04-25T10:25:54.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.927+0000 7f6043400640 1 -- 172.21.15.87:0/2122350718 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7f603c0f1590 con 0x7f604c06eee0 2024-04-25T10:25:54.927 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:25:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6028066f60 msgr2=0x7f6028069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6028066f60 0x7f6028069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f604c15c320 tx=0x7f6034007cb0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c06eee0 msgr2=0x7f604c1f6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c06eee0 0x7f604c1f6980 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f603c0af040 tx=0x7f603c019700 comp rx=0 tx=0).stop 2024-04-25T10:25:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 shutdown_connections 2024-04-25T10:25:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6028066f60 0x7f6028069420 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f604c072220 0x7f604c151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f604c071e40 0x7f604c14bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 --2- 172.21.15.87:0/2122350718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f604c06eee0 0x7f604c1f6980 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 >> 172.21.15.87:0/2122350718 conn(0x7f604c06ac90 msgr2=0x7f604c150ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 shutdown_connections 2024-04-25T10:25:54.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:54.931+0000 7f6051800640 1 -- 172.21.15.87:0/2122350718 wait complete. 2024-04-25T10:25:54.948 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:25:55.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd753000640 1 Processor -- start 2024-04-25T10:25:55.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd753000640 1 -- start start 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd753000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd74c0aef80 0x7fd74c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd753000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74c15b080 con 0x7fd74c0b31c0 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd753000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74c15adb0 con 0x7fd74c070350 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd753000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74c15eb60 con 0x7fd74c0aef80 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd74c0b31c0 0x7fd74c0cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38450/0 (socket says 172.21.15.87:38450) 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 -- 172.21.15.87:0/4127552203 learned_addr learned my addr 172.21.15.87:0/4127552203 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854041486 0 0) 0x7fd74c15b080 con 0x7fd74c0b31c0 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74be00640 1 --2- 172.21.15.87:0/4127552203 >> v2:172.21.15.169:3300/0 conn(0x7fd74c0aef80 0x7fd74c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd730003530 con 0x7fd74c0b31c0 2024-04-25T10:25:55.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062017771 0 0) 0x7fd730003530 con 0x7fd74c0b31c0 2024-04-25T10:25:55.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 >> v2:172.21.15.169:3300/0 conn(0x7fd74c0aef80 msgr2=0x7fd74c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 --2- 172.21.15.87:0/4127552203 >> v2:172.21.15.169:3300/0 conn(0x7fd74c0aef80 0x7fd74c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 >> v1:172.21.15.169:6789/0 conn(0x7fd74c070350 legacy=0x7fd74c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c15ee40 con 0x7fd74c0b31c0 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd744001a00 con 0x7fd74c0b31c0 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 >> v1:172.21.15.87:6789/0 conn(0x7fd74c0b31c0 legacy=0x7fd74c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7300039c0 0x7fd730003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd730004810 0x7fd730004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd73000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c15eb60 con 0x7fd7300051d0 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73000c070 con 0x7fd7300039c0 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73000c270 con 0x7fd730004810 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd73000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd74be00640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7300039c0 0x7fd730003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd743e00640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd730004810 0x7fd730004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 -- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd730004810 msgr2=0x7fd730004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd730004810 0x7fd730004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 -- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7300039c0 msgr2=0x7fd730003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7300039c0 0x7fd730003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.087+0000 7fd751000640 1 -- 172.21.15.87:0/4127552203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd730003530 con 0x7fd7300051d0 2024-04-25T10:25:55.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd751000640 1 -- 172.21.15.87:0/4127552203 reap_dead start 2024-04-25T10:25:55.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd751000640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd73000b9d0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fd74400a860 tx=0x7fd74401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cedd3fbd07225b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400a480 con 0x7fd7300051d0 2024-04-25T10:25:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7440ad630 con 0x7fd7300051d0 2024-04-25T10:25:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd74b400640 1 -- 172.21.15.87:0/4127552203 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440ada00 con 0x7fd7300051d0 2024-04-25T10:25:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd753000640 1 -- 172.21.15.87:0/4127552203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 msgr2=0x7fd73000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd753000640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd73000b9d0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fd74400a860 tx=0x7fd74401d380 comp rx=0 tx=0).stop 2024-04-25T10:25:55.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd753000640 1 -- 172.21.15.87:0/4127552203 shutdown_connections 2024-04-25T10:25:55.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd753000640 1 --2- 172.21.15.87:0/4127552203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd73000b9d0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd753000640 1 -- 172.21.15.87:0/4127552203 >> 172.21.15.87:0/4127552203 conn(0x7fd74c0669a0 msgr2=0x7fd74c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd753000640 1 -- 172.21.15.87:0/4127552203 shutdown_connections 2024-04-25T10:25:55.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.091+0000 7fd753000640 1 -- 172.21.15.87:0/4127552203 wait complete. 2024-04-25T10:25:55.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 Processor -- start 2024-04-25T10:25:55.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 -- start start 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd74c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd74c155ac0 0x7fd74c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd74c152ab0 0x7fd74c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c0b3610 con 0x7fd7300051d0 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c071a00 con 0x7fd74c155ac0 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c0b3790 con 0x7fd74c152ab0 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd74c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd74c0b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37538/0 (socket says 172.21.15.87:37538) 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 -- 172.21.15.87:0/4156233281 learned_addr learned my addr 172.21.15.87:0/4156233281 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd743e00640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd74c155ac0 0x7fd74c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd751000640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd74c152ab0 0x7fd74c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 -- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd74c152ab0 msgr2=0x7fd74c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd74c152ab0 0x7fd74c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 -- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd74c155ac0 msgr2=0x7fd74c158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd74c155ac0 0x7fd74c158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 -- 172.21.15.87:0/4156233281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c154a20 con 0x7fd7300051d0 2024-04-25T10:25:55.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd74be00640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd74c0b1120 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fd73c011990 tx=0x7fd73c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd748c00640 1 -- 172.21.15.87:0/4156233281 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00fad0 con 0x7fd7300051d0 2024-04-25T10:25:55.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd748c00640 1 -- 172.21.15.87:0/4156233281 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c00fc70 con 0x7fd7300051d0 2024-04-25T10:25:55.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd748c00640 1 -- 172.21.15.87:0/4156233281 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c014440 con 0x7fd7300051d0 2024-04-25T10:25:55.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c154d00 con 0x7fd7300051d0 2024-04-25T10:25:55.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.095+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c1552c0 con 0x7fd7300051d0 2024-04-25T10:25:55.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.099+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74c04ef30 con 0x7fd7300051d0 2024-04-25T10:25:55.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.099+0000 7fd748c00640 1 -- 172.21.15.87:0/4156233281 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd73c012040 con 0x7fd7300051d0 2024-04-25T10:25:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.099+0000 7fd748c00640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd728066f90 0x7fd728069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.099+0000 7fd748c00640 1 -- 172.21.15.87:0/4156233281 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd73c085400 con 0x7fd7300051d0 2024-04-25T10:25:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.099+0000 7fd743e00640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd728066f90 0x7fd728069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.099+0000 7fd743e00640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd728066f90 0x7fd728069450 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0b1c70 tx=0x7fd734009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.103+0000 7fd748c00640 1 -- 172.21.15.87:0/4156233281 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd73c019030 con 0x7fd7300051d0 2024-04-25T10:25:55.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.259+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd74c0b2fa0 con 0x7fd7300051d0 2024-04-25T10:25:55.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.259+0000 7fd748c00640 1 -- 172.21.15.87:0/4156233281 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v176) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd73c055dc0 con 0x7fd7300051d0 2024-04-25T10:25:55.260 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:25:55.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd728066f90 msgr2=0x7fd728069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd728066f90 0x7fd728069450 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0b1c70 tx=0x7fd734009040 comp rx=0 tx=0).stop 2024-04-25T10:25:55.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 msgr2=0x7fd74c0b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd74c0b1120 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fd73c011990 tx=0x7fd73c011e50 comp rx=0 tx=0).stop 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 shutdown_connections 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd728066f90 0x7fd728069450 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd74c152ab0 0x7fd74c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd74c155ac0 0x7fd74c158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 --2- 172.21.15.87:0/4156233281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7300051d0 0x7fd74c0b1120 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 >> 172.21.15.87:0/4156233281 conn(0x7fd74c0669a0 msgr2=0x7fd74c0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 shutdown_connections 2024-04-25T10:25:55.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.263+0000 7fd753000640 1 -- 172.21.15.87:0/4156233281 wait complete. 2024-04-25T10:25:55.280 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:25:55.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.419+0000 7f80cbc00640 1 Processor -- start 2024-04-25T10:25:55.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.419+0000 7f80cbc00640 1 -- start start 2024-04-25T10:25:55.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80cbc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80c4159540 0x7f80c40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80cbc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f80c40721f0 0x7f80c4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80cbc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c4159920 con 0x7f80c40aebd0 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80cbc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c415c2f0 con 0x7f80c4159540 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80cbc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c415c5d0 con 0x7f80c40721f0 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80c4159540 0x7f80c40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80c4159540 0x7f80c40ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37544/0 (socket says 172.21.15.87:37544) 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 -- 172.21.15.87:0/1150426076 learned_addr learned my addr 172.21.15.87:0/1150426076 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f80c40aebd0 0x7f80c4071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38462/0 (socket says 172.21.15.87:38462) 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9c00640 1 --2- 172.21.15.87:0/1150426076 >> v2:172.21.15.169:3300/0 conn(0x7f80c40721f0 0x7f80c4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 -- 172.21.15.87:0/1150426076 >> v2:172.21.15.169:3300/0 conn(0x7f80c40721f0 msgr2=0x7f80c4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 --2- 172.21.15.87:0/1150426076 >> v2:172.21.15.169:3300/0 conn(0x7f80c40721f0 0x7f80c4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 -- 172.21.15.87:0/1150426076 >> v1:172.21.15.87:6789/0 conn(0x7f80c40aebd0 legacy=0x7f80c4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 -- 172.21.15.87:0/1150426076 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c415c8b0 con 0x7f80c4159540 2024-04-25T10:25:55.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c3400640 1 -- 172.21.15.87:0/1150426076 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35112230 0 0) 0x7f80c4159920 con 0x7f80c40aebd0 2024-04-25T10:25:55.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c9200640 1 --2- 172.21.15.87:0/1150426076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c40ae690 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f80c40cd200 tx=0x7f80b401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6069afe3738fabc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.423+0000 7f80c3400640 1 -- 172.21.15.87:0/1150426076 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b40a57d0 con 0x7f80c4159540 2024-04-25T10:25:55.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c3400640 1 -- 172.21.15.87:0/1150426076 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b40a5970 con 0x7f80c4159540 2024-04-25T10:25:55.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c3400640 1 -- 172.21.15.87:0/1150426076 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b40a5d40 con 0x7f80c4159540 2024-04-25T10:25:55.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- 172.21.15.87:0/1150426076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 msgr2=0x7f80c40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 --2- 172.21.15.87:0/1150426076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c40ae690 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f80c40cd200 tx=0x7f80b401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- 172.21.15.87:0/1150426076 shutdown_connections 2024-04-25T10:25:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 --2- 172.21.15.87:0/1150426076 >> v2:172.21.15.169:3300/0 conn(0x7f80c40721f0 0x7f80c4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 --2- 172.21.15.87:0/1150426076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c40ae690 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- 172.21.15.87:0/1150426076 >> 172.21.15.87:0/1150426076 conn(0x7f80c40669a0 msgr2=0x7f80c4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- 172.21.15.87:0/1150426076 shutdown_connections 2024-04-25T10:25:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- 172.21.15.87:0/1150426076 wait complete. 2024-04-25T10:25:55.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 Processor -- start 2024-04-25T10:25:55.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- start start 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80c40721f0 0x7f80c4154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80c40aebd0 0x7f80c40cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c4155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4071220 con 0x7f80c4159540 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40710a0 con 0x7f80c40721f0 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80cbc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40713a0 con 0x7f80c40aebd0 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c4155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c4155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37560/0 (socket says 172.21.15.87:37560) 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 -- 172.21.15.87:0/2232354722 learned_addr learned my addr 172.21.15.87:0/2232354722 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9200640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80c40721f0 0x7f80c4154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c3e00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80c40aebd0 0x7f80c40cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 -- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80c40aebd0 msgr2=0x7f80c40cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80c40aebd0 0x7f80c40cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 -- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80c40721f0 msgr2=0x7f80c4154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80c40721f0 0x7f80c4154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9c00640 1 -- 172.21.15.87:0/2232354722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c41561e0 con 0x7f80c4159540 2024-04-25T10:25:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.427+0000 7f80c9200640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80c40721f0 0x7f80c4154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c9c00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c4155360 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d830 tx=0x7f80bc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c0c00640 1 -- 172.21.15.87:0/2232354722 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00b6c0 con 0x7f80c4159540 2024-04-25T10:25:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c0c00640 1 -- 172.21.15.87:0/2232354722 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc019070 con 0x7f80c4159540 2024-04-25T10:25:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c4155f00 con 0x7f80c4159540 2024-04-25T10:25:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c0c00640 1 -- 172.21.15.87:0/2232354722 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00baa0 con 0x7f80c4159540 2024-04-25T10:25:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c4154270 con 0x7f80c4159540 2024-04-25T10:25:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c404ef30 con 0x7f80c4159540 2024-04-25T10:25:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c0c00640 1 -- 172.21.15.87:0/2232354722 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80bc01e460 con 0x7f80c4159540 2024-04-25T10:25:55.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c0c00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80a4066f60 0x7f80a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c0c00640 1 -- 172.21.15.87:0/2232354722 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80bc0857b0 con 0x7f80c4159540 2024-04-25T10:25:55.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.431+0000 7f80c9200640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80a4066f60 0x7f80a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.435+0000 7f80c9200640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80a4066f60 0x7f80a4069420 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f80b40ae000 tx=0x7f80b40a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.435+0000 7f80c0c00640 1 -- 172.21.15.87:0/2232354722 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80bc056720 con 0x7f80c4159540 2024-04-25T10:25:55.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.591+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80c406d7f0 con 0x7f80c4159540 2024-04-25T10:25:55.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.591+0000 7f80c0c00640 1 -- 172.21.15.87:0/2232354722 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v176) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80bc056020 con 0x7f80c4159540 2024-04-25T10:25:55.591 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:25:55.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80a4066f60 msgr2=0x7f80a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80a4066f60 0x7f80a4069420 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f80b40ae000 tx=0x7f80b40a7070 comp rx=0 tx=0).stop 2024-04-25T10:25:55.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 msgr2=0x7f80c4155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c4155360 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d830 tx=0x7f80bc00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 shutdown_connections 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f80a4066f60 0x7f80a4069420 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c4159540 0x7f80c4155360 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80c40aebd0 0x7f80c40cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 --2- 172.21.15.87:0/2232354722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80c40721f0 0x7f80c4154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 >> 172.21.15.87:0/2232354722 conn(0x7f80c40669a0 msgr2=0x7f80c415fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 shutdown_connections 2024-04-25T10:25:55.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.595+0000 7f80cbc00640 1 -- 172.21.15.87:0/2232354722 wait complete. 2024-04-25T10:25:55.612 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:55.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5971200640 1 Processor -- start 2024-04-25T10:25:55.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5971200640 1 -- start start 2024-04-25T10:25:55.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5971200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 0x7f596c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5971200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f596c15b910 0x7f596c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5971200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c071050 con 0x7f596c0cfe70 2024-04-25T10:25:55.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5971200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c15bda0 con 0x7f596c15b910 2024-04-25T10:25:55.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5971200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15d040 con 0x7f596c1550b0 2024-04-25T10:25:55.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5969400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f596c0cfe70 0x7f596c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38470/0 (socket says 172.21.15.87:38470) 2024-04-25T10:25:55.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.747+0000 7f5969400640 1 -- 172.21.15.87:0/4028199932 learned_addr learned my addr 172.21.15.87:0/4028199932 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5969e00640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 0x7f596c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491025064 0 0) 0x7f596c071050 con 0x7f596c0cfe70 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f596a800640 1 --2- 172.21.15.87:0/4028199932 >> v2:172.21.15.169:3300/0 conn(0x7f596c15b910 0x7f596c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5950003530 con 0x7f596c0cfe70 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3982451500 0 0) 0x7f5950003530 con 0x7f596c0cfe70 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 msgr2=0x7f596c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 0x7f596c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 >> v2:172.21.15.169:3300/0 conn(0x7f596c15b910 msgr2=0x7f596c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 --2- 172.21.15.87:0/4028199932 >> v2:172.21.15.169:3300/0 conn(0x7f596c15b910 0x7f596c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0712f0 con 0x7f596c0cfe70 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f5958001680 con 0x7f596c0cfe70 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 >> v1:172.21.15.87:6789/0 conn(0x7f596c0cfe70 legacy=0x7f596c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 0x7f59500039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5950003f40 0x7f5950004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5950004d50 0x7f59500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15d040 con 0x7f5950004d50 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595000b9d0 con 0x7f5950003f40 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595000bb50 con 0x7f596c1550b0 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f596a800640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5950004d50 0x7f59500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5969e00640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 0x7f59500039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f5969400640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5950003f40 0x7f5950004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f596a800640 1 -- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 msgr2=0x7f59500039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f596a800640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f596c1550b0 0x7f59500039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f596a800640 1 -- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5950003f40 msgr2=0x7f5950004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f596a800640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5950003f40 0x7f5950004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.751+0000 7f596a800640 1 -- 172.21.15.87:0/4028199932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950003530 con 0x7f5950004d50 2024-04-25T10:25:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f596a800640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5950004d50 0x7f59500051d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f5964009670 tx=0x7f596401ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e470cf67c32ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640b4070 con 0x7f5950004d50 2024-04-25T10:25:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59640a78e0 con 0x7f5950004d50 2024-04-25T10:25:55.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5968a00640 1 -- 172.21.15.87:0/4028199932 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640096b0 con 0x7f5950004d50 2024-04-25T10:25:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- 172.21.15.87:0/4028199932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5950004d50 msgr2=0x7f59500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 --2- 172.21.15.87:0/4028199932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5950004d50 0x7f59500051d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f5964009670 tx=0x7f596401ec80 comp rx=0 tx=0).stop 2024-04-25T10:25:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f596a800640 1 -- 172.21.15.87:0/4028199932 reap_dead start 2024-04-25T10:25:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- 172.21.15.87:0/4028199932 shutdown_connections 2024-04-25T10:25:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- 172.21.15.87:0/4028199932 >> 172.21.15.87:0/4028199932 conn(0x7f596c066b70 msgr2=0x7f596c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- 172.21.15.87:0/4028199932 shutdown_connections 2024-04-25T10:25:55.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- 172.21.15.87:0/4028199932 wait complete. 2024-04-25T10:25:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 Processor -- start 2024-04-25T10:25:55.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- start start 2024-04-25T10:25:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5950004d50 0x7f596c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f596c0ce220 0x7f596c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596c0cacd0 0x7f596c0cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c071b90 con 0x7f596c0ce220 2024-04-25T10:25:55.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0718b0 con 0x7f596c0cacd0 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.755+0000 7f5971200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0720b0 con 0x7f5950004d50 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f596c0ce220 0x7f596c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f596c0ce220 0x7f596c0ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37588/0 (socket says 172.21.15.87:37588) 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 -- 172.21.15.87:0/121948276 learned_addr learned my addr 172.21.15.87:0/121948276 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969e00640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5950004d50 0x7f596c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f596a800640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596c0cacd0 0x7f596c0cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 -- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5950004d50 msgr2=0x7f596c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5950004d50 0x7f596c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 -- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596c0cacd0 msgr2=0x7f596c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596c0cacd0 0x7f596c0cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 -- 172.21.15.87:0/121948276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c156250 con 0x7f596c0ce220 2024-04-25T10:25:55.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5969400640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f596c0ce220 0x7f596c0ce660 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f595800e7e0 tx=0x7f595800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5962000640 1 -- 172.21.15.87:0/121948276 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595800c8d0 con 0x7f596c0ce220 2024-04-25T10:25:55.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5962000640 1 -- 172.21.15.87:0/121948276 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5958018070 con 0x7f596c0ce220 2024-04-25T10:25:55.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5962000640 1 -- 172.21.15.87:0/121948276 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595800cd10 con 0x7f596c0ce220 2024-04-25T10:25:55.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c156530 con 0x7f596c0ce220 2024-04-25T10:25:55.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c156af0 con 0x7f596c0ce220 2024-04-25T10:25:55.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c04ef30 con 0x7f596c0ce220 2024-04-25T10:25:55.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5962000640 1 -- 172.21.15.87:0/121948276 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f595801d460 con 0x7f596c0ce220 2024-04-25T10:25:55.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5962000640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f594c066f60 0x7f594c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.759+0000 7f5962000640 1 -- 172.21.15.87:0/121948276 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5958084ec0 con 0x7f596c0ce220 2024-04-25T10:25:55.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.763+0000 7f5969e00640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f594c066f60 0x7f594c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.763+0000 7f5969e00640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f594c066f60 0x7f594c069420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f595c014510 tx=0x7f595c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.763+0000 7f5962000640 1 -- 172.21.15.87:0/121948276 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59580557f0 con 0x7f596c0ce220 2024-04-25T10:25:55.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f596c0b2ca0 con 0x7f596c0ce220 2024-04-25T10:25:55.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5962000640 1 -- 172.21.15.87:0/121948276 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f5958059890 con 0x7f596c0ce220 2024-04-25T10:25:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:55.950 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":128,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1787812144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1787812144},{"type":"v1","addr":"172.21.15.169:6834","nonce":1787812144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/554048725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":554048725},{"type":"v1","addr":"172.21.15.169:6835","nonce":554048725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1720179922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1720179922},{"type":"v1","addr":"172.21.15.87:6837","nonce":1720179922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:42.498010+0000","modified":"2024-04-25T10:25:45.516622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10920":{"gid":10920,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/3053712751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3053712751},{"type":"v1","addr":"172.21.15.87:6836","nonce":3053712751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:25:55.950 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 128 2024-04-25T10:25:55.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f594c066f60 msgr2=0x7f594c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f594c066f60 0x7f594c069420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f595c014510 tx=0x7f595c00b040 comp rx=0 tx=0).stop 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f596c0ce220 msgr2=0x7f596c0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f596c0ce220 0x7f596c0ce660 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f595800e7e0 tx=0x7f595800eca0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 shutdown_connections 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f594c066f60 0x7f594c069420 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596c0cacd0 0x7f596c0cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f596c0ce220 0x7f596c0ce660 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 --2- 172.21.15.87:0/121948276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5950004d50 0x7f596c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 >> 172.21.15.87:0/121948276 conn(0x7f596c066b70 msgr2=0x7f596c06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 shutdown_connections 2024-04-25T10:25:55.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:55.951+0000 7f5971200640 1 -- 172.21.15.87:0/121948276 wait complete. 2024-04-25T10:25:55.970 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:56.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9520c00640 1 Processor -- start 2024-04-25T10:25:56.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9520c00640 1 -- start start 2024-04-25T10:25:56.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9520c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0d2490 0x7f951c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9520c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c0d06a0 con 0x7f951c0ce830 2024-04-25T10:25:56.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9520c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c0b4960 con 0x7f951c15bc00 2024-04-25T10:25:56.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9520c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b4c40 con 0x7f951c0d2490 2024-04-25T10:25:56.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f951a800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f951c0ce830 0x7f951c0b5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:38484/0 (socket says 172.21.15.87:38484) 2024-04-25T10:25:56.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f951a800640 1 -- 172.21.15.87:0/135093290 learned_addr learned my addr 172.21.15.87:0/135093290 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:56.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9511400640 1 --2- 172.21.15.87:0/135093290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0d2490 0x7f951c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534920317 0 0) 0x7f951c0d06a0 con 0x7f951c0ce830 2024-04-25T10:25:56.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9500003530 con 0x7f951c0ce830 2024-04-25T10:25:56.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305965661 0 0) 0x7f951c0b4960 con 0x7f951c15bc00 2024-04-25T10:25:56.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f951c0d06a0 con 0x7f951c15bc00 2024-04-25T10:25:56.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138492068 0 0) 0x7f9500003530 con 0x7f951c0ce830 2024-04-25T10:25:56.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9511400640 1 -- 172.21.15.87:0/135093290 >> v1:172.21.15.169:6789/0 conn(0x7f951c15bc00 legacy=0x7f951c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9511400640 1 -- 172.21.15.87:0/135093290 >> v1:172.21.15.87:6789/0 conn(0x7f951c0ce830 legacy=0x7f951c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.107+0000 7f9511400640 1 -- 172.21.15.87:0/135093290 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0b4f20 con 0x7f951c0d2490 2024-04-25T10:25:56.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9511400640 1 --2- 172.21.15.87:0/135093290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0d2490 0x7f951c0ce2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f95040097f0 tx=0x7f950401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d556b7f90b5addf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040b0070 con 0x7f951c0d2490 2024-04-25T10:25:56.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95040ab400 con 0x7f951c0d2490 2024-04-25T10:25:56.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9519400640 1 -- 172.21.15.87:0/135093290 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514001a00 con 0x7f951c0d2490 2024-04-25T10:25:56.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9520c00640 1 -- 172.21.15.87:0/135093290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0d2490 msgr2=0x7f951c0ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9520c00640 1 --2- 172.21.15.87:0/135093290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0d2490 0x7f951c0ce2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f95040097f0 tx=0x7f950401c400 comp rx=0 tx=0).stop 2024-04-25T10:25:56.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9520c00640 1 -- 172.21.15.87:0/135093290 shutdown_connections 2024-04-25T10:25:56.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9520c00640 1 --2- 172.21.15.87:0/135093290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0d2490 0x7f951c0ce2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9520c00640 1 -- 172.21.15.87:0/135093290 >> 172.21.15.87:0/135093290 conn(0x7f951c0d1900 msgr2=0x7f951c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9520c00640 1 -- 172.21.15.87:0/135093290 shutdown_connections 2024-04-25T10:25:56.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.111+0000 7f9520c00640 1 -- 172.21.15.87:0/135093290 wait complete. 2024-04-25T10:25:56.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 Processor -- start 2024-04-25T10:25:56.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 -- start start 2024-04-25T10:25:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0ce830 0x7f951c09f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c0d2490 0x7f951c15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15bc00 0x7f951c0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b66f0 con 0x7f951c0d2490 2024-04-25T10:25:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0b6570 con 0x7f951c15bc00 2024-04-25T10:25:56.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c1698b0 con 0x7f951c0ce830 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c0d2490 0x7f951c15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c0d2490 0x7f951c15ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37600/0 (socket says 172.21.15.87:37600) 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 -- 172.21.15.87:0/2386532709 learned_addr learned my addr 172.21.15.87:0/2386532709 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f951a800640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15bc00 0x7f951c0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9519e00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0ce830 0x7f951c09f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 -- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0ce830 msgr2=0x7f951c09f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0ce830 0x7f951c09f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 -- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15bc00 msgr2=0x7f951c0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15bc00 0x7f951c0b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 -- 172.21.15.87:0/2386532709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0c60b0 con 0x7f951c0d2490 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f951a800640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15bc00 0x7f951c0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9511400640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c0d2490 0x7f951c15ed70 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f95040097f0 tx=0x7f95040191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9512a00640 1 -- 172.21.15.87:0/2386532709 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040ba020 con 0x7f951c0d2490 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9512a00640 1 -- 172.21.15.87:0/2386532709 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9504004220 con 0x7f951c0d2490 2024-04-25T10:25:56.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c0c5250 con 0x7f951c0d2490 2024-04-25T10:25:56.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9512a00640 1 -- 172.21.15.87:0/2386532709 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040b0040 con 0x7f951c0d2490 2024-04-25T10:25:56.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.115+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c0c5810 con 0x7f951c0d2490 2024-04-25T10:25:56.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.119+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c04ef30 con 0x7f951c0d2490 2024-04-25T10:25:56.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.119+0000 7f9512a00640 1 -- 172.21.15.87:0/2386532709 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95040bd070 con 0x7f951c0d2490 2024-04-25T10:25:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.119+0000 7f9512a00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.119+0000 7f9512a00640 1 -- 172.21.15.87:0/2386532709 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f950411df70 con 0x7f951c0d2490 2024-04-25T10:25:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.119+0000 7f9519e00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.119+0000 7f9519e00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f950c0011f0 tx=0x7f950c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.123+0000 7f9512a00640 1 -- 172.21.15.87:0/2386532709 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9504015ff0 con 0x7f951c0d2490 2024-04-25T10:25:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.279+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f951c0cfa50 con 0x7f951c0d2490 2024-04-25T10:25:56.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.279+0000 7f9512a00640 1 -- 172.21.15.87:0/2386532709 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f95040b22f0 con 0x7f951c0d2490 2024-04-25T10:25:56.281 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:56.281 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":176,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:44.125291+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:40.866786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:41.487045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 msgr2=0x7f94f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f950c0011f0 tx=0x7f950c009040 comp rx=0 tx=0).stop 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c0d2490 msgr2=0x7f951c15ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c0d2490 0x7f951c15ed70 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f95040097f0 tx=0x7f95040191a0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 shutdown_connections 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f951c15bc00 0x7f951c0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f951c0d2490 0x7f951c15ed70 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 --2- 172.21.15.87:0/2386532709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f951c0ce830 0x7f951c09f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 >> 172.21.15.87:0/2386532709 conn(0x7f951c0d1900 msgr2=0x7f951c06fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 shutdown_connections 2024-04-25T10:25:56.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.283+0000 7f9520c00640 1 -- 172.21.15.87:0/2386532709 wait complete. 2024-04-25T10:25:56.303 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:25:56.303 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:56.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 Processor -- start 2024-04-25T10:25:56.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 -- start start 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7ec1550b0 0x7fa7ec0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa7ec0cfe70 0x7fa7ec15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec15b740 0x7fa7ec06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ec071050 con 0x7fa7ec0cfe70 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ec0712f0 con 0x7fa7ec1550b0 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7f2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0715d0 con 0x7fa7ec15b740 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa7ec0cfe70 0x7fa7ec15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa7ec0cfe70 0x7fa7ec15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37608/0 (socket says 172.21.15.87:37608) 2024-04-25T10:25:56.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 -- 172.21.15.87:0/1402792125 learned_addr learned my addr 172.21.15.87:0/1402792125 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:56.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7ebe00640 1 --2- 172.21.15.87:0/1402792125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec15b740 0x7fa7ec06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eb400640 1 --2- 172.21.15.87:0/1402792125 >> v2:172.21.15.169:3300/0 conn(0x7fa7ec1550b0 0x7fa7ec0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 -- 172.21.15.87:0/1402792125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec15b740 msgr2=0x7fa7ec06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:56.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 --2- 172.21.15.87:0/1402792125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec15b740 0x7fa7ec06d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 -- 172.21.15.87:0/1402792125 >> v2:172.21.15.169:3300/0 conn(0x7fa7ec1550b0 msgr2=0x7fa7ec0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:56.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 --2- 172.21.15.87:0/1402792125 >> v2:172.21.15.169:3300/0 conn(0x7fa7ec1550b0 0x7fa7ec0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eaa00640 1 -- 172.21.15.87:0/1402792125 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0718b0 con 0x7fa7ec0cfe70 2024-04-25T10:25:56.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.439+0000 7fa7eb400640 1 --2- 172.21.15.87:0/1402792125 >> v2:172.21.15.169:3300/0 conn(0x7fa7ec1550b0 0x7fa7ec0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:56.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7eaa00640 1 --2- 172.21.15.87:0/1402792125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec0cfe70 0x7fa7ec15b200 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009870 tx=0x7fa7d801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcb0a776f0d91cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7ea000640 1 -- 172.21.15.87:0/1402792125 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80b0070 con 0x7fa7ec0cfe70 2024-04-25T10:25:56.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7ea000640 1 -- 172.21.15.87:0/1402792125 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80ab600 con 0x7fa7ec0cfe70 2024-04-25T10:25:56.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7ea000640 1 -- 172.21.15.87:0/1402792125 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80ab9d0 con 0x7fa7ec0cfe70 2024-04-25T10:25:56.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 -- 172.21.15.87:0/1402792125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec0cfe70 msgr2=0x7fa7ec15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 --2- 172.21.15.87:0/1402792125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec0cfe70 0x7fa7ec15b200 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009870 tx=0x7fa7d801c260 comp rx=0 tx=0).stop 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 -- 172.21.15.87:0/1402792125 shutdown_connections 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 --2- 172.21.15.87:0/1402792125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec0cfe70 0x7fa7ec15b200 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 --2- 172.21.15.87:0/1402792125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec15b740 0x7fa7ec06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 --2- 172.21.15.87:0/1402792125 >> v2:172.21.15.169:3300/0 conn(0x7fa7ec1550b0 0x7fa7ec0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 -- 172.21.15.87:0/1402792125 >> 172.21.15.87:0/1402792125 conn(0x7fa7ec066b70 msgr2=0x7fa7ec066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 -- 172.21.15.87:0/1402792125 shutdown_connections 2024-04-25T10:25:56.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.443+0000 7fa7f2000640 1 -- 172.21.15.87:0/1402792125 wait complete. 2024-04-25T10:25:56.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 Processor -- start 2024-04-25T10:25:56.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 -- start start 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec0cfe70 0x7fa7ec0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec1550b0 0x7fa7ec14f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec15b740 0x7fa7ec1473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0b3260 con 0x7fa7ec1550b0 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec15bd60 con 0x7fa7ec15b740 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0b33e0 con 0x7fa7ec0cfe70 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec1550b0 0x7fa7ec14f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec1550b0 0x7fa7ec14f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37618/0 (socket says 172.21.15.87:37618) 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 -- 172.21.15.87:0/2205953413 learned_addr learned my addr 172.21.15.87:0/2205953413 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eb400640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec0cfe70 0x7fa7ec0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7ebe00640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec15b740 0x7fa7ec1473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 -- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec0cfe70 msgr2=0x7fa7ec0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec0cfe70 0x7fa7ec0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 -- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec15b740 msgr2=0x7fa7ec1473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec15b740 0x7fa7ec1473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 -- 172.21.15.87:0/2205953413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec14f630 con 0x7fa7ec1550b0 2024-04-25T10:25:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7eaa00640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec1550b0 0x7fa7ec14f0f0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80ae040 tx=0x7fa7d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7e3400640 1 -- 172.21.15.87:0/2205953413 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80b0040 con 0x7fa7ec1550b0 2024-04-25T10:25:56.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7e3400640 1 -- 172.21.15.87:0/2205953413 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80b4420 con 0x7fa7ec1550b0 2024-04-25T10:25:56.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7e3400640 1 -- 172.21.15.87:0/2205953413 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80b4810 con 0x7fa7ec1550b0 2024-04-25T10:25:56.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec0cdc10 con 0x7fa7ec1550b0 2024-04-25T10:25:56.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.447+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec0cdeb0 con 0x7fa7ec1550b0 2024-04-25T10:25:56.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.451+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ec04ef30 con 0x7fa7ec1550b0 2024-04-25T10:25:56.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.451+0000 7fa7e3400640 1 -- 172.21.15.87:0/2205953413 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7d80b49b0 con 0x7fa7ec1550b0 2024-04-25T10:25:56.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.451+0000 7fa7e3400640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc066f90 0x7fa7cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.451+0000 7fa7e3400640 1 -- 172.21.15.87:0/2205953413 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7d811ecb0 con 0x7fa7ec1550b0 2024-04-25T10:25:56.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.451+0000 7fa7eb400640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc066f90 0x7fa7cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.451+0000 7fa7eb400640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc066f90 0x7fa7cc069450 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae400 tx=0x7fa7dc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.455+0000 7fa7e3400640 1 -- 172.21.15.87:0/2205953413 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7ec14f630 con 0x7fa7ec1550b0 2024-04-25T10:25:56.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.639+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7ec0b0040 con 0x7fa7ec1550b0 2024-04-25T10:25:56.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.639+0000 7fa7e3400640 1 -- 172.21.15.87:0/2205953413 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fa7d8016110 con 0x7fa7ec1550b0 2024-04-25T10:25:56.641 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:56.641 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":128,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1787812144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1787812144},{"type":"v1","addr":"172.21.15.169:6834","nonce":1787812144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/554048725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":554048725},{"type":"v1","addr":"172.21.15.169:6835","nonce":554048725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1720179922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1720179922},{"type":"v1","addr":"172.21.15.87:6837","nonce":1720179922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:42.498010+0000","modified":"2024-04-25T10:25:45.516622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10920":{"gid":10920,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/3053712751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3053712751},{"type":"v1","addr":"172.21.15.87:6836","nonce":3053712751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:25:56.641 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 128 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc066f90 msgr2=0x7fa7cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc066f90 0x7fa7cc069450 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae400 tx=0x7fa7dc008040 comp rx=0 tx=0).stop 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec1550b0 msgr2=0x7fa7ec14f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec1550b0 0x7fa7ec14f0f0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80ae040 tx=0x7fa7d8019000 comp rx=0 tx=0).stop 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 shutdown_connections 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa7cc066f90 0x7fa7cc069450 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec15b740 0x7fa7ec1473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7ec1550b0 0x7fa7ec14f0f0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 --2- 172.21.15.87:0/2205953413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec0cfe70 0x7fa7ec0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 >> 172.21.15.87:0/2205953413 conn(0x7fa7ec066b70 msgr2=0x7fa7ec0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 shutdown_connections 2024-04-25T10:25:56.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.643+0000 7fa7f2000640 1 -- 172.21.15.87:0/2205953413 wait complete. 2024-04-25T10:25:56.661 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:56.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb244c00640 1 Processor -- start 2024-04-25T10:25:56.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb244c00640 1 -- start start 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb244c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb24015b9e0 0x7fb2400cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb244c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb2400d1c30 0x7fb2400cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb244c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2400ccfb0 con 0x7fb24015bdc0 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb244c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2400cd290 con 0x7fb24015b9e0 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb244c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2400cd570 con 0x7fb2400d1c30 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb24015b9e0 0x7fb2400cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb24015b9e0 0x7fb2400cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37630/0 (socket says 172.21.15.87:37630) 2024-04-25T10:25:56.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23de00640 1 -- 172.21.15.87:0/1308172010 learned_addr learned my addr 172.21.15.87:0/1308172010 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23de00640 1 -- 172.21.15.87:0/1308172010 >> v2:172.21.15.169:3300/0 conn(0x7fb2400d1c30 msgr2=0x7fb2400cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23e800640 1 --2- 172.21.15.87:0/1308172010 >> v2:172.21.15.169:3300/0 conn(0x7fb2400d1c30 0x7fb2400cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23de00640 1 --2- 172.21.15.87:0/1308172010 >> v2:172.21.15.169:3300/0 conn(0x7fb2400d1c30 0x7fb2400cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23de00640 1 -- 172.21.15.87:0/1308172010 >> v1:172.21.15.169:6789/0 conn(0x7fb24015bdc0 legacy=0x7fb2400d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23de00640 1 -- 172.21.15.87:0/1308172010 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2400cd850 con 0x7fb24015b9e0 2024-04-25T10:25:56.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.799+0000 7fb23e800640 1 --2- 172.21.15.87:0/1308172010 >> v2:172.21.15.169:3300/0 conn(0x7fb2400d1c30 0x7fb2400cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:56.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb23de00640 1 --2- 172.21.15.87:0/1308172010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb24015b9e0 0x7fb2400cc410 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fb240070e60 tx=0x7fb23001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f54c293ce2a258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb23ca00640 1 -- 172.21.15.87:0/1308172010 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2300a5690 con 0x7fb24015b9e0 2024-04-25T10:25:56.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb23ca00640 1 -- 172.21.15.87:0/1308172010 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2300a5830 con 0x7fb24015b9e0 2024-04-25T10:25:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb23ca00640 1 -- 172.21.15.87:0/1308172010 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2300a5c00 con 0x7fb24015b9e0 2024-04-25T10:25:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 -- 172.21.15.87:0/1308172010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb24015b9e0 msgr2=0x7fb2400cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 --2- 172.21.15.87:0/1308172010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb24015b9e0 0x7fb2400cc410 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fb240070e60 tx=0x7fb23001c920 comp rx=0 tx=0).stop 2024-04-25T10:25:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 -- 172.21.15.87:0/1308172010 shutdown_connections 2024-04-25T10:25:56.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 --2- 172.21.15.87:0/1308172010 >> v2:172.21.15.169:3300/0 conn(0x7fb2400d1c30 0x7fb2400cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 --2- 172.21.15.87:0/1308172010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb24015b9e0 0x7fb2400cc410 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 -- 172.21.15.87:0/1308172010 >> 172.21.15.87:0/1308172010 conn(0x7fb24009a9d0 msgr2=0x7fb24009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 -- 172.21.15.87:0/1308172010 shutdown_connections 2024-04-25T10:25:56.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.803+0000 7fb244c00640 1 -- 172.21.15.87:0/1308172010 wait complete. 2024-04-25T10:25:56.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 Processor -- start 2024-04-25T10:25:56.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 -- start start 2024-04-25T10:25:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2400d1c30 0x7fb240155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb24015b9e0 0x7fb2400b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb24015bdc0 0x7fb2400b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400c6630 con 0x7fb2400d1c30 2024-04-25T10:25:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400c64b0 con 0x7fb24015bdc0 2024-04-25T10:25:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400c67b0 con 0x7fb24015b9e0 2024-04-25T10:25:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2400d1c30 0x7fb240155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2400d1c30 0x7fb240155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37634/0 (socket says 172.21.15.87:37634) 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 -- 172.21.15.87:0/1173644935 learned_addr learned my addr 172.21.15.87:0/1173644935 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23d400640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb24015b9e0 0x7fb2400b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23e800640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb24015bdc0 0x7fb2400b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 -- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb24015b9e0 msgr2=0x7fb2400b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb24015b9e0 0x7fb2400b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 -- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb24015bdc0 msgr2=0x7fb2400b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb24015bdc0 0x7fb2400b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 -- 172.21.15.87:0/1173644935 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24015c680 con 0x7fb2400d1c30 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb23de00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2400d1c30 0x7fb240155580 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fb2300af040 tx=0x7fb2300196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb236000640 1 -- 172.21.15.87:0/1173644935 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2300bc020 con 0x7fb2400d1c30 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb236000640 1 -- 172.21.15.87:0/1173644935 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2300b8670 con 0x7fb2400d1c30 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24015c960 con 0x7fb2400d1c30 2024-04-25T10:25:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb236000640 1 -- 172.21.15.87:0/1173644935 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2300b8a60 con 0x7fb2400d1c30 2024-04-25T10:25:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24015cf20 con 0x7fb2400d1c30 2024-04-25T10:25:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.807+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24004ef30 con 0x7fb2400d1c30 2024-04-25T10:25:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.811+0000 7fb236000640 1 -- 172.21.15.87:0/1173644935 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2300bf020 con 0x7fb2400d1c30 2024-04-25T10:25:56.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.811+0000 7fb236000640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb220066f60 0x7fb220069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.811+0000 7fb236000640 1 -- 172.21.15.87:0/1173644935 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb230120200 con 0x7fb2400d1c30 2024-04-25T10:25:56.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.811+0000 7fb23d400640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb220066f60 0x7fb220069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.811+0000 7fb23d400640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb220066f60 0x7fb220069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb240156540 tx=0x7fb22c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.815+0000 7fb236000640 1 -- 172.21.15.87:0/1173644935 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2300c6030 con 0x7fb2400d1c30 2024-04-25T10:25:57.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:56.999+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2400b7720 con 0x7fb2400d1c30 2024-04-25T10:25:57.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb236000640 1 -- 172.21.15.87:0/1173644935 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fb2300165a0 con 0x7fb2400d1c30 2024-04-25T10:25:57.001 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:57.001 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":128,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1787812144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1787812144},{"type":"v1","addr":"172.21.15.169:6834","nonce":1787812144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/554048725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":554048725},{"type":"v1","addr":"172.21.15.169:6835","nonce":554048725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1720179922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1720179922},{"type":"v1","addr":"172.21.15.87:6837","nonce":1720179922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:42.498010+0000","modified":"2024-04-25T10:25:45.516622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10920":{"gid":10920,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/3053712751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3053712751},{"type":"v1","addr":"172.21.15.87:6836","nonce":3053712751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:25:57.001 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 128 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb220066f60 msgr2=0x7fb220069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb220066f60 0x7fb220069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb240156540 tx=0x7fb22c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2400d1c30 msgr2=0x7fb240155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2400d1c30 0x7fb240155580 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fb2300af040 tx=0x7fb2300196c0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 shutdown_connections 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb220066f60 0x7fb220069420 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb24015bdc0 0x7fb2400b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb24015b9e0 0x7fb2400b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 --2- 172.21.15.87:0/1173644935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2400d1c30 0x7fb240155580 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 >> 172.21.15.87:0/1173644935 conn(0x7fb24009a9d0 msgr2=0x7fb24015ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:57.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 shutdown_connections 2024-04-25T10:25:57.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.003+0000 7fb244c00640 1 -- 172.21.15.87:0/1173644935 wait complete. 2024-04-25T10:25:57.021 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 Processor -- start 2024-04-25T10:25:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 -- start start 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f95180cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9518070c20 0x7f951815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f95180ae020 0x7f95180ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951815bb90 con 0x7f9518070c20 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95180d0250 con 0x7f95180ae020 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95180ad840 con 0x7f95180cf930 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9518070c20 0x7f951815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9518070c20 0x7f951815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37644/0 (socket says 172.21.15.87:37644) 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 -- 172.21.15.87:0/3050171104 learned_addr learned my addr 172.21.15.87:0/3050171104 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517e00640 1 --2- 172.21.15.87:0/3050171104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f95180cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f951d000640 1 --2- 172.21.15.87:0/3050171104 >> v2:172.21.15.169:3300/0 conn(0x7f95180ae020 0x7f95180ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 -- 172.21.15.87:0/3050171104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 msgr2=0x7f95180cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:57.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 --2- 172.21.15.87:0/3050171104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f95180cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 -- 172.21.15.87:0/3050171104 >> v2:172.21.15.169:3300/0 conn(0x7f95180ae020 msgr2=0x7f95180ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:57.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 --2- 172.21.15.87:0/3050171104 >> v2:172.21.15.169:3300/0 conn(0x7f95180ae020 0x7f95180ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.159+0000 7f9517400640 1 -- 172.21.15.87:0/3050171104 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951815e9f0 con 0x7f9518070c20 2024-04-25T10:25:57.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f9517400640 1 --2- 172.21.15.87:0/3050171104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9518070c20 0x7f951815e170 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f95180d05c0 tx=0x7f950401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731c2253abb464f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f9516a00640 1 -- 172.21.15.87:0/3050171104 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040aa610 con 0x7f9518070c20 2024-04-25T10:25:57.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f9516a00640 1 -- 172.21.15.87:0/3050171104 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95040af070 con 0x7f9518070c20 2024-04-25T10:25:57.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f9516a00640 1 -- 172.21.15.87:0/3050171104 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040aa9d0 con 0x7f9518070c20 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 -- 172.21.15.87:0/3050171104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9518070c20 msgr2=0x7f951815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 --2- 172.21.15.87:0/3050171104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9518070c20 0x7f951815e170 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f95180d05c0 tx=0x7f950401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 -- 172.21.15.87:0/3050171104 shutdown_connections 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 --2- 172.21.15.87:0/3050171104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9518070c20 0x7f951815e170 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 --2- 172.21.15.87:0/3050171104 >> v2:172.21.15.169:3300/0 conn(0x7f95180ae020 0x7f95180ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 --2- 172.21.15.87:0/3050171104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f95180cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 -- 172.21.15.87:0/3050171104 >> 172.21.15.87:0/3050171104 conn(0x7f9518066b70 msgr2=0x7f9518066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:57.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 -- 172.21.15.87:0/3050171104 shutdown_connections 2024-04-25T10:25:57.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 -- 172.21.15.87:0/3050171104 wait complete. 2024-04-25T10:25:57.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.163+0000 7f951f000640 1 Processor -- start 2024-04-25T10:25:57.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 -- start start 2024-04-25T10:25:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9518070c20 0x7f9518150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95180ae020 0x7f951814e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f9518154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951815e6b0 con 0x7f95180ae020 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951806e130 con 0x7f9518070c20 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9518161790 con 0x7f95180cf930 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95180ae020 0x7f951814e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95180ae020 0x7f951814e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37660/0 (socket says 172.21.15.87:37660) 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 -- 172.21.15.87:0/685690341 learned_addr learned my addr 172.21.15.87:0/685690341 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517e00640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9518070c20 0x7f9518150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951d000640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f9518154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 -- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 msgr2=0x7f9518154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f9518154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 -- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9518070c20 msgr2=0x7f9518150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9518070c20 0x7f9518150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 -- 172.21.15.87:0/685690341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951814f410 con 0x7f95180ae020 2024-04-25T10:25:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f9517400640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95180ae020 0x7f951814e850 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f950401aba0 tx=0x7f950401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f950fe00640 1 -- 172.21.15.87:0/685690341 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040aada0 con 0x7f95180ae020 2024-04-25T10:25:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f950fe00640 1 -- 172.21.15.87:0/685690341 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95040bd3f0 con 0x7f95180ae020 2024-04-25T10:25:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f950fe00640 1 -- 172.21.15.87:0/685690341 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040bd7a0 con 0x7f95180ae020 2024-04-25T10:25:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951814f6f0 con 0x7f95180ae020 2024-04-25T10:25:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951814fcb0 con 0x7f95180ae020 2024-04-25T10:25:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.167+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951804ef30 con 0x7f95180ae020 2024-04-25T10:25:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.171+0000 7f950fe00640 1 -- 172.21.15.87:0/685690341 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95040b33f0 con 0x7f95180ae020 2024-04-25T10:25:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.171+0000 7f950fe00640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.171+0000 7f950fe00640 1 -- 172.21.15.87:0/685690341 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f950411f020 con 0x7f95180ae020 2024-04-25T10:25:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.171+0000 7f9517e00640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.171+0000 7f9517e00640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f951815e290 tx=0x7f9508008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.175+0000 7f950fe00640 1 -- 172.21.15.87:0/685690341 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95040a7070 con 0x7f95180ae020 2024-04-25T10:25:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.331+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9518067160 con 0x7f95180ae020 2024-04-25T10:25:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.331+0000 7f950fe00640 1 -- 172.21.15.87:0/685690341 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f9504016110 con 0x7f95180ae020 2024-04-25T10:25:57.332 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:25:57.332 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":176,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:25:44.125291+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:40.866786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:41.487045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 msgr2=0x7f94f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f951815e290 tx=0x7f9508008040 comp rx=0 tx=0).stop 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95180ae020 msgr2=0x7f951814e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95180ae020 0x7f951814e850 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f950401aba0 tx=0x7f950401ad30 comp rx=0 tx=0).stop 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 shutdown_connections 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95180cf930 0x7f9518154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95180ae020 0x7f951814e850 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 --2- 172.21.15.87:0/685690341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9518070c20 0x7f9518150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 >> 172.21.15.87:0/685690341 conn(0x7f9518066b70 msgr2=0x7f9518068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 shutdown_connections 2024-04-25T10:25:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.335+0000 7f951f000640 1 -- 172.21.15.87:0/685690341 wait complete. 2024-04-25T10:25:57.352 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:25:57.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.487+0000 7fd153400640 1 Processor -- start 2024-04-25T10:25:57.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.487+0000 7fd153400640 1 -- start start 2024-04-25T10:25:57.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd153400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd14c15bc40 0x7fd14c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd153400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0b2770 con 0x7fd14c0cf930 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd153400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c06c670 con 0x7fd14c0cda10 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd153400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c06c950 con 0x7fd14c15bc40 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd150a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd14c15bc40 0x7fd14c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd151400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd14c0cf930 0x7fd14c0b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:52984/0 (socket says 172.21.15.87:52984) 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd150a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd14c15bc40 0x7fd14c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50332/0 (socket says 172.21.15.87:50332) 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd151400640 1 -- 172.21.15.87:0/3919361913 learned_addr learned my addr 172.21.15.87:0/3919361913 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd150a00640 1 -- 172.21.15.87:0/3919361913 >> v1:172.21.15.169:6789/0 conn(0x7fd14c0cda10 legacy=0x7fd14c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd150a00640 1 -- 172.21.15.87:0/3919361913 >> v1:172.21.15.87:6789/0 conn(0x7fd14c0cf930 legacy=0x7fd14c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd150a00640 1 -- 172.21.15.87:0/3919361913 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c06cc30 con 0x7fd14c15bc40 2024-04-25T10:25:57.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.491+0000 7fd150a00640 1 --2- 172.21.15.87:0/3919361913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c15bc40 0x7fd14c06c090 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0097c0 tx=0x7fd13c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682dc8938ae1283e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd14be00640 1 -- 172.21.15.87:0/3919361913 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b0070 con 0x7fd14c15bc40 2024-04-25T10:25:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd14be00640 1 -- 172.21.15.87:0/3919361913 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c0ab600 con 0x7fd14c15bc40 2024-04-25T10:25:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd14be00640 1 -- 172.21.15.87:0/3919361913 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0ab9d0 con 0x7fd14c15bc40 2024-04-25T10:25:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- 172.21.15.87:0/3919361913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c15bc40 msgr2=0x7fd14c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 --2- 172.21.15.87:0/3919361913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c15bc40 0x7fd14c06c090 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0097c0 tx=0x7fd13c01c3d0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- 172.21.15.87:0/3919361913 shutdown_connections 2024-04-25T10:25:57.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 --2- 172.21.15.87:0/3919361913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c15bc40 0x7fd14c06c090 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- 172.21.15.87:0/3919361913 >> 172.21.15.87:0/3919361913 conn(0x7fd14c066b70 msgr2=0x7fd14c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:57.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- 172.21.15.87:0/3919361913 shutdown_connections 2024-04-25T10:25:57.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- 172.21.15.87:0/3919361913 wait complete. 2024-04-25T10:25:57.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 Processor -- start 2024-04-25T10:25:57.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- start start 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c0cda10 0x7fd14c1fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd14c0cf930 0x7fd14c15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c15bc40 0x7fd14c154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c169d70 con 0x7fd14c0cda10 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c169bf0 con 0x7fd14c15bc40 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd153400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c169ef0 con 0x7fd14c0cf930 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd150a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c0cda10 0x7fd14c1fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd150a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c0cda10 0x7fd14c1fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50342/0 (socket says 172.21.15.87:50342) 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd150a00640 1 -- 172.21.15.87:0/3826965188 learned_addr learned my addr 172.21.15.87:0/3826965188 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:57.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.495+0000 7fd143e00640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd14c0cf930 0x7fd14c15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd151400640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c15bc40 0x7fd14c154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd150a00640 1 -- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd14c0cf930 msgr2=0x7fd14c15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd150a00640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd14c0cf930 0x7fd14c15ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd150a00640 1 -- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c15bc40 msgr2=0x7fd14c154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd150a00640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c15bc40 0x7fd14c154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd150a00640 1 -- 172.21.15.87:0/3826965188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c154b90 con 0x7fd14c0cda10 2024-04-25T10:25:57.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd150a00640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c0cda10 0x7fd14c1fa2b0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ae040 tx=0x7fd13c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd149600640 1 -- 172.21.15.87:0/3826965188 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b0050 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd149600640 1 -- 172.21.15.87:0/3826965188 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c0bc070 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd149600640 1 -- 172.21.15.87:0/3826965188 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b78b0 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c154e70 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c155430 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c04ef30 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.499+0000 7fd149600640 1 -- 172.21.15.87:0/3826965188 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd13c0ab4c0 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.503+0000 7fd149600640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd128066f60 0x7fd128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.503+0000 7fd149600640 1 -- 172.21.15.87:0/3826965188 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd13c11f720 con 0x7fd14c0cda10 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.503+0000 7fd143e00640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd128066f60 0x7fd128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.503+0000 7fd143e00640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd128066f60 0x7fd128069420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd14c152aa0 tx=0x7fd134009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.507+0000 7fd149600640 1 -- 172.21.15.87:0/3826965188 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd13c016050 con 0x7fd14c0cda10 2024-04-25T10:25:57.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.691+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd14c06d960 con 0x7fd14c0cda10 2024-04-25T10:25:57.690 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:25:57.691+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:25:57.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.691+0000 7fd149600640 1 -- 172.21.15.87:0/3826965188 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v129) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd13c0f07b0 con 0x7fd14c0cda10 2024-04-25T10:25:57.692 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd128066f60 msgr2=0x7fd128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd128066f60 0x7fd128069420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd14c152aa0 tx=0x7fd134009040 comp rx=0 tx=0).stop 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c0cda10 msgr2=0x7fd14c1fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c0cda10 0x7fd14c1fa2b0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ae040 tx=0x7fd13c01ac10 comp rx=0 tx=0).stop 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 shutdown_connections 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd128066f60 0x7fd128069420 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c15bc40 0x7fd14c154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd14c0cf930 0x7fd14c15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 --2- 172.21.15.87:0/3826965188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14c0cda10 0x7fd14c1fa2b0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 >> 172.21.15.87:0/3826965188 conn(0x7fd14c066b70 msgr2=0x7fd14c0af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 shutdown_connections 2024-04-25T10:25:57.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.695+0000 7fd153400640 1 -- 172.21.15.87:0/3826965188 wait complete. 2024-04-25T10:25:57.724 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:25:57.777 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:25:57.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.867+0000 7f8d67600640 1 Processor -- start 2024-04-25T10:25:57.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.867+0000 7f8d67600640 1 -- start start 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d67600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d60159540 0x7f8d600ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d67600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d600aebd0 0x7f8d60071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d67600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d60071360 con 0x7f8d60072020 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d67600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d60159920 con 0x7f8d60159540 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d67600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6015c2f0 con 0x7f8d600aebd0 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d600aebd0 0x7f8d60071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d64c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d60159540 0x7f8d600ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d65600640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8d60072020 0x7f8d6015f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:53576/0 (socket says 172.21.15.87:53576) 2024-04-25T10:25:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d65600640 1 -- 172.21.15.87:0/1880413400 learned_addr learned my addr 172.21.15.87:0/1880413400 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:25:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d64c00640 1 -- 172.21.15.87:0/1880413400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d600aebd0 msgr2=0x7f8d60071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d64c00640 1 --2- 172.21.15.87:0/1880413400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d600aebd0 0x7f8d60071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d64c00640 1 -- 172.21.15.87:0/1880413400 >> v1:172.21.15.169:6789/0 conn(0x7f8d60072020 legacy=0x7f8d6015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.871+0000 7f8d64c00640 1 -- 172.21.15.87:0/1880413400 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6015c5d0 con 0x7f8d60159540 2024-04-25T10:25:57.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d64c00640 1 --2- 172.21.15.87:0/1880413400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 0x7f8d600ae690 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8d500097c0 tx=0x7f8d5001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=251313f4fabacd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d5fe00640 1 -- 172.21.15.87:0/1880413400 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500b0070 con 0x7f8d60159540 2024-04-25T10:25:57.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d5fe00640 1 -- 172.21.15.87:0/1880413400 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d500ab600 con 0x7f8d60159540 2024-04-25T10:25:57.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d5fe00640 1 -- 172.21.15.87:0/1880413400 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500ab9d0 con 0x7f8d60159540 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- 172.21.15.87:0/1880413400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 msgr2=0x7f8d600ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 --2- 172.21.15.87:0/1880413400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 0x7f8d600ae690 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8d500097c0 tx=0x7f8d5001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- 172.21.15.87:0/1880413400 shutdown_connections 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 --2- 172.21.15.87:0/1880413400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d600aebd0 0x7f8d60071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 --2- 172.21.15.87:0/1880413400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 0x7f8d600ae690 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- 172.21.15.87:0/1880413400 >> 172.21.15.87:0/1880413400 conn(0x7f8d600669a0 msgr2=0x7f8d60066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- 172.21.15.87:0/1880413400 shutdown_connections 2024-04-25T10:25:57.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- 172.21.15.87:0/1880413400 wait complete. 2024-04-25T10:25:57.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 Processor -- start 2024-04-25T10:25:57.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- start start 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d60072020 0x7f8d60151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d600aebd0 0x7f8d60156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 0x7f8d60152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60072430 con 0x7f8d600aebd0 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6006c0d0 con 0x7f8d60159540 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.875+0000 7f8d67600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60168930 con 0x7f8d60072020 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d600aebd0 0x7f8d60156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d600aebd0 0x7f8d60156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50352/0 (socket says 172.21.15.87:50352) 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 -- 172.21.15.87:0/2981530617 learned_addr learned my addr 172.21.15.87:0/2981530617 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d64c00640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d60072020 0x7f8d60151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d65600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 0x7f8d60152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 -- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d60072020 msgr2=0x7f8d60151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d60072020 0x7f8d60151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 -- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 msgr2=0x7f8d60152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 0x7f8d60152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 -- 172.21.15.87:0/2981530617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6014c9c0 con 0x7f8d600aebd0 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d57e00640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d600aebd0 0x7f8d60156f10 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800e930 tx=0x7f8d4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d5d600640 1 -- 172.21.15.87:0/2981530617 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48017070 con 0x7f8d600aebd0 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d5d600640 1 -- 172.21.15.87:0/2981530617 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4800ca00 con 0x7f8d600aebd0 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d601502c0 con 0x7f8d600aebd0 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d5d600640 1 -- 172.21.15.87:0/2981530617 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800cdb0 con 0x7f8d600aebd0 2024-04-25T10:25:57.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d60150880 con 0x7f8d600aebd0 2024-04-25T10:25:57.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6004ef30 con 0x7f8d600aebd0 2024-04-25T10:25:57.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.879+0000 7f8d5d600640 1 -- 172.21.15.87:0/2981530617 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d4801a070 con 0x7f8d600aebd0 2024-04-25T10:25:57.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.883+0000 7f8d5d600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d3c066f60 0x7f8d3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:57.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.883+0000 7f8d5d600640 1 -- 172.21.15.87:0/2981530617 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8d48085010 con 0x7f8d600aebd0 2024-04-25T10:25:57.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.883+0000 7f8d64c00640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d3c066f60 0x7f8d3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:57.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.883+0000 7f8d64c00640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d3c066f60 0x7f8d3c069420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8d500ad000 tx=0x7f8d50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:57.887+0000 7f8d5d600640 1 -- 172.21.15.87:0/2981530617 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d48055a30 con 0x7f8d600aebd0 2024-04-25T10:25:58.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.079+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8d600cf470 con 0x7f8d600aebd0 2024-04-25T10:25:58.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.695+0000 7f8d5d600640 1 -- 172.21.15.87:0/2981530617 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v130) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8d48059ad0 con 0x7f8d600aebd0 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d3c066f60 msgr2=0x7f8d3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d3c066f60 0x7f8d3c069420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8d500ad000 tx=0x7f8d50008040 comp rx=0 tx=0).stop 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d600aebd0 msgr2=0x7f8d60156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d600aebd0 0x7f8d60156f10 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800e930 tx=0x7f8d4800edf0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 shutdown_connections 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8d3c066f60 0x7f8d3c069420 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d60159540 0x7f8d60152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d600aebd0 0x7f8d60156f10 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 --2- 172.21.15.87:0/2981530617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d60072020 0x7f8d60151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 >> 172.21.15.87:0/2981530617 conn(0x7f8d600669a0 msgr2=0x7f8d6015e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:58.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 shutdown_connections 2024-04-25T10:25:58.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.699+0000 7f8d67600640 1 -- 172.21.15.87:0/2981530617 wait complete. 2024-04-25T10:25:58.717 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:25:58.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.855+0000 7ff772000640 1 Processor -- start 2024-04-25T10:25:58.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.855+0000 7ff772000640 1 -- start start 2024-04-25T10:25:58.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff772000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff76c0cf040 0x7ff76c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff772000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff76c15bc00 0x7ff76c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff772000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c0713a0 0x7ff76c071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff772000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c072260 con 0x7ff76c15bc00 2024-04-25T10:25:58.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff772000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c0b3700 con 0x7ff76c0cf040 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff772000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c071d40 con 0x7ff76c0713a0 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff76c15bc00 0x7ff76c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff76c15bc00 0x7ff76c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50356/0 (socket says 172.21.15.87:50356) 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 -- 172.21.15.87:0/3502329329 learned_addr learned my addr 172.21.15.87:0/3502329329 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76be00640 1 --2- 172.21.15.87:0/3502329329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c0713a0 0x7ff76c071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76b400640 1 --2- 172.21.15.87:0/3502329329 >> v2:172.21.15.169:3300/0 conn(0x7ff76c0cf040 0x7ff76c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 -- 172.21.15.87:0/3502329329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c0713a0 msgr2=0x7ff76c071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 --2- 172.21.15.87:0/3502329329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c0713a0 0x7ff76c071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 -- 172.21.15.87:0/3502329329 >> v2:172.21.15.169:3300/0 conn(0x7ff76c0cf040 msgr2=0x7ff76c0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 --2- 172.21.15.87:0/3502329329 >> v2:172.21.15.169:3300/0 conn(0x7ff76c0cf040 0x7ff76c0cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.859+0000 7ff76aa00640 1 -- 172.21.15.87:0/3502329329 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c071f80 con 0x7ff76c15bc00 2024-04-25T10:25:58.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff76aa00640 1 --2- 172.21.15.87:0/3502329329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c15bc00 0x7ff76c0cff20 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0d05d0 tx=0x7ff75801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b5272caf6ab77f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff76a000640 1 -- 172.21.15.87:0/3502329329 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580b1070 con 0x7ff76c15bc00 2024-04-25T10:25:58.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff76a000640 1 -- 172.21.15.87:0/3502329329 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7580ac630 con 0x7ff76c15bc00 2024-04-25T10:25:58.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff76a000640 1 -- 172.21.15.87:0/3502329329 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580a8050 con 0x7ff76c15bc00 2024-04-25T10:25:58.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 -- 172.21.15.87:0/3502329329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c15bc00 msgr2=0x7ff76c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 --2- 172.21.15.87:0/3502329329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c15bc00 0x7ff76c0cff20 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0d05d0 tx=0x7ff75801c520 comp rx=0 tx=0).stop 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 -- 172.21.15.87:0/3502329329 shutdown_connections 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 --2- 172.21.15.87:0/3502329329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c15bc00 0x7ff76c0cff20 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 --2- 172.21.15.87:0/3502329329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c0713a0 0x7ff76c071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 --2- 172.21.15.87:0/3502329329 >> v2:172.21.15.169:3300/0 conn(0x7ff76c0cf040 0x7ff76c0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 -- 172.21.15.87:0/3502329329 >> 172.21.15.87:0/3502329329 conn(0x7ff76c066b70 msgr2=0x7ff76c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 -- 172.21.15.87:0/3502329329 shutdown_connections 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 -- 172.21.15.87:0/3502329329 wait complete. 2024-04-25T10:25:58.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 Processor -- start 2024-04-25T10:25:58.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.863+0000 7ff772000640 1 -- start start 2024-04-25T10:25:58.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0713a0 0x7ff76c14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c0cf040 0x7ff76c14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c15bc00 0x7ff76c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0cf5a0 con 0x7ff76c0cf040 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0cf420 con 0x7ff76c0713a0 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0aec90 con 0x7ff76c15bc00 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c0cf040 0x7ff76c14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c0cf040 0x7ff76c14bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50370/0 (socket says 172.21.15.87:50370) 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 -- 172.21.15.87:0/1156479217 learned_addr learned my addr 172.21.15.87:0/1156479217 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76b400640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0713a0 0x7ff76c14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76be00640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c15bc00 0x7ff76c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 -- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c15bc00 msgr2=0x7ff76c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c15bc00 0x7ff76c1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 -- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0713a0 msgr2=0x7ff76c14b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0713a0 0x7ff76c14b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 -- 172.21.15.87:0/1156479217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c0b3b50 con 0x7ff76c0cf040 2024-04-25T10:25:58.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff76aa00640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c0cf040 0x7ff76c14bd70 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7ff75801abd0 tx=0x7ff7580192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff763400640 1 -- 172.21.15.87:0/1156479217 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580bc070 con 0x7ff76c0cf040 2024-04-25T10:25:58.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff763400640 1 -- 172.21.15.87:0/1156479217 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7580ab5e0 con 0x7ff76c0cf040 2024-04-25T10:25:58.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c1531f0 con 0x7ff76c0cf040 2024-04-25T10:25:58.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff763400640 1 -- 172.21.15.87:0/1156479217 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580b1070 con 0x7ff76c0cf040 2024-04-25T10:25:58.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c154df0 con 0x7ff76c0cf040 2024-04-25T10:25:58.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76c04ef30 con 0x7ff76c0cf040 2024-04-25T10:25:58.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.867+0000 7ff763400640 1 -- 172.21.15.87:0/1156479217 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7580bf070 con 0x7ff76c0cf040 2024-04-25T10:25:58.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.871+0000 7ff763400640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff74c066f60 0x7ff74c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.871+0000 7ff763400640 1 -- 172.21.15.87:0/1156479217 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff75811ea50 con 0x7ff76c0cf040 2024-04-25T10:25:58.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.871+0000 7ff76b400640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff74c066f60 0x7ff74c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.871+0000 7ff76b400640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff74c066f60 0x7ff74c069420 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0b1fa0 tx=0x7ff75c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:58.875+0000 7ff763400640 1 -- 172.21.15.87:0/1156479217 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff758016110 con 0x7ff76c0cf040 2024-04-25T10:25:59.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.031+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff76c152420 con 0x7ff76c0cf040 2024-04-25T10:25:59.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.695+0000 7ff763400640 1 -- 172.21.15.87:0/1156479217 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v178) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff7580ef3a0 con 0x7ff76c0cf040 2024-04-25T10:25:59.697 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:25:59.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff74c066f60 msgr2=0x7ff74c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff74c066f60 0x7ff74c069420 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0b1fa0 tx=0x7ff75c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:59.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c0cf040 msgr2=0x7ff76c14bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c0cf040 0x7ff76c14bd70 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7ff75801abd0 tx=0x7ff7580192c0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 shutdown_connections 2024-04-25T10:25:59.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff74c066f60 0x7ff74c069420 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff76c15bc00 0x7ff76c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff76c0cf040 0x7ff76c14bd70 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 --2- 172.21.15.87:0/1156479217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0713a0 0x7ff76c14b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 >> 172.21.15.87:0/1156479217 conn(0x7ff76c066b70 msgr2=0x7ff76c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 shutdown_connections 2024-04-25T10:25:59.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.699+0000 7ff772000640 1 -- 172.21.15.87:0/1156479217 wait complete. 2024-04-25T10:25:59.721 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.859+0000 7f9f4ae00640 1 Processor -- start 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.859+0000 7f9f4ae00640 1 -- start start 2024-04-25T10:25:59.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f4ae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9f440cf040 0x7f9f440cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f4ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4415bc00 0x7f9f440cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f4ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9f440713a0 0x7f9f440717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f4ae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f44072260 con 0x7f9f440cf040 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f4ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f440b3700 con 0x7f9f440713a0 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f4ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4415aec0 con 0x7f9f4415bc00 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9f440cf040 0x7f9f440cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9f440cf040 0x7f9f440cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50376/0 (socket says 172.21.15.87:50376) 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 -- 172.21.15.87:0/832893279 learned_addr learned my addr 172.21.15.87:0/832893279 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 -- 172.21.15.87:0/832893279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4415bc00 msgr2=0x7f9f440cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 --2- 172.21.15.87:0/832893279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4415bc00 0x7f9f440cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 -- 172.21.15.87:0/832893279 >> v2:172.21.15.169:3300/0 conn(0x7f9f440713a0 msgr2=0x7f9f440717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 --2- 172.21.15.87:0/832893279 >> v2:172.21.15.169:3300/0 conn(0x7f9f440713a0 0x7f9f440717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.863+0000 7f9f43e00640 1 -- 172.21.15.87:0/832893279 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44071f80 con 0x7f9f440cf040 2024-04-25T10:25:59.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f43e00640 1 --2- 172.21.15.87:0/832893279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440cf040 0x7f9f440cf9e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f9f4406e260 tx=0x7f9f3401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5003c8b710fa9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f42a00640 1 -- 172.21.15.87:0/832893279 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340b4070 con 0x7f9f440cf040 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f42a00640 1 -- 172.21.15.87:0/832893279 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340a7a60 con 0x7f9f440cf040 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f42a00640 1 -- 172.21.15.87:0/832893279 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340a7e30 con 0x7f9f440cf040 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- 172.21.15.87:0/832893279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440cf040 msgr2=0x7f9f440cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/832893279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440cf040 0x7f9f440cf9e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f9f4406e260 tx=0x7f9f3401ebd0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- 172.21.15.87:0/832893279 shutdown_connections 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/832893279 >> v2:172.21.15.169:3300/0 conn(0x7f9f440713a0 0x7f9f440717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/832893279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4415bc00 0x7f9f440cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/832893279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440cf040 0x7f9f440cf9e0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- 172.21.15.87:0/832893279 >> 172.21.15.87:0/832893279 conn(0x7f9f44066b70 msgr2=0x7f9f44066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- 172.21.15.87:0/832893279 shutdown_connections 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- 172.21.15.87:0/832893279 wait complete. 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 Processor -- start 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- start start 2024-04-25T10:25:59.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440713a0 0x7f9f4414c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f440cf040 0x7f9f441f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4415bc00 0x7f9f44155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440cf5a0 con 0x7f9f440713a0 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440cf420 con 0x7f9f4415bc00 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f4ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440aec90 con 0x7f9f440cf040 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.867+0000 7f9f43e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440713a0 0x7f9f4414c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440713a0 0x7f9f4414c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50390/0 (socket says 172.21.15.87:50390) 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 -- 172.21.15.87:0/3457456232 learned_addr learned my addr 172.21.15.87:0/3457456232 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43400640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f440cf040 0x7f9f441f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f48e00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4415bc00 0x7f9f44155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 -- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f440cf040 msgr2=0x7f9f441f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f440cf040 0x7f9f441f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 -- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4415bc00 msgr2=0x7f9f44155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4415bc00 0x7f9f44155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 -- 172.21.15.87:0/3457456232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f441f7d30 con 0x7f9f440713a0 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f43e00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440713a0 0x7f9f4414c2e0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400b2c0 tx=0x7f9f3401d410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f3be00640 1 -- 172.21.15.87:0/3457456232 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340bf070 con 0x7f9f440713a0 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f3be00640 1 -- 172.21.15.87:0/3457456232 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340ba660 con 0x7f9f440713a0 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f44155700 con 0x7f9f440713a0 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f3be00640 1 -- 172.21.15.87:0/3457456232 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340b4070 con 0x7f9f440713a0 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441592f0 con 0x7f9f440713a0 2024-04-25T10:25:59.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4404ef30 con 0x7f9f440713a0 2024-04-25T10:25:59.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f3be00640 1 -- 172.21.15.87:0/3457456232 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f340c2070 con 0x7f9f440713a0 2024-04-25T10:25:59.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.871+0000 7f9f3be00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f24066f60 0x7f9f24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.875+0000 7f9f3be00640 1 -- 172.21.15.87:0/3457456232 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f9f3411f6c0 con 0x7f9f440713a0 2024-04-25T10:25:59.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.875+0000 7f9f43400640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f24066f60 0x7f9f24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.875+0000 7f9f43400640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f24066f60 0x7f9f24069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9f4415ae90 tx=0x7f9f30009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:25:59.875+0000 7f9f3be00640 1 -- 172.21.15.87:0/3457456232 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f441f7d30 con 0x7f9f440713a0 2024-04-25T10:26:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.083+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f440b2a10 con 0x7f9f440713a0 2024-04-25T10:26:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f3be00640 1 -- 172.21.15.87:0/3457456232 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v179) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9f340187c0 con 0x7f9f440713a0 2024-04-25T10:26:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:26:00.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f24066f60 msgr2=0x7f9f24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f24066f60 0x7f9f24069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9f4415ae90 tx=0x7f9f30009110 comp rx=0 tx=0).stop 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440713a0 msgr2=0x7f9f4414c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440713a0 0x7f9f4414c2e0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400b2c0 tx=0x7f9f3401d410 comp rx=0 tx=0).stop 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 shutdown_connections 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9f24066f60 0x7f9f24069420 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4415bc00 0x7f9f44155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f440cf040 0x7f9f441f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 --2- 172.21.15.87:0/3457456232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f440713a0 0x7f9f4414c2e0 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 >> 172.21.15.87:0/3457456232 conn(0x7f9f44066b70 msgr2=0x7f9f440ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 shutdown_connections 2024-04-25T10:26:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.707+0000 7f9f4ae00640 1 -- 172.21.15.87:0/3457456232 wait complete. 2024-04-25T10:26:00.735 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:26:00.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.879+0000 7f246c600640 1 Processor -- start 2024-04-25T10:26:00.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.879+0000 7f246c600640 1 -- start start 2024-04-25T10:26:00.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f24640aeda0 0x7f2464071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246c600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2464072020 0x7f2464070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246c600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2464159920 con 0x7f2464159540 2024-04-25T10:26:00.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246c600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246415c2f0 con 0x7f2464072020 2024-04-25T10:26:00.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246415c5d0 con 0x7f24640aeda0 2024-04-25T10:26:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246a600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2464072020 0x7f2464070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f2469c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2464159540 0x7f24640ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53010/0 (socket says 172.21.15.87:53010) 2024-04-25T10:26:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f2469c00640 1 -- 172.21.15.87:0/845067094 learned_addr learned my addr 172.21.15.87:0/845067094 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f2469200640 1 --2- 172.21.15.87:0/845067094 >> v2:172.21.15.169:3300/0 conn(0x7f24640aeda0 0x7f2464071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246a600640 1 -- 172.21.15.87:0/845067094 >> v2:172.21.15.169:3300/0 conn(0x7f24640aeda0 msgr2=0x7f2464071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246a600640 1 --2- 172.21.15.87:0/845067094 >> v2:172.21.15.169:3300/0 conn(0x7f24640aeda0 0x7f2464071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246a600640 1 -- 172.21.15.87:0/845067094 >> v1:172.21.15.87:6789/0 conn(0x7f2464159540 legacy=0x7f24640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246a600640 1 -- 172.21.15.87:0/845067094 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246415c8b0 con 0x7f2464072020 2024-04-25T10:26:00.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f2463e00640 1 -- 172.21.15.87:0/845067094 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746527802 0 0) 0x7f2464159920 con 0x7f2464159540 2024-04-25T10:26:00.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.883+0000 7f246a600640 1 --2- 172.21.15.87:0/845067094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464072020 0x7f2464070350 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f24640cd510 tx=0x7f245c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6323a679e743e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f2463e00640 1 -- 172.21.15.87:0/845067094 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c0b2070 con 0x7f2464072020 2024-04-25T10:26:00.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f2463e00640 1 -- 172.21.15.87:0/845067094 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c0a59a0 con 0x7f2464072020 2024-04-25T10:26:00.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f2463e00640 1 -- 172.21.15.87:0/845067094 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c0a5d70 con 0x7f2464072020 2024-04-25T10:26:00.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- 172.21.15.87:0/845067094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464072020 msgr2=0x7f2464070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 --2- 172.21.15.87:0/845067094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464072020 0x7f2464070350 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f24640cd510 tx=0x7f245c01c900 comp rx=0 tx=0).stop 2024-04-25T10:26:00.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- 172.21.15.87:0/845067094 shutdown_connections 2024-04-25T10:26:00.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 --2- 172.21.15.87:0/845067094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464072020 0x7f2464070350 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 --2- 172.21.15.87:0/845067094 >> v2:172.21.15.169:3300/0 conn(0x7f24640aeda0 0x7f2464071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- 172.21.15.87:0/845067094 >> 172.21.15.87:0/845067094 conn(0x7f24640669a0 msgr2=0x7f2464066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- 172.21.15.87:0/845067094 shutdown_connections 2024-04-25T10:26:00.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- 172.21.15.87:0/845067094 wait complete. 2024-04-25T10:26:00.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 Processor -- start 2024-04-25T10:26:00.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- start start 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2464072020 0x7f246413d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24640aeda0 0x7f246413aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464159540 0x7f246414ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24640710a0 con 0x7f2464159540 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246406c0d0 con 0x7f24640aeda0 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2464071220 con 0x7f2464072020 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464159540 0x7f246414ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464159540 0x7f246414ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50408/0 (socket says 172.21.15.87:50408) 2024-04-25T10:26:00.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.887+0000 7f246a600640 1 -- 172.21.15.87:0/1082784016 learned_addr learned my addr 172.21.15.87:0/1082784016 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2469200640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24640aeda0 0x7f246413aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2469c00640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2464072020 0x7f246413d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246a600640 1 -- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2464072020 msgr2=0x7f246413d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246a600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2464072020 0x7f246413d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246a600640 1 -- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24640aeda0 msgr2=0x7f246413aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246a600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24640aeda0 0x7f246413aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246a600640 1 -- 172.21.15.87:0/1082784016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246413d900 con 0x7f2464159540 2024-04-25T10:26:00.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2469200640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24640aeda0 0x7f246413aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:00.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246a600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464159540 0x7f246414ed80 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f245c0b0040 tx=0x7f245c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2461600640 1 -- 172.21.15.87:0/1082784016 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c0b2050 con 0x7f2464159540 2024-04-25T10:26:00.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2461600640 1 -- 172.21.15.87:0/1082784016 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c0be070 con 0x7f2464159540 2024-04-25T10:26:00.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2461600640 1 -- 172.21.15.87:0/1082784016 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c0ada70 con 0x7f2464159540 2024-04-25T10:26:00.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246413dbe0 con 0x7f2464159540 2024-04-25T10:26:00.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246413e1a0 con 0x7f2464159540 2024-04-25T10:26:00.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246404ef30 con 0x7f2464159540 2024-04-25T10:26:00.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2461600640 1 -- 172.21.15.87:0/1082784016 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f245c0adc10 con 0x7f2464159540 2024-04-25T10:26:00.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.891+0000 7f2461600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2444066f60 0x7f2444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.895+0000 7f2461600640 1 -- 172.21.15.87:0/1082784016 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f245c11fff0 con 0x7f2464159540 2024-04-25T10:26:00.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.895+0000 7f2469c00640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2444066f60 0x7f2444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.895+0000 7f2469c00640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2444066f60 0x7f2444069420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2458007fd0 tx=0x7f2458007d60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:00.895+0000 7f2461600640 1 -- 172.21.15.87:0/1082784016 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f245c016520 con 0x7f2464159540 2024-04-25T10:26:01.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.051+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f24640cfc30 con 0x7f2464159540 2024-04-25T10:26:01.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f2461600640 1 -- 172.21.15.87:0/1082784016 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f245c0f13a0 con 0x7f2464159540 2024-04-25T10:26:01.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2444066f60 msgr2=0x7f2444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2444066f60 0x7f2444069420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2458007fd0 tx=0x7f2458007d60 comp rx=0 tx=0).stop 2024-04-25T10:26:01.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464159540 msgr2=0x7f246414ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464159540 0x7f246414ed80 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f245c0b0040 tx=0x7f245c0196a0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 shutdown_connections 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2444066f60 0x7f2444069420 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2464159540 0x7f246414ed80 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24640aeda0 0x7f246413aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 --2- 172.21.15.87:0/1082784016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2464072020 0x7f246413d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 >> 172.21.15.87:0/1082784016 conn(0x7f24640669a0 msgr2=0x7f246415d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 shutdown_connections 2024-04-25T10:26:01.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.055+0000 7f246c600640 1 -- 172.21.15.87:0/1082784016 wait complete. 2024-04-25T10:26:01.076 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:26:01.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.211+0000 7fbeef600640 1 Processor -- start 2024-04-25T10:26:01.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.211+0000 7fbeef600640 1 -- start start 2024-04-25T10:26:01.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.211+0000 7fbeef600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.211+0000 7fbeef600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbee80cf930 0x7fbee80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.211+0000 7fbeef600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee80b2770 con 0x7fbee815bc40 2024-04-25T10:26:01.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.211+0000 7fbeef600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee806c670 con 0x7fbee80cf930 2024-04-25T10:26:01.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.211+0000 7fbeef600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee806c950 con 0x7fbee80cda10 2024-04-25T10:26:01.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeecc00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbee815bc40 0x7fbee80b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:53586/0 (socket says 172.21.15.87:53586) 2024-04-25T10:26:01.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeecc00640 1 -- 172.21.15.87:0/1844529296 learned_addr learned my addr 172.21.15.87:0/1844529296 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:01.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeed600640 1 --2- 172.21.15.87:0/1844529296 >> v2:172.21.15.169:3300/0 conn(0x7fbee80cf930 0x7fbee80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbee7e00640 1 --2- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeed600640 1 -- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 msgr2=0x7fbee806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeed600640 1 --2- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeed600640 1 -- 172.21.15.87:0/1844529296 >> v1:172.21.15.169:6789/0 conn(0x7fbee815bc40 legacy=0x7fbee80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeed600640 1 -- 172.21.15.87:0/1844529296 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee806cc30 con 0x7fbee80cf930 2024-04-25T10:26:01.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.215+0000 7fbeed600640 1 --2- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 0x7fbee80cfde0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbee00097c0 tx=0x7fbee001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efe51440119862b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbee7400640 1 -- 172.21.15.87:0/1844529296 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00b0070 con 0x7fbee80cf930 2024-04-25T10:26:01.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbee7400640 1 -- 172.21.15.87:0/1844529296 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee00ab600 con 0x7fbee80cf930 2024-04-25T10:26:01.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbee7400640 1 -- 172.21.15.87:0/1844529296 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00ab9d0 con 0x7fbee80cf930 2024-04-25T10:26:01.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 msgr2=0x7fbee80cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 --2- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 0x7fbee80cfde0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbee00097c0 tx=0x7fbee001c410 comp rx=0 tx=0).stop 2024-04-25T10:26:01.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- 172.21.15.87:0/1844529296 shutdown_connections 2024-04-25T10:26:01.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 --2- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 0x7fbee80cfde0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 --2- 172.21.15.87:0/1844529296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- 172.21.15.87:0/1844529296 >> 172.21.15.87:0/1844529296 conn(0x7fbee8066b70 msgr2=0x7fbee8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- 172.21.15.87:0/1844529296 shutdown_connections 2024-04-25T10:26:01.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- 172.21.15.87:0/1844529296 wait complete. 2024-04-25T10:26:01.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 Processor -- start 2024-04-25T10:26:01.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- start start 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee81531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 0x7fbee8152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbee815bc40 0x7fbee815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80b3850 con 0x7fbee815bc40 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80b36d0 con 0x7fbee80cf930 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeef600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80b39d0 con 0x7fbee80cda10 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeed600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbee815bc40 0x7fbee815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeed600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbee815bc40 0x7fbee815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50418/0 (socket says 172.21.15.87:50418) 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.219+0000 7fbeed600640 1 -- 172.21.15.87:0/2629016502 learned_addr learned my addr 172.21.15.87:0/2629016502 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeecc00640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee81531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbee7e00640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 0x7fbee8152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeed600640 1 -- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 msgr2=0x7fbee81531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeed600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee81531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeed600640 1 -- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 msgr2=0x7fbee8152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeed600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 0x7fbee8152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeed600640 1 -- 172.21.15.87:0/2629016502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8165c20 con 0x7fbee815bc40 2024-04-25T10:26:01.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeed600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbee815bc40 0x7fbee815c190 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fbee00097c0 tx=0x7fbee00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbee4c00640 1 -- 172.21.15.87:0/2629016502 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00b8020 con 0x7fbee815bc40 2024-04-25T10:26:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbee4c00640 1 -- 172.21.15.87:0/2629016502 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee00b4610 con 0x7fbee815bc40 2024-04-25T10:26:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee8165f00 con 0x7fbee815bc40 2024-04-25T10:26:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbee4c00640 1 -- 172.21.15.87:0/2629016502 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00b0040 con 0x7fbee815bc40 2024-04-25T10:26:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee81664c0 con 0x7fbee815bc40 2024-04-25T10:26:01.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee804ef30 con 0x7fbee815bc40 2024-04-25T10:26:01.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.223+0000 7fbee4c00640 1 -- 172.21.15.87:0/2629016502 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbee00bb020 con 0x7fbee815bc40 2024-04-25T10:26:01.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.227+0000 7fbee4c00640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbec8066f60 0x7fbec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.227+0000 7fbee4c00640 1 -- 172.21.15.87:0/2629016502 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbee011e990 con 0x7fbee815bc40 2024-04-25T10:26:01.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.227+0000 7fbeecc00640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbec8066f60 0x7fbec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.227+0000 7fbeecc00640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbec8066f60 0x7fbec8069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbee815b660 tx=0x7fbed8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.231+0000 7fbee4c00640 1 -- 172.21.15.87:0/2629016502 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbee00c2030 con 0x7fbee815bc40 2024-04-25T10:26:01.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.439+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7fbee806d960 con 0x7fbee815bc40 2024-04-25T10:26:01.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.707+0000 7fbee4c00640 1 -- 172.21.15.87:0/2629016502 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v660) v1 ==== 147+0+0 (secure 0 0 0) 0x7fbee0016090 con 0x7fbee815bc40 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbec8066f60 msgr2=0x7fbec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbec8066f60 0x7fbec8069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbee815b660 tx=0x7fbed8009040 comp rx=0 tx=0).stop 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbee815bc40 msgr2=0x7fbee815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbee815bc40 0x7fbee815c190 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fbee00097c0 tx=0x7fbee00191b0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 shutdown_connections 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbec8066f60 0x7fbec8069420 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbee815bc40 0x7fbee815c190 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee80cf930 0x7fbee8152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 --2- 172.21.15.87:0/2629016502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee80cda10 0x7fbee81531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 >> 172.21.15.87:0/2629016502 conn(0x7fbee8066b70 msgr2=0x7fbee8159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 shutdown_connections 2024-04-25T10:26:01.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.711+0000 7fbeef600640 1 -- 172.21.15.87:0/2629016502 wait complete. 2024-04-25T10:26:01.730 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:26:01.731 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:26:01.731 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:01.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7f600640 1 Processor -- start 2024-04-25T10:26:01.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7f600640 1 -- start start 2024-04-25T10:26:01.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7f600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e780d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7f600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e7815e050 0x7f2e780b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7f600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e780b1870 con 0x7f2e780b13e0 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7f600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e780b1af0 con 0x7f2e7815e050 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7f600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e780b1dd0 con 0x7f2e780d1820 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e6fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e7815e050 0x7f2e780b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e6fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2e7815e050 0x7f2e780b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50426/0 (socket says 172.21.15.87:50426) 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e6fe00640 1 -- 172.21.15.87:0/3461853381 learned_addr learned my addr 172.21.15.87:0/3461853381 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e7cc00640 1 --2- 172.21.15.87:0/3461853381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e780d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e6fe00640 1 -- 172.21.15.87:0/3461853381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 msgr2=0x7f2e780d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/3461853381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e780d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e6fe00640 1 -- 172.21.15.87:0/3461853381 >> v1:172.21.15.169:6789/0 conn(0x7f2e780b13e0 legacy=0x7f2e780ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.867+0000 7f2e6fe00640 1 -- 172.21.15.87:0/3461853381 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e780b20b0 con 0x7f2e7815e050 2024-04-25T10:26:01.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/3461853381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780b0ea0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009870 tx=0x7f2e6001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f50ba09dc8c5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e77e00640 1 -- 172.21.15.87:0/3461853381 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e600b0070 con 0x7f2e7815e050 2024-04-25T10:26:01.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e77e00640 1 -- 172.21.15.87:0/3461853381 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e600ab600 con 0x7f2e7815e050 2024-04-25T10:26:01.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e77e00640 1 -- 172.21.15.87:0/3461853381 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e600ab9d0 con 0x7f2e7815e050 2024-04-25T10:26:01.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 -- 172.21.15.87:0/3461853381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 msgr2=0x7f2e780b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 --2- 172.21.15.87:0/3461853381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780b0ea0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009870 tx=0x7f2e6001c340 comp rx=0 tx=0).stop 2024-04-25T10:26:01.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 -- 172.21.15.87:0/3461853381 shutdown_connections 2024-04-25T10:26:01.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 --2- 172.21.15.87:0/3461853381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780b0ea0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 --2- 172.21.15.87:0/3461853381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e780d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 -- 172.21.15.87:0/3461853381 >> 172.21.15.87:0/3461853381 conn(0x7f2e7809a9d0 msgr2=0x7f2e7809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 -- 172.21.15.87:0/3461853381 shutdown_connections 2024-04-25T10:26:01.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.871+0000 7f2e7f600640 1 -- 172.21.15.87:0/3461853381 wait complete. 2024-04-25T10:26:01.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 Processor -- start 2024-04-25T10:26:01.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 -- start start 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e780b13e0 0x7f2e78150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e78154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7816ab20 con 0x7f2e7815e050 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7816a9a0 con 0x7f2e780b13e0 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7816aca0 con 0x7f2e780d1820 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e780b13e0 0x7f2e78150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50440/0 (socket says 172.21.15.87:50440) 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 -- 172.21.15.87:0/122788629 learned_addr learned my addr 172.21.15.87:0/122788629 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e6fe00640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e78154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 -- 172.21.15.87:0/122788629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 msgr2=0x7f2e78154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e78154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 -- 172.21.15.87:0/122788629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e780b13e0 msgr2=0x7f2e78150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e780b13e0 0x7f2e78150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 -- 172.21.15.87:0/122788629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e781557a0 con 0x7f2e7815e050 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7d600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780ae710 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000c7b0 tx=0x7f2e7000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e75600640 1 -- 172.21.15.87:0/122788629 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70004b70 con 0x7f2e7815e050 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e75600640 1 -- 172.21.15.87:0/122788629 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e70004d10 con 0x7f2e7815e050 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e78155a80 con 0x7f2e7815e050 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e75600640 1 -- 172.21.15.87:0/122788629 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7000aa90 con 0x7f2e7815e050 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e78156040 con 0x7f2e7815e050 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.875+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7804ef30 con 0x7f2e7815e050 2024-04-25T10:26:01.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.879+0000 7f2e75600640 1 -- 172.21.15.87:0/122788629 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e7000ac30 con 0x7f2e7815e050 2024-04-25T10:26:01.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.879+0000 7f2e75600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e54066f60 0x7f2e54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.879+0000 7f2e75600640 1 -- 172.21.15.87:0/122788629 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2e70085150 con 0x7f2e7815e050 2024-04-25T10:26:01.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.879+0000 7f2e7cc00640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e54066f60 0x7f2e54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.879+0000 7f2e7cc00640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e54066f60 0x7f2e54069420 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f2e78156c00 tx=0x7f2e68008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:01.883+0000 7f2e75600640 1 -- 172.21.15.87:0/122788629 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e70055df0 con 0x7f2e7815e050 2024-04-25T10:26:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.035+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e780740e0 con 0x7f2e7815e050 2024-04-25T10:26:02.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e75600640 1 -- 172.21.15.87:0/122788629 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+13018 (secure 0 0 0) 0x7f2e70059e90 con 0x7f2e7815e050 2024-04-25T10:26:02.037 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:02.037 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":179,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:26:00.701656+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6836/3053712751":"2024-04-26T10:25:57.694816+0000","172.21.15.87:6834/3053712751":"2024-04-26T10:25:57.694816+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e54066f60 msgr2=0x7f2e54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e54066f60 0x7f2e54069420 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f2e78156c00 tx=0x7f2e68008040 comp rx=0 tx=0).stop 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 msgr2=0x7f2e780ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780ae710 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000c7b0 tx=0x7f2e7000cc70 comp rx=0 tx=0).stop 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 shutdown_connections 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2e54066f60 0x7f2e54069420 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e7815e050 0x7f2e780ae710 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e780d1820 0x7f2e78154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 --2- 172.21.15.87:0/122788629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e780b13e0 0x7f2e78150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 >> 172.21.15.87:0/122788629 conn(0x7f2e7809a9d0 msgr2=0x7f2e7815b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 shutdown_connections 2024-04-25T10:26:02.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.039+0000 7f2e7f600640 1 -- 172.21.15.87:0/122788629 wait complete. 2024-04-25T10:26:02.059 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:26:02.059 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:26:02.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8cad000640 1 Processor -- start 2024-04-25T10:26:02.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8cad000640 1 -- start start 2024-04-25T10:26:02.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8cad000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 0x7f8ca80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8cad000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ca80aebd0 0x7f8ca8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8cad000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca8071360 con 0x7f8ca8072020 2024-04-25T10:26:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8cad000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca80af030 con 0x7f8ca80aebd0 2024-04-25T10:26:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8cad000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8159920 con 0x7f8ca8159540 2024-04-25T10:26:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca6800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8ca8072020 0x7f8ca815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53014/0 (socket says 172.21.15.87:53014) 2024-04-25T10:26:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca6800640 1 -- 172.21.15.87:0/744895759 learned_addr learned my addr 172.21.15.87:0/744895759 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8c9d400640 1 --2- 172.21.15.87:0/744895759 >> v2:172.21.15.169:3300/0 conn(0x7f8ca80aebd0 0x7f8ca8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5e00640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 0x7f8ca80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247603539 0 0) 0x7f8ca8071360 con 0x7f8ca8072020 2024-04-25T10:26:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c8c003530 con 0x7f8ca8072020 2024-04-25T10:26:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3229825574 0 0) 0x7f8c8c003530 con 0x7f8ca8072020 2024-04-25T10:26:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 msgr2=0x7f8ca80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 0x7f8ca80ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 >> v2:172.21.15.169:3300/0 conn(0x7f8ca80aebd0 msgr2=0x7f8ca8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 --2- 172.21.15.87:0/744895759 >> v2:172.21.15.169:3300/0 conn(0x7f8ca80aebd0 0x7f8ca8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca815c2f0 con 0x7f8ca8072020 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.199+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f8ca0001a00 con 0x7f8ca8072020 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 >> v1:172.21.15.87:6789/0 conn(0x7f8ca8072020 legacy=0x7f8ca815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c8c0039c0 0x7f8c8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 0x7f8c8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8c8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8159920 con 0x7f8c8c004d50 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c00b9d0 con 0x7f8c8c0039c0 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c00bb50 con 0x7f8ca8159540 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca6800640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8c8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5e00640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c8c0039c0 0x7f8c8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8c9d400640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 0x7f8c8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca6800640 1 -- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 msgr2=0x7f8c8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca6800640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca8159540 0x7f8c8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca6800640 1 -- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c8c0039c0 msgr2=0x7f8c8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca6800640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c8c0039c0 0x7f8c8c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca6800640 1 -- 172.21.15.87:0/744895759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c003530 con 0x7f8c8c004d50 2024-04-25T10:26:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5e00640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c8c0039c0 0x7f8c8c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca6800640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8c8c0051d0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000a870 tx=0x7f8ca001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48812e15102a6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000a4c0 con 0x7f8c8c004d50 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca00072d0 con 0x7f8c8c004d50 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.203+0000 7f8ca5400640 1 -- 172.21.15.87:0/744895759 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00076c0 con 0x7f8c8c004d50 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- 172.21.15.87:0/744895759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 msgr2=0x7f8c8c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 --2- 172.21.15.87:0/744895759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8c8c0051d0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000a870 tx=0x7f8ca001d3a0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca6800640 1 -- 172.21.15.87:0/744895759 reap_dead start 2024-04-25T10:26:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- 172.21.15.87:0/744895759 shutdown_connections 2024-04-25T10:26:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- 172.21.15.87:0/744895759 >> 172.21.15.87:0/744895759 conn(0x7f8ca80669a0 msgr2=0x7f8ca8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- 172.21.15.87:0/744895759 shutdown_connections 2024-04-25T10:26:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- 172.21.15.87:0/744895759 wait complete. 2024-04-25T10:26:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 Processor -- start 2024-04-25T10:26:02.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- start start 2024-04-25T10:26:02.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8ca80b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca81f7170 0x7f8ca80b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ca81f75f0 0x7f8ca80b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca815cb90 con 0x7f8c8c004d50 2024-04-25T10:26:02.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca815c8b0 con 0x7f8ca81f75f0 2024-04-25T10:26:02.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8cad000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca815bfa0 con 0x7f8ca81f7170 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8ca80b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8ca80b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50462/0 (socket says 172.21.15.87:50462) 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 -- 172.21.15.87:0/1008669169 learned_addr learned my addr 172.21.15.87:0/1008669169 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca6800640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ca81f75f0 0x7f8ca80b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8c9d400640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca81f7170 0x7f8ca80b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 -- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca81f7170 msgr2=0x7f8ca80b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca81f7170 0x7f8ca80b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 -- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ca81f75f0 msgr2=0x7f8ca80b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ca81f75f0 0x7f8ca80b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca5e00640 1 -- 172.21.15.87:0/1008669169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca81f7bc0 con 0x7f8c8c004d50 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.207+0000 7f8ca6800640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ca81f75f0 0x7f8ca80b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8ca5e00640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8ca80b1880 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009160 tx=0x7f8c98011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8c9ea00640 1 -- 172.21.15.87:0/1008669169 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98004220 con 0x7f8c8c004d50 2024-04-25T10:26:02.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8c9ea00640 1 -- 172.21.15.87:0/1008669169 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c980043c0 con 0x7f8c8c004d50 2024-04-25T10:26:02.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8c9ea00640 1 -- 172.21.15.87:0/1008669169 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98015890 con 0x7f8c8c004d50 2024-04-25T10:26:02.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca81f7ea0 con 0x7f8c8c004d50 2024-04-25T10:26:02.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca814b900 con 0x7f8c8c004d50 2024-04-25T10:26:02.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca804ef30 con 0x7f8c8c004d50 2024-04-25T10:26:02.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8c9ea00640 1 -- 172.21.15.87:0/1008669169 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c98015a30 con 0x7f8c8c004d50 2024-04-25T10:26:02.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8c9ea00640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c84066f60 0x7f8c84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8c9ea00640 1 -- 172.21.15.87:0/1008669169 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c980853b0 con 0x7f8c8c004d50 2024-04-25T10:26:02.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.211+0000 7f8c9d400640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c84066f60 0x7f8c84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.215+0000 7f8c9d400640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c84066f60 0x7f8c84069420 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80cbbc0 tx=0x7f8c90008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.215+0000 7f8c9ea00640 1 -- 172.21.15.87:0/1008669169 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c98019030 con 0x7f8c8c004d50 2024-04-25T10:26:02.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.371+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ca806d7f0 con 0x7f8c8c004d50 2024-04-25T10:26:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.371+0000 7f8c9ea00640 1 -- 172.21.15.87:0/1008669169 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c98056620 con 0x7f8c8c004d50 2024-04-25T10:26:02.373 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c84066f60 msgr2=0x7f8c84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c84066f60 0x7f8c84069420 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80cbbc0 tx=0x7f8c90008040 comp rx=0 tx=0).stop 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 msgr2=0x7f8ca80b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8ca80b1880 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009160 tx=0x7f8c98011a60 comp rx=0 tx=0).stop 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 shutdown_connections 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8c84066f60 0x7f8c84069420 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ca81f75f0 0x7f8ca80b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ca81f7170 0x7f8ca80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 --2- 172.21.15.87:0/1008669169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c8c004d50 0x7f8ca80b1880 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 >> 172.21.15.87:0/1008669169 conn(0x7f8ca80669a0 msgr2=0x7f8ca815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 shutdown_connections 2024-04-25T10:26:02.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.375+0000 7f8cad000640 1 -- 172.21.15.87:0/1008669169 wait complete. 2024-04-25T10:26:02.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:26:02.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd64600640 1 Processor -- start 2024-04-25T10:26:02.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd64600640 1 -- start start 2024-04-25T10:26:02.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd64600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdd5c0d0040 0x7fdd5c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd64600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdd5c15b740 0x7fdd5c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd64600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c071050 con 0x7fdd5c1550b0 2024-04-25T10:26:02.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd64600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c15d040 con 0x7fdd5c0d0040 2024-04-25T10:26:02.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd64600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c0712f0 con 0x7fdd5c15b740 2024-04-25T10:26:02.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd61200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdd5c0d0040 0x7fdd5c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd61200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdd5c0d0040 0x7fdd5c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50476/0 (socket says 172.21.15.87:50476) 2024-04-25T10:26:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd61200640 1 -- 172.21.15.87:0/2197983829 learned_addr learned my addr 172.21.15.87:0/2197983829 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd62600640 1 --2- 172.21.15.87:0/2197983829 >> v2:172.21.15.169:3300/0 conn(0x7fdd5c15b740 0x7fdd5c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd61200640 1 -- 172.21.15.87:0/2197983829 >> v2:172.21.15.169:3300/0 conn(0x7fdd5c15b740 msgr2=0x7fdd5c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd61200640 1 --2- 172.21.15.87:0/2197983829 >> v2:172.21.15.169:3300/0 conn(0x7fdd5c15b740 0x7fdd5c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd61200640 1 -- 172.21.15.87:0/2197983829 >> v1:172.21.15.169:6789/0 conn(0x7fdd5c1550b0 legacy=0x7fdd5c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.531+0000 7fdd61200640 1 -- 172.21.15.87:0/2197983829 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0715d0 con 0x7fdd5c0d0040 2024-04-25T10:26:02.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd61200640 1 --2- 172.21.15.87:0/2197983829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0d0040 0x7fdd5c15b200 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0d04d0 tx=0x7fdd4801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24add9219092a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd5be00640 1 -- 172.21.15.87:0/2197983829 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480aa650 con 0x7fdd5c0d0040 2024-04-25T10:26:02.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd5be00640 1 -- 172.21.15.87:0/2197983829 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd480af070 con 0x7fdd5c0d0040 2024-04-25T10:26:02.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd5be00640 1 -- 172.21.15.87:0/2197983829 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480aaa10 con 0x7fdd5c0d0040 2024-04-25T10:26:02.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 -- 172.21.15.87:0/2197983829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0d0040 msgr2=0x7fdd5c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 --2- 172.21.15.87:0/2197983829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0d0040 0x7fdd5c15b200 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0d04d0 tx=0x7fdd4801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 -- 172.21.15.87:0/2197983829 shutdown_connections 2024-04-25T10:26:02.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 --2- 172.21.15.87:0/2197983829 >> v2:172.21.15.169:3300/0 conn(0x7fdd5c15b740 0x7fdd5c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 --2- 172.21.15.87:0/2197983829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c0d0040 0x7fdd5c15b200 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 -- 172.21.15.87:0/2197983829 >> 172.21.15.87:0/2197983829 conn(0x7fdd5c066b70 msgr2=0x7fdd5c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:02.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 -- 172.21.15.87:0/2197983829 shutdown_connections 2024-04-25T10:26:02.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.535+0000 7fdd64600640 1 -- 172.21.15.87:0/2197983829 wait complete. 2024-04-25T10:26:02.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 Processor -- start 2024-04-25T10:26:02.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 -- start start 2024-04-25T10:26:02.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c0d0040 0x7fdd5c13da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c1550b0 0x7fdd5c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c15b740 0x7fdd5c1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c15bcd0 con 0x7fdd5c15b740 2024-04-25T10:26:02.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c15bb50 con 0x7fdd5c0d0040 2024-04-25T10:26:02.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c15c030 con 0x7fdd5c1550b0 2024-04-25T10:26:02.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c15b740 0x7fdd5c1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c15b740 0x7fdd5c1510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50484/0 (socket says 172.21.15.87:50484) 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 -- 172.21.15.87:0/1530175101 learned_addr learned my addr 172.21.15.87:0/1530175101 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd61c00640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c0d0040 0x7fdd5c13da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd61200640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c1550b0 0x7fdd5c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 -- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c1550b0 msgr2=0x7fdd5c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c1550b0 0x7fdd5c154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 -- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c0d0040 msgr2=0x7fdd5c13da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c0d0040 0x7fdd5c13da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 -- 172.21.15.87:0/1530175101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c157790 con 0x7fdd5c15b740 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd62600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c15b740 0x7fdd5c1510d0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007e60 tx=0x7fdd5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd59600640 1 -- 172.21.15.87:0/1530175101 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54014420 con 0x7fdd5c15b740 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd59600640 1 -- 172.21.15.87:0/1530175101 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd54019070 con 0x7fdd5c15b740 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c157a70 con 0x7fdd5c15b740 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd59600640 1 -- 172.21.15.87:0/1530175101 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54014820 con 0x7fdd5c15b740 2024-04-25T10:26:02.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c158030 con 0x7fdd5c15b740 2024-04-25T10:26:02.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.539+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c04ef30 con 0x7fdd5c15b740 2024-04-25T10:26:02.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.543+0000 7fdd59600640 1 -- 172.21.15.87:0/1530175101 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd540149c0 con 0x7fdd5c15b740 2024-04-25T10:26:02.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.543+0000 7fdd59600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd3c066f60 0x7fdd3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.543+0000 7fdd59600640 1 -- 172.21.15.87:0/1530175101 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdd54085490 con 0x7fdd5c15b740 2024-04-25T10:26:02.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.543+0000 7fdd61c00640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd3c066f60 0x7fdd3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.543+0000 7fdd61c00640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd3c066f60 0x7fdd3c069420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdd50001810 tx=0x7fdd50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.547+0000 7fdd59600640 1 -- 172.21.15.87:0/1530175101 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd54056140 con 0x7fdd5c15b740 2024-04-25T10:26:02.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:02.755+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7fdd5c13dfc0 con 0x7fdd5c15b740 2024-04-25T10:26:03.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.715+0000 7fdd59600640 1 -- 172.21.15.87:0/1530175101 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v662) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdd5405a1e0 con 0x7fdd5c15b740 2024-04-25T10:26:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd3c066f60 msgr2=0x7fdd3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd3c066f60 0x7fdd3c069420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdd50001810 tx=0x7fdd50009040 comp rx=0 tx=0).stop 2024-04-25T10:26:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c15b740 msgr2=0x7fdd5c1510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c15b740 0x7fdd5c1510d0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007e60 tx=0x7fdd5400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 shutdown_connections 2024-04-25T10:26:03.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdd3c066f60 0x7fdd3c069420 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5c15b740 0x7fdd5c1510d0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd5c1550b0 0x7fdd5c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 --2- 172.21.15.87:0/1530175101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd5c0d0040 0x7fdd5c13da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 >> 172.21.15.87:0/1530175101 conn(0x7fdd5c066b70 msgr2=0x7fdd5c15c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 shutdown_connections 2024-04-25T10:26:03.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.719+0000 7fdd64600640 1 -- 172.21.15.87:0/1530175101 wait complete. 2024-04-25T10:26:03.742 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:26:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.883+0000 7f5bcfe00640 1 Processor -- start 2024-04-25T10:26:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.883+0000 7f5bcfe00640 1 -- start start 2024-04-25T10:26:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.883+0000 7f5bcfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5bc815e130 0x7f5bc80cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.883+0000 7f5bcfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5bc8071bc0 0x7f5bc8072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.883+0000 7f5bcfe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc80ad840 con 0x7f5bc80b3480 2024-04-25T10:26:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.883+0000 7f5bcfe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc8072570 con 0x7f5bc8071bc0 2024-04-25T10:26:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.883+0000 7f5bcfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80d0090 con 0x7f5bc815e130 2024-04-25T10:26:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcde00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5bc8071bc0 0x7f5bc8072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcd400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5bc80b3480 0x7f5bc8070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53018/0 (socket says 172.21.15.87:53018) 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcd400640 1 -- 172.21.15.87:0/3491012453 learned_addr learned my addr 172.21.15.87:0/3491012453 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcca00640 1 --2- 172.21.15.87:0/3491012453 >> v2:172.21.15.169:3300/0 conn(0x7f5bc815e130 0x7f5bc80cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bc7e00640 1 -- 172.21.15.87:0/3491012453 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458469652 0 0) 0x7f5bc80ad840 con 0x7f5bc80b3480 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bc7e00640 1 -- 172.21.15.87:0/3491012453 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb0003530 con 0x7f5bc80b3480 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcde00640 1 -- 172.21.15.87:0/3491012453 >> v2:172.21.15.169:3300/0 conn(0x7f5bc815e130 msgr2=0x7f5bc80cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcde00640 1 --2- 172.21.15.87:0/3491012453 >> v2:172.21.15.169:3300/0 conn(0x7f5bc815e130 0x7f5bc80cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcde00640 1 -- 172.21.15.87:0/3491012453 >> v1:172.21.15.87:6789/0 conn(0x7f5bc80b3480 legacy=0x7f5bc8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcde00640 1 -- 172.21.15.87:0/3491012453 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc815e710 con 0x7f5bc8071bc0 2024-04-25T10:26:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bcde00640 1 --2- 172.21.15.87:0/3491012453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc8071bc0 0x7f5bc8072030 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80b4c40 tx=0x7f5bc001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477f98f47628a41d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bc7e00640 1 -- 172.21.15.87:0/3491012453 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00b2070 con 0x7f5bc8071bc0 2024-04-25T10:26:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bc7e00640 1 -- 172.21.15.87:0/3491012453 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00a57f0 con 0x7f5bc8071bc0 2024-04-25T10:26:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.887+0000 7f5bc7e00640 1 -- 172.21.15.87:0/3491012453 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00a5bc0 con 0x7f5bc8071bc0 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- 172.21.15.87:0/3491012453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc8071bc0 msgr2=0x7f5bc8072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/3491012453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc8071bc0 0x7f5bc8072030 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80b4c40 tx=0x7f5bc001c930 comp rx=0 tx=0).stop 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- 172.21.15.87:0/3491012453 shutdown_connections 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/3491012453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc8071bc0 0x7f5bc8072030 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/3491012453 >> v2:172.21.15.169:3300/0 conn(0x7f5bc815e130 0x7f5bc80cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- 172.21.15.87:0/3491012453 >> 172.21.15.87:0/3491012453 conn(0x7f5bc80669a0 msgr2=0x7f5bc8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- 172.21.15.87:0/3491012453 shutdown_connections 2024-04-25T10:26:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- 172.21.15.87:0/3491012453 wait complete. 2024-04-25T10:26:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 Processor -- start 2024-04-25T10:26:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- start start 2024-04-25T10:26:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bc8071bc0 0x7f5bc814ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc80b3480 0x7f5bc815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc815e130 0x7f5bc8156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80ae530 con 0x7f5bc815e130 2024-04-25T10:26:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815bb90 con 0x7f5bc80b3480 2024-04-25T10:26:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80ae6b0 con 0x7f5bc8071bc0 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc815e130 0x7f5bc8156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc815e130 0x7f5bc8156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50512/0 (socket says 172.21.15.87:50512) 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 -- 172.21.15.87:0/100289724 learned_addr learned my addr 172.21.15.87:0/100289724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcca00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc80b3480 0x7f5bc815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcd400640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bc8071bc0 0x7f5bc814ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 -- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bc8071bc0 msgr2=0x7f5bc814ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bc8071bc0 0x7f5bc814ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 -- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc80b3480 msgr2=0x7f5bc815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc80b3480 0x7f5bc815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcde00640 1 -- 172.21.15.87:0/100289724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc815b1f0 con 0x7f5bc815e130 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcd400640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bc8071bc0 0x7f5bc814ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.891+0000 7f5bcca00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc80b3480 0x7f5bc815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:03.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bcde00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc815e130 0x7f5bc8156f50 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0019540 tx=0x7f5bc00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:03.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bc5600640 1 -- 172.21.15.87:0/100289724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00b2050 con 0x7f5bc815e130 2024-04-25T10:26:03.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bc5600640 1 -- 172.21.15.87:0/100289724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00be070 con 0x7f5bc815e130 2024-04-25T10:26:03.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc80b11e0 con 0x7f5bc815e130 2024-04-25T10:26:03.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bc5600640 1 -- 172.21.15.87:0/100289724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00ad870 con 0x7f5bc815e130 2024-04-25T10:26:03.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc8159430 con 0x7f5bc815e130 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc8167f60 con 0x7f5bc815e130 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bc5600640 1 -- 172.21.15.87:0/100289724 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bc00ada10 con 0x7f5bc815e130 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bc5600640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ba8066f60 0x7f5ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bc5600640 1 -- 172.21.15.87:0/100289724 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5bc011fdd0 con 0x7f5bc815e130 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bcd400640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ba8066f60 0x7f5ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.895+0000 7f5bcd400640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ba8066f60 0x7f5ba8069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc001210 tx=0x7f5bbc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:03.899+0000 7f5bc5600640 1 -- 172.21.15.87:0/100289724 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bc8167f60 con 0x7f5bc815e130 2024-04-25T10:26:04.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.051+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5bc804ef30 con 0x7f5bc815e130 2024-04-25T10:26:04.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.055+0000 7f5bc5600640 1 -- 172.21.15.87:0/100289724 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5bc01210c0 con 0x7f5bc815e130 2024-04-25T10:26:04.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ba8066f60 msgr2=0x7f5ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ba8066f60 0x7f5ba8069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc001210 tx=0x7f5bbc009040 comp rx=0 tx=0).stop 2024-04-25T10:26:04.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc815e130 msgr2=0x7f5bc8156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc815e130 0x7f5bc8156f50 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0019540 tx=0x7f5bc00196d0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 shutdown_connections 2024-04-25T10:26:04.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5ba8066f60 0x7f5ba8069420 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc815e130 0x7f5bc8156f50 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc80b3480 0x7f5bc815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 --2- 172.21.15.87:0/100289724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bc8071bc0 0x7f5bc814ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 >> 172.21.15.87:0/100289724 conn(0x7f5bc80669a0 msgr2=0x7f5bc8070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 shutdown_connections 2024-04-25T10:26:04.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.059+0000 7f5bcfe00640 1 -- 172.21.15.87:0/100289724 wait complete. 2024-04-25T10:26:04.078 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:26:04.078 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:26:04.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:26:04.178 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:26:04.178 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:26:04.178 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:04.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:26:04.187 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:26:04.187 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:04.197 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:26:04.197 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:26:04.197 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:04.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:26:04.236 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:26:04.236 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:04.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:26:04.250 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:26:04.250 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:04.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 Processor -- start 2024-04-25T10:26:04.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 -- start start 2024-04-25T10:26:04.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17e815bc40 0x7f17e806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cda10 0x7f17e80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f17e80cf930 0x7f17e80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e80b2770 con 0x7f17e815bc40 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e806c670 con 0x7f17e80cf930 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17ef400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e806c950 con 0x7f17e80cda10 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.415+0000 7f17eca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17e815bc40 0x7f17e806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17e815bc40 0x7f17e806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50520/0 (socket says 172.21.15.87:50520) 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 -- 172.21.15.87:0/3069839425 learned_addr learned my addr 172.21.15.87:0/3069839425 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:04.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17ed400640 1 --2- 172.21.15.87:0/3069839425 >> v2:172.21.15.169:3300/0 conn(0x7f17e80cf930 0x7f17e80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17dfe00640 1 --2- 172.21.15.87:0/3069839425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cda10 0x7f17e80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 -- 172.21.15.87:0/3069839425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cda10 msgr2=0x7f17e80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:04.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 --2- 172.21.15.87:0/3069839425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cda10 0x7f17e80b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 -- 172.21.15.87:0/3069839425 >> v2:172.21.15.169:3300/0 conn(0x7f17e80cf930 msgr2=0x7f17e80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:04.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 --2- 172.21.15.87:0/3069839425 >> v2:172.21.15.169:3300/0 conn(0x7f17e80cf930 0x7f17e80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 -- 172.21.15.87:0/3069839425 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e806cc30 con 0x7f17e815bc40 2024-04-25T10:26:04.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17eca00640 1 --2- 172.21.15.87:0/3069839425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e815bc40 0x7f17e806c090 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f17e80b3f80 tx=0x7f17d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b917adda1ed1dbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17e7e00640 1 -- 172.21.15.87:0/3069839425 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d80b0070 con 0x7f17e815bc40 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17e7e00640 1 -- 172.21.15.87:0/3069839425 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d80ab600 con 0x7f17e815bc40 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.419+0000 7f17e7e00640 1 -- 172.21.15.87:0/3069839425 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d80ab9d0 con 0x7f17e815bc40 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- 172.21.15.87:0/3069839425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e815bc40 msgr2=0x7f17e806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 --2- 172.21.15.87:0/3069839425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e815bc40 0x7f17e806c090 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f17e80b3f80 tx=0x7f17d801c520 comp rx=0 tx=0).stop 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- 172.21.15.87:0/3069839425 shutdown_connections 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 --2- 172.21.15.87:0/3069839425 >> v2:172.21.15.169:3300/0 conn(0x7f17e80cf930 0x7f17e80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 --2- 172.21.15.87:0/3069839425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cda10 0x7f17e80b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 --2- 172.21.15.87:0/3069839425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e815bc40 0x7f17e806c090 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- 172.21.15.87:0/3069839425 >> 172.21.15.87:0/3069839425 conn(0x7f17e8066b70 msgr2=0x7f17e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- 172.21.15.87:0/3069839425 shutdown_connections 2024-04-25T10:26:04.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- 172.21.15.87:0/3069839425 wait complete. 2024-04-25T10:26:04.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 Processor -- start 2024-04-25T10:26:04.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- start start 2024-04-25T10:26:04.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e80cda10 0x7f17e80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cf930 0x7f17e814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e815bc40 0x7f17e814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8160230 con 0x7f17e80cda10 2024-04-25T10:26:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e81600b0 con 0x7f17e815bc40 2024-04-25T10:26:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ef400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e81603b0 con 0x7f17e80cf930 2024-04-25T10:26:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e80cda10 0x7f17e80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e80cda10 0x7f17e80b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50532/0 (socket says 172.21.15.87:50532) 2024-04-25T10:26:04.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 -- 172.21.15.87:0/1079071871 learned_addr learned my addr 172.21.15.87:0/1079071871 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 -- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cf930 msgr2=0x7f17e814d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ed400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e815bc40 0x7f17e814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17dfe00640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cf930 0x7f17e814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cf930 0x7f17e814d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 -- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e815bc40 msgr2=0x7f17e814a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e815bc40 0x7f17e814a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 -- 172.21.15.87:0/1079071871 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e814dda0 con 0x7f17e80cda10 2024-04-25T10:26:04.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17ed400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e815bc40 0x7f17e814a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17dfe00640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cf930 0x7f17e814d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.423+0000 7f17eca00640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e80cda10 0x7f17e80b0750 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f17d80ae040 tx=0x7f17d801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17e5600640 1 -- 172.21.15.87:0/1079071871 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d80b0040 con 0x7f17e80cda10 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17e5600640 1 -- 172.21.15.87:0/1079071871 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d80bc070 con 0x7f17e80cda10 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17e5600640 1 -- 172.21.15.87:0/1079071871 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d80b6640 con 0x7f17e80cda10 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e814e080 con 0x7f17e80cda10 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e814e640 con 0x7f17e80cda10 2024-04-25T10:26:04.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e804ef30 con 0x7f17e80cda10 2024-04-25T10:26:04.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17e5600640 1 -- 172.21.15.87:0/1079071871 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17d80b67e0 con 0x7f17e80cda10 2024-04-25T10:26:04.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17e5600640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17e5600640 1 -- 172.21.15.87:0/1079071871 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f17d811f190 con 0x7f17e80cda10 2024-04-25T10:26:04.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.427+0000 7f17dfe00640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.431+0000 7f17dfe00640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f17e814f0c0 tx=0x7f17d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.431+0000 7f17e5600640 1 -- 172.21.15.87:0/1079071871 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17d8016110 con 0x7f17e80cda10 2024-04-25T10:26:04.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17e80d0080 con 0x7f17e80cda10 2024-04-25T10:26:04.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17e5600640 1 -- 172.21.15.87:0/1079071871 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f17d80f0400 con 0x7f17e80cda10 2024-04-25T10:26:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:04.618 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":130,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1787812144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1787812144},{"type":"v1","addr":"172.21.15.169:6834","nonce":1787812144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/554048725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":554048725},{"type":"v1","addr":"172.21.15.169:6835","nonce":554048725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1720179922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":1720179922},{"type":"v1","addr":"172.21.15.87:6837","nonce":1720179922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/4210226436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4210226436},{"type":"v1","addr":"172.21.15.87:6836","nonce":4210226436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[]} 2024-04-25T10:26:04.618 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 130 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 msgr2=0x7f17c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f17e814f0c0 tx=0x7f17d0008040 comp rx=0 tx=0).stop 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e80cda10 msgr2=0x7f17e80b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e80cda10 0x7f17e80b0750 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f17d80ae040 tx=0x7f17d801ad60 comp rx=0 tx=0).stop 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 shutdown_connections 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17c4066f60 0x7f17c4069420 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17e815bc40 0x7f17e814a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17e80cf930 0x7f17e814d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 --2- 172.21.15.87:0/1079071871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e80cda10 0x7f17e80b0750 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 >> 172.21.15.87:0/1079071871 conn(0x7f17e8066b70 msgr2=0x7f17e80b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:04.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 shutdown_connections 2024-04-25T10:26:04.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.619+0000 7f17ef400640 1 -- 172.21.15.87:0/1079071871 wait complete. 2024-04-25T10:26:04.638 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:26:04.639 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:26:04.639 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:26:04.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:04.639 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:26:04.639 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:26:04.640 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:26:04.640 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:04.640 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:26:04.640 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:26:04.640 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:26:04.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:04.641 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:26:04.641 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:26:04.641 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:26:04.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:04.653 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:26:04.653 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:26:04.654 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:26:04.654 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:26:04.656 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:26:04.656 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:26:04.657 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:26:04.657 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:26:04.703 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:26:04.704 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:04.705 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:26:04.705 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:04.706 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:26:04.706 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:04.707 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:26:04.707 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:04.708 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:26:04.790 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:26:04.790 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-04-25T10:26:04.800 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T10:26:04.800 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T10:26:04.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb701600640 1 Processor -- start 2024-04-25T10:26:04.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb701600640 1 -- start start 2024-04-25T10:26:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb701600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6fc156030 0x7fb6fc071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb701600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6fc15c640 con 0x7fb6fc15c0a0 2024-04-25T10:26:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb701600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6fc15c920 con 0x7fb6fc0670b0 2024-04-25T10:26:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb701600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6fc15d260 con 0x7fb6fc156030 2024-04-25T10:26:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb6fa000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb6fc15c0a0 0x7fb6fc0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53038/0 (socket says 172.21.15.87:53038) 2024-04-25T10:26:04.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb6fa000640 1 -- 172.21.15.87:0/3483716280 learned_addr learned my addr 172.21.15.87:0/3483716280 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.855+0000 7fb6faa00640 1 --2- 172.21.15.87:0/3483716280 >> v2:172.21.15.169:3300/0 conn(0x7fb6fc156030 0x7fb6fc071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6f9600640 1 -- 172.21.15.87:0/3483716280 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237646529 0 0) 0x7fb6fc15c640 con 0x7fb6fc15c0a0 2024-04-25T10:26:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6f9600640 1 -- 172.21.15.87:0/3483716280 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e0003530 con 0x7fb6fc15c0a0 2024-04-25T10:26:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6f9600640 1 -- 172.21.15.87:0/3483716280 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938859980 0 0) 0x7fb6fc15c920 con 0x7fb6fc0670b0 2024-04-25T10:26:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6f9600640 1 -- 172.21.15.87:0/3483716280 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6fc15c640 con 0x7fb6fc0670b0 2024-04-25T10:26:04.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6faa00640 1 -- 172.21.15.87:0/3483716280 >> v1:172.21.15.169:6789/0 conn(0x7fb6fc0670b0 legacy=0x7fb6fc0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6faa00640 1 -- 172.21.15.87:0/3483716280 >> v1:172.21.15.87:6789/0 conn(0x7fb6fc15c0a0 legacy=0x7fb6fc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6faa00640 1 -- 172.21.15.87:0/3483716280 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc15cc00 con 0x7fb6fc156030 2024-04-25T10:26:04.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6faa00640 1 --2- 172.21.15.87:0/3483716280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 0x7fb6fc071f10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097c0 tx=0x7fb6ec01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b36773bdcc4a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6f9600640 1 -- 172.21.15.87:0/3483716280 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0aa5f0 con 0x7fb6fc156030 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6f9600640 1 -- 172.21.15.87:0/3483716280 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0af070 con 0x7fb6fc156030 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb6f9600640 1 -- 172.21.15.87:0/3483716280 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0aa9b0 con 0x7fb6fc156030 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb701600640 1 -- 172.21.15.87:0/3483716280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 msgr2=0x7fb6fc071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb701600640 1 --2- 172.21.15.87:0/3483716280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 0x7fb6fc071f10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097c0 tx=0x7fb6ec01c470 comp rx=0 tx=0).stop 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb701600640 1 -- 172.21.15.87:0/3483716280 shutdown_connections 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.859+0000 7fb701600640 1 --2- 172.21.15.87:0/3483716280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 0x7fb6fc071f10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 -- 172.21.15.87:0/3483716280 >> 172.21.15.87:0/3483716280 conn(0x7fb6fc06ac90 msgr2=0x7fb6fc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 -- 172.21.15.87:0/3483716280 shutdown_connections 2024-04-25T10:26:04.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 -- 172.21.15.87:0/3483716280 wait complete. 2024-04-25T10:26:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 Processor -- start 2024-04-25T10:26:04.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 -- start start 2024-04-25T10:26:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6fc0670b0 0x7fb6fc140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 0x7fb6fc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6fc15c0a0 0x7fb6fc0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc169870 con 0x7fb6fc0670b0 2024-04-25T10:26:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc1696f0 con 0x7fb6fc156030 2024-04-25T10:26:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb701600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc1699f0 con 0x7fb6fc15c0a0 2024-04-25T10:26:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6fc0670b0 0x7fb6fc140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6fc0670b0 0x7fb6fc140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50558/0 (socket says 172.21.15.87:50558) 2024-04-25T10:26:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 -- 172.21.15.87:0/3595471623 learned_addr learned my addr 172.21.15.87:0/3595471623 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6fa000640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 0x7fb6fc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6fb400640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6fc15c0a0 0x7fb6fc0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 -- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6fc15c0a0 msgr2=0x7fb6fc0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6fc15c0a0 0x7fb6fc0d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 -- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 msgr2=0x7fb6fc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 0x7fb6fc0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 -- 172.21.15.87:0/3595471623 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc158670 con 0x7fb6fc0670b0 2024-04-25T10:26:04.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6faa00640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6fc0670b0 0x7fb6fc140160 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec019080 tx=0x7fb6ec01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6f2a00640 1 -- 172.21.15.87:0/3595471623 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0be410 con 0x7fb6fc0670b0 2024-04-25T10:26:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6f2a00640 1 -- 172.21.15.87:0/3595471623 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0be5d0 con 0x7fb6fc0670b0 2024-04-25T10:26:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.863+0000 7fb6f2a00640 1 -- 172.21.15.87:0/3595471623 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0be9c0 con 0x7fb6fc0670b0 2024-04-25T10:26:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6fc158950 con 0x7fb6fc0670b0 2024-04-25T10:26:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc158f10 con 0x7fb6fc0670b0 2024-04-25T10:26:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6fc04ef30 con 0x7fb6fc0670b0 2024-04-25T10:26:04.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb6f2a00640 1 -- 172.21.15.87:0/3595471623 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6ec0bc020 con 0x7fb6fc0670b0 2024-04-25T10:26:04.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb6f2a00640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb6dc066f60 0x7fb6dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb6f2a00640 1 -- 172.21.15.87:0/3595471623 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6ec11ebc0 con 0x7fb6fc0670b0 2024-04-25T10:26:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb6fa000640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb6dc066f60 0x7fb6dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.867+0000 7fb6fa000640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb6dc066f60 0x7fb6dc069420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc15bf90 tx=0x7fb6e8007d80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:04.871+0000 7fb6f2a00640 1 -- 172.21.15.87:0/3595471623 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6ec0c5070 con 0x7fb6fc0670b0 2024-04-25T10:26:05.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.035+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb6fc1406a0 con 0x7fb6fc0670b0 2024-04-25T10:26:05.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb6f2a00640 1 -- 172.21.15.87:0/3595471623 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v181) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb6ec016090 con 0x7fb6fc0670b0 2024-04-25T10:26:05.729 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-04-25T10:26:05.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb6dc066f60 msgr2=0x7fb6dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb6dc066f60 0x7fb6dc069420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc15bf90 tx=0x7fb6e8007d80 comp rx=0 tx=0).stop 2024-04-25T10:26:05.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6fc0670b0 msgr2=0x7fb6fc140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6fc0670b0 0x7fb6fc140160 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec019080 tx=0x7fb6ec01acb0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 shutdown_connections 2024-04-25T10:26:05.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb6dc066f60 0x7fb6dc069420 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6fc15c0a0 0x7fb6fc0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6fc156030 0x7fb6fc0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 --2- 172.21.15.87:0/3595471623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6fc0670b0 0x7fb6fc140160 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 >> 172.21.15.87:0/3595471623 conn(0x7fb6fc06ac90 msgr2=0x7fb6fc068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:05.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 shutdown_connections 2024-04-25T10:26:05.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.731+0000 7fb701600640 1 -- 172.21.15.87:0/3595471623 wait complete. 2024-04-25T10:26:05.757 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:26:05.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476c600640 1 Processor -- start 2024-04-25T10:26:05.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476c600640 1 -- start start 2024-04-25T10:26:05.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f476406b0d0 0x7f476406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415e100 0x7f476406bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476c600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47640b22a0 con 0x7f476415dd20 2024-04-25T10:26:05.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47640b2580 con 0x7f476406b0d0 2024-04-25T10:26:05.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47640b2860 con 0x7f476415e100 2024-04-25T10:26:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415e100 0x7f476406bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f4769200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f476406b0d0 0x7f476406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f4769c00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f476415dd20 0x7f47640b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:53640/0 (socket says 172.21.15.87:53640) 2024-04-25T10:26:05.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f4769c00640 1 -- 172.21.15.87:0/568340465 learned_addr learned my addr 172.21.15.87:0/568340465 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476a600640 1 -- 172.21.15.87:0/568340465 >> v2:172.21.15.169:3300/0 conn(0x7f476406b0d0 msgr2=0x7f476406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476a600640 1 --2- 172.21.15.87:0/568340465 >> v2:172.21.15.169:3300/0 conn(0x7f476406b0d0 0x7f476406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476a600640 1 -- 172.21.15.87:0/568340465 >> v1:172.21.15.169:6789/0 conn(0x7f476415dd20 legacy=0x7f47640b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f476a600640 1 -- 172.21.15.87:0/568340465 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640b2b40 con 0x7f476415e100 2024-04-25T10:26:05.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.895+0000 7f4769200640 1 --2- 172.21.15.87:0/568340465 >> v2:172.21.15.169:3300/0 conn(0x7f476406b0d0 0x7f476406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:05.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476a600640 1 --2- 172.21.15.87:0/568340465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415e100 0x7f476406bae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f475c009d90 tx=0x7f475c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29207ffa04e299c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f4763e00640 1 -- 172.21.15.87:0/568340465 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0b2070 con 0x7f476415e100 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f4763e00640 1 -- 172.21.15.87:0/568340465 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c0a5940 con 0x7f476415e100 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f4763e00640 1 -- 172.21.15.87:0/568340465 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0a5d10 con 0x7f476415e100 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 -- 172.21.15.87:0/568340465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415e100 msgr2=0x7f476406bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 --2- 172.21.15.87:0/568340465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415e100 0x7f476406bae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f475c009d90 tx=0x7f475c01c970 comp rx=0 tx=0).stop 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 -- 172.21.15.87:0/568340465 shutdown_connections 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 --2- 172.21.15.87:0/568340465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415e100 0x7f476406bae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 --2- 172.21.15.87:0/568340465 >> v2:172.21.15.169:3300/0 conn(0x7f476406b0d0 0x7f476406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 -- 172.21.15.87:0/568340465 >> 172.21.15.87:0/568340465 conn(0x7f47640d1900 msgr2=0x7f47640d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 -- 172.21.15.87:0/568340465 shutdown_connections 2024-04-25T10:26:05.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.899+0000 7f476c600640 1 -- 172.21.15.87:0/568340465 wait complete. 2024-04-25T10:26:05.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 Processor -- start 2024-04-25T10:26:05.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 -- start start 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476406b0d0 0x7f4764158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415dd20 0x7f476414adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476415e100 0x7f47640c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640af1f0 con 0x7f476415e100 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640af070 con 0x7f476406b0d0 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47640af370 con 0x7f476415dd20 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476415e100 0x7f47640c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476415e100 0x7f47640c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50560/0 (socket says 172.21.15.87:50560) 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 -- 172.21.15.87:0/847769409 learned_addr learned my addr 172.21.15.87:0/847769409 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f4769200640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415dd20 0x7f476414adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f4769c00640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476406b0d0 0x7f4764158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 -- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415dd20 msgr2=0x7f476414adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415dd20 0x7f476414adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 -- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476406b0d0 msgr2=0x7f4764158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476406b0d0 0x7f4764158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 -- 172.21.15.87:0/847769409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640b5a10 con 0x7f476415e100 2024-04-25T10:26:05.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476a600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476415e100 0x7f47640c7070 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f475c01b020 tx=0x7f475c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f4761600640 1 -- 172.21.15.87:0/847769409 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0b2050 con 0x7f476415e100 2024-04-25T10:26:05.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f4761600640 1 -- 172.21.15.87:0/847769409 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c0be070 con 0x7f476415e100 2024-04-25T10:26:05.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4764159360 con 0x7f476415e100 2024-04-25T10:26:05.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f4761600640 1 -- 172.21.15.87:0/847769409 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c009450 con 0x7f476415e100 2024-04-25T10:26:05.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.903+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4764159920 con 0x7f476415e100 2024-04-25T10:26:05.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.907+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476404ef30 con 0x7f476415e100 2024-04-25T10:26:05.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.907+0000 7f4761600640 1 -- 172.21.15.87:0/847769409 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f475c0adcc0 con 0x7f476415e100 2024-04-25T10:26:05.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.907+0000 7f4761600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f60 0x7f4744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.907+0000 7f4761600640 1 -- 172.21.15.87:0/847769409 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f475c11fec0 con 0x7f476415e100 2024-04-25T10:26:05.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.907+0000 7f4769c00640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f60 0x7f4744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.907+0000 7f4769c00640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f60 0x7f4744069420 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4764160ed0 tx=0x7f4758007e90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:05.911+0000 7f4761600640 1 -- 172.21.15.87:0/847769409 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f475c016560 con 0x7f476415e100 2024-04-25T10:26:06.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f476406f880 con 0x7f476415e100 2024-04-25T10:26:06.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f4761600640 1 -- 172.21.15.87:0/847769409 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f475c0f0ce0 con 0x7f476415e100 2024-04-25T10:26:06.125 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:06.125 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmJWz5MxAABNJ5/6b/ZFn18CQXSSXllw==", 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-04-25T10:26:06.126 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXGgaBBAARdrCGANMqWlmGaD3DXBkcQ==", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmBxoCOxAA/yccKmXZTx1Fxecp5G/iOg==", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:26:06.127 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpm/SE5CxAAl+xB79mZBYiA7Fe2KCtE3g==", 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.128 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBPLCpmXlHmERAADViPLiftjFdtRSCMb3egpg==", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBSLCpmZNp8HRAAyoE1dlpsR4KGUIpKStsXrQ==", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.129 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBVLCpmNTUoJBAAIXaMwIAddJyDCjNGOdQa+w==", 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBYLCpmO6c5LhAA+JUDGiGSX05YeUbp6vx0NQ==", 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.130 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBeLCpmspuPJRAA3CkmFlxLd1ViWdrxaHGSmw==", 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBhLCpmVz0KLhAA8TjO6F4FwbGRj90o/1Xyow==", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBkLCpmn2EXORAAwjLQZiufdcw+kldN2XzmDw==", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.132 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBoLCpmSBFuBRAA4dvpiLNYXKuKnTugS/cJog==", 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow==", 2024-04-25T10:26:06.133 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmI+avGBAA7+ZIMXJdGWOjyuHgsiurOg==", 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmvmpeFRAAOm14abVZafKJ6oK5gCd+ew==", 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBMLCpmXn25HhAAoZ8P/0n8vQEER+4zq7fo4g==", 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.135 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmgujSFxAABh0p8BM9cpAwchyq5GFkHQ==", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmk/JTLxAAaN3JnwJ0kCBsy0tm36Hgcg==", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQBLLCpmavKoKxAABdPUkYw9HG2yFfeCulQVIQ==", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-04-25T10:26:06.137 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-04-25T10:26:06.138 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.138 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:06.138 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-04-25T10:26:06.138 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f60 msgr2=0x7f4744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f60 0x7f4744069420 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4764160ed0 tx=0x7f4758007e90 comp rx=0 tx=0).stop 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476415e100 msgr2=0x7f47640c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476415e100 0x7f47640c7070 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f475c01b020 tx=0x7f475c019710 comp rx=0 tx=0).stop 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 shutdown_connections 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4744066f60 0x7f4744069420 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f476415e100 0x7f47640c7070 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476415dd20 0x7f476414adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 --2- 172.21.15.87:0/847769409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476406b0d0 0x7f4764158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 >> 172.21.15.87:0/847769409 conn(0x7f47640d1900 msgr2=0x7f476406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 shutdown_connections 2024-04-25T10:26:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.127+0000 7f476c600640 1 -- 172.21.15.87:0/847769409 wait complete. 2024-04-25T10:26:06.147 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.283+0000 7f15ce000640 1 Processor -- start 2024-04-25T10:26:06.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.283+0000 7f15ce000640 1 -- start start 2024-04-25T10:26:06.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.283+0000 7f15ce000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15c81550b0 0x7f15c80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15ce000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c8071050 con 0x7f15c815b910 2024-04-25T10:26:06.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15ce000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c815d040 con 0x7f15c80cfe70 2024-04-25T10:26:06.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15ce000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c815bd80 con 0x7f15c81550b0 2024-04-25T10:26:06.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15c81550b0 0x7f15c80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15c81550b0 0x7f15c80cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50564/0 (socket says 172.21.15.87:50564) 2024-04-25T10:26:06.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7400640 1 -- 172.21.15.87:0/1537086553 learned_addr learned my addr 172.21.15.87:0/1537086553 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:06.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7e00640 1 --1- 172.21.15.87:0/1537086553 >> v1:172.21.15.87:6789/0 conn(0x7f15c815b910 0x7f15c80b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53042/0 (socket says 172.21.15.87:53042) 2024-04-25T10:26:06.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7400640 1 -- 172.21.15.87:0/1537086553 >> v1:172.21.15.169:6789/0 conn(0x7f15c80cfe70 legacy=0x7f15c815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7400640 1 -- 172.21.15.87:0/1537086553 >> v1:172.21.15.87:6789/0 conn(0x7f15c815b910 legacy=0x7f15c80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7400640 1 -- 172.21.15.87:0/1537086553 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c80712f0 con 0x7f15c81550b0 2024-04-25T10:26:06.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c6a00640 1 -- 172.21.15.87:0/1537086553 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430177618 0 0) 0x7f15c8071050 con 0x7f15c815b910 2024-04-25T10:26:06.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c7400640 1 --2- 172.21.15.87:0/1537086553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c81550b0 0x7f15c80cf930 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f15b800d180 tx=0x7f15b801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e4b0424e4e096b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c6a00640 1 -- 172.21.15.87:0/1537086553 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800b2d0 con 0x7f15c81550b0 2024-04-25T10:26:06.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.287+0000 7f15c6a00640 1 -- 172.21.15.87:0/1537086553 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b800b470 con 0x7f15c81550b0 2024-04-25T10:26:06.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c6a00640 1 -- 172.21.15.87:0/1537086553 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80a7af0 con 0x7f15c81550b0 2024-04-25T10:26:06.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- 172.21.15.87:0/1537086553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c81550b0 msgr2=0x7f15c80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 --2- 172.21.15.87:0/1537086553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c81550b0 0x7f15c80cf930 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f15b800d180 tx=0x7f15b801ec10 comp rx=0 tx=0).stop 2024-04-25T10:26:06.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- 172.21.15.87:0/1537086553 shutdown_connections 2024-04-25T10:26:06.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 --2- 172.21.15.87:0/1537086553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c81550b0 0x7f15c80cf930 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- 172.21.15.87:0/1537086553 >> 172.21.15.87:0/1537086553 conn(0x7f15c8066b70 msgr2=0x7f15c8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- 172.21.15.87:0/1537086553 shutdown_connections 2024-04-25T10:26:06.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- 172.21.15.87:0/1537086553 wait complete. 2024-04-25T10:26:06.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 Processor -- start 2024-04-25T10:26:06.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- start start 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15c80cfe70 0x7f15c814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15c81550b0 0x7f15c8155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c815b910 0x7f15c8150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c806c090 con 0x7f15c815b910 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c80720b0 con 0x7f15c81550b0 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15ce000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c806c210 con 0x7f15c80cfe70 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c815b910 0x7f15c8150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c815b910 0x7f15c8150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50580/0 (socket says 172.21.15.87:50580) 2024-04-25T10:26:06.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 -- 172.21.15.87:0/3091666402 learned_addr learned my addr 172.21.15.87:0/3091666402 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15bea00640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15c81550b0 0x7f15c8155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7400640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15c80cfe70 0x7f15c814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 -- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15c80cfe70 msgr2=0x7f15c814f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15c80cfe70 0x7f15c814f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 -- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15c81550b0 msgr2=0x7f15c8155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15c81550b0 0x7f15c8155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7e00640 1 -- 172.21.15.87:0/3091666402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c8154430 con 0x7f15c815b910 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.291+0000 7f15c7400640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15c80cfe70 0x7f15c814f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15c7e00640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c815b910 0x7f15c8150d70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c410 tx=0x7f15c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15bfe00640 1 -- 172.21.15.87:0/3091666402 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c00074e0 con 0x7f15c815b910 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15bfe00640 1 -- 172.21.15.87:0/3091666402 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c00047f0 con 0x7f15c815b910 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c8154710 con 0x7f15c815b910 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15bfe00640 1 -- 172.21.15.87:0/3091666402 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0004be0 con 0x7f15c815b910 2024-04-25T10:26:06.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c813e070 con 0x7f15c815b910 2024-04-25T10:26:06.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c8154220 con 0x7f15c815b910 2024-04-25T10:26:06.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15bfe00640 1 -- 172.21.15.87:0/3091666402 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15c0011070 con 0x7f15c815b910 2024-04-25T10:26:06.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15bfe00640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f15a4066f90 0x7f15a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15bfe00640 1 -- 172.21.15.87:0/3091666402 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f15c0084900 con 0x7f15c815b910 2024-04-25T10:26:06.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.295+0000 7f15c7400640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f15a4066f90 0x7f15a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.299+0000 7f15c7400640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f15a4066f90 0x7f15a4069450 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f15b80b0000 tx=0x7f15b80a9070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.299+0000 7f15bfe00640 1 -- 172.21.15.87:0/3091666402 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15c0055b30 con 0x7f15c815b910 2024-04-25T10:26:06.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.455+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15c804ef30 con 0x7f15c815b910 2024-04-25T10:26:06.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15bfe00640 1 -- 172.21.15.87:0/3091666402 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f15c0055430 con 0x7f15c815b910 2024-04-25T10:26:06.457 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:06.457 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":183,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:26:05.741501+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f15a4066f90 msgr2=0x7f15a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f15a4066f90 0x7f15a4069450 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f15b80b0000 tx=0x7f15b80a9070 comp rx=0 tx=0).stop 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c815b910 msgr2=0x7f15c8150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c815b910 0x7f15c8150d70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c410 tx=0x7f15c000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 shutdown_connections 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f15a4066f90 0x7f15a4069450 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15c815b910 0x7f15c8150d70 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15c81550b0 0x7f15c8155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 --2- 172.21.15.87:0/3091666402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15c80cfe70 0x7f15c814f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 >> 172.21.15.87:0/3091666402 conn(0x7f15c8066b70 msgr2=0x7f15c80b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 shutdown_connections 2024-04-25T10:26:06.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.459+0000 7f15ce000640 1 -- 172.21.15.87:0/3091666402 wait complete. 2024-04-25T10:26:06.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:26:06.478 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:26:06.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b89000640 1 Processor -- start 2024-04-25T10:26:06.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b89000640 1 -- start start 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b89000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b8415bb70 0x7f9b8406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b89000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9b84072220 0x7f9b8406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b89000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8406f250 con 0x7f9b84071e40 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b89000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8415f720 con 0x7f9b8415bb70 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b89000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8415f9c0 con 0x7f9b84072220 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b81400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b8415bb70 0x7f9b8406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b81400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b8415bb70 0x7f9b8406e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50588/0 (socket says 172.21.15.87:50588) 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b81400640 1 -- 172.21.15.87:0/909174142 learned_addr learned my addr 172.21.15.87:0/909174142 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b82800640 1 --2- 172.21.15.87:0/909174142 >> v2:172.21.15.169:3300/0 conn(0x7f9b84072220 0x7f9b8406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b81400640 1 -- 172.21.15.87:0/909174142 >> v2:172.21.15.169:3300/0 conn(0x7f9b84072220 msgr2=0x7f9b8406ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b81400640 1 --2- 172.21.15.87:0/909174142 >> v2:172.21.15.169:3300/0 conn(0x7f9b84072220 0x7f9b8406ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b81400640 1 -- 172.21.15.87:0/909174142 >> v1:172.21.15.169:6789/0 conn(0x7f9b84071e40 legacy=0x7f9b8415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.619+0000 7f9b81400640 1 -- 172.21.15.87:0/909174142 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8415fca0 con 0x7f9b8415bb70 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b81400640 1 --2- 172.21.15.87:0/909174142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b8415bb70 0x7f9b8406e7d0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f9b840ce360 tx=0x7f9b7001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47f0dbed0c5ce9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b80a00640 1 -- 172.21.15.87:0/909174142 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b700ab660 con 0x7f9b8415bb70 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b80a00640 1 -- 172.21.15.87:0/909174142 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b700b0070 con 0x7f9b8415bb70 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b80a00640 1 -- 172.21.15.87:0/909174142 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b700aba20 con 0x7f9b8415bb70 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 -- 172.21.15.87:0/909174142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b8415bb70 msgr2=0x7f9b8406e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 --2- 172.21.15.87:0/909174142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b8415bb70 0x7f9b8406e7d0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f9b840ce360 tx=0x7f9b7001c460 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 -- 172.21.15.87:0/909174142 shutdown_connections 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 --2- 172.21.15.87:0/909174142 >> v2:172.21.15.169:3300/0 conn(0x7f9b84072220 0x7f9b8406ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 --2- 172.21.15.87:0/909174142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b8415bb70 0x7f9b8406e7d0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 -- 172.21.15.87:0/909174142 >> 172.21.15.87:0/909174142 conn(0x7f9b8406ac90 msgr2=0x7f9b8406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 -- 172.21.15.87:0/909174142 shutdown_connections 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.623+0000 7f9b89000640 1 -- 172.21.15.87:0/909174142 wait complete. 2024-04-25T10:26:06.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 Processor -- start 2024-04-25T10:26:06.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 -- start start 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b84071e40 0x7f9b8414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b84072220 0x7f9b8413e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8415bb70 0x7f9b841653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840cf870 con 0x7f9b84071e40 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840cf6f0 con 0x7f9b8415bb70 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840cf9f0 con 0x7f9b84072220 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b84071e40 0x7f9b8414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b84071e40 0x7f9b8414c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50592/0 (socket says 172.21.15.87:50592) 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 -- 172.21.15.87:0/2079646131 learned_addr learned my addr 172.21.15.87:0/2079646131 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b82800640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8415bb70 0x7f9b841653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81400640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b84072220 0x7f9b8413e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 -- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b84072220 msgr2=0x7f9b8413e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b84072220 0x7f9b8413e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 -- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8415bb70 msgr2=0x7f9b841653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8415bb70 0x7f9b841653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 -- 172.21.15.87:0/2079646131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8414c9e0 con 0x7f9b84071e40 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b82800640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8415bb70 0x7f9b841653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b81e00640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b84071e40 0x7f9b8414c140 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c420 tx=0x7f9b7400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b7a000640 1 -- 172.21.15.87:0/2079646131 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740052d0 con 0x7f9b84071e40 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b7a000640 1 -- 172.21.15.87:0/2079646131 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74005470 con 0x7f9b84071e40 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b7a000640 1 -- 172.21.15.87:0/2079646131 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74007260 con 0x7f9b84071e40 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8414ccc0 con 0x7f9b84071e40 2024-04-25T10:26:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.627+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b84164ea0 con 0x7f9b84071e40 2024-04-25T10:26:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.631+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8404ef30 con 0x7f9b84071e40 2024-04-25T10:26:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.631+0000 7f9b7a000640 1 -- 172.21.15.87:0/2079646131 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b74007400 con 0x7f9b84071e40 2024-04-25T10:26:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.631+0000 7f9b7a000640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b64066f60 0x7f9b64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.631+0000 7f9b7a000640 1 -- 172.21.15.87:0/2079646131 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b74084160 con 0x7f9b84071e40 2024-04-25T10:26:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.631+0000 7f9b81400640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b64066f60 0x7f9b64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.631+0000 7f9b81400640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b64066f60 0x7f9b64069420 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9b700af000 tx=0x7f9b700a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.635+0000 7f9b7a000640 1 -- 172.21.15.87:0/2079646131 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b74054e70 con 0x7f9b84071e40 2024-04-25T10:26:06.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b84071790 con 0x7f9b84071e40 2024-04-25T10:26:06.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b7a000640 1 -- 172.21.15.87:0/2079646131 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b74058f10 con 0x7f9b84071e40 2024-04-25T10:26:06.793 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b64066f60 msgr2=0x7f9b64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b64066f60 0x7f9b64069420 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9b700af000 tx=0x7f9b700a7000 comp rx=0 tx=0).stop 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b84071e40 msgr2=0x7f9b8414c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b84071e40 0x7f9b8414c140 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c420 tx=0x7f9b7400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 shutdown_connections 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f9b64066f60 0x7f9b64069420 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8415bb70 0x7f9b841653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b84072220 0x7f9b8413e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 --2- 172.21.15.87:0/2079646131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b84071e40 0x7f9b8414c140 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 >> 172.21.15.87:0/2079646131 conn(0x7f9b8406ac90 msgr2=0x7f9b8414b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 shutdown_connections 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.795+0000 7f9b89000640 1 -- 172.21.15.87:0/2079646131 wait complete. 2024-04-25T10:26:06.816 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:26:06.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a41600640 1 Processor -- start 2024-04-25T10:26:06.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a41600640 1 -- start start 2024-04-25T10:26:06.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a41600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a3c0d0040 0x7f1a3c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a41600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c15b740 0x7f1a3c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a41600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c071050 con 0x7f1a3c1550b0 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a41600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c15d040 con 0x7f1a3c0d0040 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a41600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0712f0 con 0x7f1a3c15b740 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a3c0d0040 0x7f1a3c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1a3c1550b0 0x7f1a3c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53056/0 (socket says 172.21.15.87:53056) 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3aa00640 1 -- 172.21.15.87:0/325343920 learned_addr learned my addr 172.21.15.87:0/325343920 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3b400640 1 --2- 172.21.15.87:0/325343920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c15b740 0x7f1a3c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3a000640 1 -- 172.21.15.87:0/325343920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c15b740 msgr2=0x7f1a3c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3a000640 1 --2- 172.21.15.87:0/325343920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c15b740 0x7f1a3c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3a000640 1 -- 172.21.15.87:0/325343920 >> v1:172.21.15.87:6789/0 conn(0x7f1a3c1550b0 legacy=0x7f1a3c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3a000640 1 -- 172.21.15.87:0/325343920 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0715d0 con 0x7f1a3c0d0040 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a3b400640 1 --2- 172.21.15.87:0/325343920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c15b740 0x7f1a3c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:06.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.951+0000 7f1a39600640 1 -- 172.21.15.87:0/325343920 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051812982 0 0) 0x7f1a3c071050 con 0x7f1a3c1550b0 2024-04-25T10:26:06.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a3a000640 1 --2- 172.21.15.87:0/325343920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c0d0040 0x7f1a3c15b200 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f1a280097c0 tx=0x7f1a2801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce709355c7f2fd1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a39600640 1 -- 172.21.15.87:0/325343920 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a280a7070 con 0x7f1a3c0d0040 2024-04-25T10:26:06.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a39600640 1 -- 172.21.15.87:0/325343920 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a280b0070 con 0x7f1a3c0d0040 2024-04-25T10:26:06.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a39600640 1 -- 172.21.15.87:0/325343920 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a280ab890 con 0x7f1a3c0d0040 2024-04-25T10:26:06.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 -- 172.21.15.87:0/325343920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c0d0040 msgr2=0x7f1a3c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 --2- 172.21.15.87:0/325343920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c0d0040 0x7f1a3c15b200 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f1a280097c0 tx=0x7f1a2801c470 comp rx=0 tx=0).stop 2024-04-25T10:26:06.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 -- 172.21.15.87:0/325343920 shutdown_connections 2024-04-25T10:26:06.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 --2- 172.21.15.87:0/325343920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c15b740 0x7f1a3c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 --2- 172.21.15.87:0/325343920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c0d0040 0x7f1a3c15b200 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 -- 172.21.15.87:0/325343920 >> 172.21.15.87:0/325343920 conn(0x7f1a3c066b70 msgr2=0x7f1a3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 -- 172.21.15.87:0/325343920 shutdown_connections 2024-04-25T10:26:06.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.955+0000 7f1a41600640 1 -- 172.21.15.87:0/325343920 wait complete. 2024-04-25T10:26:06.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 Processor -- start 2024-04-25T10:26:06.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 -- start start 2024-04-25T10:26:06.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c0d0040 0x7f1a3c155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a3c1550b0 0x7f1a3c0cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c15b740 0x7f1a3c14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c15bcd0 con 0x7f1a3c15b740 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c15bb50 con 0x7f1a3c1550b0 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c15c030 con 0x7f1a3c0d0040 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c15b740 0x7f1a3c14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c15b740 0x7f1a3c14bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50600/0 (socket says 172.21.15.87:50600) 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 -- 172.21.15.87:0/3640730436 learned_addr learned my addr 172.21.15.87:0/3640730436 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3aa00640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c0d0040 0x7f1a3c155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3a000640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a3c1550b0 0x7f1a3c0cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 -- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c0d0040 msgr2=0x7f1a3c155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c0d0040 0x7f1a3c155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 -- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a3c1550b0 msgr2=0x7f1a3c0cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a3c1550b0 0x7f1a3c0cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 -- 172.21.15.87:0/3640730436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c1566a0 con 0x7f1a3c15b740 2024-04-25T10:26:06.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3a000640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a3c1550b0 0x7f1a3c0cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3aa00640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c0d0040 0x7f1a3c155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a3b400640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c15b740 0x7f1a3c14bc70 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c7f0 tx=0x7f1a3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a32a00640 1 -- 172.21.15.87:0/3640730436 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34015020 con 0x7f1a3c15b740 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a32a00640 1 -- 172.21.15.87:0/3640730436 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34004250 con 0x7f1a3c15b740 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a32a00640 1 -- 172.21.15.87:0/3640730436 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34003620 con 0x7f1a3c15b740 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c156980 con 0x7f1a3c15b740 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.959+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c156f40 con 0x7f1a3c15b740 2024-04-25T10:26:06.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.963+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c04ef30 con 0x7f1a3c15b740 2024-04-25T10:26:06.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.963+0000 7f1a32a00640 1 -- 172.21.15.87:0/3640730436 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a34018020 con 0x7f1a3c15b740 2024-04-25T10:26:06.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.963+0000 7f1a32a00640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a1c066f60 0x7f1a1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.963+0000 7f1a32a00640 1 -- 172.21.15.87:0/3640730436 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1a34085120 con 0x7f1a3c15b740 2024-04-25T10:26:06.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.963+0000 7f1a3aa00640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a1c066f60 0x7f1a1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.963+0000 7f1a3aa00640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a1c066f60 0x7f1a1c069420 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c005fd0 tx=0x7f1a2c005b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:06.967+0000 7f1a32a00640 1 -- 172.21.15.87:0/3640730436 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a3401f020 con 0x7f1a3c15b740 2024-04-25T10:26:07.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1a3c0b09b0 con 0x7f1a3c15b740 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a32a00640 1 -- 172.21.15.87:0/3640730436 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v183) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1a34056350 con 0x7f1a3c15b740 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:07.123 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a1c066f60 msgr2=0x7f1a1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a1c066f60 0x7f1a1c069420 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c005fd0 tx=0x7f1a2c005b80 comp rx=0 tx=0).stop 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c15b740 msgr2=0x7f1a3c14bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c15b740 0x7f1a3c14bc70 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c7f0 tx=0x7f1a3400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 shutdown_connections 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a1c066f60 0x7f1a1c069420 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a3c15b740 0x7f1a3c14bc70 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a3c1550b0 0x7f1a3c0cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 --2- 172.21.15.87:0/3640730436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a3c0d0040 0x7f1a3c155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.123+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 >> 172.21.15.87:0/3640730436 conn(0x7f1a3c066b70 msgr2=0x7f1a3c0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:07.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.127+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 shutdown_connections 2024-04-25T10:26:07.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.127+0000 7f1a41600640 1 -- 172.21.15.87:0/3640730436 wait complete. 2024-04-25T10:26:07.144 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:26:07.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:26:07.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.283+0000 7f61f6e00640 1 Processor -- start 2024-04-25T10:26:07.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.283+0000 7f61f6e00640 1 -- start start 2024-04-25T10:26:07.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61f6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 0x7f61f015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61f6e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f015e9b0 con 0x7f61f0162310 2024-04-25T10:26:07.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61f6e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f015ec90 con 0x7f61f00b4f50 2024-04-25T10:26:07.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61f6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015f250 con 0x7f61f01618a0 2024-04-25T10:26:07.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61efe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f61f0162310 0x7f61f00b4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:53062/0 (socket says 172.21.15.87:53062) 2024-04-25T10:26:07.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61efe00640 1 -- 172.21.15.87:0/560004316 learned_addr learned my addr 172.21.15.87:0/560004316 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61e7400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 0x7f61f015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815900809 0 0) 0x7f61f015e9b0 con 0x7f61f0162310 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d4003530 con 0x7f61f0162310 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89939977 0 0) 0x7f61f015ec90 con 0x7f61f00b4f50 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f015e9b0 con 0x7f61f00b4f50 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698339260 0 0) 0x7f61d4003530 con 0x7f61f0162310 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 msgr2=0x7f61f015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 0x7f61f015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 >> v1:172.21.15.169:6789/0 conn(0x7f61f00b4f50 legacy=0x7f61f00b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f015ef70 con 0x7f61f0162310 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f61d80016a0 con 0x7f61f0162310 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 >> v1:172.21.15.87:6789/0 conn(0x7f61f0162310 legacy=0x7f61f00b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 0x7f61d4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61d4004d50 0x7f61d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015f250 con 0x7f61d4004d50 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d400bbe0 con 0x7f61d40040e0 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d400bde0 con 0x7f61f01618a0 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61f4e00640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61d4004d50 0x7f61d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61efe00640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 0x7f61d4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61e7400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61e7400640 1 -- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 msgr2=0x7f61d4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61e7400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f01618a0 0x7f61d4003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61e7400640 1 -- 172.21.15.87:0/560004316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61d4004d50 msgr2=0x7f61d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:07.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61e7400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61d4004d50 0x7f61d40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61e7400640 1 -- 172.21.15.87:0/560004316 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4003530 con 0x7f61d40040e0 2024-04-25T10:26:07.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.287+0000 7f61f4e00640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61d4004d50 0x7f61d40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:07.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61e7400640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61d4004810 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009ae0 tx=0x7f61e001ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc1b918b3b75e717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:07.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e00096b0 con 0x7f61d40040e0 2024-04-25T10:26:07.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e00a6450 con 0x7f61d40040e0 2024-04-25T10:26:07.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61ef400640 1 -- 172.21.15.87:0/560004316 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e00a6840 con 0x7f61d40040e0 2024-04-25T10:26:07.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61f6e00640 1 -- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 msgr2=0x7f61d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61f6e00640 1 --2- 172.21.15.87:0/560004316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61d4004810 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009ae0 tx=0x7f61e001ca70 comp rx=0 tx=0).stop 2024-04-25T10:26:07.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61f4e00640 1 -- 172.21.15.87:0/560004316 reap_dead start 2024-04-25T10:26:07.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61f6e00640 1 -- 172.21.15.87:0/560004316 shutdown_connections 2024-04-25T10:26:07.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61f6e00640 1 -- 172.21.15.87:0/560004316 >> 172.21.15.87:0/560004316 conn(0x7f61f00b5610 msgr2=0x7f61f00b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:07.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61f6e00640 1 -- 172.21.15.87:0/560004316 shutdown_connections 2024-04-25T10:26:07.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.291+0000 7f61f6e00640 1 -- 172.21.15.87:0/560004316 wait complete. 2024-04-25T10:26:07.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 Processor -- start 2024-04-25T10:26:07.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 -- start start 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61f015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f0159df0 0x7f61f01606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f0147460 0x7f61f01478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00b1210 con 0x7f61f0147460 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00b1090 con 0x7f61d40040e0 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00b1390 con 0x7f61f0159df0 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f0147460 0x7f61f01478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f0147460 0x7f61f01478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50620/0 (socket says 172.21.15.87:50620) 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 -- 172.21.15.87:0/3110066032 learned_addr learned my addr 172.21.15.87:0/3110066032 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61efe00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61f015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 -- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f0159df0 msgr2=0x7f61f01606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f0159df0 0x7f61f01606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 -- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 msgr2=0x7f61f015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61f015a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 -- 172.21.15.87:0/3110066032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0158d50 con 0x7f61f0147460 2024-04-25T10:26:07.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61efe00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61f015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:07.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f4e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f0147460 0x7f61f01478e0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f760 tx=0x7f61e8001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61ecc00640 1 -- 172.21.15.87:0/3110066032 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8001c20 con 0x7f61f0147460 2024-04-25T10:26:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61ecc00640 1 -- 172.21.15.87:0/3110066032 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e8003b90 con 0x7f61f0147460 2024-04-25T10:26:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61ecc00640 1 -- 172.21.15.87:0/3110066032 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8003d80 con 0x7f61f0147460 2024-04-25T10:26:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f0159030 con 0x7f61f0147460 2024-04-25T10:26:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f01595f0 con 0x7f61f0147460 2024-04-25T10:26:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.295+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f004ef30 con 0x7f61f0147460 2024-04-25T10:26:07.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.299+0000 7f61ecc00640 1 -- 172.21.15.87:0/3110066032 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61e801a020 con 0x7f61f0147460 2024-04-25T10:26:07.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.299+0000 7f61ecc00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61cc066f90 0x7f61cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.299+0000 7f61ecc00640 1 -- 172.21.15.87:0/3110066032 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f61e8084560 con 0x7f61f0147460 2024-04-25T10:26:07.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.299+0000 7f61efe00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61cc066f90 0x7f61cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.299+0000 7f61efe00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61cc066f90 0x7f61cc069450 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f61f00676d0 tx=0x7f61d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:07.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.303+0000 7f61ecc00640 1 -- 172.21.15.87:0/3110066032 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61e8055950 con 0x7f61f0147460 2024-04-25T10:26:07.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.463+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f61f0147e20 con 0x7f61f0147460 2024-04-25T10:26:07.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.735+0000 7f61ecc00640 1 -- 172.21.15.87:0/3110066032 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v184) v1 ==== 138+0+0 (secure 0 0 0) 0x7f61e8055250 con 0x7f61f0147460 2024-04-25T10:26:07.736 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-04-25T10:26:07.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61cc066f90 msgr2=0x7f61cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61cc066f90 0x7f61cc069450 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f61f00676d0 tx=0x7f61d8009040 comp rx=0 tx=0).stop 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f0147460 msgr2=0x7f61f01478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f0147460 0x7f61f01478e0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f760 tx=0x7f61e8001b90 comp rx=0 tx=0).stop 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 shutdown_connections 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f61cc066f90 0x7f61cc069450 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f0147460 0x7f61f01478e0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61f0159df0 0x7f61f01606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 --2- 172.21.15.87:0/3110066032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40040e0 0x7f61f015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 >> 172.21.15.87:0/3110066032 conn(0x7f61f00b5610 msgr2=0x7f61f0163b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 shutdown_connections 2024-04-25T10:26:07.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.739+0000 7f61f6e00640 1 -- 172.21.15.87:0/3110066032 wait complete. 2024-04-25T10:26:07.763 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:26:07.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17cb000640 1 Processor -- start 2024-04-25T10:26:07.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17cb000640 1 -- start start 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17cb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4154f70 0x7f17c40cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17cb000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17c40b3ba0 0x7f17c40ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17cb000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c415e6e0 con 0x7f17c40cfea0 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17cb000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c40d0300 con 0x7f17c40b3ba0 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17cb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c415e9c0 con 0x7f17c4154f70 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17c40b3ba0 0x7f17c40ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17c40b3ba0 0x7f17c40ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39646/0 (socket says 172.21.15.87:39646) 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c9000640 1 -- 172.21.15.87:0/708632695 learned_addr learned my addr 172.21.15.87:0/708632695 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:07.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17bb400640 1 --1- 172.21.15.87:0/708632695 >> v1:172.21.15.87:6789/0 conn(0x7f17c40cfea0 0x7f17c40b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45032/0 (socket says 172.21.15.87:45032) 2024-04-25T10:26:07.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c3e00640 1 --2- 172.21.15.87:0/708632695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4154f70 0x7f17c40cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c9000640 1 -- 172.21.15.87:0/708632695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4154f70 msgr2=0x7f17c40cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:07.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c9000640 1 --2- 172.21.15.87:0/708632695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4154f70 0x7f17c40cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c9000640 1 -- 172.21.15.87:0/708632695 >> v1:172.21.15.87:6789/0 conn(0x7f17c40cfea0 legacy=0x7f17c40b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.907+0000 7f17c9000640 1 -- 172.21.15.87:0/708632695 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c415eca0 con 0x7f17c40b3ba0 2024-04-25T10:26:07.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17c9000640 1 --2- 172.21.15.87:0/708632695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c40b3ba0 0x7f17c40ae480 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009e80 tx=0x7f17bc01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded56b259f88e00c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:07.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17c3400640 1 -- 172.21.15.87:0/708632695 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0a56e0 con 0x7f17c40b3ba0 2024-04-25T10:26:07.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17c3400640 1 -- 172.21.15.87:0/708632695 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc0a5880 con 0x7f17c40b3ba0 2024-04-25T10:26:07.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17c3400640 1 -- 172.21.15.87:0/708632695 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0a5c50 con 0x7f17c40b3ba0 2024-04-25T10:26:07.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 -- 172.21.15.87:0/708632695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c40b3ba0 msgr2=0x7f17c40ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 --2- 172.21.15.87:0/708632695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c40b3ba0 0x7f17c40ae480 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009e80 tx=0x7f17bc01ca10 comp rx=0 tx=0).stop 2024-04-25T10:26:07.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 -- 172.21.15.87:0/708632695 shutdown_connections 2024-04-25T10:26:07.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 --2- 172.21.15.87:0/708632695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c40b3ba0 0x7f17c40ae480 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 --2- 172.21.15.87:0/708632695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4154f70 0x7f17c40cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 -- 172.21.15.87:0/708632695 >> 172.21.15.87:0/708632695 conn(0x7f17c4066b70 msgr2=0x7f17c4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:07.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 -- 172.21.15.87:0/708632695 shutdown_connections 2024-04-25T10:26:07.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.911+0000 7f17cb000640 1 -- 172.21.15.87:0/708632695 wait complete. 2024-04-25T10:26:07.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 Processor -- start 2024-04-25T10:26:07.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 -- start start 2024-04-25T10:26:07.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40b3ba0 0x7f17c4152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c40cfea0 0x7f17c414c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c4154f70 0x7f17c4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4071560 con 0x7f17c4154f70 2024-04-25T10:26:07.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40713e0 con 0x7f17c40b3ba0 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40716e0 con 0x7f17c40cfea0 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c4154f70 0x7f17c4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c4154f70 0x7f17c4159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39660/0 (socket says 172.21.15.87:39660) 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 -- 172.21.15.87:0/3198311562 learned_addr learned my addr 172.21.15.87:0/3198311562 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c3e00640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40b3ba0 0x7f17c4152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17bb400640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c40cfea0 0x7f17c414c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 -- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c40cfea0 msgr2=0x7f17c414c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c40cfea0 0x7f17c414c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 -- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40b3ba0 msgr2=0x7f17c4152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40b3ba0 0x7f17c4152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 -- 172.21.15.87:0/3198311562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c41591e0 con 0x7f17c4154f70 2024-04-25T10:26:07.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c9000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c4154f70 0x7f17c4159fa0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f17bc01a640 tx=0x7f17bc0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:07.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c0c00640 1 -- 172.21.15.87:0/3198311562 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0b8670 con 0x7f17c4154f70 2024-04-25T10:26:07.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c0c00640 1 -- 172.21.15.87:0/3198311562 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc0be070 con 0x7f17c4154f70 2024-04-25T10:26:07.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17c0c00640 1 -- 172.21.15.87:0/3198311562 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0b8a50 con 0x7f17c4154f70 2024-04-25T10:26:07.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c41594c0 con 0x7f17c4154f70 2024-04-25T10:26:07.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c4152970 con 0x7f17c4154f70 2024-04-25T10:26:07.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.915+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c404ef30 con 0x7f17c4154f70 2024-04-25T10:26:07.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.919+0000 7f17c0c00640 1 -- 172.21.15.87:0/3198311562 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17bc0b8bf0 con 0x7f17c4154f70 2024-04-25T10:26:07.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.919+0000 7f17c0c00640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17a0066f60 0x7f17a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:07.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.919+0000 7f17c0c00640 1 -- 172.21.15.87:0/3198311562 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f17bc11ffe0 con 0x7f17c4154f70 2024-04-25T10:26:07.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.919+0000 7f17c3e00640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17a0066f60 0x7f17a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:07.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.919+0000 7f17c3e00640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17a0066f60 0x7f17a0069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f17b40079c0 tx=0x7f17b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:07.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:07.923+0000 7f17c0c00640 1 -- 172.21.15.87:0/3198311562 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17bc016600 con 0x7f17c4154f70 2024-04-25T10:26:08.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.075+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f17c41666c0 con 0x7f17c4154f70 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.747+0000 7f17c0c00640 1 -- 172.21.15.87:0/3198311562 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v185) v1 ==== 170+0+0 (secure 0 0 0) 0x7f17bc0f0c70 con 0x7f17c4154f70 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17a0066f60 msgr2=0x7f17a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17a0066f60 0x7f17a0069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f17b40079c0 tx=0x7f17b4008040 comp rx=0 tx=0).stop 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c4154f70 msgr2=0x7f17c4159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c4154f70 0x7f17c4159fa0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f17bc01a640 tx=0x7f17bc0197b0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 shutdown_connections 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f17a0066f60 0x7f17a0069420 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17c4154f70 0x7f17c4159fa0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c40cfea0 0x7f17c414c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 --2- 172.21.15.87:0/3198311562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40b3ba0 0x7f17c4152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 >> 172.21.15.87:0/3198311562 conn(0x7f17c4066b70 msgr2=0x7f17c40afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:08.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 shutdown_connections 2024-04-25T10:26:08.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.751+0000 7f17cb000640 1 -- 172.21.15.87:0/3198311562 wait complete. 2024-04-25T10:26:08.787 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:26:08.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.927+0000 7f3c5fe00640 1 Processor -- start 2024-04-25T10:26:08.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.927+0000 7f3c5fe00640 1 -- start start 2024-04-25T10:26:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.927+0000 7f3c5fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3c6015dae0 0x7f3c600b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.927+0000 7f3c5fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3c6006af40 0x7f3c6006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.927+0000 7f3c5fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c60071470 con 0x7f3c600cc540 2024-04-25T10:26:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.927+0000 7f3c5fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6006b8f0 con 0x7f3c6015dae0 2024-04-25T10:26:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c60071220 con 0x7f3c6006af40 2024-04-25T10:26:08.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3c6015dae0 0x7f3c600b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3c6015dae0 0x7f3c600b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39662/0 (socket says 172.21.15.87:39662) 2024-04-25T10:26:08.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 -- 172.21.15.87:0/3659956731 learned_addr learned my addr 172.21.15.87:0/3659956731 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:08.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5f400640 1 --2- 172.21.15.87:0/3659956731 >> v2:172.21.15.169:3300/0 conn(0x7f3c6006af40 0x7f3c6006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 -- 172.21.15.87:0/3659956731 >> v2:172.21.15.169:3300/0 conn(0x7f3c6006af40 msgr2=0x7f3c6006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:08.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 --2- 172.21.15.87:0/3659956731 >> v2:172.21.15.169:3300/0 conn(0x7f3c6006af40 0x7f3c6006b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 -- 172.21.15.87:0/3659956731 >> v1:172.21.15.169:6789/0 conn(0x7f3c600cc540 legacy=0x7f3c6015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:08.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 -- 172.21.15.87:0/3659956731 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c60071750 con 0x7f3c6015dae0 2024-04-25T10:26:08.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5ea00640 1 --2- 172.21.15.87:0/3659956731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c600b5480 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f3c60162670 tx=0x7f3c5001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e73db8836d4c042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5e000640 1 -- 172.21.15.87:0/3659956731 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500b1070 con 0x7f3c6015dae0 2024-04-25T10:26:08.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5e000640 1 -- 172.21.15.87:0/3659956731 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c500a8050 con 0x7f3c6015dae0 2024-04-25T10:26:08.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.931+0000 7f3c5e000640 1 -- 172.21.15.87:0/3659956731 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500ac860 con 0x7f3c6015dae0 2024-04-25T10:26:08.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- 172.21.15.87:0/3659956731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 msgr2=0x7f3c600b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/3659956731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c600b5480 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f3c60162670 tx=0x7f3c5001c420 comp rx=0 tx=0).stop 2024-04-25T10:26:08.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- 172.21.15.87:0/3659956731 shutdown_connections 2024-04-25T10:26:08.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/3659956731 >> v2:172.21.15.169:3300/0 conn(0x7f3c6006af40 0x7f3c6006b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/3659956731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c600b5480 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- 172.21.15.87:0/3659956731 >> 172.21.15.87:0/3659956731 conn(0x7f3c60073d10 msgr2=0x7f3c60074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:08.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- 172.21.15.87:0/3659956731 shutdown_connections 2024-04-25T10:26:08.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- 172.21.15.87:0/3659956731 wait complete. 2024-04-25T10:26:08.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 Processor -- start 2024-04-25T10:26:08.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- start start 2024-04-25T10:26:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c6006af40 0x7f3c6014dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c600cc540 0x7f3c6014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c60156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600b5b40 con 0x7f3c6015dae0 2024-04-25T10:26:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600b59c0 con 0x7f3c600cc540 2024-04-25T10:26:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60067490 con 0x7f3c6006af40 2024-04-25T10:26:08.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c60156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c6006af40 0x7f3c6014dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c60156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39664/0 (socket says 172.21.15.87:39664) 2024-04-25T10:26:08.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 -- 172.21.15.87:0/1412825841 learned_addr learned my addr 172.21.15.87:0/1412825841 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:08.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c57e00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c600cc540 0x7f3c6014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 -- 172.21.15.87:0/1412825841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c6006af40 msgr2=0x7f3c6014dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c6006af40 0x7f3c6014dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 -- 172.21.15.87:0/1412825841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c600cc540 msgr2=0x7f3c6014c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c600cc540 0x7f3c6014c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.935+0000 7f3c5f400640 1 -- 172.21.15.87:0/1412825841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600ce260 con 0x7f3c6015dae0 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c5f400640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c60156870 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f3c58007e40 tx=0x7f3c5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c56a00640 1 -- 172.21.15.87:0/1412825841 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58014620 con 0x7f3c6015dae0 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c56a00640 1 -- 172.21.15.87:0/1412825841 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58019070 con 0x7f3c6015dae0 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c600b2d60 con 0x7f3c6015dae0 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c56a00640 1 -- 172.21.15.87:0/1412825841 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58014a20 con 0x7f3c6015dae0 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c600ce800 con 0x7f3c6015dae0 2024-04-25T10:26:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c60075120 con 0x7f3c6015dae0 2024-04-25T10:26:08.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.939+0000 7f3c56a00640 1 -- 172.21.15.87:0/1412825841 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c58014bc0 con 0x7f3c6015dae0 2024-04-25T10:26:08.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.943+0000 7f3c56a00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3c3c066f60 0x7f3c3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.943+0000 7f3c56a00640 1 -- 172.21.15.87:0/1412825841 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3c58086480 con 0x7f3c6015dae0 2024-04-25T10:26:08.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.943+0000 7f3c5ea00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3c3c066f60 0x7f3c3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.943+0000 7f3c5ea00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3c3c066f60 0x7f3c3c069420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3c500ae000 tx=0x7f3c500a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:08.947+0000 7f3c56a00640 1 -- 172.21.15.87:0/1412825841 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c58057010 con 0x7f3c6015dae0 2024-04-25T10:26:09.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.131+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3c600cbd30 con 0x7f3c6015dae0 2024-04-25T10:26:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.763+0000 7f3c56a00640 1 -- 172.21.15.87:0/1412825841 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v133) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3c5805b0b0 con 0x7f3c6015dae0 2024-04-25T10:26:09.763 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 28 and data pool 29 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3c3c066f60 msgr2=0x7f3c3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3c3c066f60 0x7f3c3c069420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3c500ae000 tx=0x7f3c500a7000 comp rx=0 tx=0).stop 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 msgr2=0x7f3c60156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c60156870 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f3c58007e40 tx=0x7f3c5800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 shutdown_connections 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3c3c066f60 0x7f3c3c069420 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c6015dae0 0x7f3c60156870 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c600cc540 0x7f3c6014c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 --2- 172.21.15.87:0/1412825841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c6006af40 0x7f3c6014dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 >> 172.21.15.87:0/1412825841 conn(0x7f3c60073d10 msgr2=0x7f3c6015f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:09.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.767+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 shutdown_connections 2024-04-25T10:26:09.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.771+0000 7f3c5fe00640 1 -- 172.21.15.87:0/1412825841 wait complete. 2024-04-25T10:26:09.803 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:09.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 Processor -- start 2024-04-25T10:26:09.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 -- start start 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c900d1820 0x7f6c900d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c9015e050 0x7f6c900b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c900ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c900b1850 con 0x7f6c900d1820 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c900b1af0 con 0x7f6c9015e050 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c97800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c900b1dd0 con 0x7f6c900b13e0 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c900d1820 0x7f6c900d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c900d1820 0x7f6c900d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39680/0 (socket says 172.21.15.87:39680) 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 -- 172.21.15.87:0/3701090116 learned_addr learned my addr 172.21.15.87:0/3701090116 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c8fe00640 1 --2- 172.21.15.87:0/3701090116 >> v2:172.21.15.169:3300/0 conn(0x7f6c9015e050 0x7f6c900b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c95800640 1 --2- 172.21.15.87:0/3701090116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c900ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 -- 172.21.15.87:0/3701090116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 msgr2=0x7f6c900ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:09.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 --2- 172.21.15.87:0/3701090116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c900ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 -- 172.21.15.87:0/3701090116 >> v2:172.21.15.169:3300/0 conn(0x7f6c9015e050 msgr2=0x7f6c900b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:09.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 --2- 172.21.15.87:0/3701090116 >> v2:172.21.15.169:3300/0 conn(0x7f6c9015e050 0x7f6c900b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c94e00640 1 -- 172.21.15.87:0/3701090116 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c900b20b0 con 0x7f6c900d1820 2024-04-25T10:26:09.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.939+0000 7f6c95800640 1 --2- 172.21.15.87:0/3701090116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c900ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:09.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c94e00640 1 --2- 172.21.15.87:0/3701090116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c900d1820 0x7f6c900d1c00 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f6c900c5400 tx=0x7f6c8001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b6cb3abcd74c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:09.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c8f400640 1 -- 172.21.15.87:0/3701090116 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800a55b0 con 0x7f6c900d1820 2024-04-25T10:26:09.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c8f400640 1 -- 172.21.15.87:0/3701090116 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c800a5750 con 0x7f6c900d1820 2024-04-25T10:26:09.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c8f400640 1 -- 172.21.15.87:0/3701090116 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800a5b20 con 0x7f6c900d1820 2024-04-25T10:26:09.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 -- 172.21.15.87:0/3701090116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c900d1820 msgr2=0x7f6c900d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 --2- 172.21.15.87:0/3701090116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c900d1820 0x7f6c900d1c00 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f6c900c5400 tx=0x7f6c8001c730 comp rx=0 tx=0).stop 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 -- 172.21.15.87:0/3701090116 shutdown_connections 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 --2- 172.21.15.87:0/3701090116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c900ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 --2- 172.21.15.87:0/3701090116 >> v2:172.21.15.169:3300/0 conn(0x7f6c9015e050 0x7f6c900b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 --2- 172.21.15.87:0/3701090116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c900d1820 0x7f6c900d1c00 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 -- 172.21.15.87:0/3701090116 >> 172.21.15.87:0/3701090116 conn(0x7f6c9009a9d0 msgr2=0x7f6c9009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 -- 172.21.15.87:0/3701090116 shutdown_connections 2024-04-25T10:26:09.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.943+0000 7f6c97800640 1 -- 172.21.15.87:0/3701090116 wait complete. 2024-04-25T10:26:09.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 Processor -- start 2024-04-25T10:26:09.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 -- start start 2024-04-25T10:26:09.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c90152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c900d1820 0x7f6c900af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9015e050 0x7f6c9015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c900cfdf0 con 0x7f6c9015e050 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c900cfc70 con 0x7f6c900d1820 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c900cff70 con 0x7f6c900b13e0 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9015e050 0x7f6c9015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c90152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c8fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c900d1820 0x7f6c900af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9015e050 0x7f6c9015c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39696/0 (socket says 172.21.15.87:39696) 2024-04-25T10:26:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 -- 172.21.15.87:0/1086130738 learned_addr learned my addr 172.21.15.87:0/1086130738 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 -- 172.21.15.87:0/1086130738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 msgr2=0x7f6c90152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c90152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 -- 172.21.15.87:0/1086130738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c900d1820 msgr2=0x7f6c900af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c900d1820 0x7f6c900af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 -- 172.21.15.87:0/1086130738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c900afb80 con 0x7f6c9015e050 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c95800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9015e050 0x7f6c9015c2f0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c7f0 tx=0x7f6c8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c8cc00640 1 -- 172.21.15.87:0/1086130738 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88015070 con 0x7f6c9015e050 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c8cc00640 1 -- 172.21.15.87:0/1086130738 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88007860 con 0x7f6c9015e050 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c8cc00640 1 -- 172.21.15.87:0/1086130738 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88007c50 con 0x7f6c9015e050 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9015e780 con 0x7f6c9015e050 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9015ed40 con 0x7f6c9015e050 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.947+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9004ef30 con 0x7f6c9015e050 2024-04-25T10:26:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.951+0000 7f6c8cc00640 1 -- 172.21.15.87:0/1086130738 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c88018070 con 0x7f6c9015e050 2024-04-25T10:26:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.951+0000 7f6c8cc00640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c70066f60 0x7f6c70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.951+0000 7f6c8cc00640 1 -- 172.21.15.87:0/1086130738 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c88085190 con 0x7f6c9015e050 2024-04-25T10:26:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.951+0000 7f6c94e00640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c70066f60 0x7f6c70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.951+0000 7f6c94e00640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c70066f60 0x7f6c70069420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6c800ae000 tx=0x7f6c800a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:09.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:09.955+0000 7f6c8cc00640 1 -- 172.21.15.87:0/1086130738 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c88055aa0 con 0x7f6c9015e050 2024-04-25T10:26:10.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.111+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c9006c880 con 0x7f6c9015e050 2024-04-25T10:26:10.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c8cc00640 1 -- 172.21.15.87:0/1086130738 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f6c88059b40 con 0x7f6c9015e050 2024-04-25T10:26:10.113 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:10.113 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":186,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:26:09.136802+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:07.509820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:08.125891+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c70066f60 msgr2=0x7f6c70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c70066f60 0x7f6c70069420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6c800ae000 tx=0x7f6c800a7070 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9015e050 msgr2=0x7f6c9015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9015e050 0x7f6c9015c2f0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c7f0 tx=0x7f6c8800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 shutdown_connections 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6c70066f60 0x7f6c70069420 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9015e050 0x7f6c9015c2f0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c900d1820 0x7f6c900af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 --2- 172.21.15.87:0/1086130738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c900b13e0 0x7f6c90152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 >> 172.21.15.87:0/1086130738 conn(0x7f6c9009a9d0 msgr2=0x7f6c900ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 shutdown_connections 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.115+0000 7f6c97800640 1 -- 172.21.15.87:0/1086130738 wait complete. 2024-04-25T10:26:10.135 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:10.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.271+0000 7fe77bc00640 1 Processor -- start 2024-04-25T10:26:10.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.271+0000 7fe77bc00640 1 -- start start 2024-04-25T10:26:10.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe77bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe77415bc40 0x7fe77406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe77bc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740b2770 con 0x7fe7740cda10 2024-04-25T10:26:10.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe77bc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77406c670 con 0x7fe7740cf930 2024-04-25T10:26:10.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe77bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77406c950 con 0x7fe77415bc40 2024-04-25T10:26:10.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe76be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe7740cda10 0x7fe7740b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45036/0 (socket says 172.21.15.87:45036) 2024-04-25T10:26:10.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe76be00640 1 -- 172.21.15.87:0/1541962756 learned_addr learned my addr 172.21.15.87:0/1541962756 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:10.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe779200640 1 --2- 172.21.15.87:0/1541962756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe77415bc40 0x7fe77406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe773e00640 1 -- 172.21.15.87:0/1541962756 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080100608 0 0) 0x7fe7740b2770 con 0x7fe7740cda10 2024-04-25T10:26:10.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe773e00640 1 -- 172.21.15.87:0/1541962756 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe758003530 con 0x7fe7740cda10 2024-04-25T10:26:10.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe773e00640 1 -- 172.21.15.87:0/1541962756 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3433402803 0 0) 0x7fe758003530 con 0x7fe7740cda10 2024-04-25T10:26:10.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe779200640 1 -- 172.21.15.87:0/1541962756 >> v1:172.21.15.169:6789/0 conn(0x7fe7740cf930 legacy=0x7fe7740b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe779200640 1 -- 172.21.15.87:0/1541962756 >> v1:172.21.15.87:6789/0 conn(0x7fe7740cda10 legacy=0x7fe7740b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.275+0000 7fe779200640 1 -- 172.21.15.87:0/1541962756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77406cc30 con 0x7fe77415bc40 2024-04-25T10:26:10.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe779200640 1 --2- 172.21.15.87:0/1541962756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe77415bc40 0x7fe77406c090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe7640097f0 tx=0x7fe76401c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c126b7daa5494b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe773e00640 1 -- 172.21.15.87:0/1541962756 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640b0070 con 0x7fe77415bc40 2024-04-25T10:26:10.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe773e00640 1 -- 172.21.15.87:0/1541962756 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7640ab400 con 0x7fe77415bc40 2024-04-25T10:26:10.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe773e00640 1 -- 172.21.15.87:0/1541962756 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c001680 con 0x7fe77415bc40 2024-04-25T10:26:10.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 -- 172.21.15.87:0/1541962756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe77415bc40 msgr2=0x7fe77406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 --2- 172.21.15.87:0/1541962756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe77415bc40 0x7fe77406c090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe7640097f0 tx=0x7fe76401c220 comp rx=0 tx=0).stop 2024-04-25T10:26:10.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 -- 172.21.15.87:0/1541962756 shutdown_connections 2024-04-25T10:26:10.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 --2- 172.21.15.87:0/1541962756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe77415bc40 0x7fe77406c090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 -- 172.21.15.87:0/1541962756 >> 172.21.15.87:0/1541962756 conn(0x7fe774066b70 msgr2=0x7fe774066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 -- 172.21.15.87:0/1541962756 shutdown_connections 2024-04-25T10:26:10.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 -- 172.21.15.87:0/1541962756 wait complete. 2024-04-25T10:26:10.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 Processor -- start 2024-04-25T10:26:10.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.279+0000 7fe77bc00640 1 -- start start 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7740cda10 0x7fe7740af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cf930 0x7fe77415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe77415bc40 0x7fe77415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774169d70 con 0x7fe77415bc40 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774169bf0 con 0x7fe7740cf930 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe774169ef0 con 0x7fe7740cda10 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe77415bc40 0x7fe77415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe77415bc40 0x7fe77415e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39698/0 (socket says 172.21.15.87:39698) 2024-04-25T10:26:10.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 -- 172.21.15.87:0/653943321 learned_addr learned my addr 172.21.15.87:0/653943321 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779200640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7740cda10 0x7fe7740af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe76be00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cf930 0x7fe77415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 -- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7740cda10 msgr2=0x7fe7740af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7740cda10 0x7fe7740af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 -- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cf930 msgr2=0x7fe77415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cf930 0x7fe77415c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 -- 172.21.15.87:0/653943321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77415c940 con 0x7fe77415bc40 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe779c00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe77415bc40 0x7fe77415e0f0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c8d0 tx=0x7fe76c00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe771600640 1 -- 172.21.15.87:0/653943321 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c018070 con 0x7fe77415bc40 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe771600640 1 -- 172.21.15.87:0/653943321 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c003db0 con 0x7fe77415bc40 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe771600640 1 -- 172.21.15.87:0/653943321 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c006d80 con 0x7fe77415bc40 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7740afc20 con 0x7fe77415bc40 2024-04-25T10:26:10.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77415eef0 con 0x7fe77415bc40 2024-04-25T10:26:10.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77404ef30 con 0x7fe77415bc40 2024-04-25T10:26:10.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.283+0000 7fe771600640 1 -- 172.21.15.87:0/653943321 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe76c005070 con 0x7fe77415bc40 2024-04-25T10:26:10.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.287+0000 7fe771600640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe750066f60 0x7fe750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.287+0000 7fe771600640 1 -- 172.21.15.87:0/653943321 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe76c084930 con 0x7fe77415bc40 2024-04-25T10:26:10.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.287+0000 7fe779200640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe750066f60 0x7fe750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.287+0000 7fe779200640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe750066f60 0x7fe750069420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe7640ad000 tx=0x7fe7640a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.287+0000 7fe771600640 1 -- 172.21.15.87:0/653943321 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe76c021020 con 0x7fe77415bc40 2024-04-25T10:26:10.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.443+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe77406d960 con 0x7fe77415bc40 2024-04-25T10:26:10.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.443+0000 7fe771600640 1 -- 172.21.15.87:0/653943321 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16680 (secure 0 0 0) 0x7fe76c055460 con 0x7fe77415bc40 2024-04-25T10:26:10.444 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:10.444 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":186,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:26:09.136802+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:07.509820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:08.125891+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:10.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe750066f60 msgr2=0x7fe750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe750066f60 0x7fe750069420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe7640ad000 tx=0x7fe7640a6000 comp rx=0 tx=0).stop 2024-04-25T10:26:10.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe77415bc40 msgr2=0x7fe77415e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe77415bc40 0x7fe77415e0f0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c8d0 tx=0x7fe76c00cd90 comp rx=0 tx=0).stop 2024-04-25T10:26:10.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 shutdown_connections 2024-04-25T10:26:10.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe750066f60 0x7fe750069420 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe77415bc40 0x7fe77415e0f0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cf930 0x7fe77415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 --2- 172.21.15.87:0/653943321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7740cda10 0x7fe7740af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 >> 172.21.15.87:0/653943321 conn(0x7fe774066b70 msgr2=0x7fe774159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 shutdown_connections 2024-04-25T10:26:10.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.447+0000 7fe77bc00640 1 -- 172.21.15.87:0/653943321 wait complete. 2024-04-25T10:26:10.466 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:26:10.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.603+0000 7f0265c00640 1 Processor -- start 2024-04-25T10:26:10.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.603+0000 7f0265c00640 1 -- start start 2024-04-25T10:26:10.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.603+0000 7f0265c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 0x7f02600cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.603+0000 7f0265c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02600ad840 con 0x7f0260071bc0 2024-04-25T10:26:10.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.603+0000 7f0265c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026015e710 con 0x7f02600b3480 2024-04-25T10:26:10.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.603+0000 7f0265c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02600d0090 con 0x7f026015e130 2024-04-25T10:26:10.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0260071bc0 0x7f02600add20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45040/0 (socket says 172.21.15.87:45040) 2024-04-25T10:26:10.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 -- 172.21.15.87:0/1358002677 learned_addr learned my addr 172.21.15.87:0/1358002677 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:10.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f0257e00640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 0x7f02600cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139003900 0 0) 0x7f02600ad840 con 0x7f0260071bc0 2024-04-25T10:26:10.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0244003530 con 0x7f0260071bc0 2024-04-25T10:26:10.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33303684 0 0) 0x7f026015e710 con 0x7f02600b3480 2024-04-25T10:26:10.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02600ad840 con 0x7f02600b3480 2024-04-25T10:26:10.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1921227110 0 0) 0x7f0244003530 con 0x7f0260071bc0 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 msgr2=0x7f02600cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 0x7f02600cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 >> v1:172.21.15.169:6789/0 conn(0x7f02600b3480 legacy=0x7f0260070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026015e9f0 con 0x7f0260071bc0 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0258001a00 con 0x7f0260071bc0 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f0257e00640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 0x7f02600cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 >> v1:172.21.15.87:6789/0 conn(0x7f0260071bc0 legacy=0x7f02600add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 0x7f0244003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440040c0 0x7f0244004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02600d0090 con 0x7f0244004d50 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024400ba80 con 0x7f02440040c0 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024400bc80 con 0x7f026015e130 2024-04-25T10:26:10.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025ea00640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 0x7f0244003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f0257e00640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440040c0 0x7f0244004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 -- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 msgr2=0x7f0244003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026015e130 0x7f0244003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 -- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440040c0 msgr2=0x7f0244004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440040c0 0x7f0244004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.607+0000 7f025f400640 1 -- 172.21.15.87:0/1358002677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0244003530 con 0x7f0244004d50 2024-04-25T10:26:10.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f025f400640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02440051d0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f025800a7c0 tx=0x7f025801fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33201612b2929096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800a410 con 0x7f0244004d50 2024-04-25T10:26:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02580aa690 con 0x7f0244004d50 2024-04-25T10:26:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f025e000640 1 -- 172.21.15.87:0/1358002677 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580aaa80 con 0x7f0244004d50 2024-04-25T10:26:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 -- 172.21.15.87:0/1358002677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 msgr2=0x7f02440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 --2- 172.21.15.87:0/1358002677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02440051d0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f025800a7c0 tx=0x7f025801fa40 comp rx=0 tx=0).stop 2024-04-25T10:26:10.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f025f400640 1 -- 172.21.15.87:0/1358002677 reap_dead start 2024-04-25T10:26:10.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 -- 172.21.15.87:0/1358002677 shutdown_connections 2024-04-25T10:26:10.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 -- 172.21.15.87:0/1358002677 >> 172.21.15.87:0/1358002677 conn(0x7f02600669a0 msgr2=0x7f0260066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 -- 172.21.15.87:0/1358002677 shutdown_connections 2024-04-25T10:26:10.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 -- 172.21.15.87:0/1358002677 wait complete. 2024-04-25T10:26:10.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 Processor -- start 2024-04-25T10:26:10.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.611+0000 7f0265c00640 1 -- start start 2024-04-25T10:26:10.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02601536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026013ecc0 0x7f0260153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260156d60 0x7f02601571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026015efb0 con 0x7f0244004d50 2024-04-25T10:26:10.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026015bb90 con 0x7f0260156d60 2024-04-25T10:26:10.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02600cfb50 con 0x7f026013ecc0 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02601536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02601536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39714/0 (socket says 172.21.15.87:39714) 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 -- 172.21.15.87:0/1541524176 learned_addr learned my addr 172.21.15.87:0/1541524176 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0257e00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026013ecc0 0x7f0260153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025f400640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260156d60 0x7f02601571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 -- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026013ecc0 msgr2=0x7f0260153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026013ecc0 0x7f0260153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 -- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260156d60 msgr2=0x7f02601571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260156d60 0x7f02601571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 -- 172.21.15.87:0/1541524176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02601f33d0 con 0x7f0244004d50 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0257e00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026013ecc0 0x7f0260153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f025ea00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02601536c0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f025000f7d0 tx=0x7f025000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0256a00640 1 -- 172.21.15.87:0/1541524176 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500052e0 con 0x7f0244004d50 2024-04-25T10:26:10.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0256a00640 1 -- 172.21.15.87:0/1541524176 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0250018070 con 0x7f0244004d50 2024-04-25T10:26:10.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0256a00640 1 -- 172.21.15.87:0/1541524176 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250005480 con 0x7f0244004d50 2024-04-25T10:26:10.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0260157ac0 con 0x7f0244004d50 2024-04-25T10:26:10.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0260158080 con 0x7f0244004d50 2024-04-25T10:26:10.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026004ef30 con 0x7f0244004d50 2024-04-25T10:26:10.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.615+0000 7f0256a00640 1 -- 172.21.15.87:0/1541524176 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f025000d950 con 0x7f0244004d50 2024-04-25T10:26:10.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.619+0000 7f0256a00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.619+0000 7f0256a00640 1 -- 172.21.15.87:0/1541524176 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0250084ff0 con 0x7f0244004d50 2024-04-25T10:26:10.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.619+0000 7f0257e00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.619+0000 7f0257e00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f023c066f60 0x7f023c069420 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0260162440 tx=0x7f0248008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.623+0000 7f0256a00640 1 -- 172.21.15.87:0/1541524176 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f025001a310 con 0x7f0244004d50 2024-04-25T10:26:10.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:10.815+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0260071490 con 0x7f0244004d50 2024-04-25T10:26:11.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.771+0000 7f0256a00640 1 -- 172.21.15.87:0/1541524176 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v136) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0250055fa0 con 0x7f0244004d50 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f023c066f60 msgr2=0x7f023c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f023c066f60 0x7f023c069420 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0260162440 tx=0x7f0248008040 comp rx=0 tx=0).stop 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 msgr2=0x7f02601536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02601536c0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f025000f7d0 tx=0x7f025000fc90 comp rx=0 tx=0).stop 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 shutdown_connections 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260156d60 0x7f02601571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026013ecc0 0x7f0260153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 --2- 172.21.15.87:0/1541524176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244004d50 0x7f02601536c0 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 >> 172.21.15.87:0/1541524176 conn(0x7f02600669a0 msgr2=0x7f0260159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 shutdown_connections 2024-04-25T10:26:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.775+0000 7f0265c00640 1 -- 172.21.15.87:0/1541524176 wait complete. 2024-04-25T10:26:11.800 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:26:11.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:26:11.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f8702000640 1 Processor -- start 2024-04-25T10:26:11.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f8702000640 1 -- start start 2024-04-25T10:26:11.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f8702000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86fc15a0e0 0x7f86fc15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f8702000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc0aea50 0x7f86fc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f8702000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc0b2fd0 con 0x7f86fc0b42d0 2024-04-25T10:26:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f8702000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86fc0af1d0 con 0x7f86fc15a0e0 2024-04-25T10:26:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f8702000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0b32b0 con 0x7f86fc0aea50 2024-04-25T10:26:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86fbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f86fc0b42d0 0x7f86fc06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45042/0 (socket says 172.21.15.87:45042) 2024-04-25T10:26:11.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86fbe00640 1 -- 172.21.15.87:0/3934376360 learned_addr learned my addr 172.21.15.87:0/3934376360 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:11.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535703454 0 0) 0x7f86fc0b2fd0 con 0x7f86fc0b42d0 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e0003530 con 0x7f86fc0b42d0 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86fb400640 1 --2- 172.21.15.87:0/3934376360 >> v2:172.21.15.169:3300/0 conn(0x7f86fc15a0e0 0x7f86fc15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86f2a00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc0aea50 0x7f86fc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2367858351 0 0) 0x7f86e0003530 con 0x7f86fc0b42d0 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc0aea50 msgr2=0x7f86fc0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc0aea50 0x7f86fc0b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 >> v2:172.21.15.169:3300/0 conn(0x7f86fc15a0e0 msgr2=0x7f86fc15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 --2- 172.21.15.87:0/3934376360 >> v2:172.21.15.169:3300/0 conn(0x7f86fc15a0e0 0x7f86fc15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0b3590 con 0x7f86fc0b42d0 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.943+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f86f4001680 con 0x7f86fc0b42d0 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 >> v1:172.21.15.87:6789/0 conn(0x7f86fc0b42d0 legacy=0x7f86fc06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc0aea50 0x7f86e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004040 0x7f86e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86e0004d50 0x7f86e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0af1d0 con 0x7f86e0004d50 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0b32b0 con 0x7f86e0004040 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e000bbe0 con 0x7f86fc0aea50 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fb400640 1 --2- 172.21.15.87:0/3934376360 >> v2:172.21.15.169:3300/0 conn(0x7f86fc15a0e0 0x7f86fc15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fbe00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86e0004d50 0x7f86e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fbe00640 1 -- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc0aea50 msgr2=0x7f86e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fbe00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc0aea50 0x7f86e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fbe00640 1 -- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004040 msgr2=0x7f86e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fbe00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004040 0x7f86e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fbe00640 1 -- 172.21.15.87:0/3934376360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0003530 con 0x7f86e0004d50 2024-04-25T10:26:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86fbe00640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86e0004d50 0x7f86e00051d0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c190 tx=0x7f86f401f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e245b8b915ce41b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4007c90 con 0x7f86e0004d50 2024-04-25T10:26:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4005ae0 con 0x7f86e0004d50 2024-04-25T10:26:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.947+0000 7f86faa00640 1 -- 172.21.15.87:0/3934376360 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4007070 con 0x7f86e0004d50 2024-04-25T10:26:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- 172.21.15.87:0/3934376360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86e0004d50 msgr2=0x7f86e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 --2- 172.21.15.87:0/3934376360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86e0004d50 0x7f86e00051d0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c190 tx=0x7f86f401f350 comp rx=0 tx=0).stop 2024-04-25T10:26:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 -- 172.21.15.87:0/3934376360 reap_dead start 2024-04-25T10:26:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- 172.21.15.87:0/3934376360 shutdown_connections 2024-04-25T10:26:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- 172.21.15.87:0/3934376360 >> 172.21.15.87:0/3934376360 conn(0x7f86fc0669a0 msgr2=0x7f86fc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- 172.21.15.87:0/3934376360 shutdown_connections 2024-04-25T10:26:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- 172.21.15.87:0/3934376360 wait complete. 2024-04-25T10:26:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 Processor -- start 2024-04-25T10:26:11.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- start start 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004d50 0x7f86fc153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc143980 0x7f86fc146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86fc1f6c40 0x7f86fc1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc06c7e0 con 0x7f86fc1f6c40 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0aee90 con 0x7f86e0004d50 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f8702000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc06c960 con 0x7f86fc143980 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86fc1f6c40 0x7f86fc1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86fc1f6c40 0x7f86fc1f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39728/0 (socket says 172.21.15.87:39728) 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 -- 172.21.15.87:0/95856986 learned_addr learned my addr 172.21.15.87:0/95856986 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fb400640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004d50 0x7f86fc153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86f2a00640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc143980 0x7f86fc146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 -- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc143980 msgr2=0x7f86fc146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc143980 0x7f86fc146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 -- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004d50 msgr2=0x7f86fc153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004d50 0x7f86fc153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fbe00640 1 -- 172.21.15.87:0/95856986 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc1e76e0 con 0x7f86fc1f6c40 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.951+0000 7f86fb400640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004d50 0x7f86fc153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86fbe00640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86fc1f6c40 0x7f86fc1f70c0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c190 tx=0x7f86e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86f3e00640 1 -- 172.21.15.87:0/95856986 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4005c90 con 0x7f86fc1f6c40 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86f3e00640 1 -- 172.21.15.87:0/95856986 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4005e30 con 0x7f86fc1f6c40 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc1e7160 con 0x7f86fc1f6c40 2024-04-25T10:26:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86f3e00640 1 -- 172.21.15.87:0/95856986 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40049f0 con 0x7f86fc1f6c40 2024-04-25T10:26:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86fc1544a0 con 0x7f86fc1f6c40 2024-04-25T10:26:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86fc146850 con 0x7f86fc1f6c40 2024-04-25T10:26:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86f3e00640 1 -- 172.21.15.87:0/95856986 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86f40ba030 con 0x7f86fc1f6c40 2024-04-25T10:26:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86f3e00640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f60 0x7f86d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86f3e00640 1 -- 172.21.15.87:0/95856986 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86f41219a0 con 0x7f86fc1f6c40 2024-04-25T10:26:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86fb400640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f60 0x7f86d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.955+0000 7f86fb400640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f60 0x7f86d8069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f86ec007440 tx=0x7f86ec00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:11.959+0000 7f86f3e00640 1 -- 172.21.15.87:0/95856986 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86f40c1030 con 0x7f86fc1f6c40 2024-04-25T10:26:12.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.147+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f86fc04ef30 con 0x7f86fc1f6c40 2024-04-25T10:26:12.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.775+0000 7f86f3e00640 1 -- 172.21.15.87:0/95856986 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v137) v1 ==== 122+0+0 (secure 0 0 0) 0x7f86f4019060 con 0x7f86fc1f6c40 2024-04-25T10:26:12.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f60 msgr2=0x7f86d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f60 0x7f86d8069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f86ec007440 tx=0x7f86ec00b040 comp rx=0 tx=0).stop 2024-04-25T10:26:12.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86fc1f6c40 msgr2=0x7f86fc1f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86fc1f6c40 0x7f86fc1f70c0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c190 tx=0x7f86e0005840 comp rx=0 tx=0).stop 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 shutdown_connections 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f86d8066f60 0x7f86d8069420 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86fc1f6c40 0x7f86fc1f70c0 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86fc143980 0x7f86fc146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 --2- 172.21.15.87:0/95856986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e0004d50 0x7f86fc153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 >> 172.21.15.87:0/95856986 conn(0x7f86fc0669a0 msgr2=0x7f86fc15b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 shutdown_connections 2024-04-25T10:26:12.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.779+0000 7f8702000640 1 -- 172.21.15.87:0/95856986 wait complete. 2024-04-25T10:26:12.799 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:26:12.799 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:12.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.935+0000 7fb865600640 1 Processor -- start 2024-04-25T10:26:12.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.935+0000 7fb865600640 1 -- start start 2024-04-25T10:26:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb865600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb86006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb865600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8600b2770 con 0x7fb8600cf930 2024-04-25T10:26:12.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb865600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86006c670 con 0x7fb86015bc40 2024-04-25T10:26:12.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb865600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86006c950 con 0x7fb8600cda10 2024-04-25T10:26:12.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb85f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb8600cf930 0x7fb8600b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45054/0 (socket says 172.21.15.87:45054) 2024-04-25T10:26:12.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb85f400640 1 -- 172.21.15.87:0/3402047200 learned_addr learned my addr 172.21.15.87:0/3402047200 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb857e00640 1 --2- 172.21.15.87:0/3402047200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb86006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163581643 0 0) 0x7fb8600b2770 con 0x7fb8600cf930 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb844003530 con 0x7fb8600cf930 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291631453 0 0) 0x7fb86006c670 con 0x7fb86015bc40 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8600b2770 con 0x7fb86015bc40 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb857e00640 1 -- 172.21.15.87:0/3402047200 >> v1:172.21.15.169:6789/0 conn(0x7fb86015bc40 legacy=0x7fb8600b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb857e00640 1 -- 172.21.15.87:0/3402047200 >> v1:172.21.15.87:6789/0 conn(0x7fb8600cf930 legacy=0x7fb8600b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234086936 0 0) 0x7fb844003530 con 0x7fb8600cf930 2024-04-25T10:26:12.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.939+0000 7fb857e00640 1 -- 172.21.15.87:0/3402047200 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86006cc30 con 0x7fb8600cda10 2024-04-25T10:26:12.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb857e00640 1 --2- 172.21.15.87:0/3402047200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb86006c090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb8480097f0 tx=0x7fb84801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd5dc6a3ba00abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:12.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480b0070 con 0x7fb8600cda10 2024-04-25T10:26:12.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8480ab400 con 0x7fb8600cda10 2024-04-25T10:26:12.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb85e000640 1 -- 172.21.15.87:0/3402047200 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858001a00 con 0x7fb8600cda10 2024-04-25T10:26:12.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 -- 172.21.15.87:0/3402047200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 msgr2=0x7fb86006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 --2- 172.21.15.87:0/3402047200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb86006c090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb8480097f0 tx=0x7fb84801c370 comp rx=0 tx=0).stop 2024-04-25T10:26:12.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 -- 172.21.15.87:0/3402047200 shutdown_connections 2024-04-25T10:26:12.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 --2- 172.21.15.87:0/3402047200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb86006c090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 -- 172.21.15.87:0/3402047200 >> 172.21.15.87:0/3402047200 conn(0x7fb860066b70 msgr2=0x7fb860066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:12.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 -- 172.21.15.87:0/3402047200 shutdown_connections 2024-04-25T10:26:12.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 -- 172.21.15.87:0/3402047200 wait complete. 2024-04-25T10:26:12.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 Processor -- start 2024-04-25T10:26:12.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.943+0000 7fb865600640 1 -- start start 2024-04-25T10:26:12.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb860158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8600cf930 0x7fb86014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86015bc40 0x7fb86015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860169d70 con 0x7fb8600cf930 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860169bf0 con 0x7fb86015bc40 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb860169ef0 con 0x7fb8600cda10 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8600cf930 0x7fb86014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8600cf930 0x7fb86014f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39730/0 (socket says 172.21.15.87:39730) 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 -- 172.21.15.87:0/2459037810 learned_addr learned my addr 172.21.15.87:0/2459037810 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb85f400640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86015bc40 0x7fb86015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 -- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 msgr2=0x7fb860158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb85ea00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb860158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb860158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 -- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86015bc40 msgr2=0x7fb86015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86015bc40 0x7fb86015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 -- 172.21.15.87:0/2459037810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8601566d0 con 0x7fb8600cf930 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb85ea00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb860158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb857e00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8600cf930 0x7fb86014f7f0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fb8480097f0 tx=0x7fb848019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb856a00640 1 -- 172.21.15.87:0/2459037810 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480b8070 con 0x7fb8600cf930 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb856a00640 1 -- 172.21.15.87:0/2459037810 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb848004020 con 0x7fb8600cf930 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb860156a10 con 0x7fb8600cf930 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb856a00640 1 -- 172.21.15.87:0/2459037810 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480b0040 con 0x7fb8600cf930 2024-04-25T10:26:12.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb860157250 con 0x7fb8600cf930 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86004ef30 con 0x7fb8600cf930 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.947+0000 7fb856a00640 1 -- 172.21.15.87:0/2459037810 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb848003620 con 0x7fb8600cf930 2024-04-25T10:26:12.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.951+0000 7fb856a00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb83c066f60 0x7fb83c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.951+0000 7fb856a00640 1 -- 172.21.15.87:0/2459037810 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb84811eb70 con 0x7fb8600cf930 2024-04-25T10:26:12.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.951+0000 7fb85ea00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb83c066f60 0x7fb83c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.951+0000 7fb85ea00640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb83c066f60 0x7fb83c069420 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb8500011f0 tx=0x7fb850009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:12.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:12.955+0000 7fb856a00640 1 -- 172.21.15.87:0/2459037810 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb860156a10 con 0x7fb8600cf930 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.139+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb86006d960 con 0x7fb8600cf930 2024-04-25T10:26:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb856a00640 1 -- 172.21.15.87:0/2459037810 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4955 (secure 0 0 0) 0x7fb848015ff0 con 0x7fb8600cf930 2024-04-25T10:26:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:13.141 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":137,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:12.151950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11304":{"gid":11304,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/997618220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":997618220},{"type":"v1","addr":"172.21.15.87:6836","nonce":997618220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:26:13.141 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 137 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb83c066f60 msgr2=0x7fb83c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb83c066f60 0x7fb83c069420 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb8500011f0 tx=0x7fb850009040 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8600cf930 msgr2=0x7fb86014f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8600cf930 0x7fb86014f7f0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fb8480097f0 tx=0x7fb848019110 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 shutdown_connections 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb83c066f60 0x7fb83c069420 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86015bc40 0x7fb86015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8600cf930 0x7fb86014f7f0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 --2- 172.21.15.87:0/2459037810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8600cda10 0x7fb860158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 >> 172.21.15.87:0/2459037810 conn(0x7fb860066b70 msgr2=0x7fb860153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 shutdown_connections 2024-04-25T10:26:13.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.143+0000 7fb865600640 1 -- 172.21.15.87:0/2459037810 wait complete. 2024-04-25T10:26:13.162 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6b200640 1 Processor -- start 2024-04-25T10:26:13.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6b200640 1 -- start start 2024-04-25T10:26:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a6c0670b0 0x7f1a6c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1a6c0b1ec0 0x7f1a6c0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0b5850 con 0x7f1a6c15dca0 2024-04-25T10:26:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b5af0 con 0x7f1a6c0670b0 2024-04-25T10:26:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b5dd0 con 0x7f1a6c0b1ec0 2024-04-25T10:26:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a69400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a6c0670b0 0x7f1a6c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a69e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1a6c15dca0 0x7f1a6c0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45058/0 (socket says 172.21.15.87:45058) 2024-04-25T10:26:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a69e00640 1 -- 172.21.15.87:0/434379413 learned_addr learned my addr 172.21.15.87:0/434379413 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a6a800640 1 --2- 172.21.15.87:0/434379413 >> v2:172.21.15.169:3300/0 conn(0x7f1a6c0b1ec0 0x7f1a6c0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a69400640 1 -- 172.21.15.87:0/434379413 >> v2:172.21.15.169:3300/0 conn(0x7f1a6c0b1ec0 msgr2=0x7f1a6c0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:13.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a69400640 1 --2- 172.21.15.87:0/434379413 >> v2:172.21.15.169:3300/0 conn(0x7f1a6c0b1ec0 0x7f1a6c0cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a69400640 1 -- 172.21.15.87:0/434379413 >> v1:172.21.15.87:6789/0 conn(0x7f1a6c15dca0 legacy=0x7f1a6c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a69400640 1 -- 172.21.15.87:0/434379413 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0b60b0 con 0x7f1a6c0670b0 2024-04-25T10:26:13.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.299+0000 7f1a68a00640 1 -- 172.21.15.87:0/434379413 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40707795 0 0) 0x7f1a6c0b5850 con 0x7f1a6c15dca0 2024-04-25T10:26:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a69400640 1 --2- 172.21.15.87:0/434379413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c0670b0 0x7f1a6c071a40 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0b6980 tx=0x7f1a5801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beec568e20a7bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a68a00640 1 -- 172.21.15.87:0/434379413 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a580b1070 con 0x7f1a6c0670b0 2024-04-25T10:26:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a68a00640 1 -- 172.21.15.87:0/434379413 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a580a8050 con 0x7f1a6c0670b0 2024-04-25T10:26:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a68a00640 1 -- 172.21.15.87:0/434379413 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a580ac860 con 0x7f1a6c0670b0 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 -- 172.21.15.87:0/434379413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c0670b0 msgr2=0x7f1a6c071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 --2- 172.21.15.87:0/434379413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c0670b0 0x7f1a6c071a40 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0b6980 tx=0x7f1a5801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 -- 172.21.15.87:0/434379413 shutdown_connections 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 --2- 172.21.15.87:0/434379413 >> v2:172.21.15.169:3300/0 conn(0x7f1a6c0b1ec0 0x7f1a6c0cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 --2- 172.21.15.87:0/434379413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c0670b0 0x7f1a6c071a40 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 -- 172.21.15.87:0/434379413 >> 172.21.15.87:0/434379413 conn(0x7f1a6c0b2440 msgr2=0x7f1a6c0b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 -- 172.21.15.87:0/434379413 shutdown_connections 2024-04-25T10:26:13.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.303+0000 7f1a6b200640 1 -- 172.21.15.87:0/434379413 wait complete. 2024-04-25T10:26:13.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 Processor -- start 2024-04-25T10:26:13.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 -- start start 2024-04-25T10:26:13.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c0670b0 0x7f1a6c160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0b1ec0 0x7f1a6c15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c15dca0 0x7f1a6c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cfb60 con 0x7f1a6c15dca0 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cf9e0 con 0x7f1a6c0b1ec0 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cfce0 con 0x7f1a6c0670b0 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c15dca0 0x7f1a6c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c15dca0 0x7f1a6c15cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39738/0 (socket says 172.21.15.87:39738) 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 -- 172.21.15.87:0/4137676186 learned_addr learned my addr 172.21.15.87:0/4137676186 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a69e00640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c0670b0 0x7f1a6c160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a69400640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0b1ec0 0x7f1a6c15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 -- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c0670b0 msgr2=0x7f1a6c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c0670b0 0x7f1a6c160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 -- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0b1ec0 msgr2=0x7f1a6c15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0b1ec0 0x7f1a6c15c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 -- 172.21.15.87:0/4137676186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c14ee30 con 0x7f1a6c15dca0 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a69400640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0b1ec0 0x7f1a6c15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6a800640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c15dca0 0x7f1a6c15cd90 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1a64007d20 tx=0x7f1a6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a62000640 1 -- 172.21.15.87:0/4137676186 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64014420 con 0x7f1a6c15dca0 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a62000640 1 -- 172.21.15.87:0/4137676186 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64019070 con 0x7f1a6c15dca0 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a62000640 1 -- 172.21.15.87:0/4137676186 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64014820 con 0x7f1a6c15dca0 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c14f110 con 0x7f1a6c15dca0 2024-04-25T10:26:13.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c14fbf0 con 0x7f1a6c15dca0 2024-04-25T10:26:13.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.307+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c075120 con 0x7f1a6c15dca0 2024-04-25T10:26:13.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.311+0000 7f1a62000640 1 -- 172.21.15.87:0/4137676186 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a640149c0 con 0x7f1a6c15dca0 2024-04-25T10:26:13.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.311+0000 7f1a62000640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a4c066f90 0x7f1a4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.311+0000 7f1a62000640 1 -- 172.21.15.87:0/4137676186 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a64085900 con 0x7f1a6c15dca0 2024-04-25T10:26:13.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.311+0000 7f1a69e00640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a4c066f90 0x7f1a4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.311+0000 7f1a69e00640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a4c066f90 0x7f1a4c069450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c007fd0 tx=0x7f1a5c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.315+0000 7f1a62000640 1 -- 172.21.15.87:0/4137676186 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a64056870 con 0x7f1a6c15dca0 2024-04-25T10:26:13.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.471+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a6c1550b0 con 0x7f1a6c15dca0 2024-04-25T10:26:13.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.471+0000 7f1a62000640 1 -- 172.21.15.87:0/4137676186 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f1a64085090 con 0x7f1a6c15dca0 2024-04-25T10:26:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:13.471 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":187,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:26:10.753704+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:07.509820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:08.125891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a4c066f90 msgr2=0x7f1a4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a4c066f90 0x7f1a4c069450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c007fd0 tx=0x7f1a5c009040 comp rx=0 tx=0).stop 2024-04-25T10:26:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c15dca0 msgr2=0x7f1a6c15cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c15dca0 0x7f1a6c15cd90 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1a64007d20 tx=0x7f1a6400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 shutdown_connections 2024-04-25T10:26:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1a4c066f90 0x7f1a4c069450 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a6c15dca0 0x7f1a6c15cd90 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0b1ec0 0x7f1a6c15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 --2- 172.21.15.87:0/4137676186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c0670b0 0x7f1a6c160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 >> 172.21.15.87:0/4137676186 conn(0x7f1a6c0b2440 msgr2=0x7f1a6c1599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:13.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 shutdown_connections 2024-04-25T10:26:13.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.475+0000 7f1a6b200640 1 -- 172.21.15.87:0/4137676186 wait complete. 2024-04-25T10:26:13.493 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:26:13.493 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:26:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.627+0000 7ffae5400640 1 Processor -- start 2024-04-25T10:26:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.627+0000 7ffae5400640 1 -- start start 2024-04-25T10:26:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffae5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffae00cda10 0x7ffae006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffae5400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae00b2770 con 0x7ffae00cf930 2024-04-25T10:26:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffae5400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae006c670 con 0x7ffae015bc40 2024-04-25T10:26:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffae5400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae006c950 con 0x7ffae00cda10 2024-04-25T10:26:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadf200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ffae00cf930 0x7ffae00b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45072/0 (socket says 172.21.15.87:45072) 2024-04-25T10:26:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadf200640 1 -- 172.21.15.87:0/3275510949 learned_addr learned my addr 172.21.15.87:0/3275510949 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadde00640 1 --2- 172.21.15.87:0/3275510949 >> v2:172.21.15.169:3300/0 conn(0x7ffae00cda10 0x7ffae006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937103525 0 0) 0x7ffae00b2770 con 0x7ffae00cf930 2024-04-25T10:26:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffac4003530 con 0x7ffae00cf930 2024-04-25T10:26:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686995508 0 0) 0x7ffae006c670 con 0x7ffae015bc40 2024-04-25T10:26:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae00b2770 con 0x7ffae015bc40 2024-04-25T10:26:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 510732276 0 0) 0x7ffac4003530 con 0x7ffae00cf930 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 >> v2:172.21.15.169:3300/0 conn(0x7ffae00cda10 msgr2=0x7ffae006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 --2- 172.21.15.87:0/3275510949 >> v2:172.21.15.169:3300/0 conn(0x7ffae00cda10 0x7ffae006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 >> v1:172.21.15.169:6789/0 conn(0x7ffae015bc40 legacy=0x7ffae00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae006cc30 con 0x7ffae00cf930 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7ffad8001680 con 0x7ffae00cf930 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 >> v1:172.21.15.87:6789/0 conn(0x7ffae00cf930 legacy=0x7ffae00b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4003ba0 0x7ffac4003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4004810 0x7ffac4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac4005220 0x7ffac400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae006c950 con 0x7ffac4004810 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac400c150 con 0x7ffac4005220 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac400c350 con 0x7ffac4003ba0 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadde00640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4004810 0x7ffac4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffade800640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4003ba0 0x7ffac4003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadf200640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac4005220 0x7ffac400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadde00640 1 -- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4003ba0 msgr2=0x7ffac4003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadde00640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4003ba0 0x7ffac4003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadde00640 1 -- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac4005220 msgr2=0x7ffac400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadde00640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac4005220 0x7ffac400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadde00640 1 -- 172.21.15.87:0/3275510949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac4003530 con 0x7ffac4004810 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadf200640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac4005220 0x7ffac400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.631+0000 7ffadf200640 1 -- 172.21.15.87:0/3275510949 reap_dead start 2024-04-25T10:26:13.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffadde00640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4004810 0x7ffac4004c90 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009ae0 tx=0x7ffacc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fafa31b693d90c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0096b0 con 0x7ffac4004810 2024-04-25T10:26:13.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc0a6450 con 0x7ffac4004810 2024-04-25T10:26:13.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffadd400640 1 -- 172.21.15.87:0/3275510949 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0a6840 con 0x7ffac4004810 2024-04-25T10:26:13.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffae5400640 1 -- 172.21.15.87:0/3275510949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4004810 msgr2=0x7ffac4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffae5400640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4004810 0x7ffac4004c90 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009ae0 tx=0x7ffacc01ca70 comp rx=0 tx=0).stop 2024-04-25T10:26:13.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffae5400640 1 -- 172.21.15.87:0/3275510949 shutdown_connections 2024-04-25T10:26:13.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffae5400640 1 --2- 172.21.15.87:0/3275510949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4004810 0x7ffac4004c90 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffae5400640 1 -- 172.21.15.87:0/3275510949 >> 172.21.15.87:0/3275510949 conn(0x7ffae0066b70 msgr2=0x7ffae0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:13.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffae5400640 1 -- 172.21.15.87:0/3275510949 shutdown_connections 2024-04-25T10:26:13.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.635+0000 7ffae5400640 1 -- 172.21.15.87:0/3275510949 wait complete. 2024-04-25T10:26:13.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 Processor -- start 2024-04-25T10:26:13.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 -- start start 2024-04-25T10:26:13.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4004810 0x7ffae01f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffae014a860 0x7ffae0147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffae0159e80 0x7ffae015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00d00d0 con 0x7ffae0159e80 2024-04-25T10:26:13.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00cff50 con 0x7ffae014a860 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00d0250 con 0x7ffac4004810 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffae0159e80 0x7ffae015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffae0159e80 0x7ffae015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39748/0 (socket says 172.21.15.87:39748) 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 -- 172.21.15.87:0/3518489214 learned_addr learned my addr 172.21.15.87:0/3518489214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffade800640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4004810 0x7ffae01f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadde00640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffae014a860 0x7ffae0147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 -- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4004810 msgr2=0x7ffae01f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4004810 0x7ffae01f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 -- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffae014a860 msgr2=0x7ffae0147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffae014a860 0x7ffae0147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 -- 172.21.15.87:0/3518489214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae0147e60 con 0x7ffae0159e80 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffade800640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4004810 0x7ffae01f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadde00640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffae014a860 0x7ffae0147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:13.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffadf200640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffae0159e80 0x7ffae015a300 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7ffad8001ba0 tx=0x7ffad800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffad6a00640 1 -- 172.21.15.87:0/3518489214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8019070 con 0x7ffae0159e80 2024-04-25T10:26:13.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffad6a00640 1 -- 172.21.15.87:0/3518489214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8010510 con 0x7ffae0159e80 2024-04-25T10:26:13.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffad6a00640 1 -- 172.21.15.87:0/3518489214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80108e0 con 0x7ffae0159e80 2024-04-25T10:26:13.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae01f7c10 con 0x7ffae0159e80 2024-04-25T10:26:13.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae01f81d0 con 0x7ffae0159e80 2024-04-25T10:26:13.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.639+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae004ef30 con 0x7ffae0159e80 2024-04-25T10:26:13.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.643+0000 7ffad6a00640 1 -- 172.21.15.87:0/3518489214 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffad8010a80 con 0x7ffae0159e80 2024-04-25T10:26:13.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.643+0000 7ffad6a00640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffac0066f60 0x7ffac0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.643+0000 7ffad6a00640 1 -- 172.21.15.87:0/3518489214 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffad8084590 con 0x7ffae0159e80 2024-04-25T10:26:13.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.643+0000 7ffade800640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffac0066f60 0x7ffac0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.643+0000 7ffade800640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffac0066f60 0x7ffac0069420 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffad0007fd0 tx=0x7ffad0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.647+0000 7ffad6a00640 1 -- 172.21.15.87:0/3518489214 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffad8055590 con 0x7ffae0159e80 2024-04-25T10:26:13.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.799+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffae0155880 con 0x7ffac0066f60 2024-04-25T10:26:13.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.807+0000 7ffad6a00640 1 -- 172.21.15.87:0/3518489214 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252459 (secure 0 0 0) 0x7ffae0155880 con 0x7ffac0066f60 2024-04-25T10:26:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:13.811 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:26:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffac0066f60 msgr2=0x7ffac0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffac0066f60 0x7ffac0069420 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffad0007fd0 tx=0x7ffad0009040 comp rx=0 tx=0).stop 2024-04-25T10:26:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffae0159e80 msgr2=0x7ffae015a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffae0159e80 0x7ffae015a300 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7ffad8001ba0 tx=0x7ffad800e960 comp rx=0 tx=0).stop 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 shutdown_connections 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ffac0066f60 0x7ffac0069420 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffae0159e80 0x7ffae015a300 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffae014a860 0x7ffae0147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 --2- 172.21.15.87:0/3518489214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac4004810 0x7ffae01f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 >> 172.21.15.87:0/3518489214 conn(0x7ffae0066b70 msgr2=0x7ffae00af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 shutdown_connections 2024-04-25T10:26:13.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:13.811+0000 7ffae5400640 1 -- 172.21.15.87:0/3518489214 wait complete. 2024-04-25T10:26:13.829 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2024-04-25T10:26:13.302164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":486960,"kb_used_data":19628,"kb_used_omap":12,"kb_used_meta":467187,"kb_avail":754487760,"statfs":{"total":773094113280,"available":772595466240,"internally_reserved":0,"allocated":20099072,"data_stored":9464840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":478400049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.558564"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778251+0000","last_change":"2024-04-25T10:26:09.789967+0000","last_active":"2024-04-25T10:26:10.778251+0000","last_peered":"2024-04-25T10:26:10.778251+0000","last_clean":"2024-04-25T10:26:10.778251+0000","last_became_active":"2024-04-25T10:26:09.785061+0000","last_became_peered":"2024-04-25T10:26:09.785061+0000","last_unstale":"2024-04-25T10:26:10.778251+0000","last_undegraded":"2024-04-25T10:26:10.778251+0000","last_fullsized":"2024-04-25T10:26:10.778251+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.581572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760412+0000","last_change":"2024-04-25T10:26:09.789548+0000","last_active":"2024-04-25T10:26:10.760412+0000","last_peered":"2024-04-25T10:26:10.760412+0000","last_clean":"2024-04-25T10:26:10.760412+0000","last_became_active":"2024-04-25T10:26:09.789266+0000","last_became_peered":"2024-04-25T10:26:09.789266+0000","last_unstale":"2024-04-25T10:26:10.760412+0000","last_undegraded":"2024-04-25T10:26:10.760412+0000","last_fullsized":"2024-04-25T10:26:10.760412+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.884230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792908+0000","last_change":"2024-04-25T10:26:09.786079+0000","last_active":"2024-04-25T10:26:09.792908+0000","last_peered":"2024-04-25T10:26:09.792908+0000","last_clean":"2024-04-25T10:26:09.792908+0000","last_became_active":"2024-04-25T10:26:09.785787+0000","last_became_peered":"2024-04-25T10:26:09.785787+0000","last_unstale":"2024-04-25T10:26:09.792908+0000","last_undegraded":"2024-04-25T10:26:09.792908+0000","last_fullsized":"2024-04-25T10:26:09.792908+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.779063+0000","last_change":"2024-04-25T10:26:09.778035+0000","last_active":"2024-04-25T10:26:09.779063+0000","last_peered":"2024-04-25T10:26:09.779063+0000","last_clean":"2024-04-25T10:26:09.779063+0000","last_became_active":"2024-04-25T10:26:09.777780+0000","last_became_peered":"2024-04-25T10:26:09.777780+0000","last_unstale":"2024-04-25T10:26:09.779063+0000","last_undegraded":"2024-04-25T10:26:09.779063+0000","last_fullsized":"2024-04-25T10:26:09.779063+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.766411+0000","last_change":"2024-04-25T10:26:09.765034+0000","last_active":"2024-04-25T10:26:09.766411+0000","last_peered":"2024-04-25T10:26:09.766411+0000","last_clean":"2024-04-25T10:26:09.766411+0000","last_became_active":"2024-04-25T10:26:09.763888+0000","last_became_peered":"2024-04-25T10:26:09.763888+0000","last_unstale":"2024-04-25T10:26:09.766411+0000","last_undegraded":"2024-04-25T10:26:09.766411+0000","last_fullsized":"2024-04-25T10:26:09.766411+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:24.976889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758117+0000","last_change":"2024-04-25T10:26:09.765280+0000","last_active":"2024-04-25T10:26:10.758117+0000","last_peered":"2024-04-25T10:26:10.758117+0000","last_clean":"2024-04-25T10:26:10.758117+0000","last_became_active":"2024-04-25T10:26:09.764961+0000","last_became_peered":"2024-04-25T10:26:09.764961+0000","last_unstale":"2024-04-25T10:26:10.758117+0000","last_undegraded":"2024-04-25T10:26:10.758117+0000","last_fullsized":"2024-04-25T10:26:10.758117+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:06.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792795+0000","last_change":"2024-04-25T10:26:09.787540+0000","last_active":"2024-04-25T10:26:09.792795+0000","last_peered":"2024-04-25T10:26:09.792795+0000","last_clean":"2024-04-25T10:26:09.792795+0000","last_became_active":"2024-04-25T10:26:09.785656+0000","last_became_peered":"2024-04-25T10:26:09.785656+0000","last_unstale":"2024-04-25T10:26:09.792795+0000","last_undegraded":"2024-04-25T10:26:09.792795+0000","last_fullsized":"2024-04-25T10:26:09.792795+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:43.146461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779409+0000","last_change":"2024-04-25T10:26:09.778885+0000","last_active":"2024-04-25T10:26:10.779409+0000","last_peered":"2024-04-25T10:26:10.779409+0000","last_clean":"2024-04-25T10:26:10.779409+0000","last_became_active":"2024-04-25T10:26:09.778541+0000","last_became_peered":"2024-04-25T10:26:09.778541+0000","last_unstale":"2024-04-25T10:26:10.779409+0000","last_undegraded":"2024-04-25T10:26:10.779409+0000","last_fullsized":"2024-04-25T10:26:10.779409+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:09.103513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796770+0000","last_change":"2024-04-25T10:26:09.795181+0000","last_active":"2024-04-25T10:26:09.796770+0000","last_peered":"2024-04-25T10:26:09.796770+0000","last_clean":"2024-04-25T10:26:09.796770+0000","last_became_active":"2024-04-25T10:26:09.794687+0000","last_became_peered":"2024-04-25T10:26:09.794687+0000","last_unstale":"2024-04-25T10:26:09.796770+0000","last_undegraded":"2024-04-25T10:26:09.796770+0000","last_fullsized":"2024-04-25T10:26:09.796770+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:42.547135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758363+0000","last_change":"2024-04-25T10:26:09.762737+0000","last_active":"2024-04-25T10:26:10.758363+0000","last_peered":"2024-04-25T10:26:10.758363+0000","last_clean":"2024-04-25T10:26:10.758363+0000","last_became_active":"2024-04-25T10:26:09.762018+0000","last_became_peered":"2024-04-25T10:26:09.762018+0000","last_unstale":"2024-04-25T10:26:10.758363+0000","last_undegraded":"2024-04-25T10:26:10.758363+0000","last_fullsized":"2024-04-25T10:26:10.758363+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:58.320915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778164+0000","last_change":"2024-04-25T10:26:09.782899+0000","last_active":"2024-04-25T10:26:10.778164+0000","last_peered":"2024-04-25T10:26:10.778164+0000","last_clean":"2024-04-25T10:26:10.778164+0000","last_became_active":"2024-04-25T10:26:09.781870+0000","last_became_peered":"2024-04-25T10:26:09.781870+0000","last_unstale":"2024-04-25T10:26:10.778164+0000","last_undegraded":"2024-04-25T10:26:10.778164+0000","last_fullsized":"2024-04-25T10:26:10.778164+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:37.116951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760337+0000","last_change":"2024-04-25T10:26:09.782249+0000","last_active":"2024-04-25T10:26:10.760337+0000","last_peered":"2024-04-25T10:26:10.760337+0000","last_clean":"2024-04-25T10:26:10.760337+0000","last_became_active":"2024-04-25T10:26:09.781962+0000","last_became_peered":"2024-04-25T10:26:09.781962+0000","last_unstale":"2024-04-25T10:26:10.760337+0000","last_undegraded":"2024-04-25T10:26:10.760337+0000","last_fullsized":"2024-04-25T10:26:10.760337+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.040552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760094+0000","last_change":"2024-04-25T10:26:09.787262+0000","last_active":"2024-04-25T10:26:10.760094+0000","last_peered":"2024-04-25T10:26:10.760094+0000","last_clean":"2024-04-25T10:26:10.760094+0000","last_became_active":"2024-04-25T10:26:09.786838+0000","last_became_peered":"2024-04-25T10:26:09.786838+0000","last_unstale":"2024-04-25T10:26:10.760094+0000","last_undegraded":"2024-04-25T10:26:10.760094+0000","last_fullsized":"2024-04-25T10:26:10.760094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:00.455133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760654+0000","last_change":"2024-04-25T10:26:09.769261+0000","last_active":"2024-04-25T10:26:10.760654+0000","last_peered":"2024-04-25T10:26:10.760654+0000","last_clean":"2024-04-25T10:26:10.760654+0000","last_became_active":"2024-04-25T10:26:09.768281+0000","last_became_peered":"2024-04-25T10:26:09.768281+0000","last_unstale":"2024-04-25T10:26:10.760654+0000","last_undegraded":"2024-04-25T10:26:10.760654+0000","last_fullsized":"2024-04-25T10:26:10.760654+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:30.066167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.772288+0000","last_change":"2024-04-25T10:26:09.770663+0000","last_active":"2024-04-25T10:26:09.772288+0000","last_peered":"2024-04-25T10:26:09.772288+0000","last_clean":"2024-04-25T10:26:09.772288+0000","last_became_active":"2024-04-25T10:26:09.770410+0000","last_became_peered":"2024-04-25T10:26:09.770410+0000","last_unstale":"2024-04-25T10:26:09.772288+0000","last_undegraded":"2024-04-25T10:26:09.772288+0000","last_fullsized":"2024-04-25T10:26:09.772288+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:58.878870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760213+0000","last_change":"2024-04-25T10:26:09.781858+0000","last_active":"2024-04-25T10:26:10.760213+0000","last_peered":"2024-04-25T10:26:10.760213+0000","last_clean":"2024-04-25T10:26:10.760213+0000","last_became_active":"2024-04-25T10:26:09.781358+0000","last_became_peered":"2024-04-25T10:26:09.781358+0000","last_unstale":"2024-04-25T10:26:10.760213+0000","last_undegraded":"2024-04-25T10:26:10.760213+0000","last_fullsized":"2024-04-25T10:26:10.760213+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:24.976889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761086+0000","last_change":"2024-04-25T10:26:09.767788+0000","last_active":"2024-04-25T10:26:10.761086+0000","last_peered":"2024-04-25T10:26:10.761086+0000","last_clean":"2024-04-25T10:26:10.761086+0000","last_became_active":"2024-04-25T10:26:09.767509+0000","last_became_peered":"2024-04-25T10:26:09.767509+0000","last_unstale":"2024-04-25T10:26:10.761086+0000","last_undegraded":"2024-04-25T10:26:10.761086+0000","last_fullsized":"2024-04-25T10:26:10.761086+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:18.288245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758577+0000","last_change":"2024-04-25T10:26:09.776387+0000","last_active":"2024-04-25T10:26:10.758577+0000","last_peered":"2024-04-25T10:26:10.758577+0000","last_clean":"2024-04-25T10:26:10.758577+0000","last_became_active":"2024-04-25T10:26:09.776083+0000","last_became_peered":"2024-04-25T10:26:09.776083+0000","last_unstale":"2024-04-25T10:26:10.758577+0000","last_undegraded":"2024-04-25T10:26:10.758577+0000","last_fullsized":"2024-04-25T10:26:10.758577+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:55.464088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.779880+0000","last_change":"2024-04-25T10:26:09.778279+0000","last_active":"2024-04-25T10:26:09.779880+0000","last_peered":"2024-04-25T10:26:09.779880+0000","last_clean":"2024-04-25T10:26:09.779880+0000","last_became_active":"2024-04-25T10:26:09.778038+0000","last_became_peered":"2024-04-25T10:26:09.778038+0000","last_unstale":"2024-04-25T10:26:09.779880+0000","last_undegraded":"2024-04-25T10:26:09.779880+0000","last_fullsized":"2024-04-25T10:26:09.779880+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.797740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758311+0000","last_change":"2024-04-25T10:26:09.773194+0000","last_active":"2024-04-25T10:26:10.758311+0000","last_peered":"2024-04-25T10:26:10.758311+0000","last_clean":"2024-04-25T10:26:10.758311+0000","last_became_active":"2024-04-25T10:26:09.772105+0000","last_became_peered":"2024-04-25T10:26:09.772105+0000","last_unstale":"2024-04-25T10:26:10.758311+0000","last_undegraded":"2024-04-25T10:26:10.758311+0000","last_fullsized":"2024-04-25T10:26:10.758311+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:39.352728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.776647+0000","last_change":"2024-04-25T10:26:09.773893+0000","last_active":"2024-04-25T10:26:09.776647+0000","last_peered":"2024-04-25T10:26:09.776647+0000","last_clean":"2024-04-25T10:26:09.776647+0000","last_became_active":"2024-04-25T10:26:09.773448+0000","last_became_peered":"2024-04-25T10:26:09.773448+0000","last_unstale":"2024-04-25T10:26:09.776647+0000","last_undegraded":"2024-04-25T10:26:09.776647+0000","last_fullsized":"2024-04-25T10:26:09.776647+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:56.644989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786822+0000","last_change":"2024-04-25T10:26:09.785528+0000","last_active":"2024-04-25T10:26:09.786822+0000","last_peered":"2024-04-25T10:26:09.786822+0000","last_clean":"2024-04-25T10:26:09.786822+0000","last_became_active":"2024-04-25T10:26:09.785242+0000","last_became_peered":"2024-04-25T10:26:09.785242+0000","last_unstale":"2024-04-25T10:26:09.786822+0000","last_undegraded":"2024-04-25T10:26:09.786822+0000","last_fullsized":"2024-04-25T10:26:09.786822+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.122787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777786+0000","last_change":"2024-04-25T10:26:09.781282+0000","last_active":"2024-04-25T10:26:10.777786+0000","last_peered":"2024-04-25T10:26:10.777786+0000","last_clean":"2024-04-25T10:26:10.777786+0000","last_became_active":"2024-04-25T10:26:09.780997+0000","last_became_peered":"2024-04-25T10:26:09.780997+0000","last_unstale":"2024-04-25T10:26:10.777786+0000","last_undegraded":"2024-04-25T10:26:10.777786+0000","last_fullsized":"2024-04-25T10:26:10.777786+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:55.464088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.766151+0000","last_change":"2024-04-25T10:26:09.764617+0000","last_active":"2024-04-25T10:26:09.766151+0000","last_peered":"2024-04-25T10:26:09.766151+0000","last_clean":"2024-04-25T10:26:09.766151+0000","last_became_active":"2024-04-25T10:26:09.764279+0000","last_became_peered":"2024-04-25T10:26:09.764279+0000","last_unstale":"2024-04-25T10:26:09.766151+0000","last_undegraded":"2024-04-25T10:26:09.766151+0000","last_fullsized":"2024-04-25T10:26:09.766151+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:00.455133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778334+0000","last_change":"2024-04-25T10:26:09.785894+0000","last_active":"2024-04-25T10:26:10.778334+0000","last_peered":"2024-04-25T10:26:10.778334+0000","last_clean":"2024-04-25T10:26:10.778334+0000","last_became_active":"2024-04-25T10:26:09.785249+0000","last_became_peered":"2024-04-25T10:26:09.785249+0000","last_unstale":"2024-04-25T10:26:10.778334+0000","last_undegraded":"2024-04-25T10:26:10.778334+0000","last_fullsized":"2024-04-25T10:26:10.778334+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:42.732330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779734+0000","last_change":"2024-04-25T10:26:09.766590+0000","last_active":"2024-04-25T10:26:10.779734+0000","last_peered":"2024-04-25T10:26:10.779734+0000","last_clean":"2024-04-25T10:26:10.779734+0000","last_became_active":"2024-04-25T10:26:09.766293+0000","last_became_peered":"2024-04-25T10:26:09.766293+0000","last_unstale":"2024-04-25T10:26:10.779734+0000","last_undegraded":"2024-04-25T10:26:10.779734+0000","last_fullsized":"2024-04-25T10:26:10.779734+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:38.172419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778344+0000","last_change":"2024-04-25T10:26:09.783142+0000","last_active":"2024-04-25T10:26:10.778344+0000","last_peered":"2024-04-25T10:26:10.778344+0000","last_clean":"2024-04-25T10:26:10.778344+0000","last_became_active":"2024-04-25T10:26:09.782380+0000","last_became_peered":"2024-04-25T10:26:09.782380+0000","last_unstale":"2024-04-25T10:26:10.778344+0000","last_undegraded":"2024-04-25T10:26:10.778344+0000","last_fullsized":"2024-04-25T10:26:10.778344+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:14.181376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758464+0000","last_change":"2024-04-25T10:26:09.762518+0000","last_active":"2024-04-25T10:26:10.758464+0000","last_peered":"2024-04-25T10:26:10.758464+0000","last_clean":"2024-04-25T10:26:10.758464+0000","last_became_active":"2024-04-25T10:26:09.762265+0000","last_became_peered":"2024-04-25T10:26:09.762265+0000","last_unstale":"2024-04-25T10:26:10.758464+0000","last_undegraded":"2024-04-25T10:26:10.758464+0000","last_fullsized":"2024-04-25T10:26:10.758464+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:22.466158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760891+0000","last_change":"2024-04-25T10:26:09.781240+0000","last_active":"2024-04-25T10:26:10.760891+0000","last_peered":"2024-04-25T10:26:10.760891+0000","last_clean":"2024-04-25T10:26:10.760891+0000","last_became_active":"2024-04-25T10:26:09.780975+0000","last_became_peered":"2024-04-25T10:26:09.780975+0000","last_unstale":"2024-04-25T10:26:10.760891+0000","last_undegraded":"2024-04-25T10:26:10.760891+0000","last_fullsized":"2024-04-25T10:26:10.760891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:48.946289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777867+0000","last_change":"2024-04-25T10:26:09.784366+0000","last_active":"2024-04-25T10:26:10.777867+0000","last_peered":"2024-04-25T10:26:10.777867+0000","last_clean":"2024-04-25T10:26:10.777867+0000","last_became_active":"2024-04-25T10:26:09.784030+0000","last_became_peered":"2024-04-25T10:26:09.784030+0000","last_unstale":"2024-04-25T10:26:10.777867+0000","last_undegraded":"2024-04-25T10:26:10.777867+0000","last_fullsized":"2024-04-25T10:26:10.777867+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:16.501974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.767976+0000","last_change":"2024-04-25T10:26:09.766966+0000","last_active":"2024-04-25T10:26:09.767976+0000","last_peered":"2024-04-25T10:26:09.767976+0000","last_clean":"2024-04-25T10:26:09.767976+0000","last_became_active":"2024-04-25T10:26:09.766229+0000","last_became_peered":"2024-04-25T10:26:09.766229+0000","last_unstale":"2024-04-25T10:26:09.767976+0000","last_undegraded":"2024-04-25T10:26:09.767976+0000","last_fullsized":"2024-04-25T10:26:09.767976+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.040552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.780154+0000","last_change":"2024-04-25T10:26:09.778592+0000","last_active":"2024-04-25T10:26:09.780154+0000","last_peered":"2024-04-25T10:26:09.780154+0000","last_clean":"2024-04-25T10:26:09.780154+0000","last_became_active":"2024-04-25T10:26:09.778322+0000","last_became_peered":"2024-04-25T10:26:09.778322+0000","last_unstale":"2024-04-25T10:26:09.780154+0000","last_undegraded":"2024-04-25T10:26:09.780154+0000","last_fullsized":"2024-04-25T10:26:09.780154+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:18.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796741+0000","last_change":"2024-04-25T10:26:09.794892+0000","last_active":"2024-04-25T10:26:09.796741+0000","last_peered":"2024-04-25T10:26:09.796741+0000","last_clean":"2024-04-25T10:26:09.796741+0000","last_became_active":"2024-04-25T10:26:09.794574+0000","last_became_peered":"2024-04-25T10:26:09.794574+0000","last_unstale":"2024-04-25T10:26:09.796741+0000","last_undegraded":"2024-04-25T10:26:09.796741+0000","last_fullsized":"2024-04-25T10:26:09.796741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.337257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"186'4","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786186+0000","last_change":"2024-04-25T10:26:08.771499+0000","last_active":"2024-04-25T10:26:09.786186+0000","last_peered":"2024-04-25T10:26:09.786186+0000","last_clean":"2024-04-25T10:26:09.786186+0000","last_became_active":"2024-04-25T10:26:08.770367+0000","last_became_peered":"2024-04-25T10:26:08.770367+0000","last_unstale":"2024-04-25T10:26:09.786186+0000","last_undegraded":"2024-04-25T10:26:09.786186+0000","last_fullsized":"2024-04-25T10:26:09.786186+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:26.131803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779804+0000","last_change":"2024-04-25T10:26:09.766971+0000","last_active":"2024-04-25T10:26:10.779804+0000","last_peered":"2024-04-25T10:26:10.779804+0000","last_clean":"2024-04-25T10:26:10.779804+0000","last_became_active":"2024-04-25T10:26:09.766697+0000","last_became_peered":"2024-04-25T10:26:09.766697+0000","last_unstale":"2024-04-25T10:26:10.779804+0000","last_undegraded":"2024-04-25T10:26:10.779804+0000","last_fullsized":"2024-04-25T10:26:10.779804+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:03.864597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761252+0000","last_change":"2024-04-25T10:26:08.772936+0000","last_active":"2024-04-25T10:26:10.761252+0000","last_peered":"2024-04-25T10:26:10.761252+0000","last_clean":"2024-04-25T10:26:10.761252+0000","last_became_active":"2024-04-25T10:26:08.772593+0000","last_became_peered":"2024-04-25T10:26:08.772593+0000","last_unstale":"2024-04-25T10:26:10.761252+0000","last_undegraded":"2024-04-25T10:26:10.761252+0000","last_fullsized":"2024-04-25T10:26:10.761252+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:29.059222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761275+0000","last_change":"2024-04-25T10:26:09.790769+0000","last_active":"2024-04-25T10:26:10.761275+0000","last_peered":"2024-04-25T10:26:10.761275+0000","last_clean":"2024-04-25T10:26:10.761275+0000","last_became_active":"2024-04-25T10:26:09.790216+0000","last_became_peered":"2024-04-25T10:26:09.790216+0000","last_unstale":"2024-04-25T10:26:10.761275+0000","last_undegraded":"2024-04-25T10:26:10.761275+0000","last_fullsized":"2024-04-25T10:26:10.761275+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:27.889360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760383+0000","last_change":"2024-04-25T10:26:08.769155+0000","last_active":"2024-04-25T10:26:10.760383+0000","last_peered":"2024-04-25T10:26:10.760383+0000","last_clean":"2024-04-25T10:26:10.760383+0000","last_became_active":"2024-04-25T10:26:08.768892+0000","last_became_peered":"2024-04-25T10:26:08.768892+0000","last_unstale":"2024-04-25T10:26:10.760383+0000","last_undegraded":"2024-04-25T10:26:10.760383+0000","last_fullsized":"2024-04-25T10:26:10.760383+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:17.281301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778096+0000","last_change":"2024-04-25T10:26:09.784456+0000","last_active":"2024-04-25T10:26:10.778096+0000","last_peered":"2024-04-25T10:26:10.778096+0000","last_clean":"2024-04-25T10:26:10.778096+0000","last_became_active":"2024-04-25T10:26:09.784183+0000","last_became_peered":"2024-04-25T10:26:09.784183+0000","last_unstale":"2024-04-25T10:26:10.778096+0000","last_undegraded":"2024-04-25T10:26:10.778096+0000","last_fullsized":"2024-04-25T10:26:10.778096+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.775514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779612+0000","last_change":"2024-04-25T10:26:08.769095+0000","last_active":"2024-04-25T10:26:10.779612+0000","last_peered":"2024-04-25T10:26:10.779612+0000","last_clean":"2024-04-25T10:26:10.779612+0000","last_became_active":"2024-04-25T10:26:08.768849+0000","last_became_peered":"2024-04-25T10:26:08.768849+0000","last_unstale":"2024-04-25T10:26:10.779612+0000","last_undegraded":"2024-04-25T10:26:10.779612+0000","last_fullsized":"2024-04-25T10:26:10.779612+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:21.263935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.784690+0000","last_change":"2024-04-25T10:26:09.782189+0000","last_active":"2024-04-25T10:26:09.784690+0000","last_peered":"2024-04-25T10:26:09.784690+0000","last_clean":"2024-04-25T10:26:09.784690+0000","last_became_active":"2024-04-25T10:26:09.781908+0000","last_became_peered":"2024-04-25T10:26:09.781908+0000","last_unstale":"2024-04-25T10:26:09.784690+0000","last_undegraded":"2024-04-25T10:26:09.784690+0000","last_fullsized":"2024-04-25T10:26:09.784690+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.337257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"186'1","reported_seq":13,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796182+0000","last_change":"2024-04-25T10:26:08.775857+0000","last_active":"2024-04-25T10:26:09.796182+0000","last_peered":"2024-04-25T10:26:09.796182+0000","last_clean":"2024-04-25T10:26:09.796182+0000","last_became_active":"2024-04-25T10:26:08.775560+0000","last_became_peered":"2024-04-25T10:26:08.775560+0000","last_unstale":"2024-04-25T10:26:09.796182+0000","last_undegraded":"2024-04-25T10:26:09.796182+0000","last_fullsized":"2024-04-25T10:26:09.796182+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:29.718152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758253+0000","last_change":"2024-04-25T10:26:09.762282+0000","last_active":"2024-04-25T10:26:10.758253+0000","last_peered":"2024-04-25T10:26:10.758253+0000","last_clean":"2024-04-25T10:26:10.758253+0000","last_became_active":"2024-04-25T10:26:09.761990+0000","last_became_peered":"2024-04-25T10:26:09.761990+0000","last_unstale":"2024-04-25T10:26:10.758253+0000","last_undegraded":"2024-04-25T10:26:10.758253+0000","last_fullsized":"2024-04-25T10:26:10.758253+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:06.230346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.759812+0000","last_change":"2024-04-25T10:26:08.770209+0000","last_active":"2024-04-25T10:26:10.759812+0000","last_peered":"2024-04-25T10:26:10.759812+0000","last_clean":"2024-04-25T10:26:10.759812+0000","last_became_active":"2024-04-25T10:26:08.769565+0000","last_became_peered":"2024-04-25T10:26:08.769565+0000","last_unstale":"2024-04-25T10:26:10.759812+0000","last_undegraded":"2024-04-25T10:26:10.759812+0000","last_fullsized":"2024-04-25T10:26:10.759812+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:36.202336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760520+0000","last_change":"2024-04-25T10:26:09.791042+0000","last_active":"2024-04-25T10:26:10.760520+0000","last_peered":"2024-04-25T10:26:10.760520+0000","last_clean":"2024-04-25T10:26:10.760520+0000","last_became_active":"2024-04-25T10:26:09.790778+0000","last_became_peered":"2024-04-25T10:26:09.790778+0000","last_unstale":"2024-04-25T10:26:10.760520+0000","last_undegraded":"2024-04-25T10:26:10.760520+0000","last_fullsized":"2024-04-25T10:26:10.760520+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:56.644989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758196+0000","last_change":"2024-04-25T10:26:08.772494+0000","last_active":"2024-04-25T10:26:10.758196+0000","last_peered":"2024-04-25T10:26:10.758196+0000","last_clean":"2024-04-25T10:26:10.758196+0000","last_became_active":"2024-04-25T10:26:08.772223+0000","last_became_peered":"2024-04-25T10:26:08.772223+0000","last_unstale":"2024-04-25T10:26:10.758196+0000","last_undegraded":"2024-04-25T10:26:10.758196+0000","last_fullsized":"2024-04-25T10:26:10.758196+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:57.994885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760586+0000","last_change":"2024-04-25T10:26:09.789318+0000","last_active":"2024-04-25T10:26:10.760586+0000","last_peered":"2024-04-25T10:26:10.760586+0000","last_clean":"2024-04-25T10:26:10.760586+0000","last_became_active":"2024-04-25T10:26:09.789030+0000","last_became_peered":"2024-04-25T10:26:09.789030+0000","last_unstale":"2024-04-25T10:26:10.760586+0000","last_undegraded":"2024-04-25T10:26:10.760586+0000","last_fullsized":"2024-04-25T10:26:10.760586+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.797740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778449+0000","last_change":"2024-04-25T10:26:08.773430+0000","last_active":"2024-04-25T10:26:10.778449+0000","last_peered":"2024-04-25T10:26:10.778449+0000","last_clean":"2024-04-25T10:26:10.778449+0000","last_became_active":"2024-04-25T10:26:08.771655+0000","last_became_peered":"2024-04-25T10:26:08.771655+0000","last_unstale":"2024-04-25T10:26:10.778449+0000","last_undegraded":"2024-04-25T10:26:10.778449+0000","last_fullsized":"2024-04-25T10:26:10.778449+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:19.319975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778371+0000","last_change":"2024-04-25T10:26:09.786102+0000","last_active":"2024-04-25T10:26:10.778371+0000","last_peered":"2024-04-25T10:26:10.778371+0000","last_clean":"2024-04-25T10:26:10.778371+0000","last_became_active":"2024-04-25T10:26:09.785620+0000","last_became_peered":"2024-04-25T10:26:09.785620+0000","last_unstale":"2024-04-25T10:26:10.778371+0000","last_undegraded":"2024-04-25T10:26:10.778371+0000","last_fullsized":"2024-04-25T10:26:10.778371+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:05.164114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.770029+0000","last_change":"2024-04-25T10:26:08.773287+0000","last_active":"2024-04-25T10:26:09.770029+0000","last_peered":"2024-04-25T10:26:09.770029+0000","last_clean":"2024-04-25T10:26:09.770029+0000","last_became_active":"2024-04-25T10:26:08.772947+0000","last_became_peered":"2024-04-25T10:26:08.772947+0000","last_unstale":"2024-04-25T10:26:09.770029+0000","last_undegraded":"2024-04-25T10:26:09.770029+0000","last_fullsized":"2024-04-25T10:26:09.770029+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:57.994885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761185+0000","last_change":"2024-04-25T10:26:09.779115+0000","last_active":"2024-04-25T10:26:10.761185+0000","last_peered":"2024-04-25T10:26:10.761185+0000","last_clean":"2024-04-25T10:26:10.761185+0000","last_became_active":"2024-04-25T10:26:09.778776+0000","last_became_peered":"2024-04-25T10:26:09.778776+0000","last_unstale":"2024-04-25T10:26:10.761185+0000","last_undegraded":"2024-04-25T10:26:10.761185+0000","last_fullsized":"2024-04-25T10:26:10.761185+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.515662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"186'3","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.787775+0000","last_change":"2024-04-25T10:26:08.765902+0000","last_active":"2024-04-25T10:26:09.787775+0000","last_peered":"2024-04-25T10:26:09.787775+0000","last_clean":"2024-04-25T10:26:09.787775+0000","last_became_active":"2024-04-25T10:26:08.765627+0000","last_became_peered":"2024-04-25T10:26:08.765627+0000","last_unstale":"2024-04-25T10:26:09.787775+0000","last_undegraded":"2024-04-25T10:26:09.787775+0000","last_fullsized":"2024-04-25T10:26:09.787775+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:56.313798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778160+0000","last_change":"2024-04-25T10:26:09.788367+0000","last_active":"2024-04-25T10:26:10.778160+0000","last_peered":"2024-04-25T10:26:10.778160+0000","last_clean":"2024-04-25T10:26:10.778160+0000","last_became_active":"2024-04-25T10:26:09.788088+0000","last_became_peered":"2024-04-25T10:26:09.788088+0000","last_unstale":"2024-04-25T10:26:10.778160+0000","last_undegraded":"2024-04-25T10:26:10.778160+0000","last_fullsized":"2024-04-25T10:26:10.778160+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:51.229311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"186'6","reported_seq":20,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758522+0000","last_change":"2024-04-25T10:26:08.768844+0000","last_active":"2024-04-25T10:26:10.758522+0000","last_peered":"2024-04-25T10:26:10.758522+0000","last_clean":"2024-04-25T10:26:10.758522+0000","last_became_active":"2024-04-25T10:26:08.768570+0000","last_became_peered":"2024-04-25T10:26:08.768570+0000","last_unstale":"2024-04-25T10:26:10.758522+0000","last_undegraded":"2024-04-25T10:26:10.758522+0000","last_fullsized":"2024-04-25T10:26:10.758522+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:17.666693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.778678+0000","last_change":"2024-04-25T10:26:09.774293+0000","last_active":"2024-04-25T10:26:09.778678+0000","last_peered":"2024-04-25T10:26:09.778678+0000","last_clean":"2024-04-25T10:26:09.778678+0000","last_became_active":"2024-04-25T10:26:09.773129+0000","last_became_peered":"2024-04-25T10:26:09.773129+0000","last_unstale":"2024-04-25T10:26:09.778678+0000","last_undegraded":"2024-04-25T10:26:09.778678+0000","last_fullsized":"2024-04-25T10:26:09.778678+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:17.101525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779613+0000","last_change":"2024-04-25T10:26:08.772605+0000","last_active":"2024-04-25T10:26:10.779613+0000","last_peered":"2024-04-25T10:26:10.779613+0000","last_clean":"2024-04-25T10:26:10.779613+0000","last_became_active":"2024-04-25T10:26:08.772343+0000","last_became_peered":"2024-04-25T10:26:08.772343+0000","last_unstale":"2024-04-25T10:26:10.779613+0000","last_undegraded":"2024-04-25T10:26:10.779613+0000","last_fullsized":"2024-04-25T10:26:10.779613+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.422079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786145+0000","last_change":"2024-04-25T10:26:09.780490+0000","last_active":"2024-04-25T10:26:09.786145+0000","last_peered":"2024-04-25T10:26:09.786145+0000","last_clean":"2024-04-25T10:26:09.786145+0000","last_became_active":"2024-04-25T10:26:09.780172+0000","last_became_peered":"2024-04-25T10:26:09.780172+0000","last_unstale":"2024-04-25T10:26:09.786145+0000","last_undegraded":"2024-04-25T10:26:09.786145+0000","last_fullsized":"2024-04-25T10:26:09.786145+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:33.855220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.770467+0000","last_change":"2024-04-25T10:26:08.773593+0000","last_active":"2024-04-25T10:26:09.770467+0000","last_peered":"2024-04-25T10:26:09.770467+0000","last_clean":"2024-04-25T10:26:09.770467+0000","last_became_active":"2024-04-25T10:26:08.771018+0000","last_became_peered":"2024-04-25T10:26:08.771018+0000","last_unstale":"2024-04-25T10:26:09.770467+0000","last_undegraded":"2024-04-25T10:26:09.770467+0000","last_fullsized":"2024-04-25T10:26:09.770467+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:19.319975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.787626+0000","last_change":"2024-04-25T10:26:09.782454+0000","last_active":"2024-04-25T10:26:09.787626+0000","last_peered":"2024-04-25T10:26:09.787626+0000","last_clean":"2024-04-25T10:26:09.787626+0000","last_became_active":"2024-04-25T10:26:09.782184+0000","last_became_peered":"2024-04-25T10:26:09.782184+0000","last_unstale":"2024-04-25T10:26:09.787626+0000","last_undegraded":"2024-04-25T10:26:09.787626+0000","last_fullsized":"2024-04-25T10:26:09.787626+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:24.976889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.761165+0000","last_change":"2024-04-25T10:26:08.767366+0000","last_active":"2024-04-25T10:26:09.761165+0000","last_peered":"2024-04-25T10:26:09.761165+0000","last_clean":"2024-04-25T10:26:09.761165+0000","last_became_active":"2024-04-25T10:26:08.767109+0000","last_became_peered":"2024-04-25T10:26:08.767109+0000","last_unstale":"2024-04-25T10:26:09.761165+0000","last_undegraded":"2024-04-25T10:26:09.761165+0000","last_fullsized":"2024-04-25T10:26:09.761165+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:29.718152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760122+0000","last_change":"2024-04-25T10:26:09.770551+0000","last_active":"2024-04-25T10:26:10.760122+0000","last_peered":"2024-04-25T10:26:10.760122+0000","last_clean":"2024-04-25T10:26:10.760122+0000","last_became_active":"2024-04-25T10:26:09.770271+0000","last_became_peered":"2024-04-25T10:26:09.770271+0000","last_unstale":"2024-04-25T10:26:10.760122+0000","last_undegraded":"2024-04-25T10:26:10.760122+0000","last_fullsized":"2024-04-25T10:26:10.760122+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:20.272652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"186'8","reported_seq":22,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760269+0000","last_change":"2024-04-25T10:26:08.768259+0000","last_active":"2024-04-25T10:26:10.760269+0000","last_peered":"2024-04-25T10:26:10.760269+0000","last_clean":"2024-04-25T10:26:10.760269+0000","last_became_active":"2024-04-25T10:26:08.767942+0000","last_became_peered":"2024-04-25T10:26:08.767942+0000","last_unstale":"2024-04-25T10:26:10.760269+0000","last_undegraded":"2024-04-25T10:26:10.760269+0000","last_fullsized":"2024-04-25T10:26:10.760269+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:47.939344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779852+0000","last_change":"2024-04-25T10:26:09.768006+0000","last_active":"2024-04-25T10:26:10.779852+0000","last_peered":"2024-04-25T10:26:10.779852+0000","last_clean":"2024-04-25T10:26:10.779852+0000","last_became_active":"2024-04-25T10:26:09.767614+0000","last_became_peered":"2024-04-25T10:26:09.767614+0000","last_unstale":"2024-04-25T10:26:10.779852+0000","last_undegraded":"2024-04-25T10:26:10.779852+0000","last_fullsized":"2024-04-25T10:26:10.779852+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:36.718139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"186'1","reported_seq":13,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796263+0000","last_change":"2024-04-25T10:26:08.778204+0000","last_active":"2024-04-25T10:26:09.796263+0000","last_peered":"2024-04-25T10:26:09.796263+0000","last_clean":"2024-04-25T10:26:09.796263+0000","last_became_active":"2024-04-25T10:26:08.777927+0000","last_became_peered":"2024-04-25T10:26:08.777927+0000","last_unstale":"2024-04-25T10:26:09.796263+0000","last_undegraded":"2024-04-25T10:26:09.796263+0000","last_fullsized":"2024-04-25T10:26:09.796263+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:24.841692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758576+0000","last_change":"2024-04-25T10:26:09.789862+0000","last_active":"2024-04-25T10:26:10.758576+0000","last_peered":"2024-04-25T10:26:10.758576+0000","last_clean":"2024-04-25T10:26:10.758576+0000","last_became_active":"2024-04-25T10:26:09.789602+0000","last_became_peered":"2024-04-25T10:26:09.789602+0000","last_unstale":"2024-04-25T10:26:10.758576+0000","last_undegraded":"2024-04-25T10:26:10.758576+0000","last_fullsized":"2024-04-25T10:26:10.758576+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:13.652223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"186'2","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.789078+0000","last_change":"2024-04-25T10:26:08.774643+0000","last_active":"2024-04-25T10:26:09.789078+0000","last_peered":"2024-04-25T10:26:09.789078+0000","last_clean":"2024-04-25T10:26:09.789078+0000","last_became_active":"2024-04-25T10:26:08.774385+0000","last_became_peered":"2024-04-25T10:26:08.774385+0000","last_unstale":"2024-04-25T10:26:09.789078+0000","last_undegraded":"2024-04-25T10:26:09.789078+0000","last_fullsized":"2024-04-25T10:26:09.789078+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:17.666693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.794250+0000","last_change":"2024-04-25T10:26:09.789044+0000","last_active":"2024-04-25T10:26:09.794250+0000","last_peered":"2024-04-25T10:26:09.794250+0000","last_clean":"2024-04-25T10:26:09.794250+0000","last_became_active":"2024-04-25T10:26:09.784272+0000","last_became_peered":"2024-04-25T10:26:09.784272+0000","last_unstale":"2024-04-25T10:26:09.794250+0000","last_undegraded":"2024-04-25T10:26:09.794250+0000","last_fullsized":"2024-04-25T10:26:09.794250+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:43.468414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778305+0000","last_change":"2024-04-25T10:26:08.772628+0000","last_active":"2024-04-25T10:26:10.778305+0000","last_peered":"2024-04-25T10:26:10.778305+0000","last_clean":"2024-04-25T10:26:10.778305+0000","last_became_active":"2024-04-25T10:26:08.771096+0000","last_became_peered":"2024-04-25T10:26:08.771096+0000","last_unstale":"2024-04-25T10:26:10.778305+0000","last_undegraded":"2024-04-25T10:26:10.778305+0000","last_fullsized":"2024-04-25T10:26:10.778305+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:17.666693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792886+0000","last_change":"2024-04-25T10:26:09.784938+0000","last_active":"2024-04-25T10:26:09.792886+0000","last_peered":"2024-04-25T10:26:09.792886+0000","last_clean":"2024-04-25T10:26:09.792886+0000","last_became_active":"2024-04-25T10:26:09.784636+0000","last_became_peered":"2024-04-25T10:26:09.784636+0000","last_unstale":"2024-04-25T10:26:09.792886+0000","last_undegraded":"2024-04-25T10:26:09.792886+0000","last_fullsized":"2024-04-25T10:26:09.792886+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:56.644989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758565+0000","last_change":"2024-04-25T10:26:08.775364+0000","last_active":"2024-04-25T10:26:10.758565+0000","last_peered":"2024-04-25T10:26:10.758565+0000","last_clean":"2024-04-25T10:26:10.758565+0000","last_became_active":"2024-04-25T10:26:08.774983+0000","last_became_peered":"2024-04-25T10:26:08.774983+0000","last_unstale":"2024-04-25T10:26:10.758565+0000","last_undegraded":"2024-04-25T10:26:10.758565+0000","last_fullsized":"2024-04-25T10:26:10.758565+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:16.584614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.765035+0000","last_change":"2024-04-25T10:26:09.763794+0000","last_active":"2024-04-25T10:26:09.765035+0000","last_peered":"2024-04-25T10:26:09.765035+0000","last_clean":"2024-04-25T10:26:09.765035+0000","last_became_active":"2024-04-25T10:26:09.763530+0000","last_became_peered":"2024-04-25T10:26:09.763530+0000","last_unstale":"2024-04-25T10:26:09.765035+0000","last_undegraded":"2024-04-25T10:26:09.765035+0000","last_fullsized":"2024-04-25T10:26:09.765035+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:33.855220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"186'2","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.797014+0000","last_change":"2024-04-25T10:26:08.778795+0000","last_active":"2024-04-25T10:26:09.797014+0000","last_peered":"2024-04-25T10:26:09.797014+0000","last_clean":"2024-04-25T10:26:09.797014+0000","last_became_active":"2024-04-25T10:26:08.778474+0000","last_became_peered":"2024-04-25T10:26:08.778474+0000","last_unstale":"2024-04-25T10:26:09.797014+0000","last_undegraded":"2024-04-25T10:26:09.797014+0000","last_fullsized":"2024-04-25T10:26:09.797014+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:56.313798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792994+0000","last_change":"2024-04-25T10:26:09.786159+0000","last_active":"2024-04-25T10:26:09.792994+0000","last_peered":"2024-04-25T10:26:09.792994+0000","last_clean":"2024-04-25T10:26:09.792994+0000","last_became_active":"2024-04-25T10:26:09.785886+0000","last_became_peered":"2024-04-25T10:26:09.785886+0000","last_unstale":"2024-04-25T10:26:09.792994+0000","last_undegraded":"2024-04-25T10:26:09.792994+0000","last_fullsized":"2024-04-25T10:26:09.792994+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.797740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760750+0000","last_change":"2024-04-25T10:26:08.767339+0000","last_active":"2024-04-25T10:26:10.760750+0000","last_peered":"2024-04-25T10:26:10.760750+0000","last_clean":"2024-04-25T10:26:10.760750+0000","last_became_active":"2024-04-25T10:26:08.766821+0000","last_became_peered":"2024-04-25T10:26:08.766821+0000","last_unstale":"2024-04-25T10:26:10.760750+0000","last_undegraded":"2024-04-25T10:26:10.760750+0000","last_fullsized":"2024-04-25T10:26:10.760750+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:43.273630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779401+0000","last_change":"2024-04-25T10:26:09.764101+0000","last_active":"2024-04-25T10:26:10.779401+0000","last_peered":"2024-04-25T10:26:10.779401+0000","last_clean":"2024-04-25T10:26:10.779401+0000","last_became_active":"2024-04-25T10:26:09.763856+0000","last_became_peered":"2024-04-25T10:26:09.763856+0000","last_unstale":"2024-04-25T10:26:10.779401+0000","last_undegraded":"2024-04-25T10:26:10.779401+0000","last_fullsized":"2024-04-25T10:26:10.779401+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:21.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760070+0000","last_change":"2024-04-25T10:26:08.775664+0000","last_active":"2024-04-25T10:26:10.760070+0000","last_peered":"2024-04-25T10:26:10.760070+0000","last_clean":"2024-04-25T10:26:10.760070+0000","last_became_active":"2024-04-25T10:26:08.775347+0000","last_became_peered":"2024-04-25T10:26:08.775347+0000","last_unstale":"2024-04-25T10:26:10.760070+0000","last_undegraded":"2024-04-25T10:26:10.760070+0000","last_fullsized":"2024-04-25T10:26:10.760070+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:20.510597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.780498+0000","last_change":"2024-04-25T10:26:09.779977+0000","last_active":"2024-04-25T10:26:09.780498+0000","last_peered":"2024-04-25T10:26:09.780498+0000","last_clean":"2024-04-25T10:26:09.780498+0000","last_became_active":"2024-04-25T10:26:09.779591+0000","last_became_peered":"2024-04-25T10:26:09.779591+0000","last_unstale":"2024-04-25T10:26:09.780498+0000","last_undegraded":"2024-04-25T10:26:09.780498+0000","last_fullsized":"2024-04-25T10:26:09.780498+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:43.468414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.777929+0000","last_change":"2024-04-25T10:26:08.779454+0000","last_active":"2024-04-25T10:26:09.777929+0000","last_peered":"2024-04-25T10:26:09.777929+0000","last_clean":"2024-04-25T10:26:09.777929+0000","last_became_active":"2024-04-25T10:26:08.779132+0000","last_became_peered":"2024-04-25T10:26:08.779132+0000","last_unstale":"2024-04-25T10:26:09.777929+0000","last_undegraded":"2024-04-25T10:26:09.777929+0000","last_fullsized":"2024-04-25T10:26:09.777929+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:04.508717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.780283+0000","last_change":"2024-04-25T10:26:09.779557+0000","last_active":"2024-04-25T10:26:09.780283+0000","last_peered":"2024-04-25T10:26:09.780283+0000","last_clean":"2024-04-25T10:26:09.780283+0000","last_became_active":"2024-04-25T10:26:09.779296+0000","last_became_peered":"2024-04-25T10:26:09.779296+0000","last_unstale":"2024-04-25T10:26:09.780283+0000","last_undegraded":"2024-04-25T10:26:09.780283+0000","last_fullsized":"2024-04-25T10:26:09.780283+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:43.146461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"186'2","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.793819+0000","last_change":"2024-04-25T10:26:08.772670+0000","last_active":"2024-04-25T10:26:09.793819+0000","last_peered":"2024-04-25T10:26:09.793819+0000","last_clean":"2024-04-25T10:26:09.793819+0000","last_became_active":"2024-04-25T10:26:08.772409+0000","last_became_peered":"2024-04-25T10:26:08.772409+0000","last_unstale":"2024-04-25T10:26:09.793819+0000","last_undegraded":"2024-04-25T10:26:09.793819+0000","last_fullsized":"2024-04-25T10:26:09.793819+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:24.841692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786580+0000","last_change":"2024-04-25T10:26:09.780561+0000","last_active":"2024-04-25T10:26:09.786580+0000","last_peered":"2024-04-25T10:26:09.786580+0000","last_clean":"2024-04-25T10:26:09.786580+0000","last_became_active":"2024-04-25T10:26:09.780254+0000","last_became_peered":"2024-04-25T10:26:09.780254+0000","last_unstale":"2024-04-25T10:26:09.786580+0000","last_undegraded":"2024-04-25T10:26:09.786580+0000","last_fullsized":"2024-04-25T10:26:09.786580+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:00.455133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"187'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779658+0000","last_change":"2024-04-25T10:26:08.773979+0000","last_active":"2024-04-25T10:26:10.779658+0000","last_peered":"2024-04-25T10:26:10.779658+0000","last_clean":"2024-04-25T10:26:10.779658+0000","last_became_active":"2024-04-25T10:26:08.773654+0000","last_became_peered":"2024-04-25T10:26:08.773654+0000","last_unstale":"2024-04-25T10:26:10.779658+0000","last_undegraded":"2024-04-25T10:26:10.779658+0000","last_fullsized":"2024-04-25T10:26:10.779658+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:57.994885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.789179+0000","last_change":"2024-04-25T10:26:09.784589+0000","last_active":"2024-04-25T10:26:09.789179+0000","last_peered":"2024-04-25T10:26:09.789179+0000","last_clean":"2024-04-25T10:26:09.789179+0000","last_became_active":"2024-04-25T10:26:09.782799+0000","last_became_peered":"2024-04-25T10:26:09.782799+0000","last_unstale":"2024-04-25T10:26:09.789179+0000","last_undegraded":"2024-04-25T10:26:09.789179+0000","last_fullsized":"2024-04-25T10:26:09.789179+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.884230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778407+0000","last_change":"2024-04-25T10:26:08.776951+0000","last_active":"2024-04-25T10:26:10.778407+0000","last_peered":"2024-04-25T10:26:10.778407+0000","last_clean":"2024-04-25T10:26:10.778407+0000","last_became_active":"2024-04-25T10:26:08.776616+0000","last_became_peered":"2024-04-25T10:26:08.776616+0000","last_unstale":"2024-04-25T10:26:10.778407+0000","last_undegraded":"2024-04-25T10:26:10.778407+0000","last_fullsized":"2024-04-25T10:26:10.778407+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:16.584614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.776552+0000","last_change":"2024-04-25T10:26:09.773061+0000","last_active":"2024-04-25T10:26:09.776552+0000","last_peered":"2024-04-25T10:26:09.776552+0000","last_clean":"2024-04-25T10:26:09.776552+0000","last_became_active":"2024-04-25T10:26:09.772816+0000","last_became_peered":"2024-04-25T10:26:09.772816+0000","last_unstale":"2024-04-25T10:26:09.776552+0000","last_undegraded":"2024-04-25T10:26:09.776552+0000","last_fullsized":"2024-04-25T10:26:09.776552+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.884230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.779430+0000","last_change":"2024-04-25T10:26:08.771906+0000","last_active":"2024-04-25T10:26:09.779430+0000","last_peered":"2024-04-25T10:26:09.779430+0000","last_clean":"2024-04-25T10:26:09.779430+0000","last_became_active":"2024-04-25T10:26:08.771643+0000","last_became_peered":"2024-04-25T10:26:08.771643+0000","last_unstale":"2024-04-25T10:26:09.779430+0000","last_undegraded":"2024-04-25T10:26:09.779430+0000","last_fullsized":"2024-04-25T10:26:09.779430+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.871193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760531+0000","last_change":"2024-04-25T10:26:09.790561+0000","last_active":"2024-04-25T10:26:10.760531+0000","last_peered":"2024-04-25T10:26:10.760531+0000","last_clean":"2024-04-25T10:26:10.760531+0000","last_became_active":"2024-04-25T10:26:09.785068+0000","last_became_peered":"2024-04-25T10:26:09.785068+0000","last_unstale":"2024-04-25T10:26:10.760531+0000","last_undegraded":"2024-04-25T10:26:10.760531+0000","last_fullsized":"2024-04-25T10:26:10.760531+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760845+0000","last_change":"2024-04-25T10:26:08.774587+0000","last_active":"2024-04-25T10:26:10.760845+0000","last_peered":"2024-04-25T10:26:10.760845+0000","last_clean":"2024-04-25T10:26:10.760845+0000","last_became_active":"2024-04-25T10:26:08.773480+0000","last_became_peered":"2024-04-25T10:26:08.773480+0000","last_unstale":"2024-04-25T10:26:10.760845+0000","last_undegraded":"2024-04-25T10:26:10.760845+0000","last_fullsized":"2024-04-25T10:26:10.760845+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:07.134806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786122+0000","last_change":"2024-04-25T10:26:09.776106+0000","last_active":"2024-04-25T10:26:09.786122+0000","last_peered":"2024-04-25T10:26:09.786122+0000","last_clean":"2024-04-25T10:26:09.786122+0000","last_became_active":"2024-04-25T10:26:09.775830+0000","last_became_peered":"2024-04-25T10:26:09.775830+0000","last_unstale":"2024-04-25T10:26:09.786122+0000","last_undegraded":"2024-04-25T10:26:09.786122+0000","last_fullsized":"2024-04-25T10:26:09.786122+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:18.288245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.770326+0000","last_change":"2024-04-25T10:26:08.769391+0000","last_active":"2024-04-25T10:26:09.770326+0000","last_peered":"2024-04-25T10:26:09.770326+0000","last_clean":"2024-04-25T10:26:09.770326+0000","last_became_active":"2024-04-25T10:26:08.769117+0000","last_became_peered":"2024-04-25T10:26:08.769117+0000","last_unstale":"2024-04-25T10:26:09.770326+0000","last_undegraded":"2024-04-25T10:26:09.770326+0000","last_fullsized":"2024-04-25T10:26:09.770326+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:16.584614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.787601+0000","last_change":"2024-04-25T10:26:09.783413+0000","last_active":"2024-04-25T10:26:09.787601+0000","last_peered":"2024-04-25T10:26:09.787601+0000","last_clean":"2024-04-25T10:26:09.787601+0000","last_became_active":"2024-04-25T10:26:09.780641+0000","last_became_peered":"2024-04-25T10:26:09.780641+0000","last_unstale":"2024-04-25T10:26:09.787601+0000","last_undegraded":"2024-04-25T10:26:09.787601+0000","last_fullsized":"2024-04-25T10:26:09.787601+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.040552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.761315+0000","last_change":"2024-04-25T10:26:08.774202+0000","last_active":"2024-04-25T10:26:09.761315+0000","last_peered":"2024-04-25T10:26:09.761315+0000","last_clean":"2024-04-25T10:26:09.761315+0000","last_became_active":"2024-04-25T10:26:08.772994+0000","last_became_peered":"2024-04-25T10:26:08.772994+0000","last_unstale":"2024-04-25T10:26:09.761315+0000","last_undegraded":"2024-04-25T10:26:09.761315+0000","last_fullsized":"2024-04-25T10:26:09.761315+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.871193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777761+0000","last_change":"2024-04-25T10:26:09.783598+0000","last_active":"2024-04-25T10:26:10.777761+0000","last_peered":"2024-04-25T10:26:10.777761+0000","last_clean":"2024-04-25T10:26:10.777761+0000","last_became_active":"2024-04-25T10:26:09.783330+0000","last_became_peered":"2024-04-25T10:26:09.783330+0000","last_unstale":"2024-04-25T10:26:10.777761+0000","last_undegraded":"2024-04-25T10:26:10.777761+0000","last_fullsized":"2024-04-25T10:26:10.777761+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:10.370110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.778095+0000","last_change":"2024-04-25T10:26:08.770381+0000","last_active":"2024-04-25T10:26:09.778095+0000","last_peered":"2024-04-25T10:26:09.778095+0000","last_clean":"2024-04-25T10:26:09.778095+0000","last_became_active":"2024-04-25T10:26:08.769980+0000","last_became_peered":"2024-04-25T10:26:08.769980+0000","last_unstale":"2024-04-25T10:26:09.778095+0000","last_undegraded":"2024-04-25T10:26:09.778095+0000","last_fullsized":"2024-04-25T10:26:09.778095+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:26.882416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778048+0000","last_change":"2024-04-25T10:26:09.781631+0000","last_active":"2024-04-25T10:26:10.778048+0000","last_peered":"2024-04-25T10:26:10.778048+0000","last_clean":"2024-04-25T10:26:10.778048+0000","last_became_active":"2024-04-25T10:26:09.781370+0000","last_became_peered":"2024-04-25T10:26:09.781370+0000","last_unstale":"2024-04-25T10:26:10.778048+0000","last_undegraded":"2024-04-25T10:26:10.778048+0000","last_fullsized":"2024-04-25T10:26:10.778048+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:28.520771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761311+0000","last_change":"2024-04-25T10:26:08.777232+0000","last_active":"2024-04-25T10:26:10.761311+0000","last_peered":"2024-04-25T10:26:10.761311+0000","last_clean":"2024-04-25T10:26:10.761311+0000","last_became_active":"2024-04-25T10:26:08.776932+0000","last_became_peered":"2024-04-25T10:26:08.776932+0000","last_unstale":"2024-04-25T10:26:10.761311+0000","last_undegraded":"2024-04-25T10:26:10.761311+0000","last_fullsized":"2024-04-25T10:26:10.761311+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:24.509794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":473,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777573+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:26:10.777573+0000","last_peered":"2024-04-25T10:26:10.777573+0000","last_clean":"2024-04-25T10:26:10.777573+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:26:10.777573+0000","last_undegraded":"2024-04-25T10:26:10.777573+0000","last_fullsized":"2024-04-25T10:26:10.777573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64592,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307248,"statfs":{"total":96636764160,"available":96570621952,"internally_reserved":0,"allocated":2949120,"data_stored":1622986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65364,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306476,"statfs":{"total":96636764160,"available":96569831424,"internally_reserved":0,"allocated":2359296,"data_stored":1032604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51536,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320304,"statfs":{"total":96636764160,"available":96583991296,"internally_reserved":0,"allocated":2363392,"data_stored":1038848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575019,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64012,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307828,"statfs":{"total":96636764160,"available":96571215872,"internally_reserved":0,"allocated":2355200,"data_stored":1033130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575020,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59880,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311960,"statfs":{"total":96636764160,"available":96575447040,"internally_reserved":0,"allocated":2379776,"data_stored":1040530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575019,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2383872,"data_stored":1040516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69576,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302264,"statfs":{"total":96636764160,"available":96565518336,"internally_reserved":0,"allocated":2355200,"data_stored":1032638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56276,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315564,"statfs":{"total":96636764160,"available":96579137536,"internally_reserved":0,"allocated":2953216,"data_stored":1623588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:26:13.834 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:26:14.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.011+0000 7f8a00600640 1 Processor -- start 2024-04-25T10:26:14.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.011+0000 7f8a00600640 1 -- start start 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.011+0000 7f8a00600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f89f80d0040 0x7f89f815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.011+0000 7f8a00600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89f815b740 0x7f89f80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.011+0000 7f8a00600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f8071050 con 0x7f89f81550b0 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.011+0000 7f8a00600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f815d040 con 0x7f89f815b740 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.011+0000 7f8a00600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f80712f0 con 0x7f89f80d0040 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fdc00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f89f81550b0 0x7f89f80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45086/0 (socket says 172.21.15.87:45086) 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fdc00640 1 -- 172.21.15.87:0/3164863512 learned_addr learned my addr 172.21.15.87:0/3164863512 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:14.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fe600640 1 --2- 172.21.15.87:0/3164863512 >> v2:172.21.15.87:3300/0 conn(0x7f89f815b740 0x7f89f80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fd200640 1 --2- 172.21.15.87:0/3164863512 >> v2:172.21.15.169:3300/0 conn(0x7f89f80d0040 0x7f89f815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89f7e00640 1 -- 172.21.15.87:0/3164863512 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663550778 0 0) 0x7f89f8071050 con 0x7f89f81550b0 2024-04-25T10:26:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fe600640 1 -- 172.21.15.87:0/3164863512 >> v2:172.21.15.169:3300/0 conn(0x7f89f80d0040 msgr2=0x7f89f815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fe600640 1 --2- 172.21.15.87:0/3164863512 >> v2:172.21.15.169:3300/0 conn(0x7f89f80d0040 0x7f89f815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fe600640 1 -- 172.21.15.87:0/3164863512 >> v1:172.21.15.87:6789/0 conn(0x7f89f81550b0 legacy=0x7f89f80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fe600640 1 -- 172.21.15.87:0/3164863512 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80715d0 con 0x7f89f815b740 2024-04-25T10:26:14.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fd200640 1 --2- 172.21.15.87:0/3164863512 >> v2:172.21.15.169:3300/0 conn(0x7f89f80d0040 0x7f89f815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:14.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89fe600640 1 --2- 172.21.15.87:0/3164863512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80b3260 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f89f8070160 tx=0x7f89f001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbea0f9197a44c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89f7e00640 1 -- 172.21.15.87:0/3164863512 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00b2070 con 0x7f89f815b740 2024-04-25T10:26:14.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89f7e00640 1 -- 172.21.15.87:0/3164863512 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f00a59c0 con 0x7f89f815b740 2024-04-25T10:26:14.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.015+0000 7f89f7e00640 1 -- 172.21.15.87:0/3164863512 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00a5d90 con 0x7f89f815b740 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- 172.21.15.87:0/3164863512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 msgr2=0x7f89f80b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 --2- 172.21.15.87:0/3164863512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80b3260 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f89f8070160 tx=0x7f89f001c900 comp rx=0 tx=0).stop 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- 172.21.15.87:0/3164863512 shutdown_connections 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 --2- 172.21.15.87:0/3164863512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80b3260 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 --2- 172.21.15.87:0/3164863512 >> v2:172.21.15.169:3300/0 conn(0x7f89f80d0040 0x7f89f815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- 172.21.15.87:0/3164863512 >> 172.21.15.87:0/3164863512 conn(0x7f89f8066b70 msgr2=0x7f89f8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- 172.21.15.87:0/3164863512 shutdown_connections 2024-04-25T10:26:14.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- 172.21.15.87:0/3164863512 wait complete. 2024-04-25T10:26:14.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 Processor -- start 2024-04-25T10:26:14.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- start start 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f80d0040 0x7f89f813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f81550b0 0x7f89f814d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f815bd10 con 0x7f89f815b740 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f815bb90 con 0x7f89f80d0040 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f8a00600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f815c030 con 0x7f89f81550b0 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39766/0 (socket says 172.21.15.87:39766) 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 -- 172.21.15.87:0/856474791 learned_addr learned my addr 172.21.15.87:0/856474791 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fdc00640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f80d0040 0x7f89f813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fd200640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f81550b0 0x7f89f814d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 -- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f81550b0 msgr2=0x7f89f814d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f81550b0 0x7f89f814d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 -- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f80d0040 msgr2=0x7f89f813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f80d0040 0x7f89f813c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 -- 172.21.15.87:0/856474791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f814d9d0 con 0x7f89f815b740 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fdc00640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f80d0040 0x7f89f813c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.019+0000 7f89fe600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80ad840 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f89f00b0040 tx=0x7f89f00196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89f5600640 1 -- 172.21.15.87:0/856474791 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00b2040 con 0x7f89f815b740 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89f5600640 1 -- 172.21.15.87:0/856474791 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f00bf070 con 0x7f89f815b740 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89f5600640 1 -- 172.21.15.87:0/856474791 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00ad980 con 0x7f89f815b740 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f814dcb0 con 0x7f89f815b740 2024-04-25T10:26:14.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f814e270 con 0x7f89f815b740 2024-04-25T10:26:14.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f804ef30 con 0x7f89f815b740 2024-04-25T10:26:14.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89f5600640 1 -- 172.21.15.87:0/856474791 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89f00adb20 con 0x7f89f815b740 2024-04-25T10:26:14.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89f5600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d8066f60 0x7f89d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89f5600640 1 -- 172.21.15.87:0/856474791 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89f01203d0 con 0x7f89f815b740 2024-04-25T10:26:14.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89fdc00640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d8066f60 0x7f89d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.023+0000 7f89fdc00640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d8066f60 0x7f89d8069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007fd0 tx=0x7f89ec007c20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.027+0000 7f89f5600640 1 -- 172.21.15.87:0/856474791 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89f0016520 con 0x7f89f815b740 2024-04-25T10:26:14.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.187+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89f80cc190 con 0x7f89d8066f60 2024-04-25T10:26:14.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.191+0000 7f89f5600640 1 -- 172.21.15.87:0/856474791 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252459 (secure 0 0 0) 0x7f89f80cc190 con 0x7f89d8066f60 2024-04-25T10:26:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:14.196 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d8066f60 msgr2=0x7f89d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d8066f60 0x7f89d8069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007fd0 tx=0x7f89ec007c20 comp rx=0 tx=0).stop 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 msgr2=0x7f89f80ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80ad840 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f89f00b0040 tx=0x7f89f00196a0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 shutdown_connections 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f89d8066f60 0x7f89d8069420 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f815b740 0x7f89f80ad840 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89f81550b0 0x7f89f814d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 --2- 172.21.15.87:0/856474791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89f80d0040 0x7f89f813c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 >> 172.21.15.87:0/856474791 conn(0x7f89f8066b70 msgr2=0x7f89f80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 shutdown_connections 2024-04-25T10:26:14.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.195+0000 7f8a00600640 1 -- 172.21.15.87:0/856474791 wait complete. 2024-04-25T10:26:14.215 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2024-04-25T10:26:13.302164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":486960,"kb_used_data":19628,"kb_used_omap":12,"kb_used_meta":467187,"kb_avail":754487760,"statfs":{"total":773094113280,"available":772595466240,"internally_reserved":0,"allocated":20099072,"data_stored":9464840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":478400049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.558564"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778251+0000","last_change":"2024-04-25T10:26:09.789967+0000","last_active":"2024-04-25T10:26:10.778251+0000","last_peered":"2024-04-25T10:26:10.778251+0000","last_clean":"2024-04-25T10:26:10.778251+0000","last_became_active":"2024-04-25T10:26:09.785061+0000","last_became_peered":"2024-04-25T10:26:09.785061+0000","last_unstale":"2024-04-25T10:26:10.778251+0000","last_undegraded":"2024-04-25T10:26:10.778251+0000","last_fullsized":"2024-04-25T10:26:10.778251+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.581572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760412+0000","last_change":"2024-04-25T10:26:09.789548+0000","last_active":"2024-04-25T10:26:10.760412+0000","last_peered":"2024-04-25T10:26:10.760412+0000","last_clean":"2024-04-25T10:26:10.760412+0000","last_became_active":"2024-04-25T10:26:09.789266+0000","last_became_peered":"2024-04-25T10:26:09.789266+0000","last_unstale":"2024-04-25T10:26:10.760412+0000","last_undegraded":"2024-04-25T10:26:10.760412+0000","last_fullsized":"2024-04-25T10:26:10.760412+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.884230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792908+0000","last_change":"2024-04-25T10:26:09.786079+0000","last_active":"2024-04-25T10:26:09.792908+0000","last_peered":"2024-04-25T10:26:09.792908+0000","last_clean":"2024-04-25T10:26:09.792908+0000","last_became_active":"2024-04-25T10:26:09.785787+0000","last_became_peered":"2024-04-25T10:26:09.785787+0000","last_unstale":"2024-04-25T10:26:09.792908+0000","last_undegraded":"2024-04-25T10:26:09.792908+0000","last_fullsized":"2024-04-25T10:26:09.792908+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.779063+0000","last_change":"2024-04-25T10:26:09.778035+0000","last_active":"2024-04-25T10:26:09.779063+0000","last_peered":"2024-04-25T10:26:09.779063+0000","last_clean":"2024-04-25T10:26:09.779063+0000","last_became_active":"2024-04-25T10:26:09.777780+0000","last_became_peered":"2024-04-25T10:26:09.777780+0000","last_unstale":"2024-04-25T10:26:09.779063+0000","last_undegraded":"2024-04-25T10:26:09.779063+0000","last_fullsized":"2024-04-25T10:26:09.779063+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.766411+0000","last_change":"2024-04-25T10:26:09.765034+0000","last_active":"2024-04-25T10:26:09.766411+0000","last_peered":"2024-04-25T10:26:09.766411+0000","last_clean":"2024-04-25T10:26:09.766411+0000","last_became_active":"2024-04-25T10:26:09.763888+0000","last_became_peered":"2024-04-25T10:26:09.763888+0000","last_unstale":"2024-04-25T10:26:09.766411+0000","last_undegraded":"2024-04-25T10:26:09.766411+0000","last_fullsized":"2024-04-25T10:26:09.766411+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:24.976889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758117+0000","last_change":"2024-04-25T10:26:09.765280+0000","last_active":"2024-04-25T10:26:10.758117+0000","last_peered":"2024-04-25T10:26:10.758117+0000","last_clean":"2024-04-25T10:26:10.758117+0000","last_became_active":"2024-04-25T10:26:09.764961+0000","last_became_peered":"2024-04-25T10:26:09.764961+0000","last_unstale":"2024-04-25T10:26:10.758117+0000","last_undegraded":"2024-04-25T10:26:10.758117+0000","last_fullsized":"2024-04-25T10:26:10.758117+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:06.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792795+0000","last_change":"2024-04-25T10:26:09.787540+0000","last_active":"2024-04-25T10:26:09.792795+0000","last_peered":"2024-04-25T10:26:09.792795+0000","last_clean":"2024-04-25T10:26:09.792795+0000","last_became_active":"2024-04-25T10:26:09.785656+0000","last_became_peered":"2024-04-25T10:26:09.785656+0000","last_unstale":"2024-04-25T10:26:09.792795+0000","last_undegraded":"2024-04-25T10:26:09.792795+0000","last_fullsized":"2024-04-25T10:26:09.792795+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:43.146461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779409+0000","last_change":"2024-04-25T10:26:09.778885+0000","last_active":"2024-04-25T10:26:10.779409+0000","last_peered":"2024-04-25T10:26:10.779409+0000","last_clean":"2024-04-25T10:26:10.779409+0000","last_became_active":"2024-04-25T10:26:09.778541+0000","last_became_peered":"2024-04-25T10:26:09.778541+0000","last_unstale":"2024-04-25T10:26:10.779409+0000","last_undegraded":"2024-04-25T10:26:10.779409+0000","last_fullsized":"2024-04-25T10:26:10.779409+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:09.103513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796770+0000","last_change":"2024-04-25T10:26:09.795181+0000","last_active":"2024-04-25T10:26:09.796770+0000","last_peered":"2024-04-25T10:26:09.796770+0000","last_clean":"2024-04-25T10:26:09.796770+0000","last_became_active":"2024-04-25T10:26:09.794687+0000","last_became_peered":"2024-04-25T10:26:09.794687+0000","last_unstale":"2024-04-25T10:26:09.796770+0000","last_undegraded":"2024-04-25T10:26:09.796770+0000","last_fullsized":"2024-04-25T10:26:09.796770+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:42.547135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758363+0000","last_change":"2024-04-25T10:26:09.762737+0000","last_active":"2024-04-25T10:26:10.758363+0000","last_peered":"2024-04-25T10:26:10.758363+0000","last_clean":"2024-04-25T10:26:10.758363+0000","last_became_active":"2024-04-25T10:26:09.762018+0000","last_became_peered":"2024-04-25T10:26:09.762018+0000","last_unstale":"2024-04-25T10:26:10.758363+0000","last_undegraded":"2024-04-25T10:26:10.758363+0000","last_fullsized":"2024-04-25T10:26:10.758363+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:58.320915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778164+0000","last_change":"2024-04-25T10:26:09.782899+0000","last_active":"2024-04-25T10:26:10.778164+0000","last_peered":"2024-04-25T10:26:10.778164+0000","last_clean":"2024-04-25T10:26:10.778164+0000","last_became_active":"2024-04-25T10:26:09.781870+0000","last_became_peered":"2024-04-25T10:26:09.781870+0000","last_unstale":"2024-04-25T10:26:10.778164+0000","last_undegraded":"2024-04-25T10:26:10.778164+0000","last_fullsized":"2024-04-25T10:26:10.778164+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:37.116951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760337+0000","last_change":"2024-04-25T10:26:09.782249+0000","last_active":"2024-04-25T10:26:10.760337+0000","last_peered":"2024-04-25T10:26:10.760337+0000","last_clean":"2024-04-25T10:26:10.760337+0000","last_became_active":"2024-04-25T10:26:09.781962+0000","last_became_peered":"2024-04-25T10:26:09.781962+0000","last_unstale":"2024-04-25T10:26:10.760337+0000","last_undegraded":"2024-04-25T10:26:10.760337+0000","last_fullsized":"2024-04-25T10:26:10.760337+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.040552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760094+0000","last_change":"2024-04-25T10:26:09.787262+0000","last_active":"2024-04-25T10:26:10.760094+0000","last_peered":"2024-04-25T10:26:10.760094+0000","last_clean":"2024-04-25T10:26:10.760094+0000","last_became_active":"2024-04-25T10:26:09.786838+0000","last_became_peered":"2024-04-25T10:26:09.786838+0000","last_unstale":"2024-04-25T10:26:10.760094+0000","last_undegraded":"2024-04-25T10:26:10.760094+0000","last_fullsized":"2024-04-25T10:26:10.760094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:00.455133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760654+0000","last_change":"2024-04-25T10:26:09.769261+0000","last_active":"2024-04-25T10:26:10.760654+0000","last_peered":"2024-04-25T10:26:10.760654+0000","last_clean":"2024-04-25T10:26:10.760654+0000","last_became_active":"2024-04-25T10:26:09.768281+0000","last_became_peered":"2024-04-25T10:26:09.768281+0000","last_unstale":"2024-04-25T10:26:10.760654+0000","last_undegraded":"2024-04-25T10:26:10.760654+0000","last_fullsized":"2024-04-25T10:26:10.760654+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:30.066167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.772288+0000","last_change":"2024-04-25T10:26:09.770663+0000","last_active":"2024-04-25T10:26:09.772288+0000","last_peered":"2024-04-25T10:26:09.772288+0000","last_clean":"2024-04-25T10:26:09.772288+0000","last_became_active":"2024-04-25T10:26:09.770410+0000","last_became_peered":"2024-04-25T10:26:09.770410+0000","last_unstale":"2024-04-25T10:26:09.772288+0000","last_undegraded":"2024-04-25T10:26:09.772288+0000","last_fullsized":"2024-04-25T10:26:09.772288+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:58.878870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760213+0000","last_change":"2024-04-25T10:26:09.781858+0000","last_active":"2024-04-25T10:26:10.760213+0000","last_peered":"2024-04-25T10:26:10.760213+0000","last_clean":"2024-04-25T10:26:10.760213+0000","last_became_active":"2024-04-25T10:26:09.781358+0000","last_became_peered":"2024-04-25T10:26:09.781358+0000","last_unstale":"2024-04-25T10:26:10.760213+0000","last_undegraded":"2024-04-25T10:26:10.760213+0000","last_fullsized":"2024-04-25T10:26:10.760213+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:24.976889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761086+0000","last_change":"2024-04-25T10:26:09.767788+0000","last_active":"2024-04-25T10:26:10.761086+0000","last_peered":"2024-04-25T10:26:10.761086+0000","last_clean":"2024-04-25T10:26:10.761086+0000","last_became_active":"2024-04-25T10:26:09.767509+0000","last_became_peered":"2024-04-25T10:26:09.767509+0000","last_unstale":"2024-04-25T10:26:10.761086+0000","last_undegraded":"2024-04-25T10:26:10.761086+0000","last_fullsized":"2024-04-25T10:26:10.761086+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:18.288245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758577+0000","last_change":"2024-04-25T10:26:09.776387+0000","last_active":"2024-04-25T10:26:10.758577+0000","last_peered":"2024-04-25T10:26:10.758577+0000","last_clean":"2024-04-25T10:26:10.758577+0000","last_became_active":"2024-04-25T10:26:09.776083+0000","last_became_peered":"2024-04-25T10:26:09.776083+0000","last_unstale":"2024-04-25T10:26:10.758577+0000","last_undegraded":"2024-04-25T10:26:10.758577+0000","last_fullsized":"2024-04-25T10:26:10.758577+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:55.464088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.779880+0000","last_change":"2024-04-25T10:26:09.778279+0000","last_active":"2024-04-25T10:26:09.779880+0000","last_peered":"2024-04-25T10:26:09.779880+0000","last_clean":"2024-04-25T10:26:09.779880+0000","last_became_active":"2024-04-25T10:26:09.778038+0000","last_became_peered":"2024-04-25T10:26:09.778038+0000","last_unstale":"2024-04-25T10:26:09.779880+0000","last_undegraded":"2024-04-25T10:26:09.779880+0000","last_fullsized":"2024-04-25T10:26:09.779880+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.797740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758311+0000","last_change":"2024-04-25T10:26:09.773194+0000","last_active":"2024-04-25T10:26:10.758311+0000","last_peered":"2024-04-25T10:26:10.758311+0000","last_clean":"2024-04-25T10:26:10.758311+0000","last_became_active":"2024-04-25T10:26:09.772105+0000","last_became_peered":"2024-04-25T10:26:09.772105+0000","last_unstale":"2024-04-25T10:26:10.758311+0000","last_undegraded":"2024-04-25T10:26:10.758311+0000","last_fullsized":"2024-04-25T10:26:10.758311+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:39.352728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.776647+0000","last_change":"2024-04-25T10:26:09.773893+0000","last_active":"2024-04-25T10:26:09.776647+0000","last_peered":"2024-04-25T10:26:09.776647+0000","last_clean":"2024-04-25T10:26:09.776647+0000","last_became_active":"2024-04-25T10:26:09.773448+0000","last_became_peered":"2024-04-25T10:26:09.773448+0000","last_unstale":"2024-04-25T10:26:09.776647+0000","last_undegraded":"2024-04-25T10:26:09.776647+0000","last_fullsized":"2024-04-25T10:26:09.776647+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:56.644989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786822+0000","last_change":"2024-04-25T10:26:09.785528+0000","last_active":"2024-04-25T10:26:09.786822+0000","last_peered":"2024-04-25T10:26:09.786822+0000","last_clean":"2024-04-25T10:26:09.786822+0000","last_became_active":"2024-04-25T10:26:09.785242+0000","last_became_peered":"2024-04-25T10:26:09.785242+0000","last_unstale":"2024-04-25T10:26:09.786822+0000","last_undegraded":"2024-04-25T10:26:09.786822+0000","last_fullsized":"2024-04-25T10:26:09.786822+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:49.122787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777786+0000","last_change":"2024-04-25T10:26:09.781282+0000","last_active":"2024-04-25T10:26:10.777786+0000","last_peered":"2024-04-25T10:26:10.777786+0000","last_clean":"2024-04-25T10:26:10.777786+0000","last_became_active":"2024-04-25T10:26:09.780997+0000","last_became_peered":"2024-04-25T10:26:09.780997+0000","last_unstale":"2024-04-25T10:26:10.777786+0000","last_undegraded":"2024-04-25T10:26:10.777786+0000","last_fullsized":"2024-04-25T10:26:10.777786+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:55.464088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.766151+0000","last_change":"2024-04-25T10:26:09.764617+0000","last_active":"2024-04-25T10:26:09.766151+0000","last_peered":"2024-04-25T10:26:09.766151+0000","last_clean":"2024-04-25T10:26:09.766151+0000","last_became_active":"2024-04-25T10:26:09.764279+0000","last_became_peered":"2024-04-25T10:26:09.764279+0000","last_unstale":"2024-04-25T10:26:09.766151+0000","last_undegraded":"2024-04-25T10:26:09.766151+0000","last_fullsized":"2024-04-25T10:26:09.766151+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:00.455133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778334+0000","last_change":"2024-04-25T10:26:09.785894+0000","last_active":"2024-04-25T10:26:10.778334+0000","last_peered":"2024-04-25T10:26:10.778334+0000","last_clean":"2024-04-25T10:26:10.778334+0000","last_became_active":"2024-04-25T10:26:09.785249+0000","last_became_peered":"2024-04-25T10:26:09.785249+0000","last_unstale":"2024-04-25T10:26:10.778334+0000","last_undegraded":"2024-04-25T10:26:10.778334+0000","last_fullsized":"2024-04-25T10:26:10.778334+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:42.732330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779734+0000","last_change":"2024-04-25T10:26:09.766590+0000","last_active":"2024-04-25T10:26:10.779734+0000","last_peered":"2024-04-25T10:26:10.779734+0000","last_clean":"2024-04-25T10:26:10.779734+0000","last_became_active":"2024-04-25T10:26:09.766293+0000","last_became_peered":"2024-04-25T10:26:09.766293+0000","last_unstale":"2024-04-25T10:26:10.779734+0000","last_undegraded":"2024-04-25T10:26:10.779734+0000","last_fullsized":"2024-04-25T10:26:10.779734+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:38.172419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778344+0000","last_change":"2024-04-25T10:26:09.783142+0000","last_active":"2024-04-25T10:26:10.778344+0000","last_peered":"2024-04-25T10:26:10.778344+0000","last_clean":"2024-04-25T10:26:10.778344+0000","last_became_active":"2024-04-25T10:26:09.782380+0000","last_became_peered":"2024-04-25T10:26:09.782380+0000","last_unstale":"2024-04-25T10:26:10.778344+0000","last_undegraded":"2024-04-25T10:26:10.778344+0000","last_fullsized":"2024-04-25T10:26:10.778344+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:14.181376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758464+0000","last_change":"2024-04-25T10:26:09.762518+0000","last_active":"2024-04-25T10:26:10.758464+0000","last_peered":"2024-04-25T10:26:10.758464+0000","last_clean":"2024-04-25T10:26:10.758464+0000","last_became_active":"2024-04-25T10:26:09.762265+0000","last_became_peered":"2024-04-25T10:26:09.762265+0000","last_unstale":"2024-04-25T10:26:10.758464+0000","last_undegraded":"2024-04-25T10:26:10.758464+0000","last_fullsized":"2024-04-25T10:26:10.758464+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:22.466158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760891+0000","last_change":"2024-04-25T10:26:09.781240+0000","last_active":"2024-04-25T10:26:10.760891+0000","last_peered":"2024-04-25T10:26:10.760891+0000","last_clean":"2024-04-25T10:26:10.760891+0000","last_became_active":"2024-04-25T10:26:09.780975+0000","last_became_peered":"2024-04-25T10:26:09.780975+0000","last_unstale":"2024-04-25T10:26:10.760891+0000","last_undegraded":"2024-04-25T10:26:10.760891+0000","last_fullsized":"2024-04-25T10:26:10.760891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:48.946289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777867+0000","last_change":"2024-04-25T10:26:09.784366+0000","last_active":"2024-04-25T10:26:10.777867+0000","last_peered":"2024-04-25T10:26:10.777867+0000","last_clean":"2024-04-25T10:26:10.777867+0000","last_became_active":"2024-04-25T10:26:09.784030+0000","last_became_peered":"2024-04-25T10:26:09.784030+0000","last_unstale":"2024-04-25T10:26:10.777867+0000","last_undegraded":"2024-04-25T10:26:10.777867+0000","last_fullsized":"2024-04-25T10:26:10.777867+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:16.501974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.767976+0000","last_change":"2024-04-25T10:26:09.766966+0000","last_active":"2024-04-25T10:26:09.767976+0000","last_peered":"2024-04-25T10:26:09.767976+0000","last_clean":"2024-04-25T10:26:09.767976+0000","last_became_active":"2024-04-25T10:26:09.766229+0000","last_became_peered":"2024-04-25T10:26:09.766229+0000","last_unstale":"2024-04-25T10:26:09.767976+0000","last_undegraded":"2024-04-25T10:26:09.767976+0000","last_fullsized":"2024-04-25T10:26:09.767976+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.040552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.780154+0000","last_change":"2024-04-25T10:26:09.778592+0000","last_active":"2024-04-25T10:26:09.780154+0000","last_peered":"2024-04-25T10:26:09.780154+0000","last_clean":"2024-04-25T10:26:09.780154+0000","last_became_active":"2024-04-25T10:26:09.778322+0000","last_became_peered":"2024-04-25T10:26:09.778322+0000","last_unstale":"2024-04-25T10:26:09.780154+0000","last_undegraded":"2024-04-25T10:26:09.780154+0000","last_fullsized":"2024-04-25T10:26:09.780154+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:18.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796741+0000","last_change":"2024-04-25T10:26:09.794892+0000","last_active":"2024-04-25T10:26:09.796741+0000","last_peered":"2024-04-25T10:26:09.796741+0000","last_clean":"2024-04-25T10:26:09.796741+0000","last_became_active":"2024-04-25T10:26:09.794574+0000","last_became_peered":"2024-04-25T10:26:09.794574+0000","last_unstale":"2024-04-25T10:26:09.796741+0000","last_undegraded":"2024-04-25T10:26:09.796741+0000","last_fullsized":"2024-04-25T10:26:09.796741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.337257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"186'4","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786186+0000","last_change":"2024-04-25T10:26:08.771499+0000","last_active":"2024-04-25T10:26:09.786186+0000","last_peered":"2024-04-25T10:26:09.786186+0000","last_clean":"2024-04-25T10:26:09.786186+0000","last_became_active":"2024-04-25T10:26:08.770367+0000","last_became_peered":"2024-04-25T10:26:08.770367+0000","last_unstale":"2024-04-25T10:26:09.786186+0000","last_undegraded":"2024-04-25T10:26:09.786186+0000","last_fullsized":"2024-04-25T10:26:09.786186+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:26.131803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779804+0000","last_change":"2024-04-25T10:26:09.766971+0000","last_active":"2024-04-25T10:26:10.779804+0000","last_peered":"2024-04-25T10:26:10.779804+0000","last_clean":"2024-04-25T10:26:10.779804+0000","last_became_active":"2024-04-25T10:26:09.766697+0000","last_became_peered":"2024-04-25T10:26:09.766697+0000","last_unstale":"2024-04-25T10:26:10.779804+0000","last_undegraded":"2024-04-25T10:26:10.779804+0000","last_fullsized":"2024-04-25T10:26:10.779804+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:03.864597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761252+0000","last_change":"2024-04-25T10:26:08.772936+0000","last_active":"2024-04-25T10:26:10.761252+0000","last_peered":"2024-04-25T10:26:10.761252+0000","last_clean":"2024-04-25T10:26:10.761252+0000","last_became_active":"2024-04-25T10:26:08.772593+0000","last_became_peered":"2024-04-25T10:26:08.772593+0000","last_unstale":"2024-04-25T10:26:10.761252+0000","last_undegraded":"2024-04-25T10:26:10.761252+0000","last_fullsized":"2024-04-25T10:26:10.761252+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:29.059222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761275+0000","last_change":"2024-04-25T10:26:09.790769+0000","last_active":"2024-04-25T10:26:10.761275+0000","last_peered":"2024-04-25T10:26:10.761275+0000","last_clean":"2024-04-25T10:26:10.761275+0000","last_became_active":"2024-04-25T10:26:09.790216+0000","last_became_peered":"2024-04-25T10:26:09.790216+0000","last_unstale":"2024-04-25T10:26:10.761275+0000","last_undegraded":"2024-04-25T10:26:10.761275+0000","last_fullsized":"2024-04-25T10:26:10.761275+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:27.889360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760383+0000","last_change":"2024-04-25T10:26:08.769155+0000","last_active":"2024-04-25T10:26:10.760383+0000","last_peered":"2024-04-25T10:26:10.760383+0000","last_clean":"2024-04-25T10:26:10.760383+0000","last_became_active":"2024-04-25T10:26:08.768892+0000","last_became_peered":"2024-04-25T10:26:08.768892+0000","last_unstale":"2024-04-25T10:26:10.760383+0000","last_undegraded":"2024-04-25T10:26:10.760383+0000","last_fullsized":"2024-04-25T10:26:10.760383+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:17.281301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778096+0000","last_change":"2024-04-25T10:26:09.784456+0000","last_active":"2024-04-25T10:26:10.778096+0000","last_peered":"2024-04-25T10:26:10.778096+0000","last_clean":"2024-04-25T10:26:10.778096+0000","last_became_active":"2024-04-25T10:26:09.784183+0000","last_became_peered":"2024-04-25T10:26:09.784183+0000","last_unstale":"2024-04-25T10:26:10.778096+0000","last_undegraded":"2024-04-25T10:26:10.778096+0000","last_fullsized":"2024-04-25T10:26:10.778096+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.775514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779612+0000","last_change":"2024-04-25T10:26:08.769095+0000","last_active":"2024-04-25T10:26:10.779612+0000","last_peered":"2024-04-25T10:26:10.779612+0000","last_clean":"2024-04-25T10:26:10.779612+0000","last_became_active":"2024-04-25T10:26:08.768849+0000","last_became_peered":"2024-04-25T10:26:08.768849+0000","last_unstale":"2024-04-25T10:26:10.779612+0000","last_undegraded":"2024-04-25T10:26:10.779612+0000","last_fullsized":"2024-04-25T10:26:10.779612+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:21.263935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.784690+0000","last_change":"2024-04-25T10:26:09.782189+0000","last_active":"2024-04-25T10:26:09.784690+0000","last_peered":"2024-04-25T10:26:09.784690+0000","last_clean":"2024-04-25T10:26:09.784690+0000","last_became_active":"2024-04-25T10:26:09.781908+0000","last_became_peered":"2024-04-25T10:26:09.781908+0000","last_unstale":"2024-04-25T10:26:09.784690+0000","last_undegraded":"2024-04-25T10:26:09.784690+0000","last_fullsized":"2024-04-25T10:26:09.784690+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.337257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"186'1","reported_seq":13,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796182+0000","last_change":"2024-04-25T10:26:08.775857+0000","last_active":"2024-04-25T10:26:09.796182+0000","last_peered":"2024-04-25T10:26:09.796182+0000","last_clean":"2024-04-25T10:26:09.796182+0000","last_became_active":"2024-04-25T10:26:08.775560+0000","last_became_peered":"2024-04-25T10:26:08.775560+0000","last_unstale":"2024-04-25T10:26:09.796182+0000","last_undegraded":"2024-04-25T10:26:09.796182+0000","last_fullsized":"2024-04-25T10:26:09.796182+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:29.718152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758253+0000","last_change":"2024-04-25T10:26:09.762282+0000","last_active":"2024-04-25T10:26:10.758253+0000","last_peered":"2024-04-25T10:26:10.758253+0000","last_clean":"2024-04-25T10:26:10.758253+0000","last_became_active":"2024-04-25T10:26:09.761990+0000","last_became_peered":"2024-04-25T10:26:09.761990+0000","last_unstale":"2024-04-25T10:26:10.758253+0000","last_undegraded":"2024-04-25T10:26:10.758253+0000","last_fullsized":"2024-04-25T10:26:10.758253+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:06.230346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.759812+0000","last_change":"2024-04-25T10:26:08.770209+0000","last_active":"2024-04-25T10:26:10.759812+0000","last_peered":"2024-04-25T10:26:10.759812+0000","last_clean":"2024-04-25T10:26:10.759812+0000","last_became_active":"2024-04-25T10:26:08.769565+0000","last_became_peered":"2024-04-25T10:26:08.769565+0000","last_unstale":"2024-04-25T10:26:10.759812+0000","last_undegraded":"2024-04-25T10:26:10.759812+0000","last_fullsized":"2024-04-25T10:26:10.759812+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:36.202336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760520+0000","last_change":"2024-04-25T10:26:09.791042+0000","last_active":"2024-04-25T10:26:10.760520+0000","last_peered":"2024-04-25T10:26:10.760520+0000","last_clean":"2024-04-25T10:26:10.760520+0000","last_became_active":"2024-04-25T10:26:09.790778+0000","last_became_peered":"2024-04-25T10:26:09.790778+0000","last_unstale":"2024-04-25T10:26:10.760520+0000","last_undegraded":"2024-04-25T10:26:10.760520+0000","last_fullsized":"2024-04-25T10:26:10.760520+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:56.644989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758196+0000","last_change":"2024-04-25T10:26:08.772494+0000","last_active":"2024-04-25T10:26:10.758196+0000","last_peered":"2024-04-25T10:26:10.758196+0000","last_clean":"2024-04-25T10:26:10.758196+0000","last_became_active":"2024-04-25T10:26:08.772223+0000","last_became_peered":"2024-04-25T10:26:08.772223+0000","last_unstale":"2024-04-25T10:26:10.758196+0000","last_undegraded":"2024-04-25T10:26:10.758196+0000","last_fullsized":"2024-04-25T10:26:10.758196+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:57.994885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760586+0000","last_change":"2024-04-25T10:26:09.789318+0000","last_active":"2024-04-25T10:26:10.760586+0000","last_peered":"2024-04-25T10:26:10.760586+0000","last_clean":"2024-04-25T10:26:10.760586+0000","last_became_active":"2024-04-25T10:26:09.789030+0000","last_became_peered":"2024-04-25T10:26:09.789030+0000","last_unstale":"2024-04-25T10:26:10.760586+0000","last_undegraded":"2024-04-25T10:26:10.760586+0000","last_fullsized":"2024-04-25T10:26:10.760586+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.797740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778449+0000","last_change":"2024-04-25T10:26:08.773430+0000","last_active":"2024-04-25T10:26:10.778449+0000","last_peered":"2024-04-25T10:26:10.778449+0000","last_clean":"2024-04-25T10:26:10.778449+0000","last_became_active":"2024-04-25T10:26:08.771655+0000","last_became_peered":"2024-04-25T10:26:08.771655+0000","last_unstale":"2024-04-25T10:26:10.778449+0000","last_undegraded":"2024-04-25T10:26:10.778449+0000","last_fullsized":"2024-04-25T10:26:10.778449+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:19.319975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778371+0000","last_change":"2024-04-25T10:26:09.786102+0000","last_active":"2024-04-25T10:26:10.778371+0000","last_peered":"2024-04-25T10:26:10.778371+0000","last_clean":"2024-04-25T10:26:10.778371+0000","last_became_active":"2024-04-25T10:26:09.785620+0000","last_became_peered":"2024-04-25T10:26:09.785620+0000","last_unstale":"2024-04-25T10:26:10.778371+0000","last_undegraded":"2024-04-25T10:26:10.778371+0000","last_fullsized":"2024-04-25T10:26:10.778371+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:05.164114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.770029+0000","last_change":"2024-04-25T10:26:08.773287+0000","last_active":"2024-04-25T10:26:09.770029+0000","last_peered":"2024-04-25T10:26:09.770029+0000","last_clean":"2024-04-25T10:26:09.770029+0000","last_became_active":"2024-04-25T10:26:08.772947+0000","last_became_peered":"2024-04-25T10:26:08.772947+0000","last_unstale":"2024-04-25T10:26:09.770029+0000","last_undegraded":"2024-04-25T10:26:09.770029+0000","last_fullsized":"2024-04-25T10:26:09.770029+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:57.994885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761185+0000","last_change":"2024-04-25T10:26:09.779115+0000","last_active":"2024-04-25T10:26:10.761185+0000","last_peered":"2024-04-25T10:26:10.761185+0000","last_clean":"2024-04-25T10:26:10.761185+0000","last_became_active":"2024-04-25T10:26:09.778776+0000","last_became_peered":"2024-04-25T10:26:09.778776+0000","last_unstale":"2024-04-25T10:26:10.761185+0000","last_undegraded":"2024-04-25T10:26:10.761185+0000","last_fullsized":"2024-04-25T10:26:10.761185+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.515662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"186'3","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.787775+0000","last_change":"2024-04-25T10:26:08.765902+0000","last_active":"2024-04-25T10:26:09.787775+0000","last_peered":"2024-04-25T10:26:09.787775+0000","last_clean":"2024-04-25T10:26:09.787775+0000","last_became_active":"2024-04-25T10:26:08.765627+0000","last_became_peered":"2024-04-25T10:26:08.765627+0000","last_unstale":"2024-04-25T10:26:09.787775+0000","last_undegraded":"2024-04-25T10:26:09.787775+0000","last_fullsized":"2024-04-25T10:26:09.787775+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:56.313798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778160+0000","last_change":"2024-04-25T10:26:09.788367+0000","last_active":"2024-04-25T10:26:10.778160+0000","last_peered":"2024-04-25T10:26:10.778160+0000","last_clean":"2024-04-25T10:26:10.778160+0000","last_became_active":"2024-04-25T10:26:09.788088+0000","last_became_peered":"2024-04-25T10:26:09.788088+0000","last_unstale":"2024-04-25T10:26:10.778160+0000","last_undegraded":"2024-04-25T10:26:10.778160+0000","last_fullsized":"2024-04-25T10:26:10.778160+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:51.229311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"186'6","reported_seq":20,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758522+0000","last_change":"2024-04-25T10:26:08.768844+0000","last_active":"2024-04-25T10:26:10.758522+0000","last_peered":"2024-04-25T10:26:10.758522+0000","last_clean":"2024-04-25T10:26:10.758522+0000","last_became_active":"2024-04-25T10:26:08.768570+0000","last_became_peered":"2024-04-25T10:26:08.768570+0000","last_unstale":"2024-04-25T10:26:10.758522+0000","last_undegraded":"2024-04-25T10:26:10.758522+0000","last_fullsized":"2024-04-25T10:26:10.758522+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:17.666693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.778678+0000","last_change":"2024-04-25T10:26:09.774293+0000","last_active":"2024-04-25T10:26:09.778678+0000","last_peered":"2024-04-25T10:26:09.778678+0000","last_clean":"2024-04-25T10:26:09.778678+0000","last_became_active":"2024-04-25T10:26:09.773129+0000","last_became_peered":"2024-04-25T10:26:09.773129+0000","last_unstale":"2024-04-25T10:26:09.778678+0000","last_undegraded":"2024-04-25T10:26:09.778678+0000","last_fullsized":"2024-04-25T10:26:09.778678+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:17.101525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779613+0000","last_change":"2024-04-25T10:26:08.772605+0000","last_active":"2024-04-25T10:26:10.779613+0000","last_peered":"2024-04-25T10:26:10.779613+0000","last_clean":"2024-04-25T10:26:10.779613+0000","last_became_active":"2024-04-25T10:26:08.772343+0000","last_became_peered":"2024-04-25T10:26:08.772343+0000","last_unstale":"2024-04-25T10:26:10.779613+0000","last_undegraded":"2024-04-25T10:26:10.779613+0000","last_fullsized":"2024-04-25T10:26:10.779613+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.422079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786145+0000","last_change":"2024-04-25T10:26:09.780490+0000","last_active":"2024-04-25T10:26:09.786145+0000","last_peered":"2024-04-25T10:26:09.786145+0000","last_clean":"2024-04-25T10:26:09.786145+0000","last_became_active":"2024-04-25T10:26:09.780172+0000","last_became_peered":"2024-04-25T10:26:09.780172+0000","last_unstale":"2024-04-25T10:26:09.786145+0000","last_undegraded":"2024-04-25T10:26:09.786145+0000","last_fullsized":"2024-04-25T10:26:09.786145+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:33.855220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.770467+0000","last_change":"2024-04-25T10:26:08.773593+0000","last_active":"2024-04-25T10:26:09.770467+0000","last_peered":"2024-04-25T10:26:09.770467+0000","last_clean":"2024-04-25T10:26:09.770467+0000","last_became_active":"2024-04-25T10:26:08.771018+0000","last_became_peered":"2024-04-25T10:26:08.771018+0000","last_unstale":"2024-04-25T10:26:09.770467+0000","last_undegraded":"2024-04-25T10:26:09.770467+0000","last_fullsized":"2024-04-25T10:26:09.770467+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:19.319975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.787626+0000","last_change":"2024-04-25T10:26:09.782454+0000","last_active":"2024-04-25T10:26:09.787626+0000","last_peered":"2024-04-25T10:26:09.787626+0000","last_clean":"2024-04-25T10:26:09.787626+0000","last_became_active":"2024-04-25T10:26:09.782184+0000","last_became_peered":"2024-04-25T10:26:09.782184+0000","last_unstale":"2024-04-25T10:26:09.787626+0000","last_undegraded":"2024-04-25T10:26:09.787626+0000","last_fullsized":"2024-04-25T10:26:09.787626+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:24.976889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.761165+0000","last_change":"2024-04-25T10:26:08.767366+0000","last_active":"2024-04-25T10:26:09.761165+0000","last_peered":"2024-04-25T10:26:09.761165+0000","last_clean":"2024-04-25T10:26:09.761165+0000","last_became_active":"2024-04-25T10:26:08.767109+0000","last_became_peered":"2024-04-25T10:26:08.767109+0000","last_unstale":"2024-04-25T10:26:09.761165+0000","last_undegraded":"2024-04-25T10:26:09.761165+0000","last_fullsized":"2024-04-25T10:26:09.761165+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:29.718152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760122+0000","last_change":"2024-04-25T10:26:09.770551+0000","last_active":"2024-04-25T10:26:10.760122+0000","last_peered":"2024-04-25T10:26:10.760122+0000","last_clean":"2024-04-25T10:26:10.760122+0000","last_became_active":"2024-04-25T10:26:09.770271+0000","last_became_peered":"2024-04-25T10:26:09.770271+0000","last_unstale":"2024-04-25T10:26:10.760122+0000","last_undegraded":"2024-04-25T10:26:10.760122+0000","last_fullsized":"2024-04-25T10:26:10.760122+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:20.272652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"186'8","reported_seq":22,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760269+0000","last_change":"2024-04-25T10:26:08.768259+0000","last_active":"2024-04-25T10:26:10.760269+0000","last_peered":"2024-04-25T10:26:10.760269+0000","last_clean":"2024-04-25T10:26:10.760269+0000","last_became_active":"2024-04-25T10:26:08.767942+0000","last_became_peered":"2024-04-25T10:26:08.767942+0000","last_unstale":"2024-04-25T10:26:10.760269+0000","last_undegraded":"2024-04-25T10:26:10.760269+0000","last_fullsized":"2024-04-25T10:26:10.760269+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:47.939344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779852+0000","last_change":"2024-04-25T10:26:09.768006+0000","last_active":"2024-04-25T10:26:10.779852+0000","last_peered":"2024-04-25T10:26:10.779852+0000","last_clean":"2024-04-25T10:26:10.779852+0000","last_became_active":"2024-04-25T10:26:09.767614+0000","last_became_peered":"2024-04-25T10:26:09.767614+0000","last_unstale":"2024-04-25T10:26:10.779852+0000","last_undegraded":"2024-04-25T10:26:10.779852+0000","last_fullsized":"2024-04-25T10:26:10.779852+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:36.718139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"186'1","reported_seq":13,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.796263+0000","last_change":"2024-04-25T10:26:08.778204+0000","last_active":"2024-04-25T10:26:09.796263+0000","last_peered":"2024-04-25T10:26:09.796263+0000","last_clean":"2024-04-25T10:26:09.796263+0000","last_became_active":"2024-04-25T10:26:08.777927+0000","last_became_peered":"2024-04-25T10:26:08.777927+0000","last_unstale":"2024-04-25T10:26:09.796263+0000","last_undegraded":"2024-04-25T10:26:09.796263+0000","last_fullsized":"2024-04-25T10:26:09.796263+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:24.841692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758576+0000","last_change":"2024-04-25T10:26:09.789862+0000","last_active":"2024-04-25T10:26:10.758576+0000","last_peered":"2024-04-25T10:26:10.758576+0000","last_clean":"2024-04-25T10:26:10.758576+0000","last_became_active":"2024-04-25T10:26:09.789602+0000","last_became_peered":"2024-04-25T10:26:09.789602+0000","last_unstale":"2024-04-25T10:26:10.758576+0000","last_undegraded":"2024-04-25T10:26:10.758576+0000","last_fullsized":"2024-04-25T10:26:10.758576+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:13.652223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"186'2","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.789078+0000","last_change":"2024-04-25T10:26:08.774643+0000","last_active":"2024-04-25T10:26:09.789078+0000","last_peered":"2024-04-25T10:26:09.789078+0000","last_clean":"2024-04-25T10:26:09.789078+0000","last_became_active":"2024-04-25T10:26:08.774385+0000","last_became_peered":"2024-04-25T10:26:08.774385+0000","last_unstale":"2024-04-25T10:26:09.789078+0000","last_undegraded":"2024-04-25T10:26:09.789078+0000","last_fullsized":"2024-04-25T10:26:09.789078+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:17.666693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.794250+0000","last_change":"2024-04-25T10:26:09.789044+0000","last_active":"2024-04-25T10:26:09.794250+0000","last_peered":"2024-04-25T10:26:09.794250+0000","last_clean":"2024-04-25T10:26:09.794250+0000","last_became_active":"2024-04-25T10:26:09.784272+0000","last_became_peered":"2024-04-25T10:26:09.784272+0000","last_unstale":"2024-04-25T10:26:09.794250+0000","last_undegraded":"2024-04-25T10:26:09.794250+0000","last_fullsized":"2024-04-25T10:26:09.794250+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:43.468414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778305+0000","last_change":"2024-04-25T10:26:08.772628+0000","last_active":"2024-04-25T10:26:10.778305+0000","last_peered":"2024-04-25T10:26:10.778305+0000","last_clean":"2024-04-25T10:26:10.778305+0000","last_became_active":"2024-04-25T10:26:08.771096+0000","last_became_peered":"2024-04-25T10:26:08.771096+0000","last_unstale":"2024-04-25T10:26:10.778305+0000","last_undegraded":"2024-04-25T10:26:10.778305+0000","last_fullsized":"2024-04-25T10:26:10.778305+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:17.666693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792886+0000","last_change":"2024-04-25T10:26:09.784938+0000","last_active":"2024-04-25T10:26:09.792886+0000","last_peered":"2024-04-25T10:26:09.792886+0000","last_clean":"2024-04-25T10:26:09.792886+0000","last_became_active":"2024-04-25T10:26:09.784636+0000","last_became_peered":"2024-04-25T10:26:09.784636+0000","last_unstale":"2024-04-25T10:26:09.792886+0000","last_undegraded":"2024-04-25T10:26:09.792886+0000","last_fullsized":"2024-04-25T10:26:09.792886+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:56.644989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.758565+0000","last_change":"2024-04-25T10:26:08.775364+0000","last_active":"2024-04-25T10:26:10.758565+0000","last_peered":"2024-04-25T10:26:10.758565+0000","last_clean":"2024-04-25T10:26:10.758565+0000","last_became_active":"2024-04-25T10:26:08.774983+0000","last_became_peered":"2024-04-25T10:26:08.774983+0000","last_unstale":"2024-04-25T10:26:10.758565+0000","last_undegraded":"2024-04-25T10:26:10.758565+0000","last_fullsized":"2024-04-25T10:26:10.758565+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:16.584614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.765035+0000","last_change":"2024-04-25T10:26:09.763794+0000","last_active":"2024-04-25T10:26:09.765035+0000","last_peered":"2024-04-25T10:26:09.765035+0000","last_clean":"2024-04-25T10:26:09.765035+0000","last_became_active":"2024-04-25T10:26:09.763530+0000","last_became_peered":"2024-04-25T10:26:09.763530+0000","last_unstale":"2024-04-25T10:26:09.765035+0000","last_undegraded":"2024-04-25T10:26:09.765035+0000","last_fullsized":"2024-04-25T10:26:09.765035+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:33.855220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"186'2","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.797014+0000","last_change":"2024-04-25T10:26:08.778795+0000","last_active":"2024-04-25T10:26:09.797014+0000","last_peered":"2024-04-25T10:26:09.797014+0000","last_clean":"2024-04-25T10:26:09.797014+0000","last_became_active":"2024-04-25T10:26:08.778474+0000","last_became_peered":"2024-04-25T10:26:08.778474+0000","last_unstale":"2024-04-25T10:26:09.797014+0000","last_undegraded":"2024-04-25T10:26:09.797014+0000","last_fullsized":"2024-04-25T10:26:09.797014+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:56.313798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.792994+0000","last_change":"2024-04-25T10:26:09.786159+0000","last_active":"2024-04-25T10:26:09.792994+0000","last_peered":"2024-04-25T10:26:09.792994+0000","last_clean":"2024-04-25T10:26:09.792994+0000","last_became_active":"2024-04-25T10:26:09.785886+0000","last_became_peered":"2024-04-25T10:26:09.785886+0000","last_unstale":"2024-04-25T10:26:09.792994+0000","last_undegraded":"2024-04-25T10:26:09.792994+0000","last_fullsized":"2024-04-25T10:26:09.792994+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.797740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760750+0000","last_change":"2024-04-25T10:26:08.767339+0000","last_active":"2024-04-25T10:26:10.760750+0000","last_peered":"2024-04-25T10:26:10.760750+0000","last_clean":"2024-04-25T10:26:10.760750+0000","last_became_active":"2024-04-25T10:26:08.766821+0000","last_became_peered":"2024-04-25T10:26:08.766821+0000","last_unstale":"2024-04-25T10:26:10.760750+0000","last_undegraded":"2024-04-25T10:26:10.760750+0000","last_fullsized":"2024-04-25T10:26:10.760750+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:43.273630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779401+0000","last_change":"2024-04-25T10:26:09.764101+0000","last_active":"2024-04-25T10:26:10.779401+0000","last_peered":"2024-04-25T10:26:10.779401+0000","last_clean":"2024-04-25T10:26:10.779401+0000","last_became_active":"2024-04-25T10:26:09.763856+0000","last_became_peered":"2024-04-25T10:26:09.763856+0000","last_unstale":"2024-04-25T10:26:10.779401+0000","last_undegraded":"2024-04-25T10:26:10.779401+0000","last_fullsized":"2024-04-25T10:26:10.779401+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:21.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760070+0000","last_change":"2024-04-25T10:26:08.775664+0000","last_active":"2024-04-25T10:26:10.760070+0000","last_peered":"2024-04-25T10:26:10.760070+0000","last_clean":"2024-04-25T10:26:10.760070+0000","last_became_active":"2024-04-25T10:26:08.775347+0000","last_became_peered":"2024-04-25T10:26:08.775347+0000","last_unstale":"2024-04-25T10:26:10.760070+0000","last_undegraded":"2024-04-25T10:26:10.760070+0000","last_fullsized":"2024-04-25T10:26:10.760070+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:20.510597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.780498+0000","last_change":"2024-04-25T10:26:09.779977+0000","last_active":"2024-04-25T10:26:09.780498+0000","last_peered":"2024-04-25T10:26:09.780498+0000","last_clean":"2024-04-25T10:26:09.780498+0000","last_became_active":"2024-04-25T10:26:09.779591+0000","last_became_peered":"2024-04-25T10:26:09.779591+0000","last_unstale":"2024-04-25T10:26:09.780498+0000","last_undegraded":"2024-04-25T10:26:09.780498+0000","last_fullsized":"2024-04-25T10:26:09.780498+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:43.468414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.777929+0000","last_change":"2024-04-25T10:26:08.779454+0000","last_active":"2024-04-25T10:26:09.777929+0000","last_peered":"2024-04-25T10:26:09.777929+0000","last_clean":"2024-04-25T10:26:09.777929+0000","last_became_active":"2024-04-25T10:26:08.779132+0000","last_became_peered":"2024-04-25T10:26:08.779132+0000","last_unstale":"2024-04-25T10:26:09.777929+0000","last_undegraded":"2024-04-25T10:26:09.777929+0000","last_fullsized":"2024-04-25T10:26:09.777929+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:04.508717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.780283+0000","last_change":"2024-04-25T10:26:09.779557+0000","last_active":"2024-04-25T10:26:09.780283+0000","last_peered":"2024-04-25T10:26:09.780283+0000","last_clean":"2024-04-25T10:26:09.780283+0000","last_became_active":"2024-04-25T10:26:09.779296+0000","last_became_peered":"2024-04-25T10:26:09.779296+0000","last_unstale":"2024-04-25T10:26:09.780283+0000","last_undegraded":"2024-04-25T10:26:09.780283+0000","last_fullsized":"2024-04-25T10:26:09.780283+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:43.146461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"186'2","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.793819+0000","last_change":"2024-04-25T10:26:08.772670+0000","last_active":"2024-04-25T10:26:09.793819+0000","last_peered":"2024-04-25T10:26:09.793819+0000","last_clean":"2024-04-25T10:26:09.793819+0000","last_became_active":"2024-04-25T10:26:08.772409+0000","last_became_peered":"2024-04-25T10:26:08.772409+0000","last_unstale":"2024-04-25T10:26:09.793819+0000","last_undegraded":"2024-04-25T10:26:09.793819+0000","last_fullsized":"2024-04-25T10:26:09.793819+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:24.841692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786580+0000","last_change":"2024-04-25T10:26:09.780561+0000","last_active":"2024-04-25T10:26:09.786580+0000","last_peered":"2024-04-25T10:26:09.786580+0000","last_clean":"2024-04-25T10:26:09.786580+0000","last_became_active":"2024-04-25T10:26:09.780254+0000","last_became_peered":"2024-04-25T10:26:09.780254+0000","last_unstale":"2024-04-25T10:26:09.786580+0000","last_undegraded":"2024-04-25T10:26:09.786580+0000","last_fullsized":"2024-04-25T10:26:09.786580+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:00.455133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"187'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.779658+0000","last_change":"2024-04-25T10:26:08.773979+0000","last_active":"2024-04-25T10:26:10.779658+0000","last_peered":"2024-04-25T10:26:10.779658+0000","last_clean":"2024-04-25T10:26:10.779658+0000","last_became_active":"2024-04-25T10:26:08.773654+0000","last_became_peered":"2024-04-25T10:26:08.773654+0000","last_unstale":"2024-04-25T10:26:10.779658+0000","last_undegraded":"2024-04-25T10:26:10.779658+0000","last_fullsized":"2024-04-25T10:26:10.779658+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:57.994885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.789179+0000","last_change":"2024-04-25T10:26:09.784589+0000","last_active":"2024-04-25T10:26:09.789179+0000","last_peered":"2024-04-25T10:26:09.789179+0000","last_clean":"2024-04-25T10:26:09.789179+0000","last_became_active":"2024-04-25T10:26:09.782799+0000","last_became_peered":"2024-04-25T10:26:09.782799+0000","last_unstale":"2024-04-25T10:26:09.789179+0000","last_undegraded":"2024-04-25T10:26:09.789179+0000","last_fullsized":"2024-04-25T10:26:09.789179+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.884230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778407+0000","last_change":"2024-04-25T10:26:08.776951+0000","last_active":"2024-04-25T10:26:10.778407+0000","last_peered":"2024-04-25T10:26:10.778407+0000","last_clean":"2024-04-25T10:26:10.778407+0000","last_became_active":"2024-04-25T10:26:08.776616+0000","last_became_peered":"2024-04-25T10:26:08.776616+0000","last_unstale":"2024-04-25T10:26:10.778407+0000","last_undegraded":"2024-04-25T10:26:10.778407+0000","last_fullsized":"2024-04-25T10:26:10.778407+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:16.584614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.776552+0000","last_change":"2024-04-25T10:26:09.773061+0000","last_active":"2024-04-25T10:26:09.776552+0000","last_peered":"2024-04-25T10:26:09.776552+0000","last_clean":"2024-04-25T10:26:09.776552+0000","last_became_active":"2024-04-25T10:26:09.772816+0000","last_became_peered":"2024-04-25T10:26:09.772816+0000","last_unstale":"2024-04-25T10:26:09.776552+0000","last_undegraded":"2024-04-25T10:26:09.776552+0000","last_fullsized":"2024-04-25T10:26:09.776552+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.884230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.779430+0000","last_change":"2024-04-25T10:26:08.771906+0000","last_active":"2024-04-25T10:26:09.779430+0000","last_peered":"2024-04-25T10:26:09.779430+0000","last_clean":"2024-04-25T10:26:09.779430+0000","last_became_active":"2024-04-25T10:26:08.771643+0000","last_became_peered":"2024-04-25T10:26:08.771643+0000","last_unstale":"2024-04-25T10:26:09.779430+0000","last_undegraded":"2024-04-25T10:26:09.779430+0000","last_fullsized":"2024-04-25T10:26:09.779430+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.871193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760531+0000","last_change":"2024-04-25T10:26:09.790561+0000","last_active":"2024-04-25T10:26:10.760531+0000","last_peered":"2024-04-25T10:26:10.760531+0000","last_clean":"2024-04-25T10:26:10.760531+0000","last_became_active":"2024-04-25T10:26:09.785068+0000","last_became_peered":"2024-04-25T10:26:09.785068+0000","last_unstale":"2024-04-25T10:26:10.760531+0000","last_undegraded":"2024-04-25T10:26:10.760531+0000","last_fullsized":"2024-04-25T10:26:10.760531+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.760845+0000","last_change":"2024-04-25T10:26:08.774587+0000","last_active":"2024-04-25T10:26:10.760845+0000","last_peered":"2024-04-25T10:26:10.760845+0000","last_clean":"2024-04-25T10:26:10.760845+0000","last_became_active":"2024-04-25T10:26:08.773480+0000","last_became_peered":"2024-04-25T10:26:08.773480+0000","last_unstale":"2024-04-25T10:26:10.760845+0000","last_undegraded":"2024-04-25T10:26:10.760845+0000","last_fullsized":"2024-04-25T10:26:10.760845+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:07.134806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.786122+0000","last_change":"2024-04-25T10:26:09.776106+0000","last_active":"2024-04-25T10:26:09.786122+0000","last_peered":"2024-04-25T10:26:09.786122+0000","last_clean":"2024-04-25T10:26:09.786122+0000","last_became_active":"2024-04-25T10:26:09.775830+0000","last_became_peered":"2024-04-25T10:26:09.775830+0000","last_unstale":"2024-04-25T10:26:09.786122+0000","last_undegraded":"2024-04-25T10:26:09.786122+0000","last_fullsized":"2024-04-25T10:26:09.786122+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:18.288245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.770326+0000","last_change":"2024-04-25T10:26:08.769391+0000","last_active":"2024-04-25T10:26:09.770326+0000","last_peered":"2024-04-25T10:26:09.770326+0000","last_clean":"2024-04-25T10:26:09.770326+0000","last_became_active":"2024-04-25T10:26:08.769117+0000","last_became_peered":"2024-04-25T10:26:08.769117+0000","last_unstale":"2024-04-25T10:26:09.770326+0000","last_undegraded":"2024-04-25T10:26:09.770326+0000","last_fullsized":"2024-04-25T10:26:09.770326+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:16.584614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.787601+0000","last_change":"2024-04-25T10:26:09.783413+0000","last_active":"2024-04-25T10:26:09.787601+0000","last_peered":"2024-04-25T10:26:09.787601+0000","last_clean":"2024-04-25T10:26:09.787601+0000","last_became_active":"2024-04-25T10:26:09.780641+0000","last_became_peered":"2024-04-25T10:26:09.780641+0000","last_unstale":"2024-04-25T10:26:09.787601+0000","last_undegraded":"2024-04-25T10:26:09.787601+0000","last_fullsized":"2024-04-25T10:26:09.787601+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.040552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.761315+0000","last_change":"2024-04-25T10:26:08.774202+0000","last_active":"2024-04-25T10:26:09.761315+0000","last_peered":"2024-04-25T10:26:09.761315+0000","last_clean":"2024-04-25T10:26:09.761315+0000","last_became_active":"2024-04-25T10:26:08.772994+0000","last_became_peered":"2024-04-25T10:26:08.772994+0000","last_unstale":"2024-04-25T10:26:09.761315+0000","last_undegraded":"2024-04-25T10:26:09.761315+0000","last_fullsized":"2024-04-25T10:26:09.761315+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.871193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777761+0000","last_change":"2024-04-25T10:26:09.783598+0000","last_active":"2024-04-25T10:26:10.777761+0000","last_peered":"2024-04-25T10:26:10.777761+0000","last_clean":"2024-04-25T10:26:10.777761+0000","last_became_active":"2024-04-25T10:26:09.783330+0000","last_became_peered":"2024-04-25T10:26:09.783330+0000","last_unstale":"2024-04-25T10:26:10.777761+0000","last_undegraded":"2024-04-25T10:26:10.777761+0000","last_fullsized":"2024-04-25T10:26:10.777761+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:10.370110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T10:26:09.778095+0000","last_change":"2024-04-25T10:26:08.770381+0000","last_active":"2024-04-25T10:26:09.778095+0000","last_peered":"2024-04-25T10:26:09.778095+0000","last_clean":"2024-04-25T10:26:09.778095+0000","last_became_active":"2024-04-25T10:26:08.769980+0000","last_became_peered":"2024-04-25T10:26:08.769980+0000","last_unstale":"2024-04-25T10:26:09.778095+0000","last_undegraded":"2024-04-25T10:26:09.778095+0000","last_fullsized":"2024-04-25T10:26:09.778095+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:26.882416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.778048+0000","last_change":"2024-04-25T10:26:09.781631+0000","last_active":"2024-04-25T10:26:10.778048+0000","last_peered":"2024-04-25T10:26:10.778048+0000","last_clean":"2024-04-25T10:26:10.778048+0000","last_became_active":"2024-04-25T10:26:09.781370+0000","last_became_peered":"2024-04-25T10:26:09.781370+0000","last_unstale":"2024-04-25T10:26:10.778048+0000","last_undegraded":"2024-04-25T10:26:10.778048+0000","last_fullsized":"2024-04-25T10:26:10.778048+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:08.741057+0000","last_clean_scrub_stamp":"2024-04-25T10:26:08.741057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:28.520771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.761311+0000","last_change":"2024-04-25T10:26:08.777232+0000","last_active":"2024-04-25T10:26:10.761311+0000","last_peered":"2024-04-25T10:26:10.761311+0000","last_clean":"2024-04-25T10:26:10.761311+0000","last_became_active":"2024-04-25T10:26:08.776932+0000","last_became_peered":"2024-04-25T10:26:08.776932+0000","last_unstale":"2024-04-25T10:26:10.761311+0000","last_undegraded":"2024-04-25T10:26:10.761311+0000","last_fullsized":"2024-04-25T10:26:10.761311+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:07.734112+0000","last_clean_scrub_stamp":"2024-04-25T10:26:07.734112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:24.509794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":473,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:26:10.777573+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:26:10.777573+0000","last_peered":"2024-04-25T10:26:10.777573+0000","last_clean":"2024-04-25T10:26:10.777573+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:26:10.777573+0000","last_undegraded":"2024-04-25T10:26:10.777573+0000","last_fullsized":"2024-04-25T10:26:10.777573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64592,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307248,"statfs":{"total":96636764160,"available":96570621952,"internally_reserved":0,"allocated":2949120,"data_stored":1622986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65364,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306476,"statfs":{"total":96636764160,"available":96569831424,"internally_reserved":0,"allocated":2359296,"data_stored":1032604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51536,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320304,"statfs":{"total":96636764160,"available":96583991296,"internally_reserved":0,"allocated":2363392,"data_stored":1038848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575019,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64012,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307828,"statfs":{"total":96636764160,"available":96571215872,"internally_reserved":0,"allocated":2355200,"data_stored":1033130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575020,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59880,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311960,"statfs":{"total":96636764160,"available":96575447040,"internally_reserved":0,"allocated":2379776,"data_stored":1040530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575019,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2383872,"data_stored":1040516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69576,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302264,"statfs":{"total":96636764160,"available":96565518336,"internally_reserved":0,"allocated":2355200,"data_stored":1032638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56276,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315564,"statfs":{"total":96636764160,"available":96579137536,"internally_reserved":0,"allocated":2953216,"data_stored":1623588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:26:14.219 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:26:14.219 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:26:14.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 Processor -- start 2024-04-25T10:26:14.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 -- start start 2024-04-25T10:26:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d8c15bbb0 0x7f5d8c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d8c0b4660 0x7f5d8c06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0cf960 0x7f5d8c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c06d050 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c06ca90 con 0x7f5d8c0b4660 2024-04-25T10:26:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d92000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c06cd70 con 0x7f5d8c0cf960 2024-04-25T10:26:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d8c15bbb0 0x7f5d8c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d8c15bbb0 0x7f5d8c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39774/0 (socket says 172.21.15.87:39774) 2024-04-25T10:26:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 -- 172.21.15.87:0/1686688264 learned_addr learned my addr 172.21.15.87:0/1686688264 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8be00640 1 --2- 172.21.15.87:0/1686688264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0cf960 0x7f5d8c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8aa00640 1 --2- 172.21.15.87:0/1686688264 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c0b4660 0x7f5d8c06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 -- 172.21.15.87:0/1686688264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0cf960 msgr2=0x7f5d8c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 --2- 172.21.15.87:0/1686688264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0cf960 0x7f5d8c0cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 -- 172.21.15.87:0/1686688264 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c0b4660 msgr2=0x7f5d8c06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:14.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 --2- 172.21.15.87:0/1686688264 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c0b4660 0x7f5d8c06c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.399+0000 7f5d8b400640 1 -- 172.21.15.87:0/1686688264 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c0b2880 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d8b400640 1 --2- 172.21.15.87:0/1686688264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c0cda80 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b25a0 tx=0x7f5d7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdf4828787b8eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d8a000640 1 -- 172.21.15.87:0/1686688264 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b0070 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d8a000640 1 -- 172.21.15.87:0/1686688264 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0ab600 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d8a000640 1 -- 172.21.15.87:0/1686688264 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0ab9d0 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 -- 172.21.15.87:0/1686688264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 msgr2=0x7f5d8c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 --2- 172.21.15.87:0/1686688264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c0cda80 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b25a0 tx=0x7f5d7c01c520 comp rx=0 tx=0).stop 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 -- 172.21.15.87:0/1686688264 shutdown_connections 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 --2- 172.21.15.87:0/1686688264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0cf960 0x7f5d8c0cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 --2- 172.21.15.87:0/1686688264 >> v2:172.21.15.169:3300/0 conn(0x7f5d8c0b4660 0x7f5d8c06c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 --2- 172.21.15.87:0/1686688264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c0cda80 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 -- 172.21.15.87:0/1686688264 >> 172.21.15.87:0/1686688264 conn(0x7f5d8c066b70 msgr2=0x7f5d8c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 -- 172.21.15.87:0/1686688264 shutdown_connections 2024-04-25T10:26:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.403+0000 7f5d92000640 1 -- 172.21.15.87:0/1686688264 wait complete. 2024-04-25T10:26:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 Processor -- start 2024-04-25T10:26:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 -- start start 2024-04-25T10:26:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0b4660 0x7f5d8c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c0cf960 0x7f5d8c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0b3890 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c06c870 con 0x7f5d8c0cf960 2024-04-25T10:26:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0b3a10 con 0x7f5d8c0b4660 2024-04-25T10:26:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39778/0 (socket says 172.21.15.87:39778) 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 -- 172.21.15.87:0/3545690107 learned_addr learned my addr 172.21.15.87:0/3545690107 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8b400640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0b4660 0x7f5d8c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8aa00640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c0cf960 0x7f5d8c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 -- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0b4660 msgr2=0x7f5d8c157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0b4660 0x7f5d8c157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 -- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c0cf960 msgr2=0x7f5d8c0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c0cf960 0x7f5d8c0c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 -- 172.21.15.87:0/3545690107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c158410 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8b400640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0b4660 0x7f5d8c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d8be00640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c148de0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400e930 tx=0x7f5d8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d83400640 1 -- 172.21.15.87:0/3545690107 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84017070 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d83400640 1 -- 172.21.15.87:0/3545690107 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8400c9a0 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d83400640 1 -- 172.21.15.87:0/3545690107 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400cd70 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c157890 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c157e50 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.407+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c04ef30 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.411+0000 7f5d83400640 1 -- 172.21.15.87:0/3545690107 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d8401a070 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.411+0000 7f5d83400640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d68066f60 0x7f5d68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.411+0000 7f5d83400640 1 -- 172.21.15.87:0/3545690107 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d84085310 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.411+0000 7f5d8b400640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d68066f60 0x7f5d68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.411+0000 7f5d8b400640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d68066f60 0x7f5d68069420 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0ad000 tx=0x7f5d7c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.415+0000 7f5d83400640 1 -- 172.21.15.87:0/3545690107 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d84055d90 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.619+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5d8c06d980 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.623+0000 7f5d83400640 1 -- 172.21.15.87:0/3545690107 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5d84085130 con 0x7f5d8c15bbb0 2024-04-25T10:26:14.624 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T10:26:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQBMLCpmNHQ6DxAA+OD3Hdw2u2HYdAUxIHHBow== 2024-04-25T10:26:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-04-25T10:26:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-04-25T10:26:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:26:14.624 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d68066f60 msgr2=0x7f5d68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d68066f60 0x7f5d68069420 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0ad000 tx=0x7f5d7c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 msgr2=0x7f5d8c148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c148de0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400e930 tx=0x7f5d8400edf0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 shutdown_connections 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f5d68066f60 0x7f5d68069420 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c15bbb0 0x7f5d8c148de0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d8c0cf960 0x7f5d8c0c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 --2- 172.21.15.87:0/3545690107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8c0b4660 0x7f5d8c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 >> 172.21.15.87:0/3545690107 conn(0x7f5d8c066b70 msgr2=0x7f5d8c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 shutdown_connections 2024-04-25T10:26:14.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.627+0000 7f5d92000640 1 -- 172.21.15.87:0/3545690107 wait complete. 2024-04-25T10:26:14.644 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773e00640 1 Processor -- start 2024-04-25T10:26:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773e00640 1 -- start start 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47740ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f477406ca80 0x7f47740cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477406d060 con 0x7f47740ca560 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47740ca9c0 con 0x7f477406ca80 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477406d340 con 0x7f477415b390 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f477406ca80 0x7f47740cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f477406ca80 0x7f47740cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39786/0 (socket says 172.21.15.87:39786) 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4772000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f47740ca560 0x7f477406c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45088/0 (socket says 172.21.15.87:45088) 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773400640 1 -- 172.21.15.87:0/3635281441 learned_addr learned my addr 172.21.15.87:0/3635281441 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4772a00640 1 --2- 172.21.15.87:0/3635281441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47740ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773400640 1 -- 172.21.15.87:0/3635281441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 msgr2=0x7f47740ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773400640 1 --2- 172.21.15.87:0/3635281441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47740ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773400640 1 -- 172.21.15.87:0/3635281441 >> v1:172.21.15.87:6789/0 conn(0x7f47740ca560 legacy=0x7f477406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.783+0000 7f4773400640 1 -- 172.21.15.87:0/3635281441 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477406d620 con 0x7f477406ca80 2024-04-25T10:26:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.787+0000 7f4773400640 1 --2- 172.21.15.87:0/3635281441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f477406ca80 0x7f47740cd710 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89700a8b0fca85b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.787+0000 7f4771600640 1 -- 172.21.15.87:0/3635281441 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0ab620 con 0x7f477406ca80 2024-04-25T10:26:14.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.787+0000 7f4771600640 1 -- 172.21.15.87:0/3635281441 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c0b0070 con 0x7f477406ca80 2024-04-25T10:26:14.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.787+0000 7f4771600640 1 -- 172.21.15.87:0/3635281441 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0ab9e0 con 0x7f477406ca80 2024-04-25T10:26:14.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- 172.21.15.87:0/3635281441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f477406ca80 msgr2=0x7f47740cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 --2- 172.21.15.87:0/3635281441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f477406ca80 0x7f47740cd710 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- 172.21.15.87:0/3635281441 shutdown_connections 2024-04-25T10:26:14.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 --2- 172.21.15.87:0/3635281441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f477406ca80 0x7f47740cd710 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 --2- 172.21.15.87:0/3635281441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47740ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- 172.21.15.87:0/3635281441 >> 172.21.15.87:0/3635281441 conn(0x7f47740b16a0 msgr2=0x7f47740b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- 172.21.15.87:0/3635281441 shutdown_connections 2024-04-25T10:26:14.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- 172.21.15.87:0/3635281441 wait complete. 2024-04-25T10:26:14.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 Processor -- start 2024-04-25T10:26:14.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- start start 2024-04-25T10:26:14.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f477406ca80 0x7f47740b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47740ca560 0x7f47740c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47741521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47740ce020 con 0x7f47740ca560 2024-04-25T10:26:14.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47740cdea0 con 0x7f477406ca80 2024-04-25T10:26:14.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47740ce1a0 con 0x7f477415b390 2024-04-25T10:26:14.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4772000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47740ca560 0x7f47740c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4772a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f477406ca80 0x7f47740b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4772a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f477406ca80 0x7f47740b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:57770/0 (socket says 172.21.15.87:57770) 2024-04-25T10:26:14.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4773400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47741521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.791+0000 7f4772a00640 1 -- 172.21.15.87:0/3020960855 learned_addr learned my addr 172.21.15.87:0/3020960855 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772000640 1 -- 172.21.15.87:0/3020960855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 msgr2=0x7f47741521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772000640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47741521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772000640 1 -- 172.21.15.87:0/3020960855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f477406ca80 msgr2=0x7f47740b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772000640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f477406ca80 0x7f47740b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772000640 1 -- 172.21.15.87:0/3020960855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47741fafa0 con 0x7f47740ca560 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772a00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f477406ca80 0x7f47740b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772000640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47740ca560 0x7f47740c8ef0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f476000c390 tx=0x7f476000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f476aa00640 1 -- 172.21.15.87:0/3020960855 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600074e0 con 0x7f47740ca560 2024-04-25T10:26:14.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f476aa00640 1 -- 172.21.15.87:0/3020960855 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760007680 con 0x7f47740ca560 2024-04-25T10:26:14.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f476aa00640 1 -- 172.21.15.87:0/3020960855 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760004a10 con 0x7f47740ca560 2024-04-25T10:26:14.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4774070900 con 0x7f47740ca560 2024-04-25T10:26:14.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4774070ec0 con 0x7f47740ca560 2024-04-25T10:26:14.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4774075120 con 0x7f47740ca560 2024-04-25T10:26:14.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f476aa00640 1 -- 172.21.15.87:0/3020960855 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f476001a020 con 0x7f47740ca560 2024-04-25T10:26:14.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f476aa00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4754066f60 0x7f4754069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f476aa00640 1 -- 172.21.15.87:0/3020960855 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4760084680 con 0x7f47740ca560 2024-04-25T10:26:14.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.795+0000 7f4772a00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4754066f60 0x7f4754069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.799+0000 7f4772a00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4754066f60 0x7f4754069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4764007860 tx=0x7f47640098b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.799+0000 7f476aa00640 1 -- 172.21.15.87:0/3020960855 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4760055030 con 0x7f47740ca560 2024-04-25T10:26:14.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.983+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47740728b0 con 0x7f47740ca560 2024-04-25T10:26:14.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f476aa00640 1 -- 172.21.15.87:0/3020960855 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4955 (secure 0 0 0) 0x7f47600590d0 con 0x7f47740ca560 2024-04-25T10:26:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:14.986 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":137,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:12.151950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11304":{"gid":11304,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/997618220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":997618220},{"type":"v1","addr":"172.21.15.87:6836","nonce":997618220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:26:14.986 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 137 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4754066f60 msgr2=0x7f4754069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4754066f60 0x7f4754069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4764007860 tx=0x7f47640098b0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47740ca560 msgr2=0x7f47740c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47740ca560 0x7f47740c8ef0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f476000c390 tx=0x7f476000c850 comp rx=0 tx=0).stop 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 shutdown_connections 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4754066f60 0x7f4754069420 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f477415b390 0x7f47741521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47740ca560 0x7f47740c8ef0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 --2- 172.21.15.87:0/3020960855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f477406ca80 0x7f47740b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 >> 172.21.15.87:0/3020960855 conn(0x7f47740b16a0 msgr2=0x7f47740ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 shutdown_connections 2024-04-25T10:26:14.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:14.987+0000 7f4773e00640 1 -- 172.21.15.87:0/3020960855 wait complete. 2024-04-25T10:26:15.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 137, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:26:09.137144+0000', 'modified': '2024-04-25T10:26:12.151950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11304': {'gid': 11304, 'name': 'a', 'rank': 0, 'incarnation': 133, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6836/997618220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 997618220}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 997618220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:26:15.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:26:15.410 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:26:15.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.547+0000 7f24d1000640 1 Processor -- start 2024-04-25T10:26:15.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.547+0000 7f24d1000640 1 -- start start 2024-04-25T10:26:15.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.547+0000 7f24d1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.547+0000 7f24d1000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24cc0cfe70 0x7f24cc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24d1000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc071050 con 0x7f24cc15b740 2024-04-25T10:26:15.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24d1000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24cc15d040 con 0x7f24cc0cfe70 2024-04-25T10:26:15.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24d1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0712f0 con 0x7f24cc1550b0 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24cc0cfe70 0x7f24cc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24ca800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f24cc15b740 0x7f24cc06d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45100/0 (socket says 172.21.15.87:45100) 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24ca800640 1 -- 172.21.15.87:0/3387897628 learned_addr learned my addr 172.21.15.87:0/3387897628 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c9e00640 1 --2- 172.21.15.87:0/3387897628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c1400640 1 -- 172.21.15.87:0/3387897628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 msgr2=0x7f24cc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c1400640 1 --2- 172.21.15.87:0/3387897628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c1400640 1 -- 172.21.15.87:0/3387897628 >> v1:172.21.15.87:6789/0 conn(0x7f24cc15b740 legacy=0x7f24cc06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c1400640 1 -- 172.21.15.87:0/3387897628 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0715d0 con 0x7f24cc0cfe70 2024-04-25T10:26:15.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c9400640 1 -- 172.21.15.87:0/3387897628 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138672880 0 0) 0x7f24cc071050 con 0x7f24cc15b740 2024-04-25T10:26:15.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c1400640 1 --2- 172.21.15.87:0/3387897628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc0cfe70 0x7f24cc15b200 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097c0 tx=0x7f24b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fe1576ca73a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c9400640 1 -- 172.21.15.87:0/3387897628 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b0070 con 0x7f24cc0cfe70 2024-04-25T10:26:15.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c9400640 1 -- 172.21.15.87:0/3387897628 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b40ab600 con 0x7f24cc0cfe70 2024-04-25T10:26:15.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.551+0000 7f24c9400640 1 -- 172.21.15.87:0/3387897628 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40ab9d0 con 0x7f24cc0cfe70 2024-04-25T10:26:15.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- 172.21.15.87:0/3387897628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc0cfe70 msgr2=0x7f24cc15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 --2- 172.21.15.87:0/3387897628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc0cfe70 0x7f24cc15b200 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097c0 tx=0x7f24b401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- 172.21.15.87:0/3387897628 shutdown_connections 2024-04-25T10:26:15.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 --2- 172.21.15.87:0/3387897628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc0cfe70 0x7f24cc15b200 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 --2- 172.21.15.87:0/3387897628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- 172.21.15.87:0/3387897628 >> 172.21.15.87:0/3387897628 conn(0x7f24cc066b70 msgr2=0x7f24cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- 172.21.15.87:0/3387897628 shutdown_connections 2024-04-25T10:26:15.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- 172.21.15.87:0/3387897628 wait complete. 2024-04-25T10:26:15.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 Processor -- start 2024-04-25T10:26:15.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- start start 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24cc0cfe70 0x7f24cc150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc15b740 0x7f24cc0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc15bd10 con 0x7f24cc15b740 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc15bb90 con 0x7f24cc0cfe70 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24d1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc15c030 con 0x7f24cc1550b0 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc15b740 0x7f24cc0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc15b740 0x7f24cc0b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39818/0 (socket says 172.21.15.87:39818) 2024-04-25T10:26:15.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 -- 172.21.15.87:0/1661576850 learned_addr learned my addr 172.21.15.87:0/1661576850 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24cc0cfe70 0x7f24cc150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24c1400640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 -- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 msgr2=0x7f24cc0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 -- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24cc0cfe70 msgr2=0x7f24cc150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24cc0cfe70 0x7f24cc150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.555+0000 7f24ca800640 1 -- 172.21.15.87:0/1661576850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc1483a0 con 0x7f24cc15b740 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24ca800640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc15b740 0x7f24cc0b2a60 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c700 tx=0x7f24c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24c2a00640 1 -- 172.21.15.87:0/1661576850 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4018070 con 0x7f24cc15b740 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24c2a00640 1 -- 172.21.15.87:0/1661576850 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c400ce10 con 0x7f24cc15b740 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24c2a00640 1 -- 172.21.15.87:0/1661576850 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c400a860 con 0x7f24cc15b740 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24cc148680 con 0x7f24cc15b740 2024-04-25T10:26:15.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc148c40 con 0x7f24cc15b740 2024-04-25T10:26:15.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24c2a00640 1 -- 172.21.15.87:0/1661576850 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24c400aa00 con 0x7f24cc15b740 2024-04-25T10:26:15.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24c2a00640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8066f60 0x7f24a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24c2a00640 1 -- 172.21.15.87:0/1661576850 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24c4084610 con 0x7f24cc15b740 2024-04-25T10:26:15.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8066f60 0x7f24a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24d43e5000 1 Processor -- start 2024-04-25T10:26:15.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.559+0000 7f24d43e5000 1 -- start start 2024-04-25T10:26:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8066f60 0x7f24a8069420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00b9d0 tx=0x7f24bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193ca240 0x56041937f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x560419380310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604192675f0 con 0x5604193ca240 2024-04-25T10:26:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604191ceda0 con 0x5604193c9920 2024-04-25T10:26:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560419157150 con 0x56041937ff30 2024-04-25T10:26:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193ca240 0x56041937f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193ca240 0x56041937f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39824/0 (socket says 172.21.15.87:39824) 2024-04-25T10:26:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 -- 172.21.15.87:0/4289759097 learned_addr learned my addr 172.21.15.87:0/4289759097 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:15.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c9e00640 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24ca800640 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x560419380310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 -- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 msgr2=0x560419380310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x560419380310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 -- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 msgr2=0x5604193c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 -- 172.21.15.87:0/4289759097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560419170ca0 con 0x5604193ca240 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24ca800640 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x560419380310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24c1400640 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193ca240 0x56041937f9f0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f24b40ae040 tx=0x7f24b4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78863bf3877cb2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24baa00640 1 -- 172.21.15.87:0/4289759097 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b0040 con 0x5604193ca240 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24baa00640 1 -- 172.21.15.87:0/4289759097 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b40ab800 con 0x5604193ca240 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24baa00640 1 -- 172.21.15.87:0/4289759097 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40abbd0 con 0x5604193ca240 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- 172.21.15.87:0/4289759097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193ca240 msgr2=0x56041937f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193ca240 0x56041937f9f0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f24b40ae040 tx=0x7f24b4004740 comp rx=0 tx=0).stop 2024-04-25T10:26:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- 172.21.15.87:0/4289759097 shutdown_connections 2024-04-25T10:26:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x560419380310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193ca240 0x56041937f9f0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 --2- 172.21.15.87:0/4289759097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- 172.21.15.87:0/4289759097 >> 172.21.15.87:0/4289759097 conn(0x5604193862a0 msgr2=0x56041937e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- 172.21.15.87:0/4289759097 shutdown_connections 2024-04-25T10:26:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- 172.21.15.87:0/4289759097 wait complete. 2024-04-25T10:26:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 -- start start 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.563+0000 7f24d43e5000 1 Processor -- start 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x560419383aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560419189990 con 0x5604193c9920 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604191cae40 con 0x5604193ca240 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56041915a900 con 0x56041937ff30 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 10 client.0 _ll_register_callbacks cb 0x5604192821c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39840/0 (socket says 172.21.15.87:39840) 2024-04-25T10:26:15.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 -- 172.21.15.87:0/3447438191 learned_addr learned my addr 172.21.15.87:0/3447438191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24ca800640 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x560419383aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c9e00640 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 -- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 msgr2=0x5604193d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 -- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 msgr2=0x560419383aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x560419383aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 -- 172.21.15.87:0/3447438191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560419187630 con 0x5604193c9920 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c1400640 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d11c0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f24b401ab50 tx=0x7f24b401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62afde8885e1658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 1 -- 172.21.15.87:0/3447438191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b0040 con 0x5604193c9920 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24b40ab800 con 0x5604193c9920 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 1 -- 172.21.15.87:0/3447438191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24b40b6440 con 0x5604193c9920 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 1 -- 172.21.15.87:0/3447438191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=137) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b40cf050 con 0x5604193c9920 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 10 client.11448 fetch_fsmap learned FSMap version 137 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=137,monmap=2+}) v3 -- 0x5604191573a0 con 0x5604193c9920 2024-04-25T10:26:15.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 1 -- 172.21.15.87:0/3447438191 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b40ab800 con 0x5604193c9920 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 1 -- 172.21.15.87:0/3447438191 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 137) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24b40d2020 con 0x5604193c9920 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 10 client.11448 fetch_fsmap finished waiting for FSMap version 137 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 10 client.11448 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 20 client.11448 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x560419385160 0x5604193dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 4 client.11448 mds_command: new command op to 11304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 --> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56041916aa50 con 0x560419385160 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c9e00640 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x560419385160 0x5604193dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24c9e00640 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x560419385160 0x5604193dace0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768b6e2d1db0a1d9 server_cookie=76bb855c477527a in_seq=0 out_seq=0 2024-04-25T10:26:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 10 client.11448 ms_handle_connect on v2:172.21.15.87:6834/997618220 2024-04-25T10:26:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 1 -- 172.21.15.87:0/3447438191 <== mds.0 v2:172.21.15.87:6834/997618220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56041916aa50 con 0x560419385160 2024-04-25T10:26:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24af400640 10 client.11448 handle_command_reply: tid=0 2024-04-25T10:26:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.567+0000 7f24d43e5000 1 client.11448 shutdown 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 msgr2=0x5604193d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d11c0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f24b401ab50 tx=0x7f24b401ace0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 shutdown_connections 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x560419385160 0x5604193dace0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x560419383aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d11c0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 --2- 172.21.15.87:0/3447438191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24af400640 0 client.11448 ms_handle_reset on v2:172.21.15.87:6834/997618220 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 >> 172.21.15.87:0/3447438191 conn(0x5604193862a0 msgr2=0x56041937e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 shutdown_connections 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 1 -- 172.21.15.87:0/3447438191 wait complete. 2024-04-25T10:26:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.571+0000 7f24d43e5000 20 client.11448 trim_cache size 0 max 16384 2024-04-25T10:26:15.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.583+0000 7f24d43e5000 1 Processor -- start 2024-04-25T10:26:15.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.583+0000 7f24d43e5000 1 -- start start 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x5604193d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc136df0 con 0x5604193c9920 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc1361f0 con 0x5604193ca240 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc1367f0 con 0x56041937ff30 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39854/0 (socket says 172.21.15.87:39854) 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 -- 172.21.15.87:0/1174258790 learned_addr learned my addr 172.21.15.87:0/1174258790 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24ca800640 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x5604193d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 -- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 msgr2=0x5604193d6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 -- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 msgr2=0x5604193d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x5604193d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 -- 172.21.15.87:0/1174258790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560419250600 con 0x5604193c9920 2024-04-25T10:26:15.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:15.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24c1400640 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d73b0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f24b40ae040 tx=0x7f24b400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bf13539c2205ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24ba000640 1 -- 172.21.15.87:0/1174258790 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b0040 con 0x5604193c9920 2024-04-25T10:26:15.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24ba000640 1 -- 172.21.15.87:0/1174258790 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b4000e50 con 0x5604193c9920 2024-04-25T10:26:15.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24ba000640 1 -- 172.21.15.87:0/1174258790 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b6b90 con 0x5604193c9920 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- 172.21.15.87:0/1174258790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 msgr2=0x5604193d73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d73b0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f24b40ae040 tx=0x7f24b400b050 comp rx=0 tx=0).stop 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- 172.21.15.87:0/1174258790 shutdown_connections 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193ca240 0x5604193d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604193c9920 0x5604193d73b0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1174258790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56041937ff30 0x5604193d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- 172.21.15.87:0/1174258790 >> 172.21.15.87:0/1174258790 conn(0x5604193862a0 msgr2=0x5604193d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- 172.21.15.87:0/1174258790 shutdown_connections 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- 172.21.15.87:0/1174258790 wait complete. 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 -- start start 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.587+0000 7f24d43e5000 1 Processor -- start 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56041937ff30 0x560419429690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5604193ca240 0x5604193e4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604191d33d0 con 0x56041937ff30 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041928ad50 con 0x5604193c9920 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560419169980 con 0x5604193ca240 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 10 client.0 _ll_register_callbacks cb 0x7f24b40ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56041937ff30 0x560419429690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56041937ff30 0x560419429690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39870/0 (socket says 172.21.15.87:39870) 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 -- 172.21.15.87:0/1674662203 learned_addr learned my addr 172.21.15.87:0/1674662203 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c1400640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24ca800640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5604193ca240 0x5604193e4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 -- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5604193ca240 msgr2=0x5604193e4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5604193ca240 0x5604193e4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 -- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 msgr2=0x5604193e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 -- 172.21.15.87:0/1674662203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604191639b0 con 0x56041937ff30 2024-04-25T10:26:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24ca800640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5604193ca240 0x5604193e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c1400640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c9e00640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56041937ff30 0x560419429690 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00fcb0 tx=0x7f24bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d4615ae5377a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 1 -- 172.21.15.87:0/1674662203 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc01b050 con 0x56041937ff30 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 1 -- 172.21.15.87:0/1674662203 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24bc014b70 con 0x56041937ff30 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24b4000e50 con 0x56041937ff30 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 1 -- 172.21.15.87:0/1674662203 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=137) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24bc038050 con 0x56041937ff30 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 10 client.11454 fetch_fsmap learned FSMap version 137 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=137,monmap=2+}) v3 -- 0x5604191760b0 con 0x56041937ff30 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 1 -- 172.21.15.87:0/1674662203 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b4000e50 con 0x56041937ff30 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 1 -- 172.21.15.87:0/1674662203 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 137) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24bc03b020 con 0x56041937ff30 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 10 client.11454 fetch_fsmap finished waiting for FSMap version 137 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 10 client.11454 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 20 client.11454 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x5604193e51e0 0x5604193e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 4 client.11454 mds_command: new command op to 11304 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:26:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 --> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5604191dc2e0 con 0x5604193e51e0 2024-04-25T10:26:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c1400640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x5604193e51e0 0x5604193e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24c1400640 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x5604193e51e0 0x5604193e55c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3cf5f3857e15e6 server_cookie=b53e7927dc1144aa in_seq=0 out_seq=0 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 10 client.11454 ms_handle_connect on v2:172.21.15.87:6834/997618220 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 1 -- 172.21.15.87:0/1674662203 <== mds.0 v2:172.21.15.87:6834/997618220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x5604191dc2e0 con 0x5604193e51e0 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 10 client.11454 handle_command_reply: tid=0 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 client.11454 shutdown 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56041937ff30 msgr2=0x560419429690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56041937ff30 0x560419429690 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00fcb0 tx=0x7f24bc004740 comp rx=0 tx=0).stop 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 shutdown_connections 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:6834/997618220,v1:172.21.15.87:6836/997618220] conn(0x5604193e51e0 0x5604193e55c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5604193ca240 0x5604193e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604193c9920 0x5604193e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24af400640 0 client.11454 ms_handle_reset on v2:172.21.15.87:6834/997618220 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 --2- 172.21.15.87:0/1674662203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56041937ff30 0x560419429690 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 >> 172.21.15.87:0/1674662203 conn(0x5604193862a0 msgr2=0x5604193c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 shutdown_connections 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 1 -- 172.21.15.87:0/1674662203 wait complete. 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d43e5000 20 client.11454 trim_cache size 0 max 16384 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:26:15.596 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 11304, 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 5.8284307389999999, 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 137, 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 187, 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 187, 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 10.797661749 2024-04-25T10:26:15.597 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8066f60 msgr2=0x7f24a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8066f60 0x7f24a8069420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00b9d0 tx=0x7f24bc008040 comp rx=0 tx=0).stop 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc15b740 msgr2=0x7f24cc0b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc15b740 0x7f24cc0b2a60 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c700 tx=0x7f24c400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 shutdown_connections 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f24a8066f60 0x7f24a8069420 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24cc15b740 0x7f24cc0b2a60 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24cc1550b0 0x7f24cc0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 --2- 172.21.15.87:0/1661576850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24cc0cfe70 0x7f24cc150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.591+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 >> 172.21.15.87:0/1661576850 conn(0x7f24cc066b70 msgr2=0x7f24cc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.595+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 shutdown_connections 2024-04-25T10:26:15.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.595+0000 7f24d1000640 1 -- 172.21.15.87:0/1661576850 wait complete. 2024-04-25T10:26:15.612 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:26:15.734 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:26:15.735+0000 7fbf26200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:26:15.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8ab200640 1 Processor -- start 2024-04-25T10:26:15.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8ab200640 1 -- start start 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a41550b0 0x7ff8a40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8ab200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff8a415b910 0x7ff8a40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8ab200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a4071050 con 0x7ff8a40cfe70 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8ab200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a415bda0 con 0x7ff8a415b910 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a415d040 con 0x7ff8a41550b0 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff89b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff8a40cfe70 0x7ff8a415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45114/0 (socket says 172.21.15.87:45114) 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff89b400640 1 -- 172.21.15.87:0/3653026593 learned_addr learned my addr 172.21.15.87:0/3653026593 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8a3e00640 1 --2- 172.21.15.87:0/3653026593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a41550b0 0x7ff8a40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8a9200640 1 --2- 172.21.15.87:0/3653026593 >> v2:172.21.15.169:3300/0 conn(0x7ff8a415b910 0x7ff8a40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8a3400640 1 -- 172.21.15.87:0/3653026593 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378553647 0 0) 0x7ff8a4071050 con 0x7ff8a40cfe70 2024-04-25T10:26:15.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.747+0000 7ff8a3400640 1 -- 172.21.15.87:0/3653026593 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff888005740 con 0x7ff8a40cfe70 2024-04-25T10:26:15.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3e00640 1 -- 172.21.15.87:0/3653026593 >> v2:172.21.15.169:3300/0 conn(0x7ff8a415b910 msgr2=0x7ff8a40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:15.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3e00640 1 --2- 172.21.15.87:0/3653026593 >> v2:172.21.15.169:3300/0 conn(0x7ff8a415b910 0x7ff8a40b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3e00640 1 -- 172.21.15.87:0/3653026593 >> v1:172.21.15.87:6789/0 conn(0x7ff8a40cfe70 legacy=0x7ff8a415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3e00640 1 -- 172.21.15.87:0/3653026593 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40712f0 con 0x7ff8a41550b0 2024-04-25T10:26:15.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3e00640 1 --2- 172.21.15.87:0/3653026593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a41550b0 0x7ff8a40cf930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff8940097c0 tx=0x7ff89401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a813ae3ca801bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3400640 1 -- 172.21.15.87:0/3653026593 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8940b2070 con 0x7ff8a41550b0 2024-04-25T10:26:15.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3400640 1 -- 172.21.15.87:0/3653026593 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8940ad640 con 0x7ff8a41550b0 2024-04-25T10:26:15.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8a3400640 1 -- 172.21.15.87:0/3653026593 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8940ada10 con 0x7ff8a41550b0 2024-04-25T10:26:15.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8ab200640 1 -- 172.21.15.87:0/3653026593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a41550b0 msgr2=0x7ff8a40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3653026593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a41550b0 0x7ff8a40cf930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff8940097c0 tx=0x7ff89401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8ab200640 1 -- 172.21.15.87:0/3653026593 shutdown_connections 2024-04-25T10:26:15.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3653026593 >> v2:172.21.15.169:3300/0 conn(0x7ff8a415b910 0x7ff8a40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3653026593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a41550b0 0x7ff8a40cf930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.751+0000 7ff8ab200640 1 -- 172.21.15.87:0/3653026593 >> 172.21.15.87:0/3653026593 conn(0x7ff8a4066b70 msgr2=0x7ff8a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 -- 172.21.15.87:0/3653026593 shutdown_connections 2024-04-25T10:26:15.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 -- 172.21.15.87:0/3653026593 wait complete. 2024-04-25T10:26:15.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 Processor -- start 2024-04-25T10:26:15.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 -- start start 2024-04-25T10:26:15.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40cfe70 0x7ff8a413c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a41550b0 0x7ff8a40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a415b910 0x7ff8a40add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a415c030 con 0x7ff8a41550b0 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40720b0 con 0x7ff8a40cfe70 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a415c1b0 con 0x7ff8a415b910 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a41550b0 0x7ff8a40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a41550b0 0x7ff8a40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39876/0 (socket says 172.21.15.87:39876) 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 -- 172.21.15.87:0/3224558918 learned_addr learned my addr 172.21.15.87:0/3224558918 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8a3e00640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40cfe70 0x7ff8a413c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8a9200640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a415b910 0x7ff8a40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 -- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a415b910 msgr2=0x7ff8a40add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a415b910 0x7ff8a40add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 -- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40cfe70 msgr2=0x7ff8a413c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40cfe70 0x7ff8a413c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 -- 172.21.15.87:0/3224558918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a413c7b0 con 0x7ff8a41550b0 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff89b400640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a41550b0 0x7ff8a40ad840 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c450 tx=0x7ff88c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8a0c00640 1 -- 172.21.15.87:0/3224558918 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c017070 con 0x7ff8a41550b0 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8a0c00640 1 -- 172.21.15.87:0/3224558918 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c005cf0 con 0x7ff8a41550b0 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a41537b0 con 0x7ff8a41550b0 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.755+0000 7ff8a0c00640 1 -- 172.21.15.87:0/3224558918 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c007290 con 0x7ff8a41550b0 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.759+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a4153d70 con 0x7ff8a41550b0 2024-04-25T10:26:15.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.759+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a404ef30 con 0x7ff8a41550b0 2024-04-25T10:26:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.759+0000 7ff8a0c00640 1 -- 172.21.15.87:0/3224558918 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff88c01a070 con 0x7ff8a41550b0 2024-04-25T10:26:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.759+0000 7ff8a0c00640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff880066f60 0x7ff880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.759+0000 7ff8a0c00640 1 -- 172.21.15.87:0/3224558918 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff88c084430 con 0x7ff8a41550b0 2024-04-25T10:26:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.759+0000 7ff8a3e00640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff880066f60 0x7ff880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.759+0000 7ff8a3e00640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff880066f60 0x7ff880069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff8940af000 tx=0x7ff894008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:15.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.763+0000 7ff8a0c00640 1 -- 172.21.15.87:0/3224558918 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff88c054d50 con 0x7ff8a41550b0 2024-04-25T10:26:15.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff8a406c090 con 0x7ff8a41550b0 2024-04-25T10:26:15.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8a0c00640 1 -- 172.21.15.87:0/3224558918 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v137) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff88c058df0 con 0x7ff8a41550b0 2024-04-25T10:26:15.951 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff880066f60 msgr2=0x7ff880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff880066f60 0x7ff880069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff8940af000 tx=0x7ff894008040 comp rx=0 tx=0).stop 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a41550b0 msgr2=0x7ff8a40ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a41550b0 0x7ff8a40ad840 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c450 tx=0x7ff88c00c910 comp rx=0 tx=0).stop 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 shutdown_connections 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff880066f60 0x7ff880069420 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8a415b910 0x7ff8a40add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a41550b0 0x7ff8a40ad840 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.951+0000 7ff8ab200640 1 --2- 172.21.15.87:0/3224558918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40cfe70 0x7ff8a413c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.955+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 >> 172.21.15.87:0/3224558918 conn(0x7ff8a4066b70 msgr2=0x7ff8a40b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:15.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.955+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 shutdown_connections 2024-04-25T10:26:15.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:15.955+0000 7ff8ab200640 1 -- 172.21.15.87:0/3224558918 wait complete. 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:26:15.972 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:26:15.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:15.973 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T10:26:15.981 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:26:15.981 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:26:15.981 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:26:15.981 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.981 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T10:26:15.981 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41638sec preferred_lft 41638sec 2024-04-25T10:26:15.982 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:89:25:59:f3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:26:15.983 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.984 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:26:15.984 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:26:15.984 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:26:15.984 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:15.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:15.985 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:26:16.046 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T10:26:16.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.048 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T10:26:16.108 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T10:26:16.108 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:26:16.108 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:26:16.108 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:26:16.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.109 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-04-25T10:26:16.110 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T10:26:16.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:26:16.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:26:16.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:26:16.110 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T10:26:16.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.200 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:26:16.257 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:26:16.257 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:26:16.258 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:26:16.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.258 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T10:26:16.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:26:16.325 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:16.380 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:26:16.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:26:16.381 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:16.435 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:26:16.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.436 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:26:16.524 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:26:16.524 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:26:16.524 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:26:16.539 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:26:16.539 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:26:16.539 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:26:16.540 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:26:16.540 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:26:16.547 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:26:16.547 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:26:16.550 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-04-25T10:26:16.550 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T10:26:16.550 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:26:16.550 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-25T10:26:16.551 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-25T10:26:16.551 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=84c5cae3-3d36-4b8f-9ce8-349d9b4e086b" will pass to kernel 2024-04-25T10:26:16.552 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:26:16.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.552 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:16.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.572 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:26:16.633 INFO:teuthology.orchestra.run.smithi087.stdout:client11463 2024-04-25T10:26:16.633 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:26:16.634 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:16.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.685 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-25T10:26:16.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.742 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-25T10:26:16.799 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:26:16.799 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:16.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:16.848 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:26:17.015 INFO:teuthology.orchestra.run.smithi087.stderr:6+0 records in 2024-04-25T10:26:17.015 INFO:teuthology.orchestra.run.smithi087.stderr:6+0 records out 2024-04-25T10:26:17.015 INFO:teuthology.orchestra.run.smithi087.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.113908 s, 55.2 MB/s 2024-04-25T10:26:17.016 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:26:17.016 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:17.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:17.025 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:26:17.157 INFO:teuthology.orchestra.run.smithi087.stderr:6+0 records in 2024-04-25T10:26:17.157 INFO:teuthology.orchestra.run.smithi087.stderr:6+0 records out 2024-04-25T10:26:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0805149 s, 78.1 MB/s 2024-04-25T10:26:17.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:26:17.162 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:26:17.202 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-04-25T10:26:17.203 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-04-25T10:26:17.203 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:26:17.203 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-04-25T10:26:17.203 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.203 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-04-25T10:26:17.203 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.204 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-04-25T10:26:17.204 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.204 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-04-25T10:26:17.204 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-04-25T10:26:17.204 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.204 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-04-25T10:26:17.205 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-04-25T10:26:17.205 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.205 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:26:17.205 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-04-25T10:26:17.205 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.205 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:26:17.206 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:26:17.206 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.206 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-04-25T10:26:17.206 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.206 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-04-25T10:26:17.206 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-04-25T10:26:17.207 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:26:17.207 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:26:17.207 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-04-25T10:26:17.207 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:26:17.207 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-04-25T10:26:17.207 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.208 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on skipped message = False 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.209 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-04-25T10:26:17.210 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.211 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-04-25T10:26:17.211 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-04-25T10:26:17.211 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-04-25T10:26:17.211 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-04-25T10:26:17.211 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.211 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-04-25T10:26:17.211 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.212 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: bdev async discard = True 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: bdev enable discard = True 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore allocator = bitmap 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 1/20 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 1/20 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 4/10 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:26:17.213 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-04-25T10:26:17.214 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-04-25T10:26:17.216 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T10:26:17.217 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: mds cache max size = 1000 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = False 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: mds log max segments = 1 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = False 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.218 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.219 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-25T10:26:17.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-25T10:26:17.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-04-25T10:26:17.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-25T10:26:17.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:26:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-25T10:26:17.228 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.229 INFO:teuthology.orchestra.run.smithi169.stdout: mds cache max size = 1000 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = False 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: mds log max segments = 1 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = False 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.232 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:26:17.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-04-25T10:26:17.315 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-04-25T10:26:17.316 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-04-25T10:26:17.317 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on skipped message = False 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.318 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-04-25T10:26:17.319 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: bdev async discard = True 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: bdev enable discard = True 2024-04-25T10:26:17.320 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore allocator = bitmap 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 1/20 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 1/20 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 4/10 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.321 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T10:26:17.322 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-04-25T10:26:17.323 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.324 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout: mds cache max size = 1000 2024-04-25T10:26:17.325 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = False 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi087.stdout: mds log max segments = 1 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = False 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.326 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-25T10:26:17.327 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-25T10:26:17.328 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.329 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.87,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 84c5cae3-3d36-4b8f-9ce8-349d9b4e086b 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:26:17.330 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.331 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-04-25T10:26:17.332 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:26:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.334 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:26:17.335 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:26:17.336 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-25T10:26:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: mds cache max size = 1000 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = False 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: mds log max segments = 1 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = False 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T10:26:17.338 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T10:26:17.340 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:26:17.340 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:17.388 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:26:17.388 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:17.441 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:17.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7327600640 1 Processor -- start 2024-04-25T10:26:17.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7327600640 1 -- start start 2024-04-25T10:26:17.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7327600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73200d0040 0x7f732015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7327600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7320071050 con 0x7f73201550b0 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7327600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732015d040 con 0x7f732015b740 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7327600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200712f0 con 0x7f73200d0040 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7317e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73200d0040 0x7f732015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7324c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f73201550b0 0x7f73200cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48280/0 (socket says 172.21.15.87:48280) 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7324c00640 1 -- 172.21.15.87:0/2637291404 learned_addr learned my addr 172.21.15.87:0/2637291404 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7317e00640 1 -- 172.21.15.87:0/2637291404 >> v1:172.21.15.169:6789/0 conn(0x7f732015b740 legacy=0x7f73200b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7317e00640 1 -- 172.21.15.87:0/2637291404 >> v1:172.21.15.87:6789/0 conn(0x7f73201550b0 legacy=0x7f73200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.623+0000 7f7317e00640 1 -- 172.21.15.87:0/2637291404 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200715d0 con 0x7f73200d0040 2024-04-25T10:26:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7317e00640 1 --2- 172.21.15.87:0/2637291404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73200d0040 0x7f732015b200 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f73080097c0 tx=0x7f730801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db18b926fc1408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f731fe00640 1 -- 172.21.15.87:0/2637291404 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080b0070 con 0x7f73200d0040 2024-04-25T10:26:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f731fe00640 1 -- 172.21.15.87:0/2637291404 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73080ab600 con 0x7f73200d0040 2024-04-25T10:26:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f731fe00640 1 -- 172.21.15.87:0/2637291404 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080ab9d0 con 0x7f73200d0040 2024-04-25T10:26:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7327600640 1 -- 172.21.15.87:0/2637291404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73200d0040 msgr2=0x7f732015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7327600640 1 --2- 172.21.15.87:0/2637291404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73200d0040 0x7f732015b200 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f73080097c0 tx=0x7f730801c470 comp rx=0 tx=0).stop 2024-04-25T10:26:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7327600640 1 -- 172.21.15.87:0/2637291404 shutdown_connections 2024-04-25T10:26:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7327600640 1 --2- 172.21.15.87:0/2637291404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73200d0040 0x7f732015b200 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7327600640 1 -- 172.21.15.87:0/2637291404 >> 172.21.15.87:0/2637291404 conn(0x7f7320066b70 msgr2=0x7f7320066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7327600640 1 -- 172.21.15.87:0/2637291404 shutdown_connections 2024-04-25T10:26:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.627+0000 7f7327600640 1 -- 172.21.15.87:0/2637291404 wait complete. 2024-04-25T10:26:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 Processor -- start 2024-04-25T10:26:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 -- start start 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73200d0040 0x7f732014b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73201550b0 0x7f732014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f732015b740 0x7f732015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732015bcd0 con 0x7f732015b740 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732015bb50 con 0x7f73201550b0 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73201684c0 con 0x7f73200d0040 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f732015b740 0x7f732015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f732015b740 0x7f732015d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42872/0 (socket says 172.21.15.87:42872) 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 -- 172.21.15.87:0/3715045778 learned_addr learned my addr 172.21.15.87:0/3715045778 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7317e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73201550b0 0x7f732014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7324c00640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73200d0040 0x7f732014b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 -- 172.21.15.87:0/3715045778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73200d0040 msgr2=0x7f732014b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73200d0040 0x7f732014b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 -- 172.21.15.87:0/3715045778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73201550b0 msgr2=0x7f732014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73201550b0 0x7f732014f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 -- 172.21.15.87:0/3715045778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320159780 con 0x7f732015b740 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7325600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f732015b740 0x7f732015d4a0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f731800c730 tx=0x7f731800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f731d600640 1 -- 172.21.15.87:0/3715045778 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318003b80 con 0x7f732015b740 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f731d600640 1 -- 172.21.15.87:0/3715045778 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318003d20 con 0x7f732015b740 2024-04-25T10:26:17.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f731d600640 1 -- 172.21.15.87:0/3715045778 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731800aa30 con 0x7f732015b740 2024-04-25T10:26:17.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7320159a60 con 0x7f732015b740 2024-04-25T10:26:17.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.631+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732015a020 con 0x7f732015b740 2024-04-25T10:26:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.635+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732004ef30 con 0x7f732015b740 2024-04-25T10:26:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.635+0000 7f731d600640 1 -- 172.21.15.87:0/3715045778 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f731800abd0 con 0x7f732015b740 2024-04-25T10:26:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.635+0000 7f731d600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f72fc066f60 0x7f72fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.635+0000 7f731d600640 1 -- 172.21.15.87:0/3715045778 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7318084df0 con 0x7f732015b740 2024-04-25T10:26:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.635+0000 7f7324c00640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f72fc066f60 0x7f72fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.635+0000 7f7324c00640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f72fc066f60 0x7f72fc069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7310005890 tx=0x7f7310009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.639+0000 7f731d600640 1 -- 172.21.15.87:0/3715045778 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7318055710 con 0x7f732015b740 2024-04-25T10:26:17.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.799+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73200cf3e0 con 0x7f732015b740 2024-04-25T10:26:17.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.799+0000 7f731d600640 1 -- 172.21.15.87:0/3715045778 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f73180597b0 con 0x7f732015b740 2024-04-25T10:26:17.800 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:17.800 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":187,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:26:10.753704+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:07.509820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:08.125891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:17.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f72fc066f60 msgr2=0x7f72fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f72fc066f60 0x7f72fc069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7310005890 tx=0x7f7310009040 comp rx=0 tx=0).stop 2024-04-25T10:26:17.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f732015b740 msgr2=0x7f732015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f732015b740 0x7f732015d4a0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f731800c730 tx=0x7f731800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 shutdown_connections 2024-04-25T10:26:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f72fc066f60 0x7f72fc069420 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f732015b740 0x7f732015d4a0 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73201550b0 0x7f732014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 --2- 172.21.15.87:0/3715045778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73200d0040 0x7f732014b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 >> 172.21.15.87:0/3715045778 conn(0x7f7320066b70 msgr2=0x7f732015bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 shutdown_connections 2024-04-25T10:26:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.803+0000 7f7327600640 1 -- 172.21.15.87:0/3715045778 wait complete. 2024-04-25T10:26:17.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:26:07.509820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:26:08.125891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:26:17.822 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:26:17.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.958+0000 7f7fe6400640 1 Processor -- start 2024-04-25T10:26:17.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.958+0000 7f7fe6400640 1 -- start start 2024-04-25T10:26:17.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fe6400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7fe00aeda0 0x7f7fe0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fe6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe0159920 con 0x7f7fe0072020 2024-04-25T10:26:17.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fe6400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe015c2f0 con 0x7f7fe0159540 2024-04-25T10:26:17.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fe6400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe015c5d0 con 0x7f7fe00aeda0 2024-04-25T10:26:17.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fdfe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7fe0072020 0x7f7fe0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48292/0 (socket says 172.21.15.87:48292) 2024-04-25T10:26:17.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fd6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7fe00aeda0 0x7f7fe0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fdfe00640 1 -- 172.21.15.87:0/733125260 learned_addr learned my addr 172.21.15.87:0/733125260 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:17.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fd6a00640 1 -- 172.21.15.87:0/733125260 >> v1:172.21.15.169:6789/0 conn(0x7f7fe0159540 legacy=0x7f7fe00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fd6a00640 1 -- 172.21.15.87:0/733125260 >> v1:172.21.15.87:6789/0 conn(0x7f7fe0072020 legacy=0x7f7fe0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fd6a00640 1 -- 172.21.15.87:0/733125260 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe015c8b0 con 0x7f7fe00aeda0 2024-04-25T10:26:17.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.962+0000 7f7fdea00640 1 -- 172.21.15.87:0/733125260 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308778426 0 0) 0x7f7fe0159920 con 0x7f7fe0072020 2024-04-25T10:26:17.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fd6a00640 1 --2- 172.21.15.87:0/733125260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe0071ae0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80097c0 tx=0x7f7fc801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1eb6e9ce7ab1413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fdea00640 1 -- 172.21.15.87:0/733125260 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80ab610 con 0x7f7fe00aeda0 2024-04-25T10:26:17.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fdea00640 1 -- 172.21.15.87:0/733125260 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc80b0070 con 0x7f7fe00aeda0 2024-04-25T10:26:17.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fdea00640 1 -- 172.21.15.87:0/733125260 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80ab9d0 con 0x7f7fe00aeda0 2024-04-25T10:26:17.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 -- 172.21.15.87:0/733125260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 msgr2=0x7f7fe0071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 --2- 172.21.15.87:0/733125260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe0071ae0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80097c0 tx=0x7f7fc801c470 comp rx=0 tx=0).stop 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 -- 172.21.15.87:0/733125260 shutdown_connections 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 --2- 172.21.15.87:0/733125260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe0071ae0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 -- 172.21.15.87:0/733125260 >> 172.21.15.87:0/733125260 conn(0x7f7fe00669a0 msgr2=0x7f7fe0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 -- 172.21.15.87:0/733125260 shutdown_connections 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 -- 172.21.15.87:0/733125260 wait complete. 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 Processor -- start 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.966+0000 7f7fe6400640 1 -- start start 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0072020 0x7f7fe014cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe013e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe0159540 0x7f7fe00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0169940 con 0x7f7fe00aeda0 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe006c0d0 con 0x7f7fe0159540 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0169ac0 con 0x7f7fe0072020 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe013e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe013e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42880/0 (socket says 172.21.15.87:42880) 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 -- 172.21.15.87:0/819788609 learned_addr learned my addr 172.21.15.87:0/819788609 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fdfe00640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe0159540 0x7f7fe00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fdf400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0072020 0x7f7fe014cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 -- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0072020 msgr2=0x7f7fe014cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0072020 0x7f7fe014cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 -- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe0159540 msgr2=0x7f7fe00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe0159540 0x7f7fe00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 -- 172.21.15.87:0/819788609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe014d280 con 0x7f7fe00aeda0 2024-04-25T10:26:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fdf400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0072020 0x7f7fe014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd6a00640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe013e070 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80097c0 tx=0x7f7fc801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd7e00640 1 -- 172.21.15.87:0/819788609 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8004220 con 0x7f7fe00aeda0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd7e00640 1 -- 172.21.15.87:0/819788609 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc80043e0 con 0x7f7fe00aeda0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd7e00640 1 -- 172.21.15.87:0/819788609 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8003860 con 0x7f7fe00aeda0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe014d560 con 0x7f7fe00aeda0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe014db20 con 0x7f7fe00aeda0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe004ef30 con 0x7f7fe00aeda0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.970+0000 7f7fd7e00640 1 -- 172.21.15.87:0/819788609 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fe0071270 con 0x7f7fe00aeda0 2024-04-25T10:26:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.974+0000 7f7fd7e00640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7fbc066f90 0x7f7fbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.974+0000 7f7fd7e00640 1 -- 172.21.15.87:0/819788609 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fc811f5b0 con 0x7f7fe00aeda0 2024-04-25T10:26:17.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.974+0000 7f7fdf400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7fbc066f90 0x7f7fbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.974+0000 7f7fdf400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7fbc066f90 0x7f7fbc069450 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00b4a50 tx=0x7f7fd0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:17.978+0000 7f7fd7e00640 1 -- 172.21.15.87:0/819788609 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fc80a8050 con 0x7f7fe00aeda0 2024-04-25T10:26:18.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.130+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fe0066f20 con 0x7f7fe00aeda0 2024-04-25T10:26:18.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fd7e00640 1 -- 172.21.15.87:0/819788609 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v187) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fc8016090 con 0x7f7fe00aeda0 2024-04-25T10:26:18.133 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:26:18.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7fbc066f90 msgr2=0x7f7fbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7fbc066f90 0x7f7fbc069450 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00b4a50 tx=0x7f7fd0009040 comp rx=0 tx=0).stop 2024-04-25T10:26:18.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 msgr2=0x7f7fe013e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe013e070 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80097c0 tx=0x7f7fc801acb0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 shutdown_connections 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7fbc066f90 0x7f7fbc069450 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe0159540 0x7f7fe00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe00aeda0 0x7f7fe013e070 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 --2- 172.21.15.87:0/819788609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0072020 0x7f7fe014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 >> 172.21.15.87:0/819788609 conn(0x7f7fe00669a0 msgr2=0x7f7fe0070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 shutdown_connections 2024-04-25T10:26:18.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.134+0000 7f7fe6400640 1 -- 172.21.15.87:0/819788609 wait complete. 2024-04-25T10:26:18.154 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:26:18.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e77000640 1 Processor -- start 2024-04-25T10:26:18.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e77000640 1 -- start start 2024-04-25T10:26:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e77000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e7015dd30 0x7f7e7006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e77000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700d1fb0 0x7f7e700b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e77000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7006b1d0 con 0x7f7e700cfb30 2024-04-25T10:26:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e77000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7015e110 con 0x7f7e7015dd30 2024-04-25T10:26:18.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e77000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7006b470 con 0x7f7e700d1fb0 2024-04-25T10:26:18.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e6fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e7015dd30 0x7f7e7006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e6fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e7015dd30 0x7f7e7006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42882/0 (socket says 172.21.15.87:42882) 2024-04-25T10:26:18.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e6fe00640 1 -- 172.21.15.87:0/2351807333 learned_addr learned my addr 172.21.15.87:0/2351807333 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:18.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e6fe00640 1 -- 172.21.15.87:0/2351807333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700d1fb0 msgr2=0x7f7e700b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:18.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e6fe00640 1 --2- 172.21.15.87:0/2351807333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700d1fb0 0x7f7e700b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e6fe00640 1 -- 172.21.15.87:0/2351807333 >> v1:172.21.15.169:6789/0 conn(0x7f7e700cfb30 legacy=0x7f7e700d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:18.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.294+0000 7f7e6fe00640 1 -- 172.21.15.87:0/2351807333 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7006b750 con 0x7f7e7015dd30 2024-04-25T10:26:18.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e6fe00640 1 --2- 172.21.15.87:0/2351807333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e7015dd30 0x7f7e7006ac90 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009e50 tx=0x7f7e6001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2e95a4880c2436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e6ea00640 1 -- 172.21.15.87:0/2351807333 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e600a5760 con 0x7f7e7015dd30 2024-04-25T10:26:18.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e6ea00640 1 -- 172.21.15.87:0/2351807333 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e600a5900 con 0x7f7e7015dd30 2024-04-25T10:26:18.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e6ea00640 1 -- 172.21.15.87:0/2351807333 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e600a5cd0 con 0x7f7e7015dd30 2024-04-25T10:26:18.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 -- 172.21.15.87:0/2351807333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e7015dd30 msgr2=0x7f7e7006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 --2- 172.21.15.87:0/2351807333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e7015dd30 0x7f7e7006ac90 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009e50 tx=0x7f7e6001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 -- 172.21.15.87:0/2351807333 shutdown_connections 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 --2- 172.21.15.87:0/2351807333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700d1fb0 0x7f7e700b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 --2- 172.21.15.87:0/2351807333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e7015dd30 0x7f7e7006ac90 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 -- 172.21.15.87:0/2351807333 >> 172.21.15.87:0/2351807333 conn(0x7f7e7009a9d0 msgr2=0x7f7e7009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 -- 172.21.15.87:0/2351807333 shutdown_connections 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.298+0000 7f7e77000640 1 -- 172.21.15.87:0/2351807333 wait complete. 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 Processor -- start 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 -- start start 2024-04-25T10:26:18.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb30 0x7f7e7015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700d1fb0 0x7f7e7014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7015dd30 0x7f7e7015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7016ab60 con 0x7f7e700d1fb0 2024-04-25T10:26:18.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b28d0 con 0x7f7e7015dd30 2024-04-25T10:26:18.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7016ace0 con 0x7f7e700cfb30 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700d1fb0 0x7f7e7014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700d1fb0 0x7f7e7014d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42894/0 (socket says 172.21.15.87:42894) 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 -- 172.21.15.87:0/2224908131 learned_addr learned my addr 172.21.15.87:0/2224908131 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6fe00640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb30 0x7f7e7015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e75000640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7015dd30 0x7f7e7015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 -- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb30 msgr2=0x7f7e7015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb30 0x7f7e7015ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 -- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7015dd30 msgr2=0x7f7e7015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7015dd30 0x7f7e7015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 -- 172.21.15.87:0/2224908131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7015d370 con 0x7f7e700d1fb0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e6f400640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700d1fb0 0x7f7e7014d510 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c4c0 tx=0x7f7e5c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e67e00640 1 -- 172.21.15.87:0/2224908131 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c017070 con 0x7f7e700d1fb0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e67e00640 1 -- 172.21.15.87:0/2224908131 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c003600 con 0x7f7e700d1fb0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e67e00640 1 -- 172.21.15.87:0/2224908131 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0039f0 con 0x7f7e700d1fb0 2024-04-25T10:26:18.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e70158d60 con 0x7f7e700d1fb0 2024-04-25T10:26:18.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e70159320 con 0x7f7e700d1fb0 2024-04-25T10:26:18.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.302+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7004ef30 con 0x7f7e700d1fb0 2024-04-25T10:26:18.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.306+0000 7f7e67e00640 1 -- 172.21.15.87:0/2224908131 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e5c003b90 con 0x7f7e700d1fb0 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.306+0000 7f7e67e00640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e50066f60 0x7f7e50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.306+0000 7f7e67e00640 1 -- 172.21.15.87:0/2224908131 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e5c084240 con 0x7f7e700d1fb0 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.306+0000 7f7e6fe00640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e50066f60 0x7f7e50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.306+0000 7f7e6fe00640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e50066f60 0x7f7e50069420 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7e600ae000 tx=0x7f7e600a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.310+0000 7f7e67e00640 1 -- 172.21.15.87:0/2224908131 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e5c054b60 con 0x7f7e700d1fb0 2024-04-25T10:26:18.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.462+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e700b7920 con 0x7f7e700d1fb0 2024-04-25T10:26:18.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e67e00640 1 -- 172.21.15.87:0/2224908131 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v187) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e5c058c00 con 0x7f7e700d1fb0 2024-04-25T10:26:18.465 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e50066f60 msgr2=0x7f7e50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e50066f60 0x7f7e50069420 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7e600ae000 tx=0x7f7e600a7070 comp rx=0 tx=0).stop 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700d1fb0 msgr2=0x7f7e7014d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700d1fb0 0x7f7e7014d510 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c4c0 tx=0x7f7e5c001e80 comp rx=0 tx=0).stop 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 shutdown_connections 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7e50066f60 0x7f7e50069420 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7015dd30 0x7f7e7015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e700d1fb0 0x7f7e7014d510 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 --2- 172.21.15.87:0/2224908131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700cfb30 0x7f7e7015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 >> 172.21.15.87:0/2224908131 conn(0x7f7e7009a9d0 msgr2=0x7f7e700cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 shutdown_connections 2024-04-25T10:26:18.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.466+0000 7f7e77000640 1 -- 172.21.15.87:0/2224908131 wait complete. 2024-04-25T10:26:18.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:26:18.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4363600640 1 Processor -- start 2024-04-25T10:26:18.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4363600640 1 -- start start 2024-04-25T10:26:18.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4363600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4363600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f435c0d1830 0x7f435c0d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4363600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c15fec0 con 0x7f435c15bc10 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4363600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c1601a0 con 0x7f435c0d1830 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4363600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0cc5b0 con 0x7f435c0cf980 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4360c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f435c15bc10 0x7f435c15f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48294/0 (socket says 172.21.15.87:48294) 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4360c00640 1 -- 172.21.15.87:0/296011776 learned_addr learned my addr 172.21.15.87:0/296011776 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4361600640 1 --2- 172.21.15.87:0/296011776 >> v2:172.21.15.169:3300/0 conn(0x7f435c0d1830 0x7f435c0d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f435be00640 1 --2- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f435b400640 1 -- 172.21.15.87:0/296011776 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307291369 0 0) 0x7f435c15fec0 con 0x7f435c15bc10 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f435b400640 1 -- 172.21.15.87:0/296011776 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4340003530 con 0x7f435c15bc10 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4361600640 1 -- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 msgr2=0x7f435c0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4361600640 1 --2- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c0b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4361600640 1 -- 172.21.15.87:0/296011776 >> v1:172.21.15.87:6789/0 conn(0x7f435c15bc10 legacy=0x7f435c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.626+0000 7f4361600640 1 -- 172.21.15.87:0/296011776 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0cc890 con 0x7f435c0d1830 2024-04-25T10:26:18.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f4361600640 1 --2- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 0x7f435c0d1c60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f43540097c0 tx=0x7f435401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e795e3e3a4f509f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f435b400640 1 -- 172.21.15.87:0/296011776 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540b0070 con 0x7f435c0d1830 2024-04-25T10:26:18.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f435b400640 1 -- 172.21.15.87:0/296011776 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540ab600 con 0x7f435c0d1830 2024-04-25T10:26:18.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f435b400640 1 -- 172.21.15.87:0/296011776 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540ab9b0 con 0x7f435c0d1830 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f4363600640 1 -- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 msgr2=0x7f435c0d1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f4363600640 1 --2- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 0x7f435c0d1c60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f43540097c0 tx=0x7f435401c410 comp rx=0 tx=0).stop 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f4363600640 1 -- 172.21.15.87:0/296011776 shutdown_connections 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f4363600640 1 --2- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 0x7f435c0d1c60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f4363600640 1 --2- 172.21.15.87:0/296011776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c0b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.630+0000 7f4363600640 1 -- 172.21.15.87:0/296011776 >> 172.21.15.87:0/296011776 conn(0x7f435c09a9d0 msgr2=0x7f435c09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 -- 172.21.15.87:0/296011776 shutdown_connections 2024-04-25T10:26:18.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 -- 172.21.15.87:0/296011776 wait complete. 2024-04-25T10:26:18.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 Processor -- start 2024-04-25T10:26:18.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 -- start start 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 0x7f435c0c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f435c15bc10 0x7f435c165fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0cd740 con 0x7f435c15bc10 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0cd5c0 con 0x7f435c0d1830 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4363600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c168960 con 0x7f435c0cf980 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f435c15bc10 0x7f435c165fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f435c15bc10 0x7f435c165fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42904/0 (socket says 172.21.15.87:42904) 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 -- 172.21.15.87:0/4008908227 learned_addr learned my addr 172.21.15.87:0/4008908227 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:18.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f435be00640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 0x7f435c0c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4360c00640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 -- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 msgr2=0x7f435c153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 -- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 msgr2=0x7f435c0c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 0x7f435c0c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 -- 172.21.15.87:0/4008908227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0c5a10 con 0x7f435c15bc10 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4361600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f435c15bc10 0x7f435c165fb0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f435401a040 tx=0x7f43540191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.634+0000 7f4358c00640 1 -- 172.21.15.87:0/4008908227 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540b0040 con 0x7f435c15bc10 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4358c00640 1 -- 172.21.15.87:0/4008908227 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540b9070 con 0x7f435c15bc10 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4358c00640 1 -- 172.21.15.87:0/4008908227 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354006e20 con 0x7f435c15bc10 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c1664f0 con 0x7f435c15bc10 2024-04-25T10:26:18.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c15b2e0 con 0x7f435c15bc10 2024-04-25T10:26:18.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c04ef30 con 0x7f435c15bc10 2024-04-25T10:26:18.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4358c00640 1 -- 172.21.15.87:0/4008908227 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43540b4630 con 0x7f435c15bc10 2024-04-25T10:26:18.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4358c00640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f433c066f60 0x7f433c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4358c00640 1 -- 172.21.15.87:0/4008908227 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f435411ec70 con 0x7f435c15bc10 2024-04-25T10:26:18.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.638+0000 7f4360c00640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f433c066f60 0x7f433c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.642+0000 7f4360c00640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f433c066f60 0x7f433c069420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f434c005c10 tx=0x7f434c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.642+0000 7f4358c00640 1 -- 172.21.15.87:0/4008908227 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43540a4e20 con 0x7f435c15bc10 2024-04-25T10:26:18.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.798+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f435c0c8500 con 0x7f435c15bc10 2024-04-25T10:26:18.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.798+0000 7f4358c00640 1 -- 172.21.15.87:0/4008908227 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v187) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4354016090 con 0x7f435c15bc10 2024-04-25T10:26:18.799 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:26:18.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f433c066f60 msgr2=0x7f433c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f433c066f60 0x7f433c069420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f434c005c10 tx=0x7f434c009040 comp rx=0 tx=0).stop 2024-04-25T10:26:18.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f435c15bc10 msgr2=0x7f435c165fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f435c15bc10 0x7f435c165fb0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f435401a040 tx=0x7f43540191b0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 shutdown_connections 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f433c066f60 0x7f433c069420 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f435c15bc10 0x7f435c165fb0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f435c0d1830 0x7f435c0c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 --2- 172.21.15.87:0/4008908227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f435c0cf980 0x7f435c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 >> 172.21.15.87:0/4008908227 conn(0x7f435c09a9d0 msgr2=0x7f435c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 shutdown_connections 2024-04-25T10:26:18.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.802+0000 7f4363600640 1 -- 172.21.15.87:0/4008908227 wait complete. 2024-04-25T10:26:18.821 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:26:18.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4919000640 1 Processor -- start 2024-04-25T10:26:18.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4919000640 1 -- start start 2024-04-25T10:26:18.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4919000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f491415a0e0 0x7f491415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4919000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4914071f70 0x7f49140b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4919000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491415b6e0 con 0x7f49140b2fc0 2024-04-25T10:26:18.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4919000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4914070ed0 con 0x7f491415a0e0 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4919000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49140711b0 con 0x7f4914071f70 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4911e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f491415a0e0 0x7f491415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4911e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f491415a0e0 0x7f491415b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42920/0 (socket says 172.21.15.87:42920) 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4911e00640 1 -- 172.21.15.87:0/1149776359 learned_addr learned my addr 172.21.15.87:0/1149776359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4909400640 1 --2- 172.21.15.87:0/1149776359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4914071f70 0x7f49140b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4911e00640 1 -- 172.21.15.87:0/1149776359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4914071f70 msgr2=0x7f49140b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4911e00640 1 --2- 172.21.15.87:0/1149776359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4914071f70 0x7f49140b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4911e00640 1 -- 172.21.15.87:0/1149776359 >> v1:172.21.15.169:6789/0 conn(0x7f49140b2fc0 legacy=0x7f491406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4911e00640 1 -- 172.21.15.87:0/1149776359 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4914071490 con 0x7f491415a0e0 2024-04-25T10:26:18.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.958+0000 7f4909400640 1 --2- 172.21.15.87:0/1149776359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4914071f70 0x7f49140b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:18.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4911e00640 1 --2- 172.21.15.87:0/1149776359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491415a0e0 0x7f491415b1a0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f4904009870 tx=0x7f490401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a910d7d0827de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4911400640 1 -- 172.21.15.87:0/1149776359 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49040b0070 con 0x7f491415a0e0 2024-04-25T10:26:18.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4911400640 1 -- 172.21.15.87:0/1149776359 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49040ab600 con 0x7f491415a0e0 2024-04-25T10:26:18.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4911400640 1 -- 172.21.15.87:0/1149776359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49040ab9d0 con 0x7f491415a0e0 2024-04-25T10:26:18.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 -- 172.21.15.87:0/1149776359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491415a0e0 msgr2=0x7f491415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 --2- 172.21.15.87:0/1149776359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491415a0e0 0x7f491415b1a0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f4904009870 tx=0x7f490401c340 comp rx=0 tx=0).stop 2024-04-25T10:26:18.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 -- 172.21.15.87:0/1149776359 shutdown_connections 2024-04-25T10:26:18.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 --2- 172.21.15.87:0/1149776359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4914071f70 0x7f49140b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 --2- 172.21.15.87:0/1149776359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f491415a0e0 0x7f491415b1a0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 -- 172.21.15.87:0/1149776359 >> 172.21.15.87:0/1149776359 conn(0x7f4914066b70 msgr2=0x7f4914066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 -- 172.21.15.87:0/1149776359 shutdown_connections 2024-04-25T10:26:18.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.962+0000 7f4919000640 1 -- 172.21.15.87:0/1149776359 wait complete. 2024-04-25T10:26:18.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 Processor -- start 2024-04-25T10:26:18.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 -- start start 2024-04-25T10:26:18.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4914071f70 0x7f4914141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49140b2fc0 0x7f49141569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491415a0e0 0x7f491414f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491415a4c0 con 0x7f49140b2fc0 2024-04-25T10:26:18.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49140af030 con 0x7f4914071f70 2024-04-25T10:26:18.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4914070b50 con 0x7f491415a0e0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49140b2fc0 0x7f49141569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49140b2fc0 0x7f49141569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42924/0 (socket says 172.21.15.87:42924) 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 -- 172.21.15.87:0/1419047022 learned_addr learned my addr 172.21.15.87:0/1419047022 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4911e00640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4914071f70 0x7f4914141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4912800640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491415a0e0 0x7f491414f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 -- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491415a0e0 msgr2=0x7f491414f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491415a0e0 0x7f491414f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 -- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4914071f70 msgr2=0x7f4914141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4914071f70 0x7f4914141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 -- 172.21.15.87:0/1419047022 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49141462b0 con 0x7f49140b2fc0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4912800640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491415a0e0 0x7f491414f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4909400640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49140b2fc0 0x7f49141569d0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c7c0 tx=0x7f48fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f490aa00640 1 -- 172.21.15.87:0/1419047022 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc007a30 con 0x7f49140b2fc0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f490aa00640 1 -- 172.21.15.87:0/1419047022 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc007bd0 con 0x7f49140b2fc0 2024-04-25T10:26:18.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4914141a90 con 0x7f49140b2fc0 2024-04-25T10:26:18.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f490aa00640 1 -- 172.21.15.87:0/1419047022 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0113f0 con 0x7f49140b2fc0 2024-04-25T10:26:18.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.966+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49141575e0 con 0x7f49140b2fc0 2024-04-25T10:26:18.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.970+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491404ef30 con 0x7f49140b2fc0 2024-04-25T10:26:18.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.970+0000 7f490aa00640 1 -- 172.21.15.87:0/1419047022 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48fc011590 con 0x7f49140b2fc0 2024-04-25T10:26:18.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.970+0000 7f490aa00640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48f0066f60 0x7f48f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.970+0000 7f490aa00640 1 -- 172.21.15.87:0/1419047022 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48fc085170 con 0x7f49140b2fc0 2024-04-25T10:26:18.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.970+0000 7f4911e00640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48f0066f60 0x7f48f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.970+0000 7f4911e00640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48f0066f60 0x7f48f0069420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f49040ad000 tx=0x7f49040a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:18.974+0000 7f490aa00640 1 -- 172.21.15.87:0/1419047022 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48fc015020 con 0x7f49140b2fc0 2024-04-25T10:26:19.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.138+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f49140703e0 con 0x7f49140b2fc0 2024-04-25T10:26:19.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.138+0000 7f490aa00640 1 -- 172.21.15.87:0/1419047022 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v187) v1 ==== 80+0+0 (secure 0 0 0) 0x7f48fc055b20 con 0x7f49140b2fc0 2024-04-25T10:26:19.140 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-04-25T10:26:19.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48f0066f60 msgr2=0x7f48f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48f0066f60 0x7f48f0069420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f49040ad000 tx=0x7f49040a6000 comp rx=0 tx=0).stop 2024-04-25T10:26:19.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49140b2fc0 msgr2=0x7f49141569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49140b2fc0 0x7f49141569d0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c7c0 tx=0x7f48fc00cc80 comp rx=0 tx=0).stop 2024-04-25T10:26:19.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 shutdown_connections 2024-04-25T10:26:19.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f48f0066f60 0x7f48f0069420 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f491415a0e0 0x7f491414f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49140b2fc0 0x7f49141569d0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 --2- 172.21.15.87:0/1419047022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4914071f70 0x7f4914141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 >> 172.21.15.87:0/1419047022 conn(0x7f4914066b70 msgr2=0x7f491415a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 shutdown_connections 2024-04-25T10:26:19.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.142+0000 7f4919000640 1 -- 172.21.15.87:0/1419047022 wait complete. 2024-04-25T10:26:19.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:26:19.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:19.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:19.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:26:19.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:19.210 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:26:19.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:19.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfdae00640 1 Processor -- start 2024-04-25T10:26:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfdae00640 1 -- start start 2024-04-25T10:26:19.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfdae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cd2e0 0x7fbfd40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfdae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd415f3b0 con 0x7fbfd40cfd90 2024-04-25T10:26:19.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfdae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd40ca810 con 0x7fbfd40b0f70 2024-04-25T10:26:19.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfdae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd415ef20 con 0x7fbfd40cd2e0 2024-04-25T10:26:19.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbfd40cfd90 0x7fbfd40ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48296/0 (socket says 172.21.15.87:48296) 2024-04-25T10:26:19.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3400640 1 -- 172.21.15.87:0/1804910646 learned_addr learned my addr 172.21.15.87:0/1804910646 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:19.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495162847 0 0) 0x7fbfd415f3b0 con 0x7fbfd40cfd90 2024-04-25T10:26:19.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb8003530 con 0x7fbfd40cfd90 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd8e00640 1 --2- 172.21.15.87:0/1804910646 >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cd2e0 0x7fbfd40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 110578007 0 0) 0x7fbfb8003530 con 0x7fbfd40cfd90 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cd2e0 msgr2=0x7fbfd40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 --2- 172.21.15.87:0/1804910646 >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cd2e0 0x7fbfd40cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 >> v1:172.21.15.169:6789/0 conn(0x7fbfd40b0f70 legacy=0x7fbfd40b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd415e5f0 con 0x7fbfd40cfd90 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fbfc0001680 con 0x7fbfd40cfd90 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd8e00640 1 --2- 172.21.15.87:0/1804910646 >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cd2e0 0x7fbfd40cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 >> v1:172.21.15.87:6789/0 conn(0x7fbfd40cfd90 legacy=0x7fbfd40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfb8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfb8004810 0x7fbfb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfb8005240 0x7fbfb800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd415ef20 con 0x7fbfb8003a10 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb800c1a0 con 0x7fbfb8005240 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb800c3a0 con 0x7fbfb8004810 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3e00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfb8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3400640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfb8004810 0x7fbfb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd8e00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfb8005240 0x7fbfb800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3e00640 1 -- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfb8004810 msgr2=0x7fbfb8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3e00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfb8004810 0x7fbfb8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3e00640 1 -- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfb8005240 msgr2=0x7fbfb800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3e00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfb8005240 0x7fbfb800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3e00640 1 -- 172.21.15.87:0/1804910646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8003530 con 0x7fbfb8003a10 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd8e00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfb8005240 0x7fbfb800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd3400640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfb8004810 0x7fbfb8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:19.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.406+0000 7fbfd8e00640 1 -- 172.21.15.87:0/1804910646 reap_dead start 2024-04-25T10:26:19.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfd3e00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfb8003e50 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001e70 tx=0x7fbfc401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368ea6e444a3e6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40b0070 con 0x7fbfb8003a10 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc4003b80 con 0x7fbfb8003a10 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfd2a00640 1 -- 172.21.15.87:0/1804910646 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4001a90 con 0x7fbfb8003a10 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfdae00640 1 -- 172.21.15.87:0/1804910646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 msgr2=0x7fbfb8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfdae00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfb8003e50 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001e70 tx=0x7fbfc401c760 comp rx=0 tx=0).stop 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfdae00640 1 -- 172.21.15.87:0/1804910646 shutdown_connections 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfdae00640 1 --2- 172.21.15.87:0/1804910646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfb8003e50 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfdae00640 1 -- 172.21.15.87:0/1804910646 >> 172.21.15.87:0/1804910646 conn(0x7fbfd40b1d50 msgr2=0x7fbfd40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfdae00640 1 -- 172.21.15.87:0/1804910646 shutdown_connections 2024-04-25T10:26:19.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.410+0000 7fbfdae00640 1 -- 172.21.15.87:0/1804910646 wait complete. 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 Processor -- start 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 -- start start 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfd41f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4156130 0x7fbfd40afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd40c5c30 0x7fbfd40c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40b18e0 con 0x7fbfb8003a10 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40b1760 con 0x7fbfd4156130 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40cc230 con 0x7fbfd40c5c30 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfd41f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfd41f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42942/0 (socket says 172.21.15.87:42942) 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 -- 172.21.15.87:0/2012749664 learned_addr learned my addr 172.21.15.87:0/2012749664 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd8e00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd40c5c30 0x7fbfd40c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3400640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4156130 0x7fbfd40afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 -- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd40c5c30 msgr2=0x7fbfd40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd40c5c30 0x7fbfd40c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 -- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4156130 msgr2=0x7fbfd40afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4156130 0x7fbfd40afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 -- 172.21.15.87:0/2012749664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40cc5c0 con 0x7fbfb8003a10 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfd3e00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfd41f8720 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fbfc401cc40 tx=0x7fbfb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfcbe00640 1 -- 172.21.15.87:0/2012749664 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40b9020 con 0x7fbfb8003a10 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfcbe00640 1 -- 172.21.15.87:0/2012749664 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc40050d0 con 0x7fbfb8003a10 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfcbe00640 1 -- 172.21.15.87:0/2012749664 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40b0040 con 0x7fbfb8003a10 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd41565b0 con 0x7fbfb8003a10 2024-04-25T10:26:19.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.414+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd41fb5f0 con 0x7fbfb8003a10 2024-04-25T10:26:19.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.418+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd404ef30 con 0x7fbfb8003a10 2024-04-25T10:26:19.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.418+0000 7fbfcbe00640 1 -- 172.21.15.87:0/2012749664 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfc40bc020 con 0x7fbfb8003a10 2024-04-25T10:26:19.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.418+0000 7fbfcbe00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfb4066f90 0x7fbfb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.418+0000 7fbfcbe00640 1 -- 172.21.15.87:0/2012749664 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfc411ed60 con 0x7fbfb8003a10 2024-04-25T10:26:19.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.418+0000 7fbfd3400640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfb4066f90 0x7fbfb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.418+0000 7fbfd3400640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfb4066f90 0x7fbfb4069450 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0001040 tx=0x7fbfc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.422+0000 7fbfcbe00640 1 -- 172.21.15.87:0/2012749664 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfc40c3030 con 0x7fbfb8003a10 2024-04-25T10:26:19.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfd40cda20 con 0x7fbfb8003a10 2024-04-25T10:26:19.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfcbe00640 1 -- 172.21.15.87:0/2012749664 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4955 (secure 0 0 0) 0x7fbfc4016470 con 0x7fbfb8003a10 2024-04-25T10:26:19.610 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:19.610 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":137,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:12.151950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11304":{"gid":11304,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6836/997618220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":997618220},{"type":"v1","addr":"172.21.15.87:6836","nonce":997618220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:26:19.610 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 137 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfb4066f90 msgr2=0x7fbfb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfb4066f90 0x7fbfb4069450 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0001040 tx=0x7fbfc0009040 comp rx=0 tx=0).stop 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 msgr2=0x7fbfd41f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfd41f8720 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fbfc401cc40 tx=0x7fbfb8005840 comp rx=0 tx=0).stop 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 shutdown_connections 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fbfb4066f90 0x7fbfb4069450 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd40c5c30 0x7fbfd40c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4156130 0x7fbfd40afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 --2- 172.21.15.87:0/2012749664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfb8003a10 0x7fbfd41f8720 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 >> 172.21.15.87:0/2012749664 conn(0x7fbfd40b1d50 msgr2=0x7fbfd41617d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 shutdown_connections 2024-04-25T10:26:19.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.610+0000 7fbfdae00640 1 -- 172.21.15.87:0/2012749664 wait complete. 2024-04-25T10:26:19.632 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:26:19.792 INFO:teuthology.orchestra.run.smithi087.stdout:{"message":"","return_code":0} 2024-04-25T10:26:19.793 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:26:19.793 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:26:19.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.930+0000 7fa443200640 1 Processor -- start 2024-04-25T10:26:19.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.930+0000 7fa443200640 1 -- start start 2024-04-25T10:26:19.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa443200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa43c0cda10 0x7fa43c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa443200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43c0b2770 con 0x7fa43c0cf930 2024-04-25T10:26:19.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa443200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43c06c670 con 0x7fa43c15bc40 2024-04-25T10:26:19.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa443200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c06c950 con 0x7fa43c0cda10 2024-04-25T10:26:19.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa441200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa43c0cf930 0x7fa43c0b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48310/0 (socket says 172.21.15.87:48310) 2024-04-25T10:26:19.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa441200640 1 -- 172.21.15.87:0/1129590988 learned_addr learned my addr 172.21.15.87:0/1129590988 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:19.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa433400640 1 --2- 172.21.15.87:0/1129590988 >> v2:172.21.15.169:3300/0 conn(0x7fa43c0cda10 0x7fa43c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368541427 0 0) 0x7fa43c0b2770 con 0x7fa43c0cf930 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa420003530 con 0x7fa43c0cf930 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314257535 0 0) 0x7fa43c06c670 con 0x7fa43c15bc40 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa43c0b2770 con 0x7fa43c15bc40 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa433400640 1 -- 172.21.15.87:0/1129590988 >> v1:172.21.15.169:6789/0 conn(0x7fa43c15bc40 legacy=0x7fa43c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa433400640 1 -- 172.21.15.87:0/1129590988 >> v1:172.21.15.87:6789/0 conn(0x7fa43c0cf930 legacy=0x7fa43c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3050369135 0 0) 0x7fa420003530 con 0x7fa43c0cf930 2024-04-25T10:26:19.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.934+0000 7fa433400640 1 -- 172.21.15.87:0/1129590988 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c06cc30 con 0x7fa43c0cda10 2024-04-25T10:26:19.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa433400640 1 --2- 172.21.15.87:0/1129590988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cda10 0x7fa43c06c090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097c0 tx=0x7fa42401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb113cf4e530d17d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b0070 con 0x7fa43c0cda10 2024-04-25T10:26:19.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4240ab400 con 0x7fa43c0cda10 2024-04-25T10:26:19.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa43b400640 1 -- 172.21.15.87:0/1129590988 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434001a00 con 0x7fa43c0cda10 2024-04-25T10:26:19.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- 172.21.15.87:0/1129590988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cda10 msgr2=0x7fa43c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 --2- 172.21.15.87:0/1129590988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cda10 0x7fa43c06c090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097c0 tx=0x7fa42401c330 comp rx=0 tx=0).stop 2024-04-25T10:26:19.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- 172.21.15.87:0/1129590988 shutdown_connections 2024-04-25T10:26:19.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 --2- 172.21.15.87:0/1129590988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cda10 0x7fa43c06c090 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- 172.21.15.87:0/1129590988 >> 172.21.15.87:0/1129590988 conn(0x7fa43c066b70 msgr2=0x7fa43c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- 172.21.15.87:0/1129590988 shutdown_connections 2024-04-25T10:26:19.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- 172.21.15.87:0/1129590988 wait complete. 2024-04-25T10:26:19.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 Processor -- start 2024-04-25T10:26:19.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- start start 2024-04-25T10:26:19.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0cda10 0x7fa43c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cf930 0x7fa43c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c15bc40 0x7fa43c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c169d70 con 0x7fa43c15bc40 2024-04-25T10:26:19.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c169bf0 con 0x7fa43c0cf930 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa443200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c169ef0 con 0x7fa43c0cda10 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.938+0000 7fa441200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c15bc40 0x7fa43c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c15bc40 0x7fa43c0b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42948/0 (socket says 172.21.15.87:42948) 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 -- 172.21.15.87:0/1180419277 learned_addr learned my addr 172.21.15.87:0/1180419277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa433400640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cf930 0x7fa43c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa43be00640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0cda10 0x7fa43c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 -- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0cda10 msgr2=0x7fa43c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0cda10 0x7fa43c15a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 -- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cf930 msgr2=0x7fa43c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cf930 0x7fa43c15a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 -- 172.21.15.87:0/1180419277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c13cd10 con 0x7fa43c15bc40 2024-04-25T10:26:19.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa433400640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cf930 0x7fa43c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa43be00640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0cda10 0x7fa43c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa441200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c15bc40 0x7fa43c0b0430 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fa434001fc0 tx=0x7fa43400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa438c00640 1 -- 172.21.15.87:0/1180419277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434017070 con 0x7fa43c15bc40 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa438c00640 1 -- 172.21.15.87:0/1180419277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa434012400 con 0x7fa43c15bc40 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa438c00640 1 -- 172.21.15.87:0/1180419277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434012830 con 0x7fa43c15bc40 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c13fab0 con 0x7fa43c15bc40 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c1532a0 con 0x7fa43c15bc40 2024-04-25T10:26:19.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c04ef30 con 0x7fa43c15bc40 2024-04-25T10:26:19.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa438c00640 1 -- 172.21.15.87:0/1180419277 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa43401a070 con 0x7fa43c15bc40 2024-04-25T10:26:19.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa438c00640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f90 0x7fa418069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa438c00640 1 -- 172.21.15.87:0/1180419277 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa434084ff0 con 0x7fa43c15bc40 2024-04-25T10:26:19.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.942+0000 7fa43be00640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f90 0x7fa418069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.946+0000 7fa43be00640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f90 0x7fa418069450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0011f0 tx=0x7fa42c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:19.950+0000 7fa438c00640 1 -- 172.21.15.87:0/1180419277 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa434055ff0 con 0x7fa43c15bc40 2024-04-25T10:26:20.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.138+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa43c06d960 con 0x7fa43c15bc40 2024-04-25T10:26:20.139 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:26:20.138+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:26:20.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa438c00640 1 -- 172.21.15.87:0/1180419277 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v138) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa4340558f0 con 0x7fa43c15bc40 2024-04-25T10:26:20.141 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f90 msgr2=0x7fa418069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f90 0x7fa418069450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0011f0 tx=0x7fa42c009040 comp rx=0 tx=0).stop 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c15bc40 msgr2=0x7fa43c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c15bc40 0x7fa43c0b0430 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fa434001fc0 tx=0x7fa43400a430 comp rx=0 tx=0).stop 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 shutdown_connections 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa418066f90 0x7fa418069450 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa43c15bc40 0x7fa43c0b0430 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa43c0cf930 0x7fa43c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 --2- 172.21.15.87:0/1180419277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43c0cda10 0x7fa43c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 >> 172.21.15.87:0/1180419277 conn(0x7fa43c066b70 msgr2=0x7fa43c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 shutdown_connections 2024-04-25T10:26:20.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.142+0000 7fa443200640 1 -- 172.21.15.87:0/1180419277 wait complete. 2024-04-25T10:26:20.154 INFO:tasks.ceph.mds.a.smithi087.stderr: -5277> 2024-04-25T10:26:15.735+0000 7fbf26200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:26:20.165 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-25T10:26:20.203 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:26:20.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 Processor -- start 2024-04-25T10:26:20.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- start start 2024-04-25T10:26:20.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b4f09a0 0x56203b574f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f0d80 0x56203b578560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b57a870 0x56203b4a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56203b4499f0 con 0x56203b57a870 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56203b44d950 con 0x56203b4f0d80 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56203b4a26d0 con 0x56203b4f09a0 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b57a870 0x56203b4a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b57a870 0x56203b4a4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42972/0 (socket says 172.21.15.87:42972) 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 -- 172.21.15.87:0/708881359 learned_addr learned my addr 172.21.15.87:0/708881359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 -- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b4f09a0 msgr2=0x56203b574f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee13a00640 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b4f09a0 0x56203b574f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee13000640 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f0d80 0x56203b578560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b4f09a0 0x56203b574f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 -- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f0d80 msgr2=0x56203b578560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f0d80 0x56203b578560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 -- 172.21.15.87:0/708881359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56203b5742b0 con 0x56203b57a870 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee13a00640 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b4f09a0 0x56203b574f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee13000640 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f0d80 0x56203b578560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee14400640 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b57a870 0x56203b4a4100 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fee0c001610 tx=0x7fee0c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c16fa7a6343f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee11c00640 1 -- 172.21.15.87:0/708881359 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00f510 con 0x56203b57a870 2024-04-25T10:26:20.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee11c00640 1 -- 172.21.15.87:0/708881359 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c00f6b0 con 0x56203b57a870 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee11c00640 1 -- 172.21.15.87:0/708881359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00fae0 con 0x56203b57a870 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- 172.21.15.87:0/708881359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b57a870 msgr2=0x56203b4a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b57a870 0x56203b4a4100 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fee0c001610 tx=0x7fee0c006f90 comp rx=0 tx=0).stop 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- 172.21.15.87:0/708881359 shutdown_connections 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b57a870 0x56203b4a4100 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f0d80 0x56203b578560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 --2- 172.21.15.87:0/708881359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b4f09a0 0x56203b574f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- 172.21.15.87:0/708881359 >> 172.21.15.87:0/708881359 conn(0x56203b4d1340 msgr2=0x56203b579790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- 172.21.15.87:0/708881359 shutdown_connections 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.214+0000 7fee1676a080 1 -- 172.21.15.87:0/708881359 wait complete. 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 Processor -- start 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 -- start start 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f09a0 0x56203b572680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b4f0d80 0x56203b570be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b57a870 0x56203b571120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56203b5747f0 con 0x56203b4f0d80 2024-04-25T10:26:20.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56203b574550 con 0x56203b4f09a0 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56203b574a90 con 0x56203b57a870 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b4f0d80 0x56203b570be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b4f0d80 0x56203b570be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42982/0 (socket says 172.21.15.87:42982) 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 -- 172.21.15.87:0/4052777124 learned_addr learned my addr 172.21.15.87:0/4052777124 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 -- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b57a870 msgr2=0x56203b571120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56203b57a870 0x56203b571120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 -- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f09a0 msgr2=0x56203b572680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56203b4f09a0 0x56203b572680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 -- 172.21.15.87:0/4052777124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56203b5704f0 con 0x56203b4f0d80 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee13000640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b4f0d80 0x56203b570be0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fee00018f50 tx=0x7fee000180c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee0be00640 1 -- 172.21.15.87:0/4052777124 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee000b9020 con 0x56203b4f0d80 2024-04-25T10:26:20.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee0be00640 1 -- 172.21.15.87:0/4052777124 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee000b0040 con 0x56203b4f0d80 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee0be00640 1 -- 172.21.15.87:0/4052777124 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee000af070 con 0x56203b4f0d80 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56203b571760 con 0x56203b4f0d80 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.218+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56203b571ca0 con 0x56203b4f0d80 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee0be00640 1 -- 172.21.15.87:0/4052777124 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee000bc020 con 0x56203b4f0d80 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee0be00640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fedf4065970 0x7fedf4067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee0be00640 1 -- 172.21.15.87:0/4052777124 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fee0011eb20 con 0x56203b4f0d80 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee13a00640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fedf4065970 0x7fedf4067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x56203b56e9b0 0x56203b56edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 --> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] -- osd_op(unknown.0.0:1 28.10 28:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e188) v9 -- 0x56203b559240 con 0x56203b56e9b0 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee13a00640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fedf4065970 0x7fedf4067e30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fee040010f0 tx=0x7fee04001470 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee14400640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x56203b56e9b0 0x56203b56edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee14400640 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x56203b56e9b0 0x56203b56edc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee14400640 1 -- 172.21.15.87:0/4052777124 <== osd.4 v2:172.21.15.169:6800/3149326747 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=535b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+535 (crc 0 0 0) 0x7fee0c00f070 con 0x56203b56e9b0 2024-04-25T10:26:20.234 INFO:teuthology.orchestra.run.smithi087.stderr:Writing to - 2024-04-25T10:26:20.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x56203b56e9b0 msgr2=0x56203b56edc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x56203b56e9b0 0x56203b56edc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fedf4065970 msgr2=0x7fedf4067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fedf4065970 0x7fedf4067e30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fee040010f0 tx=0x7fee04001470 comp rx=0 tx=0).stop 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b4f0d80 msgr2=0x56203b570be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 --2- 172.21.15.87:0/4052777124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56203b4f0d80 0x56203b570be0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fee00018f50 tx=0x7fee000180c0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee14400640 1 -- 172.21.15.87:0/4052777124 reap_dead start 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 shutdown_connections 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 >> 172.21.15.87:0/4052777124 conn(0x56203b4d1340 msgr2=0x56203b4a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 shutdown_connections 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.222+0000 7fee1676a080 1 -- 172.21.15.87:0/4052777124 wait complete. 2024-04-25T10:26:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:i����/*f`V�7����@@`���������/*f`V�7�/*f �5(8`�/*f`V�78`�/*f`V�7���������/*f �5����`�������� 2024-04-25T10:26:20.236 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-25T10:26:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 Processor -- start 2024-04-25T10:26:20.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- start start 2024-04-25T10:26:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9accdb80 0x55ce9ad8a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86b40 0x55ce9acd0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9ad86f20 0x55ce9acd08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9ad87b30 con 0x55ce9ad86b40 2024-04-25T10:26:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9ac729f0 con 0x55ce9accdb80 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce9ad87dd0 con 0x55ce9ad86f20 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86b40 0x55ce9acd0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86b40 0x55ce9acd0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42996/0 (socket says 172.21.15.87:42996) 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 -- 172.21.15.87:0/2380957054 learned_addr learned my addr 172.21.15.87:0/2380957054 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4718e00640 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9accdb80 0x55ce9ad8a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4719800640 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9ad86f20 0x55ce9acd08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 -- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9ad86f20 msgr2=0x55ce9acd08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9ad86f20 0x55ce9acd08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 -- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9accdb80 msgr2=0x55ce9ad8a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9accdb80 0x55ce9ad8a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 -- 172.21.15.87:0/2380957054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce9ad85770 con 0x55ce9ad86b40 2024-04-25T10:26:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4719800640 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9ad86f20 0x55ce9acd08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4713e00640 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86b40 0x55ce9acd0370 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f47080b2040 tx=0x7f4708003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebd7f9b5b15e3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4712a00640 1 -- 172.21.15.87:0/2380957054 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080b4070 con 0x55ce9ad86b40 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4712a00640 1 -- 172.21.15.87:0/2380957054 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47080be070 con 0x55ce9ad86b40 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f4712a00640 1 -- 172.21.15.87:0/2380957054 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080b98a0 con 0x55ce9ad86b40 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- 172.21.15.87:0/2380957054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86b40 msgr2=0x55ce9acd0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86b40 0x55ce9acd0370 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f47080b2040 tx=0x7f4708003030 comp rx=0 tx=0).stop 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- 172.21.15.87:0/2380957054 shutdown_connections 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9ad86f20 0x55ce9acd08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86b40 0x55ce9acd0370 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 --2- 172.21.15.87:0/2380957054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9accdb80 0x55ce9ad8a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- 172.21.15.87:0/2380957054 >> 172.21.15.87:0/2380957054 conn(0x55ce9acf8d40 msgr2=0x55ce9ad17830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- 172.21.15.87:0/2380957054 shutdown_connections 2024-04-25T10:26:20.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.322+0000 7f471bc90080 1 -- 172.21.15.87:0/2380957054 wait complete. 2024-04-25T10:26:20.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 Processor -- start 2024-04-25T10:26:20.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 -- start start 2024-04-25T10:26:20.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9accdb80 0x55ce9ada2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9ad86b40 0x55ce9ada52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86f20 0x55ce9ad13090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9ad85510 con 0x55ce9ad86f20 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9ad85a10 con 0x55ce9ad86b40 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce9ad85cb0 con 0x55ce9accdb80 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86f20 0x55ce9ad13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86f20 0x55ce9ad13090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43000/0 (socket says 172.21.15.87:43000) 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 -- 172.21.15.87:0/2287260031 learned_addr learned my addr 172.21.15.87:0/2287260031 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4718e00640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9accdb80 0x55ce9ada2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4713e00640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9ad86b40 0x55ce9ada52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 -- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9accdb80 msgr2=0x55ce9ada2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce9accdb80 0x55ce9ada2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 -- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9ad86b40 msgr2=0x55ce9ada52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9ad86b40 0x55ce9ada52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 -- 172.21.15.87:0/2287260031 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce9ada2ac0 con 0x55ce9ad86f20 2024-04-25T10:26:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4713e00640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9ad86b40 0x55ce9ada52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4719800640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86f20 0x55ce9ad13090 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f4714004260 tx=0x7f471400c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4711600640 1 -- 172.21.15.87:0/2287260031 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714017070 con 0x55ce9ad86f20 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4711600640 1 -- 172.21.15.87:0/2287260031 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471400ce10 con 0x55ce9ad86f20 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce9ada4290 con 0x55ce9ad86f20 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f4711600640 1 -- 172.21.15.87:0/2287260031 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714001770 con 0x55ce9ad86f20 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.326+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce9ada47d0 con 0x55ce9ad86f20 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f4711600640 1 -- 172.21.15.87:0/2287260031 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f471401a070 con 0x55ce9ad86f20 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f4711600640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46fc065970 0x7f46fc067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f4711600640 1 -- 172.21.15.87:0/2287260031 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4714083500 con 0x55ce9ad86f20 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f4718e00640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46fc065970 0x7f46fc067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f471bc90080 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x55ce9ada3db0 0x55ce9ad8af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 --> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] -- osd_op(unknown.0.0:1 28.10 28:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=531b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188) v9 -- 0x55ce9ad8b4d0 con 0x55ce9ada3db0 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f4718e00640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46fc065970 0x7f46fc067e30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f470c0096f0 tx=0x7f470c005a60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f4713e00640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x55ce9ada3db0 0x55ce9ad8af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.330+0000 7f4713e00640 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x55ce9ada3db0 0x55ce9ad8af90 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f4713e00640 1 -- 172.21.15.87:0/2287260031 <== osd.4 v2:172.21.15.169:6800/3149326747 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v188'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f47080b7070 con 0x55ce9ada3db0 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x55ce9ada3db0 msgr2=0x55ce9ad8af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x55ce9ada3db0 0x55ce9ad8af90 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46fc065970 msgr2=0x7f46fc067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f46fc065970 0x7f46fc067e30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f470c0096f0 tx=0x7f470c005a60 comp rx=0 tx=0).stop 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86f20 msgr2=0x55ce9ad13090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 --2- 172.21.15.87:0/2287260031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce9ad86f20 0x55ce9ad13090 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f4714004260 tx=0x7f471400c5e0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f4719800640 1 -- 172.21.15.87:0/2287260031 reap_dead start 2024-04-25T10:26:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 shutdown_connections 2024-04-25T10:26:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 >> 172.21.15.87:0/2287260031 conn(0x55ce9acf8d40 msgr2=0x55ce9acd0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 shutdown_connections 2024-04-25T10:26:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.334+0000 7f471bc90080 1 -- 172.21.15.87:0/2287260031 wait complete. 2024-04-25T10:26:20.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-25T10:26:20.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.518+0000 7f03e2a00640 1 Processor -- start 2024-04-25T10:26:20.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.518+0000 7f03e2a00640 1 -- start start 2024-04-25T10:26:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 0x7f03dc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e2a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f03dc0721f0 0x7f03dc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e2a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc159920 con 0x7f03dc0aebd0 2024-04-25T10:26:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e2a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0af030 con 0x7f03dc0721f0 2024-04-25T10:26:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc15c2f0 con 0x7f03dc159540 2024-04-25T10:26:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03db400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f03dc0aebd0 0x7f03dc071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48344/0 (socket says 172.21.15.87:48344) 2024-04-25T10:26:20.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03db400640 1 -- 172.21.15.87:0/3664837544 learned_addr learned my addr 172.21.15.87:0/3664837544 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:20.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e0a00640 1 --2- 172.21.15.87:0/3664837544 >> v2:172.21.15.169:3300/0 conn(0x7f03dc0721f0 0x7f03dc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03dbe00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 0x7f03dc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418235255 0 0) 0x7f03dc159920 con 0x7f03dc0aebd0 2024-04-25T10:26:20.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c0003530 con 0x7f03dc0aebd0 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2420720572 0 0) 0x7f03c0003530 con 0x7f03dc0aebd0 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 msgr2=0x7f03dc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 0x7f03dc0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 >> v2:172.21.15.169:3300/0 conn(0x7f03dc0721f0 msgr2=0x7f03dc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 --2- 172.21.15.87:0/3664837544 >> v2:172.21.15.169:3300/0 conn(0x7f03dc0721f0 0x7f03dc070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc15c5d0 con 0x7f03dc0aebd0 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f03c8001680 con 0x7f03dc0aebd0 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 >> v1:172.21.15.87:6789/0 conn(0x7f03dc0aebd0 legacy=0x7f03dc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c00039c0 0x7f03c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 0x7f03c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03c0004d50 0x7f03c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc15c2f0 con 0x7f03c0004d50 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c000b9d0 con 0x7f03c00039c0 2024-04-25T10:26:20.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c000bb50 con 0x7f03dc159540 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e0a00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03c0004d50 0x7f03c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03dbe00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c00039c0 0x7f03c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03db400640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 0x7f03c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e0a00640 1 -- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 msgr2=0x7f03c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e0a00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 0x7f03c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e0a00640 1 -- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c00039c0 msgr2=0x7f03c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e0a00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c00039c0 0x7f03c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03e0a00640 1 -- 172.21.15.87:0/3664837544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0003530 con 0x7f03c0004d50 2024-04-25T10:26:20.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.522+0000 7f03db400640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc159540 0x7f03c0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:20.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e0a00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03c0004d50 0x7f03c00051d0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009ae0 tx=0x7f03d401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdfeb050e01438e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d40b2070 con 0x7f03c0004d50 2024-04-25T10:26:20.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d40a56a0 con 0x7f03c0004d50 2024-04-25T10:26:20.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03daa00640 1 -- 172.21.15.87:0/3664837544 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d40096b0 con 0x7f03c0004d50 2024-04-25T10:26:20.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e2a00640 1 -- 172.21.15.87:0/3664837544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03c0004d50 msgr2=0x7f03c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e2a00640 1 --2- 172.21.15.87:0/3664837544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03c0004d50 0x7f03c00051d0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009ae0 tx=0x7f03d401cab0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e0a00640 1 -- 172.21.15.87:0/3664837544 reap_dead start 2024-04-25T10:26:20.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e2a00640 1 -- 172.21.15.87:0/3664837544 shutdown_connections 2024-04-25T10:26:20.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e2a00640 1 -- 172.21.15.87:0/3664837544 >> 172.21.15.87:0/3664837544 conn(0x7f03dc0669a0 msgr2=0x7f03dc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e2a00640 1 -- 172.21.15.87:0/3664837544 shutdown_connections 2024-04-25T10:26:20.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.526+0000 7f03e2a00640 1 -- 172.21.15.87:0/3664837544 wait complete. 2024-04-25T10:26:20.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 Processor -- start 2024-04-25T10:26:20.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 -- start start 2024-04-25T10:26:20.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c0004d50 0x7f03dc150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03dc0b13d0 0x7f03dc0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc0b36f0 0x7f03dc0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc15bfa0 con 0x7f03dc0b13d0 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc15cb90 con 0x7f03c0004d50 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc06c0d0 con 0x7f03dc0b36f0 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03dc0b13d0 0x7f03dc0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03dc0b13d0 0x7f03dc0b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43018/0 (socket says 172.21.15.87:43018) 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 -- 172.21.15.87:0/342116479 learned_addr learned my addr 172.21.15.87:0/342116479 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03dbe00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c0004d50 0x7f03dc150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e0a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc0b36f0 0x7f03dc0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 -- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc0b36f0 msgr2=0x7f03dc0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc0b36f0 0x7f03dc0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 -- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c0004d50 msgr2=0x7f03dc150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c0004d50 0x7f03dc150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 -- 172.21.15.87:0/342116479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc1516d0 con 0x7f03dc0b13d0 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e0a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc0b36f0 0x7f03dc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03db400640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03dc0b13d0 0x7f03dc0b1850 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f03c8001bf0 tx=0x7f03c800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03d3e00640 1 -- 172.21.15.87:0/342116479 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8019070 con 0x7f03dc0b13d0 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03d3e00640 1 -- 172.21.15.87:0/342116479 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8004a00 con 0x7f03dc0b13d0 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03d3e00640 1 -- 172.21.15.87:0/342116479 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8004db0 con 0x7f03dc0b13d0 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc151150 con 0x7f03dc0b13d0 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc0b45b0 con 0x7f03dc0b13d0 2024-04-25T10:26:20.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.530+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc04ef30 con 0x7f03dc0b13d0 2024-04-25T10:26:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.534+0000 7f03d3e00640 1 -- 172.21.15.87:0/342116479 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03c801c070 con 0x7f03dc0b13d0 2024-04-25T10:26:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.534+0000 7f03d3e00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03bc066f60 0x7f03bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.534+0000 7f03d3e00640 1 -- 172.21.15.87:0/342116479 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f03c8084a30 con 0x7f03dc0b13d0 2024-04-25T10:26:20.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.534+0000 7f03dbe00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03bc066f60 0x7f03bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.534+0000 7f03dbe00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03bc066f60 0x7f03bc069420 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009fd0 tx=0x7f03cc00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.538+0000 7f03d3e00640 1 -- 172.21.15.87:0/342116479 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03c80552f0 con 0x7f03dc0b13d0 2024-04-25T10:26:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.690+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) v1 -- 0x7f03dc15ec20 con 0x7f03dc0b13d0 2024-04-25T10:26:20.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.694+0000 7f03d3e00640 1 -- 172.21.15.87:0/342116479 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v35)=0 v35) v1 ==== 140+0+0 (secure 0 0 0) 0x7f03c8059390 con 0x7f03dc0b13d0 2024-04-25T10:26:20.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03bc066f60 msgr2=0x7f03bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03bc066f60 0x7f03bc069420 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009fd0 tx=0x7f03cc00b040 comp rx=0 tx=0).stop 2024-04-25T10:26:20.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03dc0b13d0 msgr2=0x7f03dc0b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03dc0b13d0 0x7f03dc0b1850 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f03c8001bf0 tx=0x7f03c800ed70 comp rx=0 tx=0).stop 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 shutdown_connections 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f03bc066f60 0x7f03bc069420 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03dc0b36f0 0x7f03dc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03dc0b13d0 0x7f03dc0b1850 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 --2- 172.21.15.87:0/342116479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c0004d50 0x7f03dc150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 >> 172.21.15.87:0/342116479 conn(0x7f03dc0669a0 msgr2=0x7f03dc15df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 shutdown_connections 2024-04-25T10:26:20.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.698+0000 7f03e2a00640 1 -- 172.21.15.87:0/342116479 wait complete. 2024-04-25T10:26:20.717 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:26:20.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05f2600640 1 Processor -- start 2024-04-25T10:26:20.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05f2600640 1 -- start start 2024-04-25T10:26:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05f2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05ec0d0600 0x7f05ec166db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05f2600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec160650 con 0x7f05ec1600c0 2024-04-25T10:26:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05f2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec160930 con 0x7f05ec15fce0 2024-04-25T10:26:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05f2600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec0cd0e0 con 0x7f05ec0d0600 2024-04-25T10:26:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05ebe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05ec0d0600 0x7f05ec166db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05eaa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f05ec1600c0 0x7f05ec15ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48356/0 (socket says 172.21.15.87:48356) 2024-04-25T10:26:20.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05eaa00640 1 -- 172.21.15.87:0/3130254935 learned_addr learned my addr 172.21.15.87:0/3130254935 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:20.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05ebe00640 1 -- 172.21.15.87:0/3130254935 >> v1:172.21.15.169:6789/0 conn(0x7f05ec15fce0 legacy=0x7f05ec0b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:20.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05ebe00640 1 -- 172.21.15.87:0/3130254935 >> v1:172.21.15.87:6789/0 conn(0x7f05ec1600c0 legacy=0x7f05ec15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05ea000640 1 -- 172.21.15.87:0/3130254935 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560793496 0 0) 0x7f05ec160650 con 0x7f05ec1600c0 2024-04-25T10:26:20.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.858+0000 7f05ebe00640 1 -- 172.21.15.87:0/3130254935 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0cd3c0 con 0x7f05ec0d0600 2024-04-25T10:26:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05ebe00640 1 --2- 172.21.15.87:0/3130254935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec0d0600 0x7f05ec166db0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4785765df827a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05ea000640 1 -- 172.21.15.87:0/3130254935 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40b2070 con 0x7f05ec0d0600 2024-04-25T10:26:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05ea000640 1 -- 172.21.15.87:0/3130254935 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40a59e0 con 0x7f05ec0d0600 2024-04-25T10:26:20.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05ea000640 1 -- 172.21.15.87:0/3130254935 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40a5db0 con 0x7f05ec0d0600 2024-04-25T10:26:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05f2600640 1 -- 172.21.15.87:0/3130254935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec0d0600 msgr2=0x7f05ec166db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05f2600640 1 --2- 172.21.15.87:0/3130254935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec0d0600 0x7f05ec166db0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e401c900 comp rx=0 tx=0).stop 2024-04-25T10:26:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05f2600640 1 -- 172.21.15.87:0/3130254935 shutdown_connections 2024-04-25T10:26:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05f2600640 1 --2- 172.21.15.87:0/3130254935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec0d0600 0x7f05ec166db0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05f2600640 1 -- 172.21.15.87:0/3130254935 >> 172.21.15.87:0/3130254935 conn(0x7f05ec0b3430 msgr2=0x7f05ec0b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:20.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05f2600640 1 -- 172.21.15.87:0/3130254935 shutdown_connections 2024-04-25T10:26:20.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.862+0000 7f05f2600640 1 -- 172.21.15.87:0/3130254935 wait complete. 2024-04-25T10:26:20.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 Processor -- start 2024-04-25T10:26:20.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 -- start start 2024-04-25T10:26:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05ec0d0600 0x7f05ec152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05ec15fce0 0x7f05ec152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec1600c0 0x7f05ec0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec169c90 con 0x7f05ec1600c0 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec169b10 con 0x7f05ec15fce0 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec169e10 con 0x7f05ec0d0600 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec1600c0 0x7f05ec0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec1600c0 0x7f05ec0b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43040/0 (socket says 172.21.15.87:43040) 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 -- 172.21.15.87:0/3009932946 learned_addr learned my addr 172.21.15.87:0/3009932946 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05eaa00640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05ec15fce0 0x7f05ec152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05eb400640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05ec0d0600 0x7f05ec152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 -- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05ec0d0600 msgr2=0x7f05ec152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05ec0d0600 0x7f05ec152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 -- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05ec15fce0 msgr2=0x7f05ec152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05ec15fce0 0x7f05ec152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:20.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 -- 172.21.15.87:0/3009932946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0cb510 con 0x7f05ec1600c0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05ebe00640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec1600c0 0x7f05ec0b44f0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e40196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05e3400640 1 -- 172.21.15.87:0/3009932946 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40bd020 con 0x7f05ec1600c0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05e3400640 1 -- 172.21.15.87:0/3009932946 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40ad7c0 con 0x7f05ec1600c0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05e3400640 1 -- 172.21.15.87:0/3009932946 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40b2040 con 0x7f05ec1600c0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec0cb7f0 con 0x7f05ec1600c0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec155150 con 0x7f05ec1600c0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.866+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec155440 con 0x7f05ec1600c0 2024-04-25T10:26:20.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.870+0000 7f05e3400640 1 -- 172.21.15.87:0/3009932946 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05e40c0020 con 0x7f05ec1600c0 2024-04-25T10:26:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.870+0000 7f05e3400640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05cc066f60 0x7f05cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.870+0000 7f05e3400640 1 -- 172.21.15.87:0/3009932946 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f05e41202c0 con 0x7f05ec1600c0 2024-04-25T10:26:20.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.870+0000 7f05eb400640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05cc066f60 0x7f05cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:20.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.870+0000 7f05eb400640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05cc066f60 0x7f05cc069420 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007fd0 tx=0x7f05dc007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:20.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:20.874+0000 7f05e3400640 1 -- 172.21.15.87:0/3009932946 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05e40c7030 con 0x7f05ec1600c0 2024-04-25T10:26:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.062+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f05ec04ef30 con 0x7f05ec1600c0 2024-04-25T10:26:21.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.142+0000 7f05e3400640 1 -- 172.21.15.87:0/3009932946 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v139) v1 ==== 162+0+0 (secure 0 0 0) 0x7f05e4016520 con 0x7f05ec1600c0 2024-04-25T10:26:21.143 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05cc066f60 msgr2=0x7f05cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05cc066f60 0x7f05cc069420 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007fd0 tx=0x7f05dc007e40 comp rx=0 tx=0).stop 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec1600c0 msgr2=0x7f05ec0b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec1600c0 0x7f05ec0b44f0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e40196a0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 shutdown_connections 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f05cc066f60 0x7f05cc069420 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ec1600c0 0x7f05ec0b44f0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05ec15fce0 0x7f05ec152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 --2- 172.21.15.87:0/3009932946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05ec0d0600 0x7f05ec152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 >> 172.21.15.87:0/3009932946 conn(0x7f05ec0b3430 msgr2=0x7f05ec15e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:21.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 shutdown_connections 2024-04-25T10:26:21.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.146+0000 7f05f2600640 1 -- 172.21.15.87:0/3009932946 wait complete. 2024-04-25T10:26:21.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:21.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2fb600640 1 Processor -- start 2024-04-25T10:26:21.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2fb600640 1 -- start start 2024-04-25T10:26:21.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2fb600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2f4071710 0x7fd2f40cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2fb600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f406e350 con 0x7fd2f4071af0 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2fb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f40cf6c0 con 0x7fd2f415ac60 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2fb600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f415e790 con 0x7fd2f4071710 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2f8c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2f4071710 0x7fd2f40cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2ebe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd2f4071af0 0x7fd2f40cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48364/0 (socket says 172.21.15.87:48364) 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2ebe00640 1 -- 172.21.15.87:0/418026987 learned_addr learned my addr 172.21.15.87:0/418026987 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2f8c00640 1 -- 172.21.15.87:0/418026987 >> v1:172.21.15.169:6789/0 conn(0x7fd2f415ac60 legacy=0x7fd2f415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2f8c00640 1 -- 172.21.15.87:0/418026987 >> v1:172.21.15.87:6789/0 conn(0x7fd2f4071af0 legacy=0x7fd2f40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2f3e00640 1 -- 172.21.15.87:0/418026987 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192571742 0 0) 0x7fd2f406e350 con 0x7fd2f4071af0 2024-04-25T10:26:21.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.306+0000 7fd2f8c00640 1 -- 172.21.15.87:0/418026987 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f415ea70 con 0x7fd2f4071710 2024-04-25T10:26:21.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2f8c00640 1 --2- 172.21.15.87:0/418026987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40cf0e0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40097c0 tx=0x7fd2e401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a82e2f403fad027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:21.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2f3e00640 1 -- 172.21.15.87:0/418026987 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40b1070 con 0x7fd2f4071710 2024-04-25T10:26:21.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2f3e00640 1 -- 172.21.15.87:0/418026987 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e40ac640 con 0x7fd2f4071710 2024-04-25T10:26:21.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2f3e00640 1 -- 172.21.15.87:0/418026987 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40aca10 con 0x7fd2f4071710 2024-04-25T10:26:21.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2fb600640 1 -- 172.21.15.87:0/418026987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 msgr2=0x7fd2f40cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2fb600640 1 --2- 172.21.15.87:0/418026987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40cf0e0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40097c0 tx=0x7fd2e401c400 comp rx=0 tx=0).stop 2024-04-25T10:26:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2fb600640 1 -- 172.21.15.87:0/418026987 shutdown_connections 2024-04-25T10:26:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2fb600640 1 --2- 172.21.15.87:0/418026987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40cf0e0 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2fb600640 1 -- 172.21.15.87:0/418026987 >> 172.21.15.87:0/418026987 conn(0x7fd2f40cfb70 msgr2=0x7fd2f40cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2fb600640 1 -- 172.21.15.87:0/418026987 shutdown_connections 2024-04-25T10:26:21.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.310+0000 7fd2fb600640 1 -- 172.21.15.87:0/418026987 wait complete. 2024-04-25T10:26:21.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 Processor -- start 2024-04-25T10:26:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 -- start start 2024-04-25T10:26:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f4071af0 0x7fd2f40c3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f415ac60 0x7fd2f40c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4168ab0 con 0x7fd2f4071710 2024-04-25T10:26:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4168930 con 0x7fd2f415ac60 2024-04-25T10:26:21.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4168c30 con 0x7fd2f4071af0 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43056/0 (socket says 172.21.15.87:43056) 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 -- 172.21.15.87:0/3001980433 learned_addr learned my addr 172.21.15.87:0/3001980433 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2ebe00640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f4071af0 0x7fd2f40c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 -- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f4071af0 msgr2=0x7fd2f40c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f4071af0 0x7fd2f40c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 -- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f415ac60 msgr2=0x7fd2f40c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f9600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f415ac60 0x7fd2f40c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f415ac60 0x7fd2f40c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 -- 172.21.15.87:0/3001980433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40c3a50 con 0x7fd2f4071710 2024-04-25T10:26:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2ebe00640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f4071af0 0x7fd2f40c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f9600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f415ac60 0x7fd2f40c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f8c00640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40b4400 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40097c0 tx=0x7fd2e401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f1600640 1 -- 172.21.15.87:0/3001980433 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40be070 con 0x7fd2f4071710 2024-04-25T10:26:21.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f1600640 1 -- 172.21.15.87:0/3001980433 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e4004030 con 0x7fd2f4071710 2024-04-25T10:26:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2f1600640 1 -- 172.21.15.87:0/3001980433 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40b1040 con 0x7fd2f4071710 2024-04-25T10:26:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f40c3d30 con 0x7fd2f4071710 2024-04-25T10:26:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f40c42f0 con 0x7fd2f4071710 2024-04-25T10:26:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.314+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f404ef30 con 0x7fd2f4071710 2024-04-25T10:26:21.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.318+0000 7fd2f1600640 1 -- 172.21.15.87:0/3001980433 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2e4003620 con 0x7fd2f4071710 2024-04-25T10:26:21.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.318+0000 7fd2f1600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2d0066f90 0x7fd2d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:21.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.318+0000 7fd2f1600640 1 -- 172.21.15.87:0/3001980433 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd2e411ed80 con 0x7fd2f4071710 2024-04-25T10:26:21.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.318+0000 7fd2ebe00640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2d0066f90 0x7fd2d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:21.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.318+0000 7fd2ebe00640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2d0066f90 0x7fd2d0069450 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd2f415a2a0 tx=0x7fd2dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:21.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.322+0000 7fd2f1600640 1 -- 172.21.15.87:0/3001980433 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2e4016050 con 0x7fd2f4071710 2024-04-25T10:26:21.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.506+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2f4166700 con 0x7fd2f4071710 2024-04-25T10:26:21.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.506+0000 7fd2f1600640 1 -- 172.21.15.87:0/3001980433 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fd2e40efc90 con 0x7fd2f4071710 2024-04-25T10:26:21.508 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:21.508 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":140,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:21.150204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:replay","state_seq":1,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:26:21.508 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 140 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2d0066f90 msgr2=0x7fd2d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2d0066f90 0x7fd2d0069450 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd2f415a2a0 tx=0x7fd2dc009040 comp rx=0 tx=0).stop 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 msgr2=0x7fd2f40b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40b4400 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40097c0 tx=0x7fd2e401aca0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 shutdown_connections 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd2d0066f90 0x7fd2d0069450 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2f415ac60 0x7fd2f40c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2f4071af0 0x7fd2f40c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 --2- 172.21.15.87:0/3001980433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4071710 0x7fd2f40b4400 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:21.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 >> 172.21.15.87:0/3001980433 conn(0x7fd2f40cfb70 msgr2=0x7fd2f40b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:21.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 shutdown_connections 2024-04-25T10:26:21.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:21.510+0000 7fd2fb600640 1 -- 172.21.15.87:0/3001980433 wait complete. 2024-04-25T10:26:21.931 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:26:09.137144+0000', 'modified': '2024-04-25T10:26:21.150204+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 188, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11520': {'gid': 11520, 'name': 'a', 'rank': 0, 'incarnation': 140, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.87:6836/3730332371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 3730332371}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 3730332371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:26:21.932 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11520:up:replay 2024-04-25T10:26:22.933 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:23.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0be800640 1 Processor -- start 2024-04-25T10:26:23.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0be800640 1 -- start start 2024-04-25T10:26:23.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0be800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0b80cda10 0x7fc0b806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0be800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0b80cf930 0x7fc0b80cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0be800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b80b2770 con 0x7fc0b815bc40 2024-04-25T10:26:23.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0be800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b806c670 con 0x7fc0b80cda10 2024-04-25T10:26:23.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0be800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b806c950 con 0x7fc0b80cf930 2024-04-25T10:26:23.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0afe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0b80cda10 0x7fc0b806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0b7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc0b815bc40 0x7fc0b80b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48374/0 (socket says 172.21.15.87:48374) 2024-04-25T10:26:23.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0b7400640 1 -- 172.21.15.87:0/2456866146 learned_addr learned my addr 172.21.15.87:0/2456866146 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0b7e00640 1 --2- 172.21.15.87:0/2456866146 >> v2:172.21.15.169:3300/0 conn(0x7fc0b80cf930 0x7fc0b80cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0b6a00640 1 -- 172.21.15.87:0/2456866146 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599849222 0 0) 0x7fc0b80b2770 con 0x7fc0b815bc40 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0b6a00640 1 -- 172.21.15.87:0/2456866146 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc09c003530 con 0x7fc0b815bc40 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0afe00640 1 -- 172.21.15.87:0/2456866146 >> v2:172.21.15.169:3300/0 conn(0x7fc0b80cf930 msgr2=0x7fc0b80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0afe00640 1 --2- 172.21.15.87:0/2456866146 >> v2:172.21.15.169:3300/0 conn(0x7fc0b80cf930 0x7fc0b80cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0afe00640 1 -- 172.21.15.87:0/2456866146 >> v1:172.21.15.87:6789/0 conn(0x7fc0b815bc40 legacy=0x7fc0b80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0afe00640 1 -- 172.21.15.87:0/2456866146 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b806cc30 con 0x7fc0b80cda10 2024-04-25T10:26:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.078+0000 7fc0b7e00640 1 --2- 172.21.15.87:0/2456866146 >> v2:172.21.15.169:3300/0 conn(0x7fc0b80cf930 0x7fc0b80cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:23.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0afe00640 1 --2- 172.21.15.87:0/2456866146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b806c090 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80ae4c0 tx=0x7fc0a001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a50a5329ffcbac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0b6a00640 1 -- 172.21.15.87:0/2456866146 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00b0070 con 0x7fc0b80cda10 2024-04-25T10:26:23.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0b6a00640 1 -- 172.21.15.87:0/2456866146 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a00ab600 con 0x7fc0b80cda10 2024-04-25T10:26:23.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0b6a00640 1 -- 172.21.15.87:0/2456866146 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00ab9d0 con 0x7fc0b80cda10 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 -- 172.21.15.87:0/2456866146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 msgr2=0x7fc0b806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 --2- 172.21.15.87:0/2456866146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b806c090 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80ae4c0 tx=0x7fc0a001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 -- 172.21.15.87:0/2456866146 shutdown_connections 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 --2- 172.21.15.87:0/2456866146 >> v2:172.21.15.169:3300/0 conn(0x7fc0b80cf930 0x7fc0b80cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 --2- 172.21.15.87:0/2456866146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b806c090 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 -- 172.21.15.87:0/2456866146 >> 172.21.15.87:0/2456866146 conn(0x7fc0b8066b70 msgr2=0x7fc0b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 -- 172.21.15.87:0/2456866146 shutdown_connections 2024-04-25T10:26:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 -- 172.21.15.87:0/2456866146 wait complete. 2024-04-25T10:26:23.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.082+0000 7fc0be800640 1 Processor -- start 2024-04-25T10:26:23.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 -- start start 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b81f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0b80cf930 0x7fc0b80ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0b815bc40 0x7fc0b80ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80b3850 con 0x7fc0b80cda10 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80b36d0 con 0x7fc0b80cf930 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80b39d0 con 0x7fc0b815bc40 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b81f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b81f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43082/0 (socket says 172.21.15.87:43082) 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 -- 172.21.15.87:0/3326800688 learned_addr learned my addr 172.21.15.87:0/3326800688 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0afe00640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0b80cf930 0x7fc0b80ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7e00640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0b815bc40 0x7fc0b80ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 -- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0b815bc40 msgr2=0x7fc0b80ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0b815bc40 0x7fc0b80ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 -- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0b80cf930 msgr2=0x7fc0b80ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0b80cf930 0x7fc0b80ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 -- 172.21.15.87:0/3326800688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b81f8ce0 con 0x7fc0b80cda10 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0b7400640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b81f7e60 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c3c0 tx=0x7fc0a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0af400640 1 -- 172.21.15.87:0/3326800688 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8017070 con 0x7fc0b80cda10 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0af400640 1 -- 172.21.15.87:0/3326800688 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8005cf0 con 0x7fc0b80cda10 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0af400640 1 -- 172.21.15.87:0/3326800688 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80072f0 con 0x7fc0b80cda10 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b81f8440 con 0x7fc0b80cda10 2024-04-25T10:26:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b81f8a00 con 0x7fc0b80cda10 2024-04-25T10:26:23.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.086+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b804ef30 con 0x7fc0b80cda10 2024-04-25T10:26:23.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.090+0000 7fc0af400640 1 -- 172.21.15.87:0/3326800688 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0a801a070 con 0x7fc0b80cda10 2024-04-25T10:26:23.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.090+0000 7fc0af400640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc094066f60 0x7fc094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.090+0000 7fc0af400640 1 -- 172.21.15.87:0/3326800688 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc0a8084530 con 0x7fc0b80cda10 2024-04-25T10:26:23.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.090+0000 7fc0afe00640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc094066f60 0x7fc094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.090+0000 7fc0afe00640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc094066f60 0x7fc094069420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00ad000 tx=0x7fc0a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.094+0000 7fc0af400640 1 -- 172.21.15.87:0/3326800688 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0a8055440 con 0x7fc0b80cda10 2024-04-25T10:26:23.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.282+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0b80af1c0 con 0x7fc0b80cda10 2024-04-25T10:26:23.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.282+0000 7fc0af400640 1 -- 172.21.15.87:0/3326800688 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fc0a8054d40 con 0x7fc0b80cda10 2024-04-25T10:26:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:23.285 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":142,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:22.164020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:26:23.285 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 142 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc094066f60 msgr2=0x7fc094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc094066f60 0x7fc094069420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00ad000 tx=0x7fc0a0008040 comp rx=0 tx=0).stop 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 msgr2=0x7fc0b81f7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b81f7e60 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c3c0 tx=0x7fc0a800c880 comp rx=0 tx=0).stop 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 shutdown_connections 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc094066f60 0x7fc094069420 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0b815bc40 0x7fc0b80ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0b80cf930 0x7fc0b80ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 --2- 172.21.15.87:0/3326800688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b80cda10 0x7fc0b81f7e60 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 >> 172.21.15.87:0/3326800688 conn(0x7fc0b8066b70 msgr2=0x7fc0b8066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 shutdown_connections 2024-04-25T10:26:23.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:23.286+0000 7fc0be800640 1 -- 172.21.15.87:0/3326800688 wait complete. 2024-04-25T10:26:23.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 142, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:26:09.137144+0000', 'modified': '2024-04-25T10:26:22.164020+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 188, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11520': {'gid': 11520, 'name': 'a', 'rank': 0, 'incarnation': 140, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.87:6836/3730332371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 3730332371}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 3730332371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:26:23.709 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11520:up:rejoin 2024-04-25T10:26:24.566 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:26:24.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:26:24.710 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:24.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c76600640 1 Processor -- start 2024-04-25T10:26:24.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c76600640 1 -- start start 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c76600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c70071f70 0x7f7c700b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c76600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c700b2fc0 0x7f7c7006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c76600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c70070ed0 con 0x7f7c7015a0e0 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c76600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c700711b0 con 0x7f7c700b2fc0 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c76600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c70071490 con 0x7f7c70071f70 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c6fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c700b2fc0 0x7f7c7006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c6fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c700b2fc0 0x7f7c7006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43088/0 (socket says 172.21.15.87:43088) 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c6fe00640 1 -- 172.21.15.87:0/948197238 learned_addr learned my addr 172.21.15.87:0/948197238 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:24.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c6f400640 1 --1- 172.21.15.87:0/948197238 >> v1:172.21.15.87:6789/0 conn(0x7f7c7015a0e0 0x7f7c7015b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48382/0 (socket says 172.21.15.87:48382) 2024-04-25T10:26:24.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.858+0000 7f7c66a00640 1 --2- 172.21.15.87:0/948197238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c70071f70 0x7f7c700b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6fe00640 1 -- 172.21.15.87:0/948197238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c70071f70 msgr2=0x7f7c700b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6fe00640 1 --2- 172.21.15.87:0/948197238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c70071f70 0x7f7c700b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6fe00640 1 -- 172.21.15.87:0/948197238 >> v1:172.21.15.87:6789/0 conn(0x7f7c7015a0e0 legacy=0x7f7c7015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6fe00640 1 -- 172.21.15.87:0/948197238 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70071770 con 0x7f7c700b2fc0 2024-04-25T10:26:24.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c66a00640 1 --2- 172.21.15.87:0/948197238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c70071f70 0x7f7c700b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:24.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6fe00640 1 --2- 172.21.15.87:0/948197238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c700b2fc0 0x7f7c7006c090 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009e50 tx=0x7f7c6801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6e2df3375fb470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6ea00640 1 -- 172.21.15.87:0/948197238 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c680a5790 con 0x7f7c700b2fc0 2024-04-25T10:26:24.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6ea00640 1 -- 172.21.15.87:0/948197238 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c680a5930 con 0x7f7c700b2fc0 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c6ea00640 1 -- 172.21.15.87:0/948197238 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c680a5d00 con 0x7f7c700b2fc0 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 -- 172.21.15.87:0/948197238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c700b2fc0 msgr2=0x7f7c7006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 --2- 172.21.15.87:0/948197238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c700b2fc0 0x7f7c7006c090 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009e50 tx=0x7f7c6801ca30 comp rx=0 tx=0).stop 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 -- 172.21.15.87:0/948197238 shutdown_connections 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 --2- 172.21.15.87:0/948197238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c700b2fc0 0x7f7c7006c090 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 --2- 172.21.15.87:0/948197238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c70071f70 0x7f7c700b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 -- 172.21.15.87:0/948197238 >> 172.21.15.87:0/948197238 conn(0x7f7c70066b70 msgr2=0x7f7c70066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 -- 172.21.15.87:0/948197238 shutdown_connections 2024-04-25T10:26:24.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.862+0000 7f7c76600640 1 -- 172.21.15.87:0/948197238 wait complete. 2024-04-25T10:26:24.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 Processor -- start 2024-04-25T10:26:24.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 -- start start 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c70071f70 0x7f7c7015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b2fc0 0x7f7c70149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015a0e0 0x7f7c70155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c70072350 con 0x7f7c70071f70 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c70070b50 con 0x7f7c7015a0e0 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c700724d0 con 0x7f7c700b2fc0 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c70071f70 0x7f7c7015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c70071f70 0x7f7c7015d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43098/0 (socket says 172.21.15.87:43098) 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 -- 172.21.15.87:0/4176761415 learned_addr learned my addr 172.21.15.87:0/4176761415 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c66a00640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b2fc0 0x7f7c70149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6fe00640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015a0e0 0x7f7c70155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 -- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b2fc0 msgr2=0x7f7c70149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b2fc0 0x7f7c70149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 -- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015a0e0 msgr2=0x7f7c70155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015a0e0 0x7f7c70155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 -- 172.21.15.87:0/4176761415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7015d770 con 0x7f7c70071f70 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c6f400640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c70071f70 0x7f7c7015d230 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e730 tx=0x7f7c6000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c67e00640 1 -- 172.21.15.87:0/4176761415 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6001a070 con 0x7f7c70071f70 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c67e00640 1 -- 172.21.15.87:0/4176761415 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c60006c00 con 0x7f7c70071f70 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c70154a80 con 0x7f7c70071f70 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.866+0000 7f7c67e00640 1 -- 172.21.15.87:0/4176761415 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000c640 con 0x7f7c70071f70 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.870+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c70155040 con 0x7f7c70071f70 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.870+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7004ef30 con 0x7f7c70071f70 2024-04-25T10:26:24.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.870+0000 7f7c67e00640 1 -- 172.21.15.87:0/4176761415 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c6001d070 con 0x7f7c70071f70 2024-04-25T10:26:24.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.870+0000 7f7c67e00640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c4c066f60 0x7f7c4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.870+0000 7f7c67e00640 1 -- 172.21.15.87:0/4176761415 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7c60086320 con 0x7f7c70071f70 2024-04-25T10:26:24.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.870+0000 7f7c66a00640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c4c066f60 0x7f7c4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.874+0000 7f7c66a00640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c4c066f60 0x7f7c4c069420 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7c70158f20 tx=0x7f7c58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:24.874+0000 7f7c67e00640 1 -- 172.21.15.87:0/4176761415 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c7015d770 con 0x7f7c70071f70 2024-04-25T10:26:25.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c7014a7b0 con 0x7f7c70071f70 2024-04-25T10:26:25.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c67e00640 1 -- 172.21.15.87:0/4176761415 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f7c60056c70 con 0x7f7c70071f70 2024-04-25T10:26:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:26:25.066 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:23.173736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:26:25.066 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 143 2024-04-25T10:26:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c4c066f60 msgr2=0x7f7c4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c4c066f60 0x7f7c4c069420 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7c70158f20 tx=0x7f7c58008040 comp rx=0 tx=0).stop 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c70071f70 msgr2=0x7f7c7015d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c70071f70 0x7f7c7015d230 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e730 tx=0x7f7c6000ebf0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 shutdown_connections 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7c4c066f60 0x7f7c4c069420 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015a0e0 0x7f7c70155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b2fc0 0x7f7c70149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 --2- 172.21.15.87:0/4176761415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c70071f70 0x7f7c7015d230 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 >> 172.21.15.87:0/4176761415 conn(0x7f7c70066b70 msgr2=0x7f7c7006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 shutdown_connections 2024-04-25T10:26:25.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.066+0000 7f7c76600640 1 -- 172.21.15.87:0/4176761415 wait complete. 2024-04-25T10:26:25.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 143, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:26:09.137144+0000', 'modified': '2024-04-25T10:26:23.173736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 188, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11520': {'gid': 11520, 'name': 'a', 'rank': 0, 'incarnation': 140, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.87:6836/3730332371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 3730332371}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 3730332371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:26:25.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:26:25.490 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:26:25.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.626+0000 7f8683200640 1 Processor -- start 2024-04-25T10:26:25.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.626+0000 7f8683200640 1 -- start start 2024-04-25T10:26:25.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f8683200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f867c15a0e0 0x7f867c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f8683200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c15b6e0 con 0x7f867c071f70 2024-04-25T10:26:25.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f8683200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c070ed0 con 0x7f867c0b3190 2024-04-25T10:26:25.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f8683200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c0711b0 con 0x7f867c15a0e0 2024-04-25T10:26:25.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f867be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f867c15a0e0 0x7f867c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f8673400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f867c071f70 0x7f867c0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48386/0 (socket says 172.21.15.87:48386) 2024-04-25T10:26:25.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f8673400640 1 -- 172.21.15.87:0/1594011886 learned_addr learned my addr 172.21.15.87:0/1594011886 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:25.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f867be00640 1 -- 172.21.15.87:0/1594011886 >> v1:172.21.15.169:6789/0 conn(0x7f867c0b3190 legacy=0x7f867c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f867be00640 1 -- 172.21.15.87:0/1594011886 >> v1:172.21.15.87:6789/0 conn(0x7f867c071f70 legacy=0x7f867c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f867be00640 1 -- 172.21.15.87:0/1594011886 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c071490 con 0x7f867c15a0e0 2024-04-25T10:26:25.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f867b400640 1 -- 172.21.15.87:0/1594011886 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936819074 0 0) 0x7f8664001120 con 0x7f867c071f70 2024-04-25T10:26:25.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.630+0000 7f867be00640 1 --2- 172.21.15.87:0/1594011886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c15b1a0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f866c0097c0 tx=0x7f866c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29f86ffc434ca6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f867b400640 1 -- 172.21.15.87:0/1594011886 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0b2070 con 0x7f867c15a0e0 2024-04-25T10:26:25.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f867b400640 1 -- 172.21.15.87:0/1594011886 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c0ad680 con 0x7f867c15a0e0 2024-04-25T10:26:25.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f867b400640 1 -- 172.21.15.87:0/1594011886 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0ada50 con 0x7f867c15a0e0 2024-04-25T10:26:25.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- 172.21.15.87:0/1594011886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 msgr2=0x7f867c15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 --2- 172.21.15.87:0/1594011886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c15b1a0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f866c0097c0 tx=0x7f866c01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- 172.21.15.87:0/1594011886 shutdown_connections 2024-04-25T10:26:25.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 --2- 172.21.15.87:0/1594011886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c15b1a0 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- 172.21.15.87:0/1594011886 >> 172.21.15.87:0/1594011886 conn(0x7f867c066b70 msgr2=0x7f867c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- 172.21.15.87:0/1594011886 shutdown_connections 2024-04-25T10:26:25.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- 172.21.15.87:0/1594011886 wait complete. 2024-04-25T10:26:25.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 Processor -- start 2024-04-25T10:26:25.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- start start 2024-04-25T10:26:25.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f867c071f70 0x7f867c152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867c0b3190 0x7f867c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15a4c0 con 0x7f867c15a0e0 2024-04-25T10:26:25.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0af030 con 0x7f867c071f70 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8683200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c070b50 con 0x7f867c0b3190 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8681200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8681200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43114/0 (socket says 172.21.15.87:43114) 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8681200640 1 -- 172.21.15.87:0/380128137 learned_addr learned my addr 172.21.15.87:0/380128137 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f867be00640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f867c071f70 0x7f867c152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.634+0000 7f8673400640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867c0b3190 0x7f867c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8681200640 1 -- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867c0b3190 msgr2=0x7f867c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8681200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867c0b3190 0x7f867c155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8681200640 1 -- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f867c071f70 msgr2=0x7f867c152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8681200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f867c071f70 0x7f867c152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8681200640 1 -- 172.21.15.87:0/380128137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c14ba30 con 0x7f867c15a0e0 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8681200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c150290 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f867400c770 tx=0x7f867400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8678c00640 1 -- 172.21.15.87:0/380128137 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674003b80 con 0x7f867c15a0e0 2024-04-25T10:26:25.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8678c00640 1 -- 172.21.15.87:0/380128137 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674003d20 con 0x7f867c15a0e0 2024-04-25T10:26:25.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c155880 con 0x7f867c15a0e0 2024-04-25T10:26:25.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8678c00640 1 -- 172.21.15.87:0/380128137 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400aa90 con 0x7f867c15a0e0 2024-04-25T10:26:25.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c151890 con 0x7f867c15a0e0 2024-04-25T10:26:25.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.638+0000 7f8678c00640 1 -- 172.21.15.87:0/380128137 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f867400ac30 con 0x7f867c15a0e0 2024-04-25T10:26:25.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8678c00640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8658066f90 0x7f8658069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8678c00640 1 -- 172.21.15.87:0/380128137 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8674084f00 con 0x7f867c15a0e0 2024-04-25T10:26:25.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 Processor -- start 2024-04-25T10:26:25.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f867be00640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8658066f90 0x7f8658069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- start start 2024-04-25T10:26:25.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f867be00640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8658066f90 0x7f8658069450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f866c0af000 tx=0x7f866c0a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a0920 0x55b1ce6a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a1240 0x55b1ce6569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce657310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce53e5f0 con 0x55b1ce6a1240 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce4a5da0 con 0x55b1ce656f30 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce42e150 con 0x55b1ce6a0920 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a1240 0x55b1ce6569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a1240 0x55b1ce6569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43120/0 (socket says 172.21.15.87:43120) 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 -- 172.21.15.87:0/3000281667 learned_addr learned my addr 172.21.15.87:0/3000281667 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f867be00640 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a0920 0x55b1ce6a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8681200640 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce657310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 -- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a0920 msgr2=0x55b1ce6a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a0920 0x55b1ce6a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 -- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 msgr2=0x55b1ce657310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce657310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 -- 172.21.15.87:0/3000281667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ce447ca0 con 0x55b1ce6a1240 2024-04-25T10:26:25.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8681200640 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce657310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:25.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8673400640 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a1240 0x55b1ce6569f0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f8664005d00 tx=0x7f8664007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e41c8ad48b7281e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8670c00640 1 -- 172.21.15.87:0/3000281667 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664019070 con 0x55b1ce6a1240 2024-04-25T10:26:25.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8670c00640 1 -- 172.21.15.87:0/3000281667 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86640016d0 con 0x55b1ce6a1240 2024-04-25T10:26:25.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8670c00640 1 -- 172.21.15.87:0/3000281667 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664001aa0 con 0x55b1ce6a1240 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- 172.21.15.87:0/3000281667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a1240 msgr2=0x55b1ce6569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a1240 0x55b1ce6569f0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f8664005d00 tx=0x7f8664007420 comp rx=0 tx=0).stop 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- 172.21.15.87:0/3000281667 shutdown_connections 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce657310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a1240 0x55b1ce6569f0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 --2- 172.21.15.87:0/3000281667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a0920 0x55b1ce6a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- 172.21.15.87:0/3000281667 >> 172.21.15.87:0/3000281667 conn(0x55b1ce65d2a0 msgr2=0x55b1ce6552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- 172.21.15.87:0/3000281667 shutdown_connections 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.642+0000 7f8686706000 1 -- 172.21.15.87:0/3000281667 wait complete. 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 -- start start 2024-04-25T10:26:25.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 Processor -- start 2024-04-25T10:26:25.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce656f30 0x55b1ce6a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6a8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce6a1240 0x55b1ce65aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce460990 con 0x55b1ce6a0920 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce4a1e40 con 0x55b1ce6a1240 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce431900 con 0x55b1ce656f30 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 10 client.0 _ll_register_callbacks cb 0x55b1ce5591c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6a8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43130/0 (socket says 172.21.15.87:43130) 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 -- 172.21.15.87:0/2439014134 learned_addr learned my addr 172.21.15.87:0/2439014134 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f867be00640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce656f30 0x55b1ce6a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8681200640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce6a1240 0x55b1ce65aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 -- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce656f30 msgr2=0x55b1ce6a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce656f30 0x55b1ce6a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 -- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce6a1240 msgr2=0x55b1ce65aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce6a1240 0x55b1ce65aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 -- 172.21.15.87:0/2439014134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1ce45e630 con 0x55b1ce6a0920 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f867be00640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce656f30 0x55b1ce6a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8681200640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce6a1240 0x55b1ce65aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8673400640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6a8230 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f8664007460 tx=0x7f8664007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923b183d027e9a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8669600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8669600640 1 -- 172.21.15.87:0/2439014134 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664019030 con 0x55b1ce6a0920 2024-04-25T10:26:25.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8669600640 1 -- 172.21.15.87:0/2439014134 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f866401a070 con 0x55b1ce6a0920 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86640016d0 con 0x55b1ce6a0920 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8669600640 1 -- 172.21.15.87:0/2439014134 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f866402c050 con 0x55b1ce6a0920 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 10 client.11595 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55b1ce42e3a0 con 0x55b1ce6a0920 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8669600640 1 -- 172.21.15.87:0/2439014134 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86640016d0 con 0x55b1ce6a0920 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8669600640 1 -- 172.21.15.87:0/2439014134 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f866401f070 con 0x55b1ce6a0920 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 10 client.11595 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 10 client.11595 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 20 client.11595 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce65c160 0x55b1ce6b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 4 client.11595 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1ce441a50 con 0x55b1ce65c160 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.646+0000 7f867be00640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce65c160 0x55b1ce6b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f867be00640 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce65c160 0x55b1ce6b1d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ac9d8be4874606 server_cookie=23022829392d3e0f in_seq=0 out_seq=0 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8669600640 10 client.11595 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8669600640 1 -- 172.21.15.87:0/2439014134 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1ce441a50 con 0x55b1ce65c160 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8669600640 10 client.11595 handle_command_reply: tid=0 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 client.11595 shutdown 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 msgr2=0x55b1ce6a8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6a8230 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f8664007460 tx=0x7f8664007b50 comp rx=0 tx=0).stop 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 shutdown_connections 2024-04-25T10:26:25.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce65c160 0x55b1ce6b1d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce6a1240 0x55b1ce65aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6a8230 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8669600640 0 client.11595 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 --2- 172.21.15.87:0/2439014134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce656f30 0x55b1ce6a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 >> 172.21.15.87:0/2439014134 conn(0x55b1ce65d2a0 msgr2=0x55b1ce6559e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 shutdown_connections 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 1 -- 172.21.15.87:0/2439014134 wait complete. 2024-04-25T10:26:25.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.650+0000 7f8686706000 20 client.11595 trim_cache size 0 max 16384 2024-04-25T10:26:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 Processor -- start 2024-04-25T10:26:25.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- start start 2024-04-25T10:26:25.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce6fc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c14a780 con 0x55b1ce6a0920 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c149b80 con 0x55b1ce656f30 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c14a180 con 0x55b1ce6a1240 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6fc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43144/0 (socket says 172.21.15.87:43144) 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 -- 172.21.15.87:0/531827341 learned_addr learned my addr 172.21.15.87:0/531827341 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8681200640 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f867be00640 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce6fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 -- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 msgr2=0x55b1ce6fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 -- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 msgr2=0x55b1ce6fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce6fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 -- 172.21.15.87:0/531827341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ce6a1070 con 0x55b1ce6a0920 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8673400640 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6fc950 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f86640014c0 tx=0x7f866401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5059a16617813280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f866be00640 1 -- 172.21.15.87:0/531827341 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664019040 con 0x55b1ce6a0920 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f866be00640 1 -- 172.21.15.87:0/531827341 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8664014500 con 0x55b1ce6a0920 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f866be00640 1 -- 172.21.15.87:0/531827341 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664001570 con 0x55b1ce6a0920 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- 172.21.15.87:0/531827341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 msgr2=0x55b1ce6fc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6fc950 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f86640014c0 tx=0x7f866401f000 comp rx=0 tx=0).stop 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- 172.21.15.87:0/531827341 shutdown_connections 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce6fc950 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 --2- 172.21.15.87:0/531827341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce6fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- 172.21.15.87:0/531827341 >> 172.21.15.87:0/531827341 conn(0x55b1ce65d2a0 msgr2=0x55b1ce65a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- 172.21.15.87:0/531827341 shutdown_connections 2024-04-25T10:26:25.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.666+0000 7f8686706000 1 -- 172.21.15.87:0/531827341 wait complete. 2024-04-25T10:26:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 -- start start 2024-04-25T10:26:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 Processor -- start 2024-04-25T10:26:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce701660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce701ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce4aa3d0 con 0x55b1ce6a0920 2024-04-25T10:26:25.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce561d50 con 0x55b1ce656f30 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce440980 con 0x55b1ce6a1240 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 10 client.0 _ll_register_callbacks cb 0x7f8664007ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce701ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce701ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43154/0 (socket says 172.21.15.87:43154) 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 -- 172.21.15.87:0/4010313927 learned_addr learned my addr 172.21.15.87:0/4010313927 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f867be00640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce701660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8681200640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 -- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 msgr2=0x55b1ce6fdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 -- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 msgr2=0x55b1ce701660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce701660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 -- 172.21.15.87:0/4010313927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1ce527600 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f867be00640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce701660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8673400640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce701ba0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f8664000fa0 tx=0x7f8664005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53809215edecd89c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8669600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8669600640 1 -- 172.21.15.87:0/4010313927 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866403e060 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8669600640 1 -- 172.21.15.87:0/4010313927 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8664025040 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8664014500 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8669600640 1 -- 172.21.15.87:0/4010313927 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8664044050 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 10 client.11601 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55b1ce43a9b0 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8669600640 1 -- 172.21.15.87:0/4010313927 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8664014500 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8669600640 1 -- 172.21.15.87:0/4010313927 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8664047020 con 0x55b1ce6a0920 2024-04-25T10:26:25.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 10 client.11601 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 10 client.11601 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 20 client.11601 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce6fe160 0x55b1ce6fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 4 client.11601 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b1ce4b32e0 con 0x55b1ce6fe160 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.670+0000 7f867be00640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce6fe160 0x55b1ce6fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8669600640 10 client.11601 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f867be00640 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce6fe160 0x55b1ce6fe540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5464a769eaa4e8 server_cookie=f04a1e0de2d1a2a9 in_seq=0 out_seq=0 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8669600640 1 -- 172.21.15.87:0/4010313927 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55b1ce4b32e0 con 0x55b1ce6fe160 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8669600640 10 client.11601 handle_command_reply: tid=0 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 client.11601 shutdown 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 msgr2=0x55b1ce701ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce701ba0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f8664000fa0 tx=0x7f8664005880 comp rx=0 tx=0).stop 2024-04-25T10:26:25.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 shutdown_connections 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55b1ce6fe160 0x55b1ce6fe540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b1ce6a1240 0x55b1ce6fdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1ce6a0920 0x55b1ce701ba0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8669600640 0 client.11601 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 --2- 172.21.15.87:0/4010313927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b1ce656f30 0x55b1ce701660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 >> 172.21.15.87:0/4010313927 conn(0x55b1ce65d2a0 msgr2=0x55b1ce6a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 shutdown_connections 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 1 -- 172.21.15.87:0/4010313927 wait complete. 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8686706000 20 client.11601 trim_cache size 0 max 16384 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:25.679 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 11520, 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 4.5223392149999997, 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 143, 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 188, 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 188, 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 5.4663001519999996 2024-04-25T10:26:25.680 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8658066f90 msgr2=0x7f8658069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8658066f90 0x7f8658069450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f866c0af000 tx=0x7f866c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 msgr2=0x7f867c150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c150290 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f867400c770 tx=0x7f867400cc30 comp rx=0 tx=0).stop 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 shutdown_connections 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8658066f90 0x7f8658069450 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f867c15a0e0 0x7f867c150290 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f867c0b3190 0x7f867c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 --2- 172.21.15.87:0/380128137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f867c071f70 0x7f867c152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 >> 172.21.15.87:0/380128137 conn(0x7f867c066b70 msgr2=0x7f867c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 shutdown_connections 2024-04-25T10:26:25.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.674+0000 7f8683200640 1 -- 172.21.15.87:0/380128137 wait complete. 2024-04-25T10:26:25.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.693 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:26:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.834+0000 7f6d2aa00640 1 Processor -- start 2024-04-25T10:26:25.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.834+0000 7f6d2aa00640 1 -- start start 2024-04-25T10:26:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d2aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6d2415ee00 0x7f6d24168d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d2aa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d240729c0 con 0x7f6d2415deb0 2024-04-25T10:26:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d2aa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d24071920 con 0x7f6d2415e290 2024-04-25T10:26:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d2aa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d24072530 con 0x7f6d2415ee00 2024-04-25T10:26:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d23e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6d2415deb0 0x7f6d241623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:48402/0 (socket says 172.21.15.87:48402) 2024-04-25T10:26:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d23e00640 1 -- 172.21.15.87:0/1261412514 learned_addr learned my addr 172.21.15.87:0/1261412514 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d28a00640 1 --2- 172.21.15.87:0/1261412514 >> v2:172.21.15.87:3300/0 conn(0x7f6d2415ee00 0x7f6d24168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d23400640 1 -- 172.21.15.87:0/1261412514 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381825485 0 0) 0x7f6d14001120 con 0x7f6d2415deb0 2024-04-25T10:26:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d23400640 1 -- 172.21.15.87:0/1261412514 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d08003530 con 0x7f6d2415deb0 2024-04-25T10:26:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d28a00640 1 -- 172.21.15.87:0/1261412514 >> v1:172.21.15.169:6789/0 conn(0x7f6d2415e290 legacy=0x7f6d2415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d28a00640 1 -- 172.21.15.87:0/1261412514 >> v1:172.21.15.87:6789/0 conn(0x7f6d2415deb0 legacy=0x7f6d241623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d28a00640 1 -- 172.21.15.87:0/1261412514 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d24071c00 con 0x7f6d2415ee00 2024-04-25T10:26:25.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.838+0000 7f6d28a00640 1 --2- 172.21.15.87:0/1261412514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24168d40 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f6d240726b0 tx=0x7f6d1c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a312f305d5d8c48d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d23400640 1 -- 172.21.15.87:0/1261412514 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0b2070 con 0x7f6d2415ee00 2024-04-25T10:26:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d23400640 1 -- 172.21.15.87:0/1261412514 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0a5950 con 0x7f6d2415ee00 2024-04-25T10:26:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d23400640 1 -- 172.21.15.87:0/1261412514 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0a5d20 con 0x7f6d2415ee00 2024-04-25T10:26:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- 172.21.15.87:0/1261412514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 msgr2=0x7f6d24168d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/1261412514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24168d40 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f6d240726b0 tx=0x7f6d1c01c890 comp rx=0 tx=0).stop 2024-04-25T10:26:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- 172.21.15.87:0/1261412514 shutdown_connections 2024-04-25T10:26:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/1261412514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24168d40 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- 172.21.15.87:0/1261412514 >> 172.21.15.87:0/1261412514 conn(0x7f6d240669a0 msgr2=0x7f6d24066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- 172.21.15.87:0/1261412514 shutdown_connections 2024-04-25T10:26:25.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- 172.21.15.87:0/1261412514 wait complete. 2024-04-25T10:26:25.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 Processor -- start 2024-04-25T10:26:25.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- start start 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2415deb0 0x7f6d240cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2415e290 0x7f6d240cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240ae9d0 con 0x7f6d2415ee00 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240ae850 con 0x7f6d2415e290 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d2aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2416baa0 con 0x7f6d2415deb0 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d28a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d28a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43178/0 (socket says 172.21.15.87:43178) 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d28a00640 1 -- 172.21.15.87:0/3281752146 learned_addr learned my addr 172.21.15.87:0/3281752146 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.842+0000 7f6d23e00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2415deb0 0x7f6d240cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d1b400640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2415e290 0x7f6d240cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d28a00640 1 -- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2415deb0 msgr2=0x7f6d240cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d28a00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2415deb0 0x7f6d240cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d28a00640 1 -- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2415e290 msgr2=0x7f6d240cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d28a00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2415e290 0x7f6d240cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d28a00640 1 -- 172.21.15.87:0/3281752146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240cf550 con 0x7f6d2415ee00 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d28a00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24151410 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0b0040 tx=0x7f6d1c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d20c00640 1 -- 172.21.15.87:0/3281752146 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0b2050 con 0x7f6d2415ee00 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d20c00640 1 -- 172.21.15.87:0/3281752146 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0be070 con 0x7f6d2415ee00 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d24150090 con 0x7f6d2415ee00 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d20c00640 1 -- 172.21.15.87:0/3281752146 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0ada90 con 0x7f6d2415ee00 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d24150650 con 0x7f6d2415ee00 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d20c00640 1 -- 172.21.15.87:0/3281752146 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d1c0adc30 con 0x7f6d2415ee00 2024-04-25T10:26:25.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d20c00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d00066f60 0x7f6d00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d20c00640 1 -- 172.21.15.87:0/3281752146 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d1c120220 con 0x7f6d2415ee00 2024-04-25T10:26:25.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.846+0000 7f6d23e00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d00066f60 0x7f6d00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 Processor -- start 2024-04-25T10:26:25.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- start start 2024-04-25T10:26:25.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d23e00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d00066f60 0x7f6d00069420 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6d140012c0 tx=0x7f6d14009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd4502c90 0x55cfd4503070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd45035b0 0x55cfd44b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd439ff40 con 0x55cfd45035b0 2024-04-25T10:26:25.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4307070 con 0x55cfd4502c90 2024-04-25T10:26:25.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd428f1a0 con 0x55cfd44b9290 2024-04-25T10:26:25.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd45035b0 0x55cfd44b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd45035b0 0x55cfd44b8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43184/0 (socket says 172.21.15.87:43184) 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 -- 172.21.15.87:0/2998390230 learned_addr learned my addr 172.21.15.87:0/2998390230 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d23e00640 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd4502c90 0x55cfd4503070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d28a00640 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 -- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 msgr2=0x55cfd44b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44b9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 -- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd4502c90 msgr2=0x55cfd4503070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd4502c90 0x55cfd4503070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 -- 172.21.15.87:0/2998390230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd428d510 con 0x55cfd45035b0 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d23e00640 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd4502c90 0x55cfd4503070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d1b400640 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd45035b0 0x55cfd44b8d50 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c001530 tx=0x7f6d0c0019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed39f351ba46cf87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d18c00640 1 -- 172.21.15.87:0/2998390230 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c019070 con 0x55cfd45035b0 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d18c00640 1 -- 172.21.15.87:0/2998390230 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c0052e0 con 0x55cfd45035b0 2024-04-25T10:26:25.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d18c00640 1 -- 172.21.15.87:0/2998390230 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c003840 con 0x55cfd45035b0 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- 172.21.15.87:0/2998390230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd45035b0 msgr2=0x55cfd44b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd45035b0 0x55cfd44b8d50 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c001530 tx=0x7f6d0c0019f0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- 172.21.15.87:0/2998390230 shutdown_connections 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd45035b0 0x55cfd44b8d50 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2998390230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd4502c90 0x55cfd4503070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- 172.21.15.87:0/2998390230 >> 172.21.15.87:0/2998390230 conn(0x55cfd44b5240 msgr2=0x55cfd44b7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- 172.21.15.87:0/2998390230 shutdown_connections 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- 172.21.15.87:0/2998390230 wait complete. 2024-04-25T10:26:25.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.850+0000 7f6d2df52000 1 -- start start 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 Processor -- start 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd44c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd42c16c0 con 0x55cfd4502c90 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4303110 con 0x55cfd45035b0 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd44b50c0 con 0x55cfd44b9290 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 10 client.0 _ll_register_callbacks cb 0x55cfd44a4570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43192/0 (socket says 172.21.15.87:43192) 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 -- 172.21.15.87:0/1157007967 learned_addr learned my addr 172.21.15.87:0/1157007967 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d23e00640 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d28a00640 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd44c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 -- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 msgr2=0x55cfd44c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 -- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 msgr2=0x55cfd44c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd44c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 -- 172.21.15.87:0/1157007967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd428f3b0 con 0x55cfd4502c90 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d1b400640 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44c4370 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c013000 tx=0x7f6d0c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7993615d90e216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d11600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d11600640 1 -- 172.21.15.87:0/1157007967 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c019030 con 0x55cfd4502c90 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d11600640 1 -- 172.21.15.87:0/1157007967 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d0c003bc0 con 0x55cfd4502c90 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d0c0052e0 con 0x55cfd4502c90 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d11600640 1 -- 172.21.15.87:0/1157007967 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d0c02c050 con 0x55cfd4502c90 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 10 client.11616 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55cfd4388e00 con 0x55cfd4502c90 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d11600640 1 -- 172.21.15.87:0/1157007967 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c0052e0 con 0x55cfd4502c90 2024-04-25T10:26:25.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d11600640 1 -- 172.21.15.87:0/1157007967 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d0c02f020 con 0x55cfd4502c90 2024-04-25T10:26:25.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 10 client.11616 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:25.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 10 client.11616 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:25.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 20 client.11616 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:25.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd44c5150 0x55cfd4514410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 4 client.11616 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:25.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfd42a2d60 con 0x55cfd44c5150 2024-04-25T10:26:25.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d23e00640 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd44c5150 0x55cfd4514410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d23e00640 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd44c5150 0x55cfd4514410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969dc64dc617a296 server_cookie=30e36b3e819eb103 in_seq=0 out_seq=0 2024-04-25T10:26:25.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.854+0000 7f6d11600640 10 client.11616 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d11600640 1 -- 172.21.15.87:0/1157007967 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfd42a2d60 con 0x55cfd44c5150 2024-04-25T10:26:25.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d11600640 10 client.11616 handle_command_reply: tid=0 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 client.11616 shutdown 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 msgr2=0x55cfd44c4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44c4370 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c013000 tx=0x7f6d0c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 shutdown_connections 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd44c5150 0x55cfd4514410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd44c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44c4370 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d11600640 0 client.11616 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1157007967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd44b9290 0x55cfd44c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 >> 172.21.15.87:0/1157007967 conn(0x55cfd44b5240 msgr2=0x55cfd44b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 shutdown_connections 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 1 -- 172.21.15.87:0/1157007967 wait complete. 2024-04-25T10:26:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.858+0000 7f6d2df52000 20 client.11616 trim_cache size 0 max 16384 2024-04-25T10:26:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.870+0000 7f6d2df52000 1 Processor -- start 2024-04-25T10:26:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.870+0000 7f6d2df52000 1 -- start start 2024-04-25T10:26:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd44b9290 0x55cfd44b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd45035b0 0x55cfd4514410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24202d30 con 0x55cfd4502c90 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240ae490 con 0x55cfd44b9290 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240ad890 con 0x55cfd45035b0 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43200/0 (socket says 172.21.15.87:43200) 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 -- 172.21.15.87:0/1101851512 learned_addr learned my addr 172.21.15.87:0/1101851512 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d23e00640 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd44b9290 0x55cfd44b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 -- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd45035b0 msgr2=0x55cfd4514410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd45035b0 0x55cfd4514410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 -- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd44b9290 msgr2=0x55cfd44b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd44b9290 0x55cfd44b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 -- 172.21.15.87:0/1101851512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd435ec50 con 0x55cfd4502c90 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d23e00640 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd44b9290 0x55cfd44b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d1b400640 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44b8b80 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c000eb0 tx=0x7f6d0c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a25149fe8815f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d13e00640 1 -- 172.21.15.87:0/1101851512 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c019070 con 0x55cfd4502c90 2024-04-25T10:26:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d13e00640 1 -- 172.21.15.87:0/1101851512 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c01a040 con 0x55cfd4502c90 2024-04-25T10:26:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d13e00640 1 -- 172.21.15.87:0/1101851512 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0071f0 con 0x55cfd4502c90 2024-04-25T10:26:25.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- 172.21.15.87:0/1101851512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 msgr2=0x55cfd44b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44b8b80 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c000eb0 tx=0x7f6d0c01e000 comp rx=0 tx=0).stop 2024-04-25T10:26:25.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- 172.21.15.87:0/1101851512 shutdown_connections 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd45035b0 0x55cfd4514410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd4502c90 0x55cfd44b8b80 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 --2- 172.21.15.87:0/1101851512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd44b9290 0x55cfd44b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- 172.21.15.87:0/1101851512 >> 172.21.15.87:0/1101851512 conn(0x55cfd44b5240 msgr2=0x55cfd44bdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- 172.21.15.87:0/1101851512 shutdown_connections 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- 172.21.15.87:0/1101851512 wait complete. 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 -- start start 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.874+0000 7f6d2df52000 1 Processor -- start 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd44b9290 0x55cfd4563050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd4502c90 0x55cfd4563590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd451de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd42a9bd0 con 0x55cfd44b9290 2024-04-25T10:26:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240ade90 con 0x55cfd45035b0 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4303110 con 0x55cfd4502c90 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 10 client.0 _ll_register_callbacks cb 0x7f6d0c007ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd44b9290 0x55cfd4563050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd44b9290 0x55cfd4563050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43212/0 (socket says 172.21.15.87:43212) 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 -- 172.21.15.87:0/2533582523 learned_addr learned my addr 172.21.15.87:0/2533582523 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d1b400640 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd4502c90 0x55cfd4563590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d28a00640 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd451de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 -- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd4502c90 msgr2=0x55cfd4563590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd4502c90 0x55cfd4563590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 -- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 msgr2=0x55cfd451de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd451de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 -- 172.21.15.87:0/2533582523 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd42cf120 con 0x55cfd44b9290 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d23e00640 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd44b9290 0x55cfd4563050 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f6d14005bd0 tx=0x7f6d14000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaaaa59c281675f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 1 -- 172.21.15.87:0/2533582523 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400e7b0 con 0x55cfd44b9290 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 1 -- 172.21.15.87:0/2533582523 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d1401e040 con 0x55cfd44b9290 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d0c01a040 con 0x55cfd44b9290 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 1 -- 172.21.15.87:0/2533582523 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d14038050 con 0x55cfd44b9290 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 10 client.11622 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55cfd42ae170 con 0x55cfd44b9290 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 1 -- 172.21.15.87:0/2533582523 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c01a040 con 0x55cfd44b9290 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 1 -- 172.21.15.87:0/2533582523 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d1403b020 con 0x55cfd44b9290 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 10 client.11622 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 10 client.11622 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 20 client.11622 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd451ed00 0x55cfd451f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 4 client.11622 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55cfd43145b0 con 0x55cfd451ed00 2024-04-25T10:26:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d1b400640 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd451ed00 0x55cfd451f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:25.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d1b400640 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd451ed00 0x55cfd451f110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319c27493f41802 server_cookie=dc29ca6486a960df in_seq=0 out_seq=0 2024-04-25T10:26:25.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 10 client.11622 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 1 -- 172.21.15.87:0/2533582523 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cfd43145b0 con 0x55cfd451ed00 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 10 client.11622 handle_command_reply: tid=0 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 client.11622 shutdown 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd44b9290 msgr2=0x55cfd4563050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd44b9290 0x55cfd4563050 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f6d14005bd0 tx=0x7f6d14000ee0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 shutdown_connections 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55cfd451ed00 0x55cfd451f110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfd45035b0 0x55cfd451de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cfd4502c90 0x55cfd4563590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 --2- 172.21.15.87:0/2533582523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cfd44b9290 0x55cfd4563050 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d11600640 0 client.11622 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.878+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 >> 172.21.15.87:0/2533582523 conn(0x55cfd44b5240 msgr2=0x55cfd44be470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 shutdown_connections 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2df52000 1 -- 172.21.15.87:0/2533582523 wait complete. 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2df52000 20 client.11622 trim_cache size 0 max 16384 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "03c0fa9b-15ea-410c-ada5-40088610c1c8", 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:26:25.885 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d00066f60 msgr2=0x7f6d00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d00066f60 0x7f6d00069420 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6d140012c0 tx=0x7f6d14009040 comp rx=0 tx=0).stop 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 msgr2=0x7f6d24151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24151410 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0b0040 tx=0x7f6d1c019630 comp rx=0 tx=0).stop 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 shutdown_connections 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f6d00066f60 0x7f6d00069420 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2415ee00 0x7f6d24151410 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2415e290 0x7f6d240cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 --2- 172.21.15.87:0/3281752146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2415deb0 0x7f6d240cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 >> 172.21.15.87:0/3281752146 conn(0x7f6d240669a0 msgr2=0x7f6d241604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 shutdown_connections 2024-04-25T10:26:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:25.882+0000 7f6d2aa00640 1 -- 172.21.15.87:0/3281752146 wait complete. 2024-04-25T10:26:25.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.901 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:26:26.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1af6200640 1 Processor -- start 2024-04-25T10:26:26.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1af6200640 1 -- start start 2024-04-25T10:26:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1af6200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1af00aeef0 0x7f1af0163140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1af6200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1af00cf640 0x7f1af00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1af6200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af0071d90 con 0x7f1af00b43a0 2024-04-25T10:26:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1af6200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af00b2e80 con 0x7f1af00cf640 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1af6200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af00d0010 con 0x7f1af00aeef0 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aefe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1af00cf640 0x7f1af00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aefe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1af00cf640 0x7f1af00cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43216/0 (socket says 172.21.15.87:43216) 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aefe00640 1 -- 172.21.15.87:0/324787018 learned_addr learned my addr 172.21.15.87:0/324787018 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aef400640 1 --2- 172.21.15.87:0/324787018 >> v2:172.21.15.169:3300/0 conn(0x7f1af00aeef0 0x7f1af0163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aefe00640 1 -- 172.21.15.87:0/324787018 >> v2:172.21.15.169:3300/0 conn(0x7f1af00aeef0 msgr2=0x7f1af0163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aefe00640 1 --2- 172.21.15.87:0/324787018 >> v2:172.21.15.169:3300/0 conn(0x7f1af00aeef0 0x7f1af0163140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aefe00640 1 -- 172.21.15.87:0/324787018 >> v1:172.21.15.169:6789/0 conn(0x7f1af00b43a0 legacy=0x7f1af00b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.038+0000 7f1aefe00640 1 -- 172.21.15.87:0/324787018 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af006c170 con 0x7f1af00cf640 2024-04-25T10:26:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1aefe00640 1 --2- 172.21.15.87:0/324787018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00cf640 0x7f1af00cfad0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f1af006d700 tx=0x7f1ae801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cec52ffeaf547a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1aeea00640 1 -- 172.21.15.87:0/324787018 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80a57a0 con 0x7f1af00cf640 2024-04-25T10:26:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1aeea00640 1 -- 172.21.15.87:0/324787018 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae80a5940 con 0x7f1af00cf640 2024-04-25T10:26:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1aeea00640 1 -- 172.21.15.87:0/324787018 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80a5d10 con 0x7f1af00cf640 2024-04-25T10:26:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 -- 172.21.15.87:0/324787018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00cf640 msgr2=0x7f1af00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 --2- 172.21.15.87:0/324787018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00cf640 0x7f1af00cfad0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f1af006d700 tx=0x7f1ae801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 -- 172.21.15.87:0/324787018 shutdown_connections 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 --2- 172.21.15.87:0/324787018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00cf640 0x7f1af00cfad0 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 --2- 172.21.15.87:0/324787018 >> v2:172.21.15.169:3300/0 conn(0x7f1af00aeef0 0x7f1af0163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 -- 172.21.15.87:0/324787018 >> 172.21.15.87:0/324787018 conn(0x7f1af00669a0 msgr2=0x7f1af0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 -- 172.21.15.87:0/324787018 shutdown_connections 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 -- 172.21.15.87:0/324787018 wait complete. 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.042+0000 7f1af6200640 1 Processor -- start 2024-04-25T10:26:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 -- start start 2024-04-25T10:26:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00aeef0 0x7f1af0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00b43a0 0x7f1af0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf640 0x7f1af0161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af006ce60 con 0x7f1af00b43a0 2024-04-25T10:26:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00b47a0 con 0x7f1af00aeef0 2024-04-25T10:26:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af006cfe0 con 0x7f1af00cf640 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00b43a0 0x7f1af0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00b43a0 0x7f1af0154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43220/0 (socket says 172.21.15.87:43220) 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 -- 172.21.15.87:0/100017384 learned_addr learned my addr 172.21.15.87:0/100017384 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1aef400640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00aeef0 0x7f1af0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1aefe00640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf640 0x7f1af0161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 -- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf640 msgr2=0x7f1af0161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf640 0x7f1af0161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 -- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00aeef0 msgr2=0x7f1af0151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00aeef0 0x7f1af0151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 -- 172.21.15.87:0/100017384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af014eb40 con 0x7f1af00b43a0 2024-04-25T10:26:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00b43a0 0x7f1af0154090 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c3a0 tx=0x7f1ad800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7400640 1 -- 172.21.15.87:0/100017384 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80052d0 con 0x7f1af00b43a0 2024-04-25T10:26:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7400640 1 -- 172.21.15.87:0/100017384 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad8005470 con 0x7f1af00b43a0 2024-04-25T10:26:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1ae7400640 1 -- 172.21.15.87:0/100017384 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8007260 con 0x7f1af00b43a0 2024-04-25T10:26:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af0154800 con 0x7f1af00b43a0 2024-04-25T10:26:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.046+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af014ede0 con 0x7f1af00b43a0 2024-04-25T10:26:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1ae7400640 1 -- 172.21.15.87:0/100017384 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ad8007400 con 0x7f1af00b43a0 2024-04-25T10:26:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1ae7400640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1acc066f60 0x7f1acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1ae7400640 1 -- 172.21.15.87:0/100017384 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1ad8084510 con 0x7f1af00b43a0 2024-04-25T10:26:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1acc066f60 0x7f1acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 Processor -- start 2024-04-25T10:26:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 -- start start 2024-04-25T10:26:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1acc066f60 0x7f1acc069420 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009ac0 tx=0x7f1ae0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d8fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d8b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c14d700ff0 con 0x55c14d8fd9d0 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c14d689150 con 0x55c14d8fe2f0 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c14d6fd090 con 0x55c14d8b40a0 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d8fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d8fddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43232/0 (socket says 172.21.15.87:43232) 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 -- 172.21.15.87:0/1804958542 learned_addr learned my addr 172.21.15.87:0/1804958542 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aefe00640 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d8b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 -- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 msgr2=0x55c14d8b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d8b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 -- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 msgr2=0x55c14d8b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 -- 172.21.15.87:0/1804958542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c14d8970d0 con 0x55c14d8fd9d0 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.050+0000 7f1aef400640 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d8fddb0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0006aa0 tx=0x7f1ae0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1706386556a8971c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1adfe00640 1 -- 172.21.15.87:0/1804958542 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae001b070 con 0x55c14d8fd9d0 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1adfe00640 1 -- 172.21.15.87:0/1804958542 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0027070 con 0x55c14d8fd9d0 2024-04-25T10:26:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1adfe00640 1 -- 172.21.15.87:0/1804958542 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0003a30 con 0x55c14d8fd9d0 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- 172.21.15.87:0/1804958542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 msgr2=0x55c14d8fddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d8fddb0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0006aa0 tx=0x7f1ae0014000 comp rx=0 tx=0).stop 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- 172.21.15.87:0/1804958542 shutdown_connections 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d8b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 --2- 172.21.15.87:0/1804958542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d8fddb0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- 172.21.15.87:0/1804958542 >> 172.21.15.87:0/1804958542 conn(0x55c14d8b0020 msgr2=0x55c14d8b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:26.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- 172.21.15.87:0/1804958542 shutdown_connections 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- 172.21.15.87:0/1804958542 wait complete. 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- start start 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 Processor -- start 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d8e27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d8e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8e3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c14d6c5250 con 0x55c14d8b40a0 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c14d6bb690 con 0x55c14d8fe2f0 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c14d689a40 con 0x55c14d8fd9d0 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1af965a000 10 client.0 _ll_register_callbacks cb 0x55c14d89ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d8e27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d8e27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43236/0 (socket says 172.21.15.87:43236) 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 -- 172.21.15.87:0/1001876505 learned_addr learned my addr 172.21.15.87:0/1001876505 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:26.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d8e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aefe00640 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 -- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 msgr2=0x55c14d8e2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d8e2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 -- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 msgr2=0x55c14d8e3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8e3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 -- 172.21.15.87:0/1001876505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c14d7ae1c0 con 0x55c14d8b40a0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1aef400640 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d8e27f0 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0024040 tx=0x7f1ae0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485183a91df0daa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1adcc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1adcc00640 1 -- 172.21.15.87:0/1001876505 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae001b040 con 0x55c14d8b40a0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.054+0000 7f1adcc00640 1 -- 172.21.15.87:0/1001876505 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1ae0036b40 con 0x55c14d8b40a0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ae0027070 con 0x55c14d8b40a0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1adcc00640 1 -- 172.21.15.87:0/1001876505 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ae0036d40 con 0x55c14d8b40a0 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 10 client.11634 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:26.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55c14d7aa170 con 0x55c14d8b40a0 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1adcc00640 1 -- 172.21.15.87:0/1001876505 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0027070 con 0x55c14d8b40a0 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1adcc00640 1 -- 172.21.15.87:0/1001876505 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1ae0040080 con 0x55c14d8b40a0 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 10 client.11634 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 10 client.11634 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 20 client.11634 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d90dc80 0x55c14d910070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 4 client.11634 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c14d69cc60 con 0x55c14d90dc80 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d90dc80 0x55c14d910070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1adcc00640 10 client.11634 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d90dc80 0x55c14d910070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e9904cb85c7f0f server_cookie=9b313a460d0f14f7 in_seq=0 out_seq=0 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1adcc00640 1 -- 172.21.15.87:0/1001876505 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c14d69cc60 con 0x55c14d90dc80 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1adcc00640 10 client.11634 handle_command_reply: tid=0 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 client.11634 shutdown 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 msgr2=0x55c14d8e27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d8e27f0 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0024040 tx=0x7f1ae0004740 comp rx=0 tx=0).stop 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 shutdown_connections 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d90dc80 0x55c14d910070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d8e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d8e2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1adcc00640 0 client.11634 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 --2- 172.21.15.87:0/1001876505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d8e27f0 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 >> 172.21.15.87:0/1001876505 conn(0x55c14d8b0020 msgr2=0x55c14d8b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:26.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 shutdown_connections 2024-04-25T10:26:26.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 1 -- 172.21.15.87:0/1001876505 wait complete. 2024-04-25T10:26:26.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.058+0000 7f1af965a000 20 client.11634 trim_cache size 0 max 16384 2024-04-25T10:26:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 Processor -- start 2024-04-25T10:26:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- start start 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d95a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d95a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d95ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af02071f0 con 0x55c14d8b40a0 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af02063e0 con 0x55c14d8fe2f0 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af014ff40 con 0x55c14d8fd9d0 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d95a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d95a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43238/0 (socket says 172.21.15.87:43238) 2024-04-25T10:26:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 -- 172.21.15.87:0/1153820696 learned_addr learned my addr 172.21.15.87:0/1153820696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d95a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aefe00640 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d95ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 -- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 msgr2=0x55c14d95a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d95a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 -- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 msgr2=0x55c14d95ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d95ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 -- 172.21.15.87:0/1153820696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c14d747de0 con 0x55c14d8b40a0 2024-04-25T10:26:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1aef400640 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d95a300 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000b010 tx=0x7f1ae002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a62fd318831163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:26.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1adf400640 1 -- 172.21.15.87:0/1153820696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0043030 con 0x55c14d8b40a0 2024-04-25T10:26:26.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1adf400640 1 -- 172.21.15.87:0/1153820696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0027070 con 0x55c14d8b40a0 2024-04-25T10:26:26.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1adf400640 1 -- 172.21.15.87:0/1153820696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae001b040 con 0x55c14d8b40a0 2024-04-25T10:26:26.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- 172.21.15.87:0/1153820696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 msgr2=0x55c14d95a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d95a300 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000b010 tx=0x7f1ae002e070 comp rx=0 tx=0).stop 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- 172.21.15.87:0/1153820696 shutdown_connections 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d95ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8fd9d0 0x55c14d95a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 --2- 172.21.15.87:0/1153820696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8b40a0 0x55c14d95a300 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- 172.21.15.87:0/1153820696 >> 172.21.15.87:0/1153820696 conn(0x55c14d8b0020 msgr2=0x55c14d90afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- 172.21.15.87:0/1153820696 shutdown_connections 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.074+0000 7f1af965a000 1 -- 172.21.15.87:0/1153820696 wait complete. 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 -- start start 2024-04-25T10:26:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 Processor -- start 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d910770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d910cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d9111f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c14d69be00 con 0x55c14d8fd9d0 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af0150540 con 0x55c14d8fe2f0 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c14d700300 con 0x55c14d8b40a0 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 10 client.0 _ll_register_callbacks cb 0x55c14d87f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d910cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d910cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43250/0 (socket says 172.21.15.87:43250) 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 -- 172.21.15.87:0/1051325371 learned_addr learned my addr 172.21.15.87:0/1051325371 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:26.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1aef400640 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d910770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1aefe00640 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d9111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 -- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 msgr2=0x55c14d910770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d910770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 -- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 msgr2=0x55c14d9111f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d9111f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 -- 172.21.15.87:0/1051325371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c14d6a28e0 con 0x55c14d8fd9d0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1ae7e00640 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d910cb0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800cd60 tx=0x7f1ad800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6fb2424a8512d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 1 -- 172.21.15.87:0/1051325371 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80530d0 con 0x55c14d8fd9d0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 1 -- 172.21.15.87:0/1051325371 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1ad8051440 con 0x55c14d8fd9d0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ae0027070 con 0x55c14d8fd9d0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 1 -- 172.21.15.87:0/1051325371 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ad80840b0 con 0x55c14d8fd9d0 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 10 client.11640 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:26.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55c14d75a8f0 con 0x55c14d8fd9d0 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 1 -- 172.21.15.87:0/1051325371 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0027070 con 0x55c14d8fd9d0 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 1 -- 172.21.15.87:0/1051325371 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1ad8068ce0 con 0x55c14d8fd9d0 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 10 client.11640 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 10 client.11640 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 20 client.11640 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d963f50 0x55c14d966340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 4 client.11640 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c14d70e530 con 0x55c14d963f50 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1aef400640 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d963f50 0x55c14d966340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1aef400640 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d963f50 0x55c14d966340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f304abb5a360c0 server_cookie=cfb1e7f1932105d7 in_seq=0 out_seq=0 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 10 client.11640 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 1 -- 172.21.15.87:0/1051325371 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55c14d70e530 con 0x55c14d963f50 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.078+0000 7f1adcc00640 10 client.11640 handle_command_reply: tid=0 2024-04-25T10:26:26.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 client.11640 shutdown 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 msgr2=0x55c14d910cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d910cb0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800cd60 tx=0x7f1ad800a000 comp rx=0 tx=0).stop 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 shutdown_connections 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x55c14d963f50 0x55c14d966340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c14d8fe2f0 0x55c14d9111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c14d8fd9d0 0x55c14d910cb0 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1adcc00640 0 client.11640 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 --2- 172.21.15.87:0/1051325371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c14d8b40a0 0x55c14d910770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 >> 172.21.15.87:0/1051325371 conn(0x55c14d8b0020 msgr2=0x55c14d8fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 shutdown_connections 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 1 -- 172.21.15.87:0/1051325371 wait complete. 2024-04-25T10:26:26.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af965a000 20 client.11640 trim_cache size 0 max 16384 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: "03c0fa9b-15ea-410c-ada5-40088610c1c8": { 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/", 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "03c0fa9b-15ea-410c-ada5-40088610c1c8", 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive,repair" 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:26.086 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:26.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1acc066f60 msgr2=0x7f1acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1acc066f60 0x7f1acc069420 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009ac0 tx=0x7f1ae0009190 comp rx=0 tx=0).stop 2024-04-25T10:26:26.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00b43a0 msgr2=0x7f1af0154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:26.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00b43a0 0x7f1af0154090 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c3a0 tx=0x7f1ad800c860 comp rx=0 tx=0).stop 2024-04-25T10:26:26.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 shutdown_connections 2024-04-25T10:26:26.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1acc066f60 0x7f1acc069420 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf640 0x7f1af0161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af00b43a0 0x7f1af0154090 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 --2- 172.21.15.87:0/100017384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00aeef0 0x7f1af0151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:26.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 >> 172.21.15.87:0/100017384 conn(0x7f1af00669a0 msgr2=0x7f1af0066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:26.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 shutdown_connections 2024-04-25T10:26:26.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:26.082+0000 7f1af6200640 1 -- 172.21.15.87:0/100017384 wait complete. 2024-04-25T10:26:26.100 INFO:tasks.cephfs.filesystem:scrub status for tag:03c0fa9b-15ea-410c-ada5-40088610c1c8 - {'path': '/', 'tag': '03c0fa9b-15ea-410c-ada5-40088610c1c8', 'options': 'recursive,repair'} 2024-04-25T10:26:26.154 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-04-25T10:26:26.154+0000 7fb93c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:26:54.631 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:26:54.635 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:26:56.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:56.102 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:26:56.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.250+0000 7f3cef400640 1 Processor -- start 2024-04-25T10:26:56.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.250+0000 7f3cef400640 1 -- start start 2024-04-25T10:26:56.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cef400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ce80aef80 0x7f3ce815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cef400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3ce8070350 0x7f3ce815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cef400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce815b080 con 0x7f3ce815bac0 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cef400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce815adb0 con 0x7f3ce8070350 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cef400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce815eb60 con 0x7f3ce80aef80 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cdfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3ce8070350 0x7f3ce815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cdfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3ce8070350 0x7f3ce815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37950/0 (socket says 172.21.15.87:37950) 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2893868924 learned_addr learned my addr 172.21.15.87:0/2893868924 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3ceca00640 1 --2- 172.21.15.87:0/2893868924 >> v2:172.21.15.169:3300/0 conn(0x7f3ce80aef80 0x7f3ce815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2893868924 >> v2:172.21.15.169:3300/0 conn(0x7f3ce80aef80 msgr2=0x7f3ce815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2893868924 >> v2:172.21.15.169:3300/0 conn(0x7f3ce80aef80 0x7f3ce815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2893868924 >> v1:172.21.15.169:6789/0 conn(0x7f3ce815bac0 legacy=0x7f3ce80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.254+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2893868924 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce815ee40 con 0x7f3ce8070350 2024-04-25T10:26:56.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2893868924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce815b580 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f3ce806e220 tx=0x7f3cd001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e8f4b1de8f94b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3ce7e00640 1 -- 172.21.15.87:0/2893868924 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00aa660 con 0x7f3ce8070350 2024-04-25T10:26:56.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3ce7e00640 1 -- 172.21.15.87:0/2893868924 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd00af070 con 0x7f3ce8070350 2024-04-25T10:26:56.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3ce7e00640 1 -- 172.21.15.87:0/2893868924 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00aaa20 con 0x7f3ce8070350 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 -- 172.21.15.87:0/2893868924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 msgr2=0x7f3ce815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 --2- 172.21.15.87:0/2893868924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce815b580 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f3ce806e220 tx=0x7f3cd001c520 comp rx=0 tx=0).stop 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 -- 172.21.15.87:0/2893868924 shutdown_connections 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 --2- 172.21.15.87:0/2893868924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce815b580 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 --2- 172.21.15.87:0/2893868924 >> v2:172.21.15.169:3300/0 conn(0x7f3ce80aef80 0x7f3ce815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 -- 172.21.15.87:0/2893868924 >> 172.21.15.87:0/2893868924 conn(0x7f3ce80669a0 msgr2=0x7f3ce8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 -- 172.21.15.87:0/2893868924 shutdown_connections 2024-04-25T10:26:56.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 -- 172.21.15.87:0/2893868924 wait complete. 2024-04-25T10:26:56.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 Processor -- start 2024-04-25T10:26:56.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.258+0000 7f3cef400640 1 -- start start 2024-04-25T10:26:56.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce80aef80 0x7f3ce814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce815bac0 0x7f3ce80ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81679a0 con 0x7f3ce8070350 2024-04-25T10:26:56.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce8167820 con 0x7f3ce815bac0 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce8167b20 con 0x7f3ce80aef80 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce80b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37952/0 (socket says 172.21.15.87:37952) 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 -- 172.21.15.87:0/328910537 learned_addr learned my addr 172.21.15.87:0/328910537 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ced400640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce815bac0 0x7f3ce80ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce80aef80 0x7f3ce814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 -- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce80aef80 msgr2=0x7f3ce814fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce80aef80 0x7f3ce814fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 -- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce815bac0 msgr2=0x7f3ce80ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce815bac0 0x7f3ce80ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 -- 172.21.15.87:0/328910537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80ab980 con 0x7f3ce8070350 2024-04-25T10:26:56.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ceca00640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce80b0fd0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800d800 tx=0x7f3cd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ce5600640 1 -- 172.21.15.87:0/328910537 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800b6c0 con 0x7f3ce8070350 2024-04-25T10:26:56.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ce5600640 1 -- 172.21.15.87:0/328910537 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8019070 con 0x7f3ce8070350 2024-04-25T10:26:56.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce80abc00 con 0x7f3ce8070350 2024-04-25T10:26:56.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3ce5600640 1 -- 172.21.15.87:0/328910537 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800bae0 con 0x7f3ce8070350 2024-04-25T10:26:56.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.262+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce80ac1c0 con 0x7f3ce8070350 2024-04-25T10:26:56.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3ce5600640 1 -- 172.21.15.87:0/328910537 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cd800bc80 con 0x7f3ce8070350 2024-04-25T10:26:56.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3ce5600640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3cc4066f60 0x7f3cc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3ce5600640 1 -- 172.21.15.87:0/328910537 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3cd80854b0 con 0x7f3ce8070350 2024-04-25T10:26:56.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3cc4066f60 0x7f3cc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 Processor -- start 2024-04-25T10:26:56.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 -- start start 2024-04-25T10:26:56.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3cc4066f60 0x7f3cc4069420 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00ae000 tx=0x7f3cd00a7000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138df9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138dafb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138db00a0 0x565138db0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x565138bfcff0 con 0x565138dfa2f0 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x565138b85150 con 0x565138db00a0 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x565138bf9090 con 0x565138df99d0 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138dafb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138dafb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37960/0 (socket says 172.21.15.87:37960) 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2718169004 learned_addr learned my addr 172.21.15.87:0/2718169004 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3ceca00640 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138df9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3ced400640 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138db00a0 0x565138db0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 msgr2=0x565138df9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138df9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138db00a0 msgr2=0x565138db0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138db00a0 0x565138db0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.266+0000 7f3cdfe00640 1 -- 172.21.15.87:0/2718169004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565138d930d0 con 0x565138dfa2f0 2024-04-25T10:26:56.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138dafb60 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0097fd0 tx=0x7f3cd0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459bcfd45a64dd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cdd600640 1 -- 172.21.15.87:0/2718169004 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00c8630 con 0x565138dfa2f0 2024-04-25T10:26:56.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cdd600640 1 -- 172.21.15.87:0/2718169004 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd00af040 con 0x565138dfa2f0 2024-04-25T10:26:56.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cdd600640 1 -- 172.21.15.87:0/2718169004 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00c89d0 con 0x565138dfa2f0 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- 172.21.15.87:0/2718169004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 msgr2=0x565138dafb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138dafb60 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0097fd0 tx=0x7f3cd0004740 comp rx=0 tx=0).stop 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- 172.21.15.87:0/2718169004 shutdown_connections 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138db00a0 0x565138db0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138dafb60 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2718169004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138df9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- 172.21.15.87:0/2718169004 >> 172.21.15.87:0/2718169004 conn(0x565138dac020 msgr2=0x565138dae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- 172.21.15.87:0/2718169004 shutdown_connections 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- 172.21.15.87:0/2718169004 wait complete. 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- start start 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 Processor -- start 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138dde7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138dded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138dfa2f0 0x565138ddf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x565138bc1250 con 0x565138db00a0 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x565138bb7690 con 0x565138df99d0 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x565138b85a40 con 0x565138dfa2f0 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cf2950000 10 client.0 _ll_register_callbacks cb 0x565138d9ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138dde7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138dde7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37972/0 (socket says 172.21.15.87:37972) 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 -- 172.21.15.87:0/2057906548 learned_addr learned my addr 172.21.15.87:0/2057906548 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138dded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ced400640 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138dfa2f0 0x565138ddf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 -- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138dfa2f0 msgr2=0x565138ddf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138dfa2f0 0x565138ddf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 -- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 msgr2=0x565138dded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138dded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.270+0000 7f3ceca00640 1 -- 172.21.15.87:0/2057906548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565138caa1c0 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3ceca00640 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138dde7f0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8007fd0 tx=0x7f3cd8059ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9375f1307a85ec52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 1 -- 172.21.15.87:0/2057906548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd806d070 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 1 -- 172.21.15.87:0/2057906548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3cd80850c0 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cd00af040 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 1 -- 172.21.15.87:0/2057906548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cd80852c0 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 10 client.11652 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x565138ca6170 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 1 -- 172.21.15.87:0/2057906548 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd00af040 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 1 -- 172.21.15.87:0/2057906548 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3cd8080070 con 0x565138db00a0 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 10 client.11652 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 10 client.11652 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:56.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 20 client.11652 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e09c80 0x565138e0c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 4 client.11652 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565138b98c60 con 0x565138e09c80 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e09c80 0x565138e0c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e09c80 0x565138e0c070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff44e8888ced2a95 server_cookie=33cc34bbf8042ac6 in_seq=0 out_seq=0 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 10 client.11652 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 1 -- 172.21.15.87:0/2057906548 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565138b98c60 con 0x565138e09c80 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 10 client.11652 handle_command_reply: tid=0 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 client.11652 shutdown 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 msgr2=0x565138dde7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138dde7f0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8007fd0 tx=0x7f3cd8059ed0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 shutdown_connections 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e09c80 0x565138e0c070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138dfa2f0 0x565138ddf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138dded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 --2- 172.21.15.87:0/2057906548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138dde7f0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cd6000640 0 client.11652 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 >> 172.21.15.87:0/2057906548 conn(0x565138dac020 msgr2=0x565138daeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 shutdown_connections 2024-04-25T10:26:56.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 1 -- 172.21.15.87:0/2057906548 wait complete. 2024-04-25T10:26:56.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.274+0000 7f3cf2950000 20 client.11652 trim_cache size 0 max 16384 2024-04-25T10:26:56.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 Processor -- start 2024-04-25T10:26:56.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 -- start start 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138e560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138e56630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138dfa2f0 0x565138e56b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce8207340 con 0x565138db00a0 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce8206520 con 0x565138dfa2f0 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81fb7c0 con 0x565138df99d0 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138e560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138e560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37988/0 (socket says 172.21.15.87:37988) 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 -- 172.21.15.87:0/1964365104 learned_addr learned my addr 172.21.15.87:0/1964365104 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138e56630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ced400640 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138dfa2f0 0x565138e56b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 -- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 msgr2=0x565138e56630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138e56630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 -- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138dfa2f0 msgr2=0x565138e56b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138dfa2f0 0x565138e56b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 -- 172.21.15.87:0/1964365104 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565138c43de0 con 0x565138db00a0 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3ceca00640 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138e560f0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f3cd805c010 tx=0x7f3cd8085050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141f402b4edf255e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cdcc00640 1 -- 172.21.15.87:0/1964365104 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd806d040 con 0x565138db00a0 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cdcc00640 1 -- 172.21.15.87:0/1964365104 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8055210 con 0x565138db00a0 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cdcc00640 1 -- 172.21.15.87:0/1964365104 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8055d70 con 0x565138db00a0 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 -- 172.21.15.87:0/1964365104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 msgr2=0x565138e560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138e560f0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f3cd805c010 tx=0x7f3cd8085050 comp rx=0 tx=0).stop 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.290+0000 7f3cf2950000 1 -- 172.21.15.87:0/1964365104 shutdown_connections 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138dfa2f0 0x565138e56b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138df99d0 0x565138e56630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 --2- 172.21.15.87:0/1964365104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138db00a0 0x565138e560f0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- 172.21.15.87:0/1964365104 >> 172.21.15.87:0/1964365104 conn(0x565138dac020 msgr2=0x565138e06fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- 172.21.15.87:0/1964365104 shutdown_connections 2024-04-25T10:26:56.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- 172.21.15.87:0/1964365104 wait complete. 2024-04-25T10:26:56.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- start start 2024-04-25T10:26:56.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 Processor -- start 2024-04-25T10:26:56.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138db00a0 0x565138e0c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138e0c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138e0ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x565138b97e00 con 0x565138dfa2f0 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81fbdc0 con 0x565138df99d0 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x565138bfc300 con 0x565138db00a0 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 10 client.0 _ll_register_callbacks cb 0x565138d7b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138e0ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138e0ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37990/0 (socket says 172.21.15.87:37990) 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 -- 172.21.15.87:0/10611605 learned_addr learned my addr 172.21.15.87:0/10611605 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ceca00640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138db00a0 0x565138e0c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138e0c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 -- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138db00a0 msgr2=0x565138e0c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138db00a0 0x565138e0c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 -- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 msgr2=0x565138e0c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138e0c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 -- 172.21.15.87:0/10611605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565138b9e8e0 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ceca00640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138db00a0 0x565138e0c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cdfe00640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138e0c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ced400640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138e0ce00 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00155b0 tx=0x7f3ce0015a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae30eb7762b8b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cd6000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cd6000640 1 -- 172.21.15.87:0/10611605 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce001b070 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cd6000640 1 -- 172.21.15.87:0/10611605 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3ce0003c70 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cd8055210 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cd6000640 1 -- 172.21.15.87:0/10611605 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce0026050 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 10 client.11658 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x565138c568f0 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cd6000640 1 -- 172.21.15.87:0/10611605 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8055210 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cd6000640 1 -- 172.21.15.87:0/10611605 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ce0029020 con 0x565138dfa2f0 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 10 client.11658 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:56.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 10 client.11658 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 20 client.11658 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e5ff50 0x565138e62340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 4 client.11658 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565138c0a530 con 0x565138e5ff50 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.294+0000 7f3ceca00640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e5ff50 0x565138e62340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3ceca00640 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e5ff50 0x565138e62340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc43453f57e557c0 server_cookie=2f2b7d6be3c138e7 in_seq=0 out_seq=0 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cd6000640 10 client.11658 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cd6000640 1 -- 172.21.15.87:0/10611605 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x565138c0a530 con 0x565138e5ff50 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cd6000640 10 client.11658 handle_command_reply: tid=0 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 client.11658 shutdown 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 msgr2=0x565138e0ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138e0ce00 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00155b0 tx=0x7f3ce0015a70 comp rx=0 tx=0).stop 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 shutdown_connections 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x565138e5ff50 0x565138e62340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565138dfa2f0 0x565138e0ce00 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565138df99d0 0x565138e0c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 --2- 172.21.15.87:0/10611605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565138db00a0 0x565138e0c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cd6000640 0 client.11658 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 >> 172.21.15.87:0/10611605 conn(0x565138dac020 msgr2=0x565138df80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 shutdown_connections 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 1 -- 172.21.15.87:0/10611605 wait complete. 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cf2950000 20 client.11658 trim_cache size 0 max 16384 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:56.302 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:26:56.303 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:26:56.303 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:56.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3cc4066f60 msgr2=0x7f3cc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3cc4066f60 0x7f3cc4069420 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00ae000 tx=0x7f3cd00a7000 comp rx=0 tx=0).stop 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 msgr2=0x7f3ce80b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce80b0fd0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800d800 tx=0x7f3cd800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 shutdown_connections 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3cc4066f60 0x7f3cc4069420 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce815bac0 0x7f3ce80ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce80aef80 0x7f3ce814fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 --2- 172.21.15.87:0/328910537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ce8070350 0x7f3ce80b0fd0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 >> 172.21.15.87:0/328910537 conn(0x7f3ce80669a0 msgr2=0x7f3ce80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 shutdown_connections 2024-04-25T10:26:56.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.298+0000 7f3cef400640 1 -- 172.21.15.87:0/328910537 wait complete. 2024-04-25T10:26:56.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:26:56.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:56.318 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:26:56.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.454+0000 7f2d9b800640 1 Processor -- start 2024-04-25T10:26:56.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.454+0000 7f2d9b800640 1 -- start start 2024-04-25T10:26:56.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d9b800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2d9415bd10 0x7f2d940b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d9b800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9406c0e0 con 0x7f2d9415c0f0 2024-04-25T10:26:56.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d9b800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9415c6a0 con 0x7f2d940cf9e0 2024-04-25T10:26:56.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d9b800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d940b2ef0 con 0x7f2d9415bd10 2024-04-25T10:26:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2d9415c0f0 0x7f2d940b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:45782/0 (socket says 172.21.15.87:45782) 2024-04-25T10:26:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 -- 172.21.15.87:0/4278893389 learned_addr learned my addr 172.21.15.87:0/4278893389 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:26:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014071875 0 0) 0x7f2d9406c0e0 con 0x7f2d9415c0f0 2024-04-25T10:26:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d98e00640 1 --2- 172.21.15.87:0/4278893389 >> v2:172.21.15.169:3300/0 conn(0x7f2d9415bd10 0x7f2d940b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d78003530 con 0x7f2d9415c0f0 2024-04-25T10:26:56.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1689383845 0 0) 0x7f2d78003530 con 0x7f2d9415c0f0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 >> v2:172.21.15.169:3300/0 conn(0x7f2d9415bd10 msgr2=0x7f2d940b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 --2- 172.21.15.87:0/4278893389 >> v2:172.21.15.169:3300/0 conn(0x7f2d9415bd10 0x7f2d940b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 >> v1:172.21.15.169:6789/0 conn(0x7f2d940cf9e0 legacy=0x7f2d9406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9415c980 con 0x7f2d9415c0f0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f2d7c001680 con 0x7f2d9415c0f0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 >> v1:172.21.15.87:6789/0 conn(0x7f2d9415c0f0 legacy=0x7f2d940b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780039c0 0x7f2d78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d78004810 0x7f2d78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780051d0 0x7f2d7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940b2ef0 con 0x7f2d78004810 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7800c020 con 0x7f2d780039c0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7800c220 con 0x7f2d780051d0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d78004810 0x7f2d78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d98e00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780039c0 0x7f2d78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d99800640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780051d0 0x7f2d7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 -- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780051d0 msgr2=0x7f2d7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780051d0 0x7f2d7800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 -- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780039c0 msgr2=0x7f2d78003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780039c0 0x7f2d78003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d8be00640 1 -- 172.21.15.87:0/4278893389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78003530 con 0x7f2d78004810 2024-04-25T10:26:56.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.458+0000 7f2d99800640 1 -- 172.21.15.87:0/4278893389 reap_dead start 2024-04-25T10:26:56.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d8be00640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d78004810 0x7f2d78004c90 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c510 tx=0x7f2d7c01f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678159301c090a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00c0e0 con 0x7f2d78004810 2024-04-25T10:26:56.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c005b00 con 0x7f2d78004810 2024-04-25T10:26:56.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d93e00640 1 -- 172.21.15.87:0/4278893389 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c007070 con 0x7f2d78004810 2024-04-25T10:26:56.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d9b800640 1 -- 172.21.15.87:0/4278893389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d78004810 msgr2=0x7f2d78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d9b800640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d78004810 0x7f2d78004c90 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c510 tx=0x7f2d7c01f5e0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d9b800640 1 -- 172.21.15.87:0/4278893389 shutdown_connections 2024-04-25T10:26:56.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d9b800640 1 --2- 172.21.15.87:0/4278893389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d78004810 0x7f2d78004c90 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d9b800640 1 -- 172.21.15.87:0/4278893389 >> 172.21.15.87:0/4278893389 conn(0x7f2d94066b70 msgr2=0x7f2d94066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d9b800640 1 -- 172.21.15.87:0/4278893389 shutdown_connections 2024-04-25T10:26:56.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.462+0000 7f2d9b800640 1 -- 172.21.15.87:0/4278893389 wait complete. 2024-04-25T10:26:56.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 Processor -- start 2024-04-25T10:26:56.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 -- start start 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d78004810 0x7f2d9414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d940ac060 0x7f2d940ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940ae110 0x7f2d940ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940cfe10 con 0x7f2d940ac060 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9415cf40 con 0x7f2d78004810 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940cff90 con 0x7f2d940ae110 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d940ac060 0x7f2d940ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d940ac060 0x7f2d940ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38000/0 (socket says 172.21.15.87:38000) 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 -- 172.21.15.87:0/3744905444 learned_addr learned my addr 172.21.15.87:0/3744905444 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d78004810 0x7f2d9414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d99800640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940ae110 0x7f2d940ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 -- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940ae110 msgr2=0x7f2d940ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940ae110 0x7f2d940ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 -- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d78004810 msgr2=0x7f2d9414c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d78004810 0x7f2d9414c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 -- 172.21.15.87:0/3744905444 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d940a9a20 con 0x7f2d940ac060 2024-04-25T10:26:56.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d940ac060 0x7f2d940ac4e0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c510 tx=0x7f2d78005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d91600640 1 -- 172.21.15.87:0/3744905444 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0047f0 con 0x7f2d940ac060 2024-04-25T10:26:56.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d91600640 1 -- 172.21.15.87:0/3744905444 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c004990 con 0x7f2d940ac060 2024-04-25T10:26:56.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d91600640 1 -- 172.21.15.87:0/3744905444 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c004d40 con 0x7f2d940ac060 2024-04-25T10:26:56.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d940a9d00 con 0x7f2d940ac060 2024-04-25T10:26:56.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.466+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d941597a0 con 0x7f2d940ac060 2024-04-25T10:26:56.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d91600640 1 -- 172.21.15.87:0/3744905444 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d7c007070 con 0x7f2d940ac060 2024-04-25T10:26:56.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d91600640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d70066f60 0x7f2d70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d91600640 1 -- 172.21.15.87:0/3744905444 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2d7c121e00 con 0x7f2d940ac060 2024-04-25T10:26:56.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d70066f60 0x7f2d70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 Processor -- start 2024-04-25T10:26:56.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 -- start start 2024-04-25T10:26:56.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d70066f60 0x7f2d70069420 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009c60 tx=0x7f2d84009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 0x557779607070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x5577795bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577794a3f40 con 0x5577796075b0 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777940b070 con 0x5577795bd290 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577793931a0 con 0x557779606c90 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795bcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38014/0 (socket says 172.21.15.87:38014) 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 -- 172.21.15.87:0/3206483207 learned_addr learned my addr 172.21.15.87:0/3206483207 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 -- 172.21.15.87:0/3206483207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 msgr2=0x557779607070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3206483207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 0x557779607070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 -- 172.21.15.87:0/3206483207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 msgr2=0x5577795bd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3206483207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x5577795bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 -- 172.21.15.87:0/3206483207 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557779391510 con 0x5577796075b0 2024-04-25T10:26:56.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3206483207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795bcd50 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0fa7f0 tx=0x7f2d7c0f20e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4bbbbbc2f848bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d89600640 1 -- 172.21.15.87:0/3206483207 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0c0070 con 0x5577796075b0 2024-04-25T10:26:56.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d89600640 1 -- 172.21.15.87:0/3206483207 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c0fad20 con 0x5577796075b0 2024-04-25T10:26:56.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.470+0000 7f2d89600640 1 -- 172.21.15.87:0/3206483207 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0f09c0 con 0x5577796075b0 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3206483207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 msgr2=0x5577795bcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3206483207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795bcd50 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0fa7f0 tx=0x7f2d7c0f20e0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3206483207 shutdown_connections 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3206483207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x5577795bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3206483207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795bcd50 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3206483207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 0x557779607070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3206483207 >> 172.21.15.87:0/3206483207 conn(0x5577795b9240 msgr2=0x5577795bb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3206483207 shutdown_connections 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3206483207 wait complete. 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- start start 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 Processor -- start 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779612240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 0x557779612780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577793c56c0 con 0x5577796075b0 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557779407110 con 0x5577795bd290 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577794a3f40 con 0x557779606c90 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 10 client.0 _ll_register_callbacks cb 0x5577795a8570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38030/0 (socket says 172.21.15.87:38030) 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 -- 172.21.15.87:0/3173179711 learned_addr learned my addr 172.21.15.87:0/3173179711 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 0x557779612780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779612240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 -- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 msgr2=0x557779612780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 0x557779612780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 -- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 msgr2=0x557779612240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779612240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 -- 172.21.15.87:0/3173179711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577793933b0 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d99800640 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795cbe50 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007e30 tx=0x7f2d8c011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6853421742e492e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d82000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d82000640 1 -- 172.21.15.87:0/3173179711 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c006e30 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d82000640 1 -- 172.21.15.87:0/3173179711 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2d8c010040 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.474+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d7c0fad20 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d82000640 1 -- 172.21.15.87:0/3173179711 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d8c003980 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 10 client.11670 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55777948ce00 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d82000640 1 -- 172.21.15.87:0/3173179711 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c0fad20 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d82000640 1 -- 172.21.15.87:0/3173179711 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d8c003b20 con 0x5577796075b0 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 10 client.11670 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 10 client.11670 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 20 client.11670 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5577795ccbf0 0x5577795cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 4 client.11670 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577793a6d60 con 0x5577795ccbf0 2024-04-25T10:26:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5577795ccbf0 0x5577795cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5577795ccbf0 0x5577795cd000 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2f8b08f9e73df2 server_cookie=e0b1db4ff6a6cc18 in_seq=0 out_seq=0 2024-04-25T10:26:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d82000640 10 client.11670 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d82000640 1 -- 172.21.15.87:0/3173179711 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577793a6d60 con 0x5577795ccbf0 2024-04-25T10:26:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d82000640 10 client.11670 handle_command_reply: tid=0 2024-04-25T10:26:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 client.11670 shutdown 2024-04-25T10:26:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 msgr2=0x5577795cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795cbe50 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007e30 tx=0x7f2d8c011910 comp rx=0 tx=0).stop 2024-04-25T10:26:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 shutdown_connections 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5577795ccbf0 0x5577795cd000 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577796075b0 0x5577795cbe50 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557779606c90 0x557779612780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3173179711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779612240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 >> 172.21.15.87:0/3173179711 conn(0x5577795b9240 msgr2=0x5577795bbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d82000640 0 client.11670 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 shutdown_connections 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3173179711 wait complete. 2024-04-25T10:26:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.478+0000 7f2d9ecb0000 20 client.11670 trim_cache size 0 max 16384 2024-04-25T10:26:56.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 Processor -- start 2024-04-25T10:26:56.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- start start 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779606160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557779606c90 0x5577796066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x55777961d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941fc3c0 con 0x557779606c90 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d94071b70 con 0x5577795bd290 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940ab420 con 0x5577796075b0 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557779606c90 0x5577796066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557779606c90 0x5577796066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38040/0 (socket says 172.21.15.87:38040) 2024-04-25T10:26:56.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 -- 172.21.15.87:0/3438634311 learned_addr learned my addr 172.21.15.87:0/3438634311 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779606160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d99800640 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x55777961d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 -- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 msgr2=0x55777961d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x55777961d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 -- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 msgr2=0x557779606160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779606160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 -- 172.21.15.87:0/3438634311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577796073e0 con 0x557779606c90 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d98e00640 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779606160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d99800640 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x55777961d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d8be00640 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557779606c90 0x5577796066a0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c026250 tx=0x7f2d7c0c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcecf6891ee33c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d88c00640 1 -- 172.21.15.87:0/3438634311 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0f09c0 con 0x557779606c90 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d88c00640 1 -- 172.21.15.87:0/3438634311 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c1202b0 con 0x557779606c90 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d88c00640 1 -- 172.21.15.87:0/3438634311 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c119b90 con 0x557779606c90 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3438634311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557779606c90 msgr2=0x5577796066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557779606c90 0x5577796066a0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c026250 tx=0x7f2d7c0c0000 comp rx=0 tx=0).stop 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3438634311 shutdown_connections 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x55777961d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557779606c90 0x5577796066a0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/3438634311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577795bd290 0x557779606160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3438634311 >> 172.21.15.87:0/3438634311 conn(0x5577795b9240 msgr2=0x557779605a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3438634311 shutdown_connections 2024-04-25T10:26:56.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.494+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/3438634311 wait complete. 2024-04-25T10:26:56.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 -- start start 2024-04-25T10:26:56.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 Processor -- start 2024-04-25T10:26:56.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577795bd290 0x5577796205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557779606c90 0x557779620b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x557779628900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777940b070 con 0x5577795bd290 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d94071570 con 0x557779606c90 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577795c6200 con 0x5577796075b0 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 10 client.0 _ll_register_callbacks cb 0x7f2d7c007a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577795bd290 0x5577796205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577795bd290 0x5577796205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38042/0 (socket says 172.21.15.87:38042) 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 -- 172.21.15.87:0/396884700 learned_addr learned my addr 172.21.15.87:0/396884700 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d8be00640 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557779606c90 0x557779620b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d99800640 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x557779628900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 -- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 msgr2=0x557779628900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x557779628900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 -- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557779606c90 msgr2=0x557779620b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557779606c90 0x557779620b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 -- 172.21.15.87:0/396884700 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557779462c50 con 0x5577795bd290 2024-04-25T10:26:56.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d98e00640 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577795bd290 0x5577796205e0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f2d8402ada0 tx=0x7f2d840046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c18bfeb895ec41c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d82000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d82000640 1 -- 172.21.15.87:0/396884700 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8401b070 con 0x5577795bd290 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d82000640 1 -- 172.21.15.87:0/396884700 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2d84003b20 con 0x5577795bd290 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d7c1202b0 con 0x5577795bd290 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d82000640 1 -- 172.21.15.87:0/396884700 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d84003d20 con 0x5577795bd290 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 10 client.11676 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x5577793d3120 con 0x5577795bd290 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d82000640 1 -- 172.21.15.87:0/396884700 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c1202b0 con 0x5577795bd290 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d82000640 1 -- 172.21.15.87:0/396884700 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d8402a220 con 0x5577795bd290 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 10 client.11676 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 10 client.11676 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 20 client.11676 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x557779671d60 0x557779674150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 4 client.11676 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:26:56.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x5577794185b0 con 0x557779671d60 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d8be00640 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x557779671d60 0x557779674150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d8be00640 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x557779671d60 0x557779674150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3edbb37818594e1c server_cookie=89aad93d388f5b63 in_seq=0 out_seq=0 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.498+0000 7f2d82000640 10 client.11676 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d82000640 1 -- 172.21.15.87:0/396884700 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5577794185b0 con 0x557779671d60 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d82000640 10 client.11676 handle_command_reply: tid=0 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 client.11676 shutdown 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577795bd290 msgr2=0x5577796205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577795bd290 0x5577796205e0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f2d8402ada0 tx=0x7f2d840046d0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 shutdown_connections 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x557779671d60 0x557779674150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577796075b0 0x557779628900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557779606c90 0x557779620b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 --2- 172.21.15.87:0/396884700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577795bd290 0x5577796205e0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 >> 172.21.15.87:0/396884700 conn(0x5577795b9240 msgr2=0x557779606160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d82000640 0 client.11676 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 shutdown_connections 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 1 -- 172.21.15.87:0/396884700 wait complete. 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9ecb0000 20 client.11676 trim_cache size 0 max 16384 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "12361a57-c910-44a9-96e0-42e343bf8266", 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-04-25T10:26:56.507 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:56.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d70066f60 msgr2=0x7f2d70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d70066f60 0x7f2d70069420 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009c60 tx=0x7f2d84009190 comp rx=0 tx=0).stop 2024-04-25T10:26:56.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d940ac060 msgr2=0x7f2d940ac4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d940ac060 0x7f2d940ac4e0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c510 tx=0x7f2d78005840 comp rx=0 tx=0).stop 2024-04-25T10:26:56.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 shutdown_connections 2024-04-25T10:26:56.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2d70066f60 0x7f2d70069420 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940ae110 0x7f2d940ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d940ac060 0x7f2d940ac4e0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 --2- 172.21.15.87:0/3744905444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d78004810 0x7f2d9414c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 >> 172.21.15.87:0/3744905444 conn(0x7f2d94066b70 msgr2=0x7f2d940cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 shutdown_connections 2024-04-25T10:26:56.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.502+0000 7f2d9b800640 1 -- 172.21.15.87:0/3744905444 wait complete. 2024-04-25T10:26:56.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:56.522 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:26:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.658+0000 7ff8fba00640 1 Processor -- start 2024-04-25T10:26:56.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.658+0000 7ff8fba00640 1 -- start start 2024-04-25T10:26:56.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.658+0000 7ff8fba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8f40aef80 0x7ff8f415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.658+0000 7ff8fba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b31c0 0x7ff8f40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.658+0000 7ff8fba00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f415b080 con 0x7ff8f4070350 2024-04-25T10:26:56.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.658+0000 7ff8fba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f415adb0 con 0x7ff8f40aef80 2024-04-25T10:26:56.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.658+0000 7ff8fba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f415eb60 con 0x7ff8f40b31c0 2024-04-25T10:26:56.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8f40aef80 0x7ff8f415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8f40aef80 0x7ff8f415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38056/0 (socket says 172.21.15.87:38056) 2024-04-25T10:26:56.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 -- 172.21.15.87:0/3260463604 learned_addr learned my addr 172.21.15.87:0/3260463604 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/3260463604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b31c0 0x7ff8f40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 -- 172.21.15.87:0/3260463604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b31c0 msgr2=0x7ff8f40b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 --2- 172.21.15.87:0/3260463604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b31c0 0x7ff8f40b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 -- 172.21.15.87:0/3260463604 >> v1:172.21.15.169:6789/0 conn(0x7ff8f4070350 legacy=0x7ff8f415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 -- 172.21.15.87:0/3260463604 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f415ee40 con 0x7ff8f40aef80 2024-04-25T10:26:56.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f9000640 1 --2- 172.21.15.87:0/3260463604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40aef80 0x7ff8f415e480 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009870 tx=0x7ff8e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8505f773015bb43c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f3e00640 1 -- 172.21.15.87:0/3260463604 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40a7070 con 0x7ff8f40aef80 2024-04-25T10:26:56.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f3e00640 1 -- 172.21.15.87:0/3260463604 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e40b0070 con 0x7ff8f40aef80 2024-04-25T10:26:56.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.662+0000 7ff8f3e00640 1 -- 172.21.15.87:0/3260463604 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40ab890 con 0x7ff8f40aef80 2024-04-25T10:26:56.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- 172.21.15.87:0/3260463604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40aef80 msgr2=0x7ff8f415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3260463604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40aef80 0x7ff8f415e480 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009870 tx=0x7ff8e401c560 comp rx=0 tx=0).stop 2024-04-25T10:26:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- 172.21.15.87:0/3260463604 shutdown_connections 2024-04-25T10:26:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3260463604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40b31c0 0x7ff8f40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3260463604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f40aef80 0x7ff8f415e480 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- 172.21.15.87:0/3260463604 >> 172.21.15.87:0/3260463604 conn(0x7ff8f40669a0 msgr2=0x7ff8f4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- 172.21.15.87:0/3260463604 shutdown_connections 2024-04-25T10:26:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- 172.21.15.87:0/3260463604 wait complete. 2024-04-25T10:26:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 Processor -- start 2024-04-25T10:26:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- start start 2024-04-25T10:26:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4070350 0x7ff8f414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40aef80 0x7ff8f40b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f40b31c0 0x7ff8f4155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41679a0 con 0x7ff8f4070350 2024-04-25T10:26:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4167820 con 0x7ff8f40b31c0 2024-04-25T10:26:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8fba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4167b20 con 0x7ff8f40aef80 2024-04-25T10:26:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4070350 0x7ff8f414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4070350 0x7ff8f414fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38064/0 (socket says 172.21.15.87:38064) 2024-04-25T10:26:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 -- 172.21.15.87:0/3685859270 learned_addr learned my addr 172.21.15.87:0/3685859270 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 -- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40aef80 msgr2=0x7ff8f40b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40aef80 0x7ff8f40b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 -- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f40b31c0 msgr2=0x7ff8f4155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f40b31c0 0x7ff8f4155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f40b31c0 0x7ff8f4155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 -- 172.21.15.87:0/3685859270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4155c80 con 0x7ff8f4070350 2024-04-25T10:26:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f40b31c0 0x7ff8f4155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f9000640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4070350 0x7ff8f414fd90 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40ad040 tx=0x7ff8e401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f1600640 1 -- 172.21.15.87:0/3685859270 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40be5f0 con 0x7ff8f4070350 2024-04-25T10:26:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.666+0000 7ff8f1600640 1 -- 172.21.15.87:0/3685859270 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e40be7b0 con 0x7ff8f4070350 2024-04-25T10:26:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8f1600640 1 -- 172.21.15.87:0/3685859270 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40beba0 con 0x7ff8f4070350 2024-04-25T10:26:56.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f4155f60 con 0x7ff8f4070350 2024-04-25T10:26:56.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f4156520 con 0x7ff8f4070350 2024-04-25T10:26:56.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8f1600640 1 -- 172.21.15.87:0/3685859270 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8e40b43f0 con 0x7ff8f4070350 2024-04-25T10:26:56.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8f1600640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d0066f60 0x7ff8d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8f1600640 1 -- 172.21.15.87:0/3685859270 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff8e411f610 con 0x7ff8f4070350 2024-04-25T10:26:56.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d0066f60 0x7ff8d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.670+0000 7ff8feee7000 1 Processor -- start 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- start start 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d0066f60 0x7ff8d0069420 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc007fd0 tx=0x7ff8dc007980 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8719d0 0x556eef871db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef827b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8280a0 0x556eef828480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eef674ff0 con 0x556eef8280a0 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eef5fd150 con 0x556eef8722f0 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x556eef671090 con 0x556eef8719d0 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8280a0 0x556eef828480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8280a0 0x556eef828480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38072/0 (socket says 172.21.15.87:38072) 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 -- 172.21.15.87:0/2038868862 learned_addr learned my addr 172.21.15.87:0/2038868862 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9000640 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8719d0 0x556eef871db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef827b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 -- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8719d0 msgr2=0x556eef871db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8719d0 0x556eef871db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 -- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 msgr2=0x556eef827b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef827b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 -- 172.21.15.87:0/2038868862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eef80b0d0 con 0x556eef8280a0 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef827b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8280a0 0x556eef828480 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0074f0 tx=0x7ff8ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cecf1ce3c52155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8e9600640 1 -- 172.21.15.87:0/2038868862 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0147e0 con 0x556eef8280a0 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8e9600640 1 -- 172.21.15.87:0/2038868862 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec01a070 con 0x556eef8280a0 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8e9600640 1 -- 172.21.15.87:0/2038868862 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec014b80 con 0x556eef8280a0 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- 172.21.15.87:0/2038868862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8280a0 msgr2=0x556eef828480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8280a0 0x556eef828480 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0074f0 tx=0x7ff8ec009750 comp rx=0 tx=0).stop 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- 172.21.15.87:0/2038868862 shutdown_connections 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8280a0 0x556eef828480 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef827b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 --2- 172.21.15.87:0/2038868862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8719d0 0x556eef871db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- 172.21.15.87:0/2038868862 >> 172.21.15.87:0/2038868862 conn(0x556eef824020 msgr2=0x556eef826440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- 172.21.15.87:0/2038868862 shutdown_connections 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- 172.21.15.87:0/2038868862 wait complete. 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 -- start start 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.674+0000 7ff8feee7000 1 Processor -- start 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8719d0 0x556eef856d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8722f0 0x556eef857270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eef639250 con 0x556eef8722f0 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eef62f690 con 0x556eef8719d0 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x556eef5fda40 con 0x556eef8280a0 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 10 client.0 _ll_register_callbacks cb 0x556eef812e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8722f0 0x556eef857270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8722f0 0x556eef857270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38088/0 (socket says 172.21.15.87:38088) 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 -- 172.21.15.87:0/157350851 learned_addr learned my addr 172.21.15.87:0/157350851 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8719d0 0x556eef856d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9000640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 -- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 msgr2=0x556eef8567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 -- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8719d0 msgr2=0x556eef856d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8719d0 0x556eef856d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 -- 172.21.15.87:0/157350851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556eef7221c0 con 0x556eef8722f0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9000640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8722f0 0x556eef857270 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec000f90 tx=0x7ff8ec004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624ebe18360ae805 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 1 -- 172.21.15.87:0/157350851 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec02a680 con 0x556eef8722f0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 1 -- 172.21.15.87:0/157350851 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff8ec02a820 con 0x556eef8722f0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8ec01a070 con 0x556eef8722f0 2024-04-25T10:26:56.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 1 -- 172.21.15.87:0/157350851 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8ec032050 con 0x556eef8722f0 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 10 client.11688 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x556eef71e170 con 0x556eef8722f0 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 1 -- 172.21.15.87:0/157350851 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec01a070 con 0x556eef8722f0 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 1 -- 172.21.15.87:0/157350851 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff8ec035020 con 0x556eef8722f0 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 10 client.11688 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 10 client.11688 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 20 client.11688 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef881c80 0x556eef884070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 4 client.11688 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556eef610c60 con 0x556eef881c80 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9000640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef881c80 0x556eef884070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8f9000640 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef881c80 0x556eef884070 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103fd4585c1cfd67 server_cookie=20eaacc61a3fe84c in_seq=0 out_seq=0 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 10 client.11688 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 1 -- 172.21.15.87:0/157350851 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556eef610c60 con 0x556eef881c80 2024-04-25T10:26:56.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 10 client.11688 handle_command_reply: tid=0 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 client.11688 shutdown 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8722f0 msgr2=0x556eef857270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8722f0 0x556eef857270 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec000f90 tx=0x7ff8ec004ac0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 shutdown_connections 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef881c80 0x556eef884070 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8722f0 0x556eef857270 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8719d0 0x556eef856d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 --2- 172.21.15.87:0/157350851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8e2000640 0 client.11688 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.678+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 >> 172.21.15.87:0/157350851 conn(0x556eef824020 msgr2=0x556eef826ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.682+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 shutdown_connections 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.682+0000 7ff8feee7000 1 -- 172.21.15.87:0/157350851 wait complete. 2024-04-25T10:26:56.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.682+0000 7ff8feee7000 20 client.11688 trim_cache size 0 max 16384 2024-04-25T10:26:56.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 Processor -- start 2024-04-25T10:26:56.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 -- start start 2024-04-25T10:26:56.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8280a0 0x556eef870de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef827420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8722f0 0x556eef827960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4203250 con 0x556eef8719d0 2024-04-25T10:26:56.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f42023f0 con 0x556eef8280a0 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.694+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41512e0 con 0x556eef8722f0 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef827420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef827420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38100/0 (socket says 172.21.15.87:38100) 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 -- 172.21.15.87:0/570693308 learned_addr learned my addr 172.21.15.87:0/570693308 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8f9000640 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8280a0 0x556eef870de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8722f0 0x556eef827960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 -- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8722f0 msgr2=0x556eef827960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8722f0 0x556eef827960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 -- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8280a0 msgr2=0x556eef870de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8280a0 0x556eef870de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 -- 172.21.15.87:0/570693308 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eef6bbde0 con 0x556eef8719d0 2024-04-25T10:26:56.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef827420 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc003600 tx=0x7ff8dc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58626ce1ea2c8096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8e8c00640 1 -- 172.21.15.87:0/570693308 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc027070 con 0x556eef8719d0 2024-04-25T10:26:56.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8e8c00640 1 -- 172.21.15.87:0/570693308 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc018a40 con 0x556eef8719d0 2024-04-25T10:26:56.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8e8c00640 1 -- 172.21.15.87:0/570693308 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc018df0 con 0x556eef8719d0 2024-04-25T10:26:56.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- 172.21.15.87:0/570693308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 msgr2=0x556eef827420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef827420 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc003600 tx=0x7ff8dc00f000 comp rx=0 tx=0).stop 2024-04-25T10:26:56.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- 172.21.15.87:0/570693308 shutdown_connections 2024-04-25T10:26:56.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8722f0 0x556eef827960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef827420 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 --2- 172.21.15.87:0/570693308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8280a0 0x556eef870de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- 172.21.15.87:0/570693308 >> 172.21.15.87:0/570693308 conn(0x556eef824020 msgr2=0x556eef8700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- 172.21.15.87:0/570693308 shutdown_connections 2024-04-25T10:26:56.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- 172.21.15.87:0/570693308 wait complete. 2024-04-25T10:26:56.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- start start 2024-04-25T10:26:56.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 Processor -- start 2024-04-25T10:26:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef8d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef884380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eef60fe00 con 0x556eef8719d0 2024-04-25T10:26:56.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41518e0 con 0x556eef8722f0 2024-04-25T10:26:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x556eef674300 con 0x556eef8280a0 2024-04-25T10:26:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8feee7000 10 client.0 _ll_register_callbacks cb 0x556eef7f3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef8d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef8d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38114/0 (socket says 172.21.15.87:38114) 2024-04-25T10:26:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8ebe00640 1 -- 172.21.15.87:0/188683766 learned_addr learned my addr 172.21.15.87:0/188683766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:26:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8f9000640 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.698+0000 7ff8f9a00640 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef884380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8ebe00640 1 -- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 msgr2=0x556eef8d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8ebe00640 1 -- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 msgr2=0x556eef884380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef884380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8ebe00640 1 -- 172.21.15.87:0/188683766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556eef6168e0 con 0x556eef8719d0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8ebe00640 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef8d2a60 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc000f70 tx=0x7ff8dc001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e517d9d96d7a16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 1 -- 172.21.15.87:0/188683766 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc027070 con 0x556eef8719d0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 1 -- 172.21.15.87:0/188683766 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff8dc022610 con 0x556eef8719d0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8dc018a40 con 0x556eef8719d0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 1 -- 172.21.15.87:0/188683766 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8dc022850 con 0x556eef8719d0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 10 client.11694 fetch_fsmap learned FSMap version 143 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x556eef6ce8f0 con 0x556eef8719d0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 1 -- 172.21.15.87:0/188683766 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc03a070 con 0x556eef8719d0 2024-04-25T10:26:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 1 -- 172.21.15.87:0/188683766 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff8dc0229f0 con 0x556eef8719d0 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 10 client.11694 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 10 client.11694 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 20 client.11694 populate_metadata read hostname 'smithi087' 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef884fc0 0x556eef8853d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 4 client.11694 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556eef682530 con 0x556eef884fc0 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8f9000640 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef884fc0 0x556eef8853d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 10 client.11694 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8f9000640 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef884fc0 0x556eef8853d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a35ac224a9a287b server_cookie=e22663a30418aea in_seq=0 out_seq=0 2024-04-25T10:26:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 1 -- 172.21.15.87:0/188683766 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x556eef682530 con 0x556eef884fc0 2024-04-25T10:26:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 10 client.11694 handle_command_reply: tid=0 2024-04-25T10:26:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 client.11694 shutdown 2024-04-25T10:26:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 msgr2=0x556eef8d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef8d2a60 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc000f70 tx=0x7ff8dc001130 comp rx=0 tx=0).stop 2024-04-25T10:26:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 shutdown_connections 2024-04-25T10:26:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x556eef884fc0 0x556eef8853d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556eef8722f0 0x556eef884380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556eef8719d0 0x556eef8d2a60 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 --2- 172.21.15.87:0/188683766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556eef8280a0 0x556eef8d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8e2000640 0 client.11694 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 >> 172.21.15.87:0/188683766 conn(0x556eef824020 msgr2=0x556eef870930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 shutdown_connections 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 1 -- 172.21.15.87:0/188683766 wait complete. 2024-04-25T10:26:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8feee7000 20 client.11694 trim_cache size 0 max 16384 2024-04-25T10:26:56.717 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:26:56.717 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:26:56.717 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-04-25T10:26:56.717 INFO:teuthology.orchestra.run.smithi087.stdout: "12361a57-c910-44a9-96e0-42e343bf8266": { 2024-04-25T10:26:56.718 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/", 2024-04-25T10:26:56.718 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "12361a57-c910-44a9-96e0-42e343bf8266", 2024-04-25T10:26:56.718 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive,repair" 2024-04-25T10:26:56.718 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:56.718 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T10:26:56.718 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:26:56.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d0066f60 msgr2=0x7ff8d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d0066f60 0x7ff8d0069420 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc007fd0 tx=0x7ff8dc007980 comp rx=0 tx=0).stop 2024-04-25T10:26:56.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4070350 msgr2=0x7ff8f414fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4070350 0x7ff8f414fd90 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40ad040 tx=0x7ff8e401ada0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 shutdown_connections 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff8d0066f60 0x7ff8d0069420 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8f40b31c0 0x7ff8f4155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8f40aef80 0x7ff8f40b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 --2- 172.21.15.87:0/3685859270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8f4070350 0x7ff8f414fd90 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 >> 172.21.15.87:0/3685859270 conn(0x7ff8f40669a0 msgr2=0x7ff8f406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 shutdown_connections 2024-04-25T10:26:56.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:26:56.702+0000 7ff8fba00640 1 -- 172.21.15.87:0/3685859270 wait complete. 2024-04-25T10:26:56.723 INFO:tasks.cephfs.filesystem:scrub status for tag:12361a57-c910-44a9-96e0-42e343bf8266 - {'path': '/', 'tag': '12361a57-c910-44a9-96e0-42e343bf8266', 'options': 'recursive,repair'} 2024-04-25T10:27:24.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:27:24.692 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:27:26.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:26.725 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:27:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.873+0000 7fb7a8e00640 1 Processor -- start 2024-04-25T10:27:26.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a8e00640 1 -- start start 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a4159540 0x7fb7a40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a8e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7a40aebd0 0x7fb7a4071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a8e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a4071360 con 0x7fb7a4072020 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a8e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a40af030 con 0x7fb7a40aebd0 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4159920 con 0x7fb7a4159540 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7a40aebd0 0x7fb7a4071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7a40aebd0 0x7fb7a4071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48132/0 (socket says 172.21.15.87:48132) 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a2800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb7a4072020 0x7fb7a415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:54940/0 (socket says 172.21.15.87:54940) 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1400640 1 -- 172.21.15.87:0/4176035667 learned_addr learned my addr 172.21.15.87:0/4176035667 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/4176035667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a4159540 0x7fb7a40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1400640 1 -- 172.21.15.87:0/4176035667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a4159540 msgr2=0x7fb7a40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1400640 1 --2- 172.21.15.87:0/4176035667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a4159540 0x7fb7a40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1400640 1 -- 172.21.15.87:0/4176035667 >> v1:172.21.15.87:6789/0 conn(0x7fb7a4072020 legacy=0x7fb7a415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a1400640 1 -- 172.21.15.87:0/4176035667 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a415c2f0 con 0x7fb7a40aebd0 2024-04-25T10:27:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.877+0000 7fb7a0a00640 1 -- 172.21.15.87:0/4176035667 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253547526 0 0) 0x7fb7a4071360 con 0x7fb7a4072020 2024-04-25T10:27:26.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a1400640 1 --2- 172.21.15.87:0/4176035667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a40aebd0 0x7fb7a4071ae0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fb790009870 tx=0x7fb79001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78af26f88394315e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a0a00640 1 -- 172.21.15.87:0/4176035667 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900ab650 con 0x7fb7a40aebd0 2024-04-25T10:27:26.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a0a00640 1 -- 172.21.15.87:0/4176035667 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7900b0070 con 0x7fb7a40aebd0 2024-04-25T10:27:26.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a0a00640 1 -- 172.21.15.87:0/4176035667 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900aba10 con 0x7fb7a40aebd0 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 -- 172.21.15.87:0/4176035667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a40aebd0 msgr2=0x7fb7a4071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/4176035667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a40aebd0 0x7fb7a4071ae0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fb790009870 tx=0x7fb79001c530 comp rx=0 tx=0).stop 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 -- 172.21.15.87:0/4176035667 shutdown_connections 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/4176035667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a40aebd0 0x7fb7a4071ae0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/4176035667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a4159540 0x7fb7a40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 -- 172.21.15.87:0/4176035667 >> 172.21.15.87:0/4176035667 conn(0x7fb7a40669a0 msgr2=0x7fb7a4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 -- 172.21.15.87:0/4176035667 shutdown_connections 2024-04-25T10:27:26.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 -- 172.21.15.87:0/4176035667 wait complete. 2024-04-25T10:27:26.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 Processor -- start 2024-04-25T10:27:26.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.881+0000 7fb7a8e00640 1 -- start start 2024-04-25T10:27:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a4072020 0x7fb7a41411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a40aebd0 0x7fb7a4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7a4159540 0x7fb7a415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a406c0d0 con 0x7fb7a4072020 2024-04-25T10:27:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a415bfa0 con 0x7fb7a4159540 2024-04-25T10:27:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4072430 con 0x7fb7a40aebd0 2024-04-25T10:27:26.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a4072020 0x7fb7a41411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a4072020 0x7fb7a41411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48134/0 (socket says 172.21.15.87:48134) 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1097807602 learned_addr learned my addr 172.21.15.87:0/1097807602 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1097807602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a40aebd0 msgr2=0x7fb7a4152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a40aebd0 0x7fb7a4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1097807602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7a4159540 msgr2=0x7fb7a415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7a4159540 0x7fb7a415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1097807602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40af710 con 0x7fb7a4072020 2024-04-25T10:27:26.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a4072020 0x7fb7a41411c0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fb794009d10 tx=0x7fb79400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb79a000640 1 -- 172.21.15.87:0/1097807602 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794018020 con 0x7fb7a4072020 2024-04-25T10:27:26.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb79a000640 1 -- 172.21.15.87:0/1097807602 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794014480 con 0x7fb7a4072020 2024-04-25T10:27:26.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a40af9f0 con 0x7fb7a4072020 2024-04-25T10:27:26.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb79a000640 1 -- 172.21.15.87:0/1097807602 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794014830 con 0x7fb7a4072020 2024-04-25T10:27:26.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.885+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a40affb0 con 0x7fb7a4072020 2024-04-25T10:27:26.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb79a000640 1 -- 172.21.15.87:0/1097807602 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb79401b020 con 0x7fb7a4072020 2024-04-25T10:27:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb79a000640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb784066f90 0x7fb784069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb79a000640 1 -- 172.21.15.87:0/1097807602 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb794085c50 con 0x7fb7a4072020 2024-04-25T10:27:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1400640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb784066f90 0x7fb784069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 Processor -- start 2024-04-25T10:27:26.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 -- start start 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1400640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb784066f90 0x7fb784069450 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb7900af000 tx=0x7fb790004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66289d0 0x555dd6628db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66292f0 0x555dd65deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd65df480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd642bff0 con 0x555dd66289d0 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd63b4150 con 0x555dd65df0a0 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dd6428090 con 0x555dd66292f0 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66289d0 0x555dd6628db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66289d0 0x555dd6628db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48142/0 (socket says 172.21.15.87:48142) 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1797982541 learned_addr learned my addr 172.21.15.87:0/1797982541 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1797982541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66292f0 msgr2=0x555dd65deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/1797982541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66292f0 0x555dd65deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1797982541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 msgr2=0x555dd65df480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/1797982541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd65df480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 -- 172.21.15.87:0/1797982541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd65c20d0 con 0x555dd66289d0 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/1797982541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66289d0 0x555dd6628db0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fb79405e5b0 tx=0x7fb794056020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d1e706a5e66477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb78ea00640 1 -- 172.21.15.87:0/1797982541 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940546d0 con 0x555dd66289d0 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb78ea00640 1 -- 172.21.15.87:0/1797982541 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794052a40 con 0x555dd66289d0 2024-04-25T10:27:26.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb78ea00640 1 -- 172.21.15.87:0/1797982541 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794055820 con 0x555dd66289d0 2024-04-25T10:27:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 -- 172.21.15.87:0/1797982541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66289d0 msgr2=0x555dd6628db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.889+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1797982541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66289d0 0x555dd6628db0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fb79405e5b0 tx=0x7fb794056020 comp rx=0 tx=0).stop 2024-04-25T10:27:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- 172.21.15.87:0/1797982541 shutdown_connections 2024-04-25T10:27:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1797982541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd65df480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1797982541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66292f0 0x555dd65deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1797982541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66289d0 0x555dd6628db0 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- 172.21.15.87:0/1797982541 >> 172.21.15.87:0/1797982541 conn(0x555dd65db020 msgr2=0x555dd65dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- 172.21.15.87:0/1797982541 shutdown_connections 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- 172.21.15.87:0/1797982541 wait complete. 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- start start 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 Processor -- start 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd65df0a0 0x555dd660d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd660dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd66292f0 0x555dd660e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd63f0250 con 0x555dd65df0a0 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd63e6690 con 0x555dd66292f0 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dd63b4a40 con 0x555dd66289d0 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 10 client.0 _ll_register_callbacks cb 0x555dd65c9e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd65df0a0 0x555dd660d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd65df0a0 0x555dd660d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48144/0 (socket says 172.21.15.87:48144) 2024-04-25T10:27:26.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 -- 172.21.15.87:0/2053445149 learned_addr learned my addr 172.21.15.87:0/2053445149 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1400640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd660dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a2800640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd66292f0 0x555dd660e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 -- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 msgr2=0x555dd660dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd660dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 -- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd66292f0 msgr2=0x555dd660e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd66292f0 0x555dd660e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 -- 172.21.15.87:0/2053445149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dd64d91c0 con 0x555dd65df0a0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a2800640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd66292f0 0x555dd660e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd65df0a0 0x555dd660d7f0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fb79405e5b0 tx=0x7fb794000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491e1ecd0c791fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb783400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb783400640 1 -- 172.21.15.87:0/2053445149 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794055820 con 0x555dd65df0a0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb783400640 1 -- 172.21.15.87:0/2053445149 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb7940559c0 con 0x555dd65df0a0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb794052a40 con 0x555dd65df0a0 2024-04-25T10:27:26.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb783400640 1 -- 172.21.15.87:0/2053445149 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79407c050 con 0x555dd65df0a0 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 10 client.11709 fetch_fsmap learned FSMap version 143 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x555dd64d5170 con 0x555dd65df0a0 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb783400640 1 -- 172.21.15.87:0/2053445149 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794052a40 con 0x555dd65df0a0 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb783400640 1 -- 172.21.15.87:0/2053445149 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb79401ab40 con 0x555dd65df0a0 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 10 client.11709 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 10 client.11709 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 20 client.11709 populate_metadata read hostname 'smithi087' 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd6638c80 0x555dd663b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 4 client.11709 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dd63c7c60 con 0x555dd6638c80 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.893+0000 7fb7a1400640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd6638c80 0x555dd663b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb783400640 10 client.11709 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:26.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7a1400640 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd6638c80 0x555dd663b070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a856f916aa9d3dde server_cookie=a0896e3fbb270eb7 in_seq=0 out_seq=0 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb783400640 1 -- 172.21.15.87:0/2053445149 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555dd63c7c60 con 0x555dd6638c80 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb783400640 10 client.11709 handle_command_reply: tid=0 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 client.11709 shutdown 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd65df0a0 msgr2=0x555dd660d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd65df0a0 0x555dd660d7f0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fb79405e5b0 tx=0x7fb794000f50 comp rx=0 tx=0).stop 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 shutdown_connections 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd6638c80 0x555dd663b070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd66292f0 0x555dd660e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd660dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 --2- 172.21.15.87:0/2053445149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd65df0a0 0x555dd660d7f0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb783400640 0 client.11709 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 >> 172.21.15.87:0/2053445149 conn(0x555dd65db020 msgr2=0x555dd65ddca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 shutdown_connections 2024-04-25T10:27:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 1 -- 172.21.15.87:0/2053445149 wait complete. 2024-04-25T10:27:26.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.897+0000 7fb7ac205000 20 client.11709 trim_cache size 0 max 16384 2024-04-25T10:27:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 Processor -- start 2024-04-25T10:27:26.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- start start 2024-04-25T10:27:26.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd65de420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd65de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6646b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4200dd0 con 0x555dd66292f0 2024-04-25T10:27:26.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41f6c70 con 0x555dd65df0a0 2024-04-25T10:27:26.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a415a6c0 con 0x555dd66289d0 2024-04-25T10:27:26.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6646b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6646b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48150/0 (socket says 172.21.15.87:48150) 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 -- 172.21.15.87:0/1613638275 learned_addr learned my addr 172.21.15.87:0/1613638275 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a1400640 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd65de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd65de420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 -- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 msgr2=0x555dd65de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd65de960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 -- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 msgr2=0x555dd65de420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd65de420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 -- 172.21.15.87:0/1613638275 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd6472de0 con 0x555dd66292f0 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7a2800640 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6646b00 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fb79c005a80 tx=0x7fb79c005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f093d2f52fb222ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb78e000640 1 -- 172.21.15.87:0/1613638275 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c019070 con 0x555dd66292f0 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb78e000640 1 -- 172.21.15.87:0/1613638275 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c0014d0 con 0x555dd66292f0 2024-04-25T10:27:26.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb78e000640 1 -- 172.21.15.87:0/1613638275 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0018a0 con 0x555dd66292f0 2024-04-25T10:27:26.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- 172.21.15.87:0/1613638275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 msgr2=0x555dd6646b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6646b00 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fb79c005a80 tx=0x7fb79c005f40 comp rx=0 tx=0).stop 2024-04-25T10:27:26.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- 172.21.15.87:0/1613638275 shutdown_connections 2024-04-25T10:27:26.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6646b00 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd65de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 --2- 172.21.15.87:0/1613638275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd65de420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- 172.21.15.87:0/1613638275 >> 172.21.15.87:0/1613638275 conn(0x555dd65db020 msgr2=0x555dd66270d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- 172.21.15.87:0/1613638275 shutdown_connections 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.913+0000 7fb7ac205000 1 -- 172.21.15.87:0/1613638275 wait complete. 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 -- start start 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 Processor -- start 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd668d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd668def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6648e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd63c6e00 con 0x555dd66292f0 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41f6670 con 0x555dd65df0a0 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dd642b300 con 0x555dd66289d0 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 10 client.0 _ll_register_callbacks cb 0x555dd65aa040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6648e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd668d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd668d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.87:49374/0 (socket says 172.21.15.87:49374) 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a1e00640 1 -- 172.21.15.87:0/378820389 learned_addr learned my addr 172.21.15.87:0/378820389 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a1400640 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd668def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a2800640 1 -- 172.21.15.87:0/378820389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 msgr2=0x555dd668def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a2800640 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd668def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a2800640 1 -- 172.21.15.87:0/378820389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 msgr2=0x555dd668d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a2800640 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd668d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a2800640 1 -- 172.21.15.87:0/378820389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dd63cd8e0 con 0x555dd66292f0 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a2800640 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6648e20 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fb79c016000 tx=0x7fb79c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b25cfb83243910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 1 -- 172.21.15.87:0/378820389 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c025060 con 0x555dd66292f0 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 1 -- 172.21.15.87:0/378820389 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb79c001c40 con 0x555dd66292f0 2024-04-25T10:27:26.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb79c0014d0 con 0x555dd66292f0 2024-04-25T10:27:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 1 -- 172.21.15.87:0/378820389 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79c02b050 con 0x555dd66292f0 2024-04-25T10:27:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 10 client.11715 fetch_fsmap learned FSMap version 143 2024-04-25T10:27:26.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x555dd64858f0 con 0x555dd66292f0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 1 -- 172.21.15.87:0/378820389 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c0014d0 con 0x555dd66292f0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 1 -- 172.21.15.87:0/378820389 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb79c02e020 con 0x555dd66292f0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 10 client.11715 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 10 client.11715 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.a 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 20 client.11715 populate_metadata read hostname 'smithi087' 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd66498a0 0x555dd6649cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 4 client.11715 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555dd6439530 con 0x555dd66498a0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd66498a0 0x555dd6649cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7a1e00640 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd66498a0 0x555dd6649cb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a0d8d0f1756a6d server_cookie=620f1c1c13c220ec in_seq=0 out_seq=0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 10 client.11715 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 1 -- 172.21.15.87:0/378820389 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555dd6439530 con 0x555dd66498a0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb783400640 10 client.11715 handle_command_reply: tid=0 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.917+0000 7fb7ac205000 1 client.11715 shutdown 2024-04-25T10:27:26.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 msgr2=0x555dd6648e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6648e20 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fb79c016000 tx=0x7fb79c000ee0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 shutdown_connections 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x555dd66498a0 0x555dd6649cb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555dd66292f0 0x555dd6648e20 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555dd66289d0 0x555dd668def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 --2- 172.21.15.87:0/378820389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dd65df0a0 0x555dd668d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb783400640 0 client.11715 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 >> 172.21.15.87:0/378820389 conn(0x555dd65db020 msgr2=0x555dd6627960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 shutdown_connections 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 1 -- 172.21.15.87:0/378820389 wait complete. 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7ac205000 20 client.11715 trim_cache size 0 max 16384 2024-04-25T10:27:26.921 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:26.922 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-04-25T10:27:26.922 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-04-25T10:27:26.922 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb784066f90 msgr2=0x7fb784069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb784066f90 0x7fb784069450 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb7900af000 tx=0x7fb790004740 comp rx=0 tx=0).stop 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a4072020 msgr2=0x7fb7a41411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a4072020 0x7fb7a41411c0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fb794009d10 tx=0x7fb79400c490 comp rx=0 tx=0).stop 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 shutdown_connections 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb784066f90 0x7fb784069450 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7a4159540 0x7fb7a415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7a40aebd0 0x7fb7a4152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 --2- 172.21.15.87:0/1097807602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7a4072020 0x7fb7a41411c0 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 >> 172.21.15.87:0/1097807602 conn(0x7fb7a40669a0 msgr2=0x7fb7a4066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 shutdown_connections 2024-04-25T10:27:26.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:26.921+0000 7fb7a8e00640 1 -- 172.21.15.87:0/1097807602 wait complete. 2024-04-25T10:27:26.937 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:27:27.340 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:27.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.477+0000 7f7f5cc00640 1 Processor -- start 2024-04-25T10:27:27.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.477+0000 7f7f5cc00640 1 -- start start 2024-04-25T10:27:27.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f5cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f5815bc40 0x7f7f5806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f5cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f580cf930 0x7f7f580cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f5cc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f580b2770 con 0x7f7f580cda10 2024-04-25T10:27:27.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f5cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5806c670 con 0x7f7f5815bc40 2024-04-25T10:27:27.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f5cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5806c950 con 0x7f7f580cf930 2024-04-25T10:27:27.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f55e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f5815bc40 0x7f7f5806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f55e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f5815bc40 0x7f7f5806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44208/0 (socket says 172.21.15.87:44208) 2024-04-25T10:27:27.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f55e00640 1 -- 172.21.15.87:0/406536213 learned_addr learned my addr 172.21.15.87:0/406536213 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f56800640 1 --2- 172.21.15.87:0/406536213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f580cf930 0x7f7f580cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f55e00640 1 -- 172.21.15.87:0/406536213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f580cf930 msgr2=0x7f7f580cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f55e00640 1 --2- 172.21.15.87:0/406536213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f580cf930 0x7f7f580cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f55e00640 1 -- 172.21.15.87:0/406536213 >> v1:172.21.15.169:6789/0 conn(0x7f7f580cda10 legacy=0x7f7f580b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.481+0000 7f7f55e00640 1 -- 172.21.15.87:0/406536213 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5806cc30 con 0x7f7f5815bc40 2024-04-25T10:27:27.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f55e00640 1 --2- 172.21.15.87:0/406536213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5815bc40 0x7f7f5806c090 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009e50 tx=0x7f7f4801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac139f5a4839dbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f54a00640 1 -- 172.21.15.87:0/406536213 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480a5660 con 0x7f7f5815bc40 2024-04-25T10:27:27.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f54a00640 1 -- 172.21.15.87:0/406536213 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f480a5800 con 0x7f7f5815bc40 2024-04-25T10:27:27.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f54a00640 1 -- 172.21.15.87:0/406536213 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480a5bd0 con 0x7f7f5815bc40 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- 172.21.15.87:0/406536213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5815bc40 msgr2=0x7f7f5806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/406536213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5815bc40 0x7f7f5806c090 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009e50 tx=0x7f7f4801c9a0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- 172.21.15.87:0/406536213 shutdown_connections 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/406536213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f580cf930 0x7f7f580cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/406536213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5815bc40 0x7f7f5806c090 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- 172.21.15.87:0/406536213 >> 172.21.15.87:0/406536213 conn(0x7f7f58066b70 msgr2=0x7f7f58066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- 172.21.15.87:0/406536213 shutdown_connections 2024-04-25T10:27:27.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- 172.21.15.87:0/406536213 wait complete. 2024-04-25T10:27:27.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 Processor -- start 2024-04-25T10:27:27.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- start start 2024-04-25T10:27:27.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f580cda10 0x7f7f58151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f580cf930 0x7f7f580a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f5815bc40 0x7f7f58159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f580b3890 con 0x7f7f580cf930 2024-04-25T10:27:27.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f580b3710 con 0x7f7f580cda10 2024-04-25T10:27:27.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f5cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f580b3a10 con 0x7f7f5815bc40 2024-04-25T10:27:27.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.485+0000 7f7f55400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f580cf930 0x7f7f580a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f580cf930 0x7f7f580a9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44212/0 (socket says 172.21.15.87:44212) 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 -- 172.21.15.87:0/3118593663 learned_addr learned my addr 172.21.15.87:0/3118593663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55e00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f580cda10 0x7f7f58151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f56800640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f5815bc40 0x7f7f58159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 -- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f5815bc40 msgr2=0x7f7f58159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f5815bc40 0x7f7f58159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 -- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f580cda10 msgr2=0x7f7f58151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f580cda10 0x7f7f58151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 -- 172.21.15.87:0/3118593663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5815ea40 con 0x7f7f580cf930 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f56800640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f5815bc40 0x7f7f58159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55400640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f580cf930 0x7f7f580a9e40 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400c3a0 tx=0x7f7f4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f4e000640 1 -- 172.21.15.87:0/3118593663 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f440052d0 con 0x7f7f580cf930 2024-04-25T10:27:27.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f4e000640 1 -- 172.21.15.87:0/3118593663 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f44005470 con 0x7f7f580cf930 2024-04-25T10:27:27.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f4e000640 1 -- 172.21.15.87:0/3118593663 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44007260 con 0x7f7f580cf930 2024-04-25T10:27:27.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5815d650 con 0x7f7f580cf930 2024-04-25T10:27:27.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5815dc10 con 0x7f7f580cf930 2024-04-25T10:27:27.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5804ef30 con 0x7f7f580cf930 2024-04-25T10:27:27.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f4e000640 1 -- 172.21.15.87:0/3118593663 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f44007400 con 0x7f7f580cf930 2024-04-25T10:27:27.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f4e000640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f38066f90 0x7f7f38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f4e000640 1 -- 172.21.15.87:0/3118593663 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f7f440844c0 con 0x7f7f580cf930 2024-04-25T10:27:27.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.489+0000 7f7f55e00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f38066f90 0x7f7f38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.493+0000 7f7f55e00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f38066f90 0x7f7f38069450 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f480ae000 tx=0x7f7f480a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.497+0000 7f7f4e000640 1 -- 172.21.15.87:0/3118593663 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f44054d80 con 0x7f7f580cf930 2024-04-25T10:27:27.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f5806d960 con 0x7f7f580cf930 2024-04-25T10:27:27.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f4e000640 1 -- 172.21.15.87:0/3118593663 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f7f44058e20 con 0x7f7f580cf930 2024-04-25T10:27:27.678 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:27.678 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:23.173736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:27:27.679 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 143 2024-04-25T10:27:27.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f38066f90 msgr2=0x7f7f38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f38066f90 0x7f7f38069450 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f480ae000 tx=0x7f7f480a7070 comp rx=0 tx=0).stop 2024-04-25T10:27:27.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f580cf930 msgr2=0x7f7f580a9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f580cf930 0x7f7f580a9e40 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400c3a0 tx=0x7f7f4400c860 comp rx=0 tx=0).stop 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 shutdown_connections 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f7f38066f90 0x7f7f38069450 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f5815bc40 0x7f7f58159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f580cf930 0x7f7f580a9e40 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 --2- 172.21.15.87:0/3118593663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f580cda10 0x7f7f58151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 >> 172.21.15.87:0/3118593663 conn(0x7f7f58066b70 msgr2=0x7f7f58157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 shutdown_connections 2024-04-25T10:27:27.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.681+0000 7f7f5cc00640 1 -- 172.21.15.87:0/3118593663 wait complete. 2024-04-25T10:27:27.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 143, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:26:09.137144+0000', 'modified': '2024-04-25T10:26:23.173736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 188, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11520': {'gid': 11520, 'name': 'a', 'rank': 0, 'incarnation': 140, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.87:6836/3730332371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 3730332371}, {'type': 'v1', 'addr': '172.21.15.87:6836', 'nonce': 3730332371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:27:27.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:27:27.700 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.837+0000 7fb82e600640 1 Processor -- start 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.837+0000 7fb82e600640 1 -- start start 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb82e600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb82815bc40 0x7fb82806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb82e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8280cf930 0x7fb8280cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb82e600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8280b2770 con 0x7fb8280cda10 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb82e600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82806c670 con 0x7fb82815bc40 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb82e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82806c950 con 0x7fb8280cf930 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb82815bc40 0x7fb82806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb82815bc40 0x7fb82806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44226/0 (socket says 172.21.15.87:44226) 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 -- 172.21.15.87:0/595575226 learned_addr learned my addr 172.21.15.87:0/595575226 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb826a00640 1 --1- 172.21.15.87:0/595575226 >> v1:172.21.15.87:6789/0 conn(0x7fb8280cda10 0x7fb8280b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57904/0 (socket says 172.21.15.87:57904) 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827e00640 1 --2- 172.21.15.87:0/595575226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8280cf930 0x7fb8280cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 -- 172.21.15.87:0/595575226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8280cf930 msgr2=0x7fb8280cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 --2- 172.21.15.87:0/595575226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8280cf930 0x7fb8280cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 -- 172.21.15.87:0/595575226 >> v1:172.21.15.87:6789/0 conn(0x7fb8280cda10 legacy=0x7fb8280b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 -- 172.21.15.87:0/595575226 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82806cc30 con 0x7fb82815bc40 2024-04-25T10:27:27.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb826000640 1 -- 172.21.15.87:0/595575226 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414248895 0 0) 0x7fb8280b2770 con 0x7fb8280cda10 2024-04-25T10:27:27.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827e00640 1 --2- 172.21.15.87:0/595575226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8280cf930 0x7fb8280cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:27.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.841+0000 7fb827400640 1 --2- 172.21.15.87:0/595575226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb82815bc40 0x7fb82806c090 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fb818009870 tx=0x7fb81801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baad05bdfff6af98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb826000640 1 -- 172.21.15.87:0/595575226 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180b0070 con 0x7fb82815bc40 2024-04-25T10:27:27.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb826000640 1 -- 172.21.15.87:0/595575226 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8180ab600 con 0x7fb82815bc40 2024-04-25T10:27:27.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb826000640 1 -- 172.21.15.87:0/595575226 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180ab9d0 con 0x7fb82815bc40 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- 172.21.15.87:0/595575226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb82815bc40 msgr2=0x7fb82806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 --2- 172.21.15.87:0/595575226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb82815bc40 0x7fb82806c090 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fb818009870 tx=0x7fb81801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- 172.21.15.87:0/595575226 shutdown_connections 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 --2- 172.21.15.87:0/595575226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8280cf930 0x7fb8280cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 --2- 172.21.15.87:0/595575226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb82815bc40 0x7fb82806c090 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- 172.21.15.87:0/595575226 >> 172.21.15.87:0/595575226 conn(0x7fb828066b70 msgr2=0x7fb828066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- 172.21.15.87:0/595575226 shutdown_connections 2024-04-25T10:27:27.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- 172.21.15.87:0/595575226 wait complete. 2024-04-25T10:27:27.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 Processor -- start 2024-04-25T10:27:27.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- start start 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8280cda10 0x7fb828147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8280cf930 0x7fb828159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb82815bc40 0x7fb828159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280b3890 con 0x7fb8280cda10 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280b3710 con 0x7fb8280cf930 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb82e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280b3a10 con 0x7fb82815bc40 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb827400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8280cda10 0x7fb828147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb827400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8280cda10 0x7fb828147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44240/0 (socket says 172.21.15.87:44240) 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb827400640 1 -- 172.21.15.87:0/1573379303 learned_addr learned my addr 172.21.15.87:0/1573379303 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.845+0000 7fb826a00640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8280cf930 0x7fb828159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb827e00640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb82815bc40 0x7fb828159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb827400640 1 -- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb82815bc40 msgr2=0x7fb828159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb827400640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb82815bc40 0x7fb828159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb827400640 1 -- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8280cf930 msgr2=0x7fb828159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb827400640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8280cf930 0x7fb828159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb827400640 1 -- 172.21.15.87:0/1573379303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828148060 con 0x7fb8280cda10 2024-04-25T10:27:27.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb827400640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8280cda10 0x7fb828147b20 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb8180ae040 tx=0x7fb81801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb81f400640 1 -- 172.21.15.87:0/1573379303 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180b0040 con 0x7fb8280cda10 2024-04-25T10:27:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb81f400640 1 -- 172.21.15.87:0/1573379303 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8180bc070 con 0x7fb8280cda10 2024-04-25T10:27:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82815c3a0 con 0x7fb8280cda10 2024-04-25T10:27:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb81f400640 1 -- 172.21.15.87:0/1573379303 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180aaa30 con 0x7fb8280cda10 2024-04-25T10:27:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82815c960 con 0x7fb8280cda10 2024-04-25T10:27:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.849+0000 7fb81f400640 1 -- 172.21.15.87:0/1573379303 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8180aabd0 con 0x7fb8280cda10 2024-04-25T10:27:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb81f400640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb808066f60 0x7fb808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb81f400640 1 -- 172.21.15.87:0/1573379303 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb81811f580 con 0x7fb8280cda10 2024-04-25T10:27:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb826a00640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb808066f60 0x7fb808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 Processor -- start 2024-04-25T10:27:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- start start 2024-04-25T10:27:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb826a00640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb808066f60 0x7fb808069420 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb8280aeba0 tx=0x7fb814007d10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878265ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x5587826149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782615310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587824fc5f0 con 0x558782614f30 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558782463da0 con 0x55878265f240 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587823ec150 con 0x55878265e920 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782615310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782615310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44254/0 (socket says 172.21.15.87:44254) 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 -- 172.21.15.87:0/2981145772 learned_addr learned my addr 172.21.15.87:0/2981145772 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb826a00640 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x5587826149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827400640 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878265ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 -- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 msgr2=0x55878265ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878265ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 -- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 msgr2=0x5587826149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x5587826149f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 -- 172.21.15.87:0/2981145772 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558782405ca0 con 0x558782614f30 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb826a00640 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x5587826149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb827e00640 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782615310 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fb8200044f0 tx=0x7fb8200095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459b4d5597786936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb813e00640 1 -- 172.21.15.87:0/2981145772 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820014610 con 0x558782614f30 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb813e00640 1 -- 172.21.15.87:0/2981145772 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820019070 con 0x558782614f30 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb813e00640 1 -- 172.21.15.87:0/2981145772 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820014a10 con 0x558782614f30 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- 172.21.15.87:0/2981145772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 msgr2=0x558782615310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782615310 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fb8200044f0 tx=0x7fb8200095f0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- 172.21.15.87:0/2981145772 shutdown_connections 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782615310 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x5587826149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2981145772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878265ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- 172.21.15.87:0/2981145772 >> 172.21.15.87:0/2981145772 conn(0x55878261b2a0 msgr2=0x5587826132d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- 172.21.15.87:0/2981145772 shutdown_connections 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.853+0000 7fb831b8f000 1 -- 172.21.15.87:0/2981145772 wait complete. 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 -- start start 2024-04-25T10:27:27.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 Processor -- start 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782665cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x558782666230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x558782618aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55878241e990 con 0x558782614f30 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55878245fe40 con 0x55878265f240 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587823ef900 con 0x55878265e920 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 10 client.0 _ll_register_callbacks cb 0x5587825171c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782665cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782665cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44256/0 (socket says 172.21.15.87:44256) 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 -- 172.21.15.87:0/109973542 learned_addr learned my addr 172.21.15.87:0/109973542 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827e00640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x558782618aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb826a00640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x558782666230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 -- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 msgr2=0x558782666230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x558782666230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 -- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 msgr2=0x558782618aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x558782618aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 -- 172.21.15.87:0/109973542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55878241c630 con 0x558782614f30 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb826a00640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x558782666230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827e00640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x558782618aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb827400640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782665cf0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fb8180efbc0 tx=0x7fb8180ef820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa69cba585ce6156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb810c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb810c00640 1 -- 172.21.15.87:0/109973542 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180f05c0 con 0x558782614f30 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb810c00640 1 -- 172.21.15.87:0/109973542 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb8180f0120 con 0x558782614f30 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb820019070 con 0x558782614f30 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb810c00640 1 -- 172.21.15.87:0/109973542 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8180efec0 con 0x558782614f30 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 10 client.11736 fetch_fsmap learned FSMap version 143 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x5587823ec3a0 con 0x558782614f30 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb810c00640 1 -- 172.21.15.87:0/109973542 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820019070 con 0x558782614f30 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb810c00640 1 -- 172.21.15.87:0/109973542 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8180b3030 con 0x558782614f30 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 10 client.11736 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 10 client.11736 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 20 client.11736 populate_metadata read hostname 'smithi087' 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826194b0 0x558782619890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 4 client.11736 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587823ffa50 con 0x5587826194b0 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.857+0000 7fb826a00640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826194b0 0x558782619890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb826a00640 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826194b0 0x558782619890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48f082de28bb44a server_cookie=d998deefa048b111 in_seq=0 out_seq=0 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb810c00640 10 client.11736 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb810c00640 1 -- 172.21.15.87:0/109973542 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587823ffa50 con 0x5587826194b0 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb810c00640 10 client.11736 handle_command_reply: tid=0 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 client.11736 shutdown 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 msgr2=0x558782665cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782665cf0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fb8180efbc0 tx=0x7fb8180ef820 comp rx=0 tx=0).stop 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 shutdown_connections 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826194b0 0x558782619890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x558782618aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x558782666230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 --2- 172.21.15.87:0/109973542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x558782665cf0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb810c00640 0 client.11736 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 >> 172.21.15.87:0/109973542 conn(0x55878261b2a0 msgr2=0x5587826139e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 shutdown_connections 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 1 -- 172.21.15.87:0/109973542 wait complete. 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.861+0000 7fb831b8f000 20 client.11736 trim_cache size 0 max 16384 2024-04-25T10:27:27.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 Processor -- start 2024-04-25T10:27:27.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- start start 2024-04-25T10:27:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x55878266bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878266c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x55878267a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82814bfd0 con 0x558782614f30 2024-04-25T10:27:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82814d9b0 con 0x55878265f240 2024-04-25T10:27:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82814b9d0 con 0x55878265e920 2024-04-25T10:27:27.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x55878266bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x55878266bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44266/0 (socket says 172.21.15.87:44266) 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 -- 172.21.15.87:0/985971819 learned_addr learned my addr 172.21.15.87:0/985971819 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb826a00640 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878266c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827e00640 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x55878267a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 -- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 msgr2=0x55878266c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878266c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 -- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 msgr2=0x55878267a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x55878267a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 -- 172.21.15.87:0/985971819 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587824e5600 con 0x558782614f30 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827e00640 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x55878267a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb826a00640 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878266c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb827400640 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x55878266bbe0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fb8180ec320 tx=0x7fb8180ef950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3917405366a4107f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb813400640 1 -- 172.21.15.87:0/985971819 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180ff070 con 0x558782614f30 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb813400640 1 -- 172.21.15.87:0/985971819 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8180f05c0 con 0x558782614f30 2024-04-25T10:27:27.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb813400640 1 -- 172.21.15.87:0/985971819 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180f4060 con 0x558782614f30 2024-04-25T10:27:27.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- 172.21.15.87:0/985971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 msgr2=0x55878266bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x55878266bbe0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fb8180ec320 tx=0x7fb8180ef950 comp rx=0 tx=0).stop 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- 172.21.15.87:0/985971819 shutdown_connections 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55878265f240 0x55878267a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265e920 0x55878266c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 --2- 172.21.15.87:0/985971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558782614f30 0x55878266bbe0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- 172.21.15.87:0/985971819 >> 172.21.15.87:0/985971819 conn(0x55878261b2a0 msgr2=0x558782618220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- 172.21.15.87:0/985971819 shutdown_connections 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- 172.21.15.87:0/985971819 wait complete. 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.877+0000 7fb831b8f000 1 -- start start 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 Processor -- start 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558782614f30 0x5587826a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55878265e920 0x5587826a7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265f240 0x5587826a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587824683d0 con 0x55878265e920 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55878251fd50 con 0x558782614f30 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587823fe980 con 0x55878265f240 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 10 client.0 _ll_register_callbacks cb 0x7fb820003cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55878265e920 0x5587826a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55878265e920 0x5587826a7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44276/0 (socket says 172.21.15.87:44276) 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 -- 172.21.15.87:0/2513135878 learned_addr learned my addr 172.21.15.87:0/2513135878 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb827400640 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558782614f30 0x5587826a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb827e00640 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265f240 0x5587826a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 -- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265f240 msgr2=0x5587826a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265f240 0x5587826a7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 -- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558782614f30 msgr2=0x5587826a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558782614f30 0x5587826a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 -- 172.21.15.87:0/2513135878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587823f89b0 con 0x55878265e920 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb826a00640 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55878265e920 0x5587826a7820 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fb814005fd0 tx=0x7fb814007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57ba5b69b8e07c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 1 -- 172.21.15.87:0/2513135878 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81401e040 con 0x55878265e920 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 1 -- 172.21.15.87:0/2513135878 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb814029980 con 0x55878265e920 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8180f05c0 con 0x55878265e920 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 1 -- 172.21.15.87:0/2513135878 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814034050 con 0x55878265e920 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 10 client.11742 fetch_fsmap learned FSMap version 143 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x55878240b0b0 con 0x55878265e920 2024-04-25T10:27:27.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 1 -- 172.21.15.87:0/2513135878 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8180f05c0 con 0x55878265e920 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 1 -- 172.21.15.87:0/2513135878 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb814029b80 con 0x55878265e920 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 10 client.11742 fetch_fsmap finished waiting for FSMap version 143 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 10 client.11742 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 20 client.11742 populate_metadata read hostname 'smithi087' 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826a82a0 0x5587826a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 4 client.11742 mds_command: new command op to 11520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 --> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5587824712e0 con 0x5587826a82a0 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb827400640 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826a82a0 0x5587826a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb827400640 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826a82a0 0x5587826a8680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bad139b711191ac server_cookie=def7f4a955e3afb in_seq=0 out_seq=0 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 10 client.11742 ms_handle_connect on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 1 -- 172.21.15.87:0/2513135878 <== mds.0 v2:172.21.15.87:6834/3730332371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x5587824712e0 con 0x5587826a82a0 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb810c00640 10 client.11742 handle_command_reply: tid=0 2024-04-25T10:27:27.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.881+0000 7fb831b8f000 1 client.11742 shutdown 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55878265e920 msgr2=0x5587826a7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55878265e920 0x5587826a7820 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fb814005fd0 tx=0x7fb814007b50 comp rx=0 tx=0).stop 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 shutdown_connections 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:6834/3730332371,v1:172.21.15.87:6836/3730332371] conn(0x5587826a82a0 0x5587826a8680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55878265f240 0x5587826a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55878265e920 0x5587826a7820 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 --2- 172.21.15.87:0/2513135878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558782614f30 0x5587826a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb810c00640 0 client.11742 ms_handle_reset on v2:172.21.15.87:6834/3730332371 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 >> 172.21.15.87:0/2513135878 conn(0x55878261b2a0 msgr2=0x5587826667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 shutdown_connections 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 1 -- 172.21.15.87:0/2513135878 wait complete. 2024-04-25T10:27:27.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb831b8f000 20 client.11742 trim_cache size 0 max 16384 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "84c5cae3-3d36-4b8f-9ce8-349d9b4e086b", 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 11520, 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 66.729890741999995, 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 143, 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 190, 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 188, 2024-04-25T10:27:27.888 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 67.673852127999993 2024-04-25T10:27:27.889 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:27.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb808066f60 msgr2=0x7fb808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb808066f60 0x7fb808069420 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb8280aeba0 tx=0x7fb814007d10 comp rx=0 tx=0).stop 2024-04-25T10:27:27.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8280cda10 msgr2=0x7fb828147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8280cda10 0x7fb828147b20 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb8180ae040 tx=0x7fb81801ad10 comp rx=0 tx=0).stop 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 shutdown_connections 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb808066f60 0x7fb808069420 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb82815bc40 0x7fb828159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8280cf930 0x7fb828159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 --2- 172.21.15.87:0/1573379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8280cda10 0x7fb828147b20 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 >> 172.21.15.87:0/1573379303 conn(0x7fb828066b70 msgr2=0x7fb82815ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 shutdown_connections 2024-04-25T10:27:27.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:27.885+0000 7fb82e600640 1 -- 172.21.15.87:0/1573379303 wait complete. 2024-04-25T10:27:27.900 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:27:27.900 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:27:27.948 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:27:27.997 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:27:27.997 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:27:27.997 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:28.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:27:28.006 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:27:28.006 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:28.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:28.015 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:27:28.015 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:28.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:27:28.057 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:27:28.057 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:28.066 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:28.066 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:28.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.241+0000 7feaeca00640 1 Processor -- start 2024-04-25T10:27:28.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.241+0000 7feaeca00640 1 -- start start 2024-04-25T10:27:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feaeca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feae81550b0 0x7feae80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feaeca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae815b910 0x7feae80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feaeca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae8071050 con 0x7feae80cfe70 2024-04-25T10:27:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feaeca00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae815bda0 con 0x7feae81550b0 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feaeca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae815d040 con 0x7feae815b910 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feae81550b0 0x7feae80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feae81550b0 0x7feae80cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44286/0 (socket says 172.21.15.87:44286) 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae5e00640 1 -- 172.21.15.87:0/168296212 learned_addr learned my addr 172.21.15.87:0/168296212 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae6800640 1 --2- 172.21.15.87:0/168296212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae815b910 0x7feae80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae5e00640 1 -- 172.21.15.87:0/168296212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae815b910 msgr2=0x7feae80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae5e00640 1 --2- 172.21.15.87:0/168296212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae815b910 0x7feae80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae5e00640 1 -- 172.21.15.87:0/168296212 >> v1:172.21.15.169:6789/0 conn(0x7feae80cfe70 legacy=0x7feae815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae5e00640 1 -- 172.21.15.87:0/168296212 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80712f0 con 0x7feae81550b0 2024-04-25T10:27:28.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.245+0000 7feae6800640 1 --2- 172.21.15.87:0/168296212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae815b910 0x7feae80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feae5e00640 1 --2- 172.21.15.87:0/168296212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae81550b0 0x7feae80cf930 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e80 tx=0x7fead801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ce513e8937da48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feae4a00640 1 -- 172.21.15.87:0/168296212 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead80a5740 con 0x7feae81550b0 2024-04-25T10:27:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feae4a00640 1 -- 172.21.15.87:0/168296212 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead80a58e0 con 0x7feae81550b0 2024-04-25T10:27:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feae4a00640 1 -- 172.21.15.87:0/168296212 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead80a5cb0 con 0x7feae81550b0 2024-04-25T10:27:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- 172.21.15.87:0/168296212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae81550b0 msgr2=0x7feae80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 --2- 172.21.15.87:0/168296212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae81550b0 0x7feae80cf930 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e80 tx=0x7fead801c9d0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- 172.21.15.87:0/168296212 shutdown_connections 2024-04-25T10:27:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 --2- 172.21.15.87:0/168296212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae815b910 0x7feae80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 --2- 172.21.15.87:0/168296212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae81550b0 0x7feae80cf930 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- 172.21.15.87:0/168296212 >> 172.21.15.87:0/168296212 conn(0x7feae8066b70 msgr2=0x7feae8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- 172.21.15.87:0/168296212 shutdown_connections 2024-04-25T10:27:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- 172.21.15.87:0/168296212 wait complete. 2024-04-25T10:27:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 Processor -- start 2024-04-25T10:27:28.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- start start 2024-04-25T10:27:28.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae80cfe70 0x7feae8159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae81550b0 0x7feae80b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae815b910 0x7feae80b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae815c030 con 0x7feae80cfe70 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80720b0 con 0x7feae815b910 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.249+0000 7feaeca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae815c1b0 con 0x7feae81550b0 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae80cfe70 0x7feae8159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae80cfe70 0x7feae8159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44296/0 (socket says 172.21.15.87:44296) 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 -- 172.21.15.87:0/1534241794 learned_addr learned my addr 172.21.15.87:0/1534241794 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5400640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae81550b0 0x7feae80b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae6800640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae815b910 0x7feae80b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 -- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae81550b0 msgr2=0x7feae80b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae81550b0 0x7feae80b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 -- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae815b910 msgr2=0x7feae80b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae815b910 0x7feae80b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 -- 172.21.15.87:0/1534241794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8072310 con 0x7feae80cfe70 2024-04-25T10:27:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feae5e00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae80cfe70 0x7feae8159490 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7fead801b080 tx=0x7fead8019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feade000640 1 -- 172.21.15.87:0/1534241794 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead80b8670 con 0x7feae80cfe70 2024-04-25T10:27:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feade000640 1 -- 172.21.15.87:0/1534241794 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead80be070 con 0x7feae80cfe70 2024-04-25T10:27:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae8158aa0 con 0x7feae80cfe70 2024-04-25T10:27:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feade000640 1 -- 172.21.15.87:0/1534241794 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead80b8a50 con 0x7feae80cfe70 2024-04-25T10:27:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae80af760 con 0x7feae80cfe70 2024-04-25T10:27:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae804ef30 con 0x7feae80cfe70 2024-04-25T10:27:28.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feade000640 1 -- 172.21.15.87:0/1534241794 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fead80b8bf0 con 0x7feae80cfe70 2024-04-25T10:27:28.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.253+0000 7feade000640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feac8066f60 0x7feac8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.257+0000 7feade000640 1 -- 172.21.15.87:0/1534241794 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fead8120470 con 0x7feae80cfe70 2024-04-25T10:27:28.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.257+0000 7feae5400640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feac8066f60 0x7feac8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.257+0000 7feae5400640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feac8066f60 0x7feac8069420 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7feae815b7d0 tx=0x7fead4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.261+0000 7feade000640 1 -- 172.21.15.87:0/1534241794 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fead80165c0 con 0x7feae80cfe70 2024-04-25T10:27:28.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.449+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feae80cbec0 con 0x7feae80cfe70 2024-04-25T10:27:28.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feade000640 1 -- 172.21.15.87:0/1534241794 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fead80f13b0 con 0x7feae80cfe70 2024-04-25T10:27:28.449 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:28.449 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:23.173736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:27:28.450 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 143 2024-04-25T10:27:28.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feac8066f60 msgr2=0x7feac8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feac8066f60 0x7feac8069420 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7feae815b7d0 tx=0x7fead4009040 comp rx=0 tx=0).stop 2024-04-25T10:27:28.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae80cfe70 msgr2=0x7feae8159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae80cfe70 0x7feae8159490 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7fead801b080 tx=0x7fead8019770 comp rx=0 tx=0).stop 2024-04-25T10:27:28.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 shutdown_connections 2024-04-25T10:27:28.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7feac8066f60 0x7feac8069420 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae815b910 0x7feae80b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feae81550b0 0x7feae80b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 --2- 172.21.15.87:0/1534241794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feae80cfe70 0x7feae8159490 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 >> 172.21.15.87:0/1534241794 conn(0x7feae8066b70 msgr2=0x7feae8154200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:28.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 shutdown_connections 2024-04-25T10:27:28.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.453+0000 7feaeca00640 1 -- 172.21.15.87:0/1534241794 wait complete. 2024-04-25T10:27:28.471 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:28.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0aa2400640 1 Processor -- start 2024-04-25T10:27:28.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0aa2400640 1 -- start start 2024-04-25T10:27:28.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0aa2400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0a9c15dce0 0x7f0a9c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0aa2400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c06e6a0 con 0x7f0a9c06eec0 2024-04-25T10:27:28.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0aa2400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c15e0c0 con 0x7f0a9c0ad850 2024-04-25T10:27:28.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0aa2400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0674d0 con 0x7f0a9c15dce0 2024-04-25T10:27:28.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0a9c06eec0 0x7f0a9c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57916/0 (socket says 172.21.15.87:57916) 2024-04-25T10:27:28.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9aa00640 1 -- 172.21.15.87:0/702825698 learned_addr learned my addr 172.21.15.87:0/702825698 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840565254 0 0) 0x7f0a9c06e6a0 con 0x7f0a9c06eec0 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a80003530 con 0x7f0a9c06eec0 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9b400640 1 --2- 172.21.15.87:0/702825698 >> v2:172.21.15.169:3300/0 conn(0x7f0a9c15dce0 0x7f0a9c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 156813452 0 0) 0x7f0a80003530 con 0x7f0a9c06eec0 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 >> v2:172.21.15.169:3300/0 conn(0x7f0a9c15dce0 msgr2=0x7f0a9c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 --2- 172.21.15.87:0/702825698 >> v2:172.21.15.169:3300/0 conn(0x7f0a9c15dce0 0x7f0a9c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 >> v1:172.21.15.169:6789/0 conn(0x7f0a9c0ad850 legacy=0x7f0a9c0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0671f0 con 0x7f0a9c06eec0 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0a8c001680 con 0x7f0a9c06eec0 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 >> v1:172.21.15.87:6789/0 conn(0x7f0a9c06eec0 legacy=0x7f0a9c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a800039c0 0x7f0a80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a80004810 0x7f0a80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a800052f0 0x7f0a8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0674d0 con 0x7f0a80004810 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8000c1c0 con 0x7f0a800052f0 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.609+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8000c3c0 con 0x7f0a800039c0 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9aa00640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a80004810 0x7f0a80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9b400640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a800039c0 0x7f0a80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9be00640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a800052f0 0x7f0a8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9aa00640 1 -- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a800039c0 msgr2=0x7f0a80003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9aa00640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a800039c0 0x7f0a80003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9aa00640 1 -- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a800052f0 msgr2=0x7f0a8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9aa00640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a800052f0 0x7f0a8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9aa00640 1 -- 172.21.15.87:0/702825698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80003530 con 0x7f0a80004810 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9be00640 1 -- 172.21.15.87:0/702825698 reap_dead start 2024-04-25T10:27:28.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9aa00640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a80004810 0x7f0a80004c90 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c190 tx=0x7f0a8c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a03443350d87e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c007d00 con 0x7f0a80004810 2024-04-25T10:27:28.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c0a9020 con 0x7f0a80004810 2024-04-25T10:27:28.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.613+0000 7f0a9a000640 1 -- 172.21.15.87:0/702825698 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c005ae0 con 0x7f0a80004810 2024-04-25T10:27:28.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- 172.21.15.87:0/702825698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a80004810 msgr2=0x7f0a80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a80004810 0x7f0a80004c90 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c190 tx=0x7f0a8c01cc40 comp rx=0 tx=0).stop 2024-04-25T10:27:28.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- 172.21.15.87:0/702825698 shutdown_connections 2024-04-25T10:27:28.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 --2- 172.21.15.87:0/702825698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a80004810 0x7f0a80004c90 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- 172.21.15.87:0/702825698 >> 172.21.15.87:0/702825698 conn(0x7f0a9c0b2410 msgr2=0x7f0a9c0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:28.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- 172.21.15.87:0/702825698 shutdown_connections 2024-04-25T10:27:28.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- 172.21.15.87:0/702825698 wait complete. 2024-04-25T10:27:28.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 Processor -- start 2024-04-25T10:27:28.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- start start 2024-04-25T10:27:28.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a80004810 0x7f0a9c1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a9c1574a0 0x7f0a9c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a9c151960 0x7f0a9c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c066f30 con 0x7f0a9c1574a0 2024-04-25T10:27:28.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c066db0 con 0x7f0a80004810 2024-04-25T10:27:28.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0aa2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0cfad0 con 0x7f0a9c151960 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a9c1574a0 0x7f0a9c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a9c1574a0 0x7f0a9c155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44318/0 (socket says 172.21.15.87:44318) 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 -- 172.21.15.87:0/2590321640 learned_addr learned my addr 172.21.15.87:0/2590321640 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9b400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a80004810 0x7f0a9c1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9be00640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a9c151960 0x7f0a9c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 -- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a9c151960 msgr2=0x7f0a9c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a9c151960 0x7f0a9c15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 -- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a80004810 msgr2=0x7f0a9c1579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a80004810 0x7f0a9c1579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 -- 172.21.15.87:0/2590321640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c1599f0 con 0x7f0a9c1574a0 2024-04-25T10:27:28.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9be00640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a9c151960 0x7f0a9c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:28.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a9aa00640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a9c1574a0 0x7f0a9c155330 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c190 tx=0x7f0a80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.617+0000 7f0a93400640 1 -- 172.21.15.87:0/2590321640 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c001c40 con 0x7f0a9c1574a0 2024-04-25T10:27:28.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0a93400640 1 -- 172.21.15.87:0/2590321640 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c001440 con 0x7f0a9c1574a0 2024-04-25T10:27:28.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0a93400640 1 -- 172.21.15.87:0/2590321640 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c007d00 con 0x7f0a9c1574a0 2024-04-25T10:27:28.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c159cb0 con 0x7f0a9c1574a0 2024-04-25T10:27:28.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c15a270 con 0x7f0a9c1574a0 2024-04-25T10:27:28.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c04ef30 con 0x7f0a9c1574a0 2024-04-25T10:27:28.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0a93400640 1 -- 172.21.15.87:0/2590321640 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a8c0a5810 con 0x7f0a9c1574a0 2024-04-25T10:27:28.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0a93400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a78066f90 0x7f0a78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0a93400640 1 -- 172.21.15.87:0/2590321640 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f0a8c11e1a0 con 0x7f0a9c1574a0 2024-04-25T10:27:28.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0a9b400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a78066f90 0x7f0a78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.621+0000 7f0a9b400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a78066f90 0x7f0a78069450 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c06a000 tx=0x7f0a84009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.625+0000 7f0a93400640 1 -- 172.21.15.87:0/2590321640 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a8c0bf070 con 0x7f0a9c1574a0 2024-04-25T10:27:28.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.781+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a9c06eca0 con 0x7f0a9c1574a0 2024-04-25T10:27:28.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0a93400640 1 -- 172.21.15.87:0/2590321640 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+17034 (secure 0 0 0) 0x7f0a8c016950 con 0x7f0a9c1574a0 2024-04-25T10:27:28.782 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:28.782 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":190,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:27:08.279567+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:07.509820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:08.125891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]},{"pgid":"29.23","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6836/997618220":"2024-04-26T10:26:20.143429+0000","172.21.15.87:6834/997618220":"2024-04-26T10:26:20.143429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:28.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a78066f90 msgr2=0x7f0a78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a78066f90 0x7f0a78069450 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c06a000 tx=0x7f0a84009190 comp rx=0 tx=0).stop 2024-04-25T10:27:28.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a9c1574a0 msgr2=0x7f0a9c155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a9c1574a0 0x7f0a9c155330 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c190 tx=0x7f0a80005840 comp rx=0 tx=0).stop 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 shutdown_connections 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0a78066f90 0x7f0a78069450 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a9c151960 0x7f0a9c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a9c1574a0 0x7f0a9c155330 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 --2- 172.21.15.87:0/2590321640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a80004810 0x7f0a9c1579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 >> 172.21.15.87:0/2590321640 conn(0x7f0a9c0b2410 msgr2=0x7f0a9c15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 shutdown_connections 2024-04-25T10:27:28.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.785+0000 7f0aa2400640 1 -- 172.21.15.87:0/2590321640 wait complete. 2024-04-25T10:27:28.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:12:11.188224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:26:07.509820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:26:08.125891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:27:28.803 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:27:28.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa45b000640 1 Processor -- start 2024-04-25T10:27:28.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa45b000640 1 -- start start 2024-04-25T10:27:28.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa45b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159540 0x7fa4540ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa45b000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa4540721f0 0x7fa454070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa45b000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa454159920 con 0x7fa4540aebd0 2024-04-25T10:27:28.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa45b000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540af030 con 0x7fa4540721f0 2024-04-25T10:27:28.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa45b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415c2f0 con 0x7fa454159540 2024-04-25T10:27:28.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa459000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa4540721f0 0x7fa454070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa44b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa4540aebd0 0x7fa454071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57926/0 (socket says 172.21.15.87:57926) 2024-04-25T10:27:28.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa44b400640 1 -- 172.21.15.87:0/1619670199 learned_addr learned my addr 172.21.15.87:0/1619670199 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:28.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa453e00640 1 --2- 172.21.15.87:0/1619670199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159540 0x7fa4540ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa459000640 1 -- 172.21.15.87:0/1619670199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159540 msgr2=0x7fa4540ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa459000640 1 --2- 172.21.15.87:0/1619670199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159540 0x7fa4540ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa459000640 1 -- 172.21.15.87:0/1619670199 >> v1:172.21.15.87:6789/0 conn(0x7fa4540aebd0 legacy=0x7fa454071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa459000640 1 -- 172.21.15.87:0/1619670199 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45415c5d0 con 0x7fa4540721f0 2024-04-25T10:27:28.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.945+0000 7fa453400640 1 -- 172.21.15.87:0/1619670199 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852252557 0 0) 0x7fa454159920 con 0x7fa4540aebd0 2024-04-25T10:27:28.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa459000640 1 --2- 172.21.15.87:0/1619670199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4540721f0 0x7fa454070350 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009d90 tx=0x7fa44c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da35cf0d9c85745e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa453400640 1 -- 172.21.15.87:0/1619670199 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0b2070 con 0x7fa4540721f0 2024-04-25T10:27:28.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa453400640 1 -- 172.21.15.87:0/1619670199 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c0a5a00 con 0x7fa4540721f0 2024-04-25T10:27:28.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa453400640 1 -- 172.21.15.87:0/1619670199 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0a5dd0 con 0x7fa4540721f0 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 -- 172.21.15.87:0/1619670199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4540721f0 msgr2=0x7fa454070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 --2- 172.21.15.87:0/1619670199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4540721f0 0x7fa454070350 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009d90 tx=0x7fa44c01c940 comp rx=0 tx=0).stop 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 -- 172.21.15.87:0/1619670199 shutdown_connections 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 --2- 172.21.15.87:0/1619670199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4540721f0 0x7fa454070350 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 --2- 172.21.15.87:0/1619670199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159540 0x7fa4540ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 -- 172.21.15.87:0/1619670199 >> 172.21.15.87:0/1619670199 conn(0x7fa4540669a0 msgr2=0x7fa454066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 -- 172.21.15.87:0/1619670199 shutdown_connections 2024-04-25T10:27:28.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.949+0000 7fa45b000640 1 -- 172.21.15.87:0/1619670199 wait complete. 2024-04-25T10:27:28.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 Processor -- start 2024-04-25T10:27:28.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 -- start start 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4540721f0 0x7fa45414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4540aebd0 0x7fa4540af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa454159540 0x7fa45414f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540710a0 con 0x7fa454159540 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45406c0d0 con 0x7fa4540721f0 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa454071220 con 0x7fa4540aebd0 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa454159540 0x7fa45414f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa454159540 0x7fa45414f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44338/0 (socket says 172.21.15.87:44338) 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 -- 172.21.15.87:0/67552653 learned_addr learned my addr 172.21.15.87:0/67552653 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa44b400640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4540aebd0 0x7fa4540af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa453e00640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4540721f0 0x7fa45414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 -- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4540aebd0 msgr2=0x7fa4540af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4540aebd0 0x7fa4540af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 -- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4540721f0 msgr2=0x7fa45414e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4540721f0 0x7fa45414e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 -- 172.21.15.87:0/67552653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540afc50 con 0x7fa454159540 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa44b400640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4540aebd0 0x7fa4540af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa459000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa454159540 0x7fa45414f980 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7fa44c01aff0 tx=0x7fa44c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa450c00640 1 -- 172.21.15.87:0/67552653 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0b2050 con 0x7fa454159540 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa450c00640 1 -- 172.21.15.87:0/67552653 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c0be070 con 0x7fa454159540 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa454150620 con 0x7fa454159540 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa450c00640 1 -- 172.21.15.87:0/67552653 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0adb50 con 0x7fa454159540 2024-04-25T10:27:28.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.953+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4540afef0 con 0x7fa454159540 2024-04-25T10:27:28.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.957+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45404ef30 con 0x7fa454159540 2024-04-25T10:27:28.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.957+0000 7fa450c00640 1 -- 172.21.15.87:0/67552653 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa44c0adcf0 con 0x7fa454159540 2024-04-25T10:27:28.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.957+0000 7fa450c00640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa430066f60 0x7fa430069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.957+0000 7fa450c00640 1 -- 172.21.15.87:0/67552653 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fa44c1205c0 con 0x7fa454159540 2024-04-25T10:27:28.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.957+0000 7fa453e00640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa430066f60 0x7fa430069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.957+0000 7fa453e00640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa430066f60 0x7fa430069420 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa4440079c0 tx=0x7fa444008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:28.961+0000 7fa450c00640 1 -- 172.21.15.87:0/67552653 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa44c016560 con 0x7fa454159540 2024-04-25T10:27:29.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.113+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa45406c550 con 0x7fa454159540 2024-04-25T10:27:29.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.113+0000 7fa450c00640 1 -- 172.21.15.87:0/67552653 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa44c0f1450 con 0x7fa454159540 2024-04-25T10:27:29.112 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-04-25T10:27:29.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa430066f60 msgr2=0x7fa430069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa430066f60 0x7fa430069420 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa4440079c0 tx=0x7fa444008040 comp rx=0 tx=0).stop 2024-04-25T10:27:29.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa454159540 msgr2=0x7fa45414f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa454159540 0x7fa45414f980 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7fa44c01aff0 tx=0x7fa44c0196e0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 shutdown_connections 2024-04-25T10:27:29.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa430066f60 0x7fa430069420 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa454159540 0x7fa45414f980 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4540aebd0 0x7fa4540af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 --2- 172.21.15.87:0/67552653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4540721f0 0x7fa45414e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 >> 172.21.15.87:0/67552653 conn(0x7fa4540669a0 msgr2=0x7fa454066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 shutdown_connections 2024-04-25T10:27:29.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.117+0000 7fa45b000640 1 -- 172.21.15.87:0/67552653 wait complete. 2024-04-25T10:27:29.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:27:29.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.273+0000 7fb049a00640 1 Processor -- start 2024-04-25T10:27:29.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.273+0000 7fb049a00640 1 -- start start 2024-04-25T10:27:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0440cde90 0x7fb0440b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04406be90 con 0x7fb0440b1f50 2024-04-25T10:27:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04406b8d0 con 0x7fb0440b23c0 2024-04-25T10:27:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0440d0b50 con 0x7fb0440cde90 2024-04-25T10:27:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb043e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb0440b1f50 0x7fb04406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57942/0 (socket says 172.21.15.87:57942) 2024-04-25T10:27:29.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb043e00640 1 -- 172.21.15.87:0/85031942 learned_addr learned my addr 172.21.15.87:0/85031942 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049000640 1 --2- 172.21.15.87:0/85031942 >> v2:172.21.15.169:3300/0 conn(0x7fb0440cde90 0x7fb0440b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042100872 0 0) 0x7fb034001120 con 0x7fb0440b1f50 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb028003530 con 0x7fb0440b1f50 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2005364435 0 0) 0x7fb028003530 con 0x7fb0440b1f50 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 >> v2:172.21.15.169:3300/0 conn(0x7fb0440cde90 msgr2=0x7fb0440b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 --2- 172.21.15.87:0/85031942 >> v2:172.21.15.169:3300/0 conn(0x7fb0440cde90 0x7fb0440b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 >> v1:172.21.15.169:6789/0 conn(0x7fb0440b23c0 legacy=0x7fb0440cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04406bbb0 con 0x7fb0440b1f50 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb034001850 con 0x7fb0440b1f50 2024-04-25T10:27:29.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 >> v1:172.21.15.87:6789/0 conn(0x7fb0440b1f50 legacy=0x7fb04406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028003a30 0x7fb028003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb028004810 0x7fb028004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028005240 0x7fb02800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02800c170 con 0x7fb028005240 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02800c320 con 0x7fb028004810 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02800c520 con 0x7fb028003a30 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049000640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028005240 0x7fb02800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb043e00640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028003a30 0x7fb028003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb043400640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb028004810 0x7fb028004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049000640 1 -- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028003a30 msgr2=0x7fb028003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049000640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028003a30 0x7fb028003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049000640 1 -- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb028004810 msgr2=0x7fb028004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049000640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb028004810 0x7fb028004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.277+0000 7fb049000640 1 -- 172.21.15.87:0/85031942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb028003530 con 0x7fb028005240 2024-04-25T10:27:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049000640 1 -- 172.21.15.87:0/85031942 reap_dead start 2024-04-25T10:27:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049000640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028005240 0x7fb02800b9d0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009f90 tx=0x7fb03c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac81701e19d6c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0b2070 con 0x7fb028005240 2024-04-25T10:27:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c0a5c90 con 0x7fb028005240 2024-04-25T10:27:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb042a00640 1 -- 172.21.15.87:0/85031942 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c009c10 con 0x7fb028005240 2024-04-25T10:27:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 -- 172.21.15.87:0/85031942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028005240 msgr2=0x7fb02800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028005240 0x7fb02800b9d0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009f90 tx=0x7fb03c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:27:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 -- 172.21.15.87:0/85031942 shutdown_connections 2024-04-25T10:27:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 --2- 172.21.15.87:0/85031942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028005240 0x7fb02800b9d0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 -- 172.21.15.87:0/85031942 >> 172.21.15.87:0/85031942 conn(0x7fb0440b32b0 msgr2=0x7fb0440b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 -- 172.21.15.87:0/85031942 shutdown_connections 2024-04-25T10:27:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 -- 172.21.15.87:0/85031942 wait complete. 2024-04-25T10:27:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 Processor -- start 2024-04-25T10:27:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.281+0000 7fb049a00640 1 -- start start 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028005240 0x7fb0441f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb04415cec0 0x7fb0440aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0440ccfd0 0x7fb0440cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b3dd0 con 0x7fb0440ccfd0 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04406b680 con 0x7fb04415cec0 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b3f50 con 0x7fb028005240 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0440ccfd0 0x7fb0440cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0440ccfd0 0x7fb0440cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44362/0 (socket says 172.21.15.87:44362) 2024-04-25T10:27:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 -- 172.21.15.87:0/1362639331 learned_addr learned my addr 172.21.15.87:0/1362639331 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb043400640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb04415cec0 0x7fb0440aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb043e00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028005240 0x7fb0441f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 -- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028005240 msgr2=0x7fb0441f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028005240 0x7fb0441f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 -- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb04415cec0 msgr2=0x7fb0440aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb04415cec0 0x7fb0440aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 -- 172.21.15.87:0/1362639331 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440c85e0 con 0x7fb0440ccfd0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb043400640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb04415cec0 0x7fb0440aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049000640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0440ccfd0 0x7fb0440cd430 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fb03c01cab0 tx=0x7fb03c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb03be00640 1 -- 172.21.15.87:0/1362639331 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0bc070 con 0x7fb0440ccfd0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb03be00640 1 -- 172.21.15.87:0/1362639331 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c0b63f0 con 0x7fb0440ccfd0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440ae8c0 con 0x7fb0440ccfd0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb03be00640 1 -- 172.21.15.87:0/1362639331 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0b2040 con 0x7fb0440ccfd0 2024-04-25T10:27:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0441f34d0 con 0x7fb0440ccfd0 2024-04-25T10:27:29.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.285+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044075120 con 0x7fb0440ccfd0 2024-04-25T10:27:29.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.289+0000 7fb03be00640 1 -- 172.21.15.87:0/1362639331 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb03c0bf070 con 0x7fb0440ccfd0 2024-04-25T10:27:29.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.289+0000 7fb03be00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb024066f60 0x7fb024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.289+0000 7fb03be00640 1 -- 172.21.15.87:0/1362639331 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb03c11f5f0 con 0x7fb0440ccfd0 2024-04-25T10:27:29.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.289+0000 7fb043e00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb024066f60 0x7fb024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.289+0000 7fb043e00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb024066f60 0x7fb024069420 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb034000c10 tx=0x7fb034009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.293+0000 7fb03be00640 1 -- 172.21.15.87:0/1362639331 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb03c016990 con 0x7fb0440ccfd0 2024-04-25T10:27:29.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.445+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0440cc970 con 0x7fb0440ccfd0 2024-04-25T10:27:29.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.445+0000 7fb03be00640 1 -- 172.21.15.87:0/1362639331 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb03c0efeb0 con 0x7fb0440ccfd0 2024-04-25T10:27:29.445 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb024066f60 msgr2=0x7fb024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb024066f60 0x7fb024069420 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb034000c10 tx=0x7fb034009040 comp rx=0 tx=0).stop 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0440ccfd0 msgr2=0x7fb0440cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0440ccfd0 0x7fb0440cd430 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fb03c01cab0 tx=0x7fb03c009750 comp rx=0 tx=0).stop 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 shutdown_connections 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb024066f60 0x7fb024069420 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0440ccfd0 0x7fb0440cd430 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb04415cec0 0x7fb0440aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 --2- 172.21.15.87:0/1362639331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb028005240 0x7fb0441f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 >> 172.21.15.87:0/1362639331 conn(0x7fb0440b32b0 msgr2=0x7fb04415dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 shutdown_connections 2024-04-25T10:27:29.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.449+0000 7fb049a00640 1 -- 172.21.15.87:0/1362639331 wait complete. 2024-04-25T10:27:29.466 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:27:29.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.605+0000 7f2629600640 1 Processor -- start 2024-04-25T10:27:29.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.605+0000 7f2629600640 1 -- start start 2024-04-25T10:27:29.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2629600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f262415b750 0x7f26240cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2629600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26240cd590 0x7f26240cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2629600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26240b2460 con 0x7f26240b3f80 2024-04-25T10:27:29.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2629600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240b2d70 con 0x7f26240cd590 2024-04-25T10:27:29.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2629600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2624070fb0 con 0x7f262415b750 2024-04-25T10:27:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2623400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26240cd590 0x7f26240cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2623e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f262415b750 0x7f26240cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2628c00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f26240b3f80 0x7f26240b3200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:57998/0 (socket says 172.21.15.87:57998) 2024-04-25T10:27:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2628c00640 1 -- 172.21.15.87:0/1148348913 learned_addr learned my addr 172.21.15.87:0/1148348913 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2623400640 1 -- 172.21.15.87:0/1148348913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f262415b750 msgr2=0x7f26240cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2623400640 1 --2- 172.21.15.87:0/1148348913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f262415b750 0x7f26240cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2623400640 1 -- 172.21.15.87:0/1148348913 >> v1:172.21.15.169:6789/0 conn(0x7f26240b3f80 legacy=0x7f26240b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.609+0000 7f2623400640 1 -- 172.21.15.87:0/1148348913 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240b2720 con 0x7f26240cd590 2024-04-25T10:27:29.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2623400640 1 --2- 172.21.15.87:0/1148348913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 0x7f26240cd9b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f26240b2ef0 tx=0x7f261001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d8eaaa28cc3ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2622a00640 1 -- 172.21.15.87:0/1148348913 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100ab430 con 0x7f26240cd590 2024-04-25T10:27:29.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2622a00640 1 -- 172.21.15.87:0/1148348913 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26100b0070 con 0x7f26240cd590 2024-04-25T10:27:29.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2622a00640 1 -- 172.21.15.87:0/1148348913 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100ab7f0 con 0x7f26240cd590 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 -- 172.21.15.87:0/1148348913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 msgr2=0x7f26240cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 --2- 172.21.15.87:0/1148348913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 0x7f26240cd9b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f26240b2ef0 tx=0x7f261001c470 comp rx=0 tx=0).stop 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 -- 172.21.15.87:0/1148348913 shutdown_connections 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 --2- 172.21.15.87:0/1148348913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 0x7f26240cd9b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 --2- 172.21.15.87:0/1148348913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f262415b750 0x7f26240cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 -- 172.21.15.87:0/1148348913 >> 172.21.15.87:0/1148348913 conn(0x7f26240743f0 msgr2=0x7f26240669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 -- 172.21.15.87:0/1148348913 shutdown_connections 2024-04-25T10:27:29.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 -- 172.21.15.87:0/1148348913 wait complete. 2024-04-25T10:27:29.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 Processor -- start 2024-04-25T10:27:29.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.613+0000 7f2629600640 1 -- start start 2024-04-25T10:27:29.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26240b3f80 0x7f2624153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 0x7f26241fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262415b750 0x7f26241faa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240b47a0 con 0x7f262415b750 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624070b60 con 0x7f26240cd590 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262406da10 con 0x7f26240b3f80 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262415b750 0x7f26241faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262415b750 0x7f26241faa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44364/0 (socket says 172.21.15.87:44364) 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 -- 172.21.15.87:0/3911011544 learned_addr learned my addr 172.21.15.87:0/3911011544 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2623400640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 0x7f26241fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2623e00640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26240b3f80 0x7f2624153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 -- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26240b3f80 msgr2=0x7f2624153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26240b3f80 0x7f2624153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 -- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 msgr2=0x7f26241fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 0x7f26241fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 -- 172.21.15.87:0/3911011544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262414ea60 con 0x7f262415b750 2024-04-25T10:27:29.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2628c00640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262415b750 0x7f26241faa10 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c430 tx=0x7f261c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f261be00640 1 -- 172.21.15.87:0/3911011544 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c017070 con 0x7f262415b750 2024-04-25T10:27:29.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f261be00640 1 -- 172.21.15.87:0/3911011544 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c005cf0 con 0x7f262415b750 2024-04-25T10:27:29.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262414ed40 con 0x7f262415b750 2024-04-25T10:27:29.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f261be00640 1 -- 172.21.15.87:0/3911011544 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0072d0 con 0x7f262415b750 2024-04-25T10:27:29.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262414f020 con 0x7f262415b750 2024-04-25T10:27:29.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624075120 con 0x7f262415b750 2024-04-25T10:27:29.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.617+0000 7f261be00640 1 -- 172.21.15.87:0/3911011544 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f261c01a070 con 0x7f262415b750 2024-04-25T10:27:29.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.621+0000 7f261be00640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2604066f60 0x7f2604069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.621+0000 7f261be00640 1 -- 172.21.15.87:0/3911011544 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f261c0843c0 con 0x7f262415b750 2024-04-25T10:27:29.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.621+0000 7f2623e00640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2604066f60 0x7f2604069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.621+0000 7f2623e00640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2604066f60 0x7f2604069420 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2614011340 tx=0x7f261400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.625+0000 7f261be00640 1 -- 172.21.15.87:0/3911011544 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f261c054c80 con 0x7f262415b750 2024-04-25T10:27:29.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f262415f130 con 0x7f262415b750 2024-04-25T10:27:29.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f261be00640 1 -- 172.21.15.87:0/3911011544 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) v1 ==== 100+0+11 (secure 0 0 0) 0x7f261c058d20 con 0x7f262415b750 2024-04-25T10:27:29.777 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2604066f60 msgr2=0x7f2604069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2604066f60 0x7f2604069420 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2614011340 tx=0x7f261400b040 comp rx=0 tx=0).stop 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262415b750 msgr2=0x7f26241faa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262415b750 0x7f26241faa10 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c430 tx=0x7f261c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 shutdown_connections 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2604066f60 0x7f2604069420 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262415b750 0x7f26241faa10 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26240cd590 0x7f26241fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 --2- 172.21.15.87:0/3911011544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26240b3f80 0x7f2624153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 >> 172.21.15.87:0/3911011544 conn(0x7f26240743f0 msgr2=0x7f26240ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 shutdown_connections 2024-04-25T10:27:29.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.781+0000 7f2629600640 1 -- 172.21.15.87:0/3911011544 wait complete. 2024-04-25T10:27:29.799 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:29.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe3600640 1 Processor -- start 2024-04-25T10:27:29.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe3600640 1 -- start start 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe3600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fdc15e050 0x7f0fdc0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe3600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fdc0b13e0 0x7f0fdc0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe3600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fdc0b1af0 con 0x7f0fdc0d1820 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe3600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fdc0b1dd0 con 0x7f0fdc15e050 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe3600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fdc0b20b0 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fd3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fdc15e050 0x7f0fdc0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe0c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f0fdc0d1820 0x7f0fdc0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57954/0 (socket says 172.21.15.87:57954) 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe0c00640 1 -- 172.21.15.87:0/270771964 learned_addr learned my addr 172.21.15.87:0/270771964 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fd3e00640 1 -- 172.21.15.87:0/270771964 >> v2:172.21.15.169:3300/0 conn(0x7f0fdc0b13e0 msgr2=0x7f0fdc0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:29.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe1600640 1 --2- 172.21.15.87:0/270771964 >> v2:172.21.15.169:3300/0 conn(0x7f0fdc0b13e0 0x7f0fdc0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fd3e00640 1 --2- 172.21.15.87:0/270771964 >> v2:172.21.15.169:3300/0 conn(0x7f0fdc0b13e0 0x7f0fdc0cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fd3e00640 1 -- 172.21.15.87:0/270771964 >> v1:172.21.15.87:6789/0 conn(0x7f0fdc0d1820 legacy=0x7f0fdc0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fdbe00640 1 -- 172.21.15.87:0/270771964 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173258153 0 0) 0x7f0fdc0b1af0 con 0x7f0fdc0d1820 2024-04-25T10:27:29.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fd3e00640 1 -- 172.21.15.87:0/270771964 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0b2390 con 0x7f0fdc15e050 2024-04-25T10:27:29.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.937+0000 7f0fe1600640 1 --2- 172.21.15.87:0/270771964 >> v2:172.21.15.169:3300/0 conn(0x7f0fdc0b13e0 0x7f0fdc0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:29.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fd3e00640 1 --2- 172.21.15.87:0/270771964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15e050 0x7f0fdc0b0ea0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0b0e20 tx=0x7f0fc401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2730673818d029af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fdbe00640 1 -- 172.21.15.87:0/270771964 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40b1070 con 0x7f0fdc15e050 2024-04-25T10:27:29.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fdbe00640 1 -- 172.21.15.87:0/270771964 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc40ac5f0 con 0x7f0fdc15e050 2024-04-25T10:27:29.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fdbe00640 1 -- 172.21.15.87:0/270771964 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40ac9c0 con 0x7f0fdc15e050 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 -- 172.21.15.87:0/270771964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15e050 msgr2=0x7f0fdc0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 --2- 172.21.15.87:0/270771964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15e050 0x7f0fdc0b0ea0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0b0e20 tx=0x7f0fc401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 -- 172.21.15.87:0/270771964 shutdown_connections 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 --2- 172.21.15.87:0/270771964 >> v2:172.21.15.169:3300/0 conn(0x7f0fdc0b13e0 0x7f0fdc0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 --2- 172.21.15.87:0/270771964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc15e050 0x7f0fdc0b0ea0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 -- 172.21.15.87:0/270771964 >> 172.21.15.87:0/270771964 conn(0x7f0fdc09a9d0 msgr2=0x7f0fdc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 -- 172.21.15.87:0/270771964 shutdown_connections 2024-04-25T10:27:29.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.941+0000 7f0fe3600640 1 -- 172.21.15.87:0/270771964 wait complete. 2024-04-25T10:27:29.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 Processor -- start 2024-04-25T10:27:29.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 -- start start 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc0b13e0 0x7f0fdc14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0d1820 0x7f0fdc1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc15e050 0x7f0fdc158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0d0dd0 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0d0c50 con 0x7f0fdc0d1820 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0d0f50 con 0x7f0fdc15e050 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc0b13e0 0x7f0fdc14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc0b13e0 0x7f0fdc14aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44374/0 (socket says 172.21.15.87:44374) 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 -- 172.21.15.87:0/1263560695 learned_addr learned my addr 172.21.15.87:0/1263560695 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fd3e00640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0d1820 0x7f0fdc1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe1600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc15e050 0x7f0fdc158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 -- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc15e050 msgr2=0x7f0fdc158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc15e050 0x7f0fdc158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 -- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0d1820 msgr2=0x7f0fdc1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0d1820 0x7f0fdc1580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 -- 172.21.15.87:0/1263560695 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc14af80 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe1600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc15e050 0x7f0fdc158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe0c00640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc0b13e0 0x7f0fdc14aa40 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0011f0 tx=0x7f0fcc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fd9600640 1 -- 172.21.15.87:0/1263560695 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc018070 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fd9600640 1 -- 172.21.15.87:0/1263560695 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc0074e0 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fd9600640 1 -- 172.21.15.87:0/1263560695 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc004a10 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc15ada0 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc15b360 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.945+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc15bb30 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.949+0000 7f0fd9600640 1 -- 172.21.15.87:0/1263560695 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fcc01b070 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.949+0000 7f0fd9600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066f60 0x7f0fb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.949+0000 7f0fd9600640 1 -- 172.21.15.87:0/1263560695 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f0fcc084490 con 0x7f0fdc0b13e0 2024-04-25T10:27:29.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.949+0000 7f0fd3e00640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066f60 0x7f0fb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.949+0000 7f0fd3e00640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066f60 0x7f0fb8069420 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40ae000 tx=0x7f0fc4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:29.953+0000 7f0fd9600640 1 -- 172.21.15.87:0/1263560695 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0fcc0553a0 con 0x7f0fdc0b13e0 2024-04-25T10:27:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.137+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fdc14b2a0 con 0x7f0fdc0b13e0 2024-04-25T10:27:30.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.137+0000 7f0fd9600640 1 -- 172.21.15.87:0/1263560695 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f0fcc054ca0 con 0x7f0fdc0b13e0 2024-04-25T10:27:30.137 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:30.137 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:23.173736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:27:30.137 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 143 2024-04-25T10:27:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066f60 msgr2=0x7f0fb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066f60 0x7f0fb8069420 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40ae000 tx=0x7f0fc4008040 comp rx=0 tx=0).stop 2024-04-25T10:27:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc0b13e0 msgr2=0x7f0fdc14aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc0b13e0 0x7f0fdc14aa40 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0011f0 tx=0x7f0fcc00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 shutdown_connections 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f0fb8066f60 0x7f0fb8069420 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fdc15e050 0x7f0fdc158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fdc0d1820 0x7f0fdc1580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 --2- 172.21.15.87:0/1263560695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fdc0b13e0 0x7f0fdc14aa40 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 >> 172.21.15.87:0/1263560695 conn(0x7f0fdc09a9d0 msgr2=0x7f0fdc15d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 shutdown_connections 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.141+0000 7f0fe3600640 1 -- 172.21.15.87:0/1263560695 wait complete. 2024-04-25T10:27:30.158 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:30.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ada200640 1 Processor -- start 2024-04-25T10:27:30.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ada200640 1 -- start start 2024-04-25T10:27:30.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ada200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ad40ccb20 0x7f8ad40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ada200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415bdc0 0x7f8ad40d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ada200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad40ccfb0 con 0x7f8ad415b9e0 2024-04-25T10:27:30.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ada200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40cd290 con 0x7f8ad40ccb20 2024-04-25T10:27:30.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ada200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40cd570 con 0x7f8ad415bdc0 2024-04-25T10:27:30.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415bdc0 0x7f8ad40d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad3400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8ad415b9e0 0x7f8ad40cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:58008/0 (socket says 172.21.15.87:58008) 2024-04-25T10:27:30.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad3400640 1 -- 172.21.15.87:0/303737389 learned_addr learned my addr 172.21.15.87:0/303737389 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:30.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad2a00640 1 --2- 172.21.15.87:0/303737389 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40ccb20 0x7f8ad40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad2000640 1 -- 172.21.15.87:0/303737389 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673991277 0 0) 0x7f8ad40ccfb0 con 0x7f8ad415b9e0 2024-04-25T10:27:30.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad2000640 1 -- 172.21.15.87:0/303737389 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab8003530 con 0x7f8ad415b9e0 2024-04-25T10:27:30.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad3e00640 1 -- 172.21.15.87:0/303737389 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40ccb20 msgr2=0x7f8ad40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:30.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad3e00640 1 --2- 172.21.15.87:0/303737389 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40ccb20 0x7f8ad40d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad3e00640 1 -- 172.21.15.87:0/303737389 >> v1:172.21.15.169:6789/0 conn(0x7f8ad415b9e0 legacy=0x7f8ad40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad3e00640 1 -- 172.21.15.87:0/303737389 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40cd850 con 0x7f8ad415bdc0 2024-04-25T10:27:30.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.297+0000 7f8ad2a00640 1 --2- 172.21.15.87:0/303737389 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40ccb20 0x7f8ad40d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ad3e00640 1 --2- 172.21.15.87:0/303737389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415bdc0 0x7f8ad40d1a60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0097c0 tx=0x7f8acc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c41c249d1a89c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ad2000640 1 -- 172.21.15.87:0/303737389 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0b0070 con 0x7f8ad415bdc0 2024-04-25T10:27:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ad2000640 1 -- 172.21.15.87:0/303737389 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc0ab600 con 0x7f8ad415bdc0 2024-04-25T10:27:30.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ad2000640 1 -- 172.21.15.87:0/303737389 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0ab9b0 con 0x7f8ad415bdc0 2024-04-25T10:27:30.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 -- 172.21.15.87:0/303737389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415bdc0 msgr2=0x7f8ad40d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 --2- 172.21.15.87:0/303737389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415bdc0 0x7f8ad40d1a60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0097c0 tx=0x7f8acc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 -- 172.21.15.87:0/303737389 shutdown_connections 2024-04-25T10:27:30.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 --2- 172.21.15.87:0/303737389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415bdc0 0x7f8ad40d1a60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 --2- 172.21.15.87:0/303737389 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40ccb20 0x7f8ad40d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 -- 172.21.15.87:0/303737389 >> 172.21.15.87:0/303737389 conn(0x7f8ad409a9d0 msgr2=0x7f8ad409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:30.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 -- 172.21.15.87:0/303737389 shutdown_connections 2024-04-25T10:27:30.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.301+0000 7f8ada200640 1 -- 172.21.15.87:0/303737389 wait complete. 2024-04-25T10:27:30.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 Processor -- start 2024-04-25T10:27:30.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 -- start start 2024-04-25T10:27:30.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415b9e0 0x7f8ad40cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad415bdc0 0x7f8ad41403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad40c89d0 0x7f8ad41408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40c65b0 con 0x7f8ad415bdc0 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40c6430 con 0x7f8ad40c89d0 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40c6730 con 0x7f8ad415b9e0 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad415bdc0 0x7f8ad41403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad415bdc0 0x7f8ad41403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44388/0 (socket says 172.21.15.87:44388) 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 -- 172.21.15.87:0/2361262709 learned_addr learned my addr 172.21.15.87:0/2361262709 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad3e00640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad40c89d0 0x7f8ad41408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad3400640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415b9e0 0x7f8ad40cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 -- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415b9e0 msgr2=0x7f8ad40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415b9e0 0x7f8ad40cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 -- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad40c89d0 msgr2=0x7f8ad41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad40c89d0 0x7f8ad41408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 -- 172.21.15.87:0/2361262709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad414e090 con 0x7f8ad415bdc0 2024-04-25T10:27:30.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad3400640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415b9e0 0x7f8ad40cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:30.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ad2a00640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad415bdc0 0x7f8ad41403a0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000e9d0 tx=0x7f8ac000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8acb400640 1 -- 172.21.15.87:0/2361262709 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0017070 con 0x7f8ad415bdc0 2024-04-25T10:27:30.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8acb400640 1 -- 172.21.15.87:0/2361262709 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac000ca60 con 0x7f8ad415bdc0 2024-04-25T10:27:30.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad40cb090 con 0x7f8ad415bdc0 2024-04-25T10:27:30.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8acb400640 1 -- 172.21.15.87:0/2361262709 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac000ce50 con 0x7f8ad415bdc0 2024-04-25T10:27:30.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.305+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad40cb650 con 0x7f8ad415bdc0 2024-04-25T10:27:30.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.309+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad404ef30 con 0x7f8ad415bdc0 2024-04-25T10:27:30.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.309+0000 7f8acb400640 1 -- 172.21.15.87:0/2361262709 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ac001a070 con 0x7f8ad415bdc0 2024-04-25T10:27:30.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.309+0000 7f8acb400640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ab4067100 0x7f8ab40695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.309+0000 7f8acb400640 1 -- 172.21.15.87:0/2361262709 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f8ac0085070 con 0x7f8ad415bdc0 2024-04-25T10:27:30.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.309+0000 7f8ad3400640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ab4067100 0x7f8ab40695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.309+0000 7f8ad3400640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ab4067100 0x7f8ab40695c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4001590 tx=0x7f8ac4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.313+0000 7f8acb400640 1 -- 172.21.15.87:0/2361262709 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ac0055930 con 0x7f8ad415bdc0 2024-04-25T10:27:30.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.469+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ad40c6fc0 con 0x7f8ad415bdc0 2024-04-25T10:27:30.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.469+0000 7f8acb400640 1 -- 172.21.15.87:0/2361262709 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+17034 (secure 0 0 0) 0x7f8ac00599d0 con 0x7f8ad415bdc0 2024-04-25T10:27:30.469 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:30.469 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":190,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:27:08.279567+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:07.509820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:08.125891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]},{"pgid":"29.23","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6836/997618220":"2024-04-26T10:26:20.143429+0000","172.21.15.87:6834/997618220":"2024-04-26T10:26:20.143429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ab4067100 msgr2=0x7f8ab40695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ab4067100 0x7f8ab40695c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4001590 tx=0x7f8ac4009040 comp rx=0 tx=0).stop 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad415bdc0 msgr2=0x7f8ad41403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad415bdc0 0x7f8ad41403a0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000e9d0 tx=0x7f8ac000ee90 comp rx=0 tx=0).stop 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 shutdown_connections 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8ab4067100 0x7f8ab40695c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad40c89d0 0x7f8ad41408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad415bdc0 0x7f8ad41403a0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 --2- 172.21.15.87:0/2361262709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad415b9e0 0x7f8ad40cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 >> 172.21.15.87:0/2361262709 conn(0x7f8ad409a9d0 msgr2=0x7f8ad40c77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 shutdown_connections 2024-04-25T10:27:30.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.473+0000 7f8ada200640 1 -- 172.21.15.87:0/2361262709 wait complete. 2024-04-25T10:27:30.490 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:27:30.491 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:30.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e7a00640 1 Processor -- start 2024-04-25T10:27:30.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e7a00640 1 -- start start 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e015bc40 0x7ff9e006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e7a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9e00cda10 0x7ff9e00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e7a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00b2770 con 0x7ff9e00cf930 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e7a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e006c670 con 0x7ff9e00cda10 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e7a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e006c950 con 0x7ff9e015bc40 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9d7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9e00cda10 0x7ff9e00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9d7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9e00cda10 0x7ff9e00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44394/0 (socket says 172.21.15.87:44394) 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9d7e00640 1 -- 172.21.15.87:0/4214319683 learned_addr learned my addr 172.21.15.87:0/4214319683 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:30.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9d7e00640 1 -- 172.21.15.87:0/4214319683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e015bc40 msgr2=0x7ff9e006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:30.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9e5000640 1 --2- 172.21.15.87:0/4214319683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e015bc40 0x7ff9e006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9d7e00640 1 --2- 172.21.15.87:0/4214319683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e015bc40 0x7ff9e006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9d7e00640 1 -- 172.21.15.87:0/4214319683 >> v1:172.21.15.169:6789/0 conn(0x7ff9e00cf930 legacy=0x7ff9e015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:30.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.629+0000 7ff9d7e00640 1 -- 172.21.15.87:0/4214319683 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e006cc30 con 0x7ff9e00cda10 2024-04-25T10:27:30.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9d7e00640 1 --2- 172.21.15.87:0/4214319683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e00cda10 0x7ff9e00b4610 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2b3edc19aed817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9dfe00640 1 -- 172.21.15.87:0/4214319683 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80ab5f0 con 0x7ff9e00cda10 2024-04-25T10:27:30.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9dfe00640 1 -- 172.21.15.87:0/4214319683 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c80b0070 con 0x7ff9e00cda10 2024-04-25T10:27:30.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9dfe00640 1 -- 172.21.15.87:0/4214319683 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80ab9b0 con 0x7ff9e00cda10 2024-04-25T10:27:30.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 -- 172.21.15.87:0/4214319683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e00cda10 msgr2=0x7ff9e00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/4214319683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e00cda10 0x7ff9e00b4610 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c801c520 comp rx=0 tx=0).stop 2024-04-25T10:27:30.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 -- 172.21.15.87:0/4214319683 shutdown_connections 2024-04-25T10:27:30.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/4214319683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e00cda10 0x7ff9e00b4610 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/4214319683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e015bc40 0x7ff9e006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 -- 172.21.15.87:0/4214319683 >> 172.21.15.87:0/4214319683 conn(0x7ff9e0066b70 msgr2=0x7ff9e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:30.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 -- 172.21.15.87:0/4214319683 shutdown_connections 2024-04-25T10:27:30.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.633+0000 7ff9e7a00640 1 -- 172.21.15.87:0/4214319683 wait complete. 2024-04-25T10:27:30.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 Processor -- start 2024-04-25T10:27:30.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 -- start start 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9e00cda10 0x7ff9e014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e00cf930 0x7ff9e0150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e015bc40 0x7ff9e014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0168b50 con 0x7ff9e015bc40 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01689d0 con 0x7ff9e00cda10 2024-04-25T10:27:30.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0168cd0 con 0x7ff9e00cf930 2024-04-25T10:27:30.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e015bc40 0x7ff9e014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e015bc40 0x7ff9e014e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44410/0 (socket says 172.21.15.87:44410) 2024-04-25T10:27:30.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 -- 172.21.15.87:0/3900689606 learned_addr learned my addr 172.21.15.87:0/3900689606 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:30.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5000640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9e00cda10 0x7ff9e014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9d7e00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e00cf930 0x7ff9e0150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 -- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e00cf930 msgr2=0x7ff9e0150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e00cf930 0x7ff9e0150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 -- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9e00cda10 msgr2=0x7ff9e014ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9e00cda10 0x7ff9e014ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 -- 172.21.15.87:0/3900689606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e014e9f0 con 0x7ff9e015bc40 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5000640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9e00cda10 0x7ff9e014ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9d7e00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e00cf930 0x7ff9e0150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e5a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e015bc40 0x7ff9e014e4b0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c780 tx=0x7ff9d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9dd600640 1 -- 172.21.15.87:0/3900689606 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8004b70 con 0x7ff9e015bc40 2024-04-25T10:27:30.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9dd600640 1 -- 172.21.15.87:0/3900689606 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d8004d10 con 0x7ff9e015bc40 2024-04-25T10:27:30.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e014f210 con 0x7ff9e015bc40 2024-04-25T10:27:30.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9dd600640 1 -- 172.21.15.87:0/3900689606 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800aa90 con 0x7ff9e015bc40 2024-04-25T10:27:30.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e014f7d0 con 0x7ff9e015bc40 2024-04-25T10:27:30.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.637+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e004ef30 con 0x7ff9e015bc40 2024-04-25T10:27:30.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.641+0000 7ff9dd600640 1 -- 172.21.15.87:0/3900689606 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9d800ac30 con 0x7ff9e015bc40 2024-04-25T10:27:30.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.641+0000 7ff9dd600640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9bc066f60 0x7ff9bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.641+0000 7ff9dd600640 1 -- 172.21.15.87:0/3900689606 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7ff9d8085400 con 0x7ff9e015bc40 2024-04-25T10:27:30.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.641+0000 7ff9e5000640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9bc066f60 0x7ff9bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.641+0000 7ff9e5000640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9bc066f60 0x7ff9bc069420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00b3540 tx=0x7ff9d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.645+0000 7ff9dd600640 1 -- 172.21.15.87:0/3900689606 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9d8020070 con 0x7ff9e015bc40 2024-04-25T10:27:30.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.829+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9e006b390 con 0x7ff9e015bc40 2024-04-25T10:27:30.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.829+0000 7ff9dd600640 1 -- 172.21.15.87:0/3900689606 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4960 (secure 0 0 0) 0x7ff9d8056280 con 0x7ff9e015bc40 2024-04-25T10:27:30.827 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:30.827 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:23.173736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:27:30.827 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 143 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.829+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9bc066f60 msgr2=0x7ff9bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.829+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9bc066f60 0x7ff9bc069420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00b3540 tx=0x7ff9d0008040 comp rx=0 tx=0).stop 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e015bc40 msgr2=0x7ff9e014e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e015bc40 0x7ff9e014e4b0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c780 tx=0x7ff9d800cc40 comp rx=0 tx=0).stop 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 shutdown_connections 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff9bc066f60 0x7ff9bc069420 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9e015bc40 0x7ff9e014e4b0 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9e00cf930 0x7ff9e0150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 --2- 172.21.15.87:0/3900689606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9e00cda10 0x7ff9e014ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 >> 172.21.15.87:0/3900689606 conn(0x7ff9e0066b70 msgr2=0x7ff9e0066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:30.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 shutdown_connections 2024-04-25T10:27:30.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.833+0000 7ff9e7a00640 1 -- 172.21.15.87:0/3900689606 wait complete. 2024-04-25T10:27:30.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:30.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb947e00640 1 Processor -- start 2024-04-25T10:27:30.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb947e00640 1 -- start start 2024-04-25T10:27:30.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb947e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9400af9c0 0x7fb94006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb947e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9400afda0 0x7fb9400cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb947e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94006f200 con 0x7fb940071f50 2024-04-25T10:27:30.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb947e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94006d380 con 0x7fb9400afda0 2024-04-25T10:27:30.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb947e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400ccc10 con 0x7fb9400af9c0 2024-04-25T10:27:30.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb944a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9400afda0 0x7fb9400cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb945400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9400af9c0 0x7fb94006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb945e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb940071f50 0x7fb9400669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:58034/0 (socket says 172.21.15.87:58034) 2024-04-25T10:27:30.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb945e00640 1 -- 172.21.15.87:0/2547490817 learned_addr learned my addr 172.21.15.87:0/2547490817 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:30.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb944a00640 1 -- 172.21.15.87:0/2547490817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9400af9c0 msgr2=0x7fb94006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:30.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb944a00640 1 --2- 172.21.15.87:0/2547490817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9400af9c0 0x7fb94006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb944a00640 1 -- 172.21.15.87:0/2547490817 >> v1:172.21.15.169:6789/0 conn(0x7fb940071f50 legacy=0x7fb9400669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.985+0000 7fb944a00640 1 -- 172.21.15.87:0/2547490817 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400ccef0 con 0x7fb9400afda0 2024-04-25T10:27:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb944a00640 1 --2- 172.21.15.87:0/2547490817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 0x7fb9400cc3a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0097c0 tx=0x7fb92c01c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468b0848896ecd6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb93fe00640 1 -- 172.21.15.87:0/2547490817 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0b0070 con 0x7fb9400afda0 2024-04-25T10:27:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb93fe00640 1 -- 172.21.15.87:0/2547490817 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c004230 con 0x7fb9400afda0 2024-04-25T10:27:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb93fe00640 1 -- 172.21.15.87:0/2547490817 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c003620 con 0x7fb9400afda0 2024-04-25T10:27:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 -- 172.21.15.87:0/2547490817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 msgr2=0x7fb9400cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 --2- 172.21.15.87:0/2547490817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 0x7fb9400cc3a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0097c0 tx=0x7fb92c01c350 comp rx=0 tx=0).stop 2024-04-25T10:27:30.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 -- 172.21.15.87:0/2547490817 shutdown_connections 2024-04-25T10:27:30.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 --2- 172.21.15.87:0/2547490817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 0x7fb9400cc3a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 --2- 172.21.15.87:0/2547490817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9400af9c0 0x7fb94006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 -- 172.21.15.87:0/2547490817 >> 172.21.15.87:0/2547490817 conn(0x7fb9400b2410 msgr2=0x7fb9400b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:30.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 -- 172.21.15.87:0/2547490817 shutdown_connections 2024-04-25T10:27:30.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.989+0000 7fb947e00640 1 -- 172.21.15.87:0/2547490817 wait complete. 2024-04-25T10:27:30.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 Processor -- start 2024-04-25T10:27:30.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 -- start start 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb940071f50 0x7fb9400cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9400af9c0 0x7fb940159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 0x7fb94015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb940067230 con 0x7fb9400af9c0 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400670b0 con 0x7fb9400afda0 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9401600d0 con 0x7fb940071f50 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9400af9c0 0x7fb940159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9400af9c0 0x7fb940159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44424/0 (socket says 172.21.15.87:44424) 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 -- 172.21.15.87:0/26027948 learned_addr learned my addr 172.21.15.87:0/26027948 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb945400640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb940071f50 0x7fb9400cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb945e00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 0x7fb94015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 -- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb940071f50 msgr2=0x7fb9400cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb940071f50 0x7fb9400cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 -- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 msgr2=0x7fb94015a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:30.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 0x7fb94015a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:30.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 -- 172.21.15.87:0/26027948 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400ae7d0 con 0x7fb9400af9c0 2024-04-25T10:27:30.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb944a00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9400af9c0 0x7fb940159df0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ae040 tx=0x7fb92c0190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb93d600640 1 -- 172.21.15.87:0/26027948 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0bb020 con 0x7fb9400af9c0 2024-04-25T10:27:30.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb93d600640 1 -- 172.21.15.87:0/26027948 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c003dd0 con 0x7fb9400af9c0 2024-04-25T10:27:30.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb93d600640 1 -- 172.21.15.87:0/26027948 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0b0040 con 0x7fb9400af9c0 2024-04-25T10:27:30.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9400aeab0 con 0x7fb9400af9c0 2024-04-25T10:27:30.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.993+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9400af070 con 0x7fb9400af9c0 2024-04-25T10:27:30.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.997+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94004ef30 con 0x7fb9400af9c0 2024-04-25T10:27:30.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.997+0000 7fb93d600640 1 -- 172.21.15.87:0/26027948 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb92c0be020 con 0x7fb9400af9c0 2024-04-25T10:27:30.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.997+0000 7fb93d600640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb920066f60 0x7fb920069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:30.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.997+0000 7fb93d600640 1 -- 172.21.15.87:0/26027948 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fb92c11f540 con 0x7fb9400af9c0 2024-04-25T10:27:30.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.997+0000 7fb945400640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb920066f60 0x7fb920069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:30.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:30.997+0000 7fb945400640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb920066f60 0x7fb920069420 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb934009b60 tx=0x7fb934009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:30.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.001+0000 7fb93d600640 1 -- 172.21.15.87:0/26027948 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb92c0c5030 con 0x7fb9400af9c0 2024-04-25T10:27:31.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.185+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb94015f1d0 con 0x7fb9400af9c0 2024-04-25T10:27:31.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.185+0000 7fb93d600640 1 -- 172.21.15.87:0/26027948 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fb92c016090 con 0x7fb9400af9c0 2024-04-25T10:27:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:31.185 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:09.137144+0000","modified":"2024-04-25T10:26:23.173736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":188,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11520":{"gid":11520,"name":"a","rank":0,"incarnation":140,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6836/3730332371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3730332371},{"type":"v1","addr":"172.21.15.87:6836","nonce":3730332371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:27:31.185 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 143 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb920066f60 msgr2=0x7fb920069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb920066f60 0x7fb920069420 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb934009b60 tx=0x7fb934009190 comp rx=0 tx=0).stop 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9400af9c0 msgr2=0x7fb940159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9400af9c0 0x7fb940159df0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ae040 tx=0x7fb92c0190f0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 shutdown_connections 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb920066f60 0x7fb920069420 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9400afda0 0x7fb94015a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9400af9c0 0x7fb940159df0 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 --2- 172.21.15.87:0/26027948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb940071f50 0x7fb9400cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 >> 172.21.15.87:0/26027948 conn(0x7fb9400b2410 msgr2=0x7fb9400b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:31.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 shutdown_connections 2024-04-25T10:27:31.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.189+0000 7fb947e00640 1 -- 172.21.15.87:0/26027948 wait complete. 2024-04-25T10:27:31.206 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:31.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 Processor -- start 2024-04-25T10:27:31.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 -- start start 2024-04-25T10:27:31.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef881550b0 0x7fef880cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef880cfe70 0x7fef8815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef8815b740 0x7fef8806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef88071050 con 0x7fef880cfe70 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef880712f0 con 0x7fef8815b740 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef90200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef880715d0 con 0x7fef881550b0 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef880cfe70 0x7fef8815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef880cfe70 0x7fef8815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44436/0 (socket says 172.21.15.87:44436) 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 -- 172.21.15.87:0/1523204219 learned_addr learned my addr 172.21.15.87:0/1523204219 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8e200640 1 --2- 172.21.15.87:0/1523204219 >> v2:172.21.15.169:3300/0 conn(0x7fef8815b740 0x7fef8806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8d800640 1 --2- 172.21.15.87:0/1523204219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef881550b0 0x7fef880cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 -- 172.21.15.87:0/1523204219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef881550b0 msgr2=0x7fef880cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 --2- 172.21.15.87:0/1523204219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef881550b0 0x7fef880cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 -- 172.21.15.87:0/1523204219 >> v2:172.21.15.169:3300/0 conn(0x7fef8815b740 msgr2=0x7fef8806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 --2- 172.21.15.87:0/1523204219 >> v2:172.21.15.169:3300/0 conn(0x7fef8815b740 0x7fef8806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8ce00640 1 -- 172.21.15.87:0/1523204219 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef880718b0 con 0x7fef880cfe70 2024-04-25T10:27:31.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.345+0000 7fef8d800640 1 --2- 172.21.15.87:0/1523204219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef881550b0 0x7fef880cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:31.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef8ce00640 1 --2- 172.21.15.87:0/1523204219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef880cfe70 0x7fef8815b200 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fef880d04d0 tx=0x7fef7401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a341478598842ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:31.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef87e00640 1 -- 172.21.15.87:0/1523204219 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef740b0070 con 0x7fef880cfe70 2024-04-25T10:27:31.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef87e00640 1 -- 172.21.15.87:0/1523204219 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef740ab600 con 0x7fef880cfe70 2024-04-25T10:27:31.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef87e00640 1 -- 172.21.15.87:0/1523204219 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef740ab9d0 con 0x7fef880cfe70 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 -- 172.21.15.87:0/1523204219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef880cfe70 msgr2=0x7fef8815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 --2- 172.21.15.87:0/1523204219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef880cfe70 0x7fef8815b200 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fef880d04d0 tx=0x7fef7401c260 comp rx=0 tx=0).stop 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 -- 172.21.15.87:0/1523204219 shutdown_connections 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 --2- 172.21.15.87:0/1523204219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef880cfe70 0x7fef8815b200 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 --2- 172.21.15.87:0/1523204219 >> v2:172.21.15.169:3300/0 conn(0x7fef8815b740 0x7fef8806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 --2- 172.21.15.87:0/1523204219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef881550b0 0x7fef880cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 -- 172.21.15.87:0/1523204219 >> 172.21.15.87:0/1523204219 conn(0x7fef88066b70 msgr2=0x7fef88066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 -- 172.21.15.87:0/1523204219 shutdown_connections 2024-04-25T10:27:31.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.349+0000 7fef90200640 1 -- 172.21.15.87:0/1523204219 wait complete. 2024-04-25T10:27:31.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 Processor -- start 2024-04-25T10:27:31.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 -- start start 2024-04-25T10:27:31.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef880cfe70 0x7fef88153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef881550b0 0x7fef8814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8815b740 0x7fef8814b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef880b3260 con 0x7fef8815b740 2024-04-25T10:27:31.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8815bd60 con 0x7fef881550b0 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef880b33e0 con 0x7fef880cfe70 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8815b740 0x7fef8814b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8815b740 0x7fef8814b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44450/0 (socket says 172.21.15.87:44450) 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 -- 172.21.15.87:0/1728835671 learned_addr learned my addr 172.21.15.87:0/1728835671 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8ce00640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef881550b0 0x7fef8814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8d800640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef880cfe70 0x7fef88153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 -- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef880cfe70 msgr2=0x7fef88153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef880cfe70 0x7fef88153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 -- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef881550b0 msgr2=0x7fef8814d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef881550b0 0x7fef8814d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 -- 172.21.15.87:0/1728835671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8814c0a0 con 0x7fef8815b740 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8ce00640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef881550b0 0x7fef8814d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef8e200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8815b740 0x7fef8814b140 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fef80007e90 tx=0x7fef8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef85600640 1 -- 172.21.15.87:0/1728835671 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80014420 con 0x7fef8815b740 2024-04-25T10:27:31.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef85600640 1 -- 172.21.15.87:0/1728835671 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80019070 con 0x7fef8815b740 2024-04-25T10:27:31.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef85600640 1 -- 172.21.15.87:0/1728835671 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80014800 con 0x7fef8815b740 2024-04-25T10:27:31.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8814c380 con 0x7fef8815b740 2024-04-25T10:27:31.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.353+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8814c940 con 0x7fef8815b740 2024-04-25T10:27:31.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.357+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8804ef30 con 0x7fef8815b740 2024-04-25T10:27:31.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.357+0000 7fef85600640 1 -- 172.21.15.87:0/1728835671 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef800149a0 con 0x7fef8815b740 2024-04-25T10:27:31.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.357+0000 7fef85600640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef68066f60 0x7fef68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.357+0000 7fef85600640 1 -- 172.21.15.87:0/1728835671 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fef80085900 con 0x7fef8815b740 2024-04-25T10:27:31.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.357+0000 7fef8d800640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef68066f60 0x7fef68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.357+0000 7fef8d800640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef68066f60 0x7fef68069420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00a9a0 tx=0x7fef7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:31.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.361+0000 7fef85600640 1 -- 172.21.15.87:0/1728835671 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef80011260 con 0x7fef8815b740 2024-04-25T10:27:31.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.521+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef8815abe0 con 0x7fef8815b740 2024-04-25T10:27:31.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.521+0000 7fef85600640 1 -- 172.21.15.87:0/1728835671 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+17034 (secure 0 0 0) 0x7fef80056810 con 0x7fef8815b740 2024-04-25T10:27:31.520 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:31.520 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":190,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:27:08.279567+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:07.509820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:08.125891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]},{"pgid":"29.23","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6836/997618220":"2024-04-26T10:26:20.143429+0000","172.21.15.87:6834/997618220":"2024-04-26T10:26:20.143429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef68066f60 msgr2=0x7fef68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef68066f60 0x7fef68069420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00a9a0 tx=0x7fef7c008040 comp rx=0 tx=0).stop 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8815b740 msgr2=0x7fef8814b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8815b740 0x7fef8814b140 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fef80007e90 tx=0x7fef8000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 shutdown_connections 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fef68066f60 0x7fef68069420 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8815b740 0x7fef8814b140 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef881550b0 0x7fef8814d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 --2- 172.21.15.87:0/1728835671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef880cfe70 0x7fef88153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 >> 172.21.15.87:0/1728835671 conn(0x7fef88066b70 msgr2=0x7fef881590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:31.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 shutdown_connections 2024-04-25T10:27:31.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.525+0000 7fef90200640 1 -- 172.21.15.87:0/1728835671 wait complete. 2024-04-25T10:27:31.542 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:27:31.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8346a00640 1 Processor -- start 2024-04-25T10:27:31.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8346a00640 1 -- start start 2024-04-25T10:27:31.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8346a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83400cd2e0 0x7f83400cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8346a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834015f3b0 con 0x7f83400cfd90 2024-04-25T10:27:31.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8346a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400ca810 con 0x7f83400b0f70 2024-04-25T10:27:31.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8346a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834015ef20 con 0x7f83400cd2e0 2024-04-25T10:27:31.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f83400cfd90 0x7f83400ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57960/0 (socket says 172.21.15.87:57960) 2024-04-25T10:27:31.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833f400640 1 -- 172.21.15.87:0/2056766588 learned_addr learned my addr 172.21.15.87:0/2056766588 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8344a00640 1 --2- 172.21.15.87:0/2056766588 >> v2:172.21.15.169:3300/0 conn(0x7f83400cd2e0 0x7f83400cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221734082 0 0) 0x7f834015f3b0 con 0x7f83400cfd90 2024-04-25T10:27:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8324003530 con 0x7f83400cfd90 2024-04-25T10:27:31.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090739062 0 0) 0x7f83400ca810 con 0x7f83400b0f70 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f834015f3b0 con 0x7f83400b0f70 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747614233 0 0) 0x7f8324003530 con 0x7f83400cfd90 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 >> v2:172.21.15.169:3300/0 conn(0x7f83400cd2e0 msgr2=0x7f83400cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 --2- 172.21.15.87:0/2056766588 >> v2:172.21.15.169:3300/0 conn(0x7f83400cd2e0 0x7f83400cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 >> v1:172.21.15.169:6789/0 conn(0x7f83400b0f70 legacy=0x7f83400b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834015e5f0 con 0x7f83400cfd90 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f832c001680 con 0x7f83400cfd90 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 >> v1:172.21.15.87:6789/0 conn(0x7f83400cfd90 legacy=0x7f83400ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8324003ba0 0x7f8324003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8324004810 0x7f8324004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83240051d0 0x7f832400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834015ef20 con 0x7f8324004810 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832400c0d0 con 0x7f8324003ba0 2024-04-25T10:27:31.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832400c2d0 con 0x7f83240051d0 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833f400640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8324004810 0x7f8324004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833fe00640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8324003ba0 0x7f8324003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f8344a00640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83240051d0 0x7f832400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.681+0000 7f833f400640 1 -- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83240051d0 msgr2=0x7f832400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833f400640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83240051d0 0x7f832400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833f400640 1 -- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8324003ba0 msgr2=0x7f8324003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833f400640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8324003ba0 0x7f8324003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833f400640 1 -- 172.21.15.87:0/2056766588 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324003530 con 0x7f8324004810 2024-04-25T10:27:31.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8344a00640 1 -- 172.21.15.87:0/2056766588 reap_dead start 2024-04-25T10:27:31.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833f400640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8324004810 0x7f8324004c90 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f832c007fd0 tx=0x7f832c01cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dadd7a02998ecff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:31.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c007bf0 con 0x7f8324004810 2024-04-25T10:27:31.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c0a9070 con 0x7f8324004810 2024-04-25T10:27:31.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f833ea00640 1 -- 172.21.15.87:0/2056766588 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c005d80 con 0x7f8324004810 2024-04-25T10:27:31.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8346a00640 1 -- 172.21.15.87:0/2056766588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8324004810 msgr2=0x7f8324004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8346a00640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8324004810 0x7f8324004c90 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f832c007fd0 tx=0x7f832c01cb00 comp rx=0 tx=0).stop 2024-04-25T10:27:31.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8346a00640 1 -- 172.21.15.87:0/2056766588 shutdown_connections 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8346a00640 1 --2- 172.21.15.87:0/2056766588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8324004810 0x7f8324004c90 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8346a00640 1 -- 172.21.15.87:0/2056766588 >> 172.21.15.87:0/2056766588 conn(0x7f83400b1d50 msgr2=0x7f83400b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8346a00640 1 -- 172.21.15.87:0/2056766588 shutdown_connections 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.685+0000 7f8346a00640 1 -- 172.21.15.87:0/2056766588 wait complete. 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 Processor -- start 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 -- start start 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8324004810 0x7f8340157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83400cb4c0 0x7f8340157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83400c82e0 0x7f83400c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400b1350 con 0x7f83400cb4c0 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834015ebb0 con 0x7f83400c82e0 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400b14d0 con 0x7f8324004810 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83400cb4c0 0x7f8340157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83400cb4c0 0x7f8340157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44466/0 (socket says 172.21.15.87:44466) 2024-04-25T10:27:31.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 -- 172.21.15.87:0/3495443136 learned_addr learned my addr 172.21.15.87:0/3495443136 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833fe00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8324004810 0x7f8340157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8344a00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83400c82e0 0x7f83400c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 -- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8324004810 msgr2=0x7f8340157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8324004810 0x7f8340157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 -- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83400c82e0 msgr2=0x7f83400c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83400c82e0 0x7f83400c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 -- 172.21.15.87:0/3495443136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400b5860 con 0x7f83400cb4c0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f833f400640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83400cb4c0 0x7f8340157fe0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f832c01c930 tx=0x7f8324005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8337e00640 1 -- 172.21.15.87:0/3495443136 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c006270 con 0x7f83400cb4c0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8337e00640 1 -- 172.21.15.87:0/3495443136 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c006410 con 0x7f83400cb4c0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83400b5b40 con 0x7f83400cb4c0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8337e00640 1 -- 172.21.15.87:0/3495443136 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0a5800 con 0x7f83400cb4c0 2024-04-25T10:27:31.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.689+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400c4880 con 0x7f83400cb4c0 2024-04-25T10:27:31.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.693+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834004ef30 con 0x7f83400cb4c0 2024-04-25T10:27:31.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.693+0000 7f8337e00640 1 -- 172.21.15.87:0/3495443136 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f832c0a59c0 con 0x7f83400cb4c0 2024-04-25T10:27:31.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.693+0000 7f8337e00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8320066f90 0x7f8320069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:31.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.693+0000 7f8337e00640 1 -- 172.21.15.87:0/3495443136 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f832c11ef20 con 0x7f83400cb4c0 2024-04-25T10:27:31.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.693+0000 7f833fe00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8320066f90 0x7f8320069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:31.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.693+0000 7f833fe00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8320066f90 0x7f8320069450 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f83300079b0 tx=0x7f8330007940 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:31.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.697+0000 7f8337e00640 1 -- 172.21.15.87:0/3495443136 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f832c0be060 con 0x7f83400cb4c0 2024-04-25T10:27:31.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.881+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f83400cda20 con 0x7f83400cb4c0 2024-04-25T10:27:31.881 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-04-25T10:27:31.885+0000 7ff594200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:27:31.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.885+0000 7f8337e00640 1 -- 172.21.15.87:0/3495443136 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v144) v1 ==== 161+0+0 (secure 0 0 0) 0x7f832c016810 con 0x7f83400cb4c0 2024-04-25T10:27:31.883 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:27:31.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8320066f90 msgr2=0x7f8320069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8320066f90 0x7f8320069450 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f83300079b0 tx=0x7f8330007940 comp rx=0 tx=0).stop 2024-04-25T10:27:31.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83400cb4c0 msgr2=0x7f8340157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:31.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83400cb4c0 0x7f8340157fe0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f832c01c930 tx=0x7f8324005840 comp rx=0 tx=0).stop 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 shutdown_connections 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f8320066f90 0x7f8320069450 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83400c82e0 0x7f83400c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83400cb4c0 0x7f8340157fe0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 --2- 172.21.15.87:0/3495443136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8324004810 0x7f8340157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 >> 172.21.15.87:0/3495443136 conn(0x7f83400b1d50 msgr2=0x7f8340072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 shutdown_connections 2024-04-25T10:27:31.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:31.889+0000 7f8346a00640 1 -- 172.21.15.87:0/3495443136 wait complete. 2024-04-25T10:27:31.894 INFO:tasks.ceph.mds.a.smithi087.stderr: -2577> 2024-04-25T10:26:26.154+0000 7fb93c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:27:31.908 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:27:31.941 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-04-25T10:27:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66cda00640 1 Processor -- start 2024-04-25T10:27:32.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66cda00640 1 -- start start 2024-04-25T10:27:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66cda00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66c80cfb30 0x7f66c80d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66cda00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 0x7f66c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66cda00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c806b470 con 0x7f66c815dd30 2024-04-25T10:27:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66cda00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c815e110 con 0x7f66c80cfb30 2024-04-25T10:27:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66cda00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80b2940 con 0x7f66c80ce220 2024-04-25T10:27:32.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66c6a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f66c815dd30 0x7f66c806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57976/0 (socket says 172.21.15.87:57976) 2024-04-25T10:27:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66c6a00640 1 -- 172.21.15.87:0/3944935939 learned_addr learned my addr 172.21.15.87:0/3944935939 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66be000640 1 --2- 172.21.15.87:0/3944935939 >> v2:172.21.15.87:3300/0 conn(0x7f66c80cfb30 0x7f66c80d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66be000640 1 -- 172.21.15.87:0/3944935939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 msgr2=0x7f66c80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66be000640 1 --2- 172.21.15.87:0/3944935939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 0x7f66c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66be000640 1 -- 172.21.15.87:0/3944935939 >> v1:172.21.15.87:6789/0 conn(0x7f66c815dd30 legacy=0x7f66c806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:32.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.049+0000 7f66be000640 1 -- 172.21.15.87:0/3944935939 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c806b750 con 0x7f66c80cfb30 2024-04-25T10:27:32.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66be000640 1 --2- 172.21.15.87:0/3944935939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c80d18a0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f66b00097c0 tx=0x7f66b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db0df9f978fe45c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:32.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66c6000640 1 -- 172.21.15.87:0/3944935939 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b0070 con 0x7f66c80cfb30 2024-04-25T10:27:32.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66c6000640 1 -- 172.21.15.87:0/3944935939 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b00ab600 con 0x7f66c80cfb30 2024-04-25T10:27:32.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66c6000640 1 -- 172.21.15.87:0/3944935939 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00ab9d0 con 0x7f66c80cfb30 2024-04-25T10:27:32.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 -- 172.21.15.87:0/3944935939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 msgr2=0x7f66c80d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:32.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 --2- 172.21.15.87:0/3944935939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c80d18a0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f66b00097c0 tx=0x7f66b001c480 comp rx=0 tx=0).stop 2024-04-25T10:27:32.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 -- 172.21.15.87:0/3944935939 shutdown_connections 2024-04-25T10:27:32.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 --2- 172.21.15.87:0/3944935939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 0x7f66c80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 --2- 172.21.15.87:0/3944935939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c80d18a0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 -- 172.21.15.87:0/3944935939 >> 172.21.15.87:0/3944935939 conn(0x7f66c809a9d0 msgr2=0x7f66c809adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:32.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 -- 172.21.15.87:0/3944935939 shutdown_connections 2024-04-25T10:27:32.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.053+0000 7f66cda00640 1 -- 172.21.15.87:0/3944935939 wait complete. 2024-04-25T10:27:32.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 Processor -- start 2024-04-25T10:27:32.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 -- start start 2024-04-25T10:27:32.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 0x7f66c814b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c8154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815dd30 0x7f66c814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80b24f0 con 0x7f66c80cfb30 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80b2370 con 0x7f66c815dd30 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80b2670 con 0x7f66c80ce220 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c8154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c8154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44502/0 (socket says 172.21.15.87:44502) 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 -- 172.21.15.87:0/1488697973 learned_addr learned my addr 172.21.15.87:0/1488697973 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66c6a00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 0x7f66c814b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66c7400640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815dd30 0x7f66c814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 -- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 msgr2=0x7f66c814b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 0x7f66c814b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 -- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815dd30 msgr2=0x7f66c814bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:32.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815dd30 0x7f66c814bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 -- 172.21.15.87:0/1488697973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8155e30 con 0x7f66c80cfb30 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66be000640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c8154540 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f66b001a0b0 tx=0x7f66b001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66bf400640 1 -- 172.21.15.87:0/1488697973 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b0050 con 0x7f66c80cfb30 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66bf400640 1 -- 172.21.15.87:0/1488697973 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b00bc070 con 0x7f66c80cfb30 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66bf400640 1 -- 172.21.15.87:0/1488697973 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b5a70 con 0x7f66c80cfb30 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c8154d90 con 0x7f66c80cfb30 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c8155350 con 0x7f66c80cfb30 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.057+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c804ef30 con 0x7f66c80cfb30 2024-04-25T10:27:32.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.061+0000 7f66bf400640 1 -- 172.21.15.87:0/1488697973 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66b00ab480 con 0x7f66c80cfb30 2024-04-25T10:27:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.061+0000 7f66bf400640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66a4066f60 0x7f66a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.061+0000 7f66bf400640 1 -- 172.21.15.87:0/1488697973 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6026+0+0 (secure 0 0 0) 0x7f66b011fa70 con 0x7f66c80cfb30 2024-04-25T10:27:32.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.061+0000 7f66c6a00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66a4066f60 0x7f66a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:32.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.061+0000 7f66c6a00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66a4066f60 0x7f66a4069420 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f66b8000f70 tx=0x7f66b8007e90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:32.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.065+0000 7f66bf400640 1 -- 172.21.15.87:0/1488697973 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66b0016070 con 0x7f66c80cfb30 2024-04-25T10:27:32.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.249+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f66c80b7920 con 0x7f66c80cfb30 2024-04-25T10:27:32.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66bf400640 1 -- 172.21.15.87:0/1488697973 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v145) v1 ==== 104+0+0 (secure 0 0 0) 0x7f66b00f01c0 con 0x7f66c80cfb30 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66a4066f60 msgr2=0x7f66a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66a4066f60 0x7f66a4069420 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f66b8000f70 tx=0x7f66b8007e90 comp rx=0 tx=0).stop 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 msgr2=0x7f66c8154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c8154540 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f66b001a0b0 tx=0x7f66b001acc0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 shutdown_connections 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f66a4066f60 0x7f66a4069420 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815dd30 0x7f66c814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66c80cfb30 0x7f66c8154540 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 --2- 172.21.15.87:0/1488697973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c80ce220 0x7f66c814b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 >> 172.21.15.87:0/1488697973 conn(0x7f66c809a9d0 msgr2=0x7f66c80ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 shutdown_connections 2024-04-25T10:27:32.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:32.889+0000 7f66cda00640 1 -- 172.21.15.87:0/1488697973 wait complete. 2024-04-25T10:27:32.911 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:27:33.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4698200640 1 Processor -- start 2024-04-25T10:27:33.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4698200640 1 -- start start 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4698200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f469015c0f0 0x7f46900b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4698200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469006c0e0 con 0x7f469015bd10 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4698200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469015c6a0 con 0x7f46900cf9e0 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4698200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469015c980 con 0x7f469015c0f0 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4695800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f469015bd10 0x7f46900b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57988/0 (socket says 172.21.15.87:57988) 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4695800640 1 -- 172.21.15.87:0/3801836870 learned_addr learned my addr 172.21.15.87:0/3801836870 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4694e00640 1 --2- 172.21.15.87:0/3801836870 >> v2:172.21.15.169:3300/0 conn(0x7f469015c0f0 0x7f46900b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230766122 0 0) 0x7f469006c0e0 con 0x7f469015bd10 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4678003530 con 0x7f469015bd10 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257528073 0 0) 0x7f469015c6a0 con 0x7f46900cf9e0 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f469006c0e0 con 0x7f46900cf9e0 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 77852183 0 0) 0x7f4678003530 con 0x7f469015bd10 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 >> v2:172.21.15.169:3300/0 conn(0x7f469015c0f0 msgr2=0x7f46900b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 --2- 172.21.15.87:0/3801836870 >> v2:172.21.15.169:3300/0 conn(0x7f469015c0f0 0x7f46900b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 >> v1:172.21.15.169:6789/0 conn(0x7f46900cf9e0 legacy=0x7f469006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469015cc60 con 0x7f469015bd10 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f4684001680 con 0x7f469015bd10 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 >> v1:172.21.15.87:6789/0 conn(0x7f469015bd10 legacy=0x7f46900b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f4678003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4678004810 0x7f4678004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46780051d0 0x7f467800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469015c980 con 0x7f4678003ba0 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467800c0a0 con 0x7f46780051d0 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467800c2a0 con 0x7f4678004810 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4695800640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f4678003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4694e00640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4678004810 0x7f4678004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.097+0000 7f4696200640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46780051d0 0x7f467800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4695800640 1 -- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4678004810 msgr2=0x7f4678004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4695800640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4678004810 0x7f4678004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4695800640 1 -- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46780051d0 msgr2=0x7f467800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4695800640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46780051d0 0x7f467800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4695800640 1 -- 172.21.15.87:0/3801836870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678003530 con 0x7f4678003ba0 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4696200640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46780051d0 0x7f467800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:33.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4696200640 1 -- 172.21.15.87:0/3801836870 reap_dead start 2024-04-25T10:27:33.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4695800640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f4678003fa0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f4684007fd0 tx=0x7f468401cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e69dc4ab654d200 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684007bf0 con 0x7f4678003ba0 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840a9020 con 0x7f4678003ba0 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f468fe00640 1 -- 172.21.15.87:0/3801836870 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684005b00 con 0x7f4678003ba0 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4698200640 1 -- 172.21.15.87:0/3801836870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 msgr2=0x7f4678003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4698200640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f4678003fa0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f4684007fd0 tx=0x7f468401cb10 comp rx=0 tx=0).stop 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4698200640 1 -- 172.21.15.87:0/3801836870 shutdown_connections 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4698200640 1 --2- 172.21.15.87:0/3801836870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f4678003fa0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4698200640 1 -- 172.21.15.87:0/3801836870 >> 172.21.15.87:0/3801836870 conn(0x7f4690066b70 msgr2=0x7f4690066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4698200640 1 -- 172.21.15.87:0/3801836870 shutdown_connections 2024-04-25T10:27:33.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.101+0000 7f4698200640 1 -- 172.21.15.87:0/3801836870 wait complete. 2024-04-25T10:27:33.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 Processor -- start 2024-04-25T10:27:33.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 -- start start 2024-04-25T10:27:33.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f469014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4690152900 0x7f4690152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46900b02b0 0x7f46900b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900b2b70 con 0x7f4678003ba0 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900b29f0 con 0x7f46900b02b0 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46900b2cf0 con 0x7f4690152900 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f469014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f469014a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44526/0 (socket says 172.21.15.87:44526) 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 -- 172.21.15.87:0/1669986601 learned_addr learned my addr 172.21.15.87:0/1669986601 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4694e00640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4690152900 0x7f4690152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4696200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46900b02b0 0x7f46900b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 -- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4690152900 msgr2=0x7f4690152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4690152900 0x7f4690152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 -- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46900b02b0 msgr2=0x7f46900b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46900b02b0 0x7f46900b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 -- 172.21.15.87:0/1669986601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4690153410 con 0x7f4678003ba0 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4696200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46900b02b0 0x7f46900b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4695800640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f469014a120 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f46840ab040 tx=0x7f4678005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f468d600640 1 -- 172.21.15.87:0/1669986601 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840bb070 con 0x7f4678003ba0 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f468d600640 1 -- 172.21.15.87:0/1669986601 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840070d0 con 0x7f4678003ba0 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f468d600640 1 -- 172.21.15.87:0/1669986601 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684007bf0 con 0x7f4678003ba0 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46901536f0 con 0x7f4678003ba0 2024-04-25T10:27:33.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.105+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4690153cb0 con 0x7f4678003ba0 2024-04-25T10:27:33.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.109+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469004ef30 con 0x7f4678003ba0 2024-04-25T10:27:33.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.109+0000 7f468d600640 1 -- 172.21.15.87:0/1669986601 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4684007480 con 0x7f4678003ba0 2024-04-25T10:27:33.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.109+0000 7f468d600640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4670066f90 0x7f4670069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:33.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.109+0000 7f468d600640 1 -- 172.21.15.87:0/1669986601 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6026+0+0 (secure 0 0 0) 0x7f468411e450 con 0x7f4678003ba0 2024-04-25T10:27:33.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.109+0000 7f4694e00640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4670066f90 0x7f4670069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:33.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.109+0000 7f4694e00640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4670066f90 0x7f4670069450 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f46901fa0e0 tx=0x7f467c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:33.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.113+0000 7f468d600640 1 -- 172.21.15.87:0/1669986601 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4684016850 con 0x7f4678003ba0 2024-04-25T10:27:33.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.269+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4690155a30 con 0x7f4678003ba0 2024-04-25T10:27:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.897+0000 7f468d600640 1 -- 172.21.15.87:0/1669986601 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v192) v1 ==== 181+0+0 (secure 0 0 0) 0x7f46840ef310 con 0x7f4678003ba0 2024-04-25T10:27:33.895 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:27:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4670066f90 msgr2=0x7f4670069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4670066f90 0x7f4670069450 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f46901fa0e0 tx=0x7f467c008040 comp rx=0 tx=0).stop 2024-04-25T10:27:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 msgr2=0x7f469014a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f469014a120 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f46840ab040 tx=0x7f4678005840 comp rx=0 tx=0).stop 2024-04-25T10:27:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 shutdown_connections 2024-04-25T10:27:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f4670066f90 0x7f4670069450 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46900b02b0 0x7f46900b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4690152900 0x7f4690152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 --2- 172.21.15.87:0/1669986601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4678003ba0 0x7f469014a120 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 >> 172.21.15.87:0/1669986601 conn(0x7f4690066b70 msgr2=0x7f469015d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 shutdown_connections 2024-04-25T10:27:33.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:33.901+0000 7f4698200640 1 -- 172.21.15.87:0/1669986601 wait complete. 2024-04-25T10:27:33.921 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:27:34.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 Processor -- start 2024-04-25T10:27:34.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 -- start start 2024-04-25T10:27:34.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0cf040 0x7f16fc0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16fc15bc00 0x7f16fc0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16fc0713a0 0x7f16fc0717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc072260 con 0x7f16fc0713a0 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc0b3700 con 0x7f16fc15bc00 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1703a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc15aec0 con 0x7f16fc0cf040 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16fc0713a0 0x7f16fc0717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16fc0713a0 0x7f16fc0717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44530/0 (socket says 172.21.15.87:44530) 2024-04-25T10:27:34.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 -- 172.21.15.87:0/3570827097 learned_addr learned my addr 172.21.15.87:0/3570827097 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:34.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 -- 172.21.15.87:0/3570827097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0cf040 msgr2=0x7f16fc0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:34.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 --2- 172.21.15.87:0/3570827097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0cf040 0x7f16fc0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 -- 172.21.15.87:0/3570827097 >> v2:172.21.15.169:3300/0 conn(0x7f16fc15bc00 msgr2=0x7f16fc0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:34.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 --2- 172.21.15.87:0/3570827097 >> v2:172.21.15.169:3300/0 conn(0x7f16fc15bc00 0x7f16fc0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.065+0000 7f1701a00640 1 -- 172.21.15.87:0/3570827097 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc071f80 con 0x7f16fc0713a0 2024-04-25T10:27:34.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1701a00640 1 --2- 172.21.15.87:0/3570827097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0713a0 0x7f16fc0717e0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f16fc06e2c0 tx=0x7f16f401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183280a8d6735bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:34.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f16fb400640 1 -- 172.21.15.87:0/3570827097 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f40a7870 con 0x7f16fc0713a0 2024-04-25T10:27:34.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f16fb400640 1 -- 172.21.15.87:0/3570827097 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f40a7a10 con 0x7f16fc0713a0 2024-04-25T10:27:34.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f16fb400640 1 -- 172.21.15.87:0/3570827097 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f40a7de0 con 0x7f16fc0713a0 2024-04-25T10:27:34.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 -- 172.21.15.87:0/3570827097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0713a0 msgr2=0x7f16fc0717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:34.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 --2- 172.21.15.87:0/3570827097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0713a0 0x7f16fc0717e0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f16fc06e2c0 tx=0x7f16f401ebe0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 -- 172.21.15.87:0/3570827097 shutdown_connections 2024-04-25T10:27:34.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 --2- 172.21.15.87:0/3570827097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc0713a0 0x7f16fc0717e0 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 --2- 172.21.15.87:0/3570827097 >> v2:172.21.15.169:3300/0 conn(0x7f16fc15bc00 0x7f16fc0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 --2- 172.21.15.87:0/3570827097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0cf040 0x7f16fc0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 -- 172.21.15.87:0/3570827097 >> 172.21.15.87:0/3570827097 conn(0x7f16fc066b70 msgr2=0x7f16fc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:34.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 -- 172.21.15.87:0/3570827097 shutdown_connections 2024-04-25T10:27:34.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 -- 172.21.15.87:0/3570827097 wait complete. 2024-04-25T10:27:34.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 Processor -- start 2024-04-25T10:27:34.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.069+0000 7f1703a00640 1 -- start start 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0713a0 0x7f16fc1faf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc0cf040 0x7f16fc14b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc15bc00 0x7f16fc1f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0cf5a0 con 0x7f16fc15bc00 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0cf420 con 0x7f16fc0cf040 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0aec90 con 0x7f16fc0713a0 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc15bc00 0x7f16fc1f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc15bc00 0x7f16fc1f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44534/0 (socket says 172.21.15.87:44534) 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 -- 172.21.15.87:0/185966915 learned_addr learned my addr 172.21.15.87:0/185966915 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f16fbe00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc0cf040 0x7f16fc14b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:34.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701000640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0713a0 0x7f16fc1faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 -- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0713a0 msgr2=0x7f16fc1faf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0713a0 0x7f16fc1faf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 -- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc0cf040 msgr2=0x7f16fc14b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc0cf040 0x7f16fc14b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 -- 172.21.15.87:0/185966915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc1f79f0 con 0x7f16fc15bc00 2024-04-25T10:27:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f16fbe00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc0cf040 0x7f16fc14b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:34.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1701a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc15bc00 0x7f16fc1f6f00 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f16f401d290 tx=0x7f16f401b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:34.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f16f8c00640 1 -- 172.21.15.87:0/185966915 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f40be070 con 0x7f16fc15bc00 2024-04-25T10:27:34.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f16f8c00640 1 -- 172.21.15.87:0/185966915 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f4007600 con 0x7f16fc15bc00 2024-04-25T10:27:34.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f16f8c00640 1 -- 172.21.15.87:0/185966915 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4009270 con 0x7f16fc15bc00 2024-04-25T10:27:34.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16fc1f7cd0 con 0x7f16fc15bc00 2024-04-25T10:27:34.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc1faa50 con 0x7f16fc15bc00 2024-04-25T10:27:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc04ef30 con 0x7f16fc15bc00 2024-04-25T10:27:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.073+0000 7f16f8c00640 1 -- 172.21.15.87:0/185966915 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16f40079f0 con 0x7f16fc15bc00 2024-04-25T10:27:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.077+0000 7f16f8c00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.077+0000 7f16f8c00640 1 -- 172.21.15.87:0/185966915 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f16f411eec0 con 0x7f16fc15bc00 2024-04-25T10:27:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.077+0000 7f1701000640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:34.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.077+0000 7f1701000640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f16f0006fd0 tx=0x7f16f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:34.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.081+0000 7f16f8c00640 1 -- 172.21.15.87:0/185966915 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16f40187d0 con 0x7f16fc15bc00 2024-04-25T10:27:34.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.241+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16fc1fa520 con 0x7f16fc15bc00 2024-04-25T10:27:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.901+0000 7f16f8c00640 1 -- 172.21.15.87:0/185966915 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v193) v1 ==== 169+0+0 (secure 0 0 0) 0x7f16f40efea0 con 0x7f16fc15bc00 2024-04-25T10:27:34.898 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-04-25T10:27:34.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.901+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 msgr2=0x7f16dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:34.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.901+0000 7f1703a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f16f0006fd0 tx=0x7f16f0008040 comp rx=0 tx=0).stop 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.901+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc15bc00 msgr2=0x7f16fc1f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.901+0000 7f1703a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc15bc00 0x7f16fc1f6f00 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f16f401d290 tx=0x7f16f401b980 comp rx=0 tx=0).stop 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 shutdown_connections 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f16dc066f60 0x7f16dc069420 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16fc15bc00 0x7f16fc1f6f00 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16fc0cf040 0x7f16fc14b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 --2- 172.21.15.87:0/185966915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16fc0713a0 0x7f16fc1faf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:34.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 >> 172.21.15.87:0/185966915 conn(0x7f16fc066b70 msgr2=0x7f16fc1506b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 shutdown_connections 2024-04-25T10:27:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:34.905+0000 7f1703a00640 1 -- 172.21.15.87:0/185966915 wait complete. 2024-04-25T10:27:34.926 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:27:35.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb608800640 1 Processor -- start 2024-04-25T10:27:35.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb608800640 1 -- start start 2024-04-25T10:27:35.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb608800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6000aef80 0x7fb60015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb608800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60015b080 con 0x7fb600070350 2024-04-25T10:27:35.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb608800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60015adb0 con 0x7fb6000b31c0 2024-04-25T10:27:35.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb608800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60015eb60 con 0x7fb6000aef80 2024-04-25T10:27:35.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb600070350 0x7fb60015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:57990/0 (socket says 172.21.15.87:57990) 2024-04-25T10:27:35.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605400640 1 -- 172.21.15.87:0/48058232 learned_addr learned my addr 172.21.15.87:0/48058232 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:35.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605e00640 1 --2- 172.21.15.87:0/48058232 >> v2:172.21.15.169:3300/0 conn(0x7fb6000aef80 0x7fb60015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779779333 0 0) 0x7fb60015b080 con 0x7fb600070350 2024-04-25T10:27:35.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e8003530 con 0x7fb600070350 2024-04-25T10:27:35.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043854246 0 0) 0x7fb60015adb0 con 0x7fb6000b31c0 2024-04-25T10:27:35.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb60015b080 con 0x7fb6000b31c0 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3775502578 0 0) 0x7fb5e8003530 con 0x7fb600070350 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 >> v2:172.21.15.169:3300/0 conn(0x7fb6000aef80 msgr2=0x7fb60015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 --2- 172.21.15.87:0/48058232 >> v2:172.21.15.169:3300/0 conn(0x7fb6000aef80 0x7fb60015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 >> v1:172.21.15.169:6789/0 conn(0x7fb6000b31c0 legacy=0x7fb6000cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60015ee40 con 0x7fb600070350 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fb5f0001680 con 0x7fb600070350 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 >> v1:172.21.15.87:6789/0 conn(0x7fb600070350 legacy=0x7fb60015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb5e8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5e8004810 0x7fb5e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5e80051d0 0x7fb5e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60015eb60 con 0x7fb5e8003b50 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e800c050 con 0x7fb5e80051d0 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e800c250 con 0x7fb5e8004810 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605e00640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb5e8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605400640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5e8004810 0x7fb5e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb606800640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5e80051d0 0x7fb5e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605e00640 1 -- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5e8004810 msgr2=0x7fb5e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605e00640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5e8004810 0x7fb5e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605e00640 1 -- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5e80051d0 msgr2=0x7fb5e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605e00640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5e80051d0 0x7fb5e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb605e00640 1 -- 172.21.15.87:0/48058232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8003530 con 0x7fb5e8003b50 2024-04-25T10:27:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.069+0000 7fb606800640 1 -- 172.21.15.87:0/48058232 reap_dead start 2024-04-25T10:27:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb605e00640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb5e8003f50 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009ae0 tx=0x7fb5f401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5499a62716635b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40b2070 con 0x7fb5e8003b50 2024-04-25T10:27:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f40a5640 con 0x7fb5e8003b50 2024-04-25T10:27:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb604a00640 1 -- 172.21.15.87:0/48058232 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40096b0 con 0x7fb5e8003b50 2024-04-25T10:27:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb608800640 1 -- 172.21.15.87:0/48058232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 msgr2=0x7fb5e8003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb608800640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb5e8003f50 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009ae0 tx=0x7fb5f401ca70 comp rx=0 tx=0).stop 2024-04-25T10:27:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb608800640 1 -- 172.21.15.87:0/48058232 shutdown_connections 2024-04-25T10:27:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb608800640 1 --2- 172.21.15.87:0/48058232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb5e8003f50 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb608800640 1 -- 172.21.15.87:0/48058232 >> 172.21.15.87:0/48058232 conn(0x7fb6000669a0 msgr2=0x7fb600066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb608800640 1 -- 172.21.15.87:0/48058232 shutdown_connections 2024-04-25T10:27:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.073+0000 7fb608800640 1 -- 172.21.15.87:0/48058232 wait complete. 2024-04-25T10:27:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 Processor -- start 2024-04-25T10:27:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 -- start start 2024-04-25T10:27:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb6000b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb600149c10 0x7fb6000b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60014c810 0x7fb6000b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000b39e0 con 0x7fb5e8003b50 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60015bd90 con 0x7fb60014c810 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000cf660 con 0x7fb600149c10 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb6000b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb6000b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44550/0 (socket says 172.21.15.87:44550) 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 -- 172.21.15.87:0/2879457162 learned_addr learned my addr 172.21.15.87:0/2879457162 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605400640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb600149c10 0x7fb6000b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb606800640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60014c810 0x7fb6000b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 -- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb600149c10 msgr2=0x7fb6000b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb600149c10 0x7fb6000b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 -- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60014c810 msgr2=0x7fb6000b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60014c810 0x7fb6000b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 -- 172.21.15.87:0/2879457162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000b0dd0 con 0x7fb5e8003b50 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb605e00640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb6000b0350 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fb5f401c8a0 tx=0x7fb5f4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb5fa000640 1 -- 172.21.15.87:0/2879457162 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40b2040 con 0x7fb5e8003b50 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb5fa000640 1 -- 172.21.15.87:0/2879457162 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f400f000 con 0x7fb5e8003b50 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb5fa000640 1 -- 172.21.15.87:0/2879457162 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40096b0 con 0x7fb5e8003b50 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb600152c30 con 0x7fb5e8003b50 2024-04-25T10:27:35.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.077+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6001534d0 con 0x7fb5e8003b50 2024-04-25T10:27:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.081+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60004ef30 con 0x7fb5e8003b50 2024-04-25T10:27:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.081+0000 7fb5fa000640 1 -- 172.21.15.87:0/2879457162 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5f40bf030 con 0x7fb5e8003b50 2024-04-25T10:27:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.081+0000 7fb5fa000640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5e4066f90 0x7fb5e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.081+0000 7fb5fa000640 1 -- 172.21.15.87:0/2879457162 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb5f411e990 con 0x7fb5e8003b50 2024-04-25T10:27:35.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.081+0000 7fb605400640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5e4066f90 0x7fb5e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.081+0000 7fb605400640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5e4066f90 0x7fb5e4069450 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb60014a030 tx=0x7fb5f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.085+0000 7fb5fa000640 1 -- 172.21.15.87:0/2879457162 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5f40c6060 con 0x7fb5e8003b50 2024-04-25T10:27:35.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.241+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb6000b2fa0 con 0x7fb5e8003b50 2024-04-25T10:27:35.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb5fa000640 1 -- 172.21.15.87:0/2879457162 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5f4016780 con 0x7fb5e8003b50 2024-04-25T10:27:35.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5e4066f90 msgr2=0x7fb5e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5e4066f90 0x7fb5e4069450 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb60014a030 tx=0x7fb5f0009040 comp rx=0 tx=0).stop 2024-04-25T10:27:35.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 msgr2=0x7fb6000b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb6000b0350 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fb5f401c8a0 tx=0x7fb5f4019810 comp rx=0 tx=0).stop 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 shutdown_connections 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb5e4066f90 0x7fb5e4069450 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60014c810 0x7fb6000b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb600149c10 0x7fb6000b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 --2- 172.21.15.87:0/2879457162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003b50 0x7fb6000b0350 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 >> 172.21.15.87:0/2879457162 conn(0x7fb6000669a0 msgr2=0x7fb600159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 shutdown_connections 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.245+0000 7fb608800640 1 -- 172.21.15.87:0/2879457162 wait complete. 2024-04-25T10:27:35.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-25T10:27:35.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe977e00640 1 Processor -- start 2024-04-25T10:27:35.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe977e00640 1 -- start start 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe977e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe97015e130 0x7fe9700cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe977e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe970071bc0 0x7fe970072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe977e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9700ad840 con 0x7fe9700b3480 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe977e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9700d0090 con 0x7fe97015e130 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe977e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97015bb90 con 0x7fe970071bc0 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe974a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe97015e130 0x7fe9700cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe975400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe9700b3480 0x7fe970070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58002/0 (socket says 172.21.15.87:58002) 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe974a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe97015e130 0x7fe9700cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44552/0 (socket says 172.21.15.87:44552) 2024-04-25T10:27:35.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe975400640 1 -- 172.21.15.87:0/4015517598 learned_addr learned my addr 172.21.15.87:0/4015517598 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:35.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe975e00640 1 --2- 172.21.15.87:0/4015517598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe970071bc0 0x7fe970072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe974a00640 1 -- 172.21.15.87:0/4015517598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe970071bc0 msgr2=0x7fe970072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:35.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe974a00640 1 --2- 172.21.15.87:0/4015517598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe970071bc0 0x7fe970072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe974a00640 1 -- 172.21.15.87:0/4015517598 >> v1:172.21.15.87:6789/0 conn(0x7fe9700b3480 legacy=0x7fe970070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe974a00640 1 -- 172.21.15.87:0/4015517598 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97015e710 con 0x7fe97015e130 2024-04-25T10:27:35.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe96fe00640 1 -- 172.21.15.87:0/4015517598 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114094421 0 0) 0x7fe9700ad840 con 0x7fe9700b3480 2024-04-25T10:27:35.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.405+0000 7fe975e00640 1 --2- 172.21.15.87:0/4015517598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe970071bc0 0x7fe970072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:35.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe974a00640 1 --2- 172.21.15.87:0/4015517598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015e130 0x7fe9700cfb50 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0097c0 tx=0x7fe95c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f56a62546128f52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe96fe00640 1 -- 172.21.15.87:0/4015517598 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0b0070 con 0x7fe97015e130 2024-04-25T10:27:35.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe96fe00640 1 -- 172.21.15.87:0/4015517598 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c0ab600 con 0x7fe97015e130 2024-04-25T10:27:35.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe96fe00640 1 -- 172.21.15.87:0/4015517598 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0ab9d0 con 0x7fe97015e130 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 -- 172.21.15.87:0/4015517598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015e130 msgr2=0x7fe9700cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 --2- 172.21.15.87:0/4015517598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015e130 0x7fe9700cfb50 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0097c0 tx=0x7fe95c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 -- 172.21.15.87:0/4015517598 shutdown_connections 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 --2- 172.21.15.87:0/4015517598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe970071bc0 0x7fe970072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 --2- 172.21.15.87:0/4015517598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe97015e130 0x7fe9700cfb50 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 -- 172.21.15.87:0/4015517598 >> 172.21.15.87:0/4015517598 conn(0x7fe9700669a0 msgr2=0x7fe970066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 -- 172.21.15.87:0/4015517598 shutdown_connections 2024-04-25T10:27:35.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 -- 172.21.15.87:0/4015517598 wait complete. 2024-04-25T10:27:35.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 Processor -- start 2024-04-25T10:27:35.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.409+0000 7fe977e00640 1 -- start start 2024-04-25T10:27:35.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe970071bc0 0x7fe970156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700b3480 0x7fe97015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015e130 0x7fe970158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700ae6b0 con 0x7fe9700b3480 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700ae530 con 0x7fe970071bc0 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700ae830 con 0x7fe97015e130 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700b3480 0x7fe97015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700b3480 0x7fe97015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44568/0 (socket says 172.21.15.87:44568) 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 -- 172.21.15.87:0/2421027236 learned_addr learned my addr 172.21.15.87:0/2421027236 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe975400640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe970071bc0 0x7fe970156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe975e00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015e130 0x7fe970158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 -- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015e130 msgr2=0x7fe970158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015e130 0x7fe970158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 -- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe970071bc0 msgr2=0x7fe970156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe970071bc0 0x7fe970156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 -- 172.21.15.87:0/2421027236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97015f370 con 0x7fe9700b3480 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe975400640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe970071bc0 0x7fe970156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:35.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe974a00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700b3480 0x7fe97015a9f0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0ae040 tx=0x7fe95c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe96d600640 1 -- 172.21.15.87:0/2421027236 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0b0040 con 0x7fe9700b3480 2024-04-25T10:27:35.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe96d600640 1 -- 172.21.15.87:0/2421027236 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c004030 con 0x7fe9700b3480 2024-04-25T10:27:35.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe96d600640 1 -- 172.21.15.87:0/2421027236 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0043e0 con 0x7fe9700b3480 2024-04-25T10:27:35.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97015f650 con 0x7fe9700b3480 2024-04-25T10:27:35.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe970156dd0 con 0x7fe9700b3480 2024-04-25T10:27:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.413+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97015fa40 con 0x7fe9700b3480 2024-04-25T10:27:35.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.417+0000 7fe96d600640 1 -- 172.21.15.87:0/2421027236 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe95c0b3070 con 0x7fe9700b3480 2024-04-25T10:27:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.417+0000 7fe96d600640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.417+0000 7fe96d600640 1 -- 172.21.15.87:0/2421027236 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe95c11f3d0 con 0x7fe9700b3480 2024-04-25T10:27:35.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.417+0000 7fe975400640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.417+0000 7fe975400640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe964007fd0 tx=0x7fe964007c20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.421+0000 7fe96d600640 1 -- 172.21.15.87:0/2421027236 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe95c0ba020 con 0x7fe9700b3480 2024-04-25T10:27:35.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.573+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) v1 -- 0x7fe97015f820 con 0x7fe9700b3480 2024-04-25T10:27:35.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.577+0000 7fe96d600640 1 -- 172.21.15.87:0/2421027236 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe95c016090 con 0x7fe9700b3480 2024-04-25T10:27:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 msgr2=0x7fe950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe964007fd0 tx=0x7fe964007c20 comp rx=0 tx=0).stop 2024-04-25T10:27:35.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700b3480 msgr2=0x7fe97015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700b3480 0x7fe97015a9f0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0ae040 tx=0x7fe95c019240 comp rx=0 tx=0).stop 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 shutdown_connections 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97015e130 0x7fe970158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9700b3480 0x7fe97015a9f0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 --2- 172.21.15.87:0/2421027236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe970071bc0 0x7fe970156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 >> 172.21.15.87:0/2421027236 conn(0x7fe9700669a0 msgr2=0x7fe9701586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 shutdown_connections 2024-04-25T10:27:35.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.581+0000 7fe977e00640 1 -- 172.21.15.87:0/2421027236 wait complete. 2024-04-25T10:27:35.597 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:27:35.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.733+0000 7f0136000640 1 Processor -- start 2024-04-25T10:27:35.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0136000640 1 -- start start 2024-04-25T10:27:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0136000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01300cfd90 0x7f01300ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0136000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013015f3b0 con 0x7f01300b0f70 2024-04-25T10:27:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0136000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01300ca810 con 0x7f01300cd110 2024-04-25T10:27:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0136000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f013015ef20 con 0x7f01300cfd90 2024-04-25T10:27:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f01300b0f70 0x7f01300b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:58008/0 (socket says 172.21.15.87:58008) 2024-04-25T10:27:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 -- 172.21.15.87:0/2654992767 learned_addr learned my addr 172.21.15.87:0/2654992767 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:35.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0126a00640 1 --2- 172.21.15.87:0/2654992767 >> v2:172.21.15.169:3300/0 conn(0x7f01300cfd90 0x7f01300ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419382992 0 0) 0x7f013015f3b0 con 0x7f01300b0f70 2024-04-25T10:27:35.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0114003530 con 0x7f01300b0f70 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3149521760 0 0) 0x7f0114003530 con 0x7f01300b0f70 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 >> v2:172.21.15.169:3300/0 conn(0x7f01300cfd90 msgr2=0x7f01300ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 --2- 172.21.15.87:0/2654992767 >> v2:172.21.15.169:3300/0 conn(0x7f01300cfd90 0x7f01300ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 >> v1:172.21.15.169:6789/0 conn(0x7f01300cd110 legacy=0x7f01300cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013015e5f0 con 0x7f01300b0f70 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f0120001680 con 0x7f01300b0f70 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 >> v1:172.21.15.87:6789/0 conn(0x7f01300b0f70 legacy=0x7f01300b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0114003a10 0x7f0114003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0114004810 0x7f0114004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01140051d0 0x7f011400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013015ef20 con 0x7f0114003a10 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011400c100 con 0x7f0114004810 2024-04-25T10:27:35.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011400c300 con 0x7f01140051d0 2024-04-25T10:27:35.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0114003a10 0x7f0114003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0126a00640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0114004810 0x7f0114004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012fe00640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01140051d0 0x7f011400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 -- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01140051d0 msgr2=0x7f011400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:35.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01140051d0 0x7f011400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 -- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0114004810 msgr2=0x7f0114004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:35.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0114004810 0x7f0114004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012f400640 1 -- 172.21.15.87:0/2654992767 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0114003530 con 0x7f0114003a10 2024-04-25T10:27:35.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f0126a00640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0114004810 0x7f0114004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:35.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.737+0000 7f012fe00640 1 -- 172.21.15.87:0/2654992767 reap_dead start 2024-04-25T10:27:35.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f012f400640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0114003a10 0x7f0114003e50 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f012000c110 tx=0x7f012001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44a9a58aa8e9ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120007c50 con 0x7f0114003a10 2024-04-25T10:27:35.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01200a9070 con 0x7f0114003a10 2024-04-25T10:27:35.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f012ea00640 1 -- 172.21.15.87:0/2654992767 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120005d80 con 0x7f0114003a10 2024-04-25T10:27:35.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f0136000640 1 -- 172.21.15.87:0/2654992767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0114003a10 msgr2=0x7f0114003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f0136000640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0114003a10 0x7f0114003e50 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f012000c110 tx=0x7f012001cbc0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f0136000640 1 -- 172.21.15.87:0/2654992767 shutdown_connections 2024-04-25T10:27:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f0136000640 1 --2- 172.21.15.87:0/2654992767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0114003a10 0x7f0114003e50 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f0136000640 1 -- 172.21.15.87:0/2654992767 >> 172.21.15.87:0/2654992767 conn(0x7f01300b1d50 msgr2=0x7f01300b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f0136000640 1 -- 172.21.15.87:0/2654992767 shutdown_connections 2024-04-25T10:27:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.741+0000 7f0136000640 1 -- 172.21.15.87:0/2654992767 wait complete. 2024-04-25T10:27:35.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 Processor -- start 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 -- start start 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0114003a10 0x7f01300b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01300af040 0x7f01301570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0130162820 0x7f0130162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01300b18e0 con 0x7f0130162820 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01300b1760 con 0x7f01300af040 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01300cc230 con 0x7f0114003a10 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0130162820 0x7f0130162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0130162820 0x7f0130162ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44580/0 (socket says 172.21.15.87:44580) 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 -- 172.21.15.87:0/1863375464 learned_addr learned my addr 172.21.15.87:0/1863375464 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012f400640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0114003a10 0x7f01300b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 -- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0114003a10 msgr2=0x7f01300b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0114003a10 0x7f01300b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 -- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01300af040 msgr2=0x7f01301570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0126a00640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01300af040 0x7f01301570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01300af040 0x7f01301570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 -- 172.21.15.87:0/1863375464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300af890 con 0x7f0130162820 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012f400640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0114003a10 0x7f01300b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0126a00640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01300af040 0x7f01301570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f012fe00640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0130162820 0x7f0130162ca0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f01300b2d50 tx=0x7f012800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0127e00640 1 -- 172.21.15.87:0/1863375464 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128003980 con 0x7f0130162820 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0127e00640 1 -- 172.21.15.87:0/1863375464 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0128018070 con 0x7f0130162820 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0127e00640 1 -- 172.21.15.87:0/1863375464 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128003d80 con 0x7f0130162820 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01300afb70 con 0x7f0130162820 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01300c4b60 con 0x7f0130162820 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.745+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013004ef30 con 0x7f0130162820 2024-04-25T10:27:35.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.749+0000 7f0127e00640 1 -- 172.21.15.87:0/1863375464 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01280050d0 con 0x7f0130162820 2024-04-25T10:27:35.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.749+0000 7f0127e00640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f010c066f60 0x7f010c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:35.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.749+0000 7f0127e00640 1 -- 172.21.15.87:0/1863375464 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0128084ae0 con 0x7f0130162820 2024-04-25T10:27:35.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.749+0000 7f012f400640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f010c066f60 0x7f010c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:35.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.749+0000 7f012f400640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f010c066f60 0x7f010c069420 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f01200ab000 tx=0x7f01200a7070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:35.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.753+0000 7f0127e00640 1 -- 172.21.15.87:0/1863375464 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0128055cf0 con 0x7f0130162820 2024-04-25T10:27:35.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:35.961+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f01300cda20 con 0x7f0130162820 2024-04-25T10:27:36.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.913+0000 7f0127e00640 1 -- 172.21.15.87:0/1863375464 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v724) v1 ==== 133+0+0 (secure 0 0 0) 0x7f01280555f0 con 0x7f0130162820 2024-04-25T10:27:36.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f010c066f60 msgr2=0x7f010c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f010c066f60 0x7f010c069420 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f01200ab000 tx=0x7f01200a7070 comp rx=0 tx=0).stop 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0130162820 msgr2=0x7f0130162ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0130162820 0x7f0130162ca0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f01300b2d50 tx=0x7f012800ec10 comp rx=0 tx=0).stop 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 shutdown_connections 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f010c066f60 0x7f010c069420 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0130162820 0x7f0130162ca0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01300af040 0x7f01301570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 --2- 172.21.15.87:0/1863375464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0114003a10 0x7f01300b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 >> 172.21.15.87:0/1863375464 conn(0x7f01300b1d50 msgr2=0x7f0130157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 shutdown_connections 2024-04-25T10:27:36.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:36.917+0000 7f0136000640 1 -- 172.21.15.87:0/1863375464 wait complete. 2024-04-25T10:27:36.938 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:27:36.939 INFO:tasks.cephfs_test_runner: 2024-04-25T10:27:36.939 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:27:36.939 INFO:tasks.cephfs_test_runner:Ran 13 tests in 885.098s 2024-04-25T10:27:36.939 INFO:tasks.cephfs_test_runner: 2024-04-25T10:27:36.939 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:27:36.939 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:27:36.950 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:27:36.959 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:27:36.959 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:27:36.959 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:36.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:27:36.969 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T10:27:36.969 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:37.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:27:37.021 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:27:37.021 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:37.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:37.030 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T10:27:37.031 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:37.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3235 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:37.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.083 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:27:37.092 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:27:37.092 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:27:37.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:27:37.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.159 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:27:37.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T10:27:37.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.399 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T10:27:37.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.409 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T10:27:37.474 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-04-25T10:27:37.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:27:37.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.476 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T10:27:37.486 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:27:37.486 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:27:37.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:27:37.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:27:37.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.624 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T10:27:37.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.773 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T10:27:37.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.782 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T10:27:37.851 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-25T10:27:37.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:27:37.853 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:27:37.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:27:37.864 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:27:38.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6caa00640 1 Processor -- start 2024-04-25T10:27:38.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6caa00640 1 -- start start 2024-04-25T10:27:38.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6caa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe6c40ccb20 0x7fe6c40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6caa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe6c415bdc0 0x7fe6c40d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6caa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c40ccfb0 con 0x7fe6c415b9e0 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6caa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40cd290 con 0x7fe6c415bdc0 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6caa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40cd570 con 0x7fe6c40ccb20 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe6c415bdc0 0x7fe6c40d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe6c415b9e0 0x7fe6c40cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33562/0 (socket says 172.21.15.87:33562) 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c3e00640 1 -- 172.21.15.87:0/3440405155 learned_addr learned my addr 172.21.15.87:0/3440405155 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c8a00640 1 -- 172.21.15.87:0/3440405155 >> v2:172.21.15.169:3300/0 conn(0x7fe6c40ccb20 msgr2=0x7fe6c40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c8a00640 1 --2- 172.21.15.87:0/3440405155 >> v2:172.21.15.169:3300/0 conn(0x7fe6c40ccb20 0x7fe6c40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c8a00640 1 -- 172.21.15.87:0/3440405155 >> v1:172.21.15.87:6789/0 conn(0x7fe6c415b9e0 legacy=0x7fe6c40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c8a00640 1 -- 172.21.15.87:0/3440405155 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40cd850 con 0x7fe6c415bdc0 2024-04-25T10:27:38.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.005+0000 7fe6c3400640 1 -- 172.21.15.87:0/3440405155 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057693198 0 0) 0x7fe6c40ccfb0 con 0x7fe6c415b9e0 2024-04-25T10:27:38.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6c8a00640 1 --2- 172.21.15.87:0/3440405155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415bdc0 0x7fe6c40d1a60 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4067980 tx=0x7fe6bc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1747c4fb748bfcbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6c3400640 1 -- 172.21.15.87:0/3440405155 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0b2070 con 0x7fe6c415bdc0 2024-04-25T10:27:38.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6c3400640 1 -- 172.21.15.87:0/3440405155 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0a57f0 con 0x7fe6c415bdc0 2024-04-25T10:27:38.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6c3400640 1 -- 172.21.15.87:0/3440405155 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0a5bc0 con 0x7fe6c415bdc0 2024-04-25T10:27:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 -- 172.21.15.87:0/3440405155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415bdc0 msgr2=0x7fe6c40d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 --2- 172.21.15.87:0/3440405155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415bdc0 0x7fe6c40d1a60 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4067980 tx=0x7fe6bc01c930 comp rx=0 tx=0).stop 2024-04-25T10:27:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 -- 172.21.15.87:0/3440405155 shutdown_connections 2024-04-25T10:27:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 --2- 172.21.15.87:0/3440405155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415bdc0 0x7fe6c40d1a60 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 --2- 172.21.15.87:0/3440405155 >> v2:172.21.15.169:3300/0 conn(0x7fe6c40ccb20 0x7fe6c40d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 -- 172.21.15.87:0/3440405155 >> 172.21.15.87:0/3440405155 conn(0x7fe6c409a9d0 msgr2=0x7fe6c409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:38.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 -- 172.21.15.87:0/3440405155 shutdown_connections 2024-04-25T10:27:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 -- 172.21.15.87:0/3440405155 wait complete. 2024-04-25T10:27:38.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 Processor -- start 2024-04-25T10:27:38.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.009+0000 7fe6caa00640 1 -- start start 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415b9e0 0x7fe6c415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c415bdc0 0x7fe6c4154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c4154a50 0x7fe6c41650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c65b0 con 0x7fe6c415b9e0 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c6430 con 0x7fe6c4154a50 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c6730 con 0x7fe6c415bdc0 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415b9e0 0x7fe6c415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415b9e0 0x7fe6c415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58548/0 (socket says 172.21.15.87:58548) 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 -- 172.21.15.87:0/244850915 learned_addr learned my addr 172.21.15.87:0/244850915 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c8a00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c4154a50 0x7fe6c41650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6bbe00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c415bdc0 0x7fe6c4154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 -- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c415bdc0 msgr2=0x7fe6c4154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c415bdc0 0x7fe6c4154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 -- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c4154a50 msgr2=0x7fe6c41650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c4154a50 0x7fe6c41650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 -- 172.21.15.87:0/244850915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c415c1a0 con 0x7fe6c415b9e0 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6bbe00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c415bdc0 0x7fe6c4154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c3e00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415b9e0 0x7fe6c415a0e0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40011f0 tx=0x7fe6b400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c0c00640 1 -- 172.21.15.87:0/244850915 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4018070 con 0x7fe6c415b9e0 2024-04-25T10:27:38.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c0c00640 1 -- 172.21.15.87:0/244850915 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b40074e0 con 0x7fe6c415b9e0 2024-04-25T10:27:38.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c0c00640 1 -- 172.21.15.87:0/244850915 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4004a10 con 0x7fe6c415b9e0 2024-04-25T10:27:38.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c415a620 con 0x7fe6c415b9e0 2024-04-25T10:27:38.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c415c740 con 0x7fe6c415b9e0 2024-04-25T10:27:38.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c404ef30 con 0x7fe6c415b9e0 2024-04-25T10:27:38.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.013+0000 7fe6c0c00640 1 -- 172.21.15.87:0/244850915 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6b401b070 con 0x7fe6c415b9e0 2024-04-25T10:27:38.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.017+0000 7fe6c0c00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.017+0000 7fe6c0c00640 1 -- 172.21.15.87:0/244850915 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe6b4084150 con 0x7fe6c415b9e0 2024-04-25T10:27:38.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.017+0000 7fe6bbe00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.017+0000 7fe6bbe00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6a0066f60 0x7fe6a0069420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4165f10 tx=0x7fe6ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.021+0000 7fe6c0c00640 1 -- 172.21.15.87:0/244850915 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6b4055360 con 0x7fe6c415b9e0 2024-04-25T10:27:38.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.169+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6c415a850 con 0x7fe6a0066f60 2024-04-25T10:27:38.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6c0c00640 1 -- 172.21.15.87:0/244850915 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13500 (secure 0 0 0) 0x7fe6c415a850 con 0x7fe6a0066f60 2024-04-25T10:27:38.169 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:38.170 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6a0066f60 msgr2=0x7fe6a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6a0066f60 0x7fe6a0069420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4165f10 tx=0x7fe6ac008040 comp rx=0 tx=0).stop 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415b9e0 msgr2=0x7fe6c415a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415b9e0 0x7fe6c415a0e0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40011f0 tx=0x7fe6b400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 shutdown_connections 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c4154a50 0x7fe6c41650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c415bdc0 0x7fe6c4154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 --2- 172.21.15.87:0/244850915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c415b9e0 0x7fe6c415a0e0 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 >> 172.21.15.87:0/244850915 conn(0x7fe6c409a9d0 msgr2=0x7fe6c406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 shutdown_connections 2024-04-25T10:27:38.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.173+0000 7fe6caa00640 1 -- 172.21.15.87:0/244850915 wait complete. 2024-04-25T10:27:38.191 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2024-04-25T10:27:37.323394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":485212,"kb_used_data":34536,"kb_used_omap":12,"kb_used_meta":450547,"kb_avail":754489508,"statfs":{"total":773094113280,"available":772597256192,"internally_reserved":0,"allocated":35364864,"data_stored":24443200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":461360689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12622614,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001316"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":486,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:27:34.907879+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:27:34.907879+0000","last_peered":"2024-04-25T10:27:34.907879+0000","last_clean":"2024-04-25T10:27:34.907879+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:27:34.907879+0000","last_undegraded":"2024-04-25T10:27:34.907879+0000","last_fullsized":"2024-04-25T10:27:34.907879+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48044,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323796,"statfs":{"total":96636764160,"available":96587567104,"internally_reserved":0,"allocated":3039232,"data_stored":1666036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65440,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306400,"statfs":{"total":96636764160,"available":96569753600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51616,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320224,"statfs":{"total":96636764160,"available":96583909376,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64080,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307760,"statfs":{"total":96636764160,"available":96571146240,"internally_reserved":0,"allocated":2428928,"data_stored":1069932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59940,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63956,"kb_used_data":10564,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307884,"statfs":{"total":96636764160,"available":96571273216,"internally_reserved":0,"allocated":10817536,"data_stored":9458540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71696,"kb_used_data":4420,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300144,"statfs":{"total":96636764160,"available":96563347456,"internally_reserved":0,"allocated":4526080,"data_stored":3167084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60440,"kb_used_data":7048,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311400,"statfs":{"total":96636764160,"available":96574873600,"internally_reserved":0,"allocated":7217152,"data_stored":5854604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:27:38.191 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:27:38.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.329+0000 7f4262600640 1 Processor -- start 2024-04-25T10:27:38.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.329+0000 7f4262600640 1 -- start start 2024-04-25T10:27:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.329+0000 7f4262600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c15bc40 0x7f425c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.329+0000 7f4262600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f425c0cda10 0x7f425c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.329+0000 7f4262600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c0b2770 con 0x7f425c0cf930 2024-04-25T10:27:38.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.329+0000 7f4262600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c06c670 con 0x7f425c0cda10 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.329+0000 7f4262600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c06c950 con 0x7f425c15bc40 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c15bc40 0x7f425c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f425c0cda10 0x7f425c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c15bc40 0x7f425c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.87:35534/0 (socket says 172.21.15.87:35534) 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 -- 172.21.15.87:0/3760203614 learned_addr learned my addr 172.21.15.87:0/3760203614 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425be00640 1 --1- 172.21.15.87:0/3760203614 >> v1:172.21.15.169:6789/0 conn(0x7f425c0cf930 0x7f425c15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39400/0 (socket says 172.21.15.87:39400) 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 -- 172.21.15.87:0/3760203614 >> v2:172.21.15.169:3300/0 conn(0x7f425c0cda10 msgr2=0x7f425c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 --2- 172.21.15.87:0/3760203614 >> v2:172.21.15.169:3300/0 conn(0x7f425c0cda10 0x7f425c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 -- 172.21.15.87:0/3760203614 >> v1:172.21.15.169:6789/0 conn(0x7f425c0cf930 legacy=0x7f425c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 -- 172.21.15.87:0/3760203614 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c06cc30 con 0x7f425c15bc40 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425aa00640 1 --2- 172.21.15.87:0/3760203614 >> v2:172.21.15.169:3300/0 conn(0x7f425c0cda10 0x7f425c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:38.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425a000640 1 -- 172.21.15.87:0/3760203614 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875416260 0 0) 0x7f425c0b2770 con 0x7f425c0cf930 2024-04-25T10:27:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425b400640 1 --2- 172.21.15.87:0/3760203614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c15bc40 0x7f425c06c090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f424c009e50 tx=0x7f424c01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7216843a802f6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425a000640 1 -- 172.21.15.87:0/3760203614 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c0a5810 con 0x7f425c15bc40 2024-04-25T10:27:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425a000640 1 -- 172.21.15.87:0/3760203614 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c0a59b0 con 0x7f425c15bc40 2024-04-25T10:27:38.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.333+0000 7f425a000640 1 -- 172.21.15.87:0/3760203614 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c0a5d80 con 0x7f425c15bc40 2024-04-25T10:27:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- 172.21.15.87:0/3760203614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c15bc40 msgr2=0x7f425c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 --2- 172.21.15.87:0/3760203614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c15bc40 0x7f425c06c090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f424c009e50 tx=0x7f424c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:27:38.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- 172.21.15.87:0/3760203614 shutdown_connections 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 --2- 172.21.15.87:0/3760203614 >> v2:172.21.15.169:3300/0 conn(0x7f425c0cda10 0x7f425c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 --2- 172.21.15.87:0/3760203614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c15bc40 0x7f425c06c090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- 172.21.15.87:0/3760203614 >> 172.21.15.87:0/3760203614 conn(0x7f425c066b70 msgr2=0x7f425c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- 172.21.15.87:0/3760203614 shutdown_connections 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- 172.21.15.87:0/3760203614 wait complete. 2024-04-25T10:27:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 Processor -- start 2024-04-25T10:27:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- start start 2024-04-25T10:27:38.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f425c0cda10 0x7f425c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c0cf930 0x7f425c14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f425c15bc40 0x7f425c0b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c168b50 con 0x7f425c0cda10 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1689d0 con 0x7f425c15bc40 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f4262600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c168cd0 con 0x7f425c0cf930 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f425c0cda10 0x7f425c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f425c0cda10 0x7f425c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58562/0 (socket says 172.21.15.87:58562) 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 -- 172.21.15.87:0/2069615483 learned_addr learned my addr 172.21.15.87:0/2069615483 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425be00640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f425c15bc40 0x7f425c0b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425aa00640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c0cf930 0x7f425c14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 -- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c0cf930 msgr2=0x7f425c14a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c0cf930 0x7f425c14a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 -- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f425c15bc40 msgr2=0x7f425c0b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f425c15bc40 0x7f425c0b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425b400640 1 -- 172.21.15.87:0/2069615483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0b2260 con 0x7f425c0cda10 2024-04-25T10:27:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.337+0000 7f425aa00640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c0cf930 0x7f425c14a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f425b400640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f425c0cda10 0x7f425c0af3e0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f424c01a660 tx=0x7f424c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4253400640 1 -- 172.21.15.87:0/2069615483 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c0bc020 con 0x7f425c0cda10 2024-04-25T10:27:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4253400640 1 -- 172.21.15.87:0/2069615483 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c0b8670 con 0x7f425c0cda10 2024-04-25T10:27:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c153250 con 0x7f425c0cda10 2024-04-25T10:27:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4253400640 1 -- 172.21.15.87:0/2069615483 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c0b8a40 con 0x7f425c0cda10 2024-04-25T10:27:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c153810 con 0x7f425c0cda10 2024-04-25T10:27:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c04ef30 con 0x7f425c0cda10 2024-04-25T10:27:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4253400640 1 -- 172.21.15.87:0/2069615483 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f424c0bf020 con 0x7f425c0cda10 2024-04-25T10:27:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4253400640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f423c066f60 0x7f423c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f4253400640 1 -- 172.21.15.87:0/2069615483 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f424c11fcf0 con 0x7f425c0cda10 2024-04-25T10:27:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.341+0000 7f425aa00640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f423c066f60 0x7f423c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.345+0000 7f425aa00640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f423c066f60 0x7f423c069420 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f425c0b15f0 tx=0x7f4248008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.345+0000 7f4253400640 1 -- 172.21.15.87:0/2069615483 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f424c0c6030 con 0x7f425c0cda10 2024-04-25T10:27:38.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.497+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f425c06d960 con 0x7f423c066f60 2024-04-25T10:27:38.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.497+0000 7f4253400640 1 -- 172.21.15.87:0/2069615483 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13500 (secure 0 0 0) 0x7f425c06d960 con 0x7f423c066f60 2024-04-25T10:27:38.496 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:38.496 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:27:38.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f423c066f60 msgr2=0x7f423c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f423c066f60 0x7f423c069420 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f425c0b15f0 tx=0x7f4248008040 comp rx=0 tx=0).stop 2024-04-25T10:27:38.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f425c0cda10 msgr2=0x7f425c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f425c0cda10 0x7f425c0af3e0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f424c01a660 tx=0x7f424c0197d0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 shutdown_connections 2024-04-25T10:27:38.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f423c066f60 0x7f423c069420 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f425c15bc40 0x7f425c0b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f425c0cf930 0x7f425c14a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 --2- 172.21.15.87:0/2069615483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f425c0cda10 0x7f425c0af3e0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 >> 172.21.15.87:0/2069615483 conn(0x7f425c066b70 msgr2=0x7f425c158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:38.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 shutdown_connections 2024-04-25T10:27:38.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.501+0000 7f4262600640 1 -- 172.21.15.87:0/2069615483 wait complete. 2024-04-25T10:27:38.517 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2024-04-25T10:27:37.323394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":485212,"kb_used_data":34536,"kb_used_omap":12,"kb_used_meta":450547,"kb_avail":754489508,"statfs":{"total":773094113280,"available":772597256192,"internally_reserved":0,"allocated":35364864,"data_stored":24443200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":461360689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12622614,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001316"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":486,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:27:34.907879+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:27:34.907879+0000","last_peered":"2024-04-25T10:27:34.907879+0000","last_clean":"2024-04-25T10:27:34.907879+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:27:34.907879+0000","last_undegraded":"2024-04-25T10:27:34.907879+0000","last_fullsized":"2024-04-25T10:27:34.907879+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48044,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323796,"statfs":{"total":96636764160,"available":96587567104,"internally_reserved":0,"allocated":3039232,"data_stored":1666036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65440,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306400,"statfs":{"total":96636764160,"available":96569753600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51616,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320224,"statfs":{"total":96636764160,"available":96583909376,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64080,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307760,"statfs":{"total":96636764160,"available":96571146240,"internally_reserved":0,"allocated":2428928,"data_stored":1069932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59940,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63956,"kb_used_data":10564,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307884,"statfs":{"total":96636764160,"available":96571273216,"internally_reserved":0,"allocated":10817536,"data_stored":9458540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71696,"kb_used_data":4420,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300144,"statfs":{"total":96636764160,"available":96563347456,"internally_reserved":0,"allocated":4526080,"data_stored":3167084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60440,"kb_used_data":7048,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311400,"statfs":{"total":96636764160,"available":96574873600,"internally_reserved":0,"allocated":7217152,"data_stored":5854604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:27:38.517 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:27:38.517 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:27:38.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 Processor -- start 2024-04-25T10:27:38.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 -- start start 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e300cbf10 0x7f1e3006c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300728d0 0x7f1e3006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1e300cc2f0 0x7f1e3006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e300cb040 con 0x7f1e300cbf10 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e300ca430 con 0x7f1e300cc2f0 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300ca710 con 0x7f1e300728d0 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e300cbf10 0x7f1e3006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e300cbf10 0x7f1e3006c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58576/0 (socket says 172.21.15.87:58576) 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 -- 172.21.15.87:0/1289321680 learned_addr learned my addr 172.21.15.87:0/1289321680 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36200640 1 --2- 172.21.15.87:0/1289321680 >> v2:172.21.15.169:3300/0 conn(0x7f1e300cc2f0 0x7f1e3006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e34e00640 1 --2- 172.21.15.87:0/1289321680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300728d0 0x7f1e3006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 -- 172.21.15.87:0/1289321680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300728d0 msgr2=0x7f1e3006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:38.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 --2- 172.21.15.87:0/1289321680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300728d0 0x7f1e3006cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 -- 172.21.15.87:0/1289321680 >> v2:172.21.15.169:3300/0 conn(0x7f1e300cc2f0 msgr2=0x7f1e3006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:38.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 --2- 172.21.15.87:0/1289321680 >> v2:172.21.15.169:3300/0 conn(0x7f1e300cc2f0 0x7f1e3006d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e35800640 1 -- 172.21.15.87:0/1289321680 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300ca9f0 con 0x7f1e300cbf10 2024-04-25T10:27:38.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.657+0000 7f1e36200640 1 --2- 172.21.15.87:0/1289321680 >> v2:172.21.15.169:3300/0 conn(0x7f1e300cc2f0 0x7f1e3006d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:38.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e35800640 1 --2- 172.21.15.87:0/1289321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300cbf10 0x7f1e3006c720 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f1e300af650 tx=0x7f1e2401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2893735b2f0a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e2fe00640 1 -- 172.21.15.87:0/1289321680 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240ab640 con 0x7f1e300cbf10 2024-04-25T10:27:38.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e2fe00640 1 -- 172.21.15.87:0/1289321680 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e240b0070 con 0x7f1e300cbf10 2024-04-25T10:27:38.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e2fe00640 1 -- 172.21.15.87:0/1289321680 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240aba00 con 0x7f1e300cbf10 2024-04-25T10:27:38.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 -- 172.21.15.87:0/1289321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300cbf10 msgr2=0x7f1e3006c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1289321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300cbf10 0x7f1e3006c720 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f1e300af650 tx=0x7f1e2401c460 comp rx=0 tx=0).stop 2024-04-25T10:27:38.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 -- 172.21.15.87:0/1289321680 shutdown_connections 2024-04-25T10:27:38.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1289321680 >> v2:172.21.15.169:3300/0 conn(0x7f1e300cc2f0 0x7f1e3006d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1289321680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300728d0 0x7f1e3006cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1289321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300cbf10 0x7f1e3006c720 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 -- 172.21.15.87:0/1289321680 >> 172.21.15.87:0/1289321680 conn(0x7f1e300cfde0 msgr2=0x7f1e300d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:38.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 -- 172.21.15.87:0/1289321680 shutdown_connections 2024-04-25T10:27:38.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 -- 172.21.15.87:0/1289321680 wait complete. 2024-04-25T10:27:38.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 Processor -- start 2024-04-25T10:27:38.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.661+0000 7f1e36c00640 1 -- start start 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300728d0 0x7f1e30158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300cbf10 0x7f1e300c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e300cc2f0 0x7f1e300c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30167690 con 0x7f1e300728d0 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30073f30 con 0x7f1e300cc2f0 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30167810 con 0x7f1e300cbf10 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300728d0 0x7f1e30158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300728d0 0x7f1e30158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58586/0 (socket says 172.21.15.87:58586) 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 -- 172.21.15.87:0/1113528821 learned_addr learned my addr 172.21.15.87:0/1113528821 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36200640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e300cc2f0 0x7f1e300c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e34e00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300cbf10 0x7f1e300c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 -- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300cbf10 msgr2=0x7f1e300c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300cbf10 0x7f1e300c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 -- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e300cc2f0 msgr2=0x7f1e300c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e300cc2f0 0x7f1e300c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 -- 172.21.15.87:0/1113528821 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300aea60 con 0x7f1e300728d0 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e35800640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300728d0 0x7f1e30158bf0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ad040 tx=0x7f1e2401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e2d600640 1 -- 172.21.15.87:0/1113528821 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240be3f0 con 0x7f1e300728d0 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e2d600640 1 -- 172.21.15.87:0/1113528821 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e240be5b0 con 0x7f1e300728d0 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e301597c0 con 0x7f1e300728d0 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e2d600640 1 -- 172.21.15.87:0/1113528821 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240be9a0 con 0x7f1e300728d0 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e300aed00 con 0x7f1e300728d0 2024-04-25T10:27:38.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.665+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e30075120 con 0x7f1e300728d0 2024-04-25T10:27:38.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.669+0000 7f1e2d600640 1 -- 172.21.15.87:0/1113528821 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e240c8420 con 0x7f1e300728d0 2024-04-25T10:27:38.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.669+0000 7f1e2d600640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1e10066f60 0x7f1e10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.669+0000 7f1e2d600640 1 -- 172.21.15.87:0/1113528821 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1e2411f500 con 0x7f1e300728d0 2024-04-25T10:27:38.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.669+0000 7f1e34e00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1e10066f60 0x7f1e10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.669+0000 7f1e34e00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1e10066f60 0x7f1e10069420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ce650 tx=0x7f1e1c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.673+0000 7f1e2d600640 1 -- 172.21.15.87:0/1113528821 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e240c4060 con 0x7f1e300728d0 2024-04-25T10:27:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.821+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e300b0b70 con 0x7f1e10066f60 2024-04-25T10:27:38.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.821+0000 7f1e2d600640 1 -- 172.21.15.87:0/1113528821 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13500 (secure 0 0 0) 0x7f1e300b0b70 con 0x7f1e10066f60 2024-04-25T10:27:38.820 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:38.820 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:27:38.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1e10066f60 msgr2=0x7f1e10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1e10066f60 0x7f1e10069420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ce650 tx=0x7f1e1c008040 comp rx=0 tx=0).stop 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300728d0 msgr2=0x7f1e30158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300728d0 0x7f1e30158bf0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ad040 tx=0x7f1e2401aca0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 shutdown_connections 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1e10066f60 0x7f1e10069420 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e300cc2f0 0x7f1e300c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e300cbf10 0x7f1e300c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 --2- 172.21.15.87:0/1113528821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e300728d0 0x7f1e30158bf0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 >> 172.21.15.87:0/1113528821 conn(0x7f1e300cfde0 msgr2=0x7f1e300b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 shutdown_connections 2024-04-25T10:27:38.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.825+0000 7f1e36c00640 1 -- 172.21.15.87:0/1113528821 wait complete. 2024-04-25T10:27:38.840 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2024-04-25T10:27:37.323394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":485212,"kb_used_data":34536,"kb_used_omap":12,"kb_used_meta":450547,"kb_avail":754489508,"statfs":{"total":773094113280,"available":772597256192,"internally_reserved":0,"allocated":35364864,"data_stored":24443200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":461360689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12622614,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001316"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":486,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:27:34.907879+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:27:34.907879+0000","last_peered":"2024-04-25T10:27:34.907879+0000","last_clean":"2024-04-25T10:27:34.907879+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:27:34.907879+0000","last_undegraded":"2024-04-25T10:27:34.907879+0000","last_fullsized":"2024-04-25T10:27:34.907879+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48044,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323796,"statfs":{"total":96636764160,"available":96587567104,"internally_reserved":0,"allocated":3039232,"data_stored":1666036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65440,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306400,"statfs":{"total":96636764160,"available":96569753600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51616,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320224,"statfs":{"total":96636764160,"available":96583909376,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64080,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307760,"statfs":{"total":96636764160,"available":96571146240,"internally_reserved":0,"allocated":2428928,"data_stored":1069932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59940,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63956,"kb_used_data":10564,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307884,"statfs":{"total":96636764160,"available":96571273216,"internally_reserved":0,"allocated":10817536,"data_stored":9458540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71696,"kb_used_data":4420,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300144,"statfs":{"total":96636764160,"available":96563347456,"internally_reserved":0,"allocated":4526080,"data_stored":3167084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60440,"kb_used_data":7048,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311400,"statfs":{"total":96636764160,"available":96574873600,"internally_reserved":0,"allocated":7217152,"data_stored":5854604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:27:38.841 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:38.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c7c00640 1 Processor -- start 2024-04-25T10:27:38.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c7c00640 1 -- start start 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c7c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08c00b0f70 0x7f08c00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c7c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c7c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c015f3b0 con 0x7f08c00cfd90 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c7c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c015ef20 con 0x7f08c00b0f70 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c7c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00ca810 con 0x7f08c00cd2e0 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08bfe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f08c00cfd90 0x7f08c00ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33566/0 (socket says 172.21.15.87:33566) 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08bfe00640 1 -- 172.21.15.87:0/867855622 learned_addr learned my addr 172.21.15.87:0/867855622 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c5200640 1 --2- 172.21.15.87:0/867855622 >> v2:172.21.15.169:3300/0 conn(0x7f08c00b0f70 0x7f08c00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c5c00640 1 --2- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08bf400640 1 -- 172.21.15.87:0/867855622 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715393092 0 0) 0x7f08c015f3b0 con 0x7f08c00cfd90 2024-04-25T10:27:38.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08bf400640 1 -- 172.21.15.87:0/867855622 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a8003530 con 0x7f08c00cfd90 2024-04-25T10:27:38.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c5200640 1 -- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 msgr2=0x7f08c00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:38.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c5200640 1 --2- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c00cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c5200640 1 -- 172.21.15.87:0/867855622 >> v1:172.21.15.87:6789/0 conn(0x7f08c00cfd90 legacy=0x7f08c00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.981+0000 7f08c5200640 1 -- 172.21.15.87:0/867855622 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00b1890 con 0x7f08c00b0f70 2024-04-25T10:27:38.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c5200640 1 --2- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 0x7f08c00b1350 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f08b40097c0 tx=0x7f08b401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f269b16e285c5e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08bf400640 1 -- 172.21.15.87:0/867855622 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40b0070 con 0x7f08c00b0f70 2024-04-25T10:27:38.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08bf400640 1 -- 172.21.15.87:0/867855622 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b40ab600 con 0x7f08c00b0f70 2024-04-25T10:27:38.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08bf400640 1 -- 172.21.15.87:0/867855622 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40ab9b0 con 0x7f08c00b0f70 2024-04-25T10:27:38.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 -- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 msgr2=0x7f08c00b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 --2- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 0x7f08c00b1350 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f08b40097c0 tx=0x7f08b401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 -- 172.21.15.87:0/867855622 shutdown_connections 2024-04-25T10:27:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 --2- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c00cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 --2- 172.21.15.87:0/867855622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 0x7f08c00b1350 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 -- 172.21.15.87:0/867855622 >> 172.21.15.87:0/867855622 conn(0x7f08c00b1d50 msgr2=0x7f08c00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 -- 172.21.15.87:0/867855622 shutdown_connections 2024-04-25T10:27:38.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.985+0000 7f08c7c00640 1 -- 172.21.15.87:0/867855622 wait complete. 2024-04-25T10:27:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 Processor -- start 2024-04-25T10:27:38.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 -- start start 2024-04-25T10:27:38.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 0x7f08c00c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c0099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00cfd90 0x7f08c00c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0161b50 con 0x7f08c00cfd90 2024-04-25T10:27:38.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01619d0 con 0x7f08c00b0f70 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0161cd0 con 0x7f08c00cd2e0 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00cfd90 0x7f08c00c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00cfd90 0x7f08c00c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58600/0 (socket says 172.21.15.87:58600) 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 -- 172.21.15.87:0/171010710 learned_addr learned my addr 172.21.15.87:0/171010710 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08bfe00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c0099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 -- 172.21.15.87:0/171010710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 msgr2=0x7f08c0099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c0099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 -- 172.21.15.87:0/171010710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 msgr2=0x7f08c00c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 0x7f08c00c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 -- 172.21.15.87:0/171010710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00c53e0 con 0x7f08c00cfd90 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c5c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00cfd90 0x7f08c00c6440 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f08b800efc0 tx=0x7f08b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08bcc00640 1 -- 172.21.15.87:0/171010710 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8018020 con 0x7f08c00cfd90 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08bcc00640 1 -- 172.21.15.87:0/171010710 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b8014480 con 0x7f08c00cfd90 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c00c5680 con 0x7f08c00cfd90 2024-04-25T10:27:38.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08bcc00640 1 -- 172.21.15.87:0/171010710 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8014870 con 0x7f08c00cfd90 2024-04-25T10:27:38.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c00c5c40 con 0x7f08c00cfd90 2024-04-25T10:27:38.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.989+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c004ef30 con 0x7f08c00cfd90 2024-04-25T10:27:38.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.993+0000 7f08bcc00640 1 -- 172.21.15.87:0/171010710 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08b801b020 con 0x7f08c00cfd90 2024-04-25T10:27:38.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.993+0000 7f08bcc00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a0066f60 0x7f08a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:38.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.993+0000 7f08bcc00640 1 -- 172.21.15.87:0/171010710 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f08b80858d0 con 0x7f08c00cfd90 2024-04-25T10:27:38.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.993+0000 7f08c5200640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a0066f60 0x7f08a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:38.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.993+0000 7f08c5200640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a0066f60 0x7f08a0069420 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f08b40ad000 tx=0x7f08b4004740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:38.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:38.997+0000 7f08bcc00640 1 -- 172.21.15.87:0/171010710 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08b8022070 con 0x7f08c00cfd90 2024-04-25T10:27:39.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.157+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08c00cda20 con 0x7f08c00cfd90 2024-04-25T10:27:39.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.157+0000 7f08bcc00640 1 -- 172.21.15.87:0/171010710 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+13146 (secure 0 0 0) 0x7f08b8010040 con 0x7f08c00cfd90 2024-04-25T10:27:39.155 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:39.155 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":193,"fsid":"84c5cae3-3d36-4b8f-9ce8-349d9b4e086b","created":"2024-04-25T10:11:58.086259+0000","modified":"2024-04-25T10:27:34.899665+0000","last_up_change":"2024-04-25T10:12:10.411313+0000","last_in_change":"2024-04-25T10:12:00.631609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:12:11.188224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9fd7de2-a5ee-4694-9239-9f30b30d37da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6801","nonce":2221934810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6805","nonce":2221934810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6814","nonce":2221934810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":2221934810},{"type":"v1","addr":"172.21.15.87:6809","nonce":2221934810}]},"public_addr":"172.21.15.87:6801/2221934810","cluster_addr":"172.21.15.87:6805/2221934810","heartbeat_back_addr":"172.21.15.87:6814/2221934810","heartbeat_front_addr":"172.21.15.87:6809/2221934810","state":["exists","up"]},{"osd":1,"uuid":"7baf96bf-f0b4-4b34-b5ca-78bcd8118464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6817","nonce":191811739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6819","nonce":191811739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6823","nonce":191811739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":191811739},{"type":"v1","addr":"172.21.15.87:6821","nonce":191811739}]},"public_addr":"172.21.15.87:6817/191811739","cluster_addr":"172.21.15.87:6819/191811739","heartbeat_back_addr":"172.21.15.87:6823/191811739","heartbeat_front_addr":"172.21.15.87:6821/191811739","state":["exists","up"]},{"osd":2,"uuid":"7bc3a5c5-e581-4d3f-827e-038a5b979a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6825","nonce":2913449321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6827","nonce":2913449321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6831","nonce":2913449321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2913449321},{"type":"v1","addr":"172.21.15.87:6829","nonce":2913449321}]},"public_addr":"172.21.15.87:6825/2913449321","cluster_addr":"172.21.15.87:6827/2913449321","heartbeat_back_addr":"172.21.15.87:6831/2913449321","heartbeat_front_addr":"172.21.15.87:6829/2913449321","state":["exists","up"]},{"osd":3,"uuid":"cff9fb22-0831-4513-94ef-d7839c7f5b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6804","nonce":327355267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6808","nonce":327355267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6815","nonce":327355267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":327355267},{"type":"v1","addr":"172.21.15.87:6812","nonce":327355267}]},"public_addr":"172.21.15.87:6804/327355267","cluster_addr":"172.21.15.87:6808/327355267","heartbeat_back_addr":"172.21.15.87:6815/327355267","heartbeat_front_addr":"172.21.15.87:6812/327355267","state":["exists","up"]},{"osd":4,"uuid":"ccc3700f-0f90-438a-b39a-86e71ffa860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6801","nonce":3149326747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6803","nonce":3149326747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6807","nonce":3149326747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3149326747},{"type":"v1","addr":"172.21.15.169:6805","nonce":3149326747}]},"public_addr":"172.21.15.169:6801/3149326747","cluster_addr":"172.21.15.169:6803/3149326747","heartbeat_back_addr":"172.21.15.169:6807/3149326747","heartbeat_front_addr":"172.21.15.169:6805/3149326747","state":["exists","up"]},{"osd":5,"uuid":"5a7c34b3-a7cc-451c-b735-4aa82ca6a6c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6811","nonce":1054103150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6817","nonce":1054103150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6823","nonce":1054103150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1054103150},{"type":"v1","addr":"172.21.15.169:6821","nonce":1054103150}]},"public_addr":"172.21.15.169:6811/1054103150","cluster_addr":"172.21.15.169:6817/1054103150","heartbeat_back_addr":"172.21.15.169:6823/1054103150","heartbeat_front_addr":"172.21.15.169:6821/1054103150","state":["exists","up"]},{"osd":6,"uuid":"4a593c61-a6b9-47a5-a008-fc0837c29d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6810","nonce":3889701998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6813","nonce":3889701998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6819","nonce":3889701998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":3889701998},{"type":"v1","addr":"172.21.15.169:6816","nonce":3889701998}]},"public_addr":"172.21.15.169:6810/3889701998","cluster_addr":"172.21.15.169:6813/3889701998","heartbeat_back_addr":"172.21.15.169:6819/3889701998","heartbeat_front_addr":"172.21.15.169:6816/3889701998","state":["exists","up"]},{"osd":7,"uuid":"d3a5c2d1-069c-4959-bcd3-75418023a734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6825","nonce":2370587194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6827","nonce":2370587194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6831","nonce":2370587194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2370587194},{"type":"v1","addr":"172.21.15.169:6829","nonce":2370587194}]},"public_addr":"172.21.15.169:6825/2370587194","cluster_addr":"172.21.15.169:6827/2370587194","heartbeat_back_addr":"172.21.15.169:6831/2370587194","heartbeat_front_addr":"172.21.15.169:6829/2370587194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.462082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.517649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.425874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.348478+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.308926+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.587136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.599564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:12:06.431880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6836/3730332371":"2024-04-26T10:27:31.885550+0000","172.21.15.87:6834/3730332371":"2024-04-26T10:27:31.885550+0000","172.21.15.87:6836/997618220":"2024-04-26T10:26:20.143429+0000","172.21.15.87:6834/997618220":"2024-04-26T10:26:20.143429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a0066f60 msgr2=0x7f08a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a0066f60 0x7f08a0069420 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f08b40ad000 tx=0x7f08b4004740 comp rx=0 tx=0).stop 2024-04-25T10:27:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00cfd90 msgr2=0x7f08c00c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00cfd90 0x7f08c00c6440 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f08b800efc0 tx=0x7f08b800c490 comp rx=0 tx=0).stop 2024-04-25T10:27:39.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 shutdown_connections 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f08a0066f60 0x7f08a0069420 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08c00cfd90 0x7f08c00c6440 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c00cd2e0 0x7f08c0099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 --2- 172.21.15.87:0/171010710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00b0f70 0x7f08c00c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 >> 172.21.15.87:0/171010710 conn(0x7f08c00b1d50 msgr2=0x7f08c0160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 shutdown_connections 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:39.161+0000 7f08c7c00640 1 -- 172.21.15.87:0/171010710 wait complete. 2024-04-25T10:27:40.177 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:27:40.177 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:40.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.325+0000 7fc4fb400640 1 Processor -- start 2024-04-25T10:27:40.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.325+0000 7fc4fb400640 1 -- start start 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.325+0000 7fc4fb400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc4f415c0f0 0x7fc4f40b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.325+0000 7fc4fb400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f406c0e0 con 0x7fc4f40cf9e0 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.325+0000 7fc4fb400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f415c6a0 con 0x7fc4f415bd10 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.325+0000 7fc4fb400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f415c980 con 0x7fc4f415c0f0 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc4f415c0f0 0x7fc4f40b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc4f415c0f0 0x7fc4f40b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58614/0 (socket says 172.21.15.87:58614) 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3e00640 1 -- 172.21.15.87:0/1271284214 learned_addr learned my addr 172.21.15.87:0/1271284214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:40.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f9400640 1 --1- 172.21.15.87:0/1271284214 >> v1:172.21.15.87:6789/0 conn(0x7fc4f40cf9e0 0x7fc4f406c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33580/0 (socket says 172.21.15.87:33580) 2024-04-25T10:27:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3e00640 1 -- 172.21.15.87:0/1271284214 >> v1:172.21.15.169:6789/0 conn(0x7fc4f415bd10 legacy=0x7fc4f40b4610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:27:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3e00640 1 -- 172.21.15.87:0/1271284214 >> v1:172.21.15.87:6789/0 conn(0x7fc4f40cf9e0 legacy=0x7fc4f406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3e00640 1 -- 172.21.15.87:0/1271284214 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f415cc60 con 0x7fc4f415c0f0 2024-04-25T10:27:40.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3e00640 1 --2- 172.21.15.87:0/1271284214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40b27e0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009870 tx=0x7fc4e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21c02edd19b66af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3400640 1 -- 172.21.15.87:0/1271284214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00ab430 con 0x7fc4f415c0f0 2024-04-25T10:27:40.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3400640 1 -- 172.21.15.87:0/1271284214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e00b0070 con 0x7fc4f415c0f0 2024-04-25T10:27:40.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.329+0000 7fc4f3400640 1 -- 172.21.15.87:0/1271284214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00ab7f0 con 0x7fc4f415c0f0 2024-04-25T10:27:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- 172.21.15.87:0/1271284214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 msgr2=0x7fc4f40b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 --2- 172.21.15.87:0/1271284214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40b27e0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009870 tx=0x7fc4e001c520 comp rx=0 tx=0).stop 2024-04-25T10:27:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- 172.21.15.87:0/1271284214 shutdown_connections 2024-04-25T10:27:40.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 --2- 172.21.15.87:0/1271284214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40b27e0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- 172.21.15.87:0/1271284214 >> 172.21.15.87:0/1271284214 conn(0x7fc4f4066b70 msgr2=0x7fc4f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:40.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- 172.21.15.87:0/1271284214 shutdown_connections 2024-04-25T10:27:40.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- 172.21.15.87:0/1271284214 wait complete. 2024-04-25T10:27:40.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 Processor -- start 2024-04-25T10:27:40.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- start start 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40cf9e0 0x7fc4f4157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f415bd10 0x7fc4f414ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f416a460 con 0x7fc4f415c0f0 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f416a2e0 con 0x7fc4f415bd10 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4fb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f416a5e0 con 0x7fc4f40cf9e0 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58616/0 (socket says 172.21.15.87:58616) 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 -- 172.21.15.87:0/3837230599 learned_addr learned my addr 172.21.15.87:0/3837230599 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f8a00640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40cf9e0 0x7fc4f4157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 -- 172.21.15.87:0/3837230599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40cf9e0 msgr2=0x7fc4f4157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40cf9e0 0x7fc4f4157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 -- 172.21.15.87:0/3837230599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f415bd10 msgr2=0x7fc4f414ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f415bd10 0x7fc4f414ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.333+0000 7fc4f9400640 1 -- 172.21.15.87:0/3837230599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f40b40c0 con 0x7fc4f415c0f0 2024-04-25T10:27:40.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f9400640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40afbb0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c3f0 tx=0x7fc4ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0074e0 con 0x7fc4f415c0f0 2024-04-25T10:27:40.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec007680 con 0x7fc4f415c0f0 2024-04-25T10:27:40.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec004a10 con 0x7fc4f415c0f0 2024-04-25T10:27:40.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f414bfa0 con 0x7fc4f415c0f0 2024-04-25T10:27:40.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f40b0770 con 0x7fc4f415c0f0 2024-04-25T10:27:40.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4f404ef30 con 0x7fc4f415c0f0 2024-04-25T10:27:40.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4ec01a020 con 0x7fc4f415c0f0 2024-04-25T10:27:40.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4d4066f90 0x7fc4d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc4ec084fc0 con 0x7fc4f415c0f0 2024-04-25T10:27:40.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f8a00640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4d4066f90 0x7fc4d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7fc4d406cef0 0x7fc4d406f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 --> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4ec010ec0 con 0x7fc4d406cef0 2024-04-25T10:27:40.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4ec085370 con 0x7fc4f415c0f0 2024-04-25T10:27:40.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.337+0000 7fc4f3e00640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7fc4d406cef0 0x7fc4d406f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.341+0000 7fc4f8a00640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4d4066f90 0x7fc4d4069450 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc4f415b930 tx=0x7fc4e40094f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.341+0000 7fc4f3e00640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7fc4d406cef0 0x7fc4d406f310 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.341+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== osd.0 v2:172.21.15.87:6800/2221934810 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc4ec010ec0 con 0x7fc4d406cef0 2024-04-25T10:27:40.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 --> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc4f406d9b0 con 0x7fc4d406cef0 2024-04-25T10:27:40.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4f0c00640 1 -- 172.21.15.87:0/3837230599 <== osd.0 v2:172.21.15.87:6800/2221934810 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc4f406d9b0 con 0x7fc4d406cef0 2024-04-25T10:27:40.352 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:40.352 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:40.352 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:40.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7fc4d406cef0 msgr2=0x7fc4d406f310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6800/2221934810,v1:172.21.15.87:6801/2221934810] conn(0x7fc4d406cef0 0x7fc4d406f310 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4d4066f90 msgr2=0x7fc4d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fc4d4066f90 0x7fc4d4069450 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc4f415b930 tx=0x7fc4e40094f0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 msgr2=0x7fc4f40afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 --2- 172.21.15.87:0/3837230599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4f415c0f0 0x7fc4f40afbb0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c3f0 tx=0x7fc4ec00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4f9400640 1 -- 172.21.15.87:0/3837230599 reap_dead start 2024-04-25T10:27:40.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 shutdown_connections 2024-04-25T10:27:40.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 >> 172.21.15.87:0/3837230599 conn(0x7fc4f4066b70 msgr2=0x7fc4f415e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:40.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 shutdown_connections 2024-04-25T10:27:40.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.353+0000 7fc4fb400640 1 -- 172.21.15.87:0/3837230599 wait complete. 2024-04-25T10:27:40.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:27:40.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.505+0000 7fd4f5800640 1 Processor -- start 2024-04-25T10:27:40.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.505+0000 7fd4f5800640 1 -- start start 2024-04-25T10:27:40.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4f5800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4f00cda10 0x7fd4f006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cf930 0x7fd4f00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4f5800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00b2770 con 0x7fd4f015bc40 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4f5800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f006c670 con 0x7fd4f00cda10 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4f5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f006c950 con 0x7fd4f00cf930 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4e6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4f00cda10 0x7fd4f006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4e6000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4f00cda10 0x7fd4f006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58620/0 (socket says 172.21.15.87:58620) 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4e6000640 1 -- 172.21.15.87:0/2822762835 learned_addr learned my addr 172.21.15.87:0/2822762835 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4ef400640 1 --2- 172.21.15.87:0/2822762835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cf930 0x7fd4f00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4e6000640 1 -- 172.21.15.87:0/2822762835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cf930 msgr2=0x7fd4f00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4e6000640 1 --2- 172.21.15.87:0/2822762835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cf930 0x7fd4f00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4e6000640 1 -- 172.21.15.87:0/2822762835 >> v1:172.21.15.169:6789/0 conn(0x7fd4f015bc40 legacy=0x7fd4f00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4e6000640 1 -- 172.21.15.87:0/2822762835 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f006cc30 con 0x7fd4f00cda10 2024-04-25T10:27:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.509+0000 7fd4ef400640 1 --2- 172.21.15.87:0/2822762835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cf930 0x7fd4f00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:40.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4e6000640 1 --2- 172.21.15.87:0/2822762835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cda10 0x7fd4f006c090 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009870 tx=0x7fd4d801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e93c238931ac780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4ee000640 1 -- 172.21.15.87:0/2822762835 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80b0070 con 0x7fd4f00cda10 2024-04-25T10:27:40.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4ee000640 1 -- 172.21.15.87:0/2822762835 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d80ab600 con 0x7fd4f00cda10 2024-04-25T10:27:40.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4ee000640 1 -- 172.21.15.87:0/2822762835 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80ab9d0 con 0x7fd4f00cda10 2024-04-25T10:27:40.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 -- 172.21.15.87:0/2822762835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cda10 msgr2=0x7fd4f006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 --2- 172.21.15.87:0/2822762835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cda10 0x7fd4f006c090 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009870 tx=0x7fd4d801c490 comp rx=0 tx=0).stop 2024-04-25T10:27:40.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 -- 172.21.15.87:0/2822762835 shutdown_connections 2024-04-25T10:27:40.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 --2- 172.21.15.87:0/2822762835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cf930 0x7fd4f00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 --2- 172.21.15.87:0/2822762835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cda10 0x7fd4f006c090 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 -- 172.21.15.87:0/2822762835 >> 172.21.15.87:0/2822762835 conn(0x7fd4f0066b70 msgr2=0x7fd4f0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:40.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 -- 172.21.15.87:0/2822762835 shutdown_connections 2024-04-25T10:27:40.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 -- 172.21.15.87:0/2822762835 wait complete. 2024-04-25T10:27:40.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 Processor -- start 2024-04-25T10:27:40.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.513+0000 7fd4f5800640 1 -- start start 2024-04-25T10:27:40.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cda10 0x7fd4f014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cf930 0x7fd4f013ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015bc40 0x7fd4f0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00b3890 con 0x7fd4f00cf930 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00b3710 con 0x7fd4f015bc40 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00b3a10 con 0x7fd4f00cda10 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cf930 0x7fd4f013ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cf930 0x7fd4f013ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58622/0 (socket says 172.21.15.87:58622) 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 -- 172.21.15.87:0/233392546 learned_addr learned my addr 172.21.15.87:0/233392546 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4ef400640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015bc40 0x7fd4f0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4eea00640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cda10 0x7fd4f014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 -- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cda10 msgr2=0x7fd4f014b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cda10 0x7fd4f014b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 -- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015bc40 msgr2=0x7fd4f0153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015bc40 0x7fd4f0153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 -- 172.21.15.87:0/233392546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f015d050 con 0x7fd4f00cf930 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4eea00640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cda10 0x7fd4f014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e6000640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cf930 0x7fd4f013ba10 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fd4d801ab40 tx=0x7fd4d801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e7400640 1 -- 172.21.15.87:0/233392546 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80ba020 con 0x7fd4f00cf930 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e7400640 1 -- 172.21.15.87:0/233392546 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d80aa9f0 con 0x7fd4f00cf930 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e7400640 1 -- 172.21.15.87:0/233392546 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80b0040 con 0x7fd4f00cf930 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f015d330 con 0x7fd4f00cf930 2024-04-25T10:27:40.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f015d8f0 con 0x7fd4f00cf930 2024-04-25T10:27:40.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f004ef30 con 0x7fd4f00cf930 2024-04-25T10:27:40.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.517+0000 7fd4e7400640 1 -- 172.21.15.87:0/233392546 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4d80bd020 con 0x7fd4f00cf930 2024-04-25T10:27:40.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.521+0000 7fd4e7400640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4cc066f60 0x7fd4cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.521+0000 7fd4e7400640 1 -- 172.21.15.87:0/233392546 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd4d811f490 con 0x7fd4f00cf930 2024-04-25T10:27:40.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.521+0000 7fd4eea00640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4cc066f60 0x7fd4cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.521+0000 7fd4eea00640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4cc066f60 0x7fd4cc069420 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009fc0 tx=0x7fd4e0009550 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.525+0000 7fd4e7400640 1 -- 172.21.15.87:0/233392546 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4d80c4030 con 0x7fd4f00cf930 2024-04-25T10:27:40.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4f013bf50 con 0x7fd4cc066f60 2024-04-25T10:27:40.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4e7400640 1 -- 172.21.15.87:0/233392546 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd4f013bf50 con 0x7fd4cc066f60 2024-04-25T10:27:40.674 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:27:40.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4cc066f60 msgr2=0x7fd4cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4cc066f60 0x7fd4cc069420 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009fc0 tx=0x7fd4e0009550 comp rx=0 tx=0).stop 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cf930 msgr2=0x7fd4f013ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cf930 0x7fd4f013ba10 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fd4d801ab40 tx=0x7fd4d801acd0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 shutdown_connections 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd4cc066f60 0x7fd4cc069420 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015bc40 0x7fd4f0153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00cf930 0x7fd4f013ba10 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 --2- 172.21.15.87:0/233392546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4f00cda10 0x7fd4f014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 >> 172.21.15.87:0/233392546 conn(0x7fd4f0066b70 msgr2=0x7fd4f0066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 shutdown_connections 2024-04-25T10:27:40.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.677+0000 7fd4f5800640 1 -- 172.21.15.87:0/233392546 wait complete. 2024-04-25T10:27:40.694 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:27:40.695 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:40.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bea00640 1 Processor -- start 2024-04-25T10:27:40.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bea00640 1 -- start start 2024-04-25T10:27:40.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9b815ac40 0x7fd9b815be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b815b100 con 0x7fd9b80d0090 2024-04-25T10:27:40.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b815b3e0 con 0x7fd9b80b37d0 2024-04-25T10:27:40.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b815bc80 con 0x7fd9b815ac40 2024-04-25T10:27:40.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bd600640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd9b80d0090 0x7fd9b80b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33590/0 (socket says 172.21.15.87:33590) 2024-04-25T10:27:40.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bd600640 1 -- 172.21.15.87:0/3212571633 learned_addr learned my addr 172.21.15.87:0/3212571633 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:40.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341143709 0 0) 0x7fd9b815b100 con 0x7fd9b80d0090 2024-04-25T10:27:40.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a0005740 con 0x7fd9b80d0090 2024-04-25T10:27:40.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927413820 0 0) 0x7fd9a0005740 con 0x7fd9b80d0090 2024-04-25T10:27:40.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9b815ac40 msgr2=0x7fd9b815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:40.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9b815ac40 0x7fd9b815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 >> v1:172.21.15.169:6789/0 conn(0x7fd9b80b37d0 legacy=0x7fd9b815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b815b6c0 con 0x7fd9b80d0090 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7fd9ac001680 con 0x7fd9b80d0090 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 >> v1:172.21.15.87:6789/0 conn(0x7fd9b80d0090 legacy=0x7fd9b80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a0005c00 0x7fd9a0006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a0006a20 0x7fd9a0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9b815ac40 0x7fd9a000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b815bc80 con 0x7fd9a0005c00 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a0007820 con 0x7fd9a0006a20 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a000e120 con 0x7fd9b815ac40 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bd600640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a0005c00 0x7fd9a0006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.837+0000 7fd9bcc00640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a0006a20 0x7fd9a0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bd600640 1 -- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9b815ac40 msgr2=0x7fd9a000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:40.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bd600640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9b815ac40 0x7fd9a000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bd600640 1 -- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a0006a20 msgr2=0x7fd9a0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:40.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bd600640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a0006a20 0x7fd9a0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bd600640 1 -- 172.21.15.87:0/3212571633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0005740 con 0x7fd9a0005c00 2024-04-25T10:27:40.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bd600640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a0005c00 0x7fd9a0006000 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00c270 tx=0x7fd9ac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f1d1dc604dec35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac007d20 con 0x7fd9a0005c00 2024-04-25T10:27:40.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac0a9020 con 0x7fd9a0005c00 2024-04-25T10:27:40.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9b7e00640 1 -- 172.21.15.87:0/3212571633 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac005ae0 con 0x7fd9a0005c00 2024-04-25T10:27:40.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bea00640 1 -- 172.21.15.87:0/3212571633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a0005c00 msgr2=0x7fd9a0006000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bea00640 1 --2- 172.21.15.87:0/3212571633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a0005c00 0x7fd9a0006000 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00c270 tx=0x7fd9ac01cc80 comp rx=0 tx=0).stop 2024-04-25T10:27:40.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9be000640 1 -- 172.21.15.87:0/3212571633 reap_dead start 2024-04-25T10:27:40.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bea00640 1 -- 172.21.15.87:0/3212571633 shutdown_connections 2024-04-25T10:27:40.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bea00640 1 -- 172.21.15.87:0/3212571633 >> 172.21.15.87:0/3212571633 conn(0x7fd9b8074450 msgr2=0x7fd9b80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:40.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bea00640 1 -- 172.21.15.87:0/3212571633 shutdown_connections 2024-04-25T10:27:40.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.841+0000 7fd9bea00640 1 -- 172.21.15.87:0/3212571633 wait complete. 2024-04-25T10:27:40.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 Processor -- start 2024-04-25T10:27:40.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 -- start start 2024-04-25T10:27:40.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a0005c00 0x7fd9b80b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b815be50 0x7fd9b815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b81f5aa0 0x7fd9b81f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b815ed00 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8072440 con 0x7fd9b815be50 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b815ee80 con 0x7fd9a0005c00 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b81f5aa0 0x7fd9b81f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b81f5aa0 0x7fd9b81f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58634/0 (socket says 172.21.15.87:58634) 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 -- 172.21.15.87:0/2236972633 learned_addr learned my addr 172.21.15.87:0/2236972633 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bcc00640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b815be50 0x7fd9b815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bd600640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a0005c00 0x7fd9b80b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 -- 172.21.15.87:0/2236972633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a0005c00 msgr2=0x7fd9b80b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a0005c00 0x7fd9b80b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 -- 172.21.15.87:0/2236972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b815be50 msgr2=0x7fd9b815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b815be50 0x7fd9b815c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 -- 172.21.15.87:0/2236972633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b81f6930 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bcc00640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b815be50 0x7fd9b815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9be000640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b81f5aa0 0x7fd9b81f5f00 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c4f0 tx=0x7fd9b000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0007110 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b00072d0 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b81f6c10 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00076c0 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.845+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b81f6440 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9b81f72a0 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9b0007860 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9b5600640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd998066f60 0x7fd998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd9b0084990 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9b5600640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7fd99806cec0 0x7fd99806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:40.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9bd600640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd998066f60 0x7fd998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 --> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd99806f990 con 0x7fd99806cec0 2024-04-25T10:27:40.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b0084d80 con 0x7fd9b81f5aa0 2024-04-25T10:27:40.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9bcc00640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7fd99806cec0 0x7fd99806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:40.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9bd600640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd998066f60 0x7fd998069420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0aa000 tx=0x7fd9b815b2d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.849+0000 7fd9bcc00640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7fd99806cec0 0x7fd99806f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:40.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.853+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== osd.1 v2:172.21.15.87:6816/191811739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd99806f990 con 0x7fd99806cec0 2024-04-25T10:27:40.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 --> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd9b81f7790 con 0x7fd99806cec0 2024-04-25T10:27:40.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9b5600640 1 -- 172.21.15.87:0/2236972633 <== osd.1 v2:172.21.15.87:6816/191811739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd9b81f7790 con 0x7fd99806cec0 2024-04-25T10:27:40.863 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:40.863 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:40.863 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7fd99806cec0 msgr2=0x7fd99806f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6816/191811739,v1:172.21.15.87:6817/191811739] conn(0x7fd99806cec0 0x7fd99806f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd998066f60 msgr2=0x7fd998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fd998066f60 0x7fd998069420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0aa000 tx=0x7fd9b815b2d0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b81f5aa0 msgr2=0x7fd9b81f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 --2- 172.21.15.87:0/2236972633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b81f5aa0 0x7fd9b81f5f00 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c4f0 tx=0x7fd9b000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:27:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9be000640 1 -- 172.21.15.87:0/2236972633 reap_dead start 2024-04-25T10:27:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 shutdown_connections 2024-04-25T10:27:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 >> 172.21.15.87:0/2236972633 conn(0x7fd9b8074450 msgr2=0x7fd9b815e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 shutdown_connections 2024-04-25T10:27:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:40.865+0000 7fd9bea00640 1 -- 172.21.15.87:0/2236972633 wait complete. 2024-04-25T10:27:40.880 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:27:41.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64dde00640 1 Processor -- start 2024-04-25T10:27:41.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64dde00640 1 -- start start 2024-04-25T10:27:41.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64dde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80b0f70 0x7f64d80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64dde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64d80cfd90 0x7f64d80ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64dde00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d815f3b0 con 0x7f64d80cd110 2024-04-25T10:27:41.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64dde00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d815ef20 con 0x7f64d80cfd90 2024-04-25T10:27:41.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64dde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80ca810 con 0x7f64d80b0f70 2024-04-25T10:27:41.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80b0f70 0x7f64d80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d7400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f64d80cd110 0x7f64d80cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39430/0 (socket says 172.21.15.87:39430) 2024-04-25T10:27:41.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d7400640 1 -- 172.21.15.87:0/3387554499 learned_addr learned my addr 172.21.15.87:0/3387554499 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:41.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d6000640 1 --2- 172.21.15.87:0/3387554499 >> v2:172.21.15.169:3300/0 conn(0x7f64d80cfd90 0x7f64d80ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d5600640 1 -- 172.21.15.87:0/3387554499 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537547621 0 0) 0x7f64d815f3b0 con 0x7f64d80cd110 2024-04-25T10:27:41.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d6a00640 1 -- 172.21.15.87:0/3387554499 >> v2:172.21.15.169:3300/0 conn(0x7f64d80cfd90 msgr2=0x7f64d80ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:41.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d6a00640 1 --2- 172.21.15.87:0/3387554499 >> v2:172.21.15.169:3300/0 conn(0x7f64d80cfd90 0x7f64d80ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d6a00640 1 -- 172.21.15.87:0/3387554499 >> v1:172.21.15.169:6789/0 conn(0x7f64d80cd110 legacy=0x7f64d80cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.021+0000 7f64d6a00640 1 -- 172.21.15.87:0/3387554499 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80b1890 con 0x7f64d80b0f70 2024-04-25T10:27:41.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64d6a00640 1 --2- 172.21.15.87:0/3387554499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80b0f70 0x7f64d80b1350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097c0 tx=0x7f64c801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dff6b2f3fb065c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64d5600640 1 -- 172.21.15.87:0/3387554499 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80b0070 con 0x7f64d80b0f70 2024-04-25T10:27:41.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64d5600640 1 -- 172.21.15.87:0/3387554499 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c80ab600 con 0x7f64d80b0f70 2024-04-25T10:27:41.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64d5600640 1 -- 172.21.15.87:0/3387554499 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80ab9d0 con 0x7f64d80b0f70 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 -- 172.21.15.87:0/3387554499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80b0f70 msgr2=0x7f64d80b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 --2- 172.21.15.87:0/3387554499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80b0f70 0x7f64d80b1350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097c0 tx=0x7f64c801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 -- 172.21.15.87:0/3387554499 shutdown_connections 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 --2- 172.21.15.87:0/3387554499 >> v2:172.21.15.169:3300/0 conn(0x7f64d80cfd90 0x7f64d80ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 --2- 172.21.15.87:0/3387554499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80b0f70 0x7f64d80b1350 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 -- 172.21.15.87:0/3387554499 >> 172.21.15.87:0/3387554499 conn(0x7f64d80b1d50 msgr2=0x7f64d80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 -- 172.21.15.87:0/3387554499 shutdown_connections 2024-04-25T10:27:41.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.025+0000 7f64dde00640 1 -- 172.21.15.87:0/3387554499 wait complete. 2024-04-25T10:27:41.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 Processor -- start 2024-04-25T10:27:41.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 -- start start 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64d80b0f70 0x7f64d80c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80cd110 0x7f64d80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d80cfd90 0x7f64d80afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d81619d0 con 0x7f64d80b0f70 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80cd540 con 0x7f64d80cfd90 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8161b50 con 0x7f64d80cd110 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64d80b0f70 0x7f64d80c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64d80b0f70 0x7f64d80c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58642/0 (socket says 172.21.15.87:58642) 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 -- 172.21.15.87:0/196827313 learned_addr learned my addr 172.21.15.87:0/196827313 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:41.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 -- 172.21.15.87:0/196827313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80cd110 msgr2=0x7f64d80af7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:27:41.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80cd110 0x7f64d80af7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 -- 172.21.15.87:0/196827313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d80cfd90 msgr2=0x7f64d80afd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:27:41.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d80cfd90 0x7f64d80afd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 -- 172.21.15.87:0/196827313 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80c4fa0 con 0x7f64d80b0f70 2024-04-25T10:27:41.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64d6a00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64d80b0f70 0x7f64d80c5f30 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f64c80190b0 tx=0x7f64d80cc940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64cf400640 1 -- 172.21.15.87:0/196827313 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80b0040 con 0x7f64d80b0f70 2024-04-25T10:27:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64cf400640 1 -- 172.21.15.87:0/196827313 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c80bc070 con 0x7f64d80b0f70 2024-04-25T10:27:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64cf400640 1 -- 172.21.15.87:0/196827313 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80b5680 con 0x7f64d80b0f70 2024-04-25T10:27:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d80c3c20 con 0x7f64d80b0f70 2024-04-25T10:27:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.029+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d80c41e0 con 0x7f64d80b0f70 2024-04-25T10:27:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.033+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d804ef30 con 0x7f64d80b0f70 2024-04-25T10:27:41.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.033+0000 7f64cf400640 1 -- 172.21.15.87:0/196827313 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64c80b3070 con 0x7f64d80b0f70 2024-04-25T10:27:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.033+0000 7f64cf400640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64b8066f60 0x7f64b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.033+0000 7f64cf400640 1 -- 172.21.15.87:0/196827313 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f64c811ea20 con 0x7f64d80b0f70 2024-04-25T10:27:41.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.033+0000 7f64d6000640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64b8066f60 0x7f64b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.033+0000 7f64d6000640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64b8066f60 0x7f64b8069420 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64d8073470 tx=0x7f64c4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.037+0000 7f64cf400640 1 -- 172.21.15.87:0/196827313 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64c8016090 con 0x7f64d80b0f70 2024-04-25T10:27:41.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.188+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d80ccdf0 con 0x7f64b8066f60 2024-04-25T10:27:41.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.188+0000 7f64cf400640 1 -- 172.21.15.87:0/196827313 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f64d80ccdf0 con 0x7f64b8066f60 2024-04-25T10:27:41.188 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64b8066f60 msgr2=0x7f64b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64b8066f60 0x7f64b8069420 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64d8073470 tx=0x7f64c4009190 comp rx=0 tx=0).stop 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64d80b0f70 msgr2=0x7f64d80c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64d80b0f70 0x7f64d80c5f30 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f64c80190b0 tx=0x7f64d80cc940 comp rx=0 tx=0).stop 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 shutdown_connections 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f64b8066f60 0x7f64b8069420 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d80cfd90 0x7f64d80afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64d80cd110 0x7f64d80af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 --2- 172.21.15.87:0/196827313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64d80b0f70 0x7f64d80c5f30 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 >> 172.21.15.87:0/196827313 conn(0x7f64d80b1d50 msgr2=0x7f64d8072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 shutdown_connections 2024-04-25T10:27:41.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.192+0000 7f64dde00640 1 -- 172.21.15.87:0/196827313 wait complete. 2024-04-25T10:27:41.210 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:27:41.210 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:41.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.352+0000 7f3864200640 1 Processor -- start 2024-04-25T10:27:41.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.352+0000 7f3864200640 1 -- start start 2024-04-25T10:27:41.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3864200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f385c15bd10 0x7f385c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3864200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 0x7f385c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3864200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c06c0e0 con 0x7f385c0cf9e0 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3864200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c15c6a0 con 0x7f385c15bd10 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3864200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c15c980 con 0x7f385c15c0f0 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f385c0cf9e0 0x7f385c06c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33606/0 (socket says 172.21.15.87:33606) 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 -- 172.21.15.87:0/2899291768 learned_addr learned my addr 172.21.15.87:0/2899291768 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3860e00640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 0x7f385c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3861800640 1 --2- 172.21.15.87:0/2899291768 >> v2:172.21.15.169:3300/0 conn(0x7f385c15bd10 0x7f385c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403268838 0 0) 0x7f385c06c0e0 con 0x7f385c0cf9e0 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3844003530 con 0x7f385c0cf9e0 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863680702 0 0) 0x7f3844003530 con 0x7f385c0cf9e0 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 msgr2=0x7f385c0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 0x7f385c0b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 >> v2:172.21.15.169:3300/0 conn(0x7f385c15bd10 msgr2=0x7f385c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 --2- 172.21.15.87:0/2899291768 >> v2:172.21.15.169:3300/0 conn(0x7f385c15bd10 0x7f385c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c15cc60 con 0x7f385c0cf9e0 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f3854001680 con 0x7f385c0cf9e0 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 >> v1:172.21.15.87:6789/0 conn(0x7f385c0cf9e0 legacy=0x7f385c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38440039c0 0x7f3844003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 0x7f3844004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3844004d50 0x7f38440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c15c6a0 con 0x7f3844004d50 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c15c980 con 0x7f38440039c0 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384400b9d0 con 0x7f385c15c0f0 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3844004d50 0x7f38440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3861800640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38440039c0 0x7f3844003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3860e00640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 0x7f3844004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 -- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 msgr2=0x7f3844004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:41.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 0x7f3844004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 -- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38440039c0 msgr2=0x7f3844003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:41.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38440039c0 0x7f3844003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3862200640 1 -- 172.21.15.87:0/2899291768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844003530 con 0x7f3844004d50 2024-04-25T10:27:41.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.356+0000 7f3860e00640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f385c15c0f0 0x7f3844004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:41.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3862200640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3844004d50 0x7f38440051d0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f385400c160 tx=0x7f385401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50c5c3268c3e91f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854007c90 con 0x7f3844004d50 2024-04-25T10:27:41.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38540a9020 con 0x7f3844004d50 2024-04-25T10:27:41.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f385be00640 1 -- 172.21.15.87:0/2899291768 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854005ae0 con 0x7f3844004d50 2024-04-25T10:27:41.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 -- 172.21.15.87:0/2899291768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3844004d50 msgr2=0x7f38440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 --2- 172.21.15.87:0/2899291768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3844004d50 0x7f38440051d0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f385400c160 tx=0x7f385401cbe0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3862200640 1 -- 172.21.15.87:0/2899291768 reap_dead start 2024-04-25T10:27:41.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 -- 172.21.15.87:0/2899291768 shutdown_connections 2024-04-25T10:27:41.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 -- 172.21.15.87:0/2899291768 >> 172.21.15.87:0/2899291768 conn(0x7f385c066b70 msgr2=0x7f385c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 -- 172.21.15.87:0/2899291768 shutdown_connections 2024-04-25T10:27:41.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 -- 172.21.15.87:0/2899291768 wait complete. 2024-04-25T10:27:41.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 Processor -- start 2024-04-25T10:27:41.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.360+0000 7f3864200640 1 -- start start 2024-04-25T10:27:41.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3844004d50 0x7f385c14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f385c15e640 0x7f385c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f385c145a50 0x7f385c145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0b4cd0 con 0x7f385c145a50 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0b4b50 con 0x7f385c15e640 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c06ccf0 con 0x7f3844004d50 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f385c145a50 0x7f385c145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f385c145a50 0x7f385c145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58664/0 (socket says 172.21.15.87:58664) 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 -- 172.21.15.87:0/349970466 learned_addr learned my addr 172.21.15.87:0/349970466 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3861800640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3844004d50 0x7f385c14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3860e00640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f385c15e640 0x7f385c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 -- 172.21.15.87:0/349970466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3844004d50 msgr2=0x7f385c14b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3844004d50 0x7f385c14b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 -- 172.21.15.87:0/349970466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f385c15e640 msgr2=0x7f385c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f385c15e640 0x7f385c15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 -- 172.21.15.87:0/349970466 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c146660 con 0x7f385c145a50 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3862200640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f385c145a50 0x7f385c145ed0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f385400c160 tx=0x7f3844005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854007c70 con 0x7f385c145a50 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38540b3070 con 0x7f385c145a50 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38540049f0 con 0x7f385c145a50 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c146940 con 0x7f385c145a50 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c146f00 con 0x7f385c145a50 2024-04-25T10:27:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f385c147e40 con 0x7f385c145a50 2024-04-25T10:27:41.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.364+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3854004b90 con 0x7f385c145a50 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3859600640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f383c066f60 0x7f383c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f385411e6f0 con 0x7f385c145a50 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3859600640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f383c06cec0 0x7f383c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3861800640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f383c066f60 0x7f383c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 --> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f38540a9e50 con 0x7f383c06cec0 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3854125050 con 0x7f385c145a50 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3860e00640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f383c06cec0 0x7f383c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3861800640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f383c066f60 0x7f383c069420 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3850009fd0 tx=0x7f385000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3860e00640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f383c06cec0 0x7f383c06f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.368+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== osd.2 v2:172.21.15.87:6824/2913449321 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f38540a9e50 con 0x7f383c06cec0 2024-04-25T10:27:41.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 --> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f385c146410 con 0x7f383c06cec0 2024-04-25T10:27:41.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3859600640 1 -- 172.21.15.87:0/349970466 <== osd.2 v2:172.21.15.87:6824/2913449321 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f385c146410 con 0x7f383c06cec0 2024-04-25T10:27:41.380 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:41.380 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:41.380 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f383c06cec0 msgr2=0x7f383c06f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6824/2913449321,v1:172.21.15.87:6825/2913449321] conn(0x7f383c06cec0 0x7f383c06f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f383c066f60 msgr2=0x7f383c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f383c066f60 0x7f383c069420 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3850009fd0 tx=0x7f385000b040 comp rx=0 tx=0).stop 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f385c145a50 msgr2=0x7f385c145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 --2- 172.21.15.87:0/349970466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f385c145a50 0x7f385c145ed0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f385400c160 tx=0x7f3844005840 comp rx=0 tx=0).stop 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3862200640 1 -- 172.21.15.87:0/349970466 reap_dead start 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.380+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 shutdown_connections 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.384+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 >> 172.21.15.87:0/349970466 conn(0x7f385c066b70 msgr2=0x7f385c15d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.384+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 shutdown_connections 2024-04-25T10:27:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.384+0000 7f3864200640 1 -- 172.21.15.87:0/349970466 wait complete. 2024-04-25T10:27:41.400 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:27:41.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.536+0000 7fec3ca00640 1 Processor -- start 2024-04-25T10:27:41.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.536+0000 7fec3ca00640 1 -- start start 2024-04-25T10:27:41.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec3ca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec380c6a20 0x7fec38166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec3ca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380ccfb0 con 0x7fec3815b9e0 2024-04-25T10:27:41.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec3ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380cd290 con 0x7fec380ccb20 2024-04-25T10:27:41.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec3ca00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec380cd570 con 0x7fec380c6a20 2024-04-25T10:27:41.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec36800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec380c6a20 0x7fec38166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec35e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fec3815b9e0 0x7fec380cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33616/0 (socket says 172.21.15.87:33616) 2024-04-25T10:27:41.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec35e00640 1 -- 172.21.15.87:0/1369711022 learned_addr learned my addr 172.21.15.87:0/1369711022 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:41.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec36800640 1 -- 172.21.15.87:0/1369711022 >> v1:172.21.15.169:6789/0 conn(0x7fec380ccb20 legacy=0x7fec380d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec36800640 1 -- 172.21.15.87:0/1369711022 >> v1:172.21.15.87:6789/0 conn(0x7fec3815b9e0 legacy=0x7fec380cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec36800640 1 -- 172.21.15.87:0/1369711022 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380cd850 con 0x7fec380c6a20 2024-04-25T10:27:41.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.540+0000 7fec35400640 1 -- 172.21.15.87:0/1369711022 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356315835 0 0) 0x7fec380ccfb0 con 0x7fec3815b9e0 2024-04-25T10:27:41.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec36800640 1 --2- 172.21.15.87:0/1369711022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec380c6a20 0x7fec38166820 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fec30009d90 tx=0x7fec3001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e0eb68bf7d9128 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec35400640 1 -- 172.21.15.87:0/1369711022 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300b2070 con 0x7fec380c6a20 2024-04-25T10:27:41.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec35400640 1 -- 172.21.15.87:0/1369711022 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300a58e0 con 0x7fec380c6a20 2024-04-25T10:27:41.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec35400640 1 -- 172.21.15.87:0/1369711022 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300a5cb0 con 0x7fec380c6a20 2024-04-25T10:27:41.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- 172.21.15.87:0/1369711022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec380c6a20 msgr2=0x7fec38166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 --2- 172.21.15.87:0/1369711022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec380c6a20 0x7fec38166820 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fec30009d90 tx=0x7fec3001c970 comp rx=0 tx=0).stop 2024-04-25T10:27:41.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- 172.21.15.87:0/1369711022 shutdown_connections 2024-04-25T10:27:41.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 --2- 172.21.15.87:0/1369711022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec380c6a20 0x7fec38166820 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- 172.21.15.87:0/1369711022 >> 172.21.15.87:0/1369711022 conn(0x7fec3809a9d0 msgr2=0x7fec3809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- 172.21.15.87:0/1369711022 shutdown_connections 2024-04-25T10:27:41.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- 172.21.15.87:0/1369711022 wait complete. 2024-04-25T10:27:41.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 Processor -- start 2024-04-25T10:27:41.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- start start 2024-04-25T10:27:41.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec380c6a20 0x7fec38150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3815b9e0 0x7fec380b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec380b4d60 0x7fec380c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec38169910 con 0x7fec3815b9e0 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec38169790 con 0x7fec380c6a20 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec3ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec38169a90 con 0x7fec380b4d60 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec2d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3815b9e0 0x7fec380b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec2d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3815b9e0 0x7fec380b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58674/0 (socket says 172.21.15.87:58674) 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec2d400640 1 -- 172.21.15.87:0/823558536 learned_addr learned my addr 172.21.15.87:0/823558536 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.544+0000 7fec35e00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec380c6a20 0x7fec38150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec36800640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec380b4d60 0x7fec380c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2d400640 1 -- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec380b4d60 msgr2=0x7fec380c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2d400640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec380b4d60 0x7fec380c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2d400640 1 -- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec380c6a20 msgr2=0x7fec38150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2d400640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec380c6a20 0x7fec38150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2d400640 1 -- 172.21.15.87:0/823558536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38150e70 con 0x7fec3815b9e0 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2d400640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3815b9e0 0x7fec380b4820 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c4d0 tx=0x7fec2000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2ea00640 1 -- 172.21.15.87:0/823558536 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200052d0 con 0x7fec3815b9e0 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2ea00640 1 -- 172.21.15.87:0/823558536 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20005470 con 0x7fec3815b9e0 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2ea00640 1 -- 172.21.15.87:0/823558536 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20012830 con 0x7fec3815b9e0 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec380c8d70 con 0x7fec3815b9e0 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec380c9330 con 0x7fec3815b9e0 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3804ef30 con 0x7fec3815b9e0 2024-04-25T10:27:41.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2ea00640 1 -- 172.21.15.87:0/823558536 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec200129d0 con 0x7fec3815b9e0 2024-04-25T10:27:41.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2ea00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec14066f60 0x7fec14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec2ea00640 1 -- 172.21.15.87:0/823558536 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fec20084880 con 0x7fec3815b9e0 2024-04-25T10:27:41.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.548+0000 7fec35e00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec14066f60 0x7fec14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.552+0000 7fec35e00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec14066f60 0x7fec14069420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fec28000f70 tx=0x7fec28007240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.552+0000 7fec2ea00640 1 -- 172.21.15.87:0/823558536 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec20055b20 con 0x7fec3815b9e0 2024-04-25T10:27:41.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fec380cdea0 con 0x7fec14066f60 2024-04-25T10:27:41.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec2ea00640 1 -- 172.21.15.87:0/823558536 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fec380cdea0 con 0x7fec14066f60 2024-04-25T10:27:41.706 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:27:41.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec14066f60 msgr2=0x7fec14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec14066f60 0x7fec14069420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fec28000f70 tx=0x7fec28007240 comp rx=0 tx=0).stop 2024-04-25T10:27:41.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3815b9e0 msgr2=0x7fec380b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3815b9e0 0x7fec380b4820 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c4d0 tx=0x7fec2000c990 comp rx=0 tx=0).stop 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 shutdown_connections 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fec14066f60 0x7fec14069420 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec380b4d60 0x7fec380c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3815b9e0 0x7fec380b4820 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 --2- 172.21.15.87:0/823558536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec380c6a20 0x7fec38150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 >> 172.21.15.87:0/823558536 conn(0x7fec3809a9d0 msgr2=0x7fec380c5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 shutdown_connections 2024-04-25T10:27:41.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.708+0000 7fec3ca00640 1 -- 172.21.15.87:0/823558536 wait complete. 2024-04-25T10:27:41.726 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:27:41.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:41.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.864+0000 7fab99400640 1 Processor -- start 2024-04-25T10:27:41.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab99400640 1 -- start start 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab99400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fab940d0090 0x7fab9415be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab99400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab940b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab99400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9415b100 con 0x7fab940721f0 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab99400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9415bc80 con 0x7fab940d0090 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab99400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9415c390 con 0x7fab9415ac40 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab93e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fab940d0090 0x7fab9415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab98a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fab940721f0 0x7fab9415e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33618/0 (socket says 172.21.15.87:33618) 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab98a00640 1 -- 172.21.15.87:0/4159630373 learned_addr learned my addr 172.21.15.87:0/4159630373 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab8b400640 1 --2- 172.21.15.87:0/4159630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab940b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab93e00640 1 -- 172.21.15.87:0/4159630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 msgr2=0x7fab940b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab93e00640 1 --2- 172.21.15.87:0/4159630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab940b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab93e00640 1 -- 172.21.15.87:0/4159630373 >> v1:172.21.15.87:6789/0 conn(0x7fab940721f0 legacy=0x7fab9415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab93e00640 1 -- 172.21.15.87:0/4159630373 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9415b3e0 con 0x7fab940d0090 2024-04-25T10:27:41.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab93400640 1 -- 172.21.15.87:0/4159630373 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627745419 0 0) 0x7fab9415b100 con 0x7fab940721f0 2024-04-25T10:27:41.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.868+0000 7fab8b400640 1 --2- 172.21.15.87:0/4159630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab940b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:41.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab93e00640 1 --2- 172.21.15.87:0/4159630373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940d0090 0x7fab9415be50 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fab840097c0 tx=0x7fab8401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757896bdaa004c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab93400640 1 -- 172.21.15.87:0/4159630373 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840b0070 con 0x7fab940d0090 2024-04-25T10:27:41.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab93400640 1 -- 172.21.15.87:0/4159630373 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab840ab600 con 0x7fab940d0090 2024-04-25T10:27:41.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab93400640 1 -- 172.21.15.87:0/4159630373 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840ab9d0 con 0x7fab940d0090 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 -- 172.21.15.87:0/4159630373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940d0090 msgr2=0x7fab9415be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 --2- 172.21.15.87:0/4159630373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940d0090 0x7fab9415be50 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fab840097c0 tx=0x7fab8401c380 comp rx=0 tx=0).stop 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 -- 172.21.15.87:0/4159630373 shutdown_connections 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 --2- 172.21.15.87:0/4159630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab940b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 --2- 172.21.15.87:0/4159630373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940d0090 0x7fab9415be50 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 -- 172.21.15.87:0/4159630373 >> 172.21.15.87:0/4159630373 conn(0x7fab94074450 msgr2=0x7fab940669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 -- 172.21.15.87:0/4159630373 shutdown_connections 2024-04-25T10:27:41.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 -- 172.21.15.87:0/4159630373 wait complete. 2024-04-25T10:27:41.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 Processor -- start 2024-04-25T10:27:41.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.872+0000 7fab99400640 1 -- start start 2024-04-25T10:27:41.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940721f0 0x7fab94156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab940d0090 0x7fab9414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab9414e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9415f090 con 0x7fab940721f0 2024-04-25T10:27:41.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9415ef10 con 0x7fab940d0090 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9415f210 con 0x7fab9415ac40 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940721f0 0x7fab94156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940721f0 0x7fab94156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58680/0 (socket says 172.21.15.87:58680) 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 -- 172.21.15.87:0/1865801253 learned_addr learned my addr 172.21.15.87:0/1865801253 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab8b400640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab940d0090 0x7fab9414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab98a00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab9414e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 -- 172.21.15.87:0/1865801253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 msgr2=0x7fab9414e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab9415ac40 0x7fab9414e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 -- 172.21.15.87:0/1865801253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab940d0090 msgr2=0x7fab9414c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab940d0090 0x7fab9414c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 -- 172.21.15.87:0/1865801253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab940b1880 con 0x7fab940721f0 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab93e00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940721f0 0x7fab94156160 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fab840ae040 tx=0x7fab8401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840b0050 con 0x7fab940721f0 2024-04-25T10:27:41.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab840bc070 con 0x7fab940721f0 2024-04-25T10:27:41.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab940b1b60 con 0x7fab940721f0 2024-04-25T10:27:41.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840b5a50 con 0x7fab940721f0 2024-04-25T10:27:41.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9415d230 con 0x7fab940721f0 2024-04-25T10:27:41.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab94075120 con 0x7fab940721f0 2024-04-25T10:27:41.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.876+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab840ab4a0 con 0x7fab940721f0 2024-04-25T10:27:41.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab90c00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fab70066f60 0x7fab70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fab8411f740 con 0x7fab940721f0 2024-04-25T10:27:41.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab8b400640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fab70066f60 0x7fab70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab90c00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7fab7006cec0 0x7fab7006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:41.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 --> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab84001d40 con 0x7fab7006cec0 2024-04-25T10:27:41.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab84126050 con 0x7fab940721f0 2024-04-25T10:27:41.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab98a00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7fab7006cec0 0x7fab7006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:41.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab8b400640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fab70066f60 0x7fab70069420 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab98a00640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7fab7006cec0 0x7fab7006f2e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:41.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.880+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== osd.3 v2:172.21.15.87:6802/327355267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fab84001d40 con 0x7fab7006cec0 2024-04-25T10:27:41.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 --> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fab940cdcf0 con 0x7fab7006cec0 2024-04-25T10:27:41.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab90c00640 1 -- 172.21.15.87:0/1865801253 <== osd.3 v2:172.21.15.87:6802/327355267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fab940cdcf0 con 0x7fab7006cec0 2024-04-25T10:27:41.892 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:41.892 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:41.892 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:41.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7fab7006cec0 msgr2=0x7fab7006f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6802/327355267,v1:172.21.15.87:6804/327355267] conn(0x7fab7006cec0 0x7fab7006f2e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fab70066f60 msgr2=0x7fab70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fab70066f60 0x7fab70069420 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).stop 2024-04-25T10:27:41.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940721f0 msgr2=0x7fab94156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:41.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 --2- 172.21.15.87:0/1865801253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab940721f0 0x7fab94156160 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fab840ae040 tx=0x7fab8401abc0 comp rx=0 tx=0).stop 2024-04-25T10:27:41.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab98a00640 1 -- 172.21.15.87:0/1865801253 reap_dead start 2024-04-25T10:27:41.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 shutdown_connections 2024-04-25T10:27:41.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 >> 172.21.15.87:0/1865801253 conn(0x7fab94074450 msgr2=0x7fab94068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:41.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 shutdown_connections 2024-04-25T10:27:41.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:41.892+0000 7fab99400640 1 -- 172.21.15.87:0/1865801253 wait complete. 2024-04-25T10:27:41.911 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:27:42.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.048+0000 7faa13000640 1 Processor -- start 2024-04-25T10:27:42.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.048+0000 7faa13000640 1 -- start start 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa13000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faa0c0cda10 0x7faa0c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa13000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa0c0cf930 0x7faa0c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa13000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0c0b2770 con 0x7faa0c15bc40 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa13000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c06c670 con 0x7faa0c0cda10 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa13000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c06c950 con 0x7faa0c0cf930 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa03400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faa0c0cda10 0x7faa0c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa03400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faa0c0cda10 0x7faa0c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58682/0 (socket says 172.21.15.87:58682) 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa03400640 1 -- 172.21.15.87:0/3980488616 learned_addr learned my addr 172.21.15.87:0/3980488616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa11000640 1 --2- 172.21.15.87:0/3980488616 >> v2:172.21.15.169:3300/0 conn(0x7faa0c0cf930 0x7faa0c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa03400640 1 -- 172.21.15.87:0/3980488616 >> v2:172.21.15.169:3300/0 conn(0x7faa0c0cf930 msgr2=0x7faa0c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa03400640 1 --2- 172.21.15.87:0/3980488616 >> v2:172.21.15.169:3300/0 conn(0x7faa0c0cf930 0x7faa0c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa03400640 1 -- 172.21.15.87:0/3980488616 >> v1:172.21.15.169:6789/0 conn(0x7faa0c15bc40 legacy=0x7faa0c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.052+0000 7faa03400640 1 -- 172.21.15.87:0/3980488616 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c06cc30 con 0x7faa0c0cda10 2024-04-25T10:27:42.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa03400640 1 --2- 172.21.15.87:0/3980488616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c0cda10 0x7faa0c06c090 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0ae4c0 tx=0x7fa9f401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4855557e936bc2dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa0b400640 1 -- 172.21.15.87:0/3980488616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40b0070 con 0x7faa0c0cda10 2024-04-25T10:27:42.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa0b400640 1 -- 172.21.15.87:0/3980488616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f40ab600 con 0x7faa0c0cda10 2024-04-25T10:27:42.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa0b400640 1 -- 172.21.15.87:0/3980488616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40ab9d0 con 0x7faa0c0cda10 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- 172.21.15.87:0/3980488616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c0cda10 msgr2=0x7faa0c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 --2- 172.21.15.87:0/3980488616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c0cda10 0x7faa0c06c090 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0ae4c0 tx=0x7fa9f401c490 comp rx=0 tx=0).stop 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- 172.21.15.87:0/3980488616 shutdown_connections 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 --2- 172.21.15.87:0/3980488616 >> v2:172.21.15.169:3300/0 conn(0x7faa0c0cf930 0x7faa0c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 --2- 172.21.15.87:0/3980488616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c0cda10 0x7faa0c06c090 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- 172.21.15.87:0/3980488616 >> 172.21.15.87:0/3980488616 conn(0x7faa0c066b70 msgr2=0x7faa0c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- 172.21.15.87:0/3980488616 shutdown_connections 2024-04-25T10:27:42.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- 172.21.15.87:0/3980488616 wait complete. 2024-04-25T10:27:42.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 Processor -- start 2024-04-25T10:27:42.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- start start 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa0c0cda10 0x7faa0c14a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa0c0cf930 0x7faa0c15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c15bc40 0x7faa0c14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0b3850 con 0x7faa0c15bc40 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0b36d0 con 0x7faa0c0cf930 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa13000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0b39d0 con 0x7faa0c0cda10 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa11000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c15bc40 0x7faa0c14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa11000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c15bc40 0x7faa0c14bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58690/0 (socket says 172.21.15.87:58690) 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.056+0000 7faa11000640 1 -- 172.21.15.87:0/2451351122 learned_addr learned my addr 172.21.15.87:0/2451351122 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa03400640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa0c0cf930 0x7faa0c15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa0be00640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa0c0cda10 0x7faa0c14a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa11000640 1 -- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa0c0cda10 msgr2=0x7faa0c14a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa11000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa0c0cda10 0x7faa0c14a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa11000640 1 -- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa0c0cf930 msgr2=0x7faa0c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa11000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa0c0cf930 0x7faa0c15cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa11000640 1 -- 172.21.15.87:0/2451351122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c14e170 con 0x7faa0c15bc40 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa11000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c15bc40 0x7faa0c14bd40 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7faa04009d30 tx=0x7faa0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa08c00640 1 -- 172.21.15.87:0/2451351122 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04018020 con 0x7faa0c15bc40 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa08c00640 1 -- 172.21.15.87:0/2451351122 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa04014480 con 0x7faa0c15bc40 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa08c00640 1 -- 172.21.15.87:0/2451351122 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04014870 con 0x7faa0c15bc40 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c14e450 con 0x7faa0c15bc40 2024-04-25T10:27:42.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c14ea10 con 0x7faa0c15bc40 2024-04-25T10:27:42.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c04ef30 con 0x7faa0c15bc40 2024-04-25T10:27:42.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa08c00640 1 -- 172.21.15.87:0/2451351122 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa0401b020 con 0x7faa0c15bc40 2024-04-25T10:27:42.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa08c00640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9e8066f60 0x7fa9e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.060+0000 7faa08c00640 1 -- 172.21.15.87:0/2451351122 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7faa040859c0 con 0x7faa0c15bc40 2024-04-25T10:27:42.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.064+0000 7faa0be00640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9e8066f60 0x7fa9e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.064+0000 7faa0be00640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9e8066f60 0x7fa9e8069420 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0b10e0 tx=0x7fa9fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.064+0000 7faa08c00640 1 -- 172.21.15.87:0/2451351122 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa04087e20 con 0x7faa0c15bc40 2024-04-25T10:27:42.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.220+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7faa0c155a80 con 0x7fa9e8066f60 2024-04-25T10:27:42.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.220+0000 7faa08c00640 1 -- 172.21.15.87:0/2451351122 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faa0c155a80 con 0x7fa9e8066f60 2024-04-25T10:27:42.220 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:27:42.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9e8066f60 msgr2=0x7fa9e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9e8066f60 0x7fa9e8069420 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0b10e0 tx=0x7fa9fc009040 comp rx=0 tx=0).stop 2024-04-25T10:27:42.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c15bc40 msgr2=0x7faa0c14bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c15bc40 0x7faa0c14bd40 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7faa04009d30 tx=0x7faa0400c490 comp rx=0 tx=0).stop 2024-04-25T10:27:42.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 shutdown_connections 2024-04-25T10:27:42.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fa9e8066f60 0x7fa9e8069420 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa0c15bc40 0x7faa0c14bd40 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa0c0cf930 0x7faa0c15cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 --2- 172.21.15.87:0/2451351122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa0c0cda10 0x7faa0c14a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 >> 172.21.15.87:0/2451351122 conn(0x7faa0c066b70 msgr2=0x7faa0c0cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 shutdown_connections 2024-04-25T10:27:42.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.224+0000 7faa13000640 1 -- 172.21.15.87:0/2451351122 wait complete. 2024-04-25T10:27:42.241 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:27:42.241 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:42.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe01ce00640 1 Processor -- start 2024-04-25T10:27:42.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe01ce00640 1 -- start start 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe01ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0180cf540 0x7fe0180b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe01ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe01815baf0 0x7fe01815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe01ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180b4700 con 0x7fe0180b3000 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe01ce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0180b49e0 con 0x7fe0180cf540 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe01ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01815f470 con 0x7fe01815baf0 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe017400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0180cf540 0x7fe0180b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe017400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0180cf540 0x7fe0180b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58696/0 (socket says 172.21.15.87:58696) 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe017400640 1 -- 172.21.15.87:0/2882754656 learned_addr learned my addr 172.21.15.87:0/2882754656 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe017400640 1 -- 172.21.15.87:0/2882754656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe01815baf0 msgr2=0x7fe01815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe017400640 1 --2- 172.21.15.87:0/2882754656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe01815baf0 0x7fe01815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe017400640 1 -- 172.21.15.87:0/2882754656 >> v1:172.21.15.169:6789/0 conn(0x7fe0180b3000 legacy=0x7fe0180cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.384+0000 7fe017400640 1 -- 172.21.15.87:0/2882754656 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01815f750 con 0x7fe0180cf540 2024-04-25T10:27:42.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe017400640 1 --2- 172.21.15.87:0/2882754656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180cf540 0x7fe0180b4120 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fe008009e50 tx=0x7fe00801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e5d519287ebb9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe016000640 1 -- 172.21.15.87:0/2882754656 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080a5670 con 0x7fe0180cf540 2024-04-25T10:27:42.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe016000640 1 -- 172.21.15.87:0/2882754656 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0080a5810 con 0x7fe0180cf540 2024-04-25T10:27:42.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe016000640 1 -- 172.21.15.87:0/2882754656 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080a5be0 con 0x7fe0180cf540 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 -- 172.21.15.87:0/2882754656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180cf540 msgr2=0x7fe0180b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 --2- 172.21.15.87:0/2882754656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180cf540 0x7fe0180b4120 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fe008009e50 tx=0x7fe00801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 -- 172.21.15.87:0/2882754656 shutdown_connections 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 --2- 172.21.15.87:0/2882754656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe01815baf0 0x7fe01815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 --2- 172.21.15.87:0/2882754656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180cf540 0x7fe0180b4120 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 -- 172.21.15.87:0/2882754656 >> 172.21.15.87:0/2882754656 conn(0x7fe018074450 msgr2=0x7fe0180669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 -- 172.21.15.87:0/2882754656 shutdown_connections 2024-04-25T10:27:42.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 -- 172.21.15.87:0/2882754656 wait complete. 2024-04-25T10:27:42.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 Processor -- start 2024-04-25T10:27:42.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.388+0000 7fe01ce00640 1 -- start start 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180b3000 0x7fe01814e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0180cf540 0x7fe0180af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe01815baf0 0x7fe0180af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180ce230 con 0x7fe0180b3000 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180ce0b0 con 0x7fe0180cf540 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180ce3b0 con 0x7fe01815baf0 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180b3000 0x7fe01814e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180b3000 0x7fe01814e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58710/0 (socket says 172.21.15.87:58710) 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 -- 172.21.15.87:0/2458657323 learned_addr learned my addr 172.21.15.87:0/2458657323 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 -- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe01815baf0 msgr2=0x7fe0180af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe01815baf0 0x7fe0180af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 -- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0180cf540 msgr2=0x7fe0180af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0180cf540 0x7fe0180af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 -- 172.21.15.87:0/2458657323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01815e550 con 0x7fe0180b3000 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe017400640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180b3000 0x7fe01814e4b0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fe0080af040 tx=0x7fe008019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080bc020 con 0x7fe0180b3000 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0080b8670 con 0x7fe0180b3000 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080b8a60 con 0x7fe0180b3000 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01815e830 con 0x7fe0180b3000 2024-04-25T10:27:42.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01815edf0 con 0x7fe0180b3000 2024-04-25T10:27:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0180702d0 con 0x7fe0180b3000 2024-04-25T10:27:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.392+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0080bf020 con 0x7fe0180b3000 2024-04-25T10:27:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe00f400640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff8066f60 0x7fdff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe00811ffc0 con 0x7fe0180b3000 2024-04-25T10:27:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe00f400640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fdff806cec0 0x7fdff806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe016a00640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff8066f60 0x7fdff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 --> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0080b0ec0 con 0x7fdff806cec0 2024-04-25T10:27:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0081225e0 con 0x7fe0180b3000 2024-04-25T10:27:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe016a00640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff8066f60 0x7fdff8069420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe018152c00 tx=0x7fe004006c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe017e00640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fdff806cec0 0x7fdff806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe017e00640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fdff806cec0 0x7fdff806f2e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.396+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== osd.4 v2:172.21.15.169:6800/3149326747 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe0080b0ec0 con 0x7fdff806cec0 2024-04-25T10:27:42.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.408+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 --> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe018153fb0 con 0x7fdff806cec0 2024-04-25T10:27:42.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe00f400640 1 -- 172.21.15.87:0/2458657323 <== osd.4 v2:172.21.15.169:6800/3149326747 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe018153fb0 con 0x7fdff806cec0 2024-04-25T10:27:42.409 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:42.409 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:42.409 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:42.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fdff806cec0 msgr2=0x7fdff806f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.169:6800/3149326747,v1:172.21.15.169:6801/3149326747] conn(0x7fdff806cec0 0x7fdff806f2e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff8066f60 msgr2=0x7fdff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fdff8066f60 0x7fdff8069420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe018152c00 tx=0x7fe004006c00 comp rx=0 tx=0).stop 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180b3000 msgr2=0x7fe01814e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 --2- 172.21.15.87:0/2458657323 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0180b3000 0x7fe01814e4b0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fe0080af040 tx=0x7fe008019790 comp rx=0 tx=0).stop 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe017e00640 1 -- 172.21.15.87:0/2458657323 reap_dead start 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 shutdown_connections 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 >> 172.21.15.87:0/2458657323 conn(0x7fe018074450 msgr2=0x7fe0180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 shutdown_connections 2024-04-25T10:27:42.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.412+0000 7fe01ce00640 1 -- 172.21.15.87:0/2458657323 wait complete. 2024-04-25T10:27:42.428 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:27:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0ecc00640 1 Processor -- start 2024-04-25T10:27:42.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0ecc00640 1 -- start start 2024-04-25T10:27:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0ecc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0e815bc40 0x7fb0e806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0ecc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0e80cda10 0x7fb0e80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0ecc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e80b2770 con 0x7fb0e80cf930 2024-04-25T10:27:42.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0ecc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e806c670 con 0x7fb0e815bc40 2024-04-25T10:27:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0ecc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e806c950 con 0x7fb0e80cda10 2024-04-25T10:27:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0e5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0e815bc40 0x7fb0e806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0e5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0e815bc40 0x7fb0e806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58720/0 (socket says 172.21.15.87:58720) 2024-04-25T10:27:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0e5e00640 1 -- 172.21.15.87:0/397275234 learned_addr learned my addr 172.21.15.87:0/397275234 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.564+0000 7fb0e5400640 1 --2- 172.21.15.87:0/397275234 >> v2:172.21.15.169:3300/0 conn(0x7fb0e80cda10 0x7fb0e80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e5e00640 1 -- 172.21.15.87:0/397275234 >> v2:172.21.15.169:3300/0 conn(0x7fb0e80cda10 msgr2=0x7fb0e80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:42.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e5e00640 1 --2- 172.21.15.87:0/397275234 >> v2:172.21.15.169:3300/0 conn(0x7fb0e80cda10 0x7fb0e80b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e5e00640 1 -- 172.21.15.87:0/397275234 >> v1:172.21.15.169:6789/0 conn(0x7fb0e80cf930 legacy=0x7fb0e815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e5e00640 1 -- 172.21.15.87:0/397275234 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e806cc30 con 0x7fb0e815bc40 2024-04-25T10:27:42.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e5e00640 1 --2- 172.21.15.87:0/397275234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e815bc40 0x7fb0e806c090 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80726f0 tx=0x7fb0d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82098e1d7a7e7df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e4a00640 1 -- 172.21.15.87:0/397275234 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80ab430 con 0x7fb0e815bc40 2024-04-25T10:27:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e4a00640 1 -- 172.21.15.87:0/397275234 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80b0070 con 0x7fb0e815bc40 2024-04-25T10:27:42.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0e4a00640 1 -- 172.21.15.87:0/397275234 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80ab7f0 con 0x7fb0e815bc40 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0ecc00640 1 -- 172.21.15.87:0/397275234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e815bc40 msgr2=0x7fb0e806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/397275234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e815bc40 0x7fb0e806c090 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80726f0 tx=0x7fb0d801c520 comp rx=0 tx=0).stop 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0ecc00640 1 -- 172.21.15.87:0/397275234 shutdown_connections 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/397275234 >> v2:172.21.15.169:3300/0 conn(0x7fb0e80cda10 0x7fb0e80b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/397275234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e815bc40 0x7fb0e806c090 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0ecc00640 1 -- 172.21.15.87:0/397275234 >> 172.21.15.87:0/397275234 conn(0x7fb0e8066b70 msgr2=0x7fb0e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.568+0000 7fb0ecc00640 1 -- 172.21.15.87:0/397275234 shutdown_connections 2024-04-25T10:27:42.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 -- 172.21.15.87:0/397275234 wait complete. 2024-04-25T10:27:42.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 Processor -- start 2024-04-25T10:27:42.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 -- start start 2024-04-25T10:27:42.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e80cda10 0x7fb0e81460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e80cf930 0x7fb0e80b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815bc40 0x7fb0e8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8168b50 con 0x7fb0e80cda10 2024-04-25T10:27:42.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e81689d0 con 0x7fb0e80cf930 2024-04-25T10:27:42.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8168cd0 con 0x7fb0e815bc40 2024-04-25T10:27:42.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e80cda10 0x7fb0e81460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e80cda10 0x7fb0e81460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58722/0 (socket says 172.21.15.87:58722) 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 -- 172.21.15.87:0/1864027738 learned_addr learned my addr 172.21.15.87:0/1864027738 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5400640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e80cf930 0x7fb0e80b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e6800640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815bc40 0x7fb0e8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 -- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815bc40 msgr2=0x7fb0e8157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815bc40 0x7fb0e8157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 -- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e80cf930 msgr2=0x7fb0e80b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e80cf930 0x7fb0e80b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 -- 172.21.15.87:0/1864027738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80b0f90 con 0x7fb0e80cda10 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0e5e00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e80cda10 0x7fb0e81460f0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80ad040 tx=0x7fb0d801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0de000640 1 -- 172.21.15.87:0/1864027738 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80be5f0 con 0x7fb0e80cda10 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0de000640 1 -- 172.21.15.87:0/1864027738 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80be790 con 0x7fb0e80cda10 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e80b1270 con 0x7fb0e80cda10 2024-04-25T10:27:42.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0de000640 1 -- 172.21.15.87:0/1864027738 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80beb60 con 0x7fb0e80cda10 2024-04-25T10:27:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.572+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e80b1830 con 0x7fb0e80cda10 2024-04-25T10:27:42.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.576+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e804ef30 con 0x7fb0e80cda10 2024-04-25T10:27:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.576+0000 7fb0de000640 1 -- 172.21.15.87:0/1864027738 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0d80c7430 con 0x7fb0e80cda10 2024-04-25T10:27:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.576+0000 7fb0de000640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0c8066f60 0x7fb0c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.576+0000 7fb0de000640 1 -- 172.21.15.87:0/1864027738 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb0d80a8030 con 0x7fb0e80cda10 2024-04-25T10:27:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.576+0000 7fb0e5400640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0c8066f60 0x7fb0c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.576+0000 7fb0e5400640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0c8066f60 0x7fb0c8069420 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb0e815c670 tx=0x7fb0d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.580+0000 7fb0de000640 1 -- 172.21.15.87:0/1864027738 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0d8016110 con 0x7fb0e80cda10 2024-04-25T10:27:42.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.732+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0e806d960 con 0x7fb0c8066f60 2024-04-25T10:27:42.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0de000640 1 -- 172.21.15.87:0/1864027738 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb0e806d960 con 0x7fb0c8066f60 2024-04-25T10:27:42.733 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:27:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0c8066f60 msgr2=0x7fb0c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0c8066f60 0x7fb0c8069420 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb0e815c670 tx=0x7fb0d4008040 comp rx=0 tx=0).stop 2024-04-25T10:27:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e80cda10 msgr2=0x7fb0e81460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e80cda10 0x7fb0e81460f0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80ad040 tx=0x7fb0d801ad60 comp rx=0 tx=0).stop 2024-04-25T10:27:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 shutdown_connections 2024-04-25T10:27:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fb0c8066f60 0x7fb0c8069420 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815bc40 0x7fb0e8157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e80cf930 0x7fb0e80b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 --2- 172.21.15.87:0/1864027738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0e80cda10 0x7fb0e81460f0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 >> 172.21.15.87:0/1864027738 conn(0x7fb0e8066b70 msgr2=0x7fb0e815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 shutdown_connections 2024-04-25T10:27:42.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.736+0000 7fb0ecc00640 1 -- 172.21.15.87:0/1864027738 wait complete. 2024-04-25T10:27:42.754 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:27:42.755 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216e800640 1 Processor -- start 2024-04-25T10:27:42.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216e800640 1 -- start start 2024-04-25T10:27:42.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f21680cf6a0 0x7f21681622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216806ac90 0x7f216806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216e800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21680cc350 con 0x7f2168072870 2024-04-25T10:27:42.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216e800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21680cc630 con 0x7f21680cf6a0 2024-04-25T10:27:42.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21681617f0 con 0x7f216806ac90 2024-04-25T10:27:42.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f21680cf6a0 0x7f21681622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216ca00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2168072870 0x7f21680cb950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33622/0 (socket says 172.21.15.87:33622) 2024-04-25T10:27:42.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216ca00640 1 -- 172.21.15.87:0/1395917630 learned_addr learned my addr 172.21.15.87:0/1395917630 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:42.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216d400640 1 -- 172.21.15.87:0/1395917630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216806ac90 msgr2=0x7f216806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:42.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216d400640 1 --2- 172.21.15.87:0/1395917630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216806ac90 0x7f216806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216d400640 1 -- 172.21.15.87:0/1395917630 >> v1:172.21.15.87:6789/0 conn(0x7f2168072870 legacy=0x7f21680cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f216d400640 1 -- 172.21.15.87:0/1395917630 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168161ad0 con 0x7f21680cf6a0 2024-04-25T10:27:42.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.896+0000 7f2167e00640 1 -- 172.21.15.87:0/1395917630 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722741536 0 0) 0x7f21680cc350 con 0x7f2168072870 2024-04-25T10:27:42.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216d400640 1 --2- 172.21.15.87:0/1395917630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21680cf6a0 0x7f21681622d0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f215c0097f0 tx=0x7f215c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1865fdd73e87fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f2167e00640 1 -- 172.21.15.87:0/1395917630 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0b0070 con 0x7f21680cf6a0 2024-04-25T10:27:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f2167e00640 1 -- 172.21.15.87:0/1395917630 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c0ab600 con 0x7f21680cf6a0 2024-04-25T10:27:42.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f2167e00640 1 -- 172.21.15.87:0/1395917630 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0ab9d0 con 0x7f21680cf6a0 2024-04-25T10:27:42.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 -- 172.21.15.87:0/1395917630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21680cf6a0 msgr2=0x7f21681622d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 --2- 172.21.15.87:0/1395917630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21680cf6a0 0x7f21681622d0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f215c0097f0 tx=0x7f215c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 -- 172.21.15.87:0/1395917630 shutdown_connections 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 --2- 172.21.15.87:0/1395917630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216806ac90 0x7f216806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 --2- 172.21.15.87:0/1395917630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21680cf6a0 0x7f21681622d0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 -- 172.21.15.87:0/1395917630 >> 172.21.15.87:0/1395917630 conn(0x7f21680cfea0 msgr2=0x7f21680d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 -- 172.21.15.87:0/1395917630 shutdown_connections 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.900+0000 7f216e800640 1 -- 172.21.15.87:0/1395917630 wait complete. 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 Processor -- start 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 -- start start 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f216806ac90 0x7f21680b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2168072870 0x7f2168157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21680cf6a0 0x7f21680cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168072190 con 0x7f2168072870 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21680cfc90 con 0x7f216806ac90 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2168072310 con 0x7f21680cf6a0 2024-04-25T10:27:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2168072870 0x7f2168157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2168072870 0x7f2168157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58744/0 (socket says 172.21.15.87:58744) 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 -- 172.21.15.87:0/3988468458 learned_addr learned my addr 172.21.15.87:0/3988468458 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216d400640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f216806ac90 0x7f21680b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216de00640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21680cf6a0 0x7f21680cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 -- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21680cf6a0 msgr2=0x7f21680cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21680cf6a0 0x7f21680cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 -- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f216806ac90 msgr2=0x7f21680b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f216806ac90 0x7f21680b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 -- 172.21.15.87:0/3988468458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168157810 con 0x7f2168072870 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216ca00640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2168072870 0x7f2168157070 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f215400c520 tx=0x7f215400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154018070 con 0x7f2168072870 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21540074e0 con 0x7f2168072870 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21540049f0 con 0x7f2168072870 2024-04-25T10:27:42.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2168157af0 con 0x7f2168072870 2024-04-25T10:27:42.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21681580b0 con 0x7f2168072870 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.904+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21681575b0 con 0x7f2168072870 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f215401b070 con 0x7f2168072870 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f2165600640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2148066f60 0x7f2148069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2154084190 con 0x7f2168072870 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f216d400640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2148066f60 0x7f2148069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f2165600640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7f214806cec0 0x7f214806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 --> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2154017ec0 con 0x7f214806cec0 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f215408b050 con 0x7f2168072870 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f216d400640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2148066f60 0x7f2148069420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f215c0ad000 tx=0x7f215c0a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f216de00640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7f214806cec0 0x7f214806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f216de00640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7f214806cec0 0x7f214806f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:42.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.908+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== osd.5 v2:172.21.15.169:6809/1054103150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2154017ec0 con 0x7f214806cec0 2024-04-25T10:27:42.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 --> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2168075120 con 0x7f214806cec0 2024-04-25T10:27:42.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f2165600640 1 -- 172.21.15.87:0/3988468458 <== osd.5 v2:172.21.15.169:6809/1054103150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2168075120 con 0x7f214806cec0 2024-04-25T10:27:42.922 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:42.922 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:42.922 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:42.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7f214806cec0 msgr2=0x7f214806f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.169:6809/1054103150,v1:172.21.15.169:6811/1054103150] conn(0x7f214806cec0 0x7f214806f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2148066f60 msgr2=0x7f2148069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f2148066f60 0x7f2148069420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f215c0ad000 tx=0x7f215c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2168072870 msgr2=0x7f2168157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 --2- 172.21.15.87:0/3988468458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2168072870 0x7f2168157070 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f215400c520 tx=0x7f215400c9e0 comp rx=0 tx=0).stop 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216de00640 1 -- 172.21.15.87:0/3988468458 reap_dead start 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 shutdown_connections 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 >> 172.21.15.87:0/3988468458 conn(0x7f21680cfea0 msgr2=0x7f21680af990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 shutdown_connections 2024-04-25T10:27:42.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:42.924+0000 7f216e800640 1 -- 172.21.15.87:0/3988468458 wait complete. 2024-04-25T10:27:42.943 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:27:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.080+0000 7f44f0800640 1 Processor -- start 2024-04-25T10:27:43.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.080+0000 7f44f0800640 1 -- start start 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44f0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e81550b0 0x7f44e80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44f0800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f44e80cfe70 0x7f44e815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44f0800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44e815b740 0x7f44e806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44f0800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e8071050 con 0x7f44e80cfe70 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44f0800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80712f0 con 0x7f44e815b740 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44f0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80715d0 con 0x7f44e81550b0 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f44e80cfe70 0x7f44e815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f44e80cfe70 0x7f44e815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58760/0 (socket says 172.21.15.87:58760) 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 -- 172.21.15.87:0/1230010906 learned_addr learned my addr 172.21.15.87:0/1230010906 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ede00640 1 --2- 172.21.15.87:0/1230010906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e81550b0 0x7f44e80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ee800640 1 --2- 172.21.15.87:0/1230010906 >> v2:172.21.15.169:3300/0 conn(0x7f44e815b740 0x7f44e806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 -- 172.21.15.87:0/1230010906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e81550b0 msgr2=0x7f44e80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 --2- 172.21.15.87:0/1230010906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e81550b0 0x7f44e80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 -- 172.21.15.87:0/1230010906 >> v2:172.21.15.169:3300/0 conn(0x7f44e815b740 msgr2=0x7f44e806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 --2- 172.21.15.87:0/1230010906 >> v2:172.21.15.169:3300/0 conn(0x7f44e815b740 0x7f44e806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 -- 172.21.15.87:0/1230010906 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80718b0 con 0x7f44e80cfe70 2024-04-25T10:27:43.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.084+0000 7f44ed400640 1 --2- 172.21.15.87:0/1230010906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e815b200 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f44e80d04d0 tx=0x7f44d801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14af6e6340332a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44eca00640 1 -- 172.21.15.87:0/1230010906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d80b0070 con 0x7f44e80cfe70 2024-04-25T10:27:43.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44eca00640 1 -- 172.21.15.87:0/1230010906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d80ab600 con 0x7f44e80cfe70 2024-04-25T10:27:43.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44eca00640 1 -- 172.21.15.87:0/1230010906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d80ab9d0 con 0x7f44e80cfe70 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- 172.21.15.87:0/1230010906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 msgr2=0x7f44e815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 --2- 172.21.15.87:0/1230010906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e815b200 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f44e80d04d0 tx=0x7f44d801c260 comp rx=0 tx=0).stop 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- 172.21.15.87:0/1230010906 shutdown_connections 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 --2- 172.21.15.87:0/1230010906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e815b200 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 --2- 172.21.15.87:0/1230010906 >> v2:172.21.15.169:3300/0 conn(0x7f44e815b740 0x7f44e806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 --2- 172.21.15.87:0/1230010906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e81550b0 0x7f44e80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- 172.21.15.87:0/1230010906 >> 172.21.15.87:0/1230010906 conn(0x7f44e8066b70 msgr2=0x7f44e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- 172.21.15.87:0/1230010906 shutdown_connections 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- 172.21.15.87:0/1230010906 wait complete. 2024-04-25T10:27:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 Processor -- start 2024-04-25T10:27:43.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- start start 2024-04-25T10:27:43.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44e81550b0 0x7f44e814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e815b740 0x7f44e8150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80b3260 con 0x7f44e80cfe70 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e815bd60 con 0x7f44e81550b0 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44f0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80b33e0 con 0x7f44e815b740 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44ede00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44ede00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e814a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58772/0 (socket says 172.21.15.87:58772) 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.088+0000 7f44ede00640 1 -- 172.21.15.87:0/1283024632 learned_addr learned my addr 172.21.15.87:0/1283024632 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ed400640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44e81550b0 0x7f44e814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ee800640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e815b740 0x7f44e8150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ede00640 1 -- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e815b740 msgr2=0x7f44e8150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ede00640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e815b740 0x7f44e8150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ede00640 1 -- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44e81550b0 msgr2=0x7f44e814e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ede00640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44e81550b0 0x7f44e814e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ede00640 1 -- 172.21.15.87:0/1283024632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e814a920 con 0x7f44e80cfe70 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ede00640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e814a3e0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e930 tx=0x7f44dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44e2000640 1 -- 172.21.15.87:0/1283024632 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc017070 con 0x7f44e80cfe70 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44e2000640 1 -- 172.21.15.87:0/1283024632 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc00ca00 con 0x7f44e80cfe70 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e814df10 con 0x7f44e80cfe70 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44e2000640 1 -- 172.21.15.87:0/1283024632 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00cdb0 con 0x7f44e80cfe70 2024-04-25T10:27:43.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e815c390 con 0x7f44e80cfe70 2024-04-25T10:27:43.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e804ef30 con 0x7f44e80cfe70 2024-04-25T10:27:43.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44e2000640 1 -- 172.21.15.87:0/1283024632 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44dc01a070 con 0x7f44e80cfe70 2024-04-25T10:27:43.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44e2000640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f44cc066f60 0x7f44cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44e2000640 1 -- 172.21.15.87:0/1283024632 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f44dc084dd0 con 0x7f44e80cfe70 2024-04-25T10:27:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.092+0000 7f44ed400640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f44cc066f60 0x7f44cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.096+0000 7f44ed400640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f44cc066f60 0x7f44cc069420 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f44d80ad000 tx=0x7f44d80a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.100+0000 7f44e2000640 1 -- 172.21.15.87:0/1283024632 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44e814a920 con 0x7f44e80cfe70 2024-04-25T10:27:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.252+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f44e80ce810 con 0x7f44cc066f60 2024-04-25T10:27:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44e2000640 1 -- 172.21.15.87:0/1283024632 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f44e80ce810 con 0x7f44cc066f60 2024-04-25T10:27:43.253 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:27:43.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f44cc066f60 msgr2=0x7f44cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f44cc066f60 0x7f44cc069420 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f44d80ad000 tx=0x7f44d80a6000 comp rx=0 tx=0).stop 2024-04-25T10:27:43.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 msgr2=0x7f44e814a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e814a3e0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e930 tx=0x7f44dc00edf0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 shutdown_connections 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f44cc066f60 0x7f44cc069420 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44e815b740 0x7f44e8150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44e81550b0 0x7f44e814e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 --2- 172.21.15.87:0/1283024632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44e80cfe70 0x7f44e814a3e0 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 >> 172.21.15.87:0/1283024632 conn(0x7f44e8066b70 msgr2=0x7f44e8158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 shutdown_connections 2024-04-25T10:27:43.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.256+0000 7f44f0800640 1 -- 172.21.15.87:0/1283024632 wait complete. 2024-04-25T10:27:43.275 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:27:43.275 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:43.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.416+0000 7f0a14200640 1 Processor -- start 2024-04-25T10:27:43.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.416+0000 7f0a14200640 1 -- start start 2024-04-25T10:27:43.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.416+0000 7f0a14200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0a0c15deb0 0x7f0a0c1623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.416+0000 7f0a14200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c0b2d00 0x7f0a0c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a14200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0729c0 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a14200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c072530 con 0x7f0a0c15deb0 2024-04-25T10:27:43.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a14200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c162900 con 0x7f0a0c0b2d00 2024-04-25T10:27:43.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a10e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c0b2d00 0x7f0a0c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a12200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f0a0c0b36a0 0x7f0a0c15e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.87:39470/0 (socket says 172.21.15.87:39470) 2024-04-25T10:27:43.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a12200640 1 -- 172.21.15.87:0/794975387 learned_addr learned my addr 172.21.15.87:0/794975387 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:43.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a11800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0a0c15deb0 0x7f0a0c1623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a10e00640 1 -- 172.21.15.87:0/794975387 >> v2:172.21.15.169:3300/0 conn(0x7f0a0c15deb0 msgr2=0x7f0a0c1623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a10e00640 1 --2- 172.21.15.87:0/794975387 >> v2:172.21.15.169:3300/0 conn(0x7f0a0c15deb0 0x7f0a0c1623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a10e00640 1 -- 172.21.15.87:0/794975387 >> v1:172.21.15.169:6789/0 conn(0x7f0a0c0b36a0 legacy=0x7f0a0c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.420+0000 7f0a10e00640 1 -- 172.21.15.87:0/794975387 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0b3af0 con 0x7f0a0c0b2d00 2024-04-25T10:27:43.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a10e00640 1 --2- 172.21.15.87:0/794975387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c0b2d00 0x7f0a0c0b3160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09f80097c0 tx=0x7f09f801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f44c50452b264fab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a0be00640 1 -- 172.21.15.87:0/794975387 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b1070 con 0x7f0a0c0b2d00 2024-04-25T10:27:43.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a0be00640 1 -- 172.21.15.87:0/794975387 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80a8050 con 0x7f0a0c0b2d00 2024-04-25T10:27:43.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a0be00640 1 -- 172.21.15.87:0/794975387 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80ac860 con 0x7f0a0c0b2d00 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- 172.21.15.87:0/794975387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c0b2d00 msgr2=0x7f0a0c0b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 --2- 172.21.15.87:0/794975387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c0b2d00 0x7f0a0c0b3160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09f80097c0 tx=0x7f09f801c3e0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- 172.21.15.87:0/794975387 shutdown_connections 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 --2- 172.21.15.87:0/794975387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c0b2d00 0x7f0a0c0b3160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 --2- 172.21.15.87:0/794975387 >> v2:172.21.15.169:3300/0 conn(0x7f0a0c15deb0 0x7f0a0c1623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- 172.21.15.87:0/794975387 >> 172.21.15.87:0/794975387 conn(0x7f0a0c0669a0 msgr2=0x7f0a0c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- 172.21.15.87:0/794975387 shutdown_connections 2024-04-25T10:27:43.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- 172.21.15.87:0/794975387 wait complete. 2024-04-25T10:27:43.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 Processor -- start 2024-04-25T10:27:43.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- start start 2024-04-25T10:27:43.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a0c0b2d00 0x7f0a0c0cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a0c0b36a0 0x7f0a0c15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c15deb0 0x7f0a0c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0ae850 con 0x7f0a0c0b2d00 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0721c0 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.424+0000 7f0a14200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0ae9d0 con 0x7f0a0c15deb0 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a11800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a0c0b2d00 0x7f0a0c0cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a11800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a0c0b2d00 0x7f0a0c0cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58786/0 (socket says 172.21.15.87:58786) 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a11800640 1 -- 172.21.15.87:0/256834075 learned_addr learned my addr 172.21.15.87:0/256834075 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a10e00640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a0c0b36a0 0x7f0a0c15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a12200640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c15deb0 0x7f0a0c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a10e00640 1 -- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c15deb0 msgr2=0x7f0a0c14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a10e00640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0c15deb0 0x7f0a0c14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a10e00640 1 -- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a0c0b2d00 msgr2=0x7f0a0c0cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a10e00640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a0c0b2d00 0x7f0a0c0cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a10e00640 1 -- 172.21.15.87:0/256834075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0cf540 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a11800640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a0c0b2d00 0x7f0a0c0cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:43.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a10e00640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a0c0b36a0 0x7f0a0c15ff60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ac000 tx=0x7f09f8019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b9020 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80b5410 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b1040 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0cc780 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1604a0 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a0c153740 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.428+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09f80bc020 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a09600640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f09ec066f90 0x7f09ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f09f811edc0 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a11800640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f09ec066f90 0x7f09ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a09600640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7f09ec06cef0 0x7f09ec06f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 --> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09f80c0e70 con 0x7f09ec06cef0 2024-04-25T10:27:43.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09f811f250 con 0x7f0a0c0b36a0 2024-04-25T10:27:43.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a11800640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f09ec066f90 0x7f09ec069450 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0a000094c0 tx=0x7f0a0000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a12200640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7f09ec06cef0 0x7f09ec06f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a12200640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7f09ec06cef0 0x7f09ec06f310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.432+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== osd.6 v2:172.21.15.169:6808/3889701998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f09f80c0e70 con 0x7f09ec06cef0 2024-04-25T10:27:43.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.444+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 --> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0a0c04ef30 con 0x7f09ec06cef0 2024-04-25T10:27:43.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.444+0000 7f0a09600640 1 -- 172.21.15.87:0/256834075 <== osd.6 v2:172.21.15.169:6808/3889701998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0a0c04ef30 con 0x7f09ec06cef0 2024-04-25T10:27:43.445 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:43.445 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:43.445 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7f09ec06cef0 msgr2=0x7f09ec06f310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:6808/3889701998,v1:172.21.15.169:6810/3889701998] conn(0x7f09ec06cef0 0x7f09ec06f310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f09ec066f90 msgr2=0x7f09ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f09ec066f90 0x7f09ec069450 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0a000094c0 tx=0x7f0a0000b040 comp rx=0 tx=0).stop 2024-04-25T10:27:43.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a0c0b36a0 msgr2=0x7f0a0c15ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 --2- 172.21.15.87:0/256834075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a0c0b36a0 0x7f0a0c15ff60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ac000 tx=0x7f09f8019180 comp rx=0 tx=0).stop 2024-04-25T10:27:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a12200640 1 -- 172.21.15.87:0/256834075 reap_dead start 2024-04-25T10:27:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 shutdown_connections 2024-04-25T10:27:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 >> 172.21.15.87:0/256834075 conn(0x7f0a0c0669a0 msgr2=0x7f0a0c156750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 shutdown_connections 2024-04-25T10:27:43.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.448+0000 7f0a14200640 1 -- 172.21.15.87:0/256834075 wait complete. 2024-04-25T10:27:43.463 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:27:43.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.600+0000 7f27efa00640 1 Processor -- start 2024-04-25T10:27:43.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.600+0000 7f27efa00640 1 -- start start 2024-04-25T10:27:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27efa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f27e80b3190 0x7f27e806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27efa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e8070ed0 con 0x7f27e8071f70 2024-04-25T10:27:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27efa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80711b0 con 0x7f27e815a0e0 2024-04-25T10:27:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27efa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e8071490 con 0x7f27e80b3190 2024-04-25T10:27:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f27e8071f70 0x7f27e80b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33636/0 (socket says 172.21.15.87:33636) 2024-04-25T10:27:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7e00640 1 -- 172.21.15.87:0/4281921439 learned_addr learned my addr 172.21.15.87:0/4281921439 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:43.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27eda00640 1 --2- 172.21.15.87:0/4281921439 >> v2:172.21.15.169:3300/0 conn(0x7f27e80b3190 0x7f27e806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149383884 0 0) 0x7f27e8070ed0 con 0x7f27e8071f70 2024-04-25T10:27:43.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc003530 con 0x7f27e8071f70 2024-04-25T10:27:43.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2956489667 0 0) 0x7f27cc003530 con 0x7f27e8071f70 2024-04-25T10:27:43.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 >> v2:172.21.15.169:3300/0 conn(0x7f27e80b3190 msgr2=0x7f27e806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 --2- 172.21.15.87:0/4281921439 >> v2:172.21.15.169:3300/0 conn(0x7f27e80b3190 0x7f27e806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 >> v1:172.21.15.169:6789/0 conn(0x7f27e815a0e0 legacy=0x7f27e815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e8071770 con 0x7f27e8071f70 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f27d4001680 con 0x7f27e8071f70 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 >> v1:172.21.15.87:6789/0 conn(0x7f27e8071f70 legacy=0x7f27e80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27cc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27cc0043c0 0x7f27cc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27cc005130 0x7f27cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8071490 con 0x7f27cc0039c0 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc00c070 con 0x7f27cc005130 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc00c270 con 0x7f27cc0043c0 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27ed000640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27cc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27eda00640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27cc005130 0x7f27cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27ed000640 1 -- 172.21.15.87:0/4281921439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27cc0043c0 msgr2=0x7f27cc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27ed000640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27cc0043c0 0x7f27cc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27ed000640 1 -- 172.21.15.87:0/4281921439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27cc005130 msgr2=0x7f27cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27ed000640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27cc005130 0x7f27cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27ed000640 1 -- 172.21.15.87:0/4281921439 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc003530 con 0x7f27cc0039c0 2024-04-25T10:27:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.604+0000 7f27eda00640 1 -- 172.21.15.87:0/4281921439 reap_dead start 2024-04-25T10:27:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27ed000640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27cc003da0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001f60 tx=0x7f27d801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16fcf32015b3da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80ac070 con 0x7f27cc0039c0 2024-04-25T10:27:43.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d80a6610 con 0x7f27cc0039c0 2024-04-25T10:27:43.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27e7400640 1 -- 172.21.15.87:0/4281921439 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8001b60 con 0x7f27cc0039c0 2024-04-25T10:27:43.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27efa00640 1 -- 172.21.15.87:0/4281921439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 msgr2=0x7f27cc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27efa00640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27cc003da0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001f60 tx=0x7f27d801c850 comp rx=0 tx=0).stop 2024-04-25T10:27:43.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27efa00640 1 -- 172.21.15.87:0/4281921439 shutdown_connections 2024-04-25T10:27:43.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27efa00640 1 --2- 172.21.15.87:0/4281921439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27cc003da0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27efa00640 1 -- 172.21.15.87:0/4281921439 >> 172.21.15.87:0/4281921439 conn(0x7f27e8066b70 msgr2=0x7f27e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27efa00640 1 -- 172.21.15.87:0/4281921439 shutdown_connections 2024-04-25T10:27:43.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.608+0000 7f27efa00640 1 -- 172.21.15.87:0/4281921439 wait complete. 2024-04-25T10:27:43.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 Processor -- start 2024-04-25T10:27:43.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 -- start start 2024-04-25T10:27:43.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27e81e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e81e6c80 0x7f27e80af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e8155e40 0x7f27e81562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80b2e90 con 0x7f27cc0039c0 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8070b50 con 0x7f27e8155e40 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80b3010 con 0x7f27e81e6c80 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27e81e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27e81e90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58800/0 (socket says 172.21.15.87:58800) 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 -- 172.21.15.87:0/1591073470 learned_addr learned my addr 172.21.15.87:0/1591073470 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27e7e00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e81e6c80 0x7f27e80af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27eda00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e8155e40 0x7f27e81562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 -- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e81e6c80 msgr2=0x7f27e80af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e81e6c80 0x7f27e80af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 -- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e8155e40 msgr2=0x7f27e81562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e8155e40 0x7f27e81562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 -- 172.21.15.87:0/1591073470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e81e9820 con 0x7f27cc0039c0 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27eda00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e8155e40 0x7f27e81562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27e7e00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e81e6c80 0x7f27e80af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27ed000640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27e81e90f0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000f90 tx=0x7f27cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27e4c00640 1 -- 172.21.15.87:0/1591073470 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80ac070 con 0x7f27cc0039c0 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27e4c00640 1 -- 172.21.15.87:0/1591073470 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d80a69c0 con 0x7f27cc0039c0 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27e4c00640 1 -- 172.21.15.87:0/1591073470 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80a6d70 con 0x7f27cc0039c0 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e81e9b00 con 0x7f27cc0039c0 2024-04-25T10:27:43.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81e70e0 con 0x7f27cc0039c0 2024-04-25T10:27:43.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.612+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e81ea020 con 0x7f27cc0039c0 2024-04-25T10:27:43.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.616+0000 7f27e4c00640 1 -- 172.21.15.87:0/1591073470 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27d80af070 con 0x7f27cc0039c0 2024-04-25T10:27:43.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.616+0000 7f27e4c00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f27c8066f60 0x7f27c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.616+0000 7f27e4c00640 1 -- 172.21.15.87:0/1591073470 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f27d811e9b0 con 0x7f27cc0039c0 2024-04-25T10:27:43.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.616+0000 7f27e7e00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f27c8066f60 0x7f27c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.616+0000 7f27e7e00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f27c8066f60 0x7f27c8069420 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f27e80b0330 tx=0x7f27d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.620+0000 7f27e4c00640 1 -- 172.21.15.87:0/1591073470 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27e81ea020 con 0x7f27cc0039c0 2024-04-25T10:27:43.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.776+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f27e804ef30 con 0x7f27c8066f60 2024-04-25T10:27:43.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.776+0000 7f27e4c00640 1 -- 172.21.15.87:0/1591073470 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f27e804ef30 con 0x7f27c8066f60 2024-04-25T10:27:43.777 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f27c8066f60 msgr2=0x7f27c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f27c8066f60 0x7f27c8069420 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f27e80b0330 tx=0x7f27d4009040 comp rx=0 tx=0).stop 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 msgr2=0x7f27e81e90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27e81e90f0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000f90 tx=0x7f27cc005840 comp rx=0 tx=0).stop 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 shutdown_connections 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f27c8066f60 0x7f27c8069420 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e8155e40 0x7f27e81562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e81e6c80 0x7f27e80af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 --2- 172.21.15.87:0/1591073470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27cc0039c0 0x7f27e81e90f0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 >> 172.21.15.87:0/1591073470 conn(0x7f27e8066b70 msgr2=0x7f27e815b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 shutdown_connections 2024-04-25T10:27:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.780+0000 7f27efa00640 1 -- 172.21.15.87:0/1591073470 wait complete. 2024-04-25T10:27:43.799 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:27:43.799 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:27:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c58c00640 1 Processor -- start 2024-04-25T10:27:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c58c00640 1 -- start start 2024-04-25T10:27:43.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c58c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 0x7f1c540d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c58c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c540b0930 0x7f1c540cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c58c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c540cf5a0 con 0x7f1c540d01a0 2024-04-25T10:27:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c58c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c540cedb0 con 0x7f1c540b0930 2024-04-25T10:27:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c58c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540cf880 con 0x7f1c5415bcd0 2024-04-25T10:27:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c52800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1c540d01a0 0x7f1c5415f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:33642/0 (socket says 172.21.15.87:33642) 2024-04-25T10:27:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c52800640 1 -- 172.21.15.87:0/1510934081 learned_addr learned my addr 172.21.15.87:0/1510934081 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:27:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c4be00640 1 --2- 172.21.15.87:0/1510934081 >> v2:172.21.15.169:3300/0 conn(0x7f1c540b0930 0x7f1c540cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.940+0000 7f1c51e00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 0x7f1c540d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884853110 0 0) 0x7f1c540cf5a0 con 0x7f1c540d01a0 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c38003530 con 0x7f1c540d01a0 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2602245293 0 0) 0x7f1c38003530 con 0x7f1c540d01a0 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 msgr2=0x7f1c540d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 0x7f1c540d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 >> v2:172.21.15.169:3300/0 conn(0x7f1c540b0930 msgr2=0x7f1c540cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 --2- 172.21.15.87:0/1510934081 >> v2:172.21.15.169:3300/0 conn(0x7f1c540b0930 0x7f1c540cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c540cf000 con 0x7f1c540d01a0 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f1c4c001a00 con 0x7f1c540d01a0 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c4be00640 1 --2- 172.21.15.87:0/1510934081 >> v2:172.21.15.169:3300/0 conn(0x7f1c540b0930 0x7f1c540cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 >> v1:172.21.15.87:6789/0 conn(0x7f1c540d01a0 legacy=0x7f1c5415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 0x7f1c380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38003f40 0x7f1c38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c38004d50 0x7f1c380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540cedb0 con 0x7f1c38003f40 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540cf880 con 0x7f1c38004d50 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3800fde0 con 0x7f1c5415bcd0 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c4be00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38003f40 0x7f1c38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51e00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 0x7f1c380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c52800640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c38004d50 0x7f1c380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c4be00640 1 -- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 msgr2=0x7f1c380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c4be00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 0x7f1c380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c4be00640 1 -- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c38004d50 msgr2=0x7f1c380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c4be00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c38004d50 0x7f1c380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c4be00640 1 -- 172.21.15.87:0/1510934081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c38003530 con 0x7f1c38003f40 2024-04-25T10:27:43.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.944+0000 7f1c51e00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c5415bcd0 0x7f1c380039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c4be00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38003f40 0x7f1c38004810 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009cb0 tx=0x7f1c3c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74093eac45f7037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c0b2070 con 0x7f1c38003f40 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c0a5840 con 0x7f1c38003f40 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c51400640 1 -- 172.21.15.87:0/1510934081 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c009880 con 0x7f1c38003f40 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 -- 172.21.15.87:0/1510934081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38003f40 msgr2=0x7f1c38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 --2- 172.21.15.87:0/1510934081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38003f40 0x7f1c38004810 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009cb0 tx=0x7f1c3c01cc30 comp rx=0 tx=0).stop 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c52800640 1 -- 172.21.15.87:0/1510934081 reap_dead start 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 -- 172.21.15.87:0/1510934081 shutdown_connections 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 -- 172.21.15.87:0/1510934081 >> 172.21.15.87:0/1510934081 conn(0x7f1c540d19d0 msgr2=0x7f1c540d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 -- 172.21.15.87:0/1510934081 shutdown_connections 2024-04-25T10:27:43.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 -- 172.21.15.87:0/1510934081 wait complete. 2024-04-25T10:27:43.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 Processor -- start 2024-04-25T10:27:43.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 -- start start 2024-04-25T10:27:43.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c38003f40 0x7f1c5406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.948+0000 7f1c58c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c540b7440 0x7f1c54159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c58c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c540caa90 0x7f1c541ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c58c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5415c230 con 0x7f1c540caa90 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c58c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5415c0b0 con 0x7f1c540b7440 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c58c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540b1330 con 0x7f1c38003f40 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c540caa90 0x7f1c541ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c540caa90 0x7f1c541ee9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58816/0 (socket says 172.21.15.87:58816) 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 -- 172.21.15.87:0/828553125 learned_addr learned my addr 172.21.15.87:0/828553125 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c51e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c38003f40 0x7f1c5406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c4be00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c540b7440 0x7f1c54159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 -- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c38003f40 msgr2=0x7f1c5406b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c38003f40 0x7f1c5406b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 -- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c540b7440 msgr2=0x7f1c54159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c540b7440 0x7f1c54159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 -- 172.21.15.87:0/828553125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c540cb430 con 0x7f1c540caa90 2024-04-25T10:27:43.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c52800640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c540caa90 0x7f1c541ee9c0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f1c540cef30 tx=0x7f1c4c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c010040 con 0x7f1c540caa90 2024-04-25T10:27:43.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c01a070 con 0x7f1c540caa90 2024-04-25T10:27:43.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00faa0 con 0x7f1c540caa90 2024-04-25T10:27:43.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c540cb710 con 0x7f1c540caa90 2024-04-25T10:27:43.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5406bc20 con 0x7f1c540caa90 2024-04-25T10:27:43.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c5404ef30 con 0x7f1c540caa90 2024-04-25T10:27:43.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c4c00fc40 con 0x7f1c540caa90 2024-04-25T10:27:43.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.952+0000 7f1c4a000640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c30066f60 0x7f1c30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1c4c087bf0 con 0x7f1c540caa90 2024-04-25T10:27:43.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c51e00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c30066f60 0x7f1c30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c4a000640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f1c3006cec0 0x7f1c3006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:43.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 --> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c4c023ec0 con 0x7f1c3006cec0 2024-04-25T10:27:43.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c4c08c2a0 con 0x7f1c540caa90 2024-04-25T10:27:43.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c4be00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f1c3006cec0 0x7f1c3006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:43.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c51e00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c30066f60 0x7f1c30069420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009a80 tx=0x7f1c44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c4be00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f1c3006cec0 0x7f1c3006f2e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:43.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.956+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== osd.7 v2:172.21.15.169:6824/2370587194 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1c4c023ec0 con 0x7f1c3006cec0 2024-04-25T10:27:43.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.968+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 --> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1c540b7950 con 0x7f1c3006cec0 2024-04-25T10:27:43.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.968+0000 7f1c4a000640 1 -- 172.21.15.87:0/828553125 <== osd.7 v2:172.21.15.169:6824/2370587194 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1c540b7950 con 0x7f1c3006cec0 2024-04-25T10:27:43.969 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T10:27:43.969 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:27:43.969 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f1c3006cec0 msgr2=0x7f1c3006f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.169:6824/2370587194,v1:172.21.15.169:6825/2370587194] conn(0x7f1c3006cec0 0x7f1c3006f2e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c30066f60 msgr2=0x7f1c30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f1c30066f60 0x7f1c30069420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009a80 tx=0x7f1c44008040 comp rx=0 tx=0).stop 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c540caa90 msgr2=0x7f1c541ee9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 --2- 172.21.15.87:0/828553125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c540caa90 0x7f1c541ee9c0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f1c540cef30 tx=0x7f1c4c011c20 comp rx=0 tx=0).stop 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c52800640 1 -- 172.21.15.87:0/828553125 reap_dead start 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 shutdown_connections 2024-04-25T10:27:43.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 >> 172.21.15.87:0/828553125 conn(0x7f1c540d19d0 msgr2=0x7f1c5415af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:43.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 shutdown_connections 2024-04-25T10:27:43.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:43.972+0000 7f1c58c00640 1 -- 172.21.15.87:0/828553125 wait complete. 2024-04-25T10:27:43.987 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:27:44.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56eb000640 1 Processor -- start 2024-04-25T10:27:44.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56eb000640 1 -- start start 2024-04-25T10:27:44.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56eb000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56e41550b0 0x7f56e40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56eb000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f56e415b910 0x7f56e40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56eb000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e4071050 con 0x7f56e40cfe70 2024-04-25T10:27:44.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56eb000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e415bd80 con 0x7f56e41550b0 2024-04-25T10:27:44.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56eb000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e415d040 con 0x7f56e415b910 2024-04-25T10:27:44.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56e41550b0 0x7f56e40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56e41550b0 0x7f56e40cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58822/0 (socket says 172.21.15.87:58822) 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e3e00640 1 -- 172.21.15.87:0/813273532 learned_addr learned my addr 172.21.15.87:0/813273532 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e9000640 1 --2- 172.21.15.87:0/813273532 >> v2:172.21.15.169:3300/0 conn(0x7f56e415b910 0x7f56e40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e3e00640 1 -- 172.21.15.87:0/813273532 >> v2:172.21.15.169:3300/0 conn(0x7f56e415b910 msgr2=0x7f56e40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e3e00640 1 --2- 172.21.15.87:0/813273532 >> v2:172.21.15.169:3300/0 conn(0x7f56e415b910 0x7f56e40b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e3e00640 1 -- 172.21.15.87:0/813273532 >> v1:172.21.15.169:6789/0 conn(0x7f56e40cfe70 legacy=0x7f56e415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e3e00640 1 -- 172.21.15.87:0/813273532 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40712f0 con 0x7f56e41550b0 2024-04-25T10:27:44.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.124+0000 7f56e9000640 1 --2- 172.21.15.87:0/813273532 >> v2:172.21.15.169:3300/0 conn(0x7f56e415b910 0x7f56e40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:44.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56e3e00640 1 --2- 172.21.15.87:0/813273532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e40cf930 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8026dc5ceec022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56e3400640 1 -- 172.21.15.87:0/813273532 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0b1070 con 0x7f56e41550b0 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56e3400640 1 -- 172.21.15.87:0/813273532 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc0ac630 con 0x7f56e41550b0 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56e3400640 1 -- 172.21.15.87:0/813273532 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0a8050 con 0x7f56e41550b0 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 -- 172.21.15.87:0/813273532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 msgr2=0x7f56e40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 --2- 172.21.15.87:0/813273532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e40cf930 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 -- 172.21.15.87:0/813273532 shutdown_connections 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 --2- 172.21.15.87:0/813273532 >> v2:172.21.15.169:3300/0 conn(0x7f56e415b910 0x7f56e40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 --2- 172.21.15.87:0/813273532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e40cf930 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 -- 172.21.15.87:0/813273532 >> 172.21.15.87:0/813273532 conn(0x7f56e4066b70 msgr2=0x7f56e4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 -- 172.21.15.87:0/813273532 shutdown_connections 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.128+0000 7f56eb000640 1 -- 172.21.15.87:0/813273532 wait complete. 2024-04-25T10:27:44.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 Processor -- start 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 -- start start 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e40cfe70 0x7f56e41514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e415b910 0x7f56e41fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e415c030 con 0x7f56e41550b0 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40720b0 con 0x7f56e40cfe70 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e415c1b0 con 0x7f56e415b910 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e414f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58830/0 (socket says 172.21.15.87:58830) 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 -- 172.21.15.87:0/3103835603 learned_addr learned my addr 172.21.15.87:0/3103835603 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56e9000640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e415b910 0x7f56e41fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56e3e00640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e40cfe70 0x7f56e41514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 -- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e415b910 msgr2=0x7f56e41fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e415b910 0x7f56e41fa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 -- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e40cfe70 msgr2=0x7f56e41514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e40cfe70 0x7f56e41514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 -- 172.21.15.87:0/3103835603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4155490 con 0x7f56e41550b0 2024-04-25T10:27:44.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56db400640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e414f860 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c480 tx=0x7f56d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56e0c00640 1 -- 172.21.15.87:0/3103835603 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4012830 con 0x7f56e41550b0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56e0c00640 1 -- 172.21.15.87:0/3103835603 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d4017070 con 0x7f56e41550b0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56e0c00640 1 -- 172.21.15.87:0/3103835603 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4012c30 con 0x7f56e41550b0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e41fad80 con 0x7f56e41550b0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.132+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e4155730 con 0x7f56e41550b0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.136+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e404ef30 con 0x7f56e41550b0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.136+0000 7f56e0c00640 1 -- 172.21.15.87:0/3103835603 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56d401c460 con 0x7f56e41550b0 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.136+0000 7f56e0c00640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f56c0066f60 0x7f56c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.136+0000 7f56e0c00640 1 -- 172.21.15.87:0/3103835603 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f56d4084b40 con 0x7f56e41550b0 2024-04-25T10:27:44.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.136+0000 7f56e3e00640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f56c0066f60 0x7f56c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.136+0000 7f56e3e00640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f56c0066f60 0x7f56c0069420 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0ae000 tx=0x7f56cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:44.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.140+0000 7f56e0c00640 1 -- 172.21.15.87:0/3103835603 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56d4055e80 con 0x7f56e41550b0 2024-04-25T10:27:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.292+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f56e406cca0 con 0x7f56c0066f60 2024-04-25T10:27:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.292+0000 7f56e0c00640 1 -- 172.21.15.87:0/3103835603 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56e406cca0 con 0x7f56c0066f60 2024-04-25T10:27:44.292 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:27:44.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f56c0066f60 msgr2=0x7f56c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f56c0066f60 0x7f56c0069420 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0ae000 tx=0x7f56cc008040 comp rx=0 tx=0).stop 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 msgr2=0x7f56e414f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e414f860 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c480 tx=0x7f56d400c940 comp rx=0 tx=0).stop 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 shutdown_connections 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f56c0066f60 0x7f56c0069420 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e415b910 0x7f56e41fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56e41550b0 0x7f56e414f860 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 --2- 172.21.15.87:0/3103835603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e40cfe70 0x7f56e41514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 >> 172.21.15.87:0/3103835603 conn(0x7f56e4066b70 msgr2=0x7f56e40ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 shutdown_connections 2024-04-25T10:27:44.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.296+0000 7f56eb000640 1 -- 172.21.15.87:0/3103835603 wait complete. 2024-04-25T10:27:44.313 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:27:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.448+0000 7ff4eaa00640 1 Processor -- start 2024-04-25T10:27:44.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.448+0000 7ff4eaa00640 1 -- start start 2024-04-25T10:27:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.448+0000 7ff4eaa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4e41550b0 0x7ff4e40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.448+0000 7ff4eaa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4e40cfe70 0x7ff4e415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.448+0000 7ff4eaa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e4071050 con 0x7ff4e415b740 2024-04-25T10:27:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4eaa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e415d040 con 0x7ff4e41550b0 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4eaa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e40712f0 con 0x7ff4e40cfe70 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4e41550b0 0x7ff4e40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4e41550b0 0x7ff4e40cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58834/0 (socket says 172.21.15.87:58834) 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 -- 172.21.15.87:0/2011933179 learned_addr learned my addr 172.21.15.87:0/2011933179 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4db400640 1 --2- 172.21.15.87:0/2011933179 >> v2:172.21.15.169:3300/0 conn(0x7ff4e40cfe70 0x7ff4e415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 -- 172.21.15.87:0/2011933179 >> v2:172.21.15.169:3300/0 conn(0x7ff4e40cfe70 msgr2=0x7ff4e415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 --2- 172.21.15.87:0/2011933179 >> v2:172.21.15.169:3300/0 conn(0x7ff4e40cfe70 0x7ff4e415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 -- 172.21.15.87:0/2011933179 >> v1:172.21.15.169:6789/0 conn(0x7ff4e415b740 legacy=0x7ff4e406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 -- 172.21.15.87:0/2011933179 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e40715d0 con 0x7ff4e41550b0 2024-04-25T10:27:44.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3e00640 1 --2- 172.21.15.87:0/2011933179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e40cf930 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009870 tx=0x7ff4d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a4c8c75e62863c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3400640 1 -- 172.21.15.87:0/2011933179 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40b0070 con 0x7ff4e41550b0 2024-04-25T10:27:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3400640 1 -- 172.21.15.87:0/2011933179 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d40ab600 con 0x7ff4e41550b0 2024-04-25T10:27:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.452+0000 7ff4e3400640 1 -- 172.21.15.87:0/2011933179 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40ab9d0 con 0x7ff4e41550b0 2024-04-25T10:27:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- 172.21.15.87:0/2011933179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 msgr2=0x7ff4e40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/2011933179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e40cf930 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009870 tx=0x7ff4d401c520 comp rx=0 tx=0).stop 2024-04-25T10:27:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- 172.21.15.87:0/2011933179 shutdown_connections 2024-04-25T10:27:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/2011933179 >> v2:172.21.15.169:3300/0 conn(0x7ff4e40cfe70 0x7ff4e415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/2011933179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e40cf930 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- 172.21.15.87:0/2011933179 >> 172.21.15.87:0/2011933179 conn(0x7ff4e4066b70 msgr2=0x7ff4e4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- 172.21.15.87:0/2011933179 shutdown_connections 2024-04-25T10:27:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- 172.21.15.87:0/2011933179 wait complete. 2024-04-25T10:27:44.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 Processor -- start 2024-04-25T10:27:44.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- start start 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e40cfe70 0x7ff4e4155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e4159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415b740 0x7ff4e4159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e415bcd0 con 0x7ff4e41550b0 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e415bb50 con 0x7ff4e415b740 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4eaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e415c030 con 0x7ff4e40cfe70 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e4159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e4159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58850/0 (socket says 172.21.15.87:58850) 2024-04-25T10:27:44.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 -- 172.21.15.87:0/76045925 learned_addr learned my addr 172.21.15.87:0/76045925 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 -- 172.21.15.87:0/76045925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e40cfe70 msgr2=0x7ff4e4155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e40cfe70 0x7ff4e4155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 -- 172.21.15.87:0/76045925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415b740 msgr2=0x7ff4e4159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415b740 0x7ff4e4159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.456+0000 7ff4db400640 1 -- 172.21.15.87:0/76045925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e4158400 con 0x7ff4e41550b0 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4db400640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e4159a50 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00e930 tx=0x7ff4cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e0c00640 1 -- 172.21.15.87:0/76045925 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc017070 con 0x7ff4e41550b0 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e0c00640 1 -- 172.21.15.87:0/76045925 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc00ca00 con 0x7ff4e41550b0 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e413ad20 con 0x7ff4e41550b0 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e0c00640 1 -- 172.21.15.87:0/76045925 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc00cdb0 con 0x7ff4e41550b0 2024-04-25T10:27:44.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e413a240 con 0x7ff4e41550b0 2024-04-25T10:27:44.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e404ef30 con 0x7ff4e41550b0 2024-04-25T10:27:44.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e0c00640 1 -- 172.21.15.87:0/76045925 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4cc01a070 con 0x7ff4e41550b0 2024-04-25T10:27:44.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e0c00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4c0066f60 0x7ff4c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:44.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e0c00640 1 -- 172.21.15.87:0/76045925 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff4cc084dd0 con 0x7ff4e41550b0 2024-04-25T10:27:44.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e3e00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4c0066f60 0x7ff4c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:44.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.460+0000 7ff4e3e00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4c0066f60 0x7ff4c0069420 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40ad000 tx=0x7ff4d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:44.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.464+0000 7ff4e0c00640 1 -- 172.21.15.87:0/76045925 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4e4158400 con 0x7ff4e41550b0 2024-04-25T10:27:44.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.616+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4e40b2710 con 0x7ff4c0066f60 2024-04-25T10:27:44.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4e0c00640 1 -- 172.21.15.87:0/76045925 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13418 (secure 0 0 0) 0x7ff4e40b2710 con 0x7ff4c0066f60 2024-04-25T10:27:44.618 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:27:44.618 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:27:44.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4c0066f60 msgr2=0x7ff4c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4c0066f60 0x7ff4c0069420 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40ad000 tx=0x7ff4d4008040 comp rx=0 tx=0).stop 2024-04-25T10:27:44.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 msgr2=0x7ff4e4159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:44.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e4159a50 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00e930 tx=0x7ff4cc00edf0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 shutdown_connections 2024-04-25T10:27:44.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7ff4c0066f60 0x7ff4c0069420 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415b740 0x7ff4e4159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e41550b0 0x7ff4e4159a50 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 --2- 172.21.15.87:0/76045925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e40cfe70 0x7ff4e4155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:44.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 >> 172.21.15.87:0/76045925 conn(0x7ff4e4066b70 msgr2=0x7ff4e4151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:44.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 shutdown_connections 2024-04-25T10:27:44.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:27:44.620+0000 7ff4eaa00640 1 -- 172.21.15.87:0/76045925 wait complete. 2024-04-25T10:27:44.638 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":652,"stamp":"2024-04-25T10:27:43.324408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":470932,"kb_used_data":20264,"kb_used_omap":12,"kb_used_meta":450547,"kb_avail":754503788,"statfs":{"total":773094113280,"available":772611878912,"internally_reserved":0,"allocated":20750336,"data_stored":9786080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":461360689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12582912,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-8,"num_write_kb":-12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.424122"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":486,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:27:34.907879+0000","last_change":"2024-04-25T10:12:12.520853+0000","last_active":"2024-04-25T10:27:34.907879+0000","last_peered":"2024-04-25T10:27:34.907879+0000","last_clean":"2024-04-25T10:27:34.907879+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:27:34.907879+0000","last_undegraded":"2024-04-25T10:27:34.907879+0000","last_fullsized":"2024-04-25T10:27:34.907879+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:11.413799+0000","last_clean_scrub_stamp":"2024-04-25T10:12:11.413799+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:15.553741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48044,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323796,"statfs":{"total":96636764160,"available":96587567104,"internally_reserved":0,"allocated":3039232,"data_stored":1666036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65440,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306400,"statfs":{"total":96636764160,"available":96569753600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51616,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320224,"statfs":{"total":96636764160,"available":96583909376,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64096,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307744,"statfs":{"total":96636764160,"available":96571129856,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59940,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55776,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316064,"statfs":{"total":96636764160,"available":96579649536,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69664,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302176,"statfs":{"total":96636764160,"available":96565428224,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56356,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315484,"statfs":{"total":96636764160,"available":96579055616,"internally_reserved":0,"allocated":3039232,"data_stored":1666036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:27:44.642 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:12:11.413799+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=12, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=27, tm_sec=39, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:27:44.642 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:27:54.742 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:27:54.746 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:04.643 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:28:04.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8fa00640 1 Processor -- start 2024-04-25T10:28:04.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8fa00640 1 -- start start 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8fa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb880cda10 0x7fcb8806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8fa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb880cf930 0x7fcb880cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8fa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb880b2770 con 0x7fcb8815bc40 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8fa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8806c670 con 0x7fcb880cda10 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8fa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8806c950 con 0x7fcb880cf930 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb87e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb880cda10 0x7fcb8806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8d000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fcb8815bc40 0x7fcb880b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36686/0 (socket says 172.21.15.87:36686) 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb87e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb880cda10 0x7fcb8806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58274/0 (socket says 172.21.15.87:58274) 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8d000640 1 -- 172.21.15.87:0/1572180232 learned_addr learned my addr 172.21.15.87:0/1572180232 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:28:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb8da00640 1 --2- 172.21.15.87:0/1572180232 >> v2:172.21.15.169:3300/0 conn(0x7fcb880cf930 0x7fcb880cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb87e00640 1 -- 172.21.15.87:0/1572180232 >> v2:172.21.15.169:3300/0 conn(0x7fcb880cf930 msgr2=0x7fcb880cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:04.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb87e00640 1 --2- 172.21.15.87:0/1572180232 >> v2:172.21.15.169:3300/0 conn(0x7fcb880cf930 0x7fcb880cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb87e00640 1 -- 172.21.15.87:0/1572180232 >> v1:172.21.15.87:6789/0 conn(0x7fcb8815bc40 legacy=0x7fcb880b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb87e00640 1 -- 172.21.15.87:0/1572180232 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8806cc30 con 0x7fcb880cda10 2024-04-25T10:28:04.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.792+0000 7fcb87400640 1 -- 172.21.15.87:0/1572180232 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495849415 0 0) 0x7fcb880b2770 con 0x7fcb8815bc40 2024-04-25T10:28:04.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb87e00640 1 --2- 172.21.15.87:0/1572180232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8806c090 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ae4c0 tx=0x7fcb7001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ee1d3e23cc7e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:04.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb87400640 1 -- 172.21.15.87:0/1572180232 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700b0070 con 0x7fcb880cda10 2024-04-25T10:28:04.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb87400640 1 -- 172.21.15.87:0/1572180232 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb700ab600 con 0x7fcb880cda10 2024-04-25T10:28:04.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb87400640 1 -- 172.21.15.87:0/1572180232 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700ab9d0 con 0x7fcb880cda10 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 -- 172.21.15.87:0/1572180232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 msgr2=0x7fcb8806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/1572180232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8806c090 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ae4c0 tx=0x7fcb7001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 -- 172.21.15.87:0/1572180232 shutdown_connections 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/1572180232 >> v2:172.21.15.169:3300/0 conn(0x7fcb880cf930 0x7fcb880cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/1572180232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8806c090 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 -- 172.21.15.87:0/1572180232 >> 172.21.15.87:0/1572180232 conn(0x7fcb88066b70 msgr2=0x7fcb88066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 -- 172.21.15.87:0/1572180232 shutdown_connections 2024-04-25T10:28:04.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.796+0000 7fcb8fa00640 1 -- 172.21.15.87:0/1572180232 wait complete. 2024-04-25T10:28:04.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 Processor -- start 2024-04-25T10:28:04.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 -- start start 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8814a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb880cf930 0x7fcb8814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8815bc40 0x7fcb8814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b3850 con 0x7fcb880cda10 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b36d0 con 0x7fcb880cf930 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b39d0 con 0x7fcb8815bc40 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8814a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8814a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58282/0 (socket says 172.21.15.87:58282) 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 -- 172.21.15.87:0/462546850 learned_addr learned my addr 172.21.15.87:0/462546850 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb87e00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb880cf930 0x7fcb8814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8da00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8815bc40 0x7fcb8814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 -- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8815bc40 msgr2=0x7fcb8814c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8815bc40 0x7fcb8814c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 -- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb880cf930 msgr2=0x7fcb8814bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb880cf930 0x7fcb8814bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 -- 172.21.15.87:0/462546850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8814a560 con 0x7fcb880cda10 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8d000640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8814a020 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c4b0 tx=0x7fcb7800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb84c00640 1 -- 172.21.15.87:0/462546850 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78018070 con 0x7fcb880cda10 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb84c00640 1 -- 172.21.15.87:0/462546850 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb78007500 con 0x7fcb880cda10 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8815ab80 con 0x7fcb880cda10 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb84c00640 1 -- 172.21.15.87:0/462546850 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78004a30 con 0x7fcb880cda10 2024-04-25T10:28:04.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8815a5c0 con 0x7fcb880cda10 2024-04-25T10:28:04.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.800+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8804ef30 con 0x7fcb880cda10 2024-04-25T10:28:04.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.804+0000 7fcb84c00640 1 -- 172.21.15.87:0/462546850 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb7801b070 con 0x7fcb880cda10 2024-04-25T10:28:04.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.804+0000 7fcb84c00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb64066f60 0x7fcb64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:04.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.804+0000 7fcb84c00640 1 -- 172.21.15.87:0/462546850 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcb780840b0 con 0x7fcb880cda10 2024-04-25T10:28:04.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.804+0000 7fcb87e00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb64066f60 0x7fcb64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:04.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.804+0000 7fcb87e00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb64066f60 0x7fcb64069420 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcb700ad000 tx=0x7fcb700a6000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:04.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.808+0000 7fcb84c00640 1 -- 172.21.15.87:0/462546850 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb780552c0 con 0x7fcb880cda10 2024-04-25T10:28:04.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.964+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 --> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb880cf4e0 con 0x7fcb64066f60 2024-04-25T10:28:04.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.964+0000 7fcb84c00640 1 -- 172.21.15.87:0/462546850 <== mgr.4113 v2:172.21.15.87:6832/3202965986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13385 (secure 0 0 0) 0x7fcb880cf4e0 con 0x7fcb64066f60 2024-04-25T10:28:04.964 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:28:04.964 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb64066f60 msgr2=0x7fcb64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb64066f60 0x7fcb64069420 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcb700ad000 tx=0x7fcb700a6000 comp rx=0 tx=0).stop 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 msgr2=0x7fcb8814a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8814a020 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c4b0 tx=0x7fcb7800c970 comp rx=0 tx=0).stop 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 shutdown_connections 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7fcb64066f60 0x7fcb64069420 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8815bc40 0x7fcb8814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb880cf930 0x7fcb8814bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 --2- 172.21.15.87:0/462546850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb880cda10 0x7fcb8814a020 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:04.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 >> 172.21.15.87:0/462546850 conn(0x7fcb88066b70 msgr2=0x7fcb880ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:04.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 shutdown_connections 2024-04-25T10:28:04.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:04.968+0000 7fcb8fa00640 1 -- 172.21.15.87:0/462546850 wait complete. 2024-04-25T10:28:04.985 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":662,"stamp":"2024-04-25T10:28:03.328586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":470916,"kb_used_data":20264,"kb_used_omap":12,"kb_used_meta":450547,"kb_avail":754503804,"statfs":{"total":773094113280,"available":772611895296,"internally_reserved":0,"allocated":20750336,"data_stored":9786080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":461360689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002701"},"pg_stats":[{"pgid":"1.0","version":"193'119","reported_seq":497,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:27:44.600432+0000","last_change":"2024-04-25T10:27:44.600460+0000","last_active":"2024-04-25T10:27:44.600432+0000","last_peered":"2024-04-25T10:27:44.600432+0000","last_clean":"2024-04-25T10:27:44.600432+0000","last_became_active":"2024-04-25T10:12:12.519722+0000","last_became_peered":"2024-04-25T10:12:12.519722+0000","last_unstale":"2024-04-25T10:27:44.600432+0000","last_undegraded":"2024-04-25T10:27:44.600432+0000","last_fullsized":"2024-04-25T10:27:44.600432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'119","last_scrub_stamp":"2024-04-25T10:27:44.600356+0000","last_deep_scrub":"193'119","last_deep_scrub_stamp":"2024-04-25T10:27:44.600356+0000","last_clean_scrub_stamp":"2024-04-25T10:27:44.600356+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:52.721891+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575040,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48032,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323808,"statfs":{"total":96636764160,"available":96587579392,"internally_reserved":0,"allocated":3039232,"data_stored":1666036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65440,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306400,"statfs":{"total":96636764160,"available":96569753600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51616,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320224,"statfs":{"total":96636764160,"available":96583909376,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64096,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307744,"statfs":{"total":96636764160,"available":96571129856,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59940,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55776,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316064,"statfs":{"total":96636764160,"available":96579649536,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69664,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302176,"statfs":{"total":96636764160,"available":96565428224,"internally_reserved":0,"allocated":2445312,"data_stored":1075668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56352,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315488,"statfs":{"total":96636764160,"available":96579059712,"internally_reserved":0,"allocated":3039232,"data_stored":1666036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:28:04.986 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:28:05.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f3375a00640 1 Processor -- start 2024-04-25T10:28:05.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f3375a00640 1 -- start start 2024-04-25T10:28:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f3375a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3370159540 0x7f33700ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f3375a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f33700721f0 0x7f3370070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f3375a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3370159920 con 0x7f33700aebd0 2024-04-25T10:28:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f3375a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337015c2f0 con 0x7f33700721f0 2024-04-25T10:28:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f3375a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337015c5d0 con 0x7f3370159540 2024-04-25T10:28:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f33700721f0 0x7f3370070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f33700721f0 0x7f3370070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58290/0 (socket says 172.21.15.87:58290) 2024-04-25T10:28:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336f400640 1 -- 172.21.15.87:0/1191121115 learned_addr learned my addr 172.21.15.87:0/1191121115 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:28:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336ea00640 1 --2- 172.21.15.87:0/1191121115 >> v2:172.21.15.169:3300/0 conn(0x7f3370159540 0x7f33700ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336f400640 1 -- 172.21.15.87:0/1191121115 >> v2:172.21.15.169:3300/0 conn(0x7f3370159540 msgr2=0x7f33700ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336f400640 1 --2- 172.21.15.87:0/1191121115 >> v2:172.21.15.169:3300/0 conn(0x7f3370159540 0x7f33700ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336f400640 1 -- 172.21.15.87:0/1191121115 >> v1:172.21.15.169:6789/0 conn(0x7f33700aebd0 legacy=0x7f3370071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336f400640 1 -- 172.21.15.87:0/1191121115 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337015c8b0 con 0x7f33700721f0 2024-04-25T10:28:05.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.124+0000 7f336ea00640 1 --2- 172.21.15.87:0/1191121115 >> v2:172.21.15.169:3300/0 conn(0x7f3370159540 0x7f33700ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:05.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f336f400640 1 --2- 172.21.15.87:0/1191121115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700721f0 0x7f3370070350 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f33700b4c20 tx=0x7f336801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949dd877bb685ead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f336d600640 1 -- 172.21.15.87:0/1191121115 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33680a5780 con 0x7f33700721f0 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f336d600640 1 -- 172.21.15.87:0/1191121115 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33680a5920 con 0x7f33700721f0 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f336d600640 1 -- 172.21.15.87:0/1191121115 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33680a5cf0 con 0x7f33700721f0 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 -- 172.21.15.87:0/1191121115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700721f0 msgr2=0x7f3370070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 --2- 172.21.15.87:0/1191121115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700721f0 0x7f3370070350 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f33700b4c20 tx=0x7f336801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 -- 172.21.15.87:0/1191121115 shutdown_connections 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 --2- 172.21.15.87:0/1191121115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700721f0 0x7f3370070350 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 --2- 172.21.15.87:0/1191121115 >> v2:172.21.15.169:3300/0 conn(0x7f3370159540 0x7f33700ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 -- 172.21.15.87:0/1191121115 >> 172.21.15.87:0/1191121115 conn(0x7f33700669a0 msgr2=0x7f3370066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:05.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 -- 172.21.15.87:0/1191121115 shutdown_connections 2024-04-25T10:28:05.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 -- 172.21.15.87:0/1191121115 wait complete. 2024-04-25T10:28:05.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.128+0000 7f3375a00640 1 Processor -- start 2024-04-25T10:28:05.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 -- start start 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33700721f0 0x7f33701505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700aebd0 0x7f33700b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3370159540 0x7f3370159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3370071220 con 0x7f33700aebd0 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33700710a0 con 0x7f3370159540 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33700713a0 con 0x7f33700721f0 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700aebd0 0x7f33700b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700aebd0 0x7f33700b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58296/0 (socket says 172.21.15.87:58296) 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 -- 172.21.15.87:0/3400851687 learned_addr learned my addr 172.21.15.87:0/3400851687 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336f400640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3370159540 0x7f3370159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336ea00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33700721f0 0x7f33701505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 -- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33700721f0 msgr2=0x7f33701505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33700721f0 0x7f33701505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 -- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3370159540 msgr2=0x7f3370159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3370159540 0x7f3370159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 -- 172.21.15.87:0/3400851687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33700b0950 con 0x7f33700aebd0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f336e000640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700aebd0 0x7f33700b00e0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c480 tx=0x7f335c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c012830 con 0x7f33700aebd0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c017070 con 0x7f33700aebd0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33700b0c30 con 0x7f33700aebd0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c012c30 con 0x7f33700aebd0 2024-04-25T10:28:05.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33700b11f0 con 0x7f33700aebd0 2024-04-25T10:28:05.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337004ef30 con 0x7f33700aebd0 2024-04-25T10:28:05.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.132+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f335c01c460 con 0x7f33700aebd0 2024-04-25T10:28:05.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.136+0000 7f3366a00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3350066f60 0x7f3350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.136+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f335c084b70 con 0x7f33700aebd0 2024-04-25T10:28:05.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.136+0000 7f336ea00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3350066f60 0x7f3350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.136+0000 7f336ea00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3350066f60 0x7f3350069420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3360006fd0 tx=0x7f3360008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.140+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f335c055e10 con 0x7f33700aebd0 2024-04-25T10:28:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.296+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3370166700 con 0x7f33700aebd0 2024-04-25T10:28:05.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.300+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v38)=0 v38) v1 ==== 123+0+0 (secure 0 0 0) 0x7f335c055710 con 0x7f33700aebd0 2024-04-25T10:28:05.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.300+0000 7f3366a00640 1 -- 172.21.15.87:0/3400851687 <== mon.0 v2:172.21.15.87:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f335c08a080 con 0x7f33700aebd0 2024-04-25T10:28:05.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3350066f60 msgr2=0x7f3350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3350066f60 0x7f3350069420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3360006fd0 tx=0x7f3360008040 comp rx=0 tx=0).stop 2024-04-25T10:28:05.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700aebd0 msgr2=0x7f33700b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700aebd0 0x7f33700b00e0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f335c00c480 tx=0x7f335c00c940 comp rx=0 tx=0).stop 2024-04-25T10:28:05.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 shutdown_connections 2024-04-25T10:28:05.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3350066f60 0x7f3350069420 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3370159540 0x7f3370159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33700aebd0 0x7f33700b00e0 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 --2- 172.21.15.87:0/3400851687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33700721f0 0x7f33701505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 >> 172.21.15.87:0/3400851687 conn(0x7f33700669a0 msgr2=0x7f3370066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:05.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 shutdown_connections 2024-04-25T10:28:05.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.304+0000 7f3375a00640 1 -- 172.21.15.87:0/3400851687 wait complete. 2024-04-25T10:28:05.322 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:28:05.323 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b22000640 1 Processor -- start 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b22000640 1 -- start start 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b22000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b1c0ccb20 0x7f3b1c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b22000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 0x7f3b1c0d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b22000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c0ccfb0 con 0x7f3b1c15b9e0 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b22000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c0cd290 con 0x7f3b1c0ccb20 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b22000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0cd570 con 0x7f3b1c15bdc0 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1b400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3b1c15b9e0 0x7f3b1c0cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:36702/0 (socket says 172.21.15.87:36702) 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1b400640 1 -- 172.21.15.87:0/3382890278 learned_addr learned my addr 172.21.15.87:0/3382890278 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1aa00640 1 --2- 172.21.15.87:0/3382890278 >> v2:172.21.15.169:3300/0 conn(0x7f3b1c0ccb20 0x7f3b1c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1be00640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 0x7f3b1c0d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 <== mon.0 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004834434 0 0) 0x7f3b1c0ccfb0 con 0x7f3b1c15b9e0 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b00003530 con 0x7f3b1c15b9e0 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 <== mon.0 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221982072 0 0) 0x7f3b00003530 con 0x7f3b1c15b9e0 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 msgr2=0x7f3b1c0d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 0x7f3b1c0d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 >> v2:172.21.15.169:3300/0 conn(0x7f3b1c0ccb20 msgr2=0x7f3b1c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 --2- 172.21.15.87:0/3382890278 >> v2:172.21.15.169:3300/0 conn(0x7f3b1c0ccb20 0x7f3b1c0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0cd850 con 0x7f3b1c15b9e0 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 <== mon.0 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2796968820 0 0) 0x7f3b0c0016a0 con 0x7f3b1c15b9e0 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 >> v1:172.21.15.87:6789/0 conn(0x7f3b1c15b9e0 legacy=0x7f3b1c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b000039c0 0x7f3b00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 0x7f3b00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b00004d50 0x7f3b000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0cd290 con 0x7f3b00004d50 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0cd570 con 0x7f3b000039c0 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0000b9d0 con 0x7f3b1c15bdc0 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1be00640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b00004d50 0x7f3b000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1b400640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b000039c0 0x7f3b00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.460+0000 7f3b1aa00640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 0x7f3b00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1be00640 1 -- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 msgr2=0x7f3b00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1be00640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b1c15bdc0 0x7f3b00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1be00640 1 -- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b000039c0 msgr2=0x7f3b00003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1be00640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b000039c0 0x7f3b00003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1be00640 1 -- 172.21.15.87:0/3382890278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00003530 con 0x7f3b00004d50 2024-04-25T10:28:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1be00640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b00004d50 0x7f3b000051d0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e170 tx=0x7f3b1401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdebcdeab08be8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140b2070 con 0x7f3b00004d50 2024-04-25T10:28:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3b140a5c70 con 0x7f3b00004d50 2024-04-25T10:28:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1a000640 1 -- 172.21.15.87:0/3382890278 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14009c10 con 0x7f3b00004d50 2024-04-25T10:28:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b22000640 1 -- 172.21.15.87:0/3382890278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b00004d50 msgr2=0x7f3b000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b22000640 1 --2- 172.21.15.87:0/3382890278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b00004d50 0x7f3b000051d0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e170 tx=0x7f3b1401cc60 comp rx=0 tx=0).stop 2024-04-25T10:28:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.464+0000 7f3b1be00640 1 -- 172.21.15.87:0/3382890278 reap_dead start 2024-04-25T10:28:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- 172.21.15.87:0/3382890278 shutdown_connections 2024-04-25T10:28:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- 172.21.15.87:0/3382890278 >> 172.21.15.87:0/3382890278 conn(0x7f3b1c09a9d0 msgr2=0x7f3b1c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- 172.21.15.87:0/3382890278 shutdown_connections 2024-04-25T10:28:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- 172.21.15.87:0/3382890278 wait complete. 2024-04-25T10:28:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 Processor -- start 2024-04-25T10:28:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- start start 2024-04-25T10:28:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b00004d50 0x7f3b1c06c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1c0ca200 0x7f3b1c06b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1c1f7090 0x7f3b1c1f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0d1a60 con 0x7f3b1c0ca200 2024-04-25T10:28:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0cf740 con 0x7f3b1c1f7090 2024-04-25T10:28:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0d1be0 con 0x7f3b00004d50 2024-04-25T10:28:05.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1c0ca200 0x7f3b1c06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1c0ca200 0x7f3b1c06b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58310/0 (socket says 172.21.15.87:58310) 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 -- 172.21.15.87:0/4037615034 learned_addr learned my addr 172.21.15.87:0/4037615034 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 -- 172.21.15.87:0/4037615034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b00004d50 msgr2=0x7f3b1c06c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b00004d50 0x7f3b1c06c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 -- 172.21.15.87:0/4037615034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1c1f7090 msgr2=0x7f3b1c1f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1c1f7090 0x7f3b1c1f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 -- 172.21.15.87:0/4037615034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0ca680 con 0x7f3b1c0ca200 2024-04-25T10:28:05.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b1aa00640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1c0ca200 0x7f3b1c06b610 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f3b080119b0 tx=0x7f3b08011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b13400640 1 -- 172.21.15.87:0/4037615034 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08019070 con 0x7f3b1c0ca200 2024-04-25T10:28:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b13400640 1 -- 172.21.15.87:0/4037615034 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3b08004240 con 0x7f3b1c0ca200 2024-04-25T10:28:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b13400640 1 -- 172.21.15.87:0/4037615034 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800f860 con 0x7f3b1c0ca200 2024-04-25T10:28:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.468+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c0c9480 con 0x7f3b1c0ca200 2024-04-25T10:28:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.472+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c0c9a00 con 0x7f3b1c0ca200 2024-04-25T10:28:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.472+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c04ef30 con 0x7f3b1c0ca200 2024-04-25T10:28:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.472+0000 7f3b13400640 1 -- 172.21.15.87:0/4037615034 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b0801c070 con 0x7f3b1c0ca200 2024-04-25T10:28:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.472+0000 7f3b13400640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.472+0000 7f3b13400640 1 -- 172.21.15.87:0/4037615034 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3b080853c0 con 0x7f3b1c0ca200 2024-04-25T10:28:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.472+0000 7f3b1b400640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.472+0000 7f3b1b400640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3afc066f60 0x7f3afc069420 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0c8450 tx=0x7f3b0c000bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:05.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.476+0000 7f3b13400640 1 -- 172.21.15.87:0/4037615034 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b1c0ca680 con 0x7f3b1c0ca200 2024-04-25T10:28:05.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.664+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b1c0b29f0 con 0x7f3b1c0ca200 2024-04-25T10:28:05.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.664+0000 7f3b13400640 1 -- 172.21.15.87:0/4037615034 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+3712 (secure 0 0 0) 0x7f3b08056540 con 0x7f3b1c0ca200 2024-04-25T10:28:05.664 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:28:05.664 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":145,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3250372792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":3250372792},{"type":"v1","addr":"172.21.15.87:6837","nonce":3250372792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":8897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1364964567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1364964567},{"type":"v1","addr":"172.21.15.169:6834","nonce":1364964567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":10267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/4171274616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":4171274616},{"type":"v1","addr":"172.21.15.169:6835","nonce":4171274616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132},{"gid":11811,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/1544524850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1544524850},{"type":"v1","addr":"172.21.15.87:6836","nonce":1544524850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":145}],"filesystems":[]} 2024-04-25T10:28:05.664 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 145 2024-04-25T10:28:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3afc066f60 msgr2=0x7f3afc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3afc066f60 0x7f3afc069420 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0c8450 tx=0x7f3b0c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1c0ca200 msgr2=0x7f3b1c06b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1c0ca200 0x7f3b1c06b610 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f3b080119b0 tx=0x7f3b08011e70 comp rx=0 tx=0).stop 2024-04-25T10:28:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 shutdown_connections 2024-04-25T10:28:05.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:6832/3202965986,v1:172.21.15.87:6833/3202965986] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1c1f7090 0x7f3b1c1f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b1c0ca200 0x7f3b1c06b610 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 --2- 172.21.15.87:0/4037615034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b00004d50 0x7f3b1c06c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 >> 172.21.15.87:0/4037615034 conn(0x7f3b1c09a9d0 msgr2=0x7f3b1c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 shutdown_connections 2024-04-25T10:28:05.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T10:28:05.668+0000 7f3b22000640 1 -- 172.21.15.87:0/4037615034 wait complete. 2024-04-25T10:28:05.685 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:28:05.685 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:28:05.686 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:28:05.686 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:05.698 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:28:05.698 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:28:05.698 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:05.751 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:28:05.751 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:28:05.751 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:05.765 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:28:05.765 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:28:05.765 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:05.824 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:28:05.825 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:28:05.825 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:28:05.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:05.922 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:28:05.922 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:28:05.922 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:08.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-25T10:28:12.024 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:28:12.024 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:28:12.025 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:12.101 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:28:12.102 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:28:12.102 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:12.202 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:28:12.203 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:28:12.203 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:12.296 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:28:12.296 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:28:12.296 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:14.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T10:28:14.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-25T10:28:18.398 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:28:18.399 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:28:18.399 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:18.464 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:28:18.464 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:28:18.464 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:18.540 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:28:18.540 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:28:18.540 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:28:18.540 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:18.599 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:28:18.600 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:28:18.600 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:18.653 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:28:18.653 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:28:18.653 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:28:18.653 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:18.674 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:28:18.675 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:28:18.675 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:18.692 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:28:18.692 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:28:18.692 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:18.755 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:28:18.755 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:28:18.756 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T10:28:18.789 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:28:18.789 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:28:19.095 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:28:19.095 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:28:19.295 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:28:19.295 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:28:19.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T10:28:19.498 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:28:19.498 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:28:19.676 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:28:19.677 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:28:19.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:28:19.828 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:28:20.016 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:28:20.016 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:28:20.165 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:28:20.165 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:28:20.319 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:28:20.320 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/data/mon.a.tgz 2024-04-25T10:28:20.320 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-04-25T10:28:20.329 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.MkIRZ4YKr9 2024-04-25T10:28:20.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MkIRZ4YKr9 2024-04-25T10:28:20.480 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.MkIRZ4YKr9 2024-04-25T10:28:20.553 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.MkIRZ4YKr9 is 462KB 2024-04-25T10:28:20.615 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.MkIRZ4YKr9 2024-04-25T10:28:20.623 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/data/mon.b.tgz 2024-04-25T10:28:20.623 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-25T10:28:20.632 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.rhy5LDmhee 2024-04-25T10:28:20.632 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rhy5LDmhee 2024-04-25T10:28:20.786 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.rhy5LDmhee 2024-04-25T10:28:20.863 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.rhy5LDmhee is 462KB 2024-04-25T10:28:20.928 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.rhy5LDmhee 2024-04-25T10:28:20.937 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/data/mon.c.tgz 2024-04-25T10:28:20.937 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-25T10:28:20.985 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.P9raWMA9XX 2024-04-25T10:28:20.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.P9raWMA9XX 2024-04-25T10:28:21.133 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.P9raWMA9XX 2024-04-25T10:28:21.206 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.P9raWMA9XX is 462KB 2024-04-25T10:28:21.268 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.P9raWMA9XX 2024-04-25T10:28:21.276 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:28:21.277 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:28:21.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:28:21.352 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:28:21.386 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:28:21.386 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:28:21.388 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/remote/smithi087/crash 2024-04-25T10:28:21.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:28:21.411 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/remote/smithi169/crash 2024-04-25T10:28:21.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:28:21.447 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:28:21.447 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:21.463 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:21.548 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:21.567 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:21.586 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:28:21.587 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:28:21.615 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:28:21.644 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84688.log 2024-04-25T10:28:21.645 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88713.log 2024-04-25T10:28:21.645 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81699.log 2024-04-25T10:28:21.646 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.84688.log.gz 2024-04-25T10:28:21.646 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90183.log 2024-04-25T10:28:21.646 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88713.log.gz 2024-04-25T10:28:21.647 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74701.log 2024-04-25T10:28:21.647 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81699.log.gz 2024-04-25T10:28:21.648 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85260.log 2024-04-25T10:28:21.648 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90183.log.gz 2024-04-25T10:28:21.649 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54595.log 2024-04-25T10:28:21.649 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74701.log.gz 2024-04-25T10:28:21.649 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90368.log 2024-04-25T10:28:21.650 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54595.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.85260.log.gz 2024-04-25T10:28:21.650 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91425.log 2024-04-25T10:28:21.650 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54595.log.gz 2024-04-25T10:28:21.651 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50254.log 2024-04-25T10:28:21.651 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90368.log.gz 2024-04-25T10:28:21.652 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36344.log 2024-04-25T10:28:21.652 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91425.log.gz 2024-04-25T10:28:21.653 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-04-25T10:28:21.653 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50254.log.gz 2024-04-25T10:28:21.653 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84400.log 2024-04-25T10:28:21.654 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36344.log.gz 2024-04-25T10:28:21.654 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85165.log 2024-04-25T10:28:21.654 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-04-25T10:28:21.655 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55724.log 2024-04-25T10:28:21.655 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84400.log.gz 2024-04-25T10:28:21.656 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.85165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85165.log.gz 2024-04-25T10:28:21.656 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37333.log 2024-04-25T10:28:21.657 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84296.log 2024-04-25T10:28:21.658 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55724.log.gz 2024-04-25T10:28:21.658 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72763.log 2024-04-25T10:28:21.658 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37333.log.gz 2024-04-25T10:28:21.659 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33983.log 2024-04-25T10:28:21.659 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84296.log.gz 2024-04-25T10:28:21.659 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27728.log 2024-04-25T10:28:21.660 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81121.log 2024-04-25T10:28:21.660 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72763.log.gz 2024-04-25T10:28:21.660 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54482.log 2024-04-25T10:28:21.660 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33983.log.gz 2024-04-25T10:28:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:28:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27752.log 2024-04-25T10:28:21.664 INFO:teuthology.orchestra.run.smithi169.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27728.log.gz 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27848.log 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27752.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27752.log.gz 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33405.log 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27848.log.gz 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27776.log 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.3.33405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33405.log.gz 2024-04-25T10:28:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:28:21.666 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36103.log 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81121.log.gz 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85523.log 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54482.log.gz 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54214.log 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36103.log.gz 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87640.log 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85523.log.gz 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81667.log 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54214.log.gz 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77816.log 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87640.log.gz 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83834.log 2024-04-25T10:28:21.667 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81667.log.gz 2024-04-25T10:28:21.668 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54945.log 2024-04-25T10:28:21.668 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77816.log.gz 2024-04-25T10:28:21.669 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27800.log 2024-04-25T10:28:21.669 INFO:teuthology.orchestra.run.smithi169.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27776.log.gz 2024-04-25T10:28:21.669 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:28:21.669 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27800.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27800.log.gz 2024-04-25T10:28:21.669 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:28:21.669 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27872.log 2024-04-25T10:28:21.670 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27824.log 2024-04-25T10:28:21.671 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27872.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:28:21.671 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27872.log.gz 2024-04-25T10:28:21.671 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27824.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:28:21.672 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27824.log.gz 2024-04-25T10:28:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80725.log 2024-04-25T10:28:21.675 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83834.log.gz 2024-04-25T10:28:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51945.log 2024-04-25T10:28:21.675 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54945.log.gz 2024-04-25T10:28:21.675 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77894.log 2024-04-25T10:28:21.675 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80725.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72885.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51945.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56262.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77894.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.72885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72885.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74308.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85867.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56262.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74943.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74308.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74538.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85867.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74943.log.gz 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49805.log 2024-04-25T10:28:21.676 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74538.log.gz 2024-04-25T10:28:21.678 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:28:21.678 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:28:21.678 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27896.log 2024-04-25T10:28:21.679 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.27896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27896.log.gz 2024-04-25T10:28:21.679 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33311.log 2024-04-25T10:28:21.680 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75482.log 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55542.log 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75482.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49805.log.gz 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75862.log 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75482.log.gz 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34402.log 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55542.log.gz 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75862.log.gz 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72044.log 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34402.log.gz 2024-04-25T10:28:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52120.log 2024-04-25T10:28:21.683 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.33311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33311.log.gz 2024-04-25T10:28:21.683 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33329.log 2024-04-25T10:28:21.683 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:28:21.683 INFO:teuthology.orchestra.run.smithi169.stderr: 94.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:28:21.683 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:28:21.683 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.33329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33329.log.gz 2024-04-25T10:28:21.683 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72044.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72044.log.gz 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68917.log 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mgr.y.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52120.log.gz 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27839.log 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51789.log 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68917.log.gz 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.27839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75430.log 2024-04-25T10:28:21.685 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27839.log.gz 2024-04-25T10:28:21.686 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:28:21.687 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78247.log 2024-04-25T10:28:21.687 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75430.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51789.log.gz 2024-04-25T10:28:21.688 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68891.log 2024-04-25T10:28:21.688 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75430.log.gz 2024-04-25T10:28:21.688 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51062.log 2024-04-25T10:28:21.688 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78247.log.gz 2024-04-25T10:28:21.688 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33687.log 2024-04-25T10:28:21.689 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68891.log.gz 2024-04-25T10:28:21.689 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71966.log 2024-04-25T10:28:21.689 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51062.log.gz 2024-04-25T10:28:21.690 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37137.log 2024-04-25T10:28:21.690 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33687.log.gz 2024-04-25T10:28:21.690 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33410.log 2024-04-25T10:28:21.691 INFO:teuthology.orchestra.run.smithi169.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:28:21.691 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55386.log 2024-04-25T10:28:21.691 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71966.log.gz 2024-04-25T10:28:21.692 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53928.log 2024-04-25T10:28:21.692 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37137.log.gz 2024-04-25T10:28:21.692 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87770.log 2024-04-25T10:28:21.692 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55386.log.gz 2024-04-25T10:28:21.693 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91685.log 2024-04-25T10:28:21.693 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53928.log.gz 2024-04-25T10:28:21.694 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-04-25T10:28:21.694 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87770.log.gz 2024-04-25T10:28:21.695 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55750.log 2024-04-25T10:28:21.695 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91685.log.gz 2024-04-25T10:28:21.696 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69888.log 2024-04-25T10:28:21.697 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.3.33410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33410.log.gz 2024-04-25T10:28:21.697 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz 2024-04-25T10:28:21.697 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55750.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55750.log.gz 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78273.log 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.69888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35334.log 0.0% 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.0.69888.log.gz 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90100.log 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78273.log.gz 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85234.log 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35334.log.gz 2024-04-25T10:28:21.700 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50116.log 2024-04-25T10:28:21.700 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90100.log.gz 2024-04-25T10:28:21.701 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69413.log 2024-04-25T10:28:21.701 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85234.log.gz 2024-04-25T10:28:21.702 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85919.log 2024-04-25T10:28:21.702 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50116.log.gz 2024-04-25T10:28:21.702 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35778.log 2024-04-25T10:28:21.703 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69413.log.gz 2024-04-25T10:28:21.703 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36863.log 2024-04-25T10:28:21.703 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85919.log.gz 2024-04-25T10:28:21.704 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36077.log 2024-04-25T10:28:21.704 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35778.log.gz 2024-04-25T10:28:21.705 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86552.log 2024-04-25T10:28:21.705 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36863.log.gz 2024-04-25T10:28:21.706 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36077.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.84206.log 2024-04-25T10:28:21.706 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36077.log.gz 2024-04-25T10:28:21.707 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90074.log 2024-04-25T10:28:21.707 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86552.log.gz 2024-04-25T10:28:21.707 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54858.log 2024-04-25T10:28:21.707 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84206.log.gz 2024-04-25T10:28:21.708 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84180.log 2024-04-25T10:28:21.709 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90074.log.gz 2024-04-25T10:28:21.709 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55055.log 2024-04-25T10:28:21.710 INFO:teuthology.orchestra.run.smithi087.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54858.log.gz 2024-04-25T10:28:21.710 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49905.log 2024-04-25T10:28:21.710 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84180.log.gz 2024-04-25T10:28:21.711 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53706.log 2024-04-25T10:28:21.711 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55055.log.gz 2024-04-25T10:28:21.712 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89327.log 2024-04-25T10:28:21.712 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53706.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49905.log.gz 2024-04-25T10:28:21.712 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88957.log 2024-04-25T10:28:21.713 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53706.log.gz 2024-04-25T10:28:21.713 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55601.log 2024-04-25T10:28:21.713 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89327.log.gz 2024-04-25T10:28:21.714 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69334.log 2024-04-25T10:28:21.714 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88957.log.gz 2024-04-25T10:28:21.715 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89009.log 2024-04-25T10:28:21.715 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55601.log.gz 2024-04-25T10:28:21.716 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78721.log 2024-04-25T10:28:21.716 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69334.log.gz 2024-04-25T10:28:21.716 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89640.log 2024-04-25T10:28:21.717 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89009.log.gz 2024-04-25T10:28:21.717 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88827.log 2024-04-25T10:28:21.718 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78721.log.gz 2024-04-25T10:28:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88059.log 2024-04-25T10:28:21.719 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89640.log.gz 2024-04-25T10:28:21.719 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72890.log 2024-04-25T10:28:21.719 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88827.log.gz 2024-04-25T10:28:21.720 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87926.log 2024-04-25T10:28:21.720 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88059.log.gz 2024-04-25T10:28:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.72890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72890.log.gz 2024-04-25T10:28:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50663.log 2024-04-25T10:28:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35830.log 2024-04-25T10:28:21.722 INFO:teuthology.orchestra.run.smithi087.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.87926.log.gz 2024-04-25T10:28:21.723 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75535.log 2024-04-25T10:28:21.723 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50663.log.gz 2024-04-25T10:28:21.723 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34206.log 2024-04-25T10:28:21.724 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35830.log.gz 2024-04-25T10:28:21.724 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71549.log 2024-04-25T10:28:21.725 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.75535.log.gz 2024-04-25T10:28:21.725 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85184.log 2024-04-25T10:28:21.725 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34206.log.gz 2024-04-25T10:28:21.727 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85633.log 2024-04-25T10:28:21.727 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71549.log.gz 2024-04-25T10:28:21.727 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87283.log 2024-04-25T10:28:21.728 INFO:teuthology.orchestra.run.smithi087.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.85184.log.gz 2024-04-25T10:28:21.728 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85633.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:28:21.728 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85633.log.gz 2024-04-25T10:28:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-04-25T10:28:21.729 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87283.log.gz 2024-04-25T10:28:21.730 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75678.log 2024-04-25T10:28:21.730 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55289.log 2024-04-25T10:28:21.730 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-04-25T10:28:21.731 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50611.log 2024-04-25T10:28:21.732 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75678.log.gz 2024-04-25T10:28:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55289.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55289.log.gz 2024-04-25T10:28:21.733 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35181.log 2024-04-25T10:28:21.734 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36422.log 2024-04-25T10:28:21.734 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50611.log.gz 2024-04-25T10:28:21.734 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.35181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35181.log.gz 2024-04-25T10:28:21.734 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55107.log 2024-04-25T10:28:21.735 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55490.log 2024-04-25T10:28:21.735 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36422.log.gz 2024-04-25T10:28:21.736 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53366.log 2024-04-25T10:28:21.736 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55107.log.gz 2024-04-25T10:28:21.737 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52504.log 2024-04-25T10:28:21.737 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55490.log.gz 2024-04-25T10:28:21.739 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74727.log 2024-04-25T10:28:21.739 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53366.log.gz 2024-04-25T10:28:21.739 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36046.log 2024-04-25T10:28:21.740 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52504.log.gz 2024-04-25T10:28:21.740 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33923.log 2024-04-25T10:28:21.740 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74727.log.gz 2024-04-25T10:28:21.740 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91215.log 2024-04-25T10:28:21.741 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36046.log.gz 2024-04-25T10:28:21.741 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33923.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33923.log.gz 2024-04-25T10:28:21.741 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86926.log 2024-04-25T10:28:21.742 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33652.log 2024-04-25T10:28:21.742 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91215.log.gz 2024-04-25T10:28:21.743 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83886.log 2024-04-25T10:28:21.743 INFO:teuthology.orchestra.run.smithi087.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.86926.log.gz 2024-04-25T10:28:21.744 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69893.log 2024-04-25T10:28:21.744 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33652.log.gz 2024-04-25T10:28:21.745 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33998.log 2024-04-25T10:28:21.745 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83886.log.gz 2024-04-25T10:28:21.746 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.69893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69893.log.gz 2024-04-25T10:28:21.746 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50384.log 2024-04-25T10:28:21.747 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50900.log 2024-04-25T10:28:21.747 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33998.log.gz 2024-04-25T10:28:21.748 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83752.log 2024-04-25T10:28:21.748 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50384.log.gz 2024-04-25T10:28:21.748 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54791.log 2024-04-25T10:28:21.749 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50900.log.gz 2024-04-25T10:28:21.749 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79348.log 2024-04-25T10:28:21.750 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54791.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83752.log.gz 2024-04-25T10:28:21.750 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54791.log.gz 2024-04-25T10:28:21.750 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90962.log 2024-04-25T10:28:21.751 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88434.log 2024-04-25T10:28:21.751 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79348.log.gz 2024-04-25T10:28:21.752 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74282.log 2024-04-25T10:28:21.752 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90962.log.gz 2024-04-25T10:28:21.753 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53208.log 2024-04-25T10:28:21.753 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88434.log.gz 2024-04-25T10:28:21.754 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27720.log 2024-04-25T10:28:21.754 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74282.log.gz 2024-04-25T10:28:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79181.log 2024-04-25T10:28:21.755 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53208.log.gz 2024-04-25T10:28:21.755 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.27720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34885.log 2024-04-25T10:28:21.756 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27720.log.gz 2024-04-25T10:28:21.756 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79283.log 2024-04-25T10:28:21.756 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79181.log.gz 2024-04-25T10:28:21.757 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53954.log 2024-04-25T10:28:21.757 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34885.log.gz 2024-04-25T10:28:21.758 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85893.log 2024-04-25T10:28:21.758 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79283.log.gz 2024-04-25T10:28:21.759 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54008.log 2024-04-25T10:28:21.759 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53954.log.gz 2024-04-25T10:28:21.760 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78865.log 2024-04-25T10:28:21.760 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85893.log.gz 2024-04-25T10:28:21.760 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34859.log 2024-04-25T10:28:21.761 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54008.log.gz 2024-04-25T10:28:21.761 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81899.log 2024-04-25T10:28:21.762 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78865.log.gz 2024-04-25T10:28:21.762 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55464.log 2024-04-25T10:28:21.762 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34859.log.gz 2024-04-25T10:28:21.763 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34911.log 2024-04-25T10:28:21.763 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81899.log.gz 2024-04-25T10:28:21.764 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37571.log 2024-04-25T10:28:21.764 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55464.log.gz 2024-04-25T10:28:21.765 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71523.log 2024-04-25T10:28:21.765 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34911.log.gz 2024-04-25T10:28:21.766 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34625.log 2024-04-25T10:28:21.766 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37571.log.gz 2024-04-25T10:28:21.767 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78377.log 2024-04-25T10:28:21.767 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71523.log.gz 2024-04-25T10:28:21.768 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55965.log 2024-04-25T10:28:21.768 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34625.log.gz 2024-04-25T10:28:21.768 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86784.log 2024-04-25T10:28:21.768 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78377.log.gz 2024-04-25T10:28:21.769 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75021.log 2024-04-25T10:28:21.769 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55965.log.gz 2024-04-25T10:28:21.770 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78169.log 2024-04-25T10:28:21.770 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86784.log.gz 2024-04-25T10:28:21.771 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71445.log 2024-04-25T10:28:21.771 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75021.log.gz 2024-04-25T10:28:21.772 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81251.log 2024-04-25T10:28:21.772 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78169.log.gz 2024-04-25T10:28:21.773 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79233.log 2024-04-25T10:28:21.773 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71445.log.gz 2024-04-25T10:28:21.774 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91633.log 2024-04-25T10:28:21.774 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81251.log.gz 2024-04-25T10:28:21.774 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91040.log 2024-04-25T10:28:21.775 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79233.log.gz 2024-04-25T10:28:21.775 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74360.log 2024-04-25T10:28:21.775 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91633.log.gz 2024-04-25T10:28:21.776 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88983.log 2024-04-25T10:28:21.776 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91040.log.gz 2024-04-25T10:28:21.777 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89392.log 2024-04-25T10:28:21.777 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74360.log.gz 2024-04-25T10:28:21.778 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34192.log 2024-04-25T10:28:21.778 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88983.log.gz 2024-04-25T10:28:21.779 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37387.log 2024-04-25T10:28:21.779 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89392.log.gz 2024-04-25T10:28:21.780 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35283.log 2024-04-25T10:28:21.780 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34192.log.gz 2024-04-25T10:28:21.780 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54485.log 2024-04-25T10:28:21.781 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37387.log.gz 2024-04-25T10:28:21.781 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.2.35283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35283.log.gz 2024-04-25T10:28:21.781 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53641.log 2024-04-25T10:28:21.782 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91529.log 2024-04-25T10:28:21.782 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54485.log.gz 2024-04-25T10:28:21.783 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68597.log 2024-04-25T10:28:21.783 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53641.log.gz 2024-04-25T10:28:21.784 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91321.log 2024-04-25T10:28:21.784 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91529.log.gz 2024-04-25T10:28:21.785 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68995.log 2024-04-25T10:28:21.785 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68597.log.gz 2024-04-25T10:28:21.786 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75984.log 2024-04-25T10:28:21.786 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91321.log.gz 2024-04-25T10:28:21.787 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69439.log 2024-04-25T10:28:21.787 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68995.log.gz 2024-04-25T10:28:21.787 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.75984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75984.log.gz 2024-04-25T10:28:21.788 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83555.log 2024-04-25T10:28:21.788 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35672.log 2024-04-25T10:28:21.789 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69439.log.gz 2024-04-25T10:28:21.789 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2024-04-25T10:28:21.790 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35672.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.83555.log.gz 2024-04-25T10:28:21.790 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35672.log.gz 2024-04-25T10:28:21.791 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50200.log 2024-04-25T10:28:21.791 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37111.log 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86409.log 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50200.log.gz 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72359.log 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37111.log.gz 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91555.log 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86409.log.gz 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55412.log 2024-04-25T10:28:21.795 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72359.log.gz 2024-04-25T10:28:21.796 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36805.log 2024-04-25T10:28:21.796 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91555.log.gz 2024-04-25T10:28:21.797 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81765.log 2024-04-25T10:28:21.797 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55412.log.gz 2024-04-25T10:28:21.797 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77329.log 2024-04-25T10:28:21.798 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36805.log.gz 2024-04-25T10:28:21.798 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85310.log 2024-04-25T10:28:21.798 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81765.log.gz 2024-04-25T10:28:21.799 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87199.log 2024-04-25T10:28:21.799 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77329.log.gz 2024-04-25T10:28:21.800 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86179.log 2024-04-25T10:28:21.800 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87199.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.85310.log.gz 2024-04-25T10:28:21.801 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87199.log.gz 2024-04-25T10:28:21.801 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36020.log 2024-04-25T10:28:21.802 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34186.log 2024-04-25T10:28:21.802 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86179.log.gz 2024-04-25T10:28:21.803 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85990.log 2024-04-25T10:28:21.803 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36020.log.gz 2024-04-25T10:28:21.803 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51270.log 2024-04-25T10:28:21.804 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34186.log.gz 2024-04-25T10:28:21.804 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55023.log 2024-04-25T10:28:21.804 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85990.log.gz 2024-04-25T10:28:21.805 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35360.log 2024-04-25T10:28:21.805 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51270.log.gz 2024-04-25T10:28:21.806 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35516.log 2024-04-25T10:28:21.806 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55023.log.gz 2024-04-25T10:28:21.807 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72070.log 2024-04-25T10:28:21.807 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35360.log.gz 2024-04-25T10:28:21.808 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51713.log 2024-04-25T10:28:21.808 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35516.log.gz 2024-04-25T10:28:21.809 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89106.log 2024-04-25T10:28:21.809 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72070.log.gz 2024-04-25T10:28:21.809 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36552.log 2024-04-25T10:28:21.810 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51713.log.gz 2024-04-25T10:28:21.811 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37597.log 2024-04-25T10:28:21.811 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89106.log.gz 2024-04-25T10:28:21.811 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52633.log 2024-04-25T10:28:21.812 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36552.log.gz 2024-04-25T10:28:21.812 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72129.log 2024-04-25T10:28:21.813 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37597.log.gz 2024-04-25T10:28:21.813 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75274.log 2024-04-25T10:28:21.814 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52633.log.gz 2024-04-25T10:28:21.814 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75652.log 2024-04-25T10:28:21.814 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72129.log.gz 2024-04-25T10:28:21.815 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71360.log 2024-04-25T10:28:21.815 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75274.log.gz 2024-04-25T10:28:21.816 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35412.log 2024-04-25T10:28:21.816 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75652.log.gz 2024-04-25T10:28:21.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35942.log 2024-04-25T10:28:21.817 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71360.log.gz 2024-04-25T10:28:21.818 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35412.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35412.log.gz 2024-04-25T10:28:21.818 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84911.log 2024-04-25T10:28:21.819 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90775.log 2024-04-25T10:28:21.819 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35942.log.gz 2024-04-25T10:28:21.819 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81147.log 2024-04-25T10:28:21.820 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84911.log.gz 2024-04-25T10:28:21.820 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69360.log 2024-04-25T10:28:21.821 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81147.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.90775.log.gz 2024-04-25T10:28:21.821 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81147.log.gz 2024-04-25T10:28:21.822 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72161.log 2024-04-25T10:28:21.822 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33988.log 2024-04-25T10:28:21.823 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69360.log.gz 2024-04-25T10:28:21.823 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84426.log 2024-04-25T10:28:21.823 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72161.log.gz 2024-04-25T10:28:21.824 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77460.log 2024-04-25T10:28:21.824 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33988.log.gz 2024-04-25T10:28:21.825 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84742.log 2024-04-25T10:28:21.825 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.84426.log.gz 2024-04-25T10:28:21.826 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77460.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77460.log.gz 2024-04-25T10:28:21.826 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55081.log 2024-04-25T10:28:21.827 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75170.log 2024-04-25T10:28:21.827 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84742.log.gz 2024-04-25T10:28:21.828 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90884.log 2024-04-25T10:28:21.828 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55081.log.gz 2024-04-25T10:28:21.829 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75170.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:28:21.829 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75170.log.gz 2024-04-25T10:28:21.829 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81988.log 2024-04-25T10:28:21.830 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90884.log.gz 2024-04-25T10:28:21.830 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71765.log 2024-04-25T10:28:21.831 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81988.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81988.log.gz 2024-04-25T10:28:21.831 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88005.log 2024-04-25T10:28:21.832 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.71765.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.75732.log 2024-04-25T10:28:21.833 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88005.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71765.log.gz 2024-04-25T10:28:21.833 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37491.log 2024-04-25T10:28:21.833 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88005.log.gz 2024-04-25T10:28:21.834 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82198.log 2024-04-25T10:28:21.834 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75732.log.gz 2024-04-25T10:28:21.835 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53288.log 2024-04-25T10:28:21.836 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82198.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37491.log.gz 2024-04-25T10:28:21.836 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78111.log 2024-04-25T10:28:21.836 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82198.log.gz 2024-04-25T10:28:21.837 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86211.log 2024-04-25T10:28:21.837 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53288.log.gz 2024-04-25T10:28:21.838 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85464.log 2024-04-25T10:28:21.839 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78111.log.gz 2024-04-25T10:28:21.839 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52783.log 2024-04-25T10:28:21.840 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86211.log.gz 2024-04-25T10:28:21.840 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81173.log 2024-04-25T10:28:21.841 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85464.log.gz 2024-04-25T10:28:21.841 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33949.log 2024-04-25T10:28:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81173.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52783.log.gz 2024-04-25T10:28:21.843 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34395.log 2024-04-25T10:28:21.843 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81173.log.gz 2024-04-25T10:28:21.844 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77486.log 2024-04-25T10:28:21.844 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33949.log.gz 2024-04-25T10:28:21.845 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50637.log 2024-04-25T10:28:21.845 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34395.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:28:21.845 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77486.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34395.log.gz 2024-04-25T10:28:21.845 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74969.log 2024-04-25T10:28:21.846 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.77486.log.gz 2024-04-25T10:28:21.846 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88480.log 2024-04-25T10:28:21.847 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz 2024-04-25T10:28:21.847 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74969.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74969.log.gz -5 2024-04-25T10:28:21.847 INFO:teuthology.orchestra.run.smithi087.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80751.log 2024-04-25T10:28:21.848 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.88480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89035.log 2024-04-25T10:28:21.848 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.88480.log.gz 2024-04-25T10:28:21.849 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69203.log 2024-04-25T10:28:21.849 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80751.log.gz 2024-04-25T10:28:21.850 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53524.log 2024-04-25T10:28:21.850 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89035.log.gz 2024-04-25T10:28:21.851 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89614.log 2024-04-25T10:28:21.851 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69203.log.gz 2024-04-25T10:28:21.852 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68787.log 2024-04-25T10:28:21.852 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53524.log.gz 2024-04-25T10:28:21.853 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88905.log 2024-04-25T10:28:21.853 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89614.log.gz 2024-04-25T10:28:21.854 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86604.log 2024-04-25T10:28:21.854 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68787.log.gz 2024-04-25T10:28:21.855 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72387.log 2024-04-25T10:28:21.855 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88905.log.gz 2024-04-25T10:28:21.855 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90274.log 2024-04-25T10:28:21.856 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86604.log.gz 2024-04-25T10:28:21.857 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77638.log 2024-04-25T10:28:21.857 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72387.log.gz 2024-04-25T10:28:21.857 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86472.log 2024-04-25T10:28:21.858 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90274.log.gz 2024-04-25T10:28:21.858 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37413.log 2024-04-25T10:28:21.859 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.77638.log.gz 2024-04-25T10:28:21.860 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51970.log 2024-04-25T10:28:21.860 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86472.log.gz 2024-04-25T10:28:21.860 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37413.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37413.log.gz 2024-04-25T10:28:21.861 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80572.log 2024-04-25T10:28:21.861 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-04-25T10:28:21.862 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80572.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51970.log.gz 2024-04-25T10:28:21.862 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69842.log 2024-04-25T10:28:21.863 INFO:teuthology.orchestra.run.smithi087.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.80572.log.gz 2024-04-25T10:28:21.863 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35994.log 2024-04-25T10:28:21.864 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53234.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-04-25T10:28:21.864 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69842.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69842.log.gz 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50927.log 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84859.log 2024-04-25T10:28:21.866 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35994.log: /var/log/ceph/ceph-client.admin.50927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69047.log 2024-04-25T10:28:21.866 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35994.log.gz 2024-04-25T10:28:21.866 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50927.log.gz 2024-04-25T10:28:21.867 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52020.log 2024-04-25T10:28:21.867 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84859.log.gz 2024-04-25T10:28:21.868 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69047.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:28:21.868 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69047.log.gz 2024-04-25T10:28:21.869 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86734.log 2024-04-25T10:28:21.869 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mon.a.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52020.log.gz 2024-04-25T10:28:21.870 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88332.log 2024-04-25T10:28:21.871 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78786.log 2024-04-25T10:28:21.872 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86734.log: /var/log/ceph/ceph-client.admin.88332.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88332.log.gz 2024-04-25T10:28:21.872 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53550.log 2024-04-25T10:28:21.872 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.86734.log.gz 2024-04-25T10:28:21.872 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85789.log 2024-04-25T10:28:21.873 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78786.log.gz 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77920.log 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53550.log.gz 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90494.log 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85789.log.gz 2024-04-25T10:28:21.876 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84050.log 2024-04-25T10:28:21.876 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77920.log.gz 2024-04-25T10:28:21.877 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51561.log 2024-04-25T10:28:21.878 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84050.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.90494.log.gz 2024-04-25T10:28:21.878 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36210.log 2024-04-25T10:28:21.878 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84050.log.gz 2024-04-25T10:28:21.879 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89694.log 2024-04-25T10:28:21.880 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36210.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51561.log.gz 2024-04-25T10:28:21.880 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36210.log.gz 2024-04-25T10:28:21.880 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85117.log 2024-04-25T10:28:21.881 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68649.log 2024-04-25T10:28:21.882 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85117.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89694.log.gz 2024-04-25T10:28:21.882 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85117.log.gz 2024-04-25T10:28:21.882 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2024-04-25T10:28:21.883 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88931.log 2024-04-25T10:28:21.883 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68649.log.gz 2024-04-25T10:28:21.884 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68464.log 2024-04-25T10:28:21.884 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2024-04-25T10:28:21.885 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90546.log 2024-04-25T10:28:21.885 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88931.log.gz 2024-04-25T10:28:21.886 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89773.log 2024-04-25T10:28:21.887 INFO:teuthology.orchestra.run.smithi087.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.68464.log.gz 2024-04-25T10:28:21.887 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34001.log 2024-04-25T10:28:21.889 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89773.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.90546.log.gz 2024-04-25T10:28:21.889 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82042.log 2024-04-25T10:28:21.889 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89773.log.gz 2024-04-25T10:28:21.890 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54086.log 2024-04-25T10:28:21.890 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34001.log.gz 2024-04-25T10:28:21.890 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.82042.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.90699.log 2024-04-25T10:28:21.891 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54086.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82042.log.gz 2024-04-25T10:28:21.892 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86120.log 2024-04-25T10:28:21.892 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54086.log.gz 2024-04-25T10:28:21.893 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36526.log 2024-04-25T10:28:21.893 INFO:teuthology.orchestra.run.smithi087.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.90699.log.gz 2024-04-25T10:28:21.894 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78973.log 2024-04-25T10:28:21.894 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86120.log.gz 2024-04-25T10:28:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36526.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36526.log.gz 2024-04-25T10:28:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80807.log 2024-04-25T10:28:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5/var/log/ceph/ceph-client.admin.78973.log: --verbose -- /var/log/ceph/ceph-client.admin.37764.log 2024-04-25T10:28:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80807.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78973.log.gz 2024-04-25T10:28:21.897 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80807.log.gzgzip 2024-04-25T10:28:21.897 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72450.log 2024-04-25T10:28:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37764.log: -- /var/log/ceph/ceph-client.0.51497.log 2024-04-25T10:28:21.898 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37764.log.gz 2024-04-25T10:28:21.899 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85549.log 2024-04-25T10:28:21.899 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72450.log.gz 2024-04-25T10:28:21.900 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.51497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51497.log.gz 2024-04-25T10:28:21.900 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86146.log 2024-04-25T10:28:21.901 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.85549.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69021.log 2024-04-25T10:28:21.901 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86146.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85549.log.gz 2024-04-25T10:28:21.902 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90048.log 2024-04-25T10:28:21.902 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86146.log.gz 2024-04-25T10:28:21.903 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87465.log 2024-04-25T10:28:21.903 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69021.log.gz 2024-04-25T10:28:21.904 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5/var/log/ceph/ceph-client.admin.90048.log: --verbose -- /var/log/ceph/ceph-client.admin.79025.log 2024-04-25T10:28:21.904 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87465.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90048.log.gz 2024-04-25T10:28:21.905 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78656.log 2024-04-25T10:28:21.905 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87465.log.gz 2024-04-25T10:28:21.906 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88879.log 2024-04-25T10:28:21.906 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79025.log.gz 2024-04-25T10:28:21.907 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81063.log 2024-04-25T10:28:21.907 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78656.log: /var/log/ceph/ceph-client.admin.88879.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78656.log.gz 2024-04-25T10:28:21.908 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52246.log 2024-04-25T10:28:21.908 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88879.log.gz 2024-04-25T10:28:21.909 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36656.log 2024-04-25T10:28:21.909 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81063.log.gz 2024-04-25T10:28:21.910 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52246.log: /var/log/ceph/ceph-client.admin.90832.log 2024-04-25T10:28:21.910 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36656.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52246.log.gz 2024-04-25T10:28:21.911 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53576.log 2024-04-25T10:28:21.911 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36656.log.gz 2024-04-25T10:28:21.912 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68839.log 2024-04-25T10:28:21.912 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90832.log.gz 2024-04-25T10:28:21.913 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69766.log 2024-04-25T10:28:21.914 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68839.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53576.log.gz 2024-04-25T10:28:21.914 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84661.log 2024-04-25T10:28:21.915 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68839.log.gz 2024-04-25T10:28:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87666.log 2024-04-25T10:28:21.915 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69766.log.gz 2024-04-25T10:28:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56243.log 2024-04-25T10:28:21.917 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84661.log: /var/log/ceph/ceph-client.admin.87666.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84661.log.gz 2024-04-25T10:28:21.917 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91503.log 2024-04-25T10:28:21.918 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87666.log.gz 2024-04-25T10:28:21.918 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.56243.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56243.log.gz 2024-04-25T10:28:21.918 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77434.log 2024-04-25T10:28:21.919 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72633.log 2024-04-25T10:28:21.920 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91503.log.gz 2024-04-25T10:28:21.920 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55698.log 2024-04-25T10:28:21.921 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77434.log.gz 2024-04-25T10:28:21.921 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71079.log 2024-04-25T10:28:21.922 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72633.log.gz 2024-04-25T10:28:21.922 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87978.log 2024-04-25T10:28:21.923 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55698.log.gz 2024-04-25T10:28:21.923 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74386.log 2024-04-25T10:28:21.924 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71079.log.gz 2024-04-25T10:28:21.925 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84989.log 2024-04-25T10:28:21.925 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87978.log.gz 2024-04-25T10:28:21.926 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78403.log 2024-04-25T10:28:21.927 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84989.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74386.log.gz 2024-04-25T10:28:21.927 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79334.log 2024-04-25T10:28:21.927 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84989.log.gz 2024-04-25T10:28:21.928 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75073.log 2024-04-25T10:28:21.928 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78403.log.gz 2024-04-25T10:28:21.929 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.79334.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22752.log 2024-04-25T10:28:21.929 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79334.log.gz 2024-04-25T10:28:21.930 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-04-25T10:28:21.930 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75073.log.gz 2024-04-25T10:28:21.931 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.tmp-client.admin.22752.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22752.log.gz 2024-04-25T10:28:21.931 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88747.log 2024-04-25T10:28:21.932 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51898.log 2024-04-25T10:28:21.932 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-04-25T10:28:21.933 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78474.log 2024-04-25T10:28:21.933 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88747.log.gz 2024-04-25T10:28:21.934 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78526.log 2024-04-25T10:28:21.934 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51898.log.gz 2024-04-25T10:28:21.935 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37307.log 2024-04-25T10:28:21.935 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78474.log.gz 2024-04-25T10:28:21.936 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83971.log 2024-04-25T10:28:21.936 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78526.log.gz 2024-04-25T10:28:21.937 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50410.log 2024-04-25T10:28:21.937 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37307.log.gz 2024-04-25T10:28:21.938 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55777.log 2024-04-25T10:28:21.939 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83971.log.gz 2024-04-25T10:28:21.939 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50410.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50410.log.gz 2024-04-25T10:28:21.940 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75758.log 2024-04-25T10:28:21.940 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91737.log 2024-04-25T10:28:21.941 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55777.log.gz 2024-04-25T10:28:21.942 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74255.log 2024-04-25T10:28:21.942 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75758.log.gz 2024-04-25T10:28:21.943 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68571.log 2024-04-25T10:28:21.943 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91737.log.gz 2024-04-25T10:28:21.944 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51401.log 2024-04-25T10:28:21.944 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68571.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74255.log.gz 2024-04-25T10:28:21.945 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68571.log.gz 2024-04-25T10:28:21.945 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84770.log 2024-04-25T10:28:21.946 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68761.log 2024-04-25T10:28:21.947 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-04-25T10:28:21.947 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78351.log 2024-04-25T10:28:21.947 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84770.log.gz 2024-04-25T10:28:21.948 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77536.log 2024-04-25T10:28:21.948 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68761.log.gz 2024-04-25T10:28:21.949 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54687.log 2024-04-25T10:28:21.949 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78351.log.gz 2024-04-25T10:28:21.951 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83703.log 2024-04-25T10:28:21.951 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.77536.log.gz 2024-04-25T10:28:21.951 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35542.log 2024-04-25T10:28:21.952 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54687.log.gz 2024-04-25T10:28:21.953 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36604.log 2024-04-25T10:28:21.953 INFO:teuthology.orchestra.run.smithi087.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.83703.log.gz 2024-04-25T10:28:21.954 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87861.log 2024-04-25T10:28:21.954 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35542.log.gz 2024-04-25T10:28:21.955 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36954.log 2024-04-25T10:28:21.955 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36604.log.gz 2024-04-25T10:28:21.956 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85414.log 2024-04-25T10:28:21.956 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87861.log.gz 2024-04-25T10:28:21.957 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53314.log 2024-04-25T10:28:21.957 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36954.log.gz 2024-04-25T10:28:21.958 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53046.log 2024-04-25T10:28:21.959 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53314.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.85414.log.gz 2024-04-25T10:28:21.959 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91014.log 2024-04-25T10:28:21.959 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53314.log.gz 2024-04-25T10:28:21.960 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35163.log 2024-04-25T10:28:21.960 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53046.log.gz 2024-04-25T10:28:21.961 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36980.log 2024-04-25T10:28:21.961 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91014.log.gz 2024-04-25T10:28:21.962 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.35163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35163.log.gz 2024-04-25T10:28:21.962 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91092.log 2024-04-25T10:28:21.964 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56095.log 2024-04-25T10:28:21.964 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36980.log.gz 2024-04-25T10:28:21.965 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88148.log 2024-04-25T10:28:21.965 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91092.log.gz 2024-04-25T10:28:21.965 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86303.log 2024-04-25T10:28:21.967 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88148.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56095.log.gz 2024-04-25T10:28:21.968 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35698.log 2024-04-25T10:28:21.968 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88148.log.gz 2024-04-25T10:28:21.968 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51244.log 2024-04-25T10:28:21.968 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86303.log.gz 2024-04-25T10:28:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89471.log 2024-04-25T10:28:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35698.log: /var/log/ceph/ceph-client.admin.51244.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35698.log.gz 2024-04-25T10:28:21.971 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51244.log.gz 2024-04-25T10:28:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36500.log 2024-04-25T10:28:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84154.log 2024-04-25T10:28:21.973 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36500.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89471.log.gz 2024-04-25T10:28:21.973 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81556.log 2024-04-25T10:28:21.973 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36500.log.gz 2024-04-25T10:28:21.973 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83529.log 2024-04-25T10:28:21.974 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84154.log.gz 2024-04-25T10:28:21.976 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.81556.log: -5 --verbose -- /var/log/ceph/ceph-client.0.54837.log 2024-04-25T10:28:21.976 INFO:teuthology.orchestra.run.smithi087.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.81556.log.gz 2024-04-25T10:28:21.976 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87147.log 2024-04-25T10:28:21.976 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83529.log.gz 2024-04-25T10:28:21.976 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.54837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54837.log.gz 2024-04-25T10:28:21.977 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53130.log 2024-04-25T10:28:21.979 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87796.log 2024-04-25T10:28:21.979 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53130.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87147.log.gz 2024-04-25T10:28:21.979 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53130.log.gz 2024-04-25T10:28:21.980 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34599.log 2024-04-25T10:28:21.980 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75365.log 2024-04-25T10:28:21.980 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87796.log.gz 2024-04-25T10:28:21.982 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34989.log 2024-04-25T10:28:21.982 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34599.log.gz 2024-04-25T10:28:21.982 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55885.log 2024-04-25T10:28:21.983 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75365.log.gz 2024-04-25T10:28:21.983 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84609.log 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34989.log: /var/log/ceph/ceph-client.admin.55885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53785.log 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34989.log.gz 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55885.log.gz 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75888.log 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84609.log.gz 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89553.log 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75888.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53785.log.gz 2024-04-25T10:28:21.987 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33999.log 2024-04-25T10:28:21.988 INFO:teuthology.orchestra.run.smithi087.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.75888.log.gz 2024-04-25T10:28:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72685.log 2024-04-25T10:28:21.988 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89553.log.gz 2024-04-25T10:28:21.988 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74753.log 2024-04-25T10:28:21.990 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33999.log.gz 2024-04-25T10:28:21.990 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91189.log 2024-04-25T10:28:21.990 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72685.log.gz 2024-04-25T10:28:21.990 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91815.log 2024-04-25T10:28:21.990 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74753.log.gz 2024-04-25T10:28:21.991 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78500.log 2024-04-25T10:28:21.991 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91189.log.gz 2024-04-25T10:28:21.992 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90022.log 2024-04-25T10:28:21.992 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91815.log.gz 2024-04-25T10:28:21.993 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37085.log 2024-04-25T10:28:21.993 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78500.log.gz 2024-04-25T10:28:21.994 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77177.log 2024-04-25T10:28:21.994 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90022.log.gz 2024-04-25T10:28:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90160.log 2024-04-25T10:28:22.002 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37085.log.gz 2024-04-25T10:28:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51375.log 2024-04-25T10:28:22.002 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77177.log: /var/log/ceph/ceph-client.admin.90160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84512.log 2024-04-25T10:28:22.002 INFO:teuthology.orchestra.run.smithi087.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.77177.log.gz 2024-04-25T10:28:22.002 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90160.log.gz 2024-04-25T10:28:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37033.log 2024-04-25T10:28:22.003 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-04-25T10:28:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68310.log 2024-04-25T10:28:22.003 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84512.log.gz 2024-04-25T10:28:22.003 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71229.log 2024-04-25T10:28:22.003 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37033.log.gz 2024-04-25T10:28:22.004 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71334.log 2024-04-25T10:28:22.004 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.68310.log.gz 2024-04-25T10:28:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54715.log 2024-04-25T10:28:22.008 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71229.log.gz 2024-04-25T10:28:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82224.log 2024-04-25T10:28:22.008 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71334.log.gz 2024-04-25T10:28:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34807.log 2024-04-25T10:28:22.008 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54715.log.gz 2024-04-25T10:28:22.008 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89262.log 2024-04-25T10:28:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34807.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.82224.log.gz 2024-04-25T10:28:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86874.log 2024-04-25T10:28:22.009 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34807.log.gz 2024-04-25T10:28:22.009 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35856.log 2024-04-25T10:28:22.009 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89262.log.gz 2024-04-25T10:28:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89720.log 2024-04-25T10:28:22.010 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86874.log.gz 2024-04-25T10:28:22.010 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85815.log 2024-04-25T10:28:22.010 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35856.log.gz 2024-04-25T10:28:22.011 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55185.log 2024-04-25T10:28:22.012 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89720.log.gz 2024-04-25T10:28:22.012 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89996.log 2024-04-25T10:28:22.013 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85815.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85815.log.gz 2024-04-25T10:28:22.013 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84885.log 2024-04-25T10:28:22.013 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55185.log.gz 2024-04-25T10:28:22.016 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81037.log 2024-04-25T10:28:22.016 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89996.log.gz 2024-04-25T10:28:22.016 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.84885.log: -- /var/log/ceph/ceph-client.admin.54483.log 2024-04-25T10:28:22.016 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81037.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84885.log.gz 2024-04-25T10:28:22.016 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84374.log 2024-04-25T10:28:22.017 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81037.log.gz 2024-04-25T10:28:22.018 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54313.log 2024-04-25T10:28:22.019 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54483.log.gz 2024-04-25T10:28:22.019 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84374.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:28:22.019 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54313.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84374.log.gz 2024-04-25T10:28:22.021 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83860.log 2024-04-25T10:28:22.021 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54313.log.gz 2024-04-25T10:28:22.021 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-04-25T10:28:22.021 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36474.log 2024-04-25T10:28:22.022 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83860.log.gz 2024-04-25T10:28:22.022 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52402.log 2024-04-25T10:28:22.023 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-04-25T10:28:22.023 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69610.log 2024-04-25T10:28:22.024 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36474.log.gz 2024-04-25T10:28:22.024 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86381.log 2024-04-25T10:28:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69610.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52402.log.gz 2024-04-25T10:28:22.026 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2024-04-25T10:28:22.026 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69610.log.gz 2024-04-25T10:28:22.027 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34937.log 2024-04-25T10:28:22.027 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86381.log.gz 2024-04-25T10:28:22.028 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-04-25T10:28:22.028 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34937.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2024-04-25T10:28:22.030 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34003.log 2024-04-25T10:28:22.030 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34937.log.gz 2024-04-25T10:28:22.030 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55315.log 2024-04-25T10:28:22.031 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-04-25T10:28:22.032 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33594.log 2024-04-25T10:28:22.032 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34003.log.gz 2024-04-25T10:28:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88306.log 2024-04-25T10:28:22.033 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55315.log.gz 2024-04-25T10:28:22.033 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90312.log 2024-04-25T10:28:22.034 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33594.log.gz 2024-04-25T10:28:22.035 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36129.log 2024-04-25T10:28:22.035 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88306.log.gz 2024-04-25T10:28:22.036 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86830.log 2024-04-25T10:28:22.036 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90312.log.gz 2024-04-25T10:28:22.037 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85364.log 2024-04-25T10:28:22.037 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36129.log.gz 2024-04-25T10:28:22.038 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.86830.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.86830.log.gz 2024-04-25T10:28:22.038 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74229.log 2024-04-25T10:28:22.039 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55341.log 2024-04-25T10:28:22.040 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74229.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.85364.log.gz 2024-04-25T10:28:22.041 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89418.log 2024-04-25T10:28:22.041 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74229.log.gz 2024-04-25T10:28:22.041 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89825.log 2024-04-25T10:28:22.042 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55341.log.gz 2024-04-25T10:28:22.042 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53262.log 2024-04-25T10:28:22.043 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89418.log.gz 2024-04-25T10:28:22.043 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50585.log 2024-04-25T10:28:22.044 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89825.log.gz 2024-04-25T10:28:22.045 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69151.log 2024-04-25T10:28:22.045 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53262.log.gz 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34729.log 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50585.log.gz 2024-04-25T10:28:22.047 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36708.log 2024-04-25T10:28:22.047 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69151.log.gz 2024-04-25T10:28:22.048 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91399.log 2024-04-25T10:28:22.048 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34729.log.gz 2024-04-25T10:28:22.049 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51451.log 2024-04-25T10:28:22.049 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36708.log.gz 2024-04-25T10:28:22.050 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35308.log 2024-04-25T10:28:22.050 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91399.log.gz 2024-04-25T10:28:22.051 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51870.log 2024-04-25T10:28:22.051 INFO:teuthology.orchestra.run.smithi087.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51451.log.gz 2024-04-25T10:28:22.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55633.log 2024-04-25T10:28:22.053 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35308.log.gz 2024-04-25T10:28:22.053 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34194.log 2024-04-25T10:28:22.054 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51870.log.gz 2024-04-25T10:28:22.055 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33271.log 2024-04-25T10:28:22.055 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55633.log.gz 2024-04-25T10:28:22.056 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78085.log 2024-04-25T10:28:22.056 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34194.log.gz 2024-04-25T10:28:22.057 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87614.log 2024-04-25T10:28:22.057 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33271.log.gz 2024-04-25T10:28:22.058 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50848.log 2024-04-25T10:28:22.058 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78085.log.gz 2024-04-25T10:28:22.059 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84963.log 2024-04-25T10:28:22.059 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87614.log.gz 2024-04-25T10:28:22.060 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88280.log 2024-04-25T10:28:22.060 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50848.log.gz 2024-04-25T10:28:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75938.log 2024-04-25T10:28:22.061 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84963.log.gz 2024-04-25T10:28:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69740.log 2024-04-25T10:28:22.062 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88280.log.gz 2024-04-25T10:28:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53394.log 2024-04-25T10:28:22.064 INFO:teuthology.orchestra.run.smithi087.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.75938.log.gz 2024-04-25T10:28:22.065 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82172.log 2024-04-25T10:28:22.065 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69740.log.gz 2024-04-25T10:28:22.066 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79129.log 2024-04-25T10:28:22.066 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53394.log.gz 2024-04-25T10:28:22.067 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-04-25T10:28:22.067 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82172.log.gz 2024-04-25T10:28:22.068 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90646.log 2024-04-25T10:28:22.068 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79129.log.gz 2024-04-25T10:28:22.069 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52302.log 2024-04-25T10:28:22.070 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-04-25T10:28:22.070 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35490.log 2024-04-25T10:28:22.071 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.90646.log.gz 2024-04-25T10:28:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54138.log 2024-04-25T10:28:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35490.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52302.log.gz 2024-04-25T10:28:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89210.log 2024-04-25T10:28:22.073 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35490.log.gz 2024-04-25T10:28:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55859.log 2024-04-25T10:28:22.074 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54138.log.gz 2024-04-25T10:28:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89210.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89210.log.gz 2024-04-25T10:28:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88384.log 2024-04-25T10:28:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55859.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55859.log.gz 2024-04-25T10:28:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35464.log 2024-04-25T10:28:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89525.log 2024-04-25T10:28:22.078 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.88384.log.gz 2024-04-25T10:28:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69714.log 2024-04-25T10:28:22.079 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35464.log.gz 2024-04-25T10:28:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34211.log 2024-04-25T10:28:22.080 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89525.log.gz 2024-04-25T10:28:22.081 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84805.log 2024-04-25T10:28:22.081 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69714.log.gz 2024-04-25T10:28:22.082 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36682.log 2024-04-25T10:28:22.083 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34211.log.gz 2024-04-25T10:28:22.083 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33372.log 2024-04-25T10:28:22.084 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84805.log.gz 2024-04-25T10:28:22.085 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36155.log 2024-04-25T10:28:22.085 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36682.log.gz 2024-04-25T10:28:22.086 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68410.log 2024-04-25T10:28:22.086 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33372.log.gz 2024-04-25T10:28:22.087 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86498.log 2024-04-25T10:28:22.087 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36155.log.gz 2024-04-25T10:28:22.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68969.log 2024-04-25T10:28:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86498.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.68410.log.gz 2024-04-25T10:28:22.089 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74787.log 2024-04-25T10:28:22.090 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86498.log.gz 2024-04-25T10:28:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51841.log 2024-04-25T10:28:22.090 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68969.log.gz 2024-04-25T10:28:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77381.log 2024-04-25T10:28:22.092 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74787.log.gz 2024-04-25T10:28:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84937.log 2024-04-25T10:28:22.093 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51841.log.gz 2024-04-25T10:28:22.093 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77738.log 2024-04-25T10:28:22.094 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77381.log.gz 2024-04-25T10:28:22.095 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88228.log 2024-04-25T10:28:22.095 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84937.log.gz 2024-04-25T10:28:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52733.log 2024-04-25T10:28:22.096 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77738.log.gz 2024-04-25T10:28:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53902.log 2024-04-25T10:28:22.097 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88228.log.gz 2024-04-25T10:28:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69237.log 2024-04-25T10:28:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53902.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52733.log.gz 2024-04-25T10:28:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71129.log 2024-04-25T10:28:22.099 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53902.log.gz 2024-04-25T10:28:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72529.log 2024-04-25T10:28:22.101 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69237.log.gz 2024-04-25T10:28:22.101 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68865.log 2024-04-25T10:28:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72529.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.71129.log.gz 2024-04-25T10:28:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54164.log 2024-04-25T10:28:22.103 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72529.log.gz 2024-04-25T10:28:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33462.log 2024-04-25T10:28:22.104 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68865.log.gz 2024-04-25T10:28:22.105 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34677.log 2024-04-25T10:28:22.105 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54164.log.gz 2024-04-25T10:28:22.106 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68675.log 2024-04-25T10:28:22.106 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33462.log.gz 2024-04-25T10:28:22.107 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87439.log 2024-04-25T10:28:22.107 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34677.log.gz 2024-04-25T10:28:22.108 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71282.log 2024-04-25T10:28:22.108 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68675.log.gz 2024-04-25T10:28:22.109 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81381.log 2024-04-25T10:28:22.109 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87439.log.gz 2024-04-25T10:28:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35386.log 2024-04-25T10:28:22.110 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71282.log.gz 2024-04-25T10:28:22.111 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75509.log 2024-04-25T10:28:22.112 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81381.log.gz 2024-04-25T10:28:22.113 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85497.log 2024-04-25T10:28:22.113 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35386.log.gz 2024-04-25T10:28:22.113 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75509.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:28:22.114 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75509.log.gz 2024-04-25T10:28:22.115 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81199.log 2024-04-25T10:28:22.115 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85497.log.gz 2024-04-25T10:28:22.116 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88358.log 2024-04-25T10:28:22.117 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81871.log 2024-04-25T10:28:22.117 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81199.log.gz 2024-04-25T10:28:22.119 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88358.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88358.log.gz 2024-04-25T10:28:22.119 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91118.log 2024-04-25T10:28:22.120 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81871.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81871.log.gz 2024-04-25T10:28:22.120 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85841.log 2024-04-25T10:28:22.122 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75591.log 2024-04-25T10:28:22.122 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91118.log.gz 2024-04-25T10:28:22.123 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85841.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85841.log.gz 2024-04-25T10:28:22.124 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75591.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75591.log.gz 2024-04-25T10:28:22.124 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37545.log 2024-04-25T10:28:22.126 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36314.log 2024-04-25T10:28:22.127 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77407.log 2024-04-25T10:28:22.127 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37545.log.gz 2024-04-25T10:28:22.128 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36314.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36314.log.gz 2024-04-25T10:28:22.129 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77407.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77407.log.gz 2024-04-25T10:28:22.130 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84833.log 2024-04-25T10:28:22.131 INFO:teuthology.orchestra.run.smithi169.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:28:22.131 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33793.log 2024-04-25T10:28:22.132 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82016.log 2024-04-25T10:28:22.132 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84833.log.gz 2024-04-25T10:28:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33793.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33793.log.gz 2024-04-25T10:28:22.134 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82016.log.gz 2024-04-25T10:28:22.135 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83912.log 2024-04-25T10:28:22.136 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77948.log 2024-04-25T10:28:22.137 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72422.log 2024-04-25T10:28:22.137 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83912.log.gz 2024-04-25T10:28:22.138 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77948.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77948.log.gz 2024-04-25T10:28:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72422.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72422.log.gz 2024-04-25T10:28:22.140 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69269.log 2024-04-25T10:28:22.141 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36448.log 2024-04-25T10:28:22.143 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77868.log 2024-04-25T10:28:22.143 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69269.log.gz 2024-04-25T10:28:22.144 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36448.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36448.log.gz 2024-04-25T10:28:22.145 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77868.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77868.log.gz 2024-04-25T10:28:22.145 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51296.log 2024-04-25T10:28:22.146 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53182.log 2024-04-25T10:28:22.148 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84452.log 2024-04-25T10:28:22.148 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51296.log.gz 2024-04-25T10:28:22.149 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53182.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53182.log.gz 2024-04-25T10:28:22.150 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84452.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84452.log.gz 2024-04-25T10:28:22.151 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89184.log 2024-04-25T10:28:22.152 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74334.log 2024-04-25T10:28:22.153 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91893.log 2024-04-25T10:28:22.154 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89184.log.gz 2024-04-25T10:28:22.154 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74334.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74334.log.gz 2024-04-25T10:28:22.155 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91893.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91893.log.gz 2024-04-25T10:28:22.156 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77355.log 2024-04-25T10:28:22.157 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52276.log 2024-04-25T10:28:22.158 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54484.log 2024-04-25T10:28:22.158 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77355.log.gz 2024-04-25T10:28:22.160 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52276.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52276.log.gz 2024-04-25T10:28:22.161 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54484.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54484.log.gz 2024-04-25T10:28:22.161 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87491.log 2024-04-25T10:28:22.162 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33436.log 2024-04-25T10:28:22.164 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53446.log 2024-04-25T10:28:22.164 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87491.log.gz 2024-04-25T10:28:22.165 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33436.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33436.log.gz 2024-04-25T10:28:22.166 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53446.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53446.log.gz 2024-04-25T10:28:22.166 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91451.log 2024-04-25T10:28:22.167 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69386.log 2024-04-25T10:28:22.169 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55803.log 2024-04-25T10:28:22.169 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91451.log.gz 2024-04-25T10:28:22.170 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69386.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69386.log.gz 2024-04-25T10:28:22.171 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55803.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55803.log.gz 2024-04-25T10:28:22.171 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91373.log 2024-04-25T10:28:22.172 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50488.log 2024-04-25T10:28:22.174 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33490.log 2024-04-25T10:28:22.174 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91373.log.gz 2024-04-25T10:28:22.175 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50488.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50488.log.gz 2024-04-25T10:28:22.176 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33490.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33490.log.gz 2024-04-25T10:28:22.176 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54112.log 2024-04-25T10:28:22.178 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34410.log 2024-04-25T10:28:22.179 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55438.log 2024-04-25T10:28:22.179 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54112.log.gz 2024-04-25T10:28:22.180 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34410.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34410.log.gz 2024-04-25T10:28:22.181 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55438.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55438.log.gz 2024-04-25T10:28:22.182 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88485.log 2024-04-25T10:28:22.183 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85015.log 2024-04-25T10:28:22.184 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.88485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81817.log 2024-04-25T10:28:22.184 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.88485.log.gz 2024-04-25T10:28:22.186 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85015.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85015.log.gz 2024-04-25T10:28:22.187 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81817.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.81817.log.gz 2024-04-25T10:28:22.187 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36928.log 2024-04-25T10:28:22.188 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84322.log 2024-04-25T10:28:22.190 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78919.log 2024-04-25T10:28:22.190 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36928.log.gz 2024-04-25T10:28:22.191 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84322.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84322.log.gz 2024-04-25T10:28:22.192 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78919.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78919.log.gz 2024-04-25T10:28:22.192 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81927.log 2024-04-25T10:28:22.194 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71256.log 2024-04-25T10:28:22.195 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71659.log 2024-04-25T10:28:22.195 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81927.log.gz 2024-04-25T10:28:22.196 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71256.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71256.log.gz 2024-04-25T10:28:22.197 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71659.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71659.log.gz 2024-04-25T10:28:22.198 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88773.log 2024-04-25T10:28:22.199 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81277.log 2024-04-25T10:28:22.200 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72581.log 2024-04-25T10:28:22.200 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88773.log.gz 2024-04-25T10:28:22.201 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81277.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81277.log.gz 2024-04-25T10:28:22.205 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72581.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72581.log.gz 2024-04-25T10:28:22.205 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84128.log 2024-04-25T10:28:22.206 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79329.log 2024-04-25T10:28:22.206 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90596.log 2024-04-25T10:28:22.206 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84128.log.gz 2024-04-25T10:28:22.206 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.79329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79329.log.gz 2024-04-25T10:28:22.208 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90596.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.90596.log.gz 2024-04-25T10:28:22.209 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91841.log 2024-04-25T10:28:22.210 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51349.log 2024-04-25T10:28:22.212 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51535.log 2024-04-25T10:28:22.212 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91841.log.gz 2024-04-25T10:28:22.212 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51349.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51349.log.gz 2024-04-25T10:28:22.214 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51535.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51535.log.gz 2024-04-25T10:28:22.214 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81011.log 2024-04-25T10:28:22.215 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89746.log 2024-04-25T10:28:22.217 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86578.log 2024-04-25T10:28:22.217 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81011.log.gz 2024-04-25T10:28:22.218 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89746.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89746.log.gz 2024-04-25T10:28:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86578.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86578.log.gz 2024-04-25T10:28:22.219 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77688.log 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi169.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91267.log 2024-04-25T10:28:22.222 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.77688.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.83605.log 2024-04-25T10:28:22.223 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91267.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.77688.log.gz 2024-04-25T10:28:22.223 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91267.log.gz 2024-04-25T10:28:22.225 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83605.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.83605.log.gz 2024-04-25T10:28:22.225 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54971.log 2024-04-25T10:28:22.226 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71633.log 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72226.log 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54971.log.gz 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71633.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71633.log.gz 2024-04-25T10:28:22.230 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72226.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72226.log.gz 2024-04-25T10:28:22.231 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75989.log 2024-04-25T10:28:22.232 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54265.log 2024-04-25T10:28:22.233 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.75989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75989.log.gz 2024-04-25T10:28:22.233 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34755.log 2024-04-25T10:28:22.234 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.54265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54265.log.gz 2024-04-25T10:28:22.235 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34755.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34755.log.gz 2024-04-25T10:28:22.236 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91241.log 2024-04-25T10:28:22.237 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69495.log 2024-04-25T10:28:22.238 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91163.log 2024-04-25T10:28:22.239 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91241.log.gz 2024-04-25T10:28:22.239 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69495.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69495.log.gz 2024-04-25T10:28:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91163.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91163.log.gz 2024-04-25T10:28:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51687.log 2024-04-25T10:28:22.243 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74486.log 2024-04-25T10:28:22.244 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80672.log 2024-04-25T10:28:22.244 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51687.log.gz 2024-04-25T10:28:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74486.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74486.log.gz 2024-04-25T10:28:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80672.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80672.log.gz 2024-04-25T10:28:22.247 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74891.log 2024-04-25T10:28:22.248 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77764.log 2024-04-25T10:28:22.249 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87173.log 2024-04-25T10:28:22.249 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74891.log.gz 2024-04-25T10:28:22.250 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77764.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77764.log.gz 2024-04-25T10:28:22.252 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87173.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87173.log.gz 2024-04-25T10:28:22.252 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71687.log 2024-04-25T10:28:22.253 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34419.log 2024-04-25T10:28:22.254 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51166.log 2024-04-25T10:28:22.255 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71687.log.gz 2024-04-25T10:28:22.256 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34419.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34419.log.gz 2024-04-25T10:28:22.256 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51166.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51166.log.gz 2024-04-25T10:28:22.257 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82068.log 2024-04-25T10:28:22.258 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51763.log 2024-04-25T10:28:22.260 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68735.log 2024-04-25T10:28:22.260 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82068.log.gz 2024-04-25T10:28:22.260 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51763.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51763.log.gz 2024-04-25T10:28:22.262 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68735.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68735.log.gz 2024-04-25T10:28:22.262 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86901.log 2024-04-25T10:28:22.263 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88853.log 2024-04-25T10:28:22.265 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78221.log 2024-04-25T10:28:22.265 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86901.log.gz 2024-04-25T10:28:22.266 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88853.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88853.log.gz 2024-04-25T10:28:22.267 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78221.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78221.log.gz 2024-04-25T10:28:22.267 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34963.log 2024-04-25T10:28:22.268 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90130.log 2024-04-25T10:28:22.270 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34963.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34963.log.gz 2024-04-25T10:28:22.271 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90130.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90130.log.gz 2024-04-25T10:28:22.271 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87543.log 2024-04-25T10:28:22.272 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71029.log 2024-04-25T10:28:22.274 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75563.log 2024-04-25T10:28:22.274 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87543.log.gz 2024-04-25T10:28:22.275 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71029.log: /var/log/ceph/ceph-client.admin.75563.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.71029.log.gz 2024-04-25T10:28:22.276 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75563.log.gz 2024-04-25T10:28:22.277 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54260.log 2024-04-25T10:28:22.278 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75222.log 2024-04-25T10:28:22.279 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.54260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89952.log 2024-04-25T10:28:22.279 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54260.log.gz 2024-04-25T10:28:22.280 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75222.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75222.log.gz 2024-04-25T10:28:22.281 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86976.log 2024-04-25T10:28:22.281 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.89952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89952.log.gz 2024-04-25T10:28:22.282 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82146.log 2024-04-25T10:28:22.283 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81329.log 2024-04-25T10:28:22.284 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1%/var/log/ceph/ceph-client.admin.82146.log: -- replaced with /var/log/ceph/ceph-client.admin.86976.log.gz 2024-04-25T10:28:22.284 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82146.log.gz 2024-04-25T10:28:22.285 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81329.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81329.log.gz 2024-04-25T10:28:22.286 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79207.log 2024-04-25T10:28:22.287 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-04-25T10:28:22.288 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90223.log 2024-04-25T10:28:22.289 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79207.log.gz 2024-04-25T10:28:22.290 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50783.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz 2024-04-25T10:28:22.291 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90223.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.90223.log.gz 2024-04-25T10:28:22.291 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33992.log 2024-04-25T10:28:22.293 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71791.log 2024-04-25T10:28:22.294 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84714.log 2024-04-25T10:28:22.294 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33992.log.gz 2024-04-25T10:28:22.295 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71791.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71791.log.gz 2024-04-25T10:28:22.296 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84714.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84714.log.gz 2024-04-25T10:28:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87952.log 2024-04-25T10:28:22.298 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53104.log 2024-04-25T10:28:22.299 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71308.log 2024-04-25T10:28:22.299 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87952.log.gz 2024-04-25T10:28:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53104.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53104.log.gz 2024-04-25T10:28:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71308.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71308.log.gz 2024-04-25T10:28:22.302 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:28:22.303 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74177.log 2024-04-25T10:28:22.307 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82094.log 2024-04-25T10:28:22.307 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74177.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74177.log.gz 2024-04-25T10:28:22.307 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82094.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82094.log.gz 2024-04-25T10:28:22.307 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84024.log 2024-04-25T10:28:22.308 INFO:teuthology.orchestra.run.smithi087.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:28:22.308 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69467.log 2024-04-25T10:28:22.309 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84024.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84024.log.gz 2024-04-25T10:28:22.310 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69467.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69467.log.gz 2024-04-25T10:28:22.311 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71739.log 2024-04-25T10:28:22.312 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53982.log 2024-04-25T10:28:22.313 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83492.log 2024-04-25T10:28:22.313 INFO:teuthology.orchestra.run.smithi087.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71739.log.gz 2024-04-25T10:28:22.315 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53982.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53982.log.gz 2024-04-25T10:28:22.316 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83492.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83492.log.gz 2024-04-25T10:28:22.316 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88687.log 2024-04-25T10:28:22.317 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74675.log 2024-04-25T10:28:22.318 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71386.log 2024-04-25T10:28:22.319 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88687.log.gz 2024-04-25T10:28:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74675.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74675.log.gz 2024-04-25T10:28:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71386.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71386.log.gz 2024-04-25T10:28:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85763.log 2024-04-25T10:28:22.322 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35049.log 2024-04-25T10:28:22.324 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81791.log 2024-04-25T10:28:22.324 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85763.log.gz 2024-04-25T10:28:22.325 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35049.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35049.log.gz 2024-04-25T10:28:22.326 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81791.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81791.log.gz 2024-04-25T10:28:22.326 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89444.log 2024-04-25T10:28:22.328 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69073.log 2024-04-25T10:28:22.329 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84102.log 2024-04-25T10:28:22.329 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89444.log.gz 2024-04-25T10:28:22.330 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69073.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69073.log.gz 2024-04-25T10:28:22.331 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84102.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84102.log.gz 2024-04-25T10:28:22.332 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80861.log 2024-04-25T10:28:22.333 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83654.log 2024-04-25T10:28:22.334 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87569.log 2024-04-25T10:28:22.334 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80861.log.gz 2024-04-25T10:28:22.336 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83654.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.83654.log.gz 2024-04-25T10:28:22.336 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87569.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87569.log.gz 2024-04-25T10:28:22.337 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50982.log 2024-04-25T10:28:22.338 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71914.log 2024-04-25T10:28:22.340 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75118.log 2024-04-25T10:28:22.340 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50982.log.gz 2024-04-25T10:28:22.341 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71914.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71914.log.gz 2024-04-25T10:28:22.342 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75118.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75118.log.gz 2024-04-25T10:28:22.342 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52833.log 2024-04-25T10:28:22.343 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80952.log 2024-04-25T10:28:22.345 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36831.log 2024-04-25T10:28:22.345 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52833.log.gz 2024-04-25T10:28:22.346 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80952.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80952.log.gz 2024-04-25T10:28:22.347 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36831.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36831.log.gz 2024-04-25T10:28:22.348 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71869.log 2024-04-25T10:28:22.349 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27604.log 2024-04-25T10:28:22.350 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55263.log 2024-04-25T10:28:22.351 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71869.log.gz 2024-04-25T10:28:22.351 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.27604.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27604.log.gz 2024-04-25T10:28:22.352 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55263.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55263.log.gz 2024-04-25T10:28:22.353 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78059.log 2024-04-25T10:28:22.354 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74077.log 2024-04-25T10:28:22.355 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81478.log 2024-04-25T10:28:22.355 INFO:teuthology.orchestra.run.smithi087.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78059.log.gz 2024-04-25T10:28:22.357 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74077.log: /var/log/ceph/ceph-client.admin.81478.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74077.log.gz 2024-04-25T10:28:22.357 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81478.log.gz 2024-04-25T10:28:22.358 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72607.log 2024-04-25T10:28:22.359 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53156.log 2024-04-25T10:28:22.360 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55911.log 2024-04-25T10:28:22.361 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72607.log.gz 2024-04-25T10:28:22.361 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53156.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53156.log.gz 2024-04-25T10:28:22.363 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55911.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55911.log.gz 2024-04-25T10:28:22.363 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33624.log 2024-04-25T10:28:22.365 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74839.log 2024-04-25T10:28:22.366 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52683.log 2024-04-25T10:28:22.366 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33624.log.gz 2024-04-25T10:28:22.367 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74839.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74839.log.gz 2024-04-25T10:28:22.369 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52683.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52683.log.gz 2024-04-25T10:28:22.369 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71843.log 2024-04-25T10:28:22.370 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78891.log 2024-04-25T10:28:22.372 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82320.log 2024-04-25T10:28:22.372 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71843.log.gz 2024-04-25T10:28:22.373 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78891.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78891.log.gz 2024-04-25T10:28:22.373 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.82320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82320.log.gz 2024-04-25T10:28:22.374 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89799.log 2024-04-25T10:28:22.375 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71471.log 2024-04-25T10:28:22.376 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51637.log 2024-04-25T10:28:22.377 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89799.log.gz 2024-04-25T10:28:22.377 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71471.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71471.log.gz 2024-04-25T10:28:22.380 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51637.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51637.log.gz 2024-04-25T10:28:22.380 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54661.log 2024-04-25T10:28:22.381 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71179.log 2024-04-25T10:28:22.382 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37439.log 2024-04-25T10:28:22.382 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54661.log.gz 2024-04-25T10:28:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71179.log: /var/log/ceph/ceph-client.admin.37439.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71179.log.gz 2024-04-25T10:28:22.385 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37439.log.gz 2024-04-25T10:28:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86437.log 2024-04-25T10:28:22.386 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36734.log 2024-04-25T10:28:22.388 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77127.log 2024-04-25T10:28:22.388 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86437.log.gz 2024-04-25T10:28:22.388 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36734.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36734.log.gz 2024-04-25T10:28:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77127.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.77127.log.gz 2024-04-25T10:28:22.391 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78143.log 2024-04-25T10:28:22.392 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33767.log 2024-04-25T10:28:22.394 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78143.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78143.log.gz 2024-04-25T10:28:22.394 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33767.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33767.log.gz 2024-04-25T10:28:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79051.log 2024-04-25T10:28:22.396 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50462.log 2024-04-25T10:28:22.398 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54391.log 2024-04-25T10:28:22.398 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79051.log.gz 2024-04-25T10:28:22.399 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50462.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50462.log.gz 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54391.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54391.log.gz 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69688.log 2024-04-25T10:28:22.401 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79077.log 2024-04-25T10:28:22.403 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53732.log 2024-04-25T10:28:22.403 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69688.log.gz 2024-04-25T10:28:22.404 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79077.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79077.log.gz 2024-04-25T10:28:22.405 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53732.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53732.log.gz 2024-04-25T10:28:22.405 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87033.log 2024-04-25T10:28:22.406 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80622.log 2024-04-25T10:28:22.409 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81843.log 2024-04-25T10:28:22.410 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80622.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.87033.log.gz 2024-04-25T10:28:22.410 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81843.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.80622.log.gz 2024-04-25T10:28:22.410 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81843.log.gz 2024-04-25T10:28:22.410 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50280.log 2024-04-25T10:28:22.411 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78325.log 2024-04-25T10:28:22.413 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86656.log 2024-04-25T10:28:22.413 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50280.log.gz 2024-04-25T10:28:22.414 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78325.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78325.log.gz 2024-04-25T10:28:22.415 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86656.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86656.log.gz 2024-04-25T10:28:22.415 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34408.log 2024-04-25T10:28:22.416 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68623.log 2024-04-25T10:28:22.418 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35438.log 2024-04-25T10:28:22.419 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34408.log.gz 2024-04-25T10:28:22.419 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68623.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68623.log.gz 2024-04-25T10:28:22.420 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35438.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35438.log.gz 2024-04-25T10:28:22.421 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37745.log 2024-04-25T10:28:22.422 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72096.log 2024-04-25T10:28:22.423 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.37745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37745.log.gz 2024-04-25T10:28:22.423 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51611.log 2024-04-25T10:28:22.424 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72096.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72096.log.gz 2024-04-25T10:28:22.425 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51611.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51611.log.gz 2024-04-25T10:28:22.426 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83998.log 2024-04-25T10:28:22.427 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78578.log 2024-04-25T10:28:22.428 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87114.log 2024-04-25T10:28:22.428 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83998.log.gz 2024-04-25T10:28:22.429 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78578.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78578.log.gz 2024-04-25T10:28:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87114.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.87114.log.gz 2024-04-25T10:28:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86068.log 2024-04-25T10:28:22.433 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69636.log 2024-04-25T10:28:22.434 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89236.log 2024-04-25T10:28:22.434 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86068.log.gz 2024-04-25T10:28:22.435 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69636.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69636.log.gz 2024-04-25T10:28:22.436 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89236.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89236.log.gz 2024-04-25T10:28:22.436 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81355.log 2024-04-25T10:28:22.437 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68514.log 2024-04-25T10:28:22.439 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72252.log 2024-04-25T10:28:22.439 INFO:teuthology.orchestra.run.smithi087.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81355.log.gz 2024-04-25T10:28:22.441 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68514.log: /var/log/ceph/ceph-client.admin.72252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78689.log 2024-04-25T10:28:22.441 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68514.log.gz 2024-04-25T10:28:22.442 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72252.log.gz 2024-04-25T10:28:22.442 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-04-25T10:28:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86042.log 2024-04-25T10:28:22.444 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78689.log.gz 2024-04-25T10:28:22.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50142.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz 2024-04-25T10:28:22.445 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86042.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86042.log.gz 2024-04-25T10:28:22.446 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54421.log 2024-04-25T10:28:22.447 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84478.log 2024-04-25T10:28:22.449 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71497.log 2024-04-25T10:28:22.449 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54421.log.gz 2024-04-25T10:28:22.449 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84478.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84478.log.gz 2024-04-25T10:28:22.451 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71497.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71497.log.gz 2024-04-25T10:28:22.451 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84635.log 2024-04-25T10:28:22.452 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69582.log 2024-04-25T10:28:22.454 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84635.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84635.log.gz 2024-04-25T10:28:22.455 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69582.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69582.log.gz 2024-04-25T10:28:22.455 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71581.log 2024-04-25T10:28:22.456 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54365.log 2024-04-25T10:28:22.458 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50306.log 2024-04-25T10:28:22.458 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71581.log.gz 2024-04-25T10:28:22.459 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54365.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54365.log.gz 2024-04-25T10:28:22.460 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50306.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50306.log.gz 2024-04-25T10:28:22.460 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87744.log 2024-04-25T10:28:22.461 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72018.log 2024-04-25T10:28:22.463 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72659.log 2024-04-25T10:28:22.463 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87744.log.gz 2024-04-25T10:28:22.464 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72018.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72018.log.gz 2024-04-25T10:28:22.465 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72659.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72659.log.gz 2024-04-25T10:28:22.465 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81452.log 2024-04-25T10:28:22.467 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50007.log 2024-04-25T10:28:22.468 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35594.log 2024-04-25T10:28:22.468 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81452.log.gz 2024-04-25T10:28:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50007.log: /var/log/ceph/ceph-client.admin.35594.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50007.log.gz 2024-04-25T10:28:22.471 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35594.log.gz 2024-04-25T10:28:22.471 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80835.log 2024-04-25T10:28:22.472 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74813.log 2024-04-25T10:28:22.474 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88202.log 2024-04-25T10:28:22.474 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80835.log.gz 2024-04-25T10:28:22.474 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74813.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74813.log.gz 2024-04-25T10:28:22.475 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88202.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88202.log.gz 2024-04-25T10:28:22.476 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37281.log 2024-04-25T10:28:22.477 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75810.log 2024-04-25T10:28:22.479 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86835.log 2024-04-25T10:28:22.479 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37281.log.gz 2024-04-25T10:28:22.479 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75810.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75810.log.gz 2024-04-25T10:28:22.481 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.86835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78429.log 2024-04-25T10:28:22.481 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.86835.log.gz 2024-04-25T10:28:22.482 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50358.log 2024-04-25T10:28:22.483 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87718.log 2024-04-25T10:28:22.483 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78429.log.gz 2024-04-25T10:28:22.485 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50358.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50358.log.gz 2024-04-25T10:28:22.486 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87718.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87718.log.gz 2024-04-25T10:28:22.486 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50332.log 2024-04-25T10:28:22.487 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53498.log 2024-04-25T10:28:22.488 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83808.log 2024-04-25T10:28:22.489 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50332.log.gz 2024-04-25T10:28:22.490 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53498.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53498.log.gz 2024-04-25T10:28:22.491 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83808.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83808.log.gz 2024-04-25T10:28:22.491 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86277.log 2024-04-25T10:28:22.493 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51036.log 2024-04-25T10:28:22.494 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54060.log 2024-04-25T10:28:22.494 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86277.log.gz 2024-04-25T10:28:22.495 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51036.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51036.log.gz 2024-04-25T10:28:22.495 INFO:teuthology.orchestra.run.smithi087.stderr: /var/log/ceph/ceph-client.admin.78812.log 2024-04-25T10:28:22.497 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54060.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90858.log 2024-04-25T10:28:22.497 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78812.log: -- replaced with /var/log/ceph/ceph-client.admin.54060.log.gz 2024-04-25T10:28:22.498 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78812.log.gz 2024-04-25T10:28:22.498 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56017.log 2024-04-25T10:28:22.499 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83945.log 2024-04-25T10:28:22.500 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90858.log.gz 2024-04-25T10:28:22.502 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56017.log: /var/log/ceph/ceph-client.admin.83945.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56017.log.gz 2024-04-25T10:28:22.502 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83945.log.gz 2024-04-25T10:28:22.502 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80985.log 2024-04-25T10:28:22.503 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51140.log 2024-04-25T10:28:22.504 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56069.log 2024-04-25T10:28:22.505 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80985.log.gz 2024-04-25T10:28:22.506 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51140.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51140.log.gz 2024-04-25T10:28:22.507 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56069.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56069.log.gz 2024-04-25T10:28:22.507 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84544.log 2024-04-25T10:28:22.509 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78630.log 2024-04-25T10:28:22.510 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89295.log 2024-04-25T10:28:22.510 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84544.log.gz 2024-04-25T10:28:22.512 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78630.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78630.log.gz 2024-04-25T10:28:22.512 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89295.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.89295.log.gz 2024-04-25T10:28:22.513 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34418.log 2024-04-25T10:28:22.514 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69792.log 2024-04-25T10:28:22.516 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50168.log 2024-04-25T10:28:22.516 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34418.log.gz 2024-04-25T10:28:22.517 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69792.log: /var/log/ceph/ceph-client.admin.50168.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.69792.log.gz 2024-04-25T10:28:22.517 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50168.log.gz 2024-04-25T10:28:22.518 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37361.log 2024-04-25T10:28:22.519 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72839.log 2024-04-25T10:28:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74203.log 2024-04-25T10:28:22.521 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37361.log.gz 2024-04-25T10:28:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72839.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72839.log.gz 2024-04-25T10:28:22.523 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74203.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74203.log.gz 2024-04-25T10:28:22.523 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75248.log 2024-04-25T10:28:22.524 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35750.log 2024-04-25T10:28:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75784.log 2024-04-25T10:28:22.526 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75248.log.gz 2024-04-25T10:28:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35750.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35750.log.gz 2024-04-25T10:28:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75784.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75784.log.gz 2024-04-25T10:28:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88174.log 2024-04-25T10:28:22.530 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75144.log 2024-04-25T10:28:22.532 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90910.log 2024-04-25T10:28:22.532 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88174.log.gz 2024-04-25T10:28:22.532 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75144.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75144.log.gz 2024-04-25T10:28:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90910.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90910.log.gz 2024-04-25T10:28:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82120.log 2024-04-25T10:28:22.535 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34210.log 2024-04-25T10:28:22.537 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81582.log 2024-04-25T10:28:22.537 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82120.log.gz 2024-04-25T10:28:22.537 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34210.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34210.log.gz 2024-04-25T10:28:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81582.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81582.log.gz 2024-04-25T10:28:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75333.log 2024-04-25T10:28:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91347.log 2024-04-25T10:28:22.542 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75333.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:28:22.542 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75333.log.gz 2024-04-25T10:28:22.542 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91347.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91347.log.gz 2024-04-25T10:28:22.545 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-04-25T10:28:22.546 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50751.log 2024-04-25T10:28:22.548 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91919.log 2024-04-25T10:28:22.548 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50751.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-04-25T10:28:22.548 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:28:22.548 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50751.log.gz 2024-04-25T10:28:22.550 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80781.log 2024-04-25T10:28:22.550 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91919.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:28:22.550 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91919.log.gz 2024-04-25T10:28:22.551 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80781.log: /var/log/ceph/ceph-mds.c.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80781.log.gz 2024-04-25T10:28:22.552 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55159.log 2024-04-25T10:28:22.553 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86355.log 2024-04-25T10:28:22.555 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55159.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55159.log.gz 2024-04-25T10:28:22.556 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86355.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86355.log.gz 2024-04-25T10:28:22.556 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36760.log 2024-04-25T10:28:22.557 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78552.log 2024-04-25T10:28:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36760.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36760.log.gz 2024-04-25T10:28:22.560 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78552.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78552.log.gz 2024-04-25T10:28:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74564.log 2024-04-25T10:28:22.562 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85170.log 2024-04-25T10:28:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74564.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74564.log.gz 2024-04-25T10:28:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.85170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85170.log.gz 2024-04-25T10:28:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88087.log 2024-04-25T10:28:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78604.log 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88087.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88087.log.gz 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78604.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78604.log.gz 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34411.log 2024-04-25T10:28:22.569 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:28:22.571 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34411.log.gz 2024-04-25T10:28:22.573 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56197.log 2024-04-25T10:28:22.576 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90988.log 2024-04-25T10:28:22.576 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51088.log 2024-04-25T10:28:22.576 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56197.log.gz 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89947.log 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90988.log.gz 2024-04-25T10:28:22.584 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54997.log 2024-04-25T10:28:22.584 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51088.log.gz 2024-04-25T10:28:22.584 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.89947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89947.log.gz 2024-04-25T10:28:22.585 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54997.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54997.log.gz 2024-04-25T10:28:22.585 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56121.log 2024-04-25T10:28:22.585 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84348.log 2024-04-25T10:28:22.587 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78999.log 2024-04-25T10:28:22.588 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56121.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56121.log.gz 2024-04-25T10:28:22.590 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84348.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80926.log 2024-04-25T10:28:22.590 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78999.log: -- replaced with /var/log/ceph/ceph-client.admin.84348.log.gz 2024-04-25T10:28:22.593 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72711.log 2024-04-25T10:28:22.594 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78999.log.gz 2024-04-25T10:28:22.595 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72711.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80926.log.gz 2024-04-25T10:28:22.596 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87692.log 2024-04-25T10:28:22.597 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72711.log.gz 2024-04-25T10:28:22.599 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80522.log 2024-04-25T10:28:22.601 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87692.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87692.log.gz 2024-04-25T10:28:22.602 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91581.log 2024-04-25T10:28:22.604 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80522.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.80522.log.gz 2024-04-25T10:28:22.608 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37699.log 2024-04-25T10:28:22.608 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91581.log.gz 2024-04-25T10:28:22.610 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37190.log 2024-04-25T10:28:22.615 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37190.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37190.log.gz 2024-04-25T10:28:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37699.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37699.log.gz 2024-04-25T10:28:22.616 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74995.log 2024-04-25T10:28:22.617 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74995.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74995.log.gz 2024-04-25T10:28:22.617 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51323.log 2024-04-25T10:28:22.620 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51323.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51323.log.gz 2024-04-25T10:28:22.620 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84076.log 2024-04-25T10:28:22.623 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84076.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84076.log.gz 2024-04-25T10:28:22.623 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77790.log 2024-04-25T10:28:22.626 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72555.log 2024-04-25T10:28:22.626 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77790.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77790.log.gz 2024-04-25T10:28:22.629 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72555.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72555.log.gz 2024-04-25T10:28:22.629 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36181.log 2024-04-25T10:28:22.632 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36181.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36181.log.gz 2024-04-25T10:28:22.632 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56043.log 2024-04-25T10:28:22.635 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56043.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56043.log.gz 2024-04-25T10:28:22.635 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85575.log 2024-04-25T10:28:22.637 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77842.log 2024-04-25T10:28:22.641 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77842.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77842.log.gz 2024-04-25T10:28:22.641 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85575.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85575.log.gz 2024-04-25T10:28:22.642 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81303.log 2024-04-25T10:28:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81303.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81303.log.gz 2024-04-25T10:28:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87309.log 2024-04-25T10:28:22.647 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87309.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87309.log.gz 2024-04-25T10:28:22.647 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89668.log 2024-04-25T10:28:22.650 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89668.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89668.log.gz 2024-04-25T10:28:22.650 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55991.log 2024-04-25T10:28:22.653 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72305.log 2024-04-25T10:28:22.653 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55991.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55991.log.gz 2024-04-25T10:28:22.655 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72305.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72305.log.gz 2024-04-25T10:28:22.655 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74436.log 2024-04-25T10:28:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74436.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.74436.log.gz 2024-04-25T10:28:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85041.log 2024-04-25T10:28:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85041.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85041.log.gz 2024-04-25T10:28:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90749.log 2024-04-25T10:28:22.665 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90749.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90749.log.gz 2024-04-25T10:28:22.665 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35568.log 2024-04-25T10:28:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74512.log 2024-04-25T10:28:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35568.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35568.log.gz 2024-04-25T10:28:22.671 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74512.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74512.log.gz 2024-04-25T10:28:22.671 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88661.log 2024-04-25T10:28:22.673 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52533.log 2024-04-25T10:28:22.673 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88661.log.gz 2024-04-25T10:28:22.674 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86329.log 2024-04-25T10:28:22.676 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52533.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52533.log.gz 2024-04-25T10:28:22.678 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86329.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86329.log.gz 2024-04-25T10:28:22.679 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89901.log 2024-04-25T10:28:22.682 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89901.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89901.log.gz 2024-04-25T10:28:22.682 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33996.log 2024-04-25T10:28:22.685 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33996.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33996.log.gz 2024-04-25T10:28:22.685 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77277.log 2024-04-25T10:28:22.688 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77277.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77277.log.gz 2024-04-25T10:28:22.688 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74865.log 2024-04-25T10:28:22.691 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86682.log 2024-04-25T10:28:22.691 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74865.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74865.log.gz 2024-04-25T10:28:22.693 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88254.log 2024-04-25T10:28:22.694 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86682.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86682.log.gz 2024-04-25T10:28:22.697 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71607.log 2024-04-25T10:28:22.697 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88254.log.gz 2024-04-25T10:28:22.698 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72331.log 2024-04-25T10:28:22.700 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71607.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71607.log.gz 2024-04-25T10:28:22.702 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72331.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72331.log.gz 2024-04-25T10:28:22.703 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91607.log 2024-04-25T10:28:22.706 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91607.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91607.log.gz 2024-04-25T10:28:22.706 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37623.log 2024-04-25T10:28:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37623.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37623.log.gz 2024-04-25T10:28:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54034.log 2024-04-25T10:28:22.712 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54034.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54034.log.gz 2024-04-25T10:28:22.712 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54741.log 2024-04-25T10:28:22.716 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88502.log 2024-04-25T10:28:22.716 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54741.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54741.log.gz 2024-04-25T10:28:22.719 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88502.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.88502.log.gz 2024-04-25T10:28:22.719 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52608.log 2024-04-25T10:28:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52608.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.52608.log.gz 2024-04-25T10:28:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53020.log 2024-04-25T10:28:22.727 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53020.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53020.log.gz 2024-04-25T10:28:22.728 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82325.log 2024-04-25T10:28:22.728 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.82325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82325.log.gz 2024-04-25T10:28:22.728 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86630.log 2024-04-25T10:28:22.733 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86630.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86630.log.gz 2024-04-25T10:28:22.733 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84232.log 2024-04-25T10:28:22.733 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.84232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54842.log 2024-04-25T10:28:22.734 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84232.log.gz 2024-04-25T10:28:22.735 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85601.log 2024-04-25T10:28:22.737 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.54842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54842.log.gz 2024-04-25T10:28:22.738 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55516.log 2024-04-25T10:28:22.740 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85601.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85601.log.gz 2024-04-25T10:28:22.742 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86524.log 2024-04-25T10:28:22.743 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55516.log.gz 2024-04-25T10:28:22.745 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36578.log 2024-04-25T10:28:22.746 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86524.log.gz 2024-04-25T10:28:22.749 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.36578.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36578.log.gz 2024-04-25T10:28:22.750 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87829.log 2024-04-25T10:28:22.756 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87829.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87829.log.gz 2024-04-25T10:28:22.756 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77227.log 2024-04-25T10:28:22.756 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77227.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.77227.log.gz 2024-04-25T10:28:22.756 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37519.log 2024-04-25T10:28:22.759 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37519.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37519.log.gz 2024-04-25T10:28:22.759 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2024-04-25T10:28:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72503.log 2024-04-25T10:28:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51010.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2024-04-25T10:28:22.765 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72503.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72503.log.gz 2024-04-25T10:28:22.765 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71992.log 2024-04-25T10:28:22.768 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71992.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71992.log.gz 2024-04-25T10:28:22.768 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78947.log 2024-04-25T10:28:22.771 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78947.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78947.log.gz 2024-04-25T10:28:22.771 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37059.log 2024-04-25T10:28:22.774 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78299.log 2024-04-25T10:28:22.774 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37059.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37059.log.gz 2024-04-25T10:28:22.775 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69125.log 2024-04-25T10:28:22.775 INFO:teuthology.orchestra.run.smithi087.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78299.log.gz 2024-04-25T10:28:22.777 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81608.log 2024-04-25T10:28:22.778 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69125.log.gz 2024-04-25T10:28:22.779 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77303.log 2024-04-25T10:28:22.781 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81608.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53340.log 2024-04-25T10:28:22.781 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81608.log.gz 2024-04-25T10:28:22.783 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77303.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77303.log.gz 2024-04-25T10:28:22.786 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53340.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53340.log.gz 2024-04-25T10:28:22.786 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75626.log 2024-04-25T10:28:22.789 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75626.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75626.log.gz 2024-04-25T10:28:22.789 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2024-04-25T10:28:22.792 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35724.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2024-04-25T10:28:22.792 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33568.log 2024-04-25T10:28:22.794 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33568.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33568.log.gz 2024-04-25T10:28:22.795 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54287.log 2024-04-25T10:28:22.798 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54287.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54287.log.gz 2024-04-25T10:28:22.798 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54919.log 2024-04-25T10:28:22.801 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54919.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54919.log.gz 2024-04-25T10:28:22.801 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53758.log 2024-04-25T10:28:22.804 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53758.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53758.log.gz 2024-04-25T10:28:22.804 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71713.log 2024-04-25T10:28:22.806 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71713.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71713.log.gz 2024-04-25T10:28:22.806 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72278.log 2024-04-25T10:28:22.809 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52942.log 2024-04-25T10:28:22.809 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72278.log.gz 2024-04-25T10:28:22.810 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36288.log 2024-04-25T10:28:22.812 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52942.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52942.log.gz 2024-04-25T10:28:22.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36288.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36288.log.gz 2024-04-25T10:28:22.817 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33516.log 2024-04-25T10:28:22.818 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33516.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33516.log.gz 2024-04-25T10:28:22.818 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53867.log 2024-04-25T10:28:22.821 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53867.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53867.log.gz 2024-04-25T10:28:22.822 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52561.log 2024-04-25T10:28:22.825 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52561.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52561.log.gz 2024-04-25T10:28:22.825 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74590.log 2024-04-25T10:28:22.828 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75706.log 2024-04-25T10:28:22.828 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74590.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74590.log.gz 2024-04-25T10:28:22.831 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75706.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75706.log.gz 2024-04-25T10:28:22.831 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88031.log 2024-04-25T10:28:22.834 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88031.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88031.log.gz 2024-04-25T10:28:22.834 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55568.log 2024-04-25T10:28:22.837 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55568.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55568.log.gz 2024-04-25T10:28:22.837 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90248.log 2024-04-25T10:28:22.840 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90248.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90248.log.gz 2024-04-25T10:28:22.840 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52968.log 2024-04-25T10:28:22.842 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52968.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52968.log.gz 2024-04-25T10:28:22.843 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51192.log 2024-04-25T10:28:22.846 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51192.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51192.log.gz 2024-04-25T10:28:22.846 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72737.log 2024-04-25T10:28:22.849 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72737.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72737.log.gz 2024-04-25T10:28:22.849 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74616.log 2024-04-25T10:28:22.852 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74616.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74616.log.gz 2024-04-25T10:28:22.852 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87387.log 2024-04-25T10:28:22.853 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50226.log 2024-04-25T10:28:22.854 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87387.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87387.log.gz 2024-04-25T10:28:22.857 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52352.log 2024-04-25T10:28:22.857 INFO:teuthology.orchestra.run.smithi087.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50226.log.gz 2024-04-25T10:28:22.860 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69556.log 2024-04-25T10:28:22.860 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52352.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52352.log.gz 2024-04-25T10:28:22.861 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36236.log 2024-04-25T10:28:22.863 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69556.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69556.log.gz 2024-04-25T10:28:22.865 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87517.log 2024-04-25T10:28:22.865 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36236.log.gz 2024-04-25T10:28:22.866 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88122.log 2024-04-25T10:28:22.868 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87517.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87517.log.gz 2024-04-25T10:28:22.871 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88122.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88122.log.gz 2024-04-25T10:28:22.871 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33713.log 2024-04-25T10:28:22.874 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33713.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33713.log.gz 2024-04-25T10:28:22.874 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85659.log 2024-04-25T10:28:22.877 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85659.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85659.log.gz 2024-04-25T10:28:22.877 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33871.log 2024-04-25T10:28:22.880 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33871.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33871.log.gz 2024-04-25T10:28:22.880 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33542.log 2024-04-25T10:28:22.884 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91659.log 2024-04-25T10:28:22.884 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33542.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33542.log.gz 2024-04-25T10:28:22.885 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54448.log 2024-04-25T10:28:22.885 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91659.log.gz 2024-04-25T10:28:22.888 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54448.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54448.log.gz 2024-04-25T10:28:22.888 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86708.log 2024-04-25T10:28:22.891 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51502.log 2024-04-25T10:28:22.892 INFO:teuthology.orchestra.run.smithi087.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86708.log.gz 2024-04-25T10:28:22.893 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88552.log 2024-04-25T10:28:22.895 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.51502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51502.log.gz 2024-04-25T10:28:22.897 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.88552.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.71817.log 2024-04-25T10:28:22.899 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.88552.log.gz 2024-04-25T10:28:22.901 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35882.log 2024-04-25T10:28:22.905 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35882.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35882.log.gz 2024-04-25T10:28:22.906 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71817.log.gz 2024-04-25T10:28:22.906 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79155.log 2024-04-25T10:28:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79155.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79155.log.gz 2024-04-25T10:28:22.909 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68943.log 2024-04-25T10:28:22.911 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68943.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68943.log.gz 2024-04-25T10:28:22.912 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91477.log 2024-04-25T10:28:22.914 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91477.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91477.log.gz 2024-04-25T10:28:22.914 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75456.log 2024-04-25T10:28:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78195.log 2024-04-25T10:28:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75456.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75456.log.gz 2024-04-25T10:28:22.920 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78195.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78195.log.gz 2024-04-25T10:28:22.920 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74027.log 2024-04-25T10:28:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74027.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74027.log.gz 2024-04-25T10:28:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74917.log 2024-04-25T10:28:22.926 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74917.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74917.log.gz 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68709.log 2024-04-25T10:28:22.929 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68709.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68709.log.gz 2024-04-25T10:28:22.930 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51218.log 2024-04-25T10:28:22.933 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51218.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51218.log.gz 2024-04-25T10:28:22.933 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34394.log 2024-04-25T10:28:22.936 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34394.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34394.log.gz 2024-04-25T10:28:22.936 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88609.log 2024-04-25T10:28:22.940 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88609.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88609.log.gz 2024-04-25T10:28:22.941 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-04-25T10:28:22.941 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50559.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-04-25T10:28:22.942 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91066.log 2024-04-25T10:28:22.944 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2024-04-25T10:28:22.946 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55831.log 2024-04-25T10:28:22.946 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50953.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91066.log.gz 2024-04-25T10:28:22.948 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5/var/log/ceph/ceph-client.admin.55831.log: --verbose -- /var/log/ceph/ceph-client.admin.91869.log 2024-04-25T10:28:22.949 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55831.log.gz 2024-04-25T10:28:22.950 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87361.log 2024-04-25T10:28:22.953 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91869.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91869.log.gz 2024-04-25T10:28:22.953 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87361.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87361.log.gz 2024-04-25T10:28:22.953 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53078.log 2024-04-25T10:28:22.954 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2024-04-25T10:28:22.956 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53078.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53078.log.gz 2024-04-25T10:28:22.957 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85067.log 2024-04-25T10:28:22.961 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56147.log 2024-04-25T10:28:22.961 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85067.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.85067.log.gz 2024-04-25T10:28:22.961 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68813.log 2024-04-25T10:28:22.963 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56147.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56147.log.gz 2024-04-25T10:28:22.964 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75047.log 2024-04-25T10:28:22.965 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.68813.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68813.log.gz 2024-04-25T10:28:22.966 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.75047.log: /var/log/ceph/ceph-client.admin.78838.log 2024-04-25T10:28:22.972 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75047.log.gz 2024-04-25T10:28:22.972 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69177.log 2024-04-25T10:28:22.972 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33819.log 2024-04-25T10:28:22.972 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78838.log.gz 2024-04-25T10:28:22.972 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69177.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69177.log.gz 2024-04-25T10:28:22.973 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77974.log 2024-04-25T10:28:22.975 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33819.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33819.log.gz 2024-04-25T10:28:22.976 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33739.log 2024-04-25T10:28:22.981 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77974.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77974.log.gz 2024-04-25T10:28:22.981 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33739.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85685.log 2024-04-25T10:28:22.981 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33739.log.gz 2024-04-25T10:28:22.983 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89132.log 2024-04-25T10:28:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85685.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85685.log.gz 2024-04-25T10:28:22.986 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75196.log 2024-04-25T10:28:22.988 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89132.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89132.log.gz 2024-04-25T10:28:22.990 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33845.log 2024-04-25T10:28:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33845.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33845.log.gz 2024-04-25T10:28:22.994 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34833.log 2024-04-25T10:28:22.994 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75196.log.gz 2024-04-25T10:28:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34833.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34833.log.gz 2024-04-25T10:28:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34195.log 2024-04-25T10:28:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34195.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34195.log.gz 2024-04-25T10:28:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34781.log 2024-04-25T10:28:23.003 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34781.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34781.log.gz 2024-04-25T10:28:23.003 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35288.log 2024-04-25T10:28:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.2.35288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35288.log.gz 2024-04-25T10:28:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75836.log 2024-04-25T10:28:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85737.log 2024-04-25T10:28:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75836.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75836.log.gz 2024-04-25T10:28:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85737.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.85737.log.gz 2024-04-25T10:28:23.012 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55133.log 2024-04-25T10:28:23.014 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55133.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55133.log.gz 2024-04-25T10:28:23.015 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85711.log 2024-04-25T10:28:23.017 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85711.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.85711.log.gz 2024-04-25T10:28:23.018 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35968.log 2024-04-25T10:28:23.021 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52859.log 2024-04-25T10:28:23.021 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35968.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35968.log.gz 2024-04-25T10:28:23.024 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52859.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52859.log.gz 2024-04-25T10:28:23.024 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77612.log 2024-04-25T10:28:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77612.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77612.log.gz 2024-04-25T10:28:23.027 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50874.log 2024-04-25T10:28:23.029 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49855.log 2024-04-25T10:28:23.030 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50874.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50874.log.gz 2024-04-25T10:28:23.031 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79374.log 2024-04-25T10:28:23.034 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49855.log: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37750.log 2024-04-25T10:28:23.034 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49855.log.gz/var/log/ceph/ceph-client.admin.79374.log: 2024-04-25T10:28:23.035 INFO:teuthology.orchestra.run.smithi087.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79374.log.gz 2024-04-25T10:28:23.036 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79103.log 2024-04-25T10:28:23.037 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.37750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37750.log.gz 2024-04-25T10:28:23.040 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.79103.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79103.log.gz 2024-04-25T10:28:23.040 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87413.log 2024-04-25T10:28:23.043 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87413.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87413.log.gz 2024-04-25T10:28:23.043 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74649.log 2024-04-25T10:28:23.046 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74649.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74649.log.gz 2024-04-25T10:28:23.046 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37164.log 2024-04-25T10:28:23.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37164.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37164.log.gz 2024-04-25T10:28:23.052 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69530.log 2024-04-25T10:28:23.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69530.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69530.log.gz 2024-04-25T10:28:23.052 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82274.log 2024-04-25T10:28:23.055 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.82274.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82274.log.gz 2024-04-25T10:28:23.055 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78033.log 2024-04-25T10:28:23.059 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89588.log 2024-04-25T10:28:23.059 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78033.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78033.log.gz/var/log/ceph/ceph-client.admin.89588.log: 2024-04-25T10:28:23.059 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81089.log 89.6% -- replaced with /var/log/ceph/ceph-client.admin.89588.log.gz 2024-04-25T10:28:23.059 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-04-25T10:28:23.062 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81089.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81089.log.gz 2024-04-25T10:28:23.062 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88635.log 2024-04-25T10:28:23.065 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88635.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88635.log.gz 2024-04-25T10:28:23.065 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71940.log 2024-04-25T10:28:23.068 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71940.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71940.log.gz 2024-04-25T10:28:23.068 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80698.log 2024-04-25T10:28:23.073 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75300.log 2024-04-25T10:28:23.073 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80698.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80698.log.gz 2024-04-25T10:28:23.073 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.75300.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87225.log 2024-04-25T10:28:23.073 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75300.log.gz 2024-04-25T10:28:23.074 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56248.log 2024-04-25T10:28:23.076 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37007.log 2024-04-25T10:28:23.078 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.56248.log: gzip -5 --verbose 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87225.log.gz 2024-04-25T10:28:23.078 INFO:teuthology.orchestra.run.smithi087.stderr: -- /var/log/ceph/ceph-client.admin.89158.log 2024-04-25T10:28:23.078 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0%/var/log/ceph/ceph-client.admin.37007.log: -- replaced with /var/log/ceph/ceph-client.0.56248.log.gz 2024-04-25T10:28:23.079 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37007.log.gz 2024-04-25T10:28:23.080 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91789.log 2024-04-25T10:28:23.083 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91789.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91789.log.gz 2024-04-25T10:28:23.084 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89158.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89158.log.gz 2024-04-25T10:28:23.085 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72476.log 2024-04-25T10:28:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72476.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72476.log.gz 2024-04-25T10:28:23.088 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78007.log 2024-04-25T10:28:23.092 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.78007.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78007.log.gz 2024-04-25T10:28:23.093 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36370.log 2024-04-25T10:28:23.094 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36370.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36370.log.gz 2024-04-25T10:28:23.094 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52070.log 2024-04-25T10:28:23.096 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83466.log 2024-04-25T10:28:23.098 INFO:teuthology.orchestra.run.smithi087.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52070.log.gz 2024-04-25T10:28:23.099 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68360.log 2024-04-25T10:28:23.102 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.83466.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83466.log.gz 2024-04-25T10:28:23.103 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.68360.log: -- /var/log/ceph/ceph-client.admin.34651.log 2024-04-25T10:28:23.104 INFO:teuthology.orchestra.run.smithi087.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68360.log.gz 2024-04-25T10:28:23.106 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89080.log 2024-04-25T10:28:23.107 INFO:teuthology.orchestra.run.smithi087.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34651.log.gz 2024-04-25T10:28:23.109 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51114.log 2024-04-25T10:28:23.114 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51114.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51114.log.gz 2024-04-25T10:28:23.114 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89080.log.gz 2024-04-25T10:28:23.114 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53811.log 2024-04-25T10:28:23.115 INFO:teuthology.orchestra.run.smithi087.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:28:23.116 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53811.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53811.log.gz 2024-04-25T10:28:23.116 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90394.log 2024-04-25T10:28:23.118 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69662.log 2024-04-25T10:28:23.120 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90394.log: /var/log/ceph/ceph-client.admin.69662.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.90394.log.gz 2024-04-25T10:28:23.120 INFO:teuthology.orchestra.run.smithi087.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69662.log.gz 2024-04-25T10:28:23.121 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85964.log 2024-04-25T10:28:23.122 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35804.log 2024-04-25T10:28:23.123 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.85964.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85964.log.gz 2024-04-25T10:28:23.124 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35804.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35804.log.gz 2024-04-25T10:28:23.125 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90338.log 2024-04-25T10:28:23.126 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34209.log 2024-04-25T10:28:23.128 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90338.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90338.log.gz 2024-04-25T10:28:23.128 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.34209.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34209.log.gz 2024-04-25T10:28:23.129 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90444.log 2024-04-25T10:28:23.130 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53472.log 2024-04-25T10:28:23.132 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90444.log: /var/log/ceph/ceph-client.admin.53472.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.90444.log.gz 2024-04-25T10:28:23.133 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91763.log 2024-04-25T10:28:23.133 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53472.log.gz 2024-04-25T10:28:23.134 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81407.log 2024-04-25T10:28:23.135 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91763.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91763.log.gz 2024-04-25T10:28:23.136 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81407.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81407.log.gz 2024-04-25T10:28:23.137 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86094.log 2024-04-25T10:28:23.138 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81530.log 2024-04-25T10:28:23.140 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86094.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86094.log.gz 2024-04-25T10:28:23.141 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81530.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81530.log.gz 2024-04-25T10:28:23.141 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88801.log 2024-04-25T10:28:23.142 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33897.log 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.88801.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88801.log.gz 2024-04-25T10:28:23.144 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33897.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33897.log.gz 2024-04-25T10:28:23.145 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81504.log 2024-04-25T10:28:23.146 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81634.log 2024-04-25T10:28:23.148 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81504.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81504.log.gz 2024-04-25T10:28:23.148 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81634.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81634.log.gz 2024-04-25T10:28:23.149 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49957.log 2024-04-25T10:28:23.150 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55937.log 2024-04-25T10:28:23.152 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49957.log: /var/log/ceph/ceph-client.admin.55937.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49957.log.gz 2024-04-25T10:28:23.153 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74127.log 2024-04-25T10:28:23.153 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55937.log.gz 2024-04-25T10:28:23.153 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89851.log 2024-04-25T10:28:23.157 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.74127.log: /var/log/ceph/ceph-client.admin.89851.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74127.log.gz 2024-04-25T10:28:23.157 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50718.log 2024-04-25T10:28:23.157 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.89851.log.gz 2024-04-25T10:28:23.157 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35646.log 2024-04-25T10:28:23.158 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50718.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50718.log.gz 2024-04-25T10:28:23.159 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35646.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35646.log.gz 2024-04-25T10:28:23.160 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55211.log 2024-04-25T10:28:23.161 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-04-25T10:28:23.162 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55211.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55211.log.gz 2024-04-25T10:28:23.163 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52170.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-04-25T10:28:23.164 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91711.log 2024-04-25T10:28:23.165 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36262.log 2024-04-25T10:28:23.167 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.91711.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91711.log.gz 2024-04-25T10:28:23.168 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36262.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36262.log.gz 2024-04-25T10:28:23.168 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37246.log 2024-04-25T10:28:23.170 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81225.log 2024-04-25T10:28:23.171 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37246.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37246.log.gz 2024-04-25T10:28:23.172 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81225.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81225.log.gz 2024-04-25T10:28:23.172 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71412.log 2024-04-25T10:28:23.173 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50533.log 2024-04-25T10:28:23.175 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.71412.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71412.log.gz 2024-04-25T10:28:23.176 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50533.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50533.log.gz 2024-04-25T10:28:23.176 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87251.log 2024-04-25T10:28:23.178 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36630.log 2024-04-25T10:28:23.179 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87251.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87251.log.gz 2024-04-25T10:28:23.180 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36630.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36630.log.gz 2024-04-25T10:28:23.180 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90936.log 2024-04-25T10:28:23.182 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52994.log 2024-04-25T10:28:23.183 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.90936.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90936.log.gz 2024-04-25T10:28:23.184 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52994.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52994.log.gz 2024-04-25T10:28:23.184 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54339.log 2024-04-25T10:28:23.186 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86016.log 2024-04-25T10:28:23.188 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54339.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54339.log.gz 2024-04-25T10:28:23.188 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.86016.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86016.log.gz 2024-04-25T10:28:23.189 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87335.log 2024-04-25T10:28:23.190 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72789.log 2024-04-25T10:28:23.191 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87335.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87335.log.gz 2024-04-25T10:28:23.193 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.72789.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.72789.log.gz 2024-04-25T10:28:23.193 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69099.log 2024-04-25T10:28:23.194 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81962.log 2024-04-25T10:28:23.196 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.69099.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69099.log.gz 2024-04-25T10:28:23.197 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.81962.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81962.log.gz 2024-04-25T10:28:23.197 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53420.log 2024-04-25T10:28:23.199 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89497.log 2024-04-25T10:28:23.200 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53420.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53420.log.gz 2024-04-25T10:28:23.201 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.89497.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89497.log.gz 2024-04-25T10:28:23.201 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52909.log 2024-04-25T10:28:23.203 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55237.log 2024-04-25T10:28:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52909.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52909.log.gz 2024-04-25T10:28:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80900.log 2024-04-25T10:28:23.206 INFO:teuthology.orchestra.run.smithi087.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55237.log.gz 2024-04-25T10:28:23.206 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87087.log 2024-04-25T10:28:23.208 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.80900.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80900.log.gz 2024-04-25T10:28:23.208 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.87087.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87087.log.gz 2024-04-25T10:28:23.209 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77586.log 2024-04-25T10:28:23.210 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50436.log 2024-04-25T10:28:23.212 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.77586.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77586.log.gz 2024-04-25T10:28:23.213 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50436.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50436.log.gz 2024-04-25T10:28:23.213 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-04-25T10:28:23.214 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35620.log 2024-04-25T10:28:23.216 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35201.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-04-25T10:28:23.217 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35620.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35620.log.gz 2024-04-25T10:28:23.217 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35908.log 2024-04-25T10:28:23.219 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.35908.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35908.log.gz 2024-04-25T10:28:23.773 INFO:teuthology.orchestra.run.smithi169.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:28:23.926 INFO:teuthology.orchestra.run.smithi169.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:28:23.970 INFO:teuthology.orchestra.run.smithi087.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:28:23.993 INFO:teuthology.orchestra.run.smithi169.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:28:24.093 INFO:teuthology.orchestra.run.smithi169.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:28:24.207 INFO:teuthology.orchestra.run.smithi087.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:28:24.294 INFO:teuthology.orchestra.run.smithi087.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:28:24.771 INFO:teuthology.orchestra.run.smithi087.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:28:24.859 INFO:teuthology.orchestra.run.smithi087.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:28:44.444 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:28:44.445 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-25T10:28:44.446 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m22.810s 2024-04-25T10:28:44.446 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m30.417s 2024-04-25T10:28:44.446 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m2.396s 2024-04-25T10:28:46.461 INFO:teuthology.orchestra.run.smithi087.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:28:46.463 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-04-25T10:28:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m24.845s 2024-04-25T10:28:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m35.646s 2024-04-25T10:28:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m4.171s 2024-04-25T10:28:46.465 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:28:46.465 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/remote/smithi087/log 2024-04-25T10:28:46.466 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:28:51.278 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/remote/smithi169/log 2024-04-25T10:28:51.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:28:54.848 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:28:54.857 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:28:54.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:28:54.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:28:54.943 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:28:54.953 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:28:54.953 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:28:54.956 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:28:54.969 INFO:teuthology.orchestra.run.smithi087.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:28:54.969 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================== 2024-04-25T10:28:54.969 INFO:teuthology.orchestra.run.smithi087.stdout:*hv01.front.sepi 67.205.162.81 3 u 45 64 377 0.068 -5.467 1.058 2024-04-25T10:28:54.969 INFO:teuthology.orchestra.run.smithi087.stdout:+hv02.front.sepi 63.231.80.2 3 u 35 64 377 0.098 -4.485 1.504 2024-04-25T10:28:54.969 INFO:teuthology.orchestra.run.smithi087.stdout:+hv03.front.sepi 74.6.168.72 3 u 39 64 377 0.094 -4.933 1.576 2024-04-25T10:28:54.969 INFO:teuthology.orchestra.run.smithi087.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:28:54.973 INFO:teuthology.orchestra.run.smithi169.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:28:54.973 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================== 2024-04-25T10:28:54.973 INFO:teuthology.orchestra.run.smithi169.stdout:+hv01.front.sepi 67.205.162.81 3 u 39 64 377 0.100 -6.183 0.887 2024-04-25T10:28:54.973 INFO:teuthology.orchestra.run.smithi169.stdout:*hv02.front.sepi 63.231.80.2 3 u 34 64 377 0.098 -4.119 0.899 2024-04-25T10:28:54.973 INFO:teuthology.orchestra.run.smithi169.stdout:+hv03.front.sepi 74.6.168.72 3 u 35 64 377 0.077 -4.453 0.805 2024-04-25T10:28:54.973 INFO:teuthology.orchestra.run.smithi169.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:28:54.974 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:28:54.983 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:28:54.984 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:28:54.992 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:28:55.001 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:28:55.009 INFO:teuthology.task.internal:Duration was 1692.472029 seconds 2024-04-25T10:28:55.010 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:28:55.018 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:28:55.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:28:55.022 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:28:55.108 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:28:55.108 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:28:55.109 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:28:55.129 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T10:28:55.129 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:28:55.149 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:28:55.150 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:28:55.179 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:28:55.235 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:28:55.235 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:28:55.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:28:55.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:28:55.367 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:28:55.368 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:28:55.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:28:55.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:28:55.416 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:28:55.431 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:28:55.445 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-04-25T10:28:55.466 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-25T10:28:55.484 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:28:55.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:28:55.511 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:28:55.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:28:55.533 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:28:55.543 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:28:55.544 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/remote/smithi087 2024-04-25T10:28:55.544 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:28:55.580 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673049/remote/smithi169 2024-04-25T10:28:55.580 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:28:55.605 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:28:55.605 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:28:55.619 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:28:55.649 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:28:55.659 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:28:55.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:28:55.727 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:28:55.727 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:28:55.730 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:28:55.735 INFO:teuthology.orchestra.run.smithi087.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:28 /home/ubuntu/cephtest 2024-04-25T10:28:55.738 INFO:teuthology.orchestra.run.smithi169.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:28 /home/ubuntu/cephtest 2024-04-25T10:28:55.740 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:28:55.751 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:28:55.774 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1692.4720289707184 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:28:55.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:28:55.851 INFO:teuthology.run:pass